Beautiful, AI-native markdown IDE and LLM wiki
TypeScript 2nd-brainagent-skillsclaudecodexcompany-braindocsknowledge-knowledge-baseknowledge-managementllmllm-wikillm-wiki-karpathymarkdownmarkdown-editormdmdxnotespkmsecond-brainskills
fix(ok): PRD-7858 stop treating a body `---` as frontmatter on write and edit (#3339)
tim-inkeepad3bf4c9cdc7
diff --git a/.changeset/write-thematic-break-frontmatter.md b/.changeset/write-thematic-break-frontmatter.mdnew file mode 100644index 000000000..57647d08c--- /dev/null+++ b/.changeset/write-thematic-break-frontmatter.md@@ -0,0 +1,7 @@+---+"@inkeep/open-knowledge": patch+---++Agents can write `---` again. A body-level `---` (a thematic break, or a section an agent separated with rules) was being mistaken for a frontmatter fence in two places: `edit` refused any find containing `---` or a `key: value` line before it even read the document, and `write` with `position: "append"` / `"prepend"` partitioned a leading `---` span off the payload and then rejected the whole write when that span was not parseable YAML. Both refusals pushed agents into a full-document rewrite, which clobbers whatever a concurrent writer had put in the rest of the file. `edit` now decides purely on where the match lands, so a `---` or `key: value` find in the body applies normally and only a match inside the frontmatter region is refused; append/prepend only treat a leading `---` block as frontmatter when it parses as a YAML mapping, and otherwise write it through verbatim as the body text it is.++One narrow case stays refused, now with an error that names the fix: a `---` fence pair that would land at byte 0 of a document with no frontmatter. There the composed bytes re-read as a frontmatter block, so the content would vanish from the rendered document and could not be edited back. All three write positions agree on refusing it, and the message points at the workarounds (a leading blank line, or `***` / `___` for the thematic break).diff --git a/packages/app/tests/integration/fm-collision-guard.test.ts b/packages/app/tests/integration/fm-collision-guard.test.tsindex 2185d782d..76fe24d68 100644--- a/packages/app/tests/integration/fm-collision-guard.test.ts+++ b/packages/app/tests/integration/fm-collision-guard.test.ts@@ -413,13 +413,26 @@ describe('A3 — append/prepend payload partition', () => {}/**- * Append and prepend partition the payload with the same regex and discard- * the claimed span. When that span is not a YAML mapping the payload never- * had frontmatter and the partition just ate the agent's BODY — the same- * payload sent as `replace` is already refused, so the asymmetry was the bug.+ * The verdict follows the COMPOSED DOCUMENT, not the payload's shape.+ *+ * This file's whole thesis is a document whose body OPENS with a `---` fence+ * pair — offset 0 is what makes the span satisfy `FRONTMATTER_RE` and vanish+ * into the YAML region. `replace` puts the payload at offset 0 by+ * definition, and `prepend` puts it there whenever the document carries no+ * frontmatter, so both still refuse. An `append` onto a non-empty body+ * cannot reach offset 0: the same bytes compose into a document with no+ * frontmatter region at all, where the fence pair is an ordinary thematic+ * break. Refusing that was the bug this pins against — it left the agent no+ * move but a whole-document rewrite, which clobbers concurrent writers.+ *+ * The earlier reading judged the payload in isolation and called the+ * position split an asymmetry. It is one rule ("may the composed document+ * end up with a frontmatter region the agent never asked for?") reaching+ * three different composed documents. Same anchor the sibling WYSIWYG guard+ * above uses: `normalizeBridge`'s doc-start step is anchored at offset 0.**/- test('an ambiguous payload is refused, matching replace', async () => {+ test('an ambiguous payload is refused where it would land at offset 0', async () => {const client = await createTestClient(server.port);try {await agentWriteMd(server.port, 'seed\n', {@@ -429,17 +442,82 @@ describe('A3 — append/prepend payload partition', () => {await settled(client);const ambiguous = '---\n\nx\n\n---\n\ny\n';+ // `replace` makes the payload the whole document: the fence pair IS the+ // frontmatter region, and it is not a mapping.expect(await writeRaw(client.docName, ambiguous, 'replace')).toBe(400);- expect(await writeRaw(client.docName, ambiguous, 'append')).toBe(400);+ // `prepend` onto a frontmatter-less document lands at offset 0 too.expect(await writeRaw(client.docName, ambiguous, 'prepend')).toBe(400);- // Refused writes leave the document untouched.+ // Both refusals leave the document untouched.expect(getServerState(server, client.docName)?.ytext.toString()).toBe('seed\n');} finally {await client.cleanup();}});+ /**+ * The other side of the same rule, and the case the bug report was filed+ * about. Appending onto a non-empty body puts the fence pair mid-document,+ * where it is a thematic break and nothing is claimed as frontmatter — so+ * the write lands, and the content survives all the way into the fragment+ * rather than disappearing into a YAML region.+ *+ */+ test('the same payload appended onto a non-empty body lands, with no collision', async () => {+ const client = await createTestClient(server.port);+ const violations: unknown[] = [];+ const detach = attachBridgeInvariantWatcher(client.doc, {+ onViolation: (info) => violations.push(info),+ });+ try {+ await agentWriteMd(server.port, 'seed\n', {+ docName: client.docName,+ position: 'replace',+ });+ await settled(client);++ expect(await writeRaw(client.docName, '---\n\nx\n\n---\n\ny\n', 'append')).toBe(200);+ const ytext = await settled(client);++ // No frontmatter region was created — the document still opens with its+ // own body, so the fence pair never reaches offset 0.+ expect(stripFrontmatter(ytext).frontmatter).toBe('');+ expect(ytext.startsWith('seed')).toBe(true);++ // The agent's bytes are intact: no re-spell, no dropped span. This is+ // the byte-sacred agent path, not the WYSIWYG mint.+ expect(ytext).toContain('---');+ expect(ytext).toContain('x');+ expect(ytext).toContain('y');++ // And the content reached the fragment rather than vanishing into a+ // YAML region — the collision this file exists to catch did not happen.+ const state = getServerState(server, client.docName);+ expect(state?.md).toContain('x');+ expect(state?.md).toContain('y');+ expect(violations).toEqual([]);+ } finally {+ detach();+ await client.cleanup();+ }+ });++ /**+ * The offset-0 sibling of the append case: with an EMPTY body there is+ * nothing in front of the payload, so append lands at offset 0 exactly like+ * prepend and refuses on the same rule.+ *+ */+ test('append onto an EMPTY body refuses — it lands at offset 0 too', async () => {+ const client = await createTestClient(server.port);+ try {+ expect(await writeRaw(client.docName, '---\n\nx\n\n---\n\ny\n', 'append')).toBe(400);+ expect(getServerState(server, client.docName)?.ytext.toString() ?? '').toBe('');+ } finally {+ await client.cleanup();+ }+ });+/*** A well-formed frontmatter payload keeps its documented silent drop: the* span parses as a mapping, the body still lands, and the bound is specified.diff --git a/packages/server/src/agent-sessions.ts b/packages/server/src/agent-sessions.tsindex 30df2e39b..a499f33db 100644--- a/packages/server/src/agent-sessions.ts+++ b/packages/server/src/agent-sessions.ts@@ -26,6 +26,7 @@ import {stripFrontmatter,unwrapFrontmatterFences,} from '@inkeep/open-knowledge-core';+import { splitPayloadFrontmatter } from './payload-frontmatter.ts';export { colorFromSeed } from '@inkeep/open-knowledge-core';@@ -353,7 +354,9 @@ export function snapshotBlocks(document: Document): string[] {* body-only; we handle both. On 'replace', an FM in the payload supersedes* the existing FM. On 'prepend'/'append', the payload's FM (if any) is* dropped defensively to avoid producing a document with two FM blocks- * (double-FM is a CommonMark invalid state). Stripping FM is orthogonal to+ * (double-FM is a CommonMark invalid state) — but ONLY when it parses as a+ * YAML mapping, since `FRONTMATTER_RE` also claims ordinary bodies that open+ * and close with a `---` thematic break. Stripping FM is orthogonal to* the byte-faithful contract — body bytes survive verbatim, only the* FM-handling logic prevents structural breakage.*@@ -372,14 +375,6 @@ export function snapshotBlocks(document: Document): string[] {interface ComposedAgentWrite {existingFm: string;finalFm: string;- /**- * The span append/prepend partitioned off the payload and discarded. Empty- * on replace/patch, which supersede the FM region instead of dropping it.- * Dropping a well-formed block is the documented bound; dropping a span that- * is NOT a YAML mapping means the payload never had frontmatter and the- * partition just ate the agent's BODY, which the gate refuses.- */- droppedPayloadFm: string;/** Full document bytes (FM + body) the primitives apply verbatim. */newContent: string;}@@ -390,7 +385,17 @@ function composeAgentWrite(position: 'append' | 'prepend' | 'replace' | 'patch',): ComposedAgentWrite | undefined {const { frontmatter: existingFm, body: currentBody } = stripFrontmatter(currentYText);- const { frontmatter: payloadFm, body: payloadBody } = stripFrontmatter(markdown);+ // Append/prepend DISCARD whatever the frontmatter partition claims, so the+ // mapping-only rule in `splitPayloadFrontmatter` is what keeps the regex+ // from silently eating body content that merely opens with a `---` fence+ // pair. `replace`/`patch` supersede the FM region rather than discarding+ // it, so they keep the raw `stripFrontmatter` partition. The rule lives in+ // its own module because the MCP `write` tool's advisory notes must agree+ // with it byte-for-byte — see `splitPayloadFrontmatter`.+ const { frontmatter: payloadFm, body: payloadBody } =+ position === 'append' || position === 'prepend'+ ? splitPayloadFrontmatter(markdown)+ : stripFrontmatter(markdown);// append/prepend with an empty body is a no-op — there is nothing to add,// so return before the write and leave the document byte-unchanged.@@ -439,7 +444,6 @@ function composeAgentWrite(return {existingFm,finalFm,- droppedPayloadFm: position === 'append' || position === 'prepend' ? payloadFm : '',newContent: prependFrontmatter(finalFm, newBody),};}@@ -462,7 +466,7 @@ function applyAgentMarkdownWriteInner(if (composed === undefined) {return;}- const { existingFm, finalFm, droppedPayloadFm, newContent } = composed;+ const { existingFm, finalFm, newContent } = composed;// A `replace`/`patch` that rebuilds the fragment over un-propagated WYSIWYG// content silently discards it — wire the paired-intake derive-loss detector@@ -494,10 +498,12 @@ function applyAgentMarkdownWriteInner(//// Gate is targeted: only fires when the agent CHANGES the FM// (`finalFm !== existingFm`). Append/prepend can never satisfy that —- // they inherit `existingFm` by construction — so they carry their own- // arm below, keyed on the payload span they discard. Existing docs that- // already carry malformed FM keep accepting body-only writes — the- // rejection follows the introducer, not the inheritor.+ // they inherit `existingFm` by construction — and they no longer need+ // an arm of their own: `composeAgentWrite` only partitions a payload FM+ // off an append/prepend when it parses as a YAML mapping, so there is no+ // unparseable span left to refuse. Existing docs that already carry+ // malformed FM keep accepting body-only writes — the rejection follows+ // the introducer, not the inheritor.//// No byte mutation: we parse for validation only. The agent's bytes// are preserved verbatim once they pass (Y.Text-is-truth, precedent@@ -530,21 +536,35 @@ function applyAgentMarkdownWriteInner(// `frontmatter-malformed-write-refused` structured log already// carries the refusal signal.recordFrontmatterEditSurface('mcp-write');- } else if (droppedPayloadFm !== '') {- // Append/prepend partition the payload with the same `FRONTMATTER_RE` and- // discard the claimed span. Dropping a well-formed block is the- // documented bound and stays silent — the payload's BODY still lands- // intact. But a span that isn't a YAML mapping was never frontmatter:- // the partition just ate the agent's content, and the same payload sent- // as `replace` is already refused with this exact error and class. Same- // predicate, same envelope, same event — the asymmetry was the bug.- const droppedParsed = parseFrontmatterYaml(unwrapFrontmatterFences(droppedPayloadFm));- if (droppedParsed.map === null) {- throw new FrontmatterMalformedError({- file: docNameToFile(document.name),- parseError: droppedParsed.parseError ?? 'unknown YAML parse error',- });- }+ } else if (finalFm === '' && stripFrontmatter(newContent).frontmatter !== '') {+ // Frontmatter PROMOTION, the residue of the un-split above. Writing a+ // `---`-fenced non-mapping span through as body is right everywhere it+ // lands mid-document, but append/prepend can put it at byte 0 (prepend+ // onto a document with no frontmatter, or append onto one whose body is+ // empty). There the composed bytes re-partition: every other consumer —+ // `stripFrontmatter`, the XmlFragment derive, the FM bridge, the next+ // `edit` — reads that span as the frontmatter region, so the content+ // vanishes from the rendered document, the property panel shows the+ // malformed-FM banner, and editing it back is refused as an FM+ // intersect. Refusing here keeps the three write positions in agreement+ // about which byte sequences make an acceptable document: `replace`+ // already refuses these exact bytes at the gate above.+ //+ // The span is necessarily a non-mapping — a well-formed mapping block+ // stays partitioned off as `payloadFm` and is dropped, never un-split —+ // so the malformed-FM envelope is the honest one. It carries its own+ // class and hint because nothing here was parsed: the default advice+ // ("quote your YAML-significant characters") would send the agent after+ // syntax when the problem is placement, and bucketing this as a parse+ // error would put placement refusals in the counter that exists to+ // catch schema regressions.+ throw new FrontmatterMalformedError({+ file: docNameToFile(document.name),+ parseError:+ "the payload's leading `---` fence pair would land at byte 0, where the composed document re-reads it as the frontmatter region instead of body",+ refusalClass: 'byte-0-promotion',+ hint: 'Start the payload with a blank line so the fence pair cannot open the document, or use `***` / `___` for the thematic break.',+ });}// Hand the composed full bytes (FM + body) to the shared primitive.diff --git a/packages/server/src/api-agent-frontmatter.test.ts b/packages/server/src/api-agent-frontmatter.test.tsindex 4adbc41c4..23848fd2a 100644--- a/packages/server/src/api-agent-frontmatter.test.ts+++ b/packages/server/src/api-agent-frontmatter.test.ts@@ -7,7 +7,8 @@* 1. write_document with payload FM updates the YAML region of Y.Text.* 2. write_document with body-only payload preserves existing FM.* 3. append/prepend never duplicate or stomp FM.- * 4. agent-patch refuses spliced edits to the FM region.+ * 4. agent-patch refuses spliced edits that LAND in the FM region (and only+ * those — a `---` or `key: value` find in the body applies normally).* 5. body-only agent-patch continues to work.* 6. agent-undo reverts the FM region in lock-step with body changes.*/@@ -217,6 +218,156 @@ describe('POST /api/agent-write-md (write_document) — frontmatter handling', (}});+ test('append payload opening with a `---`-fenced NON-mapping span lands verbatim (PRD-7858)', async () => {+ // `FRONTMATTER_RE` claims any payload that opens and closes with a `---`+ // fence — including an ordinary body whose first line is a thematic break.+ // Append used to partition that span off and then REFUSE the write because+ // it wasn't parseable YAML, leaving the agent no option but a whole-doc+ // rewrite that clobbers concurrent writers. It's body text: write it.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const existingFm = '---\ntitle: Stable\n---\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(session.dc.document, `${existingFm}# Header\n`, 'replace');+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(+ hocuspocus,+ sessionManager,+ contentDir,+ '/api/agent-write-md',+ {+ docName: 'test-doc',+ markdown: '---\n\n## Findings\n\n---\n\nTrailing note.\n',+ position: 'append',+ },+ );++ expect(response.status).toBe(200);+ expect(ytextFm(session.dc.document)).toBe(existingFm);+ expect(fmMap(session.dc.document)).toEqual({ title: 'Stable' });++ const ytext = session.dc.document.getText('source').toString();+ expect(ytext).toContain('## Findings');+ expect(ytext).toContain('Trailing note.');+ } finally {+ await cleanup();+ }+ });++ test('prepend of a `---`-fenced non-mapping span onto an FM-LESS doc is refused (byte-0 promotion)', async () => {+ // Un-splitting the span is right everywhere it lands mid-document. At byte+ // 0 the composed bytes re-partition and the agent's body text BECOMES the+ // frontmatter region — content disappears from the rendered doc, the+ // property panel shows the malformed-FM banner, and editing it back is+ // refused as an FM intersect. Refuse instead, matching what `replace`+ // already does with the same bytes.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const original = '# Header\n\nExisting body.\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(session.dc.document, original, 'replace');+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(+ hocuspocus,+ sessionManager,+ contentDir,+ '/api/agent-write-md',+ {+ docName: 'test-doc',+ markdown: '---\n\n## Findings\n\n---\n\nTrailing note.\n',+ position: 'prepend',+ },+ );++ expect(response.status).toBe(400);+ const parsed = JSON.parse(response.body);+ expect(parsed.type).toBe('urn:ok:error:frontmatter-malformed');+ expect(parsed.parseError).toContain('byte 0');+ // The advice is about placement. The default YAML-quoting hint would+ // send the agent after syntax that was never parsed.+ expect(parsed.detail).toContain('blank line');+ expect(parsed.detail).not.toContain('Quote string values');++ // Bytes never landed — the document is untouched and still FM-less.+ expect(session.dc.document.getText('source').toString()).toBe(original);+ expect(ytextFm(session.dc.document)).toBe('');+ } finally {+ await cleanup();+ }+ });++ test('append of the same span onto an FM-less doc with an EMPTY body is refused too', async () => {+ // `currentBody.length > 0` is false, so append also places the payload at+ // byte 0 — the sibling of the prepend case above.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const response = await callApi(+ hocuspocus,+ sessionManager,+ contentDir,+ '/api/agent-write-md',+ {+ docName: 'test-doc',+ markdown: '---\n\n## Findings\n\n---\n',+ position: 'append',+ },+ );++ expect(response.status).toBe(400);+ expect(JSON.parse(response.body).type).toBe('urn:ok:error:frontmatter-malformed');+ expect(session.dc.document.getText('source').toString()).toBe('');+ } finally {+ await cleanup();+ }+ });++ test('prepend of the same span onto an FM-BEARING doc lands (the guard is byte-0, not `---`)', async () => {+ // Sibling of the append case above, on the other position. With existing+ // frontmatter the payload can never reach offset 0, so the promotion guard+ // is structurally unreachable here — this pins that, so a future widening+ // of the guard that forgets the FM-bearing case fails loudly.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const existingFm = '---\ntitle: Stable\n---\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(session.dc.document, `${existingFm}# Header\n`, 'replace');+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(+ hocuspocus,+ sessionManager,+ contentDir,+ '/api/agent-write-md',+ {+ docName: 'test-doc',+ markdown: '---\n\n## Findings\n\n---\n\nTrailing note.\n',+ position: 'prepend',+ },+ );++ expect(response.status).toBe(200);+ expect(ytextFm(session.dc.document)).toBe(existingFm);+ expect(fmMap(session.dc.document)).toEqual({ title: 'Stable' });++ const ytext = session.dc.document.getText('source').toString();+ expect(ytext).toContain('## Findings');+ expect(ytext).toContain('Trailing note.');+ expect(ytext).toContain('# Header');+ } finally {+ await cleanup();+ }+ });+test('append payload that itself starts with FM does NOT double-write FM', async () => {const { contentDir, hocuspocus, sessionManager, cleanup } = setup();try {@@ -379,6 +530,9 @@ describe('POST /api/frontmatter-patch (edit_frontmatter) — fence spacing (PRD-});describe('POST /api/agent-patch (edit_document) — frontmatter rejection', () => {+ // Every rejection here is position-based: the find MATCHED inside the FM+ // region. A find that merely LOOKS like frontmatter but lands in the body+ // applies — see the body-`---` / yaml-shape cases at the end of this block.test('rejects yaml-shape find (e.g. "cluster: misc") with 400 + migration hint', async () => {const { contentDir, hocuspocus, sessionManager, cleanup } = setup();try {@@ -488,6 +642,99 @@ describe('POST /api/agent-patch (edit_document) — frontmatter rejection', () =}});+ test('refuses a byte-0 edit whose replace opens a VALID YAML-mapping fence (promotion)', async () => {+ // The document has no frontmatter, so there is no region for the position+ // check to protect — but a match at byte 0 whose replacement opens a fence+ // pair would CREATE frontmatter through the one surface whose contract is+ // body-only. Refused, consistent with this handler's own error text.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const original = 'Lead paragraph.\n\nRest of body.\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(session.dc.document, original, 'replace');+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(hocuspocus, sessionManager, contentDir, '/api/agent-patch', {+ docName: 'test-doc',+ find: 'Lead paragraph.',+ replace: '---\ntitle: Sneaky\n---\n\nLead paragraph.',+ });++ expect(response.status).toBe(400);+ const parsed = JSON.parse(response.body);+ expect(parsed.type).toBe('urn:ok:error:frontmatter-edit-not-supported');+ expect(parsed.title).toContain(+ "would turn the replacement text into the document's frontmatter",+ );++ expect(session.dc.document.getText('source').toString()).toBe(original);+ expect(ytextFm(session.dc.document)).toBe('');+ } finally {+ await cleanup();+ }+ });++ test('refuses a byte-0 edit whose replace opens a NON-mapping fence (same verdict)', async () => {+ // Same shape, span that is not YAML at all. Without the promotion guard+ // this one silently lost the fenced span instead of writing it.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const original = 'Lead paragraph.\n\nRest of body.\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(session.dc.document, original, 'replace');+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(hocuspocus, sessionManager, contentDir, '/api/agent-patch', {+ docName: 'test-doc',+ find: 'Lead paragraph.',+ replace: '---\n\n## Findings\n\n---\n\nLead paragraph.',+ });++ expect(response.status).toBe(400);+ expect(JSON.parse(response.body).type).toBe('urn:ok:error:frontmatter-edit-not-supported');+ expect(session.dc.document.getText('source').toString()).toBe(original);+ } finally {+ await cleanup();+ }+ });++ test('a `---` replace that lands mid-body on an FM-LESS doc still applies', async () => {+ // The promotion guard is keyed on byte 0, not on the presence of `---`.+ // The same replacement one paragraph in composes to a document whose first+ // bytes are still prose, so nothing re-partitions and the write goes+ // through — this is the case the PR exists to unblock.+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(+ session.dc.document,+ 'Lead paragraph.\n\nOld section.\n',+ 'replace',+ );+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(hocuspocus, sessionManager, contentDir, '/api/agent-patch', {+ docName: 'test-doc',+ find: 'Old section.',+ replace: '---\n\n## New section\n\n---\n',+ });++ expect(response.status).toBe(200);+ const ytext = session.dc.document.getText('source').toString();+ expect(ytext).toContain('## New section');+ expect(ytext.startsWith('Lead paragraph.')).toBe(true);+ expect(ytextFm(session.dc.document)).toBe('');+ } finally {+ await cleanup();+ }+ });+test('body-only patch with non-yaml find still applies (regression — body path unaffected)', async () => {const { contentDir, hocuspocus, sessionManager, cleanup } = setup();try {@@ -553,7 +800,11 @@ describe('POST /api/agent-patch (edit_document) — frontmatter rejection', () =}});- test('rejects yaml-shape find even when no FM exists in the doc (heuristic precheck is doc-stateless)', async () => {+ test('applies a yaml-SHAPE find that lands in the body (PRD-7858 — no doc-stateless precheck)', async () => {+ // The rejection follows where the match LANDS, never what the string+ // looks like. `foo: bar` is prose in the body of a doc with no FM at all;+ // the old string-shape precheck refused it before reading the doc, and+ // the agent's only workaround was a whole-document rewrite.const { contentDir, hocuspocus, sessionManager, cleanup } = setup();try {const session = await sessionManager.getSession('test-doc');@@ -572,13 +823,47 @@ describe('POST /api/agent-patch (edit_document) — frontmatter rejection', () =replace: 'baz: qux',});- expect(response.status).toBe(400);- const parsed = JSON.parse(response.body);- expect(parsed.title).toContain('Frontmatter edits are not supported');+ expect(response.status).toBe(200);const ytext = session.dc.document.getText('source').toString();- expect(ytext).toContain('foo: bar appears here.');- expect(ytext).not.toContain('baz: qux');+ expect(ytext).toContain('baz: qux appears here.');+ expect(ytext).not.toContain('foo: bar');+ } finally {+ await cleanup();+ }+ });++ test('applies an edit whose find AND replace carry a body `---` thematic break (PRD-7858)', async () => {+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ const existingFm = '---\ntitle: Doc\n---\n';+ session.dc.document.transact(() => {+ applyAgentMarkdownWrite(+ session.dc.document,+ `${existingFm}# Body\n\n---\n\n## Old Section\n`,+ 'replace',+ );+ }, AGENT_WRITE_ORIGIN);++ const response = await callApi(hocuspocus, sessionManager, contentDir, '/api/agent-patch', {+ docName: 'test-doc',+ find: '---\n\n## Old Section\n',+ replace: '---\n\n## New Section\n\nWith content.\n',+ });++ expect(response.status).toBe(200);++ // The doc's own frontmatter is untouched — the find matched the body's+ // thematic break, which starts well after the FM-end byte.+ expect(ytextFm(session.dc.document)).toBe(existingFm);+ expect(fmMap(session.dc.document)).toEqual({ title: 'Doc' });++ const ytext = session.dc.document.getText('source').toString();+ expect(ytext).toContain('## New Section');+ expect(ytext).toContain('With content.');+ expect(ytext).not.toContain('## Old Section');} finally {await cleanup();}@@ -613,6 +898,77 @@ describe('POST /api/agent-patch (edit_document) — frontmatter rejection', () =});});+describe('byte-0 promotion refusal reaches every write surface', () => {+ // The guard lives in `applyAgentMarkdownWriteInner`, so it fires from every+ // handler that composes an append. Each handler catches+ // `FrontmatterMalformedError` separately, and two of those catches had no+ // test — a comment was the only thing standing between them and being+ // deleted as dead code, which is exactly what the comment warns against.++ test('/api/agent-write (the non-md surface) returns the typed envelope', async () => {+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('test-doc');++ // Empty body + no frontmatter: `${content}\n` lands at byte 0.+ const response = await callApi(hocuspocus, sessionManager, contentDir, '/api/agent-write', {+ docName: 'test-doc',+ content: '---\n\n## Findings\n\n---',+ });++ expect(response.status).toBe(400);+ const parsed = JSON.parse(response.body);+ expect(parsed.type).toBe('urn:ok:error:frontmatter-malformed');+ expect(parsed.parseError).toContain('byte 0');+ // Placement advice, not the YAML-quoting default.+ expect(parsed.detail).toContain('blank line');+ expect(parsed.detail).not.toContain('Quote string values');++ // Refused writes leave the document untouched.+ expect(session.dc.document.getText('source').toString()).toBe('');+ } finally {+ await cleanup();+ }+ });++ test('/api/agent-write-batch reports it per entry, with the placement hint', async () => {+ const { contentDir, hocuspocus, sessionManager, cleanup } = setup();+ try {+ const session = await sessionManager.getSession('batch-doc');++ const response = await callApi(+ hocuspocus,+ sessionManager,+ contentDir,+ '/api/agent-write-batch',+ {+ docs: [+ {+ docName: 'batch-doc',+ markdown: '---\n\n## Findings\n\n---\n',+ position: 'append',+ },+ ],+ },+ );++ // The batch envelope is 200; the refusal rides on the entry.+ expect(response.status).toBe(200);+ const parsed = JSON.parse(response.body);+ const entry = parsed.results[0];+ expect(entry.error.type).toBe('urn:ok:error:frontmatter-malformed');+ // The entry detail carries the same hint the single-doc surface gives —+ // this is what the shared helpers exist to keep aligned.+ expect(entry.error.detail).toContain('blank line');+ expect(entry.error.detail).not.toContain('Quote string values');++ expect(session.dc.document.getText('source').toString()).toBe('');+ } finally {+ await cleanup();+ }+ });+});+describe('agent-undo round-trip across FM-touching writes', () => {test('applyAgentUndo reverts FM region in lock-step with body changes', async () => {const { sessionManager, cleanup } = setup();diff --git a/packages/server/src/api-extension.ts b/packages/server/src/api-extension.tsindex e7f8d02a3..2c4885d29 100644--- a/packages/server/src/api-extension.ts+++ b/packages/server/src/api-extension.ts@@ -364,8 +364,9 @@ import {type SemanticSearchService,} from './embeddings/index.ts';import {- classifyParseError,FrontmatterMalformedError,+ frontmatterRefusalDetail,+ logFrontmatterRefusal,respondFrontmatterMalformed,} from './frontmatter-malformed-error.ts';import { assertNoSymlinkEscape } from './fs-safety.ts';@@ -740,41 +741,12 @@ function agentPatchFmTouchCounter(): ReturnType<ReturnType<typeof getMeter>['cre'ok.frontmatter.agent_patch_fm_touch_total',{description:- 'Count of agent-patch calls whose find string targets the frontmatter region. Measures incidence during the soft-deprecation window before agent-patch FM-intersecting calls are enforced as 400. Bounded label: result ∈ {rejected, pre_deprecation_passthrough}.',+ 'Count of agent-patch calls refused for touching the frontmatter region. Bounded labels: result ∈ {rejected, pre_deprecation_passthrough}, reason ∈ {intersect, promoted}. `intersect` is a find that MATCHED inside the existing frontmatter; `promoted` is a byte-0 replace that would CREATE frontmatter on a document that had none. They refuse for opposite reasons, so a spike in one says nothing about the other — the append/prepend surface separates the same pair via the `byte-0-promotion` class on `frontmatter-malformed-write-refused`.',},);return _agentPatchFmTouchCounter;}-/**- * Heuristic FM-intersection check for `agent-patch` find strings. Pure- * function on the find string — runs before any doc state is read.- *- * Rejection signal:- * - find contains `---` (FM/body separator — opening or closing fence)- * - find matches `/^\s*[\w-]+:/` (yaml-style key-value at start)- *- * Catches the common case: agents that copy a YAML line verbatim into- * `find` to splice an FM property. The position-based check inside the- * transact block catches the rarer case where a non-yaml-shape find- * happens to land in the FM region (e.g., `find: 'draft'` matching- * `status: draft`). Together they cover both "find looks like FM" and- * "find lands in FM."- */-function findLooksLikeFrontmatter(find: string): boolean {- // Line-anchored `---` (YAML document fence). Mid-string `---` (e.g. body- // text containing em-dash sequences or markdown thematic breaks embedded- // in larger find strings) flows to the position-based check below.- if (/(^|\n)---(\s|\n|$)/.test(find)) return true;- // YAML key-value shape — require an actual value (`\s+\S` after the- // colon) so prose like `Note:` / `IMPORTANT:` / `Warning:` (no value)- // is left to the position-based check, which rejects only when the find- // actually lands inside the FM region. Empty-value YAML keys like- // `draft:` similarly fall through to position-based rejection.- if (/^\s*[\w-]+:\s+\S/.test(find)) return true;- return false;-}-let _renameAttributionCounter: ReturnType<ReturnType<typeof getMeter>['createCounter']> | null =null;function renameAttributionCounter(): ReturnType<ReturnType<typeof getMeter>['createCounter']> {@@ -5263,14 +5235,13 @@ export function createApiExtension(respondDocInConflict(res, e, 'agent-write');return;}- // Symmetry-only catch: `agent-write` calls `applyAgentMarkdownWrite`- // with `position: 'append'`, which routes through the FM-dropping- // branch (`finalFm = existingFm`). The malformed-FM gate fires only- // when `finalFm !== existingFm`, so this catch is structurally- // unreachable from this handler today. Kept as a forward-compat- // slot in case a future shape lets `agent-write` accept FM-bearing- // payloads — at that point the existing test coverage on- // `agent-write-md` already pins the envelope shape.+ // Live catch. `agent-write` appends, so it inherits `existingFm` and+ // can never trip the malformed-FM gate's first arm+ // (`finalFm !== existingFm`). It DOES reach the second arm: appending+ // content that opens with a `---` fence pair to a document with no+ // frontmatter and an empty body places that pair at byte 0, where the+ // composed document would re-read it as its frontmatter region. Not+ // dead code — deleting it turns that refusal into a 500.if (e instanceof FrontmatterMalformedError) {respondFrontmatterMalformed(res, e, 'agent-write');return;@@ -5724,20 +5695,16 @@ export function createApiExtension();}if (e instanceof FrontmatterMalformedError) {- console.warn(- JSON.stringify({- event: 'frontmatter-malformed-write-refused',- handler: 'agent-write-batch',- class: classifyParseError(e.parseError),- 'doc.name': docName,- parseError: e.parseError,- }),- );+ // Same event and same detail composition as the single-doc+ // surface. Reading `parseError` directly here is what made a+ // byte-0 promotion refusal count as a YAML parse error on this+ // path alone and drop its placement hint.+ logFrontmatterRefusal(e, 'agent-write-batch');return entryError(docName,'urn:ok:error:frontmatter-malformed','Frontmatter YAML is malformed.',- e.parseError,+ frontmatterRefusalDetail(e),);}if (e instanceof AgentSessionCapacityError) {@@ -6513,23 +6480,6 @@ export function createApiExtension(const { agentId, agentName, colorSeed, clientName, clientVersion, label } =extractAgentIdentity(body);- // Heuristic precheck: reject `find` strings that look like a YAML- // frontmatter block before doing any Y.Doc work. The position-based- // postcheck below catches non-yaml strings whose first match falls- // inside the FM region. Frontmatter edits must go through- // write with position:"replace", not a body find/replace.- if (findLooksLikeFrontmatter(find)) {- agentPatchFmTouchCounter().add(1, { result: 'rejected' });- errorResponse(- res,- 400,- 'urn:ok:error:frontmatter-edit-not-supported',- 'Frontmatter edits are not supported via a body find/replace. Use edit({ document: { path, frontmatter } }) to change frontmatter, or write({ document: { path, content, position: "replace" } }) to rewrite the whole document including its YAML block.',- { handler: 'agent-patch' },- );- return;- }-if (isSystemDoc(docName) || isConfigDoc(docName)) {errorResponse(res,@@ -6600,6 +6550,7 @@ export function createApiExtension(let notFound = false;let staleTarget = false;let fmIntersect = false;+ let fmPromoted = false;// Site A content-divergence captured from the in-transact gate.// Surfaced as the response's `warning` field on successful patches.let patchDivergence: AgentWriteContentDivergence | undefined;@@ -6677,29 +6628,51 @@ export function createApiExtension(return;}- // Position-based FM-intersection check. The string-shape- // heuristic above handles yaml-style find strings; this catches- // the residual class where a non-yaml find (e.g. a single word- // like `draft`) happens to first-match in the FM region.- // `pos < currentFm.length` is the necessary-and-sufficient- // signal — FM is contiguous at doc start, so any match starting- // before the FM-end byte overlaps the FM region.+ // The FM-intersection check, and the ONLY one. `pos <+ // currentFm.length` is necessary and sufficient: the FM region is+ // contiguous at doc start, so a match starting before its end byte+ // overlaps it and a match at or after cannot. Catches both the+ // yaml-shaped find copied out of the FM block and the plain-word+ // find (e.g. `draft`) that happens to first-match inside it —+ // without refusing those same strings when they live in the body.+ // A string-shape precheck used to reject any find containing a+ // line-anchored `---` or a `key: value` line before reading the+ // doc at all; it refused body thematic breaks and prose, and the+ // agent's only recourse was a whole-document `write` that clobbers+ // concurrent writers.if (pos < currentFm.length) {fmIntersect = true;return;}- // Splice at the character level, then write the recomposed body- // via the `'patch'` position. Only body-region patches reach here,- // so this branch never modifies the FM: applyAgentMarkdownWrite- // reads the current FM from the YAML region of Y.Text and keeps it- // intact for a body-only payload. `'patch'` (NOT `'replace'`) routes- // the write through the INCREMENTAL primitive, so this surgical- // find/replace produces a minimal item-preserving Y.Text delta- // instead of an atomic whole-doc overwrite — replace stays atomic,- // the edit body find/replace stays surgical.const newFull =currentFull.slice(0, pos) + replace + currentFull.slice(pos + find.length);++ // Frontmatter-PROMOTION check, the mirror of the intersection+ // check above. A document with no FM has no region to protect by+ // position, so a match at byte 0 whose `replace` opens a `---`+ // fence pair silently turns the agent's body text into the+ // document's frontmatter region — creating frontmatter through the+ // one surface whose whole contract is "body only". Refuse it: this+ // handler already tells agents that frontmatter edits go through+ // `edit({ document: { path, frontmatter } })`, and a find/replace+ // that CREATES frontmatter is a frontmatter edit.+ if (currentFm === '' && stripFrontmatter(newFull).frontmatter !== '') {+ fmPromoted = true;+ return;+ }++ // The promotion guard above is what keeps this splice honest: every+ // payload that survives it recomposes with its FM region byte-+ // identical to `currentFm`, so stripping it back off here cannot+ // drop anything. `applyAgentMarkdownWrite` then reads the current+ // FM from the YAML region of Y.Text and keeps it intact for the+ // body-only payload — `finalFm === existingFm` still always holds+ // from this handler. `'patch'` (NOT `'replace'`) routes the write+ // through the INCREMENTAL primitive, so this surgical find/replace+ // produces a minimal item-preserving Y.Text delta instead of an+ // atomic whole-doc overwrite — replace stays atomic, the edit body+ // find/replace stays surgical.const { body: newBody } = stripFrontmatter(newFull);const beforeBlocks = snapshotBlocks(session.dc.document);patchDivergence = applyAgentMarkdownWrite(@@ -6736,7 +6709,7 @@ export function createApiExtension(}),);}- if (!notFound && !staleTarget && !fmIntersect) {+ if (!notFound && !staleTarget && !fmIntersect && !fmPromoted) {// Only count + record when the patch actually applied. The// adoption-rate denominator excludes 404/409 + FM-intersect 400// so the metric reflects successful writes, not total attempts.@@ -6776,7 +6749,7 @@ export function createApiExtension(return;}if (fmIntersect) {- agentPatchFmTouchCounter().add(1, { result: 'rejected' });+ agentPatchFmTouchCounter().add(1, { result: 'rejected', reason: 'intersect' });errorResponse(res,400,@@ -6786,6 +6759,17 @@ export function createApiExtension();return;}+ if (fmPromoted) {+ agentPatchFmTouchCounter().add(1, { result: 'rejected', reason: 'promoted' });+ errorResponse(+ res,+ 400,+ 'urn:ok:error:frontmatter-edit-not-supported',+ "This edit would turn the replacement text into the document's frontmatter: the document has no frontmatter, the match starts at byte 0, and `replace` opens a `---` fence pair — so the composed document would re-read that block as its YAML region. Use edit({ document: { path, frontmatter } }) to set frontmatter, or keep the `---` out of the first line (a leading blank line, or `***` / `___` for a thematic break).",+ { handler: 'agent-patch' },+ );+ return;+ }// Await the L1 disk store so a swallowed persistence failure surfaces as// an error instead of a false success. Mirrors agent-write-md.diff --git a/packages/server/src/frontmatter-malformed-error.test.ts b/packages/server/src/frontmatter-malformed-error.test.tsindex 1ebddf150..abfb3020f 100644--- a/packages/server/src/frontmatter-malformed-error.test.ts+++ b/packages/server/src/frontmatter-malformed-error.test.ts@@ -1,5 +1,12 @@-import { describe, expect, test } from 'vitest';-import { classifyParseError } from './frontmatter-malformed-error.ts';+import type { ServerResponse } from 'node:http';+import { describe, expect, test, vi } from 'vitest';+import {+ classifyParseError,+ FrontmatterMalformedError,+ frontmatterRefusalDetail,+ logFrontmatterRefusal,+ respondFrontmatterMalformed,+} from './frontmatter-malformed-error.ts';describe('classifyParseError — bounded-cardinality refusal class', () => {test('top-level non-mapping bucket', () => {@@ -69,4 +76,126 @@ describe('classifyParseError — bounded-cardinality refusal class', () => {]).toContain(c);}});++ // `byte-0-promotion` is deliberately absent from the list above: the+ // classifier reads parser messages, and that refusal never parsed anything.+ // Its class rides on the error instead — pinned in the block below.+});++/** Minimal `ServerResponse` double — captures status + body, nothing else. */+function captureRes(): { res: ServerResponse; read: () => { status: number; body: string } } {+ const captured = { status: 0, body: '' };+ const res = {+ writeHead(status: number) {+ captured.status = status;+ },+ end(body?: string) {+ captured.body = body ?? '';+ },+ } as unknown as ServerResponse;+ return { res, read: () => captured };+}++describe('respondFrontmatterMalformed — throw-site class and hint win over prose-sniffing', () => {+ test('a byte-0 promotion refusal is NOT counted as a yaml parse error', () => {+ // The whole point of the `class` label is that a spike in+ // `yaml-parse-error` means the parser or schema regressed. Placement+ // refusals landing in that bucket would make the signal unreadable — and+ // they would, since the classifier falls through to it for any non-empty+ // prose. This fails if the `err.refusalClass ??` short-circuit is removed.+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ try {+ const { res, read } = captureRes();+ respondFrontmatterMalformed(+ res,+ new FrontmatterMalformedError({+ file: 'doc.md',+ parseError: "the payload's leading `---` fence pair would land at byte 0",+ refusalClass: 'byte-0-promotion',+ hint: 'Start the payload with a blank line.',+ }),+ 'agent-write-md',+ );++ const event = JSON.parse(warn.mock.calls.at(-1)?.[0] as string);+ expect(event.event).toBe('frontmatter-malformed-write-refused');+ expect(event.class).toBe('byte-0-promotion');+ // Confirms the bucket it would have landed in without the explicit class.+ expect(+ classifyParseError("the payload's leading `---` fence pair would land at byte 0"),+ ).toBe('yaml-parse-error');++ // The hint replaces the YAML-quoting advice rather than stacking on it.+ const body = JSON.parse(read().body);+ expect(body.detail).toContain('Start the payload with a blank line.');+ expect(body.detail).not.toContain('Quote string values');+ } finally {+ warn.mockRestore();+ }+ });++ test('a real YAML failure still classifies from the parser message and keeps the default hint', () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ try {+ const { res, read } = captureRes();+ respondFrontmatterMalformed(+ res,+ new FrontmatterMalformedError({+ file: 'doc.md',+ parseError: 'top-level value is not a mapping',+ }),+ 'agent-write-md',+ );++ expect(JSON.parse(warn.mock.calls.at(-1)?.[0] as string).class).toBe('non-mapping-top-level');+ expect(JSON.parse(read().body).detail).toContain('Quote string values');+ } finally {+ warn.mockRestore();+ }+ });+});++describe('logFrontmatterRefusal / frontmatterRefusalDetail — the batch surface agrees', () => {+ // `agent-write-batch` builds a per-entry error object instead of an HTTP+ // response, so it cannot call `respondFrontmatterMalformed` — it calls these+ // two directly. Reading `err.parseError` there instead is what made byte-0+ // promotion refusals count as YAML parse errors on that path alone, so this+ // pins that both surfaces resolve the class and the detail identically.+ test('a promotion refusal keeps its class and hint on the batch path', () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ try {+ const err = new FrontmatterMalformedError({+ file: 'doc.md',+ parseError: "the payload's leading `---` fence pair would land at byte 0",+ refusalClass: 'byte-0-promotion',+ hint: 'Start the payload with a blank line.',+ });++ logFrontmatterRefusal(err, 'agent-write-batch');+ const event = JSON.parse(warn.mock.calls.at(-1)?.[0] as string);+ expect(event.event).toBe('frontmatter-malformed-write-refused');+ expect(event.handler).toBe('agent-write-batch');+ expect(event.class).toBe('byte-0-promotion');++ expect(frontmatterRefusalDetail(err)).toContain('Start the payload with a blank line.');+ expect(frontmatterRefusalDetail(err)).not.toContain('Quote string values');+ } finally {+ warn.mockRestore();+ }+ });++ test('a YAML failure still classifies from the parser message on the batch path', () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ try {+ const err = new FrontmatterMalformedError({+ file: 'doc.md',+ parseError: 'top-level value is not a mapping',+ });+ logFrontmatterRefusal(err, 'agent-write-batch');+ expect(JSON.parse(warn.mock.calls.at(-1)?.[0] as string).class).toBe('non-mapping-top-level');+ expect(frontmatterRefusalDetail(err)).toContain('Quote string values');+ } finally {+ warn.mockRestore();+ }+ });});diff --git a/packages/server/src/frontmatter-malformed-error.ts b/packages/server/src/frontmatter-malformed-error.tsindex f70549720..94dd02016 100644--- a/packages/server/src/frontmatter-malformed-error.ts+++ b/packages/server/src/frontmatter-malformed-error.ts@@ -10,12 +10,20 @@* string value containing an unquoted YAML-significant character (`:`, `#`,* leading `-`), e.g. `title: The End of 3% Mortgages: Why ...`.*- * The gate lives at `applyAgentMarkdownWriteInner` and ONLY fires when the- * agent's write actually CHANGES the FM region (`finalFm !== existingFm`).- * Append/prepend never touch FM (payload FM is dropped defensively in that- * path), so the gate skips them. Existing docs that already carry malformed- * FM on disk continue to accept body-only writes — the rejection is targeted- * at the introducer, not the inheritor.+ * The gate lives at `applyAgentMarkdownWriteInner` and fires when the agent's+ * write actually CHANGES the FM region (`finalFm !== existingFm`). Existing+ * docs that already carry malformed FM on disk continue to accept body-only+ * writes — the rejection is targeted at the introducer, not the inheritor.+ *+ * A second arm catches the same outcome by a different route: an append or+ * prepend inherits `existingFm` by construction, so it can never trip the+ * first arm, but it CAN place a `---`-fenced non-mapping span at byte 0 of a+ * document that has no frontmatter. The composed bytes then re-partition and+ * that span becomes the FM region — malformed frontmatter the agent never+ * asked for. Same envelope, but it carries its own `refusalClass`+ * (`byte-0-promotion`) and `hint`: nothing was parsed, so the YAML-quoting+ * advice would misdirect, and counting it as a parse error would blunt the+ * one signal the class label exists to give.** Wire shape — slim RFC 9457 envelope at HTTP 400:*@@ -51,12 +59,15 @@ const FIX_HINT ='Frontmatter must be a top-level YAML mapping. Quote string values containing YAML-significant characters (`:`, `#`, leading `-`), e.g. `title: "Foo: bar"`.';/**- * Typed throw from `applyAgentMarkdownWriteInner` when the COMPOSED- * frontmatter (`finalFm`) is unparseable AND the agent is the cause (i.e.,- * `finalFm !== existingFm`). The `file` field carries the doc path so the- * HTTP boundary can surface it as the `file` extension on the RFC 9457- * envelope. `parseError` is the raw `yaml@2` parser message — useful for- * agents to diagnose the offending line/column without round-tripping.+ * Typed throw from `applyAgentMarkdownWriteInner` when the agent's write is+ * the cause of a malformed FM region — either because the COMPOSED+ * frontmatter it introduces is unparseable (`finalFm !== existingFm`), or+ * because an append/prepend would promote body bytes into the FM region at+ * byte 0. The `file` field carries the doc path so the HTTP boundary can+ * surface it as the `file` extension on the RFC 9457 envelope. `parseError`+ * is the raw `yaml@2` parser message for the first case — useful for agents+ * to diagnose the offending line/column without round-tripping — and a+ * placement description for the second.** Throw shape rather than return-value branching: this matches the same* uniform error-composition contract as `DocInConflictError` — every write@@ -66,12 +77,28 @@ const FIX_HINT =export class FrontmatterMalformedError extends Error {readonly file: string;readonly parseError: string;+ /**+ * Set by a throw site that already KNOWS its class, so the response helper+ * does not have to re-derive it by sniffing prose it was just handed.+ * Omitted for the YAML gates, whose `parseError` is a real parser message+ * that `classifyParseError` reads correctly.+ */+ readonly refusalClass?: FrontmatterMalformedClass;+ /** Fix advice replacing `FIX_HINT` when the YAML-quoting hint would not apply. */+ readonly hint?: string;override readonly name = 'FrontmatterMalformedError' as const;- constructor(opts: { file: string; parseError: string }) {+ constructor(opts: {+ file: string;+ parseError: string;+ refusalClass?: FrontmatterMalformedClass;+ hint?: string;+ }) {super(`Frontmatter YAML is malformed in ${opts.file}: ${opts.parseError}`);this.file = opts.file;this.parseError = opts.parseError;+ if (opts.refusalClass !== undefined) this.refusalClass = opts.refusalClass;+ if (opts.hint !== undefined) this.hint = opts.hint;}}@@ -96,6 +123,17 @@ export class FrontmatterMalformedError extends Error {* FrontmatterValueSchema (e.g. function,* Symbol leaf; `null` shapes are coerced to* empty values, not rejected)+ * - `'byte-0-promotion'` ← NOT a parse outcome: the bytes never got+ * as far as YAML. An append/prepend would+ * have placed a `---` fence pair at offset+ * 0, so the composed document would read+ * the agent's body as its frontmatter.+ * Carried on the error by the throw site+ * rather than inferred from `parseError`,+ * which keeps placement refusals out of the+ * `yaml-parse-error` bucket — that bucket's+ * whole job is spotting a schema/parser+ * regression by its count.* - `'unknown'` ← fallback (should be unreachable; logged* so a future parseError prefix that* doesn't match any branch surfaces@@ -105,6 +143,7 @@ export type FrontmatterMalformedClass =| 'yaml-parse-error'| 'non-mapping-top-level'| 'schema-rejection'+ | 'byte-0-promotion'| 'unknown';export function classifyParseError(parseError: string): FrontmatterMalformedClass {@@ -124,6 +163,42 @@ export function classifyParseError(parseError: string): FrontmatterMalformedClasreturn 'unknown';}+/**+ * The refusal's bounded-cardinality class. A throw site that knows its own+ * class wins; only the YAML gates fall through to prose classification.+ *+ * Module-private on purpose: both refusal surfaces reach it through+ * `logFrontmatterRefusal`, which is the single emission site. Exporting it+ * would re-open the door this consolidation closed — a caller reading the+ * class itself is one step from composing its own event and drifting on the+ * label, which is exactly how placement refusals ended up counted as YAML+ * parse errors on the batch path.+ */+function frontmatterRefusalClass(err: FrontmatterMalformedError): FrontmatterMalformedClass {+ return err.refusalClass ?? classifyParseError(err.parseError);+}++/** Agent-facing detail: the diagnosis plus whichever fix advice applies. */+export function frontmatterRefusalDetail(err: FrontmatterMalformedError): string {+ return `${err.parseError}. ${err.hint ?? FIX_HINT}`;+}++/**+ * Emit the structured refusal event. Both refusal surfaces route through here+ * so the grouped-by-handler counter stays consistent across gates.+ */+export function logFrontmatterRefusal(err: FrontmatterMalformedError, handler: string): void {+ console.warn(+ JSON.stringify({+ event: 'frontmatter-malformed-write-refused',+ handler,+ class: frontmatterRefusalClass(err),+ 'doc.name': stripDocExtension(err.file),+ parseError: err.parseError,+ }),+ );+}+/*** Translate a `FrontmatterMalformedError` into the slim RFC 9457 problem+json* envelope at HTTP 400. The exact wire shape is a 1-way-door contract — see@@ -131,10 +206,9 @@ export function classifyParseError(parseError: string): FrontmatterMalformedClas** Callers wrap their handler body in `try { ... } catch (err) { if (err* instanceof FrontmatterMalformedError) { respondFrontmatterMalformed(res, err, 'handler-name'); return; } throw err; }`.- *- * Emits the structured log event `frontmatter-malformed-write-refused` on- * every refusal — single emission site so the grouped-by-handler counter- * stays consistent across gates.+ * A surface that composes something other than an HTTP response (see+ * `agent-write-batch`) calls `logFrontmatterRefusal` + `frontmatterRefusalDetail`+ * instead, so the event and the detail stay identical across both.*/export function respondFrontmatterMalformed(res: ServerResponse,@@ -150,19 +224,10 @@ export function respondFrontmatterMalformed(// nested mappings + arrays of objects — a spike in `schema-rejection` is// now the signal that the schema regressed or a new non-representable// value class surfaced.- const refusalClass = classifyParseError(err.parseError);- console.warn(- JSON.stringify({- event: 'frontmatter-malformed-write-refused',- handler,- class: refusalClass,- 'doc.name': stripDocExtension(err.file),- parseError: err.parseError,- }),- );+ logFrontmatterRefusal(err, handler);errorResponse(res, 400, 'urn:ok:error:frontmatter-malformed', 'Frontmatter YAML is malformed.', {handler,- detail: `${err.parseError}. ${FIX_HINT}`,+ detail: frontmatterRefusalDetail(err),extensions: {file: err.file,parseError: err.parseError,diff --git a/packages/server/src/mcp/tools/write.test.ts b/packages/server/src/mcp/tools/write.test.tsindex 68af74757..1a1d4724a 100644--- a/packages/server/src/mcp/tools/write.test.ts+++ b/packages/server/src/mcp/tools/write.test.ts@@ -11,7 +11,8 @@import { stripFrontmatter } from '@inkeep/open-knowledge-core';import { describe, expect, it } from 'vitest';-import { composeWithFrontmatter } from './write.ts';+import { splitPayloadFrontmatter } from '../../payload-frontmatter.ts';+import { composeWithFrontmatter, frontmatterIgnoredNote } from './write.ts';/** Count leading-or-anywhere `---` fence lines that open a frontmatter block. */function frontmatterBlockCount(markdown: string): number {@@ -108,3 +109,52 @@ describe('composeWithFrontmatter', () => {expect(result.markdown).toBe('\nBody.');});});++describe('splitPayloadFrontmatter — the shared append/prepend partition rule', () => {+ it('treats a `---`-fenced NON-mapping span as body, so no drop is announced', () => {+ const payload = '---\n\n## Findings\n\n---\n\nTrailing note.\n';+ const split = splitPayloadFrontmatter(payload);+ expect(split.frontmatter).toBe('');+ expect(split.body).toBe(payload);+ });++ it('still partitions a real YAML-mapping block off the payload', () => {+ const split = splitPayloadFrontmatter('---\ntitle: Second\n---\n\nExtra.\n');+ expect(split.frontmatter).toBe('---\ntitle: Second\n---\n');+ expect(split.body).toBe('\nExtra.\n');+ });++ it('leaves a payload with no leading fence untouched', () => {+ const split = splitPayloadFrontmatter('Just a paragraph.\n');+ expect(split.frontmatter).toBe('');+ expect(split.body).toBe('Just a paragraph.\n');+ });+});++describe('frontmatterIgnoredNote — the two outcomes are distinguishable', () => {+ // A YAML typo is the only difference between "your frontmatter was dropped"+ // and "your frontmatter is now body text". The note has to say which.+ const dropped = frontmatterIgnoredNote('append', '---\ntitle: Fine\n---\nbody\n');+ const asBody = frontmatterIgnoredNote('append', '---\ntitle: Foo: Bar\n---\nbody\n');++ it('says DROPPED for a well-formed mapping', () => {+ expect(dropped).toContain('was ignored');+ expect(dropped).not.toContain('written as BODY');+ });++ it('says WRITTEN AS BODY for a span that is not a mapping', () => {+ // Without this branch the malformed case is silent — the write returns 200+ // and the agent's keys land in the document as literal text with no signal.+ expect(asBody).toContain('written as BODY');+ expect(asBody).toContain('not a YAML mapping');+ });++ it('stays silent when the payload never opened with a fence pair', () => {+ expect(frontmatterIgnoredNote('append', 'just a paragraph\n')).toBeNull();+ expect(frontmatterIgnoredNote('append', '---\njust a break\n')).toBeNull();+ });++ it('stays silent on replace, which does not drop anything', () => {+ expect(frontmatterIgnoredNote('replace', '---\ntitle: Fine\n---\nbody\n')).toBeNull();+ });+});diff --git a/packages/server/src/mcp/tools/write.ts b/packages/server/src/mcp/tools/write.tsindex f53588072..c3258837a 100644--- a/packages/server/src/mcp/tools/write.ts+++ b/packages/server/src/mcp/tools/write.ts@@ -37,6 +37,7 @@ import { parentFolderOf } from '../../content/nested-folder-rules.ts';import { applySubstitution, todayIsoUtc } from '../../content/substitution.ts';import { resolveTemplatesAvailable } from '../../content/templates-resolver.ts';import type { LocalApiDispatch } from '../../http/local-api-dispatch.ts';+import { splitPayloadFrontmatter } from '../../payload-frontmatter.ts';import type { AgentIdentity } from '../agent-identity.ts';import {formatAdvisoryBriefs,@@ -148,20 +149,39 @@ type WriteOneResult =// notes)./**- * A `---` frontmatter block in a `prepend`/`append` payload is dropped by the- * CRDT write path (a second FM block would be invalid). Surface a note instead- * of silently discarding it.+ * Describe what the write path did with a `---` block opening a+ * `prepend`/`append` payload. Two outcomes, and the agent needs to tell them+ * apart because they differ by nothing more than a YAML typo:+ *+ * - Parses as a mapping → real frontmatter, DROPPED (a second FM block would+ * be invalid). The body still lands.+ * - Does not parse as a mapping → never frontmatter, so the whole payload+ * lands as BODY, fence lines included.+ *+ * The second case is the one that bites: an agent that meant to set+ * frontmatter and mistyped the YAML gets its keys written into the document as+ * literal text. Reporting only the first case would leave that silent, which+ * is worse than the refusal this path used to return — the bytes are fine, but+ * the agent has no signal that its intent was not honoured.*/-function frontmatterIgnoredNote(position: string, markdown: string | undefined): string | null {+export function frontmatterIgnoredNote(+ position: string,+ markdown: string | undefined,+): string | null {if ((position !== 'prepend' && position !== 'append') || !markdown) return null;- if (stripFrontmatter(markdown).frontmatter.trim() === '') return null;- return `Note: a \`---\` frontmatter block in this \`${position}\` payload was ignored — frontmatter is written only with \`position: "replace"\`. To change frontmatter, use \`edit({ document: { path, frontmatter } })\` (patch) or \`write({ document: { path, content, position: "replace" } })\` (full rewrite).`;+ // The raw regex claim vs. what the mapping-only rule actually partitions.+ const claimed = stripFrontmatter(markdown).frontmatter.trim() !== '';+ if (!claimed) return null;+ if (splitPayloadFrontmatter(markdown).frontmatter.trim() !== '') {+ return `Note: a \`---\` frontmatter block in this \`${position}\` payload was ignored — frontmatter is written only with \`position: "replace"\`. To change frontmatter, use \`edit({ document: { path, frontmatter } })\` (patch) or \`write({ document: { path, content, position: "replace" } })\` (full rewrite).`;+ }+ return `Note: this \`${position}\` payload opens with a \`---\` fence pair, but what is between the fences is not a YAML mapping, so it was written as BODY text (fence lines included) rather than treated as frontmatter. If you meant a thematic break, that is the correct outcome — \`***\` or \`___\` avoids the ambiguity. If you meant to set frontmatter, the YAML did not parse: use \`edit({ document: { path, frontmatter } })\` (patch) or \`write({ document: { path, content, position: "replace" } })\` (full rewrite).`;}/** An `append`/`prepend` whose body is empty is a server-side no-op. */function emptyAppendNoOpNote(position: string, markdown: string | undefined): string | null {if ((position !== 'prepend' && position !== 'append') || markdown === undefined) return null;- if (stripFrontmatter(markdown).body !== '') return null;+ if (splitPayloadFrontmatter(markdown).body !== '') return null;return `No content to ${position} — document unchanged. To clear a document, use \`position: "replace"\` with empty \`content\`.`;}diff --git a/packages/server/src/payload-frontmatter.ts b/packages/server/src/payload-frontmatter.tsnew file mode 100644index 000000000..a206826c6--- /dev/null+++ b/packages/server/src/payload-frontmatter.ts@@ -0,0 +1,47 @@+/**+ * The append/prepend payload partition rule, shared by the two server-side+ * consumers that must agree on it byte-for-byte.+ *+ * Server-local on purpose. `@inkeep/open-knowledge-core` is a mirrored public+ * package, so an export there is a permanent surface commitment; both callers+ * live in this package, and the rule is an internal write-path policy rather+ * than a markdown primitive. Kept out of `agent-sessions.ts` (the composer+ * that owns the rule) because the MCP `write` tool would then drag Yjs,+ * Hocuspocus and the parse pool in for one predicate.+ */++import {+ parseFrontmatterYaml,+ stripFrontmatter,+ unwrapFrontmatterFences,+} from '@inkeep/open-knowledge-core';++/**+ * Partition an append/prepend payload into (frontmatter, body) under the rule+ * that only a YAML MAPPING counts as frontmatter.+ *+ * `FRONTMATTER_RE` claims any text that opens with a `---` fence and closes+ * with another one, which an ordinary body hits routinely — a thematic break,+ * a section separated by rules, a pasted `---`-delimited quote. Append and+ * prepend DISCARD whatever the partition claims, so a mis-claim silently eats+ * body content the caller asked to write. A claimed span that does not parse+ * as a mapping was never frontmatter, so it is un-split and the whole payload+ * is body.+ *+ * The two consumers must not diverge: `composeAgentWrite` writes by this rule,+ * and the MCP `write` tool's advisory notes describe the result to the agent.+ * When they disagree the note reports a drop that did not happen.+ * `replace`/`patch` do NOT use this — they supersede the frontmatter region+ * rather than discarding it, so they keep the raw `stripFrontmatter` partition.+ */+export function splitPayloadFrontmatter(markdown: string): {+ frontmatter: string;+ body: string;+} {+ const split = stripFrontmatter(markdown);+ if (split.frontmatter === '') return split;+ if (parseFrontmatterYaml(unwrapFrontmatterFences(split.frontmatter)).map === null) {+ return { frontmatter: '', body: markdown };+ }+ return split;+}