Skip to content

Beautiful, AI-native markdown IDE and LLM wiki

fix(ok): memoize Observer A splice parse per drain (PRD-8273) (#3853)

Andrew Mikofalvy38c38f363287
diff --git a/.changeset/observer-a-splice-parse-memo.md b/.changeset/observer-a-splice-parse-memo.md
new file mode 100644
index 000000000..56f10d60d
--- /dev/null
+++ b/.changeset/observer-a-splice-parse-memo.md
@@ -0,0 +1,5 @@
+---
+"@inkeep/open-knowledge": patch
+---
+
+Typing in a large document no longer costs the server twice the markdown parsing it needs. Every keystroke in the visual editor makes the server re-derive the document's source text, and that work included parsing the document's current bytes from scratch even though the previous keystroke had just produced those exact bytes and parsed them already. On a 231 KB document that redundant parse was roughly half the per-keystroke cost. The server now reuses the parse it already has when the bytes are unchanged, and falls back to parsing whenever they are not, so documents whose source is not what the serializer would emit behave exactly as before. Typing on large documents should feel more responsive; there is no change to what gets written to disk.
diff --git a/knip.config.ts b/knip.config.ts
index 8ad68f996..b61debe5e 100644
--- a/knip.config.ts
+++ b/knip.config.ts
@@ -124,9 +124,7 @@ export default {
'packages/server': {
entry: ['src/**/*.test.ts', 'src/parse-worker.ts'],
project: 'src/**',
- ignoreDependencies: [
- '@types/shell-quote',
- ],
+ ignoreDependencies: ['@types/shell-quote'],
},
'packages/cli': {
entry: ['src/**/*.test.ts', 'scripts/*.ts', 'tests/**/*.ts', 'src/parse-worker.ts'],
diff --git a/packages/app/tests/stress/crdt-stress.e2e.ts b/packages/app/tests/stress/crdt-stress.e2e.ts
index 3649fc2a9..c25270d81 100644
--- a/packages/app/tests/stress/crdt-stress.e2e.ts
+++ b/packages/app/tests/stress/crdt-stress.e2e.ts
@@ -93,19 +93,38 @@ test('S6: multi-turn stress — large content + user edits', async ({ page, api,
// Wait for Observer A to sync the user-typed marker into Y.Text('source').
// Same Observer-A-mediated Y.Text convergence as the content-propagation wait
// above, so it carries the same 30s budget. The marker round-trips client
- // keystroke → XmlFragment → server Observer A → Y.Text; across the 3 turns the
- // doc accumulates to ~3× the fixture, so by the later turns Observer A re-derives
- // Y.Text from a much larger fragment and that round-trip grows with it. The
- // marker always lands (it is never dropped — the final assertion below confirms
- // all three survive), just slowly under accumulating load.
+ // keystroke → XmlFragment → server Observer A → Y.Text against a document
+ // that is ~3× the fixture by the last turn. How many server drains the 15
+ // characters actually produce here is NOT measured — `delay: 5` makes one
+ // drain per keystroke the plausible shape, but the numbers below come from
+ // an offline probe that drove one character per transaction by
+ // construction, so treat per-drain costs as a per-drain figure and not as a
+ // count of this test's drains. The marker always lands (it is never dropped
+ // — the final assertion below confirms all three survive), just slowly.
//
- // This budget was once raised from 10s to 30s on the theory that it was merely
- // ungenerous. That turned out not to be the whole story, so do not read the
- // current number as a solved problem: this wait STILL times out under load, and
- // it is the open residual the comment at the top of the loop refers to. The
- // suspected cost is Observer A re-deriving Y.Text per keystroke against a
- // fragment that keeps growing, which is a server-side cost no client-side budget
- // fixes. Measure that re-derive before considering another bump.
+ // The budget was raised 10s → 30s once, on the theory that it was merely
+ // ungenerous. Do NOT raise it again: a third bump hides the mechanism, and the
+ // mechanism has since been measured, so there is nothing left to guess at.
+ //
+ // What it is NOT. The long-standing suspicion was that the cost is Observer A
+ // re-deriving Y.Text from the XmlFragment per keystroke, growing superlinearly
+ // with the fragment. Both halves are wrong. Measured offline (no browser, no
+ // dev server, no websocket) the bare re-derive — yXmlFragmentToProseMirrorRootNode
+ // + serialize — is ~28ms at 231 KB / 1502 children, about 6.6% of the drain, so
+ // even 15 drains of it is under half a second. And the drain scales LINEARLY,
+ // not superlinearly: 135 / 280 / 424 ms at 1× / 2× / 3× the fixture.
+ //
+ // What it is. `computeMapDrivenBodySplice` (packages/server/src/map-driven-splice.ts)
+ // is ~87% of the per-keystroke drain, doing full-document passes whose cost its
+ // own docblock has flagged as unbounded by document size since long before
+ // anyone measured it. One of its two parses is now memoized — a drain's
+ // `oldBody` is the previous drain's `newBody` — removing ~46% of the function's
+ // cost, pinned by parse-count tests in packages/server/src/map-driven-*.test.ts.
+ // The surviving serialize + parse are still O(document) and still why this wait
+ // needs a budget this large. See that docblock for what is left: the parse wants
+ // an incremental parser, while the serialize duplicates one the caller already
+ // did and is reducible under a condition spelled out there. Either way, not by
+ // raising the number below.
await page.waitForFunction(
(m: string) => window.__activeProvider?.document?.getText('source')?.toString()?.includes(m),
marker,
diff --git a/packages/core/src/bridge/bind-frontmatter-doc.ts b/packages/core/src/bridge/bind-frontmatter-doc.ts
index dfd95ef49..d7a845280 100644
--- a/packages/core/src/bridge/bind-frontmatter-doc.ts
+++ b/packages/core/src/bridge/bind-frontmatter-doc.ts
@@ -1,4 +1,3 @@
-
import type * as Y from 'yjs';
import type { Err, Ok, Result } from '../config/result.ts';
import { type FrontmatterValidationError, toFrontmatterIssue } from '../frontmatter/errors.ts';
diff --git a/packages/core/src/bridge/bridge-invariant.ts b/packages/core/src/bridge/bridge-invariant.ts
index 8bfdb40cd..005627195 100644
--- a/packages/core/src/bridge/bridge-invariant.ts
+++ b/packages/core/src/bridge/bridge-invariant.ts
@@ -1,4 +1,3 @@
-
import { fnv1aDigest } from './hash-util.ts';
export type BridgeInvariantSite = 'observer-b' | 'persistence' | 'test-harness';
diff --git a/packages/core/src/bridge/compose.ts b/packages/core/src/bridge/compose.ts
index 2a597ec3d..f4cc8a286 100644
--- a/packages/core/src/bridge/compose.ts
+++ b/packages/core/src/bridge/compose.ts
@@ -1,4 +1,3 @@
-
import { prependFrontmatter, stripFrontmatter } from '../extensions/frontmatter.ts';
export type ComposeAdjustment = 'none' | 'body' | 'frontmatter' | 'unresolved';
diff --git a/packages/core/src/bridge/divergence-locate.ts b/packages/core/src/bridge/divergence-locate.ts
index d979f4b39..6b505314d 100644
--- a/packages/core/src/bridge/divergence-locate.ts
+++ b/packages/core/src/bridge/divergence-locate.ts
@@ -1,4 +1,3 @@
-
import { findFirstDivergenceIndex } from './tolerance-telemetry.ts';
export const BRIDGE_LINE_KINDS = [
diff --git a/packages/core/src/bridge/doc-boundary-space.ts b/packages/core/src/bridge/doc-boundary-space.ts
index 01ef76a9f..8fb428b90 100644
--- a/packages/core/src/bridge/doc-boundary-space.ts
+++ b/packages/core/src/bridge/doc-boundary-space.ts
@@ -1,4 +1,3 @@
-
import { FRONTMATTER_RE, stripFrontmatter } from '../extensions/frontmatter.ts';
import { carriedEdgeEmpties, type DocEdgeEmpties } from '../markdown/doc-edge-blank-runs.ts';
diff --git a/packages/core/src/bridge/growth-detect.ts b/packages/core/src/bridge/growth-detect.ts
index 463e89595..9d9c244b0 100644
--- a/packages/core/src/bridge/growth-detect.ts
+++ b/packages/core/src/bridge/growth-detect.ts
@@ -1,4 +1,3 @@
-
const DEFAULT_MIN_SUBSTANTIVE_LINE_LENGTH = 16;
export const DUPLICATION_GATE_MIN_LINE_LENGTH = 8;
diff --git a/packages/core/src/bridge/intraline-predicate.ts b/packages/core/src/bridge/intraline-predicate.ts
index 0939885bb..c43fdd7e7 100644
--- a/packages/core/src/bridge/intraline-predicate.ts
+++ b/packages/core/src/bridge/intraline-predicate.ts
@@ -1,4 +1,3 @@
-
export function rawLineCounts(doc: string): Map<string, number> {
const counts = new Map<string, number>();
for (const line of doc.split('\n')) {
diff --git a/packages/core/src/bridge/merge-three-way.ts b/packages/core/src/bridge/merge-three-way.ts
index c0dc65705..251560a35 100644
--- a/packages/core/src/bridge/merge-three-way.ts
+++ b/packages/core/src/bridge/merge-three-way.ts
@@ -65,7 +65,6 @@ export function tryLineLevelCombine(
return { clean: true, merged };
}
-
export type BridgeMergeContentLossSide = 'user' | 'agent';
export type BridgeMergeContentLossWhich = 'substring' | 'order' | 'growth';
diff --git a/packages/core/src/bridge/normalize.ts b/packages/core/src/bridge/normalize.ts
index dcc5b0690..01ac80099 100644
--- a/packages/core/src/bridge/normalize.ts
+++ b/packages/core/src/bridge/normalize.ts
@@ -1,4 +1,3 @@
-
import { stripFrontmatter } from '../extensions/frontmatter.ts';
import { bodyEdgeEmpties } from './doc-boundary-space.ts';
diff --git a/packages/core/src/bridge/pm-structural-equivalence.ts b/packages/core/src/bridge/pm-structural-equivalence.ts
index 0be5203e4..754dc5cc2 100644
--- a/packages/core/src/bridge/pm-structural-equivalence.ts
+++ b/packages/core/src/bridge/pm-structural-equivalence.ts
@@ -32,7 +32,6 @@ const BR_SENTINEL: PmStructuralNode = { type: '__br__' };
const BR_LITERAL_RE = /^<br\s*\/?>$/i;
-
interface DegradeEntry {
readonly label: string;
readonly severity: ToleranceClassSeverity;
@@ -128,7 +127,6 @@ export interface ComparePmStructuralOptions {
ignoreAttrs?: (attrKey: string) => boolean;
}
-
/** Rebuild a node with `fn` applied to each child; identity when childless.
* Shared by the degrade canonicalizers so none re-implements the walk. */
function mapChildren(
@@ -250,7 +248,6 @@ function applyDegrades(tree: PmStructuralNode): {
return { normalized: current, fired };
}
-
export function comparePmStructural(
expected: PmStructuralNode,
actual: PmStructuralNode,
diff --git a/packages/core/src/bridge/subsequence.ts b/packages/core/src/bridge/subsequence.ts
index dca644141..af20de1e2 100644
--- a/packages/core/src/bridge/subsequence.ts
+++ b/packages/core/src/bridge/subsequence.ts
@@ -1,4 +1,3 @@
-
/** True when `needle` is a subsequence of `haystack` (two-pointer, O(n+m)):
* insertions in `haystack` are free; any dropped or substituted `needle` byte
* fails. */
diff --git a/packages/core/src/markdown/callout-transformer.ts b/packages/core/src/markdown/callout-transformer.ts
index a363dfd54..83d47404a 100644
--- a/packages/core/src/markdown/callout-transformer.ts
+++ b/packages/core/src/markdown/callout-transformer.ts
@@ -1,4 +1,3 @@
-
import type { Blockquote, Paragraph, Root } from 'mdast';
import type { MdxJsxAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/code-fence.ts b/packages/core/src/markdown/code-fence.ts
index 4499dbc92..9807a7e82 100644
--- a/packages/core/src/markdown/code-fence.ts
+++ b/packages/core/src/markdown/code-fence.ts
@@ -1,4 +1,3 @@
-
export function selectFenceChar(info: string): '`' | '~' {
return info.includes('`') ? '~' : '`';
}
diff --git a/packages/core/src/markdown/comment-promoter.ts b/packages/core/src/markdown/comment-promoter.ts
index 34704165b..7415c474a 100644
--- a/packages/core/src/markdown/comment-promoter.ts
+++ b/packages/core/src/markdown/comment-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Paragraph, PhrasingContent, Root, RootContent, Text } from 'mdast';
import { SKIP, visit } from 'unist-util-visit';
import type { VFile } from 'vfile';
diff --git a/packages/core/src/markdown/dedent-block-jsx-close.ts b/packages/core/src/markdown/dedent-block-jsx-close.ts
index b6299a948..ece5c02a1 100644
--- a/packages/core/src/markdown/dedent-block-jsx-close.ts
+++ b/packages/core/src/markdown/dedent-block-jsx-close.ts
@@ -1,4 +1,3 @@
-
import { findFencedRegions, isInsideFence } from './fence-regions.ts';
const INDENTED_BLOCK_JSX_CLOSE_RE = /^([ ]{1,3})(<\/[A-Z][A-Za-z0-9_]*\s*>)([ \t]*)$/gm;
diff --git a/packages/core/src/markdown/details-accordion-promoter.ts b/packages/core/src/markdown/details-accordion-promoter.ts
index acd398aa6..54774764a 100644
--- a/packages/core/src/markdown/details-accordion-promoter.ts
+++ b/packages/core/src/markdown/details-accordion-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Paragraph, Parent, Root, Text } from 'mdast';
import type { MdxJsxAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/div-align-promoter.ts b/packages/core/src/markdown/div-align-promoter.ts
index ba9b56384..43ecdc709 100644
--- a/packages/core/src/markdown/div-align-promoter.ts
+++ b/packages/core/src/markdown/div-align-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Paragraph, Parent, Root, Text } from 'mdast';
import type { MdxJsxFlowElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/fence-regions.ts b/packages/core/src/markdown/fence-regions.ts
index 4513da9bb..6d3f3643e 100644
--- a/packages/core/src/markdown/fence-regions.ts
+++ b/packages/core/src/markdown/fence-regions.ts
@@ -1,4 +1,3 @@
-
const FENCE_RE = /^(`{3,}|~{3,})/gm;
export function findFencedRegions(src: string): Array<[number, number]> {
diff --git a/packages/core/src/markdown/fixtures/index.ts b/packages/core/src/markdown/fixtures/index.ts
index 6931623c7..00f5771ee 100644
--- a/packages/core/src/markdown/fixtures/index.ts
+++ b/packages/core/src/markdown/fixtures/index.ts
@@ -1,4 +1,3 @@
-
import { readFileSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
@@ -9,7 +8,6 @@ function fixturePath(...segments: string[]): string {
return resolve(FIXTURES_DIR, ...segments);
}
-
interface GfmExample {
section: string;
markdown: string;
@@ -19,7 +17,6 @@ export function loadGfmExamples(): GfmExample[] {
return JSON.parse(readFileSync(fixturePath('gfm', 'examples.json'), 'utf8')) as GfmExample[];
}
-
interface MdxCrashEntry {
id: string;
input: string;
@@ -71,7 +68,6 @@ export function loadLargeEmbedFixtures(): LargeEmbedFixture[] {
) as LargeEmbedFixture[];
}
-
export function loadPrd6955Before(): string {
return readFileSync(fixturePath('regression', 'prd-6955-before.md'), 'utf8');
}
@@ -80,7 +76,6 @@ export function loadPrd6955CorruptedTriplicated(): string {
return readFileSync(fixturePath('regression', 'prd-6955-corrupted-triplicated.md'), 'utf8');
}
-
export interface NgPinnedCase {
id: string;
name: string;
@@ -97,7 +92,6 @@ export function loadNgPinnedCases(): NgPinnedCase[] {
) as NgPinnedCase[];
}
-
export function loadLargeRealistic(): string {
return readFileSync(fixturePath('perf', 'large-realistic.md'), 'utf8');
}
diff --git a/packages/core/src/markdown/fixtures/perf/generate.ts b/packages/core/src/markdown/fixtures/perf/generate.ts
index daa3ef82b..8b1eb8519 100644
--- a/packages/core/src/markdown/fixtures/perf/generate.ts
+++ b/packages/core/src/markdown/fixtures/perf/generate.ts
@@ -1,4 +1,3 @@
-
import { writeFileSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
@@ -11,7 +10,6 @@ const BASELINE_ONLY_COUNTS = [500, 2500] as const;
const SEED = 0xf1de1117;
-
function mulberry32(seed: number): () => number {
let a = seed >>> 0;
return () => {
@@ -38,7 +36,6 @@ function pickWeighted<T>(rand: () => number, items: readonly [T, number][]): T {
return items[items.length - 1][0];
}
-
const WORDS = [
'lorem',
'ipsum',
@@ -156,7 +153,6 @@ function mdxBlock(rand: () => number): string {
return `<${name}>\n\n${sentence(rand)}\n\n</${name}>`;
}
-
type BlockKind = 'paragraph' | 'heading' | 'list' | 'code' | 'table' | 'mdx';
const MIX: readonly [BlockKind, number][] = [
@@ -195,7 +191,6 @@ function generateDocument(blockCount: number, seed: number): string {
return `${blocks.join('\n\n')}\n`;
}
-
function main(): void {
for (const count of [...BLOCK_COUNTS, ...BASELINE_ONLY_COUNTS]) {
const doc = generateDocument(count, SEED);
diff --git a/packages/core/src/markdown/guard-flanking-matrix.ts b/packages/core/src/markdown/guard-flanking-matrix.ts
index 32c055c5b..420dc0fe5 100644
--- a/packages/core/src/markdown/guard-flanking-matrix.ts
+++ b/packages/core/src/markdown/guard-flanking-matrix.ts
@@ -7,7 +7,6 @@ import {
import { BACKSLASH_GUARD_SUBSTITUTIONS, encodeBackslashEscapes } from './backslash-escape-guard.ts';
import { ENTITY_REF_GUARD_SUBSTITUTIONS, encodeEntityRefs } from './entity-ref-guard.ts';
-
export const ATTENTION_DELIMITERS = ['*', '_', '**', '~~', '=='] as const;
export type FlankClass = 'whitespace' | 'punctuation' | 'other';
diff --git a/packages/core/src/markdown/handler-shadow-audit.ts b/packages/core/src/markdown/handler-shadow-audit.ts
index 283913255..2653473da 100644
--- a/packages/core/src/markdown/handler-shadow-audit.ts
+++ b/packages/core/src/markdown/handler-shadow-audit.ts
@@ -1,4 +1,3 @@
-
export interface HandlerShadowWitness {
input: string;
expect: 'byte' | 'byte-and-reparse-type';
diff --git a/packages/core/src/markdown/highlight-promoter.ts b/packages/core/src/markdown/highlight-promoter.ts
index 5d1000cc4..da0a81d23 100644
--- a/packages/core/src/markdown/highlight-promoter.ts
+++ b/packages/core/src/markdown/highlight-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Parent, PhrasingContent, Root, Text } from 'mdast';
import type { MdxJsxTextElement } from 'mdast-util-mdx';
import { SKIP, visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/html-to-mdast.ts b/packages/core/src/markdown/html-to-mdast.ts
index 9681c0056..787315157 100644
--- a/packages/core/src/markdown/html-to-mdast.ts
+++ b/packages/core/src/markdown/html-to-mdast.ts
@@ -1,4 +1,3 @@
-
import type { Element as HastElement, Root as HastRoot } from 'hast';
import type { List, Nodes as MdastNodes, Root as MdastRoot } from 'mdast';
import rehypeParse from 'rehype-parse';
@@ -129,8 +128,7 @@ export function htmlToMdast(html: string, options?: HtmlToMdastOptions): MdastRo
throw new HtmlPayloadTooLargeError(html.length, maxBytes);
}
- const processor = unified()
- .use(rehypeParse, { fragment: true });
+ const processor = unified().use(rehypeParse, { fragment: true });
for (const plugin of cleanupPlugins) {
processor.use(plugin);
diff --git a/packages/core/src/markdown/image-promoter.ts b/packages/core/src/markdown/image-promoter.ts
index 994eb6917..706fa9afb 100644
--- a/packages/core/src/markdown/image-promoter.ts
+++ b/packages/core/src/markdown/image-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Image, Paragraph, Root } from 'mdast';
import type { MdxJsxAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/index.ts b/packages/core/src/markdown/index.ts
index ec1ac1b7e..779e27500 100644
--- a/packages/core/src/markdown/index.ts
+++ b/packages/core/src/markdown/index.ts
@@ -1,4 +1,3 @@
-
import {
type FromProseMirrorOptions,
fromPmMark,
@@ -207,7 +206,6 @@ export class MarkdownManager {
}
}
-
const registry = createRegistry();
function destructureAttrs(
@@ -256,7 +254,6 @@ function destructureAttrs(
return result;
}
-
function hasDirtyDescendant(node: PmNode): boolean {
let found = false;
node.descendants((child) => {
@@ -279,7 +276,6 @@ function effectiveDirty(node: PmNode, freshnessChecker?: StructuralFreshnessChec
return freshnessChecker ? freshnessChecker.isDiverged(node.toJSON()) : false;
}
-
function isEmptyMdastParagraph(node: MdastNodes): boolean {
if (node.type !== 'paragraph') return false;
const children = node.children ?? [];
@@ -348,7 +344,6 @@ function extractTextFromMdastNodes(nodes: MdastNodes[]): string {
return out;
}
-
import {
AUDIO_EXTENSIONS,
FILE_ATTACHMENT_EXTENSIONS,
@@ -601,7 +596,6 @@ function buildMdastToPmHandlers(
}));
}
-
if (m.emphasis) {
handlers.emphasis = toPmMark(m.emphasis, (node: Emphasis) => ({
sourceDelimiter: node.data?.sourceDelimiter ?? '*',
@@ -682,7 +676,6 @@ function buildMdastToPmHandlers(
}));
}
-
if (m.link) {
const sourceLiteralMark = m.sourceLiteral;
handlers.link = (node: Link, _parent: MdastParent, state: MdastToPmState) => {
@@ -980,7 +973,6 @@ function buildMdastToPmHandlers(
};
}
-
const blockUnknownHandler = (node: {
type: string;
position?: { start: { offset: number }; end: { offset: number } };
@@ -1106,7 +1098,6 @@ function buildMdastToPmHandlers(
return handlers as RemarkProseMirrorOptions['handlers'];
}
-
function buildPmToMdastHandlers(
schema: Schema,
freshness: FreshnessCheckerHolder,
diff --git a/packages/core/src/markdown/lint/applies-to.ts b/packages/core/src/markdown/lint/applies-to.ts
index b79e9aef7..fa0be9e86 100644
--- a/packages/core/src/markdown/lint/applies-to.ts
+++ b/packages/core/src/markdown/lint/applies-to.ts
@@ -1,4 +1,3 @@
-
import picomatch from 'picomatch';
import { SUPPORTED_DOC_EXTENSIONS } from '../../constants/doc-extensions.ts';
diff --git a/packages/core/src/markdown/lint/config-files.ts b/packages/core/src/markdown/lint/config-files.ts
index 94b34dee1..312c9c74b 100644
--- a/packages/core/src/markdown/lint/config-files.ts
+++ b/packages/core/src/markdown/lint/config-files.ts
@@ -1,4 +1,3 @@
-
const MARKDOWNLINT_JSON_CONFIG_FILES: ReadonlySet<string> = new Set([
'.markdownlint.json',
'.markdownlint.jsonc',
diff --git a/packages/core/src/markdown/lint/config-schemas.ts b/packages/core/src/markdown/lint/config-schemas.ts
index fbbd309df..1a81aa524 100644
--- a/packages/core/src/markdown/lint/config-schemas.ts
+++ b/packages/core/src/markdown/lint/config-schemas.ts
@@ -52,7 +52,6 @@ const fullPluginShape = Object.fromEntries(
LINT_PLUGINS.map((plugin) => [plugin.id, plugin.sliceSchema]),
) as z.ZodRawShape;
-
export const LinterConfigSchema = z.object({
enabled: z.boolean(),
plugins: z.object(fullPluginShape),
diff --git a/packages/core/src/markdown/lint/default-config.ts b/packages/core/src/markdown/lint/default-config.ts
index a51f2ffb0..d48eea5e4 100644
--- a/packages/core/src/markdown/lint/default-config.ts
+++ b/packages/core/src/markdown/lint/default-config.ts
@@ -6,7 +6,6 @@ export const DEFAULT_MARKDOWNLINT_CONFIG: Record<string, MarkdownlintRuleSetting
MD013: false,
};
-
export function resolveMarkdownlintConfig(
rules: Record<string, MarkdownlintRuleSetting> | undefined,
): Configuration {
diff --git a/packages/core/src/markdown/lint/frontmatter-schema-edit.ts b/packages/core/src/markdown/lint/frontmatter-schema-edit.ts
index 9c9542661..ef478e268 100644
--- a/packages/core/src/markdown/lint/frontmatter-schema-edit.ts
+++ b/packages/core/src/markdown/lint/frontmatter-schema-edit.ts
@@ -1,4 +1,3 @@
-
import { CANONICAL_SCHEMA_DIALECT_URIS, DEFAULT_SCHEMA_DIALECT } from './frontmatter-validate.ts';
export interface FrontmatterFieldConstraint {
diff --git a/packages/core/src/markdown/lint/frontmatter-validate.ts b/packages/core/src/markdown/lint/frontmatter-validate.ts
index b0e7833fd..5f953d667 100644
--- a/packages/core/src/markdown/lint/frontmatter-validate.ts
+++ b/packages/core/src/markdown/lint/frontmatter-validate.ts
@@ -1,4 +1,3 @@
-
import Ajv, { type ValidateFunction } from 'ajv';
import Ajv2019 from 'ajv/dist/2019.js';
import Ajv2020 from 'ajv/dist/2020.js';
diff --git a/packages/core/src/markdown/lint/index.ts b/packages/core/src/markdown/lint/index.ts
index ffd277a90..11b5abdfd 100644
--- a/packages/core/src/markdown/lint/index.ts
+++ b/packages/core/src/markdown/lint/index.ts
@@ -1,4 +1,3 @@
-
import { LINT_PLUGINS, type LinterConfig } from './plugins.ts';
import type { LintDiagnostic } from './types.ts';
diff --git a/packages/core/src/markdown/lint/okf-frontmatter/materialize.ts b/packages/core/src/markdown/lint/okf-frontmatter/materialize.ts
index f438f1b16..7a5af097e 100644
--- a/packages/core/src/markdown/lint/okf-frontmatter/materialize.ts
+++ b/packages/core/src/markdown/lint/okf-frontmatter/materialize.ts
@@ -1,4 +1,3 @@
-
import type { OkfRuleId } from '../okf-rule-meta.ts';
import { fileNameFor, OKF_SCHEMA_DIR, okfSchemaPathFor } from './paths.ts';
import { OKF_FRONTMATTER_REGISTRY } from './registry.ts';
diff --git a/packages/core/src/markdown/lint/okf-frontmatter/paths.ts b/packages/core/src/markdown/lint/okf-frontmatter/paths.ts
index c290b6878..1ee913527 100644
--- a/packages/core/src/markdown/lint/okf-frontmatter/paths.ts
+++ b/packages/core/src/markdown/lint/okf-frontmatter/paths.ts
@@ -1,4 +1,3 @@
-
import type { OkfRuleId } from '../okf-rule-meta.ts';
export const OKF_SCHEMA_DIR = '.ok/okf';
diff --git a/packages/core/src/markdown/lint/okf-frontmatter/registry.ts b/packages/core/src/markdown/lint/okf-frontmatter/registry.ts
index 53ecb619d..f325fbbb4 100644
--- a/packages/core/src/markdown/lint/okf-frontmatter/registry.ts
+++ b/packages/core/src/markdown/lint/okf-frontmatter/registry.ts
@@ -1,4 +1,3 @@
-
import { compileAppliesTo } from '../applies-to.ts';
import { validateFrontmatterSource } from '../frontmatter-validate.ts';
import {
diff --git a/packages/core/src/markdown/lint/okf-frontmatter/schemas.ts b/packages/core/src/markdown/lint/okf-frontmatter/schemas.ts
index 89845d3d5..f1c37671b 100644
--- a/packages/core/src/markdown/lint/okf-frontmatter/schemas.ts
+++ b/packages/core/src/markdown/lint/okf-frontmatter/schemas.ts
@@ -1,4 +1,3 @@
-
// file-level form is deliberate: the sibling `biome-ignore` must sit on the line
// oxlint-disable unicorn/no-thenable
diff --git a/packages/core/src/markdown/lint/okf-project/rules.ts b/packages/core/src/markdown/lint/okf-project/rules.ts
index 8b476332e..6d691ae2b 100644
--- a/packages/core/src/markdown/lint/okf-project/rules.ts
+++ b/packages/core/src/markdown/lint/okf-project/rules.ts
@@ -1,4 +1,3 @@
-
import { SUPPORTED_DOC_EXTENSIONS } from '../../../constants/doc-extensions.ts';
import { isOkfRuleEnabled, type OkfRuleId, type OkfRuleToggles } from '../okf-rule-meta.ts';
diff --git a/packages/core/src/markdown/lint/okf-rule-meta.ts b/packages/core/src/markdown/lint/okf-rule-meta.ts
index 44913a9dc..4de50d79f 100644
--- a/packages/core/src/markdown/lint/okf-rule-meta.ts
+++ b/packages/core/src/markdown/lint/okf-rule-meta.ts
@@ -1,4 +1,3 @@
-
export const OKF_RULE_IDS = [
'no-wiki-links',
'log-shape',
diff --git a/packages/core/src/markdown/lint/okf-rules.ts b/packages/core/src/markdown/lint/okf-rules.ts
index d0999b62e..1d5538421 100644
--- a/packages/core/src/markdown/lint/okf-rules.ts
+++ b/packages/core/src/markdown/lint/okf-rules.ts
@@ -1,4 +1,3 @@
-
import { isOkfRuleEnabled, type OkfRuleId, type OkfRuleToggles } from './okf-rule-meta.ts';
import type { OkfRule } from './okf-runner.ts';
import { indexShape } from './rules/index-shape.ts';
diff --git a/packages/core/src/markdown/lint/okf-runner.ts b/packages/core/src/markdown/lint/okf-runner.ts
index fabb08d6e..de130e3f4 100644
--- a/packages/core/src/markdown/lint/okf-runner.ts
+++ b/packages/core/src/markdown/lint/okf-runner.ts
@@ -1,4 +1,3 @@
-
import type { Root } from 'mdast';
import remarkFrontmatter from 'remark-frontmatter';
import remarkGfm from 'remark-gfm';
diff --git a/packages/core/src/markdown/lint/plugins.ts b/packages/core/src/markdown/lint/plugins.ts
index 0a089990e..09c8450ee 100644
--- a/packages/core/src/markdown/lint/plugins.ts
+++ b/packages/core/src/markdown/lint/plugins.ts
@@ -1,4 +1,3 @@
-
import { z } from 'zod';
import { DEFAULT_MARKDOWNLINT_CONFIG, resolveMarkdownlintConfig } from './default-config.ts';
import {
@@ -24,7 +23,6 @@ import {
type OkfSlice,
} from './types.ts';
-
const MarkdownlintRuleSettingSchema = z.union([
z.boolean(),
z.enum(MARKDOWNLINT_RULE_SEVERITIES),
diff --git a/packages/core/src/markdown/lint/rules/index-shape.ts b/packages/core/src/markdown/lint/rules/index-shape.ts
index fcbb1b56c..59fe9b87d 100644
--- a/packages/core/src/markdown/lint/rules/index-shape.ts
+++ b/packages/core/src/markdown/lint/rules/index-shape.ts
@@ -1,4 +1,3 @@
-
import type { List, ListItem } from 'mdast';
import { EXIT, visit } from 'unist-util-visit';
import { defineScopedOkfRule } from '../okf-runner.ts';
diff --git a/packages/core/src/markdown/lint/rules/log-shape.ts b/packages/core/src/markdown/lint/rules/log-shape.ts
index 431e4e9c8..e0159519f 100644
--- a/packages/core/src/markdown/lint/rules/log-shape.ts
+++ b/packages/core/src/markdown/lint/rules/log-shape.ts
@@ -1,4 +1,3 @@
-
import type { Heading } from 'mdast';
import { toString as headingText } from 'mdast-util-to-string';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/lint/rules/no-wiki-links.ts b/packages/core/src/markdown/lint/rules/no-wiki-links.ts
index f75a04d15..2123e977f 100644
--- a/packages/core/src/markdown/lint/rules/no-wiki-links.ts
+++ b/packages/core/src/markdown/lint/rules/no-wiki-links.ts
@@ -1,4 +1,3 @@
-
import type { Point } from 'unist';
import { visit } from 'unist-util-visit';
import { defineOkfRule } from '../okf-runner.ts';
diff --git a/packages/core/src/markdown/lint/rules/reserved-casing.ts b/packages/core/src/markdown/lint/rules/reserved-casing.ts
index c8b4d8f28..89ef3e4a9 100644
--- a/packages/core/src/markdown/lint/rules/reserved-casing.ts
+++ b/packages/core/src/markdown/lint/rules/reserved-casing.ts
@@ -1,4 +1,3 @@
-
import { SUPPORTED_DOC_EXTENSIONS } from '../../../constants/doc-extensions.ts';
import { defineOkfRule } from '../okf-runner.ts';
diff --git a/packages/core/src/markdown/lint/types.ts b/packages/core/src/markdown/lint/types.ts
index c04fa4de0..6836b962d 100644
--- a/packages/core/src/markdown/lint/types.ts
+++ b/packages/core/src/markdown/lint/types.ts
@@ -1,4 +1,3 @@
-
import type { BrokenLinkReason } from '../../schemas/api/agent-write.ts';
import type { OkfRuleId } from './okf-rule-meta.ts';
@@ -113,7 +112,6 @@ export interface OkfSlice {
generate?: { index?: boolean };
}
-
interface RuleOptionSpecBase {
key: string;
description: string;
diff --git a/packages/core/src/markdown/lint/validation-counts.ts b/packages/core/src/markdown/lint/validation-counts.ts
index 437cc539a..98238656e 100644
--- a/packages/core/src/markdown/lint/validation-counts.ts
+++ b/packages/core/src/markdown/lint/validation-counts.ts
@@ -1,4 +1,3 @@
-
export interface ValidationSourceCounts {
errorCount: number;
warningCount: number;
diff --git a/packages/core/src/markdown/math-promoter.ts b/packages/core/src/markdown/math-promoter.ts
index bd79424c7..a5b511f25 100644
--- a/packages/core/src/markdown/math-promoter.ts
+++ b/packages/core/src/markdown/math-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Code, Root } from 'mdast';
import type { Math as MdastMath } from 'mdast-util-math';
import type { MdxJsxAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
diff --git a/packages/core/src/markdown/mdast-to-hast-handlers.ts b/packages/core/src/markdown/mdast-to-hast-handlers.ts
index 00fbc0035..ed89371d0 100644
--- a/packages/core/src/markdown/mdast-to-hast-handlers.ts
+++ b/packages/core/src/markdown/mdast-to-hast-handlers.ts
@@ -1,4 +1,3 @@
-
import type { Comment, Element, ElementContent, Properties } from 'hast';
import type { FootnoteDefinition, FootnoteReference } from 'mdast';
import type { MdxJsxFlowElement, MdxJsxTextElement } from 'mdast-util-mdx';
diff --git a/packages/core/src/markdown/mdast-to-html.ts b/packages/core/src/markdown/mdast-to-html.ts
index df788a80d..8ce2b2492 100644
--- a/packages/core/src/markdown/mdast-to-html.ts
+++ b/packages/core/src/markdown/mdast-to-html.ts
@@ -1,4 +1,3 @@
-
import type { Element, Root as HastRoot, Text as HastText } from 'hast';
import type { Root as MdastRoot, Text as MdastText } from 'mdast';
import type { Handler } from 'mdast-util-to-hast';
diff --git a/packages/core/src/markdown/merged-walker.ts b/packages/core/src/markdown/merged-walker.ts
index fa3cfcd86..bfd7e3109 100644
--- a/packages/core/src/markdown/merged-walker.ts
+++ b/packages/core/src/markdown/merged-walker.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Parent, Root } from 'mdast';
import { SKIP, visit } from 'unist-util-visit';
import type { VFile } from 'vfile';
diff --git a/packages/core/src/markdown/mermaid-promoter.ts b/packages/core/src/markdown/mermaid-promoter.ts
index 17ed2b829..1a897fca8 100644
--- a/packages/core/src/markdown/mermaid-promoter.ts
+++ b/packages/core/src/markdown/mermaid-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Code, Root } from 'mdast';
import type { MdxJsxAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/parse-with-fallback.ts b/packages/core/src/markdown/parse-with-fallback.ts
index 4aa0490c4..d988f4b8b 100644
--- a/packages/core/src/markdown/parse-with-fallback.ts
+++ b/packages/core/src/markdown/parse-with-fallback.ts
@@ -168,7 +168,6 @@ export function parseRecursive(
}
}
-
interface VFilePlace {
offset?: number;
start?: { offset?: number };
@@ -187,7 +186,6 @@ function extractErrorOffset(err: unknown): number | undefined {
return undefined;
}
-
interface Region {
start: number;
end: number;
@@ -211,7 +209,6 @@ function nearestBlankLineAfter(src: string, offset: number): number | null {
return null;
}
-
export interface TagEvent {
kind: 'open' | 'close' | 'self-close';
name: string;
@@ -358,7 +355,6 @@ function findFallbackRegion(src: string, errorOffset: number): Region {
return { start: blockStart, end: blockEnd };
}
-
interface SourceBlock {
src: string;
start: number;
@@ -456,7 +452,6 @@ function tryPerBlockFallback(
};
}
-
function wholeDocRawText(source: string): JSONContent {
return {
type: 'doc',
diff --git a/packages/core/src/markdown/parser-drop-closure.ts b/packages/core/src/markdown/parser-drop-closure.ts
index 43d420187..e6ab4fd57 100644
--- a/packages/core/src/markdown/parser-drop-closure.ts
+++ b/packages/core/src/markdown/parser-drop-closure.ts
@@ -1,4 +1,3 @@
-
export type DroppedTokenAdjudication =
| {
kind: 'format-dof-axis';
diff --git a/packages/core/src/markdown/pipeline.ts b/packages/core/src/markdown/pipeline.ts
index 55736041c..0e3224206 100644
--- a/packages/core/src/markdown/pipeline.ts
+++ b/packages/core/src/markdown/pipeline.ts
@@ -1,4 +1,3 @@
-
import {
type FromProseMirrorOptions,
fromProseMirror,
diff --git a/packages/core/src/markdown/position-aware-join.ts b/packages/core/src/markdown/position-aware-join.ts
index ccf06e94d..91cdf658f 100644
--- a/packages/core/src/markdown/position-aware-join.ts
+++ b/packages/core/src/markdown/position-aware-join.ts
@@ -1,4 +1,3 @@
-
import type { Join } from 'mdast-util-to-markdown';
import type { Position } from 'unist';
import { isBlankLineParagraph } from './zero-emission-blocks.ts';
diff --git a/packages/core/src/markdown/position-slice.ts b/packages/core/src/markdown/position-slice.ts
index e5b4c9eba..0f399d34c 100644
--- a/packages/core/src/markdown/position-slice.ts
+++ b/packages/core/src/markdown/position-slice.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Root } from 'mdast';
import { visit } from 'unist-util-visit';
import type { VFile } from 'vfile';
@@ -477,8 +476,7 @@ export function applyPositionSliceToNode(
const first = source[startOff];
if (first !== '[' && first !== '<' && !node.title) {
node.data.sourceStyle = 'gfm-autolink';
- }
- else if (first === '[') {
+ } else if (first === '[') {
const slice = source.slice(startOff, endOff);
const closeBracketIdx = slice.lastIndexOf('](');
if (closeBracketIdx !== -1) {
diff --git a/packages/core/src/markdown/rehype-plugins/skip-notion-whitespace.ts b/packages/core/src/markdown/rehype-plugins/skip-notion-whitespace.ts
index ca225ce9b..db10fa54b 100644
--- a/packages/core/src/markdown/rehype-plugins/skip-notion-whitespace.ts
+++ b/packages/core/src/markdown/rehype-plugins/skip-notion-whitespace.ts
@@ -1,4 +1,3 @@
-
import type { Comment, Element, Root, Text } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-cocoa-meta.ts b/packages/core/src/markdown/rehype-plugins/strip-cocoa-meta.ts
index 3a2524a9a..34068d103 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-cocoa-meta.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-cocoa-meta.ts
@@ -1,4 +1,3 @@
-
import type { Element, ElementContent, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-gdocs-wrapper.ts b/packages/core/src/markdown/rehype-plugins/strip-gdocs-wrapper.ts
index 2279a0189..93151d43b 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-gdocs-wrapper.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-gdocs-wrapper.ts
@@ -1,4 +1,3 @@
-
import type { Element, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-github-hovercard.ts b/packages/core/src/markdown/rehype-plugins/strip-github-hovercard.ts
index a0f467c42..188f02d83 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-github-hovercard.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-github-hovercard.ts
@@ -1,4 +1,3 @@
-
import type { Element, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-gmail-classes.ts b/packages/core/src/markdown/rehype-plugins/strip-gmail-classes.ts
index 4e8542a21..4ba2c3418 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-gmail-classes.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-gmail-classes.ts
@@ -1,4 +1,3 @@
-
import type { Element, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-gsheets-wrapper.ts b/packages/core/src/markdown/rehype-plugins/strip-gsheets-wrapper.ts
index a3c96b24a..1a74b1272 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-gsheets-wrapper.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-gsheets-wrapper.ts
@@ -1,4 +1,3 @@
-
import type { Element, ElementContent, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-mso-styles.ts b/packages/core/src/markdown/rehype-plugins/strip-mso-styles.ts
index e95cf43d6..de35e7b79 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-mso-styles.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-mso-styles.ts
@@ -1,4 +1,3 @@
-
import type { Comment, Element, ElementContent, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-slack-classes.ts b/packages/core/src/markdown/rehype-plugins/strip-slack-classes.ts
index ffdb4d5b5..b4be1b2ea 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-slack-classes.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-slack-classes.ts
@@ -1,4 +1,3 @@
-
import type { Element, ElementContent, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/rehype-plugins/strip-vscode-spans.ts b/packages/core/src/markdown/rehype-plugins/strip-vscode-spans.ts
index e2a96d1a7..3c8d88b7c 100644
--- a/packages/core/src/markdown/rehype-plugins/strip-vscode-spans.ts
+++ b/packages/core/src/markdown/rehype-plugins/strip-vscode-spans.ts
@@ -1,4 +1,3 @@
-
import type { Element, Root } from 'hast';
import type { Plugin } from 'unified';
diff --git a/packages/core/src/markdown/resolve-image-url.ts b/packages/core/src/markdown/resolve-image-url.ts
index 0389575ac..cf2c9da8e 100644
--- a/packages/core/src/markdown/resolve-image-url.ts
+++ b/packages/core/src/markdown/resolve-image-url.ts
@@ -1,4 +1,3 @@
-
import { isRelativeUrl } from './safe-url.ts';
function isDevDiagnosticContext(): boolean {
diff --git a/packages/core/src/markdown/safe-url.ts b/packages/core/src/markdown/safe-url.ts
index 8be632ba4..4d3250d3a 100644
--- a/packages/core/src/markdown/safe-url.ts
+++ b/packages/core/src/markdown/safe-url.ts
@@ -1,4 +1,3 @@
-
export const SAFE_URL_SCHEMES = ['https', 'http', 'mailto', 'tel', 'ftp', 'sms'] as const;
const SCHEME_ALT = SAFE_URL_SCHEMES.map((s) => `${s}:`).join('|');
diff --git a/packages/core/src/markdown/serialize-fidelity.test-helper.ts b/packages/core/src/markdown/serialize-fidelity.test-helper.ts
index 96304e7a5..0dd54070e 100644
--- a/packages/core/src/markdown/serialize-fidelity.test-helper.ts
+++ b/packages/core/src/markdown/serialize-fidelity.test-helper.ts
@@ -1,4 +1,3 @@
-
import type { JSONContent } from '@tiptap/core';
import type { Nodes } from 'mdast';
import { expect } from 'vitest';
diff --git a/packages/core/src/markdown/serialize-helpers.ts b/packages/core/src/markdown/serialize-helpers.ts
index 93798cf83..7cac5b042 100644
--- a/packages/core/src/markdown/serialize-helpers.ts
+++ b/packages/core/src/markdown/serialize-helpers.ts
@@ -1,4 +1,3 @@
-
import type { Node as PmNode } from '@tiptap/pm/model';
import type {
MdxJsxAttribute,
diff --git a/packages/core/src/markdown/single-dollar-math-promoter.ts b/packages/core/src/markdown/single-dollar-math-promoter.ts
index 8c08b7a04..b012a5aff 100644
--- a/packages/core/src/markdown/single-dollar-math-promoter.ts
+++ b/packages/core/src/markdown/single-dollar-math-promoter.ts
@@ -1,4 +1,3 @@
-
import type { PhrasingContent, Root, Text } from 'mdast';
import type { InlineMath } from 'mdast-util-math';
import { SKIP, visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/to-markdown-handlers.ts b/packages/core/src/markdown/to-markdown-handlers.ts
index 83d4af672..b50bea5ec 100644
--- a/packages/core/src/markdown/to-markdown-handlers.ts
+++ b/packages/core/src/markdown/to-markdown-handlers.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Parents } from 'mdast';
import type { MdxJsxAttribute, MdxJsxExpressionAttribute, MdxJsxFlowElement } from 'mdast-util-mdx';
import type { Handle, Info, State } from 'mdast-util-to-markdown';
diff --git a/packages/core/src/markdown/underline-promoter.ts b/packages/core/src/markdown/underline-promoter.ts
index 6e6d73400..0f1b0f27d 100644
--- a/packages/core/src/markdown/underline-promoter.ts
+++ b/packages/core/src/markdown/underline-promoter.ts
@@ -1,4 +1,3 @@
-
import type { Nodes, Parent, Root } from 'mdast';
import type { MdxJsxTextElement } from 'mdast-util-mdx';
import { visit } from 'unist-util-visit';
diff --git a/packages/core/src/markdown/unknown-mdast-guard.ts b/packages/core/src/markdown/unknown-mdast-guard.ts
index 932d7d2a8..1a082bda7 100644
--- a/packages/core/src/markdown/unknown-mdast-guard.ts
+++ b/packages/core/src/markdown/unknown-mdast-guard.ts
@@ -1,4 +1,3 @@
-
import type { Root as MdastRoot } from 'mdast';
import type { VFile } from 'vfile';
diff --git a/packages/core/src/markdown/whitespace-char-ref.ts b/packages/core/src/markdown/whitespace-char-ref.ts
index ac9f5239b..d578476a7 100644
--- a/packages/core/src/markdown/whitespace-char-ref.ts
+++ b/packages/core/src/markdown/whitespace-char-ref.ts
@@ -1,4 +1,3 @@
-
const INLINE_WHITESPACE_BY_CODE: ReadonlyMap<number, string> = new Map([
[0x20, ' '],
[0x09, '\t'],
diff --git a/packages/core/src/markdown/wiki-link-micromark.ts b/packages/core/src/markdown/wiki-link-micromark.ts
index afd9936fe..6dad26200 100644
--- a/packages/core/src/markdown/wiki-link-micromark.ts
+++ b/packages/core/src/markdown/wiki-link-micromark.ts
@@ -17,7 +17,6 @@ declare module 'micromark-util-types' {
}
}
-
const CODE_BANG = 33; // !
const CODE_LBRACKET = 91; // [
const CODE_RBRACKET = 93; // ]
@@ -261,7 +260,6 @@ export function wikiLinkSyntax(): Extension {
};
}
-
function enterWikiLink(this: CompileContext, token: Token) {
this.enter(
{
@@ -387,7 +385,6 @@ export const wikiLinkToMarkdown: {
unsafe: [{ character: '[', inConstruct: ['phrasing'] }],
};
-
const MICROMARK_EXT = wikiLinkSyntax();
export function remarkWikiLink(this: Processor) {
diff --git a/packages/server/src/map-driven-observer-a.test.ts b/packages/server/src/map-driven-observer-a.test.ts
index 1635f9a0f..c513d6641 100644
--- a/packages/server/src/map-driven-observer-a.test.ts
+++ b/packages/server/src/map-driven-observer-a.test.ts
@@ -20,6 +20,7 @@ import { composeAndWriteRawBody } from './bridge-intake.ts';
import { getLogger } from './logger.ts';
import { computeMapDrivenBodySplice } from './map-driven-splice.ts';
import { getMetrics } from './metrics.ts';
+import { createCountingManager } from './parse-counting.test-helper.ts';
import {
__resetMapDrivenParseErrorWarnForTests,
OBSERVER_SYNC_ORIGIN,
@@ -444,3 +445,67 @@ describe('map-driven Observer A — default Path A behavior', () => {
});
});
});
+
+describe('typing-burst parse economy (PRD-8273)', () => {
+ /** Append one character to the fragment's deepest last text node — a keystroke. */
+ function typeChar(doc: Y.Doc, xmlFragment: Y.XmlFragment, ch: string): void {
+ doc.transact(() => {
+ let node: Y.XmlElement | Y.XmlText | Y.XmlHook | undefined = xmlFragment.get(
+ xmlFragment.length - 1,
+ );
+ while (node instanceof Y.XmlElement && node.length > 0) {
+ node = node.get(node.length - 1);
+ }
+ if (!(node instanceof Y.XmlText)) throw new Error('no text node to type into');
+ node.insert(node.length, ch);
+ });
+ }
+
+ /**
+ * Each keystroke drain runs `computeMapDrivenBodySplice`. Its two parses are
+ * `oldBody` (what Y.Text holds now) and `newBody` (the canonical
+ * serialization of the fragment) — and across consecutive drains those are
+ * the SAME bytes, because the splice a drain applies leaves Y.Text holding
+ * exactly the `newBody` it just parsed.
+ *
+ * The memo serves that `oldBody` side, so a drain parses once rather than
+ * twice — which is what this test pins. Without it a 15-character marker
+ * costs 30 full-document parses at any document size; with it, 15.
+ *
+ * It pins the COUNT rather than a wall-clock budget deliberately: a parse
+ * count is machine-independent in a way milliseconds are not, so this cannot
+ * go flaky on a contended runner.
+ */
+ test('consecutive keystroke drains parse each body once, not once per drain', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const { doc, xmlFragment, ytext } = createTestDoc();
+ const cleanup = setupServerObservers({
+ doc,
+ xmlFragment,
+ ytext,
+ mdManager: counted,
+ schema,
+ });
+
+ doc.transact(() => {
+ const pmNode = schema.nodeFromJSON(counted.parse('# H\n\nalpha\n'));
+ updateYFragment(doc, xmlFragment, pmNode, { mapping: new Map(), isOMark: new Map() });
+ });
+
+ const before = parses();
+ for (const ch of 'XYZ') typeChar(doc, xmlFragment, ch);
+ const burstParses = parses() - before;
+
+ // The marker must actually have landed — a stalled bridge would also
+ // report a low parse count.
+ expect(ytext.toString()).toContain('alphaXYZ');
+
+ // One parse per keystroke, not two: each drain's `oldBody` is the body the
+ // PREVIOUS drain already parsed as its `newBody` (the seeding drain above
+ // primes the memo the same way), so only the newly-serialized side is real
+ // work. Three keystrokes, three parses.
+ expect(burstParses).toBe(3);
+
+ cleanup();
+ });
+});
diff --git a/packages/server/src/map-driven-splice.test.ts b/packages/server/src/map-driven-splice.test.ts
index b6b9ac03e..666619a84 100644
--- a/packages/server/src/map-driven-splice.test.ts
+++ b/packages/server/src/map-driven-splice.test.ts
@@ -1,7 +1,8 @@
import { MarkdownManager, sharedExtensions } from '@inkeep/open-knowledge-core';
import type { JSONContent } from '@tiptap/core';
import { describe, expect, test } from 'vitest';
-import { computeMapDrivenBodySplice } from './map-driven-splice.ts';
+import { computeMapDrivenBodySplice, createEditorMdastMemo } from './map-driven-splice.ts';
+import { createCountingManager } from './parse-counting.test-helper.ts';
const mdManager = new MarkdownManager({ extensions: sharedExtensions });
@@ -204,3 +205,190 @@ describe('computeMapDrivenBodySplice', () => {
});
});
});
+
+describe('editor-mdast parse memo (PRD-8273)', () => {
+ test('a repeated body is parsed once, not once per call', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const memo = createEditorMdastMemo();
+ const bodyA = '# H\n\nalpha\n';
+
+ // Drain 1: old=A, new=B. Drain 2 then arrives with old=B.
+ const first = computeMapDrivenBodySplice(
+ bodyA,
+ counted.parse('# H\n\nalphaX\n'),
+ counted,
+ undefined,
+ memo,
+ );
+ expect(first).not.toBeNull();
+ if (!first) return;
+ const bodyB = applySplice(bodyA, first);
+ const afterFirst = parses();
+
+ computeMapDrivenBodySplice(bodyB, counted.parse('# H\n\nalphaXY\n'), counted, undefined, memo);
+
+ // Drain 2 parsed only its own `newBody`; `bodyB` came from the memo.
+ expect(parses() - afterFirst).toBe(1);
+ });
+
+ /**
+ * The memo's entire safety argument is that its key is the body BYTES. An
+ * external write, a reconnect resync, or a concurrent client can change the
+ * body between drains; a memo that served its cached tree there would splice
+ * against a document that no longer exists — silent content corruption, far
+ * worse than the latency the memo saves. So: prime it, then hand it a
+ * DIFFERENT body and assert the splice is computed against the bytes
+ * actually passed in.
+ */
+ test('a body changed out from under the memo misses rather than serving a stale parse', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const memo = createEditorMdastMemo();
+
+ // Prime: this call leaves the memo holding the canonical newBody.
+ const primed = '# H\n\nalpha\n';
+ computeMapDrivenBodySplice(primed, counted.parse('# H\n\nalphaX\n'), counted, undefined, memo);
+ const afterPrime = parses();
+
+ // A concurrent writer replaced the body wholesale. Same doc, different bytes.
+ const external = '# DIFFERENT\n\nomega\n\ntail\n';
+ const splice = computeMapDrivenBodySplice(
+ external,
+ counted.parse('# DIFFERENT\n\nomega EDITED\n\ntail\n'),
+ counted,
+ undefined,
+ memo,
+ );
+ expect(splice).not.toBeNull();
+ if (!splice) return;
+
+ // It re-parsed both sides — no stale hit.
+ expect(parses() - afterPrime).toBe(2);
+
+ // And the splice is anchored in the EXTERNAL bytes: applying it reproduces
+ // the intended body, and the untouched tail survives byte-identically.
+ const result = applySplice(external, splice);
+ expect(result).toContain('omega EDITED');
+ expect(result).toContain('# DIFFERENT');
+ expect(result.endsWith('tail\n')).toBe(true);
+ });
+
+ /**
+ * The equivalence that matters is "a splice built from a memo HIT equals one
+ * built from a fresh parse". Passing a FRESH memo does not test that: both
+ * of its lookups miss, so the memoized call runs the identical cold path as
+ * the unmemoized one and the comparison asserts nothing about the memo. This
+ * test therefore primes the memo first and PROVES the hit landed — via the
+ * parse count — before comparing. Without that assertion the test could
+ * silently decay back into comparing two cold paths.
+ */
+ test('a splice built from a memo hit equals one built from a fresh parse', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const memo = createEditorMdastMemo();
+ const bodyA = '# H\n\none\n\ntwo\n\nthree\n';
+
+ // Drain 1 primes the memo: the `newBody` it parses is the byte-identical
+ // `oldBody` that drain 2 arrives with.
+ const first = computeMapDrivenBodySplice(
+ bodyA,
+ counted.parse('# H\n\none EDITED\n\ntwo\n\nthree\n'),
+ counted,
+ undefined,
+ memo,
+ );
+ expect(first).not.toBeNull();
+ if (!first) return;
+ const bodyB = applySplice(bodyA, first);
+
+ // Drain 2: `oldBody` is served from the memo, `newBody` is parsed fresh.
+ const newPm = counted.parse('# H\n\none EDITED\n\ntwo CHANGED\n\nthree\n');
+ const before = parses();
+ const fromHit = computeMapDrivenBodySplice(bodyB, newPm, counted, undefined, memo);
+ // ONE parse, not two — this is the assertion that makes the comparison
+ // below meaningful rather than tautological.
+ expect(parses() - before).toBe(1);
+
+ const fromFreshParse = computeMapDrivenBodySplice(bodyB, newPm, counted);
+ expect(fromHit).toEqual(fromFreshParse);
+ expect(fromHit).not.toBeNull();
+ });
+
+ /**
+ * Pins the precondition the docblock states: the hit depends on the
+ * preserved region's bytes being what the serializer would emit. Trailing
+ * whitespace survives the splice into Y.Text but is stripped from `newBody`,
+ * so the next drain's `oldBody` is NOT the previous `newBody` and must
+ * re-parse. Documented behaviour, so it gets a test rather than only a
+ * sentence — a stated precondition nobody exercises is how the sentence
+ * drifts away from the code.
+ */
+ test('preserved bytes the serializer would not emit cause the next drain to miss', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const memo = createEditorMdastMemo();
+ // The trailing run on `one` is preserved by the splice, absent from newBody.
+ const bodyA = 'one \n\ntwo\n\nthree\n';
+
+ const first = computeMapDrivenBodySplice(
+ bodyA,
+ counted.parse('one \n\ntwo A\n\nthree\n'),
+ counted,
+ undefined,
+ memo,
+ );
+ expect(first).not.toBeNull();
+ if (!first) return;
+ const bodyB = applySplice(bodyA, first);
+
+ const before = parses();
+ computeMapDrivenBodySplice(
+ bodyB,
+ counted.parse('one \n\ntwo B\n\nthree\n'),
+ counted,
+ undefined,
+ memo,
+ );
+ // Two parses: the memo held the canonical newBody, not these bytes.
+ expect(parses() - before).toBe(2);
+ });
+
+ /**
+ * The safety argument for this memo is that it keys on the body BYTES, so a
+ * changed body is a different key and therefore a miss. A same-length,
+ * different-content body is the one input shape that separates a genuine byte
+ * comparison from a length check — or from any other shortcut that happens to
+ * satisfy the other tests. The two fixtures below are byte-equal in length and
+ * differ in block STRUCTURE (three paragraphs vs two), so a length-based
+ * comparison would serve the wrong tree and compute a splice anchored in the
+ * wrong block boundaries.
+ */
+ test('a same-length body with different content misses — the key is bytes, not length', () => {
+ const { manager: counted, parses } = createCountingManager();
+ const memo = createEditorMdastMemo();
+
+ // A call leaves the memo holding that call's `newBody`, so the primed key
+ // is DERIVED from the serializer rather than assumed to equal a literal.
+ const primed = counted.serialize(counted.parse('one\n\ntwo\n\nthree\n'));
+ // A different body of the same byte length, with different block structure
+ // (two paragraphs, not three). Both premises are asserted, not assumed, so
+ // a serializer change or a fixture edit fails loudly instead of quietly
+ // turning this into a test of nothing.
+ const actual = 'one two\n\nthreeX\n';
+ expect(actual.length).toBe(primed.length);
+ expect(actual).not.toBe(primed);
+
+ // Prime: this call's `newBody` IS `primed`, so the memo ends up keyed on it.
+ computeMapDrivenBodySplice('zzz\n', counted.parse(primed), counted, undefined, memo);
+
+ const newPm = counted.parse('one two\n\nthreeY\n');
+ const before = parses();
+ const withMemo = computeMapDrivenBodySplice(actual, newPm, counted, undefined, memo);
+ // TWO parses — both sides are new bytes. Measured against a length-based
+ // comparison this reads ZERO, because `newBody` here is also 16 bytes and
+ // so would "match" as well; that is the shape this test exists to reject.
+ expect(parses() - before).toBe(2);
+
+ // And the splice is anchored in `actual`'s block boundaries, not `primed`'s.
+ const withoutMemo = computeMapDrivenBodySplice(actual, newPm, counted);
+ expect(withMemo).toEqual(withoutMemo);
+ expect(withMemo).not.toBeNull();
+ });
+});
diff --git a/packages/server/src/map-driven-splice.ts b/packages/server/src/map-driven-splice.ts
index 4084df8ef..6c8e27c6e 100644
--- a/packages/server/src/map-driven-splice.ts
+++ b/packages/server/src/map-driven-splice.ts
@@ -7,22 +7,80 @@
* structurally between old and new. Untouched prefix + suffix blocks
* stay in Y.Text byte-identical.
*
- * The computation is pure (no Y.Doc / no observers / no side effects):
- * parse oldBody → mdast, serialize new PM JSON → canonical newBody, parse
- * that → mdast, then walk the two tree.children sequences to find the
+ * The computation touches no Y.Doc and no observers, and its RESULT is a pure
+ * function of its inputs. It is not side-effect-free: when a `memo` is passed
+ * it writes the parse it just performed into that memo. The write is
+ * cache-only — it can change how long a later call takes, never what any call
+ * returns — but "pure, no side effects" would now be a false claim, and the
+ * distinction matters to anyone reasoning about calling this concurrently.
+ *
+ * Shape: parse oldBody → mdast, serialize new PM JSON → canonical newBody,
+ * parse that → mdast, then walk the two tree.children sequences to find the
* longest common prefix + suffix under structural equality (position
* stripped) and emit splice = [oldPrefixEnd, oldSuffixStart] replaced by
* newBody.slice(newPrefixEnd, newSuffixStart).
*
- * "Structural equality" ignores `position` so a block that round-trips
- * to a canonical form (`*italic*` → `_italic_`) is treated as equal —
+ * "Structural equality" ignores `position` so a block whose authored bytes
+ * differ from what the serializer would emit (a paragraph carrying a trailing
+ * whitespace run, say: `one ` serializes to `one`) is treated as equal —
* the OLD bytes survive in Y.Text untouched.
*
- * Perf envelope: three full-document passes (parse + serialize + parse)
- * per drain-settle, synchronous, unbounded by doc size. Measured on a
- * 675 KB fuzz document this is hundreds of ms per drain — acceptable for
- * the settle path (it replaces incremental-diff work of similar order),
- * but a future large-doc latency report should start here.
+ * Perf envelope: TWO full-document passes per drain-settle when the `memo`
+ * hits (serialize + one parse), three when it misses. Synchronous, unbounded
+ * by doc size. On a 231 KB document this function is ~87% of Observer A's
+ * per-keystroke drain cost; on a 675 KB fuzz document, hundreds of ms per
+ * drain. That is acceptable for the settle path — it replaces incremental-diff
+ * work of similar order — but it is where large-doc drain latency lives, which
+ * an earlier revision of this docblock predicted before anyone measured it.
+ *
+ * The pass the memo removes is the `oldBody` parse. Across consecutive drains
+ * the `oldBody` handed in is byte-identical to the `newBody` the PREVIOUS
+ * drain parsed, because the splice that drain applied left Y.Text holding
+ * exactly those bytes (see `EditorMdastMemo`).
+ *
+ * That byte-identity is CONDITIONAL, and the condition is worth stating because
+ * it is the whole reason the saving is real: the splice deliberately preserves
+ * the OLD bytes outside the changed region, so the drain's result equals its
+ * canonical `newBody` only when those preserved bytes are what the serializer
+ * would itself emit. Measured: a body whose preserved region round-trips
+ * byte-exact hits on the next drain (1 parse); the same shape carrying trailing
+ * whitespace misses (2 parses), because the preserved run survives into Y.Text
+ * while `newBody` has it stripped. The hit is the normal case: this pipeline
+ * preserves an unusually wide class byte-exact — emphasis markers (`*italic*`,
+ * `_italic_`, `**bold**`, `__bold__` all survive as authored), setext headings,
+ * multi-blank runs, unpadded tables, `+` bullets, indented code, a heading
+ * tight against its paragraph — so the miss class is essentially dirty bytes
+ * (trailing whitespace, CRLF, a missing final newline). A document full of
+ * those pays both parses on every drain, correctly and silently: a miss is only
+ * ever slower, never wrong.
+ *
+ * The surviving serialize + parse are still O(document) and unbounded by doc
+ * size. The parse needs a real incremental parser to shrink. The SERIALIZE is
+ * NOT irreducible, and saying so would send the next investigator at a hard
+ * problem while an easy one sits upstream: Observer A already serialized this
+ * same PM JSON before calling here (the `body` it derives for the
+ * composition), so a caller-supplied `newBody` would drop this call entirely.
+ * Measured at ~37 ms on the 231 KB fixture.
+ *
+ * It is not threaded through because the two are not always the same string.
+ * The caller passes `skipFreshnessDerive: !freshnessSafe`; this call passes no
+ * options. Measured against the server manager (which sets
+ * `deriveStructuralFreshness`, so the flag is live): on a quiescent drain the
+ * two are byte-identical, but on a non-quiescent one they diverge — a
+ * diverged-but-pristine `jsxComponent` emits stale `sourceRaw` for the caller
+ * and freshly re-derived bytes here (50 vs 51 bytes on a `<Callout>` whose body
+ * text was edited away from the `sourceRaw` its pristine fast path still holds,
+ * the shape a `.private.` freshness fixture exercises). WHICH side of that
+ * divergence is the hazard is not settled here, and the next reader should
+ * inherit the question rather than one answer: the caller's suppressed string
+ * is knowingly historical, while `SerializeCallOptions` names this splice's own
+ * text-match as a convergence mechanism and assigns the risk to the unsuppressed
+ * side. What this call site actually needs is only that the two are not
+ * interchangeable. Threading the caller's string in only
+ * when `freshnessSafe` holds would settle it, but it couples this otherwise
+ * input-only computation to the observer's freshness state on the bridge's
+ * highest-risk path, so it wants its own fuzz run rather than a ride on
+ * someone else's.
*
* When the changed region is a single container block on both sides
* (blockquote / list / listItem) whose own fields are untouched, the walk
@@ -38,7 +96,7 @@
*/
import type { MarkdownManager } from '@inkeep/open-knowledge-core';
import type { JSONContent } from '@tiptap/core';
-import type { RootContent } from 'mdast';
+import type { Root, RootContent } from 'mdast';
export interface MapDrivenSplice {
readonly spliceStart: number;
@@ -46,11 +104,78 @@ export interface MapDrivenSplice {
readonly newSlice: string;
}
+/**
+ * One-entry parse memo for the splice computer, owned per-document by the
+ * Observer A closure (created at attach, dropped at detach) so two documents
+ * editing concurrently never evict each other and nothing outlives the
+ * observers.
+ *
+ * The key is the body BYTES, not a generation counter or a doc revision, and
+ * that is the whole safety argument: a hit is only ever returned for a string
+ * equal to the one that produced the tree, so an external write, a reconnect
+ * resync, or a concurrent client that changes the body out from under the memo
+ * simply misses and re-parses. There is no state under which a stale tree can
+ * be served — the failure mode of a wrong splice is far worse than the latency
+ * this saves, so the memo is designed to have no such mode rather than to
+ * detect one. The one thing the key does NOT carry is parse configuration, so
+ * a memo is scoped to a single `MarkdownManager`: sharing one across managers
+ * configured differently would let a content hit serve a tree the other
+ * manager would not have produced.
+ *
+ * Safe to share the returned tree because the splice walk never mutates it:
+ * every consumer either reads `position` offsets or compares via
+ * `stringifyIgnorePosition`, and `tryNarrowIntoContainer` spreads into a fresh
+ * object rather than editing in place. No tree escapes the computer either —
+ * `MapDrivenSplice` carries two numbers and a string.
+ *
+ * Retained cost, since latency is not the only budget: one entry holds an
+ * mdast tree plus its body string, measured at 2.57 MB for a 231 KB body
+ * (~11.6x the bytes). Before this memo those trees were transient. For
+ * proportion, the loaded Y.Doc for that same content already retains 6.1 MB
+ * (~27.6x) in its XmlFragment and Y.Text, so a memoized document costs about
+ * 42% more than it did — a fraction of an already-large per-document
+ * retention, not a new order of magnitude. It is charged only for documents
+ * with observers attached and is released on detach.
+ *
+ * Deliberately NOT bounded by body size. The saving scales with document size,
+ * so a size cap would decline exactly the documents where a drain costs the
+ * most — paying full latency to save the memory on the worst case is the wrong
+ * trade. If a bound is ever wanted, the memo already degrades to a miss, so
+ * declining to store is a one-line change that costs only the parse it was
+ * saving.
+ */
+export interface EditorMdastMemo {
+ /**
+ * The single entry, or `null` when empty. Body and tree live together in one
+ * nullable rather than as two independent ones so that "half-populated" —
+ * a body with no tree, or a tree with no body to key it — cannot be
+ * constructed. The invariant is both-or-neither, so it is spelled that way.
+ */
+ entry: { readonly body: string; readonly tree: Root } | null;
+}
+
+/** Fresh, empty memo. One per document. */
+export function createEditorMdastMemo(): EditorMdastMemo {
+ return { entry: null };
+}
+
+function parseEditorMdastMemoized(
+ mdManager: MarkdownManager,
+ body: string,
+ memo: EditorMdastMemo | undefined,
+): Root {
+ if (memo?.entry?.body === body) return memo.entry.tree;
+ const tree = mdManager.parseToEditorMdast(body);
+ if (memo !== undefined) memo.entry = { body, tree };
+ return tree;
+}
+
export function computeMapDrivenBodySplice(
oldBody: string,
newPmJson: JSONContent,
mdManager: MarkdownManager,
onFallback?: (reason: 'parse-error' | 'missing-position', err?: unknown) => void,
+ memo?: EditorMdastMemo,
): MapDrivenSplice | null {
let oldChildren: readonly RootContent[];
let newBody: string;
@@ -61,9 +186,12 @@ export function computeMapDrivenBodySplice(
// parse, so the sibling walk below would see two identical block lists
// and splice a byte range that excludes the very change it is meant to
// carry.
- oldChildren = mdManager.parseToEditorMdast(oldBody).children;
+ // Memo order matters: `oldBody` is looked up FIRST because it is the
+ // side that repeats across drains, then `newBody` overwrites the single
+ // slot so the NEXT drain's `oldBody` lookup hits.
+ oldChildren = parseEditorMdastMemoized(mdManager, oldBody, memo).children;
newBody = mdManager.serialize(newPmJson);
- newChildren = mdManager.parseToEditorMdast(newBody).children;
+ newChildren = parseEditorMdastMemoized(mdManager, newBody, memo).children;
} catch (err) {
// Swallowing is the contract (caller falls back to the whole-body diff
// path), but the swallow must not be silent: a systemic parse/serialize
diff --git a/packages/server/src/parse-counting.test-helper.ts b/packages/server/src/parse-counting.test-helper.ts
new file mode 100644
index 000000000..f8d6cbcf0
--- /dev/null
+++ b/packages/server/src/parse-counting.test-helper.ts
@@ -0,0 +1,29 @@
+/**
+ * Shared rig for the map-driven splice parse-economy tests.
+ *
+ * Wraps a `MarkdownManager` so every `parseToEditorMdast` call is counted.
+ * `computeMapDrivenBodySplice` is the only caller of that method on the
+ * Observer A drain path, so the count IS the drain's full-document parse
+ * count — which is what the economy pins assert. Counts, not milliseconds:
+ * a parse count is identical on a quiet laptop and a saturated CI runner,
+ * so these tests cannot go flaky under load the way a wall-clock budget can.
+ */
+import { MarkdownManager, sharedExtensions } from '@inkeep/open-knowledge-core';
+
+export interface CountingManager {
+ /** The instrumented manager. Pass this wherever a `MarkdownManager` is wanted. */
+ readonly manager: MarkdownManager;
+ /** Full-document `parseToEditorMdast` calls so far. */
+ readonly parses: () => number;
+}
+
+export function createCountingManager(): CountingManager {
+ const manager = new MarkdownManager({ extensions: sharedExtensions });
+ let calls = 0;
+ const original = manager.parseToEditorMdast.bind(manager);
+ manager.parseToEditorMdast = (markdown: string) => {
+ calls += 1;
+ return original(markdown);
+ };
+ return { manager, parses: () => calls };
+}
diff --git a/packages/server/src/server-observers.ts b/packages/server/src/server-observers.ts
index 6bd60bca2..fc5340cab 100644
--- a/packages/server/src/server-observers.ts
+++ b/packages/server/src/server-observers.ts
@@ -75,7 +75,11 @@ import {
LOSS_EVENT_GUARD_DEFER,
type LossCaptureRing,
} from './loss-capture.ts';
-import { computeMapDrivenBodySplice } from './map-driven-splice.ts';
+import {
+ computeMapDrivenBodySplice,
+ createEditorMdastMemo,
+ type EditorMdastMemo,
+} from './map-driven-splice.ts';
import {
incrementBridgeMergeCheckpointCreated,
incrementBridgeMergeContentGrowth,
@@ -329,6 +333,12 @@ interface TryComputeMapDrivenSpliceArgs {
readonly json: unknown;
readonly mdManager: MarkdownManager;
readonly docName: string | undefined;
+ /**
+ * This document's parse memo. Per-document so concurrent editing of two
+ * docs cannot evict each other, and so the retained tree dies with the
+ * observers rather than living as module state.
+ */
+ readonly mdastMemo: EditorMdastMemo;
}
// Warn-once: the parse-error fallback metric is a bounded-cardinality counter
@@ -355,7 +365,7 @@ function warnOnceMapDrivenParseError(docName: string | undefined, err: unknown):
function tryComputeMapDrivenSplice(
args: TryComputeMapDrivenSpliceArgs,
): YTextMapDrivenSplice | null {
- const { currentText, lastSyncedXmlMd, json, mdManager, docName } = args;
+ const { currentText, lastSyncedXmlMd, json, mdManager, docName, mdastMemo } = args;
if (currentText !== lastSyncedXmlMd) {
incrementMapDrivenSpliceFallback('text-mismatch');
return null;
@@ -375,6 +385,7 @@ function tryComputeMapDrivenSplice(
incrementMapDrivenSpliceFallback(reason);
if (reason === 'parse-error') warnOnceMapDrivenParseError(docName, err);
},
+ mdastMemo,
);
if (!splice) return null;
@@ -1325,6 +1336,14 @@ export function setupServerObservers(opts: SetupServerObserversOpts): () => void
// change on any residual-bearing doc to Path B.
let lastSyncedCanonicalMd = '';
let lastSyncedYTextBytes = '';
+ // This document's splice parse memo. Declared alongside the
+ // witnesses because it has the same lifecycle — created when the observers
+ // attach, dropped when they detach — and the same per-document scope. It is
+ // a pure latency optimisation keyed on body BYTES, so it can never change
+ // which splice is computed: a body it has not seen is a miss and a fresh
+ // parse. Holding it here rather than in module state keeps two concurrently
+ // edited documents from evicting each other's entry.
+ const mdastMemo = createEditorMdastMemo();
// Coherence flag — true iff BOTH witnesses were recorded together at a
// real settlement. The router's witness-vs-witness residual-tolerance
// comparison is only meaningful within one settlement generation:
@@ -2103,15 +2122,21 @@ export function setupServerObservers(opts: SetupServerObserversOpts): () => void
json,
mdManager,
docName: opts.docName,
+ mdastMemo,
});
if (mapDrivenSplice) {
setActiveSpanAttributes({
'observer.a.path': 'map-driven-splice',
- // The splice's three full-document passes are the documented
+ // The splice's full-document passes are the documented
// unbounded-by-doc-size cost on this path (map-driven-splice.ts
// perf envelope) — stamped so a large-doc drain-latency trace
- // answers "where did the time go" without reproduction. Integer ms
- // keeps the attribute bounded-cardinality.
+ // answers "where did the time go" without reproduction. Steady
+ // state here is TWO passes (serialize + one parse): this call site
+ // passes a memo, and during a typing burst the `oldBody` parse hits
+ // it. A drain that shows the third pass took a memo miss — a body
+ // the previous drain did not produce, i.e. an external write, a
+ // resync, or the first drain after attach. Integer ms keeps the
+ // attribute bounded-cardinality.
'observer.a.splice.compute_ms': Math.round(performance.now() - spliceComputeStart),
});
}