Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript
31.7 KB
Raw Blame
/**
 * Write-time neutralizer for structure-bearing markdown (lane A,
 * trust-and-responsiveness-hardening, t_c2ac4fc8).
 *
 * Third-party text written into the vault can plant structure the Brain
 * then treats as its own: a `[[wikilink]]` joins the link graph, an
 * `@osb` line joins the marker scanners, a ```osb fence plants a block
 * marker, and an ATX heading plants a semantic section the section
 * parsers read back. The read-time payload neutralizer
 * (`untrusted-source.ts`) and the ingest trust lane defend reads; nothing
 * defended writes. This module is the write-time leg: it escapes the
 * sigils of planted structure so the text keeps reading verbatim while
 * every line-grammar detector in src/core stops matching.
 *
 * ## Doctrine
 *
 *   - **Structure only, never vocabulary.** Detection keys on grammar a
 *     parser consumes, one family per consumer grammar, and on no word
 *     list: wikilinks (`wikilink.ts` regex family), `@osb` inline
 *     markers (the `inline.ts` parser itself, so "would a scanner parse
 *     this" has exactly one answer), ```osb fence lines (the
 *     `discoverMarkers` fence walk) and ATX headings (the CommonMark
 *     opening shape, which covers the semantic headings and the log
 *     heading). Task checkboxes, markdown links, reference definitions
 *     and tags are deliberately out of the set.
 *   - **Escape, never delete, never invisible characters.** One uniform
 *     mechanism: a backslash before each sigil of a detected span
 *     (brackets in wikilinks, `@` on marker lines, backticks on fence
 *     lines, the `#` run on headings). CommonMark renders a backslash
 *     before ASCII punctuation as the literal character, so the text
 *     stays readable and the bytes stay honest; no HTML entities, no
 *     zero-width anything.
 *   - **Fence-aware, per consumer grammar.** Every family is masked by the
 *     fence model the readers of that family actually run, because the
 *     grammars disagree about what a fence even is: the marker scanner
 *     (`inline.ts`) knows exactly-3-backtick lines and nothing else, the
 *     section readers (`source-links.ts` `fencedLineMask`, shared with the
 *     block resolver) know 3-or-more backtick or tilde runs, the search
 *     chunker (`chunker.ts` `readCodeBlock`) closes any fence on the first
 *     line starting with the same three-character mark, and the link
 *     readers share {@link maskCodeRegions}. Masking an `@osb` row with the
 *     one model and scanning it with the other is how a planted marker
 *     inside a tilde fence would slip through: detection answers
 *     "would MY consumer parse this" for each family separately.
 *   - **Escape-aware detectors.** A sigil preceded by an odd backslash
 *     run is data the caller already escaped, not structure; it is never
 *     re-escaped. (Line-anchored families get this for free: `\@osb` and
 *     `\#\#` fail the anchor.)
 *   - **Line breaks normalized first.** The enforce-mode walk folds CRLF
 *     to `\n` and a lone CR, NEL (U+0085), U+2028 and U+2029 to `\n`
 *     before detection, and records the rewrite as the `line-break` kind:
 *     CommonMark, Obsidian and `/m`-regex readers end lines on those
 *     characters while the `\n`-splitting detectors do not, so an
 *     unfolded `x\r# Planted` renders a heading no family would see.
 *   - **Idempotent by construction.** Escaping a fence line dissolves the
 *     fence, which can expose previously masked interior text - so
 *     {@link neutralizeMarkdown} re-detects its own output until the
 *     detector finds nothing left (bounded by
 *     {@link NEUTRALIZE_FIXPOINT_STEPS}). The walk terminates because
 *     each round only inserts backslashes before sigils that lacked an
 *     odd run - a backslashed sigil fails the same detection on the next
 *     pass (the odd-run rule, and the line anchors), so no sigil is ever
 *     escaped twice, and escaping cannot CREATE a sigil: a backslash
 *     before a sigil never starts a line the way `@osb`, `#` or ` ``` `
 *     must. A round can only ever EXPOSE sigils that were already in the
 *     text (a dissolved fence's interior), so the walk visits each sigil
 *     at most once and the bound is a safety net, never reached. The exit
 *     condition IS the doctrine: the output no longer matches the
 *     detector, therefore `neutralize(neutralize(x))` is byte-identical
 *     to `neutralize(x)`. Should the bound ever run out with live
 *     structure still detected - an input the walk cannot converge on -
 *     the walk fails CLOSED with a {@link MarkdownNeutralizeFixpointError}
 *     instead of returning those bytes as if they were neutralized: the
 *     bound guards against the impossible by refusing it, not by
 *     shipping it.
 *
 * ## One detection grammar, two consumers
 *
 * {@link detectMarkdownStructure} is the shared layer: the writer
 * ({@link neutralizeMarkdown}) escapes the spans it reports, and the
 * doctor's planted-rows planner counts them - so the finding, its count
 * and what enforce mode would change cannot drift apart.
 *
 * The gate ({@link resolveMarkdownNeutralizeMode}) is pure and defaults
 * to `off`, where the seams do not call this module at all and every
 * write stays byte-identical to before it existed.
 */

import {
  isLiveWikilinkSpan,
  maskCodeRegions,
  precededByOddBackslashRun,
  WIKILINK_TARGET_RE,
} from "../wikilink.ts";
import { parseInlineMarker } from "../inline.ts";
import { fencedLineMask } from "../source-links.ts";
import { envOrConfig } from "../../validate.ts";
import type { FrontmatterMap } from "../../types.ts";
import { discoverConfig } from "../../config.ts";

// ----- The shared detection grammar -----------------------------------------

/** The structure families the neutralizer escapes and the auditor counts,
 * plus the line-break token: the enforce-mode walk also folds unusual line
 * separators to `\n` before detection and records that rewrite here, so the
 * `neutralized_markers` provenance names every way the output departed from
 * the input. The auditor counts only the structure families - the
 * line-break token never appears in {@link detectMarkdownStructure} output. */
export const MARKDOWN_STRUCTURE_KIND = Object.freeze({
  wikilink: "wikilink",
  osbInline: "osb-inline",
  osbFence: "osb-fence",
  heading: "atx-heading",
  lineBreak: "line-break",
} as const);

/** One {@link MARKDOWN_STRUCTURE_KIND} token. */
export type MarkdownStructureKind =
  (typeof MARKDOWN_STRUCTURE_KIND)[keyof typeof MARKDOWN_STRUCTURE_KIND];

/** One planted structure row: a span of `text` whose sigils are live. */
export interface MarkdownStructureMatch {
  readonly kind: MarkdownStructureKind;
  /** Inclusive start offset of the sigil span, in code units of `text`. */
  readonly start: number;
  /** Exclusive end offset of the sigil span. */
  readonly end: number;
}

/**
 * The ```osb fence-open shape `discoverMarkers` walks: exactly three backticks,
 * a plain info string, end of line. The marker grammar never learned
 * about longer runs or tilde fences, and this module does not teach it -
 * "would a marker scanner consume this" is the question the osb-fence
 * family answers.
 */
const OSB_FENCE_OPEN_RE = /^```([A-Za-z0-9_-]*)\s*$/;

/** The CommonMark ATX opening shape, widened to the union of consumer grammars: any
 * indentation (the section readers and the log reader disagree about how much a
 * heading may carry, so the detector under-cuts none of them), a `#` run of 1-6,
 * then a space, the CR of a CRLF body, or the line end. */
const ATX_HEADING_RE = /^[ \t]*(#{1,6})(?:[ \t\r]|$)/;

/** The fence marker both grammars are built from. */
const BACKTICK = "`";

/** One physical line and its start offset in the scanned text. */
interface Line {
  readonly text: string;
  readonly start: number;
}

function splitLines(text: string): Line[] {
  const out: Line[] = [];
  let start = 0;
  for (;;) {
    const nl = text.indexOf("\n", start);
    if (nl === -1) {
      out.push({ text: text.slice(start), start });
      return out;
    }
    out.push({ text: text.slice(start, nl), start });
    start = nl + 1;
  }
}

/** Length of the backtick run starting at `s[0]` (`s` is a pre-trimmed line head). */
function backtickRun(s: string): number {
  let n = 0;
  while (n < s.length && s[n] === BACKTICK) n++;
  return n;
}

/**
 * The ```osb fence lines plus the body mask, found by the same walk
 * `discoverMarkers` runs: an open fence line per {@link OSB_FENCE_OPEN_RE}
 * consumes lines until a line whose trimmed form starts with three backticks
 * (or the end of text), and an `osb`-info block contributes its open and
 * close fence lines. This is the marker grammar's ENTIRE fence concept -
 * exactly-3-backtick boundaries and nothing else: a tilde fence or a
 * 4-backtick fence fences nothing for the scanner, so the rows inside are
 * live and the mask must not cover them, while an unterminated fence
 * consumes the rest of the text and the mask must cover that.
 */
interface MarkerFenceStructure {
  /** The osb fence boundary-line sigil spans, in document order. */
  readonly spans: MarkdownStructureMatch[];
  /** True for body lines the marker scanner consumes as fenced (not parsed). */
  readonly bodyMasked: boolean[];
}

function markerFenceStructure(lines: ReadonlyArray<Line>): MarkerFenceStructure {
  const spans: MarkdownStructureMatch[] = [];
  const bodyMasked = Array.from({ length: lines.length }, () => false);
  let i = 0;
  while (i < lines.length) {
    const line = lines[i]!;
    const open = OSB_FENCE_OPEN_RE.exec(line.text.trim());
    if (open === null) {
      i++;
      continue;
    }
    const isOsb = (open[1] ?? "") === "osb";
    if (isOsb) {
      const indent = line.text.length - line.text.trimStart().length;
      spans.push({ kind: "osb-fence", start: line.start + indent, end: line.start + indent + 3 });
    }
    i++;
    while (i < lines.length) {
      const inner = lines[i]!;
      if (inner.text.trim().startsWith("```")) {
        if (isOsb) {
          const indent = inner.text.length - inner.text.trimStart().length;
          const run = backtickRun(inner.text.trimStart());
          spans.push({
            kind: "osb-fence",
            start: inner.start + indent,
            end: inner.start + indent + run,
          });
        }
        i++;
        break;
      }
      bodyMasked[i] = true;
      i++;
    }
  }
  return { spans, bodyMasked };
}

/** The sigil span of a line's leading `@` when the line parses as a marker. */
function inlineMarkerMatch(line: Line, lineNo: number): MarkdownStructureMatch | null {
  if (parseInlineMarker(line.text, lineNo + 1) === null) return null;
  const at = line.text.length - line.text.trimStart().length;
  return { kind: "osb-inline", start: line.start + at, end: line.start + at + 1 };
}

/**
 * The search chunker's code-block walk (`chunker.ts` `FENCE_RE` +
 * `readCodeBlock`), mirrored line for line: a block opens on a line whose
 * RAW form starts with three backticks or tildes (the chunker tests the
 * untrimmed line) and closes on the first later line whose TRIMMED form
 * starts with the same three-character mark - info string or not - and an
 * unclosed block runs to end of text. This is deliberately NOT
 * {@link fencedLineMask}'s rule: the chunker is pinned by
 * `CHUNKER_VERSION` (changing how it reads a page forces a full vault
 * re-chunk), so the detector carries the chunker's shape as its own
 * second heading mask rather than the chunker carrying the section
 * readers'.
 */
const CHUNKER_FENCE_OPEN_RE = /^(?:```|~~~)/;

function chunkerCodeBlockMask(lines: ReadonlyArray<Line>): boolean[] {
  const mask = Array.from({ length: lines.length }, () => false);
  let i = 0;
  while (i < lines.length) {
    if (!CHUNKER_FENCE_OPEN_RE.test(lines[i]!.text)) {
      i++;
      continue;
    }
    const mark = lines[i]!.text.trim().startsWith("```") ? "```" : "~~~";
    mask[i] = true;
    i++;
    while (i < lines.length) {
      mask[i] = true;
      const closed = lines[i]!.text.trim().startsWith(mark);
      i++;
      if (closed) break;
    }
  }
  return mask;
}

/**
 * Every planted structure row in `text`, left to right. Each family is
 * masked by the fence model its own readers run, because "is this row
 * inert?" has one answer per consumer grammar and they do not agree:
 *
 *   - the osb-inline and osb-fence families answer to the marker scanner
 *     (`inline.ts`): its exactly-3-backtick boundary walk is the only
 *     fencing that exists there, so a planted marker inside a tilde fence
 *     or a 4-backtick fence is live and gets reported;
 *   - the atx-heading family answers to TWO fence grammars: the section
 *     readers (`source-links.ts` `fencedLineMask`, shared with the block
 *     resolver; a 3-or-more backtick or tilde fence closes only on a run
 *     at least as long carrying nothing but whitespace, and an
 *     unterminated fence fences nothing) and the search chunker
 *     (`chunker.ts` `readCodeBlock`, which closes ANY fence on the first
 *     later line whose trimmed form starts with the same three-character
 *     mark, info string or not). A heading is escaped when EITHER grammar
 *     leaves it live - the chunker's shorter-close rule exposes a heading
 *     after a ``` line inside a ```` fence, which masking with
 *     `fencedLineMask` alone left unescaped and attributable as a chunk
 *     `headingPath`;
 *   - the wikilink family answers to the link readers (`vault.ts`
 *     `extractWikilinks`, page-lint) and runs the readers' own recipe:
 *     code regions (fences and inline spans) are masked out of the scan
 *     view first, then the multi-line-tolerant {@link WIKILINK_TARGET_RE}
 *     shape and {@link isLiveWikilinkSpan} run on that view, so a
 *     `[[...\n...]]` pair across a line break is detected exactly when
 *     they would extract it. The mask is length-preserving, so every
 *     match offset in the view is the same offset in the raw text and the
 *     brackets the readers can see are exactly the brackets this scan
 *     sees - a `[[target|`code`]]` span whose brackets sit outside the
 *     region is reported (and escaped) even though the raw span overlaps
 *     the region, which is the hole a raw-text scan with a region-overlap
 *     test left open: the readers still extracted the live target from
 *     their own masked view.
 *
 * Fence lines for the osb family are boundaries rather than masked
 * content. Pure and linear in the length of the text.
 */
export function detectMarkdownStructure(text: string): ReadonlyArray<MarkdownStructureMatch> {
  const lines = splitLines(text);
  const fence = markerFenceStructure(lines);
  const matches: MarkdownStructureMatch[] = [...fence.spans];
  // The wikilink family keys on the multi-line-tolerant shape its readers
  // run (`vault.ts` extractWikilinks, page-lint) over the same masked view
  // those readers match on. The fill preserves length, so match offsets
  // are raw-text offsets; the bracket-span match set itself is identical
  // under any space fill, because a code region contributes no bracket
  // characters in either view.
  const masked = maskCodeRegions(text, (region) => " ".repeat(region.length));
  for (const m of masked.matchAll(WIKILINK_TARGET_RE)) {
    const start = m.index;
    const end = start + m[0].length;
    if (!isLiveWikilinkSpan(masked, start, end)) continue;
    matches.push({ kind: "wikilink", start, end });
  }
  const headingMasked = fencedLineMask(lines.map((line) => line.text));
  const chunkerMasked = chunkerCodeBlockMask(lines);
  for (let i = 0; i < lines.length; i++) {
    const line = lines[i]!;
    // The heading escapes unless BOTH fence grammars mask the line: the
    // family's consumers disagree about where a fence ends (see the
    // family bullet above), and a line either consumer treats as
    // fence-open but the other does not is live for the latter.
    if (!headingMasked[i] || !chunkerMasked[i]) {
      const heading = ATX_HEADING_RE.exec(line.text);
      if (heading !== null) {
        const runStart = line.start + heading[0].indexOf("#");
        const start = runStart;
        const end = runStart + heading[1]!.length;
        matches.push({ kind: "atx-heading", start, end });
      }
    }
    if (fence.bodyMasked[i]) continue;
    const marker = inlineMarkerMatch(line, i);
    if (marker !== null) matches.push(marker);
  }
  return matches.toSorted((a, b) => a.start - b.start || a.end - b.end);
}

// ----- The neutralizer ------------------------------------------------------

/**
 * The line-separator normalization the enforce-mode walk runs before
 * detection: `\r\n` folds to `\n`, and a lone CR, NEL (U+0085), LINE
 * SEPARATOR (U+2028) or PARAGRAPH SEPARATOR (U+2029) becomes `\n`. These
 * are the characters CommonMark, Obsidian and every `/m`-regex line reader
 * treat as a line end while the `\n`-splitting detectors - OSB's own line
 * readers included - do not, so a planted row hiding behind one (`x\r#
 * Planted` renders a heading) would otherwise pass detection untouched.
 * One pass, and the output carries none of the separators, so the rewrite
 * is inherently idempotent.
 */
const LINE_BREAK_NORMALIZE_RE = /\r\n?|[\u0085\u2028\u2029]/g;

function normalizeLineBreaks(text: string): { text: string; changed: boolean } {
  const normalized = text.replace(LINE_BREAK_NORMALIZE_RE, "\n");
  return normalized === text ? { text, changed: false } : { text: normalized, changed: true };
}

/**
 * How many detect-and-escape rounds {@link neutralizeMarkdown} may run
 * before it refuses: two rounds are the deep case (round one escapes an
 * ```osb fence's boundary lines, the escape dissolves the fence for the
 * line-grammar consumers, and round two escapes the interior the dissolve
 * exposed), and no round can create a sigil or escape one twice (see the
 * doctrine block above), so the walk converges and the bound is a safety
 * net honest input never reaches. The post-condition is asserted, not
 * assumed: a bound exhausted with the output's own detection still
 * non-empty is a {@link MarkdownNeutralizeFixpointError}, never a
 * `changed: true` carrying live structure.
 */
const NEUTRALIZE_FIXPOINT_STEPS = 8;

/**
 * Backslash-escape the minimal sigil characters of one detected span.
 * The heading, marker and fence spans are pure sigil runs (`#`s, the
 * `@`, backticks) so every character escapes; a wikilink span covers its
 * whole `[[target]]` body, of which only the brackets are sigils - the
 * target stays verbatim, and a bracket the caller already escaped (an odd
 * backslash run before it, e.g. the closing `]` of `[[a\]]`) is data the
 * same rule the span's opening pair was held to, and is left alone. Each
 * bracket's run is read against the span as detected, so the backslashes
 * this pass inserts never make a later bracket look escaped.
 */
function escapeSigils(span: string, kind: MarkdownStructureKind): string {
  if (kind !== MARKDOWN_STRUCTURE_KIND.wikilink) {
    return [...span].map((ch) => `\\${ch}`).join("");
  }
  let out = "";
  for (let i = 0; i < span.length; i++) {
    const ch = span[i]!;
    if ((ch === "[" || ch === "]") && !precededByOddBackslashRun(span, i)) out += `\\${ch}`;
    else out += ch;
  }
  return out;
}

function escapeRound(
  context: string,
  text: string,
): { text: string; kinds: ReadonlyArray<MarkdownStructureKind> } | null {
  // Detection runs over the exact bytes the page's scanners see before
  // `text`; only spans reaching past the context are this call's business
  // (see {@link NeutralizeMarkdownOptions.precedingContext}).
  const stitched = context + text;
  const matches = detectMarkdownStructure(stitched).filter((m) => m.end > context.length);
  if (matches.length === 0) return null;
  let out = "";
  let last = context.length;
  const kinds = new Set<MarkdownStructureKind>();
  for (const m of matches) {
    const start = Math.max(m.start, context.length);
    if (start < last) continue; // by construction disjoint; a guard, not a contract
    out += text.slice(last - context.length, start - context.length);
    out += escapeSigils(stitched.slice(start, m.end), m.kind);
    last = m.end;
    kinds.add(m.kind);
  }
  out += text.slice(last - context.length);
  return { text: out, kinds: [...kinds].toSorted() };
}

/** What {@link neutralizeMarkdown} returns. */
export interface NeutralizedMarkdown {
  /** The text with planted structure escaped; byte-identical input when `changed` is false. */
  readonly text: string;
  /** Whether any structure was found and escaped, or line separators were rewritten. */
  readonly changed: boolean;
  /**
   * The families that were neutralized plus the `line-break` token when
   * unusual line separators were folded, sorted, unique; empty when
   * unchanged.
   */
  readonly kinds: ReadonlyArray<MarkdownStructureKind>;
}

const UNCHANGED: NeutralizedMarkdown = Object.freeze({
  text: "",
  changed: false,
  kinds: Object.freeze([]),
});

/** Optional inputs of {@link neutralizeMarkdown}. */
export interface NeutralizeMarkdownOptions {
  /**
   * The bytes rendered before `text` on the same page, verbatim: an
   * already-neutralized caller half plus whatever renderer-authored
   * separator and heading the writer placed between the halves (the
   * capture seam passes the neutralized body, the blank-line separator
   * and its own `## Guidance` heading).
   *
   * The seam needs this because a rendered page is ONE text to every
   * scanner: a fence the preceding half opens and `text`'s own bytes close
   * (or leave open) has combined-page meaning that neither half's
   * standalone scan can see - the split-fence hole, where a planted marker
   * live on the combined page scanned clean in both halves. The three
   * fence models the families answer to also disagree about what a fence
   * even is, and the boundary state differs per model in ways a boolean
   * cannot carry (an unterminated fence masks the rest of the page for the
   * marker scanner while fencing nothing for the section readers unless a
   * later fence closes it). So no state is summarized: detection runs over
   * `precedingContext + text` - the exact bytes the page's scanners see
   * ahead of `text` - and only spans reaching past the context are
   * escaped, which makes the half's verdict agree with the combined page's
   * in both directions.
   *
   * Skipping the context's own rows is safe by construction: caller
   * context is the output of an earlier neutralize call, whose detection
   * is empty (the fixpoint exit), and renderer-authored rows are the
   * writer's, never this call's business. No span can straddle the
   * boundary when the context carries the writer heading - the wikilink
   * target grammar stops at its `#`, and every other family is
   * line-anchored - so each escaped span lies wholly in `text`; a custom
   * context that let one straddle would have only its `text`-side sigils
   * escaped and fail closed at the fixpoint bound rather than ship a
   * half-escaped span.
   */
  readonly precedingContext?: string;
}

/**
 * Escape the structure-bearing markdown in third-party text.
 *
 * Unusual line separators (a lone CR, NEL, U+2028, U+2029 - and CRLF) are
 * folded to `\n` first and the fold is recorded as the `line-break` kind:
 * the line-grammar consumers these families answer to disagree about what
 * ends a line, and the fold removes the disagreement before detection. The
 * gate puts every call under `enforce`, so the other two modes stay
 * byte-identical by never reaching this function.
 *
 * Returns the input unchanged - `changed: false`, identical bytes, zero
 * added bytes - when nothing was found to normalize or escape, which is
 * the shape of the overwhelming majority of caller text. When structure
 * was found, each detected span's sigils are backslash-escaped (see the
 * module docblock for the doctrine) and {@link kinds} names every family
 * that was neutralized (plus `line-break` when separators were folded),
 * for the seams' conditional provenance key.
 *
 * The walk is bounded by {@link NEUTRALIZE_FIXPOINT_STEPS} and FAILS
 * CLOSED: a bound exhausted with live structure still detected throws
 * {@link MarkdownNeutralizeFixpointError} rather than returning the exact
 * bytes this module exists to neutralize (see
 * {@link neutralizeMarkdownWithBound}).
 */
export function neutralizeMarkdown(
  text: string,
  options: NeutralizeMarkdownOptions = {},
): NeutralizedMarkdown {
  return neutralizeMarkdownWithBound(text, NEUTRALIZE_FIXPOINT_STEPS, options);
}

/**
 * {@link neutralizeMarkdown} with the fixpoint bound made explicit -
 * `maxRounds` is the highest step index the detect-and-escape loop may
 * reach, so `0` still runs the one round that decides "nothing to do"
 * versus "escape once". Exported as the seam that lets tests force the
 * bound low (the production bound is a safety net that honest input
 * cannot reach); callers in the vault lanes use
 * {@link neutralizeMarkdown} and its settled bound.
 *
 * The post-condition is the doctrine's exit condition: the returned text
 * no longer matches {@link detectMarkdownStructure} on the spans that
 * reach past {@link NeutralizeMarkdownOptions.precedingContext} (the
 * context's own rows are skipped by construction, see there). If the
 * bound runs out with detection still finding such spans, the walk did
 * not converge and returning `changed: true` would ship live structure as
 * if it were neutralized - so the refusal is thrown, by name
 * ({@link MarkdownNeutralizeFixpointError}), and the caller's write dies
 * before any bytes are persisted.
 */
export function neutralizeMarkdownWithBound(
  text: string,
  maxRounds: number,
  options: NeutralizeMarkdownOptions = {},
): NeutralizedMarkdown {
  const context = options.precedingContext ?? "";
  // Unusual line separators fold to `\n` before any detection: the
  // line-grammar consumers the families answer to disagree about what ends
  // a line, and the fold puts `text` on the `\n`-only view every detector
  // here scans. The context is prior neutralizer output (see
  // {@link NeutralizeMarkdownOptions.precedingContext}), so it is already
  // folded and needs no second pass.
  const folded = normalizeLineBreaks(text);
  let current = folded.text;
  let kinds = new Set<MarkdownStructureKind>();
  if (folded.changed) kinds.add(MARKDOWN_STRUCTURE_KIND.lineBreak);
  for (let step = 0; step <= maxRounds; step++) {
    const round = escapeRound(context, current);
    if (round === null) break;
    current = round.text;
    for (const kind of round.kinds) kinds.add(kind);
  }
  if (kinds.size === 0) {
    return { text, changed: false, kinds: UNCHANGED.kinds };
  }
  const remaining = detectMarkdownStructure(context + current).filter(
    (m) => m.end > context.length,
  );
  if (remaining.length > 0) {
    throw new MarkdownNeutralizeFixpointError(
      `markdown neutralization did not reach its fixpoint within ` +
        `${maxRounds + 1} detect-and-escape round(s): ` +
        `${remaining.length} planted structure span(s) would remain in the output`,
    );
  }
  return Object.freeze({
    text: current,
    changed: true,
    kinds: Object.freeze([...kinds].toSorted()),
  });
}

/**
 * Refusal raised when {@link neutralizeMarkdown}'s fixpoint bound ran out
 * with live structure still detected - the fail-closed posture for the
 * safety net honest input never reaches (see the doctrine block: the exit
 * condition IS an empty detection, so anything else is a refusal, never a
 * silently under-neutralized write).
 */
export class MarkdownNeutralizeFixpointError extends Error {
  constructor(message: string) {
    super(message);
    this.name = "MarkdownNeutralizeFixpointError";
  }
}

// ----- The provenance record ------------------------------------------------

/**
 * Frontmatter key recording the structure families an enforce-mode write
 * neutralized (and the `line-break` rewrite, when separators were folded).
 * A page carrying it says "these bytes are not what the caller sent" - the
 * audit trail for the one mode that edits content - and a write that
 * changed nothing adds nothing, keeping the untouched shape byte-identical
 * to before the gate existed.
 */
export const MARKDOWN_NEUTRALIZED_FRONTMATTER_KEY = "neutralized_markers";

/**
 * The frontmatter fragment a seam merges when neutralization changed
 * bytes: the key naming the neutralized kinds, sorted. An empty kind set
 * adds NOTHING - the `untrustedSourceFrontmatter` conditional-spread
 * precedent, so an unchanged body gains zero bytes.
 */
export function neutralizedMarkersFrontmatter(
  kinds: ReadonlyArray<MarkdownStructureKind>,
): FrontmatterMap {
  return kinds.length === 0 ? {} : { [MARKDOWN_NEUTRALIZED_FRONTMATTER_KEY]: [...kinds] };
}

// ----- The gate -------------------------------------------------------------

/**
 * The resolved posture of the write-time neutralization gate:
 *   - `off`     - the seams never invoke the neutralizer; every write
 *                 stays byte-identical to a build without this module,
 *   - `shadow`  - byte-identical to `off` at the seams. Shadow has no
 *                 work to do there: the doctor's planted-rows audit
 *                 recomputes the would-be rows from the vault's bytes on
 *                 its own and reads this mode only for its wording, so a
 *                 seam call under shadow would compute an answer nobody
 *                 consumes,
 *   - `enforce` - the seams write neutralized bytes.
 */
export const MARKDOWN_NEUTRALIZE_MODE = Object.freeze({
  off: "off",
  shadow: "shadow",
  enforce: "enforce",
} as const);

export type MarkdownNeutralizeMode =
  (typeof MARKDOWN_NEUTRALIZE_MODE)[keyof typeof MARKDOWN_NEUTRALIZE_MODE];

/** Environment variable for the markdown neutralization gate mode. */
export const MARKDOWN_NEUTRALIZE_MODE_ENV = "OPEN_SECOND_BRAIN_MARKDOWN_NEUTRALIZE_MODE";

/** Machine-config key for the markdown neutralization gate mode. */
export const MARKDOWN_NEUTRALIZE_MODE_CONFIG = "markdown_neutralize_mode";

/** True when `value` is one of the three mode tokens. */
export function isMarkdownNeutralizeMode(value: string): value is MarkdownNeutralizeMode {
  return (Object.values(MARKDOWN_NEUTRALIZE_MODE) as string[]).includes(value);
}

/** Refusal raised for a gate mode value that is not in the vocabulary. */
export class MarkdownNeutralizeModeError extends Error {
  constructor(message: string) {
    super(message);
    this.name = "MarkdownNeutralizeModeError";
  }
}

/**
 * Resolve the gate's mode: an explicit mode key wins (env over machine
 * config; an invalid value is a misconfiguration refused with a
 * {@link MarkdownNeutralizeModeError} naming the key actually in force -
 * never silently defaulted); with neither set the default is `off`, the
 * posture every write had before the gate existed. The
 * `resolveSearchTrustGateMode` shape minus the legacy boolean: this gate
 * never had one. Reads only its arguments.
 */
export function resolveMarkdownNeutralizeMode(input: {
  readonly env?: NodeJS.ProcessEnv;
  readonly config?: Readonly<Record<string, string>>;
}): MarkdownNeutralizeMode {
  const env = input.env ?? {};
  const config = input.config ?? {};
  const raw = envOrConfig(
    env,
    config,
    MARKDOWN_NEUTRALIZE_MODE_ENV,
    MARKDOWN_NEUTRALIZE_MODE_CONFIG,
  );
  if (raw !== null) {
    if (isMarkdownNeutralizeMode(raw)) return raw;
    const inForce = env[MARKDOWN_NEUTRALIZE_MODE_ENV]
      ? MARKDOWN_NEUTRALIZE_MODE_ENV
      : MARKDOWN_NEUTRALIZE_MODE_CONFIG;
    throw new MarkdownNeutralizeModeError(
      `${inForce} must be 'off', 'shadow' or 'enforce', got '${raw}'`,
    );
  }
  return MARKDOWN_NEUTRALIZE_MODE.off;
}

/**
 * Resolve the gate's mode the way the write seams do: the live process
 * env over the machine config at its default discovery path. The one
 * impure wrapper, so the capture and ingest seams cannot drift about
 * where the inputs come from; tests drive the pure
 * {@link resolveMarkdownNeutralizeMode} instead.
 */
export function resolveMarkdownNeutralizeModeForProcess(): MarkdownNeutralizeMode {
  return resolveMarkdownNeutralizeMode({ env: process.env, config: discoverConfig().data });
}