Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

docs(key-files): keep graph-and-facts under its size cap after the #6983 entry (#6983)

garrytan9782aa881c1f
diff --git a/docs/architecture/key-files/graph-and-facts.md b/docs/architecture/key-files/graph-and-facts.md
index 57f9a7f49d..3648d5adf3 100644
--- a/docs/architecture/key-files/graph-and-facts.md
+++ b/docs/architecture/key-files/graph-and-facts.md
@@ -103,7 +103,7 @@ rejection of serve-only flags before a brain opens.
- `src/core/fence-shared.ts` — shared pipe-table primitives for the `## Takes` (`takes-fence.ts`) and `## Facts` (`facts-fence.ts`) fences: `parseRowCells`, `isSeparatorRow`, `stripStrikethrough`, `parseStringCell`, `escapeFenceCell`, and the rewrite guard (#6385 R12) `assertFenceRewritable` / `FenceRewriteRefusal` that `upsertFactRow`, `upsertTakeRow`, `supersedeRow` and `mergePhantomFenceRows` throw when their input fence parses with warnings (the message names the fence only; the warnings, which embed row text, stay on the error). `parseRowCells` is escape-aware: `\|` stays inside its cell and decodes back to a literal `|` (exact inverse of `escapeFenceCell`), while any other backslash passes through verbatim so existing fence text (e.g. Windows paths) is byte-stable across a render → parse cycle. This matters because the fence is canonical and reconcile is wipe-and-reinsert — a pipe inside a cell shifting the column layout would corrupt every row behind it on the next reconcile. `escapeFenceCell` also encodes fence-marker text (`gbrain:facts:begin` and the three siblings) by lengthening the ampersand run before `#58;` (`gbrain:X` → `gbrain:X`, a literal `gbrain:X` → `gbrain:X`), and `decodeFenceCell` (the decoder `parseRowCells` and `fence-repair/raw-rows.ts` share) removes exactly one level, so a claim that quotes a marker never becomes a live marker and a literal entity round-trips byte-identically; the lexical fail-closed privacy rule is unchanged. Pinned by the escape cases in `test/facts-fence.test.ts` + the full render → parse → reconcile round-trip in `test/e2e/facts-fence-reconcile-postgres.test.ts`.
-- `src/core/link-extraction.ts` — shared library for the graph layer. `extractEntityRefs` (canonical) matches `[Name](../people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks; `extractPageLinks`, `resolveMarkdownLinkSlug` (one markdown href resolved from a page exactly as `extractPageLinks` resolves it; the writer's link validator, `src/core/output/validators/link.ts`, uses it so its dangling-link verdict matches the graph; a `.md` dir link with no `/`/`./`/`../` prefix first tries `pageRelativeMarkdownSlugs`, the page's directory then its ancestors like the FS walker's `resolveSlug`, and the first existing page wins, #6983), `inferLinkType` heuristics (attended/works_at/invested_in/founded/advises/source/mentions), `parseTimelineEntries`, `isAutoLinkEnabled`. In `inferLinkType` a per-edge verb counts only when it belongs to the link being typed (`attachedVerb`: a match with another link between it and this one, or written immediately before another link, is that link's), then precedence founded > invested_in > advises > works_at decides; `joined [X] as <role>` reads as works_at. `attendanceEvidenceRanges` accepts an `Attendees:`/`Participants:` line (plain or bold) and a `## Attendees`/`## Participants` link-list section (`Participants` since gbrain-evals N12-6, with a `LINK_EXTRACTOR_VERSION_TS` bump so meeting pages re-extract). Markdown links, bare-slug prose refs, and slash-shaped wikilinks match ANY dir-shaped path (`ANY_DIR_SEGMENT`), not a directory whitelist — nonexistent targets are dropped by the persist paths' page-existence checks (`resolveCandidateSources`, put_page's allSlugs filter, `addLinksBatch` INNER JOINs) and counted as `skippedMissingTarget` in the extract summaries; the `DIR_PATTERN` whitelist is only the typed fast-path for pass-2b wikilinks (non-whitelisted `[[dir/...]]` get an equivalent direct typed candidate in pass 2c, plus the flag-gated suffix rescue for non-exact matches). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. **Opt-in global-basename wikilink resolution** (default off): `WIKILINK_GENERIC_RE` catches bare `[[name]]` wikilinks outside `DIR_PATTERN` (third pass `2c` in `extractEntityRefs`); `EntityRef.needsResolution: true` tags refs from this pass (the ref's `slug` is the wikilink TARGET, `name` the optional display alias). `SlugResolver` gains optional `resolveBasenameMatches(name): Promise<string[]>` (multi-match by design — emits one edge per matching page). The single shared basename matcher is `buildBasenameIndex(slugs)` + `queryBasenameIndex(index, name)` + `normalizeBasename` (keys raw/lower/slugified tail, stable-sorted shorter-first then lexical), used by `makeResolver`, the FS `resolveBasenameMatchesFromSlugs`, AND the doctor check so they cannot drift. `makeResolver` step 1 and the FS resolver try `exactSlugCandidates` (`exact-slug-candidates.ts`: the value, then `slugifyPath`), then `resolveSlugWithAlias` (rename aliases); its fuzzy title step and live keyword step (source-scoped) accept a person/company/fund/organization candidate only under `sameEntityName`, so a near-name stays unresolved instead of linking to a different entity (`test/entity-resolution-near-names.test.ts`). With `link_resolution.global_basename` on, `extractFrontmatterLinks` passes `resolve(name, dirHint, { globalBasename, selfSlug })`, and a unique basename match (self excluded) resolves right after the exact steps and before the fuzzy and live keyword steps, the exact-name-first precedence of `resolveEntitySlug` (#5749, `test/link-resolution-basename-before-search.test.ts`). `makeResolver(engine, {mode, sourceId})` builds the index lazily via `engine.getAllSlugs({sourceId})` — source-scoped so a bare `[[name]]` never resolves to a same-tail page in a different source. `extractPageLinks` gains `opts.globalBasename` (routes `needsResolution` refs through `resolveBasenameMatches` keyed on `ref.slug`, emits candidates tagged `linkType: 'wikilink_basename'` + `linkSource: 'wikilink-resolved'`, skips self-loops) and `opts.skipFrontmatter`. All three surfaces (FS extract, DB extract, `put_page` auto-link) tag provenance with `link_source='wikilink-resolved'`; `put_page` includes it in its reconcilable-edge set so stale basename edges are removed when the wikilink or the flag goes away. Exports `WIKILINK_BASENAME_LINK_TYPE` + `isGlobalBasenameEnabled(engine)` (resolution order: env `GBRAIN_LINK_RESOLUTION_GLOBAL_BASENAME` → DB config `link_resolution.global_basename` → default false). `gbrain doctor`'s `link_resolution_opportunity` check surfaces a paste-ready enable hint when ≥5 bare wikilinks would resolve AND ≥20% match. Migration v113 widens `links_link_source_check` to admit `'wikilink-resolved'`; v114 then opens it to any kebab-case provenance (`^[a-z][a-z0-9]*(-[a-z0-9]+)*$`, ≤64 chars) so external derivers register their own tag (e.g. `citation-graph`) without a migration. `LINK_EXTRACTOR_VERSION_TS` also lives here (bump like `CHUNKER_VERSION` to invalidate prior extract-stale stamps). Pinned by `test/link-extraction.test.ts`, `test/extract-fs.test.ts`, `test/doctor.test.ts`, `test/e2e/global-basename-pglite.test.ts`. #6191: `inferLinkType` applies `link-target-roles.ts` `targetTakesVerb` to every per-edge and fallback verb: `works_at` never types a link to a person, meeting, calendar-event, media, image or transcript target (unknown type: a `people/`, `meetings/`, `calendar/`, `cal/` slug), `founded`/`invested_in` never a temporal or media one; a refused verb keeps scanning lower-precedence rules, then `mentions` (`test/link-type-target-guard.test.ts`).
+- `src/core/link-extraction.ts` — shared library for the graph layer. `extractEntityRefs` (canonical) matches `[Name](../people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks; `extractPageLinks`, `resolveMarkdownLinkSlug` (an href resolved as `extractPageLinks` does, after `pageRelativeMarkdownSlugs` for a prefix-less `.md` dir link, page dir then ancestors like FS `resolveSlug`, #6983; `output/validators/link.ts` uses both), `inferLinkType` heuristics (attended/works_at/invested_in/founded/advises/source/mentions), `parseTimelineEntries`, `isAutoLinkEnabled`. In `inferLinkType` a per-edge verb counts only when it belongs to the link being typed (`attachedVerb`: a match with another link between it and this one, or written immediately before another link, is that link's), then precedence founded > invested_in > advises > works_at decides; `joined [X] as <role>` reads as works_at. `attendanceEvidenceRanges` accepts an `Attendees:`/`Participants:` line (plain or bold) and a `## Attendees`/`## Participants` link-list section (`Participants` since gbrain-evals N12-6, with a `LINK_EXTRACTOR_VERSION_TS` bump so meeting pages re-extract). Markdown links, bare-slug prose refs, and slash-shaped wikilinks match ANY dir-shaped path (`ANY_DIR_SEGMENT`), not a directory whitelist — nonexistent targets are dropped by the persist paths' page-existence checks (`resolveCandidateSources`, put_page's allSlugs filter, `addLinksBatch` INNER JOINs) and counted as `skippedMissingTarget` in the extract summaries; the `DIR_PATTERN` whitelist is only the typed fast-path for pass-2b wikilinks (non-whitelisted `[[dir/...]]` get an equivalent direct typed candidate in pass 2c, plus the flag-gated suffix rescue for non-exact matches). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. **Opt-in global-basename wikilink resolution** (default off): `WIKILINK_GENERIC_RE` catches bare `[[name]]` wikilinks outside `DIR_PATTERN` (third pass `2c` in `extractEntityRefs`); `EntityRef.needsResolution: true` tags refs from this pass (the ref's `slug` is the wikilink TARGET, `name` the optional display alias). `SlugResolver` gains optional `resolveBasenameMatches(name): Promise<string[]>` (multi-match by design — emits one edge per matching page). The single shared basename matcher is `buildBasenameIndex(slugs)` + `queryBasenameIndex(index, name)` + `normalizeBasename` (keys raw/lower/slugified tail, stable-sorted shorter-first then lexical), used by `makeResolver`, the FS `resolveBasenameMatchesFromSlugs`, AND the doctor check so they cannot drift. `makeResolver` step 1 and the FS resolver try `exactSlugCandidates` (`exact-slug-candidates.ts`: the value, then `slugifyPath`), then `resolveSlugWithAlias` (rename aliases); its fuzzy title step and live keyword step (source-scoped) accept a person/company/fund/organization candidate only under `sameEntityName`, so a near-name stays unresolved instead of linking to a different entity (`test/entity-resolution-near-names.test.ts`). With `link_resolution.global_basename` on, `extractFrontmatterLinks` passes `resolve(name, dirHint, { globalBasename, selfSlug })`, and a unique basename match (self excluded) resolves right after the exact steps and before the fuzzy and live keyword steps, the exact-name-first precedence of `resolveEntitySlug` (#5749, `test/link-resolution-basename-before-search.test.ts`). `makeResolver(engine, {mode, sourceId})` builds the index lazily via `engine.getAllSlugs({sourceId})` — source-scoped so a bare `[[name]]` never resolves to a same-tail page in a different source. `extractPageLinks` gains `opts.globalBasename` (routes `needsResolution` refs through `resolveBasenameMatches` keyed on `ref.slug`, emits candidates tagged `linkType: 'wikilink_basename'` + `linkSource: 'wikilink-resolved'`, skips self-loops) and `opts.skipFrontmatter`. All three surfaces (FS extract, DB extract, `put_page` auto-link) tag provenance with `link_source='wikilink-resolved'`; `put_page` includes it in its reconcilable-edge set so stale basename edges are removed when the wikilink or the flag goes away. Exports `WIKILINK_BASENAME_LINK_TYPE` + `isGlobalBasenameEnabled(engine)` (resolution order: env `GBRAIN_LINK_RESOLUTION_GLOBAL_BASENAME` → DB config `link_resolution.global_basename` → default false). `gbrain doctor`'s `link_resolution_opportunity` check surfaces a paste-ready enable hint when ≥5 bare wikilinks would resolve AND ≥20% match. Migration v113 widens `links_link_source_check` to admit `'wikilink-resolved'`; v114 then opens it to any kebab-case provenance (`^[a-z][a-z0-9]*(-[a-z0-9]+)*$`, ≤64 chars) so external derivers register their own tag (e.g. `citation-graph`) without a migration. `LINK_EXTRACTOR_VERSION_TS` also lives here (bump like `CHUNKER_VERSION` to invalidate prior extract-stale stamps). Pinned by `test/link-extraction.test.ts`, `test/extract-fs.test.ts`, `test/doctor.test.ts`, `test/e2e/global-basename-pglite.test.ts`. #6191: `inferLinkType` applies `link-target-roles.ts` `targetTakesVerb` to every per-edge and fallback verb: `works_at` never types a link to a person, meeting, calendar-event, media, image or transcript target (unknown type: a `people/`, `meetings/`, `calendar/`, `cal/` slug), `founded`/`invested_in` never a temporal or media one; a refused verb keeps scanning lower-precedence rules, then `mentions` (`test/link-type-target-guard.test.ts`).
- `src/core/timeline-dedup-repair.ts` — schema-drift self-heal for `idx_timeline_dedup`. A master-merge migration renumbering can leave a brain's version counter stamped past the index change while the index keeps an old shape — and every `addTimelineEntry` batch then fails its `ON CONFLICT` inference, silently breaking timeline writes brain-wide. The version counter can't detect this, so the repair is keyed off the actual index SHAPE. Canonical shape is `(page_id, date, md5(summary), source)` — md5-keyed (migration v138) because a raw long/incompressible summary would overflow the btree v4 row cap (~2704 bytes) and abort every timeline insert for that page; both engines' insert sites infer `ON CONFLICT` on the md5 tuple, so `EXPECTED_COLUMNS` MUST carry the md5 form (a raw-shape expectation would make the repair revert v138 on every migrate pass). `checkTimelineDedupIndex(engine)` returns `{tablePresent, indexPresent, columns, needsRepair}` (read-only; powers the `timeline_dedup_index` doctor check; the indexdef column parser is paren-depth-aware so `md5(summary)` — or any future expression column — parses as ONE column instead of flagging a correct index as drifted forever) and `repairTimelineDedupIndex(engine)` dedupes-then-rebuilds to the canonical shape (raw-summary grouping ⟺ md5 grouping modulo negligible collisions). `runMigrations` invokes the repair on every pass (including the no-pending early-return path); idempotent no-op when the index already matches. `gbrain apply-migrations --force-schema` triggers it on demand. Pinned by `test/timeline-dedup-repair.test.ts`.