Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(persistence): frontmatter-only drift never blocks remember; last_used-only contact drift is additive; dedupe contact rows; gbrain repair contact-rows (#8109)
garrytan2d6186bbc773
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex debf81335f..e1b595c7b4 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,21 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**A frontmatter-only disagreement between a page's file and its database copy no longer blocks an unrelated `remember`, a contact row whose `last_used` moved forward reconciles automatically, and duplicated contact rows can be collapsed with one previewed repair (#8109).**++### What changed++| Situation | Before | Now |+|---|---|---|+| A person page's file differs from the database only in frontmatter (for example one contact row's `last_used` moved forward), and an agent saves a fact | `remember` refused `source_changed` (`file_database_drift`) until someone reconciled the page | the fact lands; the file keeps its own frontmatter with the new body, and the frontmatter drift stays for reconcile. Body, timeline, title, type, tag and policy or privacy key drift still refuse, and so does a write that changes frontmatter |+| `gbrain sources reconcile <source> <slug> --preview --auto-additive` on a contact row whose only change is a later `last_used` | `review` (`list_entries_changed_or_reordered`) | `auto` (rule `contacts_activity_advance`); a removed, reordered or otherwise changed row, or a date moved backward or into the future, still needs review. Unquoted YAML dates compare as dates |+| A `put_page` or `capture` whose frontmatter `contacts:` list repeats a `(type, value)` row | stored every copy | keeps one row per `(type, value)` with the newest `last_used`; rows that differ in another field and every distinct value stay |+| Pages that already hold repeated contact rows | no repair | `gbrain repair contact-rows [--source <id>] [--slug <slug>]...` previews (writes nothing), then `--apply --expect <hash> --yes` rewrites exactly the previewed pages through revision-bound `put_page`: the file through the coordinated writer on sources with a checkout, the database on repo-less brains. Drifted pages are listed `needs_reconcile` with the reconcile command |++Nothing runs on upgrade.+## [0.60.528.0] - 2026-10-31**The test-isolation scan is about 25 times faster on Linux and 12 times faster on macOS, so `bun run verify` and the macOS validation job no longer come close to their per-check caps (#8053, #8099).**diff --git a/docs/guides/concurrent-writes.md b/docs/guides/concurrent-writes.mdindex af330554ce..1356d5c86d 100644--- a/docs/guides/concurrent-writes.md+++ b/docs/guides/concurrent-writes.md@@ -188,7 +188,7 @@ outcomes and reasons, never values), and `classified` counts the batch:| Classification | Meaning | Next step || --- | --- | --- |-| `structurally_additive` | Only additive metadata changed: a `contacts` list with entries appended after every stored entry (order and repeats kept), an activity date that moved forward (`updated`, `last_*`, `*_last_used`), or a field present on one side only. | Preview with `--auto-additive`, then apply. |+| `structurally_additive` | Only additive metadata changed: a `contacts` list with entries appended after every stored entry (order and repeats kept) or whose stored rows changed only by an activity date moving forward (a row's `last_used`), an activity date that moved forward (`updated`, `last_*`, `*_last_used`), or a field present on one side only. | Preview with `--auto-additive`, then apply. || `additive_with_suggestions` | The above, plus body or timeline lines inserted without changing or removing any stored line. | Read the inserted lines, then preview with `--auto-additive --accept-suggested`. || `review_required` | Stored text changed or was removed; or a policy, privacy, title, type, tag, alias or fence change; or a date that moved backward, changed format or is in the future. | Resolve manually as above, after asking the user. || `formatting_only` | The parsed content already agrees. | Preview; the result is the database content. |diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex fbfc157404..41a87db38c 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -1155,6 +1155,41 @@ It drops generated bullets that carry markup (yours are left alone), deletesmarkup-only rows and strips the rest, under the page lock. A second apply findsnothing, and a timeline-only rewrite queues no facts extraction.+<a id="contact-rows"></a>+### Contact rows++A writer that appended a `{type, value, last_used}` row to a page's+frontmatter `contacts:` list on every touch, instead of updating the existing+row, left the same value many times (#8109). `put_page` and `capture` now keep+one row per (type, value); this explicit-only, preview-bound kind collapses the+rows already stored. It never runs from `--all`, doctor's remediation plan, the+maintenance cycle or an upgrade.++**Say to your agent:** *"Preview which pages repeat contact rows, show me the+counts, then apply after I agree."*++```bash+gbrain repair contact-rows --source <id> [--slug <slug>]... # preview: dedupe / needs_reconcile+gbrain repair contact-rows --source <id> --apply --expect <hash> --yes # rewrites exactly the previewed pages+```++Rows group by type (trimmed, case-insensitive) and value (trimmed), so two+spellings of one number stay two rows. A group collapses into its first row,+keeping the newest `last_used` (and any other activity date); a field present on+one row only is kept. A group whose rows differ in any other field only loses+exact copies. Rows that are not `{type, value}` objects are left alone, and only+the `contacts` key changes.++The preview counts `pages_to_dedupe`, `rows_to_remove` and `needs_reconcile`.+A page whose canonical file differs from the database is listed+`needs_reconcile` and never written: preview+`gbrain sources reconcile <source> <slug> --brain host --preview --auto-additive`+(a `last_used` that moved forward is structurally additive), apply it, then+preview this repair again. The apply publishes one revision-bound `put_page` per+page, so a source with a checkout commits the file through the coordinated+writer and a repo-less source writes the database; a page that changed since the+preview refuses with `preview_changed`.+<a id="timeline-order"></a>### Timeline orderdiff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex e43d770ea2..3c8d87f608 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -51,7 +51,7 @@ of making a duplicate.| Reason | Error code | What it means | Recovery || --- | --- | --- | --- |-| <a id="file_database_drift"></a>`file_database_drift` | `source_changed` | The page's canonical file and its database copy disagree, usually because the file was edited outside a coordinated write. Neither copy was overwritten. A file with no `type:` line keeps the stored type, and titles compare trimmed, so those alone do not cause this. | The receipt's fix (`gbrain sources writer failures <source>`, `gbrain write-request <request_id> --json`) previews keeping the database copy for that one page with `gbrain repair file-drift --source <source> --slug <slug> --prefer database` (`--prefer file` previews importing the file instead); a person picks the side, then the agent runs the apply command the preview prints. A group of such receipts gets `--all-drifted --since <instant>` with the oldest receipt's time, so only pages that drifted since then are selected. For a hand merge: `gbrain sources reconcile <source> <slug> --brain <brain> --preview`, review the preview, resolve it, then `--apply <resolved-preview-file> --request-id <new uuid>`. Retry the original write with a **new** request ID. A page that records no file of its own (no `source_path`, typically written by an older release whose write-through also put `ingested_via` / `ingested_at` / `source_kind` in the file only) reconciles against the file at its slug path only when you add `--adopt-slug-path` to the preview; the refusal names the flag in that case, and apply records that file as the page's origin. See [repair a file/database disagreement](concurrent-writes.md#repair-a-filedatabase-disagreement). Many pages at once: `gbrain sources reconcile <source> --brain host --audit --conflicts` lists them and `gbrain repair file-drift` resolves them, preferring the file or the database copy ([file drift](repair.md#file-drift)). |+| <a id="file_database_drift"></a>`file_database_drift` | `source_changed` | The page's canonical file and its database copy disagree, usually because the file was edited outside a coordinated write. Neither copy was overwritten. A file with no `type:` line keeps the stored type, and titles compare trimmed, so those alone do not cause this. Drift only in ordinary frontmatter keys (not a policy or privacy key, title, type or tags) does not refuse a write that leaves the frontmatter unchanged, such as `remember`: the write lands and the file keeps its own frontmatter, so that drift is left for reconcile (#8109). | The receipt's fix (`gbrain sources writer failures <source>`, `gbrain write-request <request_id> --json`) previews keeping the database copy for that one page with `gbrain repair file-drift --source <source> --slug <slug> --prefer database` (`--prefer file` previews importing the file instead); a person picks the side, then the agent runs the apply command the preview prints. A group of such receipts gets `--all-drifted --since <instant>` with the oldest receipt's time, so only pages that drifted since then are selected. For a hand merge: `gbrain sources reconcile <source> <slug> --brain <brain> --preview`, review the preview, resolve it, then `--apply <resolved-preview-file> --request-id <new uuid>`. Retry the original write with a **new** request ID. A page that records no file of its own (no `source_path`, typically written by an older release whose write-through also put `ingested_via` / `ingested_at` / `source_kind` in the file only) reconciles against the file at its slug path only when you add `--adopt-slug-path` to the preview; the refusal names the flag in that case, and apply records that file as the page's origin. See [repair a file/database disagreement](concurrent-writes.md#repair-a-filedatabase-disagreement). Many pages at once: `gbrain sources reconcile <source> --brain host --audit --conflicts` lists them and `gbrain repair file-drift` resolves them, preferring the file or the database copy ([file drift](repair.md#file-drift)). || <a id="canonical_file_missing"></a>`canonical_file_missing` | `source_changed` (a caller write; maintenance refuses before admission) | The page's recorded canonical file is gone while its database page is still live (a ghost page: the file was moved or deleted outside gbrain). A caller write refuses with "The canonical file was removed outside coordinated publication." and nothing is written; retrying it refuses the same way. A maintenance write (fact-fence adoption, enrichment, `dream`) is refused before admission: no request id, no receipt (before #6696 each cycle settled a `conflict` receipt). `gbrain sources writer failures <source>` reports such receipts as category `canonical_file_missing`. Neither copy was changed. | Decide which copy is right. If the file should exist, restore it at its recorded path and preview the page with `gbrain sources reconcile <source> <slug> --brain host --preview`, or sync. If the page should go, preview the ghost-page repair with `gbrain repair ghost-pages --source <source>` (read-only; it prints the apply command with its hash) and apply it with `--apply --expect <hash> --yes` only after the user agrees; a soft-deleted page can be restored with `gbrain restore <slug> --force` until the autopilot purge 72 h later. `gbrain pages ghosts --source <source>` and doctor `ghost_pages` list them, doctor `maintenance_write_drift` lists held-back maintenance writes. See [ghost pages](repair.md#ghost-pages). || <a id="refusal_breaker"></a>`refusal_breaker` | the code of the earlier refusals | The same maintenance write (same source, slug, page revision, planner and intent) was refused `persistence.maintenance.refusal_breaker_threshold` times (default 3; `0` turns it off) in the last 24 hours, so maintenance submits nothing for it instead of adding another receipt (#6696). It is tried again once the page, the planner or the write changes, or when the oldest refusal is 24 hours old. | Read the last refusal with `gbrain write-request -- <request_id>` (named in the message) and fix what it names. Doctor `maintenance_write_drift` lists every held-back write with its exit. || <a id="canonical_file_absent"></a>`canonical_file_absent` | `source_changed` (`restore_page` / `gbrain restore`) | On a brain without managed persistence, a restore that writes no file (`sync.write_through` off, a sandbox, a database-only source) of a soft-deleted page whose recorded file is no longer among the files sync imports (#6609). A plain restore would be undone by the next `gbrain sync --full`, so nothing was restored. Not retryable; the fix asks the user. | Ask the user. To keep the page with its file: put the file back at its recorded path, commit it and `gbrain sync --source <source>`. To keep it database-only: `gbrain restore <slug> --source-id <source> --detach` (MCP `restore_page` with `detach: true`), which forgets the recorded file and stamps `database_only_reason='detached'`. See [restoring a page whose file is gone](../storage-tiering.md#restoring-a-page-whose-file-is-gone). |diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex b2e11fd8fb..3410ab21bf 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: linksrc/core/search/evidence-delivery.ts 1755 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned)src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)-src/core/behavior-change-notice.ts 1575 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +1 GBRA-250 #8071 rows_behind_checkpoint_8071 row (since TBD until the merge slot); +1 GBRA-250 #8073 derived_write_through_commits_8073 row (since TBD until the merge slot); +2 GBRA-250 #8072 rows (oversize hold left unread, refresh sync budget; since TBD until the merge slot); +1 GBRA-156 #7593: private_pages_granted_sources row (re-summed at the 0.60.527.0 merge: 1574 + 1)+src/core/behavior-change-notice.ts 1578 ratchet +3 GBRA-253 #8109 rows (frontmatter-only drift write, contacts activity advance, contact-rows dedupe and repair); since 0.60.528.0 until the merge slot restamps; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +1 GBRA-250 #8071 rows_behind_checkpoint_8071 row (since TBD until the merge slot); +1 GBRA-250 #8073 derived_write_through_commits_8073 row (since TBD until the merge slot); +2 GBRA-250 #8072 rows (oversize hold left unread, refresh sync budget; since TBD until the merge slot); +1 GBRA-156 #7593: private_pages_granted_sources row (re-summed at the 0.60.527.0 merge: 1574 + 1)diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsvindex c756cf2759..a74764a148 100644--- a/scripts/page-body-readers-baseline.tsv+++ b/scripts/page-body-readers-baseline.tsv@@ -467,6 +467,7 @@ src/core/persistence/page-input.ts normalizeSubagentPageInput call:serializeMarksrc/core/persistence/page-input.ts normalizeSubagentPageInput field SEGWsrc/core/persistence/page-prepare.ts canonical field SEGW lane 1d file; segmented page: getter throws page_body_segmentedsrc/core/persistence/page-prepare.ts fileMatchesSnapshot field SEGW lane 1d file; segmented page: getter throws page_body_segmented+src/core/persistence/page-prepare.ts frontmatterDriftWrite field SEGW #8109: called only for an inline page (bodyUnread excluded at the call site); compares the file's parsed body with the snapshot bodysrc/core/persistence/page-prepare.ts prepareInlineOrSegmentedMutation call:serializePageToMarkdown SEGW lane 1d filesrc/core/persistence/page-prepare.ts prepareInlineOrSegmentedMutation field SEGW lane 1d file: preserveProtectedTakes reads the snapshot body; segmented page: getter throws page_body_segmentedsrc/core/persistence/page-prepare.ts segmentedFileMatches field NONE the body fields hold region digests (body-digests.ts) for the canonical comparison; no page body is read@@ -526,6 +527,8 @@ src/core/repair/connector-fences.ts candidates sql STREAMsrc/core/repair/connector-fences.ts connectorFencesRepair call:serializePageToMarkdown SEGWsrc/core/repair/connector-fences.ts connectorFencesRepair field STREAMsrc/core/repair/connector-fences.ts relocateTimelineFences field STREAM+src/core/repair/contact-rows.ts plannedChange call:serializePageToMarkdown SEGW SQL exclusion in scan: body_storage='inline' only+src/core/repair/contact-rows.ts plannedChange field SEGW SQL exclusion in scan: body_storage='inline' only; the read-back compares the re-rendered bodysrc/core/repair/conversation-labels.ts classifyLabelFacts sql STREAMsrc/core/repair/conversation-labels.ts countLabelFacts sql STREAMsrc/core/repair/duplicate-facts.ts strikeFencedCopies call:serializePageToMarkdown INLINE inline pages only: a segmented page returns before the read, and its fenced copies stay active (listed as segmented_page)diff --git a/src/commands/repair.ts b/src/commands/repair.tsindex 6b73bafd6a..0a4a0a5591 100644--- a/src/commands/repair.ts+++ b/src/commands/repair.ts@@ -46,6 +46,8 @@ const REPAIR_CONTENT_HELP = ` gbrain repair content [--source <id>] [--onl[--apply [--expect <hash>[,<hash>]]] [--json]`;/** #7247: the usage line of the timeline-order kind, which also takes --slug (the hash covers the selection). */const REPAIR_ORDER_HELP = ' gbrain repair timeline-order [--source <id>] [--slug <slug>]... [--apply --expect <preview-hash> --yes] [--json]';+/** #8109: the usage line of the contact-rows kind, which also takes --slug. */+const REPAIR_CONTACTS_HELP = ' gbrain repair contact-rows [--source <id>] [--slug <slug>]... [--apply --expect <preview-hash> --yes] [--json]';const REPAIR_CONTENT_SUMMARY = ` content The content-repair lane (#6377), not a kind: runs ${CONTENT_REPAIR_KINDS.join(', then ')} over the same--source/--only selection, so the holds managed sync records for a file's content (invalid_fence,frontmatter_slug_conflict) clear in one pass. The preview plans every kind before anything is written and@@ -99,6 +101,7 @@ export const REPAIR_HELP = `Usage: gbrain repair [<kind>] [--apply] [--source <igbrain repair provenance [--source <id>] [--slug <slug>]... [--include-ambiguous] [--apply --expect <preview-hash> --yes] [--json]gbrain repair secret-spans --source <id> --slug <slug>... [--rules <rule,...>] [--no-embed] [--apply --expect <preview-hash>] [--json]${REPAIR_ORDER_HELP}+${REPAIR_CONTACTS_HELP}gbrain repair rename-identity --source <id> [--slug <new-slug> [--from <old-slug>] [--close][--apply --expect <preview-hash> --yes]] [--json]${REPAIR_CONTENT_HELP}@@ -206,7 +209,7 @@ const VALUE_FLAGS = new Set(['--rules', '--source', '--limit', '--expect', '--onconst KIND_FLAGS: Record<string, readonly string[]> = {'--only': ['frontmatter', 'fences', 'slug-conflicts', 'file-drift', CONTENT_LANE], '--skip': ['frontmatter', 'fences', 'slug-conflicts', 'file-drift', CONTENT_LANE], '--diff': ['frontmatter', 'fences', 'slug-conflicts', CONTENT_LANE], '--yes': ['frontmatter', 'conversation-labels', 'stuck-recovery', 'ghost-pages', 'legacy-twins', 'rename-identity', 'url-clone-fences'],'--include-cross-entity': ['duplicate-facts'],- '--slug': ['fences', 'file-drift', 'rename-identity', 'overwritten-edits', 'origin-twins', 'timeline-order', 'provenance', 'secret-spans'], '--from': ['rename-identity'], '--close': ['rename-identity'], '--prefer': ['file-drift'], '--request': ['file-drift'], '--all-drifted': ['file-drift'], '--class': ['file-drift', 'visibility'], '--since': ['file-drift', 'generations'],+ '--slug': ['fences', 'file-drift', 'rename-identity', 'overwritten-edits', 'origin-twins', 'timeline-order', 'contact-rows', 'provenance', 'secret-spans'], '--from': ['rename-identity'], '--close': ['rename-identity'], '--prefer': ['file-drift'], '--request': ['file-drift'], '--all-drifted': ['file-drift'], '--class': ['file-drift', 'visibility'], '--since': ['file-drift', 'generations'],'--held': ['file-drift', 'generations'], '--side': ['file-drift'], '--state': ['file-drift'], '--cursor': ['file-drift'], '--provenance': ['file-drift'],'--keep': ['stuck-recovery'], '--request-id': ['stuck-recovery'],'--older-than': ['receipt-compaction'], '--include-failures': ['receipt-compaction'], '--force': ['receipt-compaction'],@@ -490,7 +493,7 @@ export async function runRepairCommand(engine: BrainEngine, args: string[]): Pro...(olderThan ? ['--older-than', olderThan] : []), ...(includeFailures ? ['--include-failures'] : []), ...(force ? ['--force'] : []),...(afterPage !== undefined ? ['--after', String(afterPage)] : []), ...(day ? ['--day', day] : []),...(kind === 'rename-identity' ? [...slugs.flatMap(slug => ['--slug', slug]), ...(from ? ['--from', from] : []), ...(close ? ['--close'] : [])] : []),- ...(kind === 'origin-twins' || kind === 'timeline-order' || kind === 'provenance' ? slugs.flatMap(slug => ['--slug', slug]) : []),+ ...(kind === 'origin-twins' || kind === 'timeline-order' || kind === 'contact-rows' || kind === 'provenance' ? slugs.flatMap(slug => ['--slug', slug]) : []),...(generations ? generationsSelectionArgv(generations) : []),...(kind === 'agent-session-facts' ? agentSessionFlagArgs({ cwdPrefixes, sessions, principal }) : [])];const custom = repairSpec(kind as RepairKind).consentText;diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex d2bcaa71a8..cfd2510ec4 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1376,6 +1376,9 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.525.0', id: 'rows_behind_checkpoint_8071', text: 'New doctor check `sync_rows_behind_checkpoint` and explicit-only repair kind `gbrain repair rows-behind --source <id>` (#8071): on each managed source\'s owner host, doctor compares every live Markdown page with its file read from Git at the sync checkpoint (`sources.last_commit`) and warns about rows that hold an older version with no hold, which no later sync re-imports because the import record names the current file. The repair previews each row with its path, checkpoint blob and action and writes nothing; `--apply --expect <hash>` re-imports the committed file through the managed import (no file is written), or opens the hold sync would have opened when that import is refused or the file is over the read bound. Rows with no import record join only with `--include-ambiguous`. Bounded by `GBRAIN_DOCTOR_ROWS_BEHIND_TIMEOUT_MS` (default 10 s); not applicable on repo-less brains and off the owner host. It never runs on upgrade or automatically.' },{ since: '0.60.525.0', id: 'derived_write_through_commits_8073', text: 'Files the dream cycle writes into a source that is a Git checkout are now committed: atom and concept files (with `cycle.extract_atoms.write_through` or `cycle.synthesize_concepts.write_through` on) and the reverse-written synthesis, patterns and cycle-summary pages and their removals. Each batch (an item\'s atoms plus the files of the atoms it retired, the backfill, one concept, or one phase\'s reverse-written pages) is one local commit naming only those files, made through the checkout\'s commit lock (a hardened checkout\'s post-commit hook pushes it; a checkout with a canonical owner marker gets a named-paths-only commit). They used to stay untracked or modified, which made the scheduled sync skip every run. A commit the checkout refuses rolls back the files it wrote and reports `write_through_failed` with a warn status; atoms and concepts are written again on the next run. Each written file is also recorded in the pending write-through ledger, so files a crash left uncommitted go into the next commit gbrain makes in that checkout. Files left uncommitted by earlier versions are not touched; commit them with `gbrain git commit-push -m <msg> -- <paths>`.' },{ since: '0.60.525.0', id: 'oversize_hold_unread_8072', text: 'A Markdown file over 24 MiB that sync had to read whole to refuse (segmented storage on, held `file_too_large`, for example for a facts fence) is no longer read again by every sync: it stays held, unread, until it changes, gbrain is upgraded, or `gbrain sources retry-held <source>` schedules it. After changing a setting the hold message names, run `gbrain sources retry-held <source>`. A file held from its size alone still re-checks on every sync.' },+ { since: '0.60.528.0', id: 'frontmatter_drift_write_8109', text: 'A write that leaves a page\'s frontmatter unchanged (`remember`, a fact or timeline write, a `put_page` of the stored frontmatter) is no longer refused `source_changed` (`file_database_drift`) when the page\'s canonical file differs from the database only in ordinary frontmatter keys (for example a contact row\'s `last_used` moved forward in the file): the write lands in the database and the file keeps its own frontmatter with the new body, so the frontmatter drift stays for `gbrain sources reconcile <source> <slug> --preview --auto-additive`. Drift in the body, timeline, title, type, tags or a policy or privacy key (`access`, `visibility`, `aliases`, ...) refuses as before, and so does a write that changes frontmatter (#8109).' },+ { since: '0.60.528.0', id: 'contacts_activity_advance_8109', text: '`gbrain sources reconcile --auto-additive` now decides a `contacts` row whose only change is an activity date (`last_used`, `last_*`, `*_last_used`, `updated`) moving forward as structurally additive (rule `contacts_activity_advance`) instead of `review` (`list_entries_changed_or_reordered`); a row that changed in any other way, moved, was removed or moved a date backward or into the future still needs review. Unquoted YAML dates compare as the instants they are instead of `date_not_text` (#8109).' },+ { since: '0.60.528.0', id: 'contact_rows_dedupe_8109', text: '`put_page` and `capture` keep one frontmatter `contacts` row per (type, value), in first-seen order with the newest `last_used`, in the file and the database alike; a row that differs in another field and every distinct value are kept. New explicit repair `gbrain repair contact-rows [--source <id>] [--slug <slug>]...` collapses rows already duplicated: the preview writes nothing and prints a hash, `--apply --expect <hash> --yes` rewrites exactly the previewed pages through revision-bound `put_page` (the canonical file on sources with a checkout, the database on repo-less ones), and a page whose file drifted is listed `needs_reconcile`. Nothing runs on upgrade (#8109).' },{ since: '0.60.525.0', id: 'refresh_sync_budget_8072', text: 'The member syncs of one `gbrain sources refresh` run stop starting work after `sources.refresh_sync_budget_ms` (default 5 minutes; `GBRAIN_REFRESH_SYNC_BUDGET_MS`). Each member gets an equal share of what is left. A run past it reports `sync_deferred` with the members it left `deferred` and still completes the refresh, so writes to the checkout are admitted again and the next `gbrain sources refresh <source>` continues them; a member that committed nothing in its share is reported `refresh_sync_timeout` in `sync_blocked`. A stalled sync no longer holds the refresh lock across scheduled runs or starves the other members.' },];diff --git a/src/core/contact-rows.ts b/src/core/contact-rows.tsnew file mode 100644index 0000000000..227d4d4fe8--- /dev/null+++ b/src/core/contact-rows.ts@@ -0,0 +1,84 @@+/**+ * #8109: duplicated contact rows. A page's frontmatter `contacts:` list of+ * `{type, value, last_used, ...}` rows collapses to one row per (type, value):+ * a writer that appends a row on every touch instead of updating the existing+ * one leaves the same value many times. Pure; no I/O.+ *+ * Never drops a distinct value: rows group by (type trimmed and lower-cased,+ * value trimmed), so two spellings of a value stay two rows; a merged row keeps+ * its first row's spelling. A group merges into its first row only when every field other than an activity date agrees+ * or is absent on some rows; activity dates (`last_*`, `*_last_used`,+ * `updated`) keep the newest valid date. A group with any other conflicting+ * field only loses rows that are exact copies of an earlier one. Rows that are+ * not `{type, value}` objects are left as they are.+ */+import { digest } from './persistence/digest.ts';++export const CONTACT_ROWS_KEY = 'contacts';+export const ACTIVITY_DATE_KEY = /^(?:updated|updated_at|modified|last_modified|last_[a-z0-9_]+|[a-z0-9_]+_last_used)$/;++export interface ContactRowsDedupe { rows: unknown[]; removed: number; kept_conflicting: number }++type Row = Record<string, unknown>;+const isRow = (value: unknown): value is Row => value !== null && typeof value === 'object' && !Array.isArray(value) && !(value instanceof Date);+const keyOf = (row: unknown): string | null => {+ if (!isRow(row) || typeof row.type !== 'string' || (typeof row.value !== 'string' && typeof row.value !== 'number')) return null;+ const type = row.type.trim().toLowerCase(), value = String(row.value).trim();+ return type && value ? JSON.stringify([type, value]) : null;+};+const instant = (value: unknown): number => {+ const text = value instanceof Date ? value.toISOString() : typeof value === 'string' ? value : null;+ return text === null ? NaN : Date.parse(text);+};++/** One row for the group, or null when a non-activity field conflicts or an activity date is not a date. */+function merge(group: Row[]): Row | null {+ const merged: Row = {};+ for (const row of group) {+ for (const [field, value] of Object.entries(row)) {+ if (!Object.hasOwn(merged, field)) { merged[field] = value; continue; }+ if (field === 'type' || field === 'value' || digest(merged[field]) === digest(value)) continue;+ if (!ACTIVITY_DATE_KEY.test(field)) return null;+ const kept = instant(merged[field]), next = instant(value);+ if (!Number.isFinite(kept) || !Number.isFinite(next)) return null;+ if (next > kept) merged[field] = value;+ }+ }+ return merged;+}++/** The deduplicated list, or null when `list` is not an array or nothing would change. */+export function dedupeContactRows(list: unknown): ContactRowsDedupe | null {+ if (!Array.isArray(list)) return null;+ const groups = new Map<string, Row[]>();+ for (const row of list) {+ const key = keyOf(row);+ if (key) groups.set(key, [...(groups.get(key) ?? []), row as Row]);+ }+ if (![...groups.values()].some(group => group.length > 1)) return null;+ const merged = new Map<string, Row | null>([...groups].map(([key, group]) => [key, group.length > 1 ? merge(group) : group[0]]));+ const rows: unknown[] = [];+ const emitted = new Set<string>();+ const seenExact = new Set<string>();+ let kept_conflicting = 0;+ for (const row of list) {+ const key = keyOf(row);+ if (!key) { rows.push(row); continue; }+ const one = merged.get(key);+ if (one) { if (!emitted.has(key)) { emitted.add(key); rows.push(one); } continue; }+ const exact = digest(row);+ if (seenExact.has(exact)) continue;+ seenExact.add(exact);+ if (emitted.has(key)) kept_conflicting++;+ emitted.add(key);+ rows.push(row);+ }+ const removed = list.length - rows.length;+ return removed > 0 ? { rows, removed, kept_conflicting } : null;+}++/** `frontmatter` with its contact rows deduplicated, or null when nothing changes. */+export function dedupeFrontmatterContacts(frontmatter: Record<string, unknown> | null | undefined): { frontmatter: Record<string, unknown>; dedupe: ContactRowsDedupe } | null {+ const dedupe = frontmatter && Object.hasOwn(frontmatter, CONTACT_ROWS_KEY) ? dedupeContactRows(frontmatter[CONTACT_ROWS_KEY]) : null;+ return dedupe ? { frontmatter: { ...frontmatter, [CONTACT_ROWS_KEY]: dedupe.rows }, dedupe } : null;+}diff --git a/src/core/persistence/page-prepare.ts b/src/core/persistence/page-prepare.tsindex 4cd0ffd2de..c696df3cc3 100644--- a/src/core/persistence/page-prepare.ts+++ b/src/core/persistence/page-prepare.ts@@ -89,6 +89,9 @@ import { pageFencesNormalized } from '../fence-repair/report.ts';import { parseFenceRepairReceipt } from '../fence-repair/receipt.ts';import { fenceRepairCommit } from './effect-model.ts';import { withoutSafetyAssessments } from './reconcile-safety.ts';+import { POLICY_KEYS } from './reconcile-additive.ts';+import { dedupeFrontmatterContacts } from '../contact-rows.ts';+import { protectedReconcileKey } from './reconcile-merge.ts';import { preparePagePurge, purgePageInTransaction, type PreparedPagePurge } from './page-purge.ts';import { withTrustKeep } from './context.ts';import { isFenceEditWrite, pageGateTrust, pageWriteTrust, recordAgentPageLowering, stampPageTrustMarker, storedPageTier } from '../trust/page-write.ts';@@ -128,6 +131,12 @@ function canonical(page: Pick<Page, 'type' | 'title' | 'compiled_truth' | 'timelfrontmatter: comparableFrontmatter(page.frontmatter), tags: [...new Set(tags)].sort() };}interface CanonicalProvenance { source_kind: string; ingested_via: string; ingested_at: string; }+/** #8109: a caller-authored page (put_page, capture) keeps one contact row per (type, value), the newest last_used kept; file and database alike. */+function callerContactsDeduped(row: Pick<WriteRequest, 'operation'>, page: ParsedPage): ParsedPage {+ const contacts = row.operation === 'put_page' || row.operation === 'capture' ? dedupeFrontmatterContacts(page.frontmatter) : null;+ if (contacts) page.frontmatter = contacts.frontmatter;+ return page;+}/*** Writes whose caller supplies the page text, and so could supply provenance stamps in its frontmatter. Unlike* trust/page-write.ts CONTENT_OPERATIONS, revert_version and restore_page are absent: they replay a stored version.@@ -230,6 +239,36 @@ export async function fileMatchesSnapshot(engine: BrainEngine, slug: string, bytconst reading = readCanonicalAsImported(bytes, typeof sourcePath === 'string' ? sourcePath : await sourcePath());return reading.inferred && await matches(reading.content, true);}+/**+ * #8109: the file a write publishes over a canonical file that differs from the database only in frontmatter, or null+ * when the write must refuse as drift. The file and the database must agree on everything but frontmatter (body,+ * timeline, title, type and tags, as fileMatchesSnapshot compares them), no drifted key may be a policy or protected+ * key, and the write must leave the stored frontmatter unchanged. The file then keeps its own frontmatter block and+ * takes the write's body, so neither side's frontmatter is lost; the drift stays for reconcile.+ */+async function frontmatterDriftWrite(engine: BrainEngine, slug: string, fileText: string, snapshot: PageSnapshot, content: string,+ activePack?: ParseOpts['activePack']): Promise<string | null> {+ const file = parseMarkdown(fileText, slug, { activePack });+ const type = importedFileType(file, snapshot.page, false);+ const bodies = await overlayCanonicalBodies(engine.executeRaw.bind(engine), file.compiled_truth, file.timeline ?? '', snapshot.withdrawals,+ { sourceId: snapshot.page.source_id, marker: snapshot.globalPurges });+ const bare = (page: Parameters<typeof canonical>[0], tags: string[]) => digest(canonical({ ...page, frontmatter: {} }, tags));+ if (bare({ ...file, type, ...bodies }, file.tags) !== bare(snapshot.page, snapshot.tags)) return null;+ const fileFrontmatter = comparableFrontmatter(withoutSafetyAssessments(file.frontmatter));+ const stored = comparableFrontmatter(withoutSafetyAssessments(snapshot.page.frontmatter ?? {}));+ const drifted = [...new Set([...Object.keys(fileFrontmatter), ...Object.keys(stored)])].filter(key => digest(fileFrontmatter[key] ?? null) !== digest(stored[key] ?? null));+ if (!drifted.length || drifted.some(key => POLICY_KEYS.has(key) || protectedReconcileKey(key))) return null;+ const incoming = parseMarkdown(content, slug, { activePack });+ if (digest(comparableFrontmatter(withoutSafetyAssessments(incoming.frontmatter))) !== digest(stored)+ || bare({ ...incoming, compiled_truth: '', timeline: '' }, incoming.tags) !== bare({ ...snapshot.page, compiled_truth: '', timeline: '' }, snapshot.tags)) return null;+ const head = /^\uFEFF?---\r?\n[\s\S]*?\r?\n---[ \t]*\r?\n/;+ const fileHead = head.exec(fileText), contentHead = head.exec(content);+ if (!fileHead || !contentHead) return null;+ const spliced = fileHead[0] + content.slice(contentHead[0].length);+ const check = parseMarkdown(spliced, slug, { activePack });+ return digest(check.frontmatter) === digest(file.frontmatter) && check.title === file.title && check.type === incoming.type && digest(check.tags) === digest(file.tags)+ && check.compiled_truth === incoming.compiled_truth && (check.timeline ?? '') === (incoming.timeline ?? '') ? spliced : null;+}/** The type a file's parse compares with (`fileMatchesSnapshot`), after aligning the parse with what its import over `page` stores. */function importedFileType(parsed: ParsedMarkdown, page: Pick<Page, 'type' | 'frontmatter'>, inferred: boolean): Page['type'] {// #5521 parity for subtype (#5928): a pack rule is not part of the file's bytes, so a file@@ -421,6 +460,10 @@ async function compareFileTarget(engine: BrainEngine, row: Pick<WriteRequest, 'sconst adopt = !snapshot.page.source_path && !snapshot.page.source_uri ? ['--adopt-slug-path'] : [];// #6693: a committed Git change sync has not imported yet (another agent's commit), on a page that records its file.if (!adopt.length && await waitingOnWhile(options.clock, 'git', () => committedPendingImport(engine, row.source_id, root, path, options.clock?.signal))) throw pendingImportRefusal(row, options.remote === true);+ // #8109: drift confined to ordinary frontmatter keys never blocks a write that leaves the frontmatter alone.+ const spliced = !adopt.length && options.deleting !== true && typeof content === 'string' && before instanceof Buffer && !bodyUnread(snapshot.page) && before.length <= PUSH_SCAN_MAX_FILE_BYTES+ ? await frontmatterDriftWrite(engine, row.slug, before.toString('utf8'), snapshot, content, options.activePack) : null;+ if (spliced !== null) return { path, root, content: spliced, expectedBeforeHash: beforeHash() };throw opError('source_changed', 'The canonical file contains an uncoordinated local edit.',`On the brain host, run gbrain sources reconcile ${row.source_id} ${row.slug} --brain <brain id, host by default> --preview${adopt.length ? ' --adopt-slug-path (the page records no file of its own, so the preview adopts the file at its slug path)' : ''}, review and apply the resolved preview, then retry this write with a new request_id. Neither copy was overwritten.`,{ detail: 'file_database_drift', fix: readFix(`Previews how page ${row.slug}'s canonical file and database copy reconcile; a preview never changes canonical content.`,@@ -920,7 +963,7 @@ async function prepareInlineOrSegmentedMutation(unbounded: BrainEngine, row: Wri...(gateTrust ? { writeGate: gateInput(gateTrust, row.id) } : {}),// Provenance compares the caller's page with the stored one, so it runs before the server's trust marker is stamped.prepareFrontmatter: page => {- provenance = putProvenance(row, snapshot, page);+ provenance = putProvenance(row, snapshot, callerContactsDeduped(row, page));suppliedTrustMarker = page.frontmatter.trust_tier;stampPageTrustMarker(row, page.frontmatter, trust, storedTier, snapshot?.page.frontmatter);if (privateRequest) assertPrivateRequestKept(row, page.frontmatter);diff --git a/src/core/persistence/preview-approval.ts b/src/core/persistence/preview-approval.tsindex 6ba5bba4c1..7ce65392c9 100644--- a/src/core/persistence/preview-approval.ts+++ b/src/core/persistence/preview-approval.ts@@ -40,7 +40,7 @@ export const PREVIEW_APPROVAL_OP = 'repair-approval';/** The default `purgeStaleCheckpoints` TTL; an older approved set may already be gone. */export const APPROVAL_MAX_AGE_DAYS = 7;-export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'extractor-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'provenance' | 'secret-spans' | 'rows-behind';+export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'extractor-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'contact-rows' | 'provenance' | 'secret-spans' | 'rows-behind';export interface ApprovedSetKey {command: PreviewApprovalCommand;diff --git a/src/core/persistence/reconcile-additive.ts b/src/core/persistence/reconcile-additive.tsindex 1437244710..dd81c22a37 100644--- a/src/core/persistence/reconcile-additive.ts+++ b/src/core/persistence/reconcile-additive.ts@@ -3,28 +3,28 @@ import { FACTS_FENCE_BEGIN, FACTS_FENCE_END } from '../facts-fence.ts';import { TAKES_FENCE_BEGIN, TAKES_FENCE_END } from '../takes-fence.ts';import { OperationError } from '../ops/contract.ts';import { digest } from './digest.ts';+import { ACTIVITY_DATE_KEY as ACTIVITY_DATE } from '../contact-rows.ts';import { protectedReconcileKey, type ReconcileConflict, type ReconcileDecision } from './reconcile-merge.ts';/*** #5974: structural classification of file/database drift. It proves only* structure, never truth: an inserted line can still contradict an older one,* so inserted text is `suggested` (an explicit operator acceptance applies it)- * while only append-only contact lists and allowlisted activity dates are- * `auto`. Everything else is `review`. Pure; no I/O.+ * while only append-only contact lists (whose existing rows may advance an+ * activity date, #8109) and allowlisted activity dates are `auto`. Everything else is `review`. Pure; no I/O.*/export const ADDITIVE_RULE_VERSION = 1;-export type AdditiveRule = 'text_insertion_only' | 'contacts_append_only' | 'activity_date_advance' | 'file_only_field' | 'database_only_kept';+export type AdditiveRule = 'text_insertion_only' | 'contacts_append_only' | 'contacts_activity_advance' | 'activity_date_advance' | 'file_only_field' | 'database_only_kept';export type DriftClass = 'auto' | 'suggested' | 'review';-export interface DriftPath { path: string; class: DriftClass; rule?: AdditiveRule; reason: string; inserted_lines?: number; appended?: number }+export interface DriftPath { path: string; class: DriftClass; rule?: AdditiveRule; reason: string; inserted_lines?: number; appended?: number; advanced?: number }export interface DriftClassification { paths: DriftPath[]; verdict: 'structurally_additive' | 'additive_with_suggestions' | 'review_required' | 'no_drift' }export interface AutoDecision { path: string; rule: AdditiveRule; rule_version: number; evidence_digest: string }const APPEND_ONLY_ARRAYS = new Set(['contacts']);-const ACTIVITY_DATE = /^(?:updated|updated_at|modified|last_modified|last_[a-z0-9_]+|[a-z0-9_]+_last_used)$/;// Policy-bearing metadata a structural rule must never decide, beyond the protected merge keys.-const POLICY_KEYS = new Set(['access', 'confidentiality', 'grants', 'grant', 'owner', 'owners', 'permissions', 'acl', 'sharing',+export const POLICY_KEYS = new Set(['access', 'confidentiality', 'grants', 'grant', 'owner', 'owners', 'permissions', 'acl', 'sharing','private', 'visibility', 'status', 'aliases', 'slug', 'id', 'type', 'title', 'tags', 'withdrawn', 'withdrawals', 'expires', 'expires_at',- 'valid_until', 'archived', 'archived_at', 'deleted', 'deleted_at', 'consent', 'consented_at']);+ 'valid_until', 'archived', 'archived_at', 'deleted', 'deleted_at', 'consent', 'consented_at', 'validate']);const MAX_APPENDED = 500;const DAY_MS = 86_400_000;@@ -67,7 +67,10 @@ export function classifyTextInsertion(file: string, database: string): { ok: trureturn { ok: true, inserted };}-function classifyDate(file: unknown, database: unknown, now: number): { ok: true } | { ok: false; reason: string } {+function classifyDate(fileValue: unknown, databaseValue: unknown, now: number): { ok: true } | { ok: false; reason: string } {+ // An unquoted YAML date parses to a Date; the database stores the same instant as its ISO string.+ const text = (value: unknown) => value instanceof Date && Number.isFinite(value.getTime()) ? value.toISOString() : value;+ const file = text(fileValue), database = text(databaseValue);if (typeof file !== 'string' || typeof database !== 'string') return { ok: false, reason: 'date_not_text' };const dateOnly = /^\d{4}-\d{2}-\d{2}$/, stamp = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d+)?)?(?:Z|[+-]\d{2}:\d{2})$/;const format = (v: string) => dateOnly.test(v) ? 'date' : stamp.test(v) ? 'timestamp' : null;@@ -80,12 +83,27 @@ function classifyDate(file: unknown, database: unknown, now: number): { ok: truereturn { ok: true };}-function classifyAppend(file: unknown, database: unknown): { ok: true; appended: number } | { ok: false; reason: string } {+/** #8109: the same row with only activity dates (never a policy key) moved forward under classifyDate's rules. */+function activityOnlyAdvance(file: unknown, database: unknown, now: number): boolean {+ const row = (value: unknown): value is Record<string, unknown> => value !== null && typeof value === 'object' && !Array.isArray(value) && !(value instanceof Date);+ if (!row(file) || !row(database)) return false;+ const keys = Object.keys(file);+ if (keys.length !== Object.keys(database).length || !keys.every(key => Object.hasOwn(database, key))) return false;+ return keys.every(key => digest(file[key]) === digest(database[key])+ || (ACTIVITY_DATE.test(key) && !POLICY_KEYS.has(key) && !protectedReconcileKey(key) && classifyDate(file[key], database[key], now).ok));+}++function classifyAppend(file: unknown, database: unknown, now: number): { ok: true; appended: number; advanced: number } | { ok: false; reason: string } {if (!Array.isArray(file) || !Array.isArray(database)) return { ok: false, reason: 'not_a_list' };if (file.length < database.length) return { ok: false, reason: 'list_entries_removed' };- for (let k = 0; k < database.length; k++) if (digest(file[k]) !== digest(database[k])) return { ok: false, reason: 'list_entries_changed_or_reordered' };+ let advanced = 0;+ for (let k = 0; k < database.length; k++) {+ if (digest(file[k]) === digest(database[k])) continue;+ if (!activityOnlyAdvance(file[k], database[k], now)) return { ok: false, reason: 'list_entries_changed_or_reordered' };+ advanced++;+ }if (file.length - database.length > MAX_APPENDED) return { ok: false, reason: 'too_many_appended' };- return { ok: true, appended: file.length - database.length };+ return { ok: true, appended: file.length - database.length, advanced };}const absent = Symbol('absent');@@ -113,9 +131,10 @@ export function classifyDrift(file: ParsedPage, database: ParsedPage, now = Dateif (f === absent) { paths.push({ path, class: 'auto', rule: 'database_only_kept', reason: 'database_value_kept' }); continue; }if (d === absent) { paths.push({ path, class: 'auto', rule: 'file_only_field', reason: 'file_field_added' }); continue; }if (APPEND_ONLY_ARRAYS.has(key)) {- const append = classifyAppend(f, d);- paths.push(append.ok ? { path, class: 'auto', rule: 'contacts_append_only', reason: 'entries_appended', appended: append.appended }- : { path, class: 'review', reason: append.reason });+ const append = classifyAppend(f, d, now);+ paths.push(!append.ok ? { path, class: 'review', reason: append.reason }+ : append.advanced ? { path, class: 'auto', rule: 'contacts_activity_advance', reason: 'activity_dates_advanced', appended: append.appended, advanced: append.advanced }+ : { path, class: 'auto', rule: 'contacts_append_only', reason: 'entries_appended', appended: append.appended });continue;}if (ACTIVITY_DATE.test(key)) {diff --git a/src/core/repair/contact-rows.ts b/src/core/repair/contact-rows.tsnew file mode 100644index 0000000000..837e0ab1aa--- /dev/null+++ b/src/core/repair/contact-rows.ts@@ -0,0 +1,159 @@+/**+ * `gbrain repair contact-rows` (#8109): collapse duplicated frontmatter contact+ * rows. A writer that appended a `{type, value, last_used}` row on every touch+ * instead of updating the existing one left the same value many times; the+ * repair keeps one row per (type, value) with the newest `last_used`+ * (`dedupeContactRows`, contact-rows.ts), and never drops a distinct value.+ *+ * Explicit-only and preview-bound: the preview lists each page with its+ * revision and row counts under a hash and writes nothing but the approval+ * checkpoint; `--apply --expect <hash> --yes` publishes exactly that set, one+ * revision-bound `put_page` per page with a request id derived from the hash,+ * so a source with a checkout commits the canonical file through the+ * coordinated writer and a database-only (repo-less) source writes the+ * database. Only the `contacts` frontmatter key changes. A page whose canonical+ * file differs from the database is listed `needs_reconcile` with the+ * `--auto-additive` reconcile command (a `last_used`-only drift is+ * structurally additive), never written. Segmented pages are left alone. It+ * never runs from `--all`, the remediation plan, maintenance or an upgrade.+ */+import type { BrainEngine, PageSnapshot } from '../engine.ts';+import { OperationError } from '../ops/contract.ts';+import { parseMarkdown, serializePageToMarkdown } from '../markdown.ts';+import { dedupeFrontmatterContacts } from '../contact-rows.ts';+import { digest } from '../persistence/digest.ts';+import { liveFileDrift } from '../persistence/drift-conflicts.ts';+import { submitPageMutation } from '../persistence/page-mutations.ts';+import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';+import { afterCursor, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairListing, type RepairPlan, type RepairScope } from './core.ts';++const BATCH = 200;+const DOCS = 'docs/guides/repair.md#contact-rows';++interface Selection { source_ids: string[]; slugs: string[] }+interface PlannedPage { source_id: string; slug: string; id: number; revision: string; rows_before: number; rows_after: number }+interface RowsItem extends RepairItem { page: PlannedPage; hash: string; last: boolean }++function previewCommand(scope: RepairScope, slugs: string[]): string {+ return `gbrain repair contact-rows${scope.source_ids.length === 1 ? ` --source ${scope.source_ids[0]}` : ''}${slugs.map(slug => ` --slug ${slug}`).join('')}`;+}++const hashOf = async (engine: BrainEngine, scope: RepairScope, selection: Selection, pages: PlannedPage[]) => previewHash({ kind: 'contact-rows-v1', brain_id: scope.brain_id,+ sources: await engine.executeRaw<{ id: string; incarnation: string }>('SELECT id, incarnation::text AS incarnation FROM sources WHERE id=ANY($1::text[]) ORDER BY id', [scope.source_ids]),+ selection, pages });++/** One request id per page of an approved set: a resumed apply replays it, and the plan knows which pages already published. */+function requestIdOf(hash: string, page: Pick<PlannedPage, 'source_id' | 'slug' | 'revision'>): string {+ const h = digest(['contact-rows-request-v1', hash, page.source_id, page.slug, page.revision]);+ return `${h.slice(0, 8)}-${h.slice(8, 12)}-4${h.slice(13, 16)}-a${h.slice(17, 20)}-${h.slice(20, 32)}`;+}++/** The page's repaired file, or null when its contact rows need no change or the rendered page would not read back with exactly those rows. */+function plannedChange(snapshot: PageSnapshot): { content: string; rows_before: number; rows_after: number } | null {+ const { page, tags } = snapshot;+ const deduped = dedupeFrontmatterContacts(page.frontmatter);+ if (!deduped) return null;+ const content = serializePageToMarkdown({ ...page, frontmatter: deduped.frontmatter }, tags);+ const reread = parseMarkdown(content, `${page.slug}.md`);+ const { contacts: _after, ...restAfter } = reread.frontmatter;+ const { contacts: _before, ...restBefore } = page.frontmatter;+ if (digest(reread.frontmatter.contacts) !== digest(deduped.dedupe.rows) || digest(restAfter) !== digest(restBefore)+ || reread.compiled_truth.trim() !== page.compiled_truth.trim() || (reread.timeline ?? '').trim() !== (page.timeline ?? '').trim()) return null;+ return { content, rows_before: (page.frontmatter.contacts as unknown[]).length, rows_after: deduped.dedupe.rows.length };+}++const reconcileCommand = (page: Pick<PlannedPage, 'source_id' | 'slug'>) => `gbrain sources reconcile ${page.source_id} ${page.slug} --brain host --preview --auto-additive`;++/** Every inline page of the selection whose contact rows the repair would collapse, in id order. Read-only. */+async function scan(engine: BrainEngine, scope: RepairScope, selection: Selection) {+ const planned: PlannedPage[] = [];+ const listing: RepairListing[] = [];+ const residuals: Record<string, number> = { pages_to_dedupe: 0, rows_to_remove: 0, needs_reconcile: 0 };+ for (let cursor = 0; ;) {+ const batch = await engine.executeRaw<{ id: number; source_id: string; slug: string }>(+ `SELECT id, source_id, slug FROM pages+ WHERE source_id=ANY($1::text[]) AND deleted_at IS NULL AND body_storage='inline' AND id>$2+ AND jsonb_typeof(frontmatter->'contacts')='array' AND jsonb_array_length(frontmatter->'contacts')>1+ AND (cardinality($3::text[])=0 OR slug=ANY($3::text[]))+ ORDER BY id LIMIT ${BATCH}`, [scope.source_ids, cursor, selection.slugs]);+ for (const row of batch) {+ cursor = Number(row.id);+ const snapshot = await engine.readPageSnapshot(row.slug, { sourceId: row.source_id });+ const change = snapshot && plannedChange(snapshot);+ if (!snapshot || !change) continue;+ const page = { source_id: row.source_id, slug: row.slug, id: Number(row.id), revision: snapshot.revision, rows_before: change.rows_before, rows_after: change.rows_after };+ const label = `${row.source_id}:${row.slug} (revision ${snapshot.revision})`;+ if ((await liveFileDrift(engine, row.source_id, row.slug, snapshot)).still_present === true) {+ residuals.needs_reconcile!++;+ listing.push({ item: label, class: 'needs_reconcile', detail: `its canonical file differs from the database; preview ${reconcileCommand(page)}, apply it, then preview this repair again` });+ continue;+ }+ planned.push(page);+ residuals.pages_to_dedupe!++;+ residuals.rows_to_remove! += page.rows_before - page.rows_after;+ listing.push({ item: label, class: 'dedupe', detail: `${page.rows_before} contact rows become ${page.rows_after}` });+ }+ if (batch.length < BATCH) return { planned, listing, residuals };+ }+}++function item(page: PlannedPage, hash: string, last: boolean): RowsItem {+ return { cursor: { phase: 0, id: page.id }, source_id: page.source_id, slug: page.slug, chars: 0, action: `collapses ${page.rows_before} contact rows to ${page.rows_after}`, page, hash, last };+}++export const contactRowsRepair: RepairHandler = {+ kind: 'contact-rows',+ async plan(engine, scope, after, opts): Promise<RepairPlan> {+ const selection: Selection = { source_ids: scope.source_ids, slugs: [...new Set(opts?.slugs ?? [])].sort() };+ const command = previewCommand(scope, selection.slugs);+ if (!opts?.apply) {+ const { planned, listing, residuals } = await scan(engine, scope, selection);+ const hash = await hashOf(engine, scope, selection, planned);+ if (planned.length) await saveApprovedSet<PlannedPage & { selection: Selection }>(engine, { command: 'contact-rows', hash }, planned.map(page => ({ ...page, selection })));+ return { items: planned.map((page, index) => item(page, hash, index === planned.length - 1)), preview_hash: hash, residuals, listing };+ }+ if (!opts.expect) {+ throw new OperationError('invalid_params', 'gbrain repair contact-rows --apply rewrites only the set a preview printed.',+ `Preview first: ${command} — show the user the listing, then run the apply command it prints: ${command} --apply --expect <preview-hash> --yes`, DOCS);+ }+ const expect = opts.expect;+ const approved = (await loadApprovedSet<PlannedPage & { selection: Selection }>(engine, { command: 'contact-rows', hash: expect, previewCommand: command,+ emptyHash: () => hashOf(engine, scope, selection, []) })).items;+ if (approved.some(page => digest(page.selection) !== digest(selection))) throw previewChangedError(expect, command);+ const pages = approved.map(({ selection: _selection, ...page }) => page);+ const published = new Set((await engine.executeRaw<{ request_id: string }>(+ "SELECT request_id::text AS request_id FROM persistence_requests WHERE request_id=ANY($1::uuid[]) AND state='committed'",+ [pages.map(page => requestIdOf(expect, page))])).map(row => row.request_id));+ // Every page not yet published under this hash still has the revision the user saw, or nothing is written.+ for (const page of pages) {+ if (published.has(requestIdOf(expect, page))) continue;+ const snapshot = await engine.readPageSnapshot(page.slug, { sourceId: page.source_id });+ if (snapshot?.revision !== page.revision || Number(snapshot.page.id) !== page.id) throw previewChangedError(expect, command);+ }+ return { items: pages.map((page, index) => item(page, expect, index === pages.length - 1)).filter(entry => afterCursor(entry.cursor, after)),+ preview_hash: expect, residuals: {} };+ },+ async apply(ctx, entry): Promise<RepairItemOutcome> {+ const { page, hash, last } = entry as RowsItem;+ try {+ const requestId = requestIdOf(hash, page);+ const detail = { rows_before: page.rows_before, rows_after: page.rows_after };+ const [done] = await ctx.engine.executeRaw<{ state: string }>('SELECT state FROM persistence_requests WHERE request_id=$1::uuid', [requestId]);+ if (done?.state === 'committed') return { applied: true, outcome: 'deduped', detail };+ const snapshot = await ctx.engine.readPageSnapshot(page.slug, { sourceId: page.source_id });+ if (!snapshot || snapshot.revision !== page.revision) return { applied: false, outcome: 'changed_since_preview', reason: 'the page changed since the preview' };+ const change = plannedChange(snapshot);+ if (!change) return { applied: false, outcome: 'changed_since_preview', reason: 'the contact rows need no change' };+ try {+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: page.slug, source_id: page.source_id, content: change.content,+ expected_revision: snapshot.revision, request_id: requestId } });+ } catch (error) {+ if (!(error instanceof OperationError) || error.code !== 'source_changed' || error.detail !== 'file_database_drift') throw error;+ return { applied: false, outcome: 'needs_reconcile', reason: `the canonical file differs from the database; preview ${reconcileCommand(page)}, apply it, then preview this repair again` };+ }+ return { applied: true, outcome: 'deduped', detail };+ } finally {+ if (last) await clearApprovedSet(ctx.engine, { command: 'contact-rows', hash });+ }+ },+};diff --git a/src/core/repair/core.ts b/src/core/repair/core.tsindex 267e14a3fd..1b43ca57fb 100644--- a/src/core/repair/core.ts+++ b/src/core/repair/core.ts@@ -51,7 +51,7 @@ import { classifyItemFailure, ItemRun, type ItemFailure } from '../item-failure.import type { ProgressReporter } from '../progress.ts';import type { GenerationsSelection } from '../generations/repair.ts';-export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'] as const;+export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'] as const;export type RepairKind = typeof REPAIR_KINDS[number];export interface RepairScope { brain_id: string; source_ids: string[] }diff --git a/src/core/repair/registry.ts b/src/core/repair/registry.tsindex 334008b9e0..3657b1721b 100644--- a/src/core/repair/registry.ts+++ b/src/core/repair/registry.ts@@ -37,6 +37,7 @@ import { REPAIR_KINDS, runRepair, type RepairHandler, type RepairKind, type Repaimport { timelineRepair } from './timeline.ts';import { timelineCommentsRepair } from './timeline-comments.ts';import { timelineOrderRepair } from './timeline-order.ts';+import { contactRowsRepair } from './contact-rows.ts';import { visibilityRepair } from './visibility.ts';import { safeChunksRepair } from './safe-chunks.ts';import { contextualModeRepair } from './contextual-mode.ts';@@ -363,6 +364,20 @@ const SPECS: Record<RepairKind, Omit<RepairKindSpec, 'kind'>> = {+ 'page with a line the parser cannot place, or that would not read back with the same text and timeline rows, is listed needs_human. --slug '+ '<slug> narrows. Preview-bound: --apply --expect <hash> --yes rewrites exactly the previewed pages through revision-bound put_page; runs only when named.',},+ 'contact-rows': {+ handler: contactRowsRepair, embeds: 'effect', checks: [], explicit_only: true, preview_bound: true, consent: 'destructive',+ consentText: {+ what: 'Rewrite the previewed pages so each frontmatter contacts list keeps one row per (type, value)',+ why: 'The preview listed each page whose contacts list repeats a (type, value) row, with its row counts and the page revision it read.',+ risk: 'Only the contacts key changes: repeated rows collapse into the first, keeping the newest last_used; rows that differ in any other field and every distinct value stay. A page that changed since the preview refuses the apply (preview_changed). Managed sources commit the file through the Git effect; undo with git revert or gbrain revert.',+ user_message: 'Collapse the previewed duplicated contact rows? One row per (type, value) is kept, with the newest last_used',+ },+ summary: 'Collapse duplicated frontmatter contact rows (#8109): a writer that appended a {type, value, last_used} row on every touch left the same value '+ + 'many times. Keeps one row per (type, value), in first-seen order, with the newest last_used; a row that differs in another field is kept, and no '+ + 'distinct value is dropped. A page whose canonical file differs from the database is listed needs_reconcile with the --auto-additive reconcile '+ + 'command. --slug <slug> narrows. Preview-bound: --apply --expect <hash> --yes rewrites exactly the previewed pages through revision-bound put_page '+ + '(the canonical file on sources with a checkout, the database on repo-less ones); runs only when named.',+ },fences: {handler: fencesRepair, embeds: 'effect', checks: ['fence_integrity'], preview_bound: true, spends: 'llm',summary: 'Repair malformed facts and takes fences (#6188) that sync held or that pages store: per file or page the free tiers first (the lossless 'diff --git a/test/contact-rows-drift-8109.test.ts b/test/contact-rows-drift-8109.test.tsnew file mode 100644index 0000000000..85dfcc74c1--- /dev/null+++ b/test/contact-rows-drift-8109.test.ts@@ -0,0 +1,154 @@+/**+ * #8109: a person page whose frontmatter contact rows drifted (one row's+ * last_used moved forward in the file) no longer blocks an unrelated remember,+ * and the drift reconciles under --auto-additive as structurally additive.+ * Body, title and policy-field drift keep refusing. PGLite, and an isolated+ * Postgres database when DATABASE_URL is set.+ */+import { afterAll, beforeAll, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import type { BrainEngine } from '../src/core/engine.ts';+import { importFromContent } from '../src/core/import-file.ts';+import { parseMarkdown, serializePageToMarkdown } from '../src/core/markdown.ts';+import { parseFactsFence } from '../src/core/facts-fence.ts';+import { claimWorktree } from '../src/core/persistence/ownership.ts';+import { registerLocalWriter, withVerifiedLocalRegistration } from '../src/core/persistence/identity.ts';+import { runReconcileApply, runReconcilePreview } from '../src/core/persistence/reconcile.ts';+import { liveFileDrift } from '../src/core/persistence/drift-conflicts.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { operationsByName } from '../src/core/operations.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { withEnv } from './helpers/with-env.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';++const home = mkdtempSync(join(tmpdir(), 'gbrain-contact-drift-'));+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;+beforeAll(async () => {+ const engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); engines.push(engine);+ if (process.env.DATABASE_URL) { const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL); engines.push(pg.engine); closePostgres = pg.close; }+}, 120_000);+afterAll(async () => {+ await withEnv({ GBRAIN_HOME: home }, async () => { for (const engine of engines) { await disposePersistenceConsumer(engine); await engine.disconnect(); } });+ await closePostgres?.(); rmSync(home, { recursive: true, force: true });+});+const isolated = (fn: (engine: BrainEngine) => Promise<void>) => withEnv({ GBRAIN_HOME: home, OPENAI_API_KEY: undefined, ANTHROPIC_API_KEY: undefined }, async () => {+ for (const engine of engines) await fn(engine);+});++const SLUG = 'people/alice-example';+// The writer's shape: every touch appended a row, so the same phone is written three times. Unquoted YAML dates.+const STORED = `---+type: person+title: Alice Example+visibility: private+contacts:+ - type: phone+ value: '+15550100'+ last_used: 2026-10-01+ - type: email+ value: [email protected]+ last_used: 2026-09-01+ - type: phone+ value: '+15550100'+ last_used: 2026-10-02+ - type: phone+ value: '+15550100'+ last_used: 2026-10-03+---+Alice Example is a synthetic person.++## Timeline++- **2026-01-01** | meeting — Met about the plan.+`;+type Editable = { title: string; compiled_truth: string; timeline: string; frontmatter: Record<string, unknown> };++async function fixture(engine: BrainEngine, edit: (page: Editable) => void) {+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ const id = `contacts-${randomUUID().slice(0, 12)}`, root = join(home, id);+ mkdirSync(join(root, 'people'), { recursive: true });+ await engine.executeRaw('INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,$2,\'{}\')', [id, root]);+ await importFromContent(engine, SLUG, STORED, { sourceId: id, sourcePath: `${SLUG}.md`, noEmbed: true });+ const snapshot = (await engine.readPageSnapshot(SLUG, { sourceId: id }))!;+ const next: Editable = { title: snapshot.page.title, compiled_truth: snapshot.page.compiled_truth, timeline: snapshot.page.timeline ?? '',+ frontmatter: structuredClone(snapshot.page.frontmatter) };+ edit(next);+ writeFileSync(join(root, `${SLUG}.md`), serializePageToMarkdown({ ...snapshot.page, ...next }, snapshot.tags));+ await claimWorktree(engine, id, root);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ const registration = await registerLocalWriter(engine, 'cli');+ const ctx: OperationContext = { engine, remote: false, sourceId: id, config: { engine: engine.kind }, dryRun: false, logger: { info() {}, warn() {}, error() {} } };+ const local = <T>(fn: () => Promise<T>) => withVerifiedLocalRegistration(engine, registration, fn);+ return { id, root, file: join(root, `${SLUG}.md`), ctx, local, stored: snapshot };+}+const contactsOf = (frontmatter: Record<string, unknown>) => frontmatter.contacts as Array<Record<string, unknown>>;+const advance = (page: Editable) => { contactsOf(page.frontmatter)[2].last_used = '2026-10-30T00:00:00.000Z'; };+const remember = (ctx: OperationContext, fact: string) => operationsByName.remember!.handler(ctx,+ { fact, provenance: 'Synthetic operator conversation', entity: SLUG, visibility: 'private', request_id: randomUUID() }) as Promise<Record<string, unknown>>;++test('#8109: a last_used bump in the file does not block remember; the file keeps its frontmatter, and the drift reconciles as structurally additive', () => isolated(async engine => {+ const f = await fixture(engine, advance);+ await f.local(async () => {+ expect((await liveFileDrift(engine, f.id, SLUG)).reason).toBe('file_database_drift');+ const fact = 'Prefers the synthetic morning review.';+ const saved = await remember(f.ctx, fact);+ expect(saved).toMatchObject({ status: 'inserted', state: 'committed' });++ const db = (await engine.readPageSnapshot(SLUG, { sourceId: f.id }))!;+ expect(parseFactsFence(db.page.compiled_truth).facts).toEqual(expect.arrayContaining([expect.objectContaining({ claim: fact })]));+ // The database's frontmatter is untouched; the file keeps its advanced last_used and carries the same body.+ expect(db.page.frontmatter).toEqual(f.stored.page.frontmatter);+ const onDisk = parseMarkdown(readFileSync(f.file, 'utf8'), SLUG);+ expect(contactsOf(onDisk.frontmatter)[2].last_used).toBe('2026-10-30T00:00:00.000Z');+ expect(onDisk.compiled_truth).toBe(db.page.compiled_truth);+ expect(onDisk.timeline).toBe(db.page.timeline);+ // The frontmatter drift is still reported, for its own path.+ expect((await liveFileDrift(engine, f.id, SLUG)).reason).toBe('file_database_drift');++ // A second write lands the same way.+ expect(await remember(f.ctx, 'Reads synthetic newsletters.')).toMatchObject({ status: 'inserted', state: 'committed' });++ const ready = await runReconcilePreview(engine, { source_id: f.id, slug: SLUG, auto_additive: true });+ expect(ready).toMatchObject({ status: 'ready', classification: 'structurally_additive', auto_decided_paths: ['/frontmatter/contacts'] });+ expect(ready.preview.auto_decisions!.map(d => d.rule)).toEqual(['contacts_activity_advance']);+ expect((await runReconcileApply(engine, { source_id: f.id, slug: SLUG, preview: ready.preview, request_id: randomUUID() })).state).toBe('committed');+ const reconciled = (await engine.readPageSnapshot(SLUG, { sourceId: f.id }))!;+ expect(contactsOf(reconciled.page.frontmatter)[2].last_used).toBe('2026-10-30T00:00:00.000Z');+ expect(parseFactsFence(reconciled.page.compiled_truth).facts.map(row => row.claim)).toEqual(expect.arrayContaining([fact, 'Reads synthetic newsletters.']));+ expect((await liveFileDrift(engine, f.id, SLUG)).reason).toBe('agrees');+ });+}), 180_000);++for (const [name, edit] of [+ ['a policy field', (page: Editable) => { advance(page); page.frontmatter.visibility = 'public'; }],+ ['the body', (page: Editable) => { advance(page); page.compiled_truth += '\n\nA line added by hand.'; }],+ ['the title', (page: Editable) => { page.title = 'Alice Renamed'; page.frontmatter.title = 'Alice Renamed'; }],+] as const) {+ test(`#8109: drift in ${name} still refuses remember and leaves both copies as they were`, () => isolated(async engine => {+ const f = await fixture(engine, edit);+ const before = readFileSync(f.file, 'utf8');+ await f.local(async () => {+ await expect(remember(f.ctx, 'A fact that must wait.')).rejects.toMatchObject({ writeError: 'source_changed' });+ });+ expect(readFileSync(f.file, 'utf8')).toBe(before);+ expect((await engine.readPageSnapshot(SLUG, { sourceId: f.id }))!.revision).toBe(f.stored.revision);+ }), 180_000);+}++test('#8109: a write that changes frontmatter over frontmatter drift still refuses', () => isolated(async engine => {+ const f = await fixture(engine, advance);+ const before = readFileSync(f.file, 'utf8');+ await f.local(async () => {+ const content = serializePageToMarkdown({ ...f.stored.page, frontmatter: { ...f.stored.page.frontmatter, company: 'acme-example' } }, f.stored.tags);+ await expect(submitPageMutation({ ...f.ctx, sourceId: f.id }, { operation: 'put_page', params: { slug: SLUG, source_id: f.id, content,+ expected_revision: f.stored.revision, request_id: randomUUID() } })).rejects.toMatchObject({ code: 'source_changed' });+ });+ expect(readFileSync(f.file, 'utf8')).toBe(before);+}), 180_000);diff --git a/test/contact-rows.test.ts b/test/contact-rows.test.tsnew file mode 100644index 0000000000..5de1556bfb--- /dev/null+++ b/test/contact-rows.test.ts@@ -0,0 +1,54 @@+/**+ * #8109: the pure contact-row dedupe. One row per (type, value), newest+ * activity date kept; no distinct value is ever dropped.+ */+import { describe, expect, test } from 'bun:test';+import { dedupeContactRows, dedupeFrontmatterContacts } from '../src/core/contact-rows.ts';++const phone = (value: string, last_used: unknown, extra: Record<string, unknown> = {}) => ({ type: 'phone', value, last_used, ...extra });++describe('#8109 dedupeContactRows', () => {+ test('a value written three times collapses to one row with the newest last_used, in first-seen order', () => {+ const list = [phone('+15550100', '2026-10-01'), { type: 'email', value: '[email protected]', last_used: '2026-09-01' },+ phone('+15550100', '2026-10-30'), phone('+15550100', '2026-10-15'), { type: 'email', value: '[email protected]', last_used: '2026-09-02' }];+ expect(dedupeContactRows(list)).toEqual({+ rows: [phone('+15550100', '2026-10-30'), { type: 'email', value: '[email protected]', last_used: '2026-09-02' }], removed: 3, kept_conflicting: 0 });+ });++ test('keys compare type case-insensitively and trimmed; distinct spellings of a value stay distinct rows', () => {+ const list = [phone('+15550100', '2026-10-01'), { type: ' Phone ', value: '+15550100 ', last_used: '2026-10-02' }, phone('555-0100', '2026-10-03'), phone('+1 555 0100', '2026-10-04')];+ const out = dedupeContactRows(list)!;+ expect(out.removed).toBe(1);+ expect(out.rows.map(row => (row as { value: string }).value)).toEqual(['+15550100', '555-0100', '+1 555 0100']);+ expect((out.rows[0] as { last_used: string }).last_used).toBe('2026-10-02');+ });++ test('a field present on one row only is kept; Date objects and timestamps compare as instants', () => {+ const out = dedupeContactRows([phone('+15550100', new Date('2026-10-01T00:00:00Z')), phone('+15550100', new Date('2026-10-05T00:00:00Z'), { label: 'mobile' })])!;+ expect(out.rows).toEqual([{ type: 'phone', value: '+15550100', last_used: new Date('2026-10-05T00:00:00Z'), label: 'mobile' }]);+ });++ test('a conflicting non-date field keeps every distinct row and drops only exact copies', () => {+ const list = [phone('+15550100', '2026-10-01', { label: 'mobile' }), phone('+15550100', '2026-10-01', { label: 'work' }), phone('+15550100', '2026-10-01', { label: 'mobile' })];+ expect(dedupeContactRows(list)).toEqual({ rows: list.slice(0, 2), removed: 1, kept_conflicting: 1 });+ // An activity date that is not a date cannot be ordered: the rows are kept apart, exact copies still go.+ expect(dedupeContactRows([phone('+15550100', 'yesterday'), phone('+15550100', '2026-10-01'), phone('+15550100', 'yesterday')])!.rows).toHaveLength(2);+ });++ test('non-row entries, rows without a type or value, and lists with nothing to merge are left alone', () => {+ expect(dedupeContactRows(['alice-example', 'alice-example'])).toBeNull();+ expect(dedupeContactRows([{ value: '+15550100' }, { value: '+15550100' }])).toBeNull();+ expect(dedupeContactRows([phone('+15550100', '2026-10-01'), phone('+15550101', '2026-10-01')])).toBeNull();+ expect(dedupeContactRows('not a list')).toBeNull();+ const mixed = dedupeContactRows(['alice-example', phone('+15550100', '2026-10-01'), 'alice-example', phone('+15550100', '2026-10-02')])!;+ expect(mixed.rows).toEqual(['alice-example', phone('+15550100', '2026-10-02'), 'alice-example']);+ });++ test('dedupeFrontmatterContacts touches only the contacts key', () => {+ expect(dedupeFrontmatterContacts({ phones: ['+15550100', '+15550100'] })).toBeNull();+ expect(dedupeFrontmatterContacts(null)).toBeNull();+ const out = dedupeFrontmatterContacts({ company: 'acme-example', contacts: [phone('+15550100', '2026-10-01'), phone('+15550100', '2026-10-02')] })!;+ expect(out.frontmatter).toEqual({ company: 'acme-example', contacts: [phone('+15550100', '2026-10-02')] });+ expect(out.dedupe.removed).toBe(1);+ });+});diff --git a/test/e2e/repair-contact-rows-postgres.test.ts b/test/e2e/repair-contact-rows-postgres.test.tsnew file mode 100644index 0000000000..5b0096ec5f--- /dev/null+++ b/test/e2e/repair-contact-rows-postgres.test.ts@@ -0,0 +1,3 @@+import { registerPostgresTests } from '../helpers/test-backends.ts';++await registerPostgresTests(() => import('../repair-contact-rows.test.ts'));diff --git a/test/fix-wave-5-integration.test.ts b/test/fix-wave-5-integration.test.tsindex 3ee52db0a3..319d5ba56d 100644--- a/test/fix-wave-5-integration.test.ts+++ b/test/fix-wave-5-integration.test.ts@@ -195,7 +195,7 @@ for (const backend of testBackends()) {const untouched = async () => { expect(await liveAtoms()).toBe(30); expect(await activeFacts()).toBe(0); };const previews = { 'accepted-takes': 'gbrain repair accepted-takes', 'google-file-modes': 'gbrain repair google-file-modes', 'stale-atoms': 'gbrain repair stale-atoms', 'atoms-restore': 'gbrain repair atoms-restore', 'extractor-facts': 'gbrain repair extractor-facts','conversation-labels': 'gbrain repair conversation-labels', 'captured-facts': 'gbrain repair captured-facts', 'agent-session-facts': 'gbrain repair agent-session-facts', 'redacted-facts': 'gbrain repair redacted-facts', 'loop-facts': 'gbrain repair loop-facts', 'ontology-facts': 'gbrain repair ontology-facts', 'facts-rehome': 'gbrain repair facts-rehome', 'consolidate-ties': 'gbrain repair consolidate-ties', 'duplicate-facts': 'gbrain repair duplicate-facts', 'orphan-children': 'gbrain repair orphan-children', 'failed-writes': 'gbrain repair failed-writes', 'split-facts': 'gbrain repair split-facts',- frontmatter: 'gbrain repair frontmatter', 'timeline-comments': 'gbrain repair timeline-comments', 'timeline-order': 'gbrain repair timeline-order', 'managed-sync-orphans': 'gbrain repair managed-sync-orphans', 'content-receipt-root': 'gbrain repair content-receipt-root', 'frontmatter-keys': 'gbrain repair frontmatter-keys', 'file-drift': 'gbrain repair file-drift',+ frontmatter: 'gbrain repair frontmatter', 'timeline-comments': 'gbrain repair timeline-comments', 'timeline-order': 'gbrain repair timeline-order', 'contact-rows': 'gbrain repair contact-rows', 'managed-sync-orphans': 'gbrain repair managed-sync-orphans', 'content-receipt-root': 'gbrain repair content-receipt-root', 'frontmatter-keys': 'gbrain repair frontmatter-keys', 'file-drift': 'gbrain repair file-drift','stuck-recovery': 'gbrain repair stuck-recovery', 'ghost-pages': 'gbrain repair ghost-pages', 'origin-twins': 'gbrain repair origin-twins', 'receipt-compaction': 'gbrain repair receipt-compaction', aliases: 'gbrain repair aliases', 'rename-identity': 'gbrain repair rename-identity','overwritten-edits': 'gbrain repair overwritten-edits', 'legacy-twins': 'gbrain repair legacy-twins', 'missing-tables': 'gbrain repair missing-tables', 'url-clone-fences': 'gbrain repair url-clone-fences', 'segmented-mirror': 'gbrain repair segmented-mirror','uncommitted-deletions': 'gbrain repair uncommitted-deletions',diff --git a/test/persistence-reconcile-additive-5974.test.ts b/test/persistence-reconcile-additive-5974.test.tsindex 3a7427425d..c8219adc70 100644--- a/test/persistence-reconcile-additive-5974.test.ts+++ b/test/persistence-reconcile-additive-5974.test.ts@@ -81,6 +81,33 @@ describe('structural drift classifier', () => {expect(classifyDrift(page({ compiled_truth: 'Line one.\nNew.\nLine two.' }), page(), NOW).verdict).toBe('additive_with_suggestions');expect(classifyDrift(page({ frontmatter: { ...page().frontmatter, access: 'public' } }), page(), NOW).verdict).toBe('review_required');});++ test('#8109: a contact row whose only change is an activity date moving forward is auto; any other row change stays review', () => {+ const row = (value: string, last_used: unknown, extra: Record<string, unknown> = {}) => ({ type: 'phone', value, last_used, ...extra });+ const stored = page({ frontmatter: { contacts: [row('+15550100', '2026-10-01'), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')] } });+ const file = (contacts: unknown[]) => page({ frontmatter: { contacts } });+ const one = classifyDrift(file([row('+15550100', '2026-10-01'), row('+15550100', '2026-10-02'), row('+15550101', '2026-09-01')]), stored, NOW).paths;+ expect(one).toEqual([{ path: '/frontmatter/contacts', class: 'auto', rule: 'contacts_activity_advance', reason: 'activity_dates_advanced', appended: 0, advanced: 1 }]);+ expect(classifyDrift(file([row('+15550100', '2026-10-01'), row('+15550100', '2026-10-02'), row('+15550101', '2026-09-01')]), stored, NOW).verdict).toBe('structurally_additive');+ // Advanced and appended together is still additive.+ expect(classOf(file([row('+15550100', '2026-10-03'), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01'), row('+15550102', '2026-10-02')]), stored)['/frontmatter/contacts'])+ .toBe('auto:activity_dates_advanced');+ // Unquoted YAML dates parse to Date objects; they compare as the instants they are.+ expect(classOf(file([row('+15550100', new Date('2026-10-02T00:00:00Z')), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')]),+ page({ frontmatter: { contacts: [row('+15550100', new Date('2026-10-01T00:00:00Z')), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')] } }))['/frontmatter/contacts'])+ .toBe('auto:activity_dates_advanced');+ expect(classOf(page({ frontmatter: { last_used: new Date('2026-10-02T00:00:00Z') } }), page({ frontmatter: { last_used: '2026-10-01T00:00:00.000Z' } }))['/frontmatter/last_used'])+ .toBe('auto:activity_date_advanced');+ for (const [contacts, reason] of [+ [[row('+15550100', '2026-09-30'), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550100', '2030-01-01'), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550199', '2026-10-02'), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550100', '2026-10-02', { label: 'work' }), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550100', '2026-10-01', { access: 'public' }), row('+15550100', '2026-10-01'), row('+15550101', '2026-09-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550101', '2026-09-01'), row('+15550100', '2026-10-01'), row('+15550100', '2026-10-01')], 'list_entries_changed_or_reordered'],+ [[row('+15550100', '2026-10-02'), row('+15550101', '2026-09-01')], 'list_entries_removed'],+ ] as const) expect(classOf(file([...contacts]), stored)['/frontmatter/contacts']).toBe(`review:${reason}`);+ });});const home = mkdtempSync(join(tmpdir(), 'gbrain-reconcile-additive-'));diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 306d1894d5..eb04799314 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -44,6 +44,7 @@ test/company-brain-policy.test.tstest/company-brain-runtime.test.tstest/consolidate-fenced-close-reimport.test.tstest/consolidate-same-instant-6023.test.ts+test/contact-rows-drift-8109.test.tstest/conversation-facts-outcome-marker-purge-7778.test.tstest/conversation-facts-oversized-sidecar.test.tstest/conversation-facts-purged-claim-7438.test.tsdiff --git a/test/repair-contact-rows.test.ts b/test/repair-contact-rows.test.tsnew file mode 100644index 0000000000..34611d99a4--- /dev/null+++ b/test/repair-contact-rows.test.ts@@ -0,0 +1,128 @@+/**+ * #8109: `gbrain repair contact-rows` on real publication, PGLite and Postgres. A managed source (claimed worktree: the+ * canonical file is published through the coordinated writer) and a database-only (repo-less) source share one run.+ * The preview writes nothing, a moved preview refuses, the apply collapses exactly the planned pages without dropping a+ * distinct value, a page whose file drifted is listed needs_reconcile and left alone, and a second preview finds nothing.+ * put_page itself keeps one row per (type, value).+ */+import { describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { parseMarkdown } from '../src/core/markdown.ts';+import { importFromContent } from '../src/core/import-file.ts';+import { resolveRepairScope } from '../src/core/repair/core.ts';+import { AUTO_REPAIR_REGISTRY, repairRunner } from '../src/core/repair/registry.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { durableGitRepo } from './helpers/git-publication.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const logger = { info() {}, warn() {}, error() {} };+const gitRoot = async ({ engine, root }: { engine: BrainEngine; root: string }) => {+ writeFileSync(join(root, '.gitignore'), '\n'); durableGitRepo(root, ['.gitignore']);+ await engine.executeRaw("INSERT INTO sources(id,name) VALUES('source-a','source-a') ON CONFLICT DO NOTHING");+};+const row = (type: string, value: string, last_used: string, extra = '') => ` - type: ${type}\n value: '${value}'\n last_used: '${last_used}'${extra}`;+const doc = (title: string, rows: string[]) => `---\ntype: person\ntitle: ${title}\ncompany: acme-example\ncontacts:\n${rows.join('\n')}\n---\n\n${title} is a synthetic person.\n`;+const TRIPLED = [row('phone', '+15550100', '2026-10-01'), row('email', '[email protected]', '2026-09-01'), row('phone', '+15550100', '2026-10-03'),+ row('phone', '+15550100', '2026-10-02'), row('phone', '+1 555 0100', '2026-08-01')];+const CONFLICT = [row('phone', '+15550111', '2026-10-01', "\n label: 'mobile'"), row('phone', '+15550111', '2026-10-02', "\n label: 'work'"),+ row('phone', '+15550111', '2026-10-01', "\n label: 'mobile'")];++/**+ * Pages imported before the brain is managed, as sync imports a file another writer produced, so they keep their+ * duplicates: a default-source page also gets its file in the checkout.+ */+const seeded = (pages: Array<[sourceId: string, slug: string, content: string]>) => async ({ engine, root }: { engine: BrainEngine; root: string }) => {+ await gitRoot({ engine, root });+ for (const [sourceId, slug, content] of pages) {+ const managed = sourceId === 'default';+ if (managed) { mkdirSync(join(root, 'people'), { recursive: true }); writeFileSync(join(root, `${slug}.md`), content); }+ await importFromContent(engine, slug, content, { sourceId, ...(managed ? { sourcePath: `${slug}.md` } : {}), noEmbed: true });+ }+};+const contacts = async (engine: BrainEngine, sourceId: string, slug: string) => (await engine.readPageSnapshot(slug, { sourceId }))!.page.frontmatter.contacts as Array<Record<string, unknown>>;+const revision = async (engine: BrainEngine, sourceId: string, slug: string) => (await engine.readPageSnapshot(slug, { sourceId }))!.revision;++for (const backend of testBackends()) {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ describe(`#8109 gbrain repair contact-rows (${backend})`, () => {+ test('preview writes nothing; the apply collapses exactly the planned pages on a managed and a repo-less source; drift is left for reconcile', () => managedBrain(async ({ engine, ctx, root }) => {+ const file = (slug: string) => readFileSync(join(root, `${slug}.md`), 'utf8');+ // Dana's file moved one last_used forward outside gbrain: file/database drift.+ writeFileSync(join(root, 'people/dana-example.md'), file('people/dana-example').replace("'2026-10-03'", "'2026-10-30'"));+ const files = { alice: file('people/alice-example'), bob: file('people/bob-example'), dana: file('people/dana-example') };+ const revisions = { bob: await revision(engine, 'default', 'people/bob-example'), dana: await revision(engine, 'default', 'people/dana-example') };++ expect(AUTO_REPAIR_REGISTRY.map(spec => spec.kind)).not.toContain('contact-rows');+ const scope = await resolveRepairScope(engine);+ const preview = () => repairRunner(engine, { apply: false, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true }));+ const apply = (hash: string) => repairRunner(engine, { apply: true, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true, expect: hash }));+ await expect(repairRunner(engine, { apply: true, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true }))).rejects.toThrow(/preview/i);++ const first = await preview();+ expect(first.affected).toBe(3);+ expect(first.residuals).toEqual({ pages_to_dedupe: 3, rows_to_remove: 5, needs_reconcile: 1 });+ expect(first.listing!.map(entry => [entry.class, entry.item.split(' ')[0]])).toEqual([+ ['dedupe', 'default:people/alice-example'], ['dedupe', 'default:people/bob-example'], ['needs_reconcile', 'default:people/dana-example'], ['dedupe', 'source-a:people/carol-example']]);+ expect(first.listing!.find(entry => entry.class === 'needs_reconcile')!.detail).toContain('gbrain sources reconcile default people/dana-example --brain host --preview --auto-additive');+ expect({ alice: file('people/alice-example'), bob: file('people/bob-example'), dana: file('people/dana-example') }).toEqual(files);+ expect(await contacts(engine, 'source-a', 'people/carol-example')).toHaveLength(5);++ await expect(apply('f'.repeat(64))).rejects.toMatchObject({ code: 'preview_changed' });+ const applied = await apply(first.preview_hash!);+ expect(applied.applied).toBe(3);+ expect(applied.outcomes).toEqual({ deduped: 3 });++ const expected = [{ type: 'phone', value: '+15550100', last_used: '2026-10-03' }, { type: 'email', value: '[email protected]', last_used: '2026-09-01' },+ { type: 'phone', value: '+1 555 0100', last_used: '2026-08-01' }];+ expect(await contacts(engine, 'default', 'people/alice-example')).toEqual(expected);+ expect(await contacts(engine, 'source-a', 'people/carol-example')).toEqual(expected);+ // The managed page's canonical file carries the same rows; nothing else in the page changed.+ const alice = parseMarkdown(file('people/alice-example'), 'people/alice-example.md');+ expect(alice.frontmatter.contacts).toEqual(expected);+ expect(alice.frontmatter.company).toBe('acme-example');+ expect(alice.compiled_truth).toBe(parseMarkdown(files.alice, 'people/alice-example.md').compiled_truth);+ // Rows that differ in another field are all kept; only the exact copy went.+ expect((await contacts(engine, 'default', 'people/bob-example')).map(r => r.label)).toEqual(['mobile', 'work']);+ expect(await revision(engine, 'default', 'people/bob-example')).not.toBe(revisions.bob);+ // The drifted page was not written.+ expect(file('people/dana-example')).toBe(files.dana);+ expect(await revision(engine, 'default', 'people/dana-example')).toBe(revisions.dana);++ const second = await preview();+ expect(second.affected).toBe(0);+ expect(second.residuals).toEqual({ pages_to_dedupe: 0, rows_to_remove: 0, needs_reconcile: 1 });+ await expect(apply(first.preview_hash!)).rejects.toMatchObject({ code: 'preview_changed' });+ }, { databaseUrl, setup: seeded([['default', 'people/alice-example', doc('Alice Example', TRIPLED)], ['default', 'people/bob-example', doc('Bob Example', CONFLICT)],+ ['default', 'people/dana-example', doc('Dana Example', TRIPLED)], ['source-a', 'people/carol-example', doc('Carol Example', TRIPLED)]]) }), 180_000);++ test('a previewed page that moved refuses the apply; --slug narrows the preview', () => managedBrain(async ({ engine, ctx }) => {+ const scope = await resolveRepairScope(engine, 'source-a');+ const narrowed = await repairRunner(engine, { apply: false, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true, slugs: ['people/carol-example'] }));+ expect(narrowed.listing!.map(entry => entry.item.split(' ')[0])).toEqual(['source-a:people/carol-example']);+ const first = await repairRunner(engine, { apply: false, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true }));+ expect(first.affected).toBe(2);+ const snapshot = (await engine.readPageSnapshot('people/carol-example', { sourceId: 'source-a' }))!;+ await submitPageMutation({ ...ctx, sourceId: 'source-a' } as OperationContext, { operation: 'put_page', params: { slug: 'people/carol-example', source_id: 'source-a',+ request_id: randomUUID(), expected_revision: snapshot.revision, content: doc('Carol Example', TRIPLED).replace('is a synthetic person.', 'was edited after the preview.') } });+ await expect(repairRunner(engine, { apply: true, logger }).then(runner => runner.run('contact-rows', scope, { explicit: true, expect: first.preview_hash! })))+ .rejects.toMatchObject({ code: 'preview_changed' });+ expect(await contacts(engine, 'source-a', 'people/alice-example')).toHaveLength(5);+ }, { databaseUrl, setup: seeded([['source-a', 'people/alice-example', doc('Alice Example', TRIPLED)], ['source-a', 'people/carol-example', doc('Carol Example', TRIPLED)]]) }), 180_000);++ test('put_page keeps one contact row per (type, value), in the file and the database', () => managedBrain(async ({ engine, ctx, root }) => {+ for (const sourceId of ['default', 'source-a']) {+ await submitPageMutation({ ...ctx, sourceId } as OperationContext, { operation: 'put_page', params: { slug: 'people/alice-example', source_id: sourceId,+ request_id: randomUUID(), content: doc('Alice Example', TRIPLED) } });+ expect(await contacts(engine, sourceId, 'people/alice-example')).toEqual([{ type: 'phone', value: '+15550100', last_used: '2026-10-03' },+ { type: 'email', value: '[email protected]', last_used: '2026-09-01' }, { type: 'phone', value: '+1 555 0100', last_used: '2026-08-01' }]);+ }+ expect(parseMarkdown(readFileSync(join(root, 'people/alice-example.md'), 'utf8'), 'people/alice-example.md').frontmatter.contacts)+ .toEqual(await contacts(engine, 'default', 'people/alice-example'));+ }, { databaseUrl, setup: gitRoot }), 180_000);+ });+}diff --git a/test/repair-explicit-only.test.ts b/test/repair-explicit-only.test.tsindex 42c226789d..7e1907e5f5 100644--- a/test/repair-explicit-only.test.ts+++ b/test/repair-explicit-only.test.ts@@ -24,7 +24,7 @@ import { REPAIR_HELP, parseRepairArgs, runRepairCommand } from '../src/commands/import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';import { withEnv } from './helpers/with-env.ts';-const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'];+const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'];// GBRA-161: the generations preview carries its ledger kind and the restore window, so the printed command runs as is.const PREVIEW = (kind: RepairKind): string => kind === 'generations' ? 'gbrain repair generations --kind atom --since 30d' : `gbrain repair ${kind}`;let engine: PGLiteEngine;