Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(persistence): the frontmatter-drift splice refuses reader-list, lineage and trust-marker drift (#8109)

garrytan31d685571a27
diff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.md
index a88d35a006..d34b0ebef2 100644
--- a/docs/guides/write-refusals.md
+++ b/docs/guides/write-refusals.md
@@ -51,7 +51,7 @@ of making a duplicate.
| Reason | Error code | What it means | Recovery |
| --- | --- | --- | --- |
-| <a id="file_database_drift"></a>`file_database_drift` | `source_changed` | The page's canonical file and its database copy disagree, usually because the file was edited outside a coordinated write. Neither copy was overwritten. A file with no `type:` line keeps the stored type, and titles compare trimmed, so those alone do not cause this. Drift only in ordinary frontmatter keys (not a policy or privacy key, title, type or tags) does not refuse a write that leaves the frontmatter unchanged, such as `remember`: the write lands and the file keeps its own frontmatter, so that drift is left for reconcile (#8109). | The receipt's fix (`gbrain sources writer failures <source>`, `gbrain write-request <request_id> --json`) previews keeping the database copy for that one page with `gbrain repair file-drift --source <source> --slug <slug> --prefer database` (`--prefer file` previews importing the file instead); a person picks the side, then the agent runs the apply command the preview prints. A group of such receipts gets `--all-drifted --since <instant>` with the oldest receipt's time, so only pages that drifted since then are selected. For a hand merge: `gbrain sources reconcile <source> <slug> --brain <brain> --preview`, review the preview, resolve it, then `--apply <resolved-preview-file> --request-id <new uuid>`. Retry the original write with a **new** request ID. A page that records no file of its own (no `source_path`, typically written by an older release whose write-through also put `ingested_via` / `ingested_at` / `source_kind` in the file only) reconciles against the file at its slug path only when you add `--adopt-slug-path` to the preview; the refusal names the flag in that case, and apply records that file as the page's origin. See [repair a file/database disagreement](concurrent-writes.md#repair-a-filedatabase-disagreement). Many pages at once: `gbrain sources reconcile <source> --brain host --audit --conflicts` lists them and `gbrain repair file-drift` resolves them, preferring the file or the database copy ([file drift](repair.md#file-drift)). |
+| <a id="file_database_drift"></a>`file_database_drift` | `source_changed` | The page's canonical file and its database copy disagree, usually because the file was edited outside a coordinated write. Neither copy was overwritten. A file with no `type:` line keeps the stored type, and titles compare trimmed, so those alone do not cause this. Drift only in ordinary frontmatter keys (not a policy, privacy, lineage or trust key such as `readable_by`, `derived_from` or `trust_tier`, nor title, type or tags) does not refuse a write that leaves the frontmatter unchanged, such as `remember`: the write lands and the file keeps its own frontmatter, so that drift is left for reconcile (#8109). | The receipt's fix (`gbrain sources writer failures <source>`, `gbrain write-request <request_id> --json`) previews keeping the database copy for that one page with `gbrain repair file-drift --source <source> --slug <slug> --prefer database` (`--prefer file` previews importing the file instead); a person picks the side, then the agent runs the apply command the preview prints. A group of such receipts gets `--all-drifted --since <instant>` with the oldest receipt's time, so only pages that drifted since then are selected. For a hand merge: `gbrain sources reconcile <source> <slug> --brain <brain> --preview`, review the preview, resolve it, then `--apply <resolved-preview-file> --request-id <new uuid>`. Retry the original write with a **new** request ID. A page that records no file of its own (no `source_path`, typically written by an older release whose write-through also put `ingested_via` / `ingested_at` / `source_kind` in the file only) reconciles against the file at its slug path only when you add `--adopt-slug-path` to the preview; the refusal names the flag in that case, and apply records that file as the page's origin. See [repair a file/database disagreement](concurrent-writes.md#repair-a-filedatabase-disagreement). Many pages at once: `gbrain sources reconcile <source> --brain host --audit --conflicts` lists them and `gbrain repair file-drift` resolves them, preferring the file or the database copy ([file drift](repair.md#file-drift)). |
| <a id="canonical_file_missing"></a>`canonical_file_missing` | `source_changed` (a caller write; maintenance refuses before admission) | The page's recorded canonical file is gone while its database page is still live (a ghost page: the file was moved or deleted outside gbrain). A caller write refuses with "The canonical file was removed outside coordinated publication." and nothing is written; retrying it refuses the same way. A maintenance write (fact-fence adoption, enrichment, `dream`) is refused before admission: no request id, no receipt (before #6696 each cycle settled a `conflict` receipt). `gbrain sources writer failures <source>` reports such receipts as category `canonical_file_missing`. Neither copy was changed. | Decide which copy is right. If the file should exist, restore it at its recorded path and preview the page with `gbrain sources reconcile <source> <slug> --brain host --preview`, or sync. If the page should go, preview the ghost-page repair with `gbrain repair ghost-pages --source <source>` (read-only; it prints the apply command with its hash) and apply it with `--apply --expect <hash> --yes` only after the user agrees; a soft-deleted page can be restored with `gbrain restore <slug> --force` until the autopilot purge 72 h later. `gbrain pages ghosts --source <source>` and doctor `ghost_pages` list them, doctor `maintenance_write_drift` lists held-back maintenance writes. See [ghost pages](repair.md#ghost-pages). |
| <a id="refusal_breaker"></a>`refusal_breaker` | the code of the earlier refusals | The same maintenance write (same source, slug, page revision, planner and intent) was refused `persistence.maintenance.refusal_breaker_threshold` times (default 3; `0` turns it off) in the last 24 hours, so maintenance submits nothing for it instead of adding another receipt (#6696). It is tried again once the page, the planner or the write changes, or when the oldest refusal is 24 hours old. | Read the last refusal with `gbrain write-request -- <request_id>` (named in the message) and fix what it names. Doctor `maintenance_write_drift` lists every held-back write with its exit. |
| <a id="canonical_file_absent"></a>`canonical_file_absent` | `source_changed` (`restore_page` / `gbrain restore`) | On a brain without managed persistence, a restore that writes no file (`sync.write_through` off, a sandbox, a database-only source) of a soft-deleted page whose recorded file is no longer among the files sync imports (#6609). A plain restore would be undone by the next `gbrain sync --full`, so nothing was restored. Not retryable; the fix asks the user. | Ask the user. To keep the page with its file: put the file back at its recorded path, commit it and `gbrain sync --source <source>`. To keep it database-only: `gbrain restore <slug> --source-id <source> --detach` (MCP `restore_page` with `detach: true`), which forgets the recorded file and stamps `database_only_reason='detached'`. See [restoring a page whose file is gone](../storage-tiering.md#restoring-a-page-whose-file-is-gone). |
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index cd8a7ab3c5..3be9e7c63e 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1377,7 +1377,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.525.0', id: 'rows_behind_checkpoint_8071', text: 'New doctor check `sync_rows_behind_checkpoint` and explicit-only repair kind `gbrain repair rows-behind --source <id>` (#8071): on each managed source\'s owner host, doctor compares every live Markdown page with its file read from Git at the sync checkpoint (`sources.last_commit`) and warns about rows that hold an older version with no hold, which no later sync re-imports because the import record names the current file. The repair previews each row with its path, checkpoint blob and action and writes nothing; `--apply --expect <hash>` re-imports the committed file through the managed import (no file is written), or opens the hold sync would have opened when that import is refused or the file is over the read bound. Rows with no import record join only with `--include-ambiguous`. Bounded by `GBRAIN_DOCTOR_ROWS_BEHIND_TIMEOUT_MS` (default 10 s); not applicable on repo-less brains and off the owner host. It never runs on upgrade or automatically.' },
{ since: '0.60.525.0', id: 'derived_write_through_commits_8073', text: 'Files the dream cycle writes into a source that is a Git checkout are now committed: atom and concept files (with `cycle.extract_atoms.write_through` or `cycle.synthesize_concepts.write_through` on) and the reverse-written synthesis, patterns and cycle-summary pages and their removals. Each batch (an item\'s atoms plus the files of the atoms it retired, the backfill, one concept, or one phase\'s reverse-written pages) is one local commit naming only those files, made through the checkout\'s commit lock (a hardened checkout\'s post-commit hook pushes it; a checkout with a canonical owner marker gets a named-paths-only commit). They used to stay untracked or modified, which made the scheduled sync skip every run. A commit the checkout refuses rolls back the files it wrote and reports `write_through_failed` with a warn status; atoms and concepts are written again on the next run. Each written file is also recorded in the pending write-through ledger, so files a crash left uncommitted go into the next commit gbrain makes in that checkout. Files left uncommitted by earlier versions are not touched; commit them with `gbrain git commit-push -m <msg> -- <paths>`.' },
{ since: '0.60.525.0', id: 'oversize_hold_unread_8072', text: 'A Markdown file over 24 MiB that sync had to read whole to refuse (segmented storage on, held `file_too_large`, for example for a facts fence) is no longer read again by every sync: it stays held, unread, until it changes, gbrain is upgraded, or `gbrain sources retry-held <source>` schedules it. After changing a setting the hold message names, run `gbrain sources retry-held <source>`. A file held from its size alone still re-checks on every sync.' },
- { since: '0.60.582.0', id: 'frontmatter_drift_write_8109', text: 'A write that leaves a page\'s frontmatter unchanged (`remember`, a fact or timeline write, a `put_page` of the stored frontmatter) is no longer refused `source_changed` (`file_database_drift`) when the page\'s canonical file differs from the database only in ordinary frontmatter keys (for example a contact row\'s `last_used` moved forward in the file): the write lands in the database and the file keeps its own frontmatter with the new body, so the frontmatter drift stays for `gbrain sources reconcile <source> <slug> --preview --auto-additive`. Drift in the body, timeline, title, type, tags or a policy or privacy key (`access`, `visibility`, `aliases`, ...) refuses as before, and so does a write that changes frontmatter (#8109).' },
+ { since: '0.60.582.0', id: 'frontmatter_drift_write_8109', text: 'A write that leaves a page\'s frontmatter unchanged (`remember`, a fact or timeline write, a `put_page` of the stored frontmatter) is no longer refused `source_changed` (`file_database_drift`) when the page\'s canonical file differs from the database only in ordinary frontmatter keys (for example a contact row\'s `last_used` moved forward in the file): the write lands in the database and the file keeps its own frontmatter with the new body, so the frontmatter drift stays for `gbrain sources reconcile <source> <slug> --preview --auto-additive`. Drift in the body, timeline, title, type, tags or a policy, privacy, lineage or trust key (`access`, `visibility`, `readable_by`, `derived_from`, `trust_tier`, `aliases`, ...) refuses as before, and so does a write that changes frontmatter (#8109).' },
{ since: '0.60.582.0', id: 'contacts_activity_advance_8109', text: '`gbrain sources reconcile --auto-additive` now decides a `contacts` row whose only change is an activity date (`last_used`, `last_*`, `*_last_used`, `updated`) moving forward as structurally additive (rule `contacts_activity_advance`) instead of `review` (`list_entries_changed_or_reordered`); a row that changed in any other way, moved, was removed or moved a date backward or into the future still needs review. Unquoted YAML dates compare as the instants they are instead of `date_not_text` (#8109).' },
{ since: '0.60.582.0', id: 'contact_rows_dedupe_8109', text: 'New explicit repair `gbrain repair contact-rows [--source <id>] [--slug <slug>]...` collapses duplicated frontmatter `contacts` rows: one row per (type, value), in first-seen order with the newest `last_used`; a row that differs in another field and every distinct value are kept. the preview writes nothing and prints a hash, `--apply --expect <hash> --yes` rewrites exactly the previewed pages through revision-bound `put_page` (the canonical file on sources with a checkout, the database on repo-less ones), and a page whose file drifted is listed `needs_reconcile`. Nothing runs on upgrade (#8109).' },
{ since: '0.60.525.0', id: 'refresh_sync_budget_8072', text: 'The member syncs of one `gbrain sources refresh` run stop starting work after `sources.refresh_sync_budget_ms` (default 5 minutes; `GBRAIN_REFRESH_SYNC_BUDGET_MS`). Each member gets an equal share of what is left. A run past it reports `sync_deferred` with the members it left `deferred` and still completes the refresh, so writes to the checkout are admitted again and the next `gbrain sources refresh <source>` continues them; a member that committed nothing in its share is reported `refresh_sync_timeout` in `sync_blocked`. A stalled sync no longer holds the refresh lock across scheduled runs or starves the other members.' },
diff --git a/src/core/persistence/page-prepare.ts b/src/core/persistence/page-prepare.ts
index 1e9955fcb8..e435cd9a12 100644
--- a/src/core/persistence/page-prepare.ts
+++ b/src/core/persistence/page-prepare.ts
@@ -92,6 +92,7 @@ import { fenceRepairCommit } from './effect-model.ts';
import { withoutSafetyAssessments } from './reconcile-safety.ts';
import { POLICY_KEYS } from './reconcile-additive.ts';
import { protectedReconcileKey } from './reconcile-merge.ts';
+import { PROTECTED_FRONTMATTER_KEYS } from '../frontmatter-recovery.ts';
import { preparePagePurge, purgePageInTransaction, type PreparedPagePurge } from './page-purge.ts';
import { withTrustKeep } from './context.ts';
import { isFenceEditWrite, pageGateTrust, pageWriteTrust, recordAgentPageLowering, stampPageTrustMarker, storedPageTier } from '../trust/page-write.ts';
@@ -237,10 +238,11 @@ export async function fileMatchesSnapshot(engine: BrainEngine, slug: string, byt
/**
* #8109: the file a write publishes over a canonical file that differs from the database only in frontmatter, or null
* when the write must refuse as drift. The file and the database must agree on everything but frontmatter (body,
- * timeline, title, type and tags, as fileMatchesSnapshot compares them), no drifted key may be a policy or protected
- * key, and the write must leave the stored frontmatter unchanged. The file then keeps its own frontmatter block and
+ * timeline, title, type and tags, as fileMatchesSnapshot compares them), no drifted key may be a policy, protected,
+ * access, lineage or trust-lowering key, and the write must leave the stored frontmatter unchanged. The file then keeps its own frontmatter block and
* takes the write's body, so neither side's frontmatter is lost; the drift stays for reconcile.
*/
+const SPLICE_REFUSED_KEYS = new Set([...PROTECTED_FRONTMATTER_KEYS, 'trust_tier', 'transcript_import', 'source_url', 'clipped_at', 'clipper', 'dream_generated']);
async function frontmatterDriftWrite(engine: BrainEngine, slug: string, fileText: string, snapshot: PageSnapshot, content: string,
activePack?: ParseOpts['activePack']): Promise<string | null> {
snapshot = (await materializedSnapshot(engine, snapshot))!;
@@ -253,7 +255,7 @@ async function frontmatterDriftWrite(engine: BrainEngine, slug: string, fileText
const fileFrontmatter = comparableFrontmatter(withoutSafetyAssessments(file.frontmatter));
const stored = comparableFrontmatter(withoutSafetyAssessments(snapshot.page.frontmatter ?? {}));
const drifted = [...new Set([...Object.keys(fileFrontmatter), ...Object.keys(stored)])].filter(key => digest(fileFrontmatter[key] ?? null) !== digest(stored[key] ?? null));
- if (!drifted.length || drifted.some(key => POLICY_KEYS.has(key) || protectedReconcileKey(key))) return null;
+ if (!drifted.length || drifted.some(key => POLICY_KEYS.has(key) || protectedReconcileKey(key) || SPLICE_REFUSED_KEYS.has(key))) return null;
const incoming = parseMarkdown(content, slug, { activePack });
if (digest(comparableFrontmatter(withoutSafetyAssessments(incoming.frontmatter))) !== digest(stored)
|| bare({ ...incoming, compiled_truth: '', timeline: '' }, incoming.tags) !== bare({ ...snapshot.page, compiled_truth: '', timeline: '' }, snapshot.tags)) return null;
diff --git a/test/contact-rows-drift-8109.test.ts b/test/contact-rows-drift-8109.test.ts
index 85dfcc74c1..04be5b4fe1 100644
--- a/test/contact-rows-drift-8109.test.ts
+++ b/test/contact-rows-drift-8109.test.ts
@@ -128,6 +128,9 @@ test('#8109: a last_used bump in the file does not block remember; the file keep
for (const [name, edit] of [
['a policy field', (page: Editable) => { advance(page); page.frontmatter.visibility = 'public'; }],
+ ['a reader list', (page: Editable) => { advance(page); page.frontmatter.readable_by = ['alice-example']; }],
+ ['a lineage key', (page: Editable) => { advance(page); page.frontmatter.derived_from = ['notes/beta-example']; }],
+ ['a trust marker', (page: Editable) => { advance(page); page.frontmatter.trust_tier = 'agent_written'; }],
['the body', (page: Editable) => { advance(page); page.compiled_truth += '\n\nA line added by hand.'; }],
['the title', (page: Editable) => { page.title = 'Alice Renamed'; page.frontmatter.title = 'Alice Renamed'; }],
] as const) {