Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Show the parked pages a manual purge would free in its preview, and write nothing before consent
garrytandaa856c4bf4a
diff --git a/docs/architecture/key-files/memory-purge.md b/docs/architecture/key-files/memory-purge.mdindex e8e3a17657..f21a428ecb 100644--- a/docs/architecture/key-files/memory-purge.md+++ b/docs/architecture/key-files/memory-purge.md@@ -34,6 +34,6 @@ User-facing contract: [expire versus purge](../../guides/memory-boundaries.md#pu- `src/commands/doctor/checks/atom-fence-rows.ts` — doctor `atom_fence_rows` (BRAIN, a host-only `WAVE_CHECKS` repair entry cleared by the explicit-only `atom-fence-rows`, so remediation classifies it `explicit_kind_required`): runs `findAtomFenceRows` per source, each statement bounded by `PLAN_STATEMENT_MS`; `warn` names the count per source and the read-only preview `gbrain repair atom-fence-rows --source <id> --json` as an agent `fix`, never a slug or row text; an overrun reports the count as unknown.- `src/core/persistence/purge-parks.ts` — #6741 C: per-tombstone purge parks in op_checkpoints (op `purge-deleted-park`, key source+slug+deleted_at, so a re-deleted page starts fresh); park at `PURGE_PARK_LIMIT` (5) non-deferral failures, a success clears it, and parks of a source seen inactive rearm the first run its writer is active again (`notePurgeSourceWriter`). `purgeStaleCheckpoints` exempts the op; `prunePurgeParks` drops parks whose tombstone is gone. A park records the gbrain `version` of its last failure; `isParked` (the managed and unmanaged selection, doctor `purge_parked_pages`) counts only the running version's, so a park another version wrote (or one with no version) gets one automatic attempt and a failure re-parks it under the running version. Tests: `test/purge-deleted-parks.test.ts`, `test/purge-parks-version.test.ts` (Postgres: `test/e2e/purge-parks-version-postgres.test.ts`).-- `src/core/persistence/derived-origin-hold.ts` + `src/core/repair/origin-carry.ts` — GST-2689: a page that reads as world by its own label but is hidden from remote callers only through a private origin (`derived_from`, an atom's `source_slug`, followed through `slug_aliases` and `slug_rename_history`, or a concept's provenance atom) keeps that privacy once the origin row is gone. `origin-carry.ts` finds the dependents with `private-labels.ts` `dependentsSql('remove')` (it reads only the pages whose frontmatter names a removed slug, or a slug a rename moved to it, through `idx_pages_frontmatter`, one jsonpath scan per 100 slugs of a source, so its cost does not grow with the source; the postimage drops the origin rows; a page that names readers is returned when its listed readers would gain it, and gets the label with its `readable_by` kept; a life-chronicle event is returned when the removal opens it, never when its `event.depth` page is removed, since it then fails closed) and gives each `visibility: private` through the label door (`publishMaintenancePages` with `keepTrust` and the label edit on a managed brain, batch `derived-origin-carry:<request_id>`; `writeLabelFile` or `importAtRevision` otherwise). `deletePageCarryingPrivate` is the `delete_page` handler body: a soft delete (local or remote) labels after it committed and returns the unchanged receipt; a purge labels first and, when that fails, soft-deletes instead (an existing tombstone stays), parks the origin and returns `purge_held`. `carryPrivateBeforePurge` runs before every expiry over all the origins (`type = 'atom'` or hidden by `privacyPartSql`) expired or expiring within `CARRY_LEAD_HOURS` (24) at once, so a page private through two of them is found and none crosses the cutoff mid-run uncarried; it publishes the labels 200 per maintenance batch, parks the blocked ones and returns the ones whose park no longer applies, which `purgeDeletedPagesCoordinated` clears only after its purge, so an origin parked when the purge was previewed stays that run; a consented manual purge (`gbrain pages purge-deleted`, `purge_deleted_pages`) first runs `retryParkedOrigins` so its preview lists the origins it frees. A soft-deleted dependent is returned too but never labelled: it blocks its origin (`dependent_soft_deleted`) until it is purged or restored, since a restore would bring it back with the origin row gone; a removed origin tied to another removed origin counts as such a dependent (judged on its own row), since another hold may keep it. Inside each delete transaction `parkUncarriedInPurge` rechecks the origins about to go (the page purge for an explicit or managed retention purge, the unmanaged bulk delete, and each recorded tombstone): an origin a page written after the carry still depends on is parked in that transaction and its row is kept (a live page is soft-deleted instead, and the receipt carries `purge_held` naming no page), so the next purge labels the page and the one after removes the origin. The origin is looked up by the lowercased slug, as `submitPageMutation` deletes it. `derived-origin-hold.ts` is the park (op_checkpoints op `derived-origin-park`, fingerprint the page id, with `blocking_slug`, `code` and `reason`), `derivedOriginPurgeHold` (no engine-sql imports), prune of restored or purged pages, and `readDerivedOriginParks` for doctor `derived_origin_parked`; `purgeStaleCheckpoints` exempts the op. Tests: `test/derived-origin-carry-private.test.ts` (repo-less, minted tokens), `test/derived-origin-carry-managed.test.ts` (managed Git brain), `test/derived-origin-recheck-plan.test.ts` (pages rows the recheck visits).+- `src/core/persistence/derived-origin-hold.ts` + `src/core/repair/origin-carry.ts` — GST-2689: a page that reads as world by its own label but is hidden from remote callers only through a private origin (`derived_from`, an atom's `source_slug`, followed through `slug_aliases` and `slug_rename_history`, or a concept's provenance atom) keeps that privacy once the origin row is gone. `origin-carry.ts` finds the dependents with `private-labels.ts` `dependentsSql('remove')` (it reads only the pages whose frontmatter names a removed slug, or a slug a rename moved to it, through `idx_pages_frontmatter`, one jsonpath scan per 100 slugs of a source, so its cost does not grow with the source; the postimage drops the origin rows; a page that names readers is returned when its listed readers would gain it, and gets the label with its `readable_by` kept; a life-chronicle event is returned when the removal opens it, never when its `event.depth` page is removed, since it then fails closed) and gives each `visibility: private` through the label door (`publishMaintenancePages` with `keepTrust` and the label edit on a managed brain, batch `derived-origin-carry:<request_id>`; `writeLabelFile` or `importAtRevision` otherwise). `deletePageCarryingPrivate` is the `delete_page` handler body: a soft delete (local or remote) labels after it committed and returns the unchanged receipt; a purge labels first and, when that fails, soft-deletes instead (an existing tombstone stays), parks the origin and returns `purge_held`. `carryPrivateBeforePurge` runs before every expiry over all the origins (`type = 'atom'` or hidden by `privacyPartSql`) expired or expiring within `CARRY_LEAD_HOURS` (24) at once, so a page private through two of them is found and none crosses the cutoff mid-run uncarried; it publishes the labels 200 per maintenance batch, parks the blocked ones and returns the ones whose park no longer applies, which `purgeDeletedPagesCoordinated` clears only after its purge, so an origin parked when the purge was previewed stays that run; a consented manual purge (`gbrain pages purge-deleted`, `purge_deleted_pages`) previews with `forecastCarry` (read-only: the parked origins the carry would free, `released_origins`, and the origins it would park, both judged as `carryPrivate` judges them) passed to `selectPurgeCandidates`, so `--dry-run` and the consent list exactly what `--yes` purges and nothing is labelled before the consent; the run passes the freed ids as `release`, and `purgeDeletedPagesCoordinated` clears those parks once its carry landed, before it purges. A soft-deleted dependent is returned too but never labelled: it blocks its origin (`dependent_soft_deleted`) until it is purged or restored, since a restore would bring it back with the origin row gone; a removed origin tied to another removed origin counts as such a dependent (judged on its own row), since another hold may keep it. Inside each delete transaction `parkUncarriedInPurge` rechecks the origins about to go (the page purge for an explicit or managed retention purge, the unmanaged bulk delete, and each recorded tombstone): an origin a page written after the carry still depends on is parked in that transaction and its row is kept (a live page is soft-deleted instead, and the receipt carries `purge_held` naming no page), so the next purge labels the page and the one after removes the origin. The origin is looked up by the lowercased slug, as `submitPageMutation` deletes it. `derived-origin-hold.ts` is the park (op_checkpoints op `derived-origin-park`, fingerprint the page id, with `blocking_slug`, `code` and `reason`), `derivedOriginPurgeHold` (no engine-sql imports), prune of restored or purged pages, and `readDerivedOriginParks` for doctor `derived_origin_parked`; `purgeStaleCheckpoints` exempts the op. Tests: `test/derived-origin-carry-private.test.ts` (repo-less, minted tokens), `test/derived-origin-carry-managed.test.ts` (managed Git brain), `test/derived-origin-recheck-plan.test.ts` (pages rows the recheck visits).- `src/core/repair/segmented-mirror.ts` — `gbrain repair segmented-mirror` (LPH a), explicit-only, preview-bound, destructive consent, `checks: []` (no doctor scan: detection reads every segmented file). `scanSegmentedMirrors` (read-only, owner host; a non-owner source is held `not_owner_host`) walks live `body_storage='segmented'` pages of sources with a worktree binding whose snapshot ledger applies (`ledgerApplies`), runs `prepareFileTarget(..., null, { allowMissing })` (the mirror's own drift check; `file_database_drift` or any other `OperationError` is held, never rewritten), then lists the page only when overlaying the ledger on the file's facts fences changes its region digests (`fileBodyDigests` vs `overlaidFileBodyDigests` over `FileBytes`, bounded memory) and the streamed render (`unreadPageFile`) hashes differently. The hash (`segmented-mirror-v1`) binds source selection, slug, page id, revision, path and both SHA-256s; the approved set lives in `op_checkpoints`. Apply re-checks revision and file hash of every page not yet admitted (`preview_changed`), then per page admits and commits, in one transaction like a withdrawal, a `repair_segmented_mirror` request (request id derived from the hash; its receipt) that inserts a version-2 targeted `withdrawal-mirror` + `git` effect (no embedding) whose `data.approved_file` carries the preview's file and render SHA-256 (`mirrorPage` skips the target `preview_changed` when the file's before-hash or the streamed render differs, before the gate or any recovery record; `publishGitPage` skips it when the file is not the approved render; the outcome is `refused` `preview_changed` with a read-only fix re-running the preview); it wakes this process's consumer and waits up to the write wait for them (`mirrored` / `mirror_queued` / `held` / `failed`; a replay is `replayed`). Test: `test/pathological/lp-a-segmented-mirror.test.ts`.diff --git a/src/commands/pages.ts b/src/commands/pages.tsindex dba0e03d4a..50f069d2de 100644--- a/src/commands/pages.ts+++ b/src/commands/pages.ts@@ -47,10 +47,13 @@ async function runPurgeDeleted(engine: BrainEngine, args: string[]): Promise<voiconst { olderThanHours, dryRun, json, retryParked } = parsePurgeArgs(args);// #6741: one selector for the preview, the consent and the run (selectPurgeCandidates): the same cutoff, rename// holds, writer state and parks; --retry-parked puts the parked pages the cutoff selects back among the candidates.- if (!dryRun) await (await import('../core/repair/origin-carry.ts')).retryParkedOrigins(engine, olderThanHours);- const selection = await selectPurgeCandidates(engine, olderThanHours, { retryParked });+ // GST-2689: the preview reads ahead what the run's carry does (forecastCarry, read-only), so it lists the parked origins+ // the run releases and leaves out the ones it parks; nothing is labelled before the consent.+ const forecast = await (await import('../core/repair/origin-carry.ts')).forecastCarry(engine, olderThanHours);+ const selection = await selectPurgeCandidates(engine, olderThanHours, { retryParked, forecast });const retried = selection.parked.filter(park => park.retried).length;- const notes = { ...(selection.parked.length ? { parked: selection.parked, retry_parked: retryParked } : {}), ...(selection.blocked.length ? { blocked: selection.blocked } : {}) };+ const notes = { ...(selection.parked.length ? { parked: selection.parked, retry_parked: retryParked } : {}), ...(selection.blocked.length ? { blocked: selection.blocked } : {}),+ ...(selection.released_origins.length ? { released_origins: selection.released_origins } : {}) };if (dryRun) {// Read-only: the same predicate and DB now() clock as the real purge; only the verb differs.@@ -65,6 +68,7 @@ async function runPurgeDeleted(engine: BrainEngine, args: string[]): Promise<voifor (const c of selection.candidates) console.log(` ${c.source_id}/${c.slug}${c.park ? ' (parked; retried once more)' : ''}${c.sync_unreviewed ? ' (deleted by a managed sync; no live twin)' : ''}`);for (const p of selection.parked.filter(park => !park.retried)) console.log(` parked: ${p.source_id}/${p.slug} after ${p.failures} failures (${p.code ?? 'error'}); skipped unless --retry-parked`);for (const b of selection.blocked.filter(entry => entry.code === 'writer_not_active')) console.log(` waiting: ${b.source_id}/${b.slug}: ${b.reason}`);+ for (const o of selection.released_origins) console.log(` released: ${o.source_id}/${o.slug} is parked (gbrain doctor: derived_origin_parked); the run labels the pages derived from it private first and keeps it parked if that fails`);return;}@@ -82,7 +86,7 @@ async function runPurgeDeleted(engine: BrainEngine, args: string[]): Promise<voiconst authorized = await consentGate(purgeConsentRequest(selection.count, olderThanHours, json, args, retryParked, retried, selection.sync_unreviewed), { json });if (!authorized) return;- const result = await purgeDeletedPagesCoordinated(engine, olderThanHours, { retryParked });+ const result = await purgeDeletedPagesCoordinated(engine, olderThanHours, { retryParked, release: forecast.release.map(origin => origin.id) });if (json) {console.log(JSON.stringify({ older_than_hours: olderThanHours, count: result.count, slugs: result.slugs,...(result.capacity_deferred ? { capacity_deferred: result.capacity_deferred } : {}), ...(result.blocked.length ? { blocked: result.blocked } : {}) }, null, 2));diff --git a/src/core/persistence/purge-deleted.ts b/src/core/persistence/purge-deleted.tsindex ee609ad8e2..fa3da414c3 100644--- a/src/core/persistence/purge-deleted.ts+++ b/src/core/persistence/purge-deleted.ts@@ -16,6 +16,7 @@ import { atomRetirementPurgeHold } from './atom-retirement-hold.ts';import { derivedRetirementPurgeHold } from './derived-retirement-hold.ts';import { faultPoint } from './fault-points.ts';import { clearDerivedOriginParks, DERIVED_ORIGIN_PARKED_CODE, derivedOriginPurgeHold, readDerivedOriginParks, type DerivedOriginPark } from './derived-origin-hold.ts';+import type { CarryForecast } from '../repair/origin-carry.ts';import { writerOnboardingPreflight } from './onboarding.ts';import { clearPurgePark, isParked, notePurgeSourceWriter, PURGE_PARK_LIMIT, purgeParkKey, prunePurgeParks, readPurgeParks, recordPurgeFailure, type PurgePark } from './purge-parks.ts';import { ItemRun, type ItemFailure } from '../item-failure.ts';@@ -91,23 +92,23 @@ export function purgeConsentRequest(count: number, olderThanHours: number, json:/** The `purge_deleted_pages` op (`gbrain call`): the D6 consent first (`yes` is `--yes`); an empty purge set needs none. */export async function consentedPurgeDeletedPages(engine: BrainEngine, olderThanHours: number, yes: boolean) {- await (await import('../repair/origin-carry.ts')).retryParkedOrigins(engine, olderThanHours);- const preview = await selectPurgeCandidates(engine, olderThanHours);+ const forecast = await (await import('../repair/origin-carry.ts')).forecastCarry(engine, olderThanHours);+ const preview = await selectPurgeCandidates(engine, olderThanHours, { forecast });if (preview.count === 0) return { status: 'purged', count: 0, slugs: [] };await requireConsent(purgeConsentRequest(preview.count, olderThanHours, false, yes ? ['--yes'] : [], false, 0, preview.sync_unreviewed));- const result = await purgeDeletedPagesCoordinated(engine, olderThanHours);+ const result = await purgeDeletedPagesCoordinated(engine, olderThanHours, { release: forecast.release.map(origin => origin.id) });return { status: result.failed ? 'partial' : 'purged', count: result.count, slugs: result.slugs, ...(result.blocked.length ? { blocked: result.blocked } : {}) };}/** An expired tombstone of a source the belief ledger records (unmanaged brain), in scan order, with its live failure record. */interface RecordedTombstone { id: number; source_id: string; slug: string; deleted_us: string; key: string; park?: PurgePark }-async function recordedTombstones(engine: BrainEngine, hours: string): Promise<RecordedTombstone[]> {+async function recordedTombstones(engine: BrainEngine, hours: string, released: readonly number[] = []): Promise<RecordedTombstone[]> {const rows = await engine.executeRaw<{ id: number; source_id: string; slug: string; deleted_us: string }>(`SELECT p.id, p.source_id, p.slug,floor(extract(epoch FROM p.deleted_at)*1000000)::bigint::text AS deleted_us FROM pages p- WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${identityDowngradePurgeHold('p')} AND ${atomRetirementPurgeHold('p')} AND ${derivedRetirementPurgeHold('p')} AND ${derivedOriginPurgeHold('p')}+ WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${identityDowngradePurgeHold('p')} AND ${atomRetirementPurgeHold('p')} AND ${derivedRetirementPurgeHold('p')} AND (${derivedOriginPurgeHold('p')} OR p.id = ANY($2::int[]))AND EXISTS (SELECT 1 FROM belief_events e WHERE e.source_id = p.source_id AND e.subject_kind = 'coverage')- ORDER BY p.deleted_at ASC, p.source_id ASC, p.slug ASC`, [hours]);+ ORDER BY p.deleted_at ASC, p.source_id ASC, p.slug ASC`, [hours, [...released]]);const parks = rows.length ? await readPurgeParks(engine) : new Map<string, PurgePark>();return rows.map(row => {const key = purgeParkKey(row.source_id, row.slug, row.deleted_us);@@ -219,23 +220,44 @@ export interface PurgeSelection {sync_unreviewed: number;/** Per source in the scan: whether its writer passed the owner verification (the park rearm reads it). */writers: Record<string, boolean>;+ /** GST-2689: the candidates that are parked origins the run's carry releases first (`forecast.release`). */+ released_origins: Array<{ source_id: string; slug: string }>;}-/** What keeps an expired tombstone of page `page` (source `source`) out of every managed purge: an archived source, or the rename-recovery, restorable-atom, derived-retirement or derived-origin hold. */-export function purgeHoldsSql(page: string, source: string): string {- return `NOT ${source}.archived AND ${identityDowngradePurgeHold(page)} AND ${atomRetirementPurgeHold(page)} AND ${derivedRetirementPurgeHold(page)} AND ${derivedOriginPurgeHold(page)}`;+/**+ * What keeps an expired tombstone of page `page` (source `source`) out of every managed purge: an archived source, or the+ * rename-recovery, restorable-atom, derived-retirement or derived-origin hold; `released` (an SQL int[]) lifts the+ * derived-origin hold for the parked origins a manual purge's preview counts as released.+ */+export function purgeHoldsSql(page: string, source: string, released?: string): string {+ const origin = released ? `(${derivedOriginPurgeHold(page)} OR ${page}.id = ANY(${released}))` : derivedOriginPurgeHold(page);+ return `NOT ${source}.archived AND ${identityDowngradePurgeHold(page)} AND ${atomRetirementPurgeHold(page)} AND ${derivedRetirementPurgeHold(page)} AND ${origin}`;}-export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours: number, opts: { retryParked?: boolean; automatic?: boolean } = {}): Promise<PurgeSelection> {+/**+ * GST-2689 `forecast` (repair/origin-carry.ts forecastCarry; the consented manual purge's preview and consent): the+ * selection the run makes once its carry has run, so it counts the parked origins the carry releases and leaves out the+ * origins it parks; the run itself reads the parks as they are.+ */+export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours: number, opts: { retryParked?: boolean; automatic?: boolean; forecast?: CarryForecast } = {}): Promise<PurgeSelection> {+ const release = (opts.forecast?.release ?? []).map(origin => origin.id), hold = (opts.forecast?.hold ?? []).map(origin => origin.id);if (!await managedPersistenceEnabled(engine)) {- const parked = (await recordedTombstones(engine, String(Math.max(0, Math.floor(olderThanHours)))))+ const parked = (await recordedTombstones(engine, String(Math.max(0, Math.floor(olderThanHours))), release)).filter((row): row is RecordedTombstone & { park: PurgePark } => isParked(row.park));const skipped = new Set(opts.retryParked ? [] : parked.map(row => row.id));const unreviewed = (await readUnreviewedSyncTombstones(engine, olderThanHours)).filter(row => !skipped.has(row.id));const kept = opts.automatic ? unreviewed : [];- const preview = await engine.purgeDeletedPages(olderThanHours, { dryRun: true, excludeIds: [...skipped, ...kept.map(row => row.id)] });+ const excludeIds = [...skipped, ...kept.map(row => row.id), ...hold];+ const preview = await engine.purgeDeletedPages(olderThanHours, { dryRun: true, excludeIds });+ // The engine's preview keeps every parked origin; the ones the run frees are read here under the purge's other holds.+ const released = release.length ? (await engine.executeRaw<{ slug: string; source_id: string; deleted_at: Date | string }>(`SELECT p.slug, p.source_id, p.deleted_at FROM pages p+ WHERE p.id = ANY($1::int[]) AND p.id <> ALL($2::int[]) AND p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($3 || ' hours')::interval+ AND ${identityDowngradePurgeHold('p')} AND ${atomRetirementPurgeHold('p')} AND ${derivedRetirementPurgeHold('p')}`,+ [release, excludeIds, String(Math.max(0, Math.floor(olderThanHours)))])).map(row => ({ slug: row.slug, source_id: row.source_id, deleted_at: new Date(row.deleted_at) }))+ .filter(row => !(preview.pages ?? []).some(page => page.source_id === row.source_id && page.slug === row.slug)) : [];+ const pages = [...(preview.pages ?? []), ...released].sort((a, b) => a.deleted_at.getTime() - b.deleted_at.getTime() || (a.slug < b.slug ? -1 : a.slug > b.slug ? 1 : 0));const held = kept.filter(row => row.expired);- return { managed: false, count: preview.count, slugs: preview.slugs, pages: preview.pages, candidates: [],+ return { managed: false, count: pages.length, slugs: pages.map(page => page.slug), pages, candidates: [], released_origins: released.map(({ source_id, slug }) => ({ source_id, slug })),blocked: [...(opts.retryParked ? [] : parked.map(row => ({ source_id: row.source_id, slug: row.slug, code: 'purge_parked', reason: parkedReason(row.park) }))),...held.map(row => syncDeleteHeld(row, olderThanHours))],parked: parked.map(row => ({ source_id: row.source_id, slug: row.slug, failures: row.park.failures, code: row.park.code, retried: opts.retryParked === true })),@@ -245,13 +267,13 @@ export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours:const rows = await engine.executeRaw<{ id: number | string; source_id: string; slug: string; cutoff: string; deleted_us: string }>(`SELECT p.id, p.source_id, p.slug, (now() - ($1 || ' hours')::interval)::text AS cutoff,floor(extract(epoch FROM p.deleted_at)*1000000)::bigint::text AS deleted_us FROM pages p JOIN sources s ON s.id = p.source_id- WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${purgeHoldsSql('p', 's')}- ORDER BY p.deleted_at ASC, p.source_id ASC, p.slug ASC`, [String(hours)]);+ WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${purgeHoldsSql('p', 's', '$2::int[]')} AND p.id <> ALL($3::int[])+ ORDER BY p.deleted_at ASC, p.source_id ASC, p.slug ASC`, [String(hours), release, hold]);const states = new Map<string, { state: string; physical_error?: string }>();if (rows.length) for (const source of (await writerOnboardingPreflight(engine)).sources) states.set(source.source_id, source);const parks = rows.length ? await readPurgeParks(engine) : new Map<string, PurgePark>();const unreviewed = new Set(rows.length ? (await readUnreviewedSyncTombstones(engine, hours)).map(row => row.id) : []);- const selection: PurgeSelection = { managed: true, count: 0, slugs: [], candidates: [], blocked: [], parked: [], inactive: 0, held: 0, sync_unreviewed: 0, writers: {} };+ const selection: PurgeSelection = { managed: true, count: 0, slugs: [], candidates: [], blocked: [], parked: [], inactive: 0, held: 0, sync_unreviewed: 0, writers: {}, released_origins: [] };for (const { id, ...row } of rows) {const writer = states.get(row.source_id);const active = writer?.state !== 'recovery_required';@@ -279,6 +301,7 @@ export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours:}}if (syncUnreviewed) selection.sync_unreviewed++;+ if (release.includes(Number(id))) selection.released_origins.push({ source_id: row.source_id, slug: row.slug });selection.candidates.push({ ...row, key, ...(park ? { park } : {}), ...(syncUnreviewed ? { sync_unreviewed: true as const } : {}) });}selection.count = selection.candidates.length;@@ -334,11 +357,14 @@ export function purgePhaseNote(result: DeletedPagePurge): string {/** Limits a purge to pages the caller already showed the person (`gbrain maintain --purge --expect`). */export type PurgeFilter = (page: { source_id: string; slug: string }) => boolean;-export async function purgeDeletedPagesCoordinated(engine: BrainEngine, olderThanHours: number, opts: { retryParked?: boolean; automatic?: boolean; only?: PurgeFilter } = {}): Promise<DeletedPagePurge> {+export async function purgeDeletedPagesCoordinated(engine: BrainEngine, olderThanHours: number, opts: { retryParked?: boolean; automatic?: boolean; only?: PurgeFilter; release?: readonly number[] } = {}): Promise<DeletedPagePurge> {+ const { release = [], ...purge } = opts;const carried = await (await import('../repair/origin-carry.ts')).carryPrivateBeforePurge(engine, olderThanHours);const origins = originParked(carried.parks);+ // GST-2689: a consented manual purge releases now the parked origins its preview counted, once their carry landed.+ await clearDerivedOriginParks(engine, carried.unparked.filter(origin => release.includes(origin.id)).map(origin => origin.id));await faultPoint('purge:after_carry');- const result = await purgeExpired(engine, olderThanHours, opts);+ const result = await purgeExpired(engine, olderThanHours, purge);const reparked = new Set(result.blocked.filter(b => b.code === DERIVED_ORIGIN_PARKED_CODE).map(b => `${b.source_id}/${b.slug}`));await clearDerivedOriginParks(engine, carried.unparked.filter(origin => !reparked.has(`${origin.source_id}/${origin.slug}`)).map(origin => origin.id));const parked = origins.length + (result.origin_parked ?? 0);diff --git a/src/core/repair/origin-carry.ts b/src/core/repair/origin-carry.tsindex a6baac09f0..b6df9baa26 100644--- a/src/core/repair/origin-carry.ts+++ b/src/core/repair/origin-carry.ts@@ -33,7 +33,7 @@ const EDIT: LabelEdit = { set: { visibility: 'private' }, remove: [] };/** A page whose removal can open another: hidden by the privacy rules itself, or an atom a concept's provenance names. */const originSql = (alias: string) => `(${alias}.type = 'atom' OR NOT ${privacyPartSql(alias)})`;-interface Origin { id: number; source_id: string; slug: string }+export interface Origin { id: number; source_id: string; slug: string }interface Dependent { id: number; origin_id: number; source_id: string; slug: string; deleted: boolean }interface Failure { code: string | null; reason: string }export interface CarryBlock extends Failure { slug: string | null }@@ -175,6 +175,16 @@ export async function deletePageCarryingPrivate(ctx: OperationContext, p: Record/** Hours before its cutoff a tombstone is carried, so none crosses the cutoff while the carry runs and is purged uncarried. */export const CARRY_LEAD_HOURS = 24;+/** The soft-deleted origins expired or expiring within CARRY_LEAD_HOURS of this cutoff, the set every carry judges at once. */+async function expiringOrigins(engine: BrainEngine, olderThanHours: number): Promise<Origin[]> {+ return withoutCallerAccess(async () => {+ await readHonorAliases(engine);+ return (await engine.executeRaw<{ id: number; source_id: string; slug: string }>(`SELECT p.id, p.source_id, p.slug FROM pages p+ WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${originSql('p')} ORDER BY p.id`,+ [String(Math.max(0, Math.floor(olderThanHours) - CARRY_LEAD_HOURS))], bounded)).map(row => ({ id: Number(row.id), source_id: row.source_id, slug: row.slug }));+ });+}+/*** Before a purge: the soft-deleted origins expired or expiring within CARRY_LEAD_HOURS get their dependents labelled* private, judged against all of them at once (a page private through two of them opens only when both go); each one@@ -183,15 +193,9 @@ export const CARRY_LEAD_HOURS = 24;*/export async function carryPrivateBeforePurge(engine: BrainEngine, olderThanHours: number): Promise<{ parks: DerivedOriginPark[]; unparked: Origin[] }> {await pruneDerivedOriginParks(engine);- const origins = await withoutCallerAccess(async () => {- await readHonorAliases(engine);- return engine.executeRaw<{ id: number; source_id: string; slug: string }>(`SELECT p.id, p.source_id, p.slug FROM pages p- WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${originSql('p')} ORDER BY p.id`,- [String(Math.max(0, Math.floor(olderThanHours) - CARRY_LEAD_HOURS))], bounded);- });const parks: DerivedOriginPark[] = [];const batch = `derived-origin-carry:purge-${randomUUID()}`;- const expired = origins.map(row => ({ id: Number(row.id), source_id: row.source_id, slug: row.slug }));+ const expired = await expiringOrigins(engine, olderThanHours);const blocked = await carryPrivate(engine, expired, batch);for (const origin of expired) {const block = blocked.get(origin.id);@@ -202,13 +206,34 @@ export async function carryPrivateBeforePurge(engine: BrainEngine, olderThanHourreturn { parks, unparked: expired.filter(origin => !blocked.has(origin.id)) };}+/** What the carry of a consented manual purge would do, read ahead: the parked origins it frees and the others it parks. */+export interface CarryForecast { release: Origin[]; hold: Origin[] }+/**- * A consented manual purge first retries the label for every parked origin and unparks the ones it frees, so its preview- * and consent list them (otherwise a brain whose only expired pages are parked origins never gets past "nothing to purge").+ * Read-only, for a consented manual purge's preview and consent: what carryPrivateBeforePurge would do over the same+ * expiring origins, judged as carryPrivate judges them. An origin is blocked when a page it alone keeps private is+ * soft-deleted, stored in segments or unreadable; the label write is assumed to land on every other page. A parked origin+ * not blocked is released (the run clears its park once its carry lands, purgeDeletedPagesCoordinated `release`), and an+ * origin not parked but blocked is held. A failed read holds every origin, as a failed carry parks every origin.*/-export async function retryParkedOrigins(engine: BrainEngine, olderThanHours: number): Promise<void> {- if (!(await readDerivedOriginParks(engine)).length) return;- await clearDerivedOriginParks(engine, (await carryPrivateBeforePurge(engine, olderThanHours)).unparked.map(origin => origin.id));+export async function forecastCarry(engine: BrainEngine, olderThanHours: number): Promise<CarryForecast> {+ const parks = new Set((await readDerivedOriginParks(engine)).map(park => park.page_id));+ return withoutCallerAccess(async () => {+ let origins: Origin[] = [];+ try {+ origins = await expiringOrigins(engine, olderThanHours);+ const blocked = new Set<number>();+ if (origins.length) for (const page of await dependentsOf(engine, origins)) {+ if (blocked.has(page.origin_id)) continue;+ const snapshot = page.deleted ? null : await engine.readPageSnapshot(page.slug, { sourceId: page.source_id });+ if (!snapshot || Number(snapshot.page.id) !== page.id || isSegmentedPage(snapshot.page)) blocked.add(page.origin_id);+ }+ return { release: origins.filter(origin => parks.has(origin.id) && !blocked.has(origin.id)),+ hold: origins.filter(origin => !parks.has(origin.id) && blocked.has(origin.id)) };+ } catch {+ return { release: [], hold: origins.filter(origin => !parks.has(origin.id)) };+ }+ });}/** The receipt note of an explicit purge held because its origin lookup failed; it names no page and counts none. */diff --git a/test/derived-origin-carry-managed.test.ts b/test/derived-origin-carry-managed.test.tsindex 806d68ef9e..db4828c3de 100644--- a/test/derived-origin-carry-managed.test.ts+++ b/test/derived-origin-carry-managed.test.ts@@ -27,6 +27,9 @@ import { runPages } from '../src/commands/pages.ts';import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';import { installFaultHook } from '../src/core/persistence/fault-points.ts';import { purgeDeletedPagesCoordinated } from '../src/core/persistence/purge-deleted.ts';+import * as preparedMaintenance from '../src/core/persistence/prepared-maintenance.ts';+import { readDerivedOriginParks } from '../src/core/persistence/derived-origin-hold.ts';+import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';import { importFromContent } from '../src/core/import-file.ts';import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';import { sqlQueryForEngine } from '../src/core/sql-query.ts';@@ -62,7 +65,7 @@ function fileText(k: string, shape: Shape): string {};return ['---', `title: ${k} ${shape}`, ...fm[shape], '---', '', `Harbor ledger planning notes ${canaryOf(k, shape)} for the roadmap.`, ''].join('\n');}-const KEYS = ['gitrm', 'purge', 'remote', 'midpurge'];+const KEYS = ['gitrm', 'purge', 'remote', 'midpurge', 'preview'];/** A private origin, a public meeting, an event derived from the origin whose depth is the meeting, and an event whose depth is the origin. */const EVENT_SLUGS = { origin: 'notes/event-origin', depth: 'notes/event-depth', event: 'notes/event-event', closed: 'notes/event-closed' };const EVENT_FILES: Record<keyof typeof EVENT_SLUGS, string[]> = {@@ -100,6 +103,21 @@ async function cli(engine: BrainEngine, args: string[]): Promise<void> {try { await withEnv({ GBRAIN_CONSENT: undefined }, () => runPages(engine, args)); }finally { log.mockRestore(); err.mockRestore(); }}+/** `gbrain pages <args>` as the CLI runs it: its JSON output and exit verdict. */+async function cliJson(engine: BrainEngine, args: string[]): Promise<{ out: Record<string, any> | null; exit: number }> {+ _resetCliExitVerdictForTests();+ const prior = process.exitCode;+ const lines: string[] = [];+ const log = spyOn(console, 'log').mockImplementation((...parts: unknown[]) => { lines.push(parts.join(' ')); });+ const err = spyOn(console, 'error').mockImplementation(() => {});+ try { await withEnv({ GBRAIN_CONSENT: undefined }, () => runPages(engine, args)); }+ finally { log.mockRestore(); err.mockRestore(); }+ const exit = currentExitCode();+ _resetCliExitVerdictForTests();+ process.exitCode = prior;+ const text = lines.join('\n').trim();+ return { out: text.startsWith('{') ? JSON.parse(text) : null, exit };+}const page = async (engine: BrainEngine, slug: string) => (await engine.executeRaw<{ deleted_at: string | null; frontmatter: unknown }>("SELECT deleted_at, frontmatter FROM pages WHERE source_id='default' AND slug=$1", [slug]))[0];const fmOf = async (engine: BrainEngine, slug: string) => { const fm = (await page(engine, slug))!.frontmatter; return (typeof fm === 'string' ? JSON.parse(fm) : fm) as Record<string, unknown>; };@@ -162,6 +180,36 @@ for (const backend of testBackends()) {await expectCarried(engine, root, k, files);}, { setup, databaseUrl }), 300_000);+ test('purge-deleted --dry-run lists the parked origin the consented run releases and labels nothing; a refused consent labels nothing; --yes purges exactly that list', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ const k = 'preview', origin = slugOf(k, 'origin'), digest = slugOf(k, 'digest');+ const files = await before(engine, root, k);+ git(root, 'rm', '-q', `${origin}.md`); commit(root, 'remove the origin');+ await performManagedSync(engine, OPTS);+ await age(engine, 100);+ const failing = spyOn(preparedMaintenance, 'publishMaintenancePages').mockImplementation(async () => { throw new Error('forced label failure'); });+ try { expect((await purgeDeletedPagesCoordinated(engine, 72, { automatic: true })).slugs).not.toContain(origin); }+ finally { failing.mockRestore(); }+ const untouched = async () => {+ await settleManagedGitEffects(engine);+ expect(readFileSync(join(root, `${digest}.md`), 'utf8')).toBe(files.digest!);+ expect((await fmOf(engine, digest)).visibility).toBeUndefined();+ expect((await readDerivedOriginParks(engine)).map(park => park.slug)).toContain(origin);+ expect((await page(engine, origin))?.deleted_at).not.toBeNull();+ };+ await untouched();+ const preview = await cliJson(engine, ['purge-deleted', '--dry-run', '--json']);+ expect(preview.exit).toBe(0);+ expect(preview.out!.slugs).toContain(origin);+ await untouched();+ expect((await cliJson(engine, ['purge-deleted', '--json'])).exit).toBe(3);+ await untouched();+ const run = await cliJson(engine, ['purge-deleted', '--json', '--yes']);+ expect(run.exit).toBe(0);+ expect([...run.out!.slugs].sort()).toEqual([...preview.out!.slugs].sort());+ await expectCarried(engine, root, k, files);+ }, { setup, databaseUrl }), 300_000);+test('the managed retention purge rechecks inside its transaction: a derived page written after the carry parks the origin and skips its row', () => managedBrain(async ({ engine, root }) => {await performManagedSync(engine, OPTS);const k = 'midpurge', origin = slugOf(k, 'origin'), late = `notes/${k}-late`;diff --git a/test/derived-origin-carry-private.test.ts b/test/derived-origin-carry-private.test.tsindex 137bce55e4..f705edf522 100644--- a/test/derived-origin-carry-private.test.ts+++ b/test/derived-origin-carry-private.test.ts@@ -35,6 +35,9 @@ import { setOwner } from '../src/core/access/owner.ts';import { bindCredential } from '../src/core/access/members.ts';import { consentedPurgeDeletedPages, purgeDeletedPagesCoordinated, selectPurgeCandidates } from '../src/core/persistence/purge-deleted.ts';import { checkDerivedOriginParked } from '../src/commands/doctor/checks/purge-parks.ts';+import { runPages } from '../src/commands/pages.ts';+import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';+import { readDerivedOriginParks } from '../src/core/persistence/derived-origin-hold.ts';import { installFaultHook } from '../src/core/persistence/fault-points.ts';import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';import * as labelWrite from '../src/core/repair/label-write.ts';@@ -394,6 +397,54 @@ for (const kind of testBackends()) {}}+ /** `gbrain pages purge-deleted <args>` as the CLI runs it: its JSON output and exit verdict. */+ async function pagesCli(args: string[]): Promise<{ out: Record<string, any> | null; exit: number }> {+ _resetCliExitVerdictForTests();+ const prior = process.exitCode;+ const lines: string[] = [];+ const log = spyOn(console, 'log').mockImplementation((...parts: unknown[]) => { lines.push(parts.join(' ')); });+ const err = spyOn(console, 'error').mockImplementation(() => {});+ try { await withEnv({ GBRAIN_CONSENT: undefined, GBRAIN_HOME: home }, () => runPages(engine, ['purge-deleted', ...args])); }+ finally { log.mockRestore(); err.mockRestore(); }+ const exit = currentExitCode();+ _resetCliExitVerdictForTests();+ process.exitCode = prior;+ const text = lines.join('\n').trim();+ return { out: text.startsWith('{') ? JSON.parse(text) : null, exit };+ }++ test('purge-deleted --dry-run lists the parked origin a consented run releases and writes nothing; refused consent writes nothing; --yes purges exactly that list', async () => {+ const k = 'preview';+ await seed(k);+ const origin = slugOf(k, 'origin'), blocking = slugOf(k, 'digest'), plain = slugOf(k, 'plain');+ await parkByFailedLabel(origin);+ await softDelete(plain);+ await backdate(origin);+ await backdate(plain);+ const untouched = async () => {+ expect((await fm(blocking)).visibility).toBeUndefined();+ expect((await readDerivedOriginParks(engine)).map(park => park.slug)).toContain(origin);+ expect((await row(origin))?.deleted_at).not.toBeNull();+ expect((await row(plain))?.deleted_at).not.toBeNull();+ };+ const preview = await pagesCli(['--dry-run', '--json']);+ expect(preview.exit).toBe(0);+ expect(preview.out!.slugs).toEqual(expect.arrayContaining([origin, plain]));+ await untouched();+ const refused = await pagesCli(['--json']);+ expect(refused.exit).toBe(3);+ await untouched();+ await expect(consentedPurgeDeletedPages(engine, 72, false)).rejects.toThrow();+ await untouched();+ const run = await pagesCli(['--json', '--yes']);+ expect(run.exit).toBe(0);+ expect([...run.out!.slugs].sort()).toEqual([...preview.out!.slugs].sort());+ expect((await fm(blocking)).visibility).toBe('private');+ await nothingKept(origin);+ expect(await row(plain)).toBeUndefined();+ await expectCarriedHidden(k);+ }, 240_000);+test('an explicit purge rechecks inside its transaction: a derived page written after the carry keeps the origin soft-deleted and parked', async () => {const k = 'midsingle';await seed(k);