Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(doctor): embedding_state warns only on deleted-page chunks the purge would remove (#7825)

garrytan14252a4557cf
diff --git a/docs/architecture/key-files/commands-2-continued.md b/docs/architecture/key-files/commands-2-continued.md
index 8b8acc9553..419eb91164 100644
--- a/docs/architecture/key-files/commands-2-continued.md
+++ b/docs/architecture/key-files/commands-2-continued.md
@@ -7,7 +7,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/commands/doctor/checks/routing-federation.ts` — `checkOauthClientScopeHealth` (`oauth_client_scope_health`, re-exported by the doctor.ts façade) has three warn arms: (a) dangling federated read grants, (b) orphaned empty `<name>-workspace` sources, (c) privileged self-registered clients — active `oauth_clients` rows whose scope word-matches `DCR_PRIVILEGED_SCOPES` — the DERIVED complement `ALLOWED_SCOPES_LIST \ DCR_REGISTRABLE_SCOPES` (exported; bound as a `$1` regex parameter, never interpolated) — and the `(read/write)` / `--scopes read,write` copy is built from `DCR_REGISTRABLE_SCOPES`; pinned by the set-equality + per-scope loop in `test/doctor-oauth-client-scope-health.test.ts`; rows with `grant_revision = 0` and no `oauth_grant_audit` row with `action = 'register'` (every operator path writes one; the anonymous DCR insert does not). Arm (c) is advisory (the signature also matches operator clients that predate the grant audit log) and carries the `gbrain auth rescope-client <client_id> --scopes read,write` / `gbrain auth revoke-client <client_id>` remedy; when the audit table or `grant_revision` column is absent it fails OPEN and the ok message states the self-registered-client audit was skipped (unknown). Pinned by `test/doctor-oauth-client-scope-health.test.ts`.
- `src/commands/doctor/checks/embedding-identity.ts` — `embedding_identity_unqualified` (#6113, BRAIN, right after `embedding_env_override`): warns when the DB `config` row `embedding_model` has no provider prefix (managed fact extraction reads only that row and `config set` refuses it). The fix is the read-only `gbrain migrate embeddings --to <provider:model> --dry-run`, filled from `loadConfig()`'s provider-qualified `embedding_model`, or `<PROVIDER_MODEL>` plus an `inputs` entry when none is configured; `details.target_differs` flags a configured model with another name (applying it switches models). Applying re-embeds active facts (paid, user's call). The free relabel is parked (TODOS). Test: `test/doctor-embedding-identity-unqualified.test.ts`.
-- `src/commands/doctor/checks/embedding-state.ts` — `embedding_state` (#6696 B3, BRAIN, `tier: 'ops'`, `opsPglite: 'not_applicable'`, after `embeddings`): `readEmbeddingState` in `ops` mode on the ops tier, `full` otherwise. Warns only on chunks of pages soft-deleted past the 72 h purge window (`deleted_page_chunks_overdue`, fix `pages purge-deleted`) and, on a keyed brain, live pages with body text but no chunks (`chunkless_pages_healable`, fix `embed --stale`); the pending backlog stays `embeddings`' warning and quarantined pages `quarantined_pages`'. keyword_only, deleted chunks inside the window and a keyless brain are `infoCheck`; ops without a covering index warns with `details.health: 'unknown'` (`embedding_state_unknown`). Test: `test/doctor-embedding-state.test.ts`.
+- `src/commands/doctor/checks/embedding-state.ts` — `embedding_state` (#6696 B3, BRAIN, `tier: 'ops'`, `opsPglite: 'not_applicable'`, after `embeddings`): `readEmbeddingState` in `ops` mode on the ops tier, `full` otherwise. Warns only on purgeable chunks of pages soft-deleted past the 72 h window (`deleted_page_chunks_overdue`, fix `pages purge-deleted`) and, on a keyed brain, live pages with body text but no chunks (`chunkless_pages_healable`, fix `embed --stale`); the pending backlog stays `embeddings`' warning and quarantined pages `quarantined_pages`'. keyword_only, held or in-window deleted chunks and a keyless brain are `infoCheck`; ops without a covering index warns with `details.health: 'unknown'` (`embedding_state_unknown`). Test: `test/doctor-embedding-state.test.ts`.
- `src/commands/doctor/checks/knowledge-health.ts` — `facts_extraction_health` reads the last 24 h of `facts:absorb` rows in `ingest_log` (written by `src/core/facts/absorb-log.ts`) per source and reason (`split_part(summary, ':', 1)`) and warns when a failure reason reaches `facts.absorb_warn_threshold` (default 10) in a source. Two reasons are not failures and never count: `write_pending_accepted` (the absorb log read the facts write's receipt back from the journal and it is still queued, running or recovering after the job's publish wait) and `write_deferred_unadmitted` (`writer_lock_unavailable` before admission, no receipt; the job retries). Deferrals warn on their own when, in a source, more than 10 or one over an hour old have no later completed `facts-absorb` job for the same page; that warning's fix is the read-only `gbrain sources writer status --source <id> --json`. The failure count is history: the message gives the incidents in the rolling 24 h window and when the newest ages out, and `details.facts_absorb` (`incidents_24h`, `ages_out_at`, `jobs_still_dead`, `dead_jobs`) adds the current state, the `facts-absorb` jobs dead or failed in the last 24 h with no later completed job for the same page (source and slug); they never change the status, and when no other fix applies the newest one gets the paid `gbrain jobs retry <id>` fix (`retry_job` over MCP, asks first). Tests: `test/doctor-facts-absorb-contention-6778.test.ts`, `test/facts-absorb-publish-wait-6778.test.ts`, `test/doctor-facts-absorb-history-6761.test.ts`.
- `src/commands/doctor/checks/persistence-consumers.ts` — #6317 (B3, T1c): doctor `two_consumers_on_host` (warn only when two live rows of resident kinds on this host have both been alive longer than 30 s; every other row, a `put` or a row younger than 30 s, is listed in `details`) and `consumers_without_heartbeat` (a `claim_phase.owner` pid on a running claim of this host with no heartbeat row: an older process that never defers; named with its kind and version from the stamp). Both carry a structured `agentFix` (`gbrain sources writer status --json`) and `doctorVerify`; with `persistence.single_consumer` off the overlap is reported as the configured behavior. `serve_build_drift` (#6696 C6, its own `serveBuildDriftEntry`, one `listHostConsumers` read, ok on PGLite with no read) compares the live resident rows (not this process) with this binary's `VERSION` through `compareWriterVersions` (an unparsable version counts as older): an older row warns "serve is running an older build than this checkout; restart serve and workers" with `details { count, this_version, owner, older, newer }` and a `fix` (actor `user`, `doctorVerify`) whose argv is the installed serve service's restart (`systemctl --user restart gbrain-serve.service` when `systemdUnitPath` exists on Linux, `launchctl kickstart -k gui/<uid>/com.gbrain.serve` when `launchdPlistPath` exists on macOS; `serveBuildDriftCheck`'s `ServiceHost` argument injects platform, home and uid) or else `kill <pid>`; a newer row is reported in an ok message; no rows names `consumers_without_heartbeat` for a pre-heartbeat serve. Registered in `DOCTOR_CHECK_REGISTRY` (the consumer entry, then `serveBuildDriftEntry`) before `searchModeEntry` and categorised `ops` in `src/core/doctor-categories.ts`. Tests: `test/doctor-serve-build-drift.test.ts`.
diff --git a/docs/architecture/key-files/memory-purge.md b/docs/architecture/key-files/memory-purge.md
index e9fd67d541..528974ee76 100644
--- a/docs/architecture/key-files/memory-purge.md
+++ b/docs/architecture/key-files/memory-purge.md
@@ -22,7 +22,7 @@ User-facing contract: [expire versus purge](../../guides/memory-boundaries.md#pu
- Withdrawn-claim refusal on `remember` (`persistence/memory-mutations.ts`, `memory-prepare.ts`) — admission refuses a withdrawn claim (`fact_withdrawn`, `FACT_WITHDRAWN_MESSAGE`/`_SUGGESTION` in `src/core/facts/withdrawal.ts`) before a request exists, and preparation re-checks it; `writeFailureDiagnostic` allowlists that prefix, so a failed receipt and `get_write_request` name it. `replaces` accepts a target saved without an entity (links it by supersession). Test: `test/remember-withdrawn-claim.test.ts`. (R1, Cat 40 Hard.) The refusal carries an additive `withdrawn` object (`factWithdrawnRefusal` in `facts/single-prepare.ts`, from `fact_withdrawals` only: the named entity's row over `all_entities`, then the newest, omitted when the caller cannot read the forgotten rows); `withdrawalFailureDetail`/`withdrawnFromDetail` in `facts/withdrawal.ts` keep it in the failed receipt's `error_detail` for `get_write_request` and replay. Tests: `test/withdrawn-detail.test.ts`, `test/e2e/withdrawn-detail-postgres.test.ts`.
-- `src/core/persistence/purge-deleted.ts` — `purgeDeletedPagesCoordinated` backs the cycle `purge` phase, the purge job, `purge_deleted_pages` and `gbrain pages purge-deleted`. Unmanaged brains keep the engine bulk delete, after the belief-ledger expiry of recorded sources' tombstones: each is its own transaction and item (ItemRun site `purge.recorded_tombstones`), a run-scoped failure or a systemic run stops the purge, and any other failure lists the tombstone in `blocked` with its classified code, counts it in `failed` (no `error`, so the phase warns) and keeps it out of the bulk delete (`purgeDeletedPages` `excludeIds`); a content failure counts toward the same park as the managed purge, a transient one is retried next run, and parked tombstones are skipped as `purge_parked` unless `retryParked` (the unmanaged preview excludes them too). A managed brain purges each tombstone older than the window (skipping archived sources, which the source lifecycle purges) through `delete_page` with `purge: true` and the internal `retention: true` bound to its revision (retention is not an owner purge; see [belief ledger](belief-ledger.md)). Writer state is read once per run (`writerOnboardingPreflight`): a source whose writer is `recovery_required` submits nothing and its tombstones are listed in `blocked` as `writer_not_active` (counted in `inactive`). Revision races are deferred to the next run; a parked tombstone (`purge-parks.ts`) is skipped as `purge_parked` (counted in `parked`) unless `retryParked`, and any other failure is listed in `blocked` and sets `error`. `selectPurgeCandidates` is the one eligibility selector (preview, consent, `purge_deleted_pages` op, run); #6884: the run submits only while the writer has outstanding-request room (`readOutstandingRoom` in `journal.ts`, at `persistence.capacity.background_pause_pct` of the cap for a background run); the rest, and the rest after a `queue_capacity` refusal, is `blocked` as `write_capacity_deferred` and counted in `capacity_deferred`, never a failure or a park. `purgePhaseStatus`/`purgePhaseNote` shape the cycle purge phase (fail on `error`; warn on `writer_not_active`, parked, capacity-deferred or an unmanaged `failed` tombstone). The engine bulk delete (`engine-sql/pages.ts` `purgeDeletedPages`) also deletes, in its transaction, the purged pages' stagings (by source and slug) and the segment texts only their manifests referenced. Tests: `test/managed-writers-w3.test.ts`, `test/purge-deleted-parks.test.ts`, `test/purge-deleted-capacity-6884.test.ts`, `test/pathological/a49-unmanaged-purge-tombstones.test.ts`.
+- `src/core/persistence/purge-deleted.ts` — `purgeDeletedPagesCoordinated` backs the cycle `purge` phase, the purge job, `purge_deleted_pages` and `gbrain pages purge-deleted`. Unmanaged brains keep the engine bulk delete, after the belief-ledger expiry of recorded sources' tombstones: each is its own transaction and item (ItemRun site `purge.recorded_tombstones`), a run-scoped failure or a systemic run stops the purge, and any other failure lists the tombstone in `blocked` with its classified code, counts it in `failed` (no `error`, so the phase warns) and keeps it out of the bulk delete (`purgeDeletedPages` `excludeIds`); a content failure counts toward the same park as the managed purge, a transient one is retried next run, and parked tombstones are skipped as `purge_parked` unless `retryParked` (the unmanaged preview excludes them too). A managed brain purges each tombstone older than the window (skipping archived sources, which the source lifecycle purges) through `delete_page` with `purge: true` and the internal `retention: true` bound to its revision (retention is not an owner purge; see [belief ledger](belief-ledger.md)). Writer state is read once per run (`writerOnboardingPreflight`): a source whose writer is `recovery_required` submits nothing and its tombstones are listed in `blocked` as `writer_not_active` (counted in `inactive`). Revision races are deferred to the next run; a parked tombstone (`purge-parks.ts`) is skipped as `purge_parked` (counted in `parked`) unless `retryParked`, and any other failure is listed in `blocked` and sets `error`. `selectPurgeCandidates` is the one eligibility selector (preview, consent, `purge_deleted_pages` op, run), and its holds are `purgeHoldsSql` (archived source, rename-identity, restorable-atom and derived-retirement holds), which doctor `embedding_state` also counts with; #6884: the run submits only while the writer has outstanding-request room (`readOutstandingRoom` in `journal.ts`, at `persistence.capacity.background_pause_pct` of the cap for a background run); the rest, and the rest after a `queue_capacity` refusal, is `blocked` as `write_capacity_deferred` and counted in `capacity_deferred`, never a failure or a park. `purgePhaseStatus`/`purgePhaseNote` shape the cycle purge phase (fail on `error`; warn on `writer_not_active`, parked, capacity-deferred or an unmanaged `failed` tombstone). The engine bulk delete (`engine-sql/pages.ts` `purgeDeletedPages`) also deletes, in its transaction, the purged pages' stagings (by source and slug) and the segment texts only their manifests referenced. Tests: `test/managed-writers-w3.test.ts`, `test/purge-deleted-parks.test.ts`, `test/purge-deleted-capacity-6884.test.ts`, `test/pathological/a49-unmanaged-purge-tombstones.test.ts`.
- `src/core/persistence/derived-retirement-hold.ts` — GBRA-161: the purge hold for every derived retirement. `DERIVED_RETIRED_BY` lists the `retired_by` stamps derived writers put on the pages they retire (extract-atoms, managed-reextract, stale-atoms, life-chronicle); `derivedRetirementPurgeHold(alias)` keeps a stamped page out of every purge route (engine bulk purge and its preview, the coordinated selection and delete) until `DERIVED_RETIREMENT_HOLD_DAYS` (30) after `deleted_at` (the database clock, never the `retired_at` text); `retireDerivedPages` versions, soft-deletes and stamps a batch in one attributed transaction (unmanaged extract_atoms). `assertRetirementStampUnchanged` is the page preparer's refusal (`reserved_frontmatter_key`) of a remote write that adds or changes one of `RESERVED_RETIREMENT_KEYS`. The hold length follows `generations.restore_days` (`RESTORE_DAYS_SQL`, read inside the purge statement; 30 by default) and `retireDerivedPagesInTransaction` is the in-transaction form generation adapters retire through. An explicit `gbrain delete --purge` is not held.
- `src/core/generations/ledger-scrub.ts` — GBRA-161: purge reaches the generation ledger. A page purge (`page-purge.ts`) nulls the purged item's preimages in `derived_generation_items` and every held proposal in `derived_generations` naming it; a fact purge (`facts/purge.ts`) nulls every preimage, proposal and held context carrying the claim. Both are `swept` (`generation_ledger`) in the deletion inventory, so the verification probes them; `gbrain repair generations` then refuses a scrubbed item with `no_preimage`.
- `src/core/persistence/atom-retirement-hold.ts` — #7013: the one SQL rule for atoms `gbrain repair atoms-restore` restores. `atomRetirementsSql(scope)` is set-based (CTEs over `pages` joined on source and origin key, linear in the atoms in scope) and returns every non-imported atom soft-deleted with `retired_by: managed-reextract` with `origin_live` (a live page origin in its source, or a transcript), `replaced` (a live atom of the same origin carries another stripped source hash), `latest` (its hash is the hash of the origin's latest `managed-reextract` retirement) and `restorable` (all three right); `atomOriginScopeSql` narrows it to one atom's source and origin through the frontmatter GIN index for the apply recheck; `atomRetirementPurgeHold`, which keeps such an atom out of every batch purge selection (engine `purgeDeletedPages` preview and delete, the belief-ledger expiry, `selectPurgeCandidates`) for `ATOM_RESTORE_HOLD_DAYS` (30) after `deleted_at`, next to the #6729 `identityDowngradePurgeHold`. An explicit `delete --purge` is not held. Static strings only, so `engine-sql/pages.ts` imports it.
diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.md
index 85da49e76e..9e921a862f 100644
--- a/docs/guides/error-codes.md
+++ b/docs/guides/error-codes.md
@@ -1049,7 +1049,7 @@ More: [docs/ENGINES.md#serve-deep-health](../../docs/ENGINES.md#serve-deep-healt
| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable |
|---|---|---|---|---|---|---|
-| Chunks of pages soft-deleted longer than the purge window (72h) are still stored. | Search never reads them; the purge lifecycle (`gbrain pages purge-deleted`, the autopilot purge phase) hard-deletes such pages and their chunks. The purge is brain-wide and irreversible, so it needs the user. | Preview with `gbrain pages purge-deleted --dry-run --json`, then run `gbrain pages purge-deleted --yes` once the user agrees. Run: gbrain pages purge-deleted --dry-run --json | agent | `gbrain doctor --only embedding_state --json` | 1 | no |
+| Chunks of pages soft-deleted longer than the purge window (72h), which the purge would remove, are still stored. | Search never reads them; the purge lifecycle (`gbrain pages purge-deleted`, the autopilot purge phase) hard-deletes such pages and their chunks. Pages its holds keep (a derived retirement inside generations.restore_days, a restorable atom, an open rename-identity pair, an archived source) are not counted. The purge is brain-wide and irreversible, so it needs the user. | Preview with `gbrain pages purge-deleted --dry-run --json`, then run `gbrain pages purge-deleted --yes` once the user agrees. Run: gbrain pages purge-deleted --dry-run --json | agent | `gbrain doctor --only embedding_state --json` | 1 | no |
More: [docs/guides/troubleshooting.md#embedding-state](../../docs/guides/troubleshooting.md#embedding-state)
diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md
index 1638111290..3d1011c756 100644
--- a/docs/guides/troubleshooting.md
+++ b/docs/guides/troubleshooting.md
@@ -431,7 +431,7 @@ derived from the rows against the active embedding column (no stored status):
| `keyword_only` | The page carries `embed_skip`: the import gate found its body over `content_sanity.bytes_block` (info, by design; its chunks still feed keyword search and are never purged). Removing the marker alone is undone on the next import. | A marker under the current `bytes_block` (written by an older release under its 500 KB default): `gbrain reindex --oversized --source <id>` clears it for free. Otherwise split the page, or, after the operator approves, `gbrain config set content_sanity.bytes_block <bytes>` then `gbrain sync --source <id> --full`, which re-imports and embeds it. |
| `quarantined` | The content-quality gate hid the page as junk; it has no searchable chunks. Doctor `quarantined_pages` owns its warning. | Review with `gbrain quarantine list --json`; clear a false positive with `gbrain quarantine clear <slug> --source-id <id> --force` (the owner's call). |
| `empty` | A live page with no body text: nothing to index. | Write content to it, or delete the stub (`gbrain delete <slug> --source <id> --force`, restorable) after the user agrees. |
-| `deleted_page_chunks` | Chunks of a soft-deleted page. Search never reads them; the purge lifecycle removes them after 72 h (info until then, a warning past it, code `deleted_page_chunks_overdue`). | `gbrain pages purge-deleted --dry-run --json`, then `gbrain pages purge-deleted --yes` once the user agrees (brain-wide, irreversible). |
+| `deleted_page_chunks` | Chunks of a soft-deleted page. Search never reads them; the purge lifecycle removes them after 72 h (info until then, a warning past it, code `deleted_page_chunks_overdue`). Chunks the purge holds stay info past 72 h: a derived retirement for `generations.restore_days` (30 days by default), a restorable atom, an open rename-identity pair or an archived source. | `gbrain pages purge-deleted --dry-run --json`, then `gbrain pages purge-deleted --yes` once the user agrees (brain-wide, irreversible). |
| `chunkless_healable` | A live page with body text but no chunk rows (written without the chunking step, or an `embed_skip` page an older gbrain stored without chunks or whose `reindex --oversized` rewrite failed). Keyword search cannot find it. Doctor warns on a keyed brain (code `chunkless_pages_healable`). | `gbrain embed --stale --catch-up --source <id> --yes` chunks and embeds it (paid); an `embed_skip` page gets chunks and no vectors. The cycle's embed phase runs the same safety net. |
`unsearchable` sums `pending`, `quarantined`, `empty` and `chunkless_healable`;
diff --git a/llms-full.txt b/llms-full.txt
index ffafd12558..74251e841d 100644
--- a/llms-full.txt
+++ b/llms-full.txt
@@ -10352,7 +10352,7 @@ derived from the rows against the active embedding column (no stored status):
| `keyword_only` | The page carries `embed_skip`: the import gate found its body over `content_sanity.bytes_block` (info, by design; its chunks still feed keyword search and are never purged). Removing the marker alone is undone on the next import. | A marker under the current `bytes_block` (written by an older release under its 500 KB default): `gbrain reindex --oversized --source <id>` clears it for free. Otherwise split the page, or, after the operator approves, `gbrain config set content_sanity.bytes_block <bytes>` then `gbrain sync --source <id> --full`, which re-imports and embeds it. |
| `quarantined` | The content-quality gate hid the page as junk; it has no searchable chunks. Doctor `quarantined_pages` owns its warning. | Review with `gbrain quarantine list --json`; clear a false positive with `gbrain quarantine clear <slug> --source-id <id> --force` (the owner's call). |
| `empty` | A live page with no body text: nothing to index. | Write content to it, or delete the stub (`gbrain delete <slug> --source <id> --force`, restorable) after the user agrees. |
-| `deleted_page_chunks` | Chunks of a soft-deleted page. Search never reads them; the purge lifecycle removes them after 72 h (info until then, a warning past it, code `deleted_page_chunks_overdue`). | `gbrain pages purge-deleted --dry-run --json`, then `gbrain pages purge-deleted --yes` once the user agrees (brain-wide, irreversible). |
+| `deleted_page_chunks` | Chunks of a soft-deleted page. Search never reads them; the purge lifecycle removes them after 72 h (info until then, a warning past it, code `deleted_page_chunks_overdue`). Chunks the purge holds stay info past 72 h: a derived retirement for `generations.restore_days` (30 days by default), a restorable atom, an open rename-identity pair or an archived source. | `gbrain pages purge-deleted --dry-run --json`, then `gbrain pages purge-deleted --yes` once the user agrees (brain-wide, irreversible). |
| `chunkless_healable` | A live page with body text but no chunk rows (written without the chunking step, or an `embed_skip` page an older gbrain stored without chunks or whose `reindex --oversized` rewrite failed). Keyword search cannot find it. Doctor warns on a keyed brain (code `chunkless_pages_healable`). | `gbrain embed --stale --catch-up --source <id> --yes` chunks and embeds it (paid); an `embed_skip` page gets chunks and no vectors. The cycle's embed phase runs the same safety net. |
`unsearchable` sums `pending`, `quarantined`, `empty` and `chunkless_healable`;
diff --git a/src/commands/doctor/checks/embedding-state.ts b/src/commands/doctor/checks/embedding-state.ts
index b01c39fc0e..a1ef8e6767 100644
--- a/src/commands/doctor/checks/embedding-state.ts
+++ b/src/commands/doctor/checks/embedding-state.ts
@@ -5,7 +5,8 @@
* (unknown without a covering partial index); every other tier counts exactly.
*
* It warns only on what no other check owns: chunks of pages soft-deleted
- * longer than the purge window (fix: `gbrain pages purge-deleted`), and live
+ * longer than the purge window that the purge would remove (fix: `gbrain
+ * pages purge-deleted`; chunks its holds keep are information), and live
* pages with body text but no chunk rows on a keyed brain (fix: `gbrain embed
* --stale`). The pending backlog stays the `embeddings` check's warning (it
* ages the backlog; this check only counts it), and quarantined pages stay
@@ -37,7 +38,7 @@ async function runEmbeddingState(ctx: DoctorContext): Promise<Check[]> {
return checks;
}
const counts = state.counts;
- const deleted = state.deleted_page_chunks as { within_window: number; past_window: number; window_hours: number };
+ const deleted = state.deleted_page_chunks as { within_window: number; past_window: number; purgeable: number; held: number; window_hours: number };
const enabled = state.embeddings_enabled === true;
const healable = enabled ? counts.chunkless_healable ?? 0 : 0;
const listed = (Object.keys(LABELS) as EmbeddingState[]).filter(s => counts[s] !== undefined).map(s => `${counts[s]} ${LABELS[s]}`).join(', ');
@@ -47,19 +48,20 @@ async function runEmbeddingState(ctx: DoctorContext): Promise<Check[]> {
if ((counts.quarantined ?? 0) > 0) notes.push('Quarantined pages are reviewed under quarantined_pages.');
if ((counts.keyword_only ?? 0) > 0) notes.push('Keyword-only chunks sit on embed_skip pages by design (split the page or approve a higher content_sanity.bytes_block; `gbrain pages unsearchable --state keyword_only` lists them).');
if (deleted.within_window > 0) notes.push(`${deleted.within_window} chunk(s) on pages soft-deleted within ${deleted.window_hours}h wait for the purge lifecycle.`);
- if (deleted.past_window > 0 || healable > 0) {
+ if (deleted.held > 0) notes.push(`${deleted.held} chunk(s) on pages soft-deleted more than ${deleted.window_hours}h ago are held from the purge by design (a derived retirement inside generations.restore_days, a restorable atom, an open rename-identity pair or an archived source).`);
+ if (deleted.purgeable > 0 || healable > 0) {
const warnings: string[] = [];
- if (deleted.past_window > 0) warnings.push(`${deleted.past_window} chunk(s) belong to pages soft-deleted more than ${deleted.window_hours}h ago; \`gbrain pages purge-deleted\` removes them (preview with --dry-run --json; it is brain-wide and irreversible, so ask the user first).`);
+ if (deleted.purgeable > 0) warnings.push(`${deleted.purgeable} chunk(s) belong to pages soft-deleted more than ${deleted.window_hours}h ago; \`gbrain pages purge-deleted\` removes them (preview with --dry-run --json; it is brain-wide and irreversible, so ask the user first).`);
if (healable > 0) warnings.push(`${healable} live page(s) have body text but no chunk rows, so search cannot read their body; \`gbrain embed --stale\` chunks and embeds them (paid).`);
- const fix = embeddingStateFix(deleted.past_window > 0 ? 'deleted_page_chunks' : 'chunkless_healable', { embeddingsEnabled: enabled, verify: doctorVerify('embedding_state') });
+ const fix = embeddingStateFix(deleted.purgeable > 0 ? 'deleted_page_chunks' : 'chunkless_healable', { embeddingsEnabled: enabled, verify: doctorVerify('embedding_state') });
checks.push({ name: 'embedding_state', status: 'warn', message: [summary, ...warnings, ...notes].join(' '),
- details: { ...details, code: deleted.past_window > 0 ? 'deleted_page_chunks_overdue' : 'chunkless_pages_healable' }, ...(fix ? { fix } : {}) });
+ details: { ...details, code: deleted.purgeable > 0 ? 'deleted_page_chunks_overdue' : 'chunkless_pages_healable' }, ...(fix ? { fix } : {}) });
return checks;
}
const message = [summary, ...notes].join(' ');
if (!enabled) {
checks.push(infoCheck('embedding_state', message, 'disabled_by_choice', keylessEnablementFix(), details));
- } else if ((counts.keyword_only ?? 0) > 0 || deleted.within_window > 0) {
+ } else if ((counts.keyword_only ?? 0) > 0 || deleted.within_window > 0 || deleted.held > 0) {
checks.push(infoCheck('embedding_state', message, (counts.keyword_only ?? 0) > 0 ? 'disabled_by_choice' : 'not_applicable', undefined, details));
} else {
checks.push({ name: 'embedding_state', status: 'ok', message, details });
diff --git a/src/core/embedding-state.ts b/src/core/embedding-state.ts
index 205aefb786..861bae63f9 100644
--- a/src/core/embedding-state.ts
+++ b/src/core/embedding-state.ts
@@ -50,6 +50,7 @@ import { DEFAULT_BYTES_BLOCK } from './content-sanity.ts';
import { loadConfig } from './config.ts';
import { embeddingEnablement } from './readiness.ts';
import { VERSION } from '../version.ts';
+import { chunkGenerationFilter } from './search/safe-chunks.ts';
export type EmbeddingState = 'embedded' | 'pending' | 'keyword_only' | 'quarantined' | 'empty' | 'deleted_page_chunks' | 'chunkless_healable';
@@ -128,6 +129,7 @@ export async function readEmbeddingState(engine: BrainEngine, opts: { mode: 'ops
const counts = raw.counts;
const pending = counts.pending ?? 0;
const threshold = await effectiveBlockThreshold(engine);
+ const purgeable = raw.deleted_overdue ? Math.min(raw.deleted_overdue, await purgeableDeletedChunks(engine, opts)) : 0;
return {
status: 'known',
basis: raw.basis,
@@ -145,7 +147,8 @@ export async function readEmbeddingState(engine: BrainEngine, opts: { mode: 'ops
? { basis: 'estimated', ratio: ratio(counts.embedded, pending) }
: { basis: 'exact', embedded: raw.eligible_embedded, eligible: raw.eligible_embedded + pending, ratio: ratio(raw.eligible_embedded, pending) },
keyword_only: { reason: 'embed_skip', pages: raw.embed_skip_pages, marker_bytes: raw.embed_skip_marker_bytes, threshold_bytes: threshold },
- deleted_page_chunks: { within_window: (counts.deleted_page_chunks ?? 0) - raw.deleted_overdue, past_window: raw.deleted_overdue, window_hours: DELETED_PAGE_PURGE_WINDOW_HOURS },
+ deleted_page_chunks: { within_window: (counts.deleted_page_chunks ?? 0) - raw.deleted_overdue, past_window: raw.deleted_overdue,
+ purgeable, held: raw.deleted_overdue - purgeable, window_hours: DELETED_PAGE_PURGE_WINDOW_HOURS },
unstamped_pending: { count: raw.unstamped_pending, created_at_lower_bound: raw.unstamped_created_at_min?.toISOString() ?? null, writer_version: VERSION },
};
}
@@ -155,6 +158,17 @@ function ratio(embedded: number | undefined, pending: number): number | null {
return total === 0 ? null : Math.round(((embedded ?? 0) / total) * 10_000) / 10_000;
}
+/** #7825: chunks past the window that `gbrain pages purge-deleted` would remove, under the purge's own holds (`purgeHoldsSql`); counted as the classifier counts them. */
+async function purgeableDeletedChunks(engine: BrainEngine, opts: { mode: 'ops' | 'full'; sourceIds?: string[] }): Promise<number> {
+ const { purgeHoldsSql } = await import('./persistence/purge-deleted.ts');
+ const [row] = await engine.executeRaw<{ n: string | number }>(`SELECT count(*)::bigint AS n
+ FROM pages p JOIN sources s ON s.id = p.source_id JOIN content_chunks cc ON cc.page_id = p.id
+ WHERE p.deleted_at < now() - make_interval(hours => $1::int) AND ${purgeHoldsSql('p', 's')}
+ ${opts.mode === 'full' ? `AND ${chunkGenerationFilter('p', 'cc')}` : ''}${opts.sourceIds ? ' AND p.source_id = ANY($2::text[])' : ''}`,
+ opts.sourceIds ? [DELETED_PAGE_PURGE_WINDOW_HOURS, opts.sourceIds] : [DELETED_PAGE_PURGE_WINDOW_HOURS]);
+ return Number(row?.n ?? 0);
+}
+
/** The effective `content_sanity.bytes_block` the import gate applies (env > file > DB > default, capped at the page ceiling). */
async function effectiveBlockThreshold(engine: BrainEngine): Promise<number> {
try {
diff --git a/src/core/error-registry.ts b/src/core/error-registry.ts
index 950eadc398..4c44cbdf01 100644
--- a/src/core/error-registry.ts
+++ b/src/core/error-registry.ts
@@ -225,7 +225,7 @@ export const CODES = {
suggestion: "Repeat the delta call with the cursor from fix (next_cursor.cursor) instead of since/since_slug, and keep passing next_cursor.cursor afterwards.",
docs: 'docs/guides/ambient-recall.md#replay-after-a-degraded-wake',
},
- deleted_page_chunks_overdue: { class: 'consent', summary: "Chunks of pages soft-deleted longer than the purge window (72h) are still stored.", why: 'Search never reads them; the purge lifecycle (`gbrain pages purge-deleted`, the autopilot purge phase) hard-deletes such pages and their chunks. The purge is brain-wide and irreversible, so it needs the user.', docs: 'docs/guides/troubleshooting.md#embedding-state', suggestion: 'Preview with `gbrain pages purge-deleted --dry-run --json`, then run `gbrain pages purge-deleted --yes` once the user agrees.', fix: { argv: ['gbrain', 'pages', 'purge-deleted', '--dry-run', '--json'], consent: [], actor: 'agent', why: 'Lists the soft-deleted pages the purge would remove; changes nothing.', requires_exclusive: false, verify: { argv: ['gbrain', 'doctor', '--only', 'embedding_state', '--json'] } } },
+ deleted_page_chunks_overdue: { class: 'consent', summary: "Chunks of pages soft-deleted longer than the purge window (72h), which the purge would remove, are still stored.", why: 'Search never reads them; the purge lifecycle (`gbrain pages purge-deleted`, the autopilot purge phase) hard-deletes such pages and their chunks. Pages its holds keep (a derived retirement inside generations.restore_days, a restorable atom, an open rename-identity pair, an archived source) are not counted. The purge is brain-wide and irreversible, so it needs the user.', docs: 'docs/guides/troubleshooting.md#embedding-state', suggestion: 'Preview with `gbrain pages purge-deleted --dry-run --json`, then run `gbrain pages purge-deleted --yes` once the user agrees.', fix: { argv: ['gbrain', 'pages', 'purge-deleted', '--dry-run', '--json'], consent: [], actor: 'agent', why: 'Lists the soft-deleted pages the purge would remove; changes nothing.', requires_exclusive: false, verify: { argv: ['gbrain', 'doctor', '--only', 'embedding_state', '--json'] } } },
demo_graph_failed: { class: 'server', summary: "The fictional graph did not verify." },
demo_import_failed: { class: 'server', summary: "The fictional page did not import successfully." },
demo_verification_failed: { class: 'server', summary: "Expected ownership or decision provenance is missing." },
diff --git a/src/core/persistence/purge-deleted.ts b/src/core/persistence/purge-deleted.ts
index 66ca381397..d8dae79512 100644
--- a/src/core/persistence/purge-deleted.ts
+++ b/src/core/persistence/purge-deleted.ts
@@ -187,6 +187,11 @@ export interface PurgeSelection {
writers: Record<string, boolean>;
}
+/** What keeps an expired tombstone of page `page` (source `source`) out of every managed purge: an archived source, or the rename-recovery, restorable-atom or derived-retirement hold. */
+export function purgeHoldsSql(page: string, source: string): string {
+ return `NOT ${source}.archived AND ${identityDowngradePurgeHold(page)} AND ${atomRetirementPurgeHold(page)} AND ${derivedRetirementPurgeHold(page)}`;
+}
+
export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours: number, opts: { retryParked?: boolean; automatic?: boolean } = {}): Promise<PurgeSelection> {
if (!await managedPersistenceEnabled(engine)) {
const parked = (await recordedTombstones(engine, String(Math.max(0, Math.floor(olderThanHours)))))
@@ -206,7 +211,7 @@ export async function selectPurgeCandidates(engine: BrainEngine, olderThanHours:
const rows = await engine.executeRaw<{ id: number | string; source_id: string; slug: string; cutoff: string; deleted_us: string }>(
`SELECT p.id, p.source_id, p.slug, (now() - ($1 || ' hours')::interval)::text AS cutoff,
floor(extract(epoch FROM p.deleted_at)*1000000)::bigint::text AS deleted_us FROM pages p JOIN sources s ON s.id = p.source_id
- WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND NOT s.archived AND ${identityDowngradePurgeHold('p')} AND ${atomRetirementPurgeHold('p')} AND ${derivedRetirementPurgeHold('p')}
+ WHERE p.deleted_at IS NOT NULL AND p.deleted_at < now() - ($1 || ' hours')::interval AND ${purgeHoldsSql('p', 's')}
ORDER BY p.deleted_at ASC, p.source_id ASC, p.slug ASC`, [String(hours)]);
const states = new Map<string, { state: string; physical_error?: string }>();
if (rows.length) for (const source of (await writerOnboardingPreflight(engine)).sources) states.set(source.source_id, source);
diff --git a/test/doctor-embedding-state.test.ts b/test/doctor-embedding-state.test.ts
index af3707c420..41299bee01 100644
--- a/test/doctor-embedding-state.test.ts
+++ b/test/doctor-embedding-state.test.ts
@@ -100,6 +100,20 @@ describe('doctor embedding_state', () => {
expect(c.fix?.verify?.argv).toEqual(['gbrain', 'doctor', '--only', 'embedding_state', '--json']);
});
+ test('#7825: chunks of a held derived retirement past the window are information, not a purge warning', async () => {
+ await chunk(await page('atoms/held-retirement', { deletedHoursAgo: 120, frontmatter: { retired_by: 'managed-reextract' } }), true);
+ const held = await run('standard');
+ expect(held.status).not.toBe('warn');
+ expect(held.details.deleted_page_chunks).toMatchObject({ past_window: 1, purgeable: 0, held: 1 });
+ expect(held.message).not.toContain('purge-deleted');
+ await chunk(await page('notes/deleted-old', { deletedHoursAgo: 100 }), true);
+ const mixed = await run('standard');
+ expect(mixed.status).toBe('warn');
+ expect(mixed.details.code).toBe('deleted_page_chunks_overdue');
+ expect(mixed.details.deleted_page_chunks).toMatchObject({ past_window: 2, purgeable: 1, held: 1 });
+ expect(mixed.message).toContain('1 chunk(s) belong to pages soft-deleted more than 72h ago');
+ });
+
test('a chunkless page with body text warns on a keyed brain and is info on a keyless one', async () => {
await page('notes/stub-with-body');
const keyed = await run('standard');
diff --git a/test/embedding-state.test.ts b/test/embedding-state.test.ts
index 08108aab7e..78b1b548a0 100644
--- a/test/embedding-state.test.ts
+++ b/test/embedding-state.test.ts
@@ -87,7 +87,7 @@ describe('embedding state classifier (full mode)', () => {
expect(Object.keys(s.counts).sort()).toEqual([...EMBEDDING_STATES].sort());
expect(s.unsearchable).toBe(2 + 3 + 1 + 2);
expect(s.unsearchable_complete).toBe(true);
- expect(s.deleted_page_chunks).toEqual({ within_window: 1, past_window: 2, window_hours: 72 });
+ expect(s.deleted_page_chunks).toEqual({ within_window: 1, past_window: 2, purgeable: 2, held: 0, window_hours: 72 });
// The default bytes_block is the 24 MiB page ceiling (large pages Wave A); these markers predate it.
expect(s.keyword_only).toMatchObject({ reason: 'embed_skip', pages: 2, marker_bytes: 1_300_000, threshold_bytes: PAGE_MAX_BYTES });
const unstamped = s.unstamped_pending as { count: number; created_at_lower_bound: string | null; writer_version: string };
diff --git a/test/fixtures/goldens/doctor/content-pglite-json.json b/test/fixtures/goldens/doctor/content-pglite-json.json
index 6274069f76..ffd48e3cae 100644
--- a/test/fixtures/goldens/doctor/content-pglite-json.json
+++ b/test/fixtures/goldens/doctor/content-pglite-json.json
@@ -784,7 +784,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
diff --git a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
index b979583551..2aa1e83364 100644
--- a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
+++ b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
@@ -806,7 +806,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
@@ -5687,7 +5689,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
diff --git a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
index 2a1c4acc2c..e1714b5925 100644
--- a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
+++ b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
@@ -804,7 +804,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
@@ -5686,7 +5688,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
diff --git a/test/fixtures/goldens/doctor/json-pglite-degraded.json b/test/fixtures/goldens/doctor/json-pglite-degraded.json
index a6bbad91d0..f25a9ec25e 100644
--- a/test/fixtures/goldens/doctor/json-pglite-degraded.json
+++ b/test/fixtures/goldens/doctor/json-pglite-degraded.json
@@ -752,7 +752,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
diff --git a/test/fixtures/goldens/doctor/json-pglite-fresh.json b/test/fixtures/goldens/doctor/json-pglite-fresh.json
index ca9f480348..f639979350 100644
--- a/test/fixtures/goldens/doctor/json-pglite-fresh.json
+++ b/test/fixtures/goldens/doctor/json-pglite-fresh.json
@@ -755,7 +755,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},
diff --git a/test/fixtures/goldens/doctor/json-postgres-fresh.json b/test/fixtures/goldens/doctor/json-postgres-fresh.json
index f2354ac43d..0e23c487b3 100644
--- a/test/fixtures/goldens/doctor/json-postgres-fresh.json
+++ b/test/fixtures/goldens/doctor/json-postgres-fresh.json
@@ -753,7 +753,9 @@
},
"covering_index": null,
"deleted_page_chunks": {
+ "held": 0,
"past_window": 0,
+ "purgeable": 0,
"window_hours": 72,
"within_window": 0
},