Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Bring back a page when Git restores its file, and keep committed files during purge
garrytan0a90c5fde843
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 38731b8879..acd2fe657e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,18 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**A file Git puts back after sync deleted its page brings the page back, and the retention purge never deletes a committed file.** On a managed brain, a commit that removed a Markdown file made sync soft-delete its page. When the same file came back (a `git revert`, switching to a branch and back, a collaborator's delete-then-restore, a file missing from disk for one working-tree sync), sync settled the path as "deleted, file unchanged" and left the page out of get, search and recall while `gbrain sync status` said synced. `gbrain pages purge-deleted --yes` then deleted the restored file from the checkout. Sync now tells a tombstone it made from a missing file apart from a person's delete and imports the file over it, so the same page comes back; a brain an older version left in that state heals on its next sync. The purge leaves a file committed at the source's last synced commit or at HEAD alone, unless a person deleted the page and the file stayed only because another page shared it. A page purge also stops leaving timeline text in the write journal.++### Itemized changes++- `src/core/persistence/sync-pass.ts`, `sync-classify.ts`: the level read marks a tombstone a managed sync deletion made (its open `sync-delete-tombstone` record or uncompacted `managed_sync_delete` receipt); once its file is back with clean committed bytes it classifies `import` (the import revives the same page row), and a `file_drift` hold an older version raised over it re-runs. A person's tombstone keeps `deleted_same` / `deleted_edited`.+- `src/core/persistence/sync-pass-run.ts`: the selection fingerprint carries `sync_deleted_restore`, so the first sync after the upgrade reads the whole source once and brings back pages left deleted beside their file.+- `src/core/persistence/page-prepare.ts`, `sync-markers.ts`: a retention purge of a tombstone whose file is committed at the source's `last_commit` or HEAD removes no file: a sync-made tombstone defers (`revision_conflict`) to the sync that restores it; any other is purged database-only (`write_through.skipped: committed_file`), except a person's delete that kept the file only because another live page shared it (`write_through.skipped: shared_origin`), which still removes the file once the tombstone owns it alone. An unreadable checkout counts as committed; a checkout with no commit yet does not. Repo-less sources are unchanged.+- `src/core/persistence/page-purge.ts`: a page purge redacts the `entry` and `bullet` text of the page's `add_timeline_entry` receipts, compacted ones included.+- Tests: `test/sync-restored-file-purge.test.ts` (PGLite and Postgres), `test/sync-classify-6712.test.ts`, `test/purge-eval-fixes.test.ts`.+## [0.60.625.0] - 2026-11-09**`gbrain upgrade` no longer consults the ClawHub registry; a large Google address book, and a large source's removal, can now finish.** Advisory: an unofficial package on the ClawHub registry shares the name gbrain. `gbrain upgrade`, and autopilot's unattended self-upgrade, treated any install it could not otherwise classify (for example a release binary run under its published file name, `gbrain-linux-x64`) as a ClawHub install whenever the `clawhub` CLI was on the host, and ran `clawhub update gbrain`, which installs whatever the registry resolves for that name from any publisher. Upgrade now never runs it: a compiled binary is recognised whatever its file name, and an install it cannot classify prints the manual steps. When Google's People sync token expires, the contacts re-list now checkpoints after every page and resumes there, so a large address book no longer restarts from page one in every sync job and Calendar and Gmail sync again. A managed `gbrain sources remove` or `purge` of a large source now deletes its pages in bounded batches under the 5-second statement bound instead of one cascading statement that could never finish, and a rerun resumes. The secret scanner also catches keys right after terminal color codes, `\xNN` and `\UXXXXXXXX` escapes and quoted-printable `=3D`, and redacts a bearer token containing an escaped slash in full.diff --git a/docs/architecture/key-files/canonical-reconciliation-continued.md b/docs/architecture/key-files/canonical-reconciliation-continued.mdindex 04af8a7a73..bda6b7f80a 100644--- a/docs/architecture/key-files/canonical-reconciliation-continued.md+++ b/docs/architecture/key-files/canonical-reconciliation-continued.md@@ -6,3 +6,4 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/persistence/rows-behind.ts` — #8071: `scanRowsBehind(engine, sourceId, { deadlineAt })` finds live Markdown pages of a managed source whose row holds an older version of its file than `sources.last_commit`, with no hold. It reads the checkpoint tree from Git (`readHeadTree(root, commit)`), compares each page in id batches of 256 with the committed blob (`compareInlineExportFile`, or `compareSegmentedExportBlob` for a page stored in segments) and confirms a difference with `prepareRepairPublication`; direction comes from the page's sync import record (`behind` when its revision is the recorded one, `unproven` without a record, `database_moved` skipped). Paths sync would not import (`loadSyncEligibility` from `sync-discovery.ts`) are skipped `not_eligible`. A blob over `SYNC_READ_BOUND` is never read and reported with `file_too_large`. Blobs are read and compared in chunks of at most `ROWS_BEHIND_BATCH_BYTES` (64 MiB) and dropped after the comparison, and the deadline is checked before each tree read and each chunk, so one batch of large files neither holds every blob in memory nor runs past the budget. Owner host only; a repo-less brain is `not_applicable`.- `src/commands/doctor/checks/rows-behind.ts` — doctor `sync_rows_behind_checkpoint` (BRAIN, after `sync_overwritten_db_edits`): per source counts and first slugs from `scanRowsBehind`; `warn` on `behind` rows with the read-only `gbrain repair rows-behind --source <id> --json` preview as the fix; `unknown` (info) when the `GBRAIN_DOCTOR_ROWS_BEHIND_TIMEOUT_MS` budget (default 10 s) stopped a scan or a checkout is unreadable; not applicable (ok) off the owner host and on repo-less brains.- `src/core/repair/rows-behind.ts` — `gbrain repair rows-behind --source <id>`: explicit-only, preview-bound (`--apply --expect <hash>`, `--include-ambiguous` adds rows with no import record). `reimport` submits the committed bytes through `submitManagedFileRepair`; `hold:<code>` writes the hold sync would have written (`writeGitHold`). Refused off the owner host (`not_owner_host`). Never runs on upgrade. Tests: `test/rows-behind-checkpoint-8071.test.ts` (PGLite and Postgres via `test/e2e/rows-behind-checkpoint-8071-postgres.test.ts`).+- GST-2201, a file that comes back after sync deleted its page: `readShadowLevel` (`sync-pass.ts`) reads whether a tombstone is sync-made (`syncDeleteOrigin`, `sync-delete-tombstones.ts`) into `page.syncDeleted`, and `sync-classify.ts` classifies such a path `import` ahead of `deleted_same` / `deleted_edited` (uncommitted bytes take `dirty`; an old hold on it re-runs). The selection fingerprint carries `sync_deleted_restore`, so the first pass after the upgrade reads the whole level once. `originCommitted` (`sync-markers.ts`) says whether a page's file is in the tree at the source's `last_commit` or at HEAD (false off a managed checkout or for a company-brain source; an unreadable checkout counts as committed). A retention purge (`page-prepare.ts`) of a tombstone whose file is committed defers a sync-made tombstone (`revision_conflict`) and purges any other database-only (`databaseOnlyReason: committed_file`), so the file stays. Tests: `test/sync-restored-file-purge.test.ts` (PGLite and Postgres), `test/sync-classify-6712.test.ts`.diff --git a/docs/guides/sync-unblock-runbook.md b/docs/guides/sync-unblock-runbook.mdindex fcd69b24a5..8234631d65 100644--- a/docs/guides/sync-unblock-runbook.md+++ b/docs/guides/sync-unblock-runbook.md@@ -429,8 +429,8 @@ it, and a `--full` continues until every path it covers is settled.`adopt` (the import record is updated to the file without a page write),`kept` (the commit no longer has a file that is still on disk, or the pageis under a declared `storage.db_only` dir, so its page stays), `untracked` (a file on disk the commit does not have, with no- page), `gone` (no file and no live page), `deleted_same` and `legacy_same` (a deleted page whose file is- unchanged), and `db_ahead` (the page changed in the database and the file+ page), `gone` (no file and no live page), `deleted_same` and `legacy_same` (a page a person deleted whose+ file is unchanged; a page sync deleted because its file was gone is an `import` once the file is back), and `db_ahead` (the page changed in the database and the filedid not; the sub-labels `pending`, `db_only` and `unproven` areinformational, and `gbrain repair file-drift --prefer database` writes oneback to its file if the owner wants it).diff --git a/plugin-variants/gbrain-coding/skills/data-loss-gate/SKILL.md b/plugin-variants/gbrain-coding/skills/data-loss-gate/SKILL.mdindex 249ca4d905..6697effde6 100644--- a/plugin-variants/gbrain-coding/skills/data-loss-gate/SKILL.md+++ b/plugin-variants/gbrain-coding/skills/data-loss-gate/SKILL.md@@ -91,7 +91,9 @@ Brain / database level (gbrain-specific):from every source of the brain. This is the point of no return for thesoft-delete safety net. Show the user `gbrain pages purge-deleted --dry-run --json`first; the command itself asks (exit 3 without a terminal) and runs with- `--yes` only after the user agrees.+ `--yes` only after the user agrees. It keeps a file committed at the source's+ last synced commit or at HEAD (the page waits or goes database-only), unless a+ person deleted the page and the file stayed only because another page shared it.- **Source removal** — `gbrain sources remove <id>` deletes the source ANDevery page in it. The `--confirm-destructive` flag does not substitute forthe card. A managed removal deletes pages in committed batches: one thatdiff --git a/plugin/skills/data-loss-gate/SKILL.md b/plugin/skills/data-loss-gate/SKILL.mdindex 249ca4d905..6697effde6 100644--- a/plugin/skills/data-loss-gate/SKILL.md+++ b/plugin/skills/data-loss-gate/SKILL.md@@ -91,7 +91,9 @@ Brain / database level (gbrain-specific):from every source of the brain. This is the point of no return for thesoft-delete safety net. Show the user `gbrain pages purge-deleted --dry-run --json`first; the command itself asks (exit 3 without a terminal) and runs with- `--yes` only after the user agrees.+ `--yes` only after the user agrees. It keeps a file committed at the source's+ last synced commit or at HEAD (the page waits or goes database-only), unless a+ person deleted the page and the file stayed only because another page shared it.- **Source removal** — `gbrain sources remove <id>` deletes the source ANDevery page in it. The `--confirm-destructive` flag does not substitute forthe card. A managed removal deletes pages in committed batches: one thatdiff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex c3c8060bb4..813cbffe2f 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: linksrc/core/search/evidence-delivery.ts 1759 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned); +4 GST-687: page evidence and the extractive fallback quote below-owner-tier text for untrusted readers (untrusted flag; logic in search/transcript-quote.ts)src/commands/config.ts 1526 ratchet +1 GBRA-216: recall.* config set value refusal (facts/recall-config.ts); +1 GBRA-216: config set refuses a dream.claim_conflicts.mode other than propose|off (refusal in facts/claim-conflicts.ts); new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)-src/core/behavior-change-notice.ts 1691 ratchet +4 train R: #7491/#7492 keyword_identifier_probe and keyword_slug_handles rows and their comment, #7175 keyword phrase row; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +1 GBRA-156 #7593: private_pages_granted_sources row (re-summed at the 0.60.527.0 merge: 1574 + 1); +3 GBRA-200 comment, write-grant refusal and degraded-stamp rows (re-summed at the 0.60.562.0 merge: 1615 + 3); +2 GBRA-280 get_page selector-precedence row and comment (re-summed at the 0.60.569.0 merge: 1619 + 2); the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move); +1 GST-1209: segmented_tiling_gst1209 row (rows are data and cannot move); +1 GST-1251: skillopt_loop_deprecated_gst1251 row (rows are data and cannot move); +1 GST-1209 follow-up: untiled_hold_repair_gst1209 row (rows are data and cannot move); +1 GBRA-283 B3a: embedded_scope_required_gbra283 row (rows are data and cannot move); re-summed at the train R merge with v0.60.574.0: 1623 + 4; +2 GST-1387: refresh_fence_refusal_gst1387 row and its comment (rows are data and cannot move); re-summed at the memory train #8481 merge with v0.60.578.0 (1643 measured); +3 GBRA-253 #8109 rows (frontmatter_drift_write_8109, contacts_activity_advance_8109, contact_rows_dedupe_8109; rows are data and cannot move), re-summed at the master v0.60.581.0 merge: 1643 + 3 = 1646 measured; +3 #8123 part C rows: sync_write_through_off_writeback_7084, managed_sync_dry_run_counts_7087, frontmatter_missing_source_dir_7889 (since TBD until the merge slot; re-summed at the 0.60.583.0 merge: 1646 + 3); +4 PR B commit_ledger_prb, doctor_foreign_writers_prb and sources_guard_prb rows and their comment (rows are data and cannot move; since TBD until the merge slot); +1 PR B #8615: file_drift_held_needs_prefer_8615 row; +1 PR B #8617: refresh_source_changed_retry_8617 row; +1 PR B #8662: commit_push_checkout_not_a_source_8662 row (since TBD until the merge slot; rows are data and cannot move); +2 PR B refresh_barrier_prb and commit_push_base_prb rows (re-summed: 1658 measured); +3 convergence PR A: the wave comment and two rows (conversion_content_evidence_pr_a, git_gap_live_pages_pr_a; since TBD until the merge slot; rows are data and cannot move); re-summed at the gate-ahead on master v0.60.592.0 plus #8711: 1658 + 3 = 1661 measured; +4 GST-1094/GST-900 rows (memory_recall scope-read, member-bound acting_user, recall max_chars, unknown_param _meta-only) and +1 qm_scope_writes_gst1113 and +1 recall_question_terms_gst1304 rows (rows are data and cannot move), re-summed at the 0.60.595.0 slot merge: 1661 + 6; +1 GST-1582 refresh_barrier_import_gst1582 row (since TBD until the merge slot; rows are data and cannot move); +2 GST-1601 source_changed_origin_gst1601 and adopt_local_edits_gst1601 rows (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.596.0 merge: 1668 + 2; +1 GST-1623 refresh_barrier_capacity_gst1623 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.599.0 merge: 1670 + 1; +2 GBRA-224 W1 release_channel_stable_default row and its comment; +1 GST-1623 refresh_barrier_later_write_gst1623 row (#8765; rows are data and cannot move); +5 #8766 GST-1623 items 4/6-9 and #8755: the wave comment and four rows (doctor_foreign_writers_partial_gst1623, commit_push_enclosing_repo_gst1623, owner_guard_fail_closed_gst1623, adopt_local_edits_hardening_8755) and +1 managed_sync_kind_reserved_8765 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on master v0.60.608.0 plus #8798: 1674 + 6; +1 GST-1853 refresh_barrier_aborted_refresh_sync_gst1853 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.613.0 merge: 1680 + 1; +1 GST-1674 refresh_refused_merge_recovery_gst1674 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.614.0 merge: 1681 + 1; +1 #8824/#8826/#8767: adopt_local_edits_replay_markers_8824 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8837: 1682 + 1; +1 url_source_origin row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8834: 1683 + 1; +1 #8828: the refresh_replan_own_commits_8828 row (1684 + 1 at the 0.60.618.0 slot); +1 the managed commit-push unpublished-count row; +1 the refresh publish-diverged default row; +1 GST-2051 ungranted_member_writes_gst2051 row (since TBD until the merge slot; rows are data and cannot move); +1 GST-731 think_window_trajectory_gst731 row and +1 GST-1602 purge_cancels_deferred_write_gst1602 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8915 v0.60.621.0: 1688 + 2; +1 GST-2059: owner_reads_private row+src/core/behavior-change-notice.ts 1693 ratchet +4 train R: #7491/#7492 keyword_identifier_probe and keyword_slug_handles rows and their comment, #7175 keyword phrase row; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +1 GBRA-156 #7593: private_pages_granted_sources row (re-summed at the 0.60.527.0 merge: 1574 + 1); +3 GBRA-200 comment, write-grant refusal and degraded-stamp rows (re-summed at the 0.60.562.0 merge: 1615 + 3); +2 GBRA-280 get_page selector-precedence row and comment (re-summed at the 0.60.569.0 merge: 1619 + 2); the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move); +1 GST-1209: segmented_tiling_gst1209 row (rows are data and cannot move); +1 GST-1251: skillopt_loop_deprecated_gst1251 row (rows are data and cannot move); +1 GST-1209 follow-up: untiled_hold_repair_gst1209 row (rows are data and cannot move); +1 GBRA-283 B3a: embedded_scope_required_gbra283 row (rows are data and cannot move); re-summed at the train R merge with v0.60.574.0: 1623 + 4; +2 GST-1387: refresh_fence_refusal_gst1387 row and its comment (rows are data and cannot move); re-summed at the memory train #8481 merge with v0.60.578.0 (1643 measured); +3 GBRA-253 #8109 rows (frontmatter_drift_write_8109, contacts_activity_advance_8109, contact_rows_dedupe_8109; rows are data and cannot move), re-summed at the master v0.60.581.0 merge: 1643 + 3 = 1646 measured; +3 #8123 part C rows: sync_write_through_off_writeback_7084, managed_sync_dry_run_counts_7087, frontmatter_missing_source_dir_7889 (since TBD until the merge slot; re-summed at the 0.60.583.0 merge: 1646 + 3); +4 PR B commit_ledger_prb, doctor_foreign_writers_prb and sources_guard_prb rows and their comment (rows are data and cannot move; since TBD until the merge slot); +1 PR B #8615: file_drift_held_needs_prefer_8615 row; +1 PR B #8617: refresh_source_changed_retry_8617 row; +1 PR B #8662: commit_push_checkout_not_a_source_8662 row (since TBD until the merge slot; rows are data and cannot move); +2 PR B refresh_barrier_prb and commit_push_base_prb rows (re-summed: 1658 measured); +3 convergence PR A: the wave comment and two rows (conversion_content_evidence_pr_a, git_gap_live_pages_pr_a; since TBD until the merge slot; rows are data and cannot move); re-summed at the gate-ahead on master v0.60.592.0 plus #8711: 1658 + 3 = 1661 measured; +4 GST-1094/GST-900 rows (memory_recall scope-read, member-bound acting_user, recall max_chars, unknown_param _meta-only) and +1 qm_scope_writes_gst1113 and +1 recall_question_terms_gst1304 rows (rows are data and cannot move), re-summed at the 0.60.595.0 slot merge: 1661 + 6; +1 GST-1582 refresh_barrier_import_gst1582 row (since TBD until the merge slot; rows are data and cannot move); +2 GST-1601 source_changed_origin_gst1601 and adopt_local_edits_gst1601 rows (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.596.0 merge: 1668 + 2; +1 GST-1623 refresh_barrier_capacity_gst1623 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.599.0 merge: 1670 + 1; +2 GBRA-224 W1 release_channel_stable_default row and its comment; +1 GST-1623 refresh_barrier_later_write_gst1623 row (#8765; rows are data and cannot move); +5 #8766 GST-1623 items 4/6-9 and #8755: the wave comment and four rows (doctor_foreign_writers_partial_gst1623, commit_push_enclosing_repo_gst1623, owner_guard_fail_closed_gst1623, adopt_local_edits_hardening_8755) and +1 managed_sync_kind_reserved_8765 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on master v0.60.608.0 plus #8798: 1674 + 6; +1 GST-1853 refresh_barrier_aborted_refresh_sync_gst1853 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.613.0 merge: 1680 + 1; +1 GST-1674 refresh_refused_merge_recovery_gst1674 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the master v0.60.614.0 merge: 1681 + 1; +1 #8824/#8826/#8767: adopt_local_edits_replay_markers_8824 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8837: 1682 + 1; +1 url_source_origin row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8834: 1683 + 1; +1 #8828: the refresh_replan_own_commits_8828 row (1684 + 1 at the 0.60.618.0 slot); +1 the managed commit-push unpublished-count row; +1 the refresh publish-diverged default row; +1 GST-2051 ungranted_member_writes_gst2051 row (since TBD until the merge slot; rows are data and cannot move); +1 GST-731 think_window_trajectory_gst731 row and +1 GST-1602 purge_cancels_deferred_write_gst1602 row (since TBD until the merge slot; rows are data and cannot move), re-summed at the gate-ahead on #8915 v0.60.621.0: 1688 + 2; +1 GST-2059: owner_reads_private row; +2 GST-2201 sync_restored_file_gst2201 and purge_keeps_committed_file_gst2201 rows (since is the current VERSION until the merge slot; rows are data and cannot move)diff --git a/skills/data-loss-gate/SKILL.md b/skills/data-loss-gate/SKILL.mdindex 148b9e4fdb..f333d1e2b0 100644--- a/skills/data-loss-gate/SKILL.md+++ b/skills/data-loss-gate/SKILL.md@@ -90,7 +90,9 @@ Brain / database level (gbrain-specific):from every source of the brain. This is the point of no return for thesoft-delete safety net. Show the user `gbrain pages purge-deleted --dry-run --json`first; the command itself asks (exit 3 without a terminal) and runs with- `--yes` only after the user agrees.+ `--yes` only after the user agrees. It keeps a file committed at the source's+ last synced commit or at HEAD (the page waits or goes database-only), unless a+ person deleted the page and the file stayed only because another page shared it.- **Source removal** — `gbrain sources remove <id>` deletes the source ANDevery page in it. The `--confirm-destructive` flag does not substitute forthe card. A managed removal deletes pages in committed batches: one thatdiff --git a/skills/skills.lock.json b/skills/skills.lock.jsonindex f72f8f0ad1..2f5e7139dc 100644--- a/skills/skills.lock.json+++ b/skills/skills.lock.json@@ -72,7 +72,7 @@"daily-task-manager/SKILL.md": "e93264a83b5fd0a19b2f06a322b3a4ad92d418f6a968c1cd51f4bea79cbca4d9","daily-task-manager/routing-eval.jsonl": "e98b2592e65c26952242436a960d8b2061e375b269004a18c230cbcf1d692ff9","daily-task-prep/SKILL.md": "5ea839012b48761ed8d7a9bab056a2566587d6f7fe044e3907d77b20f8eec5ac",- "data-loss-gate/SKILL.md": "656d7e108a88fa3ac7d7d41dfb79f02b7dc592d286a7cd6a7fa63503550e49ba",+ "data-loss-gate/SKILL.md": "99ab81bd998f5e5f7cb203d79a9ba16d852c3764a3a9ebb6e8296692284c2fd7","data-loss-gate/routing-eval.jsonl": "825358a5600641061a02b25097cd84c1c28bb0359a7608a10b9f25f661bd88f9","data-research/SKILL.md": "39f9648a21a7074d6ad7db789e93e6d7085449a6cdaffacf8b1660eb68636b9d","db-repair/SKILL.md": "87a34fe18812f3b6ec37381ae69c205e30747300098ebe3fe41f8460d0ba8346",diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex bc7a3f675b..f914d9f366 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1493,6 +1493,8 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.622.0', id: 'think_window_trajectory_gst731', text: '`think` on a question that names one month or day (or given `since`/`until`) still gets the entity\'s trajectory: each metric\'s latest reading before the window now joins the readings inside it, as the value in force when the window opens (unless its `valid_until` ended by then), and an entity page dated outside the window still names whose trajectory to read (it stays out of the evidence). Before, "MRR in April" with readings in March and June, or a company page last edited outside the month, injected no trajectory. `WINDOW_EXCLUDED_<n>_TRAJECTORY_POINTS` no longer counts the carried reading.' },{ since: '0.60.622.0', id: 'purge_cancels_deferred_write_gst1602', text: 'With `persistence.refresh_barrier` on, `gbrain forget <id> --purge` no longer refuses `purge_blocked_pending_recovery` while a write deferred behind a refresh holds the claim in its staged body: the deferred write was never claimed, so the purge cancels it with `purged_content`, as it does a queued one, and removes the content at once instead of up to 15 minutes later.' },{ since: '0.60.623.0', id: 'owner_reads_private_gst2059', text: 'A brain can now name its owner: `gbrain access owner set <member>` (config `access.owner_member`, an enabled member). Every remote credential or stdio server bound to that member reads the whole brain, `visibility: private` pages, privacy-alias pages (`access: private`, ...), derived and pinned-question pages included, through `get_page`, `search`, `query`, `recall` and the other page reads in the sources its grant reads, and `recall` returns its private facts. `readable_by` and folder rules still bind it, and it is not relaxed when it speaks for a shared-room audience. A page the owner\'s credential creates in `default` through `put_page` is not kept to the owner (no `readable_by: [member:<id>]`). Other members, unbound and public callers are unchanged. `_meta.access.relaxation` reads `owner` for such a call, and `gbrain access check --token <name>` reports it. `gbrain repair private-labels` (named runs only) previews the `access: private` labels stamped on ordinary pages with counts per category, keeps real restrictions, and clears the rest in reviewed batches (`--apply --expect <hash>`), recording each change so `--class restore` can put it back. The bulk-ingestion page template no longer writes an `access:` line; what each designation means is in docs/guides/access-designations.md.' },+ { since: '0.60.625.0', id: 'sync_restored_file_gst2201', text: 'On a managed source, a page sync deleted because its file was gone comes back on the next sync once the file returns (a `git revert`, a branch or detached checkout and back, a later commit that re-adds it with the same or edited bytes, a file missing from disk for one working-tree sync). It used to stay deleted (`deleted_same`, or a `file_drift` hold whose suggested fix removed the file) while `gbrain sync status` said synced. The first sync after the upgrade reads the whole source once, so pages an older version left deleted beside their file come back and such holds clear without a manual step. A page a person deleted keeps its deletion.' },+ { since: '0.60.625.0', id: 'purge_keeps_committed_file_gst2201', text: 'The retention purge (the autopilot purge phase, the `purge` job and `gbrain pages purge-deleted`) never removes a soft-deleted page\'s file that is committed at its source\'s last synced commit or at HEAD. A page sync deleted whose file is back is left for the sync that restores it (counted as deferred); any other such page is removed database-only (receipt `write_through.skipped: committed_file`) and its file stays. A page whose file is gone, or a person\'s delete that kept the file only because another page shared it, purges as before. A page purge (`gbrain delete <slug> --purge` and the retention purge) now also redacts the timeline text the page\'s `add_timeline_entry` receipts kept (`entry`, `bullet`), compacted ones included.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/persistence/coordinator.ts b/src/core/persistence/coordinator.tsindex cef9d0a01c..ba2272a257 100644--- a/src/core/persistence/coordinator.ts+++ b/src/core/persistence/coordinator.ts@@ -73,7 +73,7 @@ interface PreparedMutationBase {/** #7866: a content no-op whose chunks all hold a vector for their text: publication queues no embedding effect (`embedding_state: current`). */embeddingCurrent?: boolean;/** Why a page write bound to a worktree publishes no file (receipt `write_through.skipped`). */- databaseOnlyReason?: 'db_only' | 'unbound_source' | 'mirror_read_only' | 'sync_excluded' | 'shared_origin' | 'origin_twin' | 'detached';+ databaseOnlyReason?: 'db_only' | 'unbound_source' | 'mirror_read_only' | 'sync_excluded' | 'shared_origin' | 'origin_twin' | 'detached' | 'committed_file';/** #6517: the receipt's `write_through.warning` for a database-only publication, set by `validate` under the publication guards. */databaseOnlyWarning?: string;/**diff --git a/src/core/persistence/page-prepare.ts b/src/core/persistence/page-prepare.tsindex fed0044699..11017c2672 100644--- a/src/core/persistence/page-prepare.ts+++ b/src/core/persistence/page-prepare.ts@@ -94,6 +94,8 @@ import { POLICY_KEYS } from './reconcile-additive.ts';import { protectedReconcileKey } from './reconcile-merge.ts';import { PROTECTED_FRONTMATTER_KEYS } from '../frontmatter-recovery.ts';import { preparePagePurge, purgePageInTransaction, type PreparedPagePurge } from './page-purge.ts';+import { originCommitted } from './sync-markers.ts';+import { syncDeleteOrigin } from './sync-delete-tombstones.ts';import { withTrustKeep } from './context.ts';import { isFenceEditWrite, pageGateTrust, pageWriteTrust, recordAgentPageLowering, stampPageTrustMarker, storedPageTier } from '../trust/page-write.ts';import { compareTrust, type WriteTrust } from '../trust/tier.ts';@@ -657,6 +659,19 @@ async function prepareDeletePageMutation(engine: BrainEngine, row: WriteRequest,error.detail = 'shared_origin';throw error;}+ // GST-2201: a retention purge never removes a file committed at the source's last commit or at HEAD. A tombstone sync made+ // from the file's absence waits for the sync that restores it; a person's tombstone is removed database-only and the file stays.+ // A person's delete that kept the file only because another live page shared it (`write_through.skipped: shared_origin`)+ // still removes the file once the tombstone owns it alone.+ const committedFile = purge && p.retention === true && snapshot.page.deleted_at != null && !!file && !shared && excludedPath === null+ && await originCommitted(engine, row.source_id, snapshot.page.source_path)+ && (await engine.executeRaw(`SELECT 1 FROM persistence_requests r WHERE r.source_id=$1 AND r.slug=$2 AND (r.page_id IS NULL OR r.page_id=$3) AND r.operation='delete_page'+ AND r.state='committed' AND r.outcome->'write_through'->>'skipped'='shared_origin' LIMIT 1`, [row.source_id, row.slug, snapshot.page.id])).length === 0;+ if (committedFile) {+ const [made] = await engine.executeRaw<{ sync: boolean }>(`SELECT ${syncDeleteOrigin('p')} AS sync FROM pages p WHERE p.id=$1`, [snapshot.page.id]);+ if (made?.sync) throw pageRefusal('revision_conflict', 'The file of this sync-deleted page is committed again, so the retention purge left the page and the file alone; the next sync restores the page.', row,+ `Page ${row.slug}'s file is committed in source ${row.source_id}'s checkout.`, pageFix(row.source_id, row.slug));+ }// #6511: the purge's journal sweep runs here, under the preparation budget, never inside the 5 s publication transaction.const preparedPurge: PreparedPagePurge | null = purge ? await preparePagePurge(engine, row, snapshot, clock) : null;// #6948: a page merged into its duplicate holds that page's body, so an owner purge would tombstone the survivor's content.@@ -667,7 +682,7 @@ async function prepareDeletePageMutation(engine: BrainEngine, row: WriteRequest,error.detail = 'origin_twin';throw error;}- const prepared: PreparedMutation = { observedRevision, noop, file, ...await pageDatabaseOnlyPublication(engine, row, file),+ const prepared: PreparedMutation = { observedRevision, noop, file: committedFile ? undefined : file, ...await pageDatabaseOnlyPublication(engine, row, committedFile ? undefined : file),// W1: a purge bumps sources.purge_epoch, so it holds the source row exclusively from the start (retention bumps nothing, #26);// the derived rows' and cascaded links' streams are locked with the page....(purge && !preparedPurge?.retention ? { sourceExclusive: true } : {}),@@ -675,6 +690,7 @@ async function prepareDeletePageMutation(engine: BrainEngine, row: WriteRequest,...(preparedPurge && !preparedPurge.retention ? { afterCommit: async () => { appendPurgeCommitted(row.id); } } : {}),...(preparedPurge?.derivedSlugs.length ? { additionalPageKeys: preparedPurge.derivedSlugs.map(slug => ({ sourceId: row.source_id, slug })) } : {}),...(excludedPath !== null ? { databaseOnlyReason: 'sync_excluded' as const } : {}), ...(shared ? { databaseOnlyReason: 'shared_origin' as const } : {}),+ ...(committedFile ? { databaseOnlyReason: 'committed_file' as const } : {}),...(shared ? { validate: async (tx: BrainEngine) => { prepared.databaseOnlyWarning = await validateSharedOrigin(tx, row, shared, sharedRetention); } } : {}),apply: async tx => {if (purge && excludedPath !== null) await assertStillSyncExcluded(tx, row, excludedPath);diff --git a/src/core/persistence/page-purge.ts b/src/core/persistence/page-purge.tsindex 876c35f763..7539c520ec 100644--- a/src/core/persistence/page-purge.ts+++ b/src/core/persistence/page-purge.ts@@ -46,7 +46,7 @@ import { faultPoint } from './fault-points.ts';import { writeOriginDeletedMarker } from './sync-markers.ts';import { clearRebuildableCache } from './rebuildable-cache.ts';import { carryingNeedles, findRequestsForSlugs, JOURNAL_SWEEP_MATCH_CAP, recheckRequestsCarrying, redactRequestIntents, sweepRequestsCarrying, sweepRequestsForGroups,- type JournalSweep } from './intent-redaction.ts';+ type IntentRedactionTarget, type JournalSweep } from './intent-redaction.ts';import { dropPurgedFenceRows } from '../facts/purge-overlay.ts';import { isOutcomeMarker, outcomeMarkerSql } from '../facts/audit-sources.ts';import { derivedRowsFrom, hideDerivedRows } from '../facts/derivation-inputs.ts';@@ -327,12 +327,18 @@ export async function purgePageInTransaction(tx: BrainEngine, row: WriteRequest,// #6511 [R2]: the preparation sweep's targets, plus the tail past its ceiling and the rows pending at scan time, re-tested here.const recheck = await lookup('tail_recheck', () => recheckRequestsCarrying(tx, sourceId, needles, prepared.sweep,{ exclude: row.id, beforePending: () => faultPoint('purge:pending_recheck', { requestId: row.request_id, sourceId, operation: row.operation }) }));- const forSlug = await lookup('slug_lookup', async () => [...await findRequestsForSlugs(tx, sourceId, [slug]), ...reasons.length ? await requestsRecordingReasons(tx, sourceId, slug) : []]);+ // The page's add_timeline_entry receipts keep the entry text in their outcome (`entry`, `bullet`), compacted ones included:+ // that text is a needle in the page's own requests, as its reasons are.+ const timeline = await lookup('slug_lookup', () => tx.executeRaw<IntentRedactionTarget & { summary: string | null; bullet: string | null }>(`SELECT id::text AS id, principal_kind, principal_id,+ outcome->'entry'->>'summary' AS summary, outcome->>'bullet' AS bullet FROM persistence_requests+ WHERE source_id=$1 AND slug=$2 AND operation='add_timeline_entry' AND outcome ? 'entry' ORDER BY sequence LIMIT 10000`, [sourceId, slug]));+ const timelineText = [...new Set(timeline.flatMap(t => [t.summary, t.bullet]).map(text => text?.trim() ?? '').filter(Boolean))];+ const forSlug = await lookup('slug_lookup', async () => [...await findRequestsForSlugs(tx, sourceId, [slug]), ...reasons.length ? await requestsRecordingReasons(tx, sourceId, slug) : [], ...timeline]);const own = new Set(forSlug.map(r => r.id));const requests = [...new Map([...forSlug, ...prepared.sweep.targets, ...recheck.targets].map(r => [r.id, r])).values()];// GBRA-217: the page's reasons are needles in its own requests only, so a short reason never redacts another page's receipt.removed.persistence_requests = await lookup('redaction_read', async () => await redactRequestIntents(tx, requests.filter(r => own.has(r.id)), row.id,- { skipPending: true, needles: [...needles, ...reasons] }) + await redactRequestIntents(tx, requests.filter(r => !own.has(r.id)), row.id, { skipPending: true, needles }));+ { skipPending: true, needles: [...needles, ...reasons, ...timelineText] }) + await redactRequestIntents(tx, requests.filter(r => !own.has(r.id)), row.id, { skipPending: true, needles }));removed.persistence_effects = await stripPageGitNotes(tx, sourceId, slug);const journalSweep = { scanned: prepared.sweep.scanned, ceiling: prepared.sweep.ceiling, watermark: prepared.sweep.watermark ?? null, batch: prepared.sweep.batch ?? 1,tail: recheck.tail, pending_rechecked: recheck.pendingRechecked, complete: prepared.sweep.complete && recheck.complete };diff --git a/src/core/persistence/sync-classify.ts b/src/core/persistence/sync-classify.tsindex a23502bb32..820026430a 100644--- a/src/core/persistence/sync-classify.ts+++ b/src/core/persistence/sync-classify.ts@@ -48,7 +48,9 @@ export interface SyncPathState {/** The blob id of the bytes on disk, when read. */diskOid: string | null;/** The page whose origin is `p`: live, a soft-deleted tombstone, or none. */- page: { live: boolean; revision: string; originMatches: boolean } | null;+ page: { live: boolean; revision: string; originMatches: boolean;+ /** A tombstone a managed sync deletion made from the file's absence (sync-delete-tombstones.ts), not a person's delete. */+ syncDeleted?: boolean } | null;/** The page's import record (none, or one written before PR 0 without a revision). */record: { blobOids: readonly string[]; revision: string | null; parserSame: boolean } | null;/** `att(R)`: the page's current revision is file-backed; `hash` is the attested bytes' sha256 when an effect recorded one; `arm1` a committed sync request wrote it. */@@ -157,7 +159,9 @@ type Matched = Omit<SyncClassified, 'inFlight' | 'classAction'>;* is at its level and not re-screen due, the pass re-runs it (`held`, `rescreenDue`).*/function firstMatch(s: SyncPathState, strict = false): Matched {- if (s.hold && s.hold.atLevel && !s.hold.rescreenDue && !s.hold.retried) return { cls: 'held', settled: true, action: 'none' };+ // GST-2201: a hold an older version raised over a sync-made tombstone whose file came back (tombstone_edited) is re-run.+ const syncRestored = s.page?.live === false && s.page.syncDeleted === true && s.level !== null;+ if (s.hold && s.hold.atLevel && !s.hold.rescreenDue && !s.hold.retried && !syncRestored) return { cls: 'held', settled: true, action: 'none' };const rest = strict ? gitWins(s) : afterHold(s);// The re-run is the import the hold refused, where an import is what the level would get anyway (`import`) or changes// nothing (`matched`). A level that needs another action (a drift side, a delete proof, a reparse) keeps that class, so@@ -190,10 +194,15 @@ function afterHold(s: SyncPathState): Matched {}if (!live && s.level === null) return { cls: 'gone', settled: true, action: 'none' };if (!live) {+ // GST-2201: sync deleted this page because its file was absent; the file is back (a revert, a branch switch, a restore),+ // so Git is the parent and the file imports over the tombstone, once its committed bytes are clean (uncommitted bytes+ // take the dirty row). A person's delete keeps the deleted_same and deleted_edited rows.+ const syncRestored = tombstone && s.page!.syncDeleted === true && s.level !== null;+ if (syncRestored && (s.mode !== 'committed' || CLEAN.has(s.dirty) || s.dirty === 'deleted')) return { cls: 'import', settled: false, action: 'import' };const deletedMarker = s.marker?.kind === 'deleted' ? s.marker : null;- if ((tombstone && fileSame) || (deletedMarker && s.level !== null && deletedMarker.blobOids.includes(s.level))) return { cls: 'deleted_same', settled: true, action: 'none' };- if ((tombstone || deletedMarker) && !s.published) return { cls: 'deleted_edited', settled: false, action: 'hold', side: 'deleted', state: tombstone ? 'tombstone_edited' : 'purged_differs' };- if (s.marker?.kind === 'legacy' && s.marker.deletion && !s.fullObligation && s.level !== null && s.marker.blobOids.includes(s.level)) return { cls: 'legacy_same', settled: true, action: 'none' };+ if (!syncRestored && ((tombstone && fileSame) || (deletedMarker && s.level !== null && deletedMarker.blobOids.includes(s.level)))) return { cls: 'deleted_same', settled: true, action: 'none' };+ if (!syncRestored && (tombstone || deletedMarker) && !s.published) return { cls: 'deleted_edited', settled: false, action: 'hold', side: 'deleted', state: tombstone ? 'tombstone_edited' : 'purged_differs' };+ if (!syncRestored && s.marker?.kind === 'legacy' && s.marker.deletion && !s.fullObligation && s.level !== null && s.marker.blobOids.includes(s.level)) return { cls: 'legacy_same', settled: true, action: 'none' };}if (s.mode === 'committed' && s.tree !== null && !CLEAN.has(s.dirty) && s.dirty !== 'deleted') {const absorbed = (s.tree !== null && !!s.record && s.record.blobOids.includes(s.tree)) || (s.diskOid !== null && !!s.record && s.record.blobOids.includes(s.diskOid));diff --git a/src/core/persistence/sync-markers.ts b/src/core/persistence/sync-markers.tsindex d71d191089..8ba419b2cd 100644--- a/src/core/persistence/sync-markers.ts+++ b/src/core/persistence/sync-markers.ts@@ -50,24 +50,57 @@ const TREE_READ_LIMITS = { timeoutMs: 10_000, maxBytes: 1024 * 1024 };* cannot answer counts as present (fail closed), so an unreadable checkout or tree writes the marker.*/function originStillBacked(root: string | null, lastCommit: string | null, slugMode: string | null, sourcePath: string): boolean {- if (!root) return true;+ const origin = originGitPaths(root, slugMode, sourcePath);+ if (!origin) return true;+ for (const gitPath of origin.candidates) {+ try { if (readSyncFile(root!, origin.scope ? gitPath.slice(origin.scope.length + 1) : gitPath) !== null) return true; } catch { return true; }+ }+ return originInTree(origin, lastCommit);+}++/** The Git paths `sourcePath` may have in `root`'s repository, or null when the checkout cannot be read. */+function originGitPaths(root: string | null, slugMode: string | null, sourcePath: string): { gitRoot: string; scope: string; candidates: Set<string> } | null {+ if (!root) return null;const toplevel = hardenedGitSync(root, ['rev-parse', '--show-toplevel'], { timeoutMs: 10_000, maxBytes: 4096 });- if (!toplevel.ok) return true;+ if (!toplevel.ok) return null;const gitRoot = toplevel.stdout.toString('utf8').trim();let scope: string;- try { scope = relative(realpathSync.native(gitRoot), realpathSync.native(root)).split(sep).join('/'); } catch { return true; }- if (scope.startsWith('..')) return true;+ try { scope = relative(realpathSync.native(gitRoot), realpathSync.native(root)).split(sep).join('/'); } catch { return null; }+ if (scope.startsWith('..')) return null;const candidates = new Set<string>();if (slugMode !== 'git-root') candidates.add(scope ? `${scope}/${sourcePath}` : sourcePath);if (slugMode !== 'source-root' && (!scope || sourcePath.startsWith(`${scope}/`))) candidates.add(sourcePath);- for (const gitPath of candidates) {- try { if (readSyncFile(root, scope ? gitPath.slice(scope.length + 1) : gitPath) !== null) return true; } catch { return true; }- }- if (!lastCommit || candidates.size === 0) return false;- const tree = hardenedGitSync(gitRoot, ['ls-tree', '-z', '--full-tree', lastCommit, '--', ...candidates], TREE_READ_LIMITS);+ return { gitRoot, scope, candidates };+}++/** Whether one of the origin's paths is in `lastCommit`'s tree; a tree that cannot be read counts as present. */+function originInTree(origin: { gitRoot: string; candidates: Set<string> }, lastCommit: string | null): boolean {+ if (!lastCommit || origin.candidates.size === 0) return false;+ const tree = hardenedGitSync(origin.gitRoot, ['ls-tree', '-z', '--full-tree', lastCommit, '--', ...origin.candidates], TREE_READ_LIMITS);return !tree.ok || tree.stdout.length > 0;}+/**+ * GST-2201: whether a page's file is committed at its source's `last_commit` or at the checkout's HEAD (a commit sync has not+ * read yet), so a retention purge must not remove it. False on an unmanaged brain, for a source with no worktree binding (a+ * repo-less brain), a company-brain source or a checkout with no commit yet; a checkout or tree that cannot be read counts as committed (fail closed: the+ * file stays).+ */+export async function originCommitted(engine: BrainEngine, sourceId: string, sourcePath: string | null | undefined): Promise<boolean> {+ if (!sourcePath || !await managedPersistenceEnabled(engine)) return false;+ const [source] = await engine.executeRaw<{ last_commit: string | null; config: Record<string, unknown> | null }>('SELECT last_commit, config FROM sources WHERE id=$1', [sourceId]);+ if (!source || companyBrainProfile(source.config) !== null) return false;+ const binding = await getWorktreeBinding(engine, sourceId);+ if (!binding) return false;+ const root = binding.local_path ? resolve(binding.local_path, binding.relative_path) : null;+ const origin = originGitPaths(root, typeof source.config?.slug_root_mode === 'string' ? source.config.slug_root_mode : null, sourcePath);+ if (!origin) return true;+ if (originInTree(origin, source.last_commit)) return true;+ const head = hardenedGitSync(origin.gitRoot, ['rev-parse', '--verify', '--quiet', 'HEAD^{commit}'], TREE_READ_LIMITS);+ if (!head.ok) return !(head.reason === 'exit' && head.status === 1);+ return originInTree(origin, head.stdout.toString('utf8').trim());+}+/*** Writes the deletion marker of `page`'s origin inside `tx`, the transaction that deletes the page row (call it before* the delete). Writes nothing on an unmanaged brain, for a page without an origin, while another live page records thediff --git a/src/core/persistence/sync-pass-run.ts b/src/core/persistence/sync-pass-run.tsindex 4f4ddbc3b3..592a8bdef1 100644--- a/src/core/persistence/sync-pass-run.ts+++ b/src/core/persistence/sync-pass-run.ts@@ -409,7 +409,10 @@ async function pinPass(engine: BrainEngine, input: SyncPassInput, lease: NonNull// GBRA-133: so does `sync.apply_held_deletions` when on (sync-held-deletions.ts): turning it on reads the level, and its held deletions, once.// `sync.level_reuse` rides the same read (sync-tree-reuse.ts); it changes how the level is read, not what it is.const { heldDeletions, levelReuse } = await readPassSwitches(engine, sourceId);- const selectionFp = digest({ ...selection, ...(metafilePages === 'off' ? {} : { metafile_pages: metafilePages }), ...(heldDeletions ? { apply_held_deletions: true } : {}), ...(strict ? { canonical: 'git' } : {}) }), parserFp = parserFingerprint();+ // GST-2201: `sync_deleted_restore` marks the classifier that imports a file restored over its sync-made tombstone, so the first+ // pass after the upgrade reads the whole level once and brings back pages an older pass left deleted beside their file.+ const selectionFp = digest({ ...selection, ...(metafilePages === 'off' ? {} : { metafile_pages: metafilePages }), ...(heldDeletions ? { apply_held_deletions: true } : {}), ...(strict ? { canonical: 'git' } : {}),+ sync_deleted_restore: 1 }), parserFp = parserFingerprint();const obligations: ManagedSyncPassValue['obligations'] = prior ? structuredClone(prior.value.obligations): { baseline_commit: context.source.last_commit, mismatch: {}, not_converged_streak: 0, faults: {} };obligations.mismatch ??= {}; obligations.faults ??= {}; obligations.not_converged_streak ??= 0;diff --git a/src/core/persistence/sync-pass.ts b/src/core/persistence/sync-pass.tsindex bf8d3a277c..4db8f5d50f 100644--- a/src/core/persistence/sync-pass.ts+++ b/src/core/persistence/sync-pass.ts@@ -48,6 +48,7 @@ import { MARKDOWN_CHUNKER_VERSION } from '../chunkers/recursive.ts';import { filterYielding, loopYield, progressYield, sortYielding, turnYield, whileActive, type LoopYield } from './sync-yield.ts';import { countsMatch, proveConfirm, readLevelMark, type ConfirmProof, type LevelMark } from './sync-pass-confirm.ts';import { domainCache, levelDomain, narrowedWalk, walkAttestations, type LevelDomainCache } from './sync-pass-narrow.ts';+import { syncDeleteOrigin } from './sync-delete-tombstones.ts';export { MANAGED_SYNC_PASS_OP, SYNC_ORIGIN_DELETED_OP };@@ -324,7 +325,7 @@ const recordOids = (record: PageRow['record'], level?: string | null): string[]const recordNames = (record: PageRow['record'], oid: string | null): boolean => !!record && typeof oid === 'string' && oid.length > 0&& (record.blob_oid === oid || record.disk_blob_oid === oid);-const PAGE_DELETED = 1, PAGE_RECORD = 2, PAGE_REVISION_NULL = 4, PAGE_PARSER_EXPECTED = 8, PAGE_WORKING = 16, PAGE_WORKING_TRUE = 32, PAGE_DERIVED_CURRENT = 64;+const PAGE_DELETED = 1, PAGE_RECORD = 2, PAGE_REVISION_NULL = 4, PAGE_PARSER_EXPECTED = 8, PAGE_WORKING = 16, PAGE_WORKING_TRUE = 32, PAGE_DERIVED_CURRENT = 64, PAGE_SYNC_DELETED = 128;/*** `I⋈P` as columns, one slot per page, instead of a `PageRow` and a record object per page (about 200 B a page at 600k). A@@ -342,6 +343,8 @@ export class PageTable {hasRecord(i: number): boolean { return (this.bits[i]! & PAGE_RECORD) !== 0; }/** `SyncPathState.derivedCurrent`: the page's chunks, text projection and contextual mode are current (`readPages`). */derivedCurrent(i: number): boolean { return (this.bits[i]! & PAGE_DERIVED_CURRENT) !== 0; }+ /** `SyncPathState.page.syncDeleted`: a tombstone a managed sync deletion made (sync-delete-tombstones.ts), still open. */+ syncDeleted(i: number): boolean { return (this.bits[i]! & PAGE_SYNC_DELETED) !== 0; }/** The record's fields `row(i).record` would carry (undefined without a record), read without building it: the classifier's per-path reads. */diskBlobOidOf(i: number): string | null | undefined { return !this.hasRecord(i) ? undefined : this.diskBlobOid.has(i) ? this.diskBlobOid.get(i)! : this.blobOid[i] ?? null; }recordRevisionOf(i: number): string | undefined { return !this.hasRecord(i) || this.bits[i]! & PAGE_REVISION_NULL ? undefined : this.recordRevision.get(i) ?? this.revision[i]!; }@@ -412,7 +415,8 @@ async function readPages(exec: Exec, kind: 'postgres' | 'pglite', sourceId: striCASE WHEN c.parser IS DISTINCT FROM $4::jsonb THEN c.parser::text END AS rec_parser, c.parser = $4::jsonb AS rec_parser_expected,c.full_episode AS rec_full_episode, c.working AS rec_working,(p.page_kind = 'markdown' AND p.chunker_version = ${MARKDOWN_CHUNKER_VERSION} AND p.body_storage IS DISTINCT FROM 'segmented'- AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current+ AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current,+ CASE WHEN p.deleted_at IS NOT NULL THEN ${syncDeleteOrigin('p')} ELSE false END AS sync_deletedFROM pages p LEFT JOIN (SELECT fingerprint, completed_keys->0->>'origin' AS origin, completed_keys->0->>'blob_oid' AS blob_oid,completed_keys->0->>'disk_blob_oid' AS disk_blob_oid, completed_keys->0->>'revision' AS revision,completed_keys->0->'parser' AS parser, completed_keys->0->>'full_episode' AS full_episode, (completed_keys->0->'working')::text AS working@@ -420,7 +424,7 @@ async function readPages(exec: Exec, kind: 'postgres' | 'pglite', sourceId: striWHERE p.source_id=$1 AND p.source_path IS NOT NULL`;const params = [sourceId, SYNC_IMPORT_PROVENANCE_OP, `${sourceId}:${incarnation}:`, JSON.stringify(expected)];type Raw = Omit<PageRow, 'record' | 'arm1' | 'attested_hash'> & { has_record: boolean; rec_origin_same: boolean; rec_origin: string | null; rec_blob_oid: string | null;- rec_disk_same: boolean; rec_disk_blob_oid: string | null; rec_revision_null: boolean; rec_revision: string | null; rec_parser: string | null; rec_parser_expected: boolean | null; rec_full_episode: string | null; rec_working: string | null; derived_current: boolean | null };+ rec_disk_same: boolean; rec_disk_blob_oid: string | null; rec_revision_null: boolean; rec_revision: string | null; rec_parser: string | null; rec_parser_expected: boolean | null; rec_full_episode: string | null; rec_working: string | null; derived_current: boolean | null; sync_deleted: boolean | null };const pages = only?.table ?? new PageTable(expected);const parsers = new Map<string, SyncImportProvenance['parser'] | undefined>(), episodes = new Map<string, string>();const parserOf = (text: string | null) => {@@ -434,7 +438,7 @@ async function readPages(exec: Exec, kind: 'postgres' | 'pglite', sourceId: stripages.id[i] = raw.id; pages.slug[i] = raw.slug; pages.sourcePath[i] = raw.source_path; pages.revision[i] = raw.revision; pages.requestId[i] = raw.request_id;pages.bits[i] = (raw.deleted ? PAGE_DELETED : 0) | (raw.has_record ? PAGE_RECORD : 0) | (raw.rec_revision_null ? PAGE_REVISION_NULL : 0)| (raw.rec_parser_expected ? PAGE_PARSER_EXPECTED : 0) | (working ? PAGE_WORKING : 0) | (raw.rec_working === 'true' ? PAGE_WORKING_TRUE : 0)- | (raw.derived_current ? PAGE_DERIVED_CURRENT : 0);+ | (raw.derived_current ? PAGE_DERIVED_CURRENT : 0) | (raw.deleted && raw.sync_deleted ? PAGE_SYNC_DELETED : 0);const episode = raw.has_record ? raw.rec_full_episode : null;if (typeof episode === 'string' && !episodes.has(episode)) episodes.set(episode, episode);pages.blobOid[i] = raw.has_record ? raw.rec_blob_oid ?? undefined : undefined; pages.fullEpisode[i] = typeof episode === 'string' ? episodes.get(episode)! : episode;@@ -812,7 +816,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: stconst needsBaseline = live && recordRevision == null && level !== null && !recordBlobOids.includes(level);const state: SyncPathState = {mode, tree: committed?.oid ?? null, level, present: presentNow, dirty, diskOid: probe?.rawOid ?? null,- page: row ? { live, revision: pages.revision[at!]!, originMatches: record && originPath(pages.origin.get(at!) ?? pages.sourcePath[at!]!) === pageOrigin(pages.sourcePath[at!]!, pages.slug[at!]!) } : null,+ page: row ? { live, revision: pages.revision[at!]!, originMatches: record && originPath(pages.origin.get(at!) ?? pages.sourcePath[at!]!) === pageOrigin(pages.sourcePath[at!]!, pages.slug[at!]!), ...(!live && pages.syncDeleted(at!) ? { syncDeleted: true } : {}) } : null,record: record ? { blobOids: recordBlobOids, revision: recordRevision != null ? String(recordRevision) : null, parserSame: sameParser(pages.parserOf(at!), fp(path)) } : null,attested, levelSha256: probe?.rawSha256 ?? null,published: !!attested?.hash && probe?.rawSha256 === attested.hash,diff --git a/test/e2e/managed-sync-shared-origin-6517.test.ts b/test/e2e/managed-sync-shared-origin-6517.test.tsindex 495d00ed68..debb53c6fb 100644--- a/test/e2e/managed-sync-shared-origin-6517.test.ts+++ b/test/e2e/managed-sync-shared-origin-6517.test.ts@@ -89,7 +89,7 @@ for (const backend of testBackends()) describe(`[${backend}] #6517 managed syncrmSync(home, { recursive: true, force: true });}, 60_000);- test('two live claimants hold their file (the rest syncs); after the loser is soft-deleted the file imports, edits and deletes through both guards; a single edited tombstone is held', async () => {+ test('two live claimants hold their file (the rest syncs); after the loser is soft-deleted the file imports, edits and deletes through both guards; a single sync-made tombstone whose file returns edited is restored', async () => {const f = await twins();writeFileSync(f.file, note('Alpha, version two.')); commit(f.root, 'version two');// #6948: the twin no longer refuses the source; only its file is held origin_conflict (the bodies differ: a person decides).@@ -131,21 +131,19 @@ for (const backend of testBackends()) describe(`[${backend}] #6517 managed syncexpect((await page(f.id, 'alpha'))!.deleted_at).not.toBeNull();expect((await page(f.id, 'alpha'))!.compiled_truth).toContain('version three');expect((await page(f.id, 'alpha-twin'))!.deleted_at).not.toBeNull();- // Point the twin tombstone elsewhere: one tombstone left. #6712 PR 4b (design v3.2 §2.3 row 9, `deleted_edited`): the file- // changed after its page was deleted, so the pass holds it (side deleted, state tombstone_edited) for a person's Decision- // instead of reviving the page; repair file-drift --held --prefer file is the way back.+ // Point the twin tombstone elsewhere: one tombstone left. GST-2201: sync made that tombstone (the file left in a commit), so the+ // file's return is the page's return: the pass imports the edited bytes into the same page, and the identity hold clears.await coordinated(f.id, 'UPDATE pages SET source_path=NULL WHERE source_id=$1 AND slug=$2', [f.id, 'alpha-twin']);- // The file is already held page_identity_changed, so this run re-screens it (holdRescreenDue) rather than holding it anew:- // the result lists no new hold, and the source's one hold becomes the file_drift above in place of the identity hold.const returned = await sync(f.opts);expect(returned.managedWrite).toBeUndefined();- expect(returned).toMatchObject({ status: 'up_to_date', filesImported: 0, held_count: 0, holds_outstanding: 1 });- const rescreened = (await readGitSourceHolds(engine, { sourceIds: [f.id] }))[0]?.holds ?? [];- expect(rescreened.map(item => [item.path, item.code, item.meta.side, item.meta.state, item.slug])).toEqual([['alpha.md', 'file_drift', 'deleted', 'tombstone_edited', 'alpha']]);+ expect(returned).toMatchObject({ status: 'synced', filesImported: 1 });+ expect(returned.held_count ?? 0).toBe(0);+ expect((await readGitSourceHolds(engine, { sourceIds: [f.id] }))[0]?.holds ?? []).toEqual([]);const back = (await page(f.id, 'alpha'))!;expect(back.id).toBe(survivor.id);- expect(back.deleted_at).not.toBeNull();- expect(back.compiled_truth).toContain('version three');+ expect(back.deleted_at).toBeNull();+ expect(back.compiled_truth).toContain('Alpha returns.');+ expect((await page(f.id, 'alpha-twin'))!.deleted_at).not.toBeNull();expect(await readManagedSyncFailures(engine, [f.id])).toEqual([]);}, 180_000);diff --git a/test/fixtures/fold-hook-noop-6712.json b/test/fixtures/fold-hook-noop-6712.jsonindex ae1061dce9..0dfacecaf8 100644--- a/test/fixtures/fold-hook-noop-6712.json+++ b/test/fixtures/fold-hook-noop-6712.json@@ -117,7 +117,7 @@"SELECT last_commit, config->>'slug_root_mode' AS mode FROM sources WHERE id=$1 AND incarnation=$2::uuid","WITH x AS MATERIALIZED (SELECT txid_snapshot_xmin(txid_current_snapshot()) AS now_lo, txid_snapshot_xmax(txid_current_snapshot()) AS hi, age((($4::bigint) % 4294967296)::text::xid) AS lo_age, txid_current_snapshot()::text AS now_snapshot, txid_current_if_assigned()::text AS now_own), s1 AS MATERIALIZED (SELECT age(((c.cut) % 4294967296)::text::xid) AS cut_age, ARRAY(SELECT ((t) % 4294967296)::text::xid FROM unnest(c.xip) t) AS xip, ((c.own) % 4294967296)::text::xid AS own FROM (SELECT ARRAY(SELECT txid_snapshot_xip($7::txid_snapshot)) AS xip, $8::bigint AS own, LEAST(txid_snapshot_xmax($7::txid_snapshot), (SELECT min(t) FROM txid_snapshot_xip($7::txid_snapshot) t WHERE txid_visible_in_snapshot(t, txid_current_snapshot()))) AS cut) c), pg AS (SELECT count(*) FILTER (WHERE p.source_path IS NOT NULL)::int AS n, COALESCE(jsonb_agg(jsonb_build_array(p.id::text, EXISTS (SELECT 1 FROM persistence_requests r WHERE r.id=p.revision_write_request_id AND r.source_id=$1 AND (r.id=$6::uuid OR r.state='committed' AND r.intent->>'passId'=$5 AND r.outcome->>'revision'=p.knowledge_revision::text)), p.knowledge_revision::text)) FILTER (WHERE NOT (age(p.xmin) > x.lo_age OR COALESCE(age(p.xmin) > s1.cut_age AND p.xmin <> ALL(s1.xip) AND p.xmin IS DISTINCT FROM s1.own, false))), '[]'::jsonb) AS changed FROM pages p, x, s1 WHERE p.source_id=$1), oc AS (SELECT o.op, count(*)::int AS n, jsonb_agg(jsonb_build_array(o.op, o.fingerprint, o.completed_keys->0->>'revision', o.completed_keys->0->>'run_id')) FILTER (WHERE NOT (age(o.xmin) > x.lo_age OR COALESCE(age(o.xmin) > s1.cut_age AND o.xmin <> ALL(s1.xip) AND o.xmin IS DISTINCT FROM s1.own, false))) AS changed FROM op_checkpoints o, x, s1 WHERE o.op = ANY($2::text[]) AND (o.fingerprint = $3 OR starts_with(o.fingerprint, $3 || ':')) GROUP BY o.op) SELECT x.now_lo::text AS xmin, x.now_snapshot AS snapshot, x.now_own AS own, (x.hi - $4::bigint)::float8 AS span, pg.n AS pages, pg.changed AS pages_changed, (SELECT COALESCE(jsonb_object_agg(op, n), '{}'::jsonb) FROM oc) AS ops, (SELECT COALESCE(jsonb_agg(c), '[]'::jsonb) FROM oc, jsonb_array_elements(COALESCE(oc.changed, '[]'::jsonb)) c) AS ops_changed, (SELECT COALESCE(jsonb_agg(fingerprint), '[]'::jsonb) FROM op_checkpoints WHERE op='sync-hold' AND starts_with(fingerprint, $3 || ':')) AS holds FROM x, pg","SELECT 1 FROM persistence_requests r WHERE r.source_id=$1 AND r.slug='__managed_sync_checkpoint__' AND r.intent IS NOT NULL AND NOT r.compacted AND r.operation='submit_job' AND r.intent->>'kind'='managed_sync_checkpoint' AND r.state IN ('failed','conflict','cancelled') AND NOT EXISTS (SELECT 1 FROM persistence_requests c WHERE (c.source_id=r.source_id AND c.slug=r.slug AND c.sequence>r.sequence AND ( (c.digest=r.digest AND c.state IN ('committed','failed','queued','running','recovering')) OR (c.state IN ('committed','queued','running','recovering') AND (c.operation IN ('delete_page','purge_page') OR r.operation='put_page')) OR (r.slug='__managed_sync_checkpoint__' AND r.operation='submit_job' AND c.state='committed' AND c.operation='submit_job' AND COALESCE((SELECT k.intent->>'kind' FROM persistence_requests k WHERE k.id=r.id),'managed_sync_checkpoint')='managed_sync_checkpoint' AND COALESCE(c.intent->>'kind','managed_sync_checkpoint')='managed_sync_checkpoint') OR (r.operation='put_page' AND c.state='failed' AND c.operation='put_page' AND COALESCE(NOT (c.intent ? 'kind'), false)) OR (r.operation='submit_job' AND c.operation='submit_job' AND c.state='committed' AND COALESCE(c.intent ? 'retryOf', false) AND EXISTS (SELECT 1 FROM persistence_requests k, persistence_requests cc WHERE k.id=r.id AND cc.sequence=c.sequence AND k.intent->>'kind' IN ('connector_v2_import','connector_v2_delete','connector_v2_checkpoint') AND cc.intent->>'kind'=k.intent->>'kind' AND cc.principal_kind=k.principal_kind AND cc.principal_id=k.principal_id AND cc.source_incarnation=k.source_incarnation AND EXISTS (WITH RECURSIVE lineage(request_id, sequence, intent, depth) AS ( SELECT cc.request_id, cc.sequence, cc.intent, 0 UNION ALL SELECT p.request_id, p.sequence, p.intent, l.depth+1 FROM lineage l JOIN persistence_requests p ON p.principal_kind=k.principal_kind AND p.principal_id=k.principal_id AND p.request_id=CASE WHEN l.intent->>'retryOf' ~ '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' THEN (l.intent->>'retryOf')::uuid END WHERE l.depth<32 AND p.sequence<l.sequence AND p.source_id=k.source_id AND p.source_incarnation=k.source_incarnation AND p.operation=k.operation AND p.slug=k.slug AND p.intent->>'kind'=k.intent->>'kind') SELECT 1 FROM lineage WHERE request_id=k.request_id)))))) LIMIT 1",- "SELECT p.id, p.slug, p.source_path, p.knowledge_revision::text AS revision, p.deleted_at IS NOT NULL AS deleted, p.revision_write_request_id::text AS request_id, c.fingerprint IS NOT NULL AS has_record, c.origin IS NOT DISTINCT FROM p.source_path AS rec_origin_same, CASE WHEN c.origin IS DISTINCT FROM p.source_path THEN c.origin END AS rec_origin, c.blob_oid AS rec_blob_oid, c.disk_blob_oid IS NOT DISTINCT FROM c.blob_oid AS rec_disk_same, CASE WHEN c.disk_blob_oid IS DISTINCT FROM c.blob_oid THEN c.disk_blob_oid END AS rec_disk_blob_oid, c.revision IS NULL AS rec_revision_null, NULLIF(c.revision, p.knowledge_revision::text) AS rec_revision, CASE WHEN c.parser IS DISTINCT FROM $4::jsonb THEN c.parser::text END AS rec_parser, c.parser = $4::jsonb AS rec_parser_expected, c.full_episode AS rec_full_episode, c.working AS rec_working, (p.page_kind = 'markdown' AND p.chunker_version = 4 AND p.body_storage IS DISTINCT FROM 'segmented' AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current FROM pages p LEFT JOIN (SELECT fingerprint, completed_keys->0->>'origin' AS origin, completed_keys->0->>'blob_oid' AS blob_oid, completed_keys->0->>'disk_blob_oid' AS disk_blob_oid, completed_keys->0->>'revision' AS revision, completed_keys->0->'parser' AS parser, completed_keys->0->>'full_episode' AS full_episode, (completed_keys->0->'working')::text AS working FROM op_checkpoints WHERE op=$2) c ON c.fingerprint=$3||p.id::text WHERE p.source_id=$1 AND p.source_path IS NOT NULL AND p.id = ANY($5::bigint[]) ORDER BY p.id",+ "SELECT p.id, p.slug, p.source_path, p.knowledge_revision::text AS revision, p.deleted_at IS NOT NULL AS deleted, p.revision_write_request_id::text AS request_id, c.fingerprint IS NOT NULL AS has_record, c.origin IS NOT DISTINCT FROM p.source_path AS rec_origin_same, CASE WHEN c.origin IS DISTINCT FROM p.source_path THEN c.origin END AS rec_origin, c.blob_oid AS rec_blob_oid, c.disk_blob_oid IS NOT DISTINCT FROM c.blob_oid AS rec_disk_same, CASE WHEN c.disk_blob_oid IS DISTINCT FROM c.blob_oid THEN c.disk_blob_oid END AS rec_disk_blob_oid, c.revision IS NULL AS rec_revision_null, NULLIF(c.revision, p.knowledge_revision::text) AS rec_revision, CASE WHEN c.parser IS DISTINCT FROM $4::jsonb THEN c.parser::text END AS rec_parser, c.parser = $4::jsonb AS rec_parser_expected, c.full_episode AS rec_full_episode, c.working AS rec_working, (p.page_kind = 'markdown' AND p.chunker_version = 4 AND p.body_storage IS DISTINCT FROM 'segmented' AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current, CASE WHEN p.deleted_at IS NOT NULL THEN (EXISTS (SELECT 1 FROM op_checkpoints sdt WHERE sdt.op = 'sync-delete-tombstone' AND sdt.fingerprint = p.id::text AND p.deleted_at = (sdt.completed_keys->0->>'deleted_at')::timestamptz) OR EXISTS (SELECT 1 FROM persistence_requests sdr WHERE sdr.source_id = p.source_id AND sdr.slug = p.slug AND sdr.page_id = p.id AND sdr.intent IS NOT NULL AND NOT sdr.compacted AND sdr.state = 'committed' AND sdr.intent->>'kind' = 'managed_sync_delete' AND sdr.completed_at >= p.deleted_at)) ELSE false END AS sync_deleted FROM pages p LEFT JOIN (SELECT fingerprint, completed_keys->0->>'origin' AS origin, completed_keys->0->>'blob_oid' AS blob_oid, completed_keys->0->>'disk_blob_oid' AS disk_blob_oid, completed_keys->0->>'revision' AS revision, completed_keys->0->'parser' AS parser, completed_keys->0->>'full_episode' AS full_episode, (completed_keys->0->'working')::text AS working FROM op_checkpoints WHERE op=$2) c ON c.fingerprint=$3||p.id::text WHERE p.source_id=$1 AND p.source_path IS NOT NULL AND p.id = ANY($5::bigint[]) ORDER BY p.id","SELECT p.id::text AS id, r.intent->>'level' AS level, r.intent->>'path' AS path FROM pages p JOIN persistence_requests r ON r.id=p.revision_write_request_id WHERE p.source_id=$1 AND p.id = ANY($2::bigint[])","UPDATE sources SET last_commit=$3,last_sync_at=now(), config=jsonb_set(( WITH RECURSIVE root_layers(value, depth) AS ( SELECT COALESCE(config, '{}'::jsonb), 0 UNION ALL SELECT gbrain_try_jsonb(value #>> '{}'), depth + 1 FROM root_layers WHERE depth < 10 AND jsonb_typeof(value) = 'string' AND gbrain_try_jsonb(value #>> '{}') IS NOT NULL ), root(value) AS ( SELECT value FROM root_layers ORDER BY depth DESC LIMIT 1 ), fragment_seeds(ordinality, value) AS ( SELECT 0::bigint, value FROM root WHERE jsonb_typeof(value) = 'object' UNION ALL SELECT item.ordinality, item.value FROM root, LATERAL jsonb_array_elements( CASE WHEN jsonb_typeof(root.value) = 'array' THEN root.value ELSE '[]'::jsonb END ) WITH ORDINALITY AS item(value, ordinality) ), fragment_layers(ordinality, value, depth) AS ( SELECT ordinality, value, 0 FROM fragment_seeds UNION ALL SELECT ordinality, gbrain_try_jsonb(value #>> '{}'), depth + 1 FROM fragment_layers WHERE depth < 10 AND jsonb_typeof(value) = 'string' AND gbrain_try_jsonb(value #>> '{}') IS NOT NULL ), fragments AS ( SELECT DISTINCT ON (ordinality) ordinality, value FROM fragment_layers ORDER BY ordinality, depth DESC ) SELECT COALESCE( jsonb_object_agg(entry.key, entry.value ORDER BY fragments.ordinality), '{}'::jsonb ) FROM fragments CROSS JOIN LATERAL jsonb_each( CASE WHEN jsonb_typeof(fragments.value) = 'object' THEN fragments.value ELSE '{}'::jsonb END ) AS entry(key, value) ),'{slug_root_mode}',to_jsonb(CASE WHEN $6::boolean THEN COALESCE(config->>'slug_root_mode',$5::text) ELSE $5::text END)), newest_content_at=(SELECT MAX(updated_at) FROM pages WHERE source_id=$1 AND deleted_at IS NULL) WHERE id=$1 AND incarnation=$2::uuid AND (last_commit IS NOT DISTINCT FROM $4 OR last_commit=$3) AND ($6::boolean OR config->>'slug_root_mode' IS NULL OR config->>'slug_root_mode'=$5) RETURNING id","INSERT INTO op_checkpoints(op,fingerprint,completed_keys,updated_at) VALUES ($1, $2, jsonb_build_array(jsonb_strip_nulls(jsonb_build_object('run_id',$3::text,'at',now(),'changed',$4::int,'extract',$5::boolean,'since',$6::text))), now()) ON CONFLICT (op,fingerprint) DO UPDATE SET updated_at=now(), completed_keys=CASE WHEN jsonb_array_length(op_checkpoints.completed_keys)>=16 THEN (op_checkpoints.completed_keys-0)||EXCLUDED.completed_keys ELSE op_checkpoints.completed_keys||EXCLUDED.completed_keys END",@@ -250,7 +250,7 @@"WITH x AS MATERIALIZED (SELECT txid_snapshot_xmin(txid_current_snapshot()) AS now_lo, txid_snapshot_xmax(txid_current_snapshot()) AS hi, age((($4::bigint) % 4294967296)::text::xid) AS lo_age, txid_current_snapshot()::text AS now_snapshot, txid_current_if_assigned()::text AS now_own), s1 AS MATERIALIZED (SELECT age(((c.cut) % 4294967296)::text::xid) AS cut_age, ARRAY(SELECT ((t) % 4294967296)::text::xid FROM unnest(c.xip) t) AS xip, ((c.own) % 4294967296)::text::xid AS own FROM (SELECT ARRAY(SELECT txid_snapshot_xip($7::txid_snapshot)) AS xip, $8::bigint AS own, LEAST(txid_snapshot_xmax($7::txid_snapshot), (SELECT min(t) FROM txid_snapshot_xip($7::txid_snapshot) t WHERE txid_visible_in_snapshot(t, txid_current_snapshot()))) AS cut) c), pg AS (SELECT count(*) FILTER (WHERE p.source_path IS NOT NULL)::int AS n, COALESCE(jsonb_agg(jsonb_build_array(p.id::text, EXISTS (SELECT 1 FROM persistence_requests r WHERE r.id=p.revision_write_request_id AND r.source_id=$1 AND (r.id=$6::uuid OR r.state='committed' AND r.intent->>'passId'=$5 AND r.outcome->>'revision'=p.knowledge_revision::text)), p.knowledge_revision::text)) FILTER (WHERE NOT (age(p.xmin) > x.lo_age OR COALESCE(age(p.xmin) > s1.cut_age AND p.xmin <> ALL(s1.xip) AND p.xmin IS DISTINCT FROM s1.own, false))), '[]'::jsonb) AS changed FROM pages p, x, s1 WHERE p.source_id=$1), oc AS (SELECT o.op, count(*)::int AS n, jsonb_agg(jsonb_build_array(o.op, o.fingerprint, o.completed_keys->0->>'revision', o.completed_keys->0->>'run_id')) FILTER (WHERE NOT (age(o.xmin) > x.lo_age OR COALESCE(age(o.xmin) > s1.cut_age AND o.xmin <> ALL(s1.xip) AND o.xmin IS DISTINCT FROM s1.own, false))) AS changed FROM op_checkpoints o, x, s1 WHERE o.op = ANY($2::text[]) AND (o.fingerprint = $3 OR starts_with(o.fingerprint, $3 || ':')) GROUP BY o.op) SELECT x.now_lo::text AS xmin, x.now_snapshot AS snapshot, x.now_own AS own, (x.hi - $4::bigint)::float8 AS span, pg.n AS pages, pg.changed AS pa","SELECT 1 FROM persistence_requests r WHERE r.source_id=$1 AND r.slug='__managed_sync_checkpoint__' AND r.intent IS NOT NULL AND NOT r.compacted AND r.operation='submit_job' AND r.intent->>'kind'='managed_sync_checkpoint' AND r.state IN ('failed','conflict','cancelled') AND NOT EXISTS (SELECT 1 FROM persistence_requests c WHERE (c.source_id=r.source_id AND c.slug=r.slug AND c.sequence>r.sequence AND ( (c.digest=r.digest AND c.state IN ('committed','failed','queued','running','recovering')) OR (c.state IN ('committed','queued','running','recovering') AND (c.operation IN ('delete_page','purge_page') OR r.operation='put_page')) OR (r.slug='__managed_sync_checkpoint__' AND r.operation='submit_job' AND c.state='committed' AND c.operation='submit_job' AND COALESCE((SELECT k.intent->>'kind' FROM persistence_requests k WHERE k.id=r.id),'managed_sync_checkpoint')='managed_sync_checkpoint' AND COALESCE(c.intent->>'kind','managed_sync_checkpoint')='managed_sync_checkpoint') OR (r.operation='put_page' AND c.state='failed' AND c.operation='put_page' AND COALESCE(NOT (c.intent ? 'kind'), false)) OR (r.operation='submit_job' AND c.operation='submit_job' AND c.state='committed' AND COALESCE(c.intent ? 'retryOf', false) AND EXISTS (SELECT 1 FROM persistence_requests k, persistence_requests cc WHERE k.id=r.id AND cc.sequence=c.sequence AND k.intent->>'kind' IN ('connector_v2_import','connector_v2_delete','connector_v2_checkpoint') AND cc.intent->>'kind'=k.intent->>'kind' AND cc.principal_kind=k.principal_kind AND cc.principal_id=k.principal_id AND cc.source_incarnation=k.source_incarnation AND EXISTS (WITH RECURSIVE lineage(request_id, sequence, intent, depth) AS ( SELECT cc.request_id, cc.sequence, cc.intent, 0 UNION ALL SELECT p.request_id, p.sequence, p.intent, l.depth+1 FROM lineage l JOIN persistence_requests p ON p.principal_kind=k.principal_kind AN","(SELECT 'r:'||request_id::text AS id FROM persistence_requests WHERE state IN ('queued','running','recovering') AND (worktree_id=$1::uuid OR worktree_id IS NULL AND source_id=$4 AND source_incarnation=$5::uuid) AND request_id IS DISTINCT FROM $2::uuid LIMIT $3) UNION ALL (SELECT 'e:'||e.id::text FROM persistence_effects e JOIN persistence_requests r ON r.id=e.request_id WHERE r.worktree_id=$1::uuid AND e.kind='git' AND e.state IN ('queued','running') LIMIT $3)",- "SELECT p.id, p.slug, p.source_path, p.knowledge_revision::text AS revision, p.deleted_at IS NOT NULL AS deleted, p.revision_write_request_id::text AS request_id, c.fingerprint IS NOT NULL AS has_record, c.origin IS NOT DISTINCT FROM p.source_path AS rec_origin_same, CASE WHEN c.origin IS DISTINCT FROM p.source_path THEN c.origin END AS rec_origin, c.blob_oid AS rec_blob_oid, c.disk_blob_oid IS NOT DISTINCT FROM c.blob_oid AS rec_disk_same, CASE WHEN c.disk_blob_oid IS DISTINCT FROM c.blob_oid THEN c.disk_blob_oid END AS rec_disk_blob_oid, c.revision IS NULL AS rec_revision_null, NULLIF(c.revision, p.knowledge_revision::text) AS rec_revision, CASE WHEN c.parser IS DISTINCT FROM $4::jsonb THEN c.parser::text END AS rec_parser, c.parser = $4::jsonb AS rec_parser_expected, c.full_episode AS rec_full_episode, c.working AS rec_working, (p.page_kind = 'markdown' AND p.chunker_version = 4 AND p.body_storage IS DISTINCT FROM 'segmented' AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current FROM pages p LEFT JOIN (SELECT fingerprint, completed_keys->0->>'origin' AS origin, completed_keys->0->>'blob_oid' AS blob_oid, completed_keys->0->>'disk_blob_oid' AS disk_blob_oid, completed_keys->0->>'revision' AS revision, completed_keys->0->'parser' AS parser, completed_keys->0->>'full_episode' AS full_episode, (completed_keys->0->'working')::text AS working FROM op_checkpoints WHERE op=$2) c ON c.fingerprint=$3||p.id::text WHERE p.source_id=$1 AND p.source_path IS NOT NULL AND p.id = ANY($5::bigint[]) ORDER BY p.id",+ "SELECT p.id, p.slug, p.source_path, p.knowledge_revision::text AS revision, p.deleted_at IS NOT NULL AS deleted, p.revision_write_request_id::text AS request_id, c.fingerprint IS NOT NULL AS has_record, c.origin IS NOT DISTINCT FROM p.source_path AS rec_origin_same, CASE WHEN c.origin IS DISTINCT FROM p.source_path THEN c.origin END AS rec_origin, c.blob_oid AS rec_blob_oid, c.disk_blob_oid IS NOT DISTINCT FROM c.blob_oid AS rec_disk_same, CASE WHEN c.disk_blob_oid IS DISTINCT FROM c.blob_oid THEN c.disk_blob_oid END AS rec_disk_blob_oid, c.revision IS NULL AS rec_revision_null, NULLIF(c.revision, p.knowledge_revision::text) AS rec_revision, CASE WHEN c.parser IS DISTINCT FROM $4::jsonb THEN c.parser::text END AS rec_parser, c.parser = $4::jsonb AS rec_parser_expected, c.full_episode AS rec_full_episode, c.working AS rec_working, (p.page_kind = 'markdown' AND p.chunker_version = 4 AND p.body_storage IS DISTINCT FROM 'segmented' AND p.text_projection_revision IS NOT DISTINCT FROM p.knowledge_revision AND p.contextual_retrieval_mode IS NOT NULL) AS derived_current, CASE WHEN p.deleted_at IS NOT NULL THEN (EXISTS (SELECT 1 FROM op_checkpoints sdt WHERE sdt.op = 'sync-delete-tombstone' AND sdt.fingerprint = p.id::text AND p.deleted_at = (sdt.completed_keys->0->>'deleted_at')::timestamptz) OR EXISTS (SELECT 1 FROM persistence_requests sdr WHERE sdr.source_id = p.source_id AND sdr.slug = p.slug AND sdr.page_id = p.id AND sdr.intent IS NOT NULL AND NOT sdr.compacted AND sdr.state = 'committed' AND sdr.intent->>'kind' = 'managed_sync_delete' AND sdr.completed_at >= p.deleted_at)) ELSE false END AS sync_deleted FROM pages p LEFT JOIN (SELECT fingerprint, completed_keys->0->>'origin' AS origin, completed_keys->0->>'blob_oid' AS blob_oid, completed_keys->0->>'disk_blob_oid' AS disk_blob_oid, completed_keys->0->>'revision' AS revision, completed_keys->0->'pa","SELECT p.id::text AS id, r.intent->>'level' AS level, r.intent->>'path' AS path FROM pages p JOIN persistence_requests r ON r.id=p.revision_write_request_id WHERE p.source_id=$1 AND p.id = ANY($2::bigint[])","UPDATE sources SET last_commit=$3,last_sync_at=now(), config=jsonb_set(( WITH RECURSIVE root_layers(value, depth) AS ( SELECT COALESCE(config, '{}'::jsonb), 0 UNION ALL SELECT gbrain_try_jsonb(value #>> '{}'), depth + 1 FROM root_layers WHERE depth < 10 AND jsonb_typeof(value) = 'string' AND gbrain_try_jsonb(value #>> '{}') IS NOT NULL ), root(value) AS ( SELECT value FROM root_layers ORDER BY depth DESC LIMIT 1 ), fragment_seeds(ordinality, value) AS ( SELECT 0::bigint, value FROM root WHERE jsonb_typeof(value) = 'object' UNION ALL SELECT item.ordinality, item.value FROM root, LATERAL jsonb_array_elements( CASE WHEN jsonb_typeof(root.value) = 'array' THEN root.value ELSE '[]'::jsonb END ) WITH ORDINALITY AS item(value, ordinality) ), fragment_layers(ordinality, value, depth) AS ( SELECT ordinality, value, 0 FROM fragment_seeds UNION ALL SELECT ordinality, gbrain_try_jsonb(value #>> '{}'), depth + 1 FROM fragment_layers WHERE depth < 10 AND jsonb_typeof(value) = 'string' AND gbrain_try_jsonb(value #>> '{}') IS NOT NULL ), fragments AS ( SELECT DISTINCT ON (ordinality) ordinality, value FROM fragment_layers ORDER BY ordinality, depth DESC ) SELECT COALESCE( jsonb_object_agg(entry.key, entry.value ORDER BY fragments.ordinality), '{}'::jsonb ) FROM fragments CROSS JOIN LATERAL jsonb_each( CASE WHEN jsonb_typeof(fragments.value) = 'object' THEN fragments.value ELSE '{}'::jsonb END ) AS entry(key, value) ),'{slug_root_mode}',to_jsonb(CASE WHEN $6::boolean THEN COALESCE(config->>'slug_root_mode',$5::text) ELSE $5::text END)), newest_content_at=(SELECT MAX(updated_at) FROM pages WHERE source_id=$1 AND deleted_at IS NULL) WHERE id=$1 AND incarnation=$2::uuid AND (last_commit IS NOT DISTINCT FROM $4 OR last_commit=$3) AND ($6::bo","INSERT INTO op_checkpoints(op,fingerprint,completed_keys,updated_at) VALUES ($1, $2, jsonb_build_array(jsonb_strip_nulls(jsonb_build_object('run_id',$3::text,'at',now(),'changed',$4::int,'extract',$5::boolean,'since',$6::text))), now()) ON CONFLICT (op,fingerprint) DO UPDATE SET updated_at=now(), completed_keys=CASE WHEN jsonb_array_length(op_checkpoints.completed_keys)>=16 THEN (op_checkpoints.completed_keys-0)||EXCLUDED.completed_keys ELSE op_checkpoints.completed_keys||EXCLUDED.completed_keys END",diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 5204d12ad7..9d33754154 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -573,6 +573,7 @@ test/sync-pass-stop-in-level-gbra133.test.tstest/sync-plain-directory-7973.test.tstest/sync-prescreen-both-delta-6712.test.tstest/sync-prescreen-drift-6712.test.ts+test/sync-restored-file-purge.test.tstest/sync-slug-root-mode-6694.test.tstest/sync-staged-catchup.test.tstest/sync-status-failure-detail-6513.test.tsdiff --git a/test/purge-eval-fixes.test.ts b/test/purge-eval-fixes.test.tsindex 564f953714..5a1b8367c9 100644--- a/test/purge-eval-fixes.test.ts+++ b/test/purge-eval-fixes.test.ts@@ -92,6 +92,19 @@ describe('purge eval fixes', () => {expect(done.stores.find((s: { store: string }) => s.store === 'persistence_requests')).toMatchObject({ status: 'deleted' });}, 60_000);+ test('W3: a page purge leaves no timeline text in its add_timeline_entry receipts, compacted ones included', async () => {+ await seed();+ const kept = await call('add_timeline_entry', { slug: 'people/alice-example', date: '2026-02-03', summary: 'Moved house TIMELINE-MARK-ONE', request_id: randomUUID() });+ const compacted = await call('add_timeline_entry', { slug: 'people/alice-example', date: '2026-02-04', summary: 'Changed banks TIMELINE-MARK-TWO', request_id: randomUUID() });+ expect([kept.status, compacted.status]).toEqual(['ok', 'ok']);+ await engine.executeRaw(`UPDATE persistence_requests SET intent=NULL, compacted=true WHERE operation='add_timeline_entry' AND outcome->'entry'->>'summary' LIKE '%MARK-TWO'`);+ expect(await journalCarries('TIMELINE-MARK')).toBe(2);+ expect((await purgePage('people/alice-example')).status).toBe('purged');+ expect(await journalCarries('TIMELINE-MARK')).toBe(0);+ const outcomes = await engine.executeRaw<{ outcome: { entry: { summary: string }; bullet: string } }>(`SELECT outcome FROM persistence_requests WHERE operation='add_timeline_entry' ORDER BY sequence`);+ expect(outcomes.map(o => [o.outcome.entry.summary, o.outcome.bullet])).toEqual([['[purged]', '[purged]'], ['[purged]', '[purged]']]);+ }, 60_000);+test('39-3: remembering a purged claim refuses with purged_content (verb path, local and remote mapping)', async () => {const id = await seed();await purgeFact(id);diff --git a/test/sync-classify-6712.test.ts b/test/sync-classify-6712.test.tsindex 125274ae08..51fed3183a 100644--- a/test/sync-classify-6712.test.ts+++ b/test/sync-classify-6712.test.ts@@ -24,7 +24,8 @@ const CLEAN: ReadonlySet<SyncDirtiness | null> = new Set(['clean', 'clean_eol',/** The reference model: page state first, then level, then dirtiness, then the fileSame × dbSame × proof regions. */function reference(s: SyncPathState): SyncClass {- if (s.hold && s.hold.atLevel && !s.hold.rescreenDue && !s.hold.retried) return 'held';+ const syncRestored = s.page?.live === false && s.page.syncDeleted === true && s.level !== null;+ if (s.hold && s.hold.atLevel && !s.hold.rescreenDue && !s.hold.retried && !syncRestored) return 'held';const rest = referenceAfterHold(s);if (!s.hold) return rest;if (s.level !== null && (rest === 'matched' || rest === 'import')) return 'held';@@ -44,6 +45,7 @@ function referenceAfterHold(s: SyncPathState): SyncClass {if (state !== 'live') {if (s.mode === 'committed' && s.tree === null) return 'untracked';if (s.level === null) return 'gone';+ if (state === 'tombstone' && s.page!.syncDeleted) return dirtyCommitted ? dirtyClass() : 'import';const markerHolds = s.marker?.kind === 'deleted' && s.marker.blobOids.includes(s.level);if ((state === 'tombstone' && fileSame) || markerHolds) return 'deleted_same';const deletedEvidence = state === 'tombstone' || s.marker?.kind === 'deleted';@@ -92,6 +94,9 @@ describe('rows (v3.2 §2.3)', () => {['8 deleted_same (tombstone)', { page: { live: false, revision: 'R1', originMatches: true } }, 'deleted_same', true],['8 deleted_same (marker)', { page: null, record: null, marker: { kind: 'deleted', blobOids: ['A'] } }, 'deleted_same', true],['9 deleted_edited (tombstone)', { tree: 'B', level: 'B', page: { live: false, revision: 'R1', originMatches: true } }, 'deleted_edited', false],+ ['8 a sync-made tombstone whose file is back unchanged imports (a revert, a branch switch)', { page: { live: false, revision: 'R1', originMatches: true, syncDeleted: true } }, 'import', false],+ ['9 a sync-made tombstone whose file is back edited imports', { tree: 'B', level: 'B', page: { live: false, revision: 'R1', originMatches: true, syncDeleted: true } }, 'import', false],+ ['7 a sync-made tombstone whose file is still gone stays gone', { mode: 'working', tree: 'A', level: null, present: false, dirty: null, page: { live: false, revision: 'R1', originMatches: true, syncDeleted: true } }, 'gone', true],['9 deleted_edited (marker without OIDs, fail closed)', { page: null, record: null, marker: { kind: 'deleted', blobOids: [] } }, 'deleted_edited', false],['10 legacy_same: a marker that records a deletion', { page: null, record: null, marker: { kind: 'legacy', blobOids: ['A'], deletion: 'req-1' } }, 'legacy_same', true],['10 a baseline-equal legacy marker without a deletion settles nothing (b1)', { page: null, record: null, marker: { kind: 'legacy', blobOids: ['A'] } }, 'import', false],@@ -162,7 +167,7 @@ const stateArb: fc.Arbitrary<SyncPathState> = fc.record({dirty: fc.constantFrom<SyncDirtiness | null>('clean', 'clean_eol', 'clean_lfs', 'modified', 'deleted', 'staged', 'untracked', 'unknown', null),diskOid: oid,page: fc.constantFrom<SyncPathState['page']>(null, { live: true, revision: 'R1', originMatches: true }, { live: true, revision: 'R1', originMatches: false },- { live: true, revision: 'R2', originMatches: true }, { live: false, revision: 'R1', originMatches: true }),+ { live: true, revision: 'R2', originMatches: true }, { live: false, revision: 'R1', originMatches: true }, { live: false, revision: 'R1', originMatches: true, syncDeleted: true }),record: fc.constantFrom<SyncPathState['record']>(null, { blobOids: ['A'], revision: null, parserSame: true }, { blobOids: ['A'], revision: 'R1', parserSame: true },{ blobOids: ['A'], revision: 'R1', parserSame: false }, { blobOids: ['A', 'B'], revision: 'R1', parserSame: true }, { blobOids: ['A'], revision: 'R2', parserSame: true }),attested: fc.constantFrom<SyncPathState['attested']>(null, { hash: sha('A'), arm1: false }, { hash: sha('B'), arm1: false }, { hash: null, arm1: true }),@@ -213,15 +218,23 @@ describe('P0 properties', () => {expect(n).toBeGreaterThan(100_000);});- test('(j) no page-less import while a deleted marker or tombstone matches the level', () => {+ test('(j) no page-less import while a deleted marker or a person\'s tombstone matches the level', () => {fc.assert(fc.property(stateArb, s => {const live = s.page?.live === true;const tombstoneMatches = s.page !== null && !s.page.live && s.level !== null && !!s.record?.blobOids.includes(s.level);const markerMatches = s.marker?.kind === 'deleted' && s.level !== null && s.marker.blobOids.includes(s.level);- if (!live && (tombstoneMatches || markerMatches)) expect(classifySyncPath(s).action).not.toBe('import');+ if (!live && !s.page?.syncDeleted && (tombstoneMatches || markerMatches)) expect(classifySyncPath(s).action).not.toBe('import');}), { numRuns: 40_000, seed: 67121 });});+ test('(j2) a tombstone sync made from a file\'s absence imports the file once it is back, a hold over it included', () => {+ fc.assert(fc.property(stateArb, s => {+ if (s.page?.live !== false || !s.page.syncDeleted || s.level === null || s.spelling) return;+ if (s.mode === 'committed' && (s.tree === null || !['clean', 'clean_eol', 'clean_lfs', 'deleted'].includes(s.dirty!))) return;+ expect(classifySyncPath(s)).toMatchObject(s.hold ? { cls: 'held', action: 'import', settled: false } : { cls: 'import', action: 'import', settled: false });+ }), { numRuns: 40_000, seed: 22011 });+ });+test('(k) an edit to a swept legacy page imports with no Decision', () => {// The legacy sweep (v3.2 §2.7) writes {blob_oid, disk_blob_oid, revision, parser: fp} for an equal legacy page.const swept = at({ record: { blobOids: ['A'], revision: 'R1', parserSame: true }, baseline: 'A' });diff --git a/test/sync-restored-file-purge.test.ts b/test/sync-restored-file-purge.test.tsnew file mode 100644index 0000000000..24de2ef2db--- /dev/null+++ b/test/sync-restored-file-purge.test.ts@@ -0,0 +1,294 @@+/**+ * A file a managed sync deleted (its commit removed the file) and Git later put back stays in the brain (GST-2201).+ *+ * Protects: when a commit deletes a Markdown file, sync soft-deletes its page; when the same file comes back (a revert, a+ * branch switch and back, a later commit that re-adds it, with the same or edited bytes), the next sync brings the page+ * back (same page row) instead of settling it as deleted, so search, get and recall see it again and sync status never+ * says synced while the page is missing. A brain an older version left in that state heals on its next plain sync. The+ * retention purge (the cycle purge phase, the purge job and the consented `gbrain pages purge-deleted`) never removes a+ * file that is committed at the source's last commit or at HEAD: a sync-made tombstone whose file is back waits for the+ * sync that restores it; a person's delete undone in Git, and a tombstone gbrain can no longer tell was sync-made, are+ * purged database-only with the file left alone; a person's delete whose file stays gone purges as before.+ *+ * PGLite always; Postgres with DATABASE_URL (test/postgres-unit-arms.txt).+ */+import { describe, expect, spyOn, test } from 'bun:test';+import { execFileSync } from 'node:child_process';+import { randomUUID } from 'node:crypto';+import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { runCycle } from '../src/core/cycle.ts';+import { makePurgeHandler } from '../src/core/minions/handlers/purge.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { performManagedSync } from '../src/core/persistence/sync-run.ts';+import { readSyncStatus } from '../src/core/persistence/sync-status.ts';+import { MANAGED_SYNC_PASS_OP } from '../src/core/persistence/sync-pass-row.ts';+import { SYNC_DELETE_TOMBSTONE_OP } from '../src/core/persistence/sync-delete-tombstones.ts';+import { declarePersistenceProtocol } from '../src/core/persistence/protocol.ts';+import { __resetTreeReuseForTests } from '../src/core/persistence/sync-tree-reuse.ts';+import { runPages } from '../src/commands/pages.ts';+import { managedBrain, settleManagedGitEffects } from './helpers/managed-brain.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';+import { withEnv } from './helpers/with-env.ts';++const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();+const commit = (root: string, message: string) => { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', message); };+const note = (title: string, body: string) => `---\ntype: concept\ntitle: ${title}\n---\n\n${body}\n`;+const OPTS = { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true };+const A = 'notes/doc-a', B = 'notes/doc-b', TWIN = 'notes/twin', COPY = 'notes/twin-copy', USER = 'notes/user', KEEP = 'notes/keep';+const BODY: Record<string, string> = { [A]: note('Doc A', 'Alice-example wrote about topic a.'), [B]: note('Doc B', 'Notes on acme-example, topic b.'),+ [TWIN]: note('Twin', 'The same words in two files.'), [COPY]: note('Twin', 'The same words in two files.'), [USER]: note('User', 'A page a person deletes.'),+ [KEEP]: note('Keep', 'A page nobody touches.') };+const setup = ({ root }: { root: string }) => {+ mkdirSync(join(root, 'notes'), { recursive: true }); git(root, 'init', '-q', '-b', 'main');+ for (const [slug, body] of Object.entries(BODY)) writeFileSync(join(root, `${slug}.md`), body);+ commit(root, 'notes');+};+const CONSENT_ENV = { GBRAIN_CONSENT: undefined };++const page = async (engine: BrainEngine, slug: string) => (await engine.executeRaw<{ id: number | string; deleted_at: string | null; compiled_truth: string }>(+ 'SELECT id, deleted_at, compiled_truth FROM pages WHERE source_id=$1 AND slug=$2', ['default', slug]))[0];+const sync = async (engine: BrainEngine) => { await performManagedSync(engine, OPTS); };+const status = async (engine: BrainEngine) => { const s = await readSyncStatus(engine, 'default', { detail: false }); return { state: s.state, reason: s.reason }; };+const removeAndSync = async (engine: BrainEngine, root: string, slugs: string[]) => {+ git(root, 'rm', '-q', ...slugs.map(slug => `${slug}.md`)); commit(root, 'remove notes');+ await sync(engine);+ for (const slug of slugs) expect((await page(engine, slug))?.deleted_at).not.toBeNull();+};+/** Ages every tombstone (and its sync-deletion record) by `hours`, as time would. A fixture write, outside coordination. */+async function age(engine: BrainEngine, hours: number): Promise<void> {+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw(`UPDATE pages SET deleted_at = deleted_at - ($1 || ' hours')::interval WHERE source_id='default' AND deleted_at IS NOT NULL`, [String(hours)]);+ await engine.executeRaw(`UPDATE op_checkpoints c SET completed_keys = jsonb_set(c.completed_keys, '{0,deleted_at}', to_jsonb(p.deleted_at)) FROM pages p+ WHERE c.op=$1 AND p.id::text = c.fingerprint`, [SYNC_DELETE_TOMBSTONE_OP]);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+}+async function cli(engine: BrainEngine, args: string[]): Promise<void> {+ const log = spyOn(console, 'log').mockImplementation(() => {});+ const err = spyOn(console, 'error').mockImplementation(() => {});+ try { await withEnv(CONSENT_ENV, () => runPages(engine, args)); }+ finally { log.mockRestore(); err.mockRestore(); }+}+type Purge = 'cycle phase' | 'purge job' | 'CLI purge-deleted';+const PURGES: Purge[] = ['cycle phase', 'purge job', 'CLI purge-deleted'];+async function purge(engine: BrainEngine, root: string, how: Purge): Promise<void> {+ if (how === 'cycle phase') await withEnv(CONSENT_ENV, () => runCycle(engine, { brainDir: root, phases: ['purge'], sourceId: 'default' }));+ else if (how === 'purge job') await withEnv(CONSENT_ENV, () => makePurgeHandler(engine)({ data: { scope: 'pages' } } as never));+ else await cli(engine, ['purge-deleted', '--json', '--yes']);+}+/** The file is on disk with its committed bytes and the checkout has no change to it. */+const fileKept = (root: string, slug: string) => {+ expect(existsSync(join(root, `${slug}.md`))).toBe(true);+ expect(readFileSync(join(root, `${slug}.md`), 'utf8')).toBe(BODY[slug]!);+ expect(git(root, 'status', '--porcelain', '--', `${slug}.md`)).toBe('');+};+/** A person deletes `slug` through delete_page; its write-through publication settles. */+async function personDelete(engine: BrainEngine, ctx: OperationContext, slug: string): Promise<void> {+ const snapshot = await engine.readPageSnapshot(slug, { sourceId: 'default' });+ await submitPageMutation(ctx, { operation: 'delete_page', params: { slug, expected_revision: snapshot!.revision, request_id: randomUUID() } });+ await settleManagedGitEffects(engine);+ expect((await page(engine, slug))?.deleted_at).not.toBeNull();+}+/** Commits the removal the delete left in the checkout, then puts `slug`'s original bytes back in a new commit (an undo in Git). */+const restoreInGit = (root: string, slug: string) => {+ if (git(root, 'status', '--porcelain', '--', `${slug}.md`)) commit(root, `remove ${slug}`);+ writeFileSync(join(root, `${slug}.md`), BODY[slug]!); commit(root, `restore ${slug}`);+};++for (const backend of testBackends()) {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ describe(`${backend}: a file Git restores after a sync deleted its page comes back`, () => {+ test('git revert of the deleting commit: the same page row is live again and status is synced', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ const before = await page(engine, A);+ await removeAndSync(engine, root, [A]);+ git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'revert', '--no-edit', 'HEAD');+ await sync(engine);+ const after = await page(engine, A);+ expect(after?.deleted_at).toBeNull();+ expect(String(after?.id)).toBe(String(before?.id));+ expect(after?.compiled_truth).toContain('topic a');+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ expect((await engine.executeRaw('SELECT 1 FROM op_checkpoints c JOIN pages p ON p.id::text=c.fingerprint WHERE c.op=$1 AND p.deleted_at IS NOT NULL', [SYNC_DELETE_TOMBSTONE_OP])).length).toBe(0);+ }, { setup, databaseUrl }), 300_000);++ test('a sync on a branch without the files, then back on main: the pages are live again', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ git(root, 'checkout', '-q', '-b', 'side');+ await removeAndSync(engine, root, [A, B]);+ git(root, 'checkout', '-q', 'main');+ await sync(engine);+ for (const slug of [A, B]) expect((await page(engine, slug))?.deleted_at).toBeNull();+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);++ test('deleted in one commit, re-added by a later one (same bytes and edited): both pages are live with the committed text', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ await removeAndSync(engine, root, [A, B]);+ writeFileSync(join(root, `${A}.md`), BODY[A]!);+ writeFileSync(join(root, `${B}.md`), note('Doc B', 'Notes on acme-example, topic b, revised by a collaborator.'));+ commit(root, 're-add notes');+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).toBeNull();+ const b = await page(engine, B);+ expect(b?.deleted_at).toBeNull();+ expect(b?.compiled_truth).toContain('revised by a collaborator');+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);++ test('a sync on a detached older commit, then back on main: the page the newest commit added is live again', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ writeFileSync(join(root, 'notes/new.md'), note('New', 'Added by the newest commit.'));+ commit(root, 'add a note');+ await sync(engine);+ git(root, 'checkout', '-q', '--detach', 'HEAD~1');+ await sync(engine);+ expect((await page(engine, 'notes/new'))?.deleted_at).not.toBeNull();+ git(root, 'checkout', '-q', 'main');+ await sync(engine);+ expect((await page(engine, 'notes/new'))?.deleted_at).toBeNull();+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);++ test('a file moved into a dot-folder and back: the page is live again', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ mkdirSync(join(root, '.parked'));+ git(root, 'mv', `${A}.md`, '.parked/doc-a.md'); commit(root, 'park a note');+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).not.toBeNull();+ git(root, 'mv', '.parked/doc-a.md', `${A}.md`); commit(root, 'unpark the note');+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).toBeNull();+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);++ test('working-tree mode: a file missing from disk for one sync and then back is live again', () => managedBrain(async ({ engine, root }) => {+ await engine.setConfig('sync.include_working_tree', 'true');+ await sync(engine);+ const parked = join(root, '..', 'doc-a.parked');+ renameSync(join(root, `${A}.md`), parked);+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).not.toBeNull();+ renameSync(parked, join(root, `${A}.md`));+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).toBeNull();+ fileKept(root, A);+ }, { setup, databaseUrl }), 300_000);++ test('a hold an older version raised over a sync-deleted page whose file came back edited is re-run: the page imports and the hold clears', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ await removeAndSync(engine, root, [A]);+ // An older version did not read the sync-deletion record: hide it while the edited file comes back, so that pass holds the path.+ const receipts = await engine.executeRaw<{ id: string }>(`SELECT id::text AS id FROM persistence_requests WHERE intent->>'kind'='managed_sync_delete'`);+ const setKind = (kind: string) => engine.transaction(async tx => {+ await declarePersistenceProtocol(tx);+ await tx.executeRaw(`UPDATE persistence_requests SET intent = jsonb_set(intent, '{kind}', to_jsonb($2::text)) WHERE id = ANY($1::uuid[])`, [receipts.map(r => r.id), kind]);+ });+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('UPDATE op_checkpoints SET op=$2 WHERE op=$1', [SYNC_DELETE_TOMBSTONE_OP, 'hidden-tombstone-record']);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ await setKind('hidden');+ __resetTreeReuseForTests();+ writeFileSync(join(root, `${A}.md`), note('Doc A', 'Alice-example wrote about topic a, then edited it.'));+ commit(root, 're-add edited');+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).not.toBeNull();+ expect((await status(engine)).reason).toBe('held_needs_human');+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('UPDATE op_checkpoints SET op=$1 WHERE op=$2', [SYNC_DELETE_TOMBSTONE_OP, 'hidden-tombstone-record']);+ // ...and its pass stamped its own fingerprint.+ await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys = jsonb_set(completed_keys, '{0,obligations,converged,selection_fp}', '"older-version"') WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ await setKind('managed_sync_delete');+ __resetTreeReuseForTests();+ await sync(engine);+ const a = await page(engine, A);+ expect(a?.deleted_at).toBeNull();+ expect(a?.compiled_truth).toContain('then edited it');+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);++ test('a brain an older version left with the page deleted beside its restored file heals on the next plain sync', () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ await removeAndSync(engine, root, [A]);+ git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'revert', '--no-edit', 'HEAD');+ // What an older version's pass left: converged at the restored HEAD (it settled the path as deleted_same) under its own fingerprint.+ const head = git(root, 'rev-parse', 'HEAD');+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys = jsonb_set(jsonb_set(completed_keys, '{0,obligations,converged,target}', to_jsonb($2::text)),+ '{0,obligations,converged,selection_fp}', '"older-version"') WHERE op=$1`, [MANAGED_SYNC_PASS_OP, head]);+ await engine.executeRaw("UPDATE sources SET last_commit=$1 WHERE id='default'", [head]);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ expect((await page(engine, A))?.deleted_at).not.toBeNull();+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).toBeNull();+ expect(await status(engine)).toEqual({ state: 'synced', reason: 'nothing_pending' });+ }, { setup, databaseUrl }), 300_000);+ });++ describe(`${backend}: the retention purge never removes a file that is committed`, () => {+ for (const how of PURGES) {+ test(`${how}: a person's delete undone in Git is purged database-only and the file stays`, () => managedBrain(async ({ engine, ctx, root }) => {+ await sync(engine);+ await personDelete(engine, ctx, USER);+ restoreInGit(root, USER);+ await sync(engine);+ expect((await page(engine, USER))?.deleted_at).not.toBeNull();+ await age(engine, 100);+ await purge(engine, root, how);+ expect(await page(engine, USER)).toBeUndefined();+ fileKept(root, USER);+ await sync(engine);+ fileKept(root, USER);+ }, { setup, databaseUrl }), 300_000);++ test(`${how}: a sync-deleted page with a byte-identical live twin whose file is back waits for the sync that restores it`, () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ await removeAndSync(engine, root, [TWIN]);+ git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'revert', '--no-edit', 'HEAD');+ await age(engine, 100);+ await purge(engine, root, how);+ expect((await page(engine, TWIN))?.deleted_at).not.toBeNull();+ fileKept(root, TWIN);+ await sync(engine);+ expect((await page(engine, TWIN))?.deleted_at).toBeNull();+ fileKept(root, TWIN);+ }, { setup, databaseUrl }), 300_000);++ test(`${how}: a tombstone with no sync-deletion record and a compacted receipt is purged database-only and its committed file stays`, () => managedBrain(async ({ engine, root }) => {+ await sync(engine);+ await removeAndSync(engine, root, [A]);+ // A tombstone made before 0.60.330.0 whose receipt has since been compacted: nothing marks it as sync-made.+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('DELETE FROM op_checkpoints WHERE op=$1', [SYNC_DELETE_TOMBSTONE_OP]);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ await engine.transaction(async tx => {+ await declarePersistenceProtocol(tx);+ await tx.executeRaw(`UPDATE persistence_requests SET intent=NULL, compacted=true WHERE intent->>'kind'='managed_sync_delete'`);+ });+ __resetTreeReuseForTests(); // the next sync runs in a new process, as after an upgrade+ git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'revert', '--no-edit', 'HEAD');+ await sync(engine);+ expect((await page(engine, A))?.deleted_at).not.toBeNull();+ await age(engine, 100);+ await purge(engine, root, how);+ expect(await page(engine, A)).toBeUndefined();+ fileKept(root, A);+ }, { setup, databaseUrl }), 300_000);++ test(`${how}: a person's delete whose file stays gone purges as before`, () => managedBrain(async ({ engine, ctx, root }) => {+ await sync(engine);+ await personDelete(engine, ctx, USER);+ if (git(root, 'status', '--porcelain', '--', `${USER}.md`)) commit(root, 'remove user note');+ await sync(engine);+ await age(engine, 100);+ await purge(engine, root, how);+ expect(await page(engine, USER)).toBeUndefined();+ expect(existsSync(join(root, `${USER}.md`))).toBe(false);+ expect((await page(engine, KEEP))?.deleted_at).toBeNull();+ }, { setup, databaseUrl }), 300_000);+ }+ });+}