Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(sync): hold, never delete, missing-file pages while gbrain.yml storage config is invalid (#8007)

garrytane64a0d3d16f0
diff --git a/docs/architecture/key-files/core-utilities-2.md b/docs/architecture/key-files/core-utilities-2.md
index 0ebbba0623..2cd46c69f3 100644
--- a/docs/architecture/key-files/core-utilities-2.md
+++ b/docs/architecture/key-files/core-utilities-2.md
@@ -37,7 +37,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/sql-query.ts` — engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise<rows[]>`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` (Postgres via postgres.js `unsafe(sql, params)`, PGLite via `db.query(sql, params)`). Deliberately narrower than postgres.js's `sql` tag: no nested fragments, `sql.json()`, `sql.unsafe()`, `sql.begin()`, or array binding — the narrow scalar-only surface is the feature (keeps it from drifting into a partial postgres.js clone). JSONB writes go through `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` which composes positional `$N::jsonb` casts and passes JS **objects** through; an object reaches the wire with the correct type oid, so executeRawJsonb is safe (verified by `test/sql-query.test.ts` on PGLite, `test/e2e/auth-permissions.test.ts:67` on Postgres). Positional binding is NOT universally immune, though: binding a `JSON.stringify(x)` **string** to a bare `$N::jsonb` via `unsafe()` double-encodes it into a jsonb string scalar on real Postgres (PGLite hides it). Fixes: pass a raw object (executeRawJsonb / `sql.json`), or cast through `$N::text::jsonb`. `scripts/check-jsonb-pattern.sh` (template grep) doesn't fire on `executeRawJsonb(...)` because it passes objects; the positional `$N::jsonb` + `JSON.stringify` form is caught by `scripts/check-jsonb-params.mjs`. Also exports `FILES_METADATA_MERGE_SQL`, the one `ON CONFLICT (storage_path)` merge fragment every files upsert (`src/commands/files.ts` x4, `src/core/ops/files.ts` `file_upload`) interpolates: it resets a non-object `files.metadata` to `{}` before the jsonb `||` merge, because `||` array-wraps a non-object left operand and would turn a legacy string-scalar row into `["{}", {...}]` that `gbrain repair-jsonb` can no longer see (pinned by `test/file-upload-fail-closed.test.ts`). Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, `src/mcp/http-transport.ts` so all five work uniformly against PGLite and Postgres.
- `src/core/raw-file-upload.ts` — #5963 Part A: `managedRawUploadRefusal` is checked by `gbrain files upload-raw` (`src/commands/files.ts`) before any `mkdir`, so a git-storage upload into a managed canonical worktree refuses `writer_coordinator_required` (raw files are not coordinated in this release; fix: `gbrain files upload` with a storage backend) and creates nothing; `rawUploadRefusedFor` keeps `file_upload` (`src/core/ops/files.ts`) from suggesting upload-raw for such a page when no backend is configured.
-- `src/core/storage-config.ts` — Storage tiering: `loadStorageConfig` reads `gbrain.yml` (warning once per process for each unknown key under `storage:`; an optional `keep_db` prefix list, present only when declared, is normalized like the tiers and matched by `isKeepDbSlug`, consumed by `repair/keep-db-policy.ts`), normalizes deprecated keys (`git_tracked`/`supabase_only`) to canonical (`db_tracked`/`db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. `isDeclaredDbOnlySlug` matches a lowercase page slug against the declared db_only dirs case-insensitively; managed persistence uses it to publish those pages database-only. `hasUnresolvedDbOnlyDeclaration` detects a db_only key line that resolved no directory (unsupported syntax such as flow-style lists; an intentionally empty `db_only: []` is an accepted false positive); the sync baseline-commit guard refuses on it, and managed persistence turns a missing-file refusal into `storage_error`. Uses a dedicated parser for the `gbrain.yml` shape rather than gray-matter (broken on delimiter-less YAML). Also carries `DERIVE_PHASE_DB_ONLY_DEFAULTS` (`life/events/`, `atoms/`, `extracts/`, `dream-cycle-summaries/`) + `effectiveDbOnlyDirs` — the engine's derive-phase output prefixes treated as implicitly-declared db_only by the `undeclared_db_only_pages` doctor check but deliberately NOT merged into `loadStorageConfig` (a global merge would auto-gitignore those dirs and silently kill ingestion for brains that file-back them) — and `findDbOnlyCollisions` (pure collector-output vs db_only overlap detector shared by the `db_only_collector_collision` doctor check and sync's `manageGitignore` warning). Pinned by `test/storage-config.test.ts` + `test/doctor-silent-death-checks.test.ts`.
+- `src/core/storage-config.ts` — Storage tiering: `loadStorageConfig` reads `gbrain.yml` (warning once per process for each unknown key under `storage:`; an optional `keep_db` prefix list, present only when declared, is normalized like the tiers and matched by `isKeepDbSlug`, consumed by `repair/keep-db-policy.ts`), normalizes deprecated keys (`git_tracked`/`supabase_only`) to canonical (`db_tracked`/`db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. `isDeclaredDbOnlySlug` matches a lowercase page slug against the declared db_only dirs case-insensitively; managed persistence uses it to publish those pages database-only. `hasUnresolvedDbOnlyDeclaration` detects a db_only key line that resolved no directory (a multi-line flow list, a key outside `storage:`; one-line lists parse, `db_only: []` declares none, #8007); the sync baseline-commit guard refuses on it, and managed persistence turns a missing-file refusal into `storage_error`. Uses a dedicated parser for the `gbrain.yml` shape rather than gray-matter (broken on delimiter-less YAML). Also carries `DERIVE_PHASE_DB_ONLY_DEFAULTS` (`life/events/`, `atoms/`, `extracts/`, `dream-cycle-summaries/`) + `effectiveDbOnlyDirs` — the engine's derive-phase output prefixes treated as implicitly-declared db_only by the `undeclared_db_only_pages` doctor check but deliberately NOT merged into `loadStorageConfig` (a global merge would auto-gitignore those dirs and silently kill ingestion for brains that file-back them) — and `findDbOnlyCollisions` (pure collector-output vs db_only overlap detector shared by the `db_only_collector_collision` doctor check and sync's `manageGitignore` warning). Pinned by `test/storage-config.test.ts` + `test/doctor-silent-death-checks.test.ts`.
- `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local). `uploadFile` streams (#7427).
diff --git a/docs/guides/repair.md b/docs/guides/repair.md
index 8650412aef..0bf0e13518 100644
--- a/docs/guides/repair.md
+++ b/docs/guides/repair.md
@@ -2560,6 +2560,11 @@ so the page may hold the only copy. `--prefer file` deletes it after review;
committing the file again clears the hold. Reason `origin_unknown` means the
sync's history read did not finish, so it could not tell whether a commit held
the file and kept the page; the provenance below reads the whole history.
+Reason `storage_invalid` (#8007) means `gbrain.yml`'s storage section did not
+load, or declared `db_only` in a form that resolved no directory, so the sync
+could not tell whether the page is database-only and kept it: run
+`gbrain storage status`, fix `gbrain.yml`, commit it and sync again; the hold
+clears and the page is decided under the fixed config.
<a id="file-drift-provenance"></a>**Provenance: who deleted each gone file.** Every page of a
`--held --side deleted --state file_gone` preview names where its file went
diff --git a/docs/storage-tiering.md b/docs/storage-tiering.md
index 9cf3b4c028..e6379d5ada 100644
--- a/docs/storage-tiering.md
+++ b/docs/storage-tiering.md
@@ -35,6 +35,7 @@ storage:
Path requirements:
- Each directory must end with `/` for canonical form. The validator auto-normalizes missing trailing slashes (one-time info note shows what changed).
+- A tier may also be a one-line flow list, `db_only: [media/x/, 'media/articles/']`. A flow list split over several lines, or a `db_only` key outside `storage:`, resolves no directory: `gbrain storage status` names it, and until it is fixed a managed sync keeps every live page whose file is gone (held as `file_drift` with reason `storage_invalid`) instead of deleting it, because it cannot tell which pages are database-only (#8007). A tier overlap holds them the same way. Fix `gbrain.yml`, commit it and sync again; the holds clear and each page is decided under the fixed config.
- A directory cannot appear in both tiers — that's a tier-overlap error and `loadStorageConfig` throws `StorageConfigError`. Edit `gbrain.yml` to remove the overlap and try again.
- Any other key under `storage:` is ignored with a warning naming it (a typo such as `kep_db` used to be ignored silently).
diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsv
index e9793ed76c..2bfe533bb0 100644
--- a/scripts/module-size-limits.tsv
+++ b/scripts/module-size-limits.tsv
@@ -12,7 +12,7 @@ src/commands/extract.ts 2700 ratchet +8 GBRA-130 A-1: extract --stale --retry-he
src/commands/jobs.ts 687 ratchet refactor wave 1 split target (W4 jobs, cut line); ceiling rationale: docs/architecture/key-files/commands-4.md entry for src/commands/jobs.ts; help lines: authorize-legacy --select (#5157 T3) and jobs cancel --select (DX-T2); submit --queue-only + PGLite work drain note (agent-first operator wave E5); +2 spend authorization: jobs submit --yes/--max-usd and jobs list --group help lines; +4 fix wave 9: supervisor stop help documents the 45 s wait, --queue and the verified-drain exit codes; +1 #7142: doctor-refresh handler registration (logic in doctor/last-known.ts); +20 #7242 jobs wait: subcommand row, import, usage lines and its --help text with the exit codes (logic in jobs/wait.ts)
src/commands/serve-http.ts 941 ratchet refactor wave 1 W4 split into serve-http-<area>.ts modules; ceiling rationale: docs/architecture/key-files/commands-5.md entry for src/commands/serve-http.ts; +2 agent operator wave A6: ServeHttpContext.noticeLedger (import + field; logic in core/notice-ledger.ts); +2 agent contract F1/F2: ServeHttpContext.readinessCache (probed readiness for initialize tails + gbrain://capabilities); +5 Lane D facts drain: serve --http starts the drain scheduler and stops it before the engine disconnect (logic in core/facts/drain-scheduler.ts); +2 agent-operator follow-up Item 3: ServeHttpOptions.adoptServer (status-only recovery hands its bound listener to listenOrRefuse; logic in serve-http-status.ts); +5 #6317: managed-sync movement watch start/stop (watch in persistence/sync-movement.ts); +1 #6589: imports loadServeConfigs (operations see the DB-plane config read once at startup; logic in mcp/serve-config.ts); +5 #6858 ask 6: on-commit HEAD watcher start/stop (watcher in core/serve-head-watcher.ts); +4 GBRA-199: serve --http --advertise parse, help line and pass-through (logic in mcp/surface.ts parseAdvertiseFlag)
src/core/ai/gateway.ts 4533 ratchet +2 #6997: ChatResult.fallbackReason (interface field + doc; recorded in ai/chat-fallback.ts); +17 #6607: lazy OpenAI model discovery (discoverBeforeOpenAICall, once per reconfigure, at the chat/expand/embed entry points; it re-resolves through the gateway-internal resolveGatewayModels, so it cannot move); +18 W11 item 3: embedQuery query-embedding cache lookup/store + clear at every model-cache clear site and the test transport seam (cache lives in ai/query-embed-cache.ts); +7 fix wave 5 (#5137): provider-key redactor on every upstream error path and the first embedding 401/403 report (logic in ai/key-redact.ts, ai/key-warnings.ts); +20 System One v1: decide() delegating export + transport seam, RerankInput.onMeta/RerankMeta, TypeSafe reranker delegation to ai/decide/rerank-adapter.ts; +3 #5585: reconfigureGatewayWithEngine records expansion/chat model provenance (ai/gateway-model-sources.ts) + clears it with gateway state; lowered #5585: tracker record sites collapsed onto ai/budget-record.ts (recordOnTracker + failedCallUsage) while adding ledger fields (requested model, purpose, failed, estimated, countsAsCall); +20: schema-rejection fallback in chat() (retry once schemaless on a json_schema 4xx, remember the recipe for the process — mirrors expand()); re-summed after merging master v0.50.0.0 with the wave (structured-output + pricing hunks); +48: #4863 ChatOpts.responseSchema → tolerant json_schema Output on the chat() generateText call for openai-compatible recipes that declare structured outputs (facts extraction on Ollama); +74: #3657 post-sunset rerank short-circuit (effective-model check, once-per-process audit row + stderr, injected-clock seam); grown v0.46.26.0 megawave: config-plane probes + halt telemetry (#3387 #4312); +21 lines (OpenAI Responses API reasoning-item echo: ChatBlock 'reasoning' variant, chat()/toModelMessages() round-trip); wave-k absorbs: OCR routing seam (#4107 class), config-snapshot reader block (#3980); +21: #4107 OCR-model routing (getImageOcrModel); +10: keyless OPENAI_BASE_URL embedding override (#4385); wave-k #3622 reimpl: GBRAIN_EMBED_MAX_BATCH_TOKENS env cap for no_batch_cap recipes; +9: DeepSeek judge thinking pin plumbed through chat opts (#4069); +master v0.48 merge true-up (reranker succession: no_key preflight, reranker_skipped stage, readiness); ranker wave v0.48.4.0: ChatOpts.temperature threaded to the transport (+15); +24: #4172 recipe-declared thinking headroom on the chat()/toolLoop() default — shared isThinkingModel predicate (Claude-5 regex || recipe thinking_by_default, fail-closed), also consumed by the subagent handler; +20: #4861 claude-cli expansion routed to the schemaless viaText path (the CLI adapter ignores responseFormat, so the native generateObject branch had no fallback); retired provider integration removed; safety wave: migration reranker input ceilings; lowered by fix wave 3 #5680 (embed request planning peeled to ai/embed-batch-plan.ts); +5 #4616: empty inputs skip the provider and every text/multimodal result is screened for zero-norm/NaN/Inf vectors (logic lives in ai/embedding-guard.ts); fix wave 3 integration: #4616 empty-input filtering feeds the shared #5680 batch planner; +3 #5691: embedQuery takes a per-request queryPrefix (no gateway state); +26 fix wave 7 (#5642): embed spend records bank provider-reported usage.tokens (all sub-batches reported → measured; chars-per-token estimate stays the fallback and the failure charge); +2 #5892: import + load-time call of installAiSdkWarningWriter (writer lives in ai/sdk-warnings.ts); +12 #5490: chat() delegation to ai/chat-fallback.ts when chat_fallback_chain is set (attempt loop lives there), ChatOpts.allowFallback + ToolLoopOpts.allowFallback fields and the toolLoop forward; +2 #5490 review: ChatResult.fallbackFrom (the call model a chat_fallback_chain answer replaced); +1 fix wave lane B3: configureGateway keeps chat_fallback_on_refusal (the refusal switch lives in ai/chat-fallback.ts); +18 fix wave 9 lane B: ChatOpts.thinking (#5331, per-route mapping lives in ai/thinking-off.ts), ChatOpts.timeoutMs + ToolLoopOpts.turnTimeoutMs and the turn forward (#4921); interface fields and their docs cannot move; +2 P8: GBRAIN_AI_CALL_LOG observer install in configureGateway (logic in ai/call-log.ts); +2 W9F item 5: thinkingOffOutputCap export shared by chat() and the judge preflight estimates (capability table lives in ai/thinking-off.ts); +3 A1 (wave 0): ChatResult.usage.reasoning_tokens (interface field + doc) and the chat() reasoning-subset read beside the cache counts; the ai-sdk importer allowlist is shrink-only, so the seam stays here); +9 GOOGLE_GENERATIVE_AI_BASE_URL: resolveNativeBaseUrl('google') + the three native-google construction sites (eval metering proxy); re-summed at the master 6ccf30c3e merge (4417 integration + 4423 master - 4408 base); lowered -5: the expand() prompt template peeled to ai/expansion-prompt.ts (buildExpansionPrompt) so the longmemeval call planner measures the same text; +5 #7176: expand() writes its chat_usage_log row (phase expansion) beside the budget record, and the guardedExpansion instance that names its invocation gateway.expand (ledger in ai/chat-usage.ts); +2 PR 4 (Wave B incremental derivation, transcripts) 4b round 4: chat() forwards a metered page step's providerRetries (BudgetTracker) as the SDK's maxRetries, so every attempt is one it reserved for
-src/commands/sync.ts 558 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts)
+src/commands/sync.ts 559 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts); +1 #8007: SyncResult.storage_invalid_kept interface field (the keep lives in persistence/sync-discovery.ts)
src/core/cycle.ts 3345 ratchet +2 GBRA-130 C-2 (#6946): the sync phase reads an owned clone's pull block and reports it as pull_blocked (logic in url-clone.ts urlClonePullBlock/urlClonePullBlockSummary); +6 GBRA-130 A-1: the extract phase warns on failed or breaker-stopped stale drain pages (logic in cycle/connector-extract.ts staleDrainFailures); +1 W1 belief ledger: the extract phase runs the sampled belief drift reconcile (logic in belief/coverage-report.ts); +2 System One v1 S9: extract_facts phase tail runs the conflict sweep (ai/decide/sweep.ts) only when the slot is not off; +2 System One v1: purge phase prunes decision receipts (ai/decide/store.ts); grown v0.46.26.0 megawave (#4102 #2608) + master v0.46.26.0 cycle-start orphan recovery; +10 wave G #3974: cycle-sync uncommitted-drift warn; grown wave-g (#4416): synthesize_concepts source threading; grown wave-g (#4416): synthesize_concepts source threading (same fix absorbed as #4417 on wave-k); wave-k: ); +7: #4077 abort threading; +10: #4745 fullImplicitSourceCycle opt-in contract; +3: #4890 extract phase jsonMode/quiet + pages-not-links totals; +1: wave review — extract phase passes quiet:true + jsonMode:false; +1: extract_facts pagesFailed detail; +8 fix wave 3: one cycle date resolved in runCycle and threaded to synthesize/patterns/drift (C-15); +1 #5632: cycle sync resumes an unfinished managed cursor with its stored options; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +1 at the merge; -41 #5809: anyAbortSignal peeled to abort-signals.ts; +5 #5809: extract_atoms phase gets a real abort signal; +4 #5809: cycle_already_running names the lock holder; +7 #5255: cycle sync runs a managed pull request as no-pull and reports upstream_refresh; +1 #4578: a lock steal during the last selected phase reports lock_stolen, not aborted; +3 #5536: keyless embed phase clean-skips when embedding_disabled is set; +5 F4b: planner-stats refresh after the freshness phases (logic in planner-stats.ts); +2 agent operator wave A1: PhaseResult sibling code/fix; +20 #5876: Life Chronicle phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block; v0.60.46.0 merge: re-summed to the merged line count (3224); +3 retrieval feedback retention in the purge phase (logic in feedback/store.ts pruneRetrievalFeedback); 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); P3 x foundations-2 composition: deltas re-summed at the merge; 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); +15 GBRA-49 temporal typed edges: edge_contradictions phase registration + dispatch block + extract-phase relationship sweep call (logic in cycle/edge-contradictions.ts, link-relationships.ts); P3 x P1 (v0.60.57.0) composition: deltas re-summed at the merge; +1 fix wave 9 lane P (#5151): extract_facts phase passes drain so the facts watermark drain runs (logic in facts/reconcile-watermark.ts); +10 #6188: fence_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in cycle/fence-repair.ts); +19 fix wave 12 #6242: source cycles with brain-wide phases also take the shared gbrain-cycle lease (acquireDbCycleLock lease set, needsMaintenanceLease, skipBrainWidePhases, all-skipped deriveStatus case; lease logic in cycle/lock-set.ts); +16 #6377: content_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK), the REPAIR_LANE_PHASES dispatch helper shared with fence_repair (logic in cycle/content-repair.ts); +10 C4 pinned questions: standing_questions phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in questions/phase.ts); +11 #6696 (A5, A4): runCycle becomes the background-origin wrapper over runCycleUnscoped and adds the capacity notices (logic in persistence/admission-origin.ts, capacity-snapshot.ts) plus the CycleReport.notices interface field; +2 #6297: CycleOpts.directPhases (operator-named phases skip the patterns cadence); +5 #6865: unindexed_files phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its line in the REPAIR_LANE_PHASES dispatch helper (logic in cycle/unindexed-files.ts); +2 fix wave 20 #6954: the extract phase warns on extract_write_lost and its stale drain retries the pages left due; +5 #7241: CycleOpts.onPhaseStart and its best-effort call in timePhase, so the autopilot-cycle job writes its current phase to minion_jobs.progress; +1 GBRA-133: the extract phase passes the stale-drain budget to its targeted pass when sync gave no slug list; +7 #7630 (GBRA-207): the sync phase refuses sync_anchor_relative when no source matches its directory and default's local_path is relative
src/core/cycle/synthesize.ts 3244 ratchet +3 #7805: each captured session synthesizes only into its own frozen source (routing import, the outer wrapper, the routing parameter; logic in cycle/synthesize-source-routing.ts); +26 System One v1 S7/S8: triage and grounding decide delegation (triage-decide.ts, grounding-decide.ts); merged waves; grown v0.46.26.0 megawave + master rolling lease via shared throttled renewer; grown wave-g (#4506): cycle summaries can stay out of the source repo; +18: expired-verdict sweep + DeepSeek judge thinking pin (#4069); +47: #4077 cooperative-abort threading; +110: eval fix wave F6 spend surfacing (withChatPhase wrap, triage/child token telemetry, zero-page counter); +32: F1b quote-verify call site + kill switch; +2: F1a/F3/F4a OUTPUT POLICY rules; +59: F2 verified-segment rescue (gate at report construction, rescue knobs, telemetry); +19: rubric v2 peak-scoring + docblock rescue branch; +50: #4337 summary link-sample cap + dream_created_cycle_date preserve-on-rerun; +5: #4348 resolveCycleDate wiring (local-day summary bucketing); +3 round-3 rebase true-up; +55: #4805 synth-v2 completion skip (findSynthV2Completion + prefix-parameterized loadSuccessfulSynthesisKeys); +10: #4886 mode-aware buildSynthesisPrompt (oneshot children drop tool-use/final-prose guidance); #5105: small legacy writer/refusal and shared revision CAS delegates; +4: merge of #5666 durable synthesis completions with the v0.59.18.0 verify-epoch wiring; +7 #5413/#5471 discovery exclusions (self-capture session ids, dream output directories); +6 fix wave 2 (v0.60.5.0) integration: W7 paid-loop breaker check before submission and released-key daily-cap count on top of the W1b discovery exclusions; +6 v0.60.5.0 master-merge true-up: the wave-2 growth (+13 W1b, +6 W7) re-summed over master v0.59.18.0 at 3222 lines with no slack; +76 fix wave 3: per-run USD gate + fail-closed daily cap (C-13), child first-write ownership for stamp/reverse-write (C-8, #5685), cycle-date prompt hint (C-15), opt-in attribution rule (#5425); fix wave 3 x fix wave 2 (v0.60.5.0) composition: +19 at the merge; +2 #5806 triageNow clock seam forwarded to triage (interface field + call-site option); +4 #5854: a postprocess or summary publish still pending after its wait is deferred (publish_pending warn, no cooldown stamp; helper in synthesize-postprocess.ts); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); lowered to 3200 by wave 9 lane W #5145: completion-key helpers (findLegacyCompletion, findSynthV2Completion, loadSuccessfulSynthesisKeys) peeled to synthesize-completion.ts with the new pre-triage partition; +13 P2 extraction.date_grounding: transcript observation date line instead of a run-date "Today's date" (config field + prompt arg); +1 fix wave 11 #6160: oneshot_task_prefixes on the child job data; +18 fix wave 12 #6069: unreliable-verdict backoff in runTriagePass (report code, pass backoff count, triage details entry; marker logic in triage-backoff.ts); +1 fix wave 12 #6242: SYNTH_PHASE_FAIL keeps an OperationError's suggestion and code (OperationError import); +3 wave 17 #6616: slug discovery and postprocessing share the canonical committed child outputs (loadCommittedSynthesisOutputs in synthesize-postprocess.ts; one call-site line, one parameter, one lookup line); lowered to 3224 by wave 17 #6642: the summary provenance section moved to summaryProvenanceLines in dream-provenance.ts; +6 GBRA-130 B-6 (#6860): surrogate-safe triage cuts (truncateUtf8) and the per-transcript verdict-cache write call site (logic in triage-backoff.ts VerdictCacheWrites); +2 GBRA-161 PR 4 (#7065): the synthesize phase activates each synthesized transcript's outputs as its generation (activateSynthesisGenerations import and call site; logic in cycle/synthesize-generations.ts); +8 GBRA-161 PR 5 (S-2): the dream summary publishes as a single-item dream_summary generation bound to its pre-run revision (publishSinglePage with a generation_fenced skip); +1 GBRA-60: a managed synthesis child carries dream_output (verify and stamp before put_page; cycle/synthesize-child-verify.ts)
src/core/cycle/extract-atoms.ts 1658 ratchet crossed the 1500 unlisted cap in the atoms/extraction fix wave: +21 embed-aware cost gate (#4878), +65 atoms-array anchor scan (#4913); peel candidate: the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) into extract-atoms-parse.ts; +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) is peeled into extract-atoms-parse.ts (-193, fix wave 10 lane B); +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); +17 #5031: same-page case-insensitive atom adoption fallback (one page-atom scan per source page per run); +12 P2 atomsPrompt helper: extraction.date_grounding source observation date + rule in the atoms prompt; +4 GBRA-75: octet_length-bounded body-length predicate (minCompiledTruthChars) for discovery and the backlog count; GBRA-130 A-42 (#6804): dot-led atom stems and the legacy-slug adoption guard
diff --git a/src/commands/storage.ts b/src/commands/storage.ts
index 166db2aed7..ad9cb87305 100644
--- a/src/commands/storage.ts
+++ b/src/commands/storage.ts
@@ -1,7 +1,7 @@
import { existsSync } from 'fs';
import { dirname, join, relative, resolve, sep } from 'path';
import type { BrainEngine } from '../core/engine.ts';
-import { loadStorageConfig, validateStorageConfig, getStorageTier, isDbOnly } from '../core/storage-config.ts';
+import { hasUnresolvedDbOnlyDeclaration, loadStorageConfig, validateStorageConfig, getStorageTier, isDbOnly } from '../core/storage-config.ts';
import type { StorageConfig, StorageTier } from '../core/storage-config.ts';
import { walkBrainRepo, type DiskFileEntry } from '../core/disk-walk.ts';
import { getDefaultSourcePath, isResolverUserError, resolveSourceForRepoPath } from '../core/source-resolver.ts';
@@ -219,6 +219,7 @@ export async function getStorageStatus(
): Promise<StorageStatusResult> {
const config = repoPath ? loadStorageConfig(repoPath) : null;
const warnings = config ? validateStorageConfig(config) : [];
+ if (repoPath && hasUnresolvedDbOnlyDeclaration(repoPath, config)) warnings.push('gbrain.yml declares storage.db_only but no directory resolved from it (a flow list split over lines, or the key outside storage:); list the directories in block style or as one line, db_only: [a/, b/]. Until then sync keeps every page whose file is gone (#8007).');
const pagesByTier: PageCountsByTier = { db_tracked: 0, db_only: 0, unspecified: 0 };
const diskUsageByTier: DiskUsageByTier = { db_tracked: 0, db_only: 0, unspecified: 0 };
diff --git a/src/commands/sync-diagnostics.ts b/src/commands/sync-diagnostics.ts
index fd25ccbaef..d1b8cbdf6c 100644
--- a/src/commands/sync-diagnostics.ts
+++ b/src/commands/sync-diagnostics.ts
@@ -71,6 +71,7 @@ export function printManagedSyncNotes(result: SyncResult, write: (line: string)
for (const refusal of result.fileRefusals ?? []) {
write(` Refused ${refusal.code}: ${refusal.message} ${refusal.suggestion} (${refusal.docs})`);
}
+ if (result.storage_invalid_kept) write(` ${result.storage_invalid_kept} page(s) whose file is gone were kept, not deleted: gbrain.yml's storage section is invalid or declares db_only in a form that resolved no directory, so sync cannot tell which are database-only. Run \`gbrain storage status\`, fix gbrain.yml, then sync again.`);
if (result.working_tree_kept) write(` ${result.working_tree_kept} page(s) whose file is in the working tree but not in Git were kept (imported from the working tree, or import provenance unknown); delete the file, or commit it, to change that.`);
const skips = result.legacySkips;
if (skips?.contextualMode) write(` ${skips.contextualMode} legacy file(s) skipped because they parse to the same page but have no contextual retrieval mode, which a skipped import cannot stamp; to stamp it: gbrain repair contextual-mode`);
diff --git a/src/commands/sync.ts b/src/commands/sync.ts
index e584dcbe01..e59571475b 100644
--- a/src/commands/sync.ts
+++ b/src/commands/sync.ts
@@ -76,6 +76,7 @@ export interface SyncResult {
tail?: Omit<import('../core/persistence/sync-tail.ts').SyncTailReport, 'links'>;
/** #5459 (P4.5): live pages a `--full` tree walk kept because their file is in the working tree and their import provenance is the working tree or unknown. */
working_tree_kept?: number;
+ /** #8007: pages whose file is gone kept because gbrain.yml's storage section is invalid. */ storage_invalid_kept?: number;
/**
* Aggregated alias/undeclared explicit-type warnings (schema.type_warnings,
* default on) — one entry per distinct non-canonical type this run.
diff --git a/src/core/persistence/source-storage.ts b/src/core/persistence/source-storage.ts
index c9fc87571c..d8112125f6 100644
--- a/src/core/persistence/source-storage.ts
+++ b/src/core/persistence/source-storage.ts
@@ -46,6 +46,19 @@ function load(root: string): StorageConfig | null {
* logged once per version; `refuse` then throws `storage_error`, while
* `not_db_only` answers false so the caller keeps its non-db_only behavior.
*/
+/**
+ * #8007: whether a page slug is database-only at this source root, or `unknown` when gbrain.yml cannot be read, its tiers
+ * overlap or it declares db_only in syntax resolving nothing. A caller deciding whether a missing file deletes its page
+ * holds the page on `unknown` instead of guessing.
+ */
+export function sourceDbOnlyVerdict(root: string, slug: string): boolean | 'unknown' {
+ try {
+ return isDeclaredDbOnlySlug(slug, load(root));
+ } catch {
+ return 'unknown';
+ }
+}
+
/** #7423: the declared `storage.db_only` dir this page slug falls under at this source root, or null; an invalid gbrain.yml refuses (`storage_error`). */
export function sourceDbOnlyDir(root: string, slug: string): string | null {
const config = load(root);
diff --git a/src/core/persistence/sync-classify.ts b/src/core/persistence/sync-classify.ts
index 9e06cd3b2d..e08cf29392 100644
--- a/src/core/persistence/sync-classify.ts
+++ b/src/core/persistence/sync-classify.ts
@@ -83,8 +83,11 @@ export interface SyncPathState {
fullObligation?: boolean;
/** Under a `storage.keep_db` prefix, on a source that is not mirror-read-only. */
keepDb: boolean;
- /** #7324: a page under a declared `storage.db_only` dir: its database row is canonical, so an absent file is no deletion evidence. */
- dbOnly?: boolean;
+ /**
+ * #7324: a page under a declared `storage.db_only` dir: its database row is canonical, so an absent file is no deletion
+ * evidence. #8007: `unknown` while gbrain.yml's storage section is invalid; an absent file is then held, never deleted.
+ */
+ dbOnly?: boolean | 'unknown';
/** Open work on the page or path (a queued/running/recovering request, a pending Git effect). */
openWork: boolean;
/**
@@ -110,7 +113,7 @@ export interface SyncClassified {
side?: 'both' | 'unknown' | 'deleted';
state?: 'tombstone_edited' | 'purged_differs' | 'file_gone';
/** #7130: a `delete_drift` whose file Git never committed (`never_committed`), `.gitignore` excludes, or whose history read did not answer (`origin_unknown`). */
- uncommitted?: 'never_committed' | 'ignored' | 'origin_unknown';
+ uncommitted?: 'never_committed' | 'ignored' | 'origin_unknown' | 'storage_invalid';
/** `db_ahead` `db_only` under keep_db: also gets the DB-wins write. */
keepDbWrite?: true;
/** `db_ahead` `db_only`/`unproven` under an open full obligation: excluded and counted (`full_skipped_db_ahead`). */
@@ -175,7 +178,8 @@ function afterHold(s: SyncPathState): Matched {
if (live && s.level === null) {
if (s.mode === 'committed' && s.present) return { cls: 'kept', settled: true, action: 'none' };
if (!s.present) {
- if (s.dbOnly) return { cls: 'kept', settled: true, action: 'none' };
+ if (s.dbOnly === true) return { cls: 'kept', settled: true, action: 'none' };
+ if (s.dbOnly === 'unknown') return { cls: 'delete_drift', settled: false, action: 'hold', side: 'deleted', state: 'file_gone', uncommitted: 'storage_invalid' };
if (s.originEvidence && s.originEvidence !== 'committed') return { cls: 'delete_drift', settled: false, action: 'hold', side: 'deleted', state: 'file_gone',
uncommitted: s.originEvidence === 'unknown' ? 'origin_unknown' : s.originEvidence };
return proof ? { cls: 'delete', settled: false, action: 'delete' } : { cls: 'delete_drift', settled: false, action: 'hold', side: 'deleted', state: 'file_gone' };
@@ -241,7 +245,7 @@ function gitVerdict(state: SyncPathState): Matched {
case 'drift': case 'keep_db': case 'deleted_edited': case 'legacy_unverified': return wins({ cls: 'import', settled: false, action: 'import' });
case 'db_ahead': return rest.sub === 'pending' ? rest : wins({ cls: 'import', settled: false, action: 'import' });
case 'delete': case 'delete_drift':
- if (state.originEvidence === undefined) return rest;
+ if (state.originEvidence === undefined || rest.uncommitted === 'storage_invalid') return rest;
if (state.originEvidence !== 'committed') return wins({ cls: 'kept', settled: true, action: 'none' });
return rest.cls === 'delete' ? rest : wins({ cls: 'delete', settled: false, action: 'delete' });
default: return rest;
diff --git a/src/core/persistence/sync-discovery.ts b/src/core/persistence/sync-discovery.ts
index 68625ed2e7..972f72d485 100644
--- a/src/core/persistence/sync-discovery.ts
+++ b/src/core/persistence/sync-discovery.ts
@@ -36,7 +36,7 @@ import { holdBasisMoved, holdRescreenDue, readGitHoldRetryPaths, readGitSourceHo
import { readBlobContents, readTreeBlobs, SYNC_READ_BOUND } from './sync-blobs.ts';
import { gitChildEnv } from '../git-env.ts';
import { readOriginEvidence } from './sync-origin-evidence.ts';
-import { isSourceDbOnlySlug } from './source-storage.ts';
+import { sourceDbOnlyVerdict } from './source-storage.ts';
import { movedDriftHolds, prescreenSyncEntries, type SyncDriftSide } from './sync-prescreen.ts';
import { sliceEntries, type SyncSliceCursor, type SyncSliceState } from './sync-slices.ts';
@@ -85,6 +85,8 @@ export interface SyncDiscovery { binding: WorktreeBinding; root: string; gitRoot
workingTreeKept?: string[];
/** #7130: live pages whose file is absent but Git never committed it (or `.gitignore` excludes it): kept, never deleted. */
uncommittedKept?: string[];
+ /** #8007: live pages whose file is absent while gbrain.yml's storage section is invalid, so whether each is database-only is unknown: kept, never deleted. */
+ storageInvalidKept?: string[];
/** #6349 (P4.4): this discovery is one path slice of its target; the checkpoint records it instead of advancing `last_commit` until `done`. */
slice?: SyncSliceState;
/** #6729: inexact Git renames split into a delete plus an add (origin form); recorded after the run commits. */
@@ -373,6 +375,20 @@ async function dropUncommittedDeletes(gitRoot: string, tips: Array<string | null
return kept;
}
+/**
+ * #7324: a delete under a declared storage.db_only dir is retired (its database row is canonical). #8007: while gbrain.yml's
+ * storage section is invalid, whether a path is database-only is unknown, so its delete is retired too and named.
+ */
+function retireDbOnlyDeletes(root: string, entries: readonly SyncEntry[], retired: Set<SyncEntry>): { storageInvalidKept?: string[] } {
+ const storageInvalidKept: string[] = [];
+ for (const entry of entries) {
+ const dbOnly = entry.action === 'delete' && entry.slug ? sourceDbOnlyVerdict(root, entry.slug) : false;
+ if (dbOnly) retired.add(entry);
+ if (dbOnly === 'unknown') storageInvalidKept.push(entry.path);
+ }
+ return storageInvalidKept.length ? { storageInvalidKept } : {};
+}
+
export async function discoverManagedSync(engine: BrainEngine, opts: SyncOpts, context?: ManagedSyncContext,
stage?: { target: string; head: string; slice?: Pick<SyncSliceCursor, 'phase' | 'after'> }, walk = false): Promise<SyncDiscovery> {
const { binding, root, gitRoot, sourceId, incarnation, source } = context ?? await resolveManagedSyncContext(engine, opts);
@@ -659,7 +675,7 @@ export async function discoverManagedSync(engine: BrainEngine, opts: SyncOpts, c
else winner.renameHeld = held;
}
}
- if (!company) for (const entry of selected) if (entry.action === 'delete' && entry.slug && isSourceDbOnlySlug(root, entry.slug, 'not_db_only')) retired.add(entry);
+ if (!company) Object.assign(discovered, retireDbOnlyDeletes(root, selected, retired));
if (retired.size) discovered.entries = selected.filter(entry => !retired.has(entry));
if (collisions.length) discovered.slugCollisions = collisions;
// Identities and origin collisions were resolved over the whole target above; only the frozen manifest is cut.
diff --git a/src/core/persistence/sync-holds.ts b/src/core/persistence/sync-holds.ts
index d181aed9a2..8bd0343df4 100644
--- a/src/core/persistence/sync-holds.ts
+++ b/src/core/persistence/sync-holds.ts
@@ -117,7 +117,7 @@ export interface PreparationStallMeta {
}
export type GitHoldReason = InvalidFrontmatterReason | 'rename_source_changed' | 'explicit_id_conflict' | FenceReason | OriginConflictReason | PageIdentityHoldReason
| 'symlink_in_repo' | 'symlink_outside_root' | 'symlink_dangling' | 'symlink_loop' | 'permission_denied' | 'not_regular_file' | 'skip_worktree' | 'deleted_locally'
- | 'not_utf8' | 'control_character' | 'never_committed' | 'ignored' | 'origin_unknown';
+ | 'not_utf8' | 'control_character' | 'never_committed' | 'ignored' | 'origin_unknown' | 'storage_invalid';
/** GBRA-130: why a `page_identity_changed` hold's path has no page sync may write (A-17, B-3, B-4). */
export type PageIdentityHoldReason = 'several_tombstones' | 'spelling' | 'identity_differs';
/** #6712 PR 4b: the sides a `file_drift` Decision can be about, and the states of a deleted side (design v3.2 §4). */
@@ -613,6 +613,8 @@ export function holdRescreenDue(record: Pick<GitHoldRecord, 'code' | 'meta'>, re
// GBRA-133: an unknown-side drift held before #7340's baseline rule (a Git edit to a page with no import record) imports
// now when the page still holds its file at the baseline commit, so it is screened once more.
|| (record.code === 'file_drift' && fileDriftSide(record.meta) === 'unknown' && (record.meta.drift_version ?? 0) < FILE_DRIFT_VERSION)
+ // #8007: a deletion held while gbrain.yml's storage section was invalid is decided again by every sync, so a fixed file settles it.
+ || (record.code === 'file_drift' && record.meta.reason === 'storage_invalid')
|| (outOfAttemptsHold(record.code) && record.meta.stall?.gbrain_version !== VERSION)
// GBRA-130 A-6: an owner fault is the writer's, not the file's: an upgrade re-tests the path.
|| (record.code === 'page_write_failed' && record.meta.write_failure?.gbrain_version !== VERSION)
@@ -716,6 +718,13 @@ function deletedDriftFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'met
const revision = record.meta.page_revision ? ` (revision ${record.meta.page_revision})` : '';
const preview = (prefer: 'database' | 'file') => heldDriftPreviewArgv(source, 'deleted', state, prefer, record.path);
const verify = { argv: ['gbrain', 'sources', 'status', source, '--json'] };
+ if (state === 'file_gone' && record.meta.reason === 'storage_invalid') {
+ return { argv: ['gbrain', 'storage', 'status'], consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/storage-tiering.md',
+ why: `${record.path} is gone, but gbrain.yml's storage section is invalid or declares db_only in a form that resolved no directory, so sync cannot tell whether page ${slug} is database-only and kept it instead of deleting it (#8007). This read-only check names the problem; fix gbrain.yml at the source root (list db_only directories in block style or as a one-line [a/, b/] list), commit it, then sync again.`,
+ then: { argv: ['gbrain', 'sync', '--source', source, '--no-pull'], consent: [], actor: 'agent', requires_exclusive: false, verify,
+ why: `Decides ${record.path} again under the fixed gbrain.yml: a database-only page is kept, any other deletion follows its usual proof.` },
+ verify };
+ }
const uncommitted = state === 'file_gone' && (record.meta.reason === 'never_committed' || record.meta.reason === 'ignored');
const what = state === 'file_gone' && record.meta.reason === 'origin_unknown' ? `${record.path} is gone, and Git could not show whether a commit held it (its history read did not finish), so sync kept page ${slug} instead of deleting it (#7130). The --held preview reads the whole history and names the file's provenance. `
: uncommitted ? `${record.path} is gone, but Git never committed it${record.meta.reason === 'ignored' ? ' (.gitignore excludes it)' : ''}, so its absence is no evidence of a deletion and sync kept page ${slug} instead of deleting it (#7130). Put the file back and commit it (un-ignore it first) to keep the page file-backed. `
diff --git a/src/core/persistence/sync-pass-apply.ts b/src/core/persistence/sync-pass-apply.ts
index 66d4c09adc..3c73efee51 100644
--- a/src/core/persistence/sync-pass-apply.ts
+++ b/src/core/persistence/sync-pass-apply.ts
@@ -309,7 +309,10 @@ async function writePassHolds(engine: BrainEngine, level: ShadowLevel, input: Pa
continue;
}
if (r.cls === 'delete_drift' && heldRenameOrigins.has(syncOriginPath(passEntryPaths(level, entry.path).sourcePath))) continue;
- if (r.cls === 'delete_drift' && r.uncommitted) {
+ if (r.cls === 'delete_drift' && r.uncommitted === 'storage_invalid') {
+ pending.push({ entry, code: 'file_drift', slug: entry.slug, kind: 'uncommitted', meta: { side: 'deleted', state: 'file_gone', reason: 'storage_invalid', ...(entry.revision ? { page_revision: entry.revision } : {}) },
+ message: `${path}: the file is gone, but gbrain.yml's storage section is invalid or declares db_only in a form that resolved no directory, so sync cannot tell whether page ${entry.slug} is database-only; it held the path and kept the page instead of deleting it (#8007); the rest of the source synced.` });
+ } else if (r.cls === 'delete_drift' && r.uncommitted) {
const why = r.uncommitted === 'ignored' ? '.gitignore excludes it, so Git never kept it' : r.uncommitted === 'origin_unknown' ? 'Git could not show whether a commit held it (its history read did not finish)' : 'no commit in its history holds it';
pending.push({ entry, code: 'file_drift', slug: entry.slug, kind: 'uncommitted', meta: { side: 'deleted', state: 'file_gone', reason: r.uncommitted, ...(entry.revision ? { page_revision: entry.revision } : {}) },
message: `${path}: the file is gone, but ${why}, so its absence is no evidence that page ${entry.slug} was deleted; sync held it and kept the page instead of deleting it (#7130); the rest of the source synced.` });
diff --git a/src/core/persistence/sync-pass.ts b/src/core/persistence/sync-pass.ts
index 157eb897e3..4397eac77e 100644
--- a/src/core/persistence/sync-pass.ts
+++ b/src/core/persistence/sync-pass.ts
@@ -24,7 +24,7 @@ import { isReservedSkillBundlePath } from '../skill-reserved-paths.ts';
import { resolveSlugRootMode } from '../sync-anchor.ts';
import { resolveSlugForPath } from '../sync.ts';
import { join } from 'node:path';
-import { isDeclaredDbOnlySlug, isKeepDbSlug, loadStorageConfig } from '../storage-config.ts';
+import { hasUnresolvedDbOnlyDeclaration, isDeclaredDbOnlySlug, isKeepDbSlug, loadStorageConfig } from '../storage-config.ts';
import { loadActivePackForEngine } from '../schema-pack/engine-resolution.ts';
import { currentCompanyBrainSync, getCompanyBrainProfile } from '../company-brain/profile.ts';
import { sha256 } from './digest.ts';
@@ -639,7 +639,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: st
const settings = await readLevelSettings(engine, context, scope, input);
timings.setup = t();
if (!settings) return null;
- const { slugMode, exclude, pack, mirror, strategy, includeHidden, keepDbPrefixes, storage, workingConfig, metafilePages, strict } = settings;
+ const { slugMode, exclude, pack, mirror, strategy, includeHidden, keepDbPrefixes, storage, dbOnlyUnknown, workingConfig, metafilePages, strict } = settings;
const sourceRelative = (gitPath: string) => (scope ? gitPath.slice(scope.length + 1) : gitPath);
let metafilePage: (gitPath: string) => boolean = () => false;
// Eligibility of a path outside the cached tree (every path of the tree is eligible); origins validate per call: a
@@ -810,7 +810,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: st
episodeUnsatisfied: input.pass ? !!input.pass.episodeFullId && live && (record ? pages.fullEpisode[at!] : undefined) !== input.pass.episodeFullId
: input.full === true && live,
derivedCurrent: live && pages.derivedCurrent(at!), fullObligation: input.pass ? !!input.pass.episodeFullId : input.full === true,
- keepDb: !mirror && keepDbPrefixes.length > 0 && row && isKeepDbSlug(pages.slug[at!]!, keepDbPrefixes), dbOnly: row && isDeclaredDbOnlySlug(pages.slug[at!]!, storage),
+ keepDb: !mirror && keepDbPrefixes.length > 0 && row && isKeepDbSlug(pages.slug[at!]!, keepDbPrefixes), dbOnly: row && (dbOnlyUnknown ? 'unknown' : isDeclaredDbOnlySlug(pages.slug[at!]!, storage)),
openWork: row && open.pageIds.has(Number(pages.id[at!])) || open.origins.has(origin),
};
return { state, ...(row ? { at } : {}), ...(legacyPageless ? { legacyPageless } : {}) };
@@ -829,7 +829,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: st
}).then(found => { timings.legacy = t(); return found; })
: new Map<string, string>();
- const classifiedKey = JSON.stringify([treeKey, mode, strict, input.pass?.episodeFullId ?? null, input.full === true, mirror, keepDbPrefixes, storage?.db_only ?? [], pack?.identity ?? null, baselineCommit, !!input.pass?.bootstrap, [...colonRefused].sort()]);
+ const classifiedKey = JSON.stringify([treeKey, mode, strict, input.pass?.episodeFullId ?? null, input.full === true, mirror, keepDbPrefixes, storage?.db_only ?? [], dbOnlyUnknown, pack?.identity ?? null, baselineCommit, !!input.pass?.bootstrap, [...colonRefused].sort()]);
const reuse = snapshot.changed && priorClassified?.key === classifiedKey ? reusableMatched(priorClassified.paths, matchedUnchanged({ pages, tree, byOrigin, claimants, holds: holdByGitPath, markers, open,
changed: confirmChangedPaths(pages, snapshot.changed, (sourcePath, slug) => gitPathOf(pageOrigin(sourcePath, slug))), indexDelta, spelled: path => twins.has(path) || legacySpelling.has(path), originOf })) : null;
// A round that reuses nothing lets the previous round's paths go before it builds its own: one level held, not two.
@@ -917,7 +917,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: st
}
return { sourceId, incarnation, root, gitRoot, scope, slugMode, mode, strict, head, baselineCommit, full: input.full === true, kind: kind.kind, kindBlockers: kind.blockers,
heldRenames: holds.flatMap(hold => hold.meta.rename_from ? [{ path: toGit(hold.path), rename: hold.meta.rename_from }] : []),
- releasableHolds: holds.filter(hold => !domain.has(toGit(hold.path))).map(hold => hold.path),
+ releasableHolds: holds.filter(hold => !domain.has(toGit(hold.path)) || (!dbOnlyUnknown && hold.code === 'file_drift' && hold.meta.reason === 'storage_invalid')).map(hold => hold.path),
deletionCandidates: deletionCandidates(holds, path => domain.has(path) && !tree.has(path)
&& !(mode === 'working' && (untracked.has(path) || index.has(path))) && !readPresent(root, path, scope), toGit),
paths, reconcilable, ...(colon.refusals.size ? { fileRefusals: [...colon.refusals.values()] } : {}), timings_ms: timings, statements: statements + attestationStatements + 3,
@@ -963,11 +963,12 @@ async function readLevelSettings(engine: BrainEngine, context: ManagedSyncContex
]);
const settled = <T>(read: PromiseSettledResult<T>): T => { if (read.status === 'rejected') throw read.reason; return read.value; };
if (settled(company)) return null;
- const storage = (() => { try { return loadStorageConfig(root); } catch { return null; } })();
+ let storageInvalid = false;
+ const storage = (() => { try { return loadStorageConfig(root); } catch { storageInvalid = true; return null; } })();
return { slugMode: settled(slugModeRead), exclude: settled(excludeRead), pack: settled(packRead), mirror: settled(mirrorRead), workingConfig: settled(workingRead),
metafilePages: settled(metafilePagesRead), strict: context.source.config?.canonical === 'git', strategy: input.pass?.strategy ?? (context.source.config?.strategy as string | undefined) ?? 'markdown',
includeHidden: [...new Set([...input.pass?.includeHidden ?? [], ...(settled(includeHiddenRead) ?? '').split(/[\n,]/).map(v => v.trim()).filter(Boolean)].map(v => v.endsWith('/') ? `${v}**` : v))],
- keepDbPrefixes: storage?.keep_db ?? [], storage };
+ keepDbPrefixes: storage?.keep_db ?? [], storage, dbOnlyUnknown: storageInvalid || hasUnresolvedDbOnlyDeclaration(root, storage) };
}
/** A failed mark, proof or delta read: the round reads again without them (it aborted a Postgres snapshot). */
diff --git a/src/core/persistence/sync-run.ts b/src/core/persistence/sync-run.ts
index 478def5baa..94b59b0e42 100644
--- a/src/core/persistence/sync-run.ts
+++ b/src/core/persistence/sync-run.ts
@@ -304,6 +304,7 @@ function result(cursor: Cursor | CursorHeader, status: SyncResult['status'], rea
deleted: cursor.counts.deleted, renamed: cursor.counts.renamed ?? 0, chunksCreated: cursor.counts.chunks, embedded: 0, pagesAffected: [],
...(cursor.slugCollisions?.length ? { slugCollisions: cursor.slugCollisions } : {}), ...(cursor.identityDowngrades?.length ? { identityDowngrades: cursor.identityDowngrades } : {}),
...(cursor.fileRefusals?.length ? { fileRefusals: cursor.fileRefusals } : {}), ...(cursor.workingTreeKept?.length ? { working_tree_kept: cursor.workingTreeKept.length } : {}),
+ ...(cursor.storageInvalidKept?.length ? { storage_invalid_kept: cursor.storageInvalidKept.length } : {}),
...(cursor.stagedHead && !cursor.authority.writer.remote ? { staged: { target: cursor.target, head: cursor.stagedHead, ...(cursor.slice && !cursor.slice.done ? { slice: { phase: cursor.slice.phase, after: cursor.slice.after } } : {}) } } : {}),
waived: { imports: cursor.counts.waived?.imports ?? 0, deletes: cursor.counts.waived?.deletes ?? 0 }, prescreened: cursor.counts.prescreened ?? 0,
filesImported: cursor.index, bankedFiles: cursor.index,
diff --git a/src/core/storage-config.ts b/src/core/storage-config.ts
index f71f745fd7..86d2fa56eb 100644
--- a/src/core/storage-config.ts
+++ b/src/core/storage-config.ts
@@ -105,10 +105,11 @@ function parseStorageYaml(content: string, unknown: string[] = []): RawStorage |
const key = indented.slice(0, colon).trim();
if (STORAGE_KEYS.has(key)) {
currentList = key as keyof RawStorage;
- // Inline empty list: `db_only: []`.
- const remainder = indented.slice(colon + 1).trim();
- if (remainder === '[]' && !raw[currentList]) {
- raw[currentList] = [];
+ // Inline flow list on one line: `db_only: []` or `db_only: [dir/, 'other/']`.
+ const flow = /^\[(.*)\]$/.exec(indented.slice(colon + 1).trim());
+ if (flow) {
+ const values = flow[1]!.split(',').map((value) => value.trim().replace(/^["']|["']$/g, '')).filter(Boolean);
+ raw[currentList] = [...(raw[currentList] ?? []), ...values];
}
continue;
}
@@ -395,11 +396,11 @@ export function isKeepDbSlug(slug: string, keepDb: readonly string[]): boolean {
/**
* True when gbrain.yml has a `db_only` (or deprecated `supabase_only`) key
* line but no db_only directory resolved from it. `loadStorageConfig` warns
- * and resolves nothing for syntax the narrow parser does not handle (e.g.
- * flow-style `db_only: [dir/]`), so a caller that must not guess a tier
- * refuses instead. Only YAML key lines count, not comments or prose that
- * mention the word. Known false positive: an intentionally empty
- * `db_only: []` also counts, since it resolves to the same empty config.
+ * and resolves nothing for syntax the narrow parser does not handle (e.g. a
+ * flow list split over several lines, or a key outside the storage section),
+ * so a caller that must not guess a tier refuses instead. Only YAML key lines
+ * count, not comments or prose that mention the word; an explicit empty
+ * `db_only: []` declares no directory and does not count.
*/
export function hasUnresolvedDbOnlyDeclaration(repoPath: string, config: StorageConfig | null): boolean {
if (config && config.db_only.length > 0) return false;
@@ -409,7 +410,8 @@ export function hasUnresolvedDbOnlyDeclaration(repoPath: string, config: Storage
return yamlContent.split('\n').some((line) => {
const trimmed = line.trim();
if (trimmed.startsWith('#')) return false;
- return /^(db_only|supabase_only)\s*:/.test(trimmed);
+ const key = /^(db_only|supabase_only)\s*:(.*)$/.exec(trimmed.replace(/\s+#.*$/, ''));
+ return key !== null && key[2]!.replace(/\s+/g, '') !== '[]';
});
}
diff --git a/src/core/sync-git.ts b/src/core/sync-git.ts
index 85b432d45b..1575a5455e 100644
--- a/src/core/sync-git.ts
+++ b/src/core/sync-git.ts
@@ -529,14 +529,13 @@ export function createSyncBaselineCommit(repoPath: string): void {
const dbOnlyDirs = storageConfig?.db_only ?? [];
// Sniff-test fail-closed (round 6, P2): `loadStorageConfig` warns-and-
// returns an EMPTY config for syntactically-valid-but-unsupported YAML
- // (e.g. flow-style `db_only: [dir/]`; the narrow custom parser only
- // handles block-style lists), which would silently resolve zero
+ // (e.g. a flow list split over lines; the narrow custom parser handles
+ // block lists and one-line flow lists), which would silently resolve zero
// exclusions from a file that clearly intended some. If gbrain.yml has a
// db_only (or deprecated `supabase_only`) key line but nothing resolved
// from it, refuse rather than guess; `hasUnresolvedDbOnlyDeclaration`
- // owns the key-line rules. Accepted trade-off: an intentionally empty
- // `db_only: []` also refuses (same empty config); silently committing
- // db_only content into permanent git history is high-cost and hard to undo.
+ // owns the key-line rules (an explicit empty `db_only: []` declares none, #8007);
+ // silently committing db_only content into permanent git history is high-cost and hard to undo.
if (hasUnresolvedDbOnlyDeclaration(repoPath, storageConfig)) {
const yamlPath = join(repoPath, 'gbrain.yml');
throw new Error(
diff --git a/test/git-gap-census.test.ts b/test/git-gap-census.test.ts
index a2847d0697..99f0d916f2 100644
--- a/test/git-gap-census.test.ts
+++ b/test/git-gap-census.test.ts
@@ -355,8 +355,10 @@ for (const backend of testBackends()) describe(`git-gap census (git-canonical G1
const root = tempDir('git-gap-policy-');
git(root, 'init', '-q');
await useCheckout(root);
- put(root, 'gbrain.yml', 'storage:\n db_only: [notes/]\n');
+ put(root, 'gbrain.yml', 'storage:\n db_only: [notes/,\n archive/]\n');
await expect(censusGitGap(engine, 'default', { sample: 0 })).rejects.toMatchObject({ code: 'storage_error' });
+ put(root, 'gbrain.yml', 'storage:\n db_only: [notes/]\n');
+ expect((await censusGitGap(engine, 'default', { sample: 0 })).declared.db_only_prefix).toBe(1);
put(root, 'gbrain.yml', 'storage:\n db_only:\n - Notes/\n');
expect((await censusGitGap(engine, 'default', { sample: 0 })).declared.db_only_prefix).toBe(1);
rmSync(join(root, 'gbrain.yml'));
diff --git a/test/grandfather-managed.test.ts b/test/grandfather-managed.test.ts
index 4a59123fb2..675deb6e26 100644
--- a/test/grandfather-managed.test.ts
+++ b/test/grandfather-managed.test.ts
@@ -188,7 +188,7 @@ test('a derive-phase page whose recorded file went missing still refuses', () =>
const invalidStorage: Array<[string, string]> = [
['overlapping tiers', 'storage:\n db_tracked:\n - conversations/\n db_only:\n - conversations/\n'],
- ['flow-style db_only', 'storage:\n db_only: [conversations/]\n'],
+ ['a multi-line flow-style db_only', 'storage:\n db_only: [conversations/,\n archive/]\n'],
];
for (const [name, gbrainYml] of invalidStorage) {
test(`${name} in gbrain.yml only turn the missing-file refusal into storage_error`, () => fixture(async ({ engine, ctx, root }) => {
diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txt
index 5c1c7ddd45..2341121142 100644
--- a/test/postgres-unit-arms.txt
+++ b/test/postgres-unit-arms.txt
@@ -516,6 +516,7 @@ test/sync-slug-root-mode-6694.test.ts
test/sync-staged-catchup.test.ts
test/sync-status-failure-detail-6513.test.ts
test/sync-status-truthful-7243.test.ts
+test/sync-storage-invalid-8007.test.ts
test/sync-tree-reuse-gbra133.test.ts
test/sync-unblock-repair-error-gbra130.test.ts
test/sync-uncommitted-origin-7130.test.ts
diff --git a/test/storage-config.test.ts b/test/storage-config.test.ts
index 8f5a2c5032..47d245073b 100644
--- a/test/storage-config.test.ts
+++ b/test/storage-config.test.ts
@@ -163,10 +163,11 @@ describe('loadStorageConfig — real-disk loader', () => {
}
test.each([
- ['flow-style db_only', 'storage:\n db_only: [conversations/]\n', true],
- ['flow-style deprecated supabase_only', 'storage:\n supabase_only: [conversations/]\n', true],
+ ['flow-style db_only (#8007: parsed)', 'storage:\n db_only: [conversations/]\n', false],
+ ['flow-style deprecated supabase_only (#8007: parsed)', 'storage:\n supabase_only: [conversations/]\n', false],
+ ['a flow list split over lines', 'storage:\n db_only: [conversations/,\n media/]\n', true],
['db_only outside the storage section', 'db_only:\n - conversations/\n', true],
- ['literal empty db_only (accepted false positive)', 'storage:\n db_only: []\n', true],
+ ['literal empty db_only (#8007: declares none)', 'storage:\n db_only: []\n', false],
['block-style db_only', 'storage:\n db_only:\n - conversations/\n', false],
['a comment that mentions db_only', '# db_only handling later\nstorage:\n db_tracked:\n - people/\n', false],
['no gbrain.yml', null, false],
@@ -179,6 +180,15 @@ describe('loadStorageConfig — real-disk loader', () => {
}
});
+test('#8007: a one-line flow list resolves its directories, quoted or not', () => {
+ try {
+ writeFileSync(join(tmp, 'gbrain.yml'), "storage:\n db_only: [conversations/, 'media/x/', \"notes/\"]\n db_tracked: [people/]\n");
+ expect(loadStorageConfig(tmp)).toMatchObject({ db_only: ['conversations/', 'media/x/', 'notes/'], db_tracked: ['people/'] });
+ } finally {
+ cleanup();
+ }
+ });
+
test('returns null when repoPath is missing', () => {
try {
expect(loadStorageConfig(undefined)).toBeNull();
diff --git a/test/storage-pglite.test.ts b/test/storage-pglite.test.ts
index 8df01e5dcf..2ab1f0c2e4 100644
--- a/test/storage-pglite.test.ts
+++ b/test/storage-pglite.test.ts
@@ -103,6 +103,21 @@ describe('Storage tiering on PGLite — full lifecycle (D8 + D4)', () => {
}
});
+ test('#8007: getStorageStatus names a db_only declaration that resolved no directory; a one-line flow list resolves', async () => {
+ try {
+ writeFileSync(join(tmp, 'gbrain.yml'), 'storage:\n db_only: [media/x/,\n media/y/]\n');
+ const unresolved = await getStorageStatus(engine, tmp);
+ expect(unresolved.warnings.some(w => w.includes('storage.db_only but no directory resolved'))).toBe(true);
+ expect(formatStorageStatusHuman(unresolved)).toContain('db_only: [a/, b/]');
+ writeFileSync(join(tmp, 'gbrain.yml'), 'storage:\n db_only: [media/x/, "media/y/"]\n');
+ const resolved = await getStorageStatus(engine, tmp);
+ expect(resolved.config!.db_only).toEqual(['media/x/', 'media/y/']);
+ expect(resolved.warnings.some(w => w.includes('no directory resolved'))).toBe(false);
+ } finally {
+ cleanup();
+ }
+ });
+
test('getStorageStatus scopes page checks to the source that owns the repo path (#4763)', async () => {
try {
writeGbrainYml();
diff --git a/test/sync-classify-storage-invalid-8007.test.ts b/test/sync-classify-storage-invalid-8007.test.ts
new file mode 100644
index 0000000000..a9297beeec
--- /dev/null
+++ b/test/sync-classify-storage-invalid-8007.test.ts
@@ -0,0 +1,21 @@
+/** #8007: the classifier never turns an absent file into a deletion while gbrain.yml's db_only declaration is unreadable. */
+import { describe, expect, test } from 'bun:test';
+import { classifySyncPath, type SyncPathState } from '../src/core/persistence/sync-classify.ts';
+
+const gone: SyncPathState = { mode: 'committed', tree: null, level: null, dirty: null, diskOid: null, present: false,
+ page: { live: true, revision: 'R1', originMatches: true }, record: { blobOids: ['B'], revision: 'R1', parserSame: true },
+ attested: null, levelSha256: null, published: false, hold: null, marker: null, baseline: null, spelling: false, episodeUnsatisfied: false, keepDb: false, openWork: false };
+const HOLD = { cls: 'delete_drift', settled: false, action: 'hold', side: 'deleted', state: 'file_gone', uncommitted: 'storage_invalid' };
+
+describe('#8007: dbOnly unknown holds a missing-file page', () => {
+ test('a proven delete without the declaration question; held with it, on every origin evidence, strict or not', () => {
+ expect(classifySyncPath(gone)).toMatchObject({ cls: 'delete', action: 'delete' });
+ expect(classifySyncPath({ ...gone, dbOnly: true })).toMatchObject({ cls: 'kept', settled: true, action: 'none' });
+ for (const originEvidence of [undefined, 'committed', 'never_committed', 'ignored', 'unknown'] as const) {
+ const state = { ...gone, dbOnly: 'unknown' as const, ...(originEvidence ? { originEvidence } : {}) };
+ expect(classifySyncPath(state)).toMatchObject(HOLD);
+ expect(classifySyncPath(state, { strict: true })).toMatchObject(HOLD);
+ expect(classifySyncPath(state, { strict: true }).action).not.toBe('delete');
+ }
+ });
+});
diff --git a/test/sync-git-autoinit.test.ts b/test/sync-git-autoinit.test.ts
index ae0fd72eae..16a14eefc4 100644
--- a/test/sync-git-autoinit.test.ts
+++ b/test/sync-git-autoinit.test.ts
@@ -277,10 +277,10 @@ describe('#2964: sync auto-inits a never-git-initialized default brain dir', ()
test('a gbrain.yml that mentions db_only but resolves no dirs refuses the baseline commit (round 6 P2: unsupported-syntax sniff test)', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const { execSync } = await import('child_process');
- // Flow-style array — valid YAML, but the narrow custom parser only
- // handles block-style lists, so loadStorageConfig warns and resolves
- // an empty db_only list rather than throwing.
- writeFileSync(join(dir, 'gbrain.yml'), 'storage:\n db_only: [private-cache/]\n');
+ // A flow-style array split over lines — valid YAML, but the narrow
+ // custom parser reads block lists and one-line flow lists only (#8007),
+ // so loadStorageConfig resolves an empty db_only list rather than throwing.
+ writeFileSync(join(dir, 'gbrain.yml'), 'storage:\n db_only: [private-cache/,\n archive/]\n');
await expect(
performSync(engine, { noPull: true, noEmbed: true, full: true }),
diff --git a/test/sync-storage-invalid-8007.test.ts b/test/sync-storage-invalid-8007.test.ts
new file mode 100644
index 0000000000..f3580d5a6a
--- /dev/null
+++ b/test/sync-storage-invalid-8007.test.ts
@@ -0,0 +1,91 @@
+/**
+ * #8007: a managed sync decides whether a page whose file is gone is deleted. A page under a declared storage.db_only dir
+ * is kept (#7324), so when gbrain.yml's storage section cannot be read (tiers that overlap) or declares db_only in a form
+ * that resolves no directory (a flow list split over lines), sync cannot tell which pages are database-only. On a real
+ * managed brain (PGLite, and Postgres through the E2E wrapper), this protects:
+ * - the frozen discovery manifest drops every deletion and names it in storageInvalidKept;
+ * - a managed sync, incremental and full, deletes nothing: each missing-file page is held (file_drift, reason
+ * storage_invalid, fix `gbrain storage status`) and stays live, and the retention purge removes none of them;
+ * - the same holds on a strict (canonical: git) source, where Git's verdict would otherwise delete a committed-then-removed file;
+ * - once gbrain.yml is valid again (a one-line flow list, which the parser now reads), the next syncs keep the db_only
+ * pages, delete the removed page outside db_only, and release the storage_invalid holds.
+ */
+import { describe, expect, test } from 'bun:test';
+import { execFileSync } from 'node:child_process';
+import { mkdirSync, rmSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { discoverManagedSync } from '../src/core/persistence/sync-discovery.ts';
+import { performManagedSync } from '../src/core/persistence/sync-run.ts';
+import { purgeDeletedPagesCoordinated } from '../src/core/persistence/purge-deleted.ts';
+import { readGitHoldListing } from '../src/core/persistence/sync-holds.ts';
+import { managedBrain } from './helpers/managed-brain.ts';
+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';
+
+const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
+const commit = (root: string, message: string) => { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', message); };
+const note = (title: string, body: string) => `---\ntype: concept\ntitle: ${title}\n---\n\n${body}\n`;
+const OPTS = { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true };
+const CARDS = ['library/card-cached', 'library/card-gone'];
+const REMOVED = ['library/card-cached.md', 'library/card-gone.md', 'notes/control.md'];
+const setup = ({ root }: { root: string }) => {
+ git(root, 'init', '-q');
+ for (const slug of [...CARDS, 'notes/keep', 'notes/control']) {
+ mkdirSync(join(root, slug, '..'), { recursive: true });
+ writeFileSync(join(root, `${slug}.md`), note(slug, `The committed body of ${slug}.`));
+ }
+ commit(root, 'seed');
+};
+/** gbrain.yml in the given shape, library/ untracked and ignored (card-gone's file removed too), notes/control.md git rm'd. */
+function invalidateAndUntrack(root: string, yml: string) {
+ writeFileSync(join(root, 'gbrain.yml'), yml);
+ writeFileSync(join(root, '.gitignore'), 'library/\n');
+ git(root, 'rm', '-r', '-q', '--cached', 'library');
+ git(root, 'rm', '-q', 'notes/control.md');
+ rmSync(join(root, 'library/card-gone.md'));
+ commit(root, 'untrack library with gbrain.yml');
+}
+const live = async (engine: BrainEngine, slug: string) => (await engine.executeRaw<{ live: boolean }>(
+ "SELECT deleted_at IS NULL AS live FROM pages WHERE source_id='default' AND slug=$1", [slug]))[0]?.live;
+const storageHolds = async (engine: BrainEngine) => ((await readGitHoldListing(engine, ['default'], 100))[0]?.holds ?? [])
+ .filter(hold => hold.code === 'file_drift' && hold.meta.reason === 'storage_invalid').map(hold => hold.path).sort();
+
+const INVALID = [
+ ['tiers that overlap (the storage config cannot load)', 'storage:\n db_only:\n - library/\n db_tracked:\n - library/\n'],
+ ['a db_only flow list split over lines (resolves no directory)', 'storage:\n db_only: [library/,\n archive/]\n'],
+] as const;
+
+for (const backend of testBackends()) describe(`[${backend}] #8007: an invalid gbrain.yml never turns a missing file into a deletion`, () => {
+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;
+
+ for (const strict of [false, true]) for (const [name, yml] of INVALID) test(`${strict ? 'strict (canonical git) source, ' : ''}${name}: discovery and sync hold every missing-file page; the purge removes none; a fixed gbrain.yml settles them`, () => managedBrain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ if (strict) await engine.executeRaw(`UPDATE sources SET config = config || '{"canonical":"git"}'::jsonb WHERE id='default'`);
+ invalidateAndUntrack(root, yml);
+
+ for (const full of [false, true]) {
+ const discovered = await discoverManagedSync(engine, { ...OPTS, full });
+ expect(discovered.entries.filter(entry => entry.action === 'delete')).toEqual([]);
+ expect([...discovered.storageInvalidKept ?? []].sort()).toEqual(REMOVED);
+ }
+ await performManagedSync(engine, OPTS);
+ await performManagedSync(engine, { ...OPTS, full: true });
+ for (const slug of [...CARDS, 'notes/control', 'notes/keep']) expect({ slug, live: await live(engine, slug) }).toEqual({ slug, live: true });
+ expect(await storageHolds(engine)).toEqual(REMOVED);
+ const hold = ((await readGitHoldListing(engine, ['default'], 100))[0]?.holds ?? []).find(h => h.path === 'notes/control.md')!;
+ expect(hold.meta).toMatchObject({ side: 'deleted', state: 'file_gone', reason: 'storage_invalid' });
+
+ const purged = await purgeDeletedPagesCoordinated(engine, 0);
+ for (const slug of [...CARDS, 'notes/control']) expect(purged.slugs).not.toContain(slug);
+ for (const slug of [...CARDS, 'notes/control']) expect({ slug, live: await live(engine, slug) }).toEqual({ slug, live: true });
+
+ writeFileSync(join(root, 'gbrain.yml'), 'storage:\n db_only: [library/]\n');
+ commit(root, 'fix gbrain.yml');
+ await performManagedSync(engine, OPTS);
+ await performManagedSync(engine, { ...OPTS, full: true });
+ for (const slug of CARDS) expect({ slug, live: await live(engine, slug) }).toEqual({ slug, live: true });
+ expect(await live(engine, 'notes/control')).toBe(false);
+ expect(await live(engine, 'notes/keep')).toBe(true);
+ expect(await storageHolds(engine)).toEqual([]);
+ }, { databaseUrl, setup }), 240_000);
+});