Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(ghost-pages): check code and image pages at their recorded file, never at <slug>.md; a recorded path that names no file inside the source is reported recorded_path_unresolvable and never removed (#7736)

garrytan3f6d9db71910
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2634aeb65c..73615e8ce0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,27 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**`gbrain pages ghosts` and `gbrain repair ghost-pages` no longer flag live code pages as ghosts, so applying the repair can no longer soft-delete a synced `.json` (or any other code or image) page whose file is right there.**
+
+Before, the ghost check looked for a code page's file at `<slug>.md`, because the path resolver it shared with Markdown pages rejected every recorded path that did not end in `.md` or `.mdx`. A synced `projects/example/decisions.active.json` page was looked for at `projects-example-decisions-active-json.md`, found missing, and proposed for soft-delete; the repair's own recheck used the same path, so it agreed. Applying the repair removed live pages, and after the 72-hour purge they were gone for good. Now a page that records a non-Markdown file is checked at that file, both in the scan and in the recheck the repair runs before each soft delete.
+
+### What changed
+
+| Situation | Before | Now |
+|---|---|---|
+| A code or image page whose recorded file exists | listed as a ghost (missing `<slug>.md`); the apply soft-deleted it | not a ghost |
+| A code or image page whose recorded file is gone and no `<slug>.md` exists | listed at `<slug>.md` | listed at its recorded path; the apply soft-deletes it on managed and unmanaged brains |
+| A page whose recorded non-Markdown path names no file inside the source (absolute, `..`) | checked at `<slug>.md` and proposed for soft-delete | never a ghost, never removed; listed apart as `recorded_path_unresolvable` by doctor, `gbrain pages ghosts` and the preview |
+| `gbrain repair uncommitted-deletions` on a soft-deleted code page | read Git evidence for `<slug>.md` | reads it for the recorded file |
+
+### How to get it
+
+Upgrade. If you previewed `gbrain repair ghost-pages` on an older version, preview again before applying: the apply rechecks every page and keeps a code page whose file is present (`file_restored`).
+
+Reported by @Kkkkkuro (#7736).
+
## [0.60.472.0] - 2026-10-27
**A `gbrain sources refresh` whose process dies mid-refresh no longer wedges the managed checkout. The next refresh finishes it, and status tells you who holds a refresh and whether it is still running.**
diff --git a/docs/guides/repair.md b/docs/guides/repair.md
index bbb4170780..19191ca864 100644
--- a/docs/guides/repair.md
+++ b/docs/guides/repair.md
@@ -1199,6 +1199,14 @@ database only and are never ghosts. Doctor `undeclared_db_only_pages` is a
separate audit. A preview lists at most 1,000 ghosts; apply them, then preview
again for the rest. Sync never removes ghost pages by itself.
+A code or image page (a synced `.json`, `.ts` or `.png` file) is checked at
+the file it records, never at `<slug>.md`: while that file is there the page
+is not a ghost. It is a ghost only when its recorded file is gone and no file
+sits at its slug path either, and the preview lists it at the recorded path. A
+page whose recorded path names no file inside the source (an absolute path, a
+path with `..`) is never a ghost and is never removed: doctor, `gbrain pages
+ghosts` and the preview list it apart with code `recorded_path_unresolvable`.
+
<a id="uncommitted-deletions"></a>
### Uncommitted deletions
diff --git a/src/commands/doctor/checks/ghost-pages.ts b/src/commands/doctor/checks/ghost-pages.ts
index a51590b1f4..c940027211 100644
--- a/src/commands/doctor/checks/ghost-pages.ts
+++ b/src/commands/doctor/checks/ghost-pages.ts
@@ -23,7 +23,7 @@ import { agentFix, checkError } from '../check-fix.ts';
import { connectedEngine, type DoctorContext, type DoctorEntry } from '../context.ts';
import { GHOST_CHECK_LIMIT, GHOST_SCAN_LIMIT, readGhostPages } from '../../../core/ghost-pages.ts';
import { boundUnknownDetails } from '../bounded-unknown.ts';
-import { SKILL_PACK_MEMBER_CODE, skillPathDetail } from '../../../core/repair/ghost-pages.ts';
+import { SKILL_PACK_MEMBER_CODE, UNRESOLVABLE_NOTE, skillPathDetail, unresolvableDetail } from '../../../core/repair/ghost-pages.ts';
const DOCS = 'docs/guides/repair.md#ghost-pages';
@@ -33,10 +33,13 @@ async function runGhostPages(ctx: DoctorContext): Promise<Check[]> {
const summary = await readGhostPages(connectedEngine(ctx), {});
const details = { count: summary.count, by_source: summary.by_source, samples: summary.samples, inspected: summary.inspected,
truncated: summary.truncated, exact: summary.exact, docs: DOCS,
- ...(summary.skill_paths ? { skill_paths: { ...skillPathDetail(undefined, summary.skill_path_samples, summary.skill_paths), samples: summary.skill_path_samples } } : {}) };
- const skillNote = summary.skill_paths
+ ...(summary.skill_paths ? { skill_paths: { ...skillPathDetail(undefined, summary.skill_path_samples, summary.skill_paths), samples: summary.skill_path_samples } } : {}),
+ ...(summary.unresolvable ? { unresolvable: unresolvableDetail(summary.unresolvable, summary.unresolvable_samples) } : {}) };
+ const skillNote = (summary.skill_paths
? ` ${summary.skill_paths} more are files of the shared skillpack missing from the checkout (${SKILL_PACK_MEMBER_CODE}: ${summary.skill_path_samples.map(page => `${page.source_id}:${page.slug}`).join(', ')}); the repair leaves them alone: restore each file on the owner host (git -C <source root> checkout -- <path>, or republish the skill with put_skill).`
- : '';
+ : '') + (summary.unresolvable
+ ? ` ${summary.unresolvable} page(s) record a path that names no file inside the source (${summary.unresolvable_samples.map(page => `${page.source_id}:${page.slug}`).join(', ')}). ${UNRESOLVABLE_NOTE}`
+ : '');
const ghostSources = summary.by_source.filter(entry => entry.count > 0);
const unknown = summary.by_source.filter(entry => entry.status === 'unknown');
if (ghostSources.length) {
diff --git a/src/commands/pages-ghosts.ts b/src/commands/pages-ghosts.ts
index 04253b1a34..6e8000389a 100644
--- a/src/commands/pages-ghosts.ts
+++ b/src/commands/pages-ghosts.ts
@@ -10,7 +10,7 @@ import { opError } from '../core/ops/contract.ts';
import { readFix } from '../core/ops/op-fix.ts';
import { resolveSourceId } from '../core/source-resolver.ts';
import { scanGhostSource } from '../core/ghost-pages.ts';
-import { GHOST_CONSEQUENCES, GHOST_REPAIR_LIMIT, SKILL_PATH_NOTE, ghostPreviewCommand, ghostScanRefusal, skillPathRestore } from '../core/repair/ghost-pages.ts';
+import { GHOST_CONSEQUENCES, GHOST_REPAIR_LIMIT, SKILL_PATH_NOTE, UNRESOLVABLE_NOTE, ghostPreviewCommand, ghostScanRefusal, skillPathRestore, unresolvableDetail } from '../core/repair/ghost-pages.ts';
export const PAGES_GHOSTS_USAGE = 'Usage: gbrain pages ghosts [--source <id>] [--json]';
@@ -35,14 +35,18 @@ export async function runPagesGhosts(engine: BrainEngine, args: string[]): Promi
? [`Preview the removal: ${ghostPreviewCommand(sourceId)} (writes nothing; apply only after the user agrees).`, GHOST_CONSEQUENCES.keep]
: [];
if (scan.skill_paths.length) next.push(SKILL_PATH_NOTE);
+ if (scan.unresolvable_count) next.push(UNRESOLVABLE_NOTE);
if (json) {
console.log(JSON.stringify({ source_id: sourceId, status: scan.status, ...(scan.reason ? { reason: scan.reason } : {}), count: scan.ghosts.length,
- ghosts: scan.ghosts, ...(scan.skill_paths.length ? { skill_paths: scan.skill_paths } : {}), inspected: scan.inspected, file_backed: scan.file_backed, truncated: scan.truncated, exact: !scan.truncated, next }, null, 2));
+ ghosts: scan.ghosts, ...(scan.skill_paths.length ? { skill_paths: scan.skill_paths } : {}),
+ ...(scan.unresolvable_count ? { unresolvable: unresolvableDetail(scan.unresolvable_count, scan.unresolvable) } : {}), inspected: scan.inspected, file_backed: scan.file_backed, truncated: scan.truncated, exact: !scan.truncated, next }, null, 2));
return;
}
console.log(`${scan.ghosts.length}${scan.truncated ? '+' : ''} ghost page(s) in source ${sourceId} (${scan.inspected} live page(s) inspected, ${scan.file_backed} file-backed).`);
for (const ghost of scan.ghosts) console.log(` ${ghost.slug} missing ${ghost.path} revision ${ghost.revision}`);
if (scan.skill_paths.length) console.log(`${scan.skill_paths.length} more are files of the shared skillpack (listed apart; the repair leaves them alone):`);
for (const ghost of scan.skill_paths) console.log(` ${ghost.slug} missing ${ghost.path} restore: ${skillPathRestore(scan.root, ghost)}`);
+ if (scan.unresolvable_count) console.log(`${scan.unresolvable_count} page(s) record a path that names no file inside the source (not ghosts; never removed):`);
+ for (const page of scan.unresolvable) console.log(` ${page.slug} recorded path ${page.source_path}`);
for (const line of next) console.log(line);
}
diff --git a/src/core/ghost-pages.ts b/src/core/ghost-pages.ts
index 3ef27523d2..d6f1ce6d4f 100644
--- a/src/core/ghost-pages.ts
+++ b/src/core/ghost-pages.ts
@@ -29,6 +29,12 @@
* ordinary ghost; the knowledge guard lets its retirement through as a database-only
* soft delete.
*
+ * #7736: a code or image page records the path of its own file (`decisions.json`), which the Markdown
+ * resolver behind `prepareFileTarget` rejects, falling back to `<slug>.md`. Such a page is checked at its recorded
+ * file first: present, it is not a ghost; absent, it is a ghost only when `liveFileDrift` also finds no file. A page
+ * whose recorded non-Markdown path cannot be placed inside the source (absolute, `..`, NUL) is never a ghost: it is
+ * reported apart (`unresolvable`, code `recorded_path_unresolvable`) and never removed.
+ *
* Readers: doctor `ghost_pages`, `gbrain pages ghosts` and `gbrain repair
* ghost-pages` (`scanGhostSource`, `ghostPageNow`), and the read-scope MCP
* health op (`readGhostPages`, counts only for remote callers).
@@ -37,7 +43,7 @@ import { statSync } from 'node:fs';
import { join, relative, sep } from 'node:path';
import type { BrainEngine } from './engine.ts';
import type { PageSnapshot } from './page-state/types.ts';
-import { resolveSourceLocalFilePath, sourceGitScope } from './markdown.ts';
+import { resolveRecordedSourceFilePath, resolveSourceLocalFilePath, sourceGitScope } from './markdown.ts';
import { recordedPathFromFileUri, scannerSlugRootMode } from './write-through.ts';
import { isOpPageSlug } from './ops/context.ts';
import { existingLocalHostId } from './persistence/identity.ts';
@@ -49,19 +55,26 @@ import { isReservedSkillBundlePath } from './skill-reserved-paths.ts';
import { sharedPackMember } from './shared-skills/knowledge-guard.ts';
export interface GhostPage { source_id: string; slug: string; path: string; revision: string; page_id: number }
+/** #7736: a live page whose recorded non-Markdown path names no file inside its source; never a ghost, never removed. */
+export interface UnresolvablePage { source_id: string; slug: string; source_path: string; page_id: number }
+export const UNRESOLVABLE_PATH_CODE = 'recorded_path_unresolvable';
export interface GhostPagesSummary {
status: 'known' | 'unknown';
/** 'not_owner_host' | 'not_applicable' | 'storage_unavailable' | 'ambiguous_storage' */
reason?: string;
/** Ghosts found among inspected pages. */
count: number;
- by_source: Array<{ source_id: string; count: number; status: 'known' | 'unknown' | 'not_applicable'; reason?: string; skill_paths?: number }>;
+ by_source: Array<{ source_id: string; count: number; status: 'known' | 'unknown' | 'not_applicable'; reason?: string; skill_paths?: number; unresolvable?: number }>;
/** Bounded by `sampleLimit`. */
samples: GhostPage[];
/** #6910: ghosts whose path is a member of an adopted shared skillpack (the pack's file is missing); not in `count`. */
skill_paths: number;
/** Bounded by `sampleLimit`. */
skill_path_samples: GhostPage[];
+ /** #7736: live pages whose recorded path could not be resolved; not in `count`. */
+ unresolvable: number;
+ /** Bounded by `sampleLimit`. */
+ unresolvable_samples: UnresolvablePage[];
inspected: number;
truncated: boolean;
exact: boolean;
@@ -77,6 +90,10 @@ export interface GhostSourceScan {
ghosts: GhostPage[];
/** #6910: confirmed ghosts whose path is a member of the source's adopted shared skillpack; never in `ghosts`. */
skill_paths: GhostPage[];
+ /** #7736: live pages whose recorded non-Markdown path could not be resolved (bounded by `UNRESOLVABLE_SAMPLE_LIMIT`); never in `ghosts`. */
+ unresolvable: UnresolvablePage[];
+ /** How many pages were unresolvable (`unresolvable` holds at most `UNRESOLVABLE_SAMPLE_LIMIT` of them). */
+ unresolvable_count: number;
/** The source's canonical root on this host, when the scan ran (the skill-path restore step names it). */
root?: string;
/** Live pages the existence pre-filter examined. */
@@ -90,6 +107,7 @@ export interface GhostSourceScan {
}
export const GHOST_SAMPLE_LIMIT = 5;
+export const UNRESOLVABLE_SAMPLE_LIMIT = 20;
/** Candidates `readGhostPages` (doctor, health) confirms per source. */
export const GHOST_CHECK_LIMIT = 200;
/** Live pages the pre-filter reads per source. */
@@ -123,29 +141,43 @@ export async function sourceContext(engine: BrainEngine, sourceId: string): Prom
owner: { host_id: hostId, worktree_id: binding.worktree_id, owner_epoch: String(binding.owner_epoch), source_incarnation: binding.source_incarnation } };
}
-/** The path `prepareFileTarget` compares, resolved the same way (recorded source path, captured file URI, then the slug path). */
-export function canonicalPath(context: Extract<SourceContext, { ok: true }>, page: Pick<PageRow, 'slug' | 'source_path' | 'source_uri'>): string {
+/** #7736: the file a non-Markdown page records (a code or image page); undefined for a Markdown or unrecorded path, null when the path names no file inside the source. */
+function recordedFilePath(context: Extract<SourceContext, { ok: true }>, page: Pick<PageRow, 'slug' | 'source_path'>): string | null | undefined {
+ const recorded = page.source_path?.trim();
+ if (!recorded || /\.mdx?$/i.test(recorded)) return undefined;
+ return resolveRecordedSourceFilePath(context.root, recorded, page.slug, context.mode, context.gitScope);
+}
+
+/**
+ * The path `prepareFileTarget` compares, resolved the same way (recorded source path, captured file URI, then the slug path);
+ * #7736: a page recording a non-Markdown file is checked at that file, and null means its recorded path names no file inside the source.
+ */
+export function canonicalPath(context: Extract<SourceContext, { ok: true }>, page: Pick<PageRow, 'slug' | 'source_path' | 'source_uri'>): string | null {
+ const recorded = recordedFilePath(context, page);
+ if (recorded !== undefined) return recorded;
const captured = recordedPathFromFileUri(page.source_uri, context.root);
return resolveSourceLocalFilePath(context.root, page.source_path, page.slug, page.source_path ? context.mode : undefined, context.gitScope)
?? (captured ? join(context.root, captured) : join(context.root, `${page.slug}.md`));
}
-/** The existence pre-filter: `present`, `database_only`, a `candidate` at its source-relative path, or `ambiguous`. */
-function prefilter(context: Extract<SourceContext, { ok: true }>, page: PageRow): { kind: 'present' | 'database_only' | 'ambiguous' } | { kind: 'candidate'; path: string } {
+/** The existence pre-filter: `present`, `database_only`, a `candidate` at its source-relative path, `unresolvable` (#7736) or `ambiguous`. */
+function prefilter(context: Extract<SourceContext, { ok: true }>, page: PageRow): { kind: 'present' | 'database_only' | 'unresolvable' | 'ambiguous' } | { kind: 'candidate'; path: string } {
if (!isOpPageSlug(page.slug)) return { kind: 'database_only' };
const path = canonicalPath(context, page);
- try {
- statSync(path);
- return { kind: 'present' };
- } catch (error) {
- if (!missing(error)) return { kind: 'ambiguous' };
+ if (path !== null) {
+ try {
+ statSync(path);
+ return { kind: 'present' };
+ } catch (error) {
+ if (!missing(error)) return { kind: 'ambiguous' };
+ }
}
try {
if (publishesDatabaseOnly(context.root, page.slug, { page: { deleted_at: null, source_path: page.source_path, source_uri: page.source_uri } })) return { kind: 'database_only' };
} catch {
return { kind: 'ambiguous' };
}
- return { kind: 'candidate', path: relative(context.root, path).split(sep).join('/') };
+ return path === null ? { kind: 'unresolvable' } : { kind: 'candidate', path: relative(context.root, path).split(sep).join('/') };
}
/** The full check of one candidate: `ghost` with its revision, `not_ghost`, or `unknown` with the classifier's reason. */
@@ -153,10 +185,23 @@ async function confirm(engine: BrainEngine, context: Extract<SourceContext, { ok
Promise<{ kind: 'ghost'; ghost: GhostPage } | { kind: 'not_ghost'; reason: string } | { kind: 'unknown'; reason: string }> {
const snapshot: PageSnapshot | null = await engine.readPageSnapshot(slug, { sourceId });
if (!snapshot || snapshot.page.deleted_at) return { kind: 'not_ghost', reason: 'page_missing' };
+ const page = { slug, source_path: snapshot.page.source_path ?? null, source_uri: snapshot.page.source_uri ?? null };
+ // #7736: `liveFileDrift` resolves Markdown paths only, so a recorded non-Markdown file is checked here on its own; the
+ // page is a ghost only when that file and the write target `liveFileDrift` checks are both missing.
+ const recorded = recordedFilePath(context, page);
+ if (recorded === null) return { kind: 'not_ghost', reason: UNRESOLVABLE_PATH_CODE };
+ if (recorded) {
+ try {
+ statSync(recorded);
+ return { kind: 'not_ghost', reason: 'recorded_file_present' };
+ } catch (error) {
+ if (!missing(error)) return { kind: 'unknown', reason: 'ambiguous_storage' };
+ }
+ }
const live = await liveFileDrift(engine, sourceId, slug, snapshot);
if (live.reason === 'canonical_file_missing') {
- return { kind: 'ghost', ghost: { source_id: sourceId, slug, path: relative(context.root, canonicalPath(context, { slug, source_path: snapshot.page.source_path ?? null,
- source_uri: snapshot.page.source_uri ?? null })).split(sep).join('/'), revision: snapshot.revision, page_id: Number(snapshot.page.id) } };
+ return { kind: 'ghost', ghost: { source_id: sourceId, slug, path: relative(context.root, recorded ?? canonicalPath(context, page)!).split(sep).join('/'),
+ revision: snapshot.revision, page_id: Number(snapshot.page.id) } };
}
if (['agrees', 'file_database_drift', 'database_only', 'page_missing'].includes(live.reason)) return { kind: 'not_ghost', reason: live.reason };
return { kind: 'unknown', reason: live.reason === 'not_owner_host' ? 'not_owner_host' : 'ambiguous_storage' };
@@ -179,7 +224,7 @@ async function sharedPackMemberGhost(engine: BrainEngine, owner: GhostOwner, gho
export async function scanGhostSource(engine: BrainEngine, sourceId: string, opts: { candidateLimit?: number; signal?: AbortSignal } = {}): Promise<GhostSourceScan> {
const limit = opts.candidateLimit ?? GHOST_CHECK_LIMIT;
const context = await sourceContext(engine, sourceId);
- const scan: GhostSourceScan = { source_id: sourceId, status: 'known', ghosts: [], skill_paths: [], inspected: 0, file_backed: 0, candidates: 0, truncated: false };
+ const scan: GhostSourceScan = { source_id: sourceId, status: 'known', ghosts: [], skill_paths: [], unresolvable: [], unresolvable_count: 0, inspected: 0, file_backed: 0, candidates: 0, truncated: false };
if (!context.ok) return { ...scan, status: context.status, reason: context.reason };
scan.owner = context.owner;
scan.root = context.root;
@@ -195,6 +240,10 @@ export async function scanGhostSource(engine: BrainEngine, sourceId: string, opt
const verdict = prefilter(context, row);
if (verdict.kind === 'present') { scan.file_backed++; continue; }
if (verdict.kind === 'database_only') continue;
+ if (verdict.kind === 'unresolvable') {
+ if (scan.unresolvable_count++ < UNRESOLVABLE_SAMPLE_LIMIT) scan.unresolvable.push({ source_id: sourceId, slug: row.slug, source_path: row.source_path!, page_id: Number(row.id) });
+ continue;
+ }
if (verdict.kind === 'ambiguous') return { ...scan, status: 'unknown', reason: 'ambiguous_storage' };
scan.candidates++;
if (scan.candidates > limit) { scan.truncated = true; unchecked++; continue; }
@@ -233,14 +282,16 @@ export async function readGhostPages(engine: BrainEngine, opts: { sourceIds?: st
const offOwner = scans.some(scan => scan.reason === 'not_owner_host');
const count = scans.reduce((sum, scan) => sum + scan.ghosts.length, 0);
const skillPaths = scans.reduce((sum, scan) => sum + scan.skill_paths.length, 0);
+ const unresolvable = scans.reduce((sum, scan) => sum + scan.unresolvable_count, 0);
const truncated = scans.some(scan => scan.truncated);
const status = unknown || (!known.length && offOwner) ? 'unknown' : 'known';
const reason = unknown?.reason ?? (!known.length ? (offOwner ? 'not_owner_host' : 'not_applicable') : undefined);
return { status, ...(reason ? { reason } : {}), count,
by_source: scans.map(scan => ({ source_id: scan.source_id, count: scan.ghosts.length, status: scan.status, ...(scan.reason ? { reason: scan.reason } : {}),
- ...(scan.skill_paths.length ? { skill_paths: scan.skill_paths.length } : {}) })),
+ ...(scan.skill_paths.length ? { skill_paths: scan.skill_paths.length } : {}), ...(scan.unresolvable_count ? { unresolvable: scan.unresolvable_count } : {}) })),
samples: scans.flatMap(scan => scan.ghosts).slice(0, opts.sampleLimit ?? GHOST_SAMPLE_LIMIT),
skill_paths: skillPaths, skill_path_samples: scans.flatMap(scan => scan.skill_paths).slice(0, opts.sampleLimit ?? GHOST_SAMPLE_LIMIT),
+ unresolvable, unresolvable_samples: scans.flatMap(scan => scan.unresolvable).slice(0, opts.sampleLimit ?? GHOST_SAMPLE_LIMIT),
inspected: scans.reduce((sum, scan) => sum + scan.inspected, 0), truncated,
exact: status === 'known' && !truncated && !offOwner && !unknown };
}
diff --git a/src/core/markdown.ts b/src/core/markdown.ts
index e5588d99b0..cc5069efdd 100644
--- a/src/core/markdown.ts
+++ b/src/core/markdown.ts
@@ -1447,16 +1447,15 @@ export function sourceGitScope(localPath: string): string[] | null {
}
}
-export function resolveSourceLocalFilePath(
- localPath: string,
- rawSourcePath: string | null | undefined,
- pageSlug?: string | null,
- slugRootMode?: 'git-root' | 'source-root',
- gitScope?: string[] | null,
-): string | null {
+export function resolveSourceLocalFilePath(localPath: string, rawSourcePath: string | null | undefined, pageSlug?: string | null, slugRootMode?: 'git-root' | 'source-root', gitScope?: string[] | null): string | null {
+ return rawSourcePath && /\.mdx?$/i.test(rawSourcePath.trim()) ? resolveRecordedSourceFilePath(localPath, rawSourcePath, pageSlug, slugRootMode, gitScope) : null;
+}
+/** #7736: `resolveSourceLocalFilePath` for a recorded path of any extension (a code or image page records its own file, `decisions.json`):
+ * the same placement, or null when the path cannot name a file inside the source (absolute, `..`, NUL, empty). */
+export function resolveRecordedSourceFilePath(localPath: string, rawSourcePath: string | null | undefined, pageSlug?: string | null, slugRootMode?: 'git-root' | 'source-root', gitScope?: string[] | null): string | null {
if (!rawSourcePath) return null;
const value = rawSourcePath.trim();
- if (!value || value.includes('\0') || !/\.mdx?$/i.test(value)) return null;
+ if (!value || value.includes('\0')) return null;
if (isAbsolute(value) || /^[A-Za-z]:[\\/]/.test(value)) return null;
const sourceSegments = splitLocalPathSegments(value);
if (sourceSegments.length === 0 || sourceSegments.some(segment => segment === '..')) return null;
diff --git a/src/core/repair/ghost-pages.ts b/src/core/repair/ghost-pages.ts
index 83eb52fe00..edd37e7a14 100644
--- a/src/core/repair/ghost-pages.ts
+++ b/src/core/repair/ghost-pages.ts
@@ -53,7 +53,7 @@ import { readDeletionProvenance } from '../persistence/sync-origin-evidence.ts';
import { maintenancePreflight, submitMaintenanceIntent } from '../persistence/prepared-maintenance.ts';
import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';
import { MASS_RECONCILE_MIN_PAGES, MASS_RECONCILE_RATIO, massReconcileAllowed } from '../sync-reconcile.ts';
-import { ghostPageNow, scanGhostSource, type GhostOwner, type GhostPage, type GhostSourceScan } from '../ghost-pages.ts';
+import { ghostPageNow, scanGhostSource, UNRESOLVABLE_PATH_CODE, type GhostOwner, type GhostPage, type GhostSourceScan, type UnresolvablePage } from '../ghost-pages.ts';
import { afterCursor, REPAIR_ITEM_FAILURE_CODES, repairRequestId, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairPlan, type RepairScope } from './core.ts';
export const GHOST_PAGES_INTENT = GHOST_RETIREMENT_INTENT;
@@ -70,6 +70,11 @@ export const SKILL_PACK_MEMBER_CODE = 'ghost_skill_pack_member';
export const SKILL_PATH_NOTE = 'A ghost page whose path is a file of the source\'s adopted shared skillpack (skills/<name>/..., skillpack.json) is the pack\'s own file missing from the checkout; '
+ 'the shared skill publisher owns it, so this repair never removes the page. Restore the file on the owner host (git -C <source root> checkout -- <path> when it is committed; '
+ 'otherwise republish the skill with put_skill and its catalog expected_revision so the publisher writes it again); the page then stops being a ghost. These pages never block the other ghosts.';
+/** #7736: why a page whose recorded path names no file inside its source is left alone; the preview, `gbrain pages ghosts` and doctor say it. */
+export const UNRESOLVABLE_NOTE = 'A page whose recorded file path names no file inside its source (an absolute path, a path with .., or NUL) is never treated as a ghost or removed: '
+ + 'gbrain cannot tell whether its file exists. Read it with gbrain pages status <slug> --source <id> --json; the page stays as it is.';
+/** The `unresolvable` detail every reader prints. */
+export const unresolvableDetail = (count: number, samples: UnresolvablePage[]) => ({ code: UNRESOLVABLE_PATH_CODE, count, samples, note: UNRESOLVABLE_NOTE, docs: DOCS });
/** The exact restore command for one pack-member ghost. */
export const skillPathRestore = (root: string | undefined, ghost: GhostPage) => `git -C ${root ?? '<source root>'} checkout -- ${ghost.path}`;
/** The `skill_paths` detail every reader prints. */
@@ -170,19 +175,24 @@ export const ghostPagesRepair: RepairHandler = {
...(mass ? [`${scan.ghosts.length} of ${scan.file_backed} file-backed page(s) are ghosts (more than ${Math.round(MASS_RECONCILE_RATIO * 100)}%): the apply refuses (ghost_mass_delete_refused) unless GBRAIN_ALLOW_MASS_RECONCILE=1; check the checkout first.`] : []),
...(scan.truncated ? [`More ghosts may exist past the first ${GHOST_REPAIR_LIMIT} checked; apply this set, then preview again.`] : []),
...(skillPaths ? [`${skillPaths} ghost page(s) on files of the shared skillpack are listed apart (${SKILL_PACK_MEMBER_CODE}) and not applied: ${SKILL_PATH_NOTE}`] : []),
+ ...(scan.unresolvable_count ? [`${scan.unresolvable_count} page(s) record a file path that names no file inside the source; they are listed apart (${UNRESOLVABLE_PATH_CODE}) and not applied: ${UNRESOLVABLE_NOTE}`] : []),
...(ahead.length ? [`${ahead.length} ghost page(s) were written in the database after their file last held them, so each holds the only copy of that content; they are listed apart (${DATABASE_AHEAD_CODE}) and not applied.`] : []),
...(scan.reason === 'mirror_read_only' ? [`Source ${sourceId} is a read-only mirror: its pages publish no file, so none is a ghost.`] : []),
...(ghosts.length ? Object.values(GHOST_CONSEQUENCES) : []),
];
return { warnings, items: ghosts.map((ghost, index) => item(ghost, scan.owner!, hash, index === ghosts.length - 1)), preview_hash: hash,
- residuals: { ghosts: ghosts.length, file_backed: scan.file_backed, inspected: scan.inspected, ...(skillPaths ? { skill_paths: skillPaths } : {}), ...(ahead.length ? { database_ahead: ahead.length } : {}) },
+ residuals: { ghosts: ghosts.length, file_backed: scan.file_backed, inspected: scan.inspected, ...(skillPaths ? { skill_paths: skillPaths } : {}), ...(ahead.length ? { database_ahead: ahead.length } : {}),
+ ...(scan.unresolvable_count ? { unresolvable: scan.unresolvable_count } : {}) },
listing: [...ghosts.map(ghost => ({ item: `${ghost.source_id}:${ghost.slug}`, class: 'canonical_file_missing',
detail: `file ${ghost.path} is missing; revision ${ghost.revision}` })),
...ahead.map(ghost => ({ item: `${ghost.source_id}:${ghost.slug}`, class: DATABASE_AHEAD_CODE, detail: databaseAheadDetail(ghost) })),
...scan.skill_paths.map(ghost => ({ item: `${ghost.source_id}:${ghost.slug}`, class: SKILL_PACK_MEMBER_CODE,
- detail: `shared skillpack file ${ghost.path} is missing; not applied. Restore it: ${skillPathRestore(scan.root, ghost)}` }))],
+ detail: `shared skillpack file ${ghost.path} is missing; not applied. Restore it: ${skillPathRestore(scan.root, ghost)}` })),
+ ...scan.unresolvable.map(page => ({ item: `${page.source_id}:${page.slug}`, class: UNRESOLVABLE_PATH_CODE,
+ detail: `recorded path ${page.source_path} names no file inside the source; not applied` }))],
details: { purge_fuse_hours: GHOST_PURGE_FUSE_HOURS, undo: 'gbrain restore <slug> --force', keep: GHOST_CONSEQUENCES.keep, mass_delete: mass, truncated: scan.truncated, docs: DOCS,
- ...(skillPaths ? { skill_paths: skillPathDetail(scan.root, scan.skill_paths) } : {}) } };
+ ...(skillPaths ? { skill_paths: skillPathDetail(scan.root, scan.skill_paths) } : {}),
+ ...(scan.unresolvable_count ? { unresolvable: unresolvableDetail(scan.unresolvable_count, scan.unresolvable) } : {}) } };
}
if (!opts.expect) {
throw opError('invalid_params', 'gbrain repair ghost-pages --apply removes only the set a preview printed.',
@@ -252,7 +262,7 @@ export const databaseAheadDetail = (ghost: GhostPage) => `file ${ghost.path} is
/** Whether the live page is still the approved ghost under the approved owner, with a file-backed revision; otherwise the outcome that skips it. */
async function stillApproved(engine: BrainEngine, ghost: GhostPage, owner: GhostOwner): Promise<RepairItemOutcome | null> {
const now = await ghostPageNow(engine, ghost.source_id, ghost.slug);
- if (!now.ghost) return now.reason === 'agrees' || now.reason === 'file_database_drift' ? restored(ghost) : changed(`the page is no longer a ghost (${now.reason})`);
+ if (!now.ghost) return ['agrees', 'file_database_drift', 'recorded_file_present'].includes(now.reason) ? restored(ghost) : changed(`the page is no longer a ghost (${now.reason})`);
if (!sameOwner(now.owner, owner)) return changed('the source\'s canonical owner changed since the preview');
if (now.ghost.revision !== ghost.revision || now.ghost.page_id !== ghost.page_id || now.ghost.path !== ghost.path) return changed('the page or its recorded canonical path changed since the preview');
if (!await ghostFileBacked(engine, ghost, owner)) return { applied: false, outcome: 'refused', reason: DATABASE_AHEAD_CODE, detail: { message: databaseAheadDetail(ghost) } };
@@ -293,7 +303,10 @@ export async function retireGhost(ctx: OperationContext, ghost: GhostPage, owner
/**
* Preparer for `managed_maintenance_retire_ghost_page`: re-runs the classifier under the approved owner, prepares a
* `delete_page` with `allowMissingFile`, and refuses unless the prepared file target is the approved path with no
- * bytes (a file restored in between refuses here, or at filesystem publication when it appears later).
+ * bytes (a file restored in between refuses here, or at filesystem publication when it appears later). #7736: a page
+ * whose approved path is a recorded non-Markdown file is prepared at its slug path, which must hold no bytes either;
+ * the classifier re-checks the recorded file here; one restored after preparation is not seen at publication, and
+ * the soft delete stays undoable with `gbrain restore <slug> --force` until the purge.
*/
export async function prepareGhostPageRetirement(engine: BrainEngine, row: WriteRequest, config: GBrainConfig): Promise<PreparedMutation> {
const ghost = row.intent!.ghost as GhostPage | undefined;
@@ -313,7 +326,11 @@ export async function prepareGhostPageRetirement(engine: BrainEngine, row: Write
if (skip) throw refuse(skip);
await authorizeWrite(engine, row.authority, 'delete_page', row.slug);
const page = await preparePageMutation(engine, { ...row, operation: 'delete_page' }, config, undefined, undefined, { allowMissingFile: true });
- if (!page.file || relative(page.file.root, page.file.path).split(sep).join('/') !== ghost.path) throw refuse(changed('its canonical file target moved'));
+ const target = page.file ? relative(page.file.root, page.file.path).split(sep).join('/') : null;
+ // #7736: a code or image page's write target is its slug path (prepareFileTarget resolves Markdown paths only), not the
+ // recorded file the classifier just found missing again; that target must expect no bytes like any other.
+ const recordedFile = !/\.mdx?$/i.test(ghost.path) && target === `${ghost.slug}.md`;
+ if (!page.file || (target !== ghost.path && !recordedFile)) throw refuse(changed('its canonical file target moved'));
if (page.file.expectedBeforeHash !== null) throw refuse(restored(ghost));
return { ...page, validate: async tx => { await page.validate?.(tx); await authorizeWrite(tx, row.authority, 'delete_page', row.slug); },
// GBRA-133: the soft delete is the decision a held deletion of this file waited for, so its hold clears with it.
diff --git a/src/core/uncommitted-deletions.ts b/src/core/uncommitted-deletions.ts
index b1cb15a0f8..a6a52b93e0 100644
--- a/src/core/uncommitted-deletions.ts
+++ b/src/core/uncommitted-deletions.ts
@@ -85,6 +85,7 @@ export async function scanUncommittedDeletions(engine: BrainEngine, sourceId: st
const gitRoot = resolve(context.root, ...context.gitScope.map(() => '..'));
const candidates = tombstones.flatMap(row => {
const file = canonicalPath(context, row);
+ if (file === null) return [];
const gitPath = relative(gitRoot, file).split(sep).join('/');
return gitPath.startsWith('../') ? [] : [{ row, file, gitPath }];
});
diff --git a/test/fixtures/goldens/exports/runtime.json b/test/fixtures/goldens/exports/runtime.json
index 296957bd02..af9e74e977 100644
--- a/test/fixtures/goldens/exports/runtime.json
+++ b/test/fixtures/goldens/exports/runtime.json
@@ -780,6 +780,7 @@
"recoverFrontmatter",
"resolvePageFilePath",
"resolveParsedSubtype",
+ "resolveRecordedSourceFilePath",
"resolveSourceLocalFilePath",
"serializeMarkdown",
"serializePageToMarkdown",
diff --git a/test/fixtures/goldens/exports/types.json b/test/fixtures/goldens/exports/types.json
index 2e3df1bd62..11ba204583 100644
--- a/test/fixtures/goldens/exports/types.json
+++ b/test/fixtures/goldens/exports/types.json
@@ -5875,6 +5875,13 @@
"(parsed: ParsedMarkdown, existing?: { type: string; frontmatter?: Record<string, unknown> | null; } | null) => void"
]
},
+ {
+ "kind": "function",
+ "name": "resolveRecordedSourceFilePath",
+ "signature": [
+ "(localPath: string, rawSourcePath: string | null | undefined, pageSlug?: string | null, slugRootMode?: \"git-root\" | \"source-root\", gitScope?: string[] | null) => string | null"
+ ]
+ },
{
"kind": "function",
"name": "resolveSourceLocalFilePath",
diff --git a/test/ghost-pages-6696.test.ts b/test/ghost-pages-6696.test.ts
index 187aa366ad..658155da2c 100644
--- a/test/ghost-pages-6696.test.ts
+++ b/test/ghost-pages-6696.test.ts
@@ -25,7 +25,10 @@ import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';
import { installFaultHook } from '../src/core/persistence/fault-points.ts';
import { liveFileDrift } from '../src/core/persistence/drift-conflicts.ts';
import { classifyTerminal } from '../src/core/persistence/terminal-classify.ts';
-import { readGhostPages, ghostPageNow, __setGhostScanLimitForTests } from '../src/core/ghost-pages.ts';
+import { readGhostPages, ghostPageNow, scanGhostSource, __setGhostScanLimitForTests } from '../src/core/ghost-pages.ts';
+import { retireGhost } from '../src/core/repair/ghost-pages.ts';
+import { importCodeFile } from '../src/core/import-file.ts';
+import { slugifyCodePath } from '../src/core/sync.ts';
import { resolveRepairScope } from '../src/core/repair/core.ts';
import { AUTO_REPAIR_REGISTRY, EXPLICIT_REPAIR_REGISTRY, repairForCheck, repairRunner, repairSpec } from '../src/core/repair/registry.ts';
import { runRepairCommand } from '../src/commands/repair.ts';
@@ -88,6 +91,24 @@ async function refusal(fn: () => Promise<unknown>): Promise<OperationError> {
const repair = async (sourceId: string, opts: { apply?: boolean; expect?: string } = {}) => (await repairRunner(engine, { apply: opts.apply === true, logger }))
.run('ghost-pages', await resolveRepairScope(engine, sourceId), { explicit: true, sourceFlag: sourceId, expect: opts.expect });
+/** #7736: a JSON file committed to the source and indexed as a code page by the real indexer; its flattened slug has no `<slug>.md`. */
+const CODE_PATH = 'projects/example/decisions.active.json';
+const CODE_SLUG = slugifyCodePath(CODE_PATH);
+async function seedCodePage(f: Fixture, managed = true): Promise<void> {
+ const json = '{"decision":"keep"}\n';
+ mkdirSync(join(f.root, 'projects/example'), { recursive: true });
+ writeFileSync(join(f.root, CODE_PATH), json);
+ execFileSync('git', ['add', CODE_PATH], { cwd: f.root });
+ execFileSync('git', ['-c', 'user.name=test', '-c', '[email protected]', 'commit', '-q', '-m', 'code file'], { cwd: f.root });
+ if (managed) await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');
+ await importCodeFile(engine, CODE_PATH, json, { noEmbed: true, sourceId: f.sourceId });
+ if (managed) await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+ const [row] = await engine.executeRaw<{ page_kind: string; source_path: string }>('SELECT page_kind, source_path FROM pages WHERE source_id=$1 AND slug=$2 AND deleted_at IS NULL', [f.sourceId, CODE_SLUG]);
+ expect(row).toEqual({ page_kind: 'code', source_path: CODE_PATH });
+ expect(existsSync(join(f.root, `${CODE_SLUG}.md`))).toBe(false);
+}
+const json = async (argv: string[]) => JSON.parse(await withEnv({ GBRAIN_INTERACTIVE: '0', CI: '1' }, () => capture(() => runRepairCommand(engine, argv)))).results[0];
+
/** notes/agrees (file matches), notes/drift (file edited), notes/ghost (file removed), scratch/kept (declared db_only, file removed), notes/gone (deleted). */
async function seed(f: Fixture): Promise<string> {
writeFileSync(join(f.root, 'gbrain.yml'), 'storage:\n db_only:\n - scratch/\n');
@@ -260,6 +281,64 @@ for (const backend of testBackends()) describe(`#6696 B5 ghost pages (${backend}
}));
}
+ test('#7736: a live JSON code page is not a ghost: pages ghosts, the preview and the confirm step leave it, the apply keeps it', () => fixture(async f => {
+ await seed(f);
+ await seedCodePage(f);
+ const listed = JSON.parse(await capture(() => runPagesGhosts(engine, ['--source', f.sourceId, '--json'])));
+ expect(listed.ghosts.map((ghost: { slug: string }) => ghost.slug)).toEqual(['notes/ghost']);
+ expect(listed.unresolvable).toBeUndefined();
+ expect(await ghostPageNow(engine, f.sourceId, CODE_SLUG)).toMatchObject({ ghost: null, reason: 'recorded_file_present' });
+ const preview = await json(['ghost-pages', '--source', f.sourceId, '--json']);
+ expect(preview.listing.map((entry: { item: string }) => entry.item)).toEqual([`${f.sourceId}:notes/ghost`]);
+ const applied = await json(['ghost-pages', '--source', f.sourceId, '--apply', '--expect', preview.preview_hash, '--yes', '--json']);
+ expect(applied).toMatchObject({ applied: 1, outcomes: { soft_deleted: 1 } });
+ expect(await live(f.sourceId, 'notes/ghost')).toBe(false);
+ expect(await live(f.sourceId, CODE_SLUG)).toBe(true);
+ // The confirm step checks the recorded file on its own: an approval naming the page at its slug path (what a preview
+ // before #7736 listed) keeps it.
+ const snapshot = (await engine.readPageSnapshot(CODE_SLUG, { sourceId: f.sourceId }))!;
+ const owner = (await scanGhostSource(engine, f.sourceId)).owner!;
+ const stale = { source_id: f.sourceId, slug: CODE_SLUG, path: `${CODE_SLUG}.md`, revision: snapshot.revision, page_id: Number(snapshot.page.id) };
+ expect(await retireGhost(f.ctx, stale, owner, async () => randomUUID())).toMatchObject({ applied: false, outcome: 'file_restored' });
+ expect(await live(f.sourceId, CODE_SLUG)).toBe(true);
+ expect(existsSync(join(f.root, CODE_PATH))).toBe(true);
+ }));
+
+ for (const managed of [true, false]) {
+ test(`#7736: a code page whose recorded JSON file is gone is a ghost at that path and the apply retires it (${managed ? 'managed' : 'unmanaged'})`, () => fixture(async f => {
+ await seedCodePage(f, managed);
+ unlinkSync(join(f.root, CODE_PATH));
+ const preview = await repair(f.sourceId);
+ expect(preview.sample).toEqual([`${f.sourceId}:${CODE_SLUG}`]);
+ expect((await ghostPageNow(engine, f.sourceId, CODE_SLUG)).ghost).toMatchObject({ path: CODE_PATH });
+ const applied = await repair(f.sourceId, { apply: true, expect: preview.preview_hash });
+ expect(applied).toMatchObject({ applied: 1, outcomes: { soft_deleted: 1 } });
+ expect(await live(f.sourceId, CODE_SLUG)).toBe(false);
+ }, { managed }));
+ }
+
+ test('#7736: a code page whose recorded path names no file inside the source is reported unresolvable and never removed', () => fixture(async f => {
+ await seed(f);
+ await seedCodePage(f);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');
+ await engine.executeRaw('UPDATE pages SET source_path=$3 WHERE source_id=$1 AND slug=$2', [f.sourceId, CODE_SLUG, '../outside/decisions.active.json']);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+ const summary = await readGhostPages(engine, { sourceIds: [f.sourceId] });
+ expect(summary).toMatchObject({ status: 'known', count: 1, unresolvable: 1 });
+ expect(summary.unresolvable_samples).toEqual([{ source_id: f.sourceId, slug: CODE_SLUG, source_path: '../outside/decisions.active.json', page_id: expect.any(Number) }]);
+ expect(await ghostPageNow(engine, f.sourceId, CODE_SLUG)).toMatchObject({ ghost: null, reason: 'recorded_path_unresolvable' });
+ const listed = JSON.parse(await capture(() => runPagesGhosts(engine, ['--source', f.sourceId, '--json'])));
+ expect(listed.ghosts.map((ghost: { slug: string }) => ghost.slug)).toEqual(['notes/ghost']);
+ expect(listed.unresolvable).toMatchObject({ code: 'recorded_path_unresolvable', count: 1 });
+ const preview = await json(['ghost-pages', '--source', f.sourceId, '--json']);
+ expect(preview.listing).toContainEqual(expect.objectContaining({ item: `${f.sourceId}:${CODE_SLUG}`, class: 'recorded_path_unresolvable' }));
+ const applied = await json(['ghost-pages', '--source', f.sourceId, '--apply', '--expect', preview.preview_hash, '--yes', '--json']);
+ expect(applied).toMatchObject({ applied: 1, outcomes: { soft_deleted: 1 } });
+ expect(await live(f.sourceId, CODE_SLUG)).toBe(true);
+ const [check] = await ghostPagesEntry.run({ engine } as never) as Array<Record<string, any>>;
+ expect(check.details.unresolvable).toMatchObject({ code: 'recorded_path_unresolvable', count: 1 });
+ }));
+
test('an unmanaged brain soft-deletes the approved ghost directly', () => fixture(async f => {
mkdirSync(join(f.root, 'notes'));
for (const slug of ['notes/one', 'notes/two']) {