Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Connector facts on managed brains, a free takes dry run, ledgered mention-link removal and claude-cli timeouts (#9552, #9547, #9548, #9553) (#9569)
Garry Tan09fe4c5ae0d1
diff --git a/INSTALL_FOR_AGENTS.md b/INSTALL_FOR_AGENTS.mdindex a89258c8ef..56b82b2d9f 100644--- a/INSTALL_FOR_AGENTS.md+++ b/INSTALL_FOR_AGENTS.md@@ -904,6 +904,8 @@ grants.writing/originals page content to your configured chat model (defaultAnthropic Haiku). Refuses to run unless `takes.bootstrap_enabled=true`is set in config AND `--yes` is passed. Two-gate opt-in by design.+ `--dry-run` without `--yes` only lists the candidate pages and sends+ nothing to the model.- Autopilot's auto-apply tier for takes-bootstrap is `manual_only`(do not bypass).diff --git a/behavior-changes.d/takes_extract_dry_run_needs_yes_9547.md b/behavior-changes.d/takes_extract_dry_run_needs_yes_9547.mdnew file mode 100644index 0000000000..c8e5ae0871--- /dev/null+++ b/behavior-changes.d/takes_extract_dry_run_needs_yes_9547.md@@ -0,0 +1,4 @@+---+kind: action+---+`gbrain takes extract --from-pages --dry-run` without `--yes` no longer sends page content to the chat model. It lists the candidate pages and counts them, reports no claim count (`claims_extracted` is null and `dry_run` is true in `--json`), and writes nothing. A script that relied on a dry run classifying claims without `--yes` now needs `--dry-run --yes`, which classifies the candidates and still writes nothing.diff --git a/changelog.d/gbra587-silent-failures.md b/changelog.d/gbra587-silent-failures.mdnew file mode 100644index 0000000000..0b6f52d4d7--- /dev/null+++ b/changelog.d/gbra587-silent-failures.md@@ -0,0 +1,9 @@+**Four failures that used to happen silently now either work or report themselves: connector pages on a managed brain get facts again, `takes extract --dry-run` sends nothing to the model unless you also pass `--yes`, turning off mention linking records each removed link in the belief ledger, and a claude-cli call that hits the chat timeout fails only that call.**++<!-- behavior-change: takes_extract_dry_run_needs_yes_9547 -->++### Fixed+- On a managed brain, Slack and meeting pages that a connector publishes are queued for fact extraction again. The connector used to call a queue path that a managed brain refuses, and the refusal was swallowed, so those pages never got facts. Each import's publication now queues the extraction itself and keeps `--no-extract` and the 50-page limit per run. Gmail, calendar and GitHub imports queue no facts on a managed brain, the same as on a classic brain. On a classic brain, a page whose extraction could not be queued is now logged to `ingest_log` and reported on stderr instead of being dropped. Slack pages synced before this fix get their facts from `gbrain extract-conversation-facts --source-id <id> --types slack --yes` (#9552, reported by @lloydsilvertwo).+- `gbrain takes extract --from-pages --dry-run` without `--yes` lists and counts the candidate pages and calls no model, as its "preview" wording promised. It used to send every candidate page to the chat model without `--yes`. Its output says nothing was sent and no claims were classified, and `--json` carries `dry_run: true` with `claims_extracted: null`. `--dry-run --yes` keeps the old preview, which classifies the candidates and writes nothing (#9547, reported and first fixed by @Kkkkkuro).+- Turning off mention linking for a source that the belief ledger covers now writes one `link.retracted` event for each mention link it removes, so the ledger no longer asserts links that are gone. The turn-off and the mention reconcile take a per-source lock in a fixed order, so the two cannot deadlock (#9548, reported and first fixed by @akinduroifedayo).+- When a claude-cli call hits the chat timeout (`GBRAIN_AI_CHAT_TIMEOUT_MS`, 300 s by default), the caller now gets the timeout error instead of "claude-cli adapter aborted". Fact extraction records it as a provider error for that page and moves on, so one hung call no longer ends the whole conversation-facts backfill (#9553, reported by @lloydsilvertwo).diff --git a/llms-full.txt b/llms-full.txtindex 5bbbfcfbff..1fb32683ce 100644--- a/llms-full.txt+++ b/llms-full.txt@@ -2774,6 +2774,8 @@ grants.writing/originals page content to your configured chat model (defaultAnthropic Haiku). Refuses to run unless `takes.bootstrap_enabled=true`is set in config AND `--yes` is passed. Two-gate opt-in by design.+ `--dry-run` without `--yes` only lists the candidate pages and sends+ nothing to the model.- Autopilot's auto-apply tier for takes-bootstrap is `manual_only`(do not bypass).diff --git a/scripts/postgres-arm-weights.json b/scripts/postgres-arm-weights.jsonindex 4f85c99460..ae1a7937c0 100644--- a/scripts/postgres-arm-weights.json+++ b/scripts/postgres-arm-weights.json@@ -46,6 +46,7 @@"test/belief-ledger-w5-time-model.test.ts": 22130,"test/belief-ledger-weekly-diff.test.ts": 102700,"test/belief-ledger-weekly-probes.test.ts": 35614,+ "test/belief-mention-disable-9548.test.ts": 10687,"test/capacity-bounded-read-6803.test.ts": 8904,"test/capture-consent-admission.test.ts": 10088,"test/capture-dispatch-provenance-6524.test.ts": 7201,diff --git a/src/commands/takes.ts b/src/commands/takes.tsindex 6ab67dee2f..363ec5815c 100644--- a/src/commands/takes.ts+++ b/src/commands/takes.ts@@ -685,14 +685,17 @@ Common flags:* lifts gradeable claims into the takes fence.** Two-gate consent: requires `takes.bootstrap_enabled=true` in config- * AND explicit --yes flag for any non-dryRun run. Refuses LLM-bearing- * extraction without both.+ * AND explicit --yes flag for any model call. Refuses LLM-bearing+ * extraction without both. `--dry-run` writes nothing: without --yes it+ * lists the candidate pages and sends nothing to the model; with --yes it+ * also classifies them.*/async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {const sub = rest[0];if (sub !== '--from-pages') {process.stderr.write('Usage: gbrain takes extract --from-pages [--yes] [--dry-run] [--json] [--source-id <id>] [--max-pages N (clamped to 1000)] [--before <updated_at>,<id>] [--include-covered] [--holder <name>]\n' ++ '--dry-run writes nothing. Without --yes it lists the candidate pages and sends nothing to the model; --dry-run --yes also classifies them.\n' +'Runs progress: pages that already hold takes are skipped, so repeat runs sweep a large corpus in slices. Pass a run\'s next_before to --before to continue past pages that yielded no claims. --include-covered rescans everything (refresh).\n',);process.exit(1);@@ -737,7 +740,7 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {}process.stderr.write(`[takes extract] sends concept/atom/lore/briefing/writing/originals page content to ${modelLabel}.\n` +- `Pass --yes to proceed (or --dry-run to preview).\n`,+ `Pass --yes to proceed, or --dry-run to list the candidate pages without sending anything to the model (--dry-run --yes classifies them and writes nothing).\n`,);process.exit(1);}@@ -746,6 +749,7 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {const result = await extractTakesFromPages(engine, {bootstrapEnabled: true,dryRun,+ yes: skipConfirm,sourceIdFilter,maxPages,before,@@ -768,6 +772,16 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);return;}+ // Quoted: the cursor's timestamp contains a space.+ const next = result.next_before ? `next: --before '${result.next_before}'\n` : '';+ if (result.dry_run) {+ process.stdout.write(+ `takes extract --from-pages (dry-run): ${result.pages_scanned} candidate page(s); nothing was sent to the model and no claims were classified. Add --yes to classify them without writing.\n` ++ (result.candidates ?? []).map((c) => ` ${c.slug} (${c.source_id})\n`).join('') ++ next,+ );+ return;+ }// #4473: takes are markdown-canonical — pages the fence writer refused are// skipped (never written DB-only). Say so instead of silently undercounting.const unlocatable = result.skipped.filter((s) => !s.reason.startsWith('llm_error:'));@@ -795,10 +809,8 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {}process.stdout.write(`takes extract --from-pages: ${result.claims_extracted} claim(s) from ${result.pages_scanned} page(s)` +- (dryRun ? ' (dry-run)' : '') + '\n',+ (dryRun ? ' (dry-run)' : '') + '\n' + next,);- // Quoted: the cursor's timestamp contains a space.- if (result.next_before) process.stdout.write(`next: --before '${result.next_before}'\n`);}const BEFORE_CURSOR_RE = /^(\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}:\d{2}(?:\.\d{1,6})?(?:Z|[+-]\d{2}(?::?\d{2})?)?),([1-9]\d{0,15})$/;diff --git a/src/core/ai/providers/claude-cli-language-model.ts b/src/core/ai/providers/claude-cli-language-model.tsindex b3eb52e201..5c8fda4f98 100644--- a/src/core/ai/providers/claude-cli-language-model.ts+++ b/src/core/ai/providers/claude-cli-language-model.ts@@ -508,7 +508,7 @@ function runClaude(const onAbort = () => {child.kill('SIGTERM');- reject(new Error('claude-cli adapter aborted'));+ reject(signal?.reason instanceof Error && signal.reason.name === 'TimeoutError' ? signal.reason : new Error('claude-cli adapter aborted'));};if (signal) {if (signal.aborted) {diff --git a/src/core/connectors/api-source.ts b/src/core/connectors/api-source.tsindex dab0dca8f1..f5dedf9b83 100644--- a/src/core/connectors/api-source.ts+++ b/src/core/connectors/api-source.ts@@ -191,22 +191,35 @@ export async function deleteConnectorPage(ctx: ConnectorPublishCtx, relPath: strif (rows.length) { await engine.deletePages([slug], { sourceId }); summary.deleted++; }}+/** A connector run queues fact extraction for at most this many pages; the cycle's backfill covers the rest. */+export const CONNECTOR_FACTS_PAGE_LIMIT = 50;+/** Facts and embeddings for the pages a run published (the connector path has no sync finalize). */export async function afterConnectorPublish(ctx: Pick<ConnectorPublishCtx, 'engine' | 'sourceId' | 'summary' | 'managed' | 'tag'>, opts: SyncOpts): Promise<void> {const { engine, sourceId, summary, managed } = ctx;if (!summary.pagesAffected.length) return;- if (!opts.noExtract && summary.pagesAffected.length <= 50) {+ const overLimit = summary.pagesAffected.length > CONNECTOR_FACTS_PAGE_LIMIT;+ // #9552: on a managed brain each import's publication queued its own fact extraction (connector-sync.ts).+ if (!opts.noExtract && !overLimit && !managed) {const { runFactsBackstop } = await import('../facts/backstop.ts');+ let failed = 0;+ let firstError = '';for (const slug of summary.pagesAffected) {try {const page = await engine.getPage(slug, { sourceId });if (!page || isSegmentedPage(page)) continue;await runFactsBackstop({ slug, type: page.type, compiled_truth: page.compiled_truth ?? '', frontmatter: page.frontmatter ?? {} },{ engine, sourceId, sessionId: connectorFactsSession(sourceId, slug), source: 'sync:import', mode: 'queue', notabilityFilter: 'high-only', sourceSlug: slug });- } catch { /* per-page enqueue is best-effort, as in sync */ }+ } catch (error) {+ failed++;+ firstError ||= error instanceof Error ? error.message : String(error);+ const { writeFactsAbsorbFailure } = await import('../facts/absorb-log.ts');+ await writeFactsAbsorbFailure(engine, slug, error, sourceId);+ }}- } else if (!opts.noExtract) {- process.stderr.write(`[${ctx.tag}] ${summary.pagesAffected.length} pages changed; facts are extracted by the cycle's backfill\n`);+ if (failed) process.stderr.write(`[${ctx.tag}] ${sourceId}: fact extraction was not queued for ${failed} page(s): ${firstError}\n`);+ } else if (!opts.noExtract && overLimit) {+ process.stderr.write(`[${ctx.tag}] ${summary.pagesAffected.length} pages changed; ${managed ? `facts were queued for the first ${CONNECTOR_FACTS_PAGE_LIMIT}, the rest` : 'facts'} are extracted by the cycle's backfill\n`);}if (managed || opts.noEmbed || summary.pagesAffected.length > 100) return;try {diff --git a/src/core/extract-takes-from-pages.ts b/src/core/extract-takes-from-pages.tsindex d5c84c7367..35cc8dfede 100644--- a/src/core/extract-takes-from-pages.ts+++ b/src/core/extract-takes-from-pages.ts@@ -59,8 +59,13 @@ Max 15 claims per page; output [] if no gradeable claims are present.`;export interface ExtractTakesFromPagesOpts {/** Required: must be true for any work to happen (A12). */bootstrapEnabled: boolean;- /** Dry-run: classify but don't write to takes table. */+ /**+ * Dry-run: write nothing. Without `yes` it only lists the candidate pages+ * and calls no model; with `yes` it also classifies them.+ */dryRun?: boolean;+ /** The user passed --yes: page content may be sent to the chat model. */+ yes?: boolean;/** Scope to a single source. */sourceIdFilter?: string;/** Max pages to classify per run (caps cost). Default 50. */@@ -94,7 +99,12 @@ export interface ExtractTakesFromPagesOpts {export interface ExtractTakesFromPagesResult {pages_scanned: number;- claims_extracted: number;+ /** Null on a dry run without `yes`: no claims were classified. */+ claims_extracted: number | null;+ /** Set on a dry run without `yes`: candidates listed, nothing sent to the model. */+ dry_run?: true;+ /** The candidate pages a dry run without `yes` listed. */+ candidates?: Array<{ slug: string; source_id: string }>;/*** Where the next run should resume, as `<updated_at text>,<page id>`: the* position of the last page this run finished with, or null if it finished@@ -203,7 +213,9 @@ export async function extractTakesFromPages(};}- if (!isAvailable('chat')) {+ const dryRun = opts.dryRun ?? false;+ const listOnly = dryRun && !opts.yes;+ if (!listOnly && !isAvailable('chat')) {return {pages_scanned: 0,claims_extracted: 0,@@ -213,7 +225,6 @@ export async function extractTakesFromPages(};}- const dryRun = opts.dryRun ?? false;const managedJournalWrites = await managedPersistenceEnabled(engine);const maxPages = opts.maxPages ?? 50;const holder = opts.holder ?? 'system';@@ -251,6 +262,19 @@ export async function extractTakesFromPages(params,);+ if (listOnly) {+ return {+ ...emptyTail,+ dry_run: true,+ pages_scanned: pages.length,+ claims_extracted: null,+ next_before: pages.at(-1)?.resume_at ?? null,+ consent_gate_blocked: false,+ llm_unavailable: false,+ candidates: pages.map((page) => ({ slug: page.slug, source_id: page.source_id })),+ };+ }+let pagesScanned = 0;let claimsExtracted = 0;let pagesSkipped = 0;diff --git a/src/core/mentions/pass.ts b/src/core/mentions/pass.tsindex ab0079820e..0419dce93f 100644--- a/src/core/mentions/pass.ts+++ b/src/core/mentions/pass.ts@@ -184,6 +184,15 @@ async function writeStatus(engine: Pick<BrainEngine, 'executeRaw'>, sourceId: stpatch.fingerprint ?? null, patch.fingerprint !== undefined]);}+/**+ * #9548: a disable (exclusive) and each reconcile batch reading the source's generation (shared) take this first, before page+ * keys or status rows: a journaled disable locks its origin pages, which a cross-source batch holding the status row links to.+ */+async function lockMentionSources(tx: Pick<BrainEngine, 'executeRaw'>, sourceIds: string[], shared: boolean): Promise<void> {+ await tx.executeRaw(`SELECT count(${shared ? 'pg_advisory_xact_lock_shared' : 'pg_advisory_xact_lock'}(hashtext('gbrain:mention-publish:' || s)::bigint))+ FROM (SELECT DISTINCT s FROM unnest($1::text[]) AS x(s) ORDER BY s) k`, [sourceIds]);+}+/** Run `fn` in a transaction; on a managed brain, as a coordinated write for `sourceId` (page_aliases is guarded). */async function aliasWrite<T>(engine: BrainEngine, managed: boolean, sourceId: string, fn: (tx: BrainEngine) => Promise<T>): Promise<T> {if (!managed) return engine.transaction(fn);@@ -235,18 +244,26 @@ export async function writePageAliases(tx: BrainEngine, slug: string, sourceId:/** Turn the index off for one source: plain mention links, derived aliases, entries and page state go; typed_ner rows stay. */async function disableSource(engine: BrainEngine, managed: boolean, sourceId: string): Promise<number> {- let removed = 0;- await aliasWrite(engine, managed, sourceId, async tx => {- const rows = await tx.executeRaw<{ id: number }>(+ let beliefs = null as ReturnType<typeof unmanagedBeliefPoint> | null;+ const removed = await aliasWrite(engine, managed, sourceId, async tx => {+ await lockMentionSources(tx, [sourceId], false);+ // #9548: on a belief-covered source the removed links are journaled like the reconcile's; only origins whose page keys are held are swept.+ const origins = (await beliefCoveredSources(tx, [sourceId])).has(sourceId) ? (await tx.executeRaw<{ slug: string }>(+ `SELECT DISTINCT f.slug FROM links l JOIN pages f ON f.id = l.from_page_id+ WHERE f.source_id = $1 AND l.link_source = 'mentions' AND (l.link_kind IS NULL OR l.link_kind = 'plain')`, [sourceId])).map(r => r.slug) : null;+ const unlink = async () => (await tx.executeRaw<{ id: number }>(`DELETE FROM links l USING pages fWHERE f.id = l.from_page_id AND f.source_id = $1 AND l.link_source = 'mentions'- AND (l.link_kind IS NULL OR l.link_kind = 'plain') RETURNING l.id`, [sourceId]);- removed = rows.length;+ AND (l.link_kind IS NULL OR l.link_kind = 'plain') AND ($2::text[] IS NULL OR f.slug = ANY($2::text[])) RETURNING l.id`, [sourceId, origins])).length;+ if (origins?.length) beliefs = unmanagedBeliefPoint(sourceId, origins, 'mention_pass');+ const unlinked = beliefs ? await beliefs.run(tx, unlink) : await unlink();await tx.executeRaw(`DELETE FROM page_aliases WHERE source_id = $1 AND origin IN ('declared','subject')`, [sourceId]);await tx.executeRaw('DELETE FROM mention_gazetteer_entries WHERE source_id = $1', [sourceId]);await tx.executeRaw('DELETE FROM page_mention_state WHERE source_id = $1', [sourceId]);await writeStatus(tx, sourceId, { state: 'disabled', pending: 0, error: null, bumpGeneration: true, fingerprint: null });+ return unlinked;});+ await beliefs?.commit(engine);return removed;}@@ -555,6 +572,7 @@ async function reconcileSource(engine: BrainEngine, sourceId: string, gazetteer:// page keys are locked first, as every publisher locks them before the page rows.let beliefs = null as ReturnType<typeof unmanagedBeliefPoint> | null;const outcome = await engine.transaction(async tx => {+ await lockMentionSources(tx, genSources, true);const journaled = (await beliefCoveredSources(tx, [sourceId])).has(sourceId);if (journaled) await tx.lockPageKeys(readable.map(p => ({ sourceId: p.source_id, slug: p.slug })));if (!sameGenerations(generations, await readGenerations(tx, genSources, true))) return null;diff --git a/src/core/persistence/connector-sync.ts b/src/core/persistence/connector-sync.tsindex 9521bda305..2ebed8f1c1 100644--- a/src/core/persistence/connector-sync.ts+++ b/src/core/persistence/connector-sync.ts@@ -51,6 +51,8 @@ import { readConnectorV2Cutoff } from './connector-checkpoint-migration.ts';import { declarePersistenceProtocol } from './protocol.ts';import { intentCarriesPurgedContent } from './purged-intent.ts';import { PERSISTENCE_IPC_MAX_BYTES } from './ipc.ts';+import { prepareFactsBackstop, type FactsBackstopStatus } from './effect-facts.ts';+import { CONNECTOR_FACTS_PAGE_LIMIT } from '../connectors/api-source.ts';import type { GoogleSourceConfig } from '../google/types.ts';interface ConnectorLease { handle: DbLockHandle; signal: AbortSignal; }@@ -136,6 +138,8 @@ interface ConnectorIntent extends Record<string, unknown> {googlePageId?: number;noEmbed: boolean;noSchemaPack: boolean;+ /** #9552: a Slack or meetings import queues no automatic fact extraction (`--no-extract`, or past the run's limit). Not part of the request identity. */+ noExtract?: true;checkpointKey: string;checkpointBefore: unknown[];checkpointAfter?: unknown[];@@ -267,6 +271,11 @@ async function strictPrivateFile(engine: BrainEngine, row: Pick<WriteRequest, 'sreturn { file: turning ? await preparePrivateTransition(engine, row, snapshot) : undefined };}+/** #9552: the connectors whose imports queue fact extraction, as an unmanaged sync does; never Google mail or GitHub. */+function factsConnector(connector: ConnectorKind): boolean {+ return connector === 'slack' || connector === 'meetings';+}+export async function beginConnectorSync(engine: BrainEngine, sourceId: string, connector: ConnectorKind,suppliedConfig: unknown, opts: SyncOpts & { resetCheckpoint?: boolean }, lease?: ConnectorLease): Promise<ManagedConnectorSync | null> {if (!await readManagedMode(engine)) return null;@@ -304,6 +313,7 @@ export async function beginConnectorSync(engine: BrainEngine, sourceId: string,const canonicalRoot = connectorBindingRoot(engine, sourceId, source, binding);const session = new ManagedConnectorSync(engine, sourceId, identity, source, authority, binding, canonicalRoot, opts.noEmbed === true, opts.noSchemaPack === true,opts.retryFailed === true, lease, opts.resetCheckpoint === true, opts.githubItem !== undefined, opts.full === true);+ session.extractFacts = opts.noExtract !== true;await session.load();return session;}@@ -404,6 +414,9 @@ export class ManagedConnectorSync {/** The stable key plus its #6561 alias keys: retry approvals recorded under an alias still apply. */private readonly checkpointKeys: string[];private readonly connector: ConnectorKind;+ /** #9552: whether this run queues fact extraction for the pages it imports, and how many imports have queued it. */+ extractFacts = true;+ private factsOffered = 0;constructor(private engine: BrainEngine, readonly sourceId: string, private identity: ConnectorIdentity,private source: ConnectorSource, private authority: SyncAuthority, private binding: WorktreeBinding | null,private canonicalRoot: string | null, private noEmbed: boolean, private noSchemaPack: boolean, private retryFailed = false,@@ -957,8 +970,9 @@ export class ManagedConnectorSync {sourcePath, noEmbed: this.noEmbed, noSchemaPack: this.noSchemaPack, checkpointKey: this.checkpointKey,checkpointBefore: kind === 'connector_v2_checkpoint' ? this.checkpointRow : cursorOnly(this.checkpointRow),ownerEpoch: this.binding ? String(this.binding.owner_epoch) : null, canonicalRoot: this.canonicalRoot,- filePath: file?.path ?? null, fileBeforeHash: file?.expectedBeforeHash ?? null, ...extra };- const callerIntent = { ...intent, syncAuthority: undefined, newestContentAt: undefined,+ filePath: file?.path ?? null, fileBeforeHash: file?.expectedBeforeHash ?? null, ...extra,+ ...(kind === 'connector_v2_import' && factsConnector(this.connector) && (!this.extractFacts || this.factsOffered >= CONNECTOR_FACTS_PAGE_LIMIT) ? { noExtract: true as const } : {}) };+ const callerIntent = { ...intent, syncAuthority: undefined, newestContentAt: undefined, noExtract: undefined,...(kind === 'connector_v2_checkpoint' ? { receipts: undefined } : {}) };const principal = this.authority.writer.principal;const baseRequestId = stableId({ principal, sourceId: this.sourceId, incarnation: this.source.incarnation, callerIntent });@@ -1067,6 +1081,7 @@ export class ManagedConnectorSync {}if (admitted) {if (page) this.counts.page_admissions++;+ if (kind === 'connector_v2_import' && factsConnector(this.connector) && !intent.noExtract) this.factsOffered++;if (kind === 'connector_v2_checkpoint') this.counts.checkpoint_admissions++;}if (kind === 'connector_v2_google_receipts') {@@ -1225,6 +1240,9 @@ export async function prepareConnectorMutation(engine: BrainEngine, row: WriteReif (ready.observedRevision !== (snapshot?.revision ?? null)) throw connectorPublicationRefusal('revision_conflict', 'The connector page changed during preparation.', row,`Page ${row.slug} in ${row.source_id} changed while this connector write was being prepared.`);const project = await prepareCanonicalProjections(engine, ready.parsedPage, row.slug, row.source_id, snapshot, 'preserving');+ // #9552: the publication outbox queues the page's fact extraction, as it does for a page write.+ const facts = ready.noop || p.noExtract || !factsConnector(p.connector) ? undefined+ : await prepareFactsBackstop(engine, row, ready.parsedPage, snapshot).catch((): FactsBackstopStatus => ({ skipped: 'backstop_error' }));const tags = [...new Set([...(snapshot?.tags ?? []), ...ready.parsedPage.tags])].sort();const page: Page = { ...(snapshot?.page ?? { id: 0, slug: row.slug, source_id: row.source_id, created_at: new Date(row.created_at), updated_at: new Date(row.created_at) }), ...ready.parsedPage };const strictPrivate = await strictPrivateFile(engine, row, p.content, p.sourcePath, snapshot);@@ -1242,7 +1260,7 @@ export async function prepareConnectorMutation(engine: BrainEngine, row: WriteRe// A body stored in segments was sealed by its import's apply when complete (sealSegmentedPage); the inline text seal reads the whole timeline.if (!ready.noop) { await project(tx); if (!installed?.segmented) await sealPageTextProjection(tx, row.slug, row.source_id); }return { status: ready.noop ? 'skipped' : snapshot ? 'updated' : 'created', slug: row.slug, source_id: row.source_id,- chunks: result.chunks, noop: ready.noop, imported_file: true, connector_database: !row.worktree_id };+ chunks: result.chunks, noop: ready.noop, imported_file: true, connector_database: !row.worktree_id, ...(facts ? { facts_backstop: facts } : {}) };} };}diff --git a/src/core/persistence/effect-facts.ts b/src/core/persistence/effect-facts.tsindex 8fcc0190c1..de3a5d1592 100644--- a/src/core/persistence/effect-facts.ts+++ b/src/core/persistence/effect-facts.ts@@ -10,6 +10,8 @@ import type { PageSnapshot } from '../page-state/types.ts';import { OperationError, opError } from '../ops/contract.ts';import { readFix } from '../ops/op-fix.ts';import { readManagedMode } from './managed-mode.ts';+import { isConnectorIntentKind } from './connector-identity.ts';+import { connectorFactsSession } from '../facts/retire.ts';import { authorizeStoredRequest, authorizeWrite } from './authority.ts';import { completeEffect } from './effect-journal.ts';import { guardEffectSource } from './effect-recovery.ts';@@ -150,9 +152,13 @@ export async function dispatchFactsBackstopEffect(engine: BrainEngine, effect: Pif (skipped) { await completeEffect(tx, effect, { facts: 'skipped', reason: skipped }); return; }// #6231: the page-write filter in force when the effect dispatches; the job keeps it across retries.const notabilityFilter = await resolvePageWriteNotabilityFilter(tx);+ // #9552: a connector page keeps the connector path's session (its facts are retired with the item) and its high-only filter.+ const connector = isConnectorIntentKind(row!.intent?.kind);const job = await new MinionQueue(tx).add('facts-absorb', {// #6333: a stable per-page session (mirrors sync's `sync:<slug>`) so a re-extraction feeds the same-page shadow counter.- slug: effect.data.slug, sourceId: effect.source_id, source: 'mcp:put_page', sessionId: `put_page:${effect.data.slug}`, notabilityFilter,+ slug: effect.data.slug, sourceId: effect.source_id, source: connector ? 'sync:import' : 'mcp:put_page',+ sessionId: connector ? connectorFactsSession(effect.source_id, effect.data.slug!) : `put_page:${effect.data.slug}`,+ notabilityFilter: connector ? 'high-only' : notabilityFilter,visibility: effect.data.visibility === 'world' ? 'world' : 'private',persistence_request_id: effect.request_id, page_id: effect.data.page_id, revision: effect.revision,}, { queue: 'default', idempotency_key: factsAbsorbWriteKey(effect.request_id), max_attempts: 5, backoff_delay: 60_000 });diff --git a/test/belief-mention-disable-9548.test.ts b/test/belief-mention-disable-9548.test.tsnew file mode 100644index 0000000000..f084229869--- /dev/null+++ b/test/belief-mention-disable-9548.test.ts@@ -0,0 +1,162 @@+/**+ * #9548: turning mention linking off on a belief-covered source removes the plain mention links the pass wrote.+ *+ * 1. Protects: each removed link is journaled as an unmanaged `link.retracted` (op `mention_pass`), so the ledger's+ * last state for the origin page matches its stored links and the drift scan finds nothing.+ * 2. Fails when: the disable step deletes the links outside a belief batch: the ledger still asserts them and the+ * origin page reads as drift.+ * 3. Why existing coverage misses it: belief-stale-links-7031 journals the pass's reconcile writes; no probe runs the+ * disable branch on a covered source.+ */+import { describe, expect, test } from 'bun:test';+import type { BrainEngine } from '../src/core/engine.ts';+import { enableBeliefLedger } from '../src/core/belief/coverage.ts';+import { scanBeliefDrift } from '../src/core/belief/coverage-report.ts';+import { runMentionPass } from '../src/core/mentions/pass.ts';+import { maintenanceAttribution } from '../src/core/persistence/attribution.ts';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { beliefBrain, beliefEngineCase, beliefPage, type BeliefBrain } from './helpers/belief-ledger.ts';+import { testBackends } from './helpers/test-backends.ts';+import { waitFor } from './helpers/wait-for.ts';++const origin = 'companies/acme-example', target = 'people/alice-example';++for (const engineCase of testBackends().map(beliefEngineCase)) {+ describe('#9548 mention disable on a belief-covered source (' + engineCase.name + ')', () => {+ test('disabling mention linking journals each removed plain mention link and leaves no drift', async () => {+ await beliefBrain(async b => {+ await b.put(target, beliefPage({ type: 'person', title: 'Alice Example' }));+ await b.put(origin, beliefPage({ prose: 'Acme Example met Alice Example last week.' }));+ expect(await runMentionPass(b.engine, { sourceId: b.sourceId })).toMatchObject({ created: 1 });+ const asserted = (await b.events()).findLast(e => e.subject_kind === 'link')!;+ expect([asserted.slug, asserted.event_type, asserted.cause.op]).toEqual([origin, 'link.asserted', 'mention_pass']);+ expect((await scanBeliefDrift(b.engine, b.sourceId, { deep: true })).drifted).toEqual([]);+ const before = await b.lastId();+ await b.engine.setConfig('mentions.auto_link', 'false');+ expect(await runMentionPass(b.engine, { sourceId: b.sourceId })).toMatchObject({ removed: 1, state: 'disabled' });+ expect(await b.engine.executeRaw("SELECT 1 FROM links WHERE link_source = 'mentions'")).toEqual([]);+ const events = await b.events(before);+ expect(events.map(e => [e.slug, e.subject_key, e.event_type, e.origin, e.cause.op]))+ .toEqual([[origin, asserted.subject_key, 'link.retracted', 'unmanaged', 'mention_pass']]);+ const publications = [...new Set(events.map(e => e.publication_id))];+ expect(await b.engine.executeRaw('SELECT 1 FROM belief_commit_marks WHERE publication_id = ANY($1::uuid[])', [publications]))+ .toHaveLength(publications.length);+ expect((await scanBeliefDrift(b.engine, b.sourceId, { deep: true })).drifted).toEqual([]);+ }, { managed: true, databaseUrl: engineCase.databaseUrl });+ }, 180_000);++ if (engineCase.name === 'postgres') {+ const other = 'source-b', person = 'people/alice-example', colleague = 'people/bob-example', local = 'notes/source-a-note';+ const pid = async (db: Pick<BrainEngine, 'executeRaw'>) => Number((await db.executeRaw<{ pid: number }>('SELECT pg_backend_pid() AS pid'))[0]!.pid);+ const blockedBy = async (b: BeliefBrain, holder: number) => (await b.engine.executeRaw<{ pid: number }>(+ 'SELECT pid FROM pg_stat_activity WHERE datname = current_database() AND $1::int = ANY(pg_blocking_pids(pid))', [holder])).map(r => Number(r.pid));+ // The engine's transactions run `hook` on each statement of the wrapped tx, so a test can hold one mid-transaction.+ const pausing = (engine: BrainEngine, hook: (tx: BrainEngine, member: string, args: unknown[]) => Promise<void>) => new Proxy(engine, { get(target, key) {+ if (key === 'transaction') return <T>(run: (tx: BrainEngine) => Promise<T>) => target.transaction(tx => run(new Proxy(tx, { get(inner, member) {+ const value = Reflect.get(inner, member);+ if (typeof value !== 'function') return value;+ return async (...args: unknown[]) => { await hook(inner, String(member), args); return value.apply(inner, args); };+ } })));+ const value = Reflect.get(target, key);+ return typeof value === 'function' ? value.bind(target) : value;+ } });+ const twoSources = async (b: BeliefBrain) => {+ await b.engine.setConfig('link_resolution.cross_source', 'true');+ await b.engine.transaction(async tx => {+ await tx.executeRaw("SELECT set_config('gbrain.topology_change','on',true)");+ await tx.executeRaw('INSERT INTO sources(id,name) VALUES($1,$1)', [other]);+ });+ const actor = await maintenanceAttribution(b.engine);+ await b.engine.transaction(tx => withCoordinatedWrite(tx, [other], async () => {+ await tx.putPage(person, { type: 'person', title: 'Alice Example', compiled_truth: 'Works with Bob Example.' }, { sourceId: other });+ await tx.putPage(colleague, { type: 'person', title: 'Bob Example', compiled_truth: 'A synthetic person.' }, { sourceId: other });+ }, actor));+ await enableBeliefLedger(b.engine, other);+ await b.put(local, beliefPage({ type: 'note', title: 'Source A note', prose: 'No names yet.' }));+ expect(await runMentionPass(b.engine, { sourceId: other })).toMatchObject({ state: 'complete', created: 1 });+ expect(await runMentionPass(b.engine, { sourceId: b.sourceId })).toMatchObject({ state: 'complete', created: 0 });+ await b.put(local, beliefPage({ type: 'note', title: 'Source A note', prose: 'Met Alice Example.' }));+ await disposePersistenceConsumer(b.engine);+ };+ // Neither pass fails (no 40P01), the disable removes source B's one link, and only source A's mention can remain.+ const settle = async (b: BeliefBrain, passes: Array<Promise<unknown>>) => {+ const results = await Promise.allSettled(passes);+ const failures = results.flatMap(r => r.status === 'rejected' ? [{ code: (r.reason as { code?: string })?.code, message: String((r.reason as Error)?.message) }] : []);+ expect(failures).toEqual([]);+ const [reconciled, disabled] = results.map(r => (r as PromiseFulfilledResult<{ created: number }>).value);+ expect(disabled).toMatchObject({ state: 'disabled', removed: 1 });+ const remaining = await b.engine.executeRaw(+ "SELECT f.source_id, f.slug, t.source_id AS target_source, t.slug AS target FROM links l JOIN pages f ON f.id = l.from_page_id JOIN pages t ON t.id = l.to_page_id WHERE l.link_source = 'mentions'");+ expect(remaining).toEqual(reconciled!.created ? [{ source_id: b.sourceId, slug: local, target_source: other, target: person }] : []);+ return reconciled!;+ };++ test('a disable that waits behind a cross-source reconcile holding its status row completes without deadlock', async () => {+ await beliefBrain(async b => {+ await twoSources(b);+ const reached = Promise.withResolvers<number>(), release = Promise.withResolvers<void>();+ let disablePid = 0;+ const reconciling = pausing(b.engine, async (tx, member, args) => {+ if (member !== 'addLinksBatch' || !(args[0] as Array<{ to_source_id?: string }>).some(r => r.to_source_id === other)) return;+ reached.resolve(await pid(tx));+ await release.promise;+ });+ const disabling = pausing(b.engine, async (tx, member, args) => {+ if (member === 'executeRaw' && String(args[0]).includes('gbrain.write_sources') && !disablePid) disablePid = await pid(tx);+ });+ const watchdog = setTimeout(() => release.resolve(), 30_000);+ const reconcile = runMentionPass(reconciling, { sourceId: b.sourceId });+ void reconcile.catch(error => reached.reject(error));+ let disable: Promise<unknown> | undefined;+ try {+ const holder = await reached.promise;+ await b.engine.setConfig('mentions.auto_link', 'false');+ disable = runMentionPass(disabling, { sourceId: other });+ void disable.catch(() => undefined);+ await waitFor(async () => disablePid > 0 && (await blockedBy(b, holder)).includes(disablePid),+ { timeoutMs: 20_000, label: 'the disable waits on the reconcile transaction' });+ } finally {+ release.resolve();+ clearTimeout(watchdog);+ }+ expect(await settle(b, [reconcile, disable!])).toMatchObject({ state: 'complete', created: 1 });+ }, { managed: true, databaseUrl: engineCase.databaseUrl });+ }, 180_000);++ test('a cross-source reconcile that runs while the disable is mid-transaction completes without deadlock', async () => {+ await beliefBrain(async b => {+ await twoSources(b);+ const scanned = Promise.withResolvers<void>(), publish = Promise.withResolvers<void>();+ const deleted = Promise.withResolvers<number>(), release = Promise.withResolvers<void>();+ const disabling = pausing(b.engine, async (tx, member, args) => {+ if (member !== 'executeRaw' || !String(args[0]).includes('INSERT INTO mention_index_status')) return;+ deleted.resolve(await pid(tx));+ await release.promise;+ });+ const watchdog = setTimeout(() => { publish.resolve(); release.resolve(); }, 30_000);+ const reconcile = runMentionPass(b.engine, { sourceId: b.sourceId, beforePublish: async () => { scanned.resolve(); await publish.promise; } });+ void reconcile.catch(error => scanned.reject(error));+ let disable: Promise<unknown> | undefined;+ let settled = false;+ try {+ await scanned.promise;+ await b.engine.setConfig('mentions.auto_link', 'false');+ disable = runMentionPass(disabling, { sourceId: other });+ void disable.catch(error => deleted.reject(error));+ const holder = await deleted.promise;+ publish.resolve();+ void reconcile.finally(() => { settled = true; }).catch(() => undefined);+ await waitFor(async () => settled || (await blockedBy(b, holder)).length > 0,+ { timeoutMs: 20_000, label: 'the reconcile publishes or waits on the disable transaction' });+ } finally {+ publish.resolve();+ release.resolve();+ clearTimeout(watchdog);+ }+ await settle(b, [reconcile, disable!]);+ }, { managed: true, databaseUrl: engineCase.databaseUrl });+ }, 180_000);+ }+ });+}diff --git a/test/claude-cli-hung-call-9553.test.ts b/test/claude-cli-hung-call-9553.test.tsnew file mode 100644index 0000000000..d59a1bbd11--- /dev/null+++ b/test/claude-cli-hung-call-9553.test.ts@@ -0,0 +1,121 @@+/**+ * #9553: a `claude` child that never answers. The stub at+ * GBRAIN_CLAUDE_CLI_BIN reads stdin, records its pid, then sleeps for ten+ * minutes without printing an envelope.+ *+ * The calls go through extractFactsFromTurnWithOutcome, the function the+ * conversation-facts backfill uses (conversation-window-extract.ts passes the+ * backfill's per-source deadline signal as `abortSignal`), and from there+ * through gateway chat() to ClaudeCliLanguageModel.+ */+import { afterAll, afterEach, beforeAll, describe, expect, test } from 'bun:test';+import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import { tmpdir } from 'node:os';+import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';+import { extractFactsFromTurnWithOutcome } from '../src/core/facts/extract.ts';+import { withEnv } from './helpers/with-env.ts';++const MODEL = 'claude-cli:claude-sonnet-4-6';+const RACE_MS = 3_000;+const stubDir = join(tmpdir(), `claude-cli-hung-9553-${process.pid}`);+const stubBin = join(stubDir, 'claude');+const pidPath = join(stubDir, 'pid');+const src = (p: string) => JSON.stringify(join(import.meta.dir, '../src', p));++const alive = (pid: number) => { try { process.kill(pid, 0); return true; } catch { return false; } };+const sleep = (ms: number) => new Promise(r => setTimeout(r, ms));+const stubPid = () => (existsSync(pidPath) ? Number(readFileSync(pidPath, 'utf8').trim()) : 0);++beforeAll(() => {+ mkdirSync(stubDir, { recursive: true });+ writeFileSync(stubBin, [+ '#!/bin/sh',+ 'if [ "$1" = "--version" ]; then echo "2.1.316 (Claude Code)"; exit 0; fi',+ `echo $$ > "${pidPath}"`,+ 'cat > /dev/null',+ 'exec sleep 600',+ ].join('\n') + '\n');+ chmodSync(stubBin, 0o755);+});+afterEach(() => {+ const pid = stubPid();+ if (pid && alive(pid)) process.kill(pid, 'SIGKILL');+ rmSync(pidPath, { force: true });+ resetGateway();+});+afterAll(() => rmSync(stubDir, { recursive: true, force: true }));++async function race(abortSignal?: AbortSignal): Promise<{ settled: 'timeout' | 'resolved' | 'rejected'; ms: number; childAlive: boolean }> {+ configureGateway({ chat_model: MODEL, env: {} });+ return withEnv({ GBRAIN_CLAUDE_CLI_BIN: stubBin }, async () => {+ const started = Date.now();+ const call = extractFactsFromTurnWithOutcome({+ turnText: 'alice-example said acme-example closed its seed round last week.',+ source: 'test:9553',+ model: MODEL,+ abortSignal,+ }).then(() => 'resolved' as const, () => 'rejected' as const);+ const settled = await Promise.race([call, sleep(RACE_MS).then(() => 'timeout' as const)]);+ const ms = Date.now() - started;+ await sleep(200);+ const pid = stubPid();+ return { settled, ms, childAlive: pid > 0 && alive(pid) };+ });+}++/**+ * GBRAIN_AI_CHAT_TIMEOUT_MS is read once when gateway.ts loads, so the+ * shortened default backstop runs in a fresh bun process.+ */+function extractUnderOneSecondBackstop(model: string, gatewayExtra: string, preamble = ''): { ms: number; ok?: boolean; reason?: string; error?: string; thrown?: string } {+ const script = [+ `import { configureGateway } from ${src('core/ai/gateway.ts')};`,+ `import { extractFactsFromTurnWithOutcome } from ${src('core/facts/extract.ts')};`,+ preamble,+ `configureGateway({ chat_model: ${JSON.stringify(model)}, env: {}${gatewayExtra} });`,+ 'const t = Date.now();',+ `const out = await extractFactsFromTurnWithOutcome({ turnText: 'alice-example joined acme-example.', source: 'test:9553', model: ${JSON.stringify(model)} }).then(o => o, e => ({ thrown: String(e) }));`,+ 'console.log(JSON.stringify({ ms: Date.now() - t, ok: out.ok, reason: out.reason, error: out.error ? String(out.error) : undefined, thrown: out.thrown }));',+ 'process.exit(0);',+ ].join('\n');+ const r = Bun.spawnSync([process.execPath, '-e', script], {+ env: { ...process.env, GBRAIN_CLAUDE_CLI_BIN: stubBin, GBRAIN_AI_CHAT_TIMEOUT_MS: '1000' },+ timeout: 15_000,+ });+ const line = r.stdout.toString().trim().split('\n').pop() ?? '';+ console.log(`[${model}] outcome under a 1 s chat backstop: ${line}`);+ return JSON.parse(line);+}++describe('claude-cli call whose child never answers (#9553)', () => {+ test('a caller deadline (the backfill per-source walltime signal) cuts the call off mid-flight and kills the child', async () => {+ const controller = new AbortController();+ const timer = setTimeout(() => controller.abort(new Error('conversation_facts_backfill: per-source walltime cap (1min) hit for source-a')), 1_000);+ const r = await race(controller.signal);+ clearTimeout(timer);+ expect(r.settled).toBe('rejected');+ expect(r.ms).toBeLessThan(RACE_MS);+ expect(r.childAlive).toBe(false);+ }, 10_000);++ test('the default chat backstop cuts a claude-cli call off and records it as a failed call, like an HTTP provider', async () => {+ const out = extractUnderOneSecondBackstop(MODEL, '');+ await sleep(200);+ expect(out.ms).toBeLessThan(RACE_MS);+ expect(alive(stubPid())).toBe(false);+ expect(out.thrown).toBeUndefined();+ expect(out.reason).toBe('provider_error');+ }, 20_000);++ test('control: an HTTP provider that never answers is recorded as provider_error at the same backstop', () => {+ const out = extractUnderOneSecondBackstop(+ 'ollama:llama3.1',+ `, base_urls: { ollama: 'http://127.0.0.1:' + server.port + '/v1' }`,+ 'const server = Bun.serve({ port: 0, fetch: () => new Promise(() => {}) });',+ );+ expect(out.ms).toBeLessThan(RACE_MS);+ expect(out.thrown).toBeUndefined();+ expect(out.reason).toBe('provider_error');+ }, 20_000);+});diff --git a/test/connector-facts-managed-9552.test.ts b/test/connector-facts-managed-9552.test.tsnew file mode 100644index 0000000000..c302bf7d1f--- /dev/null+++ b/test/connector-facts-managed-9552.test.ts@@ -0,0 +1,188 @@+/**+ * #9552: on a managed brain, pages a connector publishes (a Slack day page, a meeting page) are queued for+ * automatic fact extraction through the publication outbox, the way a page write is, and the queued work+ * extracts facts. A connector run never calls a direct queue backstop on a managed brain (that throws+ * `writer_coordinator_required`).+ */+import { afterAll, beforeAll, expect, test } from 'bun:test';+import type { BrainEngine } from '../src/core/engine.ts';+import { createConnectorFixture, json, options, settleConnectorWrites, withGoogleAccount } from './helpers/connector-fixture.ts';+import { addThread, fakeGmail, gmailFetch } from './helpers/connector-holds-fixture.ts';+import { parseGoogleSourceConfig, runGoogleSync } from '../src/core/google/google-source.ts';+import { slackFetch, slackTs, type FakeSlack, slackTestToken } from './helpers/slack-fixture.ts';+import { withEnv } from './helpers/with-env.ts';+import { waitFor } from './helpers/wait-for.ts';+import { runSlackSync } from '../src/core/slack/slack-source.ts';+import { parseSlackSourceConfig } from '../src/core/slack/source-config.ts';+import { slackApi } from '../src/core/slack/client.ts';+import { runMeetingsSync } from '../src/core/meetings/meetings-source.ts';+import { parseMeetingsSourceConfig } from '../src/core/meetings/source-config.ts';+import { firefliesProvider } from '../src/core/meetings/providers/fireflies.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { configureGateway, resetGateway, __setChatTransportForTests, type ChatResult } from '../src/core/ai/gateway.ts';+import { runFactsDrain } from '../src/core/facts/drain.ts';+import { runPhaseConversationFactsBackfill } from '../src/core/cycle/conversation-facts-backfill.ts';++const { engines, env, setup, teardown, source } = createConnectorFixture();+beforeAll(setup, 120_000);+afterAll(async () => { __setChatTransportForTests(null); resetGateway(); await teardown(); });++const KEYS = { ANTHROPIC_API_KEY: 'sk-test-9552', OPENAI_API_KEY: 'sk-test-9552' };+const MODEL = 'anthropic:claude-sonnet-4-6';+const DAY = 86_400_000;+const NOW = Date.now();+const OWNER = '[email protected]';+const CLAIM = 'Acme Example signed a three-year renewal with Beta Example worth two million dollars in October.';+const extract = { ...options, noExtract: false };++const workspace = (): FakeSlack => ({ teamId: 'TACME00001', users: { UALICE0001: { name: 'Alice Example' } }, calls: [], channels: [+ { id: 'CGENERAL01', name: 'general', messages: [{ ts: slackTs(NOW - DAY), user: 'UALICE0001', text: `${CLAIM} The renewal closes the account review for the year.` }] }] });+const meetings = [{ id: 'mtg-9552a', title: 'Renewal review', date: NOW - DAY }];+const fetchImpl = async (_url: string, init: RequestInit) => {+ const { query, variables } = JSON.parse(String(init.body)) as { query: string; variables: Record<string, unknown> };+ if (query.includes('GbrainUser')) return json({ data: { user: { user_id: 'u-9552', email: OWNER, name: 'Owner Example' } } });+ if (query.includes('GbrainTranscripts')) return json({ data: { transcripts: Number(variables.skip) > 0 ? [] : meetings.map(m => ({ id: m.id, title: m.title, date: m.date })) } });+ const m = meetings.find(x => x.id === variables.id)!;+ return json({ data: { transcript: { id: m.id, title: m.title, date: m.date, duration: 30, organizer_email: OWNER, participants: [OWNER],+ meeting_attendees: [{ email: OWNER, displayName: 'Owner Example' }], summary: { overview: `${CLAIM} The team agreed the pricing terms and the next review date.`, action_items: [], keywords: [] } } } });+};++async function managedSource(engine: BrainEngine, id: string, config: Record<string, unknown>) {+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,NULL,$2::text::jsonb)', [id, JSON.stringify(config)]);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+}+const count = async (engine: BrainEngine, sql: string, params: unknown[]) => Number((await engine.executeRaw<{ n: number }>(sql, params))[0]?.n ?? 0);+const queuedExtractions = (engine: BrainEngine, sourceId: string) => count(engine,+ `SELECT count(*)::int AS n FROM minion_jobs WHERE name='facts-absorb' AND data->>'sourceId'=$1`, [sourceId]);+const openEffects = (engine: BrainEngine, sourceId: string) => count(engine,+ `SELECT count(*)::int AS n FROM persistence_effects WHERE source_id=$1 AND kind='facts-backstop' AND state IN ('queued','running')`, [sourceId]);+const factRows = (engine: BrainEngine, sourceId: string) => count(engine,+ "SELECT count(*)::int AS n FROM facts WHERE source_id=$1 AND expired_at IS NULL AND fact LIKE '%three-year renewal%'", [sourceId]);++async function captureStderr<T>(run: () => Promise<T>): Promise<string> {+ const original = process.stderr.write.bind(process.stderr);+ let stderr = '';+ process.stderr.write = ((chunk: string | Uint8Array, ...rest: unknown[]) => { stderr += String(chunk); return original(chunk as never, ...rest as never[]); }) as typeof process.stderr.write;+ try { await run(); return stderr; } finally { process.stderr.write = original; }+}++test('managed brain: a Slack page and a meeting page a connector publishes are queued for facts, and the queued work extracts them', async () => withEnv({ ...env, ...KEYS }, async () => {+ for (const engine of engines) {+ configureGateway({ embedding_disabled: true, env: KEYS } as never);+ await engine.setConfig('facts.extraction_model', MODEL);+ await engine.setConfig('embedding_disabled', 'true');++ const slackConfig = { kind: 'slack', s_channels: 'CGENERAL01', s_record: 'database', federated: false };+ await managedSource(engine, 'slack-9552', slackConfig);+ let stderr = await captureStderr(() => runSlackSync(engine, 'slack-9552', parseSlackSourceConfig(slackConfig, ''), extract,+ { api: slackApi({ token: slackTestToken(), userAgent: 'gbrain-test', fetchImpl: slackFetch(workspace()), sleep: async () => {}, minDelayMs: 0 }) }));+ await settleConnectorWrites(engine, 'slack-9552');++ const meetingConfig = { kind: 'meetings', m_provider: 'fireflies', m_record: 'database', federated: false };+ await managedSource(engine, 'meet-9552', meetingConfig);+ stderr += await captureStderr(() => runMeetingsSync(engine, 'meet-9552', parseMeetingsSourceConfig(meetingConfig, ''), extract,+ { provider: firefliesProvider({ token: 'synthetic-fixture-key', userAgent: 'gbrain-test', fetchImpl, sleep: async () => {} }) }));+ await settleConnectorWrites(engine, 'meet-9552');++ for (const [id, type] of [['slack-9552', 'slack'], ['meet-9552', 'meeting']] as const) {+ const pages = await engine.executeRaw<{ type: string }>('SELECT type FROM pages WHERE source_id=$1 AND deleted_at IS NULL', [id]);+ expect(pages.map(p => p.type)).toEqual([type]);+ await waitFor(async () => (await openEffects(engine, id)) === 0, { timeoutMs: 30_000 });+ }+ expect(stderr).not.toContain('writer_coordinator_required');+ expect({ slack: await queuedExtractions(engine, 'slack-9552'), meeting: await queuedExtractions(engine, 'meet-9552') }).toEqual({ slack: 1, meeting: 1 });++ __setChatTransportForTests(async () => ({ text: JSON.stringify({ facts: [{ fact: 'Acme Example signed a three-year renewal with Beta Example', kind: 'fact',+ entity: 'companies/acme-example', confidence: 0.9, notability: 'high' }] }), blocks: [], stopReason: 'end', model: MODEL, providerId: 'anthropic',+ usage: { input_tokens: 10, output_tokens: 10, cache_read_tokens: 0, cache_creation_tokens: 0 } }) as unknown as ChatResult);+ try {+ if (engine.kind === 'pglite') await runFactsDrain(engine, { owner: 'cycle', yieldMs: 0, log: () => {} });+ await runPhaseConversationFactsBackfill(engine, {});+ } finally { __setChatTransportForTests(null); }+ if (engine.kind === 'pglite') expect({ slack: await factRows(engine, 'slack-9552'), meeting: await factRows(engine, 'meet-9552') }).toEqual({ slack: 1, meeting: 1 });+ }+}), 180_000);++test('managed brain: a Gmail import queues no fact extraction, as on an unmanaged brain', async () => withEnv({ ...env, ...KEYS }, async () => {+ for (const engine of engines) {+ configureGateway({ embedding_disabled: true, env: KEYS } as never);+ await engine.setConfig('facts.extraction_model', MODEL);+ const account = '[email protected]';+ const gmailConfig = { kind: 'google', g_account: account, g_services: 'gmail', g_access: 'env', g_token_env: 'CONNECTOR_TEST_TOKEN' };+ const f = await source(engine, gmailConfig);+ const fx = fakeGmail(account);+ addThread(fx, 'a1b2c3d4e5f69552', Date.now() - 3_600_000, `${CLAIM} The renewal closes the account review for the year.`);+ await runGoogleSync(engine, f.id, parseGoogleSourceConfig(gmailConfig, f.dir), extract, withGoogleAccount(gmailFetch(fx), account));+ await settleConnectorWrites(engine, f.id);+ const pages = await engine.executeRaw<{ type: string }>('SELECT type FROM pages WHERE source_id=$1 AND deleted_at IS NULL', [f.id]);+ expect(pages.map(p => p.type)).toEqual(['email']);+ await waitFor(async () => (await openEffects(engine, f.id)) === 0, { timeoutMs: 30_000 });+ expect(await queuedExtractions(engine, f.id)).toBe(0);+ await disposePersistenceConsumer(engine);+ }+}), 120_000);++test('managed brain: a run with --no-extract queues no fact extraction', async () => withEnv({ ...env, ...KEYS }, async () => {+ for (const engine of engines) {+ const config = { kind: 'meetings', m_provider: 'fireflies', m_record: 'database', federated: false };+ await managedSource(engine, 'meet-9552-noextract', config);+ await runMeetingsSync(engine, 'meet-9552-noextract', parseMeetingsSourceConfig(config, ''), options,+ { provider: firefliesProvider({ token: 'synthetic-fixture-key', userAgent: 'gbrain-test', fetchImpl, sleep: async () => {} }) });+ await settleConnectorWrites(engine, 'meet-9552-noextract');+ const [receipt] = await engine.executeRaw<{ facts: unknown }>(+ `SELECT outcome->'facts_backstop' AS facts FROM persistence_requests WHERE source_id=$1 AND intent->>'kind'='connector_v2_import'`, ['meet-9552-noextract']);+ expect(receipt?.facts ?? null).toBeNull();+ expect(await openEffects(engine, 'meet-9552-noextract')).toBe(0);+ expect(await queuedExtractions(engine, 'meet-9552-noextract')).toBe(0);+ }+}), 120_000);++test('unmanaged brain: a page whose fact extraction could not be queued is logged to ingest_log and stderr, not dropped', async () => withEnv({ ...env, ...KEYS }, async () => {+ for (const engine of engines) {+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ const config = { kind: 'meetings', m_provider: 'fireflies', m_record: 'database', federated: false };+ await engine.executeRaw('INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,NULL,$2::text::jsonb)', ['meet-9552-plain', JSON.stringify(config)]);+ const failing = new Proxy(engine, { get: (target, key, receiver) => key === 'getConfig'+ ? async (k: string) => { if (k === 'facts.extraction_enabled') throw new Error('synthetic config read failure'); return target.getConfig(k); }+ : Reflect.get(target, key, receiver) }) as BrainEngine;+ try {+ const stderr = await captureStderr(() => runMeetingsSync(failing, 'meet-9552-plain', parseMeetingsSourceConfig(config, ''), extract,+ { provider: firefliesProvider({ token: 'synthetic-fixture-key', userAgent: 'gbrain-test', fetchImpl, sleep: async () => {} }) }));+ expect(stderr).toContain('meet-9552-plain: fact extraction was not queued for 1 page(s): synthetic config read failure');+ const logged = await engine.executeRaw<{ summary: string }>(`SELECT summary FROM ingest_log WHERE source_id=$1 AND source_type='facts:absorb'`, ['meet-9552-plain']);+ expect(logged.map(r => r.summary)).toEqual(['pipeline_error: extraction failed (Error)']);+ } finally { await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1'); }+ }+}), 120_000);++test('Slack pages a connector published before the fix get their facts from gbrain extract-conversation-facts', async () => withEnv({ ...env, ...KEYS }, async () => {+ const { runExtractConversationFacts } = await import('../src/commands/extract-conversation-facts.ts');+ const day = NOW - DAY;+ const conversation = (): FakeSlack => ({ teamId: 'TACME00001', users: { UALICE0001: { name: 'Alice Example' }, UBOB000001: { name: 'Bob Example' } }, calls: [], channels: [+ { id: 'CGENERAL01', name: 'general', messages: [+ { ts: slackTs(day, 1), user: 'UALICE0001', text: CLAIM },+ { ts: slackTs(day, 2), user: 'UBOB000001', text: 'Great news. Did Beta Example agree to the pricing terms we proposed last week?' },+ { ts: slackTs(day, 3), user: 'UALICE0001', text: 'Yes, they accepted the pricing terms and the renewal starts in January.' },+ { ts: slackTs(day, 4), user: 'UBOB000001', text: 'I will update the account plan and schedule the next review for the spring.' }] }] });+ for (const engine of engines) {+ configureGateway({ embedding_disabled: true, env: KEYS } as never);+ await engine.setConfig('facts.extraction_model', MODEL);+ await engine.setConfig('embedding_disabled', 'true');+ const config = { kind: 'slack', s_channels: 'CGENERAL01', s_record: 'database', federated: false };+ await managedSource(engine, 'slack-9552-before', config);+ await runSlackSync(engine, 'slack-9552-before', parseSlackSourceConfig(config, ''), options,+ { api: slackApi({ token: slackTestToken(), userAgent: 'gbrain-test', fetchImpl: slackFetch(conversation()), sleep: async () => {}, minDelayMs: 0 }) });+ await settleConnectorWrites(engine, 'slack-9552-before');+ expect(await queuedExtractions(engine, 'slack-9552-before')).toBe(0);+ expect(await factRows(engine, 'slack-9552-before')).toBe(0);+ __setChatTransportForTests(async () => ({ text: JSON.stringify({ facts: [{ fact: 'Acme Example signed a three-year renewal with Beta Example', kind: 'fact',+ entity: 'companies/acme-example', confidence: 0.9, notability: 'high' }] }), blocks: [], stopReason: 'end', model: MODEL, providerId: 'anthropic',+ usage: { input_tokens: 10, output_tokens: 10, cache_read_tokens: 0, cache_creation_tokens: 0 } }) as unknown as ChatResult);+ try { await runExtractConversationFacts(engine, ['--source-id', 'slack-9552-before', '--types', 'slack', '--yes']); }+ finally { __setChatTransportForTests(null); }+ expect(await factRows(engine, 'slack-9552-before')).toBeGreaterThan(0);+ }+}), 120_000);diff --git a/test/extract-takes-md-first.test.ts b/test/extract-takes-md-first.test.tsindex b58c3a5d66..fa8cc5c957 100644--- a/test/extract-takes-md-first.test.ts+++ b/test/extract-takes-md-first.test.ts@@ -126,7 +126,7 @@ describe('extractTakesFromPages — md-first (#4473)', () => {await engine.putPage(slug, { type: 'concept', title: 'Dry', compiled_truth: BODY, frontmatter: {} });writeFileSync(join(repo, `${slug}.md`), `# Dry\n\n${BODY}\n`, 'utf-8');- const r = await extractTakesFromPages(engine, { bootstrapEnabled: true, maxPages: 50, dryRun: true });+ const r = await extractTakesFromPages(engine, { bootstrapEnabled: true, maxPages: 50, dryRun: true, yes: true });// Dry-run rescans BOTH uncovered pages (db-born included — no file gate on dry-run).expect(r.claims_extracted).toBeGreaterThanOrEqual(1);const md = readFileSync(join(repo, `${slug}.md`), 'utf-8');diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 721bf69b66..13b966d9b9 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -63,6 +63,7 @@ test/belief-ledger-trust.test.tstest/belief-ledger-w5-time-model.test.tstest/belief-ledger-weekly-diff.test.tstest/belief-ledger-weekly-probes.test.ts+test/belief-mention-disable-9548.test.tstest/capacity-bounded-read-6803.test.tstest/capture-consent-admission.test.tstest/capture-dispatch-provenance-6524.test.tsdiff --git a/test/takes-extract-dry-run-consent.test.ts b/test/takes-extract-dry-run-consent.test.tsnew file mode 100644index 0000000000..864af48c14--- /dev/null+++ b/test/takes-extract-dry-run-consent.test.ts@@ -0,0 +1,119 @@+/**+ * #9547 — `gbrain takes extract --from-pages --dry-run` without `--yes`+ * sends nothing to the model. It lists and counts the candidate pages and+ * reports no claim count (null, never 0: 0 would read as "nothing to+ * extract"). `--dry-run --yes` still classifies and writes nothing.+ */+import { describe, test, expect, beforeAll, afterAll } from 'bun:test';+import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { configureGateway, resetGateway, __setChatTransportForTests } from '../src/core/ai/gateway.ts';+import { runTakes } from '../src/commands/takes.ts';++let engine: PGLiteEngine;+let repo: string;+let chatCalls = 0;+const BODY = 'An opinion-bearing body long enough to clear the 200-char eligibility floor. '.repeat(5);+const SLUG = 'concepts/dry-run-candidate';++beforeAll(async () => {+ engine = new PGLiteEngine();+ await engine.connect({});+ await engine.initSchema();+ repo = mkdtempSync(join(tmpdir(), 'gb-takes-dry-run-'));+ mkdirSync(join(repo, 'concepts'), { recursive: true });+ await engine.setConfig('sync.repo_path', repo);+ await engine.setConfig('takes.bootstrap_enabled', 'true');+ configureGateway({ chat_model: 'anthropic:claude-haiku-4-5-20251001', env: { ANTHROPIC_API_KEY: 'sk-ant-test-dry-run' } });+ __setChatTransportForTests(async () => {+ chatCalls++;+ const text = '[{"claim":"a bootstrapped claim","kind":"take","weight":0.7}]';+ return {+ text,+ blocks: [{ type: 'text' as const, text }],+ stopReason: 'end' as const,+ usage: { input_tokens: 1, output_tokens: 1, cache_read_tokens: 0, cache_creation_tokens: 0 },+ model: 'anthropic:claude-haiku-4-5-20251001',+ providerId: 'anthropic',+ };+ });+ await engine.putPage(SLUG, { type: 'concept', title: 'Dry run', compiled_truth: BODY, frontmatter: {} });+ writeFileSync(join(repo, `${SLUG}.md`), `# Dry run\n\n${BODY}\n`, 'utf-8');+});++afterAll(async () => {+ __setChatTransportForTests(null);+ resetGateway();+ await engine.disconnect();+ rmSync(repo, { recursive: true, force: true });+});++async function rowCounts(): Promise<Record<string, number>> {+ const tables = await engine.executeRaw<{ t: string }>(`SELECT tablename AS t FROM pg_tables WHERE schemaname = 'public' ORDER BY 1`);+ const counts: Record<string, number> = {};+ for (const { t } of tables) {+ const rows = await engine.executeRaw<{ n: number }>(`SELECT COUNT(*)::int AS n FROM "${t}"`);+ counts[t] = rows[0].n;+ }+ return counts;+}++async function runCaptured(args: string[]): Promise<string> {+ const chunks: string[] = [];+ const originalWrite = process.stdout.write;+ process.stdout.write = ((chunk: string | Uint8Array) => {+ chunks.push(String(chunk));+ return true;+ }) as typeof process.stdout.write;+ try {+ await runTakes(engine, args);+ } finally {+ process.stdout.write = originalWrite;+ }+ return chunks.join('');+}++describe('takes extract --from-pages --dry-run (#9547)', () => {+ test('without --yes: no model call, candidates listed, no claim count, nothing written', async () => {+ const mdPath = join(repo, `${SLUG}.md`);+ const mdBefore = readFileSync(mdPath, 'utf-8');+ const countsBefore = await rowCounts();+ chatCalls = 0;++ const result = JSON.parse(await runCaptured(['extract', '--from-pages', '--dry-run', '--json']));++ expect(chatCalls).toBe(0);+ expect(result.dry_run).toBe(true);+ expect(result.claims_extracted ?? null).toBeNull();+ expect(result.pages_scanned).toBe(1);+ expect(result.candidates).toEqual([{ slug: SLUG, source_id: 'default' }]);+ expect(readFileSync(mdPath, 'utf-8')).toBe(mdBefore);+ expect(await rowCounts()).toEqual(countsBefore);+ });++ test('without --yes: the summary says nothing was sent to the model', async () => {+ chatCalls = 0;+ const out = await runCaptured(['extract', '--from-pages', '--dry-run']);+ expect(chatCalls).toBe(0);+ expect(out).toContain(SLUG);+ expect(out).toContain('nothing was sent to the model');+ expect(out).not.toContain('0 claim(s)');+ });++ test('with --yes: classifies the candidates and writes nothing', async () => {+ const mdPath = join(repo, `${SLUG}.md`);+ const mdBefore = readFileSync(mdPath, 'utf-8');+ const countsBefore = await rowCounts();+ chatCalls = 0;++ const result = JSON.parse(await runCaptured(['extract', '--from-pages', '--dry-run', '--yes', '--json']));++ expect(chatCalls).toBe(1);+ expect(result.dry_run).toBeUndefined();+ expect(result.claims_extracted).toBe(1);+ expect(readFileSync(mdPath, 'utf-8')).toBe(mdBefore);+ expect(await rowCounts()).toEqual(countsBefore);+ });+});diff --git a/test/takes-extract-json.test.ts b/test/takes-extract-json.test.tsindex 2c50f0fccd..ba9b6c42ae 100644--- a/test/takes-extract-json.test.ts+++ b/test/takes-extract-json.test.ts@@ -50,7 +50,7 @@ afterAll(() => {describe('gbrain takes extract --from-pages --json (#3962)', () => {test('emits the extraction result as parseable JSON', async () => {const stdout = await captureStdout(() =>- runTakes(engine, ['extract', '--from-pages', '--dry-run', '--json']));+ runTakes(engine, ['extract', '--from-pages', '--dry-run', '--yes', '--json']));expect(JSON.parse(stdout)).toEqual({pages_scanned: 0,@@ -72,7 +72,7 @@ describe('gbrain takes extract --from-pages --json (#3962)', () => {queries.length = 0;const cursorTs = '2031-07-08 09:10:11.654321-05';await captureStdout(() =>- runTakes(engine, ['extract', '--from-pages', '--json', '--dry-run', '--before', `${cursorTs},977`, '--source-id', 'default']));+ runTakes(engine, ['extract', '--from-pages', '--json', '--dry-run', '--yes', '--before', `${cursorTs},977`, '--source-id', 'default']));const pageQuery = queries.find((q) => q.sql.includes('FROM pages'));expect(pageQuery).toBeDefined();expect(pageQuery!.sql).not.toContain(cursorTs);@@ -85,7 +85,7 @@ describe('gbrain takes extract --from-pages --json (#3962)', () => {test('negative control: without --before the page query carries no cursor condition', async () => {queries.length = 0;- await captureStdout(() => runTakes(engine, ['extract', '--from-pages', '--json', '--dry-run']));+ await captureStdout(() => runTakes(engine, ['extract', '--from-pages', '--json', '--dry-run', '--yes']));const pageQuery = queries.find((q) => q.sql.includes('FROM pages'));expect(pageQuery!.sql).not.toContain('(updated_at, id) <');expect(pageQuery!.params).toEqual([]);@@ -130,7 +130,7 @@ describe('takes extract --before cursor validation (#5059)', () => {? [{ id: 9, slug: 'concepts/short', source_id: 'default', type: 'concept', compiled_truth: 'too short', updated_at: new Date('2030-01-02T03:04:05.123Z'), resume_at: '2030-01-02 03:04:05.123456+00,9' }]: sql.includes('FROM persistence_brain') ? [{ enabled: false }] : [],} as unknown as BrainEngine;- const stdout = await captureStdout(() => runTakes(pageEngine, ['extract', '--from-pages', '--dry-run']));+ const stdout = await captureStdout(() => runTakes(pageEngine, ['extract', '--from-pages', '--dry-run', '--yes']));expect(stdout).toContain("next: --before '2030-01-02 03:04:05.123456+00,9'");});});