Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: record each /ingest capture once per idempotency key so a re-run or replay can't revert a newer edit (GST-1856)
garrytanafa63c30407d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f722e0705a..d5d6313072 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,17 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**A `POST /ingest` capture that runs again after its write landed no longer reverts a newer edit to the page.** A worker that died or lost its lock after the write committed, but before the job was marked complete, handed the job to another attempt, which wrote the captured body over any edit made in between. A client replaying the same POST after `gbrain jobs prune` removed the finished job got a fresh job that did the same. Each capture is now recorded under its job's idempotency key in the same transaction as the page write, and any later run of that key writes nothing and returns the first run's result. The record lasts 90 days, even after the job row is pruned. Separate captures to one page still land in order, because each has its own key.++### Itemized changes++- `src/core/minions/handlers/ingest-capture.ts` + `src/core/ingestion/applied-captures.ts`: a keyed capture checks `ingest_applied_captures` (new migration v296) before writing. On an unmanaged brain the row is inserted inside the page write's transaction, and a run that finds the row already there rolls its write back. On a managed brain each attempt sends the door a `request_id` derived from the key and the attempt number, the row is inserted once the door reports the write committed, and a committed request found without its row counts as applied and fills the row in. Tombstones and jobs queued without a key behave as before.+- `src/core/minions/types.ts`: handlers see the job's `idempotency_key` on their context; jobs already waiting when you upgrade carry it from their row.+- `src/core/cycle.ts`: the dream cycle's purge phase drops applied-capture records older than 90 days.+- Tests: `test/ingest-capture-applied-once.test.ts` (PGLite) and `test/e2e/ingest-capture-applied-once-postgres.test.ts` (Postgres): a re-run after a newer edit, a replay after the job row is pruned (through the real route), two ordered captures to one slug, and the re-run on a managed brain.+## [0.60.610.0] - 2026-11-08**A client that retries a `POST /ingest` capture accepted before v0.60.609.0 no longer queues a second job that writes the old body over the page's newer edit.** v0.60.609.0 added the target slug to the capture's queue key, and nothing looked up the old key form, so a retry of a capture that had already landed was taken as a new one. The route now also looks up a capture's pre-v0.60.609.0 key and answers 202 with that job when it was queued for the same target slug, so the retry is the same request again and nothing is queued.diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.mdindex 5bf68d9dd3..dd268e867c 100644--- a/docs/architecture/KEY_FILES.md+++ b/docs/architecture/KEY_FILES.md@@ -26,15 +26,15 @@ link it here; never raise the cap.| [Canonical reconciliation](key-files/canonical-reconciliation.md) | Exact-page repair, retained originals, [derived generations](key-files/generations.md), sync discovery and the origin rule; [rows behind the checkpoint](key-files/canonical-reconciliation-continued.md) || [Company-brain ingestion](key-files/company-brain.md) | Inspection, admission, receipts, derived relationships and schema; [operator guide](../guides/company-brain-ingestion.md) || [Agent operator contract](key-files/agent-contract.md) | `agent-output.ts`, error registry + docs, notice ledger, `isCallable`, `--json` guard, contract scanner |-| [Commands (1/6)](key-files/commands-1.md) | `src/commands/agent-logs.ts` through `db-repair.ts` |+| [Commands (1/6)](key-files/commands-1.md) | `agent-logs.ts` to `db-repair.ts` || [Commands (2/6)](key-files/commands-2.md) | `src/commands/doctor.ts` and `src/commands/doctor/` || [Commands (2/6, continued)](key-files/commands-2-continued.md) | `doctor/checks/routing-federation.ts`, `git-convergence.ts`, `keyword-plan.ts`, `dream-retriage.ts` to `embed.ts` || [Commands (3/6)](key-files/commands-3.md) | `src/commands/engine-status.ts` through `extract.ts` || [Commands (3/6, continued)](key-files/commands-3-continued.md) | `src/commands/extract-timeline-db.ts` through `frontmatter-install-hook.ts` || [Commands (4/6)](key-files/commands-4.md) | `src/commands/frontmatter.ts` through `pages-status.ts` || [Commands (4/6, continued)](key-files/commands-4-continued.md) | `persistence-admin.ts` to `reindex-search-vector.ts`, `init-mode-picker.ts`, `core/embedding-migration-cli.ts` |-| [Commands (5/6)](key-files/commands-5.md) | `src/commands/reindex.ts` through `storage.ts` |-| [Commands (6/6)](key-files/commands-6.md) | `src/commands/sync.ts` through `whoknows.ts` |+| [Commands (5/6)](key-files/commands-5.md) | `repair.ts` to `skillpack.ts`; [continued](key-files/commands-5-continued.md) |+| [Commands (6/6)](key-files/commands-6.md) | `sync.ts` to `whoknows.ts` || [Core Ai](key-files/core-ai.md) | `src/core/ai/` gateway, embedding, dims, errors, fallbacks || [Core Ai (continued)](key-files/core-ai-continued.md) | `src/core/ai/providers/*`, `recipes/*`, `voyage-gateway.ts` || [Core Decide](key-files/core-decide.md) | `src/core/ai/decide/*`, `src/core/search/decide-stage.ts`, `gbrain decide`, `decide_health` (System One) |diff --git a/docs/architecture/key-files/commands-5-continued.md b/docs/architecture/key-files/commands-5-continued.mdnew file mode 100644index 0000000000..c7fc6f5090--- /dev/null+++ b/docs/architecture/key-files/commands-5-continued.md@@ -0,0 +1,20 @@+# Key files: Commands (5/6, continued)++[Subsystem index](../KEY_FILES.md) · [Previous entries](commands-5.md). Read only the entries relevant to your change.+Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.++- `src/commands/sources-lifecycle{,-args}.ts` — strict source mutation parsing before engine connection, with stable UUIDs and source incarnation preconditions. The private CLI administration envelope delegates to a live PGLite owner, retains request IDs after lost responses, and refuses preactivation administration or legacy hardening rather than changing worktrees through an uncoordinated writer. Managed purge names one archived source; the expiry worker separately coordinates each source. `test/persistence-source-cli.test.ts` executes real CLI subprocesses, checks replay after deletion/recreation, and verifies global page/lifecycle request identity.++- `src/commands/sources-reset-upstream.ts` — `gbrain sources reset-upstream <id> --expect-local <sha> --expect-upstream <sha> [--dry-run] [--json]`, the fix for `upstream_rewritten` / `upstream_unknown` (`src/core/sync-upstream-rewrite.ts`). Refuses a managed canonical worktree (`writer_coordinator_required`) and an in-progress git operation (`git_operation_in_progress`); under the checkout's commit lock (none on `--dry-run`) refuses with `reset_upstream_refused` a HEAD or upstream other than the expected SHAs (`expect_mismatch`), no upstream (`no_upstream`), tracked edits (`dirty_tree`, names the paths) and local commits not reachable from the upstream, the recorded baseline or any old or new value of the tracking ref's reflog (`local_commits`, names them and the backup ref). Otherwise writes `refs/gbrain/backup/<UTC stamp>-<sha12>` (create-only), runs `git reset --keep <upstream>`, verifies HEAD, records the baseline and clears the divergence; never fetches. Pinned by `test/pathological/s-58-upstream-rewrite.test.ts`.+- `src/commands/sources-retry-held.ts` — `gbrain sources retry-held <id> [--dry-run] [--json]`: records a re-attempt request for a source's held connector items (and, on a managed brain, their retry-pointer rows) or a re-screen request (op `sync-hold-retry`) for its held Git files, which the next sync adds to its manifest; nothing runs now, the receipt prints the sync to run. #6278: on a managed brain that follow-up sync is built by `gitFollowUpSync`: the unfinished managed cursor's resume arguments (`readManagedSyncBacklog`, `managedSyncResumeArgs`), so `--no-embed` and the other processing options carry over instead of starting a second cursor, else the arguments a `preparation_stalled` hold recorded (`meta.stall.sync_argv`), else `--source <id> --no-pull`. `preparation_stalled` holds count under `stalled` in the repair route and need no file repair: the receipt and help point to `gbrain sources writer status --source <id> --json` first. #6865: `--unindexed` (`retryUnindexed`) runs one bounded pass of the unindexed-file detector (`persistence/unindexed-files.ts`, 60 s) for a managed Git source, schedules the found files through the same re-screen request, stores the result doctor `unindexed_files` reads (continuing an incomplete stored scan) and prints the follow-up sync with `gbrain doctor --only unindexed_files --json` as its verify; `--dry-run` lists them and writes nothing; connector and company-brain sources refuse `invalid_params`; a complete pass with nothing found prints `No unindexed files for <id>.`, an incomplete one says the scan continues. Both modes build the receipt through `receiptOf`. Tests: `test/held-files-walkthrough.test.ts`, `test/managed-sync-preparation-stalled-holds.test.ts`, `test/unindexed-files.test.ts`.+- `src/commands/sources-resolve-path.ts` — GBRA-150: `gbrain sources resolve-path <id> <path...> [--stdin] [--json]`, the CLI face of op `sources_resolve_path` (`core/ops/sources.ts`). Paths come from argv or `--stdin` (one per line, at most `RESOLVE_PATHS_MAX` = 200); the human print is one line per path (state, slug, why, fix), `--json` the op's result. Read-only. Tests: `test/sources-resolve-path.test.ts`. Harness guide: `docs/guides/resolve-paths.md`.+- `src/commands/sources-set-path.ts` — `gbrain sources set-path <id> <path> [--force]`: unmanaged installations retain the legacy pointer repair and overlap check. Managed installations use the verified lifecycle rebind; force never bypasses ownership or manifest checks. Relative paths and MSYS spellings are normalized before routing. The `default_source_local_path` doctor check names this as the repair.++- `src/commands/sources-strategy.ts` — `gbrain sources set-strategy <id> <markdown|code|auto>` (#4899, PR #5955): writes `sources.config.strategy` through `updateSourceConfig` (single-key merge; every other config key survives), the value `sync --all`, autopilot, the dream cycle and the MCP `sync` op pass to the per-source sync when the caller names no `--strategy`; unset means `markdown`. The one writer of that key (no `sources add --strategy`, no op param). Connector kinds refuse it. [R26]: refuses with `sync_run_unfinished` (exit 1) while the source has an unfinished `managed-sync` cursor, naming the resume and the orphan repair, because the frozen manifest was classified under the old strategy; re-stating the current value is allowed. `sources list --json` and `listSources` report `strategy` (`null` when unset). Pinned by `test/sources-strategy.test.ts`.++- `src/commands/sources-add-next.ts` — GBRA-156: `noGrantFloorNext(engine, id)`, the federate step `sources add` prints when a new non-federated source leaves the no-grant floor (`localFederatedSourceIds(engine, 'default', 'seed_default')`, else `['default']`) holding no live page a remote read may show (private pages excluded when remote reads exclude them); returns `{ command, argv, why }` or null. One read, no write; `sources add` treats a failed read as no step.+- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit,refresh,set-calendar}`; `set-calendar` dispatches to `src/commands/sources-set-calendar.ts`, and `set-cr-mode` prints `contextualModeHint` after the write. `refresh` dispatches to `src/commands/sources-refresh.ts` (argument parsing, stdout text or `--json` result/refusal objects, exit 1 on refusal or an unfinished sync; protocol in `src/core/persistence/worktree-refresh.ts`, where a fence lock timeout (55P03) becomes `refresh_lock_timeout` naming the open transactions holding the fenced tables, and any other untyped throw becomes an `internal_error` refusal whose `why` carries the stack and whose fix names the refresh row it left for the next run to resume, GBRA-250). `current [--json]` calls `resolveSourceWithTier()` and prints `source_id`, `tier` (`flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail` (decision table in `skills/conventions/brain-routing.md`). `status [--json]` — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` from `src/commands/sync.ts`; `--json` emits stable `{schema_version: 1, sources, ...}` on stdout; filters input to `local_path IS NOT NULL AND archived IS NOT TRUE`. `audit <id> [--json]` — read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). The live `runStatus` health table gains a `BACKFILL` column between `EMBED` and `FAILS` (`active(N)` beats `queued(N)` beats `idle`, from `SourceMetrics.backfill_active` / `backfill_queued` in `src/core/source-health.ts`) so operators see deferred `embed-backfill` minion work after `sync --all` exits 0; `jobCountsBySource` in `source-health.ts` widens its `minion_jobs` SQL with two `COUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...)` aggregates (best-effort, all-0 on pre-minions brains). `remove` pre-checks OAuth-client referents (guided refusal via `formatClientReferentsBlock` — the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (`unhardenBrainRepo`: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact. `purge` runs the same referents pre-check before its hard DELETE. `add` without `--federated` or `--no-federated` stores the source non-federated and warns on stderr that a remote token or agent connection with no source grant cannot read it, naming `gbrain sources federate <id>` and `gbrain auth rescope --token <name> --sources <id>`; when the no-grant floor (the default source plus federated sources) then holds no live page it also prints `next: gbrain sources federate <id>` on stdout (`noGrantFloorNext` in `sources-add-next.ts`). `add --json` prints one object (`id`, `name`, `local_path`, `remote_url`, `federated`, and `next` with `command`/`argv`/`why` when that step applies) and moves the human lines to stderr (`test/sources-add-federation-warning.test.ts`). Pinned by `test/content-sanity.test.ts`, `test/import-file-content-sanity.test.ts`, `test/source-health.test.ts`.++- `src/commands/status-funnel.ts` — `gbrain status --funnel [--json] [--enable | --delete]` (D8 adoption funnel): engine-free, dispatched from `src/cli/main.ts` before any connect so it works before `gbrain init` and on a thin client; reads and writes only the local record through `src/core/funnel.ts`. Guide: `docs/guides/adoption-funnel.md`. Tests: `test/funnel.test.ts`, `test/funnel-cli.test.ts`.++- `src/commands/storage.ts` — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. The repo and the counted source come from `resolveRestoreTarget` (`src/core/restore-target.ts`, the `export --restore-only` rule), and a page is listed missing exactly when that export would restore it (`isDbOnly`, then `restoreFilePath` in the page source's `scannerSlugRootMode` and `nativeFileTarget`: the recorded `source_path`, else `<slug>.md`, absent; the repo walk answers without a per-page syscall whenever it saw that exact file as a regular file under real directories, and a recorded path starting with the repo's path inside its git checkout, which export re-anchors, takes the full check), so the printed `Use: gbrain export --restore-only --source <id> --repo <path>` (the path passed through `shellQuote`, `src/core/shell-quote.ts`) writes exactly the listed files into `--dir` (default `./export`, not the repo). An unsafe recorded path or a symlinked/ambiguous file target (`OperationError` from `nativeFileTarget`) becomes one warning per reason (lowest slug plus a count of further pages) plus the refusal export would give, printed even when no file is missing; a repo whose own owner (`resolveSourceForRepoPath`: dotfile, then longest registered `local_path`) is archived is refused too (unless an active source is registered at the repo's exact path, which owns it outright), although export's exact-path rule would fall back to the only active source; a refused rule prints `Cannot suggest a restore command:` and falls back to the `--repo` / default-source path, counting the active repo owner's pages (dotfile, then longest registered `local_path`) or every source's pages when no active source owns it. `restoreSource` / `restoreRefusal` carry this in the JSON result. Pinned by `test/storage-restore-hint.test.ts`.diff --git a/docs/architecture/key-files/commands-5.md b/docs/architecture/key-files/commands-5.mdindex 05eb1a2f26..396aff9731 100644--- a/docs/architecture/key-files/commands-5.md+++ b/docs/architecture/key-files/commands-5.md@@ -22,7 +22,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/commands/serve-http-grants.ts` + `admin/src/components/ClientGrant.tsx` — typed admin grant request validation and shared profile/advanced permission editor. Register/rescope/TTL routes in `serve-http-registration.ts`, `serve-http-grants.ts`, and `serve-http-admin-api.ts` use canonical grant validation and audit; legacy response shapes remain available. Registration and permission-edit failures add safe classifications, stage, outcome and inspection remedies; an unacknowledged mutation stays outcome-unknown, while read/preview failures and known refusals remain failed. New UI creation defaults to memory-writer and requires a preview, while edits carry the displayed revision and require a fresh preview after any field change or conflict. Detail/catalog endpoints expose non-secret grants and the shared harness registry. Operation authority snapshots, source sets, delegated tools/namespaces, spend, concurrency, and future TTL are independently reviewable; profile regrant is explicit. `admin/src/pages/Agents.tsx` previews machine/public PKCE/confidential PKCE registration and current permissions, and exposes versioned lifecycle actions. Explicit downloads select machine `HarnessCredentials` or native `OAuthClientSetup`; mixed clients require a flow choice. Loading errors preserve stale rows with Retry and disable creation when required source metadata is missing. Browser downloads and native connection evidence remain distinct. Rebuild committed `admin/dist` and `src/admin-embedded.ts` after SPA edits.- `src/commands/serve-http.ts` — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. The legacy `POST /ingest` queue refuses operation-snapshot clients; they use approved MCP writes so execution stays inside the shared grant contract. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--bind HOST] [--log-full-params]`. OAuth discovery's `scopes_supported` (RFC 8414 AS metadata and both RFC 9728 protected-resource documents, one value through the SDK router) comes from `scopesSupportedForDiscovery({ enableDcr })` in `src/core/scope.ts`: with `--enable-dcr` it is exactly the self-registration ceiling (`read write`), so a client that copies discovery into its `/register` request succeeds; without DCR it is every scope except operator-only `agent`. The DCR ceiling still rejects an explicit privileged request, and operator-registered clients are never narrowed by the advertisement. Combines MCP SDK's `mcpAuthRouter` (authorize/token/register/revoke), a custom `client_credentials` handler running BEFORE the router (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; custom handler falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement + `localOnly` rejection before op dispatch, and per-IP `express-rate-limit` buckets: the MCP SDK's defaults on `/authorize` (100 / 15 min), `/register` (20 / hour) and `/token` (50 / 15 min) because `authRouterOptions` passes no `rateLimit: false`, the custom `client_credentials` handler's `ccRateLimiter`, and independent owner limits: 60 total authentication requests/min/IP, 10 failed authentications/min/IP (verified authority, independent of response status), and 60 consent requests/min/authenticated session in `serve-http-admin-limits.ts`. Admin no-store, no-referrer and framing headers precede login/link handlers. Explicit pending IDs survive fresh-browser links; invalid or restarted requests require native authorization to restart. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE broadcasts every MCP request. `cookie-parser` wired (Express 5 has no built-in). Startup logging prints port, engine, issuer URL (honors `--public-url`), client count, DCR status, and the admin bootstrap token line — but the generated token's raw value only prints when stderr is an interactive TTY (`shouldSuppressBootstrapPrint`): a non-TTY (containerized/piped) start hides it so the secret never lands in centralized log storage, env-sourced tokens (`$GBRAIN_ADMIN_BOOTSTRAP_TOKEN`) are always hidden, `--print-admin-token` forces the raw value on a trusted terminal, and `--suppress-bootstrap-token` hides everything. The `/mcp` request handler's OperationContext literal sets `remote: true` explicitly (without it `submit_job`'s protected-name guard would see a falsy undefined and a `read+write`-scoped OAuth token could submit `shell` jobs). `summarizeMcpParams` from `src/mcp/dispatch.ts` feeds both `mcp_request_log` writes and the SSE feed by default (raw via `--log-full-params`). Cookie `Secure` flag set behind HTTPS or a public-URL proxy; magic-link nonce store LRU-bounded; DCR disable routes through the `GBrainOAuthProvider` `dcrDisabled` constructor option (not a router monkey-patch); `transport.handleRequest` wrapped in try/catch to return a JSON-RPC 500 envelope; OperationError + unexpected exceptions unified through `buildError` / `serializeError` so `/mcp` always returns the same envelope. `/health` is liveness-only via `probeLiveness(engine, engineName, version, timeoutMs)` racing `engine.executeRaw('SELECT 1', undefined, { signal })` against the exported `HEALTH_TIMEOUT_MS = 3000`; when the timeout wins, an `AbortController` cancels the Postgres query (PGLite can only discard its eventual result) before the same tagged `ProbeHealthResult` 503 envelope is returned (single timer-cleanup site); body shape `{status, version, engine}` only. Full stats live at admin-only `/admin/api/full-stats` (gated by `requireAdmin`, calls `probeHealth(engine, ...)`) — keeps `getStats()`'s 6× count(*) off the public route so a saturated pool doesn't trigger orchestrator restart cascades. OAuth/admin/audit SQL outside the abortable liveness probe routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so it works against PGLite; the four `mcp_request_log.params` INSERT sites (success / auth_failed / scope_denied / server-error) go through `executeRawJsonb(engine, ...)` so the column stores real objects (`params->>'op'` returns `search`, not the quoted string). `--bind HOST` defaults `127.0.0.1` (self-hosters pass `--bind 0.0.0.0`); a stderr WARN fires when `--public-url` is set without `--bind`; the banner prints a `Bind:` line. `AuthInfo.sourceId` + `AuthInfo.allowedSources` + `AuthInfo.takesHoldersAllowList` are the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` (source scope from the `oauth_clients` row; takes-holders from `access_tokens.permissions.takes_holders` for legacy bearer tokens). The `/mcp` dispatch site reads `authInfo.takesHoldersAllowList ?? ['world']` — absent grants (OAuth-client tokens, pre-v29 brains) fail closed to world-only takes visibility, while an explicit `[]` grant is preserved as deny-all; pinned end-to-end by `test/e2e/serve-http-takes-holders.test.ts`. The HTTP MCP `tools/list` handler at `:837-849` uses `paramDefToSchema(v)` from `src/mcp/tool-defs.ts` so array params keep `items` (strict-mode OAuth clients otherwise reject the whole tool list). `POST /ingest` enforces the slug-prefix write fence at the ROUTE, not the op layer: the route hands its payload to the `ingest_capture` minion handler, which deliberately bypasses `put_page`, so no `OperationContext` exists and `enforceClientSlugFence` never runs — a slug-bound client must therefore supply `X-Gbrain-Slug` and it must satisfy `slugUnderBoundPrefixes`, else 403 (without the check a bound client could overwrite any page inside its granted source). The write source is resolved server-side as `authInfo.sourceId ?? 'default'` and travels on the job as `job.data.sourceId`; the caller-supplied `X-Gbrain-Source-Id` header routes nothing and only names the emitter (`webhook-<clientId>`), which is what the event's `source_id` and the 202's back-compat `source_id` field carry. The 202 additionally reports the routed destination as `write_source_id`, and the write source and normalized target slug (hashed) join the queue idempotency key, so a rescoped client or a second slug gets its own capture, and a slug over 1024 bytes gets 400 `invalid_slug` (`test/serve-http-ingest-write-loss.test.ts`). confidential revoke: a pre-router `/revoke` handler validates the RFC 7009 body, verifies hash-only secrets for both `client_secret_post` and `client_secret_basic`, rejects mixed authentication, preserves the SDK path for public clients, and separates opaque client-auth failures from retryable/backend failures. OAuth metadata advertises both confidential methods. Pinned by `test/e2e/serve-http-oauth.test.ts`. three admin routes: `/admin/api/calibration/profile`, `/admin/api/calibration/charts/:type` (image/svg+xml; type in {brier-trend, domain-bars, pattern-statements, abandoned-threads}), `/admin/api/calibration/pattern/:id` (drill-down). The manual mint route `/admin/api/register-client` in `serve-http-registration.ts` accepts optional `source` + `federatedRead` bindings mirroring the CLI's `--source`/`--federated-read` (validated via `normalizeSourceInput`/`normalizeFederatedReadInput` from `src/core/source-id.ts`; omitting both preserves the default binding `source_id='default'` / `federated_read=[source_id]`, invalid values return a structured 400 `invalid_source`), so an admin SPA or provisioning proxy can mint a source-confined client over HTTP. The route mirrors the CLI lane end to end: structured 400s for `unknown_source` / `archived_source` (one batched `ANY($1::text[])` existence check), `invalid_token_ttl` (shared `TOKEN_TTL_MIN/MAX_SECONDS` bounds, integer-validated BEFORE the tx), and `brain_too_old` (column pre-flight OUTSIDE the tx — a pre-scoped-clients brain refuses up front instead of aborting mid-transaction); the duplicate-name pre-check + INSERT run in ONE transaction under the SAME name-scoped advisory lock the CLI takes (`registerClientNameLockKey`), returning 409 `duplicate_name` with the existing `client_id`; the INSERT composes `registerScopedClient` (the CLI's registration core) so the two paths cannot drift; any post-commit failure includes the created `client_id` so the operator can revoke (never a false "nothing was created"). Pinned by `test/register-client-source-normalize.test.ts`. The `/mcp` tools/list is the honest catalog: per-request filters (token scope incl. the `agentCallable` carve-out, bound-client fence via `opAllowedForBoundClient`, publish gates via `disabledOpsForPublishGates`) over the surface-filtered op set, schemas via `buildToolDefs` (strict emission when `mcp.strict_params` resolves reject); the tools/list `mcp_request_log` row records the listed size as `params.tool_count`. Call-time op-level denials the list should have prevented — the inline scope deny, publish-gate backstop (`config_key=...`), fence op-level deny (`fence=op`, classified via `isListLevelDenialEnvelope`) — log `status='denied_after_list'` instead of `'error'` (the trend-to-zero honest-catalog metric, see docs/operations/mcp-surface-runbook.md) — every tools/call row's status resolves through `requestLogStatusForResult` (dispatch.ts) on BOTH transports. The admin health-indicators error rate counts `status NOT IN ('success','success_with_warnings')` and excludes `operation='surface_change'` audit rows from numerator AND denominator (audit rows record operator/self actions, not traffic). Per-request surface resolution lives in `resolveEffectiveSurface`: a `verbs` ceiling skips the default-surface config read, and a failed read applies the last successfully read default per process (fail-closed — a transient config outage cannot widen a NULL-surface client). OAuth protected-resource metadata (RFC 9728) names `/mcp` as the resource (`resourceServerUrl`), so the SDK mounts the PRM at `/.well-known/oauth-protected-resource/mcp` and the 401 `resource_metadata` URL is derived from the same value; the bare root path is kept as an alias by rewriting `req.url` onto the SDK's handler (same document, same CORS). Shutdown: `waitForHttpServerLifecycle` tracks accepted sockets weakly (`trackServerSockets`; Bun never emits socket `close`, so a strong set leaked one per request), severs the still-reachable ones right after `server.close()`, and bounds the close wait at `CLOSE_TIMEOUT_MS` (5s) — a collected keep-alive wrapper whose native handle keeps `close()` from returning gets one stderr line and shutdown proceeds instead of hanging. DCR startup copy: the SECURITY WARNING states self-registered clients are limited to read/write (read-only for `client_credentials` under `--enable-dcr-insecure`) and that every authorization_code connection needs owner approval in the admin UI; banner `DCR:` values are `enabled (consent; DCR max read write)` / `enabled (INSECURE: read-only M2M)` / `disabled`.- Layout: `runServeHttp` binds through `listenOrRefuse` (`serve-http-listen.ts`: `serve_port_in_use` on a taken port; with `options.adoptServer` from status-only recovery it hands the app to the already-bound listener instead of binding). `buildServeHttpApp` builds one shared `ServeHttpContext` (sessions, nonce stores, limiters, `requireAdmin`, SSE fan-out, OAuth provider, resource verifier, non-`localOnly` op list) and mounts in registration order `serve-http-oauth.ts` (CORS gates, DCR TTL hook, confidential credentials, SDK router, discovery, owner login links, consent), `serve-http-metrics.ts` (`/health`, `/metrics`, probes), `serve-http-admin-api.ts` (`requireAdmin`, `/admin/login`, `/admin/api/*`, `/admin/events`), `serve-http-spa.ts`, `serve-http-mcp.ts` (`/mcp`; writes `[gbrain-serve] dispatch op=<name> args_sha256=<16 hex>` to stderr before each tool call so a stall's last line names its request, never the arguments) and `serve-http-webhooks.ts` (`/ingest`, which maps a legacy-authority `permission_denied` from submission to 409 `{error, message, hint, docs_url}` (`test/serve-http-ingest-legacy.test.ts`), answers 429 `ingest_backlog_full` when the 50-capture waiting cap hands back another job, and every other submission error to 500; `/webhooks/github`, which HMACs every body, with a throwaway key when there is no source or secret); `runServeHttp` keeps listen, banner, resolve-IPC binding and shutdown, and the façade re-exports moved symbols. Order is pinned by `test/serve-http-route-golden.test.ts` (AST) and `test/serve-http-route-runtime-golden.test.ts` (router stack). Size ceiling: `scripts/module-size-limits.tsv`.+ Layout: `runServeHttp` binds through `listenOrRefuse` (`serve-http-listen.ts`: `serve_port_in_use` on a taken port; with `options.adoptServer` from status-only recovery it hands the app to the already-bound listener instead of binding). `buildServeHttpApp` builds one shared `ServeHttpContext` (sessions, nonce stores, limiters, `requireAdmin`, SSE fan-out, OAuth provider, resource verifier, non-`localOnly` op list) and mounts in registration order `serve-http-oauth.ts` (CORS gates, DCR TTL hook, confidential credentials, SDK router, discovery, owner login links, consent), `serve-http-metrics.ts` (`/health`, `/metrics`, probes), `serve-http-admin-api.ts` (`requireAdmin`, `/admin/login`, `/admin/api/*`, `/admin/events`), `serve-http-spa.ts`, `serve-http-mcp.ts` (`/mcp`; writes `[gbrain-serve] dispatch op=<name> args_sha256=<16 hex>` to stderr before each tool call so a stall's last line names its request, never the arguments) and `serve-http-webhooks.ts` (`/ingest`, which maps a legacy-authority `permission_denied` from submission to 409 `{error, message, hint, docs_url}` (`test/serve-http-ingest-legacy.test.ts`), answers 429 `ingest_backlog_full` when the 50-capture waiting cap hands back another job, and every other submission error to 500; `legacyIngestJob` answers a retried POST /ingest recorded under the pre-v0.60.609.0 key (no slug) when that row's slug matches (`test/serve-http-ingest-legacy-key-retry.test.ts`, `test/e2e/serve-http-ingest-legacy-key-retry-postgres.test.ts`); the queue key dedupes only while the job row exists, and the `ingest_capture` handler (`minions/handlers/ingest-capture.ts`) applies a capture once per job idempotency key for 90 days through `ingest_applied_captures` (`ingestion/applied-captures.ts`, migration v296), even after `gbrain jobs prune`: the row is inserted in the page write's transaction (`importFromContent`'s `beforeCommit`; a run that loses the insert rolls its write back and returns the stored result) or after a committed managed door write, whose per-attempt `request_id` (`ingest-capture:<key sha256 prefix>:<attempt>`) lets a committed request in `persistence_requests` count as applied and backfill the row; tombstones and keyless jobs are unchanged, and the purge phase drops rows past 90 days (`test/ingest-capture-applied-once.test.ts`, `test/e2e/ingest-capture-applied-once-postgres.test.ts`); `/webhooks/github`, which HMACs every body, with a throwaway key when there is no source or secret); `runServeHttp` keeps listen, banner, resolve-IPC binding and shutdown, and the façade re-exports moved symbols. Order is pinned by `test/serve-http-route-golden.test.ts` (AST) and `test/serve-http-route-runtime-golden.test.ts` (router stack). Size ceiling: `scripts/module-size-limits.tsv`.- `src/commands/serve.ts` — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Surface: stdio resolves `GBRAIN_SURFACE` > `--surface` > config > full (`resolveStdioSurface`; an invalid env value is ignored with a stderr line and handed to `startMcpServer` as `invalidSurfaceEnv`), `--http` resolves `--surface` > config > full and says on stderr that it ignores `GBRAIN_SURFACE`; both print `surface=<x> (source: …)` at start. `--http --token-ttl` goes through `parseTokenTtl` (`src/commands/auth.ts`) and refuses anything outside 60 to 7,776,000 seconds before the server starts. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` that first awaits `shutdownDelegatedSync()` (idempotent shared promise; a running delegated sync is aborted and settles its checkpoint against the live engine, with the cleanup deadline extended by `GBRAIN_SERVE_SYNC_SETTLE_MS` only while a job runs) and then releases the engine and the PGLite write-lock dir within 5 seconds (otherwise the lock is held indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnect, forcing a 5-minute stale-lock wait on next start). Watchdog reparent check is `getParentPid() !== initialParentPid` (the `=== 1` check missed the subreaper case under launchd/systemd). Bun's `process.ppid` cache is stale across reparenting ([oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable ...` stderr line so operators see the degraded mode. The idle maintenance sweep skips ticks while a delegated sync runs and drains deferred embeds afterwards (`maybeDrainDeferredEmbeds`); it targets the source `resolveMcpStdioSourceScope` resolves (GBRAIN_SOURCE > `.gbrain-source` dotfile > local_path > `sources.default` > …) — the SAME ladder as stdio dispatch and the startup sweep, so dotfile/local_path-scoped serves reconcile their in-session MCP writes (pinned by `test/serve-idle-sweep-source.serial.test.ts`). #7599: `lean` (`readOnlyServeSkipsResidentDuties`, a read-only stdio serve on Postgres; PGLite keeps both) turns off the idle sweep and `installResidentTickers` (facts drain, movement watch, on-commit head watcher). Both stdin activity listeners (the default-on idle sweep and the opt-in `--stdio-idle-timeout`) attach only AFTER the MCP SDK transport's own stdin listener — the sweep on its first interval tick, the idle timeout through the activator `installStdioLifecycle` returns, which `runServe` invokes once `startMcpServer` resolves (and which re-arms the countdown so boot time does not eat into the idle window) — so neither can flip stdin into flowing mode and consume a fast client's initialize frame before the SDK sees it. The boot-readiness deadline (`GBRAIN_SERVE_BOOT_TIMEOUT_SECONDS`, default 60, 0 disables) measures time WITHOUT boot progress: every `onBootPhase` change and every `noteForwardProgress()` note (`src/core/forward-progress.ts`; progress ticks and each completed stdio request via `trackStdioRpc` in `src/mcp/server.ts`) restarts the window, so a large brain's slow but advancing boot is never killed; only a full window without progress releases the engine and exits 1 with the `serve_boot_timeout` line (stuck phase, pool pressure, seconds since boot started). Pinned by `test/serve-stdio-lifecycle.test.ts` (boot-readiness block). #6543: after the pool stall watch, `recordServeTransport` (`src/commands/serve-health.ts`) reads the read pool's transport once (live `prepare`/`max_pipeline` with `max_pipeline_reason`, the doctor probe's `SHOW statement_timeout` and its ALTER ROLE fix when a configured timeout reads 0, `application_name` when Supavisor), logs one `[gbrain serve] transport` line and caches it for `/health?deep=1` (never re-read per probe). `GBRAIN_SERVE_RESTART_REQUIRED_EXIT_SECONDS=<n>` (>= 120, else `invalid_params` at start) arms `installRestartRequiredExit`: an in-process `restart_required` held continuously for n seconds (the hold resets when it clears) disconnects under a 5 s deadline and exits `SERVE_UNHEALTHY_EXIT_CODE` (69, `serve_unhealthy_exit`); a durable ledger (`serve-self-exit.json` under the gbrain home) suppresses the exit after 3 within an hour (`serve_self_exit_suppressed`, `self_exit_suppressed` in the deep 503). `probeDeepHealth` (`serve-http-metrics.ts`) answers 503 `persistence_restart_required` before its pooled probe, `pool_inflight_over_budget` or `deep_health_probe_failed` (`reason`), each with `fix`/`docs` from the registry and the cached `transport`, no request ids. Pinned by `test/serve-health-6543.test.ts` (a real `PersistenceConsumer` driven to restart_required).@@ -35,19 +35,3 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/commands/skillify.ts` + `src/core/skillify/{generator,templates}.ts` — `gbrain skillify scaffold <name>` creates all stubs for a new skill: SKILL.md, script, tests, routing-eval.jsonl, resolver entry, filing-rules pointer. `gbrain skillify check <script>` runs the 10-step checklist (LLM evals, routing evals, check-resolvable gate, filing audit) against a candidate skill before it lands.- `src/commands/skillpack.ts` + `src/core/skillpack/{bundle,scaffold,reference,migrate-fence,scrub-legacy,harvest,harvest-lint,copy,apply-hunks,diff-text,installer}.ts` — `install`/`uninstall` are not supported (they exit non-zero with a hint to the replacement). Surface: `scaffold` (one-time additive copy via `copyArtifacts` in `copy.ts`; refuses to overwrite; partial-state fills missing paired sources declared in SKILL.md frontmatter `sources:`), `reference` (read-only diff lens + `--apply-clean-hunks` two-way auto-apply via pure-JS unified-diff parser/applier in `apply-hunks.ts` + `diff-text.ts`; strict arguments through the `skillpack reference` row of `src/cli/strict-args.ts` (#5491): `--dry-run=true`, a typo or an unknown flag exits 2 before any write, the per-skill and `--all` grammars print separately, and `--all --apply-clean-hunks` refuses with the three-command update path; pinned by `test/skillpack-reference-strict-args.test.ts`), `migrate-fence` (one-shot strip of legacy fence; cumulative-slugs receipt → row-parsing fallback; preserves rows verbatim as user-owned routing), `scrub-legacy-fence-rows` (opt-in row cleanup with skill-present + non-empty-triggers gate), `harvest` (host→gbrain inverse with symlink-reject + canonical-path containment via a `validateUploadPath`-style gate + default-on privacy linter in `harvest-lint.ts` against `~/.gbrain/harvest-private-patterns.txt` plus built-in a built-in fork-name pattern + email + Slack-channel patterns; rollback on match). Paired-source declarations live in each SKILL.md's frontmatter `sources:` array (validated by `loadSkillSources` in `bundle.ts`). `autoDetectSkillsDir` (in `src/core/repo-root.ts`) has a `cwd_walk_up` tier ahead of `~/.openclaw/workspace` (`$OPENCLAW_WORKSPACE` precedence preserved). `gbrain skillpack check --strict` exits non-zero on drift (CI gate); top-level `gbrain skillpack-check` keeps exit-1-on-issues for cron. Companion editorial skill `skills/skillpack-harvest/SKILL.md` drives the genericization checklist. Doc: `docs/guides/skillpacks-as-scaffolding.md`. Test coverage across `test/skillpack-{copy,scaffold,reference,reference-apply,apply-hunks,migrate-fence,scrub-legacy,harvest,harvest-lint,frontmatter-sources}.test.ts` + 9-case E2E in `test/skillpack-flow.test.ts`. `installer.ts` + `test/skillpack-install.test.ts` remain because `gbrain skillpack diff` uses `diffSkill` from there.--- `src/commands/sources-lifecycle{,-args}.ts` — strict source mutation parsing before engine connection, with stable UUIDs and source incarnation preconditions. The private CLI administration envelope delegates to a live PGLite owner, retains request IDs after lost responses, and refuses preactivation administration or legacy hardening rather than changing worktrees through an uncoordinated writer. Managed purge names one archived source; the expiry worker separately coordinates each source. `test/persistence-source-cli.test.ts` executes real CLI subprocesses, checks replay after deletion/recreation, and verifies global page/lifecycle request identity.--- `src/commands/sources-reset-upstream.ts` — `gbrain sources reset-upstream <id> --expect-local <sha> --expect-upstream <sha> [--dry-run] [--json]`, the fix for `upstream_rewritten` / `upstream_unknown` (`src/core/sync-upstream-rewrite.ts`). Refuses a managed canonical worktree (`writer_coordinator_required`) and an in-progress git operation (`git_operation_in_progress`); under the checkout's commit lock (none on `--dry-run`) refuses with `reset_upstream_refused` a HEAD or upstream other than the expected SHAs (`expect_mismatch`), no upstream (`no_upstream`), tracked edits (`dirty_tree`, names the paths) and local commits not reachable from the upstream, the recorded baseline or any old or new value of the tracking ref's reflog (`local_commits`, names them and the backup ref). Otherwise writes `refs/gbrain/backup/<UTC stamp>-<sha12>` (create-only), runs `git reset --keep <upstream>`, verifies HEAD, records the baseline and clears the divergence; never fetches. Pinned by `test/pathological/s-58-upstream-rewrite.test.ts`.-- `src/commands/sources-retry-held.ts` — `gbrain sources retry-held <id> [--dry-run] [--json]`: records a re-attempt request for a source's held connector items (and, on a managed brain, their retry-pointer rows) or a re-screen request (op `sync-hold-retry`) for its held Git files, which the next sync adds to its manifest; nothing runs now, the receipt prints the sync to run. #6278: on a managed brain that follow-up sync is built by `gitFollowUpSync`: the unfinished managed cursor's resume arguments (`readManagedSyncBacklog`, `managedSyncResumeArgs`), so `--no-embed` and the other processing options carry over instead of starting a second cursor, else the arguments a `preparation_stalled` hold recorded (`meta.stall.sync_argv`), else `--source <id> --no-pull`. `preparation_stalled` holds count under `stalled` in the repair route and need no file repair: the receipt and help point to `gbrain sources writer status --source <id> --json` first. #6865: `--unindexed` (`retryUnindexed`) runs one bounded pass of the unindexed-file detector (`persistence/unindexed-files.ts`, 60 s) for a managed Git source, schedules the found files through the same re-screen request, stores the result doctor `unindexed_files` reads (continuing an incomplete stored scan) and prints the follow-up sync with `gbrain doctor --only unindexed_files --json` as its verify; `--dry-run` lists them and writes nothing; connector and company-brain sources refuse `invalid_params`; a complete pass with nothing found prints `No unindexed files for <id>.`, an incomplete one says the scan continues. Both modes build the receipt through `receiptOf`. Tests: `test/held-files-walkthrough.test.ts`, `test/managed-sync-preparation-stalled-holds.test.ts`, `test/unindexed-files.test.ts`.-- `src/commands/sources-resolve-path.ts` — GBRA-150: `gbrain sources resolve-path <id> <path...> [--stdin] [--json]`, the CLI face of op `sources_resolve_path` (`core/ops/sources.ts`). Paths come from argv or `--stdin` (one per line, at most `RESOLVE_PATHS_MAX` = 200); the human print is one line per path (state, slug, why, fix), `--json` the op's result. Read-only. Tests: `test/sources-resolve-path.test.ts`. Harness guide: `docs/guides/resolve-paths.md`.-- `src/commands/sources-set-path.ts` — `gbrain sources set-path <id> <path> [--force]`: unmanaged installations retain the legacy pointer repair and overlap check. Managed installations use the verified lifecycle rebind; force never bypasses ownership or manifest checks. Relative paths and MSYS spellings are normalized before routing. The `default_source_local_path` doctor check names this as the repair.--- `src/commands/sources-strategy.ts` — `gbrain sources set-strategy <id> <markdown|code|auto>` (#4899, PR #5955): writes `sources.config.strategy` through `updateSourceConfig` (single-key merge; every other config key survives), the value `sync --all`, autopilot, the dream cycle and the MCP `sync` op pass to the per-source sync when the caller names no `--strategy`; unset means `markdown`. The one writer of that key (no `sources add --strategy`, no op param). Connector kinds refuse it. [R26]: refuses with `sync_run_unfinished` (exit 1) while the source has an unfinished `managed-sync` cursor, naming the resume and the orphan repair, because the frozen manifest was classified under the old strategy; re-stating the current value is allowed. `sources list --json` and `listSources` report `strategy` (`null` when unset). Pinned by `test/sources-strategy.test.ts`.--- `src/commands/sources-add-next.ts` — GBRA-156: `noGrantFloorNext(engine, id)`, the federate step `sources add` prints when a new non-federated source leaves the no-grant floor (`localFederatedSourceIds(engine, 'default', 'seed_default')`, else `['default']`) holding no live page a remote read may show (private pages excluded when remote reads exclude them); returns `{ command, argv, why }` or null. One read, no write; `sources add` treats a failed read as no step.-- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit,refresh,set-calendar}`; `set-calendar` dispatches to `src/commands/sources-set-calendar.ts`, and `set-cr-mode` prints `contextualModeHint` after the write. `refresh` dispatches to `src/commands/sources-refresh.ts` (argument parsing, stdout text or `--json` result/refusal objects, exit 1 on refusal or an unfinished sync; protocol in `src/core/persistence/worktree-refresh.ts`, where a fence lock timeout (55P03) becomes `refresh_lock_timeout` naming the open transactions holding the fenced tables, and any other untyped throw becomes an `internal_error` refusal whose `why` carries the stack and whose fix names the refresh row it left for the next run to resume, GBRA-250). `current [--json]` calls `resolveSourceWithTier()` and prints `source_id`, `tier` (`flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail` (decision table in `skills/conventions/brain-routing.md`). `status [--json]` — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` from `src/commands/sync.ts`; `--json` emits stable `{schema_version: 1, sources, ...}` on stdout; filters input to `local_path IS NOT NULL AND archived IS NOT TRUE`. `audit <id> [--json]` — read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). The live `runStatus` health table gains a `BACKFILL` column between `EMBED` and `FAILS` (`active(N)` beats `queued(N)` beats `idle`, from `SourceMetrics.backfill_active` / `backfill_queued` in `src/core/source-health.ts`) so operators see deferred `embed-backfill` minion work after `sync --all` exits 0; `jobCountsBySource` in `source-health.ts` widens its `minion_jobs` SQL with two `COUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...)` aggregates (best-effort, all-0 on pre-minions brains). `remove` pre-checks OAuth-client referents (guided refusal via `formatClientReferentsBlock` — the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (`unhardenBrainRepo`: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact. `purge` runs the same referents pre-check before its hard DELETE. `add` without `--federated` or `--no-federated` stores the source non-federated and warns on stderr that a remote token or agent connection with no source grant cannot read it, naming `gbrain sources federate <id>` and `gbrain auth rescope --token <name> --sources <id>`; when the no-grant floor (the default source plus federated sources) then holds no live page it also prints `next: gbrain sources federate <id>` on stdout (`noGrantFloorNext` in `sources-add-next.ts`). `add --json` prints one object (`id`, `name`, `local_path`, `remote_url`, `federated`, and `next` with `command`/`argv`/`why` when that step applies) and moves the human lines to stderr (`test/sources-add-federation-warning.test.ts`). Pinned by `test/content-sanity.test.ts`, `test/import-file-content-sanity.test.ts`, `test/source-health.test.ts`.--- `src/commands/status-funnel.ts` — `gbrain status --funnel [--json] [--enable | --delete]` (D8 adoption funnel): engine-free, dispatched from `src/cli/main.ts` before any connect so it works before `gbrain init` and on a thin client; reads and writes only the local record through `src/core/funnel.ts`. Guide: `docs/guides/adoption-funnel.md`. Tests: `test/funnel.test.ts`, `test/funnel-cli.test.ts`.--- `src/commands/storage.ts` — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. The repo and the counted source come from `resolveRestoreTarget` (`src/core/restore-target.ts`, the `export --restore-only` rule), and a page is listed missing exactly when that export would restore it (`isDbOnly`, then `restoreFilePath` in the page source's `scannerSlugRootMode` and `nativeFileTarget`: the recorded `source_path`, else `<slug>.md`, absent; the repo walk answers without a per-page syscall whenever it saw that exact file as a regular file under real directories, and a recorded path starting with the repo's path inside its git checkout, which export re-anchors, takes the full check), so the printed `Use: gbrain export --restore-only --source <id> --repo <path>` (the path passed through `shellQuote`, `src/core/shell-quote.ts`) writes exactly the listed files into `--dir` (default `./export`, not the repo). An unsafe recorded path or a symlinked/ambiguous file target (`OperationError` from `nativeFileTarget`) becomes one warning per reason (lowest slug plus a count of further pages) plus the refusal export would give, printed even when no file is missing; a repo whose own owner (`resolveSourceForRepoPath`: dotfile, then longest registered `local_path`) is archived is refused too (unless an active source is registered at the repo's exact path, which owns it outright), although export's exact-path rule would fall back to the only active source; a refused rule prints `Cannot suggest a restore command:` and falls back to the `--repo` / default-source path, counting the active repo owner's pages (dotfile, then longest registered `local_path`) or every source's pages when no active source owns it. `restoreSource` / `restoreRefusal` carry this in the JSON result. Pinned by `test/storage-restore-hint.test.ts`.diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex 0f463a3672..2f897a7496 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -13,7 +13,7 @@ src/commands/jobs.ts 693 ratchet refactor wave 1 split target (W4 jobs, cut linesrc/commands/serve-http.ts 947 ratchet refactor wave 1 W4 split into serve-http-<area>.ts modules; ceiling rationale: docs/architecture/key-files/commands-5.md entry for src/commands/serve-http.ts; +2 agent operator wave A6: ServeHttpContext.noticeLedger (import + field; logic in core/notice-ledger.ts); +2 agent contract F1/F2: ServeHttpContext.readinessCache (probed readiness for initialize tails + gbrain://capabilities); +5 Lane D facts drain: serve --http starts the drain scheduler and stops it before the engine disconnect (logic in core/facts/drain-scheduler.ts); +2 agent-operator follow-up Item 3: ServeHttpOptions.adoptServer (status-only recovery hands its bound listener to listenOrRefuse; logic in serve-http-status.ts); +5 #6317: managed-sync movement watch start/stop (watch in persistence/sync-movement.ts); +1 #6589: imports loadServeConfigs (operations see the DB-plane config read once at startup; logic in mcp/serve-config.ts); +5 #6858 ask 6: on-commit HEAD watcher start/stop (watcher in core/serve-head-watcher.ts); +4 GBRA-199: serve --http --advertise parse, help line and pass-through (logic in mcp/surface.ts parseAdvertiseFlag); +6 GBRA-283 B3a: runServeHttp enters the local scope (withLocalScope; scope store in core/engine-scope-store.ts), the entry wrapper and its import cannot movesrc/core/ai/gateway.ts 4571 ratchet +5 GBRA-274 B1.3: withoutBudgetTracker export (runs a rollout hook outside the run's budget scope; the budget AsyncLocalStorage is gateway-private, so it cannot move); +2 #6997: ChatResult.fallbackReason (interface field + doc; recorded in ai/chat-fallback.ts); +17 #6607: lazy OpenAI model discovery (discoverBeforeOpenAICall, once per reconfigure, at the chat/expand/embed entry points; it re-resolves through the gateway-internal resolveGatewayModels, so it cannot move); +18 W11 item 3: embedQuery query-embedding cache lookup/store + clear at every model-cache clear site and the test transport seam (cache lives in ai/query-embed-cache.ts); +7 fix wave 5 (#5137): provider-key redactor on every upstream error path and the first embedding 401/403 report (logic in ai/key-redact.ts, ai/key-warnings.ts); +20 System One v1: decide() delegating export + transport seam, RerankInput.onMeta/RerankMeta, TypeSafe reranker delegation to ai/decide/rerank-adapter.ts; +3 #5585: reconfigureGatewayWithEngine records expansion/chat model provenance (ai/gateway-model-sources.ts) + clears it with gateway state; lowered #5585: tracker record sites collapsed onto ai/budget-record.ts (recordOnTracker + failedCallUsage) while adding ledger fields (requested model, purpose, failed, estimated, countsAsCall); +20: schema-rejection fallback in chat() (retry once schemaless on a json_schema 4xx, remember the recipe for the process — mirrors expand()); re-summed after merging master v0.50.0.0 with the wave (structured-output + pricing hunks); +48: #4863 ChatOpts.responseSchema → tolerant json_schema Output on the chat() generateText call for openai-compatible recipes that declare structured outputs (facts extraction on Ollama); +74: #3657 post-sunset rerank short-circuit (effective-model check, once-per-process audit row + stderr, injected-clock seam); grown v0.46.26.0 megawave: config-plane probes + halt telemetry (#3387 #4312); +21 lines (OpenAI Responses API reasoning-item echo: ChatBlock 'reasoning' variant, chat()/toModelMessages() round-trip); wave-k absorbs: OCR routing seam (#4107 class), config-snapshot reader block (#3980); +21: #4107 OCR-model routing (getImageOcrModel); +10: keyless OPENAI_BASE_URL embedding override (#4385); wave-k #3622 reimpl: GBRAIN_EMBED_MAX_BATCH_TOKENS env cap for no_batch_cap recipes; +9: DeepSeek judge thinking pin plumbed through chat opts (#4069); +master v0.48 merge true-up (reranker succession: no_key preflight, reranker_skipped stage, readiness); ranker wave v0.48.4.0: ChatOpts.temperature threaded to the transport (+15); +24: #4172 recipe-declared thinking headroom on the chat()/toolLoop() default — shared isThinkingModel predicate (Claude-5 regex || recipe thinking_by_default, fail-closed), also consumed by the subagent handler; +20: #4861 claude-cli expansion routed to the schemaless viaText path (the CLI adapter ignores responseFormat, so the native generateObject branch had no fallback); retired provider integration removed; safety wave: migration reranker input ceilings; lowered by fix wave 3 #5680 (embed request planning peeled to ai/embed-batch-plan.ts); +5 #4616: empty inputs skip the provider and every text/multimodal result is screened for zero-norm/NaN/Inf vectors (logic lives in ai/embedding-guard.ts); fix wave 3 integration: #4616 empty-input filtering feeds the shared #5680 batch planner; +3 #5691: embedQuery takes a per-request queryPrefix (no gateway state); +26 fix wave 7 (#5642): embed spend records bank provider-reported usage.tokens (all sub-batches reported → measured; chars-per-token estimate stays the fallback and the failure charge); +2 #5892: import + load-time call of installAiSdkWarningWriter (writer lives in ai/sdk-warnings.ts); +12 #5490: chat() delegation to ai/chat-fallback.ts when chat_fallback_chain is set (attempt loop lives there), ChatOpts.allowFallback + ToolLoopOpts.allowFallback fields and the toolLoop forward; +2 #5490 review: ChatResult.fallbackFrom (the call model a chat_fallback_chain answer replaced); +1 fix wave lane B3: configureGateway keeps chat_fallback_on_refusal (the refusal switch lives in ai/chat-fallback.ts); +18 fix wave 9 lane B: ChatOpts.thinking (#5331, per-route mapping lives in ai/thinking-off.ts), ChatOpts.timeoutMs + ToolLoopOpts.turnTimeoutMs and the turn forward (#4921); interface fields and their docs cannot move; +2 P8: GBRAIN_AI_CALL_LOG observer install in configureGateway (logic in ai/call-log.ts); +2 W9F item 5: thinkingOffOutputCap export shared by chat() and the judge preflight estimates (capability table lives in ai/thinking-off.ts); +3 A1 (wave 0): ChatResult.usage.reasoning_tokens (interface field + doc) and the chat() reasoning-subset read beside the cache counts; the ai-sdk importer allowlist is shrink-only, so the seam stays here); +9 GOOGLE_GENERATIVE_AI_BASE_URL: resolveNativeBaseUrl('google') + the three native-google construction sites (eval metering proxy); re-summed at the master 6ccf30c3e merge (4417 integration + 4423 master - 4408 base); lowered -5: the expand() prompt template peeled to ai/expansion-prompt.ts (buildExpansionPrompt) so the longmemeval call planner measures the same text; +5 #7176: expand() writes its chat_usage_log row (phase expansion) beside the budget record, and the guardedExpansion instance that names its invocation gateway.expand (ledger in ai/chat-usage.ts); +2 PR 4 (Wave B incremental derivation, transcripts) 4b round 4: chat() forwards a metered page step's providerRetries (BudgetTracker) as the SDK's maxRetries, so every attempt is one it reserved for; +24 GBRA-283 B2 (#8456): per-engine gateway state, the gw() accessor every config and model-cache read goes through (an embedded engine state or the process state), configure split into gatewayConfigOf, and configFor at the provider-resolution sites so a provider the engine lacks refuses before any call (state and refusal live in ai/engine-gateway.ts); +1 GBRA-283 B2: embedding diagnosis refuses a provider the embedded engine was not given; +1 GBRA-283 B2 (#8499 item 1): requireConfig refuses an unscoped call in a process holding embedded engines (engine_scope_missing; predicate in engine-scope.ts); +7 GBRA-283 B3a: every embed/chat/expand/rerank/toolLoop entry and requireConfig/configFor call refuseUnscoped (predicate and refusal in ai/engine-gateway.ts), the unscoped state, and the shrink/no_key/structured-output memory read through gw() (state in ai/engine-gateway.ts GatewayState)src/commands/sync.ts 566 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts); +1 #8007: SyncResult.storage_invalid_kept interface field (the keep lives in persistence/sync-discovery.ts); +2 GBRA-287: SyncResult.reexport_failures interface field (the re-export lives in commands/sync/full.ts); re-summed at the memory train #8481 merge with v0.60.576.0 (564 measured); +2 convergence PR A: SyncOpts.conversionBinding interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-pass.ts bindConversion)-src/core/cycle.ts 3348 ratchet +2 GBRA-130 C-2 (#6946): the sync phase reads an owned clone's pull block and reports it as pull_blocked (logic in url-clone.ts urlClonePullBlock/urlClonePullBlockSummary); +6 GBRA-130 A-1: the extract phase warns on failed or breaker-stopped stale drain pages (logic in cycle/connector-extract.ts staleDrainFailures); +1 W1 belief ledger: the extract phase runs the sampled belief drift reconcile (logic in belief/coverage-report.ts); +2 System One v1 S9: extract_facts phase tail runs the conflict sweep (ai/decide/sweep.ts) only when the slot is not off; +2 System One v1: purge phase prunes decision receipts (ai/decide/store.ts); grown v0.46.26.0 megawave (#4102 #2608) + master v0.46.26.0 cycle-start orphan recovery; +10 wave G #3974: cycle-sync uncommitted-drift warn; grown wave-g (#4416): synthesize_concepts source threading; grown wave-g (#4416): synthesize_concepts source threading (same fix absorbed as #4417 on wave-k); wave-k: ); +7: #4077 abort threading; +10: #4745 fullImplicitSourceCycle opt-in contract; +3: #4890 extract phase jsonMode/quiet + pages-not-links totals; +1: wave review — extract phase passes quiet:true + jsonMode:false; +1: extract_facts pagesFailed detail; +8 fix wave 3: one cycle date resolved in runCycle and threaded to synthesize/patterns/drift (C-15); +1 #5632: cycle sync resumes an unfinished managed cursor with its stored options; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +1 at the merge; -41 #5809: anyAbortSignal peeled to abort-signals.ts; +5 #5809: extract_atoms phase gets a real abort signal; +4 #5809: cycle_already_running names the lock holder; +7 #5255: cycle sync runs a managed pull request as no-pull and reports upstream_refresh; +1 #4578: a lock steal during the last selected phase reports lock_stolen, not aborted; +3 #5536: keyless embed phase clean-skips when embedding_disabled is set; +5 F4b: planner-stats refresh after the freshness phases (logic in planner-stats.ts); +2 agent operator wave A1: PhaseResult sibling code/fix; +20 #5876: Life Chronicle phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block; v0.60.46.0 merge: re-summed to the merged line count (3224); +3 retrieval feedback retention in the purge phase (logic in feedback/store.ts pruneRetrievalFeedback); 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); P3 x foundations-2 composition: deltas re-summed at the merge; 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); +15 GBRA-49 temporal typed edges: edge_contradictions phase registration + dispatch block + extract-phase relationship sweep call (logic in cycle/edge-contradictions.ts, link-relationships.ts); P3 x P1 (v0.60.57.0) composition: deltas re-summed at the merge; +1 fix wave 9 lane P (#5151): extract_facts phase passes drain so the facts watermark drain runs (logic in facts/reconcile-watermark.ts); +10 #6188: fence_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in cycle/fence-repair.ts); +19 fix wave 12 #6242: source cycles with brain-wide phases also take the shared gbrain-cycle lease (acquireDbCycleLock lease set, needsMaintenanceLease, skipBrainWidePhases, all-skipped deriveStatus case; lease logic in cycle/lock-set.ts); +16 #6377: content_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK), the REPAIR_LANE_PHASES dispatch helper shared with fence_repair (logic in cycle/content-repair.ts); +10 C4 pinned questions: standing_questions phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in questions/phase.ts); +11 #6696 (A5, A4): runCycle becomes the background-origin wrapper over runCycleUnscoped and adds the capacity notices (logic in persistence/admission-origin.ts, capacity-snapshot.ts) plus the CycleReport.notices interface field; +2 #6297: CycleOpts.directPhases (operator-named phases skip the patterns cadence); +5 #6865: unindexed_files phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its line in the REPAIR_LANE_PHASES dispatch helper (logic in cycle/unindexed-files.ts); +2 fix wave 20 #6954: the extract phase warns on extract_write_lost and its stale drain retries the pages left due; +5 #7241: CycleOpts.onPhaseStart and its best-effort call in timePhase, so the autopilot-cycle job writes its current phase to minion_jobs.progress; +1 GBRA-133: the extract phase passes the stale-drain budget to its targeted pass when sync gave no slug list; +7 #7630 (GBRA-207): the sync phase refuses sync_anchor_relative when no source matches its directory and default's local_path is relative; re-summed at the memory train #8481 merge with v0.60.576.0 (3348 measured)+src/core/cycle.ts 3351 ratchet +2 GBRA-130 C-2 (#6946): the sync phase reads an owned clone's pull block and reports it as pull_blocked (logic in url-clone.ts urlClonePullBlock/urlClonePullBlockSummary); +6 GBRA-130 A-1: the extract phase warns on failed or breaker-stopped stale drain pages (logic in cycle/connector-extract.ts staleDrainFailures); +1 W1 belief ledger: the extract phase runs the sampled belief drift reconcile (logic in belief/coverage-report.ts); +2 System One v1 S9: extract_facts phase tail runs the conflict sweep (ai/decide/sweep.ts) only when the slot is not off; +2 System One v1: purge phase prunes decision receipts (ai/decide/store.ts); grown v0.46.26.0 megawave (#4102 #2608) + master v0.46.26.0 cycle-start orphan recovery; +10 wave G #3974: cycle-sync uncommitted-drift warn; grown wave-g (#4416): synthesize_concepts source threading; grown wave-g (#4416): synthesize_concepts source threading (same fix absorbed as #4417 on wave-k); wave-k: ); +7: #4077 abort threading; +10: #4745 fullImplicitSourceCycle opt-in contract; +3: #4890 extract phase jsonMode/quiet + pages-not-links totals; +1: wave review — extract phase passes quiet:true + jsonMode:false; +1: extract_facts pagesFailed detail; +8 fix wave 3: one cycle date resolved in runCycle and threaded to synthesize/patterns/drift (C-15); +1 #5632: cycle sync resumes an unfinished managed cursor with its stored options; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +1 at the merge; -41 #5809: anyAbortSignal peeled to abort-signals.ts; +5 #5809: extract_atoms phase gets a real abort signal; +4 #5809: cycle_already_running names the lock holder; +7 #5255: cycle sync runs a managed pull request as no-pull and reports upstream_refresh; +1 #4578: a lock steal during the last selected phase reports lock_stolen, not aborted; +3 #5536: keyless embed phase clean-skips when embedding_disabled is set; +5 F4b: planner-stats refresh after the freshness phases (logic in planner-stats.ts); +2 agent operator wave A1: PhaseResult sibling code/fix; +20 #5876: Life Chronicle phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block; v0.60.46.0 merge: re-summed to the merged line count (3224); +3 retrieval feedback retention in the purge phase (logic in feedback/store.ts pruneRetrievalFeedback); 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); P3 x foundations-2 composition: deltas re-summed at the merge; 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); +15 GBRA-49 temporal typed edges: edge_contradictions phase registration + dispatch block + extract-phase relationship sweep call (logic in cycle/edge-contradictions.ts, link-relationships.ts); P3 x P1 (v0.60.57.0) composition: deltas re-summed at the merge; +1 fix wave 9 lane P (#5151): extract_facts phase passes drain so the facts watermark drain runs (logic in facts/reconcile-watermark.ts); +10 #6188: fence_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in cycle/fence-repair.ts); +19 fix wave 12 #6242: source cycles with brain-wide phases also take the shared gbrain-cycle lease (acquireDbCycleLock lease set, needsMaintenanceLease, skipBrainWidePhases, all-skipped deriveStatus case; lease logic in cycle/lock-set.ts); +16 #6377: content_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK), the REPAIR_LANE_PHASES dispatch helper shared with fence_repair (logic in cycle/content-repair.ts); +10 C4 pinned questions: standing_questions phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in questions/phase.ts); +11 #6696 (A5, A4): runCycle becomes the background-origin wrapper over runCycleUnscoped and adds the capacity notices (logic in persistence/admission-origin.ts, capacity-snapshot.ts) plus the CycleReport.notices interface field; +2 #6297: CycleOpts.directPhases (operator-named phases skip the patterns cadence); +5 #6865: unindexed_files phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its line in the REPAIR_LANE_PHASES dispatch helper (logic in cycle/unindexed-files.ts); +2 fix wave 20 #6954: the extract phase warns on extract_write_lost and its stale drain retries the pages left due; +5 #7241: CycleOpts.onPhaseStart and its best-effort call in timePhase, so the autopilot-cycle job writes its current phase to minion_jobs.progress; +1 GBRA-133: the extract phase passes the stale-drain budget to its targeted pass when sync gave no slug list; +7 #7630 (GBRA-207): the sync phase refuses sync_anchor_relative when no source matches its directory and default's local_path is relative; re-summed at the memory train #8481 merge with v0.60.576.0 (3348 measured); +3 GST-1856: the purge phase drops applied-capture records past 90 days (logic in ingestion/applied-captures.ts purgeStaleAppliedCaptures)src/core/cycle/synthesize.ts 3244 ratchet +3 #7805: each captured session synthesizes only into its own frozen source (routing import, the outer wrapper, the routing parameter; logic in cycle/synthesize-source-routing.ts); +26 System One v1 S7/S8: triage and grounding decide delegation (triage-decide.ts, grounding-decide.ts); merged waves; grown v0.46.26.0 megawave + master rolling lease via shared throttled renewer; grown wave-g (#4506): cycle summaries can stay out of the source repo; +18: expired-verdict sweep + DeepSeek judge thinking pin (#4069); +47: #4077 cooperative-abort threading; +110: eval fix wave F6 spend surfacing (withChatPhase wrap, triage/child token telemetry, zero-page counter); +32: F1b quote-verify call site + kill switch; +2: F1a/F3/F4a OUTPUT POLICY rules; +59: F2 verified-segment rescue (gate at report construction, rescue knobs, telemetry); +19: rubric v2 peak-scoring + docblock rescue branch; +50: #4337 summary link-sample cap + dream_created_cycle_date preserve-on-rerun; +5: #4348 resolveCycleDate wiring (local-day summary bucketing); +3 round-3 rebase true-up; +55: #4805 synth-v2 completion skip (findSynthV2Completion + prefix-parameterized loadSuccessfulSynthesisKeys); +10: #4886 mode-aware buildSynthesisPrompt (oneshot children drop tool-use/final-prose guidance); #5105: small legacy writer/refusal and shared revision CAS delegates; +4: merge of #5666 durable synthesis completions with the v0.59.18.0 verify-epoch wiring; +7 #5413/#5471 discovery exclusions (self-capture session ids, dream output directories); +6 fix wave 2 (v0.60.5.0) integration: W7 paid-loop breaker check before submission and released-key daily-cap count on top of the W1b discovery exclusions; +6 v0.60.5.0 master-merge true-up: the wave-2 growth (+13 W1b, +6 W7) re-summed over master v0.59.18.0 at 3222 lines with no slack; +76 fix wave 3: per-run USD gate + fail-closed daily cap (C-13), child first-write ownership for stamp/reverse-write (C-8, #5685), cycle-date prompt hint (C-15), opt-in attribution rule (#5425); fix wave 3 x fix wave 2 (v0.60.5.0) composition: +19 at the merge; +2 #5806 triageNow clock seam forwarded to triage (interface field + call-site option); +4 #5854: a postprocess or summary publish still pending after its wait is deferred (publish_pending warn, no cooldown stamp; helper in synthesize-postprocess.ts); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); lowered to 3200 by wave 9 lane W #5145: completion-key helpers (findLegacyCompletion, findSynthV2Completion, loadSuccessfulSynthesisKeys) peeled to synthesize-completion.ts with the new pre-triage partition; +13 P2 extraction.date_grounding: transcript observation date line instead of a run-date "Today's date" (config field + prompt arg); +1 fix wave 11 #6160: oneshot_task_prefixes on the child job data; +18 fix wave 12 #6069: unreliable-verdict backoff in runTriagePass (report code, pass backoff count, triage details entry; marker logic in triage-backoff.ts); +1 fix wave 12 #6242: SYNTH_PHASE_FAIL keeps an OperationError's suggestion and code (OperationError import); +3 wave 17 #6616: slug discovery and postprocessing share the canonical committed child outputs (loadCommittedSynthesisOutputs in synthesize-postprocess.ts; one call-site line, one parameter, one lookup line); lowered to 3224 by wave 17 #6642: the summary provenance section moved to summaryProvenanceLines in dream-provenance.ts; +6 GBRA-130 B-6 (#6860): surrogate-safe triage cuts (truncateUtf8) and the per-transcript verdict-cache write call site (logic in triage-backoff.ts VerdictCacheWrites); +2 GBRA-161 PR 4 (#7065): the synthesize phase activates each synthesized transcript's outputs as its generation (activateSynthesisGenerations import and call site; logic in cycle/synthesize-generations.ts); +8 GBRA-161 PR 5 (S-2): the dream summary publishes as a single-item dream_summary generation bound to its pre-run revision (publishSinglePage with a generation_fenced skip); +1 GBRA-60: a managed synthesis child carries dream_output (verify and stamp before put_page; cycle/synthesize-child-verify.ts)src/core/cycle/extract-atoms.ts 1658 ratchet crossed the 1500 unlisted cap in the atoms/extraction fix wave: +21 embed-aware cost gate (#4878), +65 atoms-array anchor scan (#4913); peel candidate: the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) into extract-atoms-parse.ts; +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) is peeled into extract-atoms-parse.ts (-193, fix wave 10 lane B); +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); +17 #5031: same-page case-insensitive atom adoption fallback (one page-atom scan per source page per run); +12 P2 atomsPrompt helper: extraction.date_grounding source observation date + rule in the atoms prompt; +4 GBRA-75: octet_length-bounded body-length predicate (minCompiledTruthChars) for discovery and the backlog count; GBRA-130 A-42 (#6804): dot-led atom stems and the legacy-slug adoption guardsrc/core/engine.ts 2874 ratchet +5 #7191: findPages/countPagesBy signatures and type imports (SQL in engine-sql/page-lookup.ts); grown v0.46.26.0 megawave: relational/visibility/usage contract surface (#4352 #4218); grown wave-g (#4524 #4527): orphans-mode contract + removeLink deleted count; wave-k: +4 fail-closed TrajectoryOpts.remote contract doc (#3855), +2 FactListOpts.unconsolidatedOnly (#4070), +14 getAllConfig bulk-read contract (#3980); grown v0.46.26.0 megawave: relational/visibility/usage contract surface (#4352 #4218); grown wave-g (#4524 #4527): orphans-mode contract + removeLink deleted count; wave-k: +4 fail-closed TrajectoryOpts.remote contract doc (#3855), +2 FactListOpts.unconsolidatedOnly (#4070), +14 getAllConfig bulk-read contract (#3980); +1: getBacklinkCounts page-id contract doc (#4380); +9: FactListOpts.grep SQL pre-limit contract (#3851); +13: dream-verdict TTL contract (#4069); +33: softDeletePages batch soft-delete contract (#4587); +8: #4592 scoped getStats/getHealth contract doc; +5 read-time TTL validity contract docs (active vs history read semantics); +12: #4280 findOrphanPages type/quarantined metadata contract; +14: traversePathsDetailed row-cap contract (ship-review); +12: resolveSlugWithAliasDetailed owning-source contract (ship-review) | merged v0.48.1.0: master #4788 + wave traversePathsDetailed/resolveSlugWithAliasDetailed; master: | merged v0.48.1.0 true-up; corrective release: read-policy signatures and Chronicle scope; +7: #4882 listFactsSince entitySlug/sessionId composition doc; #5105: canonical page state shared module and transaction/snapshot delegates; #5105: guarded derived projection preparation and read seals; #5105: resident-stop registration and direct transaction contract; #5105: +1 raw preservation contract for legacy engine migration; merged company ingestion/schema routing changes; safety wave: fact embedding model and text identity contract; +5 write-path wave: findDuplicatePage excludeSlug + id-first ranking contract; +2 fw3 A14: addTag tagSource provenance contract; evidence delivery: getChunkWindows interface member + type import (+3); #5761: +4 markPagesAttendanceBlocked member + countStalePagesForExtraction attendance option; +4 #5888: FactListOpts.fingerprint + FactRow.fact_fingerprint contract (hot-memory duplicate collapse); +1 GBRA-49: LinkReadScope temporal opts on the link read contract; +4 #5984: upsertChunks sealChunkerVersion and createVersion preimage contract docs (round-trip diet); v0.60.58.0 merge: #5984 diet growth re-summed over v0.60.57.0 (2683 measured); +10 P7 relationalChainHop member + ChainHopOpts/ChainHopEdge imports (SQL in search/read-enrichment.ts); +14 contributor audit wave P0: listFactsKeyset member (delta facts keyset read contract) + FactRow.created_at_iso; +1 #5375: get_versions limit/include_body type import (GetVersionsOpts, PageVersionRows); 3 P2 speaker attribution: FactAttribution type + FactRow/NewFact.attributed_to fields + findCandidateDuplicates attributedTo option contract; +4 #5984 lane apply diet: setPageAliases inline option and upsertChunks pageId contract docs; +10 #6278: executeRaw timeoutMs contract (transaction-local statement_timeout for bounded preparation reads); +2 GBRA-69: replaceDerivedLinksBatch member; master v0.60.136.0 merge into #5575 memory trust (both sides kept); +6 GBRA-67: readPageSnapshotsBatch interface member (BrainEngine signatures cannot move); +1 GBRA-75 wave 10: optional findDuplicatePages member (batched findDuplicatePage, contract in engine-sql/pages.ts); +4 FactListOpts/TrajectoryOpts.excludePrivate fact-provenance visibility contract (interface fields cannot move); +18 GBRA-76: TransactionOptions (abort signal) interface and the transaction/transactionDirect signatures (abort logic in postgres-engine/transaction-abort.ts); +2 W14 P3.11 (#4381): LinkBatchInput.resolution_type on top of W13; +1 C2: findCandidateDuplicates arm:'keyword' contract doc (facts.candidate_fusion); re-summed at the master 6ccf30c3e merge (2708 integration + 2707 master - 2707 base); +5 Cat 40 Hard fix wave F2 (#6271): countKeywordPages/searchKeywordPages BrainEngine signatures (SQL in engine-sql/keyword-pages.ts); +6 W15 P1.2 (#6454): countStaleFrontmatterKeyPages/listStaleFrontmatterKeyPages/stripStaleFrontmatterKeys contract (SQL in engine-sql/pages.ts); +2 large pages Wave A sub-a: getChunkPage signature (get_chunks paging; SQL in engine-sql/chunks.ts); +5 #6696 B3: readEmbeddingStateCounts/listEmbeddingStateRows signatures + type import (SQL in engine-sql/chunks.ts); +2 Cat 40 next wave part 1: non-ranking link types on the relationalFanout/relationalChainHop/getBacklinkCounts/getAdjacencyBoosts contract docs (policy in link-ranking-policy.ts; re-summed at the 0.60.242.0 merge: master 2794 + 2); +26 Wave B PR 1: segmented page storage members (allocateBodyOccurrenceIds..sweepSegments) + page-body type import (SQL in engine-sql/page-body.ts); merge v0.60.248.0: Wave B PR 1 and master growth compose at 2822; +1 LHF wave 2 item 1: executeRaw opts gain jitOff/prepare (the JIT-off typed-search type check; logic in ops/search-type-filter.ts); +9 PR2 fix lane X: BrainEngine.keywordBoundedRead, one unscoped read under the keyword statement bound for the segmented-region read (interface member and its doc cannot move): 2823 + 9; +9 GBRA-177: TakesListOpts.oldestFirst/sinceOnOrBefore/ungradedFirst for the grade_takes window (interface fields cannot move; SQL in engine-sql/takes.ts): 2832 + 9; +6 GBRA-156: TransactionOptions.pipelineFirst (the declaration goes out with BEGIN behind a transaction-mode pooler; predicate in persistence/protocol.ts); interface fields cannot move; +1 GBRA-161 PR 2: deprecation notes on insertFacts({ deleteForPageFirst }), deleteFactsForPage and expireFact (interface docs cannot move); +7 GBRA-156: TransactionOptions.pipelineEarly (search read bodies go out behind BEGIN; predicate in postgres-engine/search-settings.ts); interface fields cannot move; +2 GBRA-156: TransactionOptions.readOnly (search read transactions open BEGIN READ ONLY; 2858 measured); grown v0.46.26.0 megawave: relational/visibility/usage contract surface (#4352 #4218); re-summed at the Wave B PR 4 merge with v0.60.452.0 (2866 measured); re-summed at the memory train #8481 merge with v0.60.576.0 (2874 measured)diff --git a/src/commands/serve-http-webhooks.ts b/src/commands/serve-http-webhooks.tsindex 18798a715d..ac8b47d050 100644--- a/src/commands/serve-http-webhooks.ts+++ b/src/commands/serve-http-webhooks.ts@@ -567,10 +567,12 @@ async function handleIngest(...(writer ? { writer } : {}),},{- // Idempotency: same content from the same client within the- // queue's lifetime is a single job. Different content gets- // different jobs. Daemon-side dedup catches the 24h window;- // the queue-level idempotency catches simultaneous retries.+ // Idempotency: the same content from the same client to the same+ // slug is one capture. While its job row exists the queue hands+ // back that job; the ingest_capture handler applies a capture once+ // per key for 90 days, even after `gbrain jobs prune` removed its+ // job row (GST-1856), so a replay never rewrites the page. Different+ // content gets different jobs. Daemon-side dedup catches the 24h window.// The effective write source is part of the key: a client rescoped// from source X to Y must land a NEW capture in Y rather than being// deduped against its old X-bound job.diff --git a/src/core/cycle.ts b/src/core/cycle.tsindex a51026c05c..d4d8b785ac 100644--- a/src/core/cycle.ts+++ b/src/core/cycle.ts@@ -1821,6 +1821,8 @@ async function runPhasePurge(engine: BrainEngine, dryRun: boolean): Promise<Phas// System One: decision receipts past decide.receipts.retention_days (reported only when rows were pruned).let purgedDecisionReceipts = 0;try { purgedDecisionReceipts = await (await import('./ai/decide/store.ts')).pruneReceiptsForCycle(engine); } catch { /* pre-v179 brain */ }+ // GST-1856: applied-capture records past 90 days (ingestion/applied-captures.ts; 0 on a brain before v296).+ const purgedAppliedCaptures = await (await import('./ingestion/applied-captures.ts')).purgeStaleAppliedCaptures(engine);let purgedFeedback = { events: 0, weights: 0 };try { purgedFeedback = await (await import('./feedback/store.ts')).pruneRetrievalFeedback(engine, (await (await import('./feedback/settings.ts')).loadFeedbackSettings(engine)).eventRetentionDays); } catch { /* pre-v207 brain */ }return {@@ -1848,6 +1850,7 @@ async function runPhasePurge(engine: BrainEngine, dryRun: boolean): Promise<Phaspurged_batch_retry_audit_files_count: purgedBatchRetryAuditFiles,purged_volunteer_events_count: purgedVolunteerEvents,...(purgedDecisionReceipts > 0 ? { purged_decision_receipts_count: purgedDecisionReceipts } : {}),+ ...(purgedAppliedCaptures > 0 ? { purged_applied_captures_count: purgedAppliedCaptures } : {}),...(purgedFeedback.events + purgedFeedback.weights > 0 ? { purged_retrieval_feedback_events_count: purgedFeedback.events, purged_retrieval_weights_count: purgedFeedback.weights } : {}),},};diff --git a/src/core/cycle/inline-drain.ts b/src/core/cycle/inline-drain.tsindex fbac9f0887..69d519b597 100644--- a/src/core/cycle/inline-drain.ts+++ b/src/core/cycle/inline-drain.ts@@ -268,6 +268,7 @@ async function drainLoop(name: job.name,data: job.data,attempts_made: job.attempts_made,+ idempotency_key: job.idempotency_key,// #4077: fold the phase's cancellation into the per-job signal so an// in-flight handler (mid-LLM-call) stops without waiting for its own// timeout; `abort` stays the loop-local source for timedOut below.diff --git a/src/core/deletion-inventory.ts b/src/core/deletion-inventory.tsindex fb1911bcb4..9a28847f38 100644--- a/src/core/deletion-inventory.ts+++ b/src/core/deletion-inventory.ts@@ -95,6 +95,7 @@ const ROWS: readonly Row[] = [['facts', 'swept', 'fact', 'source_id entity_slug fact kind visibility notability context source source_session embedding source_markdown_slug claim_metric claim_unit claim_period event_type dimension value value_hash dim_status embedding_model embedded_text_hash write_principal_kind write_principal_id last_write_principal_kind last_write_principal_id attributed_to trust_tier write_origin source_window'],['files', 'swept', 'page_files', 'source_id page_slug filename storage_path mime_type content_hash metadata'],['gbrain_cycle_locks', 'out_of_scope', 'no_memory_text', 'id holder_host'],+ ['ingest_applied_captures', 'out_of_scope', 'no_memory_text', 'idempotency_key source_id slug result'],['ingest_log', 'out_of_scope', 'probed_reported', 'source_id source_type source_ref pages_updated summary writer_version', 'summary pages_updated'],['link_edge_proposals', 'out_of_scope', 'probed_reported', 'source_id link_type evidence_hash status model generated_line detail', 'generated_line detail'],['link_relationships', 'out_of_scope', 'no_memory_text', 'link_type scope source_id semantics status_now evidence_hash'],diff --git a/src/core/ingestion/applied-captures.ts b/src/core/ingestion/applied-captures.tsnew file mode 100644index 0000000000..9eef442599--- /dev/null+++ b/src/core/ingestion/applied-captures.ts@@ -0,0 +1,78 @@+/**+ * GST-1856: the applied-capture record (`ingest_applied_captures`, migration v296).+ *+ * An `ingest_capture` job whose write landed records it under the job's idempotency key, in the page write's own+ * transaction on an unmanaged brain and right after the door commits on a managed one. Any later run of the same key (a+ * stalled attempt re-run after its write landed, a POST /ingest replayed after `gbrain jobs prune` removed the job+ * row) returns the stored result and writes nothing, so it cannot revert a newer edit. Each capture has its own key, so+ * ordered captures to one slug all apply. A managed write's door request carries a client request_id under+ * `appliedCaptureRequestPrefix(key)`, so a committed request found without its row (the worker died between the commit+ * and the insert) counts as applied and backfills the row. Rows live 90 days, at least as long as the job rows the+ * queue-level dedupe relied on (`gbrain jobs prune` defaults to 30 days).+ */++import { createHash } from 'node:crypto';+import type { BrainEngine } from '../engine.ts';++export const APPLIED_CAPTURE_TTL_DAYS = 90;++type Exec = Pick<BrainEngine, 'executeRaw'>;++const jsonValue = (value: unknown): unknown => typeof value === 'string' ? JSON.parse(value) : value;++/** The client request_id prefix of a managed capture's door writes; the attempt number follows it. */+export function appliedCaptureRequestPrefix(key: string): string {+ return `ingest-capture:${createHash('sha256').update(key).digest('hex').slice(0, 48)}:`;+}++/**+ * The stored result of the capture recorded under `key`, or null. With `journal` (a managed brain), a committed door+ * request of this capture with no row yet counts too: its row is backfilled from `journal.result(outcome)`.+ */+export async function readAppliedCapture<T>(engine: Exec, key: string,+ journal?: { sourceIds: string[]; slug: string; result: (outcome: Record<string, unknown>) => T }): Promise<T | null> {+ const [row] = await engine.executeRaw<{ result: unknown }>('SELECT result FROM ingest_applied_captures WHERE idempotency_key = $1', [key]);+ if (row) return jsonValue(row.result) as T;+ if (!journal) return null;+ const [request] = await engine.executeRaw<{ source_id: string; outcome: unknown }>(+ `SELECT source_id, outcome FROM persistence_requests+ WHERE source_id = ANY($1::text[]) AND slug = $2 AND state = 'committed' AND client_request_id LIKE $3+ ORDER BY sequence LIMIT 1`,+ [journal.sourceIds, journal.slug, `${appliedCaptureRequestPrefix(key)}%`]);+ if (!request) return null;+ const result = journal.result((jsonValue(request.outcome) ?? {}) as Record<string, unknown>);+ await recordAppliedCapture(engine, key, request.source_id, journal.slug, result);+ return result;+}++/** Records the capture; false when a run of the same key already did (inside a write transaction: roll the write back). */+export async function recordAppliedCapture(exec: Exec, key: string, sourceId: string, slug: string, result: unknown): Promise<boolean> {+ const rows = await exec.executeRaw(+ `INSERT INTO ingest_applied_captures (idempotency_key, source_id, slug, result) VALUES ($1, $2, $3, $4::text::jsonb)+ ON CONFLICT (idempotency_key) DO NOTHING RETURNING 1`,+ [key, sourceId, slug, JSON.stringify(result)]);+ return rows.length > 0;+}++/** Stores the final result once the import returned, inserting the row when the import wrote nothing. */+export async function saveAppliedCapture(exec: Exec, key: string, sourceId: string, slug: string, result: unknown): Promise<void> {+ await exec.executeRaw(+ `INSERT INTO ingest_applied_captures (idempotency_key, source_id, slug, result) VALUES ($1, $2, $3, $4::text::jsonb)+ ON CONFLICT (idempotency_key) DO UPDATE SET result = EXCLUDED.result`,+ [key, sourceId, slug, JSON.stringify(result)]);+}++/** 90-day GC, called from the dream cycle's purge phase. Best-effort: 0 on any failure (a brain before v296 has no table). */+export async function purgeStaleAppliedCaptures(engine: Exec, ttlDays = APPLIED_CAPTURE_TTL_DAYS): Promise<number> {+ try {+ const rows = await engine.executeRaw<{ count: string | number }>(+ `WITH deleted AS (+ DELETE FROM ingest_applied_captures WHERE applied_at < now() - ($1 || ' days')::interval RETURNING 1+ )+ SELECT count(*)::text AS count FROM deleted`,+ [String(ttlDays)]);+ return Number(rows[0]?.count ?? 0);+ } catch {+ return 0;+ }+}diff --git a/src/core/minions/handlers/ingest-capture.ts b/src/core/minions/handlers/ingest-capture.tsindex 7d071168d2..ca6c600327 100644--- a/src/core/minions/handlers/ingest-capture.ts+++ b/src/core/minions/handlers/ingest-capture.ts@@ -47,6 +47,7 @@ import { authorizePageVisibility, excludesPrivateWrites } from '../../persistencimport type { Principal, WriteAuthority } from '../../persistence/model.ts';import { assertMemberPageWriter, assertScopePageWriter } from '../../access/member-facts.ts';import { validateSlug } from '../../utils.ts';+import { appliedCaptureRequestPrefix, readAppliedCapture, recordAppliedCapture, saveAppliedCapture } from '../../ingestion/applied-captures.ts';export interface IngestCaptureResult {slug: string;@@ -71,6 +72,32 @@ export function defaultSlugForEvent(event: IngestionEvent, now: Date = new Date(return `inbox/${y}-${m}-${d}-${hashPrefix}`;}+/**+ * A foreign_key_violation (23503, cf. oauth-provider.ts:1053) on the PAGES source FK. SQLSTATE alone is too broad: a+ * violation from chunks/tags/versions would be misread as "source unavailable" and silently rewritten to the default+ * partition, masking a real integrity failure.+ *+ * Matching any message mentioning "source" is not tight enough to carry a routing decision: many tables reference+ * sources(id) (files_source_id_fkey, facts_source_id_fkey, code_edges, calibration_profiles, …), so a genuine integrity+ * failure on one of those would be rewritten as reason:'fk_violation' — exactly the masking this check exists to+ * prevent. maybeRewriteSourceFkError (commands/capture.ts) can afford the looser match because it only formats a human+ * hint; here the verdict moves data. Prefer the driver's structured constraint/table fields (postgres.js and PGLite both+ * expose them) and fall back to the message for wrapped errors.+ */+function isPagesSourceFkViolation(err: unknown): boolean {+ const errMsg = err instanceof Error ? err.message : String(err);+ const fkErr = err as { code?: string; constraint?: string; table?: string };+ const namesPagesSourceFk =+ (typeof fkErr.constraint === 'string' && /^pages_source_id_fk/.test(fkErr.constraint)) ||+ (fkErr.table === 'pages' && typeof fkErr.constraint === 'string' && fkErr.constraint.includes('source')) ||+ /pages_source_id_fk/.test(errMsg) ||+ (errMsg.includes('foreign key constraint') && errMsg.includes('"pages"'));+ return fkErr?.code === '23503' && namesPagesSourceFk;+}++/** Rolls back a capture's page write when another run of its idempotency key already recorded the capture (GST-1856). */+class CaptureAlreadyApplied extends Error {}+/** The token or OAuth client the POST /ingest route stored on the job, or undefined (checked as a guest). */function webhookPrincipal(writer: unknown): Principal | undefined {const principal = (writer as { principal?: { kind?: unknown; id?: unknown } } | null | undefined)?.principal;@@ -229,13 +256,23 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {if (rows.length > 0) sourceId = event.source_id;}- const webhookWrite = untrustedPayload && event.source_kind === 'webhook' && !isTombstone;- const checkWriter = (sid: string) => webhookWrite ? assertWebhookWriterMayReplace(engine, data.writer, sid, slug) : Promise.resolve();- await checkWriter(sourceId ?? 'default');-// G6: a managed brain refuses direct writes, so its captures and tombstones go through the door (put_page /// delete_page), which publishes the file before the row.const managed = await managedPersistenceEnabled(engine);+ const captured = (status: IngestCaptureResult['status'], chunks: number): IngestCaptureResult => ({ slug, status, chunks,+ untrusted_payload: untrustedPayload, source_kind: event.source_kind, source_uri: event.source_uri, ...(sourceFallback ? { source_fallback: sourceFallback } : {}) });+ // GST-1856: a capture applies once per job idempotency key. A later run of the key (a stalled attempt re-run after+ // its write landed, a POST /ingest replayed after `gbrain jobs prune`) returns the first run's result and writes+ // nothing, so it never reverts a newer edit. Tombstones and jobs queued without a key keep their old behavior.+ const appliedKey = !isTombstone && job.idempotency_key ? job.idempotency_key : null;+ const managedOutcome = (outcome: { noop?: boolean; chunks?: number }) => captured(outcome.noop ? 'skipped' : 'imported', Number(outcome.chunks ?? 0));+ const applied = appliedKey ? await readAppliedCapture<IngestCaptureResult>(engine, appliedKey,+ managed ? { sourceIds: [...new Set([sourceId ?? 'default', 'default'])], slug: slug.toLowerCase(), result: managedOutcome } : undefined) : null;+ if (applied) return applied;++ const webhookWrite = untrustedPayload && event.source_kind === 'webhook' && !isTombstone;+ const checkWriter = (sid: string) => webhookWrite ? assertWebhookWriterMayReplace(engine, data.writer, sid, slug) : Promise.resolve();+ await checkWriter(sourceId ?? 'default');const door = (operation: 'put_page' | 'delete_page', sid: string, params: Record<string, unknown>) => submitPageMutation({ engine, config: { engine: engine.kind } as OperationContext['config'], remote: false, dryRun: false, sourceId: sid, logger: { info() {}, warn() {}, error() {} } },{ operation, params: { ...params, slug, source_id: sid } });@@ -289,24 +326,32 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {// tool_output stores and gates the capture external_untrusted (#5575 A3), as importOpts does unmanaged.// A capture replaces the page as it stands, so it binds to the revision read just before (the door requires one to// replace an existing page); a write landing in between refuses revision_conflict and the job retries.- const put = async (sid: string) => {+ // GST-1856: each attempt of a keyed capture sends its own request_id under the capture's prefix, so the applied check+ // above finds a committed attempt in the door's journal even when the worker died before recording it.+ const requestId = appliedKey ? `${appliedCaptureRequestPrefix(appliedKey)}${job.attempts_made}` : undefined;+ const put = async (sid: string, request?: string) => {const snapshot = await engine.readPageSnapshot(slug, { sourceId: sid, includeDeleted: true });return door('put_page', sid, { content: event.content, content_origin: 'tool_output', source_kind: event.source_kind, source_uri: event.source_uri,- ingested_via: 'ingest_capture', ...(snapshot ? { expected_revision: snapshot.revision } : {}),+ ingested_via: 'ingest_capture', ...(snapshot ? { expected_revision: snapshot.revision } : {}), ...(request ? { request_id: request } : {}),// #2044/#4548: as `remote: untrustedPayload` does unmanaged, an untrusted body keeps the non-world facts rows and takes rows its sender could not see....(untrustedPayload ? { keep_hidden_rows: true } : {}) });};let response: Record<string, unknown>;try {- response = await put(sourceId ?? 'default');+ response = await put(sourceId ?? 'default', requestId);} catch (err) {+ // A re-run of the same attempt whose earlier request did not commit and read another revision: retry it as a new+ // attempt (a new request_id) instead of letting the deterministic id end the job as a refusal.+ if (requestId && err instanceof OperationError && err.code === 'idempotency_conflict') {+ throw new Error(`ingest_capture: attempt ${job.attempts_made} of this capture was already submitted for an older revision; the next attempt retries it`);+ }// The door refuses a source that vanished or was archived after the pre-check (the managed form of the FK retry).const vanished = sourceId !== undefined && err instanceof OperationError && err.code === 'source_changed'? await engine.executeRaw<{ archived: boolean | null }>('SELECT archived FROM sources WHERE id = $1', [sourceId]) : null;if (!vanished || (vanished.length > 0 && vanished[0]?.archived !== true)) throw err;sourceFallback = { requested: sourceId!, effective: 'default', reason: vanished.length ? 'archived' : 'fk_violation' };await checkWriter('default');- response = await put('default');+ response = await put('default', requestId && `${requestId}:default`);}if (sourceFallback) {console.error(@@ -314,54 +359,38 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {`(${sourceFallback.reason}); wrote under default`,);}- const outcome = (response.outcome ?? {}) as { noop?: boolean; chunks?: number };- return {- slug,- status: outcome.noop ? 'skipped' : 'imported',- chunks: Number(outcome.chunks ?? 0),- untrusted_payload: untrustedPayload,- source_kind: event.source_kind,- source_uri: event.source_uri,- ...(sourceFallback ? { source_fallback: sourceFallback } : {}),- };+ // The door answers only a committed write; a pending one throws write_pending and the job retries.+ const result = managedOutcome((response.outcome ?? {}) as { noop?: boolean; chunks?: number });+ if (appliedKey) await recordAppliedCapture(engine, appliedKey, sourceFallback ? 'default' : sourceId ?? 'default', slug, result);+ return result;}+ // GST-1856: the record goes in the page write's own transaction; when another run of the key recorded it first, the+ // whole write rolls back (null) and the stored result is returned. The FK retry's first transaction rolls its record back too.+ const importAs = async (sid: string | undefined) => {+ try {+ return await importFromContent(engine, slug, event.content, { ...importOpts, sourceId: sid, ...(appliedKey ? { beforeCommit: async (tx: BrainEngine) => {+ if (!await recordAppliedCapture(tx, appliedKey, sid ?? 'default', slug, captured('imported', 0))) throw new CaptureAlreadyApplied();+ } } : {}) });+ } catch (err) {+ if (err instanceof CaptureAlreadyApplied) return null;+ throw err;+ }+ };let result;try {- result = await importFromContent(engine, slug, event.content, { ...importOpts, sourceId });+ result = await importAs(sourceId);} catch (err) {- // The sources row can vanish (ON DELETE CASCADE) between the pre-check and- // this write. 23503 = foreign_key_violation (cf. oauth-provider.ts:1053),- // but SQLSTATE alone is too broad: a violation from chunks/tags/versions- // would be misread as "source unavailable" and silently rewritten to the- // default partition, masking a real integrity failure.- //- // Match the PAGES source FK specifically. Matching any message mentioning- // "source" is not tight enough to carry a routing decision: many tables- // reference sources(id) (files_source_id_fkey, facts_source_id_fkey,- // code_edges, calibration_profiles, …), so a genuine integrity failure on- // one of those would be rewritten as reason:'fk_violation' — exactly the- // masking this check exists to prevent. maybeRewriteSourceFkError- // (commands/capture.ts) can afford the looser match because it only- // formats a human hint; here the verdict moves data. Prefer the driver's- // structured constraint/table fields (postgres.js and PGLite both expose- // them) and fall back to the message for wrapped errors.- const errMsg = err instanceof Error ? err.message : String(err);- const fkErr = err as { code?: string; constraint?: string; table?: string };- const namesPagesSourceFk =- (typeof fkErr.constraint === 'string' && /^pages_source_id_fk/.test(fkErr.constraint)) ||- (fkErr.table === 'pages' && typeof fkErr.constraint === 'string' && fkErr.constraint.includes('source')) ||- /pages_source_id_fk/.test(errMsg) ||- (errMsg.includes('foreign key constraint') && errMsg.includes('"pages"'));- const isSourceFk = fkErr?.code === '23503' && namesPagesSourceFk;- if (sourceId !== undefined && isSourceFk) {+ // The sources row can vanish (ON DELETE CASCADE) between the pre-check and this write.+ if (sourceId !== undefined && isPagesSourceFkViolation(err)) {sourceFallback = { requested: sourceId, effective: 'default', reason: 'fk_violation' };await checkWriter('default');- result = await importFromContent(engine, slug, event.content, { ...importOpts, sourceId: undefined });+ result = await importAs(undefined);} else {throw err;}}+ if (result === null) return (await readAppliedCapture<IngestCaptureResult>(engine, appliedKey!))!;const refused = result.refusal?.code === 'visibility_noncanonical';if (sourceFallback) {console.error(@@ -374,15 +403,10 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {throw privateRequestRefusal({ slug: result.slug, source_id: sourceFallback ? 'default' : sourceId ?? 'default' }, parseMarkdown(event.content, `${result.slug}.md`).frontmatter);}- return {- slug,- status: result.status,- chunks: result.chunks,- untrusted_payload: untrustedPayload,- source_kind: event.source_kind,- source_uri: event.source_uri,- ...(sourceFallback ? { source_fallback: sourceFallback } : {}),- };+ // The final result replaces the provisional one; an import that wrote nothing (no transaction ran) inserts the row here.+ const final = captured(result.status, result.chunks);+ if (appliedKey && result.status !== 'error') await saveAppliedCapture(engine, appliedKey, sourceFallback ? 'default' : sourceId ?? 'default', slug, final);+ return final;};}diff --git a/src/core/minions/job-context.ts b/src/core/minions/job-context.tsindex 66dc4efb64..e4429e3fe9 100644--- a/src/core/minions/job-context.ts+++ b/src/core/minions/job-context.ts@@ -30,6 +30,7 @@ export function buildJobContext(data: job.data,submission_authority: job.submission_authority,attempts_made: job.attempts_made,+ idempotency_key: job.idempotency_key,signal,deadlineAtMs: job.timeout_at != null ? job.timeout_at.getTime() : null,shutdownSignal,diff --git a/src/core/minions/types.ts b/src/core/minions/types.tsindex 4494c72235..6ed4f25886 100644--- a/src/core/minions/types.ts+++ b/src/core/minions/types.ts@@ -283,6 +283,8 @@ export interface MinionJobContext {/** Set by runWithJobSpend for a spend-authorized job: its group record and meter key. */spend?: JobSpendContext;attempts_made: number;+ /** The job row's idempotency key (null when it was queued without one); ingest_capture records an applied capture under it (GST-1856). */+ idempotency_key: string | null;/** AbortSignal for cooperative cancellation (fires on timeout, cancel, pause, or lock loss). */signal: AbortSignal;/** Absolute wall-clock deadline (epoch ms) from the claim-time `timeout_at` stamp,diff --git a/src/core/persistence/graduation-inventory.ts b/src/core/persistence/graduation-inventory.tsindex db6d617a6d..d21c31504b 100644--- a/src/core/persistence/graduation-inventory.ts+++ b/src/core/persistence/graduation-inventory.ts@@ -89,6 +89,7 @@ export const GRADUATION_INVENTORY: Inventory = {carry('facts', 'user_data', 'Facts with attribution columns and the superseded_by self-FK.'),carry('files', 'user_data', 'File metadata; object bytes stay in the storage backend.'),carry('git_canonical_brain', 'operational', 'Git-canonical fence counter (GBRA-95 G7a): sources whose config sets canonical = git, kept by triggers on sources that the copy bypasses; the target would otherwise read zero and its writer guard would refuse nothing.'),+ carry('ingest_applied_captures', 'operational', 'Applied /ingest captures by job idempotency key, so a re-run or a replay of a capture after graduation still writes nothing.'),carry('ingest_log', 'operational', 'Ingestion log.'),carry('links', 'user_data', 'Page graph links.'),carry('link_edge_proposals', 'user_data', 'Relationship contradiction proposals with the user\'s accept/reject/undo decisions and the paid judge results behind them.'),diff --git a/src/core/schema-migrations/latest.generated.ts b/src/core/schema-migrations/latest.generated.tsindex 0e755cb30c..a8fdb37405 100644--- a/src/core/schema-migrations/latest.generated.ts+++ b/src/core/schema-migrations/latest.generated.ts@@ -4,4 +4,4 @@// The highest schema migration version (migrate.ts LATEST_VERSION), importable without// loading the migrations themselves.-export const LATEST_SCHEMA_VERSION = 295;+export const LATEST_SCHEMA_VERSION = 296;diff --git a/src/core/schema-migrations/registry.generated.ts b/src/core/schema-migrations/registry.generated.tsindex 26ec10bdfd..852ba0067c 100644--- a/src/core/schema-migrations/registry.generated.ts+++ b/src/core/schema-migrations/registry.generated.ts@@ -296,6 +296,7 @@ import { v292 } from './v292-refresh-write-barrier.ts';import { v293 } from './v293-qm-attest-sources.ts';import { v294 } from './v294-refresh-barrier-import-deferral.ts';import { v295 } from './v295-refresh-retry-hint-history-index.ts';+import { v296 } from './v296-ingest-applied-captures.ts';// Array order is historical (see HISTORICAL_ARRAY_ORDER in the generator); the// runner sorts by version before applying.@@ -590,4 +591,5 @@ export const MIGRATIONS: Migration[] = [v293,v294,v295,+ v296,];diff --git a/src/core/schema-migrations/v296-ingest-applied-captures.ts b/src/core/schema-migrations/v296-ingest-applied-captures.tsnew file mode 100644index 0000000000..0144f22251--- /dev/null+++ b/src/core/schema-migrations/v296-ingest-applied-captures.ts@@ -0,0 +1,36 @@+import type { Migration } from './types.ts';++// Applied by src/core/migrate.ts on the next initSchema(); see docs/ENGINES.md+// ("Canonical schema sources") for when schema.sql or a TS fragment also changes.+//+// GST-1856: ingest_applied_captures records that an ingest_capture job's write+// landed, keyed by the job's idempotency key and written in the page write's+// own transaction (or, on a managed brain, right after the door commits). A+// later run of the same key returns the stored result instead of writing the+// body again, so a stalled attempt re-run or a POST /ingest replayed after+// `gbrain jobs prune` no longer reverts a newer edit. The dream cycle's purge+// phase drops rows past 90 days (purgeStaleAppliedCaptures). Operational+// state, not brain content: a repo-less brain gets it unchanged. Created empty;+// nothing is rewritten on upgrade. No schema.sql mirror: like v282-v290, it+// exists only through migrations. The number is a placeholder restamped at the+// merge slot.+export const v296: Migration = {+ version: 296,+ name: 'ingest_applied_captures',+ idempotent: true,+ sql: `+CREATE TABLE IF NOT EXISTS ingest_applied_captures (+ idempotency_key TEXT PRIMARY KEY,+ source_id TEXT NOT NULL,+ slug TEXT NOT NULL,+ result JSONB NOT NULL,+ applied_at TIMESTAMPTZ NOT NULL DEFAULT now()+);+CREATE INDEX IF NOT EXISTS ingest_applied_captures_applied_at_idx ON ingest_applied_captures (applied_at);+DO $rls$ BEGIN+ IF EXISTS (SELECT 1 FROM pg_roles pr WHERE pg_has_role(current_user, pr.oid, 'USAGE') AND (pr.rolbypassrls OR pr.rolsuper)) THEN+ ALTER TABLE ingest_applied_captures ENABLE ROW LEVEL SECURITY;+ END IF;+END $rls$;+`,+};diff --git a/test/e2e/ingest-capture-applied-once-postgres.test.ts b/test/e2e/ingest-capture-applied-once-postgres.test.tsnew file mode 100644index 0000000000..8b33c6ea1e--- /dev/null+++ b/test/e2e/ingest-capture-applied-once-postgres.test.ts@@ -0,0 +1,3 @@+import { registerPostgresTests } from '../helpers/test-backends.ts';++await registerPostgresTests(() => import('../ingest-capture-applied-once.test.ts'));diff --git a/test/e2e/ingestion-roundtrip.test.ts b/test/e2e/ingestion-roundtrip.test.tsindex 3fca5a09ae..51d43f6c14 100644--- a/test/e2e/ingestion-roundtrip.test.ts+++ b/test/e2e/ingestion-roundtrip.test.ts@@ -48,6 +48,7 @@ function makeFakeJobCtx(data: Record<string, unknown>): MinionJobContext {name: 'ingest_capture',data,attempts_made: 1,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/e2e/openrouter-anthropic-subagent-replay.live.test.ts b/test/e2e/openrouter-anthropic-subagent-replay.live.test.tsindex cdc2d7840d..cadc029513 100644--- a/test/e2e/openrouter-anthropic-subagent-replay.live.test.ts+++ b/test/e2e/openrouter-anthropic-subagent-replay.live.test.ts@@ -66,6 +66,7 @@ function makeCtx(name: 'subagent',data: { prompt, model: MODEL, max_turns: 4, system: SYSTEM },attempts_made: 0,+ idempotency_key: null,signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/e2e/openrouter-deepseek-subagent-replay.live.test.ts b/test/e2e/openrouter-deepseek-subagent-replay.live.test.tsindex ae654893e0..3aa5adf5ca 100644--- a/test/e2e/openrouter-deepseek-subagent-replay.live.test.ts+++ b/test/e2e/openrouter-deepseek-subagent-replay.live.test.ts@@ -66,6 +66,7 @@ function makeCtx(name: 'subagent',data: { prompt, model: MODEL, max_turns: 4, system: SYSTEM },attempts_made: 0,+ idempotency_key: null,signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/e2e/subagent-crash-replay-multi-provider.test.ts b/test/e2e/subagent-crash-replay-multi-provider.test.tsindex 659922fd8d..eb43eaca53 100644--- a/test/e2e/subagent-crash-replay-multi-provider.test.ts+++ b/test/e2e/subagent-crash-replay-multi-provider.test.ts@@ -274,6 +274,7 @@ async function makeCrashedCtx(jobId: number, prompt: string, modelId: string): Pname: 'subagent',data: { prompt, model: modelId },attempts_made: 1, // crashed once+ idempotency_key: null,signal: abortCtrl.signal,deadlineAtMs: null,shutdownSignal: shutdownCtrl.signal,diff --git a/test/e2e/subagent-gateway-path.test.ts b/test/e2e/subagent-gateway-path.test.tsindex 7a436aeb2a..395f7dd7a6 100644--- a/test/e2e/subagent-gateway-path.test.ts+++ b/test/e2e/subagent-gateway-path.test.ts@@ -94,6 +94,7 @@ async function makeFakeJob(opts: FakeJobOpts): Promise<{ jobId: number; ctx: Minname: 'subagent',data: { prompt: opts.prompt, model: opts.model, allowed_tools: opts.allowed_tools, mode: opts.mode },attempts_made: 0,+ idempotency_key: null,signal: abortCtrl.signal,deadlineAtMs: null,shutdownSignal: shutdownCtrl.signal,diff --git a/test/e2e/subagent-gateway-resume-reconciliation.test.ts b/test/e2e/subagent-gateway-resume-reconciliation.test.tsindex 09251d2b18..3033f7c633 100644--- a/test/e2e/subagent-gateway-resume-reconciliation.test.ts+++ b/test/e2e/subagent-gateway-resume-reconciliation.test.ts@@ -67,7 +67,7 @@ async function makeJob(prompt: string, model: string): Promise<{ jobId: number;);const jobId = rows[0].id;const ctx: MinionJobContext = {- id: jobId, name: 'subagent', data: { prompt, model }, attempts_made: 1,+ id: jobId, name: 'subagent', data: { prompt, model }, attempts_made: 1, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {},isActive: async () => true, readInbox: async () => [],diff --git a/test/embedding-recovery.serial.test.ts b/test/embedding-recovery.serial.test.tsindex 7825728d3f..203d4908e8 100644--- a/test/embedding-recovery.serial.test.ts+++ b/test/embedding-recovery.serial.test.ts@@ -1359,7 +1359,7 @@ for (const kind of backends) {}const controller = new AbortController();const result = await makeEmbedBackfillHandler(engine)({- id: 1, name: 'embed-backfill', data: { sourceId: 'default' }, attempts_made: 0,+ id: 1, name: 'embed-backfill', data: { sourceId: 'default' }, attempts_made: 0, idempotency_key: null,signal: controller.signal, shutdownSignal: controller.signal, deadlineAtMs: Date.now() + 60_000,isActive: async () => true, updateProgress: async () => {}, updateTokens: async () => {},log: async () => {}, readInbox: async () => [],diff --git a/test/facts-absorb-keyless-deferral.test.ts b/test/facts-absorb-keyless-deferral.test.tsindex eb7d50c756..a48c9e061d 100644--- a/test/facts-absorb-keyless-deferral.test.ts+++ b/test/facts-absorb-keyless-deferral.test.ts@@ -29,7 +29,7 @@ beforeEach(async () => { await resetPgliteState(engine); await engine.setConfig(const BODY = `Acme Example shipped the beta in 2026. ${'A substantive project record with names, dates and decisions. '.repeat(5)}`;function ctx(job: { id: number; data: Record<string, unknown> }, signal = new AbortController().signal): MinionJobContext {- return { id: job.id, name: 'facts-absorb', data: job.data, attempts_made: 0, signal, deadlineAtMs: null, shutdownSignal: signal,+ return { id: job.id, name: 'facts-absorb', data: job.data, attempts_made: 0, idempotency_key: null, signal, deadlineAtMs: null, shutdownSignal: signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] } as MinionJobContext;}async function queued(slug: string) {diff --git a/test/facts-absorb-publish-wait-6778.test.ts b/test/facts-absorb-publish-wait-6778.test.tsindex 617ee1265e..de2dc16d5a 100644--- a/test/facts-absorb-publish-wait-6778.test.ts+++ b/test/facts-absorb-publish-wait-6778.test.ts@@ -123,7 +123,7 @@ async function fixture(name: string, run: (f: Fixture) => Promise<void>): Promisfunction jobContext(job: Fixture['job'], deadlineAtMs: number | null): MinionJobContext {const signal = new AbortController().signal;- return { id: job.id, name: 'facts-absorb', data: job.data, attempts_made: 0, signal, deadlineAtMs, shutdownSignal: signal,+ return { id: job.id, name: 'facts-absorb', data: job.data, attempts_made: 0, idempotency_key: null, signal, deadlineAtMs, shutdownSignal: signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] } as MinionJobContext;}diff --git a/test/facts-backstop-outcome.serial.test.ts b/test/facts-backstop-outcome.serial.test.tsindex 3f7e816b8e..017268fa74 100644--- a/test/facts-backstop-outcome.serial.test.ts+++ b/test/facts-backstop-outcome.serial.test.ts@@ -301,7 +301,7 @@ describe('put_page never fails on extraction errors (E1 invariant)', () => {const handlers = new Map<string, MinionHandler>();await registerBuiltinHandlers({ register: (name: string, handler: MinionHandler) => handlers.set(name, handler) } as never, engine, { quiet: true });await expect(handlers.get('facts-absorb')!({- ...jobs[0], name: 'facts-absorb', attempts_made: 0, signal: new AbortController().signal,+ ...jobs[0], name: 'facts-absorb', attempts_made: 0, idempotency_key: null, signal: new AbortController().signal,deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {},updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [],})).rejects.toBeInstanceOf(FactsExtractionError);diff --git a/test/facts-backstop.test.ts b/test/facts-backstop.test.tsindex 7a7288d87a..6b48f9a3b9 100644--- a/test/facts-backstop.test.ts+++ b/test/facts-backstop.test.ts@@ -490,6 +490,7 @@ describe('facts-absorb minion handler honors the queued notabilityFilter (#4870)name: 'facts-absorb',data: { slug: page.slug, sourceId: 'default', source: 'mcp:put_page', notabilityFilter: 'medium-and-up' },attempts_made: 0,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-default.json b/test/fixtures/goldens/catalog/pglite-engine-init-default.jsonindex 2d52dc61f0..fa40c9aaf8 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-default.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-default.json@@ -158,6 +158,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -816,6 +817,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1410,6 +1414,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2719,6 +2725,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.jsonindex c7646f800d..aa2c137aab 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json@@ -158,6 +158,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -816,6 +817,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1410,6 +1414,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2719,6 +2725,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.jsonindex f24412af8a..22ea6da123 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json@@ -158,6 +158,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -816,6 +817,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1406,6 +1410,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2715,6 +2721,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-default.json b/test/fixtures/goldens/catalog/postgres-engine-init-default.jsonindex 5435aa0a4d..8f965096b2 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-default.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-default.json@@ -161,6 +161,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -823,6 +824,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1413,6 +1417,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2731,6 +2737,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.jsonindex 812260080b..1015bc6397 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json@@ -161,6 +161,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -823,6 +824,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1413,6 +1417,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2731,6 +2737,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.jsonindex 70e2e56154..bfd82d703f 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json@@ -161,6 +161,7 @@"git_canonical_brain.git_canonical_brain_pkey": "p PRIMARY KEY (singleton)","git_canonical_brain.git_canonical_brain_singleton_check": "c CHECK ((singleton = 1))","git_canonical_brain.git_canonical_brain_strict_sources_check": "c CHECK ((strict_sources >= 0))",+ "ingest_applied_captures.ingest_applied_captures_pkey": "p PRIMARY KEY (idempotency_key)","ingest_log.ingest_log_pkey": "p PRIMARY KEY (id)","link_edge_proposals.link_edge_proposals_a_to_page_id_fkey": "f FOREIGN KEY (a_to_page_id) REFERENCES pages(id) ON DELETE CASCADE","link_edge_proposals.link_edge_proposals_b_to_page_id_fkey": "f FOREIGN KEY (b_to_page_id) REFERENCES pages(id) ON DELETE CASCADE",@@ -823,6 +824,9 @@"git_canonical_brain": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],+ "ingest_applied_captures": [+ "<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"+ ],"ingest_log": ["<current_role> -> <current_role> WITH GRANT OPTION: DELETE,INSERT,REFERENCES,SELECT,TRIGGER,TRUNCATE,UPDATE"],@@ -1409,6 +1413,8 @@"idx_timeline_event_page": "CREATE INDEX idx_timeline_event_page ON public.timeline_entries USING btree (event_page_id) WHERE (event_page_id IS NOT NULL)","idx_timeline_page": "CREATE INDEX idx_timeline_page ON public.timeline_entries USING btree (page_id)","idx_versions_page": "CREATE INDEX idx_versions_page ON public.page_versions USING btree (page_id)",+ "ingest_applied_captures_applied_at_idx": "CREATE INDEX ingest_applied_captures_applied_at_idx ON public.ingest_applied_captures USING btree (applied_at)",+ "ingest_applied_captures_pkey": "CREATE UNIQUE INDEX ingest_applied_captures_pkey ON public.ingest_applied_captures USING btree (idempotency_key)","ingest_log_marked_idx": "CREATE INDEX ingest_log_marked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS TRUE)","ingest_log_pkey": "CREATE UNIQUE INDEX ingest_log_pkey ON public.ingest_log USING btree (id)","ingest_log_unmarked_idx": "CREATE INDEX ingest_log_unmarked_idx ON public.ingest_log USING btree (created_at) WHERE (writer_segment_reader IS NOT TRUE)",@@ -2727,6 +2733,16 @@],"flags": "relkind=r rls=true force_rls=false"},+ "ingest_applied_captures": {+ "columns": [+ "1 idempotency_key text/text NOT NULL",+ "2 source_id text/text NOT NULL",+ "3 slug text/text NOT NULL",+ "4 result jsonb/jsonb NOT NULL",+ "5 applied_at timestamp with time zone/timestamptz NOT NULL DEFAULT now()"+ ],+ "flags": "relkind=r rls=true force_rls=false"+ },"ingest_log": {"columns": ["1 id integer/int4 NOT NULL DEFAULT nextval('ingest_log_id_seq'::regclass)",diff --git a/test/fixtures/goldens/doctor/json-postgres-fresh.json b/test/fixtures/goldens/doctor/json-postgres-fresh.jsonindex 8550797000..2c90c82221 100644--- a/test/fixtures/goldens/doctor/json-postgres-fresh.json+++ b/test/fixtures/goldens/doctor/json-postgres-fresh.json@@ -1083,7 +1083,7 @@{"category": "ops","elapsed_ms": "<volatile>",- "message": "RLS enabled on 173/173 public tables",+ "message": "RLS enabled on 174/174 public tables","name": "rls","status": "ok"},@@ -1111,7 +1111,7 @@{"category": "meta","elapsed_ms": "<volatile>",- "message": "All 173 table(s) the applied migrations create exist.",+ "message": "All 174 table(s) the applied migrations create exist.","name": "schema_missing_tables","status": "ok"},diff --git a/test/fixtures/goldens/exports/types.json b/test/fixtures/goldens/exports/types.jsonindex 66e4842035..ea91f74979 100644--- a/test/fixtures/goldens/exports/types.json+++ b/test/fixtures/goldens/exports/types.json@@ -6401,6 +6401,7 @@"data: Record<string, unknown>","deadlineAtMs: number | null","id: number",+ "idempotency_key: string | null","isActive: () => Promise<boolean>","log: (message: string | TranscriptEntry) => Promise<void>","name: string",diff --git a/test/fixtures/goldens/migrations/records.json b/test/fixtures/goldens/migrations/records.jsonindex 09b0198e57..f249a6baa3 100644--- a/test/fixtures/goldens/migrations/records.json+++ b/test/fixtures/goldens/migrations/records.json@@ -1,6 +1,6 @@{"golden": {- "count": 290,+ "count": 291,"gaps": [17,18,@@ -97,7 +97,7 @@"verifyProjectionStatistics": "0d231b04ecfbc5a4109d6fca9567bf4e8c9e9db6c3fbbb35212e026579ed6f67","withSchemaLockTimeout": "95605bd3004955e11e936c7a4aa6889319b401098abf9db18ef3e48da2c3fc59"},- "latest": 295,+ "latest": 296,"records": [{"handler": {@@ -4813,6 +4813,19 @@"transaction": "absent","verify": "absent","version": 295+ },+ {+ "handler": "absent",+ "idempotent": true,+ "name": "ingest_applied_captures",+ "sql": {+ "length": 556,+ "sha256": "685b94ce31a5399b6fa9e456a3280a52a33b5d2f87d60b958f946df5b255144d"+ },+ "sqlFor": "absent",+ "transaction": "absent",+ "verify": "absent",+ "version": 296}],"versions": [@@ -5105,7 +5118,8 @@292,293,294,- 295+ 295,+ 296]},"normalizer": "migrations-record-v1"diff --git a/test/g6-ingestion-strict.test.ts b/test/g6-ingestion-strict.test.tsindex 378e5319f6..9dc8386112 100644--- a/test/g6-ingestion-strict.test.ts+++ b/test/g6-ingestion-strict.test.ts@@ -34,7 +34,7 @@ function stubWatcher() {return { watcher, add: (path: string) => emitter.emit('add', path) };}-const job = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 1, signal: new AbortController().signal,+const job = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 1, idempotency_key: null, signal: new AbortController().signal,deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {},isActive: async () => true, readInbox: async () => [] });diff --git a/test/graduation-inventory.test.ts b/test/graduation-inventory.test.tsindex 86d67b1440..c1cf3cf422 100644--- a/test/graduation-inventory.test.ts+++ b/test/graduation-inventory.test.ts@@ -60,7 +60,7 @@ describe('graduation inventory', () => {expect(new Set(names).size).toBe(names.length);const counts: Record<string, number> = {};for (const e of GRADUATION_INVENTORY.entries) counts[e.class] = (counts[e.class] ?? 0) + 1;- expect(counts).toEqual({ carry: 158, rebind: 2, rebuild: 3, discard: 9, schema_owned: 4 });+ expect(counts).toEqual({ carry: 159, rebind: 2, rebuild: 3, discard: 9, schema_owned: 4 });for (const e of GRADUATION_INVENTORY.entries) {expect({ relation: e.relation, reason: e.reason.length > 5, present: e.engines.pglite || e.engines.postgres }).toEqual({ relation: e.relation, reason: true, present: true });diff --git a/test/handlers-embed-backfill.test.ts b/test/handlers-embed-backfill.test.tsindex 72bc8dc1ab..eb443e4dd8 100644--- a/test/handlers-embed-backfill.test.ts+++ b/test/handlers-embed-backfill.test.ts@@ -54,6 +54,7 @@ function fakeJob(data: Record<string, unknown>): MinionJobContext {name: 'embed-backfill',data,attempts_made: 0,+ idempotency_key: null,signal: controller.signal,deadlineAtMs: null,shutdownSignal: controller.signal,diff --git a/test/helpers/facts-worker-config-contract.ts b/test/helpers/facts-worker-config-contract.tsindex 0402f5bfd7..a67e7c5176 100644--- a/test/helpers/facts-worker-config-contract.ts+++ b/test/helpers/facts-worker-config-contract.ts@@ -59,7 +59,7 @@ export async function exerciseFactsWorkerConfig(engine: BrainEngine): Promise<voawait registerBuiltinHandlers(worker, engine, { quiet: true });await disposePersistenceConsumer(engine);expect(persistenceConsumerStatus(engine).state).toBe('not_running');- const job: MinionJobContext = { id: Number(queued.id), name: 'facts-absorb', data: { ...queued.data, config: { embedding_disabled: false } }, attempts_made: 0,+ const job: MinionJobContext = { id: Number(queued.id), name: 'facts-absorb', data: { ...queued.data, config: { embedding_disabled: false } }, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] };expect(await worker.getHandler('facts-absorb')!(job)).toMatchObject({ inserted: 1 });diff --git a/test/helpers/managed-atoms-contract.ts b/test/helpers/managed-atoms-contract.tsindex 19a8fdcc5e..a4a28a117d 100644--- a/test/helpers/managed-atoms-contract.ts+++ b/test/helpers/managed-atoms-contract.ts@@ -157,7 +157,7 @@ export async function exerciseManagedAtoms(engine: BrainEngine, scenario: Case):const worker = new MinionWorker(engine, { queue: 'fixture' });await registerBuiltinHandlers(worker, engine, { quiet: true });__setChatTransportForTests(chat);- const job: MinionJobContext = { id: 944, name: 'extract-atoms-drain', data: { sourceId, retryRequestId: receipt.request_id }, attempts_made: 0,+ const job: MinionJobContext = { id: 944, name: 'extract-atoms-drain', data: { sourceId, retryRequestId: receipt.request_id }, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] };if (scenario === 'malformed_retry_failure') {diff --git a/test/helpers/managed-facts-contract.ts b/test/helpers/managed-facts-contract.tsindex bfbd1c63e4..8f886ccd52 100644--- a/test/helpers/managed-facts-contract.ts+++ b/test/helpers/managed-facts-contract.ts@@ -117,7 +117,7 @@ export async function exerciseManagedFacts(engine: BrainEngine, scenario: Case):expect(jobs[0].data.persistence_request_id).toBeDefined();const worker = new MinionWorker(engine, { queue: 'fixture' });await registerBuiltinHandlers(worker, engine, { quiet: true });- const job: MinionJobContext = { id: Number(jobs[0].id), name: 'facts-absorb', data: jobs[0].data, attempts_made: 0,+ const job: MinionJobContext = { id: Number(jobs[0].id), name: 'facts-absorb', data: jobs[0].data, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] };if (scenario === 'worker_revoked') await engine.executeRaw('UPDATE oauth_clients SET deleted_at=now() WHERE client_id=$1', [clientId]);diff --git a/test/ingest-capture-applied-once.test.ts b/test/ingest-capture-applied-once.test.tsnew file mode 100644index 0000000000..b42e4c8554--- /dev/null+++ b/test/ingest-capture-applied-once.test.ts@@ -0,0 +1,260 @@+/**+ * GST-1856: an ingest_capture job applies its capture once per idempotency key, so a run after the write landed never+ * reverts a newer edit.+ *+ * A worker that died or lost its lock after the page write committed, but before completion was recorded, handed the job+ * to another attempt (the stall sweep), and that attempt wrote the captured body over any edit made in between. A client+ * replaying a POST /ingest after `gbrain jobs prune` removed the completed job row got a fresh job that did the same.+ * Protects: a re-run of the key writes nothing and returns the first run's result (unmanaged, and managed through the+ * door's journal when the record row is missing); a replay after the job row is deleted leaves the newer edit; two+ * captures with different content to one slug both land in order, and re-running the first afterwards keeps the second.+ * Real token verifier, real Express route, the real ingest_capture handler; PGLite here, Postgres through+ * test/e2e/ingest-capture-applied-once-postgres.test.ts.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import express from 'express';+import type { Server } from 'node:http';+import type { AddressInfo } from 'node:net';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { mintLegacyToken } from '../src/core/token-mint.ts';+import { mountWebhooks } from '../src/commands/serve-http-webhooks.ts';+import type { ServeHttpContext } from '../src/commands/serve-http.ts';+import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';+import { sqlQueryForEngine } from '../src/core/sql-query.ts';+import { importFromContent } from '../src/core/import-file.ts';+import { computeContentHash } from '../src/core/ingestion/types.ts';+import { purgeStaleAppliedCaptures } from '../src/core/ingestion/applied-captures.ts';+import { makeIngestCaptureHandler } from '../src/core/minions/handlers/ingest-capture.ts';+import type { MinionJobContext } from '../src/core/minions/types.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const SRC = 'source-a';++interface Brain { engine: BrainEngine; server: Server; base: string; token: string; close: () => Promise<void> }+const brains = new Map<string, Brain>();++async function openBrain(engine: BrainEngine, closeEngine: () => Promise<void>): Promise<Brain> {+ await engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT (id) DO NOTHING', [SRC]);+ const token = (await mintLegacyToken(engine, { name: 'applied-once-writer', scopes: ['read', 'write'], takesHolders: ['world'], sourceGrant: [SRC] })).token;+ const provider = new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine), transaction: fn => engine.transaction(tx => fn(sqlQueryForEngine(tx))) });+ const app = express();+ const pass: express.RequestHandler = (_req, _res, next) => next();+ mountWebhooks(app, {+ engine,+ resourceVerifier: provider,+ resourceMetadataUrl: 'http://127.0.0.1/.well-known/oauth-protected-resource',+ ingestRateLimiter: pass,+ githubWebhookLimiter: pass,+ broadcastEvent: () => {},+ } as unknown as ServeHttpContext);+ const server = await new Promise<Server>(resolve => { const s = app.listen(0, '127.0.0.1', () => resolve(s)); });+ const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;+ return { engine, server, base, token, close: async () => { await new Promise<void>(resolve => server.close(() => resolve())); await closeEngine(); } };+}++const backends = testBackends();+beforeAll(async () => {+ if (backends.includes('pglite')) {+ const pglite = new PGLiteEngine();+ await pglite.connect({});+ await pglite.initSchema();+ brains.set('pglite', await openBrain(pglite, () => pglite.disconnect()));+ }+ if (backends.includes('postgres')) {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ brains.set('postgres', await openBrain(pg.engine, pg.close));+ }+}, 120_000);+afterAll(async () => { for (const brain of brains.values()) await brain.close(); }, 60_000);++const jobCtx = (id: number, data: Record<string, unknown>, key: string | null, attempts = 0): MinionJobContext => ({ id, name: 'ingest_capture', data,+ attempts_made: attempts, idempotency_key: key, signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,+ updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] });+const jsonData = (data: unknown) => (typeof data === 'string' ? JSON.parse(data) : data) as Record<string, unknown>;+const note = (title: string, line: string) => `---\ntype: note\ntitle: ${title}\n---\n# ${title}\n\n${line}\n`;+/** The data a webhook capture job carries for `content` sent to `slug` in `sourceId`. */+const captureData = (content: string, slug: string, sourceId: string) => ({ slug, sourceId, event: { source_id: 'webhook-client-example', source_kind: 'webhook',+ source_uri: 'mcp-webhook:client-example:1', received_at: new Date().toISOString(), content_type: 'text/markdown', content,+ content_hash: computeContentHash(content), untrusted_payload: true } });++for (const backend of backends) {+ describe(`ingest_capture applies a capture once per idempotency key (${backend})`, () => {+ const brain = () => brains.get(backend)!;+ const post = (body: string, slug: string) => fetch(`${brain().base}/ingest`, {+ method: 'POST',+ headers: { authorization: `Bearer ${brain().token}`, 'content-type': 'text/markdown', 'x-gbrain-slug': slug },+ body,+ });+ const ingest = async (body: string, slug: string) => {+ const res = await post(body, slug);+ expect(res.status).toBe(202);+ return (await res.json() as { job_id: number }).job_id;+ };+ /** Runs a queued job the way the worker does: data and idempotency key from its row. */+ const runJob = async (id: number) => {+ const [row] = await brain().engine.executeRaw<{ data: unknown; idempotency_key: string | null }>('SELECT data, idempotency_key FROM minion_jobs WHERE id = $1', [id]);+ return makeIngestCaptureHandler(brain().engine)(jobCtx(id, jsonData(row!.data), row!.idempotency_key));+ };+ const complete = (id: number) => brain().engine.executeRaw("UPDATE minion_jobs SET status = 'completed', finished_at = now() WHERE id = $1", [id]);+ const pageText = async (slug: string) => (await brain().engine.readPageSnapshot(slug, { sourceId: SRC }))?.page.compiled_truth ?? '';++ test('a re-run of the job after its write landed keeps the newer edit and returns the first result', async () => {+ const slug = `notes/rerun-${backend}`;+ const data = captureData(note('Rerun capture', 'Captured body.'), slug, SRC);+ const key = `ingest:webhook:client-example:${SRC}:rerun-${backend}`;+ const run = makeIngestCaptureHandler(brain().engine);+ const first = await run(jobCtx(1, data, key));+ expect(first.status).toBe('imported');+ await importFromContent(brain().engine, slug, note('Rerun capture', 'Newer edit.'), { sourceId: SRC, noEmbed: true });+ // The stall sweep hands the job to another attempt: same row, same key, same attempts_made.+ expect(await run(jobCtx(1, data, key))).toEqual(first);+ expect(await pageText(slug)).toContain('Newer edit.');+ expect(await pageText(slug)).not.toContain('Captured body.');+ const [row] = await brain().engine.executeRaw<{ source_id: string; slug: string }>('SELECT source_id, slug FROM ingest_applied_captures WHERE idempotency_key = $1', [key]);+ expect(row).toEqual({ source_id: SRC, slug });+ });++ test('a POST replayed after its job row was pruned queues a job that keeps the newer edit', async () => {+ const slug = `notes/replay-${backend}`;+ const body = note('Replayed capture', 'Body from the integration.');+ const first = await ingest(body, slug);+ const firstResult = await runJob(first);+ expect(firstResult.status).toBe('imported');+ await complete(first);+ await importFromContent(brain().engine, slug, note('Replayed capture', 'Newer edit after the capture.'), { sourceId: SRC, noEmbed: true });+ await brain().engine.executeRaw('DELETE FROM minion_jobs WHERE id = $1', [first]);++ const replay = await ingest(body, slug);+ expect(replay).not.toBe(first);+ expect(await runJob(replay)).toEqual(firstResult);+ expect(await pageText(slug)).toContain('Newer edit after the capture.');+ expect(await pageText(slug)).not.toContain('Body from the integration.');+ });++ test('two captures with different content to one slug land in order, and a re-run of the first keeps the second', async () => {+ const slug = `notes/ordered-${backend}`;+ const one = await ingest(note('Ordered capture', 'First capture body.'), slug);+ expect((await runJob(one)).status).toBe('imported');+ await complete(one);+ expect(await pageText(slug)).toContain('First capture body.');+ const two = await ingest(note('Ordered capture', 'Second capture body.'), slug);+ expect(two).not.toBe(one);+ expect((await runJob(two)).status).toBe('imported');+ expect(await pageText(slug)).toContain('Second capture body.');+ expect((await runJob(one)).status).toBe('imported');+ expect(await pageText(slug)).toContain('Second capture body.');+ expect(await pageText(slug)).not.toContain('First capture body.');+ });++ // Two connections are needed to hold the record row uncommitted while the run writes, so this one is Postgres only.+ if (backend === 'postgres') test('a run that loses the record insert to a concurrent run rolls its page write back', async () => {+ const slug = `notes/concurrent-${backend}`;+ await importFromContent(brain().engine, slug, note('Concurrent capture', 'Edit already on the page.'), { sourceId: SRC, noEmbed: true });+ const data = captureData(note('Concurrent capture', 'Concurrent captured body.'), slug, SRC);+ const key = `ingest:webhook:client-example:${SRC}:concurrent-${backend}`;+ const stored = { slug, status: 'imported' as const, chunks: 0, untrusted_payload: true, source_kind: 'webhook', source_uri: 'mcp-webhook:client-example:1' };+ let inserted!: () => void, release!: () => void;+ const insertedP = new Promise<void>(resolve => { inserted = resolve; });+ const held = new Promise<void>(resolve => { release = resolve; });+ const holder = brain().engine.transaction(async tx => {+ await tx.executeRaw('INSERT INTO ingest_applied_captures (idempotency_key, source_id, slug, result) VALUES ($1, $2, $3, $4::jsonb)', [key, SRC, slug, stored]);+ inserted();+ await held;+ });+ await insertedP;+ const run = makeIngestCaptureHandler(brain().engine)(jobCtx(4, data, key));+ let blocked = false;+ for (let i = 0; i < 200 && !blocked; i++) {+ await Bun.sleep(50);+ blocked = (await brain().engine.executeRaw<{ n: number }>(+ "SELECT count(*)::int AS n FROM pg_stat_activity WHERE wait_event_type = 'Lock' AND query LIKE '%INSERT INTO ingest_applied_captures%'"))[0]!.n > 0;+ }+ release();+ await holder;+ expect(blocked).toBe(true);+ expect(await run).toEqual(stored);+ expect(await pageText(slug)).toContain('Edit already on the page.');+ expect(await pageText(slug)).not.toContain('Concurrent captured body.');+ }, 30_000);++ test('a keyed capture whose first write hits the pages source FK lands under default, its first record rolled back', async () => {+ const racingSource = `racing-${backend}`;+ await brain().engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT (id) DO NOTHING', [racingSource]);+ let attempts = 0;+ // The first write transaction runs in full (page and record), then fails as if the source row vanished under it.+ const racing = new Proxy(brain().engine, {+ get(target, prop) {+ if (prop === 'transaction') {+ return async <T>(fn: (tx: BrainEngine) => Promise<T>): Promise<T> => {+ if (++attempts > 1) return target.transaction(fn);+ return target.transaction(async tx => {+ await fn(tx);+ throw Object.assign(new Error('insert or update on table "pages" violates foreign key constraint "pages_source_id_fk"'), { code: '23503' });+ });+ };+ }+ const value = Reflect.get(target, prop, target);+ return typeof value === 'function' ? value.bind(target) : value;+ },+ });+ const slug = `notes/fk-race-${backend}`;+ const key = `ingest:webhook:client-example:${racingSource}:fk-race`;+ const result = await makeIngestCaptureHandler(racing)(jobCtx(5, captureData(note('Racing capture', 'Racing body.'), slug, racingSource), key));+ expect(attempts).toBe(2);+ expect(result.status).toBe('imported');+ expect(result.source_fallback).toEqual({ requested: racingSource, effective: 'default', reason: 'fk_violation' });+ expect(await brain().engine.executeRaw('SELECT source_id FROM ingest_applied_captures WHERE idempotency_key = $1', [key])).toEqual([{ source_id: 'default' }]);+ expect((await brain().engine.readPageSnapshot(slug, { sourceId: 'default' }))?.page.compiled_truth).toContain('Racing body.');+ expect(await brain().engine.readPageSnapshot(slug, { sourceId: racingSource })).toBeNull();+ });++ test('the purge keeps records for 90 days', async () => {+ const insert = (key: string, age: string) => brain().engine.executeRaw(+ `INSERT INTO ingest_applied_captures (idempotency_key, source_id, slug, result, applied_at) VALUES ($1, $2, 'notes/purge', '{}'::jsonb, now() - $3::interval)`, [key, SRC, age]);+ await insert(`purge-old-${backend}`, '91 days');+ await insert(`purge-recent-${backend}`, '89 days');+ expect(await purgeStaleAppliedCaptures(brain().engine)).toBe(1);+ const kept = await brain().engine.executeRaw<{ idempotency_key: string }>("SELECT idempotency_key FROM ingest_applied_captures WHERE slug = 'notes/purge'");+ expect(kept.map(r => r.idempotency_key)).toEqual([`purge-recent-${backend}`]);+ });++ test('a job queued without a key writes and records nothing', async () => {+ const slug = `notes/keyless-${backend}`;+ const data = captureData(note('Keyless capture', 'Keyless body.'), slug, SRC);+ expect((await makeIngestCaptureHandler(brain().engine)(jobCtx(2, data, null))).status).toBe('imported');+ expect(await pageText(slug)).toContain('Keyless body.');+ expect(await brain().engine.executeRaw('SELECT 1 FROM ingest_applied_captures WHERE slug = $1', [slug])).toHaveLength(0);+ });+ });++ describe(`ingest_capture applies a capture once on a managed brain (${backend})`, () => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;++ test('a re-run after the door committed keeps the newer edit, from the journal when the record row is missing', () => managedBrain(async ({ engine, ctx }) => {+ const slug = 'notes/managed-rerun';+ const data = captureData(note('Managed capture', 'Managed captured body.'), slug, 'default');+ const key = 'ingest:webhook:client-example:default:managed-rerun';+ const run = makeIngestCaptureHandler(engine);+ const first = await run(jobCtx(3, data, key));+ expect(first.status).toBe('imported');+ const snapshot = await engine.readPageSnapshot(slug, { sourceId: 'default' });+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug, content: note('Managed capture', 'Managed newer edit.'), expected_revision: snapshot!.revision } });+ const pageText = async () => (await engine.readPageSnapshot(slug, { sourceId: 'default' }))?.page.compiled_truth ?? '';+ expect(await pageText()).toContain('Managed newer edit.');++ // The worker died after the door committed and before the record row was inserted.+ await engine.executeRaw('DELETE FROM ingest_applied_captures WHERE idempotency_key = $1', [key]);+ expect(await run(jobCtx(3, data, key))).toEqual(first);+ expect(await pageText()).toContain('Managed newer edit.');+ expect(await engine.executeRaw('SELECT 1 FROM ingest_applied_captures WHERE idempotency_key = $1', [key])).toHaveLength(1);++ expect(await run(jobCtx(3, data, key, 1))).toEqual(first);+ expect(await pageText()).toContain('Managed newer edit.');+ expect(await pageText()).not.toContain('Managed captured body.');+ }, { databaseUrl }), 120_000);+ });+}diff --git a/test/ingest-capture-private-hold-7880.test.ts b/test/ingest-capture-private-hold-7880.test.tsindex 590139abad..a91e1e4431 100644--- a/test/ingest-capture-private-hold-7880.test.ts+++ b/test/ingest-capture-private-hold-7880.test.ts@@ -55,7 +55,7 @@ function event(content: string, untrusted: boolean): IngestionEvent {return { source_id: 'source-a', source_kind: 'webhook', source_uri: 'mcp-webhook:client-example:1', received_at: new Date('2026-10-28T12:00:00Z').toISOString(),content_type: 'text/markdown', content, content_hash: computeContentHash(content), untrusted_payload: untrusted };}-const job = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 0, signal: new AbortController().signal,+const job = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 0, idempotency_key: null, signal: new AbortController().signal,deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {},isActive: async () => true, readInbox: async () => [] });type Outcome = { status?: string; thrown?: unknown; code?: string; message?: string };diff --git a/test/ingestion/ingest-capture.test.ts b/test/ingestion/ingest-capture.test.tsindex 677a6590a8..87ecf86a46 100644--- a/test/ingestion/ingest-capture.test.ts+++ b/test/ingestion/ingest-capture.test.ts@@ -58,6 +58,7 @@ function makeJob(data: Record<string, unknown>): MinionJobContext {name: 'ingest_capture',data,attempts_made: 1,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/jobs-import-errors.test.ts b/test/jobs-import-errors.test.tsindex 75e9cc6438..b5a906e091 100644--- a/test/jobs-import-errors.test.ts+++ b/test/jobs-import-errors.test.ts@@ -41,6 +41,7 @@ test('queued import rejects malformed frontmatter, preserves progress, and succename: 'import',data: { dir: root, sourceId: 'default', noEmbed: true },attempts_made: 0,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/managed-atoms-stop-reason.test.ts b/test/managed-atoms-stop-reason.test.tsindex 16bc64ae3b..f6a0195047 100644--- a/test/managed-atoms-stop-reason.test.ts+++ b/test/managed-atoms-stop-reason.test.ts@@ -85,7 +85,7 @@ for (const stop of ['length', 'refusal', 'content_filter'] as const) {__setChatTransportForTests(chat);const worker = new MinionWorker(engine, { queue: 'fixture' });await registerBuiltinHandlers(worker, engine, { quiet: true });- const job: MinionJobContext = { id: 951, name: 'extract-atoms-drain', data: { sourceId: 'default', retryRequestId: receipt.request_id }, attempts_made: 0,+ const job: MinionJobContext = { id: 951, name: 'extract-atoms-drain', data: { sourceId: 'default', retryRequestId: receipt.request_id }, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] };const retried = await worker.getHandler('extract-atoms-drain')!(job) as Record<string, unknown>;diff --git a/test/minions-shell-process-tree.test.ts b/test/minions-shell-process-tree.test.tsindex e12f7c881e..9cf7be7ac9 100644--- a/test/minions-shell-process-tree.test.ts+++ b/test/minions-shell-process-tree.test.ts@@ -24,6 +24,7 @@ function makeCtx(data: Record<string, unknown>, signal: AbortSignal): MinionJobCname: 'shell',data,attempts_made: 0,+ idempotency_key: null,signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/minions-shell.test.ts b/test/minions-shell.test.tsindex 42f801cc1a..72a47c56ac 100644--- a/test/minions-shell.test.ts+++ b/test/minions-shell.test.ts@@ -51,6 +51,7 @@ function makeCtx(name: 'shell',data,attempts_made: 0,+ idempotency_key: null,signal: opts.signal ?? new AbortController().signal,deadlineAtMs: null,shutdownSignal: opts.shutdownSignal ?? new AbortController().signal,diff --git a/test/minions/delegated-execution.serial.test.ts b/test/minions/delegated-execution.serial.test.tsindex 6f63f85f83..bd28003b95 100644--- a/test/minions/delegated-execution.serial.test.ts+++ b/test/minions/delegated-execution.serial.test.ts@@ -49,7 +49,7 @@ async function ownedJob(budget: number | null = null, extra: Record<string, unknallowed_tools: snapshot.tools, allowed_slug_prefixes: snapshot.slugPrefixes, source_id: snapshot.sourceId,...extra, __owner_client_id: 'owner', __delegation_grant: snapshot };const job = await new MinionQueue(engine).add('subagent', data, {}, { allowProtectedSubmit: true, delegatedClientId: 'owner' });- return { id: job.id, name: job.name, data, attempts_made: 0,+ return { id: job.id, name: job.name, data, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, shutdownSignal: new AbortController().signal, deadlineAtMs: null,async updateProgress() {}, async updateTokens() {}, async log() {}, async isActive() { return true; }, async readInbox() { return []; } };}diff --git a/test/minions/subagent-oneshot.test.ts b/test/minions/subagent-oneshot.test.tsindex b44dac8133..8ed133de87 100644--- a/test/minions/subagent-oneshot.test.ts+++ b/test/minions/subagent-oneshot.test.ts@@ -76,6 +76,7 @@ async function makeCtx(data: SubagentHandlerData): Promise<MinionJobContext> {name: job.name,data: data as unknown as Record<string, unknown>,attempts_made: 0,+ idempotency_key: null,signal: ac.signal,deadlineAtMs: null,shutdownSignal: shutdown.signal,diff --git a/test/minions/subagent-turn-timeout.test.ts b/test/minions/subagent-turn-timeout.test.tsindex ef57295a10..a5c713521d 100644--- a/test/minions/subagent-turn-timeout.test.ts+++ b/test/minions/subagent-turn-timeout.test.ts@@ -81,6 +81,7 @@ async function makeCtx(data: SubagentHandlerData, deadlineAtMs: number | null):name: 'subagent',data: data as unknown as Record<string, unknown>,attempts_made: 0,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs,shutdownSignal: new AbortController().signal,diff --git a/test/patterns-staged-writes.test.ts b/test/patterns-staged-writes.test.tsindex 1eeca3407f..97bc85216e 100644--- a/test/patterns-staged-writes.test.ts+++ b/test/patterns-staged-writes.test.ts@@ -58,7 +58,7 @@ function oneWrite(input: Record<string, unknown>): void {}function jobContext(job: { id: number; name: string; data: Record<string, unknown> }): MinionJobContext {- return { id: job.id, name: job.name, data: job.data, attempts_made: 0, signal: new AbortController().signal, shutdownSignal: new AbortController().signal,+ return { id: job.id, name: job.name, data: job.data, attempts_made: 0, idempotency_key: null, signal: new AbortController().signal, shutdownSignal: new AbortController().signal,deadlineAtMs: null, async updateProgress() {}, async updateTokens() {}, async log() {}, async isActive() { return true; }, async readInbox() { return []; } };}diff --git a/test/serve-http-ingest-legacy-key-retry.test.ts b/test/serve-http-ingest-legacy-key-retry.test.tsindex 45696f773a..54561c8c87 100644--- a/test/serve-http-ingest-legacy-key-retry.test.ts+++ b/test/serve-http-ingest-legacy-key-retry.test.ts@@ -68,7 +68,7 @@ beforeAll(async () => {}, 120_000);afterAll(async () => { for (const brain of brains.values()) await brain.close(); }, 60_000);-const jobCtx = (id: number, data: Record<string, unknown>): MinionJobContext => ({ id, name: 'ingest_capture', data, attempts_made: 0,+const jobCtx = (id: number, data: Record<string, unknown>): MinionJobContext => ({ id, name: 'ingest_capture', data, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {},updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] });const jsonData = (data: unknown) => (typeof data === 'string' ? JSON.parse(data) : data) as Record<string, unknown>;diff --git a/test/serve-http-ingest-member-access.test.ts b/test/serve-http-ingest-member-access.test.tsindex 0a9ed7f7a1..eaa4c3fd00 100644--- a/test/serve-http-ingest-member-access.test.ts+++ b/test/serve-http-ingest-member-access.test.ts@@ -81,7 +81,7 @@ const ingest = (slug: string, body = takeover) => fetch(`${base}/ingest`, {headers: { authorization: `Bearer ${bobToken}`, 'content-type': 'text/markdown', 'x-gbrain-slug': slug },body,});-const jobCtx = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 0,+const jobCtx = (data: Record<string, unknown>): MinionJobContext => ({ id: 1, name: 'ingest_capture', data, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {},updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] });const webhookEvent = (content: string): IngestionEvent => ({ source_id: 'webhook-client-example', source_kind: 'webhook',diff --git a/test/serve-http-ingest-write-loss.test.ts b/test/serve-http-ingest-write-loss.test.tsindex 0b1d4bd95f..c2b8a69482 100644--- a/test/serve-http-ingest-write-loss.test.ts+++ b/test/serve-http-ingest-write-loss.test.ts@@ -72,7 +72,7 @@ const ingest = async (body: string, slug?: string) => {expect(res.status).toBe(202);return (await res.json() as { job_id: number }).job_id;};-const jobCtx = (id: number, data: Record<string, unknown>): MinionJobContext => ({ id, name: 'ingest_capture', data, attempts_made: 0,+const jobCtx = (id: number, data: Record<string, unknown>): MinionJobContext => ({ id, name: 'ingest_capture', data, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal, updateProgress: async () => {},updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] });const runJob = async (id: number) => {diff --git a/test/subagent-aggregator.test.ts b/test/subagent-aggregator.test.tsindex 318fb12e3a..4467f07bdf 100644--- a/test/subagent-aggregator.test.ts+++ b/test/subagent-aggregator.test.ts@@ -39,6 +39,7 @@ function ctxWithInbox(name: 'subagent_aggregator',data,attempts_made: 0,+ idempotency_key: null,signal: new AbortController().signal,deadlineAtMs: null,shutdownSignal: new AbortController().signal,diff --git a/test/subagent-handler.test.ts b/test/subagent-handler.test.tsindex bc4c746f88..9c0a5d8dba 100644--- a/test/subagent-handler.test.ts+++ b/test/subagent-handler.test.ts@@ -91,6 +91,7 @@ async function makeCtx(input: unknown): Promise<MinionJobContext> {name: job.name,data: (input as Record<string, unknown>) ?? {},attempts_made: 0,+ idempotency_key: null,signal: ac.signal,deadlineAtMs: null,shutdownSignal: shutdown.signal,diff --git a/test/subagent-put-page-rejection.serial.test.ts b/test/subagent-put-page-rejection.serial.test.tsindex 525feb2826..2376eaee98 100644--- a/test/subagent-put-page-rejection.serial.test.ts+++ b/test/subagent-put-page-rejection.serial.test.ts@@ -52,7 +52,7 @@ async function makeContext(lane: 'anthropic' | 'gateway' | 'oneshot'): Promise<M};const job = await queue.add('subagent', data, {}, { allowProtectedSubmit: true });return {- id: job.id, name: job.name, data, attempts_made: 0, deadlineAtMs: null,+ id: job.id, name: job.name, data, attempts_made: 0, idempotency_key: null, deadlineAtMs: null,signal: new AbortController().signal, shutdownSignal: new AbortController().signal,async updateProgress() {}, async updateTokens() {}, async log() {},async isActive() { return true; }, async readInbox() { return []; },diff --git a/test/subagent-stop-reason-6448.test.ts b/test/subagent-stop-reason-6448.test.tsindex dae0723e25..3d23e4b332 100644--- a/test/subagent-stop-reason-6448.test.ts+++ b/test/subagent-stop-reason-6448.test.ts@@ -82,7 +82,7 @@ class FakeMessagesClient implements MessagesClient {async function makeCtx(input: Record<string, unknown>): Promise<MinionJobContext> {const job = await queue.add('subagent', input, {}, { allowProtectedSubmit: true });return {- id: job.id, name: job.name, data: input, attempts_made: 0,+ id: job.id, name: job.name, data: input, attempts_made: 0, idempotency_key: null,signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,async updateProgress() {}, async updateTokens() {}, async log() {},async isActive() { return true; }, async readInbox() { return []; },