Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix: CI gates on #8989: nosemgrep notes on the probe spawn and npm root join; move the doctor budgets entry to its own key-files page (size cap)

garrytanc469365c375a
diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.md
index dd268e867c..c7157eff73 100644
--- a/docs/architecture/KEY_FILES.md
+++ b/docs/architecture/KEY_FILES.md
@@ -29,6 +29,7 @@ link it here; never raise the cap.
| [Commands (1/6)](key-files/commands-1.md) | `agent-logs.ts` to `db-repair.ts` |
| [Commands (2/6)](key-files/commands-2.md) | `src/commands/doctor.ts` and `src/commands/doctor/` |
| [Commands (2/6, continued)](key-files/commands-2-continued.md) | `doctor/checks/routing-federation.ts`, `git-convergence.ts`, `keyword-plan.ts`, `dream-retriage.ts` to `embed.ts` |
+| [Doctor tiers and budgets](key-files/doctor-budgets.md) | `doctor/budget.ts`, `checks/ops-sections.ts`, `checks/orphan-estimate.ts`: tiers, `--budget-ms`, entry deadlines, `localProbe` |
| [Commands (3/6)](key-files/commands-3.md) | `src/commands/engine-status.ts` through `extract.ts` |
| [Commands (3/6, continued)](key-files/commands-3-continued.md) | `src/commands/extract-timeline-db.ts` through `frontmatter-install-hook.ts` |
| [Commands (4/6)](key-files/commands-4.md) | `src/commands/frontmatter.ts` through `pages-status.ts` |
diff --git a/docs/architecture/key-files/commands-2-continued.md b/docs/architecture/key-files/commands-2-continued.md
index dbdab4f430..70f176a0d3 100644
--- a/docs/architecture/key-files/commands-2-continued.md
+++ b/docs/architecture/key-files/commands-2-continued.md
@@ -33,7 +33,6 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/commands/doctor/checks/keyword-plan.ts` — `keyword_plan` (#6495, OPS, wave check with operator resolution; the keyword-side twin of `vector_plan` in commands-2.md). Postgres only: reads the commonest lexeme the planner knows for `content_chunks.search_vector` (`pg_stats.most_common_elems` with its frequency), EXPLAINs (no ANALYZE) the strict page-count statement for it (`buildKeywordCountStatement`, the builder the engine uses) and times the engine's own bounded `countKeywordPages(term, { timeoutMs: 2000 })`; warns `canary_timed_out` when that count hits its bound and `canary_slow` when it takes more than half of `search.keyword_statement_timeout_ms`, naming the knob with its range (1000-60000), the fail-open to the vector arm (`keyword_arm_failed`), that a longer bound buys time and not a cheaper statement, that one canary term is not workload coverage, and that a timed-out count is never "no matches". Skipped on PGLite, below 10,000 chunks (`pg_class.reltuples`) and without planner statistics (it names `ANALYZE content_chunks`); a failing probe reports unknown health. `engineKeywordPlanProbe(engine)` is the database side; `keywordPlanCheck(engine, probe?)` takes a fake for tests. Tested in `test/doctor-keyword-plan-6495.test.ts` (fake probe for every outcome; with DATABASE_URL the real probe on a seeded brain and a lock-forced timeout).
- `src/commands/doctor/checks/image-embedding-coverage.ts` — `image_embedding_coverage` (#6529, brain category, after `image_assets`). Counts live image pages (`page_kind = 'image'`), those with no `embedding_image` chunk, those with a NULL `source_path`, and the `managed_image_sync_unsupported` holds per source, and reads `isMultimodalEnabled()`. Ok when the brain has no image pages, when every image page has an image vector and a `source_path` and nothing is held, or when multimodal embedding is off (the counts are then informational: image search is not expected to work). On a multimodal install it warns with the counts (text coverage counts those pages as embedded while image search does not see them) and names `gbrain embed --stale --images` and `gbrain sync --no-pull --source <id>` for the holds. Read-only; no paid call. Test: `test/doctor-image-embedding-coverage.test.ts`.
- `src/commands/doctor/checks/provider-consent.ts` — `provider_consent` (OPS, engine-free filesystem lane right after `embedding_key_source`; not in `doctorReportRemote`): the provider-use decision `gbrain setup` recorded for this install (`localSetupRecords` in `src/core/setup/local-receipts.ts`). A decline (any live receipt; opt-outs win) is `ok` + info `disabled_by_choice`, "skipped by consent", naming the provider key variables present (names only) and carrying `providerConsentFix`: `gbrain setup <harness> --providers` (+ `--name`/`--scope project`), actor `user`, consent `paid`+`egress`, so `next: tell_user_to_run`. Accepted is plain `ok`; no receipt is info `not_applicable`. The same fix rides `embedding_provider`'s `--probe` skip (`checks/embedding-health.ts`), and `probe-consent.ts` authorizes nothing when a decline is recorded. Reads files only; sends nothing. Pinned by `test/doctor-provider-consent.serial.test.ts` (request counter: zero outbound under a decline, controls prove discovery and the probe are seen otherwise).
-- `src/commands/doctor/budget.ts` + `checks/ops-sections.ts` + `checks/orphan-estimate.ts` — doctor tiers and budgets (#6696 B1). `parseDoctorTier` (`--tier ops|quick|standard|deep`, a bad value is `invalid_params`), `parseBudgetMs` (`--budget-ms`, default 4000 under ops, 10000 under quick, none otherwise) and `startDoctorDeadline` build the `DoctorDeadline` the CLI starts before connecting. `runDoctorRegistry` runs the entries tagged `tier: 'ops'` (persistence capacity, terminal failures, managed sync movement, persistence compaction, serve build drift, schema-pack consistency, embedding state) right after the connection lane; under `--tier ops` it runs only those (each `executeRaw` gets the absolute `deadlineAt`, so `SET LOCAL statement_timeout` is the time left after the pool wait: `postgres-engine/bounded-statement.ts`), reports an entry marked `opsPglite: 'not_applicable'` as an `ok` info row on PGLite, and stays complete. Every other tier bounds each entry by `doctor.entry_budget_ms` (default 15000). An entry past its budget, an unknown reported after the overall deadline, or a connect the deadline ended becomes `budgetUnknownCheck` (warn, `details.health: 'unknown'`, `reason: 'budget'`, `exact: false`, a rerun-alone fix), never a clear result; every check carries `elapsed_ms`, and the report adds `tier`/`complete`/`unknown_sections`/`elapsed_ms`/`budget_ms` only when `--tier` or `--budget-ms` was given (`doctorRunSummary`). `doctorHealth(ctx)` memoizes `getHealth` for one run. `orphan_ratio` estimates from a `TABLESAMPLE SYSTEM` sample above `doctor.orphan_ratio.full_scan_max_pages` (default 50000) unless `--tier deep`. Capacity and compaction freshness live in `checks/ops-sections.ts`. On every tier each entry runs inside its own `StatementDeadline` (`src/core/statement-deadline.ts`: an AsyncLocalStorage scope plus `deadlineBoundEngine`, one stable Proxy per real engine that refuses any engine call past the deadline, carries the deadline to the server on Postgres `executeRaw`, and marks the scope expired on a cancel even when the check catches it; `engineCacheKey` maps the wrapper to the real engine for engine-keyed caches; PGLite gets only the pre-statement refusal). The budget-config read is one bounded statement (2 s). `CHEAP_INCIDENT_CHECKS` (by check name: persistence capacity, managed sync movement, `text_projection_readiness`, `partial_embedding`, `links_extraction_lag`) run right after the ops sections, before any full scan. #8979: with an engine and without `--fast` or `--only`, the entries before the connection lane stop at half the time left when the check run starts, so the ops sections and the cheap incident checks keep the other half. `local-audits.ts` `localProbe` runs the home-dir entry's `npm root -g`, `which -a gbrain` and `crontab -l` asynchronously and kills each with SIGKILL at the entry deadline (5 s at most), so a hung probe never holds the run past `--budget-ms`. `scanDeadline` clips the frontmatter walk and the fence census to the entry deadline less 250 ms, so they report partial results. `--only` with an explicit `--budget-ms` gives the selected entries the whole overall budget; `budgetUnknownCheck` names the limiting budget (`details.limit`: `entry` or `overall`) and fixes with `--only <check> --budget-ms N` (N ten times that budget; actor agent while N fits `AGENT_TOOL_LIMIT_MS` less 10 s, else actor user). `--budget-ms 0` is no overall budget. `default-budget.ts`: a plain non-interactive CLI `doctor --json` (stderr not a TTY; no `--budget-ms`, `--tier`, `--fix`, `--remediate` or `--remediation-plan`) caps its check run at `doctor.default_budget_ms` (file plane, default 90000, 0 off), started by `startDeferredDeadline` when the registry starts (connect and migrations are outside it), with one stderr line; the report then carries the tier fields. `capacity-memo.ts` `doctorCapacityForecast`: one forecast per run (shared by `persistence_capacity` and `persistence_request_growth`) read under the run's overall deadline and dropped from the memo on a deadline failure, which makes the waiting check unknown instead of a counter-only verdict. `persistence_session_timeouts` reads `SHOW statement_timeout` outside the entry deadline. Tests: `test/doctor-tier-6696.test.ts`, `test/doctor-day-zero.test.ts` (`--tier ops`), `test/e2e/doctor-ops-tier-postgres.test.ts`, `test/doctor-entry-deadline-6803.test.ts` (+ `test/e2e/doctor-entry-deadline-postgres.test.ts`), `test/doctor-default-budget-6803.test.ts`; bench `scripts/bench/doctor-scale.ts`. #7276: `scoredChecks` keeps over-budget unknowns out of every score (`unknown_count`); `derived_visibility` is its own entry (`test/doctor-unknown-score-7276.test.ts`).
- `src/commands/doctor/last-known.ts` — the quick tier's last known results (#7142). `--tier quick` (budget.ts: `QUICK_BUDGET_MS_DEFAULT` 10000, connect included; `entryRunsAt` orders ops ⊂ quick ⊂ standard ⊂ deep; `cheapTier` gives it the ops tier's round-trip connection check and counter-mode embedding state) runs the ops entries plus the entries tagged `tier: 'quick'` (schema version, the stub-guard entry with `sync_failures`, queue health, partial embeddings). `lastKnownMode` turns the store on only for a brain-wide local CLI run (`buildChecks(..., { lastKnown: 'cli' })` from `runDoctor`) or the refresh job, never under `--only`, `--source`, a remote scope, `--fix` or `--tier ops`. `runDoctorRegistry` records each entry's outcome (`done`, `expired` when it ran out of budget or emitted a budget unknown, `not_run` when the tier skipped a standard entry); `withLastKnown` writes the finished entries of a standard or deep run to `op_checkpoints` (op `doctor-last-known`, fingerprint = entry name, `completed_keys[0]` = `{ checks, measured_at, elapsed_ms, emits }`, one bounded statement, best-effort) and `applyLastKnown` replaces an unfinished entry's checks with its last known ones when they are at most `LAST_KNOWN_MAX_AGE_MS` (24 h) old: same status, message prefixed `Last known, N ago (...)`, `details.last_known` `{ measured_at, age_ms, reason: not_in_tier | budget }`. The rest are named in one `doctor_last_known` warning (`details.health: 'unknown'`, `unknown_checks`), which `tierFields` expands into `unknown_sections`; served checks go to `last_known_sections` and make the report incomplete. An entry with no result younger than `REFRESH_AFTER_MS` (1 h) queues one `doctor-refresh` job on Postgres (idempotency key per hour, `maxWaiting: 1`, priority 10 so sync, the autopilot cycles, global maintenance, embed backfill and loops extract claim first, `timeout_ms` 15 min; none when DB config `doctor.background_refresh` is `false`, #7263); the handler (`runDoctorRefresh`, registered in `commands/jobs.ts`) runs `doctorRefreshRegistry()` (the connection lane plus every standard entry after the connection gate; filesystem entries are recorded only by CLI runs) with `REFRESH_ENTRY_BUDGET_MS` (600000) per entry inside the overall `REFRESH_BUDGET_MS` (10 min, passed as `--budget-ms`: past it no entry starts and the running one is cut, so the job completes with what it measured and the rest waits for the next slot) and queues nothing itself. `doctor-refresh` is in `DEFAULT_LONG_JOB_NAMES` (`minions/lanes.ts`). Tests: `test/doctor-refresh-7263.test.ts`, `test/doctor-quick-tier-7142.test.ts` (+ `test/e2e/doctor-quick-tier-7142-postgres.test.ts`).
- `src/commands/doctor/brain-health-bands.ts` + `remote-tier.ts` + `ops-snapshot.ts`, `src/core/doctor-remote-tier.ts`, `src/core/persistence/capacity-view.ts` — the ops tier for agents and remote clients (#6696 B2). `readCapacityView` (for `/admin/api/ops`) reads the counter rows, `capacityDiagnostics`, the light `readCapacityForecast` (null when unreadable: days unknown) and `readLastCompaction`; the bands read `cachedCapacitySnapshot`; `capacityVerify()` is the verify of every capacity fix (`gbrain doctor --only persistence_capacity --json` plus MCP `get_brain_health`). `readBrainHealthBands` backs the read-scope `get_brain_health` op: capacity band and days band, write failures and active blockers as count bands for the caller's readable sources, no slugs or exact counters, cached 60 s per engine and scope. `runRemoteDoctorTier` backs `run_doctor`'s `tier: "ops"` / `only` (admin). `runScopedRemoteDoctor` is the thin client's `gbrain doctor --tier ops` / `--only <registry check>`: it reads `tools/list` and uses `run_doctor` with the parameters, else the bands (`BAND_CHECKS`), else refuses `doctor_tier_unavailable`; it never runs the full remote report in their place. `cachedOpsSnapshot` serves `/admin/api/ops` from one ops run a minute. Tests: `test/doctor-ops-remote-6696.test.ts`.
diff --git a/docs/architecture/key-files/doctor-budgets.md b/docs/architecture/key-files/doctor-budgets.md
new file mode 100644
index 0000000000..0323ea923b
--- /dev/null
+++ b/docs/architecture/key-files/doctor-budgets.md
@@ -0,0 +1,6 @@
+# Key files: Doctor tiers and budgets
+
+[Subsystem index](../KEY_FILES.md) · [Doctor commands](commands-2.md) · [continued](commands-2-continued.md). Read only the entries relevant to your change.
+Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
+
+- `src/commands/doctor/budget.ts` + `checks/ops-sections.ts` + `checks/orphan-estimate.ts` — doctor tiers and budgets (#6696 B1). `parseDoctorTier` (`--tier ops|quick|standard|deep`, a bad value is `invalid_params`), `parseBudgetMs` (`--budget-ms`, default 4000 under ops, 10000 under quick, none otherwise) and `startDoctorDeadline` build the `DoctorDeadline` the CLI starts before connecting. `runDoctorRegistry` runs the entries tagged `tier: 'ops'` (persistence capacity, terminal failures, managed sync movement, persistence compaction, serve build drift, schema-pack consistency, embedding state) right after the connection lane; under `--tier ops` it runs only those (each `executeRaw` gets the absolute `deadlineAt`, so `SET LOCAL statement_timeout` is the time left after the pool wait: `postgres-engine/bounded-statement.ts`), reports an entry marked `opsPglite: 'not_applicable'` as an `ok` info row on PGLite, and stays complete. Every other tier bounds each entry by `doctor.entry_budget_ms` (default 15000). An entry past its budget, an unknown reported after the overall deadline, or a connect the deadline ended becomes `budgetUnknownCheck` (warn, `details.health: 'unknown'`, `reason: 'budget'`, `exact: false`, a rerun-alone fix), never a clear result; every check carries `elapsed_ms`, and the report adds `tier`/`complete`/`unknown_sections`/`elapsed_ms`/`budget_ms` only when `--tier` or `--budget-ms` was given (`doctorRunSummary`). `doctorHealth(ctx)` memoizes `getHealth` for one run. `orphan_ratio` estimates from a `TABLESAMPLE SYSTEM` sample above `doctor.orphan_ratio.full_scan_max_pages` (default 50000) unless `--tier deep`. Capacity and compaction freshness live in `checks/ops-sections.ts`. On every tier each entry runs inside its own `StatementDeadline` (`src/core/statement-deadline.ts`: an AsyncLocalStorage scope plus `deadlineBoundEngine`, one stable Proxy per real engine that refuses any engine call past the deadline, carries the deadline to the server on Postgres `executeRaw`, and marks the scope expired on a cancel even when the check catches it; `engineCacheKey` maps the wrapper to the real engine for engine-keyed caches; PGLite gets only the pre-statement refusal). The budget-config read is one bounded statement (2 s). `CHEAP_INCIDENT_CHECKS` (by check name: persistence capacity, managed sync movement, `text_projection_readiness`, `partial_embedding`, `links_extraction_lag`) run right after the ops sections, before any full scan. #8979: with an engine and without `--fast` or `--only`, the entries before the connection lane stop at half the time left when the check run starts, so the ops sections and the cheap incident checks keep the other half. `local-audits.ts` `localProbe` runs the home-dir entry's `npm root -g`, `which -a gbrain` and `crontab -l` asynchronously and kills each with SIGKILL at the entry deadline (5 s at most), so a hung probe never holds the run past `--budget-ms`. `scanDeadline` clips the frontmatter walk and the fence census to the entry deadline less 250 ms, so they report partial results. `--only` with an explicit `--budget-ms` gives the selected entries the whole overall budget; `budgetUnknownCheck` names the limiting budget (`details.limit`: `entry` or `overall`) and fixes with `--only <check> --budget-ms N` (N ten times that budget; actor agent while N fits `AGENT_TOOL_LIMIT_MS` less 10 s, else actor user). `--budget-ms 0` is no overall budget. `default-budget.ts`: a plain non-interactive CLI `doctor --json` (stderr not a TTY; no `--budget-ms`, `--tier`, `--fix`, `--remediate` or `--remediation-plan`) caps its check run at `doctor.default_budget_ms` (file plane, default 90000, 0 off), started by `startDeferredDeadline` when the registry starts (connect and migrations are outside it), with one stderr line; the report then carries the tier fields. `capacity-memo.ts` `doctorCapacityForecast`: one forecast per run (shared by `persistence_capacity` and `persistence_request_growth`) read under the run's overall deadline and dropped from the memo on a deadline failure, which makes the waiting check unknown instead of a counter-only verdict. `persistence_session_timeouts` reads `SHOW statement_timeout` outside the entry deadline. Tests: `test/doctor-tier-6696.test.ts`, `test/doctor-day-zero.test.ts` (`--tier ops`), `test/e2e/doctor-ops-tier-postgres.test.ts`, `test/doctor-entry-deadline-6803.test.ts` (+ `test/e2e/doctor-entry-deadline-postgres.test.ts`), `test/doctor-default-budget-6803.test.ts`; bench `scripts/bench/doctor-scale.ts`. #7276: `scoredChecks` keeps over-budget unknowns out of every score (`unknown_count`); `derived_visibility` is its own entry (`test/doctor-unknown-score-7276.test.ts`).
diff --git a/src/commands/doctor/checks/local-audits.ts b/src/commands/doctor/checks/local-audits.ts
index 70fbd9d817..8fe5666a14 100644
--- a/src/commands/doctor/checks/local-audits.ts
+++ b/src/commands/doctor/checks/local-audits.ts
@@ -397,6 +397,7 @@ async function localProbe(entryDeadlineAt: number | undefined, file: string, arg
const { spawn } = await import('../../../core/spawn.ts');
const posix = process.platform !== 'win32';
return new Promise((resolve) => {
+ // nosemgrep: javascript.lang.security.audit.spawn-shell-true.spawn-shell-true -- file and args are constants from the three call sites (npm root -g, which -a gbrain, crontab -l); the shell is Windows-only, where npm resolves to npm.cmd, as with the execFile it replaced; trusted local CLI only
const child = spawn(file, args, { shell: !posix, detached: posix, stdio: ['ignore', 'pipe', 'ignore'] });
let out = '';
let done = false;
@@ -449,6 +450,7 @@ async function runHomeDirInWorktree(ctx: DoctorContext): Promise<Check[]> {
const globalPackages = [join(bunGlobalRoot, 'node_modules', 'gbrain')];
// No npm (or it failed): only Bun's global dir is checked.
const npmRoot = (await localProbe(entryDeadlineAt, 'npm', ['root', '-g']))?.trim() || undefined;
+ // nosemgrep: javascript.lang.security.audit.path-traversal.path-join-resolve-traversal.path-join-resolve-traversal -- npmRoot is the local operator's own `npm root -g` output, joined only to look for an installed gbrain package; trusted local CLI only
if (npmRoot) globalPackages.push(join(npmRoot, 'gbrain'));
// `which` exits non-zero when gbrain isn't on PATH (or is missing
// entirely on this platform) — nothing to check.