Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
52.9 KB
Raw Blame

Key files: Core Services (1/3)

Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.

  • src/core/advisor/{types,run,render,recommended-set,history,apply,collect-*}.ts + src/commands/advisor.ts — gbrain advisor: read-only ranked actions from brain state. run.runAdvisor executes the hardcoded COLLECTORS array (version [cache-only], migration, schema-pack, stalled-jobs [tolerates only missing minion_jobs, sources or sources.last_sync_at], usage-shape, setup-smells, uninstalled-brain-pack, uninstalled-bundled, chronicle, mcp-client-fit), each in its own try/catch that reports a thrown collector as the static collector_failed:<id> warn; built-in collectors rethrow read failures instead of returning no findings; collect-mcp-client-fit.ts (E3) flags full-surface MCP clients whose 30d usage fits STARTER_OPS (exact rescope-client --surface starter fix; ≥10-call threshold; automation-shaped clients excluded per D12) plus STARTER_OPS drift (top-used ops missing; starter members unused 90d) via the shared src/core/mcp-usage.ts reader — starter membership is judged against the exported ALWAYS_INCLUDED_STARTER_OPS (surface.ts) so the always-included lane never reads as unused, and the missing-from-starter arm excludes localOnly ops (never proposable for a network surface, mirroring derive-starter-ops) — REMOTE runs redact client identifiers to aggregate counts (amendment 29), and its dismiss/snooze rides the nag-state engine with its own ~/.gbrain/advisor-usage-nag-state.json (local runs only); rankFindings orders critical>warn>info then collector order, caps the info tail, and drops workspace_dependent findings when remote (A1). render.ts is the shared =-bar renderer used by the advisor AND post-install-advisory.ts (generalized to a single current-state recommended-set.RECOMMENDED, install→scaffold). history.ts appends bounded ~/.gbrain/advisor-history.jsonl (no DB migration) for since-last-run deltas; local-only. apply.resolveApplyTarget is the allowlist+injection guard for commands/advisor.ts --apply <id> (structured argv via spawnCliChild, never a shell; local-only; asks through requireConsent with per-dispatch-id effects and cost text, so --apply <id> --yes is the non-interactive approval and a refusal exits 3 with the consent payload naming the command and its cost). The advisor op (operations.ts) is read-scoped, NOT localOnly, gated by mcp.publish_advisor (readPublishGate: on by default for the owner's stdio server, off for HTTP unless set; an explicit false or a failed read hides it) and strictly read-only on remote. The writeback-consent collector reaches MCP callers and stays until the user answers (memory.auto_writeback set to any value). CLI wired in cli.ts (CLI_ONLY + dispatch). Bundled skill skills/gbrain-advisor/ + weekly cron recipe. Tests: test/advisor-{core,apply,op-gate,ranking-eval}.test.ts.

  • src/core/audit/audit-writer.ts — shared JSONL audit primitive behind the audit modules. Exports createAuditWriter({kind, recordSchema}) returning {log, readRecent} plus shared helpers computeIsoWeekFilename(kind, now?) and resolveAuditDir() (honors GBRAIN_AUDIT_DIR). ISO-week file rotation; best-effort writes (stderr warn on failure, never throws); read-path scans current-week + previous-week files for boundary spans. Built on it: src/core/rerank-audit.ts, src/core/audit-slug-fallback.ts, src/core/minions/handlers/shell-audit.ts, src/core/minions/handlers/supervisor-audit.ts, src/core/facts/phantom-audit.ts. The graph-signals-failures audit (logGraphSignalsFailure) uses the same primitive. src/core/skillpack/audit.ts is the one audit that does not use it. Pinned by test/audit/audit-writer.test.ts.

  • src/core/audit/batch-retry-audit.ts — JSONL audit primitive for batch-retry events, built on audit-writer.ts. Schema: {ts, site, batch_size, attempt, outcome: 'success' | 'exhausted', delay_ms, error_message_summary, error_code?}. Privacy: NEVER logs slugs / page IDs / content (mirrors shell-audit.ts). logBatchRetry fires per successful retry recovery; logBatchExhausted fires when retries exhaust and rows are lost. readRecentBatchRetryEvents(hours=24) returns {events, corrupted_lines, files_scanned, files_unreadable} — corruption + permission errors surface to doctor, not silently swallowed. pruneOldBatchRetryAuditFiles(daysToKeep=30) deletes old files, called from gbrain dream --phase purge. File: ~/.gbrain/audit/batch-retry-YYYY-Www.jsonl (honors GBRAIN_AUDIT_DIR). summarizeError routes error messages through the shared redactConnectionInfo helper from src/core/audit/redact-connection-info.ts BEFORE truncation so DSNs / hostnames / credentials / IPv4 octets can't leak into operator-shared JSONL dumps. Pinned by test/audit/batch-retry-audit.test.ts (12 cases) + test/audit/batch-retry-redaction.test.ts (3 privacy cases).

  • src/core/audit/content-sanity-audit.ts — ISO-week JSONL audit at ~/.gbrain/audit/content-sanity-YYYY-Www.jsonl built on the audit-writer.ts primitive. Records every hard-block, soft-block, warn-trip, and quarantine/flag event with {kind, source_id, slug, bytes, matched_pattern_name?, reason, ts}. Doctor reads the last 7 days, aggregates by (matched_pattern_name, source_id) so operators see which scraper is the problem. Honors GBRAIN_AUDIT_DIR for shared-filesystem multi-host setups. Pinned by test/audit/content-sanity-audit.test.ts.

  • src/core/audit/db-disconnect-audit.ts — JSONL audit for every call to db.disconnect() and PostgresEngine.disconnect(). Built on audit-writer.ts. Schema: {ts, engine_kind: 'postgres'|'pglite'|'unknown', connection_style: 'module'|'instance'|'unknown', caller_stack, command, pid}. caller_stack captured via new Error().stack truncated to 20 frames so operators identify the offending caller without inflating JSONL. Privacy: stack frames carry file paths but NO SQL content / row data / user strings. File: `/.gbrain/audit/db-disconnect-YYYY-Www.jsonl(honorsGBRAIN_AUDIT_DIR). readRecentDbDisconnects(hours=24)walks current + previous ISO week and returns{count, most_recent_caller, files_scanned}. Wired into src/core/db.ts:disconnectandsrc/core/postgres-engine.ts:disconnect, logging BEFORE the early-return branches so even no-op disconnects on never-connected engines are recorded (that case may itself be a caller-side bug). Pinned by test/db-disconnect-audit.test.ts` (6 cases: round-trip, stack truncation, sort order, empty-dir nulls, stable feature name, EROFS best-effort).

  • src/core/audit/lock-renewal-audit.ts — JSONL audit primitive for per-job lock-renewal faults. Sibling of batch-retry-audit.ts, built on audit-writer.ts. Four outcomes: failure (single renewLock throw, counter incremented), success_after_failure (recovery; emits the recovery count), gave_up (time-based deadline exceeded; abort fired), executeJob_rejected (the second unhandledRejection vector — the stored executeJob(...).finally(...) promise itself rejected, e.g. failJob threw during the same DB outage). Schema: {ts, job_id, job_name, attempt?, outcome, error_message_summary?, error_code?} plus additive telemetry fields (cause?, lateness_ms?, overlap_skips?, load1?, cores?, via?, deadline_deferred?) threaded via an optional trailing ctx param on the sink (compactCtx copies only DEFINED fields so absent telemetry stays absent from the JSONL). Privacy: NEVER logs lock_token or job.data; error summaries route through redactConnectionInfo BEFORE truncation. Defense-in-depth: every audit call inside the lock-renewal tick's catch block is wrapped in its own inner try/catch so a misbehaving audit-writer can't surface as an unhandledRejection. readRecentLockRenewalEvents(hours=24) walks current + previous ISO week with corrupted-line tolerance. pruneOldLockRenewalAuditFiles(daysToKeep=30) is ready for future dream-cycle purge wiring. File: ~/.gbrain/audit/lock-renewal-YYYY-Www.jsonl. Pinned by test/audit/lock-renewal-audit.test.ts (incl. ctx round-trip + telemetry-less-line readback).

  • src/core/audit/pool-recovery-audit.ts — reap/reconnect audit on the shared audit-writer primitive. Events: reap_detected (CONNECTION_ENDED), reconnect_other (network/auth/health-check), reconnect_succeeded, reconnect_failed. readRecentPoolRecoveries(hours=1) returns {reaps, recoveries, failures, others, events}. Error summaries route through redactConnectionInfo before truncation (DSN/host/IP safe). Emitted ONLY from PostgresEngine.reconnect(ctx?) (the rare reap-retry path, near-zero hot-path cost); reconnect() classifies the threaded error via isConnectionEndedError (in retry-matcher.ts) so only true pooler reaps are labeled reap_detected. The retry callback in retry.ts threads the triggering error as (ctx?: {error?}) => Promise<void>. Pinned by test/audit/pool-recovery-audit.test.ts.

  • src/core/audit/redact-connection-info.ts — Shared pure helper. redactConnectionInfo(text: string): string strips Postgres connection info before any audit JSONL write: postgres:///postgresql:// URLs, host=foo, user=foo, password=foo, pwd=foo, IPv4 octets — each match becomes <REDACTED:kind>. Negative-lookbehind/lookahead [\w.@-] on the IPv4 pattern defeats version-string false positives (v3.1.4.0, [email protected]) while still matching real IPs in PG errors ((192.168.1.42)). Order-sensitive pattern set: URL forms first so substrings inside URLs don't get double-redacted. Idempotent, pure (no I/O), hot-path-safe (regex compiled at module load). Wired into lock-renewal-audit.ts, batch-retry-audit.ts, and cli.ts's doctor DB-fallback stderr note (layered with url-redact.ts:redactUrlsInText). Known limitations: bare-quoted hostnames (at "db.example.com") and usernames (for user "postgres.foo") are NOT caught — the highest-value leak (the IP in those shapes) IS caught. Pinned by test/audit/redact-connection-info.test.ts (15 cases: all 5 patterns + Supabase fixture + ENOTFOUND fixture + version-string false-positive defense).

  • src/core/brainstorm/checkpoint.ts — crash-resilient checkpoint for gbrain brainstorm and gbrain lsd. Persists FULL idea bodies (~50KB/run) so resume MERGES pre-crash ideas with post-resume ideas before the judge runs (a resume that produces only second-run output is silent partial output). run_id = sha256(question + profile + sort(close_slugs) + sort(far_slugs)).slice(0,16) — NO embedding bits, stable across embedding-model swaps. Atomic write via .tmp + rename. ONE resume flag (--resume <run_id> covers failed crosses, never-attempted crosses, AND a failed judge phase — the orchestrator keeps the checkpoint with judge_done=false whenever the judge throws, so --resume short-circuits every completed cross from disk and re-runs only the judge; there is no separate retry-judge flag); only a fully clean run (all crosses + judge) clears the file. The checkpoint also banks the CLI's saved-page slug (idea_slug, from RunBrainstormOpts.ideaSlug) and the orchestrator echoes it back as BrainstormResult.idea_slug on resume, so the re-scored page overwrites the one the failed run saved instead of forking a second page under a fresh nonce. --list-runs prints run_ids mtime-newest-first; --force-resume bypasses the 7-day staleness gate. Cycle purge phase (gbrain dream --phase purge) GCs checkpoints older than 7 days via gcStaleCheckpoints(7). Pinned by test/e2e/brainstorm-resume.test.ts (20 unit + 5 E2E cases incl. the merge contract and judge-failed retention/re-score).

  • src/core/brainstorm/{domain-bank,orchestrator,judges}.ts + src/commands/{brainstorm,lsd,eval-brainstorm}.ts + src/core/last-retrieved.ts — bisociation-grounded idea generation pair: gbrain brainstorm <question> (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and gbrain lsd <question> (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via pages.last_retrieved_at, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (SELECT DISTINCT substring(slug from '^[^/]+/[^/]+') cached 1h-TTL in config per source) tiebroken by JOIN page_links connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via 1 - clamp(cosine_distance, 0, 2) / 2. judges.ts exports runJudge(config, ideas) + two configs (BRAINSTORM_JUDGE_CONFIG weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs LSD_JUDGE_CONFIG cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when calibration_profiles.active_bias_tags is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in the search/query/get_page/fetch handlers and recall's query arm (never its facts-only reads) fires bumpLastRetrievedAt(engine, pageIds) (fire-and-forget, 5-min throttled via SQL clause, default-on with search.track_retrieval config escape hatch, read per engine); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped Set<Promise<unknown>>; awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}> resolves once all tracked promises settle, bounded by a 5s Promise.race timeout that stderr-warns the pending count. src/cli.ts awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE engine.disconnect(), then a fallback process.exit(0) fires ONLY when outcome === 'timeout' AND shouldForceExitAfterMain(argv) (excludes serve so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. The timestamp lives in page_retrievals(page_id, last_retrieved_at) (v224): the bump is an INSERT ... ON CONFLICT DO UPDATE throttled to 5 minutes, a HOT update with no triggers that never advances page_generation_clock_seq (the old UPDATE pages advanced it on every read, expiring the query-cache bookmark and the get_health and projection-readiness memos). No foreign key; the pages_forget_retrievals statement trigger removes rows of hard-deleted pages. Readers (domain-bank samplers, volunteer stats, entity card) take GREATEST(pages.last_retrieved_at, page_retrievals.last_retrieved_at); the legacy column stays (unindexed since v224) for older binaries; full forward-reference bootstrap probe on both engines. Frontmatter mode: lsd makes the dream-cycle synthesize phase skip LSD output via isLsdOutput() in src/core/cycle/transcript-discovery.ts short-circuiting isDreamOutput(). gbrain eval brainstorm <fixture.jsonl> is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). gbrain doctor has a brainstorm_health check (migration applied, search.track_retrieval setting, an unpriced cross or judge chat model, calibration cold-start status). Cost policy lives in src/core/brainstorm/cost-gate.ts: no cap flag is the $5 default tracker cap (an unpriced model warns and runs, priced calls stay metered; the estimate, mid-run guard and pre-judge check count it at Sonnet rates), --max-usd N (legacy --max-cost) is a user cap that refuses a run touching an unpriced chat, judge or embedding model before any work with the no-pricing.ts refusal, --max-usd off drops the USD ceiling; every price reads pricing.overrides; results carry cost.cap_usd/cost.cap_source. judges.ts computes the judge token budget via computeJudgeMaxTokens(ideaCount, modelId) (named constants TOKEN_BUDGET_PER_IDEA, TOKEN_BUDGET_ENVELOPE, LEGACY_MIN_MAX_TOKENS, MAX_OUTPUT_TOKENS_CEIL; ANTHROPIC_OUTPUT_CAPS map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no modelOverride the cap routes through the gateway's actual configured chat model via getChatModel(). --save for both commands persists through the canonical ingestion path: persistSavedIdea(engine, {slug, content, provenanceVia}) calls importFromContent({noEmbed:true, sourcePath, holdNoncanonicalPrivate:true}) (a held private request refuses put_page's visibility_not_applied and any other refusal is a failed save, #7897; chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared writePageThrough helper (file rendered FROM the row so the two sinks can't diverge and gbrain sync doesn't churn it). formatSaveOutcome(outcome, ctx) returns an honest per-branch message (both-sinks, DB-only when no sync.repo_path/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud save FAILED … NOT persisted on stderr + nonzero exit) — --save never prints "Saved" when the DB write failed. buildIdeaSlug(question, label, nonce?) adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. --json callers stay DB-only. buildBrainstormFrontmatterObject(result) in orchestrator.ts returns the object form for serializeMarkdown. Pinned by test/last-retrieved.test.ts, test/last-retrieved-writers.test.ts (the writer verbs and every call site), test/e2e/pglite-cli-exit.serial.test.ts (IRON-RULE: real bun src/cli.ts subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), test/fix-wave-structural.test.ts (asserts the drain await is textually BEFORE engine.disconnect), test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts. Open Collider source: github.com/CL-ML/open-collider.

  • src/core/budget/budget-tracker.ts — the keystone budget primitive. One typed error (BudgetExhausted with reason: 'cost' | 'runtime' | 'no_pricing'), one schema-stable audit JSONL at ~/.gbrain/audit/budget-YYYY-Www.jsonl. Contracts: record() throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion); reserve() admission checks cumulative + OUTSTANDING spend and returns an opaque BudgetReservation id for each held projection; record({ reservation }) settles that id and release(id) frees an attempt that never billed (no-op when already settled), so N concurrent callers can't all pass against the same cumulative and a small same-model call finishing first can't free a large in-flight call's hold (settlement is by id, never by model; an id-less record settles nothing; a priced reservation recorded under an unpriced served id charges its held ceiling). gateway.chat reserves, then runs the attempt in one try/finally that releases the id (provider-resolution failure, invocation-policy refusal); embed reserves after its preflights; rerank, decide, the TypeSafe rerank adapter and embed-facts pass their ids. Record audit rows carry requested_model and reservation; reserve() hard-fails with reason: 'no_pricing' when maxCostUsd is set AND the model is missing from pricing maps, its message and BudgetExhausted.pricing built by no-pricing.ts (warn-once preserved when cap is unset, naming the registration command); lookupPricing normalizes recipe aliases through resolveRecipe before consulting the pricing tables, so claude-cli:haiku prices exactly like claude-cli:claude-haiku-4-5-20251001 at reserve(), record() and isModelPriceable() alike (the gateway reserves with the pre-resolution string the user configured); extractUsageFromError(err, fallback) returns err.usage when the SDK provides it, else the pessimistic fallback (caller passes maxOutputTokens, not the optimistic pre-call estimate). onExhausted(cb) fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Pricing math lives in the pure src/core/budget/reservation-cost.ts: priceFor(model, kind, overrides) → { pricing, source: 'override' | 'provider_wildcard' | 'table' } | null is the one resolver (exact override, then a <provider>:* wildcard admitted only for recipes with billing: 'subscription' by isAllowedPricingOverrideKey, then aliases, the claude-cli → Anthropic sibling, canonical rows and free local providers); isModelPriceable, usageCostUsd/reservationCostUsd, skillopt preflight and synthesize_concepts all use it. parsePricingOverrides drops disallowed keys and adds each alias key's canonical twin. Every record() also feeds an in-memory model ledger keyed by (requested model, served model, touchpoint, purpose) — updated BEFORE the cap throw and before the unpriced early return — exposed as snapshot().models; BudgetActualUsage carries requestedModelId, purpose, failed, countsAsCall (default true) and estimated, and rows track calls (gateway operations), attempts (every record), failed_calls, tokens, cost_usd (null when unpriced) and cost_basis (estimated if any contributing record was). src/core/budget/models-used.ts maps labels to touchpoints (gateway.chat→chat, gateway.expand*→expansion, gateway.embed*→embedding, gateway.rerank*→rerank, gateway.ocr*→ocr, else other), builds receipt rows (buildModelsUsed), merges resumed segments and renders the "Models called" table. No DB change. Shared by brainstorm, cycle/budget-meter, and eval-contradictions; BudgetMeter keeps its public shape over it (schema_version: 1 stamped on every dream-budget audit line). Pinned by 18 unit cases. A4 cap sources: capSource: 'derived' | 'default' | 'user' (getter capSource; an explicit maxCostUsd with no declared source is user, the pre-A4 behaviour). Only a user cap hard-fails an unpriced model, and that BudgetExhausted carries fix (noPricingFix, actor agent on the CLI, host_admin when transport is remote); derived/default caps warn once and run the model unmetered. Cost exhaustion carries capSource, and a derived one appends derived_cap_exhausted to the agent-contract log (callers report it with consent.ts's derivedCapExhaustedError). Pinned by test/budget-tracker-cap-source.test.ts.

  • src/core/budget/daily-ledger.ts — #6188 the durable per-UTC-day USD ledger over budget_ledger/budget_reservations (migration v012; no new schema), one ledger per (scope, resolver), e.g. FENCE_REPAIR_LEDGER (llm_repair/fences). reserve(estimate, { capUsd }) is one conditional upsert (reserved + committed + estimate <= cap) plus a reservation row with expires_at; dispatch marks it sent; settle(id, actual) is idempotent, bound to the reservation's own day, and records an overrun; release returns an undispatched hold; expired holds are reclaimed before each reserve (a dispatched one is charged its estimate). Any ledger error means no call (ledger_unavailable); meteredCall runs the whole protocol. Amounts are NUMERIC(12,4), spend rounded up and caps down. estimateChatCallUsd prices a chat call through pricing.overrides and the canonical table; an unpriced model is metered at the highest canonical rate (estimated: true) under a default cap and refused with noPricingGuidance under a user cap. Never opens a BudgetTracker, so a caller's ambient tracker keeps metering. Read by doctor fence_integrity (today's spend). Tests: test/daily-ledger.test.ts (+ test/e2e/daily-ledger-postgres.test.ts, concurrent sessions).

  • src/core/consent.ts + src/core/consent-preapproval.ts — the A4 consent primitive for trusted local CLI handlers. requireConsent(req, env?) evaluates the matrix (paid: --yes, finite --max-usd/--max-cost, spend.posture=tokenmax via env.getConfig, or the per-run preapproval; other effects: --yes, --apply/--trust, the per-command NON_INTERACTIVE_AUTHORIZES map (only apply-migrations → persistent_install), or the install preapproval, which never covers credentials; destructive: --yes plus --expect <plan_hash> when the request has one, never preapprovable, a mismatch is preview_changed), then a TTY prompt through interaction.readLine (EOF/timeout = decline), else throws confirmation_required carrying confirmationPayload() (isConsentRefusal; renderConsentRefusal/printConsentRefusal: --json payload document or an [AGENT] block with a fenced [SHOW USER] relay, exit 3). Caps on the returned Authorization: explicit or preapproved or configured → user; estimate → derived (x1.5, floor $0.25); no estimate → default ($5, printed). Refusals, declines and preapproved runs are logged to E11 (code, effects, outcome; never params). Destructive rail: computePlanHash(selection) (stable sha256 over brain, source, operation, records + revisions + incarnations, parameters, effects), persistApproval/readApproval under <GBRAIN_HOME>/consent/approvals/, verifyApproval/assertApproval re-ask on a changed, newly matching or dropped record. Preapprovals (consent.preapprove.paid.max_usd_per_run, consent.preapprove.persistent_install) live only in the host file plane; setConsentPreapproval refuses remote callers. Pinned by test/consent.test.ts; observational startup by test/observational-startup.serial.test.ts.

  • src/core/embed-consent.ts — consent for explicit embedding backfills (gbrain embed, jobs submit embed|embed-backfill|embed-catch-up, features --auto-fix): requireEmbedBackfillConsent (paid, an SQL estimate from the text it would embed, a user cap below the estimate refuses with cost_cap_exceeded), null when nothing can spend (embedWouldSpend: embeddings off, no credentials, or a local recipe billing $0 per embeddingProviderIsFree). embedBackfillFix is the doctor-side backlog fix. Write-path embedding and the unattended library paths (autopilot, cycles, queued jobs) are deliberately outside it. Doctor's opt-in live probe asks through src/commands/doctor/probe-consent.ts (paid + egress). Pinned by test/agent-operator-gate-fixes.{,serial.}test.ts. Fix wave 13 P1.18: scope.images makes embedWouldSpend also count a paid multimodal model and, with GBRAIN_EMBEDDING_IMAGE_OCR=true, a paid OCR model, even behind a free text provider (test/embed-consent-multimodal.test.ts).

  • src/core/agent-markers.ts — the one [AGENT] … [/AGENT] / [SHOW USER] builder (agentBlock): fixed field vocabulary, every value through inertText() so interpolated text cannot open or close a block.

  • src/core/budget/cap-flag.ts — the one USD cost-cap flag parser (D19): parseCapFlag(flag, raw, { zero }) returns { flag, usd } (usd: null for off/unlimited/none) or throws CapFlagError; a bare 0 is refused unless the command declares it (zero: 'off' for skillopt's legacy --max-cost-usd 0, 'cap' for eval longmemeval's $0 judge cap); mergeCapFlag refuses two disagreeing cap flags; capNotice renders the one cap: $X (source; how to remove) line. Used by brainstorm/lsd, skillopt, enrich, onboard and eval longmemeval. Pinned by test/budget/cap-flag.test.ts.

  • src/core/budget/no-pricing.ts — the one no_pricing refusal builder. noPricingGuidance(model, kind) returns the structured fields (code, model, provider, kind, units, lookup, register_command, register_scope: 'local_cli', docs = the ERROR_CATALOGUE.no_pricing anchor); noPricingMessage renders the one-line cause plus noPricingSteps (look up the provider's price, run the gbrain pricing set command on the brain host, retry; remote MCP callers ask the operator); pricingSetCommand gives the command alone for warnings (chat: --input/--output, other kinds: --rate), rendered from pricingSetArgv; noPricingFix(guidance, transport) is the agent-contract fix for a user-cap refusal (the registration argv with inputs for each rate and the pricing page). Used by BudgetTracker, enrich, conversation facts (CLI and cycle phase), skillopt receipts, the embedding migration authorization and the extract-atoms, budget-meter, embed-backfill and conversation-budget warnings. Pinned by test/budget/no-pricing-registration.test.ts.

  • src/core/chronicle/{contract,eligibility,config,reasons,ledger,ledger-stats,execute,publish,event-identity,invite-projection,backfill,campaign,job,upgrade-notice,extract-events,proposal-diagnostics,ontology,narrative}.ts + src/core/cycle/chronicle.ts + src/eval/chronicle/harness.ts + src/commands/eval-chronicle.ts — Life Chronicle: the temporal spine. contract.ts holds the ledger columns, the extractor version (part of the ledger key: a bump makes extracted content new to discovery and backfill), the phase bounds and the phase result details. reasons.ts is the one reason table (CHRONICLE_REASONS: stage, meaning, stored fix Action with argv/preview_argv/consent/actor, never a stored next; auto_chronicle_off has no fix because it is off by choice) and builds the chronicle_backstop receipt field ({pending:'next_cycle', daily_remaining} or {skipped, stage, why, fix}, omitted for pages that are not chronicle-shaped); docs/guides/life-chronicle.md renders its table from it. eligibility.isChronicleEligible is the one eligibility function (meeting/conversation/calendar-event + directory rescue; diary and event pages NEVER eligible — privacy + anti-loop); with a time policy it also applies the invite end (not_yet_happened), the recency window on the page's own date (an authored effective_date, else frontmatter date/start; a fallback date counts as undated) and the settle window. config.ts owns the switch and knobs: autoChronicleSetting reads auto_chronicle as on when unset, on/off for true/false words, and invalid (off) for any other word; chronicleSettings reads chronicle.job_budget_usd/auto_daily_limit/auto_recent_days/auto_settle_seconds/judge_max_tokens/max_events_per_page (#5329, default 25) with ranges (CHRONICLE_NUMERIC_KEYS, enforced by config set; a malformed stored row falls back and is listed in invalid) and chronicle.activated_at (stamped by the first write decision or phase run; earlier revisions are history); autoChronicleNeedsAcknowledgement is true while on and chronicle.default_on_acknowledged is unset (stamped by config set auto_chronicle). The opt-out is gbrain config set auto_chronicle false everywhere. ledger.ts owns chronicle_page_state (one row per source/page/content hash/extractor version: state, reason, trigger, deciding writer and request, no_extract, attempts, next_attempt_at, cost/unpriced, event slugs and hashes): recordChronicleDecision runs inside every coordinated publication that imports a page (from queuePublicationEffects), applies the shared writer-confinement predicate, --no-extract, the opt-out and eligibility, and puts chronicle_backstop on the outcome through reasons.ts; reserveChronicleSlot is the rolling 24 h reservation under a transaction advisory lock. ledger-stats.ts rolls up the ledger and reservations for doctor and the advisor; readChronicleDuplicateEvents counts the live events from earlier extraction runs that a page's newest extraction did not produce (doctor chronicle_duplicate_events, #8972: info, read-only, counts pages and surplus_events; the preview-first repair is #9012). upgrade-notice.ts is the one-shot post-upgrade notice (chronicle.default_on_notice_shown). The global chronicle cycle phase (cycle/chronicle.ts) supersedes rows whose content is no longer live, discovers pages changed since activation with no row (unmanaged brains decide there; managed ones record no_write_decision), and executes settled rows round-robin across sources (≤50 judged items and a wall-time bound per run; automatic rows only while auto_chronicle is on, backfill rows always); no chat provider → no calls; the extract_atoms cost gate refuses an explicit cap on an unpriced model. execute.ts runs one row: live-content check, time rules, the reservation (automatic rows; attempts count), the judge in a chronicle:<trigger> BudgetTracker scope with the per-page cap and an explicit post-call cap check, failure classes (chat_error/budget → failed with backoff, no provider/no_pricing → failed until config changes, refused → skipped, only a parsed [] is no_events). Before the judge runs (#8972), execute.ts hashes the judged input (extract-events.chronicleJudgedInput: the window the judge reads, the page's visibility and incarnation, chronicle.tz and chronicle.max_events_per_page): a revision whose input matches the page's newest extracted generation is skipped/judge_input_unchanged with no call, so a long page whose tail grew past the window is not paid for again; a generation that dropped future-dated proposals records cutoffInputHash (the input plus the cutoff day) instead, so it is judged once more when the cutoff day moves. An automatic row whose input already spent the attempt limit in paid failures is failed/judge_input_failed with no call (a refused input stays judge_refused); only judge_truncated, judge_parse_failed, malformed_proposal, budget_exhausted and judge_refused count, summed over the page's rows with the same input in judge_input_failures, and backfill rows are exempt. A truncated generation (a body longer than the window) with at least one event and no dropped proposal replaces the previous one when its window re-read the text of every live event (windowCoversLiveEvents, which fails closed on rows recorded before the window was stored). Migration v297 chronicle_judge_window adds judge_input_hash, judge_window_chars, judge_window_hash and judge_input_failures (default 0, no backfill) to chronicle_page_state. invite-projection.ts stands in for the judge on an ended calendar invite (slug under calendar//cal/ or type calendar-event, with a title, start and end): one Scheduled: <title> event (kind: meeting, captured_via: life-chronicle:invite, not proof of attendance) with no chat call, no daily reservation and zero cost, through the same publication and ledger path. publish.ts publishes one generation against the pinned depth snapshot: each event write re-validates the depth revision, content hash, source incarnation, privacy and the deciding writer's grant (events also read as private when their event.depth origin is private by field or declared lineage, or missing, search/private-visibility.ts) (managed_maintenance_chronicle_event/_retire intents on managed brains, one transaction per event otherwise), stamps the depth's visibility, never overwrites an operator-edited event (live hash not in the ledger) or resurrects an operator-deleted one, and retires the previous generation's owned events it did not reproduce (retired_by: life-chronicle; restored when produced again) only when the caller passes replaces: execute.ts and runChronicleExtract pass it for a complete judged generation (extract-events.replacesPreviousGeneration: at least one event, no proposal dropped by the date screen or the per-page cap, and a body no longer than the JUDGE_BODY_CHARS the judge reads, except the execute.ts window rule below; a segmented page's truth lead is read one character past it to tell) and for an ineligible revision's empty generation, so an empty or partial generation publishes what it has, retires nothing and still records its ledger outcome (#7013). event-identity.ts assigns final slugs before publication: events sharing a base slug life/events/<day>-<hash8> (who, what, depth) are told apart by instant, where and kind; one keeps the base slug and the rest get -<hash6> (identical copies -2…); assignment reads the pages already at the candidate slugs, so it is order-independent, keeps a page's existing slug across corrections and re-extractions, and a page extracted before disambiguation keeps its base slug. extract-events.runChronicleExtract is the direct-caller body over the same snapshot and publication path: deterministic when/who, injectable judge (default = chat gateway; output cap 4000 tokens by default, operator override chronicle.judge_max_tokens), an ALL-or-nothing parse barrier (isValidProposal requires a real parseable date; a refused batch carries proposal-diagnostics.diagnoseMalformedProposals: a fixed code, the schema field and JSON type found, and counts, never a value, from at most 50 proposals, 50 who entries and 256 characters per string, on the direct result's malformed, the row outcome and the phase's warning line), then screenChronicleProposals drops, never writes, each proposal without a day (date_imprecise: when is a year or month) or dated after the page's own day (future_dated: chronicleEventCutoff is the latest of the page's own date, a meeting's end and a conversation's last parsed message, as a day in chronicle.tz, capped at today), then keeps at most chronicle.max_events_per_page of the survivors in the judge's order (#5329, the rest over_cap, no model call to count); drops are counted in events_dropped (phase details, row outcome, direct result) and a page whose every proposal was dropped records the drop reason on its extracted ledger row instead of no_events; content-addressed life/events/ pages + a timeline_entries projection. An unusable judge response is never recorded as no_events: truncation, unparseable output, a provider error and a refusal surface as judge_truncated/judge_parse_failed/judge_chat_error/judge_refused. backfill.ts (chronicle_backfill) queues trigger='backfill' ledger rows, exempt from the daily limit and recency (unless --recent), across types and rescue prefixes under one global --limit; --since is the update date, --dated-since the page date; a dry run reports estimated_usd and skip reasons; a live run needs --yes; content already extracted or queued is never re-queued, even after gbrain jobs prune. --max-usd (#6199, campaign.ts) is a queue-time bound: a page queues only while queued × (per-attempt cap + chronicleCallCeilingUsd) × max attempts fits, rows are stamped (campaign_id, attempt_cap_usd, max_attempts, pricing_policy, campaign_max_usd; migration chronicle_campaign_stamps) and queueCampaignRow never takes a row another backfill queued or an executor holds; an unpriced model under the cap refuses with no_pricing. Executors obey the stamp: claimChronicleRow consumes a campaign row's attempt with the lease and refuses it past max_attempts, the tracker uses the stamped cap, finish sums spend per attempt (cost_attempts, so a replay adds nothing) and applies the outcome only while its attempt is the latest (a stale completion is fenced), and failExhaustedCampaignRows fails a crashed row whose attempts are spent as campaign_exhausted; any other ledger write clears the stamp. job.ts drains chronicle_extract jobs from older releases through the same executor. ontology.ts carries the deterministic pieces of the bi-temporal per-entity ontology that RIDES THE facts TABLE (migration v122 adds dimension/value/value_hash/dim_status): valueHash (normalized, timestamp-free; the v188 dedup index adds the stint's valid_from, so a crash-retry conflicts while a same-provenance return to an earlier value records a new interval), normalizeDimension (seed alias lexicon), isNovelDimension (novel → quarantined, excluded from resolution/context until confirmed). The engine methods (mergeOntologyFact — corroborate on same value (noop when the same provenance already observed it during the current stint), forward-supersede via valid_until+superseded_by on a new value, backdated conflicts kept + flagged; getOntology with --asof valid-time travel; discoverOntologyDimensions; findOntologyConflicts — currently-open rows only) live in BOTH engines; both engines are on the R8 valid_until write allow-list (engine-layer, dimension IS NOT NULL rows only). Chronicle reads (getTimelineForDate/getSince/getLastSeen/getOnThisDay) JOIN the depth page (deleted_at IS NULL), hide soft-deleted event projections at READ time, and order by event effective_date for intra-day sequence. Ops: chronicle_day/chronicle_since/chronicle_last_seen/chronicle_on_this_day/ontology_*/volunteer_chronicle (agent orientation via src/core/context/chronicle-context.ts)/chronicle_backfill (admin, localOnly). Diary privacy: fail-closed for ctx.remote !== false callers — diary-sourced ontology + conflict values are redacted, the four timeline read ops (chronicle_day/chronicle_since/chronicle_on_this_day + volunteer_chronicle's recent_timeline) drop rows whose depth page, event page, or source provenance lives under life/diary/, and chronicle_last_seen answers the never-seen shape (last_date/last_event_slug/days_ago all null) when the latest sighting is diary-sourced or a diary page is queried as the entity. Remote callers other than the brain owner read only visibility = 'world' observations, like recall (worldOnlyFacts, GST-2951): getOntology/findOntologyConflicts take a visibility filter applied before resolution and the conflict HAVING, plus excludeDiary (set for every remote caller), which drops life/diary/ observations before each dimension resolves so a newer diary note cannot hide the older readable value or make a false conflict. ontology_propose commits through coordinatedDatabaseWrite on a managed brain. Page visibility rides the same policy: ontology_get/ontology_conflicts/volunteer_chronicle's ontologies resolve through readPolicyOpts, and both engines' getOntology/findOntologyConflicts apply privateProvenanceFilterFragment (an observation whose provenance page, looked up in the fact's own source, is visibility: private is dropped BEFORE per-dimension resolution and before the conflict HAVING, so an untrusted caller resolves the newest value they may see rather than a private value or a hole). The chronicle reads' event-page LEFT JOIN carries the caller's source scope in BOTH engines, so a scoped read can't surface another source's event pages through the join. Search: applyChronicleTypeBoost in search/hybrid.ts (bounded [1.0,1.25], fires only inside the recency !== 'off' post-fusion branch → non-temporal search bit-for-bit unchanged). Advisor collector collect-chronicle.ts (conflicts; coverage gap with a scoped chronicle-backfill --since <date> --limit 50 --dry-run; auto_chronicle_default_on ask_user until acknowledged; auto_chronicle_invalid; chronicle_chat_unavailable; chronicle_extraction_failing; chronicle_daily_limit); doctor chronicle_projection_health and auto_chronicle (checks/auto-chronicle.ts; BRAIN category). Eval: gbrain eval chronicle — deterministic, own in-memory PGLite, 6 gold tasks (day order, last-seen, supersession, asof, conflict, isolation), exit 0 iff 6/6. Tests: test/chronicle-*.test.ts (automatic path: chronicle-auto-{decision,phase,unmanaged}, chronicle-date-quality, chronicle-event-privacy, chronicle-event-identity, chronicle-invite-projection, chronicle-backfill, chronicle-backfill-max-usd, chronicle-reasons-5876, chronicle-max-events-5329, chronicle-long-conversation-8972; Postgres arms in test/e2e/chronicle-*-postgres.test.ts), test/eval-chronicle.test.ts, test/auto-chronicle-surfaces-5876.test.ts.

  • src/core/chunkers/ — markdown chunking (recursive.ts) and code chunking. code.ts is a tree-sitter-based semantic chunker for 30 languages (plus SQL via DerekStride/tree-sitter-sql) with embedded-asset WASMs (src/assets/wasm/), @dqbd/tiktoken cl100k_base tokenizer, small-sibling merging. CHUNKER_VERSION is folded into importCodeFile's content_hash so chunker shape changes force clean re-chunks across releases; GRAMMAR_REVISIONS (per-language chunker revisions: lua: 1 after the tree-sitter-grammars v0.3.0 swap that ended the gbrain serve Lua-fence hang; typescript/tsx/javascript/python/ruby: 2 for the #5001 __module__ chunks; elixir: 1 for #7080) folds into that language's hash only, and chunkerStamp() (8;lua=1;typescript=2;…) is the sources.chunker_version gate value every sync path reads and writes, so a per-language change walks each source once and re-imports only that language's files while the cost gate (grammarOnlyDrift, grammar_drift) prices those files alone. Top-level statements that contain a call (describe(...), app.use(...), print(f()), Ruby RSpec.describe ... do) are chunked through code-module-chunks.ts (below) so their call edges have a chunk to hang on. Grammar provenance and the vendoring scripts: src/assets/wasm/README.md; pinned by test/chunkers/code-lua.test.ts. extractSymbolName has an inline SQL branch (extractSqlSymbolName) diving through DerekStride's statement wrapper into the inner DDL child (create_table/create_function/create_view/create_index/create_procedure/create_type/create_schema/create_database/create_trigger/alter_table/alter_view) and extracting the target identifier via the name field with identifier-shaped fallback; DML kinds (select/insert/update/delete/merge/with) deliberately return null so chunks emit unnamed (code-def is a DDL signal). normalizeSymbolType has parallel SQL branches mapping create_table → 'table', create_view → 'view', etc. DEF_TYPES (owned by src/core/chunkers/def-types.ts, re-exported by src/commands/code-def.ts — see that entry) carries the SQL kinds ('table' | 'view' | 'index' | 'procedure' | 'schema' | 'database' | 'trigger') so the new chunks surface in gbrain code-def <name> queries.

  • src/core/chunkers/code-module-chunks.ts — #5001 module-level statement chunks. moduleStatementSymbol(node, language, named) names a top-level node that contains a call and is not a named definition (JS/TS/TSX, Python, Ruby: a statement, a top-level if/for/try, Python's if __name__ == "__main__":, an anonymous export default …) as { symbolName: '__module__', symbolType: 'module' }; code.ts adds those nodes to its semantic set and emits them through the ordinary chunk path, with moduleChunkQualifiedName giving the edge identity <file-basename>::__module__ (vat-review.test::__module__; a Ruby module Foo keeps its own name). isModuleChunk lets mergeSmallSiblings fold adjacent small statements (they are not definitions), and mergedRunSymbol keeps the __module__ identity on a merged run that holds one or a holdsCall declaration (const app = createApp();), so code-callers helper lists the test body or route registration that calls helper instead of dropping the edge. A statement without a call (x = 1, a docstring) and every definition-only file are unchanged. The edge rows themselves come from code-chunks.ts's mapCodeEdges, the one call-site → code_edges_symbol mapping shared by importCodeFile and the prepared apply. Pinned by test/chunkers/code-module-level-calls.test.ts (chunk inventory per language, merge identity, oversized split, PGLite import through mapCodeEdges, the per-language revision stamp) and test/chunkers/code-module-anonymous-5001.test.ts.

  • src/core/chunkers/code-elixir.ts — #7080 Elixir. tree-sitter-elixir parses every construct as a call, so elixirDefinition(node) reads the macro: defmodule/defprotocol/defimpl name a module from the first argument (Foo.Bar; an impl is Protocol.Target), def/defp/defmacro(p)/defguard(p)/defdelegate name a function from the head (left of a when guard). code.ts uses it for top-level names and types and as Elixir's NESTED_EMIT_CONFIG.definition hook (do_block body), so functions get chunks qualified Foo.Bar.fun; the module chunk keeps the text outside its definitions. Other top-level calls are __module__ code. extractElixirCallEdges emits bare-token calls, skipping definition macros, heads, directives, special forms, @ attributes and field reads (s.side). Pinned by test/chunkers/code-elixir-7080.test.ts.

  • src/core/chunkers/def-types.ts — DEF_TYPES: the ONE list of definition-shaped normalized (post-normalizeSymbolType) symbol types, shared by gbrain code-def's lookup allowlist (src/commands/code-def.ts re-exports it) and the code chunker's small-sibling merge guard so the two can't drift — a symbol type code-def can resolve never has its symbol_name erased by merging a small definition into a neighbor. Includes the fallthrough forms normalizeSymbolType emits verbatim for tree-sitter node types it doesn't canonicalize ('method declaration', 'struct specifier', 'record declaration', …) — most of an OO codebase's symbols arrive in those forms. Pinned by test/chunkers/code-merge-defs.test.ts.

  • src/core/chunkers/bullet-chunks.ts — chunkTimelineBullets(text, {maxTokens}): the segmented chunker's passages for a timeline segment with bullet lines (prepareSegmentChunks, SEGMENTED_CHUNKER_VERSION 5). A passage breaks only at a bullet start chosen by a hash of that bullet and its word count, so an entry added at the top changes at most two passages; a run over the size bound re-splits and merges back from its end; a single over-limit bullet falls back to chunkProjectedText with boundaryCuts. No overlap. Returns null for text with no bullet line (the caller chunks it as before). bulletChunkWork counts units and runs for the linearity test.

  • src/core/chunkers/recursive.ts — base chunker: 300-word chunks, 50-word sentence-aware overlap, 5-level delimiter hierarchy. Applies sanitizeRemoteBody and then credentialSafeProjection (src/core/credential-projection.ts) before splitting, so only world Facts and ordinary prose enter retrieval text and private keys enter only as <REDACTED:private_key_pem>; all Takes and malformed protected sections are omitted. Non-overlapping portions reassemble to that sanitized input. The overlap extractor counts chars (not whitespace tokens) for CJK-dominant chunks — same unit as countWords — and aligns to 。!? or a whitespace-followed ASCII .!?; the L4 char-slice fallback advances through safeSplitIndex so astral pairs (emoji, non-BMP CJK) are never halved. Both are gated on isCJKDominant, so English output is byte-identical (pinned by test/chunkers/recursive-cjk-overlap.test.ts). The hard cap skips the token count for a chunk whose UTF-8 byte length already fits the token budget (fitsEmbedTokens in token-estimate.ts: bytes bound both the cl100k count and the weighted CJK estimate), so typical CJK chunks never reach the tiktoken WASM; boundaries are unchanged (test/chunkers/fits-embed-tokens.test.ts). The markdown chunker version records the current index format. With spans: true each chunk carries byte_start/byte_end, its exact UTF-8 span in the sanitized, credential-projected text; markdown prose chunks store them as content_chunks.projected_byte_start/end.

  • src/core/chunkers/symbol-resolver.ts — resolves and stamps each bounded chunk batch in one transaction. It walks only chunks that own a code_edges_symbol row of their source (OWNS_EDGE, checked again under the guards) and returns at once for a source with no such row, so a markdown-only source gets no edges_backfilled_at stamps (each is a non-HOT update). It takes ordered source/page guards before edge reads, revalidates current chunk/page/slug identity under those guards, and commits edge metadata with its backfill watermark. Concurrent projection replacement cannot certify new edges using an old resolver read; rollback leaves both metadata and watermark unchanged. Pinned by test/symbol-resolver-projection-race.test.ts with PGLite serialization and a real PostgreSQL lock-wait receipt. A calls edge the extractor flagged memberCall (an untyped receiver, obj.m() emitted as bare m; persisted as edge_metadata.member_call) is never resolved within the file, since a member call cannot invoke a same-file top-level function (gbrain-evals N13-8, test/code-intel/n13-edge-resolution.test.ts); existing code pages pick this up when they re-import (gbrain reindex-code).

  • src/core/code-intel/recursive-walk.ts (code_blast / code_flow) + the chunkId arm of getCallersOf / getCalleesOf in src/core/engine-sql/code-edges.ts (#5439 item 3, wave 15 P7.8) — the BFS keys its frontier and visited set by source-scoped chunk identity where the hop that reached a node was resolved (a code_edges_chunk row's to_chunk_id, or readEdgeResolution(edge_metadata).kind === 'resolved'), and by qualified name only for unresolved edges; the start is always a name. The next hop from a chunk node reads edges by that chunk (opts.chunkId: callees = both tables' from_chunk_id; callers = code_edges_chunk.to_chunk_id plus code_edges_symbol rows whose edge_metadata.resolved_chunk_id matches), under the same source filter and current-projection read policy, so two files whose top-level helper share the qualified name no longer fan into each other's callers or callees two hops out. WalkNode.chunk_id is directional: the caller's chunk on a callers hop, the resolved callee's chunk on a callees hop, absent for an unresolved callee (it used to be the edge's from_chunk_id on every hop). SQL lives once in the code-edges domain (SQL-text golden sql-text/code-edges.json chunkId variants). Pinned on both backends by test/code-intel/recursive-walk-resolved-ids-5439.test.ts (test/e2e/code-intel-walk-resolved-ids-postgres.test.ts).