Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
20.7 KB
Raw Blame

Hindsight calibration (key files cluster)

Subsystem index.

Calibration teaches gbrain how the user tends to be wrong and uses that knowledge at every advice surface: a six-migration schema (v67-v72), three cycle phases, eight expansions, one admin tab. Convention skill at skills/conventions/calibration.md has the agent- facing rules.

Migration v80: the takes_resolution_consistency CHECK accepts quality='unresolvable' AND outcome=NULL as the 4th valid resolution state. The column-level CHECK on resolved_quality (takes_resolved_quality_values) enumerates all 4 states. Take.resolved_quality, TakeResolution.quality, and takes-fence.ts:TakeQuality are 4-state. TakesScorecard carries unresolvable_count

  • unresolvable_rate; resolved stays 3-state (correct+incorrect+partial) so scorecards stay comparable over time. finalizeScorecard: unresolvable_rate = unresolvable_count / (resolved + unresolvable_count), NULL when both 0. Spec doc at docs/architecture/calibration-quality-gate-spec.md. Pinned by R1-R5 in test/takes-resolution.test.ts and test/migrate.test.ts's v80 structural + PGLite round-trip suite (CHECK admits unresolvable+NULL, still rejects partial+true and unresolvable+true|false, pre-v80 NULL/NULL rows survive).
  • src/core/cycle/base-phase.ts — abstract BaseCyclePhase class. Enforces sourceScopeOpts(ctx) threading at the type level; rules out the source-isolation leak class structurally for every new phase. Inherits source-scope, budget meter, error envelope, progress reporter. propose_takes / grade_takes / calibration_profile all extend it. run() honors dry-run for every subclass at once: when opts.dryRun or ctx.dryRun === true, it returns status: 'skipped' / reason: 'no_dry_run_support' before process() (no LLM calls, no take_proposals / take_grade_cache / calibration_profiles writes) — the same skip shape extract and resolve_symbol_edges use; runCycle threads dryRun into all three call sites. The ONE home of CYCLE_DEADLINE_RESERVE_MS (60s carved out of the enclosing job's remaining wall-clock — wait-poll + worker force-evict grace + cleanup headroom; patterns.ts re-exports it) and of BasePhaseOpts.deadlineAtMs (the enclosing minion job's absolute deadline, threaded via runCycle; null/unset for direct gbrain dream callers — phases then fall back to their derived defaults).
  • src/core/cycle/propose-takes.ts — LLM scans markdown prose, proposes gradeable claims to the take_proposals queue. Candidate discovery excludes extract_receipt pages before the LLM call so the phase never re-ingests its own operational receipts; nullable legacy page types remain eligible. cycle.propose_takes.types (#5213; proposeTakesTypes, a JSON array of page types in the cycle.enrich_thin.types shape) narrows the candidate query to those types with type = ANY(...) on top of the source scope; unset keeps every type eligible and a malformed value is ignored with one stderr line. Idempotency cache on (source_id, page_slug, content_hash, prompt_version) composite unique index. extractorOutcome (#6782) is the one reading of a reply, used for both the takes and the tombstone decision: it locates the first JSON value (after <think> and whole-reply fence stripping, the whole remainder or up to its own closing bracket) and returns valid_empty only for a complete empty array not followed by a JSON object or an array of take-shaped objects (a [1] citation does not count), valid_nonempty for at least one take, else invalid, which defaultExtractor throws as transient (no tombstone). A zero-claim tombstone is keyed by the page's content hash, so any edit to the page extracts it again on the next cycle. Fence-dedup: existing canonical takes passed to the extractor as context. A claim whose exact text an operator rejected on the same page (status = 'rejected' with acted_at) is dropped before the insert on any later extraction (rejected_claims_skipped); a page left with no claim gets the empty tombstone, so its hash is cached (test/propose-takes-rejected-memory-7299.test.ts). Opt-in dream.propose_takes.attribution_rules=true adds EXTRACT_TAKES_ATTRIBUTION_RULES (withdrawn claims, assistant-authored judgments not attributed to the user) under prompt version suffix +attribution5425; off by default because it measured lower cat15 recall. Ships a stub prompt; tuned prompt arrives via the synthetic corpus build. Per-page extractor failures log a warning and continue, but a whole-run condition (per classifyGlobalLlmError in src/core/ai/errors.ts) breaks the page loop with a single combined warning line, sets aborted_global_error, and records a halt in the rollup — auth/billing on the first hit, bare rate_limit only after RATE_LIMIT_HALT_STREAK (3) consecutive hits (a successful call resets the streak). Status: fail when the halt happened with ZERO successful extractor calls (the whole LLM lane is down), otherwise any warnings fold into warn + (N warning(s)) summary suffix, so swallowed failures can't read as a clean ok. llm_calls_succeeded/llm_calls_failed/halted land in details. The phase wall-clock deadline is DERIVED, never a literal (a literal equal to the autopilot-cycle handler anchor, with non-co-started clocks, would make the clean deadline_hit partial-completion path structurally unreachable): explicit opts.deadlineMs wins (test seam), else resolveProposeTakesDeadlineMs(deadlineAtMs, now) = PHASE_DEADLINE_FRACTION_OF_JOB (0.8, headroom for grade_takes + calibration_profile which run after it with no deadline of their own) × (remaining job budget − CYCLE_DEADLINE_RESERVE_MS), clamped to the fallback (0.8 × the autopilot-cycle entry in HANDLER_DEFAULT_TIMEOUT_MS — a missing anchor throws at module LOAD, failing the whole cycle visibly). A fractioned value under MIN_PROPOSE_TAKES_BUDGET_MS (2 min) resolves to null and the phase returns an honest skipped with reason: 'insufficient_cycle_budget' (after the cheap provider probe, before any rollup/DB write — records neither a halt nor a completed round; next cycle retries with a fresh budget). Pinned by test/propose-takes.test.ts + test/propose-takes-per-claim.test.ts + test/cycle-phase-deadline-drift.test.ts. The kind vocabulary is TAKE_KIND_VALUES imported from src/core/takes-fence.ts (the fence enum — no hand-copied set); an unknown kind maps through the legacy-kind table, else take. dream.propose_takes.call_timeout_ms (whole ms 1000..300000; config set refuses others, a stored out-of-range value is clamped and a non-number ignored, each with a named-key phase warning) replaces the output-cap scaling for every extractor call, held to the phase time left (never below 90 s); a call its own bound stops fails per page with an error naming the key. Pinned by test/propose-takes-config-max-tokens.test.ts + test/propose-takes-truncation.test.ts. #5212: listCandidatePages also skips the dream cycle's own output by its writer stamps (dream_generated = true, extracted_by LIKE 'extract_atoms%', synthesized_by LIKE 'synthesize_concepts%'), pinned on PGLite (test/propose-takes-per-claim.test.ts) and Postgres (test/e2e/propose-takes-jsonb-postgres.test.ts). The mechanical holder check of #5425 [UC4] (cycle/propose-takes-attribution.ts, one call after the extractor returns, behind dream.attribution_checks, default off) is separate from the prompt rule and documented with that module. GBRA-130 A-25: extractor failures go through ProposeTakesFailures (cycle/propose-takes-failures.ts, op_checkpoints op propose-takes-failures per source): a page that failed before at the same content hash and prompt version is retried after the fresh pages, only fresh failures feed the #3763 halt and the propose_takes.pages ItemRun breaker, and a run-scoped error rethrows. A content-class failure (request_too_large, content_rejected) repeated in PROPOSE_TAKES_HOLD_RUNS (3) counted runs, each with extractor success evidence, writes a failure tombstone: a rejected take_proposals row whose claim is (extraction failed: <code>), distinct from an empty extraction; transient failures never hold, and a systemic run or global halt counts nothing. The result adds failed_pages, failure_tombstones_written and systemic; the summary names gbrain takes propose --failed. A content or prompt-version change, or gbrain takes propose --retry-failed, retries a held page. Pinned by test/pathological/a25-propose-takes-poison-pages.test.ts (both engines). The extractor receives the page's publicBody (facts rows other than world and the takes fence removed); the content hash, the fence skip and the dedup rows still read the stored body (test: test/cycle-world-fence-rows.test.ts).
  • src/core/cycle/grade-takes.ts — walks unresolved takes older than 6 months, retrieves evidence, asks judge model, caches verdict. Auto-resolve DISABLED by default; cycle.grade_takes.auto_resolve.enabled turns it on (test/grade-takes-auto-resolve-config.test.ts). Conservative thresholds: >=0.95 single OR >=0.85 ensemble 3/3 unanimous. aggregateEnsemble reuses the cross-modal substrate; fires on the borderline 0.6-0.95 band. Writes to take_grade_cache. Same global-error posture as propose-takes: a judge failure that classifies as a whole-run condition breaks the take loop with aborted_global_error (auth/billing first hit; rate_limit after 3 consecutive takes; fail status when zero judge calls succeeded, else warn + warning count; judge_calls_succeeded/judge_calls_failed/halted in details). Rejected ensemble judges are classified the same way — Promise.allSettled never flattens a revoked key or exhausted spend limit into a silent null verdict. Per-take auto-apply failures stay per-take.
  • src/core/cycle/grade-takes-measured.ts — GRADE_TAKES_MEASURED_MODELS (the judges that met the W6 take-grading eval's preregistered rule, evals/take-grading/PREREGISTRATION.md; openai:gpt-6.1-sol since the 2026-10-16 counted run) and gradeJudgeMeasured(), which gbrain takes grades (cmdGrades in src/commands/takes.ts, trusted local, no MCP op) uses to mark cached verdicts no measurement backs. Auto-resolve stays unwired.
  • src/core/cycle/calibration-profile.ts — aggregates resolved takes into 2-4 narrative pattern statements + active bias tags. Voice-gated via gateVoice(). Cold-brain skip when <5 resolved. Writes to calibration_profiles with audit columns (voice_gate_passed, voice_gate_attempts, grade_completion).
  • src/core/calibration/voice-gate.ts — single gateVoice() function, mode parameter (pattern_statement | nudge | forecast_blurb | dashboard_caption | morning_pulse). 2 regens then template fallback from src/core/calibration/templates.ts. Haiku judge with mode-specific rubrics; all rubrics structurally forbid clinical/preachy voice.
  • src/core/calibration/gstack-coupling.ts — outcome-driven learnings coupling. writeIncorrectResolution(opts) shells out to the gstack-learnings-log binary. Config gate cycle.grade_takes.write_gstack_learnings (default false for external users). Namespace prefix gbrain:calibration:v0.36.1.0: so --undo-wave can scrub.
  • src/core/calibration/svg-renderer.ts — server-rendered SVG for the admin SPA Calibration tab. Pure functions: data → SVG string. Inlines design tokens; XSS-safe via escapeXml(). Four renderers: renderBrierTrend, renderDomainBars, renderAbandonedThreadsCard, renderPatternStatementsCard. SPA renders via <TrustedSVG> wrapper behind requireAdmin.
  • src/core/calibration/undo-wave.ts — undoWave reverses calibration's mutations: unsets takes.resolved_* for wave-applied resolutions (cross-checks resolved_by so manual writes persist), deletes calibration_profiles, purges nudge logs, marks grade-cache rows applied=false. --dry-run shows counts without writing. Idempotent on wave_version match.
  • src/core/calibration/think-ab.ts — A/B harness. runAbTrial calls thinkRunner twice (baseline + with-calibration), records preference to think_ab_results. buildAbReport aggregates over a 30-day window; flags calibration_net_negative when n>=20 + win rate < 45% on decisive trials.
  • src/core/think/prompt.ts — anti-bias prompt shaping. withCalibration option on buildThinkSystemPrompt adds anti-bias rules. buildCalibrationBlock() emits the <calibration> XML. buildThinkUserMessage has TWO shapes: default (question first), and with-calibration (retrieval → calibration → question) when opt-in. Wired into runThink via opts.withCalibration + opts.calibrationHolder.
  • src/commands/calibration.ts — CLI: gbrain calibration (read + print), --regenerate, --undo-wave <ver>, ab-report. MCP op get_calibration_profile (scope: read) backs the same data path. Source-scoped via sourceScopeOpts(ctx).
  • src/core/owner-holder.ts — single source of truth for "the brain owner" holder string. DEFAULT_OWNER_HOLDER = 'self' (matches the consolidate facts→takes writer + docs/takes-vs-facts.md); resolveOwnerHolder({override, configValue}) returns override > emotional_weight.user_holder config > 'self'. Consumed by the calibration_profile cycle phase, gbrain calibration CLI, the get_calibration_profile op, think's calibration block, emotional-weight's DEFAULT_USER_HOLDER, and doctor's calibration_freshness. Pure; unit-tested in test/owner-holder.test.ts. Does NOT unify owner-identity fragmentation (self/brain/people-<owner>) — tracked separately.
  • src/commands/takes.ts — the full gbrain takes subcommand dispatcher (list/search/embed/add/update/supersede/resolve/propose/scorecard/calibration/extract/revisit). takes list --limit N --offset N validates integers at the CLI; the engine clamps limit (default 100, cap 500) and floors offset at 0. takes revisit <slug> opens $EDITOR on the source page with a <!-- gbrain:revisit --> cursor marker. #5167: takes remove <slug> --row N goes through takes-mutation.ts to the local-only takes_remove op (persistence takes-prepare.ts): the row leaves the takes table and its fence row becomes a reservation row (| N | ~~(removed)~~ | take | world | 0.0 | | removed |, W9F item 4) in one coordinated publication, other row numbers stay, and a resolved row, a row another row cites as "superseded by" it, or a row whose DB claim disagrees with the fence is refused (the last names takes rebuild); thin clients are refused. Reservations: parseTakesFence returns them in reservedRowNums, never in takes, so no reader or projection sees a take there; renderTakesFence(takes, reservedRowNums) puts them back in row order, and upsertTakeRow, supersedeRow and takesPreparation.replaceFence keep the reservations of the body they rewrite. Every new take row number comes from GBRA-54's nextFreeRowNum (fence-repair/normalize.ts): one more than every fence row on the page (facts and takes, live, struck and reserved) and, in takes-prepare.ts, every stored facts/takes row (storedFenceRows), revalidated under the publication lock; the promise holds while the reservation row exists. #6450: takes_add and an explicit takes_supersede holder override are trimmed and checked with isValidHolder at the write boundary (syntax before authority: invalid_params naming the grammar, before assertHolderAllowed); the inherited supersede holder is not re-validated (the existing fence already round-tripped). Every prepared takes fence is parsed again before publication and refused invalid_params on any warning, so no write can leave a fence the next write would refuse. Test: test/takes-holder-validation-6450.test.ts. test/takes-row-reservation.test.ts (both engines), test/takes-reservation-callers.test.ts (census of parseTakesFence callers). takes rebuild <slug> [--source-id] runs the local-only takes_rebuild op (W9F item 8; runTakesRebuild in takes-mutation.ts, delegated to a resident gbrain serve owner over persistence IPC, else the local engine, with the same JSON, warnings and exit 1 on a missing page or a warning), which calls extractTakes({source:"db", slugs:[slug], sourceId, rebuild:true}), whose rebuild now deletes only indexed rows whose (row_num, claim) left the fence, so matching rows keep resolutions and vectors. The v0_28_0 hint names gbrain reindex --markdown instead of the nonexistent extract takes --rebuild. GBRA-130 A-25: takes propose --failed [--source <id>] [--json] lists pages propose_takes held with a failure tombstone (listFailedProposals); takes propose --retry-failed [--slug <slug>] [--source <id>] [--json] deletes their tombstones and ledger records (retryFailedProposals) so the next run re-extracts them.
  • src/core/extract-takes-from-pages.ts — bootstrap extraction selects uncovered pages newest first. Managed pages capture their revision before classification and publish the complete appended takes fence through coordinated put_page, whose canonical projection commits the takes rows with the page; conflicts skip that page. Unmanaged pages take the same database path when the database is their copy of record (sync.write_through off, or the page in a declared storage.db_only directory; an unbound Postgres source skips that page owner_unavailable), and so does a file-backed page of a source with a canonical owner (bound before the run, or bound by the run's own first put_page, as PGLite does), so a mixed batch never reaches the fenced md-first writer; other unmanaged pages retain the md-first writer. When the page has no takes section but still has stored takes (a rewrite dropped the section), withStoredTakesFence (src/core/takes-write.ts, also used by think --take) renders them, resolutions included, into the section first, so new claims are deduped against them and number after them; a page it refuses (fence_unparsed for malformed takes markers in the file, invalid_input for a stored resolution without a quality or a stored cell that would not read back unchanged) is skipped with that reason and the run goes on. Pinned by test/recreated-takes-section-keeps-stored-takes.test.ts. A file-backed page whose file is missing is skipped as mirror_unavailable before the classifier runs. Pinned by test/persistence-managed-takes-extract.test.ts, test/extract-takes-md-first.test.ts and test/pathological/d19-takes-bootstrap-db-only.test.ts. takes extract --before <updated_at>,<id> (CLI-only) continues strictly below a prior run's next_before, an (updated_at, id) keyset with exact timestamptz text, so zero-claim pages no longer starve older ones; a budget stop resumes at the first unclassified page, a failed page stays uncovered for the next run without --before, and a malformed cursor is an exit-2 invalid_params refusal before any page is read. Pinned by test/extract-takes-from-pages-progression.test.ts, test/takes-extract-json.test.ts and test/e2e/takes-extract-cursor-postgres.test.ts.
  • src/core/take-proposals.ts — owns the take_proposals queue's row contract. normalizeTakeProposalRow coerces Postgres driver BigInt/string values (id, weight, promoted_row_num) to numbers at the query boundary so both engines return the same numeric row shape. acceptProposal promotes through a coordinated takes_add mutation (submitPageMutation, the same path gbrain takes add uses), so accept works on a managed brain where a direct fence write is refused with writer_coordinator_required (test/takes-propose-accept-managed.test.ts). #5214: an accepted proposal's take records take_proposals#<id> as its source; acceptProposal(target, id, edits) (takes propose --accept <id> --claim/--weight/--who/--kind) applies the edits as a plain takes_add with source take_proposals#<id> (edited) while the queue row keeps the original claim text. #7271: accept needs no brain directory (the mutation resolves its write root from the proposal's source; a repo-less brain writes through the database), takes propose --accept prints the accepted claim and page, and takes list --status refuses and points at takes propose --json (test/takes-propose-cli-7271.test.ts).
  • admin/src/pages/Calibration.tsx — Calibration tab. Single-column layout. <TrustedSVG> wrapper handles dangerouslySetInnerHTML for the server-rendered SVG.
  • admin/src/index.css — --text-muted: #777 (WCAG AA contrast bump to ~5.5 on the #0a0a0f bg).
  • test/fixtures/calibration/extract-takes-corpus/ — synthetic prompt-tuning corpus. Ships 5 representative pages; full 50-page + 10-page holdout generated by gbrain calibration build-corpus. All anonymized per CLAUDE.md placeholder list.
  • scripts/check-synthetic-corpus-privacy.sh — CI guard in bun run verify. Greps for explicit dollar amounts + verifies non-essay fixtures reference at least one placeholder name.
  • DESIGN.md — repo-root design system. Formalizes the de facto admin tokens. Calibration target for future /plan-design-review and /design-review.