Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
18.0 KB
Raw Blame

Open-loop engine (key files cluster)

Subsystem index · Google connector entries.

User-facing contract: docs/guides/open-loops.md (detection, exclusion and close semantics). The Gmail sweep that feeds both lanes is in google-and-loops.md (google-source.ts, loop-catchup.ts, loops-enqueue.ts).

  • src/core/google/loops-exclusion.ts — the user's own label exclusion for the open-loop engine (#5445). excludedLabelTokens reads the source's g_loops_exclude_labels, else the brain-wide loops.extraction_exclude_labels (no merge); resolveExcludedLabels matches tokens against the account's label catalog (id verbatim, name case-insensitively; with no catalog only id-shaped tokens resolve) into a LoopsExclusionPolicy { tokens, ids, unresolved }, where a non-empty unresolved means FAIL CLOSED for new paid extraction. resolveSweepExclusion (loops-enqueue.ts) runs it once per Gmail sweep (one GmailClient.getLabels call when anything is configured), logs each unresolved name, and storeLoopsExclusion keeps the resolution in op_checkpoints (op loops-exclusion, keyed like the loop outcomes) so loadLoopsExclusionPolicy can apply the policy without a Gmail client at the two late sites: runLoopsExtract (skips excluded_label, throws retryable excluded_label_unresolved) and openDueGraceHold ('excluded' drops the hold, 'deferred' keeps it). A stored resolution is reused only for the same token list; changed tokens fall back to id-only until the next sweep. The policy gates loopExtractionEligibility right after SPAM/TRASH and before the owner override (reasons excluded_label / excluded_label_unresolved in the sweep's counters), withLabelExclusion strips the opens from a detectThreadLoop verdict (closes untouched), recordGraceVerdict records no hold for an excluded thread, runLoopsCatchup skips with excluded_label_unresolved, and enqueueLoopsExtraction reports the same skipped_reason ahead of no_candidates. The leaf is excluded from connector identity. Tests: test/loops-exclusion.test.ts, test/google-source-reconcile.test.ts ("label exclusion (#5445)"), test/loops-extract-run.serial.test.ts ("#5445 execution-time").
  • src/core/google/loops-spend.ts — the daily spend cap on loop extraction (wave 14 P4.6b): loops.extraction_max_usd_per_day (default $2.00, validated at config set by validateLoopsSpendConfigValue; a stored value is user-sourced). loopsSpendGate is what both enqueue sites call: it reads the cap and the day's group spend (readGroupSpend on group:<day group>) and returns the SpendAuthorization to stamp (loopsExtractSpendAuthorization: kind authorized, command loops_extract, via preapproval, one deterministic UUID per UTC day from loopsSpendGroupId) or the reason to queue nothing (daily_spend_cap at the cap, spend_cap_zero at 0). The sweep's enqueueLoopsExtraction and the catch-up's enqueue pass that record through MinionQueue.add's trusted spendAuthorization option, so the worker runs each job under runWithJobSpend (spend-authorization.ts): every provider attempt reserves against the group and settles its usage; refusal before the call is cost_cap_exceeded, recorded on the row. An unreadable meter admits the enqueue (the worker still enforces). Tests: test/loops-extract-spend-cap.test.ts.
  • src/core/google/counterparty.ts — where a loop's counterparty page is (#5504, wave 14 P4.1). resolveCounterparty(engine, sourceId, { name, email }) tries the loop's OWN source first through resolveEntitySlugWithSource (name, then email; never the slugify fallback), and only then crosses a source boundary by identity: an entity_identities canonical member in another source (identity_canonical, which also redirects an own-source stub to its canonical twin) or the exact email as a page_aliases row on exactly one live page in exactly one other source (email_alias). A display name never crosses, a tie (two other sources) stays unresolved and the loop keeps a NULL counterparty rather than a guess. Both writers (loop-detect.ts upsertThreadLoop, loops-extract.ts) stamp the result into open_loops.counterparty_source_id and point the owes_to / awaiting_reply_from edge at the page's source; the commitment fact stays in the connector source. Readers confine with COALESCE(counterparty_source_id, source_id) (COUNTERPARTY_SOURCE_SQL): a row written before v236 reads as its own source. src/core/loops/loops-schema.ts is the leaf the migration and the store share (column DDL + the partial index open_loops_counterparty_source_idx).
  • src/core/google/loop-detect.ts — the zero-LLM thread-state machine: last substantive message inbound + user in To: + unanswered ≥24h → unanswered_inbound; last outbound + contains a question + unanswered ≥72h → unanswered_outbound (both grace windows measured from the oldest owed message in the trailing run since the last turn flip, so a nudge or follow-up never restarts the clock — gbrain-evals N7-1); a reply closes the loop (closed_by: reply_detected), except that my acknowledgement-only reply to their question (isAcknowledgementOnly: "Thanks!", "Got it, thanks.") is dropped from the substantive run so it neither closes nor restarts the loop (N7-2); asksQuestion ignores a ? inside a link (N7-7) and reads only the note above a forward separator (textAboveForward in google-render.ts), so the message a forward carries never counts as the sender's question; each spec carries openedMs, the oldest obligation-carrying message in the run, which the apply step writes as opened_at (N7-6). Precision IS the product: noise senders, list mail (List-Unsubscribe), auto-submitted mail (RFC 3834 Auto-Submitted other than no, which also never closes a loop, so an auto-reply is not an answer), CC-only delivery, FYI/forwards without a question, self-threads, and muted senders/threads never open loops; positively automated mail (isPositivelyAutomated) and, with the opt-in LoopDetectPolicy.ownerWroteTo, CATEGORY_UPDATES mail from someone the owner never wrote to return withheld: {reason, counterpartyEmail} instead of an open (#5012; closes are unaffected) — pinned by the labeled fixture corpus in test/google-loop-detect.test.ts (every false-positive class gets a fixture before its fix). Pure verdict function + a thin apply step called per touched thread from the sync; the apply step must never fail the sync. A grace-window hold returns held: { untilMs, spec } (the loop it would open) and applyThreadLoopVerdict returns the verdict, so the sweep can record it (loop-catchup.ts); openDueGraceHold opens a stored spec after re-reading suppressions. applyThreadLoopVerdict builds the policy through ownerWroteToFor and settles a mute re-check thread (settleMuteRecheck) whose verdict neither opened nor closed its queued loops; invalidateSuppressionCache drops the per-engine suppression cache after a mute or unmute.
  • src/core/google/loops-reciprocity.ts — the opt-in Updates reciprocity rule (#5012): key loops.updates_require_reciprocity (default false; validateLoopsReciprocityConfigValue at config set). ownerWroteToFor(engine, source, myAddresses) returns null when off; on, the addresses in participants of the source's email pages whose senders include an owner address, cached per engine and source for 10 minutes. Tests: test/loops-noise-5012.test.ts, test/loops-noise-5012-store.test.ts (+ Postgres twin).
  • src/core/google/loops-extract.ts — the LLM half of the open-loop engine: ONE extractor per recent thread page projects each commitment into THREE substrates in the same pass — the open_loops row (dedup commit:<sha8> of thread, direction and the normalized verbatim quote, the text only when no quote verified; same-quote items in one response collapse, and a row already recorded for the quote keeps its key, #7289), a facts row via writeSingleFact (kind=commitment, fence-first, deduped; its id lands on open_loops.fact_id so entity cards / recall / context_pack see the commitment through existing read paths with zero new read code; a failed facts write is logged with slug, source, the refusal code and a bounded error, never the commitment text, and the loop row still lands; on a strict source a counterparty with no page under an exact page rule, or whose access rules cannot be read, is such a refusal, git_home_required or access_policy_pending, whose message names the entity's page, GST-2635), and a typed edge thread-page → person-page (owes_to / awaiting_reply_from) for relational search. extract_facts never runs separately on google-source email pages. Guardrails: injection-hardened input, ALL-or-nothing parse barrier (a malformed model response writes NOTHING), a structural eligibility gate (loopExtractionEligibility — Gmail labels, List-Unsubscribe, Auto-Submitted, calendar part, who wrote the message; no vendor lists) plus the per-source waiting-depth enqueue budget (LOOPS_EXTRACT_ENQUEUE_CEILING, a spend backstop — see google-source.ts), only the last LOOPS_EXTRACT_WINDOW_DAYS (30) of mail (the deep backfill is never extracted), kill switch loops.extraction_enabled (default ON for google sources). Suppressions gate the lane before any model/facts/edge write: a muted thread id, or a muted SENDER — fm.from ∪ fm.senders, every address that AUTHORED a message, so a muted counterparty who wrote earlier in the thread still suppresses — returns reason: 'suppressed'; recipients/CC never count (muting one person must not hide everyone else's commitments in a group thread, and an outside sender must not be able to dodge extraction by CC'ing a muted address). Pages without a senders: field fall back to fm.from alone until their next re-render. Eligibility computes the substantive (non-noise, non-calendar) message set FIRST and owner_participated requires the owner to have written a SUBSTANTIVE message — a pure-calendar thread the owner RSVP'd to stays no_substantive_messages. When the chat provider is unavailable, runLoopsExtract throws LoopsExtractRetryableError (reason llm_unavailable) rather than returning a skipped result: a completed no-work row would hold the revision-keyed idempotency slot forever, while a retried-then-dead row frees it so the same revision re-enqueues on the next sweep. On a managed brain managedFactWritePreflight checks the managed fact authority and canonical owner before chat (an unaccepted writer refuses with writer_coordinator_required), and the commitment fact publishes through writeSingleFact's coordinator path; disabled, missing-page and suppressed cases retain their existing no-work outcomes. A judge response cut off at the 2048-token cap (stopReason: length) is retried once at 8192; still truncated, the job fails retryable (truncated) with the ceiling named. Non-length errors and refusals are never escalated.
  • src/core/loops/loops-store.ts — SQL accessors for the open_loops + loop_suppressions tables over engine.executeRaw with IDENTICAL SQL text on both engines (parity by construction, the sources-ops.ts pattern — no per-engine method twins). JSONB discipline: evidence binds through $N::text::jsonb, never a bare ::jsonb cast over JSON.stringify. Loops close by state transition, never delete: reply-driven auto-close flips status to done and stamps closed_by, keeping the audit trail. openedAt sets opened_at (default now()); an open row keeps the earlier of its stored and incoming value, a reopened row takes the incoming one. The upsert's DO UPDATE carries a manual-close guard in its WHERE: a closed row (done/dropped/stale) only reopens on GENUINELY newer activity, so a routine sweep re-seeing the same thread never resurrects a hand-closed loop, and the staleness auto-close (closed_by: 'staleness') is likewise guarded against the upsert's reopen. listOpenLoops takes an optional id that composes with the status/type/counterparty filters and never replaces the source scope; counterpartySourceId narrows a counterparty slug match to the source the page lives in (COALESCE(counterparty_source_id, source_id)), leaving an email match untouched (#5504). removeSuppression(engine, sourceId, kind, value) is the exact inverse of the mute write — deletes the one (source_id, kind, value) row with the same lower-casing and returns {removed: boolean} (false = was not muted, callers treat it as a no-op success). #5012: closeMutedLoops closes (or, without apply, lists) the open reply-owed loops a mute covers as dropped / closed_by: muted, and the upsert never updates a muted row for the same counterparty; requeueMutedLoops hands an unmuted loop no other suppression covers to closed_by: mute_recheck (no thread: unmuted, reported unverifiable), which the upsert reopens and closeThreadLoops closes on the re-check's verdict, and settleMuteRecheck marks unmuted when neither happens; muteRecheckThreads feeds loadSuppressions().recheckThreads and the sweep seed; listSenderInboundLoops backs loops drop sender.
  • src/core/ops/loops.ts — the op surface: open_loops (read), loops_close (write), loops_mute (write; also closes the covered reply-owed loops and returns closed, dry run would_close, plus undo_command), loops_unmute (returns requeued, unverifiable, stale_source + recheck_command, undo_command; dry run would_requeue). The grouped open_loops view adds withheld_noise (null when none): open reply-owed loops from senders in each source's loop_withheld_senders record, every sender ordered by open loop count, each with a preview_command (#7285; gbrain waiting prints the first 10 and counts the rest). loops_close retires a commitment loop's fact through persistence/loop-fact-retirement.ts and returns { closed, id, status, fact_expired, retryable, reason? } from the real publication outcome; a loop already closed whose fact is still active re-attempts the retirement. open_loops is deliberately NOT localOnly (hosted serves it over HTTP to the authenticated owner); instead it applies fail-closed evidence redaction for ctx.remote !== false — counts, counterparty, summary, due date only; verbatim quotes, Gmail deep links, and the injectable text digest are trusted-local only. Ranking and rendered ages use one reference time, as_of (ISO, default now, echoed in the grouped result; gbrain waiting --as-of), so a published order is reproducible (N7-5). An optional id (positive safe integer, else invalid_params through invalidParam without echoing the value) finds one loop at any status unless status is given; explicit filters still intersect, scope and redaction are unchanged, so an out-of-scope id returns exactly what a missing id returns, and an id lookup carries no text digest. The result carries per-source last-successful-sync ages + a stale flag (>24h): all stale sources remain a refusal condition for gbrain waiting, while a mixed fresh/stale set continues with a warning naming stale sources (stale_sources in JSON). Per-call scope params source_id/all_sources resolve through resolveRequestedScope (an MCP client bound to another source can reach the google source's loops; remote callers stay in-grant, out-of-grant source_id is denied); a scope with NO google source returns no_google_sources: true and the digest says the engine has nothing to read instead of a false "You are clean". Scope is fail-closed too: an UNSCOPED remote open_loops read is refused outright (the op must not rely on transports to enforce it — an unscoped remote read would span every source), and the write ops (loops_close, loops_mute, loops_unmute) require a single-source remote scope that matches the caller's grants — trusting a caller-supplied source_id for a remote write would let any remote client plant (or lift) suppression rows cross-source; loops_unmute reverses a suppression via removeSuppression and reports {removed} honestly. #5871: grouped output ranks people only; loops with neither a counterparty slug nor an email return in no_counterparty (loop_count, by_type, shown loops, loops_omitted; null when none), never ranked or counted by limit, and the digest lists them last and never says "You are clean" while they exist; each group shows at most LOOPS_PER_GROUP (5) loops, due-soonest then most recently active, with loops_omitted (test/ops-loops.test.ts, test/loops-cli.test.ts).
  • src/commands/loops.ts — gbrain waiting [--top N] [--json] [--stale-ok] + gbrain loops list|show|done|drop|mute: all paths dispatch through the trusted-local op layer (handleToolCall, remote:false) so CLI and MCP share one behavior. waiting REFUSES when EVERY google source has gone >24h without a successful sync and prints the exact fix (--stale-ok bypasses; per-source sync ages are always reported); when only some sources are stale, it continues with the warning and source ids from open_loops. Output per counterparty: what's owed, evidence quotes, Gmail deep links, entity-card context, a paste-ready digest. Reads default to the __all__ brain span (loops live in google sources, not default — a default-scoped read would say "all clean" while people wait); --source <id> narrows explicitly. loops show <id> passes the id to open_loops (any status and rank); a missing positive id is a usage error (exit 2) and an unknown id renders not_found with the gbrain loops list fix (exit 1). An unqualified mute/unmute resolves the brain's google source (never default) through the SAME resolution — an unmute can never aim at a different source than the mute it reverses — and refuses with the exact fix when none or multiple exist. unmute sender <email> | thread <id> is exact and queues the loops the mute closed for re-check; both take --dry-run and print the count and the undo command. A repeated unmute is a no-op success (removed: false, exit 0) so scripts can call it unconditionally. loops drop sender <email> [--source] [--note] [--dry-run] closes one sender's open reply-owed loops through loops_close (the dry run is the consent step; a drop cannot be undone), and waiting prints the withheld_noise lines.