Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
docs: one CHANGELOG entry for the ingest and facts train
garrytan65e0d280cbea
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 231ea28691..46aff0cc3f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,36 +12,31 @@ identifiers and attribution are available in the pre-removal Git revision## [Unreleased]-**Fact extraction and capture now record who said what, and where a page came from, more faithfully.** Lines you quote with `>` in a turn are no longer saved as your own facts, a Claude Code Remote Control session is yours rather than an agent's, facts that open with a log word ("Notes from", "Watched") link to the one entity they name, a facts batch refused for a stale embedding config lands once the write owner restarts with the fixed config, `transcripts ingest --facts --json` reports a budget stop instead of reading complete, a remote timeline entry, link or tag no longer overwrites how its page entered the brain, and a live fact whose context note happens to read "superseded by #N" is no longer hidden as superseded.+**Gmail threads you purged stay purged, forwarded emails keep their body, live facts stay in recall, and more fixes across connectors, transcripts, fact capture, takes, timelines and links.**-### Itemized changes+Gmail's change feed reports a thread whenever anything on it changes, a label or a read flag, however old the mail is. gbrain imported every one of them, so a mailbox set to 90 days of history could gain years of old threads. Those threads now stay out, the same way the first backfill keeps them out. A purged Gmail thread that the change feed delivers again no longer writes its subject and quoted text back into open loops, on managed and unmanaged sources alike. Forwarded emails used to be saved with only the note typed above the "Forwarded message" line, or empty, because that line was read as the start of a quoted reply. The forwarded message is now kept. Two unmanaged Google cases that ended as held items only a person could clear now sync cleanly. One is a page whose checkout file was missing only gbrain's Facts table after one of its facts was forgotten, and the other is an item whose page the owner purged while Google kept listing it (it is skipped and stays purged).-- `src/core/facts/writeback-gate.ts`: an accepted ambient-writeback turn is banked and hashed without its `>` quote lines outside backtick or tilde fences (a line like `>90% of traffic` is still yours); the quoted-material check recognizes tilde fences too. Reported and first fixed by @Masashi-Ono0611 (#8410, #8412).-- `src/core/persistence/facts-maintenance.ts`: a page-triggered or input-keyed facts batch refused `embedding_configuration` by the write owner's consumer is re-admitted from its retained facts (no new model call, retained visibility, unattributed facts included) once no full consumer that was running at the refusal is still live; until then a retry replays the stored refusal and spends nothing, a claim forgotten in the meantime stays forgotten, and a real model change still admits nothing. Reported and first fixed by @babyspectrum (#8526, #8554).-- `src/core/facts/subject-infer.ts`: capture-stream openers (captured, watched, reviewed, logged; notes, capture, log, review, watch as labels) no longer veto subject inference as a competing name. Reported and first fixed by @mgunnin (#8656).-- `src/core/transcripts/ingest-facts.ts`, `src/commands/transcripts.ts`: `transcripts ingest --facts` returns `budgetExhausted`, `budgetReason`, `pagesSkippedTooLarge` and `pagesLockSkipped` in `--json` and prints a line for each in the human summary. Reported and first fixed by @harjothkhara (#8722, #8757).-- `src/core/context/session-speaker.ts`: a headless session that a `claude remote-control` server (or its `rc`, `remote`, `sync`, `bridge` aliases) hosts over `--sdk-url` is the owner's; a worker that session launches is still agent-launched. Existing speaker sidecars keep their first record. Reported and first fixed by @exGeni (#8080, #8892).-- `src/core/persistence/page-mutations.ts`: only remote writes that supply page text (put_page, edit_page, append_page, capture) stamp `mcp:<operation>` provenance; other remote page writes keep the stored `source_kind`, `ingested_via` and `ingested_at`. Reported and first fixed by @zsimovanforgeops (#8667).-- `src/core/facts-fence.ts`: only a struck (`~~claim~~`) row takes a `superseded by #N` link from its context cell, the way only a struck row reads `forgotten:`; a live row keeps the text as context and stays in recall. A live row an older build linked is cleared by a full reconcile, which `gbrain dream --phase extract_facts --source <id>` runs; the nightly cycle clears it only when the page changes, and on a managed brain an earlier reconcile receipt for an unchanged page can replay until then. Reported and first fixed by @alanshurafa (#9069, #9073).-- Docs: `docs/guides/ambient-writeback.md` (Remote Control sessions under automatic detection), `docs/guides/fence-format.md` (the `context` column) and the KEY_FILES entries for the touched files.-- Tests: `test/facts/writeback-gate.test.ts`, `test/facts-embedding-refusal-heal-8526.test.ts` (PGLite and Postgres), `test/facts-subject-infer.test.ts`, `test/extract-conversation-facts.test.ts`, `test/agent-session-speaker-8064.serial.test.ts`, `test/put-page-provenance.test.ts`, `test/facts-fence.test.ts`, `test/extract-facts-phase.test.ts`, `test/fence-write.test.ts`.--**Five connector fixes: Gmail stays inside its history window, forwards keep their content, GitHub webhooks refresh the item they name, and two Google pages stop getting stuck as held.**+On a GitHub source with a webhook, a refresh waiting for one issue swallowed the event for any other issue of the same source, and deleting an issue right after editing it refetched the page instead of removing it. Each item event now gets its own refresh or delete.-Gmail's change feed reports a thread whenever anything on it changes, a label or a read flag, however old the mail is. gbrain imported every one of them, so a mailbox set to 90 days of history could gain years of old threads. Those threads now stay out, the same way the first backfill keeps them out. Forwarded emails used to be saved with only the note typed above the "Forwarded message" line, or empty, because that line was read as the start of a quoted reply. The forwarded message is now kept.+Fact extraction and capture now record who said what, and where a page came from, more faithfully. Lines you quote with `>` in a turn are no longer saved as your own facts, a Claude Code Remote Control session is yours rather than an agent's, facts that open with a log word ("Notes from", "Watched") link to the one entity they name, a facts batch refused for a stale embedding config lands once the write owner restarts with the fixed config, `transcripts ingest --facts --json` reports a budget stop instead of reading complete, a remote timeline entry, link or tag no longer overwrites how its page entered the brain, and a live fact whose context note happens to read "superseded by #N" is no longer hidden as superseded. The speaker labels "User" and "Assistant" no longer block a fact from linking to the person it names.-On a GitHub source with a webhook, a refresh waiting for one issue swallowed the event for any other issue of the same source, and deleting an issue right after editing it refetched the page instead of removing it. Each item event now gets its own refresh or delete.+Paging through takes no longer repeats or skips takes that were written together, and rebuilding takes from a page keeps the "superseded by" link on struck takes. `gbrain takes propose --accept` no longer needs `--dir` and prints the accepted claim, and accepting a proposal that supersedes a fact works on a segmented entity page. `gbrain reconcile-links --source <id>` now links the pages of that source instead of reporting `ok` with no edges on a brain with named sources, and a write failure other than a missing code page now fails the run. Timeline entries built from inline `[Source: …]` citations keep the filenames, IDs and hashes written in backticks. Old timeline rows that an earlier parser split at a hyphen (`foo-bar.de` stored as `foo` / `bar.de`) are now removed by sync and `gbrain extract timeline --prune-orphans` instead of sitting next to the correct row forever. One "portfolio" in a Facts fence or Timeline row no longer types every person-to-company link on the page as an investment.-Two unmanaged Google cases ended as held items that only a person could clear: a page whose checkout file was missing only gbrain's Facts table after one of its facts was forgotten, and an item whose page the owner purged while Google kept listing it. Both now sync cleanly; the purged item is skipped and stays purged. On managed and unmanaged sources alike, a purged Gmail thread that the change feed delivers again no longer writes its subject and quoted text back into open loops.+Hermes transcript snapshots no longer fail while Hermes is writing, and `gbrain transcripts ingest --all` keeps each file's discovered format. A managed `gbrain import` with `sync.write_through` off refuses once before admitting anything, `gbrain connectors sync --dry-run` writes no state, and a connector re-render keeps the `trust_tier` marker that `gbrain repair provenance` stamped, so doctor stops listing those pages again after every import. Open loops no longer gain a twin when the model rewords a quote, `withheld_noise` lists every sender in JSON, more no-reply sender spellings count as noise, and `gbrain google status` probes Contacts with a call the read-only Contacts permission covers. Doctor's `atom_provenance_drift` no longer flags an edit that left the page body unchanged.### Things to watch- Existing pages of forwarded emails keep their cut body until the thread changes again. Nothing is rewritten on upgrade.- Old Gmail threads already imported through the change feed stay in the brain and keep updating when they change. Shrinking the window or a full sync does not remove them.-- At 50 waiting item jobs for one GitHub source, a GitHub webhook for an item event that no waiting job covers now gets HTTP 429 `webhook_backlog_full` (`Retry-After: 30`), where it used to get 202 while the event was dropped. Redeliver it from the webhook's Recent Deliveries, or let the next sweep catch it.+- At 50 waiting item jobs for one GitHub source, a GitHub webhook for an item event that no waiting job covers now gets HTTP 429 `webhook_backlog_full` (`Retry-After: 30`), where it used to get 202 while the event was dropped. GitHub does not redeliver a 429 on its own: redeliver it from the webhook's Recent Deliveries, or let the next sweep or `gbrain sync --source <id>` catch it.+- A live fact row an older build linked as superseded is cleared by a full reconcile (`gbrain dream --phase extract_facts --source <id>`); the nightly cycle clears it only when the page changes.+- Duplicate open loops written before this change and person-to-company links already typed from a machine-written section stay as they are until their thread or page is processed again.+- `gbrain takes list --status <x>` used to be ignored silently and now exits 1, pointing to `--expired` and `gbrain takes propose --json`.### Itemized changes+#### Connectors (#8941)+- `src/core/google/google-source.ts`: `processThread` takes a `cutoffMs`; the history delta and held-thread retries pass `now - g_history_days`, and a thread whose newest message is older is settled without a page and without a hold unless it already has a page, which keeps updating (#8482). Reported and first fixed by @Masashi-Ono0611.- `src/core/google/google-render.ts`: `trimQuotedReply` keeps the whole body when a Gmail `---------- Forwarded message ----------` or Apple Mail `Begin forwarded message:` line comes before any reply marker; reply trimming, including a forward quoted inside a reply, is unchanged (#8719). Reported and first fixed by @javieraldape.- `src/commands/serve-http-webhooks.ts`: the item idempotency key carries `delete` or `refresh`, the per-source waiting cap is 50 (was 1), and a cap hit answers 429 `webhook_backlog_full` with `Retry-After: 30` unless the waiting job it returns covers the event (the same item and intent, or an explicit source-wide sync for an edit; an automatic freshness sync, which can skip, covers nothing). In `src/core/minions/handlers/sync.ts` an item job that finds another sync of its source running is deferred 30 s at a time without burning an attempt, instead of completing as a source-wide skip, so its refresh or deletion is not lost (#8799). Reported and first fixed by @javieraldape.@@ -51,9 +46,19 @@ Two unmanaged Google cases ended as held items that only a person could clear: a- Docs: `docs/guides/github-source.md` (backlog refusal, busy-source retry), `docs/architecture/key-files/google-and-loops.md`, new `docs/architecture/key-files/google-sweeps.md`.- Tests: `test/google-gmail-delta-window-8482.test.ts`, `test/google-render-forward-8719.test.ts`, `test/serve-http-github-items-8799.test.ts`, `test/google-withdrawn-fence-carry-8645.test.ts`, `test/google-classic-purged-item-8644.test.ts`, `test/sync-handler-github-item-busy-8799.test.ts`.-**Five community-reported fixes for takes, timelines and links.** Paging through takes no longer repeats or skips takes that were written together. Rebuilding takes from a page keeps the "superseded by" link on struck takes. `gbrain reconcile-links --source <id>` now links the pages of that source instead of reporting `ok` with no edges on a brain with named sources, and a write failure other than a missing code page now fails the run. Timeline entries built from inline `[Source: …]` citations keep the filenames, IDs and hashes written in backticks. Old timeline rows that an earlier parser split at a hyphen (`foo-bar.de` stored as `foo` / `bar.de`) are now removed by sync and `gbrain extract timeline --prune-orphans` instead of sitting next to the correct row forever.+#### Facts and capture (#8971)-### Itemized changes+- `src/core/facts/writeback-gate.ts`: an accepted ambient-writeback turn is banked and hashed without its `>` quote lines outside backtick or tilde fences (a line like `>90% of traffic` is still yours); the quoted-material check recognizes tilde fences too. Reported and first fixed by @Masashi-Ono0611 (#8410, #8412).+- `src/core/persistence/facts-maintenance.ts`: a page-triggered or input-keyed facts batch refused `embedding_configuration` by the write owner's consumer is re-admitted from its retained facts (no new model call, retained visibility, unattributed facts included) once no full consumer that was running at the refusal is still live; until then a retry replays the stored refusal and spends nothing, a claim forgotten in the meantime stays forgotten, and a real model change still admits nothing. Reported and first fixed by @babyspectrum (#8526, #8554).+- `src/core/facts/subject-infer.ts`: capture-stream openers (captured, watched, reviewed, logged; notes, capture, log, review, watch as labels) no longer veto subject inference as a competing name. Reported and first fixed by @mgunnin (#8656).+- `src/core/transcripts/ingest-facts.ts`, `src/commands/transcripts.ts`: `transcripts ingest --facts` returns `budgetExhausted`, `budgetReason`, `pagesSkippedTooLarge` and `pagesLockSkipped` in `--json` and prints a line for each in the human summary. Reported and first fixed by @harjothkhara (#8722, #8757).+- `src/core/context/session-speaker.ts`: a headless session that a `claude remote-control` server (or its `rc`, `remote`, `sync`, `bridge` aliases) hosts over `--sdk-url` is the owner's; a worker that session launches is still agent-launched. Existing speaker sidecars keep their first record. Reported and first fixed by @exGeni (#8080, #8892).+- `src/core/persistence/page-mutations.ts`: only remote writes that supply page text (put_page, edit_page, append_page, capture) stamp `mcp:<operation>` provenance; other remote page writes keep the stored `source_kind`, `ingested_via` and `ingested_at`. Reported and first fixed by @zsimovanforgeops (#8667).+- `src/core/facts-fence.ts`: only a struck (`~~claim~~`) row takes a `superseded by #N` link from its context cell, the way only a struck row reads `forgotten:`; a live row keeps the text as context and stays in recall. A live row an older build linked is cleared by a full reconcile, which `gbrain dream --phase extract_facts --source <id>` runs; the nightly cycle clears it only when the page changes, and on a managed brain an earlier reconcile receipt for an unchanged page can replay until then. Reported and first fixed by @alanshurafa (#9069, #9073).+- Docs: `docs/guides/ambient-writeback.md` (Remote Control sessions under automatic detection), `docs/guides/fence-format.md` (the `context` column) and the KEY_FILES entries for the touched files.+- Tests: `test/facts/writeback-gate.test.ts`, `test/facts-embedding-refusal-heal-8526.test.ts` (PGLite and Postgres), `test/facts-subject-infer.test.ts`, `test/extract-conversation-facts.test.ts`, `test/agent-session-speaker-8064.serial.test.ts`, `test/put-page-provenance.test.ts`, `test/facts-fence.test.ts`, `test/extract-facts-phase.test.ts`, `test/fence-write.test.ts`.++#### Takes, timelines and links (#8936)- `src/core/engine-sql/takes.ts`: `listTakes` ends its ORDER BY with `t.id DESC`, so LIMIT/OFFSET pages over takes sharing `created_at` are stable. Test: `test/list-takes-tie-order.test.ts`. Reported and first fixed by @Masashi-Ono0611 (#8883).- `src/core/cycle/extract-takes.ts`: the reconcile and rebuild walks write rows with `takesPreparation.toCanonicalBatchInput`, the projection page writes already use, so an inactive fence row citing `superseded by #N` keeps `superseded_by = N`. Test: `test/takes-reconcile-superseded-pointer.test.ts`. Reported and first fixed by @Masashi-Ono0611 (#8391).@@ -61,6 +66,24 @@ Two unmanaged Google cases ended as held items that only a person could clear: a- `src/core/timeline-citations.ts`: a citation summary keeps the text of each same-line inline code span (backticks dropped, never emphasis-unwrapped); fenced code and spans holding citation or comment markup stay out. `supersededInlineCitationEntries` also returns the code-blanking reading, so rows stored by it are retired on reconcile. Test: `test/timeline-citations-inline-code.test.ts`. Reported and first fixed by @Masashi-Ono0611 (#8895).- `src/core/timeline-extract.ts`: `supersededTimelineReadings` (renamed from `supersededCitationTimeline`; `persistence/canonical-projections.ts` follows) adds the pre-#3280 bullet split, read by a linear scan, so `retractRemovedTimelineEntries` and the managed projection retire those rows when their detail is extractor-written. Test: `test/timeline-legacy-split-7949.test.ts`. Reported and first fixed by @Masashi-Ono0611 (#7955).+#### Transcripts, import, connectors, loops, facts and doctor (#8319, #8332)++- `src/core/transcripts/hermes.ts`: the Hermes adapter opens the store read-only and runs `VACUUM INTO` a temporary file, so a checkpoint during the copy no longer tears the snapshot (`SQLITE_IOERR_SHORT_READ`) (#7067). Test: `test/hermes-snapshot-7067.serial.test.ts`.+- `src/core/transcripts/ingest.ts`, `src/commands/transcripts.ts`: `transcripts ingest --all` carries each file's discovered format instead of detecting it again, which failed on Claude Code logs whose first record is over 2 MiB; an explicit `--format` still wins (#7700). Test: `test/transcripts-all-discovered-format-7700.test.ts`.+- `src/commands/import.ts`, `src/core/persistence/import-mutations.ts`: a managed import with `sync.write_through` off refuses `owner_unavailable` (`detail: disabled_by_config`) once, before admitting any file, and names the supported path (commit, then `gbrain sync --source <id>`) (#7085). Tests: `test/import-write-through-off-7085.test.ts`, `test/e2e/import-write-through-off-7085-postgres.test.ts`.+- `src/core/connectors/sync.ts`: `connectors sync --dry-run` writes neither `last_sync_at` nor `auth_error_at`, and a preview whose listing stopped early returns `partial` with `listComplete: false` and `wouldFetch` (#7689, part 2). Test: `test/connectors-dry-run-7689.test.ts`.+- `src/core/persistence/connector-sync.ts`, `src/core/frontmatter-recovery.ts`: a connector re-render carries the stored `trust_tier` marker forward only when it equals the tier the import assigns (`external_untrusted`), and `trust_tier` is a protected frontmatter key, so a marker can never raise trust (#8052). Tests: `test/connector-trust-marker-8052.test.ts`, `test/e2e/connector-trust-marker-8052.test.ts`.+- `src/core/google/loops-extract.ts`: the extracted-loop dedup key is thread, direction and the normalized verbatim quote, and same-quote items in one response collapse; rows written earlier keep their key when their quote matches (#7289). Test: `test/loops-extract-run.serial.test.ts`.+- `src/core/ops/loops.ts`, `src/commands/loops.ts`: `withheld_noise` returns every sender in JSON; only the human render caps at 10 and points to `gbrain waiting --json` (#7285).+- `src/core/google/api-access.ts`: the `google status` Contacts probe calls `people/me/connections?pageSize=1`, which `contacts.readonly` covers; no scopes are added (#7490). Test: `test/google-status-contacts-probe-7490.test.ts`.+- `src/core/google/google-render.ts`: `isNoiseSender` matches separator and localized no-reply local parts (`no_reply`, `no.reply`, `do_not_reply`, `noresponder`, `notificaciones`, `notificacoes`, `ne-pas-repondre`, `keine-antwort`) on the part before the @ only (#7856). Test: `test/loops-noise-5012.test.ts`.+- `src/core/facts/subject-infer.ts`: the extractor's speaker labels no longer count as competing names, and names that start or end with a digit count only when they resolve to an existing page (#7827, items 1 and 3). Tests: `test/facts-subject-infer-7827.test.ts`, `test/e2e/facts-subject-infer-7827-postgres.test.ts`.+- `src/commands/takes.ts`, `src/core/take-proposals.ts`: `takes propose --accept` resolves its own root (repo-less brains accept through the database) and prints the accepted claim; `takes list --status` exits 1 (#7271). Tests: `test/takes-propose-cli-7271.test.ts`, `test/e2e/takes-propose-cli-7271-postgres.test.ts`.+- `src/core/facts/proposal-supersede.ts`: the managed `decide_proposal` preparer reads through `wholeBodySnapshot`, so a `supersede_fact` proposal on a segmented entity page no longer refuses `page_body_segmented` (#8060). Tests: `test/trust-supersede-segmented-8060.test.ts`, `test/e2e/trust-supersede-segmented-8060-postgres.test.ts`.+- `src/core/machine-sections.ts`, `src/core/link-extraction.ts`, `src/core/page-body/segment-links.ts`: `roleTriggerText` drops the ranges `rolePriorSuppressedRanges` already excludes, for inline and segmented extraction; no extractor or pipeline version bump (#7882). Test: `test/link-role-prior-trigger-7882.test.ts`.+- `src/core/mentions/pass.ts`: on a belief-covered source the mention pass locks the page keys and journals its link writes through `unmanagedBeliefPoint(..., 'mention_pass')`, so the drift check no longer reconciles a freshly published page as drift (#7031). Tests: `test/belief-stale-links-7031.test.ts`, `test/e2e/belief-stale-links-7031-postgres.test.ts`.+- `src/commands/doctor/checks/atom-body-recheck.ts` (new), `src/commands/doctor/checks/extraction-sync.ts`: `atom_provenance_drift` finds the `page_versions` edition whose hash matches the atom's `source_hash` and compares its body with the live body, fences removed; equal bodies count under a new `body_unchanged` detail (at most 200 groups, 20 versions each). The doctor JSON goldens gain `"body_unchanged": 0`, and `scripts/page-body-readers-baseline.tsv` gains three INLINE rows for the re-check's body reads (#6825). Test: `test/doctor-atom-provenance-drift-6825.test.ts`.+## [0.60.657.0] - 2026-11-11**A running `gbrain serve`, worker or autopilot no longer fails writes after another process changes a table's columns (GST-2260).** When `gbrain apply-migrations` or an upgrade in another process added, renamed or dropped a column, every pooled connection that had already prepared a `SELECT *` or `RETURNING *` on that table failed its next run with "cached plan must not change result type". Outside a transaction the driver quietly prepared the statement again, but inside one the error had already aborted the transaction, so the caller saw it. A `put_page` came back `internal_error`, or was accepted and then failed at publication, once for each statement on each pooled connection. Version 0.60.646.0 made the write paths re-run the transaction it aborted. Now the first such error also makes every connection in the process prepare its statements again before its next query, and an OAuth token grant whose client read meets it and a minion job's completion or failure record are re-run too. Pools that do not prepare statements (transaction-mode pooler URLs, `prepare: false`) and PGLite are unchanged.