Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(sources): count pages of deleted credentials, read prior federation under the row lock
garrytanf68f5d71aaef
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2166b2568f..a110bd0d9f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,7 +12,7 @@ identifiers and attribution are available in the pre-removal Git revision## [Unreleased]-**`gbrain sources federate` now names the pages it makes readable to everyone that a member's credential wrote.** A token or OAuth client bound to a member and granted its own source (the documented `--sources private-<member>,default` shape) writes there without `readable_by`, which is right while that source is not federated: only credentials granted it read it. Federating the source adds it to what every connection with no source grant reads, so each such page became readable by every unbound token and agent connection, and nothing said so. `sources federate <id>` still federates exactly as before, with the same exit code, and then prints on stderr how many pages in the source a remote token or OAuth client wrote that name no readers and no folder rule, up to five slugs with the member each credential is bound to now, `gbrain access who-can-read <slug> --source <id>` to check one, and `gbrain access rule add --prefix <folder/> --readers <member> --source <id>` (which previews before `--yes`) to keep a folder to its member; every command is shell-quoted. No record keeps which member a credential was bound to when it wrote, so pages from a credential unbound or rebound since then are listed too. `sources federate --json` prints one object (`id`, `federated`, and `pages_exposed` when there are such pages) and moves the human lines to stderr. `sources restore`, which re-federates by default, prints the same list, and a managed restore returns it as `pages_exposed`. The pages are read from the database's write attribution after the change commits, so a repo-less brain reports them the same way. Writes made through an attested actor (`x-qm-actor`) carry the attesting system's credential and are listed under it.+**`gbrain sources federate` now names the pages it makes readable to everyone that a member's credential wrote.** A token or OAuth client bound to a member and granted its own source (the documented `--sources private-<member>,default` shape) writes there without `readable_by`, which is right while that source is not federated: only credentials granted it read it. Federating the source adds it to what every connection with no source grant reads, so each such page became readable by every unbound token and agent connection, and nothing said so. `sources federate <id>` still federates exactly as before, with the same exit code, and then prints on stderr how many pages in the source a remote token or OAuth client wrote that name no readers and no folder rule, up to five slugs with the member each credential is bound to now, `gbrain access who-can-read <slug> --source <id>` to check one, and `gbrain access rule add --prefix <folder/> --readers <member> --source <id>` (which previews before `--yes`) to keep a folder to its member; every command is shell-quoted. No record keeps which member a credential was bound to when it wrote, so pages from a credential unbound, rebound or deleted since then are listed too. A page whose remote author was followed only by a tag change (which moves the revision attribution without a version snapshot) is not listed. `sources federate --json` prints one object (`id`, `federated`, and `pages_exposed` when there are such pages) and moves the human lines to stderr. `sources restore`, which re-federates by default, prints the same list, and a managed restore returns it as `pages_exposed`. The pages are read from the database's write attribution after the change commits, so a repo-less brain reports them the same way. Writes made through an attested actor (`x-qm-actor`) carry the attesting system's credential and are listed under it.### Itemized changes@@ -20,7 +20,7 @@ identifiers and attribution are available in the pre-removal Git revision- `src/commands/sources.ts`: `sources federate` (and `--json`) and the unmanaged `sources restore` print the list on stderr after the change commits.- `src/core/persistence/administration.ts`: a managed `source_lifecycle` restore that re-federates adds `pages_exposed` to its result.- Docs: `docs/guides/multi-source-brains.md`.-- Tests: `test/sources-federate-member-pages.test.ts` (repo-less PGLite: a member-bound token's remote `put_page` into an unfederated source, then `sources federate` names that page and its member and not a page with readers or an operator page, and an unbound token then reads it; `--json`; a credential unbound since it wrote and a slug holding `$(...)`, quoted in the commands; unmanaged `restore` and `--no-federate`; the managed restore result). It fails on master with an empty stderr.+- Tests: `test/sources-federate-member-pages.test.ts` (repo-less PGLite: a member-bound token's remote `put_page` into an unfederated source, then `sources federate` names that page and its member and not a page with readers or an operator page, and an unbound token then reads it; `--json`; a credential unbound or deleted since it wrote and a slug holding `$(...)`, quoted in the commands; unmanaged `restore` and `--no-federate`; the managed restore result). It fails on master with an empty stderr.## [0.60.622.0] - 2026-11-09diff --git a/docs/architecture/key-files/access.md b/docs/architecture/key-files/access.mdindex 0411395e41..77e6e48a5b 100644--- a/docs/architecture/key-files/access.md+++ b/docs/architecture/key-files/access.md@@ -4,5 +4,5 @@Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.- `src/core/access/` (GBRA-218) — per-member read access inside one source. `principals.ts` normalizes frontmatter `readable_by` (bare id = `member:<id>`, `group:<g>`, ASCII case fold, space trim; anything malformed is `['!nobody']`); it is the row twin of the SQL `gbrain_access_readers()` (`test/access-principals.test.ts` holds them equal). A BEFORE trigger keeps `pages.page_readers` equal to that function (direct writes to the column are recomputed), and predicates test the array, never the function, which costs ~0.2 ms per row. A page that names readers with no derived value yet (written before the migration) reads as nobody until `gbrain repair access` (`repair/access.ts`). `members.ts` (members, groups, tombstoned ids never reused, bindings), `token-binding.ts` (the verifiers' join; a credential bound to a disabled, tombstoned or missing member is refused with `access_member_disabled`) and `caller.ts` (`remoteCallerAccess`: the identity dispatch puts on the caller-access record; stdio binds through `mcp.stdio_member`; publication re-resolves from the stored principal). In `search/private-visibility.ts` every page fragment is `accessAllowedSql` AND the privacy part: plain fragments use the empty identity (over-hide, never leak), `*ForCaller` fragments the caller's, and `GBRAIN_REMOTE_PRIVATE_PAGES`, `search.remote_private_pages` and `granted_sources` relax only the privacy part (`privacyForCaller`). A derivative of a page that names readers is hidden from every remote caller (`originAllowedSql`, which calls the SQL function `gbrain_access_origin_restricted()` only for atom, question, concept, event or `derived_from` rows, so each read plans one call instead of four subplans). Both parts lead with column tests (`page_readers IS NULL`, `type`) so row estimates follow column statistics; an expression test such as `->'readable_by' IS NULL` estimates almost no rows and sent restricted vector searches into nested-loop sequential scans. The index walk's page probe (`INDEX_WALK_PAGE_COLUMNS`) projects `page_readers`. Config `access.restrictions_present` (set by a trigger, sticky) keeps every pre-A1 path on a brain with no restriction. `check:remote-privacy-reads` keeps the two relaxations read only in `private-visibility.ts`. Tests: `test/member-read-access.test.ts` (remote matrix, both engines), `test/access-*.test.ts`. Folder rules (A2) keep whole folders of a source to named members. `folder-rules.ts` is the grammar (`.gbrain/access.yml`: `rules:` of `id`, `prefix`, `readable_by`), `classifyRulebook` (each change is narrowing or widening; the target installs only the narrowing part) and `matchingRuleIds`. `policy.ts` owns the one applied rulebook for both stores: `access_rules` (source-qualified ids), the append-only `access_policy_revisions` log, `access_pending_coverage` (prefixes that read as nobody while a narrowing restamps, or while a committed file is invalid) and `sources.config.access_store`, persisted at the first rule and changed only by `policy-migrate.ts` (journaled in `access_policy_migrations`). `proposeRevision` installs narrowing at once under a per-source row lock and holds widening for `applyApproved` (`plan_hash`, `preview_changed`); `restampSource` rewrites `pages.folder_rule_ids` in id windows, then clears pending coverage under the lock; re-proposing the same rulebook finishes an interrupted restamp. A trigger stamps `folder_rule_ids` from the SQL function `gbrain_access_folder_rules()` on insert and on slug or source change, never from git on the serve path. `policy-git.ts` is the git store (files committed through the commit door, applied by `syncAccessPolicyAt` from sync preflight and before and after every managed pass (`performManagedSync`: sync, drain, `sources refresh`, git conversion), read with `git show` at the synced commit; a database with no revision yet replays each committed `access.yml` since the `access.approved` baseline, so a rebuild holds a removal a plain commit made); `policy-transfer.ts` carries the applied rulebook through `gbrain export` and installs it before any page in `gbrain import`; `policy-notice.ts` is the `access_policy_pending` degraded notice. In `private-visibility.ts`, `folderRulesAllowedSql` sits inside `accessAllowedSql` (`folder_rule_ids IS NULL OR folder_rule_ids <@ <satisfied ids>` plus coverage conjuncts), so no hatch, grant or `--as` relaxes it, and the unrestricted short form tests `folder_rule_ids IS NULL` next to `page_readers IS NULL` (v283 triggers set `access.restrictions_present` on the first rule or pending prefix); `caller.ts`'s `folderAccessFor` fails closed to whole-source coverage. CLI: `src/commands/access-rules.ts`; doctor: `src/commands/doctor/checks/folder-access.ts`. Tests: `test/access-folder-rules.test.ts` (database store, both engines), `test/access-folder-rules-git.test.ts`, `test/access-folder-transfer.test.ts`, `test/access-folder-doctor.test.ts`. Shared agents (B, GBRA-231): `audience.ts` parses the audience a harness asserts (`X-Gbrain-On-Behalf-Of` or request-level `_meta.on_behalf_of`, 1-50 principals, `guest` = the empty identity; the same key inside tool arguments is refused) and appends each assertion to `access-audience-YYYY-Www.jsonl`. Dispatch resolves it on every remote call through `caller.ts` (`resolveAudienceIdentities`: every entry must be in the credential's live `may_act_for`, then name an enabled member or existing group); the caller-access record then carries `audienceIdentities`, and `accessAllowedForAllSql` needs one readers conjunct per identity, folder rules the intersection of each entry's satisfied set (`folderAccessForAll`). Privacy relaxes only for a one-member audience. Writes carry the audience as `WriteAuthority.actingFor` (re-resolved at publication) and agent jobs as `grant.actingFor`. Every remote response carries `_meta.access` (`appliedAccessSummary`), which `whoami` returns as `access.applied`; doctor `access_may_act_for_wildcard` (`checks/shared-agent-access.ts`) warns on `*`. Tests: `test/access-on-behalf-of.test.ts` (both engines), `test/e2e/serve-http-on-behalf-of.test.ts`. Attested actor (C1, GST-359): `actor-model.ts` (dependency-free: `x-qm-actor`, `acting_user`, `EXTERNAL_ID_RE` = 1-256 visible ASCII, `attestRightFrom`: the right counts only for a member-less credential whose token is legacy or `client_credentials`, read from `oauth_tokens.grant_type`) and `actor.ts`. `resolveAttestedActor` runs in dispatch before any work: an invalid id refuses `actor_not_attested` (`invalid_actor_id`) before any lookup, then the right, the op list (`attest_ops`, default `get_write_request,memory_recall,memory_write`), `acting_user` = header (stripped), and `actorCoverage` (the one coverage test: live right plus a live link in `access_member_links` to an enabled member, compared exactly) or `actor_unlinked`. `ACTOR_DEFERRED_OPS` (`capture`, `extract_facts`, take writes, `list_write_conflicts`) are never attestable until each is keyed to the member: refused by `parseAttestOps`, `bindCredential`, every call and the stored re-check, which also re-checks `attest_ops` for a queued write's operation and each job tool call. Under the header serve-http lists only attested tools and refuses `resources/read`. `attestedCallerAccess` is a one-member audience with privacy never relaxed (no granted sources, no hatch), never B's `may_act_for` path. Writes store `WriteAuthority.actingFor = {system, external_id, member}` and jobs `grant.actor`; `storedActorIdentity` re-resolves both through `actorCoverage` and refuses when the member changed. `linkMember` / `unlinkMember` / `listMemberLinks` back `gbrain access member link|unlink|links`; `--attest-actor` / `--attest-ops` on `auth rescope` (`members.ts`) only for a member-less legacy token or confidential client-credentials client. Migration v286. Tests: `test/access-qm-actor.test.ts` (both engines), `test/e2e/serve-http-qm-actor.test.ts`. Member facts (C2, GST-359): `member-facts.ts`. A remote `remember` from an attested actor or a lone-member credential (`callerMember`, never an audience of several) writes to `memory/members/<member>` (`memberFactPage`; `persistence/memory-prepare.ts` creates the page with `readable_by: [member:<id>]`), `visibility: private`, `infer_entity` off, the entity kept as `entity_slug`. A member id whose page slug `slugifyPath` would change (`a--b`, `zed-`) is refused (`invalid_params`). The page must name that member alone (`memberPageIsOwn`, checked at submission and again in preparation, `memberPageNotOwned`), and remote writes under `memory/members/` go only to the caller's own page (`assertMemberPageWriter`, called from `ops/context.ts` `enforceClientSlugFence`, so every remote page write passes it; trusted local callers are unaffected). Reads widen a world-only fact list only by `memberFactAdmitSql` (a private fact on the caller's own live member page whose readers and folder rules admit the caller; `factVisibilitySql` in `engine-sql/facts.ts` and `search/fact-relevance.ts`), and `privateProvenanceFilterUnderPolicy` judges a member-page fact's source page as the caller (every other fact keeps the empty identity, so a fact from a page that names readers stays hidden from every remote caller). Writers never cross pages: `facts/single-prepare.ts` (`SingleFactIntent.page_slug`) and `trust/supersede-handlers.ts` dedup, supersede and contest only within the member page, and no other write matches a fact there (`onMemberPageSql`); consolidation skips facts whose page names readers (`readersSourcedFactSql`). Per-person keys: `callerIdentityKey` joins the hot-memory cache key (`facts/meta-hook.ts`, `::xid=`), `callerSessionKey` the `delta` cursor (`ops/facts.ts`), `sameAttestedActor` gates receipts (`persistence/authority.ts` `ownRequestAccessible`) and replays (`persistence/journal.ts`), and `get_agent_job` and the owner-fenced `get_job`, `wait_job`, `get_job_progress`, `list_jobs` and `cancel_job` match the job's `grant.actor` (system, external id, member; `jobActorSql` in `minions/types.ts`), so a job with an actor reads as missing to the bare client and to other actors. Tests: `test/access-member-facts.test.ts` (both engines), `test/access-member-sweep.test.ts` (every attestable op as Bob, the bare client and an unbound credential; `test/helpers/member-sweep.ts`), `test/e2e/serve-http-member-sweep.test.ts` (two members over a real `serve --http` on a keyless repo-less brain).-- `src/core/access/federation-exposure.ts` (#8982) — `remotePagesFederationExposes(engine, source)`: the live pages of a source that name no readers and no folder rule and whose live revision (`pages.revision_principal_*`) or any version (`page_versions.write_principal_*`) a token or OAuth client wrote (revoked included; no record keeps a credential's member at write time, so an unbound or rebound one counts and each example names its member now): count, the first five slugs, and shell-quoted `who-can-read` / `access rule add` commands. `remotePagesFederationExposesOrWarn` runs it, after the change commits, only for a change that added the source to the no-grant floor, and turns a failed read into a stderr warning, so the caller's federation never changes. Read by `sources federate`, the unmanaged `sources restore` (stderr; `--json` as `pages_exposed`) and the managed `source_lifecycle` restore result. Pinned by `test/sources-federate-member-pages.test.ts`.+- `src/core/access/federation-exposure.ts` (#8982) — `remotePagesFederationExposes(engine, source)`: the live pages of a source that name no readers and no folder rule and whose live revision (`pages.revision_principal_*`) or any version (`page_versions.write_principal_*`) a token or OAuth client wrote (revoked or deleted included; no record keeps a credential's member at write time, so an unbound, rebound or deleted one counts and each example names its member now; a tag change moves the revision attribution without a version, hiding the earlier author): count, the first five slugs, and shell-quoted `who-can-read` / `access rule add` commands. `remotePagesFederationExposesOrWarn` runs it, after the change commits, only for a change that added the source to the no-grant floor (federate reads the prior state under the source row lock), and turns a failed read into a stderr warning, so the caller's federation never changes. Read by `sources federate`, the unmanaged `sources restore` (stderr; `--json` as `pages_exposed`) and the managed `source_lifecycle` restore result. Pinned by `test/sources-federate-member-pages.test.ts`.- `src/core/ops/qm-memory.ts` (GST-359 D) — QM's memory provider pair. `memory_recall` reads the attested person's own last-24-hour facts on their member page (`listFactsSince` with `sourceMarkdownSlug`), then `rankFactsByQuestion` over the facts they may read, renders one plain line per fact (`plainFactText` strips fence and heading markers) and cuts at a line within `max_chars` (`boundFactLines`). A call that names `source_id` or `source_ids` (full surface only) reads as that scope instead: `recallSources` resolves them through `requestSourceScope` (a source outside the credential's grant is `permission_denied`), `SCOPE_READ_ACCESS` empties identity, actor and audience, so no own facts and no member-page fact are read, restricted pages stay hidden and privacy is never relaxed; the query is ranked on its first 2,000 characters (`MEMORY_RECALL_QUERY_CHARS`). `memory_write` validates 1-20 facts, derives each request id from uuidv5(member, idempotency_key) (`memoryWriteRequestId`) and saves through `remember`'s handler with `ctx.attestedOp = 'memory_write'`, which the stored write authority keeps (`WriteAuthority.attestedOp`) so publication and receipt reads re-check that operation's attest right. `qmMemoryToolResult` (dispatch) moves the result into `structuredContent` and leaves the model text in `content`; `unlinkedRecallResult` is the empty success dispatch's `actorRefusal` returns for an unlinked person. Contract fixture: `scripts/build-qm-memory-contract.ts` writes `docs/contracts/qm-memory-provider/v1.json` (frozen, byte-identical) and `v2.json` (the scope read and write; `regen:all` artifacts). Scope writes (GST-1113): `memory_write {source_id}` saves on `memory/scope` in a scope source listed in the credential's `attest_sources` (migration v293, on `access_tokens` and `oauth_clients`); `access/attest-sources.ts` holds `perCallWriteSource` (the one exemption from the one-write-source checks in `page-mutations.ts`, `memory-mutations.ts` and `persistence/authority.ts`, keyed to that slug and re-checked at publication through `actorCoverage(..., source)`), the sticky mark `sources.config.qm_scope` (set by `applyAttestSources` under row locks; `isSourceFederated` ignores federation on it), `assertScopeSourceGrant` (in `insertClientGrant`, `persistGrant`, `writeTokenGrant` and `insertUnifiedToken`) and `withoutScopeSources` (a person recall never reads a scope source). `member-facts.ts` `assertScopePageWriter` keeps other remote writers off `memory/scope`. Tests: `test/qm-scope-writes.test.ts` (both engines), `test/qm-memory-tools.test.ts` (both engines), `test/qm-memory-contract.test.ts` (drift), `test/e2e/serve-http-qm-memory.test.ts` (keyless `serve --http`).diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex 5532ad9eb0..57b4d6c79c 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -30,7 +30,7 @@ src/commands/integrations.ts 1739 ratchet grown v0.46.26.0 megawave (#4039 connesrc/core/minions/handlers/subagent.ts 1925 ratchet merged: master v131 settle rework + dream-wave C5 peel + C6 accounting + C7 lease heartbeats/lost-lease aborts + C9 oneshot dispatch + adversarial fix batches; grown by #4078 subagent-loop capability fix; +6 lines (reasoning-item replay: adaptContentBlocksToChatBlocks handles 'reasoning' blocks); +3 wave G #4514: Anthropic-via-OpenRouter allowed on the subagent loop; +36 lines (code review, same PR): isPromptTooLongError walks a bounded .cause chain so it still matches post-normalizeAIError() wrapping, plus extractPromptTooLongDetail so the dead-lettered job's error carries the matched provider detail instead of the generic normalizeAIError() wrapper text; master integration: harness support: boundary wiring to shared grants, isolated setup, withdrawal and delegated guards; #5105: +13 call-site wiring to shared tool-write-identity helper across fresh and replay dispatch; +7 fix wave 9 lane B #4921: ai.chat.per_turn_timeout_ms read once and passed to the oneshot and gateway-loop paths (resolver lives in minions/handler-timeouts.ts)src/commands/bootstrap.ts 2154 ratchet grandfathered at merge (grew past the 1500 cap on master); raised for #4066 HOME_WORKSPACE_GUARD_EXEMPT (status/uninstall/harness); +5 for the harnessDetect test seam (bulk lives in core/bootstrap/harness.ts); +23 for the contract subcommand dispatcher glue (#4105 absorb — bulk lives in core/bootstrap/contract.ts); +codex hooks-consent gate + mcp+hooks receipt registration + uninstall arm; +42 ambient-writeback WP8: interview-time SURFACE_MULTIUSER audience apply (pre-init ordering) + applyDeclaredAudienceFromInterview; +3 #4897 harness usage clause documenting --source ID semantics; +5 #5878: --refresh-skills dispatch and usage lines (logic in core/bootstrap/harness-skills.ts); +11 #5488: SUBCOMMAND_HELP harness entry (gate on --help / -h / help) — CLI help line, cannot move; +11 #4618 seat: --seat/--no-seat help lines and the runHooks parse (parse + notes live in core/bootstrap/hooks.ts parseSeatFlags); +13 #4082: runAdoptHooks (the --adopt branch of the hooks subcommand, kept out of runHooks for the function-size ratchet) and its two help lines (adoption logic lives in core/bootstrap/adopt-connection.ts + adopted-connections.ts); +4 #8426/#8428: the user-scope MCP registration, its verify and the workspace and Codex SessionEnd hooks start gbrain through the hook launcher (logic in core/setup/hook-launcher.ts); +2 #8435: the user-global opencode entry starts serve through the hook launcher (logic in core/setup/hook-launcher.ts); +1 GBRA-224: INSTALL_PIN import (core/install-ref.ts; re-summed at the GBRA-346 merge: master 2153 + 1)src/core/minions/worker.ts 1795 ratchet grandfathered at merge (grew past the 1500 cap on master, #4170); grown v0.46.11.0 five-issue wave; security: durable queued authority and confined filesystem dispatch; typed readiness, configuration-blocked lifecycle and bounded release; post-validation readiness publication and confirmed isolated-execution stop evidence; latch typed lock-renewal faults before generic retry; +9 Lane D: JobDeferredError routing (deferJob helper); +1 spend authorization: runWithJobSpend import (both handler seams call it in place of the bare handler); +3 watchdog: gracefulShutdown records its reason and launchJob aborts a claim committed before it fired; +5 GBRA-130 A-30: watchdog-attribution import, InFlightJob.shared (interface field), the suspect-alone claim gate and option, and the watchdog settle call in the executeJob catch (logic in minions/watchdog-attribution.ts); +6 GBRA-133 2a.4 sync-queue wake: IdleWaiter import and field, claimStarted before each claim, the idle poll through it, and stop/requestShutdown ending its wait (logic in minions/idle-wait.ts); +1 GBRA-158 #7239: refusal.ts import (a deterministic refusal ends the job at once; classification in minions/refusal.ts); +3 #7242 lane reservation: lanes import and the longJobNames/shortJobReservedSlots defaults (the claim passes claimExclusions; policy in minions/lanes.ts); +2 #7242 terminal hook: onJobSettled default and the settle call (logic in terminal-hook.ts); #7252 x #7249 merge: both sides grew the file, re-summed (1794 measured); GBRA-151 #7183/#7187/#7188 re-summed at the v0.60.402.0 merge (1795 measured)-src/commands/sources.ts 2123 ratchet +3 url_source_origin: sources record-origin dispatch and help rows (body in sources-record-origin.ts, logic in core/url-source-origin.ts); +3 PR B: sources guard dispatch and help rows (body in sources-guard.ts); +3 G7b-2: sources host-repo dispatch and help row (body in sources-host-repo.ts and core/persistence/hosted-repo.ts); +5 PR #5955 / fix wave 14 P2.7: sources set-strategy dispatch and help row, the strategy field of sources list --json (body in sources-strategy.ts); +5 #5988: sources status/retry-held help rows describe Git holds (help lines cannot move); +2 #5988: sources status <id> filter and Git held files (view in persistence/connector-status.ts); +5 F0: sources refresh dispatch and help rows (command body in sources-refresh.ts, protocol in persistence/worktree-refresh.ts); +6 fix wave 5 #5409: mirror-readonly/mirror-writable dispatch, help rows and the list --json mirror_read_only field (body in sources-mirror.ts); grown v0.46.22.0 phone wave: github source kind (#4104); +129 v0.47 gmail-loops: google source-kind flags + vault preflight + duplicate-account warn + access-mode validation; +8 set-path dispatch + help rows (#4739 — command body peeled to sources-set-path.ts); +29 #4698 adoption: --calendar-id flag + duplicate-account warn scoped to overlapping services; #5105: trusted writer administration route; company handlers and receipt status wiring; parsing stays in dedicated modules; canonical reconciliation: +5 engine-free sibling-command dispatch and help row, implementation remains in source-reconcile.ts; +4 fix wave 3 lane A: sources status connector recovery state and run counts (view lives in persistence/connector-status.ts); +3 fix wave 4 lane B: sources retry-held dispatch and help row (command body in sources-retry-held.ts) and the connector first-sync note on add; bodies in source-delete.ts and sources-set-path.ts); +4 #5984: sources status shows the managed sync backlog (logic in persistence/sync-drain.ts); v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (2031 measured); +11 W-B4: sources shared-skills dispatch, its --help guard and help rows, and the shared_skills field/lines in sources status (body in sources-shared-skills.ts, view in shared-skills/source-opt-out.ts); +4 #6188: uncommitted legacy fence-repair notices in sources status (logic in fence-repair/uncommitted.ts); +7 #6317: data_moving/movement_state on sources status and its human line (reader in persistence/sync-movement.ts), writer movement help row; +1 GBRA-133: sources reset-upstream help row (body in sources-reset-upstream.ts); fix wave 20 integration: lane 3 (#6827) and master growth compose at 2062 measured; +3 GBRA-150: sources resolve-path dispatch and help rows (body in sources-resolve-path.ts); +10 #7326: sources status parses --holds/--limit/--cursor and passes the git_holds listing mode, plus 4 help lines (bounding logic in persistence/connector-status.ts gitHoldStatusJson); +2 GBRA-133 Q1: sources set-sync dispatch and help row (body in sources-sync-enabled.ts); +4 G1 git-canonical: sources git-gap dispatch, doc line and help rows (body in sources-git-gap.ts); +4 G3 git-canonical: sources reindex dispatch, doc line and help rows (body in sources-reindex.ts); +1 GBRA-156: the sources add non-federated warning line; re-summed at the master 33d73cb5 merge (2086 measured); +4 G8a git-canonical: sources canonical dispatch, doc line and help rows (body in sources-canonical.ts, plan and apply in persistence/git-conversion.ts); GBRA-156: sources add --json and its next step (logic in sources-add-next.ts); re-summed at the master ade6d638 merge (2100 measured); +2 GBRA-220 PR 2: the unmanaged sources add routes --kind meetings|slack to the shared lifecycle parser (body in sources-add-api.ts); re-summed at the master b85dd645 merge (2102 measured); +2 #7701: the archived field of SourceListEntry and its list entry (sources list now marks archived sources; an interface field cannot move): re-summed at the 0.60.583.0 merge: 2102 + 2; +1 GST-1113: sources federate refuses a QM scope source (one line; the check lives in core/access/attest-sources.ts), re-summed at the 0.60.594.0 gate-ahead: 2107 + 1; +5 GST-1113 /ship fix round: sources federate checks the scope mark and writes the config in one transaction under the source row lock (the check lives in core/access/attest-sources.ts); +7 #8982: sources federate and restore print the remote-written pages without readers they expose, federate --json (the read and the lines live in core/access/federation-exposure.ts)+src/commands/sources.ts 2124 ratchet +3 url_source_origin: sources record-origin dispatch and help rows (body in sources-record-origin.ts, logic in core/url-source-origin.ts); +3 PR B: sources guard dispatch and help rows (body in sources-guard.ts); +3 G7b-2: sources host-repo dispatch and help row (body in sources-host-repo.ts and core/persistence/hosted-repo.ts); +5 PR #5955 / fix wave 14 P2.7: sources set-strategy dispatch and help row, the strategy field of sources list --json (body in sources-strategy.ts); +5 #5988: sources status/retry-held help rows describe Git holds (help lines cannot move); +2 #5988: sources status <id> filter and Git held files (view in persistence/connector-status.ts); +5 F0: sources refresh dispatch and help rows (command body in sources-refresh.ts, protocol in persistence/worktree-refresh.ts); +6 fix wave 5 #5409: mirror-readonly/mirror-writable dispatch, help rows and the list --json mirror_read_only field (body in sources-mirror.ts); grown v0.46.22.0 phone wave: github source kind (#4104); +129 v0.47 gmail-loops: google source-kind flags + vault preflight + duplicate-account warn + access-mode validation; +8 set-path dispatch + help rows (#4739 — command body peeled to sources-set-path.ts); +29 #4698 adoption: --calendar-id flag + duplicate-account warn scoped to overlapping services; #5105: trusted writer administration route; company handlers and receipt status wiring; parsing stays in dedicated modules; canonical reconciliation: +5 engine-free sibling-command dispatch and help row, implementation remains in source-reconcile.ts; +4 fix wave 3 lane A: sources status connector recovery state and run counts (view lives in persistence/connector-status.ts); +3 fix wave 4 lane B: sources retry-held dispatch and help row (command body in sources-retry-held.ts) and the connector first-sync note on add; bodies in source-delete.ts and sources-set-path.ts); +4 #5984: sources status shows the managed sync backlog (logic in persistence/sync-drain.ts); v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (2031 measured); +11 W-B4: sources shared-skills dispatch, its --help guard and help rows, and the shared_skills field/lines in sources status (body in sources-shared-skills.ts, view in shared-skills/source-opt-out.ts); +4 #6188: uncommitted legacy fence-repair notices in sources status (logic in fence-repair/uncommitted.ts); +7 #6317: data_moving/movement_state on sources status and its human line (reader in persistence/sync-movement.ts), writer movement help row; +1 GBRA-133: sources reset-upstream help row (body in sources-reset-upstream.ts); fix wave 20 integration: lane 3 (#6827) and master growth compose at 2062 measured; +3 GBRA-150: sources resolve-path dispatch and help rows (body in sources-resolve-path.ts); +10 #7326: sources status parses --holds/--limit/--cursor and passes the git_holds listing mode, plus 4 help lines (bounding logic in persistence/connector-status.ts gitHoldStatusJson); +2 GBRA-133 Q1: sources set-sync dispatch and help row (body in sources-sync-enabled.ts); +4 G1 git-canonical: sources git-gap dispatch, doc line and help rows (body in sources-git-gap.ts); +4 G3 git-canonical: sources reindex dispatch, doc line and help rows (body in sources-reindex.ts); +1 GBRA-156: the sources add non-federated warning line; re-summed at the master 33d73cb5 merge (2086 measured); +4 G8a git-canonical: sources canonical dispatch, doc line and help rows (body in sources-canonical.ts, plan and apply in persistence/git-conversion.ts); GBRA-156: sources add --json and its next step (logic in sources-add-next.ts); re-summed at the master ade6d638 merge (2100 measured); +2 GBRA-220 PR 2: the unmanaged sources add routes --kind meetings|slack to the shared lifecycle parser (body in sources-add-api.ts); re-summed at the master b85dd645 merge (2102 measured); +2 #7701: the archived field of SourceListEntry and its list entry (sources list now marks archived sources; an interface field cannot move): re-summed at the 0.60.583.0 merge: 2102 + 2; +1 GST-1113: sources federate refuses a QM scope source (one line; the check lives in core/access/attest-sources.ts), re-summed at the 0.60.594.0 gate-ahead: 2107 + 1; +5 GST-1113 /ship fix round: sources federate checks the scope mark and writes the config in one transaction under the source row lock (the check lives in core/access/attest-sources.ts); +7 #8982: sources federate and restore print the remote-written pages without readers they expose, federate --json, the prior federation read under the row lock (the read and the lines live in core/access/federation-exposure.ts)src/core/link-extraction.ts 2424 ratchet grown v0.46.26.0 megawave: link-aware timeline split + delimiter-outside-links (#3957 #4062 #3737); +37 wave-k #3908 reimpl: isCrossSourceLinksEnabled config ladder; +10 wave #4277: parseTimelineEntries skips generated backlink receipts; +17 #4855: latin-fold in normalizeBasename + dir-hint step tries both slug grammars; +17: #4855 Latin stroke-letter fold; +5: #4873 ./-relative markdown links kept as live references; +34 wave review: `../y.md` sibling links resolve on the DB path + folded bare-wikilink grammar (#4873 class); +62 #4977: role-prior suppression in machine-list sections (rolePriorSuppressedRanges, EntityRef.index); +3 composed with #4985 (variation-selector strip) in the same wave; +4 #4995: markdown-link anchor strip + lowercase-candidate rationale comments; +6 pre-landing review: #4977 suppression on code-stripped content + truthful watermark/section-list comments; T2: source-qualified inference and atomic origin-scoped derived links; canonical attendance evidence, source-qualified orientation, strict attendee resolution, and relocated filesystem link scanner (command export preserved); +4: pack-owned fallback and ambiguous attendee guards Display-label attendance exclusion, indexed evidence membership, and bounded strict resolver caching; attendance-boundary review.; complete attendance resolution and fence-safe evidence traversal; frontmatter discovery, canonical basename admission and code-aware attendance comment masking; exclude inline HTML comment spans from admitted attendance ranges; +17 write-path wave: same-name rule for entity fuzzy matches + rename-alias fallback in makeResolver; +1 #5567: materialized marker ends bullet detail; +30 #5749: unique global-basename step before fuzzy and live keyword resolution (ResolveOptions, watermark note); #5765: bold **Attendees:** label accepted + LINK_EXTRACTOR_VERSION_TS bump note; +10 eval-wave N9-2/N9-3/N12-7: pack frontmatter mappings inherit FRONTMATTER_LINK_MAP direction, page-type-bound pack attended follows canonical attendance, LINK_EXTRACTOR_VERSION_TS bump note; +1 #5623: normalizeBasename collapses hyphen runs like slugifySegment (extractor watermark note); +11 #5562: parseTimelineEntries accepts dated ### headings (FS-extractor parity); +3 eval-wave N12-6: Participants accepted as attendance evidence + LINK_EXTRACTOR_VERSION_TS bump note; +22 P5 wanted pages: AuthoredRef on body link candidates; -33 lowered: P5 moved machine-section ranges to core/machine-sections.ts and stated relation types to core/line-grammar.ts; +7 P5: ExtractPageLinksOptions interface + wikilinkAuthoredRef helper (keeps extractPageLinks under the function-size limit); +32 P5: per-edge verb attachment (attachedVerb, VERB_RULES) + joined-as works_at + extractor watermark note; P5 x temporal-edges merge: verb attachment reads the verb at the link's own position in the window (excerptAnchor) and the within-page dedup (now dedupeCandidates) drops a mention of a target the page also links with a typed edge; links joined only by commas/conjunctions share the verb before the first (coordinated); board roles (BOARD_RE, BOARD_AFTER_RE) resolve to invested_in for an investor page and advises otherwise, never works_at; +1 fix wave 11 #6191: imports targetTakesVerb (role-verb target guard lives in link-target-roles.ts); v0.60.105.0 merge: #5984 Phase 5 link preparation (+6) re-summed over wave 11 at 2211; +109 Q2 Track C: stable verb-rule ids, traceLinkType/explainLinkType (reasons for every typing decision), per-occurrence typer hoisted out of extractPageLinks (makeTypeFor); unit logic lives in core/link-typing-units.ts; +25 Q2 Track C: typing-unit verb rules and vetoes in attachedVerb/traceLinkType, validity stints in explainLinkType; +4 U3: board-seat rule split out of INVESTED_RE (same verb and precedence) and investor-prior/link offsets passed to unit vetoes; +11 U2: locateLink shared by attachedVerb and the U2 post-pass in traceLinkType; -11 lowered: Q2 verdict reverted U2 (locateLink and the post-pass); +21 fix wave 13 P4.10 #6266: resolveSameDirSlug + exported resolveMarkdownLinkSlug, shared with the link validator; +41 Wave B PR 2 lane 2e: extractPageLinks split into extractBodyLinks (window-scoped, offset-shifted, pass-tagged candidates) and frontmatterLinks, plus entityRefPasses, candidateKey, rolePriorPhrases and the roleText/regexBudget/window options, so a segmented page streams through the same extractor (window and merge logic in core/page-body/segment-links.ts); +1 at the #6869 gate-ahead merge: the #6803 wave EXTRACTOR_BUMP_NOTES comment composed with lane 2e (2420 + 1); +3 G5 (GBRA-60): extractBodyLinks masks the ## Links fence (links-fence.ts), whose rows are projected as their own edgessrc/core/embedding-migration.ts 1587 ratchet new row: crossed 1500 in v0.46.25.0 fix waves (#4305 #4306 chunk-model truth helpers); retired provider integration removed; safety wave: canonical projection readiness and fact-aware completion; file configuration publication retains lease-row locks; +8 transactional retained-vector rebuildability checks before destructive migration boundaries; archived eligibility: source-locked reconciliation and live/current smoke revalidation; +2 #5680: authorization_generation field and its carry-forward on same-target re-apply (settle idempotency); +10 #5226: plan carries chunks_to_restamp beside chunks_to_embed; +9 #5088: schema transition defers HNSW builds into the marker (deferred_ann_indexes), carry-forward on re-apply, verify blocks on pending/INVALID ANN indexes (build phase in embedding-ann-build.ts); +10 #5088: dim-pinned tables (query_cache, facts, and any later entry of TEXT_EMBEDDING_DIM_PINNED_TABLES) defer their HNSW through the same worklist (DEFERRED_ANN_TABLES, recordDeferredAnnIndexes); +25 wave 8 lane D1 #5885: takes in the dim-pinned column list, plan (takes_to_embed), status (takes_pending) and completion verify; +17 #5334: smoke check samples one representative chunk per distinct page (bounded index-ordered batch walk, longest chunk re-read under the page lock, title-prefixed bounded query); +1 #5334: the smoke-query length limits are shared with the worst-case budgetsrc/core/github-source.ts 1530 ratchet new row: crossed the 1500 unlisted cap in v0.46.23.0 review fix wave (scope-state gating, client hardening, symlink containment); merged reliability/shared-skills source integration; managed sweep lease wrapper; fix wave 3 lane A: config parser peeled to github-source-config.ts (-47), +account pin wiring and pinned-installation refreshdiff --git a/src/commands/sources.ts b/src/commands/sources.tsindex dcf70c43e6..8916b3efcb 100644--- a/src/commands/sources.ts+++ b/src/commands/sources.ts@@ -1178,19 +1178,20 @@ async function runFederate(engine: BrainEngine, args: string[], value: boolean):}// GST-1113: the mark check and the config write hold the source row, so a concurrent --attest-sources mark commits// before (and federate refuses) or after (and sees federated, so it refuses), never between them.- await engine.transaction(async tx => {+ const wasFederated = await engine.transaction(async tx => {const [row] = await tx.executeRaw<{ config: unknown }>('SELECT config FROM sources WHERE id = $1 FOR UPDATE', [id]);if (value) await (await import('../core/access/attest-sources.ts')).assertNotScopeSource(tx, id);- const config = parseConfig(row?.config ?? src.config);+ const config = parseConfig(row?.config ?? src.config), before = isFederated(config);config.federated = value;await tx.executeRaw(`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,[JSON.stringify(normalizeSourceConfig(config)), id],);+ return before;});say(`Source "${id}" is now ${value ? 'federated (appears in cross-source default search)' : 'isolated (only searched when explicitly named)'}.`);const exposure = await import('../core/access/federation-exposure.ts');- const exposed = await exposure.remotePagesFederationExposesOrWarn(engine, id, value && id !== 'default' && !isFederated(src.config));+ const exposed = await exposure.remotePagesFederationExposesOrWarn(engine, id, value && id !== 'default' && !wasFederated);if (exposed) console.error(exposure.pagesExposedLines(id, exposed).join('\n'));if (json) console.log(JSON.stringify({ id, federated: value, ...(exposed ? { pages_exposed: exposed } : {}) }));diff --git a/src/core/access/federation-exposure.ts b/src/core/access/federation-exposure.tsindex a02ac5bafd..8d9c62c3f2 100644--- a/src/core/access/federation-exposure.ts+++ b/src/core/access/federation-exposure.ts@@ -6,9 +6,10 @@* it writes nothing and never refuses the change.** A page counts when it names no readers and no folder rule and the writer of its live revision or of any earlier- * version (the write attribution columns) is a remote credential: a token or OAuth client, revoked or not. No record- * keeps which member a credential was bound to when it wrote, so a credential unbound or rebound since then counts- * too, and each example names the member it is bound to now (null when none). The database is the record, so a+ * version (the write attribution columns) is a remote credential: a token or OAuth client, revoked or deleted since. No+ * record keeps which member a credential was bound to when it wrote, so a credential unbound or rebound since then+ * counts too, and each example names the member it is bound to now (null when none or deleted). A write that moves+ * the revision attribution without a version snapshot (a tag change) hides the earlier remote author from this read. The database is the record, so a* repo-less brain answers the same way.*/import { shellQuote } from '../agent-output.ts';@@ -24,19 +25,18 @@ export interface PagesExposed {/** The pages of `sourceId` a remote credential wrote that name no readers (the first five by slug), or null when there are none. */export async function remotePagesFederationExposes(engine: SqlEngine, sourceId: string): Promise<PagesExposed | null> {const rows = await engine.executeRaw<{ slug: string; member: string | null; total: number }>(- `WITH writers AS (- SELECT 'legacy_token'::text AS kind, id::text AS pid, member_id FROM access_tokens- UNION ALL- SELECT 'oauth_client'::text, client_id, member_id FROM oauth_clients),- open AS (SELECT id, slug, revision_principal_kind, revision_principal_id FROM pages+ `WITH open AS (SELECT id, slug, revision_principal_kind, revision_principal_id FROM pagesWHERE source_id = $1 AND deleted_at IS NULL AND page_readers IS NULL AND folder_rule_ids IS NULL),authored AS (- SELECT o.slug, w.member_id FROM open o JOIN writers w ON w.kind = o.revision_principal_kind AND w.pid = o.revision_principal_id+ SELECT slug, revision_principal_kind AS kind, revision_principal_id AS pid FROM openUNION ALL- SELECT o.slug, w.member_id FROM open o JOIN page_versions v ON v.page_id = o.id- JOIN writers w ON w.kind = v.write_principal_kind AND w.pid = v.write_principal_id)- SELECT slug, min(member_id) AS member, (count(*) OVER ())::int AS total- FROM authored GROUP BY slug ORDER BY slug LIMIT 5`, [sourceId]);+ SELECT o.slug, v.write_principal_kind, v.write_principal_id FROM open o JOIN page_versions v ON v.page_id = o.id)+ SELECT a.slug, min(coalesce(t.member_id, c.member_id)) AS member, (count(*) OVER ())::int AS total+ FROM authored a+ LEFT JOIN access_tokens t ON a.kind = 'legacy_token' AND t.id::text = a.pid+ LEFT JOIN oauth_clients c ON a.kind = 'oauth_client' AND c.client_id = a.pid+ WHERE a.kind IN ('legacy_token', 'oauth_client')+ GROUP BY a.slug ORDER BY a.slug LIMIT 5`, [sourceId]);const first = rows[0];if (!first) return null;const bound = rows.find(r => r.member !== null && r.slug.includes('/'));diff --git a/test/sources-federate-member-pages.test.ts b/test/sources-federate-member-pages.test.tsindex f265ed73a6..a889a6c755 100644--- a/test/sources-federate-member-pages.test.ts+++ b/test/sources-federate-member-pages.test.ts@@ -6,7 +6,7 @@* unbound token then reads the member's page. The federate still happens (exit and config unchanged), and its* output now names the count, the page and its member, the check and restrict commands, on stderr and under* --json as pages_exposed; restore (unmanaged and managed) does the same. Pages with readers and operator pages- * are not named; a credential unbound since it wrote still is, and a hostile slug is shell-quoted.+ * are not named; a credential unbound or deleted since it wrote still is, and a hostile slug is shell-quoted.* 2. Fails when: federate prints no preview of the member's page (master), names a page with readers or an operator* page, drops an unbound credential's page, builds a command with an unquoted slug, warns on unfederate, or stops* federating.@@ -106,25 +106,30 @@ describe('sources federate previews member-written pages it exposes (#8982, pgliexpect(await federated()).toBe(true);}, 240_000);- test('a credential unbound since it wrote still counts; a hostile slug is shell-quoted in the commands', async () => {+ test('a credential unbound or deleted since it wrote still counts; a hostile slug is shell-quoted in the commands', async () => {await engine.executeRaw("UPDATE sources SET config = config || '{\"federated\": false}'::jsonb WHERE id = $1", [SRC]);const [plan] = await engine.executeRaw<{ slug: string; kind: string; pid: string }>(`SELECT slug, revision_principal_kind AS kind, revision_principal_id AS pidFROM pages WHERE source_id = $1 AND slug LIKE 'notes/alice-plan-%'`, [SRC]);- const rename = (from: string, to: string) => engine.transaction(async tx => {- await tx.executeRaw("SELECT set_config('gbrain.write_principal_kind', $1, true), set_config('gbrain.write_principal_id', $2, true)", [plan!.kind, plan!.pid]);+ const [operator] = await engine.executeRaw<{ kind: string; pid: string }>(`SELECT revision_principal_kind AS kind, revision_principal_id AS pid+ FROM pages WHERE source_id = $1 AND slug = 'notes/operator-page'`, [SRC]);+ const attribute = (from: string, to: string, kind: string, pid: string) => engine.transaction(async tx => {+ await tx.executeRaw("SELECT set_config('gbrain.write_principal_kind', $1, true), set_config('gbrain.write_principal_id', $2, true)", [kind, pid]);await tx.executeRaw('UPDATE pages SET slug = $3 WHERE source_id = $1 AND slug = $2', [SRC, from, to]);});+ const rename = (from: string, to: string) => attribute(from, to, plan!.kind, plan!.pid);await engine.executeRaw("UPDATE access_tokens SET member_id = NULL WHERE name = 'alice'");const hostile = 'notes/$(printf injected)/plan';await rename(plan!.slug, hostile);+ await attribute('notes/operator-page', 'notes/operator-page-moved', 'oauth_client', 'deleted-client-example');try {const out = await sources(['federate', SRC, '--json']);- expect(JSON.parse(out.stdout).pages_exposed).toEqual({ count: 1, examples: [{ slug: hostile, member: null }],+ expect(JSON.parse(out.stdout).pages_exposed).toEqual({ count: 2, examples: [{ slug: hostile, member: null }, { slug: 'notes/operator-page-moved', member: null }],check: "gbrain access who-can-read 'notes/$(printf injected)/plan' --source " + SRC,restrict: 'gbrain access rule add --prefix <folder/> --readers <member> --source ' + SRC });expect(out.stderr).toContain("'notes/$(printf injected)/plan' (credential bound to no member now)");} finally {await rename(hostile, plan!.slug);+ await attribute('notes/operator-page-moved', 'notes/operator-page', operator!.kind, operator!.pid);await engine.executeRaw("UPDATE access_tokens SET member_id = 'alice-example' WHERE name = 'alice'");}}, 240_000);