Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

test: use withEnv for the file-transport arms of the partial-clone tests (GST-2331)

garrytan5177b587e300
diff --git a/CHANGELOG.md b/CHANGELOG.md
index af378b78c2..ca57f381e2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,7 +16,7 @@ identifiers and attribution are available in the pre-removal Git revision
### Itemized changes
-- `src/core/hardened-git.ts`: `NO_LAZY_FETCH_ENV` is what a managed-sync read that runs under `gitChildEnv` adds beside `HARDENED_GIT_ARGS`: `GIT_NO_LAZY_FETCH=1` (Git 2.45+) and an empty `GIT_ALLOW_PROTOCOL`, which allows no transport on any Git version and takes precedence over an inherited allowlist and every `protocol.allow` setting.
+- `src/core/hardened-git.ts`: `NO_LAZY_FETCH_ENV` is what a managed-sync read that runs under `gitChildEnv` adds beside `HARDENED_GIT_ARGS`: `GIT_NO_LAZY_FETCH=1` (Git 2.45+) and an empty `GIT_ALLOW_PROTOCOL`, which allows no transport on any Git version.
- `syncGit` and `syncGitAsync` (`sync-discovery.ts`), every read in `sync-blobs.ts` (`ls-tree`, `cat-file --batch`), the `git show` fallback, the delta and working-tree diff (`sync-delta.ts`), the rename-identity `cat-file` stream, the metafile head reads, the origin-evidence history walk and ignore check, the held-deletion read, the unindexed-file listing and the pending-import checks now run with both.
- `src/core/persistence/sync-entry-fault.ts`: in a partial clone the `git_object_missing` hold names the partial clone and its fix is `git cat-file -p <blob>`; in any other checkout it still names `git fsck --connectivity-only`.
- A partial clone with no checkout (`--no-checkout`) holds every committed file `git_object_missing`. When rename detection needs an older blob the partial clone lacks, the sync compares the whole tree instead: the new path imports as a new page and the old page is deleted, so that rename does not keep its page.
diff --git a/docs/architecture/key-files/security.md b/docs/architecture/key-files/security.md
index 2e057ebffc..662b1f1914 100644
--- a/docs/architecture/key-files/security.md
+++ b/docs/architecture/key-files/security.md
@@ -40,6 +40,6 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/write-gate-scan.ts` + `scripts/trust-dry-run-scan.ts` — read-only exposure scan: keyset batches over pages, facts, takes and timeline entries, tier projected from deterministic signals (connector source, webhook/clipper page, `mcp:*`/capture/transcript page, agent fact lanes, `persistence_requests` operation and authority; none -> `unknown`), flag/quarantine counts and families per tier and table, counts and pattern names only. The script runs it in a REPEATABLE READ READ ONLY transaction (PGLite: on a copy of the data directory). Test: `test/write-gate-scan.test.ts`.
-- `src/core/git-env.ts` + `src/core/hardened-git.ts` — #7584, the operator's `safe.directory` in scrubbed Git environments: `hardenedGitEnvironment` still drops system and global config and every `GIT_*` variable, but carries the operator's `safe.directory` entries (system and global, read from `/` with `git config --includes --show-scope` and cached for 30 s, then `GIT_CONFIG_*`) as environment config, so a source repository owned by another user reads once the operator allows it. `gitChildEnv` carries the environment entries the same way (`envSafeDirectories`, `safeDirectoryEnv`). Every other key in those places stays out. `gitChildEnv` also drops Git's global pathspec switches (`GIT_PATHSPEC_ENV_VARS`: `GIT_LITERAL_PATHSPECS`, `GIT_GLOB_PATHSPECS`, `GIT_NOGLOB_PATHSPECS`, `GIT_ICASE_PATHSPECS`; `extra` can still set one). gbrain's writers mark each path literal with `literalPathspecs` (`:(literal)<path>`), never `git --literal-pathspecs`, which Git exports to every hook and child as `GIT_LITERAL_PATHSPECS=1`, where a hook's glob pathspec then matches nothing. GST-2331: `NO_LAZY_FETCH_ENV` (`GIT_NO_LAZY_FETCH=1` and an empty `GIT_ALLOW_PROTOCOL`) is what a managed-sync read that runs under `gitChildEnv` adds beside `HARDENED_GIT_ARGS`, so a partial clone fetches nothing (Git before 2.45 ignores the first; the empty allowlist allows no transport on any Git and takes precedence over an inherited allowlist and every `protocol.allow` setting, which `protocol.allow=never` alone does not): `syncGit`/`syncGitAsync`, every read in `sync-blobs.ts`, `sync-delta.ts`'s text runner (a rename whose old blob the clone lacks fails the diff, `diff_failed`, and discovery walks the whole tree), `sync-rename-identity.ts`, `sync-metafile-pages.ts`, `sync-origin-evidence.ts`, `sync-held-deletions.ts`, `unindexed-files.ts` and `pending-import.ts`. Tests: `test/hardened-git-safe-directory-7584.test.ts` (ownership forced with `GIT_TEST_ASSUME_DIFFERENT_OWNER` through a `git` wrapper on PATH), `test/git-literal-pathspec-hooks-7759.test.ts`, `test/sync-partial-clone-2331.test.ts`.
+- `src/core/git-env.ts` + `src/core/hardened-git.ts` — #7584, the operator's `safe.directory` in scrubbed Git environments: `hardenedGitEnvironment` still drops system and global config and every `GIT_*` variable, but carries the operator's `safe.directory` entries (system and global, read from `/` with `git config --includes --show-scope` and cached for 30 s, then `GIT_CONFIG_*`) as environment config, so a source repository owned by another user reads once the operator allows it. `gitChildEnv` carries the environment entries the same way (`envSafeDirectories`, `safeDirectoryEnv`). Every other key in those places stays out. `gitChildEnv` also drops Git's global pathspec switches (`GIT_PATHSPEC_ENV_VARS`: `GIT_LITERAL_PATHSPECS`, `GIT_GLOB_PATHSPECS`, `GIT_NOGLOB_PATHSPECS`, `GIT_ICASE_PATHSPECS`; `extra` can still set one). gbrain's writers mark each path literal with `literalPathspecs` (`:(literal)<path>`), never `git --literal-pathspecs`, which Git exports to every hook and child as `GIT_LITERAL_PATHSPECS=1`, where a hook's glob pathspec then matches nothing. GST-2331: `NO_LAZY_FETCH_ENV` (`GIT_NO_LAZY_FETCH=1` and an empty `GIT_ALLOW_PROTOCOL`) is what a managed-sync read that runs under `gitChildEnv` adds beside `HARDENED_GIT_ARGS`, so a partial clone fetches nothing (Git before 2.45 ignores the first; the empty allowlist allows no transport on any Git): `syncGit`/`syncGitAsync`, every read in `sync-blobs.ts`, `sync-delta.ts`'s text runner (a rename whose old blob the clone lacks fails the diff, `diff_failed`, and discovery walks the whole tree), `sync-rename-identity.ts`, `sync-metafile-pages.ts`, `sync-origin-evidence.ts`, `sync-held-deletions.ts`, `unindexed-files.ts` and `pending-import.ts`. Tests: `test/hardened-git-safe-directory-7584.test.ts` (ownership forced with `GIT_TEST_ASSUME_DIFFERENT_OWNER` through a `git` wrapper on PATH), `test/git-literal-pathspec-hooks-7759.test.ts`, `test/sync-partial-clone-2331.test.ts`.
- `src/core/outside-content.ts` — GBRA-221: `neutralizeOutsideText(text)`, the one neutralizer for text a third party wrote before it lands in a page body (Gmail/Calendar/Contacts renders in `google-render.ts`, GitHub `renderItemPage`/`renderRepoCard`, chat transcripts in `transcripts/render.ts`, `book-mirror` chapter output). It escapes every `gbrain<ws>:<ws><name>` marker token to `gbrain\:<name>` (case-insensitive), `[Source:` to `[Source\:`, and backslash-prefixes each timeline sentinel, bare `## Timeline`/`## History` heading, dated timeline row or heading, transcript message anchor and date heading, using the parsers' own predicates. Idempotent and readable; apply it per outside field (or a composed line holding one), never to a whole page that may hold gbrain's own fences or citations. Also owns `escapeAnchorLines` and `MESSAGE_ANCHOR_RE` (re-exported by `transcripts/render.ts`). New connectors use it.
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 6bfdfd4b0f..d4e14510b9 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1503,7 +1503,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.637.0', id: 'purge_keeps_committed_file_gst2201', text: 'The retention purge (the autopilot purge phase, the `purge` job and `gbrain pages purge-deleted`) never removes a soft-deleted page\'s file that is committed at its source\'s last synced commit or at HEAD. A page sync deleted whose file is back is left for the sync that restores it (counted as deferred); any other such page is removed database-only (receipt `write_through.skipped: committed_file`) and its file stays. A page whose file is gone, or a person\'s delete that never removed its file (for example because another page shared it), purges as before. A page purge (`gbrain delete <slug> --purge` and the retention purge) now also redacts the timeline text the page\'s `add_timeline_entry` receipts kept (the entry\'s summary and source, `bullet`), compacted ones included.' },
{ since: '0.60.647.0', id: 'remember_share_gst2243', text: 'A member credential\'s `remember` still saves privately on `memory/members/<member>` by default, and can now pass `share: true` to save the fact on its subject\'s page for everyone who reads the source, as every non-member caller does. A member\'s explicit `visibility: "world"`, which used to be saved as private without a word, is now refused (`invalid_params`, reason `member_share_required`) and nothing is saved; `share: true` with `visibility: "private"`, or onto a page whose facts are kept private, is refused too (`share_target_restricted` for the page). Only the brain host can withdraw a shared fact. With ambient writeback on, a member credential\'s instructions say to omit `visibility` and use `share: true` for team facts, and a credential that may name a member through `x-qm-actor` is told to do so on those calls and pass `visibility: "world"` on the rest. Every `remember` reply, for any caller, now names the `page` the fact went to and its `visibility`. QM\'s `memory_write` is unchanged.' },
{ since: '0.60.648.0', id: 'sync_filter_owned_hold_9024', text: 'A managed sync now holds a committed file that Git stores through a filter (a path whose `filter` attribute at the synced commit names a driver other than Git LFS, whether or not this checkout configures that driver, or a file starting with git-crypt\'s header) as `content_filter_owned`, and the rest of the source syncs. Before, such a file imported its stored bytes (git-crypt ciphertext, for one) as the page text, or a git-crypt file was held `content_null_bytes`. gbrain still never runs a repository\'s filter programs (or `core.fsmonitor`) during sync, and reading the filter attribute fetches nothing, so a partial clone that lacks a `.gitattributes` that applies to the file holds it `git_object_missing`; the hold names git-crypt when the header matches and says to leave the file out with `sync.exclude` or commit it without the filter, and for a plain-text file under a filter attribute nobody uses, to remove the attribute and run `gbrain sources retry-held`. A page an older version already imported from such a file is left as it is until the file changes; then the file is held. With `sync.holds=fail`, and on a company-brain source (which never holds), the run stops with `content_filter_owned`. Classic (unmanaged) sync is unchanged.' },
- { since: 'TBD', id: 'sync_reads_fetch_nothing_gst2331', text: 'A managed sync now makes no object fetches in a partial clone (`git clone --filter`): every Git read it makes runs with Git\'s transports and lazy fetching switched off, whatever the environment or Git config allows. A committed file whose blob the partial clone never fetched imports from the working tree when those bytes are the committed blob, and is otherwise held `git_object_missing` with a message that names the partial clone and the `git cat-file -p <blob>` that fetches it; a partial clone with no checkout holds every committed file this way. A blobless clone with a normal checkout syncs its files as before, but an older object it never fetched is read only once it is fetched: when rename detection needs an older blob the partial clone lacks, the sync compares the whole tree instead (the new path imports as a new page and the old page is deleted, so that rename does not keep its page), and folder rules wait as below. A treeless clone (`--filter=tree:0`) that lacks a tree of the synced commit stops the run with `git_object_missing`, naming the partial clone and the `git ls-tree -r -d <commit>` that fetches the trees, and syncs or deletes nothing until then. A source whose commit names a folder-rule file (`.gbrain/access.yml`, `.gbrain/access.approved`) this clone has not fetched, or whose rule history a rebuild cannot read (a first sync replays every earlier version of `access.yml`), reads as nobody to remote members until the objects are fetched, the same as the shallow-checkout rule; its problem names the exact command that fetches them (`git -C <root> cat-file -p <rev>:./<path>` for a file, `git -C <root> log --first-parent -p <rev> -- .gbrain/access.yml` for the history), and the next sync after that command applies the rules and clears the mark with no other step. Before, the sync fetched a missing object from the partial clone\'s remote.' },
+ { since: 'TBD', id: 'sync_reads_fetch_nothing_gst2331', text: 'A managed sync now makes no object fetches in a partial clone (`git clone --filter`): every Git read it makes runs with Git\'s transports and lazy fetching switched off. A committed file whose blob the partial clone never fetched imports from the working tree when those bytes are the committed blob, and is otherwise held `git_object_missing` with a message that names the partial clone and the `git cat-file -p <blob>` that fetches it; a partial clone with no checkout holds every committed file this way. A blobless clone with a normal checkout syncs its files as before, but an older object it never fetched is read only once it is fetched: when rename detection needs an older blob the partial clone lacks, the sync compares the whole tree instead (the new path imports as a new page and the old page is deleted, so that rename does not keep its page), and folder rules wait as below. A treeless clone (`--filter=tree:0`) that lacks a tree of the synced commit stops the run with `git_object_missing`, naming the partial clone and the `git ls-tree -r -d <commit>` that fetches the trees, and syncs or deletes nothing until then. A source whose commit names a folder-rule file (`.gbrain/access.yml`, `.gbrain/access.approved`) this clone has not fetched, or whose rule history a rebuild cannot read (a first sync replays every earlier version of `access.yml`), reads as nobody to remote members until the objects are fetched, the same as the shallow-checkout rule; its problem names the exact command that fetches them (`git -C <root> cat-file -p <rev>:./<path>` for a file, `git -C <root> log --first-parent -p <rev> -- .gbrain/access.yml` for the history), and the next sync after that command applies the rules and clears the mark with no other step. Before, the sync fetched a missing object from the partial clone\'s remote.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/test/sync-partial-clone-2331.test.ts b/test/sync-partial-clone-2331.test.ts
index bd3dbf1e00..685db79759 100644
--- a/test/sync-partial-clone-2331.test.ts
+++ b/test/sync-partial-clone-2331.test.ts
@@ -11,6 +11,7 @@ import { OperationError } from '../src/core/ops/contract.ts';
import { managedBrain } from './helpers/managed-brain.ts';
import { persistencePostgresTemplate } from './helpers/persistence-postgres.ts';
import { testBackends } from './helpers/test-backends.ts';
+import { withEnv } from './helpers/with-env.ts';
// GST-2331: in a partial clone, a managed sync read of an object the repository lacks fetches nothing, so the program the
// checkout's config names for object fetches (remote.origin.uploadpack) never runs. The file is held git_object_missing
@@ -35,12 +36,6 @@ const rules = async (engine: BrainEngine) => (await engine.executeRaw<{ rule_id:
const present = (root: string, spec: string) => { execFileSync('git', ['-C', root, 'cat-file', '-e', spec], { env: { ...process.env, GIT_NO_LAZY_FETCH: '1' }, stdio: 'ignore' }); return true; };
/** Runs the git command a folder-rule problem names, exactly as written there. */
const runNamed = (message: string, shape: RegExp) => { const [command, ...args] = message.match(shape)![0].split(' '); execFileSync(command!, args, { stdio: 'ignore' }); };
-/** Runs `body` with GIT_ALLOW_PROTOCOL=file in this process, as an operator's environment may carry it. */
-async function allowingFile<T>(body: () => Promise<T>): Promise<T> {
- const prior = process.env.GIT_ALLOW_PROTOCOL;
- process.env.GIT_ALLOW_PROTOCOL = 'file';
- try { return await body(); } finally { if (prior === undefined) delete process.env.GIT_ALLOW_PROTOCOL; else process.env.GIT_ALLOW_PROTOCOL = prior; }
-}
/** Upstream with notes under hr/ and a folder-rule file, served bare and cloned blobless with a checkout into `root`. */
function rulesClone(root: string, versions: string[][]): ReturnType<typeof server> {
const upstream = join(root, '..', 'upstream');
@@ -181,7 +176,7 @@ for (const backend of testBackends()) describe('[' + backend + '] managed sync i
writeFileSync(join(root, 'notes/missing.md'), note('missing, edited in the checkout'));
git(root, 'update-index', '--assume-unchanged', 'notes/missing.md');
const restoreWindow = __setPinnedWindowForTests(arm.window), restoreLazy = __setNoLazyFetchForTests(arm.lazyOff);
- try { await (arm.allowFile ? allowingFile(() => performManagedSync(engine, OPTS)) : performManagedSync(engine, OPTS)); } finally { restoreLazy(); restoreWindow(); }
+ try { await (arm.allowFile ? withEnv({ GIT_ALLOW_PROTOCOL: 'file' }, () => performManagedSync(engine, OPTS)) : performManagedSync(engine, OPTS)); } finally { restoreLazy(); restoreWindow(); }
expect(existsSync(marker)).toBe(false);
const held = await holds(engine);
expect(held.map(hold => [hold.path, hold.code])).toEqual([['notes/missing.md', 'git_object_missing']]);
@@ -329,7 +324,7 @@ for (const backend of testBackends()) describe('[' + backend + '] managed sync i
let remote: ReturnType<typeof server> | undefined;
return brain(async ({ root }) => {
__resetHistoryIndexesForTest();
- const evidence = await allowingFile(() => readOriginEvidence(root, [git(root, 'rev-parse', 'HEAD')], ['notes/a.md']));
+ const evidence = await withEnv({ GIT_ALLOW_PROTOCOL: 'file' }, () => readOriginEvidence(root, [git(root, 'rev-parse', 'HEAD')], ['notes/a.md']));
expect(existsSync(remote!.marker)).toBe(false);
expect(evidence.get('notes/a.md')).toBe('unknown');
}, root => { remote = bloblessClone(root, ['seed', 'a'], false, 'tree:0'); });