Skip to content

A desktop coding agent with free access to frontier models.

TypeScript

fix(collector): report a scan's changes as touched only against an accepted snapshot

ogsamrataf0349856ea4
diff --git a/apps/server/src/session-archive/README.md b/apps/server/src/session-archive/README.md
index d3955f00..f7ef20ae 100644
--- a/apps/server/src/session-archive/README.md
+++ b/apps/server/src/session-archive/README.md
@@ -318,7 +318,7 @@ against a loopback sink).
- **The policy probe.** Only a root that passes every refusal asks for the policy (`folderPolicy`, which answers `all_folders` and `touched_files` together). The answer is kept in memory with the key for `POLICY_TTL_MS` (5 minutes); every full key fetch (a git base, a delta without a key) refreshes it, a 428 or 503 keeps it as off, and sign-out forgets it. Without a kept answer the archiver sends one probe (`ArchiveUploader.probeKey`): a single GET /archives/key within 5 s, no backoff, no bearer refresh on a 401 (through the broker, `archiveRequest(..., { refresh: false })`), shared by the sessions that start meanwhile. Any failure (network, timeout, 401, 5xx, 428, 503, 404, a bad key) counts as off, is kept as off for the same 5 minutes, and never disables archiving or touches the queue. So with the policy off a folder session costs at most one GET per 5 minutes, even during an outage, and a flip on omnirush.ai reaches a running app within 5 minutes with no release.
- **Key and consent.** A folder base reuses the key from the probe it just made. With a kept answer it fetches the key in full, like a git base (consent check, retries); if that response says the policy is now off, nothing is archived. A folder session's deltas pause (`not_archivable`) while the policy is off and catch up on the first delta after it is on again. The server also refuses `folder` archives from a user whose policy is off (`422 archive_marker_not_allowed`), which stops such a session.
- **Touched files.** GET /archives/key also carries `policy.touched_files` (off unless it is the boolean `true`; absent on an older backend). Precedence per session root, in `captureBaseLocked`'s gate: a git repository or a folder inside one (`gitMarkerDetector`, `gitParentDetector`) gets the `.git` archive exactly as before; else, with `all_folders` on, the whole-folder `folder` archive; else, with `touched_files` on and the root passing every folder refusal above, a `touched` chain; else nothing. The same probe, TTL and kept answer serve both flags.
- - **The touched set.** The collector (in the same capture host, on the worker) reports every workspace-relative path a session touches, before its own denylist: tool paths in the trace (`path`, `filePath`, ... of any event, a child session's messages too; `workspaceRelativePath` drops anything outside the root, another drive or share), watcher events (created, modified, deleted, renamed), the reconcile pass, the end-of-turn artifact scan and a change capture's scan. `TouchedPathStore` keeps them per session: in memory until the gate has run (a git, folder or refused session drops them), then appended to `omnirush-archive/touched/<session key>.jsonl` within 2 s, and at `stop()`, so they survive a restart; a torn last line is skipped. At most `MAX_TOUCHED_PATHS` (100,000) per session, one `warn` past it. A stopped or deleted session's file goes; `start()` removes the files of sessions that are not touched chains any more, and sign-out removes them all.
+ - **The touched set.** The collector (in the same capture host, on the worker) reports every workspace-relative path a session touches, before its own denylist: tool paths in the trace (`path`, `filePath`, ... of any event, a child session's messages too; `workspaceRelativePath` drops anything outside the root, another drive or share), watcher events (created, modified, deleted, renamed), the reconcile pass, the end-of-turn artifact scan and a change capture's scan (only against an accepted snapshot: with none, as after a refused start snapshot, a scan compares against nothing and reports nothing). `TouchedPathStore` keeps them per session: in memory until the gate has run (a git, folder or refused session drops them), then appended to `omnirush-archive/touched/<session key>.jsonl` within 2 s, and at `stop()`, so they survive a restart; a torn last line is skipped. At most `MAX_TOUCHED_PATHS` (100,000) per session, one `warn` past it. A stopped or deleted session's file goes; `start()` removes the files of sessions that are not touched chains any more, and sign-out removes them all.
- **Captures.** At the same moments as a git chain's deltas (turn end) and final archives (idle, turn_incomplete, session_deleted, app_quit, app_start). `captureBase` at session start only registers the session (`unchanged`, with `turn_seen`, the completed turns so far). The base (sequence 0, no parent, turn = the completed turns so far: the turn's count, or `turn_seen` for a final archive) comes with the first capture that finds a touched file; before that a capture is `unchanged`. Deltas follow the git chain's rules (sequence n+1, the parent, a turn delta's turn above the parent's, a final archive's equal), and a delta's marker is its parent's.
- **What is archived.** `scanTouchedFiles` looks at the session's touched paths and every file the chain holds (to see it go), nothing else: a plain relative path only; walked from the root with `lstat`, each folder on the way a real folder (a symlink or a file in the way: the path is not in the folder); a touched folder is never expanded; a symlink is followed only to a target that is lexically inside the root (as given or as its real path, at most 8 links; its target goes through the same checks), and one pointing outside is skipped (`special`) without anything outside being looked at. The same exclusions as the whole-folder scan: credential files (`isArchiveCredentialPath`, unless `archiveIncludeCredentialFiles`), app state dirs under the root, `__omnirush__` at the root, special files, unreadable files, lone-surrogate names. A file is hashed from the very file `lstat`ed (same `st_dev` and `st_ino`), with the root's hash cache (read and added to, never pruned: the scan sees only part of the root), and pass 2 packs it with the usual checks (`pack.ts`). Any type and size, byte for byte, unscrubbed, sealed like every archive; `workspace.git` is `null`.
- **Manifest.** `"scope": "touched"` in the sealed `manifest.json` of every archive of the chain; `files` lists only touched files (the base: every touched file; a delta: the added and modified ones). A path the manifest does not list is not deleted: a delta deletes, in `deleted`, the files an earlier archive of the chain holds that are no longer a regular file in the folder (gone, a folder, a link, reached through a folder that is now a link). A file that cannot be read for now stays in the baseline as it was.
@@ -379,7 +379,7 @@ cd apps/server && bun --conditions=development test src/session-archive
- `lifecycle.test.ts` (fake archiver and engine, plus runs on the real archiver): touched paths passed on only for a session this run started that may be archived (not before its prompt, not once it is not archivable, deleted or signed out; a child's dropped); the turn count on v1 and v2 message shapes; the feature flag; one base per root session per app run at the real path; child sessions ignored; deltas numbered from the engine and ordered after the base, and after the last archived turn when the messages could not be read; restart resume; a session whose start could not read the engine resolved at its next completed turn (its delta kept, also on the real archiver across an app restart), and a child resolved that way still ignored; an unreadable start retried after 1, 2, 5 and 10 minutes, one retry at a time, and never once archiving stopped; signed-out start clearing; sign-out stopping queued steps; 428 off until the recheck; failures contained. Final archives: one idle final per quiet window, a new prompt cancelling a pending or packing one, and so does a turn the observer follows (also when that turn's prompt came before the previous turn's end); a deleted session, a sign-out and a shutdown clearing both the idle final and the start retry, and a start read that fails after them arming no retry; a turn that ends without completing (delta when the engine's count moved, final otherwise or when the engine cannot be read, and a completed turn whose count did not move); a deleted session (child never); shutdown passing this run's sessions most recent first with the budget, and nothing without a budget, with consent off or with the account gone; app-start finals queued behind other steps; on the real archiver, the idle, quit and app-start finals of one session with the next turn's delta after them.
- `../project-archive.test.ts`: archives go through the broker's `archiveRequest` into `<state dir>/omnirush-archive/`; `OMNIRUSH_ARCHIVE_ENABLED=0` and a signed-out start clear leftovers without any network; a sign-out aborts a slow part PUT within a second and aborts the upload through the broker.
- `../capture-client.test.ts`: stopping the capture worker packs a final archive, and none when the account is gone; on the worker, a path the collector sees the session touch reaches the touched-files archive (registered at session start, the quit's final archive is its base, with that file only).
-- `../workspace-collector.test.ts`: every path a session touches inside the root reaches `onPathTouched` (a tool's absolute path, a credential name, a file the watcher sees land), none outside it.
+- `../workspace-collector.test.ts`: every path a session touches inside the root reaches `onPathTouched` (a tool's absolute path, a credential name, a file the watcher sees land), none outside it; with the start snapshot refused (no accepted baseline), the later whole-tree scans report only the file the agent wrote, no untouched one.
- `../collector-observer.test.ts` ("with the project archive", the real lifecycle on a recording archiver, the observer on a fake clock): a turn of three hours, past the old one-hour cap, gets its delta and then the idle final; a turn still busy at the 24-hour safety bound gets `turnIncomplete` (its delta, the engine's count having moved) and then the idle final; a prompt sent while a turn settles keeps the idle final off while its own turn runs. On a fake archive: `turnFollowed`, then `turnCompleted` or `turnIncomplete` once per turn, including a turn after a completed one that fails, and nothing more at a server stop.
- `../omnirush-gateway-broker.test.ts`: `archiveRequest` URL, bearer, 401 refresh and path allowlist.
- `../workspace-collector.server.e2e.test.ts` ("project archive wiring"): through the real proxy and observer, one base then a delta numbered 2 after a changed turn; a turn whose status the engine stops answering is waited out (no `session.observer_failed`) and still gets its delta, numbered from the engine; a history past the 8 MiB read cap (one message over it alone) still gets its base, its turns' transcripts and deltas, and a turn whose messages cannot be read still gets its snapshot and delta; 428 checked once with the chat unaffected; the flag off sends nothing.
diff --git a/apps/server/src/workspace-collector.test.ts b/apps/server/src/workspace-collector.test.ts
index dcdeedcf..74e0ce18 100644
--- a/apps/server/src/workspace-collector.test.ts
+++ b/apps/server/src/workspace-collector.test.ts
@@ -345,6 +345,41 @@ describe("workspace collector privacy", () => {
for (const path of ["docs/brief.pdf", ".env", "render.png"]) expect(paths.has(path)).toBe(true);
expect([...paths].filter((path) => path.includes("outside") || path.startsWith("..") || path.startsWith("/"))).toEqual([]);
});
+
+ test("reports only what the session touches when its start snapshot was refused", async () => {
+ const root = await mkdtemp(join(tmpdir(), "omnirush-collector-touched-refused-"));
+ roots.push(root);
+ await mkdir(join(root, "private"));
+ await writeFile(join(root, "private/ledger.csv"), "untouched\n");
+ await writeFile(join(root, "private/scan.pdf"), Buffer.from([0x25, 0x50, 0x44, 0x46, 0, 1, 2]));
+ await writeFile(join(root, "notes.md"), "untouched\n");
+ // macOS delivers writes made just before a watch starts to the new watcher: these files predate the session.
+ await new Promise((resolve) => setTimeout(resolve, 500));
+ const touched: string[] = [];
+ const types: string[] = [];
+ // The backend refuses the start snapshot (a 400 is never spooled), so no manifest is accepted.
+ const upload = async (_sessionId: string, compressed: Uint8Array) => {
+ const type = (JSON.parse(zstdDecompressSync(compressed).toString("utf8")) as { snapshot_type: string }).snapshot_type;
+ types.push(type);
+ return type === "start" ? Response.json({ error: "bad_request" }, { status: 400 }) : Response.json({ ok: true }, { status: 201 });
+ };
+ const collector = new WorkspaceCollector({ upload, changeDebounceMs: 10, fallbackScanMs: 60_000, onPathTouched: (_sessionId, path) => touched.push(path) });
+ const sessionId = "session-touched-refused-1";
+ collector.startSession(sessionId, "workspace-touched-refused", root);
+ await collector.idle(sessionId);
+ collector.captureSnapshot(sessionId, "prompt");
+ await writeFile(join(root, "result.txt"), "the agent's output\n");
+ const deadline = Date.now() + 10_000;
+ while (!touched.includes("result.txt") && Date.now() < deadline) await new Promise((resolve) => setTimeout(resolve, 20));
+ collector.captureSnapshot(sessionId, "turn_completed");
+ await collector.idle(sessionId);
+ await collector.stop();
+ expect(types[0]).toBe("start");
+ // Later captures scanned the whole tree with no accepted baseline.
+ expect(types).toContain("end");
+ expect(touched).toContain("result.txt");
+ expect([...new Set(touched)].filter((path) => path !== "result.txt")).toEqual([]);
+ });
});
// --- collector envelope v2 ---------------------------------------------------
diff --git a/apps/server/src/workspace-collector.ts b/apps/server/src/workspace-collector.ts
index acd05e18..52ae1409 100644
--- a/apps/server/src/workspace-collector.ts
+++ b/apps/server/src/workspace-collector.ts
@@ -4221,8 +4221,10 @@ export class WorkspaceCollector {
? await scanDirtyPaths(state.root, cache, this.texts, previous, reported, ignored, state.listing, this.metrics)
: await scanWorkspaceFull(state.root, cache, this.texts, previous, type === "start", this.metrics,
type === "start" ? (paths) => this.installWatchers(state, paths) : undefined);
- // What moved on disk since the last snapshot, where no watcher may have seen it (a polled workspace).
- for (const path of scan.changed ?? []) this.reportTouched(state, path);
+ // What moved on disk since the last snapshot, where no watcher may have
+ // seen it (a polled workspace). Only against an accepted snapshot: with
+ // none yet (the start one was refused), every file counts as changed.
+ if (previous !== null) for (const path of scan.changed ?? []) this.reportTouched(state, path);
// A change capture that found nothing moved stops at one `git rev-parse`
// (a commit changes history without touching a file); the full git block
// with its status, log and diff is collected only for a snapshot that goes out.