The open-source app everyone uses to manage agents at work
TypeScript acpxagent-orchestrationagent-skillsagentic-aiai-agentsai-engineeringai-governanceai-toolsautonomous-agentsclaudeclaude-codeclaude-skillscodexhermesllmmcpmulti-agentopenclawproject-managementworkflow-automation
fix(server): compact finished task worktrees and shorten the reaper cooldown (#16304)
Devin Foley5b29b9062e9b
diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.mdindex 3f960499f7..f5b7edaa27 100644--- a/doc/DEVELOPING.md+++ b/doc/DEVELOPING.md@@ -2007,9 +2007,16 @@ Environment overrides:- `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=<days>` sets how long theterminal-workspace reaper waits after an issue tree becomes terminal before itarchives the execution workspace and deletes the worktree. A person can reopen- the work inside this window. The default is `7`. A value of `0` disables the+ the work inside this window. The default is `1`. A value of `0` disables thecooldown and restores immediate reaping. A negative or non-numeric value fallsback to the default.+- `PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS=<hours>` sets how long the reaper+ waits after an issue tree becomes terminal before it removes regenerable files+ from the workspace (see "Terminal workspace compaction" below). The default is+ `1`. A value of `0` compacts on the next sweep. A negative, empty, or+ non-numeric value falls back to the default.+- `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. Any other+ value keeps it on.Without `PAPERCLIP_DB_BACKUP_ALERT_FILE`, health checks look for`db-backup-to-s3.failure` in the backup directory, beside the backup directory,@@ -2319,6 +2326,55 @@ fields before disabling a connection. Verify workspace ownership, active runs,Git state, and runtime-service readiness before closing a workspace. A missingURL or old workspace timestamp alone does not prove that a row is disposable.+### Terminal workspace compaction++A finished task often keeps its worktree for a long time: the reaper archives a+worktree only when its delivery is merged and its files are clean, and then only+after the cooldown. Most of a worktree's size is installed dependencies and build+output, not work. Compaction removes those files and keeps everything else, so a+finished task does not hold gigabytes while it waits for the reaper (or forever,+when it has no merged pull request or has leftover files).++The reaper compacts a workspace once per terminal cycle, after+`PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS`, when all of these hold:++- the workspace is an isolated linked Git worktree (never a shared workspace or a+ project's main checkout);+- every issue in its tree is done or cancelled;+- no queued or running heartbeat run works on the tree or the workspace;+- no runtime service runs from the workspace.++Compaction removes only directories that have a regenerable name+(`node_modules`, `dist`, `build`, `coverage`, `storybook-static`, `target`,+`.cache`, `.next`, `.nuxt`, `.parcel-cache`, `.svelte-kit`, `.turbo`, `.vite`),+that the repository's own `.gitignore` files ignore (a negated pattern, a global+excludes file, or `.git/info/exclude` does not count), and that hold no tracked+file. A directory with such a name+that does not qualify (for example a tracked `build/`) is searched in turn, so+ignored dependencies inside it still go. It never enters `.git` or+`.paperclip`, and it never follows a symlink. It checks each directory's ignore+status on its own, so dependencies below an ignored parent (for example+`vendor/tool/node_modules`) are found. Its Git reads have a deadline and stop+when the sweep stops. Tracked files, uncommitted changes,+untracked files, other ignored files (such as `.env` or local data), the branch,+and its commits all stay. The workspace row stays active.++The removal is a rename into `.paperclip-compaction-trash` beside the worktree,+made while the reaper holds the workspace lifecycle lock. The reaper checks the+conditions above before the rename and again after it, in the same transaction.+If the task reopened or a run started in between, the directories move back.+Otherwise the trash is deleted. Trash that an interrupted compaction or a failed+delete leaves behind is deleted by a later sweep once it is more than an hour+old. The sweep checks each worktree parent directory at most every 30 minutes,+whether or not anything is left to compact.++A compacted worktree is in the same state as a freshly created one, so when the+task resumes, whatever produced those files the first time produces them again:+reuse runs the workspace provision command (or `scripts/provision-worktree.sh`),+which reinstalls dependencies when `node_modules` is missing. Repositories without+a provision command rely on the agent's own install step, as they do for a new+worktree.+### Browser realtime connection recoveryIf a browser cannot construct a WebSocket, the live-update and run transcriptdiff --git a/doc/SPEC-implementation.md b/doc/SPEC-implementation.mdindex 48f9182580..804e488aca 100644--- a/doc/SPEC-implementation.md+++ b/doc/SPEC-implementation.md@@ -1355,6 +1355,8 @@ The current app also exposes V1-supporting surfaces for:derive `deliveryState` as `merged_via_pr | merged_by_ancestry | unmerged | unknown`; terminal issue trees with amerged delivery and no active checkout run become cleanup-eligible with reason `issue_terminal` and are archivedthrough the workspace cleanup path. Reopening the source issue records activity but does not restore that workspace.+ Before that, terminal isolated worktrees are compacted: ignored, untracked dependency, build, and cache directories+ are removed while the workspace, its branch, and all source changes stay, and reuse regenerates them.- task watchdog configuration and reusable watchdog issue orchestration for explicitly watched issue subtrees- routines and scheduled/API/webhook triggers- plugin installation, configuration, state, jobs, logs, webhooks, and plugin database namespace migrationdiff --git a/server/src/__tests__/execution-workspace-compaction.test.ts b/server/src/__tests__/execution-workspace-compaction.test.tsnew file mode 100644index 0000000000..b929a6aef0--- /dev/null+++ b/server/src/__tests__/execution-workspace-compaction.test.ts@@ -0,0 +1,504 @@+import { EMBEDDED_POSTGRES_TEST_TIMEOUT_MS } from "@paperclipai/db";+import { execFile } from "node:child_process";+import { randomUUID } from "node:crypto";+import fs from "node:fs/promises";+import os from "node:os";+import path from "node:path";+import { promisify } from "node:util";+import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";+import { and, eq } from "drizzle-orm";+import {+ activityLog,+ agents,+ companies,+ createDb,+ executionWorkspaces,+ heartbeatRuns,+ issues,+ projectWorkspaces,+ projects,+ workspaceRuntimeServices,+} from "@paperclipai/db";+import {+ getEmbeddedPostgresTestSupport,+ startEmbeddedPostgresTestDatabase,+} from "./helpers/embedded-postgres.js";+import {+ executionWorkspaceService,+ type ExecutionWorkspaceServiceOptions,+} from "../services/execution-workspaces.ts";+import { realizeExecutionWorkspace } from "../services/workspace-runtime.ts";+import { WORKSPACE_COMPACTION_TRASH_DIR } from "../services/workspace-compaction.ts";++const execFileAsync = promisify(execFile);+const HOUR_MS = 60 * 60 * 1000;++const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();+const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;++if (!embeddedPostgresSupport.supported) {+ console.warn(+ `Skipping embedded Postgres workspace compaction tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,+ );+}++async function git(cwd: string, args: string[]) {+ const output = await execFileAsync("git", ["-C", cwd, ...args], { cwd });+ return output.stdout.trim();+}++async function exists(filePath: string) {+ return fs.lstat(filePath).then(() => true, () => false);+}++async function writeFile(filePath: string, contents: string) {+ await fs.mkdir(path.dirname(filePath), { recursive: true });+ await fs.writeFile(filePath, contents, "utf8");+}++async function createRepo() {+ const repoRoot = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-compaction-")));+ await git(repoRoot, ["init"]);+ await git(repoRoot, ["config", "user.name", "Paperclip Test"]);+ await git(repoRoot, ["config", "user.email", "[email protected]"]);+ await writeFile(path.join(repoRoot, "README.md"), "# Test repo\n");+ await writeFile(+ path.join(repoRoot, ".gitignore"),+ ["node_modules/", "dist/", ".env", ".paperclip/", "local-data/", ""].join("\n"),+ );+ await git(repoRoot, ["add", "README.md", ".gitignore"]);+ await git(repoRoot, ["commit", "-m", "Initial commit"]);+ await git(repoRoot, ["branch", "-M", "main"]);+ return repoRoot;+}++// Files a finished task leaves behind. The regenerable directories are the+// only paths compaction may remove; everything else is work or data.+const REGENERABLE_PATHS = ["node_modules", "packages/app/node_modules", "dist"];+const PRESERVED_PATHS = [+ "notes/findings.md",+ ".env",+ "local-data/state.json",+ ".paperclip/instances/default/db.txt",+];++async function populateWorktree(worktreePath: string) {+ await writeFile(path.join(worktreePath, "node_modules/left-pad/index.js"), "module.exports = 1;\n");+ await writeFile(path.join(worktreePath, "packages/app/node_modules/dep/index.js"), "module.exports = 2;\n");+ await writeFile(path.join(worktreePath, "dist/server.js"), "built\n");+ await writeFile(path.join(worktreePath, "notes/findings.md"), "untracked agent notes\n");+ await writeFile(path.join(worktreePath, ".env"), "TOKEN=local\n");+ await writeFile(path.join(worktreePath, "local-data/state.json"), "{}\n");+ await writeFile(path.join(worktreePath, ".paperclip/instances/default/db.txt"), "instance data\n");+ await fs.appendFile(path.join(worktreePath, "README.md"), "uncommitted edit\n");+}++describeEmbeddedPostgres("terminal workspace compaction", () => {+ let db!: ReturnType<typeof createDb>;+ let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;+ const tempDirs = new Set<string>();++ beforeAll(async () => {+ tempDb = await startEmbeddedPostgresTestDatabase("paperclip-workspace-compaction-");+ db = createDb(tempDb.connectionString);+ }, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);++ afterEach(async () => {+ await db.delete(workspaceRuntimeServices);+ await db.delete(activityLog);+ await db.update(issues).set({ checkoutRunId: null, executionRunId: null, parentId: null });+ await db.delete(issues);+ await db.delete(executionWorkspaces);+ await db.delete(projectWorkspaces);+ await db.delete(projects);+ await db.delete(heartbeatRuns);+ await db.delete(agents);+ await db.delete(companies);+ for (const dir of tempDirs) await fs.rm(dir, { recursive: true, force: true });+ tempDirs.clear();+ });++ afterAll(async () => {+ await tempDb?.cleanup();+ });++ function service(options: ExecutionWorkspaceServiceOptions = {}) {+ return executionWorkspaceService(db, {+ resolvePullRequestDetails: async () => ({ state: "unknown", headRef: null, headSha: null }),+ workspaceReaperCooldownDays: 0,+ workspaceCompactionDelayMs: 0,+ ...options,+ });+ }++ async function seedCompany() {+ const companyId = randomUUID();+ const projectId = randomUUID();+ const agentId = randomUUID();+ const issuePrefix = `C${companyId.slice(0, 7).toUpperCase()}`;+ await db.insert(companies).values({+ id: companyId,+ name: "Paperclip",+ issuePrefix,+ requireBoardApprovalForNewAgents: false,+ });+ await db.insert(projects).values({ id: projectId, companyId, name: "Compaction", status: "in_progress" });+ await db.insert(agents).values({+ id: agentId,+ companyId,+ name: "Coder",+ role: "engineer",+ status: "active",+ adapterType: "codex_local",+ adapterConfig: {},+ runtimeConfig: {},+ permissions: {},+ });+ return { companyId, projectId, agentId, issuePrefix };+ }++ async function seedWorkspace(options: {+ worktreePath?: string;+ repoRoot?: string;+ commitOnBranch?: boolean;+ populate?: boolean;+ terminalAgoMs?: number;+ mode?: "isolated_workspace" | "shared_workspace";+ } = {}) {+ const repoRoot = options.repoRoot ?? await createRepo();+ tempDirs.add(repoRoot);+ const branchName = `PAP-${Math.floor(Math.random() * 100000)}-finished-task`;+ let worktreePath = options.worktreePath;+ if (!worktreePath) {+ worktreePath = path.join(repoRoot, ".paperclip", "worktrees", branchName);+ await git(repoRoot, ["worktree", "add", "-b", branchName, worktreePath, "main"]);+ }+ if (options.commitOnBranch) {+ await writeFile(path.join(worktreePath, "src/feature.ts"), "export const unmerged = true;\n");+ await git(worktreePath, ["add", "src/feature.ts"]);+ await git(worktreePath, ["commit", "-m", "Unmerged work"]);+ }+ if (options.populate !== false) await populateWorktree(worktreePath);++ const company = await seedCompany();+ const projectWorkspaceId = randomUUID();+ const executionWorkspaceId = randomUUID();+ const sourceIssueId = randomUUID();+ await db.insert(projectWorkspaces).values({+ id: projectWorkspaceId,+ companyId: company.companyId,+ projectId: company.projectId,+ name: "Primary",+ sourceType: "local_path",+ isPrimary: true,+ cwd: repoRoot,+ });+ await db.insert(executionWorkspaces).values({+ id: executionWorkspaceId,+ companyId: company.companyId,+ projectId: company.projectId,+ projectWorkspaceId,+ mode: options.mode ?? "isolated_workspace",+ strategyType: "git_worktree",+ name: branchName,+ status: "active",+ cwd: worktreePath,+ providerRef: worktreePath,+ providerType: "git_worktree",+ baseRef: "main",+ branchName,+ });+ await db.insert(issues).values({+ id: sourceIssueId,+ companyId: company.companyId,+ projectId: company.projectId,+ identifier: `${company.issuePrefix}-1`,+ title: "Finished task",+ status: "done",+ priority: "medium",+ completedAt: new Date(Date.now() - (options.terminalAgoMs ?? 2 * HOUR_MS)),+ executionWorkspaceId,+ });+ await db+ .update(executionWorkspaces)+ .set({ sourceIssueId })+ .where(eq(executionWorkspaces.id, executionWorkspaceId));+ return { ...company, repoRoot, worktreePath, branchName, executionWorkspaceId, sourceIssueId };+ }++ async function workspaceStatus(executionWorkspaceId: string) {+ const [row] = await db+ .select({ status: executionWorkspaces.status })+ .from(executionWorkspaces)+ .where(eq(executionWorkspaces.id, executionWorkspaceId));+ return row?.status ?? null;+ }++ async function expectRegenerablePresent(worktreePath: string, present: boolean) {+ for (const relativePath of REGENERABLE_PATHS) {+ expect(await exists(path.join(worktreePath, relativePath)), relativePath).toBe(present);+ }+ }++ async function expectWorkPreserved(worktreePath: string) {+ for (const relativePath of PRESERVED_PATHS) {+ expect(await exists(path.join(worktreePath, relativePath)), relativePath).toBe(true);+ }+ expect(await fs.readFile(path.join(worktreePath, "README.md"), "utf8")).toContain("uncommitted edit");+ }++ it("compacts an undelivered no-PR workspace with leftover files and keeps every source change", async () => {+ const seeded = await seedWorkspace({ commitOnBranch: true });+ const headBefore = await git(seeded.worktreePath, ["rev-parse", "HEAD"]);+ const statusBefore = await git(seeded.worktreePath, ["status", "--porcelain", "--untracked-files=all"]);++ const sweep = await service().sweepTerminalWorkspaces();++ // The branch carries unmerged work, so the reaper keeps the workspace.+ expect(sweep).toMatchObject({ compacted: 1, compactionFailed: 0, archived: 0, skippedUndelivered: 1 });+ expect(await workspaceStatus(seeded.executionWorkspaceId)).toBe("active");+ await expectRegenerablePresent(seeded.worktreePath, false);+ await expectWorkPreserved(seeded.worktreePath);+ expect(await exists(path.join(seeded.worktreePath, "src/feature.ts"))).toBe(true);+ expect(await git(seeded.worktreePath, ["rev-parse", "HEAD"])).toBe(headBefore);+ expect(await git(seeded.repoRoot, ["rev-parse", `refs/heads/${seeded.branchName}`])).toBe(headBefore);+ // Removing ignored directories never changes what Git sees as work.+ expect(await git(seeded.worktreePath, ["status", "--porcelain", "--untracked-files=all"])).toBe(statusBefore);+ // The moved files are deleted, not parked.+ const trashRoot = path.join(path.dirname(seeded.worktreePath), WORKSPACE_COMPACTION_TRASH_DIR);+ expect(await fs.readdir(trashRoot).catch(() => [])).toEqual([]);++ const [activity] = await db+ .select({ details: activityLog.details })+ .from(activityLog)+ .where(and(+ eq(activityLog.entityId, seeded.executionWorkspaceId),+ eq(activityLog.action, "execution_workspace.regenerable_files_removed"),+ ));+ expect((activity?.details as { removedPaths?: string[] } | undefined)?.removedPaths?.sort())+ .toEqual([...REGENERABLE_PATHS].sort());+ }, 30_000);++ it("compacts a delivered workspace that is still inside the archive cooldown", async () => {+ const seeded = await seedWorkspace();+ await git(seeded.worktreePath, ["checkout", "--", "README.md"]);++ const sweep = await service({ workspaceReaperCooldownDays: 7 }).sweepTerminalWorkspaces();++ expect(sweep).toMatchObject({ compacted: 1, archived: 0, skippedCooldown: 1 });+ expect(await workspaceStatus(seeded.executionWorkspaceId)).toBe("active");+ await expectRegenerablePresent(seeded.worktreePath, false);+ expect(await exists(path.join(seeded.worktreePath, "notes/findings.md"))).toBe(true);+ }, 30_000);++ it("waits for the compaction delay after the issue tree becomes terminal", async () => {+ const seeded = await seedWorkspace({ terminalAgoMs: 10 * 60 * 1000 });+ const svc = service({ workspaceReaperCooldownDays: 7, workspaceCompactionDelayMs: HOUR_MS });++ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ // The worker wakes when the delay ends, not when the archive cooldown ends.+ const next = await svc.nextTerminalWorkspaceSweepAt();+ expect(next).not.toBeNull();+ expect(next!).toBeLessThan(Date.now() + HOUR_MS);++ await db+ .update(issues)+ .set({ completedAt: new Date(Date.now() - 2 * HOUR_MS) })+ .where(eq(issues.id, seeded.sourceIssueId));+ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 1 });+ await expectRegenerablePresent(seeded.worktreePath, false);+ }, 30_000);++ it("leaves files alone while a run is active on the task", async () => {+ const seeded = await seedWorkspace();+ const runId = randomUUID();+ await db.insert(heartbeatRuns).values({ id: runId, companyId: seeded.companyId, agentId: seeded.agentId, status: "running" });+ await db.update(issues).set({ checkoutRunId: runId }).where(eq(issues.id, seeded.sourceIssueId));++ const sweep = await service().sweepTerminalWorkspaces();++ expect(sweep).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);++ it("leaves files alone while a queued run targets the task through its context", async () => {+ const seeded = await seedWorkspace();+ await db.insert(heartbeatRuns).values({+ id: randomUUID(),+ companyId: seeded.companyId,+ agentId: seeded.agentId,+ status: "queued",+ contextSnapshot: { issueId: seeded.sourceIssueId },+ });++ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);++ it("leaves files alone while a child issue is unfinished", async () => {+ const seeded = await seedWorkspace();+ await db.insert(issues).values({+ id: randomUUID(),+ companyId: seeded.companyId,+ projectId: seeded.projectId,+ parentId: seeded.sourceIssueId,+ identifier: `${seeded.issuePrefix}-2`,+ title: "Follow-up",+ status: "todo",+ priority: "medium",+ });++ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 0, skippedNonTerminalTree: 1 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);++ it("leaves files alone while a runtime service runs from the worktree", async () => {+ const seeded = await seedWorkspace();+ await db.insert(workspaceRuntimeServices).values({+ id: randomUUID(),+ companyId: seeded.companyId,+ projectId: seeded.projectId,+ executionWorkspaceId: seeded.executionWorkspaceId,+ issueId: seeded.sourceIssueId,+ scopeType: "execution_workspace",+ serviceName: "web",+ status: "running",+ lifecycle: "shared",+ command: "pnpm dev",+ cwd: seeded.worktreePath,+ provider: "local_process",+ healthStatus: "healthy",+ });++ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);++ it("restores the files when the task reopens while they are being moved", async () => {+ const seeded = await seedWorkspace();+ const svc = service({+ afterWorkspaceCompactionMove: async () => {+ await db.update(issues).set({ status: "todo", completedAt: null }).where(eq(issues.id, seeded.sourceIssueId));+ },+ });++ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 0, compactionFailed: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ expect(await fs.readFile(path.join(seeded.worktreePath, "node_modules/left-pad/index.js"), "utf8"))+ .toBe("module.exports = 1;\n");+ const trashRoot = path.join(path.dirname(seeded.worktreePath), WORKSPACE_COMPACTION_TRASH_DIR);+ expect(await fs.readdir(trashRoot).catch(() => [])).toEqual([]);+ }, 30_000);++ it("restores the files when a run starts while they are being moved", async () => {+ const seeded = await seedWorkspace();+ const svc = service({+ afterWorkspaceCompactionMove: async () => {+ await db.insert(heartbeatRuns).values({+ id: randomUUID(),+ companyId: seeded.companyId,+ agentId: seeded.agentId,+ status: "queued",+ contextSnapshot: { issueId: seeded.sourceIssueId },+ });+ },+ });++ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);++ it("never compacts a shared workspace or a main checkout", async () => {+ const shared = await seedWorkspace({ mode: "shared_workspace" });+ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(shared.worktreePath, true);++ // A workspace row that points at the repository's main checkout (its .git+ // is a directory) is never compacted, whatever its strategy says.+ const repoRoot = await createRepo();+ await populateWorktree(repoRoot);+ await seedWorkspace({ repoRoot, worktreePath: repoRoot, populate: false });+ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(repoRoot, true);+ }, 30_000);++ it("compacts once per terminal cycle and again after the task finishes a later cycle", async () => {+ const seeded = await seedWorkspace();+ const svc = service({ workspaceReaperCooldownDays: 7 });++ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 1 });+ // A resumed run regenerates dependencies, and the task finishes again.+ await writeFile(path.join(seeded.worktreePath, "node_modules/left-pad/index.js"), "module.exports = 1;\n");+ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ expect(await exists(path.join(seeded.worktreePath, "node_modules"))).toBe(true);++ await db+ .update(issues)+ .set({ completedAt: new Date(Date.now() - HOUR_MS) })+ .where(eq(issues.id, seeded.sourceIssueId));+ expect(await svc.sweepTerminalWorkspaces()).toMatchObject({ compacted: 1 });+ expect(await exists(path.join(seeded.worktreePath, "node_modules"))).toBe(false);+ }, 30_000);++ it("regenerates removed files when the task resumes in the compacted worktree", async () => {+ const repoRoot = await createRepo();+ tempDirs.add(repoRoot);+ const provisionCommand = "mkdir -p node_modules/left-pad && echo regenerated > node_modules/left-pad/index.js";+ const realizationInput = {+ base: {+ baseCwd: repoRoot,+ source: "project_primary" as const,+ projectId: "project-1",+ workspaceId: "workspace-1",+ repoUrl: null,+ repoRef: "main",+ },+ config: { workspaceStrategy: { type: "git_worktree", baseRef: "main", provisionCommand } },+ issue: { id: "issue-1", identifier: "PAP-9", title: "Resume after compaction" },+ agent: { id: "agent-1", name: "Coder", companyId: "company-1" },+ } satisfies Parameters<typeof realizeExecutionWorkspace>[0];+ const first = await realizeExecutionWorkspace(realizationInput);+ expect(first.created).toBe(true);+ await writeFile(path.join(first.cwd, "notes/findings.md"), "untracked agent notes\n");+ await writeFile(path.join(first.cwd, "src/feature.ts"), "export const unmerged = true;\n");+ await git(first.cwd, ["add", "src/feature.ts"]);+ await git(first.cwd, ["commit", "-m", "Unmerged work"]);+ const head = await git(first.cwd, ["rev-parse", "HEAD"]);++ const seeded = await seedWorkspace({ repoRoot, worktreePath: first.cwd, populate: false });+ expect(await service().sweepTerminalWorkspaces()).toMatchObject({ compacted: 1 });+ expect(await exists(path.join(first.cwd, "node_modules"))).toBe(false);+ expect(await workspaceStatus(seeded.executionWorkspaceId)).toBe("active");++ const resumed = await realizeExecutionWorkspace({+ ...realizationInput,+ recordedBranchOwnership: { branchName: first.branchName!, createdByRuntime: true },+ });++ expect(resumed.created).toBe(false);+ expect(resumed.cwd).toBe(first.cwd);+ expect(await fs.readFile(path.join(first.cwd, "node_modules/left-pad/index.js"), "utf8")).toBe("regenerated\n");+ expect(await fs.readFile(path.join(first.cwd, "notes/findings.md"), "utf8")).toBe("untracked agent notes\n");+ expect(await git(first.cwd, ["rev-parse", "HEAD"])).toBe(head);+ }, 60_000);++ it("purges leftover trash even when the workspace has nothing left to compact", async () => {+ const seeded = await seedWorkspace({ populate: false });+ const trashEntry = path.join(path.dirname(seeded.worktreePath), WORKSPACE_COMPACTION_TRASH_DIR, "interrupted");+ await writeFile(path.join(trashEntry, "0/left-pad/index.js"), "module.exports = 1;\n");+ const twoHoursAgo = new Date(Date.now() - 2 * HOUR_MS);+ await fs.utimes(trashEntry, twoHoursAgo, twoHoursAgo);++ const sweep = await service().sweepTerminalWorkspaces();++ expect(sweep).toMatchObject({ compacted: 0, compactionFailed: 0 });+ expect(await exists(trashEntry)).toBe(false);+ }, 30_000);++ it("does not compact when compaction is disabled", async () => {+ const seeded = await seedWorkspace();+ expect(await service({ workspaceCompactionDelayMs: null }).sweepTerminalWorkspaces()).toMatchObject({ compacted: 0 });+ await expectRegenerablePresent(seeded.worktreePath, true);+ }, 30_000);+});diff --git a/server/src/__tests__/workspace-compaction.test.ts b/server/src/__tests__/workspace-compaction.test.tsnew file mode 100644index 0000000000..eb19c828ef--- /dev/null+++ b/server/src/__tests__/workspace-compaction.test.ts@@ -0,0 +1,214 @@+import { execFile } from "node:child_process";+import fs from "node:fs/promises";+import os from "node:os";+import path from "node:path";+import { promisify } from "node:util";+import { afterEach, describe, expect, it } from "vitest";+import {+ WORKSPACE_COMPACTION_TRASH_DIR,+ deleteWorkspaceCompactionTrash,+ findRegenerableWorkspaceDirs,+ isLinkedGitWorktree,+ moveWorkspaceDirsToTrash,+ purgeStaleWorkspaceCompactionTrash,+ restoreWorkspaceDirsFromTrash,+} from "../services/workspace-compaction.ts";++const execFileAsync = promisify(execFile);+const tempDirs: string[] = [];++async function git(cwd: string, args: string[]) {+ return (await execFileAsync("git", ["-C", cwd, ...args], { cwd })).stdout.trim();+}++async function write(filePath: string, contents = "x\n") {+ await fs.mkdir(path.dirname(filePath), { recursive: true });+ await fs.writeFile(filePath, contents, "utf8");+}++async function createRepoWithWorktree(gitignore: string[]) {+ const root = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-compaction-unit-")));+ tempDirs.push(root);+ const repoRoot = path.join(root, "repo");+ await fs.mkdir(repoRoot);+ await git(repoRoot, ["init", "-b", "main"]);+ await git(repoRoot, ["config", "user.name", "Paperclip Test"]);+ await git(repoRoot, ["config", "user.email", "[email protected]"]);+ await write(path.join(repoRoot, ".gitignore"), `${gitignore.join("\n")}\n`);+ await git(repoRoot, ["add", ".gitignore"]);+ await git(repoRoot, ["commit", "-m", "Initial commit"]);+ const worktreePath = path.join(root, "worktrees", "PAP-1-task");+ await git(repoRoot, ["worktree", "add", "-b", "PAP-1-task", worktreePath, "main"]);+ return { root, repoRoot, worktreePath };+}++afterEach(async () => {+ for (const dir of tempDirs.splice(0)) await fs.rm(dir, { recursive: true, force: true });+});++describe("findRegenerableWorkspaceDirs", () => {+ it("returns only ignored, untracked directories with a regenerable name", async () => {+ const { worktreePath } = await createRepoWithWorktree([+ "node_modules/", "dist/", ".turbo/", ".env", "local-data/", ".paperclip/", "coverage/",+ ]);+ await write(path.join(worktreePath, "node_modules/a/index.js"));+ await write(path.join(worktreePath, "packages/ui/node_modules/b/index.js"));+ await write(path.join(worktreePath, "packages/ui/dist/ui.js"));+ await write(path.join(worktreePath, ".turbo/cache"));+ // Ignored, but not a regenerable name: data and secrets stay.+ await write(path.join(worktreePath, ".env"));+ await write(path.join(worktreePath, "local-data/state.json"));+ // A regenerable name that Git does not ignore is source.+ await write(path.join(worktreePath, "build/keep.txt"));+ // The worktree instance data is never scanned.+ await write(path.join(worktreePath, ".paperclip/node_modules/c/index.js"));+ // An ignored directory with a force-added tracked file is source.+ await write(path.join(worktreePath, "coverage/baseline.json"));+ await git(worktreePath, ["add", "-f", "coverage/baseline.json"]);++ expect((await findRegenerableWorkspaceDirs(worktreePath)).sort()).toEqual([+ ".turbo",+ "node_modules",+ "packages/ui/dist",+ "packages/ui/node_modules",+ ]);+ });++ it("does not follow or return a symlinked directory", async () => {+ const { root, worktreePath } = await createRepoWithWorktree(["node_modules"]);+ const outside = path.join(root, "shared-store");+ await write(path.join(outside, "node_modules/pkg/index.js"));+ await fs.symlink(path.join(outside, "node_modules"), path.join(worktreePath, "node_modules"));+ await fs.symlink(outside, path.join(worktreePath, "linked"));++ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual([]);+ });++ it("searches inside a regenerable-named directory that is source", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/"]);+ // `build/` holds tracked source, so it stays, but its ignored dependencies go.+ await write(path.join(worktreePath, "build/scripts/release.ts"));+ await git(worktreePath, ["add", "build/scripts/release.ts"]);+ await write(path.join(worktreePath, "build/node_modules/tool/index.js"));+ // An untracked, not-ignored `dist/` is work in progress; its ignored+ // dependencies still go.+ await write(path.join(worktreePath, "dist/notes.md"));+ await write(path.join(worktreePath, "dist/node_modules/dep/index.js"));++ expect((await findRegenerableWorkspaceDirs(worktreePath)).sort()).toEqual([+ "build/node_modules",+ "dist/node_modules",+ ]);+ });++ it("finds dependencies below an ignored parent directory", async () => {+ const { worktreePath } = await createRepoWithWorktree(["vendor/", "node_modules/"]);+ await write(path.join(worktreePath, "vendor/tool/source.c"));+ await write(path.join(worktreePath, "vendor/tool/node_modules/dep/index.js"));+ // A single nested candidate once made `ls-files --directory` fail.+ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual(["vendor/tool/node_modules"]);++ await write(path.join(worktreePath, "node_modules/a/index.js"));+ expect((await findRegenerableWorkspaceDirs(worktreePath)).sort())+ .toEqual(["node_modules", "vendor/tool/node_modules"]);+ });++ it("keeps a directory that a negated pattern unignores", async () => {+ const { worktreePath } = await createRepoWithWorktree(["build/", "!build/", "dist/", "node_modules/"]);+ // Untracked work in an explicitly unignored directory is not regenerable.+ await write(path.join(worktreePath, "build/release-notes.md"));+ await write(path.join(worktreePath, "dist/bundle.js"));++ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual(["dist"]);+ await expect(fs.access(path.join(worktreePath, "build/release-notes.md"))).resolves.toBeUndefined();+ });++ it("handles directory names with glob characters in the tracked-file check", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/"]);+ await write(path.join(worktreePath, "pkg[1]/node_modules/vendored/index.js"));+ await git(worktreePath, ["add", "-f", "pkg[1]/node_modules/vendored/index.js"]);+ await write(path.join(worktreePath, "pkg1/node_modules/dep/index.js"));++ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual(["pkg1/node_modules"]);+ });++ it("ignores global and repository-local exclude files", async () => {+ const { root, repoRoot, worktreePath } = await createRepoWithWorktree(["dist/"]);+ const globalExcludes = path.join(root, "global-gitignore");+ await write(globalExcludes, "build/\n");+ await git(repoRoot, ["config", "core.excludesFile", globalExcludes]);+ await write(path.join(repoRoot, ".git/info/exclude"), "coverage/\n");+ await write(path.join(worktreePath, "build/draft.md"));+ await write(path.join(worktreePath, "coverage/report.html"));+ await write(path.join(worktreePath, "dist/bundle.js"));++ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual(["dist"]);+ });++ it("stops when the sweep is aborted", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/"]);+ await write(path.join(worktreePath, "node_modules/a/index.js"));+ const controller = new AbortController();+ controller.abort();+ await expect(findRegenerableWorkspaceDirs(worktreePath, { signal: controller.signal })).rejects.toThrow();+ });++ it("returns nothing when no directory qualifies", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/"]);+ await write(path.join(worktreePath, "src/index.ts"));+ expect(await findRegenerableWorkspaceDirs(worktreePath)).toEqual([]);+ });+});++describe("isLinkedGitWorktree", () => {+ it("accepts a linked worktree and rejects a main checkout", async () => {+ const { repoRoot, worktreePath } = await createRepoWithWorktree([]);+ expect(await isLinkedGitWorktree(worktreePath)).toBe(true);+ expect(await isLinkedGitWorktree(repoRoot)).toBe(false);+ });+});++describe("compaction trash", () => {+ it("moves directories beside the worktree and restores them unchanged", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/", "dist/"]);+ await write(path.join(worktreePath, "node_modules/a/index.js"), "dependency\n");+ await write(path.join(worktreePath, "packages/ui/dist/ui.js"), "bundle\n");++ const move = await moveWorkspaceDirsToTrash(worktreePath, ["node_modules", "packages/ui/dist", "missing"]);+ expect(move.moved.map((entry) => entry.relativePath)).toEqual(["node_modules", "packages/ui/dist"]);+ expect(path.dirname(path.dirname(move.trashDir))).toBe(path.dirname(worktreePath));+ expect(path.basename(path.dirname(move.trashDir))).toBe(WORKSPACE_COMPACTION_TRASH_DIR);+ await expect(fs.access(path.join(worktreePath, "node_modules"))).rejects.toThrow();++ await restoreWorkspaceDirsFromTrash(worktreePath, move);+ expect(await fs.readFile(path.join(worktreePath, "node_modules/a/index.js"), "utf8")).toBe("dependency\n");+ expect(await fs.readFile(path.join(worktreePath, "packages/ui/dist/ui.js"), "utf8")).toBe("bundle\n");+ await deleteWorkspaceCompactionTrash(move.trashDir);+ await expect(fs.access(move.trashDir)).rejects.toThrow();+ });++ it("keeps a directory something recreated instead of overwriting it on restore", async () => {+ const { worktreePath } = await createRepoWithWorktree(["node_modules/"]);+ await write(path.join(worktreePath, "node_modules/a/index.js"), "old\n");+ const move = await moveWorkspaceDirsToTrash(worktreePath, ["node_modules"]);+ await write(path.join(worktreePath, "node_modules/a/index.js"), "reinstalled\n");++ await restoreWorkspaceDirsFromTrash(worktreePath, move);+ expect(await fs.readFile(path.join(worktreePath, "node_modules/a/index.js"), "utf8")).toBe("reinstalled\n");+ });++ it("purges only trash older than the threshold", async () => {+ const { worktreePath } = await createRepoWithWorktree([]);+ const parent = path.dirname(worktreePath);+ const stale = path.join(parent, WORKSPACE_COMPACTION_TRASH_DIR, "stale");+ const fresh = path.join(parent, WORKSPACE_COMPACTION_TRASH_DIR, "fresh");+ await write(path.join(stale, "0/file"));+ await write(path.join(fresh, "0/file"));+ const twoHoursAgo = new Date(Date.now() - 2 * 60 * 60 * 1000);+ await fs.utimes(stale, twoHoursAgo, twoHoursAgo);++ await purgeStaleWorkspaceCompactionTrash(parent, 60 * 60 * 1000);+ await expect(fs.access(stale)).rejects.toThrow();+ await expect(fs.access(fresh)).resolves.toBeUndefined();+ });+});diff --git a/server/src/__tests__/workspace-reaper-cooldown-config.test.ts b/server/src/__tests__/workspace-reaper-cooldown-config.test.tsindex 2bcde2f4d6..51d8134560 100644--- a/server/src/__tests__/workspace-reaper-cooldown-config.test.ts+++ b/server/src/__tests__/workspace-reaper-cooldown-config.test.ts@@ -2,7 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";import { loadConfig } from "../config.ts";// The terminal-workspace reaper reads PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS.-// These tests lock the parser contract: the default is 7 days, an explicit 0+// These tests lock the parser contract: the default is 1 day, an explicit 0// means immediate reaping, and empty, whitespace-only, negative, or non-numeric// values fall back to the default. An empty or whitespace-only value must not// become 0, because that would delete terminal workspaces immediately.@@ -12,19 +12,19 @@ describe("workspace reaper cooldown config parsing", () => {vi.unstubAllEnvs();});- it("uses the 7 day default when the variable is not set", () => {+ it("uses the 1 day default when the variable is not set", () => {vi.stubEnv("PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", undefined);- expect(loadConfig().workspaceReaperCooldownDays).toBe(7);+ expect(loadConfig().workspaceReaperCooldownDays).toBe(1);});- it("uses the 7 day default for an empty value", () => {+ it("uses the 1 day default for an empty value", () => {vi.stubEnv("PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", "");- expect(loadConfig().workspaceReaperCooldownDays).toBe(7);+ expect(loadConfig().workspaceReaperCooldownDays).toBe(1);});- it("uses the 7 day default for a whitespace-only value", () => {+ it("uses the 1 day default for a whitespace-only value", () => {vi.stubEnv("PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", " ");- expect(loadConfig().workspaceReaperCooldownDays).toBe(7);+ expect(loadConfig().workspaceReaperCooldownDays).toBe(1);});it("keeps an explicit 0 as immediate reaping", () => {@@ -42,13 +42,50 @@ describe("workspace reaper cooldown config parsing", () => {expect(loadConfig().workspaceReaperCooldownDays).toBe(3);});- it("uses the 7 day default for a negative value", () => {+ it("uses the 1 day default for a negative value", () => {vi.stubEnv("PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", "-1");- expect(loadConfig().workspaceReaperCooldownDays).toBe(7);+ expect(loadConfig().workspaceReaperCooldownDays).toBe(1);});- it("uses the 7 day default for a non-numeric value", () => {+ it("uses the 1 day default for a non-numeric value", () => {vi.stubEnv("PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", "soon");- expect(loadConfig().workspaceReaperCooldownDays).toBe(7);+ expect(loadConfig().workspaceReaperCooldownDays).toBe(1);+ });+});++// The reaper reads PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS and+// PAPERCLIP_WORKSPACE_COMPACTION_ENABLED. The default delay is 1 hour, an+// explicit 0 compacts on the next sweep, invalid values fall back to the+// default, and only an explicit "false" turns compaction off.+describe("workspace compaction config parsing", () => {+ afterEach(() => {+ vi.unstubAllEnvs();+ });++ it("uses the 1 hour default when the variables are not set", () => {+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS", undefined);+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_ENABLED", undefined);+ expect(loadConfig().workspaceCompactionDelayHours).toBe(1);+ });++ it("keeps an explicit 0 and reads fractional hours", () => {+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS", "0");+ expect(loadConfig().workspaceCompactionDelayHours).toBe(0);+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS", " 0.5 ");+ expect(loadConfig().workspaceCompactionDelayHours).toBe(0.5);+ });++ it("uses the 1 hour default for empty, negative, or non-numeric values", () => {+ for (const value of ["", " ", "-2", "later"]) {+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS", value);+ expect(loadConfig().workspaceCompactionDelayHours).toBe(1);+ }+ });++ it("turns compaction off only for an explicit false", () => {+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_ENABLED", "false");+ expect(loadConfig().workspaceCompactionDelayHours).toBeNull();+ vi.stubEnv("PAPERCLIP_WORKSPACE_COMPACTION_ENABLED", "no");+ expect(loadConfig().workspaceCompactionDelayHours).toBe(1);});});diff --git a/server/src/config.ts b/server/src/config.tsindex db5efb2cbd..69860d95f9 100644--- a/server/src/config.ts+++ b/server/src/config.ts@@ -80,6 +80,7 @@ export interface Config {databaseBackupRetentionDays: number;databaseBackupDir: string;workspaceReaperCooldownDays: number;+ workspaceCompactionDelayHours: number | null;serveUi: boolean;uiDevMiddleware: boolean;secretsProvider: SecretProvider;@@ -281,8 +282,10 @@ export function loadConfig(): Config {// becomes terminal before it archives the workspace. A person can reopen the// work inside this window. A value of 0 disables the cooldown and restores// immediate reaping. A negative or non-numeric value falls back to the- // default. The day granularity and the default of 7 obey the- // PAPERCLIP_DB_BACKUP_RETENTION_DAYS precedent above.+ // default. The day granularity obeys the PAPERCLIP_DB_BACKUP_RETENTION_DAYS+ // precedent above. The default is 1 day: regenerable files are compacted+ // after a few hours anyway, so a longer hold only keeps source checkouts of+ // delivered work, and a recurring task produces one workspace per run.const workspaceReaperCooldownDaysEnv =process.env.PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS?.trim();const workspaceReaperCooldownDaysRaw = Number(workspaceReaperCooldownDaysEnv);@@ -291,7 +294,25 @@ export function loadConfig(): Config {&& Number.isFinite(workspaceReaperCooldownDaysRaw)&& workspaceReaperCooldownDaysRaw >= 0? workspaceReaperCooldownDaysRaw- : 7;+ : 1;+ // The terminal-workspace reaper removes regenerable files (installed+ // dependencies, build output, tool caches) from a terminal workspace this+ // many hours after its issue tree becomes terminal. The worktree and every+ // source file stay; a resumed task regenerates the files when its workspace+ // is provisioned again. 0 compacts on the next sweep. A negative, empty, or+ // non-numeric value falls back to the default of 1 hour.+ // PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false turns compaction off.+ const workspaceCompactionDelayHoursEnv =+ process.env.PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS?.trim();+ const workspaceCompactionDelayHoursRaw = Number(workspaceCompactionDelayHoursEnv);+ const workspaceCompactionDelayHours =+ process.env.PAPERCLIP_WORKSPACE_COMPACTION_ENABLED?.trim().toLowerCase() === "false"+ ? null+ : workspaceCompactionDelayHoursEnv+ && Number.isFinite(workspaceCompactionDelayHoursRaw)+ && workspaceCompactionDelayHoursRaw >= 0+ ? workspaceCompactionDelayHoursRaw+ : 1;const bindValidationErrors = validateConfiguredBindMode({deploymentMode,deploymentExposure,@@ -338,6 +359,7 @@ export function loadConfig(): Config {databaseBackupRetentionDays,databaseBackupDir,workspaceReaperCooldownDays,+ workspaceCompactionDelayHours,serveUi:process.env.SERVE_UI !== undefined? process.env.SERVE_UI === "true"diff --git a/server/src/index.ts b/server/src/index.tsindex 9b5f85562b..d2c89452af 100644--- a/server/src/index.ts+++ b/server/src/index.ts@@ -1374,6 +1374,9 @@ async function startServerWithDatabaseTeardown(});const terminalWorkspaces = executionWorkspaceService(db as any, {workspaceReaperCooldownDays: config.workspaceReaperCooldownDays,+ workspaceCompactionDelayMs: config.workspaceCompactionDelayHours === null+ ? null+ : config.workspaceCompactionDelayHours * 60 * 60 * 1000,terminalWorkspaceInspectionBackoffMs: TERMINAL_WORKSPACE_RETRY_MS,});const scheduleMergedPullRequestConfirmationSweep = () => {@@ -1394,7 +1397,7 @@ async function startServerWithDatabaseTeardown(...terminalWorkspaces,sweepTerminalWorkspaces: async (limit, signal) => {const result = await terminalWorkspaces.sweepTerminalWorkspaces(limit, signal);- if (result.archived > 0 || result.cleanupFailed > 0) {+ if (result.archived > 0 || result.cleanupFailed > 0 || result.compacted > 0 || result.compactionFailed > 0) {logger.info(result, "terminal issue workspace reaper changed workspace state");} else if (result.checked > 0 && Date.now() - lastTerminalWorkspaceSkipLogAt >= 10 * 60 * 1000) {lastTerminalWorkspaceSkipLogAt = Date.now();diff --git a/server/src/services/execution-workspaces.ts b/server/src/services/execution-workspaces.tsindex 43c990e119..05e0bc77a6 100644--- a/server/src/services/execution-workspaces.ts+++ b/server/src/services/execution-workspaces.ts@@ -12,6 +12,14 @@ import { authorizationService, executionWorkspaceReadSqlCondition, projectReadSqimport type { SQL } from "drizzle-orm";import { createWorkspaceGitInspectionCache } from "./workspace-git-inspection-cache.js";import { notifyTerminalWorkspaceWork } from "./terminal-workspace-work.js";+import {+ deleteWorkspaceCompactionTrash,+ findRegenerableWorkspaceDirs,+ isLinkedGitWorktree,+ moveWorkspaceDirsToTrash,+ purgeStaleWorkspaceCompactionTrash,+ restoreWorkspaceDirsFromTrash,+} from "./workspace-compaction.js";import { execFile } from "node:child_process";import { createHash } from "node:crypto";import fs from "node:fs/promises";@@ -282,8 +290,17 @@ export type ExecutionWorkspaceServiceOptions = {beforeTerminalWorkspaceCleanup?: (workspace: ExecutionWorkspaceRow) => Promise<void>;// The terminal-workspace reaper waits this many days after an issue tree// becomes terminal before it archives the workspace. A value of 0 disables- // the cooldown. The default is 7 days.+ // the cooldown. The default is 1 day.workspaceReaperCooldownDays?: number;+ // The reaper removes regenerable files (installed dependencies, build output,+ // tool caches) from a terminal workspace this long after its issue tree+ // becomes terminal. The worktree, its branch, its commits, and every source+ // file stay, so the task can resume in place. Unset or null disables+ // compaction.+ workspaceCompactionDelayMs?: number | null;+ // Test hook. Runs after compaction moves directories out of the worktree and+ // before it re-checks the guard under the lifecycle lock.+ afterWorkspaceCompactionMove?: (workspace: ExecutionWorkspaceRow) => Promise<void>;inspectGitCloseReadiness?: (workspace: ExecutionWorkspace) => Promise<{git: ExecutionWorkspaceCloseGitReadiness | null;warnings: string[];@@ -1366,6 +1383,11 @@ async function hasPendingTaskWorkspaceRecovery(tx: DbTransaction, input: {});}+// Compaction trash older than this belongs to no compaction in progress: the+// move and the guard finish in one short transaction.+const WORKSPACE_COMPACTION_TRASH_STALE_MS = 60 * 60 * 1000;+const WORKSPACE_COMPACTION_TRASH_PURGE_INTERVAL_MS = 30 * 60 * 1000;+export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServiceOptions = {}) {async function resolveSourceWorkspaceStrategy(input: {companyId: string; issueId?: string | null; assigneeAgentId?: string | null;@@ -1602,8 +1624,17 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic// disables the cooldown.const workspaceReaperCooldownMs = Math.max(0,- (opts.workspaceReaperCooldownDays ?? 7) * 24 * 60 * 60 * 1000,+ (opts.workspaceReaperCooldownDays ?? 1) * 24 * 60 * 60 * 1000,);+ const workspaceCompactionDelayMs = typeof opts.workspaceCompactionDelayMs === "number"+ ? Math.max(0, opts.workspaceCompactionDelayMs)+ : null;+ // The latest terminal timestamp each workspace was compacted for. A workspace+ // is compacted once per terminal cycle: a resumed task can regenerate its+ // files, and the next terminal cycle compacts them again. The memo is only a+ // cache; after a restart compaction re-scans, finds nothing, and records it.+ const compactedTerminalAt = new Map<string, number>();+ const compactionTrashPurgedAt = new Map<string, number>();const pullRequestStateCache = new Map<string,{@@ -1990,6 +2021,187 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic});}+ // Reads, inside the caller's transaction, whether compaction may touch the+ // workspace files right now: the issue tree is terminal, no queued or running+ // heartbeat run works on the tree or the workspace, and no runtime service+ // still runs from the worktree.+ async function terminalWorkspaceCompactionGuardPasses(tx: DbTransaction, workspace: ExecutionWorkspaceRow) {+ const rows = await tx.execute<{ allowed: boolean }>(sql`+ WITH RECURSIVE issue_tree(id, status) AS (+ SELECT root.id, root.status+ FROM ${issues} root+ WHERE root.company_id = ${workspace.companyId}+ AND root.id = ${workspace.sourceIssueId}+ UNION ALL+ SELECT child.id, child.status+ FROM ${issues} child+ JOIN issue_tree parent ON child.parent_id = parent.id+ WHERE child.company_id = ${workspace.companyId}+ )+ SELECT (+ EXISTS (SELECT 1 FROM issue_tree)+ AND NOT EXISTS (SELECT 1 FROM issue_tree WHERE status NOT IN ('done', 'cancelled'))+ AND NOT EXISTS (+ SELECT 1+ FROM ${issues} linked_issue+ JOIN ${heartbeatRuns} live_run+ ON live_run.id = linked_issue.checkout_run_id+ OR live_run.id = linked_issue.execution_run_id+ WHERE linked_issue.company_id = ${workspace.companyId}+ AND (+ linked_issue.execution_workspace_id = ${workspace.id}+ OR linked_issue.id IN (SELECT id FROM issue_tree)+ )+ AND live_run.company_id = ${workspace.companyId}+ AND live_run.status IN ('queued', 'running')+ )+ AND NOT EXISTS (+ SELECT 1+ FROM ${heartbeatRuns} issue_run+ WHERE issue_run.company_id = ${workspace.companyId}+ AND issue_run.status IN ('queued', 'running')+ AND coalesce(+ issue_run.native_issue_id::text,+ nullif(issue_run.context_snapshot->>'issueId', ''),+ nullif(issue_run.context_snapshot->>'taskId', '')+ ) IN (SELECT id::text FROM issue_tree)+ )+ AND NOT EXISTS (+ SELECT 1+ FROM ${workspaceRuntimeServices} service+ WHERE service.company_id = ${workspace.companyId}+ AND service.execution_workspace_id = ${workspace.id}+ AND service.status <> 'stopped'+ )+ ) AS allowed+ `);+ return rows[0]?.allowed === true;+ }++ // Deletes compaction trash that an interrupted compaction or a failed delete+ // left beside a worktree. The sweep calls this for every worktree candidate,+ // independent of the per-cycle compaction memo, so leftover trash is found+ // after a restart and after a workspace has nothing left to compact. Each+ // worktree parent directory is checked at most once per interval.+ async function purgeCompactionTrashIfDue(workspace: ExecutionWorkspaceRow) {+ const worktreePath = workspace.providerRef ?? workspace.cwd;+ if (workspace.providerType !== "git_worktree" || !worktreePath) return;+ const parentDir = path.dirname(path.resolve(worktreePath));+ const nowMs = now().getTime();+ const lastPurgedAt = compactionTrashPurgedAt.get(parentDir);+ if (lastPurgedAt !== undefined && nowMs - lastPurgedAt < WORKSPACE_COMPACTION_TRASH_PURGE_INTERVAL_MS) return;+ if (compactionTrashPurgedAt.size >= 1024) compactionTrashPurgedAt.delete(compactionTrashPurgedAt.keys().next().value!);+ compactionTrashPurgedAt.set(parentDir, nowMs);+ await purgeStaleWorkspaceCompactionTrash(parentDir, WORKSPACE_COMPACTION_TRASH_STALE_MS).catch((error) => {+ logger.warn({ err: error, parentDir }, "failed to purge stale workspace compaction trash");+ });+ }++ // Removes regenerable files (installed dependencies, build output, tool+ // caches) from a terminal workspace. The workspace row stays active and the+ // worktree, its branch, its commits, and every tracked or untracked source+ // file stay in place, so a resumed task continues where it stopped and its+ // provisioning regenerates the removed files. This applies to workspaces the+ // reaper cannot archive: an undelivered branch, leftover files, or a workspace+ // still inside the archive cooldown.+ async function compactTerminalWorkspace(+ workspace: ExecutionWorkspaceRow,+ signal?: AbortSignal,+ ): Promise<"compacted" | "nothing_to_compact" | "not_compactable" | "busy"> {+ const worktreePath = workspace.providerRef ?? workspace.cwd;+ if (+ workspace.providerType !== "git_worktree"+ || workspace.mode === "shared_workspace"+ || !workspace.sourceIssueId+ || !worktreePath+ ) {+ return "not_compactable";+ }+ if (!(await isLinkedGitWorktree(worktreePath))) return "not_compactable";+ const projectWorkspaceCwd = workspace.projectWorkspaceId+ ? await db+ .select({ cwd: projectWorkspaces.cwd })+ .from(projectWorkspaces)+ .where(and(+ eq(projectWorkspaces.companyId, workspace.companyId),+ eq(projectWorkspaces.id, workspace.projectWorkspaceId),+ ))+ .then((rows) => rows[0]?.cwd ?? null)+ : null;+ if (projectWorkspaceCwd && path.resolve(projectWorkspaceCwd) === path.resolve(worktreePath)) {+ return "not_compactable";+ }++ const relativePaths = await findRegenerableWorkspaceDirs(worktreePath, { signal });+ if (relativePaths.length === 0) return "nothing_to_compact";++ const snapshotGeneration = readExecutionWorkspaceLifecycleGeneration(+ workspace.metadata as Record<string, unknown> | null,+ );+ // Hold the lifecycle lock while the directories move. The guard runs before+ // the move and again after it, in the same transaction. A task that resumes+ // in between leaves the terminal state or starts a run first, so the second+ // check fails and the directories move back before anything is deleted.+ const move = await db.transaction(async (tx) => {+ await acquireExecutionWorkspaceLifecycleLock(tx, workspace.id);+ const [fresh] = await tx+ .select({+ status: executionWorkspaces.status,+ closedAt: executionWorkspaces.closedAt,+ metadata: executionWorkspaces.metadata,+ })+ .from(executionWorkspaces)+ .where(and(eq(executionWorkspaces.id, workspace.id), eq(executionWorkspaces.companyId, workspace.companyId)));+ const freshMetadata = (fresh?.metadata as Record<string, unknown> | null) ?? null;+ if (+ !fresh+ || isClosedExecutionWorkspaceStatus(fresh.status)+ || fresh.closedAt+ || metadataHasReopenPendingConsumption(freshMetadata)+ || readExecutionWorkspaceLifecycleGeneration(freshMetadata) !== snapshotGeneration+ || !(await terminalWorkspaceCompactionGuardPasses(tx, workspace))+ ) {+ return null;+ }+ const moved = await moveWorkspaceDirsToTrash(worktreePath, relativePaths);+ try {+ await opts.afterWorkspaceCompactionMove?.(workspace);+ if (!(await terminalWorkspaceCompactionGuardPasses(tx, workspace))) {+ await restoreWorkspaceDirsFromTrash(worktreePath, moved);+ await deleteWorkspaceCompactionTrash(moved.trashDir);+ return null;+ }+ } catch (error) {+ await restoreWorkspaceDirsFromTrash(worktreePath, moved);+ await deleteWorkspaceCompactionTrash(moved.trashDir);+ throw error;+ }+ return moved;+ });+ if (!move) return "busy";++ await deleteWorkspaceCompactionTrash(move.trashDir).catch((error) => {+ logger.warn(+ { err: error, executionWorkspaceId: workspace.id, trashDir: move.trashDir },+ "failed to delete compacted workspace files; a later compaction retries",+ );+ });+ if (move.moved.length === 0) return "nothing_to_compact";+ await logActivity(db, {+ companyId: workspace.companyId,+ actorType: "system",+ actorId: "workspace_terminality_reaper",+ action: "execution_workspace.regenerable_files_removed",+ entityType: "execution_workspace",+ entityId: workspace.id,+ details: {+ sourceIssueId: workspace.sourceIssueId,+ removedPaths: move.moved.map((entry) => entry.relativePath),+ },+ });+ return "compacted";+ }+async function cleanupTerminalWorkspace(workspace: ExecutionWorkspaceRow,expectedHeadSha: string | null,@@ -2885,6 +3097,8 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServicskippedCooldown: 0,skippedBackoff: 0,clearedStaleReopenPending: 0,+ compacted: 0,+ compactionFailed: 0,};}const previousCursor = terminalSweepCursor;@@ -2954,11 +3168,14 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServicskippedCooldown: 0,skippedBackoff: 0,clearedStaleReopenPending: 0,+ compacted: 0,+ compactionFailed: 0,};const readinessByWorkingTree = new Map<string, ReturnType<typeof inspectGitCloseReadiness>>();for (const workspace of candidates) {signal?.throwIfAborted();+ await purgeCompactionTrashIfDue(workspace);const issueTree = await listWorkspaceIssueTree(workspace);const sourceIssue = issueTree.find((issue) => issue.id === workspace.sourceIssueId);const reopenPending = metadataHasReopenPendingConsumption(@@ -2977,8 +3194,34 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServiccontinue;}const latestTerminalAt = Math.max(...issueTree.map(issue => issueTerminalTimestamp(issue)!.getTime()));+ // Compaction runs once per terminal cycle, after its own delay and+ // before the archive cooldown, so a workspace the reaper cannot archive+ // yet (or ever) stops holding regenerable files.+ const compactionPending = workspaceCompactionDelayMs !== null+ && !reopenPending+ && compactedTerminalAt.get(workspace.id) !== latestTerminalAt;+ const compactionDueAt = latestTerminalAt + (workspaceCompactionDelayMs ?? 0);+ const archiveDueAt = latestTerminalAt + workspaceReaperCooldownMs;terminalWorkspaceDeadlines.set(workspace.id, reopenPending ? now().getTime()- : latestTerminalAt + workspaceReaperCooldownMs);+ : compactionPending ? Math.min(compactionDueAt, archiveDueAt) : archiveDueAt);+ if (compactionPending && compactionDueAt <= now().getTime()) {+ try {+ const outcome = await compactTerminalWorkspace(workspace, signal);+ if (outcome === "compacted") result.compacted += 1;+ if (outcome !== "busy") {+ if (compactedTerminalAt.size >= 4096) compactedTerminalAt.delete(compactedTerminalAt.keys().next().value!);+ compactedTerminalAt.set(workspace.id, latestTerminalAt);+ terminalWorkspaceDeadlines.set(workspace.id, archiveDueAt);+ }+ } catch (error) {+ if (signal?.aborted) throw error;+ result.compactionFailed += 1;+ logger.warn(+ { err: error, executionWorkspaceId: workspace.id, sourceIssueId: workspace.sourceIssueId },+ "terminal workspace compaction failed; a later sweep retries",+ );+ }+ }if (workspaceReaperCooldownMs > 0 && latestTerminalAt + workspaceReaperCooldownMs > now().getTime()) {result.skippedCooldown += 1;continue;diff --git a/server/src/services/run-failure-diagnostics.ts b/server/src/services/run-failure-diagnostics.tsindex eb53316d4e..009d7264ce 100644--- a/server/src/services/run-failure-diagnostics.ts+++ b/server/src/services/run-failure-diagnostics.ts@@ -154,14 +154,15 @@ const PUBLIC_BOOLEAN_ENV_KEYS = new Set(["CI", "NO_COLOR", "FORCE_COLOR","VITEST", "MALLOCNANOZONE", "CODEX_CI", "CODEX_SHELL", "CODEX_SAGE_BACKFILL_TRACKER_TAB_REUSE","DEV", "PROD", "SSR", "PAPERCLIP_REQUIRE_SENTRY_TEST_SDK",- "PAPERCLIP_DB_BACKUP_ENABLED", "PAPERCLIP_AUTH_DISABLE_SIGN_UP", "PAPERCLIP_ENABLE_COMPANY_DELETION",+ "PAPERCLIP_DB_BACKUP_ENABLED", "PAPERCLIP_WORKSPACE_COMPACTION_ENABLED", "PAPERCLIP_AUTH_DISABLE_SIGN_UP", "PAPERCLIP_ENABLE_COMPANY_DELETION","PAPERCLIP_STORAGE_S3_FORCE_PATH_STYLE", "PAPERCLIP_SECRETS_STRICT_MODE","SERVE_UI", "PAPERCLIP_UI_DEV_MIDDLEWARE", "HEARTBEAT_SCHEDULER_ENABLED", "PAPERCLIP_ANNOUNCEMENTS_ENABLED","GITHUB_ACTIONS", "GITHUB_REF_PROTECTED", "RUNNER_DEBUG", "ACTIONS_RUNNER_DEBUG", "ACTIONS_STEP_DEBUG",]);const PUBLIC_NUMBER_ENV_KEYS = new Set(["PORT", "PAPERCLIP_DB_BACKUP_INTERVAL_MINUTES", "PAPERCLIP_DB_BACKUP_RETENTION_DAYS",- "PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", "HEARTBEAT_SCHEDULER_INTERVAL_MS",+ "PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS", "PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS",+ "HEARTBEAT_SCHEDULER_INTERVAL_MS","GITHUB_RUN_ATTEMPT", "GITHUB_RUN_NUMBER", "GITHUB_RUN_ID", "GITHUB_RETENTION_DAYS",]);diff --git a/server/src/services/workspace-compaction.ts b/server/src/services/workspace-compaction.tsnew file mode 100644index 0000000000..e9eac8026f--- /dev/null+++ b/server/src/services/workspace-compaction.ts@@ -0,0 +1,232 @@+import { execFile } from "node:child_process";+import { randomUUID } from "node:crypto";+import fs from "node:fs/promises";+import path from "node:path";++// Directory names that hold regenerable output: installed dependencies, build+// output, and tool caches. Compaction removes a directory only when its name is+// in this set, Git ignores it, and it holds no tracked file. Removing one leaves+// the worktree in the same state as a freshly created worktree, so whatever+// produced it the first time (the workspace provision command, or the agent)+// produces it again when the task resumes.+export const REGENERABLE_WORKSPACE_DIR_NAMES: ReadonlySet<string> = new Set([+ "node_modules",+ "dist",+ "build",+ "coverage",+ "storybook-static",+ "target",+ ".cache",+ ".next",+ ".nuxt",+ ".parcel-cache",+ ".svelte-kit",+ ".turbo",+ ".vite",+]);++// Compaction never descends into these directories. `.paperclip` holds the+// worktree's own Paperclip instance data, which is not regenerable.+const PROTECTED_WORKSPACE_DIR_NAMES: ReadonlySet<string> = new Set([".git", ".paperclip"]);++// Moved directories wait here before deletion. The trash sits beside the+// worktree so the move is a same-filesystem rename, which is atomic and fast.+// A branch name cannot start with ".", so the name never collides with a+// worktree directory.+export const WORKSPACE_COMPACTION_TRASH_DIR = ".paperclip-compaction-trash";++const MAX_WALK_DEPTH = 12;++// Each Git read has a deadline and stops when the sweep is aborted, so a stalled+// Git process cannot hold the reaper.+const GIT_READ_TIMEOUT_MS = 60_000;++async function runGitRead(+ worktreePath: string,+ args: string[],+ options: { signal?: AbortSignal; input?: string; okExitCodes?: number[] } = {},+) {+ return await new Promise<string>((resolve, reject) => {+ const child = execFile(+ "git",+ ["-C", worktreePath, ...args],+ { cwd: worktreePath, timeout: GIT_READ_TIMEOUT_MS, signal: options.signal, maxBuffer: 16 * 1024 * 1024 },+ (error, stdout) => {+ const exitCode = (error as { code?: unknown } | null)?.code;+ if (!error || (typeof exitCode === "number" && options.okExitCodes?.includes(exitCode))) {+ resolve(stdout);+ return;+ }+ reject(error);+ },+ );+ // Git can exit before it reads stdin (or never reads it). A write to the+ // closed pipe then fails with EPIPE; the exit status above already reports+ // any real failure, so the stream error is not a second outcome.+ child.stdin?.on("error", () => {});+ if (options.input === undefined) child.stdin?.end();+ else child.stdin?.end(options.input);+ });+}++function splitNul(output: string) {+ return output.split("\0");+}++function isRepositoryGitignore(source: string) {+ return !path.isAbsolute(source)+ && !source.split("/").includes("..")+ && path.posix.basename(source) === ".gitignore";+}++// Keeps the candidates that Git ignores and that hold no tracked file.+// `check-ignore` answers per path, also for a path below an ignored parent.+// It exits 1 when no path is ignored, which is a normal answer.+async function filterRemovableCandidates(worktreePath: string, candidates: string[], signal?: AbortSignal) {+ // With --verbose --non-matching -z, each path yields four fields: source,+ // line number, pattern, and path. The source is empty for a path that no+ // pattern matches. Verbose output also reports a negating pattern such as+ // `!build/`, which means Git does NOT ignore the path, so only a match by a+ // non-negated pattern counts. Only the repository's own `.gitignore` files+ // count as evidence: a user's global excludes file or `.git/info/exclude`+ // says nothing about whether this repository's directory is regenerable.+ const fields = splitNul(await runGitRead(+ worktreePath,+ ["check-ignore", "--stdin", "-z", "--verbose", "--non-matching"],+ { signal, input: candidates.map((candidate) => `${candidate}\0`).join(""), okExitCodes: [1] },+ ));+ const ignored = new Set<string>();+ for (let index = 0; index + 3 < fields.length; index += 4) {+ const [source, , pattern, candidate] = fields.slice(index, index + 4);+ if (source && pattern && !pattern.startsWith("!") && isRepositoryGitignore(source)) ignored.add(candidate!);+ }+ const ignoredCandidates = candidates.filter((candidate) => ignored.has(candidate));+ if (ignoredCandidates.length === 0) return [];+ // A directory that holds a tracked file is source, even when Git ignores it.+ // Literal pathspecs: a candidate is a path, never a pattern.+ const tracked = splitNul(await runGitRead(+ worktreePath,+ ["--literal-pathspecs", "ls-files", "-z", "--", ...ignoredCandidates],+ { signal },+ ))+ .filter((entry) => entry.length > 0);+ return ignoredCandidates.filter((candidate) =>+ !tracked.some((trackedPath) => trackedPath === candidate || trackedPath.startsWith(`${candidate}/`)));+}++// Returns true only for a linked Git worktree: its `.git` entry is a file that+// points at the main repository. A main checkout has a `.git` directory, and+// compaction never touches one.+export async function isLinkedGitWorktree(worktreePath: string) {+ const stat = await fs.lstat(path.join(worktreePath, ".git")).catch(() => null);+ return stat?.isFile() === true;+}++// Lists the regenerable directories in a worktree, as paths relative to the+// worktree root. Symlinks are never followed or returned. A directory with a+// regenerable name that does not qualify (for example a tracked `build/`) is+// searched in turn, so eligible directories inside it are still found.+export async function findRegenerableWorkspaceDirs(+ worktreePath: string,+ options: { signal?: AbortSignal } = {},+): Promise<string[]> {+ const removable: string[] = [];++ async function collect(relativeDir: string, depth: number, candidates: string[]) {+ options.signal?.throwIfAborted();+ if (depth > MAX_WALK_DEPTH) return;+ const entries = await fs.readdir(path.join(worktreePath, relativeDir), { withFileTypes: true })+ .catch(() => []);+ for (const entry of entries) {+ if (!entry.isDirectory()) continue;+ if (PROTECTED_WORKSPACE_DIR_NAMES.has(entry.name)) continue;+ const relativePath = relativeDir ? path.posix.join(relativeDir, entry.name) : entry.name;+ if (REGENERABLE_WORKSPACE_DIR_NAMES.has(entry.name)) {+ candidates.push(relativePath);+ continue;+ }+ await collect(relativePath, depth + 1, candidates);+ }+ }++ let roots = [""];+ while (roots.length > 0) {+ const candidates: string[] = [];+ for (const root of roots) {+ await collect(root, root ? root.split("/").length : 0, candidates);+ }+ if (candidates.length === 0) break;+ const eligible = new Set(await filterRemovableCandidates(worktreePath, candidates, options.signal));+ removable.push(...candidates.filter((candidate) => eligible.has(candidate)));+ roots = candidates.filter((candidate) => !eligible.has(candidate));+ }+ return removable;+}++export type MovedWorkspaceDir = { relativePath: string; trashPath: string };++export type WorkspaceCompactionMove = {+ trashDir: string;+ moved: MovedWorkspaceDir[];+};++// Moves each directory into a fresh trash directory beside the worktree. Each+// move is one rename, so the whole step is fast enough to run while the caller+// holds the workspace lifecycle lock. A path that is no longer a real+// directory, or that sits on another filesystem, is left in place.+export async function moveWorkspaceDirsToTrash(+ worktreePath: string,+ relativePaths: string[],+): Promise<WorkspaceCompactionMove> {+ const trashDir = path.join(+ path.dirname(worktreePath),+ WORKSPACE_COMPACTION_TRASH_DIR,+ `${path.basename(worktreePath)}-${Date.now()}-${randomUUID().slice(0, 8)}`,+ );+ await fs.mkdir(trashDir, { recursive: true });+ const moved: MovedWorkspaceDir[] = [];+ for (const [index, relativePath] of relativePaths.entries()) {+ const sourcePath = path.join(worktreePath, relativePath);+ const stat = await fs.lstat(sourcePath).catch(() => null);+ if (!stat?.isDirectory()) continue;+ const trashPath = path.join(trashDir, String(index));+ try {+ await fs.rename(sourcePath, trashPath);+ moved.push({ relativePath, trashPath });+ } catch (error) {+ if ((error as NodeJS.ErrnoException).code === "EXDEV") continue;+ await restoreWorkspaceDirsFromTrash(worktreePath, { trashDir, moved });+ throw error;+ }+ }+ return { trashDir, moved };+}++// Puts moved directories back. A path that something recreated in the meantime+// keeps the new copy, and the moved copy stays in the trash for deletion.+export async function restoreWorkspaceDirsFromTrash(worktreePath: string, move: WorkspaceCompactionMove) {+ for (const entry of [...move.moved].reverse()) {+ const targetPath = path.join(worktreePath, entry.relativePath);+ const occupied = await fs.lstat(targetPath).then(() => true, () => false);+ if (occupied) continue;+ await fs.rename(entry.trashPath, targetPath);+ }+}++export async function deleteWorkspaceCompactionTrash(trashDir: string) {+ await fs.rm(trashDir, { recursive: true, force: true });+}++// Deletes trash directories that a crash or restart left behind. A trash entry+// younger than `olderThanMs` may belong to a compaction in progress, so it stays.+export async function purgeStaleWorkspaceCompactionTrash(worktreeParentDir: string, olderThanMs: number) {+ const trashRoot = path.join(worktreeParentDir, WORKSPACE_COMPACTION_TRASH_DIR);+ const entries = await fs.readdir(trashRoot, { withFileTypes: true }).catch(() => []);+ const cutoff = Date.now() - olderThanMs;+ for (const entry of entries) {+ const entryPath = path.join(trashRoot, entry.name);+ const stat = await fs.lstat(entryPath).catch(() => null);+ if (!stat || stat.mtimeMs > cutoff) continue;+ await fs.rm(entryPath, { recursive: true, force: true }).catch(() => {});+ }+}