The open-source app everyone uses to manage agents at work
TypeScript acpxagent-orchestrationagent-skillsagentic-aiai-agentsai-engineeringai-governanceai-toolsautonomous-agentsclaudeclaude-codeclaude-skillscodexhermesllmmcpmulti-agentopenclawproject-managementworkflow-automation
fix(runtime): continue from saved sources after workspace loss (#16690)
Dotta7c7d2d8d2cfc
diff --git a/doc/native-workspace-finalization-recovery.md b/doc/native-workspace-finalization-recovery.mdindex 3d861bb82a..570c221d0e 100644--- a/doc/native-workspace-finalization-recovery.md+++ b/doc/native-workspace-finalization-recovery.md@@ -359,3 +359,27 @@ A retained source receipt records control-plane evidence; it does not prove theprovider still holds the original bytes or that copy-back succeeded. Operatorsmust verify and recover the exact source separately. Later successful runs andcompleted tasks do not certify recovery of earlier workspace files.++### Continue after confirmed source destruction++When an old export failed and its exact sandbox has a confirmed destruction+receipt, the task can continue from surviving attachments. This is reconstruction,+not recovery of the missing edits. The task's recovery notice exposes **Continue+from saved sources** to board operators with `runtime:manage`. Select the surviving+attachments, describe the missing work, and record the observed external action+outcomes. A retained or unverified sandbox must use the existing export or file+recovery path instead.++`POST /api/issues/{id}/recovery-actions/reconstruct-workspace` records this choice.+It validates the source run, accepted result, prepared descriptor, exact destroyed+lease, current task owner and status version. It rejects active copyback owners,+competing recovery, newer executions, and attachments from another task or company.+The transaction records attachment hashes and a durable fresh-session delivery+intent. The old run, result, descriptor and failed finalization remain intact.+Repeated submissions adopt the original intent. Normal admission rechecks source+identity, attachments, authority, budgets, holds and task eligibility. It preserves+the retry count and does not grant a new work window or replay completed actions.+The agent must verify the selected attachments before rebuilding files.+Queued successors recheck their exact reconstruction receipt, sources and current+execution gates when claimed and immediately before provider handoff. A later+human block cannot be overwritten by automatic checkout.diff --git a/maps/agent-access-map/recovery.md b/maps/agent-access-map/recovery.mdindex 016f34aef4..5a764e09a7 100644--- a/maps/agent-access-map/recovery.md+++ b/maps/agent-access-map/recovery.md@@ -176,6 +176,7 @@ Review: `baseline`.API: `present` — representative operation IDs:- `POST /api/issues/{id}/recovery-actions/retry-workspace-export`.+- `POST /api/issues/{id}/recovery-actions/reconstruct-workspace`.- `POST /api/issues/{id}/recovery-actions/resume-workspace-finalization`.- `POST /api/issues/{id}/recovery-actions/resolve` for retained-source file repair.- `POST /api/execution-workspaces/{id}/reconcile-branch`.@@ -184,6 +185,10 @@ Agent access: `restricted` — Board/runtime management APIs; agent reads diagnoperforms permitted workspace work.Policy: Export retry and branch reconciliation require board runtime management.+Reconstruction also requires board runtime management. It verifies exact source+destruction and same-task attachment hashes, records lost work and external action+outcomes, and admits one fresh continuation through normal execution gates. The+old failed export remains failed. A retained or uncertain source cannot use it.Retained legacy file-only repair requires the board, exact stopped-source receipts,and recovered-file evidence. It can record repair for a successful legacy turnwithout changing its result or authorizing replay. Queued and running sourcesdiff --git a/maps/agent-access-map/workspaces.md b/maps/agent-access-map/workspaces.mdindex 6d050c32fd..ffaef2f422 100644--- a/maps/agent-access-map/workspaces.md+++ b/maps/agent-access-map/workspaces.md@@ -2,6 +2,14 @@Product map: [Execution workspaces, services, and files](../feature-map/workspaces.md).+Lost-source reconstruction adds the board-only+`POST /api/issues/{id}/recovery-actions/reconstruct-workspace` action. It requires+`runtime:manage` and ordinary task visibility. Agents cannot authorize it. The+result is a durable fresh-session admission request, subject to current execution+gates. It grants no additional tool access, budget, work window or permission to+repeat completed actions. Attachments must belong to the same task and company.+The original failed run and lost-export evidence remain unchanged.+Review date: 2026-10-12. Source commit: `58e16dbddfc9c8d0cf60867f9d9847f0feda055b`.Native REST transport reconciled in this change; see the[shared transport contract](README.md#native-rest-transport).diff --git a/maps/feature-map/workspaces.md b/maps/feature-map/workspaces.mdindex 4bda0d21c3..b1508f6778 100644--- a/maps/feature-map/workspaces.md+++ b/maps/feature-map/workspaces.md@@ -18,6 +18,14 @@ Agent access: [companion](../agent-access-map/workspaces.md).- `files-git`: inspect workspace files and repository state used for the deliverable.- `closure`: close/reopen or reconcile through permitted controls without losing work.+For a historical failed export whose exact sandbox was destroyed, the task recovery+notice offers **Continue from saved sources** to an authorized board operator.+Select surviving task attachments, record lost work and external action outcomes,+then rebuild through fresh-session admission. The old failed run remains failed.+Retained sandboxes keep their existing export recovery path. Automated coverage:+[reconstruction](../../server/src/services/native-runtime/native-workspace-reconstruction.test.ts).+Live staging acceptance remains separate from these regression tests.+## How to get to it (user POV)### `task-workspace`diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.tsindex af3387712b..7d9a9799b5 100644--- a/packages/shared/src/index.ts+++ b/packages/shared/src/index.ts@@ -2001,6 +2001,7 @@ export {resolveIssueRecoveryActionSchema,isValidExistingBranchName,retryWorkspaceExportSchema,+ reconstructWorkspaceSchema,resumeWorkspaceFinalizationSchema,issueReviewRequestSchema,issueExecutionWorkspaceSettingsSchema,diff --git a/packages/shared/src/types/execution-projection.ts b/packages/shared/src/types/execution-projection.tsindex c84678fec6..2a8fb29bbc 100644--- a/packages/shared/src/types/execution-projection.ts+++ b/packages/shared/src/types/execution-projection.ts@@ -14,6 +14,9 @@ export interface ExecutionBlocker {/** Required workspace files must be recovered before another provider turn. */workspaceRepairRequired?: boolean;canRecordWorkspaceRepair?: boolean;+ canReconstructWorkspace?: boolean;+ workspaceReconstructionPending?: boolean;+ workspaceRecoveryStatusVersion?: number;savedMessageCount?: number;}@@ -77,4 +80,12 @@ export interface ExecutionReconciliation {outcomeEvidence: string;/** Operator evidence bound to this failed run; required after workspace restore failure. */workspaceRepairEvidence?: string;+ workspaceReconstruction?: {+ schema: "paperclip.workspace-reconstruction.v1";+ leaseId: string;+ resultId: string;+ descriptorSha256: string;+ missingWork: string;+ sources: Array<{ attachmentId: string; sha256: string; byteSize: number; filename: string | null }>;+ };}diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.tsindex 8e224de092..1684fdeaa4 100644--- a/packages/shared/src/validators/index.ts+++ b/packages/shared/src/validators/index.ts@@ -441,6 +441,7 @@ export {resolveIssueRecoveryActionSchema,isValidExistingBranchName,retryWorkspaceExportSchema,+ reconstructWorkspaceSchema,resumeWorkspaceFinalizationSchema,issueReviewRequestSchema,issueExecutionWorkspaceSettingsSchema,diff --git a/packages/shared/src/validators/issue.ts b/packages/shared/src/validators/issue.tsindex 23c0fd7e10..cf398d1f8b 100644--- a/packages/shared/src/validators/issue.ts+++ b/packages/shared/src/validators/issue.ts@@ -573,6 +573,17 @@ export const retryWorkspaceExportSchema = z.object({repairNote: z.string().trim().min(20).max(12000),}).strict();+export const reconstructWorkspaceSchema = z.object({+ actionId: z.string().guid(),+ runId: z.string().guid(),+ expectedStatusVersion: z.number().int().nonnegative(),+ attachmentIds: z.array(z.string().guid()).min(1).max(20).refine(ids => new Set(ids).size === ids.length),+ missingWork: z.string().trim().min(20).max(12000),+ actionOutcome: z.enum(["completed", "not_performed", "mixed"]),+ outcomeEvidence: z.string().trim().min(20).max(12000),+ acknowledgeLostWork: z.literal(true),+}).strict();+export const resolveIssueRecoveryActionSchema = z.object({workspaceBaseRef: z.object({diff --git a/server/src/__tests__/satisfied-connection-intents.test.ts b/server/src/__tests__/satisfied-connection-intents.test.tsindex 8c8f4a5330..6b5fd7da05 100644--- a/server/src/__tests__/satisfied-connection-intents.test.ts+++ b/server/src/__tests__/satisfied-connection-intents.test.ts@@ -7,7 +7,7 @@ import { activityLog, agentWakeupRequests, agents, approvals, authUsers, compani} from "@paperclipai/db";import { createPostgresRunDispatchAdapter } from "../modules/run-dispatch/adapters/postgres.js";import { connectionIntentDeliveryService } from "../services/connection-intent-delivery.js";-import { findSatisfiedToolConnection, satisfiedConnectionIntentService } from "../services/satisfied-connection-intents.js";+import { connectionContinuationPendingResponse, findSatisfiedToolConnection, satisfiedConnectionIntentService } from "../services/satisfied-connection-intents.js";import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";import { materializeNativeInteractionResponses } from "../services/native-runtime/native-interaction-bridge.js";import { DELIVERY_QUEUES, subscribeDeliveryWork } from "../services/delivery-work-notifications.js";@@ -64,6 +64,28 @@ describePostgres("already available connection requests", () => {expect((await db.select().from(issueThreadInteractions))[0].status).toBe("pending");});+ it.each(["ready", "revoked", "question", "approval"] as const)("checks retired connection cards while a continuation is todo: %s", async state => {+ const f = await seed();+ const run = await retry(f);+ await satisfiedConnectionIntentService(db).sweepPending();+ expect((await db.select().from(issueThreadInteractions))[0].status).toBe("expired");+ await db.update(issues).set({ status: "todo" }).where(eq(issues.id, f.issueId));+ if (state === "revoked") await db.update(connectionGrants).set({ status: "revoked" }).where(eq(connectionGrants.id, f.grant.id));+ if (state === "question") await db.insert(issueThreadInteractions).values({ companyId: f.companyId, issueId: f.issueId,+ kind: "ask_user_questions", status: "pending", addresseeUserId: "connection-owner", payload: { version: 1, questions: [] } });+ if (state === "approval") {+ const [approval] = await db.insert(approvals).values({ companyId: f.companyId, type: "tool_action", status: "pending", payload: {} }).returning();+ await db.insert(issueApprovals).values({ companyId: f.companyId, issueId: f.issueId, approvalId: approval.id });+ }+ const [task] = await db.select().from(issues).where(eq(issues.id, f.issueId));+ expect(await findSatisfiedToolConnection(db, task, f.intent, run.id)).toBeNull();+ expect(await connectionContinuationPendingResponse(db, task, "connection-owner", run.id))+ .toBe(state === "ready" ? null : state === "approval" ? "approval" : "interaction");+ expect(await createPostgresRunDispatchAdapter(db).evaluateScheduledRetryGate({ companyId: f.companyId,+ runId: run.id, retryReasonOverride: "native_safe_replacement", now: new Date() }))+ .toMatchObject({ allowed: state === "ready" });+ });+it("keeps a retry under another user's identity waiting for its response", async () => {const f = await seed(); const run = await retry(f);await db.update(heartbeatRuns).set({ responsibleUserId: "someone-else" }).where(eq(heartbeatRuns.id, run.id));diff --git a/server/src/modules/run-dispatch/adapters/postgres.ts b/server/src/modules/run-dispatch/adapters/postgres.tsindex 947e7d3386..2807033653 100644--- a/server/src/modules/run-dispatch/adapters/postgres.ts+++ b/server/src/modules/run-dispatch/adapters/postgres.ts@@ -1,5 +1,6 @@import { hasConversationContinuationPolicy } from "../../../services/conversation-continuation.js";import { getExecutionBlocker } from "../../../services/execution-blocker.js";+import { validateReconstructedWorkspaceDispatch } from "../../../services/native-runtime/native-workspace-reconstruction.js";import { getNativeReviewAssignment } from "../../../services/native-runtime/native-review-participant.js";import { and, asc, eq, gte, inArray, lte, or, sql } from "drizzle-orm";import type { Db } from "@paperclipai/db";@@ -981,10 +982,27 @@ export function createPostgresRunDispatchAdapter(};}- async function decideCurrentRunStaleness(tx: Db, run: HeartbeatRun, now: Date) {+ async function decideCurrentRunStaleness(tx: Db, run: HeartbeatRun, now: Date, publications?: ActivityPublication[]) {const contextSnapshot = parseObject(run.contextSnapshot);const issueId = readNonEmptyString(contextSnapshot.issueId);if (!issueId) return { issueId: null, facts: null, executionBlocker: null, decision: { stale: false as const } };+ if (typeof contextSnapshot.workspaceReconstructionActionId === "string") {+ try {+ await validateReconstructedWorkspaceDispatch(tx, run);+ const current = await loadGateFacts({ runId: run.id, companyId: run.companyId, agentId: run.agentId,+ responsibleUserId: run.responsibleUserId, conversationContinuation: true, contextSnapshot,+ scheduledRetryReason: run.scheduledRetryReason, retryReasonOverride: "native_safe_replacement", wakeupRequestId: run.wakeupRequestId,+ }, now, tx, publications);+ if (!current.agentFound || !decideScheduledRetryGate(current.facts, now).allowed) throw new Error("reconstruction_execution_not_eligible");+ }+ catch {+ return { issueId, facts: null, executionBlocker: null, decision: { stale: true as const,+ errorCode: "execution_reconciliation_required" as const,+ reason: "Workspace reconstruction changed before execution. Inspect the saved sources and current task before continuing.",+ details: { issueId, recoveryActionId: contextSnapshot.workspaceReconstructionActionId },+ } };+ }+ }const recovery = await getExecutionBlocker(tx, run.companyId, issueId, { conversationResetCommentId: deriveCommentId(contextSnapshot) });if (recovery) return { issueId, facts: null, executionBlocker: recovery, decision: { stale: true as const,errorCode: "execution_reconciliation_required" as const, reason: recovery.nextAction,@@ -1070,9 +1088,9 @@ export function createPostgresRunDispatchAdapter(async function cancelStaleQueuedRun(input: CancelStaleQueuedRunInput,): Promise<CancelStaleQueuedRunOutcome> {- const cancelLockedRun = async (tx: Db, run: HeartbeatRun) => {+ const cancelLockedRun = async (tx: Db, run: HeartbeatRun, publications: ActivityPublication[]) => {if (run.status !== input.expectedStatus) return { outcome: "lost_race" as const };- const { issueId, facts, decision, executionBlocker } = await decideCurrentRunStaleness(tx, run, input.now);+ const { issueId, facts, decision, executionBlocker } = await decideCurrentRunStaleness(tx, run, input.now, publications);if (!decision.stale || !issueId) {if (input.expectedStatus === "queued" && facts?.isInteractionWake) {// Preserve the authority accepted under the issue/run locks. Later@@ -1109,7 +1127,7 @@ export function createPostgresRunDispatchAdapter(async function dispatchResolvedInteractionIfCurrent<T>(input: DispatchResolvedInteractionInput<T>,): Promise<DispatchResolvedInteractionOutcome<T>> {- const dispatchLockedRun = async (tx: Db, run: HeartbeatRun) => {+ const dispatchLockedRun = async (tx: Db, run: HeartbeatRun, publications: ActivityPublication[]) => {if (run.status !== input.expectedStatus || run.resultJson?.cancellation || run.resultJson?.startupCancellation) {return { dispatched: false as const, cancellation: { outcome: "lost_race" as const } };}@@ -1117,6 +1135,7 @@ export function createPostgresRunDispatchAdapter(tx,run,input.now,+ publications,);const decision =!initialDecision.stale && issueIddiff --git a/server/src/routes/issues.ts b/server/src/routes/issues.tsindex eacaea637c..6852b42280 100644--- a/server/src/routes/issues.ts+++ b/server/src/routes/issues.ts@@ -19,6 +19,8 @@ import { createIssueReadTiming } from "../services/issue-read-timing.js";import { isNativeWorkspaceExportRepairCause } from "@paperclipai/shared";import { resumeNativeWorkspaceAfterOwnerStop } from "../services/native-runtime/native-workspace-owner-recovery.js";import { retryNativeWorkspaceExport } from "../services/native-runtime/native-workspace-export-retry.js";+import { reconstructNativeWorkspace, unreconciledNativeExportPredicate } from "../services/native-runtime/native-workspace-reconstruction.js";+import { reconstructWorkspaceSchema } from "@paperclipai/shared";import { queuedInteractionId, readQueuedInteractionResponse, hasQueuedInteractionResponse } from "../services/queued-interaction-response.js";import { deliverConversationComments, isConversation } from "../services/agent-conversations.js";import { issueRecoveryActionReadModel } from "../services/issue-recovery-actions.js";@@ -9501,11 +9503,11 @@ export function issueRoutes(});const retainedHolds = await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.companyId, issue.companyId), eq(issueRecoveryActions.sourceIssueId, issue.id),- sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`,- or(+ or(unreconciledNativeExportPredicate(), and(+ sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`, or(sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,inArray(issueRecoveryActions.cause, [...EXECUTION_RECONCILIATION_CAUSES]),- ),+ ))),)).orderBy(desc(issueRecoveryActions.createdAt));res.json({active,@@ -9537,6 +9539,17 @@ export function issueRoutes(res.status(202).json(receipt);});+ router.post("/issues/:id/recovery-actions/reconstruct-workspace", validate(reconstructWorkspaceSchema), async (req, res) => {+ assertBoard(req);+ const issue = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Issue not found");+ if (!issue || !(await assertIssueReadAllowed(req, res, issue))) return;+ const decision = await access.decide({ actor: req.actor, action: "runtime:manage", resource: { type: "company", companyId: issue.companyId } });+ if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision));+ const receipt = await reconstructNativeWorkspace({ db, companyId: issue.companyId, issueId: issue.id,+ actorId: getActorInfo(req).actorId, request: req.body });+ res.status(202).json(receipt);+ });+router.post("/issues/:id/recovery-actions/resolve",validate(resolveIssueRecoveryActionSchema),diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.tsindex 120312ba87..d51610850f 100644--- a/server/src/routes/openapi.ts+++ b/server/src/routes/openapi.ts@@ -243,6 +243,7 @@ import {createAcceptedPlanDecompositionSchema,resolveIssueRecoveryActionSchema,retryWorkspaceExportSchema,+ reconstructWorkspaceSchema,resumeWorkspaceFinalizationSchema,cancelIssueThreadInteractionSchema,// Secret provider configs and remote import@@ -10668,6 +10669,15 @@ registerCurrentRoute({responses: { 202: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict },});+registerCurrentRoute({+ method: "post",+ path: "/api/issues/{id}/recovery-actions/reconstruct-workspace",+ tags: ["issues"],+ summary: "Continue from saved task attachments after confirmed destruction of the original workspace",+ body: reconstructWorkspaceSchema,+ responses: { 202: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict },+});+registerCurrentRoute({method: "post",path: "/api/issues/{id}/recovery-actions/resolve",diff --git a/server/src/services/execution-blocker.ts b/server/src/services/execution-blocker.tsindex 38d615dfbd..2d2ae87c85 100644--- a/server/src/services/execution-blocker.ts+++ b/server/src/services/execution-blocker.ts@@ -8,6 +8,9 @@ import { canRetryStoppedRun } from "./cancelled-native-startup.js";import { queuedCommentIdsFromWakePayload } from "./issue-queued-comment-queue.js";import { EXECUTION_RECONCILIATION_CAUSES, type ExecutionBlocker } from "@paperclipai/shared";import { hasRequiredWorkspaceRecovery, LEGACY_WORKSPACE_RECOVERY_SCHEMA, nativeRunHasNoWorkspaceAccess } from "./workspace-restore-recovery-state.js";+import { unreconciledNativeExportPredicate } from "./native-runtime/native-workspace-reconstruction.js";+import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";+import { readNativeWorkspaceSyncReference } from "./native-runtime/native-workspace-sync.js";export async function hasPendingRunCleanup(db: Db, companyId: string, runId: string): Promise<boolean> {const [state] = await db.execute<{ pending: boolean }>(sql`SELECT@@ -40,16 +43,18 @@ export async function hasPendingRunCleanup(db: Db, companyId: string, runId: strexport function executionBlockerPredicate() {return and(not(conversationRecoveryActionPredicate()!),- inArray(issueRecoveryActions.cause, [...EXECUTION_RECONCILIATION_CAUSES]),- or(inArray(issueRecoveryActions.status, ["active", "escalated"]),- sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`),+ or(unreconciledNativeExportPredicate(), and(+ inArray(issueRecoveryActions.cause, [...EXECUTION_RECONCILIATION_CAUSES]),+ or(inArray(issueRecoveryActions.status, ["active", "escalated"]),+ sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`),+ )),);}export async function getExecutionBlocker(db: Db, companyId: string, issueId: string, options?: { conversationResetCommentId?: string | null }): Promise<ExecutionBlocker | null> {const [conversation] = await db.select({ agentId: issues.conversationAgentId,boundaryId: issues.conversationBoundaryCommentId, status: issues.status,- assigneeAgentId: issues.assigneeAgentId }).from(issues).where(and(+ assigneeAgentId: issues.assigneeAgentId, statusVersion: issues.statusVersion }).from(issues).where(and(eq(issues.companyId, companyId), eq(issues.id, issueId),)).limit(1);// Resetting model context cannot repair or recover a workspace. This hold@@ -59,6 +64,7 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: steq(issueRecoveryActions.sourceIssueId, issueId),executionBlockerPredicate(),or(+ unreconciledNativeExportPredicate(),sql`${issueRecoveryActions.evidence}->>'workspaceRestoreFailure' = 'restore_unsafe_archive'`,sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,),@@ -94,6 +100,17 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st)).limit(1) : [];const withoutWorkspace = Boolean(run?.driverKind === "grok_bot_mcp" && nativeRunHasNoWorkspaceAccess(run));const workspaceRepairRequired = Boolean(restoreHold && !withoutWorkspace);+ const reconstructableExport = ["native_workspace_sync_out_unrecoverable", "native_workspace_sync_out_retry_exhausted"].includes(action.cause);+ const exportSource = reconstructableExport ? readNativeWorkspaceSyncReference(run?.runnerProfileJson?.nativeWorkspaceSync) : null;+ const [exportLease] = exportSource ? await db.select().from(environmentLeases).where(and(+ eq(environmentLeases.companyId, companyId), eq(environmentLeases.heartbeatRunId, run!.id), eq(environmentLeases.id, exportSource.leaseId),+ eq(environmentLeases.providerLeaseId, exportSource.providerLeaseId),+ )).limit(1) : [];+ const canReconstructWorkspace = Boolean(exportLease && hasRemoteTerminationReceipt(exportLease)+ && (exportLease.metadata?.remoteExecutionTermination as { state?: string } | undefined)?.state === "destroyed"+ && !action.evidence.executionReconciliation && conversation?.status === "blocked" && conversation.assigneeAgentId === run?.agentId);+ const reconstructionPending = Boolean((action.evidence.executionReconciliation as { workspaceReconstruction?: unknown } | undefined)?.workspaceReconstruction+ && action.evidence.continuationDelivery === "pending");const queue = await db.select({ payload: agentWakeupRequests.payload }).from(agentWakeupRequests).where(and(eq(agentWakeupRequests.companyId, companyId), eq(agentWakeupRequests.status, "deferred_issue_execution"),sql`${agentWakeupRequests.payload}->>'issueId' = ${issueId}`,@@ -163,12 +180,14 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st// A stopped reviewer can differ from the task owner who receives the work back.agentId: run?.agentId ?? null,cause: action.cause,- nextAction,+ nextAction: reconstructionPending ? "Continuation from saved sources is queued and waiting for execution checks." : nextAction,runStatus: run?.status ?? null,runError: runError?.slice(0, 1024) ?? null,canContinue: eligibleContinuation && !chatBinding,canRetry: canRetry && !chatBinding,...(restoreHold ? { workspaceRepairRequired, canRecordWorkspaceRepair: workspaceRepairRequired && hasRequiredWorkspaceRecovery(action.evidence) } : {}),savedMessageCount: saved.length,+ ...(canReconstructWorkspace ? { canReconstructWorkspace, workspaceRecoveryStatusVersion: conversation!.statusVersion } : {}),+ ...(reconstructionPending ? { workspaceReconstructionPending: true } : {}),};}diff --git a/server/src/services/explicit-native-continuation.ts b/server/src/services/explicit-native-continuation.tsindex aabb3e1b78..46686f7d7b 100644--- a/server/src/services/explicit-native-continuation.ts+++ b/server/src/services/explicit-native-continuation.ts@@ -144,6 +144,9 @@ export async function admitExplicitNativeContinuation(input: {if (!actions.length) return null;const blocker = await getExecutionBlocker(db, companyId, issueId);if (blocker && blocker.recoveryActionId === null) return null;+ if (blocker?.workspaceRepairRequired && ["native_workspace_sync_out_unrecoverable", "native_workspace_sync_out_retry_exhausted"].includes(blocker.cause)) {+ return blocked("workspace_reconstruction_required", "Recover the saved files or record reconstruction before continuing. Your message is saved.");+ }const [pendingInteraction] = await db.select({ id: issueThreadInteractions.id }).from(issueThreadInteractions).where(and(eq(issueThreadInteractions.companyId, companyId), eq(issueThreadInteractions.issueId, issueId),eq(issueThreadInteractions.status, "pending"),diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.tsindex 81632b1b83..474559cd29 100644--- a/server/src/services/heartbeat.ts+++ b/server/src/services/heartbeat.ts@@ -2173,7 +2173,7 @@ export function heartbeatService(await issuesSvc.checkout(issueId,agent.id,- ["todo", "backlog", "blocked"],+ typeof context.workspaceReconstructionActionId === "string" ? ["todo"] : ["todo", "backlog", "blocked"],run.id,);context[PAPERCLIP_HARNESS_CHECKOUT_KEY] = true;@@ -2844,6 +2844,7 @@ export function heartbeatService(if (!issueId ||(!isResolvedInteractionContinuationWakeContext(context) &&+ typeof context.workspaceReconstructionActionId !== "string" &&!["native_safe_replacement", "native_provider_overloaded"].includes(run.scheduledRetryReason ?? ""))) {return { dispatched: true, resultPromise: dispatch(() => {}) };diff --git a/server/src/services/heartbeat/queue.ts b/server/src/services/heartbeat/queue.tsindex be493ce9b4..758910670f 100644--- a/server/src/services/heartbeat/queue.ts+++ b/server/src/services/heartbeat/queue.ts@@ -33,6 +33,7 @@ import {} from "../explicit-native-continuation.js";import { canRetryStoppedRun } from "../cancelled-native-startup.js";import { getExecutionBlocker, hasPendingRunCleanup } from "../execution-blocker.js";+import { validateNativeWorkspaceReconstruction } from "../native-runtime/native-workspace-reconstruction.js";import { isConversationAdapter } from "../conversation-continuation.js";import { recordExecutionWait } from "../execution-wait.js";import {@@ -3102,6 +3103,13 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDependeconst [reconciledRun] = await tx.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, issue.companyId), eq(heartbeatRuns.id, sourceRunId),));+ if (decision.workspaceReconstruction) {+ await validateNativeWorkspaceReconstruction(tx as unknown as Db, {+ companyId: issue.companyId, issueId: issue.id, runId: sourceRunId, actionId,+ });+ reconciledRestoreRetryCount = executionFailureRetryCount(reconciledRun!);+ enrichedContextSnapshot.workspaceReconstructionActionId = action.id;+ }if (hasWorkspaceRestoreFailure(reconciledRun?.resultJson)) {if ((readNonEmptyString(decision.workspaceRepairEvidence)?.length ?? 0) < 20)return { kind: "skipped" as const };@@ -3170,7 +3178,9 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende// A bound chat request has already rechecked its current principal// above. Treat its new user message like a board comment, but keep// failed-run retry actions on their separate exact-request path.- if (executionBlocker && !(await admitExplicitNativeContinuation({+ const ownsReconstructionContinuation = reconciledSourceRunId && executionBlocker?.runId === reconciledSourceRunId+ && executionBlocker.recoveryActionId === enrichedContextSnapshot.recoveryActionId;+ if (executionBlocker && !ownsReconstructionContinuation && !(await admitExplicitNativeContinuation({db: tx as unknown as Db, companyId: issue.companyId, issueId: issue.id,agentId, actorType: opts.requestedByActorType, actorId: opts.requestedByActorId,reason: durableRequest && !failedChatRetry ? "issue_commented" : reason,@@ -3968,7 +3978,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDependequeuedCommentInterruptId: opts.queuedCommentInterruptId,queuedCommentRequestId: opts.queuedCommentRequestId,});- if (!explicitContinuation && executionBlocker) return deferBlockedExecution(executionBlocker);+ if (!explicitContinuation && executionBlocker && !ownsReconstructionContinuation) return deferBlockedExecution(executionBlocker);if (explicitContinuation) {enrichedContextSnapshot.forceFreshSession = true;enrichedContextSnapshot.previousRunId = explicitContinuation.previousRunId;@@ -4013,7 +4023,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende// in separate turns, or finalization will deliver it a second time.const pendingComments =(executionWaitRequestId || canCoalesceComments) &&- !(await getExecutionBlocker(tx as unknown as Db, issue.companyId, issue.id))+ (ownsReconstructionContinuation || !(await getExecutionBlocker(tx as unknown as Db, issue.companyId, issue.id)))? await tx.select().from(agentWakeupRequests)@@ -4095,6 +4105,14 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende.returning().then((rows) => rows[0]);+ if (ownsReconstructionContinuation) {+ await tx.update(issueRecoveryActions).set({ evidence: sql`${issueRecoveryActions.evidence} || ${JSON.stringify({+ continuationDelivery: "delivered", continuationRunId: newRun.id,+ })}::jsonb` }).where(and(eq(issueRecoveryActions.companyId, issue.companyId),+ eq(issueRecoveryActions.id, String(enrichedContextSnapshot.recoveryActionId)),+ sql`${issueRecoveryActions.evidence}->>'continuationDelivery' = 'pending'`));+ }+await tx.update(agentWakeupRequests).set({diff --git a/server/src/services/native-runtime/native-workspace-reconstruction.test.ts b/server/src/services/native-runtime/native-workspace-reconstruction.test.tsnew file mode 100644index 0000000000..52a65d4b78--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-reconstruction.test.ts@@ -0,0 +1,222 @@+import { randomUUID } from "node:crypto";+import express from "express";+import request from "supertest";+import { eq, sql } from "drizzle-orm";+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";+import { activityLog, agents, agentWakeupRequests, assets, authUsers, companies, completionContracts, createDb, environmentLeases, heartbeatRuns, issueAttachments, issueComments, issueRecoveryActions, issues, nativeRunFinalizations, nativeRunResults } from "@paperclipai/db";+import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";+import { errorHandler } from "../../middleware/index.js";+import { issueRoutes } from "../../routes/issues.js";+import { getExecutionBlocker } from "../execution-blocker.js";+import { admitExplicitNativeContinuation } from "../explicit-native-continuation.js";+import { deliverReconciledExecutions } from "../execution-recovery-resolution.js";+import { heartbeatService } from "../heartbeat.js";+import { remoteTerminationReceipt } from "../remote-execution-termination.js";+import { reconstructNativeWorkspace } from "./native-workspace-reconstruction.js";+import { createRunDispatch } from "../../modules/run-dispatch/index.js";+import { drainHeartbeatRunsToQuiescence } from "../../__tests__/helpers/drain-heartbeat-runs.js";+const providerExecute = vi.hoisted(() => vi.fn(async () => ({ exitCode: 0, signal: null, timedOut: false,+ errorMessage: null, summary: "Bounded reconstruction completed.", provider: "test", model: "test" })));+vi.mock("../../adapters/index.js", async () => ({+ ...await vi.importActual<typeof import("../../adapters/index.js")>("../../adapters/index.js"),+ getServerAdapter: () => ({ supportsLocalAgentJwt: false, execute: providerExecute }),+}));++describe("reconstruction after confirmed native workspace loss", () => {+ let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;+ let db: ReturnType<typeof createDb>;+ let runningHeartbeat: ReturnType<typeof heartbeatService> | null = null;+ beforeAll(async () => {+ temporary = await startEmbeddedPostgresTestDatabase("workspace-reconstruction-");+ db = createDb(temporary.connectionString);+ }, 30_000);+ afterAll(async () => { await temporary.cleanup(); });+ afterEach(async () => {+ if (runningHeartbeat) await drainHeartbeatRunsToQuiescence(db, runningHeartbeat);+ runningHeartbeat = null;+ providerExecute.mockClear();+ await db.transaction(async tx => {+ await tx.execute(sql`set local client_min_messages = warning`);+ await tx.execute(sql`truncate companies cascade`);+ }); });++ async function seed() {+ const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID(), resultId = randomUUID(),+ contractId = randomUUID(), leaseId = randomUUID(), actionId = randomUUID(), providerLeaseId = randomUUID(), userId = randomUUID(), attachmentId = randomUUID(), assetId = randomUUID();+ await db.insert(authUsers).values({ id: userId, name: "Operator", email: `${userId}@example.test`, emailVerified: true, createdAt: new Date(), updatedAt: new Date() });+ await db.insert(companies).values({ id: companyId, name: "Reconstruction", issuePrefix: `REC${companyId.replaceAll("-", "").slice(0, 6)}`, defaultResponsibleUserId: userId, budgetMonthlyCents: 10000 });+ await db.insert(agents).values({ id: agentId, companyId, name: "Worker", status: "idle", adapterType: "paperclip_runner", budgetMonthlyCents: 5000, runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } } });+ await db.insert(issues).values({ id: issueId, companyId, title: "Finish bounded work", status: "blocked", assigneeAgentId: agentId });+ await db.insert(completionContracts).values({ id: contractId, companyId, issueId, revision: 1, schemaVersion: "paperclip.completion-contract.v1", policyVersion: "test", risk: "standard", completionAuthority: "server_arbiter", incompleteCriteriaPolicy: "preserve_non_terminal", contractJson: { objective: "Finish bounded work" }, canonicalSha256: contractId, createdByActorType: "system", createdByActorId: "test" });+ await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "failed", runtimeMode: "native", nativeIssueId: issueId, nativePhase: "terminal_failure", completionContractId: contractId,+ createdAt: new Date("2026-05-01T00:00:00Z"), finishedAt: new Date("2026-05-01T01:30:00Z"), responsibleUserId: userId, scheduledRetryAttempt: 2,+ contextSnapshot: { issueId, originalWorkDeadline: "2026-05-01T01:30:00Z" }, errorCode: "native_session_interrupted", processPid: process.pid,+ runnerProfileJson: { nativeWorkspaceSync: { schema: "paperclip.native-workspace-sync/v1", state: "prepared", descriptorSha256: "a".repeat(64), baselineSha256: "b".repeat(64), finalHostSha256: null, workspaceId: randomUUID(), leaseId, providerLeaseId, remoteCwd: "/work", resourceDisposition: "destroy" } } });+ await db.insert(nativeRunResults).values({ id: resultId, companyId, issueId, runId, completionContractId: contractId, serverFingerprint: resultId, schemaStatus: "accepted", resultJson: { work: "Unfinished local edits; no published PR" }, canonicalSha256: resultId });+ const cause = "native_workspace_sync_out_unrecoverable";+ await db.insert(nativeRunFinalizations).values({ runId, companyId, issueId, phase: "terminal_failure", resultId, failureCode: cause });+ const identity = { id: leaseId, companyId, heartbeatRunId: runId, provider: "daytona", providerLeaseId };+ await db.insert(environmentLeases).values({ ...identity, issueId, status: "expired", releasedAt: new Date(), cleanupStatus: "success", metadata: { remoteExecutionTermination: remoteTerminationReceipt(identity, { providerLeaseId, state: "destroyed" }) } });+ await db.insert(issueRecoveryActions).values({ id: actionId, companyId, sourceIssueId: issueId, kind: "active_run_watchdog", ownerType: "board", returnOwnerAgentId: agentId, cause, fingerprint: runId, evidence: { runId }, nextAction: "Recover saved files", status: "resolved", outcome: "restored", resolutionNote: "new_source_execution_path" });+ await db.insert(assets).values({ id: assetId, companyId, provider: "local", objectKey: assetId, contentType: "text/plain", byteSize: 100, sha256: "c".repeat(64), originalFilename: "saved-baseline.ts" });+ await db.insert(issueAttachments).values({ id: attachmentId, companyId, issueId, assetId });+ const [issue] = await db.select().from(issues).where(eq(issues.id, issueId));+ const body = { actionId, runId, expectedStatusVersion: issue.statusVersion, attachmentIds: [attachmentId], missingWork: "The unexported patch is lost. Rebuild only from this earlier verified baseline.", actionOutcome: "not_performed" as const, outcomeEvidence: "The saved execution receipt and remote repository show no published PR or submitted external change.", acknowledgeLostWork: true as const };+ const input = { db, companyId, issueId, actorId: userId, request: body };+ return { ...input, body, agentId, leaseId, resultId, runId, actionId, providerLeaseId, attachmentId, assetId };+ }+ function app(actor: unknown = { type: "board", source: "local_implicit" }) {+ const app = express(); app.use(express.json());+ app.use((req, _res, next) => { (req as unknown as { actor: unknown }).actor = actor; next(); });+ app.use("/api", issueRoutes(db, {} as never)); app.use(errorHandler); return app;+ }+ it("keeps a falsely resolved export visible and prevents conversation reset from hiding it", async () => {+ const f = await seed();+ expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({ runId: f.runId, workspaceRepairRequired: true, canReconstructWorkspace: true });+ const commentId = randomUUID();+ await db.insert(issueComments).values({ id: commentId, companyId: f.companyId, issueId: f.issueId, authorType: "user", authorUserId: f.actorId, body: "/new" });+ await db.update(issues).set({ conversationAgentId: f.agentId, conversationUserId: f.actorId, conversationState: "active" }).where(eq(issues.id, f.issueId));+ expect(await getExecutionBlocker(db, f.companyId, f.issueId, { conversationResetCommentId: commentId })).not.toBeNull();+ expect(await admitExplicitNativeContinuation({ db, companyId: f.companyId, issueId: f.issueId, agentId: f.agentId,+ actorType: "user", actorId: f.actorId, reason: "issue_commented", commentId, successorRunId: randomUUID() })).toBeNull();+ const response = await request(app()).get(`/api/issues/${f.issueId}/recovery-actions`).expect(200);+ expect(response.body.actions).toEqual(expect.arrayContaining([expect.objectContaining({ id: f.actionId })]));+ });+ it("records partial sources atomically, preserves failed history and budgets, and deduplicates concurrent submissions", async () => {+ const f = await seed();+ const [before] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));+ await Promise.all([reconstructNativeWorkspace(f), reconstructNativeWorkspace(f)]);+ expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId)))[0]).toEqual(before);+ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, f.runId)))[0]).toMatchObject({ phase: "terminal_failure", resultId: f.resultId });+ expect((await db.select().from(nativeRunResults).where(eq(nativeRunResults.id, f.resultId)))[0].schemaStatus).toBe("accepted");+ const [record] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));+ expect(record).toMatchObject({ outcome: "handed_back", evidence: { continuationDelivery: "pending", executionReconciliation: { workspaceReconstruction: { missingWork: f.body.missingWork, sources: [{ attachmentId: f.attachmentId, sha256: "c".repeat(64) }] } } } });+ expect(record.evidence.executionReconciliation).not.toHaveProperty("workspaceRepairEvidence");+ expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({ runId: f.runId, workspaceRepairRequired: true });+ expect(await db.select().from(activityLog).where(eq(activityLog.entityId, f.issueId))).toHaveLength(1);+ expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId))).toHaveLength(0);+ expect((await db.select().from(companies).where(eq(companies.id, f.companyId)))[0].budgetMonthlyCents).toBe(10000);+ expect((await db.select().from(agents).where(eq(agents.id, f.agentId)))[0].budgetMonthlyCents).toBe(5000);+ });+ it.each(["retained", "receipt", "lease", "owner", "coordinator", "copyback", "newer", "status", "version", "attachment", "result"] as const)("rejects stale or unsafe reconstruction: %s", async kind => {+ const f = await seed();+ const [lease] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, f.leaseId));+ if (kind === "retained") await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination: remoteTerminationReceipt(lease, { providerLeaseId: f.providerLeaseId, state: "stopped" }) } }).where(eq(environmentLeases.id, f.leaseId));+ if (kind === "receipt") await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination: { ...(lease.metadata!.remoteExecutionTermination as object), runId: randomUUID() } } }).where(eq(environmentLeases.id, f.leaseId));+ if (kind === "lease") await db.update(environmentLeases).set({ status: "pending_cleanup" }).where(eq(environmentLeases.id, f.leaseId));+ if (kind === "owner") await db.update(issues).set({ assigneeAgentId: null }).where(eq(issues.id, f.issueId));+ if (kind === "coordinator") await db.update(nativeRunFinalizations).set({ leaseOwner: "other-worker" }).where(eq(nativeRunFinalizations.runId, f.runId));+ if (kind === "copyback") await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: { token: "other-worker" } } }).where(eq(heartbeatRuns.id, f.runId));+ if (kind === "newer") await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, contextSnapshot: { issueId: f.issueId }, status: "succeeded" });+ if (kind === "status") await db.update(issues).set({ status: "done" }).where(eq(issues.id, f.issueId));+ if (kind === "version") f.request.expectedStatusVersion += 1;+ if (kind === "attachment") f.request.attachmentIds = [(await seed()).attachmentId];+ if (kind === "result") await db.update(nativeRunResults).set({ schemaStatus: "rejected" }).where(eq(nativeRunResults.id, f.resultId));+ await expect(reconstructNativeWorkspace(f)).rejects.toThrow();+ expect(await db.select().from(activityLog).where(eq(activityLog.entityId, f.issueId))).toHaveLength(0);+ expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId)))[0].evidence).not.toHaveProperty("executionReconciliation");+ });+ it("exposes board recovery through the assembled API and rejects missing loss acknowledgement", async () => {+ const f = await seed();+ await request(app()).post(`/api/issues/${f.issueId}/recovery-actions/reconstruct-workspace`).send({ ...f.body, acknowledgeLostWork: false }).expect(400);+ await request(app({ type: "agent", companyId: f.companyId, agentId: f.agentId, runId: f.runId })).post(`/api/issues/${f.issueId}/recovery-actions/reconstruct-workspace`).send(f.body).expect(403);+ await request(app()).post(`/api/issues/${f.issueId}/recovery-actions/reconstruct-workspace`).send(f.body).expect(202);+ });+ it.each(["status", "attachment", "paused"] as const)("keeps delivery pending when %s changes after reconstruction is recorded", async kind => {+ const f = await seed();+ await reconstructNativeWorkspace(f);+ if (kind === "status") await db.update(issues).set({ status: "blocked" }).where(eq(issues.id, f.issueId));+ if (kind === "attachment") await db.delete(issueAttachments).where(eq(issueAttachments.id, f.attachmentId));+ if (kind === "paused") await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agentId));+ const heartbeat = heartbeatService(db, { runtimeEnv: {} });+ await deliverReconciledExecutions(db, heartbeat.wakeup);+ expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agentId))).toHaveLength(1);+ expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId)))[0].evidence.continuationDelivery).toBe("pending");+ });+ it("admits one fresh continuation through public heartbeat, keeps retry budget, and adopts a lost acknowledgement", async () => {+ const f = await seed();+ await reconstructNativeWorkspace(f);+ await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, status: "running" });+ const heartbeat = heartbeatService(db, { runtimeEnv: {} });+ const wake = vi.fn(heartbeat.wakeup);+ let successorId: string | undefined;+ let admissionError: unknown;+ await deliverReconciledExecutions(db, async (...args) => {+ let successor;+ try { successor = await wake(...args); } catch (error) { admissionError = error; throw error; }+ expect(successor).not.toBeNull(); successorId = successor!.id;+ expect(successor).toMatchObject({ retryOfRunId: f.runId, scheduledRetryAttempt: 2, contextSnapshot: { forceFreshSession: true, previousRunId: f.runId } });+ throw new Error("lost acknowledgement");+ });+ if (admissionError) throw admissionError;+ const queued = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId));+ expect(successorId, JSON.stringify(queued.map(row => ({ status: row.status, reason: row.reason, payload: row.payload })))).toBeDefined();+ await deliverReconciledExecutions(db, wake);+ expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.idempotencyKey, `execution-reconciliation:${f.actionId}`))).toHaveLength(1);+ expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId)))[0].evidence).toMatchObject({ continuationDelivery: "delivered", continuationRunId: successorId });+ expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toBeNull();+ });+ it.each(["unchanged", "blocked", "attachment", "paused", "forged_successor"] as const)("rechecks a queued reconstruction at the final provider handoff: %s", async kind => {+ const f = await seed();+ await reconstructNativeWorkspace(f);+ await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, status: "running" });+ const heartbeat = heartbeatService(db, { runtimeEnv: {} });+ await deliverReconciledExecutions(db, heartbeat.wakeup);+ const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));+ const successorId = String(action.evidence.continuationRunId);+ expect(successorId).not.toBe("undefined");+ await db.update(heartbeatRuns).set({ status: "running" }).where(eq(heartbeatRuns.id, successorId));+ await db.update(issues).set({ status: "in_progress", executionRunId: successorId }).where(eq(issues.id, f.issueId));+ if (kind === "blocked") await db.update(issues).set({ status: "blocked" }).where(eq(issues.id, f.issueId));+ if (kind === "attachment") await db.delete(issueAttachments).where(eq(issueAttachments.id, f.attachmentId));+ if (kind === "paused") await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agentId));+ if (kind === "forged_successor") await db.update(issueRecoveryActions).set({ evidence: { ...action.evidence, continuationRunId: randomUUID() } }).where(eq(issueRecoveryActions.id, f.actionId));+ const dispatch = vi.fn(async () => "provider entered");+ const outcome = await createRunDispatch(db).dispatchResolvedInteractionIfCurrent({ companyId: f.companyId,+ runId: successorId, expectedStatus: "running", dispatch });+ expect(outcome.dispatched).toBe(kind === "unchanged");+ expect(dispatch).toHaveBeenCalledTimes(kind === "unchanged" ? 1 : 0);+ if (!outcome.dispatched) expect(outcome.cancellation).toMatchObject({ outcome: "cancelled" });+ });+ it.each([false, true])("runs the assembled heartbeat without overwriting a block after claim: %s", async blockAfterClaim => {+ const f = await seed();+ await db.update(agents).set({ adapterType: "codex_local" }).where(eq(agents.id, f.agentId));+ await reconstructNativeWorkspace(f);+ const [busy] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, status: "running" }).returning();+ const heartbeat = runningHeartbeat = heartbeatService(db, { runtimeEnv: { PAPERCLIP_IN_WORKTREE: "false" } });+ await deliverReconciledExecutions(db, heartbeat.wakeup);+ const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));+ const successorId = String(action.evidence.continuationRunId);+ expect(successorId).not.toBe("undefined");+ if (blockAfterClaim) await db.execute(sql.raw(`+ create or replace function block_reconstruction_after_claim() returns trigger as $$+ begin+ if NEW.id = '${successorId}'::uuid and NEW.status = 'running' then+ update issues set status = 'blocked' where id = '${f.issueId}'::uuid;+ end if;+ return NEW;+ end; $$ language plpgsql;+ create trigger block_reconstruction_after_claim after update of status on heartbeat_runs+ for each row execute function block_reconstruction_after_claim();+ `));+ try {+ await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, busy.id));+ await heartbeat.resumeQueuedRuns();+ const deadline = Date.now() + 15000;+ let current;+ do {+ [current] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, successorId));+ if (["succeeded", "failed", "cancelled"].includes(current.status)) break;+ await new Promise(resolve => setTimeout(resolve, 25));+ } while (Date.now() < deadline);+ expect(providerExecute).toHaveBeenCalledTimes(blockAfterClaim ? 0 : 1);+ if (blockAfterClaim) {+ expect(current).toMatchObject({ status: "cancelled", errorCode: "execution_reconciliation_required" });+ expect((await db.select().from(issues).where(eq(issues.id, f.issueId)))[0].status).toBe("blocked");+ } else expect(current.status).toBe("succeeded");+ } finally {+ if (blockAfterClaim) await db.execute(sql`drop trigger block_reconstruction_after_claim on heartbeat_runs; drop function block_reconstruction_after_claim()`);+ await db.update(issues).set({ status: "done" }).where(eq(issues.id, f.issueId));+ }+ }, 30_000);+});diff --git a/server/src/services/native-runtime/native-workspace-reconstruction.ts b/server/src/services/native-runtime/native-workspace-reconstruction.tsnew file mode 100644index 0000000000..a238dd95b2--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-reconstruction.ts@@ -0,0 +1,152 @@+import { and, eq, gt, inArray, ne, or, sql } from "drizzle-orm";+import { assets, environmentLeases, heartbeatRuns, issueAttachments, issueRecoveryActions, issues, nativeRunFinalizations, nativeRunResults, type Db } from "@paperclipai/db";+import { reconstructWorkspaceSchema, type ExecutionReconciliation } from "@paperclipai/shared";+import type { z } from "zod";+import { conflict } from "../../errors.js";+import { persistActivity, publishActivity } from "../activity-log.js";+import { markExecutionReconciliation } from "../execution-recovery-resolution.js";+import { hasRemoteTerminationReceipt } from "../remote-execution-termination.js";+import { readNativeWorkspaceSyncReference } from "./native-workspace-sync.js";+import { nativeSha256 } from "./canonical.js";++const causes = ["native_workspace_sync_out_unrecoverable", "native_workspace_sync_out_retry_exhausted"];+const object = (value: unknown): Record<string, unknown> => value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};+const changed = () => conflict("The lost workspace recovery changed. Refresh the task before continuing.", { code: "workspace_reconstruction_stale" });++export function unreconciledNativeExportPredicate() {+ return and(+ inArray(issueRecoveryActions.cause, causes),+ or(and(+ or(inArray(issueRecoveryActions.status, ["active", "escalated"]), and(+ eq(issueRecoveryActions.status, "resolved"), eq(issueRecoveryActions.resolutionNote, "new_source_execution_path"),+ )),+ sql`${issueRecoveryActions.evidence}->'executionReconciliation'->'workspaceReconstruction' is null`,+ ), and(eq(issueRecoveryActions.status, "resolved"),+ sql`${issueRecoveryActions.evidence}->'executionReconciliation'->'workspaceReconstruction'->>'schema' = 'paperclip.workspace-reconstruction.v1'`,+ sql`${issueRecoveryActions.evidence}->>'continuationDelivery' = 'pending'`,+ )),+ sql`exists (select 1 from ${heartbeatRuns} where ${heartbeatRuns.companyId} = ${issueRecoveryActions.companyId}+ and ${heartbeatRuns.nativeIssueId} = ${issueRecoveryActions.sourceIssueId}+ and ${heartbeatRuns.id}::text = ${issueRecoveryActions.evidence}->>'runId'+ and ${heartbeatRuns.runtimeMode} = 'native' and ${heartbeatRuns.status} = 'failed'+ and ${heartbeatRuns.runnerProfileJson}->'nativeWorkspaceSync'->>'state' = 'prepared')`,+ );+}++export async function validateNativeWorkspaceReconstruction(db: Db, input: {+ companyId: string; issueId: string; runId: string; actionId: string;+ successorRunId?: string;+}) {+ const [issue] = await db.select().from(issues).where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.issueId))).for("update");+ const [coordinator] = await db.select().from(nativeRunFinalizations).where(and(eq(nativeRunFinalizations.companyId, input.companyId), eq(nativeRunFinalizations.runId, input.runId))).for("update");+ const [run] = await db.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, input.companyId), eq(heartbeatRuns.id, input.runId))).for("update");+ const [action] = await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.sourceIssueId, input.issueId), eq(issueRecoveryActions.id, input.actionId))).for("update");+ if (!issue || !run || !coordinator || !action || run.runtimeMode !== "native" || run.nativeIssueId !== issue.id+ || run.status !== "failed" || run.nativePhase !== "terminal_failure" || coordinator.issueId !== issue.id+ || coordinator.phase !== "terminal_failure" || coordinator.leaseOwner || coordinator.failureDetail?.successorRunId+ || object(run.runnerProfileJson).nativeWorkspaceFinalizationOwner+ || action.kind !== "active_run_watchdog" || action.ownerType !== "board" || !causes.includes(action.cause)+ || action.evidence.runId !== run.id || coordinator.failureCode !== action.cause+ || !["active", "escalated", "resolved"].includes(action.status)+ || (action.status === "resolved" && action.resolutionNote !== "new_source_execution_path" && !object(action.evidence.executionReconciliation).workspaceReconstruction)+ || issue.assigneeAgentId !== run.agentId || action.returnOwnerAgentId !== run.agentId+ || !(input.successorRunId ? ["todo", "in_progress"] : ["blocked", "todo"]).includes(issue.status)+ || (issue.executionRunId && issue.executionRunId !== input.successorRunId)+ || (issue.checkoutRunId && issue.checkoutRunId !== input.successorRunId)) throw changed();+ const reference = readNativeWorkspaceSyncReference(object(run.runnerProfileJson).nativeWorkspaceSync);+ if (!reference || reference.state !== "prepared") throw changed();+ const leases = await db.select().from(environmentLeases).where(and(eq(environmentLeases.companyId, input.companyId), eq(environmentLeases.heartbeatRunId, run.id))).for("update");+ const lease = leases.find(row => row.id === reference.leaseId);+ if (!lease || lease.issueId !== issue.id || lease.providerLeaseId !== reference.providerLeaseId+ || !leases.every(hasRemoteTerminationReceipt) || object(lease.metadata?.remoteExecutionTermination).state !== "destroyed"+ || leases.some(row => row.metadata?.pendingCleanupInFlight === true)) {+ throw conflict("Paperclip has not confirmed destruction of the original workspace. Recover its files or retry its export first.", { code: "workspace_reconstruction_source_not_destroyed" });+ }+ const [otherLease] = await db.select({ id: environmentLeases.id }).from(environmentLeases).where(and(+ ne(environmentLeases.id, lease.id), eq(environmentLeases.provider, lease.provider!), eq(environmentLeases.providerLeaseId, lease.providerLeaseId!),+ inArray(environmentLeases.status, ["active", "retained", "pending_cleanup"]),+ )).limit(1);+ const [newer] = await db.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(and(+ eq(heartbeatRuns.companyId, input.companyId), ne(heartbeatRuns.id, run.id),+ input.successorRunId ? ne(heartbeatRuns.id, input.successorRunId) : undefined,+ or(eq(heartbeatRuns.nativeIssueId, issue.id), sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${issue.id}`),+ or(gt(heartbeatRuns.createdAt, run.createdAt), inArray(heartbeatRuns.status, ["queued", "running"])),+ )).limit(1);+ const [otherAction] = await db.select({ id: issueRecoveryActions.id }).from(issueRecoveryActions).where(and(+ eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.sourceIssueId, issue.id),+ ne(issueRecoveryActions.id, action.id), inArray(issueRecoveryActions.status, ["active", "escalated"]),+ )).limit(1);+ const [result] = await db.select().from(nativeRunResults).where(and(+ eq(nativeRunResults.companyId, input.companyId), eq(nativeRunResults.issueId, issue.id), eq(nativeRunResults.runId, run.id),+ eq(nativeRunResults.id, coordinator.resultId ?? "00000000-0000-0000-0000-000000000000"),+ eq(nativeRunResults.completionContractId, run.completionContractId ?? "00000000-0000-0000-0000-000000000000"), eq(nativeRunResults.schemaStatus, "accepted"),+ )).limit(1);+ if (otherLease || newer || otherAction || !result) throw changed();+ const decision = action.evidence.executionReconciliation as ExecutionReconciliation | undefined;+ if (decision?.workspaceReconstruction) {+ const reconstruction = decision.workspaceReconstruction;+ if ((!input.successorRunId && issue.status !== "todo") || decision.runId !== run.id || reconstruction.schema !== "paperclip.workspace-reconstruction.v1"+ || (input.successorRunId && (action.evidence.continuationRunId !== input.successorRunId || action.evidence.continuationDelivery !== "delivered"))+ || reconstruction.leaseId !== lease.id || reconstruction.resultId !== result.id+ || reconstruction.descriptorSha256 !== reference.descriptorSha256 || !reconstruction.sources?.length) throw changed();+ const files = await db.select({ attachmentId: issueAttachments.id, sha256: assets.sha256 }).from(issueAttachments)+ .innerJoin(assets, and(eq(assets.id, issueAttachments.assetId), eq(assets.companyId, input.companyId)))+ .where(and(eq(issueAttachments.companyId, input.companyId), eq(issueAttachments.issueId, input.issueId),+ inArray(issueAttachments.id, reconstruction.sources.map(source => source.attachmentId)))).for("share");+ if (files.length !== reconstruction.sources.length || reconstruction.sources.some(source => !files.some(file => file.attachmentId === source.attachmentId && file.sha256 === source.sha256))) throw changed();+ }+ return { issue, run, action, reference, lease, result };+}++export async function validateReconstructedWorkspaceDispatch(db: Db, run: typeof heartbeatRuns.$inferSelect) {+ const context = object(run.contextSnapshot);+ if (typeof context.workspaceReconstructionActionId !== "string") return;+ const actionId = context.workspaceReconstructionActionId;+ if (typeof context.issueId !== "string" || typeof context.previousRunId !== "string"+ || context.source !== "execution.reconciled" || context.recoveryActionId !== actionId || context.forceFreshSession !== true+ || run.retryOfRunId !== context.previousRunId) throw changed();+ const source = await validateNativeWorkspaceReconstruction(db, { companyId: run.companyId, issueId: context.issueId,+ runId: context.previousRunId, actionId, successorRunId: run.id });+ if (source.run.agentId !== run.agentId || !object(source.action.evidence.executionReconciliation).workspaceReconstruction) throw changed();+}++export async function reconstructNativeWorkspace(input: {+ db: Db; companyId: string; issueId: string; actorId: string;+ request: z.infer<typeof reconstructWorkspaceSchema>;+}) {+ const request = reconstructWorkspaceSchema.parse(input.request);+ const requestSha256 = nativeSha256(request);+ const outcome = await input.db.transaction(async tx => {+ const db = tx as unknown as Db;+ const [issue] = await tx.select().from(issues).where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.issueId))).for("update");+ const [prior] = await tx.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.sourceIssueId, input.issueId), eq(issueRecoveryActions.id, request.actionId)));+ if (prior?.evidence.workspaceReconstructionRequest === requestSha256) return { replayed: true, publication: null };+ if (!issue || issue.status !== "blocked" || issue.statusVersion !== request.expectedStatusVersion || prior?.evidence.executionReconciliation) throw changed();+ const source = await validateNativeWorkspaceReconstruction(db, { ...input, ...request });+ const files = await tx.select({ attachmentId: issueAttachments.id, sha256: assets.sha256, byteSize: assets.byteSize, filename: assets.originalFilename })+ .from(issueAttachments).innerJoin(assets, and(eq(assets.id, issueAttachments.assetId), eq(assets.companyId, input.companyId)))+ .where(and(eq(issueAttachments.companyId, input.companyId), eq(issueAttachments.issueId, input.issueId), inArray(issueAttachments.id, request.attachmentIds)));+ if (files.length !== request.attachmentIds.length || files.some(file => !/^[a-f0-9]{64}$/.test(file.sha256) || file.byteSize <= 0)) {+ throw conflict("Choose saved attachments from this task before reconstructing its workspace.", { code: "workspace_reconstruction_sources_invalid" });+ }+ const decision: ExecutionReconciliation = { runId: request.runId, providerStopped: true,+ actionOutcome: request.actionOutcome, outcomeEvidence: request.outcomeEvidence,+ workspaceReconstruction: { schema: "paperclip.workspace-reconstruction.v1", leaseId: source.lease.id,+ resultId: source.result.id, descriptorSha256: source.reference.descriptorSha256, missingWork: request.missingWork, sources: files } };+ await markExecutionReconciliation(db, source.action, decision, input.actorId);+ const now = new Date();+ await tx.update(issueRecoveryActions).set({ status: "resolved", outcome: "handed_back", resolvedAt: now, updatedAt: now,+ resolutionNote: "workspace_reconstruction_authorized", nextAction: "Continue from saved sources. The original unexported edits are lost; verify the selected attachments before rebuilding. Do not repeat completed external actions or reset the original work limits.",+ evidence: sql`${issueRecoveryActions.evidence} || ${JSON.stringify({ workspaceReconstructionRequest: requestSha256 })}::jsonb`,+ wakePolicy: null, monitorPolicy: null,+ }).where(eq(issueRecoveryActions.id, source.action.id));+ await tx.update(issues).set({ status: "todo", unblockDescriptor: null, blockedTransitionAt: null,+ blockedOwnerNotifiedAt: null, updatedAt: now }).where(eq(issues.id, issue.id));+ const publication = await persistActivity(db, { companyId: input.companyId, actorType: "user", actorId: input.actorId,+ action: "issue.workspace_reconstruction_authorized", entityType: "issue", entityId: input.issueId, runId: request.runId,+ details: { recoveryActionId: request.actionId, ...decision } });+ return { replayed: false, publication };+ });+ if (outcome.publication) publishActivity(outcome.publication.publication);+ return { runId: request.runId, actionId: request.actionId, status: "queued" as const, replayed: outcome.replayed };+}diff --git a/server/src/services/satisfied-connection-intents.ts b/server/src/services/satisfied-connection-intents.tsindex 49d295ae97..f6a3eb39ef 100644--- a/server/src/services/satisfied-connection-intents.ts+++ b/server/src/services/satisfied-connection-intents.ts@@ -11,10 +11,10 @@ type Task = Pick<typeof issues.$inferSelect, "id" | "companyId" | "projectId" |type Intent = Pick<typeof issueThreadInteractions.$inferSelect, "id" | "companyId" | "issueId" | "kind" | "status" | "payload" | "addresseeUserId"> & Partial<Pick<typeof issueThreadInteractions.$inferSelect, "sourceRunId">>;/** Verify existing access only. This never installs tools, grants access, or adopts an identity. */-export async function findSatisfiedToolConnection(db: Db, task: Task, intent: Intent, policyRunId?: string | null) {+async function findAvailableToolConnection(db: Db, task: Task, intent: Intent, policyRunId?: string | null) {if (intent.kind !== "connection_intent" || intent.status !== "pending"|| intent.companyId !== task.companyId || intent.issueId !== task.id- || task.assigneeUserId || !["in_progress", "in_review"].includes(task.status)) return null;+ || task.assigneeUserId) return null;const parsed = connectionIntentPayloadSchema.safeParse(intent.payload);if (!parsed.success) return null;const payload = parsed.data;@@ -42,6 +42,11 @@ export async function findSatisfiedToolConnection(db: Db, task: Task, intent: Inreturn access.allowedCount + access.askFirstCount > 0 ? connection : null;}+export async function findSatisfiedToolConnection(db: Db, task: Task, intent: Intent, policyRunId?: string | null) {+ if (!["in_progress", "in_review"].includes(task.status)) return null;+ return findAvailableToolConnection(db, task, intent, policyRunId);+}+/** Recheck both live requests and system-retired cards against the next run. */export async function connectionContinuationPendingResponse(db: Db, task: Task, responsibleUserId: string | null,policyRunId?: string | null): Promise<"interaction" | "approval" | null> {@@ -55,7 +60,7 @@ export async function connectionContinuationPendingResponse(db: Db, task: Task,if (interaction.status !== "pending" && !retired) continue;if (retired && (interaction.payload as { requestingAgentId?: string }).requestingAgentId !== task.assigneeAgentId) continue;if (interaction.addresseeUserId !== responsibleUserId) return "interaction";- const connection = await findSatisfiedToolConnection(db, task, { ...interaction, status: "pending" }, policyRunId);+ const connection = await findAvailableToolConnection(db, task, { ...interaction, status: "pending" }, policyRunId);// Historical system retirements describe a service need, not a permanent// binding to an account the user may later replace. Delivery and native// response projection still validate their own exact recorded connection.diff --git a/ui/src/api/issues.ts b/ui/src/api/issues.tsindex 40881632c3..af057efd11 100644--- a/ui/src/api/issues.ts+++ b/ui/src/api/issues.ts@@ -287,6 +287,9 @@ export const issuesApi = {api.post<{ runId: string; resultId: string; status: "queued" }>(`/issues/${id}/recovery-actions/resume-workspace-finalization`, data),retryWorkspaceExport: (id: string, data: { actionId: string; runId: string; repairNote: string }) =>api.post<{ runId: string; resultId: string; leaseId: string; status: "queued" }>(`/issues/${id}/recovery-actions/retry-workspace-export`, data),+ reconstructWorkspace: (id: string, data: { actionId: string; runId: string; expectedStatusVersion: number; attachmentIds: string[];+ missingWork: string; actionOutcome: "completed" | "not_performed" | "mixed"; outcomeEvidence: string; acknowledgeLostWork: true }) =>+ api.post<{ runId: string; actionId: string; status: "queued"; replayed: boolean }>(`/issues/${id}/recovery-actions/reconstruct-workspace`, data),recoveryActions: (id: string) =>api.get<{ active: IssueRecoveryAction | null; actions: IssueRecoveryAction[] }>(`/issues/${id}/recovery-actions`),resolveRecoveryAction: (diff --git a/ui/src/components/ExecutionBlockerNotice.tsx b/ui/src/components/ExecutionBlockerNotice.tsxindex f6f82de177..676d3c347f 100644--- a/ui/src/components/ExecutionBlockerNotice.tsx+++ b/ui/src/components/ExecutionBlockerNotice.tsx@@ -1,6 +1,7 @@import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";import type { ExecutionBlocker, Issue } from "@paperclipai/shared";import { WorkspaceFileRecovery } from "./WorkspaceFileRecovery";+import { WorkspaceReconstructionRecovery } from "./WorkspaceReconstructionRecovery";import { agentsApi } from "../api/agents";import { activityApi } from "../api/activity";import { issuesApi } from "../api/issues";@@ -50,15 +51,16 @@ export function ExecutionBlockerNotice({ companyId, issueId, issueStatus, canManreturn (<div role="status" aria-label="Task recovery" className={cn("task-context-notice flex flex-wrap items-center justify-between gap-2", className)}><div className="min-w-0 flex-1 break-words">- <p>{modelRejected ? "Model unavailable." : "Recovery needed."}{blocker.runError ? ` ${blocker.runError}` : ""}</p>+ <p>{blocker.workspaceReconstructionPending ? "Reconstruction queued." : modelRejected ? "Model unavailable." : "Recovery needed."}{blocker.runError && !blocker.workspaceReconstructionPending ? ` ${blocker.runError}` : ""}</p>{modelRejected && <p>Choose a supported model or clear the task's model override, then retry.</p>}{blocker.workspaceRepairRequired ? (<>{/* The stored next action is an operator runbook step; lead with what happened. This hold also coversunsafe archives and local runs, so do not claim retries or a retained sandbox here. */}<p>- Paperclip could not restore the agent's workspace files after its last run. A board operator must- recover the files before the agent can continue.+ {blocker.workspaceReconstructionPending ? blocker.nextAction : blocker.canReconstructWorkspace+ ? "The workspace was destroyed before its edits were saved. A board operator can continue from surviving source attachments."+ : "Paperclip could not restore the agent's workspace files after its last run. A board operator must recover the files before the agent can continue."}</p><details><summary className="cursor-pointer">Technical details</summary>@@ -94,6 +96,8 @@ export function ExecutionBlockerNotice({ companyId, issueId, issueStatus, canMan{recoveryError && <p role="alert" className="w-full text-destructive">Unable to load recovery details: {recoveryError.message}</p>}{issueStatus && <WorkspaceFileRecovery key={`${blocker.recoveryActionId}:${blocker.runId}`} issueId={issueId}issueStatus={issueStatus} blocker={blocker} canManage={canManage} onRecorded={onRetried} />}+ {blocker.canReconstructWorkspace && <WorkspaceReconstructionRecovery key={`reconstruct:${blocker.recoveryActionId}:${blocker.runId}`}+ issueId={issueId} blocker={blocker} canManage={canManage} onRecorded={onRetried} />}</div>);}diff --git a/ui/src/components/WorkspaceReconstructionRecovery.test.tsx b/ui/src/components/WorkspaceReconstructionRecovery.test.tsxnew file mode 100644index 0000000000..eb025f2f8a--- /dev/null+++ b/ui/src/components/WorkspaceReconstructionRecovery.test.tsx@@ -0,0 +1,74 @@+// @vitest-environment jsdom+import { act } from "react";+import { createRoot, type Root } from "react-dom/client";+import { QueryClient, QueryClientProvider } from "@tanstack/react-query";+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";+import { WorkspaceReconstructionRecovery } from "./WorkspaceReconstructionRecovery";+import { queryKeys } from "../lib/queryKeys";+const api = vi.hoisted(() => ({ listAttachments: vi.fn(), reconstructWorkspace: vi.fn() }));+vi.mock("../api/issues", () => ({ issuesApi: api }));++describe("rebuild a destroyed workspace from partial sources", () => {+ let root: Root, container: HTMLDivElement, client: QueryClient;+ const onRecorded = vi.fn();+ beforeEach(() => {+ vi.clearAllMocks();+ api.listAttachments.mockResolvedValue([{ id: "source", originalFilename: "baseline.ts", byteSize: 100 }]);+ api.reconstructWorkspace.mockResolvedValue({ status: "queued" });+ container = document.createElement("div"); document.body.append(container); root = createRoot(container);+ client = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } });+ });+ afterEach(async () => { await act(async () => root.unmount()); client.clear(); container.remove(); });+ async function mount(canManage = true) {+ await act(async () => root.render(<QueryClientProvider client={client}><WorkspaceReconstructionRecovery issueId="task" canManage={canManage}+ blocker={{ recoveryActionId: "recovery", runId: "lost-run", agentId: "agent", cause: "native_workspace_sync_out_unrecoverable", nextAction: "", workspaceRecoveryStatusVersion: 3 }} onRecorded={onRecorded} />+ </QueryClientProvider>));+ await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); });+ }+ async function fill() {+ await act(async () => {+ for (const area of container.querySelectorAll("textarea")) {+ Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, "value")!.set!.call(area, "Verified source and receipts show what must be rebuilt without repeating published work.");+ area.dispatchEvent(new Event("input", { bubbles: true }));+ }+ const checkboxes = container.querySelectorAll<HTMLButtonElement>('[role="checkbox"]');+ checkboxes[0].click(); checkboxes[1].click();+ });+ }+ it("requires surviving sources, lost-work acknowledgement and outcome evidence without claiming original-file recovery", async () => {+ await mount();+ const button = Array.from(container.querySelectorAll("button")).find(button => button.textContent === "Rebuild and continue")!;+ expect(button.disabled).toBe(true);+ expect(container.textContent).toContain("missing edits have not been recovered");+ expect(container.textContent).toContain("Existing work limits still apply");+ await fill(); expect(button.disabled).toBe(false);+ await act(async () => button.click());+ expect(api.reconstructWorkspace).toHaveBeenCalledWith("task", expect.objectContaining({ runId: "lost-run", expectedStatusVersion: 3, attachmentIds: ["source"], acknowledgeLostWork: true, actionOutcome: "mixed" }));+ expect(onRecorded).toHaveBeenCalledOnce();+ });+ it("preserves the reconstruction decision when current source checks reject it", async () => {+ api.reconstructWorkspace.mockRejectedValue(new Error("The source owner changed."));+ await mount(); await fill();+ await act(async () => Array.from(container.querySelectorAll("button")).find(button => button.textContent === "Rebuild and continue")!.click());+ await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); });+ expect(container.querySelector('[role="alert"]')?.textContent).toContain("source owner changed");+ expect(container.querySelector("textarea")?.value).toContain("Verified source");+ expect(onRecorded).not.toHaveBeenCalled();+ });+ it("offers no reconstruction form to a reader without runtime management authority", async () => {+ await mount(false); expect(container.querySelector("button")).toBeNull();+ expect(api.listAttachments).not.toHaveBeenCalled(); expect(api.reconstructWorkspace).not.toHaveBeenCalled();+ });+ it("refreshes uploaded sources with the task attachment cache and drops deleted selections", async () => {+ await mount(); await fill();+ await act(async () => client.setQueryData(queryKeys.issues.attachments("task"), [+ { id: "new-source", originalFilename: "uploaded-source.ts", byteSize: 200 },+ ]));+ await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); });+ expect(container.textContent).toContain("uploaded-source.ts");+ expect(container.textContent).not.toContain("baseline.ts");+ const button = Array.from(container.querySelectorAll("button")).find(button => button.textContent === "Rebuild and continue")!;+ expect(button.disabled).toBe(true);+ expect(api.reconstructWorkspace).not.toHaveBeenCalled();+ });+});diff --git a/ui/src/components/WorkspaceReconstructionRecovery.tsx b/ui/src/components/WorkspaceReconstructionRecovery.tsxnew file mode 100644index 0000000000..b3a1da8bcf--- /dev/null+++ b/ui/src/components/WorkspaceReconstructionRecovery.tsx@@ -0,0 +1,67 @@+import { useId, useState } from "react";+import { useMutation, useQuery } from "@tanstack/react-query";+import type { ExecutionBlocker } from "@paperclipai/shared";+import { issuesApi } from "../api/issues";+import { queryKeys } from "../lib/queryKeys";+import { Button } from "./ui/button";+import { Checkbox } from "./ui/checkbox";+import { Label } from "./ui/label";+import { Textarea } from "./ui/textarea";+import { NativeSelect } from "./ui/select";++export function WorkspaceReconstructionRecovery({ issueId, blocker, canManage, onRecorded }: {+ issueId: string; blocker: ExecutionBlocker; canManage: boolean; onRecorded: () => void;+}) {+ const id = useId();+ const [selected, setSelected] = useState<string[]>([]);+ const [missingWork, setMissingWork] = useState("");+ const [outcomeEvidence, setOutcomeEvidence] = useState("");+ const [actionOutcome, setActionOutcome] = useState<"completed" | "not_performed" | "mixed">("mixed");+ const [acknowledged, setAcknowledged] = useState(false);+ const files = useQuery({ queryKey: queryKeys.issues.attachments(issueId),+ queryFn: () => issuesApi.listAttachments(issueId), enabled: canManage });+ const currentSelection = selected.filter(id => files.data?.some(file => file.id === id));+ const submit = useMutation({+ mutationFn: () => issuesApi.reconstructWorkspace(issueId, {+ actionId: blocker.recoveryActionId!, runId: blocker.runId!, expectedStatusVersion: blocker.workspaceRecoveryStatusVersion!,+ attachmentIds: currentSelection, missingWork: missingWork.trim(), actionOutcome, outcomeEvidence: outcomeEvidence.trim(), acknowledgeLostWork: true,+ }), onSuccess: onRecorded,+ });+ return <details className="w-full">+ <summary className="cursor-pointer">Continue from saved sources</summary>+ <p>The original workspace was destroyed before its edits were saved. Its last run will remain failed.</p>+ {canManage ? <div className="flex flex-col gap-2">+ <p>Select task attachments to rebuild from and record what remains missing. Paperclip starts a fresh session through its normal execution checks. Existing work limits still apply.</p>+ <fieldset disabled={submit.isPending} className="flex flex-col gap-2">+ <legend>Saved source attachments</legend>+ {files.data?.map(file => <div key={file.id} className="flex items-center gap-2">+ <Checkbox id={`${id}-${file.id}`} checked={selected.includes(file.id)} onCheckedChange={checked =>+ setSelected(current => checked === true ? [...current, file.id] : current.filter(value => value !== file.id))} />+ <Label htmlFor={`${id}-${file.id}`}>{file.originalFilename ?? "Attachment"} ({file.byteSize} bytes)</Label>+ </div>)}+ {files.isPending && <p>Loading saved attachments…</p>}+ {files.data?.length === 0 && <p>Attach the surviving source files to this task first.</p>}+ <Label htmlFor={`${id}-missing`}>Missing work and reconstruction plan</Label>+ <Textarea id={`${id}-missing`} maxLength={12000} value={missingWork} onChange={event => setMissingWork(event.target.value)} />+ <Label htmlFor={`${id}-outcome`}>Previously completed external actions</Label>+ <NativeSelect id={`${id}-outcome`} value={actionOutcome} onChange={event => setActionOutcome(event.target.value as typeof actionOutcome)}>+ <option value="mixed">Some completed, some not performed</option>+ <option value="not_performed">None performed</option>+ <option value="completed">All completed</option>+ </NativeSelect>+ <Label htmlFor={`${id}-evidence`}>Evidence and actions that must not be repeated</Label>+ <Textarea id={`${id}-evidence`} maxLength={12000} value={outcomeEvidence} onChange={event => setOutcomeEvidence(event.target.value)} />+ <div className="flex items-center gap-2">+ <Checkbox id={`${id}-ack`} checked={acknowledged} onCheckedChange={checked => setAcknowledged(checked === true)} />+ <Label htmlFor={`${id}-ack`}>Continue from these sources. The missing edits have not been recovered.</Label>+ </div>+ </fieldset>+ <div className="flex justify-end">+ <Button disabled={submit.isPending || !acknowledged || currentSelection.length === 0 || missingWork.trim().length < 20 || outcomeEvidence.trim().length < 20}+ onClick={() => submit.mutate()}>{submit.isPending ? "Queuing…" : "Rebuild and continue"}</Button>+ </div>+ {files.isError && <p role="alert" className="text-destructive">{files.error.message}</p>}+ {submit.isError && <p role="alert" className="text-destructive">{submit.error.message}</p>}+ </div> : <p>A board member with runtime access can continue from the surviving task attachments.</p>}+ </details>;+}