The open-source app everyone uses to manage agents at work
TypeScript acpxagent-orchestrationagent-skillsagentic-aiai-agentsai-engineeringai-governanceai-toolsautonomous-agentsclaudeclaude-codeclaude-skillscodexhermesllmmcpmulti-agentopenclawproject-managementworkflow-automation
fix: recover saved workspace results automatically (#15824)
Dotta0b0e53029fa5
diff --git a/doc/native-workspace-finalization-recovery.md b/doc/native-workspace-finalization-recovery.mdindex 0120131541..a9a6e6365f 100644--- a/doc/native-workspace-finalization-recovery.md+++ b/doc/native-workspace-finalization-recovery.md@@ -35,6 +35,46 @@ leave tar/Git children alive, so incomplete copyback requires operator stopverification even on the same host. PID reuse is checked against its recordedstart time rather than trusted by PID alone.+## Automatic cloud recovery++New Railway copyback receipts record the exact project, environment, service and+deployment ID. On takeover, the existing finalization reconciler asks the cloud+harness to inspect that deployment. The harness resolves the stack's owned+resources independently and issues a short-lived Ed25519 receipt only when the+exact deployment is `REMOVED` with `deploymentStopped: true`. This covers all+processes in the deployment, including orphaned copyback children. Missing,+truncated, live or unavailable provider evidence leaves the physical fence intact.++The tenant verifies the configured harness key, disjoint receipt audience, stack,+run, owner token, deployment identity and expiry. It records proof and an activity+entry when replacing only the exact owner under the existing advisory lock. The+normal reconciler resumes the accepted result; it does not create a provider turn+or change approval/monitor waits. A changed owner during verification fails closed.++Stop proof alone does not authorize old work to overwrite a current task. After+the provider read, automatic recovery uses the operator path's shared binding+checks under short coordinator, issue and run row locks, atomically with the+owner claim. The task must still be open, assigned to the run or its authoritative+review participant, and bound to that running native execution and accepted+result. A different checkout, newer execution or queued/running legacy execution+prevents takeover. Publication checks repeat this validation; row locks are never+held across physical copyback I/O.++The harness endpoint is `POST /v1/tenant/workspace-worker-stop`. The fleet-wide+server token authorizes a read request, never stop evidence: responses must be+signed by the harness. This endpoint requires the companion cloud harness change.+Deploying only the application keeps unknown owners fenced. No cloud provider+credentials are delivered to tenant applications.++The task view shows only “Restoring saved work…” after the server confirms recovery+or an operator queues it,+then removes the notice when commitment resolves recovery. It exposes no process+identifiers, shutdown form or manual resume control. Legacy receipts without+platform identity and unsupported platforms say operator assistance is needed;+operators use the documented exact-owner API below after independent stop proof.+The three diagnosed historical receipts do not gain a deployment identity merely+by upgrading. Never infer one from elapsed time or a hostname.+## Unverified copyback after controller replacementGeneric orphan recovery does not turn a missing provider PID into `process_lost`@@ -66,6 +106,48 @@ exact prior controller **and its copyback subprocesses** have stopped. Retain thsandbox, accepted native result, and workspace descriptor. Do not run a newprovider turn, delete workspace contents, or relax archive confinement.+For legacy receipts, an authorized company owner, company administrator or+instance administrator records independently verified platform evidence through+the operator API. The task has no shutdown form or resume control. The API is+`POST /api/issues/:id/recovery-actions/resume-workspace-finalization` with:++```json+{+ "actionId": "<recovery-action-id>",+ "runId": "<run-id>",+ "ownerToken": "<owner-token-from-recovery-evidence>",+ "controllerAndCopybackStopped": true,+ "stopEvidence": "<deployment-platform evidence for this controller and its children>"+}+```++This is an operator attestation, not an automatic deployment-platform probe.+The separate `runtime:confirm_owner_stop` authorization action limits this+attestation to active company owners or administrators, the trusted local board,+or an instance administrator. Ordinary runtime managers and agents cannot release+the physical owner receipt. Missing role data keeps the client notice read-only.+A+missing advisory connection alone is insufficient. Admission rejects an active+copyback lock, an observably live local controller, a changed owner, a newer run,+or an unbound result. It records the exact owner and operator evidence atomically+with releasing the receipt. The accepted result, workspace descriptor, source+lease, task status, monitors, and approval waits are unchanged. The ordinary+reconciler performs copyback and applies the saved result through the status+arbiter. No provider wake is enqueued. The recovery action remains visible until+finalization commits. Admission supports saved-result copyback from `result_accepted`,+`workspace_finalizing`, and `retryable_failure`; an abandoned retry does not need+a replacement run. The existing retry schedule remains unchanged.+Repeating the same confirmation before reconciliation is+idempotent; it cannot release a newer owner. Generic recovery resolution, including+false-positive dispositions, cannot dismiss this hold or bypass saved-result+finalization.++Repeated sweeps preserve these copyback-specific instructions, including for+older actions whose recovery budget was already exhausted. They must never+recommend starting a replacement provider run for this hold.++For older servers without this action, the maintenance fallback follows.+After that platform verification, use a database maintenance transaction torelease only the exact receipt shown by the recovery action. Replace the fourplaceholders with the action's company, run, token, and source issue. The advisory@@ -138,8 +220,8 @@ A board member with runtime management access can complete the saved result:the export failure while preserving all user files. Restore transport accessor other provider prerequisites before retrying. Extending the sandboxretention window can be necessary during this repair.-3. In the task's **Workspace export needs repair** notice, describe the repair and- choose **Retry workspace export**. The equivalent board API is+3. Use the operator API to record the repair and request export-only retry. The+ task shows operator assistance is needed, without an infrastructure form. The board API is`POST /api/issues/:id/recovery-actions/retry-workspace-export` with`{ "actionId": "<recovery-action-id>", "runId": "<run-id>", "repairNote": "<repair and preservation evidence>" }`.4. Confirm the same run commits, the saved result determines the task outcome,diff --git a/maps/agent-access-map/recovery.md b/maps/agent-access-map/recovery.mdindex 1b90fa083d..c7cc042f96 100644--- a/maps/agent-access-map/recovery.md+++ b/maps/agent-access-map/recovery.md@@ -129,23 +129,40 @@ Review: `baseline`.API: `present` — representative operation IDs:- `POST /api/issues/{id}/recovery-actions/retry-workspace-export`.+- `POST /api/issues/{id}/recovery-actions/resume-workspace-finalization`.- `POST /api/execution-workspaces/{id}/reconcile-branch`.Agent access: `restricted` — Board/runtime management APIs; agent reads diagnosis andperforms permitted workspace work.-Policy: Task/run visibility, run ownership and source/action policy; workspace export-retry and some operator controls require board.--Evidence: `source-only` — [implementation](../../server/src/routes/issues.ts#L9478) at-the reviewed commit above.--Verification: `not-run` — assert the documented agent denial with an owned test fixture;+Policy: Export retry and branch reconciliation require board runtime management.+Physical-owner stop confirmation uses the separate `runtime:confirm_owner_stop`+action. It requires active company owner/admin membership, trusted local board+access, or instance administration. Agents and ordinary runtime managers are denied.+The exact source task, running native run, accepted result and owner receipt must+match; active copyback locks, live local writers, and newer runs prevent admission.+An authorized administrator must first verify through the deployment platform that+the exact controller and all copyback children stopped, then attest that evidence.+Elapsed time, an expired coordinator lease or a lost advisory connection is not proof.+Only the exact owner receipt is released; normal reconciliation performs export and+commitment without another provider turn or bypassing existing task waits.++Evidence: `source-only` — [route](../../server/src/routes/issues.ts),+[administrator authorization](../../server/src/services/authorization.ts), and+[owner admission](../../server/src/services/native-runtime/native-workspace-owner-recovery.ts)+reviewed for this change. Local authorization, route, UI and finalization regressions+cover member/agent denial, exact-owner admission and the real reconciler preserving+monitor and approval waits. This is not live provider-stop proof.++Verification: `not-run` for live agent/deployment execution — assert the documented agent denial with an owned test fixture;verify any listed agent request/result path separately. A delegated operation needs itsown policy and access review.Gaps: Export retry and branch reconciliation require board runtime:manage at-their HTTP routes. Native `call_api` preserves these denials.+their HTTP routes. The exact-owner operator fallback additionally requires+runtime:confirm_owner_stop. Native call_api preserves these agent denials.+Automatic managed-cloud recovery uses independently verified, signed platform+proof through the finalization reconciler; agents cannot attest owner shutdown.## `unknown-outcome`diff --git a/maps/feature-map/recovery.md b/maps/feature-map/recovery.mdindex 3cd71abaff..7d5a5945d6 100644--- a/maps/feature-map/recovery.md+++ b/maps/feature-map/recovery.md@@ -119,7 +119,22 @@ Automated: [disposition notices](../../ui/src/components/DispositionRecoveryNotitests pending/error/gated states and waiting for the real retry acknowledgement.The recovery-card suite covers workspace divergence, reissue, reconcile, andconfirmation. [Workspace export](../../ui/src/components/WorkspaceExportRecovery.test.tsx)-covers its separate repair surface.+covers quiet restoration, legacy/operator assistance and resolved states.++For an abandoned native copyback owner, the harness verifies the exact stopped+deployment and the reconciler restores saved work automatically. Task users see+no process details or shutdown form. The documented operator API handles older+receipts after independent platform verification, through a separate administrator+gate. The [authorization regressions](../../server/src/__tests__/authorization-service.test.ts)+and [route regressions](../../server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts)+cover administrator admission, ordinary-member and agent denial, suspended and+foreign membership, missing role data, and stale cloud admin rows.+The [native workspace finalizer suite](../../server/src/services/native-runtime/native-workspace-finalizer.test.ts)+runs the real reconciler with a valid saved yield from `result_accepted`,+`workspace_finalizing`, and `retryable_failure`. It verifies physical fencing+before verified stop, automatic and operator recovery, successful export and commitment, hold resolution,+unchanged saved results, retained approval/monitor waits, and no new provider wake.+These local regressions do not prove a deployment platform stopped a foreign writer.Manual: reproduce an exhausted disposable action. Click once, verify pendingstate prevents repeated clicks, and inspect a successful continuation or an@@ -128,6 +143,17 @@ repair, record the compared branches/revisions and preserve uncommitted work;confirm the repair's actual result before resuming. Check the denied and staleaction paths. A changed card label is insufficient proof of workspace integrity.+For an abandoned native controller, follow the [stop-verification procedure](../../doc/native-workspace-finalization-recovery.md)+with a disposable deployment. A new platform-bound owner recovers automatically+only after the harness signs proof that the exact deployment fully stopped.+Legacy owners require independently verified stop evidence through the operator+API; the task has no shutdown form. Confirm+the original accepted result commits, the recovery hold resolves, saved files+remain intact, and existing approval waits or monitor checks remain. No new agent+turn should start. Repeat with a retry-phase owner and an ordinary runtime manager;+the first can resume through automatic proof or authorized operator confirmation,+while the second cannot use the operator API.+### `post-stop-message`Automated: [legacy continuation](../../server/src/services/recovery/legacy-continuation.test.ts),diff --git a/maps/storybook-map/INDEX.md b/maps/storybook-map/INDEX.mdindex ec3a4c08b5..cc36bca0cc 100644--- a/maps/storybook-map/INDEX.md+++ b/maps/storybook-map/INDEX.md@@ -4,7 +4,7 @@[Contract and maintenance](README.md) · [Review findings](REVIEW.md)-223 story files; 2215 indexed stories. Counts describe inventory, not passed tests or complete feature coverage.+225 story files; 2224 indexed stories. Counts describe inventory, not passed tests or complete feature coverage.Status describes product alignment; implementation describes the rendered subject; boundary describes simulated data. Sources and limitations are in catalog.json. Related capabilities cross-list one canonical story file.@@ -335,10 +335,12 @@ Stopped work, workspace repair, bounded continuation, and reconnect.| [Projects and execution/Recovery/Tasks/Model capacity retry](../../ui/storybook/stories/recovery/model-capacity-retry.stories.tsx) | `recovery/bounded-continuation` | unreviewed / production / fixtures | 1 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. || [Projects and execution/Recovery/Design previews/Recovery notice](../../ui/storybook/stories/recovery/recovery-notice-preview.stories.tsx) | `recovery/diagnosis`, `recovery/source-action` | current / production / fixtures | 9 / 3 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. || [Projects and execution/Recovery/Product/Issue Scheduled retry surfaces](../../ui/storybook/stories/recovery/scheduled-retry.stories.tsx) | `recovery/bounded-continuation` | unreviewed / production / fixtures | 1 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. |-| [Projects and execution/Recovery/Paperclip/Source Issue Recovery](../../ui/storybook/stories/recovery/source-issue-recovery.stories.tsx) | `recovery/diagnosis`, `recovery/source-action` | unreviewed / production / fixtures | 5 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. |+| [Projects and execution/Recovery/Paperclip/Source Issue Recovery](../../ui/storybook/stories/recovery/source-issue-recovery.stories.tsx) | `recovery/diagnosis`, `recovery/source-action` | unreviewed / production / fixtures | 5 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Source review confirms ordinary recovery fixtures remain applicable after the native-owner resolution gate; native-owner stop verification has its own suite. || [Projects and execution/Recovery/Paperclip/Source-resolved Fold](../../ui/storybook/stories/recovery/source-resolved-fold.stories.tsx) | `recovery/diagnosis`, `recovery/source-action` | unreviewed / production / fixtures | 5 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. || [Projects and execution/Recovery/Workspace branch recovery/01 Repair card](../../ui/storybook/stories/recovery/workspace-branch-recovery/01-component.stories.tsx) | `recovery/workspace` | unreviewed / production / fixtures | 8 / 4 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. || [Projects and execution/Recovery/Workspace branch recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-branch-recovery/02-task-journey.stories.tsx) | `recovery/workspace` | unreviewed / production / fixtures | 2 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Master reconciliation inspected assignment-history recipient suggestions and manual-selection retention; existing stories do not establish the new assignment-history behavior. Expanded upstream verification recipes were inspected; the existing fixture scenarios do not prove the added backend handoff, workspace, recovery or authorization contracts. |+| [Projects and execution/Recovery/Workspace finalization recovery/01 Notice](../../ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx) | `recovery/workspace`, `recovery/source-action` | current / production / fixtures | 6 / 2 | Production quiet status; automatic restoration, legacy operator fallback, queued and resolved states. No task repair form, process details or API simulation. Provider proof is verified by separate server and harness tests. |+| [Projects and execution/Recovery/Workspace finalization recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx) | `recovery/workspace`, `recovery/source-action` | unreviewed / mixed / fixtures | 3 / 0 | Production shell, status, bubbles and composer in a local conversation composition; shows restoration and saved work waiting for confirmation. Fixture messages; not the shipped task page or live provider acceptance. |## Apps, channels, and extensions@@ -667,9 +669,9 @@ References are navigation links, not proof of parity. An absent link means no ca- `execution-environments/images`: No cataloged story.- `recovery/diagnosis`: [Tasks and conversations/Task creation and lifecycle/Paperclip/Assigned Backlog Safeguards](../../ui/storybook/stories/tasks/assigned-backlog-safeguards.stories.tsx) (unreviewed), [Tasks and conversations/Inbox, decisions, and search/Product/Inbox/Blocked tab](../../ui/storybook/stories/inbox-search/blocked-inbox.stories.tsx) (unreviewed), [Apps, channels, and extensions/Secrets and proposals/Task credential access/01 Notice](../../ui/storybook/stories/secrets/credential-access/01-notice.stories.tsx) (unreviewed), [Apps, channels, and extensions/Secrets and proposals/Task credential access/02 Task page](../../ui/storybook/stories/secrets/credential-access/02-task-page.stories.tsx) (unreviewed), [Projects and execution/Recovery/Task chat/Recovery notice](../../ui/storybook/stories/recovery/execution-blocker-notice.stories.tsx) (unreviewed), [Projects and execution/Recovery/Tasks/Execution recovery](../../ui/storybook/stories/recovery/execution-recovery.stories.tsx) (unreviewed), [Projects and execution/Recovery/Product/Issue/Blocked notice](../../ui/storybook/stories/recovery/issue-blocked-notice.stories.tsx) (unreviewed), [Projects and execution/Recovery/Design previews/Recovery notice](../../ui/storybook/stories/recovery/recovery-notice-preview.stories.tsx) (current), [Projects and execution/Recovery/Paperclip/Source Issue Recovery](../../ui/storybook/stories/recovery/source-issue-recovery.stories.tsx) (unreviewed), [Projects and execution/Recovery/Paperclip/Source-resolved Fold](../../ui/storybook/stories/recovery/source-resolved-fold.stories.tsx) (unreviewed), [Tasks and conversations/Task creation and lifecycle/Paperclip/Successful Run Handoff](../../ui/storybook/stories/tasks/successful-run-handoff.stories.tsx) (unreviewed)- `recovery/bounded-continuation`: [Projects and execution/Recovery/Tasks/Model capacity retry](../../ui/storybook/stories/recovery/model-capacity-retry.stories.tsx) (unreviewed), [Projects and execution/Recovery/Product/Issue Scheduled retry surfaces](../../ui/storybook/stories/recovery/scheduled-retry.stories.tsx) (unreviewed)-- `recovery/source-action`: [Projects and execution/Recovery/Design previews/Recovery notice](../../ui/storybook/stories/recovery/recovery-notice-preview.stories.tsx) (current), [Projects and execution/Recovery/Paperclip/Source Issue Recovery](../../ui/storybook/stories/recovery/source-issue-recovery.stories.tsx) (unreviewed), [Projects and execution/Recovery/Paperclip/Source-resolved Fold](../../ui/storybook/stories/recovery/source-resolved-fold.stories.tsx) (unreviewed)+- `recovery/source-action`: [Projects and execution/Recovery/Design previews/Recovery notice](../../ui/storybook/stories/recovery/recovery-notice-preview.stories.tsx) (current), [Projects and execution/Recovery/Paperclip/Source Issue Recovery](../../ui/storybook/stories/recovery/source-issue-recovery.stories.tsx) (unreviewed), [Projects and execution/Recovery/Paperclip/Source-resolved Fold](../../ui/storybook/stories/recovery/source-resolved-fold.stories.tsx) (unreviewed), [Projects and execution/Recovery/Workspace finalization recovery/01 Notice](../../ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx) (current), [Projects and execution/Recovery/Workspace finalization recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx) (unreviewed)- `recovery/exhaustion`: No cataloged story.-- `recovery/workspace`: [Projects and execution/Recovery/Workspace branch recovery/01 Repair card](../../ui/storybook/stories/recovery/workspace-branch-recovery/01-component.stories.tsx) (unreviewed), [Projects and execution/Recovery/Workspace branch recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-branch-recovery/02-task-journey.stories.tsx) (unreviewed)+- `recovery/workspace`: [Projects and execution/Recovery/Workspace branch recovery/01 Repair card](../../ui/storybook/stories/recovery/workspace-branch-recovery/01-component.stories.tsx) (unreviewed), [Projects and execution/Recovery/Workspace branch recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-branch-recovery/02-task-journey.stories.tsx) (unreviewed), [Projects and execution/Recovery/Workspace finalization recovery/01 Notice](../../ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx) (current), [Projects and execution/Recovery/Workspace finalization recovery/02 Task journey](../../ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx) (unreviewed)- `recovery/unknown-outcome`: No cataloged story.- `recovery/user-continuation`: No cataloged story.- `recovery/reconnect`: [Projects and execution/Recovery/App/Connection recovery](../../ui/storybook/stories/recovery/cloud-access-recovery.stories.tsx) (unreviewed)diff --git a/maps/storybook-map/REVIEW.md b/maps/storybook-map/REVIEW.mdindex b1ab397fee..e3575ea0ee 100644--- a/maps/storybook-map/REVIEW.md+++ b/maps/storybook-map/REVIEW.md@@ -274,6 +274,35 @@ Reviewed the same twelve workspace-recipe-dependent entries after extending theReviewed the twelve unchanged workspace recipe subjects and fixtures after correcting implicit default selection. Task creation and controller classification now share configured-policy, primary, creation-time and ID precedence. Explicit choices and retained denial markers still suppress controller hooks. Project strategy fixtures show configuration, while runtime status and file/browser/artifact/service fixtures do not execute source scripts or prove this backend ordering. Refreshed only their reviewed recipe fingerprints and retained all classifications and stable IDs. No browser render, play, screenshot, Storybook build or live runtime acceptance was performed.+## Workspace finalization UI refinement — 2026-10-13++Source base: `f07efa2802`, followed by scoped edits to the production recovery notice and its two story suites. Reviewed `recovery/workspace` and `recovery/source-action`; stable story IDs and administrator admission remain unchanged. The notice leads with saved work, discloses controller metadata, explains evidence/confirmation requirements, and distinguishes queue acknowledgement from completion. A stale controller error preserves the note and clears confirmation so the operator must verify again. The task composition uses viewer identity for human bubbles and omits a routine status badge.++MacOS in-app browser inspection passed for the task journey at desktop and 390px mobile widths: controller disclosure with Enter, evidence without confirmation remaining disabled, explicit confirmation enabling submission, and a simulated queued result in place. The `owner-changed` story's play assertions passed; the friendly error was visible, the note remained, confirmation cleared and the action was disabled. The `long-controller-name` story rendered at 390px without horizontal overflow. The mobile task shell showed its bottom navigation; evidence text computed to 16px and Tab reached the enabled action. No physical mobile keyboard, Linux screenshot baseline, live deployment stop, real API admission or provider copyback acceptance is claimed. Other story states received source review, not a full browser execution pass.++Refreshed the two reviewed catalog entries only. Global catalog validation already reported unrelated source-fingerprint drift before this work; those entries require their own review.++## Automatic workspace recovery revision — 2026-10-13++Supersedes the task shutdown form reviewed above. The production task now renders+only a quiet restoration status or an honest operator-assistance fallback. It+contains no process metadata, evidence field, confirmation checkbox or resume+button. The task thread omits the duplicate technical recovery card. Obsolete+form stories and their simulated API fixture were removed; suite IDs remain+stable. The composed task journey includes a saved state without the notice.++Observed in the macOS in-app browser: automatic status in the production shell,+completed example without the status, retained approval-wait message, and mobile+bottom navigation with equal content and viewport widths (no horizontal overflow).+These are fixture UI observations, not live provider shutdown or task commitment.+The production component test verifies removal on a resolved action update. The+real reconciler's database tests separately preserve accepted-result identity,+monitor and approval waits, and create no new run or wake request.++Only the two reviewed suites have refreshed fingerprints. Global validation has+71 other source-fingerprint errors, including recovery-recipe dependents after+the operator instructions changed. It is not globally green. No unrelated review+fingerprints or visual baselines were accepted.## Inherited activity, inbox and recovery recipes — 2026-10-12@@ -435,3 +464,52 @@ and three also reference the Slack-only settings copy. Other source bytes andall classifications, fixture boundaries, story IDs and sidebar titles remainunchanged from that baseline. Catalog checks validate these references andfingerprints, not live Slack behavior.++### Current-master reconciliation++Merged master `5e8e992df2`. The newer catalog supplies the baseline associations+and reviews. Compared every declared input against that master: only seven+inputs changed for this feature (recovery recipe, recovery card, task page, quiet+status, two stories and their fixture). Reviewed those changes and the unchanged+subjects/fixtures they affect; refreshed the 26 dependent fingerprints while+retaining their classifications and simulation boundaries. The task-page change+adds the quiet status and suppresses only the owner-unverified duplicate card;+other task variants retain their behavior. The recovery-card change blocks+unsafe generic dismissal. Existing card regressions cover that denial.++Five remaining errors have unchanged source bytes and metadata inherited from+master: shared setup, decision-card, decisions-desk, search and what-needs-me.+These require a separate review; no hashes or visual baselines were accepted for+them. The earlier 71-error count above describes the pre-merge baseline.++### Inherited index metadata follow-up++The current master's personal-toast recipe also left shared setup and four inbox+suites with stale metadata. Reviewed the recipe's added toast capability and+verification limits: decision-card, decisions-desk and search still demonstrate+their existing isolated subjects; what-needs-me's toast fixture does not prove+viewer eligibility. Their subject and fixture bytes are unchanged from master;+retained all unreviewed classifications and fixture boundaries. The generator+adds the missing toast capability and description to the existing inbox feature,+without changing feature order or IDs. Reviewed that generated diff and refreshed+these four entries, the shared setup and Overview fingerprints. No toast behavior,+new browser acceptance or visual baseline is claimed. Global catalog validation+now passes. This supersedes the five inherited-error count above.++### Confirmed recovery status++Reviewed the shipped notice, server evidence contract and both recovery fixtures.+Platform identity alone no longer means work is restoring: the server marks the+exact recovery receipt resuming only after verified stop proof and a successful+owner claim. Missing proof remains operator-required. The new unavailable-proof+story covers a platform-bound receipt without progress; confirmed and queued+stories retain the quiet status. Updated only the two affected fingerprints.+This is component/fixture evidence, not deployed harness acceptance.++### Latest master reconciliation++Retained master’s task-creation visibility scenarios, source-review notes and+unreviewed classifications. Seven overlapping entries differ from master only+in the already reviewed recovery recipe or task-detail recovery notice wiring.+Refreshed those combined-source fingerprints and regenerated the index. No new+creation-visibility acceptance or visual baseline is claimed.diff --git a/maps/storybook-map/catalog.json b/maps/storybook-map/catalog.jsonindex 1c81c9a3ad..005c7a6f2b 100644--- a/maps/storybook-map/catalog.json+++ b/maps/storybook-map/catalog.json@@ -304,7 +304,7 @@"ui/storybook/prototypes/chat-entry/ChatEntryReview.tsx","ui/storybook/stories/agent-chat/agent-chat-sidebar.stories.tsx"],- "fingerprint": "6994574cfda7901b4cec355add8dadb58146bd4cf0d3488796fe27e19e34cdc3",+ "fingerprint": "0fc8fe3663215c79493116ec27e13daf92b8893806ee4e19e7e4143e97e4fe5f","stableId": "design-explorations-agent-chat-sidebar-pages","sidebarTitle": "Tasks and conversations/Agent conversations and project handoff/Design explorations/Agent chat sidebar/Pages"},@@ -364,7 +364,7 @@"ui/storybook/prototypes/chat-entry/ChatEntryReview.tsx","ui/storybook/stories/agent-chat/agent-chat.stories.tsx"],- "fingerprint": "c3faad0b551a84f6c7ddf5cf9566b96e0d435d28a4ac3035d147b069723b7152",+ "fingerprint": "96b384fffda6225b3f00714d1fa33dc208806d9a502e2dcc46fba3c667010d18","stableId": "design-explorations-agent-chat","sidebarTitle": "Tasks and conversations/Agent conversations and project handoff/Design explorations/Agent chat"},@@ -513,7 +513,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/agents/agent-persona-pages.stories.tsx"],- "fingerprint": "35454c02bbae403677d0d79644dc92da6c96172265d0f7581508e7d111463e0f",+ "fingerprint": "672f406658a3ade96fed4f5a18d2a413ec9888e348b74244b5d37cb3db760dde","stableId": "agents-personas-full-pages","sidebarTitle": "Agents and reusable capabilities/Hiring, configuration, and organization/Agents/Personas/Full pages"},@@ -1196,7 +1196,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/tasks/assigned-backlog-safeguards.stories.tsx"],- "fingerprint": "cdb63642517b31c68df9ca8d62e60039f9f4d69d95d1a96dc96235fb7db1487d",+ "fingerprint": "d6e735a2c56092274f139709aba0c1c555f3f3cb0fbf9820d50ee02874907b79","stableId": "paperclip-assigned-backlog-safeguards","sidebarTitle": "Tasks and conversations/Task creation and lifecycle/Paperclip/Assigned Backlog Safeguards"},@@ -1220,7 +1220,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/inbox-search/blocked-inbox.stories.tsx"],- "fingerprint": "bcb6c0a3cb34fcd06e7b056521ad84fb968f73ba46a7ba00eda2a86bb060b11b",+ "fingerprint": "70b54f5ce7c083b8e64c28501d66247d4753adabd732b377f7ad18309dc90ffe","stableId": "product-inbox-blocked-tab","sidebarTitle": "Tasks and conversations/Inbox, decisions, and search/Product/Inbox/Blocked tab"},@@ -1456,7 +1456,7 @@"ui/storybook/prototypes/chat-entry/ChatEntryReview.tsx","ui/storybook/stories/agent-chat/chat-entry.stories.tsx"],- "fingerprint": "444465b306686cc36d85d4d2af43f9cd31f3c7c34a5fd05f8413aa639becf535",+ "fingerprint": "3e85e934a82dae8089b73a1bddb0538640259ce1c70e7d16da38ee1e91ab0efc","replacement": "ui/storybook/stories/agents/primary-agent/05-chat.stories.tsx","stableId": "design-explorations-chat-entry","sidebarTitle": "Tasks and conversations/Agent conversations and project handoff/Design explorations/Chat entry"@@ -1499,13 +1499,13 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/CloudAccessGate.tsx","ui/storybook/stories/recovery/cloud-access-recovery.stories.tsx"],- "fingerprint": "bbec224b127ec6898cf95dfae922e6747483c0d0d4ec55ff7d1560c0c4a5c041",+ "fingerprint": "64b2422bdabe2b07b563354481000171455841e49ccbf6a72270066d4e27e9e0","stableId": "app-connection-recovery","sidebarTitle": "Projects and execution/Recovery/App/Connection recovery"},@@ -1829,14 +1829,14 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","maps/feature-map/secrets.md","ui/src/components/task-chat/TaskChatMarker.tsx","ui/storybook/stories/secrets/credential-access/01-notice.stories.tsx"],- "fingerprint": "2eba045db9127e5cd8d8bf7c9664e4598a5c84be117ae6159ce7171b24f2005d",+ "fingerprint": "14757d026311d143e9301d6294d2f46f40dad25af8ea3f8d5cf1893b260f82db","stableId": "task-credential-access-01-notice","sidebarTitle": "Apps, channels, and extensions/Secrets and proposals/Task credential access/01 Notice"},@@ -1862,7 +1862,7 @@"ui/storybook/stories/secrets/credential-access/02-task-page.stories.tsx","ui/storybook/support/credential-access/CredentialTaskPage.tsx"],- "fingerprint": "70a07a45f3633984db4bbadd599c3745e098f17100f8421b0c3cd1fec46e2186",+ "fingerprint": "1289c5e6659cd318f00b63d4f16790bbac988d8d18f6726e10262223c11e66fe","stableId": "task-credential-access-02-task-page","sidebarTitle": "Apps, channels, and extensions/Secrets and proposals/Task credential access/02 Task page"},@@ -2210,13 +2210,13 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/ExecutionBlockerNotice.tsx","ui/storybook/stories/recovery/execution-blocker-notice.stories.tsx"],- "fingerprint": "bee366ba1266e6692c8f0ee027c449fdda6b51762369ba431871a5af5ee018c6",+ "fingerprint": "018d8d687921ea109cb98f587abd59b94722f1f148940852326417ce8a457b04","stableId": "task-chat-recovery-notice","sidebarTitle": "Projects and execution/Recovery/Task chat/Recovery notice"},@@ -2230,7 +2230,7 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/ActiveAgentsPanel.tsx",@@ -2240,7 +2240,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/recovery/execution-recovery.stories.tsx"],- "fingerprint": "6f8b245b169d95046bdb52513dfaf4937a96f807b72470c198863471f59c8149",+ "fingerprint": "82ddff0cf6d98e558f9038d59960758347ad39f3c7a9aaf18f29214a8ad81756","stableId": "tasks-execution-recovery","sidebarTitle": "Projects and execution/Recovery/Tasks/Execution recovery"},@@ -2861,7 +2861,7 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","maps/feature-map/task-coordination.md",@@ -2869,7 +2869,7 @@"ui/src/components/task-chat/TaskChatBlockerLinks.tsx","ui/storybook/stories/recovery/issue-blocked-notice.stories.tsx"],- "fingerprint": "6a84d7f60318d33a1835a668873095aaad1c215a83bbbb4735823fdcb3291140",+ "fingerprint": "52f861389de245bda4d9c582581b49ce0b9442d60ee7eff950adc6801aaca8a4","stableId": "product-issue-blocked-notice","sidebarTitle": "Projects and execution/Recovery/Product/Issue/Blocked notice"},@@ -3093,13 +3093,13 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/IssueScheduledRetryCard.tsx","ui/storybook/stories/recovery/model-capacity-retry.stories.tsx"],- "fingerprint": "5e08f681f3f96cb7be32e316d3f4066e5239dcf0f4393cb9ec09d599734f201e",+ "fingerprint": "1db367a2b77b8c747ab5c81c1fe3317da02bf294308e9e09af827bcd12ee0b9f","stableId": "tasks-model-capacity-retry","sidebarTitle": "Projects and execution/Recovery/Tasks/Model capacity retry"},@@ -3358,7 +3358,7 @@"ui/storybook/prototypes/originating-tasks/OriginatingTasks.tsx","ui/storybook/stories/task-coordination/originating-tasks.stories.tsx"],- "fingerprint": "59123e73a0286fd11a48376ee152a974d730c59bd5421fd273cfca3a185f6e08",+ "fingerprint": "32c9981ce024550b26636b97d7337f7b34a1207e8bcdd5759a11d9a7ddb204fe","stableId": "ux-labs-tasks-created-from-a-task","sidebarTitle": "Tasks and conversations/Delegation, dependencies, and signoff/UX Labs/Tasks Created From a Task"},@@ -3370,7 +3370,7 @@"implementation": "guide","boundary": "static","notes": "Matches the upstream feature index and order, including features with no cataloged examples. Related suite links and classifications do not imply execution coverage.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["ui/src/components/ui/badge.tsx","ui/src/components/ui/input.tsx",@@ -3793,7 +3793,7 @@"ui/storybook/stories/access/private-tasks/06-full-product-pages.stories.tsx","ui/storybook/support/private-tasks/PrivacyStory.tsx"],- "fingerprint": "d4b5ba6bce33a9657d859d3278b1f84bbc9b77cffd24e370214a0fce24c0a02b",+ "fingerprint": "ebd9132a1f0228f775b1f382fc30dcae68e65284c8118c7b8a2eee3219c741ed","stableId": "private-tasks-01-creation","sidebarTitle": "Getting started and access/Login, invitations, and access/Private tasks/01 Creation"},@@ -3939,7 +3939,7 @@"ui/storybook/stories/access/private-tasks/06-full-product-pages.stories.tsx","ui/storybook/support/private-tasks/PrivacyStory.tsx"],- "fingerprint": "5fb218d88c3b6815839732e4226a27f4b40a2df46c8cbad6709cb665546b4248",+ "fingerprint": "6069116fffc3a661df1fb10245f3f6d8045086efd71c886a8064a27eed09ac42","stableId": "private-tasks-06-full-product-pages","sidebarTitle": "Getting started and access/Login, invitations, and access/Private tasks/06 Full product pages"},@@ -3986,7 +3986,7 @@"ui/storybook/stories/access/private-tasks/07-user-stories.stories.tsx","ui/storybook/support/private-tasks/PrivacyStory.tsx"],- "fingerprint": "5fbe5c61f8754bc0c1db1c6a990b9c44bc5d941b7b61de77fac76300e6c42a22",+ "fingerprint": "483a8bfae265844bfa10ebd8529572f43c23c8474d414b45ba0b663efc280dbd","stableId": "private-tasks-07-user-stories","sidebarTitle": "Getting started and access/Login, invitations, and access/Private tasks/07 User stories"},@@ -4437,7 +4437,7 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/DispositionRecoveryNotice.tsx",@@ -4446,7 +4446,7 @@"ui/storybook/prototypes/recovery-notice/RecoveryNoticePreview.tsx","ui/storybook/stories/recovery/recovery-notice-preview.stories.tsx"],- "fingerprint": "cfd7345545112ea9ecc13035ebd187451117de5ed010f439cfb8c6962461fb3f",+ "fingerprint": "e83bcaf099227bfcd17d571f680121b5b51d5f605d76479165000a575229a884","stableId": "design-previews-recovery-notice","sidebarTitle": "Projects and execution/Recovery/Design previews/Recovery notice"},@@ -4697,7 +4697,7 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/IssueProperties.tsx",@@ -4705,7 +4705,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/recovery/scheduled-retry.stories.tsx"],- "fingerprint": "43ef1917079251d558b3f52e5af750dfb69a54d3326d31494dbf037baeefe193",+ "fingerprint": "f360f4207ef2b1cef18eee2307c50a7bce3a5f5bbf8966b24cd86b3ec50d21b4","stableId": "product-issue-scheduled-retry-surfaces","sidebarTitle": "Projects and execution/Recovery/Product/Issue Scheduled retry surfaces"},@@ -5009,8 +5009,8 @@"status": "unreviewed","implementation": "production","boundary": "fixtures",- "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Source review confirms ordinary recovery fixtures remain applicable after the native-owner resolution gate; native-owner stop verification has its own suite.",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/IssueBlockedNotice.tsx",@@ -5019,7 +5019,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/recovery/source-issue-recovery.stories.tsx"],- "fingerprint": "1b6e44d4fb4ab7e2e63fb8816aa31fbe3267e10647f238ed82e6591a82d2e918",+ "fingerprint": "949814263788ae0e3501b67bbd26118f150db219fc04e5cf653f18dd345ec8d5","stableId": "paperclip-source-issue-recovery","sidebarTitle": "Projects and execution/Recovery/Paperclip/Source Issue Recovery"},@@ -5034,14 +5034,14 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/SourceResolvedFoldBadge.tsx","ui/src/components/SourceResolvedFoldCallout.tsx","ui/storybook/stories/recovery/source-resolved-fold.stories.tsx"],- "fingerprint": "3af3daff618b8c278bcbf10ba3e5d081d87240c680758ccb3078eccdf9283122",+ "fingerprint": "70cab38574df19267e35ab13c10c4a3eea444bb3b6e72c81f739f21cfc519df0","stableId": "paperclip-source-resolved-fold","sidebarTitle": "Projects and execution/Recovery/Paperclip/Source-resolved Fold"},@@ -5122,7 +5122,7 @@"ui/storybook/fixtures/paperclipData.ts","ui/storybook/stories/tasks/successful-run-handoff.stories.tsx"],- "fingerprint": "2a7db108d46f35f37dd9346d2fc039e61dff8c688e09d9ed7b0e777e7f79af0d",+ "fingerprint": "27ad7c3c2b09acf5ac07163d44026511bd0c2b806387b591fef5fca343b0c63b","stableId": "paperclip-successful-run-handoff","sidebarTitle": "Tasks and conversations/Task creation and lifecycle/Paperclip/Successful Run Handoff"},@@ -5575,14 +5575,14 @@"implementation": "production","boundary": "fixtures","notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed.",- "reviewedAt": "2026-10-12",+ "reviewedAt": "2026-10-13","sources": ["maps/feature-map/recovery.md","ui/src/components/WorkspaceBaseRefRecoveryNotice.tsx","ui/storybook/stories/recovery/workspace-branch-recovery/01-component.stories.tsx","ui/storybook/support/workspace-branch-recovery/fixtures.ts"],- "fingerprint": "77f708cc660f6829cadaffa6139930a102109976c41f7e24e2f662ed687d2412",+ "fingerprint": "eeaa7044a1d5a2b5355a8ac2375820e3611c71920e88c04d87e5d7ce64ed929a","stableId": "workspace-branch-recovery-01-repair-card","sidebarTitle": "Projects and execution/Recovery/Workspace branch recovery/01 Repair card"},@@ -5607,7 +5607,7 @@"ui/storybook/stories/recovery/workspace-branch-recovery/02-task-journey.stories.tsx","ui/storybook/support/workspace-branch-recovery/fixtures.ts"],- "fingerprint": "5eedfcb5738c125d3fa8ed7dc4b3d20a300b216de51f1ae5e58ede98d38fd75c",+ "fingerprint": "f8a090cfdbadfe77d8ee064b99fc11b8f5e097d4e72173df2496280ef52152d1","stableId": "workspace-branch-recovery-02-task-journey","sidebarTitle": "Projects and execution/Recovery/Workspace branch recovery/02 Task journey"},@@ -5671,6 +5671,53 @@"fingerprint": "b9cd51c58b31ab445e2997b9f2a23c7a7b3f4ecd7b6dd39fc0f35ae0779311df","stableId": "workspaces-service-control-bar","sidebarTitle": "Projects and execution/Workspaces, services, and files/Workspaces/Service control bar"+ },+ {+ "file": "ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx",+ "capabilities": [+ "recovery/workspace",+ "recovery/source-action"+ ],+ "purpose": "Workspace finalization recovery/01 Notice",+ "status": "current",+ "implementation": "production",+ "boundary": "fixtures",+ "notes": "Production quiet status; automatic restoration, legacy operator fallback, queued and resolved states. No task repair form, process details or API simulation. Provider proof is verified by separate server and harness tests.",+ "reviewedAt": "2026-10-13",+ "sources": [+ "maps/feature-map/recovery.md",+ "ui/src/components/WorkspaceExportRecovery.tsx",+ "ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx",+ "ui/storybook/support/workspace-finalization-recovery/fixtures.tsx"+ ],+ "fingerprint": "45bf8c793c91362634b7ce32ed0ec2007861b7fefbfd0e2da692c3dc804a53c9",+ "stableId": "workspace-finalization-recovery-01-notice",+ "sidebarTitle": "Projects and execution/Recovery/Workspace finalization recovery/01 Notice"+ },+ {+ "file": "ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx",+ "capabilities": [+ "recovery/workspace",+ "recovery/source-action"+ ],+ "purpose": "Workspace finalization recovery/02 Task journey",+ "status": "unreviewed",+ "implementation": "mixed",+ "boundary": "fixtures",+ "notes": "Production shell, status, bubbles and composer in a local conversation composition; shows restoration and saved work waiting for confirmation. Fixture messages; not the shipped task page or live provider acceptance.",+ "reviewedAt": "2026-10-13",+ "sources": [+ "maps/feature-map/recovery.md",+ "ui/src/components/Layout.tsx",+ "ui/src/components/WorkspaceExportRecovery.tsx",+ "ui/src/components/task-chat/TaskChatBubble.tsx",+ "ui/src/components/task-chat/TaskChatComposer.tsx",+ "ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx",+ "ui/storybook/support/workspace-finalization-recovery/fixtures.tsx"+ ],+ "fingerprint": "aa76cd28dde8d50abd208b34283249170f3a85435ba82fff169a1c33433dfca1",+ "stableId": "workspace-finalization-recovery-02-task-journey",+ "sidebarTitle": "Projects and execution/Recovery/Workspace finalization recovery/02 Task journey"}]}diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.tsindex b1655cf316..48c3110dac 100644--- a/packages/shared/src/index.ts+++ b/packages/shared/src/index.ts@@ -1999,6 +1999,7 @@ export {resolveIssueRecoveryActionSchema,isValidExistingBranchName,retryWorkspaceExportSchema,+ resumeWorkspaceFinalizationSchema,issueReviewRequestSchema,issueExecutionWorkspaceSettingsSchema,checkoutIssueSchema,diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.tsindex e7b3197a55..8e224de092 100644--- a/packages/shared/src/validators/index.ts+++ b/packages/shared/src/validators/index.ts@@ -441,6 +441,7 @@ export {resolveIssueRecoveryActionSchema,isValidExistingBranchName,retryWorkspaceExportSchema,+ resumeWorkspaceFinalizationSchema,issueReviewRequestSchema,issueExecutionWorkspaceSettingsSchema,checkoutIssueSchema,diff --git a/packages/shared/src/validators/issue.ts b/packages/shared/src/validators/issue.tsindex 8fddad793b..0d9ea3bdce 100644--- a/packages/shared/src/validators/issue.ts+++ b/packages/shared/src/validators/issue.ts@@ -557,6 +557,14 @@ const RESOLVE_ISSUE_RECOVERY_ACTION_OUTCOMES = ["cancelled",] as const;+export const resumeWorkspaceFinalizationSchema = z.object({+ actionId: z.string().guid(),+ runId: z.string().guid(),+ ownerToken: z.string().guid(),+ controllerAndCopybackStopped: z.literal(true),+ stopEvidence: z.string().trim().min(20).max(12000),+}).strict();+export const retryWorkspaceExportSchema = z.object({actionId: z.string().guid(),runId: z.string().guid(),diff --git a/server/src/__tests__/authorization-service.test.ts b/server/src/__tests__/authorization-service.test.tsindex 4e1f3f63c2..a2b28921c5 100644--- a/server/src/__tests__/authorization-service.test.ts+++ b/server/src/__tests__/authorization-service.test.ts@@ -1810,6 +1810,41 @@ describeEmbeddedPostgres("authorization service", () => {expect(await auth.decide({ actor, action: "agent:wake", resource: { ...resource, companyId: otherCompany.id } })).toMatchObject({ allowed: false });});+ it.each(["session", "cloud_tenant"] as const)("restricts %s workspace-owner stop confirmation to administrators", async source => {+ const company = await createCompany(db, "OwnerStop");+ const otherCompany = await createCompany(db, "OtherOwnerStop");+ const userId = await createUser(db);+ await db.insert(companyMemberships).values({ companyId: company.id,+ principalType: "user", principalId: userId, status: "active", membershipRole: "owner" });+ const actor = { type: "board" as const, source, userId, companyIds: [company.id] };+ const auth = authorizationService(db);+ const request = { actor, action: "runtime:confirm_owner_stop" as const,+ resource: { type: "company" as const, companyId: company.id } };+ for (const membershipRole of ["owner", "admin", "operator", "member", "viewer", null]) {+ await db.update(companyMemberships).set({ membershipRole }).where(eq(companyMemberships.principalId, userId));+ expect((await auth.decide(request)).allowed, `role ${membershipRole}`).toBe(membershipRole === "owner" || membershipRole === "admin");+ }+ await db.update(companyMemberships).set({ membershipRole: "admin", status: "suspended" }).where(eq(companyMemberships.principalId, userId));+ expect((await auth.decide(request)).allowed).toBe(false);+ await db.update(companyMemberships).set({ status: "active" }).where(eq(companyMemberships.principalId, userId));+ expect((await auth.decide({ ...request, resource: { type: "company", companyId: otherCompany.id } })).allowed).toBe(false);+ });++ it("does not allow agent delegation or stale cloud admin rows to confirm an owner stop", async () => {+ const company = await createCompany(db, "OwnerStopDelegation");+ const userId = await createUser(db);+ const agent = await createAgent(db, company.id, { role: "ceo" });+ await db.insert(companyMemberships).values({ companyId: company.id,+ principalType: "user", principalId: userId, status: "active", membershipRole: "operator" });+ await db.insert(instanceUserRoles).values({ userId, role: "instance_admin" });+ const auth = authorizationService(db);+ const request = { action: "runtime:confirm_owner_stop" as const, resource: { type: "company" as const, companyId: company.id } };+ expect((await auth.decide({ ...request, actor: { type: "board", userId, source: "cloud_tenant", isInstanceAdmin: false } })).allowed).toBe(false);+ expect((await auth.decide({ ...request, actor: { type: "agent", agentId: agent.id, companyId: company.id, onBehalfOfUserId: userId } })).allowed).toBe(false);+ expect((await auth.decide({ ...request, actor: { type: "board", userId, source: "cloud_tenant", isInstanceAdmin: true } })).allowed).toBe(true);+ expect((await auth.decide({ ...request, actor: { type: "board", source: "local_implicit" } })).allowed).toBe(true);+ });+it("limits viewer members to read-only visibility actions", async () => {const company = await createCompany(db, "BoardViewerVisibility");const userId = `user-${randomUUID()}`;diff --git a/server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts b/server/src/__tests__/issue-agent-mutation-ownership-routes.test.tsindex 6aa18723cb..4afb8bad37 100644--- a/server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts+++ b/server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts@@ -162,8 +162,10 @@ const mockLogActivity = vi.hoisted(() => vi.fn(async () => undefined));const mockObserveCrossIssueInfluence = vi.hoisted(() => vi.fn(async () => null));const mockRetryWorkspaceExport = vi.hoisted(() => vi.fn());+const mockResumeWorkspaceFinalization = vi.hoisted(() => vi.fn());function registerRouteMocks() {+ vi.doMock("../services/native-runtime/native-workspace-owner-recovery.js", () => ({ resumeNativeWorkspaceAfterOwnerStop: mockResumeWorkspaceFinalization }));vi.doMock("../services/native-runtime/native-workspace-export-retry.js", () => ({ retryNativeWorkspaceExport: mockRetryWorkspaceExport }));vi.doMock("@paperclipai/shared/telemetry", () => ({trackAgentTaskCompleted: vi.fn(),@@ -2009,6 +2011,52 @@ describe("agent issue mutation checkout ownership", () => {);});+ it("resumes a verified stopped workspace owner without waking the provider", async () => {+ mockAccessService.decide.mockResolvedValue({ allowed: true });+ mockResumeWorkspaceFinalization.mockResolvedValue({ runId: ownerRunId, status: "queued" });+ const res = await request(await createApp(boardActor()))+ .post(`/api/issues/${issueId}/recovery-actions/resume-workspace-finalization`)+ .send({ actionId: recoveryActionId, runId: ownerRunId, ownerToken: ownerRunId,+ controllerAndCopybackStopped: true, stopEvidence: "Deployment platform confirms the exact old container and all its processes exited." });+ expect(res.status).toBe(202);+ expect(mockAccessService.decide).toHaveBeenCalledWith(expect.objectContaining({ action: "runtime:confirm_owner_stop" }));+ expect(mockResumeWorkspaceFinalization).toHaveBeenCalledWith(expect.objectContaining({ companyId, issueId, runId: ownerRunId, ownerToken: ownerRunId, actorId: "board-user" }));+ expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled();+ });++ it.each([{ controllerAndCopybackStopped: false }, { stopEvidence: "" }])("requires explicit stop evidence: %j", async invalid => {+ const res = await request(await createApp(boardActor()))+ .post(`/api/issues/${issueId}/recovery-actions/resume-workspace-finalization`)+ .send({ actionId: recoveryActionId, runId: ownerRunId, ownerToken: ownerRunId,+ controllerAndCopybackStopped: true, stopEvidence: "Deployment platform confirms the exact old container and all its processes exited.", ...invalid });+ expect(res.status).toBe(400);+ expect(mockResumeWorkspaceFinalization).not.toHaveBeenCalled();+ });++ it.each(["agent", "viewer"])("rejects %s workspace-owner release before admission", async kind => {+ mockAccessService.decide.mockImplementation(async (input: { action: string }) => ({ allowed: input.action === "issue:read", explanation: "No runtime access" }));+ const res = await request(await createApp(kind === "agent" ? ownerActor() : boardActor()))+ .post(`/api/issues/${issueId}/recovery-actions/resume-workspace-finalization`)+ .send({ actionId: recoveryActionId, runId: ownerRunId, ownerToken: ownerRunId,+ controllerAndCopybackStopped: true, stopEvidence: "Deployment platform confirms the exact old container and all its processes exited." });+ expect(res.status).toBe(403);+ expect(mockResumeWorkspaceFinalization).not.toHaveBeenCalled();+ });++ it("rejects an ordinary runtime manager's claimed foreign-owner stop", async () => {+ mockAccessService.decide.mockImplementation(async ({ action }: { action: string }) => ({+ allowed: action === "issue:read" || action === "runtime:manage",+ explanation: "A deployment administrator must verify the controller stopped",+ }));+ const res = await request(await createApp(boardActor()))+ .post(`/api/issues/${issueId}/recovery-actions/resume-workspace-finalization`)+ .send({ actionId: recoveryActionId, runId: ownerRunId, ownerToken: ownerRunId,+ controllerAndCopybackStopped: true, stopEvidence: "I claim that the foreign controller and all its processes have stopped." });+ expect(res.status).toBe(403);+ expect(mockResumeWorkspaceFinalization).not.toHaveBeenCalled();+ expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled();+ });+it("queues board export-only recovery without calling provider wake", async () => {mockAccessService.decide.mockResolvedValue({ allowed: true });mockRetryWorkspaceExport.mockResolvedValue({ runId: ownerRunId, status: "queued" });@@ -2030,7 +2078,25 @@ describe("agent issue mutation checkout ownership", () => {expect(res.status).toBe(403); expect(mockRetryWorkspaceExport).not.toHaveBeenCalled();});- it.each(["todo", "done", "in_review"].flatMap(sourceIssueStatus => ["native_workspace_sync_out_unsafe_archive", "native_workspace_sync_out_retry_exhausted"].map(cause => ({ sourceIssueStatus, cause }))))("does not resolve accepted export recovery through an ordinary $sourceIssueStatus transition: $cause", async ({ sourceIssueStatus, cause }) => {+ it.each(["done", "in_review"])("does not bypass unverified copyback via false_positive to %s", async sourceIssueStatus => {+ const sourceIssue = makeIssue({ status: "blocked", assigneeAgentId: ownerAgentId });+ mockIssueService.getById.mockResolvedValue(sourceIssue);+ mockIssueService.update.mockImplementation(async (_id: string, patch: Record<string, unknown>) => ({ ...sourceIssue, ...patch }));+ mockIssueRecoveryActionService.getActiveForIssue.mockResolvedValue({+ id: recoveryActionId, status: "active", kind: "active_run_watchdog", ownerType: "board",+ ownerAgentId: null, returnOwnerAgentId: ownerAgentId,+ cause: "native_workspace_finalization_owner_unverified", evidence: { runId: ownerRunId },+ });+ const res = await request(await createApp(boardActor()))+ .post(`/api/issues/${issueId}/recovery-actions/resolve`)+ .send({ actionId: recoveryActionId, outcome: "false_positive", sourceIssueStatus });+ expect(res.status).toBe(409);+ expect(res.body.details?.code).toBe("workspace_owner_stop_required");+ expect(mockIssueService.update).not.toHaveBeenCalled();+ expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled();+ });++ it.each(["todo", "done", "in_review"].flatMap(sourceIssueStatus => ["native_workspace_sync_out_unsafe_archive", "native_workspace_sync_out_retry_exhausted", "native_workspace_finalization_owner_unverified"].map(cause => ({ sourceIssueStatus, cause }))))("does not resolve accepted export recovery through an ordinary $sourceIssueStatus transition: $cause", async ({ sourceIssueStatus, cause }) => {const sourceIssue = makeIssue({ status: "blocked", assigneeAgentId: ownerAgentId });mockIssueService.getById.mockResolvedValue(sourceIssue);mockIssueService.update.mockImplementation(async (_id: string, patch: Record<string, unknown>) => ({ ...sourceIssue, ...patch }));@@ -2043,7 +2109,7 @@ describe("agent issue mutation checkout ownership", () => {.post(`/api/issues/${issueId}/recovery-actions/resolve`).send({ actionId: recoveryActionId, outcome: "restored", sourceIssueStatus });expect.soft(res.status).toBe(409);- expect.soft(res.body.details?.code).toBe(cause === "native_workspace_sync_out_unsafe_archive" ? "workspace_export_automatic_recovery" : "workspace_export_retry_required");+ expect.soft(res.body.details?.code).toBe(cause === "native_workspace_sync_out_unsafe_archive" ? "workspace_export_automatic_recovery" : cause === "native_workspace_finalization_owner_unverified" ? "workspace_owner_stop_required" : "workspace_export_retry_required");expect.soft(mockIssueService.update).not.toHaveBeenCalled();expect.soft(mockHeartbeatService.wakeup).not.toHaveBeenCalled();});diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.tsindex 7af8d4c123..fa2abf0c52 100644--- a/server/src/routes/issues.ts+++ b/server/src/routes/issues.ts@@ -17,6 +17,7 @@ import { setIssueTitleSchema } from "@paperclipai/shared";import { resolveConfirmationFromComment } from "../services/confirmation-comment-resolution.js";import { createIssueReadTiming } from "../services/issue-read-timing.js";import { isNativeWorkspaceExportRepairCause } from "@paperclipai/shared";+import { resumeNativeWorkspaceAfterOwnerStop } from "../services/native-runtime/native-workspace-owner-recovery.js";import { retryNativeWorkspaceExport } from "../services/native-runtime/native-workspace-export-retry.js";import { queuedInteractionId, readQueuedInteractionResponse, hasQueuedInteractionResponse } from "../services/queued-interaction-response.js";import { deliverConversationComments, isConversation } from "../services/agent-conversations.js";@@ -96,6 +97,7 @@ import {resolveCreateIssueStatusDefault,resolveIssueRecoveryActionSchema,retryWorkspaceExportSchema,+ resumeWorkspaceFinalizationSchema,runnerGoalActionRequestSchema,feedbackTargetTypeSchema,feedbackTraceStatusSchema,@@ -9511,6 +9513,17 @@ export function issueRoutes(});});+ router.post("/issues/:id/recovery-actions/resume-workspace-finalization", validate(resumeWorkspaceFinalizationSchema), async (req, res) => {+ assertBoard(req);+ const issue = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Issue not found");+ if (!issue || !(await assertIssueReadAllowed(req, res, issue))) return;+ const decision = await access.decide({ actor: req.actor, action: "runtime:confirm_owner_stop", resource: { type: "company", companyId: issue.companyId } });+ if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision));+ const receipt = await resumeNativeWorkspaceAfterOwnerStop({ db, companyId: issue.companyId, issueId: issue.id,+ ...req.body, actorId: getActorInfo(req).actorId });+ res.status(202).json(receipt);+ });+router.post("/issues/:id/recovery-actions/retry-workspace-export", validate(retryWorkspaceExportSchema), async (req, res) => {assertBoard(req);const issue = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Issue not found");@@ -9757,6 +9770,10 @@ export function issueRoutes(throw conflict("Unsafe workspace export recovers automatically without another provider turn.", { code: "workspace_export_automatic_recovery" });}+ if ((outcome === "restored" || outcome === "false_positive") && activeRecoveryAction.cause === "native_workspace_finalization_owner_unverified") {+ throw conflict("Verify the previous controller and its copyback processes stopped, then use Resume saved result.", { code: "workspace_owner_stop_required" });+ }+if (outcome === "restored" && isNativeWorkspaceExportRepairCause(activeRecoveryAction.cause)) {throw conflict("Repair the retained sandbox, then use Retry workspace export to finish the accepted result without another provider turn.", { code: "workspace_export_retry_required" });}diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.tsindex 532e3a66a2..e749eaeef3 100644--- a/server/src/routes/openapi.ts+++ b/server/src/routes/openapi.ts@@ -240,6 +240,7 @@ import {createAcceptedPlanDecompositionSchema,resolveIssueRecoveryActionSchema,retryWorkspaceExportSchema,+ resumeWorkspaceFinalizationSchema,cancelIssueThreadInteractionSchema,// Secret provider configs and remote importcreateSecretProviderConfigSchema,@@ -10635,6 +10636,15 @@ registerCurrentRoute({summary: "List issue recovery actions",});+registerCurrentRoute({+ method: "post",+ path: "/api/issues/{id}/recovery-actions/resume-workspace-finalization",+ tags: ["issues"],+ summary: "Resume a saved native result after operator verification of the prior workspace owner's stop",+ body: resumeWorkspaceFinalizationSchema,+ responses: { 202: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict },+});+registerCurrentRoute({method: "post",path: "/api/issues/{id}/recovery-actions/retry-workspace-export",diff --git a/server/src/services/authorization.ts b/server/src/services/authorization.tsindex 7bc119b86a..9b68990080 100644--- a/server/src/services/authorization.ts+++ b/server/src/services/authorization.ts@@ -94,6 +94,7 @@ export type AuthorizationAction =| "project:read"| "project:write_workspace"| "runtime:manage"+ | "runtime:confirm_owner_stop"| "secrets:read"| "secrets:propose";@@ -184,6 +185,7 @@ function permissionForAction(action: AuthorizationAction): PermissionKey | nullaction === "issue:read" ||action === "project:read" ||action === "runtime:manage" ||+ action === "runtime:confirm_owner_stop" ||action === "secrets:read" ||action === "secrets:propose") {@@ -1858,6 +1860,11 @@ export function authorizationService(db: Db | DbTransaction) {});}+ if (input.action === "runtime:confirm_owner_stop" && input.actor.type !== "board") {+ return deny({ action: input.action, reason: "deny_unsupported_action",+ explanation: "Only a deployment administrator can confirm a workspace controller has stopped." });+ }+if (input.actor.type === "board") {let taskAssignmentPolicyEffect: AssignmentPolicyEffect | null = null;if (input.actor.source === "local_implicit") {@@ -1886,6 +1893,15 @@ export function authorizationService(db: Db | DbTransaction) {explanation: "Allowed because the actor is an instance admin.",});}+ if (input.action === "runtime:confirm_owner_stop") {+ const membership = await getActiveMembership(companyId, "user", input.actor.userId ?? "");+ if (membership?.membershipRole === "owner" || membership?.membershipRole === "admin") {+ return allow({ action: input.action, reason: "allow_role_default",+ explanation: "Allowed by the active company owner or administrator role." });+ }+ return deny({ action: input.action, reason: "deny_missing_grant",+ explanation: "Only a company owner, company administrator, or instance administrator can confirm a workspace controller has stopped." });+ }// What instance-admin elevation used to give cloud tenant users is// replaced by company-scoped visibility: an active membership in the// resource company grants the same read surface a same-company agentdiff --git a/server/src/services/cloud-runtime-identity.ts b/server/src/services/cloud-runtime-identity.tsindex e6567f2cf9..435c7f7562 100644--- a/server/src/services/cloud-runtime-identity.ts+++ b/server/src/services/cloud-runtime-identity.ts@@ -231,7 +231,7 @@ function decodeJsonPart(part: string, label: string): Record<string, unknown> {return parsed as Record<string, unknown>;}-function publicKeyForKid(env: NodeJS.ProcessEnv, kid: string) {+export function cloudRuntimePublicKeyForKid(env: NodeJS.ProcessEnv, kid: string) {const raw = nonEmpty(env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS);if (!raw) throw new Error("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is not configured");let parsed: unknown;@@ -276,7 +276,7 @@ function verifyClaims(input: {) {throw new Error("Cloud runtime identity protected header is invalid");}- const key = publicKeyForKid(input.env, header.kid);+ const key = cloudRuntimePublicKeyForKid(input.env, header.kid);const signature = Buffer.from(encodedSignature, "base64url");const signingInput = Buffer.from(`${encodedHeader}.${encodedPayload}`, "ascii");if (!verify(null, signingInput, key, signature)) {@@ -523,7 +523,7 @@ export function verifyCloudControlAssertion(input: {) {throw new Error("Cloud control protected header is invalid");}- const key = publicKeyForKid(env, header.kid);+ const key = cloudRuntimePublicKeyForKid(env, header.kid);const signature = Buffer.from(encodedSignature, "base64url");const signingInput = Buffer.from(`${encodedHeader}.${encodedPayload}`, "ascii");if (!verify(null, signingInput, key, signature)) {diff --git a/server/src/services/issue-recovery-actions.ts b/server/src/services/issue-recovery-actions.tsindex aeb7384bee..6dd31f091c 100644--- a/server/src/services/issue-recovery-actions.ts+++ b/server/src/services/issue-recovery-actions.ts@@ -375,6 +375,18 @@ export function issueRecoveryActionService(db: Db) {// beyond the advertised cap. A distinct identity can still supersede the// exhausted action through the branch above.if (isRecoveryBudgetExhausted(existing.evidence ?? {})) {+ // Older sweeps replaced this operator-only hold with instructions to+ // start another run. Restore the exact copyback recovery instructions+ // without reopening its budget or erasing a recorded stop confirmation.+ if (existing.cause === "native_workspace_finalization_owner_unverified"+ && input.cause === existing.cause && input.fingerprint === existing.fingerprint+ && !existing.evidence?.workspaceOwnerStop && existing.nextAction !== input.nextAction) {+ const [updated] = await db.update(issueRecoveryActions).set({ nextAction: input.nextAction, updatedAt: now })+ .where(and(eq(issueRecoveryActions.id, existing.id),+ inArray(issueRecoveryActions.status, [...ACTIVE_RECOVERY_ACTION_STATUSES]),+ sql`${issueRecoveryActions.evidence}->'workspaceOwnerStop' is null`)).returning();+ if (updated) return toReadModel(updated);+ }return existing;}const nextAttemptCount =@@ -417,8 +429,9 @@ export function issueRecoveryActionService(db: Db) {fingerprint: existing.fingerprint,},},- nextAction:- `Automatic recovery exhausted after ${attemptsUsed}/${effectiveMaxAttempts} attempts. ` ++ nextAction: existing.cause === "native_workspace_finalization_owner_unverified"+ ? input.nextAction+ : `Automatic recovery exhausted after ${attemptsUsed}/${effectiveMaxAttempts} attempts. ` +"Review the infrastructure failure and explicitly choose a replacement run or provider configuration.",wakePolicy: null,monitorPolicy: null,diff --git a/server/src/services/native-runtime/native-workspace-finalization-ownership.ts b/server/src/services/native-runtime/native-workspace-finalization-ownership.tsindex fcbff05fde..5168b9bf85 100644--- a/server/src/services/native-runtime/native-workspace-finalization-ownership.ts+++ b/server/src/services/native-runtime/native-workspace-finalization-ownership.ts@@ -1,15 +1,19 @@+import { logActivity } from "../activity-log.js";import { issueRecoveryActionService } from "../issue-recovery-actions.js";import { nativeSha256 } from "./canonical.js";import { NATIVE_WORKSPACE_OWNER_UNVERIFIED } from "./native-workspace-finalization-recovery-state.js";import { randomUUID } from "node:crypto";import os from "node:os";-import { and, eq, sql } from "drizzle-orm";-import { heartbeatRuns, workspaceOperations, withDedicatedDbConnection, type Db } from "@paperclipai/db";+import { and, eq, inArray, sql } from "drizzle-orm";+import { heartbeatRuns, issues, issueRecoveryActions, nativeRunFinalizations, workspaceOperations, withDedicatedDbConnection, type Db } from "@paperclipai/db";+import { currentNativeWorkspaceRecoveryResult } from "./native-workspace-recovery-binding.js";import {currentNativeControllerIdentity,evaluateNativeControllerTakeover,} from "./native-restart-recovery.js";+import { currentWorkspaceWorker, requestWorkspaceWorkerStopProof, type WorkspaceWorker, type WorkspaceWorkerStopProof } from "./native-workspace-platform-proof.js";+const OWNER_KEY = "nativeWorkspaceFinalizationOwner";const activeOwnerTokens = new Set<string>();// Positive same-boot evidence, retained only when durable cleanup could not be@@ -21,6 +25,7 @@ type Owner = {hostname: string;pid: number;processStartedAt: string;+ worker?: WorkspaceWorker;};export type NativeWorkspaceFinalizationOwnership = {token: string;@@ -58,7 +63,7 @@ export async function withNativeWorkspaceFinalizationOwnership<T>(const identity = await currentNativeControllerIdentity();const owner: Owner = {token: randomUUID(), controllerBootId: identity.bootId, hostname: os.hostname(), pid: identity.pid,- processStartedAt: identity.processStartedAt.toISOString(),+ processStartedAt: identity.processStartedAt.toISOString(), worker: currentWorkspaceWorker(),};const scope = and(eq(heartbeatRuns.id, input.runId), eq(heartbeatRuns.companyId, input.companyId), eq(heartbeatRuns.runtimeMode, "native"));const owned = and(scope, sql`${heartbeatRuns.runnerProfileJson}->${OWNER_KEY}->>'token' = ${owner.token}`);@@ -73,13 +78,42 @@ export async function withNativeWorkspaceFinalizationOwnership<T>(if (!run) throw new Error("native_workspace_finalization_binding_missing");const rawPrior = run.profile?.[OWNER_KEY];const prior = readOwner(rawPrior);+ let automaticResultId: string | undefined;+ const currentRecoveryResult = async (db: Db) => {+ if (!run.issueId) return null;+ // Use the arbiter's lock order and release row locks before copyback I/O.+ const [coordinator] = await db.select().from(nativeRunFinalizations).where(and(+ eq(nativeRunFinalizations.companyId, input.companyId), eq(nativeRunFinalizations.runId, input.runId),+ eq(nativeRunFinalizations.issueId, run.issueId),+ )).for("update");+ const [issue] = await db.select().from(issues).where(and(+ eq(issues.companyId, input.companyId), eq(issues.id, run.issueId),+ )).for("update");+ const [currentRun] = await db.select().from(heartbeatRuns).where(scope).for("update");+ return coordinator && issue && currentRun ? currentNativeWorkspaceRecoveryResult(db, {+ companyId: input.companyId, coordinator, issue, run: currentRun,+ }) : null;+ };+ let platformProof: WorkspaceWorkerStopProof | null = null;if (rawPrior != null) {const requireStopVerification = async () => {const fingerprint = nativeSha256({ runId: input.runId, owner: rawPrior });+ const nextAction = "Verify the previous controller and its workspace-copyback processes have stopped, then release only this exact workspace owner receipt using doc/native-workspace-finalization-recovery.md. Resume workspace finalization without another provider turn.";const recovery = issueRecoveryActionService(input.db);const existing = run.issueId ? await recovery.getActiveForIssue(input.companyId, run.issueId) : null;// Rechecking the same physical owner is not a new recovery attempt.if (existing?.cause === NATIVE_WORKSPACE_OWNER_UNVERIFIED && existing.fingerprint === fingerprint) {+ if (!existing.evidence?.workspaceOwnerStop) {+ await input.db.update(issueRecoveryActions).set({ nextAction, updatedAt: new Date(),+ evidence: sql`jsonb_set(coalesce(${issueRecoveryActions.evidence}, '{}'::jsonb), '{workspaceOwnerRecovery}', '{"state":"operator_required"}'::jsonb)`,+ }).where(and(+ eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.id, existing.id),+ eq(issueRecoveryActions.fingerprint, fingerprint), eq(issueRecoveryActions.cause, NATIVE_WORKSPACE_OWNER_UNVERIFIED),+ inArray(issueRecoveryActions.status, ["active", "escalated"]),+ sql`${issueRecoveryActions.evidence}->'workspaceOwnerStop' is null`,+ sql`(${issueRecoveryActions.nextAction} is distinct from ${nextAction} or ${issueRecoveryActions.evidence}#>>'{workspaceOwnerRecovery,state}' is distinct from 'operator_required')`,+ ));+ }return { acquired: false } as const;}if (run.issueId) await recovery.upsertSourceScoped({@@ -87,18 +121,21 @@ export async function withNativeWorkspaceFinalizationOwnership<T>(kind: "active_run_watchdog", ownerType: "board", returnOwnerAgentId: run.agentId,cause: NATIVE_WORKSPACE_OWNER_UNVERIFIED,fingerprint,- evidence: { runId: input.runId, owner: rawPrior },- nextAction: "Verify the previous controller and its workspace-copyback processes have stopped, then release only this exact workspace owner receipt using doc/native-workspace-finalization-recovery.md. Resume workspace finalization without another provider turn.",+ evidence: { runId: input.runId, owner: rawPrior, workspaceOwnerRecovery: { state: "operator_required" } },+ nextAction,wakePolicy: null, maxAttempts: 1, supersedeOnIdentityChange: true,});return { acquired: false } as const;};- // A foreign host or malformed receipt cannot prove the old physical owner stopped.- if (!prior || prior.hostname !== owner.hostname) return requireStopVerification();+ // The harness checks the exact owned deployment, including orphaned children.+ // A live deployment, unsigned answer or older receipt stays fenced.+ if (prior?.worker) platformProof = await requestWorkspaceWorkerStopProof({ runId: input.runId, ownerToken: prior.token, worker: prior.worker });+ if (!prior) return requireStopVerification();+ if (!platformProof && prior.hostname !== owner.hostname) return requireStopVerification();const knownJoinedHere = prior.controllerBootId === identity.bootId&& prior.pid === identity.pid && joinedOwnerTokens.has(prior.token)&& !activeOwnerTokens.has(prior.token);- if (!knownJoinedHere) {+ if (!platformProof && !knownJoinedHere) {const takeover = await evaluateNativeControllerTakeover({owner: { leaseOwner: prior.token, leaseExpiresAt: new Date(0), controllerPid: prior.pid,controllerProcessStartedAt: new Date(prior.processStartedAt) }, now: new Date(),@@ -119,15 +156,32 @@ export async function withNativeWorkspaceFinalizationOwnership<T>(}// Claim before the potentially ambiguous write so finally can remove// only our token even if the server committed but its response was lost.- activeOwnerTokens.add(owner.token);- claimed = true;- const [claimedRun] = await input.db.update(heartbeatRuns).set({- runnerProfileJson: sql`jsonb_set(coalesce(${heartbeatRuns.runnerProfileJson}, '{}'::jsonb), array[${OWNER_KEY}], ${JSON.stringify(owner)}::jsonb)`,- }).where(and(scope, prior- ? sql`${heartbeatRuns.runnerProfileJson}->${OWNER_KEY}->>'token' = ${prior.token}`- : sql`${heartbeatRuns.runnerProfileJson}->>${OWNER_KEY} is null`- )).returning({ id: heartbeatRuns.id });+ const claimOwner = async (db: Db) => {+ activeOwnerTokens.add(owner.token);+ claimed = true;+ const [claimedRun] = await db.update(heartbeatRuns).set({+ runnerProfileJson: sql`jsonb_set(coalesce(${heartbeatRuns.runnerProfileJson}, '{}'::jsonb), array[${OWNER_KEY}], ${JSON.stringify(owner)}::jsonb) || ${JSON.stringify(platformProof ? { nativeWorkspaceOwnerStopProof: platformProof } : {})}::jsonb`,+ }).where(and(scope, prior+ ? sql`${heartbeatRuns.runnerProfileJson}->${OWNER_KEY} = ${JSON.stringify(rawPrior)}::jsonb`+ : sql`${heartbeatRuns.runnerProfileJson}->>${OWNER_KEY} is null`+ )).returning({ id: heartbeatRuns.id });+ return claimedRun;+ };+ // A stop receipt proves the old worker exited, not that its task is still+ // current. Revalidate after the provider read and atomically with the CAS.+ const claimedRun = platformProof ? await input.db.transaction(async tx => {+ const result = await currentRecoveryResult(tx as unknown as Db);+ if (!result) return null;+ const claimedRun = await claimOwner(tx as unknown as Db);+ if (claimedRun) automaticResultId = result.id;+ return claimedRun;+ }) : await claimOwner(input.db);if (!claimedRun) return { acquired: false } as const;+ if (platformProof && run.issueId) await logActivity(input.db, {+ companyId: input.companyId, actorType: "system", actorId: "workspace-recovery",+ action: "issue.workspace_owner_stop_verified", entityType: "issue", entityId: run.issueId,+ runId: input.runId, details: { proof: platformProof },+ });if (prior) joinedOwnerTokens.delete(prior.token);const ownership: NativeWorkspaceFinalizationOwnership = {token: owner.token,@@ -137,12 +191,25 @@ export async function withNativeWorkspaceFinalizationOwnership<T>(await lock.execute(sql`select 1`);const [current] = await input.db.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(owned).limit(1);if (!current) throw new Error("owner_replaced");+ if (automaticResultId) {+ const result = await input.db.transaction(tx => currentRecoveryResult(tx as unknown as Db));+ if (result?.id !== automaticResultId) throw new Error("task_replaced");+ }} catch (cause) {throw new NativeWorkspaceFinalizationOwnershipLostError({ cause });}},};await ownership.assertHeld();+ if (platformProof && run.issueId) await input.db.update(issueRecoveryActions).set({+ evidence: sql`jsonb_set(coalesce(${issueRecoveryActions.evidence}, '{}'::jsonb), '{workspaceOwnerRecovery}', '{"state":"resuming"}'::jsonb)`,+ updatedAt: new Date(),+ }).where(and(+ eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.sourceIssueId, run.issueId),+ eq(issueRecoveryActions.cause, NATIVE_WORKSPACE_OWNER_UNVERIFIED),+ eq(issueRecoveryActions.fingerprint, nativeSha256({ runId: input.runId, owner: rawPrior })),+ inArray(issueRecoveryActions.status, ["active", "escalated"]),+ ));actionPromise = action(ownership);const value = await actionPromise;await ownership.assertHeld();diff --git a/server/src/services/native-runtime/native-workspace-finalizer.test.ts b/server/src/services/native-runtime/native-workspace-finalizer.test.tsindex 766dd1cf53..30737efb30 100644--- a/server/src/services/native-runtime/native-workspace-finalizer.test.ts+++ b/server/src/services/native-runtime/native-workspace-finalizer.test.ts@@ -1,3 +1,8 @@+import * as platformProof from "./native-workspace-platform-proof.js";+import { currentNativeControllerIdentity } from "./native-restart-recovery.js";+import { nativeSha256 } from "./canonical.js";+import { resumeNativeWorkspaceAfterOwnerStop } from "./native-workspace-owner-recovery.js";+import { issueRecoveryActionService } from "../issue-recovery-actions.js";import { spawn } from "node:child_process";import { once } from "node:events";import { readProcessStartedAt } from "../hot-restart.js";@@ -9,6 +14,8 @@ import path from "node:path";import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";import { and, desc, eq, sql } from "drizzle-orm";import {+ activityLog,+ agentWakeupRequests,agents,companies,completionContracts,@@ -19,11 +26,15 @@ import {nativeRunFinalizations,nativeRunResults,issueRecoveryActions,+ issueThreadInteractions,projects,workspaceOperations,+ workAssessments,+ statusDecisions,} from "@paperclipai/db";import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";import { resumeNativeWorkspaceFinalization } from "./native-workspace-finalizer.js";+import { reconcileNativeFinalizations } from "./native-finalization-reconciler.js";describe("native workspace finalization recovery", () => {let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;@@ -159,6 +170,225 @@ describe("native workspace finalization recovery", () => {}});+ async function seedAbandonedCopyback() {+ const seeded = await seedRun({ executionWorkspaceId: randomUUID(), title: "Accepted yield, abandoned copyback" });+ const owner = { token: randomUUID(), hostname: "replaced-controller", pid: 2,+ processStartedAt: "2026-10-10T00:02:55.560Z", controllerBootId: randomUUID() };+ await db.update(heartbeatRuns).set({ status: "running", nativePhase: "workspace_finalizing",+ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner, preserved: "workspace descriptor" },+ }).where(eq(heartbeatRuns.id, seeded.runId));+ await db.update(issues).set({ executionRunId: seeded.runId }).where(eq(issues.id, seeded.issueId));+ await db.update(nativeRunFinalizations).set({ phase: "workspace_finalizing" }).where(eq(nativeRunFinalizations.runId, seeded.runId));+ const recovery = issueRecoveryActionService(db);+ const hold = { companyId, sourceIssueId: seeded.issueId, kind: "active_run_watchdog" as const,+ ownerType: "board" as const, cause: "native_workspace_finalization_owner_unverified",+ fingerprint: nativeSha256({ runId: seeded.runId, owner }), evidence: { runId: seeded.runId, owner }, maxAttempts: 1,+ nextAction: "Verify the previous controller and its workspace-copyback processes have stopped. Resume workspace finalization without another provider turn." };+ const action = await recovery.upsertSourceScoped(hold);+ await recovery.upsertSourceScoped(hold);+ return { ...seeded, owner, actionId: action.id, companyId, db, ownerToken: owner.token,+ actorId: "operator", controllerAndCopybackStopped: true as const,+ stopEvidence: "Deployment platform confirms the exact old container and all its processes exited." };+ }++ it("retains copyback-specific instructions after repeated recovery sweeps", async () => {+ const seed = await seedAbandonedCopyback();+ const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, seed.actionId));+ expect(action.status).toBe("escalated");+ expect(action.nextAction).toContain("workspace-copyback processes have stopped");+ expect(action.nextAction).not.toContain("replacement run");+ await db.update(issueRecoveryActions).set({ nextAction: "Start a replacement run" }).where(eq(issueRecoveryActions.id, seed.actionId));+ await withNativeWorkspaceFinalizationOwnership(seed, async () => "unexpected");+ const [repaired] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, seed.actionId));+ expect(repaired.nextAction).toContain("workspace-copyback processes have stopped");+ expect(repaired.attemptCount).toBe(action.attemptCount);+ expect(repaired.evidence?.recoveryBudget).toEqual(action.evidence?.recoveryBudget);+ });++ it("releases only the verified owner and resumes the saved result without changing its disposition", async () => {+ const seed = await seedAbandonedCopyback();+ const [before] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId));+ expect(await withNativeWorkspaceFinalizationOwnership(seed, async () => "unexpected")).toEqual({ acquired: false });+ const receipt = await resumeNativeWorkspaceAfterOwnerStop(seed);+ expect(receipt).toEqual({ runId: seed.runId, resultId: before.resultId, status: "queued" });+ const [after] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId));+ expect(after).toEqual(before);+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson).toEqual({ preserved: "workspace descriptor" });+ expect(run.status).toBe("running");+ const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, seed.actionId));+ expect(action.evidence?.workspaceOwnerStop).toMatchObject({ owner: seed.owner, actorId: "operator", resultId: before.resultId, stopEvidence: seed.stopEvidence });+ expect(action.status).toBe("escalated"); // Only commitment resolves the hold.+ expect(action.wakePolicy).toBeNull();+ expect(await db.select().from(activityLog).where(eq(activityLog.runId, seed.runId))).toHaveLength(1);+ const file = path.join(workspaceRoot, seed.runId);+ expect(await withNativeWorkspaceFinalizationOwnership(seed, async () => {+ await fs.writeFile(file, "accepted work");+ return "resumed";+ })).toEqual({ acquired: true, value: "resumed" });+ expect(await fs.readFile(file, "utf8")).toBe("accepted work");+ // A duplicate confirmation neither clears a newer owner nor writes another audit.+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).resolves.toEqual(receipt);+ expect(await db.select().from(activityLog).where(eq(activityLog.runId, seed.runId))).toHaveLength(1);+ });++ it.each(["expired", "live", "unknown"] as const)("checks the coordinator lease before verified recovery: %s", async state => {+ const seed = await seedAbandonedCopyback();+ const leaseExpiresAt = state === "unknown" ? null : new Date(Date.now() + (state === "expired" ? -60_000 : 60_000));+ await db.update(nativeRunFinalizations).set({ leaseOwner: "previous-reconciler", leaseExpiresAt }).where(eq(nativeRunFinalizations.runId, seed.runId));+ if (state === "expired") {+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).resolves.toMatchObject({ status: "queued" });+ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId)))[0].leaseOwner).toBe("previous-reconciler");+ } else {+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toMatchObject({ status: 409 });+ expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId)))[0].runnerProfileJson?.nativeWorkspaceFinalizationOwner).toEqual(seed.owner);+ }+ });++ it.each(["pending", "resolved", "superseded", "unassigned"] as const)("requires authoritative reviewer recovery: %s", async state => {+ const seed = await seedAbandonedCopyback();+ const reviewerId = randomUUID(), sourceRunId = randomUUID(), sourceResultId = randomUUID(), assessmentId = randomUUID();+ const decisionId = randomUUID(), interactionId = randomUUID();+ await db.insert(agents).values({ id: reviewerId, companyId, name: "Review agent", adapterType: "paperclip_runner", status: "running" });+ const [actingRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId, nativeIssueId: seed.issueId, runtimeMode: "native", status: "succeeded", completionContractId: actingRun.completionContractId, createdAt: new Date(actingRun.createdAt.getTime() - 60_000) });+ await db.insert(nativeRunResults).values({ id: sourceResultId, companyId, issueId: seed.issueId, runId: sourceRunId, completionContractId: actingRun.completionContractId!, serverFingerprint: sourceResultId, schemaStatus: "accepted", resultJson: {}, canonicalSha256: sourceResultId });+ await db.insert(workAssessments).values({ id: assessmentId, companyId, issueId: seed.issueId, runId: sourceRunId, contractId: actingRun.completionContractId!, resultId: sourceResultId, triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: "in_progress", priorStatusVersion: 0, policyVersion: "fixture", assessmentJson: {}, inputDigest: assessmentId });+ const [issue] = await db.update(issues).set({ status: "in_review" }).where(eq(issues.id, seed.issueId)).returning();+ await db.insert(statusDecisions).values({ id: decisionId, companyId, issueId: seed.issueId, runId: sourceRunId, assessmentId, decisionVersion: 1, policyVersion: "fixture", fromStatus: "in_progress", toStatus: "in_review", reasonCode: "explicit_review", decisionJson: { projectedStatusVersion: issue.statusVersion }, decisionDigest: decisionId, applicationState: "applied" });+ await db.update(issues).set({ lastStatusDecisionId: state === "superseded" ? null : decisionId }).where(eq(issues.id, seed.issueId));+ await db.insert(issueThreadInteractions).values({ id: interactionId, companyId, issueId: seed.issueId, kind: "request_confirmation", sourceRunId, addresseeAgentId: state === "unassigned" ? agentId : reviewerId, createdByAgentId: agentId, continuationPolicy: "wake_assignee", status: state === "resolved" ? "accepted" : "pending", ...(state === "resolved" ? { resolvedByRunId: seed.runId, resolvedByAgentId: reviewerId } : {}), payload: { version: 1, prompt: "Review the completed work.", target: { type: "custom", key: "native_completion_review", revisionId: decisionId } } });+ await db.update(heartbeatRuns).set({ agentId: reviewerId, contextSnapshot: { issueId: seed.issueId, nativeReviewInteractionId: interactionId, nativeReviewDecisionId: decisionId } }).where(eq(heartbeatRuns.id, seed.runId));+ if (state === "pending" || state === "resolved") {+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).resolves.toMatchObject({ status: "queued" });+ expect((await db.select().from(issues).where(eq(issues.id, seed.issueId)))[0].assigneeAgentId).toBe(agentId);+ } else {+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toMatchObject({ status: 409 });+ }+ });++ it.each(([+ ["workspace_finalizing", "monitor"], ["workspace_finalizing", "approval"],+ ["retryable_failure", "monitor"], ["retryable_failure", "approval"],+ ["result_accepted", "monitor"], ["result_accepted", "approval"],+ ] as const).flatMap(([phase, wait]) => (["operator", "automatic"] as const).map(mode => [phase, wait, mode] as const)))("commits the saved yield from %s preserving its %s wait via %s recovery", async (phase, wait, mode) => {+ const seed = await seedAbandonedCopyback();+ await db.update(nativeRunFinalizations).set({ phase,+ ...(phase === "retryable_failure" ? { attempt: 1, nextAttemptAt: new Date(Date.now() - 60_000),+ failureCode: "native_workspace_sync_out_failed" } : {}),+ }).where(eq(nativeRunFinalizations.runId, seed.runId));+ await db.update(heartbeatRuns).set({ nativePhase: phase }).where(eq(heartbeatRuns.id, seed.runId));+ const workspaceId = randomUUID();+ const cwd = path.join(workspaceRoot, seed.runId);+ await fs.mkdir(cwd);+ await fs.writeFile(path.join(cwd, "saved.txt"), "accepted work");+ await db.insert(executionWorkspaces).values({ id: workspaceId, companyId, projectId,+ mode: "local", strategyType: "local_directory", name: "Retained workspace", cwd });+ const nextCheckAt = new Date(Date.now() - 60_000); // Already due, like the staging runs.+ const policy = { monitor: { nextCheckAt: nextCheckAt.toISOString(), notes: "Check the existing PR." } };+ await db.update(issues).set({ executionWorkspaceId: workspaceId,+ ...(wait === "monitor" ? { executionPolicy: policy, monitorNextCheckAt: nextCheckAt } : {}),+ }).where(eq(issues.id, seed.issueId));+ const interactionId = randomUUID();+ if (wait === "approval") await db.insert(issueThreadInteractions).values({+ id: interactionId, companyId, issueId: seed.issueId, sourceRunId: seed.runId,+ createdByAgentId: agentId, kind: "request_confirmation", status: "pending",+ payload: { version: 1, prompt: "Approve the plan?", target: { type: "issue_document",+ issueId: seed.issueId, key: "plan", revisionId: randomUUID() } },+ });+ const resultJson = {+ result: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "yielded",+ summary: "Wait for the existing continuation.",+ completionClaim: { contractRevision: "test", objectiveSatisfied: false, criteria: [],+ remainingWork: [{ description: "Wait for the check or plan confirmation.", blocksCompletion: true }] },+ continuation: { kind: wait === "monitor" ? "monitor" : "response_wake",+ summary: "Wait for the existing continuation.", idempotencyKey: "saved-wait" },+ evidence: [], verification: [], attentionRequests: [], artifacts: [] },+ terminal: { schema: "paperclip.prp.terminal.v1", turnTerminalState: "completed",+ runTerminalState: "succeeded", reportedWorkDisposition: "yielded" },+ };+ await db.update(nativeRunResults).set({ resultJson }).where(eq(nativeRunResults.runId, seed.runId));+ const [accepted] = await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, seed.runId));++ // The real reconciler must remain fenced before operator confirmation.+ await reconcileNativeFinalizations(db, [seed.runId]);+ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId)))[0].phase).toBe(phase);+ const [beforeConfirmation] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId));+ if (mode === "operator") {+ await resumeNativeWorkspaceAfterOwnerStop(seed);+ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId)))[0]).toEqual(beforeConfirmation);+ await reconcileNativeFinalizations(db, [seed.runId]);+ } else {+ const worker = { provider: "railway" as const, projectId: "p", environmentId: "e", serviceId: "s", deploymentId: "d" };+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ await db.update(heartbeatRuns).set({ runnerProfileJson: { ...run.runnerProfileJson, nativeWorkspaceFinalizationOwner: { ...seed.owner, worker } } }).where(eq(heartbeatRuns.id, seed.runId));+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockResolvedValue({ v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", runId: seed.runId, ownerToken: seed.owner.token, worker, stopped: true, iat: 1, exp: 61 });+ try { await reconcileNativeFinalizations(db, [seed.runId]); } finally { probe.mockRestore(); }+ }++ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId)))[0]).toMatchObject({ phase: "committed", resultId: accepted.id });+ expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId)))[0]).toMatchObject({ status: "succeeded", nativePhase: "committed" });+ expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, seed.runId))).toEqual([accepted]);+ expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, seed.actionId)))[0].status).toBe("resolved");+ const [issue] = await db.select().from(issues).where(eq(issues.id, seed.issueId));+ expect(issue.status).toBe(wait === "monitor" ? "in_progress" : "in_review");+ if (wait === "monitor") {+ expect(issue.monitorNextCheckAt).toEqual(nextCheckAt);+ expect(issue.executionPolicy).toEqual(policy);+ } else {+ expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].status).toBe("pending");+ }+ expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.nativeIssueId, seed.issueId))).toHaveLength(1);+ expect(await db.select().from(agentWakeupRequests).where(sql`${agentWakeupRequests.payload}->>'issueId' = ${seed.issueId}`)).toHaveLength(0);+ expect(await db.select().from(workspaceOperations).where(and(eq(workspaceOperations.heartbeatRunId, seed.runId),+ eq(workspaceOperations.phase, "workspace_finalize"), eq(workspaceOperations.status, "succeeded")))).toHaveLength(1);+ expect(await fs.readFile(path.join(cwd, "saved.txt"), "utf8")).toBe("accepted work");+ });++ it.each(["token", "company", "result", "newer owner", "missing confirmation", "malformed owner"])("rejects stale or unverified copyback recovery: %s", async changed => {+ const seed = await seedAbandonedCopyback();+ if (changed === "malformed owner") {+ const owner = { token: seed.ownerToken };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ await db.update(issueRecoveryActions).set({ evidence: { runId: seed.runId, owner } }).where(eq(issueRecoveryActions.id, seed.actionId));+ }+ if (changed === "token") seed.ownerToken = randomUUID();+ if (changed === "company") seed.companyId = foreignCompanyId;+ if (changed === "result") await db.update(nativeRunFinalizations).set({ resultId: null }).where(eq(nativeRunFinalizations.runId, seed.runId));+ if (changed === "newer owner") await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: { ...seed.owner, token: randomUUID() } } }).where(eq(heartbeatRuns.id, seed.runId));+ if (changed === "missing confirmation") Object.assign(seed, { controllerAndCopybackStopped: false });+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toThrow();+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson?.nativeWorkspaceFinalizationOwner).toBeDefined();+ expect(await db.select().from(activityLog).where(eq(activityLog.runId, seed.runId))).toHaveLength(0);+ });++ it("rejects a locally live owner even after its advisory connection is gone", async () => {+ const seed = await seedAbandonedCopyback();+ const identity = await currentNativeControllerIdentity();+ const owner = { ...seed.owner, hostname: os.hostname(), pid: identity.pid,+ processStartedAt: identity.processStartedAt.toISOString(), controllerBootId: identity.bootId };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ await db.update(issueRecoveryActions).set({ evidence: { runId: seed.runId, owner } }).where(eq(issueRecoveryActions.id, seed.actionId));+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toThrow("still active");+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson?.nativeWorkspaceFinalizationOwner).toEqual(owner);+ });++ it("does not reopen an old result after a newer task execution exists", async () => {+ const seed = await seedAbandonedCopyback();+ await db.insert(heartbeatRuns).values({ companyId, agentId, status: "queued", nativeIssueId: seed.issueId });+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toThrow("no longer current");+ });++ it("rejects stop confirmation while another controller holds the physical lock", async () => {+ const seed = await seedAbandonedCopyback();+ await db.transaction(async tx => {+ await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`native-workspace-finalization:${companyId}:${seed.runId}`}, 0))`);+ await expect(resumeNativeWorkspaceAfterOwnerStop(seed)).rejects.toThrow("still active");+ });+ });+it("reuses a successful export even if an old controller wrote a later failed barrier", async () => {const seeded = await seedRun({ executionWorkspaceId: randomUUID(), title: "Late failed barrier" });const [success] = await db.insert(workspaceOperations).values({ companyId, heartbeatRunId: seeded.runId,@@ -311,6 +541,137 @@ describe("native workspace finalization recovery", () => {.toEqual({ acquired: true, value: "recovered" });});+ it("automatically adopts a proven stopped deployment without another agent turn", async () => {+ const seed = await seedAbandonedCopyback();+ const worker = { provider: "railway" as const, projectId: "fixture-project", environmentId: "fixture-env", serviceId: "fixture-service", deploymentId: "fixture-old" };+ const owner = { ...seed.owner, worker };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner, preserved: "workspace descriptor" } }).where(eq(heartbeatRuns.id, seed.runId));+ await db.update(issueRecoveryActions).set({ fingerprint: nativeSha256({ runId: seed.runId, owner }), evidence: { runId: seed.runId, owner, preserved: "recovery evidence" } }).where(eq(issueRecoveryActions.id, seed.actionId));+ const [before] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId));+ const proof = { v: 1 as const, iss: "paperclip-cloud" as const, aud: "paperclip-workspace-worker-stop/v1" as const, sub: "stack", runId: seed.runId, ownerToken: owner.token, worker, stopped: true as const, iat: 1, exp: 61 };+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockResolvedValue(proof);+ try {+ const work = vi.fn(async () => {+ const [recovery] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, seed.actionId));+ expect(recovery.evidence?.workspaceOwnerRecovery).toEqual({ state: "resuming" });+ expect(recovery.evidence?.preserved).toBe("recovery evidence");+ return "saved workspace copied";+ });+ expect(await withNativeWorkspaceFinalizationOwnership(seed, work)).toEqual({ acquired: true, value: "saved workspace copied" });+ expect(probe).toHaveBeenCalledWith({ runId: seed.runId, ownerToken: owner.token, worker });+ expect(work).toHaveBeenCalledOnce();+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson?.nativeWorkspaceFinalizationOwner).toBeUndefined();+ expect(run.runnerProfileJson?.nativeWorkspaceOwnerStopProof).toEqual(proof);+ expect(run.runnerProfileJson?.preserved).toBe("workspace descriptor");+ const [after] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId));+ expect(after).toEqual(before);+ expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, companyId))).toHaveLength(0);+ } finally { probe.mockRestore(); }+ });++ it("retains the fence for unknown platform evidence and owner changes during verification", async () => {+ const seed = await seedAbandonedCopyback();+ const worker = { provider: "railway" as const, projectId: "p", environmentId: "e", serviceId: "s", deploymentId: "d" };+ const owner = { ...seed.owner, worker };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockResolvedValue(null);+ const work = vi.fn();+ try {+ expect(await withNativeWorkspaceFinalizationOwnership(seed, work)).toEqual({ acquired: false });+ const [recovery] = await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.sourceIssueId, seed.issueId), eq(issueRecoveryActions.fingerprint, nativeSha256({ runId: seed.runId, owner }))));+ expect(recovery.evidence?.workspaceOwnerRecovery).toEqual({ state: "operator_required" });+ const replacement = { ...owner, token: randomUUID() };+ probe.mockImplementation(async () => {+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: replacement } }).where(eq(heartbeatRuns.id, seed.runId));+ return { v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", runId: seed.runId, ownerToken: owner.token, worker, stopped: true, iat: 1, exp: 61 };+ });+ expect(await withNativeWorkspaceFinalizationOwnership(seed, work)).toEqual({ acquired: false });+ expect(work).not.toHaveBeenCalled();+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson?.nativeWorkspaceFinalizationOwner).toEqual(replacement);+ } finally { probe.mockRestore(); }+ });++ it.each(["done", "cancelled", "execution replaced", "checkout replaced", "newer run", "queued legacy run", "reassigned", "result unbound", "result rejected", "run stopped"])("does not copy saved files after the task moves on during platform verification: %s", async changed => {+ const seed = await seedAbandonedCopyback();+ const worker = { provider: "railway" as const, projectId: "p", environmentId: "e", serviceId: "s", deploymentId: "d" };+ const owner = { ...seed.owner, worker };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ const file = path.join(workspaceRoot, `${seed.runId}.txt`);+ await fs.writeFile(file, "new task work");+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockImplementation(async () => {+ if (changed === "done" || changed === "cancelled") await db.update(issues).set({ status: changed }).where(eq(issues.id, seed.issueId));+ if (changed === "execution replaced") await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seed.issueId));+ if (changed === "checkout replaced") {+ const [other] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "succeeded" }).returning();+ await db.update(issues).set({ checkoutRunId: other.id }).where(eq(issues.id, seed.issueId));+ }+ if (changed === "reassigned") await db.update(issues).set({ assigneeAgentId: null }).where(eq(issues.id, seed.issueId));+ if (changed === "result unbound") await db.update(nativeRunFinalizations).set({ resultId: null }).where(eq(nativeRunFinalizations.runId, seed.runId));+ if (changed === "result rejected") await db.update(nativeRunResults).set({ schemaStatus: "rejected" }).where(eq(nativeRunResults.runId, seed.runId));+ if (changed === "run stopped") await db.update(heartbeatRuns).set({ status: "failed" }).where(eq(heartbeatRuns.id, seed.runId));+ if (changed === "newer run" || changed === "queued legacy run") {+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ await db.insert(heartbeatRuns).values({ companyId, agentId,+ runtimeMode: changed === "newer run" ? "native" : "legacy",+ nativeIssueId: changed === "newer run" ? seed.issueId : null,+ contextSnapshot: { issueId: seed.issueId }, status: changed === "newer run" ? "succeeded" : "queued",+ createdAt: new Date(run.createdAt.getTime() + (changed === "newer run" ? 60_000 : -60_000)),+ });+ }+ return { v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", runId: seed.runId, ownerToken: owner.token, worker, stopped: true, iat: 1, exp: 61 };+ });+ const copyback = vi.fn(async () => fs.writeFile(file, "old sandbox work"));+ try {+ expect(await withNativeWorkspaceFinalizationOwnership(seed, copyback)).toEqual({ acquired: false });+ expect(copyback).not.toHaveBeenCalled();+ expect(await fs.readFile(file, "utf8")).toBe("new task work");+ const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seed.runId));+ expect(run.runnerProfileJson?.nativeWorkspaceFinalizationOwner).toEqual(owner);+ expect(run.runnerProfileJson?.nativeWorkspaceOwnerStopProof).toBeUndefined();+ } finally { probe.mockRestore(); }+ });++ it("rechecks automatic task ownership at the copyback publication boundary", async () => {+ const seed = await seedAbandonedCopyback();+ const worker = { provider: "railway" as const, projectId: "p", environmentId: "e", serviceId: "s", deploymentId: "d" };+ const owner = { ...seed.owner, worker };+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockResolvedValue({ v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", runId: seed.runId, ownerToken: owner.token, worker, stopped: true, iat: 1, exp: 61 });+ const write = vi.fn();+ try {+ await expect(withNativeWorkspaceFinalizationOwnership(seed, async ownership => {+ await db.update(issues).set({ status: "cancelled" }).where(eq(issues.id, seed.issueId));+ await ownership.assertHeld();+ write();+ })).rejects.toThrow("native_workspace_finalization_ownership_lost");+ expect(write).not.toHaveBeenCalled();+ } finally { probe.mockRestore(); }+ });++ it.each(["closed", "superseded"])("does not start a workspace operation when the real reconciler selects a %s task", async changed => {+ const seed = await seedAbandonedCopyback();+ const worker = { provider: "railway" as const, projectId: "p", environmentId: "e", serviceId: "s", deploymentId: "d" };+ const owner = { ...seed.owner, worker };+ const workspaceId = randomUUID(), cwd = path.join(workspaceRoot, seed.runId);+ await fs.mkdir(cwd);+ await fs.writeFile(path.join(cwd, "current.txt"), "new task work");+ await db.insert(executionWorkspaces).values({ id: workspaceId, companyId, projectId, mode: "local", strategyType: "local_directory", name: "Current workspace", cwd });+ await db.update(issues).set({ executionWorkspaceId: workspaceId,+ ...(changed === "closed" ? { status: "done" } : { executionRunId: null }),+ }).where(eq(issues.id, seed.issueId));+ await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeWorkspaceFinalizationOwner: owner } }).where(eq(heartbeatRuns.id, seed.runId));+ const probe = vi.spyOn(platformProof, "requestWorkspaceWorkerStopProof").mockResolvedValue({ v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", runId: seed.runId, ownerToken: owner.token, worker, stopped: true, iat: 1, exp: 61 });+ try {+ await reconcileNativeFinalizations(db, [seed.runId]);+ expect(probe).toHaveBeenCalledOnce();+ expect(await db.select().from(workspaceOperations).where(eq(workspaceOperations.heartbeatRunId, seed.runId))).toHaveLength(0);+ expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, seed.runId)))[0].phase).toBe("workspace_finalizing");+ expect(await fs.readFile(path.join(cwd, "current.txt"), "utf8")).toBe("new task work");+ } finally { probe.mockRestore(); }+ });+it("surfaces an unverifiable foreign controller as board recovery without physical writes", async () => {const seeded = await seedRun({ executionWorkspaceId: randomUUID(), title: "Foreign copyback controller" });const owner = { token: randomUUID(), hostname: "other-controller.invalid", pid: 42, processStartedAt: new Date().toISOString() };diff --git a/server/src/services/native-runtime/native-workspace-owner-recovery.ts b/server/src/services/native-runtime/native-workspace-owner-recovery.tsnew file mode 100644index 0000000000..4b3e19e331--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-owner-recovery.ts@@ -0,0 +1,87 @@+import os from "node:os";+import { and, eq, sql } from "drizzle-orm";+import { heartbeatRuns, issues, issueRecoveryActions, nativeRunFinalizations, type Db } from "@paperclipai/db";+import { resumeWorkspaceFinalizationSchema } from "@paperclipai/shared";+import { conflict } from "../../errors.js";+import { persistActivity, publishActivity } from "../activity-log.js";+import { currentNativeWorkspaceRecoveryResult } from "./native-workspace-recovery-binding.js";+import { evaluateNativeControllerTakeover } from "./native-restart-recovery.js";++const OWNER_KEY = "nativeWorkspaceFinalizationOwner";+const record = (value: unknown): Record<string, unknown> => value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};+const changed = () => conflict("The recorded workspace owner is no longer current. Refresh the task and inspect its run.", { code: "workspace_owner_recovery_stale" });++/** Board-only admission after deployment-platform verification. A missing DB+ * lock or an expired lease alone never proves physical copyback has stopped. */+export async function resumeNativeWorkspaceAfterOwnerStop(input: {+ db: Db; companyId: string; issueId: string; actionId: string; runId: string;+ ownerToken: string; controllerAndCopybackStopped: true; stopEvidence: string; actorId: string;+}) {+ const confirmation = resumeWorkspaceFinalizationSchema.parse({+ actionId: input.actionId, runId: input.runId, ownerToken: input.ownerToken,+ controllerAndCopybackStopped: input.controllerAndCopybackStopped, stopEvidence: input.stopEvidence,+ });+ const admitted = await input.db.transaction(async tx => {+ // Share the physical copyback lock. Do not wait behind an active exporter+ // and then apply a confirmation that described an earlier owner.+ const locks = await tx.execute(sql`select pg_try_advisory_xact_lock(hashtextextended(${`native-workspace-finalization:${input.companyId}:${input.runId}`}, 0)) as acquired`);+ if (locks[0]?.acquired !== true) throw conflict("Workspace copyback is still active. Wait for it to stop before confirming recovery.");+ // Match the status arbiter's coordinator -> issue -> run lock order.+ const [coordinator] = await tx.select().from(nativeRunFinalizations).where(and(eq(nativeRunFinalizations.companyId, input.companyId), eq(nativeRunFinalizations.issueId, input.issueId), eq(nativeRunFinalizations.runId, input.runId))).for("update");+ const [issue] = await tx.select().from(issues).where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.issueId))).for("update");+ const [run] = await tx.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, input.companyId), eq(heartbeatRuns.nativeIssueId, input.issueId), eq(heartbeatRuns.id, input.runId))).for("update");+ const [action] = await tx.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.companyId, input.companyId), eq(issueRecoveryActions.sourceIssueId, input.issueId), eq(issueRecoveryActions.id, input.actionId))).for("update");+ if (!issue || !run || !coordinator || !action || action.evidence?.runId !== run.id+ || action.cause !== "native_workspace_finalization_owner_unverified" || action.ownerType !== "board"+ || action.kind !== "active_run_watchdog" || !["active", "escalated"].includes(action.status)+ || (coordinator.leaseOwner && (!coordinator.leaseExpiresAt || coordinator.leaseExpiresAt.getTime() > Date.now()))) throw changed();+ const result = await currentNativeWorkspaceRecoveryResult(tx as unknown as Db, {+ companyId: input.companyId, issue, run, coordinator,+ });+ if (!result) throw changed();+ const prior = record(run.runnerProfileJson?.[OWNER_KEY]);+ const recordedOwner = record(action.evidence?.owner);+ const stop = record(action.evidence?.workspaceOwnerStop);+ const receipt = { runId: run.id, resultId: result.id, status: "queued" as const };+ if (run.runnerProfileJson?.[OWNER_KEY] == null && record(stop.owner).token === input.ownerToken+ && stop.resultId === result.id) return { receipt, publication: null };+ if (typeof prior.hostname !== "string" || !prior.hostname+ || !Number.isInteger(prior.pid) || Number(prior.pid) <= 0+ || typeof prior.processStartedAt !== "string" || !Number.isFinite(Date.parse(prior.processStartedAt))+ || prior.token !== input.ownerToken || recordedOwner.token !== input.ownerToken+ || prior.hostname !== recordedOwner.hostname || prior.pid !== recordedOwner.pid+ || prior.processStartedAt !== recordedOwner.processStartedAt+ || prior.controllerBootId !== recordedOwner.controllerBootId) throw changed();+ // Reject a confirmation contradicted by a locally observable live owner,+ // even if that owner's advisory connection has already disconnected.+ if (prior.hostname === os.hostname()) {+ const takeover = await evaluateNativeControllerTakeover({ owner: {+ leaseOwner: input.ownerToken, leaseExpiresAt: new Date(0),+ controllerPid: typeof prior.pid === "number" ? prior.pid : null,+ controllerProcessStartedAt: typeof prior.processStartedAt === "string" ? new Date(prior.processStartedAt) : null,+ }, now: new Date() });+ if (!takeover.allowed) throw conflict("The previous workspace controller is still active or its stop cannot be verified.");+ }+ const now = new Date();+ const proof = { owner: prior, resultId: result.id, actorId: input.actorId,+ stopEvidence: confirmation.stopEvidence, confirmedAt: now.toISOString() };+ await tx.update(heartbeatRuns).set({+ runnerProfileJson: sql`${heartbeatRuns.runnerProfileJson} - ${OWNER_KEY}`, updatedAt: now,+ }).where(and(eq(heartbeatRuns.companyId, input.companyId), eq(heartbeatRuns.id, run.id),+ sql`${heartbeatRuns.runnerProfileJson}->${OWNER_KEY}->>'token' = ${input.ownerToken}`));+ await tx.update(issueRecoveryActions).set({ evidence: { ...action.evidence, workspaceOwnerStop: proof },+ nextAction: "Workspace finalization is queued for the saved result. The agent will not repeat its work.",+ wakePolicy: null, monitorPolicy: null, updatedAt: now,+ }).where(eq(issueRecoveryActions.id, action.id));+ const publication = await persistActivity(tx as unknown as Db, { companyId: input.companyId,+ actorType: "user", actorId: input.actorId, action: "issue.workspace_owner_stop_confirmed",+ entityType: "issue", entityId: input.issueId, runId: input.runId,+ details: { recoveryActionId: input.actionId, ...proof },+ });+ // Leave the result, coordinator, source lease, task status and wait paths+ // untouched. The normal reconciler resumes copyback and arbitrates it.+ return { receipt, publication };+ });+ if (admitted.publication) publishActivity(admitted.publication.publication);+ return admitted.receipt;+}diff --git a/server/src/services/native-runtime/native-workspace-platform-proof.test.ts b/server/src/services/native-runtime/native-workspace-platform-proof.test.tsnew file mode 100644index 0000000000..fb3297ca80--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-platform-proof.test.ts@@ -0,0 +1,34 @@+import { generateKeyPairSync, sign } from "node:crypto";+import { describe, expect, it, vi } from "vitest";+import { verifyWorkspaceWorkerStopReceipt, requestWorkspaceWorkerStopProof, currentWorkspaceWorker } from "./native-workspace-platform-proof.js";+const { privateKey, publicKey } = generateKeyPairSync("ed25519");+const env = { PAPERCLIP_CLOUD_STACK_ID: "stack", PAPERCLIP_CLOUD_API_ORIGIN: "https://cloud.example", PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "fixture-token", PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...publicKey.export({ format: "jwk" }), kid: "test", alg: "EdDSA", use: "sig" }] }) };+const worker = { provider: "railway" as const, projectId: "project", environmentId: "environment", serviceId: "service", deploymentId: "old-deployment" };+const binding = { runId: "run", ownerToken: "owner", worker };+const now = new Date("2099-01-01T00:00:00Z");+function receipt(overrides = {}) {+ const header = Buffer.from(JSON.stringify({ alg: "EdDSA", kid: "test", typ: "paperclip-workspace-worker-stop+jwt" })).toString("base64url");+ const payload = Buffer.from(JSON.stringify({ v: 1, iss: "paperclip-cloud", aud: "paperclip-workspace-worker-stop/v1", sub: "stack", ...binding, stopped: true, iat: now.getTime()/1000, exp: now.getTime()/1000+60, ...overrides })).toString("base64url");+ return `${header}.${payload}.${sign(null, Buffer.from(`${header}.${payload}`), privateKey).toString("base64url")}`;+}+describe("workspace platform stop proof", () => {+ it("accepts only signed proof for the exact stack, owner, run and full stopped deployment", () => {+ expect(verifyWorkspaceWorkerStopReceipt(receipt(), binding, env, now)).toMatchObject({ stopped: true, worker });+ for (const invalid of [{ sub: "other-stack" }, { runId: "other-run" }, { ownerToken: "other-owner" }, { stopped: false }, { exp: now.getTime()/1000-1 }, { worker: { ...worker, deploymentId: "other" } }, { aud: "paperclip-cloud-control/v1" }]) {+ expect(verifyWorkspaceWorkerStopReceipt(receipt(invalid), binding, env, now)).toBeNull();+ }+ expect(verifyWorkspaceWorkerStopReceipt(receipt().slice(0,-5)+"wrong", binding, env, now)).toBeNull();+ });+ it("fails closed on live, unknown, unreachable and unsigned platform responses", async () => {+ for (const response of [{ stopped: false }, { stopped: true }, { receipt: "forged" }]) {+ const fetchImpl = vi.fn().mockResolvedValue(Response.json(response));+ expect(await requestWorkspaceWorkerStopProof(binding, { env, now, fetchImpl })).toBeNull();+ }+ expect(await requestWorkspaceWorkerStopProof(binding, { env, now, fetchImpl: vi.fn().mockRejectedValue(new Error("offline")) })).toBeNull();+ expect(await requestWorkspaceWorkerStopProof(binding, { env, now, fetchImpl: vi.fn().mockResolvedValue(Response.json({ receipt: receipt() })) })).toMatchObject({ stopped: true });+ });+ it("records a complete platform identity, and does not guess legacy ownership", () => {+ expect(currentWorkspaceWorker({ RAILWAY_PROJECT_ID: "project", RAILWAY_ENVIRONMENT_ID: "environment", RAILWAY_SERVICE_ID: "service", RAILWAY_DEPLOYMENT_ID: "old-deployment" })).toEqual(worker);+ expect(currentWorkspaceWorker({ RAILWAY_DEPLOYMENT_ID: "old-deployment" })).toBeUndefined();+ });+});diff --git a/server/src/services/native-runtime/native-workspace-platform-proof.ts b/server/src/services/native-runtime/native-workspace-platform-proof.tsnew file mode 100644index 0000000000..04b452d1d7--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-platform-proof.ts@@ -0,0 +1,65 @@+import { verify } from "node:crypto";+import { cloudRuntimePublicKeyForKid } from "../cloud-runtime-identity.js";+import { getCloudStackContext } from "../cloud-instance.js";++/** Deployment identity covers the controller and every orphaned tar/Git child. */+export type WorkspaceWorker = {+ provider: "railway"; projectId: string; environmentId: string; serviceId: string; deploymentId: string;+};+export type WorkspaceStopBinding = { runId: string; ownerToken: string; worker: WorkspaceWorker };+export type WorkspaceWorkerStopProof = WorkspaceStopBinding & {+ v: 1; iss: "paperclip-cloud"; aud: "paperclip-workspace-worker-stop/v1"; sub: string;+ stopped: true; iat: number; exp: number;+};+function validWorker(worker: unknown): worker is WorkspaceWorker {+ if (!worker || typeof worker !== "object" || Array.isArray(worker)) return false;+ const value = worker as Record<string, unknown>;+ return value.provider === "railway" && ["projectId", "environmentId", "serviceId", "deploymentId"].every(key =>+ typeof value[key] === "string" && value[key].length > 0 && value[key].length <= 200);+}+export function currentWorkspaceWorker(env: NodeJS.ProcessEnv = process.env): WorkspaceWorker | undefined {+ const ids = [env.RAILWAY_PROJECT_ID, env.RAILWAY_ENVIRONMENT_ID, env.RAILWAY_SERVICE_ID, env.RAILWAY_DEPLOYMENT_ID];+ if (ids.some(id => !id?.trim())) return undefined;+ return { provider: "railway", projectId: ids[0]!, environmentId: ids[1]!, serviceId: ids[2]!, deploymentId: ids[3]! };+}+export function verifyWorkspaceWorkerStopReceipt(+ receipt: string, binding: WorkspaceStopBinding, env: NodeJS.ProcessEnv = process.env, now = new Date(),+): WorkspaceWorkerStopProof | null {+ try {+ if (receipt.length > 16_000 || !validWorker(binding.worker)) return null;+ const parts = receipt.split(".");+ if (parts.length !== 3 || parts.some(part => !/^[A-Za-z0-9_-]+$/.test(part))) return null;+ const header = JSON.parse(Buffer.from(parts[0], "base64url").toString("utf8"));+ if (header.alg !== "EdDSA" || header.typ !== "paperclip-workspace-worker-stop+jwt" || typeof header.kid !== "string") return null;+ if (!verify(null, Buffer.from(`${parts[0]}.${parts[1]}`, "ascii"), cloudRuntimePublicKeyForKid(env, header.kid), Buffer.from(parts[2], "base64url"))) return null;+ const proof = JSON.parse(Buffer.from(parts[1], "base64url").toString("utf8")) as WorkspaceWorkerStopProof;+ const seconds = Math.floor(now.getTime()/1000);+ if (proof.v !== 1 || proof.iss !== "paperclip-cloud" || proof.aud !== "paperclip-workspace-worker-stop/v1"+ || !env.PAPERCLIP_CLOUD_STACK_ID || proof.sub !== env.PAPERCLIP_CLOUD_STACK_ID || proof.stopped !== true+ || proof.runId !== binding.runId || proof.ownerToken !== binding.ownerToken+ || !Number.isInteger(proof.iat) || !Number.isInteger(proof.exp) || proof.iat > seconds+30+ || proof.exp <= seconds || proof.exp <= proof.iat || proof.exp-proof.iat > 60+ || !validWorker(proof.worker) || Object.keys(binding.worker).some(key => proof.worker[key as keyof WorkspaceWorker] !== binding.worker[key as keyof WorkspaceWorker])) return null;+ return proof;+ } catch { return null; }+}+/** Unsupported platforms, legacy receipts and unavailable evidence never authorize takeover. */+export async function requestWorkspaceWorkerStopProof(binding: WorkspaceStopBinding, options: {+ env?: NodeJS.ProcessEnv; fetchImpl?: typeof fetch; now?: Date;+} = {}): Promise<WorkspaceWorkerStopProof | null> {+ const env = options.env ?? process.env;+ const context = getCloudStackContext(env);+ const token = env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN?.trim();+ if (!context?.stackId || !context.cloudOrigin || !token || !validWorker(binding.worker)) return null;+ try {+ const origin = new URL(context.cloudOrigin);+ if (origin.protocol !== "https:" || origin.username || origin.password) return null;+ const response = await (options.fetchImpl ?? fetch)(`${origin.origin}/v1/tenant/workspace-worker-stop`, {+ method: "POST", redirect: "error", headers: { authorization: `Bearer ${token}`, "x-paperclip-cloud-stack-id": context.stackId, "content-type": "application/json" },+ body: JSON.stringify(binding), signal: AbortSignal.timeout(10_000),+ });+ if (!response.ok) return null;+ const payload = await response.json() as { receipt?: unknown };+ return typeof payload.receipt === "string" ? verifyWorkspaceWorkerStopReceipt(payload.receipt, binding, env, options.now) : null;+ } catch { return null; }+}diff --git a/server/src/services/native-runtime/native-workspace-recovery-binding.ts b/server/src/services/native-runtime/native-workspace-recovery-binding.tsnew file mode 100644index 0000000000..20c4ee6861--- /dev/null+++ b/server/src/services/native-runtime/native-workspace-recovery-binding.ts@@ -0,0 +1,36 @@+import { and, eq, gt, inArray, ne, or, sql } from "drizzle-orm";+import { heartbeatRuns, issues, nativeRunFinalizations, nativeRunResults, type Db } from "@paperclipai/db";+import { getNativeReviewAssignment } from "./native-review-participant.js";++/** Both operator and platform recovery must still own the task's accepted work.+ * Call with coordinator -> issue -> run rows locked in a short transaction. */+export async function currentNativeWorkspaceRecoveryResult(db: Db, input: {+ companyId: string;+ issue: typeof issues.$inferSelect;+ run: typeof heartbeatRuns.$inferSelect;+ coordinator: typeof nativeRunFinalizations.$inferSelect;+}) {+ const { companyId, issue, run, coordinator } = input;+ if (issue.companyId !== companyId || run.companyId !== companyId || coordinator.companyId !== companyId+ || run.nativeIssueId !== issue.id || coordinator.issueId !== issue.id || coordinator.runId !== run.id+ || run.runtimeMode !== "native" || run.status !== "running"+ || !["result_accepted", "workspace_finalizing", "retryable_failure"].includes(coordinator.phase)+ || !coordinator.resultId || ["done", "cancelled"].includes(issue.status)+ || issue.executionRunId !== run.id || (issue.checkoutRunId && issue.checkoutRunId !== run.id)) return null;+ if (issue.assigneeAgentId !== run.agentId && !await getNativeReviewAssignment(db, {+ companyId, issueId: issue.id, agentId: run.agentId,+ contextSnapshot: run.contextSnapshot, actingRunId: run.id, issueExecutionRunId: issue.executionRunId,+ allowResolvedByRunId: run.id,+ })) return null;+ const [result] = await db.select().from(nativeRunResults).where(and(+ eq(nativeRunResults.id, coordinator.resultId), eq(nativeRunResults.companyId, companyId),+ eq(nativeRunResults.issueId, issue.id), eq(nativeRunResults.runId, run.id), eq(nativeRunResults.schemaStatus, "accepted"),+ ));+ if (!result || result.completionContractId !== run.completionContractId) return null;+ const [newer] = await db.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(and(+ eq(heartbeatRuns.companyId, companyId), ne(heartbeatRuns.id, run.id),+ or(eq(heartbeatRuns.nativeIssueId, issue.id), sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${issue.id}`),+ or(gt(heartbeatRuns.createdAt, run.createdAt), inArray(heartbeatRuns.status, ["queued", "running"])),+ )).limit(1);+ return newer ? null : result;+}diff --git a/ui/src/api/issues.ts b/ui/src/api/issues.tsindex 314064ea27..f3a56c5ef6 100644--- a/ui/src/api/issues.ts+++ b/ui/src/api/issues.ts@@ -283,6 +283,8 @@ export const issuesApi = {`/issues/${id}/stalled-review-decision`,data,),+ resumeWorkspaceFinalization: (id: string, data: { actionId: string; runId: string; ownerToken: string; controllerAndCopybackStopped: true; stopEvidence: string }) =>+ api.post<{ runId: string; resultId: string; status: "queued" }>(`/issues/${id}/recovery-actions/resume-workspace-finalization`, data),retryWorkspaceExport: (id: string, data: { actionId: string; runId: string; repairNote: string }) =>api.post<{ runId: string; resultId: string; leaseId: string; status: "queued" }>(`/issues/${id}/recovery-actions/retry-workspace-export`, data),resolveRecoveryAction: (diff --git a/ui/src/components/IssueRecoveryActionCard.test.tsx b/ui/src/components/IssueRecoveryActionCard.test.tsxindex 88e6cc7b13..cd51322698 100644--- a/ui/src/components/IssueRecoveryActionCard.test.tsx+++ b/ui/src/components/IssueRecoveryActionCard.test.tsx@@ -1161,3 +1161,10 @@ it("requires export retry for ordinary restoration while keeping explicit boardexpect(document.body.textContent).not.toContain("Send for review");expect(document.body.textContent).toContain("False positive, done");});++it("requires owner-stop recovery even for board false-positive dispositions", () => {+ const node = render(<IssueRecoveryActionCard action={buildAction({ kind: "active_run_watchdog", ownerType: "board",+ cause: "native_workspace_finalization_owner_unverified" })} onResolve={() => {}} canFalsePositive />);+ expect(node.querySelector("[data-testid='recovery-action-resolve-trigger']")).toBeNull();+ expect(node.textContent).not.toContain("False positive");+});diff --git a/ui/src/components/IssueRecoveryActionCard.tsx b/ui/src/components/IssueRecoveryActionCard.tsxindex beb84076e1..e61adbbb96 100644--- a/ui/src/components/IssueRecoveryActionCard.tsx+++ b/ui/src/components/IssueRecoveryActionCard.tsx@@ -1067,6 +1067,7 @@ export function IssueRecoveryActionCard({} satisfies Record<RecoveryCardCardState, string>)[cardState];const visibleResolveOptions = RESOLVE_OPTIONS.filter((option) => {+ if (action.cause === "native_workspace_finalization_owner_unverified") return false;if (isNativeWorkspaceExportRepairCause(action.cause) && ["todo", "done", "in_review"].includes(option.outcome)) return false;if (option.outcome === "todo" && requiresExecutionReconciliation(action.cause)) return false;if (option.boardOnly && !canFalsePositive) return false;diff --git a/ui/src/components/WorkspaceExportRecovery.test.tsx b/ui/src/components/WorkspaceExportRecovery.test.tsxindex 10f4c243ea..28a2409d2f 100644--- a/ui/src/components/WorkspaceExportRecovery.test.tsx+++ b/ui/src/components/WorkspaceExportRecovery.test.tsx@@ -1,72 +1,44 @@// @vitest-environment jsdomimport { act } from "react";import { createRoot, type Root } from "react-dom/client";-import { QueryClient, QueryClientProvider } from "@tanstack/react-query";-import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";+import { afterEach, beforeEach, describe, expect, it } from "vitest";import type { IssueRecoveryAction } from "@paperclipai/shared";import { WorkspaceExportRecovery } from "./WorkspaceExportRecovery";-const retry = vi.hoisted(() => vi.fn());-vi.mock("../api/issues", () => ({ issuesApi: { retryWorkspaceExport: retry } }));-const action = { id: "action", cause: "native_workspace_sync_out_retry_exhausted", status: "active", updatedAt: "2026-01-01T00:00:00Z", ownerType: "board", evidence: { runId: "run" }, wakePolicy: null } as unknown as IssueRecoveryAction;+const action = { cause: "native_workspace_finalization_owner_unverified", status: "escalated", ownerType: "board", evidence: { runId: "run", owner: { token: "owner", hostname: "secret-host", pid: 2 } }, wakePolicy: null } as unknown as IssueRecoveryAction;describe("WorkspaceExportRecovery", () => {- let root: Root, container: HTMLDivElement, client: QueryClient;- const onQueued = vi.fn();- beforeEach(() => {- retry.mockReset().mockResolvedValue({ status: "queued" }); onQueued.mockReset();- container = document.createElement("div"); document.body.append(container); root = createRoot(container);- client = new QueryClient({ defaultOptions: { mutations: { retry: false } } });- });- afterEach(async () => { await act(async () => root.unmount()); client.clear(); container.remove(); });- async function mount(overrides: Partial<Parameters<typeof WorkspaceExportRecovery>[0]> = {}) {- await act(async () => root.render(<QueryClientProvider client={client}><WorkspaceExportRecovery issueId="issue" action={action} canManage onQueued={onQueued} {...overrides} /></QueryClientProvider>));- }- async function enterNote() {- const textarea = container.querySelector("textarea")!;- await act(async () => {- Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, "value")!.set!.call(textarea, "Restored provider connectivity while preserving all saved files.");- textarea.dispatchEvent(new Event("input", { bubbles: true }));- });- }- async function submit() {- await act(async () => container.querySelector("button")!.click());- await act(async () => { await new Promise(resolve => setTimeout(resolve, 10)); });+ let root: Root, container: HTMLDivElement;+ beforeEach(() => { container = document.createElement("div"); document.body.append(container); root = createRoot(container); });+ afterEach(async () => { await act(async () => root.unmount()); container.remove(); });+ async function mount(value: IssueRecoveryAction | null) {+ await act(async () => root.render(<WorkspaceExportRecovery action={value} />));+ expect(container.querySelector("textarea, button, details, input")).toBeNull();+ expect(container.textContent).not.toContain("secret-host");}- it("requires a repair note and submits only the exact recorded run", async () => {- await mount(); expect(container.querySelector("button")!.disabled).toBe(true);- await enterNote(); await submit();- expect(retry).toHaveBeenCalledWith("issue", { actionId: "action", runId: "run", repairNote: "Restored provider connectivity while preserving all saved files." });- expect(onQueued).toHaveBeenCalledOnce();- expect(container.querySelector('[role="status"]')?.textContent).toContain("agent will not repeat its work");- });- it("offers repair again when a queued export publishes a new permanent failure", async () => {- await mount(); await enterNote(); await submit();- expect(container.querySelector("button")).toBeNull();- await mount({ action: { ...action, updatedAt: "2026-01-01T00:01:00Z", wakePolicy: null } });- expect(container.querySelector("button")?.textContent).toBe("Retry workspace export");- });- it("keeps unavailable-sandbox errors actionable inline", async () => {- retry.mockRejectedValueOnce(new Error("Resume the retained sandbox before retrying.")); await mount();- await enterNote(); await submit();- expect(container.querySelector('[role="alert"]')?.textContent).toContain("Resume the retained sandbox");- });- it("does not offer the action without runtime access", async () => {- await mount({ canManage: false }); expect(container.querySelector("button")).toBeNull();- });- it("offers saved-result export after transient retries are exhausted without claiming an unsafe link", async () => {- await mount({ action: { ...action, cause: "native_workspace_sync_out_retry_exhausted" } });- expect(container.querySelector("button")?.textContent).toBe("Retry workspace export");- expect(container.textContent).toContain("export failure");- expect(container.textContent).not.toContain("unsafe link");- await enterNote(); await submit();- expect(retry).toHaveBeenCalledWith("issue", expect.objectContaining({ runId: "run" }));- });- it("never asks users to repair historical unsafe exports", async () => {- await mount({ action: { ...action, cause: "native_workspace_sync_out_unsafe_archive" } });- expect(container.textContent).toBe("");- expect(retry).not.toHaveBeenCalled();- });- it("does not offer a second retry while the same export is queued", async () => {- await mount({ action: { ...action, wakePolicy: { kind: "resume_native_run" } } });- expect(container.querySelector('[role="status"]')?.textContent).toContain("Export is queued"); expect(container.querySelector("button")).toBeNull();+ it("shows a quiet status while the harness restores work", async () => {+ await mount({ ...action, evidence: { ...action.evidence, workspaceOwnerRecovery: { state: "resuming" } } });+ expect(container.querySelector('[role="status"]')?.textContent).toBe("Restoring saved work…");+ });+ it("does not mistake platform identity for confirmed recovery", async () => {+ await mount({ ...action, evidence: { ...action.evidence, owner: { ...(action.evidence.owner as object), worker: { provider: "railway", deploymentId: "old" } } } });+ expect(container.textContent).toContain("operator assistance");+ });+ it("states that an operator is needed for legacy receipts", async () => {+ await mount(action);+ expect(container.textContent).toContain("Workspace recovery needs operator assistance");+ });+ it("keeps export repair out of the task", async () => {+ const cause = "native_workspace_sync_out_retry_exhausted";+ await mount({ ...action, cause });+ expect(container.textContent).toContain("operator assistance");+ });+ it("removes the status when recovery resolves, including updates in place", async () => {+ await mount({ ...action, evidence: { ...action.evidence, workspaceOwnerStop: {} } });+ expect(container.textContent).toBe("Restoring saved work…");+ await mount({ ...action, status: "resolved" }); expect(container.textContent).toBe("");+ await mount(null); expect(container.textContent).toBe("");+ });+ it("does not show unrelated or historical unsafe-export notices", async () => {+ await mount({ ...action, cause: "no_live_path", evidence: {} }); expect(container.textContent).toBe("");+ await mount({ ...action, cause: "native_workspace_sync_out_unsafe_archive" }); expect(container.textContent).toBe("");});});diff --git a/ui/src/components/WorkspaceExportRecovery.tsx b/ui/src/components/WorkspaceExportRecovery.tsxindex 9837681681..a5b3dcb45e 100644--- a/ui/src/components/WorkspaceExportRecovery.tsx+++ b/ui/src/components/WorkspaceExportRecovery.tsx@@ -1,45 +1,17 @@-import { useId, useState } from "react";-import { useMutation } from "@tanstack/react-query";import type { IssueRecoveryAction } from "@paperclipai/shared";import { isNativeWorkspaceExportRepairCause } from "@paperclipai/shared";-import { issuesApi } from "../api/issues";-import { cn } from "../lib/utils";-import { Button } from "./ui/button";-import { Label } from "./ui/label";-import { Textarea } from "./ui/textarea";+import { asObject, cn } from "../lib/utils";-/** Copyback repair preserves the accepted result and never wakes the provider. */-export function WorkspaceExportRecovery({ issueId, action, canManage, onQueued, className }: {- className?: string;- issueId: string; action: IssueRecoveryAction | null; canManage: boolean; onQueued: () => void;+/** Infrastructure repair belongs to the harness/operator, never a task form. */+export function WorkspaceExportRecovery({ action, className }: {+ className?: string; action: IssueRecoveryAction | null;}) {- const noteId = useId();- const [repairNote, setRepairNote] = useState("");- const [queuedActionVersion, setQueuedActionVersion] = useState<string | null>(null);- const retry = useMutation({- mutationFn: () => issuesApi.retryWorkspaceExport(issueId, {- actionId: action!.id, runId: action!.evidence.runId as string, repairNote: repairNote.trim(),- }),- onSuccess: () => { setQueuedActionVersion(String(action!.updatedAt)); onQueued(); },- });- if (!action || !isNativeWorkspaceExportRepairCause(action.cause) || action.ownerType !== "board"+ const ownerRecovery = action?.cause === "native_workspace_finalization_owner_unverified";+ if (!action || (!ownerRecovery && !isNativeWorkspaceExportRepairCause(action.cause)) || action.ownerType !== "board"|| !["active", "escalated"].includes(action.status) || typeof action.evidence.runId !== "string") return null;- const queued = queuedActionVersion === String(action.updatedAt) || action.wakePolicy?.kind === "resume_native_run";- return <section aria-label="Workspace export repair" className={cn("task-context-notice flex flex-col gap-2", className)}>- <p className="font-medium">Workspace export needs repair</p>- {queued ? <p role="status">Export is queued for the saved result. The agent will not repeat its work.</p> : <>- <p className="text-muted-foreground">{"Automatic workspace export retries stopped. Inspect the export failure, restore provider or destination availability, and preserve the saved files in the retained sandbox. Retry export here when the cause is resolved."}</p>- {canManage ? <>- <Label htmlFor={noteId}>Repair performed</Label>- <Textarea id={noteId} value={repairNote} onChange={event => setRepairNote(event.target.value)} maxLength={12_000}- placeholder="Describe the repair and how the saved workspace files were preserved." disabled={retry.isPending} />- <div className="flex justify-end">- <Button onClick={() => retry.mutate()} disabled={retry.isPending || repairNote.trim().length < 20}>- {retry.isPending ? "Queueing export…" : "Retry workspace export"}- </Button>- </div>- </> : <p className="text-muted-foreground">A board member with runtime access can retry this export after repair.</p>}- {retry.isError && <p role="alert" className="text-destructive">{retry.error instanceof Error ? retry.error.message : "Could not queue export. Refresh the task and inspect its run."}</p>}- </>}- </section>;+ const queued = !!action.evidence.workspaceOwnerStop || action.wakePolicy?.kind === "resume_native_run";+ const automatic = queued || (ownerRecovery && asObject(action.evidence.workspaceOwnerRecovery).state === "resuming");+ return <p role="status" className={cn("text-sm text-muted-foreground", className)}>+ {automatic ? "Restoring saved work…" : "Workspace recovery needs operator assistance. The agent’s result is saved."}+ </p>;}diff --git a/ui/src/pages/IssueDetail.tsx b/ui/src/pages/IssueDetail.tsxindex af019ce5e2..4d68f7840e 100644--- a/ui/src/pages/IssueDetail.tsx+++ b/ui/src/pages/IssueDetail.tsx@@ -2497,7 +2497,7 @@ const IssueDetailChatTab = memo(function IssueDetailChatTab({blockerAttention={blockerAttention}successfulRunHandoff={successfulRunHandoff}scheduledRetry={scheduledRetry}- recoveryAction={recoveryAction ?? null}+ recoveryAction={recoveryAction?.cause === "native_workspace_finalization_owner_unverified" ? null : recoveryAction ?? null}onResolveRecoveryAction={onResolveRecoveryAction}onReissueIsolatedRecoveryAction={onReissueIsolatedRecoveryAction}reissueIsolatedRecoveryActionPending={@@ -7659,9 +7659,9 @@ export function TaskDetailSurface({ conversation, tasksTab }: { tasksTab?: TaskS/>{resolvedDetailTab === "chat" && (- <WorkspaceExportRecovery key={`workspace-export:${issue.activeRecoveryAction?.id ?? issue.id}`} issueId={issue.id}+ <WorkspaceExportRecovery key={`workspace-export:${issue.activeRecoveryAction?.id ?? issue.id}`}className={shellSectionClass}- action={issue.activeRecoveryAction ?? null} canManage={canManageBoardRuntime} onQueued={invalidateIssueDetail} />+ action={issue.activeRecoveryAction ?? null} />)}{resolvedDetailTab === "chat" && issue.executionBlocker && (<ExecutionBlockerNotice className={shellSectionClass} companyId={issue.companyId} issueId={issue.id} blocker={issue.executionBlocker} onRetried={invalidateIssueDetail} />@@ -8046,7 +8046,7 @@ export function TaskDetailSurface({ conversation, tasksTab }: { tasksTab?: TaskSblockerAttention={issue.blockerAttention ?? null}successfulRunHandoff={issue.successfulRunHandoff ?? null}scheduledRetry={issue.scheduledRetry ?? null}- recoveryAction={issue.activeRecoveryAction ?? null}+ recoveryAction={issue.activeRecoveryAction?.cause === "native_workspace_finalization_owner_unverified" ? null : issue.activeRecoveryAction ?? null}onResolveRecoveryAction={handleResolveRecoveryAction}onReissueIsolatedRecoveryAction={handleReissueIsolatedRecoveryActiondiff --git a/ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx b/ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsxnew file mode 100644index 0000000000..eafd2bfd05--- /dev/null+++ b/ui/storybook/stories/recovery/workspace-finalization-recovery/01-notice.stories.tsx@@ -0,0 +1,29 @@+import type { Meta, StoryObj } from "@storybook/react-vite";+import { expect, within } from "storybook/test";+import { WorkspaceExportRecovery } from "@/components/WorkspaceExportRecovery";+import { automaticAction, queuedAction, recoveryAction } from "../../../support/workspace-finalization-recovery/fixtures";+const meta = {+ id: "workspace-finalization-recovery-01-notice",+ title: "Projects and execution/Recovery/Workspace finalization recovery/01 Notice",+ component: WorkspaceExportRecovery,+ args: { action: automaticAction },+ parameters: { layout: "fullscreen", docs: { story: { inline: false }, description: { component: "Controller recovery runs in the harness. The task shows a quiet status and removes it when saving finishes. Legacy receipts without platform identity require operator assistance through the documented API; no shutdown form is shown to users." } } },+ decorators: [Story => <div className="mx-auto max-w-2xl p-4 sm:p-8"><Story /></div>],+} satisfies Meta<typeof WorkspaceExportRecovery>;+export default meta;+type Story = StoryObj<typeof meta>;+export const RestoringSavedWork: Story = { play: async ({ canvasElement }) => {+ const canvas = within(canvasElement);+ await expect(canvas.getByRole("status")).toHaveTextContent("Restoring saved work");+ await expect(canvas.queryByRole("button")).not.toBeInTheDocument();+ await expect(canvas.queryByRole("textbox")).not.toBeInTheDocument();+} };+export const NeedsVerification: Story = { args: { action: recoveryAction } };+export const VerificationUnavailable: Story = { args: { action: {+ ...automaticAction, evidence: { ...automaticAction.evidence, workspaceOwnerRecovery: { state: "operator_required" } },+} }, play: async ({ canvasElement }) => {+ await expect(within(canvasElement).getByRole("status")).toHaveTextContent("operator assistance");+} };+export const AlreadyQueued: Story = { args: { action: queuedAction } };+export const Saved: Story = { args: { action: null } };+export const Mobile: Story = { globals: { viewport: { value: "mobile", isRotated: false } } };diff --git a/ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx b/ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsxnew file mode 100644index 0000000000..98a012faeb--- /dev/null+++ b/ui/storybook/stories/recovery/workspace-finalization-recovery/02-task-journey.stories.tsx@@ -0,0 +1,43 @@+import { useEffect, useState } from "react";+import type { Meta, StoryObj } from "@storybook/react-vite";+import { Route, Routes } from "@/lib/router";+import { Layout } from "@/components/Layout";+import { TaskChatBubble } from "@/components/task-chat/TaskChatBubble";+import { TaskChatComposer } from "@/components/task-chat/TaskChatComposer";+import { WorkspaceExportRecovery } from "@/components/WorkspaceExportRecovery";+import { PluginLauncherProvider } from "@/plugins/launchers";+import { useBreadcrumbs } from "@/context/BreadcrumbContext";+import { automaticAction } from "../../../support/workspace-finalization-recovery/fixtures";++function TaskConversation({ saved = false }: { saved?: boolean }) {+ const [messages, setMessages] = useState<string[]>([]);+ const { setBreadcrumbs } = useBreadcrumbs();+ useEffect(() => { setBreadcrumbs([{ label: "Tasks", href: "/PAP/issues" }, { label: "Prepare the rollout plan", identifier: "PAP-24" }]); }, [setBreadcrumbs]);+ return <main className="mx-auto flex min-h-full w-full max-w-3xl flex-col gap-6 px-4 py-6 sm:px-8">+ <header className="flex flex-col gap-2">+ <span className="font-mono text-xs text-muted-foreground">PAP-24</span>+ <h1 className="text-xl font-semibold">Prepare the rollout plan</h1>+ <p className="text-sm text-muted-foreground">Assigned to Jordan Lee</p>+ </header>+ <TaskChatBubble item={{ id: "request", kind: "message", author: "human", authorName: "Riley Board", authorUserId: "user-board", isCurrentUser: true, text: "Prepare the rollout plan and wait for my confirmation before starting implementation." }} />+ <TaskChatBubble item={{ id: "saved-result", kind: "message", author: "agent", authorName: "Jordan Lee", text: "The plan is ready. I’m waiting for your confirmation before continuing." }} />+ <WorkspaceExportRecovery action={saved ? null : automaticAction} />+ {messages.map((text, index) => <TaskChatBubble key={index} item={{ id: `followup-${index}`, kind: "message", author: "human", authorName: "Riley Board", authorUserId: "user-board", isCurrentUser: true, text }} />)}+ <div className="flex-1" />+ <TaskChatComposer workMode="standard" onAdd={async body => setMessages(current => [...current, body])} placeholder="Message Jordan Lee…" />+ </main>;+}++function Journey({ saved = false }: { saved?: boolean }) {+ return <PluginLauncherProvider><Routes><Route path="/:companyPrefix" element={<Layout />}><Route path="issues/:issueId" element={<TaskConversation saved={saved} />} /></Route></Routes></PluginLauncherProvider>;+}+const meta = {+ id: "workspace-finalization-recovery-02-task-journey",+ title: "Projects and execution/Recovery/Workspace finalization recovery/02 Task journey", component: Journey, args: { saved: false },+ parameters: { layout: "fullscreen", initialEntries: ["/PAP/issues/PAP-24"], docs: { description: { component: "A composed task conversation in the production app shell, using the shipped recovery notice, task bubbles and composer. Recovery happens automatically with no form or user action. This is a composed UI preview; finalization and approval resolution are verified separately by server tests." }, story: { inline: false } } },+} satisfies Meta<typeof Journey>;+export default meta;+type Story = StoryObj<typeof meta>;+export const ResumeInContext: Story = {};+export const SavedAndWaitingForConfirmation: Story = { args: { saved: true } };+export const Mobile: Story = { globals: { viewport: { value: "mobile", isRotated: false } } };diff --git a/ui/storybook/support/workspace-finalization-recovery/fixtures.tsx b/ui/storybook/support/workspace-finalization-recovery/fixtures.tsxnew file mode 100644index 0000000000..084a4335a4--- /dev/null+++ b/ui/storybook/support/workspace-finalization-recovery/fixtures.tsx@@ -0,0 +1,37 @@+import type { IssueRecoveryAction } from "@paperclipai/shared";++export const issueId = "00000000-0000-4000-8000-000000000824";+export const stopEvidence = "The deployment platform confirms that previous-controller-7d4f and all its workspace-copyback processes exited. The saved workspace is retained.";++export const recoveryAction: IssueRecoveryAction = {+ id: "00000000-0000-4000-8000-000000000825",+ companyId: "company-storybook", sourceIssueId: issueId, recoveryIssueId: null,+ kind: "active_run_watchdog", status: "escalated", ownerType: "board",+ ownerAgentId: null, ownerUserId: null, previousOwnerAgentId: null, returnOwnerAgentId: null,+ cause: "native_workspace_finalization_owner_unverified", fingerprint: "storybook-saved-result",+ evidence: { runId: "00000000-0000-4000-8000-000000000826", owner: {+ token: "00000000-0000-4000-8000-000000000827", hostname: "previous-controller-7d4f",+ pid: 2, processStartedAt: "2026-10-10T00:02:55.560Z",+ controllerBootId: "00000000-0000-4000-8000-000000000828",+ } },+ nextAction: "Verify the previous controller and its workspace-copyback processes have stopped. Resume workspace finalization without another provider turn.",+ wakePolicy: null, monitorPolicy: null, attemptCount: 1, maxAttempts: 1,+ timeoutAt: null, lastAttemptAt: "2026-10-10T00:10:00.000Z",+ outcome: null, resolutionNote: null, resolvedAt: null,+ createdAt: "2026-10-10T00:10:00.000Z", updatedAt: "2026-10-10T00:10:00.000Z",+};++export const queuedAction: IssueRecoveryAction = {+ ...recoveryAction, evidence: { ...recoveryAction.evidence, workspaceOwnerStop: {+ owner: recoveryAction.evidence.owner, stopEvidence,+ resultId: "00000000-0000-4000-8000-000000000829", actorId: "user-board",+ confirmedAt: "2026-10-10T00:15:00.000Z",+ } },+ nextAction: "Workspace finalization is queued for the saved result. The agent will not repeat its work.",+};++export const automaticAction: IssueRecoveryAction = {+ ...recoveryAction, evidence: { ...recoveryAction.evidence, workspaceOwnerRecovery: { state: "resuming" }, owner: { ...(recoveryAction.evidence.owner as object), worker: {+ provider: "railway", projectId: "fixture-project", environmentId: "fixture-environment", serviceId: "fixture-service", deploymentId: "fixture-old-deployment",+ } } },+};