Skip to content

The open-source app everyone uses to manage agents at work

docs(runner): define architecture and compatibility (#12084)

Dotta41bf5cafa11e
diff --git a/doc/architecture/paperclip-runner-compatibility.md b/doc/architecture/paperclip-runner-compatibility.md
new file mode 100644
index 0000000000..8630e45ca9
--- /dev/null
+++ b/doc/architecture/paperclip-runner-compatibility.md
@@ -0,0 +1,206 @@
+# Paperclip Runner compatibility and rollout
+
+- Status: Proposed
+- Date: 2026-08-24
+- Parent decision: [Paperclip Runner architecture](paperclip-runner.md)
+
+## Purpose
+
+This document defines compatibility rules for introducing the experimental
+Paperclip Runner. These rules are acceptance criteria for each implementation
+change. They are not a migration plan for existing adapters.
+
+## Compatibility invariants
+
+1. Existing adapter selection is authoritative. A direct adapter stays direct.
+2. A non-runner run must not start runnerd or open PRP.
+3. A non-runner run must not create native result, finalization, status-decision,
+ or runner-transport records.
+4. A non-runner run must not invoke native status arbitration.
+5. Direct adapters keep their current transcript, interaction, cancellation,
+ result, and finalization behavior.
+6. Runner-only UI controls depend on persisted runtime facts and are absent from
+ direct runs.
+7. The rollout flag controls fresh runner selection and fresh runner starts.
+8. A flag change does not rewrite an agent profile or a persisted run choice.
+9. Persisted native data remains readable after the flag is disabled.
+10. Recovery may finish an already persisted native run while fresh native
+ starts remain blocked.
+
+## Runtime selection
+
+The server resolves and persists the runtime once, before provider launch.
+
+| Persisted runtime | Adapter | Flag | Result |
+| --- | --- | --- | --- |
+| none | Any direct adapter | off or on | Use the existing direct path. |
+| none | `paperclip_runner` with Codex | off | Reject the fresh start with a stable rollout-disabled error. |
+| none | `paperclip_runner` with Codex | on | Use PRP v1 and runnerd. |
+| none | `paperclip_runner` with another provider | on | Reject the unsupported provider before runnerd starts. |
+| direct | Any | changed later | Keep the persisted direct path. |
+| native | Any | changed later | Keep the persisted native path for read, cancel, recovery, and finalization. |
+
+The server must not fall back from a selected `paperclip_runner` start to
+`codex_local`. A configuration or rollout error must be visible. Silent fallback
+would hide the runtime that executed the task.
+
+## Direct adapter boundary
+
+This rule applies to every built-in and plugin direct adapter. It includes:
+
+- `codex_local`;
+- `claude_local`;
+- `opencode_local`;
+- other local CLI or session adapters;
+- process and HTTP adapters;
+- gateway adapters; and
+- external adapter plugins.
+
+Adding Paperclip Runner must not add runner imports or runner branches inside a
+direct adapter implementation. The heartbeat coordinator may select the
+explicit runner adapter at one narrow seam. All other adapters continue through
+their existing code.
+
+For a flag-off `codex_local` heartbeat, compatibility proof must show:
+
+- one direct invocation;
+- the same normalized result and finalization bytes as the approved baseline;
+- zero runner processes;
+- zero PRP connections; and
+- zero native rows.
+
+## Configuration behavior
+
+When the rollout flag is off:
+
+- creation UI does not offer `paperclip_runner`;
+- edit UI does not offer switching to `paperclip_runner`;
+- server creation and import reject a new `paperclip_runner` selection;
+- the server rejects a fresh start for an existing runner-configured agent;
+- read and export preserve an existing runner configuration;
+- unrelated edits to an existing runner-configured agent do not erase its
+ configuration; and
+- switching that agent to a direct adapter remains allowed.
+
+When the rollout flag is on:
+
+- creation, import, and edit accept `paperclip_runner` only with provider
+ `codex` and valid Codex configuration;
+- switching from a direct adapter affects only future unresolved runs; and
+- switching away from the runner affects only future unresolved runs.
+
+Server validation is the authority in both states. Import files and API clients
+cannot bypass the flag or provider allowlist.
+
+## Persisted native runs
+
+The following data remains readable independent of the current flag:
+
+- the persisted runtime selection and reason;
+- run, runner, session, turn, and provider identity;
+- ordered runner and provider events;
+- accepted result and evidence assessment;
+- finalization coordinator and status decision;
+- usage, cost, cancellation, and terminal details; and
+- durable interactions and final task-thread reply.
+
+If the flag is disabled during an in-flight native run, the server may reconnect,
+cancel, reconcile, and finalize that same run. It must not use that recovery as
+authority to start a new native run.
+
+Recovery must remain idempotent. Repeating it cannot add a second final reply,
+interaction, wake, status decision, or application effect.
+
+## Task-page compatibility
+
+The task page uses one provider-neutral thread projection. Runtime facts may add
+runner event groups, semantic receipts, usage, and structured questions. They
+must not replace classic direct-adapter content.
+
+Direct-adapter coverage must include:
+
+- an active run;
+- a settled run;
+- an empty transcript;
+- a pending interaction;
+- a resolved interaction; and
+- the classic interface state.
+
+The existing composer remains usable for direct adapters. A direct run does not
+show reconnect, runner cancel, semantic receipt, or other runner-only controls.
+Final replies continue to use the existing issue-comment behavior.
+
+## Structured input compatibility
+
+New structured questions use `paperclip.question_set.v1` and the matching
+response contract. Provider-specific question objects remain inside their
+drivers.
+
+Legacy unstructured interaction records remain readable and resolvable. A
+structured form fails closed when its required schema, question mode, question
+ID, option ID, or response value is invalid. The implementation must not silently
+convert malformed structured input to a legacy text prompt.
+
+## Protocol version compatibility
+
+PRP wire versions, fixture versions, event schema versions, and typed schema
+discriminators are independently versioned.
+
+- Peers negotiate the highest common PRP version.
+- No common required version fails closed before command or provider execution.
+- An unknown required fixture or event schema version fails closed.
+- An unknown required typed discriminator fails closed.
+- Additive optional properties remain compatible only when an old v1 consumer
+ can ignore them without changing behavior.
+- Unknown object properties must survive validation when the owning schema
+ permits additive fields.
+
+Breaking meaning requires a new required version. A provider error is not a
+protocol negotiation result.
+
+## Semantic action compatibility
+
+Catalog generation and production authorization are separate steps.
+
+- A catalog entry does not authorize production use.
+- An operation without a production binding is undiscoverable.
+- An operation denied to the run is undiscoverable unless the protocol
+ explicitly defines a safe denied receipt for that discovery mode.
+- A duplicate call with the same idempotency key and canonical input returns the
+ original safe receipt.
+- Reusing the key with different input returns a conflict and performs no second
+ effect.
+- Receipts redact credentials, private provider payloads, and hidden identity.
+
+## Required compatibility matrix
+
+Each runner-related pull request updates only rows that it can execute. The
+complete first-wave matrix must cover:
+
+| Area | Required cases |
+| --- | --- |
+| Runtime selection | Every built-in direct adapter, explicit runner selection, unsupported provider, flag on, and flag off. |
+| Direct regression | Flag-off `codex_local` invocation count, byte-stable result/finalization, and zero native rows. |
+| Configuration | Enabled and disabled create, import, edit, read, export, and adapter switch. |
+| Recovery | Persisted native run after flag disable, reconnect, duplicate event, duplicate command, cancellation, and server restart. |
+| Protocol security | Cross-company binding, ticket replay, ticket expiry, malformed frame, unsupported version, and revoked lease. |
+| Semantic actions | Discovery denial, unbound action, duplicate call, conflicting retry, redaction, and governed action. |
+| Task page | Active, settled, empty transcript, interaction, and classic direct-adapter states. |
+| Structured input | Valid response, malformed response, stale response, duplicate response, provider loss, and cancellation. |
+
+## Pull request acceptance
+
+Every implementation pull request must:
+
+- build and work against its declared base;
+- keep its changed-file count below 100;
+- update manifests and exports only for implemented surfaces;
+- run the smallest relevant tests before the repository handoff gate;
+- run typecheck, tests, and build for handoff;
+- run protocol parity, Cargo, clean-consumer, migration, token, and browser gates
+ when those areas change;
+- contain no unexplained failed, cancelled, or path-skipped verification; and
+- resolve actionable review and security findings before it is ready.
+
+Generated files land with their source and a drift check. `pnpm-lock.yaml` is
+owned by CI and is not part of these pull requests.
diff --git a/doc/architecture/paperclip-runner.md b/doc/architecture/paperclip-runner.md
new file mode 100644
index 0000000000..85b41813ba
--- /dev/null
+++ b/doc/architecture/paperclip-runner.md
@@ -0,0 +1,286 @@
+# ADR: Paperclip Runner architecture
+
+- Status: Proposed
+- Date: 2026-08-24
+- Owners: Paperclip control plane and runner maintainers
+- Related: [Paperclip Runner compatibility and rollout](paperclip-runner-compatibility.md)
+
+## Context
+
+Paperclip is a control plane. It owns companies, agents, issues, budgets,
+approvals, and durable workflow state. Agent providers remain execution
+services. Existing adapters invoke those services directly from the Paperclip
+server.
+
+Paperclip Runner introduces a separate execution process for provider sessions.
+This process needs durable delivery, restart recovery, and governed access to
+Paperclip actions. It must not become a second control plane. It must also land
+without changing the behavior of existing adapters.
+
+The initial implementation is intentionally narrow. It supports Codex through
+an explicit, experimental adapter. Other providers and developer tools remain
+outside this decision.
+
+## Decision
+
+Add a standalone package named `@paperclipai/paperclip-runner`. The package owns
+the language-neutral Paperclip Runner Protocol (PRP), the Rust runner process,
+provider drivers, deterministic replay, and semantic action dispatch contracts.
+
+Add one explicit adapter named `paperclip_runner`. The adapter is available only
+when an instance-level, default-off rollout flag is enabled. Its first supported
+provider is Codex.
+
+Do not route existing adapters through Paperclip Runner. A direct adapter keeps
+its current invocation, transcript, interaction, cancellation, and finalization
+paths.
+
+## Goals
+
+- Keep runner process ownership outside the Paperclip server process.
+- Preserve Paperclip as the authority for identity, policy, and workflow state.
+- Recover a run after runner or network interruption without duplicate effects.
+- Expose only actions that the current run is allowed to use.
+- Make protocol behavior deterministic across TypeScript and Rust.
+- Keep existing adapter behavior unchanged while the runner is experimental.
+
+## Non-goals
+
+- Replace existing direct adapters.
+- Move business authorization or issue status policy into Rust.
+- Give runnerd a broad Paperclip API credential.
+- Support OpenCode, ACPX, Claude Managed, AWS AgentCore, or remote sandboxes in
+ the first production slice.
+- Expose browser SDK, React SDK, eval, lab, or scenario-explorer package entry
+ points in the initial release.
+- Commit recorded screenshots, stress logs, or construction history as product
+ architecture.
+
+## Topology
+
+The initial local topology is:
+
+```text
+Paperclip server
+ | authenticated PRP v1 WebSocket
+ v
+paperclip-runnerd
+ | Codex app-server protocol
+ v
+Codex
+```
+
+The server opens a native run and launches a verified runnerd artifact in the
+realized execution environment. Runnerd opens the outbound PRP connection. It
+then owns the provider process group and the durable transport state for that
+run.
+
+The browser does not connect to runnerd. It reads projections from the existing
+Paperclip APIs and task-thread models.
+
+## Dependency direction
+
+The runner package must build and test without importing Paperclip server, UI,
+CLI, database, or other private workspace implementation modules.
+
+```text
+JSON Schema and fixtures
+ |
+ +------------------+
+ v v
+ TypeScript contracts Rust runner core
+ | |
+ +--------+---------+
+ v
+ deterministic parity
+
+Paperclip server ----implements----> runner public ports
+```
+
+The dependency points from an implementation to a contract. The Paperclip
+server may implement a public runner port. The runner package must not import
+the server implementation.
+
+The initial public package surfaces are:
+
+- `@paperclipai/paperclip-runner` for runtime contracts and clients.
+- `@paperclipai/paperclip-runner/testing` for deterministic fakes and
+ conformance helpers.
+
+Every export must have an implementation and a clean-consumer test before it is
+published. Later SDK, eval, and lab surfaces require separate decisions.
+
+## Protocol boundary
+
+PRP v1 uses a WebSocket at:
+
+```text
+/api/runner/v1/connect/:runId
+```
+
+JSON Schema is the language-neutral protocol authority. TypeScript and Rust use
+the same canonical fixtures and must produce the same replay result.
+
+Protocol rules are fail closed:
+
+- Unknown required protocol or schema versions are rejected.
+- Unknown required discriminators and enum values are rejected.
+- Additive optional object fields may be accepted when v1 consumers can ignore
+ them safely.
+- Frames, headers, durable state, diagnostic tails, and replay windows are
+ bounded.
+- Event identity and ordering remain stable across reconnect and restart.
+
+Provider-native messages do not cross this boundary. Drivers translate them to
+provider-neutral PRP events, results, usage, cancellations, and structured
+input.
+
+## Trust boundary
+
+The Paperclip server is authoritative for:
+
+- company, agent, issue, run, session, and user attribution;
+- rollout and runtime selection;
+- action discovery and authorization;
+- approval, budget, secret, revision, and workspace policy;
+- durable application records and activity history;
+- result acceptance, finalization, and issue status.
+
+Runnerd is authoritative only for its local responsibilities:
+
+- provider process supervision;
+- provider session and turn transport;
+- durable PRP outbox and command receipts;
+- stable runner-side event identity;
+- bounded process diagnostics.
+
+Runnerd receives a short-lived, one-use bootstrap ticket. The ticket is bound to
+the company, agent, issue, run, runner, session, turn, and verified artifact. The
+server exchanges it for a short-lived connection lease. Raw tickets are never
+stored. Runnerd never receives a broad Paperclip API key.
+
+The server rejects expired, replayed, revoked, cross-company, mismatched,
+malformed, oversized, or protocol-incompatible connections. Cancellation,
+timeout, supersession, and environment-lease loss revoke runner authority.
+
+## Durable delivery and recovery
+
+Runnerd persists an event before it sends the event. The server acknowledges a
+cumulative source cursor. Runnerd removes acknowledged data only after it
+validates that cursor.
+
+A reconnect reports the last processed command, the next source sequence, the
+last acknowledged source sequence, and the unacknowledged range. The server may
+then replay one pending command and accept byte-equivalent event retries.
+
+Command identity is idempotent. Reusing a command ID with the same canonical
+input returns the stored result. Reusing the ID with different input fails
+closed. A repeated semantic tool call cannot repeat an application effect.
+
+Recovery is bounded. Exhausted storage, reconnect, command, or time limits end
+the run with a classified failure instead of an unbounded loop.
+
+## Semantic actions
+
+PRP carries provider-neutral semantic operation IDs. The package may define an
+operation catalog, but catalog presence does not grant authority.
+
+For each run, the server projects only operations that have a production
+binding and that the current actor may discover. An unbound or unauthorized
+operation is absent from discovery. The server validates inputs, executes the
+existing application authority, and returns a redacted receipt.
+
+This keeps these invariants in one place:
+
+- company scoping;
+- actor and run attribution;
+- authorization and approval;
+- revision and idempotency checks;
+- budget and secret policy;
+- activity logging and safe error details.
+
+Runnerd cannot forge identity through tool input. Identity and scope come from
+the authenticated connection binding.
+
+## Structured input
+
+Questions use `paperclip.question_set.v1`. Responses use the matching canonical
+response contract. Drivers translate between these provider-neutral records and
+provider-native input APIs.
+
+The server validates a response at the untrusted API edge and again against the
+persisted question set before delivery. A process loss may materialize a durable
+task-thread interaction, but cancellation and an already resolved request must
+not create a second interaction or continuation.
+
+## Persistence and finalization
+
+Native run records are additive. Existing heartbeat and issue records remain
+readable. A runner result is an untrusted claim. It does not directly update an
+issue status.
+
+The server validates the result, classifies durable evidence, applies status
+policy, commits workflow effects, and records finalization. Each phase is
+idempotent and recoverable. Existing direct adapters keep their existing
+finalization paths and do not invoke native status arbitration.
+
+Disabling the rollout flag blocks fresh runner starts. It does not make an
+already persisted native run unreadable or prevent bounded recovery and
+finalization of that run.
+
+## Rollout
+
+The rollout has three gates:
+
+1. The instance flag is enabled.
+2. The agent explicitly selects `paperclip_runner`.
+3. The adapter selects a supported provider. The initial provider is `codex`.
+
+The adapter is hidden from creation and selection surfaces while the flag is
+off. Server validation also rejects a fresh runner selection or start while the
+flag is off. UI hiding is not the security boundary.
+
+Runtime selection is persisted before launch. Later setting changes cannot
+silently move an in-flight run between the direct and native execution paths.
+
+The detailed compatibility rules are in
+[Paperclip Runner compatibility and rollout](paperclip-runner-compatibility.md).
+
+## Observability
+
+The server records structured runner, provider, semantic action, interaction,
+usage, and terminal events. Logs and receipts must not contain bootstrap
+tickets, leases, provider credentials, secret values, complete environments, or
+private host paths.
+
+The task page may project these records through existing thread components. A
+runner-only control must depend on persisted runtime facts. It must not depend
+only on the agent adapter profile.
+
+## Consequences
+
+This design adds process, protocol, and recovery complexity. In return, it gives
+provider sessions a durable and testable execution boundary without moving
+Paperclip governance into the runner.
+
+The default-off, explicit-adapter rollout duplicates some provider
+configuration during the experiment. This is intentional. It keeps comparison
+and rollback simple and prevents a global migration of existing agents.
+
+The standalone package boundary requires generated artifacts, clean-consumer
+tests, and cross-language parity gates. These checks add build cost, but they
+prevent server implementation details from becoming accidental public API.
+
+## Required proof before general availability
+
+- TypeScript and Rust accept and reject the same protocol fixtures.
+- Lost acknowledgements, reconnects, runner restarts, and duplicate commands do
+ not duplicate events or effects.
+- Bootstrap replay, binding mismatch, and cross-company access fail closed.
+- Only authorized and production-bound actions appear in discovery.
+- Cancellation settles provider, runner, transport, and application state.
+- Flag-off direct-adapter runs create no native records and start no runner
+ process.
+- Persisted native runs remain readable and recoverable after the flag changes.
+- Codex completes the server to PRP to runnerd to provider to server path.
+- Existing adapter compatibility tests remain byte stable where specified.