The open-source app everyone uses to manage agents at work
TypeScript acpxagent-orchestrationagent-skillsagentic-aiai-agentsai-engineeringai-governanceai-toolsautonomous-agentsclaudeclaude-codeclaude-skillscodexhermesllmmcpmulti-agentopenclawproject-managementworkflow-automation
docs(runner): define architecture and compatibility (#12084)
Dotta41bf5cafa11e
diff --git a/doc/architecture/paperclip-runner-compatibility.md b/doc/architecture/paperclip-runner-compatibility.mdnew file mode 100644index 0000000000..8630e45ca9--- /dev/null+++ b/doc/architecture/paperclip-runner-compatibility.md@@ -0,0 +1,206 @@+# Paperclip Runner compatibility and rollout++- Status: Proposed+- Date: 2026-08-24+- Parent decision: [Paperclip Runner architecture](paperclip-runner.md)++## Purpose++This document defines compatibility rules for introducing the experimental+Paperclip Runner. These rules are acceptance criteria for each implementation+change. They are not a migration plan for existing adapters.++## Compatibility invariants++1. Existing adapter selection is authoritative. A direct adapter stays direct.+2. A non-runner run must not start runnerd or open PRP.+3. A non-runner run must not create native result, finalization, status-decision,+ or runner-transport records.+4. A non-runner run must not invoke native status arbitration.+5. Direct adapters keep their current transcript, interaction, cancellation,+ result, and finalization behavior.+6. Runner-only UI controls depend on persisted runtime facts and are absent from+ direct runs.+7. The rollout flag controls fresh runner selection and fresh runner starts.+8. A flag change does not rewrite an agent profile or a persisted run choice.+9. Persisted native data remains readable after the flag is disabled.+10. Recovery may finish an already persisted native run while fresh native+ starts remain blocked.++## Runtime selection++The server resolves and persists the runtime once, before provider launch.++| Persisted runtime | Adapter | Flag | Result |+| --- | --- | --- | --- |+| none | Any direct adapter | off or on | Use the existing direct path. |+| none | `paperclip_runner` with Codex | off | Reject the fresh start with a stable rollout-disabled error. |+| none | `paperclip_runner` with Codex | on | Use PRP v1 and runnerd. |+| none | `paperclip_runner` with another provider | on | Reject the unsupported provider before runnerd starts. |+| direct | Any | changed later | Keep the persisted direct path. |+| native | Any | changed later | Keep the persisted native path for read, cancel, recovery, and finalization. |++The server must not fall back from a selected `paperclip_runner` start to+`codex_local`. A configuration or rollout error must be visible. Silent fallback+would hide the runtime that executed the task.++## Direct adapter boundary++This rule applies to every built-in and plugin direct adapter. It includes:++- `codex_local`;+- `claude_local`;+- `opencode_local`;+- other local CLI or session adapters;+- process and HTTP adapters;+- gateway adapters; and+- external adapter plugins.++Adding Paperclip Runner must not add runner imports or runner branches inside a+direct adapter implementation. The heartbeat coordinator may select the+explicit runner adapter at one narrow seam. All other adapters continue through+their existing code.++For a flag-off `codex_local` heartbeat, compatibility proof must show:++- one direct invocation;+- the same normalized result and finalization bytes as the approved baseline;+- zero runner processes;+- zero PRP connections; and+- zero native rows.++## Configuration behavior++When the rollout flag is off:++- creation UI does not offer `paperclip_runner`;+- edit UI does not offer switching to `paperclip_runner`;+- server creation and import reject a new `paperclip_runner` selection;+- the server rejects a fresh start for an existing runner-configured agent;+- read and export preserve an existing runner configuration;+- unrelated edits to an existing runner-configured agent do not erase its+ configuration; and+- switching that agent to a direct adapter remains allowed.++When the rollout flag is on:++- creation, import, and edit accept `paperclip_runner` only with provider+ `codex` and valid Codex configuration;+- switching from a direct adapter affects only future unresolved runs; and+- switching away from the runner affects only future unresolved runs.++Server validation is the authority in both states. Import files and API clients+cannot bypass the flag or provider allowlist.++## Persisted native runs++The following data remains readable independent of the current flag:++- the persisted runtime selection and reason;+- run, runner, session, turn, and provider identity;+- ordered runner and provider events;+- accepted result and evidence assessment;+- finalization coordinator and status decision;+- usage, cost, cancellation, and terminal details; and+- durable interactions and final task-thread reply.++If the flag is disabled during an in-flight native run, the server may reconnect,+cancel, reconcile, and finalize that same run. It must not use that recovery as+authority to start a new native run.++Recovery must remain idempotent. Repeating it cannot add a second final reply,+interaction, wake, status decision, or application effect.++## Task-page compatibility++The task page uses one provider-neutral thread projection. Runtime facts may add+runner event groups, semantic receipts, usage, and structured questions. They+must not replace classic direct-adapter content.++Direct-adapter coverage must include:++- an active run;+- a settled run;+- an empty transcript;+- a pending interaction;+- a resolved interaction; and+- the classic interface state.++The existing composer remains usable for direct adapters. A direct run does not+show reconnect, runner cancel, semantic receipt, or other runner-only controls.+Final replies continue to use the existing issue-comment behavior.++## Structured input compatibility++New structured questions use `paperclip.question_set.v1` and the matching+response contract. Provider-specific question objects remain inside their+drivers.++Legacy unstructured interaction records remain readable and resolvable. A+structured form fails closed when its required schema, question mode, question+ID, option ID, or response value is invalid. The implementation must not silently+convert malformed structured input to a legacy text prompt.++## Protocol version compatibility++PRP wire versions, fixture versions, event schema versions, and typed schema+discriminators are independently versioned.++- Peers negotiate the highest common PRP version.+- No common required version fails closed before command or provider execution.+- An unknown required fixture or event schema version fails closed.+- An unknown required typed discriminator fails closed.+- Additive optional properties remain compatible only when an old v1 consumer+ can ignore them without changing behavior.+- Unknown object properties must survive validation when the owning schema+ permits additive fields.++Breaking meaning requires a new required version. A provider error is not a+protocol negotiation result.++## Semantic action compatibility++Catalog generation and production authorization are separate steps.++- A catalog entry does not authorize production use.+- An operation without a production binding is undiscoverable.+- An operation denied to the run is undiscoverable unless the protocol+ explicitly defines a safe denied receipt for that discovery mode.+- A duplicate call with the same idempotency key and canonical input returns the+ original safe receipt.+- Reusing the key with different input returns a conflict and performs no second+ effect.+- Receipts redact credentials, private provider payloads, and hidden identity.++## Required compatibility matrix++Each runner-related pull request updates only rows that it can execute. The+complete first-wave matrix must cover:++| Area | Required cases |+| --- | --- |+| Runtime selection | Every built-in direct adapter, explicit runner selection, unsupported provider, flag on, and flag off. |+| Direct regression | Flag-off `codex_local` invocation count, byte-stable result/finalization, and zero native rows. |+| Configuration | Enabled and disabled create, import, edit, read, export, and adapter switch. |+| Recovery | Persisted native run after flag disable, reconnect, duplicate event, duplicate command, cancellation, and server restart. |+| Protocol security | Cross-company binding, ticket replay, ticket expiry, malformed frame, unsupported version, and revoked lease. |+| Semantic actions | Discovery denial, unbound action, duplicate call, conflicting retry, redaction, and governed action. |+| Task page | Active, settled, empty transcript, interaction, and classic direct-adapter states. |+| Structured input | Valid response, malformed response, stale response, duplicate response, provider loss, and cancellation. |++## Pull request acceptance++Every implementation pull request must:++- build and work against its declared base;+- keep its changed-file count below 100;+- update manifests and exports only for implemented surfaces;+- run the smallest relevant tests before the repository handoff gate;+- run typecheck, tests, and build for handoff;+- run protocol parity, Cargo, clean-consumer, migration, token, and browser gates+ when those areas change;+- contain no unexplained failed, cancelled, or path-skipped verification; and+- resolve actionable review and security findings before it is ready.++Generated files land with their source and a drift check. `pnpm-lock.yaml` is+owned by CI and is not part of these pull requests.diff --git a/doc/architecture/paperclip-runner.md b/doc/architecture/paperclip-runner.mdnew file mode 100644index 0000000000..85b41813ba--- /dev/null+++ b/doc/architecture/paperclip-runner.md@@ -0,0 +1,286 @@+# ADR: Paperclip Runner architecture++- Status: Proposed+- Date: 2026-08-24+- Owners: Paperclip control plane and runner maintainers+- Related: [Paperclip Runner compatibility and rollout](paperclip-runner-compatibility.md)++## Context++Paperclip is a control plane. It owns companies, agents, issues, budgets,+approvals, and durable workflow state. Agent providers remain execution+services. Existing adapters invoke those services directly from the Paperclip+server.++Paperclip Runner introduces a separate execution process for provider sessions.+This process needs durable delivery, restart recovery, and governed access to+Paperclip actions. It must not become a second control plane. It must also land+without changing the behavior of existing adapters.++The initial implementation is intentionally narrow. It supports Codex through+an explicit, experimental adapter. Other providers and developer tools remain+outside this decision.++## Decision++Add a standalone package named `@paperclipai/paperclip-runner`. The package owns+the language-neutral Paperclip Runner Protocol (PRP), the Rust runner process,+provider drivers, deterministic replay, and semantic action dispatch contracts.++Add one explicit adapter named `paperclip_runner`. The adapter is available only+when an instance-level, default-off rollout flag is enabled. Its first supported+provider is Codex.++Do not route existing adapters through Paperclip Runner. A direct adapter keeps+its current invocation, transcript, interaction, cancellation, and finalization+paths.++## Goals++- Keep runner process ownership outside the Paperclip server process.+- Preserve Paperclip as the authority for identity, policy, and workflow state.+- Recover a run after runner or network interruption without duplicate effects.+- Expose only actions that the current run is allowed to use.+- Make protocol behavior deterministic across TypeScript and Rust.+- Keep existing adapter behavior unchanged while the runner is experimental.++## Non-goals++- Replace existing direct adapters.+- Move business authorization or issue status policy into Rust.+- Give runnerd a broad Paperclip API credential.+- Support OpenCode, ACPX, Claude Managed, AWS AgentCore, or remote sandboxes in+ the first production slice.+- Expose browser SDK, React SDK, eval, lab, or scenario-explorer package entry+ points in the initial release.+- Commit recorded screenshots, stress logs, or construction history as product+ architecture.++## Topology++The initial local topology is:++```text+Paperclip server+ | authenticated PRP v1 WebSocket+ v+paperclip-runnerd+ | Codex app-server protocol+ v+Codex+```++The server opens a native run and launches a verified runnerd artifact in the+realized execution environment. Runnerd opens the outbound PRP connection. It+then owns the provider process group and the durable transport state for that+run.++The browser does not connect to runnerd. It reads projections from the existing+Paperclip APIs and task-thread models.++## Dependency direction++The runner package must build and test without importing Paperclip server, UI,+CLI, database, or other private workspace implementation modules.++```text+JSON Schema and fixtures+ |+ +------------------++ v v+ TypeScript contracts Rust runner core+ | |+ +--------+---------++ v+ deterministic parity++Paperclip server ----implements----> runner public ports+```++The dependency points from an implementation to a contract. The Paperclip+server may implement a public runner port. The runner package must not import+the server implementation.++The initial public package surfaces are:++- `@paperclipai/paperclip-runner` for runtime contracts and clients.+- `@paperclipai/paperclip-runner/testing` for deterministic fakes and+ conformance helpers.++Every export must have an implementation and a clean-consumer test before it is+published. Later SDK, eval, and lab surfaces require separate decisions.++## Protocol boundary++PRP v1 uses a WebSocket at:++```text+/api/runner/v1/connect/:runId+```++JSON Schema is the language-neutral protocol authority. TypeScript and Rust use+the same canonical fixtures and must produce the same replay result.++Protocol rules are fail closed:++- Unknown required protocol or schema versions are rejected.+- Unknown required discriminators and enum values are rejected.+- Additive optional object fields may be accepted when v1 consumers can ignore+ them safely.+- Frames, headers, durable state, diagnostic tails, and replay windows are+ bounded.+- Event identity and ordering remain stable across reconnect and restart.++Provider-native messages do not cross this boundary. Drivers translate them to+provider-neutral PRP events, results, usage, cancellations, and structured+input.++## Trust boundary++The Paperclip server is authoritative for:++- company, agent, issue, run, session, and user attribution;+- rollout and runtime selection;+- action discovery and authorization;+- approval, budget, secret, revision, and workspace policy;+- durable application records and activity history;+- result acceptance, finalization, and issue status.++Runnerd is authoritative only for its local responsibilities:++- provider process supervision;+- provider session and turn transport;+- durable PRP outbox and command receipts;+- stable runner-side event identity;+- bounded process diagnostics.++Runnerd receives a short-lived, one-use bootstrap ticket. The ticket is bound to+the company, agent, issue, run, runner, session, turn, and verified artifact. The+server exchanges it for a short-lived connection lease. Raw tickets are never+stored. Runnerd never receives a broad Paperclip API key.++The server rejects expired, replayed, revoked, cross-company, mismatched,+malformed, oversized, or protocol-incompatible connections. Cancellation,+timeout, supersession, and environment-lease loss revoke runner authority.++## Durable delivery and recovery++Runnerd persists an event before it sends the event. The server acknowledges a+cumulative source cursor. Runnerd removes acknowledged data only after it+validates that cursor.++A reconnect reports the last processed command, the next source sequence, the+last acknowledged source sequence, and the unacknowledged range. The server may+then replay one pending command and accept byte-equivalent event retries.++Command identity is idempotent. Reusing a command ID with the same canonical+input returns the stored result. Reusing the ID with different input fails+closed. A repeated semantic tool call cannot repeat an application effect.++Recovery is bounded. Exhausted storage, reconnect, command, or time limits end+the run with a classified failure instead of an unbounded loop.++## Semantic actions++PRP carries provider-neutral semantic operation IDs. The package may define an+operation catalog, but catalog presence does not grant authority.++For each run, the server projects only operations that have a production+binding and that the current actor may discover. An unbound or unauthorized+operation is absent from discovery. The server validates inputs, executes the+existing application authority, and returns a redacted receipt.++This keeps these invariants in one place:++- company scoping;+- actor and run attribution;+- authorization and approval;+- revision and idempotency checks;+- budget and secret policy;+- activity logging and safe error details.++Runnerd cannot forge identity through tool input. Identity and scope come from+the authenticated connection binding.++## Structured input++Questions use `paperclip.question_set.v1`. Responses use the matching canonical+response contract. Drivers translate between these provider-neutral records and+provider-native input APIs.++The server validates a response at the untrusted API edge and again against the+persisted question set before delivery. A process loss may materialize a durable+task-thread interaction, but cancellation and an already resolved request must+not create a second interaction or continuation.++## Persistence and finalization++Native run records are additive. Existing heartbeat and issue records remain+readable. A runner result is an untrusted claim. It does not directly update an+issue status.++The server validates the result, classifies durable evidence, applies status+policy, commits workflow effects, and records finalization. Each phase is+idempotent and recoverable. Existing direct adapters keep their existing+finalization paths and do not invoke native status arbitration.++Disabling the rollout flag blocks fresh runner starts. It does not make an+already persisted native run unreadable or prevent bounded recovery and+finalization of that run.++## Rollout++The rollout has three gates:++1. The instance flag is enabled.+2. The agent explicitly selects `paperclip_runner`.+3. The adapter selects a supported provider. The initial provider is `codex`.++The adapter is hidden from creation and selection surfaces while the flag is+off. Server validation also rejects a fresh runner selection or start while the+flag is off. UI hiding is not the security boundary.++Runtime selection is persisted before launch. Later setting changes cannot+silently move an in-flight run between the direct and native execution paths.++The detailed compatibility rules are in+[Paperclip Runner compatibility and rollout](paperclip-runner-compatibility.md).++## Observability++The server records structured runner, provider, semantic action, interaction,+usage, and terminal events. Logs and receipts must not contain bootstrap+tickets, leases, provider credentials, secret values, complete environments, or+private host paths.++The task page may project these records through existing thread components. A+runner-only control must depend on persisted runtime facts. It must not depend+only on the agent adapter profile.++## Consequences++This design adds process, protocol, and recovery complexity. In return, it gives+provider sessions a durable and testable execution boundary without moving+Paperclip governance into the runner.++The default-off, explicit-adapter rollout duplicates some provider+configuration during the experiment. This is intentional. It keeps comparison+and rollback simple and prevents a global migration of existing agents.++The standalone package boundary requires generated artifacts, clean-consumer+tests, and cross-language parity gates. These checks add build cost, but they+prevent server implementation details from becoming accidental public API.++## Required proof before general availability++- TypeScript and Rust accept and reject the same protocol fixtures.+- Lost acknowledgements, reconnects, runner restarts, and duplicate commands do+ not duplicate events or effects.+- Bootstrap replay, binding mismatch, and cross-company access fail closed.+- Only authorized and production-bound actions appear in discovery.+- Cancellation settles provider, runner, transport, and application state.+- Flag-off direct-adapter runs create no native records and start no runner+ process.+- Persisted native runs remain readable and recoverable after the flag changes.+- Codex completes the server to PRP to runnerd to provider to server path.+- Existing adapter compatibility tests remain byte stable where specified.