🙌 OpenHands: AI-Driven Development
docs(codereview): clarify Agent Canvas checkpoints (#17523)
Graham Neubigd07e1126a37d
diff --git a/.agents/skills/custom-codereview-guide.md b/.agents/skills/custom-codereview-guide.mdindex 38697679b2..351cb755e8 100644--- a/.agents/skills/custom-codereview-guide.md+++ b/.agents/skills/custom-codereview-guide.md@@ -11,191 +11,198 @@ This guide supplements the public `code-review` skill with rules specific to`OpenHands/OpenHands`, the Agent Canvas frontend. Read `AGENTS.md` first; it isthe detailed source of truth for current architecture and test conventions.-Be direct and constructive. Review correctness and architecture, not formatting-that lint or the compiler already checks.+## Review Sequence and Decision-## Review Decision+Review the current PR head in this order:-- Submit exactly one review: **APPROVE** or **COMMENT**. Never use- **REQUEST_CHANGES**.-- Default to **APPROVE** when there are no important findings. Nitpicks and- optional cleanup are not reasons to withhold approval.-- Use **COMMENT** for correctness, security, architecture, missing evidence, or- unmet acceptance criteria. Let a human maintainer make the blocking decision.-- Do not approve changes that can affect agent or benchmark behavior—prompts,- tool selection, conversation payloads, terminal behavior, planning, memory,- or evaluation paths—without human review and appropriate lightweight evals.-- Read the linked issue and include a compact checklist covering each acceptance- criterion. Meeting the checklist is necessary but does not replace review for- regressions, security, or maintainability.+1. Read the linked issue, its acceptance criteria, and unresolved review threads.+2. Confirm that the change belongs in this repository and follows the dependency+ direction below.+3. Apply every relevant blocking checkpoint in this guide.+4. Inspect tests and production-facing evidence for the behavior changed.-## Repository Ownership+Submit exactly one review:-Put behavior in the repository that owns it:+- **APPROVE** when every applicable checkpoint passes and there are no material+ correctness, security, compatibility, architecture, or evidence gaps.+- **COMMENT** when a material gap remains. State the concrete consequence, the+ unmet checkpoint or acceptance criterion, and the smallest viable correction.+- Never use **REQUEST_CHANGES**. A human maintainer owns the blocking decision.++For changes that can affect agent or benchmark behavior—prompts, tool selection,+conversation payloads, terminal behavior, planning, memory, or evaluation+paths—state the eval risk in the review. Missing optional eval evidence is not a+material code finding: when the current head otherwise passes review, APPROVE so+the automation can request a human maintainer to choose the appropriate+lightweight evaluation. Use COMMENT when an acceptance criterion or required+check calls for specific eval evidence and that evidence is missing or failing,+or when available results show a regression.++Include a compact checklist for every linked acceptance criterion. Meeting the+checklist is necessary but does not replace review for regressions, security, or+maintainability.++## Repository Ownership and Dependency Direction| Repository | Owns || ------------------------------ | --------------------------------------------------------------------------------------------------------------- || `OpenHands/OpenHands` | Agent Canvas UI, frontend state, backend selection, frontend service integration, and local-stack orchestration |-| `OpenHands/software-agent-sdk` | Agent Server, agents, tools, conversations, events, workspaces, and the canonical server API |-| `OpenHands/typescript-client` | Browser-compatible typed access to the Agent Server API |+| `OpenHands/software-agent-sdk` | Agent Server, SDK, canonical server API, and browser-compatible client in `clients/typescript/` || `OpenHands/extensions` | Reusable skills, plugins, and integrations || `OpenHands/automation` | Scheduling, webhooks, run history, and automation dispatch |The normal dependency direction is Agent Server contract → TypeScript client →-Canvas. Flag raw endpoint reimplementations, Canvas-local copies of server-contracts, and changes opened in the wrong repository.--## Architecture That Guides Agents--Agents tend to copy the nearest pattern and choose the shortest compiling path.-Review the codebase as part of the product surface that guides those choices:--1. **Make the conventional path cheapest.** New work should naturally reuse a- named hook, service, store, or feature module instead of adding another branch- to a shared root.-2. **Fail forbidden dependencies mechanically.** Repeated review guidance should- become a lint rule, compiler boundary, or architecture test. Do not grow this- document when a small executable guard would be clearer.-3. **Give durable state one obvious writer.** A backend setting, consent value,- conversation cache entry, or persisted browser value should have one named- owner. Flag second writers and component-local mirrors of authoritative state.-4. **Prefer owned feature files over shared switches.** Product work should- usually extend a feature-owned module. Shared registries and root conditionals- need a concrete reason.-5. **Keep exceptions narrow and visible.** Exceptions belong in a small allowlist- next to the guard that enforces the rule and should be reviewed as architecture- changes.--Treat “deep module” as a design heuristic, not a line-count target. A good module-has a narrow, stable interface and hides cohesive complexity. Do not split a file-merely because it is long, and do not create layers that only rename or forward-arguments. Prefer a small pure seam when it removes duplicated decisions, makes-ownership explicit, or enables focused tests.--### React effects--`useEffect` is for synchronizing React with an external system. Flag effects used-to:--- derive render data from props or state;-- respond to a user action that can run in the event handler;-- initialize a value that belongs in a lazy state initializer;-- mirror one store or cache into another component state value; or-- repair ordering created by competing writers.--An effect is not automatically wrong. Subscription, browser API, timer, and-network synchronization still belong in effects when cleanup and dependency-semantics are explicit.--## Blocking Architecture Checkpoints+Canvas. Submit **COMMENT** for raw endpoint reimplementations, Canvas-local copies+of server contracts, or behavior implemented in the wrong repository.++## Blocking Checkpoints### Agent Server and Cloud API access-`src/api/no-direct-agent-server-calls.test.ts` is the executable source of truth.-Do not approve new raw `fetch`, `axios`, shared `openHands`, or low-level HTTP-client access to Agent Server endpoints. Use `@openhands/typescript-client` with-the options from `src/api/agent-server-client-options.ts`.+Apply this checkpoint when a change calls or models an Agent Server, Cloud, or+runtime-sandbox API.++- `src/api/no-direct-agent-server-calls.test.ts` is the executable source of+ truth. Agent Server access must use `@openhands/typescript-client` with options+ from `src/api/agent-server-client-options.ts`.+- Cloud and runtime-sandbox requests must use `callCloudProxy`; runtime requests+ must provide the correct `hostOverride` and authentication mode.+- Treat changes to the guard's allowlist as architecture changes. Do not copy the+ allowlist into this guide.++Submit **COMMENT** if the PR adds raw `fetch`, `axios`, shared `openHands`, or+low-level HTTP client access to an Agent Server endpoint.++### Agent Server compatibility++Apply this checkpoint when Canvas begins relying on a new Agent Server endpoint,+field, schema, or behavior. Canvas and the Agent Server are independently+versioned.++Require an increase to `minimumAgentServer` to the first compatible released version.++Verify the compatibility boundary. Adding a TypeScript-client method does not+make older Agent Servers support it. Submit **COMMENT** if a supported backend+can reach the new code and fail because the required server behavior is absent.++### Affected Canvas modes++Apply this checkpoint when a change touches a shared adapter, conversation+builder, setting, state selector, or presentation helper.++Enumerate the affected consumers across these dimensions:-Cloud and runtime-sandbox requests must go through `callCloudProxy`; runtime-requests must provide the correct `hostOverride` and authentication mode. Review-changes to the guard's allowlist as architecture changes. Do not copy its current-entries into this guide—the test should remain the one authoritative list.+- Local and Cloud backends;+- OpenHands and ACP agents;+- standard, planning, and delegated conversations; and+- standalone and embedded Canvas.++Verify every affected path. Do not require the full Cartesian product when+control or data flow proves a dimension is isolated. Submit **COMMENT** when an+affected variant can take a distinct path but the implementation or evidence+covers only the default.### Event wire contracts-The SDK event model is the wire authority, the TypeScript client mirrors it, and-Canvas consumes the published client type. Do not approve Canvas-local-redeclarations, partial intersections, module augmentation, or presentation-fields added to wire-event interfaces.+Apply this checkpoint when a change reads, extends, or renders Agent Server+events.-A contract change should land in this order:+The SDK event model is the wire authority, the TypeScript client mirrors it, and+Canvas consumes the published client type. A contract change must land in this+order:1. SDK model/schema and serialization coverage.2. TypeScript-client mirror derived from the SDK payload.3. Published client release.-4. Canvas consumption and rendering/telemetry coverage.+4. Canvas consumption and rendering or telemetry coverage.Canvas-only presentation state belongs in a separate view model keyed by event-identity.+identity. Submit **COMMENT** for Canvas-local wire redeclarations, partial+intersections, module augmentation, or presentation fields added to wire types.++### Durable state and transitions++Apply this checkpoint when a change reads or writes a backend setting, profile,+conversation cache entry, or persisted browser value.++- Give the value one named owner and one obvious writer. Do not mirror an+ authoritative store into component-local state.+- When a stored shape or selection rule changes, verify existing-state hydration+ or migration as well as create, update, delete, default, and active-selection+ transitions that the feature supports.+- A destructive transition must leave a deterministic valid state or an explicit+ empty state that the UI handles.-### Telemetry and durable frontend state+Submit **COMMENT** if existing users can lose state, a delete or reset can leave+an invalid selection, or multiple writers can race or overwrite one another.-- `src/services/telemetry.ts` is the only owner of the Canvas PostHog client.-- React events go through typed functions in `src/hooks/use-tracking.ts`; components- must not call PostHog directly.-- Consent rendering uses the telemetry consent external store, not mirrored local- state. `setTelemetryConsent` remains the single consent controller.-- A business milestone has one canonical capture. Flag duplicate conditional- captures.-- For other durable values, prefer the existing named service/store/hook and flag- new storage writes from arbitrary components.+Telemetry has stricter named owners:++- `src/services/telemetry.ts` exclusively owns the Canvas PostHog client.+- React events use typed functions from `src/hooks/use-tracking.ts`.+- Consent rendering uses the telemetry consent external store;+ `setTelemetryConsent` is the only consent controller.+- A business milestone has one canonical capture.++## Design Review++Prefer named hooks, services, stores, and feature modules over shared-root+branches or switches. Keep necessary exceptions in a narrow allowlist beside an+executable guard. Do not add layers that only rename or forward arguments, or+split a cohesive file because it is long.++`useEffect` is for synchronization with an external system, not derived render+data, user actions, lazy initialization, store mirroring, or ordering repairs.+Subscriptions, browser APIs, timers, and network synchronization remain valid+when cleanup and dependencies are explicit.## Dependencies and Releases-- Direct dependencies are exact-pinned. Keep `package.json` and- `package-lock.json` synchronized through npm; do not hand-edit one side only.-- Treat changes to dependency exemptions, git pins, and security overrides as- reviewable policy changes. `__tests__/package-library.test.ts` is the executable- source of truth for allowed specs.-- Scrutinize newly published third-party dependency versions for supply-chain- risk. First-party OpenHands packages are exempt from a waiting period but not- from contract and release-order review.-- Package version changes belong in explicit release PRs and must match the- release workflow expectations.--## Testing and Evidence--- Require evidence proportional to the behavior changed. For UI behavior, use a- screenshot or video from the real app. For CLI, API, or scripts, require the+- Direct dependencies are exact-pinned. Update `package.json` and+ `package-lock.json` together through npm.+- Treat dependency exemptions, git pins, and security overrides as policy+ changes. `__tests__/package-library.test.ts` is the executable source of truth.+- Scrutinize newly published third-party versions for supply-chain risk.+ First-party OpenHands packages are exempt from a waiting period, not from+ contract and release-order review.+- Package version changes belong in explicit release PRs and must match release+ workflow expectations.++## Testing and Production Evidence++- Require evidence proportional to the behavior changed. UI changes need a+ screenshot or video from the real app; CLI, API, and script changes need theexact runtime command and observed result.-- Runtime and user-visible bug fixes require before-and-after evidence through- the real production-facing path. The before evidence must reproduce the bug on- the base branch or released version; the after evidence must repeat the same- setup on the PR head and show the corrected behavior. Include exact commands,- relevant output, and screenshots or video when the behavior is visual.-- For lifecycle fixes, evidence must also verify the resulting process or resource- state—for example, the parent exit code and whether child services or listening- ports remain after shutdown.-- Unit and integration tests are regression proof, not a substitute for live- evidence. If required live evidence is missing, submit **COMMENT**, not- **APPROVE**, and identify the exact production-facing verification still needed.-- Prefer tests that exercise real logic and observable state. Do not reward mocks- that only prove another mock was called.-- Keep tests focused: one meaningful assertion path per behavior, no duplicated- coverage of library behavior, and no brittle presentation-only snapshots.-- Follow the test routing in `AGENTS.md`. The mock-LLM, Docker mock-LLM, and- live LLM-backed E2E suites run after changes reach `main`, not from PR labels.- If a risky PR needs pre-merge E2E evidence, recommend manually dispatching the- relevant workflow against the PR branch.-- Never broaden live E2E triggers or secret exposure for convenience.--## Review Context Integrity--Before submitting the review, compare its summary and every finding against the-current PR title, changed-file manifest, linked issues, and acceptance criteria.-If the review describes files, behavior, issues, or release paths that are not in-that context, stop and re-read the PR rather than submitting stale or mismatched-feedback. Do not approve until this final context check passes.--## What Not to Comment On--Do not leave review comments for:--- formatting or minor style that tooling handles;-- optional “nice to have” refactors unrelated to the change;-- praise-only observations—approve instead;-- extra tests for straightforward data/config changes when existing checks cover- the risk; or-- temporary `.pr/` artifacts, which are cleaned up by repository automation.--When raising a finding, trace the relevant call or data flow far enough to show-the concrete failure mode. Prefer one high-signal comment over several symptoms-of the same ownership problem.--## Communication Style--- Be concise, specific, and friendly.-- Explain the user-visible or architectural consequence.-- Suggest the smallest viable correction.-- Use GitHub suggestion syntax for local fixes.-- If the PR is sound, approve it without manufacturing feedback.+- Runtime and user-visible bug fixes require the same production-facing setup+ before and after the change. The base or released version must reproduce the+ bug; the PR head must show the corrected behavior.+- Lifecycle fixes must also verify resulting process or resource state, such as+ the parent exit code and remaining child services or listening ports.+- Tests must exercise real logic and observable state. A claimed regression test+ must reach the target behavior and fail when that behavior regresses; mocks+ that only prove another mock was called are insufficient.+- Do not duplicate library behavior or add brittle presentation-only snapshots.++Tests are regression proof, not a substitute for required live evidence. Submit+**COMMENT** when production-facing evidence is required but absent, and name the+exact verification still needed.++Follow the test routing in `AGENTS.md`. Mock-LLM, Docker mock-LLM, and live+LLM-backed E2E suites run after changes reach `main`, not from PR labels. For+risky pre-merge changes, recommend manually dispatching the relevant workflow+against the PR branch. Never broaden secret exposure for convenience.++## Final Context and Comment Discipline++Before submitting, compare the review summary and every finding with the current+PR title, head commit, changed-file manifest, linked issues, acceptance criteria,+and existing review threads. If a finding describes files or behavior outside+that context, stop and re-read the PR.++Do not comment on formatting handled by tooling, minor style, praise, optional+unrelated refactors, extra tests for straightforward data/config changes, or+temporary `.pr/` artifacts. Do not manufacture feedback to avoid approval.++For every finding, trace the call or data flow far enough to show the concrete+user-visible or architectural consequence. Prefer one root-cause comment over+several symptoms, and suggest the smallest viable correction.diff --git a/AGENTS.md b/AGENTS.mdindex fd93b9bb6f..c43efb9373 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -28,13 +28,12 @@ piece of a multi-repo system. Before adding code here, check the change belongs| Repo | Owns | Add code here when… ||------|------|---------------------|| **`OpenHands/OpenHands`** (this repo) | The React/TypeScript **frontend** (agent-canvas): UI, routes, frontend services in `src/api/` that *consume* backend APIs. | You are changing UI, frontend state, or how the frontend *calls* an existing backend endpoint. |-| **`OpenHands/software-agent-sdk`** | The Python **SDK + agent-server**: agents, tools, conversations, events, and the REST/WebSocket **API surface** (`openhands-sdk`, `openhands-tools`, `openhands-agent-server`, `openhands-workspace`). | You are adding or changing a backend endpoint, agent/tool behaviour, or server-side logic. New API **endpoints** live here, not in the frontend. |-| **`OpenHands/typescript-client`** (`@openhands/typescript-client`) | The generated/maintained **TypeScript client** that mirrors the agent-server API. The frontend's *only* sanctioned way to reach the agent-server (see "API Access Rules"). | You are adding client-side **access to an agent-server endpoint** (typed client method, request/response types). API-access code belongs here, **not** re-implemented in this repo. |+| **`OpenHands/software-agent-sdk`** | The Python **SDK + agent-server** and `clients/typescript/`: agents, tools, conversations, events, the REST/WebSocket **API surface**, and the browser-compatible client that mirrors it. | You are adding or changing a backend endpoint, agent/tool behaviour, server-side logic, or typed client access to that API. New endpoints and client methods live there, not in the frontend. || **`OpenHands/extensions`** (`@openhands/extensions`) | Public **skills, automations, and integrations** (loaded here at build time via `SKILLS_CATALOG`). | You are adding or editing a skill, automation, or MCP integration. |Common mis-placements to avoid:-- **API endpoint access** → belongs in `typescript-client`, then consumed here. Do **not**+- **API endpoint access** → belongs in `software-agent-sdk/clients/typescript/`, then consumed here. Do **not**add raw `axios`/`fetch` endpoint code to the frontend (CI guard:`src/api/no-direct-agent-server-calls.test.ts`; see "API Access Rules").- **New server endpoints / agent or tool logic** → belongs in `software-agent-sdk`.@@ -47,11 +46,28 @@ The four repositories have distinct ownership boundaries:| Repository | Owns ||---|---|| [`OpenHands/OpenHands`](https://github.com/OpenHands/OpenHands) | Agent Canvas frontend, user-facing control center, backend selection, and local-stack orchestration. |-| [`OpenHands/software-agent-sdk`](https://github.com/OpenHands/software-agent-sdk) | Python SDK, Agent Server, agent/tool behavior, conversations, workspaces, events, and the canonical server API. |-| [`OpenHands/typescript-client`](https://github.com/OpenHands/typescript-client) | Browser-compatible TypeScript client and generated/maintained types for the Agent Server API. |+| [`OpenHands/software-agent-sdk`](https://github.com/OpenHands/software-agent-sdk) | Python SDK, Agent Server, agent/tool behavior, conversations, workspaces, events, the canonical server API, and `clients/typescript/`. || [`OpenHands/automation`](https://github.com/OpenHands/automation) | Automation definitions, scheduling, webhooks, run history, and dispatching. It manages when automations run; the Agent Server/SDK executes them. |-The usual dependency direction is `software-agent-sdk` / Agent Server → OpenAPI contract → `typescript-client` → Agent Canvas. Automation scheduling and dispatching flow from Agent Canvas to `automation`, which starts work on the Agent Server/SDK. Put new server behavior and endpoints in `software-agent-sdk`, client access in `typescript-client`, UI and frontend integration in this repository, and scheduling/webhook lifecycle behavior in `automation`.+The usual dependency direction is Agent Server → OpenAPI contract → `software-agent-sdk/clients/typescript/` → Agent Canvas. Automation scheduling and dispatching flow from Agent Canvas to `automation`, which starts work on the Agent Server/SDK. Put new server behavior, endpoints, and typed client access in `software-agent-sdk`, UI and frontend integration in this repository, and scheduling/webhook lifecycle behavior in `automation`.++## Shared Frontend Change Checklist++Before changing a shared adapter, conversation builder, setting, state selector,+or presentation helper:++- If Canvas begins relying on a new Agent Server endpoint, field, schema, or+ behavior, raise `compatibility.minimumAgentServer` in `config/defaults.json`+ to the first compatible released version.+- Enumerate the affected consumers across Local and Cloud backends, OpenHands+ and ACP agents, standard/planning/delegated conversations, and standalone and+ embedded Canvas. Implement and test every path that has distinct control or+ data flow; do not test an irrelevant Cartesian product.+- Give durable frontend state one named owner and one obvious writer. When a+ stored shape or selection rule changes, handle existing-state hydration or+ migration and the supported create, update, delete, default, and active-item+ transitions. A destructive transition must leave a deterministic valid state+ or an explicit empty state that the UI handles.All pull requests for this repository must comply with [`.agents/skills/custom-codereview-guide.md`](.agents/skills/custom-codereview-guide.md), in addition to the general contribution requirements and CI checks.