Skip to content

🙌 OpenHands: AI-Driven Development

docs(verify-openhands): map the automation setup page as F28 (#18452)

Devin5cbbb90b5b9c
diff --git a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md
index f86e98c6a7..692265e723 100644
--- a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md
+++ b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md
@@ -10,7 +10,8 @@ MCP integrations, or seeds a conversation with the entry's launch prompt.
**Add Automation → Create Automation** opens the agent-assisted setup page at
`/automations/setup`, where the form opens before a conversation exists. Saving a server-backed
draft and testing its generated custom script need no LLM; sending a prompt
-starts genuine model assistance through `automation_form_update`. **Add Automation → Import automation** reads
+starts genuine model assistance through `automation_form_update`. The setup page's own fields,
+draft validation, Test and Create each have their own IDs in F28. **Add Automation → Import automation** reads
an exported JSON file, previews it and creates it disabled.
Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-route.tsx`, `src/routes/automation-setup-new.tsx`, `src/components/features/automations/` (`recommended-automations-launcher.tsx`, `recommended-automations-section.tsx`, `responder-deployment-modal.tsx`, `create-instructions.tsx`, `import-automation-modal.tsx`, `agent-profile-selector.tsx`, `setup/`), `src/components/features/conversation/conversation-main/conversation-main.tsx`, `src/components/features/manifest/`, `src/manifests/`, `src/hooks/use-start-automation-setup.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/hooks/use-responder-url-secret.ts`, `src/utils/automation-export.ts`, `src/api/automation-service/automation-service.api.ts`, `src/api/automation-form-client-tool.ts`, `src/api/automation-form-session.ts`, `src/services/automation-form.ts`, `src/constants/automation-form.ts`, `node_modules/@openhands/extensions/automations/`.
diff --git a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md
index 653a9bc607..f3ca82048d 100644
--- a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md
+++ b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md
@@ -51,6 +51,8 @@ Source: `src/routes/automation-detail.tsx`, `src/components/features/automations
- `F23.edit-close`: **Close** leaves without saving and lands on `/automations` (the dashboard, not the detail page); Escape does nothing on the page.
- `F23.edit-validation`: an empty prompt is refused with the toast **Add a prompt before continuing.** (no request); a bad cron, a cron that never runs or a timeout above 1800 should be refused with a readable message. Known failure (reproduced 2026-10-13, #18380; regression from #17688): Save sends the PATCH and shows only the toast `Request failed with status code 422`; the API detail (`Value error, Invalid cron expression: …`) is dropped. BASE's modal refused the cron inline (`Enter a valid cron expression`).
- `F23.edit-save`: Save sends one PATCH (also when nothing changed), shows **Saving…** then **Saved just now**, the toast **Automation updated**, and the detail page and API reflect it; an Active profile model stays `null`.
+- `F23.edit-test`: on the edit page **Test** dispatches the automation as last saved (`POST /api/automation/v1/<id>/dispatch`, no PATCH first) and shows `Test run started.` on the form; unsaved edits are not part of that run. Whether it should test the current form is a product decision (Gotchas).
+- `F23.edit-dirty-state`: the edit page shows no save state for unsaved edits (no **Unsaved changes**, which create mode shows, F28); the label appears only while saving (**Saving…**) and after (**Saved just now** or **Save failed**). **Close** discards the edits.
- `F23.edit-schedule`: Frequency offers Once, Hourly, Daily, Weekdays (Mon–Fri), Weekly and Custom; Weekly adds **Day of week** and **At**; they are saved as a cron expression and read back after a reload.
- `F23.edit-time-cleared`: clearing **At** on a preset schedule and saving should be refused with a message instead of being dropped or replaced. Known failure (reproduced 2026-10-13, #17948 item, new symptom since #17688): Save silently stores midnight (`0 0 * * 3`) and toasts **Automation updated**; Save draft in the create flow stores `0 0 * * *` the same way. Open PR #18260 changes only `edit-automation-modal.tsx`, which nothing imports on TARGET, so it does not fix this.
- `F23.edit-cron`: a custom cron schedule shows Frequency **Custom** with an editable **Custom** cron field and a timezone select.
@@ -106,11 +108,13 @@ Preconditions:
- **Run now and save errors (`F23.run-now-error`, `F23.edit-save-error`).** Arrange a stale page: open QA_F23 Fail, then `control-openhands api DELETE /api/automation/v1/<Fail id> --write`. Run `control-openhands browser toasts --history --clear` (empties the toast history), click `role=button[name="Run now"]`, then `control-openhands browser toasts --history` and `control-openhands browser screenshot --feature F23.run-now-error --name toasts`: exactly one toast, `Automation not found`, and no `Request failed with status code 404`. Expected: the page then shows **Automation not found**: `control-openhands browser wait 'role=heading[name="QA_F23 Fail"]' --state detached --timeout 15000` succeeds (the not-found state takes about 7 s, as in States) and `control-openhands browser snapshot 'role=paragraph >> text=Automation not found >> xpath=..'` lists link `Back to Automations` (do not use `wait-text 'Automation not found'` here: the toast has the same text). Once the heading is gone, run `control-openhands browser toasts --history` again: it still holds only `Automation not found` (a late second toast, `Request failed with status code 404`, would come from the page's own refetch of the deleted automation). Known failure: the wait times out and the page keeps heading `QA_F23 Fail`, `Active` and an enabled **Run now** (`control-openhands browser enabled 'role=button[name="Run now"]'` is `true`) until a reload (#18062). The save error does not reuse that page (it is gone once #18062 is fixed): arrange a second fixture, `control-openhands api POST /api/automation/v1/preset/prompt --write --data '{"name":"QA_F23 Stale","prompt":"Reply with the single word: pong.","trigger":{"type":"cron","schedule":"0 0 1 1 *","timezone":"UTC"},"enabled":false}'` (`body.id` is `<stale-id>`), open QA_F23 Stale, its kebab and `role=list >> has-text=Tarball >> role=button[name="Edit"]` (`--expect-url '/automations/setup'`). With the edit page open, `control-openhands api DELETE /api/automation/v1/<stale-id> --write` and `control-openhands browser toasts --history --clear`. Fill `testid=automation-setup-name` with `QA_F23 Stale renamed` and click `testid=automation-setup-save`: the page keeps the edit (`browser value 'testid=automation-setup-name'` still `QA_F23 Stale renamed`) and `browser toasts --history` holds exactly one toast. Expected: `Automation not found` (BASE's modal showed it). Known failure (reproduced 2026-10-13, regression from #17688, #18380): the toast reads `Request failed with status code 404`. Screenshot with `--feature F23.edit-save-error --name toast`, move the pointer off the toast and Close. `browser errors --app-only` lists the induced `POST /api/automation/v1/<Fail id>/dispatch` and `PATCH /api/automation/v1/<stale-id>` 404s; both fixtures are already deleted.
- **Load more runs (`F23.activity-log-more`).** Not run: it needs more than 20 runs of one automation (about 35 s and one model call each). With them, `control-openhands browser click 'testid=automation-activity-log >> role=button[name="Load more"]'` grows the list from 20 to 40 rows.
- **Script automation (`F23.script-section`).** Arrange a script automation through F22's bundle entry (UI arrange; F22 owns the creation proof): `control-openhands browser goto /automations/templates`, `control-openhands browser click 'testid=recommended-automation-card-news-digest' --expect-url '/automations/new/news-digest'`, `control-openhands browser fill 'testid=setup-field-schedule' '0 0 1 1 *'`, `control-openhands browser click 'testid=setup-continue-button'`, `control-openhands browser wait 'testid=setup-review' --timeout 10000`, `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}' --timeout 60000`. The detail page of **Daily news digest** shows `role=heading[name="Script"]` and no Prompt (`browser count 'role=heading[name="Prompt"]'` is `0`); `control-openhands browser text 'testid=automation-script-entrypoint'` is `python3 main.py`, and `browser eval "Array.from(document.querySelectorAll('[data-testid=automation-script-file]')).map(e=>e.innerText.split('\n').slice(0,2).join(' | '))"` lists `main.py | Entrypoint` first, then `config.json` (both with line-numbered source). Screenshot with `--feature F23.script-section --name news-digest`. `automation-script-unavailable` (bundle unreadable) was not reached. Do not Run now: the script fetches public feeds and starts a model-backed conversation. Delete it in the Delete bullet.
+- **Edit-mode Test and unsaved edits (`F23.edit-test`, `F23.edit-dirty-state`).** Open QA_F23 Event and its kebab **Edit** (`--expect-url 'automationId='`). Fill `testid=automation-setup-name` with `QA_F23 Unsaved`: `control-openhands browser count 'testid=automation-setup-save-state'` is `0`. `control-openhands browser network --clear`, click `testid=automation-setup-test`; `control-openhands browser network --filter 'automation/v1'` shows `POST …/<id>/dispatch 201` and no `PATCH`, and the form shows `Test run started.` Move the pointer off the toast (`control-openhands browser hover 'testid=automation-setup-prompt'`) and click `testid=automation-setup-close` (`--expect-url '/automations$'`): the card still reads `QA_F23 Event`. The dispatched run is not checked here; Delete below removes it with the automation.
- **Delete (`F23.delete`).** Open QA_F23 Event, click the kebab and `role=list >> has-text=Tarball >> role=button[name="Delete"]`. `control-openhands browser snapshot 'role=dialog[name="Delete automation"]'` shows `Delete "QA_F23 Event"? This cannot be undone.`, `Cancel` and `Delete` (the same dialog as F21.delete). `control-openhands browser click 'role=dialog[name="Delete automation"] >> role=button[name="Cancel"]'` keeps the page (`control-openhands browser count 'role=dialog[name="Delete automation"]'` is `0`). Reopen and `control-openhands browser click 'role=dialog[name="Delete automation"] >> role=button[name="Delete"]' --expect-url '/automations(\?|$)'`; `control-openhands api GET /api/automation/v1` no longer lists it. Delete QA_F23 Pong, QA_F23 Blocked and Daily news digest the same way (and QA_F23 Fail if the error bullet was skipped); `api GET /api/automation/v1` shows `"total": 0`.
- **Errors.** After each group `control-openhands browser errors --app-only` shows no page errors; the only app errors are the induced ones (404s for the unknown id and the deleted fixture, 502s while the service is stopped).
## Gotchas
+- The automations QA checklist (F28) expects edit-mode **Test** to run the current form state; the panel calls `dispatchAutomation(<id>)`, which runs the saved automation. `F23.edit-test` records current behavior; whether it should change is a product decision, not a map correction.
- The Logs dialog's panel keeps long content reachable (`F23.run-logs-modal`, #18175 / #18173). Re-verified 2026-10-13 (revision `2bdc0e9d`, SDK 1.54.0, automation 1.19.3) with a 30-line `outcome_summary` at 1440×1000, 390×844 and 390×500: the panel is `testid=run-logs-body` (`max-h-[80vh]`, `overflow-y: auto`) with a sticky header (Logs + X stay visible); the Output box no longer scrolls on its own, the whole panel does, and its end (`=== RESULT === ALL_OK`) is reachable at the bottom. Verified 2026-10-10 on an independent fresh Linux Local stack at product revision `5a8c3832154b14d2dad651801c0ab3f74c96f02e`, SDK 1.54.0 and automation 1.19.3: a genuine DeepSeek finish produced a 4,596-character `outcome_summary` and 1,079-character metadata `message`; summary/metadata endings, Output/Error tabs and the true stdout end were reachable at 1440×1000, 390×844, 1440×500 and 390×500. The header/X remained visible, closing did not navigate, and phone had no horizontal overflow. Preserve those long-content and short-height checks when retesting. The verified scope is pinned Linux Local; macOS, Cloud and legacy/custom finish schemas need their own checks. `browser bbox 'role=dialog'` measures the full-screen wrapper, not the panel: measure the panel with `browser eval`.
- The detail page has no `main` landmark: `browser snapshot 'role=main'` times out. Snapshot the page, or scope by section headings (`role=heading[name="Configuration"] >> xpath=ancestor::div[2]`).
- The kebab menu has no test ids; `role=list >> has-text=Tarball` scopes the detail kebab (the dashboard kebab has no Tarball, so F21 uses `has-text=Export`).
diff --git a/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md
new file mode 100644
index 0000000000..ffaa6539d1
--- /dev/null
+++ b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md
@@ -0,0 +1,75 @@
+# F28 — Automation setup page
+
+The setup page is the one form for making and changing automations outside the
+template dialog. **Add Automation → Create Automation** opens it blank at
+`/automations/setup`; a saved draft reopens at `/automations/setup?draftId=<id>`;
+kebab **Edit** opens an existing automation at
+`/automations/setup?automationId=<id>`. The form collects Name, a Prompt (or
+Custom Python with entrypoint and setup script), agent profile, model and
+repositories, a Schedule or Event trigger, and optional plugins and timeout.
+A docked composer under the form can ask the agent to fill it. Create mode's
+toolbar has **Save draft**, **Test** and **Create Automation**; edit mode's has
+**Test**, **Save** and **Close**. A server-backed draft carries the server's
+validation result and test runs; Create turns it into a real, enabled
+automation and removes the draft.
+
+F22 keeps `F22.agent-assisted-setup` (the end-to-end agent, plugin, phone and
+draft-Test recipes); F23 owns edit mode (`F23.edit-*`, including Test and the save state on the edit
+page). This family gives each create-mode behavior its own ID so a pass can fail one without
+failing the others.
+
+Source: `src/routes/automation-setup-new.tsx`, `src/components/features/automations/setup/` (`automation-setup-panel.tsx`, `automation-setup-prompt-stack.tsx`, `automation-setup-draft-service.ts`), `src/hooks/use-start-automation-setup.ts`, `src/hooks/use-open-automation-editor.ts`.
+
+## Sub-features
+
+- `F28.open-blank`: Create Automation opens `/automations/setup` with Name pre-filled **New Automation** (placeholder `Standup digest`), an empty Prompt, Schedule selected with Daily at `09:00` in the browser's time zone, the docked composer (`Need help creating an automation?`) and no new conversation. The save state already reads **Unsaved changes** and the toolbar **Test** is disabled.
+- `F28.kind-toggle`: the button beside the Prompt label switches between Prompt and **Custom** (Custom Python). Custom shows a generated Python script that calls `AUTOMATION_CALLBACK_URL`, and **Entrypoint and setup** reveals Entrypoint `python3 main.py`, Setup script path `setup.sh` and a no-op setup script; switching back shows the Prompt again.
+- `F28.schedule-presets`: in create mode the Frequency control (the same control as on the edit page, whose presets and saving F23 owns as edit-schedule and edit-cron) starts on Daily at `09:00`, and switching presets fills create-mode defaults: Once a date-time at the next whole hour today, Weekly Monday, Custom the last custom cron, `0 9 * * *` until one is typed (it is not derived from the preset: from Weekly on Monday it also reads `0 9 * * *`). Every preset keeps the 13-entry time zone select.
+- `F28.event-trigger`: choosing **Event** replaces the schedule with Event source (default `github`; its menu lists the webhook sources registered on this backend plus **Custom**), Event key (default `issue_comment.created`), **Add Filter** (one free-text filter input), **Ask Agent**, the notice `Event sources need a public URL to deliver webhooks to this instance.`, and a **Test event payload** pre-filled with a JSON object for that source and key. A source the backend has not registered (menu **Custom**, then a typed name) adds **Register this as a custom webhook source**, checked by default.
+- `F28.timeout`: **Add Timeout** adds an optional **Timeout seconds** number field with a remove button; `1800` saves and the draft reads **Ready to test**.
+- `F28.draft-save`: **Save draft** works on any form, valid or not; the first save replaces the URL with `?draftId=<id>`, shows the **Draft saved** bar (relative time, **View test runs**, **Test**) and the toast `Draft saved in this conversation.`, and the save state reads **Saved just now**. The draft appears in the dashboard Drafts group (F21).
+- `F28.draft-validation`: the server validates on save, not on blur. An invalid draft is still saved (`dispatchable: false`) and its first error replaces **Ready to test** in the draft bar: an empty prompt shows `Field required` (also under Prompt), a bad custom cron shows `Value error, Invalid cron expression: every day`. A field error stays until the next save.
+- `F28.draft-save-error`: a draft the server refuses outright (a timeout above 1800 returns `422`) sets the save state to **Save failed** and shows one error toast with the reason. Known failure (reproduced 2026-10-14, not filed yet): the toast reads only `Draft body does not match endpoint schema`; the reason the API returns in `detail.errors` (`Value error, timeout must not exceed 1800 seconds (30 minutes)`) is never shown, and the draft bar keeps the previous save's **Ready to test**.
+- `F28.required-before-test`: Test and Create check locally first and send nothing when a required field is empty: `Add a prompt before continuing.` (Prompt or Plugin kind), `Add a plugin source before continuing.`, `Add Python code before continuing.`, `Add an entrypoint command before continuing.`, `Add a setup script path before continuing.`, `Add a setup script before continuing.`; an event Test with a payload that is not a JSON object shows `Enter a valid JSON object before testing this event automation.` A blank Name is not an error: it is derived from the prompt.
+- `F28.test-gated`: in create mode **Test** stays disabled until the draft is saved and unchanged since; editing any field after a save disables it again until the next **Save draft**.
+- `F28.draft-test`: **Test** on a saved, valid draft opens **Test runs** (with **Back**), shows `Test run started.` and a Pending row; it runs a hidden disabled copy of the draft, not a dashboard automation. Expected: the row reaches Successful or Failed by itself. Known failure #18263 (reproduced again 2026-10-14 on a Custom Python draft): the row stays Pending and the page does not poll the runs after the first request.
+- `F28.create`: **Create Automation** creates the automation enabled, navigates to `/automations/<id>` (a Custom Python automation shows **Script**), and removes the draft from the dashboard Drafts group.
+- `F28.error-contrast`: the draft bar's error pill and the Test runs error alert are readable in every color theme. Known failure (computed 2026-10-14 from the Light+ theme tokens and confirmed live the same day for the draft pill, not filed yet): both use raw `text-red-300` on `bg-red-500/10`, about 1.5:1 against Light+'s `#F3F3F3` surface; the theme's `--oh-feedback-error` gives 4.6:1. Dark themes read about 7.8:1.
+
+## How to get to it (user POV)
+
+- Dashboard header **Add Automation → Create Automation** (`automations-add-automation`, then `automations-add-automation-create`): blank create mode. Every create recipe below starts here.
+- Empty dashboard **Create Automation** (`automations-empty >> automations-create-automation`): the same blank form (`F21.empty-state`).
+- Dashboard Drafts group: name or **Resume** (`automation-setup-draft-resume-<draft-id>`) and the pencil (**Edit**, `automation-setup-draft-edit-<draft-id>`) open `?draftId=<id>`.
+- Kebab **Edit** on a dashboard card or the detail page header: edit mode (`?automationId=<id>`, `F21.edit`, `F23.edit-open`).
+- A direct URL with `draftId` or `automationId` (see the cold-load gotcha).
+
+## Driving it with control-openhands
+
+Preconditions:
+
+- Baseline state (`launch`, `doctor`, `onboard --skip`). No LLM is needed: every bullet uses the Custom Python kind or never dispatches, and the agent-assisted path stays in F22.
+- Desktop viewport; the `F28.error-contrast` bullet needs the Light+ theme (Settings → Application, `F16`), which it restores.
+- Every recipe below was run as written on 2026-10-14 on a fresh local `control-openhands launch` with Agent Server 1.54.0 and no LLM, along with `F23.edit-test` and `F23.edit-dirty-state` ([run results](https://github.com/OpenHands/OpenHands/pull/18452#issuecomment-6177650450)). The corrections from that run are included here; documented known failures remain failures.
+
+- **Blank form (`F28.open-blank`).** Run `control-openhands conversation list`, `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup$'`. Then `browser value 'testid=automation-setup-name'` is `New Automation`, `browser attr 'testid=automation-setup-name' placeholder` is `Standup digest`, `browser value 'testid=automation-setup-prompt'` is empty, `browser attr 'testid=automation-setup-frequency-daily' aria-checked` is `true`, `browser value 'testid=automation-setup-time'` is `09:00`, `browser text 'testid=automation-setup-save-state'` is `Unsaved changes` and `browser enabled 'testid=automation-setup-test'` is `false`. `conversation list` is unchanged.
+- **Presets (`F28.schedule-presets`).** For each of `once`, `hourly`, `daily`, `weekdays`, `weekly`, `custom` run `control-openhands browser click 'testid=automation-setup-frequency-<preset>'` and `control-openhands browser snapshot 'testid=automation-setup-at-row'`: Once has `automation-setup-datetime`, Hourly only `automation-setup-timezone`, Daily and Weekdays `automation-setup-time`, Weekly also `automation-setup-weekday` (value `1`), Custom `automation-setup-custom-schedule`. Click Weekly, then Custom: `browser value 'testid=automation-setup-custom-schedule'` is `0 9 * * *` (the default, not Weekly's `0 9 * * 1`). `browser count 'testid=automation-setup-timezone >> option'` is `13`. Click Daily again.
+- **Event fields (`F28.event-trigger`).** Run `control-openhands browser click 'role=radio[name=/^Event/]'`. `browser value 'testid=automation-setup-event-key'` is `issue_comment.created`, `browser visible 'testid=automation-setup-event-public-url-notice'` is `true`, and `browser count 'testid=automation-setup-custom-webhook-enabled'` is `0` for `github`. Click `testid=automation-setup-event-source-toggle`, `browser snapshot 'testid=automation-setup-event-source-menu'` lists the registered sources and `Custom` (GitHub, GitLab, Slack, Linear, Jira on a local stack). Click `testid=automation-setup-event-source-menu >> role=button[name="Custom"]`, `browser fill 'testid=automation-setup-event-source' qa-hooks`: `browser visible 'testid=automation-setup-custom-webhook-enabled'` is `true` and `browser eval "document.querySelector('[data-testid=automation-setup-custom-webhook-enabled]').checked"` is `true`. Pick GitHub again from the same menu. Click `testid=automation-setup-add-event-filter` (`automation-setup-event-filter` appears) and `testid=automation-setup-event-test-payload-toggle`; `browser value 'testid=automation-setup-event-test-payload'` is a JSON object with `"source": "github"` and `"event": "issue_comment.created"`. Click `role=radio[name=/^Schedule/]` to return.
+- **Invalid drafts save (`F28.draft-save`, `F28.draft-validation`).** Fill `testid=automation-setup-name` with `QA_F28 Probe`, leave the prompt empty, `browser network --clear`, click `testid=automation-setup-save-draft`. `browser wait-url 'draftId='`; `browser toasts --history` includes `Draft saved in this conversation.`; `browser text 'testid=automation-setup-draft-validity'` is `Field required`; `control-openhands api GET /api/automation/v1/drafts/<draft-id>` (the id in the URL) has `validation_errors` `[{"field":"prompt","code":"missing",…}]` and `dispatchable: false` (`browser network --bodies` shows request bodies, not the response); `browser text 'testid=automation-setup-save-state'` is `Saved just now`. Now fill the prompt with `Reply pong. Do not run any tools.`, click `testid=automation-setup-frequency-custom`, fill `testid=automation-setup-custom-schedule` with `every day`, press Tab, and `browser network --filter 'automation/v1'` lists no request (no blur validation). Click Save draft: the validity text is `Value error, Invalid cron expression: every day`. Fill the cron with `0 0 1 1 *` and Save draft: it reads `Ready to test`.
+- **Required fields before Test (`F28.required-before-test`).** Clear `testid=automation-setup-prompt`, Save draft, `browser network --clear`, click `testid=automation-setup-test`: `browser text 'testid=automation-setup-draft-runs-page'` contains `Add a prompt before continuing.` and `browser network --filter 'dispatch'` lists nothing. Click `testid=automation-setup-draft-runs-back`, fill the prompt back in and Save draft.
+- **Timeout (`F28.timeout`, `F28.draft-save-error`).** Click `testid=automation-setup-add-timeout`, fill `testid=automation-setup-timeout` with `1800`, Save draft: `Ready to test`. Fill `1801`, `browser toasts --history --clear`, `browser network --clear`, Save draft. `browser network --bodies --filter 'automation/v1/drafts'` shows `PATCH … 422` with `detail.errors[0].message` `Value error, timeout must not exceed 1800 seconds (30 minutes)`, and `browser text 'testid=automation-setup-save-state'` is `Save failed`. Expected: `browser toasts --history` shows that message. Known failure (reproduced twice 2026-10-14, not filed yet): the toast is `Draft body does not match endpoint schema` and the draft bar still reads `Ready to test`. Click `testid=automation-setup-timeout-remove` and Save draft.
+- **Test gating (`F28.test-gated`).** After a successful Save draft `browser enabled 'testid=automation-setup-test'` is `true`; fill `testid=automation-setup-name` with `QA_F28 Probe 2` and it is `false`; Save draft and it is `true`.
+- **Custom kind and draft Test (`F28.kind-toggle`, `F28.draft-test`).** Click `testid=automation-setup-kind-custom`: `browser count 'testid=automation-setup-prompt'` is `0` and `browser value 'testid=automation-setup-custom-code'` contains `AUTOMATION_CALLBACK_URL`. Click `testid=automation-setup-custom-details-toggle`: Entrypoint `python3 main.py`, Setup script path `setup.sh`. Save draft (`Ready to test`), click `testid=automation-setup-test` and `browser wait 'testid=automation-setup-draft-run'`: the row reads Pending and the page shows `Test run started.` Expected: `browser wait 'testid=automation-setup-draft-run >> text=Successful' --timeout 45000`. Known failure #18263 (reproduced 2026-10-14): the row stays Pending for 40 s with a single runs request in `browser network --filter '/runs'`. Click `testid=automation-setup-draft-runs-back`. Switching back with `testid=automation-setup-kind-prompt` restores the Prompt; leave the draft on Custom for the next bullet.
+- **Create (`F28.create`).** Run `control-openhands browser click 'testid=automation-setup-create' --expect-url '/automations/[0-9a-f-]{36}$'`. The detail h1 is `QA_F28 Probe 2`, the badge reads Active and the page shows **Script**. `browser goto /automations`: `browser count 'testid=automation-setup-draft-resume-<draft-id>'` is `0`.
+- **Error contrast (`F28.error-contrast`).** Switch to Light+ (`F16`), open a draft whose cron is `every day` (as in the invalid-drafts bullet), `browser screenshot 'testid=automation-setup-draft-details' --feature F28.error-contrast --name light-plus`, and read the colors with `browser eval "getComputedStyle(document.querySelector('[data-testid=automation-setup-draft-validity]')).color"`. Expected: at least 4.5:1 against the bar. Known failure (reproduced 2026-10-14, not filed yet): the color is `oklch(0.808 0.114 19.571)` (`text-red-300`) over `#F3F3F3` with a 10% red tint, 1.49:1. Restore the theme.
+- **Cleanup.** Delete `QA_F28 Probe 2` from its kebab and any leftover `QA_F28` draft with its trash icon, as in F21; `api GET /api/automation/v1/drafts` lists none.
+
+## Gotchas
+
+- A toast can cover the toolbar (Save draft, Test, Close) and stays while the pointer rests on it, so the next toolbar click times out ("subtree intercepts pointer events"). After each save, `browser hover` a form field (for example `testid=automation-setup-name`) before clicking the toolbar again, as F22's Gotchas say.
+- Drive inputs with `browser fill`: setting `.value` from `browser eval` does not reach React state for every field, and a save then sends the old value.
+- Change one field per Save draft when checking a validation message: a save shows only the first server error, so two edits in one save hide the second.
+- A cold load of `/automations/setup?draftId=<id>` or `?automationId=<id>` (a reload or a typed URL) opens the form: re-checked 2026-10-14 on a `control-openhands launch` of main with this family (both read their Name back). An earlier instance of unknown build stayed on **Loading saved draft…** for more than 20 s; if that returns, compare with opening the same draft from the dashboard.
+- The draft bar duplicates the prompt error: `Field required` appears both under Prompt and in the bar.
+- Kebab **Edit** on a Custom Python automation shows the code editor instead of the saved bundle summary: open bug #18341 (`F23`), with fixes in PRs #18353 and #18444.
+- Escape on the setup page does nothing, by design (`F23.edit-close`).
diff --git a/.agents/skills/verify-openhands/references/feature-map/README.md b/.agents/skills/verify-openhands/references/feature-map/README.md
index 60c0107822..03e0aa03db 100644
--- a/.agents/skills/verify-openhands/references/feature-map/README.md
+++ b/.agents/skills/verify-openhands/references/feature-map/README.md
@@ -251,13 +251,14 @@ How to write and prove new entries: [../mapping.md](../mapping.md).
| F20 | [Canvas apps](F20-canvas-apps.md) | install from path or git, enable, update, uninstall, extension pages | `/apps` | none | 24 |
| F21 | [Automations dashboard and actions](F21-automations-dashboard.md) | cards and list, filters, sort, Run now, enable, export, import, delete, pin | `/automations` | LLM; automation service | 35 |
| F22 | [Creating automations](F22-automation-creation.md) | templates, setup dialog, custom automations, import | `/automations/templates`, `/automations/new/<id>` | LLM | 29 |
-| F23 | [Automation detail, runs and editing](F23-automation-detail.md) | detail page, runs, logs, edit dialog, triggers, debug | `/automations/<id>` | LLM; runs | 39 |
+| F23 | [Automation detail, runs and editing](F23-automation-detail.md) | detail page, runs, logs, edit dialog, triggers, debug | `/automations/<id>` | LLM; runs | 41 |
| F24 | [Automation Git Sync](F24-git-sync.md) | configure, sync cycles, encryption, status | `/automations/git-sync` | `fixture git-repo` | 26 |
| F25 | [Backends, Cloud and sharing](F25-backends-and-cloud.md) | add, edit, remove and switch backends, per-backend consent, Cloud login, shared pages | backend selector | a second stack; Cloud account (blocked) | 33 |
| F26 | [Launcher modes, Docker, desktop and library](F26-runtime-variants.md) | launcher flags, partial stacks, LAN bind, Docker, Electron, embeddable library | a terminal | Docker/Electron where available | 33 |
| F27 | [Workspace tools](F27-workspace-tools.md) | terminal, browser, planner, task list, usage, `canvas_ui_control` | drawer tabs | LLM | 30 |
+| F28 | [Automation setup page](F28-automation-setup-page.md) | blank form, kind toggle, schedule presets, event fields, timeout, draft save and validation, draft Test, Create | **Add Automation → Create Automation**, `/automations/setup`, draft Resume, kebab Edit | automation service; no LLM | 13 |
-27 families, 760 sub-features. `control-openhands map ids` lists every ID with its file.
+28 families, 775 sub-features. `control-openhands map ids` lists every ID with its file.
### Neighbouring families
@@ -266,7 +267,7 @@ Several pages are shared. Each behavior has one owner; the others reference its
- Conversation page: the composer is F05, what the agent produces is F06, the header and its menus are F07, the drawer is F08 (files, changes) and F27 (tools).
- Starting work: Home is F03; the recommended-automation launcher is shared by F01 (onboarding), F03 (rail) and F22 (templates), and owned by F22.
- LLM settings: profiles are F10, provider connections F11 (same page), routers F12, agent profiles F13.
-- Automations: dashboard F21, creation F22, detail and runs F23, Git Sync F24.
+- Automations: dashboard F21, creation F22, detail and runs F23, Git Sync F24; the setup page's form, drafts and Test are F28 (F22 keeps the agent-assisted recipes, F23 owns edit mode).
- Customize: MCP F17, skills F18, plugins F19, apps F20; the hub and its navigation are in F17.
- Backends: adding and switching backends is F25; launcher flags that create them are F26.
@@ -279,7 +280,7 @@ Everything else a user can reach is mapped. These are left out on purpose:
- Suspended Cloud workspaces (`src/components/features/backends/cloud-organization-boundary.tsx`, [#17988](https://github.com/OpenHands/OpenHands/pull/17988)): **Mapped prerequisites/local absence:** [`F25.cloud-org-suspended`](F25-backends-and-cloud.md) records the expected alert and the prerequisite; a local backend can check only that the Cloud suspension surface is absent. **Positive paths not live-mapped:** the selected Cloud organization must receive a genuine scoped HTTP 403 with `detail: "Organization is suspended"` or `detail: "User membership is suspended"`; recovery also needs another accessible workspace. Desktop/phone alerts, workspace switching and request/backend/organization isolation remain blocked until those genuine states are available. An unrelated 403 is not that prerequisite.
- Enterprise Super Admin setup guide (`src/components/features/setup-guide`, [#17969](https://github.com/OpenHands/OpenHands/pull/17969)): **Mapped prerequisites/local absence:** [`F02.super-admin-setup-guide`](F02-app-shell.md) records its prerequisites and the local check for no guide or `/api/admin/setup-state` request. **Positive paths not live-mapped:** an authenticated enterprise Cloud backend must have a selected organization whose `/me` permissions include `manage_super_admins`, plus a genuine `/api/admin/setup-state` response with a non-null `guide_org_id`, `guide_dismissed: false`, and at least one unfinished required guide step. A local backend cannot establish those prerequisites. Progress, step destinations, collapse/reopen, server-driven guide removal, progress persistence and desktop/phone layouts remain blocked until an entitled account can drive them; source inspection is not runtime proof.
- Setup-guide continuation and Canvas tours ([#18138](https://github.com/OpenHands/OpenHands/pull/18138), `src/components/features/setup-guide/`, `src/components/features/manifest/manifest-setup-dialog.tsx`, `src/hooks/mutation/use-add-mcp-server.ts`) are **not live-mapped**. They share `F02.super-admin-setup-guide`'s genuine Cloud account and setup-state prerequisites above, with the next required step set to `first-automation` or `add-integration`. Verification additionally needs a successful direct automation creation or MCP-server addition and a subsequent server response confirming that step complete; assisted setup and out-of-order or unconfirmed completion have different control flow. The next Canvas step/tour, Cloud-host same-tab invitation handoff with `org` and `setup_tour`, standalone/Electron Start fallback, and tour entry via `setup_tour` remain blocked across desktop/phone and those deployment boundaries. The invitation handoff also needs an accessible enterprise invitation page for the guide organization. No positive recipe is accepted from source or mocked enterprise responses.
-- New automation setup boundaries ([#17199](https://github.com/OpenHands/OpenHands/pull/17199), [#17200](https://github.com/OpenHands/OpenHands/pull/17200), [#17201](https://github.com/OpenHands/OpenHands/pull/17201), [#17242](https://github.com/OpenHands/OpenHands/pull/17242), [#17688](https://github.com/OpenHands/OpenHands/pull/17688)): [`F22.agent-assisted-setup`](F22-automation-creation.md) maps credential-free draft save/reload/resume/custom-script tests and genuine `automation_form_update`, desktop Hide/Show, phone Form/Conversation, associated-draft resumption, final prompt/plugin creation and execution. These model paths were verified with real DeepSeek on Linux Local, SDK 1.54.0/automation 1.19.3, on 2026-10-10. Draft-Test completion is mapped with known failure #18263; ordinary final-automation polling passes. **Not yet live-mapped:** home Code/Automate handoff (`src/components/features/home/home-launcher-mode-toggle.tsx`), full dashboard draft management beyond the verified Resume and credential-free Delete paths (`src/routes/automations-list.tsx`), and the replacement existing-automation editor (`src/hooks/use-open-automation-editor.ts`). Editor migration needs a fresh literal user-path recipe; F21/F23's legacy Edit-modal recipes are not current live proof of that editor. The distinct template Plugin review/timezone path remains unverified by the agent-created plugin checks. This partial delta retains its baseline.
+- New automation setup boundaries ([#17199](https://github.com/OpenHands/OpenHands/pull/17199), [#17200](https://github.com/OpenHands/OpenHands/pull/17200), [#17201](https://github.com/OpenHands/OpenHands/pull/17201), [#17242](https://github.com/OpenHands/OpenHands/pull/17242), [#17688](https://github.com/OpenHands/OpenHands/pull/17688)): [`F22.agent-assisted-setup`](F22-automation-creation.md) maps credential-free draft save/reload/resume/custom-script tests and genuine `automation_form_update`, desktop Hide/Show, phone Form/Conversation, associated-draft resumption, final prompt/plugin creation and execution. These model paths were verified with real DeepSeek on Linux Local, SDK 1.54.0/automation 1.19.3, on 2026-10-10. Draft-Test completion is mapped with known failure #18263; ordinary final-automation polling passes. [`F28`](F28-automation-setup-page.md) maps the create-mode setup form, drafts and Test; [`F23`](F23-automation-detail.md) maps the existing-automation editor, including edit-mode Test and unsaved edits. Every F28 recipe and those two F23 rows were run on a fresh local `control-openhands launch` with Agent Server 1.54.0 and no LLM on 2026-10-14 ([run results](https://github.com/OpenHands/OpenHands/pull/18452#issuecomment-6177650450)). **Not yet live-mapped:** home Code/Automate handoff (`src/components/features/home/home-launcher-mode-toggle.tsx`) and full dashboard draft management beyond the verified Resume and credential-free Delete paths (`src/routes/automations-list.tsx`). The distinct template Plugin review/timezone path remains unverified by the agent-created plugin checks. This partial delta retains its baseline.
- Built-in Pi/OpenCode ([#17758](https://github.com/OpenHands/OpenHands/pull/17758), `src/constants/acp-providers.ts`, `src/constants/acp-brand-marks.ts`): **Not yet live-mapped:** the added local onboarding/profile presets, Pi credentials, multiline `OPENCODE_AUTH_CONTENT`, actual SDK commands/models and Cloud exclusion. Needs fresh local onboarding/profile drives and genuine ACP execution. Pi needs its corresponding credentials/login; OpenCode credentials are optional (the SDK provides a keyless default), so absence of credentials alone does not block its preset. Provider authorization and bounded execution still apply. Cloud exclusion/reset needs a genuine Cloud backend. Source review and the existing Claude/Codex/Gemini recipes do not verify these additions.
- Cloud organization handback ([#18228](https://github.com/OpenHands/OpenHands/pull/18228), `src/api/backend-registry/url-selection.ts`): **Not yet live-mapped:** locked Cloud `?org=` handback must select the returned accessible organization and consume only that parameter. Needs a genuine locked Cloud deployment, an authenticated account with multiple accessible organizations and the actual web-app handoff. A local backend cannot supply that state.
- Locked-to-Cloud deployments: `scripts/static-server.mjs --lock-to-cloud` exists, but `bin/agent-canvas.mjs` does not forward it, so `control-openhands launch` cannot start that mode.