🙌 OpenHands: AI-Driven Development
docs(verify-openhands): map maintenance 2026-10-17 (#18590)
Engel Nyst7cc8716b77e7
diff --git a/.agents/skills/verify-openhands/references/feature-map/F03-home.md b/.agents/skills/verify-openhands/references/feature-map/F03-home.mdindex 27eb406ffb..1ad0a30588 100644--- a/.agents/skills/verify-openhands/references/feature-map/F03-home.md+++ b/.agents/skills/verify-openhands/references/feature-map/F03-home.md@@ -168,7 +168,7 @@ Phone and cleanup:- Known issue [#16801](https://github.com/OpenHands/OpenHands/issues/16801) (`F03.launch-without-workspace`): Agent starts by looking in workspace, even when started with "no workspace". Live prerequisite: A working Claude Opus agent and the original task/prompt that makes it explore the default workspace; the prompt is absent.- Known issue [#16210](https://github.com/OpenHands/OpenHands/issues/16210) (`F03.plugin-picker`): [Cloud Backends] Plugins not available when starting a new conversation. Live prerequisite: An authenticated Cloud organization with configured account plugins before opening New Chat > Plugins.- Known issue [#15701](https://github.com/OpenHands/OpenHands/issues/15701) (`F03.launch-in-workspace`): Creating Conversation toast stuck visible. Live prerequisite: The original stuck creation request/outcome sequence; the report is intermittent and gives no deterministic trigger.-- Known issue [#18480](https://github.com/OpenHands/OpenHands/issues/18480) (`F03.open-repository-dialog`): an Azure DevOps Enterprise user belonging to several organizations can see only one organization's repositories. Live prerequisite: a real Cloud/Enterprise sign-in with two organizations and distinct repositories; the report's mocked two-organization capture is not a real-backend reproduction.+- `F03.open-repository-dialog` on a Cloud backend lists Azure DevOps repositories the way it lists GitHub ones: it asks for the user's organizations once (`/api/v1/git/installations/search`), then pages through each organization's repositories by `installation_id`, so a user in several organizations sees every organization's repositories; typed search goes to `/api/v1/git/repositories/search` without an `installation_id`, so what it finds is up to the Cloud server (`shouldUseInstallationRepos` in `src/utils/utils.ts`; a local backend keeps the direct search flow). Source-only, not driven live: needs an OpenHands Cloud/Enterprise sign-in with Azure DevOps access to two organizations with distinct repositories.- Known issue [#18418](https://github.com/OpenHands/OpenHands/issues/18418) (`F03.create-error-toast`, `F03.launch-with-plugin`): hosted runtime 1.50.1 can fail during skills/plugins setup because its packaged plugin-schema resource is missing. Live prerequisite: the affected Cloud runtime/PyInstaller image and original plugin setup plus its account. A Local 1.55.0 launch does not verify that deployment.- Known issue [OpenHands/software-agent-sdk#5837](https://github.com/OpenHands/software-agent-sdk/issues/5837) (`F03.folder-browser`): case-only directory names at a page boundary can repeat the server cursor; Canvas detects the repeated cursor and the directory listing fails instead of showing all choices. Live prerequisite: a case-sensitive filesystem with enough directories to cross the real picker page boundary, including two distinct case-only names. A case-insensitive Mac volume cannot hold that fixture.- Known issue [OpenHands/software-agent-sdk#5849](https://github.com/OpenHands/software-agent-sdk/issues/5849) (`F03.folder-browser`, `F03.manage-workspaces`): re-adding a saved workspace or removing an already-removed stale row can return 500 because the registry writes even for a no-op. Live prerequisite: an owned read-only workspace registry plus the already-saved or stale picker row; changes to registry mtime/inode alone are not a Canvas failure.diff --git a/.agents/skills/verify-openhands/references/feature-map/F05-composer.md b/.agents/skills/verify-openhands/references/feature-map/F05-composer.mdindex 3110085470..c124fff0b0 100644--- a/.agents/skills/verify-openhands/references/feature-map/F05-composer.md+++ b/.agents/skills/verify-openhands/references/feature-map/F05-composer.md@@ -14,7 +14,7 @@ Source: `src/components/features/chat/`, `src/components/features/controls/tools- `F05.send-message`: Send is disabled while the field is empty; Shift+Enter inserts a newline; Enter sends, clears the field and the message persists after reload.- `F05.draft-persistence`: unsent text is restored per conversation (after navigation and reload) and on the home composer.-- `F05.composer-resize`: the field grows with its content and shrinks when cleared; a top-edge grip lets the user drag the height.+- `F05.composer-resize`: the field grows with its content and shrinks when cleared; an invisible top-edge grip (resize cursor, no line drawn) lets the user drag the height.- `F05.plus-menu`: `+` opens a tools menu (Macros, Show skills/hooks/agent tools, Add Files and Images; Git tools in a repository); a macro fills the composer without sending.- `F05.agent-profile-switch`: before a conversation starts, `+` > Switch agent profile lists agent profiles and links to Manage agent profiles; it is absent in a started conversation.- `F05.attach-files`: picked files show as chips and images as thumbnails (with an Upload-as-file toggle), each removable; sent files land in the workspace and are named in the message.@@ -58,9 +58,9 @@ Preconditions:- Desktop viewport unless a bullet says otherwise. Microphone dictation is blocked in the sandbox browser (no microphone); `+` > Git tools needs a conversation on a git workspace (not driven here).- **Type and send (`F05.send-message`).** On `/conversations/<id>` run `control-openhands browser attr 'testid=submit-button' aria-label` (`Send message`), `control-openhands browser enabled 'testid=submit-button'` (`false`), `control-openhands browser type 'testid=chat-input' 'Reply with only:'`, `control-openhands browser enabled 'testid=submit-button'` (`true`), `control-openhands browser press Shift+Enter`, `control-openhands browser type 'testid=chat-input' 'ok2'` and `control-openhands browser eval "document.querySelector('[data-testid=chat-input]').innerText"`: `Reply with only:\nok2`. Run `control-openhands browser press Enter --selector 'testid=chat-input'`; the field is empty and `control-openhands browser count 'testid=stop-button'` is `1` while the agent runs. Then `control-openhands conversation wait <id> --fresh --timeout 180` (a fast model can finish before a separate `--until running` wait starts, which then times out on `finished`), `control-openhands browser reload` and `control-openhands browser count 'testid=user-message >> has-text=ok2'`: `1`; `control-openhands browser text 'testid=agent-message >> nth=-1'` is `ok2`.-- **Draft survives navigation (`F05.draft-persistence`).** Run `control-openhands browser type 'testid=chat-input' 'QA draft text'` and wait out the 500 ms save: `control-openhands browser eval "new Promise(r=>setTimeout(()=>r(JSON.parse(localStorage.getItem('conversation-state-<id>')).draftMessage),1500))"` returns `QA draft text`. Then `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url settings`, `control-openhands browser back` and the `innerText` eval above: `QA draft text`. `control-openhands browser reload` restores it again. A draft typed just before leaving is flushed, not lost: `control-openhands browser type 'testid=chat-input' ' plus fast'` and at once `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url settings`, `control-openhands browser back`: the eval reads `QA draft text plus fast`. The same holds for an immediate sidebar switch to another conversation and back (that other conversation's composer stays empty). Clear with `control-openhands browser fill 'testid=chat-input' ''` and wait a second before navigating. A slash command the chat handles itself (`/model`, `/new`) clears the saved draft with the composer. Known failure (reproduced 2026-10-16, [#18552](https://github.com/OpenHands/OpenHands/issues/18552)): after `browser type 'testid=chat-input' '/model'`, a pause of 1 second, `browser press Escape` and `browser press Enter --selector 'testid=chat-input'`, leaving and returning shows `/model` in the composer again (the same for `/new`).+- **Draft survives navigation (`F05.draft-persistence`).** Run `control-openhands browser type 'testid=chat-input' 'QA draft text'` and wait out the 500 ms save: `control-openhands browser eval "new Promise(r=>setTimeout(()=>r(JSON.parse(localStorage.getItem('conversation-state-<id>')).draftMessage),1500))"` returns `QA draft text`. Then `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url settings`, `control-openhands browser back` and the `innerText` eval above: `QA draft text`. `control-openhands browser reload` restores it again. A draft typed just before leaving is flushed, not lost: `control-openhands browser type 'testid=chat-input' ' plus fast'` and at once `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url settings`, `control-openhands browser back`: the eval reads `QA draft text plus fast`. The same holds for an immediate sidebar switch to another conversation and back (that other conversation's composer stays empty). Clear with `control-openhands browser fill 'testid=chat-input' ''` and wait a second before navigating. A slash command the chat handles itself (`/model`, `/new`) clears the saved draft with the composer: after `control-openhands browser type 'testid=chat-input' '/model'`, a pause of 1 second, `control-openhands browser press Escape` and `control-openhands browser press Enter --selector 'testid=chat-input'`, the Settings round trip above brings back an empty composer and the `draftMessage` eval returns `null`. `/new` the same way opens a new conversation (toast `Starting a new conversation.`); `control-openhands browser goto /conversations/<id>` shows the source conversation's composer empty and its `draftMessage` `null`.- **Home draft (`F05.draft-persistence`).** Run `control-openhands browser goto /`, `control-openhands browser type 'testid=chat-input' 'QA home draft'`, `control-openhands browser goto /settings/secrets`, `control-openhands browser goto /`, then the `innerText` eval: `QA home draft`; `control-openhands browser storage --session` lists `oh:home-prompt-draft`. The automation setup page's docked composer (`/automations/setup`, F22) keeps no draft and never touches the Home one. With `QA home draft` still in the Home composer, run `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'`, `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'` and `control-openhands browser wait 'testid=chat-input'`: the `innerText` eval is empty. `control-openhands browser type 'testid=chat-input' 'Reply with only the word ok2. Do not run any tools.'`, then `control-openhands browser eval "new Promise(r=>setTimeout(()=>r(sessionStorage.getItem('oh:home-prompt-draft')),1500))"` still returns `QA home draft`. `control-openhands browser click 'testid=submit-button' --expect-url '/conversations/'` and `control-openhands browser click 'testid=sidebar-conversations-link' --expect-url '/conversations(\?|$)'`: the Home composer shows `QA home draft`. Clear it (`fill 'testid=chat-input' ''`, wait a second) and repeat the setup send: New Chat is empty and `browser storage --session --values` has no `oh:home-prompt-draft`. Unsent text in the setup composer is lost when the user leaves that page (by design).-- **Auto-resize (`F05.composer-resize`).** On `/conversations/<id>` (`control-openhands browser goto /conversations/<id>`; Home draft left `/`) type `line`, then eight times `control-openhands browser press Shift+Enter` and `control-openhands browser type 'testid=chat-input' 'lineN'` (`line2`…`line9`): `control-openhands browser bbox 'testid=chat-input'` grows by 20 px per line, from `height` `20` to `180`; after `control-openhands browser fill 'testid=chat-input' ''` it is `20`. `control-openhands browser count '#resize-grip'` is `1`. `control-openhands browser drag '#resize-grip' --by 0,-200` makes the empty field `220` high and `control-openhands browser drag '#resize-grip' --by 0,150` brings it to `70`; after `control-openhands browser reload` it is `20` again (the dragged height is not kept).+- **Auto-resize (`F05.composer-resize`).** On `/conversations/<id>` (`control-openhands browser goto /conversations/<id>`; Home draft left `/`) type `line`, then eight times `control-openhands browser press Shift+Enter` and `control-openhands browser type 'testid=chat-input' 'lineN'` (`line2`…`line9`): `control-openhands browser bbox 'testid=chat-input'` grows by 20 px per line, from `height` `20` to `180`; after `control-openhands browser fill 'testid=chat-input' ''` it is `20`. `control-openhands browser count '#resize-grip'` is `1`. The grip is an invisible 12 px strip with a resize cursor: `control-openhands browser eval "(()=>{const g=document.getElementById('resize-grip');return [g.children.length,getComputedStyle(g).cursor,getComputedStyle(g).backgroundColor].join()})()"` is `0,ns-resize,rgba(0, 0, 0, 0)`, and no line appears above the field on `browser hover '#resize-grip'`, during a drag or after `browser click '#resize-grip'` (screenshots). `control-openhands browser drag '#resize-grip' --by 0,-200` makes the empty field `220` high and `control-openhands browser drag '#resize-grip' --by 0,150` brings it to `70`; after `control-openhands browser reload` it is `20` again (the dragged height is not kept).- **Tools menu and macros (`F05.plus-menu`).** On `/conversations/<id>` run `control-openhands browser click 'testid=chat-plus-button'` and `control-openhands browser testids 'testid=tools-context-menu'`: `macros-button`, `show-skills-button`, `show-hooks-button`, `show-agent-tools-button`, `add-files-and-images-button` (no `git-tools-button` outside a repository). `control-openhands browser hover 'testid=macros-button'` shows `macros-submenu` with `increase-test-coverage-button`, `fix-readme-button`, `auto-merge-prs-button`, `clean-dependencies-button`. `control-openhands browser click 'testid=fix-readme-button'` closes the menu and fills the composer with `Please look at the README and make the following improvements…`; the `testid=user-message` count is unchanged. Clear with `control-openhands browser fill 'testid=chat-input' ''`. Open the menu again and `control-openhands browser press Escape`: `browser count 'testid=tools-context-menu'` is `0` and `control-openhands browser eval "document.activeElement.dataset.testid"` is `chat-plus-button`.- **Show dialogs (`F05.plus-menu`).** For each of `show-skills-button`, `show-hooks-button`, `show-agent-tools-button`: `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser click 'testid=show-hooks-button'` (etc.), `control-openhands browser text 'role=dialog'` (starts `Available Skills`, `Available Hooks`, `Agent Tools & Metadata`), then `control-openhands browser press Escape` and `control-openhands browser count 'role=dialog'`: `0`.- **Agent profile before start (`F05.agent-profile-switch`).** Run `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser hover 'testid=switch-agent-profile-button'` and `control-openhands browser text 'testid=agent-profile-submenu'`: `AVAILABLE PROFILES`, `default` (checked; `chat-input-agent-profile-option-default`), `Manage agent profiles`. `control-openhands browser click 'testid=agent-profile-submenu >> text=Manage agent profiles' --expect-url 'settings/agents'` lands on `/settings/agents`. In a started conversation (`control-openhands browser goto /conversations/<id>`) `control-openhands browser click 'testid=chat-plus-button'` then `control-openhands browser count 'testid=switch-agent-profile-button'` is `0`.@@ -100,7 +100,7 @@ Preconditions:- **`/goal` (`F05.slash-goal`).** Bare `/goal` + Enter: `control-openhands browser toasts` shows `Please provide an objective — e.g. /goal <objective>`. Send `/goal --max 1 Reply with only: goal-ok`: `control-openhands browser text 'testid=goal-status'` reads `Goal: … Round 0/1 running Stop`; `control-openhands browser wait 'testid=goal-done' --timeout 150000`, then `Round 1/1 complete score 100%`. Send `/goal --max 2 Reply with only: goal-two` and at once `control-openhands browser click 'testid=goal-status >> nth=-1 >> testid=goal-stop'`; `control-openhands browser wait 'testid=goal-resume'` and the banner reads `Round 0/2 interrupted Resume`. `control-openhands browser click 'testid=goal-resume'` sets it `running`; poll `control-openhands browser text 'testid=goal-status >> nth=-1'` until a new row ends `Round 1/2 complete score 100%`. The old `Round 0/2 interrupted` row no longer offers Resume: `control-openhands browser count 'testid=goal-resume'` is `0`. After `control-openhands browser reload`, `control-openhands browser count 'testid=goal-status'` is `3` and the `goal-resume` count is still `0`.- **`/plan` and `/code` (`F05.slash-plan-code`).** Send bare `/plan` (Escape, Enter): `control-openhands browser attr 'testid=chat-input' data-placeholder` turns `Let’s work on a plan`; `control-openhands conversation list` gains a planner conversation (title `null` at first, `<planner-id>`). Bare `/code` turns it back to `What do you want to build?`. `control-openhands browser count 'role=button[name="Code"]'` is `0` on local (the pill is Cloud only). `/plan` sent while the agent runs should be refused with feedback; today it is dropped silently: the field clears, no toast, the placeholder stays `What do you want to build?` (fail row; `use-plan-mode-interceptor.ts` swallows it by design but tells the user nothing).- **Plan card (`F05.plan-preview`).** In plan mode send `Plan creating a file hello.txt containing hi. Keep the plan to three short steps.`, then `control-openhands conversation wait <planner-id> --until running --timeout 60`, `control-openhands conversation wait <planner-id> --timeout 240` and `control-openhands browser wait 'testid=plan-preview-content' --timeout 60000`. `control-openhands browser text 'testid=plan-preview-content'` starts `1. OBJECTIVE` and ends `Read more`; `control-openhands browser enabled 'testid=plan-preview-build-button'` is `true`. Use `--fresh` instead of the `--until running` wait if the planner is fast. `control-openhands browser click 'testid=plan-preview-view-button'` (Read more does the same) opens the Planner tab: `control-openhands browser testids --filter planner` lists `conversation-tab-planner` and `planner-tab-build-button` (`Build ⌘↩`).-- **Build shortcut (`F05.build-plan-shortcut`).** With the plan card shown and the agent idle, `control-openhands browser press Control+Enter`: the placeholder returns to `What do you want to build?`, `control-openhands conversation wait <id> --until running --timeout 60` then `--timeout 240` finishes, `control-openhands browser text 'testid=user-message >> nth=-1'` is `Execute the plan based on the .agents_tmp/PLAN.md file.` (the agent's reply wording varies, e.g. `Executed the plan in .agents_tmp/PLAN.md`), and `hello.txt` exists in the `workspace` from `control-openhands conversation status <id>`. The same holds with focus in the empty composer (`control-openhands browser focus 'testid=chat-input'`, then `browser press Control+Enter --selector 'testid=chat-input'`), where focus sits after sending `/plan`. Known failure (reproduced 2026-10-16, [#18543](https://github.com/OpenHands/OpenHands/issues/18543)): from the focused composer nothing happens; only a body-focused press (`browser mouse-click 700 300` first) builds.+- **Build shortcut (`F05.build-plan-shortcut`).** With the plan card shown and the agent idle, `control-openhands browser press Control+Enter`: the placeholder returns to `What do you want to build?`, `control-openhands conversation wait <id> --until running --timeout 60` then `--timeout 240` finishes, `control-openhands browser text 'testid=user-message >> nth=-1'` is `Execute the plan based on the .agents_tmp/PLAN.md file.` (the agent's reply wording varies, e.g. `Executed the plan in .agents_tmp/PLAN.md`), and `hello.txt` exists in the `workspace` from `control-openhands conversation status <id>`. The same holds with focus in the empty composer (`control-openhands browser focus 'testid=chat-input'`, then `control-openhands browser press Control+Enter --selector 'testid=chat-input'`), where focus sits after sending `/plan`. With text in the composer Ctrl/Cmd+Enter sends that text instead of building: type `Reply with only: noted. Do not change the plan.` and `control-openhands browser press Control+Enter --selector 'testid=chat-input'`; the placeholder stays `Let’s work on a plan`, and after `control-openhands conversation wait <planner-id> --fresh --timeout 180` the planner's last MessageEvents are that text and `noted.`.- **`/plan <task>` and `/code <task>` (`F05.slash-plan-code-task`).** With the agent idle, send bare `/plan`, then `/code Reply with only: code-ok` (Escape, Enter): the placeholder turns `What do you want to build?` at once, `control-openhands conversation wait <id> --fresh --timeout 180`, `control-openhands browser text 'testid=user-message >> nth=-1'` is `Reply with only: code-ok` (no `/code` prefix) and the last agent message is `code-ok`. Then send `/plan Plan writing bye.txt containing bye. One step only.`: the placeholder turns `Let’s work on a plan`, `control-openhands conversation wait <planner-id> --fresh --timeout 240`, `control-openhands conversation events <planner-id> --kinds MessageEvent --last 100` ends with the user text `Plan writing bye.txt containing bye. One step only.` and a planner reply, and `control-openhands conversation list` still has only the two conversations (the existing planner is reused).- **Build button (`F05.plan-preview`).** With the newer plan card from `/plan Plan writing bye.txt containing bye. One step only.` (above) shown and the agent idle, `control-openhands browser click 'testid=plan-preview-build-button >> nth=-1'` does the same as the shortcut: placeholder `What do you want to build?`, `control-openhands conversation wait <id> --fresh --timeout 240`, the same last user message, and `bye.txt` (content `bye`) in the workspace.- **Phone (`F05.phone`).** On `/conversations/<id>` run `control-openhands browser viewport phone`, `control-openhands browser bbox 'testid=interactive-chat-box'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F05.phone --name composer` (`+`, model pill, ring, mic and Send on one row). Return with `control-openhands browser viewport desktop`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md b/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.mdindex b7efe858e4..70a57b4c53 100644--- a/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md+++ b/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md@@ -22,7 +22,7 @@ Source: `src/components/features/chat/chat-interface.tsx`, `src/components/conve- `F06.pending-messages`: a sent message shows "Sending..." until the server echoes it; a failed send shows Failed to send with Retry and Dismiss; Retry delivers it exactly once.- `F06.clock-skew-send`: a message sent while the browser clock runs ahead of the server is still confirmed by the server echo: no lingering "Sending..." and no Retry.- `F06.failed-send-persists`: a failed send keeps its Retry row after the backend returns and after leaving and reopening the conversation from the sidebar; sending the same text again succeeds without clearing it, and Retry then delivers that attempt exactly once.-- `F06.markdown-rendering`: agent replies render markdown: tables inside a horizontal scroller with edge fades, fenced code, inline code.+- `F06.markdown-rendering`: agent replies render markdown: tables inside a horizontal scroller with edge fades, fenced code (highlighted tokens stay inline, so a TOML `[build-system]` header is one line), inline code, and links that are always underlined in the theme's link colour.- `F06.streamed-reply`: a streamed agent reply types out at a steady reading pace (about 60 characters a second, faster only when it would otherwise trail the model by more than about 10 seconds), each new word fading in; it keeps typing to the end after the model has finished, then settles into the finished markdown (tables, code blocks) with no per-word spans. While it types, the transcript stays pinned to the bottom; scrolling up stops the follow without pulling the user back, and returning to the bottom (button or scroll) resumes it. With reduced motion the words appear without the fade.- `F06.workspace-path-links`: a workspace file path in an agent reply, and the path chip of a file-editor card, open that file in the Files tab.- `F06.thinking`: model reasoning appears as a collapsed "Thinking" row that expands.@@ -67,7 +67,7 @@ Preconditions:- Baseline state (launched, doctored, `onboard --skip` done) and `control-openhands llm preset deepseek` (deepseek-flash active). Every prompt below is tiny and stays in the conversation's own workspace.- Conversation ids are printed as `"id"` by `control-openhands conversation start`, or read from `control-openhands browser url` (`/conversations/<id>`). `<id>` below is always the conversation the step just created or opened.- `F06.llm-not-configured-banner` needs a second, fresh `control-openhands launch --new --build never` with no `llm preset` (export its run dir as `OH_VERIFY_RUN` for those commands, then `control-openhands stop` it).-- `F06.critic-result`: if an authorized OpenHands (`openhands/…`) or compatible OpenHands LLM-proxy profile is available, use it for the critic check, regardless of the profile used for the rest of the pass. The SDK reuses that profile's LLM key when Critic API Key is empty; a separate critic key is not required. For a non-default proxy deployment, use its compatible critic endpoint. Restore the previous profile after the check. Record `blocked` only when no authorized working critic configuration is available, not merely because the main profile uses another provider. The automatic-fetch half of `F06.load-older-history` (its 16-command conversation) needs a model; the scroll-to-top half is driven without one, on the dummy profile below, with thirty refused user messages. `F06.hook-events` needs a configured hook; none is set in a fresh run.+- `F06.critic-result`: if an authorized OpenHands (`openhands/…`) or compatible OpenHands LLM-proxy profile is available, use it for the critic check, regardless of the profile used for the rest of the pass. The SDK reuses that profile's LLM key when Critic API Key is empty; a separate critic key is not required. For a non-default proxy deployment, use its compatible critic endpoint. Restore the previous profile after the check. Record `blocked` only when no authorized working critic configuration is available, not merely because the main profile uses another provider; its bullet also says how to arrange a locally scored block with a model only. The automatic-fetch half of `F06.load-older-history` (its 16-command conversation) needs a model; the scroll-to-top half is driven without one, on the dummy profile below, with thirty refused user messages. `F06.hook-events` needs a configured hook; none is set in a fresh run.- `F06.clock-skew-send` and `F06.failed-send-persists` need no model reply: a conversation that exists is enough (without a key the agent errors at once, and the user rows still land). Without a key, create the family's conversations with a dummy profile (`QA_DUMMY_KEY=qa control-openhands llm set --profile qa-zeta --model openai/gpt-4o --api-key-env QA_DUMMY_KEY --no-validate`) and `control-openhands conversation start --prompt ...` without `--wait`. `F06.corrective-nudge` is blocked: the nudge needs a model turn with neither text nor a tool call, which a normal prompt to deepseek-flash does not produce. When one happens, `control-openhands browser count 'testid=corrective-nudge-message'` is `1` and `browser count 'testid=user-message >> has-text=Your last response did not include'` is `0`.- The harness browser grants clipboard access: `control-openhands browser clipboard` reads what a Copy button wrote and `browser clipboard --write qa-empty` resets it first.- A terminal call can come back as "The terminal session was reset..." after this run's agent server restarts; the model reruns it, which turns one action into a group of two (see Gotchas).@@ -80,6 +80,7 @@ Preconditions:- **Copy a message and a code block (`F06.message-copy`, `F06.code-block-copy`).** On the same page run `control-openhands browser clipboard --write qa-empty`, `control-openhands browser hover 'testid=agent-message >> nth=0'` and `control-openhands browser click 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0'` (the bubble's button; the code block has a second, hidden `copy-to-clipboard`). `control-openhands browser attr 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0' aria-label` is `Copied to clipboard`, and `Copy to clipboard` again about 2 s later; `control-openhands browser clipboard` starts with `| Step | Result |` and ends with the fenced `print(1)` block. Then `control-openhands browser hover 'testid=agent-message >> pre'`, `control-openhands browser click 'testid=agent-message >> pre >> testid=copy-to-clipboard'` and `control-openhands browser clipboard` is exactly `print(1)`.- **Streamed reply (`F06.streamed-reply`).** From `/` run `control-openhands browser record start --feature F06.streamed-reply --name reveal`, fill `testid=chat-input` with `Write a markdown answer of about 300 words explaining Python virtual environments. Include a level-2 heading, a bulleted list of 4 items, a python code block of about 6 lines, and a table with 3 columns and 3 rows. Do not run any tools.` and `control-openhands browser click 'testid=submit-button' --expect-url '/conversations/'`. While it types, `control-openhands browser count 'testid=agent-message >> .stream-word-fade'` is at most about 8 (0 while the reveal waits for the model), `browser count 'testid=agent-message >> pre >> [data-stream-word]'` is `0`, and the distance from the bottom (`browser eval "(()=>{const e=document.querySelector('[data-testid=chat-scroll-container]');return Math.round(e.scrollHeight-e.scrollTop-e.clientHeight)})()"`) stays `0` to `1`. The bubble keeps growing after `conversation events <id> --kinds MessageEvent` already lists the agent reply (about 20 seconds for 1,600 characters). Once done, `browser count 'testid=agent-message >> [data-stream-word]'` is `0`, the message holds one `table` and one `pre`, and its text is unchanged after `browser reload`. In a longer follow-up (about 800 words), run `control-openhands browser scroll 'testid=chat-scroll-container' --by -700` mid-stream: the distance grows as the text grows, `count 'testid=scroll-to-bottom'` is `1`, and `browser click 'testid=scroll-to-bottom'` returns it to `0` while the reveal continues. Finish with `control-openhands browser record stop --gif`. Reduced motion has no emulation verb: `grep -o '@media (prefers-reduced-motion:reduce){\[data-agent-server-ui\] .stream-word-fade{animation:none}' build/assets/root-*.css` finds the rule.- **Long table cell (`F06.markdown-rendering`).** On the stream conversation write a prompt file whose first line is `Reply with only the word OK. Do not run any tools.`, then a blank line and the table `| Name | Description | Status |`, `|---|---|---|`, `| Alpha | This is a really long sentence in a single cell that goes on and on to explain a very involved aspect of the system, which nobody could read on one line without scrolling forever to the right. | ok |`, `| Beta | brief | fail |`. Run `control-openhands browser fill 'testid=chat-input' x --value-file <file>`, `control-openhands browser click 'testid=submit-button'` and `control-openhands conversation wait <id> --fresh --timeout 120`; the user bubble renders the table. Cells wrap once they reach 28rem (`max-w-md`): `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).whiteSpace + ' ' + getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).maxWidth"` is `normal 448px`. At desktop with the right panel closed, `control-openhands browser bbox 'testid=user-message >> nth=-1 >> testid=markdown-table-scroll'` has `scrollWidth` `582` equal to `clientWidth` and `Status` is visible (`browser screenshot 'testid=chat-interface' --feature F06.markdown-rendering --name long-cell-table`). A table still wider than the conversation keeps the horizontal scroll with its edge fade: with the panel open (`control-openhands browser click 'testid=right-panel-toggle'`) `clientWidth` is `474`, and at phone width `326`, so `Status` is scrolled off there while the long cell stays wrapped. An agent reply with such a table behaves the same.+- **Links and TOML (`F06.markdown-rendering`).** Run `control-openhands conversation start --prompt "Write a short markdown answer on Python packaging with pyproject.toml. Start with the sentence: See [QA docs](https://example.com/qa) for the verification page. Include one toml code block with [build-system] and [project] sections. Do not run any tools." --wait --timeout 200`. `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=agent-message] .token.table')).display"` is `inline`, and each section header (`[build-system]`, `[project]`) is one line in `control-openhands browser screenshot 'testid=agent-message' --feature F06.markdown-rendering --name toml-and-link`. The link `testid=agent-message >> a[href="https://example.com/qa"]` has `text-decoration-line` `underline` before any hover. Send the same `See [QA docs](https://example.com/qa) ...` sentence as a follow-up to see it in a user bubble. In Light+ (F16.color-theme) the link is `rgb(0, 90, 153)`: 7.19:1 on the chat panel, 5.70:1 in the user bubble; in Solarized Light `rgb(0, 90, 136)`: 6.91:1 and 5.60:1 (contrast from the computed `color` and the nearest opaque background, read with `browser eval`). The same holds at phone width.- **Non-zero exit badge (`F06.tool-visualizers`).** In an open conversation run `control-openhands browser fill 'testid=chat-input' "Run exactly this terminal command, verbatim, with nothing appended: ls /qa_missing_dir2 . Then reply OK."`, `control-openhands browser click 'testid=submit-button'` and `control-openhands conversation wait <id> --fresh --timeout 150` (without `--fresh` it returns the previous run's `finished` at once). If the reply's actions render as a single row, run `control-openhands browser click 'testid=chat-scroll-container >> role=button[name="Expand"][exact] >> nth=-1'`; if they render as a group (`browser count 'testid=event-group'` grew, e.g. `2 actions completed` after a terminal reset and rerun), run `control-openhands browser click 'testid=event-group-toggle >> nth=-1'` and `control-openhands browser click 'testid=event-group-content >> nth=-1 >> role=button[name="Expand"][exact] >> nth=-1'`. Then `control-openhands browser count 'testid=chat-scroll-container >> text="exit 2"'` is `1`; the badge sits above the `No such file or directory` output (`browser screenshot 'testid=event-group >> nth=-1' --feature F06.tool-visualizers --name exit-code`). Repeating the check in the same conversation needs a new directory name and a count that grows, because an earlier expanded badge stays on the page.- **Task list (`F06.task-list`, `F06.status-indicator`).** From `/` run `control-openhands browser goto /`, `control-openhands browser fill 'testid=chat-input' "Use your task_tracker tool to plan exactly two tasks titled 'qa step one' and 'qa step two'; mark 'qa step one' done and 'qa step two' todo. Do nothing else and reply DONE."` and `control-openhands browser click 'testid=submit-button' --expect-url '/conversations/' --observe '[data-testid=chat-status-indicator]' --observe-ms 8000`. `observed` contains `Starting` for a moment (the pill). Then `control-openhands conversation wait <id> --timeout 150` and `control-openhands browser text 'testid=chat-scroll-container'`: it contains `Tasks`, `qa step one`, `qa step two`; the screenshot shows a check icon and muted text for the done task, an empty circle for the todo one.- **Empty state (`F06.empty-state-suggestions`).** Run `control-openhands browser click 'testid=conversation-panel-new-thread-picker'`, `control-openhands browser click 'testid=launch-no-workspace' --expect-url '/conversations/'`, then `control-openhands browser text 'testid=chat-suggestions'`: `Let's start building!` plus `Increase test coverage`, `Auto-merge PRs`, `Fix README`, `Clean dependencies`. Run `control-openhands browser click 'testid=chat-suggestions >> text=Fix README'` and `control-openhands browser text 'testid=chat-input'`: the composer holds the README-improvement prompt.@@ -90,7 +91,7 @@ Preconditions:- **Failed send survives leaving (`F06.failed-send-persists`).** With `/conversations/<id>` open and healthy, run `control-openhands service stop agent-server`, `control-openhands browser wait 'testid=error-message-banner' --timeout 20000`, `control-openhands browser fill 'testid=chat-input' 'Reply with only: retry-ok'` and `control-openhands browser click 'testid=submit-button' --observe '[data-testid=chat-message-sending],[data-testid=chat-message-error]' --observe-ms 8000`: `Sending...` turns into `Failed to send Retry Dismiss` and `control-openhands browser count 'testid=chat-message-retry'` is `1` (`control-openhands browser screenshot 'testid=chat-interface' --feature F06.failed-send-persists --name failed-before-leaving`). Run `control-openhands restart` and `control-openhands browser wait 'testid=error-message-banner' --state hidden --timeout 90000` (the page reconnects by itself; `testid=error-message-banner-retry` hurries it while the banner is still up): the Retry row is still there (count `1`). Leave through the sidebar and come back: `control-openhands browser click 'role=link[name="New Chat"]'`, `control-openhands browser wait 'testid=home-chat-launcher'`, `control-openhands browser click 'a[href*="/conversations/<id>"] >> nth=0' --expect-url '/conversations/<id>'`, `control-openhands browser wait 'testid=user-message'`; `control-openhands browser count 'testid=chat-message-retry'` is still `1` after the history reloaded (`control-openhands browser screenshot 'testid=chat-interface' --feature F06.failed-send-persists --name retry-after-reopen`). Send the same text again (fill `Reply with only: retry-ok`, click `testid=submit-button` with the same `--observe`): the new bubble goes `Sending...` and settles while `Failed to send Retry Dismiss` stays; `control-openhands browser count 'testid=chat-message-sending'` is `0`, `control-openhands browser count 'testid=chat-message-retry'` is `1`, and the rows of `control-openhands conversation events <id> --kinds MessageEvent --grep 'retry-ok'` whose `source` is `user` number exactly one, `Reply with only: retry-ok` (in a model-free run `count` is `1`; with a model the agent's `retry-ok` reply matches the grep too, so count the `user` rows, not `count`). Then `control-openhands browser click 'testid=chat-message-retry'` and `control-openhands browser wait 'testid=chat-message-retry' --state detached --timeout 15000`: `control-openhands browser count 'testid=chat-message-sending'` is `0` and the same `--grep` now lists exactly two `user` rows `Reply with only: retry-ok` (one per attempt, none duplicated). With a model, `control-openhands conversation wait <id> --fresh --timeout 120` before the next bullet.- **Live chip, stop and resume (`F06.live-activity`, `F06.stop-resume`).** Run `control-openhands conversation start --prompt "Run exactly one terminal command: sleep 25 && echo qa-slept. Then reply with only its output."` (no `--wait`), `control-openhands browser wait 'testid=live-activity-chip' --timeout 30000` and `control-openhands browser text 'testid=live-activity-chip'`: first `Thinking`, then the action title (e.g. `Run sleep 25 then echo qa-slept`); `browser text 'testid=interactive-chat-box'` contains `Running`. Run `control-openhands browser click 'testid=stop-button'`: `control-openhands conversation status <id>` is `paused`, the composer reads `Stopped`, `count 'testid=play-button'` is `1` and the chip is gone. Stop interrupts the running `sleep` too: `control-openhands conversation events <id>` shows an `AgentErrorEvent` (`Tool call interrupted before completion. The conversation was paused.`) followed by an `InterruptEvent`, and `browser text 'testid=chat-scroll-container'` shows an `Agent error` row under the action title. Run `control-openhands browser click 'testid=play-button'`: status `running`, then `control-openhands conversation wait <id> --timeout 120` ends `finished` with a new agent `MessageEvent` (the model either reports the interruption or reruns the command as a new action and replies `qa-slept`).- **Reload mid-run (`F06.reload-mid-run`).** Run `control-openhands conversation start --prompt "Run exactly one terminal command: sleep 15 && echo qa-reloaded. Then reply with only its output."`, wait for `testid=live-activity-chip`, then `control-openhands browser reload`. The composer still reads `Running` and the chip is back. After `control-openhands conversation wait <id> --timeout 120`: `testid=agent-message` is `qa-reloaded`, `count 'testid=user-message'` is `1`, `control-openhands browser eval "document.querySelectorAll('[data-testid=chat-scroll-container] [data-testid=generic-event-message-title]').length"` is `1`.-- **Older history (`F06.load-older-history`).** Run `control-openhands conversation start --prompt "Run these 16 terminal commands strictly one per tool call, sequentially (never combine them): echo qa1, echo qa2, echo qa3, echo qa4, echo qa5, echo qa6, echo qa7, echo qa8, echo qa9, echo qa10, echo qa11, echo qa12, echo qa13, echo qa14, echo qa15, echo qa16. Then reply DONE." --wait --timeout 400` (note `<history-id>`); `control-openhands conversation events <history-id> --last 500` reports a count above 50. Run `control-openhands browser network --clear`, `control-openhands browser reload`, then `control-openhands browser network --last 300`: two `GET /api/conversations/<history-id>/events/search` requests (newest page, then the older page). `control-openhands browser text 'testid=chat-scroll-container'` starts with the prompt and shows `16 actions completed` and `DONE`. The other trigger is the user scrolling to the top, driven without a model on the dummy profile from Preconditions (`qa-zeta` active; in a keyed run create and activate it with that same `llm set` command for this bullet, then re-run `control-openhands llm preset deepseek` and remove it with `control-openhands api DELETE /api/profiles/qa-zeta --write`, so later families still count two profiles; arrange, not proof). Run `control-openhands conversation start --prompt "qa older 1"` (no `--wait`; note `<older-id>`), then `for N in $(seq 2 30); do control-openhands conversation send <older-id> --prompt "qa older $N"; done`: every send is accepted while the agent sits in error, about 1.5 s apiece (42 s for the 29 here), and each refused message adds five events (a `MessageEvent`, three `ConversationStateUpdateEvent`s, a `ConversationErrorEvent`), so the conversation ends with about 150 events (152 here). `control-openhands conversation events <older-id> --last 500` reports `"total": 152`, `"pages": 2`, `"more": false` (both server pages read, see Gotchas); its first row is the `SystemPromptEvent` and the first `MessageEvent` is `qa older 1`, while `--last 25` reports `"pages": 1`, `"more": true` and starts at `qa older 26`. Run `control-openhands browser network --clear`, `control-openhands browser reload`, `control-openhands browser wait 'testid=chat-input'`, then `control-openhands browser network --filter 'events/search' --last 10`: two requests, `?limit=50&sort_order=TIMESTAMP_DESC` and one with `timestamp__lt=<ts of qa older 21>`, because the newest 50 events hold only ten bubbles, shorter than the viewport, so the automatic fetch ran once and then stopped. `control-openhands browser count 'testid=user-message'` is `20`, `control-openhands browser text 'testid=chat-scroll-container'` starts `qa older 11` and holds none of `qa older 1`…`qa older 10`, and `control-openhands browser bbox 'testid=chat-scroll-container'` has `scrollHeight` `1576` against `clientHeight` `814`: taller than the viewport, so the third page waits for the user. Run `control-openhands browser scroll 'testid=chat-scroll-container' --by -100000`: `control-openhands browser network --filter 'events/search' --last 5` gains a request with `timestamp__lt=<ts of qa older 11>`, `control-openhands browser count 'testid=user-message'` is `29` and the text starts `qa older 2` (that page held messages 2–10 plus the tail of the first message's events; not seen in two runs: if the first refusal ever logs six events instead of seven, the arithmetic gives count `30`, text `qa older 1` and a last page of only the `SystemPromptEvent`), and the view stays on `qa older 11`: `control-openhands browser eval "document.querySelector('[data-testid=chat-scroll-container]').scrollTop"` is `684`, the height of the prepended rows. Scroll to the top again the same way: a fourth request (`timestamp__lt=` the oldest loaded event), the count is `30`, the text starts `qa older 1` and `scrollTop` is `76` (`control-openhands browser screenshot 'testid=chat-interface' --feature F06.load-older-history --name scroll-top-loaded-qa-older-1`). A third scroll adds no request: the short page ended the paging. `control-openhands browser count 'testid=loading-older-events'` is `0` after each scroll because the `Fetching older messages…` row lasts a few tens of milliseconds on a local stack; to see it, scroll with `control-openhands browser scroll 'testid=chat-scroll-container' --by -100000 --observe 'testid=loading-older-events' --observe-ms 1500` instead (both scrolls were driven this way here, with the same counts), whose `observed` goes `<absent>` at `0` ms, `Fetching older messages…` at about 20 ms and `<absent>` again within about 40 to 120 ms (`116` ms on the first scroll, `39` on the second; `observedBy` `mutation`); the `timestamp__lt=` network rows remain the durable proof. Return to the 16-command conversation for the next bullet: `control-openhands browser goto /conversations/<history-id>` and `control-openhands browser wait 'testid=chat-input'`. Known failure (reproduced 2026-10-16, [#18523](https://github.com/OpenHands/OpenHands/issues/18523)): the transient `Fetching older messages…` label measures 2.58:1 on the Light+ chat panel and 2.40:1 on Solarized Light, below 4.5:1.+- **Older history (`F06.load-older-history`).** Run `control-openhands conversation start --prompt "Run these 16 terminal commands strictly one per tool call, sequentially (never combine them): echo qa1, echo qa2, echo qa3, echo qa4, echo qa5, echo qa6, echo qa7, echo qa8, echo qa9, echo qa10, echo qa11, echo qa12, echo qa13, echo qa14, echo qa15, echo qa16. Then reply DONE." --wait --timeout 400` (note `<history-id>`); `control-openhands conversation events <history-id> --last 500` reports a count above 50. Run `control-openhands browser network --clear`, `control-openhands browser reload`, then `control-openhands browser network --last 300`: two `GET /api/conversations/<history-id>/events/search` requests (newest page, then the older page). `control-openhands browser text 'testid=chat-scroll-container'` starts with the prompt and shows `16 actions completed` and `DONE`. The other trigger is the user scrolling to the top, driven without a model on the dummy profile from Preconditions (`qa-zeta` active; in a keyed run create and activate it with that same `llm set` command for this bullet, then re-run `control-openhands llm preset deepseek` and remove it with `control-openhands api DELETE /api/profiles/qa-zeta --write`, so later families still count two profiles; arrange, not proof). Run `control-openhands conversation start --prompt "qa older 1"` (no `--wait`; note `<older-id>`), then `for N in $(seq 2 30); do control-openhands conversation send <older-id> --prompt "qa older $N"; done`: every send is accepted while the agent sits in error, about 1.5 s apiece (42 s for the 29 here), and each refused message adds five events (a `MessageEvent`, three `ConversationStateUpdateEvent`s, a `ConversationErrorEvent`), so the conversation ends with about 150 events (152 here). `control-openhands conversation events <older-id> --last 500` reports `"total": 152`, `"pages": 2`, `"more": false` (both server pages read, see Gotchas); its first row is the `SystemPromptEvent` and the first `MessageEvent` is `qa older 1`, while `--last 25` reports `"pages": 1`, `"more": true` and starts at `qa older 26`. Run `control-openhands browser network --clear`, `control-openhands browser reload`, `control-openhands browser wait 'testid=chat-input'`, then `control-openhands browser network --filter 'events/search' --last 10`: two requests, `?limit=50&sort_order=TIMESTAMP_DESC` and one with `timestamp__lt=<ts of qa older 21>`, because the newest 50 events hold only ten bubbles, shorter than the viewport, so the automatic fetch ran once and then stopped. `control-openhands browser count 'testid=user-message'` is `20`, `control-openhands browser text 'testid=chat-scroll-container'` starts `qa older 11` and holds none of `qa older 1`…`qa older 10`, and `control-openhands browser bbox 'testid=chat-scroll-container'` has `scrollHeight` `1576` against `clientHeight` `814`: taller than the viewport, so the third page waits for the user. Run `control-openhands browser scroll 'testid=chat-scroll-container' --by -100000`: `control-openhands browser network --filter 'events/search' --last 5` gains a request with `timestamp__lt=<ts of qa older 11>`, `control-openhands browser count 'testid=user-message'` is `29` and the text starts `qa older 2` (that page held messages 2–10 plus the tail of the first message's events; not seen in two runs: if the first refusal ever logs six events instead of seven, the arithmetic gives count `30`, text `qa older 1` and a last page of only the `SystemPromptEvent`), and the view stays on `qa older 11`: `control-openhands browser eval "document.querySelector('[data-testid=chat-scroll-container]').scrollTop"` is `684`, the height of the prepended rows. Scroll to the top again the same way: a fourth request (`timestamp__lt=` the oldest loaded event), the count is `30`, the text starts `qa older 1` and `scrollTop` is `76` (`control-openhands browser screenshot 'testid=chat-interface' --feature F06.load-older-history --name scroll-top-loaded-qa-older-1`). A third scroll adds no request: the short page ended the paging. `control-openhands browser count 'testid=loading-older-events'` is `0` after each scroll because the `Fetching older messages…` row lasts a few tens of milliseconds on a local stack; to see it, scroll with `control-openhands browser scroll 'testid=chat-scroll-container' --by -100000 --observe 'testid=loading-older-events' --observe-ms 1500` instead (both scrolls were driven this way here, with the same counts), whose `observed` goes `<absent>` at `0` ms, `Fetching older messages…` at about 20 ms and `<absent>` again within about 40 to 120 ms (`116` ms on the first scroll, `39` on the second; `observedBy` `mutation`); the `timestamp__lt=` network rows remain the durable proof. Return to the 16-command conversation for the next bullet: `control-openhands browser goto /conversations/<history-id>` and `control-openhands browser wait 'testid=chat-input'`. The `Fetching older messages…` row uses the tertiary text colour and stays readable in the light themes: measured while it is shown (a `browser eval` MutationObserver that reads its computed colour when `testid=loading-older-events` appears, then the scroll above), it is 8.45:1 on the Light+ chat panel and 4.99:1 on Solarized Light (theme switching: F16.color-theme).- **Scroll to bottom (`F06.scroll-to-bottom`).** On the 16-command conversation (`<history-id>`; in a model-free run this bullet is blocked with the automatic-fetch half, prerequisite `DEEPSEEK_API_KEY`), `control-openhands browser click 'testid=event-group-toggle'` makes the transcript taller than the viewport while the view stays where it was. Run `control-openhands browser scroll 'testid=chat-scroll-container' --by 2000` first (the button only reacts to scroll events, so a page that is already at the top never shows it): `count 'testid=scroll-to-bottom'` is `0`. Run `control-openhands browser scroll 'testid=chat-scroll-container' --by -2000`; the count is `1`. `control-openhands browser click 'testid=scroll-to-bottom'`, then the count is `0` and `control-openhands browser eval "(()=>{const e=document.querySelector('[data-testid=chat-scroll-container]');return Math.round(e.scrollHeight-e.scrollTop-e.clientHeight)})()"` is `0`.- **Confirmation mode on (`F06.confirmation-mode`).** Arrange through the settings UI, in this order (the Security Analyzer field appears only once Confirmation Mode is on): `control-openhands browser goto /settings/verification`, `control-openhands browser click 'testid=verification-settings-screen >> text=Confirmation Mode'`, `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser click 'testid=verification-settings-screen >> role=button[name="Show suggestions"]'`, `control-openhands browser click 'role=listbox >> role=option[name="None"]'`, `control-openhands browser value 'testid=verification-settings-screen >> role=combobox[name="Security Analyzer"]'` (`None`), `control-openhands browser click 'testid=verification-settings-screen >> testid=save-button'`; `control-openhands api GET /api/settings` shows `"confirmation_mode": true` and `"security_analyzer": "none"` (None means every action is confirmed; the default LLM analyzer only stops HIGH-risk actions, so `echo` would run unprompted). Run `control-openhands conversation start --prompt "Run the terminal command 'echo qa-confirm-ok' and reply with its output." --wait --timeout 180`: it returns `"status": "waiting_for_confirmation"`. `control-openhands browser text 'testid=chat-scroll-container'` ends `Do you want to continue with this action? Cancel Ctrl+⇧+⌫ Continue Ctrl+↩` on Linux (`Cancel ⇧⌘⌫ Continue ⌘↩` on macOS, #18127), `control-openhands browser tooltip 'testid=action-confirm-button'` is `Confirm the requested action`, and a lock icon sits above the composer (screenshot). The tooltip stays open over the Cancel button, so move the pointer first: `control-openhands browser hover 'testid=chat-input'`. Run `control-openhands browser click 'testid=action-reject-button'`: status `idle`, the buttons are gone, `conversation events <id> --last 6` ends with `UserRejectObservation`, and the action's row reads `Rejected` (`control-openhands browser count 'testid=rejected-indicator'` is `1`) with no spinner. Send `Run the terminal command 'echo qa-confirm-two' and reply with its output.` through `testid=chat-input` / `testid=submit-button`, `control-openhands browser wait 'testid=action-confirm-button' --timeout 20000`, `control-openhands browser click 'testid=action-confirm-button'`; about 15 s later the status is `finished` and the events contain the observation `qa-confirm-two`. A rejection inside a group resolves it too: send `Run the terminal command 'echo qa-g1'. After it finishes, run the terminal command 'echo qa-g2' as a separate tool call. Then reply DONE.`, continue the first action (`browser wait 'testid=action-confirm-button'`, then `control-openhands browser press Control+Enter`, which is also the next bullet's shortcut) and cancel the second (`browser wait 'testid=action-confirm-button' --timeout 60000`, `control-openhands browser press Control+Shift+Backspace`): status `idle`, `control-openhands browser text 'testid=event-group >> nth=-1'` ends `2 actions completed`, `control-openhands browser count 'testid=event-group >> nth=-1 >> .animate-spin'` is `0`, the same after `browser reload`, and the expanded group (`control-openhands browser click 'testid=event-group >> nth=-1 >> button >> nth=0'`) shows `Rejected` on the `qa-g2` row (`browser screenshot 'testid=chat-interface' --feature F06.confirmation-mode --name group-after-reload`). Each button keeps its label and hint on one line: while the prompt is waiting, `control-openhands browser eval "[...document.querySelectorAll('[data-testid=action-reject-button],[data-testid=action-confirm-button]')].map(b=>b.offsetHeight+'/'+b.scrollHeight).join(' ')"` is `26/26 26/26` at desktop, also with the side panel open (`testid=right-panel-toggle`). At phone width (`browser viewport phone`) the shortcut hints are hidden (no hardware keyboard), the buttons read `Cancel` and `Continue`, and `browser text 'testid=chat-scroll-container'` ends `Do you want to continue with this action? Cancel Continue` (screenshot the awaiting prompt at phone width).- **Confirmation shortcuts (`F06.confirmation-shortcuts`).** The previous bullet resolved its action, so arrange a new one in the same conversation: send `Run the terminal command 'echo qa-confirm-three' and reply with its output.` through `testid=chat-input` / `testid=submit-button` and `control-openhands browser wait 'testid=action-confirm-button' --timeout 20000`. `control-openhands browser text 'testid=action-confirm-button'` is `Continue Ctrl+↩` off macOS. Ctrl works off macOS (#17928, fixed by #18127): the previous bullet's `Control+Enter` continued and `Control+Shift+Backspace` cancelled. `control-openhands browser press Meta+Enter` continues too: `control-openhands conversation wait <id> --fresh --timeout 120` ends `finished` and `control-openhands conversation events <id> --kinds ObservationEvent` contains `qa-confirm-three`. Restore afterwards: `browser goto /settings/verification`, click `testid=sdk-section-all-toggle`, `Show suggestions`, `role=listbox >> role=option[name="LLM"]`, then `testid=verification-settings-screen >> text=Confirmation Mode`, then `save-button`; `api GET /api/settings` shows `"confirmation_mode": false`, `"security_analyzer": "llm"`.@@ -99,7 +100,7 @@ Preconditions:- **Error events (`F06.error-events`).** Any run that ends in a `ConversationErrorEvent` (today: any message sent in a branched conversation) leaves `conversation wait <id>` at `"status": "error"`; `control-openhands browser testids 'testid=chat-interface'` lists `error-message-banner` with `warning-message-banner-icon` and the error's text, and the composer status reads `Error`. Read the code with `control-openhands api GET "/api/conversations/<id>/events/search?limit=3&sort_order=TIMESTAMP_DESC"`.- **Skill installed banner (`F06.skill-install-banner`).** The banner keys on the installer's success line in terminal output. Put this one line in a prompt file: `Run exactly this one terminal command, verbatim, then reply DONE: mkdir -p .agents/skills/qa-skill && printf '# qa-skill\n' > .agents/skills/qa-skill/SKILL.md && echo "✅ Successfully installed 'qa-skill' to $PWD/.agents/skills/qa-skill"`, then run `control-openhands conversation start --prompt "$(cat <prompt file>)" --wait --timeout 150`. `control-openhands browser text 'testid=skill-install-restart-banner'` reads `Installed to this workspace: qa-skill. Skills load when a conversation starts, so this conversation can't use them yet.` Run `control-openhands browser click 'testid=skill-install-restart-action' --expect-url '/conversations/(?!<id>)'`; `control-openhands api GET /api/conversations/<new id>` has the same `workspace.working_dir` as the source. Back on the source (`control-openhands browser goto /conversations/<id>`), `control-openhands browser click 'testid=skill-install-restart-dismiss'` hides it (count `0`); after `browser reload` it is back (dismissal is session-only by design).- **No LLM (`F06.llm-not-configured-banner`).** On the fresh no-LLM run: `control-openhands onboard --skip`, `control-openhands browser click 'testid=conversation-panel-new-thread-picker'`, `control-openhands browser click 'testid=launch-no-workspace' --expect-url '/conversations/'`. `control-openhands browser text 'testid=home-llm-not-configured-banner'` reads `Your LLM isn't set up yet, so conversations won't run. Finish setup to get started. Set up LLM`; `count 'testid=chat-suggestions'` is `0`; the composer cannot take text (`control-openhands browser attr 'testid=chat-input' contenteditable` is `false`, so `browser fill` fails) and `control-openhands browser enabled 'testid=submit-button'` is `false`. `control-openhands browser click 'testid=home-llm-not-configured-action'` then `browser url` ends in `/settings/llm`.-- **Critic (`F06.critic-result`).** Toggle the hidden switch through its label on `/settings/verification`: `control-openhands browser goto /settings/verification`, `control-openhands browser click 'label:has([data-testid="sdk-settings-verification.critic_enabled"])'`, save, `api GET /api/settings` shows `"critic_enabled": true`. Use the authorized OpenHands/proxy-backed profile described above for this check; leave Critic API Key empty to reuse its LLM key. With that profile and the critic enabled, run `control-openhands conversation start --prompt "Create qa_critic.txt containing hi, then finish." --wait --timeout 180`: every scored event carries a block, so `control-openhands browser count 'text=Critic: agent success likelihood'` is at least `1` and a finishing action (`finish` tool row) shows its own block next to the agent's message; the stars carry the score as their label, which an ARIA snapshot does not list: `control-openhands browser count '[aria-label^="Score: "]'` is at least `2` and `control-openhands browser attr '[aria-label^="Score: "] >> nth=0' aria-label` reads `Score: <N.N>%` with one decimal (`Score: 82.0%`), the same number as the `(82.0%)` text beside the stars. `control-openhands browser click 'role=button[name="Expand details"] >> nth=0'` opens the categories `Potential Issues:`, `Infrastructure:` and `Likely Follow-up:` with their named issues in `control-openhands browser text 'testid=chat-scroll-container'`, and the button's label flips to `Collapse details`. Turn it off again the same way and restore the previous profile. Profile availability is a precondition, not proof that the service accepted the credential: require real scored events before recording a pass.+- **Critic (`F06.critic-result`).** Toggle the hidden switch through its label on `/settings/verification`: `control-openhands browser goto /settings/verification`, `control-openhands browser click 'label:has([data-testid="sdk-settings-verification.critic_enabled"])'`, save, `api GET /api/settings` shows `"critic_enabled": true`. Use the authorized OpenHands/proxy-backed profile described above for this check; leave Critic API Key empty to reuse its LLM key. With that profile and the critic enabled, run `control-openhands conversation start --prompt "Create qa_critic.txt containing hi, then finish." --wait --timeout 180`: every scored event carries a block, so `control-openhands browser count 'text=Critic: agent success likelihood'` is at least `1` and a finishing action (`finish` tool row) shows its own block next to the agent's message; the stars carry the score as their label, which an ARIA snapshot does not list: `control-openhands browser count '[aria-label^="Score: "]'` is at least `2` and `control-openhands browser attr '[aria-label^="Score: "] >> nth=0' aria-label` reads `Score: <N.N>%` with one decimal (`Score: 82.0%`), the same number as the `(82.0%)` text beside the stars. `control-openhands browser click 'role=button[name="Expand details"] >> nth=0'` opens the categories `Potential Issues:`, `Infrastructure:` and `Likely Follow-up:` with their named issues in `control-openhands browser text 'testid=chat-scroll-container'`, and the button's label flips to `Collapse details`. Turn it off again the same way and restore the previous profile. Profile availability is a precondition, not proof that the service accepted the credential: require real scored events before recording a pass. A locally scored block needs no Cloud key: create a conversation through `control-openhands api POST /api/conversations --write --data @/dev/stdin`, piping a payload built from the environment (so the model key is neither printed nor written) with a direct `agent` `{kind:"Agent", llm:{model:"deepseek/deepseek-flash", api_key:<from $DEEPSEEK_API_KEY>}, tools:[], include_default_tools:["FinishTool"], critic:{kind:"AgentFinishedCritic"}}`, a `workspace.working_dir` under `$OH_VERIFY_RUN/workspace` and an `initial_message` (`run: true`) `Call the finish tool immediately with the message qa-critic-done. Do not do anything else.`; arrange, not proof. After `control-openhands conversation wait <id>` (`finished`) open `/conversations/<id>`: the block reads `Critic: agent success likelihood ☆☆☆☆☆ (0.0%)` with the iterative-refinement hint below, and the stars carry `aria-label` `Score: 0.0%`. Its colours follow the theme: header, `(0.0%)` and hint use tertiary text (8.45:1 on the Light+ chat panel, 4.99:1 on Solarized Light) and the score-0 stars use error feedback (5.86:1 and 5.61:1); this critic lists no issue categories, so Expand details is absent.- **Phone (`F06.phone`).** Run `control-openhands browser viewport phone`, open the table conversation (`control-openhands browser goto /conversations/<stream id>`), `control-openhands browser bbox 'testid=chat-interface'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `control-openhands browser bbox 'testid=markdown-table-scroll'` (`scrollWidth` larger than `clientWidth`: the table scrolls inside itself). `control-openhands browser screenshot --feature F06.phone --name conversation`, then `control-openhands browser viewport desktop`.- After each group run `control-openhands browser errors --app-only`. Outages driven with `service stop` log connection-refused console errors by design; `pageErrors` must stay `0`.@@ -144,7 +145,7 @@ Preconditions:- `control-openhands conversation events <id>` returns the last 25 events unless you pass `--last N`; it reads the server's 100-event pages until `N` rows are in hand (`pages` says how many it read, `more: true` that older pages remain), so `--last 500` covers a 150-event conversation in two pages (`total` `152`, `pages` `2`, `more` `false` on the Older history conversation; `--last 25` there reads one page, `more` `true`) and `--from-start` reads from the oldest. `--kinds` and `--grep` filter before `--last` counts, and pages are read until `--last` matching rows are in hand: `--kinds MessageEvent --last 3` is the three newest messages, `--kinds MessageEvent --last 500` all thirty, and `--grep 'qa older 1'` reaches `qa older 1` on the oldest page with the default `--last` (`count` `11`, `pages` `2`). Event texts are cut at about 160 characters.- `--observe` takes the same selector syntax as the verbs: `testid=` and plain CSS segments (joined with ` >> `) are watched by an in-page MutationObserver (`observedBy` `mutation`; the Older history bullet's `--observe 'testid=loading-older-events'` is one), while other engines (`role=`, `text=`, `nth=`) are polled every 20 ms instead (not driven in this family).- `role=button[name="Expand"]` also matches "Expand thinking" and "Expand actions"; use `[exact]` to target tool rows. Each click flips the row to Collapse, so `nth=0` walks to the next collapsed row.-- Tool rows show no success icon; the only indicators are the `exit N` badge for non-zero exits and a clock icon (`status-icon`) for timeouts. A single action is not grouped: it renders as a bare row without `event-group`.+- Tool rows show no success icon; the only indicators are the `exit N` badge for non-zero exits and a clock icon (`status-icon`) for timeouts. The clock uses the theme's warning colour (`fill-feedback-warning`): 6.10:1 on the Light+ chat panel and 6.50:1 on Solarized Light. A follow-up `Call the terminal tool exactly once with command 'sleep 3' and its timeout parameter set to 1 (seconds). Do not run anything else. Then reply DONE.` produces one (`control-openhands browser count 'testid=status-icon'` is `1`). A single action is not grouped: it renders as a bare row without `event-group`.- The model writes the tool titles and may append to commands (it turned `ls /missing` into `ls ...; echo "EXIT:$?"`, exit 0). Ask for the command "verbatim, with nothing appended" when the exit code matters.- The agent itself may open files with `canvas_ui_control` (a third action in the echo/file run). Close the file tab before proving that a path link opens it.- On Settings → Verification the Security Analyzer field (All tab) is shown only while Confirmation Mode is on: switch Confirmation Mode on first, then pick the analyzer. Check `api GET /api/settings` after saving.@@ -152,7 +153,7 @@ Preconditions:- The confirm and cancel shortcuts accept Meta or Ctrl and ignore a held key's auto-repeat; once a response is sent the listener is detached, so a second press sends nothing (#18127). Hints read `Ctrl+↩` / `Ctrl+⇧+⌫` off Apple platforms.- Known failure, Canvas UI ([#18202](https://github.com/OpenHands/OpenHands/issues/18202), reproduced 2026-10-16): after a user-message branch the composer holds the prefilled text but Send stays disabled (`browser enabled 'testid=submit-button'` is `false`, and a click on it times out with `element is not enabled`). The prefill is a saved draft that `src/hooks/chat/use-draft-persistence.ts` writes into the field without an input event, and the composer is reused across the client-side switch, so its Send state keeps the source conversation's empty field. Editing the text, a reload, or Enter in the field (`browser press Enter --selector 'testid=chat-input'`) all work. A draft restored by a sidebar switch behaves the same (F05 Gotchas).- Known failure, likely Agent Server: every message in a branched conversation ends with `ConversationErrorEvent` `LLMAuthenticationError` ("Your LLM API key appears to be invalid or has expired."), while the source conversation keeps working (OpenHands/software-agent-sdk#5465). Do not use branches as fixtures for other checks.-- "The terminal session was reset because the underlying tmux server/session disappeared" (clock `status-icon` on the row) appears after a `service stop agent-server` / `restart` cycle. Since `launch` gives each run its own short `TMUX_TMPDIR`, other runs no longer cause it. The underlying product bug is still open (OpenHands/OpenHands#17946) for plain `agent-canvas` instances: `bin/agent-canvas.mjs` → `scripts/dev-with-automation.mjs` passes `TMUX_TMPDIR=<state>/tmux` to the Agent Server but `ensureDirectories` never creates it (`dev-safe.mjs` does), so tmux 3.4 falls back to the shared `/tmp/tmux-<uid>/openhands` socket and another instance's restart or stop kills this run's windows. The model usually reruns the command; expect an extra action.+- The agent server starts each terminal session on its own tmux socket (`openhands-<random hex>`) inside its own `TMUX_TMPDIR`: `launch` sets a per-run `/tmp/ohv-tmux-<hash>`, and a plain `agent-canvas` creates `<state>/tmux` and passes it on, so another instance stopping or restarting does not touch this run's terminals. After this run's own `restart`, a follow-up terminal command ran once with its output and no reset (2026-10-17). If a row still comes back as "The terminal session was reset because the underlying tmux server/session disappeared" (clock `status-icon`), the model usually reruns the command; expect an extra action.- The `ConversationErrorEvent` banner is not restored after navigating away and back (or a reload); read the error from the events API instead.- After `restart`, reconnection takes about 20 s without a click; click `error-message-banner-retry` to reconnect at once. The error banner does not come back after a page reload once the stack is healthy.- "Sending..." lasts about 100 to 650 ms on a local stack, too short to hover and click `chat-message-stop`; that stop path is not driven here. The `F06.pending-messages` failure path needs `service stop agent-server`.@@ -212,8 +213,4 @@ Preconditions:- Session reconnects use the latest received `after_seq` cursor (`F06.reconnect`). For a local metadata check, retain the open page, send genuine `run:false` user events, record `/events/count`, restart the owned backend and inspect finite WebSocket lifecycle metadata: the reconnect cursor is the previous count minus one and a subsequent recovery marker appears once. This does not measure frame-body replay or Cloud behavior.- Stop persists `paused` before an in-flight model request completes (`F06.stop-resume`); the persisted state and reloaded Stopped UI survive an Agent Server restart. A bounded credential-free check can use the tracked accept-only endpoint and an exact listener PID proven to descend from the owned launcher; never kill by pattern or resume the paused conversation.-- Known issue [#18523](https://github.com/OpenHands/OpenHands/issues/18523) (`F06.tool-visualizers`): a timed-out terminal action's clock (`testid=status-icon`) measures 1.91:1 on the Light+ chat panel and 1.77:1 on Solarized Light, below 3:1. Reproduce with a follow-up `Call the terminal tool exactly once with command 'sleep 3' and its timeout parameter set to 1 (seconds). Do not run anything else. Then reply DONE.`-- Known issue [#18522](https://github.com/OpenHands/OpenHands/issues/18522) (`F06.markdown-rendering`): chat Markdown links have no underline before hover, and a reply `See [QA docs](https://example.com/qa) for the verification page.` shows the link at 3.76:1 on the Light+ chat panel and 3.49:1 on Solarized Light (2.99:1 and 2.83:1 in the user bubble), below 4.5:1.- Known issue [#18521](https://github.com/OpenHands/OpenHands/issues/18521) (`F06.hook-events`): with real `user_prompt_submit` hooks in `<workspace>/.openhands/hooks.json` (stderr then exit 0; `sleep 2` with timeout 1; a JSON deny with exit 2), expanded hook rows in Light+ show the ok/failed/blocked badges at 1.84:1, 1.50:1 and 1.80:1, stderr at 1.30:1, the timeout error at 2.89:1 and the blocked reason at 1.72:1; Solarized Light gives 1.18:1 to 2.68:1.-- Known issue [#18524](https://github.com/OpenHands/OpenHands/issues/18524) (`F06.critic-result`): a real score-0 critic block measures header 1.48:1, stars 2.89:1 and percentage 4.74:1 on the Light+ chat panel (1.37:1, 2.68:1 and 4.40:1 on Solarized Light). For a local rendering-only check without an authorized remote critic profile, arrange it with `control-openhands api POST /api/conversations --write` using a direct `agent` with `critic:{kind:"AgentFinishedCritic"}`, `tools:[]`, `include_default_tools:["FinishTool"]` and a finish-immediately prompt; that critic scores 0 locally.-- Known issue [#18541](https://github.com/OpenHands/OpenHands/issues/18541) (`F06.markdown-rendering`): in a fenced `toml` block, section headers like `[build-system]` render on three lines (`[`, name, `]`) because the highlighter's `token table` span takes `display: table`; `browser eval "getComputedStyle(document.querySelector('[data-testid=agent-message] .token.table')).display"` returns `table` instead of `inline`. Reproduce with `Write a short markdown answer on Python packaging with pyproject.toml. Include one toml code block with [build-system] and [project] sections. Do not run any tools.` The Files tab preview of a `.toml` file (F08) shows the same break.diff --git a/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md b/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.mdindex 8afb7308ae..19f4c281c1 100644--- a/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md+++ b/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md@@ -25,7 +25,7 @@ Source: `src/routes/conversation.tsx`, `src/components/features/conversation/`,- `F07.agent-tools-modal`: Show Agent Tools & Metadata shows the system message and the tool list.- `F07.export-transcript`: Export… downloads a Markdown or HTML transcript with optional tool details and timestamps.- `F07.download-zip`: Download conversation data downloads `conversation_<id>.zip`.-- `F07.display-cost`: Display Usage and Cost opens the right panel on the Usage tab.+- `F07.display-cost`: Display Usage and Cost opens the right panel on the Usage tab; at phone width it opens the panel page (`/conversations/<id>/panel`) on Usage.- `F07.stop-confirm`: Stop Conversation asks for confirmation; Confirm Close pauses the conversation and returns to `/conversations`.- `F07.delete-confirm`: Delete Conversation names the title in a confirmation; Cancel keeps it, Confirm Delete removes it and returns to `/conversations`.- `F07.branch-from-message`: Branch from here on a message forks the conversation as `<title> (branch)`; on a user message the message is left out and its text goes back into the composer.@@ -76,7 +76,7 @@ Preconditions:- **Agent tools (`F07.agent-tools-modal`).** Open the menu, `control-openhands browser click 'testid=conversation-name-context-menu >> testid=show-agent-tools-button'`; the dialog `testid=system-message-modal` has tabs `System Message` (selected, the system prompt) and `Available Tools`. `control-openhands browser click 'role=dialog >> role=tab[name="Available Tools"]'`, `control-openhands browser count 'testid=system-message-modal >> testid=toggle-button'` (backend-provided count; `10` on Agent Server 1.54.0: terminal, file_editor, task_tracker, canvas_ui_control, launch_child_conversation, request_secret (local backends, see `F14.agent-request`), finish, think, switch_llm, invoke_skill), `control-openhands browser click 'testid=system-message-modal >> testid=toggle-button >> nth=0'` expands `terminal` with its description. Close with `control-openhands browser click 'testid=close-system-message-modal'`.- **Export transcript (`F07.export-transcript`).** Open the menu, `control-openhands browser click 'testid=conversation-name-context-menu >> testid=export-transcript-button'`; `control-openhands browser snapshot 'role=dialog'` shows `Export conversation`, radio `Markdown document (.md)` checked, `Standalone web page (.html)`, and checkboxes `Include tool call details` and `Include timestamps` checked. `control-openhands browser click 'testid=confirm-transcript-export'` then `control-openhands browser downloads --last 1 --inspect`: a `<ms>-conversation-<id>.md` (the harness prefixes a timestamp to the suggested `conversation-<id>.md`) lands in `<run>/private/downloads` starting `# QA_F07 renamed`, `**Model:** deepseek/deepseek-flash`, `## User` with `<sub>` timestamps. Reopen, `control-openhands browser click 'role=dialog >> role=radio[name="Standalone web page (.html)"]'`, `control-openhands browser click 'role=dialog >> role=checkbox[name="Include timestamps"]'`, export: the `.html` has `<title>QA_F07 renamed</title>` and no timestamps. Reopen once more: `control-openhands browser click 'testid=cancel-transcript-export'` closes it without a download (the file count in `<run>/private/downloads` is unchanged); `browser press Escape` and `testid=close-transcript-export-modal` close it too.- **Download data (`F07.download-zip`).** Open the menu, `control-openhands browser click 'testid=conversation-name-context-menu >> testid=download-trajectory-button'`, then `control-openhands browser downloads`. A `<ms>-conversation_<id>.zip` lands in `<run>/private/downloads`; `browser downloads --last 1 --inspect` lists `<id-without-dashes>/base_state.json` and `bash_events/`, and every LLM `api_key` in `base_state.json` is masked as asterisks (`**********`).-- **Usage (`F07.display-cost`).** Open the menu, `control-openhands browser click 'testid=conversation-name-context-menu >> testid=display-cost-button'`, `control-openhands browser wait 'testid=usage-panel'` and `control-openhands browser attr 'testid=right-panel-toggle' aria-pressed` (`true`). The panel shows Context Window, Token Usage and Total Cost (contents are F27). At phone width (`browser viewport phone`) the same click is expected to open `/conversations/<id>/panel` on the Usage tab. Known failure (reproduced 2026-10-16, [#18536](https://github.com/OpenHands/OpenHands/issues/18536)): at phone width the menu closes, the URL stays `/conversations/<id>` and `testid=usage-panel` never appears; the composer context ring's Usage row and bar (`F27.composer-context-meter`) do the same.+- **Usage (`F07.display-cost`).** Open the menu, `control-openhands browser click 'testid=conversation-name-context-menu >> testid=display-cost-button'`, `control-openhands browser wait 'testid=usage-panel'` and `control-openhands browser attr 'testid=right-panel-toggle' aria-pressed` (`true`). The panel shows Context Window, Token Usage and Total Cost (contents are F27). At desktop the URL stays `/conversations/<id>` (the side drawer opens). At phone width (`control-openhands browser viewport phone`, then `control-openhands browser goto /conversations/<id>`) the same two clicks open the panel page: `control-openhands browser wait 'testid=usage-panel' --timeout 5000` succeeds and `control-openhands browser url` ends `/conversations/<id>/panel`. The composer context ring's Usage row and bar (`F27.composer-context-meter`: `browser click 'testid=context-window-meter'`, then `testid=context-window-plan-usage` or `testid=context-window-meter-bar-button`) open the same page there. Return with `control-openhands browser viewport desktop`.- **Right panel (`F07.right-panel-toggle`).** Run `control-openhands browser click 'testid=right-panel-toggle'` and `control-openhands browser attr 'testid=right-panel-toggle' aria-label`: it alternates `Show panel` / `Hide panel`, and `control-openhands browser wait 'testid=tabs-pane-header' --state hidden --timeout 5000` (after `Show panel`) or `--state visible` (after `Hide panel`) follows; a plain `browser visible` right after the click still reads the old value while the panel animates. With the panel open, `control-openhands browser reload` and `browser attr 'testid=right-panel-toggle' aria-pressed` reads `false` (closed by design; the selected tab is remembered).- **Overview (`F07.overview-toggle`).** Run `control-openhands browser click 'testid=conversation-overview-toggle'`, `control-openhands browser visible 'testid=conversation-overview-panel'` (`true`) and `browser attr 'testid=conversation-overview-toggle' aria-label` (`Hide overview`). Then `control-openhands browser click 'testid=right-panel-toggle'`: `control-openhands browser wait 'testid=conversation-overview-column' --state hidden --timeout 5000` succeeds (an immediate `browser visible` can still read `true`). With the panel open, `control-openhands browser hover 'testid=conversation-overview-toggle'` and `control-openhands browser wait 'testid=conversation-overview-peek' --timeout 5000` shows the peek. Clicking the overview toggle now closes the panel (`aria-pressed` `false`) and shows the overview (`browser wait 'testid=conversation-overview-column' --state visible`). Click the overview toggle once more to restore the closed state.- **Git actions (`F07.git-actions-menu`).** Run `control-openhands browser click 'testid=conversation-git-actions-toggle'` and `control-openhands browser testids 'testid=conversation-git-actions-menu'` (`-commit`, `-pull`, `-push`, `-create-pr`, `-create-new-branch`, labels `Commits`, `Pull`, `Push`, `Create PR`, `Create New Branch`). `control-openhands browser click 'testid=conversation-git-actions-menu >> testid=conversation-git-actions-commit'` then `control-openhands browser text 'testid=chat-input'`: `Please review the current changes and create a git commit with a concise, descriptive message.`; `conversation events <id> --kinds MessageEvent` shows no new message. `control-openhands browser enabled 'testid=submit-button'` should be `true`, without typing. Each item replaces the composer text: `-pull` gives `Please pull the latest code from the repository.`, `-create-new-branch` gives `Please create a new branch with a descriptive name related to the work you plan to do.`, and the Push and Create PR prompts name `GitHub` by default. `control-openhands browser press Escape` closes the open menu. Clear the composer with `control-openhands browser fill 'testid=chat-input' ''`. Known failure ([#18202](https://github.com/OpenHands/OpenHands/issues/18202), reproduced 2026-10-09 on `main@3500d9e5`): after Pull, Send stays disabled until the user types.diff --git a/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md b/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.mdindex 9541f868ea..fc9c51856a 100644--- a/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md+++ b/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md@@ -82,7 +82,7 @@ Preconditions:- **Tree toggle (`F08.files-tree-toggle`).** Run `control-openhands browser bbox 'testid=files-tab-tree'` (width `224`). Then `control-openhands browser click 'testid=file-quick-row-tree-toggle'`, `control-openhands browser attr 'testid=file-quick-row-tree-toggle' aria-label` (`Show file tree`) and `control-openhands browser count 'testid=files-tab-tree'` (`0`). Reload and reopen the drawer: the count is still `0`. Click the toggle again to restore (count `1`).- **Tree resize (`F08.files-tree-resize`).** Run `control-openhands browser drag 'testid=files-tab-tree-resize-handle >> .cursor-ew-resize' --by 100,0`; `browser bbox 'testid=files-tab-tree'` is `324` wide and `control-openhands browser eval "localStorage.getItem('files-tab-tree-width')"` is `"324"`. `--by 500,0` clamps at `480`, `--by -600,0` at `160`, and the width survives `browser reload` plus the toggle. Restore with `--by 64,0` (`224`).- **File discovery (`F08.files-discovery-settings`).** With the drawer open on Files (open it with `right-panel-toggle` and click `testid=conversation-tab-files` only if Files is not already the active tab: clicking the active tab closes the drawer), run `control-openhands browser click 'testid=file-discovery-settings'` and `control-openhands browser snapshot 'role=dialog[name="File discovery"]'`: `Applies to this workspace on this backend, including other conversations using it.`, textbox `Excluded directories` (`.git`, `node_modules`, `.venv`, … `target`), spinbutton `Maximum files (0 for unlimited)` `2000`, checkbox `Include symbolic links to files`, buttons `Restore defaults`, `Cancel`, `Save`. With `D='role=dialog[name="File discovery"]'`: `control-openhands browser fill "$D >> role=spinbutton" '-1'` makes `control-openhands browser enabled "$D >> role=button[name=\"Save\"]"` `false`; `2000` makes it `true`. Run `control-openhands browser fill "$D >> role=textbox" "$(printf '.git\nnode_modules\nsrc')"` and click Save: the dialog closes, `control-openhands api GET /api/settings` holds `workspace_file_discovery` keyed by the workspace path with `excludedPatterns` `.git`, `node_modules`, `src`, and after `control-openhands browser click 'testid=files-tab-refresh'` `control-openhands browser count 'testid=file-tree-dir-src'` is `0`, also after `browser reload` and reopening the drawer (`browser screenshot --feature F08.files-discovery-settings --name dialog` before saving). Restore: reopen the dialog, click `Restore defaults` (the textbox returns to the default list; the dialog stays open), then `Save`; after Refresh the `file-tree-dir-src` count is `1`. `control-openhands browser press Escape` closes the dialog without saving. At phone width the dialog fits the viewport: on the 390 px `/panel` page (`control-openhands browser viewport phone`, `browser goto /conversations/<id>/panel`, `browser click 'testid=conversation-tab-files'`, then `file-discovery-settings`) `control-openhands browser bbox 'testid=file-discovery-settings-modal'` is about 351 px wide at x about 20 with `insideViewport` `true` (520 px at desktop), and `browser screenshot --feature F08.files-discovery-settings --name phone-fits` shows the title, the textbox and `Restore defaults`, `Cancel`, `Save` whole. Measure the `file-discovery-settings-modal` test id: `role=dialog` is the full-screen backdrop. Focus starts in the dialog (the Excluded directories textbox) and Tab cycles its fields and buttons without leaving it.-- **Rich/Plain (`F08.files-rich-plain`).** Click `testid=file-tree-file-qa-notes.md`; `testid=file-content-viewer-markdown` exists. Click `testid=file-tree-file-qa-page.html`, then run `control-openhands browser attr 'testid=file-content-viewer-iframe' sandbox` (`allow-same-origin`) and `control-openhands browser eval "document.querySelector('[data-testid=file-content-viewer-iframe]').contentDocument.body.innerText"` (`QA page`). Run `control-openhands browser click 'testid=files-tab-content-mode-toggle-option-plain'`; `control-openhands browser text 'testid=files-tab-content >> testid=file-content-viewer-highlighted'` reads `1<h1>QA page</h1>`. After `browser reload` and reopening, `control-openhands browser attr 'testid=files-tab-content-mode-toggle-option-plain' aria-checked` is `true`. Switch back with `testid=files-tab-content-mode-toggle-option-rich`.+- **Rich/Plain (`F08.files-rich-plain`).** Click `testid=file-tree-file-qa-notes.md`; `testid=file-content-viewer-markdown` exists. Click `testid=file-tree-file-qa-page.html`, then run `control-openhands browser attr 'testid=file-content-viewer-iframe' sandbox` (`allow-same-origin`) and `control-openhands browser eval "document.querySelector('[data-testid=file-content-viewer-iframe]').contentDocument.body.innerText"` (`QA page`). Run `control-openhands browser click 'testid=files-tab-content-mode-toggle-option-plain'`; `control-openhands browser text 'testid=files-tab-content >> testid=file-content-viewer-highlighted'` reads `1<h1>QA page</h1>`. After `browser reload` and reopening, `control-openhands browser attr 'testid=files-tab-content-mode-toggle-option-plain' aria-checked` is `true`. Switch back with `testid=files-tab-content-mode-toggle-option-rich`. Highlighted code keeps its tokens inline: a `.toml` file's section headers (`[build-system]`, `[project]`) each stay on one line in Rich mode (`control-openhands browser eval "[...document.querySelectorAll('[data-testid=files-tab] .token.table')].map(e=>getComputedStyle(e).display)"` lists `inline` for each).- **Open in new window (`F08.files-open-new-window`).** With `qa-page.html` selected, run `control-openhands browser click 'testid=files-tab-open-in-new-window'` and `control-openhands browser tabs`. Page 1's URL is `/api/conversations/<id>/workspace/qa-page.html?v=<n>`. Run `control-openhands browser tab 1`, `control-openhands browser text 'body'` (`QA page`) and `control-openhands browser close-tab 1`.- **Open-file strip (`F08.files-open-tabs`).** With several files opened, run `control-openhands browser attr 'testid=file-quick-row-item-qa-page.html' title` (the path). Select `qa-notes.md` with `control-openhands browser click 'testid=file-quick-row-item-qa-notes.md'`, then close it with `control-openhands browser click 'testid=file-quick-row-close-qa-notes.md'`. `control-openhands browser eval "[...document.querySelectorAll('[data-testid^=file-quick-row-item-]')].filter(e=>e.getAttribute('aria-selected')==='true').map(e=>e.dataset.testid)"` names the neighbour. Run `browser reload`, reopen the drawer and `control-openhands browser text 'testid=file-quick-row'`: the same files remain. Opening a different conversation (see Commits empty state) shows no quick-row items: `browser text 'testid=file-quick-row'` is empty. `browser testids` omits strip tabs scrolled out of view; count them with `browser eval "[...document.querySelectorAll('[data-testid^=file-quick-row-item-]')].length"`.- **No selection (`F08.files-no-selection`).** Close every open file with its `file-quick-row-close-<path>` button. `control-openhands browser text 'testid=files-tab-content'` reads `Select a file to view its contents`, and `control-openhands browser count 'testid=files-tab-content-mode-toggle'` is `0`.@@ -214,7 +214,7 @@ Preconditions:- `browser click ... --observe` accepts the usual selector syntax (`testid=...`).- Two `.cursor-ew-resize` grips sit in the conversation page (chat/drawer, then the Files tree); scope the tree one with `testid=files-tab-tree-resize-handle`.- At phone width, chat path links (and the agent's `canvas_ui_control` tab and file requests, through `src/services/canvas-ui.ts`) navigate to `/conversations/<id>/panel` (#18045); at desktop width they open the side drawer and the URL does not change. A `/panel` URL above 1024 px redirects to `/conversations/<id>`, so a bullet that leaves the phone viewport on `/panel` ends on the chat route.-- If the agent's terminal resets mid-command ("terminal session was reset", for example after a `restart`), the agent reruns the command. Check the workspace with `git status` through a second prompt rather than trusting the first observation. Runs no longer share a tmux server (OpenHands/OpenHands#17946).+- If the agent's terminal resets mid-command ("terminal session was reset", for example after a `restart`), the agent reruns the command. Check the workspace with `git status` through a second prompt rather than trusting the first observation. Each instance's agent server keeps its tmux sockets in its own `TMUX_TMPDIR` (see F06 Gotchas), so only this run's own agent server going away can reset its terminal; a command after this run's `restart` ran normally on 2026-10-17.- Known issue: [#17567](https://github.com/OpenHands/OpenHands/issues/17567) (Monaco disposed-model page error on diff unmount; it also fires on collapse and on a diff→old switch).- Known issue [#18108](https://github.com/OpenHands/OpenHands/issues/18108) (`F08.files-tree`): Files tab shows "No files in workspace" on cloud when the agent writes outside /workspace/project. Live prerequisite: An authenticated Enterprise Cloud sandbox, no repository selected, with files outside its default working directory.- Known issue [#17773](https://github.com/OpenHands/OpenHands/issues/17773) (`F08.files-tree`): Files tab always shows "No files in workspace" on Windows local backend. Live prerequisite: A Windows local Agent Server using cmd.exe; this run is macOS.diff --git a/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md b/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.mdindex 5fb585c2cf..f7a52db5aa 100644--- a/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md+++ b/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md@@ -10,7 +10,7 @@ the collapsed rail, the command menu and several deep links (LLM banner, gettingstarted checklist, composer pickers). The individual pages are other families(F10–F16; Secrets is F14).-Source: `src/routes.ts`, `src/routes/settings.tsx`, `src/routes/settings-index.tsx`, `src/routes/agent-settings.tsx`, `src/constants/settings-nav.tsx`, `src/hooks/use-settings-nav-items.ts`, `src/utils/settings-utils.ts`, `src/components/features/settings/settings-layout.tsx`, `src/components/features/settings/settings-desktop-sidebar.tsx`, `src/components/shared/compact-section-navigation.tsx`, `src/components/features/settings/settings-mobile-hub.tsx`, `src/components/features/settings/settings-mobile-drawer.tsx`, `src/components/features/settings/settings-navigation.tsx`, `src/components/features/settings/mobile-header.tsx`, `src/components/features/settings/backend-synced-settings-badge.tsx`, `src/components/features/settings/agent-canvas-update-card.tsx`, `src/components/features/settings/agent-canvas-version-tile.tsx`, `src/utils/mobile-section-nav.ts`, `src/components/features/command-menu/command-menu-items.tsx`.+Source: `src/routes.ts`, `src/routes/settings.tsx`, `src/routes/settings-index.tsx`, `src/routes/agent-settings.tsx`, `src/constants/settings-nav.tsx`, `src/hooks/use-settings-nav-items.ts`, `src/utils/settings-utils.ts`, `src/components/features/settings/settings-layout.tsx`, `src/components/features/settings/settings-desktop-sidebar.tsx`, `src/components/shared/compact-section-navigation.tsx`, `src/components/features/settings/settings-mobile-hub.tsx`, `src/components/features/settings/settings-mobile-drawer.tsx`, `src/components/features/settings/settings-navigation.tsx`, `src/components/features/settings/mobile-header.tsx`, `src/components/features/settings/backend-synced-settings-badge.tsx`, `src/components/features/settings/agent-canvas-update-card.tsx`, `src/components/features/settings/agent-canvas-version-tile.tsx`, `src/utils/mobile-section-nav.ts`, `src/components/features/command-menu/command-menu-items.tsx`, `src/components/features/settings/brand-button.tsx`.## Sub-featuresdiff --git a/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md b/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.mdindex 8393f89f28..a4c786dc74 100644--- a/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md+++ b/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md@@ -35,7 +35,7 @@ Source: `src/routes/llm-settings.tsx`, `src/components/features/settings/llm-pro- `F10.model-picker`: the Basic provider and model comboboxes group options as Verified Models and Other Models; changing provider clears the model; the OpenHands provider shows account and key help.- `F10.provider-list-complete`: the Basic provider combobox offers every provider the backend reports or has verified models for (over a hundred, `xai` included); picking a late-sorting one fills the field and loads its models.- `F10.cloud-provider-pagination`: on a Cloud organization backend the provider list is fetched from `providers/search` page by page, so providers past the first page (`xai`, `openrouter`) are offered too (blocked here).-- `F10.edit`: Edit loads the profile (`Editing profile "<name>" - save to apply changes`) with the stored key masked; Save stays disabled until something changes; a changed name renames before saving; the stored key is kept.+- `F10.edit`: Edit loads the profile (`Editing profile "<name>" - save to apply changes`) with the stored key masked; Save stays disabled until something changes; a changed name renames before saving; the stored key is kept; a save the Agent Server refuses toasts the server's reason and keeps the editor open.- `F10.basic-save-keeps-base-url`: re-saving a profile from the Basic view without changing its model keeps the Base URL that view hides.- `F10.edit-default-reapply`: saving the Default profile re-applies it to the agent's LLM settings.- `F10.editor-discard`: Back and Cancel return to the list without saving and without an unsaved-changes prompt.@@ -85,12 +85,13 @@ Preconditions:- **Rename (`F10.rename`).** Open the `QA_pro-copy-copy` row menu and click `testid=profile-actions-menu >> testid=profile-rename`. `control-openhands browser value 'testid=rename-profile-input'` is `QA_pro-copy-copy` and `browser eval "document.activeElement.dataset.testid"` is `rename-profile-input`. `control-openhands browser fill 'testid=rename-profile-input' '-bad'` makes `control-openhands browser enabled 'role=dialog[name="Rename Profile"] >> role=button[name="Rename"]'` `false` and `control-openhands browser attr 'testid=rename-profile-rule' class` contains `text-danger` (the theme danger colour, #18199). Filling the old name and clicking Rename closes the modal with no toast (`browser count 'testid=rename-profile-modal'` is `0`); `control-openhands browser click 'role=dialog[name="Rename Profile"] >> role=button[name="Cancel"]'` also closes it. To rename, reopen, fill `QA_renamed`, `control-openhands browser press Enter`, `control-openhands browser wait-text 'Profile renamed to "QA_renamed"'`, reload, and the count of `'[data-testid=profile-row]:has([title="QA_renamed"])'` is `1` while the old name's is `0`. Renaming the Default row (`QA_pro` → `QA_pro2`, submit with `testid=rename-profile-submit`) keeps its badge (`browser text '[data-testid=profile-row]:has([data-testid=profile-active-badge])'` reads `QA_pro2 … Default`) and `active_profile` follows.- **Rename onto an existing name (`F10.rename-conflict`).** The Rename step above already renamed `QA_pro-copy-copy` and `QA_pro`, so use the remaining fixtures: open the `QA_pro-copy` row menu, click `testid=profile-actions-menu >> testid=profile-rename`, fill `testid=rename-profile-input` with `QA_pro2` and click `testid=rename-profile-submit`. The modal stays open (`count 'testid=rename-profile-modal'` is `1`), `control-openhands browser toasts` has the server's message `Profile 'QA_pro2' already exists` (no `HTTP request failed` prefix, no JSON), and `browser screenshot --feature F10.rename-conflict --name toast` shows that toast above the open modal. Close it with `control-openhands browser click 'role=dialog[name="Rename Profile"] >> role=button[name="Cancel"]'`.- **Edit (`F10.edit`).** Open the `QA_renamed` row menu and click `testid=profile-actions-menu >> testid=profile-edit`. `browser text 'testid=profile-editor-title'` is `Edit LLM Profile`, `browser text 'testid=profile-editor-description'` is `Editing profile "QA_renamed" - save to apply changes`, `browser attr 'testid=llm-api-key-input' placeholder` is `<hidden>`, `browser count 'testid=set-indicator'` is `1` and `browser enabled 'testid=save-profile-btn'` is `false` (screenshot `--feature F10.edit --name loaded`). Filling another profile's name (`QA_flash`) sets `aria-invalid` `true` with the taken-name message of `F10.name-validation` and keeps Save disabled; refilling `QA_renamed` clears it. Run `control-openhands browser click 'testid=llm-model-input'`, `control-openhands browser click 'testid=model-item-deepseek-flash'` (Save turns `true`), `control-openhands browser fill 'testid=profile-name-input' QA_edited`, Save, and `control-openhands browser wait-text 'Profile "QA_edited" updated' --timeout 60000`. After a reload `browser text '[data-testid=profile-row]:has([title="QA_edited"])'` reads `QA_edited deepseek/deepseek-flash` and `QA_renamed` is gone. The save passed validation without retyping the key, which proves the stored key was kept.+- **Refused save shows the server's reason (`F10.edit`).** Arrange a disposable dummy-key loopback profile (`QA_DUMMY=sk-qa-dummy control-openhands llm set --profile QA_LOOPBACK --model openai/gpt-4o --api-key-env QA_DUMMY --base-url http://127.0.0.1:1/v1 --no-activate --no-validate`), `browser goto /settings/llm`, open its row menu and Edit, `control-openhands browser click 'testid=sdk-section-all-toggle'` and `control-openhands browser fill 'testid=sdk-settings-llm.max_input_tokens' 16000`. Run `browser network --clear`, click `testid=save-profile-btn` and read `browser toasts --history`: one error toast with the server's reason, `The configured model has a context window of 16,000 tokens, which is below the minimum of 16,384 tokens required for OpenHands to function properly. …`; `browser network --filter /api/` shows `POST /api/profiles/QA_LOOPBACK/validate` and `POST /api/profiles/QA_LOOPBACK` each answering `500` once ([OpenHands/software-agent-sdk#6026](https://github.com/OpenHands/software-agent-sdk/issues/6026) tracks the status code). The editor stays open (`browser count 'testid=save-profile-btn'` is `1`) at desktop and phone (`--feature F10.edit --name server-reason-<viewport>`), and `api GET /api/profiles/QA_LOOPBACK` keeps `max_input_tokens` `null`. Delete `QA_LOOPBACK` through its row menu.- **Basic re-save keeps a hidden Base URL (`F10.basic-save-keeps-base-url`).** `QA_adv` carries `base_url` `https://api.deepseek.com` (Custom model above). Open its row menu, click `testid=profile-actions-menu >> testid=profile-edit`, then `control-openhands browser click 'testid=sdk-section-advanced-toggle'`: `control-openhands browser value 'testid=base-url-input'` is `https://api.deepseek.com`. Run `control-openhands browser click 'testid=sdk-section-basic-toggle'`: `control-openhands browser count 'testid=base-url-input'` is `0` (the view hides it), `control-openhands browser value 'testid=llm-model-input'` is `deepseek-flash` and `control-openhands browser enabled 'testid=save-profile-btn'` is `false` (`control-openhands browser screenshot --feature F10.basic-save-keeps-base-url --name basic-view`). Without touching the model, run `control-openhands browser fill 'testid=llm-api-key-input' --value-env DEEPSEEK_API_KEY` (Save turns `true`), `control-openhands browser click 'testid=save-profile-btn' --observe 'testid=save-profile-btn' --observe-ms 4000` (`Save`, `Validating...`, `Saving...`) and `control-openhands browser wait-text 'Profile "QA_adv" updated' --timeout 60000`. Then `control-openhands api GET /api/profiles/QA_adv --pick config.base_url` is still `https://api.deepseek.com` and `control-openhands api GET /api/profiles/QA_adv --pick config.model` is `deepseek/deepseek-flash`; after `control-openhands browser reload` the same GETs agree. Without a key, arrange `QA_DUMMY_KEY=qa control-openhands llm set --profile QA_base --model deepseek/deepseek-chat --api-key-env QA_DUMMY_KEY --base-url https://api.deepseek.com --no-validate --no-activate` (it prints `activated` `false` and `control-openhands api GET /api/profiles --pick active_profile` is unchanged: the Default stays where it is and the editors below inherit it, as long as some profile is Default; on a run where `QA_base` is the only profile the page makes it Default at the next load, see Gotchas) and `control-openhands browser reload`, then drive the same steps on `QA_base`'s row with its own readbacks: its editor opens on Advanced (the Base URL is a non-default value) with `control-openhands browser value 'testid=base-url-input'` `https://api.deepseek.com`, Basic hides it (`count` `0`), `control-openhands browser value 'testid=llm-model-input'` is `deepseek-chat` and `control-openhands browser enabled 'testid=save-profile-btn'` is `false`; fill the key with `QA_DUMMY_KEY=qa-dummy-key control-openhands browser fill 'testid=llm-api-key-input' --value-env QA_DUMMY_KEY` (Save turns `true`) and Save: the observed states are `Save`, `Validating...` (`[disabled] [busy]`), `Save` within a second, `control-openhands browser toasts` has the provider's message `litellm.BadRequestError: DeepseekException - {"error":{"message":"Authentication Fails, Your api key: ****-key is invalid …`, `browser text 'testid=profile-editor-title'` is still `Edit LLM Profile`, `control-openhands api GET /api/profiles/QA_base --pick config.base_url` is still `https://api.deepseek.com` and `--pick config.model` is `deepseek/deepseek-chat`; record `blocked` with `DEEPSEEK_API_KEY` as the prerequisite, leave with `control-openhands browser click 'testid=cancel-profile-btn'` and remove the arranged profile with `control-openhands api DELETE /api/profiles/QA_base --write` (arrange, not proof).- **Leave without saving (`F10.editor-discard`).** Edit `QA_adv`, `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser fill 'testid=sdk-settings-llm.top_p' 0.5` (Save turns enabled), then `control-openhands browser click 'testid=back-to-profiles'`. The list returns at once (`browser count 'testid=add-llm-profile'` is `1`) with no prompt (`control-openhands browser events --kinds dialog` is empty), and reopening Edit → All shows `top_p` empty. `testid=cancel-profile-btn` behaves the same.- **Schema field validation and re-apply (`F10.schema-validation`, `F10.edit-default-reapply`).** Edit the Default profile and run `control-openhands browser click 'testid=sdk-section-all-toggle'`. `control-openhands browser fill 'testid=sdk-settings-llm.temperature' 5`, Save, then `control-openhands browser wait-text 'must be at most 2'` (toast `Temperature must be at most 2`). Clear it, fill `testid=sdk-settings-llm.top_p` with `3`: `Top P must be at most 1`. Clear it, fill `testid=sdk-settings-llm.extra_headers` with `{bad`: `Invalid JSON for Extra Headers`. Clear that, fill temperature `0.3`, Save, and `wait-text 'Profile "<name>" updated'`. `control-openhands api GET /api/settings` then has `agent_settings.llm.temperature` `0.3`, and reopening Edit → All shows `0.3`.- **New profile prefill (`F10.create-prefill`).** With that Default (temperature `0.3`) run `control-openhands browser click 'testid=add-llm-profile'`. The editor opens on All (`browser eval "[...document.querySelectorAll('[role=tab]')].map(t=>t.innerText+':'+t.getAttribute('aria-selected')).join(',')"` is `Basic:false,Advanced:false,All:true`), `browser value 'testid=llm-custom-model-input'` is the backend default `openai/gpt-5.6-sol`, and `control-openhands assert --field value --equals 0.3 -- browser value 'testid=sdk-settings-llm.temperature'` passes: the temperature is inherited from the Default. Run `control-openhands browser fill 'testid=llm-custom-model-input' deepseek/deepseek-flash`, `browser fill 'testid=profile-name-input' QA_inherit`, fill the key as in Create, `control-openhands browser scroll 'testid=sdk-settings-llm.temperature'`, `control-openhands browser screenshot --feature F10.create-prefill --name inherited-before-save`, Save and `wait-text 'Profile "QA_inherit" created' --timeout 60000`. The saved profile has the temperature the form showed: `control-openhands api GET /api/profiles/QA_inherit --pick config.temperature` is `0.3`. After `browser reload`, open `QA_inherit`'s row menu (`'[data-testid=profile-row]:has([title="QA_inherit"]) >> testid=profile-menu-trigger'`), click `testid=profile-actions-menu >> testid=profile-edit` and `testid=sdk-section-all-toggle`, `control-openhands browser scroll 'testid=sdk-settings-llm.temperature'` and `control-openhands assert --field value --equals 0.3 -- browser value 'testid=sdk-settings-llm.temperature'` passes (screenshot `--feature F10.create-prefill --name edit-after-create`); leave with `testid=cancel-profile-btn`. Repeat at `browser viewport phone` after `control-openhands api DELETE /api/profiles/QA_inherit --write` (the desktop pass already took the name); the same holds.- **Link to a provider connection (`F10.connection-link`, `F10.list-grouped`).** Arrange a connection through the F11 form: `control-openhands browser click 'testid=add-provider-connection'`, `control-openhands browser fill 'testid=provider-connection-name-input' QA_ds_conn`, `control-openhands browser click 'testid=provider-connection-provider-input'`, `control-openhands browser fill 'testid=provider-connection-provider-input' deepseek`, `control-openhands browser click 'testid=provider-item-deepseek'`, `control-openhands browser fill 'testid=provider-connection-api-key-input' --value-file <key file>`, `control-openhands browser click 'testid=provider-connection-submit'` and `control-openhands browser wait-text 'Connection "QA_ds_conn" created'`. Edit `QA_adv`, click `testid=sdk-section-basic-toggle`, run `control-openhands browser click 'testid=llm-provider-connection-input'` (options `None`, `QA_ds_conn`) and `control-openhands browser click 'role=option[name="QA_ds_conn"]'`; `browser count 'testid=llm-api-key-input'` is `0`, and on the Advanced tab `browser count 'testid=base-url-input'` is `0`. Picking `role=option[name="None"]` brings the key field back (`1`). Re-select `QA_ds_conn`, then `control-openhands browser click 'testid=save-profile-btn' --observe 'testid=save-profile-btn' --observe-ms 3000`: states `Save`, `Saving...` (no `Validating...`), and `wait-text 'Profile "QA_adv" updated'`. After a reload `control-openhands browser eval "[...document.querySelectorAll('[data-testid=profile-group-header]')].map(e=>e.innerText).join(' | ')"` is `QA_DS_CONN | NOT LINKED` (CSS upper-case) and `control-openhands browser count 'div:has(> [data-testid=profile-group-header]:text-is("QA_ds_conn")) [data-testid=profile-row]:has([title="QA_adv"])'` is `1` (screenshot `--feature F10.list-grouped --name grouped`).-- **Add models from a provider (`F10.add-models`).** Run `control-openhands browser click 'testid=add-models-from-provider'`; `browser enabled 'testid=add-models-submit'` is `false`. Keyless: `control-openhands browser select 'testid=add-models-provider' moonshot` and `control-openhands browser wait 'testid=add-models-select-all'`; `browser text 'testid=add-models-keyless-note'` is `No connection — profiles will need a key added later`, rows start unchecked (`Add 0 profiles`), and `control-openhands browser check 'testid=add-models-select-all'` makes `browser text 'testid=add-models-submit'` read `Add 12 profiles`; uncheck it again. Linked: `control-openhands browser select 'testid=add-models-provider' deepseek`; the connection field (`testid=add-models-connection-field`, options `No connection (keyless)` and `QA_ds_conn`) is pre-bound to `QA_ds_conn`. Run `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-chat'`, `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-v3'`, `control-openhands browser click 'testid=add-models-submit'` and `control-openhands browser wait-text 'Added 2'` (toast `Added 2 profiles`). The modal closes; after a reload `deepseek-chat` and `deepseek-v3` sit under the `QA_DS_CONN` header and `api GET /api/profiles` shows both with a `provider_connection_id`. Preselect entry: `control-openhands browser click 'testid=provider-connection-row >> has-text=QA_ds_conn >> testid=provider-connection-menu-trigger'` (menu: Bulk add, Edit, Delete), `control-openhands browser click 'testid=provider-connection-actions-menu >> testid=provider-connection-add-models'`; provider `deepseek` and the connection are preset, every remaining row is checked (`Add 10 profiles`), and `browser count 'testid=add-models-row-deepseek/deepseek-chat'` is `0` (already added). Close with `control-openhands browser click 'role=dialog[name="Add models as profiles"] >> role=button[name="Cancel"]'`. Known failure (reproduced 2026-10-16, [#18516](https://github.com/OpenHands/OpenHands/issues/18516)): in Light+ the chooser's text is unreadable: the provider select measures 1.00:1, labels, Select all and model names 1.11:1, and a partial save at the 50-profile limit shows Saved at 1.60:1 and Failed at 2.60:1; Solarized Light gives 1.08:1 to 1.23:1.+- **Add models from a provider (`F10.add-models`).** Run `control-openhands browser click 'testid=add-models-from-provider'`; `browser enabled 'testid=add-models-submit'` is `false`. Keyless: `control-openhands browser select 'testid=add-models-provider' moonshot` and `control-openhands browser wait 'testid=add-models-select-all'`; `browser text 'testid=add-models-keyless-note'` is `No connection — profiles will need a key added later` (readable, at least 4.5:1, in every theme; Known failure (reproduced 2026-10-17, [#18546](https://github.com/OpenHands/OpenHands/issues/18546)): the muted note measures 2.58:1 in Solarized Light, 5.58:1 in Light+), rows start unchecked (`Add 0 profiles`), and `control-openhands browser check 'testid=add-models-select-all'` makes `browser text 'testid=add-models-submit'` read `Add 12 profiles`; uncheck it again. Linked: `control-openhands browser select 'testid=add-models-provider' deepseek`; the connection field (`testid=add-models-connection-field`, options `No connection (keyless)` and `QA_ds_conn`) is pre-bound to `QA_ds_conn`. Run `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-chat'`, `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-v3'`, `control-openhands browser click 'testid=add-models-submit'` and `control-openhands browser wait-text 'Added 2'` (toast `Added 2 profiles`). The modal closes; after a reload `deepseek-chat` and `deepseek-v3` sit under the `QA_DS_CONN` header and `api GET /api/profiles` shows both with a `provider_connection_id`. Preselect entry: `control-openhands browser click 'testid=provider-connection-row >> has-text=QA_ds_conn >> testid=provider-connection-menu-trigger'` (menu: Bulk add, Edit, Delete), `control-openhands browser click 'testid=provider-connection-actions-menu >> testid=provider-connection-add-models'`; provider `deepseek` and the connection are preset, every remaining row is checked (`Add 10 profiles`), and `browser count 'testid=add-models-row-deepseek/deepseek-chat'` is `0` (already added). Close with `control-openhands browser click 'role=dialog[name="Add models as profiles"] >> role=button[name="Cancel"]'`. In Light+ and Solarized Light the chooser stays readable: the provider select, labels, Select all and model names, and a partial save's per-row **Saved** and **Failed** outcomes all measure at least 4.5:1 against the modal (arrange a partial save by filling the profile list to 48 through the real profiles API, then add four keyless moonshot rows including `moonshot-v1-8k`, whose 8,192-token window the server refuses, and `moonshot-v1-auto`, which hits the 50-profile limit; the toast reads `Added 2 profiles; 2 failed` and the modal stays open).- **Broken link (`F10.broken-link`).** The local server refuses to delete a referenced connection (row menu → `testid=provider-connection-delete` → `testid=delete-provider-connection-confirm` toasts `Provider connection cannot be deleted while it is referenced by LLM profile(s): ...`), so arrange the state directly: `control-openhands api POST /api/profiles/QA_broken --write --data '{"llm":{"model":"deepseek/deepseek-chat","provider_connection_id":"00000000000000000000000000000000"},"include_secrets":false}'`. Then `control-openhands browser reload` and `control-openhands browser text '[data-testid=profile-row]:has([title="QA_broken"])'` reads `QA_broken deepseek/deepseek-chat Broken link`; `control-openhands browser attr '[data-testid=profile-row]:has([title="QA_broken"]) >> testid=profile-broken-connection-badge' title` is `The linked provider connection was deleted. Edit this profile to re-link it.` (screenshot `--feature F10.broken-link --name badge`). Edit it: `browser value 'testid=llm-provider-connection-input'` is the orphaned id and the listbox offers `None`, `QA_ds_conn` and that id. The row's group header shows the raw id, not a name (see Gotchas).- **ChatGPT subscription, pre-auth (`F10.subscription`).** In a fresh editor run `control-openhands browser click 'testid=sdk-section-basic-toggle'` first (after the schema step the Default profile has a temperature, so a new editor opens on All; see `F10.create-prefill`), then `control-openhands browser click 'testid=llm-auth-type-input'` (options `API key`, `ChatGPT subscription`) and `control-openhands browser click 'role=option[name="ChatGPT subscription"]'`. `browser text 'testid=subscription-status'` is `ChatGPT subscription not connected`, `browser value 'testid=llm-subscription-model-input'` is `gpt-5.5` (list: gpt-5.5, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, gpt-6-astra), and `llm-provider-input` and `llm-api-key-input` are gone (count `0`). Known failure (reproduced 2026-10-13, #17135): run `control-openhands browser fill 'testid=profile-name-input' QA_sub`, `control-openhands browser network --clear`, `control-openhands browser click 'testid=save-profile-btn'` and `control-openhands browser network --bodies --filter '/api/profiles'`. Expected: Save validates and saves the shown `gpt-5.5`. Actual: one `POST /api/profiles/QA_sub/validate` whose body has `"model":"openai/gpt-5.6-sol"`, the API-key form's hidden default, although the dropdown shows `gpt-5.5` (it shows the first subscription model without writing it to the form); `browser toasts --history` has `OpenAI subscription login is required`, because no subscription is connected. Picking a model in `testid=llm-subscription-model-input`, even `gpt-5.5` again, makes the next Save send `POST /api/profiles/QA_sub/validate`, which answers `OpenAI subscription login is required` here (`browser toasts --history`), because no subscription is connected. `control-openhands browser click 'testid=subscription-connect'` and `control-openhands browser wait 'testid=subscription-user-code'`: the card shows the code, `Open sign-in page`, `I've finished signing in` (`subscription-poll`) and Cancel, and `control-openhands browser tabs` lists a second page on `auth.openai.com` (it opens a second or two after the code appears; rerun `browser tabs` if it lists one page). The card polls on its own, so `Sign-in is not complete yet. ...` can already show before you click poll. Close it with `control-openhands browser close-tab 1` and `control-openhands browser tab 0`. `control-openhands browser click 'testid=subscription-poll'` leaves `Sign-in is not complete yet. Finish the browser flow and try again.` on the card, and `control-openhands browser click 'testid=subscription-cancel'` removes the code (`count 'testid=subscription-user-code'` `0`). Switching back with `role=option[name="API key"]` restores provider `OpenAI` and model `gpt-5.6-sol`. Completing sign-in is blocked (needs a ChatGPT Plus/Pro account).- **Delete (`F10.delete`, `F10.delete-default`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_pro-copy"]) >> testid=profile-menu-trigger'` and `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-delete'`. `control-openhands browser snapshot 'role=dialog'` shows `Delete Profile` with `Are you sure you want to delete the profile "QA_pro-copy"? This action cannot be undone.`, and `browser eval "document.activeElement.innerText"` is `Cancel`. `control-openhands browser click 'role=dialog[name="Delete Profile"] >> role=button[name="Cancel"]'` keeps the row (count `1`). Repeat, then `control-openhands browser click 'testid=delete-profile-confirm'`, `control-openhands browser wait-text 'Profile "QA_pro-copy" deleted'`, reload, and the count is `0`. Deleting the Default row the same way leaves exactly one `profile-active-badge` on another profile (`browser text '[data-testid=profile-row]:has([data-testid=profile-active-badge])'`), and `api GET /api/profiles` names it in `active_profile`.@@ -179,4 +180,3 @@ Preconditions:- Known issue [OpenHands/software-agent-sdk#5862](https://github.com/OpenHands/software-agent-sdk/issues/5862): DeepSeek invalid-key401 responses can be classified as bad-request errors. Blocked by this pass's loopback-only provider routing: a real invalid-key validation request to DeepSeek must be authorized to capture its actual 401 shape. No usable key or successful inference is needed, but a loopback connection error does not prove this classification. (`F10.create-validation`).- Known issue [OpenHands/software-agent-sdk#5838](https://github.com/OpenHands/software-agent-sdk/issues/5838): A live LLM switch during a running tool step can be acknowledged before the new LLM is persisted for GET/restart. Blocked: needs a genuine model-issued bounded tool step (for example `sleep 20`) and an approved starting model. Refused-loopback initialization never enters that step. (`F10.default-used`).- Known issue [#17806](https://github.com/OpenHands/OpenHands/issues/17806): A model-only Save must preserve an untouched key. Current SDK 1.55 refused-loopback validation rejects Save before persistence; the stored key marker remains set. Completing Basic/Advanced preserved-key and runtime-switch proof needs an approved route that accepts preflight. Keep dummy traffic on refused loopback until an external route is authorized; a rejected Save does not establish either key loss or successful preservation through Save. (`F10.edit`).-- Known issue [#18548](https://github.com/OpenHands/OpenHands/issues/18548) (`F10.edit`): a save the Agent Server refuses (for example All > Max Input Tokens `16000`, below its 16,384 minimum; validate and save both answer 500, [OpenHands/software-agent-sdk#6026](https://github.com/OpenHands/software-agent-sdk/issues/6026)) shows only `An error occurred` instead of the server's reason; the editor stays open.diff --git a/.agents/skills/verify-openhands/references/feature-map/F14-secrets.md b/.agents/skills/verify-openhands/references/feature-map/F14-secrets.mdindex 0d851600ee..2c4d96da4f 100644--- a/.agents/skills/verify-openhands/references/feature-map/F14-secrets.md+++ b/.agents/skills/verify-openhands/references/feature-map/F14-secrets.md@@ -50,7 +50,7 @@ Preconditions:- **Back discards the draft (`F14.form-back`).** On `/settings/secrets` run `control-openhands browser click 'testid=add-secret-button'` and `control-openhands browser text 'testid=secret-editor-title'` (`Add a Secret`). Fill `testid=add-secret-form >> testid=name-input` with `QA_BACK_SECRET` and `value-input` with `dummy`, then `control-openhands browser click 'testid=back-to-secrets'`. `control-openhands browser count 'testid=add-secret-form'` is `0`, the list is back (`browser count 'testid=secret-item'` > `0`) and `control-openhands api GET /api/settings/secrets` lists no `QA_BACK_SECRET`. Reopening Add shows an empty `name-input` (`browser value`); leave with `testid=back-to-secrets`. The Edit form shows `Edit a Secret` in the same `secret-editor-title`.- **Rename keeps the value (`F14.rename`).** Needs `QA_AGENT_SECRET` from Agent receives the secret. Run `control-openhands browser goto /settings/secrets`, `control-openhands browser click 'testid=secret-item >> has-text=QA_AGENT_SECRET >> testid=edit-secret-button'`, fill `testid=edit-secret-form >> testid=name-input` with `OPENHANDS_AUTOMATION_API_KEY` and click `testid=edit-secret-form >> testid=submit-button`: `control-openhands browser snapshot 'testid=edit-secret-form'` shows the paragraph `Secret already exists` and the form stays open. Fill `name-input` with `QA_RENAMED_SECRET` (value left blank), click submit, `control-openhands browser reload`; `browser count 'testid=secret-item >> has-text=QA_RENAMED_SECRET'` is `1`, the `QA_AGENT_SECRET` count is `0`, and `api GET /api/settings/secrets` lists only the new name. Then run `control-openhands conversation start --prompt "Run exactly this command and reply with only its output: python3 -c \"import os; print(os.environ.get('QA_RENAMED_SECRET') == 'qa-dummy-4821', 'QA_AGENT_SECRET' in os.environ)\"" --wait --timeout 240` and `control-openhands conversation events <id> --kinds ObservationEvent`: the observation is `True False`.- **Replace the value (`F14.edit-value`).** Run `control-openhands browser goto /settings/secrets`, `control-openhands browser click 'testid=secret-item >> has-text=QA_RENAMED_SECRET >> testid=edit-secret-button'` and `control-openhands browser snapshot 'testid=edit-secret-form'` (the value textbox is named `Secret Value (leave blank to preserve the existing value)`). Fill `testid=edit-secret-form >> testid=value-input` with `qa-dummy-7350`; `browser enabled 'testid=edit-secret-form >> testid=submit-button'` is `true`. Click submit; `browser count 'testid=edit-secret-form'` is `0`. Start a conversation with the prompt `Run exactly this command and reply with only its output: python3 -c "import os; print(os.environ.get('QA_RENAMED_SECRET') == 'qa-dummy-7350')"` (quoted as above) and read its `ObservationEvent`: `True`.-- **Agent requests a secret (`F14.agent-request`).** Run `control-openhands browser goto /`, then `control-openhands conversation start --prompt "Call the request_secret tool with name QA_REQ_SECRET and reason 'QA check: a dummy token for verification.' Do not run any other tools. After calling it, end your turn and wait." --wait --timeout 240` (prints `<req-id>`; it leaves the browser on the conversation). `control-openhands conversation events <req-id> --kinds ActionEvent,ObservationEvent` shows the `request_secret` action and the observation `Tool call dispatched to client.`; `control-openhands browser snapshot 'testid=secret-request-card'` reads `region "The agent needs a secret"`, the reason `QA check: a dummy token for verification.`, textbox `Name` holding `QA_REQ_SECRET`, `Value`, `Description Optional`, `Cancel` and `Add secret [disabled]` (`browser screenshot --feature F14.agent-request --name card`). The chat input stays usable while the card is pending, and `control-openhands browser reload` keeps the card with the name still prefilled. Run `control-openhands browser fill 'testid=secret-request-card >> testid=value-input' qa-dummy-5190` and `control-openhands browser click 'testid=secret-request-card >> testid=submit-button'`: the card disappears, `control-openhands api GET /api/settings/secrets` lists `QA_REQ_SECRET`, and after `control-openhands conversation wait <req-id> --fresh --timeout 180` `control-openhands conversation events <req-id> --kinds MessageEvent` holds the user message `[secret-request] {"status":"saved","name":"QA_REQ_SECRET"}`, which the chat does not show (`browser count 'text=[secret-request]'` is `0`). The running conversation now has the value: `control-openhands conversation send <req-id> --prompt "Run exactly this command and reply with only its output: python3 -c \"import os; print(os.environ.get('QA_REQ_SECRET') == 'qa-dummy-5190')\"" --wait --timeout 240`, then the last terminal `ObservationEvent` is `True`. Cancel: `browser goto /`, start a second conversation with the same prompt naming `QA_REQ_CANCEL` and reason `QA cancel check.`, fill `value-input` with `should-not-save` and `control-openhands browser click 'testid=secret-request-card >> testid=cancel-button'`: the card disappears, `api GET /api/settings/secrets` lists no `QA_REQ_CANCEL`, the agent receives `[secret-request] {"status":"not_saved","name":"QA_REQ_CANCEL","guidance":...}`, and a reload does not bring the card back. Already saved: a third conversation asking for `QA_REQ_SECRET` again shows no card (`browser count 'testid=secret-request-card'` `0`) and its events hold `[secret-request] {"status":"saved","name":"QA_REQ_SECRET"}`. Expected: at phone width the card fits the chat column (`insideViewport` `true`) with the title, every field and both buttons visible; run the block below. Return with `browser viewport desktop`. `QA_REQ_SECRET` is deleted in the cleanup below.+- **Agent requests a secret (`F14.agent-request`).** Run `control-openhands browser goto /`, then `control-openhands conversation start --prompt "Call the request_secret tool with name QA_REQ_SECRET and reason 'QA check: a dummy token for verification.' Do not run any other tools. After calling it, end your turn and wait." --wait --timeout 240` (prints `<req-id>`; it leaves the browser on the conversation). `control-openhands conversation events <req-id> --kinds ActionEvent,ObservationEvent` shows the `request_secret` action and the observation `Tool call dispatched to client.`; `control-openhands browser snapshot 'testid=secret-request-card'` reads `region "The agent needs a secret"`, the reason `QA check: a dummy token for verification.`, textbox `Name` holding `QA_REQ_SECRET`, `Value`, `Description Optional`, `Cancel` and `Add secret [disabled]` (`browser screenshot --feature F14.agent-request --name card`). The chat input stays usable while the card is pending, and `control-openhands browser reload` keeps the card with the name still prefilled. Run `control-openhands browser fill 'testid=secret-request-card >> testid=value-input' qa-dummy-5190` and `control-openhands browser click 'testid=secret-request-card >> testid=submit-button'`: the card disappears, `control-openhands api GET /api/settings/secrets` lists `QA_REQ_SECRET`, and after `control-openhands conversation wait <req-id> --fresh --timeout 180` `control-openhands conversation events <req-id> --kinds MessageEvent` holds the user message `[secret-request] {"status":"saved","name":"QA_REQ_SECRET"}`, which the chat does not show (`browser count 'text=[secret-request]'` is `0`). The running conversation now has the value: `control-openhands conversation send <req-id> --prompt "Run exactly this command and reply with only its output: python3 -c \"import os; print(os.environ.get('QA_REQ_SECRET') == 'qa-dummy-5190')\"" --wait --timeout 240`, then the last terminal `ObservationEvent` is `True`. Cancel: `browser goto /`, start a second conversation with the same prompt naming `QA_REQ_CANCEL` and reason `QA cancel check.`, fill `value-input` with `should-not-save` and `control-openhands browser click 'testid=secret-request-card >> testid=cancel-button'`: the card disappears, `api GET /api/settings/secrets` lists no `QA_REQ_CANCEL`, the agent receives `[secret-request] {"status":"not_saved","name":"QA_REQ_CANCEL","guidance":...}`, and a reload does not bring the card back. Already saved: a third conversation asking for `QA_REQ_SECRET` again shows no card (`browser count 'testid=secret-request-card'` `0`) and its events hold `[secret-request] {"status":"saved","name":"QA_REQ_SECRET"}`. At phone width the card fits the chat column (`insideViewport` `true`, `x` 16 and `width` 358 at 390 px, no page overflow) with the title, every field and both buttons; the block below checks it. Return with `browser viewport desktop`. `QA_REQ_SECRET` is deleted in the cleanup below.```shcontrol-openhands browser viewport desktopdiff --git a/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md b/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.mdindex 86b6ee1daf..ae14563277 100644--- a/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md+++ b/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md@@ -58,7 +58,7 @@ Preconditions:- **Save and reload (`F15.condenser-save`).** Fill Max Size with `120`, then `control-openhands browser click 'testid=save-button' --observe 'testid=save-button' --observe-ms 3000` (observed states `Save Changes` → `Saving...` → `Save Changes`) and `control-openhands browser toasts` (`Settings saved. For old conversations, you will need to stop and restart the conversation to see the changes.`). `control-openhands browser enabled 'testid=save-button'` is `false`. Run `control-openhands browser reload`, `control-openhands browser snapshot 'role=tablist'` (`All` [selected]: the page opens on the tier that shows the overridden field) and `control-openhands browser value 'testid=sdk-settings-condenser.max_size'` (`120`). `control-openhands api GET /api/settings` shows `max_size` `120`. Keep it for the next bullet.- **Dependent fields (`F15.condenser-dependents`).** On the All tier run `control-openhands browser click 'testid=condenser-settings-screen >> text=Enable Memory Condensation'`; the switch reads `false` and `control-openhands browser count 'testid=condenser-settings-screen >> input'` is `2` (the switch and Condenser Kind). Click the label again: the count is `8` and `browser enabled 'testid=save-button'` is `false`.- **Basic save keeps hidden fields (`F15.view-scoped-save`).** With Max Size stored as `120`, run `control-openhands browser click 'testid=sdk-section-basic-toggle'`, `control-openhands browser click 'testid=condenser-settings-screen >> text=Enable Memory Condensation'` and `control-openhands browser click 'testid=save-button'`. `control-openhands api GET /api/settings` now shows `enabled` `false` and `max_size` `120`. Known failure (reproduced 2026-10-06): `max_size` is `240`, because the save sends defaults for every field Basic does not show (#18049). After `browser reload` the page opens on Basic with the switch off. Restore: click the label again, `browser click 'testid=save-button'`, `browser reload`; the switch reads `true` and the API shows `enabled` `true` (with `max_size` `240` today, from the reset above).-- **Invalid values (`F15.save-errors`).** On the All tier run `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 1.5`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser toasts` (`Expected an integer value, received: 1.5`; no request is sent). Then wait for the toast to close, fill `-5`, `control-openhands browser network --clear`, `control-openhands browser click 'testid=save-button' --observe 'testid=save-button' --observe-ms 2500` (`Saving...` for about 1.5 s), `control-openhands browser toasts` (`HTTP request failed (422 Unprocessable Entity): {"detail":"Settings validation failed"}`) and `control-openhands browser network` (three `PATCH /api/settings`). The API still shows `max_size` `240`; `browser reload` drops the bad draft.+- **Invalid values (`F15.save-errors`).** On the All tier run `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 1.5`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser toasts` (`Expected an integer value, received: 1.5`; no request is sent). Then wait for the toast to close, fill `-5`, `control-openhands browser network --clear`, `control-openhands browser click 'testid=save-button' --observe 'testid=save-button' --observe-ms 2500` (`Saving...` only briefly), `control-openhands browser toasts` (`Settings validation failed`) and `control-openhands browser network` (one `PATCH /api/settings` answering 422; a 4xx is not retried). The API still shows `max_size` `240`; `browser reload` drops the bad draft.- **Condenser Kind (`F15.condenser-kind`).** On the All tier run `control-openhands browser click 'testid=sdk-settings-condenser.condenser_kind'`, `control-openhands browser snapshot 'role=listbox'` (options `LLM Summarizer` [selected] and `None`), `control-openhands browser click 'role=listbox >> role=option[name="None"]'` and `control-openhands browser click 'testid=save-button'`. After `browser reload` the page opens on All, `browser value 'testid=sdk-settings-condenser.condenser_kind'` is `None`, and `control-openhands api GET /api/settings --pick agent_settings.condenser` is just `{"enabled": true, "condenser_kind": "no_op"}` (the labels are Canvas copy; the stored values are unchanged). The summarizer fields should hide while **None** is selected. Known failure (reproduced 2026-10-06): Max Size, Max Tokens, Keep First Events, Minimum Progress, Hard Reset Retries and Hard Reset Scaling stay with their defaults, and `browser fill 'testid=sdk-settings-condenser.max_size' 100` then Save gives `Settings saved` while the API still shows only `enabled` and `condenser_kind` and a reload shows `240` again (#18049; `browser screenshot --feature F15.condenser-kind --name none-shows-summarizer-fields`). Restore by choosing `LLM Summarizer` the same way and saving: after reload the page opens on Basic and the API shows all eight defaults again (`condenser_kind` `llm_summarizing`).- **Agent Context (`F15.agent-context-page`).** Run `control-openhands browser click 'testid=sidebar-settings-/settings/agent-context' --expect-url '/settings/agent-context(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Let the agent keep notes and recall them in new conversations.`), `control-openhands browser count 'role=tablist'` (`0`) and `control-openhands browser snapshot 'testid=agent-context-settings-screen'` (text `Persistent Agent Memory`, the paragraph about `.openhands/memory/`, `button "Save Changes" [disabled]`).- **Persistent memory persists (`F15.agent-context-save`).** Run `control-openhands browser click 'testid=agent-context-settings-screen >> text=Persistent Agent Memory'`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser reload` and `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-agent_context.load_memory\"]').checked"` (`true`; API `agent_context.load_memory` `true`). Restore: click the label, save, reload; the value is `false`.@@ -94,10 +94,10 @@ Preconditions:- Saving is tier-scoped: Basic or Advanced saves reset every field the tier hides to its schema default (`F15.view-scoped-save`). Make all edits on All when other non-default values must survive.- The page opens on the least detailed tier that shows every stored override (any minor override → All). Do not assume Basic after a reload.- Condenser Kind **None** (`no_op`) still shows the summarizer fields with defaults, although the stored condenser no longer has them, and edits to them are accepted (`Settings saved`) but dropped while **None** is stored. #18049 covers hiding fields that do not apply to the selected kind.-- A server-side validation error is retried three times (about 1.5 s of `Saving...`) before the toast, and the toast only says `Settings validation failed` without naming the field (#17932).+- A server-side validation error (422) is not retried: one `PATCH /api/settings`, `Saving...` for a moment, then the toast `Settings validation failed`, which does not name the field.- Condenser Kind is served by the Agent Server's settings schema as a user field. Canvas shows its own choice labels and help text rather than raw `llm_summarizing`/`no_op` values or reStructuredText. The dropdown (`F15.condenser-kind`) remains a user setting so additional kinds can be selected. [OpenHands/software-agent-sdk#5495](https://github.com/OpenHands/software-agent-sdk/issues/5495), the schema's developer-facing description, remains an upstream issue.- Toasts close after a few seconds; read them right after the click, or assert the stored value with `api GET /api/settings`.-- Under confirmation mode the first confirmed command sometimes returned `The terminal session was reset because the underlying tmux server/session disappeared...` while several runs shared one tmux server (OpenHands/OpenHands#17946); `launch` now isolates them, so treat a reset as a finding. The agent then asks to run it again; a confirmed command took up to two minutes to produce its observation. Confirm again and wait with `conversation wait`, not a fixed sleep.+- Each instance's agent server keeps its tmux sockets in its own `TMUX_TMPDIR` (see F06 Gotchas), so a confirmed command that returns `The terminal session was reset because the underlying tmux server/session disappeared...` is a finding, not interference from another run; after one, the agent asks to run it again. A confirmed command took up to two minutes to produce its observation. Confirm again and wait with `conversation wait`, not a fixed sleep.- The confirmation buttons have test ids (`action-confirm-button`, `action-reject-button`) but their accessible names are `Confirm action` / `Reject action`, not the visible `Continue` / `Cancel`: `role=button[name*="Continue"]` finds nothing.- Basic-tier saves on Verification reset Security Analyzer to `llm` (it is an Advanced field). That is a convenient restore, and a trap if you meant to keep `None`.- The memory instructions are only visible through the conversation's first event; `conversation events` truncates text, so read `api GET /api/conversations/<id>/events/search?limit=1` and search the JSON.diff --git a/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md b/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.mdindex 0ecc62b391..0ab3188723 100644--- a/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md+++ b/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md@@ -111,7 +111,7 @@ Preconditions:## Gotchas-- Server errors on MCP mutations show one toast with the server's message, or `An error occurred` when the body has none. A custom server's **Save Server** or **Add Server** action whose pre-save connection test fails keeps the editor open. To check backend loss, stop the owned Agent Server with the editor open, click Save, then restore it with `control-openhands restart`; background settings polls can add separate error toasts, so count only the toast caused by the click. A failed add, edit or delete is retried three times (`withRetry` in `settings-service.api.ts`, 0.5 s and 1 s apart) before its single toast; that retry behavior is open [#17932](https://github.com/OpenHands/OpenHands/issues/17932).+- Server errors on MCP mutations show one toast with the server's message, or `An error occurred` when the body has none. A custom server's **Save Server** or **Add Server** action whose pre-save connection test fails keeps the editor open. To check backend loss, stop the owned Agent Server with the editor open, click Save, then restore it with `control-openhands restart`; background settings polls can add separate error toasts, so count only the toast caused by the click. That Save fails at its pre-save connection test (one `GET /server_info` `502`, toast `An error occurred`, editor open) and sends no `PATCH`. A delete or enable toggle with no answer or a 5xx answer is sent three times in all (`withRetry` in `src/api/with-retry.ts`; backend stopped: three `502`s, 0.5 s and 1 s apart) before its single toast; a 4xx answer other than 408 or 429 is not retried: the stale delete in `F17.delete-server` sends one `DELETE` that answers `404`.- Desktop `/customize` is a client-side redirect: `browser url` may still read `/customize` for a moment; use `wait-url '/mcp(\?|$)'`.- From 768 to 1023 px the sidebar rail stays and the phone top bar is gone. The compact Customize selector above each detail page provides sibling navigation; the rail's **Customize** link (`sidebar-skills-link`, unscoped at this width) still returns to the hub.- On a phone the drawer and the hidden desktop rail both render `sidebar-skills-link`; scope it with `testid=sidebar-mobile-drawer >> ...` or the click fails with a strict-mode error, even though `browser testids` lists it once.diff --git a/.agents/skills/verify-openhands/references/feature-map/F19-plugins.md b/.agents/skills/verify-openhands/references/feature-map/F19-plugins.mdindex c828697689..ce6b0f8abe 100644--- a/.agents/skills/verify-openhands/references/feature-map/F19-plugins.md+++ b/.agents/skills/verify-openhands/references/feature-map/F19-plugins.md@@ -81,8 +81,8 @@ Preconditions:- **Review screen (`F19.launch-review`).** Run `control-openhands browser goto '/launch?plugins=W3sic291cmNlIjoiZ2l0aHViOk9wZW5IYW5kcy9leHRlbnNpb25zIiwicmVwb19wYXRoIjoicGx1Z2lucy9jaXR5LXdlYXRoZXIifSx7InNvdXJjZSI6Imh0dHBzOi8vZ2l0aHViLmNvbS9PcGVuSGFuZHMvZXh0ZW5zaW9ucy5naXQiLCJyZXBvX3BhdGgiOiJwbHVnaW5zL21hZ2ljLXRlc3QiLCJyZWYiOiJtYWluIn1d'` (two plugins from `OpenHands/extensions`, one as an HTTPS git URL) and `control-openhands browser snapshot 'testid=plugin-launch-modal'`: the heading `Launch Plugin` (`Launch` once); under `Plugins`, the text `city-weather OpenHands/extensions/ plugins/city-weather magic-test OpenHands/extensions/ plugins/magic-test@ main` (the screenshot renders `OpenHands/extensions / plugins/magic-test @ main`); and the checkbox `I trust this plugin from OpenHands/extensions with the agent secrets defined in my account.`, which names `OpenHands/extensions` once (`browser screenshot --feature F19.launch-review --name multi`). Message: `control-openhands browser goto '/launch?plugin_source=github%3AOpenHands%2Fextensions&message=%3Cb%3EWeather%3C%2Fb%3E%20for%20QA'` and `control-openhands browser text 'testid=plugin-launch-modal >> p'`: `Weather for QA` (tags stripped). With a 600-character `message` (`/launch?plugin_source=github%3AOpenHands%2Fextensions&message=aaa…`), `control-openhands browser eval "document.querySelector('[data-testid=plugin-launch-modal] p').textContent.length"` is `500`. Close: `control-openhands browser click 'testid=plugin-launch-modal >> testid=close-button' --expect-url '/conversations(\?|$)'`; on a fresh `/launch?...`, `control-openhands browser press Escape` also lands on `/conversations`.- **Parameters (`F19.launch-parameters`).** Run `control-openhands browser goto '/launch?plugins=W3sic291cmNlIjoiZ2l0aHViOk9wZW5IYW5kcy9leHRlbnNpb25zIiwicmVwb19wYXRoIjoicGx1Z2lucy9jaXR5LXdlYXRoZXIiLCJyZWYiOiJtYWluIiwicGFyYW1ldGVycyI6eyJjaXR5IjoiUGFyaXMiLCJkYXlzIjozLCJtZXRyaWMiOnRydWV9fV0%3D'` (city-weather with `{"city":"Paris","days":3,"metric":true}`) and `control-openhands browser snapshot 'testid=plugin-launch-modal'`: a `city-weather` section button, `ref: main`, `path: plugins/city-weather`, textbox `city` (`Paris`), spinbutton `days` (`3`) and a checked checkbox `metric`. `control-openhands browser click 'testid=plugin-section-0'` collapses it (`browser count 'testid=plugin-0-param-city'` is `0`); click again to expand. `control-openhands browser fill 'testid=plugin-0-param-city' Berlin`, `control-openhands browser fill 'testid=plugin-0-param-days' ''` (then `control-openhands browser value 'testid=plugin-0-param-days'` is `0`) and `control-openhands browser uncheck 'testid=plugin-0-param-metric'` are accepted. A link that mixes plugins with and without parameters (`/launch?plugins=W3sic291cmNlIjoiZ2l0aHViOk9wZW5IYW5kcy9leHRlbnNpb25zIiwicmVwb19wYXRoIjoicGx1Z2lucy9tYWdpYy10ZXN0In0seyJzb3VyY2UiOiJnaXRodWI6T3BlbkhhbmRzL2V4dGVuc2lvbnMiLCJyZXBvX3BhdGgiOiJwbHVnaW5zL2NpdHktd2VhdGhlciIsInBhcmFtZXRlcnMiOnsiY2l0eSI6IlBhcmlzIn19XQ%3D%3D`) lists the parameter-less one under `Additional Plugins`. Leave with Escape; do not start this one (see Gotchas).- **Dev format (`F19.launch-dev-params`).** Run `control-openhands browser goto '/launch?plugin_source=github%3AOpenHands%2Fextensions&plugin_ref=main&plugin_repo_path=plugins%2Fmagic-test&message=hi%20QA'` and `control-openhands browser snapshot 'testid=plugin-launch-modal'`: `Launch magic-test`, paragraph `hi QA`, text `magic-test OpenHands/extensions/ plugins/magic-test@ main`.-- **Error states (`F19.launch-errors`).** For each of `/launch`, `/launch?plugins=garbage`, `/launch?plugins=e30%3D` (`{}`), `/launch?plugins=W10%3D` (`[]`) and `/launch?plugins=W3sicmVmIjoibWFpbiJ9XQ%3D%3D` (`[{"ref":"main"}]`) run `control-openhands browser goto '<url>'` and `control-openhands browser text 'testid=launch-error'`. Bare and `[]` read `Unable to Launch` / `No plugins were specified. Please provide at least one plugin to launch.`; the other three read `The plugin configuration is invalid. Please check the URL and try again.` (`browser screenshot --feature F19.launch-errors --name invalid-format`). `control-openhands browser count 'testid=try-again-button'` is `0` in all of them. `control-openhands browser click 'testid=go-home-button' --expect-url '/conversations(\?|$)'` leaves. Known failure (reproduced 2026-10-16, [#18514](https://github.com/OpenHands/OpenHands/issues/18514)): in Light+ and Solarized Light the `Unable to Launch` heading measures 2.60:1 and 2.36:1 against the error card, below 4.5:1.-- **Creation failure (`F19.launch-creation-failed`).** Run `control-openhands browser goto '/launch?plugin_source=github%3AOpenHands%2Fextensions&plugin_repo_path=plugins%2Fmagic-test'`, `control-openhands browser check 'testid=trust-checkbox'`, `control-openhands service stop agent-server`, `control-openhands browser click 'testid=start-conversation-button'`, `control-openhands browser wait 'testid=launch-error' --timeout 60000` and `control-openhands browser text 'testid=launch-error'`: `Unable to Launch`, `Failed to Start Conversation`, `Go Home`, `Try Again` (`browser screenshot --feature F19.launch-creation-failed --name error`). `control-openhands browser click 'testid=try-again-button'` reloads the page; with the server still down the app shows **Manage backends** with the Local backend `Disconnected`. Restore with `control-openhands restart --timeout 240` and `control-openhands browser reload`: the review modal is back and `control-openhands browser eval "document.querySelector('[data-testid=trust-checkbox]').checked"` is `false`. Run `control-openhands doctor` before continuing.+- **Error states (`F19.launch-errors`).** For each of `/launch`, `/launch?plugins=garbage`, `/launch?plugins=e30%3D` (`{}`), `/launch?plugins=W10%3D` (`[]`) and `/launch?plugins=W3sicmVmIjoibWFpbiJ9XQ%3D%3D` (`[{"ref":"main"}]`) run `control-openhands browser goto '<url>'` and `control-openhands browser text 'testid=launch-error'`. Bare and `[]` read `Unable to Launch` / `No plugins were specified. Please provide at least one plugin to launch.`; the other three read `The plugin configuration is invalid. Please check the URL and try again.` (`browser screenshot --feature F19.launch-errors --name invalid-format`). `control-openhands browser count 'testid=try-again-button'` is `0` in all of them. `control-openhands browser click 'testid=go-home-button' --expect-url '/conversations(\?|$)'` leaves. The `Unable to Launch` heading stays at least 4.5:1 against the error card in all five themes at desktop and phone width (it uses the theme's error colour).+- **Creation failure (`F19.launch-creation-failed`).** Run `control-openhands browser goto '/launch?plugin_source=github%3AOpenHands%2Fextensions&plugin_repo_path=plugins%2Fmagic-test'`, `control-openhands browser check 'testid=trust-checkbox'`, `control-openhands service stop agent-server`, `control-openhands browser click 'testid=start-conversation-button'`, `control-openhands browser wait 'testid=launch-error' --timeout 60000` and `control-openhands browser text 'testid=launch-error'`: `Unable to Launch`, `Failed to Start Conversation`, `Go Home`, `Try Again` (`browser screenshot --feature F19.launch-creation-failed --name error`). Heading and **Try Again** stay at least 4.5:1 against their backgrounds, Try Again also on hover (checked in Solarized Light at desktop and phone width; Try Again uses the theme's contrast colours). `control-openhands browser click 'testid=try-again-button'` reloads the page; with the server still down the app shows **Manage backends** with the Local backend `Disconnected`. Restore with `control-openhands restart --timeout 240` and `control-openhands browser reload`: the review modal is back and `control-openhands browser eval "document.querySelector('[data-testid=trust-checkbox]').checked"` is `false`. Run `control-openhands doctor` before continuing.- **Non-ASCII deep link (`F19.launch-unicode`).** Run `control-openhands browser goto '/launch?plugins=W3sic291cmNlIjoiZ2l0aHViOnFhL3LDqXBvIn1d'` (UTF-8 base64 of `[{"source":"github:qa/répo"}]`) and `control-openhands browser text 'testid=plugin-launch-modal >> h2'`. Expected `Launch qa/répo`; today `Launch qa/répo`.- **Phone (`F19.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /plugins`, `control-openhands browser bbox 'testid=skills-plugins-screen'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `control-openhands browser eval "(()=>{const s=document.querySelector('[data-testid=plugins-search-input]').getBoundingClientRect(), f=document.querySelector('[data-testid=plugins-status-filter]').getBoundingClientRect();return {searchBottom:Math.round(s.bottom), filterTop:Math.round(f.top), filterWidth:Math.round(f.width)}})()"`: `filterTop` is below `searchBottom` (stacked) and `filterWidth` is `348`. `control-openhands browser screenshot --feature F19.phone --name list` shows one column of cards. Open `testid=plugin-card-city-weather` and `control-openhands browser bbox 'testid=plugin-detail-modal'` (`insideViewport` `true`, `browser screenshot --feature F19.phone --name detail`), Escape; `control-openhands browser click 'testid=plugins-add-plugin-button'` and `control-openhands browser bbox 'testid=add-plugin-modal'` (`insideViewport` `true`), Escape. The parameters link from above gives `control-openhands browser bbox 'testid=plugin-launch-modal'` `insideViewport` `true` (`browser screenshot --feature F19.phone --name launch`). Hub entry: `control-openhands browser goto /customize`, `control-openhands browser click 'testid=extensions-mobile-hub >> testid=sidebar-extensions-/plugins' --expect-url '/plugins(\?|$)'` and `control-openhands browser click 'testid=sidebar-mobile-back-button' --expect-url '/customize(\?|$)'`. Return with `control-openhands browser viewport desktop`.- **Errors sweep (`F19.page`).** `control-openhands browser errors --app-only` shows no page errors; the only app HTTP errors are the deliberate ones (refresh 500, install 400, 502s while the Agent Server was stopped).diff --git a/.agents/skills/verify-openhands/references/feature-map/F20-canvas-apps.md b/.agents/skills/verify-openhands/references/feature-map/F20-canvas-apps.mdindex 090b020535..ce8af3c5b3 100644--- a/.agents/skills/verify-openhands/references/feature-map/F20-canvas-apps.md+++ b/.agents/skills/verify-openhands/references/feature-map/F20-canvas-apps.md@@ -63,7 +63,7 @@ Preconditions:- **Card (`F20.card`).** Run `control-openhands browser snapshot 'testid=canvas-extension-card-demo-page'`: heading `Demo page`, the source path, switch `Enable`, `A dependency-free fixture for the Canvas Extension page ABI.`, text `Disabled v0.1.0 Pages: 1`, list item `Hello from an extension`, buttons `Update` and `Uninstall`. `control-openhands browser screenshot --feature F20.card --name disabled`.- **Trust confirmation (`F20.enable-confirm`).** Run `control-openhands browser click 'testid=canvas-extension-card-demo-page >> role=switch'` and `control-openhands browser snapshot 'testid=confirmation-modal'`: the paragraph `This app runs trusted JavaScript inside Agent Canvas and can make authenticated requests to the active Agent Server. Review its source and revision before enabling it.`, buttons `Cancel` and `Enable trusted app`. `control-openhands browser click 'testid=confirmation-modal >> testid=cancel-button'` closes it and `aria-checked` stays `false`. Keyboard: `control-openhands browser focus 'testid=canvas-extension-card-demo-page >> role=switch'` and `control-openhands browser press Space` open it again (`browser count 'testid=confirmation-modal'` is `1`); `control-openhands browser press Escape` closes it and `api GET /api/canvas-extensions/installed` still says `"enabled": false`. Backdrop: click the switch again, `control-openhands browser mouse-click 20 500`, then `browser count 'testid=confirmation-modal'` is `0` and the API still says `"enabled": false`. Reopen with the click, then `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'`, `control-openhands browser reload` and read `aria-checked`: `true`; `control-openhands browser text 'testid=canvas-extension-card-demo-page'` contains `Enabled`.- **Rail entry (`F20.rail-entry`, `F20.rail-label`).** After enabling, run `control-openhands browser wait 'testid=sidebar-canvas-extension-demo-page-hello' --timeout 10000`, `control-openhands browser attr 'testid=sidebar-canvas-extension-demo-page-hello' href` (`/extensions/demo-page/hello`) and `control-openhands browser text 'testid=sidebar-canvas-extension-demo-page-hello'`. Expected `Extension demo` (the fixture's `nav_label`); today it reads `Hello from an extension` (see Gotchas).-- **Render the page (`F20.page-render`).** Run `control-openhands browser click 'testid=sidebar-canvas-extension-demo-page-hello' --expect-url '/extensions/demo-page/hello(\?|$)'` and `control-openhands browser snapshot 'role=main[name="Hello from an extension"]'`: heading `Hello from a Canvas Extension` and paragraph `Host API 1 on backend default-local`. Direct URL: `control-openhands browser goto /extensions/demo-page/hello/nested` and the same snapshot show the paragraph `Nested extension path: nested`. `control-openhands browser screenshot --feature F20.page-render --name rail`.+- **Render the page (`F20.page-render`).** Run `control-openhands browser click 'testid=sidebar-canvas-extension-demo-page-hello' --expect-url '/extensions/demo-page/hello(\?|$)'` and `control-openhands browser snapshot 'role=main[name="Hello from an extension"]'`: heading `Hello from a Canvas Extension` and paragraph `Host API 1 on backend default-local`. Direct URL: `control-openhands browser goto /extensions/demo-page/hello/nested` and the same snapshot show the paragraph `Nested extension path: nested`. `control-openhands browser screenshot --feature F20.page-render --name rail`. The browser tab names the app page: `control-openhands browser eval "document.title"` on `/extensions/demo-page/hello` (and its nested path) should read the page title with an app suffix (`Hello from an extension - OpenHands App`), and leaving the app restores the normal title. Known failure (reproduced 2026-10-17, [#18557](https://github.com/OpenHands/OpenHands/issues/18557)): `document.title` is `OpenHands` after the rail click, on the direct and nested URLs, twice.- **App backend view (`F20.app-backend-view`).** Still on `/extensions/demo-page/hello/nested` (where Render the page ends) run `control-openhands browser text 'testid=demo-extension-app-backend-status'`, `control-openhands api GET /server_info --pick capabilities` and `control-openhands api GET /server_info --pick app_backend_ingress_url`. The host hands a page `appBackendView` only when the capabilities list `canvas_app_backend_bridge_v1` and the ingress URL is not `null`: with the pinned server (1.53.0: neither) the line reads `App backend view unavailable`; a server advertising both would make the demo read `App backend view available` (not driven here). `control-openhands browser goto /extensions/demo-page/hello` shows the same line, `control-openhands browser snapshot 'role=main[name="Hello from an extension"]'` lists it as the last paragraph, and `control-openhands browser screenshot --feature F20.app-backend-view --name unavailable`.- **Active and collapsed rail entry (`F20.rail-active`).** With the demo enabled, run `control-openhands browser goto /apps`; `control-openhands browser attr 'testid=sidebar-canvas-extension-demo-page-hello' aria-current` is `null`; after `control-openhands browser click 'testid=sidebar-canvas-extension-demo-page-hello' --expect-url '/extensions/demo-page/hello(\?|$)'` it is `page`. Run `control-openhands browser click 'testid=sidebar-collapse-toggle'`: `browser visible 'testid=sidebar-canvas-extension-demo-page-hello'` stays `true` (about 40 px wide), `browser attr ... aria-label` and `control-openhands browser tooltip 'testid=sidebar-canvas-extension-demo-page-hello'` read `Hello from an extension`; `control-openhands browser screenshot --feature F20.rail-active --name collapsed` shows the highlighted icon. Expand again with `browser click 'testid=sidebar-collapse-toggle'` (its `aria-label` is `Collapse sidebar` again). Command menu: `browser goto /`, `browser press Control+k`, `browser type 'testid=command-menu >> role=combobox' Hello` and `browser snapshot 'testid=command-menu'` read `No commands found`; `browser press Escape`.- **Unavailable page (`F20.page-unavailable`).** Run `control-openhands browser goto /extensions/qa-missing/x` and `control-openhands browser snapshot 'role=main'`: heading `Not available` and `This app is disabled, missing, or does not provide this page.` `control-openhands browser goto /extensions/demo-page/nope` shows the same card while the demo is enabled.@@ -72,7 +72,7 @@ Preconditions:- **Survives a restart (`F20.persist-restart`).** Run `control-openhands restart`, `control-openhands doctor`, `control-openhands browser goto /`, `control-openhands browser wait 'testid=sidebar-canvas-extension-demo-page-hello' --timeout 15000`, then `control-openhands browser goto /apps` and read the switch's `aria-checked`: `true`.- **Update with the backend down (`F20.refresh`).** On `/apps` run `control-openhands service stop agent-server`, `control-openhands browser click 'testid=canvas-extension-refresh-demo-page'` and `control-openhands browser toasts`: one toast, `An error occurred` (the ingress answers the Update's `POST /api/canvas-extensions/install` with a `502` that carries no server reason; see Gotchas), and the card still shows `Enabled` (`browser screenshot --feature F20.refresh --name backend-down`). Bring the stack back with `control-openhands restart` and `control-openhands doctor` before going on.- **Disable (`F20.disable`).** On `/apps` run `control-openhands browser click 'testid=canvas-extension-card-demo-page >> role=switch'`; `control-openhands browser count 'testid=confirmation-modal'` is `0` and `control-openhands browser wait 'testid=sidebar-canvas-extension-demo-page-hello' --state detached --timeout 10000` succeeds. After `control-openhands browser reload` the switch's `aria-checked` is `false`; `control-openhands browser goto /extensions/demo-page/hello` and `browser snapshot 'role=main'` show `Not available`.-- **Uninstall (`F20.uninstall`).** Re-enable the demo on `/apps` (`control-openhands browser goto /apps`, since Disable ends on the extension page; the switch, then `testid=confirmation-modal >> testid=confirm-button`, then `browser wait 'testid=sidebar-canvas-extension-demo-page-hello'`). Run `control-openhands browser click 'testid=canvas-extension-uninstall-demo-page'` and `control-openhands browser snapshot 'testid=confirmation-modal'`: `Uninstall Demo page from this backend?`, buttons `Cancel` and `Confirm`. `control-openhands browser press Escape` leaves `browser count 'testid=canvas-extension-card-demo-page'` at `1`; so does clicking Uninstall again and `control-openhands browser click 'testid=confirmation-modal >> testid=cancel-button'`. Click Uninstall again, `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'` and `control-openhands browser toasts` (`App uninstalled.`); `control-openhands browser wait 'testid=sidebar-canvas-extension-demo-page-hello' --state detached --timeout 10000` succeeds. After `control-openhands browser reload` the card count is `0`, the page shows `No apps are installed on this backend.`, and `/extensions/demo-page/hello` shows `Not available`. Known failure (reproduced 2026-10-16, [#18512](https://github.com/OpenHands/OpenHands/issues/18512)): the **Uninstall** label measures 4.30:1 in Light+, 3.89:1 in Solarized Light and 3.21:1 in OpenHands-DeepSea against the card (2.39:1 on DeepSea hover), below 4.5:1.+- **Uninstall (`F20.uninstall`).** Re-enable the demo on `/apps` (`control-openhands browser goto /apps`, since Disable ends on the extension page; the switch, then `testid=confirmation-modal >> testid=confirm-button`, then `browser wait 'testid=sidebar-canvas-extension-demo-page-hello'`). Run `control-openhands browser click 'testid=canvas-extension-uninstall-demo-page'` and `control-openhands browser snapshot 'testid=confirmation-modal'`: `Uninstall Demo page from this backend?`, buttons `Cancel` and `Confirm`. `control-openhands browser press Escape` leaves `browser count 'testid=canvas-extension-card-demo-page'` at `1`; so does clicking Uninstall again and `control-openhands browser click 'testid=confirmation-modal >> testid=cancel-button'`. Click Uninstall again, `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'` and `control-openhands browser toasts` (`App uninstalled.`); `control-openhands browser wait 'testid=sidebar-canvas-extension-demo-page-hello' --state detached --timeout 10000` succeeds. After `control-openhands browser reload` the card count is `0`, the page shows `No apps are installed on this backend.`, and `/extensions/demo-page/hello` shows `Not available`. The **Uninstall** label stays at least 4.5:1 against the card, normal and hovered, in all five themes at desktop and phone width (it uses the theme's error colour).- **Local source plus Path (`F20.install`).** Run `control-openhands browser goto /apps` (the Uninstall check ends on `/extensions/demo-page/hello`), `control-openhands browser click 'testid=canvas-extensions-add-button'`, `control-openhands browser fill 'testid=add-canvas-extension-source-input' "$PWD/src/fixtures/canvas-extensions"`, `control-openhands browser fill 'testid=add-canvas-extension-repo-path-input' demo-page`, `control-openhands browser click 'testid=add-canvas-extension-submit'` and `control-openhands browser wait 'testid=add-canvas-extension-modal' --state detached --timeout 30000`. The toast is `App installed. …` and `api GET /api/canvas-extensions/installed` shows `"source"` ending in `/src/fixtures/canvas-extensions/demo-page` with `"repo_path": null`. Uninstall it as above (switch untouched, so no rail check).- **Git tree URL (`F20.git-tree-url`).** Needs the Git precondition (upstream reachable, fixture identical). Run `control-openhands browser click 'testid=canvas-extensions-add-button'`, `control-openhands browser fill 'testid=add-canvas-extension-source-input' 'https://github.com/OpenHands/OpenHands/tree/main/src/fixtures/canvas-extensions/demo-page'` (Ref and Path empty), `control-openhands browser click 'testid=add-canvas-extension-submit' --observe 'testid=add-canvas-extension-submit'` (`Installing… [disabled]` for about 0.4 to 2 s, then `<absent>`) and `control-openhands browser wait 'testid=add-canvas-extension-modal' --state detached --timeout 90000`; the toast is `App installed. …`. After `control-openhands browser reload`, `control-openhands browser snapshot 'testid=canvas-extension-card-demo-page'` shows source `https://github.com/OpenHands/OpenHands`, switch `Enable`, term `Ref` with the 40-character commit from the precondition and term `Path` with `src/fixtures/canvas-extensions/demo-page`; `api GET /api/canvas-extensions/installed` has the same `source`, `resolved_ref` and `repo_path`. `control-openhands browser screenshot --feature F20.git-tree-url --name card`.- **Duplicate install (`F20.install-duplicate`).** With the Git-installed demo present, run `control-openhands browser errors --clear`, `control-openhands browser click 'testid=canvas-extensions-add-button'`, `control-openhands browser fill 'testid=add-canvas-extension-source-input' "$PWD/src/fixtures/canvas-extensions/demo-page"`, `control-openhands browser click 'testid=add-canvas-extension-submit'` and `control-openhands browser wait 'testid=add-canvas-extension-submit >> text=Install' --timeout 30000`. `control-openhands browser toasts` reads `This app is already installed. Use Update or Uninstall on its card.` (`browser screenshot --feature F20.install-duplicate --name already-installed`), `browser count 'testid=add-canvas-extension-modal'` stays `1`, `browser count '[data-testid^="canvas-extension-card-"]'` stays `1`, `api GET /api/canvas-extensions/installed` still shows the GitHub `source`, and `browser errors --app-only` lists the expected `409` on `POST /api/canvas-extensions/install`. Close the form with `control-openhands browser press Escape`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.mdindex fe28a19ba1..3d6929d476 100644--- a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md+++ b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md@@ -31,7 +31,7 @@ Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-rout- `F22.setup-form-validation`: Continue checks fields locally (`This field is required.`, `Must be at least N characters.`, `Must be at most N.`) and a field blur sends the draft to the service, whose errors appear under the field (`Value error, Invalid cron expression: …`).- `F22.setup-agent-profile`: the **Agent profile** combobox appears only for actions that create through `/v1` (Upload tarball and bundle entries), not for Prompt or Plugin; it lists `Default` and every saved agent profile and shows `Default` while no profile is chosen (also after a switch through Prompt or Plugin and after picking `Default`, #18111), picking a profile hides the **LLM profile** field, switching to Prompt or Plugin clears the choice, and the automation is created with that `agent_profile_id`.- `F22.setup-plugin-create`: the **Plugin** action takes plugin sources and a prompt, and a picked **Timezone** carries through Review into the created automation (`preset_metadata.plugins`, `trigger.timezone`).-- `F22.setup-tarball-create`: the **Upload tarball** action requires a `.tar`/`.tar.gz` file, shows its file name in Review and creates an automation with the uploaded tarball and the entrypoint.+- `F22.setup-tarball-create`: the **Upload tarball** action requires a `.tar`/`.tar.gz` file, shows its file name in Review and creates an automation with the uploaded tarball and the entrypoint. The native file chooser button follows the color theme (its interactive-hover background with contrast text).- `F22.setup-repository-list`: a repository list field has **Add** (disabled while the input is empty) and a **Remove** per row; an empty required list is refused.- `F22.setup-review-confirm`: **Review** lists every value (`Not set` for blanks); **Back** keeps the values; **Confirm and create** creates the automation (enabled) and replaces the URL with its detail page.- `F22.setup-bundle-create`: a bundle entry (Daily news digest) uploads its script tarball and creates an automation with the bundle's entrypoint, without any account.@@ -39,7 +39,7 @@ Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-rout- `F22.setup-fallback-conversation`: **Set up in a conversation** creates a conversation seeded with the entry's setup message.- `F22.setup-close`: X or Escape goes back in history; on a cold deep link it replaces the location with `/automations`.- `F22.setup-unknown-id`: `/automations/new/<id>` for an id no catalog entry claims is a 404.-- `F22.agent-assisted-setup`: **Add Automation → Create Automation** opens `/automations/setup`, whose setup panel collects name, action, trigger, model/profile and custom code, can ask the agent for help, saves server-backed drafts, opens test runs behind a Back button and dispatches the finished automation.+- `F22.agent-assisted-setup`: **Add Automation → Create Automation** opens `/automations/setup`, whose setup panel collects name, action, trigger, model/profile and custom code, can ask the agent for help, saves server-backed drafts, opens test runs behind a Back button and dispatches the finished automation. Expected: in create mode a Prompt or Plugin draft is not offered an agent profile its endpoint rejects, as the catalog dialog's Agent profile field already behaves, so Save draft never fails on one. Known failure (reproduced 2026-10-17, [#18569](https://github.com/OpenHands/OpenHands/issues/18569)): the agent profile pill under the Prompt (`automation-setup-agent-profile`) lists **Active profile** and every saved profile for Prompt and Plugin drafts; after picking a saved profile, Save draft sends `agent_profile_id` to `/v1/preset/prompt` or `/v1/preset/plugin`, the drafts API answers `422`, the toast reads `Extra inputs are not permitted` and the save state **Save failed**. **Active profile** saves.- `F22.import-picker`: **Add Automation → Import automation** opens a dialog that explains the format (docs link), says the import is disabled, and offers a dropzone and **Choose file**.- `F22.import-validation`: a file that is not JSON shows the toast `The selected file is not valid JSON.`; a JSON file that breaks the format shows one toast listing every issue; the dialog stays on the picker.- `F22.import-preview`: a valid file switches the dialog to a preview (Name, Trigger, Prompt, Plugins) with the disabled notice, **Cancel** and **Import automation**.@@ -80,7 +80,7 @@ Preconditions:- **Agent profile (`F22.setup-agent-profile`).** In the same form (Prompt action, name `QA_F22 Custom`, prompt and schedule `0 0 1 1 *` filled), `control-openhands browser count 'testid=automation-agent-profile'` is `0`; after `control-openhands browser choose 'testid=setup-action-kind' 'Plugin'` it is still `0`, and after `control-openhands browser choose 'testid=setup-action-kind' 'Upload tarball'` it is `1` (only actions that create through `/v1` can carry a profile; the preset endpoints behind Prompt and Plugin reject `agent_profile_id`). Run `control-openhands browser value 'testid=automation-agent-profile'`, `control-openhands browser click 'testid=automation-agent-profile'` and `control-openhands browser snapshot 'role=listbox'`: the value is `Default` and the listbox has `option "Default" [selected]` and `option "default"` (the seeded agent profile from `api GET /api/agent-profiles`), the same as on a fresh Upload tarball form (`F22.setup-tarball-create`). `control-openhands browser press Escape` closes the listbox and leaves the dialog open (`browser count 'testid=setup-dialog'` is `1`). Run `control-openhands browser choose 'testid=automation-agent-profile' 'default'`: `browser count 'testid=setup-field-model'` is `0` (LLM profile hidden); `browser choose 'testid=automation-agent-profile' 'Default'` brings it back (`1`). Pick `default` again, then `control-openhands browser choose 'testid=setup-action-kind' 'Prompt'`: the selector is gone and `setup-field-model` is back (`1`). Choose `Upload tarball` once more and run `control-openhands browser value 'testid=automation-agent-profile'`: `Default` (the choice was cleared, shown as on a fresh form; choosing `Default` from the list also reads `Default`). Choose `Prompt` again before continuing. Creating an automation with a profile is in the Upload tarball bullet below.- **Review and create (`F22.setup-review-confirm`).** Run `control-openhands browser click 'testid=setup-field-model'`, `control-openhands browser click 'role=listbox >> role=option[name="deepseek-flash"]'`, `control-openhands browser click 'testid=setup-continue-button'`, `control-openhands browser wait 'testid=setup-review' --timeout 10000` and `control-openhands browser snapshot 'testid=setup-dialog'`: heading `Review`, terms Action `Prompt`, Schedule `0 0 1 1 *`, Timezone `UTC`, Automation name, LLM profile `deepseek-flash`, Timeout `60`, Prompt, Repository `Not set`, Branch or ref `Not set`, and buttons `Back` and `Confirm and create`. Take `browser screenshot --feature F22.setup-review-confirm --name review`. `control-openhands browser click 'testid=setup-back-button'` returns to the form with `setup-field-name` still `QA_F22 Custom`; click Continue, wait for `setup-review` again, then `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}'`. `browser url` is `/automations/<id>` (the id of the new automation); after `control-openhands browser reload`, `control-openhands browser text 'role=heading[level=1]'` is `QA_F22 Custom` and `browser text 'testid=active-status-badge-active'` is `Active` (the detail page has no `main` landmark: `'role=main >> …'` times out there), and `api GET '/api/automation/v1?limit=100'` shows it `enabled: true`, `trigger.schedule` `0 0 1 1 *`, `timeout` `60`, `model` `deepseek-flash`. `control-openhands browser back` lands on `/automations/templates` (the setup URL was replaced).- **Plugin action and timezone (`F22.setup-plugin-create`).** Run `control-openhands browser goto /automations/new/custom-automation`, click `testid=setup-continue-button`, then `control-openhands browser choose 'testid=setup-action-kind' 'Plugin'` and `control-openhands browser choose 'testid=setup-field-timezone' 'Europe/Berlin'`. Fill `testid=setup-field-schedule` with `0 0 1 1 *`, `testid=setup-field-name` with `QA_F22 Plugin`, `testid=setup-field-plugins` with `github:OpenHands/extensions/plugins/qa-none` and `testid=setup-field-prompt` with `Reply with the single word: pong. Do not run any tools.`; click Continue and `browser wait 'testid=setup-review' --timeout 10000`. `browser snapshot 'testid=setup-dialog'`: Action `Plugin`, Timezone `Europe/Berlin`, LLM profile `Not set`, Timeout `Not set`, Plugins `github:OpenHands/extensions/plugins/qa-none`. `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}' --timeout 60000`; `api GET '/api/automation/v1?limit=100'` shows `QA_F22 Plugin` enabled with `trigger.timezone` `Europe/Berlin` and `preset_metadata` `{preset_type: plugin, plugins: [{source: github:OpenHands/extensions/plugins/qa-none}]}`.-- **Upload tarball action, with an agent profile (`F22.setup-tarball-create`, `F22.setup-agent-profile`).** Run `control-openhands browser goto /automations/new/custom-automation`, click `testid=setup-continue-button`, `control-openhands browser choose 'testid=setup-action-kind' 'Upload tarball'` (Entrypoint defaults to `python3 main.py`, and on this fresh form `control-openhands browser value 'testid=automation-agent-profile'` is `Default`), fill `testid=setup-field-name` with `QA_F22 Tarball` and `testid=setup-field-schedule` with `0 0 1 1 *`, click Continue: `browser text 'testid=setup-field-tarball-error'` is `This field is required.` Run `control-openhands browser upload 'testid=setup-field-tarball' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-tarball.tar.gz"` and `control-openhands browser choose 'testid=automation-agent-profile' 'default'` (`browser count 'testid=setup-field-model'` is `0`); take `control-openhands browser screenshot 'testid=setup-dialog' --feature F22.setup-agent-profile --name upload-profile`. Click Continue, `browser wait 'testid=setup-review' --timeout 10000`: Review shows Action `Upload tarball`, LLM profile `Not set`, Tarball `qa-f22-tarball.tar.gz`, Entrypoint `python3 main.py`, Setup script `Not set` (Review does not list the agent profile). `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}' --timeout 60000`; `browser text 'role=heading[level=1]'` is `QA_F22 Tarball`, and `control-openhands api GET /api/automation/v1/<id>` (the id in the URL) shows it enabled with `entrypoint` `python3 main.py`, a `tarball_path` `oh-internal://uploads/…`, `model` `null` and `agent_profile_id` equal to the `default` profile's `id` from `control-openhands api GET /api/agent-profiles`.+- **Upload tarball action, with an agent profile (`F22.setup-tarball-create`, `F22.setup-agent-profile`).** Run `control-openhands browser goto /automations/new/custom-automation`, click `testid=setup-continue-button`, `control-openhands browser choose 'testid=setup-action-kind' 'Upload tarball'` (Entrypoint defaults to `python3 main.py`, and on this fresh form `control-openhands browser value 'testid=automation-agent-profile'` is `Default`), fill `testid=setup-field-name` with `QA_F22 Tarball` and `testid=setup-field-schedule` with `0 0 1 1 *`, click Continue: `browser text 'testid=setup-field-tarball-error'` is `This field is required.` The chooser button follows the theme: `control-openhands browser eval "(s=>s.backgroundColor+' '+s.color)(getComputedStyle(document.querySelector('[data-testid=setup-field-tarball]'),'::file-selector-button'))"` is `rgb(64, 64, 64) rgb(255, 255, 255)` on OpenHands-Neutral and Neo, `rgb(75, 84, 104) rgb(255, 255, 255)` on DeepSea, `rgb(229, 229, 229) rgb(31, 31, 31)` on Light+ and `rgb(230, 223, 204) rgb(0, 43, 54)` on Solarized Light (`F16.color-theme`; restore OpenHands-Neutral). Run `control-openhands browser upload 'testid=setup-field-tarball' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-tarball.tar.gz"` and `control-openhands browser choose 'testid=automation-agent-profile' 'default'` (`browser count 'testid=setup-field-model'` is `0`); take `control-openhands browser screenshot 'testid=setup-dialog' --feature F22.setup-agent-profile --name upload-profile`. Click Continue, `browser wait 'testid=setup-review' --timeout 10000`: Review shows Action `Upload tarball`, LLM profile `Not set`, Tarball `qa-f22-tarball.tar.gz`, Entrypoint `python3 main.py`, Setup script `Not set` (Review does not list the agent profile). `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}' --timeout 60000`; `browser text 'role=heading[level=1]'` is `QA_F22 Tarball`, and `control-openhands api GET /api/automation/v1/<id>` (the id in the URL) shows it enabled with `entrypoint` `python3 main.py`, a `tarball_path` `oh-internal://uploads/…`, `model` `null` and `agent_profile_id` equal to the `default` profile's `id` from `control-openhands api GET /api/agent-profiles`.- **Close (`F22.setup-close`).** From `/automations/templates` (`control-openhands browser goto /automations/templates`; the previous bullet ends on the new automation's page), `control-openhands browser click 'testid=recommended-automation-card-news-digest' --expect-url '/automations/new/news-digest'` then `control-openhands browser click 'testid=setup-dialog-close' --expect-url '/automations/templates'`; the same with `control-openhands browser press Escape` instead of X also returns to `/automations/templates`. Cold deep link: `control-openhands browser goto /automations/new/news-digest`, `control-openhands browser click 'testid=setup-dialog-close' --expect-url '/automations$'` (replaced, not back).- **Bundle entry, min length (`F22.setup-bundle-create`, `F22.setup-form-validation`).** From `/automations/templates` (`control-openhands browser goto /automations/templates`; Close ends on `/automations`) click `testid=recommended-automation-card-news-digest` (`--expect-url '/automations/new/news-digest'`): there is no prerequisites step and the form shows `automation-agent-profile` (a bundle creates through `/v1`, so it offers a profile; leave it on `Default`), `setup-field-schedule` (`0 8 * * *`), `setup-field-timezone`, `setup-field-feeds` (three default feeds) and `setup-field-topics`. Fill `testid=setup-field-feeds` with `short` and click Continue: `browser text 'testid=setup-field-feeds-error'` is `Must be at least 8 characters.` Close (`testid=setup-dialog-close`), click the card again (the draft is gone), fill `testid=setup-field-schedule` with `0 0 1 1 *`, click Continue, `browser wait 'testid=setup-review' --timeout 10000`, then `control-openhands browser click 'testid=setup-continue-button' --expect-url '/automations/[0-9a-f-]{8,}' --timeout 60000`. The detail h1 is `Daily news digest`; `api GET '/api/automation/v1?limit=100'` shows it enabled with `entrypoint` `python3 main.py`, a `tarball_path` `oh-internal://uploads/…` and `timeout` `900`.- **Repository list (`F22.setup-repository-list`).** Run `control-openhands browser goto /automations/new/github-pr-reviewer` (no prerequisites step locally). `control-openhands browser enabled 'testid=setup-list-repositories-add'` is `false`; fill `testid=setup-field-repositories` with `qa-org/qa-repo` (`enabled` turns `true`), click `testid=setup-list-repositories-add`: `browser text 'testid=setup-list-repositories'` starts `qa-org/qa-repo` / `Remove` and the input is empty again. `control-openhands browser click 'testid=setup-list-repositories-remove-qa-org/qa-repo'` removes it (count `0`); Continue then shows `setup-field-repositories-error` `This field is required.` Close the dialog.@@ -93,6 +93,17 @@ Preconditions:- **Prompt launch (`F22.template-launch-conversation`).** Use a card whose required integrations all need external setup, so no install dialog queues: on `/automations/templates` run `control-openhands browser click 'testid=recommended-automation-pills-jira-issue-to-bitbucket-pr-overflow'` (`dialog: Bitbucket Needs external setup`), `control-openhands browser press Escape`, then `control-openhands browser click 'testid=recommended-automation-card-jira-issue-to-bitbucket-pr' --expect-url '/conversations/[0-9a-f]' --timeout 60000`. `control-openhands browser eval "document.querySelector('[data-testid=chat-input]').innerText"` is `/ticket-to-code-change:setup`, and `control-openhands conversation events <id> --kinds MessageEvent --from-start` (id from the URL) has `total` `0` even 20 s later: the launch command is pre-filled, not sent (the featured copy promises "a pre-filled conversation"). Do not press send (no model cost). Take `browser screenshot --feature F22.template-launch-conversation --name prefilled`. Cards with an installable missing integration (`research-brief-writer`, Tavily) go through the install queue first and are blocked here (see Preconditions).- **Open and save a draft (`F22.agent-assisted-setup`, desktop and phone).** This scoped recipe needs a doctored baseline stack with automation draft support (1.19.3 here); its non-dispatch steps need no LLM key or profile. From `/automations` run `control-openhands browser click 'testid=automations-add-automation'`, then `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`. The form and docked composer are visible; `control-openhands conversation list` is unchanged. Run `control-openhands browser fill 'testid=automation-setup-name' 'QA_F22 Delta Draft'`, `control-openhands browser fill 'testid=automation-setup-prompt' 'Reply pong. Do not run any tools.'`, `control-openhands browser click 'testid=automation-setup-frequency-custom'`, `control-openhands browser fill 'testid=automation-setup-custom-schedule' '0 0 1 1 *'`, then `control-openhands browser click 'testid=automation-setup-save-draft'`. `control-openhands browser wait 'testid=automation-setup-draft-details' --timeout 10000` and `browser snapshot 'testid=automation-setup-draft-details'` show **Draft saved** and **Ready to test**. `control-openhands api GET /api/automation/v1/drafts` supplies `<draft-id>` and confirms the name, prompt and yearly cron; the URL carries `draftId`. After `browser reload`, read each field with `browser value 'testid=automation-setup-name'`, `browser value 'testid=automation-setup-prompt'` and `browser value 'testid=automation-setup-custom-schedule'`: the values match. Screenshot with `--feature F22.agent-assisted-setup --name saved-draft`. Saving a prompt draft does not prove model execution; use the configured flash profile and the **Genuine prompt/plugin draft Test** checks below for that path.- **Credential-free script test (`F22.agent-assisted-setup`).** Run `control-openhands browser click 'testid=automation-setup-kind-custom'`, then inspect `browser value 'testid=automation-setup-custom-code'`: the generated Python prints its prompt and calls the real automation callback, without an agent/LLM. Keep its code, setup and entrypoint unchanged; click Save draft. Run `browser record start --feature F22.agent-assisted-setup --name custom-script-test --max-seconds 180`, then `browser click 'testid=automation-setup-draft-test'` and `browser wait 'testid=automation-setup-draft-runs-page' --timeout 10000`. Its snapshot initially shows Pending. Immediately run `api GET /api/automation/v1/drafts/<draft-id>` for `materialized_automation_id` and `last_test_run_id`, then `api GET '/api/automation/v1/<materialized-id>/runs?limit=10'`. Repeat the quoted runs GET after short `sleep 5` intervals until that run has terminal backend status (bounded to 90 seconds; no single wait over 45 seconds). Once it is COMPLETED, `browser wait 'testid=automation-setup-draft-run >> text=Successful' --timeout 45000` tests automatic UI completion. Expected: Successful without a reload. Record the actual row and a second backend receipt before any recovery. As a separate readback, run `browser reload`, click `testid=automation-setup-view-test-runs`, screenshot Successful, then `browser click 'testid=automation-setup-draft-runs-back'`: the draft details read **Latest test passed**. End with `browser record stop`. A saved draft, Pending row or successful reload does not prove automatic completion updates.+- **Agent profile on a Prompt or Plugin draft (`F22.agent-assisted-setup`).** Arrange a second saved agent profile: `control-openhands api POST /api/agent-profiles/QA_F22_Profile --write --data '{"schema_version":3,"agent_kind":"openhands","name":"QA_F22_Profile","llm_profile_ref":"deepseek-flash","agent":"CodeActAgent"}'`. Open a blank form (Add Automation → Create Automation), fill `testid=automation-setup-name` with `QA_F22 Profile` and the prompt with `Reply pong. Do not run any tools.`. Expected: `browser count 'testid=automation-setup-agent-profile'` is `0` (no profile offered to a Prompt draft), and Save draft succeeds. Known failure (reproduced 2026-10-17, [#18569](https://github.com/OpenHands/OpenHands/issues/18569)): the pill reads `default`; clicking it lists `automation-setup-agent-profile-option-active`, `-default` and `-QA_F22_Profile`. Pick `QA_F22_Profile`, `browser network --clear`, Save draft: `browser network --bodies --filter 'automation/v1'` shows `POST /api/automation/v1/drafts` `422` with `"endpoint":"/v1/preset/prompt"` and `agent_profile_id` in the draft, `browser toasts --history` holds `Extra inputs are not permitted` and `browser text 'testid=automation-setup-save-state'` is `Save failed`. Picking `automation-setup-agent-profile-option-active` and saving again returns `201`. With a plugin added (`testid=automation-setup-add-plugin`, source `github:OpenHands/extensions/plugins/qa-none`) and the profile picked again, the draft `PATCH` to `/v1/preset/plugin` fails the same way. Delete the draft with its dashboard trash icon (`F21.saved-drafts`) and run `control-openhands api DELETE /api/agent-profiles/QA_F22_Profile --write`.++ ```sh known-bug F22.agent-assisted-setup prompt-profile+ trap 'control-openhands api DELETE /api/agent-profiles/QA_F22_Profile --write >/dev/null' EXIT+ control-openhands api POST /api/agent-profiles/QA_F22_Profile --write --data '{"schema_version":3,"agent_kind":"openhands","name":"QA_F22_Profile","llm_profile_ref":"deepseek-flash","agent":"CodeActAgent"}'+ control-openhands browser viewport desktop+ control-openhands browser goto /automations/setup+ control-openhands browser wait 'testid=automation-setup-prompt'+ control-openhands assert --field value --equals '' -- browser value 'testid=automation-setup-prompt'+ control-openhands assert --field count --equals 0 --bug-shows 1 -- browser count 'testid=automation-setup-agent-profile' # bug+ ```- **Resume at phone width (`F22.agent-assisted-setup`).** Run `browser viewport phone`, `browser goto /automations`, then click `testid=automation-setup-draft-resume-<draft-id>` with `--expect-url '/automations/setup'`. Read Name, rename to `QA_F22 Phone Draft`, Save draft and reload: `browser value 'testid=automation-setup-name'` and `api GET /api/automation/v1/drafts/<draft-id>` both have the new name. `browser bbox 'testid=automation-setup-form'` has `pageHorizontalOverflow: false`; this long form scrolls vertically and need not fit wholly inside the viewport. Screenshot with `--feature F22.agent-assisted-setup --name resumed-phone`. Repeat initial open/save/reload on a fresh phone draft when checking that entry point; desktop creation is not phone creation. Restore desktop afterward.- **Agent fills a prompt form (`F22.agent-assisted-setup`, desktop).** Start a fresh form with `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`. Run `control-openhands conversation start --stay --prompt 'Call automation_form_update exactly once with fields {"name":"QA_F22 DeepSeek Desktop","kind":"prompt","prompt":"Reply pong. Do not run any tools.","frequency":"custom","customSchedule":"0 0 1 1 *","timezone":"UTC","model":"deepseek-flash"}. Do not call any other tools, run terminal commands, inspect files, or read environment variables. Briefly confirm the filled fields and stop.'`. `--stay` sends through the docked composer; omitting it would navigate away to New Chat. The returned `<setup-conversation-id>` opens `/conversations/<id>` with the form beside it. Run `control-openhands conversation wait <setup-conversation-id> --until finished,idle --timeout 60`, then `control-openhands conversation events <setup-conversation-id> --kinds ActionEvent --full`: a genuine `automation_form_update` supplied the exact fields. Read `control-openhands browser value 'testid=automation-setup-name'`, `control-openhands browser value 'testid=automation-setup-prompt'`, `control-openhands browser value 'testid=automation-setup-custom-schedule'` and `control-openhands browser text 'testid=automation-setup-model'`; they match the requested values. `control-openhands browser snapshot 'testid=automation-setup-form'` also shows **Filled by OpenHands** markers. Capture `control-openhands browser screenshot --feature F22.agent-assisted-setup --name desktop-genuine-tool-filled`; the UI field readbacks and real action establish the update, not the agent's confirmation alone. The sent prompt is not kept as a draft: `control-openhands browser eval "sessionStorage.getItem('oh:home-prompt-draft')"` is `null` (with no earlier Home draft), and the New Chat composer (`browser eval "document.querySelector('[data-testid=chat-input]').innerText"` after clicking `text=New Chat`) and a later `/automations/setup` docked composer are empty; return to `/conversations/<setup-conversation-id>` for the next bullet.- **Desktop Hide/Show and phone views (`F22.agent-assisted-setup`).** On that associated conversation, `control-openhands browser attr 'testid=automation-setup-agent-toggle' aria-label` reads `Hide Agent`. Click the toggle: the label becomes `Show Agent`, and `control-openhands browser bbox 'testid=conversation-chat-panel'` has width `0`; form values remain. Click again: `Hide Agent` and a visible conversation panel return. Save with `control-openhands browser click 'testid=automation-setup-save-draft'`, `control-openhands browser wait 'testid=automation-setup-draft-details' --timeout 10000` and `control-openhands api GET /api/automation/v1/drafts` to obtain `<assisted-draft-id>` and the stored form. Run `control-openhands browser viewport phone`, `control-openhands browser click 'testid=automation-setup-mobile-view-form'`, read Name, then `control-openhands browser click 'testid=automation-setup-mobile-view-conversation'`; the selected view fits without horizontal overflow. From `control-openhands browser goto /automations`, click `testid=automation-setup-draft-resume-<assisted-draft-id>` with `--expect-url '/automations/setup'`: dashboard Resume opens `/automations/setup?draftId=<id>`, not the original conversation URL. Reload and read Name/prompt/schedule again. Separately `control-openhands browser goto /conversations/<setup-conversation-id>` and `control-openhands browser click 'testid=automation-setup-mobile-view-form'` restore the same draft inside its associated conversation. Capture each entry separately. `control-openhands browser click 'testid=automation-setup-create' --expect-url '/automations/[0-9a-f-]{8,}'` creates the final prompt automation; `control-openhands api GET /api/automation/v1/<id>` shows the requested fields and `enabled: false`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.mdindex e4392a0ea1..14d356e482 100644--- a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md+++ b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md@@ -53,7 +53,7 @@ Source: `src/routes/automation-detail.tsx`, `src/components/features/automations- `F23.edit-save`: Save sends one PATCH (also when nothing changed), shows **Saving…** then **Saved just now**, the toast **Automation updated**, and the detail page and API reflect it; an Active profile model stays `null`.- `F23.edit-test`: on the edit page **Test** dispatches the automation as last saved (`POST /api/automation/v1/<id>/dispatch`, no PATCH first) and shows `Test run started.` on the form; unsaved edits are not part of that run. Whether it should test the current form is a product decision (Gotchas).- `F23.edit-dirty-state`: the edit page shows no save state for unsaved edits (no **Unsaved changes**, which create mode shows, F28); the label appears only while saving (**Saving…**) and after (**Saved just now** or **Save failed**). **Close** discards the edits.-- `F23.edit-schedule`: Frequency offers Once, Hourly, Daily, Weekdays (Mon–Fri), Weekly and Custom; Weekly adds **Day of week** and **At**; they are saved as a cron expression and read back after a reload.+- `F23.edit-schedule`: Frequency offers Once, Hourly, Daily, Weekdays (Mon–Fri), Weekly and Custom; Weekly adds **Day of week** and **At**; they are saved as a cron expression and read back after a reload. Switching to Custom starts the Custom field from the cron of the preset it replaces (Weekly, Wednesday at `10:30` → `30 10 * * 3`), and a saved cron that matches a preset reads back as that preset, not as Custom.- `F23.edit-time-cleared`: clearing **At** on a preset schedule and saving should be refused with a message instead of being dropped or replaced. Known failure (reproduced 2026-10-13, [#17948](https://github.com/OpenHands/OpenHands/issues/17948) item, new symptom since #17688): Save silently stores midnight (`0 0 * * 3`) and toasts **Automation updated**; Save draft in the create flow stores `0 0 * * *` the same way. Open PR #18260 (head `0f812ac68902d2c71896568311bffb8aa6f216a6`, checked 2026-10-15) targets the live setup panel: it rejects an invalid or cleared time before Save and Save draft, with an edit-mode regression test. It is not merged; the known failure above remains the observed behavior on this map’s baseline.- `F23.edit-cron`: a custom cron schedule shows Frequency **Custom** with an editable **Custom** cron field and a timezone select.- `F23.edit-event-readonly`: for an event automation the edit page shows Event selected with Event source, event key and filter controls (editable now, no longer read-only text) and no schedule fields.@@ -97,7 +97,7 @@ Preconditions:- **Edit fields and closing (`F23.edit-open`, `F23.edit-close`).** On Pong run `control-openhands browser click 'role=button[name="Automation actions"]'` and `control-openhands browser click 'role=list >> has-text=Tarball >> role=button[name="Edit"]' --expect-url '/automations/setup'`: the URL is `/automations/setup?automationId=<id>`, `browser count 'testid=edit-automation-save'` is `0` (no modal), and `browser value` reads `automation-setup-name` (`QA_F23 Pong`), `automation-setup-prompt`, the stored schedule (a custom cron shows Frequency Custom with `automation-setup-custom-schedule`) and `automation-setup-timeout` (`600`); `browser text 'testid=automation-setup-agent-profile'` is `default` and `browser text 'testid=automation-setup-model'` names the active profile (the stored `model` stays `null`). `browser testids 'testid=automation-setup-toolbar'` lists `automation-setup-test`, `automation-setup-save` and `automation-setup-close`; `control-openhands conversation list` is unchanged. `control-openhands browser press Escape` leaves the URL unchanged; `control-openhands browser click 'testid=automation-setup-close' --expect-url '/automations$'` lands on the dashboard without a PATCH (the dashboard's own GETs follow). A cold `control-openhands browser goto '/automations/setup?automationId=<id>'` opens the same form.- **Validation (`F23.edit-validation`).** On the edit page run `control-openhands browser fill 'testid=automation-setup-prompt' ''`, `control-openhands browser network --clear` and `control-openhands browser click 'testid=automation-setup-save'`: `control-openhands browser toasts --history` includes the toast `Add a prompt before continuing.` and `browser network --filter 'automation/v1'` lists no request. Refill the prompt. On a Custom-cron automation (QA_F23 Fail) fill `testid=automation-setup-custom-schedule` with `not a cron` (then `0 0 31 2 *`, then a valid cron with `testid=automation-setup-timeout` `1801`), click Save and read `control-openhands browser toasts --history` after `browser toasts --history --clear`. Each sends a `PATCH` that returns `422` (`browser network --filter 'automation/v1'`), and the history holds exactly one toast with the API message: `Value error, Invalid cron expression: not a cron`, `Value error, Cron expression cannot produce any future fire times: 0 0 31 2 *` and `Value error, timeout must not exceed 1800 seconds (30 minutes)`; `api GET /api/automation/v1/<Fail id>` keeps `0 0 1 1 *` and timeout `600`. Before each new click move the pointer off the toast (`browser hover 'testid=automation-setup-name'`, about 5 s; see Gotchas).- **No-change Save (`F23.edit-save`).** Open Edit, `control-openhands browser network --clear`, click `testid=automation-setup-save`, then `control-openhands browser network --filter 'automation/v1'`: one `PATCH` with the unchanged values and the toast `Automation updated` (the old modal sent nothing).-- **Weekly schedule, model and timeout (`F23.edit-schedule`, `F23.edit-save`).** On the edit page run `control-openhands browser click 'testid=automation-setup-frequency-weekly'`, `control-openhands browser select 'testid=automation-setup-weekday' 3` (Wednesday; options `0:Sunday` … `6:Saturday`), `control-openhands browser fill 'testid=automation-setup-time' 10:30`, `control-openhands browser click 'testid=automation-setup-model'`, `control-openhands browser click 'testid=automation-setup-model-option-deepseek-pro'`, `control-openhands browser fill 'testid=automation-setup-timeout' 300` and `control-openhands browser screenshot --feature F23.edit-schedule --name weekly-form`. Then `control-openhands browser network --clear`, `control-openhands browser click 'testid=automation-setup-save' --observe 'testid=automation-setup-save-state' --observe-ms 2000` (`Saving…` → `Saved just now`) and `control-openhands browser toasts --history` (`Automation updated`). `browser network --bodies --filter 'automation/v1'` shows one `PATCH /api/automation/v1/<id>` with `schedule` `30 10 * * 3`, `model` `deepseek-pro` and `timeout` `300`. After `control-openhands browser reload` the weekday is `3`, At `10:30`, the model `deepseek-pro` and Timeout `300`; the detail page Configuration reads `Schedule` `30 10 * * 3` and `LLM profile` `deepseek-pro`.+- **Weekly schedule, model and timeout (`F23.edit-schedule`, `F23.edit-save`).** On the edit page run `control-openhands browser click 'testid=automation-setup-frequency-weekly'`, `control-openhands browser select 'testid=automation-setup-weekday' 3` (Wednesday; options `0:Sunday` … `6:Saturday`), `control-openhands browser fill 'testid=automation-setup-time' 10:30`, `control-openhands browser click 'testid=automation-setup-model'`, `control-openhands browser click 'testid=automation-setup-model-option-deepseek-pro'`, `control-openhands browser fill 'testid=automation-setup-timeout' 300` and `control-openhands browser screenshot --feature F23.edit-schedule --name weekly-form`. Click `testid=automation-setup-frequency-custom`: `browser value 'testid=automation-setup-custom-schedule'` is `30 10 * * 3`; click `testid=automation-setup-frequency-weekly` again (the weekday and time stay). Then `control-openhands browser network --clear`, `control-openhands browser click 'testid=automation-setup-save' --observe 'testid=automation-setup-save-state' --observe-ms 2000` (`Saving…` → `Saved just now`) and `control-openhands browser toasts --history` (`Automation updated`). `browser network --bodies --filter 'automation/v1'` shows one `PATCH /api/automation/v1/<id>` with `schedule` `30 10 * * 3`, `model` `deepseek-pro` and `timeout` `300`. After `control-openhands browser reload` the weekday is `3`, At `10:30`, the model `deepseek-pro` and Timeout `300`; the detail page Configuration reads `Schedule` `30 10 * * 3` and `LLM profile` `deepseek-pro`.- **Cleared time (`F23.edit-time-cleared`).** On the weekly edit page `control-openhands browser fill 'testid=automation-setup-time' ''`, `control-openhands browser network --clear`, click Save. Expected: an inline message and nothing saved. Known failure (reproduced 2026-10-13, [#17948](https://github.com/OpenHands/OpenHands/issues/17948) item, new symptom since #17688): `browser network --bodies --filter 'automation/v1'` shows a `PATCH` with `schedule` `0 0 * * 3`, the toast reads `Automation updated` and the automation now runs at midnight. Save draft in the create flow (`/automations/setup`) stores `0 0 * * *` the same way; open PR #18260 targets both paths in `automation-setup-panel.tsx` at head `0f812ac68902d2c71896568311bffb8aa6f216a6` (checked 2026-10-15), but is not merged.- **Custom cron (`F23.edit-cron`).** Open QA_F23 Fail and Edit: Frequency **Custom** is selected and `control-openhands browser value 'testid=automation-setup-custom-schedule'` is `0 0 1 1 *` beside the timezone select (`UTC`). Fill `0 0 2 1 *` and `testid=automation-setup-timeout` with `1` (this makes the next run fail), Save and `browser toasts --history` (`Automation updated`); after a reload of the detail page the Configuration shows `0 0 2 1 *`. Invalid crons: see Validation.- **Failed run, its row and Debug with OpenHands (`F23.run-status-polling`, `F23.run-no-conversation`, `F23.run-logs-modal`, `F23.debug-with-openhands`).** On QA_F23 Fail (timeout `1` from the previous bullet) click `role=button[name="Run now"]` and `control-openhands browser wait-text 'Automation dispatched'`. The run stays Running for about 3-4 minutes and ends `FAILED` (poll `control-openhands api GET /api/automation/v1/<id>/runs`; drive the event automation bullets meanwhile). Then `control-openhands browser goto /automations`, open QA_F23 Fail, `control-openhands browser wait 'testid=automation-activity-log >> testid=run-status-icon-failed'` and `control-openhands browser text 'testid=automation-activity-log'`: the newest row reads `<time>`, `Timed out: command timed out or was killed`, `Failed`. Expected beside the time: `No Conversation` (this is a prompt automation); today it reads `No conversation — this run executed a script. Use View logs for its output.` Run `control-openhands browser click 'testid=automation-activity-log >> title=View logs >> nth=0'` and `control-openhands browser snapshot 'role=dialog'`: `Run` `FAILED`, `Task` `No task outcome reported.`, `System` `Error: Timed out: command timed out or was killed` and `Status detail: ...`, the Output tab with the `[setup] Fetching SDK version ...` lines, and button `Debug with OpenHands`. Screenshot with `--feature F23.debug-with-openhands --name failed-logs`. `control-openhands browser click 'testid=debug-automation-button' --expect-url '/conversations/'` opens a new conversation; `control-openhands conversation events <conversation id from browser url> --kinds MessageEvent` starts `The scheduled automation "QA_F23 Fail" failed during a run. Please investigate the error and fix the root cause.`, and `control-openhands browser eval "document.body.innerText.includes('Timed out: command timed out or was killed')"` is `true` (the page also shows the run id). The agent then works on that prompt with the active profile.@@ -120,7 +120,7 @@ Preconditions:## Gotchas- The automations QA checklist (F28) expects edit-mode **Test** to run the current form state; the panel calls `dispatchAutomation(<id>)`, which runs the saved automation. `F23.edit-test` records current behavior; whether it should change is a product decision, not a map correction.-- The Logs dialog's panel keeps long content reachable (`F23.run-logs-modal`, #18175 / #18173). Re-verified 2026-10-13 (revision `2bdc0e9d`, SDK 1.54.0, automation 1.19.3) with a 30-line `outcome_summary` at 1440×1000, 390×844 and 390×500: the panel is `testid=run-logs-body` (`max-h-[80vh]`, `overflow-y: auto`) with a sticky header (Logs + X stay visible); the Output box no longer scrolls on its own, the whole panel does, and its end (`=== RESULT === ALL_OK`) is reachable at the bottom. Verified 2026-10-10 on an independent fresh Linux Local stack at product revision `5a8c3832154b14d2dad651801c0ab3f74c96f02e`, SDK 1.54.0 and automation 1.19.3: a genuine DeepSeek finish produced a 4,596-character `outcome_summary` and 1,079-character metadata `message`; summary/metadata endings, Output/Error tabs and the true stdout end were reachable at 1440×1000, 390×844, 1440×500 and 390×500. The header/X remained visible, closing did not navigate, and phone had no horizontal overflow. Preserve those long-content and short-height checks when retesting. The verified scope is pinned Linux Local; macOS, Cloud and legacy/custom finish schemas need their own checks. `browser bbox 'role=dialog'` measures the full-screen wrapper, not the panel: measure the panel with `browser eval`.+- The Logs dialog's panel keeps long content reachable (`F23.run-logs-modal`, #18175 / #18173). Re-verified 2026-10-17 (revision `04529073`, Agent Server 1.55.1) with a genuine 30-line `outcome_summary` at 1440×1000 and 390×844 (2026-10-13 also at 390×500): the panel is `testid=run-logs-body` (`max-h-[80vh]`, `overflow-y: auto`) with a sticky header (Logs + X stay visible); the Output box no longer scrolls on its own, the whole panel does, and its end (`=== RESULT === ALL_OK`) is reachable at the bottom. Verified 2026-10-10 on an independent fresh Linux Local stack at product revision `5a8c3832154b14d2dad651801c0ab3f74c96f02e`, SDK 1.54.0 and automation 1.19.3: a genuine DeepSeek finish produced a 4,596-character `outcome_summary` and 1,079-character metadata `message`; summary/metadata endings, Output/Error tabs and the true stdout end were reachable at 1440×1000, 390×844, 1440×500 and 390×500. The header/X remained visible, closing did not navigate, and phone had no horizontal overflow. Preserve those long-content and short-height checks when retesting. The verified scope is pinned Linux Local; macOS, Cloud and legacy/custom finish schemas need their own checks. `browser bbox 'role=dialog'` measures the full-screen wrapper, not the panel: measure the panel with `browser eval`.- The detail page has no `main` landmark: `browser snapshot 'role=main'` times out. Snapshot the page, or scope by section headings (`role=heading[name="Configuration"] >> xpath=ancestor::div[2]`).- The kebab menu has no test ids; `role=list >> has-text=Tarball` scopes the detail kebab (the dashboard kebab has no Tarball, so F21 uses `has-text=Export`).- Kebab **Edit** (detail and dashboard) opens `/automations/setup?automationId=<id>`. The edit page's Save and Test report API failures with the API's message (see `F23.edit-validation`, `F23.edit-save-error`), and its `Automation updated` toast covers Test/Save/Close until the pointer moves away.diff --git a/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md b/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.mdindex f7d50d9a72..599dbc2a28 100644--- a/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md+++ b/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md@@ -22,20 +22,20 @@ Source: `src/components/features/backends/`, `src/contexts/active-backend-contex## Sub-features- `F25.selector-dropdown`: clicking (or hovering) the selector opens a listbox of backends, each option named `<status> <name>` with a status dot, and a footer with **Add Backend** and **Manage Backends**.-- `F25.add-backend-modal`: **Add Backend** opens "Choose how you want to connect" with a description, a **Learn more** link to the deployment docs and the tabs **OpenHands Cloud** (selected) and **Agent-server** (plus **Providers** with the bridge, see the Providers row). It takes keyboard focus on open (the close button), Tab and Shift+Tab wrap inside it, and at desktop width closing returns focus to the backend selector (at phone width the drawer closes with the modal and focus falls back to the page body). Escape does not close it; the X does.+- `F25.add-backend-modal`: **Add Backend** opens "Choose how you want to connect" with a description, a **Learn more** link to the deployment docs and the tabs **OpenHands Cloud** (selected) and **Agent-server** (plus **Providers** with the bridge, see the Providers row). It takes keyboard focus on open (the close button), Tab and Shift+Tab wrap inside it, and at desktop width closing returns focus to the backend selector (at phone width the drawer closes with the modal and focus falls back to the page body). Escape does not close it; the X does. Known failure (reproduced 2026-10-17, [#18589](https://github.com/OpenHands/OpenHands/issues/18589)): at phone width, with the modal opened from the drawer's backend selector, Escape closes the drawer and the modal with it.- `F25.cloud-advanced-host`: on the Cloud tab, **Advanced** reveals a **Host** field (placeholder `https://app.all-hands.dev`) for a self-hosted Cloud.-- `F25.cloud-device-flow`: **Connect to OpenHands** opens a popup and shows `Starting authentication...`, then `Waiting for authorization...` with the verification link and **Cancel**; Cancel returns to the idle button and closes the popup. An unreachable host shows `Failed to start device flow: ...` with **Try again**. Approving the code (adds a backend named OpenHands Cloud) needs a Cloud account.+- `F25.cloud-device-flow`: **Connect to OpenHands** opens a popup and shows `Starting authentication...`, then `Waiting for authorization...` with the verification link and **Cancel**; Cancel returns to the idle button and closes the popup. An unreachable host shows `Failed to start device flow: ...` with **Try again** and closes the popup (Try again opens a fresh one); the error text, the `Waiting for authorization...` status, the `Browser opened...` line and the verification link stay readable (at least 4.5:1) in every theme, Light+ and Solarized Light included. Known failure (reproduced 2026-10-17, [#18546](https://github.com/OpenHands/OpenHands/issues/18546)): in Solarized Light the `Browser opened. Complete sign-in to continue...` line (`text-tertiary` `#586E75` on the modal's `#EEE8D5`) measures 4.39:1. Approving the code (adds a backend named OpenHands Cloud) needs a Cloud account.- `F25.agent-server-guidance`: the Agent-server tab has a **Local / Remote** toggle. Local shows a collapsible **Before you connect** with `agent-canvas --backend-only --port 8001` and a local-setup docs link; Remote shows **Recommended setup**, **Connection details** and a remote-setup docs link.- `F25.add-agent-server`: **Connect** stays disabled until Host Name and a valid Host are filled; the API Key field is optional for Connect on Local and required on Remote, but a key-protected host still rejects an empty key. A dead host, an empty key or a wrong key shows `Could not connect to <host>` with the reason inline (`Disconnected (check URL or network)` for a dead host, `Invalid API key` for an empty or wrong key); a reachable host with the right key, entered on the default Local location, adds the backend, makes it active (BM-001) and redirects a `/conversations/<id>` page to `/conversations` without an error toast, also on the backend's first activation, without asking the new backend for the old conversation id.- `F25.switch-backend`: choosing another option shows a full-screen `Switching to <name>` overlay for about a second, then the sidebar and pages show that backend's data; on `/conversations/<id>` the app moves to `/conversations` (BM-002).- `F25.backend-pinned-url`: in-app links carry `?backend=<id>`; opening such a URL pins the tab to that backend; an unknown id is ignored.- `F25.per-tab-backend`: the active backend is per browser tab: switching it in one tab leaves the other open tabs where they were, a new tab opened from a plain link starts from the last choice made in any tab, and a sidebar conversation row opened in a new tab (Control-click) lands on the backend that owns the conversation because its link carries `?backend=<owner>`.- `F25.manage-backends`: Manage backends lists every backend with status dot, name, version (`v1.50.1`), host, status text and a LOCAL/CLOUD pill, plus Edit and Remove; it takes keyboard focus on open and keeps Tab inside; Escape, a backdrop click, X and **Done** close it and return focus to the selector.-- `F25.manage-add`: **Add Backend** in Manage backends opens the same Add form stacked over Manage; its X returns to Manage, and a successful add keeps Manage open with the new row while the new backend becomes active.+- `F25.manage-add`: **Add Backend** in Manage backends opens the same Add form stacked over Manage; its X returns to Manage, Escape leaves both open (Known failure (reproduced 2026-10-17, [#18589](https://github.com/OpenHands/OpenHands/issues/18589)): at phone width, with Manage opened from the drawer's backend selector, Escape closes the drawer, Add and Manage), and a successful add keeps Manage open with the new row while the new backend becomes active.- `F25.manage-select`: clicking a healthy row makes that backend active and closes the modal; on a detail page it leaves it for that section's list, like the selector does (BM-002).- `F25.edit-backend`: the pencil opens "Edit backend" pre-filled (key masked) with a `Connected · Local v<version>` badge; empty name and invalid host show `Name is required` and `Enter a valid URL (e.g. http://localhost:8080)` and disable Save; Save re-tests the connection, keeps the modal open with `Could not connect to <host>` on failure, and persists on success.-- `F25.edit-cancel`: **Cancel** and the X in Edit backend close only the Edit modal and discard the draft; Manage backends stays open with the stored values.-- `F25.edit-escape`: Escape in the Edit modal should not discard the draft (the modal disables Escape). Known failure, see Gotchas.+- `F25.edit-cancel`: **Cancel**, the X and a backdrop click in Edit backend close only the Edit modal and discard the draft; Manage backends stays open with the stored values.+- `F25.edit-escape`: Escape in the Edit modal does nothing: Edit keeps its draft and Manage backends stays open underneath. Known failure (reproduced 2026-10-17, [#18589](https://github.com/OpenHands/OpenHands/issues/18589)): at phone width, with Manage opened from the drawer's backend selector, Escape closes the drawer, Manage and Edit, and the draft is lost.- `F25.health-status`: a backend that stops answering turns its dot red (`Disconnected`) within one health probe (every 30 s for local and remote agent servers; Cloud backends every 5 min and not on window focus or reconnect, #18128); its Manage row reads `Disconnected (check URL or network)` with a red detail line and cannot be selected.- `F25.recovery-gate`: reloading while the active backend is down shows only `agent-server-onboarding-screen` with Manage backends in recovery mode (no X, no Done, Escape ignored, primary **Add Backend**); choosing a healthy row restores the app.- `F25.remove-backend`: the trash icon asks `Remove backend "<name>"? If it is active, the app will switch back to Local.`; Cancel keeps it, Confirm removes it and an active removed backend falls back to Local (BM-003).@@ -81,31 +81,31 @@ Preconditions:- **Open the selector (`F25.selector-dropdown`).** From `/conversations` run `control-openhands browser click 'testid=backend-selector'` and `control-openhands browser snapshot 'testid=backend-selector'`. The snapshot shows `listbox` with `option "Connected Local"`, a separator, `button "Add Backend"` and `button "Manage Backends"`. Screenshot with `control-openhands browser screenshot --feature F25.selector-dropdown --name open`. Hover opens it too: `browser press Escape`, move the pointer away with `control-openhands browser hover 'testid=command-menu-trigger'` (hovering a selector the pointer is already on does nothing), `control-openhands browser hover 'testid=backend-selector'`, then the same snapshot shows `combobox "Local" [expanded]` and the two footer buttons.- **Add Backend modal (`F25.add-backend-modal`).** Run `control-openhands browser click 'testid=add-backend-menu-item'` and `control-openhands browser snapshot 'testid=add-backend-modal'`: heading `Choose how you want to connect`, link `Compare Cloud, self-hosted, and Enterprise deployment options` (`/url: https://docs.openhands.dev/overview/introduction`, text `Learn more`) and tab `OpenHands Cloud ...` `[selected]` next to tab `Agent-server ...`. Keyboard focus: `control-openhands browser eval "(()=>{const a=document.activeElement;const d=document.querySelector('[data-testid=add-backend-modal]');return {tid:a.getAttribute('data-testid'),inDialog:!!(d&&d.contains(a))}})()"` is `{tid: "add-backend-close", inDialog: true}` right after opening; after eight `control-openhands browser press Tab` and three `browser press Shift+Tab` (reading the same eval after each) `inDialog` stays `true` (Tab past **Advanced** wraps to the close button), and `browser press q` then `control-openhands browser eval "document.querySelector('[data-testid=chat-input]').textContent"` is `""` (nothing reaches the Home composer behind). `browser screenshot --feature F25.add-backend-modal --name focus-trap-open`. `control-openhands browser press Escape` leaves `browser count 'testid=add-backend-modal'` at `1`; `control-openhands browser click 'testid=add-backend-close'` makes it `0`, and `control-openhands browser eval "!!document.activeElement.closest('[data-testid=backend-selector]')"` is `true` (focus is back on the selector). Keyboard focus in a light theme: arrange `control-openhands browser eval "localStorage.setItem('openhands-color-theme','light-plus')"` and `browser reload`, reopen the modal, run `control-openhands browser focus 'testid=add-backend-option-cloud'` and `browser press ArrowRight`, then `control-openhands browser eval "getComputedStyle(document.activeElement).outlineColor"`. Expected: the theme's focus colour (`--oh-focus`, `#007ACC` in Light+). Connection tabs, the guidance toggle and the DigitalOcean provider choice use the theme focus role and preserve host `--oh-focus` overrides; default focus contrast is at least 3:1 against base, modal, raised and hovered neutral surfaces in all five themes. Solarized Light uses `#227FBE`. Check the settled outline after its transition, preserving the 2 px width and existing offsets. Restore with `browser eval "localStorage.removeItem('openhands-color-theme')"`.- **Advanced host (`F25.cloud-advanced-host`).** Reopen with `browser click 'testid=backend-selector'` and `browser click 'testid=add-backend-menu-item'`, run `control-openhands browser click 'testid=add-backend-advanced-toggle'`, then `control-openhands browser attr 'testid=add-backend-advanced-toggle' aria-expanded` (`true`) and `control-openhands browser snapshot 'testid=add-backend-cloud-panel'` (textbox `Host` with placeholder `https://app.all-hands.dev`).-- **Device flow error (`F25.cloud-device-flow`).** Run `control-openhands browser fill 'testid=add-backend-cloud-host' http://127.0.0.1:9`, `control-openhands browser click 'testid=add-backend-login-button' --observe 'testid=add-backend-device-flow'` and `control-openhands browser wait 'testid=add-backend-auth-error' --timeout 15000`. The observation goes `Starting authentication...` → `Failed to start device flow: Failed to fetch` / `Try again`, and `control-openhands browser network --last 5` shows `POST /oauth/device/authorize` to `http://127.0.0.1:9`. `control-openhands browser click 'testid=add-backend-auth-retry'` starts again and fails the same way. Each attempt leaves an `about:blank` popup: `control-openhands browser tabs`, then `control-openhands browser close-tab <i>` for each extra page and `control-openhands browser tab 0`. Close the modal with `testid=add-backend-close`. Known failure (reproduced 2026-10-16, [#18527](https://github.com/OpenHands/OpenHands/issues/18527)): in Light+ the `Failed to start device flow: Failed to fetch` text measures 1.80:1 on its red tint, below 4.5:1.-- **Device flow awaiting and Cancel (`F25.cloud-device-flow`).** Reopen Add Backend (Cloud tab, default host), run `control-openhands browser click 'testid=add-backend-login-button' --observe 'testid=add-backend-device-flow' --observe-ms 6000`, then `control-openhands browser snapshot 'testid=add-backend-device-flow'`. If `https://app.all-hands.dev` is reachable (`control-openhands browser network --external --last 5` lists it), the snapshot shows `Waiting for authorization...`, `Browser opened. Complete sign-in to continue. If browser didn't open, visit:`, a link `https://app.all-hands.dev/oauth/device/verify?user_code=<code>` and button `Cancel`; `browser tabs` lists a second page on that URL. Screenshot with `--feature F25.cloud-device-flow --name awaiting`, then `control-openhands browser click 'testid=add-backend-auth-cancel'`: the panel is back to `Connect to OpenHands` / `Advanced` and `browser tabs` lists one page. Do not approve the code here (the **Cloud backend** bullet does, when it can). If Cloud is unreachable you get the error state of the previous bullet instead.+- **Device flow error (`F25.cloud-device-flow`).** Run `control-openhands browser fill 'testid=add-backend-cloud-host' http://127.0.0.1:9`, `control-openhands browser click 'testid=add-backend-login-button' --observe 'testid=add-backend-device-flow'` and `control-openhands browser wait 'testid=add-backend-auth-error' --timeout 15000`. The observation goes `Starting authentication...` → `Failed to start device flow: Failed to fetch` / `Try again`, and `control-openhands browser network --last 5` shows `POST /oauth/device/authorize` to `http://127.0.0.1:9`. `control-openhands browser click 'testid=add-backend-auth-retry'` starts again and fails the same way. The failed attempt closes its popup: `control-openhands browser tabs` lists one page after each failure and after the retry (`browser screenshot --feature F25.cloud-device-flow --name error-no-popup`). Close the modal with `testid=add-backend-close`. Contrast in a light theme: arrange `control-openhands browser eval "localStorage.setItem('openhands-color-theme','light-plus')"` and `browser reload`, repeat the failed attempt, and measure `[data-testid=add-backend-auth-error] p` against its composited background (`browser eval` with getComputedStyle up the ancestor chain): the `Failed to start device flow: Failed to fetch` text is at least 4.5:1 on its red tint (4.93:1 in Light+, 4.62:1 in Solarized Light on 2026-10-17). Restore with `browser eval "localStorage.removeItem('openhands-color-theme')"`.+- **Device flow awaiting and Cancel (`F25.cloud-device-flow`).** Reopen Add Backend (Cloud tab, default host), run `control-openhands browser click 'testid=add-backend-login-button' --observe 'testid=add-backend-device-flow' --observe-ms 6000`, then `control-openhands browser snapshot 'testid=add-backend-device-flow'`. If `https://app.all-hands.dev` is reachable (`control-openhands browser network --external --last 5` lists it), the snapshot shows `Waiting for authorization...`, `Browser opened. Complete sign-in to continue. If browser didn't open, visit:`, a link `https://app.all-hands.dev/oauth/device/verify?user_code=<code>` and button `Cancel`; `browser tabs` lists a second page on that URL. The status line, the `Browser opened...` line and the link stay at least 4.5:1 in Light+ and Solarized Light (link 14.85:1 and 12.25:1, `Browser opened...` 7.62:1 in Light+ on 2026-10-17). Known failure (reproduced 2026-10-17, [#18546](https://github.com/OpenHands/OpenHands/issues/18546)): the `Browser opened...` line measures 4.39:1 in Solarized Light. Screenshot with `--feature F25.cloud-device-flow --name awaiting`, then `control-openhands browser click 'testid=add-backend-auth-cancel'`: the panel is back to `Connect to OpenHands` / `Advanced` and `browser tabs` lists one page. Do not approve the code here (the **Cloud backend** bullet does, when it can). If Cloud is unreachable you get the error state of the previous bullet instead.- **Agent-server guidance (`F25.agent-server-guidance`).** Run `control-openhands browser click 'testid=add-backend-option-agent-server'` and `control-openhands browser snapshot 'testid=add-backend-agent-server-panel'`: radiogroup `Agent-server location` with `Local` checked, heading `Before you connect`, textboxes `Host Name`, `Host`, `API Key` and `button "Connect" [disabled]`. `control-openhands browser click 'testid=add-backend-local-guidance-toggle'` and `browser snapshot 'testid=add-backend-local-guidance'` show the code `agent-canvas --backend-only --port 8001` and link `Read the local backend setup guide`. `control-openhands browser click 'testid=add-backend-location-option-remote'` and `control-openhands browser click 'testid=add-backend-remote-guidance-toggle'`; `browser snapshot 'testid=add-backend-remote-guidance'` shows `Recommended setup`, `Connection details` and `Read the remote backend setup guide`.- **Connect rules and errors (`F25.add-agent-server`).** Still on Remote: `control-openhands browser fill 'testid=add-backend-name' QA_Second`, `control-openhands browser fill 'testid=add-backend-host' 127.0.0.1:9`; `control-openhands browser enabled 'testid=add-backend-submit'` is `false` (Remote needs a key). `control-openhands browser fill 'testid=add-backend-api-key' wrong-key` makes it `true`. `control-openhands browser click 'testid=add-backend-location-option-local'` keeps the fields. `control-openhands browser click 'testid=add-backend-submit' --observe 'testid=add-backend-submit'` shows `Checking…`, then `control-openhands browser text 'testid=add-backend-error'` starts `Could not connect to http://127.0.0.1:9` / `Disconnected (check URL or network). ...`. Fill `add-backend-host` with `<second-url>` and submit again: the error reads `Could not connect to <second-url>` / `Invalid API key`. On Local the key is optional for Connect but not for a key-protected host: `control-openhands browser fill 'testid=add-backend-api-key' ''` leaves `control-openhands browser enabled 'testid=add-backend-submit'` at `true` (`browser value 'testid=add-backend-api-key'` reports `length` `0`), and `control-openhands browser click 'testid=add-backend-submit'` then `control-openhands browser wait 'testid=add-backend-error' --timeout 15000` shows the same `Could not connect to <second-url>` / `Invalid API key` (`browser screenshot --feature F25.add-agent-server --name local-no-key-error`). Close with `testid=add-backend-close`.-- **Add the second backend (`F25.add-agent-server`).** Run `control-openhands browser goto /conversations/<id>`, `browser click 'testid=backend-selector'`, `browser click 'testid=add-backend-menu-item'`, `browser click 'testid=add-backend-option-agent-server'`, fill `add-backend-name` with `QA_Second` and `add-backend-host` with `<second-url>`, then `control-openhands browser fill 'testid=add-backend-api-key' --value-file "$OH_VERIFY_RUN_2/private/session-key"` and `control-openhands browser click 'testid=add-backend-submit' --expect-url '/conversations(\?|$)'`. The URL is `/conversations`, `browser count 'testid=add-backend-modal'` is `0`, `browser snapshot 'testid=backend-selector'` shows `combobox "QA_Second"`, and `browser count 'testid=conversation-card'` is `0`. Before anything reloads the page, `control-openhands browser toasts --history` lists only `Loading...` and the new backend's `Your LLM isn't set up yet, so conversations won't run. ...` (the second stack has no LLM profile; no error toast), and `control-openhands browser network --external --filter 'ids='` lists no request: the new backend is never asked for the first stack's conversation id. Known failure (reproduced 2026-10-13, #18390): on the run's first activation of `QA_Second` the list sometimes holds `GET <second-url>/api/conversations?ids=<id>` (`200`); record the bullet as `fail` then (see Gotchas). (`--external` leaves out this stack's own `GET /api/conversations?ids=<id>` from the `goto /conversations/<id>` above, which the request log keeps across page loads; `<second-url>` is another origin, so a request to it is listed.) The new backend asks its own telemetry consent ("This preference is saved for the local backend “QA_Second”"): run `control-openhands onboard --skip` (it leaves the browser on `/`). After `control-openhands browser reload` the selector still reads `QA_Second`.+- **Add the second backend (`F25.add-agent-server`).** Run `control-openhands browser goto /conversations/<id>`, `browser click 'testid=backend-selector'`, `browser click 'testid=add-backend-menu-item'`, `browser click 'testid=add-backend-option-agent-server'`, fill `add-backend-name` with `QA_Second` and `add-backend-host` with `<second-url>`, then `control-openhands browser fill 'testid=add-backend-api-key' --value-file "$OH_VERIFY_RUN_2/private/session-key"` and `control-openhands browser click 'testid=add-backend-submit' --expect-url '/conversations(\?|$)'`. The URL is `/conversations`, `browser count 'testid=add-backend-modal'` is `0`, `browser snapshot 'testid=backend-selector'` shows `combobox "QA_Second"`, and `browser count 'testid=conversation-card'` is `0`. Before anything reloads the page, `control-openhands browser toasts --history` lists only `Loading...` and the new backend's `Your LLM isn't set up yet, so conversations won't run. ...` (the second stack has no LLM profile; no error toast), and `control-openhands browser network --external --filter 'ids='` lists no request: the new backend is never asked for the first stack's conversation id. Known failure (reproduced 2026-10-13 and 2026-10-17, #18390): on the run's first activation of `QA_Second` the list sometimes holds `GET <second-url>/api/conversations?ids=<id>` (`200`); record the bullet as `fail` then (see Gotchas). (`--external` leaves out this stack's own `GET /api/conversations?ids=<id>` from the `goto /conversations/<id>` above, which the request log keeps across page loads; `<second-url>` is another origin, so a request to it is listed.) The new backend asks its own telemetry consent ("This preference is saved for the local backend “QA_Second”"): run `control-openhands onboard --skip` (it leaves the browser on `/`). After `control-openhands browser reload` the selector still reads `QA_Second`.- **Switch with the overlay (`F25.switch-backend`).** From `/` run `browser click 'testid=backend-selector'` and `control-openhands browser click 'testid=backend-selector >> role=option[name="Connected Local"]' --observe 'testid=environment-switch-overlay' --observe-ms 2500`. The observation shows `Switching to Local`, then `<absent>` about one second later; the selector reads `Local` and `browser count 'testid=conversation-card'` is `1`. Then `control-openhands browser click 'testid=conversation-card' --expect-url '/conversations/[0-9a-f-]+'`, open the selector and `control-openhands browser click 'testid=backend-selector >> role=option[name="Connected QA_Second"]' --expect-url '/conversations(\?|$)'`: the URL is `/conversations`, the selector reads `QA_Second` and the card count is `0`.- **Pinned URL (`F25.backend-pinned-url`).** With `QA_Second` active run `control-openhands browser goto '/conversations/<id>?backend=default-local'` and `control-openhands browser wait 'testid=chat-interface'`: the selector reads `Local` and the conversation opens. `control-openhands browser goto '/conversations?backend=qa-unknown-id'` keeps `Local` (card count `1`). The sidebar card links themselves end in `?backend=default-local` (`browser snapshot`).- **Per-tab backend (`F25.per-tab-backend`).** With `Local` active and `QA_Second` registered, run `control-openhands browser goto /conversations` and `control-openhands browser eval "JSON.parse(sessionStorage.getItem('openhands-active-backend')).backendId"` (`default-local`); note QA_Second's id with `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).find(b=>b.name==='QA_Second').id"` (`<second-id>`, a generated UUID). Open a plain second tab, as a user opening the app again: `control-openhands browser tab new /conversations` prints `index` `1` and makes it the daemon's active tab (`control-openhands browser tabs` lists two pages at `/conversations` with `active` `1`); `control-openhands browser wait-tab '/conversations$' --new --timeout 5000` then prints `index` `0`, the tab you came from, because `--new` leaves out the tab the daemon is on and that is now the new one (run from `browser tab 0` it prints `index` `1`, the second tab). `control-openhands browser tab 1`, `control-openhands browser wait 'testid=backend-selector'`: `browser snapshot 'testid=backend-selector'` reads `combobox "Local"` and `control-openhands browser eval "sessionStorage.getItem('openhands-active-backend')"` is `null` (a new tab inherits nothing and starts from the localStorage fallback). Switch this tab: `browser click 'testid=backend-selector'`, `control-openhands browser click 'testid=backend-selector >> role=option[name="Connected QA_Second"]'`, `control-openhands browser wait 'testid=environment-switch-overlay' --state detached --timeout 10000`; the selector reads `QA_Second`, `browser count 'testid=conversation-card'` is `0` and `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-active-backend')).backendId"` is `<second-id>`. Back in `control-openhands browser tab 0` nothing moved: the selector still reads `Local`, the card count is `1`, the sessionStorage eval is still `default-local` while the localStorage eval is `<second-id>` (`browser screenshot --feature F25.per-tab-backend --name first-tab-unchanged`). A plain new tab starts from the last choice made in any tab: from tab 0 run `control-openhands browser tab new /conversations` (`index` `2`, now active), `browser wait 'testid=backend-selector'`: the selector reads `QA_Second`, the card count is `0`, its sessionStorage item is `null` and the localStorage eval is `<second-id>` (`--name plain-new-tab-last-choice`). A conversation row opens on its owner instead: `control-openhands browser tab 0`, `control-openhands browser click 'testid=conversation-card' --modifiers Control` (add `>> nth=0` when the run holds more conversations than the precondition's one: the click is strict), `control-openhands browser wait-tab '/conversations/<id>' --timeout 20000` prints `index` `3` and the URL `/conversations/<id>?backend=default-local`; `control-openhands browser tab 3`, `control-openhands browser wait 'testid=chat-interface' --timeout 30000`, `browser url` still ends in `/conversations/<id>?backend=default-local` (no bounce to `/conversations`), the selector reads `Local` and `browser toasts --history` lists only `Loading...` (no `This conversation does not exist…`); `browser screenshot --feature F25.per-tab-backend --name new-tab-on-owner`. Close the extra tabs highest first: `control-openhands browser close-tab 3`, `control-openhands browser close-tab 2`, `control-openhands browser close-tab 1`, `control-openhands browser tab 0`; `browser tabs` lists one page and the selector reads `Local`. Opening the pinned link also recorded `default-local` as the last choice (the localStorage eval in tab 3 reads `default-local`), so run the plain-new-tab check before the row check.- **Manage list and closing (`F25.manage-backends`).** Run `browser click 'testid=backend-selector'`, `control-openhands browser click 'testid=manage-backends-menu-item'` and `control-openhands browser snapshot 'testid=manage-backends-modal'`: heading `Manage backends`, two rows `Connected Local v1.50.1 <this run's URL> Connected Local` and `Connected QA_Second v1.50.1 <second-url> Connected Local`, each with `Edit` and `Remove`, then `Add Backend` and `Done`. `browser text 'testid=manage-backends-status-QA_Second'` is `Connected` and `browser text 'testid=manage-backends-version-QA_Second'` is `v1.50.1` (whatever `doctor` reports). Screenshot with `--feature F25.manage-backends --name list`. Focus is inside: the Add Backend bullet's focus eval with `manage-backends-modal` in place of `add-backend-modal` reads `close-manage-backends-modal` on open and stays `inDialog` `true` over nine Tabs and two Shift+Tabs (with two backends listed, Tab 8 reaches **Done** and Tab 9 wraps to the close button); after Escape, focus is back on the selector. Close it four ways, reopening between: `browser press Escape`, `control-openhands browser mouse-click 20 500` (backdrop), `browser click 'testid=close-manage-backends-modal'`, `browser click 'testid=manage-backends-done'`; each time `browser count 'testid=manage-backends-modal'` is `0`.- **Select from Manage (`F25.manage-select`).** With `Local` active, `control-openhands browser click 'testid=conversation-card' --expect-url '/conversations/[0-9a-f-]+'`, open Manage Backends, then `control-openhands browser click 'testid=manage-backends-row-QA_Second >> role=button >> nth=0' --expect-url '/conversations(\?|$)'`. Like the selector, the app leaves the detail page for `/conversations` (BM-002): `browser count 'testid=manage-backends-modal'` is `0`, the selector reads `QA_Second`, `browser count 'testid=chat-interface'` and `browser count 'testid=conversation-card'` are `0`, and `browser screenshot --feature F25.manage-select --name after-select` shows QA_Second's Home with its empty conversation list.- **Edit validation and errors (`F25.edit-backend`).** Open Manage Backends, `control-openhands browser click 'testid=manage-backends-edit-QA_Second'`, then `browser value 'testid=edit-backend-host'` (`<second-url>`), `browser attr 'testid=edit-backend-api-key' type` (`password`) and `browser text 'testid=edit-backend-status'` (`Connected`, `·`, `Local`, `v1.50.1` on separate lines). Fill `edit-backend-name` with an empty string and `edit-backend-host` with `not a host`, then `browser focus 'testid=edit-backend-api-key'` and `browser snapshot 'testid=edit-backend-form'`: alerts `Name is required` and `Enter a valid URL (e.g. http://localhost:8080)`; `browser enabled 'testid=edit-backend-submit'` is `false`.-- **Escape in Edit (`F25.edit-escape`).** In the same Edit modal press `control-openhands browser press Escape`. Expected: the Edit modal stays (it disables Escape) or only it closes. Today `browser count 'testid=edit-backend-modal'` and `browser count 'testid=manage-backends-modal'` are both `0`: the whole stack closes and the draft is lost. Nothing was saved (`browser snapshot 'testid=backend-selector'` still reads `QA_Second`).-- **Edit and save (`F25.edit-backend`).** Reopen Manage Backends and the pencil. Fill `edit-backend-host` with `http://127.0.0.1:9`, `browser click 'testid=edit-backend-submit'`, `browser wait 'testid=edit-backend-error' --timeout 10000`: the text starts `Could not connect to http://127.0.0.1:9` and the modal stays. Fill the host back with `<second-url>` and `edit-backend-name` with `QA_Renamed`, click `edit-backend-submit`, then `control-openhands browser wait 'testid=edit-backend-modal' --state detached --timeout 10000`; `browser count 'testid=manage-backends-row-QA_Renamed'` is `1`. Click `manage-backends-done`, `browser reload`: the selector reads `QA_Renamed`.+- **Escape in Edit (`F25.edit-escape`).** In the same Edit modal run `control-openhands browser fill 'testid=edit-backend-name' QA_Draft` and `control-openhands browser press Escape`: `browser count 'testid=edit-backend-modal'` and `browser count 'testid=manage-backends-modal'` both stay `1` and `browser value 'testid=edit-backend-name'` is still `QA_Draft` (`browser screenshot --feature F25.edit-escape --name after-escape`). Leave with `browser click 'testid=edit-backend-cancel'`; nothing was saved (the Manage row is still `QA_Second`).+- **Edit and save (`F25.edit-backend`).** Manage Backends is still open: click the pencil `testid=manage-backends-edit-QA_Second` again. Fill `edit-backend-host` with `http://127.0.0.1:9`, `browser click 'testid=edit-backend-submit'`, `browser wait 'testid=edit-backend-error' --timeout 10000`: the text starts `Could not connect to http://127.0.0.1:9` and the modal stays. Fill the host back with `<second-url>` and `edit-backend-name` with `QA_Renamed`, click `edit-backend-submit`, then `control-openhands browser wait 'testid=edit-backend-modal' --state detached --timeout 10000`; `browser count 'testid=manage-backends-row-QA_Renamed'` is `1`. Click `manage-backends-done`, `browser reload`: the selector reads `QA_Renamed`.- **Health status (`F25.health-status`).** With QA_Renamed the active backend (as the Edit bullet's reload leaves it; the closed selector shows only the active backend's dot), run `control-openhands service stop agent-server --run "$OH_VERIFY_RUN_2"`, then on the open page `control-openhands browser wait 'testid=backend-selector >> role=status[name="Disconnected"]' --timeout 40000`. `browser click 'testid=backend-selector'` and `browser snapshot 'testid=backend-selector'` show `option "Disconnected QA_Renamed"` next to `option "Connected Local"`. In Manage Backends `browser text 'testid=manage-backends-status-QA_Renamed'` is `Disconnected (check URL or network)`, `testid=manage-backends-status-detail-QA_Renamed` repeats it with `Check that the backend URL is correct ...`, and `browser enabled 'testid=manage-backends-row-QA_Renamed >> role=button >> nth=0'` is `false`. An error toast with the same text also appears. Screenshot with `--feature F25.health-status --name manage-down`.- **Recovery gate (`F25.recovery-gate`).** Close the modal, `control-openhands browser reload`, then `control-openhands browser wait 'testid=manage-backends-modal' --timeout 20000`. `browser testids` lists `agent-server-onboarding-screen` and the Manage rows but no sidebar; `browser count 'testid=close-manage-backends-modal'` and `browser count 'testid=manage-backends-done'` are `0`, and `browser press Escape` leaves the modal (`count` `1`). `control-openhands browser click 'testid=manage-backends-row-Local >> role=button >> nth=0'` and `browser wait 'testid=agent-server-onboarding-screen' --state detached --timeout 15000` restore the app: selector `Local`, card count `1`. Screenshot the gate with `--feature F25.recovery-gate --name gate` before choosing.- **Remove the active backend (`F25.remove-backend`).** Run `control-openhands restart --run "$OH_VERIFY_RUN_2"`, `browser click 'testid=backend-selector'`, `browser wait 'testid=backend-selector >> role=option[name="Connected QA_Renamed"]' --timeout 25000` and click that option (selector reads `QA_Renamed`). Open Manage Backends, `control-openhands browser click 'testid=manage-backends-remove-QA_Renamed'`; `browser text 'testid=confirmation-modal'` reads `Remove backend "QA_Renamed"? If it is active, the app will switch back to Local.` `control-openhands browser click 'testid=confirmation-modal >> role=button[name="Cancel"]'` keeps the row (`count 'testid=manage-backends-row-QA_Renamed'` `1`). Click the trash again, then `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'`: the row count is `0`. Click Done and `browser reload`: the selector reads `Local`, `browser count 'testid=backend-selector >> role=option'` (dropdown open) is `1`, and the card count is `1`.-- **Add from Manage, Edit Cancel and X (`F25.manage-add`, `F25.edit-cancel`).** The second stack is running again after the previous bullet. Run `browser goto /conversations/<id>`, `browser wait 'testid=chat-interface'`, open the selector, click `manage-backends-menu-item`, then `control-openhands browser click 'testid=manage-backends-add'` and `control-openhands browser wait 'testid=add-backend-modal' --timeout 5000` (`visible`); `browser count 'testid=manage-backends-modal'` is `1` underneath. `browser click 'testid=add-backend-close'`: the Add count is `0` and the Manage count stays `1`. Click `manage-backends-add` again, `add-backend-option-agent-server`, fill `add-backend-name` with `QA_Second`, `add-backend-host` with `<second-url>` and `add-backend-api-key` with `--value-file "$OH_VERIFY_RUN_2/private/session-key"`, then `browser click 'testid=add-backend-submit' --expect-url '/conversations(\?|$)'`: the URL is `/conversations`, `count 'testid=add-backend-modal'` is `0`, `count 'testid=manage-backends-modal'` is still `1`, `count 'testid=manage-backends-row-QA_Second'` is `1` and the selector reads `QA_Second` (screenshot `--feature F25.manage-add --name after-add`). No consent prompt this time: it is stored on the second backend. Then `control-openhands browser click 'testid=manage-backends-edit-QA_Second'`, `browser fill 'testid=edit-backend-name' QA_Draft` and `control-openhands browser click 'testid=edit-backend-cancel'`: `count 'testid=edit-backend-modal'` `0`, `count 'testid=manage-backends-modal'` `1`, row `QA_Second` `1`, row `QA_Draft` `0`. Repeat with `control-openhands browser click 'testid=edit-backend-close'`: same counts, and reopening the pencil shows `browser value 'testid=edit-backend-name'` `QA_Second`; leave with `edit-backend-cancel`. Remove it again: `browser click 'testid=manage-backends-remove-QA_Second'`, `browser click 'testid=confirmation-modal >> testid=confirm-button'`, `browser click 'testid=manage-backends-done'` and `browser reload`: the selector reads `Local`.+- **Add from Manage, Edit Cancel and X (`F25.manage-add`, `F25.edit-cancel`).** The second stack is running again after the previous bullet. Run `browser goto /conversations/<id>`, `browser wait 'testid=chat-interface'`, open the selector, click `manage-backends-menu-item`, then `control-openhands browser click 'testid=manage-backends-add'` and `control-openhands browser wait 'testid=add-backend-modal' --timeout 5000` (`visible`); `browser count 'testid=manage-backends-modal'` is `1` underneath. `control-openhands browser press Escape` leaves both (`add-backend-modal` and `manage-backends-modal` counts `1`); `browser click 'testid=add-backend-close'`: the Add count is `0` and the Manage count stays `1`. Click `manage-backends-add` again, `add-backend-option-agent-server`, fill `add-backend-name` with `QA_Second`, `add-backend-host` with `<second-url>` and `add-backend-api-key` with `--value-file "$OH_VERIFY_RUN_2/private/session-key"`, then `browser click 'testid=add-backend-submit' --expect-url '/conversations(\?|$)'`: the URL is `/conversations`, `count 'testid=add-backend-modal'` is `0`, `count 'testid=manage-backends-modal'` is still `1`, `count 'testid=manage-backends-row-QA_Second'` is `1` and the selector reads `QA_Second` (screenshot `--feature F25.manage-add --name after-add`). No consent prompt this time: it is stored on the second backend. Then `control-openhands browser click 'testid=manage-backends-edit-QA_Second'`, `browser fill 'testid=edit-backend-name' QA_Draft` and `control-openhands browser click 'testid=edit-backend-cancel'`: `count 'testid=edit-backend-modal'` `0`, `count 'testid=manage-backends-modal'` `1`, row `QA_Second` `1`, row `QA_Draft` `0`. Repeat with `control-openhands browser click 'testid=edit-backend-close'` and with a backdrop click (`control-openhands browser mouse-click 20 500`): same counts, and reopening the pencil shows `browser value 'testid=edit-backend-name'` `QA_Second`; leave with `edit-backend-cancel`. Remove it again: `browser click 'testid=manage-backends-remove-QA_Second'`, `browser click 'testid=confirmation-modal >> testid=confirm-button'`, `browser click 'testid=manage-backends-done'` and `browser reload`: the selector reads `Local`.- **Collapsed rail (`F25.collapsed-entry`).** Run `control-openhands browser click 'testid=sidebar-collapse-toggle'` (`attr 'aside[data-collapsed]' data-collapsed` is `true`), `control-openhands browser hover 'testid=collapsed-backend-selector-link'`, `control-openhands browser click 'testid=add-backend-menu-item'` and `control-openhands browser wait 'testid=add-backend-modal' --timeout 5000` (`visible`). Close it, hover again, click `manage-backends-menu-item` and `browser wait 'testid=manage-backends-modal' --timeout 5000`. Close with Done and expand with `browser click 'testid=sidebar-collapse-toggle'`.- **Phone (`F25.phone`).** Run `control-openhands browser viewport phone`, `browser goto /conversations`, `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'`, `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=backend-selector'`, `browser click 'testid=add-backend-menu-item'`, `browser click 'testid=add-backend-option-agent-server'` and `control-openhands browser bbox 'testid=add-backend-modal'`: `insideViewport` `true`, `pageHorizontalOverflow` `false`. Screenshot `--feature F25.phone --name add-modal`. Close it (the drawer stays open), open Manage Backends from the drawer selector the same way and check `bbox 'testid=manage-backends-modal'` the same. Click `testid=manage-backends-edit-Local` and check `bbox 'testid=edit-backend-modal'`: `insideViewport` `true`; repeat at `browser viewport 320x568`. At `browser viewport 320x400`, clear `edit-backend-name`, fill `edit-backend-host` with `not a host` and focus `edit-backend-api-key` to show both validation messages. Run `browser scroll 'testid=edit-backend-form' --by 600`: Cancel and Save must come into view while the title and `edit-backend-close` stay visible. Check the modal and all three controls with `bbox`, then close with Cancel. Return with `browser viewport desktop`; reopen Edit and confirm its width is still 520px.- **Providers tab (`F25.providers-tab`).** On the default run `browser eval "!!window.marsBridge"` is `true` (Preconditions); `curl -s "http://127.0.0.1:$(control-openhands status | jq -r .ports.frontend)/mars/health"` (the static child behind the ingress) returns the HTML index, not JSON: one bridge per stack. Open Add Backend: the tablist adds `tab "Providers Connect through DigitalOcean and more."` (`testid=add-backend-option-providers`); `control-openhands browser click 'testid=add-backend-option-providers'` and `browser snapshot 'testid=add-backend-providers-panel'` show `button "DigitalOcean Your team's OpenHands agents"` and `More providers coming soon.` (`browser screenshot --feature F25.add-backend-modal --name with-providers-tab`). At phone width the modal stays inside the viewport with the three tab titles cut to `Ope…`, `Age…`, `Pro…` (`--feature F25.providers-tab --name phone-tabs`).- **DigitalOcean sign-in (`F25.digitalocean-sign-in`).** On the default run, run `control-openhands browser click 'testid=add-backend-provider-digitalocean'` and `browser wait 'testid=managed-agents-sign-in'`: heading `Connect your DigitalOcean team`, `textbox "Personal access token"` (`browser attr 'testid=managed-agents-token' type` is `password`), link `Create a token` (`/url: https://cloud.digitalocean.com/account/api/tokens`) and `button "Connect with token" [disabled]`; no OAuth button in a browser (`browser count 'testid=managed-agents-oauth'` is `0`). Fill `testid=managed-agents-token` with `dop_v1_qa_invalid_token_0000` and click `testid=managed-agents-token-submit`: `browser text 'testid=managed-agents-auth-error'` reads `That does not look like a DigitalOcean personal access token.` (`POST /mars/rpc/savePat` `500`). Fill a well-formed dummy, `dop_v1_` plus 64 zeros, and submit again: after a few seconds the error reads `401 Unauthorized` (`POST /mars/rpc/savePat` `401`, DigitalOcean's answer; `browser screenshot --feature F25.digitalocean-sign-in --name rejected-token`). Never type a real token here. `control-openhands browser click 'testid=add-backend-providers-back'` returns to the Providers panel (`browser count 'testid=managed-agents-sign-in'` is `0`; the providers panel itself stays mounted behind the sign-in screen, so its count is `1` either way); close with `testid=add-backend-close`. At phone width the sign-in screen fits (`bbox 'testid=add-backend-modal'` `insideViewport` `true`). Both errors also raise an error toast with the same text. Expected on a page served without a baked key (F26 `--host 0.0.0.0`, Docker), once the server's key was typed on the API-key screen: the same two answers as above. Known failure (reproduced 2026-10-14, #18434): every token, even the malformed one, ends in `session key required` (`POST /mars/rpc/getAuthState` and `savePat` `401`), because the web bridge client sends only the baked page key, never the typed one; it predates #18349 (BASE with `MARS_WEB=1` fails the same way). A `--frontend-only` page connected to another stack's backend fails the same way for a different reason (the typed key belongs to the other stack); #18434 leaves that case to a maintainer decision.- **Managed Agents with an account (`F25.digitalocean-agents`, `F25.digitalocean-backend-removal`).** Blocked: needs a DigitalOcean account with Managed Agents. With one, after a real token the panel lists agents (`digitalocean-agent-<id>`), **New Agent** opens `digitalocean-new-agent-form` (name, size), a session's Connect adds a backend and the sidebar shows `mars-session-panel`; removing that backend opens `mars-backend-removal-modal` with `disconnect-only-button` and `stop-session-button`.- **Cloud backend (`F25.cloud-device-flow`, `F25.cloud-org-rows`, `F25.cloud-settings-link`, `F25.share-publicly`, `F25.cloud-sandbox-states`).** Only with a Cloud account (Preconditions); it creates a device grant and one conversation on that account, and shows the account's own conversations in the sidebar (keep those screenshots private). Reopen Add Backend on the Cloud tab, `browser click 'testid=add-backend-login-button'`, then `control-openhands browser tab 1` (the verify popup: `Device Authorization Request`, the code, **Authorize Device**), `control-openhands browser click 'role=button[name="Authorize Device"]'` (`Success!`) and `control-openhands browser tab 0`. Within about 15 s (token polling answers `400` until approved, then `200`) `browser count 'testid=add-backend-modal'` is `0`, the popup is gone and `browser snapshot 'testid=backend-selector'` reads `combobox "OpenHands Cloud": OpenHands Cloud – Personal Workspace`; the registry holds `{name: "OpenHands Cloud", host: "https://app.all-hands.dev", kind: "cloud"}` (`browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({name:b.name,host:b.host,kind:b.kind}))"`). Open the selector: `option "Connected OpenHands Cloud – Personal Workspace"` (one option per organization) next to `Connected Local`; Manage Backends shows the row with `Personal Workspace` and the `Cloud` pill. `browser attr 'testid=backend-selector-settings-link' href` is `https://app.all-hands.dev/settings?org=<org id>` with `target` `_blank`; clicking it opens that tab (`browser wait-tab 'app.all-hands.dev/settings'`, then `browser close-tab 1`, `browser tab 0`). Start one conversation from Home: fill `testid=chat-input` with `Reply with exactly: QA_F25_CLOUD. Do not run any tools.` and `browser click 'testid=submit-button' --expect-url '/conversations/'`: the URL is `/conversations/task-<id>` while the sandbox starts (toasts `Creating conversation…`, `Sending...`), then moves to `/conversations/<id>?backend=<cloud id>&org=<org id>` and the reply `QA_F25_CLOUD` appears (about a minute). Its `conversation-name >> ellipsis-button` menu has `share-publicly-button` (**Public Share**, off); do not flip it unless the conversation may be public. Delete it with `delete-button` and `'role=dialog >> testid=confirm-button'`, then remove the Cloud backend in Manage Backends (`manage-backends-remove-OpenHands Cloud`, confirm): after `browser reload` the selector reads `Local`.-- **Device verify page (`F25.device-verify-page`).** Run `control-openhands browser goto /oauth/device/verify` and `browser snapshot`: heading `Device Authorization`, `Enter the code displayed on your device:`, textbox `Device Code:` and button `Continue`, inside the normal app shell. `control-openhands browser fill 'role=textbox[name="Device Code:"]' QA-0000` and `browser click 'role=button[name="Continue"]'` go straight to the result: heading `Error`, `Failed to authorize device. Please try again.`, button `Try Again`; `browser network --filter verify-authenticated` shows `POST /oauth/device/verify-authenticated` `404`. Then `control-openhands browser goto '/oauth/device/verify?user_code=QA-0000'`: heading `Device Authorization Request`, `DEVICE CODE`, `QA-0000`, `Security Notice`, buttons `Cancel` and `Authorize Device` (screenshot `--feature F25.device-verify-page --name with-code`). `browser click 'role=button[name="Cancel"]'` does nothing in a tab the page did not open (URL unchanged). `browser click 'role=button[name="Authorize Device"]'` and `browser wait 'role=heading[name="Error"]'` show the same Error card; **Try Again** reloads to the request. Known failure (reproduced 2026-10-16, [#18527](https://github.com/OpenHands/OpenHands/issues/18527)): with `?user_code=QA-NOT-REAL` the Security Notice heading and instruction measure 1.66:1 and 1.74:1 in Light+ (1.71:1 and 1.16:1 in Solarized Light) on their amber tint, below 4.5:1.-- **Shared view (`F25.shared-conversation-view`).** Run `control-openhands browser goto /shared/conversations/<id>` and `control-openhands browser wait-text 'Conversation not found' --timeout 15000`. The page has no sidebar; `browser network --filter shared` shows `GET /api/shared-conversations` and `/api/shared-events/search` `404`. Screenshot `--feature F25.shared-conversation-view --name local-not-found`.+- **Device verify page (`F25.device-verify-page`).** Run `control-openhands browser goto /oauth/device/verify` and `browser snapshot`: heading `Device Authorization`, `Enter the code displayed on your device:`, textbox `Device Code:` and button `Continue`, inside the normal app shell. `control-openhands browser fill 'role=textbox[name="Device Code:"]' QA-0000` and `browser click 'role=button[name="Continue"]'` go straight to the result: heading `Error`, `Failed to authorize device. Please try again.`, button `Try Again`; `browser network --filter verify-authenticated` shows `POST /oauth/device/verify-authenticated` `404`. Then `control-openhands browser goto '/oauth/device/verify?user_code=QA-0000'`: heading `Device Authorization Request`, `DEVICE CODE`, `QA-0000`, `Security Notice`, buttons `Cancel` and `Authorize Device` (screenshot `--feature F25.device-verify-page --name with-code`). `browser click 'role=button[name="Cancel"]'` does nothing in a tab the page did not open (URL unchanged). `browser click 'role=button[name="Authorize Device"]'` and `browser wait 'role=heading[name="Error"]'` show the same Error card; **Try Again** reloads to the request. In Light+ and Solarized Light (arrange `localStorage.setItem('openhands-color-theme', ...)` as in the Add Backend bullet), `?user_code=QA-NOT-REAL` keeps the Security Notice heading and instruction at least 4.5:1 on their amber tint, at desktop and phone width (2026-10-17: 5.03:1 and 13.58:1 in Light+, 5.53:1 and 11.84:1 in Solarized Light; `--name notice-light-plus-phone`).+- **Shared view (`F25.shared-conversation-view`).** Run `control-openhands browser goto /shared/conversations/<id>` and `control-openhands browser wait-text 'Conversation not found' --timeout 15000`. The page has no sidebar; `browser network --filter shared` shows `GET /api/shared-conversations` and `/api/shared-events/search` `404`, and `control-openhands browser toasts --history` stays `[]` (the page shows its own not-found state, no toast). Screenshot `--feature F25.shared-conversation-view --name local-not-found`.- **No Public Share locally (`F25.share-publicly`).** Run `browser goto /conversations/<id>`, `control-openhands browser click 'testid=conversation-name >> testid=ellipsis-button'` and `control-openhands browser count 'testid=share-publicly-button'`: `0` on a local backend (`browser testids --filter button` lists `rename-button`, `show-skills-button`, `show-agent-tools-button`, `export-transcript-button`, `download-trajectory-button`, `display-cost-button` and `delete-button`; `show-hooks-button` and `stop-button` appear only while the conversation is active (idle, running, waiting for confirmation or finished), so a paused or errored conversation has neither). The toggle, `copy-share-link-button` and `open-share-link-button` need a Cloud backend: blocked.- **Cleanup.** `browser press Escape`, then `control-openhands stop --run "$OH_VERIFY_RUN_2"` and `unset OH_VERIFY_RUN_2`. The conversation stays with this run's state.- **Errors.** `control-openhands browser errors --app-only` after the family shows no page errors. Expected app-origin entries: `POST /oauth/device/verify-authenticated` 404 (twice), the shared-view 404s, the warning `Scripts may close only the windows that were opened by them.` from the device-verify Cancel, and a burst of CORS `console.error`s for `<second-url>` while the second agent server is stopped or restarting (see Gotchas).@@ -128,10 +128,9 @@ Preconditions:- A tab opened with `click --modifiers Control` starts with an empty sessionStorage, so it boots from the localStorage fallback (the backend last chosen in any tab) unless its URL carries `?backend=`. Opening a `?backend=` link writes that backend to localStorage too: a per-tab check that relies on "last choice" must run before opening pinned links.- The Add Backend Agent-server form has no inline "required" messages; it only disables **Connect**. The messages exist in the Edit form.- Footer clicks from the collapsed popover or the phone drawer open the modals a moment later: `browser wait` for them instead of `count`.-- The device-flow popup survives errors: every failed attempt leaves an `about:blank` tab ([#17953](https://github.com/OpenHands/OpenHands/issues/17953)). Cancel closes it; after an error close it with `browser close-tab`.- With the default host the device flow talks to the real `app.all-hands.dev` (`POST /oauth/device/authorize`, then token polling). Starting and cancelling is harmless; approve a code only in the **Cloud backend** bullet, with an account you may use.- While the second stack's agent server is stopped or restarting, its ingress answers without CORS headers, so the browser logs `Access to fetch at '<second-url>/api/settings' ... blocked by CORS policy` (also `/server_info`, `/api/conversations/search`, `/api/llm/models/verified`). That is the dead upstream, not a CORS bug; a healthy second stack sends `access-control-allow-origin` for this origin.-- Expected: Escape never clears the backend selector. Known failure (reproduced 2026-10-09, 2026-10-12, 2026-10-13 and 2026-10-14, [#18233](https://github.com/OpenHands/OpenHands/issues/18233)): Escape on the focused backend selector while its menu is closed (a second Escape after the one that closes the menu, and sometimes the Escape that closes Manage Backends opened from it) clears the selection: the closed selector shows the active backend's name as an italic placeholder (`browser value 'testid=backend-selector >> role=combobox'` is `""`) and its status dot is gone, so `browser wait 'testid=backend-selector >> role=status[name="Disconnected"]'` times out although the backend is down; a reload restores it. Reload before a closed-selector status check, as the Edit bullet does. Repro: `browser click 'testid=backend-selector'`, `browser press Escape` twice ([#18233](https://github.com/OpenHands/OpenHands/issues/18233)). On 2026-10-12 `bug proof` verified fix PR #18238 at 77de0ad24 against main f07aebdf5 (two `xfail`, two `xpass`, desktop). `control-openhands bug run F25.selector-dropdown` replays it (desktop, after a reload with the mouse off the selector: a reload under a mouse left on it opens its menu and empties its field, which would look like the bug) and reports `xfail` while the bug lasts:+- Expected: Escape never clears the backend selector. Known failure (reproduced 2026-10-09, 2026-10-12, 2026-10-13, 2026-10-14 and 2026-10-17, [#18233](https://github.com/OpenHands/OpenHands/issues/18233)): Escape on the focused backend selector while its menu is closed (a second Escape after the one that closes the menu, and sometimes the Escape that closes Manage Backends opened from it) clears the selection: the closed selector shows the active backend's name as an italic placeholder (`browser value 'testid=backend-selector >> role=combobox'` is `""`) and its status dot is gone, so `browser wait 'testid=backend-selector >> role=status[name="Disconnected"]'` times out although the backend is down; a reload restores it. Reload before a closed-selector status check, as the Edit bullet does. Repro: `browser click 'testid=backend-selector'`, `browser press Escape` twice ([#18233](https://github.com/OpenHands/OpenHands/issues/18233)). On 2026-10-12 `bug proof` verified fix PR #18238 at 77de0ad24 against main f07aebdf5 (two `xfail`, two `xpass`, desktop). `control-openhands bug run F25.selector-dropdown` replays it (desktop, after a reload with the mouse off the selector: a reload under a mouse left on it opens its menu and empties its field, which would look like the bug) and reports `xfail` while the bug lasts:```sh known-bug F25.selector-dropdown escape-closedcontrol-openhands browser viewport desktop@@ -148,8 +147,6 @@ Preconditions:```- Local and remote backends are probed every 30 s (settings + server_info), Cloud backends every 5 min (#18128), so allow up to 40 s for a local dot to change. `browser network --clear`, wait about 70 s, then `browser network --filter server_info` shows two probes per local origin 30 s apart. While the active backend is down, any reload lands on the recovery gate: drive the disconnected rows on the page that was already loaded.- `/oauth/device/verify` is a Cloud page that also ships in Canvas: on a local stack its POST target does not exist, so Authorize always ends in `Error`. The "Authentication Required" state is unreachable (`useIsAuthed` always resolves `true`).-- `/shared/conversations/<id>` on a local backend also raises a raw toast `HTTP request failed (404 Not Found): {"detail":"Not Found"}` next to `Conversation not found` ([#17953](https://github.com/OpenHands/OpenHands/issues/17953)).-- Known failure (repro candidate, not an exemption): `F25.edit-escape` closes the Manage modal underneath and drops the Edit draft (Cancel and X close only Edit, `F25.edit-cancel`) ([#17953](https://github.com/OpenHands/OpenHands/issues/17953)).- The add bullet's `network --external --filter 'ids='` check guards an earlier race: [#18390](https://github.com/OpenHands/OpenHands/issues/18390) tracks the same first-activation symptom; on a backend's first activation from `/conversations/<id>`, about 1 attempt in 5 asked the new backend for the old conversation id. It did not happen in 15 first activations at 4eff80237: each time `QA_Second` was removed, its consent unset with `control-openhands api PATCH /api/settings --run "$OH_VERIFY_RUN_2" --write --data '{"misc_settings_diff":{"app_preferences":{"user_consents_to_analytics":null}}}'` (arrange only) and the backend added again from `/conversations/<id>`. A backend that lacks an id answers `[null]`, which now reads as "not found" (only the `This conversation does not exist…` toast), not as incompatible data. No merged change was shown to fix the race itself (#18009 only changed how a `[null]` answer reads; #18024 only moved the redirect into a shared hook, still called after `addBackend`), so a hit (an `ids=` request to `<second-url>`) is a bug to file. At 95115e84f6dd it hit on 1 of 2 first activations from `/conversations/<id>` (one `GET <second-url>/api/conversations?ids=<id>` answered `200`, with no `This conversation does not exist…` toast after it). The hit followed a failed Connect in the same modal (an empty key, `Invalid API key`) before the successful one; the clean retry, which connected once, had none, so a repro should try both sequences. On 2026-10-08 at d2c89252d it did not reproduce in 17 first activations (8 clean, 9 failed-then-clean, some under CPU load and some human-paced): every `ids=` capture was empty, and no merged change touched the activation logic since the hit, so it stays a watch item rather than a filed defect. Record the Add bullet as `fail` when the request appears, even though the redirect and the selector look right. Known failure (reproduced 2026-10-13, #18390): two independent first activations hit it that day, each the run's first activation of `QA_Second` after a clean Connect. On 2026-10-13 at 2bdc0e9d6 it hit on 1 of 6 first activations from `/conversations/<id>`: the first one of the run (`GET <second-url>/api/conversations?ids=<id>` `200`, no toast), a clean Connect in a modal opened after earlier failed Connects in other Add Backend openings; the next 5 (2 failed-then-clean, 3 clean, plus one Manage → Add) had none.- Add Backend and Manage Backends take and keep keyboard focus, and Edit backend takes it on open (`ModalBackdrop`; at phone width open the selector with `'testid=sidebar-mobile-drawer >> testid=backend-selector'`, since the bare test id matches two selectors). At desktop width focus returns to the selector input when they close (at phone width Add Backend's X closes the drawer too and focus falls to the page body), so an Escape pressed after closing lands on the selector and can trigger [#18233](https://github.com/OpenHands/OpenHands/issues/18233) below.- Toasts are not page or HTTP errors, so `browser errors --app-only` misses them: read `browser toasts --history` after every add or switch, before anything reloads the page (the history starts with each page load).diff --git a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.mdindex dfd5e12f7b..c9b083d7bd 100644--- a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md+++ b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md@@ -26,7 +26,7 @@ Source: `bin/agent-canvas.mjs`, `scripts/dev-with-automation.mjs`, `scripts/dev-- `F26.session-key-persist`: in local mode without `LOCAL_BACKEND_API_KEY` the launcher generates a session key, saves it under `$HOME/.openhands/agent-canvas/api-key.txt` and injects the same key again on the next start.- `F26.session-key-pinned`: `LOCAL_BACKEND_API_KEY=<key>` in local mode injects exactly that key into the page and the API accepts only it (the saved generated key gets 401).- `F26.frontend-only`: `--frontend-only` serves the SPA; `/server_info`, `/api/*`, `/sockets/*` and `/api/automation/*` answer 503, and a browser new to that origin opens on the **Add a backend** onboarding step without an error toast.-- `F26.frontend-only-returning`: on a frontend-only origin, a browser that already stores a backend for that origin (the `default-local` entry a full launcher on the same port seeded, or one added there earlier) gets the recovery gate instead of onboarding: `agent-server-onboarding-screen` with Manage backends in recovery mode and the `Local` row `Disconnected` with the 503 detail (the gate itself is F25's recovery-gate row); the failing probes raise generic `An error occurred` toasts there (Known failure, reproduced 2026-10-08: [#18160](https://github.com/OpenHands/OpenHands/issues/18160)).+- `F26.frontend-only-returning`: on a frontend-only origin, a browser that already stores a backend for that origin (the `default-local` entry a full launcher on the same port seeded, or one added there earlier) gets the recovery gate instead of onboarding: `agent-server-onboarding-screen` with Manage backends in recovery mode and the `Local` row `Disconnected` with the 503 detail (the gate itself is F25's recovery-gate row); its failing 503 probes (`/server_info`, `/api/settings`, `/api/llm/models/verified`) raise no toast.- `F26.backend-only`: `--backend-only` serves the APIs (key-protected) and answers 503 `No backend configured for this route` for `/` and static assets.- `F26.cross-connect`: a frontend-only UI connects to a separate backend-only instance through **Add a backend**; the shell then shows that backend as Connected and all API traffic goes to it.- `F26.remote-backend`: from a normal Canvas, backend selector → **Add Backend** → **Agent-server** → **Remote** shows the self-hosting guidance and connects to another instance (stand-in for a self-hosted VM, `docs/SELF_HOSTING.md`).@@ -34,7 +34,7 @@ Source: `bin/agent-canvas.mjs`, `scripts/dev-with-automation.mjs`, `scripts/dev-- `F26.host-bind-lan`: `--host 0.0.0.0` warns, listens on the machine's LAN address, and serves an index.html that carries `__AGENT_CANVAS_AUTH_REQUIRED__` and neither `__AGENT_CANVAS_SESSION_API_KEY__` nor the key itself (the loopback launcher's page carries the key marker); the UI behaves like `--public` (Add a backend with Next disabled; Skip leads to the API-key screen).- `F26.host-bind-lan-optin`: the opt-in the launcher's warning recommends (`--allow-lan-session-key`) should restore key injection on a LAN bind. Known failure (reproduced 2026-10-08): the npm launcher ignores the flag and the page still carries only `__AGENT_CANVAS_AUTH_REQUIRED__`; [#17949](https://github.com/OpenHands/OpenHands/issues/17949).- `F26.host-bind-env`: `OH_BIND_HOST=0.0.0.0` (the environment form of `--host`) prints the same two "not loopback" warnings and does not inject the session key, even with `LOCAL_BACKEND_API_KEY` set.-- `F26.automation-startup-failure`: when the requested automation backend exits before it is ready, the launcher should print no success banner, stop the services it started and exit nonzero with the automation's exit reason. Known failure (reproduced 2026-10-13, #18362): it prints the **Main UI** banner and keeps the stack up with `/api/automation/health` answering `502`.+- `F26.automation-startup-failure`: the launcher announces the stack (the **Main UI** banner) without waiting for automation, then watches it: when the automation backend exits before it is healthy, the launcher prints `✗ The automation backend exited during startup (code <N>). Its [automation] log lines show why.`, stops every service it started and exits with code 1, leaving its ports closed. A slow start that only times out keeps the stack up.- `F26.mars-web-bridge`: every launcher-served web stack hosts the DigitalOcean Managed Agents bridge on its ingress by default (`MARS_WEB=0` opts out): `/mars/health` answers `{"ok":true,"authRequired":true}` on a full, `--backend-only`, `--frontend-only` and `--host 0.0.0.0` launcher, `/mars/rpc/*` answers `401` without the session key and works with it, the static child behind the ingress hosts none, and the Electron stack hosts none either (its preload provides `window.marsBridge`).- `F26.runtime-services`: `/server_info.runtime_services` lists the agent-server, ingress, frontend and automation URLs as the agent sees them, and a new conversation's system prompt carries a matching `<RUNTIME_SERVICES>` block.- `F26.runtime-services-agent-use`: with that block alone, the agent can reach the automation backend from its terminal: asked to, it reads the OpenAPI at the URL the block names, creates an enabled cron automation, and dispatches a run that completes; the automation and its run show up in the Automations UI.@@ -91,8 +91,8 @@ Preconditions:- **Generated session key persists (`F26.session-key-persist`).** With 18960–18963 free, define a helper and start a loopback stack without a key: `Q=$OH_VERIFY_RUN/private/f26-keys; mkdir -p $Q; start(){ HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 "$@" > $Q/launcher.log 2>&1 & echo $! > $Q/pid; for i in $(seq 1 60); do c=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/); [ "$c" = 200 ] && break; sleep 3; done; echo "code=$c"; }; start env -u LOCAL_BACKEND_API_KEY nohup node bin/agent-canvas.mjs --port 18960` (`code=200`). `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_SESSION_API_KEY__[^;<]*' | md5sum` prints a hash (never print the key itself); `$Q/home/.openhands/agent-canvas/api-key.txt` exists (64 hex characters) and `curl -s -o /dev/null -w '%{http_code}' -H "X-Session-API-Key: $(cat $Q/home/.openhands/agent-canvas/api-key.txt)" http://127.0.0.1:18960/api/settings` is `200`. `kill -TERM $(cat $Q/pid); sleep 6`, run the same `start ...` line again in the same shell and repeat the `md5sum` line: the hash is identical.- **Pinned session key (`F26.session-key-pinned`).** Same shell, after `kill -TERM $(cat $Q/pid); sleep 6`: `start env LOCAL_BACKEND_API_KEY=qa-f26-pin-key nohup node bin/agent-canvas.mjs --port 18960`. `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_SESSION_API_KEY__[^;<]*'` prints `__AGENT_CANVAS_SESSION_API_KEY__="qa-f26-pin-key"`; `/api/settings` is `200` with `-H 'X-Session-API-Key: qa-f26-pin-key'` and `401` with the saved generated key. In the browser, `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser eval "({key: window.__AGENT_CANVAS_SESSION_API_KEY__ === 'qa-f26-pin-key', authRequired: window.__AGENT_CANVAS_AUTH_REQUIRED__ === true})"` → `{key: true, authRequired: false}`.- **Bind host from the environment (`F26.host-bind-env`).** Same shell, after `kill -TERM $(cat $Q/pid); sleep 6`: `start env LOCAL_BACKEND_API_KEY=qa-f26-pin-key OH_BIND_HOST=0.0.0.0 nohup node bin/agent-canvas.mjs --port 18960`. `grep -a 'not loopback' $Q/launcher.log` shows the same `[auth]` and `[static]` warnings as `--host 0.0.0.0`, and `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u` prints only `__AGENT_CANVAS_AUTH_REQUIRED__`. Stop it with `kill -TERM $(cat $Q/pid)`; after about 6 s ports 18960–18963 are free.-- **Frontend-only, returning user (`F26.frontend-only-returning`).** Same shell, after `kill -TERM $(cat $Q/pid); sleep 6`. Arrange a browser that onboarded on this origin against the full launcher: `start env LOCAL_BACKEND_API_KEY=qa-f26-pin-key nohup node bin/agent-canvas.mjs --port 18960` (`code=200`), `control-openhands browser reset` (a fresh profile, which also forgets your run's onboarding: restore it at the end), `control-openhands browser goto http://127.0.0.1:18960/ --allow-external`, `control-openhands browser wait 'testid=telemetry-consent-form' --timeout 15000` (this backend state has never answered consent; a second pass on the same `$Q` skips the form), `control-openhands browser uncheck 'testid=telemetry-consent-form >> role=checkbox'`, `control-openhands browser click 'testid=confirm-telemetry-preferences'`, `control-openhands browser wait 'testid=onboarding-step-choose-agent' --timeout 15000`, `control-openhands browser click 'testid=onboarding-skip'` and `control-openhands browser wait 'testid=root-layout' --timeout 20000`. `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({id:b.id,name:b.name,host:b.host}))"` is `[{id: "default-local", name: "Local", host: "http://127.0.0.1:18960"}]`. Now serve the same port without backends: `kill -TERM $(cat $Q/pid); sleep 6; start nohup node bin/agent-canvas.mjs --frontend-only --port 18960` (`code=200`, and `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/server_info` is `503`). `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser wait 'testid=agent-server-onboarding-screen' --timeout 20000` succeed: the gate of `F25.recovery-gate`, not onboarding. `control-openhands browser count` is `1` for `testid=manage-backends-modal` and `0` for `testid=onboarding-step-check-backend`, `testid=first-run-onboarding-screen`, `testid=api-key-entry-screen`, `testid=root-layout`, `testid=close-manage-backends-modal` and `testid=manage-backends-done`; `control-openhands browser press Escape` leaves the modal (`count` `1`). `control-openhands browser text 'testid=manage-backends-row-Local'` reads `Local`, `http://127.0.0.1:18960`, `Disconnected`, `HTTP request failed (503 Service Unavailable): "Service Unavailable (no backend configured for this route)"` and `LOCAL`, and `control-openhands browser testids` lists `manage-backends-add` (**Add Backend**) as the way on. `control-openhands browser screenshot --feature F26.frontend-only-returning --name recovery-gate`. The 503 probes are expected errors in the sweep: `control-openhands browser errors` (without `--app-only`, which keeps only your run's origin) lists `external:http-error` rows with `503` for `http://127.0.0.1:18960/server_info`, `/api/settings` and `/api/llm/models/verified`, and `pageErrors` is `0`. Stop it with `kill -TERM $(cat $Q/pid)`; after about 6 s ports 18960–18963 are free. Restore your run: `control-openhands browser goto /` and `control-openhands onboard --skip`.-- **Automation exits during startup (`F26.automation-startup-failure`).** With 18960–18963 free, start a loopback launcher whose automation cannot install: `Q=$OH_VERIFY_RUN/private/f26-autofail; mkdir -p $Q; HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 OH_AUTOMATION_VERSION=0.0.1 LOCAL_BACKEND_API_KEY=qa-f26-autofail-key OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 nohup node bin/agent-canvas.mjs --port 18960 > $Q/launcher.log 2>&1 & echo $! > $Q/pid`, then poll the log until `Main UI` appears or the pid exits. Expected (#18362): no `Main UI` banner, a message naming the automation startup failure, a nonzero exit and closed ports. Known failure (reproduced 2026-10-13 twice, with `0.0.1` and `0.0.0-qa-missing`): the log shows `[automation] ... No solution found ...` and `[automation] Exited with code 1` (code 2 for the unparsable version), then `Main UI: http://localhost:18960/` two seconds later; `kill -0 $(cat $Q/pid)` succeeds, `curl http://127.0.0.1:18960/` is `200` and `/api/automation/health` (with the key) is `502`. Stop it with `kill -TERM $(cat $Q/pid)`; the ports free within seconds. The #18299 migration failure in the issue is the same path through a different exit cause.+- **Frontend-only, returning user (`F26.frontend-only-returning`).** Same shell, after `kill -TERM $(cat $Q/pid); sleep 6`. Arrange a browser that onboarded on this origin against the full launcher: `start env LOCAL_BACKEND_API_KEY=qa-f26-pin-key nohup node bin/agent-canvas.mjs --port 18960` (`code=200`), `control-openhands browser reset` (a fresh profile, which also forgets your run's onboarding: restore it at the end), `control-openhands browser goto http://127.0.0.1:18960/ --allow-external`, `control-openhands browser wait 'testid=telemetry-consent-form' --timeout 15000` (this backend state has never answered consent; a second pass on the same `$Q` skips the form), `control-openhands browser uncheck 'testid=telemetry-consent-form >> role=checkbox'`, `control-openhands browser click 'testid=confirm-telemetry-preferences'`, `control-openhands browser wait 'testid=onboarding-step-choose-agent' --timeout 15000`, `control-openhands browser click 'testid=onboarding-skip'` and `control-openhands browser wait 'testid=root-layout' --timeout 20000`. `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({id:b.id,name:b.name,host:b.host}))"` is `[{id: "default-local", name: "Local", host: "http://127.0.0.1:18960"}]`. Now serve the same port without backends: `kill -TERM $(cat $Q/pid); sleep 6; start nohup node bin/agent-canvas.mjs --frontend-only --port 18960` (`code=200`, and `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/server_info` is `503`). `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser wait 'testid=agent-server-onboarding-screen' --timeout 20000` succeed: the gate of `F25.recovery-gate`, not onboarding. `control-openhands browser count` is `1` for `testid=manage-backends-modal` and `0` for `testid=onboarding-step-check-backend`, `testid=first-run-onboarding-screen`, `testid=api-key-entry-screen`, `testid=root-layout`, `testid=close-manage-backends-modal` and `testid=manage-backends-done`; `control-openhands browser press Escape` leaves the modal (`count` `1`). `control-openhands browser text 'testid=manage-backends-row-Local'` reads `Local`, `http://127.0.0.1:18960`, `Disconnected`, `HTTP request failed (503 Service Unavailable): "Service Unavailable (no backend configured for this route)"` and `LOCAL`, and `control-openhands browser testids` lists `manage-backends-add` (**Add Backend**) as the way on. After about 15 s, `control-openhands browser toasts --history` is `[]` (the same at `browser viewport phone` after a reload). `control-openhands browser screenshot --feature F26.frontend-only-returning --name recovery-gate`. The 503 probes are expected errors in the sweep: `control-openhands browser errors` (without `--app-only`, which keeps only your run's origin) lists `external:http-error` rows with `503` for `http://127.0.0.1:18960/server_info`, `/api/settings` and `/api/llm/models/verified`, and `pageErrors` is `0`. Stop it with `kill -TERM $(cat $Q/pid)`; after about 6 s ports 18960–18963 are free. Restore your run: `control-openhands browser goto /` and `control-openhands onboard --skip`.+- **Automation exits during startup (`F26.automation-startup-failure`).** With 18960–18963 free, start a loopback launcher whose automation cannot install: `Q=$OH_VERIFY_RUN/private/f26-autofail; mkdir -p $Q; HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 OH_AUTOMATION_VERSION=0.0.1 LOCAL_BACKEND_API_KEY=qa-f26-autofail-key OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 nohup node bin/agent-canvas.mjs --port 18960 > $Q/launcher.log 2>&1 & echo $! > $Q/pid`, then poll the log until the pid exits (about 5 s after the automation failure). The log shows `[automation] ... No solution found ...` and `[automation] Exited with code 1`, the `Main UI: http://localhost:18960/` banner (printed before automation is healthy), then `✗ The automation backend exited during startup (code 1). Its [automation] log lines show why.` and `[agent-server] Stopping...`, `[static] Stopping...`, `[ingress] Stopping...`; `kill -0 $(cat $Q/pid)` fails and 18960–18963 are free. To read the exit code, run the same line in the foreground with `OH_AUTOMATION_VERSION=0.0.0-qa-missing`, `timeout 240` in place of `nohup` and no `&`, then `echo "exit=$?"`: automation exits with code 2, the message names `code 2` and `exit=1`. The partial-migration bullet below reaches the same path through a different exit cause.- **Runtime services (`F26.runtime-services`).** Run `control-openhands api GET /server_info --pick runtime_services.mode` (`agent-canvas`) and `control-openhands api GET /server_info --pick runtime_services.services.automation.url_from_agent` (`http://localhost:<automation port>`; `control-openhands status` prints the ports). Then `control-openhands conversation start --prompt "Reply with the single word: ok" --wait --timeout 180` (prints `<id>`) and `control-openhands conversation events <id> --grep RUNTIME_SERVICES --from-start`: the `SystemPromptEvent` matches, with an excerpt `<RUNTIME_SERVICES>\nYou are running inside an agent-canvas dev stack started ...`. `control-openhands conversation events <id> --grep "localhost:<automation port>" --from-start` shows `* Automation backend: http://localhost:<port>` plus its `Docs:` and `OpenAPI:` URLs.- **The agent uses the runtime services (`F26.runtime-services-agent-use`).** Write `qa-f26-agent.txt` (outside the workspace) containing: ``Using the Automation backend described in your RUNTIME_SERVICES (read its OpenAPI to find the routes), create one automation named QA_rt_auto from a prompt: the prompt is "Reply with the single word: pong. Do not run any tools.", the schedule is the cron 0 9 * * *, and it is enabled. Then dispatch one run of it now. Do not change anything else. Reply with the automation id and the run id only.`` Run `control-openhands conversation start --prompt "$(cat qa-f26-agent.txt)" --wait --timeout 400` (note `<agent-id>`); the reply names `<automation-id>` and a run id. `control-openhands conversation events <agent-id> --kinds ActionEvent --from-start` shows the agent's own `curl` calls: `http://localhost:<automation port>/api/automation/openapi.json`, `POST …/api/automation/v1/preset/prompt` and `POST …/api/automation/v1/<automation-id>/dispatch`, each with the `X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY` header the block names. Second view in the UI: `control-openhands browser goto /automations` and `control-openhands browser text '[data-testid^=automation-card-] >> has-text=QA_rt_auto'` (`QA_rt_auto`, the prompt, `cron`, `Runs 1`). Open it with `control-openhands browser click '[data-testid^=automation-card-] >> has-text=QA_rt_auto >> text=QA_rt_auto' --expect-url '/automations/[0-9a-f-]+'`. Within about a minute, `control-openhands browser text 'testid=automation-activity-log'` shows the run's summary (`Replied with the single word "pong" …`), a cost and `Successful` (`control-openhands browser screenshot --feature F26.runtime-services-agent-use --name agent-created-automation-run`). `control-openhands api GET /api/automation/v1/<automation-id>` has `name` `QA_rt_auto`, `trigger.schedule` `0 9 * * *` and `enabled` `true`. Delete it afterwards (`control-openhands api DELETE /api/automation/v1/<automation-id> --write`, arrange). It runs daily at 09:00 UTC otherwise.@@ -103,7 +103,7 @@ Preconditions:- **External links (`F26.desktop-external-links`).** Not driven: record `not-run` (harness gap: no verb clicks inside the Electron window or observes the OS browser handoff).- **KV survives authentication rotation (`F26.session-key-rotated`, `F26.session-key-persist`).** With no explicit `AUTOMATION_KV_SECRET`, create an owned inert automation and PUT a harmless marker to `/api/automation/v1/kv/qa-session-rotation?automation_id=<id>` through `api --write`; GET must return that marker. Run `restart --rotate-key` and `doctor`, then GET again using the helper's fresh private key. Expected: rotating authentication preserves encrypted automation state. Known failure, reproduced 2026-10-12 against automation 1.19.3: the pre-rotation read is 200 and the fresh-auth read is 500 `Failed to decrypt value` ([#18301](https://github.com/OpenHands/OpenHands/issues/18301)). This uses only disposable state and does not establish the original incident's data-loss history.-- **Partial SQLite migration startup (`F26.runtime-services`).** Stop only the owned automation service, make a cold backup of `<run>/private/automation/automations.db`, and confirm its `automation_git_sync_org_config` already has `consecutive_auth_failures`. In that disposable DB only, set `alembic_version.version_num` to `028`; restart with a bounded timeout, then open `/automations`. Expected: required-service startup fails explicitly or the partial schema recovers without corrupting state. Known failure, reproduced 2026-10-12 against automation 1.19.3: migration 029 reports the duplicate column and exits 3, the launcher remains alive, and Canvas shows **Automations Unavailable** ([#18299](https://github.com/OpenHands/OpenHands/issues/18299)). Restore the cold backup, restart and doctor before continuing. Never apply this fixture to an operator database.+- **Partial SQLite migration startup (`F26.runtime-services`).** Stop only the owned automation service, make a cold backup of `<run>/private/automation/automations.db`, and confirm its `automation_git_sync_org_config` already has `consecutive_auth_failures`. In that disposable DB only, set `alembic_version.version_num` to `028`; restart with a bounded timeout (`timeout 200 control-openhands restart`). Expected: required-service startup fails explicitly or the partial schema recovers without corrupting state. Observed 2026-10-17 against automation 1.20.1: migration 029 reports the duplicate column and exits 3, then the launcher prints `✗ The automation backend exited during startup (code 3). ...`, stops every service and exits (`F26.automation-startup-failure`); `restart` answers `The launcher exited before the stack became ready.` and `status` shows `launcherAlive` `false`. The schema itself is not repaired; that half stays open in [#18299](https://github.com/OpenHands/OpenHands/issues/18299). Restore the cold backup, restart and doctor before continuing. Never apply this fixture to an operator database.- **Occupied internal frontend port (`F26.cli-port-in-use`).** Reserve a loopback frontend port in an owned listener and run the genuine launcher with separate free ingress/backend/automation ports. The reported default-port branch can also be observed read-only when port 3001 is already occupied; do not stop its owner. Expected: an occupied internal frontend port receives a free fallback while explicit service-port conflicts keep their existing semantics. Known failure, reproduced 2026-10-12: `agent-canvas --port 18940` exits 1 with `frontend: port 3001`, despite the ingress override ([#17546](https://github.com/OpenHands/OpenHands/issues/17546), [#16226](https://github.com/OpenHands/OpenHands/issues/16226)). An explicit `OH_CANVAS_SAFE_VITE_PORT` is a current workaround; it does not provide automatic fallback.- **Required frontend death (`F26.runtime-services`).** After a healthy private launch, identify only its required static-server PID using launcher ancestry plus its owned frontend port, then send that child SIGKILL. Open `/` and check the owned launcher's liveness and authenticated `/api/settings`. Expected: an unexpected required-service death stops the stack and reports failure. Known failure, reproduced 2026-10-12: the launcher and backend remain alive, settings returns 200, and the main UI returns 502 **Bad Gateway** ([#16927](https://github.com/OpenHands/OpenHands/issues/16927)). This live trigger is SIGKILL, not a port-race EADDRINUSE; the failing required-child exit branch is shared. Restart and doctor to restore the run.- **Static-server credential argv (`F26.runtime-services`).** Inspect only this run's owned static-server process for the frontend port. Read the private key in memory and retain booleans indicating whether `--session-api-key` is present and whether that literal key occurs in argv; never print argv or the key. Expected: other host users cannot recover the session key from command-line arguments. Known failure, reproduced 2026-10-12: both booleans are true for the owned static frontend ([#16383](https://github.com/OpenHands/OpenHands/issues/16383)). Docker remains a separate unavailable prerequisite here.@@ -117,7 +117,7 @@ Preconditions:- The browser daemon refuses other origins unless you pass `--allow-external` to `browser goto`. Each port is its own origin, so a partial-stack page starts at first run again (onboarding and a consent dialog), independent of your run's onboarding.- In a frontend-only UI the **Add a backend** step's Next is enabled with an empty key, while on a LAN or `--public` stack it is disabled until a key is typed: the host the step probes differs (503 vs. auth-required).- A partial-stack origin keeps its stored backend in the browser profile across launches, so a second pass lands straight in the shell. `control-openhands browser reset` brings back the first run on every origin; then `control-openhands browser goto /` and `control-openhands onboard --skip` restore your own run's baseline.-- Which screen a frontend-only origin shows depends on the browser, not the launcher: a profile new to the origin gets **Add a backend** (first run owns the initial backend), while a profile that stores a backend for that origin (the `default-local` entry a full launcher on the same port seeded, or any backend added there earlier) gets the recovery gate of `F25.recovery-gate` as soon as that backend answers 503 (`F26.frontend-only-returning`). The no-toast check belongs to the first-run state only: on the gate, `browser toasts --history` lists generic `An error occurred` status toasts raised by the failing 503 probes (two within 12 s here), with no hint of which request failed. Known failure (reproduced 2026-10-08): both toasts come from the free-models hydrator's `GET /api/llm/models/verified` 503, which no `meta.disableToast` guards on that screen (nor does `backend-version.tsx`); [#18160](https://github.com/OpenHands/OpenHands/issues/18160).+- Which screen a frontend-only origin shows depends on the browser, not the launcher: a profile new to the origin gets **Add a backend** (first run owns the initial backend), while a profile that stores a backend for that origin (the `default-local` entry a full launcher on the same port seeded, or any backend added there earlier) gets the recovery gate of `F25.recovery-gate` as soon as that backend answers 503 (`F26.frontend-only-returning`). Both states are toast-free: the gate's failing 503 probes (the free-models hydrator's `GET /api/llm/models/verified` and the backend version read included) are silenced, and the row's red detail carries the 503 text.- Stopping a backend-only instance while the frontend-only page that uses it stays open is the same backend-down state as `F25.recovery-gate` after a reload; keep the returning-user bullet above as the frontend-only-specific check (it reproduces the `default-local` case).- The launcher serves the bundled VS Code editor. #17660 made that opt-in (`OH_CANVAS_ENABLE_VSCODE=true`) and #18048 reverted it, so only a checkout between the two needs the variable (`launch --vscode` sets it). A launcher that serves it reserves `/vscode/`: a frontend-only stack answers `503` there and logs `[static] /vscode -> 503 (rejected)`; a stack with an agent-server lists `/vscode → http://127.0.0.1:<agent-server port + 1000>` in its ingress banner and hands the agent-server that editor port (19961 for the hand-started launchers here, 19000 for the desktop app), which the launcher's own port check does not cover. On a checkout between the two without the variable, `/vscode/` is an ordinary SPA URL (`200`). Here the agent-server has no editor binary (`VSCode server binary not found, VSCode will be disabled` in the launcher log, `/api/vscode/status` `{"running":false,"enabled":true}`), so the editor port stays free even then.- The port-in-use error is printed twice, the second time with a Node stack trace, and suggests `PORT=<other>` rather than `--port` ([#17949](https://github.com/OpenHands/OpenHands/issues/17949)). The `--public` key error's example says `npm run dev -- --public` instead of the npx command.@@ -130,11 +130,12 @@ Preconditions:- Electron's splash shows each raw service-log line (`agent-server: {"asctime": ...`) as its status text during boot; the named phases only appear between them.- `build:lib` from a copy whose `node_modules` is a symlink emits bundled dependencies under `dist/<absolute node_modules path>/...`; that path is an artifact of the copy, not of the product.- **Bug candidate (`F26.host-bind-lan-optin`):** `scripts/bind-host.mjs` tells npm users to pass `--allow-lan-session-key`, but neither `bin/agent-canvas.mjs` nor `scripts/dev-with-automation.mjs` parses or forwards it (only `scripts/static-server.mjs` and the Docker entrypoint do), and unknown flags are silently ignored ([#17949](https://github.com/OpenHands/OpenHands/issues/17949), whose first item is exactly this; reproduced again on 2026-10-08 at d2c89252d).-- Known issue [#17946](https://github.com/OpenHands/OpenHands/issues/17946): the npm launcher never creates its `TMUX_TMPDIR`, so hand-started instances share one tmux server and a restart of one resets the others' terminals.+- Each launcher instance gets its own tmux server: the npm launcher (and `npm run dev:static`) creates the resolved `TMUX_TMPDIR` before starting the Agent Server, `<state>/tmux` by default or an explicit `TMUX_TMPDIR` it preserves (then no `<state>/tmux` is made), and nothing falls back to the shared `/tmp/tmux-<uid>`. When the default path is too long for a Unix socket (the deep `$OH_VERIFY_RUN/private/...` states here), the Agent Server logs `TMUX_TMPDIR <path> is too long for tmux socket paths; using per-server tmux directory /tmp/openhands-agent-server-<pid> instead`, still one per instance; `control-openhands launch` passes a short `/tmp/ohv-tmux-<hash>` for the same reason. Check with `tr '\0' '\n' < /proc/<agent-server pid>/environ | grep TMUX_TMPDIR` and `ls -d <dir>` (2026-10-17: two hand-started launchers, one default and one with `TMUX_TMPDIR=/tmp/qa-f26-tmux-<n>`, each had its own directory).- Known issue [#17763](https://github.com/OpenHands/OpenHands/issues/17763) (`F26.docker-image`): The Docker entrypoint can leave protected Agent Server APIs unauthenticated when only `LOCAL_BACKEND_API_KEY` is supplied, because its export to `OH_SESSION_API_KEYS_0` is inside the generate-key branch. Recheck an owned container with a dummy custom key: unauthenticated `GET /api/conversations/search` must be 401 and the same request with that dummy key 200. `/server_info` is intentionally public and is not an auth probe. Blocked here without a Docker daemon.- Known issue [#17623](https://github.com/OpenHands/OpenHands/issues/17623) (`F26.helm-chart`): The chart defaults service-account-token automounting to true even with RBAC disabled, exposing an unused cluster credential to agent processes. Recheck a disposable Kubernetes installation using the default values and an explicit opt-in; inspect token existence/readability only, never its contents. Blocked without Helm and a Kubernetes cluster.- Known issue [#17470](https://github.com/OpenHands/OpenHands/issues/17470) (`F26.docker-image`): On Apple Silicon under Colima/Virtualization.framework, the image's Python services can exit with SIGILL while the frontend leaves onboarding at Disconnected. The report isolates `import litellm` and compares `OPENSSL_armcap=0`; recheck on the affected ARM/Colima guest with the reported cryptography/OpenSSL combination. A native macOS npm stack is not that container environment.- Known issue [#17112](https://github.com/OpenHands/OpenHands/issues/17112) (`F26.desktop-main-window` `F26.desktop-external-links`): Desktop main-window navigation, an OAuth popup redirect, or a nested popup can render a remote page inside the app frame. Recheck all three on an owned running Electron shell with a harmless remote page and loopback redirect fixture; the browser harness cannot operate Electron windows and the installed Electron binary is unavailable here.+- Known issue [#18547](https://github.com/OpenHands/OpenHands/issues/18547) (`F26.desktop-main-window`): on Windows the desktop title bar and menu bar stay dark when the Light+ theme is selected (`electron/main.mjs` keeps `nativeTheme.themeSource` on `dark`). Live prerequisite: the desktop app on Windows.- Known issue [#16872](https://github.com/OpenHands/OpenHands/issues/16872) (`F26.desktop-external-links`): Non-browser URI schemes were reported to reach the OS handler from the desktop popup flow. Current source now parses hosts and restricts external protocols; recheck refusal in a running owned Electron shell before concluding resolution. Observe an inert denied test scheme without invoking any real OS handler. The browser harness cannot drive Electron and no installed Electron binary is available here.- Known issue [#16774](https://github.com/OpenHands/OpenHands/issues/16774) (`F26.desktop-boot-splash` `F26.desktop-main-window`): The macOS 1.14.0 DMG installs an app that Gatekeeper reports as damaged instead of booting. Recheck the actual downloaded signed/notarized release artifact through install and first launch on macOS; starting source Electron or removing quarantine would bypass the reported failure. That release DMG/signature fixture is unavailable in this run.- Known issue [#16720](https://github.com/OpenHands/OpenHands/issues/16720) (`F26.runtime-services`): CPU-heavy commands in several conversations can starve Agent Server and stop the UI loading; the report requests lower priority for agent commands than the service. Recheck with a disposable bounded workload, recorded CPU/resource limits and real command observations, never arbitrary stress on the shared machine. The exact Rust workload and affected resource envelope are missing, and real agent execution needs a model key. The backend counterpart is [OpenHands/software-agent-sdk#4575](https://github.com/OpenHands/software-agent-sdk/issues/4575).diff --git a/.agents/skills/verify-openhands/references/feature-map/F27-workspace-tools.md b/.agents/skills/verify-openhands/references/feature-map/F27-workspace-tools.mdindex 9c62030a9c..27b576aebd 100644--- a/.agents/skills/verify-openhands/references/feature-map/F27-workspace-tools.md+++ b/.agents/skills/verify-openhands/references/feature-map/F27-workspace-tools.md@@ -23,7 +23,7 @@ Source: `src/components/features/terminal/`, `src/hooks/use-terminal.ts`, `src/c- `F27.browser-screenshot`: after the agent's browser tool ran, the tab shows the last screenshot under a bar with its URL and an enabled "Open in New Tab" link.- `F27.planner-empty`: with no plan the Planner reads "There is currently no plan for this repo" with an enabled "Create a plan" button; the drawer's "Build ⌘↩" button is disabled.- `F27.planner-create`: "Create a plan" toasts "Planning agent initialized", switches the composer to plan mode ("Let’s work on a plan") and disables itself, also after a reload.-- `F27.planner-plan`: when the workspace has `.agents_tmp/PLAN.md`, the Planner renders it as Markdown with an enabled "Build ⌘↩" bar.+- `F27.planner-plan`: when the workspace has `.agents_tmp/PLAN.md`, the Planner renders it as Markdown (links underlined in the theme's link colour) with an enabled "Build ⌘↩" bar.- `F27.planner-build`: "Build ⌘↩" sends "Execute the plan based on the .agents_tmp/PLAN.md file." to the agent.- `F27.tasklist`: the Task List tab appears only once the agent used `task_tracker`; it lists each task with its status icon and highlights the in-progress one; the drawer's tabs menu opens it too.- `F27.usage-metrics`: the Usage tab shows the context meter ("X% used (Y% left)", tokens / window), Token Usage (Input, Cache Hit, Cache Write, Output, Total) and Total Cost. Above 70% fill, the meter label and bar use the warning feedback role; above 90%, its label uses error feedback and its bar uses status error. Text reaches 4.5:1 against the raised Usage surface and fills reach 3:1 against the track in all five themes.@@ -76,7 +76,7 @@ Preconditions:- **Terminal after reload (`F27.terminal-history-reload`, `F27.terminal-live-append`).** Before clicking any other tab in `<tools-id>` (a user tab click is persisted and would mask the next check), run `control-openhands browser reload`, `control-openhands browser click 'testid=right-panel-toggle'`, `control-openhands browser text 'testid=tabs-pane-header'` (read now for `F27.canvas-ui-open-tab` below; it starts with `Terminal`, so the drawer is already on Terminal: do not click `conversation-tab-terminal`, because clicking the active tab closes the drawer) and `control-openhands browser text '.xterm-rows'`. Expected: the earlier commands are still listed. Known failure ([#17566](https://github.com/OpenHands/OpenHands/issues/17566) reproduces, 2026-10-06): the text is empty and the tab reads "No terminal output yet" while the chat still lists the actions. Live append is checked after the Build step below.- **Agent tab survives reload (`F27.canvas-ui-open-tab`).** The `tabs-pane-header` text read in the reload step above (right after `right-panel-toggle`) starts with `Terminal`, the agent's last choice: `src/services/canvas-ui.ts` saves the tab per conversation like a user click (#18045), and `control-openhands browser eval "JSON.parse(localStorage.getItem('conversation-state-<tools-id>')).selectedTab"` is `terminal` (`browser screenshot --feature F27.canvas-ui-open-tab --name reload-reopens-terminal`). For contrast, `control-openhands browser click 'testid=conversation-tab-usage'`, `control-openhands browser reload`, `control-openhands browser click 'testid=right-panel-toggle'`: a user-clicked tab is restored (`Usage`).- **Task List (`F27.tasklist`).** In `<tools-id>` run `control-openhands browser click 'testid=conversation-tab-tasklist'`, `control-openhands browser wait '[data-active="true"] >> has-text=qa-two' --timeout 10000` (the rows render a moment after the click; an immediate count reads `0`), `control-openhands browser count '[data-active="true"] >> has-text=qa-two'` (`1`) and `control-openhands browser count '[data-active="false"] >> has-text=qa-three'` (`1`); screenshot with `control-openhands browser screenshot --feature F27.tasklist --name tasks` (done/in-progress/todo icons). Menu entry: `control-openhands browser click 'testid=tabs-pane-header >> testid=ellipsis-button'` then `control-openhands browser click 'testid=conversation-tabs-menu-open-tasklist'`; the header starts with `Task List`.-- **Plan and Build (`F27.planner-plan`, `F27.planner-build`).** In `<tools-id>` run `control-openhands browser click 'testid=conversation-tab-planner'` and `control-openhands browser snapshot 'testid=app-route'`: it ends with `heading "QA Plan" [level=1]` and a list `alpha`, `beta`, `gamma`; `control-openhands browser enabled 'testid=planner-tab-build-button'` is `true`. Run `control-openhands browser click 'testid=planner-tab-build-button'`, then `control-openhands browser count 'testid=user-message >> has-text=Execute the plan based on the .agents_tmp/PLAN.md file.'` (`1`) and `control-openhands conversation wait <tools-id> --timeout 240`; the agent reads PLAN.md and answers. Live append (`F27.terminal-live-append`): `control-openhands browser click 'testid=conversation-tab-terminal'` and `control-openhands browser text '.xterm-rows'` now show the new `$ cat .agents_tmp/PLAN.md` block and its output (but not the pre-reload history).+- **Plan and Build (`F27.planner-plan`, `F27.planner-build`).** In `<tools-id>` run `control-openhands browser click 'testid=conversation-tab-planner'` and `control-openhands browser snapshot 'testid=app-route'`: it ends with `heading "QA Plan" [level=1]` and a list `alpha`, `beta`, `gamma`; `control-openhands browser enabled 'testid=planner-tab-build-button'` is `true`. Links in the plan stay readable: append `See [QA docs](https://example.com/qa) for the verification page.` to the workspace's `.agents_tmp/PLAN.md` (arrange), reload and reopen Planner; the link (`[role=tabpanel] >> a[href="https://example.com/qa"]`) is underlined before hover, `rgb(0, 90, 153)` at 6.48:1 on the Light+ panel and `rgb(0, 90, 136)` at 6.08:1 on Solarized Light (F16.color-theme; `browser screenshot 'testid=app-route' --feature F27.planner-plan --name plan-link`), and the phone panel page shows it too. Run `control-openhands browser click 'testid=planner-tab-build-button'`, then `control-openhands browser count 'testid=user-message >> has-text=Execute the plan based on the .agents_tmp/PLAN.md file.'` (`1`) and `control-openhands conversation wait <tools-id> --timeout 240`; the agent reads PLAN.md and answers. Live append (`F27.terminal-live-append`): `control-openhands browser click 'testid=conversation-tab-terminal'` and `control-openhands browser text '.xterm-rows'` now show the new `$ cat .agents_tmp/PLAN.md` block and its output (but not the pre-reload history).- **Create a plan (`F27.planner-create`).** In `<quiet-id>` (no plan yet) run `control-openhands browser goto /conversations/<quiet-id>` and `control-openhands browser click 'testid=right-panel-toggle'` (the browser is on `<tools-id>`, and a fresh load has the drawer closed; it reopens on the tab last chosen there, Usage after the context-ring bullet (its Usage item and bar save `usage`): if that is already Planner, skip the next click, which would close the drawer), then `control-openhands browser click 'testid=conversation-tab-planner'`, `control-openhands browser click 'role=button[name="Create a plan"]'`, then `control-openhands browser toasts` (`Planning agent initialized`) and `control-openhands browser enabled 'role=button[name="Create a plan"]'` (`false`). The composer placeholder is `Let’s work on a plan` (`browser screenshot --feature F27.planner-create --name after-create`). After `control-openhands browser reload` and reopening Planner the button stays disabled, and `control-openhands api GET /api/conversations/<quiet-id>` lists one `sub_conversation_ids` entry (the helper; it is not in the sidebar list).- **Open a file from the agent (`F27.canvas-ui-navigate-file`).** On `/conversations/<tools-id>` (`control-openhands browser goto /conversations/<tools-id>`; Create a plan left `<quiet-id>`, and `--stay` sends into the open page) run `control-openhands conversation start --stay --prompt "Call the canvas_ui_control tool with command navigate_to_file and path .agents_tmp/PLAN.md, then reply OK." --wait --timeout 200`, then `control-openhands browser count 'testid=file-quick-row-item-.agents_tmp/PLAN.md'` (`1`) and `control-openhands browser text 'testid=files-tab-content'` (`Rich`, `Plain`, `QA Plan`, `alpha`…). Chat link entry: `control-openhands browser click 'testid=right-panel-toggle'` (closes the drawer), `control-openhands browser click 'testid=markdown-file-path-link >> nth=0'`, then `control-openhands browser attr 'testid=right-panel-toggle' aria-pressed` is `true` with the same file selected.- **Launch a child (`F27.launch-child`).** On `/conversations/<tools-id>` (code mode, not a plan-mode conversation) run `control-openhands conversation start --stay --prompt "Call the launch_child_conversation tool with target local, isolation shared and task 'Reply with only the word PONG. Do not run any tools.' Then reply OK."` and poll `control-openhands browser toasts` every few seconds: one toast reads `Launched a local child conversation Open`. Take the child id from `control-openhands conversation events <tools-id> --kinds MessageEvent` (the `[child-conversation] {"status":"launched",…,"conversation_id":"<child-id>"…}` message), then `control-openhands conversation wait <child-id> --timeout 120` (`finished`) and `control-openhands conversation events <child-id> --kinds MessageEvent` (agent `PONG`). `control-openhands browser count 'testid=conversation-card >> has-text=PONG'` is `1`.@@ -104,7 +104,7 @@ Preconditions:- The Overview peek appears only while the drawer is open (`canPeekOnHover` needs `isRightPanelShown`); a `browser goto` or reload starts with the drawer closed.- The Overview's repo and branch rows depend on a bash probe (`useLocalGitInfo`) that runs only while the conversation's status is idle, running, waiting for confirmation or finished and the agent is not in error. Right after `conversation start` the probe has run and its result stays cached on the page, but on an errored conversation a reload hides the rows again, and `conversation pause` does not apply to an errored conversation. Read the rows before any `goto` or `reload`.- At phone width (and in a narrow drawer) the tab row shows only the tabs that fit; the rest are reachable only through the tabs menu (`conversation-tabs-menu-open-<tab>`), and which tabs fit depends on the active tab's label width.-- Plain `agent-canvas` instances on one machine share the tmux socket `/tmp/tmux-<uid>/openhands` ([OpenHands/OpenHands#17946](https://github.com/OpenHands/OpenHands/issues/17946)), so another instance stopping can reset this one's terminal. `control-openhands launch` gives each run its own `TMUX_TMPDIR`, so in verification runs a reset means this run's agent server restarted.+- Each instance's agent server keeps its terminal sessions' tmux sockets (one `openhands-<random hex>` per terminal session) in its own `TMUX_TMPDIR` (`launch`: `/tmp/ohv-tmux-<hash>`; a plain `agent-canvas`: `<state>/tmux`, which the launcher creates), so another instance stopping cannot reset this one's terminal. After this run's own `restart`, `echo QA_TERM_MARK_2` ran once with its output and no reset (2026-10-17); a reset row is a finding.- Agent Tools & Metadata remembers the last tab (it reopens on Available Tools).- The Overview menu opens only Pull Requests (`conversation-overview-menu-open-git-pull_requests`); its testids are `conversation-overview-menu-open-*` and `-pin-*` for `workspace` and the Git rows only.- Known open issues: terminal history after reload ([#17566](https://github.com/OpenHands/OpenHands/issues/17566)) still fails (reproduced 2026-10-06); info modals at phone width (#17562) pass.@@ -119,5 +119,4 @@ Preconditions:- Known issue [#17383](https://github.com/OpenHands/OpenHands/issues/17383) (`F27.browser-screenshot`; dependency counterpart [OpenHands/software-agent-sdk#5152](https://github.com/OpenHands/software-agent-sdk/issues/5152)): unlocked dependency resolution can silently remove the agent's browser tools. The actual 2026-10-12 pinned runtime resolves browser-use 0.11.13, mcp 1.30.0 and fastmcp 3.4.8; its genuine BrowserToolExecutor constructs successfully. That narrowly fails to reproduce the SDK report's specified mcp 2.x import failure. Fresh constructor recheck on 2026-10-16 at `b34ac59`, using the actual Agent Server 1.55.0 interpreter and private HOME, again resolves browser-use 0.11.13, mcp 1.30.0 and fastmcp 3.4.8 and constructs successfully; no browser operation or model was run. The Canvas/global browser-operation path still needs an authorized real LLM and a genuine browser-enabled agent issuing a `browser_*` operation; construction alone does not establish that feature works.- Browser availability also depends on [OpenHands/software-agent-sdk#5899](https://github.com/OpenHands/software-agent-sdk/issues/5899) (`F27.browser-screenshot`; canonical tool-selection note in F13): a Chromium installed only under `PLAYWRIGHT_BROWSERS_PATH` is missed by the agent-tool probe. Recheck the real browser-tool operation on a host with only that custom cache and no standard browser, default cache or PATH executable. This Mac's user Chrome must remain installed; an executor constructor or the verification browser is not a successful agent browser operation.- Known issue [OpenHands/software-agent-sdk#5835](https://github.com/OpenHands/software-agent-sdk/issues/5835) (`F27.overview-identity`): the local Overview Git probe uses conversation-scoped Bash execution; an idle-eviction pass can kill scoped work without a final output. Recheck a disposable short-idle-TTL server while a genuine Canvas Git probe is still executing, with independently observed command lifetime and the eventual output/result. The report's standalone asynchronous `sleep` reproduction is supporting backend evidence, not proof of a Terminal-tab command, which uses the agent event stream instead.-- Known issue [#18522](https://github.com/OpenHands/OpenHands/issues/18522) (`F27.planner-plan`): a Markdown link in `.agents_tmp/PLAN.md` renders in the Planner at 3.39:1 in Light+ and 3.07:1 in Solarized Light, below 4.5:1.- Known issue [#18546](https://github.com/OpenHands/OpenHands/issues/18546) (`F27.usage-metrics`): in Solarized Light the Usage panel's primary text measures 3.68:1 and its muted text (percentage, token count, Cache rows) 2.61:1 on the raised panel surface, below 4.5:1; the other four themes pass.diff --git a/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.mdindex 3ecc89cfe5..67d95733cc 100644--- a/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md+++ b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md@@ -24,7 +24,7 @@ Source: `src/routes/automation-setup-new.tsx`, `src/components/features/automati- `F28.open-blank`: Create Automation opens `/automations/setup` with Name pre-filled **New Automation** (placeholder `Standup digest`), an empty Prompt, Schedule selected with Daily at `09:00` in `America/New_York` (the default, whatever the browser's time zone), the docked composer (`Need help creating an automation?`) and no new conversation. The save state already reads **Unsaved changes** and the toolbar **Test** is disabled.- `F28.kind-toggle`: the button beside the Prompt label switches between Prompt and **Custom** (Custom Python). Custom shows a generated Python script that calls `AUTOMATION_CALLBACK_URL`, and **Entrypoint and setup** reveals Entrypoint `python3 main.py`, Setup script path `setup.sh` and a no-op setup script; switching back shows the Prompt again.-- `F28.schedule-presets`: in create mode the Frequency control (the same control as on the edit page, whose presets and saving F23 owns as edit-schedule and edit-cron) starts on Daily at `09:00`, and switching presets fills create-mode defaults: Once a date-time at the next whole hour today, Weekly Monday, and Custom the cron of the preset it replaces (from Weekly, Monday at `10:30` it reads `30 10 * * 1`), while a cron already typed in Custom stays. Known failure (reproduced 2026-10-15, [#18461](https://github.com/OpenHands/OpenHands/issues/18461)): Custom shows the last custom cron, `0 9 * * *` until one is typed, whatever the preset; from Weekly on Monday it reads `0 9 * * *`. Every preset keeps the 13-entry time zone select.+- `F28.schedule-presets`: in create mode the Frequency control (the same control as on the edit page, whose presets and saving F23 owns as edit-schedule and edit-cron) starts on Daily at `09:00`, and switching presets fills create-mode defaults: Once a date-time at the next whole hour today, Weekly Monday, and Custom the cron of the preset it replaces (from Weekly, Monday at `10:30` it reads `30 10 * * 1`; Weekdays `30 10 * * 1-5`, Hourly `0 * * * *`, Once on 17 October at `09:00` `0 9 17 10 *`). Clicking Custom again keeps a cron already typed there; leaving Custom for a preset and coming back starts from that preset again. Every preset keeps the 13-entry time zone select.- `F28.event-trigger`: choosing **Event** replaces the schedule with Event source (default **GitHub**, sent as `github`; its menu lists the webhook sources registered on this backend plus **Custom**), Event key (default `issue_comment.created`), **Add Filter** (one free-text filter input), **Ask Agent**, the notice `Event sources need a public URL to deliver webhooks to this instance.`, and a **Test event payload** pre-filled with a JSON object for that source and key. A source the backend has not registered (menu **Custom**, then a typed name) adds **Register this as a custom webhook source**, checked by default.- `F28.timeout`: **Add Timeout** adds an optional **Timeout seconds** number field with a remove button; `1800` saves and the draft reads **Ready to test**.- `F28.draft-save`: **Save draft** works on any form, valid or not; the first save replaces the URL with `?draftId=<id>`, shows the **Draft saved** bar (relative time, **View test runs**, **Test**) and the toast `Draft saved in this conversation.`, and the save state reads **Saved just now**. The draft appears in the dashboard Drafts group (F21).@@ -34,7 +34,7 @@ Source: `src/routes/automation-setup-new.tsx`, `src/components/features/automati- `F28.test-gated`: in create mode **Test** stays disabled until the draft is saved and unchanged since; editing any field after a save disables it again until the next **Save draft**.- `F28.draft-test`: **Test** on a saved, valid draft opens **Test runs** (with **Back**), shows `Test run started.` and a Pending row; it runs a hidden disabled copy of the draft, not a dashboard automation. Expected: the row reaches Successful or Failed by itself. Known failure #18263 (reproduced again 2026-10-15 on a Custom Python draft): the row stays Pending and the page does not poll the runs after the first request.- `F28.create`: **Create Automation** creates the automation enabled, navigates to `/automations/<id>` (a Custom Python automation shows **Script**), and removes the draft from the dashboard Drafts group.-- `F28.error-contrast`: the draft bar's error pill and the Test runs error alert use the theme's error color (`text-feedback-error`) and reach at least 4.5:1 in all five color themes. The pill uses a 5% red tint over the draft bar surface.+- `F28.error-contrast`: the draft bar's error pill and the Test runs error alert use the theme's error color (`text-feedback-error`) and reach at least 4.5:1 in all five color themes. Their tints follow the same theme error color: the pill a 5% tint over the draft bar surface, the alert a 10% tint with a 40% border.## How to get to it (user POV)@@ -50,10 +50,10 @@ Preconditions:- Baseline state (`launch`, `doctor`, `onboard --skip`). No LLM is needed: every bullet uses the Custom Python kind or never dispatches, and the agent-assisted path stays in F22.- Desktop viewport; the `F28.error-contrast` bullet switches color themes (Settings → Application, `F16`) and restores the default.-- Every recipe below was run as written on 2026-10-16 on a fresh local `control-openhands launch` of main with Agent Server 1.55.1 and no LLM. Documented known failures remain failures.+- Every recipe below was run as written on 2026-10-16 on a fresh local `control-openhands launch` of main with Agent Server 1.55.1 and no LLM; the blank-form, presets and error-contrast recipes again on 2026-10-17. Documented known failures remain failures.- **Blank form (`F28.open-blank`).** Run `control-openhands conversation list`, `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup$'`. Then `browser value 'testid=automation-setup-name'` is `New Automation`, `browser attr 'testid=automation-setup-name' placeholder` is `Standup digest`, `browser value 'testid=automation-setup-prompt'` is empty, `browser attr 'testid=automation-setup-frequency-daily' aria-checked` is `true`, `browser value 'testid=automation-setup-time'` is `09:00`, `browser text 'testid=automation-setup-save-state'` is `Unsaved changes` and `browser enabled 'testid=automation-setup-test'` is `false`. `conversation list` is unchanged.-- **Presets (`F28.schedule-presets`).** For each of `once`, `hourly`, `daily`, `weekdays`, `weekly`, `custom` run `control-openhands browser click 'testid=automation-setup-frequency-<preset>'` and `control-openhands browser testids 'testid=automation-setup-at-row'` (`browser snapshot` is an ARIA tree and does not list test ids): Once has `automation-setup-datetime`, Hourly only `automation-setup-timezone`, Daily and Weekdays `automation-setup-time`, Weekly also `automation-setup-weekday` (value `1`), Custom `automation-setup-custom-schedule`. Click Weekly, then Custom: expected `browser value 'testid=automation-setup-custom-schedule'` is Weekly's `0 9 * * 1`. Known failure (reproduced 2026-10-15, [#18461](https://github.com/OpenHands/OpenHands/issues/18461)): it reads `0 9 * * *`. `browser count 'testid=automation-setup-timezone >> option'` is `13`. Click Daily again.+- **Presets (`F28.schedule-presets`).** For each of `once`, `hourly`, `daily`, `weekdays`, `weekly`, `custom` run `control-openhands browser click 'testid=automation-setup-frequency-<preset>'` and `control-openhands browser testids 'testid=automation-setup-at-row'` (`browser snapshot` is an ARIA tree and does not list test ids): Once has `automation-setup-datetime`, Hourly only `automation-setup-timezone`, Daily and Weekdays `automation-setup-time`, Weekly also `automation-setup-weekday` (value `1`), Custom `automation-setup-custom-schedule`. Click Weekly, then Custom: `browser value 'testid=automation-setup-custom-schedule'` is Weekly's `0 9 * * 1`. Click Weekly, `browser fill 'testid=automation-setup-time' 10:30`, click Custom: `30 10 * * 1`. Fill the Custom field with `15 8 * * 2` and click Custom again: it stays `15 8 * * 2`. Click Daily, then Custom: `30 10 * * *` (the preset's, not the typed cron). `browser count 'testid=automation-setup-timezone >> option'` is `13`. Click Daily again.- **Event fields (`F28.event-trigger`).** Run `control-openhands browser click 'role=radio[name=/^Event/]'`. `browser value 'testid=automation-setup-event-key'` is `issue_comment.created`, `browser visible 'testid=automation-setup-event-public-url-notice'` is `true`, and `browser count 'testid=automation-setup-custom-webhook-enabled'` is `0` for `github`. Click `testid=automation-setup-event-source-toggle`, `browser snapshot 'testid=automation-setup-event-source-menu'` lists the registered sources and `Custom` (GitHub, GitLab, Slack, Linear, Jira on a local stack). Click `testid=automation-setup-event-source-menu >> role=button[name="Custom"]`, `browser fill 'testid=automation-setup-event-source' qa-hooks`: `browser visible 'testid=automation-setup-custom-webhook-enabled'` is `true` and `browser eval "document.querySelector('[data-testid=automation-setup-custom-webhook-enabled]').checked"` is `true`. Pick GitHub again from the same menu. Click `testid=automation-setup-add-event-filter` (`automation-setup-event-filter` appears) and `testid=automation-setup-event-test-payload-toggle`; `browser value 'testid=automation-setup-event-test-payload'` is a JSON object with `"source": "github"` and `"event": "issue_comment.created"`. Click `role=radio[name=/^Schedule/]` to return.- **Invalid drafts save (`F28.draft-save`, `F28.draft-validation`).** Fill `testid=automation-setup-name` with `QA_F28 Probe`, leave the prompt empty, `browser network --clear`, click `testid=automation-setup-save-draft`. `browser wait-url 'draftId='`; `browser toasts --history` includes `Draft saved in this conversation.`; `browser text 'testid=automation-setup-draft-validity'` is `Field required`; `control-openhands api GET /api/automation/v1/drafts/<draft-id>` (the id in the URL) has `validation_errors` `[{"field":"prompt","code":"missing",…}]` and `dispatchable: false` (`browser network --bodies` shows request bodies, not the response); `browser text 'testid=automation-setup-save-state'` is `Saved just now`. Now fill the prompt with `Reply pong. Do not run any tools.`, click `testid=automation-setup-frequency-custom`, fill `testid=automation-setup-custom-schedule` with `every day`, `browser network --clear`, press Tab, and `browser network --filter 'automation/v1'` lists no request (no blur validation). Click Save draft: the validity text is `Value error, Invalid cron expression: every day`. `browser network --clear`, click `testid=automation-setup-test`: `browser text 'testid=automation-setup-draft-runs-page'` contains the same cron error, `browser network` lists only the draft `PATCH` (200) and nothing under `dispatch`; click `testid=automation-setup-draft-runs-back`. Fill the cron with `0 0 1 1 *` and Save draft: it reads `Ready to test`.- **Required fields before Test (`F28.required-before-test`).** Run `control-openhands browser fill 'testid=automation-setup-prompt' ''`, Save draft, `browser network --clear`, click `testid=automation-setup-test`: `browser text 'testid=automation-setup-draft-runs-page'` contains `Add a prompt before continuing.` and `browser network --filter 'dispatch'` lists nothing. Click `testid=automation-setup-draft-runs-back`, fill the prompt back in and Save draft.@@ -61,7 +61,7 @@ Preconditions:- **Test gating (`F28.test-gated`).** After a successful Save draft `browser enabled 'testid=automation-setup-test'` is `true`; fill `testid=automation-setup-name` with `QA_F28 Probe 2` and it is `false`; Save draft and it is `true`.- **Custom kind and draft Test (`F28.kind-toggle`, `F28.draft-test`).** Click `testid=automation-setup-kind-custom` and `browser wait 'testid=automation-setup-custom-code'`; once the crossfade ends `browser count 'testid=automation-setup-prompt'` is `0` (right after the click it can still read `1`, and `visible` `true`) and `browser value 'testid=automation-setup-custom-code'` contains `AUTOMATION_CALLBACK_URL`. Click `testid=automation-setup-custom-details-toggle`: Entrypoint `python3 main.py`, Setup script path `setup.sh`. Save draft (`Ready to test`), click `testid=automation-setup-test` and `browser wait 'testid=automation-setup-draft-run'`: the row reads Pending and the page shows `Test run started.` Expected: `browser wait 'testid=automation-setup-draft-run >> text=Successful' --timeout 45000`. Known failure #18263 (reproduced 2026-10-15): the row stays Pending for 45 s with a single runs request in `browser network --filter '/runs'`; after `browser reload`, `testid=automation-setup-view-test-runs` shows it Successful. Click `testid=automation-setup-draft-runs-back`. Switching back with `testid=automation-setup-kind-prompt` restores the Prompt; leave the draft on Custom for the next bullet.- **Create (`F28.create`).** Run `control-openhands browser click 'testid=automation-setup-create' --expect-url '/automations/[0-9a-f-]{36}$'`. The detail h1 is `QA_F28 Probe 2`, the badge reads Active and the page shows **Script**. `browser goto /automations`: `browser count 'testid=automation-setup-draft-resume-<draft-id>'` is `0`.-- **Error contrast (`F28.error-contrast`).** Switch to Light+ (`F16`), open a draft whose cron is `every day` (as in the invalid-drafts bullet), and `browser screenshot 'testid=automation-setup-draft-details' --feature F28.error-contrast --name light-plus`. Read the pill's computed foreground and background and its parent surface with `browser eval`; composite its 5% red tint (`#fb2c36`) over the draft bar background. Expected contrast is at least 4.5:1. Click `testid=automation-setup-test` and inspect `testid=automation-setup-draft-runs-error` the same way. Repeat both checks in OpenHands-DeepSea, OpenHands-Neutral, OpenHands-Neo and Solarized Light, then restore OpenHands-Neutral. Known failure (reproduced 2026-10-16, [#18519](https://github.com/OpenHands/OpenHands/issues/18519)): the text follows the theme's error color, but the pill tint and the Test runs alert's background and border keep the same fixed red in every theme.+- **Error contrast (`F28.error-contrast`).** Switch to Light+ (`F16`), open a draft whose cron is `every day` (as in the invalid-drafts bullet), and `browser screenshot 'testid=automation-setup-draft-details' --feature F28.error-contrast --name light-plus`. Read the pill's computed `color` and `backgroundColor` with `browser eval`: the background is the text color at alpha `0.05` (Light+ `rgb(192, 40, 45)` and `oklab(0.528 0.170 0.080 / 0.05)`). Composite every ancestor's background down to the pill (painting each on a 1×1 canvas and reading the pixel normalizes `oklab()`), then the text color, and compute the WCAG ratio: at least 4.5:1. Click `testid=automation-setup-test` and inspect `testid=automation-setup-draft-runs-error` the same way: background alpha `0.1`, `borderTopColor` alpha `0.4`, same hue as the text. Repeat both checks in OpenHands-DeepSea, OpenHands-Neutral, OpenHands-Neo and Solarized Light, then restore OpenHands-Neutral. On 2026-10-17 the pill/alert ratios were Light+ 4.86/4.99, DeepSea 5.10/6.16, Neo and Neutral 4.86/4.95, Solarized Light 4.57/4.78.- **Cleanup.** Delete `QA_F28 Probe 2` from its kebab and any leftover `QA_F28` draft with its trash icon, as in F21; `api GET /api/automation/v1/drafts` lists none.## Gotchasdiff --git a/.agents/skills/verify-openhands/references/feature-map/README.md b/.agents/skills/verify-openhands/references/feature-map/README.mdindex d3d79d419d..7845d9fcd5 100644--- a/.agents/skills/verify-openhands/references/feature-map/README.md+++ b/.agents/skills/verify-openhands/references/feature-map/README.md@@ -5,7 +5,7 @@ of Agent Canvas. Read this index before driving the app, then open the matchingfeature file and run its recipe with `control-openhands`. The map is not a claimthat every row passes today; the run's evidence ledger is.-Maintenance baseline: main@d3a58cd1cc045bbb1b24f2c8c0d79a8404ef239b (2026-10-16). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.+Maintenance baseline: main@045290739f0af2ae4cc89d2a209c6bd930bb81ee (2026-10-17). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.## Baseline preconditionsdiff --git a/.agents/skills/verify-openhands/scripts/control-openhands.mjs b/.agents/skills/verify-openhands/scripts/control-openhands.mjsindex e3c40a822f..ce74bcaf1f 100755--- a/.agents/skills/verify-openhands/scripts/control-openhands.mjs+++ b/.agents/skills/verify-openhands/scripts/control-openhands.mjs@@ -751,9 +751,8 @@ async function cmdLaunch({ flags }) {OH_CANVAS_SAFE_BACKEND_PORT: String(ports.agentServer),OH_CANVAS_SAFE_AUTOMATION_PORT: String(ports.automation),OH_CANVAS_SAFE_VITE_PORT: String(ports.frontend),- // A private tmux server per run (OpenHands/OpenHands#17946: the launcher's- // <stateDir>/tmux is never created, so runs would share one server). It- // lives under /tmp because a socket below the long run path would exceed+ // A private tmux socket directory per run. It lives under /tmp, not the launcher's+ // <stateDir>/tmux, because a socket below the long run path would exceed// the 108-byte Unix socket limit.TMUX_TMPDIR: tmuxDirFor(dir),});diff --git a/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs b/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjsindex b10f6b9e1d..c7ecb78b8c 100644--- a/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs+++ b/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs@@ -1,7 +1,6 @@-// Where a run's private tmux server lives (OpenHands/OpenHands#17946: the-// launcher's <stateDir>/tmux is never created, so runs would share one-// server). Under /tmp because a socket below the long run path would exceed-// the 108-byte Unix socket limit; named by a hash of the whole run path so+// Where a run's private tmux server lives. Under /tmp, not the launcher's+// <stateDir>/tmux, because a socket below the long run path would exceed the+// 108-byte Unix socket limit; named by a hash of the whole run path so// two runs whose ids end alike (`qa-run-a1`, `qb-run-a1`) never share one.import { createHash } from "node:crypto";import { join, resolve } from "node:path";diff --git a/.pr/bugs/2026-10-17/phone-escape-backend-modals/desktop-edit-after-esc.png b/.pr/bugs/2026-10-17/phone-escape-backend-modals/desktop-edit-after-esc.pngnew file mode 100644index 0000000000..4a37cc27bdBinary files /dev/null and b/.pr/bugs/2026-10-17/phone-escape-backend-modals/desktop-edit-after-esc.png differdiff --git a/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-after-esc.png b/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-after-esc.pngnew file mode 100644index 0000000000..5059548511Binary files /dev/null and b/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-after-esc.png differdiff --git a/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-before-esc.png b/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-before-esc.pngnew file mode 100644index 0000000000..fa62a79746Binary files /dev/null and b/.pr/bugs/2026-10-17/phone-escape-backend-modals/phone-edit-before-esc.png differdiff --git a/.pr/map-maintenance/2026-10-17/F05.build-plan-shortcut-composer-ctrl-enter.png b/.pr/map-maintenance/2026-10-17/F05.build-plan-shortcut-composer-ctrl-enter.pngnew file mode 100644index 0000000000..9c322de3dcBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F05.build-plan-shortcut-composer-ctrl-enter.png differdiff --git a/.pr/map-maintenance/2026-10-17/F05.composer-resize-hover.png b/.pr/map-maintenance/2026-10-17/F05.composer-resize-hover.pngnew file mode 100644index 0000000000..6ece1bd6ffBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F05.composer-resize-hover.png differdiff --git a/.pr/map-maintenance/2026-10-17/F05.draft-persistence-model-returned-empty.png b/.pr/map-maintenance/2026-10-17/F05.draft-persistence-model-returned-empty.pngnew file mode 100644index 0000000000..6f9395d8a6Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F05.draft-persistence-model-returned-empty.png differdiff --git a/.pr/map-maintenance/2026-10-17/F06.critic-result-score0-lightplus.png b/.pr/map-maintenance/2026-10-17/F06.critic-result-score0-lightplus.pngnew file mode 100644index 0000000000..6ef800d764Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F06.critic-result-score0-lightplus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F06.markdown-rendering-link-lightplus.png b/.pr/map-maintenance/2026-10-17/F06.markdown-rendering-link-lightplus.pngnew file mode 100644index 0000000000..c59d9c14c9Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F06.markdown-rendering-link-lightplus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F06.tool-visualizers-timeout-clock-lightplus.png b/.pr/map-maintenance/2026-10-17/F06.tool-visualizers-timeout-clock-lightplus.pngnew file mode 100644index 0000000000..7a606a3e26Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F06.tool-visualizers-timeout-clock-lightplus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F07.display-cost-phone.png b/.pr/map-maintenance/2026-10-17/F07.display-cost-phone.pngnew file mode 100644index 0000000000..16f53ac23dBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F07.display-cost-phone.png differdiff --git a/.pr/map-maintenance/2026-10-17/F08.files-rich-plain-toml-sections-inline.png b/.pr/map-maintenance/2026-10-17/F08.files-rich-plain-toml-sections-inline.pngnew file mode 100644index 0000000000..7268d89246Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F08.files-rich-plain-toml-sections-inline.png differdiff --git a/.pr/map-maintenance/2026-10-17/F10.add-models-partial-light-plus-2.png b/.pr/map-maintenance/2026-10-17/F10.add-models-partial-light-plus-2.pngnew file mode 100644index 0000000000..2c81510b8fBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F10.add-models-partial-light-plus-2.png differdiff --git a/.pr/map-maintenance/2026-10-17/F10.edit-server-reason-desktop.png b/.pr/map-maintenance/2026-10-17/F10.edit-server-reason-desktop.pngnew file mode 100644index 0000000000..eeb93763dcBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F10.edit-server-reason-desktop.png differdiff --git a/.pr/map-maintenance/2026-10-17/F14.agent-request-card.png b/.pr/map-maintenance/2026-10-17/F14.agent-request-card.pngnew file mode 100644index 0000000000..94db249908Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F14.agent-request-card.png differdiff --git a/.pr/map-maintenance/2026-10-17/F14.agent-request-phone-width.png b/.pr/map-maintenance/2026-10-17/F14.agent-request-phone-width.pngnew file mode 100644index 0000000000..76402b87f7Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F14.agent-request-phone-width.png differdiff --git a/.pr/map-maintenance/2026-10-17/F15.save-errors-server-422-one-attempt.png b/.pr/map-maintenance/2026-10-17/F15.save-errors-server-422-one-attempt.pngnew file mode 100644index 0000000000..1baffe5be1Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F15.save-errors-server-422-one-attempt.png differdiff --git a/.pr/map-maintenance/2026-10-17/F17.delete-server-desktop-stale-delete-404-once.png b/.pr/map-maintenance/2026-10-17/F17.delete-server-desktop-stale-delete-404-once.pngnew file mode 100644index 0000000000..89f8c48b2bBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F17.delete-server-desktop-stale-delete-404-once.png differdiff --git a/.pr/map-maintenance/2026-10-17/F19.launch-errors-light-plus-desktop.png b/.pr/map-maintenance/2026-10-17/F19.launch-errors-light-plus-desktop.pngnew file mode 100644index 0000000000..78082a26cdBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F19.launch-errors-light-plus-desktop.png differdiff --git a/.pr/map-maintenance/2026-10-17/F20.page-render-title-18557.png b/.pr/map-maintenance/2026-10-17/F20.page-render-title-18557.pngnew file mode 100644index 0000000000..edf9523326Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F20.page-render-title-18557.png differdiff --git a/.pr/map-maintenance/2026-10-17/F20.uninstall-deepsea-hover-desktop.png b/.pr/map-maintenance/2026-10-17/F20.uninstall-deepsea-hover-desktop.pngnew file mode 100644index 0000000000..a549a7e56cBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F20.uninstall-deepsea-hover-desktop.png differdiff --git a/.pr/map-maintenance/2026-10-17/F22.agent-assisted-setup-profile-save-422.png b/.pr/map-maintenance/2026-10-17/F22.agent-assisted-setup-profile-save-422.pngnew file mode 100644index 0000000000..804cbeec49Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F22.agent-assisted-setup-profile-save-422.png differdiff --git a/.pr/map-maintenance/2026-10-17/F22.setup-tarball-create-chooser-light-plus.png b/.pr/map-maintenance/2026-10-17/F22.setup-tarball-create-chooser-light-plus.pngnew file mode 100644index 0000000000..a1145dc543Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F22.setup-tarball-create-chooser-light-plus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F23.edit-schedule-weekly-to-custom.png b/.pr/map-maintenance/2026-10-17/F23.edit-schedule-weekly-to-custom.pngnew file mode 100644index 0000000000..c45841fcceBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F23.edit-schedule-weekly-to-custom.png differdiff --git a/.pr/map-maintenance/2026-10-17/F25.cloud-device-flow-error-light-plus.png b/.pr/map-maintenance/2026-10-17/F25.cloud-device-flow-error-light-plus.pngnew file mode 100644index 0000000000..c09f6da576Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F25.cloud-device-flow-error-light-plus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F25.device-verify-page-notice-light-plus-desktop.png b/.pr/map-maintenance/2026-10-17/F25.device-verify-page-notice-light-plus-desktop.pngnew file mode 100644index 0000000000..eb3217df0eBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F25.device-verify-page-notice-light-plus-desktop.png differdiff --git a/.pr/map-maintenance/2026-10-17/F25.edit-escape-after-escape.png b/.pr/map-maintenance/2026-10-17/F25.edit-escape-after-escape.pngnew file mode 100644index 0000000000..9ea05e5b77Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F25.edit-escape-after-escape.png differdiff --git a/.pr/map-maintenance/2026-10-17/F25.shared-conversation-view-local-not-found.png b/.pr/map-maintenance/2026-10-17/F25.shared-conversation-view-local-not-found.pngnew file mode 100644index 0000000000..861120c818Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F25.shared-conversation-view-local-not-found.png differdiff --git a/.pr/map-maintenance/2026-10-17/F26.frontend-only-returning-recovery-gate.png b/.pr/map-maintenance/2026-10-17/F26.frontend-only-returning-recovery-gate.pngnew file mode 100644index 0000000000..2be4801446Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F26.frontend-only-returning-recovery-gate.png differdiff --git a/.pr/map-maintenance/2026-10-17/F27.planner-plan-plan-link-lightplus.png b/.pr/map-maintenance/2026-10-17/F27.planner-plan-plan-link-lightplus.pngnew file mode 100644index 0000000000..c8cb13543cBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F27.planner-plan-plan-link-lightplus.png differdiff --git a/.pr/map-maintenance/2026-10-17/F28.error-contrast-lightp-alert.png b/.pr/map-maintenance/2026-10-17/F28.error-contrast-lightp-alert.pngnew file mode 100644index 0000000000..3383c45c76Binary files /dev/null and b/.pr/map-maintenance/2026-10-17/F28.error-contrast-lightp-alert.png differdiff --git a/.pr/map-maintenance/2026-10-17/F28.schedule-presets-weekly-1030-custom.png b/.pr/map-maintenance/2026-10-17/F28.schedule-presets-weekly-1030-custom.pngnew file mode 100644index 0000000000..b6dd54a9dfBinary files /dev/null and b/.pr/map-maintenance/2026-10-17/F28.schedule-presets-weekly-1030-custom.png differ