🙌 OpenHands: AI-Driven Development
docs(verify-openhands): map maintenance 2026-10-22 (#18811)
Engel Nysta832d9a2850f
diff --git a/.agents/skills/verify-openhands/references/feature-map/F01-first-run-and-sign-in.md b/.agents/skills/verify-openhands/references/feature-map/F01-first-run-and-sign-in.mdindex f12bd63162..a825bf6bee 100644--- a/.agents/skills/verify-openhands/references/feature-map/F01-first-run-and-sign-in.md+++ b/.agents/skills/verify-openhands/references/feature-map/F01-first-run-and-sign-in.md@@ -34,7 +34,7 @@ Source: `src/root.tsx`, `src/components/features/onboarding/`, `src/components/f- `F01.route-error-boundary`: an unknown URL renders a "Page not found" page inside the app shell, with the sidebar and a **Home** link back to `/`, and logs no page errors.- `F01.document-title`: the tab title is `OpenHands`. On a conversation it is `<status emoji> <conversation title> | OpenHands`.- `F01.bootstrap-loading`: a centered spinner card shows while `/server_info` loads (transient, not-run).-- `F01.locked-cloud-first-run`: on a canvas locked to an OpenHands Cloud host, first run shows only the Cloud login (Add backend) without a progress bar or close button (blocked).+- `F01.locked-cloud-first-run`: on a canvas locked to an OpenHands Cloud host, first run shows only the Cloud login (an "OpenHands Cloud" card with **Connect to OpenHands**) without a progress bar or close button. After the device-flow login the app opens on Home with no onboarding modal: a locked canvas never opens the agent/LLM onboarding flow, because the Cloud app owns that setup.- `F01.cookie-auth-redirect`: OHE cookie-auth deployments probe the main-app session behind a spinner and redirect to its `/login` when it is missing (blocked).## How to get to it (user POV)@@ -58,7 +58,7 @@ Preconditions:Local run (`launch --new`):-- **First-run gate (`F01.first-run-gate`).** Run `control-openhands browser goto /conversations` and `control-openhands browser testids`. The list includes `first-run-onboarding-screen`, `onboarding-modal` and `telemetry-consent-form` (the portaled consent form and its Confirm button come last); `root-layout` is absent. Run `control-openhands browser screenshot --feature F01.first-run-gate --name fresh`. The screenshot shows the consent modal over "Choose your agent".+- **First-run gate (`F01.first-run-gate`).** Run `control-openhands browser goto /conversations` and `control-openhands browser testids`. The list includes `first-run-onboarding-screen`, `onboarding-modal` and `telemetry-consent-form` (the portaled consent form and its Confirm button are listed right after `first-run-onboarding-screen`, before `onboarding-modal`); `root-layout` is absent. Run `control-openhands browser screenshot --feature F01.first-run-gate --name fresh`. The screenshot shows the consent modal over "Choose your agent".- **Consent (`F01.telemetry-consent`).** Run `control-openhands browser snapshot 'testid=telemetry-consent-form'`. It reads `This preference is saved for the local backend “Local” at http://127.0.0.1:<port>.` and the checkbox is `[checked]`. Run `control-openhands browser press Escape` and `control-openhands browser count 'testid=telemetry-consent-form'`; the count stays `1`. Run `control-openhands browser uncheck 'testid=telemetry-consent-form >> role=checkbox'`, `control-openhands browser click 'testid=confirm-telemetry-preferences'`, `control-openhands browser wait 'testid=telemetry-consent-form' --state detached`, then `control-openhands api GET /api/settings --pick misc_settings.app_preferences.user_consents_to_analytics` is `false`. Run `control-openhands browser reload` and `control-openhands browser count 'testid=telemetry-consent-form'`; the count is `0`. The dialog's description text reads at least 4.5:1 in every theme. Known failure (reproduced 2026-10-16, [#18539](https://github.com/OpenHands/OpenHands/issues/18539)): in Light+ and Solarized Light it stays a fixed `#a3a3a3`, measuring 2.27:1 and 2.06:1 on the modal surface.- **Backend down on first run (`F01.onboarding-backend-step`).** Run `control-openhands browser reset`, `control-openhands service stop agent-server`, `control-openhands browser goto /` and `control-openhands browser wait 'testid=onboarding-backend-disconnected' --timeout 20000`. The modal shows four progress steps; step 0 "Check your backend" has the alert `Disconnected` / `Update the connection details below and save to retry.` above the prefilled Local form. The alert text reads at least 4.5:1 in all five themes (arrange each theme with `control-openhands browser eval "localStorage.setItem('openhands-color-theme','<id>')"` and `browser reload`). A backend marked down is not re-probed: after `control-openhands restart` the alert stays until **Next** re-tests the connection and moves on to Choose your agent. Restore with `localStorage.removeItem('openhands-color-theme')`; a later `onboard --skip` closes the modal.- **Modal shell (`F01.onboarding-modal`).** Run `control-openhands browser press Escape`, `control-openhands browser count 'testid=onboarding-modal'` (`1`) and `control-openhands browser attr 'testid=onboarding-modal' data-current-step` (`0`). Run `control-openhands browser mouse-click 40 40` on the bare backdrop; the count stays `1` and the step is unchanged. `onboarding-progress-step-0..2` exist and `-3` does not, because the backend is healthy.@@ -92,7 +92,7 @@ Public run B (`launch --new --public`), onboarding skipped:Not reachable locally:-- **Locked Cloud first run (`F01.locked-cloud-first-run`).** Blocked: needs a build locked to a Cloud host (`VITE_LOCK_TO_CLOUD` or `static-server --lock-to-cloud`) and an OpenHands Cloud account. `launch` has no such flag.+- **Locked Cloud first run (`F01.locked-cloud-first-run`).** Needs an OpenHands Cloud account (the F25 Cloud precondition; without it, blocked) and a locked frontend, which `launch` cannot start: serve this checkout's build by hand on a free port, `Q=$OH_VERIFY_RUN/private/f01-locked; mkdir -p $Q; nohup node scripts/static-server.mjs --dir build --port 18990 --host 127.0.0.1 --lock-to-cloud https://app.all-hands.dev > $Q/static.log 2>&1 & echo $! > $Q/pid` (`curl -s http://127.0.0.1:18990/ | grep -o '__AGENT_CANVAS_LOCK_TO_CLOUD__[^;]*'` prints the host). Run `control-openhands browser goto http://127.0.0.1:18990/ --allow-external` and `control-openhands browser wait 'testid=add-backend-login-button'`: the page (`browser text body`) holds only `OpenHands Cloud`, `Connect instantly to OpenHands Cloud to use Agent Canvas` and **Connect to OpenHands**, with no progress bar or close button (the card is the portaled `testid=onboarding-modal`; `first-run-onboarding-screen` itself is empty) (`browser screenshot --feature F01.locked-cloud-first-run --name first-run`). Approve as in F25's **Cloud backend** bullet (`browser click 'testid=add-backend-login-button'`, `browser wait-tab 'oauth/device/verify'`, `browser tab 1`, `browser click 'role=button[name="Authorize Device"]'`, `browser tab 0`); `browser wait 'testid=root-layout' --timeout 60000` passes on `/`, and a few seconds later `browser count 'testid=onboarding-modal'` and `browser count 'testid=first-run-onboarding-screen'` are both `0`. The same page drives F02's locked-Cloud checklist check and `F25.locked-cloud`. Clean up: `kill -TERM $(cat $Q/pid)` and delete this login's `Device Link Access Key (<code>)` on the Cloud API keys page as F25 describes.- **Cookie-auth redirect (`F01.cookie-auth-redirect`).** Blocked: needs an OHE cookie-auth deployment.- **Bootstrap spinner (`F01.bootstrap-loading`).** Blocked on the 2026-10-09 run: the agent attempted a fresh first paint, but the spinner lasted under a second on the healthy stack; slowing `/server_info` would require request interception rather than the real service response.diff --git a/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md b/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.mdindex a4c9170ffb..70f08b2a62 100644--- a/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md+++ b/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md@@ -21,14 +21,14 @@ Source: `src/components/features/sidebar/`, `src/components/features/command-men- `F02.conversation-list`: the expanded rail embeds the conversation list (`conversation-panel`). The list itself belongs to the conversations family.- `F02.command-menu-open-close`: Control+k, Meta+k and the sidebar **Search commands** row (also the search icon on the collapsed rail) open a modal `Command menu` dialog with the input focused. Escape or the backdrop closes it, and the query is cleared on close. Escape returns focus to the element that had it before the menu opened (the **Search commands** row when it opened the menu). The row and the menu header show the shortcut as `Ctrl+K` off Apple platforms and `⌘K` on macOS and iOS.- `F02.command-menu-focus`: the shortcut works while typing in another field (the composer) and moves focus to the search input. Tab moves focus to the first option. Escape closes the menu from any focused element inside it and returns focus to the composer.-- `F02.command-menu-items`: the menu shows Navigation (New chat, Customize, Automations, MCP servers), Settings (Settings, Agent settings, LLM profiles, Model Router, Condenser, Agent Context, Verification, Application, Secrets) and Actions (Toggle sidebar). Each **Go** item navigates to its route.+- `F02.command-menu-items`: the menu shows Navigation (New chat, Customize, Automations, MCP servers), Settings (Settings, Agent settings, LLM profiles, Model Router, Agent Context, Verification, Application, Secrets) and Actions (Toggle sidebar). Each **Go** item navigates to its route. **Condenser** is listed only while a Cloud backend is active (between Model Router and Agent Context); on a local backend each agent profile has its own condenser and the menu has no Condenser item.- `F02.command-menu-toggle-sidebar`: the **Toggle sidebar** action collapses or expands the desktop rail.- `F02.command-menu-search`: every search term must match the title, description or keywords. **No commands found** is the empty state. The X button clears the query. Arrow keys wrap around the list, hovering an item selects it, Enter runs the selected item, and Enter does nothing when nothing matches.- `F02.command-menu-new-tab`: a middle-click or modifier-click on a **Go** item opens the route in a new tab and leaves the menu open.- `F02.command-menu-phone`: at 390 px the menu fits the screen and opens from the drawer's Search row. Escape closes the menu and leaves the drawer open. Running an item closes the drawer.- `F02.mobile-drawer`: at phone width the rail is hidden and a hamburger (`sidebar-mobile-menu-toggle`) opens a 300 px drawer. The hamburger is in the top bar, or in the chat header on conversation pages. The drawer closes on its chevron, Escape, a backdrop click or any route change. A modal opened over the drawer takes Escape first: Escape closes only that modal (or nothing, for one that ignores Escape) and the drawer stays open.- `F02.mobile-top-bar`: at phone width, settings sub-pages and Customize detail pages (`/skills`, `/mcp`, `/plugins`, `/apps`) show a Back chevron that returns to `/settings` or `/customize`.-- `F02.checklist`: after onboarding, the expanded rail shows a **Getting started** card with **N complete** and six items. They link to `/settings/llm`, `/conversations`, `/automations`, `/settings/agents`, `/mcp` and the external Slack invite.+- `F02.checklist`: after onboarding, the expanded rail shows a **Getting started** card with **N complete** and six items. They link to `/settings/llm`, `/conversations`, `/automations`, `/settings/agents`, `/mcp` and the external Slack invite. A canvas locked to a Cloud host never shows the card, and its Settings → Application page has no **Show Getting Started checklist** switch.- `F02.checklist-item-links`: clicking an item's row navigates in the same tab to its route (for example **Schedule a task** opens `/automations`) without marking the item done.- `F02.checklist-progress`: items get a check and a strikethrough when done. The LLM item completes when a profile is configured, Start your first chat when a conversation exists, Customize your agent after a visit to `/settings/agents`, and Join Slack on click.- `F02.checklist-preview`: hovering an item shows a preview with a description, a **Documentation** link and an action button that navigates.@@ -70,9 +70,9 @@ Preconditions:- **Settings gear (`F02.settings-link`).** The previous bullet expanded the rail: collapse it with `control-openhands browser click 'testid=sidebar-collapse-toggle'`, then run `control-openhands browser click 'testid=collapsed-settings-link'` and `control-openhands browser wait-url '/settings'`. Expand it again with `control-openhands browser click 'testid=sidebar-collapse-toggle'`, then run `control-openhands browser goto /conversations`, `control-openhands browser click 'testid=backend-selector-settings-link'` and `control-openhands browser url`; the URL ends in `/settings/agents`. `control-openhands browser attr 'testid=backend-selector-settings-link' target` is `null` (same tab).- **Conversation list (`F02.conversation-list`).** After `conversation start` (preconditions), run `control-openhands browser goto /conversations` and `control-openhands browser testids 'testid=conversation-panel'`. The list includes `conversation-card`.- **Open and close the menu (`F02.command-menu-open-close`).** From `/conversations` run `control-openhands browser text 'testid=command-menu-trigger'`. It reads `Search commands` and `CTRL+K` on Linux and Windows (the hint is `Ctrl+K`, drawn in capitals; macOS and iOS show `⌘K`). Run `control-openhands browser press Control+k`, then `control-openhands browser visible 'testid=command-menu'` (`true`), `control-openhands browser eval "document.activeElement && document.activeElement.id"` (`command-menu-search`) and `control-openhands browser text 'testid=command-menu >> css=kbd'` (`CTRL+K`, the header hint). `control-openhands browser screenshot --feature F02.command-menu-open-close --name open` shows the dimmed, blurred backdrop. Run `control-openhands browser press Escape`; `browser count 'testid=command-menu'` is `0`. Run `control-openhands browser press Meta+k` (count `1`), `control-openhands browser type 'testid=command-menu >> role=combobox' secr`, then `control-openhands browser click 'testid=command-menu >> role=button[name="Close command menu"]'` (the backdrop; count `0`). Run `control-openhands browser click 'testid=command-menu-trigger'` and `control-openhands browser value 'testid=command-menu >> role=combobox'`; the value is `""` because the query was cleared on close. Close it with `control-openhands browser press Escape`; focus is back on the row (`control-openhands browser eval "document.activeElement.dataset.testid"` is `command-menu-trigger`). Collapsed rail: run `control-openhands browser click 'testid=sidebar-collapse-toggle'`, `control-openhands browser click 'testid=command-menu-trigger'` (the search icon; count `1`), `control-openhands browser press Escape`, then expand with `control-openhands browser click 'aside[data-collapsed]'`.-- **Every item navigates (`F02.command-menu-items`).** The previous bullet closed the menu: run `control-openhands browser press Control+k`, then `control-openhands browser snapshot 'testid=command-menu' --feature F02.command-menu-items --name all-items`; it lists 14 options under Navigation, Settings and Actions, with Model Router after LLM profiles and Agent Context after Condenser settings. Close it with `control-openhands browser press Escape` and rest the pointer off the list with `control-openhands browser hover 'testid=command-menu-trigger'` (see Gotchas). For each item run `control-openhands browser goto /conversations`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' '<term>'`, `control-openhands browser press Enter` and `control-openhands browser wait-url '<route>$'`. Expected routes: `New chat` → `/conversations`, `Customize` → `/mcp` (via `/customize`), `Automations` → `/automations`, `MCP servers` → `/mcp`, `Settings overview` → `/settings/agents` (via `/settings`), `Agent settings` → `/settings/agents`, `LLM profiles` → `/settings/llm`, `Model Router` → `/settings/meta-llm`, `Condenser` → `/settings/condenser`, `Agent Context` → `/settings/agent-context`, `Verification` → `/settings/verification`, `Application` → `/settings/app`, `Secrets` → `/settings/secrets`. `browser count 'testid=command-menu'` is `0` after each run. Click path: `control-openhands browser goto /settings/app`, open the menu, type `new chat`, then `control-openhands browser click 'testid=command-menu >> role=option[name=/New chat/]'`; the URL becomes `/conversations`.+- **Every item navigates (`F02.command-menu-items`).** The previous bullet closed the menu: run `control-openhands browser press Control+k`, then `control-openhands browser snapshot 'testid=command-menu' --feature F02.command-menu-items --name all-items`; on a local backend it lists 13 options under Navigation, Settings and Actions, with Model Router after LLM profiles and Agent Context after Model Router; `browser fill 'testid=command-menu >> role=combobox' condenser` leaves `0` options. Close it with `control-openhands browser press Escape` and rest the pointer off the list with `control-openhands browser hover 'testid=command-menu-trigger'` (see Gotchas). For each item run `control-openhands browser goto /conversations`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' '<term>'`, `control-openhands browser press Enter` and `control-openhands browser wait-url '<route>$'`. Expected routes: `New chat` → `/conversations`, `Customize` → `/mcp` (via `/customize`), `Automations` → `/automations`, `MCP servers` → `/mcp`, `Settings overview` → `/settings/agents` (via `/settings`), `Agent settings` → `/settings/agents`, `LLM profiles` → `/settings/llm`, `Model Router` → `/settings/meta-llm`, `Agent Context` → `/settings/agent-context`, `Verification` → `/settings/verification`, `Application` → `/settings/app`, `Secrets` → `/settings/secrets`. `browser count 'testid=command-menu'` is `0` after each run. Click path: `control-openhands browser goto /settings/app`, open the menu, type `new chat`, then `control-openhands browser click 'testid=command-menu >> role=option[name=/New chat/]'`; the URL becomes `/conversations`.- **Toggle sidebar (`F02.command-menu-toggle-sidebar`).** Run `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' toggle`, `control-openhands browser press Enter`, then `control-openhands browser attr 'aside[data-collapsed]' data-collapsed` (`true`). Repeat with the term `sidebar`; the value returns to `false`.-- **Search, empty state, keyboard (`F02.command-menu-search`).** The Toggle sidebar action closed the menu: open it with `control-openhands browser press Control+k`, then run `control-openhands browser fill 'testid=command-menu >> role=combobox' theme` and `control-openhands browser count 'testid=command-menu >> role=option'` (`1`, Application, matched by its keywords). The term `settings secret` leaves `1` option, `secret automations` leaves `0` (all terms must match) and `model` leaves `3` (MCP servers, LLM profiles, Model Router). Fill `qa-zzz-nothing`, then run `control-openhands browser text 'testid=command-menu >> role=listbox'` (`No commands found` / `Try a page, action, or setting name.`) and `control-openhands browser screenshot --feature F02.command-menu-search --name no-results`. Run `control-openhands browser press Enter`; the menu stays open and the URL does not change. Run `control-openhands browser click 'testid=command-menu >> role=button[name="Clear search"]'`; the value is `""` and the count is back to `14`. Keyboard: `control-openhands browser attr 'testid=command-menu >> role=combobox' aria-activedescendant` starts at `command-menu-option-new-chat`. `control-openhands browser press ArrowUp` wraps to `command-menu-option-toggle-sidebar`, and `ArrowDown` returns to `new-chat`, then moves to `customize`. `control-openhands browser hover 'testid=command-menu >> role=option[name=/Secrets settings/]'` sets the value to `command-menu-option-secrets-settings`.+- **Search, empty state, keyboard (`F02.command-menu-search`).** The Toggle sidebar action closed the menu: open it with `control-openhands browser press Control+k`, then run `control-openhands browser fill 'testid=command-menu >> role=combobox' theme` and `control-openhands browser count 'testid=command-menu >> role=option'` (`1`, Application, matched by its keywords). The term `settings secret` leaves `1` option, `secret automations` leaves `0` (all terms must match) and `model` leaves `3` (MCP servers, LLM profiles, Model Router). Fill `qa-zzz-nothing`, then run `control-openhands browser text 'testid=command-menu >> role=listbox'` (`No commands found` / `Try a page, action, or setting name.`) and `control-openhands browser screenshot --feature F02.command-menu-search --name no-results`. Run `control-openhands browser press Enter`; the menu stays open and the URL does not change. Run `control-openhands browser click 'testid=command-menu >> role=button[name="Clear search"]'`; the value is `""` and the count is back to `13`. Keyboard: `control-openhands browser attr 'testid=command-menu >> role=combobox' aria-activedescendant` starts at `command-menu-option-new-chat`. `control-openhands browser press ArrowUp` wraps to `command-menu-option-toggle-sidebar`, and `ArrowDown` returns to `new-chat`, then moves to `customize`. `control-openhands browser hover 'testid=command-menu >> role=option[name=/Secrets settings/]'` sets the value to `command-menu-option-secrets-settings`.- **Open in a new tab (`F02.command-menu-new-tab`).** With the menu open and `secrets` typed, `control-openhands browser attr 'testid=command-menu >> role=option[name=/Secrets settings/]' href` is `/settings/secrets`. Run `control-openhands browser click 'testid=command-menu >> role=option[name=/Secrets settings/]' --button middle` and `control-openhands browser tabs`; tab 1 is `/settings/secrets`, tab 0 keeps its URL, and `browser count 'testid=command-menu'` is still `1`. Run `control-openhands browser close-tab 1`. The modifier path works the same: `control-openhands browser click 'testid=command-menu >> role=option[name=/Secrets settings/]' --modifiers Control` opens tab 1 with the menu still open; close that tab too. The click leaves focus on the option; `control-openhands browser press Escape` still closes the menu (count `0`).- **Keyboard focus (`F02.command-menu-focus`).** From `/conversations` run `control-openhands browser click 'testid=chat-input'`, `control-openhands browser press Control+k`, `browser count 'testid=command-menu'` (`1`) and `control-openhands browser eval "document.activeElement && document.activeElement.id"` (`command-menu-search`): the shortcut opens the menu from inside the composer. Run `control-openhands browser press Tab`; the same `eval` returns `command-menu-option-new-chat`. `control-openhands browser press Escape` closes the menu from the option (count `0`), and `control-openhands browser eval "document.activeElement.dataset.testid"` returns `chat-input`: focus is back in the composer.- **Phone drawer (`F02.mobile-drawer`).** Run `control-openhands browser viewport phone` and `control-openhands browser goto /conversations`. `control-openhands browser visible 'aside[data-collapsed]'` is `false`. Run `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'` and `control-openhands browser wait 'testid=sidebar-mobile-drawer'`. `control-openhands browser bbox 'testid=sidebar-mobile-drawer'` gives width `300`, `insideViewport` `true` and `pageHorizontalOverflow` `false`, and `control-openhands browser screenshot --feature F02.mobile-drawer --name open` shows the full rail. Close paths, each followed by `control-openhands browser wait 'testid=sidebar-mobile-drawer' --state detached`: `control-openhands browser click 'testid=sidebar-mobile-drawer-close'`; `control-openhands browser press Escape`; `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=sidebar-automations-link'` (route change; the URL becomes `/automations`); and a backdrop click to the right of the drawer, `control-openhands browser mouse-click 350 400` (on `/conversations`, `control-openhands browser click 'testid=submit-button' --force` also lands on the backdrop, because the empty composer's send button sits at x≈326 under it). On a conversation page (`browser goto` the URL from `conversation start`), `control-openhands browser bbox 'testid=sidebar-mobile-menu-toggle'` is at the top-left of the chat header. `control-openhands browser screenshot --feature F02.mobile-drawer --name conversation-header` shows it beside the title, and clicking it opens the drawer. Modals over the drawer: on `/conversations` open the drawer, `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=conversation-panel-new-thread-picker'`, `control-openhands browser click 'text=+ Add Workspace'` (the `Add workspaces` dialog, `testid=folder-browser-modal`) and `control-openhands browser press Escape`: `browser count 'testid=folder-browser-modal'` is `0` and the drawer stays: `browser wait 'testid=sidebar-mobile-drawer' --state detached --timeout 3000` times out (an immediate count can still read `1` while a closing drawer animates out); a second Escape closes the drawer (the same wait without `--timeout` passes). The same holds for a row's Delete confirmation: hover `'testid=sidebar-mobile-drawer >> testid=conversation-card'`, click its `ellipsis-button`, then `testid=delete-button`, and press Escape: no `role=dialog` is left, the drawer and the conversation stay (`--name delete-confirm-after-escape`). Return with `control-openhands browser viewport desktop`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F03-home.md b/.agents/skills/verify-openhands/references/feature-map/F03-home.mdindex 2a8ec74833..de2be600f9 100644--- a/.agents/skills/verify-openhands/references/feature-map/F03-home.md+++ b/.agents/skills/verify-openhands/references/feature-map/F03-home.md@@ -10,7 +10,7 @@ warning banner replaces a silent failure when no LLM is configured. Below thecomposer, home offers recommended automations, a pinned-automations grid and anactivity list of enabled automations with per-item actions.-Source: `src/routes/home.tsx`, `src/routes/index-home.tsx`, `src/components/features/home/`, `src/components/features/plugins/plugin-picker-modal.tsx`, `src/components/features/automations/recommended-automations-launcher.tsx`, `src/hooks/use-home-pinned-automations.ts`, `src/fixtures/home-automations-demo.ts`, `src/hooks/use-home-automation-actions.ts`, `src/utils/recommended-automation-rail.ts`, `src/components/features/automations/responder-deployment-modal.tsx`.+Source: `src/routes/home.tsx`, `src/routes/index-home.tsx`, `src/components/features/home/`, `src/components/features/plugins/plugin-picker-modal.tsx`, `src/components/features/automations/recommended-automations-launcher.tsx`, `src/hooks/use-home-pinned-automations.ts`, `src/fixtures/home-automations-demo.ts`, `src/hooks/use-home-automation-actions.ts`, `src/utils/recommended-automation-rail.ts`, `src/components/features/automations/responder-deployment-modal.tsx`, `src/components/features/automations/built-in-integration-choice-modal.tsx`.## Sub-features@@ -45,6 +45,8 @@ Workspaces (local backend):Automations on home:- `F03.recommended-automations-rail`: a horizontally scrolling rail of recommended automations with edge fades; cards with a setup form open `/automations/new/<id>`.+- `F03.recommended-rail-dismiss`: the rail header has an X named **Hide Recommended automations**, revealed while the rail is hovered or holds keyboard focus (always visible on touch and coarse-pointer devices). Clicking it removes the whole rail from Home at once and turns **Show Recommended automations** in Application settings (F16.recommended-automations) off without a reload; the choice is stored in this browser and survives a reload. The rails on the empty automations dashboard and in onboarding have no X and ignore it.+- `F03.recommended-built-in-integration-choice`: on a Cloud backend whose built-in integration covers the card (GitHub/GitLab issue to PR/MR, the Jira issue-to-PR cards, Slack channel monitor), the card first opens **Choose your setup**: when the user has connected that integration it shows how to use it (an `@openhands Please complete this task!` example and **Got it**), otherwise it offers **Connect to <name>**; the polling automation is the alternative either way (blocked locally; on a local backend these cards go straight to setup or to the responder choice).- `F03.recommended-responder-choice`: GitHub/Slack responder cards on a local backend first ask **Choose how your responder should run**.- `F03.responder-cloud-option`: in that dialog **Open OpenHands Cloud integrations** opens `https://app.all-hands.dev/settings/integrations` in a new tab, closes the dialog and saves no secret; X closes it without a choice.- `F03.recommended-missing-integration`: a card whose required integration is missing opens that integration's install dialog first.@@ -84,7 +86,7 @@ Preconditions:- Workspace fixtures: `control-openhands fixture git-repo --name qa-f03-repo` and `control-openhands fixture folder --name qa-f03-folder` (both under `<run>/workspace`).- Automation fixtures (arranged through F22 import and F21 Turn on, not proof): `control-openhands fixture file --name qa-f03-pong.automation.json --content '{"version":1,"kind":"automation","spec":{"name":"QA_F03 Pong","trigger":{"type":"cron","schedule":"0 0 1 1 *","timezone":"UTC"},"enabled":true,"prompt":"Reply with the single word: pong. Do not run any tools."}}'` and the same with `qa-f03-beta` / `QA_F03 Beta`. Run `control-openhands browser goto /automations`, then for each file run `control-openhands browser click 'testid=automations-add-automation'`, `control-openhands browser click 'testid=automations-import-automation'`, `control-openhands browser upload 'testid=automations-import-file' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f03-pong.automation.json"`, `control-openhands browser click 'testid=import-automation-confirm'`, `control-openhands browser wait 'testid=import-automation-modal' --state detached`. Imports arrive disabled: turn each on with `control-openhands browser click '[data-testid^="automation-card-"] >> has-text=QA_F03 Pong >> role=button[name="Automation actions"]'` and `control-openhands browser click 'role=list >> has-text=Export >> role=button[name="Turn on"]'`. The yearly cron never fires by itself.- **Isolated workspace notice (`F03.isolated-workspace-notice`).** Requires a real Local backend whose successful `/server_info` reports `conversation_runtime: "docker"`; native computer-use tools and a model are not prerequisites. In an owned disposable run, first select an owned host workspace on Home while runtime is Local. Preserve the mounted Home page, save the original launcher runtime fields, configure the real Docker runtime, then restart only that owned run. Refresh actual capability through the normal Manage Backends host edit (an equivalent `127.0.0.1` ↔ `localhost` host change updates `connectionRevision`; saving the unchanged host can leave the cached Local result). Verify `/server_info` is Docker and Home shows the stale-selection notice plus **Clear host project selection**. Click Clear: the host chip disappears, status becomes **New isolated workspace**, and Open workspace is disabled with the isolated-runtime explanation. Repeat actual Clear at desktop and phone widths; arrange the second stale selection through another real Local-to-Docker transition, not store/DOM injection. No conversation, model call or child container is required. Restore the exact original runtime fields, restart/doctor Local, remove only the owned workspace and stop/purge the run. Proven 2026-10-17 at frozen `61a3bd93e642a0ff4def73b29496a73245628209`, SDK 1.55.1: `weekly-isolated-notice/evidence/F03.isolated-workspace-notice/` retains actual notice/Clear screenshots; runtime receipts are under `evidence/receipts/`.-- Blocked here: `F03.open-repository-dialog` (OpenHands Cloud backend with a connected GitHub/GitLab/Bitbucket provider), `F03.workspaces-unsupported` (an Agent Server without `/api/workspaces`), `F03.recommended-prompt-launch` (a Tavily/Linear/Jira integration installed so some card has nothing missing).+- Blocked here: `F03.open-repository-dialog` (OpenHands Cloud backend with a connected GitHub/GitLab/Bitbucket provider), `F03.workspaces-unsupported` (an Agent Server without `/api/workspaces`), `F03.recommended-prompt-launch` (a Tavily/Linear/Jira integration installed so some card has nothing missing), `F03.recommended-built-in-integration-choice` (an OpenHands Cloud backend with the built-in GitHub, GitLab, Jira or Slack integration enabled, once connected and once not connected).Starting work:@@ -117,6 +119,7 @@ Workspaces:Automations on home (fixtures imported and turned on):- **Recommended rail (`F03.recommended-automations-rail`).** On `/` after `browser reload`, `control-openhands browser attr 'testid=recommended-automations-rail-fade-left' data-visible` is `false` and `...-fade-right` is `true`; `control-openhands browser scroll 'testid=recommended-automation-rail-card-github-agents-md-maintainer'` turns the left fade `true` (`browser screenshot --feature F03.recommended-automations-rail --name scrolled-end`). `control-openhands browser click 'testid=recommended-automation-rail-card-custom-automation' --expect-url 'automations/new'` lands on `/automations/new/custom-automation` (`github-pr-reviewer`, `github-issue-to-pr` and `news-digest` do the same).+- **Dismiss the rail (`F03.recommended-rail-dismiss`).** On `/` with the rail shown, `control-openhands browser attr 'testid=recommended-automations-rail-dismiss' aria-label` is `Hide Recommended automations` and `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=recommended-automations-rail-dismiss]')).opacity"` is `0`. Mouse: `control-openhands browser hover 'testid=recommended-automations-rail'` makes the same `eval` read `1` (`browser screenshot --feature F03.recommended-rail-dismiss --name x-hover`), then `control-openhands browser click 'testid=recommended-automations-rail-dismiss'`: `browser count 'testid=recommended-automations-rail'` is `0` at once and `browser eval "localStorage.getItem('openhands.home_recommended_automations_dismissed')"` is `true`. Without a reload, `control-openhands browser click 'testid=backend-selector-settings-link'` and `control-openhands browser click 'testid=sidebar-settings-/settings/app' --expect-url '/settings/app'`: `browser eval "document.querySelector('[data-testid=show-recommended-automations-switch]').checked"` is `false`. Back on `/`, `browser reload`: the rail count stays `0` while the Automations list still renders. Keyboard: with the rail shown, `control-openhands browser focus 'testid=recommended-automation-rail-card-github-pr-reviewer'` reveals the X (opacity `1`), `control-openhands browser press Shift+Tab` focuses it (`document.activeElement` aria-label `Hide Recommended automations`) and `control-openhands browser press Enter` removes the rail. On the empty dashboard (`control-openhands browser goto /automations` with no automations: run this check before importing the fixtures, or after removing them) the rail is still shown while dismissed on Home, and `browser count 'testid=recommended-automations-rail-dismiss'` is `0`. Repeat the mouse path at `browser viewport phone` after `browser scroll 'testid=recommended-automations-rail'` (`browser bbox 'testid=recommended-automations-rail-dismiss'` is `insideViewport`). Restore with the Application settings switch (`F16.recommended-automations`); the rail is back on the next Home render.- **Responder choice (`F03.recommended-responder-choice`).** From `/conversations` (`control-openhands browser goto /conversations`; the rail click left `/automations/new/...`), `control-openhands browser click 'testid=recommended-automation-rail-card-github-agents-md-maintainer'`; `control-openhands browser snapshot 'testid=responder-deployment-modal'` shows `Choose how your responder should run`, **Continue with local setup** and **Open OpenHands Cloud integrations** (`slack-channel-monitor` too). `control-openhands browser click 'testid=responder-deployment-continue-local'` saves an `OPENHANDS_URL` secret and opens the GitHub MCP install dialog (`browser snapshot 'role=dialog'`: `dialog "GitHub"`, Personal access token); `control-openhands browser click 'role=dialog >> role=button[name="Cancel"]'` closes it on `/conversations`. Delete `OPENHANDS_URL` afterwards in Settings → Secrets (F14): `control-openhands browser goto /settings/secrets`, `control-openhands browser click 'testid=secret-item >> has-text=OPENHANDS_URL >> testid=delete-secret-button'`, then `control-openhands browser click 'role=dialog >> role=button[name="Confirm"]'`.- **Cloud option (`F03.responder-cloud-option`).** From `/conversations` (`control-openhands browser goto /conversations`), `control-openhands browser click 'testid=recommended-automation-rail-card-slack-channel-monitor'`, then `control-openhands browser click 'testid=responder-deployment-open-openhands-cloud'`: `browser count 'testid=responder-deployment-modal'` is `0`, `control-openhands browser tabs` lists a second page `https://app.all-hands.dev/settings/integrations` (it can take a second to appear: rerun `tabs` before concluding) (close it with `control-openhands browser close-tab 1`), and `api GET /api/settings/secrets` has no `OPENHANDS_URL`. Reopen the card and `control-openhands browser click 'testid=responder-deployment-modal-close'`: the dialog is gone and nothing else opens.- **Missing integration (`F03.recommended-missing-integration`).** `control-openhands browser click 'testid=recommended-automation-rail-card-research-brief-writer'` opens `dialog "Tavily"` (Command `npx -y tavily-mcp`, Tavily API key); Cancel closes it and no conversation is created.@@ -156,6 +159,7 @@ Phone and cleanup:- A workspace and its parent can share a name (`workspace`); their test ids differ only by prefix (`manage-workspaces-row-` vs `manage-workspaces-parent-row-`).- `chat-input` is a contenteditable `div`: it stays editable when the composer is "disabled"; assert the Send button and the absence of a new conversation instead.- `conversation start` without `--stay` navigates home and loses a picked workspace and plugins; drive the submit button as above.+- The rail's X is `visibility: hidden` until the rail is hovered or holds focus on a fine pointer: `click --hover-first` on the X fails ("element is not visible"), and `browser focus` on the X returns ok without moving focus (check `document.activeElement`); hover the rail or focus a card first. `browser viewport phone` keeps a fine pointer (`matchMedia('(hover:hover) and (pointer:fine)')` is `true`), so the always-visible X on touch devices cannot be driven: the CLI has no touch or coarse-pointer emulation. A dismissed rail hides every rail bullet's cards; restore it before them.- Responder **Continue with local setup** writes an `OPENHANDS_URL` secret; delete it so F14 counts stay clean.- The rail lists about 15 cards (proven ones, then conversation launchers such as `research-brief-writer`); `browser testids` reports only the first few, so check a card with `browser count 'testid=recommended-automation-rail-card-<id>'`. Any automation whose name matches a card (`Daily news digest`) hides that card, so avoid catalog names in fixtures.- The home draft (`oh:home-prompt-draft`) survives the no-LLM bullets; `browser type` appends to it, so `browser fill 'testid=chat-input' ''` before typing a slash command.diff --git a/.agents/skills/verify-openhands/references/feature-map/F04-conversation-list.md b/.agents/skills/verify-openhands/references/feature-map/F04-conversation-list.mdindex f20266896f..6a15c640b5 100644--- a/.agents/skills/verify-openhands/references/feature-map/F04-conversation-list.md+++ b/.agents/skills/verify-openhands/references/feature-map/F04-conversation-list.md@@ -7,9 +7,10 @@ header has a folder-plus picker that starts a conversation in a chosenworkspace and a **View** menu with layout presets, tag filters, **Moreoptions** (organize, sort, scope, archived, age cutoff, automation filter, rowmetadata) and **Delete all**. Grouped by workspace, folders collapse, preview-five rows and start new conversations in place. Preferences, pins, archives and-the workspace each conversation was started in live in this browser's-localStorage.+five rows and start new conversations in place. Preferences, pins and archives+live in this browser's localStorage; the workspace a conversation was started+in is also stamped on the server as its `workspace` tag, so grouping holds in+any browser.Source: `src/components/features/sidebar/sidebar-conversation-list.tsx`, `src/components/features/conversation-panel/` (panel, layouts menu, advanced options modal, new-thread picker, folder list/row, pinned section, active filter chips, confirm and tags modals, `conversation-card/`), `src/stores/conversation-panel-preferences-store.ts`, `src/stores/pinned-conversations-store.ts`, `src/stores/archived-conversations-store.ts`, `src/hooks/query/use-paginated-conversations.ts`, `src/hooks/use-download-conversation.ts`, `src/api/conversation-metadata-store.ts`.@@ -82,7 +83,7 @@ Preconditions:- **Start in a workspace (`F04.new-thread-picker`).** Run `control-openhands browser goto /`, `control-openhands browser click 'testid=conversation-panel-new-thread-picker'` and `control-openhands browser snapshot 'testid=conversation-panel-new-thread-popover'`: buttons `No workspace`, `qa-f04-folder`, `qa-f04-repo`, `+ Add Workspace`, `Manage Workspaces`. Then `control-openhands browser click 'testid=conversation-panel-new-thread-popover >> testid=launch-workspace >> has-text=qa-f04-repo' --expect-url '/conversations/'` returns `/conversations/<id>`, and `control-openhands conversation list` shows that id with `workspace` ending in `/workspace/qa-f04-repo`. From `/` again, `testid=launch-no-workspace` opens a new conversation whose row sits in the No workspace folder. `control-openhands browser click 'testid=manage-workspaces-button'` opens `testid=manage-workspaces-modal` (F03); Escape closes it. Immediately after creating that empty workspace conversation, `control-openhands conversation status <id>` reads `idle`; inspecting the `conversation-status-active` class shows a green dot with no animation class. Known failure (reproduced 2026-10-12, [#16752](https://github.com/OpenHands/OpenHands/issues/16752)): this empty idle row retains active green styling instead of the expected neutral state; the older report's flashing animation was not observed.- **Row opens its conversation (`F04.card`).** From `/` (`control-openhands browser goto /`; the previous bullet's launch left a conversation page, where `--expect-url '/conversations/'` would pass at once) run `control-openhands browser click 'testid=conversation-card >> has-text=pong' --expect-url '/conversations/'`, then `control-openhands browser attr 'testid=conversation-card >> has-text=pong' data-active` (`true`; other rows `false`). `control-openhands browser eval "document.querySelector('[data-testid=conversation-card][data-active=true]').closest('a').getAttribute('href')"` is `/conversations/<id>?backend=default-local`. The finished conversation shows `testid=conversation-status-check`, idle ones `conversation-status-active` (green), paused ones `conversation-status-paused` (grey). The reopened chat has its transcript back: `control-openhands browser wait 'testid=agent-message'`, then `control-openhands browser text 'testid=agent-message >> nth=-1'` is `pong` and `control-openhands browser text 'testid=user-message >> nth=-1'` is the prompt, with `control-openhands browser count 'testid=error-message-banner'` `0` (`browser screenshot --feature F04.card --name reopened`).- **Live status (`F04.card-live-status`).** Run `control-openhands conversation start --prompt "Reply with only the word ping. Do not run any tools."` (no `--wait`; one tiny model call), then `control-openhands browser wait 'testid=conversation-panel >> testid=conversation-status-working' --timeout 20000`: the active row (`Conversation <id5>`) shows the working indicator. After `control-openhands conversation wait <id> --timeout 180` reports `finished`, `control-openhands browser wait 'testid=conversation-panel >> [data-testid=conversation-card][data-active=true] >> testid=conversation-status-check' --timeout 60000` succeeds and `control-openhands browser text 'testid=conversation-panel >> [data-testid=conversation-card][data-active=true] >> testid=conversation-card-title'` is the generated title (it varies: `✅ Reply with only ping`, `✅ Reply only ping, no tools`). The row lags the conversation by up to about 40 s (list refresh).-- **Group by workspace (`F04.grouped-folders`).** Run `control-openhands browser click 'testid=conversation-layouts-toggle'`, `control-openhands browser click 'testid=layout-preset-by-workspace'`, then `control-openhands browser snapshot 'testid=conversation-panel'`. Regions `No workspace`, `qa-f04-repo` and `qa-f04-folder` each list their own conversations; a workspace without conversations still gets a folder. Second view in another browser: `control-openhands browser reset`, `control-openhands onboard --skip`, choose By workspace again; the workspace conversations must still be in their folders (currently they all fall into No workspace, a known failure).+- **Group by workspace (`F04.grouped-folders`).** Run `control-openhands browser click 'testid=conversation-layouts-toggle'`, `control-openhands browser click 'testid=layout-preset-by-workspace'`, then `control-openhands browser snapshot 'testid=conversation-panel'`. Regions `No workspace`, `qa-f04-repo` and `qa-f04-folder` each list their own conversations; a workspace without conversations still gets a folder. A conversation started in a workspace carries that path as a reserved server tag: `control-openhands api GET /api/conversations/<id> --pick tags` has `workspace` `<run>/workspace/qa-f04-repo` next to `clientsource` (no chip shows it). The fresh-browser second view is the last bullet before Delete all.- **Collapse one folder (`F04.folder-collapse`).** Run `control-openhands browser click '[data-testid^="thread-folder-drag-ws-"][data-testid$="qa-f04-repo"]'`, `control-openhands browser attr '[data-testid^="thread-folder-drag-ws-"][data-testid$="qa-f04-repo"]' aria-expanded` (`false`) and `control-openhands browser count '[data-testid^="thread-folder-ws-"][data-testid$="qa-f04-repo"] >> testid=conversation-card'` (`0`). Click again: `true` and the cards return.- **Collapse all (`F04.collapse-all`).** Run `control-openhands browser click 'testid=conversations-header-toggle'`, `control-openhands browser attr 'testid=conversations-header-toggle' aria-expanded` (`false`) and `control-openhands browser count 'testid=conversation-panel-list-scroll >> testid=conversation-card'` (`0`). Click again: `true` and every card is back.- **New conversation in a folder (`F04.folder-new-conversation`).** Run `control-openhands browser goto /`, then `control-openhands browser click 'role=button[name="Start new conversation in qa-f04-folder"]' --expect-url '/conversations/'` (returns the new `/conversations/<id>`), then `control-openhands browser count '[data-testid^="thread-folder-ws-"][data-testid$="qa-f04-folder"] >> testid=conversation-card'`; it grows by one. Repeating `browser goto /` plus the click for `qa-f04-repo` 18 times arranges the data for the next two bullets.@@ -104,7 +105,7 @@ Preconditions:- **Pinned preview (`F04.pinned-preview-more`).** Pin six rows (`browser scroll`, `browser hover`, then click each row's `[data-testid^=conversation-pin-toggle-]`). `control-openhands browser count 'testid=conversation-panel-pinned-section >> testid=conversation-card'` is `5` and `control-openhands browser text 'testid=conversation-panel-pinned-view-more'` is `More`; `control-openhands browser click 'testid=conversation-panel-pinned-view-more'` makes the count `6` and the label `Less`; click again: `5`. Unpin all six with `browser hover` and a pin click on `'testid=conversation-panel-pinned-section >> testid=conversation-card >> nth=0'`, repeated.- **Edit tags (`F04.edit-tags`).** Open ⋮ on a row (hover, `testid=ellipsis-button`) and run `control-openhands browser click 'testid=context-menu >> testid=edit-tags-button'`. Then: `control-openhands browser fill 'testid=new-tag-key-input' 'Bad Key'` and `control-openhands browser press Enter --selector 'testid=new-tag-key-input'` → `control-openhands browser text 'testid=edit-tags-error'` is `Tag names may only contain lowercase letters and numbers` (it uses the `feedback-error` colour, #18330: `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=edit-tags-error]')).color"` is `rgb(231, 106, 94)` on OpenHands-Neutral and `rgb(192, 40, 45)` on Light+, readable on the light dialog; switch themes as in `F16.color-theme` and restore OpenHands-Neutral); key `title` with `testid=add-tag-button` → `This tag name is reserved`; key `qaenv`, `browser fill 'testid=new-tag-value-input' one`, Enter → `browser count 'testid=edit-tag-row-qaenv'` is `1`; key `qaenv` again → `This tag name is already in use`; key `qalong` with a 257-character value → `Tag values must be 256 characters or fewer`; key `qabare` with an empty value left in the inputs, then `control-openhands browser click 'testid=edit-conversation-tags-modal >> testid=confirm-button'`. `control-openhands browser toasts` shows `Tags updated`; after `browser reload` the row shows chips `Qabare` and `Qaenv: one` (`conversation-card-tag-chip`), and `control-openhands api GET '/api/conversations/search?limit=3'` has tags `qaenv: one`, `qabare: ""`.- **Tag overflow (`F04.tag-overflow`).** On the tagged row add three more tags in Edit tags (keys `qaregion`, `qastage`, `qaowner`, values `value-for-<key>`, Enter after each) and save. After `browser reload` the row shows `Qabare` and `Qaenv: one` plus `control-openhands browser text 'testid=conversation-card >> has-text=Conversation <id5> >> testid=conversation-card-tag-overflow'` `+3` (aria-label `Show 3 more tags`). Clicking it keeps the URL, sets `aria-expanded` `true`, and `control-openhands browser text 'testid=conversation-card-tag-overflow-popover'` lists `Qaowner value-for-qaowner`, `Qaregion …`, `Qastage …` (3 `conversation-card-tag-overflow-row`); `control-openhands browser press Escape` closes it (count `0`).-- **Remove tags (`F04.edit-tags-remove`).** Reopen Edit tags on that row and run `control-openhands browser click 'testid=remove-tag-qaregion'` (likewise `remove-tag-qastage`, `remove-tag-qaowner`); `control-openhands browser count 'testid=edit-tags-rows >> [data-testid^=edit-tag-row-]'` is `2`. Save with `testid=edit-conversation-tags-modal >> testid=confirm-button` (`Tags updated`); after `browser reload` the `+3` chip is gone and `control-openhands api GET '/api/conversations/search?limit=1' --pick items.0.tags` is `clientsource`, `qabare`, `qaenv` only.+- **Remove tags (`F04.edit-tags-remove`).** Reopen Edit tags on that row and run `control-openhands browser click 'testid=remove-tag-qaregion'` (likewise `remove-tag-qastage`, `remove-tag-qaowner`); `control-openhands browser count 'testid=edit-tags-rows >> [data-testid^=edit-tag-row-]'` is `2`. Save with `testid=edit-conversation-tags-modal >> testid=confirm-button` (`Tags updated`); after `browser reload` the `+3` chip is gone and `control-openhands api GET /api/conversations/<tagged-id> --pick tags` is `clientsource`, `qabare`, `qaenv` and, on a workspace conversation such as this recipe's qa-f04-repo row, the reserved `workspace` (kept by every save) only.- **Tag filter (`F04.tag-filter`).** After Edit tags (above) gave one row `qaenv=one`, run `control-openhands browser click 'testid=conversation-layouts-toggle'`, `control-openhands browser click 'testid=tag-filters-section'`, `control-openhands browser click 'testid=tag-facet-row-qaenv=one'` (`aria-checked` `true`; the menu stays open), `control-openhands browser press Escape`, then `control-openhands browser count 'testid=conversation-panel-list-scroll >> testid=conversation-card'`: only the tagged row plus pinned rows. The count is the same after `browser reload`.- **Filter chips (`F04.filter-chips`).** With the facet active, `control-openhands browser text 'testid=conversation-active-tag-filters'` is `qaenv=one` and `Clear all`. `control-openhands browser click 'testid=active-tag-filter-qaenv=one'` removes it (`browser count 'testid=conversation-active-tag-filters'` is `0`). Select `tag-facet-row-qaenv=one` and `tag-facet-row-qabare=` again, then `control-openhands browser click 'testid=clear-tag-filters'`: the strip disappears and the full list returns.- **Automation setup tag (`F04.automation-setup-tag`).** Arrange one setup conversation through the setup page (F22): `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'`, `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`, then `control-openhands conversation start --stay --prompt 'Reply with only the word ok. Do not run any tools.' --wait --timeout 180` (one tiny model call; note `<setup-id>`). `control-openhands api GET /api/conversations/<setup-id> --pick tags` has `automationsetup: draft`. In the sidebar, `control-openhands browser text 'testid=conversation-card >> has-text=Reply >> testid=conversation-card-tag-chip'` is `Automation setup: Draft` (the first chip; the internal draft-id keys stay hidden). Run `control-openhands browser click 'testid=conversation-layouts-toggle'`, `control-openhands browser click 'testid=tag-filters-section'`: the facet `testid=tag-facet-row-automationsetup=draft` reads `Automation setup: Draft`; click it, `control-openhands browser press Escape`, and `control-openhands browser text 'testid=conversation-active-tag-filters'` is `Automation setup: Draft` and `Clear all`, with only setup conversations (and pinned rows) left; `control-openhands browser click 'testid=active-tag-filter-automationsetup=draft'` removes it. After `control-openhands browser mouse-click 900 900`, `control-openhands browser tooltip 'testid=conversation-card >> has-text=Reply'` lists `Automation setup` and the raw value `draft` (`browser screenshot --feature F04.automation-setup-tag --name facet`). End with `control-openhands browser goto /`; the setup send leaves no Home draft, and opening another conversation from the sidebar leaves that conversation's tags alone (see Gotchas).@@ -116,7 +117,7 @@ Preconditions:- **Delete (`F04.delete`).** Open a conversation from its row, open ⋮ and run `control-openhands browser click 'testid=context-menu >> testid=delete-button'`; the dialog reads `Confirm Delete` and `Are you sure you want to delete the "<title>" conversation? This action cannot be undone.` with `cancel-button` and `confirm-button`. `control-openhands browser click 'role=dialog >> testid=cancel-button'` keeps the row. Repeat and `control-openhands browser click 'role=dialog >> testid=confirm-button' --expect-url '/conversations(\?|$)'`; after `browser reload` the row count is `0` and `control-openhands api GET /api/conversations/<id>` is `404`.- **Phone (`F04.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /`, `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'`, then `control-openhands browser bbox 'testid=sidebar-mobile-drawer >> testid=conversation-panel'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F04.phone --name drawer`. Hover a row in the drawer and click its `testid=ellipsis-button`; `control-openhands browser bbox 'testid=context-menu'` is inside the viewport. `control-openhands browser mouse-click 380 20` closes the menu (`browser count 'testid=context-menu'` `0`) and the drawer (`control-openhands browser wait 'testid=sidebar-mobile-drawer' --state hidden --timeout 5000`; an immediate `visible` still reads `true` during the slide-out). Reopen the drawer and `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=conversation-card >> has-text=QA_F04 renamed' --expect-url '/conversations/'`; `control-openhands browser wait 'testid=sidebar-mobile-drawer' --state hidden --timeout 5000` succeeds. Return with `control-openhands browser viewport desktop`.- **A later-page pin (`F04.pin`).** With more than 20 conversations as the pagination precondition already arranges, choose Recent, reload, click `role=button[name="Load more"]`, and pin a conversation from the newly loaded rows (`browser hover` its `conversation-card`, then click its `[data-testid^=conversation-pin-toggle-]`). Read the pinned section, then `control-openhands browser reload` and `control-openhands browser count 'testid=conversation-panel-pinned-section'`. Known failure (reproduced 2026-10-12, [#16399](https://github.com/OpenHands/OpenHands/issues/16399)): the later-page pin is pruned on first-page reload even though the conversation still exists. Tested with 23 empty conversations and one older pin.-- **Fresh-browser grouping (`F04.grouped-folders`).** Run this after the other list checks: select By workspace, record its groups, `control-openhands browser reset`, `control-openhands onboard --skip`, then `control-openhands browser click 'testid=conversation-layouts-toggle'` and `control-openhands browser click 'testid=layout-preset-by-workspace'`. Compare `browser snapshot 'testid=conversation-panel'` with `conversation status <id>`. Known failure (reproduced 2026-10-12, [#17386](https://github.com/OpenHands/OpenHands/issues/17386)): existing conversations move to No workspace although their server working directory still matches the registered workspace. The reset clears browser metadata; do not use it before checks that depend on the original browser's grouping.+- **Fresh-browser grouping (`F04.grouped-folders`).** Run this after the other list checks: select By workspace, record its groups, `control-openhands browser reset`, `control-openhands onboard --skip`, then `control-openhands browser click 'testid=conversation-layouts-toggle'` and `control-openhands browser click 'testid=layout-preset-by-workspace'`. Compare `browser snapshot 'testid=conversation-panel'` with `conversation status <id>`: every workspace conversation is still in its own folder (`qa-f04-repo`, `qa-f04-folder`) and only the no-workspace ones are under No workspace, although `control-openhands browser storage` no longer lists `openhands-agent-server-conversation-metadata` (`browser screenshot --feature F04.grouped-folders --name fresh-browser`). The reset clears pins, archives and other browser metadata; do not use it before checks that depend on them.- **Delete all (`F04.delete-all`).** Run `control-openhands browser click 'testid=conversation-layouts-toggle'`, `control-openhands browser click 'testid=delete-all-conversations'` and `control-openhands browser snapshot 'role=dialog'`: `Delete all conversations`, `Are you sure you want to delete N conversations?`. Cancel (`role=dialog >> testid=cancel-button`) keeps every row. Confirm with `control-openhands browser click 'role=dialog >> testid=confirm-button'` and `control-openhands browser wait 'role=dialog' --state detached --timeout 30000`; `control-openhands api GET '/api/conversations/search?limit=100'` has no items, Delete all is disabled again, and in By date mode the empty state returns. Expected for every conversation; today N and the deletion cover only the loaded pages (22 conversations, 20 loaded: 2 remain).- **Collapsed rail (`F04.collapsed-rail`).** Run `control-openhands browser click 'testid=sidebar-collapse-toggle'`, then `control-openhands browser count 'testid=conversation-panel'` (`0`). Expand with `control-openhands browser click 'aside[data-collapsed]'` (see F02); the count is `1`.- **Cloud picker and start tasks (`F04.cloud-picker`, `F04.start-task-cards`).** Blocked without an OpenHands Cloud backend and git provider. On Cloud, the same `conversation-panel-new-thread-picker` opens `cloud-provider-tabs`, `cloud-repo-search-input` (`Search repositories`) and `launch-repository` rows; start tasks render `testid=start-task-card` linking to `/conversations/task-<id>`.@@ -125,8 +126,8 @@ Preconditions:## Gotchas-- A plain `--expect-url '/conversations/'` returns at once when the browser is already on a conversation page. Start creation clicks from `/` (`browser goto /`), and never navigate away before the click returns: leaving early skips the client-side metadata write, so the new conversation lands in **No workspace** although the server has its working directory.-- Workspace grouping, pins, archives and preferences live in this browser's localStorage (`openhands-agent-server-conversation-metadata`, `pinned-conversations`, `archived-conversations`, `conversation-panel-preferences`). `browser reset` loses them: archived rows reappear and every conversation moves to No workspace.+- A plain `--expect-url '/conversations/'` returns at once when the browser is already on a conversation page. Start creation clicks from `/` (`browser goto /`). Navigating away right after the click still groups the new conversation in its workspace: the workspace travels as a server tag in the create request.+- Pins, archives and preferences live in this browser's localStorage (`pinned-conversations`, `archived-conversations`, `conversation-panel-preferences`; `openhands-agent-server-conversation-metadata` caches per-conversation metadata). `browser reset` loses them: archived rows reappear. Workspace grouping survives it, from the server `workspace` tag, or for a Local Repo conversation from a working directory that equals a registered workspace path.- Folder test ids embed the sanitized absolute workspace path (`thread-folder-ws--tmp-...-qa-f04-repo`). Select with `[data-testid^="thread-folder-ws-"][data-testid$="<folder>"]` or by the `+` button's accessible name.- Row links carry `?backend=default-local`; anchor URL regexes without `$`.- Hover and ⋮ need the row on screen: `browser scroll '<row>'` first, and `browser hover` before clicking `ellipsis-button` or the pin.diff --git a/.agents/skills/verify-openhands/references/feature-map/F05-composer.md b/.agents/skills/verify-openhands/references/feature-map/F05-composer.mdindex 46080890b9..b7a16edab9 100644--- a/.agents/skills/verify-openhands/references/feature-map/F05-composer.md+++ b/.agents/skills/verify-openhands/references/feature-map/F05-composer.md@@ -17,12 +17,13 @@ Source: `src/components/features/chat/`, `src/components/features/controls/tools- `F05.composer-resize`: the field grows with its content and shrinks when cleared; an invisible top-edge grip (resize cursor, no line drawn) lets the user drag the height.- `F05.plus-menu`: `+` opens a tools menu (Macros, Show skills/hooks/agent tools, Add Files and Images; Git tools in a repository); a macro fills the composer without sending.- `F05.agent-profile-switch`: before a conversation starts, `+` > Switch agent profile lists agent profiles and links to Manage agent profiles; it is absent in a started conversation.+- `F05.agent-profile-pill`: before a conversation starts, the composer shows an agent-profile pill (robot icon, `Switch agent profile: <name>`) next to the model pill; it opens the same profile list, closes on an outside click and on Escape (focus back on the pill; Known failure (reproduced 2026-10-22, [#18808](https://github.com/OpenHands/OpenHands/issues/18808)): Escape leaves it open, and the model pill's popover the same), is the first control to drop when the row is narrow (hidden at 390 px and 320 px) and is absent in a started conversation. Source: `src/components/features/chat/components/chat-input-profile-picker.tsx`, `src/components/features/chat/components/chat-input-actions.tsx`.- `F05.attach-files`: picked files show as chips and images as thumbnails (with an Upload-as-file toggle), each removable; sent files land in the workspace and are named in the message.- `F05.attach-size-limit`: a file over 3 MB, or a selection over 3 MB in total, is refused with an error toast.- `F05.image-only-send`: an attached image alone, with an empty field, enables Send; the sent user message shows the thumbnail and carries the picture as an embedded data URL.- `F05.paste`: pasted clipboard text is inserted as plain text; pasted images and dropped files attach.- `F05.dictation`: the mic button dictates into the composer; without a microphone it toasts a failure and resets.-- `F05.llm-profile-picker`: the model pill switches the running conversation's LLM profile; the next reply uses the new model.+- `F05.llm-profile-picker`: the model pill switches the running conversation's LLM profile; the next reply uses the new model. On Home the pill picks the LLM profile the next conversation launches with, on a local backend and on a Cloud backend (where it lists the organization's LLM profiles, the pick persists after a reload, and the conversation started from Home runs on it).- `F05.profile-identity`: the pill names the profile the conversation was started with, also after a reload and when another profile uses the same model; changing the default profile in Settings changes the home pill, not an existing conversation's.- `F05.overflow-menu`: at very narrow widths the model pill collapses into a "More input actions" menu whose Model submenu fits the viewport and switches the profile; Escape closes the menu and returns focus to its trigger.- `F05.context-window-meter`: the ring shows context use; its popover opens the Usage tab and compacts the context. Above 70% the popover's percentage and bar fill use the theme's warning color and above 90% its error color, on a raised surface with a bordered track, readable in light and dark themes.@@ -42,6 +43,7 @@ Source: `src/components/features/chat/`, `src/components/features/controls/tools- Home page composer (`/`) and the composer of any conversation (`/conversations/<id>`, reached from the sidebar conversation list or by URL).- `+` (More actions, `chat-plus-button`) inside the composer opens the tools menu; its last item **Add Files and Images** opens the native file picker (`upload-image-input`).+- Agent-profile pill (`chat-input-agent-profile`) next to `+` before a conversation starts (Home, or an empty new conversation), when the backend has agent profiles.- Model pill (`chat-input-llm-profile`) next to `+`; at 320 px it moves into the vertical-dots **More input actions** button. On the home page it names the default profile (the one Settings → LLM marks as default); in a conversation it names the profile that conversation runs on.- Right side of the composer: status text with Stop/Play, context ring (`context-window-meter`), mic (`chat-dictation-button`), Send (`submit-button`).- Typing `/` at the start of a word opens the slash menu (`slash-command-menu`); `/plan` and `/code` are the only mode switches on the local backend (the Code/Plan pill is Cloud only).@@ -64,6 +66,7 @@ Preconditions:- **Tools menu and macros (`F05.plus-menu`).** On `/conversations/<id>` run `control-openhands browser click 'testid=chat-plus-button'` and `control-openhands browser testids 'testid=tools-context-menu'`: `macros-button`, `show-skills-button`, `show-hooks-button`, `show-agent-tools-button`, `add-files-and-images-button` (no `git-tools-button` outside a repository). `control-openhands browser hover 'testid=macros-button'` shows `macros-submenu` with `increase-test-coverage-button`, `fix-readme-button`, `auto-merge-prs-button`, `clean-dependencies-button`. `control-openhands browser click 'testid=fix-readme-button'` closes the menu and fills the composer with `Please look at the README and make the following improvements…`; the `testid=user-message` count is unchanged. Clear with `control-openhands browser fill 'testid=chat-input' ''`. Open the menu again and `control-openhands browser press Escape`: `browser count 'testid=tools-context-menu'` is `0` and `control-openhands browser eval "document.activeElement.dataset.testid"` is `chat-plus-button`.- **Show dialogs (`F05.plus-menu`).** For each of `show-skills-button`, `show-hooks-button`, `show-agent-tools-button`: `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser click 'testid=show-hooks-button'` (etc.), `control-openhands browser text 'role=dialog'` (starts `Available Skills`, `Available Hooks`, `Agent Tools & Metadata`), then `control-openhands browser press Escape` and `control-openhands browser count 'role=dialog'`: `0`.- **Agent profile before start (`F05.agent-profile-switch`).** Run `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser hover 'testid=switch-agent-profile-button'` and `control-openhands browser text 'testid=agent-profile-submenu'`: `AVAILABLE PROFILES`, `default` (checked; `chat-input-agent-profile-option-default`), `Manage agent profiles`. `control-openhands browser click 'testid=agent-profile-submenu >> text=Manage agent profiles' --expect-url 'settings/agents'` lands on `/settings/agents`. In a started conversation (`control-openhands browser goto /conversations/<id>`) `control-openhands browser click 'testid=chat-plus-button'` then `control-openhands browser count 'testid=switch-agent-profile-button'` is `0`.+- **Agent-profile pill (`F05.agent-profile-pill`).** Run `control-openhands browser goto /`, `control-openhands browser wait 'testid=chat-input-agent-profile'`, then `control-openhands browser attr 'testid=chat-input-agent-profile' aria-label` (`Switch agent profile: default`) and `control-openhands browser text 'testid=chat-input-agent-profile'` (`default`). `control-openhands browser click 'testid=chat-input-agent-profile'` sets `aria-expanded` `true` and `control-openhands browser text 'testid=chat-input-agent-profile-popover'` reads `AVAILABLE PROFILES`, `default` (`chat-input-agent-profile-option-default`), `Manage agent profiles` (`browser screenshot --feature F05.agent-profile-pill --name home-pill-open`). `control-openhands browser mouse-click 900 900` closes it (`browser count 'testid=chat-input-agent-profile-popover'` `0`). Reopen it and `control-openhands browser press Escape`: expected count `0` and `browser attr 'testid=chat-input-agent-profile' aria-expanded` `false`, the same for `testid=chat-input-llm-profile`. Known failure (reproduced 2026-10-22, [#18808](https://github.com/OpenHands/OpenHands/issues/18808)): the count stays `1` and `aria-expanded` `true` for both pills; close them with `browser mouse-click 900 900`. Reopen it and `control-openhands browser click 'testid=chat-input-agent-profile-popover >> text=Manage agent profiles' --expect-url 'settings/agents'` lands on `/settings/agents`. At `browser viewport phone` and `narrow` on `/`, `control-openhands browser visible 'testid=chat-input-agent-profile'` is `false` while `browser visible 'testid=home-chat-launcher >> testid=chat-input-llm-profile'` is `true` (the `+` menu still switches the profile); return with `browser viewport desktop`. In a started conversation (`browser goto /conversations/<id>`, `browser wait 'testid=chat-input'`) `control-openhands browser count 'testid=chat-input-agent-profile'` is `0`.- **Attach a file and an image (`F05.attach-files`).** On `/conversations/<id>` run `control-openhands browser upload 'testid=upload-image-input' "$OH_VERIFY_RUN/evidence/_fixtures/qa-image.png" "$OH_VERIFY_RUN/evidence/_fixtures/qa-note.txt"` and `control-openhands browser screenshot 'testid=interactive-chat-box' --feature F05.attach-files --name attached`: a `qa-note.txt TXT` chip and a thumbnail. `control-openhands browser count 'testid=interactive-chat-box >> alt=qa-image.png'` is `1`. `control-openhands browser click 'role=button[name="Upload as file"]'` turns the image toggle into `Do not upload as file` (`control-openhands browser attr 'role=button[name="Do not upload as file"]' aria-pressed` is `true`); click it again to restore. The remove buttons are named: `control-openhands browser snapshot 'testid=interactive-chat-box'` lists `button "Remove qa-note.txt"` and `button "Remove image"`. Remove the image with `control-openhands browser click 'testid=interactive-chat-box >> role=button[name="Remove image"]'`; the `alt=qa-image.png` count is `0`. Then `control-openhands browser type 'testid=chat-input' 'Reply with only the content of the attached file.'`, `control-openhands browser press Enter --selector 'testid=chat-input'`, wait as in Type and send, and `control-openhands browser text 'testid=user-message >> nth=-1'`: it ends `NEW FILES ADDED: qa-note.txt`. `control-openhands conversation events <id> --kinds MessageEvent --last 3` shows the reply `QA attachment 7731`, and `qa-note.txt` sits in the conversation `workspace` printed by `control-openhands conversation status <id>`. Attachments added on Home before the conversation exists upload after it is created: with no workspace or a Local Repo workspace they land in it (`browser network --filter upload` shows `200`). Expected: the same in New Worktree mode. Known failure (reproduced 2026-10-16, SDK 1.55.1, [#18421](https://github.com/OpenHands/OpenHands/issues/18421)): run `control-openhands fixture git-repo --name qa-repo`, `control-openhands browser goto /`, `control-openhands workspace open qa-repo`, `control-openhands browser click 'testid=workspace-mode-selector'`, `control-openhands browser click 'testid=workspace-mode-selector-option-new_worktree'`, `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser upload-via 'testid=add-files-and-images-button' "$OH_VERIFY_RUN/evidence/_fixtures/qa-note.txt"`, type `Reply with only the word received. Do not run any tools.` and `control-openhands browser click 'testid=home-chat-launcher >> testid=submit-button' --expect-url '/conversations/[^/]+$'`: `control-openhands browser toasts --history` shows `HTTP request failed (422 Unprocessable Entity): {"detail":"path must be inside the conversation workspace"}` and `browser network --filter upload` a `422` for `?path=<run>/workspace/qa-repo/qa-note.txt`, while `conversation status <id>` reports the worktree `/tmp/conversation-worktrees/<id>/qa-repo`; the message is sent without the file. The same file attached inside that conversation afterwards uploads to the worktree (`200`). Restore with `testid=workspace-mode-selector-option-local_repo`.```sh known-bug F05.attach-files home-new-worktreecontrol-openhands browser viewport desktopdiff --git a/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md b/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.mdindex c88af69b51..4d9a6879df 100644--- a/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md+++ b/.agents/skills/verify-openhands/references/feature-map/F06-agent-activity.md@@ -23,7 +23,8 @@ Source: `src/components/features/chat/chat-interface.tsx`, `src/components/conve- `F06.clock-skew-send`: a message sent while the browser clock runs ahead of the server is still confirmed by the server echo: no lingering "Sending..." and no Retry.- `F06.failed-send-persists`: a failed send keeps its Retry row after the backend returns and after leaving and reopening the conversation from the sidebar; sending the same text again succeeds without clearing it, and Retry then delivers that attempt exactly once.- `F06.markdown-rendering`: agent replies render markdown: tables inside a horizontal scroller with edge fades, fenced code (highlighted tokens stay inline, so a TOML `[build-system]` header is one line), inline code, and links that are always underlined in the theme's link colour.-- `F06.streamed-reply`: a streamed agent reply types out at a steady reading pace (about 60 characters a second, faster only when it would otherwise trail the model by more than about 10 seconds), each new word fading in; it keeps typing to the end after the model has finished, then settles into the finished markdown (tables, code blocks) with no per-word spans. While it types, the transcript stays pinned to the bottom; scrolling up stops the follow without pulling the user back, and returning to the bottom (button or scroll) resumes it. With reduced motion the words appear without the fade.+- `F06.markdown-alerts`: GitHub-style alerts in a message (`> [!NOTE]`, `[!TIP]`, `[!IMPORTANT]`, `[!WARNING]`, `[!CAUTION]`) render as panels with an icon, a bold label, a coloured left border and a faint tint in the alert's theme colour, with the body in the normal text colour; the label is readable (at least 4.5:1) in dark and light themes. Source: `src/components/features/markdown/blockquote.tsx`.+- `F06.streamed-reply`: a streamed agent reply types out at a reading pace of about 60 characters a second, or faster to stay about half a second behind a fast model, each new word fading in; it catches up within about a second or two after the model has finished, then settles into the finished markdown (tables, code blocks) with no per-word spans. While it types, the transcript stays pinned to the bottom; scrolling up stops the follow without pulling the user back, and returning to the bottom (button or scroll) resumes it. With reduced motion the words appear without the fade.- `F06.workspace-path-links`: a workspace file path in an agent reply, and the path chip of a file-editor card, open that file in the Files tab.- `F06.thinking`: model reasoning appears as a collapsed "Thinking" row that expands.- `F06.event-groups`: consecutive tool calls collapse into "N actions completed" (or "k/N actions completed" with a spinner while one is pending); expanding lists one titled row per action.@@ -78,7 +79,8 @@ Preconditions:- **Markdown file preview (`F06.markdown-file-preview`).** Run `control-openhands conversation start --prompt "Use the file_editor tool to create the file qa_notes.md containing a heading '# QA Notes' and one bullet '- first'. Do nothing else and reply DONE." --wait --timeout 150`. Without any click, `control-openhands browser text 'testid=markdown-file-preview'` reads `QA Notes first qa_notes.md View` (rendered heading and bullet, then the file name) and `count 'testid=event-group'` is `0`. `control-openhands browser count 'testid=file-quick-row-item-qa_notes.md'` is `0`; `control-openhands browser click 'testid=markdown-file-preview-view'` makes it `1` and `control-openhands browser text 'testid=file-content-viewer-markdown'` reads `QA Notes first`. Then `control-openhands browser goto /conversations/<stream id>` to return to the stream conversation for the next bullets.- **Timestamps (`F06.timestamps`).** Run `control-openhands browser tooltip 'testid=user-message'`, `control-openhands browser hover 'testid=chat-input'`, `control-openhands browser click 'testid=event-group-toggle'` (if the group is collapsed) and `control-openhands browser tooltip 'testid=event-group-content >> testid=generic-event-message-title >> nth=0'`. Both return the local date and time, e.g. `Oct 4, 2026, 11:08 PM`; the group toggle itself has no tooltip.- **Copy a message and a code block (`F06.message-copy`, `F06.code-block-copy`).** On the same page run `control-openhands browser clipboard --write qa-empty`, `control-openhands browser hover 'testid=agent-message >> nth=0'` and `control-openhands browser click 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0'` (the bubble's button; the code block has a second, hidden `copy-to-clipboard`). `control-openhands browser attr 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0' aria-label` is `Copied to clipboard`, and `Copy to clipboard` again about 2 s later; `control-openhands browser clipboard` starts with `| Step | Result |` and ends with the fenced `print(1)` block. Then `control-openhands browser hover 'testid=agent-message >> pre'`, `control-openhands browser click 'testid=agent-message >> pre >> testid=copy-to-clipboard'` and `control-openhands browser clipboard` is exactly `print(1)`.-- **Streamed reply (`F06.streamed-reply`).** From `/` run `control-openhands browser record start --feature F06.streamed-reply --name reveal`, fill `testid=chat-input` with `Write a markdown answer of about 300 words explaining Python virtual environments. Include a level-2 heading, a bulleted list of 4 items, a python code block of about 6 lines, and a table with 3 columns and 3 rows. Do not run any tools.` and `control-openhands browser click 'testid=submit-button' --expect-url '/conversations/'`. While it types, `control-openhands browser count 'testid=agent-message >> .stream-word-fade'` is at most about 8 (0 while the reveal waits for the model), `browser count 'testid=agent-message >> pre >> [data-stream-word]'` is `0`, and the distance from the bottom (`browser eval "(()=>{const e=document.querySelector('[data-testid=chat-scroll-container]');return Math.round(e.scrollHeight-e.scrollTop-e.clientHeight)})()"`) stays `0` to `1`. The bubble keeps growing after `conversation events <id> --kinds MessageEvent` already lists the agent reply (about 20 seconds for 1,600 characters). Once done, `browser count 'testid=agent-message >> [data-stream-word]'` is `0`, the message holds one `table` and one `pre`, and its text is unchanged after `browser reload`. In a longer follow-up (about 800 words), run `control-openhands browser scroll 'testid=chat-scroll-container' --by -700` mid-stream: the distance grows as the text grows, `count 'testid=scroll-to-bottom'` is `1`, and `browser click 'testid=scroll-to-bottom'` returns it to `0` while the reveal continues. Finish with `control-openhands browser record stop --gif`. For the actual reduced-motion branch, record the current `control-openhands browser media` preference, set `browser media --reduced-motion reduce`, and send a confined reply-only prompt. During real streaming, assert `matchMedia("(prefers-reduced-motion: reduce)").matches` is true and the actual `.stream-word-fade` computed animation is `none` / `0s`; capture a screenshot. Set `browser media --reduced-motion no-preference` for a real normal control (`stream-word-fade`, `0.28s`), await settled markup and restore the original preference in a finally step. This was driven with real provider replies 2026-10-17; no synthetic streaming events or CSS-only pass.+- **Streamed reply (`F06.streamed-reply`).** From `/` run `control-openhands browser record start --feature F06.streamed-reply --name reveal`, fill `testid=chat-input` with `Write a markdown answer of about 300 words explaining Python virtual environments. Include a level-2 heading, a bulleted list of 4 items, a python code block of about 6 lines, and a table with 3 columns and 3 rows. Do not run any tools.` and `control-openhands browser click 'testid=submit-button' --expect-url '/conversations/'`. While it types, `control-openhands browser count 'testid=agent-message >> .stream-word-fade'` stays small (0 while the reveal waits for the model; with deepseek-flash about 10 to 50, because the reveal keeps pace with the model at several hundred characters a second), `browser count 'testid=agent-message >> pre >> [data-stream-word]'` is `0`, and the distance from the bottom (`browser eval "(()=>{const e=document.querySelector('[data-testid=chat-scroll-container]');return Math.round(e.scrollHeight-e.scrollTop-e.clientHeight)})()"`) stays `0` to `1` (a single sample can briefly read about 60 while a block lands). The reveal does not trail far behind the model: sample the last `agent-message`'s `innerText.length` and `[data-stream-word]` count every 250 ms with one `browser eval` promise, then compare with the agent row's `ts` from `conversation events <id> --kinds MessageEvent --last 1`: the bubble stops growing and drops its word spans about 2 seconds after that timestamp (1.7 s for about 2,000 and about 5,200 characters with deepseek-flash, including delivery and sampling). Once done, `browser count 'testid=agent-message >> [data-stream-word]'` is `0`, the message holds one `table` and one `pre`, and its text is unchanged after `browser reload`. In a longer follow-up (about 800 words), run `control-openhands browser scroll 'testid=chat-scroll-container' --by -700` mid-stream: the distance grows as the text grows, `count 'testid=scroll-to-bottom'` is `1`, and `browser click 'testid=scroll-to-bottom'` returns it to `0` while the reveal continues. Finish with `control-openhands browser record stop --gif`. For the actual reduced-motion branch, record the current `control-openhands browser media` preference, set `browser media --reduced-motion reduce`, and send a confined reply-only prompt. During real streaming, assert `matchMedia("(prefers-reduced-motion: reduce)").matches` is true and the actual `.stream-word-fade` computed animation is `none` / `0s`; capture a screenshot. Set `browser media --reduced-motion no-preference` for a real normal control (`stream-word-fade`, `0.28s`), await settled markup and restore the original preference in a finally step. This was driven with real provider replies 2026-10-17; no synthetic streaming events or CSS-only pass.+- **Alerts (`F06.markdown-alerts`).** Run `control-openhands conversation start --prompt "$P" --wait --timeout 180` with `P=$'Reply with exactly the following markdown, copied verbatim, and nothing else. Do not run any tools.\n\n> [!NOTE]\n> Note body with **bold** and `code`.\n\n> [!TIP]\n> Tip body.\n\n> [!IMPORTANT]\n> Important body.\n\n> [!WARNING]\n> Warning body.\n\n> [!CAUTION]\n> Caution body.'` (the user bubble renders the same panels), then `control-openhands browser wait 'testid=markdown-alert-caution'`. The agent message has `markdown-alert-note`, `-tip`, `-important`, `-warning` and `-caution`, each labelled `Note`, `Tip`, `Important`, `Warning`, `Caution`. In each panel the label, its icon `svg` and the left border share one computed colour, and the body paragraph uses the text colour: on OpenHands-Neutral `rgb(142, 197, 255)`, `rgb(94, 233, 181)`, `rgb(216, 180, 254)`, `rgb(255, 224, 42)`, `rgb(255, 179, 191)` with a white body (labels about 9.8:1 to 13.5:1 against the page); on Light+ (switch as in `F16.color-theme`) `rgb(0, 90, 153)`, `rgb(8, 99, 63)`, `rgb(126, 34, 206)`, `rgb(109, 76, 0)`, `rgb(170, 32, 37)` with a `rgb(31, 31, 31)` body (labels about 7.0:1 to 7.8:1 against the white page). Read them with `browser eval` of `getComputedStyle(...).color` on `[data-testid=markdown-alert-<type>] p span`, its `svg` and `.borderLeftColor` on the panel. Take `browser screenshot 'testid=agent-message >> nth=-1' --feature F06.markdown-alerts --name <theme>` in both themes and one at `browser viewport phone`; restore OpenHands-Neutral and `browser viewport desktop`.- **Long table cell (`F06.markdown-rendering`).** On the stream conversation write a prompt file whose first line is `Reply with only the word OK. Do not run any tools.`, then a blank line and the table `| Name | Description | Status |`, `|---|---|---|`, `| Alpha | This is a really long sentence in a single cell that goes on and on to explain a very involved aspect of the system, which nobody could read on one line without scrolling forever to the right. | ok |`, `| Beta | brief | fail |`. Run `control-openhands browser fill 'testid=chat-input' x --value-file <file>`, `control-openhands browser click 'testid=submit-button'` and `control-openhands conversation wait <id> --fresh --timeout 120`; the user bubble renders the table. Cells wrap once they reach 28rem (`max-w-md`): `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).whiteSpace + ' ' + getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).maxWidth"` is `normal 448px`. At desktop with the right panel closed, `control-openhands browser bbox 'testid=user-message >> nth=-1 >> testid=markdown-table-scroll'` has `scrollWidth` `582` equal to `clientWidth` and `Status` is visible (`browser screenshot 'testid=chat-interface' --feature F06.markdown-rendering --name long-cell-table`). A table still wider than the conversation keeps the horizontal scroll with its edge fade: with the panel open (`control-openhands browser click 'testid=right-panel-toggle'`) `clientWidth` is `474`, and at phone width `326`, so `Status` is scrolled off there while the long cell stays wrapped. An agent reply with such a table behaves the same.- **Links and TOML (`F06.markdown-rendering`).** Run `control-openhands conversation start --prompt "Write a short markdown answer on Python packaging with pyproject.toml. Start with the sentence: See [QA docs](https://example.com/qa) for the verification page. Include one toml code block with [build-system] and [project] sections. Do not run any tools." --wait --timeout 200`. `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=agent-message] .token.table')).display"` is `inline`, and each section header (`[build-system]`, `[project]`) is one line in `control-openhands browser screenshot 'testid=agent-message' --feature F06.markdown-rendering --name toml-and-link`. The link `testid=agent-message >> a[href="https://example.com/qa"]` has `text-decoration-line` `underline` before any hover. Send the same `See [QA docs](https://example.com/qa) ...` sentence as a follow-up to see it in a user bubble. In Light+ (F16.color-theme) the link is `rgb(0, 90, 153)`: 7.19:1 on the chat panel, 5.70:1 in the user bubble; in Solarized Light `rgb(0, 90, 136)`: 6.91:1 and 5.60:1 (contrast from the computed `color` and the nearest opaque background, read with `browser eval`). The same holds at phone width.- **Non-zero exit badge (`F06.tool-visualizers`).** In an open conversation run `control-openhands browser fill 'testid=chat-input' "Run exactly this terminal command, verbatim, with nothing appended: ls /qa_missing_dir2 . Then reply OK."`, `control-openhands browser click 'testid=submit-button'` and `control-openhands conversation wait <id> --fresh --timeout 150` (without `--fresh` it returns the previous run's `finished` at once). If the reply's actions render as a single row, run `control-openhands browser click 'testid=chat-scroll-container >> role=button[name="Expand"][exact] >> nth=-1'`; if they render as a group (`browser count 'testid=event-group'` grew, e.g. `2 actions completed` after a terminal reset and rerun), run `control-openhands browser click 'testid=event-group-toggle >> nth=-1'` and `control-openhands browser click 'testid=event-group-content >> nth=-1 >> role=button[name="Expand"][exact] >> nth=-1'`. Then `control-openhands browser count 'testid=chat-scroll-container >> text="exit 2"'` is `1`; the badge sits above the `No such file or directory` output (`browser screenshot 'testid=event-group >> nth=-1' --feature F06.tool-visualizers --name exit-code`). Repeating the check in the same conversation needs a new directory name and a count that grows, because an earlier expanded badge stays on the page.diff --git a/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md b/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.mdindex 64c2564af5..4ce8d2eab6 100644--- a/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md+++ b/.agents/skills/verify-openhands/references/feature-map/F07-conversation-page.md@@ -30,9 +30,9 @@ Source: `src/routes/conversation.tsx`, `src/components/features/conversation/`,- `F07.delete-confirm`: Delete Conversation names the title in a confirmation; Cancel keeps it, Confirm Delete removes it and returns to `/conversations`.- `F07.branch-from-message`: Branch from here on a message forks the conversation as `<title> (branch)`; on a user message the message is left out and its text goes back into the composer.- `F07.right-panel-toggle`: the header panel button shows and hides the right tabs panel (closed again after a reload).-- `F07.overview-toggle`: the header (i) button shows and hides the Overview column, which is exclusive with the right panel. At phone width it should reveal the Overview content on click. Known failure (reproduced twice 2026-10-19, [#16792](https://github.com/OpenHands/OpenHands/issues/16792)): at 390×844 the label changes from Show overview to Hide overview, but neither the Overview panel nor a phone dropdown renders.+- `F07.overview-toggle`: the header (i) button shows and hides the Overview column, which is exclusive with the right panel. At phone width it should reveal the Overview content on click. Known failure (reproduced 2026-10-22, [#16792](https://github.com/OpenHands/OpenHands/issues/16792)): at 390×844 the label changes from Show overview to Hide overview, but neither the Overview panel nor a phone dropdown renders.- `F07.git-actions-menu`: the header Git actions menu offers Commits, Pull, Push, Create PR and Create New Branch; each pre-fills the composer without sending, and Send becomes enabled so the user can submit or edit the prompt.-- `F07.git-control-bar`: under the composer, a folder workspace shows an inert workspace chip and the branch; with a GitHub `origin` the repo and branch chips link to GitHub and Pull, Push and Pull Request appear. The chip icons are visible in every theme. `browser eval` of the workspace (forked-repo), branch, external-link and Pull/Push/Pull Request icon `svg` computed `color` in `testid=chat-git-control-bar` equals its chip label's (`rgb(31, 31, 31)` in Light+, `rgb(0, 43, 54)` in Solarized Light, white in the default dark theme), at desktop and phone width; the GitHub logo in the repo chip keeps its own visible colour.+- `F07.git-control-bar`: under the composer, a folder workspace shows an inert workspace chip and the branch; with a GitHub `origin` the repo and branch chips link to GitHub and Pull, Push and Pull Request appear; an Azure DevOps `origin` links to `https://dev.azure.com/<org>/<project>/_git/<repo>`, and the chip shows the decoded name (`Web Apps`, not `Web%20Apps`). Known failure (reproduced 2026-10-22, [#18809](https://github.com/OpenHands/OpenHands/issues/18809)): the chip and the Overview repository row show the percent-encoded name. The chip icons are visible in every theme. `browser eval` of the workspace (forked-repo), branch, external-link and Pull/Push/Pull Request icon `svg` computed `color` in `testid=chat-git-control-bar` equals its chip label's (`rgb(31, 31, 31)` in Light+, `rgb(0, 43, 54)` in Solarized Light, white in the default dark theme), at desktop and phone width; the GitHub logo in the repo chip keeps its own visible colour.- `F07.git-control-bar-send`: the Pull, Push and Pull Request chips send their git prompt to the agent at once.- `F07.cloud-only`: Share publicly, Connect Repo, `task-<uuid>` start URLs, paused-sandbox resume, archived read-only state and the shared-conversation redirect exist only on a Cloud backend.- `F07.phone`: at 390 px the header keeps its controls and the info dialogs fit the viewport.@@ -81,7 +81,7 @@ Preconditions:- **Overview (`F07.overview-toggle`).** Run `control-openhands browser click 'testid=conversation-overview-toggle'`, `control-openhands browser visible 'testid=conversation-overview-panel'` (`true`) and `browser attr 'testid=conversation-overview-toggle' aria-label` (`Hide overview`). Then `control-openhands browser click 'testid=right-panel-toggle'`: `control-openhands browser wait 'testid=conversation-overview-column' --state hidden --timeout 5000` succeeds (an immediate `browser visible` can still read `true`). With the panel open, `control-openhands browser hover 'testid=conversation-overview-toggle'` and `control-openhands browser wait 'testid=conversation-overview-peek' --timeout 5000` shows the peek. Clicking the overview toggle now closes the panel (`aria-pressed` `false`) and shows the overview (`browser wait 'testid=conversation-overview-column' --state visible`). Click the overview toggle once more to restore the closed state.- **Git actions (`F07.git-actions-menu`).** Run `control-openhands browser click 'testid=conversation-git-actions-toggle'` and `control-openhands browser testids 'testid=conversation-git-actions-menu'` (`-commit`, `-pull`, `-push`, `-create-pr`, `-create-new-branch`, labels `Commits`, `Pull`, `Push`, `Create PR`, `Create New Branch`). `control-openhands browser click 'testid=conversation-git-actions-menu >> testid=conversation-git-actions-commit'` then `control-openhands browser text 'testid=chat-input'`: `Please review the current changes and create a git commit with a concise, descriptive message.`; `conversation events <id> --kinds MessageEvent` shows no new message. `control-openhands browser enabled 'testid=submit-button'` should be `true`, without typing. Each item replaces the composer text: `-pull` gives `Please pull the latest code from the repository.`, `-create-new-branch` gives `Please create a new branch with a descriptive name related to the work you plan to do.`, and the Push and Create PR prompts name `GitHub` by default. `control-openhands browser press Escape` closes the open menu. Clear the composer with `control-openhands browser fill 'testid=chat-input' ''`. Known failure ([#18202](https://github.com/OpenHands/OpenHands/issues/18202), reproduced 2026-10-09 on `main@3500d9e5`): after Pull, Send stays disabled until the user types.- **Panel route (`F07.panel-route`).** Run `control-openhands browser viewport phone`, open `/conversations/<id>` (`control-openhands browser goto /conversations/<id>`), `control-openhands browser click 'testid=right-panel-toggle' --expect-url '/panel$'` and `control-openhands browser testids --filter mobile` (`conversation-mobile-panel-top`, `conversation-mobile-panel-back`), then `control-openhands browser click 'testid=conversation-mobile-panel-back' --expect-url '/conversations/[0-9a-f-]+$'`. Direct entry at phone width: `control-openhands browser goto /conversations/<id>/panel` and `control-openhands browser wait 'testid=conversation-mobile-panel-back'` succeed (`control-openhands browser screenshot --feature F07.panel-route --name phone-direct`). The panel page serves every width up to 1024 px: after `control-openhands browser viewport tablet` the URL still ends `/panel` and `control-openhands browser count 'testid=conversation-mobile-panel-back'` is `1`. Wider, there is no panel page: run `control-openhands browser viewport desktop`, then `control-openhands browser wait-url '/conversations/[0-9a-f-]+$'` (the app replaces `/panel` with `/conversations/<id>`), `control-openhands browser count 'testid=conversation-mobile-panel-back'` (`0`) and `control-openhands browser attr 'testid=right-panel-toggle' aria-pressed` (`true`: the drawer is open beside the chat). A desktop-width visit lands the same way (#18045): `control-openhands browser goto /conversations/<id>/panel`, `control-openhands browser wait-url '/conversations/[0-9a-f-]+$'`, the same count (`0`) and `aria-pressed` (`true`); `control-openhands browser screenshot --feature F07.panel-route --name desktop-redirect` shows the chat with the drawer open beside it.-- **Git control bar (`F07.git-control-bar`).** On `/conversations/<git-id>` (`control-openhands browser goto /conversations/<git-id>`; Panel route ends on `/conversations/<id>`) run `control-openhands browser snapshot 'testid=interactive-chat-box'`: `button "qa-f07-repo" [disabled]` and the branch text `main`, no Pull/Push. Add a GitHub origin to the fixture (arrange step, outside the UI): `git -C <run>/workspace/qa-f07-repo remote add origin https://github.com/qa-example/qa-f07-repo.git`. Then `control-openhands browser reload` and the snapshot again: link `qa-example/qa-f07-repo` (`https://github.com/qa-example/qa-f07-repo`), link `main` (`…/tree/main`) and buttons `Pull`, `Push`, `Pull Request`.+- **Git control bar (`F07.git-control-bar`).** On `/conversations/<git-id>` (`control-openhands browser goto /conversations/<git-id>`; Panel route ends on `/conversations/<id>`) run `control-openhands browser snapshot 'testid=interactive-chat-box'`: `button "qa-f07-repo" [disabled]` and the branch text `main`, no Pull/Push. Add a GitHub origin to the fixture (arrange step, outside the UI): `git -C <run>/workspace/qa-f07-repo remote add origin https://github.com/qa-example/qa-f07-repo.git`. Then `control-openhands browser reload` and the snapshot again: link `qa-example/qa-f07-repo` (`https://github.com/qa-example/qa-f07-repo`), link `main` (`…/tree/main`) and buttons `Pull`, `Push`, `Pull Request`. An Azure DevOps origin links to the repository's `_git` page: `git -C <run>/workspace/qa-f07-repo remote set-url origin 'https://dev.azure.com/qa-org/Web%20Apps/_git/qa-f07-repo'`, `control-openhands browser reload`, then `control-openhands browser eval "[...document.querySelectorAll('[data-testid=chat-git-control-bar] a')].map(a=>a.href).join(' | ')"` (give the bar a second after the reload) is `https://dev.azure.com/qa-org/Web%20Apps/_git/qa-f07-repo | https://dev.azure.com/qa-org/Web%20Apps/_git/qa-f07-repo?version=GBmain` (the project name encoded once); the chip label should read `qa-org/Web Apps/qa-f07-repo`. Known failure (reproduced 2026-10-22, [#18809](https://github.com/OpenHands/OpenHands/issues/18809)): `browser text 'testid=chat-git-control-bar'` starts with `qa-org/Web%20Apps/qa-f07-repo`, as does the Overview's repository row; and the Overview's git section (`conversation-overview-toggle`) links the same URLs. Restore with `git -C <run>/workspace/qa-f07-repo remote set-url origin https://github.com/qa-example/qa-f07-repo.git` and `browser reload`.- **Pull sends at once (`F07.git-control-bar-send`).** Run `control-openhands browser click 'testid=interactive-chat-box >> role=button[name="Pull"][exact]'` and `control-openhands browser wait 'testid=user-message >> has-text=Please pull the latest code' --timeout 15000`; `conversation status <git-id>` is `running`. Stop it right away with Stop Conversation as in `F07.stop-confirm` (the fake remote cannot be pulled). Push and Pull Request are blocked: they need a real remote and GitHub credentials.- **Cloud-only (`F07.cloud-only`).** Blocked without a Cloud backend. On local, confirm the absence: the menu testids (see `F07.menu-open`) have no `share-publicly-button`, and the git control bar shows no `Connect Repo` chip. With Cloud: the menu's **Share publicly** switch (`share-publicly-button`, then `copy-share-link-button`, `open-share-link-button`), the repo chip **Connect Repo** opening the Open repository modal (`close-open-repository-modal`), `/conversations/task-<uuid>` while a conversation starts, and the `Unavailable for Archives` tooltips (see F25). Connect Repo saves on the server: picking a repository and branch and clicking Launch sends `PATCH https://app.all-hands.dev/api/v1/app-conversations/<id>` (`200`, toast `Repository updated successfully`) and the agent is asked to clone it; after deleting this conversation's entry from localStorage `openhands-agent-server-conversation-metadata` and `browser reload`, the git bar still shows the repository and branch.- **Phone (`F07.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser testids 'testid=chat-pane-header'` (adds `sidebar-mobile-menu-toggle`; all header controls visible), open each info dialog from the menu and `control-openhands browser bbox 'testid=skills-modal'` (likewise `hooks-modal`, `system-message-modal`): `insideViewport` is `true` and `pageHorizontalOverflow` `false`. Then `control-openhands browser click 'testid=chat-pane-header >> testid=ellipsis-button'` and `control-openhands browser bbox 'testid=conversation-name-context-menu'`: inside the viewport too (`x` about 133, `width` about 249 when the menu lists `Show Agent Tools & Metadata`; about 143 and 239 without that row, e.g. a conversation whose system prompt event is not loaded), and `control-openhands browser screenshot --feature F07.phone --name header-menu` shows every label whole (`Show Available Skills`, `Download conversation data`, `Delete Conversation`). `control-openhands browser press Escape` closes it. Return with `control-openhands browser viewport desktop`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md b/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.mdindex 2982603f2c..8125be0b64 100644--- a/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md+++ b/.agents/skills/verify-openhands/references/feature-map/F08-workspace-files-and-changes.md@@ -45,7 +45,7 @@ Source: `src/components/features/conversation/conversation-tabs/`, `src/componen- `F08.commit-rows`: each commit row shows the short SHA, subject, author (only when there are several authors) and relative time. Rows are single-open accordions, and new agent commits appear without a reload.- `F08.commits-cap`: the history loads the latest 50 commits; a longer history ends with `Showing the latest 50 commits`.- `F08.diff-view-modes`: expanding a changed file shows its diff; the view-mode buttons (`Old version`, `Diff`, `New version`) switch between old, diff and new without page errors.-- `F08.diff-markdown-preview`: a Markdown file in new (or old) mode renders as a prose preview.+- `F08.diff-markdown-preview`: a Markdown file in new (or old) mode renders as a prose preview; a long file scrolls inside a preview at most 600 px high.- `F08.diff-deleted-file`: a deleted working-tree file reads `This file was deleted. Its previous contents are no longer available for preview.`- `F08.changes-from-overview`: the overview's Changes row opens the drawer on Commits with Uncommitted expanded and closes the overview.@@ -93,7 +93,7 @@ Preconditions:- **Tool-card path chip (`F08.files-open-from-tool-chip`).** After the file-editor turn above, close the drawer and expand its tool group and card: `control-openhands browser click 'text=2 actions completed'`, then `control-openhands browser click 'role=region[name="Collapse actions"] >> role=button[name="Expand"] >> nth=-1'` (the `Create qa-editor.txt` card). `control-openhands browser text 'testid=file-path-chip'` is the absolute `<run>/workspace/qa-repo/qa-editor.txt`. Run `control-openhands browser click 'testid=file-path-chip'`: `right-panel-toggle` `aria-pressed` turns `true`, the selected strip tab is `file-quick-row-item-qa-editor.txt` (relative) and `browser text 'testid=files-tab-content'` ends with `created by file editor`.- **Commits list (`F08.uncommitted-changes`, `F08.commit-rows`).** Click `testid=conversation-tab-commits` and run `control-openhands browser text 'testid=commit-list'`. After the first prompt it reads `-`, `Uncommitted`, `6 files`, then `4bf8272`-style SHA, `Initial fixture commit` and `… ago`; after the second prompt `QA agent commit` appears above it, with author names `QA-Agent` and `QA Fixture` shown because there are now two authors. Run `control-openhands browser click 'testid=uncommitted-changes-row-toggle'` and `control-openhands browser text 'testid=uncommitted-changes-row-content'` (the changed paths; `aria-expanded` is `true`). Run `control-openhands browser click 'testid=commit-row >> has-text=Initial fixture commit >> testid=commit-row-toggle'`; Uncommitted's `aria-expanded` turns `false`, and `control-openhands browser text 'testid=commit-row-content'` lists `README.md`, `src/calc.py`, `src/test_calc.py`. After the third prompt the Uncommitted count is `8 files`.- **Diff modes (`F08.diff-view-modes`).** In the expanded Uncommitted row, run `control-openhands browser click 'testid=uncommitted-changes-row-content >> testid=file-diff-viewer-outer >> has-text=src/calc.py >> testid=collapse'`. `control-openhands browser text 'testid=uncommitted-changes-row-content >> testid=file-diff-viewer-outer >> has-text=src/calc.py >> testid=editor-container'` shows the added `def sub` lines, and `view-mode-diff` has `aria-pressed` `true`. The row's buttons are named: `control-openhands browser snapshot 'testid=uncommitted-changes-row-content >> testid=file-diff-viewer-outer >> has-text=src/calc.py'` lists `button "Old version"`, `button "Diff" [pressed]`, `button "New version"` and `button "Collapse"` (the same `collapse` button reads `Expand` while the row is collapsed), so `… >> role=button[name="Old version"]` works as well as the test id. Click `… >> testid=view-mode-old` (2 lines), then `… >> testid=view-mode-new` (4 lines), then `view-mode-diff`. Run `control-openhands browser errors --app-only` before and after: each collapse and each diff→old switch adds `TextModel got disposed before DiffEditorWidget model got reset` (fail, #17567).-- **Markdown preview (`F08.diff-markdown-preview`).** In this order `qa-notes.md` is already committed, so check it in the `QA agent commit` row: `control-openhands browser click 'testid=commit-row >> has-text=QA agent commit >> testid=commit-row-toggle'`, then with `F='testid=commit-row >> has-text=QA agent commit >> testid=file-diff-viewer-outer >> has-text=qa-notes.md'` click `"$F >> testid=collapse"` and `"$F >> testid=view-mode-new"`. `control-openhands browser count "$F >> testid=markdown-preview"` is `1`, rendering `<h1>QA notes</h1>`, **world** in bold and the `edited by agent` list item. (Before the second prompt the same check works under `testid=uncommitted-changes-row-content`.) Re-expand Uncommitted with `uncommitted-changes-row-toggle` for the next bullet.+- **Markdown preview (`F08.diff-markdown-preview`).** In this order `qa-notes.md` is already committed, so check it in the `QA agent commit` row: `control-openhands browser click 'testid=commit-row >> has-text=QA agent commit >> testid=commit-row-toggle'`, then with `F='testid=commit-row >> has-text=QA agent commit >> testid=file-diff-viewer-outer >> has-text=qa-notes.md'` click `"$F >> testid=collapse"` and `"$F >> testid=view-mode-new"`. `control-openhands browser count "$F >> testid=markdown-preview"` is `1`, rendering `<h1>QA notes</h1>`, **world** in bold and the `edited by agent` list item. (Before the second prompt the same check works under `testid=uncommitted-changes-row-content`.) A long file stays capped: arrange one in the workspace with `(printf '# QA long\n\n'; for i in $(seq 1 80); do echo "- item $i"; done) > <run>/workspace/qa-repo/qa-long.md`, expand its row under `testid=uncommitted-changes-row-content` (`L='testid=uncommitted-changes-row-content >> testid=file-diff-viewer-outer >> has-text=qa-long.md'`, then `"$L >> testid=collapse"` and `"$L >> testid=view-mode-new"`), and `control-openhands browser eval "(()=>{const o=[...document.querySelectorAll('[data-testid=file-diff-viewer-outer]')].find(x=>x.textContent.includes('qa-long.md'));const e=o.querySelector('[data-testid=markdown-preview]');return [getComputedStyle(e).maxHeight,e.offsetHeight,e.scrollHeight].join(' | ')})()"` reads `600px | 600 | <more than 600>` (shorter files such as qa-notes.md are shorter than 600), and `browser scroll "$L >> testid=markdown-preview" --by 400` scrolls inside it; delete `qa-long.md` afterwards; the same cap holds at phone width. The preview loads a moment after the file row expands: `browser wait "$F >> testid=markdown-preview"` before measuring. Re-expand Uncommitted with `uncommitted-changes-row-toggle` for the next bullet.- **Deleted file (`F08.diff-deleted-file`).** Expand `src/test_calc.py` in Uncommitted. `control-openhands browser text 'testid=uncommitted-changes-row-content >> testid=file-diff-viewer-outer >> has-text=src/test_calc.py >> testid=file-deleted-message'` reads `This file was deleted. Its previous contents are no longer available for preview.`- **From the overview (`F08.changes-from-overview`).** With the drawer closed, run `control-openhands browser click 'testid=conversation-overview-toggle'` and `control-openhands browser click 'testid=conversation-overview-diffs'` (its siblings `conversation-overview-diffs-additions`/`-deletions` show `+N`/`-N` in the theme's status success and error colors, for example `rgb(31, 189, 83)`/`rgb(255, 104, 78)` in the default theme and `rgb(22, 130, 93)`/`rgb(209, 52, 56)` in Light+; deleted files are not counted). `right-panel-toggle` `aria-pressed` is `true`, `testid=commit-list` is visible, `uncommitted-changes-row-toggle` has `aria-expanded` `true` and `testid=conversation-overview-panel` is hidden. After `browser reload` and the toggle, Commits is still selected.- **Empty states (`F08.commits-states`, `F08.files-empty-workspace`).** Run `control-openhands fixture folder`, which makes `<run>/workspace/qa-folder`, a plain folder (no `.git`) with `notes.md`; by the time the conversation has run it holds an empty `.git` (seen live; not created by the prompt), so it ends up a repo with no commits. Run `control-openhands browser goto /` (the browser is on the qa-repo conversation) and pick it with the workspace picker as in Preconditions (`folder-browser-entry-qa-folder`). Then run `control-openhands conversation start --stay --prompt 'Run exactly this bash command, then reply only "done": rm notes.md && git status --short' --wait --timeout 180`. Open the drawer and click `testid=conversation-tab-commits`; `control-openhands browser count 'text="No commits yet"'` is `1` (`browser screenshot --feature F08.commits-states --name no-commits`). On Files, click `testid=files-tab-refresh`; `control-openhands browser text 'testid=files-tab-tree'` reads `No files in workspace`. `Loading changes...` (`testid=commits-tab-status`) shows briefly on the first Commits open after a load: on `/conversations/<id>` with Files selected, `browser reload`, open the drawer, then `control-openhands browser click 'testid=conversation-tab-commits' --observe 'testid=commits-tab-status' --observe-ms 4000` records `Loading changes...` for a few ms. `Waiting for runtime to start...` (agent state init/loading/error) was not reached: `service stop agent-server` and `restart` both left `No commits yet` on screen.diff --git a/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md b/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.mdindex 5bcaa7cb37..d8cccec4af 100644--- a/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md+++ b/.agents/skills/verify-openhands/references/feature-map/F09-settings-shell.md@@ -1,13 +1,15 @@# F09 — Settings shell and navigation-Settings is one shell around eight pages (Agent, LLM, Model Router, Condenser,-Agent Context, Verification, Application, Secrets). On desktop a pinned left+Settings is one shell around seven pages on a local backend (Agent, LLM, Model+Router, Agent Context, Verification, Application, Secrets); a Cloud backend+adds Condenser after Model Router (locally each agent profile owns its+condenser). On desktop a pinned leftcolumn lists the pages, highlights the current one and ends with an "AgentCanvas" version card and a "synced from … backend" note; each page gets a titleand a grey subline. Below 1024 px (phones and tablets) `/settings` is a hub listinstead, at tablet widths (768–1023 px) each subpage has a compact Settings selector, and at phone width every page shows a Back chevron to the hub. Users reach Settings from the sidebar gear,the collapsed rail, the command menu and several deep links (LLM banner, getting-started checklist, composer pickers). The individual pages are other families+started checklist, composer pickers and the agent-profile pill). The individual pages are other families(F10–F16; Secrets is F14).Source: `src/routes.ts`, `src/routes/settings.tsx`, `src/routes/settings-index.tsx`, `src/routes/agent-settings.tsx`, `src/constants/settings-nav.tsx`, `src/hooks/use-settings-nav-items.ts`, `src/utils/settings-utils.ts`, `src/components/features/settings/settings-layout.tsx`, `src/components/features/settings/settings-desktop-sidebar.tsx`, `src/components/shared/compact-section-navigation.tsx`, `src/components/features/settings/settings-mobile-hub.tsx`, `src/components/features/settings/settings-mobile-drawer.tsx`, `src/components/features/settings/settings-navigation.tsx`, `src/components/features/settings/mobile-header.tsx`, `src/components/features/settings/backend-synced-settings-badge.tsx`, `src/components/features/settings/agent-canvas-update-card.tsx`, `src/components/features/settings/agent-canvas-version-tile.tsx`, `src/utils/mobile-section-nav.ts`, `src/components/features/command-menu/command-menu-items.tsx`, `src/components/features/settings/brand-button.tsx`.@@ -16,17 +18,17 @@ Source: `src/routes.ts`, `src/routes/settings.tsx`, `src/routes/settings-index.t- `F09.entry-gear`: the sidebar gear opens Settings on the Agent page; the gear is marked active on every settings URL; at phone width the drawer's gear opens the hub and closes the drawer.- `F09.entry-collapsed-rail`: with the sidebar collapsed, the rail's Settings icon opens the Agent page.-- `F09.entry-command-menu`: the command menu's Settings group (Settings, Agent settings, LLM profiles, Model Router, Condenser, Agent Context, Verification, Application, Secrets settings) opens the matching page.+- `F09.entry-command-menu`: the command menu's Settings group (Settings, Agent settings, LLM profiles, Model Router, Agent Context, Verification, Application, Secrets settings; Condenser settings only on a Cloud backend) opens the matching page.- `F09.command-menu-coverage`: every page in the settings nav, including Model Router and Agent Context, can be found in the command menu and opens from it.-- `F09.entry-deep-links`: the Home "Set up LLM" banner, the checklist items "Add LLM API key" and "Customize your agent", the composer LLM picker's "LLM Profiles" link and the + menu's "Manage agent profiles" link each land on `/settings/llm` or `/settings/agents`.+- `F09.entry-deep-links`: the Home "Set up LLM" banner, the checklist items "Add LLM API key" and "Customize your agent", the composer LLM picker's "LLM Profiles" link, and the "Manage agent profiles" links of the composer's agent-profile pill and of the + menu each land on `/settings/llm` or `/settings/agents`.- `F09.index-redirect`: on desktop, `/settings` replaces itself with `/settings/agents` (Back skips `/settings`).- `F09.locked-cloud-landing`: on a canvas locked to a Cloud host, `/settings` lands on `/settings/app` instead.- `F09.legacy-agent-redirect`: the retired `/settings/agent` URL replaces itself with `/settings/agents`.- `F09.hidden-llm-redirect`: with the `hide_llm_settings` feature flag, the LLM link disappears and `/settings/llm` redirects to `/settings/agents`.-- `F09.desktop-nav`: the desktop nav lists the eight pages in order, exactly one link is `aria-current="page"`, and the nav stays pinned while the page content scrolls.+- `F09.desktop-nav`: the desktop nav lists the settings pages in order (seven on a local backend, no Condenser), exactly one link is `aria-current="page"`, and the nav stays pinned while the page content scrolls.- `F09.page-header`: each page shows its nav title as an H2 with a grey subline; Secrets draws its own header instead, and the Agent profile editor hides the shell header until you leave it.- `F09.landmarks`: the settings screen exposes a single `main` landmark, on desktop pages and on the phone hub.-- `F09.phone-hub`: below 1024 px (phones and tablets), `/settings` stays put and shows a "Settings" heading, the eight links, the version card and the synced note, with no page header and no horizontal overflow.+- `F09.phone-hub`: below 1024 px (phones and tablets), `/settings` stays put and shows a "Settings" heading, the settings links (seven on a local backend), the version card and the synced note, with no page header and no horizontal overflow.- `F09.hub-resize-redirect`: widening the window to ≥1024 px while the phone hub is open replaces `/settings` with `/settings/agents`; narrowing on a sub-page keeps its URL and adds the Back chevron.- `F09.breakpoint-sweep`: at 767, 768, 820, 1023, 1024 and 1440 px, `/settings` is the hub below 1024 px and redirects to the desktop nav from 1024 px up, and the Application page's `main` is at least 340 px wide with no horizontal overflow. Below 1024 px the settings `<main>` fills the column down to the viewport bottom and scrolls itself, so a popover inside it (the tablet selector) is not clipped on a short page (#18256).- `F09.tablet-selector`: at 768–1023 px, a Settings selector above the subpage opens the same visible section links as the desktop sidebar, including Cloud links when available. Selecting a sibling navigates directly and closes it; Tab reaches links, Escape closes and returns focus, and outside clicks dismiss it. The current section uses a slash-boundary match, so a trailing slash does not hide the selector or its selected link. It is absent on the hub, phones and desktops.@@ -45,9 +47,9 @@ Source: `src/routes.ts`, `src/routes/settings.tsx`, `src/routes/settings-index.t- Sidebar footer gear next to the backend selector (`backend-selector-settings-link`), on desktop or inside the phone drawer (`sidebar-mobile-menu-toggle`, then `sidebar-mobile-drawer >> backend-selector-settings-link`).- Collapsed sidebar: the gear icon at the bottom of the rail (`collapsed-settings-link`).-- Command menu (`Control+k`/`Meta+k` or `command-menu-trigger`): group **Settings** with Settings, Agent settings, LLM profiles, Model Router, Condenser settings, Agent Context, Verification settings, Application settings, Secrets settings.-- Deep links: Home banner **Set up LLM** (`home-llm-not-configured-action`, only while no LLM key is configured); getting-started checklist **Add LLM API key** and **Customize your agent**; composer LLM pill (`chat-input-llm-profile`) → **LLM Profiles**; composer **+** (`chat-plus-button`) → **Switch agent profile** → **Manage agent profiles**. Application settings also links "Manage LLM profiles" (F15's page).-- Direct URLs: `/settings`, `/settings/agents`, `/settings/llm`, `/settings/meta-llm`, `/settings/condenser`, `/settings/agent-context`, `/settings/verification`, `/settings/app`, `/settings/secrets`, and the legacy `/settings/agent`.+- Command menu (`Control+k`/`Meta+k` or `command-menu-trigger`): group **Settings** with Settings, Agent settings, LLM profiles, Model Router, Agent Context, Verification settings, Application settings, Secrets settings (plus Condenser settings on a Cloud backend).+- Deep links: Home banner **Set up LLM** (`home-llm-not-configured-action`, only while no LLM key is configured); getting-started checklist **Add LLM API key** and **Customize your agent**; composer LLM pill (`chat-input-llm-profile`) → **LLM Profiles**; composer agent-profile pill (`chat-input-agent-profile`) → **Manage agent profiles**; composer **+** (`chat-plus-button`) → **Switch agent profile** → **Manage agent profiles**. Application settings also links "Manage LLM profiles" (F15's page).+- Direct URLs: `/settings`, `/settings/agents`, `/settings/llm`, `/settings/meta-llm`, `/settings/condenser` (Cloud backend; on a local backend it replaces itself with `/settings/agents`, see `F15.condenser-page`), `/settings/agent-context`, `/settings/verification`, `/settings/app`, `/settings/secrets`, and the legacy `/settings/agent`.- Inside Settings: the left nav (`sidebar-settings-/settings/<page>`) on desktop; the hub list (`settings-mobile-hub`) plus the top-bar Back chevron (`sidebar-mobile-back-button`) on phones.## Driving it with control-openhands@@ -61,16 +63,16 @@ Preconditions:- **Gear entry and index redirect (`F09.entry-gear`, `F09.index-redirect`).** From `/`, run `control-openhands browser click 'testid=backend-selector-settings-link'` then `control-openhands browser wait-url '/settings/agents$'`. Then `control-openhands browser attr 'testid=sidebar-settings-/settings/agents' aria-current` is `page` and `control-openhands browser attr 'testid=backend-selector-settings-link' data-active` is `true` (it is `false` again on `/`). Run `control-openhands browser back` then `control-openhands browser url`: the URL is `/`, not `/settings`, which proves the redirect replaced history. Direct entry: `control-openhands browser goto /settings` returns `"url": ".../settings/agents"`.- **Collapsed rail (`F09.entry-collapsed-rail`).** From `/` (`control-openhands browser goto /`; the previous bullet ends on `/settings/agents`, where the URL wait would pass at once) run `control-openhands browser click 'testid=sidebar-collapse-toggle'`, `control-openhands browser click 'testid=collapsed-settings-link'`, `control-openhands browser wait-url '/settings/agents$'` and `control-openhands browser screenshot --feature F09.entry-collapsed-rail --name rail`. The screenshot shows the icon rail, the settings nav with Agent highlighted. Restore with `control-openhands browser click 'testid=sidebar-collapse-toggle'` (the label flips back to "Collapse sidebar").-- **Command menu (`F09.entry-command-menu`).** First rest the pointer off the list with `control-openhands browser hover 'testid=command-menu-trigger'`: a hover can move the menu's selection, and Enter runs the selected option (F02 Gotchas). For each title run `control-openhands browser goto /`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'LLM profiles'`, `control-openhands browser press Enter`, `control-openhands browser wait-url '/settings/llm$'`. Expected targets: `Settings` → `/settings/agents`, `Agent settings` → `/settings/agents`, `LLM profiles` → `/settings/llm`, `Model Router` → `/settings/meta-llm`, `Condenser settings` → `/settings/condenser`, `Agent Context` → `/settings/agent-context`, `Verification settings` → `/settings/verification`, `Application settings` → `/settings/app`, `Secrets settings` → `/settings/secrets`. `control-openhands browser snapshot 'testid=command-menu >> role=listbox'` after typing `LLM profiles` shows two options under the "Settings" group text: `LLM profiles` `[selected]` and `Model Router`, whose subline mentions LLM profiles. Enter runs the selected one.+- **Command menu (`F09.entry-command-menu`).** First rest the pointer off the list with `control-openhands browser hover 'testid=command-menu-trigger'`: a hover can move the menu's selection, and Enter runs the selected option (F02 Gotchas). For each title run `control-openhands browser goto /`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'LLM profiles'`, `control-openhands browser press Enter`, `control-openhands browser wait-url '/settings/llm$'`. Expected targets: `Settings` → `/settings/agents`, `Agent settings` → `/settings/agents`, `LLM profiles` → `/settings/llm`, `Model Router` → `/settings/meta-llm`, `Agent Context` → `/settings/agent-context`, `Verification settings` → `/settings/verification`, `Application settings` → `/settings/app`, `Secrets settings` → `/settings/secrets`. `control-openhands browser snapshot 'testid=command-menu >> role=listbox'` after typing `LLM profiles` shows two options under the "Settings" group text: `LLM profiles` `[selected]` and `Model Router`, whose subline mentions LLM profiles. Enter runs the selected one. On a local backend, typing `Condenser` shows `No commands found` (`browser screenshot --feature F09.entry-command-menu --name no-condenser`).- **Command menu coverage (`F09.command-menu-coverage`).** Run `control-openhands browser goto /`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'Model Router'` and `control-openhands browser snapshot 'testid=command-menu >> role=listbox'`. It shows one option under the "Settings" group text: `Model Router Meta-profiles that route each task to the right LLM profile. Go` `[selected]`. Run `control-openhands browser press Enter` and `control-openhands browser wait-url '/settings/meta-llm$'`; `control-openhands browser count 'testid=command-menu'` is `0`. Repeat from `/` with `Agent Context`: the option reads `Agent Context Let the agent keep notes and recall them in new conversations. Go` and the URL becomes `/settings/agent-context`. With the previous bullet, every page in the settings nav has opened from the menu.- **Deep links without a model (`F09.entry-deep-links`).** On `/` with no LLM key (`control-openhands browser goto /`): `control-openhands browser click 'testid=home-llm-not-configured-action'` then `control-openhands browser wait-url '/settings/'` → `/settings/llm`. `control-openhands browser goto /`, `control-openhands browser click 'testid=sidebar-onboarding-checklist-item-configure-llm'`, `control-openhands browser wait-url '/settings/'` → `/settings/llm`. `control-openhands browser goto /`, `control-openhands browser click 'testid=sidebar-onboarding-checklist-item-customize-agent'`, `control-openhands browser wait-url '/settings/'` → `/settings/agents`.-- **Deep links from the composer (`F09.entry-deep-links`).** After `control-openhands llm preset deepseek`, run `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-input-llm-profile'`, `control-openhands browser click 'testid=chat-input-llm-profile-popover >> role=link[name="LLM Profiles"]'`, `control-openhands browser wait-url '/settings/llm$'`; `control-openhands browser count 'testid=chat-input-llm-profile-popover'` is `0`. Then `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser hover 'testid=switch-agent-profile-button'`, `control-openhands browser click 'testid=agent-profile-submenu >> role=link[name="Manage agent profiles"]'`, `control-openhands browser wait-url '/settings/agents$'`; `control-openhands browser count 'testid=tools-context-menu'` is `0`.+- **Deep links from the composer (`F09.entry-deep-links`).** After `control-openhands llm preset deepseek`, run `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-input-llm-profile'`, `control-openhands browser click 'testid=chat-input-llm-profile-popover >> role=link[name="LLM Profiles"]'`, `control-openhands browser wait-url '/settings/llm$'`; `control-openhands browser count 'testid=chat-input-llm-profile-popover'` is `0`. Then `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-plus-button'`, `control-openhands browser hover 'testid=switch-agent-profile-button'`, `control-openhands browser click 'testid=agent-profile-submenu >> role=link[name="Manage agent profiles"]'`, `control-openhands browser wait-url '/settings/agents$'`; `control-openhands browser count 'testid=tools-context-menu'` is `0`. The agent-profile pill: `control-openhands browser goto /`, `control-openhands browser click 'testid=chat-input-agent-profile'`, `control-openhands browser snapshot 'testid=chat-input-agent-profile-popover'` (`Available Profiles`, a `default` button, `link "Manage agent profiles"`), `control-openhands browser click 'testid=chat-input-agent-profile-popover >> role=link[name="Manage agent profiles"]'` and `control-openhands browser wait-url '/settings/agents$'`; `control-openhands browser count 'testid=chat-input-agent-profile-popover'` is `0`.- **Legacy URL (`F09.legacy-agent-redirect`).** Run `control-openhands browser goto /`, then `control-openhands browser goto /settings/agent`; the result already reads `.../settings/agents`. `control-openhands browser back` returns to `/`.-- **Desktop nav (`F09.desktop-nav`).** On any settings page (`control-openhands browser goto /settings/agents`; Legacy URL ends on `/`) run `control-openhands browser snapshot 'testid=settings-navbar-desktop'`: links Agent, LLM, Model Router, Condenser, Agent Context, Verification, Application, Secrets (with `/url`s), then `button "Agent Canvas Version …"` and the synced text. Click each one, e.g. `control-openhands browser click 'testid=sidebar-settings-/settings/meta-llm'`, `control-openhands browser wait-url '/settings/meta-llm$'`, `control-openhands browser attr 'testid=sidebar-settings-/settings/meta-llm' aria-current` (`page`), and `control-openhands browser count 'testid=settings-navbar-desktop >> [aria-current="page"]'` (`1`). Pinned: `control-openhands browser goto /settings/app` (the only page taller than the viewport), `control-openhands browser bbox 'testid=settings-navbar-desktop'` (`y` 32), `control-openhands browser hover 'testid=settings-page-subtitle'`, `control-openhands browser scroll --by 1000`, then `control-openhands browser bbox 'testid=settings-page-subtitle'` (negative `y`, content scrolled) and `control-openhands browser bbox 'testid=settings-navbar-desktop'` (still `y` 32); `control-openhands browser screenshot --feature F09.desktop-nav --name scrolled` shows Git Settings on the right and the full nav on the left.-- **Page header (`F09.page-header`).** After each nav click, `control-openhands browser text 'testid=settings-screen >> header'` reads title + subline: Agent "Create and manage reusable agent setups.", LLM "Model, API key, and options for the agent.", Model Router "Meta-profiles that route each task to the right LLM profile.", Condenser "Summarize long chats to stay within context limits.", Agent Context "Let the agent keep notes and recall them in new conversations.", Verification "Confirmation prompts and security checks on actions.", Application "Language, theme, notifications, and Git identity.". On Secrets, `control-openhands browser count 'testid=settings-screen >> header >> h2'` is `0` (the page draws its own "Secrets" heading). Editor: on `/settings/agents` (`control-openhands browser goto /settings/agents`) run `control-openhands browser click 'testid=add-agent-profile'`; `control-openhands browser count 'testid=settings-page-subtitle'` is `0` and `control-openhands browser text 'testid=settings-screen >> h2'` is `Add agent profile`. `control-openhands browser click 'testid=back-to-agent-profiles'` brings the subtitle back (count `1`); opening the editor again and clicking `testid=sidebar-settings-/settings/condenser` also shows the Condenser subline.+- **Desktop nav (`F09.desktop-nav`).** On any settings page (`control-openhands browser goto /settings/agents`; Legacy URL ends on `/`) run `control-openhands browser snapshot 'testid=settings-navbar-desktop'`: on a local backend links Agent, LLM, Model Router, Agent Context, Verification, Application, Secrets (with `/url`s; a Cloud backend adds Condenser after Model Router), then `button "Agent Canvas Version …"` and the synced text. Click each one, e.g. `control-openhands browser click 'testid=sidebar-settings-/settings/meta-llm'`, `control-openhands browser wait-url '/settings/meta-llm$'`, `control-openhands browser attr 'testid=sidebar-settings-/settings/meta-llm' aria-current` (`page`), and `control-openhands browser count 'testid=settings-navbar-desktop >> [aria-current="page"]'` (`1`). Pinned: `control-openhands browser goto /settings/app` (the only page taller than the viewport), `control-openhands browser bbox 'testid=settings-navbar-desktop'` (`y` 32), `control-openhands browser hover 'testid=settings-page-subtitle'`, `control-openhands browser scroll --by 1000`, then `control-openhands browser bbox 'testid=settings-page-subtitle'` (negative `y`, content scrolled) and `control-openhands browser bbox 'testid=settings-navbar-desktop'` (still `y` 32); `control-openhands browser screenshot --feature F09.desktop-nav --name scrolled` shows Git Settings on the right and the full nav on the left.+- **Page header (`F09.page-header`).** After each nav click, `control-openhands browser text 'testid=settings-screen >> header'` reads title + subline: Agent "Create and manage reusable agent setups.", LLM "Model, API key, and options for the agent.", Model Router "Meta-profiles that route each task to the right LLM profile.", Agent Context "Let the agent keep notes and recall them in new conversations.", Verification "Confirmation prompts and security checks on actions.", Application "Language, theme, notifications, and Git identity." (Condenser, on a Cloud backend only: "Summarize long chats to stay within context limits."). On Secrets, `control-openhands browser count 'testid=settings-screen >> header >> h2'` is `0` (the page draws its own "Secrets" heading). Editor: on `/settings/agents` (`control-openhands browser goto /settings/agents`) run `control-openhands browser click 'testid=add-agent-profile'`; `control-openhands browser count 'testid=settings-page-subtitle'` is `0` and `control-openhands browser text 'testid=settings-screen >> h2'` is `Add agent profile`. `control-openhands browser click 'testid=back-to-agent-profiles'` brings the subtitle back (count `1`); opening the editor again and clicking `testid=sidebar-settings-/settings/verification` also shows the Verification subline.- **Landmarks (`F09.landmarks`).** Run `control-openhands browser goto /settings/agents`, `control-openhands browser count 'role=main'` (`1`) and `control-openhands browser eval "document.querySelector('[data-testid=settings-screen]').tagName"` (`DIV`: the shell wrapper is not a landmark; the layout's own `<main>` is). `control-openhands browser goto /settings/app` and the same `count` is `1`. At phone width (`control-openhands browser viewport phone`, `control-openhands browser goto /settings`) the hub also counts `1`; return with `control-openhands browser viewport desktop`.-- **Phone hub and Back (`F09.phone-hub`, `F09.phone-back`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings` (URL stays `/settings`), `control-openhands browser snapshot 'testid=settings-mobile-hub'` (heading "Settings", the eight links, the version button, the synced text), `control-openhands browser count 'testid=settings-page-subtitle'` (`0`), `control-openhands browser bbox 'testid=settings-mobile-hub'` (`pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F09.phone-hub --name hub`. Then `control-openhands browser click 'testid=settings-mobile-hub >> testid=sidebar-settings-/settings/condenser'`, `control-openhands browser wait-url '/settings/condenser$'`, `control-openhands browser attr 'testid=sidebar-mobile-back-button' aria-label` (`Settings`), `control-openhands browser click 'testid=sidebar-mobile-back-button'`, `control-openhands browser wait-url '/settings$'` and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`). Drawer entry: `control-openhands browser goto /`, `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'`, `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=backend-selector-settings-link'`, `control-openhands browser wait-url '/settings$'` and `control-openhands browser wait 'testid=sidebar-mobile-drawer' --state detached --timeout 5000` pass (the drawer closes as it animates out; an immediate `visible` can still read `true`). Return with `control-openhands browser viewport desktop`.-- **Resize from the hub (`F09.hub-resize-redirect`).** Run `control-openhands browser goto /`, `control-openhands browser viewport phone`, `control-openhands browser goto /settings` and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`). Then `control-openhands browser viewport desktop` and `control-openhands browser wait-url '/settings/agents$'`: the hub is gone (`count 'testid=settings-mobile-hub'` is `0`) and `control-openhands browser back` lands on `/` (the redirect replaced `/settings`). Reverse: `control-openhands browser goto /settings/condenser`, `control-openhands browser viewport phone`, `control-openhands browser url` (still `/settings/condenser`) and `control-openhands browser visible 'testid=sidebar-mobile-back-button'` (`true`). Tablets get the hub too, and the boundary is 1024 px: run `control-openhands browser viewport tablet` (820 px), `control-openhands browser goto /settings` (URL stays `/settings`) and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`); then `control-openhands browser viewport 1023x900`, `control-openhands browser goto /settings` and the same `visible` (`true`); then `control-openhands browser viewport 1024x900` and `control-openhands browser wait-url '/settings/agents$'` (`browser visible 'testid=settings-navbar-desktop'` is `true`). Return with `control-openhands browser viewport desktop`.+- **Phone hub and Back (`F09.phone-hub`, `F09.phone-back`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings` (URL stays `/settings`), `control-openhands browser snapshot 'testid=settings-mobile-hub'` (heading "Settings", the seven links Agent, LLM, Model Router, Agent Context, Verification, Application, Secrets, the version button, the synced text), `control-openhands browser count 'testid=settings-page-subtitle'` (`0`), `control-openhands browser bbox 'testid=settings-mobile-hub'` (`pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F09.phone-hub --name hub`. Then `control-openhands browser click 'testid=settings-mobile-hub >> testid=sidebar-settings-/settings/verification'`, `control-openhands browser wait-url '/settings/verification$'`, `control-openhands browser attr 'testid=sidebar-mobile-back-button' aria-label` (`Settings`), `control-openhands browser click 'testid=sidebar-mobile-back-button'`, `control-openhands browser wait-url '/settings$'` and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`). Drawer entry: `control-openhands browser goto /`, `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'`, `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=backend-selector-settings-link'`, `control-openhands browser wait-url '/settings$'` and `control-openhands browser wait 'testid=sidebar-mobile-drawer' --state detached --timeout 5000` pass (the drawer closes as it animates out; an immediate `visible` can still read `true`). Return with `control-openhands browser viewport desktop`.+- **Resize from the hub (`F09.hub-resize-redirect`).** Run `control-openhands browser goto /`, `control-openhands browser viewport phone`, `control-openhands browser goto /settings` and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`). Then `control-openhands browser viewport desktop` and `control-openhands browser wait-url '/settings/agents$'`: the hub is gone (`count 'testid=settings-mobile-hub'` is `0`) and `control-openhands browser back` lands on `/` (the redirect replaced `/settings`). Reverse: `control-openhands browser goto /settings/verification`, `control-openhands browser viewport phone`, `control-openhands browser url` (still `/settings/verification`) and `control-openhands browser visible 'testid=sidebar-mobile-back-button'` (`true`). Tablets get the hub too, and the boundary is 1024 px: run `control-openhands browser viewport tablet` (820 px), `control-openhands browser goto /settings` (URL stays `/settings`) and `control-openhands browser visible 'testid=settings-mobile-hub'` (`true`); then `control-openhands browser viewport 1023x900`, `control-openhands browser goto /settings` and the same `visible` (`true`); then `control-openhands browser viewport 1024x900` and `control-openhands browser wait-url '/settings/agents$'` (`browser visible 'testid=settings-navbar-desktop'` is `true`). Return with `control-openhands browser viewport desktop`.- **Widths around the breakpoint (`F09.breakpoint-sweep`).** For each width `W` in `767`, `768`, `820`, `1023`, `1024`, `1440` run `control-openhands browser viewport Wx1024` and `control-openhands browser goto /settings`. Below 1024 px the URL stays `/settings`, `control-openhands browser visible 'testid=settings-mobile-hub'` is `true` and `control-openhands browser bbox 'testid=settings-mobile-hub'` has `pageHorizontalOverflow` `false` (725, 426, 478 and 681 px wide); from 1024 px up `control-openhands browser wait-url '/settings/agents$'` passes and `control-openhands browser visible 'testid=settings-navbar-desktop'` is `true`. Then `control-openhands browser goto /settings/app` and `control-openhands browser bbox 'role=main'`: `width` is at least 340 (767, 468, 520, 723, 424 and 840 today: full width at 767 px, `W − 300` beside the rail up to 1023 px, `W − 600` beside the rail and the settings nav from 1024 px), `scrollWidth` equals `clientWidth` and `pageHorizontalOverflow` is `false` (`insideViewport` is `false` only because the page is taller than 1024 px). Take `control-openhands browser screenshot --feature F09.breakpoint-sweep --name app-767` and `--name app-1024`. Height: at `390x844`, `767x1024`, `820x1180` and `1023x900`, on `/settings/secrets` (a short page) `control-openhands browser eval "Math.round(document.querySelector('[data-testid=settings-screen] main').getBoundingClientRect().bottom)+'/'+innerHeight"` reads the viewport height twice (`1180/1180` at 820x1180), and on `/settings/app` `control-openhands browser eval "(()=>{const m=document.querySelector('[data-testid=settings-screen] main');return m.scrollHeight>m.clientHeight})()"` is `true` (the `<main>` scrolls, not an outer column). At 1440x1000 the desktop layout is unchanged (the Secrets `<main>` ends at its content). Finish with `control-openhands browser viewport desktop`.- **Tablet selector (`F09.tablet-selector`).** Run `control-openhands browser viewport tablet`, `control-openhands browser goto /settings/app`, then `control-openhands browser click 'testid=settings-compact-navigation >> role=button'`. `control-openhands browser snapshot 'testid=settings-compact-navigation >> role=navigation'` lists every visible settings destination with Application current. Select `control-openhands browser click 'testid=settings-compact-navigation >> testid=sidebar-settings-/settings/secrets' --expect-url '/settings/secrets$'`; the selector now shows Secrets and its button has `aria-expanded=false`. Open again, `browser press Tab`, `browser press Escape`: `browser eval "document.activeElement.getAttribute('aria-expanded')"` is `false` (focus returned to the trigger). Capture `browser screenshot --feature F09.tablet-selector --name settings`. Repeat at 768 and 1023 px; at 767 and 1024 px `browser count 'testid=settings-compact-navigation'` is `0`. On `/settings` below 1024 px the hub remains the list. At tablet width, visit `/settings/llm/`: the trigger reads `Settings: LLM`, the page heading is LLM, and opening the selector marks LLM current. A nested URL such as `/settings/llm/example` is not a registered page and shows the global Not Found screen (F09.unknown-subpath).- **Synced badge (`F09.synced-badge`).** Run `control-openhands browser text 'testid=backend-synced-settings-badge'` and `control-openhands browser eval 'location.origin'`. The text is `These settings are synced from Local backend (<origin>)` with the same origin.diff --git a/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md b/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.mdindex 8063e7f914..a6696de12c 100644--- a/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md+++ b/.agents/skills/verify-openhands/references/feature-map/F10-llm-profiles.md@@ -19,12 +19,12 @@ Source: `src/routes/llm-settings.tsx`, `src/components/features/settings/llm-pro- `F10.list-grouped`: once any profile links to a provider connection, rows group under the connection's name, with unlinked rows last under `Not linked`; with no linked rows the list is flat again.- `F10.broken-link`: a profile whose linked connection no longer exists shows a warning `Broken link` badge whose tooltip says to edit the profile to re-link it; its editor offers the orphaned id in the Provider connection selector so it can be cleared.- `F10.actions-menu`: the row menu offers Edit, Rename, Duplicate, Set as default (disabled on the Default row) and Delete; it focuses Edit on open (mouse or Enter), arrows cycle over the enabled items (skipping a disabled Set as default), and Escape, Tab or an outside click close it.-- `F10.set-default`: Set as default toasts `Switched to profile "<name>"`, moves the badge, persists and becomes the agent's active LLM.+- `F10.set-default`: Set as default toasts `Switched to profile "<name>"`, moves the badge, persists and becomes the agent's active LLM; the `default` agent profile's LLM follows it.- `F10.duplicate`: Duplicate saves `<name>-copy`, then `<name>-copy-1`, with the same model and stored key; the copy is not Default.- `F10.rename`: the Rename modal is prefilled and focused; invalid names disable Rename; an unchanged name or Cancel just closes; Enter submits; success toasts and persists, and the Default badge follows a renamed default.- `F10.rename-conflict`: renaming onto an existing name is refused with the server's message (`Profile '<name>' already exists`) and the modal stays open.- `F10.delete`: Delete asks `Are you sure you want to delete the profile "<name>"? This action cannot be undone.` with Cancel focused; Cancel keeps the row, Delete removes it with a toast.-- `F10.delete-default`: deleting the Default profile promotes another remaining profile to Default.+- `F10.delete-default`: deleting the Default profile promotes another remaining profile to Default; the `default` agent profile is first moved onto that profile, so the delete is not refused.- `F10.create`: Add LLM Profile → provider, model and API key → Save shows `Validating...` then `Saving...`, toasts `Profile "<name>" created` and returns to the list; the first profile on an empty list becomes Default.- `F10.create-prefill`: a new editor prefills the backend default model with a blank key, base URL and connection; the other LLM options it shows come from the agent's current LLM settings, i.e. the last applied Default profile (the editor opens on the tab they need), and Save stores what the form shows.- `F10.api-key-help`: under the API key field (Basic and Advanced), `Don't know your API key? Click here for instructions` opens the OpenHands docs on getting an API key in a new tab.@@ -32,7 +32,7 @@ Source: `src/routes/llm-settings.tsx`, `src/components/features/settings/llm-pro- `F10.name-validation`: a name that breaks the rule (letter or digit first; letters, digits, `.`, `_`, `-`; max 64) disables Save and turns the rule text red; a name that already exists marks the field invalid, shows a taken-name message about LLM profiles and disables Save (Edit accepts the profile's own name).- `F10.create-validation`: a Save whose validation call fails toasts the provider error and keeps the editor open with nothing saved.- `F10.custom-model`: the Advanced tab takes a free-text model and Base URL; an empty model is refused with `Model is required`.-- `F10.model-picker`: the Basic provider and model comboboxes group options as Verified Models and Other Models; changing provider clears the model; the OpenHands provider shows account and key help.+- `F10.model-picker`: the Basic provider and model comboboxes group options as Verified Models and Other Models; providers sort by the name the picker shows, case-insensitively, with OpenHands first among the verified ones; focusing the provider field selects its label, so typing replaces it and filters the list; changing provider clears the model; the OpenHands provider shows account and key help.- `F10.provider-list-complete`: the Basic provider combobox offers every provider the backend reports or has verified models for (over a hundred, `xai` included); picking a late-sorting one fills the field and loads its models.- `F10.cloud-provider-pagination`: on a Cloud organization backend the provider list is fetched from `providers/search` page by page, so providers past the first page (`xai`, `openrouter`) are offered too (blocked here).- `F10.edit`: Edit loads the profile (`Editing profile "<name>" - save to apply changes`) with the stored key masked; Save stays disabled until something changes; a changed name renames before saving; the stored key is kept; a save the Agent Server refuses toasts the server's reason and keeps the editor open.@@ -70,8 +70,8 @@ Preconditions:- **Empty list (`F10.list-empty`).** On a fresh run, navigate from the sidebar: `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings'`, `control-openhands browser click 'testid=sidebar-settings-/settings/llm' --expect-url '/settings/llm'`, then `control-openhands browser text 'testid=profiles-empty'`. The text is `No profiles saved yet. Add a profile to save your LLM configurations.` and `control-openhands browser testids main` lists `add-llm-profile`, `add-models-from-provider` and `profiles-empty`.- **Open the editor (`F10.create`).** Run `control-openhands browser click 'testid=add-llm-profile'`. `control-openhands browser text 'testid=profile-editor-title'` is `Add LLM Profile`, `control-openhands browser text 'testid=profile-editor-description'` is `Configure your LLM settings below, then click Save to create this profile.`, and the form opens on the Basic tab with provider `OpenAI` and model `gpt-5.6-sol` (`browser value 'testid=llm-provider-input'`, `browser value 'testid=llm-model-input'`).- **API key help link (`F10.api-key-help`).** In that editor run `control-openhands browser click 'testid=sdk-section-basic-toggle'` (a new editor opens on All once the agent's LLM settings carry a non-default option; see `F10.create-prefill`), then `control-openhands browser text 'testid=llm-api-key-help-anchor'` is `Don't know your API key? Click here for instructions`; its link has `target` `_blank` (`browser eval "document.querySelector('[data-testid=llm-api-key-help-anchor] a, a[data-testid=llm-api-key-help-anchor]').target"`). Run `control-openhands browser click 'testid=llm-api-key-help-anchor >> role=link'`, then `control-openhands browser tabs` (rerun once if it still lists one page): page 1 is `https://docs.openhands.dev/.../local-setup#getting-an-api-key`. Return with `control-openhands browser close-tab 1` and `control-openhands browser tab 0`. The Advanced tab shows the same link as `testid=llm-api-key-help-anchor-advanced`.-- **Provider and model lists (`F10.model-picker`).** Run `control-openhands browser click 'testid=llm-provider-input'` and `control-openhands browser snapshot 'role=listbox'`: groups `Verified Models` (OpenHands, Anthropic, OpenAI, Mistral AI, Gemini, deepseek, Moonshot, minimax, glm, nvidia, qwen) and `Other Models`. Pick DeepSeek with `control-openhands browser fill 'testid=llm-provider-input' deepseek` and `control-openhands browser click 'testid=provider-item-deepseek'`; `browser value 'testid=llm-model-input'` is now empty. `control-openhands browser click 'testid=llm-model-input'` and `browser snapshot 'role=listbox'` list `deepseek-chat`, `deepseek-v4-pro`, … as Verified and `deepseek-coder`, `deepseek-flash`, … under Other. For the OpenHands provider (`browser fill 'testid=llm-provider-input' openhands`, `browser click 'testid=provider-item-openhands'`), `control-openhands browser text 'testid=openhands-account-help'` is `Need an OpenHands Account? Click here`; after `control-openhands browser click 'role=option[name="claude-sonnet-5"][exact]'` in the model list, `control-openhands browser text 'testid=openhands-api-key-help'` starts `You can find your OpenHands Provider LLM Key in the OpenHands LLM Key section of OpenHands Cloud`. Screenshot: `browser screenshot --feature F10.model-picker --name openhands-provider`.-- **Every provider is offered (`F10.provider-list-complete`).** Run `control-openhands browser click 'testid=cancel-profile-btn'` (the Provider and model lists bullet leaves its editor open; skip this when the list is showing), `control-openhands browser click 'testid=add-llm-profile'`, `control-openhands browser click 'testid=sdk-section-basic-toggle'` and `control-openhands browser click 'testid=llm-provider-input'`. `control-openhands browser count 'role=listbox >> role=option'` is the size of the union of the two backend lists the page merges: the names in `control-openhands api GET /api/llm/providers --pick providers` (under `value`; `control-openhands api GET /api/llm/providers --pick providers | jq '.value | length'` is `160` today) and the provider keys of `control-openhands api GET /api/llm/models/verified --max-bytes 200000` (its `body` is `{"models": {<provider>: [<model>, …]}}`; `control-openhands api GET /api/llm/models/verified --max-bytes 200000 | jq '.body.models | keys | length'` is `12` today, four of which, `glm`, `nvidia`, `openhands` and `qwen`, are missing from the first list). The union is `jq -s '([.[0].value[]] + (.[1].body.models | keys)) | unique | length' <(control-openhands api GET /api/llm/providers --pick providers) <(control-openhands api GET /api/llm/models/verified --max-bytes 200000)` (bash process substitution): `164` today, and the listbox count is the same `164`, with every backend name present under its display label (`openhands` as `OpenHands`, `mistral` as `Mistral AI`, `azure_ai` as `Azure AI Studio`). `control-openhands browser count 'role=option[name="xai"][exact]'` is `1` (it sorts last but three, before `xiaomi_mimo`, `xinference` and `zai`). Pick it with `control-openhands browser click 'role=option[name="xai"][exact]'`: `control-openhands browser value 'testid=llm-provider-input'` is `xai`, `control-openhands browser value 'testid=llm-model-input'` is empty, and after `control-openhands browser click 'testid=llm-model-input'` the listbox offers `grok-4.20`, `grok-4.20-0309`, … under Other Models (57 today; `control-openhands browser screenshot --feature F10.provider-list-complete --name xai-picked`). Close with `control-openhands browser press Escape` (the open listbox would swallow the next click) and `control-openhands browser click 'testid=cancel-profile-btn'`; the list is back.+- **Provider and model lists (`F10.model-picker`).** Run `control-openhands browser click 'testid=llm-provider-input'` and `control-openhands browser snapshot 'role=listbox'`: groups `Verified Models` and `Other Models`. `control-openhands browser eval "[...document.querySelectorAll('[role=listbox] [role=option]')].map(e=>e.innerText.trim().split('\n')[0]).join('|')"` starts `OpenHands|Anthropic|deepseek|digitalocean|Gemini|glm|minimax|Mistral AI|Moonshot|nvidia|OpenAI|OpenRouter|qwen` (the verified ones, OpenHands first, then by shown name) and continues `a2a|a2a_agent|AI21|ai21_chat|…` with `AWS Bedrock` and `Azure` among the `a`s: Other is sorted by shown name too, ignoring case. Typing filters: run `control-openhands browser press Escape`, `control-openhands browser click 'testid=profile-editor-title'`, `control-openhands browser click 'testid=llm-provider-input'` (`browser eval "(()=>{const i=document.querySelector('[data-testid=llm-provider-input]');return i.selectionStart+'-'+i.selectionEnd+' of '+i.value.length})()"` is `0-6 of 6`: the label `OpenAI` is selected), then `control-openhands browser press a`, `press n`, `press t`, `press h`: `browser value 'testid=llm-provider-input'` is `anth` and the same eval lists `Anthropic|anthropic_text` (`browser screenshot --feature F10.model-picker --name anth-filtered`). Use key presses here: `browser type` appends to the selected label (`OpenAIanth`). Close with `control-openhands browser press Escape`. Pick DeepSeek with `control-openhands browser fill 'testid=llm-provider-input' deepseek` and `control-openhands browser click 'testid=provider-item-deepseek'`; `browser value 'testid=llm-model-input'` is now empty. `control-openhands browser click 'testid=llm-model-input'` and `browser snapshot 'role=listbox'` list `deepseek-chat`, `deepseek-v4-pro`, … as Verified and `deepseek-coder`, `deepseek-flash`, … under Other. For the OpenHands provider (`browser fill 'testid=llm-provider-input' openhands`, `browser click 'testid=provider-item-openhands'`), `control-openhands browser text 'testid=openhands-account-help'` is `Need an OpenHands Account? Click here`; after `control-openhands browser click 'role=option[name="claude-sonnet-5"][exact]'` in the model list, `control-openhands browser text 'testid=openhands-api-key-help'` starts `You can find your OpenHands Provider LLM Key in the OpenHands LLM Key section of OpenHands Cloud`. Screenshot: `browser screenshot --feature F10.model-picker --name openhands-provider`.+- **Every provider is offered (`F10.provider-list-complete`).** Run `control-openhands browser click 'testid=cancel-profile-btn'` (the Provider and model lists bullet leaves its editor open; skip this when the list is showing), `control-openhands browser click 'testid=add-llm-profile'`, `control-openhands browser click 'testid=sdk-section-basic-toggle'` and `control-openhands browser click 'testid=llm-provider-input'`. `control-openhands browser count 'role=listbox >> role=option'` is the size of the union of the two backend lists the page merges: the names in `control-openhands api GET /api/llm/providers --pick providers` (under `value`; `control-openhands api GET /api/llm/providers --pick providers | jq '.value | length'` is `165` today) and the provider keys of `control-openhands api GET /api/llm/models/verified --max-bytes 200000` (its `body` is `{"models": {<provider>: [<model>, …]}}`; `control-openhands api GET /api/llm/models/verified --max-bytes 200000 | jq '.body.models | keys | length'` is `13` today, five of which, `digitalocean`, `glm`, `nvidia`, `openhands` and `qwen`, are missing from the first list). The union is `jq -s '([.[0].value[]] + (.[1].body.models | keys)) | unique | length' <(control-openhands api GET /api/llm/providers --pick providers) <(control-openhands api GET /api/llm/models/verified --max-bytes 200000)` (bash process substitution): `170` today, and the listbox count is the same `170`, with every backend name present under its display label (`openhands` as `OpenHands`, `mistral` as `Mistral AI`, `azure_ai` as `Azure AI Studio`). `control-openhands browser count 'role=option[name="xai"][exact]'` is `1` (it sorts last but three, before `xiaomi_mimo`, `xinference` and `zai`). Pick it with `control-openhands browser click 'role=option[name="xai"][exact]'`: `control-openhands browser value 'testid=llm-provider-input'` is `xai`, `control-openhands browser value 'testid=llm-model-input'` is empty, and after `control-openhands browser click 'testid=llm-model-input'` the listbox offers `grok-4.20`, `grok-4.20-0309`, … under Other Models (60 today; `control-openhands browser screenshot --feature F10.provider-list-complete --name xai-picked`). Close with `control-openhands browser press Escape` (the open listbox would swallow the next click) and `control-openhands browser click 'testid=cancel-profile-btn'`; the list is back.- **Cloud provider pagination (`F10.cloud-provider-pagination`).** Blocked: needs a Cloud organization backend where the user can manage profiles (local backends serve the whole list in one `GET /api/llm/providers`). There, the Basic provider list comes from `providers/search` one page at a time (`control-openhands browser network --filter providers/search` lists one request per page, the later ones with a `page_id`), and `xai` and `openrouter`, which sort past the first page, are offered and pick like any other provider. Record `control-openhands evidence add --feature F10.cloud-provider-pagination --result blocked ...` naming the Cloud account.- **Create the first profile (`F10.create`, `F10.name-autofill`).** In a fresh editor pick deepseek as above, then `control-openhands browser click 'testid=llm-model-input'` and `control-openhands browser click 'testid=model-item-deepseek-flash'`. `control-openhands browser value 'testid=profile-name-input'` now follows the model: `deepseek-flash` (right after the provider pick, while the model is still empty, it still reads `gpt-5.6-sol`). Run `control-openhands browser fill 'testid=profile-name-input' QA_flash`, `control-openhands browser fill 'testid=llm-api-key-input' --value-file <key file>` (or `--value-env DEEPSEEK_API_KEY`), `control-openhands browser click 'testid=save-profile-btn' --observe 'testid=save-profile-btn' --observe-ms 4000` and `control-openhands browser wait-text 'Profile "QA_flash" created' --timeout 60000`. The observed button states are `Save`, `Validating...`, `Saving...`, then the editor closes. After `control-openhands browser reload`, `control-openhands browser text '[data-testid=profile-row]:has([title="QA_flash"])'` reads `QA_flash`, `deepseek/deepseek-flash`, `Default`, and `control-openhands api GET /api/profiles` has `active_profile` `QA_flash`.- **A typed name sticks; a cleared one follows again (`F10.name-autofill`).** In a fresh editor (`control-openhands browser click 'testid=add-llm-profile'`) run `control-openhands browser fill 'testid=profile-name-input' QA_typed`, then pick deepseek as above, `control-openhands browser click 'testid=llm-model-input'` and `control-openhands browser click 'role=option[name="deepseek-v4-pro"][exact]'`; `browser value 'testid=profile-name-input'` stays `QA_typed`. Run `control-openhands browser fill 'testid=profile-name-input' ''`, then `browser click 'testid=llm-model-input'` and `browser click 'testid=model-item-deepseek-flash'`: the name is `deepseek-flash`; pick `role=option[name="deepseek-v4-pro"][exact]` again and it is `deepseek-v4-pro`.@@ -80,7 +80,7 @@ Preconditions:- **Custom model (`F10.custom-model`).** In that editor run `control-openhands browser click 'testid=sdk-section-advanced-toggle'`. `browser value 'testid=llm-custom-model-input'` carries the Basic choice (`deepseek/deepseek-flash`; placeholder `openai/gpt-5.6-sol`), and `browser attr 'testid=base-url-input' placeholder` is `https://api.openai.com`. Run `control-openhands browser fill 'testid=llm-custom-model-input' ''`, `control-openhands browser fill 'testid=profile-name-input' QA_adv`, `control-openhands browser click 'testid=save-profile-btn'` and `control-openhands browser wait-text 'Model is required'`. Then `control-openhands browser fill 'testid=llm-custom-model-input' deepseek/deepseek-flash`, `control-openhands browser fill 'testid=base-url-input' https://api.deepseek.com`, fill the key as in Create, Save, and `control-openhands browser wait-text 'Profile "QA_adv" created' --timeout 60000`. `api GET /api/profiles` shows `QA_adv` with `base_url` `https://api.deepseek.com`.- **Second profile from a verified model.** Repeat Create choosing `control-openhands browser click 'role=option[name="deepseek-v4-pro"][exact]'` (verified models have no `model-item-*` test ID) and the name `QA_pro`. The toast reads `Profile "QA_pro" created`; the row `QA_pro deepseek/deepseek-v4-pro` has no badge.- **Row menu (`F10.actions-menu`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_flash"]) >> testid=profile-menu-trigger'` and `control-openhands browser snapshot 'testid=profile-actions-menu'`: menuitems Edit, Rename, Duplicate, `Set as default [disabled]`, Delete on the Default row (all enabled on other rows). `control-openhands browser eval "document.activeElement.dataset.testid"` is `profile-edit`: focus moves into the menu on open. Run `control-openhands browser press ArrowDown` and that eval four times: `profile-rename`, `profile-duplicate`, `profile-delete` (the disabled Set as default is skipped), `profile-edit` (wraps); `control-openhands browser press ArrowUp` twice gives `profile-delete`, `profile-duplicate`. `control-openhands browser press Tab` closes the menu (`browser count 'testid=profile-actions-menu'` is `0`) and moves focus on to the next row's trigger: the row-aware eval `control-openhands browser eval "(()=>{const a=document.activeElement;const r=a.closest('[data-testid=profile-row]');return a.dataset.testid+'@'+(r?r.querySelector('[title]').title:'-')})()"` is `profile-menu-trigger@QA_pro`. Reopen the `QA_flash` menu and `control-openhands browser press Escape`: count `0`, and the row-aware eval is `profile-menu-trigger@QA_flash` (focus is back on its own trigger). Reopen it and `control-openhands browser click 'role=heading[name="Available Profiles"]'` closes it too. Keyboard open: `control-openhands browser focus '[data-testid=profile-row]:has([title="QA_pro"]) >> testid=profile-menu-trigger'` and `control-openhands browser press Enter` put focus on `profile-edit`; on this non-default row `control-openhands browser snapshot 'testid=profile-actions-menu'` lists the same five menuitems with none `[disabled]`, and `control-openhands browser screenshot --feature F10.actions-menu --name menu-nondefault` shows them with the focus ring on Edit. Five `ArrowDown`s walk `profile-rename`, `profile-duplicate`, `profile-set-active`, `profile-delete`, `profile-edit`. `control-openhands browser press Escape` closes it, and the row-aware eval is `profile-menu-trigger@QA_pro`.-- **Set as default (`F10.set-default`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_pro"]) >> testid=profile-menu-trigger'`, `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-set-active'` and `control-openhands browser wait-text 'Switched to profile "QA_pro"'`. After `control-openhands browser reload`, `control-openhands browser count 'testid=profile-active-badge'` is `1` and `control-openhands browser count '[data-testid=profile-row]:has([title="QA_pro"]) >> testid=profile-active-badge'` is `1`; `control-openhands api GET /api/profiles` has `active_profile` `QA_pro` and `control-openhands llm show` reports `active.model` `deepseek/deepseek-v4-pro`. A profile's Base URL follows it into the agent settings: `QA_adv` (Custom model above) carries one: Set as active on its row makes `control-openhands llm show` report `active.base_url` `https://api.deepseek.com`, also under `agent_settings.llm.base_url` in `control-openhands api GET /api/settings`. With a key, `control-openhands conversation start --prompt "Reply with only: adv-ok. Do not run any tools." --wait --timeout 240` then shows the pill `QA_adv` and the reply `adv-ok` (driven 2026-10-08; `QA_pro` likewise replied `pro-ok` on `deepseek/deepseek-v4-pro`). Make `QA_pro` the Default again before Duplicate.+- **Set as default (`F10.set-default`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_pro"]) >> testid=profile-menu-trigger'`, `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-set-active'` and `control-openhands browser wait-text 'Switched to profile "QA_pro"'`. After `control-openhands browser reload`, `control-openhands browser count 'testid=profile-active-badge'` is `1` and `control-openhands browser count '[data-testid=profile-row]:has([title="QA_pro"]) >> testid=profile-active-badge'` is `1`; `control-openhands api GET /api/profiles` has `active_profile` `QA_pro` and `control-openhands llm show` reports `active.model` `deepseek/deepseek-v4-pro`. A profile's Base URL follows it into the agent settings: `QA_adv` (Custom model above) carries one: Set as active on its row makes `control-openhands llm show` report `active.base_url` `https://api.deepseek.com`, also under `agent_settings.llm.base_url` in `control-openhands api GET /api/settings`. The `default` agent profile follows the Default: `control-openhands api GET /api/agent-profiles/default --pick profile.llm_profile_ref` names the profile just set (`F13`). With a key, `control-openhands conversation start --prompt "Reply with only: adv-ok. Do not run any tools." --wait --timeout 240` then shows the pill `QA_adv` and the reply `adv-ok` (driven 2026-10-08; `QA_pro` likewise replied `pro-ok` on `deepseek/deepseek-v4-pro`). Make `QA_pro` the Default again before Duplicate.- **Duplicate (`F10.duplicate`).** From the `QA_pro` row menu click `testid=profile-actions-menu >> testid=profile-duplicate` and `control-openhands browser wait-text 'Profile duplicated as "QA_pro-copy"'`; repeat on `QA_pro` for `Profile duplicated as "QA_pro-copy-1"`, and on `QA_pro-copy` for `Profile duplicated as "QA_pro-copy-copy"`. After a reload `control-openhands browser text 'testid=profile-row >> has-text=QA_pro-copy-1'` reads `QA_pro-copy-1 deepseek/deepseek-v4-pro`, the badge stays on `QA_pro`, and `api GET /api/profiles` has `api_key_set: true` on every copy.- **Rename (`F10.rename`).** Open the `QA_pro-copy-copy` row menu and click `testid=profile-actions-menu >> testid=profile-rename`. `control-openhands browser value 'testid=rename-profile-input'` is `QA_pro-copy-copy` and `browser eval "document.activeElement.dataset.testid"` is `rename-profile-input`. `control-openhands browser fill 'testid=rename-profile-input' '-bad'` makes `control-openhands browser enabled 'role=dialog[name="Rename Profile"] >> role=button[name="Rename"]'` `false` and `control-openhands browser attr 'testid=rename-profile-rule' class` contains `text-danger` (the theme danger colour, #18199). Filling the old name and clicking Rename closes the modal with no toast (`browser count 'testid=rename-profile-modal'` is `0`); `control-openhands browser click 'role=dialog[name="Rename Profile"] >> role=button[name="Cancel"]'` also closes it. To rename, reopen, fill `QA_renamed`, `control-openhands browser press Enter`, `control-openhands browser wait-text 'Profile renamed to "QA_renamed"'`, reload, and the count of `'[data-testid=profile-row]:has([title="QA_renamed"])'` is `1` while the old name's is `0`. Renaming the Default row (`QA_pro` → `QA_pro2`, submit with `testid=rename-profile-submit`) keeps its badge (`browser text '[data-testid=profile-row]:has([data-testid=profile-active-badge])'` reads `QA_pro2 … Default`) and `active_profile` follows.- **Rename onto an existing name (`F10.rename-conflict`).** The Rename step above already renamed `QA_pro-copy-copy` and `QA_pro`, so use the remaining fixtures: open the `QA_pro-copy` row menu, click `testid=profile-actions-menu >> testid=profile-rename`, fill `testid=rename-profile-input` with `QA_pro2` and click `testid=rename-profile-submit`. The modal stays open (`count 'testid=rename-profile-modal'` is `1`), `control-openhands browser toasts` has the server's message `Profile 'QA_pro2' already exists` (no `HTTP request failed` prefix, no JSON), and `browser screenshot --feature F10.rename-conflict --name toast` shows that toast above the open modal. Close it with `control-openhands browser click 'role=dialog[name="Rename Profile"] >> role=button[name="Cancel"]'`.@@ -94,7 +94,7 @@ Preconditions:- **Add models from a provider (`F10.add-models`).** Run `control-openhands browser click 'testid=add-models-from-provider'`; `browser enabled 'testid=add-models-submit'` is `false`. Keyless: `control-openhands browser select 'testid=add-models-provider' moonshot` and `control-openhands browser wait 'testid=add-models-select-all'`; `browser text 'testid=add-models-keyless-note'` is `No connection — profiles will need a key added later` (readable, at least 4.5:1, in every theme; Known failure (reproduced 2026-10-17, [#18546](https://github.com/OpenHands/OpenHands/issues/18546)): the muted note measures 2.58:1 in Solarized Light, 5.58:1 in Light+), rows start unchecked (`Add 0 profiles`), and `control-openhands browser check 'testid=add-models-select-all'` makes `browser text 'testid=add-models-submit'` read `Add 12 profiles`; uncheck it again. Linked: `control-openhands browser select 'testid=add-models-provider' deepseek`; the connection field (`testid=add-models-connection-field`, options `No connection (keyless)` and `QA_ds_conn`) is pre-bound to `QA_ds_conn`. Run `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-chat'`, `control-openhands browser check 'testid=add-models-check-deepseek/deepseek-v3'`, `control-openhands browser click 'testid=add-models-submit'` and `control-openhands browser wait-text 'Added 2'` (toast `Added 2 profiles`). The modal closes; after a reload `deepseek-chat` and `deepseek-v3` sit under the `QA_DS_CONN` header and `api GET /api/profiles` shows both with a `provider_connection_id`. Preselect entry: `control-openhands browser click 'testid=provider-connection-row >> has-text=QA_ds_conn >> testid=provider-connection-menu-trigger'` (menu: Bulk add, Edit, Delete), `control-openhands browser click 'testid=provider-connection-actions-menu >> testid=provider-connection-add-models'`; provider `deepseek` and the connection are preset, every remaining row is checked (`Add 10 profiles`), and `browser count 'testid=add-models-row-deepseek/deepseek-chat'` is `0` (already added). Close with `control-openhands browser click 'role=dialog[name="Add models as profiles"] >> role=button[name="Cancel"]'`. In Light+ and Solarized Light the chooser stays readable: the provider select, labels, Select all and model names, and a partial save's per-row **Saved** and **Failed** outcomes all measure at least 4.5:1 against the modal (arrange a partial save by filling the profile list to 48 through the real profiles API, then add four keyless moonshot rows including `moonshot-v1-8k`, whose 8,192-token window the server refuses, and `moonshot-v1-auto`, which hits the 50-profile limit; the toast reads `Added 2 profiles; 2 failed` and the modal stays open).- **Broken link (`F10.broken-link`).** The local server refuses to delete a referenced connection (row menu → `testid=provider-connection-delete` → `testid=delete-provider-connection-confirm` toasts `Provider connection cannot be deleted while it is referenced by LLM profile(s): ...`), so arrange the state directly: `control-openhands api POST /api/profiles/QA_broken --write --data '{"llm":{"model":"deepseek/deepseek-chat","provider_connection_id":"00000000000000000000000000000000"},"include_secrets":false}'`. Then `control-openhands browser reload` and `control-openhands browser text '[data-testid=profile-row]:has([title="QA_broken"])'` reads `QA_broken deepseek/deepseek-chat Broken link`; `control-openhands browser attr '[data-testid=profile-row]:has([title="QA_broken"]) >> testid=profile-broken-connection-badge' title` is `The linked provider connection was deleted. Edit this profile to re-link it.` (screenshot `--feature F10.broken-link --name badge`). Edit it: `browser value 'testid=llm-provider-connection-input'` is the orphaned id and the listbox offers `None`, `QA_ds_conn` and that id. The row's group header shows the raw id, not a name (see Gotchas).- **ChatGPT subscription, pre-auth (`F10.subscription`).** In a fresh editor run `control-openhands browser click 'testid=sdk-section-basic-toggle'` first (after the schema step the Default profile has a temperature, so a new editor opens on All; see `F10.create-prefill`), then `control-openhands browser click 'testid=llm-auth-type-input'` (options `API key`, `ChatGPT subscription`) and `control-openhands browser click 'role=option[name="ChatGPT subscription"]'`. `browser text 'testid=subscription-status'` is `ChatGPT subscription not connected`, `browser value 'testid=llm-subscription-model-input'` is `gpt-5.5` (list: gpt-5.5, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, gpt-6-astra), and `llm-provider-input` and `llm-api-key-input` are gone (count `0`). Known failure (reproduced 2026-10-13, #17135): run `control-openhands browser fill 'testid=profile-name-input' QA_sub`, `control-openhands browser network --clear`, `control-openhands browser click 'testid=save-profile-btn'` and `control-openhands browser network --bodies --filter '/api/profiles'`. Expected: Save validates and saves the shown `gpt-5.5`. Actual: one `POST /api/profiles/QA_sub/validate` whose body has `"model":"openai/gpt-5.6-sol"`, the API-key form's hidden default, although the dropdown shows `gpt-5.5` (it shows the first subscription model without writing it to the form); `browser toasts --history` has `OpenAI subscription login is required`, because no subscription is connected. Picking a model in `testid=llm-subscription-model-input`, even `gpt-5.5` again, makes the next Save send `POST /api/profiles/QA_sub/validate`, which answers `OpenAI subscription login is required` here (`browser toasts --history`), because no subscription is connected. `control-openhands browser click 'testid=subscription-connect'` and `control-openhands browser wait 'testid=subscription-user-code'`: the card shows the code, `Open sign-in page`, `I've finished signing in` (`subscription-poll`) and Cancel, and `control-openhands browser tabs` lists a second page on `auth.openai.com` (it opens a second or two after the code appears; rerun `browser tabs` if it lists one page). The card polls on its own, so `Sign-in is not complete yet. ...` can already show before you click poll. Close it with `control-openhands browser close-tab 1` and `control-openhands browser tab 0`. `control-openhands browser click 'testid=subscription-poll'` leaves `Sign-in is not complete yet. Finish the browser flow and try again.` on the card, and `control-openhands browser click 'testid=subscription-cancel'` removes the code (`count 'testid=subscription-user-code'` `0`). Switching back with `role=option[name="API key"]` restores provider `OpenAI` and model `gpt-5.6-sol`. Completing sign-in is blocked (needs a ChatGPT Plus/Pro account).-- **Delete (`F10.delete`, `F10.delete-default`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_pro-copy"]) >> testid=profile-menu-trigger'` and `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-delete'`. `control-openhands browser snapshot 'role=dialog'` shows `Delete Profile` with `Are you sure you want to delete the profile "QA_pro-copy"? This action cannot be undone.`, and `browser eval "document.activeElement.innerText"` is `Cancel`. `control-openhands browser click 'role=dialog[name="Delete Profile"] >> role=button[name="Cancel"]'` keeps the row (count `1`). Repeat, then `control-openhands browser click 'testid=delete-profile-confirm'`, `control-openhands browser wait-text 'Profile "QA_pro-copy" deleted'`, reload, and the count is `0`. Deleting the Default row the same way leaves exactly one `profile-active-badge` on another profile (`browser text '[data-testid=profile-row]:has([data-testid=profile-active-badge])'`), and `api GET /api/profiles` names it in `active_profile`.+- **Delete (`F10.delete`, `F10.delete-default`).** Run `control-openhands browser click '[data-testid=profile-row]:has([title="QA_pro-copy"]) >> testid=profile-menu-trigger'` and `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-delete'`. `control-openhands browser snapshot 'role=dialog'` shows `Delete Profile` with `Are you sure you want to delete the profile "QA_pro-copy"? This action cannot be undone.`, and `browser eval "document.activeElement.innerText"` is `Cancel`. `control-openhands browser click 'role=dialog[name="Delete Profile"] >> role=button[name="Cancel"]'` keeps the row (count `1`). Repeat, then `control-openhands browser click 'testid=delete-profile-confirm'`, `control-openhands browser wait-text 'Profile "QA_pro-copy" deleted'`, reload, and the count is `0`. Deleting the Default row the same way leaves exactly one `profile-active-badge` on another profile (`browser text '[data-testid=profile-row]:has([data-testid=profile-active-badge])'`), and `api GET /api/profiles` names it in `active_profile`. Run `control-openhands browser network --clear` before that confirm and `control-openhands browser network --filter /api/` after it: `POST /api/agent-profiles/default` (moving the `default` agent profile onto the profile with a key that will become Default) comes before `DELETE /api/profiles/<name>` `200` and `POST /api/profiles/<next>/activate`; `control-openhands api GET /api/agent-profiles/default --pick profile.llm_profile_ref` is the new Default (`browser screenshot --feature F10.delete-default --name promoted`).- **Command menu and direct URL (`F10.list`).** Run `control-openhands browser goto /` (the bullets above end on `/settings/llm`, where the URL wait would pass at once), `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' LLM` and `control-openhands browser snapshot 'testid=command-menu'`: two options, `LLM profiles` ("Manage models, providers, and API keys.") and `Model Router` (F12; its keywords include "llm"). Hovering an option selects it, so which one Enter runs depends on where the mouse rests: click the option instead, `control-openhands browser click 'testid=command-menu >> role=option[name="LLM profiles"]' --expect-url '/settings/llm(\?|$)'`. `browser count 'testid=add-llm-profile'` is `1` on arrival. `control-openhands browser goto /settings/llm` shows the same list (`browser count 'testid=profile-row'`, one `profile-active-badge`; screenshot `--feature F10.list --name list`).- **Phone layout (`F10.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/llm`, `control-openhands browser bbox 'testid=add-llm-profile'`, `control-openhands browser bbox 'testid=profile-row >> nth=0'` and `control-openhands browser screenshot --feature F10.phone --name list`: `insideViewport` `true`, `pageHorizontalOverflow` `false`. Open a row menu: `browser bbox 'testid=profile-actions-menu'` is inside the viewport. Open Edit: `browser bbox 'testid=profile-name-input'` is 348 px wide inside the 390 px viewport and `pageHorizontalOverflow` is `false` (`llm-settings-screen` may report `insideViewport` `false` when the edited profile's form is taller than the screen; only the overflow flag matters). Return with `control-openhands browser click 'testid=cancel-profile-btn'` and `control-openhands browser viewport desktop`.- **Default profile drives a conversation (`F10.default-used`).** Make a profile with a distinct model Default through the row menu (for example `QA_pro-copy-1`, `deepseek/deepseek-v4-pro`, then `wait-text 'Switched to profile "QA_pro-copy-1"'`), and run `control-openhands conversation start --prompt "Reply with exactly the word: pong" --wait --timeout 180`. The JSON has `"status": "finished"` and `"model": "deepseek/deepseek-v4-pro"`. The browser is now on the conversation page: `control-openhands browser goto /settings/llm` before driving this family again.@@ -190,14 +190,13 @@ control-openhands assert --field toasts.0.text --matches 'configured model.*16,0- A new profile keeps the inherited LLM options the editor shows, such as temperature.- `Subscription models are not loaded yet.` uses the translated `SETTINGS$SUBSCRIPTION_MODELS_NOT_LOADED` copy in settings and onboarding. It shows when a subscription profile is saved before `GET /api/llm/subscription/openai/models` answers; checking that branch needs the real request to be pending or failed.- Known issue [#17785](https://github.com/OpenHands/OpenHands/issues/17785): Saving DeepSeek-v4-flash on Windows Desktop can fail with missing cache_creation_tokens instead of saving. The reported path needs a Windows Desktop 1.24.0 environment with its SDK dependency resolution and an approved DeepSeek key; this macOS source run cannot establish that result. (`F10.create-validation`).-- Known issue [#17706](https://github.com/OpenHands/OpenHands/issues/17706): A freshly resolved npm-launcher environment can reject openhands/claude-* conversations because LiteLLM sends prompt_cache_key. Reproduction needs an approved OpenHands proxy key with an Anthropic model and the affected fresh transitive dependency resolution; a keyless initialization cannot prove provider acceptance. (`F10.default-used`).- Known issue [#17391](https://github.com/OpenHands/OpenHands/issues/17391): ChatGPT Business subscription login can be refused or use a fallback model with a 404. Completing the entitlement and selected-model checks needs an approved ChatGPT Business workspace login and the reported Docker subscription runtime. (`F10.subscription`).- Known issue [OpenHands/software-agent-sdk#5181](https://github.com/OpenHands/software-agent-sdk/issues/5181): The SDK can turn an advertised output limit into a requested default cap and silently drop an explicitly enabled reasoning_effort. The narrowed maintainer scope needs an authenticated OpenRouter/proxy route exposing lower-limit endpoints and capability metadata, or a real endpoint accepting the explicit override; a synthetic empty-metadata response would not establish production behavior. (`F10.schema-validation`).- Known issue [OpenHands/software-agent-sdk#4934](https://github.com/OpenHands/software-agent-sdk/issues/4934): Switching to a MiniMax M3 profile can fail because the SDK sends unsupported reasoning_effort. The runtime switch needs approved working credentials for the starting model and MiniMax route; the reported provider rejection cannot be proved with a loopback connection error. (`F10.schema-validation`, `F10.default-used`).- Known issue [OpenHands/software-agent-sdk#4613](https://github.com/OpenHands/software-agent-sdk/issues/4613): A namespaced model such as openai/openai/example-model can have its inner provider prefix stripped again in Responses API and reach the provider as example-model. Reproduction needs an approved OpenAI-compatible endpoint serving the namespaced model and its exact stored API-mode settings; a loopback connection refusal cannot prove the outbound identifier. (`F10.custom-model`).- Known issue [OpenHands/software-agent-sdk#4511](https://github.com/OpenHands/software-agent-sdk/issues/4511): Disabling prompt caching can still emit cache_control on preexisting cache-marked tool messages. The report only supplies a Python serializer reproduction; a Canvas check needs the exact persisted/tool-message precondition and a provider request that exposes the cache markers, with approved model access. (`F10.schema-validation`).- Known issue [OpenHands/software-agent-sdk#4247](https://github.com/OpenHands/software-agent-sdk/issues/4247): An LM Studio configuration can fail with LLM Provider NOT provided although its models endpoint works. Reproduction needs the reachable LM Studio service and the exact model ID/provider-qualified config, which the report does not supply as text; the reported CLI/Docker SDK 1.16.1 installation is not this source stack. (`F10.custom-model`).-- Known issue [OpenHands/software-agent-sdk#5890](https://github.com/OpenHands/software-agent-sdk/issues/5890): Switching the conversation LLM can lose the managed-key refresh-on401 hook. Blocked: needs an approved managed-key refresh source and an allow-listed real model endpoint that returns 401 and accepts the refreshed key. Refused loopback cannot establish the refresh/retry outcome; do not supply a fake successful completion. (`F10.edit`, `F10.default-used`).+- Known issue [OpenHands/software-agent-sdk#5890](https://github.com/OpenHands/software-agent-sdk/issues/5890): Switching the conversation LLM can lose the managed-key refresh-on401 hook. The fix ([OpenHands/software-agent-sdk#5904](https://github.com/OpenHands/software-agent-sdk/pull/5904)) is in no agent-server release up to the pinned 1.57.0; re-drive once the pin includes it. Blocked: needs an approved managed-key refresh source and an allow-listed real model endpoint that returns 401 and accepts the refreshed key. Refused loopback cannot establish the refresh/retry outcome; do not supply a fake successful completion. (`F10.edit`, `F10.default-used`).- Known issue [OpenHands/software-agent-sdk#5862](https://github.com/OpenHands/software-agent-sdk/issues/5862): DeepSeek invalid-key401 responses can be classified as bad-request errors. Blocked by this pass's loopback-only provider routing: a real invalid-key validation request to DeepSeek must be authorized to capture its actual 401 shape. No usable key or successful inference is needed, but a loopback connection error does not prove this classification. (`F10.create-validation`).- Known issue [OpenHands/software-agent-sdk#5838](https://github.com/OpenHands/software-agent-sdk/issues/5838): A live LLM switch during a running tool step can be acknowledged before the new LLM is persisted for GET/restart. Blocked: needs a genuine model-issued bounded tool step (for example `sleep 20`) and an approved starting model. Refused-loopback initialization never enters that step. (`F10.default-used`).- Known issue [#17806](https://github.com/OpenHands/OpenHands/issues/17806): A model-only Save must preserve an untouched key. Current SDK 1.55 refused-loopback validation rejects Save before persistence; the stored key marker remains set. Completing Basic/Advanced preserved-key and runtime-switch proof needs an approved route that accepts preflight. Keep dummy traffic on refused loopback until an external route is authorized; a rejected Save does not establish either key loss or successful preservation through Save. (`F10.edit`).diff --git a/.agents/skills/verify-openhands/references/feature-map/F12-model-router.md b/.agents/skills/verify-openhands/references/feature-map/F12-model-router.mdindex 3de57e1d6b..91ce87a956 100644--- a/.agents/skills/verify-openhands/references/feature-map/F12-model-router.md+++ b/.agents/skills/verify-openhands/references/feature-map/F12-model-router.md@@ -1,16 +1,19 @@# F12 — Model router (meta-LLM profiles)A model router (the UI also calls it a "meta-profile") names a classifier LLM-profile, a prompt template and a model table. While a router is active, every-new conversation gets a `route_task_to_model` tool: the classifier picks one-saved LLM profile from the table and the agent switches to it (the chat shows-`Switched to profile <name>`). With **Run on first message** on, the agent is-told to route the first user message before doing anything else. Users create-routers from the Router Pro / Router Flash templates or from scratch, can have-the missing router LLM profiles created from a provider connection, and-activate, edit and delete routers under Settings → Model Router.--Source: `src/routes/meta-llm-settings.tsx`, `src/components/features/settings/meta-llm-profiles/`, `src/api/meta-profiles-service/meta-profiles-service.api.ts`, `src/api/agent-server-adapter.ts` (`buildRouterAtStartSystemSuffix`), `src/constants/settings-nav.tsx`.+profile, a prompt template and a model table. On a local backend each OpenHands+agent profile picks its router in Settings → Agent (`F13.profile-router`); a+conversation launched from a profile with a router gets a `route_task_to_model`+tool: the classifier picks one saved LLM profile from the table and the agent+switches to it (the chat shows `Switched to profile <name>`). With **Run on first+message** on, the agent is told to route the first user message before doing+anything else. Users create routers from the Router Pro / Router Flash templates+or from scratch, can have the missing router LLM profiles created from a+provider connection, and edit and delete routers under Settings → Model Router.+Only an organization-bound Cloud backend still activates one router on this+page.++Source: `src/routes/meta-llm-settings.tsx`, `src/components/features/settings/meta-llm-profiles/`, `src/api/meta-profiles-service/meta-profiles-service.api.ts`, `src/api/agent-server-adapter.ts` (`buildRouterAtStartSystemSuffix`), `src/constants/settings-nav.tsx`, `src/hooks/mutation/use-delete-meta-profile.ts`.## Sub-features@@ -27,16 +30,16 @@ Source: `src/routes/meta-llm-settings.tsx`, `src/components/features/settings/me- `F12.add-connection-cancel`: **Cancel** in `Add provider connection` closes it without creating a connection and keeps the editor's values and picker choice.- `F12.editor-cancel`: while the create or edit editor is open it replaces the list, **Add Model Router** and the switch; **Cancel** returns to the list without saving anything.- `F12.router-profiles`: with a connection picked under **Create missing router LLM profiles using**, Save creates an LLM profile `<provider>/<name>` linked to that connection for every model-table name (and the classifier) that does not exist yet; **Don't create profiles** creates none.-- `F12.create-first`: saving the first router lists it, makes it Active and turns **Run on first message** on.-- `F12.create-more`: saving another router lists it and leaves the Active badge where it was.-- `F12.list`: each row shows the name, `<classifier> · Direct prompt`, an **Active** badge on the active router and a `...` menu; routers and the active choice survive a stack restart.-- `F12.row-menu`: the row menu offers Edit, Set active (disabled on the active row) and Delete, and closes on Escape.-- `F12.activate`: **Set active** toasts `Meta-profile "<name>" activated` and moves the badge.+- `F12.create-first`: saving the first router lists it and turns **Run on first message** on; locally no router becomes active (`active_meta_profile` stays `null`).+- `F12.create-more`: saving another router lists it and leaves **Run on first message** as it was.+- `F12.list`: each row shows the name, `<classifier> · Direct prompt` and a `...` menu (no **Active** badge locally; Cloud shows it on the active router); routers survive a stack restart.+- `F12.row-menu`: the row menu offers Edit and Delete (Cloud adds Set active, disabled on the active row), and closes on Escape.+- `F12.activate`: on an organization-bound Cloud backend, **Set active** toasts `Meta-profile "<name>" activated` and moves the badge. Local backends have no Set active (blocked here: needs a Cloud organization).- `F12.edit`: **Edit** opens `Edit meta-profile` with the name locked and no connection picker; changes persist.-- `F12.delete`: **Delete** asks `Are you sure you want to delete "<name>"? This cannot be undone.`; Cancel keeps the row, Delete removes it.-- `F12.delete-active`: deleting the active router leaves no router active; the switch shows off and disabled, and new conversations get neither the tool nor the first-message instruction.-- `F12.run-first-message-toggle`: the switch saves immediately (no Save button, no toast) and survives a reload.-- `F12.route-first-message`: with a router active and the switch on, a new conversation routes its first message (`route_task_to_model` action, `Switched to profile <name>` in the chat); with the switch off the tool is still attached but the instruction is absent.+- `F12.delete`: **Delete** asks `Are you sure you want to delete "<name>"? This cannot be undone.`; Cancel keeps the row, Delete removes it and detaches it from every agent profile that used it (`meta_profile_ref` `null`, routing off).+- `F12.delete-active`: deleting the last router shows the empty state and the switch off and disabled (the stored preference keeps its value); profiles that used it launch without the tool or the first-message instruction.+- `F12.run-first-message-toggle`: the switch is enabled while any router exists, saves immediately (no Save button, no toast) and survives a reload; its help reads `Route the first message of every new conversation through its agent profile's Model Router before the agent proceeds. Each agent profile picks its router in Settings → Agent.`+- `F12.route-first-message`: with the launched agent profile's router set and the switch on, a new conversation routes its first message (`route_task_to_model` action, `Switched to profile <name>` in the chat); with the switch off the tool is still attached but the instruction is absent.- `F12.light-theme`: in a light colour theme the list heading and router names are readable.- `F12.phone`: the list, template modal, editor and row menu fit a 390 px viewport without horizontal overflow.@@ -55,72 +58,39 @@ Preconditions:- `F12.empty-states` needs a fresh `launch --new` with `onboard --skip` done and **before** `llm preset deepseek` (no LLM profiles, no routers).- Everything after it needs `control-openhands llm preset deepseek --api-key-file <key file>` (profiles `deepseek-flash` and `deepseek-pro`), no meta-profile named `QA_*` (`control-openhands api GET /api/meta-profiles`), no provider connection (`control-openhands api GET /api/llm/provider-connections` is `[]`) and no LLM profile named `deepseek-chat` (`control-openhands api GET /api/profiles`).- `F12.add-connection` and `F12.route-first-message` use the DeepSeek key file; never print it.-- Blocked here: `F12.unsupported` (needs an agent server older than software-agent-sdk #3744, for example `launch --new --sdk-ref <older release>`), `F12.cloud` (needs a Cloud account, with and without an organization) and `F12.load-error` (needs `/api/meta-profiles` to fail while the rest of the agent server answers; `service stop agent-server` swaps the whole app for the Manage backends card instead).+- Blocked here: `F12.unsupported` (needs an agent server older than software-agent-sdk #3744, for example `launch --new --sdk-ref <older release>`), `F12.cloud` and `F12.activate` (need a Cloud account, with and without an organization) and `F12.load-error` (needs `/api/meta-profiles` to fail while the rest of the agent server answers; `service stop agent-server` swaps the whole app for the Manage backends card instead).- Desktop viewport unless a bullet says otherwise.- **Open from navigation (`F12.page`).** From `/` run `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings'`, `control-openhands browser click 'testid=sidebar-settings-/settings/meta-llm' --expect-url '/settings/meta-llm(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` and `control-openhands browser attr 'testid=sidebar-settings-/settings/meta-llm' aria-current`. The subtitle is `Meta-profiles that route each task to the right LLM profile.` and `aria-current` is `page`.- **Command menu (`F12.page`).** Run `control-openhands browser goto /` (the previous bullet ends on `/settings/meta-llm`, where the URL wait would pass at once), `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'Model Router'` and `control-openhands browser snapshot 'testid=command-menu'`: the only option is `Model Router Meta-profiles that route each task to the right LLM profile. Go` [selected]. `control-openhands browser fill 'testid=command-menu >> role=combobox' router` (and then `meta`) keeps that single option (`control-openhands browser count 'testid=command-menu >> role=option'` is `1`). Run `control-openhands browser press Enter` and `control-openhands browser wait-url '/settings/meta-llm(\?|$)'`; `browser text 'testid=settings-page-subtitle'` is the subtitle above.-- **Empty states (`F12.empty-states`).** On the fresh run run `control-openhands browser goto /settings/meta-llm`, `control-openhands browser text 'testid=meta-profile-no-llm-profiles'`, `control-openhands browser text 'testid=meta-profile-empty'` and `control-openhands browser eval "(()=>{const i=document.querySelector('[data-testid=meta-profile-run-at-conversation-start-switch]');return {checked:i.checked,disabled:i.disabled}})()"`. The texts are the two messages in Sub-features and the switch is `{checked:false, disabled:true}`. `control-openhands browser screenshot --feature F12.empty-states --name no-llm-profiles` shows both messages above a greyed switch.+- **Empty states (`F12.empty-states`).** On the fresh run run `control-openhands browser goto /settings/meta-llm`, `control-openhands browser text 'testid=meta-profile-no-llm-profiles'`, `control-openhands browser text 'testid=meta-profile-empty'` and `control-openhands browser eval "(()=>{const i=document.querySelector('[data-testid=meta-profile-run-at-conversation-start-switch]');return {checked:i.checked,disabled:i.disabled}})()"`. The texts are the two messages in Sub-features, the switch is `{checked:false, disabled:true}` and `browser text 'testid=meta-profile-run-at-conversation-start'` carries the per-profile help text (`F12.run-first-message-toggle`). `control-openhands browser screenshot --feature F12.empty-states --name no-llm-profiles` shows both messages above a greyed switch.- **Template chooser (`F12.template-chooser`).** Run `control-openhands browser click 'testid=add-meta-profile'` and `control-openhands browser snapshot 'role=dialog'`: dialog `Choose Model Router template` with buttons `Add OpenHands Router Pro`, `Add OpenHands Router Flash`, `Custom`, `Cancel`. `control-openhands browser click 'testid=meta-profile-template-cancel'` then `control-openhands browser count 'testid=meta-profile-template-modal'` is `0`; reopen and `control-openhands browser press Escape` gives `0` too. Reopen, `control-openhands browser click 'testid=meta-profile-template-router-pro'`, then `control-openhands browser value 'testid=meta-profile-name-input'` (`openhands-router-pro`), `control-openhands browser value 'testid=meta-profile-classifier-input'` (`minimax-m3`), `control-openhands browser eval "document.querySelector('[data-testid=meta-profile-model-table]').value.split('\n').length"` (`11`), `control-openhands browser eval "document.querySelector('[data-testid=meta-profile-prompt-template]').value.includes('{{ instance_text }}')"` (`true`), `control-openhands browser value 'testid=meta-profile-router-connection'` (`Don't create profiles` while no connection exists) and `control-openhands browser enabled 'testid=meta-profile-save'` (`true`). Leave with `control-openhands browser click 'testid=meta-profile-cancel'`. The Flash template (`testid=meta-profile-template-router-flash`) gives `openhands-router-flash`, the same classifier and identical table, and a different prompt (about 5,900 characters against Pro's 5,000; compare with `browser eval "document.querySelector('[data-testid=meta-profile-prompt-template]').value.length"`). A backdrop click also closes the chooser: reopen it and `control-openhands browser mouse-click 20 980`, then `count` is `0`.- **Custom template (`F12.template-custom`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-custom'`, then `control-openhands browser value` on `testid=meta-profile-name-input`, `testid=meta-profile-classifier-input`, `testid=meta-profile-prompt-template` and `testid=meta-profile-model-table` (all `""`), `control-openhands browser enabled 'testid=meta-profile-save'` (`false`) and `control-openhands browser eval "document.querySelector('[data-testid=meta-profile-model-table]').placeholder.split('\n')"`: two example rows, `["- GPT-5.4: strong for coding and reasoning", "- MiniMax-M3: efficient fallback"]`, joined by a real line break (#18035), and `control-openhands browser screenshot --feature F12.template-custom --name custom-empty` shows them on two lines under the prompt placeholder, which breaks lines the same way.- **Validation (`F12.editor-validation`).** In that Custom editor run `control-openhands browser fill 'testid=meta-profile-name-input' 'QA router!'`, `control-openhands browser fill 'testid=meta-profile-classifier-input' qa-classifier`, `control-openhands browser fill 'testid=meta-profile-prompt-template' 'Pick a model for: {{ instance_text }}'` and `control-openhands browser enabled 'testid=meta-profile-save'` (`false`: bad name). `control-openhands browser fill 'testid=meta-profile-name-input' QA_router` turns it `true`; `control-openhands browser fill 'testid=meta-profile-prompt-template' 'Pick a model for: {{ task }}'` turns it `false`; `'Pick a model for: {{instance_text}}'` turns it `true` again; `control-openhands browser fill 'testid=meta-profile-classifier-input' ''` turns it `false`. Keep the editor open.- **Add a connection (`F12.add-connection`).** Run `control-openhands browser click 'testid=meta-profile-add-provider-connection'`, `control-openhands browser fill 'testid=provider-connection-name-input' QA_deepseek`, `control-openhands browser fill 'testid=provider-connection-provider-input' deepseek`, `control-openhands browser click 'role=option[name="deepseek"]'`, `control-openhands browser fill 'testid=provider-connection-api-key-input' --value-file <key file>`, `control-openhands browser click 'testid=provider-connection-submit'` and `control-openhands browser wait 'testid=provider-connection-modal' --state detached`. `control-openhands browser toasts` has `Connection "QA_deepseek" created` and `control-openhands browser value 'testid=meta-profile-router-connection'` is `QA_deepseek (deepseek)`.-- **Create the first router (`F12.create-first`, `F12.router-profiles`).** Write the prompt and table to files: `printf 'You route tasks. Reply only with JSON {"model": "<name>", "reason": "<short>"} using one name from this table:\n{{ model_table }}\n\nTask:\n{{ instance_text }}' > <prompt file>` and `printf -- '- deepseek-flash: cheap, for simple tasks\n- deepseek-pro: stronger, for hard tasks\n- deepseek-chat: general chat' > <table file>`. Run `control-openhands browser click 'testid=meta-profile-classifier-input'`, `control-openhands browser click 'role=option[name="deepseek-flash"]'`, `control-openhands browser fill 'testid=meta-profile-name-input' QA_router`, `control-openhands browser fill 'testid=meta-profile-prompt-template' --value-file <prompt file>`, `control-openhands browser fill 'testid=meta-profile-model-table' --value-file <table file>`, `control-openhands browser click 'testid=meta-profile-save'` and `control-openhands browser wait-text 'Meta-profile "QA_router" saved'`. After `control-openhands browser reload`: `control-openhands browser text 'testid=meta-profile-row-QA_router'` is `QA_router\ndeepseek-flash · Direct prompt\nActive`, the switch `eval` above returns `{checked:true, disabled:false}`, `control-openhands api GET /api/meta-profiles` has `active_meta_profile` `QA_router`, `control-openhands api GET /api/settings` has `"run_router_at_conversation_start": true`, and `control-openhands api GET /api/profiles` has a new `deepseek-chat` (`deepseek/deepseek-chat`, `provider_connection_id` set) while `deepseek-flash`/`deepseek-pro` are unchanged. `control-openhands browser screenshot --feature F12.create-first --name list-after-create` shows the row with the gold Active badge and the switch on.+- **Create the first router (`F12.create-first`, `F12.router-profiles`).** Write the prompt and table to files: `printf 'You route tasks. Reply only with JSON {"model": "<name>", "reason": "<short>"} using one name from this table:\n{{ model_table }}\n\nTask:\n{{ instance_text }}' > <prompt file>` and `printf -- '- deepseek-flash: cheap, for simple tasks\n- deepseek-pro: stronger, for hard tasks\n- deepseek-chat: general chat' > <table file>`. Run `control-openhands browser click 'testid=meta-profile-classifier-input'`, `control-openhands browser click 'role=option[name="deepseek-flash"]'`, `control-openhands browser fill 'testid=meta-profile-name-input' QA_router`, `control-openhands browser fill 'testid=meta-profile-prompt-template' --value-file <prompt file>`, `control-openhands browser fill 'testid=meta-profile-model-table' --value-file <table file>`, `control-openhands browser click 'testid=meta-profile-save'` and `control-openhands browser wait-text 'Meta-profile "QA_router" saved'`. After `control-openhands browser reload`: `control-openhands browser text 'testid=meta-profile-row-QA_router'` is `QA_router\ndeepseek-flash · Direct prompt`, `control-openhands browser count 'testid=meta-profile-active-badge'` is `0`, the switch `eval` above returns `{checked:true, disabled:false}`, `control-openhands api GET /api/meta-profiles` has `active_meta_profile` `null`, `control-openhands api GET /api/settings` has `"run_router_at_conversation_start": true`, and `control-openhands api GET /api/profiles` has a new `deepseek-chat` (`deepseek/deepseek-chat`, `provider_connection_id` set) while `deepseek-flash`/`deepseek-pro` are unchanged. `control-openhands browser screenshot --feature F12.create-first --name list-after-create` shows the row without a badge and the switch on.- **Duplicate name (`F12.duplicate-name`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-custom'`, `control-openhands browser fill 'testid=meta-profile-name-input' QA_router`, `control-openhands browser text 'testid=meta-profile-name-taken'` (`A meta-profile with this name already exists. Choose a different name, or edit the existing one.`), pick classifier `deepseek-pro` as above, fill the prompt with `'Route: {{ instance_text }}'`, and `control-openhands browser enabled 'testid=meta-profile-save'` (`false`). `control-openhands browser fill 'testid=meta-profile-name-input' QA_router_2` makes `control-openhands browser count 'testid=meta-profile-name-taken'` `0` and Save `true`. Click `testid=meta-profile-cancel`; `control-openhands browser count 'testid=meta-profile-row-QA_router_2'` is `0`.- **"Don't create profiles" in a template (`F12.router-profiles`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-router-pro'`, `control-openhands browser value 'testid=meta-profile-router-connection'` (`QA_deepseek (deepseek)`, preselected), `control-openhands browser click 'testid=meta-profile-router-connection'` and `control-openhands browser click "role=option[name=\"Don't create profiles\"]" --observe 'testid=meta-profile-router-connection' --observe-ms 1500`. The picker stays on `Don't create profiles`: the observed states are `QA_deepseek (deepseek)` at 0 ms and `Don't create profiles` about 300 ms later, with nothing after it, and `control-openhands browser value 'testid=meta-profile-router-connection'` is `Don't create profiles`. Cancel the editor (`control-openhands browser click 'testid=meta-profile-cancel'`); `F12.create-more` saves a template with this choice. The Custom template starts on `Don't create profiles` even with a connection present; there, pick `role=option[name="QA_deepseek (deepseek)"]` first and then `Don't create profiles`: the second pick sticks. Cancel the editor.-- **Create another router without new profiles (`F12.create-more`, `F12.router-profiles`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-router-flash'`, `control-openhands browser value 'testid=meta-profile-router-connection'` (`QA_deepseek (deepseek)`, preselected), `control-openhands browser fill 'testid=meta-profile-name-input' QA_router_flash`, `control-openhands browser click 'testid=meta-profile-router-connection'`, `control-openhands browser click "role=option[name=\"Don't create profiles\"]"` and `control-openhands browser value 'testid=meta-profile-router-connection'` (`Don't create profiles`). Then `control-openhands browser click 'testid=meta-profile-save'` and `control-openhands browser wait-text 'Meta-profile "QA_router_flash" saved'`. After `control-openhands browser reload`, `control-openhands browser text 'testid=meta-profile-list'` is `QA_router\ndeepseek-flash · Direct prompt\nActive\nQA_router_flash\nminimax-m3 · Direct prompt` and `control-openhands browser count 'testid=meta-profile-active-badge'` is `1`. `control-openhands api GET /api/profiles` still lists only `deepseek-chat`, `deepseek-flash` and `deepseek-pro`: none of the template's 11 models became a profile, so `QA_router_flash` names a classifier (`minimax-m3`) that does not exist, which saving allows (see Gotchas).-- **Row menu and activate (`F12.row-menu`, `F12.activate`).** Run `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router'` and `control-openhands browser snapshot 'testid=meta-profile-actions-menu'` (`menuitem "Edit"`, `menuitem "Set active" [disabled]`, `menuitem "Delete"`); `control-openhands browser press Escape` closes it (`count` `0`). Then `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router_flash'`, `control-openhands browser click 'testid=meta-profile-actions-menu >> testid=meta-profile-set-active'` and `control-openhands browser wait-text 'Meta-profile "QA_router_flash" activated'`. After `control-openhands browser reload`, `control-openhands browser count 'testid=meta-profile-row-QA_router_flash >> testid=meta-profile-active-badge'` is `1`, the total badge count is `1`, and `api GET /api/meta-profiles` agrees.-- **Edit (`F12.edit`).** Run `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router'`, `control-openhands browser click 'testid=meta-profile-actions-menu >> testid=meta-profile-edit'`, `control-openhands browser text 'testid=meta-profile-editor >> role=heading'` (`Edit meta-profile`), `control-openhands browser enabled 'testid=meta-profile-name-input'` (`false`), `control-openhands browser count 'testid=meta-profile-router-connection'` (`0`). Pick classifier `deepseek-pro`, fill `testid=meta-profile-model-table` with the first two table lines only, click `testid=meta-profile-save` and `control-openhands browser wait-text 'Meta-profile "QA_router" saved'`. After a reload the row reads `QA_router\ndeepseek-pro · Direct prompt`, the badge stays on `QA_router_flash`, and `control-openhands api GET /api/meta-profiles/QA_router` shows the new classifier and table.+- **Create another router without new profiles (`F12.create-more`, `F12.router-profiles`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-router-flash'`, `control-openhands browser value 'testid=meta-profile-router-connection'` (`QA_deepseek (deepseek)`, preselected), `control-openhands browser fill 'testid=meta-profile-name-input' QA_router_flash`, `control-openhands browser click 'testid=meta-profile-router-connection'`, `control-openhands browser click "role=option[name=\"Don't create profiles\"]"` and `control-openhands browser value 'testid=meta-profile-router-connection'` (`Don't create profiles`). Then `control-openhands browser click 'testid=meta-profile-save'` and `control-openhands browser wait-text 'Meta-profile "QA_router_flash" saved'`. After `control-openhands browser reload`, `control-openhands browser text 'testid=meta-profile-list'` is `QA_router\ndeepseek-flash · Direct prompt\nQA_router_flash\nminimax-m3 · Direct prompt` and `control-openhands browser count 'testid=meta-profile-active-badge'` is `0`. `control-openhands api GET /api/profiles` still lists only `deepseek-chat`, `deepseek-flash` and `deepseek-pro`: none of the template's 11 models became a profile, so `QA_router_flash` names a classifier (`minimax-m3`) that does not exist, which saving allows (see Gotchas).+- **Row menu (`F12.row-menu`).** Run `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router'` and `control-openhands browser snapshot 'testid=meta-profile-actions-menu'` (`menuitem "Edit"`, `menuitem "Delete"`; `browser count 'testid=meta-profile-set-active'` is `0`); screenshot `--feature F12.row-menu --name menu`; `control-openhands browser press Escape` closes it (`count` `0`).+- **Activate (`F12.activate`).** Blocked locally: the row menu has no Set active. On an organization-bound Cloud backend, row menu → `testid=meta-profile-set-active` and `wait-text 'Meta-profile "<name>" activated'` move the Active badge; record `blocked` naming the Cloud organization login.+- **Edit (`F12.edit`).** Run `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router'`, `control-openhands browser click 'testid=meta-profile-actions-menu >> testid=meta-profile-edit'`, `control-openhands browser text 'testid=meta-profile-editor >> role=heading'` (`Edit meta-profile`), `control-openhands browser enabled 'testid=meta-profile-name-input'` (`false`), `control-openhands browser count 'testid=meta-profile-router-connection'` (`0`). Pick classifier `deepseek-pro`, fill `testid=meta-profile-model-table` with the first two table lines only, click `testid=meta-profile-save` and `control-openhands browser wait-text 'Meta-profile "QA_router" saved'`. After a reload the row reads `QA_router\ndeepseek-pro · Direct prompt` and `control-openhands api GET /api/meta-profiles/QA_router` shows the new classifier and table.- **Cancel the connection dialog (`F12.add-connection-cancel`).** Run `control-openhands browser click 'testid=add-meta-profile'`, `control-openhands browser click 'testid=meta-profile-template-custom'`, `control-openhands browser fill 'testid=meta-profile-name-input' QA_router_cancel`, `control-openhands browser click 'testid=meta-profile-add-provider-connection'`, `control-openhands browser fill 'testid=provider-connection-name-input' QA_unused`, `control-openhands browser click 'role=dialog >> role=button[name="Cancel"]'` (the Cancel button has no test id) and `control-openhands browser wait 'testid=provider-connection-modal' --state detached`. `browser value 'testid=meta-profile-name-input'` is still `QA_router_cancel`, the picker still reads `Don't create profiles`, and `control-openhands api GET /api/llm/provider-connections` has no `QA_unused` (`display_name` is the field).- **Leave the editor (`F12.editor-cancel`).** In that editor `control-openhands browser count` on `testid=meta-profile-list`, `testid=add-meta-profile` and `testid=meta-profile-run-at-conversation-start` is `0` each. Pick classifier `deepseek-flash`, fill the prompt `'Pick: {{ instance_text }}'`, click `testid=meta-profile-cancel`: `count 'testid=meta-profile-list'` is `1`, `count 'testid=meta-profile-row-QA_router_cancel'` is `0` and `api GET /api/meta-profiles` has no `QA_router_cancel`. In edit mode (row menu → Edit on `QA_router`), `control-openhands browser fill 'testid=meta-profile-prompt-template' 'Changed: {{ instance_text }}'` then Cancel leaves `control-openhands api GET /api/meta-profiles/QA_router` with its saved `prompt_template`.- **Run on first message (`F12.run-first-message-toggle`).** Run `control-openhands browser click 'testid=meta-profile-run-at-conversation-start >> text=Run on first message'`, then `control-openhands browser reload` and the switch `eval`: `checked` flips and stays flipped; `api GET /api/settings` shows `run_router_at_conversation_start` with the same value; `control-openhands browser toasts` is empty. Click again to restore `true`.-- **Routing in a conversation (`F12.route-first-message`).** Make `QA_router` active (row menu → Set active, `wait-text 'Meta-profile "QA_router" activated'`) with the switch on. Run `control-openhands conversation start --prompt "Run echo routed in the terminal and tell me its output." --wait --timeout 300`, then `control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent` (`<id>` from the start output). The first action is `route_task_to_model`, its observation starts `Classified task as 'model: deepseek-flash' and switched to LLM profile 'deepseek-flash'`, then the terminal runs `echo routed`. On the conversation page `control-openhands browser wait-text 'Switched to profile'` succeeds and `control-openhands browser screenshot --feature F12.route-first-message --name conversation` shows the `Switched to profile deepseek-flash` line above the answer. `control-openhands api GET "/api/conversations/<id>/events/search?limit=1" --max-bytes 200000` (the system prompt) contains `ROUTE_AT_CONVERSATION_START` and `"route_task_to_model"`. Switch **Run on first message** off (`control-openhands browser goto /settings/meta-llm` first: the browser is on the conversation), start `--prompt "Reply with only the word pong."`, and the same GET contains `"route_task_to_model"` but no `ROUTE_AT_CONVERSATION_START`; there is no route action.+- **Routing in a conversation (`F12.route-first-message`).** Set the active agent profile's router to `QA_router` (Settings → Agent → Edit → `agent-settings-router-select`, `F13.profile-router`; the recipe there uses a named profile `QA_oh` pinned to `deepseek-pro`, activated with Set as active) with the switch on. Run `control-openhands conversation start --prompt "Run echo routed in the terminal and tell me its output." --wait --timeout 300`, then `control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent` (`<id>` from the start output). The first action is `route_task_to_model`, its observation starts `Classified task as 'model: deepseek-flash' and switched to LLM profile 'deepseek-flash'`, then the terminal runs `echo routed`. On the conversation page `control-openhands browser wait-text 'Switched to profile'` succeeds and `control-openhands browser screenshot --feature F12.route-first-message --name conversation` shows the `Switched to profile deepseek-flash` line above the answer. `control-openhands api GET "/api/conversations/<id>/events/search?limit=1" --max-bytes 200000` (the system prompt) contains `ROUTE_AT_CONVERSATION_START` and `"route_task_to_model"`. Switch **Run on first message** off (`control-openhands browser goto /settings/meta-llm` first: the browser is on the conversation), start `--prompt "Reply with only the word pong."`, and the same GET contains `"route_task_to_model"` but no `ROUTE_AT_CONVERSATION_START`; there is no route action.- **Delete (`F12.delete`).** Run `control-openhands browser goto /settings/meta-llm`, `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router_flash'`, `control-openhands browser click 'testid=meta-profile-actions-menu >> testid=meta-profile-delete'` and `control-openhands browser snapshot 'role=dialog'` (`Delete meta-profile`, `Are you sure you want to delete "QA_router_flash"? This cannot be undone.`, Cancel, Delete). `control-openhands browser click 'role=dialog >> role=button[name="Cancel"]'` keeps the row (`count` `1`). Repeat the menu click, then `control-openhands browser click 'testid=delete-meta-profile-confirm'`, `control-openhands browser wait-text 'Meta-profile "QA_router_flash" deleted'`, `control-openhands browser reload`; `control-openhands browser count 'testid=meta-profile-row-QA_router_flash'` is `0`.-- **Delete the active router (`F12.delete-active`).** With `QA_router` active and the switch on, delete it the same way (`wait-text 'Meta-profile "QA_router" deleted'`). After a reload `testid=meta-profile-empty` is back, the switch `eval` is `{checked:false, disabled:true}` and `api GET /api/meta-profiles` has `active_meta_profile` `null`, while `api GET /api/settings` still has `"run_router_at_conversation_start": true`. A new `conversation start --prompt "Reply with only the word pong." --wait` has neither `"route_task_to_model"` nor `ROUTE_AT_CONVERSATION_START` in its events/search output.-- **Light theme (`F12.light-theme`).** No router is left after `F12.delete-active`, whose last `conversation start` left the browser on a conversation: run `control-openhands browser goto /settings/meta-llm` and create one through Custom as in `F12.phone` (name `QA_router_light`, classifier `deepseek-flash`, prompt `'Pick one model for: {{ instance_text }}'`, `wait-text 'Meta-profile "QA_router_light" saved'`), then `control-openhands browser goto /settings/app`, `control-openhands browser click 'testid=color-theme-input'`, `control-openhands browser click 'role=option[name="Light+"]'`, `control-openhands browser goto /settings/meta-llm` and `control-openhands browser eval "(()=>{const h=[...document.querySelectorAll('h2')].find(e=>e.textContent.includes('Available meta-profiles'));const n=document.querySelector('[data-testid^=meta-profile-row-] span');return {heading:getComputedStyle(h).color,rowName:getComputedStyle(n).color,bg:getComputedStyle(document.body).backgroundColor}})()"`. It returns `{heading: "rgb(31, 31, 31)", rowName: "rgb(31, 31, 31)", bg: "rgb(255, 255, 255)"}`: dark text, the same ink as LLM profile names on `/settings/llm`. `control-openhands browser screenshot --feature F12.light-theme --name list` shows the heading `Available meta-profiles` and the name `QA_router_light` next to its summary and Active badge. Restore with `role=option[name="OpenHands-Neutral"]` the same way, then delete `QA_router_light` through its row menu (`wait-text 'Meta-profile "QA_router_light" deleted'`) so `F12.phone` starts on a router-less page.-- **Phone (`F12.phone`, also re-proves `F12.create-first` on a router-less page).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/meta-llm`, `control-openhands browser click 'testid=add-meta-profile'` and `control-openhands browser bbox 'testid=meta-profile-template-modal'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`). Choose Custom, fill `QA_router_phone`, classifier `deepseek-flash` and prompt `'Pick one model for: {{ instance_text }}'`; `control-openhands browser bbox 'testid=meta-profile-editor'` has no page overflow (taller than the screen, scrolls). Save, reload, then `control-openhands browser bbox 'testid=meta-profile-row-QA_router_phone'` and, after `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router_phone'`, `control-openhands browser bbox 'testid=meta-profile-actions-menu'` are inside the viewport without page overflow. `control-openhands browser screenshot --feature F12.phone --name list` shows `QA_route…` truncated, the summary, the badge and the switch on. Return with `control-openhands browser viewport desktop`.-- **Restart persistence (`F12.list`).** Run `control-openhands restart`, `control-openhands doctor`, `control-openhands browser goto /settings/meta-llm` and `control-openhands browser count 'testid=meta-profile-row-QA_router_phone >> testid=meta-profile-active-badge'`; it is `1`.+- **Delete the last router (`F12.delete-active`).** With `QA_router` the active agent profile's router, the switch on and no other router left (delete `QA_router_2` first if F13 created it), delete it the same way (`wait-text 'Meta-profile "QA_router" deleted'`). After a reload `testid=meta-profile-empty` is back and the switch `eval` is `{checked:false, disabled:true}`, while `api GET /api/settings` still has `"run_router_at_conversation_start": true`; `api GET /api/agent-profiles/<profile>` has `meta_profile_ref` `null` and `enable_classify_and_switch_llm_tool` `false` (screenshot `--feature F12.delete-active --name last-deleted`). A new `conversation start --prompt "Reply with only the word pong." --wait` has neither `"route_task_to_model"` nor `ROUTE_AT_CONVERSATION_START` in its events/search output.+- **Light theme (`F12.light-theme`).** No router is left after `F12.delete-active`, whose last `conversation start` left the browser on a conversation: run `control-openhands browser goto /settings/meta-llm` and create one through Custom as in `F12.phone` (name `QA_router_light`, classifier `deepseek-flash`, prompt `'Pick one model for: {{ instance_text }}'`, `wait-text 'Meta-profile "QA_router_light" saved'`), then `control-openhands browser goto /settings/app`, `control-openhands browser click 'testid=color-theme-input'`, `control-openhands browser click 'role=option[name="Light+"]'`, `control-openhands browser goto /settings/meta-llm` and `control-openhands browser eval "(()=>{const h=[...document.querySelectorAll('h2')].find(e=>e.textContent.includes('Available meta-profiles'));const n=document.querySelector('[data-testid^=meta-profile-row-] span');return {heading:getComputedStyle(h).color,rowName:getComputedStyle(n).color,bg:getComputedStyle(document.body).backgroundColor}})()"`. It returns `{heading: "rgb(31, 31, 31)", rowName: "rgb(31, 31, 31)", bg: "rgb(255, 255, 255)"}`: dark text, the same ink as LLM profile names on `/settings/llm`. `control-openhands browser screenshot --feature F12.light-theme --name list` shows the heading `Available meta-profiles` and the name `QA_router_light` next to its summary. Restore with `role=option[name="OpenHands-Neutral"]` the same way, then delete `QA_router_light` through its row menu (`wait-text 'Meta-profile "QA_router_light" deleted'`) so `F12.phone` starts on a router-less page.+- **Phone (`F12.phone`, also re-proves `F12.create-first` on a router-less page).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/meta-llm`, `control-openhands browser click 'testid=add-meta-profile'` and `control-openhands browser bbox 'testid=meta-profile-template-modal'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`). Choose Custom, fill `QA_router_phone`, classifier `deepseek-flash` and prompt `'Pick one model for: {{ instance_text }}'`; `control-openhands browser bbox 'testid=meta-profile-editor'` has no page overflow (taller than the screen, scrolls). Save, reload, then `control-openhands browser bbox 'testid=meta-profile-row-QA_router_phone'` and, after `control-openhands browser click 'testid=meta-profile-menu-trigger-QA_router_phone'`, `control-openhands browser bbox 'testid=meta-profile-actions-menu'` are inside the viewport without page overflow. `control-openhands browser screenshot --feature F12.phone --name list` shows `QA_route…` truncated, the summary and the switch on. Return with `control-openhands browser viewport desktop`.+- **Restart persistence (`F12.list`).** Run `control-openhands restart`, `control-openhands doctor`, `control-openhands browser goto /settings/meta-llm` and `control-openhands browser count 'testid=meta-profile-row-QA_router_phone'`; it is `1`.- **Cleanup.** Switch **Run on first message** off through its label first (it disables once no router is left), then delete each `QA_*` router through the row menu. Remove the arranged LLM profile and connection with `control-openhands api DELETE /api/profiles/deepseek-chat --write` and `control-openhands api DELETE /api/llm/provider-connections/<connection id> --write` (`<connection id>` from `control-openhands api GET /api/llm/provider-connections`), or through Settings → LLM (F10).-- **Router activation is honest on ACP (`F12.activate`).** ACP cannot attach the router; activation must report that refusal rather than plain success. Known failure (reproduced twice 2026-10-16, [OpenHands/software-agent-sdk#5879](https://github.com/OpenHands/software-agent-sdk/issues/5879)): actual global ACP settings produced an activated toast while both stored pointers stayed null. An active ACP profile pointer alone does not create this condition. The block restores the original owned settings bytes; its assertion rejects the misleading success toast.-- ```sh known-bug F12.activate acp-settings- QA_ACP_SETTINGS="$OH_VERIFY_RUN/private/qa-known-acp-settings.json"- cp "$OH_VERIFY_RUN/private/settings.json" "$QA_ACP_SETTINGS"- trap 'cp "$QA_ACP_SETTINGS" "$OH_VERIFY_RUN/private/settings.json"; rm -f "$QA_ACP_SETTINGS"; control-openhands api DELETE /api/meta-profiles/QA_KNOWN_ACP_ROUTER --write >/dev/null; control-openhands browser reload >/dev/null' EXIT- control-openhands api POST /api/meta-profiles/QA_KNOWN_ACP_ROUTER --data '{"classifier_model":"QA_KNOWN_CLASSIFIER","prompt_template":"Pick {{ instance_text }}"}' --write- control-openhands api PATCH /api/settings --data '{"agent_settings_diff":{"agent_kind":"acp"}}' --write --pick agent_settings.agent_kind- control-openhands assert --field value --equals acp -- api GET /api/settings --pick agent_settings.agent_kind- control-openhands browser viewport desktop- control-openhands browser goto /settings/meta-llm- control-openhands browser wait 'testid=meta-profile-row-QA_KNOWN_ACP_ROUTER'- control-openhands browser click 'testid=meta-profile-menu-trigger-QA_KNOWN_ACP_ROUTER'- control-openhands browser click 'testid=meta-profile-set-active' --observe 'role=status' --observe-ms 1000- control-openhands assert --field value --equals false --bug-shows true -- browser eval 'document.body.innerText.includes("Meta-profile \"QA_KNOWN_ACP_ROUTER\" activated")' # bug- ```--- **Unreadable settings refuse router activation (`F12.activate`).** Expected: a handled 409 without a filesystem path and no store overwrite. Known failure (reproduced twice 2026-10-16, [OpenHands/software-agent-sdk#5922](https://github.com/OpenHands/software-agent-sdk/issues/5922)): the real Set active path showed the generic 500 and the settings path. The corrupt bytes survived. Back up only this run's settings, and restore them in `EXIT` before the runner's health check.-- ```sh known-bug F12.activate corrupt-settings- QA_ROUTER_SETTINGS="$OH_VERIFY_RUN/private/qa-known-router-settings.json"- cp "$OH_VERIFY_RUN/private/settings.json" "$QA_ROUTER_SETTINGS"- trap 'cp "$QA_ROUTER_SETTINGS" "$OH_VERIFY_RUN/private/settings.json"; rm -f "$QA_ROUTER_SETTINGS"; control-openhands api DELETE /api/meta-profiles/QA_KNOWN_CORRUPT_ROUTER --write >/dev/null; control-openhands browser reload >/dev/null' EXIT- control-openhands api POST /api/meta-profiles/QA_KNOWN_CORRUPT_ROUTER --data '{"classifier_model":"QA_KNOWN_CLASSIFIER","prompt_template":"Pick {{ instance_text }}"}' --write- control-openhands browser viewport desktop- control-openhands browser goto /settings/meta-llm- control-openhands browser wait 'testid=meta-profile-row-QA_KNOWN_CORRUPT_ROUTER'- printf '{"agent_settings": ' > "$OH_VERIFY_RUN/private/settings.json"- control-openhands browser click 'testid=meta-profile-menu-trigger-QA_KNOWN_CORRUPT_ROUTER'- control-openhands browser network --clear- control-openhands browser click 'testid=meta-profile-set-active' --observe 'role=status' --observe-ms 1000- control-openhands assert --field recent.0.status --equals 409 --bug 'recent.0.status eq 500 && recent.0.path eq "/api/meta-profiles/QA_KNOWN_CORRUPT_ROUTER/activate"' -- browser network --filter '/api/meta-profiles/QA_KNOWN_CORRUPT_ROUTER/activate' # bug- ```-## Gotchas- Settings nav test ids embed the route: quote `'testid=sidebar-settings-/settings/meta-llm'` as written.- The empty-state hint about LLM profiles only exists before any profile is saved: drive `F12.empty-states` first on a fresh run.-- The first router you create is activated and turns **Run on first message** on by itself; later routers do neither. Any recipe that creates a router on an empty list changes both.+- The first router you create turns **Run on first message** on by itself; later routers do not. Locally no router is ever active here: whether a conversation routes depends on its agent profile's router (`F13.profile-router`).- `Run on first message` is an `sr-only` checkbox (keyboard reachable): click the label text inside `testid=meta-profile-run-at-conversation-start` and read `.checked`/`.disabled` with `browser eval`. The stored preference (`misc_settings.app_preferences.run_router_at_conversation_start`) is independent of what the disabled switch shows; with no active router it may still be `true`. Turn it off before deleting the last router, or a later "first router" will not re-enable it because it is already on.- Router templates name 11 models (`gpt-5.4` … `claude-sonnet-4-5`) that will not exist in a sandbox. With a provider connection present, a template preselects it, and saving creates all 11 as LLM profiles linked to that connection unless you pick **Don't create profiles** first (`F12.create-more` does). If a run created them, remove them with `for p in gpt-5.4 gpt-5.5 minimax-m3 claude-opus-4-6 claude-opus-4-7 claude-opus-4-8 claude-fable-5 kimi-k2.5 kimi-k2.6 deepseek-v3.2-reasoner claude-sonnet-4-5; do control-openhands api DELETE /api/profiles/$p --write; done` (they live in the run's private state and only add rows to Settings → LLM).- Profile matching is case-insensitive and exact by name: table rows are `- <profile name>: description`; existing `deepseek-flash`/`deepseek-pro` are not re-created, and the endpoint is derived as `<provider lowercased>/<name>`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F13-agent-profiles.md b/.agents/skills/verify-openhands/references/feature-map/F13-agent-profiles.mdindex 75f74e4c57..f43db8b933 100644--- a/.agents/skills/verify-openhands/references/feature-map/F13-agent-profiles.md+++ b/.agents/skills/verify-openhands/references/feature-map/F13-agent-profiles.md@@ -1,26 +1,34 @@# F13 — Agent profilesSettings → Agent is a library of named agent profiles. Each profile is either an-OpenHands agent (an LLM profile, a system prompt, a tool set, parallel-tool calls, and MCP-server and secret scopes) or an ACP external agent (a preset-such as Claude Code, Codex, Gemini CLI, Pi or OpenCode, its launch command, a-model the agent itself reports, and provider credentials). One profile is-active; new conversations started from Home launch from it.--Source: `src/routes/agent-profiles-settings.tsx`, `src/routes/agent-settings.tsx`, `src/components/features/settings/agent-profiles/`, `src/components/features/settings/acp-credentials-section.tsx`, `src/components/features/settings/acp-conflict-warnings.tsx`, `src/components/features/settings/acp-auth-status-banner.tsx`, `src/constants/acp-providers.ts`, `src/constants/acp-brand-marks.ts`, `src/hooks/query/use-acp-model-discovery.ts`, `src/hooks/mutation/use-create-conversation.ts`, `src/hooks/mutation/resolve-conversation-launch.ts`, `src/hooks/mutation/use-new-conversation-command.ts`.+OpenHands agent (an LLM profile, a system prompt, a Model Router, a tool set,+parallel tool calls, MCP-server and secret scopes, condenser, verification and+skills) or an ACP external agent (a preset such as Claude Code, Codex, Gemini+CLI, Pi or OpenCode, its launch command, a model the agent itself reports, and+provider credentials). One profile is active; new conversations started from+Home launch from it by id, the seeded `default` included. On a local backend the+OpenHands `default` shares its LLM, MCP servers, secrets and skills with the+global pages (the active LLM profile, the servers switched on in MCP Servers,+every secret, the Skills page list), cannot be renamed or deleted, and is the+template a new profile copies its other settings from.++Source: `src/routes/agent-profiles-settings.tsx`, `src/routes/agent-settings.tsx`, `src/components/features/settings/agent-profiles/`, `src/components/features/settings/acp-credentials-section.tsx`, `src/components/features/settings/acp-conflict-warnings.tsx`, `src/components/features/settings/acp-auth-status-banner.tsx`, `src/constants/acp-providers.ts`, `src/constants/acp-brand-marks.ts`, `src/hooks/query/use-acp-model-discovery.ts`, `src/hooks/query/use-skill-catalog.ts`, `src/hooks/mutation/use-create-conversation.ts`, `src/hooks/mutation/resolve-conversation-launch.ts`, `src/hooks/mutation/use-new-conversation-command.ts`, `src/hooks/mutation/sync-agent-profiles.ts`, `src/hooks/use-chat-input-llm-profile-state.ts`, `src/api/agent-profiles-service/profile-field-support.ts`.## Sub-features-- `F13.list`: under **Available Profiles**, rows (sorted by name) show the profile name, then its LLM profile (OpenHands) or `ACP (external subprocess)`, a **Default** badge on the active profile and a `...` menu (Edit, Set as active, Delete).-- `F13.create`: **Add agent profile** opens the editor; Save toasts `Profile "<name>" created` and the row is listed after a reload.-- `F13.name-validation`: an empty or malformed name marks the field invalid (red rule) and disables Save; a name another profile already has also marks the field invalid, shows a red taken-name message about agent profiles under it (the rule stays grey) and disables Save (Edit accepts the profile's own name).+- `F13.list`: under **Available Profiles**, rows (sorted by name) show the profile name, then its LLM profile (OpenHands) or `ACP (external subprocess)`, an **Active** badge on the active profile and a `...` menu (Edit, Set as active, Delete).+- `F13.create`: **Add agent profile** opens the editor; Save toasts `Profile "<name>" created` and the row is listed after a reload. A new OpenHands profile starts from `default`'s system prompt, Model Router, parallel tool calls, tools, condenser, verification and skills; its MCP servers and secrets start unrestricted and its LLM picker on the active LLM profile.+- `F13.name-validation`: a fresh Add editor's empty name is not flagged (Save disabled); once typed, an empty or malformed name marks the field invalid (red rule) and disables Save; a name another profile already has also marks the field invalid, shows a red taken-name message about agent profiles under it (the rule stays grey) and disables Save (Edit accepts the profile's own name).- `F13.editor-no-llm`: with no LLM profiles, an OpenHands profile shows `Create an LLM profile first to define an OpenHands agent.` and Save toasts `Select an LLM profile for this agent.` without creating anything.-- `F13.openhands-options`: OpenHands profiles offer an LLM profile dropdown (the active LLM profile by default), a **System prompt** choice (`OpenHands default`, `Default + your instructions` or `Custom persona`, each saved on the profile), **Parallel tool calls** (default 1, minimum 1) and **Tools**: `Standard tools` (default) lists the server's standard set read-only, `Choose tools` turns the server's tool catalog into switches; values persist and reopen.-- `F13.edit`: Edit reopens the stored values with Save disabled until something changes; renaming keeps the profile id; Back and Cancel leave without saving.-- `F13.stale-llm-ref`: editing a profile whose LLM profile no longer exists preselects the active LLM profile and enables Save, so saving heals the reference; launching it unhealed on a backend advertising per-launch LLM override keeps the profile and runs the active LLM profile (`launched_agent_profile.llm_profile_ref`).-- `F13.set-active`: **Set as active** toasts `Switched to profile "<name>"`, moves the Default badge and is disabled on the active row.-- `F13.active-drives-conversation`: a conversation started from Home, or with `/new` typed in a conversation's composer, after activation launches from that profile (its id and settings are stamped on the conversation); with an ACP profile active, `/new` starts an ACP agent. The seeded OpenHands `default` launches from the global agent settings (no `launched_agent_profile`).-- `F13.llm-pill-precedence`: when the active profile pins a different LLM profile than the Home LLM pill, a backend advertising per-launch LLM override uses the pill's LLM while preserving the named profile's other settings and identity.+- `F13.openhands-options`: OpenHands profiles offer an LLM profile dropdown right under the agent type (the active LLM profile by default), a **System prompt** choice (`OpenHands default`, `Default + your instructions` or `Custom persona`, each saved on the profile), **Parallel tool calls** (default 1, minimum 1) and **Tools**: `Standard tools` (default) lists the server's standard set read-only, `Choose tools` turns the server's tool catalog into switches; values persist and reopen.+- `F13.edit`: Edit reopens the stored values with Save disabled until something changes; renaming keeps the profile id; Back and Cancel leave without saving. On a local backend the `default` name field is disabled with `The default profile can't be renamed or deleted.` under it.+- `F13.stale-llm-ref`: editing a profile whose LLM profile no longer exists preselects the active LLM profile and enables Save, so saving heals the reference; launching it unhealed keeps the profile and runs the active LLM profile (`launched_agent_profile.llm_profile_ref`).+- `F13.set-active`: **Set as active** toasts `Switched to profile "<name>"`, moves the Active badge and is disabled on the active row.+- `F13.active-drives-conversation`: a conversation started from Home, or with `/new` typed in a conversation's composer, after activation launches from that profile (its id and settings are stamped on the conversation: `launched_agent_profile.agent_profile_id`, tools, parallel tool calls, condenser, router, skills); with an ACP profile active, `/new` starts an ACP agent. The seeded OpenHands `default` launches by id the same way, on the active LLM profile.+- `F13.llm-pill-precedence`: the Home LLM pill shows the active agent profile's LLM. On a named profile it shows that profile's own LLM profile, a pick there saves to the profile (the active LLM profile is unchanged) and the launch runs it; on `default` it shows the active LLM profile and a pick activates another one, which `default` then runs.+- `F13.default-shared`: on a local backend the OpenHands `default` editor shows every section, with notes where it shares a global page: LLM (`The default profile uses the active LLM profile. Picking another one here makes it active.`), MCP servers (`The default profile uses the servers switched on in MCP Servers. …`), Secrets (`The default profile always has every secret.`, no picker) and Skills (`The default profile's skills are the Skills page list. …`).+- `F13.profile-router`: an OpenHands profile has a **Model Router** dropdown: `Off` (default for a new profile when `default` has none) or one of the saved routers; saving stores `meta_profile_ref` and `enable_classify_and_switch_llm_tool`, and a launch gets `route_task_to_model` (plus the first-message instruction while F12's **Run on first message** is on). A router that no longer exists stays selected with a red warning; a profile with routing on but no router named opens on `Alphabetically first router`. Deleting a router on the Model Router page detaches it from every profile. Local only.+- `F13.profile-skills`: an OpenHands profile has a **Skills** section listing the server's skill catalog (each labelled `Catalog` or `Your skill`) with a search box; switches save `public_skills`/`disabled_skills`, and a launch loads only the switched-on skills (repository skills always load). Local only.- `F13.mcp-scope`: **MCP servers** offers **All servers** (every configured server listed, switched on and disabled) or **Choose servers** (a switch per configured server); the choice persists; a chosen server that was removed stays listed with a red warning, and launching with it is refused.- `F13.secrets-scope`: **Secrets** offers **No profile restriction** (every switch on and disabled) or **Choose secrets**; on an ACP profile, Choose preselects that provider's secret names; the selection persists.- `F13.acp-form`: choosing **ACP (external subprocess)** shows Preset, Command, Credentials and Model; each preset prefills its command and its model choice (`Agent default`, or Gemini CLI's preferred `gemini-2.5-pro`), **Custom** empties the command and disables Save until one is typed, typing a preset's exact command selects that preset, and the Model list's **Custom** opens a free-text model field.@@ -28,8 +36,8 @@ Source: `src/routes/agent-profiles-settings.tsx`, `src/routes/agent-settings.tsx- `F13.acp-model-discovery`: for a built-in preset with its default command the Model list comes from the agent: `Asking <agent> which models it offers…` while the agent-server asks it, then its models with `Agent default (<model>)` first; an agent that wants a login says `Sign in or add credentials above to see the models <agent> offers.`, one that cannot start says `<agent>'s models appear here after your first conversation with it. …`; a picked model is saved as `acp_model` and reopens selected.- `F13.acp-credentials`: built-in presets show a Credentials section (password fields), an auth banner (signed in / configured), a conflict warning for mutually exclusive Claude credentials, and `Already saved — leave blank to keep` for stored values. Credentials are saved as global secrets.- `F13.acp-preset-credentials`: the Credentials fields follow the preset: Claude Code `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`; Codex `OPENAI_API_KEY`, `CODEX_AUTH_JSON` (multi-line), `OPENAI_BASE_URL`; Gemini CLI `GOOGLE_APPLICATION_CREDENTIALS_JSON` (multi-line), `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION`, `GOOGLE_GENAI_USE_VERTEXAI`, `GEMINI_API_KEY`, `GEMINI_BASE_URL`; Pi `PI_AUTH_JSON` (multi-line), `ANTHROPIC_API_KEY`; OpenCode `OPENCODE_AUTH_CONTENT` (multi-line), `OPENCODE_API_KEY`; **Custom** has no Credentials section. With no host login and no stored credential (Gemini here) no auth banner shows.-- `F13.menu-keyboard`: opening the row's `...` menu focuses Edit; ArrowDown/ArrowUp move focus between the enabled items (wrapping, and skipping the disabled **Set as active** on the active row), Enter runs the focused item, and Tab, Escape or a click outside close the menu.-- `F13.delete`: Delete asks for confirmation; Cancel keeps the row, Delete removes it. Deleting the active profile selects `default` immediately; deleting the last profile makes the server re-seed and activate `default`.+- `F13.menu-keyboard`: opening the row's `...` menu focuses Edit; ArrowDown/ArrowUp move focus between the enabled items (wrapping, and skipping the disabled **Set as active** on the active row and the disabled **Delete** on the local `default`), Enter runs the focused item, and Tab, Escape or a click outside close the menu.+- `F13.delete`: Delete asks for confirmation; Cancel keeps the row, Delete removes it. Deleting the active profile selects `default` immediately. On a local backend `default`'s **Delete** is disabled (its title reads `The default profile can't be renamed or deleted.`).- `F13.phone`: the list, row menu and editor fit a 390 px viewport without horizontal overflow.- `F13.acp-conversation`: a conversation launched from an active ACP profile runs the external agent. Blocked without an approved provider account.- `F13.cloud-read-only`: on a Cloud backend, organization members see rows without the `...` menu or **Add agent profile**. Blocked without a Cloud account.@@ -48,14 +56,15 @@ Source: `src/routes/agent-profiles-settings.tsx`, `src/routes/agent-settings.tsxPreconditions:- Baseline state (launched, doctored, `onboard --skip` done) on a **fresh** run. `F13.editor-no-llm` must run before `control-openhands llm preset deepseek`; everything from `F13.stale-llm-ref` on needs the preset (`deepseek-flash` active, `deepseek-pro`).-- A fresh run has one profile, `default`, active, whose LLM profile is `default` (which does not exist). `control-openhands api GET /api/agent-profiles` shows it.+- A fresh run has one profile, `default`, active, whose LLM profile is `default` (which does not exist). `control-openhands api GET /api/agent-profiles` shows it. On first load the UI migrates the global settings into it once (`misc_settings.app_preferences.default_profile_migrated` `true`; `api GET /api/agent-profiles/default` then carries `public_skills`, `condenser` and `verification`).+- `F13.profile-router` needs two routers: create `QA_router` (classifier `deepseek-flash`, the `F12.create-first` prompt and a two-line table naming `deepseek-flash` and `deepseek-pro`) and `QA_router_2` through Settings → Model Router (F12).- `F13.mcp-scope` needs two configured MCP servers. Arrange them with `control-openhands api POST /api/settings/mcp/qa_mcp_a --data '{"transport":"stdio","command":"qa-no-such-command","enabled":false}' --write` and the same for `qa_mcp_b`.- `F13.acp-conversation` needs the provider's agent CLI and an approved account; `F13.cloud-read-only` needs a Cloud organization member login. Both are blocked here.- Dropdowns are autocompletes. Pick an option with `control-openhands browser choose 'testid=<testid>' '<label>'` (proven on `agent-type-selector`, `agent-preset-selector`, `agent-model-selector` (`Custom`), `agent-profile-llm-selector`, `agent-settings-mcp-mode` and `agent-settings-secrets-mode`). The manual opener, `control-openhands browser click 'div:has(> div > input[data-testid=<testid>]) >> role=button[name="Show suggestions"]'` then `browser click 'role=option[name="<label>"]'`, works for most of them but is flaky right after an editor opens (see Gotchas); clicking the input itself can open an empty, filtered list.-- **List (`F13.list`).** From `/` run `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings'`, then `control-openhands browser url` (`/settings/agents`) and `control-openhands browser text 'testid=agent-profile-row'`: `default`, its LLM profile `default`, `Default` on three lines. `control-openhands browser count 'testid=agent-profile-active-badge'` is `1`; `control-openhands browser screenshot --feature F13.list --name list`. Command menu: run `control-openhands browser goto /` (the browser is on `/settings/agents`, where the URL wait would pass at once), `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'Agent settings'`, `control-openhands browser press Enter`, `control-openhands browser wait-url '/settings/agents(\?|$)'`. Legacy URL: `control-openhands browser goto /settings/agent` returns `.../settings/agents`.+- **List (`F13.list`).** From `/` run `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings'`, then `control-openhands browser url` (`/settings/agents`) and `control-openhands browser text 'testid=agent-profile-row'`: `default`, its LLM profile `default`, `Active` on three lines. Open its menu (`browser click 'testid=agent-profile-menu-trigger'`): `control-openhands browser snapshot 'testid=agent-profile-actions-menu'` is `Edit`, `Set as active [disabled]`, `Delete [disabled]`, and `control-openhands browser attr 'testid=agent-profile-delete' title` is `The default profile can't be renamed or deleted.` (screenshot `--feature F13.delete --name default-locked-menu`); `browser press Escape`. `control-openhands browser count 'testid=agent-profile-active-badge'` is `1`; `control-openhands browser screenshot --feature F13.list --name list`. Command menu: run `control-openhands browser goto /` (the browser is on `/settings/agents`, where the URL wait would pass at once), `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' 'Agent settings'`, `control-openhands browser press Enter`, `control-openhands browser wait-url '/settings/agents(\?|$)'`. Legacy URL: `control-openhands browser goto /settings/agent` returns `.../settings/agents`.- **No LLM profile (`F13.editor-no-llm`).** Before the preset, run `control-openhands browser click 'testid=add-agent-profile'`, then `control-openhands browser text 'testid=agent-profile-no-llm'`: `Create an LLM profile first to define an OpenHands agent.` `control-openhands browser text 'testid=agent-profile-editor-title'` is `Add agent profile` and `control-openhands browser text 'testid=agent-profile-editor-description'` is `Configure the agent below, then click Save to create this profile.`. Run `control-openhands browser fill 'testid=agent-profile-name-input' QA_NoLlm`; Save turns enabled (`control-openhands browser enabled 'testid=save-agent-profile-btn'`). Click `testid=save-agent-profile-btn` and run `control-openhands browser toasts`: `Select an LLM profile for this agent.` `api GET /api/agent-profiles` lists no `QA_NoLlm`. Leave with `control-openhands browser click 'testid=cancel-agent-profile-btn'`.-- **Name rules (`F13.name-validation`).** In the Add editor, fill `testid=agent-profile-name-input` with `''`, `default`, `'bad name!'` and `QA_ok` in turn; after each run `control-openhands browser attr 'testid=agent-profile-name-input' aria-invalid` and `control-openhands browser enabled 'testid=save-agent-profile-btn'`. Results: `''` → `true`/`false`; `default` (duplicate) → `true`/`false`; `'bad name!'` → `true`/`false`; `QA_ok` → `false`/`true`. The rule under the field is always shown (grey) and turns the theme danger colour (`text-danger`, #18199) while the name is invalid; it reads `Profile name must start with a letter or digit, and contain only letters, digits, dots, underscores, or hyphens (max 64 characters)`. For the duplicate, `control-openhands browser count 'role=alert >> text=already exists'` is `1`: the alert is the field's description, in the danger colour (`rgb(231, 106, 94)` on OpenHands-Neutral, `rgb(209, 52, 56)` on Light+), while the rule text stays grey. It reads `An agent profile with this name already exists. Choose a different name, or edit the existing one.` Take `control-openhands browser screenshot --feature F13.name-validation --name duplicate` with `default` filled. In Edit the same message appears for another profile's name and not for the profile's own name (`F13.edit`).+- **Name rules (`F13.name-validation`).** In a fresh Add editor, before typing, `browser attr 'testid=agent-profile-name-input' aria-invalid` is `false` and Save is disabled. Then fill `testid=agent-profile-name-input` with `QA_ok`, `''`, `default` and `'bad name!'` in turn; after each run `control-openhands browser attr 'testid=agent-profile-name-input' aria-invalid` and `control-openhands browser enabled 'testid=save-agent-profile-btn'`. Results: `QA_ok` → `false`/`true`; `''` → `true`/`false`; `default` (duplicate) → `true`/`false`; `'bad name!'` → `true`/`false`. An untouched editor does not flag its empty name (`false`/`false`). The rule under the field is always shown (grey) and turns the theme danger colour (`text-danger`, #18199) while the name is invalid; it reads `Profile name must start with a letter or digit, and contain only letters, digits, dots, underscores, or hyphens (max 64 characters)`. For the duplicate, `control-openhands browser count 'role=alert >> text=already exists'` is `1`: the alert is the field's description, in the danger colour (`rgb(231, 106, 94)` on OpenHands-Neutral, `rgb(209, 52, 56)` on Light+), while the rule text stays grey. It reads `An agent profile with this name already exists. Choose a different name, or edit the existing one.` Take `control-openhands browser screenshot --feature F13.name-validation --name duplicate` with `default` filled. In Edit the same message appears for another profile's name and not for the profile's own name (`F13.edit`).- **ACP form (`F13.acp-form`).** In the Add editor run `control-openhands browser click 'testid=agent-type-selector'` and `control-openhands browser click 'role=option[name="ACP (external subprocess)"]'`. `control-openhands browser value 'testid=agent-preset-selector'` is `Claude Code`, `browser value 'testid=agent-command-input'` is `npx -y --prefer-offline @agentclientprotocol/[email protected]` and `browser value 'testid=agent-model-selector'` is `Agent default`; `agent-profile-no-llm` is gone (ACP needs no LLM profile). The preset list (`control-openhands browser click 'div:has(> div > input[data-testid=agent-preset-selector]) >> role=button[name="Show suggestions"]'`, then `control-openhands browser snapshot 'role=listbox'`) is `Claude Code`, `Codex`, `Gemini CLI`, `Pi`, `OpenCode`, `Custom` (`browser press Escape`). Pick `Codex` and `Gemini CLI` from `testid=agent-preset-selector`: the command becomes `npx -y --prefer-offline @agentclientprotocol/[email protected]` / `npx -y --prefer-offline @google/[email protected] --acp` and the model `Agent default` / `gemini-2.5-pro` (the models come from the agent since #18131: `F13.acp-model-discovery`). Pick `Custom`: the command is empty and Save is disabled. `control-openhands browser fill 'testid=agent-command-input' 'npx -y qa-custom-acp --stdio'` enables Save. Filling the exact Codex command flips the preset to `Codex`; appending ` --verbose` flips it back to `Custom`. Model override: with a built-in preset, run `control-openhands browser choose 'testid=agent-model-selector' 'Custom'` (or open the model list with its toggle button and `control-openhands browser click 'role=option[name="Custom"][exact]'`); `control-openhands browser count 'testid=agent-model-input'` is `1` and empty. The **Custom** preset shows `agent-model-input` directly (no model list).- **Create an ACP profile (`F13.create`).** With preset `Custom`, fill `testid=agent-profile-name-input` with `QA_acp_custom`, `testid=agent-command-input` with `'npx -y qa-custom-acp --stdio'` and `testid=agent-model-input` with `qa-model`, click `testid=save-agent-profile-btn`, then `control-openhands browser wait-text 'Profile "QA_acp_custom" created' --timeout 10000`, `control-openhands browser reload` and `control-openhands browser text 'testid=agent-profile-row >> has-text=QA_acp_custom'`: `QA_acp_custom` / `ACP (external subprocess)`. `control-openhands api GET /api/agent-profiles/QA_acp_custom` shows `acp_server` `custom`, the command and `acp_model` `qa-model`.- **Pi and OpenCode presets (`F13.acp-local-presets`).** For each of `OpenCode` and `Pi`: run `control-openhands browser click 'testid=add-agent-profile'`, `control-openhands browser choose 'testid=agent-type-selector' 'ACP (external subprocess)'` and `control-openhands browser choose 'testid=agent-preset-selector' '<preset>'`. `browser value 'testid=agent-command-input'` is `npx -y --prefer-offline [email protected] acp` / `npx -y --prefer-offline [email protected] --package=@earendil-works/[email protected] pi-acp` (the versions follow `@openhands/typescript-client`). Fill `testid=agent-profile-name-input` with `QA_opencode` / `QA_pi`, run `control-openhands browser wait 'testid=agent-model-discovering' --state detached --timeout 120000`, click `testid=save-agent-profile-btn` and `control-openhands browser wait-text 'Profile "<name>" created' --timeout 10000`. `control-openhands api GET /api/agent-profiles/<name>` has `acp_server` `opencode` / `pi`, `acp_command` `null` and `acp_model` `null`, and after `browser reload` both rows read `<name>` / `ACP (external subprocess)`. Cloud hides both presets (blocked here: needs a Cloud backend).@@ -63,22 +72,28 @@ Preconditions:- **Edit and rename (`F13.edit`).** Run `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_acp_custom >> testid=agent-profile-menu-trigger'`; `control-openhands browser snapshot 'testid=agent-profile-actions-menu'` lists `Edit`, `Set as active`, `Delete`. Click `testid=agent-profile-edit`: the title is `Edit agent profile`, the description `Editing profile "QA_acp_custom" - save to apply changes`, the type/preset/command/model values are the stored ones, and Save is disabled. Note the `id` from `api GET /api/agent-profiles`, fill the name with `QA_acp_renamed`, click Save, `control-openhands browser wait-text 'Profile "QA_acp_renamed" updated' --timeout 10000` and `browser reload`: `browser count 'testid=agent-profile-row >> has-text=QA_acp_custom'` is `0`, the renamed row is `1` and `api GET /api/agent-profiles` shows the same id under the new name. Back/Cancel: open any editor, change a field, click `testid=back-to-agent-profiles` (or `testid=cancel-agent-profile-btn`); `browser count 'testid=add-agent-profile'` is `1` and the API is unchanged.- **Credentials (`F13.acp-credentials`).** In a new ACP editor with preset `Claude Code`, `control-openhands browser testids --hidden --filter acp` lists `settings-acp-secret-CLAUDE_CODE_OAUTH_TOKEN`, `-ANTHROPIC_API_KEY`, `-ANTHROPIC_BASE_URL` and one banner (`settings-acp-auth-detected` when the agent-server machine has a Claude CLI login: `You're already signed in to Claude Code — you can leave the fields below blank.`). `control-openhands browser attr 'testid=settings-acp-secret-ANTHROPIC_API_KEY' type` is `password`. Fill `settings-acp-secret-CLAUDE_CODE_OAUTH_TOKEN` with `qa-dummy-oauth` and `settings-acp-secret-ANTHROPIC_API_KEY` with `qa-dummy-key`; `control-openhands browser text 'testid=acp-credential-conflict-warning'` is `Setting ANTHROPIC_API_KEY alongside CLAUDE_CODE_OAUTH_TOKEN breaks its authentication — leave one of them blank.` (screenshot `--feature F13.acp-credentials --name conflict` after `browser scroll 'testid=acp-credential-conflict-warning'`). In every theme the banner text reaches 4.5:1: the detected banner (`settings-acp-auth-detected`) uses success text on a success tint (Light+ about 5.2:1) and the configured banner (`settings-acp-auth-configured`) neutral text on the raised surface (Light+ about 14:1); screenshot `--feature F13.acp-credentials --name detected-banner-lightplus`. Cancel; `api GET /api/settings/secrets` gains nothing. Saved credential: new profile `QA_codex`, ACP, preset `Codex`, fill `testid=settings-acp-secret-OPENAI_API_KEY` with `qa-dummy-openai`, Save (`Profile "QA_codex" created`); `api GET /api/settings/secrets` now lists `OPENAI_API_KEY` and the profile has `acp_server` `codex`, `acp_command` `null`, `acp_model` `null` (the agent's own default since #18131). Reopen it: `control-openhands browser testids --hidden --filter settings-acp-auth` shows `settings-acp-auth-configured` (`Credentials for Codex are configured — you can leave the fields below blank.`) and `control-openhands browser attr 'testid=settings-acp-secret-OPENAI_API_KEY' placeholder` is `Already saved — leave blank to keep` with an empty value.- **Secrets scope (`F13.secrets-scope`).** In the `QA_codex` editor, `control-openhands browser eval "[...document.querySelectorAll('[data-testid=agent-settings-secret-list] input')].map(i=>i.closest('li').innerText.split('\n')[0]+'='+i.checked+(i.disabled?'(disabled)':''))"` shows every name `=true(disabled)` under `No profile restriction`. Open `testid=agent-settings-secrets-mode`, `control-openhands browser click 'role=option[name="Choose secrets"]'` and rerun the eval: `OPENHANDS_AUTOMATION_API_KEY=false`, `OPENAI_API_KEY=true`, `CODEX_AUTH_JSON=true`, `OPENAI_BASE_URL=true`. Save (`Profile "QA_codex" updated`); `api GET /api/agent-profiles/QA_codex` has those three in `secret_refs`, and after `browser reload` and Edit the mode and switches are the same. On an OpenHands profile, Choose starts with nothing selected; a switch toggles by its name: in the Add editor pick `Choose secrets`, then `control-openhands browser click 'testid=agent-settings-secret-list >> text=OPENHANDS_AUTOMATION_API_KEY'` turns `document.querySelector('[data-testid=agent-settings-secret-OPENHANDS_AUTOMATION_API_KEY]').checked` from `false` to `true`.-- **Stale LLM reference (`F13.stale-llm-ref`).** After `control-openhands llm preset deepseek`, run `control-openhands browser reload`, open `[data-testid=agent-profile-row]:has([title="default"]) >> testid=agent-profile-menu-trigger` → `testid=agent-profile-edit`, `control-openhands browser wait 'testid=agent-profile-llm-selector'`. `browser value 'testid=agent-profile-llm-selector'` is `deepseek-flash (deepseek/deepseek-flash)` and Save is enabled although nothing was touched. Save (`Profile "default" updated`), `browser reload`: the row reads `default` / `deepseek-flash` / `Default` and `api GET /api/agent-profiles/default` has `llm_profile_ref` `deepseek-flash` with the same id. Screenshot `--feature F13.stale-llm-ref --name fallback` before saving. Launch half: an LLM profile that an agent profile references cannot be deleted (`DELETE /api/profiles/<ref>` answers 409 `LLM profile is referenced by 1 agent profile(s)`), so arrange a named profile's dangling reference with `control-openhands api POST /api/agent-profiles/<name> --write` setting its `llm_profile_ref` to a missing name, activate it, and start a Home chat: `api GET /api/conversations/<id> --pick launched_agent_profile` keeps the profile's id with `llm_profile_ref` set to the active LLM profile.+- **Stale LLM reference (`F13.stale-llm-ref`).** After `control-openhands llm preset deepseek`, run `control-openhands browser reload`, open `[data-testid=agent-profile-row]:has([title="default"]) >> testid=agent-profile-menu-trigger` → `testid=agent-profile-edit`, `control-openhands browser wait 'testid=agent-profile-llm-selector'`. `browser value 'testid=agent-profile-llm-selector'` is `deepseek-flash (deepseek/deepseek-flash)` and Save is enabled although nothing was touched. Save (`Profile "default" updated`), `browser reload`: the row reads `default` / `deepseek-flash` / `Active` and `api GET /api/agent-profiles/default` has `llm_profile_ref` `deepseek-flash` with the same id. Screenshot `--feature F13.stale-llm-ref --name fallback` before saving.+- **Shared `default` (`F13.default-shared`, `F13.edit`).** In that same `default` editor (before saving), `control-openhands browser enabled 'testid=agent-profile-name-input'` is `false` and `control-openhands browser text 'testid=agent-profile-default-locked-note'` is `The default profile can't be renamed or deleted.`. `control-openhands browser testids 'testid=agent-settings-screen' --hidden --filter note` lists `agent-profile-llm-default-note`, `agent-settings-mcp-default-note`, `agent-settings-secrets-default-note` and `agent-settings-skills-default-note` (texts in Sub-features); `browser count 'testid=agent-settings-secrets-mode'` is `0` (no secrets picker), and `agent-settings-router-select` (`Off`), `agent-settings-condenser` and `agent-settings-verification` are present (screenshot `--feature F13.default-shared --name default-editor`).+- **Stale reference at launch (`F13.stale-llm-ref`).** Launch half: an LLM profile that an agent profile references cannot be deleted (`DELETE /api/profiles/<ref>` answers 409 `LLM profile is referenced by 1 agent profile(s)`), so arrange a named profile's dangling reference with `control-openhands api POST /api/agent-profiles/<name> --write` setting its `llm_profile_ref` to a missing name, activate it, and start a Home chat: `api GET /api/conversations/<id> --pick launched_agent_profile` keeps the profile's id with `llm_profile_ref` set to the active LLM profile.- **OpenHands options (`F13.openhands-options`, `F13.create`).** Click `testid=add-agent-profile`, fill the name `QA_oh`. Defaults: `browser value` on `testid=agent-profile-llm-selector` is the active LLM profile (`deepseek-flash (deepseek/deepseek-flash)`), on `testid=agent-settings-system-prompt-mode` `OpenHands default`, on `testid=agent-settings-tools-mode` `Standard tools` and on `testid=sdk-settings-tool_concurrency_limit` `1`; `control-openhands browser text 'testid=agent-settings-tool-list'` lists `file_editor`, `task_tracker`, `terminal` and `switch_llm` with their descriptions (the list follows the Agent Server; it also includes `browser_tool_set` when usable). Run `control-openhands browser choose 'testid=agent-settings-tools-mode' 'Choose tools'`; `control-openhands browser eval "[...document.querySelectorAll('[data-testid=agent-settings-tool-list] input')].map(i=>i.dataset.testid.replace('agent-settings-tool-','')+'='+i.checked)"` is the selected standard tools, including `file_editor=true`, `task_tracker=true`, `terminal=true` and `switch_llm=true`; `task_tool_set=false`, `glob=false` and `grep=false`. Additional rows follow the current server catalog. Click `'testid=agent-settings-tool-list >> text=switch_llm'` and `'testid=agent-settings-tool-list >> text=task_tool_set'` (the eval flips both). Fill `testid=sdk-settings-tool_concurrency_limit` with `0` and click Save: `browser toasts` shows `Parallel tool calls must be at least 1` and `browser eval "document.querySelector('[data-testid=sdk-settings-tool_concurrency_limit]').validationMessage"` is `Value must be greater than or equal to 1.` Fill `2`, `control-openhands browser choose 'testid=agent-profile-llm-selector' 'deepseek-pro (deepseek/deepseek-v4-pro)'`, Save (`Profile "QA_oh" created`), `browser reload`: the row reads `QA_oh` / `deepseek-pro`, and `api GET /api/agent-profiles/QA_oh` has `llm_profile_ref` `deepseek-pro`, `tools` retaining the selected standard tools plus `task_tool_set` (including `task_tracker`), `enable_classify_and_switch_llm_tool` `false` and `tool_concurrency_limit` `2`. Reopening shows `Choose tools` with the same switches, `2` and `deepseek-pro`, with Save disabled. (There are no **Enable sub-agents** or **Let the agent switch LLM profiles** switches: `browser count 'text=Enable sub-agents'` is `0`.)System prompt: reopen `QA_oh`, `control-openhands browser choose 'testid=agent-settings-system-prompt-mode' 'Default + your instructions'` and fill `testid=agent-settings-system-prompt` with `Use terse QA answers.`. Save, `browser reload` and reopen: `api GET /api/agent-profiles/QA_oh` has `system_message_suffix` `Use terse QA answers.` and `persona` `null`, and the editor keeps the mode and text with Save disabled. Then choose `Custom persona`, fill `You are the disposable QA persona.`, Save, reload and reopen: `persona` is that text, `system_message_suffix` is `null`, and the editor keeps the mode and text. Choose `OpenHands default` and Save (both fields `null`) before later checks that use this profile.+- **Model Router per profile (`F13.profile-router`).** With `QA_router` and `QA_router_2` arranged, Edit `QA_oh`, `control-openhands browser wait 'testid=agent-settings-router-select'`: its value is `Off`; opening it (`browser click 'div:has(> div > input[data-testid=agent-settings-router-select]) >> role=button[name="Show suggestions"]'`, `browser snapshot 'role=listbox'`) lists `Off`, then every saved router (`QA_router`, `QA_router_2`, …) (`browser press Escape`). `control-openhands browser choose 'testid=agent-settings-router-select' 'QA_router'`, Save (`Profile "QA_oh" updated`): `api GET /api/agent-profiles/QA_oh` has `meta_profile_ref` `QA_router` and `enable_classify_and_switch_llm_tool` `true`, and reopening shows `QA_router` with Save disabled (screenshot `--feature F13.profile-router --name editor-router`). The section is absent on ACP profiles. Missing router: arrange `api POST /api/agent-profiles/QA_oh --write` with the stored profile's `meta_profile_ref` set to `QA_gone_router`, reopen: the value is `QA_gone_router` and `control-openhands browser text 'testid=agent-settings-router-dangling'` is `QA_gone_router no longer exists. Saving with it selected will stop this agent from starting. Choose another router or turn it off.` (screenshot `--name dangling`); choose `Off` (the warning goes), Save: `meta_profile_ref` `null`, `enable_classify_and_switch_llm_tool` `false`. Arranging the flag `true` with `meta_profile_ref` `null` reopens on `Alphabetically first router` with Save disabled. Restore `QA_router` through the dropdown before `F13.active-drives-conversation`. Deleting a router on the Model Router page (`F12.delete`) leaves every profile that used it at `meta_profile_ref` `null`, flag `false`.+- **Skills per profile (`F13.profile-skills`).** In `QA_oh`'s editor, `control-openhands browser eval "[...document.querySelectorAll('[data-testid=agent-settings-skills-list] input')].filter(i=>i.checked).map(i=>i.dataset.testid.replace('agent-settings-skills-',''))"` is the Skills page selection (on a fresh run the 13 recommended skills, including `github`, `docker` and `code-review`); each row is labelled `Catalog` or `Your skill`. `control-openhands browser fill 'testid=agent-settings-skills-search' github` narrows the list; `control-openhands browser click 'testid=agent-settings-skills-list >> text="github"'` turns `document.querySelector('[data-testid=agent-settings-skills-github]').checked` `false` (an unquoted `text=github` can hit another row). Clear the search, Save: `public_skills` lacks `github`, and reopening keeps it off. A conversation launched from `QA_oh` lists the other skills but not `github` in its system prompt (`control-openhands api GET "/api/conversations/<id>/events/search?limit=1" --max-bytes 400000`, grep `<name>github</name>`: `0`), while one launched from `default` lists it.+- **New profiles copy `default` (`F13.create`).** Edit `default`, set the condenser `max_size` (All tab) to `200` and Parallel tool calls to `3`, Save; click `testid=add-agent-profile`: the new editor opens on `200` and `3`, `agent-settings-mcp-mode` `All servers`, `agent-settings-secrets-mode` `No profile restriction` (screenshot `--feature F13.create --name new-copies-default`). Cancel, and restore `default` to `240` and `1`.- **Set as active (`F13.set-active`).** Run `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_oh >> testid=agent-profile-menu-trigger'`, `control-openhands browser click 'testid=agent-profile-set-active'` and `control-openhands browser wait-text 'Switched to profile "QA_oh"' --timeout 10000`. After `browser reload`, `browser count 'testid=agent-profile-row >> has-text=QA_oh >> testid=agent-profile-active-badge'` and `browser count 'testid=agent-profile-active-badge'` are both `1`. Reopen the QA_oh menu: `browser enabled 'testid=agent-profile-set-active'` is `false` (screenshot `--feature F13.set-active --name menu`); `control-openhands browser press Escape` closes it (`browser count 'testid=agent-profile-actions-menu'` is `0`).-- **Pill wins over a pinned LLM (`F13.llm-pill-precedence`).** With `QA_oh` active (pinned to `deepseek-pro`) and the Home pill on `deepseek-flash` (`browser text 'testid=chat-input-llm-profile'` on `/`), run `control-openhands conversation start --prompt "Reply with only: qa-profile-ok. Do not run tools." --wait --timeout 240`. The output's `model` is `deepseek/deepseek-flash`. On the current backend, `control-openhands api GET /api/conversations/<id>` (id from that output) has `launched_agent_profile.agent_profile_id` equal to QA_oh's id, `launched_agent_profile.llm_profile_ref` `deepseek-flash`, and `agent.tool_concurrency_limit` `2`: the per-launch LLM override keeps the named profile's other settings. These metadata checks can be made before a successful reply; a refused-loopback fixture proves the launch configuration only.-- **Active profile drives the launch (`F13.active-drives-conversation`).** Run `control-openhands browser goto /settings/agents` (the previous `conversation start` left the browser on the conversation), edit `QA_oh`, run `control-openhands browser choose 'testid=agent-profile-llm-selector' 'deepseek-flash (deepseek/deepseek-flash)'` (the toggle-button opener failed 5 of 5 times here: see Gotchas), check `browser value 'testid=agent-profile-llm-selector'`, Save (`Profile "QA_oh" updated`). Run the same `conversation start` again; `api GET /api/conversations/<id>` now has `launched_agent_profile.agent_profile_id` equal to QA_oh's id from `api GET /api/agent-profiles` and `agent.tool_concurrency_limit` `2`; `control-openhands conversation events <id> --kinds MessageEvent` contains `qa-profile-ok`.-- **MCP scope (`F13.mcp-scope`).** With `qa_mcp_a` and `qa_mcp_b` arranged, `browser goto /settings/agents`, Edit `QA_oh`: `browser value 'testid=agent-settings-mcp-mode'` is `All servers` and the eval below (with `+(i.disabled?'(disabled)':'')` appended) shows both `=true(disabled)`. Pick `Choose servers` from `testid=agent-settings-mcp-mode`; `control-openhands browser eval "[...document.querySelectorAll('[data-testid=agent-settings-mcp-list] input')].map(i=>i.dataset.testid+'='+i.checked)"` shows both `=true`. Run `control-openhands browser click 'testid=agent-settings-mcp-list >> text=qa_mcp_b'` (now `false`), Save; `api GET /api/agent-profiles/QA_oh` has `mcp_server_refs` `["qa_mcp_a"]`. Dangling: arrange `control-openhands api DELETE /api/settings/mcp/qa_mcp_a --write`, `browser reload`, Edit `QA_oh`: `control-openhands browser text 'testid=agent-settings-mcp-dangling'` is `qa_mcp_a is no longer configured. Saving with it selected will stop this agent from starting — clear it or re-add the server.` (screenshot `--feature F13.mcp-scope --name dangling`). Leave without saving, `browser goto /`, `browser reload`, `control-openhands browser fill 'testid=chat-input' 'Reply with only: qa-dangling'`, `control-openhands browser click 'testid=submit-button' --observe 'role=status' --observe-ms 4000` and `control-openhands browser toasts --history` (QA_oh must still be active; on older backends without per-launch LLM override it must also be pinned to the pill's LLM): the launch is refused (one `POST /api/conversations` 422 in `browser errors --app-only`; earlier failures in the session are listed too): `browser toasts --history` shows `Creating conversation…`, then one toast carrying the server's `detail.message` (`MCP server(s) not configured: qa_mcp_a` on Agent Server 1.53.0), and `browser text 'testid=chat-input'` still reads `Reply with only: qa-dangling`. Clear the composer with `browser fill 'testid=chat-input' ''`, then clear the ref: `control-openhands browser goto /settings/agents`, Edit `QA_oh`, `browser click 'testid=agent-settings-mcp-list >> text=qa_mcp_a'` (the warning count drops to `0`), pick `All servers`, Save; `mcp_server_refs` is `null`.+- **The profile's LLM on Home (`F13.llm-pill-precedence`).** With `QA_oh` active (pinned to `deepseek-pro`) and `deepseek-flash` the active LLM profile, `control-openhands browser goto /`: `browser text 'testid=chat-input-agent-profile'` is `QA_oh` and `browser text 'testid=chat-input-llm-profile'` is `deepseek-pro` (screenshot `--feature F13.llm-pill-precedence --name home-pills`). `control-openhands browser click 'testid=chat-input-llm-profile'`, `browser click 'testid=chat-input-llm-profile-option-deepseek-flash'`: `api GET /api/agent-profiles/QA_oh --pick profile.llm_profile_ref` becomes `deepseek-flash` while `api GET /api/profiles --pick active_profile` stays `deepseek-flash`; pick `deepseek-pro` the same way: the profile is back on `deepseek-pro` and the active LLM profile is still `deepseek-flash`. `control-openhands conversation start --prompt "Reply with only the word pong. Do not run any tools." --wait --timeout 240` (with **Run on first message** off, or no router on the profile) reports `model` `deepseek/deepseek-v4-pro`, and `control-openhands api GET /api/conversations/<id> --pick launched_agent_profile` has `llm_profile_ref` `null`. Switch to `default` in the agent pill (`browser click 'testid=chat-input-agent-profile'`, `browser click 'testid=chat-input-agent-profile-popover >> role=button[name="default"]'`): the LLM pill reads `deepseek-flash`, the active LLM profile.+- **Profile condenser for the launch check (`F13.active-drives-conversation`).** In `QA_oh`'s editor, `control-openhands browser click 'testid=agent-settings-condenser >> testid=sdk-section-all-toggle'` and `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 120`, then Save: `api GET /api/agent-profiles/QA_oh --pick profile.condenser` has `max_size` `120`. The editor's Condenser and Verification sections themselves are `F15.profile-behavior`.+- **Active profile drives the launch (`F13.active-drives-conversation`).** With `QA_oh` active (`deepseek-pro`, Parallel tool calls `2`, router `QA_router`, condenser `120`, `github` off) and **Run on first message** on, run `control-openhands conversation start --prompt "Run echo routed in the terminal and tell me its output." --wait --timeout 300`. `api GET /api/conversations/<id>` has `launched_agent_profile.agent_profile_id` equal to QA_oh's id, `agent.tool_concurrency_limit` `2`, `agent.condenser.max_size` `120` (its `agent.llm.model` is the routed profile's, `deepseek/deepseek-flash`, once `route_task_to_model` has run) and `ClassifyAndSwitchLLMTool` among `agent.tools`; `control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent` starts with `route_task_to_model` (`Classified task as 'model: deepseek-flash' and switched to LLM profile 'deepseek-flash' …`), then `echo routed`; the page shows `Switched to profile deepseek-flash` (screenshot `--feature F13.active-drives-conversation --name qa-oh-conversation`). The skills check is in `F13.profile-skills`. Then switch to `default` in the Home agent pill and run `conversation start --prompt "Reply with only: qa-default-ok. Do not run any tools." --wait --timeout 240`: `launched_agent_profile.agent_profile_id` is default's id, the model is the active LLM profile's, `agent.tool_concurrency_limit` `1`, no router tool, and the reply is `qa-default-ok`.+- **MCP scope (`F13.mcp-scope`).** With `qa_mcp_a` and `qa_mcp_b` arranged, `browser goto /settings/agents`, Edit `QA_oh`: `browser value 'testid=agent-settings-mcp-mode'` is `All servers` and the eval below (with `+(i.disabled?'(disabled)':'')` appended) shows both `=true(disabled)`. Pick `Choose servers` from `testid=agent-settings-mcp-mode`; `control-openhands browser eval "[...document.querySelectorAll('[data-testid=agent-settings-mcp-list] input')].map(i=>i.dataset.testid+'='+i.checked)"` shows both `=true`. Run `control-openhands browser click 'testid=agent-settings-mcp-list >> text=qa_mcp_b'` (now `false`), Save; `api GET /api/agent-profiles/QA_oh` has `mcp_server_refs` `["qa_mcp_a"]`. Dangling: arrange `control-openhands api DELETE /api/settings/mcp/qa_mcp_a --write`, `browser reload`, Edit `QA_oh`: `control-openhands browser text 'testid=agent-settings-mcp-dangling'` is `qa_mcp_a is no longer configured. Saving with it selected will stop this agent from starting — clear it or re-add the server.` (screenshot `--feature F13.mcp-scope --name dangling`). Leave without saving, `browser goto /`, `browser reload`, `control-openhands browser fill 'testid=chat-input' 'Reply with only: qa-dangling'`, `control-openhands browser click 'testid=submit-button' --observe 'role=status' --observe-ms 4000` and `control-openhands browser toasts --history` (QA_oh must still be active): the launch is refused (one `POST /api/conversations` 422 in `browser errors --app-only`; earlier failures in the session are listed too): `browser toasts --history` shows `Creating conversation…`, then one toast carrying the server's `detail.message` (`MCP server(s) not configured: qa_mcp_a` on Agent Server 1.53.0), and `browser text 'testid=chat-input'` still reads `Reply with only: qa-dangling`. Clear the composer with `browser fill 'testid=chat-input' ''`, then clear the ref: `control-openhands browser goto /settings/agents`, Edit `QA_oh`, `browser click 'testid=agent-settings-mcp-list >> text=qa_mcp_a'` (the warning count drops to `0`), pick `All servers`, Save; `mcp_server_refs` is `null`.- **Credential fields per preset (`F13.acp-preset-credentials`).** In the Add editor, `control-openhands browser choose 'testid=agent-type-selector' 'ACP (external subprocess)'`, then for each of `Codex`, `Gemini CLI` and `Custom` run `control-openhands browser choose 'testid=agent-preset-selector' '<preset>'` and `control-openhands browser testids --hidden --filter settings-acp`: Codex lists `settings-acp-secret-OPENAI_API_KEY` (input), `-CODEX_AUTH_JSON` (textarea), `-OPENAI_BASE_URL`; Gemini CLI lists `-GOOGLE_APPLICATION_CREDENTIALS_JSON` (textarea), `-GOOGLE_CLOUD_PROJECT`, `-GOOGLE_CLOUD_LOCATION`, `-GOOGLE_GENAI_USE_VERTEXAI`, `-GEMINI_API_KEY`, `-GEMINI_BASE_URL`; Custom lists nothing and `browser eval "document.body.innerText.includes('Credentials')"` is `false`. For Gemini, `control-openhands browser wait 'testid=settings-acp-auth-checking' --state detached --timeout 15000`, then `browser testids --hidden --filter settings-acp-auth` has count `0` (no login, no stored credential); `browser eval "document.querySelector('[data-testid=settings-acp-secret-GEMINI_API_KEY]').type"` is `password`, `GOOGLE_CLOUD_PROJECT` is `text`. Screenshot `--feature F13.acp-preset-credentials --name gemini` after `browser scroll 'testid=settings-acp-secret-GOOGLE_APPLICATION_CREDENTIALS_JSON'`. Cancel (Codex shows `settings-acp-auth-configured` instead while `OPENAI_API_KEY` from `F13.acp-credentials` is still saved).-- **Menu keyboard (`F13.menu-keyboard`).** Run `control-openhands browser goto /settings/agents`. It needs a non-active row: run `control-openhands browser click 'testid=add-agent-profile'`, `control-openhands browser fill 'testid=agent-profile-name-input' QA_kb`, `control-openhands browser click 'testid=save-agent-profile-btn'` and `control-openhands browser wait-text 'Profile "QA_kb" created' --timeout 10000`. Open its menu with `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_kb >> testid=agent-profile-menu-trigger'` and right away run `control-openhands browser eval "document.activeElement?.dataset?.testid"`: `agent-profile-edit` (opening the menu moves focus to its first enabled item, as the LLM profile menu does in `F10.actions-menu`). Then `control-openhands browser press ArrowDown` followed by that eval four times: `agent-profile-set-active`, `agent-profile-delete`, `agent-profile-edit`, `agent-profile-set-active`; `browser press ArrowUp` returns to `agent-profile-edit`. `control-openhands browser press Escape` closes the menu (`control-openhands browser count 'testid=agent-profile-actions-menu'` is `0`) and returns focus to the row's own trigger (`control-openhands browser eval "(()=>{const a=document.activeElement;const r=a.closest('[data-testid=agent-profile-row]');return a.dataset.testid+'@'+(r?r.innerText.split('\\n')[0]:'-')})()"` is `agent-profile-menu-trigger@QA_kb`); reopen it and `control-openhands browser press Tab`: closed, and focus moves on past the trigger to the next row's trigger; keyboard open (`control-openhands browser focus` on the trigger, then `browser press Enter`) also lands on `agent-profile-edit`; reopen it and run `control-openhands browser mouse-click 900 600`: closed. Active row: open its menu with `control-openhands browser click '[data-testid=agent-profile-row]:has([data-testid=agent-profile-active-badge]) >> testid=agent-profile-menu-trigger'` (`control-openhands browser snapshot 'testid=agent-profile-actions-menu'` shows `menuitem "Set as active" [disabled]`; focus is already on `agent-profile-edit`), then `control-openhands browser press ArrowDown` and the eval, then `control-openhands browser press ArrowUp` and the eval: `agent-profile-delete` (the disabled **Set as active** is skipped), then `agent-profile-edit` (`control-openhands browser screenshot --feature F13.menu-keyboard --name active-row-menu`). At `browser viewport phone` the menu opens inside the viewport with the same focus order. With the menu still open, run `control-openhands browser focus 'testid=agent-profile-edit'` and `control-openhands browser press Enter`: `control-openhands browser text 'testid=agent-profile-editor-title'` is `Edit agent profile`. Leave the editor with `control-openhands browser click 'testid=cancel-agent-profile-btn'` (`browser count 'testid=add-agent-profile'` is `1`). Delete `QA_kb`: reopen its menu, `control-openhands browser click 'testid=agent-profile-delete'`, `control-openhands browser click 'testid=delete-agent-profile-confirm'` and `control-openhands browser wait-text 'Profile "QA_kb" deleted' --timeout 10000`.-- **Delete (`F13.delete`).** Run `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_acp_renamed >> testid=agent-profile-menu-trigger'`, `control-openhands browser click 'testid=agent-profile-delete'`; `control-openhands browser text 'role=dialog'` reads `Delete Profile` and `Are you sure you want to delete the profile "QA_acp_renamed"? This action cannot be undone.` `control-openhands browser click 'role=dialog >> role=button[name="Cancel"]'` keeps the row (count `1`). Repeat and `control-openhands browser click 'testid=delete-agent-profile-confirm'`, `browser wait-text 'Profile "QA_acp_renamed" deleted' --timeout 10000`, `browser reload`: count `0`. Delete the active `QA_oh` the same way: after `browser reload`, `browser count 'testid=agent-profile-active-badge'` is `1`, the `default` row has that badge, and `api GET /api/agent-profiles` has `active_agent_profile_id` equal to default's id. Delete `QA_codex`, `QA_opencode`, `QA_pi` and `default`: the list again shows one row, `default` / `deepseek-flash` / `Default`, with a new id (the server re-seeds it).-- **Phone (`F13.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/agents`, `control-openhands browser bbox 'testid=agent-profile-row'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `browser screenshot --feature F13.phone --name list`. Open the row menu: `browser bbox 'testid=agent-profile-actions-menu'` is inside the viewport. `browser press Escape`, click `testid=add-agent-profile`: `browser bbox 'testid=agent-settings-screen'` is 348 px wide at x 16 with no page overflow (`insideViewport` is `false` only because the form is taller than the screen). Switch to ACP, `browser scroll 'testid=settings-acp-secret-ANTHROPIC_API_KEY'` and screenshot `--name acp-credentials`. Cancel and `control-openhands browser viewport desktop`.+- **Menu keyboard (`F13.menu-keyboard`).** Run `control-openhands browser goto /settings/agents`. It needs a non-active row: run `control-openhands browser click 'testid=add-agent-profile'`, `control-openhands browser fill 'testid=agent-profile-name-input' QA_kb`, `control-openhands browser click 'testid=save-agent-profile-btn'` and `control-openhands browser wait-text 'Profile "QA_kb" created' --timeout 10000`. Open its menu with `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_kb >> testid=agent-profile-menu-trigger'` and right away run `control-openhands browser eval "document.activeElement?.dataset?.testid"`: `agent-profile-edit` (opening the menu moves focus to its first enabled item, as the LLM profile menu does in `F10.actions-menu`). Then `control-openhands browser press ArrowDown` followed by that eval four times: `agent-profile-set-active`, `agent-profile-delete`, `agent-profile-edit`, `agent-profile-set-active`; `browser press ArrowUp` returns to `agent-profile-edit`. `control-openhands browser press Escape` closes the menu (`control-openhands browser count 'testid=agent-profile-actions-menu'` is `0`) and returns focus to the row's own trigger (`control-openhands browser eval "(()=>{const a=document.activeElement;const r=a.closest('[data-testid=agent-profile-row]');return a.dataset.testid+'@'+(r?r.innerText.split('\\n')[0]:'-')})()"` is `agent-profile-menu-trigger@QA_kb`); reopen it and `control-openhands browser press Tab`: closed, and focus moves on past the trigger to the next row's trigger; keyboard open (`control-openhands browser focus` on the trigger, then `browser press Enter`) also lands on `agent-profile-edit`; reopen it and run `control-openhands browser mouse-click 900 600`: closed. Re-activate `QA_oh` first (its row menu → Set as active) if `default` is active. Active row: open its menu with `control-openhands browser click '[data-testid=agent-profile-row]:has([data-testid=agent-profile-active-badge]) >> testid=agent-profile-menu-trigger'` (`control-openhands browser snapshot 'testid=agent-profile-actions-menu'` shows `menuitem "Set as active" [disabled]`; focus is already on `agent-profile-edit`), then `control-openhands browser press ArrowDown` and the eval, then `control-openhands browser press ArrowUp` and the eval: `agent-profile-delete` (the disabled **Set as active** is skipped), then `agent-profile-edit` (`control-openhands browser screenshot --feature F13.menu-keyboard --name active-row-menu`). The non-active `default` row (`'[data-testid=agent-profile-row]:has([title="default"]) >> testid=agent-profile-menu-trigger'`) has **Delete** disabled: ArrowDown twice goes `agent-profile-set-active`, then back to `agent-profile-edit` (screenshot `--name default-row-menu`), then Escape. At `browser viewport phone` the menu opens inside the viewport with the same focus order. With the menu still open, run `control-openhands browser focus 'testid=agent-profile-edit'` and `control-openhands browser press Enter`: `control-openhands browser text 'testid=agent-profile-editor-title'` is `Edit agent profile`. Leave the editor with `control-openhands browser click 'testid=cancel-agent-profile-btn'` (`browser count 'testid=add-agent-profile'` is `1`). Delete `QA_kb`: reopen its menu, `control-openhands browser click 'testid=agent-profile-delete'`, `control-openhands browser click 'testid=delete-agent-profile-confirm'` and `control-openhands browser wait-text 'Profile "QA_kb" deleted' --timeout 10000`.+- **Delete (`F13.delete`).** Run `control-openhands browser click 'testid=agent-profile-row >> has-text=QA_acp_renamed >> testid=agent-profile-menu-trigger'`, `control-openhands browser click 'testid=agent-profile-delete'`; `control-openhands browser text 'role=dialog'` reads `Delete Profile` and `Are you sure you want to delete the profile "QA_acp_renamed"? This action cannot be undone.` `control-openhands browser click 'role=dialog >> role=button[name="Cancel"]'` keeps the row (count `1`). Repeat and `control-openhands browser click 'testid=delete-agent-profile-confirm'`, `browser wait-text 'Profile "QA_acp_renamed" deleted' --timeout 10000`, `browser reload`: count `0`. Delete the active `QA_oh` the same way: after `browser reload`, `browser count 'testid=agent-profile-active-badge'` is `1`, the `default` row has that badge, and `api GET /api/agent-profiles` has `active_agent_profile_id` equal to default's id. Delete `QA_codex`, `QA_opencode` and `QA_pi`: the list again shows one row, `default` / `deepseek-flash` / `Active`, with its original id. `default` itself cannot be deleted here (`F13.list` reads its disabled Delete).+- **Phone (`F13.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/agents`, `control-openhands browser bbox 'testid=agent-profile-row'` (`insideViewport` `true`, `pageHorizontalOverflow` `false`) and `browser screenshot --feature F13.phone --name list`. Open the row menu: `browser bbox 'testid=agent-profile-actions-menu'` is inside the viewport. `browser press Escape`, click `testid=add-agent-profile`: `browser bbox 'testid=agent-settings-screen'` is 348 px wide at x 16 with no page overflow (`insideViewport` is `false` only because the form is taller than the screen). `browser scroll 'testid=agent-settings-skills-section'` and screenshot `--name editor-skills` (the search box and switches fit the 348 px column). Switch to ACP, `browser scroll 'testid=settings-acp-secret-ANTHROPIC_API_KEY'` and screenshot `--name acp-credentials`. Cancel and `control-openhands browser viewport desktop`.- **Errors.** After the family, `control-openhands browser errors --app-only` shows no page errors and no `[i18n] Missing translation` warnings (the Parallel tool calls label and description are translated since #18035), only the deliberate 422s from the dangling-MCP launch.- **Blocked (`F13.acp-conversation`, `F13.cloud-read-only`).** Record `control-openhands evidence add --feature F13.acp-conversation --result blocked ...` naming the missing provider CLI/account, and the same for `F13.cloud-read-only` naming the Cloud member login.- **Cleanup.** Delete the `OPENAI_API_KEY` credential through Secrets: `control-openhands browser goto /settings/secrets`, `control-openhands browser click 'testid=secret-item >> has-text=OPENAI_API_KEY >> testid=delete-secret-button'`, `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'`. Remove `qa_mcp_b` with `control-openhands api DELETE /api/settings/mcp/qa_mcp_b --write`. Leave `default` active.-- **Default wording (`F13.list`, `F13.set-active`).** Run `control-openhands browser goto /settings/agents`, `control-openhands browser text 'testid=agent-profile-active-badge'`, open the active row's `testid=agent-profile-menu-trigger`, and `control-openhands browser snapshot 'testid=agent-profile-actions-menu'`. Expected: the menu uses the same Default wording as the badge. Known failure, reproduced 2026-10-12 ([#17426](https://github.com/OpenHands/OpenHands/issues/17426)): the badge reads `Default` and the menu reads `Set as active`. Close with Escape, reopen once to repeat, then Escape; no activation or model call is required.+- **Active wording (`F13.list`, `F13.set-active`).** Run `control-openhands browser goto /settings/agents`, `control-openhands browser text 'testid=agent-profile-active-badge'`, open the active row's `testid=agent-profile-menu-trigger`, and `control-openhands browser snapshot 'testid=agent-profile-actions-menu'`: the badge reads `Active`, matching the menu's **Set as active**. Close with Escape; no activation or model call is required.- **Seeded default key propagation (`F13.active-drives-conversation`).** With the seeded `default` agent profile active and the disposable `QA_LOOPBACK` LLM profile active, `control-openhands api GET /api/settings --pick llm_api_key_is_set` is `true`. Run `control-openhands conversation start --prompt 'Reply with exactly pong. Do not run tools.' --timeout 60` (the dummy endpoint refuses connections). Read only boolean presence of `agent.llm.api_key` in the run-owned persisted `base_state.json`, keeping the value private, and inspect `conversation status <id>`. Expected: the key is populated and initialization reaches the refused loopback endpoint rather than an absent-key error. This checks propagation before provider authentication or a successful reply.@@ -86,7 +101,7 @@ Preconditions:- **Published stale ACP binary on PATH (`F13.acp-conversation`).** Install the real published `@zed-industries/[email protected]` under the owned run's `workspace/qa-stale-adapter` (no global install). Create/activate a real Codex ACP profile through Settings > Agents with only a dummy API-key presence marker and loopback Base URL. Prepend that package's `node_modules/.bin` only during the own backend `restart`; keep its private HOME, set default OpenAI routes plus HTTP(S)/ALL proxy to refused loopback and `NO_PROXY=127.0.0.1,localhost`, with cached dependencies offline. Doctor, then start two actual Home chats and inspect the sanitized runtime command and initialization version. Expected: the SDK uses the pinned `1.10.0` adapter. Known failure, reproduced twice 2026-10-12 ([OpenHands/software-agent-sdk#5466](https://github.com/OpenHands/software-agent-sdk/issues/5466)): `acp_command` became bare `codex-acp`, and actual ACP initialization reported `agent_version=0.16.0`; the version warning still wrongly blamed npx fallback. Both then stopped at `Authentication required`. This proves selection/warning before authentication, not a usable session, the historical model-cache error or the Claude counterpart. Restore the launcher environment/PATH and default agent, delete the own profile/dummy secrets/package fixture, then doctor and stop.-- **Suffixed profile keeps its identity (`F13.edit`).** A normal edit preserves the profile UUID and increments its revision. Known failure (reproduced 2026-10-16, [OpenHands/software-agent-sdk#5873](https://github.com/OpenHands/software-agent-sdk/issues/5873)): two real concurrency-only saves of a `.json` name each minted a new UUID at revision0. SDK 1.55 repaired the active pointer to default; do not claim it remained dangling. The credential-free block checks the revision symptom and cleans its own fixtures.+- **Suffixed profile keeps its identity (`F13.edit`).** A normal edit preserves the profile UUID and increments its revision. Known failure (reproduced 2026-10-22, [OpenHands/software-agent-sdk#5873](https://github.com/OpenHands/software-agent-sdk/issues/5873)): two real concurrency-only saves of a `.json` name each minted a new UUID at revision0. SDK 1.55 repaired the active pointer to default; do not claim it remained dangling. The credential-free block checks the revision symptom and cleans its own fixtures.```sh known-bug F13.edit json-suffixtrap 'control-openhands api DELETE /api/agent-profiles/QA_KNOWN_SUFFIX --write >/dev/null' EXIT@@ -110,16 +125,18 @@ Preconditions:- The editor is not a route: `browser url` stays `/settings/agents` and `browser reload` drops unsaved edits back to the list. The settings page header (`settings-page-subtitle`) is hidden while it is open.- Autocomplete dropdowns: clicking an input that already shows a value can open an empty listbox (the text filters it), and an option click then times out. The toggle-button opener does not always open the list either: on the Edit editor's `agent-profile-llm-selector` (bottom of the form) the option click timed out 5 of 5 times right after opening the editor, and the model list's `Custom` option once detached mid-click after a preset change. `browser choose` avoids the input-filtering problem, but an option may still remain unstable immediately after opening an editor. Reload and reopen before retrying; the suffix-identity check can use the already-selected active LLM without changing that dropdown. A listbox also closes if you spend a command (snapshot, eval) in between, so click the option right after opening.- A taken name marks the field invalid with an alert reading `An agent profile with this name already exists. …`.-- `has-text=default` also matches the active row's `Default` badge (Playwright's text match ignores case), so with several profiles `'testid=agent-profile-row >> has-text=default'` hits two rows; select the seeded profile with `'testid=agent-profile-row >> nth=-1'` (it sorts last) or by its badge.+- `has-text=default` matches any row whose LLM profile or name contains `default` (Playwright's text match ignores case); select the seeded profile with `'[data-testid=agent-profile-row]:has([title="default"])'`.- Model lookups (`F13.acp-model-discovery`) spawn the provider CLI on the agent-server machine through npx with the profile's allowed secrets; they send no prompt. A failed lookup is retried each time the preset is picked again; a successful one is cached for about five minutes per preset and secret set.- Switch rows are `sr-only` checkboxes inside labels (keyboard reachable): click the label text (`'testid=agent-settings-tool-list >> text=<tool>'`, `'testid=agent-settings-mcp-list >> text=<name>'`) and read `.checked` with `browser eval`. A bare `label` selector segment is parsed as `label=`, not CSS.-- The Default badge, the **Set as active** item and the `Switched to profile` toast all refer to the same active pointer; the wording differs from the LLM page (**Set as default**).+- The Active badge, the **Set as active** item and the `Switched to profile` toast all refer to the same active pointer; the LLM page says **Set as default**.- The OpenHands row's grey text is the LLM profile name, not the kind. A fresh run's `default` shows `default` there although no LLM profile of that name exists until it is edited (`F13.stale-llm-ref`).-- Local seeded OpenHands `default` launches through global agent settings so the Skills selection applies; its per-profile fields do not apply on that path. Named profiles use their saved settings. Home launches follow the LLM pill. On a backend advertising per-launch LLM override, a named profile pinned to another LLM keeps its tool, concurrency and MCP settings; read `launched_agent_profile` and `agent` on the conversation (`F13.llm-pill-precedence`). The same override covers a named profile whose LLM reference no longer exists. Only older backends without that capability use the legacy settings fallback in those two cases. `/new` and local child conversations launch through the same path.+- Every local launch sends `agent_profile_id`, `default` included; read `launched_agent_profile` and `agent` on the conversation. A named profile runs its own LLM profile (`launched_agent_profile.llm_profile_ref` `null`); `default`, or a profile whose LLM reference no longer exists, runs the active LLM profile (`llm_profile_ref` is set to it when it differs from the stored reference, otherwise `null`). The Home LLM pill edits the active profile's LLM (`F13.llm-pill-precedence`). `/new` and local child conversations launch through the same path. With no active profile a local launch fails with an error instead of falling back to global settings.+- On a local backend `default` shares state with global pages both ways: switching a server in its MCP list switches it in MCP Servers for every profile, its skill switches are the Skills page's, and picking its LLM activates that LLM profile. Arranged API writes (`llm preset`) do not update `default`'s stored `llm_profile_ref`; the editor preselects the active LLM profile and enables Save (`F13.stale-llm-ref`).+- An unquoted `text=github` click in the Skills list can hit another row containing that text; quote the exact name (`text="github"`) or search first.- The ACP auth probe runs the provider CLI on the agent-server machine. If that machine has a Claude CLI login, Claude Code can show "already signed in". Use only an explicitly approved account for authenticated ACP checks.- ACP credentials are global secrets, not profile fields: cancelling the editor saves nothing, but saving any ACP profile with a typed credential writes it to Settings → Secrets (F14) for every conversation.- **Choose secrets** on an ACP profile also selects provider secret names that are not saved (`CODEX_AUTH_JSON`, `OPENAI_BASE_URL`); they show as switches without descriptions.-- The empty state (`agent-profiles-empty`) and the load-error state (`agent-profiles-load-error`) are not reachable on a local backend: deleting the last profile re-seeds `default`, and with the agent-server stopped the app swaps to **Manage backends** after a reload, or keeps the cached list without one.+- The empty state (`agent-profiles-empty`) and the load-error state (`agent-profiles-load-error`) are not reachable on a local backend: `default` cannot be deleted, and with the agent-server stopped the app swaps to **Manage backends** after a reload, or keeps the cached list without one.- The refused-launch toast text comes from the Agent Server and changes between versions (`MCP server(s) not configured: <name>` on 1.53.0): assert that one toast names the server and the prompt stays, not the exact wording.- Opening a row menu focuses its first enabled item. Arrows skip disabled **Set as active**; Escape closes the menu and returns focus to its trigger, while Tab closes it and continues normal tab order.- The Add agent editor has its own agent description. Parallel tool calls label and description are translated.diff --git a/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md b/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.mdindex 1c3a70cc30..4de070cd6b 100644--- a/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md+++ b/.agents/skills/verify-openhands/references/feature-map/F15-agent-behavior-settings.md@@ -1,46 +1,55 @@# F15 — Condenser, agent context and verification settings-Three settings pages are generated from the Agent Server's settings schemas:+Settings pages generated from the Agent Server's settings schemas:**Condenser** (summarize long chats), **Agent Context** (persistent agentmemory) and **Verification** (confirmation mode, security analyzer and the-critic). Each page shows the schema's fields in Basic/Advanced/All tiers (only-the tiers that have fields), hides fields whose parent switch is off, and saves-through one **Save Changes** button that stays disabled until a value differs-from the stored one.--Source: `src/routes/condenser-settings.tsx`, `src/routes/agent-context-settings.tsx`, `src/routes/verification-settings.tsx`, `src/components/features/settings/sdk-settings/`, `src/utils/sdk-settings-schema.ts`.+critic). On a local backend each agent profile owns its condenser and critic:+they are edited in the Condenser and Verification sections of the agent+profile editor (Settings → Agent), `/settings/condenser` opens the Agent page,+and the Verification page keeps only the global Critic API Key, Confirmation+Mode and Security Analyzer under a note that the critic is set per agent+profile. On a Cloud backend the Condenser page and the full Verification page+(with the critic switch and its fields) are global settings pages. Each page+shows the schema's fields in Basic/Advanced/All tiers (only the tiers that have+fields), hides fields whose parent switch is off, and saves through one+**Save Changes** button that stays disabled until a value differs from the+stored one.++Source: `src/routes/condenser-settings.tsx`, `src/routes/agent-context-settings.tsx`, `src/routes/verification-settings.tsx`, `src/components/features/settings/sdk-settings/`, `src/utils/sdk-settings-schema.ts`, `src/components/features/settings/agent-profiles/profile-behavior-form.ts`.## Sub-features-- `F15.condenser-page`: Condenser shows its subtitle, Basic and All tabs (no Advanced), and in Basic only the **Enable Memory Condensation** switch.-- `F15.condenser-all-view`: the All tab lists the eight condenser fields with Title Case labels, plain descriptions and their stored values (Max Size `240`, Keep First Events `2`, …).-- `F15.condenser-save`: changing Max Size and saving shows **Saving...**, then the saved toast; after a reload the page opens on All with the new value.-- `F15.condenser-dependents`: turning condensation off hides the six size/reset fields (Condenser Kind stays); turning it back on un-dirties Save.-- `F15.condenser-kind`: Condenser Kind is a dropdown (**LLM Summarizer**, **None**, stored as `llm_summarizing` and `no_op`); the choice persists, and switching back restores the summarizer defaults.-- `F15.save-dirty-state`: Save is disabled on load, enabled after an edit, and disabled again when the edit is reverted by hand.-- `F15.save-errors`: a non-integer Max Size is refused in the browser with a toast; a negative one is refused by the server with a 422 toast; nothing is stored.-- `F15.view-scoped-save`: saving from a narrower tier keeps the stored values of fields that tier does not show.+- `F15.condenser-page`: on a local backend `/settings/condenser` replaces itself with `/settings/agents` and no nav item or command leads to it; on a Cloud backend Condenser shows its subtitle, Basic and All tabs (no Advanced), and in Basic only the **Enable Memory Condensation** switch.+- `F15.condenser-all-view`: (Cloud backend) the All tab lists the eight condenser fields with Title Case labels, plain descriptions and their stored values (Max Size `240`, Keep First Events `2`, …).+- `F15.condenser-save`: (Cloud backend) changing Max Size and saving shows **Saving...**, then the saved toast; after a reload the page opens on All with the new value.+- `F15.condenser-dependents`: (Cloud backend) turning condensation off hides the six size/reset fields (Condenser Kind stays); turning it back on un-dirties Save.+- `F15.condenser-kind`: (Cloud backend) Condenser Kind is a dropdown (**LLM Summarizer**, **None**, stored as `llm_summarizing` and `no_op`); the choice persists, and switching back restores the summarizer defaults.+- `F15.save-dirty-state`: on each page Save is disabled on load, enabled after an edit, and disabled again when the edit is reverted by hand.+- `F15.save-errors`: (Cloud backend) a non-integer Max Size is refused in the browser with a toast; a negative one is refused by the server with a 422 toast; nothing is stored.+- `F15.view-scoped-save`: (Cloud backend) saving from a narrower tier keeps the stored values of fields that tier does not show.- `F15.agent-context-page`: Agent Context shows a single **Persistent Agent Memory** switch with its help text and no tab bar.- `F15.agent-context-save`: toggling Persistent Agent Memory and saving persists after reload.-- `F15.verification-page`: Verification shows Basic, Advanced and All tabs; with defaults every tier shows only **Confirmation Mode** and **Enable Critic**.-- `F15.verification-critic-fields`: Confirmation Mode reveals Security Analyzer (Advanced); Enable Critic reveals Iterative Refinement and Critic API Key (Basic) plus mode, server URL and model (All); Iterative Refinement reveals threshold and max iterations.+- `F15.verification-page`: Verification shows Basic, Advanced and All tabs. On a local backend it starts with the note `Each agent profile turns the critic on or off in Settings → Agent.` and every tier shows only **Critic API Key** (with its OpenHands LLM Key help link) and **Confirmation Mode**; on a Cloud backend every tier shows only **Confirmation Mode** and **Enable Critic** by default.+- `F15.verification-critic-fields`: Confirmation Mode reveals Security Analyzer (Advanced and All). Enable Critic reveals Iterative Refinement (Basic) plus mode, server URL and model (All), and Iterative Refinement reveals threshold and max iterations: locally in an agent profile's Verification section (without the Critic API Key), on a Cloud backend on the page, where Enable Critic also reveals the Critic API Key.- `F15.confirmation-mode-save`: Confirmation Mode and Security Analyzer (conversation settings) persist after reload, alongside the agent-settings fields on the same page.- `F15.confirmation-mode-effect`: with Confirmation Mode on and Security Analyzer **None**, a new conversation stops before its first command with **Do you want to continue with this action?** and Cancel/Continue; Continue runs it.-- `F15.critic-tuning`: Critic Mode is a dropdown (`Finish and Message`, `All Actions`); the mode, Critic Threshold and Max Refinement Iterations persist after reload, and an out-of-range threshold (`1.5`) is refused with the 422 toast.-- `F15.critic-save`: critic text fields persist after reload; the Critic API Key comes back masked; clearing the fields and turning the critic off restores the defaults.+- `F15.critic-tuning`: (Cloud backend page; locally the same fields in an agent profile's Verification section) Critic Mode is a dropdown (`Finish and Message`, `All Actions`); the mode, Critic Threshold and Max Refinement Iterations persist after reload, and an out-of-range threshold (`1.5`) is refused with the 422 toast.+- `F15.critic-save`: the Critic API Key persists after reload and comes back masked, and clearing it stores `null`; on a Cloud backend the critic text fields on the page persist too, and clearing them and turning the critic off restores the defaults.- `F15.agent-context-effect`: with Persistent Agent Memory on, a new conversation's system prompt carries the `.openhands/memory/` instructions; with it off, it does not.-- `F15.tier-after-save`: after a save the tab bar re-derives the tier from the stored values without a reload: saving the last minor field back to its default moves the page from All to Basic.+- `F15.tier-after-save`: after a save the tab bar re-derives the tier from the stored values without a reload: saving the last non-Basic field back to its default moves the page to Basic.+- `F15.profile-behavior`: on a local backend the agent profile editor (Settings → Agent, Add or Edit) has Condenser (Basic/All) and Verification (Basic/All) sections rendered from the same schema; they edit that profile's `condenser` and `verification`, never the global settings, offer no Critic API Key, and a new profile starts from the `default` profile's values; reopening a profile always shows its stored values. Known failure (reproduced 2026-10-22, [#18810](https://github.com/OpenHands/OpenHands/issues/18810)): after a reload, Edit intermittently shows the global values instead, and a condenser save from that state overwrites the profile's.- `F15.unsaved-navigation`: leaving a page with unsaved edits shows no prompt and discards them.-- `F15.command-menu`: the command menu offers **Condenser settings**, **Agent Context** and **Verification settings**, and each opens its page.-- `F15.direct-url`: each page renders when opened by URL.+- `F15.command-menu`: the command menu offers **Agent Context** and **Verification settings** (plus **Condenser settings** on a Cloud backend), and each opens its page.+- `F15.direct-url`: each page renders when opened by URL; locally `/settings/condenser` opens `/settings/agents` instead.- `F15.phone`: at 390 px the fields stack in one column without horizontal overflow.- `F15.schema-unavailable`: a backend without the settings-schema endpoints shows `SDK settings schema unavailable.` with an upgrade hint instead of the form (blocked: needs such a backend).## How to get to it (user POV)-- Sidebar settings gear (`backend-selector-settings-link`, lands on `/settings/agents`), then **Condenser**, **Agent Context** or **Verification** in the settings navigation (`sidebar-settings-/settings/condenser`, `sidebar-settings-/settings/agent-context`, `sidebar-settings-/settings/verification`).-- Direct URLs `/settings/condenser`, `/settings/agent-context`, `/settings/verification`.-- Command menu (`Control+k` or `Meta+k`, or `command-menu-trigger` in the sidebar): **Condenser settings**, **Agent Context** and **Verification settings**; "memory" finds both Condenser settings and Agent Context.+- Sidebar settings gear (`backend-selector-settings-link`, lands on `/settings/agents`), then **Agent Context** or **Verification** in the settings navigation (`sidebar-settings-/settings/agent-context`, `sidebar-settings-/settings/verification`); **Condenser** (`sidebar-settings-/settings/condenser`) is listed only on a Cloud backend.+- Direct URLs `/settings/condenser` (local: opens `/settings/agents`), `/settings/agent-context`, `/settings/verification`.+- Command menu (`Control+k` or `Meta+k`, or `command-menu-trigger` in the sidebar): **Agent Context** and **Verification settings** (plus **Condenser settings** on a Cloud backend); locally "memory" finds only Agent Context.+- Per-profile condenser and critic (local): Settings → Agent, **Add agent profile** (`add-agent-profile`) or a row's menu → **Edit**, sections `agent-settings-condenser` and `agent-settings-verification`.- The effects show up in new conversations only: the confirmation prompt in the chat, and the memory instructions in the conversation's system prompt.## Driving it with control-openhands@@ -49,10 +58,13 @@ Preconditions:- Baseline state (launched, doctored, `onboard --skip` done). No LLM is needed for the settings pages.- `F15.confirmation-mode-effect` and `F15.agent-context-effect` need an active LLM profile (`control-openhands llm preset deepseek`).+- The condenser rows marked "(Cloud backend)", `F15.critic-tuning`'s page recipe and the Cloud halves of the Verification rows need an OpenHands Cloud backend and account (blocked in a local run); a local run proves the redirect, the local Verification page and `F15.profile-behavior`.+- `F15.profile-behavior` needs an LLM profile (the agent profile editor refuses to save without one): `control-openhands llm preset deepseek` (no model call).- `F15.schema-unavailable` needs an agent server that serves `/api/settings` but answers 404 for `/api/settings/agent-schema` (an older release via `launch --new --sdk-ref <old>`); not available in this checkout's pinned stack.- Stored values are the defaults: `control-openhands api GET /api/settings` shows `agent_settings.condenser.enabled` `true`, `max_size` `240`, `condenser_kind` `llm_summarizing`; `agent_context.load_memory` `false`; `verification.critic_enabled` `false`; `conversation_settings.confirmation_mode` `false`.-- **Open Condenser from the sidebar (`F15.condenser-page`).** Run `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings/agents(\?|$)'`, `control-openhands browser click 'testid=sidebar-settings-/settings/condenser' --expect-url '/settings/condenser(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Summarize long chats to stay within context limits.`) and `control-openhands browser snapshot 'testid=condenser-settings-screen'`. The snapshot shows tabs `Basic` [selected] and `All`, the text `Enable Memory Condensation` and `button "Save Changes" [disabled]`; `control-openhands browser count 'testid=sdk-section-advanced-toggle'` is `0`. Read the switch with `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-condenser.enabled\"]').checked"` (`true`).+- **Condenser on a local backend (`F15.condenser-page`, `F15.direct-url`).** Run `control-openhands browser goto /`, `control-openhands browser goto /settings/condenser` (the result already reads `.../settings/agents`), `control-openhands browser count 'testid=condenser-settings-screen'` (`0`), `control-openhands browser text 'testid=settings-screen >> header'` (`Agent` with `Create and manage reusable agent setups.`) and `control-openhands browser screenshot --feature F15.condenser-page --name local-redirect-agents`; `control-openhands browser back` returns to `/` (the redirect replaced history). `control-openhands browser count 'testid=sidebar-settings-/settings/condenser'` is `0`. The Cloud bullets below run only on a Cloud backend.+- **Open Condenser from the sidebar (`F15.condenser-page`, Cloud backend).** Run `control-openhands browser click 'testid=backend-selector-settings-link' --expect-url '/settings/agents(\?|$)'`, `control-openhands browser click 'testid=sidebar-settings-/settings/condenser' --expect-url '/settings/condenser(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Summarize long chats to stay within context limits.`) and `control-openhands browser snapshot 'testid=condenser-settings-screen'`. The snapshot shows tabs `Basic` [selected] and `All`, the text `Enable Memory Condensation` and `button "Save Changes" [disabled]`; `control-openhands browser count 'testid=sdk-section-advanced-toggle'` is `0`. Read the switch with `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-condenser.enabled\"]').checked"` (`true`).- **All tier (`F15.condenser-all-view`).** Run `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser count 'testid=condenser-settings-screen >> input'` (`8`) and `control-openhands browser value 'testid=sdk-settings-condenser.max_size'` (`240`). `control-openhands browser snapshot 'testid=condenser-settings-screen'` labels the fields `Enable Memory Condensation`, `Max Size`, `Condenser Kind` (combobox value `LLM Summarizer`), `Max Tokens`, `Keep First Events`, `Minimum Progress`, `Hard Reset Retries` and `Hard Reset Scaling`, each with a plain-language paragraph and all but the switch marked `Optional` (Condenser Kind: `How the condenser shortens long conversations. LLM Summarizer replaces older events with a summary written by the LLM; None keeps the full history.`).- **Dirty state (`F15.save-dirty-state`).** On the All tier: `control-openhands browser enabled 'testid=save-button'` is `false`; `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 120` makes it `true`; `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 240` makes it `false` again.- **Save and reload (`F15.condenser-save`).** Fill Max Size with `120`, then `control-openhands browser click 'testid=save-button' --observe 'testid=save-button' --observe-ms 3000` (observed states `Save Changes` → `Saving...` → `Save Changes`) and `control-openhands browser toasts` (`Settings saved. For old conversations, you will need to stop and restart the conversation to see the changes.`). `control-openhands browser enabled 'testid=save-button'` is `false`. Run `control-openhands browser reload`, `control-openhands browser snapshot 'role=tablist'` (`All` [selected]: the page opens on the tier that shows the overridden field) and `control-openhands browser value 'testid=sdk-settings-condenser.max_size'` (`120`). `control-openhands api GET /api/settings` shows `max_size` `120`. Keep it for the next bullet.@@ -62,31 +74,35 @@ Preconditions:- **Condenser Kind (`F15.condenser-kind`).** On the All tier run `control-openhands browser click 'testid=sdk-settings-condenser.condenser_kind'`, `control-openhands browser snapshot 'role=listbox'` (options `LLM Summarizer` [selected] and `None`), `control-openhands browser click 'role=listbox >> role=option[name="None"]'` and `control-openhands browser click 'testid=save-button'`. After `browser reload` the page opens on All, `browser value 'testid=sdk-settings-condenser.condenser_kind'` is `None`, and `control-openhands api GET /api/settings --pick agent_settings.condenser` is just `{"enabled": true, "condenser_kind": "no_op"}` (the labels are Canvas copy; the stored values are unchanged). The summarizer fields should hide while **None** is selected. Known failure (reproduced 2026-10-06): Max Size, Max Tokens, Keep First Events, Minimum Progress, Hard Reset Retries and Hard Reset Scaling stay with their defaults, and `browser fill 'testid=sdk-settings-condenser.max_size' 100` then Save gives `Settings saved` while the API still shows only `enabled` and `condenser_kind` and a reload shows `240` again (#18049; `browser screenshot --feature F15.condenser-kind --name none-shows-summarizer-fields`). Restore by choosing `LLM Summarizer` the same way and saving: after reload the page opens on Basic and the API shows all eight defaults again (`condenser_kind` `llm_summarizing`).- **Agent Context (`F15.agent-context-page`).** Run `control-openhands browser click 'testid=sidebar-settings-/settings/agent-context' --expect-url '/settings/agent-context(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Let the agent keep notes and recall them in new conversations.`), `control-openhands browser count 'role=tablist'` (`0`) and `control-openhands browser snapshot 'testid=agent-context-settings-screen'` (text `Persistent Agent Memory`, the paragraph about `.openhands/memory/`, `button "Save Changes" [disabled]`).- **Persistent memory persists (`F15.agent-context-save`).** Run `control-openhands browser click 'testid=agent-context-settings-screen >> text=Persistent Agent Memory'`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser reload` and `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-agent_context.load_memory\"]').checked"` (`true`; API `agent_context.load_memory` `true`). Restore: click the label, save, reload; the value is `false`.-- **Verification (`F15.verification-page`).** Run `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Confirmation prompts and security checks on actions.`) and `control-openhands browser snapshot 'testid=verification-settings-screen'`: tabs `Basic` [selected], `Advanced`, `All`; texts `Confirmation Mode` and `Enable Critic`. `control-openhands browser count 'testid=verification-settings-screen >> input'` is `2` on Advanced and on All too.-- **Critic and analyzer fields (`F15.verification-critic-fields`).** On All run `control-openhands browser click 'testid=verification-settings-screen >> text=Confirmation Mode'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Critic"'` and `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Iterative Refinement"'`. `control-openhands browser testids 'testid=verification-settings-screen' --hidden` now lists `sdk-settings-security_analyzer`, `sdk-settings-verification.critic_mode`, `sdk-settings-verification.critic_threshold` (`0.6`), `sdk-settings-verification.max_refinement_iterations` (`3`), `sdk-settings-verification.critic_server_url`, `sdk-settings-verification.critic_model_name`, `sdk-settings-verification.critic_api_key` and `help-link-verification.critic_api_key`. `control-openhands browser attr 'testid=sdk-settings-verification.critic_api_key' type` is `password`; `control-openhands browser eval "document.querySelector('[data-testid=\"help-link-verification.critic_api_key\"] a')?.href"` is `https://app.all-hands.dev/settings/api-keys`. `control-openhands browser screenshot --feature F15.verification-critic-fields --name all-critic-on` shows the API key spanning both columns. Basic (`sdk-section-basic-toggle`) keeps `confirmation_mode`, `critic_enabled`, `enable_iterative_refinement` and `critic_api_key`; Advanced adds `security_analyzer`. The three switches are an unsaved draft that survives tab switches: run `control-openhands browser reload` before the next bullet, otherwise its Confirmation Mode click turns the switch off again and the save stores the critic draft.+- **Verification (`F15.verification-page`).** Run `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, `control-openhands browser text 'testid=settings-page-subtitle'` (`Confirmation prompts and security checks on actions.`), `control-openhands browser text 'testid=verification-critic-per-profile-note'` (`Each agent profile turns the critic on or off in Settings → Agent.`) and `control-openhands browser snapshot 'testid=verification-settings-screen'`: tabs `Basic` [selected], `Advanced`, `All`; `textbox "Critic API Key Optional"` with its help paragraph and `link "OpenHands LLM Key"` (`https://app.all-hands.dev/settings/api-keys`), `checkbox "Confirmation Mode"` and `button "Save Changes" [disabled]`; Advanced and All show the same (`control-openhands browser screenshot --feature F15.verification-page --name local-basic`). On a Cloud backend there is no note and the tiers show `Confirmation Mode` and `Enable Critic`.+- **Analyzer field (`F15.verification-critic-fields`, local).** On `/settings/verification` click `testid=verification-settings-screen >> text=Confirmation Mode` (unsaved); `control-openhands browser count 'testid=sdk-settings-security_analyzer'` is `0` on Basic and `1` on Advanced and on All (click `sdk-section-<tier>-toggle` before each count). `browser reload` drops the draft. The critic's own fields are in each agent profile's Verification section (`F15.profile-behavior`).+- **Critic and analyzer fields (`F15.verification-critic-fields`, Cloud backend).** On All run `control-openhands browser click 'testid=verification-settings-screen >> text=Confirmation Mode'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Critic"'` and `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Iterative Refinement"'`. `control-openhands browser testids 'testid=verification-settings-screen' --hidden` now lists `sdk-settings-security_analyzer`, `sdk-settings-verification.critic_mode`, `sdk-settings-verification.critic_threshold` (`0.6`), `sdk-settings-verification.max_refinement_iterations` (`3`), `sdk-settings-verification.critic_server_url`, `sdk-settings-verification.critic_model_name`, `sdk-settings-verification.critic_api_key` and `help-link-verification.critic_api_key`. `control-openhands browser attr 'testid=sdk-settings-verification.critic_api_key' type` is `password`; `control-openhands browser eval "document.querySelector('[data-testid=\"help-link-verification.critic_api_key\"] a')?.href"` is `https://app.all-hands.dev/settings/api-keys`. `control-openhands browser screenshot --feature F15.verification-critic-fields --name all-critic-on` shows the API key spanning both columns. Basic (`sdk-section-basic-toggle`) keeps `confirmation_mode`, `critic_enabled`, `enable_iterative_refinement` and `critic_api_key`; Advanced adds `security_analyzer`. The three switches are an unsaved draft that survives tab switches: run `control-openhands browser reload` before the next bullet, otherwise its Confirmation Mode click turns the switch off again and the save stores the critic draft.- **Confirmation mode persists (`F15.confirmation-mode-save`).** On `/settings/verification` (Basic) run `control-openhands browser click 'testid=verification-settings-screen >> text=Confirmation Mode'`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser reload` and `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-confirmation_mode\"]').checked"` (`true`). Then `control-openhands browser click 'testid=sdk-section-advanced-toggle'`, `control-openhands browser click 'testid=sdk-settings-security_analyzer'`, `control-openhands browser snapshot 'role=listbox'` (options `LLM` [selected], `None`), `control-openhands browser click 'role=listbox >> role=option[name="None"]'`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser reload`: the tab bar shows `Advanced` [selected] and `control-openhands browser value 'testid=sdk-settings-security_analyzer'` is `None`. `control-openhands api GET /api/settings` shows `conversation_settings.confirmation_mode` `true`, `security_analyzer` `none`. Keep both for the next bullet.- **Confirmation stops the agent (`F15.confirmation-mode-effect`).** Run `control-openhands conversation start --prompt "Run this shell command: echo qa-f15-confirm" --wait --until waiting_for_confirmation,finished,idle,error --timeout 180`; the status is `waiting_for_confirmation`. `control-openhands browser wait-text 'Do you want to continue with this action?' --timeout 15000` succeeds and `control-openhands browser screenshot --feature F15.confirmation-mode-effect --name awaiting` shows **Cancel ⇧⌘⌫** and **Continue ⌘↩** under the pending command. Run `control-openhands browser click 'testid=action-confirm-button'`, `control-openhands conversation wait <id> --until finished,idle,error --timeout 120` (`<id>` from the start output) and `control-openhands conversation events <id> --kinds ObservationEvent`: the terminal observation is `qa-f15-confirm`. If the status is still `waiting_for_confirmation` the agent asked again (see Gotchas): confirm again. Restore on `/settings/verification` (`control-openhands browser goto /settings/verification`; the browser is on the conversation): `browser click 'testid=sdk-section-basic-toggle'`, click the Confirmation Mode label, `browser click 'testid=save-button'`; the Basic save also resets Security Analyzer to `llm` (API `confirmation_mode` `false`, `security_analyzer` `llm`).-- **Critic values persist (`F15.critic-save`).** On the All tier click `testid=verification-settings-screen >> text="Enable Critic"`, run `control-openhands browser fill 'testid=sdk-settings-verification.critic_model_name' qa-critic-model`, `control-openhands browser fill 'testid=sdk-settings-verification.critic_api_key' qa-dummy-critic-key`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser reload`. The page opens on `All`; `browser value 'testid=sdk-settings-verification.critic_model_name'` is `qa-critic-model` and `browser value 'testid=sdk-settings-verification.critic_api_key'` reports `masked: true` and `length: 10` for the stored placeholder. The CLI redacts its displayed value; do not compare its returned star string to the input's literal placeholder or bypass masking. Restore on All: fill both fields with `''`, click `text="Enable Critic"` again, save, reload: the page opens on Basic and the API shows `critic_enabled` `false`, `critic_model_name` and `critic_api_key` `null`.-- **Critic mode, threshold and iterations (`F15.critic-tuning`).** On `/settings/verification` run `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Critic"'`, `control-openhands browser click 'testid=sdk-settings-verification.critic_mode'` and `control-openhands browser snapshot 'role=listbox'` (options `Finish and Message` [selected], `All Actions`). Run `control-openhands browser click 'role=listbox >> role=option[name="All Actions"]'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Iterative Refinement"'`, `control-openhands browser fill 'testid=sdk-settings-verification.critic_threshold' 0.75`, `control-openhands browser fill 'testid=sdk-settings-verification.max_refinement_iterations' 5`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser reload`: the page opens on `All`, `browser value` reads `All Actions`, `0.75` and `5`, and `control-openhands api GET /api/settings --pick agent_settings.verification` shows `critic_mode` `all_actions`, `critic_threshold` `0.75`, `max_refinement_iterations` `5`. Then `browser fill 'testid=sdk-settings-verification.critic_threshold' 1.5`, `browser network --clear`, `browser click 'testid=save-button'`, `browser toasts` (`Settings validation failed`, one `PATCH /api/settings` with 422 in `browser network`); the API still shows `0.75`. Restore: `browser reload`, `browser click 'testid=sdk-section-basic-toggle'`, click `text="Enable Iterative Refinement"` and `text="Enable Critic"` off, `browser click 'testid=save-button'`: the Basic save resets mode, threshold and iterations, and the API shows the verification defaults.-- **Tier follows the stored values after a save (`F15.tier-after-save`).** On `/settings/condenser` (`control-openhands browser goto /settings/condenser`; the previous bullet is on `/settings/verification`) run `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser fill 'testid=sdk-settings-condenser.keep_first' 3`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser wait-text 'Settings saved' --timeout 5000` and `control-openhands browser snapshot 'role=tablist'` (`All` [selected]). Then `browser fill 'testid=sdk-settings-condenser.keep_first' 2`, `browser click 'testid=save-button'`, `control-openhands browser wait 'testid=sdk-settings-condenser.keep_first' --state detached --timeout 5000` (`detached`) and `browser snapshot 'role=tablist'`: `Basic` [selected] without any reload; the API shows `keep_first` `2` (restored).+- **Critic API Key persists (`F15.critic-save`, local).** On `/settings/verification` run `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser fill 'testid=sdk-settings-verification.critic_api_key' qa-dummy-critic-key`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser toasts` (`Settings saved. …`) and `control-openhands browser reload`: `control-openhands browser value 'testid=sdk-settings-verification.critic_api_key'` reports `masked: true` and `length: 10`, and `control-openhands api GET /api/settings --pick agent_settings.verification` shows `critic_api_key` `**********` with `critic_enabled` `false` (screenshot `--feature F15.critic-save --name key-masked`). Restore: click `testid=sdk-section-all-toggle`, fill the key with `''`, save; the API shows `critic_api_key` `null`.+- **Critic values persist (`F15.critic-save`, Cloud backend).** On the All tier click `testid=verification-settings-screen >> text="Enable Critic"`, run `control-openhands browser fill 'testid=sdk-settings-verification.critic_model_name' qa-critic-model`, `control-openhands browser fill 'testid=sdk-settings-verification.critic_api_key' qa-dummy-critic-key`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser reload`. The page opens on `All`; `browser value 'testid=sdk-settings-verification.critic_model_name'` is `qa-critic-model` and `browser value 'testid=sdk-settings-verification.critic_api_key'` reports `masked: true` and `length: 10` for the stored placeholder. The CLI redacts its displayed value; do not compare its returned star string to the input's literal placeholder or bypass masking. Restore on All: fill both fields with `''`, click `text="Enable Critic"` again, save, reload: the page opens on Basic and the API shows `critic_enabled` `false`, `critic_model_name` and `critic_api_key` `null`.+- **Critic mode, threshold and iterations (`F15.critic-tuning`, Cloud backend).** On `/settings/verification` run `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Critic"'`, `control-openhands browser click 'testid=sdk-settings-verification.critic_mode'` and `control-openhands browser snapshot 'role=listbox'` (options `Finish and Message` [selected], `All Actions`). Run `control-openhands browser click 'role=listbox >> role=option[name="All Actions"]'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Iterative Refinement"'`, `control-openhands browser fill 'testid=sdk-settings-verification.critic_threshold' 0.75`, `control-openhands browser fill 'testid=sdk-settings-verification.max_refinement_iterations' 5`, `control-openhands browser click 'testid=save-button'` and `control-openhands browser reload`: the page opens on `All`, `browser value` reads `All Actions`, `0.75` and `5`, and `control-openhands api GET /api/settings --pick agent_settings.verification` shows `critic_mode` `all_actions`, `critic_threshold` `0.75`, `max_refinement_iterations` `5`. Then `browser fill 'testid=sdk-settings-verification.critic_threshold' 1.5`, `browser network --clear`, `browser click 'testid=save-button'`, `browser toasts` (`Settings validation failed`, one `PATCH /api/settings` with 422 in `browser network`); the API still shows `0.75`. Restore: `browser reload`, `browser click 'testid=sdk-section-basic-toggle'`, click `text="Enable Iterative Refinement"` and `text="Enable Critic"` off, `browser click 'testid=save-button'`: the Basic save resets mode, threshold and iterations, and the API shows the verification defaults.+- **Tier follows the stored values after a save (`F15.tier-after-save`).** On `/settings/verification` (`control-openhands browser goto /settings/verification`) click `testid=verification-settings-screen >> text=Confirmation Mode`, run `control-openhands browser click 'testid=sdk-section-advanced-toggle'`, `control-openhands browser click 'testid=sdk-settings-security_analyzer'`, `control-openhands browser click 'role=listbox >> role=option[name="None"]'`, `control-openhands browser click 'testid=save-button'`, `control-openhands browser wait-text 'Settings saved' --timeout 5000` and `control-openhands browser snapshot 'role=tablist'` (`Advanced` [selected]). Then `browser click 'testid=sdk-section-advanced-toggle'`, pick `role=option[name="LLM"]` the same way, `browser click 'testid=save-button'`, `control-openhands browser wait 'testid=sdk-settings-security_analyzer' --state detached --timeout 5000` (`detached`) and `browser snapshot 'role=tablist'`: `Basic` [selected] without any reload (`browser screenshot --feature F15.tier-after-save --name back-to-basic`). Restore: click the Confirmation Mode label and save; the API shows `confirmation_mode` `false`, `security_analyzer` `llm`.- **Memory reaches the agent (`F15.agent-context-effect`).** Run `control-openhands browser goto /settings/agent-context`, turn Persistent Agent Memory on and save as in `F15.agent-context-save`, then run `control-openhands conversation start --prompt "Reply with only: ok" --wait --timeout 180` and `control-openhands api GET '/api/conversations/<id>/events/search?limit=1'`. The first event is the `SystemPromptEvent`; its text contains `Project memory: ` followed by `` `.openhands/memory/` under the workspace root``. The same call for a conversation started with memory off (for example the one from `F15.confirmation-mode-effect`) has no `.openhands/memory`. Restore memory off through the page (`control-openhands browser goto /settings/agent-context` first: `conversation start` left the browser on the conversation).-- **Leaving discards edits (`F15.unsaved-navigation`).** Arrange an unsaved draft: `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, `control-openhands browser click 'testid=sdk-section-all-toggle'`, then click `testid=verification-settings-screen >> text=Confirmation Mode`, `>> text="Enable Critic"` and `>> text="Enable Iterative Refinement"` (do not save); `browser enabled 'testid=save-button'` is `true`. Run `control-openhands browser click 'testid=sidebar-settings-/settings/condenser' --expect-url '/settings/condenser(\?|$)'`, `control-openhands browser dialogs` (none), `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, then `control-openhands browser eval "[document.querySelector('[data-testid=\"sdk-settings-confirmation_mode\"]').checked, document.querySelector('[data-testid=\"sdk-settings-verification.critic_enabled\"]').checked]"` (`[false, false]`) and `browser enabled 'testid=save-button'` (`false`).+- **Dirty state on Verification (`F15.save-dirty-state`).** On `/settings/verification` `control-openhands browser enabled 'testid=save-button'` is `false`; clicking `testid=verification-settings-screen >> text=Confirmation Mode` makes it `true`, and clicking the label again makes it `false`.+- **Leaving discards edits (`F15.unsaved-navigation`).** Arrange an unsaved draft: `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, click `testid=verification-settings-screen >> text=Confirmation Mode` and `control-openhands browser fill 'testid=sdk-settings-verification.critic_api_key' qa-draft` (do not save); `browser enabled 'testid=save-button'` is `true`. Run `control-openhands browser click 'testid=sidebar-settings-/settings/agent-context' --expect-url '/settings/agent-context(\?|$)'`, `control-openhands browser dialogs` (none), `control-openhands browser click 'testid=sidebar-settings-/settings/verification' --expect-url '/settings/verification(\?|$)'`, then `control-openhands browser eval "document.querySelector('[data-testid=\"sdk-settings-confirmation_mode\"]').checked"` (`false`), `control-openhands browser value 'testid=sdk-settings-verification.critic_api_key'` (empty, `length` `0`) and `browser enabled 'testid=save-button'` (`false`).+- **Per-profile condenser and critic (`F15.profile-behavior`).** After `llm preset deepseek`, run `control-openhands browser goto /settings/agents`, `control-openhands browser click 'testid=add-agent-profile'`, `control-openhands browser fill 'testid=agent-profile-name-input' QA_f15`, `control-openhands browser click 'testid=agent-settings-condenser >> testid=sdk-section-all-toggle'` (`browser count 'testid=agent-settings-condenser >> input'` is `8`; `browser value 'testid=sdk-settings-condenser.max_size'` is `240`, the `default` profile's value), `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 120`, `control-openhands browser click 'testid=agent-settings-verification >> text="Enable Critic"'` and `control-openhands browser click 'testid=agent-settings-verification >> testid=sdk-section-all-toggle'`: `control-openhands browser testids 'testid=agent-settings-verification' --hidden` lists `sdk-settings-verification.critic_enabled`, `.critic_mode`, `.enable_iterative_refinement`, `.critic_server_url` and `.critic_model_name`, and no `critic_api_key` (`browser screenshot --feature F15.profile-behavior --name editor-filled`). `control-openhands browser click 'testid=save-agent-profile-btn'` toasts `Profile "QA_f15" created`; `control-openhands api GET /api/agent-profiles/QA_f15` has `profile.condenser.max_size` `120`, `profile.verification.critic_enabled` `true` and no critic key, while `control-openhands api GET /api/settings --pick agent_settings.condenser.max_size` stays `240` and `--pick agent_settings.verification.critic_enabled` stays `false`. Reopen it (`browser reload`, `browser click '[data-testid=agent-profile-row]:has-text("QA_f15") >> testid=agent-profile-menu-trigger'`, `browser click 'role=menu >> role=menuitem[name="Edit"]'`): the Condenser section opens on All with `max_size` `120`, the critic switch is checked and `browser enabled 'testid=save-agent-profile-btn'` is `false`, on every reopen. Known failure (reproduced 2026-10-22, [#18810](https://github.com/OpenHands/OpenHands/issues/18810)): in about one reopen in six after a `browser reload` (repeat the reload and Edit about 10 times), the section shows the global `240` and the critic off while `api GET /api/agent-profiles/QA_f15` still has `120`/`true`; saving a condenser edit from that state stores `max_size` `240`. The `default` profile's editor shows both sections too (`browser count 'testid=agent-settings-condenser'` is `1`). In the section, choosing Condenser Kind **None** should hide the summarizer fields and Max Size should refuse values below zero; Known failure (reproduced 2026-10-22, [#18049](https://github.com/OpenHands/OpenHands/issues/18049), [#15813](https://github.com/OpenHands/OpenHands/issues/15813)): with **None** picked `browser count 'testid=sdk-settings-condenser.max_size'` is still `1`, and `browser fill 'testid=sdk-settings-condenser.max_size' -5` keeps `-5` (`browser screenshot --feature F15.profile-behavior --name kind-none-fields`). Leave with `testid=cancel-agent-profile-btn` and delete `QA_f15` (`control-openhands api DELETE /api/agent-profiles/QA_f15 --write`; deleting through the UI is F13).-- **Command menu (`F15.command-menu`).** Run `control-openhands browser goto /`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' Condenser`, `control-openhands browser snapshot 'testid=command-menu'` (option `Condenser settings Tune context condensation behavior. Go` [selected]), `control-openhands browser press Enter` and `control-openhands browser wait-url '/settings/condenser(\?|$)'`. Repeat with `Verification` (option `Verification settings Adjust risk and confirmation controls. Go`, URL `/settings/verification`) and with `Agent Context` (option `Agent Context Let the agent keep notes and recall them in new conversations. Go` [selected], URL `/settings/agent-context`, subtitle `Let the agent keep notes and recall them in new conversations.`). Then `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' memory` and `control-openhands browser snapshot 'testid=command-menu'`: two options, `Condenser settings …` and `Agent Context …` (hovering selects an option, so click the one you want rather than pressing Enter); close with `control-openhands browser press Escape` (`browser count 'testid=command-menu'` is `0`).-- **Direct URLs (`F15.direct-url`).** Run `control-openhands browser goto /settings/condenser` and `control-openhands browser wait 'testid=condenser-settings-screen'`; likewise `/settings/agent-context` with `testid=agent-context-settings-screen` and `/settings/verification` with `testid=verification-settings-screen`. Each wait returns `visible`.-- **Phone (`F15.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/verification`, `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser click 'testid=verification-settings-screen >> text="Enable Critic"'` (unsaved), `control-openhands browser bbox 'testid=verification-settings-screen'` (width `348`, `pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F15.phone --name verification-all` (one column; the settings nav is replaced by a back chevron). Do the same for `/settings/condenser` All. Return with `control-openhands browser viewport desktop` and `browser reload` to drop the draft.+- **Command menu (`F15.command-menu`).** Run `control-openhands browser goto /`, `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' Condenser` and `control-openhands browser snapshot 'testid=command-menu'`: on a local backend `No commands found` (on a Cloud backend, option `Condenser settings Tune context condensation behavior. Go`, which opens `/settings/condenser`); `control-openhands browser press Escape`. Repeat with `Verification` (option `Verification settings Adjust risk and confirmation controls. Go`, URL `/settings/verification`) and with `Agent Context` (option `Agent Context Let the agent keep notes and recall them in new conversations. Go` [selected], URL `/settings/agent-context`, subtitle `Let the agent keep notes and recall them in new conversations.`). Then `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' memory` and `control-openhands browser snapshot 'testid=command-menu'`: locally one option, `Agent Context …` (a Cloud backend also lists `Condenser settings …`; hovering selects an option, so click the one you want rather than pressing Enter); close with `control-openhands browser press Escape` (`browser count 'testid=command-menu'` is `0`).+- **Direct URLs (`F15.direct-url`).** Run `control-openhands browser goto /settings/agent-context` and `control-openhands browser wait 'testid=agent-context-settings-screen'`; likewise `/settings/verification` with `testid=verification-settings-screen`. Each wait returns `visible`.+- **Phone (`F15.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/verification`, `control-openhands browser click 'testid=sdk-section-all-toggle'`, `control-openhands browser bbox 'testid=verification-settings-screen'` (width `348`, `pageHorizontalOverflow` `false`) and `control-openhands browser screenshot --feature F15.phone --name verification-all-local` (one column; the settings nav is replaced by a back chevron). On a Cloud backend also click `testid=verification-settings-screen >> text="Enable Critic"` (unsaved) first, and do the same for `/settings/condenser` All. Return with `control-openhands browser viewport desktop` and `browser reload` to drop the draft.- **Schema unavailable (`F15.schema-unavailable`, blocked).** `control-openhands service stop agent-server` does not reach this state: after `browser reload` the whole app is replaced by the **Manage backends** dialog (`Disconnected`, 502). Bring the stack back with `control-openhands restart`. The state needs a backend whose schema endpoints 404 (see Preconditions); there `browser text 'testid=sdk-schema-unavailable'` should start with `SDK settings schema unavailable.`-- **Errors sweep.** `control-openhands browser errors --clear`, open the three pages and their All tabs, then `control-openhands browser errors --app-only`: no page errors and no warnings (the condenser fields are translated since #18035).+- **Errors sweep.** `control-openhands browser errors --clear`, open the pages and their All tabs, then `control-openhands browser errors --app-only`: no page errors and no warnings (the condenser fields are translated since #18035).-- **Background refetch keeps the draft (`F15.save-dirty-state`).** With Max Size stored as `240`, run `control-openhands browser goto /settings/condenser`, `control-openhands browser click 'testid=sdk-section-all-toggle'` and `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 111`. Arrange a changed server baseline with `control-openhands api PATCH /api/settings --write --data '{"agent_settings_diff":{"condenser":{"max_size":300}}}'`; this API write is a precondition, not the UI proof. Run `control-openhands browser clock --offset-ms 360000` (makes the mounted query stale), `control-openhands browser network-state offline`, `control-openhands browser network-state online` and `control-openhands browser network --filter '/api/settings'`. After the real GET refetch, the unsaved `111` should remain and Save should stay enabled. Known failure (reproduced 2026-10-12, [#16948](https://github.com/OpenHands/OpenHands/issues/16948)): `control-openhands browser value 'testid=sdk-settings-condenser.max_size'` is `300` and `control-openhands browser enabled 'testid=save-button'` is `false`; the draft vanished without Save, reset or a scope change. Capture `control-openhands browser screenshot --feature F15.save-dirty-state --name refetch-discards-draft`. Restore with `control-openhands browser clock --offset-ms 0`, fill Max Size `240`, click Save and confirm `control-openhands api GET /api/settings --pick agent_settings.condenser.max_size` is `240`; networking stays online.+- **Background refetch keeps the draft (`F15.save-dirty-state`, Cloud backend).** With Max Size stored as `240`, run `control-openhands browser goto /settings/condenser`, `control-openhands browser click 'testid=sdk-section-all-toggle'` and `control-openhands browser fill 'testid=sdk-settings-condenser.max_size' 111`. Arrange a changed server baseline with `control-openhands api PATCH /api/settings --write --data '{"agent_settings_diff":{"condenser":{"max_size":300}}}'`; this API write is a precondition, not the UI proof. Run `control-openhands browser clock --offset-ms 360000` (makes the mounted query stale), `control-openhands browser network-state offline`, `control-openhands browser network-state online` and `control-openhands browser network --filter '/api/settings'`. After the real GET refetch, the unsaved `111` should remain and Save should stay enabled. Known failure (reproduced 2026-10-12, [#16948](https://github.com/OpenHands/OpenHands/issues/16948)): `control-openhands browser value 'testid=sdk-settings-condenser.max_size'` is `300` and `control-openhands browser enabled 'testid=save-button'` is `false`; the draft vanished without Save, reset or a scope change. Capture `control-openhands browser screenshot --feature F15.save-dirty-state --name refetch-discards-draft`. Restore with `control-openhands browser clock --offset-ms 0`, fill Max Size `240`, click Save and confirm `control-openhands api GET /api/settings --pick agent_settings.condenser.max_size` is `240`; networking stays online.-- **Negative condenser input floor (`F15.save-errors`).** On `/settings/condenser` select All, fill `testid=sdk-settings-condenser.max_size` with `-5` and read `browser enabled 'testid=save-button'`. Expected by [#15813](https://github.com/OpenHands/OpenHands/issues/15813): the input prevents values below zero. Known failure, reproduced 2026-10-12: `-5` remains and Save is enabled. Save shows `Settings validation failed`, while `api GET /api/settings --pick agent_settings.condenser.max_size` remains `240`; repeat filling `-5` and read Save enabled, then fill `240` to discard the invalid draft. The server guard works; the requested client input floor does not.+- **Negative condenser input floor (`F15.save-errors`, Cloud backend; locally see `F15.profile-behavior`).** On `/settings/condenser` select All, fill `testid=sdk-settings-condenser.max_size` with `-5` and read `browser enabled 'testid=save-button'`. Expected by [#15813](https://github.com/OpenHands/OpenHands/issues/15813): the input prevents values below zero. Known failure, reproduced 2026-10-12: `-5` remains and Save is enabled. Save shows `Settings validation failed`, while `api GET /api/settings --pick agent_settings.condenser.max_size` remains `240`; repeat filling `-5` and read Save enabled, then fill `240` to discard the invalid draft. The server guard works; the requested client input floor does not.-- **Default Verification tabs (`F15.verification-page`, `F15.verification-critic-fields`).** [#15811](https://github.com/OpenHands/OpenHands/issues/15811) asks whether Basic and Advanced should differ. On defaults, snapshot `testid=verification-settings-screen` in each tab: both show only Confirmation Mode and Enable Critic. Enable Confirmation Mode without saving; `browser count 'testid=sdk-settings-security_analyzer'` is `1` on Advanced, `0` on Basic and `1` again on Advanced. Reload to discard, then repeat the default/conditional check. Verified 2026-10-12: the default sameness is the current conditional-field behavior, not evidence that the two tiers always render the same fields.+- **Default Verification tabs (`F15.verification-page`, `F15.verification-critic-fields`).** [#15811](https://github.com/OpenHands/OpenHands/issues/15811) asks whether Basic and Advanced should differ. On defaults, snapshot `testid=verification-settings-screen` in each tab: both show only Critic API Key and Confirmation Mode (on a Cloud backend, Confirmation Mode and Enable Critic). Enable Confirmation Mode without saving; `browser count 'testid=sdk-settings-security_analyzer'` is `1` on Advanced, `0` on Basic and `1` again on Advanced. Reload to discard, then repeat the default/conditional check. Verified 2026-10-12: the default sameness is the current conditional-field behavior, not evidence that the two tiers always render the same fields.- **Datetime refreshed on Canvas launch (`F15.agent-context-page`).** Before a new chat, read `api GET /api/settings --pick agent_settings.agent_context.current_datetime`; record this public timestamp so it can be restored. Arrange an old persisted-state timestamp through the real API (`api PATCH /api/settings --write --data '{"agent_settings_diff":{"agent_context":{"current_datetime":"2001-01-01T00:00:00+00:00"}}}'`). Start a chat through the Home composer with the dummy loopback profile, then inspect the run-owned runtime `agent.agent_context.current_datetime` and `conversation events <id> --kinds SystemPromptEvent --grep 2001-01-01 --from-start --last 1`. Expected: runtime datetime matches the new launch and the old-date event search returns zero. Restore the original settings timestamp through PATCH; no successful model reply is required to inspect initialization.-Regression check (`F15.save-errors`, negative-no-retry):+Regression check (`F15.save-errors`, negative-no-retry; Cloud backend only, since locally `/settings/condenser` opens the Agent page):```shtrap 'control-openhands browser reload >/dev/null' EXIT@@ -106,6 +122,7 @@ control-openhands assert --field total --equals 1 -- browser network --filter '/- Switches are `sr-only` checkboxes (keyboard reachable): toggle by clicking the label text scoped to the page, read `.checked` with `browser eval`.- `text=Enable Critic` is a case-insensitive substring match and hits the help paragraph `Enable critic evaluation for the agent.` first, so the click does nothing. Quote it: `text="Enable Critic"` (same for `text="Enable Iterative Refinement"`).- The conversation-owned fields have no `verification.` prefix: `sdk-settings-confirmation_mode`, `sdk-settings-security_analyzer`. Critic fields keep it (`sdk-settings-verification.critic_enabled`).+- Locally the condenser and the critic switch belong to agent profiles: a recipe that opens `/settings/condenser` lands on `/settings/agents`, and `sdk-settings-verification.critic_enabled` is not on the Verification page. Use the agent profile editor's sections (`F15.profile-behavior`), whose tab toggles repeat per section: scope them, `'testid=agent-settings-condenser >> testid=sdk-section-all-toggle'`.- Saving is tier-scoped: Basic or Advanced saves reset every field the tier hides to its schema default (`F15.view-scoped-save`). Make all edits on All when other non-default values must survive.- The page opens on the least detailed tier that shows every stored override (any minor override → All). Do not assume Basic after a reload.- Condenser Kind **None** (`no_op`) still shows the summarizer fields with defaults, although the stored condenser no longer has them, and edits to them are accepted (`Settings saved`) but dropped while **None** is stored. #18049 covers hiding fields that do not apply to the selected kind.diff --git a/.agents/skills/verify-openhands/references/feature-map/F16-application-settings.md b/.agents/skills/verify-openhands/references/feature-map/F16-application-settings.mdindex a77c94c559..7b19f9126b 100644--- a/.agents/skills/verify-openhands/references/feature-map/F16-application-settings.md+++ b/.agents/skills/verify-openhands/references/feature-map/F16-application-settings.md@@ -2,23 +2,25 @@Settings → Application holds the user's app-wide preferences: UI language,colour theme, anonymous usage data, sound notifications, the sidebar Getting-Started checklist, which LLM profile writes conversation titles, an optional+Started checklist, Home's Recommended automations rail, which LLM profile writes conversation titles, an optionalvoice-input transcription endpoint, and the git identity used for commits. Thepage mixes two save models: language, analytics, sound, title model and gitidentity wait for **Save Changes** and are stored on the backend; theme,-checklist and voice input apply instantly and live in this browser only.+checklist, recommended automations and voice input apply instantly and live in+this browser only.-Source: `src/routes/app-settings.tsx`, `src/components/features/settings/app-settings/`, `src/components/features/settings/settings-switch.tsx`, `src/themes/color-theme/`, `src/utils/transcription-endpoint-storage.ts`, `src/api/settings-service/settings-service.api.ts`, `src/services/telemetry.ts`.+Source: `src/routes/app-settings.tsx`, `src/components/features/settings/app-settings/`, `src/components/features/settings/settings-switch.tsx`, `src/components/features/home/home-recommended-automations-storage.ts`, `src/themes/color-theme/`, `src/utils/transcription-endpoint-storage.ts`, `src/api/settings-service/settings-service.api.ts`, `src/services/telemetry.ts`.## Sub-features- `F16.open`: the page opens from the settings navigation, the command menu and the direct URL; every section renders and **Save Changes** is disabled while nothing has changed.- `F16.language`: choosing a language and saving switches the whole UI at once and persists after a reload; `<html lang>` and `<html dir>` follow the chosen language (`rtl` for Arabic).-- `F16.color-theme`: choosing a colour theme applies it immediately without Save, persists after a reload and on other pages, and is never sent to the backend.+- `F16.color-theme`: choosing a colour theme applies it immediately without Save, persists after a reload and on other pages, and is never sent to the backend. In the default OpenHands-Neutral theme primary buttons (such as an enabled **Save Changes**) are white with dark text.- `F16.analytics`: the "Send anonymous usage data" switch saves with the form and sets browser telemetry consent to granted/denied.- `F16.analytics-cloud`: on a Cloud backend the analytics switch is forced on and disabled.- `F16.sound`: the "Sound Notifications" switch saves with the form; with it on, a sound plays when the agent finishes or needs input.-- `F16.checklist`: the "Show Getting Started checklist" switch hides or shows the sidebar checklist immediately, without Save, and survives a reload.+- `F16.checklist`: the "Show Getting Started checklist" switch hides or shows the sidebar checklist immediately, without Save, and survives a reload. On a canvas locked to a Cloud host neither the checklist nor this switch is shown.+- `F16.recommended-automations`: the "Show Recommended automations" switch shows or hides Home's Recommended automations rail, without Save; it shares one browser-only preference with the rail's X (F03.recommended-rail-dismiss), so it reads off after the X was used and turning it on brings the rail back on the next Home render.- `F16.title-model`: the Title generation model dropdown offers "Automatic (recommended)" plus one `name · model` option per LLM profile; the choice persists and new conversations still get an automatic title.- `F16.title-model-fallback`: if the chosen profile is deleted (from LLM settings, or outside the UI), the dropdown shows Automatic and the form stays clean.- `F16.manage-profiles-link`: "Manage LLM profiles" goes to `/settings/llm`.@@ -26,12 +28,12 @@ Source: `src/routes/app-settings.tsx`, `src/components/features/settings/app-set- `F16.git-identity`: Git Username and Git Email save with the form, persist after a reload, and are what the agent's commits use.- `F16.git-identity-validation`: an email without `@` is refused by native validation; empty fields save the defaults `openhands` / `[email protected]`.- `F16.save-states`: Save is enabled only while a saved field differs from the stored value; while a save is pending the form stays mounted with a disabled fieldset and a "Saving..." button; success toasts "Settings saved", failure toasts an error and keeps the user's edits and Save enabled for a retry.-- `F16.keyboard`: every control, including the three switches, is reachable with Tab and exposed to assistive technology.+- `F16.keyboard`: every control, including the four switches, is reachable with Tab and exposed to assistive technology.- `F16.phone`: the form fits a 390 px viewport without horizontal overflow and Save is reachable by scrolling.- `F16.title-model-clear`: the Title generation model combobox has a clear (×) button, shown on hover; it resets the choice to Automatic (recommended).- `F16.dropdown-filter`: typing into the Language (or any) combobox filters its options; Escape restores the stored value and leaves Save disabled.- `F16.backend-switch`: switching the active backend in the backend selector while this page is open shows the new backend's stored preferences (the footer already names it), also when switching back to a backend the page has already loaded; an unsaved edit is dropped on switch.-- `F16.browser-scope`: backend-saved preferences (language and the rest of the Save group) follow the user to a fresh browser profile, while theme and checklist visibility return to their defaults there.+- `F16.browser-scope`: backend-saved preferences (language and the rest of the Save group) follow the user to a fresh browser profile, while theme, checklist visibility and the Recommended automations rail return to their defaults there.## How to get to it (user POV)@@ -47,20 +49,21 @@ Preconditions:- Baseline state (launched, doctored, `onboard --skip` done; it stores analytics off, so the analytics switch starts unchecked).- `F16.title-model`, `F16.title-model-fallback` and the agent half of `F16.git-identity` need LLM profiles: `control-openhands llm preset deepseek --api-key-file <key file>` (or `--api-key-env DEEPSEEK_API_KEY`), which saves `deepseek-flash` (active) and `deepseek-pro`.- Read the starting values once: `control-openhands api GET /api/settings`; backend-saved fields are under `body.misc_settings.app_preferences` (`language`, `user_consents_to_analytics`, `enable_sound_notifications`, `title_llm_profile`, `git_user_name`, `git_user_email`). Keys absent there mean defaults.-- The three switches are visually hidden native checkboxes inside a `<label>`: click the label text scoped to the form or focus them with Tab and toggle with Space, and read state with `control-openhands browser eval "document.querySelector('[data-testid=enable-analytics-switch]').checked"` (same for `enable-sound-notifications-switch`, `show-getting-started-checklist-switch`).+- The four switches are visually hidden native checkboxes inside a `<label>`: click the label text scoped to the form or focus them with Tab and toggle with Space, and read state with `control-openhands browser eval "document.querySelector('[data-testid=enable-analytics-switch]').checked"` (same for `enable-sound-notifications-switch`, `show-getting-started-checklist-switch`, `show-recommended-automations-switch`).- Every bullet starts on `/settings/app` at the desktop viewport unless it says otherwise (`control-openhands browser goto /settings/app`, then `control-openhands browser wait 'testid=submit-button'`).-- `F16.analytics-cloud` is **blocked** without an OpenHands Cloud backend and account.+- `F16.analytics-cloud` is **blocked** without an OpenHands Cloud backend and account. The locked-to-Cloud half of `F16.checklist` is **blocked**: `control-openhands launch` cannot start a locked-to-Cloud deployment (see the index's Not mapped).- **Open from navigation (`F16.open`).** From `/` run `control-openhands browser click 'testid=backend-selector-settings-link'`, `control-openhands browser click 'testid=sidebar-settings-/settings/app'`, then `control-openhands browser url`, `control-openhands browser snapshot 'testid=app-settings-screen'` and `control-openhands browser screenshot --feature F16.open --name desktop`. The URL ends in `/settings/app`; the snapshot shows comboboxes "Language" (`English`), "Color Theme" (`OpenHands-Neutral`) and "Title generation model" (`Automatic (recommended)`), headings "Conversation titles", "Voice input" and "Git Settings", and `button "Save Changes" [disabled]`.- **Command-menu and URL entries (`F16.open`).** Run `control-openhands browser goto /` (the previous bullet ends on `/settings/app`, where the URL check would pass at once), `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' Application`, `control-openhands browser snapshot 'testid=command-menu'` (option "Application settings Change app language, theme, and preferences. Go" is `[selected]`), `control-openhands browser press Enter`, then `control-openhands browser url`; it ends in `/settings/app`. For the URL entry run `control-openhands browser goto /settings/app`, `control-openhands browser enabled 'testid=submit-button'` (`false`) and `control-openhands browser attr 'testid=sidebar-settings-/settings/app' aria-current` (`page`).- **Language (`F16.language`).** Run `control-openhands browser click 'testid=language-input'` (16 options), `control-openhands browser click 'role=option[name="Français"]'`, `control-openhands browser enabled 'testid=submit-button'` (`true`), `control-openhands browser click 'testid=submit-button'`, then `control-openhands browser text 'testid=submit-button'`. It reads `Enregistrer les modifications` without a reload. Run `control-openhands browser reload`, `control-openhands browser value 'testid=language-input'` (`Français`), `control-openhands browser screenshot --feature F16.language --name french` (the screenshot, not a scoped snapshot, shows the sidebar's "Nouvelle discussion" and the headings "Titres des conversations", "Saisie vocale") and `control-openhands api GET /api/settings` (`language` is `fr`). Then check the document language with `control-openhands browser eval "document.documentElement.lang + ' ' + document.documentElement.dir"` (`fr ltr`) and the translated sidebar trigger with `control-openhands browser attr 'testid=command-menu-trigger' aria-label` (`Rechercher des commandes`). Choose `role=option[name="Arabic"]` the same way (click the input, the option, Save, reload): the `eval` reads `ar rtl`, and `control-openhands browser bbox 'aside[data-collapsed]'` has `x` `1140`, so the 300 px sidebar now sits at the right edge of the 1440 px viewport. `control-openhands browser screenshot --feature F16.language --name arabic` shows the whole layout flipped right to left: the sidebar on the right, the settings navigation to its left, and the form on the left with its Arabic labels and values aligned right. Restore: `control-openhands browser click 'testid=language-input'`, `control-openhands browser click 'role=option[name="English"]'`, `control-openhands browser click 'testid=submit-button'`, `control-openhands browser reload`, `control-openhands browser text 'testid=submit-button'` reads `Save Changes` and the `eval` reads `en ltr`. In Arabic each settings switch keeps its thumb inside the track.-- **Colour theme (`F16.color-theme`).** Run `control-openhands browser click 'testid=color-theme-input'`, `control-openhands browser snapshot 'role=listbox'` (OpenHands-DeepSea, OpenHands-Neutral [selected], OpenHands-Neo, Light+, Solarized Light), `control-openhands browser click 'role=option[name="Light+"]'`, then `control-openhands browser eval "document.documentElement.style.colorScheme + ' ' + localStorage.getItem('openhands-color-theme')"` (`light light-plus`) and `control-openhands browser enabled 'testid=submit-button'` (`false`: no Save needed). `control-openhands browser screenshot --feature F16.color-theme --name light-plus` shows a white page. Second views: `control-openhands browser reload` then `control-openhands browser value 'testid=color-theme-input'` (`Light+`); `control-openhands browser goto /` then `control-openhands browser eval "getComputedStyle(document.body).backgroundColor + ' ' + document.documentElement.style.colorScheme"` (`rgb(255, 255, 255) light`); `control-openhands api GET /api/settings` contains no theme key. Restore on `/settings/app` (`control-openhands browser goto /settings/app`; the second view left `/`) by choosing `role=option[name="OpenHands-Neutral"]` the same way (`dark openhands-neutral`).+- **Colour theme (`F16.color-theme`).** Run `control-openhands browser click 'testid=color-theme-input'`, `control-openhands browser snapshot 'role=listbox'` (OpenHands-DeepSea, OpenHands-Neutral [selected], OpenHands-Neo, Light+, Solarized Light), `control-openhands browser click 'role=option[name="Light+"]'`, then `control-openhands browser eval "document.documentElement.style.colorScheme + ' ' + localStorage.getItem('openhands-color-theme')"` (`light light-plus`) and `control-openhands browser enabled 'testid=submit-button'` (`false`: no Save needed). `control-openhands browser screenshot --feature F16.color-theme --name light-plus` shows a white page. Second views: `control-openhands browser reload` then `control-openhands browser value 'testid=color-theme-input'` (`Light+`); `control-openhands browser goto /` then `control-openhands browser eval "getComputedStyle(document.body).backgroundColor + ' ' + document.documentElement.style.colorScheme"` (`rgb(255, 255, 255) light`); `control-openhands api GET /api/settings` contains no theme key. Solarized Light the same way reads `light solarized-light` with body background `rgb(253, 246, 227)`; both light themes define the Markdown alert inks (`browser eval "getComputedStyle(document.body).getPropertyValue('--oh-markdown-alert-note')"` is `#005A99` in Light+, `#005A88` in Solarized Light). Restore on `/settings/app` (`control-openhands browser goto /settings/app`; the second view left `/`) by choosing `role=option[name="OpenHands-Neutral"]` the same way (`dark openhands-neutral`). Default primary: in OpenHands-Neutral click the Sound Notifications label (Save enables), then `control-openhands browser eval "(()=>{const s=getComputedStyle(document.querySelector('[data-testid=submit-button]'));return s.backgroundColor+' / '+s.color})()"` is `rgb(255, 255, 255) / rgb(24, 24, 24)`; `browser scroll 'testid=submit-button'` and `browser screenshot --feature F16.color-theme --name default-primary` show a white **Save Changes**. Click the label again (Save disabled).- **Analytics consent (`F16.analytics`).** Run `control-openhands browser click 'testid=app-settings-screen >> text=Send anonymous usage data'`, read the switch with the `eval` from Preconditions (`true`), `control-openhands browser click 'testid=submit-button'`, `control-openhands browser wait-text 'Settings saved'`, `control-openhands browser reload`, read the switch again (`true`), `control-openhands api GET /api/settings` (`user_consents_to_analytics: true`) and `control-openhands browser eval "localStorage.getItem('openhands-telemetry-consent')"` (`granted`). Restore by clicking the label again and saving; consent reads `denied` and the API `false`.- **Cloud forces analytics (`F16.analytics-cloud`).** Blocked: needs a Cloud backend selected in the backend selector. Expected: `control-openhands browser enabled 'testid=enable-analytics-switch'` is `false` and the switch reads checked.- **Consent reversal with an in-flight Automation write (`F16.analytics`).** With a non-DNT build using a dummy PostHog key and loopback-only `VITE_POSTHOG_HOST` (no external analytics delivery), save opt-in and confirm its consent POST returned 200. Run `service pause automation`, save opt-out then opt-in through the same switch, and `service resume automation` before the five-second request timeout. Record the UI transitions and read `browser network --filter 'telemetry/consent' --bodies`; the final granted value must be posted and acknowledged. Resume before `stop`. The default hard-DNT build cannot exercise this reversal.- **Sound notifications (`F16.sound`).** Run `control-openhands browser click 'testid=app-settings-screen >> text=Sound Notifications'`, `control-openhands browser click 'testid=submit-button'`, `control-openhands browser wait-text 'Settings saved'`, `control-openhands browser reload`, read `enable-sound-notifications-switch` (`true`) and `control-openhands api GET /api/settings` (`enable_sound_notifications: true`). Restore by toggling and saving again (`false`). Audible half: with the switch on and saved, `control-openhands conversation start --prompt "Reply with only the word ok. Do not run any tools." --wait --timeout 180`, then `control-openhands browser media`: `plays` lists one `/assets/notification-<hash>.mp3` at the moment the conversation finished. With the switch off, the same check after a conversation reads `plays: []` (the `F16.git-identity` conversation below doubles as that negative check). Turn the switch off and save afterwards (`control-openhands browser goto /settings/app` first: `conversation start` left the browser on the conversation).- **Getting Started checklist (`F16.checklist`).** Run `control-openhands browser count 'testid=sidebar-onboarding-checklist'` (`1`), `control-openhands browser click 'testid=app-settings-screen >> text=Show Getting Started checklist'`, then the same `count` (`0`, immediately) and `control-openhands browser enabled 'testid=submit-button'` (`false`). `control-openhands browser screenshot --feature F16.checklist --name hidden` shows the sidebar without the "Getting started" card. `control-openhands browser reload` keeps the count at `0` and the switch unchecked. Click the label again to restore; the count returns to `1`.+- **Recommended automations (`F16.recommended-automations`).** Run `control-openhands browser eval "document.querySelector('[data-testid=show-recommended-automations-switch]').checked"` (`true` by default), `control-openhands browser click 'testid=show-recommended-automations-switch >> xpath=ancestor::label'`, then the same `eval` (`false`), `control-openhands browser enabled 'testid=submit-button'` (`false`: no Save) and `control-openhands browser eval "localStorage.getItem('openhands.home_recommended_automations_dismissed')"` (`true`); `browser reload` keeps it `false`, and `control-openhands api GET /api/settings` holds no such key. `control-openhands browser click 'testid=sidebar-conversations-link' --expect-url '/conversations'`: `browser count 'testid=recommended-automations-rail'` is `0`. Back on `/settings/app`, click the label again (`true`, storage `false`) and return to `/conversations`: `browser wait 'testid=recommended-automations-rail'` succeeds. After the rail's X was used on Home (`F03.recommended-rail-dismiss`), this switch reads `false` without a reload. At `browser viewport phone` the label is `insideViewport` with `pageHorizontalOverflow` `false` (`browser screenshot --feature F16.recommended-automations --name phone`).- **Title generation model (`F16.title-model`).** Run `control-openhands browser click 'testid=title-llm-profile-input'` and `control-openhands browser snapshot 'role=listbox'`: options `Automatic (recommended)`, `deepseek-flash · deepseek/deepseek-flash`, `deepseek-pro · deepseek/deepseek-v4-pro`. Run `control-openhands browser click 'role=option >> has-text=deepseek-pro'`, `control-openhands browser click 'testid=submit-button'`, `control-openhands browser wait-text 'Settings saved'`, `control-openhands browser reload`, `control-openhands browser value 'testid=title-llm-profile-input'` (`deepseek-pro · deepseek/deepseek-v4-pro`) and `control-openhands api GET /api/settings` (`title_llm_profile: "deepseek-pro"`). Then `control-openhands conversation start --prompt "Reply with only the word ok. Do not run any tools." --wait --timeout 180` and, a few seconds later, `control-openhands conversation status <id>` with the `id` it printed: `title` is a generated title: an emoji and a short phrase whose wording varies (for example `🔧 Acknowledge with ok only`). The start output's own `title` may already be set; when it is still `null`, the title is written a few seconds after the run finishes. Return with `control-openhands browser goto /settings/app`, choose `role=option[name="Automatic (recommended)"]`, save; `title_llm_profile` is then absent from `app_preferences`.- **Manage LLM profiles link (`F16.manage-profiles-link`).** Run `control-openhands browser click 'role=link[name="Manage LLM profiles"]'` then `control-openhands browser url`; it ends in `/settings/llm` and `control-openhands browser count 'testid=profile-row'` lists the profiles.- **Deleted profile falls back (`F16.title-model-fallback`).** Arrange a throwaway profile: `control-openhands llm set --profile qa-title --model deepseek/deepseek-flash --api-key-file <key file> --no-activate --no-validate`. On `/settings/app` select it (`control-openhands browser click 'testid=title-llm-profile-input'`, `control-openhands browser click 'role=option >> has-text=qa-title'`, save). Delete it through the UI: `control-openhands browser click 'role=link[name="Manage LLM profiles"]'`, `control-openhands browser click 'testid=profile-row >> has-text=qa-title >> testid=profile-menu-trigger'`, `control-openhands browser click 'testid=profile-actions-menu >> testid=profile-delete'`, `control-openhands browser click 'testid=delete-profile-confirm'`, then `control-openhands browser click 'testid=sidebar-settings-/settings/app'`. `control-openhands browser value 'testid=title-llm-profile-input'` is `Automatic (recommended)`, Save is disabled, and `title_llm_profile` is absent from the API's `app_preferences`. For a preference that goes stale outside the UI, arrange `control-openhands api PATCH /api/settings --data '{"misc_settings_diff":{"app_preferences":{"title_llm_profile":"qa-missing"}}}' --write`, then `control-openhands browser reload`: the dropdown still shows `Automatic (recommended)` and Save stays disabled, while the API keeps `qa-missing` until the next Save of this form. Any Save writes the form's current preferences, which no longer hold the stale key, so saving an unrelated change removes it: toggle Sound Notifications, save, toggle back, save.@@ -105,7 +108,7 @@ Preconditions:control-openhands assert --field value --equals true -- browser eval "document.querySelector('[data-testid=enable-sound-notifications-switch]').checked"```-- **Keyboard and screen readers (`F16.keyboard`).** Run `control-openhands browser focus 'testid=color-theme-input'`, then repeat `control-openhands browser press Tab` and `control-openhands browser eval "(document.activeElement.getAttribute('data-testid')||'') + ' ' + document.activeElement.tagName"`. Expected (the acceptance criterion): focus visits the three switches before `title-llm-profile-input`. The first Tab lands on `enable-analytics-switch`, followed by `enable-sound-notifications-switch` and `show-getting-started-checklist-switch`; Space toggles the focused checkbox. `control-openhands browser snapshot 'testid=app-settings-screen' --feature F16.keyboard --name aria` exposes each checkbox label and checked state. The label draws a visible focus outline.+- **Keyboard and screen readers (`F16.keyboard`).** Run `control-openhands browser focus 'testid=color-theme-input'`, then repeat `control-openhands browser press Tab` and `control-openhands browser eval "(document.activeElement.getAttribute('data-testid')||'') + ' ' + document.activeElement.tagName"`. Expected (the acceptance criterion): focus visits the four switches before `title-llm-profile-input`. The first Tab lands on `enable-analytics-switch`, followed by `enable-sound-notifications-switch`, `show-getting-started-checklist-switch` and `show-recommended-automations-switch`; Space toggles the focused checkbox. `control-openhands browser snapshot 'testid=app-settings-screen' --feature F16.keyboard --name aria` exposes each checkbox label and checked state. The label draws a visible focus outline.```shcontrol-openhands browser viewport desktop@@ -135,7 +138,7 @@ Preconditions:- **Phone layout (`F16.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /settings/app`, `control-openhands browser bbox 'testid=app-settings-screen'` (`pageHorizontalOverflow: false`, width 348 at x 16), `control-openhands browser screenshot --feature F16.phone --name top`, `control-openhands browser scroll 'testid=submit-button'`, `control-openhands browser bbox 'testid=submit-button'` (`insideViewport: true`) and `control-openhands browser screenshot --feature F16.phone --name bottom`. Return with `control-openhands browser viewport desktop`.- **Clear the title model (`F16.title-model-clear`).** Run `control-openhands browser click 'testid=title-llm-profile-input'`, `control-openhands browser click 'role=option >> has-text=deepseek-flash'` (Save enables), `control-openhands browser hover 'testid=title-llm-profile-input'`, then `control-openhands browser click 'testid=title-llm-profile-input >> xpath=../..//button[not(@aria-label)]'`. `control-openhands browser value 'testid=title-llm-profile-input'` reads `Automatic (recommended)` and `control-openhands browser enabled 'testid=submit-button'` is `false`. Without the hover the click times out (`<div ...> intercepts pointer events`).- **Filter a dropdown by typing (`F16.dropdown-filter`).** Run `control-openhands browser fill 'testid=language-input' Deu` and `control-openhands browser snapshot 'role=listbox'`: the only option is `Deutsch`. Save is briefly `true` while the text is typed; `control-openhands browser press Escape` restores `control-openhands browser value 'testid=language-input'` to `English` and `control-openhands browser enabled 'testid=submit-button'` to `false`.-- **Browser vs backend scope (`F16.browser-scope`).** Choose `Deutsch` in `testid=language-input` and save (wait until `control-openhands browser text 'testid=submit-button'` reads `Änderungen speichern`), choose `Solarized Light` in `testid=color-theme-input`, and hide the checklist with `control-openhands browser click 'testid=show-getting-started-checklist-switch >> xpath=ancestor::label'` (the label text is German now). Then `control-openhands browser reset`, `control-openhands onboard --skip`, `control-openhands browser goto /settings/app`, `control-openhands browser wait 'testid=submit-button'`: `control-openhands browser text 'testid=submit-button'` is `Änderungen speichern` and `control-openhands browser value 'testid=language-input'` is `Deutsch` (backend), while `control-openhands browser value 'testid=color-theme-input'` is `OpenHands-Neutral`, `control-openhands browser eval "localStorage.getItem('openhands-color-theme')"` is `null` and `control-openhands browser count 'testid=sidebar-onboarding-checklist'` is `1` (browser-only, back to defaults). Restore English (the option keeps its English name in the German UI): `control-openhands browser click 'testid=language-input'`, `control-openhands browser click 'role=option[name="English"]'`, `control-openhands browser click 'testid=submit-button'`, `control-openhands browser reload`, `control-openhands browser wait 'testid=submit-button'`; `control-openhands browser text 'testid=submit-button'` reads `Save Changes` and `control-openhands api GET /api/settings --pick misc_settings.app_preferences.language` is `en`.+- **Browser vs backend scope (`F16.browser-scope`).** Choose `Deutsch` in `testid=language-input` and save (wait until `control-openhands browser text 'testid=submit-button'` reads `Änderungen speichern`), choose `Solarized Light` in `testid=color-theme-input`, hide the checklist with `control-openhands browser click 'testid=show-getting-started-checklist-switch >> xpath=ancestor::label'` (the label text is German now) and turn off `show-recommended-automations-switch` the same way. Then `control-openhands browser reset`, `control-openhands onboard --skip`, `control-openhands browser goto /settings/app`, `control-openhands browser wait 'testid=submit-button'`: `control-openhands browser text 'testid=submit-button'` is `Änderungen speichern` and `control-openhands browser value 'testid=language-input'` is `Deutsch` (backend), while `control-openhands browser value 'testid=color-theme-input'` is `OpenHands-Neutral`, `control-openhands browser eval "localStorage.getItem('openhands-color-theme')"` is `null` `control-openhands browser count 'testid=sidebar-onboarding-checklist'` is `1` and `browser eval "localStorage.getItem('openhands.home_recommended_automations_dismissed')"` is `null` with the Home rail shown (browser-only, back to defaults). Restore English (the option keeps its English name in the German UI): `control-openhands browser click 'testid=language-input'`, `control-openhands browser click 'role=option[name="English"]'`, `control-openhands browser click 'testid=submit-button'`, `control-openhands browser reload`, `control-openhands browser wait 'testid=submit-button'`; `control-openhands browser text 'testid=submit-button'` reads `Save Changes` and `control-openhands api GET /api/settings --pick misc_settings.app_preferences.language` is `en`.- **Backend switch in place (`F16.backend-switch`).** Needs a second backend with a different Git username: the second stack from F25's Preconditions added as `QA_Second` (F25 Add bullet), arranged with `control-openhands api PATCH /api/settings --run "$OH_VERIFY_RUN_2" --write --data '{"misc_settings_diff":{"app_preferences":{"git_user_name":"QA Second Git"}}}'`. With Local active, run `control-openhands browser goto /settings/app` and `control-openhands browser reload`; `browser value 'testid=git-user-name-input'` is `openhands`. Switch without leaving the page: `control-openhands browser click 'testid=backend-selector'`, `control-openhands browser click 'testid=backend-selector >> role=option[name="Connected QA_Second"]'`, `control-openhands browser wait 'testid=environment-switch-overlay' --state detached --timeout 10000`, wait a few seconds: the value is `QA Second Git` and `browser text 'testid=backend-synced-settings-badge'` names QA_Second (a first visit to a backend loads fresh). Switch back the same way to `Connected Local`: the value is `openhands` beside `synced from Local backend`, and it keeps following the active backend on further switches. The form remounts per backend, so an unsaved edit is dropped: on Local run `control-openhands browser fill 'testid=git-user-name-input' 'QA Local Edit'` (`browser enabled 'testid=submit-button'` is `true`), switch to `QA_Second` (`QA Second Git`, Save `false`) and back (`openhands`, Save `false`); `control-openhands api GET /api/settings --pick misc_settings.app_preferences.git_user_name` is unchanged. At `browser viewport phone` the selector is in the drawer (`testid=sidebar-mobile-menu-toggle`, then `'testid=sidebar-mobile-drawer >> testid=backend-selector'` and `role=option[name="Connected QA_Second"]`; close the drawer with `browser mouse-click 370 400`) and the field follows the same way. Switch back to Local and remove `QA_Second` afterwards (F25).- After the family, `control-openhands browser errors --app-only` reports `appErrors: 0`. Requests to `registry.npmjs.org` (update check) and Google Fonts fail through the sandbox proxy; they are not app errors.@@ -173,7 +176,7 @@ control-openhands assert --field value --equals true -- browser eval "['enable-a## Gotchas-- Two save models on one form: theme, checklist and voice input apply instantly and ignore Save; the rest is dropped silently if you navigate away before Save (filling `git-user-name-input`, then clicking `sidebar-settings-/settings/secrets` and back, shows the stored `openhands` again; `browser dialogs` is empty). Assert the instant ones without clicking Save, and never expect Save to enable for them.+- Two save models on one form: theme, checklist, recommended automations and voice input apply instantly and ignore Save; the rest is dropped silently if you navigate away before Save (filling `git-user-name-input`, then clicking `sidebar-settings-/settings/secrets` and back, shows the stored `openhands` again; `browser dialogs` is empty). Assert the instant ones without clicking Save, and never expect Save to enable for them.- The success toast uses the language that was active when you clicked Save, not the one you saved. In `F16.language` the French Save toasts `Settings saved`, the Arabic Save toasts `Paramètres enregistrés`, and the English restore toasts `تم حفظ الإعدادات`, so `wait-text 'Settings saved'` times out on the last two. Assert with `browser text 'testid=submit-button'` after a reload, or read the toast with `browser toasts`.- Some chrome strings stay English after a language switch: the sidebar **Automate** label in every language (the automation interface supplies it, not the translation file), and most of the command menu's own texts (placeholder `Search actions, pages, and settings...`, the search box's `Search commands` name, the group names and the item names) in French and 11 other languages. Only the Model Router and Agent Context items are translated (`Routeur de modèles` and `Contexte de l'agent` in French), because they reuse the settings navigation's titles. Do not treat the English ones as a language switch failure; the sidebar trigger's `Search commands` is translated.- In Arabic (`dir="rtl"`), check both on/off thumb containment within the track and the native `.checked` state. State readback alone does not prove correct placement; the normal RTL regression recipes above retain both assertions.diff --git a/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md b/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.mdindex 8297cf0bc2..1449925ec8 100644--- a/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md+++ b/.agents/skills/verify-openhands/references/feature-map/F17-mcp-servers.md@@ -34,7 +34,7 @@ Source: `src/routes/extensions-hub.tsx`, `src/routes/mcp.tsx`, `src/components/f- `F17.test-connection`: **Test connection** in the editor shows `Verifying…`, then a green `Connected — N tool(s) available` or a red categorized error; Add/Save tests first and stays open on failure.- `F17.custom-add`: **Add Server** saves and closes; the card is listed after a reload.- `F17.custom-add-remote`: a custom Streamable HTTP (SHTTP) server with a **Bearer token** connects, saves as an `HTTP` card showing its URL, reopens with the token masked, and its tools reach the agent as `<server>_<tool>`.-- `F17.server-health`: each card's health row runs **Test connection** (`Checking connection…`), then shows a verdict with a coloured dot; a failure is red, worded for the transport (a STDIO command that cannot start, a remote server that cannot be reached) with the Agent Server's detail, and offers **Retry**.+- `F17.server-health`: each card's health row runs **Test connection** (`Checking connection…`), then shows a verdict with a coloured dot; a failure's dot uses the theme's status-error colour (`bg-status-error`) and its text is red, worded for the transport (a STDIO command that cannot start, a remote server that cannot be reached) with the Agent Server's detail, and offers **Retry**.- `F17.enable-disable`: the card toggle disables a server without removing its configuration; the state persists and a disabled server is withheld from new conversations.- `F17.custom-edit`: clicking a card opens **Edit MCP server** prefilled (secrets shown as `**********`); Save tests and persists the change.- `F17.stored-secret-reuse`: a saved server's env values and tokens come back from the settings API and in the editor as `**********`; the editor's Test connection swaps each placeholder for the stored encrypted value (never the placeholder text, never the plaintext) and Save leaves untouched secrets out of its sparse patch, so a Save from the edit form that keeps them changes only the edited fields.@@ -88,7 +88,7 @@ Preconditions:- **Test connection (`F17.test-connection`).** Failure: in the SHTTP form left open by validation, run `control-openhands browser click 'testid=add-mcp-server-form >> testid=auth-mode-dropdown'`, `control-openhands browser click 'role=option[name="None"]'` and `control-openhands browser fill 'testid=add-mcp-server-form >> testid=timeout-input' 10`. With URL `http://127.0.0.1:9/mcp`, click `testid=add-mcp-server-form >> testid=mcp-test-connection`, `control-openhands browser wait 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000`; the text is `Could not reach the server (check the URL and server type): Client failed to connect: All connection attempts failed` (the remote wording followed by the Agent Server's detail) and its class contains `text-feedback-error`. Clicking **Add Server** leaves the editor open (`browser count 'testid=mcp-custom-editor'` is `1`). Success: run `control-openhands browser click 'testid=add-mcp-server-form >> testid=server-type-dropdown'` and `control-openhands browser click 'role=option[name="STDIO"]'`, then fill `testid=name-input` `qa_time`, `testid=command-input` `uvx`, `control-openhands browser fill 'testid=add-mcp-server-form >> testid=args-input' mcp-server-time` and `control-openhands browser fill 'testid=add-mcp-server-form >> testid=env-input' UV_NATIVE_TLS=true`, then `control-openhands browser click 'testid=add-mcp-server-form >> testid=mcp-test-connection' --observe 'testid=add-mcp-server-form >> testid=mcp-test-connection' --observe-ms 20000` (`Test connection`, `Verifying…`, `Test connection`), the same `wait` and `control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message'`: `Connected — 2 tool(s) available`, class `text-feedback-success`. `control-openhands browser screenshot 'testid=mcp-custom-editor' --feature F17.test-connection --name stdio-success`.- **Add the server (`F17.custom-add`).** In the successful STDIO form run `control-openhands browser click 'testid=add-mcp-server-form >> testid=submit-button'` and `control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000`, then `control-openhands browser reload` and `control-openhands browser text 'testid=mcp-server-item >> has-text=qa_time'`: `qa_time`, `STDIO`, `Timezone-aware current time, conversions, and timestamp formatting.`, `uvx mcp-server-time`, `Not checked yet`, `Test connection` (it runs the Time entry's command and args, so it shows that entry's description and badge under its own name, #18034). `control-openhands browser attr 'testid=mcp-server-item >> has-text=qa_time' data-server-id` is `qa_time` (the id in the per-card test ids below) and its `aria-label` is `Edit qa_time`.- **Custom remote server (`F17.custom-add-remote`).** With the Streamable HTTP fixture from Preconditions running, run `control-openhands browser click 'testid=mcp-add-custom-server'`, pick `SHTTP` (`server-type-dropdown`, then `role=option[name="SHTTP"]`), fill `testid=add-mcp-server-form >> testid=server-name-input` `qa_shttp` and `url-input` `http://127.0.0.1:38417/mcp`, pick auth `Bearer token` (`auth-mode-dropdown`, then `role=option[name="Bearer token"]`) and fill `testid=add-mcp-server-form >> testid=api-key-input` `qa-dummy-token` (`browser attr ... type` is `password`). Click `testid=add-mcp-server-form >> testid=mcp-test-connection`; `control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000` is `Connected — 13 tool(s) available`. Click `testid=add-mcp-server-form >> testid=submit-button`, wait for `testid=mcp-custom-editor` to detach and `browser reload`: `control-openhands browser text '[data-server-id=qa_shttp]'` is `qa_shttp`, `HTTP`, `http://127.0.0.1:38417/mcp`, `Not checked yet`, `Test connection`; `control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_shttp` has `transport` `http` and `auth` `{"strategy":"bearer","value":"**********"}`. Its health probe reads `Reachable — credentials not verified`. Reopen it with `control-openhands browser click 'testid=mcp-server-detail-qa_shttp'`: `browser snapshot 'testid=mcp-custom-editor'` shows `Edit MCP server`, combobox `Authentication` `Bearer token`, and `control-openhands browser value 'testid=edit-mcp-server-form >> testid=api-key-input'` is `**********`; Escape. Agent side: `control-openhands conversation start --prompt "Call the qa_shttp MCP server's echo tool with message qa-ping. Do not use the terminal. Reply with only the tool's output." --wait --timeout 240`, then `control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent`: tool `qa_shttp_echo`, observation `Echo: qa-ping`. Run `control-openhands browser goto /mcp` afterwards.-- **Card health (`F17.server-health`).** Run `control-openhands browser click 'testid=mcp-health-probe-qa_time' --observe 'testid=mcp-health-label-qa_time' --observe-ms 20000`: `Not checked yet`, `Checking connection…`, `Reachable — credentials not verified`. `control-openhands browser text 'testid=mcp-server-health-qa_time'` adds the hint `This check only proves the server is reachable and lists its tools; it does not verify your credentials.`, `control-openhands browser attr 'testid=mcp-server-health-qa_time >> testid=mcp-health-dot' data-status` is `healthy-connectivity`, and the click did not open the editor (`browser count 'testid=mcp-custom-editor'` is `0`). Failure: the editor refuses to save a broken server, so arrange one with `control-openhands api POST /api/settings/mcp/qa_broken --data '{"transport":"stdio","command":"qa-no-such-command","enabled":false}' --write`, `control-openhands browser reload`, then click `testid=mcp-health-probe-qa_broken`: the label is `Could not start the server command: Client failed to connect: [Errno 2] No such file or directory: 'qa-no-such-command'` in red (STDIO wording, no URL hint; the card clamps it to two lines, `browser text 'testid=mcp-server-health-qa_broken'` reads it whole), the dot's `data-status` is `failed` and the button reads `Retry`; `control-openhands browser screenshot 'testid=mcp-server-item >> has-text=qa_broken' --feature F17.server-health --name failed`.+- **Card health (`F17.server-health`).** Run `control-openhands browser click 'testid=mcp-health-probe-qa_time' --observe 'testid=mcp-health-label-qa_time' --observe-ms 20000`: `Not checked yet`, `Checking connection…`, `Reachable — credentials not verified`. `control-openhands browser text 'testid=mcp-server-health-qa_time'` adds the hint `This check only proves the server is reachable and lists its tools; it does not verify your credentials.`, `control-openhands browser attr 'testid=mcp-server-health-qa_time >> testid=mcp-health-dot' data-status` is `healthy-connectivity`, and the click did not open the editor (`browser count 'testid=mcp-custom-editor'` is `0`). Failure: the editor refuses to save a broken server, so arrange one with `control-openhands api POST /api/settings/mcp/qa_broken --data '{"transport":"stdio","command":"qa-no-such-command","enabled":false}' --write`, `control-openhands browser reload`, then click `testid=mcp-health-probe-qa_broken`: the label is `Could not start the server command: Client failed to connect: [Errno 2] No such file or directory: 'qa-no-such-command'` in red (STDIO wording, no URL hint; the card clamps it to two lines, `browser text 'testid=mcp-server-health-qa_broken'` reads it whole), the dot's `data-status` is `failed`, its class contains `bg-status-error` (its computed `background-color` equals that token's: `rgb(255, 104, 78)` in the default dark theme, `rgb(209, 52, 56)` in Light+) and the button reads `Retry`; `control-openhands browser screenshot 'testid=mcp-server-item >> has-text=qa_broken' --feature F17.server-health --name failed`.- **Delete (`F17.delete-server`).** Run `control-openhands browser click 'testid=mcp-server-detail-qa_broken'` (a click on the card's centre lands on its health row and opens nothing: see Gotchas), `control-openhands browser click 'testid=mcp-custom-editor-delete'` and `control-openhands browser text 'testid=confirmation-modal'` (`Are you sure you want to delete this server?`, `Cancel`, `Confirm`). Run `control-openhands browser click 'testid=confirmation-modal >> testid=cancel-button'`; the editor is still open. Click Delete again, then `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'` and `control-openhands browser toasts` (`MCP server removed.`). After `control-openhands browser reload`, `control-openhands browser count 'testid=mcp-server-item >> has-text=qa_broken'` is `0`, and `control-openhands api GET /api/settings` has no `qa_broken`.- **Edit (`F17.custom-edit`).** Run `control-openhands browser click 'testid=mcp-server-detail-qa_time'` (the detail line opens the editor on every card; a plain click on the card's centre happens to open it on `qa_time`, whose Time description fills the centre, but not on a card without a description; keyboard: `browser focus '[data-server-id=qa_time]'` then `browser press Enter`) and `control-openhands browser snapshot 'testid=mcp-custom-editor'`: heading `Edit MCP server`, no Server Type field, a `Delete` button; `control-openhands browser value 'testid=edit-mcp-server-form >> testid=env-input'` is `UV_NATIVE_TLS=**********`. Run `control-openhands browser fill 'testid=edit-mcp-server-form >> testid=args-input' $'mcp-server-time\n--local-timezone\nUTC'` (bash ANSI-C quoting for the newlines), click `testid=edit-mcp-server-form >> testid=mcp-test-connection` (still `Connected — 2 tool(s) available`: the stored env value is reused), then `control-openhands browser click 'testid=edit-mcp-server-form >> testid=submit-button'` and wait for `testid=mcp-custom-editor` to detach. The card shows a verdict at once (seeded from the save's test). After `control-openhands browser reload`, `control-openhands browser text 'testid=mcp-server-detail-qa_time'` is `uvx mcp-server-time --local-timezone UTC`.- **Stored secrets are reused, not their placeholder (`F17.stored-secret-reuse`).** Make a STDIO server that only starts with the right secret: `control-openhands fixture mcp-server --name qa-vault` prints `<node>` and `<server.mjs>`. Run `control-openhands browser click 'testid=mcp-add-custom-server'`, pick `STDIO`, fill `testid=add-mcp-server-form >> testid=name-input` `qa_vault` and `testid=add-mcp-server-form >> testid=command-input` `/bin/sh`, then `control-openhands browser fill 'testid=add-mcp-server-form >> testid=args-input' $'-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"'` and `control-openhands browser fill 'testid=add-mcp-server-form >> testid=env-input' $'QA_VAULT_TOKEN=qa-vault-secret-000\nQA_NODE=<node>\nQA_SERVER=<server.mjs>'`. Run `control-openhands browser network --clear`, click `testid=add-mcp-server-form >> testid=mcp-test-connection`; `control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000` is `Connected — 1 tool(s) available`. Direct proof of what the page sent: `control-openhands browser network --bodies --filter 'api/mcp/test' --last 1` lists one `POST /api/mcp/test` (`200`) whose `body` is `{"server":{"type":"stdio","command":"/bin/sh","args":["-c","test \"$QA_VAULT_TOKEN\" = qa-vault-secret-000 && exec \"$QA_NODE\" \"$QA_SERVER\""],"env":{"QA_VAULT_TOKEN":"<redacted 19 chars>","QA_NODE":"<redacted 40 chars>","QA_SERVER":"<redacted 142 chars>"}},"name":"qa_vault"}`: the command and args in clear, each env value replaced by the CLI with the length of the real value (19 for `qa-vault-secret-000`; 40 and 142 are `<node>` and `<server.mjs>` on this machine, `${#var}` in a shell). Click `testid=add-mcp-server-form >> testid=submit-button`, `control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000` and `control-openhands browser reload`. `control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault` has `env` `{"QA_VAULT_TOKEN":"**********","QA_NODE":"**********","QA_SERVER":"**********"}` (every env value is redacted; the page reads this same endpoint, there is no `GET /api/settings/mcp`). Run `control-openhands browser click 'testid=mcp-server-detail-qa_vault'`; `control-openhands browser value 'testid=edit-mcp-server-form >> testid=env-input'` is the three lines `QA_VAULT_TOKEN=**********`, `QA_NODE=**********`, `QA_SERVER=**********`. Run `control-openhands browser network --clear` and click `testid=edit-mcp-server-form >> testid=mcp-test-connection`: `control-openhands browser text 'testid=edit-mcp-server-form >> testid=mcp-test-message' --timeout 60000` is `Connected — 1 tool(s) available`, which the shell gate only allows when the stored token, binary and path were sent (`control-openhands browser screenshot 'testid=mcp-custom-editor' --feature F17.stored-secret-reuse --name masked-test-ok`). The same `control-openhands browser network --bodies --filter 'api/mcp/test' --last 1` row now carries `"env":{"QA_VAULT_TOKEN":"<redacted 120 chars>","QA_NODE":"<redacted 140 chars>","QA_SERVER":"<redacted 268 chars>"}`: neither the `**********` placeholder (which `--bodies` would keep as is) nor the typed lengths. `control-openhands browser network --last 5` shows why: a `GET /api/settings` (and `/server_info`) right before the POST. The page fetches the settings in `encrypted` mode and swaps each placeholder for the stored *encrypted* value (`src/api/mcp-service/mcp-redacted-credentials.ts`), which the Agent Server decrypts, so the browser never holds the plaintext and the lengths are those of Fernet tokens of the real values (19 → 120, 40 → 140, 142 → 268). Counter-check that the gate bites: `control-openhands browser fill 'testid=edit-mcp-server-form >> testid=env-input' $'QA_VAULT_TOKEN=qa-wrong\nQA_NODE=**********\nQA_SERVER=**********'`, `control-openhands browser network --clear`, click Test connection again and `control-openhands browser wait 'testid=edit-mcp-server-form >> testid=mcp-test-message >> text=Connection' --timeout 90000`; the text is red and reads `Connection failed: McpError: Connection closed` or, when the client only notices at its deadline, `Connection timed out. Check the URL and try again.` (the gate exits before the MCP handshake; see Gotchas; `--name wrong-token`), and the `--bodies` row shows `"QA_VAULT_TOKEN":"<redacted 8 chars>"` next to the two kept placeholders at `140` and `268` chars: only the retyped value replaced its stored one. `control-openhands browser click 'testid=mcp-custom-editor-close'` discards it (`count 'testid=mcp-custom-editor'` `0`); reopen with `testid=mcp-server-detail-qa_vault`: the env is the three placeholders again and Test connection is `Connected — 1 tool(s) available`. `control-openhands browser press Escape`. Save with the placeholders in place: reopen with `control-openhands browser click 'testid=mcp-server-detail-qa_vault'`, append one harmless argument (it only becomes the shell script's `$0`) with `control-openhands browser fill 'testid=edit-mcp-server-form >> testid=args-input' $'-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"\nqa-v2'`, leave `env-input` as the three placeholders, run `control-openhands browser network --clear`, then `control-openhands browser click 'testid=edit-mcp-server-form >> testid=submit-button' --observe 'testid=edit-mcp-server-form >> testid=submit-button' --observe-ms 6000` (`Save Server`, `Save Server [disabled]`, `<absent>`) and `control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000`; `control-openhands browser network --bodies --filter 'api/settings/mcp' --last 3` lists one `PATCH /api/settings/mcp/qa_vault` (`200`) whose `body` is `{"transport":"stdio","command":"/bin/sh","args":["-c","test \"$QA_VAULT_TOKEN\" = qa-vault-secret-000 && exec \"$QA_NODE\" \"$QA_SERVER\"","qa-v2"]}`: a sparse patch with no `env` key at all (untouched secrets are left out, so the server keeps its stored values; neither the placeholder nor an encrypted value travels). The same `control-openhands browser network --filter 'api/settings/mcp' --last 3` without `--bodies` lists that row without a `body` field. The card's health label reads `Reachable — credentials not verified`. After `control-openhands browser reload`, `control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault.env` is still `{"QA_VAULT_TOKEN":"**********","QA_NODE":"**********","QA_SERVER":"**********"}`, `control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault.args` ends with `qa-v2`, and `control-openhands browser text 'testid=mcp-server-detail-qa_vault'` ends with `"$QA_SERVER" qa-v2`. Reopen with `control-openhands browser click 'testid=mcp-server-detail-qa_vault'` (the env is the three placeholders), run `control-openhands browser network --clear` and click `testid=edit-mcp-server-form >> testid=mcp-test-connection`: `control-openhands browser text 'testid=edit-mcp-server-form >> testid=mcp-test-message' --timeout 60000` is `Connected — 1 tool(s) available` again, which the gate only allows if the Save kept the real token, binary and path rather than storing the placeholder text (`--name saved-with-placeholders`), and the `--bodies` row carries `qa-v2` in `args` with the three encrypted lengths (`120`, `140`, `268`) in `env`. `control-openhands browser press Escape`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F18-skills.md b/.agents/skills/verify-openhands/references/feature-map/F18-skills.mdindex d9b224d01c..39236e40da 100644--- a/.agents/skills/verify-openhands/references/feature-map/F18-skills.md+++ b/.agents/skills/verify-openhands/references/feature-map/F18-skills.md@@ -1,28 +1,32 @@# F18 — Skills catalog-The Skills page (Customize → Skills, `/skills`) lists every skill the backend can-load: the built-in catalog bundled with the frontend plus personal and project-skills reported by the Agent Server. A user searches and filters the cards+The Skills page (Customize → Skills, `/skills`) lists every skill an agent+profile can load: on a local backend the Agent Server's skill catalog+(`GET /api/skills/catalog`: the public skills it reads from the bundled+`@openhands/extensions` package, plus personal skills), falling back to the+frontend's bundled catalog plus user skills on servers without that endpoint;+a Cloud backend lists its own skills. A user searches and filters the cards(filters are mirrored into the URL), opens a detail modal, switches skills on or-off (saved to settings straight away, effective for new conversations only),+off (locally saved straight away to the `default` agent profile's skill lists,+which new agent profiles start from; effective for new conversations only),launches a skill into chat, and reads how to add one with `/add-skill`; after theagent installs a skill in a conversation, a banner offers a fresh conversationthat loads it.-Source: `src/routes/skills-settings.tsx`, `src/components/features/skills/`, `src/constants/skills-docs.ts`, `src/hooks/use-skill-enablement.ts`, `src/utils/skill-enablement.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/api/skills-service.ts`, `src/components/features/chat/skill-install-restart-banner.tsx`, `src/utils/skill-install-events.ts`.+Source: `src/routes/skills-settings.tsx`, `src/components/features/skills/`, `src/constants/skills-docs.ts`, `src/hooks/use-skill-enablement.ts`, `src/utils/skill-enablement.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/api/skills-service.ts`, `src/hooks/query/use-skill-catalog.ts`, `src/components/features/chat/skill-install-restart-banner.tsx`, `src/utils/skill-install-events.ts`.## Sub-features-- `F18.page`: the page shows the title, description, the notice "Skill changes apply to new conversations only.", an **Add skill** button, a result count, the facet rail and the card grid. (Empty state "No skills found." and loading skeletons are not reachable on a local backend.)+- `F18.page`: the page shows the title, description, a notice (locally "These are the default agent profile's skills, and new agent profiles start from this list. Other profiles keep their own list."; on a Cloud backend "Skill changes apply to new conversations only."), an **Add skill** button, a result count, the facet rail and the card grid. (Empty state "No skills found." and loading skeletons are not reachable on a local backend.)- `F18.search`: typing filters cards by name, description, content or trigger; after 300 ms the query is mirrored to `?q=` without a new history entry; the X clears it; no match shows "No skills match your search."- `F18.search-history`: browser Back/Forward put the matching query back into the search box.- `F18.facets`: the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers **Clear filters**; deep links apply and unknown values are dropped.- `F18.filters-modal`: below 768 px the rail is replaced by a **Filters** button with an active-count badge that opens the same facets in a modal with Clear filters and Close.- `F18.card`: each card shows icon, name, source, a two-line description and pills; extra pills collapse into a `+N` popover; click, Enter or Space opens the detail modal.- `F18.copy-source`: personal and project skills (path sources) have a copy-path button on the card and in the modal that flips to "Copied to clipboard" for 2 s; built-in skills (source `public`) have none.-- `F18.toggle`: the card's plus/check toggle switches a built-in skill on or off; the choice survives a reload (`enabled_skills` allow-list on a local backend).-- `F18.toggle-local`: switching a personal/project skill off writes `disabled_skills`; new conversations then do not load it, and do again once it is back on.-- `F18.toggle-error`: a failed save shows an error toast and nothing is persisted.+- `F18.toggle`: the card's plus/check toggle switches a built-in skill on or off; the choice survives a reload (on a local backend it is the `default` agent profile's `public_skills` list).+- `F18.toggle-local`: switching a personal/project skill off writes the `default` agent profile's `disabled_skills`; new conversations then do not load it, and do again once it is back on.+- `F18.toggle-error`: a failed save shows an error toast and nothing is persisted. When the `default` agent profile cannot be read, the page shows an error toast and the card and modal switches are disabled, so no edit is accepted that could not be saved.- `F18.skill-in-chat`: an enabled skill is loaded into new conversations and fires on its trigger; a disabled one is not.- `F18.project-skill`: a skill committed under `<workspace>/.agents/skills/<name>/SKILL.md` is loaded into every conversation started in that workspace, in Local Repo and New Worktree mode alike, and its trigger word activates it (the chat shows a **Skill Ready** row naming it); workspace skills are not listed on the Skills page.- `F18.skill-deleted`: a personal skill whose folder was deleted disappears from the Skills page after a reload and is not loaded into new conversations: its trigger word activates nothing.@@ -51,14 +55,14 @@ Source: `src/routes/skills-settings.tsx`, `src/components/features/skills/`, `srPreconditions:- Baseline state (launched, doctored, `onboard --skip` done); desktop viewport unless a bullet says otherwise.-- A fresh run lists only the bundled catalog, `<N>` skills, all with source `public`. `<N>` is the catalog length the checkout bundles: run `node --input-type=module -e "import {SKILLS_CATALOG} from '@openhands/extensions/skills'; console.log(SKILLS_CATALOG.length)"` in the checkout (69 with `@openhands/extensions` 0.31.0). The Source and Type facet groups stay hidden until a personal/project skill exists. `add-javadoc` and `flarglebargle` are off; `docker` is on (recommended).+- A fresh run lists only the public catalog, `<N>` skills, all with source `public`. `<N>` is the catalog length the checkout bundles: run `node --input-type=module -e "import {SKILLS_CATALOG} from '@openhands/extensions/skills'; console.log(SKILLS_CATALOG.length)"` in the checkout (69 with `@openhands/extensions` 0.33.0). The Agent Server reads the same package: `control-openhands api GET /api/skills/catalog --max-bytes 3000000 | jq '.body.skills | length'` is `<N>`, and each public entry's `source` is under `<checkout>/node_modules/@openhands/extensions/skills/`. The Source and Type facet groups stay hidden until a personal/project skill exists. `add-javadoc` and `flarglebargle` are off; `docker` is on (recommended).- `F18.skill-in-chat`, `F18.toggle-local`, `F18.copy-source` and `F18.install-banner` need an active LLM profile (`control-openhands llm preset deepseek`). `F18.install-banner` also needs the agent to reach github.com.- Personal skill fixture (arranged through the agent, not proof): run `control-openhands conversation start --prompt 'Use one shell command to create the file ~/.agents/skills/qa-hello/SKILL.md (create the folders) with exactly these 7 lines: "---", "name: qa-hello", "description: QA test skill that answers qa-ping with QA-PONG-7731.", "triggers:", "- qa-ping", "---", "When the user says qa-ping, reply with exactly QA-PONG-7731 and nothing else." Then reply done.' --wait --timeout 240`. Afterwards `/skills` shows `<N+1> result(s)` and a `qa-hello` card (type pill `Auto-discovery`), and the Enabled facet count goes from 12 to 13.- Project skill fixture for `F18.project-skill` (arranged, not proof; no model needed), in this family's own repo so the commit never lands in F08's `qa-repo`: `control-openhands fixture git-repo --name qa-f18-repo`, then `control-openhands fixture skill --repo qa-f18-repo --name qa-f18-skill --trigger qa-f18-ping --commit`: it writes `<run>/workspace/qa-f18-repo/.agents/skills/qa-f18-skill/SKILL.md` (`scope` `project`) and commits it in the fixture repo (`committed.sha` and `committed.message` `Add qa-f18-skill skill`; `git -C "$OH_VERIFY_RUN/workspace/qa-f18-repo" log --oneline` lists `Add qa-f18-skill skill` above `Initial fixture commit`). A re-run reports the same `committed.sha` with `committed.unchanged` `true`. Without `--commit` the file stays uncommitted, and a New Worktree checkout holds only committed files. A fixture skill's name must not be a substring of its trigger (see Gotchas).- `F18.skill-deleted` creates its own personal fixture right before its bullet (`control-openhands fixture skill --name qa-gone --trigger qa-farewell`, written into the run's private HOME) and deletes it inside the bullet, so the counts in the other bullets are unaffected.- `F18.cloud-link` needs an OpenHands Cloud backend and account (blocked in a sandbox).-- **Open from the sidebar (`F18.page`).** From `/` run `control-openhands browser click 'testid=sidebar-skills-link' --expect-url '/mcp(\?|$)'`, `control-openhands browser click 'testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)'`, then `control-openhands browser text 'testid=skills-settings-description'` (`Discover skills to add to your workspace. Search above the cards or filter by state, recommendation, and category, then open a card to see its details and use the skill. Enable or disable default skills. Disabled skills will not be loaded into agent context.`, describing this layout since #18035), `control-openhands browser text 'testid=skills-new-conversation-notice'` (`Skill changes apply to new conversations only.`) and `control-openhands browser text 'testid=skills-result-summary'` (`<N> result(s)` on a fresh run). `control-openhands browser screenshot --feature F18.page --name desktop` shows the Customize nav, the facet rail and the card grid.+- **Open from the sidebar (`F18.page`).** From `/` run `control-openhands browser click 'testid=sidebar-skills-link' --expect-url '/mcp(\?|$)'`, `control-openhands browser click 'testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)'`, then `control-openhands browser text 'testid=skills-settings-description'` (`Discover skills to add to your workspace. Search above the cards or filter by state, recommendation, and category, then open a card to see its details and use the skill. Enable or disable default skills. Disabled skills will not be loaded into agent context.`, describing this layout since #18035), `control-openhands browser text 'testid=skills-default-list-notice'` (`These are the default agent profile's skills, and new agent profiles start from this list. Other profiles keep their own list.`; `control-openhands browser count 'testid=skills-new-conversation-notice'` is `0` on a local backend, where Cloud shows `Skill changes apply to new conversations only.` instead) and `control-openhands browser text 'testid=skills-result-summary'` (`<N> result(s)` on a fresh run). `control-openhands browser network --filter /api/skills` lists `GET /api/skills/catalog` `200`. `control-openhands browser screenshot --feature F18.page --name desktop` shows the Customize nav, the facet rail and the card grid.- **Command-menu entry (`F18.page`).** From `/` run `control-openhands browser press Control+k`, `control-openhands browser type 'testid=command-menu >> role=combobox' Skills`, `control-openhands browser snapshot 'testid=command-menu'` (one option, `Customize Browse skills, plugins, and integrations. Go`), `control-openhands browser press Enter`, `control-openhands browser wait-url '/mcp(\?|$)'`, then `control-openhands browser click 'testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)'`.- **Search (`F18.search`).** On `/skills` run `control-openhands browser fill 'testid=skills-search-input' docker`, `control-openhands browser text 'testid=skills-result-summary'` (`5 result(s)` on a fresh run: content matches count too) and, a second later, `control-openhands browser url` (ends in `/skills?q=docker`); `control-openhands browser eval "history.length"` is the same before and after typing. Then `control-openhands browser fill 'testid=skills-search-input' zzqa-nomatch` and `control-openhands browser text 'testid=skills-no-match'`: `No skills match your search.` with `0 result(s)`. Clear with `control-openhands browser click 'testid=skills-toolbar >> role=button[name="Clear search"]'`; `control-openhands browser value 'testid=skills-search-input'` is empty, the full count is back and `?q=` leaves the URL.- **Back and forward (`F18.search-history`).** Run `control-openhands browser goto '/skills?q=docker'`, `control-openhands browser click 'testid=skill-facet-state-enabled'` (pushes `?q=docker&state=enabled`), `control-openhands browser fill 'testid=skills-search-input' github`, wait a second, then `control-openhands browser back` and `control-openhands browser value 'testid=skills-search-input'`: `docker`, URL `/skills?q=docker`. `control-openhands browser forward` gives `github` and `/skills?q=github&state=enabled`.@@ -68,14 +72,14 @@ Preconditions:- **Card and pill overflow (`F18.card`).** Run `control-openhands browser testids 'testid=skill-card-add-javadoc'` (name, `skill-source-add-javadoc` = `public`, `skill-toggle-add-javadoc`, description, pills, `skill-triggers-add-javadoc-overflow` labelled `Show 2 more`), `control-openhands browser click 'testid=skill-triggers-add-javadoc-overflow'` and `control-openhands browser text 'testid=skill-triggers-add-javadoc-overflow-popover'` (`java documentation`, `document java`); `control-openhands browser count 'testid=skill-detail-modal'` stays `0`. `control-openhands browser screenshot --feature F18.card --name overflow-popover`, then `control-openhands browser press Escape`.- **Keyboard open (`F18.card`).** Run `control-openhands browser focus 'testid=skill-card-docker'`, `control-openhands browser press Enter` and `control-openhands browser attr 'testid=skill-detail-modal' data-skill-name` (`docker`); `control-openhands browser press Escape`, focus again, `control-openhands browser press Space` and `control-openhands browser count 'testid=skill-detail-modal'` (`1`). Escape closes it.- **Copy the source path (`F18.copy-source`).** With the `qa-hello` fixture, `control-openhands browser attr 'testid=skill-source-qa-hello' title` is the full `…/.agents/skills/qa-hello/SKILL.md` path. Run `control-openhands browser click 'testid=skill-copy-source-qa-hello'`, `control-openhands browser attr 'testid=skill-copy-source-qa-hello' aria-label` (`Copied to clipboard`; `count 'testid=skill-detail-modal'` stays `0`), `control-openhands browser clipboard` (`text` is the same full `SKILL.md` path) and, after 2 s, the same `attr` (`Copy source path`). In the modal (`control-openhands browser click 'testid=skill-card-qa-hello'`) `control-openhands browser click 'testid=skill-modal-copy-source-qa-hello'` turns its label to `Copied to clipboard` and the modal stays open. `count 'testid=skill-copy-source-add-javadoc'` is `0`.-- **Toggle a built-in skill (`F18.toggle`).** Run `control-openhands browser attr 'testid=skill-toggle-add-javadoc' aria-checked` (`false`; `aria-label` is `Enable skill`), `control-openhands browser click 'testid=skill-toggle-add-javadoc'` (no modal opens; `aria-checked` turns `true` and the Enabled facet count goes up by one), wait a second, `control-openhands browser reload` and read `aria-checked` again: `true`. `control-openhands api GET /api/settings --pick misc_settings.app_preferences.enabled_skills` lists `add-javadoc` (`--pick` paths start inside `body`). To switch an enabled skill off from its card, `control-openhands browser hover 'testid=skill-toggle-add-javadoc'` first (`attr ... data-showing-remove` is `true`), then click (see Gotchas).-- **Toggle a personal skill (`F18.toggle-local`).** Run `control-openhands browser hover 'testid=skill-toggle-qa-hello'`, `control-openhands browser click 'testid=skill-toggle-qa-hello'`, wait a second, `control-openhands browser reload` and `control-openhands browser attr 'testid=skill-toggle-qa-hello' aria-checked` (`false`). `control-openhands api GET /api/settings` shows `disabled_skills: ["qa-hello"]` under `body.misc_settings.app_preferences` and `enabled_skills` without it. `control-openhands conversation start --prompt "qa-ping" --wait --timeout 240`, then `control-openhands api GET '/api/conversations/<id>/events/search?limit=5' --max-bytes 2000000` (id from `conversation start`): the user `MessageEvent` has `"activated_skills": []`. Back on `/skills` (`control-openhands browser goto /skills`; `conversation start` left the browser on the conversation), click the toggle on again (no hover needed), start another `qa-ping` conversation: `"activated_skills": ["qa-hello"]` and `control-openhands browser text 'testid=agent-message >> nth=-1'` is `QA-PONG-7731`; `disabled_skills` is `[]` again.-- **Skill reaches chat (`F18.skill-in-chat`).** Run `control-openhands browser goto /skills` (the previous bullet ends on a conversation), `control-openhands browser click 'testid=skill-toggle-flarglebargle'` (`aria-checked` `true`), then `control-openhands conversation start --prompt "flarglebargle" --wait --timeout 240`, `control-openhands conversation events <id> --kinds MessageEvent --from-start` (the user row has `skills: ["flarglebargle"]`; the `events/search?limit=5` read this bullet used before can miss the user row) and `control-openhands browser text 'testid=agent-message >> nth=-1'` (a reply praising how smart the user is). `control-openhands browser screenshot --feature F18.skill-in-chat --name enabled`. Switch it off again on `/skills` (`control-openhands browser goto /skills`, hover, click; `enabled_skills` no longer lists it) and start another conversation with `--prompt "flarglebargle. Do not run any tools; reply in one line."`: the user row has no `skills`, and the agent does not praise the user (2026-10-08: `Flarglebargle to you too — what would you like help with?`).+- **Toggle a built-in skill (`F18.toggle`).** Run `control-openhands browser attr 'testid=skill-toggle-add-javadoc' aria-checked` (`false`; `aria-label` is `Enable skill`), `control-openhands browser click 'testid=skill-toggle-add-javadoc'` (no modal opens; `aria-checked` turns `true` and the Enabled facet count goes up by one), wait a second, `control-openhands browser reload` and read `aria-checked` again: `true`. `control-openhands api GET /api/agent-profiles/default --pick profile.public_skills` lists `add-javadoc` (`browser network --filter /api/agent-profiles` shows the `POST /api/agent-profiles/default` `201` the click sent); `misc_settings.app_preferences.enabled_skills` in `control-openhands api GET /api/settings` is not changed by it. To switch an enabled skill off from its card, `control-openhands browser hover 'testid=skill-toggle-add-javadoc'` first (`attr ... data-showing-remove` is `true`), then click (see Gotchas).+- **Toggle a personal skill (`F18.toggle-local`).** Run `control-openhands browser hover 'testid=skill-toggle-qa-hello'`, `control-openhands browser click 'testid=skill-toggle-qa-hello'`, wait a second, `control-openhands browser reload` and `control-openhands browser attr 'testid=skill-toggle-qa-hello' aria-checked` (`false`). `control-openhands api GET /api/agent-profiles/default --pick profile.disabled_skills` is `["qa-hello"]` (settings `disabled_skills` stays `[]`). `control-openhands conversation start --prompt "qa-ping" --wait --timeout 240`, then `control-openhands api GET '/api/conversations/<id>/events/search?limit=5' --max-bytes 2000000` (id from `conversation start`): the user `MessageEvent` has `"activated_skills": []`. Back on `/skills` (`control-openhands browser goto /skills`; `conversation start` left the browser on the conversation), click the toggle on again (no hover needed), start another `qa-ping` conversation: `"activated_skills": ["qa-hello"]` and `control-openhands browser text 'testid=agent-message >> nth=-1'` is `QA-PONG-7731`; the profile's `disabled_skills` is `[]` again.+- **Skill reaches chat (`F18.skill-in-chat`).** Run `control-openhands browser goto /skills` (the previous bullet ends on a conversation), `control-openhands browser click 'testid=skill-toggle-flarglebargle'` (`aria-checked` `true`), then `control-openhands conversation start --prompt "flarglebargle" --wait --timeout 240`, `control-openhands conversation events <id> --kinds MessageEvent --from-start` (the user row has `skills: ["flarglebargle"]`; the `events/search?limit=5` read this bullet used before can miss the user row) and `control-openhands browser text 'testid=agent-message >> nth=-1'` (a reply praising how smart the user is). `control-openhands browser screenshot --feature F18.skill-in-chat --name enabled`. Switch it off again on `/skills` (`control-openhands browser goto /skills`, hover, click; the `default` profile's `public_skills` no longer lists it) and start another conversation with `--prompt "flarglebargle. Do not run any tools; reply in one line."`: the user row has no `skills`, and the agent does not praise the user (2026-10-08: `Flarglebargle to you too — what would you like help with?`).- **Project skill in a workspace (`F18.project-skill`).** With the committed `qa-f18-skill` fixture, run `control-openhands browser goto /skills`, `control-openhands browser reload` and `control-openhands browser count 'testid=skill-card-qa-f18-skill'`: `0` (the catalog lists the Agent Server's own skills, not a workspace's). Then `control-openhands conversation start --workspace qa-f18-repo --prompt "qa-f18-ping" --wait --timeout 240` (note `<repo-skill-id>`; its `workspace` is the fixture's path) and, in the chat it leaves open, `control-openhands browser text 'testid=generic-event-message-title'` (`Skill Ready`; with a model a second title can follow, because the agent may also call `invoke_skill`: the action's summary, such as `Load qa-f18-skill instructions`, or `Invoked skill qa-f18-skill` when it has none), `control-openhands browser click 'role=button[name="Expand"][exact] >> nth=0'` and `control-openhands browser count 'role=button[name="qa-f18-skill"]'` (`1`, under `Triggered Skill Knowledge:`); `control-openhands browser screenshot --feature F18.project-skill --name skill-ready`. With an LLM the agent also obeys the skill, and this is the moment to read it, while the chat is still open: `control-openhands browser text 'testid=agent-message >> nth=-1'` ends with `QA_SKILL_OK`; without a key the conversation ends in `error` right after the events below and that reply check is blocked (prerequisite: `DEEPSEEK_API_KEY`). Agent side, `control-openhands conversation events <repo-skill-id> --grep qa-f18-skill --from-start` matches the `SystemPromptEvent` (its `<available_skills>` lists `<name>qa-f18-skill</name>`) and the user `MessageEvent` (`skills: ["qa-f18-skill"]`, with excerpts showing `"activated_skills":["qa-f18-skill"]` and the `…/qa-f18-repo/.agents/skills/qa-f18-skill/SKILL.md` path). New Worktree: run `control-openhands workspace open qa-f18-repo --mode new_worktree` (it picks the mode in the preview bar's selector and prints `mode` `new_worktree` and `preview` `qa-f18-repo` / `New Worktree`; `control-openhands browser text 'testid=home-git-control-bar-preview'` reads the same two lines), then `control-openhands conversation start --stay --prompt "qa-f18-ping" --wait --timeout 240` (note `<worktree-id>`): its `workspace` is `/tmp/conversation-worktrees/<worktree-id>/qa-f18-repo`, a worktree of the fixture on branch `openhands/<worktree-id>` (read-only check: `git -C /tmp/conversation-worktrees/<worktree-id>/qa-f18-repo log --oneline` lists `Add qa-f18-skill skill` above `Initial fixture commit`), `control-openhands browser text 'testid=generic-event-message-title'` is `Skill Ready` again (and the same reply check applies here, with a key), and `control-openhands conversation events <worktree-id> --grep qa-f18-skill --from-start` matches the same events with the worktree's `SKILL.md` path (a `ConversationStateUpdateEvent` carrying the agent context matches in both conversations too). Restore the mode: `control-openhands workspace open qa-f18-repo --mode local_repo` (`mode` `local_repo`, `preview` `qa-f18-repo` / `Local Repo`).- **Deleted personal skill (`F18.skill-deleted`).** Run `control-openhands fixture skill --name qa-gone --trigger qa-farewell`, `control-openhands browser goto /skills`, `control-openhands browser reload`, `control-openhands browser count 'testid=skill-card-qa-gone'` (`1`; `control-openhands browser attr 'testid=skill-source-qa-gone' title` is the run's `…/private/home/.agents/skills/qa-gone/SKILL.md` path and `control-openhands browser text 'testid=skills-result-summary'` counts one result more than before) and `control-openhands browser screenshot --feature F18.skill-deleted --name present`. Prove it loads: `control-openhands conversation start --prompt "qa-farewell" --wait --timeout 240` (note `<gone-a>`), `control-openhands browser text 'testid=generic-event-message-title'` (`Skill Ready`) and `control-openhands conversation events <gone-a> --grep qa-gone --from-start`: the `SystemPromptEvent` and the user `MessageEvent` match, the latter with `skills: ["qa-gone"]`. Delete the folder (outside the UI; it is inside the run): `rm -rf "$OH_VERIFY_RUN/private/home/.agents/skills/qa-gone"`. Then `control-openhands browser goto /skills`, `control-openhands browser reload`, `control-openhands browser count 'testid=skill-card-qa-gone'` (`0`; the summary is back to its earlier count), `control-openhands conversation start --prompt "qa-farewell. Do not run any tools; reply in one line." --wait --timeout 240` (note `<gone-b>`; with a model, a bare `qa-farewell` sent deepseek-flash searching the disk for the word, and it quoted the run's `stack.log` line `Skill 'qa-gone' triggered by keyword 'qa-farewell'`, so `--grep qa-gone` matched 9 tool rows although the skill was not loaded), `control-openhands browser count 'text=Skill Ready'` (`0`; `control-openhands browser screenshot --feature F18.skill-deleted --name absent`) and `control-openhands conversation events <gone-b> --grep qa-gone --from-start`: `count` is `0`, and `control-openhands conversation events <gone-b> --kinds MessageEvent --from-start` shows the user message without a `skills` row. No model is needed: the skill set is fixed before the first model call.- **Both personal skill folders (`F18.personal-skill-dirs`).** The `qa-gone` bullet above used `~/.agents/skills`. For the other folder, write a fixture into the stack's persistence folder (outside the UI, inside the run; `fixture skill` writes only `~/.agents/skills`): `mkdir -p "$OH_VERIFY_RUN/private/skills/qa-legacy"` and a `SKILL.md` there with the front matter `name: qa-legacy`, `description: QA fixture skill in the persistence-dir skills folder.`, `triggers: [qa-legacy-ping]` and the body `When this skill is active, end your reply with the word QA_LEGACY_OK.` (`$OH_VERIFY_RUN/private` is the Agent Server's `OH_PERSISTENCE_DIR`, which replaces `~/.openhands`). Run `control-openhands browser goto /skills`, `control-openhands browser reload`, `control-openhands browser count 'testid=skill-card-qa-legacy'` (`1`) and `control-openhands browser attr 'testid=skill-source-qa-legacy' title` (`…/private/skills/qa-legacy/SKILL.md`). Then `control-openhands conversation start --prompt "qa-legacy-ping. Do not run any tools; reply in one line." --wait --timeout 240` (note `<legacy-id>`): `control-openhands browser text 'testid=generic-event-message-title'` is `Skill Ready`, `control-openhands conversation events <legacy-id> --kinds MessageEvent --from-start` shows the user row with `skills: ["qa-legacy"]`, and with a model `control-openhands browser text 'testid=agent-message >> nth=-1'` ends with `QA_LEGACY_OK` (`control-openhands browser screenshot --feature F18.personal-skill-dirs --name persistence-dir-skill`). Clean up with `rm -rf "$OH_VERIFY_RUN/private/skills/qa-legacy"`, `browser goto /skills`, `browser reload`; the card count is `0`.-- **Save failure (`F18.toggle-error`).** Run `control-openhands browser goto /skills`, wait for `testid=skill-toggle-add-javadoc` and note its `aria-checked` (`true` when this follows the toggle bullet, which left `add-javadoc` on). Then `control-openhands service stop agent-server`, `control-openhands browser click 'testid=skill-toggle-add-javadoc' --timeout 5000` (the card flips at once), wait about 2 s (the `PATCH /api/settings` is tried three times before the error shows; `browser network` lists three `502`s) and `control-openhands browser toasts --history`: `An error occurred` (the 502 body is not JSON, so the generic message shows; a background settings refetch can add a second identical toast). `control-openhands browser screenshot --feature F18.toggle-error --name toast`. Run `control-openhands restart --timeout 240`, `control-openhands browser goto /skills`: `aria-checked` on `testid=skill-toggle-add-javadoc` is back to the value noted before the click and `enabled_skills` is unchanged. Clear the expected 502 noise with `control-openhands browser errors --clear`.-- **Detail modal and switch (`F18.detail-modal`).** With `add-javadoc` on, run `control-openhands browser click 'testid=skill-card-add-javadoc'`, `control-openhands browser testids 'testid=skill-detail-modal'` (name, source, `skill-modal-enable-row-add-javadoc`, description, pills incl. license `MIT`, `skill-modal-field-content-add-javadoc`, `skill-detail-close`, `skill-detail-use-skill-add-javadoc`) and `control-openhands browser enabled 'testid=skill-detail-use-skill-add-javadoc'` (`true`). Switch off with `control-openhands browser click 'testid=skill-modal-enable-row-add-javadoc >> text=Enabled'`; `control-openhands browser eval "document.querySelector('[data-testid=skill-modal-toggle-add-javadoc]').checked"` is `false`, the row reads `Disabled`, `enabled` on Use skill is `false` and the card toggle behind reads `aria-checked` `false`. `control-openhands browser screenshot --feature F18.detail-modal --name disabled`. Close, reload: the card stays off and `enabled_skills` no longer lists `add-javadoc`.+- **Save failure (`F18.toggle-error`).** Run `control-openhands browser goto /skills`, wait for `testid=skill-toggle-add-javadoc` and note its `aria-checked` (`true` when this follows the toggle bullet, which left `add-javadoc` on). Then `control-openhands service stop agent-server`, `control-openhands browser click 'testid=skill-toggle-add-javadoc' --timeout 5000` (the card flips for a moment and reverts once the save fails), wait about 4 s and run `control-openhands browser toasts --history`: `An error occurred`, once per click (the 502 body is not JSON, so the generic message shows); `control-openhands browser network --filter /api/agent-profiles` lists `GET /api/agent-profiles/default` `502` and no `POST`, since the save reads the profile first. `control-openhands browser screenshot --feature F18.toggle-error --name toast`. Run `control-openhands restart --timeout 240`, `control-openhands browser goto /skills`: `aria-checked` on `testid=skill-toggle-add-javadoc` is back to the value noted before the click and the `default` profile's `public_skills` is unchanged. Clear the expected 502 noise with `control-openhands browser errors --clear`.+- **Detail modal and switch (`F18.detail-modal`).** With `add-javadoc` on, run `control-openhands browser click 'testid=skill-card-add-javadoc'`, `control-openhands browser testids 'testid=skill-detail-modal'` (name, source, `skill-modal-enable-row-add-javadoc`, description, pills incl. license `MIT`, `skill-modal-field-content-add-javadoc`, `skill-detail-close`, `skill-detail-use-skill-add-javadoc`) and `control-openhands browser enabled 'testid=skill-detail-use-skill-add-javadoc'` (`true`). Switch off with `control-openhands browser click 'testid=skill-modal-enable-row-add-javadoc >> text=Enabled'`; `control-openhands browser eval "document.querySelector('[data-testid=skill-modal-toggle-add-javadoc]').checked"` is `false`, the row reads `Disabled`, `enabled` on Use skill is `false` and the card toggle behind reads `aria-checked` `false`. `control-openhands browser screenshot --feature F18.detail-modal --name disabled`. Close, reload: the card stays off and the `default` profile's `public_skills` no longer lists `add-javadoc`.- **Close the modal (`F18.detail-close`).** With `testid=skill-card-docker` opened each time, each of `control-openhands browser click 'testid=skill-detail-modal-close'`, `control-openhands browser click 'testid=skill-detail-close'`, `control-openhands browser press Escape` and `control-openhands browser mouse-click 100 500` (backdrop) leaves `control-openhands browser count 'testid=skill-detail-modal'` at `0`.- **Pill overflow in the modal (`F18.detail-pills`).** Run `control-openhands browser click 'testid=skill-card-add-javadoc'`, `control-openhands browser text 'testid=skill-modal-pills-add-javadoc'` (at desktop: `Trigger-based`, `Code quality & review`, `MIT`, `+4`; license and compatibility appear only in the modal), `control-openhands browser attr 'testid=skill-modal-pills-add-javadoc-overflow' aria-label` (`Show 4 more` at desktop; `Show 5 more` at phone width), `control-openhands browser click 'testid=skill-modal-pills-add-javadoc-overflow'` and `control-openhands browser text 'testid=skill-modal-pills-add-javadoc-overflow-popover'` (`Requires Java source files`, `javadoc`, `java documentation`, `document java` at desktop; `MIT` joins them at phone width); `count 'testid=skill-detail-modal'` stays `1`. `control-openhands browser screenshot --feature F18.detail-pills --name popover`. Close the popover with `control-openhands browser click 'testid=skill-modal-description-add-javadoc'`; `control-openhands browser count 'testid=skill-modal-pills-add-javadoc-overflow-popover'` is `0` while the detail modal stays open. Reopen it with `control-openhands browser click 'testid=skill-modal-pills-add-javadoc-overflow'`. With focus on the overflow trigger, run `control-openhands browser record start --feature F18.detail-pills --name escape`, `control-openhands browser press Escape` and `control-openhands browser wait 'testid=skill-modal-pills-add-javadoc-overflow-popover' --state detached`. `control-openhands browser count 'testid=skill-detail-modal'` stays `1`; focus returns to the overflow trigger. Press Escape again and `control-openhands browser wait 'testid=skill-detail-modal' --state detached`, then `control-openhands browser record stop`. Repeat this modal/pill sequence at `browser viewport phone`, then restore `browser viewport desktop`.- **Use skill (`F18.use-skill`).** Run `control-openhands browser click 'testid=skill-card-docker'`, `control-openhands browser click 'testid=skill-detail-use-skill-docker' --expect-url '/conversations(\?|$)'`, then `control-openhands browser eval "document.querySelector('[data-testid=chat-input]').innerText"`: `/docker ` (the skill name as a slash command), with the modal gone (`control-openhands browser count 'testid=skill-detail-modal'` is `0`) and the composer focused (`control-openhands browser eval "document.activeElement.getAttribute('data-testid')"` is `chat-input`). `control-openhands browser screenshot --feature F18.use-skill --name prefilled` shows `/docker` in the Home composer. Nothing is sent, and the text is applied once: leave and come back without a page load (`control-openhands browser click 'testid=sidebar-automations-link' --expect-url '/automations$'`, then `control-openhands browser click 'testid=sidebar-conversations-link' --expect-url '/conversations(\?|$)'`) and the same `chat-input` eval returns an empty string. A `browser goto /` would prove nothing here: a page load empties the composer either way.@@ -89,7 +93,7 @@ Preconditions:- **Colliding skill names do not leave ghost cards (`F18.card`, `F18.facets`).** Arrange `fixture skill --name skill-creator --trigger qa-collision-ping` in the run's isolated personal folder beside the bundled same-name skill, plus `qa-toggle`. Reload `/skills`, search `skill-creator`, then search `qa-toggle`; repeat the two searches. Expected: only the one matching toggle card and no `skill-card-skill-creator`. Known failure, reproduced 2026-10-12 ([#18287](https://github.com/OpenHands/OpenHands/issues/18287)): `skills-result-summary` says `1 result(s)` while two then three stale same-source skill-creator cards remain. Read their source titles and screenshot the mismatched list. Remove only the own colliding fixture and reload to clear ghosts.-- **Skill state survives a delayed sample (`F18.toggle`).** With an own enabled `qa-toggle` fixture, use its real switch with `--hover-first 250` when removing; drive disabled → enabled → disabled, observe for eight seconds and read `aria-checked`, settings `disabled_skills` and a reload. Expected: the switch stays false and the server keeps `qa-toggle` disabled across reload. Repeat the cycle, then restore enabled; a bounded sample does not establish every possible delay.+- **Skill state survives a delayed sample (`F18.toggle`).** With an own enabled `qa-toggle` fixture, use its real switch with `--hover-first 250` when removing; drive disabled → enabled → disabled, observe for eight seconds and read `aria-checked`, the `default` agent profile's `disabled_skills` and a reload. Expected: the switch stays false and the server keeps `qa-toggle` disabled across reload. Repeat the cycle, then restore enabled; a bounded sample does not establish every possible delay.- **Backend switch refreshes skill data (`F18.page`).** Use Local and the disposable `QA_SCOPE` identity of the own stack described in F17. Load/reload the Local Skills page once, then arrange a new personal fixture `qa-scope-new-3`; clear the skills network log. Switch to QA_SCOPE through the actual selector and inspect the still-mounted Skills page, with no `goto` or reload between the switch and observation. Expected: a skills request for the selected backend and the new card. Known failure, reproduced twice 2026-10-12 ([#16893](https://github.com/OpenHands/OpenHands/issues/16893)): zero `/api/skills` requests and card count 0; a subsequent reload requests actual data and shows the card (1). Repeat in the other direction with a new marker `qa-scope-new-4`. Two registry identities / real fixture changes share one physical run here; separate Cloud data and conversation-scoped caches are further branches. Restore Local and remove only the fixtures/alias.@@ -98,7 +102,7 @@ Preconditions:## Gotchas- `sidebar-skills-link` is labelled **Customize** and lands on `/mcp` on desktop (`/customize` redirects); Skills is one more click. Test ids embed the route: quote `'testid=sidebar-extensions-/skills'`.-- A `browser click` on an **enabled** card toggle is lost when the pointer was not already over it: `pointerenter` swaps the checkmark icon for the X icon between mousedown and mouseup, so Chrome fires no click and no `PATCH /api/settings` goes out (#17941). `browser hover` the toggle first (or click twice). Turning a skill on is not affected. Recorded as `fail` on `F18.toggle`.+- A `browser click` on an **enabled** card toggle is lost when the pointer was not already over it: `pointerenter` swaps the checkmark icon for the X icon between mousedown and mouseup, so Chrome fires no click and no save request goes out (#17941). `browser hover` the toggle first (or click twice). Turning a skill on is not affected. Recorded as `fail` on `F18.toggle`.- The card toggle and the facet rows are `role=switch`/`role=checkbox` buttons: read `aria-checked`. The modal switch is an `sr-only` checkbox (keyboard reachable): click the row text and read `.checked` with `browser eval`.- Typing writes `?q=` with `replace` after a 300 ms debounce, so read `browser url` a second later; facet clicks push a history entry immediately.- Facet counts are disjunctive: a row counts skills matching the other groups' selections, so Disabled reads `0` (and is disabled) while Recommended is checked.diff --git a/.agents/skills/verify-openhands/references/feature-map/F21-automations-dashboard.md b/.agents/skills/verify-openhands/references/feature-map/F21-automations-dashboard.mdindex bc14cf2880..241d5e5b5e 100644--- a/.agents/skills/verify-openhands/references/feature-map/F21-automations-dashboard.md+++ b/.agents/skills/verify-openhands/references/feature-map/F21-automations-dashboard.md@@ -27,7 +27,7 @@ Source: `src/routes/automations-list.tsx`, `src/components/features/automations/- `F21.list-error`: when the list request fails with nothing cached, the page shows **Failed to load automations** with Retry.- `F21.add-menu`: the header **Add Automation** menu offers **Create Automation** and **Import automation**; Create Automation opens the guided setup page at `/automations/setup` (F22).- `F21.empty-state`: with zero automations and zero saved drafts the page shows **No automations configured**, the "Create an automation" instructions, a docs link, **Create Automation** (opens `/automations/setup`, F22, without creating a conversation) and the Recommended automations rail; the view toggle is disabled.-- `F21.empty-rail`: a Recommended automations card on the empty dashboard opens its template setup at `/automations/new/<template-id>` (F22 owns the setup flow).+- `F21.empty-rail`: a Recommended automations card on the empty dashboard opens its template setup at `/automations/new/<template-id>` (F22 owns the setup flow). This rail has no dismiss X and stays shown when Home's rail was hidden (F03.recommended-rail-dismiss).- `F21.overview-tiles`: four tiles summarize the automations the groups list from the loaded org list (hidden DRAFT test automations from a draft Test are left out): Automations (`N active`), Needs attention (`Latest run failed` or `No latest-run failures`; an enabled automation counts when its latest run is Failed or its task outcome is `failed` or `blocked`, while Partial and Needs review do not count), Total runs and Average duration.- `F21.search`: the search box narrows the visible automations client-side by name, prompt, repository or model, and the saved drafts by name, prompt, kind or trigger; the tiles do not change.- `F21.filters`: the **Filters** popover filters by status (All / Active / Needs attention / Disabled) and trigger (All / Scheduled / Event-driven), counts non-default choices in a badge and offers **Reset all**; it stays open while the user changes several choices.@@ -88,7 +88,7 @@ Preconditions:- **Missing manifest (`F21.manifest-404`).** Use an isolated worktree at the frozen target and a private copy of its pinned `@openhands/extensions` package. Remove only the `AUTOMATION_INTERFACE` export from the copied `automations/index.js` before `npm run build:app`; leave Canvas source and the lockfile unchanged. Serve that frontend against an owned healthy Local backend. Open `/` and `/automations`: the sidebar Automate link should be absent, `/automations` should render 404, and Home should have no **Schedule a task** link, recommendation section, or no-manifest error toast. On frozen `61a3bd93` (2026-10-17), an isolated private copy of pinned `@openhands/[email protected]` with only that runtime export removed reproduced the route/sidebar checks on desktop and phone, but Home retained the dead checklist link (visible in the phone drawer) and a visible `No automation interface manifest is admitted` toast ([#18210](https://github.com/OpenHands/OpenHands/issues/18210); [draft fix PR #18212](https://github.com/OpenHands/OpenHands/pull/18212)). Record the alternate build as fault injection; the unmodified pinned package does include the manifest. Stop the private frontend and remove only this test worktree/build afterward.- **Empty state (`F21.empty-state`).** With no automations run `control-openhands browser goto /automations`, `control-openhands browser text 'testid=automations-empty'`, `control-openhands browser enabled 'testid=automations-view-toggle'` and `control-openhands browser screenshot --feature F21.empty-state --name empty`. The text starts `No automations configured`, contains `Create an automation`, `Learn more in the documentation` and `Create Automation`, and lists the Recommended automations; `enabled` is `false`; the tiles read `Automations 0 / 0 active` and `Needs attention 0 / No latest-run failures`. `control-openhands browser click 'testid=automations-empty >> testid=automations-create-automation' --expect-url '/automations/setup'` opens the setup page (form plus docked composer, F22) and `control-openhands conversation list` is unchanged (no conversation until the user sends a prompt). The empty-state copy still says "Start a new conversation and tell OpenHands to Create an automation".- **Add Automation menu (`F21.add-menu`).** Run `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser snapshot 'testid=automations-add-automation-menu'`: buttons `Create Automation` and `Import automation`. `control-openhands browser press Escape` (or a click on `role=heading[name="Dashboard"]`) makes `browser count 'testid=automations-add-automation-menu'` `0`. Reopen it and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`: the agent-assisted setup page opens. Go back to `/automations`, reopen the menu, and `control-openhands browser click 'testid=automations-import-automation'`: the import modal is covered by F22.-- **Recommended rail on the empty dashboard (`F21.empty-rail`).** With no automations, `control-openhands browser click 'testid=automations-empty >> testid=recommended-automation-rail-card-custom-automation' --expect-url 'automations/new'` lands on `/automations/new/custom-automation` with the **Before you start** prerequisites dialog (F22). `control-openhands browser back` returns to `/automations`; there `browser attr 'testid=recommended-automations-rail-fade-left' data-visible` is `false` and `...-fade-right` is `true` (scrolling is F03.recommended-automations-rail).+- **Recommended rail on the empty dashboard (`F21.empty-rail`).** With no automations, `control-openhands browser click 'testid=automations-empty >> testid=recommended-automation-rail-card-custom-automation' --expect-url 'automations/new'` lands on `/automations/new/custom-automation` with the **Before you start** prerequisites dialog (F22). `control-openhands browser back` returns to `/automations`; there `browser attr 'testid=recommended-automations-rail-fade-left' data-visible` is `false` and `...-fade-right` is `true` (scrolling is F03.recommended-automations-rail). With Home's rail hidden (`browser eval "localStorage.getItem('openhands.home_recommended_automations_dismissed')"` is `true`), `browser count 'testid=automations-empty >> testid=recommended-automations-rail'` is still `1` and `browser count 'testid=recommended-automations-rail-dismiss'` is `0`.- **Saved drafts (`F21.saved-drafts`, `F21.search`).** Arrange one draft through the setup page (F22's **Open and save a draft**: name `QA_F22 Delta Draft`, prompt `Reply pong. Do not run any tools.`, custom cron `0 0 1 1 *`, Save draft), then `control-openhands browser goto /automations`. `control-openhands browser text 'role=main'` lists `Drafts 1` / `QA_F22 Delta Draft` above the groups, and `control-openhands browser testids 'role=main' --filter draft` lists `automation-setup-draft-<draft-id>`, `-resume-`, `-open-`, `-edit-` (`Edit`), `-test-` (`Test draft`) and `-delete-` (`Delete draft`). `control-openhands browser tooltip 'testid=automation-setup-draft-test-<draft-id>'` is `Test Run`; clicking it toasts `Test run started.` and the draft (`api GET /api/automation/v1/drafts/<draft-id>`) gains `materialized_automation_id` and `last_test_run_id`; that automation has `state` `DRAFT`, is not listed in any group, and its detail page shows the Draft badge (F23.header). Once its run is `COMPLETED` (poll `api GET '/api/automation/v1/<materialized-id>/runs?limit=10'`), `control-openhands browser reload` leaves the tiles at `Automations 0 / 0 active`, `Total runs 0` and `Average duration —` (`browser text 'testid=overview-tile-total-runs'`): the test automation is not counted. `testid=automation-setup-draft-resume-<draft-id>` and `testid=automation-setup-draft-edit-<draft-id>` each open `/automations/setup?draftId=<draft-id>` (`--expect-url '/automations/setup'`) with the saved values. Search narrows drafts too: filling `Delta` leaves 0 rows and 1 draft; `zzqa-nomatch` shows the filtered empty state with no draft. Delete: click `testid=automation-setup-draft-delete-<draft-id>`; `control-openhands browser snapshot 'role=dialog[name="Delete draft?"]'` shows `button "Close"`, heading `Delete draft?`, `Delete “QA_F22 Delta Draft”? This removes the saved setup draft, but does not delete any final automations.`, `Cancel` and `Delete draft`, and `control-openhands browser eval "document.activeElement.textContent"` is `Cancel`; `browser press Tab` (repeated) cycles Delete draft, Close and Cancel without leaving the dialog. `control-openhands browser press Escape` makes `browser count 'role=dialog[name="Delete draft?"]'` `0` with the draft kept and focus back on `automation-setup-draft-delete-<draft-id>`; reopening and closing with `'role=dialog[name="Delete draft?"] >> role=button[name="Close"]'`, `... role=button[name="Cancel"]` and a backdrop click (`control-openhands browser mouse-click 1400 950`) each give `0` too. At phone width the confirm button is `insideViewport` (`browser bbox 'testid=automation-setup-draft-delete-confirm'`). `control-openhands browser click 'testid=automation-setup-draft-delete-confirm'` toasts `Draft deleted.`; drafts `total` is `0` and the materialized test automation answers `404`.- **Arrange two automations (import, F22).** Run `control-openhands browser click 'testid=automations-add-automation'`, `control-openhands browser click 'testid=automations-import-automation'`, `control-openhands browser upload 'testid=automations-import-file' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f21-alpha.automation.json"`, `control-openhands browser click 'testid=import-automation-confirm'`, `control-openhands browser wait 'testid=import-automation-modal' --state detached`. Repeat with `qa-f21-beta.automation.json`. Both arrive disabled, under **Inactive 2**.- **Grid cards (`F21.card-grid`).** Run `control-openhands browser reload`, `control-openhands browser snapshot 'role=main'` and `control-openhands browser screenshot --feature F21.card-grid --name grid`. Each card shows its name, prompt, pills (`pull_request.opened` + GitHub logo for Beta, `cron` for Alpha), **No activity yet**, and `Runs 0 / Recent success — / Avg. duration —`; both Play buttons are `[disabled]`.diff --git a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.mdindex dca0557690..a8ea4b9a5b 100644--- a/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md+++ b/.agents/skills/verify-openhands/references/feature-map/F22-automation-creation.md@@ -2,10 +2,11 @@A user adds an automation in three ways. The **Templates** sub-page(`/automations/templates`) lists the prebuilt catalog from-`@openhands/extensions` (6 featured, 17 Beta) with search; picking a card-either opens a manifest-driven setup dialog at `/automations/new/<id>`+`@openhands/extensions` (6 featured, 17 Beta) with search and integration and+trigger filter chips; picking a card either opens a manifest-driven setup dialog at `/automations/new/<id>`(prerequisites → form → review → **Confirm and create**), or first asks where a-GitHub/Slack responder should run, or first asks the user to install missing+GitHub/Slack responder should run (on OpenHands Cloud, whether to use the+built-in integration instead), or first asks the user to install missingMCP integrations, or seeds a conversation with the entry's launch prompt.**Add Automation → Create Automation** opens the agent-assisted setup page at`/automations/setup`, where the form opens before a conversation exists. Saving a server-backed@@ -14,17 +15,19 @@ starts genuine model assistance through `automation_form_update`. The setup pagedraft validation, Test and Create each have their own IDs in F28. **Add Automation → Import automation** readsan exported JSON file, previews it and creates it disabled.-Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-route.tsx`, `src/routes/automation-setup-new.tsx`, `src/components/features/automations/` (`recommended-automations-launcher.tsx`, `recommended-automations-section.tsx`, `responder-deployment-modal.tsx`, `create-instructions.tsx`, `import-automation-modal.tsx`, `agent-profile-selector.tsx`, `setup/`), `src/components/features/conversation/conversation-main/conversation-main.tsx`, `src/components/features/manifest/`, `src/manifests/`, `src/hooks/use-start-automation-setup.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/hooks/use-responder-url-secret.ts`, `src/utils/automation-export.ts`, `src/utils/automation-catalog.ts`, `src/api/automation-service/automation-service.api.ts`, `src/api/automation-form-client-tool.ts`, `src/api/automation-form-session.ts`, `src/services/automation-form.ts`, `src/constants/automation-form.ts`, `node_modules/@openhands/extensions/automations/`.+Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-route.tsx`, `src/routes/automation-setup-new.tsx`, `src/components/features/automations/` (`recommended-automations-launcher.tsx`, `recommended-automations-section.tsx`, `template-filter-chips.tsx`, `built-in-integration-choice-modal.tsx`, `responder-deployment-modal.tsx`, `create-instructions.tsx`, `import-automation-modal.tsx`, `agent-profile-selector.tsx`, `setup/`), `src/components/features/conversation/conversation-main/conversation-main.tsx`, `src/components/features/manifest/`, `src/manifests/`, `src/hooks/use-start-automation-setup.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/hooks/use-responder-url-secret.ts`, `src/utils/automation-export.ts`, `src/utils/automation-catalog.ts`, `src/api/automation-service/automation-service.api.ts`, `src/api/automation-form-client-tool.ts`, `src/api/automation-form-session.ts`, `src/services/automation-form.ts`, `src/constants/automation-form.ts`, `node_modules/@openhands/extensions/automations/`.## Sub-features-- `F22.templates-page`: `/automations/templates` shows the h1 **Templates**, a description, a search box, **Start from a proven workflow** with a count badge (6) and a **Beta** section (17).-- `F22.template-card`: each card shows an icon or logo stack, name, category, two-line description, a plus badge and integration pills (`N MCPs to connect before launch`; `Needs external setup` only for an integration with no installable MCP or native connection, which no current catalog entry has; `Connected` once installed, which needs a working integration and was not reached here); a `+N` overflow button opens a popover without launching the card.-- `F22.templates-search`: the search box narrows cards by name, category, description, launch prompt and integration names; a query that matches nothing should say so.+- `F22.templates-page`: `/automations/templates` shows the h1 **Templates**, a description, a search box, the **Filter templates** chips, **Start from a proven workflow** with a count badge (6) and a **Beta** section (17).+- `F22.template-card`: each card shows an icon or logo stack, name, category, two-line description, one trigger-kind badge per way it runs (a clock for **Cron**, a bolt for **Event driven**, named in a tooltip), a plus badge and integration pills (`N MCPs to connect before launch`; `Needs external setup` only for an integration with no installable MCP or native connection, which no current catalog entry has; `Connected` once installed, which needs a working integration and was not reached here); a `+N` overflow button opens a popover without launching the card.+- `F22.templates-search`: the search box narrows cards by name, category, description, launch prompt and integration names; a query (or filter) that matches nothing shows `No matching templates found — check back again soon.` with **Clear filters**, which empties the search and resets the chips.+- `F22.template-filters`: the **Filter templates** chips above the grid are **All**, one chip per integration the offered templates use (their declared integrations plus the provider of a repository picker in their setup form) and **Cron** / **Event driven**. One integration and one trigger kind can be active at once and combine; clicking an active chip clears it and **All** clears both.- `F22.template-launch-setup`: a card whose entry ships a setup form (custom-automation, news-digest, github-pr-reviewer, qa-changes, …) navigates to `/automations/new/<id>`.- `F22.template-launch-conversation`: a card without a setup form and with no installable integration missing creates a conversation and pre-fills its composer with the entry's launch command (for example `/ticket-to-code-change:setup`); nothing is sent until the user presses send.- `F22.template-mcp-install-queue`: a card with missing required integrations opens each one's install dialog in turn; Cancel abandons the launch without creating anything.- `F22.responder-deployment-choice`: on a local backend a GitHub/Slack-only responder card first asks **Choose how your responder should run**; **Continue with local setup** creates the `OPENHANDS_URL` secret and continues, **Open OpenHands Cloud integrations** opens Cloud in a new tab, X and Escape cancel.+- `F22.built-in-integration-choice`: on an OpenHands Cloud backend whose built-in integration covers a template (GitHub issue to PR, GitLab issue to MR, the Jira issue-to-PR templates, Slack channel monitor), the card first opens **Choose your setup**. When the user has connected that integration it reads `You can use OpenHands in <name> today — here's an example:` with an example message and **Got it**; otherwise **Use Built-In <name> Integration** offers **Connect to <name>**. **Setup Polling Automation** / **Continue with polling setup** is the alternative in both states. A disabled integration, and every local backend, skips the dialog.- `F22.responder-local-launch`: with the responder's integration already installed (Slack for **Slack standup digest**), **Continue with local setup** skips the install dialog and opens a new conversation whose composer is pre-filled with the entry's setup command (`/standup-digest:setup`). Nothing is sent until the user presses send; Send delivers it as the user message.- `F22.setup-prerequisites`: **Before you start** lists the entry's integrations with the reason each is needed and **Manage integrations** (`/mcp`); Continue is disabled while a required integration is missing; an entry with nothing to check skips the step.- `F22.setup-action-kind`: the **Action** combobox (custom-automation: Prompt / Plugin / Upload tarball) swaps the action fields and keeps shared values.@@ -39,7 +42,7 @@ Source: `src/routes/automation-templates.tsx`, `src/routes/automation-setup-rout- `F22.setup-fallback-conversation`: **Set up in a conversation** creates a conversation seeded with the entry's setup message.- `F22.setup-close`: X or Escape goes back in history; on a cold deep link it replaces the location with `/automations`.- `F22.setup-unknown-id`: `/automations/new/<id>` for an id no catalog entry claims is a 404.-- `F22.agent-assisted-setup`: **Add Automation → Create Automation** opens `/automations/setup`, whose setup panel collects name, action, trigger, model/profile and custom code, can ask the agent for help, saves server-backed drafts, opens test runs behind a Back button and dispatches the finished automation. Expected: in create mode a Prompt or Plugin draft is not offered an agent profile its endpoint rejects, as the catalog dialog's Agent profile field already behaves, so Save draft never fails on one. Known failure (reproduced 2026-10-17, [#18569](https://github.com/OpenHands/OpenHands/issues/18569)): the agent profile pill under the Prompt (`automation-setup-agent-profile`) lists **Active profile** and every saved profile for Prompt and Plugin drafts; after picking a saved profile, Save draft sends `agent_profile_id` to `/v1/preset/prompt` or `/v1/preset/plugin`, the drafts API answers `422`, the toast reads `Extra inputs are not permitted` and the save state **Save failed**. **Active profile** saves.+- `F22.agent-assisted-setup`: **Add Automation → Create Automation** opens `/automations/setup`, whose setup panel collects name, action, trigger, model (LLM profile) and custom code, can ask the agent for help, saves server-backed drafts, opens test runs behind a Back button and dispatches the finished automation. A Prompt or Plugin draft offers no agent profile (their `/v1/preset/prompt` and `/v1/preset/plugin` endpoints take none), so Save draft sends no `agent_profile_id` and succeeds even when other agent profiles are saved.- `F22.import-picker`: **Add Automation → Import automation** opens a dialog that explains the format (docs link), says the import is disabled, and offers a dropzone and **Choose file**.- `F22.import-validation`: a file that is not JSON shows the toast `The selected file is not valid JSON.`; a JSON file that breaks the format shows one toast listing every issue; the dialog stays on the picker.- `F22.import-preview`: a valid file switches the dialog to a preview (Name, Trigger, Prompt, Plugins) with the disabled notice, **Cancel** and **Import automation**.@@ -68,11 +71,13 @@ Preconditions:`control-openhands fixture file --name qa-f22-schema.json --content '{"version":2,"kind":"automation","spec":{"name":"","trigger":{"type":"event"},"enabled":"yes","prompt":"x"}}'`.- Every automation created here uses the yearly cron `0 0 1 1 *`, so none fires during the run.- Tarball fixture (arrange only): `control-openhands fixture tarball --name qa-f22-tarball` writes `$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-tarball.tar.gz` (one `main.py` that prints `pong`).+- Blocked here: `F22.built-in-integration-choice` needs an OpenHands Cloud backend with a built-in GitHub, GitLab, Jira or Slack integration enabled (and, for the connected state, linked to the account). On a local backend the Slack channel monitor card opens the responder choice instead, which the **Responder choice** bullet checks.- Blocked here: completing `F22.template-mcp-install-queue` (and launching a card whose only missing integration is installable, such as `research-brief-writer`) needs a real Slack, Linear or Tavily credential: the install dialog tests the connection before saving.-- **Templates page (`F22.templates-page`).** From `/` run `control-openhands browser click 'testid=sidebar-automations-link' --expect-url '/automations$'`, `control-openhands browser click 'testid=automations-navbar-desktop >> testid=automations-navigation-templates' --expect-url '/automations/templates'`, `control-openhands browser text 'role=main >> role=heading[level=1]'` (`Templates`), `control-openhands browser count '[data-testid^="recommended-automation-card-"]'` (`23`) and `control-openhands browser count 'testid=recommended-automations-beta-section >> [data-testid^="recommended-automation-card-"]'` (`17`); `browser text 'testid=recommended-automations-beta-heading'` is `Beta` and `17`. `control-openhands browser snapshot 'role=main' --max-lines 60` shows `heading "Start from a proven workflow"` with the count `6`. Take `control-openhands browser screenshot --feature F22.templates-page --name grid`. `control-openhands browser goto /automations/templates` renders the same page.-- **Cards and pills (`F22.template-card`).** Run `control-openhands browser testids 'testid=recommended-automation-card-research-brief-writer' --hidden`: `recommended-automation-icon-…`, `recommended-automation-plus-…`, `recommended-automation-pills-…` and `recommended-automation-pills-research-brief-writer-overflow` (`Show 1 more`). `control-openhands browser count '[data-testid^="automation-integration-external-"]'` is `0`: Jira and Bitbucket connect through the installable Atlassian Rovo MCP, so `jira-issue-to-bitbucket-pr` shows `Atlassian Rovo`, `Bitbucket` and `2 MCPs to connect before launch`. `control-openhands browser click 'testid=recommended-automation-pills-research-brief-writer-overflow'` opens a popover (`browser snapshot 'role=dialog'`: `dialog: 2 MCPs to connect before launch`) and the URL stays `/automations/templates`; `control-openhands browser press Escape` closes it (`browser count 'role=dialog'` `0`).-- **Search (`F22.templates-search`).** Run `control-openhands browser fill 'role=main >> placeholder=Search automations...' tavily` and `control-openhands browser testids 'role=main' --filter recommended-automation-card`: only `recommended-automation-card-research-brief-writer` (matched through its Tavily integration). Then `control-openhands browser fill 'role=main >> placeholder=Search automations...' qa-no-such-template`, `control-openhands browser count 'testid=recommended-automations-section'` and `control-openhands browser screenshot --feature F22.templates-search --name no-match`. Expected: a "no templates match" message. Actual: the count is `0` and `browser text 'role=main'` is only the title and description; the area under the search box is blank (recorded as `fail`). Clear with `control-openhands browser fill 'role=main >> placeholder=Search automations...' ''` (23 cards again).+- **Templates page (`F22.templates-page`).** From `/` run `control-openhands browser click 'testid=sidebar-automations-link' --expect-url '/automations$'`, `control-openhands browser click 'testid=automations-navbar-desktop >> testid=automations-navigation-templates' --expect-url '/automations/templates'`, `control-openhands browser text 'role=main >> role=heading[level=1]'` (`Templates`), `control-openhands browser count '[data-testid^="recommended-automation-card-"]'` (`23`) and `control-openhands browser count 'testid=recommended-automations-beta-section >> [data-testid^="recommended-automation-card-"]'` (`17`); `browser text 'testid=recommended-automations-beta-heading'` is `Beta` and `17`. `control-openhands browser snapshot 'role=main' --max-lines 60` shows `group "Filter templates"` (buttons `All` [pressed], `GitHub`, `Slack`, `Linear`, `GitLab`, `Bitbucket`, `Atlassian Rovo`, `Tavily`, `Notion`, `Cron`, `Event driven`) and `heading "Start from a proven workflow"` with the count `6`. Take `control-openhands browser screenshot --feature F22.templates-page --name grid`. `control-openhands browser goto /automations/templates` renders the same page.+- **Cards and pills (`F22.template-card`).** Run `control-openhands browser testids 'testid=recommended-automation-card-research-brief-writer' --hidden`: `recommended-automation-icon-…`, `recommended-automation-trigger-research-brief-writer-cron`, `recommended-automation-plus-…`, `recommended-automation-pills-…` and `recommended-automation-pills-research-brief-writer-overflow` (`Show 1 more`). `control-openhands browser count '[data-testid^="automation-integration-external-"]'` is `0`: Jira and Bitbucket connect through the installable Atlassian Rovo MCP, so `jira-issue-to-bitbucket-pr` shows `Atlassian Rovo`, `Bitbucket` and `2 MCPs to connect before launch`. `control-openhands browser click 'testid=recommended-automation-pills-research-brief-writer-overflow'` opens a popover (`browser snapshot 'role=dialog'`: `dialog: 2 MCPs to connect before launch`) and the URL stays `/automations/templates`; `control-openhands browser press Escape` closes it (`browser count 'role=dialog'` `0`). Trigger badges: `control-openhands browser count '[data-testid^="recommended-automation-trigger-"][data-testid$="-cron"]'` is `22` and `... -event"]'` is `8`; seven templates carry both (Custom automation among them) and only QA changes has no Cron badge, and `control-openhands browser tooltip 'testid=recommended-automation-trigger-custom-automation-event'` reads `Event driven`.+- **Search (`F22.templates-search`).** Run `control-openhands browser fill 'role=main >> placeholder=Search automations...' tavily` and `control-openhands browser testids 'role=main' --filter recommended-automation-card`: only `recommended-automation-card-research-brief-writer` (matched through its Tavily integration). Then `control-openhands browser fill 'role=main >> placeholder=Search automations...' qa-no-such-template` and `control-openhands browser text 'testid=automations-filtered-empty'`: `No matching templates found — check back again soon.` and `Clear filters`; take `control-openhands browser screenshot --feature F22.templates-search --name no-match`. `control-openhands browser click 'testid=automations-clear-filters'` empties the search box (`browser value 'role=main >> placeholder=Search automations...'` is `''`) and brings back all 23 cards.+- **Filter chips (`F22.template-filters`, `F22.templates-search`).** Count with `control-openhands browser count '[data-testid^="recommended-automation-card-"]'` after each click (`browser testids` lists only on-screen cards). `control-openhands browser click 'testid=template-filter-integration-github'`: `12`, and `browser attr 'testid=template-filter-integration-github' aria-pressed` is `true` while `template-filter-all` is `false`. `control-openhands browser click 'testid=template-filter-trigger-event'`: `4` (GitHub code review, Custom automation, Linear issue to GitHub PR, QA changes). Clicking `testid=template-filter-integration-github` again clears only that chip: `8` (every Event driven template). `testid=template-filter-trigger-cron` replaces Event driven (`22`; `template-filter-trigger-event` `aria-pressed` `false`). Click `testid=template-filter-trigger-cron` again to clear it; then `testid=template-filter-integration-slack` alone gives `3` (Slack channel monitor, Slack standup digest, Incident retrospective drafter). `testid=template-filter-integration-tavily` plus `testid=template-filter-trigger-event` matches nothing: the same `automations-filtered-empty` message appears under the chips (`browser screenshot --feature F22.template-filters --name no-match`), and `testid=automations-clear-filters` resets them (`template-filter-all` `aria-pressed` `true`, `23` cards). Finish with `control-openhands browser click 'testid=template-filter-all'`.- **Card opens setup, prerequisites (`F22.template-launch-setup`, `F22.setup-prerequisites`).** Run `control-openhands browser click 'testid=recommended-automation-card-custom-automation' --expect-url '/automations/new/custom-automation'` and `control-openhands browser snapshot 'testid=setup-dialog'`: heading `Before you start`, `GitHub Optional. Used when you choose a GitHub repository…`, link `Manage integrations` (`/mcp`) and `Continue`; `control-openhands browser enabled 'testid=setup-continue-button'` is `true` (an optional integration only warns). Run `control-openhands browser click 'testid=setup-continue-button'`; the heading becomes `Custom automation` and `browser testids 'testid=setup-dialog'` lists `setup-action-kind`, `setup-field-schedule` (`0 9 * * *`), `setup-field-timezone` (`UTC`), `setup-field-name`, `setup-field-model`, `setup-field-timeout`, `setup-field-prompt`, `setup-field-repository` and `setup-field-ref`. There is no Trigger selector here because this deployment only supports `cron`, and no `automation-agent-profile` because the default Prompt action cannot take one (`F22.setup-agent-profile`).- **Validation (`F22.setup-form-validation`).** Click `testid=setup-continue-button` with the form untouched: `control-openhands browser text 'testid=setup-field-name-error'` and `... 'testid=setup-field-prompt-error'` are `This field is required.` Fill `testid=setup-field-timeout` with `5000`, `testid=setup-field-name` with `QA_F22 Custom`, `testid=setup-field-prompt` with `Reply with the single word: pong. Do not run any tools.`, click Continue: `browser text 'testid=setup-field-timeout-error'` is `Must be at most 1800.` Fill the timeout with `60`. Service preflight: `control-openhands browser fill 'testid=setup-field-schedule' 'every day'`, `control-openhands browser press Tab`, `control-openhands browser wait 'testid=setup-field-schedule-error' --timeout 5000`, then `browser text 'testid=setup-field-schedule-error'`: `Value error, Invalid cron expression: every day` (from `POST /api/automation/v1/validate`, see `browser network --last 5`). Fill the schedule with `0 0 1 1 *`, press Tab, and `control-openhands browser wait 'testid=setup-field-schedule-error' --state detached --timeout 5000` succeeds.- **Setup form colours (`F22.setup-form-validation`, #18330).** In the custom-automation setup dialog, the required and service errors read `rgb(231, 106, 94)` on OpenHands-Neutral and `rgb(209, 52, 56)` (field errors) / `rgb(192, 40, 45)` (prompt error) on white under Light+ (`F16.color-theme`); read them with `control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=setup-field-schedule-error]')).color"`. The schedule preflight answers only once name and prompt are filled: with them empty `/validate` returns `200` and no schedule error appears.@@ -93,16 +98,17 @@ Preconditions:- **Conditional prompt launch (`F22.template-launch-conversation`).** First exercise missing required integrations on `/automations/templates`: click `testid=recommended-automation-pills-jira-issue-to-bitbucket-pr-overflow`, inspect the integration pills, close the popover with Escape, and click `testid=recommended-automation-card-jira-issue-to-bitbucket-pr`. The current pinned catalog names Atlassian Rovo and Bitbucket; their installable MCP connection is `https://mcp.atlassian.com/v2/mcp`, so a missing connection opens the **Atlassian Rovo** install queue rather than launching a conversation. Cancel with `testid=mcp-install-cancel` and verify the location stays `/automations/templates`. To exercise the conditional prefill without an external account, arrange only stored connection configuration on the isolated run with `control-openhands api PATCH /api/settings --write --data '{"agent_settings_diff":{"mcp_config":{"qa-rovo":{"url":"https://mcp.atlassian.com/v2/mcp","transport":"streamable-http"}}}}'`; this is a launcher precondition, not OAuth, reachability, or a working integration claim. Now click the same card with `--expect-url '/conversations/[0-9a-f-]+' --timeout 60000`, wait for `testid=chat-input`, and read the composer. The 2026-10-17 pinned catalog prefills `Set up the Jira issue to Bitbucket PR automation`; the user-message count remains `0`. Take `browser screenshot --feature F22.template-launch-conversation --name prefilled` and do not press Send. Delete only the owned conversation and `api DELETE /api/settings/mcp/qa-rovo --write`. The previous recipe's `/ticket-to-code-change:setup` and “Needs external setup” expectation do not describe this catalog revision. Completing the real install queue and authenticated Jira/Bitbucket workflow still requires an authorized account; the stored-config check does not pass those branches.- **Open and save a draft (`F22.agent-assisted-setup`, desktop and phone).** This scoped recipe needs a doctored baseline stack with automation draft support (1.19.3 here); its non-dispatch steps need no LLM key or profile. From `/automations` run `control-openhands browser click 'testid=automations-add-automation'`, then `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`. The form and docked composer are visible; `control-openhands conversation list` is unchanged. Run `control-openhands browser fill 'testid=automation-setup-name' 'QA_F22 Delta Draft'`, `control-openhands browser fill 'testid=automation-setup-prompt' 'Reply pong. Do not run any tools.'`, `control-openhands browser click 'testid=automation-setup-frequency-custom'`, `control-openhands browser fill 'testid=automation-setup-custom-schedule' '0 0 1 1 *'`, then `control-openhands browser click 'testid=automation-setup-save-draft'`. `control-openhands browser wait 'testid=automation-setup-draft-details' --timeout 10000` and `browser snapshot 'testid=automation-setup-draft-details'` show **Draft saved** and **Ready to test**. `control-openhands api GET /api/automation/v1/drafts` supplies `<draft-id>` and confirms the name, prompt and yearly cron; the URL carries `draftId`. After `browser reload`, read each field with `browser value 'testid=automation-setup-name'`, `browser value 'testid=automation-setup-prompt'` and `browser value 'testid=automation-setup-custom-schedule'`: the values match. Screenshot with `--feature F22.agent-assisted-setup --name saved-draft`. Saving a prompt draft does not prove model execution; use the configured flash profile and the **Genuine prompt/plugin draft Test** checks below for that path.- **Credential-free script test (`F22.agent-assisted-setup`).** Run `control-openhands browser click 'testid=automation-setup-kind-custom'`, then inspect `browser value 'testid=automation-setup-custom-code'`: the generated Python prints its prompt and calls the real automation callback, without an agent/LLM. Keep its code, setup and entrypoint unchanged; click Save draft. Run `browser record start --feature F22.agent-assisted-setup --name custom-script-test --max-seconds 180`, then `browser click 'testid=automation-setup-draft-test'` and `browser wait 'testid=automation-setup-draft-runs-page' --timeout 10000`. Its snapshot initially shows Pending. Immediately run `api GET /api/automation/v1/drafts/<draft-id>` for `materialized_automation_id` and `last_test_run_id`, then `api GET '/api/automation/v1/<materialized-id>/runs?limit=10'`. Repeat the quoted runs GET after short `sleep 5` intervals until that run has terminal backend status (bounded to 90 seconds; no single wait over 45 seconds). Once it is COMPLETED, `browser wait 'testid=automation-setup-draft-run >> text=Successful' --timeout 45000` tests automatic UI completion. Expected: Successful without a reload. Record the actual row and a second backend receipt before any recovery. As a separate readback, run `browser reload`, click `testid=automation-setup-view-test-runs`, screenshot Successful, then `browser click 'testid=automation-setup-draft-runs-back'`: the draft details read **Latest test passed**. End with `browser record stop`. A saved draft, Pending row or successful reload does not prove automatic completion updates.-- **Agent profile on a Prompt or Plugin draft (`F22.agent-assisted-setup`).** Arrange a second saved agent profile: `control-openhands api POST /api/agent-profiles/QA_F22_Profile --write --data '{"schema_version":3,"agent_kind":"openhands","name":"QA_F22_Profile","llm_profile_ref":"deepseek-flash","agent":"CodeActAgent"}'`. Open a blank form (Add Automation → Create Automation), fill `testid=automation-setup-name` with `QA_F22 Profile` and the prompt with `Reply pong. Do not run any tools.`. Expected: `browser count 'testid=automation-setup-agent-profile'` is `0` (no profile offered to a Prompt draft), and Save draft succeeds. Known failure (reproduced 2026-10-17, [#18569](https://github.com/OpenHands/OpenHands/issues/18569)): the pill reads `default`; clicking it lists `automation-setup-agent-profile-option-active`, `-default` and `-QA_F22_Profile`. Pick `QA_F22_Profile`, `browser network --clear`, Save draft: `browser network --bodies --filter 'automation/v1'` shows `POST /api/automation/v1/drafts` `422` with `"endpoint":"/v1/preset/prompt"` and `agent_profile_id` in the draft, `browser toasts --history` holds `Extra inputs are not permitted` and `browser text 'testid=automation-setup-save-state'` is `Save failed`. Picking `automation-setup-agent-profile-option-active` and saving again returns `201`. With a plugin added (`testid=automation-setup-add-plugin`, source `github:OpenHands/extensions/plugins/qa-none`) and the profile picked again, the draft `PATCH` to `/v1/preset/plugin` fails the same way. Delete the draft with its dashboard trash icon (`F21.saved-drafts`) and run `control-openhands api DELETE /api/agent-profiles/QA_F22_Profile --write`.+- **Agent profile on a Prompt or Plugin draft (`F22.agent-assisted-setup`).** Arrange a second saved agent profile: `control-openhands api POST /api/agent-profiles/QA_F22_Profile --write --data '{"schema_version":3,"agent_kind":"openhands","name":"QA_F22_Profile","llm_profile_ref":"deepseek-flash","agent":"CodeActAgent"}'`. Open a blank form (Add Automation → Create Automation), fill `testid=automation-setup-name` with `QA_F22 Profile` and the prompt with `Reply pong. Do not run any tools.`. `browser count 'testid=automation-setup-model'` is `1` and `browser count 'testid=automation-agent-profile'` (the agent profile selector the catalog setup dialog uses) is `0`: no profile is offered to a Prompt draft, only the LLM profile pill sits under the Prompt. `browser network --clear`, Save draft: `browser network --bodies --filter 'automation/v1'` shows `POST /api/automation/v1/drafts` `201` with `"endpoint":"/v1/preset/prompt"` and no `agent_profile_id` in the draft, and the draft bar reads **Draft saved** / **Ready to test**. Add a plugin (`testid=automation-setup-add-plugin`, fill `testid=automation-setup-plugin-source` with `github:OpenHands/extensions/plugins/qa-none`): the count is still `0`, and Save draft sends `PATCH` `200` to `/v1/preset/plugin` without `agent_profile_id` (`browser text 'testid=automation-setup-save-state'` is `Saved just now`). Take `browser screenshot --feature F22.agent-assisted-setup --name plugin-draft-no-profile`. At phone width (`browser viewport phone`) a blank form and the reopened draft (`/automations/setup?draftId=<draft-id>`) also count `0`; restore `browser viewport desktop`. Delete the draft with its dashboard trash icon (`F21.saved-drafts`) and run `control-openhands api DELETE /api/agent-profiles/QA_F22_Profile --write`.- ```sh known-bug F22.agent-assisted-setup prompt-profile+ ```shtrap 'control-openhands api DELETE /api/agent-profiles/QA_F22_Profile --write >/dev/null' EXITcontrol-openhands api POST /api/agent-profiles/QA_F22_Profile --write --data '{"schema_version":3,"agent_kind":"openhands","name":"QA_F22_Profile","llm_profile_ref":"deepseek-flash","agent":"CodeActAgent"}'control-openhands browser viewport desktopcontrol-openhands browser goto /automations/setupcontrol-openhands browser wait 'testid=automation-setup-prompt'control-openhands assert --field value --equals '' -- browser value 'testid=automation-setup-prompt'- control-openhands assert --field count --equals 0 --bug-shows 1 -- browser count 'testid=automation-setup-agent-profile' # bug+ control-openhands assert --field count --equals 1 -- browser count 'testid=automation-setup-model'+ control-openhands assert --field count --equals 0 -- browser count 'testid=automation-agent-profile'```- **Resume at phone width (`F22.agent-assisted-setup`).** Run `browser viewport phone`, `browser goto /automations`, then click `testid=automation-setup-draft-resume-<draft-id>` with `--expect-url '/automations/setup'`. Read Name, rename to `QA_F22 Phone Draft`, Save draft and reload: `browser value 'testid=automation-setup-name'` and `api GET /api/automation/v1/drafts/<draft-id>` both have the new name. `browser bbox 'testid=automation-setup-form'` has `pageHorizontalOverflow: false`; this long form scrolls vertically and need not fit wholly inside the viewport. Screenshot with `--feature F22.agent-assisted-setup --name resumed-phone`. Repeat initial open/save/reload on a fresh phone draft when checking that entry point; desktop creation is not phone creation. Restore desktop afterward.- **Agent fills a prompt form (`F22.agent-assisted-setup`, desktop).** Start a fresh form with `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup'`. Run `control-openhands conversation start --stay --prompt 'Call automation_form_update exactly once with fields {"name":"QA_F22 DeepSeek Desktop","kind":"prompt","prompt":"Reply pong. Do not run any tools.","frequency":"custom","customSchedule":"0 0 1 1 *","timezone":"UTC","model":"deepseek-flash"}. Do not call any other tools, run terminal commands, inspect files, or read environment variables. Briefly confirm the filled fields and stop.'`. `--stay` sends through the docked composer; omitting it would navigate away to New Chat. The returned `<setup-conversation-id>` opens `/conversations/<id>` with the form beside it. Run `control-openhands conversation wait <setup-conversation-id> --until finished,idle --timeout 60`, then `control-openhands conversation events <setup-conversation-id> --kinds ActionEvent --full`: the summary identifies the genuine `automation_form_update` tool, but `--full` expands text rather than structured action fields. For the exact receipt run `control-openhands api GET '/api/conversations/<setup-conversation-id>/events/search?limit=100' | jq '.body.items[] | select(.kind == "ActionEvent" and .tool_name == "automation_form_update") | .action.fields'`; follow `next_page_id` if the action is beyond the first page. Read `control-openhands browser value 'testid=automation-setup-name'`, `control-openhands browser value 'testid=automation-setup-prompt'`, `control-openhands browser value 'testid=automation-setup-custom-schedule'` and `control-openhands browser text 'testid=automation-setup-model'`; they match the requested values. `control-openhands browser snapshot 'testid=automation-setup-form'` also shows **Filled by OpenHands** markers. Capture `control-openhands browser screenshot --feature F22.agent-assisted-setup --name desktop-genuine-tool-filled`; the UI field readbacks and real action establish the update, not the agent's confirmation alone. The sent prompt is not kept as a draft: `control-openhands browser eval "sessionStorage.getItem('oh:home-prompt-draft')"` is `null` (with no earlier Home draft), and the New Chat composer (`browser eval "document.querySelector('[data-testid=chat-input]').innerText"` after clicking `text=New Chat`) and a later `/automations/setup` docked composer are empty; return to `/conversations/<setup-conversation-id>` for the next bullet.@@ -117,7 +123,7 @@ Preconditions:- **Import (`F22.import-confirm`).** Reopen the dialog, upload `qa-f22-import.automation.json`, then `control-openhands browser click 'testid=import-automation-confirm' --observe 'testid=import-automation-confirm' --observe-ms 3000` (observed `Importing...` for about 50 ms, then `<absent>`) and `control-openhands browser toasts`: `Imported "QA_F22 Imported" as disabled. View automation`. The card `[data-testid^="automation-card-"] >> has-text=QA_F22 Imported` is under **Inactive**, and `api GET /api/automation/v1/<id>` has `enabled: false`, `state` `INACTIVE` and the plugin in `preset_metadata.plugins`. The toast lasts about 5 s, so follow its link in the very next command: import `qa-f22-view.automation.json` the same way, then immediately `control-openhands browser click 'role=status >> role=link[name="View automation"]' --expect-url '/automations/[0-9a-f-]{8,}'`; `browser text 'role=heading[level=1]'` is `QA_F22 View`.- **Choose file (`F22.import-choose-file`).** On `/automations` (`control-openhands browser goto /automations`; Import ends on QA_F22 View's page) open Add Automation → Import automation and run `control-openhands browser upload-via 'testid=import-automation-choose-file' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-import.automation.json"` (answers the real file chooser): `browser attr 'testid=import-automation-modal' data-view` is `preview`. Close with `testid=import-automation-modal-close`, reopen, and `control-openhands browser upload-via 'testid=import-automation-dropzone >> text=Drop a .json file here' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-import.automation.json"`: `preview` again. `control-openhands browser press Escape` closes the dialog (`browser count 'testid=import-automation-modal'` `0`).- **Drop a file (`F22.import-dropzone`).** Open the import dialog again and run `control-openhands browser drop-files 'testid=import-automation-dropzone' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-import.automation.json" --stage over`: `browser attr 'testid=import-automation-dropzone' data-active` is `true` (focus border). Close and reopen the dialog, then `control-openhands browser drop-files 'testid=import-automation-dropzone' "$OH_VERIFY_RUN/evidence/_fixtures/qa-f22-import.automation.json"`: `data-view` is `preview` with Name `QA_F22 Imported` (`browser text 'testid=import-automation-modal'`). Close with `testid=import-automation-modal-close`.-- **Phone (`F22.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /automations/templates`, `control-openhands browser bbox 'testid=recommended-automations-section'` (`pageHorizontalOverflow` `false`) and `browser screenshot --feature F22.phone --name templates`. Click `testid=recommended-automation-card-custom-automation`, then `testid=setup-continue-button`; `browser bbox 'testid=setup-dialog'` is `insideViewport` `true` with no page overflow (`--name setup`). Close it, `control-openhands browser goto /automations`, open Add Automation → Import automation, upload `qa-f22-import.automation.json`; `browser bbox 'testid=import-automation-modal'` is `insideViewport` `true` (`--name import-preview`). Close with `testid=import-automation-modal-close`, then `control-openhands browser viewport desktop`.+- **Phone (`F22.phone`).** Run `control-openhands browser viewport phone`, `control-openhands browser goto /automations/templates`, `control-openhands browser bbox 'testid=recommended-automations-section'` (`pageHorizontalOverflow` `false`), `control-openhands browser bbox 'testid=template-filters'` (the chips wrap: `insideViewport` `true`) and `browser screenshot --feature F22.phone --name templates`. Click `testid=recommended-automation-card-custom-automation`, then `testid=setup-continue-button`; `browser bbox 'testid=setup-dialog'` is `insideViewport` `true` with no page overflow (`--name setup`). Close it, `control-openhands browser goto /automations`, open Add Automation → Import automation, upload `qa-f22-import.automation.json`; `browser bbox 'testid=import-automation-modal'` is `insideViewport` `true` (`--name import-preview`). Close with `testid=import-automation-modal-close`, then `control-openhands browser viewport desktop`.- **Cleanup.** Deleting an automation is F21's flow; here remove the fixtures with `control-openhands api DELETE /api/automation/v1/<id> --write` for each `QA_F22 …` (Custom, Plugin, Tarball, Imported, View, Agent) and `Daily news digest` id from `control-openhands api GET '/api/automation/v1?limit=100'`. Delete `OPENHANDS_URL` in Settings → Secrets (`control-openhands browser click 'testid=secret-item >> has-text=OPENHANDS_URL >> testid=delete-secret-button'`, then `testid=confirmation-modal >> testid=confirm-button`, as in F14.delete) unless the next recipe wants it. The fallback and setup conversations stay in the conversation list (F04 deletes them). With zero automations, `control-openhands browser goto /automations` and `control-openhands browser click 'testid=automations-empty >> testid=automations-create-automation' --expect-url '/automations/setup'` opens the agent-assisted setup page.- **Malformed catalog entry (`F22.template-card`, `F22.templates-page`).** Archive the frozen target into an isolated private checkout, link unchanged dependencies except a private copy of the pinned `@openhands/extensions`, and remove only `requires` from `automations/catalog/custom-automation/manifest.json`. Leave Canvas source and its lockfile unchanged; run `npm run build:app` only in that copy. Serve its flat `build/` directory against an owned healthy backend, then open `/automations/templates` through `browser goto <fixture-origin>/automations/templates --allow-external`. Expected: the malformed entry degrades without crashing the valid catalog. Known failure, reproduced 2026-10-17 on frozen `61a3bd93` with a private copy of pinned `@openhands/[email protected]`: desktop and phone direct navigation to `/automations/templates`, including reload, renders **Application Error**, `TypeError: Cannot read properties of undefined (reading 'integrations')`. `/automations` also crashes before its Templates link can render, so this retest uses the actual direct route. SDK 1.55.1 and automation 1.20.1 health remained `200`, and doctor passed before and after ([#16277](https://github.com/OpenHands/OpenHands/issues/16277)). Record this as catalog-data fault injection, then stop/remove only the private fixture frontend/build; never modify the shared package or build.@@ -138,13 +144,12 @@ Preconditions:- Service preflight runs 400 ms after a blur, not on every keystroke; `browser press Tab` then `browser wait` for the `-error` element.- **Confirm and create** creates the automation enabled; only imports arrive disabled. Use a yearly cron (`0 0 1 1 *`) so nothing fires during the run.- The **Daily news digest** entry names its automation `Daily news digest` (no `QA_` prefix); remember it in cleanup.-- The Agent profile combobox exists only while the chosen action creates through `/v1` (Upload tarball, bundle entries): the automation service rejects `agent_profile_id` on `/v1/preset/prompt` and `/v1/preset/plugin`, so the Prompt and Plugin actions hide it and switching to them clears a chosen profile. Leave it on `Default` except in the Upload tarball bullet. A cleared choice is stored as an empty id and shown as `Default` (#18111).+- The Agent profile combobox exists only while the chosen action creates through `/v1` (Upload tarball, bundle entries): the automation service rejects `agent_profile_id` on `/v1/preset/prompt` and `/v1/preset/plugin`, so the Prompt and Plugin actions hide it and switching to them clears a chosen profile. Leave it on `Default` except in the Upload tarball bullet. A cleared choice is stored as an empty id and shown as `Default` (#18111). The setup page (`/automations/setup`) has no agent profile control at all: its Prompt and Plugin drafts go to the same preset endpoints.- The automation detail page has no `main` landmark: read its title with `browser text 'role=heading[level=1]'`, not `'role=main >> …'`.- The import success toast is the only path to **View automation** and disappears after about 5 s.- **Continue with local setup** silently creates the `OPENHANDS_URL` secret (value: the app origin) the first time; F14's list then has an extra row.- The **Install** button in a template's install dialog verifies the MCP connection before saving, so dummy credentials cannot complete the queue; nothing is saved on failure.- Creating the first automation ticks **Schedule a task** in the sidebar Getting-started checklist (F02); a run that checks the checklist should do so before this file.-- Known issue [#17959](https://github.com/OpenHands/OpenHands/issues/17959): a template search with no matches leaves a blank area with no message (`F22.templates-search`).- Known issue [#17436](https://github.com/OpenHands/OpenHands/issues/17436) (`F22.setup-review-confirm`): Prompt-preset automations created after the Canvas 1.17.0 upgrade fail on the first conversation events request while older tarballs work, so the UI shows Failed runs. The [reporter’s correction](https://github.com/OpenHands/OpenHands/issues/17436#issuecomment-5685142272) says both comparison tarballs contain a private patch and explicitly disproves missing FinishTool registration; [PR #17539](https://github.com/OpenHands/OpenHands/pull/17539) closed unmerged and is not fix proof. Exact historical reproduction is blocked pending those two patched tarballs (or exact base revisions plus patch bytes), the underlying traceback and deployment-specific SDK patch identity. Stock old packages alone do not reproduce that supplied-artifact comparison, and frozen Canvas minimum Agent Server 1.51.0 excludes its reported 1.46.0 UI backend. A genuine current-pin prompt-preset completion is a separate normal control, not retirement of this version-specific note. Keep it separate from the plugin-preset Cloud failure in #16611. A separate current-pin control on 2026-10-17 used the actual stock prompt-preset API with a disabled, unregistered event definition and one manual Flash dispatch: the first conversation-events POST returned 200, the run completed, and Canvas showed Successful. That preset-API control is not Review/Confirm UI coverage or the historical patched-tarball comparison; no note is retired.- Known issue [#15509](https://github.com/OpenHands/OpenHands/issues/15509) (`F22.setup-tarball-create`): A native Windows event-triggered automation fails while uploading its workspace tarball with `Path must be absolute`. Recheck a Windows local stack and real event delivery, then observe its failed run/upload; a POSIX tarball upload is not a Windows path-validation check. This run has neither Windows nor the webhook fixture.- Known issue [#16611](https://github.com/OpenHands/OpenHands/issues/16611) (`F22.setup-plugin-create`): a minimal plugin-preset Cloud automation fails on its in-sandbox conversation/events request with HTTP 500, while the identical prompt preset works. Live prerequisites: an authenticated Cloud account, a real model key, and a reviewed no-op plugin fixture for the paired cases. The reported cost-zero error is visible in run logs, but a local prompt-only check cannot establish the Cloud plugin-loading behavior.diff --git a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.mdindex 0a29330d3c..c64e32f398 100644--- a/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md+++ b/.agents/skills/verify-openhands/references/feature-map/F23-automation-detail.md@@ -10,7 +10,9 @@ when it has plugins; an Activity card (Created, Last run);and the **Activity Log** of runs with status, summary, cost, a logs dialog, alink to each run's conversation and JSON/CSV export. Kebab **Edit** opens thesetup page in edit mode (`/automations/setup?automationId=<id>`, since #17688),-which changes name, prompt, agent and LLM profile, timeout and the trigger. A+which changes name, prompt, LLM profile, timeout and the trigger; it has no+agent profile control, and saving keeps any agent profile already stored on the+automation. ADRAFT-state test automation (materialized by a draft Test, F21.saved-drafts)shows a **Draft** badge and no on/off switch.@@ -47,10 +49,10 @@ Source: `src/routes/automation-detail.tsx`, `src/components/features/automations- `F23.export`: kebab **Export** downloads `<name-slug>.automation.json` (`version`, `kind: "automation"`, `spec`); an automation's first repository and its plugins go into `spec` as `repository`, `branch` and `plugins`.- `F23.tarball`: kebab **Tarball** downloads the automation's code bundle.- `F23.delete`: kebab **Delete** asks `Delete "<name>"? This cannot be undone.`; Cancel keeps it, Delete removes it and returns to `/automations`.-- `F23.edit-open`: kebab **Edit** opens the setup page `/automations/setup?automationId=<id>` (#17688; the **Edit automation** modal is gone) prefilled with Name, Prompt, agent profile and LLM profile pickers, the trigger (Schedule with Frequency, or Event), timezone and Timeout seconds, with **Test**, **Save** and **Close** in the toolbar and a docked composer; no conversation exists until a prompt is sent. A script (tarball or bundle) automation shows `Custom Python` with the **Existing code bundle** summary (`automation-setup-existing-bundle`, the upload id) instead of a code editor.+- `F23.edit-open`: kebab **Edit** opens the setup page `/automations/setup?automationId=<id>` (#17688; the **Edit automation** modal is gone) prefilled with Name, Prompt, the LLM profile picker (no agent profile picker), the trigger (Schedule with Frequency, or Event), timezone and Timeout seconds, with **Test**, **Save** and **Close** in the toolbar and a docked composer; no conversation exists until a prompt is sent. A script (tarball or bundle) automation shows `Custom Python` with the **Existing code bundle** summary (`automation-setup-existing-bundle`, the upload id) instead of a code editor.- `F23.edit-close`: **Close** leaves without saving and lands on `/automations` (the dashboard, not the detail page); Escape does nothing on the page.- `F23.edit-validation`: an empty prompt is refused with the toast **Add a prompt before continuing.** (no request); a bad cron, a cron that never runs or a timeout above 1800 is sent to the API, refused with `422`, and its message is the single error toast (`Value error, Invalid cron expression: not a cron`, `Value error, Cron expression cannot produce any future fire times: 0 0 31 2 *`, `Value error, timeout must not exceed 1800 seconds (30 minutes)`); nothing is saved.-- `F23.edit-save`: Save sends one PATCH (also when nothing changed), shows **Saving…** then **Saved just now**, the toast **Automation updated**, and the detail page and API reflect it; an Active profile model stays `null`. On a script (tarball or bundle) automation, including one without a `setup_script_path`, Save keeps the existing bundle: it asks for no setup script path, setup script or code, the PATCH carries no bundle fields, and `tarball_path`, `entrypoint` and `setup_script_path` read back unchanged; the summary stays after each edit and a reload.+- `F23.edit-save`: Save sends one PATCH (also when nothing changed), shows **Saving…** then **Saved just now**, the toast **Automation updated**, and the detail page and API reflect it; an Active profile model stays `null`. On a Prompt or Plugin automation the PATCH carries no `agent_profile_id`, so a profile stored on it stays unchanged; on a script automation it sends the stored `agent_profile_id` back. On a script (tarball or bundle) automation, including one without a `setup_script_path`, Save keeps the existing bundle: it asks for no setup script path, setup script or code, the PATCH carries no bundle fields, and `tarball_path`, `entrypoint` and `setup_script_path` read back unchanged; the summary stays after each edit and a reload.- `F23.edit-test`: on the edit page **Test** dispatches the automation as last saved (`POST /api/automation/v1/<id>/dispatch`, no PATCH first) and shows `Test run started.` on the form; unsaved edits are not part of that run. Whether it should test the current form is a product decision (Gotchas).- `F23.edit-dirty-state`: the edit page shows no save state for unsaved edits (no **Unsaved changes**, which create mode shows, F28); the label appears only while saving (**Saving…**) and after (**Saved just now** or **Save failed**). **Close** discards the edits.- `F23.edit-schedule`: Frequency offers Once, Hourly, Daily, Weekdays (Mon–Fri), Weekly and Custom; Weekly adds **Day of week** and **At**; they are saved as a cron expression and read back after a reload. Switching to Custom starts the Custom field from the cron of the preset it replaces (Weekly, Wednesday at `10:30` → `30 10 * * 3`), and a saved cron that matches a preset reads back as that preset, not as Custom.@@ -94,9 +96,9 @@ Preconditions:- **Export the log (`F23.activity-log-export`).** `control-openhands browser click 'testid=activity-log-export-json'` then `control-openhands browser downloads --last 1 --inspect`: `qa-f23-pong.activity-log.json` with `runs[]` holding `run_id`, `automation_name`, `trigger`, `start_time`, `end_time`, `duration_seconds`, `status` `COMPLETED`, `conversation_id` and `conversation_url` (`http://127.0.0.1:<port>/conversations/<id>`). `control-openhands browser click 'testid=activity-log-export-csv'` and the same `downloads` call: `qa-f23-pong.activity-log.csv` with header `run_id,automation_id,automation_name,trigger,start_time,end_time,duration_seconds,status,conversation_id,conversation_url,error,cost,phase`.- **Export and Tarball (`F23.export`, `F23.tarball`).** `control-openhands browser click 'role=button[name="Automation actions"]'`, `control-openhands browser click 'role=list >> has-text=Tarball >> role=button[name="Export"]'`, `control-openhands browser downloads --last 1 --inspect`: `qa-f23-pong.automation.json` with `"version": 1`, `"kind": "automation"` and `spec` (name, trigger, `enabled`, prompt, `model` `null`, `timeout` `600`, `timezone`). Reopen the kebab and click `role=list >> has-text=Tarball >> role=button[name="Tarball"]`; `control-openhands browser downloads --last 1` lists `QA_F23_Pong.tar` (about 9 KB) and no toast appears. The file is gzip data despite the name: `tar -tzf <path>` lists `main.py`, `finish_tool_hook.py`, `prompt.txt`, `setup.sh`.- **Turn off and the paused banner (`F23.disabled-reason`, `F23.toggle`).** While Pong is active `control-openhands browser count 'testid=automation-disabled-reason-banner'` is `0`. Open the kebab, `control-openhands browser click 'role=list >> has-text=Tarball >> role=button[name="Turn off"]'`, `control-openhands browser count 'testid=confirmation-modal'` (`0`), `control-openhands browser reload`, `control-openhands browser text 'testid=automation-disabled-reason-banner'`: `Why this is paused` / `Turned off manually` / `Paused Just now` (`automation-disabled-reason-text`, `automation-disabled-reason-timestamp`). The badge reads `Inactive` and Run now is disabled. Screenshot with `--feature F23.disabled-reason --name manual`.-- **Edit fields and closing (`F23.edit-open`, `F23.edit-close`).** On Pong run `control-openhands browser click 'role=button[name="Automation actions"]'` and `control-openhands browser click 'role=list >> has-text=Tarball >> role=button[name="Edit"]' --expect-url '/automations/setup'`: the URL is `/automations/setup?automationId=<id>`, `browser count 'testid=edit-automation-save'` is `0` (no modal), and `browser value` reads `automation-setup-name` (`QA_F23 Pong`), `automation-setup-prompt`, the stored schedule (a custom cron shows Frequency Custom with `automation-setup-custom-schedule`) and `automation-setup-timeout` (`600`); `browser text 'testid=automation-setup-agent-profile'` is `default` and `browser text 'testid=automation-setup-model'` names the active profile (the stored `model` stays `null`). `browser testids 'testid=automation-setup-toolbar'` lists `automation-setup-test`, `automation-setup-save` and `automation-setup-close`; `control-openhands conversation list` is unchanged. `control-openhands browser press Escape` leaves the URL unchanged; `control-openhands browser click 'testid=automation-setup-close' --expect-url '/automations$'` lands on the dashboard without a PATCH (the dashboard's own GETs follow). A cold `control-openhands browser goto '/automations/setup?automationId=<id>'` opens the same form.+- **Edit fields and closing (`F23.edit-open`, `F23.edit-close`).** On Pong run `control-openhands browser click 'role=button[name="Automation actions"]'` and `control-openhands browser click 'role=list >> has-text=Tarball >> role=button[name="Edit"]' --expect-url '/automations/setup'`: the URL is `/automations/setup?automationId=<id>`, `browser count 'role=dialog'` is `0` (no modal), and `browser value` reads `automation-setup-name` (`QA_F23 Pong`), `automation-setup-prompt`, the stored schedule (a custom cron shows Frequency Custom with `automation-setup-custom-schedule`) and `automation-setup-timeout` (`600`); `browser count 'testid=automation-agent-profile'` is `0` (no agent profile picker) and `browser text 'testid=automation-setup-model'` names the active profile (the stored `model` stays `null`). `browser testids 'testid=automation-setup-toolbar'` lists `automation-setup-test`, `automation-setup-save` and `automation-setup-close`; `control-openhands conversation list` is unchanged. `control-openhands browser press Escape` leaves the URL unchanged; `control-openhands browser click 'testid=automation-setup-close' --expect-url '/automations$'` lands on the dashboard without a PATCH (the dashboard's own GETs follow). A cold `control-openhands browser goto '/automations/setup?automationId=<id>'` opens the same form.- **Validation (`F23.edit-validation`).** On the edit page run `control-openhands browser fill 'testid=automation-setup-prompt' ''`, `control-openhands browser network --clear` and `control-openhands browser click 'testid=automation-setup-save'`: `control-openhands browser toasts --history` includes the toast `Add a prompt before continuing.` and `browser network --filter 'automation/v1'` lists no request. Refill the prompt. On a Custom-cron automation (QA_F23 Fail) fill `testid=automation-setup-custom-schedule` with `not a cron` (then `0 0 31 2 *`, then a valid cron with `testid=automation-setup-timeout` `1801`), click Save and read `control-openhands browser toasts --history` after `browser toasts --history --clear`. Each sends a `PATCH` that returns `422` (`browser network --filter 'automation/v1'`), and the history holds exactly one toast with the API message: `Value error, Invalid cron expression: not a cron`, `Value error, Cron expression cannot produce any future fire times: 0 0 31 2 *` and `Value error, timeout must not exceed 1800 seconds (30 minutes)`; `api GET /api/automation/v1/<Fail id>` keeps `0 0 1 1 *` and timeout `600`. Before each new click move the pointer off the toast (`browser hover 'testid=automation-setup-name'`, about 5 s; see Gotchas).-- **No-change Save (`F23.edit-save`).** Open Edit, `control-openhands browser network --clear`, click `testid=automation-setup-save`, then `control-openhands browser network --filter 'automation/v1'`: one `PATCH` with the unchanged values and the toast `Automation updated` (the old modal sent nothing).+- **No-change Save (`F23.edit-save`).** Open Edit, `control-openhands browser network --clear`, click `testid=automation-setup-save`, then `control-openhands browser network --filter 'automation/v1'`: one `PATCH` with the unchanged values (`name`, `trigger`, `model` `null`, `timeout`, `prompt`; no `agent_profile_id`) and the toast `Automation updated` (the old modal sent nothing).- **Weekly schedule, model and timeout (`F23.edit-schedule`, `F23.edit-save`).** On the edit page run `control-openhands browser click 'testid=automation-setup-frequency-weekly'`, `control-openhands browser select 'testid=automation-setup-weekday' 3` (Wednesday; options `0:Sunday` … `6:Saturday`), `control-openhands browser fill 'testid=automation-setup-time' 10:30`, `control-openhands browser click 'testid=automation-setup-model'`, `control-openhands browser click 'testid=automation-setup-model-option-deepseek-pro'`, `control-openhands browser fill 'testid=automation-setup-timeout' 300` and `control-openhands browser screenshot --feature F23.edit-schedule --name weekly-form`. Click `testid=automation-setup-frequency-custom`: `browser value 'testid=automation-setup-custom-schedule'` is `30 10 * * 3`; click `testid=automation-setup-frequency-weekly` again (the weekday and time stay). Then `control-openhands browser network --clear`, `control-openhands browser click 'testid=automation-setup-save' --observe 'testid=automation-setup-save-state' --observe-ms 2000` (`Saving…` → `Saved just now`) and `control-openhands browser toasts --history` (`Automation updated`). `browser network --bodies --filter 'automation/v1'` shows one `PATCH /api/automation/v1/<id>` with `schedule` `30 10 * * 3`, `model` `deepseek-pro` and `timeout` `300`. After `control-openhands browser reload` the weekday is `3`, At `10:30`, the model `deepseek-pro` and Timeout `300`; the detail page Configuration reads `Schedule` `30 10 * * 3` and `LLM profile` `deepseek-pro`.- **Cleared time (`F23.edit-time-cleared`).** On the weekly edit page `control-openhands browser fill 'testid=automation-setup-time' ''`, `control-openhands browser network --clear`, click Save. Expected: an inline message and nothing saved. Known failure (reproduced 2026-10-13, [#17948](https://github.com/OpenHands/OpenHands/issues/17948) item, new symptom since #17688): `browser network --bodies --filter 'automation/v1'` shows a `PATCH` with `schedule` `0 0 * * 3`, the toast reads `Automation updated` and the automation now runs at midnight. Save draft in the create flow (`/automations/setup`) stores `0 0 * * *` the same way; open PR #18260 targets both paths in `automation-setup-panel.tsx` at head `0f812ac68902d2c71896568311bffb8aa6f216a6` (checked 2026-10-15), but is not merged.- **Custom cron (`F23.edit-cron`).** Open QA_F23 Fail and Edit: Frequency **Custom** is selected and `control-openhands browser value 'testid=automation-setup-custom-schedule'` is `0 0 1 1 *` beside the timezone select (`UTC`). Fill `0 0 2 1 *` and `testid=automation-setup-timeout` with `1` (this makes the next run fail), Save and `browser toasts --history` (`Automation updated`); after a reload of the detail page the Configuration shows `0 0 2 1 *`. Invalid crons: see Validation.diff --git a/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md b/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.mdindex 6801110e9b..d1acd138ab 100644--- a/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md+++ b/.agents/skills/verify-openhands/references/feature-map/F25-backends-and-cloud.md@@ -10,8 +10,9 @@ backends in recovery mode. Cloud backends add organization rows, a Cloudsettings link and public sharing of conversations (read-only at`/shared/conversations/:id`); `/oauth/device/verify` is the device-approval page.When the page is served with the DigitalOcean Managed Agents bridge (the-desktop app's preload, or the web server's own bridge, which the launcher's-ingress mounts by default behind the session key), Add Backend gains a+desktop app's preload, or the web server's own experimental bridge, which the+launcher's ingress mounts behind the session key only when started with+`MARS_WEB=1`), Add Backend gains a**Providers** tab whose DigitalOcean entry signs in with a personal accesstoken and turns a Managed Agents session into a backend.Everything except the Cloud and DigitalOcean accounts themselves can be driven@@ -36,7 +37,7 @@ Source: `src/components/features/backends/`, `src/contexts/active-backend-contex- `F25.edit-backend`: the pencil opens "Edit backend" pre-filled (key masked) with a `Connected · Local v<version>` badge; empty name and invalid host show `Name is required` and `Enter a valid URL (e.g. http://localhost:8080)` and disable Save; Save re-tests the connection, keeps the modal open with `Could not connect to <host>` on failure, and persists on success.- `F25.edit-cancel`: **Cancel**, the X and a backdrop click in Edit backend close only the Edit modal and discard the draft; Manage backends stays open with the stored values.- `F25.edit-escape`: Escape in the Edit modal does nothing: Edit keeps its draft and Manage backends stays open underneath; at phone width, with Manage opened from the drawer's backend selector, the drawer stays open too.-- `F25.health-status`: a backend that stops answering turns its dot red (`Disconnected`) within one health probe (every 30 s for local and remote agent servers; Cloud backends every 5 min and not on window focus or reconnect, #18128); its Manage row reads `Disconnected (check URL or network)` with a red detail line and cannot be selected.+- `F25.health-status`: a backend that stops answering turns its dot to the theme's status-error colour (`bg-status-error`, `Disconnected`) within one health probe (every 30 s for local and remote agent servers; Cloud backends every 5 min and not on window focus or reconnect, #18128); its Manage row reads `Disconnected (check URL or network)` with a red detail line and cannot be selected.- `F25.recovery-gate`: reloading while the active backend is down shows only `agent-server-onboarding-screen` with Manage backends in recovery mode (no X, no Done, Escape ignored, primary **Add Backend**); choosing a healthy row restores the app.- `F25.remove-backend`: the trash icon asks `Remove backend "<name>"? If it is active, the app will switch back to Local.`; Cancel keeps it, Confirm removes it and an active removed backend falls back to Local (BM-003).- `F25.collapsed-entry`: with the rail collapsed, the backend icon's popover offers the same Add Backend and Manage Backends items, and both modals open.@@ -49,7 +50,7 @@ Source: `src/components/features/backends/`, `src/contexts/active-backend-contex- `F25.cloud-settings-link`: with a Cloud backend active the gear next to the selector opens `<cloud host>/settings?org=<org>` in a new tab.- `F25.cloud-sandbox-states`: Cloud conversations show waiting, archived and error sandbox states in chat.- `F25.cloud-org-suspended`: when a Cloud call made for the selected organization (`X-Org-Id`) fails with 403 and `detail` `Organization is suspended` or `User membership is suspended`, the app is replaced by `<org> is suspended. Contact your administrator to restore access.` (or `Your access to <org> is suspended. Contact your administrator to restore it.`), then `Switch to another workspace` and one button per other organization (`Personal Workspace` for the user's own). A button selects that organization and the app returns; with no other organization only the message shows. A reload clears the in-memory record. Blocked here: needs an OpenHands Enterprise instance with `ENABLE_SUPER_ADMIN` where a Super Admin suspended the organization or this member's membership, signed in as a non-Super-Admin member with that organization selected (#17988).-- `F25.providers-tab`: only when the page has the Managed Agents bridge (`window.marsBridge`), Add Backend shows a third tab **Providers** ("Connect through DigitalOcean and more.") with a **DigitalOcean** card ("Your team's OpenHands agents") and `More providers coming soon.`. Every launcher-served web page has it (the ingress answers `/mars/health`); without it (`MARS_WEB=0`, or a non-loopback bind with no session key at all) there are only the Cloud and Agent-server tabs and no Managed Agents UI anywhere. A non-loopback bind that has a key (F26 `--host 0.0.0.0` with `LOCAL_BACKEND_API_KEY`) mounts the bridge behind that key and offers the tab too, although the page itself carries no key (see the sign-in bullet's Known failure).+- `F25.providers-tab`: only when the page has the Managed Agents bridge (`window.marsBridge`), Add Backend shows a third tab **Providers** ("Connect through DigitalOcean and more.") with a **DigitalOcean** card ("Your team's OpenHands agents") and `More providers coming soon.`. A launcher-served web page has it only when the launcher was started with `MARS_WEB=1` (the ingress then answers `/mars/health`); without it (no `MARS_WEB`, any value other than `1`, or a non-loopback bind with no session key at all) there are only the Cloud and Agent-server tabs and no Managed Agents UI anywhere. A non-loopback bind that has a key (F26 `--host 0.0.0.0` with `LOCAL_BACKEND_API_KEY`) and `MARS_WEB=1` mounts the bridge behind that key and offers the tab too, although the page itself carries no key (see the sign-in bullet's Known failure).- `F25.digitalocean-sign-in`: the DigitalOcean card opens "Connect your DigitalOcean team" with a Back button, a masked **Personal access token** field, a **Create a token** link to `https://cloud.digitalocean.com/account/api/tokens` and **Connect with token**, disabled until a token is typed (the **Sign in with DigitalOcean** OAuth button exists only in the desktop app). A token of the wrong shape reads `That does not look like a DigitalOcean personal access token.`; a well-formed token DigitalOcean rejects shows its HTTP error (`401 Unauthorized`). Back returns to the Providers tab.- `F25.digitalocean-agents`: signed in, the panel lists the team's OpenHands agents and their sessions, **New Agent** (name and sandbox size), refresh and sign-out; connecting a session (or **Resume and connect** a paused one) adds it as the active backend, the sidebar's conversation list becomes that agent's session panel (`mars-session-panel`), and Manage Backends' Managed Agents view offers Pause, Stop, Disconnect and Open per session (blocked: needs a DigitalOcean account with Managed Agents).- `F25.digitalocean-backend-removal`: removing a DigitalOcean backend in Manage Backends explains that it only disconnects and offers **Disconnect only** or **Stop session** (blocked: needs a connected DigitalOcean session).@@ -75,7 +76,7 @@ Preconditions:- A second stack to add as a backend, started without a browser and never driven directly: `export OH_VERIFY_RUN_2=$(OH_VERIFY_RUN= control-openhands launch --new --build never --no-browser --print-run)`, then `control-openhands status --run "$OH_VERIFY_RUN_2"`; its `baseUrl` is `<second-url>` below and its key is `$OH_VERIFY_RUN_2/private/session-key`. It has no conversations.- No backend named `QA_Second` or `QA_Renamed` exists (the selector lists only `Local`).- Bullets run in order: Add creates `QA_Second`, Edit renames it to `QA_Renamed`, Remove deletes it.-- The Managed Agents rows need the bridge, which every `control-openhands launch` run has: `control-openhands browser eval "!!window.marsBridge"` is `true` and `curl -s <baseUrl>/mars/health` answers `{"ok":true,"authRequired":true}`. `F25.digitalocean-agents` and `F25.digitalocean-backend-removal` stay blocked without a DigitalOcean account.+- The Managed Agents rows need the bridge, which is opt-in and which `control-openhands launch` cannot turn on (it forwards no `MARS_WEB`): on the default run `control-openhands browser eval "!!window.marsBridge"` is `false` and `curl -s <baseUrl>/mars/health` returns the HTML index. Start a bridge stack by hand on free ports: `Q=$OH_VERIFY_RUN/private/f25-mars; mkdir -p $Q; HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 LOCAL_BACKEND_API_KEY=qa-f25-mars-key OH_SECRET_KEY=qa-f25-secret DO_NOT_TRACK=1 MARS_WEB=1 nohup node bin/agent-canvas.mjs --port 18960 > $Q/launcher.log 2>&1 & echo $! > $Q/pid`, wait until `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/health` is `200`, then `curl -s http://127.0.0.1:18960/mars/health` answers `{"ok":true,"authRequired":true}`. Drive it with `control-openhands browser goto http://127.0.0.1:18960/ --allow-external`: answer its telemetry consent (`browser uncheck 'testid=telemetry-consent-form >> role=checkbox'`, `browser click 'testid=confirm-telemetry-preferences'`), `browser click 'testid=onboarding-skip'` and `browser wait 'testid=root-layout'`; `browser eval "!!window.marsBridge"` is `true` there. Stop it with `kill -TERM $(cat $Q/pid)` when the Managed Agents bullets are done. `F25.digitalocean-agents` and `F25.digitalocean-backend-removal` stay blocked without a DigitalOcean account.- Cloud account: where this machine's proxy injects OpenHands Cloud credentials (`curl -s -o /dev/null -w '%{http_code}' https://app.all-hands.dev/api/organizations` is `200` without a token), the **Cloud backend** bullet below can approve the device flow in its popup; otherwise it is blocked. Approving creates an API key named `Device Link Access Key (<code>)` on the account: delete that key on the Cloud API keys page when done, and leave the account's other keys alone. Additional fixtures are required for: `F25.cloud-log-back-in` (needs a logged-out Cloud row), org switching in `F25.cloud-org-rows` (needs a second organization), the archived and error states of `F25.cloud-sandbox-states`, `F25.cloud-org-suspended` (also needs a suspended OpenHands Enterprise organization or membership), `F25.share-publicly` (toggle), and `F25.locked-cloud`, which also needs a launcher flag (see Gotchas).- **Open the selector (`F25.selector-dropdown`).** From `/conversations` run `control-openhands browser click 'testid=backend-selector'` and `control-openhands browser snapshot 'testid=backend-selector'`. The snapshot shows `listbox` with `option "Connected Local"`, a separator, `button "Add Backend"` and `button "Manage Backends"`. Screenshot with `control-openhands browser screenshot --feature F25.selector-dropdown --name open`. Hover opens it too: `browser press Escape`, move the pointer away with `control-openhands browser hover 'testid=command-menu-trigger'` (hovering a selector the pointer is already on does nothing), `control-openhands browser hover 'testid=backend-selector'`, then the same snapshot shows `combobox "Local" [expanded]` and the two footer buttons.@@ -94,16 +95,16 @@ Preconditions:- **Edit validation and errors (`F25.edit-backend`).** Open Manage Backends, `control-openhands browser click 'testid=manage-backends-edit-QA_Second'`, then `browser value 'testid=edit-backend-host'` (`<second-url>`), `browser attr 'testid=edit-backend-api-key' type` (`password`) and `browser text 'testid=edit-backend-status'` (`Connected`, `·`, `Local`, `v1.50.1` on separate lines). Fill `edit-backend-name` with an empty string and `edit-backend-host` with `not a host`, then `browser focus 'testid=edit-backend-api-key'` and `browser snapshot 'testid=edit-backend-form'`: alerts `Name is required` and `Enter a valid URL (e.g. http://localhost:8080)`; `browser enabled 'testid=edit-backend-submit'` is `false`.- **Escape in Edit (`F25.edit-escape`).** In the same Edit modal run `control-openhands browser fill 'testid=edit-backend-name' QA_Draft` and `control-openhands browser press Escape`: `browser count 'testid=edit-backend-modal'` and `browser count 'testid=manage-backends-modal'` both stay `1` and `browser value 'testid=edit-backend-name'` is still `QA_Draft` (`browser screenshot --feature F25.edit-escape --name after-escape`). Leave with `browser click 'testid=edit-backend-cancel'`; nothing was saved (the Manage row is still `QA_Second`). At phone width (the **Phone** bullet's drawer → Manage Backends → `manage-backends-edit-Local`), the same fill and Escape leave `edit-backend-modal`, `manage-backends-modal` and `sidebar-mobile-drawer` at count `1` with the draft kept (`--name phone-after-escape`).- **Edit and save (`F25.edit-backend`).** Manage Backends is still open: click the pencil `testid=manage-backends-edit-QA_Second` again. Fill `edit-backend-host` with `http://127.0.0.1:9`, `browser click 'testid=edit-backend-submit'`, `browser wait 'testid=edit-backend-error' --timeout 10000`: the text starts `Could not connect to http://127.0.0.1:9` and the modal stays. Fill the host back with `<second-url>` and `edit-backend-name` with `QA_Renamed`, click `edit-backend-submit`, then `control-openhands browser wait 'testid=edit-backend-modal' --state detached --timeout 10000`; `browser count 'testid=manage-backends-row-QA_Renamed'` is `1`. Click `manage-backends-done`, `browser reload`: the selector reads `QA_Renamed`.-- **Health status (`F25.health-status`).** With QA_Renamed the active backend (as the Edit bullet's reload leaves it; the closed selector shows only the active backend's dot), run `control-openhands service stop agent-server --run "$OH_VERIFY_RUN_2"`, then on the open page `control-openhands browser wait 'testid=backend-selector >> role=status[name="Disconnected"]' --timeout 40000`. `browser click 'testid=backend-selector'` and `browser snapshot 'testid=backend-selector'` show `option "Disconnected QA_Renamed"` next to `option "Connected Local"`. In Manage Backends `browser text 'testid=manage-backends-status-QA_Renamed'` is `Disconnected (check URL or network)`, `testid=manage-backends-status-detail-QA_Renamed` repeats it with `Check that the backend URL is correct ...`, and `browser enabled 'testid=manage-backends-row-QA_Renamed >> role=button >> nth=0'` is `false`. An error toast with the same text also appears. Screenshot with `--feature F25.health-status --name manage-down`.+- **Health status (`F25.health-status`).** With QA_Renamed the active backend (as the Edit bullet's reload leaves it; the closed selector shows only the active backend's dot), run `control-openhands service stop agent-server --run "$OH_VERIFY_RUN_2"`, then on the open page `control-openhands browser wait 'testid=backend-selector >> role=status[name="Disconnected"]' --timeout 40000`. The dot's class contains `bg-status-error` (computed `rgb(255, 104, 78)` in the default dark theme, equal to that token's). `browser click 'testid=backend-selector'` and `browser snapshot 'testid=backend-selector'` show `option "Disconnected QA_Renamed"` next to `option "Connected Local"`. In Manage Backends `browser text 'testid=manage-backends-status-QA_Renamed'` is `Disconnected (check URL or network)`, `testid=manage-backends-status-detail-QA_Renamed` repeats it with `Check that the backend URL is correct ...`, and `browser enabled 'testid=manage-backends-row-QA_Renamed >> role=button >> nth=0'` is `false`. An error toast with the same text also appears after the next probe (up to about 40 s; read `browser toasts --history`). Screenshot with `--feature F25.health-status --name manage-down`.- **Recovery gate (`F25.recovery-gate`).** Close the modal, `control-openhands browser reload`, then `control-openhands browser wait 'testid=manage-backends-modal' --timeout 20000`. `browser testids` lists `agent-server-onboarding-screen` and the Manage rows but no sidebar; `browser count 'testid=close-manage-backends-modal'` and `browser count 'testid=manage-backends-done'` are `0`, and `browser press Escape` leaves the modal (`count` `1`). `control-openhands browser click 'testid=manage-backends-row-Local >> role=button >> nth=0'` and `browser wait 'testid=agent-server-onboarding-screen' --state detached --timeout 15000` restore the app: selector `Local`, card count `1`. Screenshot the gate with `--feature F25.recovery-gate --name gate` before choosing.- **Remove the active backend (`F25.remove-backend`).** Run `control-openhands restart --run "$OH_VERIFY_RUN_2"`, `browser click 'testid=backend-selector'`, `browser wait 'testid=backend-selector >> role=option[name="Connected QA_Renamed"]' --timeout 25000` and click that option (selector reads `QA_Renamed`). Open Manage Backends, `control-openhands browser click 'testid=manage-backends-remove-QA_Renamed'`; `browser text 'testid=confirmation-modal'` reads `Remove backend "QA_Renamed"? If it is active, the app will switch back to Local.` `control-openhands browser click 'testid=confirmation-modal >> role=button[name="Cancel"]'` keeps the row (`count 'testid=manage-backends-row-QA_Renamed'` `1`). Click the trash again, then `control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'`: the row count is `0`. Click Done and `browser reload`: the selector reads `Local`, `browser count 'testid=backend-selector >> role=option'` (dropdown open) is `1`, and the card count is `1`.- **Add from Manage, Edit Cancel and X (`F25.manage-add`, `F25.edit-cancel`).** The second stack is running again after the previous bullet. Run `browser goto /conversations/<id>`, `browser wait 'testid=chat-interface'`, open the selector, click `manage-backends-menu-item`, then `control-openhands browser click 'testid=manage-backends-add'` and `control-openhands browser wait 'testid=add-backend-modal' --timeout 5000` (`visible`); `browser count 'testid=manage-backends-modal'` is `1` underneath. `control-openhands browser press Escape` leaves both (`add-backend-modal` and `manage-backends-modal` counts `1`); `browser click 'testid=add-backend-close'`: the Add count is `0` and the Manage count stays `1`. Click `manage-backends-add` again, `add-backend-option-agent-server`, fill `add-backend-name` with `QA_Second`, `add-backend-host` with `<second-url>` and `add-backend-api-key` with `--value-file "$OH_VERIFY_RUN_2/private/session-key"`, then `browser click 'testid=add-backend-submit' --expect-url '/conversations(\?|$)'`: the URL is `/conversations`, `count 'testid=add-backend-modal'` is `0`, `count 'testid=manage-backends-modal'` is still `1`, `count 'testid=manage-backends-row-QA_Second'` is `1` and the selector reads `QA_Second` (screenshot `--feature F25.manage-add --name after-add`). No consent prompt this time: it is stored on the second backend. Then `control-openhands browser click 'testid=manage-backends-edit-QA_Second'`, `browser fill 'testid=edit-backend-name' QA_Draft` and `control-openhands browser click 'testid=edit-backend-cancel'`: `count 'testid=edit-backend-modal'` `0`, `count 'testid=manage-backends-modal'` `1`, row `QA_Second` `1`, row `QA_Draft` `0`. Repeat with `control-openhands browser click 'testid=edit-backend-close'` and with a backdrop click (`control-openhands browser mouse-click 20 500`): same counts, and reopening the pencil shows `browser value 'testid=edit-backend-name'` `QA_Second`; leave with `edit-backend-cancel`. Remove it again: `browser click 'testid=manage-backends-remove-QA_Second'`, `browser click 'testid=confirmation-modal >> testid=confirm-button'`, `browser click 'testid=manage-backends-done'` and `browser reload`: the selector reads `Local`.- **Collapsed rail (`F25.collapsed-entry`).** Run `control-openhands browser click 'testid=sidebar-collapse-toggle'` (`attr 'aside[data-collapsed]' data-collapsed` is `true`), `control-openhands browser hover 'testid=collapsed-backend-selector-link'`, `control-openhands browser click 'testid=add-backend-menu-item'` and `control-openhands browser wait 'testid=add-backend-modal' --timeout 5000` (`visible`). Close it, hover again, click `manage-backends-menu-item` and `browser wait 'testid=manage-backends-modal' --timeout 5000`. Close with Done and expand with `browser click 'testid=sidebar-collapse-toggle'`.- **Phone (`F25.phone`).** Run `control-openhands browser viewport phone`, `browser goto /conversations`, `control-openhands browser click 'testid=sidebar-mobile-menu-toggle'`, `control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=backend-selector'`, `browser click 'testid=add-backend-menu-item'`, `browser click 'testid=add-backend-option-agent-server'` and `control-openhands browser bbox 'testid=add-backend-modal'`: `insideViewport` `true`, `pageHorizontalOverflow` `false`. Screenshot `--feature F25.phone --name add-modal`. Before closing it, `control-openhands browser press Escape` leaves `browser count 'testid=add-backend-modal'` and `browser count 'testid=sidebar-mobile-drawer'` at `1`. Close it with `testid=add-backend-close`: the drawer stays open and `control-openhands browser eval "!!document.activeElement.closest('[data-testid=sidebar-mobile-drawer] [data-testid=backend-selector]')"` is `true`. Open Manage Backends from the drawer selector the same way and check `bbox 'testid=manage-backends-modal'` the same. Escape inside a stacked modal reaches only that modal: `manage-backends-add` then Escape keeps Add, Manage and the drawer (`--feature F25.manage-add --name phone-escape-stays`), and its X returns to Manage; `manage-backends-remove-Local` opens the Remove confirmation, and Escape closes only it (Manage, the drawer and the `Local` row stay; `--name remove-confirm-escape-local`). Escape on Manage itself closes only Manage (drawer count `1`); a further Escape then lands on the focused drawer selector (see [#18233](https://github.com/OpenHands/OpenHands/issues/18233) in Gotchas), so reopen Manage from the selector before the Edit step. Click `testid=manage-backends-edit-Local` and check `bbox 'testid=edit-backend-modal'`: `insideViewport` `true`; repeat at `browser viewport 320x568`. At `browser viewport 320x400`, clear `edit-backend-name`, fill `edit-backend-host` with `not a host` and focus `edit-backend-api-key` to show both validation messages. Run `browser scroll 'testid=edit-backend-form' --by 600`: Cancel and Save must come into view while the title and `edit-backend-close` stay visible. Check the modal and all three controls with `bbox`, then close with Cancel. Return with `browser viewport desktop`; reopen Edit and confirm its width is still 520px.-- **Providers tab (`F25.providers-tab`).** On the default run `browser eval "!!window.marsBridge"` is `true` (Preconditions); `curl -s "http://127.0.0.1:$(control-openhands status | jq -r .ports.frontend)/mars/health"` (the static child behind the ingress) returns the HTML index, not JSON: one bridge per stack. Open Add Backend: the tablist adds `tab "Providers Connect through DigitalOcean and more."` (`testid=add-backend-option-providers`); `control-openhands browser click 'testid=add-backend-option-providers'` and `browser snapshot 'testid=add-backend-providers-panel'` show `button "DigitalOcean Your team's OpenHands agents"` and `More providers coming soon.` (`browser screenshot --feature F25.add-backend-modal --name with-providers-tab`). At phone width the modal stays inside the viewport with the three tab titles cut to `Ope…`, `Age…`, `Pro…` (`--feature F25.providers-tab --name phone-tabs`).-- **DigitalOcean sign-in (`F25.digitalocean-sign-in`).** On the default run, run `control-openhands browser click 'testid=add-backend-provider-digitalocean'` and `browser wait 'testid=managed-agents-sign-in'`: heading `Connect your DigitalOcean team`, `textbox "Personal access token"` (`browser attr 'testid=managed-agents-token' type` is `password`), link `Create a token` (`/url: https://cloud.digitalocean.com/account/api/tokens`) and `button "Connect with token" [disabled]`; no OAuth button in a browser (`browser count 'testid=managed-agents-oauth'` is `0`). Fill `testid=managed-agents-token` with `dop_v1_qa_invalid_token_0000` and click `testid=managed-agents-token-submit`: `browser text 'testid=managed-agents-auth-error'` reads `That does not look like a DigitalOcean personal access token.` (`POST /mars/rpc/savePat` `500`). Fill a well-formed dummy, `dop_v1_` plus 64 zeros, and submit again: after a few seconds the error reads `401 Unauthorized` (`POST /mars/rpc/savePat` `401`, DigitalOcean's answer; `browser screenshot --feature F25.digitalocean-sign-in --name rejected-token`). Never type a real token here. `control-openhands browser click 'testid=add-backend-providers-back'` returns to the Providers panel (`browser count 'testid=managed-agents-sign-in'` is `0`; the providers panel itself stays mounted behind the sign-in screen, so its count is `1` either way); close with `testid=add-backend-close`. At phone width the sign-in screen fits (`bbox 'testid=add-backend-modal'` `insideViewport` `true`). Both errors also raise an error toast with the same text. Expected on a page served without a baked key (F26 `--host 0.0.0.0`, Docker), once the server's key was typed on the API-key screen: the same two answers as above. Known failure (reproduced 2026-10-14, #18434): every token, even the malformed one, ends in `session key required` (`POST /mars/rpc/getAuthState` and `savePat` `401`), because the web bridge client sends only the baked page key, never the typed one; it predates #18349 (BASE with `MARS_WEB=1` fails the same way). A `--frontend-only` page connected to another stack's backend fails the same way for a different reason (the typed key belongs to the other stack); #18434 leaves that case to a maintainer decision.+- **Providers tab (`F25.providers-tab`).** On the default run open Add Backend: the tablist has only `tab "OpenHands Cloud ..."` and `tab "Agent-server ..."`, and `browser count 'testid=add-backend-option-providers'` is `0` (`browser screenshot --feature F25.providers-tab --name default-no-providers`). On the bridge stack (Preconditions) `curl -s http://127.0.0.1:18963/mars/health | head -c 15` (the static child behind its ingress) prints `<!DOCTYPE html>`, not JSON: one bridge per stack. Open Add Backend there: the tablist adds `tab "Providers Connect through DigitalOcean and more."` (`testid=add-backend-option-providers`); `control-openhands browser click 'testid=add-backend-option-providers'` and `browser snapshot 'testid=add-backend-providers-panel'` show `button "DigitalOcean Your team's OpenHands agents"` and `More providers coming soon.` (`browser screenshot --feature F25.add-backend-modal --name with-providers-tab`). At phone width the modal stays inside the viewport with the three tab titles cut to `Ope…`, `Age…`, `Pro…` (`--feature F25.providers-tab --name phone-tabs`).+- **DigitalOcean sign-in (`F25.digitalocean-sign-in`).** On the bridge stack, run `control-openhands browser click 'testid=add-backend-provider-digitalocean'` and `browser wait 'testid=managed-agents-sign-in'`: heading `Connect your DigitalOcean team`, `textbox "Personal access token"` (`browser attr 'testid=managed-agents-token' type` is `password`), link `Create a token` (`/url: https://cloud.digitalocean.com/account/api/tokens`) and `button "Connect with token" [disabled]`; no OAuth button in a browser (`browser count 'testid=managed-agents-oauth'` is `0`). Fill `testid=managed-agents-token` with `dop_v1_qa_invalid_token_0000` and click `testid=managed-agents-token-submit`: `browser text 'testid=managed-agents-auth-error'` reads `That does not look like a DigitalOcean personal access token.` (`POST /mars/rpc/savePat` `500`). Fill a well-formed dummy, `dop_v1_` plus 64 zeros, and submit again: after a few seconds the error reads `401 Unauthorized` (`POST /mars/rpc/savePat` `401`, DigitalOcean's answer; `browser screenshot --feature F25.digitalocean-sign-in --name rejected-token`). Never type a real token here. `control-openhands browser click 'testid=add-backend-providers-back'` returns to the Providers panel (`browser count 'testid=managed-agents-sign-in'` is `0`; the providers panel itself stays mounted behind the sign-in screen, so its count is `1` either way); close with `testid=add-backend-close`. At phone width the sign-in screen fits (`bbox 'testid=add-backend-modal'` `insideViewport` `true`). Both errors also raise an error toast with the same text. Expected on a page served without a baked key (F26 `--host 0.0.0.0`, Docker), once the server's key was typed on the API-key screen: the same two answers as above. Known failure (reproduced 2026-10-14, #18434): every token, even the malformed one, ends in `session key required` (`POST /mars/rpc/getAuthState` and `savePat` `401`), because the web bridge client sends only the baked page key, never the typed one; it predates #18349 (BASE with `MARS_WEB=1` fails the same way). A `--frontend-only` page connected to another stack's backend fails the same way for a different reason (the typed key belongs to the other stack); #18434 leaves that case to a maintainer decision.- **Managed Agents with an account (`F25.digitalocean-agents`, `F25.digitalocean-backend-removal`).** Blocked: needs a DigitalOcean account with Managed Agents. With one, after a real token the panel lists agents (`digitalocean-agent-<id>`), **New Agent** opens `digitalocean-new-agent-form` (name, size), a session's Connect adds a backend and the sidebar shows `mars-session-panel`; removing that backend opens `mars-backend-removal-modal` with `disconnect-only-button` and `stop-session-button`.-- **Cloud backend (`F25.cloud-device-flow`, `F25.cloud-org-rows`, `F25.cloud-settings-link`, `F25.share-publicly`, `F25.cloud-sandbox-states`).** Only with a Cloud account (Preconditions); it creates a device grant and one conversation on that account, and shows the account's own conversations in the sidebar (keep those screenshots private). Reopen Add Backend on the Cloud tab, `browser click 'testid=add-backend-login-button'`, then `control-openhands browser tab 1` (the verify popup: `Device Authorization Request`, the code, **Authorize Device**), `control-openhands browser click 'role=button[name="Authorize Device"]'` (`Success!`) and `control-openhands browser tab 0`. Within about 15 s (token polling answers `400` until approved, then `200`) `browser count 'testid=add-backend-modal'` is `0`, the popup is gone and `browser snapshot 'testid=backend-selector'` reads `combobox "OpenHands Cloud – Personal Workspace"` (for about a second, until `/me` answers, the org's raw name shows in place of `Personal Workspace`); the registry holds `{name: "OpenHands Cloud", host: "https://app.all-hands.dev", kind: "cloud"}` (`browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({name:b.name,host:b.host,kind:b.kind}))"`). Open the selector: `option "Connected OpenHands Cloud – Personal Workspace"` (one option per organization) next to `Connected Local`; Manage Backends shows the row with `Personal Workspace` and the `Cloud` pill. `browser attr 'testid=backend-selector-settings-link' href` is `https://app.all-hands.dev/settings?org=<org id>` with `target` `_blank`; clicking it opens that tab (`browser wait-tab 'app.all-hands.dev/settings'`, then `browser close-tab 1`, `browser tab 0`). Start one conversation from Home: fill `testid=chat-input` with `Reply with exactly: QA_F25_CLOUD. Do not run any tools.` and `browser click 'testid=submit-button' --expect-url '/conversations/'`: the URL is `/conversations/task-<id>` while the sandbox starts (toasts `Creating conversation…`, `Sending...`), then moves to `/conversations/<id>?backend=<cloud id>&org=<org id>` and the reply `QA_F25_CLOUD` appears (about a minute). Expected: the user message is confirmed (no `Sending...`) and sits above the reply. Known failure (reproduced twice 2026-10-20, [#18717](https://github.com/OpenHands/OpenHands/issues/18717)): the reply and its hook rows render first and the user bubble below them keeps `Sending...`, turning into `Failed to send` with Retry and Dismiss after 150 s; `browser events --kinds websocket-created` shows an `after_seq` past the user `MessageEvent`; `browser reload` restores the order. Its `conversation-name >> ellipsis-button` menu has `share-publicly-button` (**Public Share**, off); do not flip it unless the conversation may be public. With only the Cloud backend registered (remove `Local` in Manage Backends; restore it afterwards with `browser reset` and `onboard --skip`), the option reads `Connected Personal Workspace` and the combobox `Personal Workspace`. Delete it with `delete-button` and `'role=dialog >> testid=confirm-button'`, then remove the Cloud backend in Manage Backends (`manage-backends-remove-OpenHands Cloud`, confirm): after `browser reload` the selector reads `Local`.+- **Cloud backend (`F25.cloud-device-flow`, `F25.cloud-org-rows`, `F25.cloud-settings-link`, `F25.share-publicly`, `F25.cloud-sandbox-states`).** Only with a Cloud account (Preconditions); it creates a device grant and one conversation on that account, and shows the account's own conversations in the sidebar (keep those screenshots private). Reopen Add Backend on the Cloud tab, `browser click 'testid=add-backend-login-button'`, then `control-openhands browser tab 1` (the verify popup: `Device Authorization Request`, the code, **Authorize Device**), `control-openhands browser click 'role=button[name="Authorize Device"]'` (`Success!`) and `control-openhands browser tab 0`. Within about 15 s (token polling answers `400` until approved, then `200`) `browser count 'testid=add-backend-modal'` is `0`, the popup is gone and `browser snapshot 'testid=backend-selector'` reads `combobox "OpenHands Cloud – Personal Workspace"` (for about a second, until `/me` answers, the org's raw name shows in place of `Personal Workspace`); the registry holds `{name: "OpenHands Cloud", host: "https://app.all-hands.dev", kind: "cloud"}` (`browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({name:b.name,host:b.host,kind:b.kind}))"`). Open the selector: `option "Connected OpenHands Cloud – Personal Workspace"` (one option per organization) next to `Connected Local`; Manage Backends shows the row with `Personal Workspace` and the `Cloud` pill. `browser attr 'testid=backend-selector-settings-link' href` is `https://app.all-hands.dev/settings?org=<org id>` with `target` `_blank`; clicking it opens that tab (`browser wait-tab 'app.all-hands.dev/settings'`, then `browser close-tab 1`, `browser tab 0`). Start one conversation from Home: fill `testid=chat-input` with `Reply with exactly: QA_F25_CLOUD. Do not run any tools.` and `browser click 'testid=submit-button' --expect-url '/conversations/'`: the URL is `/conversations/task-<id>` while the sandbox starts (toasts `Creating conversation…`, `Sending...`), then moves to `/conversations/<id>?backend=<cloud id>&org=<org id>` and the reply `QA_F25_CLOUD` appears (about a minute). The user message is confirmed (no `Sending...`) and sits above the reply: `browser eval` comparing `[data-testid=user-message]` and `[data-testid=agent-message]` with `compareDocumentPosition` puts the user bubble first, and `browser events --kinds websocket-created` shows the first connect replaying from `after_seq=-1` on a short conversation. (`wait-text` on the reply word also matches the prompt; wait for `testid=agent-message` instead.) Within a few seconds after the URL moves off `task-<id>`, and without a reload, the new conversation appears in the sidebar list (repeat until `browser eval "[...document.querySelectorAll('[data-testid=conversation-card]')].some(e=>e.outerHTML.includes(location.pathname.split('/')[2]))"` is `true`). Its `conversation-name >> ellipsis-button` menu has `share-publicly-button` (**Public Share**, off); do not flip it unless the conversation may be public. With only the Cloud backend registered (remove `Local` in Manage Backends; restore it afterwards with `browser reset` and `onboard --skip`), the option reads `Connected Personal Workspace` and the combobox `Personal Workspace`. Delete it with `delete-button` and `'role=dialog >> testid=confirm-button'`, then remove the Cloud backend in Manage Backends (`manage-backends-remove-OpenHands Cloud`, confirm): after `browser reload` the selector reads `Local`.- **Device verify page (`F25.device-verify-page`).** Run `control-openhands browser goto /oauth/device/verify` and `browser snapshot`: heading `Device Authorization`, `Enter the code displayed on your device:`, textbox `Device Code:` and button `Continue`, inside the normal app shell. `control-openhands browser fill 'role=textbox[name="Device Code:"]' QA-0000` and `browser click 'role=button[name="Continue"]'` go straight to the result: heading `Error`, `Failed to authorize device. Please try again.`, button `Try Again`; `browser network --filter verify-authenticated` shows `POST /oauth/device/verify-authenticated` `404`. Then `control-openhands browser goto '/oauth/device/verify?user_code=QA-0000'`: heading `Device Authorization Request`, `DEVICE CODE`, `QA-0000`, `Security Notice`, buttons `Cancel` and `Authorize Device` (screenshot `--feature F25.device-verify-page --name with-code`). `browser click 'role=button[name="Cancel"]'` does nothing in a tab the page did not open (URL unchanged). `browser click 'role=button[name="Authorize Device"]'` and `browser wait 'role=heading[name="Error"]'` show the same Error card; **Try Again** reloads to the request. In Light+ and Solarized Light (arrange `localStorage.setItem('openhands-color-theme', ...)` as in the Add Backend bullet), `?user_code=QA-NOT-REAL` keeps the Security Notice heading and instruction at least 4.5:1 on their amber tint, at desktop and phone width (2026-10-17: 5.03:1 and 13.58:1 in Light+, 5.53:1 and 11.84:1 in Solarized Light; `--name notice-light-plus-phone`). Known issue [#18527](https://github.com/OpenHands/OpenHands/issues/18527): current target warning text, Authorize normal/hover and error icons pass their respective contrast thresholds in all five themes at desktop/phone (minimum warning 5.03:1, body 11.80:1, button 10.78:1 and icon 4.62:1). This is a target-only observation, not paired attribution of the warning fix or retirement of #18527; device-flow error and Solarized awaiting text still reproduce failures in the separate entry above.- **Shared view (`F25.shared-conversation-view`).** Run `control-openhands browser goto /shared/conversations/<id>` and `control-openhands browser wait-text 'Conversation not found' --timeout 15000`. The page has no sidebar; `browser network --filter shared` shows `GET /api/shared-conversations` and `/api/shared-events/search` `404`, and `control-openhands browser toasts --history` stays `[]` (the page shows its own not-found state, no toast). Screenshot `--feature F25.shared-conversation-view --name local-not-found`.- **No Public Share locally (`F25.share-publicly`).** Run `browser goto /conversations/<id>`, `control-openhands browser click 'testid=conversation-name >> testid=ellipsis-button'` and `control-openhands browser count 'testid=share-publicly-button'`: `0` on a local backend (`browser testids --filter button` lists `rename-button`, `show-skills-button`, `show-agent-tools-button`, `export-transcript-button`, `download-trajectory-button`, `display-cost-button` and `delete-button`; `show-hooks-button` and `stop-button` appear only while the conversation is active (idle, running, waiting for confirmation or finished), so a paused or errored conversation has neither). The toggle, `copy-share-link-button` and `open-share-link-button` need a Cloud backend: blocked.@@ -150,7 +151,7 @@ Preconditions:- The add bullet's `network --external --filter 'ids='` check guards an earlier race: [#18390](https://github.com/OpenHands/OpenHands/issues/18390) tracks the same first-activation symptom; on a backend's first activation from `/conversations/<id>`, about 1 attempt in 5 asked the new backend for the old conversation id. It did not happen in 15 first activations at 4eff80237: each time `QA_Second` was removed, its consent unset with `control-openhands api PATCH /api/settings --run "$OH_VERIFY_RUN_2" --write --data '{"misc_settings_diff":{"app_preferences":{"user_consents_to_analytics":null}}}'` (arrange only) and the backend added again from `/conversations/<id>`. A backend that lacks an id answers `[null]`, which now reads as "not found" (only the `This conversation does not exist…` toast), not as incompatible data. No merged change was shown to fix the race itself (#18009 only changed how a `[null]` answer reads; #18024 only moved the redirect into a shared hook, still called after `addBackend`), so a hit (an `ids=` request to `<second-url>`) is a bug to file. At 95115e84f6dd it hit on 1 of 2 first activations from `/conversations/<id>` (one `GET <second-url>/api/conversations?ids=<id>` answered `200`, with no `This conversation does not exist…` toast after it). The hit followed a failed Connect in the same modal (an empty key, `Invalid API key`) before the successful one; the clean retry, which connected once, had none, so a repro should try both sequences. On 2026-10-08 at d2c89252d it did not reproduce in 17 first activations (8 clean, 9 failed-then-clean, some under CPU load and some human-paced): every `ids=` capture was empty, and no merged change touched the activation logic since the hit, so it stays a watch item rather than a filed defect. Record the Add bullet as `fail` when the request appears, even though the redirect and the selector look right. Known failure (reproduced 2026-10-13, #18390): two independent first activations hit it that day, each the run's first activation of `QA_Second` after a clean Connect. On 2026-10-13 at 2bdc0e9d6 it hit on 1 of 6 first activations from `/conversations/<id>`: the first one of the run (`GET <second-url>/api/conversations?ids=<id>` `200`, no toast), a clean Connect in a modal opened after earlier failed Connects in other Add Backend openings; the next 5 (2 failed-then-clean, 3 clean, plus one Manage → Add) had none.- Add Backend and Manage Backends take and keep keyboard focus, and Edit backend takes it on open (`ModalBackdrop`; at phone width open the selector with `'testid=sidebar-mobile-drawer >> testid=backend-selector'`, since the bare test id matches two selectors). Focus returns to the selector input when they close (at phone width the drawer's selector, with the drawer still open), so an Escape pressed after closing lands on the selector and can trigger [#18233](https://github.com/OpenHands/OpenHands/issues/18233) below.- Toasts are not page or HTTP errors, so `browser errors --app-only` misses them: read `browser toasts --history` after every add or switch, before anything reloads the page (the history starts with each page load).-- `F25.locked-cloud` needs `scripts/static-server.mjs --lock-to-cloud <url>`, which neither `bin/agent-canvas.mjs` nor `control-openhands launch` can pass today; even with it, reconnecting needs a Cloud account.+- `F25.locked-cloud` needs `scripts/static-server.mjs --lock-to-cloud <url>`, which neither `bin/agent-canvas.mjs` nor `control-openhands launch` can pass: start it by hand as in F01's **Locked Cloud first run** bullet, which also signs in (a Cloud account). On that page `browser click 'testid=backend-selector'`: `browser text 'testid=manage-backends-menu-item'` is `Reconnect to Cloud` and `browser count 'testid=add-backend-menu-item'` is `0`; the command menu lists only Settings and Application settings under Settings.- The Cloud tab title truncates to `OpenHan…` at 390 px when the Agent-server tab is selected; the layout still fits. With the Providers tab present all three titles are cut to three letters.- A token DigitalOcean rejects surfaces as the bare `401 Unauthorized`, with no hint that the token is wrong or lacks scopes.- Known issue [#18029](https://github.com/OpenHands/OpenHands/issues/18029) (`F25.locked-cloud`): A returning user in a locked Cloud deployment can reach the no-backend placeholder and fail to start conversations despite a healthy configured backend. The report proposes a stale document or alternate origin but does not capture which occurred. Recheck against the affected locked deployment with a Cloud session, its exact origin, cache response headers and injected host state; the normal local launcher does not reproduce this deployment condition.diff --git a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.mdindex c1bdcb4e80..d4f12e48b0 100644--- a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md+++ b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md@@ -35,13 +35,13 @@ Source: `bin/agent-canvas.mjs`, `scripts/dev-with-automation.mjs`, `scripts/dev-- `F26.host-bind-lan-optin`: the opt-in the launcher's warning recommends (`--allow-lan-session-key`) should restore key injection on a LAN bind. Known failure (reproduced 2026-10-08): the npm launcher ignores the flag and the page still carries only `__AGENT_CANVAS_AUTH_REQUIRED__`; [#17949](https://github.com/OpenHands/OpenHands/issues/17949).- `F26.host-bind-env`: `OH_BIND_HOST=0.0.0.0` (the environment form of `--host`) prints the same two "not loopback" warnings and does not inject the session key, even with `LOCAL_BACKEND_API_KEY` set.- `F26.automation-startup-failure`: the launcher announces the stack (the **Main UI** banner) without waiting for automation, then watches it: when the automation backend exits before it is healthy, the launcher prints `✗ The automation backend exited during startup (code <N>). Its [automation] log lines show why.`, stops every service it started and exits with code 1, leaving its ports closed. A slow start that only times out keeps the stack up.-- `F26.mars-web-bridge`: every launcher-served web stack hosts the DigitalOcean Managed Agents bridge on its ingress by default (`MARS_WEB=0` opts out): `/mars/health` answers `{"ok":true,"authRequired":true}` on a full, `--backend-only`, `--frontend-only` and `--host 0.0.0.0` launcher, `/mars/rpc/*` answers `401` without the session key and works with it, the static child behind the ingress hosts none, and the Electron stack hosts none either (its preload provides `window.marsBridge`).+- `F26.mars-web-bridge`: the DigitalOcean Managed Agents web bridge is experimental and opt-in: a launcher started without `MARS_WEB=1` hosts none (`/mars/health` falls through to the SPA's HTML on a full or `--frontend-only` launcher and to `503 No backend configured for this route` on `--backend-only`, and the page has no `window.marsBridge`). With `MARS_WEB=1` the ingress hosts it: `/mars/health` answers `{"ok":true,"authRequired":true}`, `/mars/rpc/*` answers `401` without the session key and works with it, the static child behind the ingress hosts none, and the Electron stack hosts none either (its preload provides `window.marsBridge`).- `F26.runtime-services`: `/server_info.runtime_services` lists the agent-server, ingress, frontend and automation URLs as the agent sees them, and a new conversation's system prompt carries a matching `<RUNTIME_SERVICES>` block.- `F26.runtime-services-agent-use`: with that block alone, the agent can reach the automation backend from its terminal: asked to, it reads the OpenAPI at the URL the block names, creates an enabled cron automation, and dispatches a run that completes; the automation and its run show up in the Automations UI.-- `F26.lib-build`: `npm run build:lib` produces `dist/` where every `package.json` `exports` target exists and the main entry exports `AgentServerUIProviders`, `AgentServerUIRoot`, `CloudOrganizationBoundary` and the telemetry helpers.+- `F26.lib-build`: `npm run build:lib` produces `dist/` where every `package.json` `exports` target exists (including the scoped stylesheet `@openhands/agent-canvas/styles.css`, `dist/styles.css`) and the main entry exports `AgentServerUIProviders`, `AgentServerUIRoot`, `CloudOrganizationBoundary` and the telemetry helpers.- `F26.lib-style-scope`: all bundled CSS is scoped under `[data-agent-server-ui]`, and theme tokens such as `--oh-color-base` live on that scope root (observable in the standalone app).-- `F26.lib-host-app`: the built ESM library mounts in a separate Vite browser host that imports `@openhands/agent-canvas` by bare package name with matching React peers: `AgentServerUIRoot` renders its children and they stay interactive, with no module error. Canvas styles apply inside the `[data-agent-server-ui]` scope only and `styleOverrides` restyle that subtree. Known failure (reproduced 2026-10-19, [#18655](https://github.com/OpenHands/OpenHands/issues/18655)): host elements stay unaffected and `styleOverrides` set the variable on the scope root, but the library ships no Canvas utility or theme stylesheet (`dist/` holds only the 5 KB xterm CSS and no JS imports a stylesheet), so a mounted Canvas component (`EmptyBrowserMessage`) renders unstyled and the override has no visible effect.-- `F26.docker-image`: `docker run ghcr.io/openhands/agent-canvas` serves Canvas at `/canvas`; without `AGENT_CANVAS_ALLOW_LAN_SESSION_KEY=true` the UI asks for the key. Blocked without a Docker daemon. On a host whose kernel has no IPv6 (no `/proc/net/if_inet6`) the entrypoint logs `IPv6 is not available; binding the frontend to 0.0.0.0.` and the container keeps running (#18180).+- `F26.lib-host-app`: the built ESM library mounts in a separate Vite browser host that imports `@openhands/agent-canvas` by bare package name with matching React peers: `AgentServerUIRoot` renders its children and they stay interactive, with no module error. With `import "@openhands/agent-canvas/styles.css"` loaded once by the host, Canvas styles apply inside the `[data-agent-server-ui]` scope only (a mounted `EmptyBrowserMessage` is laid out and coloured, host elements keep browser defaults) and `styleOverrides` restyle that subtree.+- `F26.docker-image`: `docker run ghcr.io/openhands/agent-canvas` serves Canvas at `/canvas`; without `AGENT_CANVAS_ALLOW_LAN_SESSION_KEY=true` the UI asks for the key. Its `/server_info` reports the agent-server version pinned in `config/defaults.json`, the agent-server reads public skills from the image's bundled `@openhands/extensions` package, and the image hosts no Managed Agents bridge unless started with `-e MARS_WEB=1`. Blocked without a Docker daemon. On a host whose kernel has no IPv6 (no `/proc/net/if_inet6`) the entrypoint logs `IPv6 is not available; binding the frontend to 0.0.0.0.` and the container keeps running (#18180).- `F26.docker-conversation-runtime`: `OH_CONVERSATION_RUNTIME=docker` runs each new conversation in its own container. Blocked without a Docker daemon.- `F26.helm-chart`: `helm install agent-canvas ./helm/agent-canvas` runs the Docker image as a Kubernetes StatefulSet with a PVC and an Ingress. Blocked without `helm` and a cluster.- `F26.desktop-boot-splash`: the Electron app first shows a dark splash (logo, **OpenHands Agent Canvas**, spinner, `Starting backend services…`, then live service-log lines, a first-launch note and **Show details**).@@ -72,7 +72,7 @@ Preconditions:- **Version (`F26.cli-version`).** Run `node bin/agent-canvas.mjs --version; echo "exit=$?"` and `node bin/agent-canvas.mjs -v; echo "exit=$?"`. Each prints one line equal to `jq -r .version package.json` (the checkout's package version, which changes with every release: compare, do not expect a number; e.g. `[ "$(node bin/agent-canvas.mjs --version)" = "$(jq -r .version package.json)" ] && echo same`) and `exit=0`.- **Info (`F26.cli-info`).** Run `node bin/agent-canvas.mjs --info`. It prints `@openhands/agent-canvas <version>` (the `--version` value), `Default stack versions:` with `agent-server: <pin>` and `automation: <pin>`, `Compatibility:` `agent-server: >= <minimum>`, `Default ports:` `ingress: 8000`, `agent-server: 18000`, `automation: 18001`, and the override variables `OH_AGENT_SERVER_VERSION, OH_AGENT_SERVER_GIT_REF, OH_AGENT_SERVER_LOCAL_PATH` / `OH_AUTOMATION_VERSION, OH_AUTOMATION_GIT_REF`. The three versions are read from `config/defaults.json` (`versions.agentServer`, `versions.automation`, `compatibility.minimumAgentServer`) and move with every pin bump, so compare them with the file instead of expecting numbers: `test "$(node bin/agent-canvas.mjs --info | awk '/^(Default stack versions|Compatibility):/{s=1;next} /^$/{s=0} s{print $NF}' | paste -sd' ' -)" = "$(jq -r '[.versions.agentServer,.versions.automation,.compatibility.minimumAgentServer]|join(" ")' config/defaults.json)" && echo match || echo mismatch` prints `match`, and `printf '%s\n' "$(jq -r .compatibility.minimumAgentServer config/defaults.json)" "$(jq -r .versions.agentServer config/defaults.json)" | sort -VC && echo pin-meets-minimum || echo below-minimum` prints `pin-meets-minimum` (the default Agent Server passes the UI's own version gate).-- **Help (`F26.cli-help`).** Run `node bin/agent-canvas.mjs --help`. It prints `USAGE:`, `AUTH MODES:` (`--public ... Users must paste it when the UI loads.`), `OPTIONS:` (`-p, --port`, `-H, --host`, `--public`, `--frontend-only`, `--backend-only`, `-v, --version`, `--info`, `-h, --help`), `ENVIRONMENT VARIABLES:` and `EXAMPLES:`, exit 0.+- **Help (`F26.cli-help`).** Run `node bin/agent-canvas.mjs --help`. It prints `USAGE:`, `AUTH MODES:` (`--public ... Users must paste it when the UI loads.`), `OPTIONS:` (`-p, --port`, `-H, --host`, `--public`, `--frontend-only`, `--backend-only`, `-v, --version`, `--info`, `-h, --help`), `ENVIRONMENT VARIABLES:` (including `UV_PYTHON` `Python version/range for uvx-run services`) and `EXAMPLES:`, exit 0. The launcher starts agent-server and automation with `uvx --python >=3.12,<3.15` (the `python.uvxRequest` in `config/defaults.json`; `UV_PYTHON` replaces it): `ps -eo args | grep -v grep | grep -o 'uvx --python [^ ]*' | sort -u` on a running stack prints `uvx --python >=3.12,<3.15`.- **Conflicting flags (`F26.cli-flag-conflicts`).** Run `node bin/agent-canvas.mjs --frontend-only --backend-only; echo "exit=$?"` → `Error: --frontend-only and --backend-only cannot be used together`, `exit=1`. Run `node bin/agent-canvas.mjs --public --frontend-only; echo "exit=$?"` → `Error: --public cannot be used with --frontend-only`, `exit=1`.- **Public mode needs a key (`F26.cli-public-needs-key`).** Run `T=$(mktemp -d); env -u LOCAL_BACKEND_API_KEY HOME=$T OH_CANVAS_SAFE_STATE_DIR=$T/state timeout 60 node bin/agent-canvas.mjs --public --port 18990; echo "exit=$?"; rm -rf $T`. After `✓ uvx found` it prints `✗ PUBLIC MODE requires LOCAL_BACKEND_API_KEY environment variable.` and `exit=1`. (The successful `--public` launch is `control-openhands launch --new --public`, mapped in F01.)- **Missing build (`F26.cli-missing-build`).** Never move this checkout's `build/` (other runs serve it); copy the launcher instead: `T=$(mktemp -d); mkdir -p $T/bin $T/config; cp bin/agent-canvas.mjs $T/bin/; cp package.json $T/; cp config/defaults.json $T/config/; node $T/bin/agent-canvas.mjs --port 18990; echo "exit=$?"; rm -rf $T`. It prints `Error: No build found at <T>/build`, the `npm install` / `npm run build` hint and `exit=1`.@@ -84,9 +84,9 @@ Preconditions:- **Remote backend from a normal Canvas (`F26.remote-backend`).** Needs the backend-only instance. Run `control-openhands browser goto /`, `control-openhands browser click 'testid=backend-selector'`, `control-openhands browser click 'testid=add-backend-menu-item'`, `control-openhands browser click 'testid=add-backend-option-agent-server'` and `control-openhands browser click 'testid=add-backend-location-option-remote'`. `control-openhands browser text 'testid=add-backend-agent-server-panel'` includes `Run the remote backend with --public, set a strong LOCAL_BACKEND_API_KEY, and expose it through an SSH tunnel, ngrok, or a TLS reverse proxy.` Fill `testid=add-backend-name` with `QA_Remote`, `testid=add-backend-host` with `http://127.0.0.1:18960` and `testid=add-backend-api-key` with `qa-f26-backend-key`, click `testid=add-backend-submit`, then `control-openhands browser wait 'testid=add-backend-modal' --state detached --timeout 15000`, `control-openhands browser reload` and `control-openhands browser snapshot 'testid=backend-selector'`: `status "Connected"`, `combobox "QA_Remote"` (the new backend becomes active). If you run this bullet without Cross-connect first, the backend-only state has not answered telemetry consent yet: the consent dialog (`This preference is saved for the local backend “QA_Remote” at http://127.0.0.1:18960.`) opens over the page and blocks the selector; answer it as in Cross-connect (`control-openhands browser wait 'testid=telemetry-consent-form' --timeout 10000`, `control-openhands browser uncheck 'testid=telemetry-consent-form >> role=checkbox'`, `control-openhands browser click 'testid=confirm-telemetry-preferences'`). Switch back with `control-openhands browser click 'testid=backend-selector'` and `control-openhands browser click 'role=option[name="Local"]'`, then `control-openhands browser wait 'testid=backend-selector >> role=combobox[name="Local"]' --timeout 5000` (the switch applies after a short delay: a snapshot taken right after the click still reads `combobox "QA_Remote": Local`; after the wait it shows `combobox "Local"`). Clean up: `backend-selector` → `testid=manage-backends-menu-item`; `control-openhands browser text 'testid=manage-backends-row-QA_Remote'` reads `QA_Remote`, `v<version>`, `http://127.0.0.1:18960`, `Connected`, `LOCAL`, where `<version>` is the backend-only instance's own report, `curl -s http://127.0.0.1:18960/server_info | jq -r .version` (the `versions.agentServer` pin in `config/defaults.json` unless your shell exports an `OH_AGENT_SERVER_*` override, which a hand-started launcher inherits); it passes the UI's gate: `printf '%s\n' "$(jq -r .compatibility.minimumAgentServer config/defaults.json)" "$(curl -s http://127.0.0.1:18960/server_info | jq -r .version)" | sort -VC && echo meets-minimum || echo below-minimum` prints `meets-minimum`. Then click `testid=manage-backends-remove-QA_Remote`, `testid=confirmation-modal >> testid=confirm-button`, `testid=manage-backends-done`, `browser reload`.- **Seeded Local backend (`F26.seeded-local-backend`).** On your run, `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({id:b.id,name:b.name,host:b.host,kind:b.kind,hasKey:!!b.apiKey}))"` returns exactly one entry `{id: "default-local", name: "Local", host: "http://127.0.0.1:<ingress>", kind: "local", hasKey: true}` (after the Remote cleanup above), and `control-openhands browser eval "({key: Boolean(window.__AGENT_CANVAS_SESSION_API_KEY__), authRequired: window.__AGENT_CANVAS_AUTH_REQUIRED__ === true})"` is `{key: true, authRequired: false}`.- **Rotated session key (`F26.session-key-rotated`).** Still on your run's `/`, `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends'))[0].apiKey === window.__AGENT_CANVAS_SESSION_API_KEY__"` is `true` (compare keys inside the page; never print one). Run `md5sum $OH_VERIFY_RUN/private/session-key | cut -c1-12`, then `control-openhands restart --rotate-key` (`rotatedKey` `true`: the launcher comes back injecting a new key) and the `md5sum` line again; the hash changed. The open page still holds the old key, so its polls get `401` until it reloads: run `control-openhands browser errors --clear`, then `control-openhands browser reload` and `control-openhands browser wait 'testid=home-screen' --timeout 20000`. `control-openhands browser count` is `1` for `testid=root-layout` and `0` for `testid=api-key-entry-screen`, `testid=agent-server-onboarding-screen` and `testid=onboarding-modal`: Home, not the key prompt public mode shows in the same state (`F01.api-key-entry`). Second view: `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).map(b=>({id:b.id,host:b.host,keyMatchesPage:b.apiKey===window.__AGENT_CANVAS_SESSION_API_KEY__}))"` is `[{id: "default-local", host: "http://127.0.0.1:<ingress>", keyMatchesPage: true}]` against the new page key, `control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-agent-server-config')).sessionApiKey === window.__AGENT_CANVAS_SESSION_API_KEY__"` is `true` (the legacy key is overwritten too), `control-openhands browser network --filter 'api/settings' --last 2` shows `status` `200` under `recent`, and `control-openhands browser errors --app-only` has `pageErrors` `0` and `appErrors` `0`. `control-openhands api GET /api/settings` works with the rotated key, and `control-openhands browser screenshot --feature F26.session-key-rotated --name home-after-rotation` shows Home.-- **Managed Agents bridge (`F26.mars-web-bridge`).** With both partial stacks up: `curl -s "$(control-openhands status | jq -r .baseUrl)/mars/health"`, `curl -s http://127.0.0.1:18960/mars/health` and `curl -s http://127.0.0.1:18970/mars/health` each print `{"ok":true,"authRequired":true}`, while `curl -s "http://127.0.0.1:$(control-openhands status | jq -r .ports.frontend)/mars/health" | head -c 15` prints `<!DOCTYPE html>` (the static child behind your run's ingress has no second bridge). `curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:18960/mars/rpc/getAuthState` is `401` (no key). The frontend-only page has the bridge but no key to use it: after the cross-connect, `control-openhands browser eval "({marsBridge: !!window.marsBridge, key: Boolean(window.__AGENT_CANVAS_SESSION_API_KEY__)})"` on `http://127.0.0.1:18970/` is `{marsBridge: true, key: false}` (what that does to DigitalOcean sign-in is F25's `F25.digitalocean-sign-in` known failure).+- **Managed Agents bridge (`F26.mars-web-bridge`).** With both partial stacks up (neither started with `MARS_WEB`): `curl -s "$(control-openhands status | jq -r .baseUrl)/mars/health" | head -c 15` and `curl -s http://127.0.0.1:18970/mars/health | head -c 15` print `<!DOCTYPE html>`, `curl -s http://127.0.0.1:18960/mars/health` prints `No backend configured for this route` (503), and `curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:18970/mars/rpc/getAuthState` is `404`: no bridge without the opt-in, and `control-openhands browser eval '!!window.marsBridge'` on your run is `false`. Opt-in: start the bridge stack from F25's Preconditions (`MARS_WEB=1`, key `qa-f25-mars-key`, port 18960, after stopping the partial stacks): `curl -s http://127.0.0.1:18960/mars/health` is `{"ok":true,"authRequired":true}`, `POST /mars/rpc/getAuthState` with `-H 'Content-Type: application/json' -d '{"args":[]}'` is `401` without and `200` with `-H 'X-Session-API-Key: qa-f25-mars-key'`, `curl -s http://127.0.0.1:18963/mars/health | head -c 15` (its static child) prints `<!DOCTYPE html>`, and on `http://127.0.0.1:18960/` `browser eval '!!window.marsBridge'` is `true`. Stop it with `kill -TERM $(cat $Q/pid)`.- **Stop the partial stacks.** `kill -TERM $(cat $OH_VERIFY_RUN/private/f26-frontend/pid) $(cat $OH_VERIFY_RUN/private/f26-backend/pid)`; after about 5 s the ports 18960–18963 and 18970–18973 are free again (check as in Preconditions). Kill by the saved pid only, never by pattern.-- **LAN bind (`F26.host-bind-lan`).** Start a full stack on all interfaces: `Q=$OH_VERIFY_RUN/private/f26-lan; mkdir -p $Q; HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 LOCAL_BACKEND_API_KEY=qa-f26-lan-key OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 nohup node bin/agent-canvas.mjs --host 0.0.0.0 --port 18960 > $Q/launcher.log 2>&1 & echo $! > $Q/pid`, and wait until `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/` is `200`. `grep -a 'not loopback' $Q/launcher.log` shows `[auth] Bind host 0.0.0.0 is not loopback — session key will not be injected into HTML` and `[static] WARNING: bind host 0.0.0.0 is not loopback; refusing to inject the session API key into HTML.` The served page proves it: `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u` prints only `__AGENT_CANVAS_AUTH_REQUIRED__` and `curl -s http://127.0.0.1:18960/ | grep -c qa-f26-lan-key` prints `0`, while your loopback run's page, `curl -s "$(control-openhands status | jq -r .baseUrl)/" | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u`, prints only `__AGENT_CANVAS_SESSION_API_KEY__`. The bind is real: with `LAN` from the loopback bullet, `curl -s -o /dev/null -w '%{http_code}\n' --connect-timeout 3 http://$LAN:18960/` is `200` here (refused on your run, `F26.loopback-bind-default`; with an empty `LAN` this check is `not-run`). The browser part below expects a profile that has never visited `127.0.0.1:18960` (true in this file's order; on a repeat pass run `control-openhands browser reset` first and restore your run afterwards, see Gotchas). Then `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser eval "({key: Boolean(window.__AGENT_CANVAS_SESSION_API_KEY__), authRequired: window.__AGENT_CANVAS_AUTH_REQUIRED__ === true})"` → `{key: false, authRequired: true}`. The onboarding shows `testid=onboarding-step-check-backend` with `control-openhands browser enabled 'testid=onboarding-backend-next'` `false` (no key typed). `control-openhands browser click 'testid=onboarding-skip'` and `control-openhands browser wait 'testid=api-key-entry-screen' --timeout 15000` succeed; after `sleep 12`, `control-openhands browser toasts --history` is `[]` (no error toast on either screen); the bridge is mounted behind the key (`F26.mars-web-bridge`): `curl -s http://127.0.0.1:18960/mars/health` is `{"ok":true,"authRequired":true}`, and `POST /mars/rpc/getAuthState` with `-H 'Content-Type: application/json' -d '{"args":[]}'` is `401` without and `200` with `-H 'X-Session-API-Key: qa-f26-lan-key'`; `control-openhands browser screenshot --feature F26.host-bind-lan --name api-key-screen`. Fill `testid=api-key-entry-name` with `QA_LAN` and `testid=api-key-entry-api-key` with `qa-f26-lan-key`, click `testid=api-key-entry-submit`; `control-openhands browser wait 'testid=root-layout' --timeout 10000` succeeds. A few seconds later the telemetry consent dialog for the new `QA_LAN` backend opens and blocks clicks; answer it (`role=button[name="Confirm preferences"]`) before driving that page further. Stop it with `kill -TERM $(cat $Q/pid)`.+- **LAN bind (`F26.host-bind-lan`).** Start a full stack on all interfaces: `Q=$OH_VERIFY_RUN/private/f26-lan; mkdir -p $Q; HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 LOCAL_BACKEND_API_KEY=qa-f26-lan-key OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 nohup node bin/agent-canvas.mjs --host 0.0.0.0 --port 18960 > $Q/launcher.log 2>&1 & echo $! > $Q/pid`, and wait until `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/` is `200`. `grep -a 'not loopback' $Q/launcher.log` shows `[auth] Bind host 0.0.0.0 is not loopback — session key will not be injected into HTML` and `[static] WARNING: bind host 0.0.0.0 is not loopback; refusing to inject the session API key into HTML.` The served page proves it: `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u` prints only `__AGENT_CANVAS_AUTH_REQUIRED__` and `curl -s http://127.0.0.1:18960/ | grep -c qa-f26-lan-key` prints `0`, while your loopback run's page, `curl -s "$(control-openhands status | jq -r .baseUrl)/" | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u`, prints only `__AGENT_CANVAS_SESSION_API_KEY__`. The bind is real: with `LAN` from the loopback bullet, `curl -s -o /dev/null -w '%{http_code}\n' --connect-timeout 3 http://$LAN:18960/` is `200` here (refused on your run, `F26.loopback-bind-default`; with an empty `LAN` this check is `not-run`). The browser part below expects a profile that has never visited `127.0.0.1:18960` (true in this file's order; on a repeat pass run `control-openhands browser reset` first and restore your run afterwards, see Gotchas). Then `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser eval "({key: Boolean(window.__AGENT_CANVAS_SESSION_API_KEY__), authRequired: window.__AGENT_CANVAS_AUTH_REQUIRED__ === true})"` → `{key: false, authRequired: true}`. The onboarding shows `testid=onboarding-step-check-backend` with `control-openhands browser enabled 'testid=onboarding-backend-next'` `false` (no key typed). `control-openhands browser click 'testid=onboarding-skip'` and `control-openhands browser wait 'testid=api-key-entry-screen' --timeout 15000` succeed; after `sleep 12`, `control-openhands browser toasts --history` is `[]` (no error toast on either screen); without `MARS_WEB=1` no bridge is mounted (`F26.mars-web-bridge`): `curl -s http://127.0.0.1:18960/mars/health | head -c 15` prints `<!DOCTYPE html>`; `control-openhands browser screenshot --feature F26.host-bind-lan --name api-key-screen`. Fill `testid=api-key-entry-name` with `QA_LAN` and `testid=api-key-entry-api-key` with `qa-f26-lan-key`, click `testid=api-key-entry-submit`; `control-openhands browser wait 'testid=root-layout' --timeout 10000` succeeds. A few seconds later the telemetry consent dialog for the new `QA_LAN` backend opens and blocks clicks; answer it (`role=button[name="Confirm preferences"]`) before driving that page further. Stop it with `kill -TERM $(cat $Q/pid)`.- **LAN opt-in (`F26.host-bind-lan-optin`).** Expected: the opt-in the warning names (`Pass --host 127.0.0.1 (default) for local mode, or --allow-lan-session-key only if you accept LAN exposure.`) makes the npm launcher inject the key again. Start the LAN stack as above with `--host 0.0.0.0 --allow-lan-session-key --port 18960` (log to `$Q/launcher2.log`), wait for `200`, then `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_\(SESSION_API_KEY\|AUTH_REQUIRED\)__' | sort -u`. Today it prints only `__AGENT_CANVAS_AUTH_REQUIRED__` and the log repeats both "not loopback" warnings: the launcher ignores the flag (fail; see Gotchas). Stop it with `kill -TERM $(cat $Q/pid)`.- **Generated session key persists (`F26.session-key-persist`).** With 18960–18963 free, define a helper and start a loopback stack without a key: `Q=$OH_VERIFY_RUN/private/f26-keys; mkdir -p $Q; start(){ HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state OH_CANVAS_SAFE_BACKEND_PORT=18961 OH_CANVAS_SAFE_AUTOMATION_PORT=18962 OH_CANVAS_SAFE_VITE_PORT=18963 OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 "$@" > $Q/launcher.log 2>&1 & echo $! > $Q/pid; for i in $(seq 1 60); do c=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:18960/); [ "$c" = 200 ] && break; sleep 3; done; echo "code=$c"; }; start env -u LOCAL_BACKEND_API_KEY nohup node bin/agent-canvas.mjs --port 18960` (`code=200`). `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_SESSION_API_KEY__[^;<]*' | md5sum` prints a hash (never print the key itself); `$Q/home/.openhands/agent-canvas/api-key.txt` exists (64 hex characters) and `curl -s -o /dev/null -w '%{http_code}' -H "X-Session-API-Key: $(cat $Q/home/.openhands/agent-canvas/api-key.txt)" http://127.0.0.1:18960/api/settings` is `200`. `kill -TERM $(cat $Q/pid); sleep 6`, run the same `start ...` line again in the same shell and repeat the `md5sum` line: the hash is identical.- **Pinned session key (`F26.session-key-pinned`).** Same shell, after `kill -TERM $(cat $Q/pid); sleep 6`: `start env LOCAL_BACKEND_API_KEY=qa-f26-pin-key nohup node bin/agent-canvas.mjs --port 18960`. `curl -s http://127.0.0.1:18960/ | grep -o '__AGENT_CANVAS_SESSION_API_KEY__[^;<]*'` prints `__AGENT_CANVAS_SESSION_API_KEY__="qa-f26-pin-key"`; `/api/settings` is `200` with `-H 'X-Session-API-Key: qa-f26-pin-key'` and `401` with the saved generated key. In the browser, `control-openhands browser goto http://127.0.0.1:18960/ --allow-external` and `control-openhands browser eval "({key: window.__AGENT_CANVAS_SESSION_API_KEY__ === 'qa-f26-pin-key', authRequired: window.__AGENT_CANVAS_AUTH_REQUIRED__ === true})"` → `{key: true, authRequired: false}`.@@ -96,11 +96,11 @@ Preconditions:- **Runtime services (`F26.runtime-services`).** Run `control-openhands api GET /server_info --pick runtime_services.mode` (`agent-canvas`) and `control-openhands api GET /server_info --pick runtime_services.services.automation.url_from_agent` (`http://localhost:<automation port>`; `control-openhands status` prints the ports). Then `control-openhands conversation start --prompt "Reply with the single word: ok" --wait --timeout 180` (prints `<id>`) and `control-openhands conversation events <id> --grep RUNTIME_SERVICES --from-start`: the `SystemPromptEvent` matches, with an excerpt `<RUNTIME_SERVICES>\nYou are running inside an agent-canvas dev stack started ...`. `control-openhands conversation events <id> --grep "localhost:<automation port>" --from-start` shows `* Automation backend: http://localhost:<port>` plus its `Docs:` and `OpenAPI:` URLs.- **The agent uses the runtime services (`F26.runtime-services-agent-use`).** Write `qa-f26-agent.txt` (outside the workspace) containing: ``Using the Automation backend described in your RUNTIME_SERVICES (read its OpenAPI to find the routes), create one automation named QA_rt_auto from a prompt: the prompt is "Reply with the single word: pong. Do not run any tools.", the schedule is the cron 0 9 * * *, and it is enabled. Then dispatch one run of it now. Do not change anything else. Reply with the automation id and the run id only.`` Run `control-openhands conversation start --prompt "$(cat qa-f26-agent.txt)" --wait --timeout 400` (note `<agent-id>`); the reply names `<automation-id>` and a run id. `control-openhands conversation events <agent-id> --kinds ActionEvent --from-start` shows the agent's own `curl` calls: `http://localhost:<automation port>/api/automation/openapi.json`, `POST …/api/automation/v1/preset/prompt` and `POST …/api/automation/v1/<automation-id>/dispatch`, each with the `X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY` header the block names. Second view in the UI: `control-openhands browser goto /automations` and `control-openhands browser text '[data-testid^=automation-card-] >> has-text=QA_rt_auto'` (`QA_rt_auto`, the prompt, `cron`, `Runs 1`). Open it with `control-openhands browser click '[data-testid^=automation-card-] >> has-text=QA_rt_auto >> text=QA_rt_auto' --expect-url '/automations/[0-9a-f-]+'`. Within about a minute, `control-openhands browser text 'testid=automation-activity-log'` shows the run's summary (`Replied with the single word "pong" …`), a cost and `Successful` (`control-openhands browser screenshot --feature F26.runtime-services-agent-use --name agent-created-automation-run`). `control-openhands api GET /api/automation/v1/<automation-id>` has `name` `QA_rt_auto`, `trigger.schedule` `0 9 * * *` and `enabled` `true`. Delete it afterwards (`control-openhands api DELETE /api/automation/v1/<automation-id> --write`, arrange). It runs daily at 09:00 UTC otherwise.-- **Library build (`F26.lib-build`).** Never build inside the shared checkout (it regenerates i18n and typegen files the other runs hash); build in a copy: `L=$OH_VERIFY_RUN/private/f26-lib; mkdir -p $L; tar --exclude=./node_modules --exclude=./build --exclude=./.git --exclude=./dist -cf - . | tar -xf - -C $L; ln -s "$PWD/node_modules" $L/node_modules; (cd $L && npm run build:lib); echo "exit=$?"`. It ends with `✓ built in` and `exit=0` (about 30 s). Then `(cd $L && node -e 'const p=require("./package.json"),fs=require("fs");let ok=0,miss=[];for(const[k,v]of Object.entries(p.exports))for(const f of(typeof v==="string"?[v]:Object.values(v)))fs.existsSync(f)?ok++:miss.push(k+" -> "+f);console.log(JSON.stringify({entries:Object.keys(p.exports),ok,missing:miss}))')` prints 9 entries (`.`, `./browser`, `./conversation`, `./files`, `./settings`, `./sidebar`, `./terminal`, `./i18n`, `./package.json`), `ok` 25 and `missing` `[]`. `grep -ohE 'AgentServerUIProviders|AgentServerUIRoot|CloudOrganizationBoundary|configureTelemetry|setTelemetryConsent' $L/dist/lib/index.d.ts | sort -u` lists all five. Remove the copy with `rm -rf $L` (after the host-app bullet below when it runs).+- **Library build (`F26.lib-build`).** Never build inside the shared checkout (it regenerates i18n and typegen files the other runs hash); build in a copy: `L=$OH_VERIFY_RUN/private/f26-lib; mkdir -p $L; tar --exclude=./node_modules --exclude=./build --exclude=./.git --exclude=./dist -cf - . | tar -xf - -C $L; ln -s "$PWD/node_modules" $L/node_modules; (cd $L && npm run build:lib); echo "exit=$?"`. It ends with `✓ built in` and `exit=0` (about 30 s). Then `(cd $L && node -e 'const p=require("./package.json"),fs=require("fs");let ok=0,miss=[];for(const[k,v]of Object.entries(p.exports))for(const f of(typeof v==="string"?[v]:Object.values(v)))fs.existsSync(f)?ok++:miss.push(k+" -> "+f);console.log(JSON.stringify({entries:Object.keys(p.exports),ok,missing:miss}))')` prints 10 entries (`.`, `./browser`, `./conversation`, `./files`, `./settings`, `./sidebar`, `./terminal`, `./i18n`, `./styles.css`, `./package.json`), `ok` 26 and `missing` `[]`; `$L/dist/styles.css` is about 440 KB. `grep -ohE 'AgentServerUIProviders|AgentServerUIRoot|CloudOrganizationBoundary|configureTelemetry|setTelemetryConsent' $L/dist/lib/index.d.ts | sort -u` lists all five. Remove the copy with `rm -rf $L` (after the host-app bullet below when it runs).- **Style scope (`F26.lib-style-scope`).** On your run's home page (`control-openhands browser goto /`) run `control-openhands browser eval "(() => { let total=0, unscoped=[]; for (const sh of document.styleSheets) { let rules; try { rules = sh.cssRules } catch { continue } const walk = (rs) => { for (const r of rs) { if (r.selectorText) { total++; if (!r.selectorText.includes('data-agent-server-ui')) unscoped.push(r.selectorText) } else if (r.cssRules) walk(r.cssRules) } }; walk(rules) } return { scopeRoots: document.querySelectorAll('[data-agent-server-ui]').length, total, unscopedCount: unscoped.length, sample: unscoped.slice(0, 12) } })()"`. `total` is in the thousands and `unscopedCount` is `2`: only `.go<hash>` and `.go<hash> > *`, react-hot-toast's runtime classes. `control-openhands browser eval "getComputedStyle(document.querySelector('[data-agent-server-ui]')).getPropertyValue('--oh-color-base').trim()"` is `#181818`. This proves the bundle is scoped, not that a host page is unaffected (`F26.lib-host-app`).-- **Host app (`F26.lib-host-app`).** After the library build above (keep `$L` until this bullet is done), copy the checked-in [library host fixture](../../scripts/fixtures/library-host/README.md) (`.agents/skills/verify-openhands/scripts/fixtures/library-host/`) to a scratch directory outside the checkout, point its `@openhands/agent-canvas` dependency at `file:$L`, `npm install --ignore-scripts` there and start `QA_CANVAS_PACKAGE_DIR=$L npx vite --host 127.0.0.1 --port <owned-port>` in the background; write `http://127.0.0.1:<owned-port>/` to `$OH_VERIFY_RUN/workspace/library-host-url`. Mount: run `control-openhands browser goto "$(cat "$OH_VERIFY_RUN/workspace/library-host-url")" --allow-external`, `control-openhands browser wait '#outside'`, `control-openhands browser wait '#inside, #module-error:not(:empty)' --timeout 30000` and `control-openhands browser screenshot --feature F26.lib-host-app --name bare-package-host`. `control-openhands assert --field value --equals true -- browser eval '!!document.querySelector("#inside") && document.querySelector("#module-error").textContent === ""'` holds: `#inside` mounts inside `AgentServerUIRoot` with no module error. Then `control-openhands browser click '#inside'`, and `control-openhands assert --field text --equals 'Inside Canvas 1' -- browser text '#inside'` holds. Styles: link `$L/dist/assets/agent-canvas-*.css` from the host page, add an `AgentServerUIProviders` with `analytics: false` and `styleOverrides: {"--oh-color-base": "#101820"}` around `EmptyBrowserMessage`, reload and read `getComputedStyle` of the component's outer `flex flex-col … p-10` div and of a host element (`control-openhands browser eval`, `browser screenshot --feature F26.lib-host-app --name host-component`). Expected: the component is laid out as a flex column with Canvas colours, the override changes its base colour, and host elements keep browser defaults. Today (Known failure above) the component computes `display: block`, `padding: 0px`; the scope root carries `--oh-color-base: #101820` but no loaded rule uses it. Stop the Vite process and remove the scratch host and `$L` afterwards.+- **Host app (`F26.lib-host-app`).** After the library build above (keep `$L` until this bullet is done), copy the checked-in [library host fixture](../../scripts/fixtures/library-host/README.md) (`.agents/skills/verify-openhands/scripts/fixtures/library-host/`) to a scratch directory outside the checkout, point its `@openhands/agent-canvas` dependency at `file:$L`, `npm install --ignore-scripts` there and start `QA_CANVAS_PACKAGE_DIR=$L npx vite --host 127.0.0.1 --port <owned-port>` in the background; write `http://127.0.0.1:<owned-port>/` to `$OH_VERIFY_RUN/workspace/library-host-url`. Mount: run `control-openhands browser goto "$(cat "$OH_VERIFY_RUN/workspace/library-host-url")" --allow-external`, `control-openhands browser wait '#outside'`, `control-openhands browser wait '#inside, #module-error:not(:empty)' --timeout 30000` and `control-openhands browser screenshot --feature F26.lib-host-app --name bare-package-host`. `control-openhands assert --field value --equals true -- browser eval '!!document.querySelector("#inside") && document.querySelector("#module-error").textContent === ""'` holds: `#inside` mounts inside `AgentServerUIRoot` with no module error. Then `control-openhands browser click '#inside'`, and `control-openhands assert --field text --equals 'Inside Canvas 1' -- browser text '#inside'` holds. Styles: in the scratch host add `host-styles.js` that does `import "@openhands/agent-canvas/styles.css";` and renders `AgentServerUIProviders` (`analytics: false`, `styleOverrides: {"--oh-color-base": "#101820"}`) around `div#probe.bg-canvas-base` holding `EmptyBrowserMessage` (both from `@openhands/agent-canvas`), and `styles.html`, a copy of `index.html` that loads it. `browser goto http://127.0.0.1:<owned-port>/styles.html --allow-external`, `browser wait '#probe > *'`, then read `getComputedStyle` with `control-openhands browser eval`: the component's outer `flex flex-col … p-10` div is `display: flex`, `flex-direction: column`, `padding: 40px`, colour `rgb(238, 242, 247)`; `#probe`'s background is `rgb(16, 24, 32)` (the override; removing `--oh-color-base` from the scope root's inline style turns it into the built-in `rgb(11, 14, 20)`); host `h1#outside` stays `"Times New Roman"`, `32px`, `rgb(0, 0, 0)` and `body` stays transparent; `#module-error` is empty (`browser screenshot --feature F26.lib-host-app --name host-component`, and the same at `browser viewport phone`). The component's text shows the raw i18n key (`BROWSER$NO_PAGE_LOADED`) because this host initializes no i18n. Stop the Vite process and remove the scratch host and `$L` afterwards.- **Docker conversation isolation (`F26.docker-conversation-runtime`).** Independently of the all-in-one Canvas deployment image, configure an owned source stack's Agent Server with `OH_CONVERSATION_RUNTIME=docker` and `OH_CONVERSATION_IMAGE` pointing to a compatible pinned Agent Server image. In SDK 1.55.1 the server creates a separate container with conversation, persistence and workspace mounts. A matching `-minimal` image can cover terminal/workspace isolation, but does not supply browser capabilities. Launch a confined conversation from Canvas, verify an actual model TerminalAction writes/reads a harmless marker under `/workspace`, and independently match the container hostname, mounted file and host-root absence. DELETE the conversation and verify its container and owned workspace disappear; restore original runtime settings. The 2026-10-17 test passed with the official 1.55.1 minimal image plus a small task-owned cloud-proxy/public-CA layer (no credentials). The unmodified image's model egress failed in this proxy-dependent environment; preserve that failure separately. This does not pass all-in-one packaging, browser or ACP recipes. Never print container/session keys or reuse a shared container as a fixture.-- **Docker and Helm prerequisites (`F26.docker-image`, `F26.docker-conversation-runtime`, `F26.helm-chart`).** If `docker info` cannot reach the daemon but `dockerd` is installed, start it (`setsid dockerd`); without one, record both Docker rows `blocked` naming the missing daemon. With a daemon, run the image with `-p 127.0.0.1:8000:8000`, open `http://localhost:8000/canvas` and expect the **Add a backend** step (`testid=onboarding-step-check-backend`, Next disabled until a key is typed); `onboarding-skip` leads to `testid=api-key-entry-screen`. With `-e AGENT_CANVAS_ALLOW_LAN_SESSION_KEY=true` the key is injected and first run opens on Choose your agent, under the telemetry consent dialog. `npm run build:docker` behind a TLS-intercepting proxy needs the proxy CA inside the build stages (without it `uv pip install` / `pip install` in the final stage fail with `CERTIFICATE_VERIFY_FAILED: self-signed certificate in certificate chain`), and Docker Hub may answer 429. Build a scratch copy of the Dockerfile that only adds the CA, never an edited `docker/Dockerfile`: after `FROM node:24-slim AS frontend-build` add `COPY --from=ccrca ca-bundle.crt /tmp/qa-ca.crt` and `ENV NODE_EXTRA_CA_CERTS=/tmp/qa-ca.crt npm_config_cafile=/tmp/qa-ca.crt`, after the first `USER root` add the same `COPY` and `ENV SSL_CERT_FILE=/tmp/qa-ca.crt PIP_CERT=/tmp/qa-ca.crt REQUESTS_CA_BUNDLE=/tmp/qa-ca.crt UV_NATIVE_TLS=1`, then `docker build -f <copy> --build-context ccrca=<dir holding ca-bundle.crt> --network host --build-arg AGENT_SERVER_IMAGE=<images.agentServer>:<versions.agentServer>-python --build-arg AGENT_SERVER_VERSION=<versions.agentServer> --build-arg AUTOMATION_VERSION=<versions.automation> --build-arg VITE_BASE_PATH=/canvas -t qa-agent-canvas:target .` (the pins `scripts/docker-build.mjs` reads from `config/defaults.json`; about 5 min on a warm cache). Publish it on a free port (`-p 127.0.0.1:<free>:8000`; 8000 and the 188xx blocks may belong to other runs). Docker Hub may answer `429 Too Many Requests` for `node:24-slim`; in the scratch copy only, replace both `FROM node:24-slim` lines with `FROM mirror.gcr.io/library/node:24-slim` (arrange; say so in the note). Run the unmodified image with `docker run -d --name <qa-name> -p 127.0.0.1:<free>:8000 qa-agent-canvas:target`. On a kernel without IPv6, `docker logs <qa-name>` shows `IPv6 is not available; binding the frontend to 0.0.0.0.` and `Static-server + proxy listening on http://0.0.0.0:8000/canvas/`, `/canvas/` answers `200` within about 25 s and `docker inspect -f '{{.State.Status}}' <qa-name>` is still `running` minutes later (#18178 fixed by #18180, verified 2026-10-13); `docker exec <qa-name> ls /opt/agent-canvas` lists the `mars-*.mjs` bridge modules (#18343). The container's generated key is `docker exec <qa-name> cat /home/openhands/.openhands/agent-canvas/api-key.txt` (write it to `$OH_VERIFY_RUN/private/`, fill with `--value-file`). Settings → Application (`/canvas/settings/app`) shows the image's `Version` (`agent-canvas-update-card`), which matches `jq -r .version package.json`. Remove the containers and the image afterwards (`docker rm -f`, `docker rmi`). For `F26.lib-host-app`, arrange a normal separate browser host with matching declared peers and the built package; the 2026-10-17 real host reproduced #126 above. A missing old example is not itself a runtime prerequisite. Install a compatible task-local Helm if the binary is missing, then run `helm lint helm/agent-canvas` and `helm template qa helm/agent-canvas`: the defaults render the StatefulSet, Services and ServiceAccount; Ingress is disabled by default and needs a separate `--set ingress.enabled=true` template check. The 2026-10-17 Helm 3.19.0 static checks passed in both variants. Live `helm install` follows `helm/agent-canvas/README.md` and requires an authorized disposable Kubernetes context; absent kubeconfig/in-cluster credentials are a concrete live prerequisite, not a missing-binary excuse.+- **Docker and Helm prerequisites (`F26.docker-image`, `F26.docker-conversation-runtime`, `F26.helm-chart`).** If `docker info` cannot reach the daemon but `dockerd` is installed, start it (`setsid dockerd`); without one, record both Docker rows `blocked` naming the missing daemon. With a daemon, run the image with `-p 127.0.0.1:8000:8000`, open `http://localhost:8000/canvas` and expect the **Add a backend** step (`testid=onboarding-step-check-backend`, Next disabled until a key is typed); `onboarding-skip` leads to `testid=api-key-entry-screen`. `curl -s http://127.0.0.1:8000/server_info | jq -r .version` equals `jq -r .versions.agentServer config/defaults.json` of the release the image was built from; `docker exec <container> sh -c 'tr "\0" "\n" < /proc/$(pgrep -f agent-server | head -1)/environ | grep EXTENSIONS_PATH'` prints `EXTENSIONS_PATH=/opt/agent-canvas/node_modules/@openhands/extensions`; `curl -s http://127.0.0.1:8000/canvas/mars/health | head -c 15` prints `<!DOCTYPE html>`. With `-e AGENT_CANVAS_ALLOW_LAN_SESSION_KEY=true` the key is injected and first run opens on Choose your agent, under the telemetry consent dialog. `npm run build:docker` behind a TLS-intercepting proxy needs the proxy CA inside the build stages (without it `uv pip install` / `pip install` in the final stage fail with `CERTIFICATE_VERIFY_FAILED: self-signed certificate in certificate chain`), and Docker Hub may answer 429. Build a scratch copy of the Dockerfile that only adds the CA, never an edited `docker/Dockerfile`: after `FROM node:24-slim AS frontend-build` add `COPY --from=ccrca ca-bundle.crt /tmp/qa-ca.crt` and `ENV NODE_EXTRA_CA_CERTS=/tmp/qa-ca.crt npm_config_cafile=/tmp/qa-ca.crt`, after the first `USER root` add the same `COPY` and `ENV SSL_CERT_FILE=/tmp/qa-ca.crt PIP_CERT=/tmp/qa-ca.crt REQUESTS_CA_BUNDLE=/tmp/qa-ca.crt UV_NATIVE_TLS=1`, then `docker build -f <copy> --build-context ccrca=<dir holding ca-bundle.crt> --network host --build-arg AGENT_SERVER_IMAGE=<images.agentServer>:<versions.agentServer>-python --build-arg AGENT_SERVER_VERSION=<versions.agentServer> --build-arg AUTOMATION_VERSION=<versions.automation> --build-arg VITE_BASE_PATH=/canvas -t qa-agent-canvas:target .` (the pins `scripts/docker-build.mjs` reads from `config/defaults.json`; about 5 min on a warm cache). Publish it on a free port (`-p 127.0.0.1:<free>:8000`; 8000 and the 188xx blocks may belong to other runs). Docker Hub may answer `429 Too Many Requests` for `node:24-slim`; in the scratch copy only, replace both `FROM node:24-slim` lines with `FROM mirror.gcr.io/library/node:24-slim` (arrange; say so in the note). Run the unmodified image with `docker run -d --name <qa-name> -p 127.0.0.1:<free>:8000 qa-agent-canvas:target`. On a kernel without IPv6, `docker logs <qa-name>` shows `IPv6 is not available; binding the frontend to 0.0.0.0.` and `Static-server + proxy listening on http://0.0.0.0:8000/canvas/`, `/canvas/` answers `200` within about 25 s and `docker inspect -f '{{.State.Status}}' <qa-name>` is still `running` minutes later (#18178 fixed by #18180, verified 2026-10-13); `docker exec <qa-name> ls /opt/agent-canvas` lists the `mars-*.mjs` bridge modules (#18343). The container's generated key is `docker exec <qa-name> cat /home/openhands/.openhands/agent-canvas/api-key.txt` (write it to `$OH_VERIFY_RUN/private/`, fill with `--value-file`). Settings → Application (`/canvas/settings/app`) shows the image's `Version` (`agent-canvas-update-card`), which matches `jq -r .version package.json`. Remove the containers and the image afterwards (`docker rm -f`, `docker rmi`). For `F26.lib-host-app`, arrange a normal separate browser host with matching declared peers and the built package; the 2026-10-17 real host reproduced #126 above. A missing old example is not itself a runtime prerequisite. Install a compatible task-local Helm if the binary is missing, then run `helm lint helm/agent-canvas` and `helm template qa helm/agent-canvas`: the defaults render the StatefulSet, Services and ServiceAccount; Ingress is disabled by default and needs a separate `--set ingress.enabled=true` template check. The 2026-10-17 Helm 3.19.0 static checks passed in both variants. Live `helm install` follows `helm/agent-canvas/README.md` and requires an authorized disposable Kubernetes context; absent kubeconfig/in-cluster credentials are a concrete live prerequisite, not a missing-binary excuse.- **Desktop splash and window (`F26.desktop-boot-splash`, `F26.desktop-main-window`).** Start a private display and the app in one process group: `Q=$OH_VERIFY_RUN/private/f26-desktop; mkdir -p $Q/shots; setsid nohup sh -c "echo \$\$ > $Q/pgid; Xvfb :78 -screen 0 1440x1000x24 -nolisten tcp & sleep 1; DISPLAY=:78 HOME=$Q/home OH_CANVAS_SAFE_STATE_DIR=$Q/state LOCAL_BACKEND_API_KEY=qa-f26-desktop-key OH_SECRET_KEY=qa-f26-secret DO_NOT_TRACK=1 exec node_modules/.bin/electron --no-sandbox electron" > $Q/desktop.log 2>&1 &`, then capture frames: `for i in $(seq 1 40); do sleep 1; DISPLAY=:78 import -window root $Q/shots/s$(printf %02d $i).png; done`. Read the frames: the first seconds show the splash (logo, `OpenHands Agent Canvas`, `AI coding agent interface`, spinner, `Starting backend services…`, then lines such as `agent-server: {"asctime": ...`, the note `First launch downloads Python + the OpenHands agent server.` and a **Show details** button); a later frame shows the main window with a `File Edit View Window` menu and the first-run **Choose your agent** step under the consent dialog naming `the local backend "Local" at http://localhost:8000` (about 30 s on a warm cache, minutes on the first download). While it runs, `curl -s http://127.0.0.1:8000/mars/health | head -c 15` prints `<!DOCTYPE html>`: the desktop stack's ingress hosts no bridge (`marsWebBridge: false`, `F26.mars-web-bridge`). Copy the frames you cite into `$OH_VERIFY_RUN/evidence/F26.desktop-boot-splash/` and `.../F26.desktop-main-window/`. Stop in two steps: `kill -TERM $(cat $Q/pgid)` signals only the Electron launcher (it quits and stops its backend services); after about 10 s ports 8000, 18000, 18001 and 3001 are free again. Then `kill -TERM -- -$(cat $Q/pgid)` ends Xvfb; `ps -o pid,cmd -g $(cat $Q/pgid)` lists nothing.- **Electron failure window and controls (`F26.desktop-boot-splash`, `F26.automation-startup-failure`).** Real Electron/Xvfb and the harness’s installed Playwright are available for this source-built Linux path; native computer-use tools are not required. On an owned display, launch the existing Electron binary with `--no-sandbox --disable-gpu --remote-debugging-address=127.0.0.1 --remote-debugging-port=<free-owned-port> <frozen-checkout>/electron` (directory entry), private `HOME`/`OH_CANVAS_SAFE_STATE_DIR`, `DO_NOT_TRACK=1` and `VITE_DO_NOT_TRACK=1`. Select the actual dependency-resolution failure with `OH_AUTOMATION_VERSION=0.0.1`. The launcher honors `PORT` plus `OH_CANVAS_SAFE_BACKEND_PORT`, `OH_CANVAS_SAFE_AUTOMATION_PORT`, and `OH_CANVAS_SAFE_VITE_PORT`; reserve backend port +1000 for VSCode. However, Electron’s later readiness/main-window URLs hardcode `localhost:8000`: an isolated failure callback before that stage does not prove healthy boot at a nondefault ingress. Use free default ports 8000/18000/18001/3001/19000 for healthy boot. Bound the owned probe, monitor resources, and never stop another owner’s service. Attach real `chromium.connectOverCDP` to the loopback debugging endpoint, select actual `loading.html`, wait for `body.failed`, click `#copy-logs`, observe **Copied** after its real IPC completion, then click `#quit-button` and wait for the parent exit. On 2026-10-17, frozen `61a3bd93e642`, SDK 1.55.1: the automatic failure splash showed a readable automation-exited(code 1) summary and expanded logs; Copy confirmed and Quit returned 0. Reviewed screenshots/receipts were retained and owned ports/display/private state cleaned. A later genuine startup-interruption check paused only the owned real automation group before health, let the main window open, then resumed/terminated that group: the original main closed, a new failure splash reopened with the real SIGTERM reason, and Quit returned 0. This target-only observation does not retire #18362 or prove packaged, Windows/macOS, menu, or hardware branches; timeout-only behavior is explicitly unchanged by PR #18311.- **External links (`F26.desktop-external-links`).** Use the healthy owned Electron shell and actual rendered Markdown links; CDP user clicks are available. For a harmless destination, an owned OS-browser wrapper must execute real Chromium with a disposable profile and an exact test-host-to-loopback resolver rule; log only invocation booleans and the owned HTTP destination’s GET count. Do not replace the OS browser with an echo-only handler or inject DOM/events. Clear consent/onboarding through ordinary UI first; a public user-message API with `run:false` can supply the Markdown without model calls. Expected behavior remains an allowed link reaching the real system browser. Observed 2026-10-17 on frozen `61a3bd93e642`, source-built Linux: two actual `_blank` Markdown link clicks kept one Electron page but produced no owned browser-wrapper invocation or destination GET, including with an owned XDG browser association. A separate real `xdg-open` control with that same association launched Chromium and reached the destination. Record this as an observed failed handoff/intake candidate with origin unconfirmed, not a passing prerequisite check or a proven new product root cause. An inert custom-scheme Markdown link had its href removed by the sanitizer and invoked no handler; that proves this renderer refusal only, not the direct Electron popup-policy branches. Main-window navigation, OAuth redirect, nested-popup and actual provider authentication remain separate unproved paths. Preserve safe screenshots/receipts, stop only owned Electron/Chromium/Xvfb groups, verify all owned ports closed, and purge private state. Manual receipt provenance must name its actual URL/ports/viewport rather than reuse an old run URL.@@ -141,7 +141,7 @@ Preconditions:- Known issue [#18547](https://github.com/OpenHands/OpenHands/issues/18547) (`F26.desktop-main-window`): on Windows the desktop title bar and menu bar stay dark when the Light+ theme is selected (`electron/main.mjs` keeps `nativeTheme.themeSource` on `dark`). Live prerequisite: the desktop app on Windows.- Known issue [#16872](https://github.com/OpenHands/OpenHands/issues/16872) (`F26.desktop-external-links`): Non-browser URI schemes were reported to reach the OS handler from the desktop popup flow. Current source now parses hosts and restricts external protocols; recheck refusal in a running owned Electron shell before concluding resolution. Observe an inert denied test scheme without invoking any real OS handler. Real Playwright CDP can drive the owned Electron window. The initial inert Markdown test was sanitized before Electron saw its scheme. A later real fixture button called `window.open` with an unregistered inert custom scheme (native page count stayed 1), and an actual loopback-popup anchor navigated to that scheme (the popup closed, page count 2→1). No real OS protocol handler was registered or invoked. This proves those observable refusal paths only; it does not trace an unlogged OS-shell rejection, cover every scheme/platform, or provide paired baseline/fix proof, so this note is not retired.- Known failure [#16257](https://github.com/OpenHands/OpenHands/issues/16257) (`F26.runtime-services`), reproduced 2026-10-17: interrupt standalone `node scripts/dev-static.mjs --skip-build --port <owned-port>` after its real static-service start but before its awaited ingress start. Expected: parent termination leaves no owned service. Observed at frozen `61a3bd93e642`, SDK 1.55.1: SIGTERM at 8.758 s with no ingress process/listener, parent exit 0 at 11.828 s, then an owned ingress remained listening. Two shutdown handlers run because `dev-static` imports `dev-with-automation`; the imported empty-registry handler exits after the same 3 s delay and can preempt the standalone force-kill while startup continues. Stop the exact owned late ingress group after recording PID/port/timing. An earlier timing attempt before the static-start phase did not reproduce; a normal fully ready stop closed all owned ports. This is new live evidence for the existing issue, not a newly attributed regression or a retired note.-- Known issue [#16774](https://github.com/OpenHands/OpenHands/issues/16774) (`F26.desktop-boot-splash` `F26.desktop-main-window`): The macOS 1.14.0 DMG installs an app that Gatekeeper reports as damaged instead of booting. Recheck the actual downloaded signed/notarized release artifact through install and first launch on macOS; starting source Electron or removing quarantine would bypass the reported failure. That release DMG/signature fixture is unavailable in this run.+- macOS release builds (`F26.desktop-boot-splash`, `F26.desktop-main-window`) are ad-hoc signed, without an Apple Developer ID or notarization, so Gatekeeper reports a downloaded, quarantined `OpenHands Agent Canvas.app` as damaged instead of booting it. The GitHub release notes carry a macOS install note with the workaround, `xattr -dr com.apple.quarantine "/Applications/OpenHands Agent Canvas.app"`, after which the app opens normally. Checking it needs macOS and the downloaded release DMG; starting source Electron does not exercise Gatekeeper.- Known issue [#16720](https://github.com/OpenHands/OpenHands/issues/16720) (`F26.runtime-services`): CPU-heavy commands in several conversations can starve Agent Server and stop the UI loading; the report requests lower priority for agent commands than the service. Recheck with a disposable bounded workload, recorded CPU/resource limits and real command observations, never arbitrary stress on the shared machine. The exact Rust workload and affected resource envelope are missing. A real model key was available and used for other confined tests; that does not supply this workload. The backend counterpart is [OpenHands/software-agent-sdk#4575](https://github.com/OpenHands/software-agent-sdk/issues/4575).- Known issue [#16690](https://github.com/OpenHands/OpenHands/issues/16690) (`F26.docker-image` `F26.runtime-services`): A Docker deployment with nine chats becomes slow/disconnected, then reloads lose the list and startup proxies report ECONNREFUSED. Recheck on the affected Windows-host Docker configuration with retained conversation state and a controlled workload; creating a few empty local chats cannot prove the reported resource/runtime failure. This scenario needs the affected Windows Docker host and original conversation/workload fixture; daemon availability alone does not supply that prerequisite.- Known issue [#16511](https://github.com/OpenHands/OpenHands/issues/16511) (`F26.docker-image`): The documented Docker bind mount can be unwritable to container uid 10001, killing Agent Server/automation while the frontend still serves Bad Gateway. Recheck a disposable host-owned mode-700 mount and a writable control against an owned image; never chmod the operator's real state directory. The daemon and a bounded real container smoke test work here; this scenario still needs a completed owned Canvas image.diff --git a/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.mdindex 1092d722df..3d2562954a 100644--- a/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md+++ b/.agents/skills/verify-openhands/references/feature-map/F28-automation-setup-page.md@@ -5,7 +5,7 @@ template dialog. **Add Automation → Create Automation** opens it blank at`/automations/setup`; a saved draft reopens at `/automations/setup?draftId=<id>`;kebab **Edit** opens an existing automation at`/automations/setup?automationId=<id>`. The form collects Name, a Prompt (or-Custom Python with entrypoint and setup script), agent profile, model and+Custom Python with entrypoint and setup script), model (LLM profile) andrepositories, a Schedule or Event trigger, and optional plugins and timeout.A docked composer under the form can ask the agent to fill it. Create mode'stoolbar has **Save draft**, **Test** and **Create Automation**; edit mode's has@@ -22,7 +22,7 @@ Source: `src/routes/automation-setup-new.tsx`, `src/components/features/automati## Sub-features-- `F28.open-blank`: Create Automation opens `/automations/setup` with Name pre-filled **New Automation** (placeholder `Standup digest`), an empty Prompt, Schedule selected with Daily at `09:00` in `America/New_York` (the default, whatever the browser's time zone), the docked composer (`Need help creating an automation?`) and no new conversation. The save state already reads **Unsaved changes** and the toolbar **Test** is disabled.+- `F28.open-blank`: Create Automation opens `/automations/setup` with Name pre-filled **New Automation** (placeholder `Standup digest`), an empty Prompt, Schedule selected with Daily at `09:00` in `America/New_York` (the default, whatever the browser's time zone), the docked composer (`Need help creating an automation?`) and no new conversation. The save state already reads **Unsaved changes** and the toolbar **Test** is disabled. Neither the Prompt nor the Custom kind offers an agent profile (`automation-agent-profile` count `0`); the Prompt kind shows only the LLM profile pill `automation-setup-model`.- `F28.kind-toggle`: the button beside the Prompt label switches between Prompt and **Custom** (Custom Python). Custom shows a generated Python script that calls `AUTOMATION_CALLBACK_URL`, and **Entrypoint and setup** reveals Entrypoint `python3 main.py`, Setup script path `setup.sh` and a no-op setup script; switching back shows the Prompt again.- `F28.schedule-presets`: in create mode the Frequency control (the same control as on the edit page, whose presets and saving F23 owns as edit-schedule and edit-cron) starts on Daily at `09:00`, and switching presets fills create-mode defaults: Once a date-time at the next whole hour today, Weekly Monday, and Custom the cron of the preset it replaces (from Weekly, Monday at `10:30` it reads `30 10 * * 1`; Weekdays `30 10 * * 1-5`, Hourly `0 * * * *`, Once on 17 October at `09:00` `0 9 17 10 *`). Clicking Custom again keeps a cron already typed there; leaving Custom for a preset and coming back starts from that preset again. Every preset keeps the 13-entry time zone select.- `F28.event-trigger`: choosing **Event** replaces the schedule with Event source (default **GitHub**, sent as `github`; its menu lists the webhook sources registered on this backend plus **Custom**), Event key (default `issue_comment.created`), **Add Filter** (one free-text filter input), **Ask Agent**, the notice `Event sources need a public URL to deliver webhooks to this instance.`, and a **Test event payload** pre-filled with a JSON object for that source and key. A source the backend has not registered (menu **Custom**, then a typed name) adds **Register this as a custom webhook source**, checked by default.@@ -52,7 +52,7 @@ Preconditions:- Desktop viewport; the `F28.error-contrast` bullet switches color themes (Settings → Application, `F16`) and restores the default.- The original recipes were run on 2026-10-16 on a fresh local `control-openhands launch` of main with Agent Server 1.55.1 and no LLM; the blank-form, presets and error-contrast recipes were run again on 2026-10-17. The dated 2026-10-17 maintenance evidence separately proves the added regression checks and known-bug block. Documented known failures remain failures.-- **Blank form (`F28.open-blank`).** Run `control-openhands conversation list`, `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup$'`. Then `browser value 'testid=automation-setup-name'` is `New Automation`, `browser attr 'testid=automation-setup-name' placeholder` is `Standup digest`, `browser value 'testid=automation-setup-prompt'` is empty, `browser attr 'testid=automation-setup-frequency-daily' aria-checked` is `true`, `browser value 'testid=automation-setup-time'` is `09:00`, `browser text 'testid=automation-setup-save-state'` is `Unsaved changes` and `browser enabled 'testid=automation-setup-test'` is `false`. `conversation list` is unchanged.+- **Blank form (`F28.open-blank`).** Run `control-openhands conversation list`, `control-openhands browser goto /automations`, `control-openhands browser click 'testid=automations-add-automation'` and `control-openhands browser click 'testid=automations-add-automation-create' --expect-url '/automations/setup$'`. Then `browser value 'testid=automation-setup-name'` is `New Automation`, `browser attr 'testid=automation-setup-name' placeholder` is `Standup digest`, `browser value 'testid=automation-setup-prompt'` is empty, `browser attr 'testid=automation-setup-frequency-daily' aria-checked` is `true`, `browser value 'testid=automation-setup-time'` is `09:00`, `browser text 'testid=automation-setup-save-state'` is `Unsaved changes`, `browser enabled 'testid=automation-setup-test'` is `false`, `browser count 'testid=automation-setup-model'` is `1` and `browser count 'testid=automation-agent-profile'` is `0`; after `control-openhands browser click 'testid=automation-setup-kind-custom'` the agent profile count is still `0`. `conversation list` is unchanged.- **Presets (`F28.schedule-presets`).** For each of `once`, `hourly`, `daily`, `weekdays`, `weekly`, `custom` run `control-openhands browser click 'testid=automation-setup-frequency-<preset>'` and `control-openhands browser testids 'testid=automation-setup-at-row'` (`browser snapshot` is an ARIA tree and does not list test ids): Once has `automation-setup-datetime`, Hourly only `automation-setup-timezone`, Daily and Weekdays `automation-setup-time`, Weekly also `automation-setup-weekday` (value `1`), Custom `automation-setup-custom-schedule`. Click Weekly, then Custom: `browser value 'testid=automation-setup-custom-schedule'` is Weekly's `0 9 * * 1`. Click Weekly, `browser fill 'testid=automation-setup-time' 10:30`, click Custom: `30 10 * * 1`. Fill the Custom field with `15 8 * * 2` and click Custom again: it stays `15 8 * * 2`. Click Daily, then Custom: `30 10 * * *` (the preset's, not the typed cron). `browser count 'testid=automation-setup-timezone >> option'` is `13`. Click Daily again.- **Event fields (`F28.event-trigger`).** Run `control-openhands browser click 'role=radio[name=/^Event/]'`. `browser value 'testid=automation-setup-event-key'` is `issue_comment.created`, `browser visible 'testid=automation-setup-event-public-url-notice'` is `true`, and `browser count 'testid=automation-setup-custom-webhook-enabled'` is `0` for `github`. Click `testid=automation-setup-event-source-toggle`, `browser snapshot 'testid=automation-setup-event-source-menu'` lists the registered sources and `Custom` (GitHub, GitLab, Slack, Linear, Jira on a local stack). Click `testid=automation-setup-event-source-menu >> role=button[name="Custom"]`, `browser fill 'testid=automation-setup-event-source' qa-hooks`: `browser visible 'testid=automation-setup-custom-webhook-enabled'` is `true` and `browser eval "document.querySelector('[data-testid=automation-setup-custom-webhook-enabled]').checked"` is `true`. Pick GitHub again from the same menu. Click `testid=automation-setup-add-event-filter` (`automation-setup-event-filter` appears) and `testid=automation-setup-event-test-payload-toggle`; `browser value 'testid=automation-setup-event-test-payload'` is a JSON object with `"source": "github"` and `"event": "issue_comment.created"`. Click `role=radio[name=/^Schedule/]` to return.- **Invalid drafts save (`F28.draft-save`, `F28.draft-validation`).** Fill `testid=automation-setup-name` with `QA_F28 Probe`, leave the prompt empty, `browser network --clear`, click `testid=automation-setup-save-draft`. `browser wait-url 'draftId='`; `browser toasts --history` includes `Draft saved in this conversation.`; `browser text 'testid=automation-setup-draft-validity'` is `Field required`; `control-openhands api GET /api/automation/v1/drafts/<draft-id>` (the id in the URL) has `validation_errors` `[{"field":"prompt","code":"missing",…}]` and `dispatchable: false` (`browser network --bodies` shows request bodies, not the response); `browser text 'testid=automation-setup-save-state'` is `Saved just now`. Now fill the prompt with `Reply pong. Do not run any tools.`, click `testid=automation-setup-frequency-custom`, fill `testid=automation-setup-custom-schedule` with `every day`, `browser network --clear`, press Tab, and `browser network --filter 'automation/v1'` lists no request (no blur validation). Click Save draft: the validity text is `Value error, Invalid cron expression: every day`. `browser network --clear`, click `testid=automation-setup-test`: `browser text 'testid=automation-setup-draft-runs-page'` contains the same cron error, `browser network` lists only the draft `PATCH` (200) and nothing under `dispatch`; click `testid=automation-setup-draft-runs-back`. Fill the cron with `0 0 1 1 *` and Save draft: it reads `Ready to test`.diff --git a/.agents/skills/verify-openhands/references/feature-map/README.md b/.agents/skills/verify-openhands/references/feature-map/README.mdindex 2b45ddeac9..c92ad11a52 100644--- a/.agents/skills/verify-openhands/references/feature-map/README.md+++ b/.agents/skills/verify-openhands/references/feature-map/README.md@@ -5,7 +5,7 @@ of Agent Canvas. Read this index before driving the app, then open the matchingfeature file and run its recipe with `control-openhands`. The map is not a claimthat every row passes today; the run's evidence ledger is.-Maintenance baseline: main@1f7b1e69a19c1ce6162463427fb59fb0106ea780 (2026-10-20). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.+Maintenance baseline: main@df87d66078f8e2a3bc95e2118157311896c8b79d (2026-10-22). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.## Baseline preconditions@@ -232,26 +232,26 @@ How to write and prove new entries: [../mapping.md](../mapping.md).|---|---|---|---|---|---|| F01 | [First run, onboarding and sign-in](F01-first-run-and-sign-in.md) | telemetry consent, onboarding modal, public-mode backend step and API-key screen, route error page | any URL on a fresh browser profile; `launch --public` | fresh run or `browser reset`; LLM for say-hello | 22 || F02 | [App shell, sidebar and command menu](F02-app-shell.md) | rail and phone drawer, collapse, home pinning, getting-started checklist, command menu, toasts | every page; `Control+k` / `Meta+k` | baseline; LLM for checklist progress | 28 |-| F03 | [Home and starting work](F03-home.md) | home composer, workspace picker, plugin picker, recommended automations rail, no-LLM banner | `/`, **New Chat**, command menu | runs before and after `llm preset`; fixture repo | 42 |+| F03 | [Home and starting work](F03-home.md) | home composer, workspace picker, plugin picker, recommended automations rail, no-LLM banner | `/`, **New Chat**, command menu | runs before and after `llm preset`; fixture repo | 44 || F04 | [Conversation list and folders](F04-conversation-list.md) | sidebar list, View presets, workspace folders, tags, rename/pin/archive/delete, load more | sidebar on every page | several conversations; fixture repos | 41 |-| F05 | [Composer, slash commands and plan mode](F05-composer.md) | send, drafts, attachments, model pill, slash commands, `/goal`, plan mode, dictation | home and conversation composer | LLM | 25 |-| F06 | [Agent activity](F06-agent-activity.md) | messages, tool events, thinking, empty state, confirmation mode, stop/resume, errors, branching | `/conversations/<id>` | LLM | 36 |+| F05 | [Composer, slash commands and plan mode](F05-composer.md) | send, drafts, attachments, model pill, slash commands, `/goal`, plan mode, dictation | home and conversation composer | LLM | 26 |+| F06 | [Agent activity](F06-agent-activity.md) | messages, tool events, thinking, empty state, confirmation mode, stop/resume, errors, branching | `/conversations/<id>` | LLM | 37 || F07 | [Conversation page, header and menu](F07-conversation-page.md) | title rename, status menu, ⋯ menu (skills, hooks, tools, export, download, cost, stop, delete), git bar, overview | sidebar row, URL, `/panel` | LLM; fixture repo; Cloud items blocked | 25 || F08 | [Workspace drawer: files and changes](F08-workspace-files-and-changes.md) | Files tab, Commits and uncommitted changes, diff viewer, file links in chat | panel toggle, file links | LLM; fixture repo | 34 || F09 | [Settings shell and navigation](F09-settings-shell.md) | settings navigation, phone hub, deep links, update card, backend note | gear, command menu, URLs | baseline; deep links before `llm preset` | 26 || F10 | [LLM profiles](F10-llm-profiles.md) | list, add, edit, rename, duplicate, delete, default, advanced fields, validation | `/settings/llm` | fresh run; DeepSeek key | 34 || F11 | [Provider connections](F11-provider-connections.md) | add, edit, rotate key, delete connections; profiles from a connection | `/settings/llm` | fresh run; DeepSeek key | 21 || F12 | [Model router](F12-model-router.md) | meta-profiles, templates, run on first message, routing effect | `/settings/meta-llm` | LLM profiles | 25 |-| F13 | [Agent profiles](F13-agent-profiles.md) | OpenHands and ACP profiles, editor, name rules, active profile, MCP and secret scoping | `/settings/agents` | fresh run; LLM | 22 |+| F13 | [Agent profiles](F13-agent-profiles.md) | OpenHands and ACP profiles, editor, name rules, active profile, MCP and secret scoping | `/settings/agents` | fresh run; LLM | 25 || F14 | [Secrets](F14-secrets.md) | list, add, edit, rename, delete, agent access | `/settings/secrets` | LLM for agent access | 12 |-| F15 | [Condenser, agent context and verification](F15-agent-behavior-settings.md) | condenser fields, agent context, confirmation mode and critic, their effect on runs | `/settings/condenser`, `/agent-context`, `/verification` | LLM for the effects | 23 |-| F16 | [Application settings](F16-application-settings.md) | language, theme, analytics, sound, checklist, title model, voice input, git identity | `/settings/app` | LLM for title and git checks | 20 |+| F15 | [Condenser, agent context and verification](F15-agent-behavior-settings.md) | condenser fields, agent context, confirmation mode and critic, their effect on runs | `/settings/condenser`, `/agent-context`, `/verification` | LLM for the effects | 24 |+| F16 | [Application settings](F16-application-settings.md) | language, theme, analytics, sound, checklist, title model, voice input, git identity | `/settings/app` | LLM for title and git checks | 21 || F17 | [Customize hub and MCP servers](F17-mcp-servers.md) | catalog, install, test, edit, delete, custom servers, agent use | **Customize**, `/mcp` | `uvx`/`npx`; LLM | 31 || F18 | [Skills catalog](F18-skills.md) | facets, search, enable/disable, add skill, Use skill, personal and project skills | `/skills` | `fixture skill`; LLM | 22 || F19 | [Plugins and plugin launch](F19-plugins.md) | catalog, install, update, uninstall, enable, launch deep links | `/plugins`, `/launch` | LLM | 25 || F20 | [Canvas apps](F20-canvas-apps.md) | install from path or git, enable, update, uninstall, extension pages | `/apps` | none | 24 || F21 | [Automations dashboard and actions](F21-automations-dashboard.md) | cards and list, filters, sort, Run now, enable, export, import, delete, pin | `/automations` | LLM; automation service | 35 |-| F22 | [Creating automations](F22-automation-creation.md) | templates, setup dialog, custom automations, import | `/automations/templates`, `/automations/new/<id>` | LLM | 29 |+| F22 | [Creating automations](F22-automation-creation.md) | templates, setup dialog, custom automations, import | `/automations/templates`, `/automations/new/<id>` | LLM | 31 || F23 | [Automation detail, runs and editing](F23-automation-detail.md) | detail page, runs, logs, edit dialog, triggers, debug | `/automations/<id>` | LLM; runs | 41 || F24 | [Automation Git Sync](F24-git-sync.md) | configure, sync cycles, encryption, status | `/automations/git-sync` | `fixture git-repo` | 26 || F25 | [Backends, Cloud and sharing](F25-backends-and-cloud.md) | add, edit, remove and switch backends, per-backend consent, Cloud login, shared pages | backend selector | a second stack; Cloud account (blocked) | 33 |@@ -259,7 +259,7 @@ How to write and prove new entries: [../mapping.md](../mapping.md).| F27 | [Workspace tools](F27-workspace-tools.md) | terminal, browser, planner, task list, usage, `canvas_ui_control` | drawer tabs | LLM | 30 || F28 | [Automation setup page](F28-automation-setup-page.md) | blank form, kind toggle, schedule presets, event fields, timeout, draft save and validation, draft Test, Create | **Add Automation → Create Automation**, `/automations/setup`, draft Resume, kebab Edit | automation service; no LLM | 13 |-28 families, 779 sub-features. `control-openhands map ids` lists every ID with its file.+28 families, 790 sub-features. `control-openhands map ids` lists every ID with its file.### Neighbouring familiesdiff --git a/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs b/.agents/skills/verify-openhands/scripts/control-openhands.test.mjsindex a5a00de488..d10088bf5c 100755--- a/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs+++ b/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs@@ -974,6 +974,23 @@ test("cited test ids resolve through literals and dynamic prefixes", () => {assert.deepEqual(result.unresolved, [{ id: "renamed-away-button", files: ["F14-secrets.md"] },]);+ // A one-word prefix accounts for an id only when the source builds ids+ // from it; otherwise a removed id such as automation-setup-agent-profile+ // would resolve through the plain word "automation".+ const words = new Set(["automation", "edit"]);+ words.heads = new Set(["edit"]);+ words.tails = new Set(["message"]);+ const strict = resolveTestids(+ new Map([+ ["automation-setup-agent-profile", new Set(["F28.md"])],+ ["edit-automation-save", new Set(["F23.md"])],+ ["agent-message", new Set(["F06.md"])],+ ]),+ words,+ );+ assert.deepEqual(strict.unresolved, [+ { id: "automation-setup-agent-profile", files: ["F28.md"] },+ ]);// The live map against the live source tree: zero unresolved today, and// --strict makes any drift a failing exit.const res = spawnSync(process.execPath, [cli, "map", "testids", "--strict"], {diff --git a/.agents/skills/verify-openhands/scripts/lib/testids.mjs b/.agents/skills/verify-openhands/scripts/lib/testids.mjsindex 367bbb17ac..10ede6d108 100644--- a/.agents/skills/verify-openhands/scripts/lib/testids.mjs+++ b/.agents/skills/verify-openhands/scripts/lib/testids.mjs@@ -4,7 +4,11 @@// A cited id resolves when it appears as a string literal in src/, or when a// prefix of it (cut at a hyphen) does: Canvas builds many ids from a prefix// prop or a template (`${testIdPrefix}-name`, `plugin-card-${id}`), so the-// literal before the dynamic part is what the source holds. Anything left is+// literal before the dynamic part is what the source holds. A one-word prefix+// counts only when the source really builds ids from it (`automation-${x}`):+// any id starting with a common word such as `automation` or `edit` would+// otherwise resolve, removed ids included. An id may also end in the literal+// tail of a template (`${type}-message`). Anything left is// reported for a human or the live pass to judge; it is not proof of drift// on its own (the id may be assembled in a way this heuristic misses).import { readFileSync, readdirSync } from "node:fs";@@ -40,16 +44,27 @@ function sourceFiles(dir, out = []) {return out;}-/** String literals and template heads in the source tree that can be test ids. */+/**+ * String literals and template heads in the source tree that can be test ids.+ * The set also carries `heads` (template heads, `prefix-${...}`) and `tails`+ * (template tails, `${...}-suffix`).+ */export function sourceLiterals(srcDir) {const literals = new Set();+ literals.heads = new Set();+ literals.tails = new Set();for (const file of sourceFiles(srcDir)) {const text = readFileSync(file, "utf8");for (const m of text.matchAll(/["'`]([a-z][a-z0-9_.:/-]*[a-z0-9])["'`$]/g))literals.add(m[1]);// `prefix-${dynamic}`: the head before the first placeholder.- for (const m of text.matchAll(/`([a-z][a-z0-9_.:/-]*)-\$\{/g))+ for (const m of text.matchAll(/`([a-z][a-z0-9_.:/-]*)-\$\{/g)) {literals.add(m[1]);+ literals.heads.add(m[1]);+ }+ // `${dynamic}-suffix`: the literal tail after the last placeholder.+ for (const m of text.matchAll(/\}-([a-z][a-z0-9_.:/-]*[a-z0-9])`/g))+ literals.tails.add(m[1]);}return literals;}@@ -59,6 +74,8 @@ export function sourceLiterals(srcDir) {* @returns {{ cited: number, unresolved: Array<{ id: string, files: string[] }> }}*/export function resolveTestids(cited, literals) {+ const heads = literals.heads ?? new Set();+ const tails = literals.tails ?? new Set();const unresolved = [];for (const [id, files] of cited) {if (literals.has(id)) continue;@@ -66,7 +83,16 @@ export function resolveTestids(cited, literals) {let found = false;for (let i = parts.length - 1; i >= 1 && !found; i -= 1) {const prefix = parts.slice(0, i).join("-");- if (prefix.length >= 4 && literals.has(prefix)) found = true;+ if (+ prefix.length >= 4 &&+ literals.has(prefix) &&+ (prefix.includes("-") || heads.has(prefix))+ )+ found = true;+ }+ for (let i = 1; i < parts.length && !found; i += 1) {+ const tail = parts.slice(i).join("-");+ if (tail.length >= 4 && tails.has(tail)) found = true;}if (!found) unresolved.push({ id, files: [...files].sort() });}diff --git a/.pr/bugs/2026-10-22/azure-encoded-label/azure-desktop-first-run.png b/.pr/bugs/2026-10-22/azure-encoded-label/azure-desktop-first-run.pngnew file mode 100644index 0000000000..45bd2af394Binary files /dev/null and b/.pr/bugs/2026-10-22/azure-encoded-label/azure-desktop-first-run.png differdiff --git a/.pr/bugs/2026-10-22/azure-encoded-label/azure-label.png b/.pr/bugs/2026-10-22/azure-encoded-label/azure-label.pngnew file mode 100644index 0000000000..88860bb97fBinary files /dev/null and b/.pr/bugs/2026-10-22/azure-encoded-label/azure-label.png differdiff --git a/.pr/bugs/2026-10-22/azure-encoded-label/azure-overview.png b/.pr/bugs/2026-10-22/azure-encoded-label/azure-overview.pngnew file mode 100644index 0000000000..4e5c2b40f0Binary files /dev/null and b/.pr/bugs/2026-10-22/azure-encoded-label/azure-overview.png differdiff --git a/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-first-run.png b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-first-run.pngnew file mode 100644index 0000000000..73cb7867efBinary files /dev/null and b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-first-run.png differdiff --git a/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-still-open.png b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-still-open.pngnew file mode 100644index 0000000000..25447b7f78Binary files /dev/null and b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-agent-pill-still-open.png differdiff --git a/.pr/bugs/2026-10-22/escape-pill-popovers/escape-llm-pill-still-open.png b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-llm-pill-still-open.pngnew file mode 100644index 0000000000..b0f72edc6fBinary files /dev/null and b/.pr/bugs/2026-10-22/escape-pill-popovers/escape-llm-pill-still-open.png differdiff --git a/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-correct-120.png b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-correct-120.pngnew file mode 100644index 0000000000..082cc5f9f9Binary files /dev/null and b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-correct-120.png differdiff --git a/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-stale-240.png b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-stale-240.pngnew file mode 100644index 0000000000..cd81e00d50Binary files /dev/null and b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/reopen-stale-240.png differdiff --git a/.pr/bugs/2026-10-22/profile-editor-stale-reopen/stale-reopen-first-run.png b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/stale-reopen-first-run.pngnew file mode 100644index 0000000000..0d30afb80eBinary files /dev/null and b/.pr/bugs/2026-10-22/profile-editor-stale-reopen/stale-reopen-first-run.png differdiff --git a/.pr/map-maintenance/2026-10-22/F01-locked-cloud-first-run.png b/.pr/map-maintenance/2026-10-22/F01-locked-cloud-first-run.pngnew file mode 100644index 0000000000..0d1a6140f0Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F01-locked-cloud-first-run.png differdiff --git a/.pr/map-maintenance/2026-10-22/F03-rail-dismiss-x.png b/.pr/map-maintenance/2026-10-22/F03-rail-dismiss-x.pngnew file mode 100644index 0000000000..0cb8fb4e4dBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F03-rail-dismiss-x.png differdiff --git a/.pr/map-maintenance/2026-10-22/F04-grouped-folders-fresh-browser.png b/.pr/map-maintenance/2026-10-22/F04-grouped-folders-fresh-browser.pngnew file mode 100644index 0000000000..d9f63c2bdcBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F04-grouped-folders-fresh-browser.png differdiff --git a/.pr/map-maintenance/2026-10-22/F05-agent-profile-pill.png b/.pr/map-maintenance/2026-10-22/F05-agent-profile-pill.pngnew file mode 100644index 0000000000..5d10b3a428Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F05-agent-profile-pill.png differdiff --git a/.pr/map-maintenance/2026-10-22/F06-markdown-alerts-light-plus.png b/.pr/map-maintenance/2026-10-22/F06-markdown-alerts-light-plus.pngnew file mode 100644index 0000000000..39e934c85dBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F06-markdown-alerts-light-plus.png differdiff --git a/.pr/map-maintenance/2026-10-22/F08-diff-markdown-preview-capped.png b/.pr/map-maintenance/2026-10-22/F08-diff-markdown-preview-capped.pngnew file mode 100644index 0000000000..cde82fd117Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F08-diff-markdown-preview-capped.png differdiff --git a/.pr/map-maintenance/2026-10-22/F09-desktop-nav-seven.png b/.pr/map-maintenance/2026-10-22/F09-desktop-nav-seven.pngnew file mode 100644index 0000000000..d1b38d4e92Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F09-desktop-nav-seven.png differdiff --git a/.pr/map-maintenance/2026-10-22/F10-provider-picker-anth.png b/.pr/map-maintenance/2026-10-22/F10-provider-picker-anth.pngnew file mode 100644index 0000000000..19556fb86cBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F10-provider-picker-anth.png differdiff --git a/.pr/map-maintenance/2026-10-22/F12-row-menu-local.png b/.pr/map-maintenance/2026-10-22/F12-row-menu-local.pngnew file mode 100644index 0000000000..0a642de9b0Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F12-row-menu-local.png differdiff --git a/.pr/map-maintenance/2026-10-22/F13-active-drives-conversation.png b/.pr/map-maintenance/2026-10-22/F13-active-drives-conversation.pngnew file mode 100644index 0000000000..294c4c47d3Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F13-active-drives-conversation.png differdiff --git a/.pr/map-maintenance/2026-10-22/F13-default-shared-editor.png b/.pr/map-maintenance/2026-10-22/F13-default-shared-editor.pngnew file mode 100644index 0000000000..c9c9213dc5Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F13-default-shared-editor.png differdiff --git a/.pr/map-maintenance/2026-10-22/F13-list-active-badge.png b/.pr/map-maintenance/2026-10-22/F13-list-active-badge.pngnew file mode 100644index 0000000000..87cf42e447Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F13-list-active-badge.png differdiff --git a/.pr/map-maintenance/2026-10-22/F13-profile-router-editor.png b/.pr/map-maintenance/2026-10-22/F13-profile-router-editor.pngnew file mode 100644index 0000000000..ec2b942466Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F13-profile-router-editor.png differdiff --git a/.pr/map-maintenance/2026-10-22/F15-profile-behavior-known-failure.png b/.pr/map-maintenance/2026-10-22/F15-profile-behavior-known-failure.pngnew file mode 100644index 0000000000..7b7e6c1cf4Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F15-profile-behavior-known-failure.png differdiff --git a/.pr/map-maintenance/2026-10-22/F16-recommended-automations-switch.png b/.pr/map-maintenance/2026-10-22/F16-recommended-automations-switch.pngnew file mode 100644index 0000000000..361a21406aBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F16-recommended-automations-switch.png differdiff --git a/.pr/map-maintenance/2026-10-22/F17-server-health-failed-dot.png b/.pr/map-maintenance/2026-10-22/F17-server-health-failed-dot.pngnew file mode 100644index 0000000000..b3b3dee65bBinary files /dev/null and b/.pr/map-maintenance/2026-10-22/F17-server-health-failed-dot.png differdiff --git a/.pr/map-maintenance/2026-10-22/F18-toggle-default-profile.png b/.pr/map-maintenance/2026-10-22/F18-toggle-default-profile.pngnew file mode 100644index 0000000000..cc762760d6Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F18-toggle-default-profile.png differdiff --git a/.pr/map-maintenance/2026-10-22/F22-prompt-draft-no-profile.png b/.pr/map-maintenance/2026-10-22/F22-prompt-draft-no-profile.pngnew file mode 100644index 0000000000..94259721c5Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F22-prompt-draft-no-profile.png differdiff --git a/.pr/map-maintenance/2026-10-22/F22-template-filters.png b/.pr/map-maintenance/2026-10-22/F22-template-filters.pngnew file mode 100644index 0000000000..15793859c4Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F22-template-filters.png differdiff --git a/.pr/map-maintenance/2026-10-22/F25-cloud-first-message-order.png b/.pr/map-maintenance/2026-10-22/F25-cloud-first-message-order.pngnew file mode 100644index 0000000000..f7816dd869Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F25-cloud-first-message-order.png differdiff --git a/.pr/map-maintenance/2026-10-22/F25-providers-tab-default-off.png b/.pr/map-maintenance/2026-10-22/F25-providers-tab-default-off.pngnew file mode 100644index 0000000000..bafd7229a3Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F25-providers-tab-default-off.png differdiff --git a/.pr/map-maintenance/2026-10-22/F26-lib-host-app-styled.png b/.pr/map-maintenance/2026-10-22/F26-lib-host-app-styled.pngnew file mode 100644index 0000000000..0ce87d91e1Binary files /dev/null and b/.pr/map-maintenance/2026-10-22/F26-lib-host-app-styled.png differ