Skip to content

πŸ™Œ OpenHands: AI-Driven Development

feat(verify-openhands): daily pass support and CLI fixes (#18093)

Engel Nyst7ea83bab4fe7
diff --git a/.agents/skills/verify-openhands/SKILL.md b/.agents/skills/verify-openhands/SKILL.md
index 1304c4607c..3acbd38f61 100644
--- a/.agents/skills/verify-openhands/SKILL.md
+++ b/.agents/skills/verify-openhands/SKILL.md
@@ -3,9 +3,10 @@ name: verify-openhands
description: >
This skill should be used to "verify OpenHands features", "test the Canvas UI
like a user", "drive Agent Canvas", "check a UI change in the real app",
- "create or update the feature map", or "run the weekly feature audit". Ships
- control-openhands (launch, doctor, browser, LLM profiles, conversations,
- evidence, cleanup) and the maintained map of every user-facing feature.
+ "create or update the feature map", "run the daily pass" or "run the weekly
+ feature audit". Ships control-openhands (launch, doctor, browser, LLM
+ profiles, conversations, evidence, map tooling, cleanup) and the maintained
+ map of every user-facing feature.
triggers:
- /verify-openhands
- feature map
@@ -42,8 +43,9 @@ control-openhands --help # then: control-openhands <command> --he
Requirements: Node >=24 (the launcher's engine), npm dependencies installed
(`npm ci --ignore-scripts`), `uv`/`uvx`, and a Chromium Playwright can launch
(set `CONTROL_OPENHANDS_BROWSER=/path/to/chrome` when the pinned browser is not
-installed). Every command prints one JSON object; exit 0 ok, 1 action failed,
-2 usage, 3 environment.
+installed; `launch` then reports the running stack and `browser start` picks it
+up). Every command prints one JSON object; exit 0 ok, 1 action failed, 2 usage,
+3 environment.
If a user path cannot be driven with the CLI, that is a **harness gap**: extend
`scripts/control-openhands.mjs` (keep it executable, document the verb in
@@ -127,7 +129,9 @@ control-openhands stop # stops only this run; evidence s
ledger `<run>/evidence/ledger.jsonl`. Nothing is overwritten: a repeated
screenshot name is saved as `<name>-2.png`, so cite the path the command
prints; `evidence report` renders the table from
- [the report contract](references/report.md). Keys, logs, browser profile and
+ [the report contract](references/report.md), fail and blocked rows first with
+ their `--note` (a blocked row names its missing prerequisite there), and
+ `--baseline <yesterday's run>` lists what changed since that ledger. Keys, logs, browser profile and
downloads stay in `<run>/private/`. Evidence is not automatically public:
review every image before publishing it. The CLI masks password fields in
`snapshot`, `value` and `testids`; a screenshot of a visible key field is
@@ -151,9 +155,21 @@ prerequisite; never substitute a mock and call it a pass.
## Choose the job
-- **Verify a change or a PR**: map the changed paths to feature IDs (the map's
- `Source:` lines help), drive every entry point those features list at desktop
- and phone viewports, and report with [the report contract](references/report.md).
+- **Verify a change or a PR**: `control-openhands map affected --base <ref>`
+ (or `--paths` with the PR's file list) maps the changed paths to the
+ families whose `Source:` lines own them, and separates shared code to widen,
+ `src/` paths no family owns (a map gap) and non-user-facing paths. Drive every entry point those features list at
+ desktop and phone viewports, and report with
+ [the report contract](references/report.md).
+- **Run the daily pass**: follow [references/daily.md](references/daily.md):
+ static checks (`map check`, `map coverage`, `map testids`), the changed
+ families since the `Maintenance baseline` line of the map index
+ (`control-openhands map baseline`; `map affected` starts there by default),
+ a smoke row per family, and a date-derived rotation that gives every family
+ a full live pass once a week. A pass that finished its changed families
+ proposes `TARGET` as the next baseline with `map baseline --set "$TARGET"`
+ in its PR; compare ledgers with `evidence report --baseline` when
+ yesterday's is at hand.
- **Create or extend the map**: follow [references/mapping.md](references/mapping.md).
It teaches how to discover features, write entries against the CLI and prove
each one live. `control-openhands map coverage` measures what is still unmapped.
diff --git a/.agents/skills/verify-openhands/references/adaptation.md b/.agents/skills/verify-openhands/references/adaptation.md
index 1d80010eb9..062e3fc7f7 100644
--- a/.agents/skills/verify-openhands/references/adaptation.md
+++ b/.agents/skills/verify-openhands/references/adaptation.md
@@ -36,6 +36,16 @@ OpenHands-specific decisions:
- Weekly maintenance adds what OpenHands needs beyond pstack: a frozen
BASE/TARGET, a PR-intent ledger, and intent kept separate from runtime results,
with multi-repository attribution (Canvas, Agent Server/SDK, automation).
+- pstack's guide asks for the maintenance pass "at least once a day" but
+ prescribes no tiers, time budget, sharding or map-to-test linkage.
+ [daily.md](daily.md) adds those for a map this size: static checks first
+ (`map check`, `map coverage`, `map testids`), the changed families from
+ `map affected` (the `Source:` lines as a change-to-feature index, in the
+ spirit of `tests/e2e/mock-llm/test-mapping.json`), a smoke row per family,
+ a weekly rotation, and `evidence report --baseline` as the day's verdict.
+ The Playwright suites under `tests/e2e/` are not referenced from the map
+ (they run differently and prove a commit, not this run); the behaviors they
+ assert are mapped as sub-features with their own live recipes instead.
- Not imported: Cursor plugin configuration, model routing,
`disable-model-invocation`, autonomous shipping or merge authority, and the
zero-pixel-difference rule (deliberate weekly product changes are expected).
diff --git a/.agents/skills/verify-openhands/references/daily.md b/.agents/skills/verify-openhands/references/daily.md
new file mode 100644
index 0000000000..88481f60aa
--- /dev/null
+++ b/.agents/skills/verify-openhands/references/daily.md
@@ -0,0 +1,138 @@
+# The daily pass
+
+The weekly pass in [maintenance.md](maintenance.md) proves every feature live.
+A daily pass cannot: 27 families and 700-odd sub-features take longer than one
+agent's day, and most of the map did not change since yesterday. The daily
+pass answers a narrower question with the same rigor: **did anything that
+merged since the last pass change what a user sees, and does the rest still
+hold where it is cheapest to look?** It is the delta pass that
+[maintenance.md](maintenance.md) allows, and it moves the same baseline.
+
+This is a procedure, not a scheduler: run it from whatever runs you daily, do
+not create automations or post externally unless asked, and hand what you
+found to a person.
+
+## Inputs and hand-off
+
+A scheduled pass starts in a fresh session with no memory of yesterday, so
+everything it needs from the previous pass lives in the repository or on an
+open PR:
+
+- `BASE` is the `Maintenance baseline: main@<sha> (<date>)` line of
+ [the map index](feature-map/README.md), the previous pass's `TARGET`.
+ `control-openhands map baseline` reads it and says how far `HEAD` is from
+ it; `map affected` starts there when no `--base` is given. Fetch `main`
+ and freeze its full SHA as `TARGET`; deepen a shallow clone until `BASE` is
+ present.
+- A pass proposes `TARGET` as the next baseline in its PR
+ (`control-openhands map baseline --set "$TARGET"` moves the line; the SHA
+ must be on `main`) only when tier 2 finished: every family `map affected`
+ listed was driven, every commit in `BASE..TARGET` was resolved to its PR,
+ and the closed-issue rows below were re-driven. Merging that PR accepts the
+ baseline. A pass that ran out of time or was blocked in tier 2 leaves the
+ line where it was, even when its PR carries other map or CLI fixes, and says
+ so in its report (maintenance.md's blocked/partial outcome); a pass that
+ changes nothing leaves it too, so the next pass covers a longer range. If an
+ earlier pass's PR is still open, the next pass continues on that branch and
+ takes `BASE` from the line there.
+- The rotation position (which families get their full recipe today) is
+ derived from the date, below, so no state carries it.
+- Yesterday's `evidence/ledger.jsonl` is an optional input: when the previous
+ run's ledger is at hand (a CI artifact, a shared directory), pass it to
+ `evidence report --baseline`; without it the report has no "Changes since
+ baseline" section and the verdict rests on today's fail and blocked rows.
+
+Record the UTC time, model and spend budget, and the accounts and keys
+available, as the weekly pass does. A pass that runs out of time says so and
+names what it did not reach; it never rolls the remainder into tomorrow
+silently.
+
+## Tiers, in order
+
+Run the tiers in this order and stop when the budget runs out; each tier is
+cheaper than the next and catches a different kind of rot.
+
+1. **Static (minutes, no stack).** `control-openhands map check`,
+ `map coverage` and `map testids`. A `Source:` path that is gone, or a test
+ id the map drives that no literal in `src/` accounts for, is drift found
+ before any browser opens. Fix it in the map
+ with a live drive later today, or report it. Zero unresolved test ids is
+ the normal state; treat a new one as a rename until a drive says otherwise.
+2. **Changed (the core).** `control-openhands map affected --target $TARGET`
+ (`--base` defaults to the recorded baseline). Launch, doctor, then drive
+ **every** sub-feature the
+ listed families map for the changed paths, on each entry point the family
+ lists, at desktop and phone viewports for UI changes. Widen `shared` paths
+ to their consumers (an API client or store change touches every page that
+ reads it; pick the families whose `Source:` lines name those consumers, not
+ one convenient screen). An `unmapped` path under `src/` is a map gap: a new
+ surface to map per [mapping.md](mapping.md), or a `Source:` line to extend.
+ Resolve each merged commit in `BASE..TARGET` to its PR and keep intent
+ (`documented`, `undocumented`, `contradictory`) separate from runtime
+ results, as [maintenance.md](maintenance.md) step 2 describes. The delta
+ pass has a second half: every bullet whose Gotchas or Known failure note
+ links an issue that closed since `BASE` (check each linked issue's state)
+ is re-driven too, its row recorded and its note updated. A fix that landed
+ in the Agent Server or the automation service shows up only this way.
+3. **Smoke (every family, cheaply).** For each family not already driven in
+ tier 2: open its first entry point, run the family's `errors --app-only`
+ sweep, and drive the one bullet that proves the page's main state (a list
+ renders, a form opens). One `evidence add` row per family, under the
+ sub-feature ID that bullet names. This catches a page that stopped loading
+ without re-proving every row.
+4. **Rotation (depth over the week).** Drive the full recipe of about a
+ seventh of the families, four a day, so every family gets a full live pass
+ once a week between weekly maintenance passes. The day's slice comes from
+ the date, not from yesterday's notes: with `d` the UTC day of the year,
+ today's families are the ones at positions `(d mod 7) * 4` to
+ `(d mod 7) * 4 + 3` of the index table, in ID order (position 24 onwards
+ holds F25 to F27). Model-backed bullets run only inside the LLM budget;
+ without a key they are `blocked` with the missing prerequisite.
+
+## Reading the result
+
+Render `control-openhands evidence report`, with `--baseline <yesterday's
+ledger>` when that ledger is at hand. Fail and blocked rows come first; the
+"Changes since baseline" section, when present, is the daily verdict:
+
+- **Newly failing** on a family that tier 2 drove for a changed path: an
+ introduced-in-range candidate. Re-drive the same minimal recipe on `BASE`
+ (a second `launch --new` on a worktree at `BASE`) before calling it a
+ regression; otherwise classify it as reproduced-on-both, environment or
+ harness gap, or origin-unconfirmed (maintenance.md step 4).
+- **Newly failing** elsewhere, or **newly passing** without a PR that explains
+ it: an undocumented change; cite the commit and ask.
+- **Not checked this run** rows are the rotation moving on, not regressions;
+ the family table shows which families got depth today.
+- Without yesterday's ledger, read today's fail rows the same way: a fail on a
+ family tier 2 drove for a changed path is a candidate to re-drive on
+ `BASE`; elsewhere it is a finding to triage, with the family's last known
+ state taken from the map (a bullet's Known failure note, a linked issue).
+
+Triage as the skill does: map drift (fix, with live evidence), harness gap
+(extend the CLI, re-drive), product defect (keep the evidence, search issues,
+file in the owning repository, keep it out of the map PR), blocked (name the
+prerequisite and the route). At most one PR of proven map and CLI corrections
+per pass, from current `main` (or from the previous pass's branch while its
+PR is open), carrying the moved baseline line and saying "delta" in its
+report; never edit product code in a pass.
+
+## Several agents
+
+Shard by family, never by bullet: each agent gets whole families, its own
+`launch --new`, its own `OH_VERIFY_RUN`, and writes only its own ledger. One
+coordinator merges the ledgers' reports and owns the hand-off. A result that
+names no run, revision or entry point does not count; a gap is not a pass.
+
+## What the daily pass is not
+
+- Not the full pass: it does not re-prove every bullet, and the baseline it
+ moves says only that the day's changes were covered, which is why an
+ unfinished tier 2 never moves it. When tier 2 finds that
+ most of the map is affected (a shared component or style change), say so
+ and run the full procedure instead. A full pass is still needed weekly or
+ before a release, because Agent Server and automation releases change
+ behavior without touching this repository.
+- Not CI: a green Playwright run or a passing `map check` is an input to a
+ verdict, never the verdict.
+- Not a bug-fix lane: product defects are filed, not fixed, inside the pass.
diff --git a/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md b/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md
index 167d2fdf25..66d2d3a02c 100644
--- a/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md
+++ b/.agents/skills/verify-openhands/references/feature-map/F02-app-shell.md
@@ -10,7 +10,7 @@ a top bar or the chat header. A user can pin Customize or Automate as the page
`/` opens. A Ctrl/Cmd+K command menu searches pages, settings and actions.
Failed requests show error toasts.
-Source: `src/components/features/sidebar/`, `src/components/features/command-menu/`, `src/stores/sidebar-store.ts`, `src/stores/command-menu-store.ts`, `src/hooks/use-pinned-home-route.ts`, `src/routes/index-home.tsx`, `src/utils/mobile-section-nav.ts`, `src/components/features/settings/agent-canvas-version-tile.tsx`, `src/components/features/alerts/alert-banner.tsx`, `src/utils/custom-toast-handlers.ts`.
+Source: `src/components/features/sidebar/`, `src/components/features/command-menu/`, `src/stores/sidebar-store.ts`, `src/stores/command-menu-store.ts`, `src/hooks/use-pinned-home-route.ts`, `src/routes/index-home.tsx`, `src/utils/mobile-section-nav.ts`, `src/components/features/settings/agent-canvas-version-tile.tsx`, `src/components/features/alerts/alert-banner.tsx`, `src/utils/custom-toast-handlers.tsx`.
## Sub-features
diff --git a/.agents/skills/verify-openhands/references/feature-map/F18-skills.md b/.agents/skills/verify-openhands/references/feature-map/F18-skills.md
index 05318893ef..157a4d4eb5 100644
--- a/.agents/skills/verify-openhands/references/feature-map/F18-skills.md
+++ b/.agents/skills/verify-openhands/references/feature-map/F18-skills.md
@@ -9,7 +9,7 @@ launches a skill into chat, and reads how to add one with `/add-skill`; after th
agent installs a skill in a conversation, a banner offers a fresh conversation
that loads it.
-Source: `src/routes/skills-settings.tsx`, `src/components/features/skills/`, `src/hooks/use-skill-enablement.ts`, `src/utils/skill-enablement.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/api/skills-service.ts`, `src/components/features/chat/skill-install-restart-banner.tsx`, `src/utils/skill-install-events.ts`.
+Source: `src/routes/skills-settings.tsx`, `src/components/features/skills/`, `src/constants/skills-docs.ts`, `src/hooks/use-skill-enablement.ts`, `src/utils/skill-enablement.ts`, `src/hooks/use-launch-skill-in-chat.ts`, `src/api/skills-service.ts`, `src/components/features/chat/skill-install-restart-banner.tsx`, `src/utils/skill-install-events.ts`.
## Sub-features
diff --git a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md
index e410682315..67b1e2757c 100644
--- a/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md
+++ b/.agents/skills/verify-openhands/references/feature-map/F26-runtime-variants.md
@@ -10,7 +10,7 @@ Electron desktop app and an embeddable React library. Launchers tell the agent
which services exist through `/server_info.runtime_services` and a
`<RUNTIME_SERVICES>` block in every new conversation's system prompt.
-Source: `bin/agent-canvas.mjs`, `scripts/dev-with-automation.mjs`, `scripts/bind-host.mjs`, `scripts/static-server.mjs`, `scripts/ingress.mjs`, `scripts/runtime-services-info.mjs`, `config/defaults.json`, `src/api/agent-server-adapter.ts`, `src/api/backend-registry/`, `docker/`, `electron/`, `helm/agent-canvas/`, `src/lib/index.ts`, `docs/SELF_HOSTING.md`.
+Source: `bin/agent-canvas.mjs`, `scripts/dev-with-automation.mjs`, `scripts/dev-safe.mjs`, `scripts/dev-static.mjs`, `scripts/dev-extra-backend.mjs`, `scripts/bind-host.mjs`, `scripts/static-server.mjs`, `scripts/ingress.mjs`, `scripts/runtime-services-info.mjs`, `config/defaults.json`, `src/api/agent-server-adapter.ts`, `src/api/backend-registry/`, `docker/`, `electron/`, `helm/agent-canvas/`, `src/lib/index.ts`, `docs/SELF_HOSTING.md`.
## Sub-features
diff --git a/.agents/skills/verify-openhands/references/feature-map/README.md b/.agents/skills/verify-openhands/references/feature-map/README.md
index da10de8bdf..7105876fdd 100644
--- a/.agents/skills/verify-openhands/references/feature-map/README.md
+++ b/.agents/skills/verify-openhands/references/feature-map/README.md
@@ -211,8 +211,8 @@ Maintenance baseline: main@ed815e141409c7b52991ab8d13c553b595da9165 (2026-10-06)
## Feature entry contract
Each feature file starts with an H1 title, a short paragraph describing the
-user-visible behavior and a `Source:` line. It then uses exactly four H2 sections
-in this order (`control-openhands map check` enforces it):
+user-visible behavior and a `Source:` line whose paths must exist. It then uses
+exactly four H2 sections in this order (`control-openhands map check` enforces it):
1. `Sub-features`: one bullet per stable ID (`` `Fnn.slug` ``) and behavior.
2. `How to get to it (user POV)`: every user entry point.
diff --git a/.agents/skills/verify-openhands/references/maintenance.md b/.agents/skills/verify-openhands/references/maintenance.md
index f782f79e61..a45af2a04d 100644
--- a/.agents/skills/verify-openhands/references/maintenance.md
+++ b/.agents/skills/verify-openhands/references/maintenance.md
@@ -3,7 +3,9 @@
A feature map rots the moment the app changes. This pass keeps it honest and
answers three separate questions about the interval since the last pass:
**Is each feature present? Does it work and look right? Was each change
-documented and intended?** A merged PR proves none of the last two.
+documented and intended?** A merged PR proves none of the last two. Between
+full passes, [daily.md](daily.md) runs the delta pass each day against the
+same `Maintenance baseline` line of the map index, and moves it the same way.
The unit of rigor is the feature: every feature file gets source coverage and
live coverage, without re-proving every sentence. This is a procedure, not a
@@ -69,9 +71,11 @@ to justify a change after the fact.
## 3. Index hygiene and source wave
-Run `control-openhands map check` and `map coverage`; fix missing, duplicate and
-dead entries (`map check --fix-counts` refreshes the index counts). Then give one read-only reader per feature file (parallel if
-delegation is available). Each reads current source for that feature and returns:
+Run `control-openhands map check`, `map coverage` and `map testids`; fix
+missing, duplicate and dead entries, `Source:` paths that are gone and test
+ids no literal in `src/` accounts for (`map check --fix-counts` refreshes the
+index counts). Then give one read-only reader per
+feature file (parallel if delegation is available). Each reads current source for that feature and returns:
summary, source entry points, likely drift with citations (or none), new
surfaces missing from the map, and one live recipe. Readers never edit files or
drive the browser.
@@ -82,10 +86,12 @@ re-drive any that a current verb can now reach (compare with
`control-openhands --help` and `help browser`).
Map **every changed path** in BASE..TARGET to a feature ID or an explicit
-non-user-facing reason. For shared components, CSS, API clients, settings
-schemas and dependency bumps, expand to their consumers rather than sampling one
-convenient screen. A new surface needs a concrete source path before it is
-called missing; then add it per [mapping.md](mapping.md).
+non-user-facing reason: `control-openhands map affected --base "$BASE" --target
+"$TARGET"` does the first cut from the `Source:` lines. Its `shared` paths (components, CSS, API clients,
+hooks, stores, settings schemas, dependency bumps) expand to their consumers
+rather than sampling one convenient screen; its `unmapped` paths under `src/`
+are map gaps. A new surface needs a concrete source path before it is called
+missing; then add it per [mapping.md](mapping.md).
## 4. Live pass
@@ -131,7 +137,9 @@ automatically a bug and a zero difference does not prove behavior.
Use [the report contract](report.md): verdict first, then family and check
counts, intent gaps, defects with issue links, blocked checks with their
-prerequisites. For **changed**, open at most one PR from current `main`, re-read
+prerequisites (`control-openhands evidence report --baseline <previous run>`
+renders the counts, the blocked rows' notes and what changed since the last
+accepted ledger). For **changed**, open at most one PR from current `main`, re-read
every changed file first, and follow the repository's PR template. Finish with
`control-openhands stop`, retained evidence paths, and the exact inputs needed to
unblock what was blocked. Name the proposed next baseline; never replace an
diff --git a/.agents/skills/verify-openhands/references/mapping.md b/.agents/skills/verify-openhands/references/mapping.md
index 98970cf0a7..c8799239ad 100644
--- a/.agents/skills/verify-openhands/references/mapping.md
+++ b/.agents/skills/verify-openhands/references/mapping.md
@@ -44,8 +44,12 @@ candidate. Sources, roughly in order of yield:
6. **Backends and modes**: local vs Cloud backend, public vs local auth, ACP
agents, partial stacks (`agent-canvas --help`), Electron, Docker, and the
embeddable library (`src/index.ts` / `build:lib`).
-7. **Docs and specs**: `README.md`, `docs/*.md`, `specs/*.md` describe intended
- behavior; mock-LLM specs in `tests/e2e/mock-llm/` are selector references only.
+7. **Docs, specs and tests**: `README.md`, `docs/*.md`, `specs/*.md` describe
+ intended behavior. The Playwright suites under `tests/e2e/` (mock-LLM, live,
+ bind-policy, live-acp) assert behaviors a user sees too: read them for
+ selectors and for behaviors the map lacks, then map those behaviors as
+ sub-features with their own recipes. The map never cites a spec: the
+ suites run differently and prove a commit, not this run.
8. **Strings**: `src/i18n/translation.json` keys reveal user-visible states
(empty, error, disabled, confirmation) that a happy path never shows.
@@ -78,10 +82,12 @@ that ID instead of re-mapping it.
Each file starts with an H1 title, one paragraph describing the user-visible
behavior, and one `Source:` line with the main implementation paths (for drift
-checks; keep other implementation detail out). For a family spread over many
-files, list the route modules and top-level component directories, not every
-file. Then exactly four H2 sections, in
-this order:
+checks and `map affected`; keep other implementation detail out). Every path
+must exist (`map check` verifies it); a directory ends with `/`, names in
+parentheses after a directory are relative to it, and `name-*.tsx` matches by
+prefix. For a family spread over many files, list the route modules and
+top-level component directories, not every file. Then exactly four H2
+sections, in this order:
1. `## Sub-features`: one bullet per ID: `` - `F14.create`: add a dummy secret; it persists after reload. ``
2. `## How to get to it (user POV)`: every entry point, in user language: the
diff --git a/.agents/skills/verify-openhands/scripts/browser-daemon.mjs b/.agents/skills/verify-openhands/scripts/browser-daemon.mjs
index c89cbc022a..d37c544506 100755
--- a/.agents/skills/verify-openhands/scripts/browser-daemon.mjs
+++ b/.agents/skills/verify-openhands/scripts/browser-daemon.mjs
@@ -814,6 +814,31 @@ const handlers = {
else await activePage.screenshot({ path, fullPage: Boolean(fullPage) });
return { path, url: activePage.url(), viewport: activePage.viewportSize() };
},
+ async clock({ offsetMs, system, fixed }) {
+ // The page's clock, for states that depend on the browser's time
+ // differing from the server's (a message stamped in the future). The
+ // app's own timers keep running; install before the navigation whose
+ // page should see the skewed time.
+ const when = (value) =>
+ /^[+-]\d+$/.test(String(value))
+ ? Date.now() + Number(value)
+ : new Date(value).getTime();
+ if (fixed !== undefined) {
+ await activePage.clock.setFixedTime(when(fixed));
+ return { clock: "fixed", time: new Date(when(fixed)).toISOString() };
+ }
+ if (system !== undefined) {
+ await activePage.clock.setSystemTime(when(system));
+ return { clock: "system", time: new Date(when(system)).toISOString() };
+ }
+ const time = Date.now() + Number(offsetMs ?? 0);
+ await activePage.clock.install({ time });
+ return {
+ clock: "installed",
+ offsetMs: Number(offsetMs ?? 0),
+ time: new Date(time).toISOString(),
+ };
+ },
async viewport({ size }) {
const preset = VIEWPORTS[size];
let next = preset;
diff --git a/.agents/skills/verify-openhands/scripts/control-openhands.mjs b/.agents/skills/verify-openhands/scripts/control-openhands.mjs
index a8a57a99ab..35d855e460 100755
--- a/.agents/skills/verify-openhands/scripts/control-openhands.mjs
+++ b/.agents/skills/verify-openhands/scripts/control-openhands.mjs
@@ -32,7 +32,15 @@ import {
portsAfterRestart,
runPorts,
} from "./lib/launcher-env.mjs";
+import { baselineLines, parseBaseline, withBaseline } from "./lib/baseline.mjs";
+import { affectedFamilies, familyHead } from "./lib/map-sources.mjs";
import { routePattern } from "./lib/route-pattern.mjs";
+import {
+ citedTestids,
+ resolveTestids,
+ sourceLiterals,
+} from "./lib/testids.mjs";
+import { tmuxPathFor } from "./lib/tmux-path.mjs";
const here = dirname(fileURLToPath(import.meta.url));
const skillDir = resolve(here, "..");
@@ -545,8 +553,21 @@ async function cmdLaunch({ flags }) {
);
}
- const existing = flags.new ? undefined : loadRun(flags, { required: false });
- if (existing && !flags.new && groupAlive(existing.launcherPgid)) {
+ let existing;
+ if (!flags.new) {
+ try {
+ existing = loadRun(flags, { required: false });
+ } catch (error) {
+ // Several live runs: reuse none of them, say how to start another.
+ if (!(error instanceof CliError) || !/runs are alive/.test(error.message))
+ throw error;
+ throw new CliError(error.message, {
+ code: error.code,
+ hint: `${error.hint} To start another independent run, pass --new.`,
+ });
+ }
+ }
+ if (existing && groupAlive(existing.launcherPgid)) {
out({
ok: true,
alreadyRunning: true,
@@ -778,7 +799,19 @@ async function cmdLaunch({ flags }) {
saveRun(run);
let browser;
- if (!flags["no-browser"]) browser = await startBrowser(run);
+ if (!flags["no-browser"]) {
+ try {
+ browser = await startBrowser(run);
+ } catch (error) {
+ if (!(error instanceof CliError)) throw error;
+ // The stack is up and the run is saved: only the browser is missing.
+ throw new CliError(error.message, {
+ code: error.code,
+ hint: `${error.hint} The stack is running: export OH_VERIFY_RUN=${dir}, fix the browser, then \`control-openhands browser start\` (or \`stop\` the run).`,
+ extra: { ...(error.extra ?? {}), run: dir, baseUrl: run.baseUrl },
+ });
+ }
+ }
if (flags["print-run"]) {
// For: export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
@@ -1222,7 +1255,7 @@ async function cmdDoctor({ flags }) {
}
function tmuxDirFor(runDir) {
- const dir = join("/tmp", `ohv-tmux-${basename(runDir).slice(-6)}`);
+ const dir = tmuxPathFor(runDir);
mkdirSync(dir, { recursive: true, mode: 0o700 });
return dir;
}
@@ -1240,10 +1273,7 @@ async function cmdStop({ flags }) {
result.launcherStopped = !groupAlive(pgid);
if (result.launcherStopped) {
releaseClaim(run);
- rmSync(join("/tmp", `ohv-tmux-${basename(run.dir).slice(-6)}`), {
- recursive: true,
- force: true,
- });
+ rmSync(tmuxPathFor(run.dir), { recursive: true, force: true });
}
result.ports = ports;
run.stoppedAt = new Date().toISOString();
@@ -2720,6 +2750,22 @@ async function cmdBrowser({ positional, flags }) {
fullPage: Boolean(flags["full-page"]),
});
break;
+ case "clock":
+ if (
+ !flags["offset-ms"] &&
+ flags.system === undefined &&
+ flags.fixed === undefined
+ )
+ usage(
+ "browser clock needs --offset-ms N | --system ISO|+MS | --fixed ISO|+MS",
+ "control-openhands browser clock --offset-ms 300000 # five minutes ahead",
+ );
+ result = await browserCall(run, "clock", {
+ offsetMs: flags["offset-ms"],
+ system: flags.system,
+ fixed: flags.fixed,
+ });
+ break;
case "viewport":
need(
1,
@@ -2786,6 +2832,50 @@ async function cmdBrowser({ positional, flags }) {
// ---------------------------------------------------------------------------
const RESULTS = ["pass", "fail", "blocked", "not-run"];
+// The ledger is append-only: the report shows each check's latest row,
+// keyed by feature and entry point, minus retractions.
+function latestRows(rows) {
+ const latest = new Map();
+ for (const row of rows) {
+ const key = `${row.feature}|${row.entry ?? ""}`;
+ if (row.retracted) {
+ // A retraction without --entry drops every row of that feature.
+ for (const k of [...latest.keys()]) {
+ if (k === key || (!row.entry && k.startsWith(`${row.feature}|`)))
+ latest.delete(k);
+ }
+ } else latest.set(key, row);
+ }
+ return latest;
+}
+
+// What changed between two ledgers' latest rows (same feature and entry).
+function compareLedgers(before, after) {
+ const newlyFailing = [];
+ const newlyPassing = [];
+ const other = [];
+ const missing = [];
+ const entry = (row) => ({
+ feature: row.feature,
+ entry: row.entry,
+ note: row.note,
+ });
+ for (const [key, row] of after) {
+ const prev = before.get(key);
+ if (prev?.result === row.result) continue;
+ const change = { ...entry(row), before: prev?.result, after: row.result };
+ if (row.result === "fail" && prev?.result !== "fail")
+ newlyFailing.push(change);
+ else if (row.result === "pass" && prev && prev.result !== "pass")
+ newlyPassing.push(change);
+ else other.push(change);
+ }
+ for (const [key, row] of before)
+ if (!after.has(key))
+ missing.push({ ...entry(row), before: row.result, after: undefined });
+ return { newlyFailing, newlyPassing, other, missing };
+}
+
async function cmdEvidence({ positional, flags }) {
const run = loadRun(flags);
const ledger = join(run.dir, "evidence", "ledger.jsonl");
@@ -2866,38 +2956,112 @@ async function cmdEvidence({ positional, flags }) {
return;
}
if (sub === "report") {
- const latest = new Map();
- for (const row of rows) {
- const key = `${row.feature}|${row.entry ?? ""}`;
- if (row.retracted) {
- // A retraction without --entry drops every row of that feature.
- for (const k of [...latest.keys()]) {
- if (k === key || (!row.entry && k.startsWith(`${row.feature}|`)))
- latest.delete(k);
- }
- } else latest.set(key, row);
- }
+ const latest = latestRows(rows);
const counts = Object.fromEntries(RESULTS.map((r) => [r, 0]));
for (const row of latest.values()) counts[row.result] += 1;
+ // Per family: the denominator the report contract asks for.
+ const families = new Map();
+ for (const row of latest.values()) {
+ const id = row.feature.slice(0, 3);
+ if (!families.has(id))
+ families.set(id, Object.fromEntries(RESULTS.map((r) => [r, 0])));
+ families.get(id)[row.result] += 1;
+ }
+ // --baseline PATH: another run's ledger.jsonl (or run directory). Rows
+ // are compared by feature and entry point; what changed is listed first.
+ let changes;
+ if (flags.baseline && flags.baseline !== true) {
+ let path = resolve(String(flags.baseline));
+ if (existsSync(join(path, "evidence", "ledger.jsonl")))
+ path = join(path, "evidence", "ledger.jsonl");
+ if (!existsSync(path))
+ throw new CliError(`No ledger at ${path}`, { code: 2 });
+ const before = latestRows(
+ readFileSync(path, "utf8")
+ .trim()
+ .split("\n")
+ .filter(Boolean)
+ .map((l) => JSON.parse(l)),
+ );
+ changes = compareLedgers(before, latest);
+ }
+ if (flags.json) {
+ out({
+ ok: true,
+ run: run.runId,
+ revision: run.revision,
+ checks: latest.size,
+ counts,
+ families: Object.fromEntries(
+ [...families.entries()].sort(([a], [b]) => a.localeCompare(b)),
+ ),
+ rows: [...latest.values()],
+ ...(changes ? { changes } : {}),
+ });
+ return;
+ }
+ const cell = (v) =>
+ String(v ?? "")
+ .replace(/\|/g, "\\|")
+ .replace(/\n/g, " ");
const lines = [
`# Verification ledger β€” run ${run.runId}`,
"",
`Revision \`${run.revision}\`, ${run.mode} mode, Agent Server ${run.versions?.agentServer ?? "?"}, base ${run.baseUrl}.`,
"",
- `Checks: ${latest.size} β€” ${RESULTS.map((r) => `${r} ${counts[r]}`).join(", ")}.`,
+ `Checks: ${latest.size} β€” ${RESULTS.map((r) => `${r} ${counts[r]}`).join(", ")}. Families: ${families.size}.`,
"",
- "| Feature/check | Entry point | Expected β†’ actual | Result | Evidence |",
- "|---|---|---|---|---|",
];
- const cell = (v) =>
- String(v ?? "")
- .replace(/\|/g, "\\|")
- .replace(/\n/g, " ");
- for (const row of [...latest.values()].sort((a, b) =>
- a.feature.localeCompare(b.feature),
+ if (changes) {
+ lines.push(`## Changes since baseline`, "");
+ const section = (title, list) => {
+ if (!list.length) return;
+ lines.push(`${title}:`, "");
+ for (const c of list)
+ lines.push(
+ `- ${c.feature}${c.entry ? ` (${c.entry})` : ""}: ${c.before ?? "absent"} β†’ ${c.after ?? "absent"}${c.note ? ` β€” ${c.note}` : ""}`,
+ );
+ lines.push("");
+ };
+ section("Newly failing", changes.newlyFailing);
+ section("Newly passing", changes.newlyPassing);
+ section("Other changes", changes.other);
+ section("Not checked this run", changes.missing);
+ if (
+ !changes.newlyFailing.length &&
+ !changes.newlyPassing.length &&
+ !changes.other.length &&
+ !changes.missing.length
+ )
+ lines.push("No result changed against the baseline.", "");
+ }
+ lines.push(
+ "## Families",
+ "",
+ "| Family | pass | fail | blocked | not-run |",
+ "|---|---|---|---|---|",
+ );
+ for (const [id, c] of [...families.entries()].sort(([a], [b]) =>
+ a.localeCompare(b),
+ ))
+ lines.push(
+ `| ${id} | ${c.pass} | ${c.fail} | ${c.blocked} | ${c["not-run"]} |`,
+ );
+ lines.push(
+ "",
+ "## Checks",
+ "",
+ "| Feature/check | Entry point | Expected β†’ actual | Result | Note | Evidence |",
+ "|---|---|---|---|---|---|",
+ );
+ // Fail and blocked rows first (report.md), then by feature ID.
+ const order = { fail: 0, blocked: 1, "not-run": 2, pass: 3 };
+ for (const row of [...latest.values()].sort(
+ (a, b) =>
+ order[a.result] - order[b.result] || a.feature.localeCompare(b.feature),
)) {
lines.push(
- `| ${cell(row.feature)} | ${cell(row.entry)} | ${cell(row.expected)} β†’ ${cell(row.actual)} | ${row.result} | ${row.artifacts.map((a) => `\`${cell(relative(run.dir, resolve(run.dir, a)))}\``).join(" ")} |`,
+ `| ${cell(row.feature)} | ${cell(row.entry)} | ${cell(row.expected)} β†’ ${cell(row.actual)} | ${row.result} | ${cell(row.note)} | ${row.artifacts.map((a) => `\`${cell(relative(run.dir, resolve(run.dir, a)))}\``).join(" ")} |`,
);
}
process.stdout.write(`${lines.join("\n")}\n`);
@@ -2978,6 +3142,7 @@ const BROWSER_VERBS = new Set([
"testids",
"screenshot",
"viewport",
+ "clock",
"errors",
"events",
"eval",
@@ -3012,6 +3177,23 @@ function mapCheck({ only } = {}) {
? readFileSync(join(mapDir, "README.md"), "utf8")
: "";
if (!index) problems.push("references/feature-map/README.md is missing");
+ // The baseline line, when present, must be the only one (a merge can leave
+ // two) and carry a full SHA and a date: map affected and the passes read it.
+ const baselines = baselineLines(index);
+ if (baselines.length > 1)
+ problems.push(
+ `README.md: ${baselines.length} Maintenance baseline lines; keep the one the latest pass proposed`,
+ );
+ for (const line of baselines) {
+ if (!parseBaseline(line))
+ problems.push(
+ "README.md: the Maintenance baseline line must read `Maintenance baseline: main@<sha> (<YYYY-MM-DD>)`",
+ );
+ else if (!/main@[0-9a-f]{40} /.test(line))
+ problems.push(
+ "README.md: the Maintenance baseline line must carry the full 40-character SHA",
+ );
+ }
const files = featureFiles();
for (const file of files) {
const text = readFileSync(join(mapDir, file), "utf8");
@@ -3055,6 +3237,16 @@ function mapCheck({ only } = {}) {
const target = resolve(mapDir, m[1]);
if (!existsSync(target)) problems.push(`${file}: dead link ${m[1]}`);
}
+ // The Source: line is what drift checks and `map affected` read: every
+ // path it names must exist (node_modules content is not checked out).
+ const head = familyHead(text);
+ if (!head.sources.length)
+ problems.push(`${file}: no Source: line with implementation paths`);
+ for (const s of head.sources) {
+ if (s.path.startsWith("node_modules/")) continue;
+ if (!sourcePathExists(s))
+ problems.push(`${file}: Source: path ${s.path} does not exist`);
+ }
}
// References (`Fnn.slug` in prose, --feature Fnn.slug in recipes) must name
// a declared ID once that family's file exists.
@@ -3132,6 +3324,31 @@ function fixIndexCounts() {
writeFileSync(path, text);
}
+// A Source: path spec exists when the file or directory is there, or, for a
+// `name-*.tsx` glob, when at least one file in its directory matches.
+function sourcePathExists(s) {
+ const full = join(repoRoot, s.path);
+ if (!s.glob) return existsSync(full);
+ const dir = dirname(full);
+ if (!existsSync(dir)) return false;
+ const [head, tail = ""] = basename(s.path).split("*");
+ return readdirSync(dir).some(
+ (name) => name.startsWith(head) && name.endsWith(tail),
+ );
+}
+
+// Families with their parsed Source: lines.
+function familyList() {
+ return featureFiles().map((file) => {
+ const text = readFileSync(join(mapDir, file), "utf8");
+ return {
+ id: /^(F\d{2})/.exec(file)?.[1] ?? file,
+ file,
+ ...familyHead(text),
+ };
+ });
+}
+
function routePaths() {
const source = readFileSync(join(repoRoot, "src", "routes.ts"), "utf8");
const routes = [];
@@ -3229,6 +3446,139 @@ function mapIdList() {
return ids;
}
+// The map index (references/feature-map/README.md), home of the baseline line.
+function indexText() {
+ const path = join(mapDir, "README.md");
+ return existsSync(path) ? readFileSync(path, "utf8") : "";
+}
+
+// The recorded baseline, with where it stands relative to HEAD.
+// Two baseline lines (a merge kept both sides) are nobody's baseline.
+function assertOneBaselineLine(index) {
+ const lines = baselineLines(index);
+ if (lines.length > 1)
+ throw new CliError(
+ `The map index has ${lines.length} Maintenance baseline lines.`,
+ {
+ code: 2,
+ hint: "Remove the line that is not the latest pass's proposal (a merge kept both sides), then re-run.",
+ },
+ );
+}
+
+function baselineInfo() {
+ const index = indexText();
+ assertOneBaselineLine(index);
+ const recorded = parseBaseline(index);
+ if (!recorded) return null;
+ let known = false;
+ let ancestorOfHead = false;
+ let commitsSince;
+ try {
+ execFileSync("git", ["cat-file", "-e", `${recorded.sha}^{commit}`], {
+ cwd: repoRoot,
+ stdio: "ignore",
+ });
+ known = true;
+ ancestorOfHead =
+ spawnSync("git", ["merge-base", "--is-ancestor", recorded.sha, "HEAD"], {
+ cwd: repoRoot,
+ }).status === 0;
+ if (ancestorOfHead)
+ commitsSince = Number(
+ git(["rev-list", "--first-parent", "--count", `${recorded.sha}..HEAD`]),
+ );
+ } catch {
+ // Not in this clone (shallow, or a SHA from another branch).
+ }
+ return { ...recorded, known, ancestorOfHead, commitsSince };
+}
+
+// Changed paths between two revisions (or from a file), mapped to families.
+function mapAffected(flags) {
+ let changed;
+ let range;
+ let baseSource = "--base";
+ if (flags.paths && flags.paths !== true) {
+ const text =
+ String(flags.paths) === "-"
+ ? readFileSync(0, "utf8")
+ : readFileSync(resolve(String(flags.paths)), "utf8");
+ changed = text
+ .split("\n")
+ .map((l) => l.trim())
+ .filter(Boolean);
+ } else {
+ // Without --base, the range starts at the maintenance baseline the map
+ // index records (the previous pass's TARGET), so a daily pass needs no
+ // memory of yesterday.
+ let base =
+ flags.base && flags.base !== true ? String(flags.base) : undefined;
+ if (!base) {
+ const index = indexText();
+ assertOneBaselineLine(index);
+ const recorded = parseBaseline(index);
+ if (!recorded)
+ usage(
+ "map affected needs --base REF [--target REF] or --paths FILE|-: the map index has no `Maintenance baseline:` line to start from",
+ "control-openhands map affected --base origin/main",
+ );
+ base = recorded.sha;
+ baseSource = "map index baseline";
+ }
+ const target =
+ flags.target && flags.target !== true ? String(flags.target) : "HEAD";
+ try {
+ changed = git(["diff", "--name-only", `${base}..${target}`])
+ .split("\n")
+ .filter(Boolean);
+ } catch (error) {
+ // Say which end is missing, and why: a shallow clone that lacks the
+ // commit is deepened; a commit that is not an ancestor of the target
+ // was recorded from a branch, or the target is not main.
+ const known = (ref) =>
+ spawnSync("git", ["cat-file", "-e", `${ref}^{commit}`], {
+ cwd: repoRoot,
+ }).status === 0;
+ let reason;
+ let hint;
+ if (!known(base)) {
+ reason = `${base} is not in this clone`;
+ hint =
+ "A shallow clone may not hold the baseline: fetch main and deepen until it is present (maintenance.md step 1).";
+ } else if (!known(target)) {
+ reason = `${target} is not in this clone`;
+ hint = "Fetch the target ref first (git fetch origin main).";
+ } else {
+ reason = `${base} is not an ancestor of ${target}`;
+ hint =
+ baseSource === "map index baseline"
+ ? "The recorded baseline is not on this history: it was set from a branch, or the target is not main. Pass --base with a commit on main, and fix the line with map baseline --set <main commit>."
+ : "Pick a base that the target descends from.";
+ }
+ throw new CliError(`git diff ${base}..${target} failed: ${reason}.`, {
+ code: 3,
+ hint,
+ extra: { git: String(error.message).split("\n")[0] },
+ });
+ }
+ range = { base, target, baseSource };
+ }
+ const result = affectedFamilies(familyList(), changed);
+ return {
+ ...(range ? { range } : {}),
+ changed: changed.length,
+ families: result.families,
+ // Shared code (API clients, hooks, stores, styles, i18n): widen to the
+ // consumers rather than sampling one screen (maintenance.md step 3).
+ shared: result.shared,
+ // Under src/ but owned by no family's Source: line: a map gap, or a
+ // Source: line to extend.
+ unmapped: result.unmapped,
+ nonUserFacing: result.nonUserFacing,
+ };
+}
+
async function cmdMap({ positional, flags }) {
const [sub] = positional;
if (sub === "check") {
@@ -3265,8 +3615,110 @@ async function cmdMap({ positional, flags }) {
out({ ok: true, routes: routePaths() });
return;
}
+ if (sub === "affected") {
+ const result = mapAffected(flags);
+ out({ ok: true, ...result });
+ return;
+ }
+ if (sub === "baseline") {
+ if (flags.set !== undefined) {
+ // Move the line to a main commit's full SHA and committer date: what a
+ // pass that finished its range does before opening its PR. The line
+ // reads main@<sha>, so the commit must be on main: a pass's branch tip
+ // (a merge commit, the PR's own fixes) never is.
+ if (flags.set === true)
+ usage(
+ "map baseline --set needs the frozen TARGET (a commit on main)",
+ 'control-openhands map baseline --set "$TARGET"',
+ );
+ const ref = String(flags.set);
+ let sha;
+ let date;
+ try {
+ sha = git(["rev-parse", "--verify", "--quiet", `${ref}^{commit}`]);
+ date = git(["show", "-s", "--format=%cs", sha]);
+ } catch {
+ throw new CliError(`${ref} is not a commit in this clone.`, {
+ code: 2,
+ hint: 'control-openhands map baseline --set "$TARGET" (a commit on main; fetch origin main first)',
+ });
+ }
+ const mainRef = ["origin/main", "main"].find(
+ (r) =>
+ spawnSync("git", ["rev-parse", "--verify", "--quiet", r], {
+ cwd: repoRoot,
+ }).status === 0,
+ );
+ let warning;
+ if (mainRef) {
+ const onMain =
+ spawnSync("git", ["merge-base", "--is-ancestor", sha, mainRef], {
+ cwd: repoRoot,
+ }).status === 0;
+ if (!onMain && !flags.force)
+ throw new CliError(`${sha.slice(0, 12)} is not on ${mainRef}.`, {
+ code: 2,
+ hint: 'The line records main@<sha>: pass the frozen TARGET (control-openhands map baseline --set "$TARGET"), not the pass branch\'s HEAD. --force records it anyway.',
+ });
+ } else {
+ warning =
+ "Neither origin/main nor main is in this clone, so the commit could not be checked against main.";
+ }
+ const path = join(mapDir, "README.md");
+ const index = indexText();
+ assertOneBaselineLine(index);
+ const before = parseBaseline(index);
+ let text;
+ try {
+ text = withBaseline(index, sha, date);
+ } catch (error) {
+ throw new CliError(error.message, {
+ code: 2,
+ hint: "Add the line after the index's intro paragraph: `Maintenance baseline: main@<sha> (<date>). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.`",
+ });
+ }
+ writeFileSync(path, text);
+ out({
+ ok: true,
+ baseline: { sha, date },
+ previous: before,
+ file: relative(repoRoot, path),
+ ...(warning ? { warning } : {}),
+ note: "Proposed in this checkout only: the baseline moves when the PR that carries this line merges.",
+ });
+ return;
+ }
+ const info = baselineInfo();
+ out({
+ ok: true,
+ baseline: info,
+ ...(info
+ ? {}
+ : {
+ hint: "The map index has no `Maintenance baseline:` line; map affected then needs --base.",
+ }),
+ });
+ return;
+ }
+ if (sub === "testids") {
+ const result = resolveTestids(
+ citedTestids(mapDir, featureFiles()),
+ sourceLiterals(join(repoRoot, "src")),
+ );
+ const ok = !flags.strict || result.unresolved.length === 0;
+ out({
+ ok,
+ ...result,
+ note:
+ result.unresolved.length === 0
+ ? "every cited test id is accounted for in src/"
+ : "unresolved ids are candidates for drift (a rename), or built in a way this check misses; confirm with a live drive before editing the map",
+ });
+ if (!ok) process.exitCode = 1;
+ return;
+ }
usage(
- "Usage: control-openhands map check|coverage|ids|routes",
+ "Usage: control-openhands map check|coverage|ids|routes|affected|testids|baseline",
"control-openhands map coverage",
);
}
@@ -3320,7 +3772,7 @@ Lifecycle
Arrange (preconditions, never UI proof)
api Call this run's API with its session key (writes need --write)
llm show | set | preset deepseek | check β€” configure LLM profiles from an env key
- fixture git-repo | folder | image | file β€” disposable fixtures in the run
+ fixture git-repo | git-remote | folder | image | file | tarball | skill | mcp-server β€” disposable fixtures in the run
Essential pathways (driven through the real UI)
login Public mode: enter the session key on the API-key screen
@@ -3333,7 +3785,7 @@ Drive and observe
Evidence and map
evidence add | list | report β€” the run's pass/fail/blocked/not-run ledger
- map check | coverage | ids | routes β€” keep the feature map honest
+ map check | coverage | ids | routes | affected | testids | baseline β€” keep the feature map honest
Run state lives in $OH_VERIFY_HOME (default: $TMPDIR/openhands-verify); the
current run is the 'current' symlink there, or --run / $OH_VERIFY_RUN.
@@ -3512,6 +3964,8 @@ Verbs
testids [<sel>] [--hidden] [--filter part] discover on-screen data-testid handles (--hidden adds hidden/off-screen)
screenshot [<sel>] --feature ID --name N [--full-page] PNG under evidence/<ID>/
viewport desktop|phone|narrow|tablet|WxH 1440x1000, 390x844, 320x700, 820x1180
+ clock --offset-ms N | --system ISO|+MS | --fixed ISO|+MS skew the page's clock (install before the goto
+ whose page should see it; the server's clock is untouched)
errors [--clear] [--all] [--app-only] [--no-warnings] page/console/HTTP errors since last clear
(--clear prints the list, then empties it: run it before the action)
events [--kinds pageerror,dialog,download] [--last N]
@@ -3539,13 +3993,35 @@ Failures return {ok:false,error,hint,failureScreenshot} and exit 1.
[--entry "UI path"] [--expected TEXT] [--actual TEXT] [--artifact path[,path]] [--note TEXT]
control-openhands evidence retract --feature ID [--entry "UI path"] [--note why] drop a wrong row from the report
control-openhands evidence list [--feature F05]
-control-openhands evidence report > report.md
+control-openhands evidence report [--baseline RUN_DIR|ledger.jsonl] [--json] > report.md
+
+'report' renders the latest row per feature and entry point: a family count
+table, then every check with fail and blocked rows first and the --note column
+(where blocked rows name their missing prerequisite). --baseline compares with
+another run's ledger and lists newly failing, newly passing and unchecked rows
+first (a daily pass against yesterday's run). --json prints the same data.
`,
map: `control-openhands map check [--file Fnn-name.md] lint the map (or one entry, skipping index links): four H2s, unique IDs, links, known commands, index counts
control-openhands map check --fix-counts rewrite the index's sub-feature counts and total from the files, then lint
control-openhands map coverage routes in src/routes.ts and src/components/features/* dirs not yet mapped
control-openhands map ids every sub-feature ID with its file
control-openhands map routes the route registry as path β†’ route module
+control-openhands map affected [--base REF] [--target REF] | --paths FILE|-
+ changed paths mapped to the families whose Source: lines own them,
+ shared code to widen, src/ paths no family owns (map gaps) and
+ non-user-facing paths; without --base
+ the range starts at the index's Maintenance baseline line
+control-openhands map baseline [--set TARGET [--force]]
+ the recorded baseline (sha, date, whether HEAD descends from it and
+ by how many commits); --set moves the line to TARGET's full SHA and
+ committer date in this checkout, for the pass's PR. TARGET must be a
+ commit on origin/main (the line reads main@<sha>); a pass branch's
+ HEAD is refused unless --force
+control-openhands map testids [--strict]
+ test ids the map drives that no literal or prefix in src/ accounts for
+ (a cheap drift check before launching; --strict exits 1 on any)
+
+check also verifies every Source: path exists and the baseline line's shape.
`,
};
diff --git a/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs b/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs
index 4c583219b0..0bf02f7f91 100755
--- a/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs
+++ b/.agents/skills/verify-openhands/scripts/control-openhands.test.mjs
@@ -25,7 +25,16 @@ import {
runPorts,
} from "./lib/launcher-env.mjs";
import { redactStorage, redactStorageValue } from "./lib/redact-storage.mjs";
+import { baselineLines, parseBaseline, withBaseline } from "./lib/baseline.mjs";
+import {
+ affectedFamilies,
+ familyHead,
+ parseSourceLine,
+ specMatches,
+} from "./lib/map-sources.mjs";
import { routePattern } from "./lib/route-pattern.mjs";
+import { resolveTestids } from "./lib/testids.mjs";
+import { tmuxPathFor } from "./lib/tmux-path.mjs";
import { buildLocator, parseRole, toCss } from "./lib/selectors.mjs";
const here = dirname(fileURLToPath(import.meta.url));
@@ -619,3 +628,443 @@ test("restart refuses, before stopping anything, when a newly served editor port
await new Promise((done) => listener.close(done));
}
});
+
+test("Source: lines parse into path specs, parenthesized names relative to their directory", () => {
+ const specs = parseSourceLine(
+ "Source: `src/routes/secrets-settings.tsx`, `src/components/features/automations/` (`automation-card.tsx`, `dashboard/`), `src/api/agent-server-adapter.ts` (`buildRouterAtStartSystemSuffix`), `src/components/features/conversation/conversation-overview-*.tsx`, `node_modules/@openhands/extensions/automations/interface.json`.",
+ );
+ assert.deepEqual(
+ specs.map((s) => s.path),
+ [
+ "src/routes/secrets-settings.tsx",
+ "src/components/features/automations/",
+ "src/components/features/automations/automation-card.tsx",
+ "src/components/features/automations/dashboard/",
+ "src/api/agent-server-adapter.ts",
+ "src/components/features/conversation/conversation-overview-*.tsx",
+ "node_modules/@openhands/extensions/automations/interface.json",
+ ],
+ );
+ const glob = specs.find((s) => s.glob);
+ assert.ok(
+ specMatches(
+ glob,
+ "src/components/features/conversation/conversation-overview-tiles.tsx",
+ ),
+ );
+ assert.ok(
+ !specMatches(
+ glob,
+ "src/components/features/conversation/sub/conversation-overview-x.tsx",
+ ),
+ );
+ const dir = specs.find(
+ (s) => s.path === "src/components/features/automations/",
+ );
+ assert.ok(
+ specMatches(dir, "src/components/features/automations/kebab-menu.tsx"),
+ );
+ assert.ok(
+ !specMatches(dir, "src/components/features/automations-extra/x.tsx"),
+ );
+});
+
+test("only the Source: line in a file's head counts", () => {
+ const head = familyHead(
+ "# F14\n\nText.\n\nSource: `src/api/secrets-service.ts`.\n\n## Sub-features\n\n- `F14.list`: x\n\nSource: `not/in/head.ts`\n",
+ );
+ assert.deepEqual(
+ head.sources.map((s) => s.path),
+ ["src/api/secrets-service.ts"],
+ );
+});
+
+test("changed paths map to the families whose Source: lines own them", () => {
+ const families = [
+ {
+ id: "F14",
+ file: "F14-secrets.md",
+ sources: parseSourceLine(
+ "Source: `src/routes/secrets-settings.tsx`, `src/components/features/settings/secrets-settings/`, `src/api/secrets-service.ts`.",
+ ),
+ },
+ {
+ id: "F21",
+ file: "F21-automations-dashboard.md",
+ sources: parseSourceLine(
+ "Source: `src/components/features/automations/` (`automation-card.tsx`, `dashboard/`).",
+ ),
+ },
+ ];
+ const result = affectedFamilies(families, [
+ "src/components/features/settings/secrets-settings/secret-form.tsx",
+ "src/components/features/automations/dashboard/overview.tsx",
+ "src/components/features/automations/dashboard/overview.test.tsx",
+ "__tests__/components/secrets.test.tsx",
+ "src/hooks/query/use-get-secrets.ts",
+ "src/i18n/translation.json",
+ "src/components/features/brand-new/page.tsx",
+ "src/mocks/conversation-handlers.ts",
+ ".agents/skills/verify-openhands/SKILL.md",
+ ]);
+ assert.deepEqual(
+ result.families.map((f) => [f.id, f.paths]),
+ [
+ [
+ "F14",
+ ["src/components/features/settings/secrets-settings/secret-form.tsx"],
+ ],
+ ["F21", ["src/components/features/automations/dashboard/overview.tsx"]],
+ ],
+ );
+ // Shared code is widened by the caller, never silently dropped.
+ assert.deepEqual(result.shared, [
+ "src/hooks/query/use-get-secrets.ts",
+ "src/i18n/translation.json",
+ ]);
+ // A src/ path no family owns is a map gap.
+ assert.deepEqual(result.unmapped, [
+ "src/components/features/brand-new/page.tsx",
+ ]);
+ // Tests, mocks and this skill change no user-facing behavior, even inside
+ // a family's directory.
+ assert.deepEqual(result.nonUserFacing, [
+ "src/components/features/automations/dashboard/overview.test.tsx",
+ "__tests__/components/secrets.test.tsx",
+ "src/mocks/conversation-handlers.ts",
+ ".agents/skills/verify-openhands/SKILL.md",
+ ]);
+});
+
+test("map affected reads a path list and classifies its paths", () => {
+ const res = spawnSync(
+ process.execPath,
+ [cli, "map", "affected", "--paths", "-"],
+ {
+ encoding: "utf8",
+ input:
+ "src/components/features/settings/secrets-settings/secret-form.tsx\ndocs/README.md\n",
+ },
+ );
+ const json = JSON.parse(res.stdout);
+ assert.equal(res.status, 0, res.stdout);
+ assert.deepEqual(
+ json.families.map((f) => f.id),
+ ["F14"],
+ );
+ assert.deepEqual(json.nonUserFacing, ["docs/README.md"]);
+ // What `map affected` does with no flags depends on the index's baseline
+ // line; the baseline test below covers both states.
+});
+
+test("map check rejects a Source: path that is gone, and the shipped map is clean", () => {
+ const dir = mkdtempSync(join(tmpdir(), "mapcheck-"));
+ // The checker reads the real map dir; prove the rules on the parser level
+ // and on the live map, which must be clean.
+ const clean = spawnSync(process.execPath, [cli, "map", "check"], {
+ encoding: "utf8",
+ });
+ assert.equal(clean.status, 0, clean.stdout);
+ assert.deepEqual(JSON.parse(clean.stdout).problems, []);
+ const head = familyHead(
+ `# F99\n\nText.\n\nSource: \`src/does-not-exist.tsx\`.\n\n## Sub-features\n`,
+ );
+ assert.equal(head.sources[0].path, "src/does-not-exist.tsx");
+ assert.ok(existsSync(dir));
+});
+
+test("cited test ids resolve through literals and dynamic prefixes", () => {
+ const cited = new Map([
+ ["add-secret-button", new Set(["F14-secrets.md"])],
+ ["api-key-entry-name", new Set(["F01-first-run-and-sign-in.md"])],
+ ["plugin-card-city-weather", new Set(["F19-plugins.md"])],
+ ["renamed-away-button", new Set(["F14-secrets.md"])],
+ ]);
+ const literals = new Set([
+ "add-secret-button",
+ "api-key-entry",
+ "plugin-card",
+ ]);
+ const result = resolveTestids(cited, literals);
+ assert.equal(result.cited, 4);
+ assert.deepEqual(result.unresolved, [
+ { id: "renamed-away-button", files: ["F14-secrets.md"] },
+ ]);
+ // The live map against the live source tree: zero unresolved today, and
+ // --strict makes any drift a failing exit.
+ const res = spawnSync(process.execPath, [cli, "map", "testids", "--strict"], {
+ encoding: "utf8",
+ });
+ const out = JSON.parse(res.stdout);
+ assert.equal(res.status, 0, res.stdout);
+ assert.deepEqual(out.unresolved, []);
+ assert.ok(out.cited > 900);
+});
+
+test("evidence report shows notes, family counts, and changes against a baseline", () => {
+ const dir = mkdtempSync(join(tmpdir(), "cov-run-"));
+ mkdirSync(join(dir, "private"));
+ mkdirSync(join(dir, "evidence"));
+ writeFileSync(
+ join(dir, "run.json"),
+ JSON.stringify({
+ runId: "r2",
+ revision: "abc",
+ mode: "local",
+ baseUrl: "http://127.0.0.1:9",
+ ports: { ingress: 9 },
+ launcherPgid: 0,
+ versions: { agentServer: "1.53.0" },
+ }),
+ );
+ const rows = [
+ {
+ feature: "F14.create",
+ entry: "Settings > Secrets > Add",
+ result: "pass",
+ artifacts: [],
+ },
+ {
+ feature: "F14.agent-access",
+ entry: "conversation start",
+ result: "blocked",
+ note: "no DEEPSEEK_API_KEY",
+ artifacts: [],
+ },
+ {
+ feature: "F09.index-redirect",
+ entry: "gear",
+ result: "fail",
+ artifacts: [],
+ },
+ ];
+ writeFileSync(
+ join(dir, "evidence", "ledger.jsonl"),
+ rows
+ .map((r) => JSON.stringify({ ts: "2026-10-06T00:00:00Z", ...r }))
+ .join("\n") + "\n",
+ );
+ const baseline = join(dir, "baseline.jsonl");
+ writeFileSync(
+ baseline,
+ [
+ {
+ feature: "F14.create",
+ entry: "Settings > Secrets > Add",
+ result: "fail",
+ artifacts: [],
+ },
+ {
+ feature: "F09.index-redirect",
+ entry: "gear",
+ result: "pass",
+ artifacts: [],
+ },
+ {
+ feature: "F14.phone",
+ entry: "Settings > Secrets",
+ result: "pass",
+ artifacts: [],
+ },
+ ]
+ .map((r) => JSON.stringify({ ts: "2026-10-05T00:00:00Z", ...r }))
+ .join("\n") + "\n",
+ );
+ const report = run(["evidence", "report", "--baseline", baseline], {
+ OH_VERIFY_RUN: dir,
+ });
+ assert.equal(report.status, 0, report.stdout + report.stderr);
+ // Blocked rows carry their prerequisite; fail and blocked rows come first.
+ assert.match(report.stdout, /\| blocked \| no DEEPSEEK_API_KEY \|/);
+ assert.ok(
+ report.stdout.indexOf("F09.index-redirect") <
+ report.stdout.indexOf("F14.create"),
+ );
+ assert.match(report.stdout, /\| F14 \| 1 \| 0 \| 1 \| 0 \|/);
+ assert.match(
+ report.stdout,
+ /Newly failing:\n\n- F09.index-redirect \(gear\): pass β†’ fail/,
+ );
+ assert.match(
+ report.stdout,
+ /Newly passing:\n\n- F14.create \(Settings > Secrets > Add\): fail β†’ pass/,
+ );
+ assert.match(report.stdout, /Not checked this run:\n\n- F14.phone/);
+ const json = run(["evidence", "report", "--json", "--baseline", baseline], {
+ OH_VERIFY_RUN: dir,
+ });
+ assert.equal(json.json.counts.blocked, 1);
+ assert.deepEqual(
+ json.json.changes.newlyFailing.map((c) => c.feature),
+ ["F09.index-redirect"],
+ );
+ assert.deepEqual(
+ json.json.changes.missing.map((c) => c.feature),
+ ["F14.phone"],
+ );
+ assert.equal(
+ run(["evidence", "report", "--baseline", join(dir, "nope.jsonl")], {
+ OH_VERIFY_RUN: dir,
+ }).status,
+ 2,
+ );
+});
+
+test("the tmux directory of a run comes from its whole path", () => {
+ // Two runs whose ids end alike must not share a tmux server (the earlier
+ // scheme used the last six characters of the run id).
+ assert.notEqual(
+ tmuxPathFor("/tmp/openhands-verify/qa-run-a1"),
+ tmuxPathFor("/tmp/openhands-verify/qb-run-a1"),
+ );
+ assert.equal(tmuxPathFor("/tmp/x/run-1"), tmuxPathFor("/tmp/x/run-1"));
+ assert.match(tmuxPathFor("/tmp/x/run-1"), /^\/tmp\/ohv-tmux-[0-9a-f]{10}$/);
+});
+
+test("help names every fixture kind and the new map verbs", () => {
+ const { stdout } = run(["--help"]);
+ for (const word of [
+ "git-remote",
+ "tarball",
+ "skill",
+ "mcp-server",
+ "affected",
+ "testids",
+ ])
+ assert.match(stdout, new RegExp(word));
+ const map = spawnSync(process.execPath, [cli, "help", "map"], {
+ encoding: "utf8",
+ });
+ assert.match(map.stdout, /map affected \[--base REF\]/);
+ assert.match(map.stdout, /map testids \[--strict\]/);
+ const evidence = spawnSync(process.execPath, [cli, "help", "evidence"], {
+ encoding: "utf8",
+ });
+ assert.match(evidence.stdout, /--baseline/);
+});
+
+test("the maintenance baseline line is read and moved in place", () => {
+ const index =
+ "# Map\n\nIntro.\n\nMaintenance baseline: main@ed815e141409c7b52991ab8d13c553b595da9165 (2026-10-06). The next maintenance pass starts from this commit; a pass proposes the next baseline in its PR, and merging that PR accepts it.\n\n## Baseline preconditions\n";
+ assert.deepEqual(parseBaseline(index), {
+ sha: "ed815e141409c7b52991ab8d13c553b595da9165",
+ date: "2026-10-06",
+ });
+ assert.equal(parseBaseline("# Map\n\nno line here\n"), null);
+ const sha = "7cd6496040dd74d75e93feecaea528c6c50efef7";
+ const moved = withBaseline(index, sha, "2026-10-07");
+ assert.match(
+ moved,
+ /^Maintenance baseline: main@7cd6496040dd74d75e93feecaea528c6c50efef7 \(2026-10-07\)\. The next maintenance pass starts from this commit/m,
+ );
+ // Only the SHA and date move; the prose and the rest of the index stay.
+ assert.equal(
+ moved.replace(sha, "X").replace("2026-10-07", "D"),
+ index.replace(/ed815e1[0-9a-f]+/, "X").replace("2026-10-06", "D"),
+ );
+ assert.throws(
+ () => withBaseline(index, "7cd6496", "2026-10-07"),
+ /full 40-hex SHA/,
+ );
+ assert.throws(
+ () => withBaseline("# Map\n", sha, "2026-10-07"),
+ /no `Maintenance baseline/,
+ );
+ // A merge that kept both sides' lines: nothing moves until one is removed.
+ const twice = `${index}\nMaintenance baseline: main@${sha} (2026-10-07). Other.\n`;
+ assert.equal(baselineLines(twice).length, 2);
+ assert.throws(
+ () => withBaseline(twice, sha, "2026-10-08"),
+ /more than one `Maintenance baseline:` line/,
+ );
+});
+
+test("map baseline reads the index line, and map affected starts from it by default", () => {
+ const baseline = spawnSync(process.execPath, [cli, "map", "baseline"], {
+ encoding: "utf8",
+ });
+ assert.equal(baseline.status, 0, baseline.stdout);
+ const { baseline: recorded, hint } = JSON.parse(baseline.stdout);
+ const affected = spawnSync(process.execPath, [cli, "map", "affected"], {
+ encoding: "utf8",
+ });
+ const json = JSON.parse(affected.stdout);
+ if (recorded === null) {
+ // Until the index carries the line (OpenHands/OpenHands#18085), the
+ // default range cannot be computed and the usage error says so.
+ assert.match(hint, /no `Maintenance baseline:` line/);
+ assert.equal(affected.status, 2, affected.stdout);
+ assert.match(json.error, /Maintenance baseline/);
+ } else {
+ assert.match(recorded.sha, /^[0-9a-f]{40}$/);
+ assert.match(recorded.date, /^\d{4}-\d{2}-\d{2}$/);
+ if (recorded.known) {
+ assert.equal(affected.status, 0, affected.stdout);
+ assert.equal(json.range.base, recorded.sha);
+ assert.equal(json.range.baseSource, "map index baseline");
+ } else {
+ // A shallow clone (CI checks out at depth 1) does not hold the
+ // baseline commit: an environment error with the deepening hint, not
+ // a stack trace.
+ assert.equal(affected.status, 3, affected.stdout);
+ assert.match(json.hint, /shallow clone/);
+ }
+ }
+ // An explicit --base never consults the line. HEAD..HEAD is the one range
+ // every clone can resolve, shallow ones included.
+ const explicit = JSON.parse(
+ spawnSync(
+ process.execPath,
+ [cli, "map", "affected", "--base", "HEAD", "--target", "HEAD"],
+ {
+ encoding: "utf8",
+ },
+ ).stdout,
+ );
+ assert.equal(explicit.range.baseSource, "--base");
+ assert.equal(explicit.range.target, "HEAD");
+ assert.equal(explicit.changed, 0);
+ // --set refuses a ref that is not a commit, and a bare --set, before
+ // touching the index.
+ const bad = spawnSync(
+ process.execPath,
+ [cli, "map", "baseline", "--set", "not-a-ref-xyz"],
+ {
+ encoding: "utf8",
+ },
+ );
+ assert.equal(bad.status, 2);
+ const bare = spawnSync(process.execPath, [cli, "map", "baseline", "--set"], {
+ encoding: "utf8",
+ });
+ assert.equal(bare.status, 2);
+ assert.match(JSON.parse(bare.stdout).hint, /\$TARGET/);
+ // The line reads main@<sha>: a commit that is not on main is refused when
+ // main is in the clone (a PR branch's HEAD, here), so a pass cannot record
+ // its own branch tip by mistake.
+ const repo = resolve(here, "../../../..");
+ const mainRef = ["origin/main", "main"].find(
+ (r) =>
+ spawnSync("git", ["rev-parse", "--verify", "--quiet", r], {
+ cwd: repo,
+ }).status === 0,
+ );
+ const headOnMain =
+ mainRef &&
+ spawnSync("git", ["merge-base", "--is-ancestor", "HEAD", mainRef], {
+ cwd: repo,
+ }).status === 0;
+ if (mainRef && !headOnMain) {
+ const branchTip = spawnSync(
+ process.execPath,
+ [cli, "map", "baseline", "--set", "HEAD"],
+ { encoding: "utf8" },
+ );
+ assert.equal(branchTip.status, 2, branchTip.stdout);
+ assert.match(JSON.parse(branchTip.stdout).error, /is not on/);
+ }
+ const help = spawnSync(process.execPath, [cli, "help", "map"], {
+ encoding: "utf8",
+ });
+ assert.match(help.stdout, /map baseline \[--set TARGET \[--force\]\]/);
+});
diff --git a/.agents/skills/verify-openhands/scripts/lib/baseline.mjs b/.agents/skills/verify-openhands/scripts/lib/baseline.mjs
new file mode 100644
index 0000000000..8deccd949f
--- /dev/null
+++ b/.agents/skills/verify-openhands/scripts/lib/baseline.mjs
@@ -0,0 +1,40 @@
+// The maintenance baseline recorded in the map index
+// (references/feature-map/README.md), on a line of the form
+// Maintenance baseline: main@<sha> (<YYYY-MM-DD>). <prose>
+// A pass reads it as BASE; a pass that changes the map rewrites the SHA and
+// date to its TARGET in its PR, and merging the PR accepts the new baseline.
+const LINE =
+ /^Maintenance baseline: main@([0-9a-f]{7,40}) \((\d{4}-\d{2}-\d{2})\)(.*)$/m;
+
+/** Every line that starts like the baseline line, well-formed or not. */
+export function baselineLines(indexText) {
+ return indexText.match(/^Maintenance baseline:.*$/gm) ?? [];
+}
+
+/** @returns {{ sha: string, date: string } | null} */
+export function parseBaseline(indexText) {
+ const m = LINE.exec(indexText);
+ return m ? { sha: m[1], date: m[2] } : null;
+}
+
+/** The index text with the baseline line moved to `sha` and `date`. */
+export function withBaseline(indexText, sha, date) {
+ if (!/^[0-9a-f]{40}$/.test(sha))
+ throw new Error(`baseline sha must be a full 40-hex SHA, got ${sha}`);
+ if (!/^\d{4}-\d{2}-\d{2}$/.test(date))
+ throw new Error(`baseline date must be YYYY-MM-DD, got ${date}`);
+ if (!LINE.test(indexText))
+ throw new Error(
+ "the map index has no `Maintenance baseline: main@<sha> (<date>)` line",
+ );
+ // A merge can leave two lines; moving one of them would hide the other.
+ if (baselineLines(indexText).length > 1)
+ throw new Error(
+ "the map index has more than one `Maintenance baseline:` line; keep one",
+ );
+ return indexText.replace(
+ LINE,
+ (_, _sha, _date, rest) =>
+ `Maintenance baseline: main@${sha} (${date})${rest}`,
+ );
+}
diff --git a/.agents/skills/verify-openhands/scripts/lib/map-sources.mjs b/.agents/skills/verify-openhands/scripts/lib/map-sources.mjs
new file mode 100644
index 0000000000..fb40886840
--- /dev/null
+++ b/.agents/skills/verify-openhands/scripts/lib/map-sources.mjs
@@ -0,0 +1,165 @@
+// The feature map's `Source:` lines, read for drift checks and for mapping a
+// set of changed paths to the families that own them.
+//
+// A `Source:` line lists the main implementation paths of a family in
+// backticks. Two shapes carry paths:
+// `src/routes/secrets-settings.tsx`, `src/components/features/settings/secrets-settings/`
+// `src/components/features/automations/` (`automation-card.tsx`, `dashboard/`)
+// In the second shape the parenthesized names are relative to the directory
+// before them. A backticked token with no `/` and no file extension (a
+// function name such as `buildRouterAtStartSystemSuffix`) is a symbol, not a
+// path, and is ignored. A trailing `*` in a file name matches by prefix.
+
+const FILE_EXT = /\.(tsx?|m?[jc]s|json|css|md|ya?ml|sh|html|toml|py|txt)$/;
+
+function looksLikePath(token) {
+ return token.includes("/") || FILE_EXT.test(token);
+}
+
+/**
+ * Parse one `Source:` line into path specs.
+ * @returns {Array<{ path: string, dir: boolean, glob: boolean }>}
+ */
+export function parseSourceLine(line) {
+ const specs = [];
+ const body = line.replace(/^Source:\s*/, "");
+ // Tokens: a backticked entry, optionally followed by a parenthesized group
+ // of backticked relative entries.
+ const entry = /`([^`]+)`(\s*\(([^)]*)\))?/g;
+ for (const m of body.matchAll(entry)) {
+ const main = m[1].trim();
+ if (!looksLikePath(main)) continue;
+ specs.push(spec(main));
+ if (m[3] && main.endsWith("/")) {
+ for (const rel of m[3].matchAll(/`([^`]+)`/g)) {
+ const name = rel[1].trim();
+ if (!looksLikePath(name)) continue;
+ specs.push(spec(main + name));
+ }
+ }
+ }
+ return specs;
+}
+
+function spec(path) {
+ return {
+ path: path.replace(/\*$/, "").replace(/\*(?=\.)/, "*"),
+ dir: path.endsWith("/"),
+ glob: path.includes("*"),
+ };
+}
+
+/** True when `changed` (a repo-relative file path) belongs to `s`. */
+export function specMatches(s, changed) {
+ if (s.dir) return changed.startsWith(s.path);
+ if (s.glob) {
+ const [head, tail = ""] = s.path.split("*");
+ return (
+ changed.startsWith(head) &&
+ changed.endsWith(tail) &&
+ !changed.slice(head.length).includes("/")
+ );
+ }
+ return changed === s.path;
+}
+
+/** The `Source:` line of a feature file's head (before the first H2). */
+export function familyHead(text) {
+ const head = text.split(/^## /m)[0];
+ const source = head.match(/^Source:.*$/m)?.[0] ?? "";
+ return { sources: source ? parseSourceLine(source) : [] };
+}
+
+// Paths outside the frontend build and the launcher are not user-facing on
+// their own; a change there needs no live recipe (tests, CI, this skill).
+const NON_USER_FACING = [
+ /^__tests__\//,
+ /^__mocks__\//,
+ /^\.github\//,
+ /^\.agents\//,
+ /^\.husky\//,
+ /^\.pr\//,
+ /^\.screenshots\//,
+ /^tests\//,
+ /^specs\//,
+ /^docs\//,
+ // The MSW mock API and dev-only helpers never reach a launched stack.
+ /^src\/mocks\//,
+ /^src\/dev\//,
+ // Build and i18n tooling under scripts/; the launchers there are F26's.
+ /^scripts\/[^/]+\.cjs$/,
+ /^scripts\/(stryker-diff|generate-icons|brand-dev-electron)\.mjs$/,
+ /\.test\.[cm]?[jt]sx?$/,
+ /^(README|CHANGELOG|AGENTS|MAINTAINERS)[^/]*\.md$/,
+ /^\.[^/]+$/,
+];
+
+// Changes here reach every page through the build, so no single family owns
+// them; the caller widens to the consumers rather than sampling one screen.
+const SHARED_PREFIXES = [
+ "src/components/shared/",
+ "src/components/ui/",
+ "src/ui/",
+ "src/styles/",
+ "src/index.css",
+ "src/tailwind.css",
+ "src/themes/",
+ "src/i18n/",
+ "src/api/",
+ "src/hooks/",
+ "src/stores/",
+ "src/utils/",
+ "src/types/",
+ "src/context/",
+ "src/contexts/",
+ "src/services/",
+ "src/lib/",
+ "src/root.tsx",
+ "src/routes.ts",
+ "package.json",
+ "package-lock.json",
+ "vite.config.ts",
+ "react-router.config.ts",
+ "tailwind.config.js",
+ "tsconfig.json",
+ "config/",
+];
+
+/**
+ * Map changed paths to families.
+ * @param {Array<{ id: string, file: string, sources: object[] }>} families
+ * @param {string[]} changed repo-relative paths
+ */
+export function affectedFamilies(families, changed) {
+ const byFamily = new Map();
+ const unmapped = [];
+ const shared = [];
+ const nonUserFacing = [];
+ for (const path of changed) {
+ if (NON_USER_FACING.some((re) => re.test(path))) {
+ nonUserFacing.push(path);
+ continue;
+ }
+ const owners = families.filter((f) =>
+ f.sources.some((s) => specMatches(s, path)),
+ );
+ if (owners.length) {
+ for (const f of owners) {
+ if (!byFamily.has(f.id))
+ byFamily.set(f.id, { file: f.file, paths: [] });
+ byFamily.get(f.id).paths.push(path);
+ }
+ continue;
+ }
+ if (SHARED_PREFIXES.some((p) => path.startsWith(p))) shared.push(path);
+ else unmapped.push(path);
+ }
+ return {
+ families: [...byFamily.entries()]
+ .sort(([a], [b]) => a.localeCompare(b))
+ .map(([id, v]) => ({ id, ...v })),
+ shared,
+ unmapped,
+ nonUserFacing,
+ };
+}
diff --git a/.agents/skills/verify-openhands/scripts/lib/testids.mjs b/.agents/skills/verify-openhands/scripts/lib/testids.mjs
new file mode 100644
index 0000000000..367bbb17ac
--- /dev/null
+++ b/.agents/skills/verify-openhands/scripts/lib/testids.mjs
@@ -0,0 +1,77 @@
+// Static check of the test ids the feature map drives against the source
+// tree, so a renamed `data-testid` is noticed before any stack is launched.
+//
+// A cited id resolves when it appears as a string literal in src/, or when a
+// prefix of it (cut at a hyphen) does: Canvas builds many ids from a prefix
+// prop or a template (`${testIdPrefix}-name`, `plugin-card-${id}`), so the
+// literal before the dynamic part is what the source holds. Anything left is
+// reported for a human or the live pass to judge; it is not proof of drift
+// on its own (the id may be assembled in a way this heuristic misses).
+import { readFileSync, readdirSync } from "node:fs";
+import { join } from "node:path";
+
+/** Every `testid=...` handle cited in the map's feature files, with its families. */
+export function citedTestids(mapDir, files) {
+ const cited = new Map();
+ for (const file of files) {
+ const text = readFileSync(join(mapDir, file), "utf8");
+ for (const m of text.matchAll(/testid=([A-Za-z0-9_./:$-]+)/g)) {
+ const id = m[1].replace(/[.,;:)]+$/, "");
+ if (!id) continue;
+ if (!cited.has(id)) cited.set(id, new Set());
+ cited.get(id).add(file);
+ }
+ }
+ return cited;
+}
+
+function sourceFiles(dir, out = []) {
+ for (const entry of readdirSync(dir, { withFileTypes: true })) {
+ const path = join(dir, entry.name);
+ if (entry.isDirectory()) {
+ if (entry.name !== "node_modules") sourceFiles(path, out);
+ } else if (
+ /\.(tsx?|m?js)$/.test(entry.name) &&
+ !/\.test\.[mc]?[jt]sx?$/.test(entry.name)
+ ) {
+ out.push(path);
+ }
+ }
+ return out;
+}
+
+/** String literals and template heads in the source tree that can be test ids. */
+export function sourceLiterals(srcDir) {
+ const literals = new Set();
+ for (const file of sourceFiles(srcDir)) {
+ const text = readFileSync(file, "utf8");
+ for (const m of text.matchAll(/["'`]([a-z][a-z0-9_.:/-]*[a-z0-9])["'`$]/g))
+ literals.add(m[1]);
+ // `prefix-${dynamic}`: the head before the first placeholder.
+ for (const m of text.matchAll(/`([a-z][a-z0-9_.:/-]*)-\$\{/g))
+ literals.add(m[1]);
+ }
+ return literals;
+}
+
+/**
+ * Which cited ids the source tree does not account for.
+ * @returns {{ cited: number, unresolved: Array<{ id: string, files: string[] }> }}
+ */
+export function resolveTestids(cited, literals) {
+ const unresolved = [];
+ for (const [id, files] of cited) {
+ if (literals.has(id)) continue;
+ const parts = id.split("-");
+ let found = false;
+ for (let i = parts.length - 1; i >= 1 && !found; i -= 1) {
+ const prefix = parts.slice(0, i).join("-");
+ if (prefix.length >= 4 && literals.has(prefix)) found = true;
+ }
+ if (!found) unresolved.push({ id, files: [...files].sort() });
+ }
+ return {
+ cited: cited.size,
+ unresolved: unresolved.sort((a, b) => a.id.localeCompare(b.id)),
+ };
+}
diff --git a/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs b/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs
new file mode 100644
index 0000000000..b10f6b9e1d
--- /dev/null
+++ b/.agents/skills/verify-openhands/scripts/lib/tmux-path.mjs
@@ -0,0 +1,12 @@
+// Where a run's private tmux server lives (OpenHands/OpenHands#17946: the
+// launcher's <stateDir>/tmux is never created, so runs would share one
+// server). Under /tmp because a socket below the long run path would exceed
+// the 108-byte Unix socket limit; named by a hash of the whole run path so
+// two runs whose ids end alike (`qa-run-a1`, `qb-run-a1`) never share one.
+import { createHash } from "node:crypto";
+import { join, resolve } from "node:path";
+
+export function tmuxPathFor(runDir) {
+ const hash = createHash("sha256").update(resolve(runDir)).digest("hex");
+ return join("/tmp", `ohv-tmux-${hash.slice(0, 10)}`);
+}