Skip to content

πŸ™Œ OpenHands: AI-Driven Development

chore: Bump automation default to 1.10.0 (#17086)

Rohit Malhotra3acdc61cce6a
diff --git a/AGENTS.md b/AGENTS.md
index 07d8e8a8ca..72afc80a4d 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -252,7 +252,7 @@ you are running inside of β€” NOT the automation backend.
- Profile pre-flight ping: agent-server β‰₯ 1.43 sends a 1-token `ping` completion whenever an LLM profile is saved (`POST /api/profiles/{name}/validate`, 30 s budget in the canvas). The mock answers it with a canned `pong` instead of feeding it to `TestLLM`, so it neither consumes a scripted turn nor 500s-and-retries past the canvas timeout when the trajectory is exhausted; it is also left out of the `/admin/requests` history.
- **Real automation backend**: The automation test uses the production automation backend (started by `bin/agent-canvas.mjs`), NOT a mock server. Terminal `curl` commands from the agent hit the automation API through the ingress proxy at the test's `BACKEND_URL` (default `http://localhost:18300`). Auth uses the `X-Session-API-Key` header matching the stack's session key.
- **Test helpers** (`tests/e2e/mock-llm/utils/mock-llm-helpers.ts`): Exports `registerTrajectory()`, `activateTrajectory()`, `resetMockLLM()`, `ensureMockLLMProfile()`, `getMockLLMRequests()` (fetches captured completion bodies from `GET /admin/requests`), `IMAGE_REPLY_TOKEN` + `MINIMAL_PNG_BASE64` (constants for the image-upload spec), ACP helpers (`configureAcpAgent()`, `verifyAcpAgentSettings()`, `resetToOpenHandsAgent()`, `ACP_REPLY_TOKEN`, `MOCK_ACP_SERVER_PATH`), and more.
-- **Padding response for internal LLM call**: The agent-server makes an internal LLM call (condenser/skill-analysis) before the agent's main loop starts when skills are activated. This consumes one trajectory response. Automation tests prepend a throwaway `{ text: "" }` response as padding. The conversation test does NOT need this because its user message doesn't trigger skill activation.
+- **Padding response for internal LLM call**: The agent-server makes an internal LLM call (condenser/skill-analysis) before the agent's main loop starts when skills are activated. This consumes one trajectory response. Automation tests prepend a throwaway `{ text: "" }` response as padding. The conversation test does NOT need this because its user message doesn't trigger skill activation. Since `openhands-automation==1.10.0` (openhands/automation#405), the automation lifecycle spec scripts the required `finish` tool turn in its run-conversation budget β€” that release requires preset automation conversations to callthe `finish` tool before the run reaches COMPLETED; blank turns alone would loop and exhaust the trajectory. The mock server logs "Mock LLM exhausted after N calls" pinpoint the exact count if it drifts again.
- **Mock ACP server** (`tests/e2e/mock-llm/scripts/mock-acp-server.py`): A minimal stdio-based ACP agent that speaks JSON-RPC using the `acp` Python library (installed as a dependency of `openhands-sdk`). Handles `initialize`, `session/new`, and `session/prompt`; sends a scripted `session/update` notification with `ACP_REPLY_TOKEN` in a text content block, then returns `stop_reason: "end_turn"`. The agent-server spawns it as a subprocess via `acp_command`. Accepts `--reply-token TOKEN` to customize the reply token.
- **Test directory layout**: Specs are organized into feature subdirectories that mirror the source code structure, enabling selective test execution based on which source files changed:
- `settings/` β€” LLM profile management, ACP agent config, model switching (`mock-llm-acp-agent.spec.ts`, `mock-llm-profile-management.spec.ts`, `mock-llm-model-switch.spec.ts`)
diff --git a/config/defaults.json b/config/defaults.json
index 1a406a2553..e9b6f9bd10 100644
--- a/config/defaults.json
+++ b/config/defaults.json
@@ -3,7 +3,7 @@
"versions": {
"agentServer": "1.44.1",
"agentCanvas": "1.16.0",
- "automation": "1.9.1"
+ "automation": "1.10.0"
},
"compatibility": {
"minimumAgentServer": "1.28.0"
diff --git a/tests/e2e/mock-llm/automations/mock-llm-automation.spec.ts b/tests/e2e/mock-llm/automations/mock-llm-automation.spec.ts
index 7dad262a52..dcae2c93cd 100644
--- a/tests/e2e/mock-llm/automations/mock-llm-automation.spec.ts
+++ b/tests/e2e/mock-llm/automations/mock-llm-automation.spec.ts
@@ -365,11 +365,18 @@ test.describe("mock-LLM automation lifecycle", () => {
// quickly, making the failure obvious. See AGENTS.md β†’ "Padding
// response for internal LLM call" for more context.
//
- // After the main conversation finishes (responses 0-3), the dispatched
+ // After the main conversation finishes (responses 0-3),the dispatched
// automation run spawns a NEW conversation on the same agent-server.
- // That conversation also calls the mock LLM. We append extra text
- // responses (4-6) so the run's conversation can finish normally, the
- // script fires its completion callback, and the run reaches COMPLETED.
+
+ // That conversation also calls the mock LLM. Recent openhands-automation
+ // versions (>= 1.10.0, PR openhands/automation#405) require
+ // preset automation conversations to call the `finish` tool before the
+ // run reaches COMPLETED β€” so we script one blank internal call followed
+ // by the required finish-tool turn (and one trailing blank safety) below.
+ // If this ever drifts again, step 2's run-status timeout will surface it,
+ // and the mock server log shows "Mock LLM exhausted after N calls"
+ // pinpointing the exact count.
+
await registerTrajectory(request, "automation-lifecycle", [
// ── Main conversation (responses 0-3) ──
{ text: "" }, // 0: consumed by skill-activation LLM call (see above)
@@ -391,11 +398,37 @@ test.describe("mock-LLM automation lifecycle", () => {
// ── Automation run's conversation (responses 4+) ──
// The run starts a fresh conversation with the automation prompt.
- // Provide enough responses for any internal LLM calls + the agent's
- // turn so the conversation finishes and the completion callback fires.
+ // Script one internal padding call +the required `finish` tool turn,
+ // followed by a final text reply (+ one trailing safety blank** so the
+ // run reaches COMPLETED.
{ text: "" }, // 4: possible internal/condenser call
- { text: "Done. Hello world echoed successfully." }, // 5: agent reply
- { text: "" }, // 6: safety buffer for any follow-up internal call
+ {
+ // 5: openhands-automation >= 1.10.0 (openhands/automation#405)
+ // requires preset automation conversations to finish via the
+ // `finish` tool β€” a hook waits for `finish_tool_used` before the
+ // run reaches COMPLETED. Blank turns alone would just loop and exhaust
+ // the trajectory, so script the required finish-tool turn explicitly.
+
+ tool_call: {
+ // The agent-server registers the finish tool under its lowercase
+ // title (`finish`). The action schema requires `message`, while the
+ // attached response schema (TaskOutcome) validates the
+ // `status` + `outcome_summary` pair (that's the field alias, not
+ // `summary`); both layers must be satisfied in the same call.
+ name: "finish",
+ arguments: {
+ message: "Hello world echoed successfully.",
+ status: "success",
+ outcome_summary: "Hello world echoed successfully.",
+ },
+ },
+ },
+ // 6: after the finish tool executes, the agent still needs one more
+ // non-empty LLM turn to end the conversation β€” a blank here would
+ // make the harness nag ("no function call") and loop on the exhausted
+ // trajectory, so reply with the final text.
+ { text: "Done. Hello world echoed successfully." },
+ { text: "" }, // 7: safety buffer for any follow-up internal call after finish
]);
// Activate it so the mock LLM uses this trajectory for the next conversation
@@ -485,9 +518,9 @@ test.describe("mock-LLM automation lifecycle", () => {
const automation = await getAutomation(request, createdAutomationId!);
automationIds.add(automation.id);
- // Wait for the run to reach COMPLETED. The trajectory includes extra
- // responses (indices 4-6) for the automation run's spawned conversation
- // so it can finish and fire the completion callback.
+ // Wait for the run to reach COMPLETED. The trajectory scripts the required
+ // `finish` tool turn (plus padding) for the automation run's spawned
+ // conversation so it can finish and fire the completion callback.
const run = await waitForRunStatus(
request,
automation.id,