π OpenHands: AI-Driven Development
Add e2e mock-LLM test for ACP agents (#1141)
Rohit Malhotra86f8b0802abb
diff --git a/AGENTS.md b/AGENTS.mdindex 0f01b459cb..de861510b1 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -185,14 +185,16 @@ you are running inside of β NOT the automation backend.- `POST /admin/trajectory/activate` β activate a previously registered trajectory- `GET /admin/requests` β return the list of all `/v1/chat/completions` request bodies captured since the last reset (used by the image-upload test to verify the image was forwarded to the LLM)- **Real automation backend**: The automation test uses the production automation backend (started by `bin/agent-canvas.mjs`), NOT a mock server. Terminal `curl` commands from the agent hit the automation API through the ingress proxy at the test's `BACKEND_URL` (default `http://localhost:18300`). Auth uses the `X-Session-API-Key` header matching the stack's session key. The `mock-automation-server.py` file still exists as a reference but is not used by current tests.-- **Test helpers** (`tests/e2e/mock-llm/utils/mock-llm-helpers.ts`): Exports `registerTrajectory()`, `activateTrajectory()`, `resetMockLLM()`, `ensureMockLLMProfile()`, `getMockLLMRequests()` (fetches captured completion bodies from `GET /admin/requests`), `IMAGE_REPLY_TOKEN` + `MINIMAL_PNG_BASE64` (constants for the image-upload spec), and more.+- **Test helpers** (`tests/e2e/mock-llm/utils/mock-llm-helpers.ts`): Exports `registerTrajectory()`, `activateTrajectory()`, `resetMockLLM()`, `ensureMockLLMProfile()`, `getMockLLMRequests()` (fetches captured completion bodies from `GET /admin/requests`), `IMAGE_REPLY_TOKEN` + `MINIMAL_PNG_BASE64` (constants for the image-upload spec), ACP helpers (`configureAcpAgent()`, `verifyAcpAgentSettings()`, `resetToOpenHandsAgent()`, `ACP_REPLY_TOKEN`, `MOCK_ACP_SERVER_PATH`), and more.- **Padding response for internal LLM call**: The agent-server makes an internal LLM call (condenser/skill-analysis) before the agent's main loop starts when skills are activated. This consumes one trajectory response. Automation tests prepend a throwaway `{ text: "" }` response as padding. The conversation test does NOT need this because its user message doesn't trigger skill activation.+- **Mock ACP server** (`tests/e2e/mock-llm/scripts/mock-acp-server.py`): A minimal stdio-based ACP agent that speaks JSON-RPC using the `acp` Python library (installed as a dependency of `openhands-sdk`). Handles `initialize`, `session/new`, and `session/prompt`; sends a scripted `session/update` notification with `ACP_REPLY_TOKEN` in a text content block, then returns `stop_reason: "end_turn"`. The agent-server spawns it as a subprocess via `acp_command`. Accepts `--reply-token TOKEN` to customize the reply token.- **Test specs**:+ - `mock-llm-acp-agent.spec.ts` β ACP (Agent Client Protocol) agent E2E test. Configures the agent through the browser UI the same way a real user would: navigates to Settings β Agent, switches the agent type dropdown to "ACP (external subprocess)", selects the "Custom" preset, types the mock ACP server command into the command textarea, and clicks Save. Then verifies the settings persisted by reloading the page, starts a conversation from the home page, verifies the ACP agent's reply token appears in the chat UI, checks the POST `/api/conversations` payload contains `agent_kind: "acp"`, and resumes the conversation from the sidebar after navigating away. Uses `selectDropdownOption()` helper to interact with HeroUI Autocomplete dropdowns via `getByRole("combobox")` + `getByRole("option")`. Works in both npm and Docker E2E paths: the Docker Playwright config volume-mounts the mock ACP script into the container and sets `MOCK_ACP_CONTAINER_PYTHON` / `MOCK_ACP_CONTAINER_SCRIPT` env vars so the test types container-side paths into the UI instead of host-local paths. The `afterAll` cleanup resets back to `agent_kind: "openhands"` and restores the mock LLM profile so subsequent alphabetically-ordered specs are not affected.- `mock-llm-conversation.spec.ts` β Creates LLM profile via UI, runs a conversation with a terminal tool call, verifies bash execution and agent reply.- `mock-llm-image-upload.spec.ts` β Attaches a 1Γ1 PNG via the hidden file input, sends "What is in this image?", verifies the agent replies, that the user message event stores image_urls, and that the mock LLM received an image_url content block with a base64 data: URL in at least one completion call.- `mock-llm-automation.spec.ts` β Full automation lifecycle: registers a trajectory (7 responses total β 4 for the main conversation + 3 for the automation run's spawned conversation) where the LLM creates a cron automation and dispatches a run via terminal `curl` commands to the real automation backend. Verifies: automation created with correct schedule, run reaches COMPLETED status with a conversation_id, automation appears on the `/automations` list page, detail page shows COMPLETED badge (`data-testid="run-status-icon-completed"`), and clicking the run's conversation link navigates to the correct `/conversations/{id}` page.- `mock-llm-partial-stack.spec.ts` β Partial stack mode tests. Unlike other specs, these spawn their own `bin/agent-canvas.mjs` child processes instead of relying on the config's webServer entries. Three describe blocks: (1) `--frontend-only` verifies static frontend is served (200 on `/`), backend routes return 503 (`/server_info`, `/api/settings`, `/api/automation/v1`), and the browser shows the manage-backends modal; (2) `--backend-only` verifies `/server_info` returns 200, `/api/settings` is reachable, automation endpoint works, and root/asset requests return 503; (3) port conflict verifies the process exits non-zero with a clear error message when the ingress port is occupied, then starts successfully on a free port. Each test uses isolated state dirs and high port numbers (18310+ range) to avoid collisions with the main full-stack instance.-- Tests run serially (`workers: 1`, `mode: "serial"` per describe block). Files are discovered alphabetically so automation tests run before conversation tests; each spec is self-contained (automation test configures its own LLM profile via the settings API). The `afterEach` hook resets the mock LLM to its default trajectory so subsequent specs start fresh even when a preceding test fails.+- Tests run serially (`workers: 1`, `mode: "serial"` per describe block). Files are discovered alphabetically so the ACP agent test runs first, followed by auth-modes, automation, conversation, etc.; each spec is self-contained (automation test configures its own LLM profile via the settings API, ACP test resets back to OpenHands agent in afterAll). The `afterEach` hook resets the mock LLM to its default trajectory so subsequent specs start fresh even when a preceding test fails.- CI workflow: `.github/workflows/mock-llm-e2e.yml` runs on PRs with the `e2e-tests` label or on manual dispatch. It builds the frontend, starts the mock LLM server, runs the tests, and posts a PR comment with results.- The custom `DoneMarkerReporter` writes `.mock-llm-markers/.tests-done` after all tests complete (before webServer teardown) so the CI wrapper can detect completion and kill the lingering teardown process.diff --git a/playwright.mock-llm-docker.config.ts b/playwright.mock-llm-docker.config.tsindex d39c2d52a8..0af78594d1 100644--- a/playwright.mock-llm-docker.config.ts+++ b/playwright.mock-llm-docker.config.ts@@ -29,6 +29,7 @@import { defineConfig, devices } from "@playwright/test";import { randomBytes } from "node:crypto";+import { resolve } from "node:path";// ββ Docker image ββββββββββββββββββββββββββββββββββββββββββββββββββββββββconst DOCKER_IMAGE =@@ -85,6 +86,19 @@ if (!process.env.MOCK_LLM_AGENT_URL) {process.env.MOCK_LLM_AGENT_URL = MOCK_LLM_URL;}+// ββ ACP test support ββββββββββββββββββββββββββββββββββββββββββββββββββ+// The mock ACP server script lives on the host. We volume-mount it into+// the container and tell the test which container-side paths to use when+// typing the `acp_command` into the Settings UI.+const MOCK_ACP_HOST_PATH = resolve(+ "tests/e2e/mock-llm/scripts/mock-acp-server.py",+);+const MOCK_ACP_CONTAINER_PATH = "/opt/mock-acp-server.py";++// The agent-server image ships Python 3 via the openhands-sdk base.+process.env.MOCK_ACP_CONTAINER_PYTHON = "python3";+process.env.MOCK_ACP_CONTAINER_SCRIPT = MOCK_ACP_CONTAINER_PATH;+export default defineConfig({testDir: "./tests/e2e/mock-llm",testMatch: /.*\.spec\.ts/,@@ -152,6 +166,9 @@ export default defineConfig({"--rm",`--name ${CONTAINER_NAME}`,"--network host",+ // Mount the mock ACP server script so the agent-server inside+ // Docker can spawn it as an ACP subprocess.+ `-v ${MOCK_ACP_HOST_PATH}:${MOCK_ACP_CONTAINER_PATH}:ro`,`-e PORT=${INGRESS_PORT}`,`-e SESSION_API_KEY=${sessionApiKey}`,`-e OH_SESSION_API_KEYS_0=${sessionApiKey}`,diff --git a/tests/e2e/mock-llm/mock-llm-acp-agent.spec.ts b/tests/e2e/mock-llm/mock-llm-acp-agent.spec.tsnew file mode 100644index 0000000000..1ce30c0563--- /dev/null+++ b/tests/e2e/mock-llm/mock-llm-acp-agent.spec.ts@@ -0,0 +1,415 @@+/**+ * Mock-LLM E2E test: ACP (Agent Client Protocol) agent conversation.+ *+ * Exercises the full ACP agent stack through the browser UI β the same+ * way a real user would configure and use an ACP agent:+ *+ * 1. Navigate to Settings β Agent and switch the agent type to ACP+ * 2. Select "Custom" preset, paste the mock ACP server command, save+ * 3. Reload and verify the UI reflects the saved ACP configuration+ * 4. Start a conversation from the home page and verify the ACP agent+ * responds with the expected reply token+ * 5. Resume the conversation from the sidebar after navigating away+ *+ * The mock ACP server (`mock-acp-server.py`) speaks JSON-RPC over stdio+ * and replies with a deterministic token so the test can verify the full+ * round-trip without any real LLM.+ */++import { test, expect, type Page } from "@playwright/test";+import {+ ACP_REPLY_TOKEN,+ MOCK_ACP_COMMAND_PYTHON,+ MOCK_ACP_COMMAND_SCRIPT,+ seedLocalStorage,+ routeSessionApiKey,+ dismissAnalyticsModal,+ waitForTestId,+ waitForPath,+ getConversationIdFromURL,+ waitForNonUserMessageText,+ deleteConversation,+ resetToOpenHandsAgent,+ resetMockLLM,+ ensureMockLLMProfile,+ setChatInput,+ BACKEND_URL,+ SESSION_API_KEY,+} from "./utils/mock-llm-helpers";++const USER_MESSAGE = "Hello ACP agent, please reply.";++/**+ * The command string the user types into the ACP command textarea.+ *+ * Uses environment-aware paths: in Docker E2E the agent-server runs+ * inside a container, so we use the container-side paths set by the+ * Docker Playwright config; in the npm path we use host-local paths.+ */+const ACP_COMMAND_TEXT = `${MOCK_ACP_COMMAND_PYTHON} ${MOCK_ACP_COMMAND_SCRIPT}`;++// ββ UI interaction helpers for HeroUI Autocomplete dropdowns ββββββββββ++/**+ * Select an option from a HeroUI Autocomplete dropdown (SettingsDropdownInput).+ *+ * HeroUI Autocomplete does NOT forward `data-testid` to the underlying+ * `<input>`, so we locate the combobox by its `aria-label` (which the+ * component sets to the label prop or the name prop). We then click to+ * open the listbox and click the matching option.+ */+async function selectDropdownOption(+ page: Page,+ /** aria-label of the combobox (= the SettingsDropdownInput label text) */+ comboboxLabel: string | RegExp,+ /** Visible text of the option to click */+ optionText: string | RegExp,+) {+ const combobox = page.getByRole("combobox", { name: comboboxLabel });+ await expect(combobox).toBeVisible({ timeout: 10_000 });+ // Clear any existing text (so all options show) and open+ await combobox.click();+ await combobox.fill("");+ // Wait for the option to appear and click it+ const option = page.getByRole("option", { name: optionText });+ await expect(option).toBeVisible({ timeout: 5_000 });+ await option.click();+}++test.describe.configure({ mode: "serial" });++test.describe("mock-LLM ACP agent conversation", () => {+ let conversationId: string | null = null;++ test.beforeEach(async ({ page }) => {+ await seedLocalStorage(page);+ });++ test.afterAll(async ({ request }) => {+ // Clean up the conversation+ if (conversationId) {+ try {+ await deleteConversation(request, conversationId);+ } catch {+ // best-effort+ }+ }++ // Reset agent-server back to OpenHands + restore mock LLM profile+ // so subsequent test suites (which expect agent_kind=openhands) are+ // not affected by our ACP configuration.+ try {+ await resetToOpenHandsAgent(request);+ } catch {+ // best-effort+ }+ try {+ await ensureMockLLMProfile(request);+ } catch {+ // best-effort+ }+ try {+ await resetMockLLM(request);+ } catch {+ // best-effort+ }+ });++ // ββ Step 1: Configure ACP agent through the Settings β Agent UI βββββ++ test("step 1: configure ACP agent via Settings β Agent UI", async ({+ page,+ request,+ }) => {+ // The agent-server may make internal LLM calls (condenser) even for+ // ACP conversations. Ensure a mock LLM profile exists so those calls+ // don't fail. This API call is not what we're testing β the ACP UI is.+ await ensureMockLLMProfile(request);++ await routeSessionApiKey(page);+ await page.goto("/settings/agent", { waitUntil: "domcontentloaded" });+ await dismissAnalyticsModal(page);++ // Wait for the agent settings form to load+ await waitForTestId(page, "agent-settings-screen");++ // ββ Switch agent type from OpenHands β ACP ββ++ await test.step("select ACP agent type", async () => {+ await selectDropdownOption(+ page,+ /Agent/,+ /ACP/,+ );+ });++ // ββ After selecting ACP, the preset dropdown + command fields appear ββ++ await test.step("select Custom preset and enter command", async () => {+ // Wait for the ACP-specific fields to appear+ await waitForTestId(page, "agent-preset-selector");++ // Select "Custom" preset so we can enter our own command+ await selectDropdownOption(+ page,+ /Preset/,+ /Custom/,+ );++ // Fill in the ACP command pointing to our mock server+ const commandInput = page.getByTestId("agent-command-input");+ await expect(commandInput).toBeVisible({ timeout: 5_000 });+ await commandInput.click();+ await commandInput.fill(ACP_COMMAND_TEXT);+ });++ // ββ Save the configuration ββ++ await test.step("save agent settings", async () => {+ const saveBtn = page.getByTestId("agent-save-button");+ await expect(saveBtn).toBeEnabled({ timeout: 5_000 });+ await saveBtn.click();++ // Wait for the save to complete β the button text changes briefly+ // to "Saving..." and then back to "Save Changes", and becomes+ // disabled again (isDirty flips to false).+ await expect(saveBtn).toBeDisabled({ timeout: 10_000 });+ });++ // ββ Verify: settings API reflects the ACP configuration ββ++ await test.step("verify settings API reflects ACP config", async () => {+ const resp = await request.get(`${BACKEND_URL}/api/settings`, {+ headers: {+ "X-Session-API-Key": SESSION_API_KEY,+ "X-Expose-Secrets": "encrypted",+ },+ });+ expect(resp.ok(), `GET /api/settings returned ${resp.status()}`).toBe(+ true,+ );+ const settings = await resp.json();+ const agentSettings = settings?.agent_settings as Record<string, unknown>;+ expect(agentSettings?.agent_kind).toBe("acp");++ const command = agentSettings?.acp_command;+ expect(+ Array.isArray(command),+ `acp_command should be an array, got: ${JSON.stringify(command)}`,+ ).toBe(true);+ expect(+ (command as string[]).some((tok: string) =>+ tok.includes("mock-acp-server.py"),+ ),+ `acp_command should reference mock-acp-server.py, got: ${JSON.stringify(command)}`,+ ).toBe(true);+ });+ });++ // ββ Step 2: Reload and verify the UI reflects saved ACP config ββββββ++ test("step 2: reload and verify ACP settings are persisted in UI", async ({+ page,+ }) => {+ await routeSessionApiKey(page);+ await page.goto("/settings/agent", { waitUntil: "domcontentloaded" });+ await dismissAnalyticsModal(page);+ await waitForTestId(page, "agent-settings-screen");++ // The command textarea should show the mock server command+ const commandInput = page.getByTestId("agent-command-input");+ await expect(commandInput).toBeVisible({ timeout: 5_000 });+ const commandValue = await commandInput.inputValue();+ expect(+ commandValue.includes("mock-acp-server.py"),+ `Command should contain mock-acp-server.py after reload, got: "${commandValue}"`,+ ).toBe(true);++ // The preset dropdown should indicate the custom server is active.+ // Since we used a custom command that doesn't match any built-in+ // provider, the preset should show "Custom".+ const presetSelector = page.getByTestId("agent-preset-selector");+ await expect(presetSelector).toBeVisible({ timeout: 5_000 });+ });++ // ββ Step 3: Start an ACP conversation from the home page ββββββββββββ++ test("step 3: start ACP conversation and verify agent reply", async ({+ page,+ request,+ }) => {+ test.setTimeout(120_000);++ // Passively capture POST /api/conversations payload to verify ACP tags+ let capturedPayload: Record<string, unknown> | null = null;+ const capturePayload = (req: import("@playwright/test").Request) => {+ if (+ req.method() === "POST" &&+ new URL(req.url()).pathname === "/api/conversations"+ ) {+ try {+ capturedPayload = req.postDataJSON();+ } catch {+ // non-JSON body+ }+ }+ };+ page.on("request", capturePayload);++ await routeSessionApiKey(page);+ await page.goto("/", { waitUntil: "domcontentloaded" });+ await dismissAnalyticsModal(page);++ // Wait for the home page chat launcher+ await waitForTestId(page, "home-chat-launcher");++ // Send a message to start the conversation+ await setChatInput(page, USER_MESSAGE);+ await page.getByTestId("submit-button").click();++ // Wait for navigation to the new conversation page+ await waitForPath(page, /\/conversations\/.+/, 30_000);+ page.off("request", capturePayload);+ conversationId = getConversationIdFromURL(page);++ // ββ Verify: POST /api/conversations payload has ACP agent settings ββ++ await test.step("verify conversation payload contains ACP settings", async () => {+ expect(+ capturedPayload,+ "POST /api/conversations payload was not captured",+ ).not.toBeNull();++ const agentSettings = capturedPayload!.agent_settings as+ | Record<string, unknown>+ | undefined;+ expect(+ agentSettings,+ "payload should contain agent_settings",+ ).toBeTruthy();+ expect(+ agentSettings!.agent_kind,+ `Expected agent_kind="acp" in payload, got: ${agentSettings!.agent_kind}`,+ ).toBe("acp");+ });++ // ββ Verify: agent reply contains the ACP reply token ββ++ await test.step("verify ACP agent reply appears in chat UI", async () => {+ try {+ await waitForNonUserMessageText(page, ACP_REPLY_TOKEN, 60_000);+ } catch (err) {+ // On failure, query the events API for diagnostic context+ let diag = "";+ try {+ const eventsResp = await request.get(+ `${BACKEND_URL}/api/conversations/${encodeURIComponent(conversationId!)}/events/search`,+ {+ headers: { "X-Session-API-Key": SESSION_API_KEY },+ params: { limit: "50", sort_order: "TIMESTAMP_DESC" },+ },+ );+ if (eventsResp.ok()) {+ const body = (await eventsResp.json()) as { items?: unknown[] };+ const items = body.items ?? [];+ diag =+ `Events API returned ${items.length} events:\n` ++ items+ .map(+ (e: any) =>+ ` [${e.kind ?? "?"}] source=${e.source ?? "?"} ${JSON.stringify(+ e.llm_message?.content ?? e.content ?? e.message ?? "",+ ).slice(0, 120)}`,+ )+ .join("\n");+ } else {+ diag = `Events API returned ${eventsResp.status()}`;+ }+ } catch (diagErr) {+ diag = `Events API query failed: ${diagErr}`;+ }++ throw new Error(+ `ACP reply token "${ACP_REPLY_TOKEN}" not found in chat UI after 60s.\n` ++ `Conversation: ${conversationId}\n` ++ `ACP command: ${ACP_COMMAND_TEXT}\n` ++ `${diag}`,+ { cause: err },+ );+ }+ });++ // ββ Verify: user message is visible in the chat UI ββ++ await test.step("verify user message is visible in chat UI", async () => {+ await expect(+ page+ .locator('[data-testid="user-message"]')+ .filter({ hasText: USER_MESSAGE }),+ ).toBeVisible({ timeout: 5_000 });+ });++ // ββ Verify: conversation appears in the sidebar ββ++ await test.step("verify conversation appears in sidebar", async () => {+ const cardLinks = page.locator(+ `a[href*="/conversations/${conversationId}"]`,+ );+ await expect(cardLinks.first()).toBeVisible({ timeout: 10_000 });+ });++ // ββ Verify: no error banners ββ++ await test.step("verify no error banners", async () => {+ const errorBanner = page.getByTestId("error-message-banner");+ await expect(errorBanner).not.toBeVisible({ timeout: 2_000 });+ });+ });++ // ββ Step 4: Resume the ACP conversation from the sidebar ββββββββββββ++ test("step 4: resume ACP conversation from sidebar after navigating away", async ({+ page,+ }) => {+ test.skip(!conversationId, "step 3 must complete first");++ await routeSessionApiKey(page);+ await page.goto("/", { waitUntil: "domcontentloaded" });+ await dismissAnalyticsModal(page);++ // Wait for the sidebar to load conversation cards+ const sidebarCards = page.locator('[data-testid="conversation-card"]');+ await expect(sidebarCards.first()).toBeVisible({ timeout: 15_000 });++ // Find the sidebar link to our conversation and click it+ const conversationLink = page.locator(+ `a[href*="/conversations/${conversationId}"]`,+ );+ await expect(conversationLink.first()).toBeVisible({ timeout: 10_000 });+ await conversationLink.first().click();++ // Wait for navigation back to the conversation page+ await waitForPath(page, /\/conversations\/.+/, 15_000);+ expect(page.url()).toContain(conversationId);++ // Verify the ACP agent's reply token is still visible after resume+ await test.step("verify ACP agent reply is still visible after resume", async () => {+ await waitForNonUserMessageText(page, ACP_REPLY_TOKEN, 15_000);+ });++ // Verify the user's original message is still visible+ await test.step("verify user message is still visible after resume", async () => {+ await expect(+ page+ .locator('[data-testid="user-message"]')+ .filter({ hasText: USER_MESSAGE }),+ ).toBeVisible({ timeout: 10_000 });+ });++ // Verify no error banners after resume+ await test.step("verify no error banners after resume", async () => {+ const errorBanner = page.getByTestId("error-message-banner");+ await expect(errorBanner).not.toBeVisible({ timeout: 2_000 });+ });+ });+});diff --git a/tests/e2e/mock-llm/scripts/mock-acp-server.py b/tests/e2e/mock-llm/scripts/mock-acp-server.pynew file mode 100644index 0000000000..65be4a8d7b--- /dev/null+++ b/tests/e2e/mock-llm/scripts/mock-acp-server.py@@ -0,0 +1,114 @@+"""Mock ACP (Agent Client Protocol) server for E2E tests.++A minimal stdio-based ACP agent that speaks JSON-RPC over stdin/stdout.+The agent-server spawns this as a subprocess via ``acp_command`` and+communicates with it using the ACP protocol.++The agent responds to prompts with a scripted text reply containing+``REPLY_TOKEN``, which the E2E test verifies appeared in the UI.++Usage:+ python mock-acp-server.py [--reply-token TOKEN]++Requires:+ pip install agent-client-protocol (installed as dep of openhands-sdk)+"""++import argparse+import asyncio+import sys++import acp+from acp.schema import (+ AgentCapabilities,+ Implementation,+ PromptCapabilities,+)++REPLY_TOKEN = "MOCK_ACP_E2E_REPLY_OK"+++class MockACPAgent(acp.Agent):+ """Minimal ACP agent that returns a scripted reply to every prompt."""++ def __init__(self, reply_token: str = REPLY_TOKEN) -> None:+ self.reply_token = reply_token+ self._conn: acp.Client | None = None++ def on_connect(self, conn: acp.Client) -> None:+ self._conn = conn++ async def initialize(+ self,+ protocol_version: int,+ client_capabilities=None,+ client_info=None,+ **kwargs,+ ) -> acp.InitializeResponse:+ print("[mock-acp] initialize", file=sys.stderr, flush=True)+ return acp.InitializeResponse(+ protocol_version=acp.PROTOCOL_VERSION,+ agent_info=Implementation(+ name="mock-acp-e2e",+ title="Mock ACP E2E Agent",+ version="1.0.0",+ ),+ agent_capabilities=AgentCapabilities(+ prompt_capabilities=PromptCapabilities(),+ ),+ )++ async def new_session(+ self,+ cwd: str,+ additional_directories=None,+ mcp_servers=None,+ **kwargs,+ ) -> acp.NewSessionResponse:+ print(f"[mock-acp] new_session cwd={cwd}", file=sys.stderr, flush=True)+ return acp.NewSessionResponse(session_id="mock-acp-session-001")++ async def prompt(+ self,+ prompt,+ session_id: str,+ message_id: str | None = None,+ **kwargs,+ ) -> acp.PromptResponse:+ # Extract user text for logging+ user_text = ""+ if prompt:+ for block in prompt:+ if hasattr(block, "text"):+ user_text += block.text+ print(+ f"[mock-acp] prompt session={session_id} text={user_text!r}",+ file=sys.stderr,+ flush=True,+ )++ # Send the agent's text reply as a session/update notification+ if self._conn:+ await self._conn.session_update(+ session_id=session_id,+ update=acp.update_agent_message_text(self.reply_token),+ )++ return acp.PromptResponse(stop_reason="end_turn")+++async def main(reply_token: str) -> None:+ agent = MockACPAgent(reply_token=reply_token)+ print(f"[mock-acp] starting (token={reply_token})", file=sys.stderr, flush=True)+ await acp.run_agent(agent)+++if __name__ == "__main__":+ parser = argparse.ArgumentParser(description="Mock ACP agent for E2E tests")+ parser.add_argument(+ "--reply-token",+ default=REPLY_TOKEN,+ help="Token to include in agent replies (default: %(default)s)",+ )+ args = parser.parse_args()+ asyncio.run(main(args.reply_token))diff --git a/tests/e2e/mock-llm/utils/mock-llm-helpers.ts b/tests/e2e/mock-llm/utils/mock-llm-helpers.tsindex b557a6affd..827002ff4a 100644--- a/tests/e2e/mock-llm/utils/mock-llm-helpers.ts+++ b/tests/e2e/mock-llm/utils/mock-llm-helpers.ts@@ -5,6 +5,7 @@* shorter timeouts (responses are instant), no real credential handling.*/+import { resolve } from "node:path";import { expect, type APIRequestContext, type Page } from "@playwright/test";// Tokens that the mock LLM server uses β must match mock-llm-server.py.@@ -482,3 +483,73 @@ export const BACKEND_ONLY_URL = `http://localhost:${BACKEND_ONLY_INGRESS_PORT}`;// Mock automation helpers removed β the automation test now hits the real// automation backend running inside the bin/agent-canvas.mjs stack.++// βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ+// ACP agent configuration helpers+// βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ++/** Reply token the mock ACP server includes in its responses. */+export const ACP_REPLY_TOKEN = "MOCK_ACP_E2E_REPLY_OK";++/**+ * Absolute path to the Python binary for the mock ACP server.+ *+ * In CI, ``MOCK_LLM_PYTHON`` is a relative venv path like+ * ``.mock-llm-venv/bin/python3``. The agent-server spawns the ACP+ * subprocess from its own CWD (which may differ from the repo root),+ * so we resolve relative paths to absolute here. Bare executable+ * names (no directory separator) are left for PATH lookup.+ */+export const MOCK_ACP_PYTHON = (() => {+ const raw = process.env.MOCK_LLM_PYTHON ?? "python3";+ // Resolve paths containing a directory separator (relative like+ // ".mock-llm-venv/bin/python3"); leave bare names like "python3"+ // for PATH lookup.+ return raw.includes("/") || raw.includes("\\") ? resolve(raw) : raw;+})();++/**+ * Absolute path to the mock ACP server script, resolved from the project root.+ * The agent-server spawns this as a subprocess via ``acp_command``.+ */+export const MOCK_ACP_SERVER_PATH = resolve(+ "tests/e2e/mock-llm/scripts/mock-acp-server.py",+);++/**+ * The Python + script path the test types into the ACP command textarea.+ *+ * When running the Docker E2E config, the agent-server lives inside a+ * container where host-filesystem paths don't exist. The Docker config+ * volume-mounts the mock ACP script and sets ``MOCK_ACP_CONTAINER_*``+ * env vars with the container-side paths. The npm config leaves those+ * vars unset, so we fall back to the host-local absolute paths.+ */+export const MOCK_ACP_COMMAND_PYTHON =+ process.env.MOCK_ACP_CONTAINER_PYTHON || MOCK_ACP_PYTHON;+export const MOCK_ACP_COMMAND_SCRIPT =+ process.env.MOCK_ACP_CONTAINER_SCRIPT || MOCK_ACP_SERVER_PATH;++/**+ * Reset the agent-server back to the default OpenHands agent.+ * Used in afterAll cleanup to avoid polluting other test suites.+ */+export async function resetToOpenHandsAgent(+ request: APIRequestContext,+) {+ const resp = await request.patch(`${BACKEND_URL}/api/settings`, {+ headers: {+ "X-Session-API-Key": SESSION_API_KEY,+ "Content-Type": "application/json",+ },+ data: {+ agent_settings_diff: {+ agent_kind: "openhands",+ },+ },+ });+ // Best-effort β don't fail the test if cleanup fails+ if (!resp.ok()) {+ console.warn(`[cleanup] Reset to OpenHands failed: ${resp.status()}`);+ }+}