Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Refuse the shell and a workspace write while a person holds the wheel (#247)
beardthelion0403be14817f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 350328b9..9acfaa3f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -36,6 +36,20 @@ uses. It still cannot address another pod, a node, or a cloud metadata endpoint.destination and so permitted everything. That rule now covers the API server alone, and`networkPolicy.kubernetesApiCidr` narrows it to your cluster's service range; left empty it stays asit was, because a chart cannot know that range.+### Taking the wheel stops the Bot's shell, not just its clicks++While a person held the wheel the Bot was refused on the page, and not in the shell. `/exec` and a+workspace write went through, so a Bot could keep running commands and rewriting its `/workspace`+underneath somebody who had taken the browser at a login wall. The guard existed and covered+navigation and the four page actions; the shell arrived later and was never wired to it.++Every acting path now asks the same question in one place, so the property the documentation states+is the property the computer has. Reading is deliberately not acting: `/files/read` and+`/files/list` still answer while a person drives, because a Bot that has just been stopped still has+to be able to say what it was doing.++Nothing to configure. A Bot that acts during a takeover gets the refusal it already got for a click,+and the trail records the attempt and the failure the same way.### A finished turn shows the page it opened, not the one open nowdiff --git a/agent-computer/src/authorisation.ts b/agent-computer/src/authorisation.tsindex 667f2d44..cbeabb10 100644--- a/agent-computer/src/authorisation.ts+++ b/agent-computer/src/authorisation.ts@@ -47,3 +47,30 @@ export function offeredToken(headers: Headers, url: URL): string {export function isOpenPath(pathname: string): boolean {return pathname === "/health";}++/**+ * Which paths act on the computer, and so are refused while a person holds the wheel.+ *+ * One list, asked once per request, rather than a check inside each handler. The shell is the reason:+ * `/exec` arrived after the wheel existed and was never given the guard the page paths had, so a Bot+ * could keep running commands and writing files underneath somebody who had taken the browser at a+ * login wall. A per-handler check is exactly the thing the next endpoint forgets, which is how that+ * happened; a list the dispatcher consults is one an endpoint has to be added to.+ *+ * Reading is not acting. `/files/read` and `/files/list` stay open so a Bot that has been stopped can+ * still read its own notes and explain what it was doing, which is the answer the person handing the+ * wheel back usually wants.+ */+const ACTING_PATHS = new Set([+ "/navigate",+ "/click",+ "/type",+ "/key",+ "/scroll",+ "/exec",+ "/files/write",+]);++export function actsOnTheComputer(pathname: string): boolean {+ return ACTING_PATHS.has(pathname);+}diff --git a/agent-computer/src/index.ts b/agent-computer/src/index.tsindex b4762635..5d7177a6 100644--- a/agent-computer/src/index.ts+++ b/agent-computer/src/index.ts@@ -1,7 +1,12 @@import { serve } from "bun";import type { Page } from "playwright";import { parseAriaSnapshot, type SnapshotElement } from "./aria-snapshot";-import { isOpenPath, matchesToken, offeredToken } from "./authorisation";+import {+ actsOnTheComputer,+ isOpenPath,+ matchesToken,+ offeredToken,+} from "./authorisation";import { isPlainBotId } from "./bot-id";import {type Control,@@ -485,6 +490,26 @@ serve<StreamData>({}const session = sessionFor(botId);+ /*+ * The wheel, asked once for everything that acts.+ *+ * Refused here rather than inside each handler because the handler that forgets is the whole+ * defect: the shell shipped without this check and ran commands underneath a person who had taken+ * the browser at a login wall. `actsOnTheComputer` is the list, and a new acting endpoint is+ * refused by being added to it rather than by remembering to repeat this.+ */+ if (actsOnTheComputer(url.pathname)) {+ try {+ session.control.assertBotMayAct();+ } catch (error) {+ // A person holding the wheel is not a failure of the action; the Bot should wait and say so.+ if (error instanceof ControlError) {+ return json({ error: error.message, humanHasControl: true }, 409);+ }+ throw error;+ }+ }+if (url.pathname === "/stream") {/** The socket carries the Bot in the query because it cannot do it in a header. Every other call here names@@ -696,7 +721,6 @@ serve<StreamData>({const startedAt = Date.now();try {- session.control.assertBotMayAct();const target = await currentPage(botId);await target.goto(body.url, {waitUntil: "domcontentloaded",@@ -715,10 +739,6 @@ serve<StreamData>({elapsedMs: Date.now() - startedAt,});} catch (error) {- // A person holding the wheel is not a failed navigation; the Bot should wait.- if (error instanceof ControlError) {- return json({ error: error.message, humanHasControl: true }, 409);- }// The page is the Bot's working surface, so a failed navigation is reported rather than// thrown: the transcript needs to say what happened, and the browser stays usable.return json(@@ -893,7 +913,6 @@ serve<StreamData>({const startedAt = Date.now();try {- session.control.assertBotMayAct();const target = await currentPage(botId);const detail = await performAction(session,@@ -932,10 +951,6 @@ serve<StreamData>({if (error instanceof StaleSnapshotError) {return json({ error: error.message, stale: true }, 409);}- // 409 as well, and for the same reason: nothing is broken, the caller simply has to wait.- if (error instanceof ControlError) {- return json({ error: error.message, humanHasControl: true }, 409);- }return json({ error: describe(error, "The action failed.") }, 502);}}diff --git a/agent-computer/tests/authorisation.test.ts b/agent-computer/tests/authorisation.test.tsindex 9ec6521b..34fcfdc4 100644--- a/agent-computer/tests/authorisation.test.ts+++ b/agent-computer/tests/authorisation.test.ts@@ -1,5 +1,10 @@import { describe, expect, test } from "bun:test";-import { isOpenPath, matchesToken, offeredToken } from "../src/authorisation";+import {+ actsOnTheComputer,+ isOpenPath,+ matchesToken,+ offeredToken,+} from "../src/authorisation";/*** The check that stands in front of a Bot's browser.@@ -88,3 +93,50 @@ describe("what an unauthenticated caller may reach", () => {}});});++/**+ * Which paths the wheel stops.+ *+ * A person takes the wheel at a login wall precisely because they no longer want the Bot acting, and+ * `control.ts` states the property outright: "While a person holds control every acting call from the+ * Bot is refused". That was true of the page from the start and untrue of the shell, which arrived+ * later (#62) and was never wired to the wheel, so a Bot could run a command and rewrite the+ * workspace underneath somebody mid-sign-in.+ *+ * The list lives here, beside the other path decision, rather than in `index.ts`, for the reason the+ * header of this file gives: a decision next to `chromium.launch()` cannot be tested without Chrome.+ *+ * Reading is not acting. `/files/read` and `/files/list` stay open so a Bot waiting to be handed the+ * wheel back can still say what it was doing.+ */+describe("what the wheel stops while a person is driving", () => {+ test("every path that acts on the computer, the shell and a workspace write included", () => {+ for (const path of [+ "/navigate",+ "/click",+ "/type",+ "/key",+ "/scroll",+ "/exec",+ "/files/write",+ ]) {+ expect(actsOnTheComputer(path)).toBeTrue();+ }+ });++ test("reading, looking and the handover itself are not acting", () => {+ for (const path of [+ "/files/read",+ "/files/list",+ "/snapshot",+ "/screenshot",+ "/health",+ "/control",+ "/control/take",+ "/control/release",+ "/stream",+ ]) {+ expect(actsOnTheComputer(path)).toBeFalse();+ }+ });+});