Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Refuse a shell the database, show a screen that ended, and unblock a taken-over sign-in (#289)
David McKayfb0c797ed2e6
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 95f9be68..c851fcd3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,37 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.## Unreleased+### A Bot's shell can no longer reach the embedded database without a password++In the all-in-one image the cluster was `trust`-auth on loopback, and the Bot's shell runs in the+same container: it could `psql -h 127.0.0.1 -U openbot` with no password and read the audit trail,+the policy store, and the credential vault as the instance owner. The cluster now uses+`scram-sha-256` with a password generated on first init and kept beside the data, handed to the API+over the container environment. The shell has no way to learn it, so the connection is refused. An+external `DATABASE_URL` deployment is unaffected.++### A live screen that ends says so, instead of freezing the last frame++When a Bot's live screen ended — the computer stopped, or the socket failed — the message explaining+why was drawn only by a component the take-the-wheel view does not mount, so the screen sat frozen on+its last frame with nothing said. The reason is now shown where the live screen is.++### A Bot's browser drops the automation flags a person needs gone to sign in++The browser announced itself as automated (`navigator.webdriver`, the enable-automation switch),+which sites like Google refuse even when a real person has taken the wheel. Those flags are now off+at the source — the browser flag, not a script that patches `navigator.webdriver` and leaves the+other tells. A headless build still reports `HeadlessChrome` in its user agent, which only running+headed under a virtual display removes; that heavier change is tracked separately.++### An empty model reply, or a run with no question, no longer ends in silence++Two failures on strict OpenAI-compatible providers (z.ai GLM, Anthropic): a follow-up run that+carried only tool deltas and no human turn was refused outright, and a reply with no text and no tool+call ended the run with nothing on screen. A run with no human turn now carries a neutral+continuation, and an empty reply ends on a visible line rather than in silence. OpenAI, which+tolerated both, is unchanged.+### Embedded PostgreSQL initialises on a platform volume, and says so when it cannot`EMBEDDED_POSTGRES=on` could not create its cluster on a platform whose persistent volume is an ext4diff --git a/agent-computer/src/profiles.ts b/agent-computer/src/profiles.tsindex 4e6fe26c..c97f041c 100644--- a/agent-computer/src/profiles.ts+++ b/agent-computer/src/profiles.ts@@ -96,6 +96,15 @@ const LAUNCH_ARGS = [...(SANDBOX_ENABLED ? [] : ["--no-sandbox"]),"--disable-dev-shm-usage","--password-store=basic",+ // Drop the automation signals Chromium sets for itself, so a real person who takes the wheel can+ // sign in to a site that refuses obvious automation (Google among them). This is the flag, not a+ // JS patch of `navigator.webdriver`: the flag turns the property off at the source, where spoofing+ // it from a script leaves the other tells a detector cross-checks. It does not change what the Bot+ // may do; the governed path is unchanged. The larger tell — a headless build reporting+ // `HeadlessChrome` in its user agent — is only removed by running headed under a virtual display,+ // which is a heavier image change tracked separately; this reduces the signals it can reduce+ // without one.+ "--disable-blink-features=AutomationControlled",];console.info(@@ -372,6 +381,11 @@ export function createProfiles(root: string, onClosed: BrowserClosed) {const proxy = egressFor(botId, process.env);const context = await chromium.launchPersistentContext(dir, {args: LAUNCH_ARGS,+ // Playwright launches with `--enable-automation`, which sets `navigator.webdriver` and the+ // "controlled by automated software" banner. Dropped for the same reason as the flag above:+ // a person who takes the wheel should be able to sign in. Named explicitly so the sandbox+ // default args Playwright still supplies are otherwise left intact.+ ignoreDefaultArgs: ["--enable-automation"],// Playwright adds `--no-sandbox` on its own unless told otherwise, so leaving this out// means the flag above decides nothing and a deployment that asked for the sandbox does// not get one. Verified by reading the launched process arguments, not by trusting either.diff --git a/agent-langgraph/src/history.ts b/agent-langgraph/src/history.tsindex ce16acf4..2f982528 100644--- a/agent-langgraph/src/history.ts+++ b/agent-langgraph/src/history.ts@@ -103,9 +103,32 @@ export function toLangChainMessages(input: RunAgentInput): BaseMessage[] {}}+ /*+ * A run that carries no human turn is answered by OpenAI and refused by the strict providers.+ *+ * OpenAI tolerates a history that opens on an assistant or tool message. Anthropic and the strict+ * OpenAI-compatible providers (z.ai GLM among them) require the first non-system message to be a+ * human one, and will not answer a history that is only deltas: an assistant turn and its tool+ * results with nothing a person said to respond to. A follow-up run continuing after a tool result+ * is exactly that shape, so on those providers it came back empty and the run ended in silence.+ *+ * A neutral continuation turn gives them one to answer. It is appended only when the history holds+ * no human turn at all, so a normal conversation is untouched, and OpenAI — which already answered+ * the same history — sees no change beyond one trailing line asking it to continue.+ */+ const hasHumanTurn = messages.some(+ (message): message is HumanMessage => message instanceof HumanMessage,+ );+ if (!hasHumanTurn) {+ messages.push(new HumanMessage(CONTINUE_TURN));+ }+return messages;}+/** The continuation a strict provider needs when a run carries only deltas. See toLangChainMessages. */+const CONTINUE_TURN = "Continue from where the conversation above left off.";+function parseArguments(raw: string): Record<string, unknown> {try {const parsed = JSON.parse(raw || "{}");diff --git a/agent-langgraph/src/index.ts b/agent-langgraph/src/index.tsindex 58a286b4..9b439a17 100644--- a/agent-langgraph/src/index.ts+++ b/agent-langgraph/src/index.ts@@ -332,7 +332,9 @@ function buildGraph(input: RunAgentInput) {return new StateGraph(MessagesAnnotation).addNode("answer", async (state) => ({- messages: [await bound.invoke(state.messages)],+ messages: [+ withVisibleReply((await bound.invoke(state.messages)) as AIMessage),+ ],})).addNode("tools", async (state) => {const last = state.messages.at(-1) as AIMessage;@@ -384,6 +386,37 @@ function buildGraph(input: RunAgentInput) {.compile();}+/**+ * A reply with nothing in it ends the run in silence, so give it a line to end on.+ *+ * When a model returns no text and no tool call, the conditional edge sees no calls and returns END,+ * and the person is left looking at a turn that produced no answer and no reason. Strict providers do+ * this on a run they will not answer. Re-asking tends to get the same empty reply, so rather than+ * loop, the run ends on a visible message saying what happened. Only a genuinely empty reply is+ * touched: a reply with any text, or any tool call, is returned exactly as the model produced it.+ */+function withVisibleReply(reply: AIMessage): AIMessage {+ const hasCall = (reply.tool_calls ?? []).length > 0;+ if (hasCall || hasVisibleText(reply.content)) return reply;+ return new AIMessage({ content: EMPTY_REPLY_FALLBACK });+}++function hasVisibleText(content: AIMessage["content"]): boolean {+ if (typeof content === "string") return content.trim().length > 0;+ if (Array.isArray(content)) {+ return content.some((part) =>+ typeof part === "string"+ ? part.trim().length > 0+ : typeof (part as { text?: unknown }).text === "string" &&+ (part as { text: string }).text.trim().length > 0,+ );+ }+ return false;+}++const EMPTY_REPLY_FALLBACK =+ "The model returned an empty reply and the run ended without an answer. This can happen with a strict provider; try asking again.";+async function runAgent(input: RunAgentInput): Promise<Response> {const encoder = new EventEncoder();const stream = new ReadableStream<Uint8Array>({diff --git a/app/src/components/computer/computer-view.tsx b/app/src/components/computer/computer-view.tsxindex 532c85b1..a0990604 100644--- a/app/src/components/computer/computer-view.tsx+++ b/app/src/components/computer/computer-view.tsx@@ -640,11 +640,24 @@ export function ComputerView({/></div>) : showLiveScreen ? (- <LiveScreen- computerId={computerId}- driving={driving}- onProblem={setProblem}- />+ <div className="relative w-full" style={{ aspectRatio }}>+ <LiveScreen+ computerId={computerId}+ driving={driving}+ onProblem={setProblem}+ />+ {/*+ A live screen that ends reports why through `onProblem`, and this is the+ branch that is mounted when it does. Without drawing it here the message+ landed in `problem`, which only the sibling `NothingToSee` reads, so the+ screen ended with the stale last frame frozen on the canvas and nothing said.+ */}+ {problem ? (+ <div className="absolute inset-0 flex items-center justify-center bg-background/85 p-4 text-center text-sm text-muted-foreground">+ <span>{problem}</span>+ </div>+ ) : null}+ </div>) : (<div className="relative w-full" style={{ aspectRatio }}><NothingToSeediff --git a/docker/s6/scripts/postgres-init.sh b/docker/s6/scripts/postgres-init.shindex ba9a1ffe..b40abf29 100755--- a/docker/s6/scripts/postgres-init.sh+++ b/docker/s6/scripts/postgres-init.sh@@ -1,13 +1,20 @@-#!/bin/sh-# Create the cluster the first time, and only the first time.+#!/command/with-contenv sh+# Create the cluster the first time, and hand the API a password every time.#-# Bound to loopback and trust-auth on purpose: the only client is the process beside it, inside this-# container, and a password would be a secret with nobody to keep it from. Publishing 5432 from this-# container would change that, which is why nothing here does.+# Bound to loopback, but no longer trust-auth. The process beside it is not the only client: the+# Bot's shell runs in this same container, and under trust it could `psql -h 127.0.0.1 -U openbot`+# with no password and reach the audit trail, the policy store, and the credential vault as the+# instance owner. A generated password closes that: the shell has no way to learn it (its own+# environment is an allow-list that does not carry the URL), and scram refuses a connection without+# it. The password lives beside the data on the same volume, so it survives a restart the way the+# data does.set -eu[ "${EMBEDDED_POSTGRES:-off}" = "on" ] || exit 0DATA=/var/lib/postgresql/data+PW_FILE=/var/lib/postgresql/pgpassword+BIN=/usr/lib/postgresql/16/bin+if [ ! -s "$DATA/PG_VERSION" ]; then# Created and owned here, as root, because this is the only step in a position to do it.#@@ -33,9 +40,35 @@ if [ ! -s "$DATA/PG_VERSION" ]; thenexit 1fi- s6-setuidgid postgres /usr/lib/postgresql/16/bin/initdb -D "$DATA" -A trust -U openbot >/dev/null- s6-setuidgid postgres /usr/lib/postgresql/16/bin/pg_ctl -D "$DATA" -o "-c listen_addresses=127.0.0.1" -w start >/dev/null- s6-setuidgid postgres /usr/lib/postgresql/16/bin/createdb -U openbot openbot- s6-setuidgid postgres /usr/lib/postgresql/16/bin/psql -U openbot -d openbot -c 'CREATE EXTENSION IF NOT EXISTS vector' >/dev/null- s6-setuidgid postgres /usr/lib/postgresql/16/bin/pg_ctl -D "$DATA" -w stop >/dev/null+ # The cluster's password, generated once and kept on the same volume as the data. openssl is not in+ # this image; /dev/urandom is. 600 and owned by postgres, so the Bot's shell (which runs as pwuser)+ # cannot read the file even if it goes looking.+ PW="$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')"+ ( umask 077; printf '%s' "$PW" > "$PW_FILE" )+ chown postgres:postgres "$PW_FILE"++ # initdb reads the superuser password from a file (never an argument, which would show in `ps`), and+ # sets scram for both local and TCP connections. The temp file is removed the moment initdb returns.+ PWTMP="$(mktemp)"+ printf '%s' "$PW" > "$PWTMP"+ chown postgres:postgres "$PWTMP"+ s6-setuidgid postgres "$BIN/initdb" -D "$DATA" -A scram-sha-256 -U openbot --pwfile="$PWTMP" >/dev/null+ rm -f "$PWTMP"++ s6-setuidgid postgres "$BIN/pg_ctl" -D "$DATA" -o "-c listen_addresses=127.0.0.1" -w start >/dev/null+ # Over TCP with the password now, since the cluster no longer trusts an unauthenticated connection.+ PGPASSWORD="$PW" s6-setuidgid postgres "$BIN/createdb" -h 127.0.0.1 -U openbot openbot+ PGPASSWORD="$PW" s6-setuidgid postgres "$BIN/psql" -h 127.0.0.1 -U openbot -d openbot -c 'CREATE EXTENSION IF NOT EXISTS vector' >/dev/null+ s6-setuidgid postgres "$BIN/pg_ctl" -D "$DATA" -w stop >/dev/null+fi++# Every boot, not only the first: hand the password-bearing URL to the services that connect over TCP+# (`api` and `migrate`, both `with-contenv`). The password persists with the cluster; the container+# environment is fresh each boot, so this has to run outside the first-init guard above. The file is+# root-written here into s6's own environment directory, which pwuser cannot write and the Bot's shell+# does not read.+if [ -s "$PW_FILE" ]; then+ PW="$(cat "$PW_FILE")"+ printf 'postgres://openbot:%[email protected]:5432/openbot' "$PW" \+ > /run/s6/container_environment/DATABASE_URLfi