Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Report a surface-owned tool call to the surface (#77)
Rahul Sethuramb24fdfa3082a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 285a1411..b448227a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -221,6 +221,13 @@ Sessions survive and nobody signs in again.laptop `http://localhost` counts as one, so this never showed up in development; on a realaddress it does not, and the surface did nothing at all when you pressed send. No message, noerror. Ids now come from an API with no such restriction.+- **A framework Bot asked for a browser action and nothing happened.** `agent-langgraph` ends a run+ when the model calls a tool the surface owns, which is how a tool that lives in the browser is+ supposed to work: the run finishes, the surface acts, and the next run carries the result. But the+ call was only reported to the surface from the node that executes this deployment's own tools, and+ that node is exactly what an ending run skips. The person saw their own message, no answer under+ it, and no explanation, because a run that finishes carrying nothing is not an error. Every Bot+ action in the browser was affected: opening a page, filling a form, asking for help at a sign-in.### Changeddiff --git a/agent-langgraph/src/index.ts b/agent-langgraph/src/index.tsindex c524fb0b..158fdc7d 100644--- a/agent-langgraph/src/index.ts+++ b/agent-langgraph/src/index.ts@@ -436,7 +436,6 @@ async function runAgent(input: RunAgentInput): Promise<Response> {// Accumulated rather than emitted per chunk, because a tool call's arguments arrive in// fragments and AG-UI wants one call. The framework hands back assembled `tool_calls` on the// final message, which is precisely the plumbing agent-bot does by hand.- let finalMessage: AIMessage | null = null;/** Calls seen on the way past, so a result can be paired with the arguments it answered. */const pending = new Map<string,@@ -470,7 +469,6 @@ async function runAgent(input: RunAgentInput): Promise<Response> {if (event.event === "on_chat_model_end") {const output = event.data?.output as AIMessage | undefined;if (output) {- finalMessage = output;for (const call of output.tool_calls ?? []) {pending.set(call.id ?? call.name, {name: call.name,@@ -520,6 +518,31 @@ async function runAgent(input: RunAgentInput): Promise<Response> {closeText();+ /*+ * Calls this process did not run, which is what a tool the surface owns looks like from here.+ *+ * The graph ends the run on one of those rather than inventing a result, so the `tools` node+ * never fires and the loop above never reports the call. Without this the run is a clean+ * RUN_STARTED/RUN_FINISHED pair carrying nothing at all: the person's message sits there with+ * no answer under it, the surface never learns there was a browser action to execute, and+ * because an empty run is not an error by the protocol, nothing says so. No result is sent+ * with them; producing it is the surface's half, and it begins the next run holding it.+ */+ for (const [id, call] of pending) {+ send({+ type: "TOOL_CALL_START",+ toolCallId: id,+ toolCallName: call.name,+ } as BaseEvent);+ send({+ type: "TOOL_CALL_ARGS",+ toolCallId: id,+ delta: JSON.stringify(call.args),+ } as BaseEvent);+ send({ type: "TOOL_CALL_END", toolCallId: id } as BaseEvent);+ }+ pending.clear();+send({type: "RUN_FINISHED",threadId: input.threadId,