Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Answer on Responses API models, and set how hard they think (#212) (#268)
Ethan Stoner70cacb2f8765
diff --git a/.env.example b/.env.exampleindex dad529df..cb3c4ac6 100644--- a/.env.example+++ b/.env.example@@ -152,15 +152,24 @@ OPENAI_API_KEY=# GOOGLE_API_KEY=# Which model the framework Bot uses. Defaults per provider: gpt-5.5, claude-sonnet-4-5,-# gemini-2.5-flash. Not a 5.6 tier: this integration answers nothing at all on gpt-5.6-* through the-# Responses API, driven against the real service. Set one here to try it and the Responses API is-# switched on automatically. The built-in Bots do run 5.6, through the package's model.yaml.+# gemini-2.5-flash. A 5.6 tier works here: set one and the Responses API is switched on+# automatically. It used to answer nothing at all on those models — RUN_STARTED, RUN_FINISHED, no+# text — because that API streams content blocks rather than a string and the run read only the+# string. The default is left at 5.5 so the two shipped Bots stay comparable out of the box.# BOT_MODEL=gpt-5.5# OpenAI only, and rarely needed: the framework Bot turns the Responses API on by itself for models# that require it. Set it when you are using a model this build has not heard of that needs it too.# BOT_RESPONSES_API=false+# How hard the framework Bot thinks, on models that reason. OpenAI's setting, and one the Responses+# API carries, so it needs a model running on that API — a 5.6 tier does by itself, anything else+# wants BOT_RESPONSES_API=true. Unset, the model keeps its provider's own default.+#+# One of: none, minimal, low, medium, high, xhigh, max. Anything else and the Bot refuses to start+# and says so, rather than starting with a setting that quietly went nowhere.+# BOT_REASONING_EFFORT=+# The Bot computer. Absent means the feature is off and its routes are not mounted.AGENT_COMPUTER_URL=http://localhost:4100# Secret every computer requires from its caller. `agent-computer` drives a browser holding realdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex bd28b9a3..09f8b136 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,20 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.## Unreleased+### The framework Bot answers on 5.6-tier models, and can be told how hard to think++Pointing `BOT_MODEL` at a `gpt-5.6-*` model gave a Bot that started, reported healthy, and then said+nothing: every run was a RUN_STARTED and a RUN_FINISHED with no text between them. Those models are+run on the Responses API, which streams content blocks where chat completions streams a string, and+the run read only the string — so every delta was dropped on the floor. Both shapes are read now.+Nothing changes for a deployment on 5.5 or on Anthropic or Google.++`BOT_REASONING_EFFORT` sets how hard a reasoning model thinks: `none`, `minimal`, `low`, `medium`,+`high`, `xhigh` or `max`. Unset, the model keeps its provider's default. A value the API does not+have, or one set where it cannot be sent — a provider that is not OpenAI, or a model not on the+Responses API — stops the Bot at startup with a message naming what to change, rather than starting+with a setting that goes nowhere.+### A Bot can be asked to do something on a schedule"Every weekday at nine, post the standup notes here" is now something a Bot can be asked rather thandiff --git a/agent-langgraph/src/deltas.ts b/agent-langgraph/src/deltas.tsnew file mode 100644index 00000000..ea197245--- /dev/null+++ b/agent-langgraph/src/deltas.ts@@ -0,0 +1,44 @@+/**+ * The prose in a streamed chunk, whichever API produced it.+ *+ * Its own module for the reason `history.ts` is: `index.ts` calls `serve()` at module scope, so+ * importing it to reach one pure function binds a port.+ *+ * Chat completions streams `content` as a string. The Responses API does not: `@langchain/openai`+ * turns every `response.output_text.delta` into a content block — `[{ type: "text", text, index }]` —+ * and a reasoning model puts its summary in that same array under a different type. Reading the+ * string shape alone is why a 5.6-tier Bot returned RUN_STARTED and RUN_FINISHED with nothing+ * between them.+ */++/** A content block as the integrations produce them. Only the two fields that decide text matter. */+interface ContentBlock {+ type?: unknown;+ text?: unknown;+}++/**+ * Text the person should see, and nothing else.+ *+ * Selected by block type rather than by "has a text field": reasoning summaries are the Bot's+ * private working, and a surface printing them would be showing the person something never meant+ * for them.+ */+export function textOfChunk(content: unknown): string {+ if (typeof content === "string") return content;+ if (!Array.isArray(content)) return "";++ let text = "";+ for (const part of content) {+ if (typeof part === "string") {+ text += part;+ continue;+ }+ if (!part || typeof part !== "object") continue;+ const block = part as ContentBlock;+ if (block.type === "text" && typeof block.text === "string") {+ text += block.text;+ }+ }+ return text;+}diff --git a/agent-langgraph/src/index.ts b/agent-langgraph/src/index.tsindex b1302a46..58a286b4 100644--- a/agent-langgraph/src/index.ts+++ b/agent-langgraph/src/index.ts@@ -12,7 +12,9 @@ import {import { ChatOpenAI } from "@langchain/openai";import { serve } from "bun";import { hasManagedAgentToken } from "../../shared/agent-authorisation";+import { textOfChunk } from "./deltas";import { toLangChainMessages } from "./history";+import { readReasoningEffort } from "./model-options";/*** The same Bot, on a framework.@@ -97,6 +99,40 @@ const ANTHROPIC_BASE_URL = process.env.ANTHROPIC_BASE_URL?.trim() || undefined;const GOOGLE_BASE_URL =process.env.GOOGLE_GENERATIVE_AI_BASE_URL?.trim() || undefined;+/**+ * OpenAI only, and Responses API only: how hard this Bot is allowed to think.+ *+ * Checked here rather than sent onward and forgotten. An effort the API does not have is dropped+ * somewhere down the stack, and a Bot that starts, looks healthy and then thinks for as long as it+ * likes is a worse outcome than one that refuses to start and says why.+ */+const { effort: REASONING_EFFORT, problem: REASONING_PROBLEM } =+ readReasoningEffort(process.env.BOT_REASONING_EFFORT);+if (REASONING_PROBLEM) {+ console.error(REASONING_PROBLEM);+ process.exit(1);+}+/*+ * The two ways this setting would reach an API with nowhere to put it.+ *+ * Anthropic and Google express thinking budgets differently, and on `/v1/chat/completions` the+ * field does not exist at all. Refusing is the same call the issue makes about invalid values:+ * configuration that goes nowhere is worse than configuration that is absent, because the Bot looks+ * configured either way. Both messages name the variable that would make it work.+ */+if (REASONING_EFFORT && PROVIDER !== "openai") {+ console.error(+ `BOT_REASONING_EFFORT is OpenAI's setting, and BOT_PROVIDER=${PROVIDER}. Unset it, or set BOT_PROVIDER=openai.`,+ );+ process.exit(1);+}+if (REASONING_EFFORT && !USE_RESPONSES_API) {+ console.error(+ `BOT_REASONING_EFFORT needs the Responses API, and BOT_MODEL=${MODEL} is not being run on it. Use a model that requires it, or set BOT_RESPONSES_API=true.`,+ );+ process.exit(1);+}+function defaultModelFor(provider: string): string {if (provider === "anthropic") return "claude-sonnet-4-5";if (provider === "google") return "gemini-2.5-flash";@@ -177,6 +213,11 @@ function buildModel() {streaming: true,...(OPENAI_BASE_URL ? { configuration: { baseURL: OPENAI_BASE_URL } } : {}),...(USE_RESPONSES_API ? { useResponsesApi: true } : {}),+ /*+ * `reasoning.effort`, not the `reasoningEffort` convenience field: the integration deprecated+ * the latter in favour of merging it into this object, and one of them is the one that survives.+ */+ ...(REASONING_EFFORT ? { reasoning: { effort: REASONING_EFFORT } } : {}),});}@@ -396,8 +437,15 @@ async function runAgent(input: RunAgentInput): Promise<Response> {const chunk = event.data?.chunk as| { content?: unknown }| undefined;- const text =- typeof chunk?.content === "string" ? chunk.content : "";+ /*+ * Both content shapes, because the API decides which one arrives.+ *+ * Chat completions streams a string. The Responses API streams content blocks, so+ * reading only the string shape dropped every delta and the run finished having said+ * nothing — the "no text at all on gpt-5.6-*" this repository documents in+ * `.env.example` and `docker-compose.yml`.+ */+ const text = textOfChunk(chunk?.content);if (!text) continue;if (!textOpen) {diff --git a/agent-langgraph/src/model-options.ts b/agent-langgraph/src/model-options.tsnew file mode 100644index 00000000..56a803b1--- /dev/null+++ b/agent-langgraph/src/model-options.ts@@ -0,0 +1,59 @@+/**+ * Model settings a deployment can turn, checked before the Bot starts.+ *+ * Its own module for the reason `history.ts` is: `index.ts` calls `serve()` at module scope, so+ * importing it to reach one pure function binds a port.+ */++/**+ * The efforts the installed OpenAI API knows, in the order it documents them.+ *+ * Kept as a list rather than described in prose because it is also the error message: an operator+ * who guessed wrong should be told what to write instead, not sent to find the API reference.+ */+export const REASONING_EFFORTS = [+ "none",+ "minimal",+ "low",+ "medium",+ "high",+ "xhigh",+ "max",+] as const;++export type ReasoningEffort = (typeof REASONING_EFFORTS)[number];++/** An effort to ask for, or what is wrong with the one that was configured. */+export interface ReasoningEffortSetting {+ effort?: ReasoningEffort;+ problem?: string;+}++/**+ * How hard this Bot should think, from the environment.+ *+ * Unset means unset: no `reasoning` is sent and the model keeps whatever default its provider+ * chose. An empty string is the same thing, because a compose file passing+ * `BOT_REASONING_EFFORT: ${BOT_REASONING_EFFORT:-}` hands this one — the same trap `BOT_MODEL`+ * already documents.+ *+ * A value the API does not have is a problem rather than a silently dropped setting. That is the+ * whole complaint: configuration that goes nowhere leaves a Bot that starts, looks healthy, and+ * thinks for as long as it likes.+ */+export function readReasoningEffort(+ raw: string | undefined,+): ReasoningEffortSetting {+ const value = raw?.trim().toLowerCase();+ if (!value) return { effort: undefined };++ if ((REASONING_EFFORTS as readonly string[]).includes(value)) {+ return { effort: value as ReasoningEffort };+ }++ return {+ problem:+ `BOT_REASONING_EFFORT=${raw?.trim()} is not an effort this API has. ` ++ `Use one of: ${REASONING_EFFORTS.join(", ")}.`,+ };+}diff --git a/agent-langgraph/tests/deltas.test.ts b/agent-langgraph/tests/deltas.test.tsnew file mode 100644index 00000000..ff973992--- /dev/null+++ b/agent-langgraph/tests/deltas.test.ts@@ -0,0 +1,66 @@+import { describe, expect, test } from "bun:test";+import { textOfChunk } from "../src/deltas";++/**+ * The Responses API streams content blocks, not a string.+ *+ * This is why the framework Bot "answers nothing at all on gpt-5.6-* through the Responses API:+ * RUN_STARTED, then RUN_FINISHED, no text" — the note `.env.example` and `docker-compose.yml` both+ * carry today. The run read `chunk.content` as a string and dropped everything that was not one, and+ * on that API it is never one: `@langchain/openai` converts each `response.output_text.delta` into+ * `[{ type: "text", text: delta, index }]`.+ *+ * Chat completions still hands back a plain string, so both shapes have to work.+ */+describe("text of a streamed chunk", () => {+ test("reads a chat-completions string", () => {+ expect(textOfChunk("Hello")).toBe("Hello");+ });++ test("reads a Responses API text block", () => {+ expect(textOfChunk([{ type: "text", text: "Hello", index: 0 }])).toBe(+ "Hello",+ );+ });++ test("joins the blocks of one chunk in order", () => {+ expect(+ textOfChunk([+ { type: "text", text: "Hel", index: 0 },+ { type: "text", text: "lo", index: 1 },+ ]),+ ).toBe("Hello");+ });++ test("leaves reasoning out of what the person is shown", () => {+ /*+ * A reasoning model streams its summary in the same content array. It is not the answer, and a+ * surface that printed it would be showing the person the Bot's private working — so only text+ * blocks are read, and the block type is what decides, not its position.+ */+ expect(+ textOfChunk([+ { type: "reasoning", reasoning: "the person greeted me, so", index: 0 },+ { type: "text", text: "Hello", index: 1 },+ ]),+ ).toBe("Hello");+ });++ test("ignores blocks carrying no text of their own", () => {+ // Annotations arrive as a text block with an empty string, and a tool call carries no text at+ // all. Neither should open a message on the surface.+ expect(+ textOfChunk([+ { type: "text", text: "", annotations: [{}], index: 0 },+ { type: "tool_call_chunk", index: 1 },+ ]),+ ).toBe("");+ });++ test("says nothing for content it does not recognise", () => {+ expect(textOfChunk(undefined)).toBe("");+ expect(textOfChunk(null)).toBe("");+ expect(textOfChunk(42)).toBe("");+ expect(textOfChunk([{ text: "no type" }])).toBe("");+ });+});diff --git a/agent-langgraph/tests/model-options.test.ts b/agent-langgraph/tests/model-options.test.tsnew file mode 100644index 00000000..aa9f1d31--- /dev/null+++ b/agent-langgraph/tests/model-options.test.ts@@ -0,0 +1,42 @@+import { describe, expect, test } from "bun:test";+import { REASONING_EFFORTS, readReasoningEffort } from "../src/model-options";++/**+ * Reasoning effort, checked where a deployment can still be fixed.+ *+ * The complaint in #212 is that invalid configuration is worse than absent configuration: a value+ * the API does not know is dropped somewhere down the stack, the Bot starts, looks healthy, and+ * thinks for as long as it likes. So an effort this build has not heard of is a refusal at startup,+ * in front of whoever is deploying, the same posture as a missing model key.+ */+describe("reasoning effort from the environment", () => {+ test("unset asks for nothing, so the model keeps its own default", () => {+ expect(readReasoningEffort(undefined)).toEqual({ effort: undefined });+ expect(readReasoningEffort("")).toEqual({ effort: undefined });+ expect(readReasoningEffort(" ")).toEqual({ effort: undefined });+ });++ test("accepts every effort the installed API knows", () => {+ for (const effort of REASONING_EFFORTS) {+ expect(readReasoningEffort(effort)).toEqual({ effort });+ }+ });++ test("takes the value as written in a compose file", () => {+ // Surrounding space and a capital are how this arrives from YAML, not a different setting.+ expect(readReasoningEffort(" High ")).toEqual({ effort: "high" });+ });++ test("refuses an effort the API does not have, and names the ones it does", () => {+ const result = readReasoningEffort("maximum");+ expect(result.effort).toBeUndefined();+ expect(result.problem).toContain("maximum");+ for (const effort of REASONING_EFFORTS) {+ expect(result.problem).toContain(effort);+ }+ });++ test("refuses a number, which is what a first guess at this setting looks like", () => {+ expect(readReasoningEffort("3").problem).toBeDefined();+ });+});diff --git a/docker-compose.yml b/docker-compose.ymlindex e89f4f93..cdfe4e35 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -260,11 +260,12 @@ services:ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-}GOOGLE_API_KEY: ${GOOGLE_API_KEY:-}GOOGLE_GENERATIVE_AI_BASE_URL: ${GOOGLE_GENERATIVE_AI_BASE_URL:-}- # gpt-5.5. This integration answers nothing at all on gpt-5.6-* through the Responses API:- # RUN_STARTED, then RUN_FINISHED, no text. Driven against the real service. 5.6 is still- # reachable by setting BOT_MODEL, and the Responses API is switched on for it automatically.+ # gpt-5.5, to stay comparable with agent-bot above rather than because 5.6 does not work:+ # set BOT_MODEL to one and the Responses API is switched on for it automatically.BOT_MODEL: ${BOT_MODEL:-gpt-5.5}BOT_RESPONSES_API: ${BOT_RESPONSES_API:-false}+ # How hard it thinks, on a model that reasons. Empty leaves the provider's own default.+ BOT_REASONING_EFFORT: ${BOT_REASONING_EFFORT:-}# Where this Bot runs a tool: back through the deployment that granted it, never at the vendor.# `host.docker.internal` because the API server runs on the host, not in this network.OPENBOT_TOOL_URL: ${OPENBOT_TOOL_URL:-http://host.docker.internal:3001/api/agent-tools/call}