Skip to content

Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.

Answer on Responses API models, and set how hard they think (#212) (#268)

Ethan Stoner70cacb2f8765
diff --git a/.env.example b/.env.example
index dad529df..cb3c4ac6 100644
--- a/.env.example
+++ b/.env.example
@@ -152,15 +152,24 @@ OPENAI_API_KEY=
# GOOGLE_API_KEY=
# Which model the framework Bot uses. Defaults per provider: gpt-5.5, claude-sonnet-4-5,
-# gemini-2.5-flash. Not a 5.6 tier: this integration answers nothing at all on gpt-5.6-* through the
-# Responses API, driven against the real service. Set one here to try it and the Responses API is
-# switched on automatically. The built-in Bots do run 5.6, through the package's model.yaml.
+# gemini-2.5-flash. A 5.6 tier works here: set one and the Responses API is switched on
+# automatically. It used to answer nothing at all on those models — RUN_STARTED, RUN_FINISHED, no
+# text — because that API streams content blocks rather than a string and the run read only the
+# string. The default is left at 5.5 so the two shipped Bots stay comparable out of the box.
# BOT_MODEL=gpt-5.5
# OpenAI only, and rarely needed: the framework Bot turns the Responses API on by itself for models
# that require it. Set it when you are using a model this build has not heard of that needs it too.
# BOT_RESPONSES_API=false
+# How hard the framework Bot thinks, on models that reason. OpenAI's setting, and one the Responses
+# API carries, so it needs a model running on that API — a 5.6 tier does by itself, anything else
+# wants BOT_RESPONSES_API=true. Unset, the model keeps its provider's own default.
+#
+# One of: none, minimal, low, medium, high, xhigh, max. Anything else and the Bot refuses to start
+# and says so, rather than starting with a setting that quietly went nowhere.
+# BOT_REASONING_EFFORT=
+
# The Bot computer. Absent means the feature is off and its routes are not mounted.
AGENT_COMPUTER_URL=http://localhost:4100
# Secret every computer requires from its caller. `agent-computer` drives a browser holding real
diff --git a/CHANGELOG.md b/CHANGELOG.md
index bd28b9a3..09f8b136 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,20 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
## Unreleased
+### The framework Bot answers on 5.6-tier models, and can be told how hard to think
+
+Pointing `BOT_MODEL` at a `gpt-5.6-*` model gave a Bot that started, reported healthy, and then said
+nothing: every run was a RUN_STARTED and a RUN_FINISHED with no text between them. Those models are
+run on the Responses API, which streams content blocks where chat completions streams a string, and
+the run read only the string — so every delta was dropped on the floor. Both shapes are read now.
+Nothing changes for a deployment on 5.5 or on Anthropic or Google.
+
+`BOT_REASONING_EFFORT` sets how hard a reasoning model thinks: `none`, `minimal`, `low`, `medium`,
+`high`, `xhigh` or `max`. Unset, the model keeps its provider's default. A value the API does not
+have, or one set where it cannot be sent — a provider that is not OpenAI, or a model not on the
+Responses API — stops the Bot at startup with a message naming what to change, rather than starting
+with a setting that goes nowhere.
+
### A Bot can be asked to do something on a schedule
"Every weekday at nine, post the standup notes here" is now something a Bot can be asked rather than
diff --git a/agent-langgraph/src/deltas.ts b/agent-langgraph/src/deltas.ts
new file mode 100644
index 00000000..ea197245
--- /dev/null
+++ b/agent-langgraph/src/deltas.ts
@@ -0,0 +1,44 @@
+/**
+ * The prose in a streamed chunk, whichever API produced it.
+ *
+ * Its own module for the reason `history.ts` is: `index.ts` calls `serve()` at module scope, so
+ * importing it to reach one pure function binds a port.
+ *
+ * Chat completions streams `content` as a string. The Responses API does not: `@langchain/openai`
+ * turns every `response.output_text.delta` into a content block — `[{ type: "text", text, index }]` —
+ * and a reasoning model puts its summary in that same array under a different type. Reading the
+ * string shape alone is why a 5.6-tier Bot returned RUN_STARTED and RUN_FINISHED with nothing
+ * between them.
+ */
+
+/** A content block as the integrations produce them. Only the two fields that decide text matter. */
+interface ContentBlock {
+ type?: unknown;
+ text?: unknown;
+}
+
+/**
+ * Text the person should see, and nothing else.
+ *
+ * Selected by block type rather than by "has a text field": reasoning summaries are the Bot's
+ * private working, and a surface printing them would be showing the person something never meant
+ * for them.
+ */
+export function textOfChunk(content: unknown): string {
+ if (typeof content === "string") return content;
+ if (!Array.isArray(content)) return "";
+
+ let text = "";
+ for (const part of content) {
+ if (typeof part === "string") {
+ text += part;
+ continue;
+ }
+ if (!part || typeof part !== "object") continue;
+ const block = part as ContentBlock;
+ if (block.type === "text" && typeof block.text === "string") {
+ text += block.text;
+ }
+ }
+ return text;
+}
diff --git a/agent-langgraph/src/index.ts b/agent-langgraph/src/index.ts
index b1302a46..58a286b4 100644
--- a/agent-langgraph/src/index.ts
+++ b/agent-langgraph/src/index.ts
@@ -12,7 +12,9 @@ import {
import { ChatOpenAI } from "@langchain/openai";
import { serve } from "bun";
import { hasManagedAgentToken } from "../../shared/agent-authorisation";
+import { textOfChunk } from "./deltas";
import { toLangChainMessages } from "./history";
+import { readReasoningEffort } from "./model-options";
/**
* The same Bot, on a framework.
@@ -97,6 +99,40 @@ const ANTHROPIC_BASE_URL = process.env.ANTHROPIC_BASE_URL?.trim() || undefined;
const GOOGLE_BASE_URL =
process.env.GOOGLE_GENERATIVE_AI_BASE_URL?.trim() || undefined;
+/**
+ * OpenAI only, and Responses API only: how hard this Bot is allowed to think.
+ *
+ * Checked here rather than sent onward and forgotten. An effort the API does not have is dropped
+ * somewhere down the stack, and a Bot that starts, looks healthy and then thinks for as long as it
+ * likes is a worse outcome than one that refuses to start and says why.
+ */
+const { effort: REASONING_EFFORT, problem: REASONING_PROBLEM } =
+ readReasoningEffort(process.env.BOT_REASONING_EFFORT);
+if (REASONING_PROBLEM) {
+ console.error(REASONING_PROBLEM);
+ process.exit(1);
+}
+/*
+ * The two ways this setting would reach an API with nowhere to put it.
+ *
+ * Anthropic and Google express thinking budgets differently, and on `/v1/chat/completions` the
+ * field does not exist at all. Refusing is the same call the issue makes about invalid values:
+ * configuration that goes nowhere is worse than configuration that is absent, because the Bot looks
+ * configured either way. Both messages name the variable that would make it work.
+ */
+if (REASONING_EFFORT && PROVIDER !== "openai") {
+ console.error(
+ `BOT_REASONING_EFFORT is OpenAI's setting, and BOT_PROVIDER=${PROVIDER}. Unset it, or set BOT_PROVIDER=openai.`,
+ );
+ process.exit(1);
+}
+if (REASONING_EFFORT && !USE_RESPONSES_API) {
+ console.error(
+ `BOT_REASONING_EFFORT needs the Responses API, and BOT_MODEL=${MODEL} is not being run on it. Use a model that requires it, or set BOT_RESPONSES_API=true.`,
+ );
+ process.exit(1);
+}
+
function defaultModelFor(provider: string): string {
if (provider === "anthropic") return "claude-sonnet-4-5";
if (provider === "google") return "gemini-2.5-flash";
@@ -177,6 +213,11 @@ function buildModel() {
streaming: true,
...(OPENAI_BASE_URL ? { configuration: { baseURL: OPENAI_BASE_URL } } : {}),
...(USE_RESPONSES_API ? { useResponsesApi: true } : {}),
+ /*
+ * `reasoning.effort`, not the `reasoningEffort` convenience field: the integration deprecated
+ * the latter in favour of merging it into this object, and one of them is the one that survives.
+ */
+ ...(REASONING_EFFORT ? { reasoning: { effort: REASONING_EFFORT } } : {}),
});
}
@@ -396,8 +437,15 @@ async function runAgent(input: RunAgentInput): Promise<Response> {
const chunk = event.data?.chunk as
| { content?: unknown }
| undefined;
- const text =
- typeof chunk?.content === "string" ? chunk.content : "";
+ /*
+ * Both content shapes, because the API decides which one arrives.
+ *
+ * Chat completions streams a string. The Responses API streams content blocks, so
+ * reading only the string shape dropped every delta and the run finished having said
+ * nothing — the "no text at all on gpt-5.6-*" this repository documents in
+ * `.env.example` and `docker-compose.yml`.
+ */
+ const text = textOfChunk(chunk?.content);
if (!text) continue;
if (!textOpen) {
diff --git a/agent-langgraph/src/model-options.ts b/agent-langgraph/src/model-options.ts
new file mode 100644
index 00000000..56a803b1
--- /dev/null
+++ b/agent-langgraph/src/model-options.ts
@@ -0,0 +1,59 @@
+/**
+ * Model settings a deployment can turn, checked before the Bot starts.
+ *
+ * Its own module for the reason `history.ts` is: `index.ts` calls `serve()` at module scope, so
+ * importing it to reach one pure function binds a port.
+ */
+
+/**
+ * The efforts the installed OpenAI API knows, in the order it documents them.
+ *
+ * Kept as a list rather than described in prose because it is also the error message: an operator
+ * who guessed wrong should be told what to write instead, not sent to find the API reference.
+ */
+export const REASONING_EFFORTS = [
+ "none",
+ "minimal",
+ "low",
+ "medium",
+ "high",
+ "xhigh",
+ "max",
+] as const;
+
+export type ReasoningEffort = (typeof REASONING_EFFORTS)[number];
+
+/** An effort to ask for, or what is wrong with the one that was configured. */
+export interface ReasoningEffortSetting {
+ effort?: ReasoningEffort;
+ problem?: string;
+}
+
+/**
+ * How hard this Bot should think, from the environment.
+ *
+ * Unset means unset: no `reasoning` is sent and the model keeps whatever default its provider
+ * chose. An empty string is the same thing, because a compose file passing
+ * `BOT_REASONING_EFFORT: ${BOT_REASONING_EFFORT:-}` hands this one — the same trap `BOT_MODEL`
+ * already documents.
+ *
+ * A value the API does not have is a problem rather than a silently dropped setting. That is the
+ * whole complaint: configuration that goes nowhere leaves a Bot that starts, looks healthy, and
+ * thinks for as long as it likes.
+ */
+export function readReasoningEffort(
+ raw: string | undefined,
+): ReasoningEffortSetting {
+ const value = raw?.trim().toLowerCase();
+ if (!value) return { effort: undefined };
+
+ if ((REASONING_EFFORTS as readonly string[]).includes(value)) {
+ return { effort: value as ReasoningEffort };
+ }
+
+ return {
+ problem:
+ `BOT_REASONING_EFFORT=${raw?.trim()} is not an effort this API has. ` +
+ `Use one of: ${REASONING_EFFORTS.join(", ")}.`,
+ };
+}
diff --git a/agent-langgraph/tests/deltas.test.ts b/agent-langgraph/tests/deltas.test.ts
new file mode 100644
index 00000000..ff973992
--- /dev/null
+++ b/agent-langgraph/tests/deltas.test.ts
@@ -0,0 +1,66 @@
+import { describe, expect, test } from "bun:test";
+import { textOfChunk } from "../src/deltas";
+
+/**
+ * The Responses API streams content blocks, not a string.
+ *
+ * This is why the framework Bot "answers nothing at all on gpt-5.6-* through the Responses API:
+ * RUN_STARTED, then RUN_FINISHED, no text" — the note `.env.example` and `docker-compose.yml` both
+ * carry today. The run read `chunk.content` as a string and dropped everything that was not one, and
+ * on that API it is never one: `@langchain/openai` converts each `response.output_text.delta` into
+ * `[{ type: "text", text: delta, index }]`.
+ *
+ * Chat completions still hands back a plain string, so both shapes have to work.
+ */
+describe("text of a streamed chunk", () => {
+ test("reads a chat-completions string", () => {
+ expect(textOfChunk("Hello")).toBe("Hello");
+ });
+
+ test("reads a Responses API text block", () => {
+ expect(textOfChunk([{ type: "text", text: "Hello", index: 0 }])).toBe(
+ "Hello",
+ );
+ });
+
+ test("joins the blocks of one chunk in order", () => {
+ expect(
+ textOfChunk([
+ { type: "text", text: "Hel", index: 0 },
+ { type: "text", text: "lo", index: 1 },
+ ]),
+ ).toBe("Hello");
+ });
+
+ test("leaves reasoning out of what the person is shown", () => {
+ /*
+ * A reasoning model streams its summary in the same content array. It is not the answer, and a
+ * surface that printed it would be showing the person the Bot's private working — so only text
+ * blocks are read, and the block type is what decides, not its position.
+ */
+ expect(
+ textOfChunk([
+ { type: "reasoning", reasoning: "the person greeted me, so", index: 0 },
+ { type: "text", text: "Hello", index: 1 },
+ ]),
+ ).toBe("Hello");
+ });
+
+ test("ignores blocks carrying no text of their own", () => {
+ // Annotations arrive as a text block with an empty string, and a tool call carries no text at
+ // all. Neither should open a message on the surface.
+ expect(
+ textOfChunk([
+ { type: "text", text: "", annotations: [{}], index: 0 },
+ { type: "tool_call_chunk", index: 1 },
+ ]),
+ ).toBe("");
+ });
+
+ test("says nothing for content it does not recognise", () => {
+ expect(textOfChunk(undefined)).toBe("");
+ expect(textOfChunk(null)).toBe("");
+ expect(textOfChunk(42)).toBe("");
+ expect(textOfChunk([{ text: "no type" }])).toBe("");
+ });
+});
diff --git a/agent-langgraph/tests/model-options.test.ts b/agent-langgraph/tests/model-options.test.ts
new file mode 100644
index 00000000..aa9f1d31
--- /dev/null
+++ b/agent-langgraph/tests/model-options.test.ts
@@ -0,0 +1,42 @@
+import { describe, expect, test } from "bun:test";
+import { REASONING_EFFORTS, readReasoningEffort } from "../src/model-options";
+
+/**
+ * Reasoning effort, checked where a deployment can still be fixed.
+ *
+ * The complaint in #212 is that invalid configuration is worse than absent configuration: a value
+ * the API does not know is dropped somewhere down the stack, the Bot starts, looks healthy, and
+ * thinks for as long as it likes. So an effort this build has not heard of is a refusal at startup,
+ * in front of whoever is deploying, the same posture as a missing model key.
+ */
+describe("reasoning effort from the environment", () => {
+ test("unset asks for nothing, so the model keeps its own default", () => {
+ expect(readReasoningEffort(undefined)).toEqual({ effort: undefined });
+ expect(readReasoningEffort("")).toEqual({ effort: undefined });
+ expect(readReasoningEffort(" ")).toEqual({ effort: undefined });
+ });
+
+ test("accepts every effort the installed API knows", () => {
+ for (const effort of REASONING_EFFORTS) {
+ expect(readReasoningEffort(effort)).toEqual({ effort });
+ }
+ });
+
+ test("takes the value as written in a compose file", () => {
+ // Surrounding space and a capital are how this arrives from YAML, not a different setting.
+ expect(readReasoningEffort(" High ")).toEqual({ effort: "high" });
+ });
+
+ test("refuses an effort the API does not have, and names the ones it does", () => {
+ const result = readReasoningEffort("maximum");
+ expect(result.effort).toBeUndefined();
+ expect(result.problem).toContain("maximum");
+ for (const effort of REASONING_EFFORTS) {
+ expect(result.problem).toContain(effort);
+ }
+ });
+
+ test("refuses a number, which is what a first guess at this setting looks like", () => {
+ expect(readReasoningEffort("3").problem).toBeDefined();
+ });
+});
diff --git a/docker-compose.yml b/docker-compose.yml
index e89f4f93..cdfe4e35 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -260,11 +260,12 @@ services:
ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-}
GOOGLE_API_KEY: ${GOOGLE_API_KEY:-}
GOOGLE_GENERATIVE_AI_BASE_URL: ${GOOGLE_GENERATIVE_AI_BASE_URL:-}
- # gpt-5.5. This integration answers nothing at all on gpt-5.6-* through the Responses API:
- # RUN_STARTED, then RUN_FINISHED, no text. Driven against the real service. 5.6 is still
- # reachable by setting BOT_MODEL, and the Responses API is switched on for it automatically.
+ # gpt-5.5, to stay comparable with agent-bot above rather than because 5.6 does not work:
+ # set BOT_MODEL to one and the Responses API is switched on for it automatically.
BOT_MODEL: ${BOT_MODEL:-gpt-5.5}
BOT_RESPONSES_API: ${BOT_RESPONSES_API:-false}
+ # How hard it thinks, on a model that reasons. Empty leaves the provider's own default.
+ BOT_REASONING_EFFORT: ${BOT_REASONING_EFFORT:-}
# Where this Bot runs a tool: back through the deployment that granted it, never at the vendor.
# `host.docker.internal` because the API server runs on the host, not in this network.
OPENBOT_TOOL_URL: ${OPENBOT_TOOL_URL:-http://host.docker.internal:3001/api/agent-tools/call}