Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Keep a remote Bot answering when it asks for a learned skill that is not there (#663)
kevind12a4cdf1325
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5a67787c..5b7a9a1b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -63,6 +63,14 @@ whoever wrote a skill under that name next, and on a Bot the deployment shares tinstructions answering everybody. It is now refused with "There is no skill called …", as a grantnaming no app already was. Revoking one by hand still works.+### A remote Bot keeps answering when it asks for a learned skill that is not there++With Automatic Learning delivering skills, a Bot reached at an AG-UI endpoint (every shipped Bot+except a built-in one) ended the whole turn with "Skill is unavailable." in place of an answer when+its model asked for a skill by a name the snapshot does not hold, for a file the skill does not+list, or sent arguments that were not JSON. A built-in Bot's model is handed that sentence as the+call's result and carries on. A remote Bot's model now gets the same result and carries on too.+### Browser challenges can be handed to a person without losing the Bot's pageBots pause for actionable browser challenges and resume from a fresh page snapshot after an explicitdiff --git a/server/src/learning/remote.ts b/server/src/learning/remote.tsindex 2d5b2053..4841d764 100644--- a/server/src/learning/remote.ts+++ b/server/src/learning/remote.ts@@ -10,7 +10,7 @@ import {type ToolCall,} from "@ag-ui/client";import { map, Observable, type Subscription } from "rxjs";-import type { AcquireLearnedSkills } from "./runtime";+import type { AcquireLearnedSkills, LearnedSkillInvocation } from "./runtime";const CATALOG_ID = "openbot:learned-skills";const CATALOG_CONTEXT = "OpenBot learned skills";@@ -189,10 +189,7 @@ export class RemoteLearnedSkillsMiddleware extends Middleware {const results = new Map<string, string>();for (const call of reads) {if (subscriber.closed) return;- const content = await invocation.execute(- call.function.name,- JSON.parse(call.function.arguments || "{}"),- );+ const content = await answerRead(invocation, call);if (subscriber.closed) return;const message: Message = {id: crypto.randomUUID(),@@ -276,6 +273,32 @@ export class RemoteLearnedSkillsMiddleware extends Middleware {}}+/**+ * One skill read's result, including a read the snapshot cannot answer.+ *+ * A model asks for a skill by a name the catalog does not hold, or for a file the skill does not+ * list, or sends arguments that are not JSON. A built-in Bot's loop (the AI SDK) hands the thrown+ * message back to the model as that call's result and runs the next step, so the model reads+ * "Skill is unavailable." and carries on. The reason goes back the same way here, rather than+ * ending a remote Bot's whole turn. The snapshot is already in memory, so nothing but the call's+ * own input can make a read fail.+ */+async function answerRead(+ invocation: LearnedSkillInvocation,+ call: ToolCall,+): Promise<string> {+ try {+ return await invocation.execute(+ call.function.name,+ JSON.parse(call.function.arguments || "{}"),+ );+ } catch (error) {+ return error instanceof Error && error.message+ ? error.message+ : "That learned Skill could not be read.";+ }+}+/** AG-UI accumulates from the agent's initial history, not input.messages. */class InvocationTransport extends AbstractAgent {constructor(diff --git a/server/tests/learning-remote.test.ts b/server/tests/learning-remote.test.tsindex b8248ca0..05731d0f 100644--- a/server/tests/learning-remote.test.ts+++ b/server/tests/learning-remote.test.ts@@ -181,6 +181,69 @@ describe("remote Automatic Learning delivery", () => {expect(remote.inputs).toHaveLength(1);});+ /*+ * A built-in Bot's loop (the AI SDK) hands a throwing tool's message back to the model as that+ * call's result and runs the next step, so a model that asks for a skill by a name the snapshot+ * lacks reads "Skill is unavailable." and carries on. A remote Bot has to meet the same answer,+ * not a failed turn.+ */+ test.each([+ ["a skill the snapshot lacks", '{"skill_name":"refunds"}'],+ ["arguments that are not JSON", '{"skill_name":'],+ ])(+ "a read the snapshot cannot answer goes back to the model: %s",+ async (_case, args) => {+ const remote = new Remote((_request, index) =>+ of(+ start,+ ...(index === 0+ ? [+ {+ type: EventType.TOOL_CALL_START,+ toolCallId: "call",+ toolCallName: load.name,+ parentMessageId: "assistant",+ },+ {+ type: EventType.TOOL_CALL_ARGS,+ toolCallId: "call",+ delta: args,+ },+ { type: EventType.TOOL_CALL_END, toolCallId: "call" },+ ]+ : []),+ finish,+ ),+ );+ const invocation: LearnedSkillInvocation = {+ ...snapshot(),+ execute: async (_name, parsed) => {+ if ((parsed as { skill_name?: unknown }).skill_name !== "refund")+ throw new Error("Skill is unavailable.");+ return "refund steps";+ },+ };+ const output = await events(+ new RemoteLearnedSkillsMiddleware("bot", async () => invocation),+ remote,+ );+ expect(output.some((event) => event.type === EventType.RUN_ERROR)).toBe(+ false,+ );+ expect(output.at(-1)?.type).toBe(EventType.RUN_FINISHED);+ // The model is asked again, with the reason on the call it made.+ expect(remote.inputs).toHaveLength(2);+ const answer = remote.inputs[1]?.messages.find(+ (message) => message.role === "tool" && message.toolCallId === "call",+ );+ expect(answer?.content).toBeString();+ expect(answer?.content).not.toBe("");+ expect(+ output.filter((event) => event.type === EventType.TOOL_CALL_RESULT),+ ).toMatchObject([{ toolCallId: "call", content: answer?.content }]);+ },+ );+test("rejects reserved tool collisions before contacting the remote", async () => {const remote = new Remote(() => of(start, finish));const output = await events(