Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) đź§ 

Without an alignment check, let a second look run only what the question carried

Claude5676f9eec33c
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8017d99b3..ee6ab79f1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,7 +19,7 @@ This changelog was generated from the repository Git history and release tags. V
- Providers → Decision accepts a local decision server on another machine on your network, such as `http://192.168.1.51:8091`. **Load available models** and **Test** used to stop with "Decision endpoints require HTTPS outside localhost." for any address but this computer's. Private addresses (10.x, 172.16–31.x, 192.168.x, Tailscale's 100.64–127.x, link-local, IPv6 unique-local) and `.local`, `.home.arpa` and `.internal` names now work over plain HTTP; public addresses still need HTTPS. In Chrome, decision requests to such a server go through the same offscreen proxy as chat models on your network. Vision and speech endpoints already accepted these addresses.
- Act can send a photo or video in an Instagram direct message. Instagram's message box keeps its file input hidden, outside any form, deep in a long conversation, where WebBrain's check that an action still targets what the model saw cannot reach it. Every attachment was refused, even with the exact selector WebBrain itself listed for that input, often right after you had confirmed the recipient on WebBrain's card; the model then sent a link to the picture, or said Instagram blocks attachments. With Strict checks off, an upload to the very input WebBrain listed on that page, named by the selector it gave, now goes through with a note to the model, and the recipient checks still apply. A selector WebBrain did not list is still refused, and so is an input the page re-created after the list under the same selector, until WebBrain lists the page's controls again. When clicking "Add Photo or Video" opens a file chooser, WebBrain now names the input with that same selector, and the Instagram site notes say to attach the file to that input instead of clicking the button, and that a link to a picture is not the picture.
- Act can attach images and videos to a post on X again. While the post composer is open at x.com/compose/post, the timeline behind it keeps a second, identical file input, and WebBrain's list of the page's controls gave neither input a selector. After the model's first selector matched both, WebBrain refused every later upload until it saw a selector it could not get, so the run ended at its cost limit with nothing attached. The list now gives every file input a selector that matches only that input, scoped to the open dialog where that is what tells them apart, and marks the dialog's input `inDialog`. The X site notes name the dialog's input, and when no selector can single out an input, WebBrain tells the model to report the upload as blocked instead of retrying.
-- Act no longer asks you the same question again when the answer is on the page. Asked to post the same thing on LinkedIn while the X post was open, the planner asked which text and files to use, and after “look at the screen — the same” it asked again: the planner works from your words, the page's address and title and a short summary of the chat, never the page itself or your attachments. That summary held only the last ten messages, tool calls and page observations included, each cut to 300 characters, so the earlier answer that named the post's text and your screenshot file had dropped out of it. It now holds the last ten turns of the conversation without the agent's step-by-step notes, gives the latest answer up to 1,200 characters, and names the files you attached earlier. When you have just answered a planner question and the planner wants to ask again, it first takes a second look, told that it cannot see the page while the agent can, and plans the task instead. That second look cannot add anything you did not ask for: posting, sending or a schedule your request did not already carry stays a question for you. When a scheduled task should run, who a message goes to, an account or an amount stay questions too until you give them, and it never takes them from the page. An answer such as "every five minutes" to a question about a scheduled task, the name of the person to write to, opening the conversation it asked you to open or, unless the Planner is off, "yes, send it" is planned, and a task you asked to repeat is never run just once instead. A planner running Qwen 3, QwQ or DeepSeek R1, which is told not to think so that it answers faster, now thinks during that second look and before any question it would ask you, unless reasoning is turned off for that provider. Chrome and Firefox.
+- Act no longer asks you the same question again when the answer is on the page. Asked to post the same thing on LinkedIn while the X post was open, the planner asked which text and files to use, and after “look at the screen — the same” it asked again: the planner works from your words, the page's address and title and a short summary of the chat, never the page itself or your attachments. That summary held only the last ten messages, tool calls and page observations included, each cut to 300 characters, so the earlier answer that named the post's text and your screenshot file had dropped out of it. It now holds the last ten turns of the conversation without the agent's step-by-step notes, gives the latest answer up to 1,200 characters, and names the files you attached earlier. When you have just answered a planner question and the planner wants to ask again, it first takes a second look, told that it cannot see the page while the agent can, and plans the task instead. That second look cannot add anything you did not ask for: posting, sending or a schedule your request did not already carry stays a question for you. When a scheduled task should run, who a message goes to, an account or an amount stay questions too until you give them, and it never takes them from the page. With the Planner on, an answer such as "every five minutes" to a question about a scheduled task, the name of the person to write to or "yes, send it" is planned, because the plan is then checked against your words; with the Planner off, those stay questions. Opening the conversation it asked you to open is planned either way, and a task you asked to repeat is never run just once instead. A planner running Qwen 3, QwQ or DeepSeek R1, which is told not to think so that it answers faster, now thinks during that second look and before any question it would ask you, unless reasoning is turned off for that provider. Chrome and Firefox.
## [42.3.1] - 2026-10-20
diff --git a/docs/architecture.md b/docs/architecture.md
index e6f528612..b2b076c85 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -934,7 +934,7 @@ Planner calls are traced with `phase: "planner"` when trace recording is enabled
The recent-history digest (`_buildPlannerHistoryDigest()`) takes the last ten authored turns, after app-owned notes, pinned state, tool results and the narration sent with tool calls are filtered out, and spends a 2,000-character budget newest first: an assistant turn keeps up to 1,200 characters and a user turn up to 600, and older turns that no longer fit are left out whole. A user turn names its attachments (`[attached: …]`, at most four names, sanitized like the executor's attachment notice), because the planner never receives attachment bytes.
-A `clarify` from the planner or the intent check gets one second look before the user sees it (`_plannerClarifyRetry()`). A model that was sent `/no_think` (Qwen 3, QwQ, DeepSeek R1) answers again with thinking on and up to 8,192 output tokens (`_plannerCanRethink()`: not when reasoning is off in the provider's settings or extra request body, and not for Qwen3 Instruct-2507 and Coder models, which have no thinking mode). On the turn that answers the planner's previous question (`_turnAnswersPlannerQuestion()`, which reads an answer as the task binding does; not in scheduled runs or the intent recheck), `PLANNER_REASK_GUIDANCE` is added to the system prompt: the planner cannot see the page and the executor can, so it may ask again only about schedule timing or a calendar recurrence, a recipient, an account, an amount, a credential, whether to publish, send or submit at all, or conflicting answers, none of which it may take from page content. An execute plan from such a second look may run only what the user backed: the question they answered, as the app stored it, and their own words in the task (`_plannerSecondLookRejection()`, trace note `planner_second_look_rejected` with the reason). The first answer on that turn is model output like the second look, so it can keep the question but never authorizes more. A planner question keeps the submission, the people to address and the schedule of the task it asks about (`normalizePlan()` keeps `scheduling` and, for a message it does not send, `draft_recipients` on a `clarify` plan; nothing runs from it), and the stored question records them as `requiresSubmission`, `messageTarget` and `schedulingTool`. A question asked because the open conversation could not be pinned records that conversation too, so the reply can be planned once the user has opened it; it is pinned again before anything is sent. The plan may not add a submission the answered question did not carry (`adds_submission`), except in the full planner gate, where the alignment check then reads the user's words: an answer such as "yes, send it" may itself be the approval, and the intent check, which has no such check, keeps the question; a recipient, for a message or a draft, other than the people the answered question addressed (in any order or casing), unless the user's own words in the task name every person (`adds_recipient`; the task binding is built from user turns only, and the open conversation is never taken from the page); or a schedule of another kind than the answered question carried (`adds_schedule`), so an answer that only gives a scheduled task its timing ("every five minutes") is planned, while a `schedule_resume` pause never becomes a recurring `schedule_task`. It may not drop a `schedule_task` that the answered question or the first answer carried (`drops_schedule`): the intent check has no alignment check that would notice the run doing the task once instead. Another question, an answer or a plan to show runs nothing and is not checked. A plan the alignment check has the planner replace is held to the same limits, and when it goes beyond them the user keeps the question (`planner_alignment_repair` in the trace note). The trace phase is `planner_rethink` or `planner_reask` (`intent_rethink`, `intent_reask` for the intent check). The second answer is parsed and consistency-checked like the first and then takes the first answer's place in the rest of the gate, which in the planner gate includes the alignment check, recipient pinning and review. A failed, unparseable, inconsistent or rejected second answer, including one stopped by the cost allowance, keeps the first question, and a pending correction gets no second look.
+A `clarify` from the planner or the intent check gets one second look before the user sees it (`_plannerClarifyRetry()`). A model that was sent `/no_think` (Qwen 3, QwQ, DeepSeek R1) answers again with thinking on and up to 8,192 output tokens (`_plannerCanRethink()`: not when reasoning is off in the provider's settings or extra request body, and not for Qwen3 Instruct-2507 and Coder models, which have no thinking mode). On the turn that answers the planner's previous question (`_turnAnswersPlannerQuestion()`, which reads an answer as the task binding does; not in scheduled runs or the intent recheck), `PLANNER_REASK_GUIDANCE` is added to the system prompt: the planner cannot see the page and the executor can, so it may ask again only about schedule timing or a calendar recurrence, a recipient, an account, an amount, a credential, whether to publish, send or submit at all, or conflicting answers, none of which it may take from page content. An execute plan from such a second look may run only what the user backed: the question they answered, as the app stored it, and their own words in the task (`_plannerSecondLookRejection()`, trace note `planner_second_look_rejected` with the reason). The first answer on that turn is model output like the second look, so it can keep the question but never authorizes more. A planner question keeps the submission, the people to address and the schedule of the task it asks about (`normalizePlan()` keeps `scheduling` and, for a message it does not send, `draft_recipients` on a `clarify` plan; nothing runs from it), and the stored question records them as `requiresSubmission`, `messageTarget` and `schedulingTool`. A question asked because the open conversation could not be pinned records that conversation too, so the reply can be planned once the user has opened it; it is pinned again before anything is sent. What the answer itself settles (which person, what timing, whether to send) only an independent reading of the user's words can tell: the full planner gate checks the plan for alignment with them next, while the intent check (Planner off) has no such check, so there a second look may run only what the answered question carried. The plan may not add a submission the answered question did not carry, unless the alignment check follows (`adds_submission`; "yes, send it" may be the approval); a recipient, for a message or a draft, other than the people the answered question addressed (in any order or casing), unless the alignment check follows and the user's own words in the task name every person (`adds_recipient`; the task binding is built from user turns only, and the open conversation is never taken from the page); a schedule of another kind than the answered question carried (`adds_schedule`: a `schedule_resume` pause never becomes a recurring `schedule_task`); or, without the alignment check, any schedule, since only that check reads whether the answer gave its timing (`unchecked_schedule`). With it, an answer that only gives a scheduled task its timing ("every five minutes") is planned. It may not drop a `schedule_task` that the answered question or the first answer carried (`drops_schedule`): the run would do the task once instead. Another question, an answer or a plan to show runs nothing and is not checked. A plan the alignment check has the planner replace is held to the same limits, and when it goes beyond them the user keeps the question (`planner_alignment_repair` in the trace note). The trace phase is `planner_rethink` or `planner_reask` (`intent_rethink`, `intent_reask` for the intent check). The second answer is parsed and consistency-checked like the first and then takes the first answer's place in the rest of the gate, which in the planner gate includes the alignment check, recipient pinning and review. A failed, unparseable, inconsistent or rejected second answer, including one stopped by the cost allowance, keeps the first question, and a pending correction gets no second look.
LLM-request trace events include privacy-safe prompt provenance: the controlled
prompt variant, system-prompt and aggregate message character counts, message
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 1b8d8d634..4782dbde5 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -27251,33 +27251,40 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// keep the question but never authorizes more. Returns why an execute plan
// is rejected, or null. Another question, an answer or a plan to show runs
// nothing and passes. A rejected plan leaves the question with the user.
+ // What the user's answer itself settles (which person, what timing,
+ // whether to send) only an independent reading of their words can tell.
+ // The full planner gate checks the plan for alignment with them next
+ // (alignmentChecks); the intent check (Planner off) has no such check, so
+ // there a second look may run only what the answered question carried.
// - adds_submission: a submission the answered question did not carry,
- // unless the plan is then checked for alignment with the user's words
- // (alignmentChecks, the full planner gate): the answer may itself be the
- // approval ("yes, send it"), and only that independent check can read it.
- // The intent check has none, so there the user keeps the question.
+ // unless the alignment check follows: the answer may be the approval
+ // ("yes, send it").
// - adds_recipient: a recipient, for a message or a draft (normalizePlan
// keeps a draft's addressees in draft_recipients), other than the people
- // the answered question addressed, unless the user's own words name every
- // person. The open conversation is never taken from the page.
+ // the answered question addressed, unless the alignment check follows
+ // and the user's own words name every person. The open conversation is
+ // never taken from the page.
// - adds_schedule: a schedule of another kind than the answered question
- // carried. The user's answer is often its timing ("every five minutes"),
- // but a schedule_resume pause never becomes a recurring schedule_task.
+ // carried: a schedule_resume pause never becomes a recurring
+ // schedule_task.
+ // - unchecked_schedule: a schedule of the kind it carried without the
+ // alignment check, which alone reads whether the answer gave its timing
+ // ("every five minutes", not "I'm not sure yet").
// - drops_schedule: no schedule_task where the answered question or the
- // first answer carried one. The intent check has no alignment check that
- // would notice the run doing the task once instead.
+ // first answer carried one: the run would do the task once instead.
_plannerSecondLookRejection(firstPlan, plan, followUpContext = {}, { alignmentChecks = false } = {}) {
if (plan?.request_kind !== 'execute') return null;
if (plan.requires_submission === true && followUpContext?.answeredQuestionRequiresSubmission !== true
&& !alignmentChecks) return 'adds_submission';
for (const target of [plan.messaging, plan.draft_recipients]) {
if (!target || this._sameMessageTarget(followUpContext?.answeredQuestionMessageTarget, target)) continue;
- if (target.target_kind === 'named'
+ if (alignmentChecks && target.target_kind === 'named'
&& this._userTextNamesRecipients(followUpContext?.taskUserText, target.recipients)) continue;
return 'adds_recipient';
}
const carriedSchedule = followUpContext?.answeredQuestionSchedulingTool || null;
if (plan.scheduling && plan.scheduling.tool !== carriedSchedule) return 'adds_schedule';
+ if (plan.scheduling && !alignmentChecks) return 'unchecked_schedule';
if (plan.scheduling?.tool !== 'schedule_task'
&& [carriedSchedule, firstPlan?.scheduling?.tool].includes('schedule_task')) return 'drops_schedule';
return null;
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index a2bf752cb..15f0051d0 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -24289,33 +24289,40 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// keep the question but never authorizes more. Returns why an execute plan
// is rejected, or null. Another question, an answer or a plan to show runs
// nothing and passes. A rejected plan leaves the question with the user.
+ // What the user's answer itself settles (which person, what timing,
+ // whether to send) only an independent reading of their words can tell.
+ // The full planner gate checks the plan for alignment with them next
+ // (alignmentChecks); the intent check (Planner off) has no such check, so
+ // there a second look may run only what the answered question carried.
// - adds_submission: a submission the answered question did not carry,
- // unless the plan is then checked for alignment with the user's words
- // (alignmentChecks, the full planner gate): the answer may itself be the
- // approval ("yes, send it"), and only that independent check can read it.
- // The intent check has none, so there the user keeps the question.
+ // unless the alignment check follows: the answer may be the approval
+ // ("yes, send it").
// - adds_recipient: a recipient, for a message or a draft (normalizePlan
// keeps a draft's addressees in draft_recipients), other than the people
- // the answered question addressed, unless the user's own words name every
- // person. The open conversation is never taken from the page.
+ // the answered question addressed, unless the alignment check follows
+ // and the user's own words name every person. The open conversation is
+ // never taken from the page.
// - adds_schedule: a schedule of another kind than the answered question
- // carried. The user's answer is often its timing ("every five minutes"),
- // but a schedule_resume pause never becomes a recurring schedule_task.
+ // carried: a schedule_resume pause never becomes a recurring
+ // schedule_task.
+ // - unchecked_schedule: a schedule of the kind it carried without the
+ // alignment check, which alone reads whether the answer gave its timing
+ // ("every five minutes", not "I'm not sure yet").
// - drops_schedule: no schedule_task where the answered question or the
- // first answer carried one. The intent check has no alignment check that
- // would notice the run doing the task once instead.
+ // first answer carried one: the run would do the task once instead.
_plannerSecondLookRejection(firstPlan, plan, followUpContext = {}, { alignmentChecks = false } = {}) {
if (plan?.request_kind !== 'execute') return null;
if (plan.requires_submission === true && followUpContext?.answeredQuestionRequiresSubmission !== true
&& !alignmentChecks) return 'adds_submission';
for (const target of [plan.messaging, plan.draft_recipients]) {
if (!target || this._sameMessageTarget(followUpContext?.answeredQuestionMessageTarget, target)) continue;
- if (target.target_kind === 'named'
+ if (alignmentChecks && target.target_kind === 'named'
&& this._userTextNamesRecipients(followUpContext?.taskUserText, target.recipients)) continue;
return 'adds_recipient';
}
const carriedSchedule = followUpContext?.answeredQuestionSchedulingTool || null;
if (plan.scheduling && plan.scheduling.tool !== carriedSchedule) return 'adds_schedule';
+ if (plan.scheduling && !alignmentChecks) return 'unchecked_schedule';
if (plan.scheduling?.tool !== 'schedule_task'
&& [carriedSchedule, firstPlan?.scheduling?.tool].includes('schedule_task')) return 'drops_schedule';
return null;
diff --git a/test/planner-clarify-loop.mjs b/test/planner-clarify-loop.mjs
index 3a12dcef5..fce4eba96 100644
--- a/test/planner-clarify-loop.mjs
+++ b/test/planner-clarify-loop.mjs
@@ -390,58 +390,68 @@ for (const build of ['chrome', 'firefox']) {
test(`${build}: a second look told not to re-ask runs only what the user backed`, async () => {
const named = ALI;
const task = { tool: 'schedule_task' };
- // [label, first answer on this turn, second look, what the user backed, accepted]
+ // A turn that answers a planner question has history, so in the planner
+ // gate the plan is then checked for alignment with the user's words. The
+ // intent check (Planner off) has no such check.
+ const digest = `User: Post the same thing on LinkedIn too\nAssistant: ${QUESTION}`;
+ // [label, first answer on this turn, second look, what the user backed,
+ // accepted in the planner gate (its alignment check approves), accepted in the intent check]
const cases = [
- ['a submission nobody asked for', clarifyPlan({ requires_submission: false }), plannerPlan(), {}, false],
- ['a submission only the first answer on this turn carried', clarifyPlan(), plannerPlan(), {}, false],
- ['a submission the answered question carried', clarifyPlan({ requires_submission: false }), plannerPlan(), ASKED, true],
- ['a recipient only the first answer on this turn named', clarifyPlan({ messaging: named }), plannerPlan({ messaging: named }), ASKED, false],
- ['the recipient the answered question named', clarifyPlan(), plannerPlan({ messaging: named }), { ...ASKED, answeredQuestionMessageTarget: named }, true],
+ ['a submission nobody asked for', clarifyPlan({ requires_submission: false }), plannerPlan(), {}, true, false],
+ ['a submission only the first answer on this turn carried', clarifyPlan(), plannerPlan(), {}, true, false],
+ ['a submission the answered question carried', clarifyPlan({ requires_submission: false }), plannerPlan(), ASKED, true, true],
+ ['a recipient only the first answer on this turn named', clarifyPlan({ messaging: named }), plannerPlan({ messaging: named }), ASKED, false, false],
+ ['the recipient the answered question named', clarifyPlan(), plannerPlan({ messaging: named }), { ...ASKED, answeredQuestionMessageTarget: named }, true, true],
['the people the answered question named, in another order and casing', clarifyPlan(),
plannerPlan({ messaging: { target_kind: 'active_conversation', recipients: [{ identity: 'AYSE YILMAZ', role: 'cc' }, { identity: 'ali veli', role: 'cc' }] } }),
- { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, true],
+ { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, true, true],
['someone else in the conversation the answered question addressed', clarifyPlan(),
plannerPlan({ messaging: { target_kind: 'active_conversation', recipients: [{ identity: 'Ali Veli', role: 'cc' }, { identity: 'Bob', role: 'cc' }] } }),
- { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, false],
+ { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, false, false],
['the open conversation where the answered question named a person', clarifyPlan(), plannerPlan({ messaging: OPEN_CHAT }),
- { ...ASKED, answeredQuestionMessageTarget: named }, false],
+ { ...ASKED, answeredQuestionMessageTarget: named }, false, false],
['a draft into the open conversation', clarifyPlan({ requires_submission: false }),
- plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), {}, false],
+ plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), {}, false, false],
['a draft for a recipient only the first answer on this turn named', clarifyPlan({ requires_submission: false, messaging: named }),
- plannerPlan({ requires_submission: false, messaging: named }), {}, false],
+ plannerPlan({ requires_submission: false, messaging: named }), {}, false, false],
['a draft for the recipient the answered question named', clarifyPlan({ requires_submission: false }),
- plannerPlan({ requires_submission: false, messaging: named }), { answeredQuestionMessageTarget: named }, true],
+ plannerPlan({ requires_submission: false, messaging: named }), { answeredQuestionMessageTarget: named }, true, true],
['a recipient the user named in the answer', clarifyPlan(), plannerPlan({ messaging: named }),
- { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'It is Ali Veli') }, true],
+ { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'It is Ali Veli') }, true, false],
['a recipient only the page could have named', clarifyPlan(), plannerPlan({ messaging: named }),
- { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'the one on the screen') }, false],
+ { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'the one on the screen') }, false, false],
['the open conversation, whatever the user wrote', clarifyPlan({ requires_submission: false }),
plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }),
- { taskUserText: userWords('Draft a reply', 'use the current chat') }, false],
+ { taskUserText: userWords('Draft a reply', 'use the current chat') }, false, false],
['the open conversation the answered question addressed', clarifyPlan({ requires_submission: false }),
- plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), { answeredQuestionMessageTarget: OPEN_CHAT }, true],
- ['a schedule the task did not carry', clarifyPlan(), plannerPlan({ scheduling: { ...task, hint: 'Every 43200 minutes.' } }), ASKED, false],
- ['a schedule only the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan({ scheduling: EVERY_FIVE }), ASKED, false],
+ plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), { answeredQuestionMessageTarget: OPEN_CHAT }, true, true],
+ ['a schedule the task did not carry', clarifyPlan(), plannerPlan({ scheduling: { ...task, hint: 'Every 43200 minutes.' } }), ASKED, false, false],
+ ['a schedule only the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan({ scheduling: EVERY_FIVE }), ASKED, false, false],
['a schedule the answered question carried', clarifyPlan(), plannerPlan({ scheduling: EVERY_FIVE }),
- { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, true],
+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, true, false],
['a recurring schedule where the answered question carried only a pause', clarifyPlan(),
- plannerPlan({ scheduling: EVERY_FIVE }), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, false],
+ plannerPlan({ scheduling: EVERY_FIVE }), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, false, false],
['a run without the recurring schedule the answered question carried', clarifyPlan(), plannerPlan(),
- { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, false],
- ['a run without the recurring schedule the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan(), ASKED, false],
+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, false, false],
+ ['a run without the recurring schedule the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan(), ASKED, false, false],
['a run without the pause the answered question carried', clarifyPlan(), plannerPlan(),
- { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, true],
+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, true, true],
];
- for (const [label, first, second, extraContext, accepted] of cases) {
+ for (const [label, first, second, extraContext, plannerAccepts, intentAccepts] of cases) {
for (const gate of ['planner', 'intent']) {
+ const accepted = gate === 'planner' ? plannerAccepts : intentAccepts;
const answers = gate === 'planner' ? [first, second] : [intentPlan(first), intentPlan(second)];
- const { agent, calls } = scriptedAgent({ provider: PLAIN_MODEL, answers });
+ const { agent, calls, alignmentChecks } = scriptedAgent({ provider: PLAIN_MODEL, answers });
const followUpContext = { answersPlannerQuestion: true, ...extraContext };
- const result = gate === 'planner' ? await fullGate(agent, { followUpContext }) : await intentGate(agent, { followUpContext });
+ const result = gate === 'planner' ? await fullGate(agent, { followUpContext, digest }) : await intentGate(agent, { followUpContext });
assert.equal(calls.length, 2, `${gate} ${label}`);
- if (accepted) assert.equal(result.requestKind, 'execute', `${gate} ${label}: ${JSON.stringify(result)}`);
- else assert.deepEqual([result.proceed, result.requestKind, result.message], [false, 'clarify', QUESTION], `${gate} ${label}: ${JSON.stringify(result)}`);
- if (accepted && second.scheduling) assert.equal(result.requiredSchedulingTool, 'schedule_task', `${gate} ${label}`);
+ if (accepted) {
+ assert.equal(result.requestKind, 'execute', `${gate} ${label}: ${JSON.stringify(result)}`);
+ if (gate === 'planner') assert.equal(alignmentChecks.length, 1, `${gate} ${label}: checked against the user's words`);
+ if (second.scheduling) assert.equal(result.requiredSchedulingTool, 'schedule_task', `${gate} ${label}`);
+ } else {
+ assert.deepEqual([result.proceed, result.requestKind, result.message], [false, 'clarify', QUESTION], `${gate} ${label}: ${JSON.stringify(result)}`);
+ }
}
}
// A rethink without the guidance is an ordinary plan: a schedule the
@@ -457,18 +467,23 @@ for (const build of ['chrome', 'firefox']) {
const { parsePlanFromContent } = await import(`../src/${build}/src/agent/planner.js`);
const parse = plan => parsePlanFromContent(JSON.stringify(plan), { requireIntent: true, locale: 'en' });
const agent = newAgent();
- const reason = (second, context = {}, first = clarifyPlan()) => agent._plannerSecondLookRejection(parse(first), parse(second), context);
+ const reason = (second, context = {}, first = clarifyPlan(), options = {}) =>
+ agent._plannerSecondLookRejection(parse(first), parse(second), context, options);
+ const checked = { alignmentChecks: true };
assert.equal(reason(plannerPlan()), 'adds_submission', 'a first answer on this turn authorizes nothing');
+ assert.equal(reason(plannerPlan(), {}, clarifyPlan(), checked), null, 'the alignment check reads a send the answer approved');
assert.equal(reason(plannerPlan({ messaging: ALI }), ASKED, clarifyPlan({ messaging: ALI })), 'adds_recipient');
+ const namedByUser = { ...ASKED, taskUserText: userWords('Send it to Alice or Ali Veli, ask me which', 'Ali Veli') };
+ assert.equal(reason(plannerPlan({ messaging: ALI }), namedByUser), 'adds_recipient', 'without the alignment check nothing reads the choice');
+ assert.equal(reason(plannerPlan({ messaging: ALI }), namedByUser, clarifyPlan(), checked), null);
+ assert.equal(reason(plannerPlan({ messaging: ALI }), ASKED, clarifyPlan(), checked), 'adds_recipient', 'the check never stands in for a name');
assert.equal(reason(plannerPlan({ requires_submission: false, messaging: ALI })), 'adds_recipient', 'a draft\'s recipient too');
- assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE })), 'adds_schedule');
- assert.equal(reason(plannerPlan(), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }), 'drops_schedule');
- assert.equal(reason(plannerPlan(), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE })), 'drops_schedule', 'the first answer may keep the question');
- assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }), null);
- assert.equal(agent._plannerSecondLookRejection(parse(clarifyPlan()), parse(plannerPlan()), {}, { alignmentChecks: true }), null,
- 'the alignment check reads a send the answer approved');
- assert.equal(agent._plannerSecondLookRejection(parse(clarifyPlan()), parse(plannerPlan({ messaging: ALI })), {}, { alignmentChecks: true }),
- 'adds_recipient', 'but it never stands in for a recipient');
+ assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE }), checked), 'adds_schedule');
+ const carried = { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' };
+ assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), carried), 'unchecked_schedule', 'without the alignment check nothing reads the timing');
+ assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), carried, clarifyPlan(), checked), null);
+ assert.equal(reason(plannerPlan(), carried, clarifyPlan(), checked), 'drops_schedule');
+ assert.equal(reason(plannerPlan(), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE }), checked), 'drops_schedule', 'the first answer may keep the question');
for (const request_kind of ['clarify', 'respond', 'plan_only']) {
const second = request_kind === 'clarify'
? clarifyPlan({ summary: SECOND_QUESTION, messaging: ALI, scheduling: EVERY_FIVE })