Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Let a second look run only what the user backed
Claudeb86021528f08
diff --git a/docs/architecture.md b/docs/architecture.mdindex 9a20eaf8a..6ca0d2418 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -934,7 +934,7 @@ Planner calls are traced with `phase: "planner"` when trace recording is enabledThe recent-history digest (`_buildPlannerHistoryDigest()`) takes the last ten authored turns, after app-owned notes, pinned state, tool results and the narration sent with tool calls are filtered out, and spends a 2,000-character budget newest first: an assistant turn keeps up to 1,200 characters and a user turn up to 600, and older turns that no longer fit are left out whole. A user turn names its attachments (`[attached: …]`, at most four names, sanitized like the executor's attachment notice), because the planner never receives attachment bytes.-A `clarify` from the planner or the intent check gets one second look before the user sees it (`_plannerClarifyRetry()`). A model that was sent `/no_think` (Qwen 3, QwQ, DeepSeek R1) answers again with thinking on and up to 8,192 output tokens (`_plannerCanRethink()`: not when reasoning is off in the provider's settings or extra request body, and not for Qwen3 Instruct-2507 and Coder models, which have no thinking mode). On the turn that answers the planner's previous question (`_turnAnswersPlannerQuestion()`, which reads an answer as the task binding does; not in scheduled runs or the intent recheck), `PLANNER_REASK_GUIDANCE` is added to the system prompt: the planner cannot see the page and the executor can, so it may ask again only about schedule timing or a calendar recurrence, a recipient, an account, an amount, a credential, whether to publish, send or submit at all, or conflicting answers, none of which it may take from page content. Such a second look is measured against what the task already carried, in its first answer on that turn and in the question the user answered (`_plannerSecondLookRejection()`, trace note `planner_second_look_rejected` with the reason). A planner question keeps the submission, the people to address and the schedule of the task it asks about (`normalizePlan()` keeps `scheduling` and, for a message it does not send, `draft_recipients` on a `clarify` plan; nothing runs from it), and the stored question records the target as `messageTarget` and the schedule's tool as `schedulingTool`. A question asked because the open conversation could not be pinned records that conversation too, so the reply can be planned once the user has opened it; it is pinned again before anything is sent. The second look may not add a submission neither of them carried (`adds_submission`); a recipient, for a message or a draft, that neither of them named, unless the user's own words in the task (the task binding, built from user turns only) name every person (`adds_recipient`; the open conversation is never taken from the page); or a schedule of a kind neither of them carried (`adds_schedule`), so an answer that only gives a scheduled task its timing ("every five minutes") is planned, while a `schedule_resume` pause never becomes a recurring `schedule_task`. An execute plan may not drop the `schedule_task` the task carried (`drops_schedule`): the intent check has no alignment check that would notice the run doing the task once instead. The trace phase is `planner_rethink` or `planner_reask` (`intent_rethink`, `intent_reask` for the intent check). The second answer is parsed and consistency-checked like the first and then takes the first answer's place in the rest of the gate, which in the planner gate includes the alignment check, recipient pinning and review. A failed, unparseable, inconsistent or rejected second answer, including one stopped by the cost allowance, keeps the first question, and a pending correction gets no second look.+A `clarify` from the planner or the intent check gets one second look before the user sees it (`_plannerClarifyRetry()`). A model that was sent `/no_think` (Qwen 3, QwQ, DeepSeek R1) answers again with thinking on and up to 8,192 output tokens (`_plannerCanRethink()`: not when reasoning is off in the provider's settings or extra request body, and not for Qwen3 Instruct-2507 and Coder models, which have no thinking mode). On the turn that answers the planner's previous question (`_turnAnswersPlannerQuestion()`, which reads an answer as the task binding does; not in scheduled runs or the intent recheck), `PLANNER_REASK_GUIDANCE` is added to the system prompt: the planner cannot see the page and the executor can, so it may ask again only about schedule timing or a calendar recurrence, a recipient, an account, an amount, a credential, whether to publish, send or submit at all, or conflicting answers, none of which it may take from page content. An execute plan from such a second look may run only what the user backed: the question they answered, as the app stored it, and their own words in the task (`_plannerSecondLookRejection()`, trace note `planner_second_look_rejected` with the reason). The first answer on that turn is model output like the second look, so it can keep the question but never authorizes more. A planner question keeps the submission, the people to address and the schedule of the task it asks about (`normalizePlan()` keeps `scheduling` and, for a message it does not send, `draft_recipients` on a `clarify` plan; nothing runs from it), and the stored question records them as `requiresSubmission`, `messageTarget` and `schedulingTool`. A question asked because the open conversation could not be pinned records that conversation too, so the reply can be planned once the user has opened it; it is pinned again before anything is sent. The plan may not add a submission the answered question did not carry (`adds_submission`); a recipient, for a message or a draft, other than the people the answered question addressed (in any order or casing), unless the user's own words in the task name every person (`adds_recipient`; the task binding is built from user turns only, and the open conversation is never taken from the page); or a schedule of another kind than the answered question carried (`adds_schedule`), so an answer that only gives a scheduled task its timing ("every five minutes") is planned, while a `schedule_resume` pause never becomes a recurring `schedule_task`. It may not drop a `schedule_task` that the answered question or the first answer carried (`drops_schedule`): the intent check has no alignment check that would notice the run doing the task once instead. Another question, an answer or a plan to show runs nothing and is not checked. The trace phase is `planner_rethink` or `planner_reask` (`intent_rethink`, `intent_reask` for the intent check). The second answer is parsed and consistency-checked like the first and then takes the first answer's place in the rest of the gate, which in the planner gate includes the alignment check, recipient pinning and review. A failed, unparseable, inconsistent or rejected second answer, including one stopped by the cost allowance, keeps the first question, and a pending correction gets no second look.LLM-request trace events include privacy-safe prompt provenance: the controlledprompt variant, system-prompt and aggregate message character counts, messagediff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 437608b21..0912135d0 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -27168,19 +27168,23 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// For the planner's second look (_plannerClarifyRetry): whether the user// turn at `index` answers the planner's previous question, and what the- // task carried (_plannerSecondLookRejection): the schedule tool and message- // target of the question it answers, and the user's own words in the task.- // A correction or a scheduled run answers no question.+ // user backed (_plannerSecondLookRejection): the submission, message target+ // and schedule tool that question stored, and the user's own words in the+ // task. A correction or a scheduled run answers no question._plannerQuestionAnswerContext(messages, index, runOptions = {}) {const answersPlannerQuestion = runOptions?.steeringRevision == null&& runOptions?.scheduledRun !== true && runOptions?.scheduledResume !== true&& this._turnAnswersPlannerQuestion(messages, index);if (!answersPlannerQuestion) {- return { answersPlannerQuestion: false, answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '' };+ return {+ answersPlannerQuestion: false, answeredQuestionRequiresSubmission: false,+ answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '',+ };}const question = messages[this._plannerClarificationIndexBefore(messages, index)]?.webbrainPlannerClarification || {};return {answersPlannerQuestion: true,+ answeredQuestionRequiresSubmission: question.requiresSubmission === true,answeredQuestionSchedulingTool: question.schedulingTool || null,answeredQuestionMessageTarget: normalizeMessageTarget(question.messageTarget),// The task binding is built from user turns only: the request and each@@ -27210,6 +27214,19 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d});}+ // Whether two message targets address the same people: the same kind and+ // the same roles and identities, in any order and casing.+ _sameMessageTarget(a, b) {+ const key = value => {+ const target = normalizeMessageTarget(value);+ return target && JSON.stringify([target.target_kind, target.recipients+ .map(recipient => `${recipient.role}:${normalizeRecipientIdentity(recipient.identity)}`)+ .sort()]);+ };+ const left = key(a);+ return !!left && left === key(b);+ }+// Whether a second look may think: only a model planning tells not to// think (/no_think). Reasoning turned off in the provider's settings or in// its extra request body stays off (mergeProviderRequestBody merges both@@ -27226,41 +27243,38 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d.reasoningEffort !== 'off';}- // What a second look told not to re-ask may plan, measured against what- // the user's task already carried: the first answer on this turn (a+ // What a second look told not to re-ask may run, measured against what the+ // user backed: the question they just answered, as the app stored it (a// question keeps the submission, the people to address and the schedule of- // the task it asks about) and the question the user just answered. Returns- // why the plan is rejected, or null. A rejected plan leaves the question- // with the user.- // - adds_submission: a submission neither of them carried.+ // the task it asks about), and their own words in the task. The first+ // answer on this turn is model output like the second look itself: it may+ // keep the question but never authorizes more. Returns why an execute plan+ // is rejected, or null. Another question, an answer or a plan to show runs+ // nothing and passes. A rejected plan leaves the question with the user.+ // - adds_submission: a submission the answered question did not carry.// - adds_recipient: a recipient, for a message or a draft (normalizePlan- // keeps a draft's addressees in draft_recipients), that neither of them- // named, unless the user's own words in the task name every person. The- // open conversation is never taken from the page on a second look.- // - adds_schedule: a schedule of a kind neither of them carried. The user's- // answer is often the timing of a schedule the task carried ("every five- // minutes"), but a schedule_resume pause never becomes a recurring- // schedule_task.- // - drops_schedule: an execute plan without the schedule_task the task- // carried. The intent check has no alignment check that would notice the- // run doing the task once instead.+ // keeps a draft's addressees in draft_recipients), other than the people+ // the answered question addressed, unless the user's own words name every+ // person. The open conversation is never taken from the page.+ // - adds_schedule: a schedule of another kind than the answered question+ // carried. The user's answer is often its timing ("every five minutes"),+ // but a schedule_resume pause never becomes a recurring schedule_task.+ // - drops_schedule: no schedule_task where the answered question or the+ // first answer carried one. The intent check has no alignment check that+ // would notice the run doing the task once instead._plannerSecondLookRejection(firstPlan, plan, followUpContext = {}) {- const submissionCarried = firstPlan?.requires_submission === true- || followUpContext?.plannerClarification?.requiresSubmission === true;- if (plan?.requires_submission === true && !submissionCarried) return 'adds_submission';- const carriedTargets = [firstPlan?.messaging, firstPlan?.draft_recipients, followUpContext?.answeredQuestionMessageTarget]- .filter(Boolean)- .map(target => JSON.stringify(target));- for (const target of [plan?.messaging, plan?.draft_recipients]) {- if (!target || carriedTargets.includes(JSON.stringify(target))) continue;+ if (plan?.request_kind !== 'execute') return null;+ if (plan.requires_submission === true && followUpContext?.answeredQuestionRequiresSubmission !== true) return 'adds_submission';+ for (const target of [plan.messaging, plan.draft_recipients]) {+ if (!target || this._sameMessageTarget(followUpContext?.answeredQuestionMessageTarget, target)) continue;if (target.target_kind === 'named'&& this._userTextNamesRecipients(followUpContext?.taskUserText, target.recipients)) continue;return 'adds_recipient';}- const carriedSchedules = [firstPlan?.scheduling?.tool, followUpContext?.answeredQuestionSchedulingTool];- if (plan?.scheduling && !carriedSchedules.includes(plan.scheduling.tool)) return 'adds_schedule';- if (plan?.request_kind === 'execute' && carriedSchedules.includes('schedule_task')- && plan.scheduling?.tool !== 'schedule_task') return 'drops_schedule';+ const carriedSchedule = followUpContext?.answeredQuestionSchedulingTool || null;+ if (plan.scheduling && plan.scheduling.tool !== carriedSchedule) return 'adds_schedule';+ if (plan.scheduling?.tool !== 'schedule_task'+ && [carriedSchedule, firstPlan?.scheduling?.tool].includes('schedule_task')) return 'drops_schedule';return null;}@@ -27276,9 +27290,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// - No re-asking: when the user has just answered the planner's previous// question, PLANNER_REASK_GUIDANCE reminds the planner that the executor// can see what it cannot, and leaves a question only for values that have- // to come from the user. Its plan may not add a submission, a recipient- // or a schedule the task did not carry, or drop a recurring schedule it- // did (_plannerSecondLookRejection).+ // to come from the user. An execute plan from it may not add a+ // submission, a recipient or a schedule the user did not back, or drop a+ // recurring schedule the task carried (_plannerSecondLookRejection).// The answer is parsed and checked like the first one, and an execute plan// then goes through alignment, recipient pinning and review like any other.// Returns null to keep the first answer: nothing to retry, or the retrydiff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 4dd6a9012..364424ae0 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -24206,19 +24206,23 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// For the planner's second look (_plannerClarifyRetry): whether the user// turn at `index` answers the planner's previous question, and what the- // task carried (_plannerSecondLookRejection): the schedule tool and message- // target of the question it answers, and the user's own words in the task.- // A correction or a scheduled run answers no question.+ // user backed (_plannerSecondLookRejection): the submission, message target+ // and schedule tool that question stored, and the user's own words in the+ // task. A correction or a scheduled run answers no question._plannerQuestionAnswerContext(messages, index, runOptions = {}) {const answersPlannerQuestion = runOptions?.steeringRevision == null&& runOptions?.scheduledRun !== true && runOptions?.scheduledResume !== true&& this._turnAnswersPlannerQuestion(messages, index);if (!answersPlannerQuestion) {- return { answersPlannerQuestion: false, answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '' };+ return {+ answersPlannerQuestion: false, answeredQuestionRequiresSubmission: false,+ answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '',+ };}const question = messages[this._plannerClarificationIndexBefore(messages, index)]?.webbrainPlannerClarification || {};return {answersPlannerQuestion: true,+ answeredQuestionRequiresSubmission: question.requiresSubmission === true,answeredQuestionSchedulingTool: question.schedulingTool || null,answeredQuestionMessageTarget: normalizeMessageTarget(question.messageTarget),// The task binding is built from user turns only: the request and each@@ -24248,6 +24252,19 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d});}+ // Whether two message targets address the same people: the same kind and+ // the same roles and identities, in any order and casing.+ _sameMessageTarget(a, b) {+ const key = value => {+ const target = normalizeMessageTarget(value);+ return target && JSON.stringify([target.target_kind, target.recipients+ .map(recipient => `${recipient.role}:${normalizeRecipientIdentity(recipient.identity)}`)+ .sort()]);+ };+ const left = key(a);+ return !!left && left === key(b);+ }+// Whether a second look may think: only a model planning tells not to// think (/no_think). Reasoning turned off in the provider's settings or in// its extra request body stays off (mergeProviderRequestBody merges both@@ -24264,41 +24281,38 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d.reasoningEffort !== 'off';}- // What a second look told not to re-ask may plan, measured against what- // the user's task already carried: the first answer on this turn (a+ // What a second look told not to re-ask may run, measured against what the+ // user backed: the question they just answered, as the app stored it (a// question keeps the submission, the people to address and the schedule of- // the task it asks about) and the question the user just answered. Returns- // why the plan is rejected, or null. A rejected plan leaves the question- // with the user.- // - adds_submission: a submission neither of them carried.+ // the task it asks about), and their own words in the task. The first+ // answer on this turn is model output like the second look itself: it may+ // keep the question but never authorizes more. Returns why an execute plan+ // is rejected, or null. Another question, an answer or a plan to show runs+ // nothing and passes. A rejected plan leaves the question with the user.+ // - adds_submission: a submission the answered question did not carry.// - adds_recipient: a recipient, for a message or a draft (normalizePlan- // keeps a draft's addressees in draft_recipients), that neither of them- // named, unless the user's own words in the task name every person. The- // open conversation is never taken from the page on a second look.- // - adds_schedule: a schedule of a kind neither of them carried. The user's- // answer is often the timing of a schedule the task carried ("every five- // minutes"), but a schedule_resume pause never becomes a recurring- // schedule_task.- // - drops_schedule: an execute plan without the schedule_task the task- // carried. The intent check has no alignment check that would notice the- // run doing the task once instead.+ // keeps a draft's addressees in draft_recipients), other than the people+ // the answered question addressed, unless the user's own words name every+ // person. The open conversation is never taken from the page.+ // - adds_schedule: a schedule of another kind than the answered question+ // carried. The user's answer is often its timing ("every five minutes"),+ // but a schedule_resume pause never becomes a recurring schedule_task.+ // - drops_schedule: no schedule_task where the answered question or the+ // first answer carried one. The intent check has no alignment check that+ // would notice the run doing the task once instead._plannerSecondLookRejection(firstPlan, plan, followUpContext = {}) {- const submissionCarried = firstPlan?.requires_submission === true- || followUpContext?.plannerClarification?.requiresSubmission === true;- if (plan?.requires_submission === true && !submissionCarried) return 'adds_submission';- const carriedTargets = [firstPlan?.messaging, firstPlan?.draft_recipients, followUpContext?.answeredQuestionMessageTarget]- .filter(Boolean)- .map(target => JSON.stringify(target));- for (const target of [plan?.messaging, plan?.draft_recipients]) {- if (!target || carriedTargets.includes(JSON.stringify(target))) continue;+ if (plan?.request_kind !== 'execute') return null;+ if (plan.requires_submission === true && followUpContext?.answeredQuestionRequiresSubmission !== true) return 'adds_submission';+ for (const target of [plan.messaging, plan.draft_recipients]) {+ if (!target || this._sameMessageTarget(followUpContext?.answeredQuestionMessageTarget, target)) continue;if (target.target_kind === 'named'&& this._userTextNamesRecipients(followUpContext?.taskUserText, target.recipients)) continue;return 'adds_recipient';}- const carriedSchedules = [firstPlan?.scheduling?.tool, followUpContext?.answeredQuestionSchedulingTool];- if (plan?.scheduling && !carriedSchedules.includes(plan.scheduling.tool)) return 'adds_schedule';- if (plan?.request_kind === 'execute' && carriedSchedules.includes('schedule_task')- && plan.scheduling?.tool !== 'schedule_task') return 'drops_schedule';+ const carriedSchedule = followUpContext?.answeredQuestionSchedulingTool || null;+ if (plan.scheduling && plan.scheduling.tool !== carriedSchedule) return 'adds_schedule';+ if (plan.scheduling?.tool !== 'schedule_task'+ && [carriedSchedule, firstPlan?.scheduling?.tool].includes('schedule_task')) return 'drops_schedule';return null;}@@ -24314,9 +24328,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// - No re-asking: when the user has just answered the planner's previous// question, PLANNER_REASK_GUIDANCE reminds the planner that the executor// can see what it cannot, and leaves a question only for values that have- // to come from the user. Its plan may not add a submission, a recipient- // or a schedule the task did not carry, or drop a recurring schedule it- // did (_plannerSecondLookRejection).+ // to come from the user. An execute plan from it may not add a+ // submission, a recipient or a schedule the user did not back, or drop a+ // recurring schedule the task carried (_plannerSecondLookRejection).// The answer is parsed and checked like the first one, and an execute plan// then goes through alignment, recipient pinning and review like any other.// Returns null to keep the first answer: nothing to retry, or the retrydiff --git a/test/planner-clarify-loop.mjs b/test/planner-clarify-loop.mjsindex 780f3dee7..d7c908b4b 100644--- a/test/planner-clarify-loop.mjs+++ b/test/planner-clarify-loop.mjs@@ -94,6 +94,9 @@ const HOW_OFTEN = 'How often should I check this page for a new release?';const PAUSE = { tool: 'schedule_resume', hint: 'Wait until the deploy finishes, then continue.' };const ALI = { target_kind: 'named', recipients: [{ identity: 'Ali Veli', role: 'to' }] };const OPEN_CHAT = { target_kind: 'active_conversation', recipients: [] };+const OPEN_CHAT_CC = { target_kind: 'active_conversation', recipients: [{ identity: 'Ali Veli', role: 'cc' }, { identity: 'Ayse Yilmaz', role: 'cc' }] };+// The question the user answered carried a submission, as the app stored it.+const ASKED = { answeredQuestionRequiresSubmission: true };// The user's words in a task binding: the request and each answer.const userWords = (request, ...answers) => `User task clarified by the latest answer: ${JSON.stringify({ request, answers })}`;@@ -258,12 +261,16 @@ for (const build of ['chrome', 'firefox']) {const invalid = asked({ schedulingTool: 'schedule_everything', messageTarget: { target_kind: 'everyone', recipients: [] } })[2].webbrainPlannerClarification;assert.deepEqual([invalid.schedulingTool, invalid.messageTarget], [null, null], 'only a known tool and a valid target');const context = agent._plannerQuestionAnswerContext(scheduled, 3);- assert.deepEqual([context.answersPlannerQuestion, context.answeredQuestionSchedulingTool, context.answeredQuestionMessageTarget],- [true, 'schedule_task', ALI]);+ assert.deepEqual([context.answersPlannerQuestion, context.answeredQuestionRequiresSubmission, context.answeredQuestionSchedulingTool,+ context.answeredQuestionMessageTarget], [true, false, 'schedule_task', ALI]);+ assert.equal(agent._plannerQuestionAnswerContext(asked({ requiresSubmission: true }), 3).answeredQuestionRequiresSubmission, true);assert.match(context.taskUserText, /Check this page for a new release on a schedule[\s\S]*every five minutes/, 'the user\'s words: request and answer');assert.doesNotMatch(context.taskUserText, /How often/, 'never the question');assert.deepEqual(agent._plannerQuestionAnswerContext(asked({}), 3).answeredQuestionSchedulingTool, null);- const none = { answersPlannerQuestion: false, answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '' };+ const none = {+ answersPlannerQuestion: false, answeredQuestionRequiresSubmission: false,+ answeredQuestionSchedulingTool: null, answeredQuestionMessageTarget: null, taskUserText: '',+ };for (const runOptions of [{ steeringRevision: 1 }, { scheduledRun: true }, { scheduledResume: true }]) {assert.deepEqual(agent._plannerQuestionAnswerContext(scheduled, 3, runOptions), none, JSON.stringify(runOptions));}@@ -297,7 +304,7 @@ for (const build of ['chrome', 'firefox']) {for (const gate of ['planner', 'intent']) {const answers = gate === 'planner' ? [clarifyPlan(), plannerPlan()] : [intentPlan(clarifyPlan()), intentPlan(plannerPlan())];const { agent, calls } = scriptedAgent({ provider: PLAIN_MODEL, answers });- const followUpContext = { answersPlannerQuestion: true };+ const followUpContext = { answersPlannerQuestion: true, ...ASKED };const result = gate === 'planner' ? await fullGate(agent, { followUpContext }) : await intentGate(agent, { followUpContext });assert.equal(result.proceed, true, `${gate}: ${JSON.stringify(result)}`);assert.equal(result.requestKind, 'execute', gate);@@ -347,7 +354,8 @@ for (const build of ['chrome', 'firefox']) {test(`${build}: a no-think model thinks during its second look, and a new plan runs`, async () => {for (const answered of [false, true]) {const { agent, calls } = scriptedAgent({ answers: [clarifyPlan(), plannerPlan()] });- const result = await fullGate(agent, { message: user('Post the same thing on LinkedIn too'), followUpContext: { answersPlannerQuestion: answered } });+ const result = await fullGate(agent, { message: user('Post the same thing on LinkedIn too'),+ followUpContext: { answersPlannerQuestion: answered, answeredQuestionRequiresSubmission: answered } });assert.equal(result.proceed, true, JSON.stringify(result));assert.equal(result.requestKind, 'execute');assert.equal(calls.length, 2);@@ -368,7 +376,7 @@ for (const build of ['chrome', 'firefox']) {await fullGate(first.agent);assert.equal(first.calls.length, 1, 'no second look that could not think anyway');const answered = scriptedAgent({ provider, answers: [clarifyPlan(), plannerPlan()] });- const result = await fullGate(answered.agent, { followUpContext: { answersPlannerQuestion: true } });+ const result = await fullGate(answered.agent, { followUpContext: { answersPlannerQuestion: true, ...ASKED } });assert.equal(result.requestKind, 'execute');const second = answered.calls[1];assert.match(second.messages[0].content, REASK);@@ -376,46 +384,50 @@ for (const build of ['chrome', 'firefox']) {assert.equal(second.options.extraBody?.chat_template_kwargs?.enable_thinking, false);});- test(`${build}: a second look told not to re-ask may not add a submission, a recipient or a schedule the task did not carry`, async () => {+ test(`${build}: a second look told not to re-ask runs only what the user backed`, async () => {const named = ALI;+ const task = { tool: 'schedule_task' };+ // [label, first answer on this turn, second look, what the user backed, accepted]const cases = [['a submission nobody asked for', clarifyPlan({ requires_submission: false }), plannerPlan(), {}, false],- ['a submission the answered question carried', clarifyPlan({ requires_submission: false }), plannerPlan(),- { plannerClarification: { requiresSubmission: true, pageUrl: TWEET_URL } }, true],- ['a submission the first answer carried', clarifyPlan(), plannerPlan(), {}, true],- ['a recipient the first answer did not name', clarifyPlan(), plannerPlan({ messaging: named }), {}, false],- ['the recipient the first answer named', clarifyPlan({ messaging: named }), plannerPlan({ messaging: named }), {}, true],+ ['a submission only the first answer on this turn carried', clarifyPlan(), plannerPlan(), {}, false],+ ['a submission the answered question carried', clarifyPlan({ requires_submission: false }), plannerPlan(), ASKED, true],+ ['a recipient only the first answer on this turn named', clarifyPlan({ messaging: named }), plannerPlan({ messaging: named }), ASKED, false],+ ['the recipient the answered question named', clarifyPlan(), plannerPlan({ messaging: named }), { ...ASKED, answeredQuestionMessageTarget: named }, true],+ ['the people the answered question named, in another order and casing', clarifyPlan(),+ plannerPlan({ messaging: { target_kind: 'active_conversation', recipients: [{ identity: 'AYSE YILMAZ', role: 'cc' }, { identity: 'ali veli', role: 'cc' }] } }),+ { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, true],+ ['someone else in the conversation the answered question addressed', clarifyPlan(),+ plannerPlan({ messaging: { target_kind: 'active_conversation', recipients: [{ identity: 'Ali Veli', role: 'cc' }, { identity: 'Bob', role: 'cc' }] } }),+ { ...ASKED, answeredQuestionMessageTarget: OPEN_CHAT_CC }, false],+ ['the open conversation where the answered question named a person', clarifyPlan(), plannerPlan({ messaging: OPEN_CHAT }),+ { ...ASKED, answeredQuestionMessageTarget: named }, false],['a draft into the open conversation', clarifyPlan({ requires_submission: false }),- plannerPlan({ requires_submission: false, messaging: { target_kind: 'active_conversation', recipients: [] } }), {}, false],- ['a draft for a recipient the first answer did not name', clarifyPlan({ requires_submission: false }),+ plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), {}, false],+ ['a draft for a recipient only the first answer on this turn named', clarifyPlan({ requires_submission: false, messaging: named }),plannerPlan({ requires_submission: false, messaging: named }), {}, false],- ['a draft for the recipient the first answer named', clarifyPlan({ messaging: named }),- plannerPlan({ requires_submission: false, messaging: named }), {}, true],- ['a draft for the recipient of the draft the first answer asked about', clarifyPlan({ requires_submission: false, messaging: named }),- plannerPlan({ requires_submission: false, messaging: named }), {}, true],- ['a recipient the answered question named', clarifyPlan(), plannerPlan({ messaging: named }),- { answeredQuestionMessageTarget: named }, true],+ ['a draft for the recipient the answered question named', clarifyPlan({ requires_submission: false }),+ plannerPlan({ requires_submission: false, messaging: named }), { answeredQuestionMessageTarget: named }, true],['a recipient the user named in the answer', clarifyPlan(), plannerPlan({ messaging: named }),- { taskUserText: userWords('Send the report to my accountant', 'It is Ali Veli') }, true],+ { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'It is Ali Veli') }, true],['a recipient only the page could have named', clarifyPlan(), plannerPlan({ messaging: named }),- { taskUserText: userWords('Send the report to my accountant', 'the one on the screen') }, false],+ { ...ASKED, taskUserText: userWords('Send the report to my accountant', 'the one on the screen') }, false],['the open conversation, whatever the user wrote', clarifyPlan({ requires_submission: false }),plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }),{ taskUserText: userWords('Draft a reply', 'use the current chat') }, false],['the open conversation the answered question addressed', clarifyPlan({ requires_submission: false }),plannerPlan({ requires_submission: false, messaging: OPEN_CHAT }), { answeredQuestionMessageTarget: OPEN_CHAT }, true],- ['a schedule the task did not carry', clarifyPlan(), plannerPlan({ scheduling: { tool: 'schedule_task', hint: 'Every 43200 minutes.' } }), {}, false],- ['a schedule the first answer carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan({ scheduling: EVERY_FIVE }), {}, true],+ ['a schedule the task did not carry', clarifyPlan(), plannerPlan({ scheduling: { ...task, hint: 'Every 43200 minutes.' } }), ASKED, false],+ ['a schedule only the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan({ scheduling: EVERY_FIVE }), ASKED, false],['a schedule the answered question carried', clarifyPlan(), plannerPlan({ scheduling: EVERY_FIVE }),- { answeredQuestionSchedulingTool: 'schedule_task' }, true],- ['a recurring schedule where the first answer carried only a pause', clarifyPlan({ scheduling: PAUSE }),- plannerPlan({ scheduling: EVERY_FIVE }), {}, false],+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, true],['a recurring schedule where the answered question carried only a pause', clarifyPlan(),- plannerPlan({ scheduling: EVERY_FIVE }), { answeredQuestionSchedulingTool: 'schedule_resume' }, false],- ['a run without the recurring schedule the first answer carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan(), {}, false],+ plannerPlan({ scheduling: EVERY_FIVE }), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, false],['a run without the recurring schedule the answered question carried', clarifyPlan(), plannerPlan(),- { answeredQuestionSchedulingTool: 'schedule_task' }, false],- ['a run without the pause the first answer carried', clarifyPlan({ scheduling: PAUSE }), plannerPlan(), {}, true],+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }, false],+ ['a run without the recurring schedule the first answer on this turn carried', clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan(), ASKED, false],+ ['a run without the pause the answered question carried', clarifyPlan(), plannerPlan(),+ { ...ASKED, answeredQuestionSchedulingTool: 'schedule_resume' }, true],];for (const [label, first, second, extraContext, accepted] of cases) {for (const gate of ['planner', 'intent']) {@@ -438,20 +450,37 @@ for (const build of ['chrome', 'firefox']) {assert.equal(result.requiredSchedulingTool, 'schedule_task');});- test(`${build}: a rejected second look names why`, async () => {+ test(`${build}: a rejected second look names why, and only a plan that runs is checked`, async () => {const { parsePlanFromContent } = await import(`../src/${build}/src/agent/planner.js`);const parse = plan => parsePlanFromContent(JSON.stringify(plan), { requireIntent: true, locale: 'en' });const agent = newAgent();- const reason = (first, second, context = {}) => agent._plannerSecondLookRejection(parse(first), parse(second), context);- assert.equal(reason(clarifyPlan({ requires_submission: false }), plannerPlan()), 'adds_submission');- assert.equal(reason(clarifyPlan(), plannerPlan({ messaging: ALI })), 'adds_recipient');- assert.equal(reason(clarifyPlan({ requires_submission: false }), plannerPlan({ requires_submission: false, messaging: ALI })), 'adds_recipient',- 'a draft\'s recipient too');- assert.equal(reason(clarifyPlan(), plannerPlan({ scheduling: EVERY_FIVE })), 'adds_schedule');- assert.equal(reason(clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan()), 'drops_schedule');- assert.equal(reason(clarifyPlan({ scheduling: OPEN_SCHEDULE }), clarifyPlan({ summary: SECOND_QUESTION })), null,- 'another question runs nothing, so it drops nothing');- assert.equal(reason(clarifyPlan({ scheduling: OPEN_SCHEDULE }), plannerPlan({ scheduling: EVERY_FIVE })), null);+ const reason = (second, context = {}, first = clarifyPlan()) => agent._plannerSecondLookRejection(parse(first), parse(second), context);+ assert.equal(reason(plannerPlan()), 'adds_submission', 'a first answer on this turn authorizes nothing');+ assert.equal(reason(plannerPlan({ messaging: ALI }), ASKED, clarifyPlan({ messaging: ALI })), 'adds_recipient');+ assert.equal(reason(plannerPlan({ requires_submission: false, messaging: ALI })), 'adds_recipient', 'a draft\'s recipient too');+ assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE })), 'adds_schedule');+ assert.equal(reason(plannerPlan(), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }), 'drops_schedule');+ assert.equal(reason(plannerPlan(), ASKED, clarifyPlan({ scheduling: OPEN_SCHEDULE })), 'drops_schedule', 'the first answer may keep the question');+ assert.equal(reason(plannerPlan({ scheduling: EVERY_FIVE }), { ...ASKED, answeredQuestionSchedulingTool: 'schedule_task' }), null);+ for (const request_kind of ['clarify', 'respond', 'plan_only']) {+ const second = request_kind === 'clarify'+ ? clarifyPlan({ summary: SECOND_QUESTION, messaging: ALI, scheduling: EVERY_FIVE })+ : plannerPlan({ request_kind, read_scope: 'none', requires_state_change: false, requires_submission: false });+ assert.equal(reason(second), null, `${request_kind} runs nothing`);+ }+ });++ test(`${build}: the same people in another order or casing are the same target`, () => {+ const agent = newAgent();+ const reordered = { target_kind: 'active_conversation', recipients: [{ identity: 'AYSE YILMAZ', role: 'cc' }, { identity: 'ali veli', role: 'cc' }] };+ assert.equal(agent._sameMessageTarget(OPEN_CHAT_CC, reordered), true);+ assert.equal(agent._sameMessageTarget(OPEN_CHAT, { target_kind: 'active_conversation', recipients: [] }), true);+ assert.equal(agent._sameMessageTarget(OPEN_CHAT_CC, { ...reordered, target_kind: 'named' }), false, 'another kind');+ assert.equal(agent._sameMessageTarget(OPEN_CHAT_CC, { ...reordered, recipients: [reordered.recipients[0], { identity: 'ali veli', role: 'to' }] }), false,+ 'another role');+ assert.equal(agent._sameMessageTarget(OPEN_CHAT_CC, { ...reordered, recipients: [reordered.recipients[0]] }), false, 'fewer people');+ assert.equal(agent._sameMessageTarget(null, OPEN_CHAT), false);+ assert.equal(agent._sameMessageTarget(null, null), false);});test(`${build}: only whole names in the user's own words name a recipient`, () => {@@ -500,7 +529,7 @@ for (const build of ['chrome', 'firefox']) {const { agent, calls } = scriptedAgent({answers: [new Error('400 Bad Request: response_format json_schema is not supported'), clarifyPlan(), plannerPlan()],});- const result = await fullGate(agent, { followUpContext: { answersPlannerQuestion: true } });+ const result = await fullGate(agent, { followUpContext: { answersPlannerQuestion: true, ...ASKED } });assert.equal(result.requestKind, 'execute', JSON.stringify(result));assert.equal(calls.length, 3);assert.equal(calls[1].options.extraBody, undefined, 'fixture: the repair went without request controls');