Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Report only what was checked: recovery summary, deletion caveat, prompts

Claudef2fb9b75f451
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index c406807fa..64ba18e7b 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -5992,7 +5992,12 @@ export class Agent extends LoopDetector {
add('plan-publication-evidence-missing', false, caveat);
if (githubFile) add('github-committed-file-verification', false, caveat);
} else if (flags.missingRequiredSubmission && !flags.missingSocialPublishTargets.length) {
- add('plan-submission-evidence-missing', false, 'I couldn\'t confirm that the submission went through.');
+ // A deletion is not a "submission" to the user who asked for it.
+ const deletions = Array.isArray(state.resourceMutationDispatches) && state.resourceMutationDispatches.length > 0
+ && state.resourceMutationDispatches.every(attempt => attempt?.mutationKind === 'delete');
+ add('plan-submission-evidence-missing', false, deletions
+ ? 'I couldn\'t confirm that the deletion took effect.'
+ : 'I couldn\'t confirm that the submission went through.');
}
if (flags.missingEvidence && findings.every(finding => ['plan-plain-final-protocol', 'plan-done-summary-shape'].includes(finding.id))) {
add('plan-execution-evidence-missing', false, state.requiresStateChange === false
@@ -7790,9 +7795,12 @@ export class Agent extends LoopDetector {
if (!current() || this._completionDoneBlock(tabId, 'done', { outcome: 'success' })) return null;
const pageUrl = await this._currentUrl(tabId);
if (!current()) return null;
+ // WebBrain writes this answer, not the model, so it claims only what it
+ // checked: the one submission on this page. A request with later steps
+ // (another site, more items) did not get them.
const summary = /^https:\/\/addons\.mozilla\.org\//i.test(pageUrl)
? 'The AMO version was submitted for review and the submission confirmation was verified.'
- : 'The requested submission completed and its result was verified.';
+ : 'The model stopped responding after the submission. WebBrain read the page and verified that submission; no later step of the request was attempted.';
const result = await runReadOnly('done', { outcome: 'success', summary });
if (current() && result?.action === 'return' && !result.status
&& this._completionVerdicts.get(tabId)?.outcome === 'succeeded') {
diff --git a/src/chrome/src/agent/tools.js b/src/chrome/src/agent/tools.js
index e5e3e948e..bf99b0fd5 100644
--- a/src/chrome/src/agent/tools.js
+++ b/src/chrome/src/agent/tools.js
@@ -2221,6 +2221,7 @@ Guidelines:
2. Be conversational and helpful β€” answer in natural language, not raw data dumps.
3. If the user asks you to do something that requires clicking or typing, let them know they need to switch to Act mode.
4. Summarize, analyze, and explain β€” that's your strength in this mode.
+5. Answer only from what you read. When a requested value is missing from it (a page failed to load, the field is not there), say it was not found for that item; never fill it in from memory, even with a caveat.
LISTINGS & PAGINATION β€” read this:
- Listing / search-result pages (URLs with ?page=, ?p=, ?sd=, ?offset=, ?after=, &cursor=; or pages with many product/result cards): EXTRACT first, paginate second.
@@ -2336,7 +2337,7 @@ Guidelines:
2. Break complex tasks into steps. For each step, plan what you need to do BEFORE acting.
3. After performing actions, verify the result by reading the page again and using any injected auto-screenshot/visual context. NEVER assume success β€” confirm it from page state or visual evidence.
4. If something fails, try alternative approaches.
-5. When the task is complete, call the "done" tool with a summary. A verification screenshot is automatically captured β€” review it to confirm the task actually succeeded before reporting completion. If the screenshot shows the task didn't work, do NOT call done β€” fix the issue first.
+5. When the task is complete, call the "done" tool with a summary. A verification screenshot is automatically captured β€” review it to confirm the task actually succeeded before reporting completion. If the screenshot shows the task didn't work, do NOT call done β€” fix the issue first. Report only what you observed: when you check a result, match the exact item (its full title, subject or text), not a similar one, and report a value the pages did not show as not found, never from memory.
6. Be concise in your reasoning but thorough in your actions.
7. Speak naturally β€” explain what you're doing and what you found in plain language.
@@ -2655,6 +2656,7 @@ DEFAULT LOOP:
2. Act with click_ax / set_field / type_ax (ref_ids are stable across calls).
3. Verify: re-read the tree or inspect injected auto-screenshot/visual context. NEVER assume success β€” confirm the page changed.
4. Repeat. When done, call done({summary, outcome:"success"}) after confirming success.
+5. Report only what you observed: match the exact item (its full title, subject or text) when you check a result, and report a value the pages did not show as not found, never from memory.
TYPING:
- For text fields prefer set_field({ref_id, text, submit}) β€” one call that focuses, clears, verifies, and only then optionally submits. Prefer submit:true for search fields. Otherwise type_ax({ref_id, text}) after reading the tree.
@@ -2750,11 +2752,11 @@ TOOLS:
- scroll({direction}), navigate({url}), wait_for_stable(): move around, and let the page settle after an action.
- upload_file: attach the user's file to a file input, using only an attachmentId or downloadId the conversation gave you; never invent a file path.
- clarify({question, options}): ask only when the request is genuinely ambiguous. Put the safe choice first, and set consequence for questions about deleting, money, credentials, the recipient or publishing.
-- done({summary, outcome}): finish. The summary is shown to the user as your answer, so include the actual result. Use "success" only after you checked it, "partial" for useful unfinished work, and "failed" for a blocker.
+- done({summary, outcome}): finish. The summary is shown to the user as your answer, so include the actual result: only what you observed, with a value the pages did not show reported as not found, never from memory. Use "success" only after you checked it, "partial" for useful unfinished work, and "failed" for a blocker.
WORKING LOOP:
1. Read the current page first; the user is on it for a reason.
2. Act on refs from your latest read. You may send several independent actions in one turn, such as the fields of one form; WebBrain stops the rest of the turn when one fails or the page navigates. Read the result before anything that depends on it.
-3. Check the outcome. Finish or close an open dialog before using the page behind it.
+3. Check the outcome against the exact item (its full title, subject or text), not a similar one. Finish or close an open dialog before using the page behind it.
4. If an approach fails twice, change it instead of repeating it.
5. Call done when the request is satisfied or a real blocker stops you.`;
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 88e9fa925..67d619096 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -5539,7 +5539,12 @@ export class Agent extends LoopDetector {
add('plan-publication-evidence-missing', false, caveat);
if (githubFile) add('github-committed-file-verification', false, caveat);
} else if (flags.missingRequiredSubmission && !flags.missingSocialPublishTargets.length) {
- add('plan-submission-evidence-missing', false, 'I couldn\'t confirm that the submission went through.');
+ // A deletion is not a "submission" to the user who asked for it.
+ const deletions = Array.isArray(state.resourceMutationDispatches) && state.resourceMutationDispatches.length > 0
+ && state.resourceMutationDispatches.every(attempt => attempt?.mutationKind === 'delete');
+ add('plan-submission-evidence-missing', false, deletions
+ ? 'I couldn\'t confirm that the deletion took effect.'
+ : 'I couldn\'t confirm that the submission went through.');
}
if (flags.missingEvidence && findings.every(finding => ['plan-plain-final-protocol', 'plan-done-summary-shape'].includes(finding.id))) {
add('plan-execution-evidence-missing', false, state.requiresStateChange === false
@@ -7331,9 +7336,12 @@ export class Agent extends LoopDetector {
if (!current() || this._completionDoneBlock(tabId, 'done', { outcome: 'success' })) return null;
const pageUrl = await this._currentUrl(tabId);
if (!current()) return null;
+ // WebBrain writes this answer, not the model, so it claims only what it
+ // checked: the one submission on this page. A request with later steps
+ // (another site, more items) did not get them.
const summary = /^https:\/\/addons\.mozilla\.org\//i.test(pageUrl)
? 'The AMO version was submitted for review and the submission confirmation was verified.'
- : 'The requested submission completed and its result was verified.';
+ : 'The model stopped responding after the submission. WebBrain read the page and verified that submission; no later step of the request was attempted.';
const result = await runReadOnly('done', { outcome: 'success', summary });
if (current() && result?.action === 'return' && !result.status
&& this._completionVerdicts.get(tabId)?.outcome === 'succeeded') {
diff --git a/src/firefox/src/agent/tools.js b/src/firefox/src/agent/tools.js
index c61e526cf..07308f13b 100644
--- a/src/firefox/src/agent/tools.js
+++ b/src/firefox/src/agent/tools.js
@@ -2107,6 +2107,7 @@ Guidelines:
2. Be conversational and helpful β€” answer in natural language, not raw data dumps.
3. If the user asks you to do something that requires clicking or typing, let them know they need to switch to Act mode.
4. Summarize, analyze, and explain β€” that's your strength in this mode.
+5. Answer only from what you read. When a requested value is missing from it (a page failed to load, the field is not there), say it was not found for that item; never fill it in from memory, even with a caveat.
LISTINGS & PAGINATION β€” read this:
- Listing / search-result pages (URLs with ?page=, ?p=, ?sd=, ?offset=, ?after=, &cursor=; or pages with many product/result cards): EXTRACT first, paginate second.
@@ -2193,7 +2194,7 @@ Guidelines:
2. Break complex tasks into steps. For each step, plan what you need to do BEFORE acting.
3. After performing actions, verify the result by reading the page/tree again and using any injected auto-screenshot/visual context. NEVER assume success β€” confirm it from page state or visual evidence.
4. If something fails, try alternative approaches.
-5. When the task is complete, call the "done" tool with a summary. A verification screenshot is automatically captured β€” review it to confirm the task actually succeeded before reporting completion. If the screenshot shows the task didn't work, do NOT call done β€” fix the issue first.
+5. When the task is complete, call the "done" tool with a summary. A verification screenshot is automatically captured β€” review it to confirm the task actually succeeded before reporting completion. If the screenshot shows the task didn't work, do NOT call done β€” fix the issue first. Report only what you observed: when you check a result, match the exact item (its full title, subject or text), not a similar one, and report a value the pages did not show as not found, never from memory.
6. Be concise in your reasoning but thorough in your actions.
7. Speak naturally β€” explain what you're doing and what you found in plain language.
@@ -2415,6 +2416,7 @@ DEFAULT LOOP:
2. Act with click_ax / set_field / type_ax (ref_ids are stable across calls).
3. Verify: re-read the tree/page or inspect injected auto-screenshot/visual context. NEVER assume success β€” confirm the page changed.
4. Repeat. When done, call done({summary, outcome:"success"}) after confirming success.
+5. Report only what you observed: match the exact item (its full title, subject or text) when you check a result, and report a value the pages did not show as not found, never from memory.
TYPING:
- For text fields prefer set_field({ref_id, text, submit}) β€” one call that focuses, clears, verifies, and only then optionally submits. Prefer submit:true for search fields. Otherwise type_ax({ref_id, text}) after reading the tree.
@@ -2510,11 +2512,11 @@ TOOLS:
- scroll({direction}), navigate({url}), wait_for_stable(): move around, and let the page settle after an action.
- upload_file: attach the user's file to a file input, using only an attachmentId or downloadId the conversation gave you; never invent a file path.
- clarify({question, options}): ask only when the request is genuinely ambiguous. Put the safe choice first, and set consequence for questions about deleting, money, credentials, the recipient or publishing.
-- done({summary, outcome}): finish. The summary is shown to the user as your answer, so include the actual result. Use "success" only after you checked it, "partial" for useful unfinished work, and "failed" for a blocker.
+- done({summary, outcome}): finish. The summary is shown to the user as your answer, so include the actual result: only what you observed, with a value the pages did not show reported as not found, never from memory. Use "success" only after you checked it, "partial" for useful unfinished work, and "failed" for a blocker.
WORKING LOOP:
1. Read the current page first; the user is on it for a reason.
2. Act on refs from your latest read. You may send several independent actions in one turn, such as the fields of one form; WebBrain stops the rest of the turn when one fails or the page navigates. Read the result before anything that depends on it.
-3. Check the outcome. Finish or close an open dialog before using the page behind it.
+3. Check the outcome against the exact item (its full title, subject or text), not a similar one. Finish or close an open dialog before using the page behind it.
4. If an approach fails twice, change it instead of repeating it.
5. Call done when the request is satisfied or a real blocker stops you.`;
diff --git a/test/guard-advisory-completion.mjs b/test/guard-advisory-completion.mjs
index 59238ce66..8cba378cd 100644
--- a/test/guard-advisory-completion.mjs
+++ b/test/guard-advisory-completion.mjs
@@ -1017,6 +1017,13 @@ for (const build of ['chrome', 'firefox']) {
guard.successfulConsequentialToolCalls = 1; guard.successfulTaskToolCalls = 1; guard.evidenceTaskKey = guard.taskKey;
}), { summary: 'Sent the form.', outcome: 'success' }, 'I couldn\'t confirm that the submission went through.', `${build} submission`);
+ // A deletion is described as one, so the caveat never speaks of a
+ // "submission" the user did not make.
+ await refusedOnceThenAccepted(Agent, planCase({ requiresStateChange: true, requiresSubmission: true }, guard => {
+ guard.successfulConsequentialToolCalls = 1; guard.successfulTaskToolCalls = 1; guard.evidenceTaskKey = guard.taskKey;
+ guard.resourceMutationDispatches = [{ mutationKind: 'delete', resourceUrl: 'https://mastoturk.org/@owned/1' }];
+ }), { summary: 'Deleted the post.', outcome: 'success' }, 'I couldn\'t confirm that the deletion took effect.', `${build} deletion`);
+
await refusedOnceThenAccepted(Agent, planCase({ requiresStateChange: true, requiresSubmission: true }, guard => {
guard.siteWorkflow = { adapterName: 'github', job: { id: 'edit-file-and-commit', template: 'publish', requiresSubmission: true } };
guard.successfulConsequentialToolCalls = 1; guard.successfulTaskToolCalls = 1; guard.evidenceTaskKey = guard.taskKey;
diff --git a/test/post-submit-model-recovery.mjs b/test/post-submit-model-recovery.mjs
index d12e4459f..c0e3137d6 100644
--- a/test/post-submit-model-recovery.mjs
+++ b/test/post-submit-model-recovery.mjs
@@ -101,6 +101,15 @@ for (const browser of ['chrome', 'firefox']) {
assert.equal(f.updates.find(x => x.type === 'failure_recovery').data.retryMode, 'resume');
assert.equal(f.updates.at(-1).type, 'error');
});
+ test(`${browser}: the recovered answer claims only the verified submission`, async () => {
+ const f = fixture();
+ f.agent._currentUrl = async () => 'https://mastoturk.org/@owned/117';
+ const result = await f.agent._recoverPostActionModelFailure(f.tab, f.messages, f.update, {}, 4);
+ assert.equal(result.status, 'done');
+ assert.match(result.content, /stopped responding after the submission/);
+ assert.match(result.content, /no later step of the request was attempted/);
+ assert.doesNotMatch(result.content, /requested submission completed/);
+ });
test(`${browser}: recovery uses the submitted frame when present`, async () => {
const f = fixture({ iframe: true });
await f.agent._recoverPostActionModelFailure(f.tab, f.messages, f.update, {}, 4);