Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Lean mode: a site's own message job keeps its delivery receipt

Barack Sokullude11729b39a2
diff --git a/docs/lean-mode.md b/docs/lean-mode.md
index ac2ea9138..a3592a3c0 100644
--- a/docs/lean-mode.md
+++ b/docs/lean-mode.md
@@ -56,7 +56,7 @@ Every guard of class `quality` in the guard registry (`src/*/src/agent/guard-pol
The page reads and model calls that only those guards use are skipped as well:
-- **At `done`:** the verification screenshot and page probes, the completion judge, the page-warning and workflow-evidence checks, the done-summary reading of a failed or partial result, and the forced read-only and done-only turns. `done` is the model's own call.
+- **At `done`:** the verification screenshot and page probes, the completion judge, the page-warning and workflow-evidence checks (except a message job's delivery, below), the done-summary reading of a failed or partial result, and the forced read-only and done-only turns. `done` is the model's own call.
- **Typing:** the readback of text writes. A write the content script cannot read back exactly is reported as `success: true, verified: false, unverifiedWrite: true`, with no note, no lock and no text-write debt, so the next write to the field goes through.
- **Around submits:** the form-validation snapshots before and after a submit and their caption readings, and the GitHub commit gates (the editor-proof binding and the split-write rule).
- **Progress and reading:** the progress ledger (its classification and its checks), the read-coverage scope reading, the arithmetic-task reading, and the readings of upload lines, authored field labels and review-thread controls for workflow evidence.
@@ -81,7 +81,7 @@ Some steps that look like quality work feed the hard checks, so they stay too:
- the social publication contract;
- the rich-text toolbar probe, which pins the typing target (only its classification is skipped);
- automatic screenshots, as set in Settings;
-- a chat message sent in the run: its delivery must be confirmed in the conversation and its receipt saved before `done` reports success, or a resumed run could send it again. A receipt that cannot be saved refuses success every time; a send the conversation cannot confirm refuses it once per send, and `partial` or `failed` always ends the run. A plain final answer gets the same check; after two refusals it ends the run as partial instead of looping.
+- a chat message sent in the run: its delivery must be confirmed in the conversation and its receipt saved before `done` reports success, or a resumed run could send it again. A receipt that cannot be saved refuses success every time; a send the conversation cannot confirm refuses it once per send, and `partial` or `failed` always ends the run. A plain final answer gets the same check; after two refusals it ends the run as partial instead of looping. A site's own message job (a direct message on X, LinkedIn or Douyin, an email in Gmail) is checked the way a full run checks it, from the conversation and the page's sent status, and its receipt is the job's evidence, which a continued run checks before it would send again.
These quality guards keep running, each because a hard check depends on it (`LEAN_KEPT_QUALITY_GUARDS` in `lean-mode.js`):
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 7efe602b6..913347319 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -1499,8 +1499,11 @@ export class Agent extends LoopDetector {
// receipt that cannot be saved refuses every time, as in a full run. A send
// the conversation cannot confirm (its read failed, the conversation
// changed, a new message arrived) refuses once per send, as a full run's
- // submission-evidence check does.
+ // submission-evidence check does. A site's own message job has its own
+ // receipt (_leanWorkflowMessageReceiptBlock).
async _leanChatReceiptBlock(tabId) {
+ const workflowSend = this._leanWorkflowMessageSend(tabId);
+ if (workflowSend) return this._leanWorkflowMessageReceiptBlock(tabId, workflowSend);
if (!this.completionInvariants.get(tabId)?.lastAction?.chatDeliveryBinding
&& !this.chatSessions.get(tabId)?.completionReceipt) return null;
let pageUrl = '';
@@ -1521,9 +1524,9 @@ export class Agent extends LoopDetector {
// done in a lean run. The completion is the model's own call: WebBrain
// takes no verification screenshot, runs no completion or incomplete-task
- // judge and checks no page warnings or workflow evidence. Only the chat
- // receipt check stays (_leanChatReceiptBlock); partial and failed always
- // end.
+ // judge and checks no page warnings or other workflow evidence. Only the
+ // message receipt checks stay (_leanChatReceiptBlock); partial and failed
+ // always end.
async _leanDoneResult(tabId, args = {}) {
const outcome = normalizeDoneOutcome(args?.outcome);
if (outcome === 'success') {
@@ -1553,6 +1556,98 @@ export class Agent extends LoopDetector {
return true;
}
+ // An adapter's own message job (send-message on X, LinkedIn or Douyin,
+ // send-email in Gmail) whose send this task dispatched: its submit receipt,
+ // bound to the job. Null for any other run.
+ _leanWorkflowMessageSend(tabId) {
+ const guard = this._planExecutionGuards.get(tabId);
+ if (!guard?.enabled || !guard.siteWorkflow?.job
+ || this._workflowVerificationKind(guard.siteWorkflow) !== 'message_sent') return null;
+ const submit = this._currentCompletionSubmitState(tabId);
+ const binding = submit?.workflowBinding;
+ if (submit?.dispatched !== true || !binding
+ || binding.bindingKey !== this._adapterWorkflowBindingKey(guard.siteWorkflow)) return null;
+ return { guard, submit, binding };
+ }
+
+ // The receipt check of a lean run for that send. Such a job has no native
+ // conversation receipt: its receipt is the job's terminal evidence, which
+ // a full done records and a continued run checks before it would send
+ // again. So the same verification runs here: the conversation is read
+ // back against the dispatched body and recipients, and the page's live
+ // messages are read in any language for the sent status. The evidence is
+ // recorded and saved before the run ends as a success. As for a chat
+ // send, evidence that cannot be saved refuses every time, and a send that
+ // cannot be confirmed refuses once per send.
+ async _leanWorkflowMessageReceiptBlock(tabId, { guard, submit, binding }) {
+ const save = async () => {
+ const saved = await this._persistNow(tabId);
+ if (saved === true || saved?.ok === true) return null;
+ return { receiptNotDurable: true,
+ error: 'The verified message delivery receipt could not be saved. Retry completion after storage is available; do not send the message again.' };
+ };
+ if (this._workflowTerminalEvidenceMatchesState(guard, guard.workflowTerminalEvidence)) return save();
+ let pageUrl = '';
+ try { pageUrl = await this._currentUrl(tabId); } catch {}
+ const adapterName = guard.siteWorkflow.adapterName;
+ let probe = null;
+ try {
+ probe = await this._messageRecipientGuardProbe(tabId, {
+ tool: 'observe_active_conversation',
+ args: {},
+ adapterName,
+ expectedMessageBody: binding.messageBody || '',
+ expectedRecipients: binding.recipientTargets || [],
+ // Google Chat routes in Gmail have no recipient set of their own.
+ supportsRecipientSets: adapterName === 'gmail' && !isGmailChatRoute(pageUrl),
+ });
+ } catch (error) {
+ if (this._checkAbort(tabId)) throw error;
+ }
+ const pageState = { liveRegionMessages: await this._liveRegionMessages(tabId) };
+ const submissionEvidence = { submit };
+ await this._readWorkflowTerminalTexts(tabId, pageState, submissionEvidence, probe);
+ const current = this._planExecutionGuards.get(tabId) === guard
+ && !this._checkAbort(tabId) && !this._hasUnvalidatedSteering(tabId);
+ const evidence = current && probe
+ ? this._workflowTerminalEvidenceFromDone(tabId, pageState, pageUrl, submissionEvidence, probe) : null;
+ if (evidence) {
+ guard.workflowTerminalEvidence = evidence;
+ guard.verifiedSubmissionEvidence = true;
+ return save();
+ }
+ const key = `workflow:${binding.bindingKey}:${submit.runToken || ''}:${submit.actionSequence ?? ''}`;
+ const refused = (this._guardRunState(tabId).leanUnconfirmedChatSends ||= new Set());
+ if (refused.has(key)) return null;
+ refused.add(key);
+ return { chatDeliveryUnconfirmed: true,
+ error: 'WebBrain could not confirm on the page that your message was delivered: the conversation does not show it sent to the intended recipient with its exact text. Do not send the message again. Read the conversation now, then call done again; if the delivery still cannot be confirmed, finish with outcome "partial" and say so.' };
+ }
+
+ // The page's visible live messages (status, alert and aria-live regions),
+ // collected as the done probe collects them and returned as they are: a
+ // model reads what they say before anything relies on one.
+ async _liveRegionMessages(tabId) {
+ try {
+ const probe = await cdpClient.evaluate(tabId, `(() => {
+ const visible = el => {
+ try {
+ const r = el.getBoundingClientRect();
+ if (r.width < 1 || r.height < 1) return false;
+ const s = window.getComputedStyle(el);
+ return !(s.visibility === 'hidden' || s.display === 'none' || parseFloat(s.opacity) === 0);
+ } catch (e) { return false; }
+ };
+ return Array.from(document.querySelectorAll('[role=status],[role=alert],[aria-live]'))
+ .filter(visible).map(e => (e.innerText || '').trim().slice(0, 120)).filter(Boolean).slice(0, 6);
+ })()`);
+ const value = probe?.result?.value;
+ return Array.isArray(value) ? value.filter(text => typeof text === 'string' && text) : [];
+ } catch {
+ return [];
+ }
+ }
+
_readCompletenessNeedsScopeClassification(tabId) {
// Lean mode enforces no read coverage, so it needs no scope reading.
if (this._leanRunActive(tabId)) return false;
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 69c91d31d..e19c772d6 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -1131,8 +1131,11 @@ export class Agent extends LoopDetector {
// receipt that cannot be saved refuses every time, as in a full run. A send
// the conversation cannot confirm (its read failed, the conversation
// changed, a new message arrived) refuses once per send, as a full run's
- // submission-evidence check does.
+ // submission-evidence check does. A site's own message job has its own
+ // receipt (_leanWorkflowMessageReceiptBlock).
async _leanChatReceiptBlock(tabId) {
+ const workflowSend = this._leanWorkflowMessageSend(tabId);
+ if (workflowSend) return this._leanWorkflowMessageReceiptBlock(tabId, workflowSend);
if (!this.completionInvariants.get(tabId)?.lastAction?.chatDeliveryBinding
&& !this.chatSessions.get(tabId)?.completionReceipt) return null;
let pageUrl = '';
@@ -1153,9 +1156,9 @@ export class Agent extends LoopDetector {
// done in a lean run. The completion is the model's own call: WebBrain
// takes no verification screenshot, runs no completion or incomplete-task
- // judge and checks no page warnings or workflow evidence. Only the chat
- // receipt check stays (_leanChatReceiptBlock); partial and failed always
- // end.
+ // judge and checks no page warnings or other workflow evidence. Only the
+ // message receipt checks stay (_leanChatReceiptBlock); partial and failed
+ // always end.
async _leanDoneResult(tabId, args = {}) {
const outcome = normalizeDoneOutcome(args?.outcome);
if (outcome === 'success') {
@@ -1185,6 +1188,98 @@ export class Agent extends LoopDetector {
return true;
}
+ // An adapter's own message job (send-message on X, LinkedIn or Douyin,
+ // send-email in Gmail) whose send this task dispatched: its submit receipt,
+ // bound to the job. Null for any other run.
+ _leanWorkflowMessageSend(tabId) {
+ const guard = this._planExecutionGuards.get(tabId);
+ if (!guard?.enabled || !guard.siteWorkflow?.job
+ || this._workflowVerificationKind(guard.siteWorkflow) !== 'message_sent') return null;
+ const submit = this._currentCompletionSubmitState(tabId);
+ const binding = submit?.workflowBinding;
+ if (submit?.dispatched !== true || !binding
+ || binding.bindingKey !== this._adapterWorkflowBindingKey(guard.siteWorkflow)) return null;
+ return { guard, submit, binding };
+ }
+
+ // The receipt check of a lean run for that send. Such a job has no native
+ // conversation receipt: its receipt is the job's terminal evidence, which
+ // a full done records and a continued run checks before it would send
+ // again. So the same verification runs here: the conversation is read
+ // back against the dispatched body and recipients, and the page's live
+ // messages are read in any language for the sent status. The evidence is
+ // recorded and saved before the run ends as a success. As for a chat
+ // send, evidence that cannot be saved refuses every time, and a send that
+ // cannot be confirmed refuses once per send.
+ async _leanWorkflowMessageReceiptBlock(tabId, { guard, submit, binding }) {
+ const save = async () => {
+ const saved = await this._persistNow(tabId);
+ if (saved === true || saved?.ok === true) return null;
+ return { receiptNotDurable: true,
+ error: 'The verified message delivery receipt could not be saved. Retry completion after storage is available; do not send the message again.' };
+ };
+ if (this._workflowTerminalEvidenceMatchesState(guard, guard.workflowTerminalEvidence)) return save();
+ let pageUrl = '';
+ try { pageUrl = await this._currentUrl(tabId); } catch {}
+ const adapterName = guard.siteWorkflow.adapterName;
+ let probe = null;
+ try {
+ probe = await this._messageRecipientGuardProbe(tabId, {
+ tool: 'observe_active_conversation',
+ args: {},
+ adapterName,
+ expectedMessageBody: binding.messageBody || '',
+ expectedRecipients: binding.recipientTargets || [],
+ // Google Chat routes in Gmail have no recipient set of their own.
+ supportsRecipientSets: adapterName === 'gmail' && !isGmailChatRoute(pageUrl),
+ });
+ } catch (error) {
+ if (this._checkAbort(tabId)) throw error;
+ }
+ const pageState = { liveRegionMessages: await this._liveRegionMessages(tabId) };
+ const submissionEvidence = { submit };
+ await this._readWorkflowTerminalTexts(tabId, pageState, submissionEvidence, probe);
+ const current = this._planExecutionGuards.get(tabId) === guard
+ && !this._checkAbort(tabId) && !this._hasUnvalidatedSteering(tabId);
+ const evidence = current && probe
+ ? this._workflowTerminalEvidenceFromDone(tabId, pageState, pageUrl, submissionEvidence, probe) : null;
+ if (evidence) {
+ guard.workflowTerminalEvidence = evidence;
+ guard.verifiedSubmissionEvidence = true;
+ return save();
+ }
+ const key = `workflow:${binding.bindingKey}:${submit.runToken || ''}:${submit.actionSequence ?? ''}`;
+ const refused = (this._guardRunState(tabId).leanUnconfirmedChatSends ||= new Set());
+ if (refused.has(key)) return null;
+ refused.add(key);
+ return { chatDeliveryUnconfirmed: true,
+ error: 'WebBrain could not confirm on the page that your message was delivered: the conversation does not show it sent to the intended recipient with its exact text. Do not send the message again. Read the conversation now, then call done again; if the delivery still cannot be confirmed, finish with outcome "partial" and say so.' };
+ }
+
+ // The page's visible live messages (status, alert and aria-live regions),
+ // collected as the done probe collects them and returned as they are: a
+ // model reads what they say before anything relies on one.
+ async _liveRegionMessages(tabId) {
+ try {
+ const results = await browser.tabs.executeScript(tabId, { code: `(() => {
+ const visible = el => {
+ try {
+ const r = el.getBoundingClientRect();
+ if (r.width < 1 || r.height < 1) return false;
+ const s = window.getComputedStyle(el);
+ return !(s.visibility === 'hidden' || s.display === 'none' || parseFloat(s.opacity) === 0);
+ } catch (e) { return false; }
+ };
+ return Array.from(document.querySelectorAll('[role=status],[role=alert],[aria-live]'))
+ .filter(visible).map(e => (e.innerText || '').trim().slice(0, 120)).filter(Boolean).slice(0, 6);
+ })()` });
+ const value = results && results[0];
+ return Array.isArray(value) ? value.filter(text => typeof text === 'string' && text) : [];
+ } catch {
+ return [];
+ }
+ }
+
_readCompletenessNeedsScopeClassification(tabId) {
// Lean mode enforces no read coverage, so it needs no scope reading.
if (this._leanRunActive(tabId)) return false;
diff --git a/test/lean-mode.mjs b/test/lean-mode.mjs
index c181b0ab7..9bd49eace 100644
--- a/test/lean-mode.mjs
+++ b/test/lean-mode.mjs
@@ -12,6 +12,7 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { readFile } from 'node:fs/promises';
import { test } from 'node:test';
+import vm from 'node:vm';
import { chromium } from 'playwright';
import { defaultDecide, installModelStub, stubPageTextReadings, withoutFixedChoiceAnswers } from './semantic-decision-stub.mjs';
@@ -570,6 +571,106 @@ for (const build of ['chrome', 'firefox']) {
assert.ok(reads >= 3, 'the conversation is read again at every success');
});
+ test(`${build}: a lean done after a site's own message send records its delivery as the job's evidence`, async () => {
+ // An X direct message sent through the adapter's send-message job has no
+ // native conversation receipt; its receipt is the job's terminal
+ // evidence, which a continued run checks before it would send again.
+ const xUrl = 'https://x.com/i/chat/123-456';
+ const fixture = ({ delivered = true, saves = true } = {}) => {
+ const agent = new Agent({ getActive: () => STRONG });
+ const tab = nextTab++;
+ agent.conversationModes.set(tab, 'act');
+ agent._runModeOverrides.set(tab, 'act');
+ leanState(agent, tab);
+ agent.conversations.set(tab, [{ role: 'system', content: 'sys' }, { role: 'user', content: 'Send Alex: Hello there' }]);
+ const siteWorkflow = agent._resolvePlannerSiteWorkflow(xUrl, { request_kind: 'execute', requires_submission: true,
+ messaging: { target_kind: 'named', recipients: ['@altryne'] } });
+ assert.equal(siteWorkflow?.job?.id, 'send-message');
+ const guard = agent._startPlanExecutionGuard(tab, 'act', { requestKind: 'execute', requiresStateChange: true,
+ requiresSubmission: true, messaging: { target_kind: 'named', recipients: ['@altryne'] }, siteWorkflow });
+ guard.workflowMetadataRequirements = [{ field: 'body', value: 'Hello there' }];
+ guard.workflowMetadataRequirementsResolved = true;
+ const workflowBinding = agent._workflowSubmitBindingForAttempt(tab, xUrl, {
+ messageRecipientGuardRequired: true, messageRecipientDispatchBinding: { token: 'x-send' },
+ messageRecipientBody: 'Hello there', messageRecipientBodyBaselineCount: 0,
+ messageRecipientExistingMessageIds: ['message-prior'] });
+ assert.ok(workflowBinding, 'the send is bound to the job');
+ agent._completionSubmitStates.set(tab, { dispatched: true, observedAfterSubmit: true, originatingUrl: xUrl,
+ currentUrl: xUrl, taskKey: agent._progressTaskKeyHash(tab), runToken: 'run-1', actionSequence: 2, workflowBinding });
+ agent._currentUrl = async () => xUrl;
+ agent._liveRegionMessages = async () => [];
+ const reads = [];
+ agent._messageRecipientGuardProbe = async (_tab, params) => {
+ reads.push(params);
+ return { success: true, conclusive: true, composerEmpty: true,
+ strongRecipientCandidates: [{ identity: '@altryne', role: 'to' }],
+ existingMessageIds: delivered ? ['message-prior', 'message-new'] : ['message-prior'],
+ matchingOutgoingMessageIds: delivered ? ['message-new'] : [],
+ matchingOutgoingMessageCount: delivered ? 1 : 0 };
+ };
+ let persisted = 0;
+ agent._persistNow = async () => { persisted++; return saves ? { ok: true } : false; };
+ return { agent, tab, guard, reads, persisted: () => persisted };
+ };
+
+ // Delivered: the job's evidence is recorded and saved, and the run ends.
+ const sent = fixture();
+ assert.deepEqual(await sent.agent.executeTool(sent.tab, 'done', { summary: 'Sent.', outcome: 'success' }),
+ { done: true, summary: 'Sent.', outcome: 'success' });
+ assert.equal(sent.reads.length, 1, 'the conversation was read');
+ assert.equal(sent.reads[0].expectedMessageBody, 'Hello there');
+ assert.equal(sent.guard.workflowTerminalEvidence?.verificationKind, 'message_sent');
+ assert.equal(sent.guard.workflowTerminalEvidence?.job, 'send-message');
+ assert.equal(sent.persisted(), 1, 'the receipt was saved before the run ended');
+ assert.equal(sent.agent._workflowTerminalEvidenceMatchesState(sent.guard, sent.guard.workflowTerminalEvidence), true,
+ 'a continued run sees the send as done');
+
+ // Delivered but not saved: refused every time.
+ const unsaved = fixture({ saves: false });
+ for (let attempt = 0; attempt < 3; attempt++) {
+ const refused = await unsaved.agent.executeTool(unsaved.tab, 'done', { summary: 'Sent.', outcome: 'success' });
+ assert.equal(refused.blockedDone, true, JSON.stringify(refused));
+ assert.match(refused.error, /delivery receipt could not be saved/);
+ }
+
+ // Not confirmed: refused once per send, and an honest ending is never refused.
+ const unconfirmed = fixture({ delivered: false });
+ const refused = await unconfirmed.agent.executeTool(unconfirmed.tab, 'done', { summary: 'Sent.', outcome: 'success' });
+ assert.equal(refused.blockedDone, true, JSON.stringify(refused));
+ assert.equal(refused.chatDeliveryUnconfirmed, true);
+ assert.match(refused.error, /Do not send the message again/);
+ assert.equal(unconfirmed.guard.workflowTerminalEvidence, null);
+ assert.deepEqual(await unconfirmed.agent.executeTool(unconfirmed.tab, 'done', { summary: 'Not confirmed.', outcome: 'partial' }),
+ { done: true, summary: 'Not confirmed.', outcome: 'partial' });
+ assert.deepEqual(await unconfirmed.agent.executeTool(unconfirmed.tab, 'done', { summary: 'Sent.', outcome: 'success' }),
+ { done: true, summary: 'Sent.', outcome: 'success' });
+ // The plain-final path shares the check.
+ const plain = fixture({ delivered: false });
+ assert.equal((await plain.agent._leanChatReceiptBlock(plain.tab))?.chatDeliveryUnconfirmed, true);
+ assert.equal(await plain.agent._leanChatReceiptBlock(plain.tab), null);
+ });
+
+ test(`${build}: the live-message probe returns the page's visible status texts`, async () => {
+ const agent = new Agent({ getActive: () => STRONG });
+ const run = code => {
+ const element = (text, width = 10) => ({ innerText: text, getBoundingClientRect: () => ({ width, height: 10 }) });
+ const nodes = [element(' Message sent '), element('Hidden', 0), element('')];
+ // The page's value arrives serialized, as CDP and executeScript return it.
+ return JSON.parse(JSON.stringify(vm.runInNewContext(code, { document: { querySelectorAll: () => nodes },
+ window: { getComputedStyle: () => ({ visibility: 'visible', display: 'block', opacity: '1' }) } })));
+ };
+ if (build === 'chrome') {
+ const { cdpClient } = await import('../src/chrome/src/cdp/cdp-client.js');
+ const original = cdpClient.evaluate;
+ cdpClient.evaluate = async (_tab, expression) => ({ result: { value: run(expression) } });
+ try { assert.deepEqual(await agent._liveRegionMessages(1), ['Message sent']); } finally { cdpClient.evaluate = original; }
+ } else {
+ const original = globalThis.browser.tabs.executeScript;
+ globalThis.browser.tabs.executeScript = async (_tab, { code }) => [run(code)];
+ try { assert.deepEqual(await agent._liveRegionMessages(1), ['Message sent']); } finally { globalThis.browser.tabs.executeScript = original; }
+ }
+ });
+
test(`${build}: in a lean run an unverified CodeMirror write is a plain success with no note or lock`, async () => {
const fileText = 'line one\n\nline three\n';
const editorMeta = { tag: 'div', contentEditable: true, codeMirror: true, ariaLabel: 'Editing README.md file contents' };