Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Placeholder guard: scope write receipts by task, read appends as results
Claude86ff5d201b01
diff --git a/docs/security-model.md b/docs/security-model.mdindex 5db8acb1b..3aeb8027d 100644--- a/docs/security-model.md+++ b/docs/security-model.md@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. || **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. || **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |-| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field (by its ref, or its name and id) in this run. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |+| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field (by its ref, or its name and id) in this task, with tracing on or off. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. || **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. || **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. || **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 7a2a7b4d2..58852fde5 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -33007,7 +33007,11 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// too, for the agent's own comparisons.const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const placeholder = whole(el.getAttribute?.('placeholder'));- const value = whole(pendingEl && el === pendingEl ? pendingValue : el.value);+ // set_field({submit, clear:false}) appends to what the field holds.+ const typed = pendingEl && el === pendingEl+ ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')+ : el.value;+ const value = whole(typed);const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));return placeholder && value.toLowerCase() === placeholder.toLowerCase()? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,@@ -36352,6 +36356,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// A cleared conversation or closed tab takes its filled credentials along.this._knownCredentialValues?.delete(tabId);this._restoredCredentialScrubs?.delete(tabId);+ this._placeholderWrites?.delete(tabId);// A conversation clear preserves the run guard while the document (and// possibly landed text) is unchanged: document-scoped mutation debt and// its recent-document backlog survive, or the next run could duplicate@@ -52461,7 +52466,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// when the user did not give that value as an answer on one of the// current task's cards: the page's example typed in as the user's. A// field counts as edited when the probe saw it differ from its default or- // WebBrain's own write put that value into that same field in this run (a+ // WebBrain's own write put that value into that same field in this task (a// framework may copy a typed value into defaultValue). The probe's// complete list is read when it sent one; the summary's changed fields// stop at eight. A placeholder without a letter, or shorter than three@@ -52500,8 +52505,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d}// A value WebBrain typed into a field that equals the field's own- // placeholder, from the write's receipt, kept for the current run with the- // field it went into: its accessibility ref, and its name and id.+ // placeholder, from the write's receipt, kept for the current task with the+ // field it went into: its accessibility ref, and its name and id. The+ // task token scopes it, as it does task proofs: a trace run ID is missing+ // when tracing is off, and a trusted continuation keeps the token._rememberPlaceholderWrite(tabId, toolName, args, response) {if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;const meta = response.fieldMeta;@@ -52509,28 +52516,34 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const normalize = text => whole(text).toLowerCase();const placeholder = normalize(meta.placeholder);- // The typed text, whole; an append (clear:false) reads the field's- // result instead, which the receipt reports shortened.- const appended = args?.clear === false && typeof response.value === 'string';- const typed = whole(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));+ // The typed text, whole, when the write replaced the value: set_field+ // replaces it unless clear is false, the typing tools only with+ // clear:true. An append reads the field's result instead, which the+ // receipt reports shortened.+ const result = this._textMutationReplacesValue(toolName, args || {}) && typeof args?.text === 'string'+ ? args.text : response.value;+ if (typeof result !== 'string') return;+ const typed = whole(result);if (!placeholder || normalize(typed) !== placeholder) return;const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';// Kept as the probe returns a whole value: its first 4,000 characters,// cut before the case is folded, as the probe cuts it.const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };if (!write.ref && !write.name && !write.id) return;- const runId = this.currentRunId.get(tabId) || null;+ const task = this._taskTokens.get(tabId);+ if (!task) return;this._placeholderWrites ??= new Map();const entry = this._placeholderWrites.get(tabId);- const writes = entry?.runId === runId ? entry.writes : [];- this._placeholderWrites.set(tabId, { runId, writes: [...writes, write].slice(-64) });+ const writes = entry?.task === task ? entry.writes : [];+ this._placeholderWrites.set(tabId, { task, writes: [...writes, write].slice(-64) });}- // Whether WebBrain's own write in this run put `value` into this probed+ // Whether WebBrain's own write in this task put `value` into this probed// field: the same ref when both have one, otherwise the same name and id._placeholderWritten(tabId, field, value) {const entry = this._placeholderWrites?.get(tabId);- if (!entry || entry.runId !== (this.currentRunId.get(tabId) || null)) return false;+ const task = this._taskTokens.get(tabId);+ if (!entry || !task || entry.task !== task) return false;const ref = String(field?.ref || '');const name = String(field?.name || '');const id = String(field?.id || '');diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 31463ebee..971ee1ee8 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -29911,7 +29911,11 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// too, for the agent's own comparisons.const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const placeholder = whole(el.getAttribute?.('placeholder'));- const value = whole(pendingEl && el === pendingEl ? pendingValue : el.value);+ // set_field({submit, clear:false}) appends to what the field holds.+ const typed = pendingEl && el === pendingEl+ ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')+ : el.value;+ const value = whole(typed);const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));return placeholder && value.toLowerCase() === placeholder.toLowerCase()? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,@@ -34370,6 +34374,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// A cleared conversation or closed tab takes its filled credentials along.this._knownCredentialValues?.delete(tabId);this._restoredCredentialScrubs?.delete(tabId);+ this._placeholderWrites?.delete(tabId);// A conversation clear preserves the run guard while the document (and// possibly landed text) is unchanged: document-scoped mutation debt and// its recent-document backlog survive, or the next run could duplicate@@ -45731,7 +45736,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d// when the user did not give that value as an answer on one of the// current task's cards: the page's example typed in as the user's. A// field counts as edited when the probe saw it differ from its default or- // WebBrain's own write put that value into that same field in this run (a+ // WebBrain's own write put that value into that same field in this task (a// framework may copy a typed value into defaultValue). The probe's// complete list is read when it sent one; the summary's changed fields// stop at eight. A placeholder without a letter, or shorter than three@@ -45770,8 +45775,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d}// A value WebBrain typed into a field that equals the field's own- // placeholder, from the write's receipt, kept for the current run with the- // field it went into: its accessibility ref, and its name and id.+ // placeholder, from the write's receipt, kept for the current task with the+ // field it went into: its accessibility ref, and its name and id. The+ // task token scopes it, as it does task proofs: a trace run ID is missing+ // when tracing is off, and a trusted continuation keeps the token._rememberPlaceholderWrite(tabId, toolName, args, response) {if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;const meta = response.fieldMeta;@@ -45779,28 +45786,34 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const normalize = text => whole(text).toLowerCase();const placeholder = normalize(meta.placeholder);- // The typed text, whole; an append (clear:false) reads the field's- // result instead, which the receipt reports shortened.- const appended = args?.clear === false && typeof response.value === 'string';- const typed = whole(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));+ // The typed text, whole, when the write replaced the value: set_field+ // replaces it unless clear is false, the typing tools only with+ // clear:true. An append reads the field's result instead, which the+ // receipt reports shortened.+ const result = this._textMutationReplacesValue(toolName, args || {}) && typeof args?.text === 'string'+ ? args.text : response.value;+ if (typeof result !== 'string') return;+ const typed = whole(result);if (!placeholder || normalize(typed) !== placeholder) return;const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';// Kept as the probe returns a whole value: its first 4,000 characters,// cut before the case is folded, as the probe cuts it.const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };if (!write.ref && !write.name && !write.id) return;- const runId = this.currentRunId.get(tabId) || null;+ const task = this._taskTokens.get(tabId);+ if (!task) return;this._placeholderWrites ??= new Map();const entry = this._placeholderWrites.get(tabId);- const writes = entry?.runId === runId ? entry.writes : [];- this._placeholderWrites.set(tabId, { runId, writes: [...writes, write].slice(-64) });+ const writes = entry?.task === task ? entry.writes : [];+ this._placeholderWrites.set(tabId, { task, writes: [...writes, write].slice(-64) });}- // Whether WebBrain's own write in this run put `value` into this probed+ // Whether WebBrain's own write in this task put `value` into this probed// field: the same ref when both have one, otherwise the same name and id._placeholderWritten(tabId, field, value) {const entry = this._placeholderWrites?.get(tabId);- if (!entry || entry.runId !== (this.currentRunId.get(tabId) || null)) return false;+ const task = this._taskTokens.get(tabId);+ if (!entry || !task || entry.task !== task) return false;const ref = String(field?.ref || '');const name = String(field?.name || '');const id = String(field?.id || '');diff --git a/test/placeholder-submit-probe.mjs b/test/placeholder-submit-probe.mjsindex cb08a80e4..18c01d6c0 100644--- a/test/placeholder-submit-probe.mjs+++ b/test/placeholder-submit-probe.mjs@@ -55,6 +55,12 @@ for (const build of ['chrome', 'firefox']) {assert.equal(result.placeholderFields?.[0]?.value, '[email protected]');const own = await probe('set_field', { ref_id: 'ref_email', text: '[email protected]', submit: true });assert.deepEqual(own.placeholderFields, []);+ // clear:false appends to what the field already holds.+ await page.evaluate(() => { document.querySelector('#email').value = 'you@'; });+ const appended = await probe('set_field', { ref_id: 'ref_email', text: 'example.com', clear: false, submit: true });+ assert.equal(appended.placeholderFields?.[0]?.fullValue, '[email protected]', 'the field ends up holding its placeholder');+ const replaced = await probe('set_field', { ref_id: 'ref_email', text: 'example.com', submit: true });+ assert.deepEqual(replaced.placeholderFields, [], 'set_field replaces the value by default');});await t.test('untouched matches cannot push an edited field out of the list', async () => {diff --git a/test/run.js b/test/run.jsindex dbd63fa73..1815aeb33 100644--- a/test/run.js+++ b/test/run.js@@ -82766,9 +82766,11 @@ test('a field holding its own placeholder text is not submitted', async () => {assert.equal(prompts, 6, `${AgentClass.name}: a whole card answer was refused`);// A framework can copy the typed value into the field's default, so the- // probe sees no edit; WebBrain's own write in this run still counts.+ // probe sees no edit; WebBrain's own write in this task still counts.+ // Tracing is off here: there is no trace run ID, only the task token.agent.conversations.set(tabId, [...task]);- agent.currentRunId.set(tabId, 'placeholder-run');+ agent.currentRunId.delete(tabId);+ agent._taskTokens.set(tabId, 'task_placeholder');field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_101', name: 'email', id: '' };agent._rememberPlaceholderWrite(tabId, 'set_field', { ref_id: 'ref_101', text: '[email protected]' },{ success: true, value: '[email protected]', fieldMeta: { type: 'email', name: 'email', id: null, placeholder: '[email protected]' } });@@ -82778,11 +82780,24 @@ test('a field holding its own placeholder text is not submitted', async () => {field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_202', name: 'email', id: '' };await submit('other-field');assert.equal(prompts, 7, `${AgentClass.name}: a write to one field counted for another`);+ // The typing tools add to what a field holds unless clear is true: the+ // receipt's result is what the field then holds, not the typed text.+ agent._taskTokens.set(tabId, 'task_append');+ const appendMeta = { type: 'email', name: 'email', id: null, placeholder: '[email protected]' };+ const appendField = { ref: '', name: 'email', id: '' };+ agent._rememberPlaceholderWrite(tabId, 'type_text', { text: '[email protected]' },+ { success: true, value: '[email protected]', fieldMeta: appendMeta });+ assert.equal(agent._placeholderWritten(tabId, appendField, '[email protected]'), false,+ `${AgentClass.name}: an append was read as replacing the value`);+ agent._rememberPlaceholderWrite(tabId, 'type_text', { text: 'example.com' },+ { success: true, value: '[email protected]', fieldMeta: appendMeta });+ assert.equal(agent._placeholderWritten(tabId, appendField, '[email protected]'), true,+ `${AgentClass.name}: an append that left the placeholder was missed`);field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_101', name: 'email', id: '' };// A long typed value is matched by the first 200 characters the probe// reports, though the write's receipt shortens what it read back.const longText = `Tell us about your project ${'in as much detail as you can '.repeat(8)}`.trim();- agent.currentRunId.set(tabId, 'long-run');+ agent._taskTokens.set(tabId, 'task_long');agent._rememberPlaceholderWrite(tabId, 'set_field', { ref_id: 'ref_303', text: longText },{ success: true, value: longText.slice(0, 100), fieldMeta: { type: 'textarea', name: 'about', id: null, placeholder: longText } });field = { label: 'About', placeholder: longText.slice(0, 200), value: longText.slice(0, 200), fullValue: longText, complete: true,@@ -82802,9 +82817,17 @@ test('a field holding its own placeholder text is not submitted', async () => {assert.equal((await submit('too-long')).placeholderValue, true, `${AgentClass.name}: a value too long to return was matched`);agent.conversations.set(tabId, [...task]);field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_101', name: 'email', id: '' };- agent.currentRunId.set(tabId, 'next-run');+ agent._taskTokens.set(tabId, 'task_next');await submit('next-run');- assert.equal(prompts, 9, `${AgentClass.name}: a write from another run still counted`);+ assert.equal(prompts, 9, `${AgentClass.name}: a write from another task still counted`);+ // Without a task, nothing is kept.+ agent._taskTokens.delete(tabId);+ agent._rememberPlaceholderWrite(tabId, 'set_field', { ref_id: 'ref_101', text: '[email protected]' },+ { success: true, value: '[email protected]', fieldMeta: { type: 'email', name: 'email', id: null, placeholder: '[email protected]' } });+ assert.equal(agent._placeholderWrites.get(tabId)?.task, 'task_long', `${AgentClass.name}: a write outside a task was kept`);+ // A cleared conversation takes the writes along.+ agent._cleanupTab(tabId, { preserveRunGuard: true });+ assert.equal(agent._placeholderWrites.has(tabId), false, `${AgentClass.name}: tab cleanup kept placeholder writes`);// An earlier, unrelated turn cannot supply the value for the current task.field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };