Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Placeholder guard: compare whole values with answers; skip disabled controls
Claude6be827f47708
diff --git a/docs/security-model.md b/docs/security-model.mdindex a8cfb2dff..5db8acb1b 100644--- a/docs/security-model.md+++ b/docs/security-model.md@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. || **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. || **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |-| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field (by its ref, or its name and id) in this run. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |+| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field (by its ref, or its name and id) in this run. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. || **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. || **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. || **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex b3edf4096..7a2a7b4d2 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -33000,13 +33000,18 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).slice(0, 5000).flatMap(el => {const type = String(el.type || '').toLowerCase();if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];- // Whole values decide; only what is returned is shortened.+ // A disabled control, its own or its fieldset's, is not sent.+ try { if (el.matches?.(':disabled')) return []; } catch { return []; }+ // Whole values decide; only what is returned for display is+ // shortened. The whole value (up to 4,000 characters) is returned+ // too, for the agent's own comparisons.const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const placeholder = whole(el.getAttribute?.('placeholder'));const value = whole(pendingEl && el === pendingEl ? pendingValue : el.value);const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));return placeholder && value.toLowerCase() === placeholder.toLowerCase()? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,+ fullValue: value.slice(0, 4000), complete: value.length <= 4000,ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];});@@ -52465,9 +52470,14 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields: Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];+ // The probe's entries carry the whole value it found equal to the+ // placeholder; an entry of the summary's changed fields is shortened+ // and is compared here.+ const wholeValue = field => normalize(typeof field?.fullValue === 'string' ? field.fullValue : field?.value);const fields = candidates.filter(field => {- const value = normalize(field?.value);- return value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder)+ const value = wholeValue(field);+ const equal = typeof field?.fullValue === 'string' || value === normalize(field?.placeholder);+ return value.length >= 3 && /\p{L}/u.test(value) && equal&& (field?.changed === true || this._placeholderWritten(tabId, field, value));});if (!fields.length) return [];@@ -52484,7 +52494,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d? message.webbrainTaskClarifications : []).filter(item => ['user', 'text', 'option'].includes(item?.source)).map(item => item?.answer)),].map(normalize).filter(Boolean);- return fields.filter(field => !answers.includes(normalize(field.value)));+ // A value too long for the probe to return whole cannot be matched to+ // an answer, so it stays refused.+ return fields.filter(field => field?.complete === false || !answers.includes(wholeValue(field)));}// A value WebBrain typed into a field that equals the field's own@@ -52494,17 +52506,18 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dif (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;const meta = response.fieldMeta;if (!meta || String(meta.type || '').toLowerCase() === 'password') return;- const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();+ const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();+ const normalize = text => whole(text).toLowerCase();const placeholder = normalize(meta.placeholder);// The typed text, whole; an append (clear:false) reads the field's// result instead, which the receipt reports shortened.const appended = args?.clear === false && typeof response.value === 'string';- const written = normalize(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));- if (!placeholder || written !== placeholder) return;+ const typed = whole(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));+ if (!placeholder || normalize(typed) !== placeholder) return;const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';- // Kept in the form the submit probe reports a value: its first 200- // characters.- const write = { value: normalize(written.slice(0, 200)), ref, name: String(meta.name || ''), id: String(meta.id || '') };+ // Kept as the probe returns a whole value: its first 4,000 characters,+ // cut before the case is folded, as the probe cuts it.+ const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };if (!write.ref && !write.name && !write.id) return;const runId = this.currentRunId.get(tabId) || null;this._placeholderWrites ??= new Map();diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex b65545c49..31463ebee 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -29904,13 +29904,18 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).slice(0, 5000).flatMap(el => {const type = String(el.type || '').toLowerCase();if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];- // Whole values decide; only what is returned is shortened.+ // A disabled control, its own or its fieldset's, is not sent.+ try { if (el.matches?.(':disabled')) return []; } catch { return []; }+ // Whole values decide; only what is returned for display is+ // shortened. The whole value (up to 4,000 characters) is returned+ // too, for the agent's own comparisons.const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();const placeholder = whole(el.getAttribute?.('placeholder'));const value = whole(pendingEl && el === pendingEl ? pendingValue : el.value);const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));return placeholder && value.toLowerCase() === placeholder.toLowerCase()? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,+ fullValue: value.slice(0, 4000), complete: value.length <= 4000,ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];});@@ -45735,9 +45740,14 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields: Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];+ // The probe's entries carry the whole value it found equal to the+ // placeholder; an entry of the summary's changed fields is shortened+ // and is compared here.+ const wholeValue = field => normalize(typeof field?.fullValue === 'string' ? field.fullValue : field?.value);const fields = candidates.filter(field => {- const value = normalize(field?.value);- return value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder)+ const value = wholeValue(field);+ const equal = typeof field?.fullValue === 'string' || value === normalize(field?.placeholder);+ return value.length >= 3 && /\p{L}/u.test(value) && equal&& (field?.changed === true || this._placeholderWritten(tabId, field, value));});if (!fields.length) return [];@@ -45754,7 +45764,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d? message.webbrainTaskClarifications : []).filter(item => ['user', 'text', 'option'].includes(item?.source)).map(item => item?.answer)),].map(normalize).filter(Boolean);- return fields.filter(field => !answers.includes(normalize(field.value)));+ // A value too long for the probe to return whole cannot be matched to+ // an answer, so it stays refused.+ return fields.filter(field => field?.complete === false || !answers.includes(wholeValue(field)));}// A value WebBrain typed into a field that equals the field's own@@ -45764,17 +45776,18 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dif (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;const meta = response.fieldMeta;if (!meta || String(meta.type || '').toLowerCase() === 'password') return;- const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();+ const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();+ const normalize = text => whole(text).toLowerCase();const placeholder = normalize(meta.placeholder);// The typed text, whole; an append (clear:false) reads the field's// result instead, which the receipt reports shortened.const appended = args?.clear === false && typeof response.value === 'string';- const written = normalize(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));- if (!placeholder || written !== placeholder) return;+ const typed = whole(appended ? response.value : (typeof args?.text === 'string' ? args.text : response.value));+ if (!placeholder || normalize(typed) !== placeholder) return;const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';- // Kept in the form the submit probe reports a value: its first 200- // characters.- const write = { value: normalize(written.slice(0, 200)), ref, name: String(meta.name || ''), id: String(meta.id || '') };+ // Kept as the probe returns a whole value: its first 4,000 characters,+ // cut before the case is folded, as the probe cuts it.+ const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };if (!write.ref && !write.name && !write.id) return;const runId = this.currentRunId.get(tabId) || null;this._placeholderWrites ??= new Map();diff --git a/test/placeholder-submit-probe.mjs b/test/placeholder-submit-probe.mjsindex eee17ba89..cb08a80e4 100644--- a/test/placeholder-submit-probe.mjs+++ b/test/placeholder-submit-probe.mjs@@ -92,6 +92,26 @@ for (const build of ['chrome', 'firefox']) {assert.deepEqual(edited.placeholderFields.map(field => [field.name, field.changed]), [['note', false]],'a changed long value no longer equals its placeholder');});++ await t.test('a disabled field is not sent and not listed; the whole value is returned', async () => {+ const long = `Describe the change you need ${'with every detail that matters '.repeat(8)}`.trim();+ await page.setContent('<form method="post" action="/change">'+ + '<input id="off" name="off" aria-label="Off" placeholder="[email protected]" disabled>'+ + '<fieldset disabled><input id="inner" name="inner" aria-label="Inner" placeholder="[email protected]"></fieldset>'+ + `<textarea id="about" name="about" aria-label="About" placeholder="${long}"></textarea><button id="go">Send</button></form>`);+ await page.evaluate(text => {+ document.querySelector('#off').value = '[email protected]';+ document.querySelector('#inner').value = '[email protected]';+ document.querySelector('#about').value = text;+ }, long);+ const result = await probe('click', { selector: '#go' });+ assert.equal(result?.isSubmit, true);+ assert.deepEqual(result.placeholderFields.map(field => field.name), ['about'], 'disabled fields are not sent');+ assert.ok(long.length > 200);+ assert.equal(result.placeholderFields[0].fullValue, long, 'the whole value');+ assert.equal(result.placeholderFields[0].complete, true);+ assert.equal(result.placeholderFields[0].value.length, 200, 'the display value is shortened');+ });} finally {await browser.close();}diff --git a/test/run.js b/test/run.jsindex 5fc449a5c..dbd63fa73 100644--- a/test/run.js+++ b/test/run.js@@ -82785,12 +82785,26 @@ test('a field holding its own placeholder text is not submitted', async () => {agent.currentRunId.set(tabId, 'long-run');agent._rememberPlaceholderWrite(tabId, 'set_field', { ref_id: 'ref_303', text: longText },{ success: true, value: longText.slice(0, 100), fieldMeta: { type: 'textarea', name: 'about', id: null, placeholder: longText } });- field = { label: 'About', placeholder: longText.slice(0, 200), value: longText.slice(0, 200), changed: false, ref: 'ref_303', name: 'about', id: '' };+ field = { label: 'About', placeholder: longText.slice(0, 200), value: longText.slice(0, 200), fullValue: longText, complete: true,+ changed: false, ref: 'ref_303', name: 'about', id: '' };assert.equal((await submit('long-write')).placeholderValue, true, `${AgentClass.name}: a long write was missed`);+ // An answer has to be the whole value: the first 200 characters the+ // probe shows are not it, and a value too long to return is never+ // matched.+ field = { ...field, changed: true };+ agent.conversations.set(tabId, [system, { ...task[1], webbrainTaskClarifications: [{ question: 'About?', answer: longText.slice(0, 200), source: 'text' }] }]);+ assert.equal((await submit('prefix-answer')).placeholderValue, true, `${AgentClass.name}: a prefix answered a long value`);+ agent.conversations.set(tabId, [system, { ...task[1], webbrainTaskClarifications: [{ question: 'About?', answer: longText, source: 'text' }] }]);+ const beforeWholeAnswer = prompts;+ await submit('whole-answer');+ assert.equal(prompts, beforeWholeAnswer + 1, `${AgentClass.name}: the whole answer was refused`);+ field = { ...field, complete: false };+ assert.equal((await submit('too-long')).placeholderValue, true, `${AgentClass.name}: a value too long to return was matched`);+ agent.conversations.set(tabId, [...task]);field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_101', name: 'email', id: '' };agent.currentRunId.set(tabId, 'next-run');await submit('next-run');- assert.equal(prompts, 8, `${AgentClass.name}: a write from another run still counted`);+ assert.equal(prompts, 9, `${AgentClass.name}: a write from another run still counted`);// An earlier, unrelated turn cannot supply the value for the current task.field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };