Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Placeholder guard: check a form-less field set_field submits

Claude93674c2d30bb
diff --git a/docs/security-model.md b/docs/security-model.md
index 8f382ba6a..0582203cf 100644
--- a/docs/security-model.md
+++ b/docs/security-model.md
@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent
| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. |
| **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. |
| **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |
-| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field in this task, with tracing on or off. The field is matched by its ref, which names one element in one document: the write's own ref, or for a write by selector or focus, the ref its target check resolved. An append whose result does not report the field's whole value counts for whatever the field then holds. A name or id is not enough, since another form, page or frame can reuse it. The receipts are kept in memory and rebuilt from the task's own fill calls and results, so a worker restart does not lose them. In Chrome, a submit control the page probe cannot reach (a closed shadow root) has its form's fields read through the debugger. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
+| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field in this task, with tracing on or off. The field is matched by its ref, which names one element in one document: the write's own ref, or for a write by selector or focus, the ref its target check resolved. An append whose result does not report the field's whole value counts for whatever the field then holds. A name or id is not enough, since another form, page or frame can reuse it. The receipts are kept in memory and rebuilt from the task's own fill calls and results, so a worker restart does not lose them. In Chrome, a submit control the page probe cannot reach (a closed shadow root) has its form's fields read through the debugger. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. Without a form, a field `set_field({submit:true})` types into and commits with Enter is checked on its own. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
| **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. |
| **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. |
| **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 156189e84..6c82919c3 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -32994,8 +32994,47 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
return compact(el.defaultValue ?? el.getAttribute?.('value') ?? '', 120);
};
+ // The placeholder guard's scan: every text control in `owned` whose
+ // whole value equals its own placeholder, edited ones first. A value
+ // `set_field` is about to type and submit counts as the field's value.
+ const placeholderScan = (owned, pendingEl = null, pendingValue = null) => {
+ // Every control is checked, however many there are: only the matches
+ // are kept, and their list is capped below.
+ const matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).flatMap(el => {
+ const type = String(el.type || '').toLowerCase();
+ if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];
+ // A disabled control, its own or its fieldset's, is not sent.
+ try { if (el.matches?.(':disabled')) return []; } catch { return []; }
+ // Whole values decide; only what is returned for display is
+ // shortened. The whole value (up to 4,000 characters) is returned
+ // too, for the agent's own comparisons.
+ const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();
+ const placeholder = whole(el.getAttribute?.('placeholder'));
+ // set_field({submit, clear:false}) appends to what the field holds.
+ const typed = pendingEl && el === pendingEl
+ ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')
+ : el.value;
+ const value = whole(typed);
+ const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));
+ return placeholder && value.toLowerCase() === placeholder.toLowerCase()
+ ? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,
+ fullValue: value.slice(0, 4000), complete: value.length <= 4000,
+ ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',
+ name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];
+ });
+ // Edited fields first, so no number of untouched matches can push
+ // one out of the list.
+ return [...matches.filter(field => field.changed), ...matches.filter(field => !field.changed)].slice(0, 500);
+ };
const summarizeForm = (form, pendingEl = null, pendingValue = null) => {
- if (!form) return { summary: '', fields: [], changedFields: [] };
+ // Without a form, a field that set_field({submit:true}) types into and
+ // commits with Enter is still checked for the placeholder guard.
+ if (!form) {
+ const placeholderFields = (() => {
+ try { return pendingEl ? placeholderScan([pendingEl], pendingEl, pendingValue) : []; } catch { return []; }
+ })();
+ return { summary: '', fields: [], changedFields: [], ...(placeholderFields.length ? { placeholderFields } : {}) };
+ }
const method = compact(form.getAttribute('method') || form.method || 'GET', 20).toUpperCase();
const actionUrl = (() => {
try {
@@ -33066,33 +33105,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// only what it contains.
const owned = String(form.tagName || '').toUpperCase() === 'FORM' && form.elements
? Array.from(form.elements) : Array.from(form.querySelectorAll('input, textarea'));
- // Every control is checked, however many the form has: only the
- // matches are kept, and their list is capped below.
- const matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).flatMap(el => {
- const type = String(el.type || '').toLowerCase();
- if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];
- // A disabled control, its own or its fieldset's, is not sent.
- try { if (el.matches?.(':disabled')) return []; } catch { return []; }
- // Whole values decide; only what is returned for display is
- // shortened. The whole value (up to 4,000 characters) is returned
- // too, for the agent's own comparisons.
- const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();
- const placeholder = whole(el.getAttribute?.('placeholder'));
- // set_field({submit, clear:false}) appends to what the field holds.
- const typed = pendingEl && el === pendingEl
- ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')
- : el.value;
- const value = whole(typed);
- const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));
- return placeholder && value.toLowerCase() === placeholder.toLowerCase()
- ? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,
- fullValue: value.slice(0, 4000), complete: value.length <= 4000,
- ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',
- name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];
- });
- // Edited fields first, so no number of untouched matches can push
- // one out of the list.
- return [...matches.filter(field => field.changed), ...matches.filter(field => !field.changed)].slice(0, 500);
+ return placeholderScan(owned, pendingEl, pendingValue);
} catch {
return [];
}
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 73453194c..d55849b8c 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -29831,8 +29831,47 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
return compact(el.defaultValue ?? el.getAttribute?.('value') ?? '', 120);
};
+ // The placeholder guard's scan: every text control in `owned` whose
+ // whole value equals its own placeholder, edited ones first. A value
+ // `set_field` is about to type and submit counts as the field's value.
+ const placeholderScan = (owned, pendingEl = null, pendingValue = null) => {
+ // Every control is checked, however many there are: only the matches
+ // are kept, and their list is capped below.
+ const matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).flatMap(el => {
+ const type = String(el.type || '').toLowerCase();
+ if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];
+ // A disabled control, its own or its fieldset's, is not sent.
+ try { if (el.matches?.(':disabled')) return []; } catch { return []; }
+ // Whole values decide; only what is returned for display is
+ // shortened. The whole value (up to 4,000 characters) is returned
+ // too, for the agent's own comparisons.
+ const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();
+ const placeholder = whole(el.getAttribute?.('placeholder'));
+ // set_field({submit, clear:false}) appends to what the field holds.
+ const typed = pendingEl && el === pendingEl
+ ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')
+ : el.value;
+ const value = whole(typed);
+ const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));
+ return placeholder && value.toLowerCase() === placeholder.toLowerCase()
+ ? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,
+ fullValue: value.slice(0, 4000), complete: value.length <= 4000,
+ ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',
+ name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];
+ });
+ // Edited fields first, so no number of untouched matches can push
+ // one out of the list.
+ return [...matches.filter(field => field.changed), ...matches.filter(field => !field.changed)].slice(0, 500);
+ };
const summarizeForm = (form, pendingEl = null, pendingValue = null) => {
- if (!form) return { summary: '', fields: [], changedFields: [] };
+ // Without a form, a field that set_field({submit:true}) types into and
+ // commits with Enter is still checked for the placeholder guard.
+ if (!form) {
+ const placeholderFields = (() => {
+ try { return pendingEl ? placeholderScan([pendingEl], pendingEl, pendingValue) : []; } catch { return []; }
+ })();
+ return { summary: '', fields: [], changedFields: [], ...(placeholderFields.length ? { placeholderFields } : {}) };
+ }
const method = compact(form.getAttribute('method') || form.method || 'GET', 20).toUpperCase();
const actionUrl = (() => {
try {
@@ -29903,33 +29942,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// only what it contains.
const owned = String(form.tagName || '').toUpperCase() === 'FORM' && form.elements
? Array.from(form.elements) : Array.from(form.querySelectorAll('input, textarea'));
- // Every control is checked, however many the form has: only the
- // matches are kept, and their list is capped below.
- const matches = owned.filter(el => /^(?:INPUT|TEXTAREA)$/i.test(String(el?.tagName || ''))).flatMap(el => {
- const type = String(el.type || '').toLowerCase();
- if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];
- // A disabled control, its own or its fieldset's, is not sent.
- try { if (el.matches?.(':disabled')) return []; } catch { return []; }
- // Whole values decide; only what is returned for display is
- // shortened. The whole value (up to 4,000 characters) is returned
- // too, for the agent's own comparisons.
- const whole = text => String(text ?? '').replace(/\s+/g, ' ').trim();
- const placeholder = whole(el.getAttribute?.('placeholder'));
- // set_field({submit, clear:false}) appends to what the field holds.
- const typed = pendingEl && el === pendingEl
- ? (toolName === 'set_field' && args?.clear === false ? String(el.value ?? '') : '') + String(pendingValue ?? '')
- : el.value;
- const value = whole(typed);
- const changed = el === pendingEl || (!!value && value !== whole(el.defaultValue ?? el.getAttribute?.('value')));
- return placeholder && value.toLowerCase() === placeholder.toLowerCase()
- ? [{ label: labelFor(el), value: compact(value, 200), placeholder: compact(placeholder, 200), changed,
- fullValue: value.slice(0, 4000), complete: value.length <= 4000,
- ref: typeof window.__wb_ax_ref === 'function' ? String(window.__wb_ax_ref(el) || '') : '',
- name: compact(el.getAttribute?.('name') || '', 120), id: compact(el.getAttribute?.('id') || '', 120) }] : [];
- });
- // Edited fields first, so no number of untouched matches can push
- // one out of the list.
- return [...matches.filter(field => field.changed), ...matches.filter(field => !field.changed)].slice(0, 500);
+ return placeholderScan(owned, pendingEl, pendingValue);
} catch {
return [];
}
diff --git a/test/placeholder-submit-probe.mjs b/test/placeholder-submit-probe.mjs
index 1436750cd..bd3b23f8d 100644
--- a/test/placeholder-submit-probe.mjs
+++ b/test/placeholder-submit-probe.mjs
@@ -63,6 +63,16 @@ for (const build of ['chrome', 'firefox']) {
assert.deepEqual(replaced.placeholderFields, [], 'set_field replaces the value by default');
});
+ await t.test('a form-less field set_field types into and commits is checked too', async () => {
+ await page.setContent('<div><input id="email" name="email" aria-label="Email" placeholder="[email protected]"></div>');
+ await page.evaluate(() => { window.__wb_ax_lookup = ref => (ref === 'ref_email' ? document.querySelector('#email') : null); });
+ const result = await probe('set_field', { ref_id: 'ref_email', text: '[email protected]', submit: true });
+ assert.equal(result?.isSubmit, true);
+ assert.deepEqual(result.placeholderFields?.map(field => [field.name, field.changed]), [['email', true]]);
+ const own = await probe('set_field', { ref_id: 'ref_email', text: '[email protected]', submit: true });
+ assert.equal(own.placeholderFields, undefined, 'another value lists nothing');
+ });
+
await t.test('untouched matches cannot push an edited field out of the list', async () => {
await page.setContent('<form method="post" action="/many">'
+ Array.from({ length: 600 }, (_, i) => `<input name="d${i}" placeholder="Example" value="Example">`).join('')