Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Placeholder guard: count WebBrain's own writes, match supplied values

Clauded38ded76b174
diff --git a/docs/security-model.md b/docs/security-model.md
index 236cf36c3..db0dc4f9f 100644
--- a/docs/security-model.md
+++ b/docs/security-model.md
@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent
| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. |
| **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. |
| **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |
-| **Placeholder-value guard** | A form submission whose changed field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless one of the user's messages or card answers contains that value. This holds with confirmations off and in scheduled runs too. The submit probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
+| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user supplied that value: as a whole answer on a card, or, for a value with a digit or symbol, written as a whole in their message. A plain word the request only names ("sign me up for email updates" beside a placeholder `Email`) does not supply it. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value there in this run. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
| **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. |
| **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. |
| **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 180e04e85..e2688ea9a 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -32449,27 +32449,65 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
};
}
- // The changed fields of a submission that hold their own placeholder text
- // when none of the user's messages or card answers gave that value: the
- // page's example typed in as the user's. The probe's complete list is
- // read when it sent one; the summary's changed fields stop at eight. A
- // placeholder without a letter, or shorter than three characters (a "1"
- // in a quantity box), is not an example of this kind.
+ // The edited fields of a submission that hold their own placeholder text
+ // when the user did not supply that value: the page's example typed in
+ // as the user's. A field counts as edited when the probe saw it differ
+ // from its default or WebBrain's own write put that value there in this
+ // run (a framework may copy a typed value into defaultValue). The
+ // probe's complete list is read when it sent one; the summary's changed
+ // fields stop at eight. A placeholder without a letter, or shorter than
+ // three characters (a "1" in a quantity box), is not an example of this
+ // kind.
_placeholderValueFields(tabId, submitInfo) {
const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();
const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields
: Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];
const fields = candidates.filter(field => {
const value = normalize(field?.value);
- return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);
+ return (field?.changed === true || this._placeholderWritten(tabId, value))
+ && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);
});
if (!fields.length) return [];
- const userText = (this.conversations.get(tabId) || [])
- .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message))
- .flatMap(message => [this._plannerUserAuthoredText(message),
- ...(Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications.map(item => item?.answer) : [])])
- .map(normalize).join('\n');
- return fields.filter(field => !userText.includes(normalize(field.value)));
+ const messages = (this.conversations.get(tabId) || [])
+ .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message));
+ const requests = messages.map(message => normalize(this._plannerUserAuthoredText(message)));
+ const answers = messages.flatMap(message => (Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications : [])
+ .filter(item => ['user', 'text', 'option'].includes(item?.source)).map(item => normalize(item?.answer)));
+ const supplied = value => {
+ const escaped = value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+ const named = new RegExp(`(?:^|[^\\p{L}\\p{N}])${escaped}(?:$|[^\\p{L}\\p{N}])`, 'u');
+ // A plain word or phrase ("Email", "Your name") is named in requests
+ // without being given; it counts only as a whole answer to a card.
+ // A value with a digit or symbol ("[email protected]") counts where
+ // the user wrote it as a whole.
+ const specific = /[^\p{L}\s]/u.test(value);
+ return answers.includes(value) || (specific && [...requests, ...answers].some(text => named.test(text)));
+ };
+ return fields.filter(field => !supplied(normalize(field.value)));
+ }
+
+ // A value WebBrain typed into a field that equals the field's own
+ // placeholder, from the write's receipt (its field metadata), kept for
+ // the current run.
+ _rememberPlaceholderWrite(tabId, toolName, args, response) {
+ if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;
+ const meta = response.fieldMeta;
+ if (!meta || String(meta.type || '').toLowerCase() === 'password') return;
+ const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();
+ const placeholder = normalize(meta.placeholder);
+ const written = normalize(typeof response.value === 'string' ? response.value : args?.text);
+ if (!placeholder || written !== placeholder) return;
+ const runId = this.currentRunId.get(tabId) || null;
+ this._placeholderWrites ??= new Map();
+ const entry = this._placeholderWrites.get(tabId);
+ const values = entry?.runId === runId ? entry.values : new Set();
+ values.add(written);
+ this._placeholderWrites.set(tabId, { runId, values });
+ }
+
+ _placeholderWritten(tabId, value) {
+ const entry = this._placeholderWrites?.get(tabId);
+ return !!entry && entry.runId === (this.currentRunId.get(tabId) || null) && entry.values.has(value);
}
_fallbackSubmitConfirmationInfo(host, tool, reason, summary = '') {
@@ -33034,9 +33072,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
try { return !!form.closest?.('[role="search"],search'); } catch { return false; }
})();
const changedFields = fields.filter(field => field.changed || field.value).slice(0, 8);
- // Every changed field that still holds its own placeholder text, for
- // the placeholder guard: unlike the summary above, not capped.
- // Passwords and other fields without a typed value are left out.
+ // Every field that holds its own placeholder text, for the
+ // placeholder guard, and whether it differs from its default: unlike
+ // the summary above, not capped. Passwords and other fields without a
+ // typed value are left out.
const placeholderFields = (() => {
try {
return Array.from(form.querySelectorAll('input, textarea')).slice(0, 500).flatMap(el => {
@@ -33045,8 +33084,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const placeholder = compact(el.getAttribute?.('placeholder') || '', 200);
const value = pendingEl && el === pendingEl ? compact(pendingValue, 200) : compact(el.value || '', 200);
const changed = el === pendingEl || (!!value && value !== defaultValue(el));
- return changed && placeholder && value.toLowerCase() === placeholder.toLowerCase()
- ? [{ label: labelFor(el), value, placeholder, changed: true }] : [];
+ return placeholder && value.toLowerCase() === placeholder.toLowerCase()
+ ? [{ label: labelFor(el), value, placeholder, changed }] : [];
}).slice(0, 16);
} catch {
return [];
@@ -52399,6 +52438,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this._recordInteractionRect(tabId, name, response, interactionUrl);
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index a37151f80..f509d5f50 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -29437,27 +29437,65 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
};
}
- // The changed fields of a submission that hold their own placeholder text
- // when none of the user's messages or card answers gave that value: the
- // page's example typed in as the user's. The probe's complete list is
- // read when it sent one; the summary's changed fields stop at eight. A
- // placeholder without a letter, or shorter than three characters (a "1"
- // in a quantity box), is not an example of this kind.
+ // The edited fields of a submission that hold their own placeholder text
+ // when the user did not supply that value: the page's example typed in
+ // as the user's. A field counts as edited when the probe saw it differ
+ // from its default or WebBrain's own write put that value there in this
+ // run (a framework may copy a typed value into defaultValue). The
+ // probe's complete list is read when it sent one; the summary's changed
+ // fields stop at eight. A placeholder without a letter, or shorter than
+ // three characters (a "1" in a quantity box), is not an example of this
+ // kind.
_placeholderValueFields(tabId, submitInfo) {
const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();
const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields
: Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];
const fields = candidates.filter(field => {
const value = normalize(field?.value);
- return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);
+ return (field?.changed === true || this._placeholderWritten(tabId, value))
+ && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);
});
if (!fields.length) return [];
- const userText = (this.conversations.get(tabId) || [])
- .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message))
- .flatMap(message => [this._plannerUserAuthoredText(message),
- ...(Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications.map(item => item?.answer) : [])])
- .map(normalize).join('\n');
- return fields.filter(field => !userText.includes(normalize(field.value)));
+ const messages = (this.conversations.get(tabId) || [])
+ .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message));
+ const requests = messages.map(message => normalize(this._plannerUserAuthoredText(message)));
+ const answers = messages.flatMap(message => (Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications : [])
+ .filter(item => ['user', 'text', 'option'].includes(item?.source)).map(item => normalize(item?.answer)));
+ const supplied = value => {
+ const escaped = value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+ const named = new RegExp(`(?:^|[^\\p{L}\\p{N}])${escaped}(?:$|[^\\p{L}\\p{N}])`, 'u');
+ // A plain word or phrase ("Email", "Your name") is named in requests
+ // without being given; it counts only as a whole answer to a card.
+ // A value with a digit or symbol ("[email protected]") counts where
+ // the user wrote it as a whole.
+ const specific = /[^\p{L}\s]/u.test(value);
+ return answers.includes(value) || (specific && [...requests, ...answers].some(text => named.test(text)));
+ };
+ return fields.filter(field => !supplied(normalize(field.value)));
+ }
+
+ // A value WebBrain typed into a field that equals the field's own
+ // placeholder, from the write's receipt (its field metadata), kept for
+ // the current run.
+ _rememberPlaceholderWrite(tabId, toolName, args, response) {
+ if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;
+ const meta = response.fieldMeta;
+ if (!meta || String(meta.type || '').toLowerCase() === 'password') return;
+ const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();
+ const placeholder = normalize(meta.placeholder);
+ const written = normalize(typeof response.value === 'string' ? response.value : args?.text);
+ if (!placeholder || written !== placeholder) return;
+ const runId = this.currentRunId.get(tabId) || null;
+ this._placeholderWrites ??= new Map();
+ const entry = this._placeholderWrites.get(tabId);
+ const values = entry?.runId === runId ? entry.values : new Set();
+ values.add(written);
+ this._placeholderWrites.set(tabId, { runId, values });
+ }
+
+ _placeholderWritten(tabId, value) {
+ const entry = this._placeholderWrites?.get(tabId);
+ return !!entry && entry.runId === (this.currentRunId.get(tabId) || null) && entry.values.has(value);
}
_fallbackSubmitConfirmationInfo(host, tool, reason, summary = '') {
@@ -29938,9 +29976,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
try { return !!form.closest?.('[role="search"],search'); } catch { return false; }
})();
const changedFields = fields.filter(field => field.changed || field.value).slice(0, 8);
- // Every changed field that still holds its own placeholder text, for
- // the placeholder guard: unlike the summary above, not capped.
- // Passwords and other fields without a typed value are left out.
+ // Every field that holds its own placeholder text, for the
+ // placeholder guard, and whether it differs from its default: unlike
+ // the summary above, not capped. Passwords and other fields without a
+ // typed value are left out.
const placeholderFields = (() => {
try {
return Array.from(form.querySelectorAll('input, textarea')).slice(0, 500).flatMap(el => {
@@ -29949,8 +29988,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const placeholder = compact(el.getAttribute?.('placeholder') || '', 200);
const value = pendingEl && el === pendingEl ? compact(pendingValue, 200) : compact(el.value || '', 200);
const changed = el === pendingEl || (!!value && value !== defaultValue(el));
- return changed && placeholder && value.toLowerCase() === placeholder.toLowerCase()
- ? [{ label: labelFor(el), value, placeholder, changed: true }] : [];
+ return placeholder && value.toLowerCase() === placeholder.toLowerCase()
+ ? [{ label: labelFor(el), value, placeholder, changed }] : [];
}).slice(0, 16);
} catch {
return [];
@@ -45546,6 +45585,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
@@ -45613,6 +45653,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
diff --git a/test/placeholder-submit-probe.mjs b/test/placeholder-submit-probe.mjs
index 5629a4e22..faef96991 100644
--- a/test/placeholder-submit-probe.mjs
+++ b/test/placeholder-submit-probe.mjs
@@ -1,7 +1,7 @@
-// The submit probe lists every changed field that still holds its own
-// placeholder text, for the placeholder guard. Unlike the summary's changed
-// fields, which stop at eight, the list covers the whole form; it never
-// carries a password, and a page default WebBrain did not change is not in it.
+// The submit probe lists every field that holds its own placeholder text,
+// for the placeholder guard, and says whether it differs from its default.
+// Unlike the summary's changed fields, which stop at eight, the list covers
+// the whole form, and it never carries a password.
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { chromium, firefox } from 'playwright';
@@ -38,8 +38,10 @@ for (const build of ['chrome', 'firefox']) {
assert.equal(result?.isSubmit, true);
assert.equal(result.changedFields.length, 8, 'the summary stops at eight');
assert.ok(!result.changedFields.some(field => field.label === 'Email'), 'the email field is past the summary');
- assert.deepEqual(result.placeholderFields, [{ label: 'Email', value: '[email protected]', placeholder: '[email protected]', changed: true }],
- 'only the changed email field holding its placeholder; no password, no unchanged page default');
+ assert.deepEqual(result.placeholderFields, [
+ { label: 'Email', value: '[email protected]', placeholder: '[email protected]', changed: true },
+ { label: 'City', value: 'Istanbul', placeholder: 'Istanbul', changed: false },
+ ], 'the edited email field, and the page default marked unchanged; never the password');
});
await t.test('a value set_field is about to type and submit is checked too', async () => {
diff --git a/test/run.js b/test/run.js
index ded6ef219..1a19021aa 100644
--- a/test/run.js
+++ b/test/run.js
@@ -82685,6 +82685,28 @@ test('a field holding its own placeholder text is not submitted', async () => {
field = { label: 'Quantity', placeholder: '1', value: '1', changed: true };
await submit('number');
assert.equal(prompts, 6, `${AgentClass.name}: an ordinary value skipped its card`);
+
+ // Naming a plain word in the request does not supply it as the value; a
+ // whole card answer does.
+ agent.conversations.set(tabId, [{ role: 'user', content: 'Sign me up for email updates.' }]);
+ field = { label: 'Email', placeholder: 'Email', value: 'Email', changed: true };
+ assert.equal((await submit('named-word')).placeholderValue, true, `${AgentClass.name}: a word the request only named exempted the field`);
+ agent.conversations.set(tabId, [{ role: 'user', content: 'Sign me up for email updates.',
+ webbrainTaskClarifications: [{ question: 'What should the email field say?', answer: 'Email', source: 'text' }] }]);
+ await submit('answered-word');
+ assert.equal(prompts, 7, `${AgentClass.name}: a whole card answer was refused`);
+
+ // A framework can copy the typed value into the field's default, so the
+ // probe sees no edit; WebBrain's own write in this run still counts.
+ agent.conversations.set(tabId, [...task]);
+ agent.currentRunId.set(tabId, 'placeholder-run');
+ field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false };
+ agent._rememberPlaceholderWrite(tabId, 'set_field', { text: '[email protected]' },
+ { success: true, value: '[email protected]', fieldMeta: { type: 'email', placeholder: '[email protected]' } });
+ assert.equal((await submit('framework-default')).placeholderValue, true, `${AgentClass.name}: WebBrain's own write was missed`);
+ agent.currentRunId.set(tabId, 'next-run');
+ await submit('next-run');
+ assert.equal(prompts, 8, `${AgentClass.name}: a write from another run still counted`);
assert.equal(executed, 0, `${AgentClass.name}: a refused or declined submission ran`);
}
});