Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Placeholder guard: match write receipts by ref only

Claude1db95bfa9cfb
diff --git a/docs/security-model.md b/docs/security-model.md
index 3aeb8027d..71c53606f 100644
--- a/docs/security-model.md
+++ b/docs/security-model.md
@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent
| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. |
| **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. |
| **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |
-| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field (by its ref, or its name and id) in this task, with tracing on or off. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
+| **Placeholder-value guard** | A form submission whose edited field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless the user gave that value as a whole answer on one of the current task's cards. The words of a request do not count, whether they give the value or refuse it ("do not use [email protected]"): a text match cannot tell which, in any language, so the model asks and the answer decides. An earlier, unrelated turn's answers do not count either. A field counts as edited when the submit probe sees it differ from its default or WebBrain's own write put the value into that same field in this task, with tracing on or off. The field is matched by its ref, which names one element in one document: the write's own ref, or for a write by selector or focus, the ref its target check resolved. A name or id is not enough, since another form, page or frame can reuse it. This holds with confirmations off and in scheduled runs too. The probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A script (`execute_js`) is not covered: what it fills is not known before it runs. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |
| **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. |
| **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. |
| **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 58852fde5..48817a60b 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -17291,6 +17291,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
promptTier,
guardAdvisories: callState.guardAdvisories,
dispatchBinding: pipelineToolbarPreflight.probe?.dispatchBinding || null,
+ // The typing target's ref, for a write by selector or focus.
+ typingTargetRef: String(pipelineToolbarPreflight.probe?.refId || ''),
...messageRecipientExecutionContext,
taskClickLockRelease: callState.taskClickLockRelease || null,
iframeTargetUnresolved: pipelineToolbarPreflight.iframeTargetUnresolved === true,
@@ -52374,7 +52376,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this._recordInteractionRect(tabId, name, response, interactionUrl);
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
- this._rememberPlaceholderWrite(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response, dispatchContext.typingTargetRef);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
@@ -52506,10 +52508,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// A value WebBrain typed into a field that equals the field's own
// placeholder, from the write's receipt, kept for the current task with the
- // field it went into: its accessibility ref, and its name and id. The
- // task token scopes it, as it does task proofs: a trace run ID is missing
+ // field it went into, by its accessibility ref: the call's own ref_id, or
+ // for a write by selector or focus, the ref its target check resolved
+ // (`targetRef`). A ref names one element in one document and route, which
+ // a name or id does not; a write without one is not kept. The task token
+ // scopes the receipts, as it does task proofs: a trace run ID is missing
// when tracing is off, and a trusted continuation keeps the token.
- _rememberPlaceholderWrite(tabId, toolName, args, response) {
+ _rememberPlaceholderWrite(tabId, toolName, args, response, targetRef = '') {
if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;
const meta = response.fieldMeta;
if (!meta || String(meta.type || '').toLowerCase() === 'password') return;
@@ -52525,11 +52530,11 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (typeof result !== 'string') return;
const typed = whole(result);
if (!placeholder || normalize(typed) !== placeholder) return;
- const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';
+ const ref = [args?.ref_id, response.ref_id, targetRef].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';
+ if (!ref) return;
// Kept as the probe returns a whole value: its first 4,000 characters,
// cut before the case is folded, as the probe cuts it.
- const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };
- if (!write.ref && !write.name && !write.id) return;
+ const write = { value: normalize(typed.slice(0, 4000)), ref };
const task = this._taskTokens.get(tabId);
if (!task) return;
this._placeholderWrites ??= new Map();
@@ -52539,16 +52544,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
// Whether WebBrain's own write in this task put `value` into this probed
- // field: the same ref when both have one, otherwise the same name and id.
+ // field, the one with the same ref.
_placeholderWritten(tabId, field, value) {
const entry = this._placeholderWrites?.get(tabId);
const task = this._taskTokens.get(tabId);
if (!entry || !task || entry.task !== task) return false;
const ref = String(field?.ref || '');
- const name = String(field?.name || '');
- const id = String(field?.id || '');
- return entry.writes.some(write => write.value === value && (write.ref && ref ? write.ref === ref
- : (write.name || write.id) && write.name === name && write.id === id));
+ return !!ref && entry.writes.some(write => write.value === value && write.ref === ref);
}
// Adds values to the tab's known credentials, keeping the most recent 64.
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 971ee1ee8..e784531c2 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -15454,6 +15454,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
promptTier,
guardAdvisories: callState.guardAdvisories,
dispatchBinding: pipelineToolbarPreflight.probe?.dispatchBinding || null,
+ // The typing target's ref, for a write by selector or focus.
+ typingTargetRef: String(pipelineToolbarPreflight.probe?.refId || ''),
...messageRecipientExecutionContext,
iframeTargetUnresolved: pipelineToolbarPreflight.iframeTargetUnresolved === true,
richTextToolbarWrongTargetAdvised: pipelineToolbarPreflight.wrongTargetAdvised === true,
@@ -45521,7 +45523,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
- this._rememberPlaceholderWrite(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response, dispatchContext.typingTargetRef);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
@@ -45589,7 +45591,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
this._annotateCredentialField(name, response, tabId);
this._rememberFilledCredential(tabId, name, args, response);
- this._rememberPlaceholderWrite(tabId, name, args, response);
+ this._rememberPlaceholderWrite(tabId, name, args, response, dispatchContext.typingTargetRef);
if (name === 'read_page') {
response = applyReadPageWindow(response, args);
}
@@ -45776,10 +45778,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// A value WebBrain typed into a field that equals the field's own
// placeholder, from the write's receipt, kept for the current task with the
- // field it went into: its accessibility ref, and its name and id. The
- // task token scopes it, as it does task proofs: a trace run ID is missing
+ // field it went into, by its accessibility ref: the call's own ref_id, or
+ // for a write by selector or focus, the ref its target check resolved
+ // (`targetRef`). A ref names one element in one document and route, which
+ // a name or id does not; a write without one is not kept. The task token
+ // scopes the receipts, as it does task proofs: a trace run ID is missing
// when tracing is off, and a trusted continuation keeps the token.
- _rememberPlaceholderWrite(tabId, toolName, args, response) {
+ _rememberPlaceholderWrite(tabId, toolName, args, response, targetRef = '') {
if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName) || response?.success !== true) return;
const meta = response.fieldMeta;
if (!meta || String(meta.type || '').toLowerCase() === 'password') return;
@@ -45795,11 +45800,11 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (typeof result !== 'string') return;
const typed = whole(result);
if (!placeholder || normalize(typed) !== placeholder) return;
- const ref = [args?.ref_id, response.ref_id].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';
+ const ref = [args?.ref_id, response.ref_id, targetRef].map(value => String(value || '')).find(value => /^ref_\d+$/.test(value)) || '';
+ if (!ref) return;
// Kept as the probe returns a whole value: its first 4,000 characters,
// cut before the case is folded, as the probe cuts it.
- const write = { value: normalize(typed.slice(0, 4000)), ref, name: String(meta.name || ''), id: String(meta.id || '') };
- if (!write.ref && !write.name && !write.id) return;
+ const write = { value: normalize(typed.slice(0, 4000)), ref };
const task = this._taskTokens.get(tabId);
if (!task) return;
this._placeholderWrites ??= new Map();
@@ -45809,16 +45814,13 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
// Whether WebBrain's own write in this task put `value` into this probed
- // field: the same ref when both have one, otherwise the same name and id.
+ // field, the one with the same ref.
_placeholderWritten(tabId, field, value) {
const entry = this._placeholderWrites?.get(tabId);
const task = this._taskTokens.get(tabId);
if (!entry || !task || entry.task !== task) return false;
const ref = String(field?.ref || '');
- const name = String(field?.name || '');
- const id = String(field?.id || '');
- return entry.writes.some(write => write.value === value && (write.ref && ref ? write.ref === ref
- : (write.name || write.id) && write.name === name && write.id === id));
+ return !!ref && entry.writes.some(write => write.value === value && write.ref === ref);
}
// Adds values to the tab's known credentials, keeping the most recent 64.
diff --git a/test/run.js b/test/run.js
index 1815aeb33..b946b6fa9 100644
--- a/test/run.js
+++ b/test/run.js
@@ -82781,18 +82781,29 @@ test('a field holding its own placeholder text is not submitted', async () => {
await submit('other-field');
assert.equal(prompts, 7, `${AgentClass.name}: a write to one field counted for another`);
// The typing tools add to what a field holds unless clear is true: the
- // receipt's result is what the field then holds, not the typed text.
+ // receipt's result is what the field then holds, not the typed text. A
+ // write by selector or focus is kept by the ref its target check
+ // resolved.
agent._taskTokens.set(tabId, 'task_append');
const appendMeta = { type: 'email', name: 'email', id: null, placeholder: '[email protected]' };
- const appendField = { ref: '', name: 'email', id: '' };
+ const appendField = { ref: 'ref_404', name: 'email', id: '' };
agent._rememberPlaceholderWrite(tabId, 'type_text', { text: '[email protected]' },
- { success: true, value: '[email protected]', fieldMeta: appendMeta });
+ { success: true, value: '[email protected]', fieldMeta: appendMeta }, 'ref_404');
assert.equal(agent._placeholderWritten(tabId, appendField, '[email protected]'), false,
`${AgentClass.name}: an append was read as replacing the value`);
agent._rememberPlaceholderWrite(tabId, 'type_text', { text: 'example.com' },
- { success: true, value: '[email protected]', fieldMeta: appendMeta });
+ { success: true, value: '[email protected]', fieldMeta: appendMeta }, 'ref_404');
assert.equal(agent._placeholderWritten(tabId, appendField, '[email protected]'), true,
`${AgentClass.name}: an append that left the placeholder was missed`);
+ // A name and id are not the field: another field (another form, page
+ // or frame) can have the same ones. Only the ref counts, and a write
+ // without one is not kept.
+ assert.equal(agent._placeholderWritten(tabId, { ref: 'ref_505', name: 'email', id: '' }, '[email protected]'), false,
+ `${AgentClass.name}: a write counted for another field with the same name`);
+ agent._taskTokens.set(tabId, 'task_no_ref');
+ agent._rememberPlaceholderWrite(tabId, 'type_text', { text: '[email protected]', clear: true },
+ { success: true, value: '[email protected]', fieldMeta: appendMeta });
+ assert.equal(agent._placeholderWrites.get(tabId)?.task, 'task_append', `${AgentClass.name}: a write without a ref was kept`);
field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false, ref: 'ref_101', name: 'email', id: '' };
// A long typed value is matched by the first 200 characters the probe
// reports, though the write's receipt shortens what it read back.