Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Don't submit a field's own placeholder text as the user's value
Claude2509c56a2511
diff --git a/docs/security-model.md b/docs/security-model.mdindex bc2f939a8..edcec7b21 100644--- a/docs/security-model.md+++ b/docs/security-model.md@@ -223,6 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. || **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. || **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |+| **Placeholder-value guard** | A form submission whose changed field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless one of the user's messages or card answers contains that value. The model is told to ask the user for it. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. || **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. || **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. || **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 6f2a81642..395f5453c 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -16621,6 +16621,31 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst submitRisk = classifySubmitRisk(submitConfirmation, submitConfirmation.url);const canGroupSearch = submitRisk.risk === SubmitRisk.LOW_RISK_SEARCH&& ['click', 'click_ax', 'press_keys'].includes(fnName);+ // A field that still holds the page's own placeholder text, which+ // no message or card answer from the user gave, is the page's+ // example typed in as a value ("[email protected]" in an email box).+ // Nothing is submitted, no card is shown, and the model is told to+ // ask the user for the value. Grouped searches keep their approval.+ const placeholderFields = canGroupSearch ? [] : this._placeholderValueFields(tabId, submitConfirmation);+ if (placeholderFields.length) {+ messages.push({+ role: 'tool',+ tool_call_id: tc.id,+ content: JSON.stringify({+ success: false,+ denied: true,+ placeholderValue: true,+ placeholderFieldCount: placeholderFields.length,+ error: `Nothing was submitted on ${submitConfirmation.host || 'this site'}: ${placeholderFields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `+ + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '+ + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',+ }),+ });+ onUpdate('warning', { message: 'Not submitted: a field holds the page\'s example text, not a value you gave.' });+ predispatchBlocked('submitConfirmationMs');+ if (interruptFailedBrowserAction(toolIndex, fnName)) { navNotices.length = 0; break; }+ continue;+ }let choice = null;if (canGroupSearch) {const intentKey = submitActionKey(submitConfirmation, submitConfirmation.url);@@ -32423,6 +32448,26 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dreturn null;}+ // The changed fields of a submission that hold their own placeholder text+ // when none of the user's messages or card answers gave that value: the+ // page's example typed in as the user's. A placeholder without a letter,+ // or shorter than three characters (a "1" in a quantity box), is not an+ // example of this kind.+ _placeholderValueFields(tabId, submitInfo) {+ const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();+ const fields = (Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : []).filter(field => {+ const value = normalize(field?.value);+ return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);+ });+ if (!fields.length) return [];+ const userText = (this.conversations.get(tabId) || [])+ .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message))+ .flatMap(message => [this._plannerUserAuthoredText(message),+ ...(Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications.map(item => item?.answer) : [])])+ .map(normalize).join('\n');+ return fields.filter(field => !userText.includes(normalize(field.value)));+ }+_fallbackSubmitConfirmationInfo(host, tool, reason, summary = '') {const normalizedHost = normalizeHost(host || '') || String(host || '').trim() || 'this site';return {diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 71cbfa5be..884aaa44a 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -14797,6 +14797,31 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst submitRisk = classifySubmitRisk(submitConfirmation, submitConfirmation.url);const canGroupSearch = submitRisk.risk === SubmitRisk.LOW_RISK_SEARCH&& ['click', 'click_ax', 'press_keys'].includes(fnName);+ // A field that still holds the page's own placeholder text, which+ // no message or card answer from the user gave, is the page's+ // example typed in as a value ("[email protected]" in an email box).+ // Nothing is submitted, no card is shown, and the model is told to+ // ask the user for the value. Grouped searches keep their approval.+ const placeholderFields = canGroupSearch ? [] : this._placeholderValueFields(tabId, submitConfirmation);+ if (placeholderFields.length) {+ messages.push({+ role: 'tool',+ tool_call_id: tc.id,+ content: JSON.stringify({+ success: false,+ denied: true,+ placeholderValue: true,+ placeholderFieldCount: placeholderFields.length,+ error: `Nothing was submitted on ${submitConfirmation.host || 'this site'}: ${placeholderFields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `+ + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '+ + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',+ }),+ });+ onUpdate('warning', { message: 'Not submitted: a field holds the page\'s example text, not a value you gave.' });+ predispatchBlocked('submitConfirmationMs');+ if (interruptFailedBrowserAction(toolIndex, fnName)) { navNotices.length = 0; break; }+ continue;+ }let choice = null;if (canGroupSearch) {const intentKey = submitActionKey(submitConfirmation, submitConfirmation.url);@@ -29411,6 +29436,26 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dreturn null;}+ // The changed fields of a submission that hold their own placeholder text+ // when none of the user's messages or card answers gave that value: the+ // page's example typed in as the user's. A placeholder without a letter,+ // or shorter than three characters (a "1" in a quantity box), is not an+ // example of this kind.+ _placeholderValueFields(tabId, submitInfo) {+ const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();+ const fields = (Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : []).filter(field => {+ const value = normalize(field?.value);+ return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);+ });+ if (!fields.length) return [];+ const userText = (this.conversations.get(tabId) || [])+ .filter(message => message?.role === 'user' && !this._isAgentInjectedUserMessage(message))+ .flatMap(message => [this._plannerUserAuthoredText(message),+ ...(Array.isArray(message.webbrainTaskClarifications) ? message.webbrainTaskClarifications.map(item => item?.answer) : [])])+ .map(normalize).join('\n');+ return fields.filter(field => !userText.includes(normalize(field.value)));+ }+_fallbackSubmitConfirmationInfo(host, tool, reason, summary = '') {const normalizedHost = normalizeHost(host || '') || String(host || '').trim() || 'this site';return {diff --git a/test/run.js b/test/run.jsindex 64e262790..13fab057d 100644--- a/test/run.js+++ b/test/run.js@@ -82577,6 +82577,90 @@ test('agent requires fresh submit confirmation across modes and prompt sources',}});+test('a field holding its own placeholder text is not submitted', async () => {+ for (const AgentClass of [AgentCh, AgentFx]) {+ const agent = new AgentClass({ getVisionProvider: async () => null });+ const tabId = 5108;+ let prompts = 0;+ let executed = 0;+ let field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };+ const updates = [];+ const task = [{ role: 'user', content: 'Sign me up for the newsletter on this page.' }];++ agent.conversationModes.set(tabId, 'act');+ agent.conversations.set(tabId, [...task]);+ agent._ensureGateSetting = async () => { };+ agent._skipPermissionGate = false;+ agent._currentUrl = async () => 'https://host.com/newsletter';+ agent._recordProgressObservation = async () => null;+ agent._autoRecordProgressAction = () => null;+ agent._progressWarningForAction = () => '';+ agent._persist = () => { };+ agent._detectLikelySubmitAction = async (_tabId, name) => ({+ isSubmit: true,+ host: 'host.com',+ tool: name,+ method: 'POST',+ actionUrl: 'https://host.com/subscribe',+ reason: 'submit button/control activation',+ summary: `Changed/filled fields: ${field.label}: ${field.value}.`,+ fields: [field],+ changedFields: [field],+ });+ agent._promptSubmitConfirmation = async () => {+ prompts += 1;+ return 'deny';+ };+ agent.executeTool = async () => {+ executed += 1;+ return { success: true };+ };+ const submit = async (id) => {+ const messages = [];+ await agent._executeToolBatch(+ tabId,+ [{ id, function: { name: 'click_ax', arguments: JSON.stringify({ ref_id: 'ref_subscribe' }) } }],+ messages,+ (type, data) => updates.push({ type, data }),+ { supportsVision: false },+ '',+ new Set(['click_ax']),+ 1,+ );+ return JSON.parse(messages[0].content);+ };++ const refused = await submit('placeholder');+ assert.equal(prompts, 0, `${AgentClass.name}: the page's example value reached a submit card`);+ assert.equal(refused.denied, true, `${AgentClass.name}: the placeholder submission is refused`);+ assert.equal(refused.placeholderValue, true, `${AgentClass.name}: the refusal names the placeholder`);+ assert.equal(refused.placeholderFieldCount, 1, `${AgentClass.name}: the refusal counts the field`);+ assert.match(refused.error, /placeholder text/, `${AgentClass.name}: the model is told why`);+ assert.match(refused.error, /Ask the user/, `${AgentClass.name}: the model is told to ask`);+ assert.ok(updates.some(update => /example text/.test(update.data?.message || '')), `${AgentClass.name}: the user sees why nothing was asked`);++ // A value the user gave, in a message or in a card answer, asks as usual.+ agent.conversations.set(tabId, [...task, { role: 'user', content: 'Use [email protected] for it.' }]);+ await submit('user-message');+ assert.equal(prompts, 1, `${AgentClass.name}: a value from the user's message was refused`);+ agent.conversations.set(tabId, [{ ...task[0], webbrainTaskClarifications: [{ question: 'Which email?', answer: '[email protected]', source: 'text' }] }]);+ await submit('user-answer');+ assert.equal(prompts, 2, `${AgentClass.name}: a value from the user's card answer was refused`);++ // Another value, a page default WebBrain did not change, and a placeholder+ // that is only a number ask as usual.+ agent.conversations.set(tabId, [...task]);+ field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };+ await submit('own-value');+ field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: false };+ await submit('page-default');+ field = { label: 'Quantity', placeholder: '1', value: '1', changed: true };+ await submit('number');+ assert.equal(prompts, 5, `${AgentClass.name}: an ordinary value skipped its card`);+ assert.equal(executed, 0, `${AgentClass.name}: a refused or declined submission ran`);+ }+});+test('submit confirmation honors scheduled and global gate bypasses', async () => {for (const AgentClass of [AgentCh, AgentFx]) {for (const scenario of [