Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Check every field for placeholders, with confirmations off too
Claude200b57fe08d5
diff --git a/docs/security-model.md b/docs/security-model.mdindex edcec7b21..236cf36c3 100644--- a/docs/security-model.md+++ b/docs/security-model.md@@ -223,7 +223,7 @@ The primary threat: a malicious page crafts content that, when read by the agent| **`done()` verification** | Decision and tool-free LLM checks prefer vision, then AX. Confident pending/failed outcomes require read-only recovery before another submission. Accepted task-bound proof can satisfy generic evidence despite an unrelated comment form, while explicit recipient, payment, download, authorization and workflow contracts remain required. Existing dialog/form checks are the final fallback. || **Grouped search approval** | A same-site GET form with no credential, payment or hidden token field, and no side-effect action address, shares one task-scoped approval for its navigation, typing and submit when the page's structure marks it as a search: a search landmark, a search-typed field or one whose keyboard says search, code names such as `q` or `query`, a search address, or a submit control a model read, in any language, as only searching or filtering. Words in a field's visible label decide nothing. Every other form keeps its fresh confirmation. || **Duplicate-submit guard** | A second text click on the same caption within 45 seconds on the same tab and URL is blocked when a model read the caption, in any language, as committing something (sending, saving, paying, publishing, creating, confirming and so on). A caption read as anything else, or that could not be read, stays repeatable; app-observed popups, links, fields and draft tools are always repeatable. Both builds. |-| **Placeholder-value guard** | A form submission whose changed field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless one of the user's messages or card answers contains that value. The model is told to ask the user for it. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. |+| **Placeholder-value guard** | A form submission whose changed field still holds that field's own placeholder text, the page's example such as `[email protected]` in an email box, is not sent and gets no card unless one of the user's messages or card answers contains that value. This holds with confirmations off and in scheduled runs too. The submit probe checks every field of the form, not only the eight its summary shows, and never reports a password. The model is told to ask the user for the value. A placeholder without a letter or shorter than three characters (a `1` in a quantity box) does not count, and grouped searches keep their approval. Both builds. || **CLICK occlusion test** | Before clicking, the resolver calls `elementFromPoint()`. If another element is visually on top, the click is refused. || **Modal-scoped click** | When a dialog is open, text clicks are scoped to that subtree so the agent doesn't click a dimmed background element. || **Universal preamble** | Every system prompt includes guidance on cookie banners and paywalls — two common injection vectors that look like benign page content. |diff --git a/package.json b/package.jsonindex 99cdd3337..d8be35c9f 100644--- a/package.json+++ b/package.json@@ -100,7 +100,7 @@"test:upload-owner": "node --test test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs","test:shard:quick": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/guard-policy.mjs test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-bridge-registration.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:provider-instances:ui && npm run test:settings-pages:ui && npm run test:openrouter-capabilities && npm run test:anthropic && npm run test:browser-batch && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs","test:shard:runtime-unit": "node --test test/skill-download-wait.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-click-guard.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/form-validation-alert-readings.mjs test/control-effect-readings.mjs test/guard-reading-carry-over.mjs test/trusted-retry-readings.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/guard-advisory-writes.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/codemirror-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/clarify-timeout-policy.mjs test/guard-ask-cards.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/page-feedback-unbindable-target.mjs test/guard-advisory-actions.mjs test/rich-text-preflight-page-change.mjs test/keyboard-receiver-frame.mjs test/ask-response-scope.mjs test/auto-mode.mjs test/model-picker-icon.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/research-escalation-page-feedback.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/workflow-field-readings.mjs test/workflow-field-anchors.mjs test/user-turn-kind.mjs test/follow-up-task-context.mjs test/completion-current-execution-evidence.mjs test/video-upload-attachment.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/incomplete-completion-recovery.mjs test/guard-advisory-completion.mjs test/lean-mode.mjs test/prompt-unverified-pages.mjs test/prompt-missing-values.mjs test/lifecycle-planner-digest.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-vision-fallback.mjs test/delivery-click-locks.mjs test/guards-steering-receipts.mjs test/terminal-user-messages.mjs test/improver-providers.mjs test/improver-providers-agent.mjs test/improver-guards-ledger.mjs test/improver-guards-tools.mjs test/improver-clarify.mjs test/improver-clarify-scheduler.mjs test/improver-misc.mjs test/improver-forms.mjs test/improver-forms-strict-secrets.mjs test/improver-content-press-keys-hint.mjs && npm run test:plan-notifications",- "test:shard:browser-a": "node --test test/captcha-draft-content.mjs test/captcha-challenge-surfaces.mjs && node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/sidepanel-permission-gate-ui.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-chat-selection.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/x-compose-trusted-typing.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-scroll-churn.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-native-button-route.mjs test/delivery-dispatch-boundary.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/submit-probe-readings.mjs test/improver-content-account-chrome.mjs test/improver-content-recipient.mjs test/improver-content-truncated-replacement.mjs test/improver-forms-probe.mjs test/improver-misc-trace-browser.mjs test/page-feedback-held-time.mjs test/page-monitor-held-observation.mjs",+ "test:shard:browser-a": "node --test test/captcha-draft-content.mjs test/captcha-challenge-surfaces.mjs && node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/sidepanel-permission-gate-ui.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-chat-selection.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/x-compose-trusted-typing.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-scroll-churn.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-native-button-route.mjs test/delivery-dispatch-boundary.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/submit-probe-readings.mjs test/placeholder-submit-probe.mjs test/improver-content-account-chrome.mjs test/improver-content-recipient.mjs test/improver-content-truncated-replacement.mjs test/improver-forms-probe.mjs test/improver-misc-trace-browser.mjs test/page-feedback-held-time.mjs test/page-monitor-held-observation.mjs","test:shard:browser-b1": "node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs test/message-control-readings.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/integration-mv3.mjs --batch && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/ask-person-activity-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs","test:shard:browser-b2": "node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs --selector-recovery && node test/quality/upload-release-binding-mv3.mjs --sequential-assets && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/shared-action-guards-mv3.mjs && node test/quality/video-upload-attachment-mv3.mjs","test:shard:browser-c": "node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-reply-route-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs --menu-read && node test/quality/mastodon-delete-confirmation-mv3.mjs --vague && node test/quality/mastodon-delete-after-correction-mv3.mjs && node test/quality/unbound-deletion-mv3.mjs && node test/quality/recipient-clarify-mv3.mjs && node test/quality/open-chat-recipients-mv3.mjs && node test/quality/correction-cross-site-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/x-compose-dialog-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 395f5453c..180e04e85 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -16446,7 +16446,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dpredispatch.add(stage, Date.now() - started);}};- const runWorkflowPreSubmitChecks = async () => await timedCheck('resourceMutationMs',+ const runWorkflowPreSubmitChecks = async () => this._placeholderSubmitBlock(tabId, fnName, detectedSubmitAction)+ || await timedCheck('resourceMutationMs',() => this._resourceMutationPreSubmitBlock(tabId, fnName, fnArgs, detectedSubmitAction, provider))|| this._steeringPriorSubmissionBlock(tabId, fnName, fnArgs, detectedSubmitAction)|| await timedCheck('githubTransitionMs',@@ -16621,31 +16622,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst submitRisk = classifySubmitRisk(submitConfirmation, submitConfirmation.url);const canGroupSearch = submitRisk.risk === SubmitRisk.LOW_RISK_SEARCH&& ['click', 'click_ax', 'press_keys'].includes(fnName);- // A field that still holds the page's own placeholder text, which- // no message or card answer from the user gave, is the page's- // example typed in as a value ("[email protected]" in an email box).- // Nothing is submitted, no card is shown, and the model is told to- // ask the user for the value. Grouped searches keep their approval.- const placeholderFields = canGroupSearch ? [] : this._placeholderValueFields(tabId, submitConfirmation);- if (placeholderFields.length) {- messages.push({- role: 'tool',- tool_call_id: tc.id,- content: JSON.stringify({- success: false,- denied: true,- placeholderValue: true,- placeholderFieldCount: placeholderFields.length,- error: `Nothing was submitted on ${submitConfirmation.host || 'this site'}: ${placeholderFields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `- + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '- + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',- }),- });- onUpdate('warning', { message: 'Not submitted: a field holds the page\'s example text, not a value you gave.' });- predispatchBlocked('submitConfirmationMs');- if (interruptFailedBrowserAction(toolIndex, fnName)) { navNotices.length = 0; break; }- continue;- }let choice = null;if (canGroupSearch) {const intentKey = submitActionKey(submitConfirmation, submitConfirmation.url);@@ -32448,14 +32424,42 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dreturn null;}+ // A submission whose changed field still holds the page's own placeholder+ // text, which no message or card answer from the user gave, is the page's+ // example typed in as a value ("[email protected]" in an email box). It is+ // refused before any card, whether or not the user asked to confirm+ // submissions (a scheduled run included), and the model is told to ask+ // the user for the value. Grouped searches keep their approval.+ _placeholderSubmitBlock(tabId, toolName, submitInfo) {+ if (submitInfo?.isSubmit !== true) return null;+ if (classifySubmitRisk(submitInfo, submitInfo.url).risk === SubmitRisk.LOW_RISK_SEARCH+ && ['click', 'click_ax', 'press_keys'].includes(toolName)) return null;+ const fields = this._placeholderValueFields(tabId, submitInfo);+ if (!fields.length) return null;+ return {+ success: false,+ denied: true,+ dispatched: false,+ noDispatch: true,+ placeholderValue: true,+ placeholderFieldCount: fields.length,+ error: `Nothing was submitted on ${submitInfo.host || 'this site'}: ${fields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `+ + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '+ + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',+ };+ }+// The changed fields of a submission that hold their own placeholder text// when none of the user's messages or card answers gave that value: the- // page's example typed in as the user's. A placeholder without a letter,- // or shorter than three characters (a "1" in a quantity box), is not an- // example of this kind.+ // page's example typed in as the user's. The probe's complete list is+ // read when it sent one; the summary's changed fields stop at eight. A+ // placeholder without a letter, or shorter than three characters (a "1"+ // in a quantity box), is not an example of this kind._placeholderValueFields(tabId, submitInfo) {const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();- const fields = (Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : []).filter(field => {+ const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields+ : Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];+ const fields = candidates.filter(field => {const value = normalize(field?.value);return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);});@@ -32602,6 +32606,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dfields: Array.isArray(detected.fields) ? detected.fields.slice(0, 12) : [],hiddenFields: Array.isArray(detected.hiddenFields) ? detected.hiddenFields.slice(0, 20) : [],changedFields: Array.isArray(detected.changedFields) ? detected.changedFields.slice(0, 8) : [],+ ...(Array.isArray(detected.placeholderFields) ? { placeholderFields: detected.placeholderFields.slice(0, 16) } : {}),method: String(detected.method || 'GET').slice(0, 20).toUpperCase(),action: String(detected.action || '').slice(0, 300),actionUrl: String(detected.actionUrl || detected.action || '').slice(0, 300),@@ -33029,6 +33034,24 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dtry { return !!form.closest?.('[role="search"],search'); } catch { return false; }})();const changedFields = fields.filter(field => field.changed || field.value).slice(0, 8);+ // Every changed field that still holds its own placeholder text, for+ // the placeholder guard: unlike the summary above, not capped.+ // Passwords and other fields without a typed value are left out.+ const placeholderFields = (() => {+ try {+ return Array.from(form.querySelectorAll('input, textarea')).slice(0, 500).flatMap(el => {+ const type = String(el.type || '').toLowerCase();+ if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];+ const placeholder = compact(el.getAttribute?.('placeholder') || '', 200);+ const value = pendingEl && el === pendingEl ? compact(pendingValue, 200) : compact(el.value || '', 200);+ const changed = el === pendingEl || (!!value && value !== defaultValue(el));+ return changed && placeholder && value.toLowerCase() === placeholder.toLowerCase()+ ? [{ label: labelFor(el), value, placeholder, changed: true }] : [];+ }).slice(0, 16);+ } catch {+ return [];+ }+ })();const changedText = changedFields.length? `Changed/filled fields: ${changedFields.map(field => `${field.label}: ${field.value || '(blank)'}`).join('; ')}.`: 'No changed or filled fields were detected.';@@ -33044,6 +33067,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dfields: fields.slice(0, 12),hiddenFields,changedFields,+ placeholderFields,method,action,actionUrl,diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 884aaa44a..a37151f80 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -14630,7 +14630,8 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dpredispatch.add(stage, Date.now() - started);}};- const runWorkflowPreSubmitChecks = async () => await timedCheck('resourceMutationMs',+ const runWorkflowPreSubmitChecks = async () => this._placeholderSubmitBlock(tabId, fnName, detectedSubmitAction)+ || await timedCheck('resourceMutationMs',() => this._resourceMutationPreSubmitBlock(tabId, fnName, fnArgs, detectedSubmitAction, provider))|| this._steeringPriorSubmissionBlock(tabId, fnName, fnArgs, detectedSubmitAction)|| await timedCheck('githubTransitionMs',@@ -14797,31 +14798,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dconst submitRisk = classifySubmitRisk(submitConfirmation, submitConfirmation.url);const canGroupSearch = submitRisk.risk === SubmitRisk.LOW_RISK_SEARCH&& ['click', 'click_ax', 'press_keys'].includes(fnName);- // A field that still holds the page's own placeholder text, which- // no message or card answer from the user gave, is the page's- // example typed in as a value ("[email protected]" in an email box).- // Nothing is submitted, no card is shown, and the model is told to- // ask the user for the value. Grouped searches keep their approval.- const placeholderFields = canGroupSearch ? [] : this._placeholderValueFields(tabId, submitConfirmation);- if (placeholderFields.length) {- messages.push({- role: 'tool',- tool_call_id: tc.id,- content: JSON.stringify({- success: false,- denied: true,- placeholderValue: true,- placeholderFieldCount: placeholderFields.length,- error: `Nothing was submitted on ${submitConfirmation.host || 'this site'}: ${placeholderFields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `- + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '- + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',- }),- });- onUpdate('warning', { message: 'Not submitted: a field holds the page\'s example text, not a value you gave.' });- predispatchBlocked('submitConfirmationMs');- if (interruptFailedBrowserAction(toolIndex, fnName)) { navNotices.length = 0; break; }- continue;- }let choice = null;if (canGroupSearch) {const intentKey = submitActionKey(submitConfirmation, submitConfirmation.url);@@ -29436,14 +29412,42 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dreturn null;}+ // A submission whose changed field still holds the page's own placeholder+ // text, which no message or card answer from the user gave, is the page's+ // example typed in as a value ("[email protected]" in an email box). It is+ // refused before any card, whether or not the user asked to confirm+ // submissions (a scheduled run included), and the model is told to ask+ // the user for the value. Grouped searches keep their approval.+ _placeholderSubmitBlock(tabId, toolName, submitInfo) {+ if (submitInfo?.isSubmit !== true) return null;+ if (classifySubmitRisk(submitInfo, submitInfo.url).risk === SubmitRisk.LOW_RISK_SEARCH+ && ['click', 'click_ax', 'press_keys'].includes(toolName)) return null;+ const fields = this._placeholderValueFields(tabId, submitInfo);+ if (!fields.length) return null;+ return {+ success: false,+ denied: true,+ dispatched: false,+ noDispatch: true,+ placeholderValue: true,+ placeholderFieldCount: fields.length,+ error: `Nothing was submitted on ${submitInfo.host || 'this site'}: ${fields.length === 1 ? 'a field still holds its' : 'fields still hold their'} `+ + 'own placeholder text, an example the page shows, not a value the user gave. Ask the user for the real value with clarify '+ + '(require_explicit_answer:true for personal values) and type that instead. Do NOT submit the placeholder.',+ };+ }+// The changed fields of a submission that hold their own placeholder text// when none of the user's messages or card answers gave that value: the- // page's example typed in as the user's. A placeholder without a letter,- // or shorter than three characters (a "1" in a quantity box), is not an- // example of this kind.+ // page's example typed in as the user's. The probe's complete list is+ // read when it sent one; the summary's changed fields stop at eight. A+ // placeholder without a letter, or shorter than three characters (a "1"+ // in a quantity box), is not an example of this kind._placeholderValueFields(tabId, submitInfo) {const normalize = text => String(text ?? '').replace(/\s+/g, ' ').trim().toLowerCase();- const fields = (Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : []).filter(field => {+ const candidates = Array.isArray(submitInfo?.placeholderFields) ? submitInfo.placeholderFields+ : Array.isArray(submitInfo?.changedFields) ? submitInfo.changedFields : [];+ const fields = candidates.filter(field => {const value = normalize(field?.value);return field?.changed === true && value.length >= 3 && /\p{L}/u.test(value) && value === normalize(field?.placeholder);});@@ -29594,6 +29598,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dfields: Array.isArray(detected.fields) ? detected.fields.slice(0, 12) : [],hiddenFields: Array.isArray(detected.hiddenFields) ? detected.hiddenFields.slice(0, 20) : [],changedFields: Array.isArray(detected.changedFields) ? detected.changedFields.slice(0, 8) : [],+ ...(Array.isArray(detected.placeholderFields) ? { placeholderFields: detected.placeholderFields.slice(0, 16) } : {}),method: String(detected.method || 'GET').slice(0, 20).toUpperCase(),action: String(detected.action || '').slice(0, 300),actionUrl: String(detected.actionUrl || detected.action || '').slice(0, 300),@@ -29933,6 +29938,24 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dtry { return !!form.closest?.('[role="search"],search'); } catch { return false; }})();const changedFields = fields.filter(field => field.changed || field.value).slice(0, 8);+ // Every changed field that still holds its own placeholder text, for+ // the placeholder guard: unlike the summary above, not capped.+ // Passwords and other fields without a typed value are left out.+ const placeholderFields = (() => {+ try {+ return Array.from(form.querySelectorAll('input, textarea')).slice(0, 500).flatMap(el => {+ const type = String(el.type || '').toLowerCase();+ if (['hidden', 'submit', 'button', 'reset', 'image', 'checkbox', 'radio', 'file', 'password'].includes(type)) return [];+ const placeholder = compact(el.getAttribute?.('placeholder') || '', 200);+ const value = pendingEl && el === pendingEl ? compact(pendingValue, 200) : compact(el.value || '', 200);+ const changed = el === pendingEl || (!!value && value !== defaultValue(el));+ return changed && placeholder && value.toLowerCase() === placeholder.toLowerCase()+ ? [{ label: labelFor(el), value, placeholder, changed: true }] : [];+ }).slice(0, 16);+ } catch {+ return [];+ }+ })();const changedText = changedFields.length? `Changed/filled fields: ${changedFields.map(field => `${field.label}: ${field.value || '(blank)'}`).join('; ')}.`: 'No changed or filled fields were detected.';@@ -29948,6 +29971,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dfields: fields.slice(0, 12),hiddenFields,changedFields,+ placeholderFields,method,action,actionUrl,diff --git a/test/placeholder-submit-probe.mjs b/test/placeholder-submit-probe.mjsnew file mode 100644index 000000000..5629a4e22--- /dev/null+++ b/test/placeholder-submit-probe.mjs@@ -0,0 +1,61 @@+// The submit probe lists every changed field that still holds its own+// placeholder text, for the placeholder guard. Unlike the summary's changed+// fields, which stop at eight, the list covers the whole form; it never+// carries a password, and a page default WebBrain did not change is not in it.+import assert from 'node:assert/strict';+import { test } from 'node:test';+import { chromium, firefox } from 'playwright';+import { fixtureControlEffectReading, probeWithReadings } from './semantic-decision-stub.mjs';++const pageUrl = 'https://signup.example/register';++for (const build of ['chrome', 'firefox']) {+ for (const engine of ['chromium', 'firefox']) {+ test(`${build} build on ${engine}: the submit probe lists placeholder values across the whole form`, async t => {+ const { Agent } = await import(`../src/${build}/src/agent/agent.js`);+ const browser = await (engine === 'chromium' ? chromium : firefox).launch({ headless: true });+ const context = await browser.newContext({ viewport: { width: 1200, height: 900 } });+ await context.route('**/*', route => route.fulfill({ contentType: 'text/html', body: '<!doctype html><title>fixture</title>' }));+ const page = await context.newPage();+ await page.goto(pageUrl);+ const source = Agent._submitActionProbe.toString();+ const runProbe = (tool, args) => page.evaluate(({ source, tool, args }) => (0, eval)(`(${source})`)(tool, args), { source, tool, args });+ const probe = (tool, args) => probeWithReadings(runProbe, tool, args, fixtureControlEffectReading);+ try {+ await t.test('a placeholder past the summary\'s first eight fields is listed', async () => {+ await page.setContent('<form method="post" action="/register">'+ + Array.from({ length: 10 }, (_, i) => `<input name="f${i}" aria-label="Field ${i}" placeholder="Example ${i}">`).join('')+ + '<input id="email" name="email" aria-label="Email" placeholder="[email protected]">'+ + '<input id="pass" name="pass" type="password" aria-label="Password" placeholder="Password">'+ + '<input id="city" name="city" aria-label="City" placeholder="Istanbul" value="Istanbul">'+ + '<button id="go">Save</button></form>');+ await page.evaluate(() => {+ for (let i = 0; i < 10; i++) document.querySelector(`[name=f${i}]`).value = `Answer ${i}`;+ document.querySelector('#email').value = '[email protected]';+ document.querySelector('#pass').value = 'Password';+ });+ const result = await probe('click', { selector: '#go' });+ assert.equal(result?.isSubmit, true);+ assert.equal(result.changedFields.length, 8, 'the summary stops at eight');+ assert.ok(!result.changedFields.some(field => field.label === 'Email'), 'the email field is past the summary');+ assert.deepEqual(result.placeholderFields, [{ label: 'Email', value: '[email protected]', placeholder: '[email protected]', changed: true }],+ 'only the changed email field holding its placeholder; no password, no unchanged page default');+ });++ await t.test('a value set_field is about to type and submit is checked too', async () => {+ await page.setContent('<form method="post" action="/subscribe">'+ + '<input id="email" name="email" aria-label="Email" placeholder="[email protected]"><button>Subscribe</button></form>');+ // set_field names its field by ref, as the accessibility tree hands it out.+ await page.evaluate(() => { window.__wb_ax_lookup = ref => (ref === 'ref_email' ? document.querySelector('#email') : null); });+ const result = await probe('set_field', { ref_id: 'ref_email', text: '[email protected]', submit: true });+ assert.equal(result?.isSubmit, true);+ assert.equal(result.placeholderFields?.[0]?.value, '[email protected]');+ const own = await probe('set_field', { ref_id: 'ref_email', text: '[email protected]', submit: true });+ assert.deepEqual(own.placeholderFields, []);+ });+ } finally {+ await browser.close();+ }+ });+ }+}diff --git a/test/run.js b/test/run.jsindex 13fab057d..ded6ef219 100644--- a/test/run.js+++ b/test/run.js@@ -82578,12 +82578,18 @@ test('agent requires fresh submit confirmation across modes and prompt sources',});test('a field holding its own placeholder text is not submitted', async () => {+ // The refusal is page-derived tool output, wrapped as untrusted content.+ const toolJson = content => {+ const text = String(content || '');+ return JSON.parse(text.slice(text.indexOf('{'), text.lastIndexOf('}') + 1));+ };for (const AgentClass of [AgentCh, AgentFx]) {const agent = new AgentClass({ getVisionProvider: async () => null });const tabId = 5108;let prompts = 0;let executed = 0;let field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };+ let completeList = null;const updates = [];const task = [{ role: 'user', content: 'Sign me up for the newsletter on this page.' }];@@ -82606,6 +82612,7 @@ test('a field holding its own placeholder text is not submitted', async () => {summary: `Changed/filled fields: ${field.label}: ${field.value}.`,fields: [field],changedFields: [field],+ ...(completeList ? { placeholderFields: completeList } : {}),});agent._promptSubmitConfirmation = async () => {prompts += 1;@@ -82627,7 +82634,7 @@ test('a field holding its own placeholder text is not submitted', async () => {new Set(['click_ax']),1,);- return JSON.parse(messages[0].content);+ return toolJson(messages[0].content);};const refused = await submit('placeholder');@@ -82637,15 +82644,36 @@ test('a field holding its own placeholder text is not submitted', async () => {assert.equal(refused.placeholderFieldCount, 1, `${AgentClass.name}: the refusal counts the field`);assert.match(refused.error, /placeholder text/, `${AgentClass.name}: the model is told why`);assert.match(refused.error, /Ask the user/, `${AgentClass.name}: the model is told to ask`);- assert.ok(updates.some(update => /example text/.test(update.data?.message || '')), `${AgentClass.name}: the user sees why nothing was asked`);+ assert.ok(updates.some(update => update.type === 'warning' && /placeholder text/.test(update.data?.message || '')),+ `${AgentClass.name}: the user sees why nothing was asked`);++ // Turning confirmations off, globally or for a scheduled run, does not+ // let the page's example through.+ agent._skipPermissionGate = true;+ assert.equal((await submit('gate-off')).placeholderValue, true, `${AgentClass.name}: confirmations off submitted the placeholder`);+ agent._skipPermissionGate = false;+ agent.scheduledRunPolicies.set(tabId, { requireConsequentialConfirmation: false });+ assert.equal((await submit('scheduled')).placeholderValue, true, `${AgentClass.name}: a scheduled run submitted the placeholder`);+ agent.scheduledRunPolicies.delete(tabId);++ // The probe's complete list covers a field past the summary's first+ // eight changed fields.+ field = { label: 'Name', placeholder: 'Your name', value: 'Ada', changed: true };+ completeList = [{ label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true }];+ assert.equal((await submit('past-summary')).placeholderValue, true, `${AgentClass.name}: a field past the summary was missed`);+ completeList = [];+ await submit('complete-list-clean');+ assert.equal(prompts, 1, `${AgentClass.name}: an empty complete list still refused`);+ completeList = null;+ field = { label: 'Email', placeholder: '[email protected]', value: '[email protected]', changed: true };// A value the user gave, in a message or in a card answer, asks as usual.agent.conversations.set(tabId, [...task, { role: 'user', content: 'Use [email protected] for it.' }]);await submit('user-message');- assert.equal(prompts, 1, `${AgentClass.name}: a value from the user's message was refused`);+ assert.equal(prompts, 2, `${AgentClass.name}: a value from the user's message was refused`);agent.conversations.set(tabId, [{ ...task[0], webbrainTaskClarifications: [{ question: 'Which email?', answer: '[email protected]', source: 'text' }] }]);await submit('user-answer');- assert.equal(prompts, 2, `${AgentClass.name}: a value from the user's card answer was refused`);+ assert.equal(prompts, 3, `${AgentClass.name}: a value from the user's card answer was refused`);// Another value, a page default WebBrain did not change, and a placeholder// that is only a number ask as usual.@@ -82656,7 +82684,7 @@ test('a field holding its own placeholder text is not submitted', async () => {await submit('page-default');field = { label: 'Quantity', placeholder: '1', value: '1', changed: true };await submit('number');- assert.equal(prompts, 5, `${AgentClass.name}: an ordinary value skipped its card`);+ assert.equal(prompts, 6, `${AgentClass.name}: an ordinary value skipped its card`);assert.equal(executed, 0, `${AgentClass.name}: a refused or declined submission ran`);}});