Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Wait for the agent's own hover to settle before pressing a native button

Claudeea9cf1a38241
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c0eab3c85..fb926a200 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -25,6 +25,7 @@ This changelog was generated from the repository Git history and release tags. V
- A Delete or Remove you ask for on an ordinary row without its own link (a cart item, a mail row, a table row) is no longer always refused. It proceeds when your latest request asks for the deletion and names that row by a word only that row shows, including the confirmation dialog the click opens. The same row is never deleted twice; toolbar and bulk deletions, menus without a linked button, and posts with a permalink keep their previous checks.
- Ordinary controls (filter chips, folder links, menus, tabs, toggles, draft attachment and media removal, composer tools) work before and after a correction; X and Bluesky no longer reject them as unidentified publication actions. After a correction, identical resubmissions are reported as already done, a corrected in-place save or a requested post on another platform is allowed once the earlier result is known, and requests to remove characters or hashtags are no longer treated as deleting the post. A submit-probe exception that left real publish controls unresolved is fixed.
- A correction sent while a run is working is no longer lost when the extension background restarts before the run uses it. The resumed run applies it first, once, in the order it was sent; if the run does not resume, the correction returns to the queue marked as paused, as after Stop. Clearing the conversation or closing the tab discards it.
+- Buttons that animate when the pointer reaches them (a hover lift, scale or border transition) are no longer refused once before working: the click waits up to 0.6 seconds, without sending anything, until the button stops moving, then presses it once at its new centre. A button that keeps moving, is covered or replaced, or any input or Stop during that wait still cancels the click.
## [40.3.0] - 2026-10-12
diff --git a/docs/page-feedback-monitor.md b/docs/page-feedback-monitor.md
index 36bb960b7..44f20aafc 100644
--- a/docs/page-feedback-monitor.md
+++ b/docs/page-feedback-monitor.md
@@ -95,8 +95,17 @@ Steer or grant authorization. Enter and Alt+Enter retain their existing behavior
acknowledgements, interrupted authority and unresolved reads retain repeat
locks; observing effects never sends a second click or clears queued feedback.
The agent's own hover may move a certified native button by up to 8 px
- (lift, border or scale); it is then pressed at its re-verified current
- centre, and any other change refuses before input.
+ (lift, border or scale), also through a CSS transition or a page hover
+ animation. Without sending input, the agent then samples that exact button
+ read-only, at most 600 ms, until two consecutive samples have the same
+ rectangle and the same hit node at its centre and no finite geometry
+ transition is still running on it or an ancestor. The settled button is
+ re-verified, the monitor fence refreshes its geometry proof only for the
+ same node with identical semantics and hit, and it is pressed once at that
+ settled centre. A button that keeps moving (including an infinite
+ animation), a covering, hidden or replaced node, a changed document or
+ route, a semantic change, a move beyond 8 px from the prepared centre, any
+ user input or Stop during the wait refuses before input.
- Trusted CDP input is marked dispatched only at the transport boundary, after
the receiving-node and page-feedback fences pass. A rejection before the
protocol call means nothing was sent; any later failure means the input was
diff --git a/package.json b/package.json
index 52795c599..750ca53ac 100644
--- a/package.json
+++ b/package.json
@@ -60,7 +60,7 @@
"test:social-contract": "node --test test/social-publish-contract.mjs test/publication-quote-markup.mjs",
"test:social-contract:dom": "node test/social-publish-contract-dom.mjs",
"test:runtime-lifecycle": "node test/bot-memory-dom.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/lifecycle-steering-restart.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/completion-current-execution-evidence.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-vision-fallback.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/guards-steering-receipts.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs && node test/message-recipient-upload.mjs && node --test test/message-recipient-whatsapp.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs && node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/unbound-deletion-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",
- "test:page-feedback": "node --test test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-passive-progress.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/delivery-click-ax-response-loss.mjs test/delivery-stop-compound.mjs test/delivery-cdp-input.mjs test/delivery-click-locks.mjs test/delivery-native-button-geometry.mjs && node test/message-recipient-upload.mjs && node --test test/message-recipient-whatsapp.mjs",
+ "test:page-feedback": "node --test test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-passive-progress.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/delivery-click-ax-response-loss.mjs test/delivery-stop-compound.mjs test/delivery-cdp-input.mjs test/delivery-click-locks.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs && node test/message-recipient-upload.mjs && node --test test/message-recipient-whatsapp.mjs",
"test:chat-steering": "node --test test/chat-steering.mjs test/chat-steering-ui.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs",
"test:rich-text-dom": "node test/ui-content-lifecycle.mjs --dom",
"test:firefox-bidi": "node --test test/firefox-bidi.mjs test/firefox-full-page-screenshot.mjs",
@@ -92,7 +92,7 @@
"test:upload-owner": "node --test test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs",
"test:shard:quick": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-bridge-registration.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:provider-instances:ui && npm run test:openrouter-capabilities && npm run test:anthropic && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs",
"test:shard:runtime-unit": "node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-click-guard.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/completion-current-execution-evidence.mjs test/video-upload-attachment.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-vision-fallback.mjs test/delivery-click-locks.mjs test/guards-steering-receipts.mjs test/terminal-user-messages.mjs",
- "test:shard:browser-a": "node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs",
+ "test:shard:browser-a": "node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs",
"test:shard:browser-b1": "node test/message-recipient-upload.mjs && node --test test/message-recipient-whatsapp.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs",
"test:shard:browser-b2": "node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/shared-action-guards-mv3.mjs && node test/quality/video-upload-attachment-mv3.mjs",
"test:shard:browser-c": "node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/unbound-deletion-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index d14fed0d0..789eb72bf 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -501,6 +501,11 @@ const VISION_SUB_CALL_TIMEOUT_MS = 90_000;
const CONTENT_ACTION_TIMEOUT_MS = 60_000;
const RUN_STARTUP_TIMEOUT_MS = 15_000;
const CONTENT_ACTION_RESPONSE_GRACE_MS = 5_000;
+// The agent's own hover may start a transition on a certified native button.
+// Its geometry is sampled (read-only) at this interval until it settles, for
+// at most this long, before the single press.
+const NATIVE_BUTTON_HOVER_SETTLE_MS = 600;
+const NATIVE_BUTTON_HOVER_POLL_MS = 50;
const ASK_MODE_HANDOFF_TIMEOUT_MS = 5_000;
const CONTENT_ACTION_SIGNAL_DEADLINES = new WeakMap();
// A completed focus click is a browser dispatch, but need not be a text write.
@@ -18748,6 +18753,17 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
assertCurrent();
if (!await finishPointer()) return failure('The acknowledged pointer operation could not be completed for the current owner. Re-observe before acting.');
assertCurrent();
+ // The hover can start a CSS transition or a page hover animation on this
+ // exact button. Wait, bounded and sending no input, until its geometry
+ // and hit node are stable, so the re-probe, fence and press below all see
+ // the settled button. Any other change, input or Stop refuses here.
+ preparationStage = 'settle_after_pointer';
+ const settled = await this._settleTrustedNativeButtonHover(tabId, args, prepared, messageOptions, assertCurrent, run);
+ if (settled.success !== true) {
+ if (timing) timing.preparationVeto = { stage: preparationStage, predicate: settled.predicate };
+ return failure(settled.error);
+ }
+ assertCurrent();
// Hover or asynchronous preparation must not change the exact target,
// private model footprint, document, route or chosen physical point.
preparationStage = 'reprobe_after_pointer';
@@ -18771,12 +18787,20 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
|| Math.abs(current.x - prepared.x) > 8 || Math.abs(current.y - prepared.y) > 8) {
return failure('The native button changed during trusted input preparation. Re-observe before acting.');
}
+ // The re-probe must see the settled geometry; a later move means the
+ // button did not settle after all.
+ if (current.x !== settled.sample.x || current.y !== settled.sample.y
+ || ['x', 'y', 'w', 'h'].some(key => current.rect?.[key] !== settled.sample.roundedRect?.[key])) {
+ if (timing) timing.preparationVeto = { stage: preparationStage, predicate: 'geometry_changed' };
+ return failure('The native button moved again after its hover settled. No click was sent. Re-observe before acting.');
+ }
// The agent's own hover may lift or resize this exact certified button
- // (:hover transform, border or size). The re-probe hit-tested its current
- // centre on the same node; the monitor fence below rechecks the certified
- // binding, and the press transport hit-tests the pressed point against
- // that node immediately before the press. Never press the stale
- // pre-hover point, which could now hit another node.
+ // (:hover transform, border or size), also through a transition. The
+ // settled re-probe hit-tested its current centre on the same node; the
+ // monitor fence below rechecks and refreshes the certified binding at
+ // that settled geometry, and the press transport hit-tests the pressed
+ // point against that node immediately before the press. Never press the
+ // stale pre-hover point, which could now hit another node.
const pressPoint = { x: current.x, y: current.y };
preparationStage = 'after_pointer_before_press';
const inputGuard = await beforePageAgentDispatch(chrome, tabId, { kind: 'click', tool: 'click_ax', ref_id: args.ref_id,
@@ -18915,6 +18939,55 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}
}
+ // After the agent's own trusted hover over a certified native button, sample
+ // the exact button read-only until two consecutive samples have the same
+ // rectangle and the same hit node at its centre, with no finite geometry
+ // transition still running. Sends no input. Bounded: a target that keeps
+ // moving, any input, Stop, a replaced or covered node, or a changed document
+ // refuses before the press.
+ async _settleTrustedNativeButtonHover(tabId, args, prepared, messageOptions, assertCurrent, run) {
+ const startedAt = Date.now();
+ const settleSession = globalThis.crypto.randomUUID();
+ const userAt = run.lastUserAt;
+ const intervened = () => run.lastUserAt !== userAt || [...run.events.values()].some(event => event.source === 'user');
+ const interventionRefusal = { success: false, predicate: 'intervention_changed',
+ error: 'Input was observed while the native button settled after the pointer move. No click was sent. Re-observe before acting.' };
+ const pause = ms => new Promise(resolve => {
+ const signal = this._runAbortSignal?.(tabId);
+ let timer = null;
+ const done = () => { clearTimeout(timer); signal?.removeEventListener?.('abort', done); resolve(); };
+ timer = setTimeout(done, ms);
+ signal?.addEventListener?.('abort', done, { once: true });
+ });
+ let previous = null;
+ for (let samples = 1; ; samples++) {
+ assertCurrent();
+ if (intervened()) return interventionRefusal;
+ const sample = await chrome.tabs.sendMessage(tabId, { target: 'content', action: 'sample_trusted_native_button',
+ params: { ref_id: args.ref_id, expectedDocumentToken: prepared.documentToken, expectedPageUrl: prepared.refScopeUrl,
+ settleSession, pointer: { x: prepared.x, y: prepared.y } } }, messageOptions);
+ assertCurrent();
+ if (intervened() || sample?.userInput === true) return interventionRefusal;
+ if (sample?.success !== true || !Number.isFinite(sample.x) || !Number.isFinite(sample.y)
+ || !Array.isArray(sample.rect) || sample.rect.length !== 4) {
+ return { success: false,
+ predicate: ['target_hit_changed', 'document_changed', 'run_ended'].includes(sample?.predicate) ? sample.predicate : 'target_changed',
+ error: typeof sample?.error === 'string' && sample.error.length <= 300 ? sample.error
+ : 'The native button could not be verified while its hover settled. No click was sent. Re-observe before acting.' };
+ }
+ if (previous && sample.animating !== true && sample.hit === previous.hit
+ && sample.rect.every((value, index) => value === previous.rect[index])) {
+ return { success: true, sample, samples, settleMs: Date.now() - startedAt };
+ }
+ previous = sample;
+ if (Date.now() - startedAt + NATIVE_BUTTON_HOVER_POLL_MS > NATIVE_BUTTON_HOVER_SETTLE_MS) {
+ return { success: false, predicate: 'geometry_changed',
+ error: 'The native button was still moving after the pointer reached it. No click was sent. Re-observe before acting.' };
+ }
+ await pause(NATIVE_BUTTON_HOVER_POLL_MS);
+ }
+ }
+
async _dispatchClickAx(
tabId,
args,
diff --git a/src/chrome/src/content/content.js b/src/chrome/src/content/content.js
index 40491b123..b974071f4 100644
--- a/src/chrome/src/content/content.js
+++ b/src/chrome/src/content/content.js
@@ -6673,6 +6673,99 @@
} catch { return null; }
};
+ // --- Trusted native button hover settle (read-only) ---
+ // After the agent's own trusted hover, the agent samples the exact certified
+ // button until its geometry is stable before the press. Sampling never
+ // focuses, scrolls, dispatches or starts a monitor operation. Hit nodes are
+ // compared by private isolated-world tokens, never by page data.
+ const _nativeButtonHitTokens = new WeakMap();
+ let _nativeButtonHitTokenSeq = 0;
+ let _nativeButtonSettleWatch = null;
+ const _nativeButtonPaintOnlyProperty = /^(?:color|opacity|box-shadow|text-shadow|filter|backdrop-filter|fill|fill-opacity|stroke|stroke-opacity|caret-color|accent-color|text-decoration-color|outline(?:-color|-width|-style|-offset)?|background(?:-color|-image|-position(?:-x|-y)?|-size)?|border(?:-(?:top|right|bottom|left|block-start|block-end|inline-start|inline-end))?-color)$/;
+
+ function _nativeButtonHitToken(node) {
+ if (!node) return 0;
+ let value = _nativeButtonHitTokens.get(node);
+ if (!value) { value = ++_nativeButtonHitTokenSeq; _nativeButtonHitTokens.set(node, value); }
+ return value;
+ }
+
+ // Physical input during the settle wait belongs to the user: the agent sends
+ // none. Pointer movement at the agent's own hover point (late or repeated
+ // browser movement events) is not intervention.
+ function _nativeButtonSettleWatchFor(session, pointer) {
+ if (_nativeButtonSettleWatch?.session === session) return _nativeButtonSettleWatch;
+ _nativeButtonSettleWatch?.stop();
+ const types = ['pointermove', 'mousemove', 'pointerdown', 'mousedown', 'pointerup', 'mouseup',
+ 'keydown', 'keyup', 'wheel', 'touchstart', 'touchmove', 'touchend'];
+ const watch = { session, input: false };
+ const onInput = event => {
+ if (!event.isTrusted) return;
+ if ((event.type === 'pointermove' || event.type === 'mousemove')
+ && Math.abs(event.clientX - pointer.x) <= 1 && Math.abs(event.clientY - pointer.y) <= 1) return;
+ watch.input = true;
+ };
+ for (const type of types) window.addEventListener(type, onInput, { capture: true, passive: true });
+ const timer = setTimeout(() => watch.stop(), 2000);
+ watch.stop = () => {
+ clearTimeout(timer);
+ for (const type of types) window.removeEventListener(type, onInput, true);
+ if (_nativeButtonSettleWatch === watch) _nativeButtonSettleWatch = null;
+ };
+ _nativeButtonSettleWatch = watch;
+ return watch;
+ }
+
+ // A finite running transition/animation that can still move or resize the
+ // button (on it or an ancestor) means it has not settled, even when two
+ // samples happen to match (a delay phase or a sub-pixel easing tail).
+ // Infinite animations never settle by waiting; their geometry samples decide.
+ function _nativeButtonGeometryAnimating(el) {
+ let animations = [];
+ try { animations = document.getAnimations(); } catch { return false; }
+ return animations.some(animation => {
+ try {
+ const target = animation.effect?.target;
+ if (animation.playState !== 'running' || !(target instanceof Element) || animation.effect.pseudoElement
+ || !_isComposedAncestor(target, el)) return false;
+ if (animation.effect.getComputedTiming?.().iterations === Infinity) return false;
+ const properties = typeof animation.transitionProperty === 'string' ? [animation.transitionProperty]
+ : (animation.effect.getKeyframes?.() || []).flatMap(frame => Object.keys(frame))
+ .filter(key => !['offset', 'computedOffset', 'easing', 'composite'].includes(key))
+ .map(key => key.startsWith('--') ? key : key.replace(/[A-Z]/g, letter => `-${letter.toLowerCase()}`));
+ return !properties.length || properties.some(property => !_nativeButtonPaintOnlyProperty.test(property));
+ } catch { return true; }
+ });
+ }
+
+ function _sampleTrustedNativeButton(params = {}) {
+ const failure = (predicate, error, extra = {}) => ({ success: false, dispatched: false, noDispatch: true, predicate, error, ...extra });
+ const { ref_id, expectedDocumentToken, expectedPageUrl, settleSession, pointer } = params;
+ if (typeof ref_id !== 'string' || typeof settleSession !== 'string' || !settleSession
+ || !Number.isFinite(pointer?.x) || !Number.isFinite(pointer?.y)) return failure('target_changed', 'Invalid native button sample.');
+ const watch = _nativeButtonSettleWatchFor(settleSession, pointer);
+ if (window.__wbPageMonitor?.active !== true) return failure('run_ended', 'The browser run ended while the native button settled. No click was sent.');
+ if ((expectedDocumentToken && expectedDocumentToken !== _axDocumentToken())
+ || (expectedPageUrl && expectedPageUrl !== location.href)) {
+ return failure('document_changed', 'The page or route changed while the native button settled. No click was sent. Re-observe before acting.');
+ }
+ const el = typeof window.__wb_ax_lookup === 'function' ? window.__wb_ax_lookup(ref_id) : null;
+ if (!el || !el.isConnected || el.tagName !== 'BUTTON') {
+ return failure('target_changed', 'The exact native button was removed or replaced while its hover settled. No click was sent. Re-observe before acting.');
+ }
+ const rect = el.getBoundingClientRect();
+ const x = Math.round(rect.left + rect.width / 2), y = Math.round(rect.top + rect.height / 2);
+ const hit = _shadowAwareElementFromPoint(x, y);
+ if (!_axFallbackVisibility(el).visible || x < 0 || y < 0 || x >= innerWidth || y >= innerHeight
+ || !hit || !_isComposedAncestor(el, hit) || _axInteractiveHitDescendant(el, hit)) {
+ return failure('target_hit_changed', 'The native button became hidden, covered or ambiguous while its hover settled. No click was sent. Re-observe before clicking.');
+ }
+ return { success: true, dispatched: false, noDispatch: true, x, y,
+ rect: [rect.left, rect.top, rect.width, rect.height],
+ roundedRect: { x: Math.round(rect.x), y: Math.round(rect.y), w: Math.round(rect.width), h: Math.round(rect.height) },
+ hit: _nativeButtonHitToken(hit), animating: _nativeButtonGeometryAnimating(el), userInput: watch.input === true };
+ }
+
function _probeMessageRecipientGuard(params = {}, receiveConversationContext = null) {
try {
const tool = String(params.tool || '');
@@ -8954,6 +9047,9 @@
return failure(e && e.message || String(e));
}
},
+ // Read-only sample of a certified native button while the agent's own
+ // hover settles. It never focuses, scrolls or dispatches.
+ 'sample_trusted_native_button': () => _sampleTrustedNativeButton(msg.params || {}),
'set_checked': () => {
let dispatched = false;
const failure = (error, extra = {}) => ({
diff --git a/test/delivery-hover-transition.mjs b/test/delivery-hover-transition.mjs
new file mode 100644
index 000000000..87d67eeda
--- /dev/null
+++ b/test/delivery-hover-transition.mjs
@@ -0,0 +1,249 @@
+// Delivery regression: the agent's own hover over a certified native BUTTON
+// can start a CSS transition or a script-driven hover animation. The press
+// must wait, without sending any input, until the exact target's geometry is
+// stable (bounded), and is then sent once at the settled, re-verified centre.
+// Every other change during that wait still refuses before any press.
+//
+// [real DOM, Chromium only: the trusted native-button path is Chrome CDP]
+// The real Agent, page-feedback run, page monitor, accessibility tree and
+// content script run unchanged; only the extension transport and debugger
+// session are bridged to Playwright. Interventions are timed from the agent's
+// own pointer move, so they land while the hover is still settling.
+import { chromium } from 'playwright';
+import { readFile } from 'node:fs/promises';
+import { fileURLToPath, pathToFileURL } from 'node:url';
+import path from 'node:path';
+import assert from 'node:assert/strict';
+import { test } from 'node:test';
+import { cdpClient } from '../src/chrome/src/cdp/cdp-client.js';
+
+const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
+const area = { get: async () => ({}), set: async () => {}, remove: async () => {} };
+let page, session;
+const hooks = { afterHover: null };
+const commands = [];
+const requests = new Set();
+const api = {
+ storage: { local: area, session: area },
+ runtime: { getURL: p => pathToFileURL(path.join(root, 'src/chrome', p)).href },
+ tabs: {
+ get: async () => ({ id: 17, url: page.url() }),
+ query: async () => [{ id: 17, url: page.url() }],
+ sendMessage: (_id, message) => page.evaluate(message => window.qaMessage(message), message),
+ },
+ webNavigation: { getAllFrames: async () => [{ frameId: 0, documentId: 'delivery-hover-document', url: page.url() }] },
+ webRequest: { onBeforeRequest: { addListener: l => requests.add(l), removeListener: l => requests.delete(l) } },
+ debugger: { sendCommand: (_target, method, params, callback) => {
+ commands.push({ method, type: params.type, x: params.x, y: params.y });
+ session.send(method, params).then(result => {
+ // The agent's own hover (pointer move without a button) was delivered.
+ if (method === 'Input.dispatchMouseEvent' && params.type === 'mouseMoved' && params.buttons === 0 && hooks.afterHover) {
+ const intervene = hooks.afterHover; hooks.afterHover = null;
+ setTimeout(() => { void intervene(); }, 90);
+ }
+ return result;
+ }).then(callback, error => {
+ api.runtime.lastError = { message: error.message };
+ try { callback(); } finally { delete api.runtime.lastError; }
+ });
+ } },
+};
+globalThis.chrome = api;
+globalThis.browser = api;
+const { Agent } = await import('../src/chrome/src/agent/agent.js');
+
+const html = ({ css = '', script = '' } = {}) => `<!doctype html><style>
+body{margin:20px} #commit{width:200px;height:45px} ${css}</style>
+<main><h1>Current record settings</h1><button id="commit" type="button">Apply settings</button>
+<label>Record title<input value="Owned neutral draft"></label><output id="result">0</output></main>
+<script>window.qaClicks=[];window.qaPressRects=[];window.qaPressColors=[];
+const qaRect=()=>{const r=document.getElementById('commit').getBoundingClientRect();return [r.left,r.top,r.width,r.height];};
+document.addEventListener('mousedown',event=>{if(!event.isTrusted)return;qaPressRects.push(qaRect());
+qaPressColors.push(getComputedStyle(document.getElementById('commit')).backgroundColor);},true);
+commit.addEventListener('click',event=>{qaClicks.push({trusted:event.isTrusted,x:event.clientX,y:event.clientY});
+if(event.isTrusted){result.textContent=String(qaClicks.length);commit.setAttribute('aria-pressed',String(qaClicks.length%2===1));}});
+${script}</script>`;
+
+async function fixture(browserInstance, options) {
+ const context = await browserInstance.newContext({ viewport: { width: 900, height: 700 } });
+ await context.route('**/*', route => route.request().url().startsWith('https://delivery-hover.invalid/')
+ ? route.fulfill({ contentType: 'text/html', body: html(options) }) : route.abort());
+ page = await context.newPage();
+ const agent = new Agent({ getActive: () => ({ model: 'delivery-model' }) });
+ agent._runningTabs.add(17);
+ await page.exposeFunction('qaBackground', message => {
+ const sender = { tab: { id: 17 }, frameId: 0, documentId: 'delivery-hover-document', url: page.url() };
+ if (message.action === 'get_page_monitor_state') return agent.pageMonitorState(sender, message.documentToken);
+ if (message.action === 'page_feedback') return agent.observePageFeedback(sender, message.feedback);
+ return {};
+ });
+ await page.addInitScript(() => {
+ window.qaListeners = [];
+ window.chrome = window.browser = { runtime: {
+ getURL: p => p,
+ onMessage: { addListener: f => qaListeners.push(f), removeListener() {} },
+ sendMessage: message => window.qaBackground(message),
+ } };
+ window.qaMessage = message => new Promise(resolve => {
+ let answered = false;
+ const respond = value => { answered = true; resolve(value); };
+ for (const f of qaListeners) { const value = f(message, {}, respond); if (answered || value === true) return; }
+ resolve(undefined);
+ });
+ });
+ for (const name of ['rich-text-toolbar-heuristic.js', 'file-upload-policy.js', 'accessibility-tree.js', 'page-monitor.js', 'content.js']) {
+ await page.addInitScript({ content: await readFile(path.join(root, 'src/chrome/src/content', name), 'utf8') });
+ }
+ await page.goto('https://delivery-hover.invalid/current');
+ session = await context.newCDPSession(page);
+ await agent._beginPageFeedbackRun(17, 'neutral');
+ await page.waitForFunction(() => window.__wbPageMonitor?.active);
+ await page.mouse.move(5, 690);
+ await page.waitForTimeout(180);
+ const run = agent._pageFeedbackRuns.get(17);
+ run.events.clear();
+ const axScope = await api.tabs.sendMessage(17, { target: 'content', action: 'get_accessibility_tree', params: { filter: 'visible', maxChars: 12000 } });
+ const capture = await api.tabs.sendMessage(17, { target: 'content', action: 'page_monitor_capture_model',
+ params: { runToken: run.token, includeTree: true } });
+ const ref = capture.page.pageContent.match(/button "Apply settings" \[(ref_[A-Za-z0-9_-]+)\]/)?.[1];
+ assert.ok(ref, capture.page.pageContent);
+ const state = { token: run.token, url: run.url, steeringRevision: 0, frames: new Map(run.frames),
+ actionBinding: { snapshotToken: capture.snapshotToken, runToken: run.token, documentToken: capture.documentToken } };
+ run.modelState = state;
+ run.dispatchCall = { name: 'click_ax', args: { ref_id: ref }, expectedModelSnapshot: capture.snapshotToken,
+ modelState: state, dispatchState: { started: false }, allowPassiveRebase: true };
+ const prepared = await api.tabs.sendMessage(17, { target: 'content', action: 'page_monitor_prepare',
+ params: { runToken: run.token, tool: 'click_ax', ref_id: ref, operationId: 'delivery-hover-operation',
+ expectedModelSnapshot: capture.snapshotToken, allowPassiveRebase: true } });
+ assert.equal(prepared.modelBindingValid, true);
+ commands.length = 0; hooks.afterHover = null;
+ agent._settleContentFilePickerGuard = async (_id, result) => result;
+ agent._annotateClickProgress = async (_id, _name, _args, result) => result;
+ agent._recordInteractionRect = () => {};
+ agent._clearUploadSelectorRecoveryAfterInspection = () => {};
+ agent._lastAxScopes.set(17, { documentToken: axScope.documentToken, pageUrl: axScope.refScopeUrl });
+ const before = await page.locator('#commit').boundingBox();
+ return { context, agent, ref, before };
+}
+
+const presses = () => commands.filter(call => call.method === 'Input.dispatchMouseEvent' && call.type === 'mousePressed');
+const hovers = () => commands.filter(call => call.method === 'Input.dispatchMouseEvent' && call.type === 'mouseMoved');
+const centre = box => ({ x: Math.round(box.x + box.width / 2), y: Math.round(box.y + box.height / 2) });
+
+test('Chrome: a certified native button is pressed once after the agent\'s own hover settles', { timeout: 300000 }, async t => {
+ const cdpAttach = cdpClient.attach;
+ cdpClient.attach = async () => cdpClient.sessions.set(17, {});
+ const browserInstance = await chromium.launch({ headless: true });
+ try {
+ for (const [name, options, moved, check] of [
+ ['a transition lift (translateY + scale over 200 ms)', {
+ css: '#commit{transition:transform .2s ease} #commit:hover{transform:translateY(-2px) scale(1.03)}',
+ }, true],
+ ['a box-shadow and border-width transition', {
+ css: '#commit{width:auto;height:auto;border:1px solid #333;transition:border-width .2s,box-shadow .2s}'
+ + ' #commit:hover{border-width:4px;box-shadow:0 4px 12px rgba(0,0,0,.3)}',
+ }, true],
+ ['a script-driven hover animation that settles', {
+ script: "commit.addEventListener('mouseenter',()=>{const start=performance.now();"
+ + "const step=now=>{const p=Math.min(1,(now-start)/180);commit.style.transform='translateY('+(-3*p)+'px)';if(p<1)requestAnimationFrame(step);};"
+ + 'requestAnimationFrame(step);},{once:true});',
+ }, true],
+ // Two samples inside the delay match; the running transition still is not settled.
+ ['a delayed transition lift (transition-delay)', {
+ css: '#commit{transition:transform .15s ease .12s} #commit:hover{transform:translateY(-2px)}',
+ }, true],
+ // Paint-only transitions cannot move the press point and do not delay it.
+ ['a long paint-only hover transition', {
+ css: '#commit{transition:background-color 2s,box-shadow 2s,color 2s} #commit:hover{background-color:#cde;color:#036;box-shadow:0 0 8px #036}',
+ }, false, async () => {
+ const [pressColor] = await page.evaluate(() => window.qaPressColors);
+ assert.notEqual(pressColor, 'rgb(204, 221, 238)', 'pressed while the 2 s paint-only transition was still running');
+ }],
+ ['an unmoved button', {}, false],
+ ]) await t.test(`${name}: one trusted press at the settled, re-verified centre`, async () => {
+ const f = await fixture(browserInstance, options);
+ try {
+ const startedAt = Date.now();
+ const result = await f.agent.executeTool(17, 'click_ax', { ref_id: f.ref });
+ const elapsed = Date.now() - startedAt;
+ const clicks = await page.evaluate(() => window.qaClicks);
+ assert.equal(result.success, true, JSON.stringify(result));
+ assert.equal(result.initialTrustedNativeButton, true);
+ assert.equal(result.dispatched, true);
+ assert.deepEqual(clicks.map(click => click.trusted), [true], 'exactly one trusted activation');
+ assert.equal(presses().length, 1, 'exactly one press was sent');
+ assert.equal(hovers().filter(move => move.x >= 0).length, 1, 'the wait sent no additional pointer input');
+ const settled = await page.locator('#commit').boundingBox();
+ assert.deepEqual({ x: presses()[0].x, y: presses()[0].y }, centre(settled), 'pressed at the settled centre');
+ assert.deepEqual({ x: clicks[0].x, y: clicks[0].y }, centre(settled));
+ const [pressRect] = await page.evaluate(() => window.qaPressRects);
+ assert.deepEqual(pressRect, await page.evaluate(() => qaRect()), 'the button had stopped moving when it was pressed');
+ if (moved) assert.notDeepEqual(centre(settled), centre(f.before), 'the hover moved the button');
+ else assert.deepEqual(centre(settled), centre(f.before));
+ await check?.();
+ assert.ok(elapsed < 4000, `bounded wait (${elapsed} ms)`);
+ } finally { await f.context.close(); }
+ });
+
+ for (const [name, options, intervene, check] of [
+ ['a never-settling hover animation', {
+ css: '@keyframes wb-bob{from{transform:translateY(0)}to{transform:translateY(-3px)}}'
+ + ' #commit:hover{animation:wb-bob .25s linear infinite alternate}',
+ }, null, (result, elapsed) => {
+ assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result));
+ assert.equal(result.preparationVeto?.predicate, 'geometry_changed');
+ assert.match(result.error, /Re-observe/);
+ assert.ok(elapsed >= 500, `refused only after the bounded wait (${elapsed} ms)`);
+ }],
+ ['an overlay appearing while the hover settles', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-2px)}'
+ + ' #cover{position:fixed;inset:0;background:#fff;z-index:9}',
+ script: "commit.addEventListener('mouseenter',()=>setTimeout(()=>{const cover=document.createElement('div');"
+ + "cover.id='cover';document.body.append(cover);},90),{once:true});",
+ }, null, result => assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result))],
+ ['the button node replaced while the hover settles', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-2px)}',
+ script: "commit.addEventListener('mouseenter',()=>setTimeout(()=>{const next=commit.cloneNode(true);"
+ + 'commit.replaceWith(next);},90),{once:true});',
+ }, null, result => assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result))],
+ ['a user pointer movement while the hover settles', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-2px)}',
+ }, () => page.mouse.move(700, 600), result => {
+ assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result));
+ assert.equal(result.preparationVeto?.predicate, 'intervention_changed');
+ }],
+ ['a user key press while the hover settles', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-2px)}',
+ }, () => page.keyboard.press('x'), result => {
+ assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result));
+ assert.equal(result.preparationVeto?.predicate, 'intervention_changed');
+ }],
+ ['Stop while the hover settles', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-2px)}',
+ }, agent => agent.abort(17), result => assert.equal(result.preparationVeto?.stage, 'settle_after_pointer', JSON.stringify(result))],
+ ['a hover move beyond the 8 px tolerance', {
+ css: '#commit{transition:transform .2s} #commit:hover{transform:translateY(-12px)}',
+ }, null, result => assert.match(result.error, /changed during trusted input preparation/)],
+ ]) await t.test(`${name} refuses before any press`, async () => {
+ const f = await fixture(browserInstance, options);
+ try {
+ if (intervene) hooks.afterHover = () => intervene(f.agent);
+ const startedAt = Date.now();
+ const result = await f.agent.executeTool(17, 'click_ax', { ref_id: f.ref });
+ const elapsed = Date.now() - startedAt;
+ await page.waitForTimeout(150);
+ assert.equal(result.success, false, JSON.stringify(result));
+ assert.equal(result.noDispatch, true, JSON.stringify(result));
+ assert.notEqual(result.dispatched, true);
+ assert.equal(presses().length, 0, 'no press was sent');
+ assert.deepEqual(await page.evaluate(() => window.qaClicks), []);
+ assert.equal(hovers().filter(move => move.x >= 0).length, 1, 'the wait sent no additional pointer input');
+ assert.ok(elapsed < 4000, `bounded refusal (${elapsed} ms)`);
+ check(result, elapsed);
+ } finally { hooks.afterHover = null; await f.context.close(); }
+ });
+ } finally {
+ cdpClient.attach = cdpAttach;
+ await browserInstance.close();
+ }
+});