Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
fix: type into X's compose window after its own focus press
Claudeaef7946f17e7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex df8bb819e..13120a345 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,7 @@ This changelog was generated from the repository Git history and release tags. V## [Unreleased]### Fixed+- X: typing a post into X's compose window no longer fails, and no longer ends with "Stopped because the page kept changing". Before typing into X's editor, WebBrain clicks it once with a real mouse click. X answers that click by drawing its bold/italic bubble right where the click landed, over the middle of the empty editor, while its timeline behind the compose window keeps updating. WebBrain took those changes as the page changing under it, and since it checks what lies under the middle of an element to be sure it still has the right one, the bubble also made it lose the editor: it refused to type, and choosing Retry met the same bubble. That click now counts as part of the typing, and from then on the typing is checked against the editor itself rather than against what covers its middle: each step puts the focus back in the editor, and the text is sent only to the editor itself: if the page moves the focus elsewhere just before the text goes out, including into a frame such as an iframe, nothing is typed. When the bubble is still there from an earlier attempt and the editor still has the focus, WebBrain types into it, or presses keys in it, without clicking it again; clicking a control whose middle is covered is still refused. Your own clicking or typing, a changed reply setting, a replaced editor or a compose window that stops accepting input still stop it before any text is sent. Separately, the agent then tried the selector `[data-testid="tweetTextarea_0"]`, which on X names both the compose box and the hidden one on the timeline behind it. That can never be typed into safely, but because the timeline kept updating, each such decision was thrown away as out of date without an answer, the agent saw the same page and decided the same again, and after five rounds the run blamed the page. Such a call now answers for itself, saying the selector matches more than one element and to use the box the agent observed, and it no longer counts as the page changing. Still not fixed: in Chrome, typing by a selector that names only the compose box is refused while the timeline keeps updating, because finding the box by selector scrolls the page before the call has been checked. That refusal now says the page kept updating and to type into the box the agent observed, which works, instead of saying that WebBrain could not attach to the tab, which usually means DevTools is open. The click and the retry without a click are Chrome's way of typing; the selector answer and typing into a focused editor whose middle is covered apply in Chrome and Firefox.- When the model provider rate-limits WebBrain (HTTP 429, or Anthropic's 529 "overloaded"), the step is now waited out and retried up to three times (about 5, 15 and 30 seconds, or longer if the provider asks), also when the limit arrives in the middle of a response, as OpenRouter and WebBrain Cloud report it, and for Anthropic (including Vertex), Azure OpenAI, AWS Bedrock and llama.cpp, whose limits ended Act runs at once. An answer you already saw part of is never generated again. A message such as "Rate limit exceeded" is no longer mistaken for an over-long conversation, so WebBrain no longer trims your conversation and asks you to start a new one. Older context is trimmed only when one request alone exceeds the provider's per-minute token limit, and then right away and only once; if the provider does not state its limit, one shortened retry is tried, and the conversation is shortened only if that retry succeeds. Each wait is lengthened at random by up to 20% so several runs do not retry at the same moment, and never comes sooner than the provider asks. An OpenAI "insufficient_quota" billing error now fails at once instead of being retried, and traces label rate limits as rate limiting even when the message mentions a quota.- The local vision fallback now also runs when an OpenAI-compatible provider such as Ollama or LM Studio rejects a screenshot as unsupported. The rejection's status was read from the wrong field, so the fallback never started.- Chat sites: a message is no longer sent to whoever's chat is open just because an e-mail address appears in that chat's earlier messages. When a chat showed no contact name at the top, WebBrain took such an address for the recipient; now it stops and says it cannot tell who the chat is with. An address shown in the conversation's own header, as in help-desk and CRM conversations, still identifies the recipient, while an address in the header's pinned text does not, and mail compose windows read their To, Cc and Bcc addresses as before. In Google Chat inside Gmail, an address mentioned in the chat is never taken as the recipient either: WebBrain stops before sending there, since nothing on the page reliably tells the chat's name from the names in its messages, while a Gmail compose window open over the Chat view still sends to its To, Cc and Bcc addresses.diff --git a/docs/page-feedback-monitor.md b/docs/page-feedback-monitor.mdindex f571defb8..75aab41d4 100644--- a/docs/page-feedback-monitor.md+++ b/docs/page-feedback-monitor.md@@ -65,6 +65,48 @@ Steer or grant authorization. Enter and Alt+Enter retain their existing behaviorcan be refreshed. Replacement editors, changed form or recipient state,later geometry changes and human intervention still require a new decision.Refusals report bounded predicate names instead of private values.+ A typing action's own single left press on its exact certified text field+ (Chrome's trusted contenteditable fallback clicks the editor once so that the+ browser, not a script, focuses it) is preparation of that typing too,+ together with its release when the release's native receiver is that field+ as well. The dispatch reply marks such a press or release+ `typingPreparation`. It does not use up the certificate, and it makes the+ operation keyboard-bound: from then on the field is compared with its+ certificate without the centre hit, because the page may answer the press by+ covering the pressed point. X draws its bold/italic bubble in a body-level+ layer exactly there, over the middle of the empty editor, while its timeline+ keeps updating. Every later step of a keyboard-bound operation focuses the+ field itself, and the receiver of its native input must be the field.+ Native keyboard input of any certified operation must name its receiver:+ input whose receiver could not be marked (no focused node, a frame without+ an active monitor, or a page that removed the revision fence) is refused as+ `target_unresolved`. When the receiver has focus in a child document (a+ page that moved focus into an iframe), the top document's monitor is asked+ about its operation before anything is sent. Such input is the operation's+ only when its certified target is the frame that has focus (or contains+ it), was not an editable host when certified (an editor's text never goes+ into a frame embedded in it), and still has the same certified semantics;+ otherwise it is refused (`receiver_frame_changed`). Later keys of the same+ action, such as `set_field`'s submit key,+ stay keyboard-bound under the same focus and receiver checks. The+ reselection before the text command keeps the certificate; when only the+ field's geometry moved meanwhile (a layout shift, not its semantics), the+ proof is rebased as for any certified decision. Genuine user input, a+ changed field, owner or reply scope, a replaced editor and an inert composer+ still stop the action before any text is sent. A press on another node, an+ activator inside the editor, a right, double or second press, a release over+ another node or any earlier input of the action is consequential as before.+ A focused text field whose centre is covered by something else (the bubble+ left by an earlier attempt) is certified `keyboardOnly`. Only keyboard input+ (`type_ax`, `set_field`, `type_text`, `type` and `press_keys`) may use such a+ certificate, and only while the field still has focus at the decision; a+ click on it is refused as `target_uncovered`. (Without this certificate such+ a field had none, and a click on it was judged by the legacy rules, which+ cannot see that the click would land on the cover.) Chrome's fallback then+ skips its press, because `ax_prepare_field_for_trusted_type` reports the+ field as not `pressable` when its centre hit does not reach the field+ without crossing an activator, and types into the focused field through the+ same receiver checks.A recent owned scroll can reconcile one native subpixel visual-viewportnotification within 200 ms when the original target, hit, rectangle, values,document scroll, full-size viewport, scale and DPR remain unchanged. This@@ -209,6 +251,27 @@ Steer or grant authorization. Enter and Alt+Enter retain their existing behaviornative-submit restrictions remain in force.A target outside the snapshot's coverage returns `action_binding_unavailable`instead of claiming that the page changed. The old call stays undispatched.+ This holds while the page keeps updating: a response whose only obstacle is+ such a target is not discarded as stale (the same view would only be decided+ again); the call returns this answer itself, later calls of its batch are+ skipped, and it does not count toward the `page_unstable` budget. User input+ and any other changed target still supersede the response. A selector that+ names several elements, such as X's `[data-testid="tweetTextarea_0"]` for the+ dialog's composer and the hidden timeline composer behind it, is reported as+ `target_ambiguous` (`bindingFailure` in the result), with the instruction to+ use the observed element's ref. For a selector call, the rich-text toolbar+ preflight answers such a target the same way before it probes the target,+ so the answer is not replaced by a probe failure. That early check answers+ only such a target; every other outcome is left to the call's own+ validation, which runs again before dispatch.+ Chrome's preflight for `type_text` with a selector resolves the target+ through the trusted CDP resolver before the call is validated, and that+ resolver's scroll is fenced. While the page keeps updating by itself the+ fence refuses it, so selector typing is still refused there before anything+ is typed. The refusal says that the page changed while the target was being+ located and to type by the observed ref with `type_ax`; it is not retryable+ as the same call, and it no longer reports a resolver that could not attach+ (DevTools) unless attaching actually failed.The next normal decision prioritizes that target and receives a fresh boundedAX observation of its owner together with the new private footprint. Targetselectors and labels retain their exact strings throughout validation anddiff --git a/package.json b/package.jsonindex f65429a8a..cfeddd14c 100644--- a/package.json+++ b/package.json@@ -59,8 +59,8 @@"benchmark:relevance": "node scripts/benchmark-offline-relevance.mjs --verbose","test:social-contract": "node --test test/social-publish-contract.mjs test/publication-quote-markup.mjs","test:social-contract:dom": "node test/social-publish-contract-dom.mjs",- "test:runtime-lifecycle": "node test/bot-memory-dom.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/history-recovery-state.mjs test/history-recovery-scheduler.mjs test/history-recovery-atomic.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/form-validation-alert-readings.mjs test/control-effect-readings.mjs test/trusted-retry-readings.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/lifecycle-steering-restart.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/workflow-field-readings.mjs test/workflow-field-anchors.mjs test/user-turn-kind.mjs test/completion-current-execution-evidence.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-vision-fallback.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/guards-steering-receipts.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/history-recovery.mjs test/history-recovery-ui.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs && node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs test/message-control-readings.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/ask-person-activity-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs && node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs --menu-read && node test/quality/mastodon-delete-confirmation-mv3.mjs --vague && node test/quality/unbound-deletion-mv3.mjs && node test/quality/recipient-clarify-mv3.mjs && node test/quality/open-chat-recipients-mv3.mjs && node test/quality/correction-cross-site-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",- "test:page-feedback": "node --test test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/delivery-click-ax-response-loss.mjs test/delivery-stop-compound.mjs test/delivery-cdp-input.mjs test/delivery-click-locks.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-dispatch-boundary.mjs && node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs",+ "test:runtime-lifecycle": "node test/bot-memory-dom.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/history-recovery-state.mjs test/history-recovery-scheduler.mjs test/history-recovery-atomic.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/form-validation-alert-readings.mjs test/control-effect-readings.mjs test/trusted-retry-readings.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/lifecycle-steering-restart.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/page-feedback-unbindable-target.mjs test/rich-text-preflight-page-change.mjs test/keyboard-receiver-frame.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/workflow-field-readings.mjs test/workflow-field-anchors.mjs test/user-turn-kind.mjs test/completion-current-execution-evidence.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-vision-fallback.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/guards-steering-receipts.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/history-recovery.mjs test/history-recovery-ui.mjs test/page-monitor-ui.mjs test/x-compose-trusted-typing.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs && node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs test/message-control-readings.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/ask-person-activity-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs && node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs --menu-read && node test/quality/mastodon-delete-confirmation-mv3.mjs --vague && node test/quality/unbound-deletion-mv3.mjs && node test/quality/recipient-clarify-mv3.mjs && node test/quality/open-chat-recipients-mv3.mjs && node test/quality/correction-cross-site-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/x-compose-dialog-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",+ "test:page-feedback": "node --test test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/page-feedback-unbindable-target.mjs test/rich-text-preflight-page-change.mjs test/keyboard-receiver-frame.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/page-monitor-ui.mjs test/x-compose-trusted-typing.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/delivery-click-ax-response-loss.mjs test/delivery-stop-compound.mjs test/delivery-cdp-input.mjs test/delivery-click-locks.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-dispatch-boundary.mjs && node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs","test:history-recovery": "node --test test/history-recovery.mjs test/history-recovery-ui.mjs test/history-recovery-state.mjs test/history-recovery-scheduler.mjs test/history-recovery-atomic.mjs test/conversation-persistence.mjs","test:chat-steering": "node --test test/chat-steering.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs","test:rich-text-dom": "node test/ui-content-lifecycle.mjs --dom",@@ -92,13 +92,13 @@"test:chat-draft-recovery": "node --test test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs && node test/quality/native-chat-presubmit-mv3.mjs","test:upload-owner": "node --test test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs","test:shard:quick": "npm run test:cms && npm run test:markdown && npm run test:generative-media && npm run test:systemone && npm run test:systemone:fast && node --test test/browser-dialogs.mjs test/capsolver-cloud-broker.mjs test/captcha-providers.mjs test/captcha-weighted-fallback.mjs test/captcha-hcaptcha-providers.mjs test/captcha-native-providers.mjs test/captcha-runtime-policy.mjs test/captcha-callback-bridge.mjs test/captcha-bridge-registration.mjs test/captcha-aws-recovery.mjs test/captcha-documented-contracts.mjs test/huggingface-signup-recovery.mjs test/trace-full-export.mjs test/feedback-traces.mjs && npm run test:firefox-bidi && npm run test:billing-notice && npm run test:compass-quota && npm run test:provider-limits && npm run test:provider-catalog && npm run test:provider-instances:ui && npm run test:openrouter-capabilities && npm run test:anthropic && npm run test:accessibility-tree-benchmark && npm run test:toolbar-guard && npm run test:cloud-bridge-approval && npm run test:discord && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && npm run test:safesocial && npm run test:build-unpacked && npm run test:attachment-drop && npm run test:workflow-editor && node --test test/profile-sync-stats.mjs && node --test test/memcode-recall.mjs",- "test:shard:runtime-unit": "node --test test/skill-download-wait.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-click-guard.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/form-validation-alert-readings.mjs test/control-effect-readings.mjs test/trusted-retry-readings.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/workflow-field-readings.mjs test/workflow-field-anchors.mjs test/user-turn-kind.mjs test/completion-current-execution-evidence.mjs test/video-upload-attachment.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-vision-fallback.mjs test/delivery-click-locks.mjs test/guards-steering-receipts.mjs test/terminal-user-messages.mjs test/improver-providers.mjs test/improver-providers-agent.mjs test/improver-guards-ledger.mjs test/improver-guards-tools.mjs test/improver-clarify.mjs test/improver-clarify-scheduler.mjs test/improver-misc.mjs test/improver-forms.mjs test/improver-forms-strict-secrets.mjs test/improver-content-press-keys-hint.mjs",- "test:shard:browser-a": "node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-dispatch-boundary.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/submit-probe-readings.mjs test/improver-content-account-chrome.mjs test/improver-content-recipient.mjs test/improver-content-truncated-replacement.mjs test/improver-forms-probe.mjs test/improver-misc-trace-browser.mjs",+ "test:shard:runtime-unit": "node --test test/skill-download-wait.mjs && node --test test/agent-lifecycle.mjs test/runtime-startup.mjs test/run-ui-scheduled-lifecycle.mjs test/transport-deadlines.mjs test/cdp-worker-restart.mjs test/bot-memory.mjs test/bot-memory-click-guard.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs test/cdp-clear-receipt.mjs test/cdp-field-submit-recovery.mjs test/conversation-persistence.mjs test/context-live-tool-group.mjs test/integration-races.mjs test/watch-completion-checkpoint.mjs test/runtime-model-identity.mjs test/provider-message-privacy.mjs test/runtime-task-authority.mjs test/planner-repair-authority.mjs test/runtime-progress-continuation.mjs test/chat-resume-chain.mjs test/chat-resume-completion.mjs test/native-chat-execution-ledger.mjs test/chat-send-dispatch-result.mjs test/chat-reply-dedup-race.mjs test/memcode-lifecycle.mjs test/runtime-peer-review.mjs test/read-continuation-schema.mjs test/completion-submit-acknowledgement.mjs test/completion-live-status.mjs test/completion-iframe-submit.mjs test/form-validation-alert-readings.mjs test/control-effect-readings.mjs test/trusted-retry-readings.mjs test/completion-semantic-commit.mjs test/completion-diagnostics.mjs test/mastodon-publication.mjs test/calculator.mjs test/chat-recovery-protocol.mjs test/chat-pending-reconciliation.mjs test/semantic-page-retry.mjs test/pdf-detection.mjs test/runtime-lifecycle.mjs test/collection-read-progress.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs test/trusted-field-preparation.mjs test/draftjs-text.mjs test/rich-text-frameworks.mjs test/cdp-managed-text-selection.mjs test/firefox-rich-text-readback.mjs test/contenteditable-beforeinput.mjs test/trusted-field-preparation-ownership.mjs test/typing-preparation-geometry.mjs test/chat-steering.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/steering-revision-recovery.mjs test/pending-clarification-hydration.mjs test/permission-steering-wakeup.mjs test/pure-calculation-reply.mjs test/pure-calculation-tools.mjs test/page-feedback.mjs test/page-feedback-progress.mjs test/page-feedback-unbindable-target.mjs test/rich-text-preflight-page-change.mjs test/keyboard-receiver-frame.mjs test/ask-response-scope.mjs test/ask-completion-churn.mjs test/page-feedback-policy.mjs test/page-feedback-transport-races.mjs test/page-feedback-recovery.mjs test/telegram-read-preparation-agent.mjs test/media-download-agent-handler.mjs test/code-scanning-url-recipient.mjs test/apocalypse-comm-parsing.mjs test/apocalypse-scalar-decoding.mjs test/bitgpu-chat-security.mjs test/social-media-url-upgrades.mjs test/downloaded-file-saved-bytes.mjs test/downloaded-file-local-probe.mjs test/formless-verification-recovery.mjs test/file-upload-accept.mjs test/file-upload-accept-agent.mjs test/workflow-route-generic-completion.mjs test/release-publication-completion.mjs test/workflow-implicit-metadata.mjs test/workflow-field-readings.mjs test/workflow-field-anchors.mjs test/user-turn-kind.mjs test/completion-current-execution-evidence.mjs test/video-upload-attachment.mjs test/feedback-automatic-churn.mjs test/feedback-navigation-idle.mjs test/feedback-run-boundaries.mjs test/feedback-ask-viewing.mjs test/lifecycle-terminal-done.mjs test/lifecycle-planner-digest.mjs test/lifecycle-steering-latency.mjs test/lifecycle-steering-restart.mjs test/lifecycle-vision-fallback.mjs test/delivery-click-locks.mjs test/guards-steering-receipts.mjs test/terminal-user-messages.mjs test/improver-providers.mjs test/improver-providers-agent.mjs test/improver-guards-ledger.mjs test/improver-guards-tools.mjs test/improver-clarify.mjs test/improver-clarify-scheduler.mjs test/improver-misc.mjs test/improver-forms.mjs test/improver-forms-strict-secrets.mjs test/improver-content-press-keys-hint.mjs",+ "test:shard:browser-a": "node test/bot-memory-dom.mjs && node test/bot-memory-click-dom.mjs && node test/ui-content-lifecycle.mjs && node --test test/native-telegram-chat.mjs test/native-telegram-recipient.mjs test/telegram-native-baseline.mjs test/telegram-native-latest.mjs test/native-chat-pre-enter-draft.mjs test/chat-draft-dispatch-ownership.mjs test/native-telegram-alias-preparation.mjs test/telegram-read-preparation-content.mjs test/queued-composer-ownership.mjs test/onboarding-provider-ready.mjs test/workflow-native-targets.mjs test/native-whatsapp-chat.mjs test/native-chat-context-identity.mjs test/native-whatsapp-contact.mjs test/whatsapp-native-bootstrap.mjs test/whatsapp-drawer-identity.mjs test/whatsapp-read-latest.mjs test/whatsapp-read-preparation.mjs test/whatsapp-chat-completion.mjs test/whatsapp-chat-restart.mjs test/chat-goal-lifecycle.mjs test/chat-completion.mjs test/chat-generic-dispatch.mjs test/completion-live-status-browser.mjs test/native-confirmation-visibility.mjs test/native-button-preparation.mjs test/read-continuation-browser.mjs test/iframe-preparation-focus.mjs test/contenteditable-literal.mjs test/native-chat-empty-editor.mjs test/native-chat-trusted-clear.mjs test/chat-steering-ui.mjs test/recipient-confirmation-ui.mjs test/assistant-message-visibility-ui.mjs test/page-monitor-ui.mjs test/x-compose-trusted-typing.mjs test/page-monitor-dialog.mjs test/page-monitor-dialog-native.mjs test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs test/page-monitor-anchored.mjs test/page-monitor-upload-owner.mjs test/page-monitor-upload-release.mjs test/page-monitor-native-controls.mjs test/page-monitor-native-preparation.mjs test/native-preparation-cold.mjs test/page-monitor-navigation.mjs test/page-monitor-main-owner.mjs test/page-monitor-collateral-controls.mjs test/accessibility-tree-scoped-budget.mjs test/ax-reference-origin.mjs test/page-monitor-decorative-hit.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/dropdown-menu-inventory.mjs test/page-monitor-passive-progress.mjs test/page-monitor-feed-items.mjs test/media-download-binding.mjs test/media-download-capture-integration.mjs test/social-media-downloader-reinjection.mjs test/upload-file-local-state-guidance.mjs test/monitor-intervention-contract.mjs test/monitor-owned-geometry.mjs test/monitor-viewport-remainder.mjs test/monitor-local-attribution.mjs test/monitor-lifecycle-leaks.mjs test/lifecycle-run-ui.mjs test/lifecycle-menu-prompts-held.mjs test/delivery-cdp-input.mjs test/delivery-click-ax-response-loss.mjs test/delivery-native-button-geometry.mjs test/delivery-hover-transition.mjs test/delivery-dispatch-boundary.mjs test/delivery-stop-compound.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/submit-probe-readings.mjs test/improver-content-account-chrome.mjs test/improver-content-recipient.mjs test/improver-content-truncated-replacement.mjs test/improver-forms-probe.mjs test/improver-misc-trace-browser.mjs","test:shard:browser-b1": "node test/message-recipient-upload.mjs && node test/message-recipient-clarify-binding.mjs && node --test test/message-recipient-whatsapp.mjs test/message-control-readings.mjs && node test/quality/integration-mv3.mjs --full-form && node test/quality/literal-calculate-midrun-mv3.mjs && node test/quality/ask-completion-churn-mv3.mjs && node test/quality/ask-person-activity-mv3.mjs && node test/quality/telegram-cold-preparation-mv3.mjs && node test/quality/native-chat-mv3.mjs && node test/quality/native-chat-presubmit-mv3.mjs && node test/quality/cloud-bot-memory-mv3.mjs","test:shard:browser-b2": "node test/quality/whatsapp-contact-matrix.mjs && node test/quality/pdf-detection-mv3.mjs && node test/quality/download-upload-mv3.mjs && node test/quality/upload-release-binding-mv3.mjs && node test/quality/release-review-world-mv3.mjs && node test/quality/downloaded-file-saved-bytes-mv3.mjs && node test/quality/file-upload-accept-mv3.mjs && node test/quality/bulk-resume-mv3.mjs && node test/quality/shared-action-guards-mv3.mjs && node test/quality/video-upload-attachment-mv3.mjs",- "test:shard:browser-c": "node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs --menu-read && node test/quality/mastodon-delete-confirmation-mv3.mjs --vague && node test/quality/unbound-deletion-mv3.mjs && node test/quality/recipient-clarify-mv3.mjs && node test/quality/open-chat-recipients-mv3.mjs && node test/quality/correction-cross-site-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs",+ "test:shard:browser-c": "node test/quality/literal-frame-mv3.mjs && node test/quality/mastodon-publication-mv3.mjs && node test/quality/mastodon-deletion-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs && node test/quality/mastodon-delete-confirmation-mv3.mjs --menu-read && node test/quality/mastodon-delete-confirmation-mv3.mjs --vague && node test/quality/unbound-deletion-mv3.mjs && node test/quality/recipient-clarify-mv3.mjs && node test/quality/open-chat-recipients-mv3.mjs && node test/quality/correction-cross-site-mv3.mjs && node test/quality/anchored-action-mv3.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/x-compose-dialog-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs && node test/quality/dropdown-menu-matrix.mjs && node test/quality/mastodon-deletion-navigation-mv3.mjs && node test/quality/semantic-navigation-matrix.mjs && node test/quality/workflow-replay-mv3.mjs && node test/quality/generative-media-mv3.mjs && node test/quality/cdp-worker-restart-mv3.mjs && node test/quality/steering-worker-restart-mv3.mjs","test:shard:tail": "node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",- "test:composer-context": "node --test test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs",+ "test:composer-context": "node --test test/page-monitor-inline-post.mjs test/bluesky-composer-context.mjs && node test/quality/inline-social-composer-mv3.mjs && node test/quality/x-compose-dialog-mv3.mjs && node test/quality/bluesky-composer-mv3.mjs","test:bot-memory": "node --test test/bot-memory.mjs test/bot-memory-registries.mjs test/bot-memory-cloud-controller.mjs && node test/bot-memory-dom.mjs && node test/quality/cloud-bot-memory-mv3.mjs","test:shared-action-guards": "node --test test/typing-preparation-geometry.mjs test/resource-mutation-guards.mjs test/completion-steering-receipt-authority.mjs test/social-ui-navigation.mjs test/native-resource-deletion-binding.mjs test/native-mastodon-confirmation.mjs test/trusted-field-preparation-ownership.mjs test/guards-control-classification.mjs test/guards-unbound-deletion.mjs test/guards-steering-receipts.mjs && node test/quality/shared-action-guards-mv3.mjs"},diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.jsindex 19034924f..2ab37bd00 100644--- a/src/chrome/src/agent/agent.js+++ b/src/chrome/src/agent/agent.js@@ -9385,6 +9385,14 @@ export class Agent extends LoopDetector {if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName)) {return { block: null, shot: null };}+ // The trusted selector resolver scrolls its target into view, and the+ // page-feedback fence refuses that scroll while the page is changing.+ const pageChangedSelectorMessage = 'The page changed while WebBrain was locating this selector\'s target, so nothing was typed. '+ + 'On a page that keeps updating, typing by selector is refused the same way each time: type into the field with type_ax and its ref_id from get_accessibility_tree instead.';+ // A target the observed certificate cannot bind (such as a selector that+ // names several elements) gets that answer before any probe of it.+ const unbindable = await this._pageFeedbackUnbindableResult?.(tabId, toolName, args);+ if (unbindable) return { block: unbindable, shot: null };const selectorBackedType = toolName === 'type_text'&& typeof args?.selector === 'string'&& !!args.selector.trim();@@ -9398,13 +9406,14 @@ export class Agent extends LoopDetector {: await this._probeRichTextToolbarRetryTarget(tabId, toolName, args, { mapAnnotation: true });if (!probe?.resolved) {const ambiguousIframeTarget = toolName === 'iframe_type' && probe?.ambiguous === true;+ const pageChanged = selectorBackedType && probe?.pageChangedDuringResolve === true;return ambiguousIframeTarget || (toolName === 'iframe_type' && recoveryPending) || selectorBackedType || focusedType? {block: {success: false,dispatched: false,noDispatch: true,- retryable: true,+ ...(pageChanged ? {} : { retryable: true }),...(ambiguousIframeTarget ? {ambiguous: true,matchCount: probe.matchCount,@@ -9417,7 +9426,8 @@ export class Agent extends LoopDetector {: 'Could not resolve one matching iframe target for the rich-text toolbar safety preflight. Re-read the iframe and retry with a specific urlFilter and selector.': focusedType? 'Could not preserve the focused target for the rich-text toolbar safety preflight. Focus the intended field again and retry.'- : 'Could not resolve the selector target for the rich-text toolbar safety preflight. Re-read the page and retry.',+ : pageChanged ? pageChangedSelectorMessage+ : 'Could not resolve the selector target for the rich-text toolbar safety preflight. Re-read the page and retry.',},shot: null,}@@ -9438,14 +9448,17 @@ export class Agent extends LoopDetector {// Without the trusted resolver the same call fails the same way every// time, so do not invite a retry that cannot succeed.const resolverUnavailable = selectorBackedType && probe.trustedResolverUnavailable === true;+ const pageChanged = selectorBackedType && !resolverUnavailable && probe.pageChangedDuringResolve === true;return {block: {success: false,dispatched: false,noDispatch: true,- ...(resolverUnavailable ? {} : { retryable: true }),+ ...(resolverUnavailable || pageChanged ? {} : { retryable: true }),error: resolverUnavailable? 'The trusted selector resolver could not attach to this tab, so selector typing cannot preserve its target. This usually means DevTools is open on the page. Use click_ax with a ref_id from get_accessibility_tree followed by type_ax instead; repeating this call will fail the same way.'+ : pageChanged+ ? pageChangedSelectorMessage: focusedType? 'Could not preserve the focused target for the rich-text toolbar safety preflight. Focus the intended field again and retry.': 'Could not preserve the selector target for the rich-text toolbar safety preflight. Re-read the page and retry.',@@ -48214,7 +48227,12 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout dthis._throwIfAborted(abortSignal);// Child-frame rectangles are frame-relative. Focus/selection are already// bound to the exact editor there; do not click them as top-frame points.- if (frameId === 0 && prepared.contentEditable && prepared.rect?.w > 0 && prepared.rect?.h > 0) {+ // Nor is an editor pressed where the page covers its centre (X's+ // formatting bubble after an earlier click): that press would reach the+ // cover. The page monitor lets such an editor take text only when it+ // already had keyboard focus when the model chose it.+ if (frameId === 0 && prepared.contentEditable && prepared.rect?.w > 0 && prepared.rect?.h > 0+ && prepared.pressable !== false) {const x = prepared.rect.x + prepared.rect.w / 2;const y = prepared.rect.y + prepared.rect.h / 2;this._throwIfAborted(abortSignal);diff --git a/src/chrome/src/agent/page-feedback.js b/src/chrome/src/agent/page-feedback.jsindex 5b6eec08a..14505198f 100644--- a/src/chrome/src/agent/page-feedback.js+++ b/src/chrome/src/agent/page-feedback.js@@ -207,11 +207,20 @@ export function pageFeedbackPendingResult() {pageFeedbackPending: true, error: 'The browser changed during action preparation. Re-observe the page before acting.' };}+// The answer of a call whose target the observed certificate cannot bind.+function actionBindingUnavailableResult(run) {+ return { success: false, skipped: true, dispatched: false, noDispatch: true, retryable: false,+ errorCode: 'action_binding_unavailable',+ ...(run.actionBindingUnavailableReason === 'target_ambiguous' ? { bindingFailure: 'target_ambiguous',+ error: 'The selector matches more than one element on this page (for example a hidden copy of the same field behind a dialog), so it cannot be bound to the element you observed. Nothing was sent. Use the ref_id of the intended element from the current page observation instead of this selector.' }+ : { error: 'The exact target could not be bound to the observed action context. Fresh available target/page context will be observed before the next decision; inspect it or read the exact target inventory before trying the action again.' }) };+}+function preparationVetoDiagnostics(veto) {const facts = {};if (!veto || typeof veto !== 'object') return facts;- if (['owner_pending_before', 'owner_pending_after', 'monitor_rejected',- 'document_mapping_changed', 'owner_ended', 'current_owner_changed'].includes(veto.cause)) facts.cause = veto.cause;+ if (['owner_pending_before', 'owner_pending_after', 'monitor_rejected', 'document_mapping_changed',+ 'receiver_frame_changed', 'owner_ended', 'current_owner_changed'].includes(veto.cause)) facts.cause = veto.cause;// Preserve diagnostics through every wrapper without copying page values.for (const key of ['ownerPendingAfter', 'monitorRejected',@@ -249,9 +258,28 @@ export async function beforePageAgentDispatch(api, tabId, details = {}) {}details = { ...details, ...(frameId === undefined ? {} : { frameId }) };}- owner.operationFrames.get(owner.operationId)?.add(Number(details.frameId) || 0);const { prepareMonitor = false, ...dispatchDetails } = details;const operationId = dispatchDetails.operationId || owner.operationId || token();+ // A model certificate binds a node of the top document. Keyboard input+ // whose receiver has focus in a child document (a page that moved focus+ // into an iframe) is that operation's input only if the top document says+ // its certified target is the frame that has focus.+ if (!details.release && details.kind === 'input' && Number(details.frameId) > 0 && owner.modelBound?.()) {+ let reply = null;+ try {+ reply = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_receiver_frame',+ params: { runToken: owner.runToken, operationId } }, { frameId: 0 });+ } catch { /* No answer cannot allow it. */ }+ if (dispatchOwners.get(tabId) !== owner) {+ const error = new Error('Browser run ended before dispatch'); error.name = 'AbortError';+ error.pagePreparationVeto = { cause: 'owner_ended' }; throw error;+ }+ if (reply?.foreignReceiverAllowed !== true) {+ const error = new Error(pageFeedbackPendingResult().error);+ error.code = 'page_feedback_pending'; error.pagePreparationVeto = { cause: 'receiver_frame_changed' }; throw error;+ }+ }+ owner.operationFrames.get(owner.operationId)?.add(Number(details.frameId) || 0);const monitorParams = { ...dispatchDetails, runToken: owner.runToken, operationId };let acknowledgement;let preparationAcknowledgement;@@ -398,6 +426,8 @@ export const pageFeedbackMethods = {};dispatchOwners.set(tabId, { runToken: run.token, operationId: '', operationFrames: new Map(),tool: () => run.dispatchCall?.name || '',+ // The current call acts on a certified node of the top document.+ modelBound: () => !!run.dispatchCall?.expectedModelSnapshot,pending: () => this._pageFeedbackBlocksCall(tabId, run.dispatchCall?.name, run.dispatchCall?.args),allowPassiveRebase: () => run.dispatchCall?.allowPassiveRebase === true,expectedModelSnapshot: () => run.dispatchCall?.dispatchState?.started ? undefined : run.dispatchCall?.expectedModelSnapshot,@@ -1101,10 +1131,12 @@ export const pageFeedbackMethods = {&& run.frames.get(id)?.id === frame.id);},- async _validatePageFeedbackAction(tabId, name, args, state) {+ // answerOnly: an early check that only answers a target the certificate+ // cannot bind; any other outcome is left to the call's own validation.+ async _validatePageFeedbackAction(tabId, name, args, state, { answerOnly = false } = {}) {const run = this._pageFeedbackRuns?.get(tabId);const binding = state?.actionBinding;- if (run) { run.actionBindingUnavailable = null; run.actionContextUnverified = false; }+ if (run) { run.actionBindingUnavailable = null; run.actionBindingUnavailableReason = null; run.actionContextUnverified = false; }if (!run || !binding || run.modelState !== state || !this._pageFeedbackStateCurrent(tabId, run, state)|| (run.events.size && !isAutomaticPageFeedback([...run.events.values()]))) return false;try {@@ -1122,13 +1154,18 @@ export const pageFeedbackMethods = {&& (response.relaxed !== true || (relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, run) === retry));const certificationFailure = ['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit','target_uncovered', 'context_uncertified'].includes(response?.certificationFailure) ? response.certificationFailure : null;+ const unbindable = owned && ['target_uncovered', 'target_unresolved', 'target_ambiguous'].includes(response.reason);+ if (answerOnly && !unbindable) {+ if (valid) (run.validatedActionBindings ??= new Map()).set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);+ return valid;+ }if (owned) {const runId = this.currentRunId?.get(tabId);if (runId) trace.recordNote(runId, 0, 'page_action_validation', {status: valid ? 'ready' : 'blocked', monitoringRelaxed: valid && response.relaxed === true,...(certificationFailure ? { certificationFailure } : {}),...(response.uncertified === true ? { certificationFailure: certificationFailure || 'context_uncertified' } : {}),- ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_uncovered',+ ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_ambiguous', 'target_uncovered','focus_changed', 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired','pending_feedback', 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed','relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable'].includes(response.reason) ? { predicate: response.reason } : {}),@@ -1139,11 +1176,12 @@ export const pageFeedbackMethods = {run.validatedActionBindings ??= new Map();run.validatedActionBindings.set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);}- if (owned && ['target_uncovered', 'target_unresolved'].includes(response.reason)) {+ if (unbindable) {// Preserve the original no-dispatch outcome. A bounded fresh observation// of this target belongs to the NEXT normal decision, never this call.run.actionCaptureTarget = safeAction(name, args);run.actionBindingUnavailable = `${name}:${JSON.stringify(args)}`;+ run.actionBindingUnavailableReason = response.reason;}if (owned && response.uncertified === true) {// The original node is unchanged, but its action context was never@@ -1158,6 +1196,18 @@ export const pageFeedbackMethods = {} catch { return false; }},+ // A typing call's own preparation probes its target before dispatch. A+ // target the certificate cannot bind at all is answered first and+ // precisely: a probe of an ambiguous selector could only pick one of its+ // elements. Anything else is left to the dispatch wrapper.+ async _pageFeedbackUnbindableResult(tabId, name, args = {}) {+ const run = this._pageFeedbackRuns?.get(tabId);+ if (!run || typeof args?.selector !== 'string' || !args.selector.trim() || !this.constructor.STATE_CHANGE_TOOLS?.has(name)+ || pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage).kind !== 'bound_target') return null;+ if (await this._validatePageFeedbackAction(tabId, name, args, run.modelState, { answerOnly: true }) !== false) return null;+ return run.actionBindingUnavailable === `${name}:${JSON.stringify(args)}` ? actionBindingUnavailableResult(run) : null;+ },+_pageFeedbackBlocksCall(tabId, name, args = {}) {const run = this._pageFeedbackRuns?.get(tabId);if (run?.dispatchCall?.expectedModelSnapshot@@ -1368,13 +1418,26 @@ export const pageFeedbackMethods = {this._monitoringRetryCurrent(tabId, run) ? current.map(event => event.source === 'unknown' && event.kind === 'dom'? { ...event, source: 'page' } : event) : current, options);};- let unverifiedActionContext = false;+ let unverifiedActionContext = false, unbindableActionTarget = false;let retained = sameDocument && canRetain(events)&& (!pending.length || canRetain(pending, { pending: true }));if (retained) for (const call of responseToolCalls) {const args = JSON.parse(call.function.arguments);if (pageFeedbackCallPolicy(call.function.name, args, state, page, events).kind === 'bound_target'&& await this._validatePageFeedbackAction(tabId, call.function.name, args, state) !== true) {+ // A target the certificate cannot bind (a selector naming several+ // elements or none, a ref the observation did not certify, a control+ // its snapshot does not cover) gets the same answer after these page+ // updates, and discarding the response would only show the model the+ // same view to decide again. Keep the call: its own no-dispatch result+ // (action_binding_unavailable) tells the model why, and the rest of+ // its batch does not run. If a real change caused it (its element+ // removed or added), the attached fresh observation shows that. A+ // changed binding, an unverified action context and user input still+ // supersede the response.+ if (run.actionBindingUnavailable === `${call.function.name}:${JSON.stringify(args)}`) {+ unbindableActionTarget = true; continue;+ }unverifiedActionContext = run.actionContextUnverified === true;retained = false; break;}@@ -1463,7 +1526,8 @@ export const pageFeedbackMethods = {if (runId) trace.recordNote(runId, 0, 'page_feedback_decision', { stage, disposition: responseToolCalls === null ? 'observed' : accepted ? 'retained' : 'superseded',reason: accepted && independentCompletion && (!isPassivePageFeedback(events)|| (finalPending.length && !isPassivePageFeedback(finalPending))) ? 'page_independent_completion'- : honestUnverifiedContext && ['passive_dom', 'automatic_page'].includes(feedbackReason) ? 'target_context_unverified' : feedbackReason,+ : honestUnverifiedContext && ['passive_dom', 'automatic_page'].includes(feedbackReason) ? 'target_context_unverified'+ : accepted && unbindableActionTarget ? 'action_target_unbindable' : feedbackReason,context: { bindingAtRefresh: { documentMatches, currentRun, stateCurrent, sameDocument },feedback: { observed: feedbackEventSummary(events), pendingAtRefresh: feedbackEventSummary(pending),pendingAtDecision: feedbackEventSummary(finalPending) } },@@ -1499,9 +1563,7 @@ export function installPageFeedback(Agent) {if (modelBound) {const validated = await this._validatePageFeedbackAction(tabId, name, args, modelState);if (validated === false) return run.actionBindingUnavailable === `${name}:${JSON.stringify(args)}`- ? { success: false, skipped: true, dispatched: false, noDispatch: true, retryable: false,- errorCode: 'action_binding_unavailable',- error: 'The exact target could not be bound to the observed action context. Fresh available target/page context will be observed before the next decision; inspect it or read the exact target inventory before trying the action again.' }+ ? actionBindingUnavailableResult(run): run.actionContextUnverified === true? { ...pageFeedbackPendingResult(), actionContextUnverified: true, errorCode: 'action_context_unverified',error: 'The requested control had no verified original action context. No action was dispatched; inspect the fresh bounded target observation before choosing another action.' }diff --git a/src/chrome/src/agent/rich-text-toolbar-probe.js b/src/chrome/src/agent/rich-text-toolbar-probe.jsindex 32680d10c..da55a65b7 100644--- a/src/chrome/src/agent/rich-text-toolbar-probe.js+++ b/src/chrome/src/agent/rich-text-toolbar-probe.js@@ -585,6 +585,10 @@ export class RichTextToolbarProbe {// and retrying the call cannot change that. Callers use this to say so// instead of asking the model to re-read the page forever.let trustedResolverUnavailable = false;+ // Set when the page-feedback fence refused the resolver's own scroll: the+ // page changed (on a live page, it keeps changing); the resolver itself+ // is available. Retrying the same selector meets the same fence.+ let pageChangedDuringResolve = false;if (toolName === 'type_text' && typeof args?.selector === 'string' && args.selector.trim()) {try {await cdpClient.attach(tabId);@@ -596,15 +600,16 @@ export class RichTextToolbarProbe {? { dispatchBinding: { backendNodeId: Number(selectorBackendNodeId), frameId: 0 } }: {}),}, 0);- } catch {- trustedResolverUnavailable = true;+ } catch (error) {+ if (error?.code === 'page_feedback_pending') pageChangedDuringResolve = true;+ else trustedResolverUnavailable = true;}}const withResolverState = probe => {const bound = withDispatchBinding(probe, 0);- return bound && trustedResolverUnavailable- ? { ...bound, trustedResolverUnavailable: true }- : bound;+ return bound && trustedResolverUnavailable ? { ...bound, trustedResolverUnavailable: true }+ : bound && pageChangedDuringResolve ? { ...bound, pageChangedDuringResolve: true }+ : bound;};try {const probe = await chrome.tabs.sendMessage(tabId, {diff --git a/src/chrome/src/cdp/cdp-client.js b/src/chrome/src/cdp/cdp-client.jsindex b3986abf0..d683f5688 100644--- a/src/chrome/src/cdp/cdp-client.js+++ b/src/chrome/src/cdp/cdp-client.js@@ -951,6 +951,10 @@ export class CDPClient {nativeWheel: params.type === 'mouseWheel',navigationCandidate: params.type === 'mousePressed',...(Number.isFinite(params.x) ? { x: params.x, y: params.y } : {}),+ // The monitor recognizes a typing action's own focus press by these.+ ...(method === 'Input.dispatchMouseEvent' ? { pointerButton: params.button, clickCount: params.clickCount } : {}),+ // Native input always tries to mark its receiver (nativeTarget).+ nativeInput: true,release,...(method === 'Input.insertText' ? { eventTypes: ['beforeinput', 'input', 'change'] } : {}),...(method === 'Input.dispatchMouseEvent' ? { eventTypes: params.type === 'mouseMoved'diff --git a/src/chrome/src/content/content.js b/src/chrome/src/content/content.jsindex 58239a40c..252464b95 100644--- a/src/chrome/src/content/content.js+++ b/src/chrome/src/content/content.js@@ -10375,12 +10375,28 @@|| !!(el.getAttribute && el.getAttribute('aria-autocomplete'))|| (el.getAttribute && el.getAttribute('aria-expanded') === 'true');const rect = el.getBoundingClientRect();+ // Whether a press at the field's centre reaches the field or its+ // plain contents, rather than something the page drew over it (X+ // draws its formatting bubble where its editor was clicked).+ const centreX = rect.left + rect.width / 2, centreY = rect.top + rect.height / 2;+ let centreHit = document.elementFromPoint(centreX, centreY);+ for (let depth = 0; depth < 24 && centreHit?.shadowRoot; depth++) {+ const inner = centreHit.shadowRoot.elementFromPoint?.(centreX, centreY);+ if (!inner || inner === centreHit) break;+ centreHit = inner;+ }+ let pressable = false;+ for (let node = centreHit, depth = 0; node && depth < 256; node = node.parentElement || node.getRootNode?.().host, depth++) {+ if (node === el) { pressable = true; break; }+ if (node.matches?.('button,a[href],input,textarea,select,summary,[onclick],[tabindex],[role="button"],[role="link"],[role="checkbox"],[role="radio"],[role="switch"],[role="tab"],[role="menuitem"]')) break;+ }return {success: true,ref_id,fieldMeta: _fieldMeta(el),isCombobox,contentEditable: !!el.isContentEditable,+ pressable,...(isDraftJsEditor(el) ? { editorKind: 'draftjs' } : {}),selectionMode: el.isContentEditable && selectionMode === 'end' ? 'end' : 'all',rect: {diff --git a/src/chrome/src/content/page-monitor.js b/src/chrome/src/content/page-monitor.jsindex 9a1a59133..008cf1a9d 100644--- a/src/chrome/src/content/page-monitor.js+++ b/src/chrome/src/content/page-monitor.js@@ -199,8 +199,11 @@|| proof.runToken !== runToken || proof.documentToken !== documentToken|| proof.userAt !== lastUserAt || proof.interventionRevision !== interventionRevision|| (proof.focusRequired !== false && deepActiveElement() !== op.preparedBinding.target)) continue;- const before = proof.binding, after = before.relaxed- ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation);+ const before = proof.binding, keyboard = !before.relaxed && keyboardBoundField(op);+ const footprint = before.relaxed+ ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation, { keyboard });+ // A keyboard-bound proof keeps its original hit (see keyboardBoundField).+ const after = footprint && keyboard ? { ...footprint, hit: before.hit } : footprint;if (!after) continue;if (before.relaxed) {const original = proof.viewport;@@ -736,10 +739,12 @@// Relaxed retry bindings carry no geometry; their grant rules apply instead.if (op.preparedBinding?.model !== true || op.preparedBinding.relaxed || op.allowPassiveRebase !== true|| op.modelBindingInvalid || op.layoutInvalidated || authorityChanged(op)) return;- const before = preparedBindingFor(op.preparedBinding);- const after = actionFootprint(op.preparedBinding.target, op.preparedBinding.readonlyPreparation);- if (samePreparationSemantics(before, after) && before.hit === after.hit) rebaseOwnedGeometry(op.preparedBinding, after,- phase.kind === 'selector-scroll' && (phase.scrollX !== window.scrollX || phase.scrollY !== window.scrollY));+ const before = preparedBindingFor(op.preparedBinding), keyboard = keyboardBoundField(op);+ const after = actionFootprint(op.preparedBinding.target, op.preparedBinding.readonlyPreparation, { keyboard });+ if (keyboard ? sameKeyboardBinding(before, after, true) : samePreparationSemantics(before, after) && before.hit === after.hit) {+ rebaseOwnedGeometry(op.preparedBinding, keyboard ? { ...after, hit: before.hit } : after,+ phase.kind === 'selector-scroll' && (phase.scrollX !== window.scrollX || phase.scrollY !== window.scrollY));+ }}function bindingPredicate(before, after) {if (!before || !after) return 'semantic_binding_changed';@@ -1375,7 +1380,10 @@}return null;}- function actionFootprint(target, readonlyPreparation = null) {+ // `keyboard` leaves out which node is on top at the target's centre; only+ // keyboard input to the focused field is checked against such a footprint+ // (see keyboardBoundField).+ function actionFootprint(target, readonlyPreparation = null, { keyboard = false } = {}) {if (readonlyPreparation && !readonlyPreparationMatches(readonlyPreparation, target)) return null;const structuralFile = target instanceof Element && target.tagName === 'INPUT' && target.type === 'file';if (!(target instanceof Element) || target.ownerDocument !== document || !target.isConnected@@ -1662,10 +1670,13 @@for (const heading of headings) if (!addNode(heading, { text: true })) return null;}const rect = structuralFile ? null : target.getBoundingClientRect();- const rawHit = structuralFile ? null : deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);- if (!structuralFile && (!rawHit || !(target === rawHit || target.contains(rawHit) || (action.contains(target) && action.contains(rawHit))))) return null;- const hit = structuralFile ? null : modelHit(target, action, rawHit);- return { model: true, structuralFile, target, scope: owner || form || action, ancestors, nodes, hit,+ const rawHit = structuralFile || keyboard ? null : deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);+ if (!structuralFile && !keyboard && (!rawHit || !(target === rawHit || target.contains(rawHit) || (action.contains(target) && action.contains(rawHit))))) return null;+ const hit = structuralFile || keyboard ? null : modelHit(target, action, rawHit);+ // Whether the target was an editable host when certified; a later+ // contenteditable/readonly flip cannot change what it was chosen as.+ const editable = target.isContentEditable === true || /^(INPUT|TEXTAREA)$/.test(target.tagName);+ return { model: true, structuralFile, target, scope: owner || form || action, ancestors, nodes, hit, editable,dialogIdentities,visualImages,...(fileWidget ? { fileWidget } : {}),@@ -1681,7 +1692,8 @@}function operationBinding(op, target = op?.preparedBinding?.target) {if (op?.preparedBinding?.relaxed) return relaxedTargetBinding(target, op.preparedBinding.grantToken);- return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation) : preparedBinding(target);+ return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation,+ { keyboard: target === op.preparedBinding.target && keyboardBoundField(op) }) : preparedBinding(target);}function inputContinuationValue(item, target) {if (item.node !== target) return item.value;@@ -1714,8 +1726,11 @@}));}function operationBindingMatches(op, target = op?.preparedBinding?.target) {- const current = operationBinding(op, target);- const matches = samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);+ let current = operationBinding(op, target);+ const before = preparedBindingFor(op?.preparedBinding);+ // Keyboard input to the focused field does not depend on the centre hit.+ if (current && before && target === op?.preparedBinding?.target && keyboardBoundField(op)) current = { ...current, hit: before.hit };+ const matches = samePreparedBinding(before, current) || inputContinuationBindingMatches(op, current);if (op?.preparedBinding?.relaxed && (!matches || !relaxedActionability(target, { input: kindFor(op.modelBindingTool || op.tool) === 'input' }))) {revokeRelaxedGrant(); return false;}@@ -1773,6 +1788,7 @@.map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b)),form.action || null, form.method || null, form.target || null, form.enctype || null]) : null;return { model: true, relaxed: true, grantToken, target, scope: bindingParent(target), form, formIdentity,+ editable: target.isContentEditable === true || /^(INPUT|TEXTAREA)$/.test(target.tagName),clickHandler: target.onclick, submitHandler: form?.onsubmit,identity: JSON.stringify([fixed, content]),continuationIdentity: JSON.stringify([fixed, target.isContentEditable ? null : target.textContent]),@@ -1856,8 +1872,16 @@if (!relaxed && pendingDOM?.source === 'user') return { reason: 'pending_feedback' };const target = modelTarget(params);if (!target) {- if (relaxed) { revokeRelaxedGrant(); return { reason: 'target_unresolved' }; }- return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };+ // A selector naming several elements (for example a composer and its+ // hidden copy behind a dialog) can never bind; say so, so the model+ // can use the observed element's ref instead.+ let ambiguous = false;+ if (typeof params.selector === 'string' && !params.ref_id && modelActionTools.has(params.tool)) {+ try { const query = queryOpenTargets(params.selector); ambiguous = query.complete && query.elements.length > 1; } catch {}+ }+ const reason = ambiguous ? 'target_ambiguous' : 'target_unresolved';+ if (relaxed) { revokeRelaxedGrant(); return { reason }; }+ return !ambiguous && params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason };}const before = snapshot.targets.get(target), previous = snapshot.uncertified.get(target);if (!relaxed && !before && !previous) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };@@ -1880,6 +1904,15 @@return { binding: original, relaxed: true, certificationFailure: snapshot.captureFailures.get(target) || undefined };}if (before?.readonlyPreparation && params.tool !== 'click_ax') return {};+ if (before?.keyboardOnly) {+ // Only keyboard input to the still focused field may use it.+ if (!keyboardInputTools.has(params.tool)) return { reason: 'target_uncovered', certificationFailure: 'target_uncovered' };+ if (deepActiveElement() !== target || !textEntryField(target)) return { reason: 'focus_changed' };+ const current = actionFootprint(target, null, { keyboard: true });+ if (!sameKeyboardBinding(preparedBindingFor(before), current, true)) return { reason: bindingPredicate(preparedBindingFor(before), current) || 'semantic_binding_changed' };+ if (preparedBindingFor(before).context !== current.context) rebaseOwnedGeometry(before, current);+ return { binding: before };+ }if (before) {const reason = modelDecisionPredicate(before, actionFootprint(target, before.readonlyPreparation));return reason ? { reason } : { binding: before };@@ -2154,7 +2187,14 @@if (binding) relaxedTargets.set(node, binding);}const limitReached = targets.size >= MODEL_TARGET_LIMIT;- const footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;+ let footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;+ // The focused text field may be covered at its centre by something that+ // does not take focus, such as the formatting bubble X draws where its+ // editor was clicked. It stays certified for keyboard input only.+ if (!footprint && !limitReached && node === deepActiveElement() && textEntryField(node)) {+ footprint = actionFootprint(node, null, { keyboard: true });+ if (footprint) footprint.keyboardOnly = true;+ }if (footprint && !observedFootprint(footprint)) { captureFailures.set(node, 'identity_unobserved'); return; }if (!footprint && page && !publicNodes.has(node)) return;if (footprint) targets.set(node, footprint);@@ -2214,6 +2254,81 @@function ownedPreparationPhase(op) {return op?.pointerPreparationOnly === true || op?.typingPreparationOnly === true || op?.selectorPreparationOnly === true;}+ const textEntryInputTypes = new Set(['text', 'search', 'email', 'url', 'tel', 'password', 'number']);+ function textEntryField(el) {+ if (!(el instanceof Element) || !el.isConnected || el.disabled === true || el.readOnly === true+ || el.closest('fieldset[disabled]') || ['aria-disabled', 'aria-readonly'].some(name => el.getAttribute(name) === 'true')) return false;+ if (el.tagName === 'TEXTAREA') return true;+ if (el.tagName === 'INPUT') return textEntryInputTypes.has(el.type);+ return el.isContentEditable === true;+ }+ // A certified typing action may press its own text field once, so that the+ // browser rather than a script focuses it, with trusted events (Chrome's+ // contenteditable fallback does so). Only one single left press whose+ // native receiver is exactly the certified field, before any other input of+ // the action, and its own release on that field qualify. Like hover or+ // scroll preparation it does not use up the certificate: what the page does+ // in reply is judged by the field's footprint, genuine user input still+ // stops the action, and the text needs its own dispatch.+ function typingFocusPhase(op, params) {+ if (!active || !op || op.finished || params.kind !== 'click' || params.fenceOnly === true+ || op.preparedBinding?.model !== true || op.preparedBinding.relaxed || op.allowPassiveRebase !== true+ || op.modelBindingInvalid || op.layoutInvalidated || op.coordinateSensitive+ || !keyboardTypingTools.has(op.modelBindingTool || op.tool)) return false;+ const field = op.preparedBinding.target;+ // Both halves of the press must reach the field itself. A page that draws+ // over the point on mousedown receives the mouseup, so that release is+ // the action's input to the page, not its preparation.+ const receiverIsField = typeof params.nativeTarget === 'string' && !!params.nativeTarget+ && resolveTarget({ nativeTarget: params.nativeTarget }) === field && textEntryField(field);+ if (params.release === true) {+ return op.typingFocusPress === true && op.typingPreparationOnly === true && !op.consequentialInputPending+ && params.x === op.x && params.y === op.y && receiverIsField;+ }+ // One press: a second press of the same action is consequential.+ if (op.consequentialInputPending || op.typingFocusPressed === true || params.pointerPreparationOnly === true+ || params.pointerButton !== 'left' || params.clickCount !== 1) return false;+ return receiverIsField;+ }+ // That press has not used the certificate. Reselecting the same field for+ // the text command still carries it (see beginContentAction).+ function ownedTypingFocus(op) {+ return op?.dispatched === true && !op.finished && op.typingFocusPress === true && op.typingPreparationOnly === true+ && !op.consequentialInputPending && op.preparedBinding?.model === true && !op.preparedBinding.relaxed;+ }+ // After that press, the action's remaining steps (restoring the selection,+ // the text command) are keyboard input to its field. Which node is on top+ // at the field's centre no longer matters to them: a page may draw a+ // formatting bubble right where the press landed, as X does. Every other+ // part of the field's footprint (field, owner, labels, values, visibility,+ // geometry, URL) must hold. Each step focuses the field itself, and the text+ // command still requires the focused node to be that field.+ const keyboardTypingTools = new Set(['type_ax', 'set_field', 'type_text', 'type']);+ // press_keys never presses a field, but its keys go to the focused one: a+ // field certified for keyboard use only (keyboardOnly) serves it as well.+ const keyboardInputTools = new Set([...keyboardTypingTools, 'press_keys']);+ function keyboardBoundField(op) {+ const field = op?.preparedBinding?.target;+ return op?.keyboardBound === true && op.preparedBinding?.model === true && !op.preparedBinding.relaxed+ && keyboardInputTools.has(op.modelBindingTool || op.tool) && textEntryField(field);+ }+ function sameKeyboardBinding(before, after, ownedGeometry = false) {+ if (!before?.model || !after?.model || before.relaxed || after.relaxed || before.target !== after.target) return false;+ return samePreparedBinding(before, { ...after, hit: before.hit,+ ...(ownedGeometry ? { context: withOriginalGeometry(before, after) } : {}) });+ }+ // modelDecisionPredicate for a keyboard-bound field: the field and every+ // semantic part of its footprint must hold, its centre hit is not compared.+ // When only geometry moved, the proof is rebased (keeping its original+ // hit), and exact checks apply from here on.+ function keyboardDecisionPredicate(binding, after) {+ const before = preparedBindingFor(binding);+ if (!sameKeyboardBinding(before, after, true)) {+ return (before && after && bindingPredicate(before, { ...after, hit: before.hit })) || 'semantic_binding_changed';+ }+ if (before.context !== after.context) rebaseOwnedGeometry(binding, { ...after, hit: before.hit });+ return '';+ }function preparedBindingChanged(op, { allowFocusRebind = false } = {}) {if (op?.allowPassiveRebase === true && op.preparedBinding?.model === true && op.focusBound&& !allowFocusRebind && deepActiveElement() !== op.focusTarget) return true;@@ -2238,6 +2353,13 @@&& !relaxedActionability(op.preparedBinding.target, { hit: true, input: params.kind === 'input', point })) {revokeRelaxedGrant(); return point ? 'target_hit_changed' : 'target_not_actionable';}+ // Native keyboard input always tries to mark its receiver. Input whose+ // receiver could not be marked (no focused node, a frame without an+ // active monitor, or a page that removed the revision fence) is not shown+ // to reach a certified target, and a keyboard-bound field is no longer+ // checked at its centre either.+ if (params.nativeInput === true && params.kind === 'input' && params.release !== true && !params.nativeTarget+ && op?.preparedBinding?.model === true) return 'target_unresolved';if (!params.nativeTarget || op?.preparedBinding?.model !== true) return '';// Resolve the actual transport marker alone; a stale marker cannot fall// back to the requested selector or ref and borrow its certificate.@@ -3147,6 +3269,8 @@modelBindingReason: proofRequired && !certified ? bindingResult?.reason || 'semantic_binding_changed' : '',preparedTargetSignature: operationTarget ? signature(operationTarget) : undefined, kinds: new Set(),preparedBinding: proofRequired ? certified : allowPassiveRebase && operationTarget ? preparedBinding(operationTarget) : null,+ // A field certified for keyboard input only (see captureTarget).+ keyboardBound: inherited ? previous.keyboardBound === true : certified?.keyboardOnly === true,preparedInterventionRevision: interventionRevision, preparedUncertainRevision: uncertainRevision, localSynthetic: false,preparedPageContext: JSON.stringify([location.href, document.baseURI, currentViewport(), window.scrollX, window.scrollY]),coordinateSensitive, coordinateTarget, coordinateHit, coordinateTargetAtPoint,@@ -3179,6 +3303,15 @@|| (op.dispatched && !op.finished && op.pointerPreparationOnly !== true && op.selectorPreparationOnly !== true && op.typingPreparationOnly !== true);op.selectorPreparationOnly = selectorPreparationOnly === true;op.typingPreparationOnly = typingPreparationOnly === true && !consequentialInputPending;+ // Only page_monitor_dispatch derives this from the actual press itself.+ op.typingFocusPress = op.typingPreparationOnly && params.kind === 'click';+ if (op.typingFocusPress) {+ op.keyboardBound = true;+ if (!params.release) op.typingFocusPressed = true;+ }+ // Any other press of the action, or a release that did not reach the+ // field, ends its keyboard binding.+ else if (params.kind === 'click' && !params.fenceOnly && params.pointerPreparationOnly !== true) op.keyboardBound = false;const movementEvents = ['pointerover', 'pointerenter', 'mouseover', 'mouseenter', 'pointermove', 'mousemove'];op.pointerPreparationOnly = !consequentialInputPending && params.pointerPreparationOnly === true&& !params.fenceOnly && !params.release && params.kind === 'click' && params.navigationCandidate === false@@ -3283,6 +3416,8 @@'hover', 'drag_drop', 'patch_element', 'revert_patch', 'highlight_element', 'execute_js','ax_prepare_field_for_trusted_type', 'ax_resolve_two_rects', 'ax_resolve_rect']);function beginContentAction(action, params = {}) {+ // The steps of a typing action that follow its own focus press.+ const afterFocusPress = ['ax_prepare_field_for_trusted_type', 'ax_paste_draft'].includes(action);// Internal DraftJS paste is the same ref-bound input operation as type_ax.// Keep its focus and synchronous editor changes inside the normal input owner.if (action === 'ax_paste_draft') action = 'type_ax';@@ -3294,12 +3429,25 @@|| (kindFor(action) === 'input' ? deepActiveElement() : null);const bindingTool = action === 'ax_prepare_field_for_trusted_type' && params.expectedModelSnapshot&& currentRelaxedGrant(params.relaxedMonitorGrantToken) ? 'set_field' : action;- const approved = [...operations.values()].reverse().find(op => op.allowPassiveRebase === true && !op.dispatched- && op.preparedBinding?.target === target && op.until >= Date.now()- && lastUserAt <= op.userAt- && (op.preparedBinding.relaxed ? currentRelaxedGrant(op.preparedBinding.grantToken) && operationBindingMatches(op, target)- : !authorityChanged(op) && (op.preparedBinding.model === true- ? !modelDecisionPredicate(op.preparedBinding, operationBinding(op, target)) : operationBindingMatches(op, target))));+ // A typing action's own focus press leaves its certificate with the+ // steps that follow it on that field: reselecting it for the text+ // command, or DraftJS's paste. They are keyboard-bound (see+ // keyboardBoundField), so the node on top at its centre does not matter.+ const approved = [...operations.values()].reverse().find(op => {+ const focusPressed = op.dispatched && afterFocusPress && ownedTypingFocus(op);+ if (op.allowPassiveRebase !== true || (op.dispatched && !focusPressed) || op.preparedBinding?.target !== target+ || !(op.until >= Date.now()) || lastUserAt > op.userAt) return false;+ if (op.preparedBinding.relaxed) return currentRelaxedGrant(op.preparedBinding.grantToken) && operationBindingMatches(op, target);+ if (authorityChanged(op)) return false;+ if (op.preparedBinding.model !== true) return operationBindingMatches(op, target);+ // A keyboard-bound field's input steps are compared without its centre+ // hit (see keyboardBoundField); anything else as a model decision.+ if (kindFor(action) === 'input' && keyboardBoundField(op)) {+ return !keyboardDecisionPredicate(op.preparedBinding, operationBinding(op, target));+ }+ if (focusPressed) return false;+ return !modelDecisionPredicate(op.preparedBinding, actionFootprint(target, op.preparedBinding.readonlyPreparation));+ });prepare({ selector: params.selector, ref_id: params.ref_id, x: params.x, y: params.y,textMatch: action === 'click' ? params.text : undefined, tool: bindingTool, operationId, runToken,...(params.expectedModelSnapshot !== undefined || approved?.expectedModelSnapshot !== undefined@@ -3318,6 +3466,8 @@local.focusBound = approved.focusBound;local.modelBindingInvalid = false;local.allowPassiveRebase = true;+ local.keyboardBound = approved.keyboardBound === true && kindFor(action) === 'input'+ && (approved.dispatched !== true || ownedTypingFocus(approved));}const installed = localOperation = { operationId, kind: kindFor(action), userAt: lastUserAt, revision,navigationCandidate: ['click', 'click_ax', 'set_checked'].includes(action),@@ -3395,19 +3545,23 @@try {const result = callback();flushPendingMutations(op); sampleFormControls(target);- const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken) : actionFootprint(target, before.readonlyPreparation);+ const keyboard = !before.relaxed && keyboardBoundField(op);+ const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken)+ : actionFootprint(target, before.readonlyPreparation, { keyboard });const changedAuthority = !active || runToken !== preparedRun || lastUserAt !== preparedUserAt|| (before.relaxed ? !currentRelaxedGrant(before.grantToken) : interventionRevision !== preparedIntervention || interventionUnreported);const changedFocus = kind === 'focus' && deepActiveElement() !== target;- if (changedAuthority || changedFocus || !samePreparationSemantics(before, after)) {+ const sameSemantics = samePreparationSemantics(before, after) || (keyboard && sameKeyboardBinding(before, after, true));+ if (changedAuthority || changedFocus || !sameSemantics) {const error = new Error('Browser changed during action preparation. Re-observe before acting.');error.code = 'page_feedback_pending'; error.dispatched = localOperation?.started === true;- error.pagePreparationVeto = { predicate: changedAuthority ? 'intervention_changed' : changedFocus ? 'focus_changed' : bindingPredicate(before, after) || 'semantic_binding_changed' };+ error.pagePreparationVeto = { predicate: changedAuthority ? 'intervention_changed' : changedFocus ? 'focus_changed'+ : bindingPredicate(before, keyboard && after ? { ...after, hit: before.hit } : after) || 'semantic_binding_changed' };throw error;}// Keep the observation's exact semantic/private values immutable. Only a// derived geometry proof for this original node is shared with its owner.- ownedPreparationBindings.set(op.preparedBinding, { binding: after, runToken, documentToken,+ ownedPreparationBindings.set(op.preparedBinding, { binding: keyboard ? { ...after, hit: before.hit } : after, runToken, documentToken,userAt: lastUserAt, interventionRevision,...(before.relaxed ? { viewport: preparationViewport } : {}),scrollPreparationUntil: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)@@ -3530,13 +3684,35 @@void lastFeedbackDelivery.then(() => respond({ ready: false, pageFeedbackPending: true, preparationVeto }));return true;}- dispatch(params); publishRevision();- respond({ ready: true, ...(active && params.runToken === runToken ? { guard: {+ // A typing action's own focus press (and its release) is preparation;+ // the reply says so, for diagnostics.+ const typingPreparation = typingFocusPhase(prepared, params);+ dispatch(params, false, typingPreparation); publishRevision();+ respond({ ready: true, ...(typingPreparation ? { typingPreparation: true } : {}), ...(active && params.runToken === runToken ? { guard: {documentToken, revision, operationId: params.operationId, runToken,navigationCandidate: params.navigationCandidate !== false,...(operations.get(params.operationId)?.nativeSecret ? { nativeSecret: operations.get(params.operationId).nativeSecret } : {}),} } : {}) });}+ else if (msg.action === 'page_monitor_receiver_frame') {+ // Keyboard input of this document's certified operation is about to+ // reach a child document. That is its input only when the certified+ // target is the frame that has focus here (or contains it), was not an+ // editable host when certified (an editor's text never goes into a+ // frame embedded in it), and its certificate still holds. An unknown+ // operation cannot be shown to allow it.+ const params = msg.params || {};+ const current = active && params.runToken === runToken;+ const op = current ? operations.get(params.operationId) : null;+ const binding = op?.preparedBinding, target = binding?.target, frame = deepActiveElement();+ const before = preparedBindingFor(binding), after = target instanceof Element ? operationBinding(op, target) : null;+ const allowed = current && binding?.model === true && binding.editable !== true && target instanceof Element+ && target.isConnected && frame instanceof Element && /^(IFRAME|FRAME)$/.test(frame.tagName)+ && (target === frame || target.contains(frame))+ // Only its geometry may have moved, as for any certified decision.+ && (before?.relaxed ? sameRelaxedBinding(before, after) : samePreparationSemantics(before, after));+ respond({ ready: current, foreignReceiverAllowed: allowed === true });+ }else if (msg.action === 'page_monitor_finish') {const params = msg.params || {};const op = operations.get(params.operationId);diff --git a/src/chrome/src/trace/privacy.js b/src/chrome/src/trace/privacy.jsindex b213e2b98..9b61aacbb 100644--- a/src/chrome/src/trace/privacy.js+++ b/src/chrome/src/trace/privacy.js@@ -77,7 +77,7 @@ const RECOVERY_STAGES = new Set([const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass', 'action_monitoring_relaxed']);const CERTIFICATION_FAILURES = new Set(['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit','target_uncovered', 'context_uncertified']);-const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_uncovered', 'focus_changed',+const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_ambiguous', 'target_uncovered', 'focus_changed','intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired', 'pending_feedback','target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed','relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable']);diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex 64ac57b13..204273abd 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -9121,6 +9121,10 @@ export class Agent extends LoopDetector {if (!['set_field', 'type_ax', 'type_text', 'iframe_type'].includes(toolName)) {return { block: null, shot: null };}+ // A target the observed certificate cannot bind (such as a selector that+ // names several elements) gets that answer before any probe of it.+ const unbindable = await this._pageFeedbackUnbindableResult?.(tabId, toolName, args);+ if (unbindable) return { block: unbindable, shot: null };const selectorBackedType = toolName === 'type_text'&& typeof args?.selector === 'string'&& !!args.selector.trim();diff --git a/src/firefox/src/agent/page-feedback.js b/src/firefox/src/agent/page-feedback.jsindex 5b6eec08a..14505198f 100644--- a/src/firefox/src/agent/page-feedback.js+++ b/src/firefox/src/agent/page-feedback.js@@ -207,11 +207,20 @@ export function pageFeedbackPendingResult() {pageFeedbackPending: true, error: 'The browser changed during action preparation. Re-observe the page before acting.' };}+// The answer of a call whose target the observed certificate cannot bind.+function actionBindingUnavailableResult(run) {+ return { success: false, skipped: true, dispatched: false, noDispatch: true, retryable: false,+ errorCode: 'action_binding_unavailable',+ ...(run.actionBindingUnavailableReason === 'target_ambiguous' ? { bindingFailure: 'target_ambiguous',+ error: 'The selector matches more than one element on this page (for example a hidden copy of the same field behind a dialog), so it cannot be bound to the element you observed. Nothing was sent. Use the ref_id of the intended element from the current page observation instead of this selector.' }+ : { error: 'The exact target could not be bound to the observed action context. Fresh available target/page context will be observed before the next decision; inspect it or read the exact target inventory before trying the action again.' }) };+}+function preparationVetoDiagnostics(veto) {const facts = {};if (!veto || typeof veto !== 'object') return facts;- if (['owner_pending_before', 'owner_pending_after', 'monitor_rejected',- 'document_mapping_changed', 'owner_ended', 'current_owner_changed'].includes(veto.cause)) facts.cause = veto.cause;+ if (['owner_pending_before', 'owner_pending_after', 'monitor_rejected', 'document_mapping_changed',+ 'receiver_frame_changed', 'owner_ended', 'current_owner_changed'].includes(veto.cause)) facts.cause = veto.cause;// Preserve diagnostics through every wrapper without copying page values.for (const key of ['ownerPendingAfter', 'monitorRejected',@@ -249,9 +258,28 @@ export async function beforePageAgentDispatch(api, tabId, details = {}) {}details = { ...details, ...(frameId === undefined ? {} : { frameId }) };}- owner.operationFrames.get(owner.operationId)?.add(Number(details.frameId) || 0);const { prepareMonitor = false, ...dispatchDetails } = details;const operationId = dispatchDetails.operationId || owner.operationId || token();+ // A model certificate binds a node of the top document. Keyboard input+ // whose receiver has focus in a child document (a page that moved focus+ // into an iframe) is that operation's input only if the top document says+ // its certified target is the frame that has focus.+ if (!details.release && details.kind === 'input' && Number(details.frameId) > 0 && owner.modelBound?.()) {+ let reply = null;+ try {+ reply = await api.tabs.sendMessage(tabId, { target: 'content', action: 'page_monitor_receiver_frame',+ params: { runToken: owner.runToken, operationId } }, { frameId: 0 });+ } catch { /* No answer cannot allow it. */ }+ if (dispatchOwners.get(tabId) !== owner) {+ const error = new Error('Browser run ended before dispatch'); error.name = 'AbortError';+ error.pagePreparationVeto = { cause: 'owner_ended' }; throw error;+ }+ if (reply?.foreignReceiverAllowed !== true) {+ const error = new Error(pageFeedbackPendingResult().error);+ error.code = 'page_feedback_pending'; error.pagePreparationVeto = { cause: 'receiver_frame_changed' }; throw error;+ }+ }+ owner.operationFrames.get(owner.operationId)?.add(Number(details.frameId) || 0);const monitorParams = { ...dispatchDetails, runToken: owner.runToken, operationId };let acknowledgement;let preparationAcknowledgement;@@ -398,6 +426,8 @@ export const pageFeedbackMethods = {};dispatchOwners.set(tabId, { runToken: run.token, operationId: '', operationFrames: new Map(),tool: () => run.dispatchCall?.name || '',+ // The current call acts on a certified node of the top document.+ modelBound: () => !!run.dispatchCall?.expectedModelSnapshot,pending: () => this._pageFeedbackBlocksCall(tabId, run.dispatchCall?.name, run.dispatchCall?.args),allowPassiveRebase: () => run.dispatchCall?.allowPassiveRebase === true,expectedModelSnapshot: () => run.dispatchCall?.dispatchState?.started ? undefined : run.dispatchCall?.expectedModelSnapshot,@@ -1101,10 +1131,12 @@ export const pageFeedbackMethods = {&& run.frames.get(id)?.id === frame.id);},- async _validatePageFeedbackAction(tabId, name, args, state) {+ // answerOnly: an early check that only answers a target the certificate+ // cannot bind; any other outcome is left to the call's own validation.+ async _validatePageFeedbackAction(tabId, name, args, state, { answerOnly = false } = {}) {const run = this._pageFeedbackRuns?.get(tabId);const binding = state?.actionBinding;- if (run) { run.actionBindingUnavailable = null; run.actionContextUnverified = false; }+ if (run) { run.actionBindingUnavailable = null; run.actionBindingUnavailableReason = null; run.actionContextUnverified = false; }if (!run || !binding || run.modelState !== state || !this._pageFeedbackStateCurrent(tabId, run, state)|| (run.events.size && !isAutomaticPageFeedback([...run.events.values()]))) return false;try {@@ -1122,13 +1154,18 @@ export const pageFeedbackMethods = {&& (response.relaxed !== true || (relaxedMonitorGrantToken && this._monitoringRetryCurrent(tabId, run) === retry));const certificationFailure = ['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit','target_uncovered', 'context_uncertified'].includes(response?.certificationFailure) ? response.certificationFailure : null;+ const unbindable = owned && ['target_uncovered', 'target_unresolved', 'target_ambiguous'].includes(response.reason);+ if (answerOnly && !unbindable) {+ if (valid) (run.validatedActionBindings ??= new Map()).set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);+ return valid;+ }if (owned) {const runId = this.currentRunId?.get(tabId);if (runId) trace.recordNote(runId, 0, 'page_action_validation', {status: valid ? 'ready' : 'blocked', monitoringRelaxed: valid && response.relaxed === true,...(certificationFailure ? { certificationFailure } : {}),...(response.uncertified === true ? { certificationFailure: certificationFailure || 'context_uncertified' } : {}),- ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_uncovered',+ ...(typeof response.reason === 'string' && ['target_changed', 'target_unresolved', 'target_ambiguous', 'target_uncovered','focus_changed', 'intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired','pending_feedback', 'target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed','relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable'].includes(response.reason) ? { predicate: response.reason } : {}),@@ -1139,11 +1176,12 @@ export const pageFeedbackMethods = {run.validatedActionBindings ??= new Map();run.validatedActionBindings.set(`${name}:${JSON.stringify(args)}`, binding.snapshotToken);}- if (owned && ['target_uncovered', 'target_unresolved'].includes(response.reason)) {+ if (unbindable) {// Preserve the original no-dispatch outcome. A bounded fresh observation// of this target belongs to the NEXT normal decision, never this call.run.actionCaptureTarget = safeAction(name, args);run.actionBindingUnavailable = `${name}:${JSON.stringify(args)}`;+ run.actionBindingUnavailableReason = response.reason;}if (owned && response.uncertified === true) {// The original node is unchanged, but its action context was never@@ -1158,6 +1196,18 @@ export const pageFeedbackMethods = {} catch { return false; }},+ // A typing call's own preparation probes its target before dispatch. A+ // target the certificate cannot bind at all is answered first and+ // precisely: a probe of an ambiguous selector could only pick one of its+ // elements. Anything else is left to the dispatch wrapper.+ async _pageFeedbackUnbindableResult(tabId, name, args = {}) {+ const run = this._pageFeedbackRuns?.get(tabId);+ if (!run || typeof args?.selector !== 'string' || !args.selector.trim() || !this.constructor.STATE_CHANGE_TOOLS?.has(name)+ || pageFeedbackCallPolicy(name, args, run.modelState, run.latestPage).kind !== 'bound_target') return null;+ if (await this._validatePageFeedbackAction(tabId, name, args, run.modelState, { answerOnly: true }) !== false) return null;+ return run.actionBindingUnavailable === `${name}:${JSON.stringify(args)}` ? actionBindingUnavailableResult(run) : null;+ },+_pageFeedbackBlocksCall(tabId, name, args = {}) {const run = this._pageFeedbackRuns?.get(tabId);if (run?.dispatchCall?.expectedModelSnapshot@@ -1368,13 +1418,26 @@ export const pageFeedbackMethods = {this._monitoringRetryCurrent(tabId, run) ? current.map(event => event.source === 'unknown' && event.kind === 'dom'? { ...event, source: 'page' } : event) : current, options);};- let unverifiedActionContext = false;+ let unverifiedActionContext = false, unbindableActionTarget = false;let retained = sameDocument && canRetain(events)&& (!pending.length || canRetain(pending, { pending: true }));if (retained) for (const call of responseToolCalls) {const args = JSON.parse(call.function.arguments);if (pageFeedbackCallPolicy(call.function.name, args, state, page, events).kind === 'bound_target'&& await this._validatePageFeedbackAction(tabId, call.function.name, args, state) !== true) {+ // A target the certificate cannot bind (a selector naming several+ // elements or none, a ref the observation did not certify, a control+ // its snapshot does not cover) gets the same answer after these page+ // updates, and discarding the response would only show the model the+ // same view to decide again. Keep the call: its own no-dispatch result+ // (action_binding_unavailable) tells the model why, and the rest of+ // its batch does not run. If a real change caused it (its element+ // removed or added), the attached fresh observation shows that. A+ // changed binding, an unverified action context and user input still+ // supersede the response.+ if (run.actionBindingUnavailable === `${call.function.name}:${JSON.stringify(args)}`) {+ unbindableActionTarget = true; continue;+ }unverifiedActionContext = run.actionContextUnverified === true;retained = false; break;}@@ -1463,7 +1526,8 @@ export const pageFeedbackMethods = {if (runId) trace.recordNote(runId, 0, 'page_feedback_decision', { stage, disposition: responseToolCalls === null ? 'observed' : accepted ? 'retained' : 'superseded',reason: accepted && independentCompletion && (!isPassivePageFeedback(events)|| (finalPending.length && !isPassivePageFeedback(finalPending))) ? 'page_independent_completion'- : honestUnverifiedContext && ['passive_dom', 'automatic_page'].includes(feedbackReason) ? 'target_context_unverified' : feedbackReason,+ : honestUnverifiedContext && ['passive_dom', 'automatic_page'].includes(feedbackReason) ? 'target_context_unverified'+ : accepted && unbindableActionTarget ? 'action_target_unbindable' : feedbackReason,context: { bindingAtRefresh: { documentMatches, currentRun, stateCurrent, sameDocument },feedback: { observed: feedbackEventSummary(events), pendingAtRefresh: feedbackEventSummary(pending),pendingAtDecision: feedbackEventSummary(finalPending) } },@@ -1499,9 +1563,7 @@ export function installPageFeedback(Agent) {if (modelBound) {const validated = await this._validatePageFeedbackAction(tabId, name, args, modelState);if (validated === false) return run.actionBindingUnavailable === `${name}:${JSON.stringify(args)}`- ? { success: false, skipped: true, dispatched: false, noDispatch: true, retryable: false,- errorCode: 'action_binding_unavailable',- error: 'The exact target could not be bound to the observed action context. Fresh available target/page context will be observed before the next decision; inspect it or read the exact target inventory before trying the action again.' }+ ? actionBindingUnavailableResult(run): run.actionContextUnverified === true? { ...pageFeedbackPendingResult(), actionContextUnverified: true, errorCode: 'action_context_unverified',error: 'The requested control had no verified original action context. No action was dispatched; inspect the fresh bounded target observation before choosing another action.' }diff --git a/src/firefox/src/content/page-monitor.js b/src/firefox/src/content/page-monitor.jsindex 9a1a59133..008cf1a9d 100644--- a/src/firefox/src/content/page-monitor.js+++ b/src/firefox/src/content/page-monitor.js@@ -199,8 +199,11 @@|| proof.runToken !== runToken || proof.documentToken !== documentToken|| proof.userAt !== lastUserAt || proof.interventionRevision !== interventionRevision|| (proof.focusRequired !== false && deepActiveElement() !== op.preparedBinding.target)) continue;- const before = proof.binding, after = before.relaxed- ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation);+ const before = proof.binding, keyboard = !before.relaxed && keyboardBoundField(op);+ const footprint = before.relaxed+ ? relaxedTargetBinding(before.target, before.grantToken) : actionFootprint(before.target, before.readonlyPreparation, { keyboard });+ // A keyboard-bound proof keeps its original hit (see keyboardBoundField).+ const after = footprint && keyboard ? { ...footprint, hit: before.hit } : footprint;if (!after) continue;if (before.relaxed) {const original = proof.viewport;@@ -736,10 +739,12 @@// Relaxed retry bindings carry no geometry; their grant rules apply instead.if (op.preparedBinding?.model !== true || op.preparedBinding.relaxed || op.allowPassiveRebase !== true|| op.modelBindingInvalid || op.layoutInvalidated || authorityChanged(op)) return;- const before = preparedBindingFor(op.preparedBinding);- const after = actionFootprint(op.preparedBinding.target, op.preparedBinding.readonlyPreparation);- if (samePreparationSemantics(before, after) && before.hit === after.hit) rebaseOwnedGeometry(op.preparedBinding, after,- phase.kind === 'selector-scroll' && (phase.scrollX !== window.scrollX || phase.scrollY !== window.scrollY));+ const before = preparedBindingFor(op.preparedBinding), keyboard = keyboardBoundField(op);+ const after = actionFootprint(op.preparedBinding.target, op.preparedBinding.readonlyPreparation, { keyboard });+ if (keyboard ? sameKeyboardBinding(before, after, true) : samePreparationSemantics(before, after) && before.hit === after.hit) {+ rebaseOwnedGeometry(op.preparedBinding, keyboard ? { ...after, hit: before.hit } : after,+ phase.kind === 'selector-scroll' && (phase.scrollX !== window.scrollX || phase.scrollY !== window.scrollY));+ }}function bindingPredicate(before, after) {if (!before || !after) return 'semantic_binding_changed';@@ -1375,7 +1380,10 @@}return null;}- function actionFootprint(target, readonlyPreparation = null) {+ // `keyboard` leaves out which node is on top at the target's centre; only+ // keyboard input to the focused field is checked against such a footprint+ // (see keyboardBoundField).+ function actionFootprint(target, readonlyPreparation = null, { keyboard = false } = {}) {if (readonlyPreparation && !readonlyPreparationMatches(readonlyPreparation, target)) return null;const structuralFile = target instanceof Element && target.tagName === 'INPUT' && target.type === 'file';if (!(target instanceof Element) || target.ownerDocument !== document || !target.isConnected@@ -1662,10 +1670,13 @@for (const heading of headings) if (!addNode(heading, { text: true })) return null;}const rect = structuralFile ? null : target.getBoundingClientRect();- const rawHit = structuralFile ? null : deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);- if (!structuralFile && (!rawHit || !(target === rawHit || target.contains(rawHit) || (action.contains(target) && action.contains(rawHit))))) return null;- const hit = structuralFile ? null : modelHit(target, action, rawHit);- return { model: true, structuralFile, target, scope: owner || form || action, ancestors, nodes, hit,+ const rawHit = structuralFile || keyboard ? null : deepElementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2);+ if (!structuralFile && !keyboard && (!rawHit || !(target === rawHit || target.contains(rawHit) || (action.contains(target) && action.contains(rawHit))))) return null;+ const hit = structuralFile || keyboard ? null : modelHit(target, action, rawHit);+ // Whether the target was an editable host when certified; a later+ // contenteditable/readonly flip cannot change what it was chosen as.+ const editable = target.isContentEditable === true || /^(INPUT|TEXTAREA)$/.test(target.tagName);+ return { model: true, structuralFile, target, scope: owner || form || action, ancestors, nodes, hit, editable,dialogIdentities,visualImages,...(fileWidget ? { fileWidget } : {}),@@ -1681,7 +1692,8 @@}function operationBinding(op, target = op?.preparedBinding?.target) {if (op?.preparedBinding?.relaxed) return relaxedTargetBinding(target, op.preparedBinding.grantToken);- return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation) : preparedBinding(target);+ return op?.preparedBinding?.model === true ? actionFootprint(target, op.preparedBinding.readonlyPreparation,+ { keyboard: target === op.preparedBinding.target && keyboardBoundField(op) }) : preparedBinding(target);}function inputContinuationValue(item, target) {if (item.node !== target) return item.value;@@ -1714,8 +1726,11 @@}));}function operationBindingMatches(op, target = op?.preparedBinding?.target) {- const current = operationBinding(op, target);- const matches = samePreparedBinding(preparedBindingFor(op?.preparedBinding), current) || inputContinuationBindingMatches(op, current);+ let current = operationBinding(op, target);+ const before = preparedBindingFor(op?.preparedBinding);+ // Keyboard input to the focused field does not depend on the centre hit.+ if (current && before && target === op?.preparedBinding?.target && keyboardBoundField(op)) current = { ...current, hit: before.hit };+ const matches = samePreparedBinding(before, current) || inputContinuationBindingMatches(op, current);if (op?.preparedBinding?.relaxed && (!matches || !relaxedActionability(target, { input: kindFor(op.modelBindingTool || op.tool) === 'input' }))) {revokeRelaxedGrant(); return false;}@@ -1773,6 +1788,7 @@.map(attribute => [attribute.name, attribute.value]).sort(([a], [b]) => a.localeCompare(b)),form.action || null, form.method || null, form.target || null, form.enctype || null]) : null;return { model: true, relaxed: true, grantToken, target, scope: bindingParent(target), form, formIdentity,+ editable: target.isContentEditable === true || /^(INPUT|TEXTAREA)$/.test(target.tagName),clickHandler: target.onclick, submitHandler: form?.onsubmit,identity: JSON.stringify([fixed, content]),continuationIdentity: JSON.stringify([fixed, target.isContentEditable ? null : target.textContent]),@@ -1856,8 +1872,16 @@if (!relaxed && pendingDOM?.source === 'user') return { reason: 'pending_feedback' };const target = modelTarget(params);if (!target) {- if (relaxed) { revokeRelaxedGrant(); return { reason: 'target_unresolved' }; }- return params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason: 'target_unresolved' };+ // A selector naming several elements (for example a composer and its+ // hidden copy behind a dialog) can never bind; say so, so the model+ // can use the observed element's ref instead.+ let ambiguous = false;+ if (typeof params.selector === 'string' && !params.ref_id && modelActionTools.has(params.tool)) {+ try { const query = queryOpenTargets(params.selector); ambiguous = query.complete && query.elements.length > 1; } catch {}+ }+ const reason = ambiguous ? 'target_ambiguous' : 'target_unresolved';+ if (relaxed) { revokeRelaxedGrant(); return { reason }; }+ return !ambiguous && params.ref_id && snapshot.refs.has(params.ref_id) ? {} : { reason };}const before = snapshot.targets.get(target), previous = snapshot.uncertified.get(target);if (!relaxed && !before && !previous) return { reason: 'target_uncovered', certificationFailure: snapshot.captureFailures.get(target) || 'identity_unobserved' };@@ -1880,6 +1904,15 @@return { binding: original, relaxed: true, certificationFailure: snapshot.captureFailures.get(target) || undefined };}if (before?.readonlyPreparation && params.tool !== 'click_ax') return {};+ if (before?.keyboardOnly) {+ // Only keyboard input to the still focused field may use it.+ if (!keyboardInputTools.has(params.tool)) return { reason: 'target_uncovered', certificationFailure: 'target_uncovered' };+ if (deepActiveElement() !== target || !textEntryField(target)) return { reason: 'focus_changed' };+ const current = actionFootprint(target, null, { keyboard: true });+ if (!sameKeyboardBinding(preparedBindingFor(before), current, true)) return { reason: bindingPredicate(preparedBindingFor(before), current) || 'semantic_binding_changed' };+ if (preparedBindingFor(before).context !== current.context) rebaseOwnedGeometry(before, current);+ return { binding: before };+ }if (before) {const reason = modelDecisionPredicate(before, actionFootprint(target, before.readonlyPreparation));return reason ? { reason } : { binding: before };@@ -2154,7 +2187,14 @@if (binding) relaxedTargets.set(node, binding);}const limitReached = targets.size >= MODEL_TARGET_LIMIT;- const footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;+ let footprint = !limitReached ? actionFootprint(node, node === readonlyPreparation?.target ? readonlyPreparation : null) : null;+ // The focused text field may be covered at its centre by something that+ // does not take focus, such as the formatting bubble X draws where its+ // editor was clicked. It stays certified for keyboard input only.+ if (!footprint && !limitReached && node === deepActiveElement() && textEntryField(node)) {+ footprint = actionFootprint(node, null, { keyboard: true });+ if (footprint) footprint.keyboardOnly = true;+ }if (footprint && !observedFootprint(footprint)) { captureFailures.set(node, 'identity_unobserved'); return; }if (!footprint && page && !publicNodes.has(node)) return;if (footprint) targets.set(node, footprint);@@ -2214,6 +2254,81 @@function ownedPreparationPhase(op) {return op?.pointerPreparationOnly === true || op?.typingPreparationOnly === true || op?.selectorPreparationOnly === true;}+ const textEntryInputTypes = new Set(['text', 'search', 'email', 'url', 'tel', 'password', 'number']);+ function textEntryField(el) {+ if (!(el instanceof Element) || !el.isConnected || el.disabled === true || el.readOnly === true+ || el.closest('fieldset[disabled]') || ['aria-disabled', 'aria-readonly'].some(name => el.getAttribute(name) === 'true')) return false;+ if (el.tagName === 'TEXTAREA') return true;+ if (el.tagName === 'INPUT') return textEntryInputTypes.has(el.type);+ return el.isContentEditable === true;+ }+ // A certified typing action may press its own text field once, so that the+ // browser rather than a script focuses it, with trusted events (Chrome's+ // contenteditable fallback does so). Only one single left press whose+ // native receiver is exactly the certified field, before any other input of+ // the action, and its own release on that field qualify. Like hover or+ // scroll preparation it does not use up the certificate: what the page does+ // in reply is judged by the field's footprint, genuine user input still+ // stops the action, and the text needs its own dispatch.+ function typingFocusPhase(op, params) {+ if (!active || !op || op.finished || params.kind !== 'click' || params.fenceOnly === true+ || op.preparedBinding?.model !== true || op.preparedBinding.relaxed || op.allowPassiveRebase !== true+ || op.modelBindingInvalid || op.layoutInvalidated || op.coordinateSensitive+ || !keyboardTypingTools.has(op.modelBindingTool || op.tool)) return false;+ const field = op.preparedBinding.target;+ // Both halves of the press must reach the field itself. A page that draws+ // over the point on mousedown receives the mouseup, so that release is+ // the action's input to the page, not its preparation.+ const receiverIsField = typeof params.nativeTarget === 'string' && !!params.nativeTarget+ && resolveTarget({ nativeTarget: params.nativeTarget }) === field && textEntryField(field);+ if (params.release === true) {+ return op.typingFocusPress === true && op.typingPreparationOnly === true && !op.consequentialInputPending+ && params.x === op.x && params.y === op.y && receiverIsField;+ }+ // One press: a second press of the same action is consequential.+ if (op.consequentialInputPending || op.typingFocusPressed === true || params.pointerPreparationOnly === true+ || params.pointerButton !== 'left' || params.clickCount !== 1) return false;+ return receiverIsField;+ }+ // That press has not used the certificate. Reselecting the same field for+ // the text command still carries it (see beginContentAction).+ function ownedTypingFocus(op) {+ return op?.dispatched === true && !op.finished && op.typingFocusPress === true && op.typingPreparationOnly === true+ && !op.consequentialInputPending && op.preparedBinding?.model === true && !op.preparedBinding.relaxed;+ }+ // After that press, the action's remaining steps (restoring the selection,+ // the text command) are keyboard input to its field. Which node is on top+ // at the field's centre no longer matters to them: a page may draw a+ // formatting bubble right where the press landed, as X does. Every other+ // part of the field's footprint (field, owner, labels, values, visibility,+ // geometry, URL) must hold. Each step focuses the field itself, and the text+ // command still requires the focused node to be that field.+ const keyboardTypingTools = new Set(['type_ax', 'set_field', 'type_text', 'type']);+ // press_keys never presses a field, but its keys go to the focused one: a+ // field certified for keyboard use only (keyboardOnly) serves it as well.+ const keyboardInputTools = new Set([...keyboardTypingTools, 'press_keys']);+ function keyboardBoundField(op) {+ const field = op?.preparedBinding?.target;+ return op?.keyboardBound === true && op.preparedBinding?.model === true && !op.preparedBinding.relaxed+ && keyboardInputTools.has(op.modelBindingTool || op.tool) && textEntryField(field);+ }+ function sameKeyboardBinding(before, after, ownedGeometry = false) {+ if (!before?.model || !after?.model || before.relaxed || after.relaxed || before.target !== after.target) return false;+ return samePreparedBinding(before, { ...after, hit: before.hit,+ ...(ownedGeometry ? { context: withOriginalGeometry(before, after) } : {}) });+ }+ // modelDecisionPredicate for a keyboard-bound field: the field and every+ // semantic part of its footprint must hold, its centre hit is not compared.+ // When only geometry moved, the proof is rebased (keeping its original+ // hit), and exact checks apply from here on.+ function keyboardDecisionPredicate(binding, after) {+ const before = preparedBindingFor(binding);+ if (!sameKeyboardBinding(before, after, true)) {+ return (before && after && bindingPredicate(before, { ...after, hit: before.hit })) || 'semantic_binding_changed';+ }+ if (before.context !== after.context) rebaseOwnedGeometry(binding, { ...after, hit: before.hit });+ return '';+ }function preparedBindingChanged(op, { allowFocusRebind = false } = {}) {if (op?.allowPassiveRebase === true && op.preparedBinding?.model === true && op.focusBound&& !allowFocusRebind && deepActiveElement() !== op.focusTarget) return true;@@ -2238,6 +2353,13 @@&& !relaxedActionability(op.preparedBinding.target, { hit: true, input: params.kind === 'input', point })) {revokeRelaxedGrant(); return point ? 'target_hit_changed' : 'target_not_actionable';}+ // Native keyboard input always tries to mark its receiver. Input whose+ // receiver could not be marked (no focused node, a frame without an+ // active monitor, or a page that removed the revision fence) is not shown+ // to reach a certified target, and a keyboard-bound field is no longer+ // checked at its centre either.+ if (params.nativeInput === true && params.kind === 'input' && params.release !== true && !params.nativeTarget+ && op?.preparedBinding?.model === true) return 'target_unresolved';if (!params.nativeTarget || op?.preparedBinding?.model !== true) return '';// Resolve the actual transport marker alone; a stale marker cannot fall// back to the requested selector or ref and borrow its certificate.@@ -3147,6 +3269,8 @@modelBindingReason: proofRequired && !certified ? bindingResult?.reason || 'semantic_binding_changed' : '',preparedTargetSignature: operationTarget ? signature(operationTarget) : undefined, kinds: new Set(),preparedBinding: proofRequired ? certified : allowPassiveRebase && operationTarget ? preparedBinding(operationTarget) : null,+ // A field certified for keyboard input only (see captureTarget).+ keyboardBound: inherited ? previous.keyboardBound === true : certified?.keyboardOnly === true,preparedInterventionRevision: interventionRevision, preparedUncertainRevision: uncertainRevision, localSynthetic: false,preparedPageContext: JSON.stringify([location.href, document.baseURI, currentViewport(), window.scrollX, window.scrollY]),coordinateSensitive, coordinateTarget, coordinateHit, coordinateTargetAtPoint,@@ -3179,6 +3303,15 @@|| (op.dispatched && !op.finished && op.pointerPreparationOnly !== true && op.selectorPreparationOnly !== true && op.typingPreparationOnly !== true);op.selectorPreparationOnly = selectorPreparationOnly === true;op.typingPreparationOnly = typingPreparationOnly === true && !consequentialInputPending;+ // Only page_monitor_dispatch derives this from the actual press itself.+ op.typingFocusPress = op.typingPreparationOnly && params.kind === 'click';+ if (op.typingFocusPress) {+ op.keyboardBound = true;+ if (!params.release) op.typingFocusPressed = true;+ }+ // Any other press of the action, or a release that did not reach the+ // field, ends its keyboard binding.+ else if (params.kind === 'click' && !params.fenceOnly && params.pointerPreparationOnly !== true) op.keyboardBound = false;const movementEvents = ['pointerover', 'pointerenter', 'mouseover', 'mouseenter', 'pointermove', 'mousemove'];op.pointerPreparationOnly = !consequentialInputPending && params.pointerPreparationOnly === true&& !params.fenceOnly && !params.release && params.kind === 'click' && params.navigationCandidate === false@@ -3283,6 +3416,8 @@'hover', 'drag_drop', 'patch_element', 'revert_patch', 'highlight_element', 'execute_js','ax_prepare_field_for_trusted_type', 'ax_resolve_two_rects', 'ax_resolve_rect']);function beginContentAction(action, params = {}) {+ // The steps of a typing action that follow its own focus press.+ const afterFocusPress = ['ax_prepare_field_for_trusted_type', 'ax_paste_draft'].includes(action);// Internal DraftJS paste is the same ref-bound input operation as type_ax.// Keep its focus and synchronous editor changes inside the normal input owner.if (action === 'ax_paste_draft') action = 'type_ax';@@ -3294,12 +3429,25 @@|| (kindFor(action) === 'input' ? deepActiveElement() : null);const bindingTool = action === 'ax_prepare_field_for_trusted_type' && params.expectedModelSnapshot&& currentRelaxedGrant(params.relaxedMonitorGrantToken) ? 'set_field' : action;- const approved = [...operations.values()].reverse().find(op => op.allowPassiveRebase === true && !op.dispatched- && op.preparedBinding?.target === target && op.until >= Date.now()- && lastUserAt <= op.userAt- && (op.preparedBinding.relaxed ? currentRelaxedGrant(op.preparedBinding.grantToken) && operationBindingMatches(op, target)- : !authorityChanged(op) && (op.preparedBinding.model === true- ? !modelDecisionPredicate(op.preparedBinding, operationBinding(op, target)) : operationBindingMatches(op, target))));+ // A typing action's own focus press leaves its certificate with the+ // steps that follow it on that field: reselecting it for the text+ // command, or DraftJS's paste. They are keyboard-bound (see+ // keyboardBoundField), so the node on top at its centre does not matter.+ const approved = [...operations.values()].reverse().find(op => {+ const focusPressed = op.dispatched && afterFocusPress && ownedTypingFocus(op);+ if (op.allowPassiveRebase !== true || (op.dispatched && !focusPressed) || op.preparedBinding?.target !== target+ || !(op.until >= Date.now()) || lastUserAt > op.userAt) return false;+ if (op.preparedBinding.relaxed) return currentRelaxedGrant(op.preparedBinding.grantToken) && operationBindingMatches(op, target);+ if (authorityChanged(op)) return false;+ if (op.preparedBinding.model !== true) return operationBindingMatches(op, target);+ // A keyboard-bound field's input steps are compared without its centre+ // hit (see keyboardBoundField); anything else as a model decision.+ if (kindFor(action) === 'input' && keyboardBoundField(op)) {+ return !keyboardDecisionPredicate(op.preparedBinding, operationBinding(op, target));+ }+ if (focusPressed) return false;+ return !modelDecisionPredicate(op.preparedBinding, actionFootprint(target, op.preparedBinding.readonlyPreparation));+ });prepare({ selector: params.selector, ref_id: params.ref_id, x: params.x, y: params.y,textMatch: action === 'click' ? params.text : undefined, tool: bindingTool, operationId, runToken,...(params.expectedModelSnapshot !== undefined || approved?.expectedModelSnapshot !== undefined@@ -3318,6 +3466,8 @@local.focusBound = approved.focusBound;local.modelBindingInvalid = false;local.allowPassiveRebase = true;+ local.keyboardBound = approved.keyboardBound === true && kindFor(action) === 'input'+ && (approved.dispatched !== true || ownedTypingFocus(approved));}const installed = localOperation = { operationId, kind: kindFor(action), userAt: lastUserAt, revision,navigationCandidate: ['click', 'click_ax', 'set_checked'].includes(action),@@ -3395,19 +3545,23 @@try {const result = callback();flushPendingMutations(op); sampleFormControls(target);- const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken) : actionFootprint(target, before.readonlyPreparation);+ const keyboard = !before.relaxed && keyboardBoundField(op);+ const after = before.relaxed ? relaxedTargetBinding(target, before.grantToken)+ : actionFootprint(target, before.readonlyPreparation, { keyboard });const changedAuthority = !active || runToken !== preparedRun || lastUserAt !== preparedUserAt|| (before.relaxed ? !currentRelaxedGrant(before.grantToken) : interventionRevision !== preparedIntervention || interventionUnreported);const changedFocus = kind === 'focus' && deepActiveElement() !== target;- if (changedAuthority || changedFocus || !samePreparationSemantics(before, after)) {+ const sameSemantics = samePreparationSemantics(before, after) || (keyboard && sameKeyboardBinding(before, after, true));+ if (changedAuthority || changedFocus || !sameSemantics) {const error = new Error('Browser changed during action preparation. Re-observe before acting.');error.code = 'page_feedback_pending'; error.dispatched = localOperation?.started === true;- error.pagePreparationVeto = { predicate: changedAuthority ? 'intervention_changed' : changedFocus ? 'focus_changed' : bindingPredicate(before, after) || 'semantic_binding_changed' };+ error.pagePreparationVeto = { predicate: changedAuthority ? 'intervention_changed' : changedFocus ? 'focus_changed'+ : bindingPredicate(before, keyboard && after ? { ...after, hit: before.hit } : after) || 'semantic_binding_changed' };throw error;}// Keep the observation's exact semantic/private values immutable. Only a// derived geometry proof for this original node is shared with its owner.- ownedPreparationBindings.set(op.preparedBinding, { binding: after, runToken, documentToken,+ ownedPreparationBindings.set(op.preparedBinding, { binding: keyboard ? { ...after, hit: before.hit } : after, runToken, documentToken,userAt: lastUserAt, interventionRevision,...(before.relaxed ? { viewport: preparationViewport } : {}),scrollPreparationUntil: kind === 'scroll' && (previousScrollX !== window.scrollX || previousScrollY !== window.scrollY)@@ -3530,13 +3684,35 @@void lastFeedbackDelivery.then(() => respond({ ready: false, pageFeedbackPending: true, preparationVeto }));return true;}- dispatch(params); publishRevision();- respond({ ready: true, ...(active && params.runToken === runToken ? { guard: {+ // A typing action's own focus press (and its release) is preparation;+ // the reply says so, for diagnostics.+ const typingPreparation = typingFocusPhase(prepared, params);+ dispatch(params, false, typingPreparation); publishRevision();+ respond({ ready: true, ...(typingPreparation ? { typingPreparation: true } : {}), ...(active && params.runToken === runToken ? { guard: {documentToken, revision, operationId: params.operationId, runToken,navigationCandidate: params.navigationCandidate !== false,...(operations.get(params.operationId)?.nativeSecret ? { nativeSecret: operations.get(params.operationId).nativeSecret } : {}),} } : {}) });}+ else if (msg.action === 'page_monitor_receiver_frame') {+ // Keyboard input of this document's certified operation is about to+ // reach a child document. That is its input only when the certified+ // target is the frame that has focus here (or contains it), was not an+ // editable host when certified (an editor's text never goes into a+ // frame embedded in it), and its certificate still holds. An unknown+ // operation cannot be shown to allow it.+ const params = msg.params || {};+ const current = active && params.runToken === runToken;+ const op = current ? operations.get(params.operationId) : null;+ const binding = op?.preparedBinding, target = binding?.target, frame = deepActiveElement();+ const before = preparedBindingFor(binding), after = target instanceof Element ? operationBinding(op, target) : null;+ const allowed = current && binding?.model === true && binding.editable !== true && target instanceof Element+ && target.isConnected && frame instanceof Element && /^(IFRAME|FRAME)$/.test(frame.tagName)+ && (target === frame || target.contains(frame))+ // Only its geometry may have moved, as for any certified decision.+ && (before?.relaxed ? sameRelaxedBinding(before, after) : samePreparationSemantics(before, after));+ respond({ ready: current, foreignReceiverAllowed: allowed === true });+ }else if (msg.action === 'page_monitor_finish') {const params = msg.params || {};const op = operations.get(params.operationId);diff --git a/src/firefox/src/trace/privacy.js b/src/firefox/src/trace/privacy.jsindex b213e2b98..9b61aacbb 100644--- a/src/firefox/src/trace/privacy.js+++ b/src/firefox/src/trace/privacy.js@@ -77,7 +77,7 @@ const RECOVERY_STAGES = new Set([const RECOVERY_RETRY_MODES = new Set(['resume', 'publication_without_audit', 'publication_audit_bypass', 'action_monitoring_relaxed']);const CERTIFICATION_FAILURES = new Set(['owner_unresolved', 'identity_unobserved', 'capture_limit', 'identity_limit','target_uncovered', 'context_uncertified']);-const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_uncovered', 'focus_changed',+const ACTION_PREDICATES = new Set(['target_changed', 'target_unresolved', 'target_ambiguous', 'target_uncovered', 'focus_changed','intervention_changed', 'model_snapshot_invalid', 'model_snapshot_expired', 'pending_feedback','target_hidden', 'target_disabled', 'target_detached', 'parent_changed', 'form_changed','relaxed_grant_expired', 'relaxed_target_changed', 'target_not_actionable']);diff --git a/test/keyboard-receiver-frame.mjs b/test/keyboard-receiver-frame.mjsnew file mode 100644index 000000000..fc3f8e7e2--- /dev/null+++ b/test/keyboard-receiver-frame.mjs@@ -0,0 +1,104 @@+import assert from 'node:assert/strict';+import { test } from 'node:test';++// Chrome attributes native keyboard input to the document whose node has+// focus. If a page moves focus into an iframe while WebBrain types into a+// certified text field of the top document, the text command must not be+// sent into that iframe: the top document is asked whether its operation can+// take input in another document, and a text field never can. Before this,+// the child document's monitor, which knows nothing of the operation, let it+// through.+const PAGE_URL = 'https://x.com/compose/post';+const area = { get: async () => ({}), set: async () => {}, remove: async () => {} };+let replies = null;+const messages = [];+const api = {+ storage: { local: area, session: area },+ runtime: { getURL: value => `extension://test/${value}`, sendMessage: async () => ({}) },+ tabs: {+ get: async id => ({ id, url: PAGE_URL, title: 'Compose post' }),+ sendMessage: async (_id, message, options) => {+ messages.push({ action: message.action, frameId: options?.frameId, params: message.params });+ if (message.action === 'page_monitor_receiver_frame') {+ if (replies.receiver instanceof Error) throw replies.receiver;+ return replies.receiver;+ }+ if (message.action === 'page_monitor_dispatch') return { ready: true, guard: { operationId: message.params.operationId } };+ return { ready: true };+ },+ },+ webNavigation: { getAllFrames: async () => [+ { frameId: 0, parentFrameId: -1, documentId: 'document-top', url: PAGE_URL },+ { frameId: 7, parentFrameId: 0, documentId: 'document-child', url: 'about:srcdoc' },+ ] },+ scripting: { executeScript: async () => [{ result: null }] },+};+globalThis.chrome = api;+globalThis.browser = api;+let nextTab = 8300;++for (const build of ['chrome', 'firefox']) {+ const { Agent } = await import(`../src/${build}/src/agent/agent.js`);+ const { beforePageAgentDispatch } = await import(`../src/${build}/src/agent/page-feedback.js`);++ async function dispatch({ receiver, details, call = { name: 'type_ax', args: { ref_id: 'ref_editor', text: 'x' }, expectedModelSnapshot: 'snapshot-1' } }) {+ const tab = nextTab++;+ const agent = new Agent({ getActive: () => null, getProvider: () => null, getVisionProvider: async () => null });+ agent._runningTabs.add(tab);+ await agent._beginPageFeedbackRun(tab, 'act');+ try {+ for (const [frameId, documentId, url, token] of [[0, 'document-top', PAGE_URL, 'top-doc'], [7, 'document-child', 'about:srcdoc', 'child-doc']]) {+ assert.equal(agent.pageMonitorState({ tab: { id: tab }, frameId, documentId, url }, token).active, true);+ }+ // Only the receiver rule is under test here.+ agent._pageFeedbackBlocksCall = () => false;+ const run = agent._pageFeedbackRuns.get(tab);+ run.dispatchCall = { ...call, allowPassiveRebase: true, dispatchState: { started: true } };+ replies = { receiver };+ messages.length = 0;+ let error = null;+ try { await beforePageAgentDispatch(api, tab, { operationId: 'op-typing', ...details }); } catch (caught) { error = caught; }+ return { error, sent: messages.filter(message => message.action !== 'page_monitor_state') };+ } finally {+ agent._finishPageFeedbackRun(tab);+ agent._runningTabs.delete(tab);+ }+ }+ const text = { kind: 'input', nativeInput: true, nativeTarget: 'marker', documentRevision: 1, eventTypes: ['beforeinput', 'input', 'change'] };++ test(`${build}: certified text input whose receiver has focus in a child document is not sent there`, async () => {+ const { error, sent } = await dispatch({ receiver: { ready: true, foreignReceiverAllowed: false },+ details: { ...text, documentToken: 'child-doc' } });+ assert.equal(error?.code, 'page_feedback_pending', String(error));+ assert.equal(error.pagePreparationVeto?.cause, 'receiver_frame_changed');+ assert.deepEqual(sent.map(item => [item.action, item.frameId]), [['page_monitor_receiver_frame', 0]]);+ assert.equal(sent[0].params.operationId, 'op-typing');+ });++ test(`${build}: without an answer from the top document such input is not sent either`, async () => {+ const { error, sent } = await dispatch({ receiver: new Error('Receiving end does not exist.'),+ details: { ...text, documentToken: 'child-doc' } });+ assert.equal(error?.pagePreparationVeto?.cause, 'receiver_frame_changed', String(error));+ assert.equal(sent.some(item => item.action === 'page_monitor_dispatch'), false);+ });++ test(`${build}: an operation that allows a child-document receiver still reaches that document`, async () => {+ const { error, sent } = await dispatch({ receiver: { ready: true, foreignReceiverAllowed: true },+ details: { ...text, documentToken: 'child-doc' } });+ assert.equal(error, null, String(error));+ assert.deepEqual(sent.map(item => [item.action, item.frameId]), [['page_monitor_receiver_frame', 0], ['page_monitor_dispatch', 7]]);+ });++ test(`${build}: top-document receivers, releases and uncertified calls are not asked`, async () => {+ for (const [details, call] of [+ [{ ...text, documentToken: 'top-doc' }, undefined],+ [{ ...text, documentToken: 'child-doc', release: true }, undefined],+ [{ ...text, documentToken: 'child-doc' }, { name: 'iframe_type', args: { selector: 'input', text: 'x' } }],+ ]) {+ const { error, sent } = await dispatch({ receiver: { ready: true, foreignReceiverAllowed: false }, details, ...(call ? { call } : {}) });+ assert.equal(error, null, String(error));+ assert.equal(sent.some(item => item.action === 'page_monitor_receiver_frame'), false, JSON.stringify(sent));+ assert.equal(sent.filter(item => item.action === 'page_monitor_dispatch').length, 1);+ }+ });+}diff --git a/test/page-feedback-unbindable-target.mjs b/test/page-feedback-unbindable-target.mjsnew file mode 100644index 000000000..0ee82282b--- /dev/null+++ b/test/page-feedback-unbindable-target.mjs@@ -0,0 +1,238 @@+import assert from 'node:assert/strict';+import { withFixtureCalculationReadings } from './calculation-fixture-readings.mjs';+import { test } from 'node:test';++// Trace 61282dbf: on x.com/compose/post the model chose+// type_text({selector: '[data-testid="tweetTextarea_0"]'}). X uses that test+// id for the composer AND the hidden timeline composer behind it, so the+// action can never be bound. Because the timeline kept updating, each such+// decision was discarded as stale before it could answer, the model saw the+// same view and decided the same again, and after five rounds the run ended+// with "the page kept changing". The call must answer for itself instead.+const PAGE_URL = 'https://x.com/compose/post';+const SELECTOR = '[data-testid="tweetTextarea_0"]';+const POST = 'Owned QA support is coming soon https://example.com/qa/pull/3195';+const TREE = 'dialog [ref_dialog]\n textbox "Post text" [ref_editor]\n button "Post" [ref_post] type="button"\n'+ + 'textbox "Post text" [ref_hidden_editor] hidden=true occluded=true';+const documents = new Map();+const area = { get: async () => ({}), set: async () => {}, remove: async () => {} };+const api = {+ storage: { local: area, session: area },+ runtime: { getURL: value => `extension://test/${value}`, sendMessage: async () => ({}) },+ tabs: { get: async id => ({ id, url: PAGE_URL, title: 'Compose post' }), sendMessage: async () => ({ ready: true }) },+ webNavigation: { getAllFrames: async ({ tabId }) => [{ frameId: 0, parentFrameId: -1, documentId: documents.get(tabId) || 'document-1', url: PAGE_URL }] },+ scripting: { executeScript: async () => [{ result: null }] },+};+globalThis.chrome = api;+globalThis.browser = api;+let nextTab = 7300;++function makeAgent(Agent, next) {+ const provider = {+ name: 'unbindable target test', model: 'test', promptTier: 'full', contextWindow: 128000,+ supportsTools: true, supportsVision: false,+ chat: withFixtureCalculationReadings(messages => messages[0]?.content?.startsWith('You classify whether one proposed terminal WebBrain Ask answer')+ ? Promise.resolve({ content: '{"pageIndependent":false}' }) : next(messages)),+ async *chatStream(messages) {+ const response = await next(messages);+ if (response.content) yield { type: 'text', content: response.content };+ if (response.toolCalls) yield { type: 'tool_call', content: response.toolCalls.map((call, index) => ({ ...call, index })) };+ yield { type: 'done' };+ },+ };+ const agent = new Agent({ getActive: () => provider, getProvider: () => provider, getVisionProvider: async () => null });+ agent._hydrate = async () => {};+ agent._persist = () => {};+ agent._persistNow = async () => ({ ok: true });+ agent._persistSubmittedTurn = async () => {};+ agent._startTraceRun = async () => null;+ agent._endTraceRun = async () => {};+ agent._beginReadCompleteness = async () => null;+ agent._manageContext = async () => {};+ agent._checkCostAllowance = async () => null;+ agent._recordCostUsage = async () => null;+ agent._currentUrl = async () => PAGE_URL;+ agent._getTabUrlTitle = async () => ({ tabUrl: PAGE_URL, tabTitle: 'Compose post' });+ agent._maybeEmitAskModeHandoff = async () => {};+ agent._maybeReinjectAdapter = async () => {};+ agent._ensureProgressSessionForCurrentTask = async () => {};+ agent._plannerMode = () => 'off';+ agent._runPlannerIntentGate = async () => ({+ proceed: true, requestKind: 'execute', requiresStateChange: false, requiresSubmission: false,+ });+ agent._skipPermissionGate = true;+ agent._ensureGateSetting = async () => true;+ for (const name of ['_chromeProtectedPageFailure', '_captchaMutationPreflight', '_workflowPreSubmitDispatchBlock',+ '_messageRecipientGuardBlock', '_detectLikelySubmitAction', '_socialPublicationPreSubmitBlock']) agent[name] = async () => null;+ agent._adoptLiveSocialPublishWorkflow = async () => false;+ agent._isFormValidationCandidate = () => false;+ agent._preflightRichTextToolbarTarget = async () => ({ block: null });+ agent._auditRichTextToolbarTarget = async () => {};+ agent._shouldAutoScreenshot = () => false;+ agent._pageFeedbackIdleMs = 0;+ agent._completionDoneBlock = () => null;+ agent.maxSteps = Infinity;+ return agent;+}++function bind(agent, tab) {+ const from = { tab: { id: tab }, frameId: 0, documentId: 'document-1', url: PAGE_URL };+ const state = agent.pageMonitorState(from, 'content-1');+ assert.equal(state.active, true);+ let seq = 0;+ return value => {+ const current = ++seq;+ assert.equal(agent.observePageFeedback(from, {+ ...state, seq: current, revision: current, kind: 'dom', source: 'page', target: 'div#timeline', ...value,+ }).accepted, true);+ };+}++const calls = (...list) => ({ toolCalls: list.map(([id, name, args]) => ({ id, function: { name, arguments: JSON.stringify(args) } })) });+const toolResults = messages => messages.filter(message => message.role === 'tool');+// Tool results may carry appended notes or an untrusted-data envelope.+function parse(message) {+ const text = typeof message?.content === 'string' ? message.content : '';+ for (let start = text.indexOf('{'); start >= 0; start = text.indexOf('{', start + 1)) {+ let depth = 0, quoted = false, escaped = false;+ for (let cursor = start; cursor < text.length; cursor++) {+ const char = text[cursor];+ if (quoted) { if (escaped) escaped = false; else if (char === '\\') escaped = true; else if (char === '"') quoted = false; }+ else if (char === '"') quoted = true;+ else if (char === '{') depth++;+ else if (char === '}' && --depth === 0) {+ try { return JSON.parse(text.slice(start, cursor + 1)); } catch { break; }+ }+ }+ }+ return null;+}++for (const build of ['chrome', 'firefox']) {+ const { Agent } = await import(`../src/${build}/src/agent/agent.js`);+ const pageFeedbackExecute = Agent.prototype.executeTool;+ for (const streaming of [false, true]) for (const reason of ['target_ambiguous', 'target_unresolved']) {+ test(`${build}: ${streaming ? 'stream' : 'chat'} an unbindable ${reason} target under live page updates answers for itself`, async () => {+ const tab = nextTab++, updates = [], modelMessages = [], validations = [];+ const oldSend = api.tabs.sendMessage;+ let agent, send, requests = 0, captures = 0, typed = 0, clicked = 0, terminalStatus;+ api.tabs.sendMessage = async (_id, message) => {+ if (message.action === 'page_monitor_capture_model') {+ captures++;+ return { ready: true, runToken: message.params.runToken, documentToken: 'content-1',+ snapshotToken: 'owned-' + captures, targetCount: 3, focusedTargetAvailable: false,+ page: { success: true, url: PAGE_URL, pageContent: TREE } };+ }+ if (message.action === 'page_monitor_validate_model') {+ validations.push(message.params);+ const unbindable = message.params.selector === SELECTOR;+ return { ready: !unbindable, ...(unbindable ? { reason } : {}), runToken: message.params.runToken,+ documentToken: 'content-1', snapshotToken: message.params.snapshotToken };+ }+ if (message.action === 'page_monitor_prepare') return { ready: true, modelBindingValid: true };+ return { ready: true };+ };+ const next = async messages => {+ modelMessages.push(structuredClone(messages)); requests++;+ assert.ok(requests <= 4, 'An unbindable target must not loop on page updates.');+ // X's timeline updates during every inference.+ send({});+ const answered = toolResults(messages).map(parse).find(result => result?.errorCode === 'action_binding_unavailable');+ if (!answered) return calls(['by-selector-' + requests, 'type_text', { selector: SELECTOR, text: POST, clear: true }],+ ['post-' + requests, 'click_ax', { ref_id: 'ref_post' }]);+ if (!typed) return calls(['by-ref', 'type_ax', { ref_id: 'ref_editor', text: POST, clear: true }]);+ return calls(['finished', 'done', { summary: 'Typed the post into the composer.', outcome: 'success' }]);+ };+ agent = makeAgent(Agent, next);+ agent._endTraceRun = async (_tab, _run, status) => { terminalStatus = status; };+ agent._enrichUserMessageWithCurrentPage = async (_tab, _messages, content) => { send = bind(agent, tab); return { role: 'user', content }; };+ agent.executeTool = async function(id, name, args, onUpdate, context) {+ if (name === 'get_accessibility_tree') return { success: true, pageContent: TREE };+ if (name === 'done') return { done: true, success: true, summary: args.summary, outcome: args.outcome };+ // The production page-feedback gate answers before any dispatch.+ if (name === 'type_text') return pageFeedbackExecute.call(this, id, name, args, onUpdate, context);+ if (name === 'click_ax') { clicked++; return { success: true, dispatched: true, verified: true }; }+ assert.equal(name, 'type_ax'); typed++;+ return { success: true, dispatched: true, verified: true };+ };+ try {+ const run = streaming+ ? agent.processMessageStream(tab, 'Type the post into the X composer.', (type, data) => updates.push({ type, data }), 'act',+ { detachedRequestId: 'unbindable-' + tab, askStreamingEnabled: false })+ : agent.processMessage(tab, 'Type the post into the X composer.', (type, data) => updates.push({ type, data }), 'act', [],+ { detachedRequestId: 'unbindable-' + tab, askStreamingEnabled: false });+ const result = await run;+ assert.equal(requests, 3, JSON.stringify({ result, terminalStatus }));+ assert.doesNotMatch(String(result), /page kept changing/);+ assert.notEqual(terminalStatus, 'page_unstable');+ assert.equal(updates.some(update => update.type === 'run_status' && update.data.status === 'page_unstable'), false);+ assert.equal(updates.some(update => update.type === 'failure_recovery'), false);+ const answers = toolResults(modelMessages[1]).map(parse);+ const answer = answers.find(item => item?.errorCode === 'action_binding_unavailable');+ assert.ok(answer, JSON.stringify(answers));+ assert.equal(answer.noDispatch, true); assert.equal(answer.dispatched, false);+ if (reason === 'target_ambiguous') {+ assert.equal(answer.bindingFailure, 'target_ambiguous');+ assert.match(answer.error, /matches more than one element/);+ assert.match(answer.error, /ref_id/);+ } else assert.equal(answer.bindingFailure, undefined);+ assert.ok(answers.some(item => item?.skipped === true && item.skippedBecause === 'fresh_turn_required'),+ 'The Post click planned with the unbindable write must not run.');+ assert.equal(clicked, 0);+ assert.equal(typed, 1);+ assert.ok(validations.some(params => params.selector === SELECTOR));+ assert.equal(agent._pageFeedbackRuns.size, 0);+ } finally { api.tabs.sendMessage = oldSend; documents.delete(tab); }+ });+ }++ for (const streaming of [false, true]) {+ test(`${build}: ${streaming ? 'stream' : 'chat'} user input during inference still supersedes an unbindable target`, async () => {+ const tab = nextTab++, updates = [], modelMessages = [];+ const oldSend = api.tabs.sendMessage;+ let agent, send, requests = 0, captures = 0;+ api.tabs.sendMessage = async (_id, message) => {+ if (message.action === 'page_monitor_capture_model') {+ captures++;+ return { ready: true, runToken: message.params.runToken, documentToken: 'content-1',+ snapshotToken: 'owned-' + captures, targetCount: 3, focusedTargetAvailable: false,+ page: { success: true, url: PAGE_URL, pageContent: TREE } };+ }+ if (message.action === 'page_monitor_validate_model') {+ const unbindable = message.params.selector === SELECTOR;+ return { ready: !unbindable, ...(unbindable ? { reason: 'target_ambiguous' } : {}), runToken: message.params.runToken,+ documentToken: 'content-1', snapshotToken: message.params.snapshotToken };+ }+ return { ready: true };+ };+ const next = async messages => {+ modelMessages.push(structuredClone(messages)); requests++;+ if (requests === 1) {+ send({ kind: 'click', source: 'user', target: 'button' });+ return calls(['by-selector', 'type_text', { selector: SELECTOR, text: POST, clear: true }]);+ }+ return calls(['finished', 'done', { summary: 'Stopped after the user acted.', outcome: 'partial' }]);+ };+ agent = makeAgent(Agent, next);+ agent._enrichUserMessageWithCurrentPage = async (_tab, _messages, content) => { send = bind(agent, tab); return { role: 'user', content }; };+ let executedTypeText = 0;+ agent.executeTool = async function(id, name, args, onUpdate, context) {+ if (name === 'get_accessibility_tree') return { success: true, pageContent: TREE };+ if (name === 'done') return { done: true, success: true, summary: args.summary, outcome: args.outcome };+ if (name === 'type_text') { executedTypeText++; return pageFeedbackExecute.call(this, id, name, args, onUpdate, context); }+ assert.fail(`Unexpected ${name}`);+ };+ try {+ const run = streaming+ ? agent.processMessageStream(tab, 'Type the post into the X composer.', (type, data) => updates.push({ type, data }), 'act',+ { detachedRequestId: 'unbindable-user-' + tab, askStreamingEnabled: false })+ : agent.processMessage(tab, 'Type the post into the X composer.', (type, data) => updates.push({ type, data }), 'act', [],+ { detachedRequestId: 'unbindable-user-' + tab, askStreamingEnabled: false });+ await run;+ assert.equal(executedTypeText, 0, 'A decision made before the user acted is discarded, not answered.');+ assert.equal(toolResults(modelMessages[1]).some(message => /action_binding_unavailable/.test(message.content)), false);+ assert.equal(requests, 2);+ } finally { api.tabs.sendMessage = oldSend; documents.delete(tab); }+ });+ }+}diff --git a/test/page-monitor-ui.mjs b/test/page-monitor-ui.mjsindex 31c0f1e7e..76a49f3be 100644--- a/test/page-monitor-ui.mjs+++ b/test/page-monitor-ui.mjs@@ -143,7 +143,7 @@ test('Chrome AX preparation attributes actual scrolling without claiming field itest('Chrome controlled-field trusted typing survives its focus-created formatting toolbar', async () => {const savedChrome = globalThis.chrome, savedBrowser = globalThis.browser, tab = 919;const savedCdpAttach = cdpClient.attach, savedCdpSend = cdpClient.sendCommand;- let changeToolbarAfterFocus = false;+ let changeToolbarAfterFocus = false, changeComposerAfterFocus = false;class FieldHost {static STATE_CHANGE_TOOLS = new Set(['set_field']);isRunning() { return true; }@@ -197,6 +197,11 @@ test('Chrome controlled-field trusted typing survives its focus-created formattiawait page.waitForTimeout(30);await page.evaluate(() => document.getElementById('formatting-toolbar').setAttribute('aria-label', 'Text formatting open'));}+ if (changeComposerAfterFocus && method === 'Input.dispatchMouseEvent' && params.type === 'mouseReleased') {+ changeComposerAfterFocus = false;+ await page.waitForTimeout(30);+ await page.evaluate(() => document.getElementById('ref_compose_dialog').setAttribute('aria-label', 'Reply to another account'));+ }return result;})().then(callback, error => {api.runtime.lastError = { message: error.message }; callback(); delete api.runtime.lastError;@@ -270,34 +275,51 @@ test('Chrome controlled-field trusted typing survives its focus-created formattiassert.equal(await page.locator('#ref_trusted_editor').innerText(), 'A verified post');assert.equal(commands.filter(item => item.method === 'Input.insertText').length, 1);+ // The page's own reaction outside the composer (its formatting toolbar)+ // after WebBrain's own focus click is not a reason to stop typing.await page.waitForTimeout(250);host._pageFeedbackRuns.get(tab).events.clear();await host._capturePageFeedbackModelState(tab, []);commands.length = 0; preparations.length = 0; changeToolbarAfterFocus = true;- const interrupted = await host.executeTool(tab, 'set_field',+ const toolbarChanged = await host.executeTool(tab, 'set_field',{ ref_id: editorRef, text: 'The next verified post', clear: true });+ assert.equal(toolbarChanged.success, true, JSON.stringify({ toolbarChanged, preparations }));+ assert.equal(await page.locator('#formatting-toolbar').getAttribute('aria-label'), 'Text formatting open');+ assert.deepEqual(preparations.map(item => item.success), [true, true]);+ assert.equal(commands.filter(item => item.method === 'Input.insertText').length, 1);+ assert.equal(await page.locator('#ref_trusted_editor').innerText(), 'The next verified post');++ // A change to the composer itself between the focus click and the+ // selection still stops text dispatch.+ await page.waitForTimeout(250);+ host._pageFeedbackRuns.get(tab).events.clear();+ await host._capturePageFeedbackModelState(tab, []);+ commands.length = 0; preparations.length = 0; changeComposerAfterFocus = true;+ const interrupted = await host.executeTool(tab, 'set_field',+ { ref_id: editorRef, text: 'The final verified post', clear: true });assert.equal(interrupted.success, false);assert.match(interrupted.error, /could not reselect it.*Browser changed during action preparation/);assert.equal(preparations.length, 2);assert.equal(preparations[0].success, true);assert.equal(preparations[1].success, false);assert.equal(commands.some(item => item.method === 'Input.insertText'), false,- 'A toolbar change between trusted focus and selection must stop text dispatch');+ 'A composer change between trusted focus and selection must stop text dispatch');assert.equal(interrupted.dispatched, true, 'The trusted focus click remains a real dispatched interaction');assert.equal(interrupted.textUnchanged, true, 'Before/after real SHA-256 readback proves the editor unchanged');assert.equal(interrupted.mutationMayHaveOccurred, false);assert.equal(interrupted.repeatBlocked, false);assert.equal(interrupted.recoveryRequired, 'fresh_tree');assert.equal(host._uncertainTextMutations?.get(tab)?.size || 0, 0);- assert.equal(await page.locator('#ref_trusted_editor').innerText(), 'A verified post');+ assert.equal(await page.locator('#ref_trusted_editor').innerText(), 'The next verified post');await page.waitForTimeout(250);host._pageFeedbackRuns.get(tab).events.clear();await host._capturePageFeedbackModelState(tab, []);+ commands.length = 0;const retried = await host.executeTool(tab, 'set_field',- { ref_id: editorRef, text: 'The next verified post', clear: true });+ { ref_id: editorRef, text: 'The final verified post', clear: true });assert.equal(retried.success, true, JSON.stringify(retried));- assert.equal(await page.locator('#ref_trusted_editor').innerText(), 'The next verified post');+ assert.equal(await page.locator('#ref_trusted_editor').innerText(), 'The final verified post');assert.equal(commands.filter(item => item.method === 'Input.insertText').length, 1);} finally {cdpClient.attach = savedCdpAttach; cdpClient.sendCommand = savedCdpSend;diff --git a/test/quality/x-compose-dialog-mv3.mjs b/test/quality/x-compose-dialog-mv3.mjsnew file mode 100644index 000000000..0849ed8d1--- /dev/null+++ b/test/quality/x-compose-dialog-mv3.mjs@@ -0,0 +1,493 @@+import assert from 'node:assert/strict';+import fs from 'node:fs/promises';+import http from 'node:http';+import { createRequire } from 'node:module';+import os from 'node:os';+import path from 'node:path';+import { chromium } from 'playwright';+import { fixturePageTextReply } from '../semantic-decision-stub.mjs';++// Trace 61282dbf (WebBrain 40.7.1) on x.com/compose/post, through the actual+// installed MV3 background, page monitor, Agent and CDP input on React/DraftJS+// document nodes. Only inference and owned HTTP responses are deterministic;+// no real X page, account or posting service is contacted.+// type_ax_after_bubble: WebBrain's own focus click makes the page draw its+// bold/italic bubble in a body-level layer while the timeline behind the+// dialog keeps updating. The write must still go through, once.+// ambiguous_selector: [data-testid="tweetTextarea_0"] names the dialog's+// composer and the hidden timeline composer. The model must be told so,+// instead of every decision being discarded until "the page kept changing".+// bubble_on_mousedown: the page draws the bubble already on mousedown, so+// WebBrain's own release lands on it. That release is the action's input+// to the page, not its preparation: the typing may be refused once, and+// the next attempt types into the focused editor without pressing it.+// iframe_focus_steal: the page moves focus from the editor into an iframe's+// input each time the editor gets it. WebBrain's text must never be typed+// into that iframe; nothing is posted.+// As in the trace's screenshot, the bubble is drawn over the point that was+// clicked, which for WebBrain's focus click is the middle of the editor.+const out = path.resolve(process.env.WEBBRAIN_QUALITY_ARTIFACTS+ || 'ci/artifacts/quality-audit-2026-10-16/x-compose-dialog-mv3');+await fs.mkdir(out, { recursive: true });+const require = createRequire(import.meta.url);+const library = (name, file) => fs.readFile(path.join(path.dirname(require.resolve(`${name}/package.json`)), file), 'utf8');+const libraryFiles = [+ ['react.js', 'react', 'umd/react.development.js'],+ ['react-dom.js', 'react-dom', 'umd/react-dom.development.js'],+ ['immutable.js', 'immutable', 'dist/immutable.js'],+ ['draft.js', 'draft-js', 'dist/Draft.js'],+];+const assets = new Map(await Promise.all(libraryFiles.map(async ([name, module, file]) => [name, await library(module, file)])));+const draftCss = await library('draft-js', 'dist/Draft.css');+const fixtureUrl = 'https://x.com/compose/post';+const account = 'ownedqa';+const SELECTOR = '[data-testid="tweetTextarea_0"]';+// The account name appears once in the task, so it anchors the account.+const singleLine = 'Owned QA support is coming soon https://example.com/qa/pull/3195';+const cases = [+ { name: 'type_ax_after_bubble', first: 'type_ax', body: singleLine },+ { name: 'ambiguous_selector', first: 'type_text', body: singleLine },+ // The trace's screenshot: the bubble of an earlier click still covers the+ // middle of the focused editor when the model decides.+ { name: 'retry_under_bubble', first: 'type_ax', body: singleLine, bubbleFirst: true },+ { name: 'bubble_on_mousedown', first: 'type_ax', body: singleLine, bubbleOnMouseDown: true, retryWrite: true },+ { name: 'iframe_focus_steal', first: 'type_ax', body: singleLine, stealFocus: true },+].filter(item => !process.env.WEBBRAIN_QUALITY_X_CASE || item.name === process.env.WEBBRAIN_QUALITY_X_CASE);+assert.ok(cases.length, 'Select an existing owned X compose case.');+const textOf = value => typeof value === 'string' ? value : Array.isArray(value)+ ? value.filter(part => part?.type === 'text').map(part => part.text || '').join('\n') : '';+function toolObject(content) {+ const text = textOf(content);+ for (let start = text.indexOf('{'); start >= 0; start = text.indexOf('{', start + 1)) {+ let depth = 0, quoted = false, escaped = false;+ for (let cursor = start; cursor < text.length; cursor++) {+ const char = text[cursor];+ if (quoted) { if (escaped) escaped = false; else if (char === '\\') escaped = true; else if (char === '"') quoted = false; }+ else if (char === '"') quoted = true;+ else if (char === '{') depth++;+ else if (char === '}' && --depth === 0) {+ try { return JSON.parse(text.slice(start, cursor + 1)); } catch { break; }+ }+ }+ }+ return null;+}+function latestTools(messages) {+ const names = new Map();+ for (const message of messages || []) for (const call of message.tool_calls || []) names.set(call.id, call.function?.name);+ return (messages || []).filter(message => message.role === 'tool')+ .map(message => ({ name: names.get(message.tool_call_id) || message.name, result: toolObject(message.content) }));+}+function references(tree) {+ const lines = String(tree?.pageContent || '').split('\n');+ const ref = expression => lines.find(line => expression.test(line))?.match(/\[(ref_\d+)\]/)?.[1];+ return { editor: ref(/textbox "Post text"(?!.*hidden=true)/), post: ref(/button "Post"(?!.*hidden=true)/) };+}+const intent = { request_kind: 'execute', scope_relation: 'new', requires_state_change: true,+ requires_submission: true, messaging: null, deliverables: ['One exact owned local QA post, freshly verified.'],+ allows_planner_shaped_result: false, allows_app_state_tool_evidence: false,+ completion_requirements: { download: false }, read_scope: 'visible_page',+ summary: 'Publish exactly one owned local QA post from the current ownedqa account and verify its confirmation.',+ steps: [{ id: '1', action: 'Read the compose dialog, write the exact draft once, re-observe Post, publish once and verify the owned confirmation.' }],+ memory: { use_progress_ledger: false, progress_action: null }, scheduling: null, risks: [] };+let current, context, worker;+const blockedRemote = [], submissions = [], summaries = [], auxiliaryRequests = [];+const server = http.createServer(async (request, response) => {+ try {+ if (request.url !== '/v1/chat/completions') { response.writeHead(404).end(); return; }+ let raw = ''; for await (const part of request) raw += part;+ const input = JSON.parse(raw);+ const system = textOf(input.messages?.[0]?.content);+ if (system.startsWith('Extract stable user preferences for WebBrain memory.')) {+ auxiliaryRequests.push(input);+ response.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({ id: 'owned-x-memory',+ choices: [{ message: { role: 'assistant', content: '{"memories":[]}' }, finish_reason: 'stop' }],+ usage: { prompt_tokens: 100, completion_tokens: 10 } }));+ return;+ }+ const pageTextReply = fixturePageTextReply(input.messages);+ if (pageTextReply) {+ auxiliaryRequests.push(input);+ response.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({ id: 'owned-x-page-text',+ choices: [{ message: { role: 'assistant', content: pageTextReply }, finish_reason: 'stop' }],+ usage: { prompt_tokens: 100, completion_tokens: 10 } }));+ return;+ }+ current.requests.push(input);+ assert.ok(current && JSON.stringify(input.messages).includes(current.marker), `Inference must retain this owned task authority: ${system.slice(0, 160)}`);+ let completion;+ if (system.startsWith("Compile the user's social-publication intent across languages.")) {+ completion = { role: 'assistant', content: JSON.stringify({ version: 1, status: 'ready',+ actions: [{ id: 'p1', platform: 'twitter', account: { source: 'task', start: account, end: account },+ posts: [{ body: { kind: 'exact', source: { source: 'task', start: current.body, end: current.body } },+ media: { kind: 'count', type: 'any', format: null, min: 0, max: 0 }, context: { kind: 'post', target: null } }] }],+ requirements: 'p1', prohibited: [], reason: 'The authentic task authorizes one exact local QA post.' }) };+ } else if (system.startsWith('Independently check whether the authentic user request authorizes this concrete public dispatch.')) {+ current.audits = (current.audits || 0) + 1;+ const audit = JSON.parse(textOf(input.messages.find(message => message.role === 'user')?.content));+ completion = { role: 'assistant', content: JSON.stringify({ key: audit.key, actionId: audit.action.id,+ authorized: true, reason: 'The exact owned body, account and plain-post scope match the authentic task.' }) };+ } else if (system.startsWith('Evaluate only the observable requirements of the genuine user task')) {+ completion = { role: 'assistant', content: JSON.stringify({ outcome: 'succeeded', reason: 'Fresh owned page confirmation records the exact authorized local post.' }) };+ } else if (!input.tools?.length) completion = { role: 'assistant', content: JSON.stringify(intent) };+ else {+ const step = current.steps++;+ assert.ok(step < 24, 'The owned compose run must finish without looping.');+ const tools = latestTools(input.messages);+ const last = tools.at(-1);+ const observed = tools.findLast(tool => tool.name === 'get_accessibility_tree' && tool.result?.pageContent);+ const typed = tools.find(tool => tool.name === 'type_ax' && tool.result?.success === true);+ const refused = tools.find(tool => tool.name === 'type_text' && tool.result?.errorCode === 'action_binding_unavailable');+ const clicked = tools.find(tool => tool.name === 'click_ax');+ if (typed && !current.afterWriteState) {+ // The page as the write left it, before anything else happens.+ current.afterWriteState = await current.page.evaluate(() => qaSnapshot());+ current.bubbleCoveredWrite = await current.page.evaluate(() => qaBubbleCoversEditor());+ }+ let call;+ if (!observed) call = { name: 'get_accessibility_tree', args: { filter: 'all' } };+ else if (!typed && !tools.some(tool => tool.name === 'type_ax')) {+ current.beforeWrite ||= references(observed.result);+ assert.ok(current.beforeWrite.editor && current.beforeWrite.post, JSON.stringify(observed.result).slice(0, 4000));+ // As in the trace: the selector first, then the observed ref once+ // the selector's answer says why it cannot be used.+ call = current.first === 'type_text' && !refused+ ? { name: 'type_text', args: { selector: SELECTOR, text: current.body, clear: true } }+ : { name: 'type_ax', args: { ref_id: current.beforeWrite.editor, text: current.body, clear: true } };+ } else if (!typed && current.retryWrite && tools.filter(tool => tool.name === 'type_ax').length < 2) {+ call = { name: 'type_ax', args: { ref_id: current.beforeWrite.editor, text: current.body, clear: true } };+ } else if (!typed) call = { name: 'done', args: { outcome: 'failed', summary: 'The owned compose write failed.' } };+ else if (!clicked && last?.name !== 'get_accessibility_tree') call = { name: 'get_accessibility_tree', args: { filter: 'all' } };+ else if (!clicked) {+ const fresh = references(observed.result);+ assert.ok(fresh.post, 'The enabled Post control must be observed after writing.');+ current.postReobserved = true;+ call = { name: 'click_ax', args: { ref_id: fresh.post } };+ } else if (last?.name === 'click_ax') call = { name: 'get_accessibility_tree', args: { filter: 'all' } };+ else call = { name: 'done', args: { outcome: 'success', summary: 'The owned local compose dialog post was published and verified.' } };+ current.calls.push(call);+ completion = { role: 'assistant', content: '', tool_calls: [{ id: `owned-x-${current.name}-${step}`,+ type: 'function', function: { name: call.name, arguments: JSON.stringify(call.args) } }] };+ }+ response.writeHead(200, { 'Content-Type': 'application/json' });+ response.end(JSON.stringify({ id: `owned-x-${current.name}-${current.requests.length}`, choices: [{ message: completion,+ finish_reason: completion.tool_calls ? 'tool_calls' : 'stop' }], usage: { prompt_tokens: 100, completion_tokens: 30 } }));+ } catch (error) {+ if (current) current.providerError = error.stack;+ response.writeHead(500, { 'Content-Type': 'application/json' }).end(JSON.stringify({ error: error.message }));+ }+});+await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));++const html = `<!doctype html><html><head><meta charset="utf-8"><title>Compose post</title>+<style>${draftCss}+body{margin:0;background:#fff;color:#111;font:17px system-ui}#app{display:flex;gap:24px;margin:20px}+#account-nav{width:170px}a{color:inherit}main{width:660px}+#timeline-composer{padding:16px;border:1px solid #bbb;min-height:120px}+#layers{position:fixed;inset:0;display:flex;justify-content:center;align-items:flex-start;z-index:10}+#backdrop{position:absolute;inset:0;background:rgba(0,0,0,.4)}+#compose{position:relative;margin-top:40px;width:600px;background:#fff;border-radius:16px;padding:12px}+#compose .public-DraftEditor-content{min-height:96px}+button{font:inherit;padding:8px;margin:3px}article{padding:16px;border:1px solid #ddd;min-height:60px}+#format-bubble{position:fixed;z-index:20;box-sizing:border-box;width:72px;height:34px;display:flex;gap:4px;background:#111;border-radius:8px;padding:2px}+[data-testid="tweetText"]{white-space:pre-wrap}</style>+${libraryFiles.map(([name]) => `<script src="/owned-fixture/${name}"></script>`).join('')}+</head><body><div id="app"></div><script>+window.qaEvents=[];window.qaSaved=[];window.qaFeedChanges=0;window.qaBubbles=0;window.qaBubblePresses=0;+const root=ReactDOM.createRoot(document.querySelector('#app'));+function App(){+ const [editorState,setEditorState]=React.useState(()=>Draft.EditorState.createEmpty());+ const [inlineState,setInlineState]=React.useState(()=>Draft.EditorState.createEmpty());+ const [open,setOpen]=React.useState(true);+ const [feedRevision,setFeedRevision]=React.useState(0);+ const [bubble,setBubble]=React.useState(null);+ const [saved,setSaved]=React.useState(null);+ window.qaEditorState=editorState;+ window.qaChurnFeed=()=>ReactDOM.flushSync(()=>{window.qaFeedChanges++;setFeedRevision(value=>value+1);});+ window.qaShowBubble=point=>ReactDOM.flushSync(()=>{window.qaBubbles++;setBubble({left:point.x-36,top:point.y-17});});+ const onChange=next=>{window.qaEditorState=next;setEditorState(next);};+ const value=editorState.getCurrentContent().getPlainText('\\n');+ const post=async()=>{+ const payload={account:'${account}',body:editorState.getCurrentContent().getPlainText('\\n'),audience:'Everyone can reply'};+ const response=await fetch('/owned-fixture/post',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)});+ if(response.ok){window.qaSaved.push(payload);setSaved(payload);setOpen(false);setBubble(null);setEditorState(Draft.EditorState.createEmpty());}+ };+ return React.createElement(React.Fragment,null,+ React.createElement('nav',{id:'account-nav','aria-label':'Account navigation'},+ React.createElement('a',{'data-testid':'AppTabBar_Profile_Link',href:'/${account}'},'Owned QA profile')),+ React.createElement('main',{'aria-label':'Home timeline','aria-hidden':open?'true':undefined},+ React.createElement('button',{id:'new-posts',type:'button'},'New posts are available ('+feedRevision+')'),+ React.createElement('div',{id:'timeline-composer'},+ React.createElement(Draft.Editor,{editorState:inlineState,onChange:setInlineState,ariaLabel:'Post text',webDriverTestID:'tweetTextarea_0',placeholder:'What is happening?!'}),+ React.createElement('button',{type:'button','data-testid':'tweetButtonInline',disabled:true},'Post')),+ React.createElement('div',{id:'confirmation',role:'status'},saved?'Your post was sent: '+saved.body:''),+ saved?React.createElement('article',{'data-testid':'tweet'},+ React.createElement('div',{'data-testid':'User-Name'},+ React.createElement('a',{href:'/'+saved.account},'Owned QA account'),+ React.createElement('a',{href:'/'+saved.account+'/status/900'},React.createElement('time',{dateTime:new Date().toISOString()},'Now'))),+ React.createElement('div',{'data-testid':'tweetText'},saved.body)):null,+ React.createElement('div',{id:'feed'},[0,1,2].map(index=>React.createElement('article',{key:feedRevision+'-'+index},+ React.createElement('a',{href:'/feedqa/status/'+(feedRevision*10+index)},'Unrelated feed account'),+ React.createElement('p',null,'Live feed record '+feedRevision+'-'+index))))),+ open?React.createElement('div',{id:'layers'},+ React.createElement('div',{id:'backdrop'}),+ React.createElement('div',{id:'compose',role:'dialog','aria-modal':'true'},+ React.createElement('div',{role:'progressbar','aria-valuemin':0,'aria-valuemax':100,'aria-valuenow':0}),+ React.createElement('button',{type:'button','aria-label':'Close'},'x'),+ React.createElement('button',{type:'button'},'Drafts'),+ React.createElement('button',{type:'button','aria-label':'Choose audience'},'Everyone'),+ React.createElement('div',{id:'compose-editor'},+ React.createElement(Draft.Editor,{editorState,onChange,ariaLabel:'Post text',webDriverTestID:'tweetTextarea_0',placeholder:'What is happening?!'})),+ React.createElement('button',{type:'button',id:'reply-scope'},'Everyone can reply'),+ React.createElement('nav',{'aria-label':'Composer tools'},+ React.createElement('button',{type:'button','aria-label':'Add photos or video'},'Media'),+ React.createElement('button',{type:'button','aria-label':'Add emoji'},'Emoji')),+ React.createElement('button',{id:'post',type:'button','data-testid':'tweetButton',disabled:!value,onClick:post},'Post'))):null,+ bubble?ReactDOM.createPortal(React.createElement('div',{id:'format-bubble',onPointerDown:()=>{window.qaBubblePresses++;},style:{left:bubble.left+'px',top:bubble.top+'px'}},+ React.createElement('button',{type:'button','aria-label':'Bold'},'B'),+ React.createElement('button',{type:'button','aria-label':'Italic'},'I')),document.body):null);+}+ReactDOM.flushSync(()=>root.render(React.createElement(App)));+// The live timeline behind the dialog keeps updating while it is open.+window.qaFeed=setInterval(()=>{if(document.getElementById('compose'))qaChurnFeed();},80);+// A real click in the dialog's editor makes the page draw its bold/italic+// bubble in a body-level layer over the clicked point as the button is+// released (in the trace, before WebBrain reselected the editor).+document.addEventListener('mouseup',event=>{+ const editor=event.target.closest?.('#compose [data-testid="tweetTextarea_0"]');+ if(!event.isTrusted||!editor||window.qaBubbleOnMouseDown)return;+ qaShowBubble({x:event.clientX,y:event.clientY});+});+// A page that draws it already as the button goes down.+document.addEventListener('mousedown',event=>{+ const editor=event.target.closest?.('#compose [data-testid="tweetTextarea_0"]');+ if(!event.isTrusted||!editor||!window.qaBubbleOnMouseDown)return;+ qaShowBubble({x:event.clientX,y:event.clientY});+});+// A page that moves focus from the dialog's editor into an iframe's input+// each time the editor receives it (after the current script has finished).+window.qaEnableFocusSteal=()=>{+ const frame=document.createElement('iframe');frame.id='qa-frame';frame.title='GIF search';+ frame.srcdoc='<!doctype html><input id="inner" aria-label="Search GIFs">';+ frame.style.cssText='position:fixed;right:8px;bottom:8px;width:240px;height:60px;z-index:5;border:1px solid #ccc';+ document.body.append(frame);+ window.qaSteals=0;+ document.addEventListener('focusin',event=>{+ if(!event.target.closest?.('#compose [data-testid="tweetTextarea_0"]'))return;+ queueMicrotask(()=>{const inner=frame.contentDocument?.getElementById('inner');if(inner){window.qaSteals++;inner.focus();}});+ });+ return new Promise(resolve=>frame.addEventListener('load',()=>resolve(true),{once:true}));+};+window.qaFrameValue=()=>document.getElementById('qa-frame')?.contentDocument?.getElementById('inner')?.value??null;+window.qaBubbleCoversEditor=()=>{+ const editor=document.querySelector('#compose [data-testid="tweetTextarea_0"]');+ if(!editor)return false;+ const rect=editor.getBoundingClientRect();+ return !!document.elementFromPoint(rect.left+rect.width/2,rect.top+rect.height/2)?.closest('#format-bubble');+};+for(const type of ['beforeinput','input'])document.addEventListener(type,event=>{+ if(event.target.closest?.('#compose'))qaEvents.push({type,trusted:event.isTrusted,inputType:event.inputType});+},true);+window.qaSnapshot=()=>{+ const editor=document.querySelector('#compose [data-testid="tweetTextarea_0"]');+ return {model:qaEditorState.getCurrentContent().getPlainText('\\n'),text:editor?.textContent??null,+ blocks:editor?[...editor.querySelectorAll('[data-block="true"]')].map(block=>block.textContent):null,+ open:!!document.getElementById('compose'),postEnabled:!!document.querySelector('#post')&&!document.querySelector('#post').disabled,+ bubbles:qaBubbles,bubblePresses:qaBubblePresses,feedChanges:qaFeedChanges,events:qaEvents,saved:qaSaved,+ editors:document.querySelectorAll('[data-testid="tweetTextarea_0"]').length};+};+</script></body></html>`;++const profile = await fs.mkdtemp(path.join(os.tmpdir(), 'webbrain-x-compose-mv3-'));+try {+ const extension = path.join(profile, 'extension');+ await fs.cp(path.resolve('src/chrome'), extension, { recursive: true });+ // Read-only diagnostics in the disposable clone leave every production+ // permission, input, publication and completion decision intact.+ await fs.appendFile(path.join(extension, 'src/background.js'), '\nglobalThis.xComposeFixtureAgent = agent;\n');+ context = await chromium.launchPersistentContext(profile, { headless: true, channel: 'chromium', viewport: { width: 1126, height: 745 },+ args: [`--disable-extensions-except=${extension}`, `--load-extension=${extension}`,+ '--host-resolver-rules=MAP * ~NOTFOUND, EXCLUDE 127.0.0.1'] });+ await context.route('**/*', async route => {+ const request = route.request(), requested = new URL(request.url());+ if (requested.hostname === '127.0.0.1' && requested.port === String(server.address().port)) return route.continue();+ if (requested.origin === 'https://x.com') {+ if (request.isNavigationRequest() && requested.href === fixtureUrl && request.method() === 'GET')+ return route.fulfill({ contentType: 'text/html', body: html });+ if (requested.pathname.startsWith('/owned-fixture/') && assets.has(requested.pathname.split('/').at(-1)) && request.method() === 'GET')+ return route.fulfill({ contentType: 'text/javascript', body: assets.get(requested.pathname.split('/').at(-1)) });+ if (requested.pathname === '/owned-fixture/post' && request.method() === 'POST') {+ submissions.push({ case: current.name, payload: request.postDataJSON() });+ return route.fulfill({ status: 201, contentType: 'application/json', body: '{"owned":true}' });+ }+ return route.fulfill({ status: 404, body: '' });+ }+ if (/^https?:$/.test(requested.protocol)) {+ blockedRemote.push({ method: request.method(), origin: requested.origin }); return route.abort('blockedbyclient');+ }+ return route.continue();+ });+ worker = context.serviceWorkers()[0] || await context.waitForEvent('serviceworker', { timeout: 30000 });+ for (let attempt = 0; attempt < 200; attempt++) {+ const ready = await worker.evaluate(() => !!(globalThis.chrome?.tabs && globalThis.chrome?.debugger && globalThis.xComposeFixtureAgent))+ .catch(() => false);+ if (ready) break;+ await new Promise(resolve => setTimeout(resolve, 50));+ }+ await worker.evaluate(() => {+ const fetchOriginal = fetch.bind(globalThis);+ globalThis.xComposeFixtureProbe = { commands: [], blocked: [] };+ globalThis.fetch = (input, options) => {+ const requested = new URL(typeof input === 'string' ? input : input.url);+ if (requested.hostname === '127.0.0.1' || ['chrome-extension:', 'data:', 'blob:'].includes(requested.protocol))+ return fetchOriginal(input, options);+ xComposeFixtureProbe.blocked.push({ origin: requested.origin, method: options?.method || 'GET' });+ return Promise.reject(new Error('The owned X compose fixture blocks external service requests.'));+ };+ const sendCommand = chrome.debugger.sendCommand.bind(chrome.debugger);+ chrome.debugger.sendCommand = (...args) => {+ xComposeFixtureProbe.commands.push({ method: args[1], params: args[2] }); return sendCommand(...args);+ };+ });+ const harness = await context.newPage();+ await harness.goto(`chrome-extension://${new URL(worker.url()).host}/src/ui/settings.html`);+ const message = (action, data = {}) => harness.evaluate(({ action, data }) =>+ chrome.runtime.sendMessage({ target: 'background', action, ...data }), { action, data });+ await harness.evaluate(() => {+ window.xRunEvents = {};+ chrome.runtime.onMessage.addListener(event => {+ if (event.action === 'agent_update' && xRunEvents[event.requestId]) xRunEvents[event.requestId].push({ type: event.type, data: event.data });+ });+ return chrome.storage.local.set({ helpImproveWebBrain: false, tracingEnabled: false,+ autoScreenshot: false, screenshotFallback: false, planBeforeActMode: 'off', planBeforeAct: false,+ planReviewMode: 'never', maxAgentSteps: 12, systemOneEnabled: false, agentAllowLocalNetwork: true,+ askBeforeConsequentialActions: false, openaiAskStreamingEnabled: false });+ });+ assert.equal((await message('update_provider', { providerId: 'openrouter', config: { apiKey: 'owned-fixture-fake-key',+ baseUrl: `http://127.0.0.1:${server.address().port}/v1`, model: 'minimax/minimax-m3', enabled: true,+ contextWindow: 1000000, supportsAskStreaming: false } })).ok, true);+ assert.equal((await message('set_active_provider', { providerId: 'openrouter' })).ok, true);++ for (const specification of cases) {+ const page = await context.newPage(), pageErrors = [];+ page.on('pageerror', error => pageErrors.push(error.message));+ await page.goto(fixtureUrl);+ await page.waitForSelector('#compose [data-testid="tweetTextarea_0"]');+ const initial = await page.evaluate(() => qaSnapshot());+ assert.equal(initial.editors, 2, 'X names the dialog composer and the hidden timeline composer alike.');+ assert.equal(initial.open, true); assert.equal(initial.postEnabled, false);+ const tabId = await harness.evaluate(async url => (await chrome.tabs.query({})).findLast(tab => tab.url === url)?.id, fixtureUrl);+ assert.ok(tabId);+ if (specification.bubbleFirst) {+ // A click of the person's own before the task, as left in the trace's screenshot.+ const centre = await page.evaluate(() => {+ const rect = document.querySelector('#compose [data-testid="tweetTextarea_0"]').getBoundingClientRect();+ return { x: rect.left + rect.width / 2, y: rect.top + rect.height / 2 };+ });+ await page.mouse.click(centre.x, centre.y);+ await page.waitForFunction(() => qaBubbleCoversEditor());+ await page.waitForTimeout(300);+ }+ if (specification.stealFocus) {+ assert.equal(await page.evaluate(() => qaEnableFocusSteal()), true);+ await page.waitForTimeout(300);+ }+ if (specification.bubbleOnMouseDown) await page.evaluate(() => { window.qaBubbleOnMouseDown = true; });+ const body = specification.body;+ const editorRect = await page.evaluate(() => {+ const rect = document.querySelector('#compose [data-testid="tweetTextarea_0"]').getBoundingClientRect();+ return { left: rect.left, top: rect.top, right: rect.right, bottom: rect.bottom };+ });+ current = { ...specification, page, tabId, marker: `Owned X compose regression case ${specification.name}.`, requests: [], calls: [], steps: 0 };+ const probeStart = await worker.evaluate(() => xComposeFixtureProbe.commands.length);+ const task = `In this intercepted owned local QA fixture, publish exactly one post from the active ${account} account with exact text ${JSON.stringify(body)}.+Re-observe the Post button after writing, click it once, then verify the owned local confirmation. ${current.marker}`;+ const requestId = `owned-x-${specification.name}`;+ await harness.evaluate(id => { xRunEvents[id] = []; }, requestId);+ const result = await message('chat', { tabId, text: task, mode: 'act', requestId, locale: 'en' });+ const runEvents = await harness.evaluate(id => xRunEvents[id], requestId);+ const state = await page.evaluate(() => qaSnapshot());+ const commands = await worker.evaluate(start => xComposeFixtureProbe.commands.slice(start), probeStart);+ const tools = result.updates?.filter(event => event.type === 'tool_result').map(event => event.data) || [];+ const localPosts = submissions.filter(post => post.case === specification.name);+ const evidence = { specification, initial, result, runEvents, tools, state, commands: commands.filter(command => command.method?.startsWith('Input.')),+ pageErrors, localPosts, provider: { ...current, page: undefined }, blockedRemote };+ await fs.writeFile(path.join(out, `${specification.name}-evidence.json`), JSON.stringify(evidence,+ (_key, value) => typeof value === 'string' && value.startsWith('data:image/') ? '[owned fixture screenshot omitted]' : value, 2));+ assert.equal(current.providerError, undefined, current.providerError);+ assert.deepEqual(pageErrors, [], 'Real React/DraftJS reconciliation must remain valid.');+ const statuses = runEvents.filter(event => event.type === 'run_status').map(event => event.data?.status);+ assert.equal(statuses.includes('page_unstable'), false, JSON.stringify({ statuses, tools: tools.map(tool => [tool.name, tool.result?.success, tool.result?.error]) }));+ assert.equal(runEvents.some(event => event.type === 'failure_recovery'), false);+ if (specification.first === 'type_text') {+ const refused = tools.filter(tool => tool.name === 'type_text');+ assert.equal(refused.length, 1, JSON.stringify(tools.map(tool => [tool.name, tool.result?.errorCode])));+ assert.equal(refused[0].result.success, false);+ assert.equal(refused[0].result.errorCode, 'action_binding_unavailable');+ assert.equal(refused[0].result.bindingFailure, 'target_ambiguous');+ assert.equal(refused[0].result.noDispatch, true);+ assert.match(refused[0].result.error, /matches more than one element/);+ }+ const writes = tools.filter(tool => tool.name === 'type_ax');+ if (specification.retryWrite) {+ // Refused at most once, without any text; then written.+ assert.ok(writes.length >= 1 && writes.length <= 2, JSON.stringify(writes));+ if (writes.length === 2) assert.equal(writes[0].result.success, false, JSON.stringify(writes[0].result));+ } else assert.equal(writes.length, 1, JSON.stringify(writes));+ if (specification.stealFocus) {+ // The text command was refused before it was sent: nothing reached the+ // iframe the page moved focus into, the editor or the posting service.+ const frameValue = await page.evaluate(() => qaFrameValue());+ assert.equal(frameValue, '', 'No text may be typed into the iframe that took focus.');+ assert.ok(await page.evaluate(() => qaSteals) >= 1, 'The page moved focus into the iframe.');+ assert.equal(writes[0].result.success, false, JSON.stringify(writes[0].result));+ assert.equal(commands.filter(command => command.method === 'Input.insertText').length, 0, 'No text command was sent.');+ assert.equal(state.model, '');+ assert.deepEqual(localPosts, []);+ assert.equal(tools.filter(tool => tool.name === 'click_ax').length, 0);+ summaries.push({ case: specification.name, inputCommands: 0, frameValue, refusal: writes[0].result.preparationVeto || writes[0].result.error,+ posts: 0, feedChanges: state.feedChanges, statuses: [...new Set(statuses)] });+ await page.close();+ continue;+ }+ assert.equal(writes.at(-1).result.success, true, JSON.stringify(writes.at(-1).result));+ assert.equal(writes.at(-1).result.verified, true);+ const insertions = commands.filter(command => command.method === 'Input.insertText');+ const presses = commands.filter(command => command.method === 'Input.dispatchMouseEvent' && command.params?.type === 'mousePressed'+ && command.params.x >= editorRect.left && command.params.x <= editorRect.right && command.params.y >= editorRect.top && command.params.y <= editorRect.bottom);+ assert.equal(insertions.length, 1, 'Dispatch the exact text through CDP once.');+ assert.equal(insertions[0].params.text, body);+ // One press focuses the editor, none when its centre was already covered.+ // Bold and Italic are never pressed.+ assert.equal(presses.length, specification.bubbleFirst ? 0 : 1, JSON.stringify(presses));+ assert.equal(state.bubblePresses, 0, 'Bold and Italic are never pressed.');+ assert.ok(state.bubbles >= 1, 'The page drew its bubble over the clicked point.');+ assert.equal(current.bubbleCoveredWrite, true, 'The bubble covered the middle of the editor while the text was written.');+ assert.ok(state.feedChanges >= 5, 'The timeline behind the dialog kept updating.');+ assert.ok(state.events.some(event => event.trusted && ['beforeinput', 'input'].includes(event.type)));+ assert.equal(current.postReobserved, true);+ assert.equal(tools.filter(tool => tool.name === 'click_ax').length, 1);+ assert.deepEqual(localPosts, [{ case: specification.name, payload: { account, body, audience: 'Everyone can reply' } }],+ 'The owned local POST must contain exactly the application draft, once.');+ assert.equal(state.open, false);+ const done = tools.findLast(tool => tool.name === 'done')?.result;+ assert.equal(done?.outcome, 'success', JSON.stringify(done));+ assert.equal(runEvents.findLast(event => event.type === 'run_complete')?.data?.status, 'completed');+ assert.equal(current.afterWriteState.model, body, 'The application draft holds exactly the text.');+ summaries.push({ case: specification.name, writes: writes.length, inputCommands: insertions.length, presses: presses.length, posts: localPosts.length,+ bubbles: state.bubbles, bubbleCoveredWrite: current.bubbleCoveredWrite, feedChanges: state.feedChanges, statuses: [...new Set(statuses)] });+ await page.close();+ }+ assert.ok(blockedRemote.every(request => ['GET', 'HEAD', 'OPTIONS'].includes(request.method)), 'No external mutation may be attempted.');+ const network = await worker.evaluate(() => xComposeFixtureProbe.blocked);+ assert.ok(network.every(request => ['GET', 'HEAD', 'OPTIONS'].includes(request.method)), 'No background external mutation may be attempted.');+ await fs.writeFile(path.join(out, 'summary.json'), JSON.stringify({ passed: true, installedMV3: true,+ framework: { react: require('react/package.json').version, draftJS: require('draft-js/package.json').version },+ summaries, externalPosts: 0 }, null, 2));+ console.log(JSON.stringify({ passed: true, installedMV3: true, actualReactDraftJS: true, cases: summaries, externalPosts: 0 }));+} finally {+ await context?.close(); await new Promise(resolve => server.close(resolve));+ const resolved = path.resolve(profile);+ assert.ok(resolved.startsWith(path.resolve(os.tmpdir()) + path.sep + 'webbrain-x-compose-mv3-'));+ await fs.rm(resolved, { recursive: true, force: true });+}diff --git a/test/rich-text-preflight-page-change.mjs b/test/rich-text-preflight-page-change.mjsnew file mode 100644index 000000000..bcdea224f--- /dev/null+++ b/test/rich-text-preflight-page-change.mjs@@ -0,0 +1,108 @@+import assert from 'node:assert/strict';+import { test } from 'node:test';++// Chrome resolves a type_text selector through the trusted CDP resolver+// before the call is validated, and that resolver's scroll is fenced. On a+// page that keeps updating by itself (X's timeline behind the composer) the+// fence refuses it before anything is typed. That refusal used to be reported+// as "the trusted selector resolver could not attach ... DevTools is open",+// which sent the model looking for a cause that did not exist. It must say+// that the page changed and point to typing by reference.+const PAGE_URL = 'https://x.com/compose/post';+const SELECTOR = 'div[role="textbox"][aria-label="Post text"]';+const area = { get: async () => ({}), set: async () => {}, remove: async () => {} };+const contentProbe = { resolved: true, refId: '', documentToken: 'content-1', refScopeUrl: PAGE_URL,+ rect: { x: 300, y: 160, w: 516, h: 96, pageX: 300, pageY: 160 },+ fieldMeta: { tag: 'div', role: 'textbox', contentEditable: true }, toolbarContext: false, toolbarRegionRef: '' };+const contentMessages = [];+let contentResolved = true;+const api = {+ storage: { local: area, session: area },+ runtime: { getURL: value => `extension://test/${value}`, sendMessage: async () => ({}) },+ tabs: {+ get: async id => ({ id, url: PAGE_URL, title: 'Compose post' }),+ sendMessage: async (_id, message) => {+ contentMessages.push(message.action);+ if (message.action !== 'probe_rich_text_toolbar_retry_target') return { ready: true };+ return contentResolved ? structuredClone(contentProbe) : { resolved: false };+ },+ },+ webNavigation: { getAllFrames: async () => [{ frameId: 0, parentFrameId: -1, documentId: 'document-1', url: PAGE_URL }] },+ scripting: { executeScript: async () => [{ result: null }] },+};+globalThis.chrome = api;+globalThis.browser = api;++const { Agent } = await import('../src/chrome/src/agent/agent.js');+const { cdpClient } = await import('../src/chrome/src/cdp/cdp-client.js');+let nextTab = 8100;++async function preflight({ attach, churn = true }) {+ const tab = nextTab++, commands = [];+ const agent = new Agent({ getActive: () => null, getProvider: () => null, getVisionProvider: async () => null });+ const original = { attach: cdpClient.attach, sendCommand: cdpClient.sendCommand };+ contentMessages.length = 0;+ cdpClient.attach = attach;+ // The real resolver runs; only the protocol transport is replaced.+ cdpClient.sendCommand = async (_tab, method) => {+ commands.push(method);+ if (method === 'Runtime.evaluate') return { result: { value: { found: true, x: 558, y: 208, width: 516, height: 96, inViewport: true, hitOk: true, tag: 'DIV', type: '' } } };+ return {};+ };+ agent._runningTabs.add(tab);+ await agent._beginPageFeedbackRun(tab, 'act');+ const from = { tab: { id: tab }, frameId: 0, documentId: 'document-1', url: PAGE_URL };+ const state = agent.pageMonitorState(from, 'content-1');+ assert.equal(state.active, true);+ // A feed item arrives: automatic page activity, no user involved.+ if (churn) assert.equal(agent.observePageFeedback(from, { ...state, seq: 1, revision: 1, kind: 'dom', source: 'page', target: 'div#timeline' }).accepted, true);+ try {+ const result = await agent._preflightRichTextToolbarTarget(tab, 'type_text', { selector: SELECTOR, text: 'Owned QA support is coming soon' }, { supportsVision: false });+ return { result, commands };+ } finally {+ cdpClient.attach = original.attach;+ cdpClient.sendCommand = original.sendCommand;+ agent._finishPageFeedbackRun(tab);+ agent._runningTabs.delete(tab);+ }+}++for (const resolved of [true, false]) test(`Chrome: selector typing refused because the page changed says so, not that DevTools is open${resolved ? '' : ' (content probe unresolved)'}`, async () => {+ contentResolved = resolved;+ const { result, commands } = await preflight({ attach: async () => {} }).finally(() => { contentResolved = true; });+ const block = result.block;+ assert.ok(block, JSON.stringify(result));+ assert.equal(block.success, false);+ assert.equal(block.dispatched, false);+ assert.equal(block.noDispatch, true);+ assert.doesNotMatch(block.error, /DevTools|could not attach|Re-read the page and retry/);+ assert.match(block.error, /page changed while WebBrain was locating/);+ assert.match(block.error, /nothing was typed/);+ assert.match(block.error, /type_ax/);+ assert.match(block.error, /ref_id/);+ // Repeating the same selector call meets the same fence.+ assert.equal(block.retryable, undefined);+ // The fence refused before the resolver touched the page.+ assert.deepEqual(commands, ['Runtime.enable']);+ assert.equal(commands.some(method => method.startsWith('Input.')), false);+ assert.ok(contentMessages.includes('probe_rich_text_toolbar_retry_target'));+});++test('Chrome: a resolver that cannot attach still reports the attach failure', async () => {+ const { result, commands } = await preflight({ attach: async () => { throw new Error('Another debugger is already attached to the tab with id: 1.'); } });+ const block = result.block;+ assert.ok(block, JSON.stringify(result));+ assert.equal(block.noDispatch, true);+ assert.match(block.error, /could not attach to this tab/);+ assert.doesNotMatch(block.error, /page changed while/);+ assert.equal(block.retryable, undefined);+ assert.deepEqual(commands, []);+});++test('Chrome: on a settled page the trusted resolver still binds the selector target', async () => {+ const { result, commands } = await preflight({ attach: async () => {}, churn: false });+ // Field inspection is a protocol call this transport answers with nothing,+ // so the probe is unresolved; what matters is that no fence refused it.+ assert.ok(commands.includes('Runtime.evaluate'), JSON.stringify(commands));+ assert.doesNotMatch(String(result.block?.error || ''), /page changed while|DevTools/);+});diff --git a/test/x-compose-trusted-typing.mjs b/test/x-compose-trusted-typing.mjsnew file mode 100644index 000000000..f8fedd6e0--- /dev/null+++ b/test/x-compose-trusted-typing.mjs@@ -0,0 +1,647 @@+import assert from 'node:assert/strict';+import fs from 'node:fs';+import { test } from 'node:test';+import { chromium, firefox } from 'playwright';+import { CDPClient, cdpClient } from '../src/chrome/src/cdp/cdp-client.js';+import { Agent } from '../src/chrome/src/agent/agent.js';+import { installPageFeedback } from '../src/chrome/src/agent/page-feedback.js';++// Trace 61282dbf (WebBrain 40.7.1, x.com/compose/post): type_ax reached the+// trusted editor fallback, WebBrain's own focus click made X draw its+// bold/italic bubble in a body-level layer while the timeline behind the+// composer kept updating, and the reselect before typing was refused as+// intervention_changed. The model then tried the selector+// [data-testid="tweetTextarea_0"], which X uses for the composer AND the+// hidden timeline composer behind it; every such decision was discarded and+// the run ended with "the page kept changing".+const read = (build, file) => fs.readFileSync(new URL(`../src/${build}/src/${file}`, import.meta.url), 'utf8');+const POST = 'Owned QA support is coming soon https://example.com/qa/pull/3195';++const xHtml = `<!doctype html><meta charset="utf-8"><title>Compose post</title><style>+body{margin:0;font:15px system-ui;color:#0f1419}+#timeline{padding:12px;width:600px}+#timeline .inline{border:1px solid #ccc;padding:8px;margin:8px 0}+#timeline [contenteditable]{min-height:40px}+#feed article{height:40px;border-bottom:1px solid #eee}+#layers{position:fixed;inset:0;display:flex;justify-content:center;align-items:flex-start}+#backdrop{position:absolute;inset:0;background:rgba(0,0,0,.4)}+#compose{position:relative;margin-top:40px;width:600px;background:#fff;border-radius:16px;padding:12px}+#compose button{margin:4px}+#editor{min-height:96px;width:516px;outline:none;margin:8px 40px}+#format-bubble{position:fixed;z-index:30;box-sizing:border-box;width:72px;height:34px;display:flex;gap:4px;background:#111;border-radius:8px;padding:2px}+</style>+<main id="timeline" aria-hidden="true">+ <button id="new-posts" type="button">New posts are available</button>+ <div class="inline"><div contenteditable="true" role="textbox" aria-label="Post text" aria-multiline="true" data-testid="tweetTextarea_0"><div><br></div></div>+ <button type="button" data-testid="tweetButtonInline" disabled>Post</button></div>+ <div id="feed"></div>+</main>+<div id="layers">+ <div id="backdrop"></div>+ <div id="compose" role="dialog" aria-modal="true">+ <div role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="0"></div>+ <button type="button" aria-label="Close">x</button>+ <button type="button">Drafts</button>+ <button type="button" aria-label="Choose audience">Everyone</button>+ <div id="editor" contenteditable="true" role="textbox" aria-label="Post text" aria-multiline="true" data-testid="tweetTextarea_0"><div><br></div></div>+ <button id="reply-scope" type="button">Everyone can reply</button>+ <nav aria-label="Composer tools"><div role="tablist">+ <button type="button" aria-label="Add photos or video">Media</button><button type="button" aria-label="Add emoji">Emoji</button>+ </div></nav>+ <button id="post" type="button" data-testid="tweetButton" disabled>Post</button>+ </div>+</div>+<script>+ window.timelineTicks = 0;+ window.startTimeline = () => { window.timeline = setInterval(() => {+ const tick = ++window.timelineTicks;+ document.getElementById('new-posts').textContent = 'New posts are available (' + tick + ')';+ const feed = document.getElementById('feed'), item = document.createElement('article');+ item.textContent = 'Live post ' + tick; feed.prepend(item);+ if (feed.children.length > 4) feed.lastElementChild.remove();+ }, 40); };+ const editor = document.getElementById('editor');+ window.bubbleShown = 0;+ // A real click in X's editor draws its bold/italic bubble in a body-level+ // layer on a later frame (React schedules it after the click's own task),+ // over the point that was clicked: in the trace's screenshot it sits in the+ // middle of the empty editor, where WebBrain had clicked.+ editor.addEventListener('pointerup', event => {+ if (!event.isTrusted) return;+ const x = event.clientX, y = event.clientY;+ requestAnimationFrame(() => setTimeout(() => {+ window.bubbleShown++;+ let bubble = document.getElementById('format-bubble');+ if (!bubble) {+ bubble = document.createElement('div'); bubble.id = 'format-bubble';+ bubble.innerHTML = '<button type="button" aria-label="Bold">B</button><button type="button" aria-label="Italic">I</button>';+ bubble.addEventListener('pointerdown', () => { window.bubblePressed = (window.bubblePressed || 0) + 1; });+ document.body.append(bubble);+ }+ bubble.style.left = (x - 36) + 'px'; bubble.style.top = (y - 17) + 'px';+ }, 0));+ });+ editor.addEventListener('input', () => { document.getElementById('post').disabled = !editor.innerText.trim(); });+</script>`;++async function fixture(engine, build, { runToken = 'test-run' } = {}) {+ const browser = await engine.launch({ headless: true });+ const context = await browser.newContext({ viewport: { width: 1126, height: 745 } });+ await context.route('https://monitor.test/**', route => route.fulfill({ contentType: 'text/html', body: xHtml }));+ await context.addInitScript(token => { window.monitorRunToken = token; }, runToken);+ await context.addInitScript(() => {+ window.monitorEnabled = true;+ window.feedback = [];+ window.messageListeners = [];+ const runtime = { onMessage: {+ addListener: fn => messageListeners.push(fn),+ removeListener: fn => { messageListeners = messageListeners.filter(item => item !== fn); },+ }, async sendMessage(msg) {+ if (msg.action === 'get_page_monitor_state') return { active: monitorEnabled, runToken: monitorRunToken, documentToken: msg.documentToken };+ if (msg.action === 'page_feedback') { feedback.push(msg.feedback); return { accepted: true }; }+ return {};+ } };+ window.chrome = { runtime }; window.browser = window.chrome;+ window.deliver = (action, params = {}) => messageListeners.forEach(fn => fn({ target: 'content', action,+ params: { runToken: monitorRunToken, ...params }, active: action === 'page_monitor_state' ? monitorEnabled : undefined,+ runToken: monitorRunToken }, {}, value => { window.lastMonitorResponse = value; }));+ });+ await context.addInitScript({ content: read(build, 'content/page-monitor-shadow.js') });+ await context.addInitScript({ content: read(build, 'content/page-monitor.js') });+ const page = await context.newPage();+ await page.goto('https://monitor.test/compose/post');+ await page.waitForTimeout(200);+ await page.evaluate(() => { feedback = []; });+ return { browser, context, page };+}++// One composer on a live page: the page monitor, the agent's page feedback,+// its trusted-typing fallback and the CDP input attribution are production+// code. Only the content-script message transport is a direct page call.+async function composeHost(page, tab, host) {+ const documentToken = await page.evaluate(() => document.documentElement.getAttribute('data-webbrain-page-revision').split(':')[0]);+ host.pageMonitorState({ tab: { id: tab }, frameId: 0 }, documentToken);+ await page.exposeFunction('feedbackToHost', feedback => host.observePageFeedback({ tab: { id: tab }, frameId: 0 }, feedback));+ await page.exposeFunction('monitorStateFromHost', token => host.pageMonitorState({ tab: { id: tab }, frameId: 0 }, token));+ await page.addScriptTag({ content: read('chrome', 'content/rich-text-toolbar-heuristic.js') });+ await page.addScriptTag({ content: read('chrome', 'content/accessibility-tree.js') });+ await page.evaluate(() => {+ const original = chrome.runtime.sendMessage;+ chrome.runtime.sendMessage = async message => {+ if (message.action === 'get_page_monitor_state') return await monitorStateFromHost(message.documentToken);+ const response = await original(message);+ return message.action === 'page_feedback' ? await feedbackToHost(message.feedback) : response;+ };+ window._fieldMeta = el => ({ tag: 'div', contentEditable: true, ariaLabel: el.getAttribute('aria-label') });+ window._editableTextValue = el => el.innerText.replace(/\n$/, '');+ window.readProseMirrorText = () => null;+ window.isDraftJsEditor = () => false;+ window.readDraftJsText = () => null;+ window.readEmptyCaretText = () => null;+ window._stableFieldSelector = el => '#' + el.id;+ window._setFieldValueMatches = (actual, prefix, text, clear) => actual === (clear ? '' : prefix) + text;+ });+ const source = read('chrome', 'content/content.js');+ const entries = ['ax_prepare_field_for_trusted_type', 'ax_verify_field_value', 'field_value_digest'].map(action => {+ const start = source.indexOf(`'${action}': ${action === 'field_value_digest' ? 'async ' : ''}() => {`), end = source.indexOf('\n },', start);+ assert.ok(start > 0 && end > start); return source.slice(start, end + 8);+ });+ const typingPreparation = source.match(/^ function _withTypingPreparation\([\s\S]*?^ }/m)?.[0];+ assert.ok(typingPreparation);+ await page.addScriptTag({ content: `${typingPreparation}\nwindow.axHelpers = { ${entries.join(',\n')} };` });+}++function fieldHostClass() {+ class FieldHost {+ static STATE_CHANGE_TOOLS = new Set(['set_field', 'type_ax', 'type_text']);+ isRunning() { return true; }+ _checkAbort() { return false; }+ _throwIfAborted() {}+ _persist() {}+ async _textMutationValueDigest(...args) { return Agent.prototype._textMutationValueDigest.call(this, ...args); }+ _appOwnedUserMessage(content) { return { role: 'user', content }; }+ async executeTool(tabId, name, args, _update, executionContext) {+ // The content-script editor write proves trusted typing is required+ // without dispatching anything, as on X's composer.+ executionContext._contentActionDispatchState.started = true;+ const result = await Agent.prototype._maybeFallbackFieldWithCdpImpl.call(this, tabId, name, args,+ { success: false, verified: false, dispatched: false, noDispatch: true, _expectedValue: args.text,+ error: 'Contenteditable fields require trusted browser typing', trustedTypeRequired: true });+ return result.textUnchanged === true+ ? Agent.prototype._finalizeTextMutationResult.call(this, tabId, name, args, result)+ : result;+ }+ }+ installPageFeedback(FieldHost);+ return FieldHost;+}++async function withComposer(tab, run, { timeline = true, beforeDecision = null } = {}) {+ const savedChrome = globalThis.chrome, savedBrowser = globalThis.browser;+ const savedCdpAttach = cdpClient.attach, savedCdpSend = cdpClient.sendCommand;+ const FieldHost = fieldHostClass();+ const host = new FieldHost();+ let browser, page, session;+ const state = { commands: [], preparations: [], afterRelease: null };+ const api = { runtime: {}, tabs: {+ get: async () => ({ url: 'https://monitor.test/compose/post' }),+ sendMessage: async (_tab, message) => {+ if (!page) return {};+ if (message.action.startsWith('ax_') || message.action === 'field_value_digest') return page.evaluate(async message => {+ window.msg = message;+ let finish = () => {};+ try {+ finish = __wbPageMonitor.beginContentAction(message.action, message.params || {});+ const result = await axHelpers[message.action]();+ if (message.action === 'field_value_digest') {+ result.documentToken = document.documentElement.getAttribute('data-webbrain-page-revision').split(':')[0];+ result.refScopeUrl = location.href;+ }+ return result;+ } catch (error) {+ return { success: false, dispatched: error.dispatched === true, noDispatch: error.dispatched !== true,+ ...(error.pagePreparationVeto ? { preparationVeto: error.pagePreparationVeto } : {}),+ ...(error.code === 'page_feedback_pending' ? { pageFeedbackPending: true } : {}),+ error: `${message.action} failed: ${error.message}` };+ } finally { finish(); }+ }, message).then(result => { if (message.action === 'ax_prepare_field_for_trusted_type') state.preparations.push(result); return result; });+ return page.evaluate(message => new Promise(resolve => messageListeners.forEach(fn => fn(message, {}, resolve))), message);+ },+ }, debugger: { sendCommand(_source, method, params, callback) {+ state.commands.push({ method, params });+ void (async () => {+ const result = await session.send(method, params);+ if (method === 'Input.dispatchMouseEvent' && params.type === 'mouseReleased' && state.afterRelease) {+ const hook = state.afterRelease; state.afterRelease = null; await hook();+ }+ return result;+ })().then(callback, error => {+ api.runtime.lastError = { message: error.message }; callback(); delete api.runtime.lastError;+ });+ } } };+ globalThis.chrome = api; delete globalThis.browser;+ try {+ await host._beginPageFeedbackRun(tab, 'interactive');+ ({ browser, page } = await fixture(chromium, 'chrome', { runToken: host._pageFeedbackRuns.get(tab).token }));+ await composeHost(page, tab, host);+ const client = new CDPClient(); client.sessions.set(tab, { attached: true });+ session = await page.context().newCDPSession(page);+ cdpClient.attach = async () => ({ attached: true });+ cdpClient.sendCommand = (...args) => client.sendCommand(...args);+ if (timeline) await page.evaluate(() => startTimeline());+ await page.waitForTimeout(250);+ const refs = await page.evaluate(() => ({ editor: __wb_ax_ref(document.getElementById('editor')) }));+ if (beforeDecision) await beforeDecision(page);+ // The decision is made from a fresh observation while the timeline runs.+ host._pageFeedbackRuns.get(tab).events.clear();+ await host._capturePageFeedbackModelState(tab, []);+ await run({ host, page, refs, state });+ } finally {+ cdpClient.attach = savedCdpAttach; cdpClient.sendCommand = savedCdpSend;+ if (session) await session.detach().catch(() => {});+ if (host._pageFeedbackRuns?.has(tab)) host._finishPageFeedbackRun(tab);+ if (savedChrome === undefined) delete globalThis.chrome; else globalThis.chrome = savedChrome;+ if (savedBrowser === undefined) delete globalThis.browser; else globalThis.browser = savedBrowser;+ if (browser) await browser.close();+ }+}++const insertions = state => state.commands.filter(item => item.method === 'Input.insertText');+const bubbleCoversEditor = page => page.evaluate(() => {+ const rect = document.getElementById('editor').getBoundingClientRect();+ return !!document.elementFromPoint(rect.left + rect.width / 2, rect.top + rect.height / 2)?.closest('#format-bubble');+});+const editorText = page => page.locator('#editor').innerText().then(text => text.replace(/\n$/, ''));++test('Chrome type_ax writes X\'s composer after the page draws its bubble for WebBrain\'s own focus click', async () => {+ await withComposer(91_001, async ({ host, page, refs, state }) => {+ // Let X's own reaction to the click land before the editor is reselected.+ state.afterRelease = () => page.waitForTimeout(30);+ const result = await host.executeTool(91_001, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, true, JSON.stringify({ result, preparations: state.preparations }));+ assert.equal(result.verified, true);+ assert.equal(await editorText(page), POST);+ assert.equal(insertions(state).length, 1, 'One trusted text command');+ assert.equal(insertions(state)[0].params.text, POST);+ assert.equal(await page.evaluate(() => window.bubbleShown), 1, 'X drew its bubble for the trusted click');+ assert.equal(await bubbleCoversEditor(page), true, 'As in the trace\'s screenshot, the bubble covers the middle of the editor');+ assert.ok(await page.evaluate(() => window.timelineTicks) > 3, 'The timeline kept updating throughout');+ assert.deepEqual(state.preparations.map(item => item.success), [true, true]);+ assert.equal(await page.evaluate(() => document.getElementById('post').disabled), false);+ });+});++for (const timeline of [true, false]) {+ test(`Chrome type_ax still writes X's composer when its reaction is already reported before the reselect${timeline ? ' (live timeline)' : ''}`, async () => {+ const tab = timeline ? 91_002 : 91_003;+ await withComposer(tab, async ({ host, page, refs, state }) => {+ // The bubble (and timeline) updates are delivered to the agent first.+ state.afterRelease = () => page.waitForTimeout(400);+ const result = await host.executeTool(tab, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, true, JSON.stringify({ result, preparations: state.preparations }));+ assert.equal(await editorText(page), POST);+ assert.equal(insertions(state).length, 1);+ assert.equal(await page.evaluate(() => window.bubbleShown), 1);+ assert.equal(await bubbleCoversEditor(page), true);+ }, { timeline });+ });+}++// The page moves the composer a little after WebBrain's click (a layout+// settling, for example). Its editor and dialog are the same, so the decision+// stands, as it would for any certified target that only moved.+for (const timeline of [true, false]) {+ test(`Chrome type_ax writes X's composer when the page moves it after WebBrain's focus click${timeline ? ' (live timeline)' : ''}`, async () => {+ const tab = timeline ? 91_006 : 91_007;+ await withComposer(tab, async ({ host, page, refs, state }) => {+ const before = await page.evaluate(() => document.getElementById('editor').getBoundingClientRect().top);+ state.afterRelease = async () => {+ await page.waitForTimeout(30);+ await page.evaluate(() => document.head.append(Object.assign(document.createElement('style'), { textContent: '#compose{margin-top:52px}' })));+ await page.waitForTimeout(50);+ };+ const result = await host.executeTool(tab, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, true, JSON.stringify({ result, preparations: state.preparations }));+ assert.equal(await editorText(page), POST);+ assert.equal(insertions(state).length, 1);+ assert.equal(await page.evaluate(() => document.getElementById('editor').getBoundingClientRect().top) - before, 12);+ assert.equal(await bubbleCoversEditor(page), true);+ }, { timeline });+ });+}++const pressEditorCentre = async page => {+ const rect = await page.evaluate(() => { const r = document.getElementById('editor').getBoundingClientRect(); return { x: r.left + r.width / 2, y: r.top + r.height / 2 }; });+ await page.mouse.click(rect.x, rect.y);+ await page.waitForFunction(() => window.bubbleShown > 0);+ await page.waitForTimeout(300);+};+const presses = state => state.commands.filter(item => item.method === 'Input.dispatchMouseEvent' && item.params.type === 'mousePressed');++// The page moving focus is page activity, not the user: the reselection+// focuses the editor again, and the text still goes only into it.+for (const delay of [30, 400]) for (const timeline of [true, false]) {+ test(`Chrome type_ax focuses X's composer again when the page moved focus after WebBrain's click (${delay} ms${timeline ? ', live timeline' : ''})`, async () => {+ const tab = 91_200 + delay + (timeline ? 1000 : 0);+ await withComposer(tab, async ({ host, page, refs, state }) => {+ state.afterRelease = async () => {+ await page.waitForTimeout(30);+ await page.evaluate(() => document.getElementById('reply-scope').focus());+ await page.waitForTimeout(delay);+ };+ const result = await host.executeTool(tab, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, true, JSON.stringify({ result, preparations: state.preparations }));+ assert.equal(await editorText(page), POST);+ assert.equal(insertions(state).length, 1);+ assert.equal(await page.evaluate(() => document.getElementById('reply-scope').textContent), 'Everyone can reply');+ }, { timeline });+ });+}++// The state in the trace's screenshot: the bubble from an earlier click still+// covers the middle of the focused editor when the model decides again.+test('Chrome type_ax writes X\'s composer while the bubble of an earlier click still covers it', async () => {+ await withComposer(91_004, async ({ host, page, refs, state }) => {+ assert.equal(await bubbleCoversEditor(page), true);+ assert.equal(await page.evaluate(() => document.activeElement?.id), 'editor');+ const result = await host.executeTool(91_004, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, true, JSON.stringify({ result, preparations: state.preparations }));+ assert.equal(await editorText(page), POST);+ assert.equal(insertions(state).length, 1);+ assert.equal(presses(state).length, 0, 'The covered centre is not pressed');+ assert.equal(await page.evaluate(() => window.bubblePressed || 0), 0, 'Bold and Italic are never pressed');+ }, { beforeDecision: pressEditorCentre });+});++test('Chrome an editor covered by the bubble after focus left it is not typed into', async () => {+ await withComposer(91_005, async ({ host, page, refs, state }) => {+ const result = await host.executeTool(91_005, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, false, JSON.stringify(result));+ assert.equal(result.dispatched, false, JSON.stringify(result));+ assert.equal(insertions(state).length, 0);+ assert.equal(presses(state).length, 0);+ assert.equal(await page.evaluate(() => window.bubblePressed || 0), 0);+ assert.equal(await editorText(page), '');+ }, { beforeDecision: async page => {+ await pressEditorCentre(page);+ // Focus leaves the editor; the bubble stays over it.+ await page.evaluate(() => document.getElementById('reply-scope').focus());+ } });+});++for (const [name, change] of [+ ['its reply audience changes', page => page.evaluate(() => { document.getElementById('reply-scope').textContent = 'Only accounts you follow can reply'; })],+ ['its editor is replaced', page => page.evaluate(() => {+ const old = document.getElementById('editor'), next = old.cloneNode(true);+ old.replaceWith(next);+ })],+ ['the user clicks elsewhere', page => page.mouse.click(40, 700)],+ ['another dialog makes the composer inert', page => page.evaluate(() => {+ const dialog = document.createElement('div');+ dialog.setAttribute('role', 'alertdialog'); dialog.textContent = 'Discard post?';+ dialog.style.cssText = 'position:fixed;left:200px;top:200px;width:300px;height:120px;background:#fff;z-index:40';+ document.body.append(dialog); document.getElementById('compose').inert = true;+ })],+]) for (const delay of [30, 400]) for (const timeline of [true, false]) {+ test(`Chrome type_ax stops without typing when, after WebBrain's focus click, ${name} (${delay} ms${timeline ? ', live timeline' : ''})`, async () => {+ const tab = 91_100 + delay + name.length + (timeline ? 1000 : 0);+ await withComposer(tab, async ({ host, page, refs, state }) => {+ state.afterRelease = async () => { await change(page); await page.waitForTimeout(delay); };+ const result = await host.executeTool(tab, 'type_ax', { ref_id: refs.editor, text: POST, clear: true });+ assert.equal(result.success, false, JSON.stringify(result));+ if (process.env.WEBBRAIN_TEST_DIAGNOSTICS) console.log(name, delay, result.preparationVeto, result.error);+ assert.equal(insertions(state).length, 0, 'No text may be sent after the composer or the user changed it');+ assert.notEqual(result.mutationMayHaveOccurred, true, JSON.stringify(result));+ const editor = await page.$('#editor');+ assert.equal(editor ? (await editor.innerText()).replace(/\n$/, '') : '', '');+ }, { timeline });+ });+}++// The page monitor is shared code; the same build file runs in both engines.+for (const [engine, build] of [[chromium, 'chrome'], [firefox, 'firefox']]) {+ test(`${build}: a selector that matches X's composer and its hidden timeline copy is reported as ambiguous`, async () => {+ const { browser, page } = await fixture(engine, build);+ try {+ await page.addScriptTag({ content: read(build, 'content/accessibility-tree.js') });+ const result = await page.evaluate(() => {+ deliver('page_monitor_capture_model', { includeTree: true });+ const snapshotToken = lastMonitorResponse.snapshotToken;+ const validate = params => { deliver('page_monitor_validate_model', { snapshotToken, ...params }); return lastMonitorResponse; };+ return {+ ambiguous: validate({ tool: 'type_text', selector: '[data-testid="tweetTextarea_0"]', text: 'x' }),+ unique: validate({ tool: 'type_text', selector: '#editor', text: 'x' }),+ missing: validate({ tool: 'type_text', selector: '#no-such-editor', text: 'x' }),+ byRef: validate({ tool: 'type_ax', ref_id: __wb_ax_ref(document.getElementById('editor')), text: 'x' }),+ };+ });+ assert.equal(result.ambiguous.ready, false);+ assert.equal(result.ambiguous.reason, 'target_ambiguous', JSON.stringify(result.ambiguous));+ assert.equal(result.unique.ready, true, JSON.stringify(result.unique));+ assert.equal(result.missing.ready, false);+ assert.equal(result.missing.reason, 'target_unresolved');+ assert.equal(result.byRef.ready, true, JSON.stringify(result.byRef));+ } finally { await browser.close(); }+ });+}++for (const [engine, build] of [[chromium, 'chrome'], [firefox, 'firefox']]) {+ test(`${build}: a focused editor under the page's bubble is certified for typing only, while it keeps focus`, async () => {+ const { browser, page } = await fixture(engine, build);+ try {+ await page.addScriptTag({ content: read(build, 'content/accessibility-tree.js') });+ const centre = await page.evaluate(() => { const r = document.getElementById('editor').getBoundingClientRect(); return { x: r.left + r.width / 2, y: r.top + r.height / 2 }; });+ await page.mouse.click(centre.x, centre.y);+ await page.waitForFunction(() => window.bubbleShown > 0);+ await page.waitForTimeout(300);+ const covered = await page.evaluate(() => {+ const r = document.getElementById('editor').getBoundingClientRect();+ return !!document.elementFromPoint(r.left + r.width / 2, r.top + r.height / 2)?.closest('#format-bubble');+ });+ assert.equal(covered, true);+ const result = await page.evaluate(() => {+ deliver('page_monitor_capture_model', { includeTree: true });+ const snapshotToken = lastMonitorResponse.snapshotToken, focused = lastMonitorResponse.focusedTargetAvailable;+ const ref = __wb_ax_ref(document.getElementById('editor'));+ const validate = params => { deliver('page_monitor_validate_model', { snapshotToken, ...params }); return lastMonitorResponse; };+ const typing = validate({ tool: 'type_ax', ref_id: ref, text: 'x' });+ // Keys also go to the focused field, so press_keys may use it too.+ const keys = validate({ tool: 'press_keys', key: 'Enter' });+ const keysByRef = validate({ tool: 'press_keys', ref_id: ref, key: 'Enter' });+ const click = validate({ tool: 'click_ax', ref_id: ref });+ document.getElementById('reply-scope').focus();+ const unfocused = validate({ tool: 'type_ax', ref_id: ref, text: 'x' });+ return { focused, typing, keys, keysByRef, click, unfocused };+ });+ assert.equal(result.focused, true, 'The focused editor stays certified under the bubble');+ assert.equal(result.typing.ready, true, JSON.stringify(result.typing));+ assert.equal(result.keys.ready, true, JSON.stringify(result.keys));+ assert.equal(result.keysByRef.ready, true, JSON.stringify(result.keysByRef));+ assert.equal(result.click.ready, false);+ assert.equal(result.click.reason, 'target_uncovered', JSON.stringify(result.click));+ assert.equal(result.unfocused.ready, false);+ assert.equal(result.unfocused.reason, 'focus_changed', JSON.stringify(result.unfocused));+ } finally { await browser.close(); }+ });+}++// The monitor recognizes the typing action's own press by its actual native+// receivers (of the press and of its release) and button, never by caller+// metadata, in both builds. Only one such press is preparation.+const pressNames = { focus: 'a single left press', right: 'a right-button press', double: 'a double press',+ second: 'a second single left press', elsewhere: 'a single left press released over another node',+ unmarked: 'a single left press followed by native text without a marked receiver' };+for (const [engine, build] of [[chromium, 'chrome'], [firefox, 'firefox']]) for (const press of Object.keys(pressNames)) {+ const kept = ['focus', 'unmarked'].includes(press);+ test(`${build}: ${pressNames[press]} on the certified editor ${press === 'unmarked' ? 'does not type' : kept ? 'keeps' : 'does not keep'}${press === 'unmarked' ? '' : ' its typing certificate'}`, async () => {+ const { browser, page } = await fixture(engine, build);+ try {+ await page.addScriptTag({ content: read(build, 'content/accessibility-tree.js') });+ const setup = await page.evaluate(() => {+ deliver('page_monitor_capture_model', { includeTree: true });+ const snapshotToken = lastMonitorResponse.snapshotToken;+ const editor = document.getElementById('editor'), ref = __wb_ax_ref(editor);+ deliver('page_monitor_prepare', { tool: 'type_ax', ref_id: ref, operationId: 'owned-typing', expectedModelSnapshot: snapshotToken, allowPassiveRebase: true });+ const prepared = lastMonitorResponse;+ const rect = editor.getBoundingClientRect();+ editor.setAttribute('data-webbrain-native-target', 'owned-press');+ return { ref, prepared, x: rect.left + rect.width / 2, y: rect.top + rect.height / 2 };+ });+ assert.equal(setup.prepared.modelBindingValid, true, JSON.stringify(setup.prepared));+ const pressTypes = ['pointerover', 'pointerenter', 'mouseover', 'mouseenter', 'pointerdown', 'mousedown', 'click', 'input', 'change'];+ const dispatched = await page.evaluate(({ x, y, press, pressTypes }) => {+ deliver('page_monitor_dispatch', { operationId: 'owned-typing', kind: 'click', x, y, navigationCandidate: true,+ pointerButton: press === 'right' ? 'right' : 'left', clickCount: press === 'double' ? 2 : 1,+ nativeTarget: 'owned-press', eventTypes: pressTypes });+ return lastMonitorResponse;+ }, { ...setup, press, pressTypes });+ assert.equal(dispatched.ready, true, JSON.stringify(dispatched));+ // The monitor's own classification of each half of the press.+ assert.equal(dispatched.typingPreparation === true, !['right', 'double'].includes(press), JSON.stringify(dispatched));+ await page.mouse.move(setup.x, setup.y);+ await page.mouse.down({ button: press === 'right' ? 'right' : 'left' });+ // As CDP marks the node under the release point before sending it.+ const released = await page.evaluate(({ x, y, press }) => {+ const receiver = press === 'elsewhere' ? document.getElementById('reply-scope') : document.getElementById('editor');+ receiver.setAttribute('data-webbrain-native-target', 'owned-release');+ deliver('page_monitor_dispatch', { operationId: 'owned-typing', kind: 'click', x, y, release: true, nativeTarget: 'owned-release' });+ return lastMonitorResponse;+ }, { ...setup, press });+ assert.equal(released.typingPreparation === true, !['right', 'double', 'elsewhere'].includes(press), JSON.stringify(released));+ await page.mouse.up({ button: press === 'right' ? 'right' : 'left' });+ await page.waitForFunction(() => window.bubbleShown > 0);+ if (press === 'second') {+ const again = await page.evaluate(({ x, y, pressTypes }) => {+ document.getElementById('editor').setAttribute('data-webbrain-native-target', 'owned-press-2');+ deliver('page_monitor_dispatch', { operationId: 'owned-typing', kind: 'click', x, y, navigationCandidate: true,+ pointerButton: 'left', clickCount: 1, nativeTarget: 'owned-press-2', eventTypes: pressTypes });+ return lastMonitorResponse;+ }, { ...setup, pressTypes });+ assert.equal(again.ready, true, JSON.stringify(again));+ // A second press is the action's input to the page, not preparation.+ assert.equal(again.typingPreparation, undefined, JSON.stringify(again));+ return;+ }+ const after = await page.evaluate(({ ref, press }) => {+ const editor = document.getElementById('editor');+ editor.focus();+ // The page keeps reacting in its body-level layer (the bubble moves):+ // passive activity that only a kept certificate is judged against.+ const bubble = document.getElementById('format-bubble');+ bubble.style.left = (parseFloat(bubble.style.left) + 1) + 'px';+ const step = action => {+ try { const finish = __wbPageMonitor.beginContentAction(action, { ref_id: ref }); finish(); return 'kept'; }+ catch (error) { return error.pagePreparationVeto?.predicate || error.message; }+ };+ const reselect = step('ax_prepare_field_for_trusted_type'), paste = step('ax_paste_draft');+ editor.setAttribute('data-webbrain-native-target', 'owned-text');+ window.lastMonitorResponse = undefined;+ // Native input always tries to mark its receiver; unmarked input to a+ // keyboard-bound field cannot be shown to be the field's.+ deliver('page_monitor_dispatch', { operationId: 'owned-typing', kind: 'input', nativeInput: true,+ ...(press === 'unmarked' ? {} : { nativeTarget: 'owned-text' }), eventTypes: ['beforeinput', 'input', 'change'] });+ return { reselect, paste, text: lastMonitorResponse };+ }, { ref: setup.ref, press });+ if (process.env.WEBBRAIN_TEST_DIAGNOSTICS) console.log(build, press, JSON.stringify(after));+ if (kept) {+ assert.deepEqual([after.reselect, after.paste], ['kept', 'kept'], JSON.stringify(after));+ } else {+ assert.notEqual(after.reselect, 'kept', JSON.stringify(after));+ assert.notEqual(after.paste, 'kept', JSON.stringify(after));+ }+ // The text command waits for the monitor's own delivery when it refuses.+ const text = after.text === undefined || after.text === null+ ? await page.waitForFunction(() => window.lastMonitorResponse, null, { timeout: 2000 }).then(handle => handle.jsonValue())+ : after.text;+ assert.equal(text.ready, press === 'focus', JSON.stringify(text));+ if (press === 'unmarked') assert.equal(text.preparationVeto?.predicate, 'target_unresolved', JSON.stringify(text));+ } finally { await browser.close(); }+ });+}++// Keyboard input of a certified operation may reach a child document only+// when the certified target is the frame that has focus (or contains it) and+// was not an editable host when certified; the top document's monitor+// answers for its operation, in both builds.+for (const [engine, build] of [[chromium, 'chrome'], [firefox, 'firefox']]) {+ test(`${build}: certified keyboard input reaches a child document only through the certified frame that has focus`, async () => {+ const { browser, page } = await fixture(engine, build);+ try {+ await page.addScriptTag({ content: read(build, 'content/accessibility-tree.js') });+ await page.evaluate(() => new Promise(resolve => {+ // An editor of its own in an iframe, wrapped in a labelled textbox,+ // and a GIF search iframe inside the composer's dialog.+ const host = document.createElement('div');+ host.id = 'frame-editor'; host.setAttribute('role', 'textbox'); host.setAttribute('aria-label', 'Embedded editor');+ const frame = document.createElement('iframe'); frame.id = 'embedded'; frame.srcdoc = '<!doctype html><body contenteditable="true"></body>';+ const gif = document.createElement('iframe'); gif.id = 'gif'; gif.srcdoc = '<!doctype html><input id="inner">';+ host.append(frame); document.getElementById('compose').append(host, gif);+ let loaded = 0; for (const item of [frame, gif]) item.addEventListener('load', () => { if (++loaded === 2) resolve(); }, { once: true });+ }));+ const answers = await page.evaluate(() => {+ deliver('page_monitor_capture_model', { includeTree: true });+ const snapshotToken = lastMonitorResponse.snapshotToken;+ const prepare = (operationId, tool, el) => deliver('page_monitor_prepare', { tool, ref_id: __wb_ax_ref(el), text: 'x',+ operationId, expectedModelSnapshot: snapshotToken, allowPassiveRebase: true });+ const ask = operationId => { deliver('page_monitor_receiver_frame', { operationId }); return lastMonitorResponse.foreignReceiverAllowed; };+ const editor = document.getElementById('editor'), host = document.getElementById('frame-editor');+ prepare('field-typing', 'type_ax', editor);+ prepare('frame-typing', 'type_ax', host);+ prepare('button-click', 'click_ax', document.getElementById('reply-scope'));+ const result = {};+ // Focus in the GIF search iframe inside the composer's dialog.+ document.getElementById('gif').contentDocument.getElementById('inner').focus();+ result.fieldWithGifFocused = ask('field-typing');+ result.frameEditorWithGifFocused = ask('frame-typing');+ result.buttonWithGifFocused = ask('button-click');+ result.unknown = ask('no-such-operation');+ // Focus in the embedded editor's own frame.+ document.getElementById('embedded').contentDocument.body.focus();+ result.frameEditorFocused = ask('frame-typing');+ result.fieldWithEmbeddedFocused = ask('field-typing');+ // The page makes the composer's editor read-only, or replaces it.+ editor.setAttribute('aria-readonly', 'true'); editor.setAttribute('contenteditable', 'false');+ result.readonlyField = ask('field-typing');+ editor.replaceWith(editor.cloneNode(true));+ result.replacedField = ask('field-typing');+ // The embedded editor's frame is replaced.+ const frame = document.getElementById('embedded'), next = frame.cloneNode(true); frame.replaceWith(next);+ result.replacedFrame = ask('frame-typing');+ return result;+ });+ assert.deepEqual(answers, {+ fieldWithGifFocused: false, frameEditorWithGifFocused: false, buttonWithGifFocused: false, unknown: false,+ frameEditorFocused: true, fieldWithEmbeddedFocused: false, readonlyField: false, replacedField: false, replacedFrame: false,+ });+ } finally { await browser.close(); }+ });+}++// Native keyboard input always tries to mark its receiver. Unmarked input is+// refused for any certified operation, pressed or not; an uncertified+// operation keeps its earlier rules.+for (const [engine, build] of [[chromium, 'chrome'], [firefox, 'firefox']]) {+ test(`${build}: unmarked native text is refused for a certified typing operation, not for an uncertified one`, async () => {+ const { browser, page } = await fixture(engine, build);+ try {+ await page.addScriptTag({ content: read(build, 'content/accessibility-tree.js') });+ const results = await page.evaluate(async () => {+ deliver('page_monitor_capture_model', { includeTree: true });+ const snapshotToken = lastMonitorResponse.snapshotToken;+ const editor = document.getElementById('editor'), ref = __wb_ax_ref(editor);+ editor.focus();+ const text = async (operationId, certified) => {+ deliver('page_monitor_prepare', { tool: 'type_ax', ref_id: ref, text: 'x', operationId,+ ...(certified ? { expectedModelSnapshot: snapshotToken, allowPassiveRebase: true } : {}) });+ window.lastMonitorResponse = undefined;+ deliver('page_monitor_dispatch', { operationId, kind: 'input', nativeInput: true, eventTypes: ['beforeinput', 'input', 'change'] });+ for (let wait = 0; wait < 100 && window.lastMonitorResponse === undefined; wait++) await new Promise(resolve => setTimeout(resolve, 20));+ return { ready: lastMonitorResponse?.ready, predicate: lastMonitorResponse?.preparationVeto?.predicate };+ };+ return { certified: await text('certified-text', true), uncertified: await text('uncertified-text', false) };+ });+ assert.deepEqual(results.certified, { ready: false, predicate: 'target_unresolved' }, JSON.stringify(results));+ assert.equal(results.uncertified.ready, true, JSON.stringify(results));+ } finally { await browser.close(); }+ });+}