Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Honour a pointerdown the page cancels in Firefox vision input
Emre Sokullu321e36726ee6
diff --git a/docs/vision-mode.md b/docs/vision-mode.mdindex d24763c27..3352fa3e0 100644--- a/docs/vision-mode.md+++ b/docs/vision-mode.md@@ -119,7 +119,7 @@ With an instant model the MV3 fixture's four steps (click and type, send, scroll`click_at` is rewritten into an ordinary coordinate `click` (`translateVisionClick`) before the batch pipeline runs, so the permission gate, the submit probe and every hard guard (recipient, deletion, money, credentials, publishing) inspect it like any other click. A guard that binds a consequential click to the control and payload it checked (a deletion, a commit after a correction) refuses coordinate clicks on the tree path, where a ref or selector click exists; in a vision run, where the point is all there is, it binds the click to the control the submit probe read at that point (`_resourceMutationActionKey`), with the same record, stage, sibling and confirmation checks and the same card, and `_visionDispatchClick` reads the point again right before input goes out (`_visionMutationBindingBlock`): when the authorized control is no longer there, nothing is sent (`resource_changed_before_dispatch`) and the model chooses again on the updated screen. A guard's advice names what the run offers (`click_at`), never a ref or selector. Only the final dispatch differs:- **Chrome**: trusted CDP pointer events at exactly the point (`_visionDispatchClick`), with the coordinate path's dispatch-time protections kept: a native `<select>` is focused for `type_text` instead of opening an OS popup no screenshot can show, the file-chooser guard, the recipient dispatch binding, `target=_blank` redirection into the run tab (also for a tab opened after the click returned, see Latency). The legacy coordinate path's semantic re-targeting, nearby-input redirect and DOM-snapshot progress check are skipped: a vision click lands where the model pointed, and whether it changed anything is for the screen update to show (a DOM text snapshot cannot see a canvas, a map or a value it does not read). Iframes, including cross-origin ones, receive the events natively.-- **Firefox**: no CDP, so `content.js` `visionInput` dispatches a synthetic pointer sequence at the exact point with real client coordinates, reproducing a press's focus (and its caret: a click in a text field or editable text puts the caret under the pointer) and a click's activation, moving off the element the pointer was last over the way a mouse does (out and leave events, so a hover menu that closes on mouseleave closes; synthetic events give nothing `:hover`, so while the pointer rests on an element it and its ancestors carry an attribute, and every `:hover` in the selectors of the style rules the content script can read also accepts it, in place (`:hover` reads `:is(:hover, [data-webbrain-hover])`, keeping the rule's place in the cascade, its specificity and the other selectors of its list), which a CSS-only menu or tooltip opens for, until the pointer moves on or a person moves their own pointer, when the page's selectors are put back; a wheel moves the pointer to its point first and goes where the browser would send it: the nearest scroller that can still move that way, an exhausted one passing it on unless its `overscroll-behavior` keeps it, never a page locked with `overflow: hidden`, and from a frame that cannot scroll to the page around it), selecting the word under the pointer for a double click and its line (a textarea's line, a whole input, the text block) for a triple click, and entering same-origin iframes: the point is mapped into the frame's own coordinates past its border and padding, undoing the scale a CSS transform or zoom draws the frame at (on itself or an ancestor), and a point on a frame drawn rotated, skewed, flipped or in 3D, which no scale maps, is refused (`frame_unmappable`). The click runs inside the same native file-chooser guard as every agent click (`clickWithoutNativeFilePicker`), and the agent settles that guard afterwards, so a chooser opened a moment later is caught too. Dispatched events reach a disabled control's listeners, unlike a person's click (or `el.click()`), so a click or drag that would press a disabled control, disabled by its fieldset included, is refused before anything is sent (`disabled_control`). A drag gives every move and the release in the coordinates of the document under the point, so a slider in a same-origin frame reads one consistent path; its press focuses what it lands on and starts what a person's press would, since synthetic events alone move no native slider, select no text and start no drag and drop: on an `<input type=range>` the value under the pointer is set as the browser sets it (snapped to the step, an `input` event per change, `change` on release; a click's primary press on the track moves the thumb to its point the same way), on selectable text or in a text field the text from the press to the pointer is selected (after a cancelable `selectstart`; not on links, images, draggable items, other controls or `user-select: none`), and on a `draggable="true"` item HTML5 drag and drop runs as the browser runs it (`dragstart` with a `DataTransfer` the page fills, `drag`/`dragenter`/`dragleave`/`dragover` per step, `drop` where the last `dragover` was accepted, `dragend`; no pointer or mouse events meanwhile). A press the page captures (`setPointerCapture` on its pointerdown, as sliders, canvases and press-and-hold controls do) keeps that capture for the rest of the click or drag: while the sequence runs, the capture methods of the press's window answer for the vision pointer's id (Firefox would throw for it, and no engine routes a synthetic pointer's events), each later pointer event and its compatibility mouse event go to the capturing element in its document's coordinates, `gotpointercapture` comes before them and `lostpointercapture` right after the release, and the page's own methods are back once the sequence ends. A click that must pass the recipient binding keeps the content click path. A click or drag on a frame the content script cannot enter (another site's frame, a plugin, or a frame still loading) is refused (`opaque_frame`): its events would stop at the frame element and reach nothing inside it.+- **Firefox**: no CDP, so `content.js` `visionInput` dispatches a synthetic pointer sequence at the exact point with real client coordinates, reproducing a press's focus (and its caret: a click in a text field or editable text puts the caret under the pointer) and a click's activation, moving off the element the pointer was last over the way a mouse does (out and leave events, so a hover menu that closes on mouseleave closes; synthetic events give nothing `:hover`, so while the pointer rests on an element it and its ancestors carry an attribute, and every `:hover` in the selectors of the style rules the content script can read also accepts it, in place (`:hover` reads `:is(:hover, [data-webbrain-hover])`, keeping the rule's place in the cascade, its specificity and the other selectors of its list), which a CSS-only menu or tooltip opens for, until the pointer moves on or a person moves their own pointer, when the page's selectors are put back; a wheel moves the pointer to its point first and goes where the browser would send it: the nearest scroller that can still move that way, an exhausted one passing it on unless its `overscroll-behavior` keeps it, never a page locked with `overflow: hidden`, and from a frame that cannot scroll to the page around it), selecting the word under the pointer for a double click and its line (a textarea's line, a whole input, the text block) for a triple click, and entering same-origin iframes: the point is mapped into the frame's own coordinates past its border and padding, undoing the scale a CSS transform or zoom draws the frame at (on itself or an ancestor), and a point on a frame drawn rotated, skewed, flipped or in 3D, which no scale maps, is refused (`frame_unmappable`). The click runs inside the same native file-chooser guard as every agent click (`clickWithoutNativeFilePicker`), and the agent settles that guard afterwards, so a chooser opened a moment later is caught too. Dispatched events reach a disabled control's listeners, unlike a person's click (or `el.click()`), so a click or drag that would press a disabled control, disabled by its fieldset included, is refused before anything is sent (`disabled_control`). A press whose `pointerdown` the page cancels gets what a browser gives it: no compatibility `mousedown`, `mousemove` or `mouseup` until its release (the boundary events still go out) and none of the press's defaults below (focus, the caret, a range thumb, a selection, drag and drop, a double click's word), while its `click` still fires. A drag gives every move and the release in the coordinates of the document under the point, so a slider in a same-origin frame reads one consistent path; its press focuses what it lands on and starts what a person's press would, since synthetic events alone move no native slider, select no text and start no drag and drop: on an `<input type=range>` the value under the pointer is set as the browser sets it (snapped to the step, an `input` event per change, `change` on release; a click's primary press on the track moves the thumb to its point the same way), on selectable text or in a text field the text from the press to the pointer is selected (after a cancelable `selectstart`; not on links, images, draggable items, other controls or `user-select: none`), and on a `draggable="true"` item HTML5 drag and drop runs as the browser runs it (`dragstart` with a `DataTransfer` the page fills, `drag`/`dragenter`/`dragleave`/`dragover` per step, `drop` where the last `dragover` was accepted, `dragend`; no pointer or mouse events meanwhile). A press the page captures (`setPointerCapture` on its pointerdown, as sliders, canvases and press-and-hold controls do) keeps that capture for the rest of the click or drag: while the sequence runs, the capture methods of the press's window answer for the vision pointer's id (Firefox would throw for it, and no engine routes a synthetic pointer's events), each later pointer event and its compatibility mouse event go to the capturing element in its document's coordinates, `gotpointercapture` comes before them and `lostpointercapture` right after the release, and the page's own methods are back once the sequence ends. A click that must pass the recipient binding keeps the content click path. A click or drag on a frame the content script cannot enter (another site's frame, a plugin, or a frame still loading) is refused (`opaque_frame`): its events would stop at the frame element and reach nothing inside it.In Firefox a vision click that must pass the recipient binding is sent by the content click, which checks that binding and sends one left click on the element at the point; there a right, middle or repeated click is refused (`recipient_bound_click`) rather than turned into a left click, and an authorized deletion is read again at its point first. The rich-text toolbar recovery guard checks a vision click's target but does not bind it to an element, so it keeps the vision dispatch in both builds.diff --git a/src/chrome/src/content/content.js b/src/chrome/src/content/content.jsindex f0eb39ba0..0836d1707 100644--- a/src/chrome/src/content/content.js+++ b/src/chrome/src/content/content.js@@ -2304,7 +2304,10 @@// or again on the same one (`arrive`), over and enter go to the target.// Documents and shadow roots whose :hover rules this input read (_visionHoverAt).const hoverPrepared = new Set();- const moveOnto = (target, extra = {}, { arrive = false } = {}) => {+ // A press whose pointerdown the page cancelled gets no compatibility+ // mousemove (`compatMouse` off) until its release; the boundary events+ // (over, out, enter, leave) still go out, as the Pointer Events spec has it.+ const moveOnto = (target, extra = {}, { arrive = false, compatMouse = true } = {}) => {// :hover follows the pointer before its events go out, as it does for// a mouse (_visionHoverAt)._visionHoverAt(target, hoverPrepared);@@ -2328,7 +2331,8 @@for (const node of entering) send(node, `${prefix}enter`, { ...extra, bubbles: false, cancelable: false, relatedTarget: related(previous, node) });}}- pointer(target, 'pointermove', extra); mouse(target, 'mousemove', extra);+ pointer(target, 'pointermove', extra);+ if (compatMouse) mouse(target, 'mousemove', extra);_visionPointerOver = target;};const hover = target => moveOnto(target, {}, { arrive: true });@@ -2384,8 +2388,11 @@const steps = Math.min(40, Math.max(2, Math.floor(Number(params.steps) || 12)));hover(el);capture.press();- pointer(el, 'pointerdown', { button: 0, buttons: 1 });- const pressAllowed = mouse(el, 'mousedown', { button: 0, buttons: 1, detail: 1 });+ // A page that cancels the pointerdown gets no compatibility mousedown,+ // mousemove or mouseup for this press, and none of the press's+ // defaults below, as in a browser.+ const pointerAllowed = pointer(el, 'pointerdown', { button: 0, buttons: 1 });+ const pressAllowed = pointerAllowed && mouse(el, 'mousedown', { button: 0, buttons: 1, detail: 1 });// The press focuses what it lands on, as a click's does, and starts// what a person's press would: on a native range input it takes the// thumb (_visionRangeDrag), on selectable text or in a text field a@@ -2445,7 +2452,7 @@}if (htmlStarted) { html.move(last, point); continue; }const holder = capture.target({ button: 0, buttons: 1, ...heldAt(point, px, py) });- moveOnto(holder || last, { button: 0, buttons: 1, ...(holder ? heldAt(point, px, py) : point) });+ moveOnto(holder || last, { button: 0, buttons: 1, ...(holder ? heldAt(point, px, py) : point) }, { compatMouse: pointerAllowed });const local = hit.toDoc(px, py);range?.to(local.x, local.y);selection?.to(local.x, local.y);@@ -2460,10 +2467,10 @@_visionPointerOver = releaseTarget;return { success: true, dispatched: true, method: 'synthetic-vision-drag', target: describe, html5Drag };}- if (releaseTarget !== _visionPointerOver) moveOnto(releaseTarget, { ...endInit, buttons: 1 });+ if (releaseTarget !== _visionPointerOver) moveOnto(releaseTarget, { ...endInit, buttons: 1 }, { compatMouse: pointerAllowed });pointer(releaseTarget, 'pointerup', endInit);capture.release(endInit);- mouse(releaseTarget, 'mouseup', { ...endInit, detail: 1 });+ if (pointerAllowed) mouse(releaseTarget, 'mouseup', { ...endInit, detail: 1 });const rangeValue = range?.release();return { success: true, dispatched: true, method: 'synthetic-vision-drag', target: describe,...(range ? { rangeValue } : {}), ...(selection ? { selectedChars: selection.selectedChars() } : {}) };@@ -2473,11 +2480,18 @@// chooser (a file input, its label, an upload button that clicks one):// the guard blocks it, and the agent settles the guard afterwards.let rangeValue;+ // Whether the last press went through to its mousedown: a double or+ // triple click's selection is that press's default.+ let lastPressAllowed = true;const filePickerGuard = clickWithoutNativeFilePicker(() => {for (let count = 1; count <= clicks; count++) {capture.press();- pointer(el, 'pointerdown', { button, buttons: buttonsMask });- const pressAllowed = mouse(el, 'mousedown', { button, buttons: buttonsMask, detail: count });+ // A page that cancels the pointerdown gets no compatibility+ // mousedown or mouseup for the press, and none of its defaults+ // (focus, the caret, a range thumb); its click still fires.+ const pointerAllowed = pointer(el, 'pointerdown', { button, buttons: buttonsMask });+ const pressAllowed = pointerAllowed && mouse(el, 'mousedown', { button, buttons: buttonsMask, detail: count });+ lastPressAllowed = pressAllowed;// A primary press on a native range input moves its thumb to the// point, as a drag's does (_visionRangeDrag): `input` now, `change`// on release.@@ -2494,7 +2508,7 @@const holder = capture.target({ button, buttons: buttonsMask }) || el;pointer(holder, 'pointerup', { button, buttons: 0 });capture.release({ button, buttons: 0 });- mouse(holder, 'mouseup', { button, buttons: 0, detail: count });+ if (pointerAllowed) mouse(holder, 'mouseup', { button, buttons: 0, detail: count });if (range) rangeValue = range.release();if (button === 0) mouse(el, 'click', { button: 0, buttons: 0, detail: count });else if (button === 1) mouse(el, 'auxclick', { button: 1, buttons: 0, detail: count });@@ -2502,10 +2516,11 @@if (button === 0 && clicks >= 2) {mouse(el, 'dblclick', { button: 0, buttons: 0, detail: 2 });// Browsers select the word under the pointer (two clicks), or its- // line or the whole field (three), as the default action; synthetic- // events need it done. A double click never takes a whole field:- // the typing that usually follows would replace all of it.- try {+ // line or the whole field (three), as the default action of the+ // last press; synthetic events need it done. A double click never+ // takes a whole field: the typing that usually follows would+ // replace all of it.+ if (lastPressAllowed) try {const caret = _visionCaretAt(el.ownerDocument, cx, cy);if (typeof el.setSelectionRange === 'function' && /^(input|textarea)$/.test(tag)) {if (clicks >= 3 && tag === 'input') el.select();diff --git a/src/firefox/src/content/content.js b/src/firefox/src/content/content.jsindex 641ce6286..d9be52316 100644--- a/src/firefox/src/content/content.js+++ b/src/firefox/src/content/content.js@@ -2598,7 +2598,10 @@// or again on the same one (`arrive`), over and enter go to the target.// Documents and shadow roots whose :hover rules this input read (_visionHoverAt).const hoverPrepared = new Set();- const moveOnto = (target, extra = {}, { arrive = false } = {}) => {+ // A press whose pointerdown the page cancelled gets no compatibility+ // mousemove (`compatMouse` off) until its release; the boundary events+ // (over, out, enter, leave) still go out, as the Pointer Events spec has it.+ const moveOnto = (target, extra = {}, { arrive = false, compatMouse = true } = {}) => {// :hover follows the pointer before its events go out, as it does for// a mouse (_visionHoverAt)._visionHoverAt(target, hoverPrepared);@@ -2622,7 +2625,8 @@for (const node of entering) send(node, `${prefix}enter`, { ...extra, bubbles: false, cancelable: false, relatedTarget: related(previous, node) });}}- pointer(target, 'pointermove', extra); mouse(target, 'mousemove', extra);+ pointer(target, 'pointermove', extra);+ if (compatMouse) mouse(target, 'mousemove', extra);_visionPointerOver = target;};const hover = target => moveOnto(target, {}, { arrive: true });@@ -2678,8 +2682,11 @@const steps = Math.min(40, Math.max(2, Math.floor(Number(params.steps) || 12)));hover(el);capture.press();- pointer(el, 'pointerdown', { button: 0, buttons: 1 });- const pressAllowed = mouse(el, 'mousedown', { button: 0, buttons: 1, detail: 1 });+ // A page that cancels the pointerdown gets no compatibility mousedown,+ // mousemove or mouseup for this press, and none of the press's+ // defaults below, as in a browser.+ const pointerAllowed = pointer(el, 'pointerdown', { button: 0, buttons: 1 });+ const pressAllowed = pointerAllowed && mouse(el, 'mousedown', { button: 0, buttons: 1, detail: 1 });// The press focuses what it lands on, as a click's does, and starts// what a person's press would: on a native range input it takes the// thumb (_visionRangeDrag), on selectable text or in a text field a@@ -2739,7 +2746,7 @@}if (htmlStarted) { html.move(last, point); continue; }const holder = capture.target({ button: 0, buttons: 1, ...heldAt(point, px, py) });- moveOnto(holder || last, { button: 0, buttons: 1, ...(holder ? heldAt(point, px, py) : point) });+ moveOnto(holder || last, { button: 0, buttons: 1, ...(holder ? heldAt(point, px, py) : point) }, { compatMouse: pointerAllowed });const local = hit.toDoc(px, py);range?.to(local.x, local.y);selection?.to(local.x, local.y);@@ -2754,10 +2761,10 @@_visionPointerOver = releaseTarget;return { success: true, dispatched: true, method: 'synthetic-vision-drag', target: describe, html5Drag };}- if (releaseTarget !== _visionPointerOver) moveOnto(releaseTarget, { ...endInit, buttons: 1 });+ if (releaseTarget !== _visionPointerOver) moveOnto(releaseTarget, { ...endInit, buttons: 1 }, { compatMouse: pointerAllowed });pointer(releaseTarget, 'pointerup', endInit);capture.release(endInit);- mouse(releaseTarget, 'mouseup', { ...endInit, detail: 1 });+ if (pointerAllowed) mouse(releaseTarget, 'mouseup', { ...endInit, detail: 1 });const rangeValue = range?.release();return { success: true, dispatched: true, method: 'synthetic-vision-drag', target: describe,...(range ? { rangeValue } : {}), ...(selection ? { selectedChars: selection.selectedChars() } : {}) };@@ -2767,11 +2774,18 @@// chooser (a file input, its label, an upload button that clicks one):// the guard blocks it, and the agent settles the guard afterwards.let rangeValue;+ // Whether the last press went through to its mousedown: a double or+ // triple click's selection is that press's default.+ let lastPressAllowed = true;const filePickerGuard = clickWithoutNativeFilePicker(() => {for (let count = 1; count <= clicks; count++) {capture.press();- pointer(el, 'pointerdown', { button, buttons: buttonsMask });- const pressAllowed = mouse(el, 'mousedown', { button, buttons: buttonsMask, detail: count });+ // A page that cancels the pointerdown gets no compatibility+ // mousedown or mouseup for the press, and none of its defaults+ // (focus, the caret, a range thumb); its click still fires.+ const pointerAllowed = pointer(el, 'pointerdown', { button, buttons: buttonsMask });+ const pressAllowed = pointerAllowed && mouse(el, 'mousedown', { button, buttons: buttonsMask, detail: count });+ lastPressAllowed = pressAllowed;// A primary press on a native range input moves its thumb to the// point, as a drag's does (_visionRangeDrag): `input` now, `change`// on release.@@ -2788,7 +2802,7 @@const holder = capture.target({ button, buttons: buttonsMask }) || el;pointer(holder, 'pointerup', { button, buttons: 0 });capture.release({ button, buttons: 0 });- mouse(holder, 'mouseup', { button, buttons: 0, detail: count });+ if (pointerAllowed) mouse(holder, 'mouseup', { button, buttons: 0, detail: count });if (range) rangeValue = range.release();if (button === 0) mouse(el, 'click', { button: 0, buttons: 0, detail: count });else if (button === 1) mouse(el, 'auxclick', { button: 1, buttons: 0, detail: count });@@ -2796,10 +2810,11 @@if (button === 0 && clicks >= 2) {mouse(el, 'dblclick', { button: 0, buttons: 0, detail: 2 });// Browsers select the word under the pointer (two clicks), or its- // line or the whole field (three), as the default action; synthetic- // events need it done. A double click never takes a whole field:- // the typing that usually follows would replace all of it.- try {+ // line or the whole field (three), as the default action of the+ // last press; synthetic events need it done. A double click never+ // takes a whole field: the typing that usually follows would+ // replace all of it.+ if (lastPressAllowed) try {const caret = _visionCaretAt(el.ownerDocument, cx, cy);if (typeof el.setSelectionRange === 'function' && /^(input|textarea)$/.test(tag)) {if (clicks >= 3 && tag === 'input') el.select();diff --git a/test/vision-pipeline-browser.mjs b/test/vision-pipeline-browser.mjsindex 4225b9d71..0d5ce6124 100644--- a/test/vision-pipeline-browser.mjs+++ b/test/vision-pipeline-browser.mjs@@ -226,6 +226,80 @@ test('content-script vision input (Firefox path) clicks, double-clicks, focuses,}});+test('content-script vision input (Firefox path) honours a pointerdown the page cancels, as a browser does', async () => {+ const browser = await chromium.launch({ channel: 'chromium' });+ try {+ const page = await browser.newPage({ viewport: { width: 1000, height: 700 } });+ await page.setContent(`<!doctype html><style>body{margin:0;font:20px monospace}+ #range{position:absolute;left:20px;top:20px;width:300px;height:20px;margin:0}+ #field{position:absolute;left:20px;top:80px;width:300px;height:30px}+ #words{position:absolute;left:20px;top:150px;margin:0;white-space:nowrap}+ #pad{position:absolute;left:20px;top:220px;width:400px;height:60px;background:#ddd}</style>+ <input id="range" type="range" min="0" max="100" step="5" value="10"><input id="field" value="hello">+ <p id="words">alpha bravo charlie</p><div id="pad"></div>+ <script>window.log=[];window.cancel=new Set(['range','field','words','pad']);+ document.addEventListener('pointerdown',e=>{if(cancel.has(e.target.id))e.preventDefault();},true);+ for(const t of ['pointerdown','mousedown','pointermove','mousemove','mouseover','pointerup','mouseup','click','dblclick','input','change'])+ document.addEventListener(t,e=>{if(e.target.id)log.push(t+':'+e.target.id)},true);</script>`);+ const source = fs.readFileSync(path.join(ROOT, 'src/firefox/src/content/content.js'), 'utf8');+ const block = source.slice(source.indexOf(' function _visionTargetAt('), source.indexOf(' let _lastClickIdent = null;'));+ const guardBlock = source.slice(source.indexOf(' function uniqueFileInputSelector(input) {'), source.indexOf(' /**\n * Click an element by selector or coordinates.'));+ await page.evaluate(({ code, guardBlock }) => {+ // eslint-disable-next-line no-new-func+ window.__visionInput = new Function(`function withLocalPageDispatch(callback) { return callback(); }+ function _uniqueFileInputSelector(input) { return input.id ? '#' + input.id : ''; }+ function _deepSelectorMatches(selector) { return [...document.querySelectorAll(selector)]; }+ ${guardBlock}\n${code}\nreturn visionInput;`)();+ }, { code: block, guardBlock });+ const run = params => page.evaluate(p => window.__visionInput(p), params);+ const take = () => page.evaluate(() => window.log.splice(0));+ const bravo = await page.evaluate(() => { const text = document.getElementById('words').firstChild; const r = document.createRange();+ r.setStart(text, 6); r.setEnd(text, 11); const b = r.getBoundingClientRect(); return { x: (b.left + b.right) / 2, y: (b.top + b.bottom) / 2 }; });++ // A click on a range track: no thumb moves and no compatibility mouse+ // events, but the click still fires.+ let result = await run({ type: 'click', x: 170, y: 30 });+ assert.equal(result.success, true);+ assert.equal(result.rangeValue, undefined);+ assert.equal(await page.evaluate(() => document.getElementById('range').value), '10');+ let log = await take();+ assert.ok(log.includes('pointerdown:range') && log.includes('pointerup:range') && log.includes('click:range'), log.join());+ assert.ok(!log.some(entry => /^(mousedown|mouseup|input|change):/.test(entry)), log.join());+ // A click in a text field: no focus.+ await run({ type: 'click', x: 100, y: 95 });+ assert.notEqual(await page.evaluate(() => document.activeElement.id), 'field');+ await take();+ // A double click on a word: dblclick fires, but no word is selected.+ await page.evaluate(() => getSelection().removeAllRanges());+ result = await run({ type: 'click', x: bravo.x, y: bravo.y, clickCount: 2 });+ log = await take();+ assert.ok(log.includes('dblclick:words'), log.join());+ assert.equal(await page.evaluate(() => String(getSelection())), '');+ // A drag: pointer moves and the release go out, no mousemove or mouseup,+ // the boundary mouseover still does, and no text is selected.+ result = await run({ type: 'drag', x: 30, y: 250, toX: 380, toY: 250, steps: 4 });+ log = await take();+ assert.equal(result.success, true);+ const pressed = log.slice(log.indexOf('pointerdown:pad'));+ assert.ok(pressed.filter(entry => entry === 'pointermove:pad').length >= 4, log.join());+ assert.ok(pressed.includes('pointerup:pad'), log.join());+ assert.ok(!pressed.some(entry => /^(mousedown|mousemove|mouseup):pad$/.test(entry)), log.join());+ // The same sequences on presses the page lets through are unchanged.+ await page.evaluate(() => window.cancel.clear());+ result = await run({ type: 'click', x: 170, y: 30 });+ assert.equal(result.rangeValue, '50');+ log = await take();+ assert.ok(log.includes('mousedown:range') && log.includes('mouseup:range'), log.join());+ result = await run({ type: 'drag', x: 30, y: 250, toX: 380, toY: 250, steps: 4 });+ log = await take();+ assert.ok(log.includes('mousemove:pad') && log.includes('mouseup:pad'), log.join());+ await run({ type: 'click', x: bravo.x, y: bravo.y, clickCount: 2 });+ assert.equal(await page.evaluate(() => String(getSelection())), 'bravo');+ } finally {+ await browser.close();+ }+});+test('content-script vision drag (Firefox path) moves and releases in a same-origin frame\'s own coordinates', async () => {const browser = await chromium.launch({ channel: 'chromium' });try {