Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Take the PDF viewer's new window from the tab-attached event

Claude3b9e687ebc25
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ed2c6c85a..9ee6162f6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,7 +10,7 @@ This changelog was generated from the repository Git history and release tags. V
- Selecting text on a page now shows a small bar, **Translate | Explain | Proofread | More…**, instead of the single `?` button. Translate, Explain and Proofread answer in a card next to the selection: Translate shows the detected source language, the original text and a target-language picker, and every card has an "Ask a follow-up" box that, once the answer is there, continues in the side panel with the card's answer as context. **More…** opens the same popup the `?` button opened (Summarize, Quiz me, Humanize, a question box and Hide this; Translate, Explain and Proofread are on the bar, so they are not repeated there). A card sends the selected text to your configured model provider, like any other selection action; selections over 4,000 characters go to the side panel instead of a card. A card answer that reaches the model's output limit says it was cut off and offers to continue in the side panel, which picks up from the unfinished text instead of starting over (a side-panel reply that is cut off again says so, including when an OpenAI Responses model reports the limit as an error, in which case the text it wrote is kept and the call still counts toward your cloud-cost allowance), and closing or redoing a card, or reloading or leaving the page, cancels the request it no longer needs. Chrome and Firefox.
### Fixed
-- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now.
+- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now, taking the new window from the tab-attached event rather than from a lookup a quick click could outrun.
- Stopping a task while WebBrain was waiting for page navigation to settle now stops it immediately. The pending settle step previously kept waiting and could let the run continue after you chose Stop.
- Mastodon: replying to toots in a narrow window no longer posts the same reply several times and stops before the next toot. There, a toot's Reply button opens the composer at /publish, and after posting Mastodon goes back to the previous column by itself. WebBrain took both page changes for something you did, so it reported each click's outcome as unknown; the agent then concluded its reply had not been posted, wrote it again, and never reached the next toot. In Chrome, a page's own Back is now told apart from your browser's Back button: the page a click opens counts as that click's verified effect, and a click after which the page goes back by itself counts as delivered, to be checked on the page. The Mastodon notes now cover replies (start with the prefilled @mentions once, check the reply under the toot before any retry) and stay active on Favourites, Bookmarks and the other columns. Firefox still treats a page's own Back as yours until its Back button reporting is verified.
- Finishing a form you asked WebBrain not to submit no longer takes most of a minute. The completion check removed every line that mentioned a password, so "fill out this form with a random password … don't submit" reached its judge as "[credential omitted]", and so did the approved plan and the agent's summary. Seeing only an unsubmitted signup form, the judge answered "pending" again and again; in the trace it took four attempts and about 50 seconds, ending only when a slow screenshot check ran out of time. The active model's check, which already holds these texts in the conversation, now receives every line of them, with only addresses shortened and bearer tokens and entered passwords hidden, as in all judge evidence. A decision model on this device, or WebBrain Cloud's own while WebBrain Cloud is also your provider, receives that same copy; any other decision model keeps whole-line redaction, and when that leaves it without the task, it is no longer asked while the active model can judge. A password WebBrain typed into a password field during the run is now hidden in the text judges receive, such as the agent's summary or a field a "show password" control revealed, unless the typing tool could not tell what kind of field it was; other entered values stay visible, so a judge can still catch a mistyped one. The judges are told that a task asking not to submit is complete once the staged values match, that a hidden value neither proves nor disproves an entry, and, for each recorded entry, whether its field read back what was entered, whether it is a credential field and which field it targeted; a credential field's line in the page evidence keeps its role and reference. Staged field edits are judged from the accessibility tree first: a screenshot shows passwords only as dots, and the focused field's blinking caret made each one stale before its verdict returned.
diff --git a/docs/architecture.md b/docs/architecture.md
index 96c15a2ed..b4dff819f 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -563,8 +563,10 @@ A panel the browser carries to another window (the tab dragged out) keeps its do
notices on its next live `windows.getCurrent()` lookup and the reporter (`setWindowId()`) names the new
window, so the background does not treat the old window as occupied and the new one as empty.
The PDF viewer does the same for the window id it sends with a selection: `pdf-viewer-window.js`
-re-reads `windows.getCurrent()` when its tab is attached, when the window gains focus and at the start
-of every pointer gesture, and the selection script reads those fields when it sends, not once at load.
+takes the new window straight from `tabs.onAttached` for its own tab (no lookup to race a click that
+follows the drop, and a lookup begun before the move is discarded), re-reads `windows.getCurrent()`
+when the window gains focus and at the start of every pointer gesture as a backstop, and the selection
+script reads those fields when it sends, not once at load.
A selection made in the bundled PDF viewer, whose message carries only a tab id, brings the
window too (Chrome's `sender.tab` for that tab, else the window the viewer reported from
`windows.getCurrent()`), so the same check applies there; that path opens the panel once in the
diff --git a/src/chrome/src/ui/pdf-handler.js b/src/chrome/src/ui/pdf-handler.js
index 28649c637..f75922592 100644
--- a/src/chrome/src/ui/pdf-handler.js
+++ b/src/chrome/src/ui/pdf-handler.js
@@ -702,7 +702,7 @@ async function initialize() {
tabId: streamInfo.tabId,
originalUrl: String(streamInfo.originalUrl || ''),
};
- await trackViewerWindow({ api, target: window, fields: submitFields }).refresh();
+ await trackViewerWindow({ api, target: window, fields: submitFields, tabId: streamInfo.tabId }).refresh();
globalThis.__webbrainSelectionShortcutConfig = {
submitMessage: 'WB_PDF_SELECTION_SHORTCUT_SUBMIT',
submitFields,
diff --git a/src/chrome/src/ui/pdf-viewer-window.js b/src/chrome/src/ui/pdf-viewer-window.js
index bb976d46e..470f19bfa 100644
--- a/src/chrome/src/ui/pdf-viewer-window.js
+++ b/src/chrome/src/ui/pdf-viewer-window.js
@@ -3,23 +3,35 @@
*
* The background uses it to tell whether WebBrain's side panel is already showing in
* the viewer's window (re-opening an open panel closes it). A viewer tab can be
- * dragged into another window after it started, so the id is read again whenever the
- * tab is attached somewhere, the window gains focus, and at the start of each pointer
- * gesture, which is always ahead of the selection it would be sent with.
+ * dragged into another window after it started. When the browser says the tab was
+ * attached to a window, that window is taken at once, with no lookup to wait for: the
+ * page lives in the tab, so it is where the tab went. The window is also read again
+ * when it gains focus and at the start of each pointer gesture, which settles any
+ * case the attach event did not cover.
*/
-export function trackViewerWindow({ api, target, fields }) {
+export function trackViewerWindow({ api, target, fields, tabId = null }) {
let latest = 0;
async function refresh() {
const request = ++latest;
try {
const id = (await api.windows?.getCurrent?.())?.id;
- // A slower, older lookup must not overwrite a newer answer.
+ // A lookup that began before a newer answer (a refresh, or a move) is stale.
if (request === latest && Number.isInteger(id)) fields.windowId = id;
} catch { /* keep the last id we confirmed */ }
}
- api.tabs?.onAttached?.addListener?.(refresh);
+ function handleAttached(attachedTabId, attachInfo) {
+ if (tabId == null) {
+ void refresh(); // not told which tab is ours, so ask where the page is
+ return;
+ }
+ if (String(attachedTabId) !== String(tabId) || !Number.isInteger(attachInfo?.newWindowId)) return;
+ latest += 1; // a lookup already under way describes the window the tab left
+ fields.windowId = attachInfo.newWindowId;
+ }
+
+ api.tabs?.onAttached?.addListener?.(handleAttached);
target.addEventListener('focus', refresh);
target.addEventListener('pointerdown', refresh, true);
return { refresh };
diff --git a/test/run.js b/test/run.js
index 828944218..7e453d7a0 100644
--- a/test/run.js
+++ b/test/run.js
@@ -52993,7 +52993,7 @@ test('a PDF viewer selection leaves a side panel that is already showing in its
assert.equal(JSON.stringify(queued[0].options), '{"panelOpened":true}', 'and told the panel was already opened');
assert.equal(queued[0].tab.id, 7);
const viewer = fs.readFileSync(path.join(ROOT, 'src/chrome/src/ui/pdf-handler.js'), 'utf8');
- assert.match(viewer, /const submitFields = \{\s*tabId: streamInfo\.tabId,\s*originalUrl: String\(streamInfo\.originalUrl \|\| ''\),\s*\};\s*await trackViewerWindow\(\{ api, target: window, fields: submitFields \}\)\.refresh\(\);[\s\S]*?submitFields,\s*allowNestedFrame: true,/,
+ assert.match(viewer, /const submitFields = \{\s*tabId: streamInfo\.tabId,\s*originalUrl: String\(streamInfo\.originalUrl \|\| ''\),\s*\};\s*await trackViewerWindow\(\{ api, target: window, fields: submitFields, tabId: streamInfo\.tabId \}\)\.refresh\(\);[\s\S]*?submitFields,\s*allowNestedFrame: true,/,
'the PDF viewer must report the window it is in, and keep that current');
const shortcut = fs.readFileSync(path.join(ROOT, 'src/chrome/src/content/selection-shortcut.js'), 'utf8');
assert.match(shortcut, /const currentSubmitFields = \(\) => shortcutConfig\?\.submitFields[\s\S]*?\? \{ \.\.\.shortcutConfig\.submitFields \}/,
@@ -53024,15 +53024,30 @@ test('the PDF viewer follows its tab into another window before a selection is s
};
const target = { addEventListener: (type, callback, capture) => domListeners.push({ type, callback, capture }) };
const fields = { tabId: 73 };
- const tracker = trackViewerWindow({ api, target, fields });
+ const tracker = trackViewerWindow({ api, target, fields, tabId: 73 });
await tracker.refresh();
assert.equal(fields.windowId, 5, 'the window the viewer starts in');
- // A tab dragged into another window is found when it is attached.
+ // A tab dragged into another window is taken from the attach event itself, at once:
+ // a click right after the drop must not wait for a lookup.
liveWindowId = 9;
attachListeners[0](73, { newWindowId: 9 });
- await new Promise(resolve => setImmediate(resolve));
- assert.equal(fields.windowId, 9, 'attaching the tab to another window updates the id');
+ assert.equal(fields.windowId, 9, 'attaching the tab to another window updates the id synchronously');
+ attachListeners[0](74, { newWindowId: 30 });
+ attachListeners[0](73, { newWindowId: undefined });
+ attachListeners[0](73, { newWindowId: '31' });
+ assert.equal(fields.windowId, 9, 'another tab, or an attach without a usable window, changes nothing');
+
+ // A lookup already under way when the tab moved describes the window it left.
+ let releaseStale;
+ gates.push(new Promise(resolve => { releaseStale = resolve; }));
+ liveWindowId = 5;
+ const stale = tracker.refresh();
+ attachListeners[0](73, { newWindowId: 9 });
+ releaseStale();
+ await stale;
+ assert.equal(fields.windowId, 9, 'a lookup that began before the move cannot undo it');
+ liveWindowId = 9;
// Window focus and the start of a pointer gesture refresh it too.
liveWindowId = 12;
@@ -53070,6 +53085,21 @@ test('the PDF viewer follows its tab into another window before a selection is s
const bare = { tabId: 1 };
await trackViewerWindow({ api: {}, target: { addEventListener() {} }, fields: bare }).refresh();
assert.deepEqual(bare, { tabId: 1 });
+
+ // Not told which tab is its own, it asks where the page is instead of guessing.
+ const unknown = {};
+ const unknownListeners = [];
+ trackViewerWindow({
+ api: {
+ windows: { getCurrent: async () => ({ id: 44 }) },
+ tabs: { onAttached: { addListener: callback => unknownListeners.push(callback) } },
+ },
+ target: { addEventListener() {} },
+ fields: unknown,
+ });
+ unknownListeners[0](3, { newWindowId: 99 });
+ await new Promise(resolve => setImmediate(resolve));
+ assert.equal(unknown.windowId, 44);
});
test('WebGPU provider drops a request whose caller aborts while the offscreen document is starting', async () => {