Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Open the PDF side panel once, and cancel a card's model call when its page unloads

Claude5fe5285dc6cd
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 13175c7a6..c0656fecd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,10 +7,10 @@ This changelog was generated from the repository Git history and release tags. V
## [Unreleased]
### Changed
-- Selecting text on a page now shows a small bar, **Translate | Explain | Proofread | More…**, instead of the single `?` button. Translate, Explain and Proofread answer in a card next to the selection: Translate shows the detected source language, the original text and a target-language picker, and every card has an "Ask a follow-up" box that, once the answer is there, continues in the side panel with the card's answer as context. **More…** opens the same popup the `?` button opened (Summarize, Quiz me, Humanize, a question box and Hide this; Translate, Explain and Proofread are on the bar, so they are not repeated there). A card sends the selected text to your configured model provider, like any other selection action; selections over 4,000 characters go to the side panel instead of a card. A card answer that reaches the model's output limit says it was cut off and offers to continue in the side panel, which picks up from the unfinished text instead of starting over (a side-panel reply that is cut off again says so, including when an OpenAI Responses model reports the limit as an error, in which case the text it wrote is kept), and closing or redoing a card cancels the request it no longer needs. Chrome and Firefox.
+- Selecting text on a page now shows a small bar, **Translate | Explain | Proofread | More…**, instead of the single `?` button. Translate, Explain and Proofread answer in a card next to the selection: Translate shows the detected source language, the original text and a target-language picker, and every card has an "Ask a follow-up" box that, once the answer is there, continues in the side panel with the card's answer as context. **More…** opens the same popup the `?` button opened (Summarize, Quiz me, Humanize, a question box and Hide this; Translate, Explain and Proofread are on the bar, so they are not repeated there). A card sends the selected text to your configured model provider, like any other selection action; selections over 4,000 characters go to the side panel instead of a card. A card answer that reaches the model's output limit says it was cut off and offers to continue in the side panel, which picks up from the unfinished text instead of starting over (a side-panel reply that is cut off again says so, including when an OpenAI Responses model reports the limit as an error, in which case the text it wrote is kept), and closing or redoing a card, or reloading or leaving the page, cancels the request it no longer needs. Chrome and Firefox.
### Fixed
-- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer.
+- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice.
- Stopping a task while WebBrain was waiting for page navigation to settle now stops it immediately. The pending settle step previously kept waiting and could let the run continue after you chose Stop.
- Mastodon: replying to toots in a narrow window no longer posts the same reply several times and stops before the next toot. There, a toot's Reply button opens the composer at /publish, and after posting Mastodon goes back to the previous column by itself. WebBrain took both page changes for something you did, so it reported each click's outcome as unknown; the agent then concluded its reply had not been posted, wrote it again, and never reached the next toot. In Chrome, a page's own Back is now told apart from your browser's Back button: the page a click opens counts as that click's verified effect, and a click after which the page goes back by itself counts as delivered, to be checked on the page. The Mastodon notes now cover replies (start with the prefilled @mentions once, check the reply under the toot before any retry) and stay active on Favourites, Bookmarks and the other columns. Firefox still treats a page's own Back as yours until its Back button reporting is verified.
- Finishing a form you asked WebBrain not to submit no longer takes most of a minute. The completion check removed every line that mentioned a password, so "fill out this form with a random password … don't submit" reached its judge as "[credential omitted]", and so did the approved plan and the agent's summary. Seeing only an unsubmitted signup form, the judge answered "pending" again and again; in the trace it took four attempts and about 50 seconds, ending only when a slow screenshot check ran out of time. The active model's check, which already holds these texts in the conversation, now receives every line of them, with only addresses shortened and bearer tokens and entered passwords hidden, as in all judge evidence. A decision model on this device, or WebBrain Cloud's own while WebBrain Cloud is also your provider, receives that same copy; any other decision model keeps whole-line redaction, and when that leaves it without the task, it is no longer asked while the active model can judge. A password WebBrain typed into a password field during the run is now hidden in the text judges receive, such as the agent's summary or a field a "show password" control revealed, unless the typing tool could not tell what kind of field it was; other entered values stay visible, so a judge can still catch a mistyped one. The judges are told that a task asking not to submit is complete once the staged values match, that a hidden value neither proves nor disproves an entry, and, for each recorded entry, whether its field read back what was entered, whether it is a credential field and which field it targeted; a credential field's line in the page evidence keeps its role and reference. Staged field edits are judged from the accessibility tree first: a screenshot shows passwords only as dots, and the focused field's blinking caret made each one stale before its verdict returned.
diff --git a/docs/architecture.md b/docs/architecture.md
index 261e56de2..285672cc7 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -535,7 +535,8 @@ reports `outputTruncated`, or OpenAI Responses raises the limit as an error carr
`partialContent`, whose text is then shown), so a long remainder never passes as complete
and can be continued by asking. Each call carries a job id, and a card
that is closed, retried or retargeted sends `WB_SELECTION_SHORTCUT_INLINE_CANCEL` so the
-background aborts the paid request it no longer needs (for the in-browser WebGPU models the
+background aborts the paid request it no longer needs (a page that unloads mid-answer sends the
+same cancel from `pagehide`, since it can no longer close the card itself; for the in-browser WebGPU models the
provider also sends `webgpu-chat-cancel`, and the offscreen worker interrupts the running
generation or drops a queued one, and the offscreen host remembers a cancel that arrives while the request is still being routed to a worker, so the generation never starts; those workers report `outputTruncated` when an answer used
its token budget, since they have no finish reason); the card also claims its request id
@@ -552,7 +553,9 @@ was dropped by a background restart reconnects at once (only a port that is refu
away backs off), so the restarted worker is not left unaware of a panel that is still showing.
A selection made in the bundled PDF viewer, whose message carries only a tab id, brings the
window too (Chrome's `sender.tab` for that tab, else the window the viewer reported from
-`windows.getCurrent()`), so the same check applies there. The card's
+`windows.getCurrent()`), so the same check applies there; that path opens the panel once in the
+click and queues the verified prompt with `panelOpened`, because a panel that is still starting
+up has not reported in yet and a second `open()` would close it. The card's
answer travels inside the selection's untrusted boundary, labelled as an earlier answer that
may contain mistakes and
capped at 6,000 characters, so model output derived from page text never becomes trusted
diff --git a/src/chrome/src/background.js b/src/chrome/src/background.js
index 2a614f8c8..31951f075 100644
--- a/src/chrome/src/background.js
+++ b/src/chrome/src/background.js
@@ -1669,15 +1669,17 @@ function notifySidePanelOfContextMenuPrompt(payload) {
pendingContextMenuNotifications.set(tabId, timerId);
}
-function openSidePanelForContextMenu(tab) {
+function openSidePanelForContextMenu(tab, { open = true } = {}) {
if (!tab?.id) return;
panelTabs.add(tab.id);
savePanelTabs();
// A panel that is already showing in this window needs nothing here: callers hand it
// the prompt by message. Running setOptions + open again on an open panel makes
// Chrome tear it down, which is how a follow-up used to close the sidebar. This check
- // must stay synchronous: open() loses the user's gesture across an await.
- if (!sidePanelPresence.isVisibleInWindow(tab.windowId)) {
+ // must stay synchronous: open() loses the user's gesture across an await. A caller
+ // that has just opened the panel itself passes open: false, since a panel that is
+ // still starting up has not reported in yet and would be opened a second time.
+ if (open && !sidePanelPresence.isVisibleInWindow(tab.windowId)) {
chrome.sidePanel.setOptions({
tabId: tab.id,
path: 'src/ui/sidepanel.html',
@@ -1825,7 +1827,7 @@ chrome.runtime.onMessage.addListener((msg, _sender, sendResponse) => {
// Selection-shortcut clicks originate in a content script. Keep this listener
// synchronous until sidePanel.open() so Chrome preserves the originating user
// gesture; prompt recovery storage can finish afterward.
-function queueSelectionShortcutPrompt(msg, tab, sendResponse) {
+function queueSelectionShortcutPrompt(msg, tab, sendResponse, { panelOpened = false } = {}) {
const selectionAction = normalizeSelectionAction(msg.action);
const includePageContext = selectionAction === 'custom' && msg.includePageContext === true;
const sourceGrounding = includePageContext
@@ -1868,7 +1870,7 @@ function queueSelectionShortcutPrompt(msg, tab, sendResponse) {
createdAt: Date.now(),
};
- openSidePanelForContextMenu(tab);
+ openSidePanelForContextMenu(tab, { open: !panelOpened });
(async () => {
try {
await contextMenuStorage.save(tab.id, payload);
@@ -1914,7 +1916,7 @@ chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {
openSidePanelForContextMenu(windowId === undefined ? { id: tabId } : { id: tabId, windowId });
} catch {}
chrome.tabs.get(tabId)
- .then(tab => queueSelectionShortcutPrompt(msg, tab, sendResponse))
+ .then(tab => queueSelectionShortcutPrompt(msg, tab, sendResponse, { panelOpened: true }))
.catch(error => sendResponse({
ok: false,
queued: false,
diff --git a/src/chrome/src/content/selection-shortcut.js b/src/chrome/src/content/selection-shortcut.js
index a599593af..3fbe4e8ed 100644
--- a/src/chrome/src/content/selection-shortcut.js
+++ b/src/chrome/src/content/selection-shortcut.js
@@ -1195,6 +1195,11 @@
window.addEventListener('keypress', containSurfaceKeyboardEvent, true);
window.addEventListener('keyup', containSurfaceKeyboardEvent, true);
window.addEventListener('blur', () => surfacePressedKeys.clear());
+ // A reload, navigation or closed tab ends this page without closing the card, so the
+ // model call it is waiting on would run to its timeout for an answer nobody can see.
+ window.addEventListener('pagehide', () => {
+ if (cardJobId) dismissSurface();
+ });
api.runtime.onMessage.addListener((message) => {
if (message?.type === 'WB_HIDE_FOR_TOOL_USE') {
diff --git a/src/firefox/src/content/selection-shortcut.js b/src/firefox/src/content/selection-shortcut.js
index a599593af..3fbe4e8ed 100644
--- a/src/firefox/src/content/selection-shortcut.js
+++ b/src/firefox/src/content/selection-shortcut.js
@@ -1195,6 +1195,11 @@
window.addEventListener('keypress', containSurfaceKeyboardEvent, true);
window.addEventListener('keyup', containSurfaceKeyboardEvent, true);
window.addEventListener('blur', () => surfacePressedKeys.clear());
+ // A reload, navigation or closed tab ends this page without closing the card, so the
+ // model call it is waiting on would run to its timeout for an answer nobody can see.
+ window.addEventListener('pagehide', () => {
+ if (cardJobId) dismissSurface();
+ });
api.runtime.onMessage.addListener((message) => {
if (message?.type === 'WB_HIDE_FOR_TOOL_USE') {
diff --git a/test/fixtures/run.mjs b/test/fixtures/run.mjs
index 84cfccaa7..3888d0b5d 100644
--- a/test/fixtures/run.mjs
+++ b/test/fixtures/run.mjs
@@ -2405,6 +2405,29 @@ for (const [label, sourcePath, manualOpen] of [
await page.waitForTimeout(60);
const cancels = await page.evaluate(() => window.__selectionCancelMessages.length);
if (cancels !== 3) throw new Error(`closing a finished card should not send a cancel: ${cancels}`);
+
+ // A page that unloads mid-flight (reload, navigation, closed tab) never closes its card,
+ // so the unload itself has to cancel the call. An idle page has nothing to cancel.
+ await page.evaluate(() => {
+ window.__selectionInlineHandler = () => new Promise(() => {});
+ });
+ const before = await page.evaluate(() => window.__selectionInlineMessages.length);
+ await openInlineCard(page, 'explain');
+ await page.waitForFunction((count) => window.__selectionInlineMessages.length === count + 1, before);
+ await page.evaluate(() => window.dispatchEvent(new PageTransitionEvent('pagehide', { persisted: false })));
+ await page.waitForFunction(() => window.__selectionCancelMessages.length === 4);
+ const unloaded = await page.evaluate(() => ({
+ cancelled: window.__selectionCancelMessages.at(-1).jobId,
+ running: window.__selectionInlineMessages.at(-1).jobId,
+ visible: window.__webbrainSelectionShortcut.getState().cardVisible,
+ }));
+ if (unloaded.cancelled !== unloaded.running || unloaded.visible) {
+ throw new Error(`unloading should cancel the running call and hide the card: ${JSON.stringify(unloaded)}`);
+ }
+ await page.evaluate(() => window.dispatchEvent(new PageTransitionEvent('pagehide', { persisted: true })));
+ await page.waitForTimeout(60);
+ const afterIdle = await page.evaluate(() => window.__selectionCancelMessages.length);
+ if (afterIdle !== 4) throw new Error(`unloading a page with no call in flight should not send a cancel: ${afterIdle}`);
});
test(`${label}: a cut-off card answer says so and continues in the side panel`, async (page) => {
diff --git a/test/run.js b/test/run.js
index 1b0c2db78..76709928b 100644
--- a/test/run.js
+++ b/test/run.js
@@ -52720,7 +52720,7 @@ test('side panel presence: the background learns which windows show a panel, and
test('background does not re-open a side panel that is already showing, and still opens one that is not', () => {
const background = fs.readFileSync(path.join(ROOT, 'src/chrome/src/background.js'), 'utf8');
- const start = background.indexOf('function openSidePanelForContextMenu(tab) {');
+ const start = background.indexOf('function openSidePanelForContextMenu(tab, { open = true } = {}) {');
const end = background.indexOf('\nasync function handleContextMenuAsk', start);
assert.ok(start > 0 && end > start, 'openSidePanelForContextMenu should be found');
const source = background.slice(start, end);
@@ -52755,6 +52755,25 @@ test('background does not re-open a side panel that is already showing, and stil
assert.deepEqual(run(4, { id: 12 }),
['save', ['setOptions', 12], ['open', 12], 'group'],
'a tab with no window id keeps opening');
+ const runWithoutOpening = (visibleWindowId, tab) => {
+ const calls = [];
+ const context = vm.createContext({
+ panelTabs: new Set(),
+ savePanelTabs: () => calls.push('save'),
+ ensureWebBrainGroup: () => { calls.push('group'); return Promise.resolve(); },
+ sidePanelPresence: { isVisibleInWindow: windowId => windowId === visibleWindowId },
+ chrome: {
+ sidePanel: {
+ setOptions: options => calls.push(['setOptions', options.tabId]),
+ open: options => calls.push(['open', options.tabId]),
+ },
+ },
+ });
+ vm.runInContext(`${source}\nopenSidePanelForContextMenu(tab, { open: false });`, Object.assign(context, { tab }));
+ return calls;
+ };
+ assert.deepEqual(runWithoutOpening(9, { id: 11, windowId: 4 }), ['save', 'group'],
+ 'a caller that has just opened the panel itself does not open it a second time, even before the panel has reported in');
assert.match(background, /chrome\.runtime\.onConnect\.addListener\(\(port\) => \{ sidePanelPresence\.attach\(port\); \}\);/,
'the presence port must be accepted at the top level of the worker');
@@ -52763,15 +52782,16 @@ test('background does not re-open a side panel that is already showing, and stil
'the side panel must tell the background when it is on screen');
});
-test('a PDF viewer selection leaves a side panel that is already showing in its window alone', () => {
+test('a PDF viewer selection leaves a side panel that is already showing in its window alone', async () => {
const background = fs.readFileSync(path.join(ROOT, 'src/chrome/src/background.js'), 'utf8');
- const helperStart = background.indexOf('function openSidePanelForContextMenu(tab) {');
+ const helperStart = background.indexOf('function openSidePanelForContextMenu(tab, { open = true } = {}) {');
const helperEnd = background.indexOf('\nasync function handleContextMenuAsk', helperStart);
const listenerStart = background.indexOf("chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {\n if (msg?.type !== 'WB_PDF_SELECTION_SHORTCUT_SUBMIT') return;");
const listenerEnd = background.indexOf('\n// (See the panel visibility comment above', listenerStart);
assert.ok(helperStart > 0 && helperEnd > helperStart && listenerStart > 0 && listenerEnd > listenerStart,
'the helper and the PDF listener should be found');
+ const queued = [];
const run = ({ visibleWindowId, message, sender }) => {
const calls = [];
let listener = null;
@@ -52780,14 +52800,14 @@ test('a PDF viewer selection leaves a side panel that is already showing in its
savePanelTabs: () => {},
ensureWebBrainGroup: () => Promise.resolve(),
isPdfHandlerSender: () => true,
- queueSelectionShortcutPrompt: () => {},
+ queueSelectionShortcutPrompt: (msg, tab, sendResponse, options) => queued.push({ tab, options }),
sidePanelPresence: { isVisibleInWindow: windowId => windowId === visibleWindowId },
chrome: {
sidePanel: {
setOptions: options => calls.push(['setOptions', options.tabId]),
open: options => calls.push(['open', options.tabId]),
},
- tabs: { get: () => new Promise(() => {}) },
+ tabs: { get: async id => ({ id, windowId: 4 }) },
runtime: { onMessage: { addListener: callback => { listener = callback; } } },
},
});
@@ -52815,9 +52835,19 @@ test('a PDF viewer selection leaves a side panel that is already showing in its
assert.deepEqual(run({ visibleWindowId: 4, message: { windowId: '4' }, sender: {} }), opened,
'a window id that is not a number is ignored');
- const viewer = fs.readFileSync(path.join(ROOT, 'src/chrome/src/ui/pdf-handler.js'), 'utf8');
- assert.match(viewer, /windowId = \(await api\.windows\?\.getCurrent\?\.\(\)\)\?\.id \?\? null;[\s\S]*?submitFields: \{[\s\S]*?\.\.\.\(Number\.isInteger\(windowId\) \? \{ windowId \} : \{\}\),/,
- 'the PDF viewer must report the window it is in');
+ // The live tab is verified after the panel was opened here; the queueing step must not open it again.
+ await new Promise(resolve => setTimeout(resolve, 0));
+ queued.length = 0;
+ run({ visibleWindowId: 9, message: { windowId: 4 }, sender: {} });
+ await new Promise(resolve => setTimeout(resolve, 0));
+ {
+ assert.equal(queued.length, 1, 'the verified tab is queued');
+ assert.equal(JSON.stringify(queued[0].options), '{"panelOpened":true}', 'and told the panel was already opened');
+ assert.equal(queued[0].tab.id, 7);
+ const viewer = fs.readFileSync(path.join(ROOT, 'src/chrome/src/ui/pdf-handler.js'), 'utf8');
+ assert.match(viewer, /windowId = \(await api\.windows\?\.getCurrent\?\.\(\)\)\?\.id \?\? null;[\s\S]*?submitFields: \{[\s\S]*?\.\.\.\(Number\.isInteger\(windowId\) \? \{ windowId \} : \{\}\),/,
+ 'the PDF viewer must report the window it is in');
+ }
});
test('WebGPU provider drops a request whose caller aborts while the offscreen document is starting', async () => {
@@ -54522,7 +54552,7 @@ test('selection shortcut is shipped, enabled by default, and keeps browser-speci
const chromeEnd = chromeBg.indexOf('// (See the panel visibility comment', chromeStart);
const chromePromptStart = chromeBg.indexOf('function queueSelectionShortcutPrompt(');
const chromeHandler = chromeBg.slice(chromePromptStart, chromeEnd);
- const chromeOpen = chromeHandler.indexOf('openSidePanelForContextMenu(tab);');
+ const chromeOpen = chromeHandler.indexOf('openSidePanelForContextMenu(tab, { open: !panelOpened });');
const chromeSave = chromeHandler.indexOf('await contextMenuStorage.save(tab.id, payload);');
assert.notEqual(chromeStart, -1, 'chrome: selection shortcut listener missing');
assert.notEqual(chromePromptStart, -1, 'chrome: shared selection prompt handler missing');