Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

Size a selection's side-panel reply to the room the context window leaves

Claude6a42c4b998c7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e45af94fd..01ce65739 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,7 +10,7 @@ This changelog was generated from the repository Git history and release tags. V
- Selecting text on a page now shows a small bar, **Translate | Explain | Proofread | More…**, instead of the single `?` button. Translate, Explain and Proofread answer in a card next to the selection: Translate shows the detected source language, the original text and a target-language picker, and every card has an "Ask a follow-up" box that, once the answer is there, continues in the side panel with the card's answer as context. **More…** opens the same popup the `?` button opened (Summarize, Quiz me, Humanize, a question box and Hide this; Translate, Explain and Proofread are on the bar, so they are not repeated there). A card sends the selected text to your configured model provider, like any other selection action; selections over 4,000 characters go to the side panel instead of a card. A card answer that reaches the model's output limit says it was cut off and offers to continue in the side panel, which picks up from the unfinished text instead of starting over (a side-panel reply that is cut off again says so, including when an OpenAI Responses model reports the limit as an error, in which case the text it wrote is kept and the call still counts toward your cloud-cost allowance), and closing or redoing a card, or reloading or leaving the page, cancels the request it no longer needs. Chrome and Firefox.
### Fixed
-- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now, taking the new window from the tab-attached event rather than from a lookup a quick click could outrun. A card answer whose request is the one to reach the cloud cost allowance now says so under the answer instead of showing it as an ordinary success. The room a card leaves for the prompt in a small context window is now estimated as a worst case (a token per ASCII character, four for anything else), so dense text such as minified code or hashes is handed to the side panel on a 4k local model instead of being sent a request the backend rejects. Continuing a cut-off Proofread in the side panel no longer asks for the whole corrected version again alongside the request for only the remainder.
+- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now, taking the new window from the tab-attached event rather than from a lookup a quick click could outrun. A card answer whose request is the one to reach the cloud cost allowance now says so under the answer instead of showing it as an ordinary success. The room a card leaves for the prompt in a small context window is now estimated as a worst case (a token per ASCII character, four for anything else), so dense text such as minified code or hashes is handed to the side panel on a 4k local model instead of being sent a request the backend rejects. Continuing a cut-off Proofread in the side panel no longer asks for the whole corrected version again alongside the request for only the remainder. Continuing a cut-off card answer, or asking a follow-up, in the side panel now sizes the reply to what the model's context window leaves, so a small local model is not sent a request it rejects; when nothing fits it says the selection is too long for the model instead of failing.
- Stopping a task while WebBrain was waiting for page navigation to settle now stops it immediately. The pending settle step previously kept waiting and could let the run continue after you chose Stop.
- Mastodon: replying to toots in a narrow window no longer posts the same reply several times and stops before the next toot. There, a toot's Reply button opens the composer at /publish, and after posting Mastodon goes back to the previous column by itself. WebBrain took both page changes for something you did, so it reported each click's outcome as unknown; the agent then concluded its reply had not been posted, wrote it again, and never reached the next toot. In Chrome, a page's own Back is now told apart from your browser's Back button: the page a click opens counts as that click's verified effect, and a click after which the page goes back by itself counts as delivered, to be checked on the page. The Mastodon notes now cover replies (start with the prefilled @mentions once, check the reply under the toot before any retry) and stay active on Favourites, Bookmarks and the other columns. Firefox still treats a page's own Back as yours until its Back button reporting is verified.
- Finishing a form you asked WebBrain not to submit no longer takes most of a minute. The completion check removed every line that mentioned a password, so "fill out this form with a random password … don't submit" reached its judge as "[credential omitted]", and so did the approved plan and the agent's summary. Seeing only an unsubmitted signup form, the judge answered "pending" again and again; in the trace it took four attempts and about 50 seconds, ending only when a slow screenshot check ran out of time. The active model's check, which already holds these texts in the conversation, now receives every line of them, with only addresses shortened and bearer tokens and entered passwords hidden, as in all judge evidence. A decision model on this device, or WebBrain Cloud's own while WebBrain Cloud is also your provider, receives that same copy; any other decision model keeps whole-line redaction, and when that leaves it without the task, it is no longer asked while the active model can judge. A password WebBrain typed into a password field during the run is now hidden in the text judges receive, such as the agent's summary or a field a "show password" control revealed, unless the typing tool could not tell what kind of field it was; other entered values stay visible, so a judge can still catch a mistyped one. The judges are told that a task asking not to submit is complete once the staged values match, that a hidden value neither proves nor disproves an entry, and, for each recorded entry, whether its field read back what was entered, whether it is a credential field and which field it targeted; a credential field's line in the page evidence keeps its role and reference. Staged field edits are judged from the accessibility tree first: a screenshot shows passwords only as dots, and the focused field's blinking caret made each one stale before its verdict returned.
diff --git a/docs/architecture.md b/docs/architecture.md
index 23a0f64a7..07a985ff3 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -537,7 +537,7 @@ cut-off, the error), and the card shows it under that answer, since the next car
fail with no warning. Continue sends the unfinished text as
`continueFrom`; the background wraps it inside the selection's untrusted block (whole, up to the
most a card can produce, because the side panel shows that block to the user) with a trusted request to resume from where it stops, because running the action again
-would meet the same output limit. A resumed Proofread gets its own instruction, without the ordinary request for one complete corrected version, so a model asked for only the remainder is not also asked to write the corrected text again. A side-panel reply that is cut off again says so (the
+would meet the same output limit. The side-panel reply to a continuation or a card follow-up carries the earlier answer as well as the selection, and a selection-only run skips context management, so that reply is sized to the room the model's context window leaves (the same worst-case estimate, `fitAnswerToContext`) rather than asking for the provider's full output ceiling; when the window leaves no room it says the selection is too long for the model and sends nothing. A resumed Proofread gets its own instruction, without the ordinary request for one complete corrected version, so a model asked for only the remainder is not also asked to write the corrected text again. A side-panel reply that is cut off again says so (the
response-only path appends a notice when the provider reports a length finish, WebGPU
reports `outputTruncated`, or OpenAI Responses raises the limit as an error carrying
`partialContent`, whose text is then shown), so a long remainder never passes as complete
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index c1c2dcfbc..92eb45a88 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -187,12 +187,14 @@ import {
import { mergeRedactionFrameRegions, mapRegionsToImage, pixelateDataUrl } from './screenshot-redaction.js';
import { buildTrustedRuntimeContext, stripTrustedRuntimeContext } from './runtime-context.js';
import {
+ fitAnswerToContext,
isSelectionSourceGrounding,
isSelectionProseAction,
normalizeSelectionAction,
normalizeSelectionSourceGrounding,
SELECTION_CONTEXT_SOURCE_GROUNDING,
SELECTION_ONLY_SOURCE_GROUNDING,
+ SELECTION_TOO_LONG_FOR_CONTEXT,
} from '../context-menu-storage.js';
import { resolveSavedDownload } from '../download-result.js';
import { executeChromeWebStoreSkillTool, isTrustedChromeWebStoreSkillTool } from '../chrome-web-store-release.js';
@@ -26027,13 +26029,21 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
...modelMessages.slice(modelMessages[0]?.role === 'system' ? 1 : 0),
];
const prunedMessages = this._pruneOldImages(this._completionEvidenceMessages(tabId, contextMessages, runOptions), provider);
+ // A selection-only run skips context management, and a resumed or follow-up prompt carries
+ // the earlier answer as well as the selection, so the reply is sized to what the model's
+ // window leaves rather than to the provider's full output ceiling.
+ let maxTokens = this._providerMaxOutputTokens(provider);
+ if (selectionScoped && phase === 'response_only') {
+ maxTokens = fitAnswerToContext({ contextWindow: provider?.contextWindow, messages: prunedMessages, wanted: maxTokens });
+ if (maxTokens === 0) return SELECTION_TOO_LONG_FOR_CONTEXT;
+ }
// Voluntary research sharing needs the exact model-facing request (with
// the context-only system prompt). Callers pass a holder when the result
// may be shared; recovery phases omit it and are never shared.
if (shareCapture) shareCapture.request = prunedMessages;
const chatOpts = {
temperature: ['delivery_recovery', 'step_limit_recovery'].includes(phase) ? 0.2 : 0.3,
- maxTokens: this._providerMaxOutputTokens(provider),
+ maxTokens,
...(Array.isArray(tools) && tools.length ? { tools } : {}),
...(toolChoice ? { toolChoice } : {}),
};
diff --git a/src/chrome/src/context-menu-storage.js b/src/chrome/src/context-menu-storage.js
index ee0387a8c..40f1d8d1b 100644
--- a/src/chrome/src/context-menu-storage.js
+++ b/src/chrome/src/context-menu-storage.js
@@ -306,6 +306,22 @@ function estimateInlinePromptTokens(messages) {
return Math.ceil(tokens);
}
+// The output a request may ask for so that its prompt and answer both fit the model's
+// context window, or 0 when the prompt leaves less than the floor (or `wanted`, if that is
+// lower). An unknown window changes nothing. Callers that skip context management, as a
+// selection-only run does, use this so a long selection plus an earlier answer is not sent
+// to a small local model that would reject it.
+export function fitAnswerToContext({ contextWindow, messages, wanted }) {
+ const window = Math.floor(Number(contextWindow));
+ if (!(window > 0)) return wanted;
+ const room = window - estimateInlinePromptTokens(messages);
+ if (room < Math.min(INLINE_MIN_ANSWER_TOKENS, wanted)) return 0;
+ return Math.min(wanted, room);
+}
+
+// Said in place of an answer when a side-panel selection run has no room left for one.
+export const SELECTION_TOO_LONG_FOR_CONTEXT = 'This selection, with the earlier text it builds on, is too long for the model\'s context window, so it was not sent. Select less text, or switch to a model with a larger context window.';
+
// 0 means the prompt alone leaves the answer less than its floor in the provider's
// context window, so the card should hand the selection to the side panel.
export function inlineSelectionMaxTokens(selectionText, action, providerCeiling, { contextWindow, messages } = {}) {
@@ -315,11 +331,7 @@ export function inlineSelectionMaxTokens(selectionText, action, providerCeiling,
: Math.max(INLINE_ANSWER_MIN_TOKENS, Math.min(INLINE_ANSWER_MAX_TOKENS, chars * 2));
const limit = Math.floor(Number(providerCeiling));
const budget = Math.min(wanted, limit > 0 ? limit : INLINE_FALLBACK_OUTPUT_CEILING);
- const window = Math.floor(Number(contextWindow));
- if (!(window > 0)) return budget;
- const room = window - estimateInlinePromptTokens(messages);
- if (room < Math.min(INLINE_MIN_ANSWER_TOKENS, budget)) return 0;
- return Math.min(budget, room);
+ return fitAnswerToContext({ contextWindow, messages, wanted: budget });
}
// Whether the model stopped because it ran out of output budget. Providers report
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index 9ab673fa5..16ae84ae3 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -179,12 +179,14 @@ import {
import { mergeRedactionFrameRegions, mapRegionsToImage, pixelateDataUrl } from './screenshot-redaction.js';
import { buildTrustedRuntimeContext, stripTrustedRuntimeContext } from './runtime-context.js';
import {
+ fitAnswerToContext,
isSelectionSourceGrounding,
isSelectionProseAction,
normalizeSelectionAction,
normalizeSelectionSourceGrounding,
SELECTION_CONTEXT_SOURCE_GROUNDING,
SELECTION_ONLY_SOURCE_GROUNDING,
+ SELECTION_TOO_LONG_FOR_CONTEXT,
} from '../context-menu-storage.js';
import { firefoxHostPermissionFailure, firefoxRestrictedDomainFailure } from '../firefox-restricted-domains.js';
import { filenameInConfiguredDownloadDirectory } from '../download-directory.js';
@@ -23182,13 +23184,21 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
...modelMessages.slice(modelMessages[0]?.role === 'system' ? 1 : 0),
];
const prunedMessages = this._pruneOldImages(this._completionEvidenceMessages(tabId, contextMessages, runOptions), provider);
+ // A selection-only run skips context management, and a resumed or follow-up prompt carries
+ // the earlier answer as well as the selection, so the reply is sized to what the model's
+ // window leaves rather than to the provider's full output ceiling.
+ let maxTokens = this._providerMaxOutputTokens(provider);
+ if (selectionScoped && phase === 'response_only') {
+ maxTokens = fitAnswerToContext({ contextWindow: provider?.contextWindow, messages: prunedMessages, wanted: maxTokens });
+ if (maxTokens === 0) return SELECTION_TOO_LONG_FOR_CONTEXT;
+ }
// Voluntary research sharing needs the exact model-facing request (with
// the context-only system prompt). Callers pass a holder when the result
// may be shared; recovery phases omit it and are never shared.
if (shareCapture) shareCapture.request = prunedMessages;
const chatOpts = {
temperature: ['delivery_recovery', 'step_limit_recovery'].includes(phase) ? 0.2 : 0.3,
- maxTokens: this._providerMaxOutputTokens(provider),
+ maxTokens,
...(Array.isArray(tools) && tools.length ? { tools } : {}),
...(toolChoice ? { toolChoice } : {}),
};
diff --git a/src/firefox/src/context-menu-storage.js b/src/firefox/src/context-menu-storage.js
index 460dd2934..99d2d9dbb 100644
--- a/src/firefox/src/context-menu-storage.js
+++ b/src/firefox/src/context-menu-storage.js
@@ -306,6 +306,22 @@ function estimateInlinePromptTokens(messages) {
return Math.ceil(tokens);
}
+// The output a request may ask for so that its prompt and answer both fit the model's
+// context window, or 0 when the prompt leaves less than the floor (or `wanted`, if that is
+// lower). An unknown window changes nothing. Callers that skip context management, as a
+// selection-only run does, use this so a long selection plus an earlier answer is not sent
+// to a small local model that would reject it.
+export function fitAnswerToContext({ contextWindow, messages, wanted }) {
+ const window = Math.floor(Number(contextWindow));
+ if (!(window > 0)) return wanted;
+ const room = window - estimateInlinePromptTokens(messages);
+ if (room < Math.min(INLINE_MIN_ANSWER_TOKENS, wanted)) return 0;
+ return Math.min(wanted, room);
+}
+
+// Said in place of an answer when a side-panel selection run has no room left for one.
+export const SELECTION_TOO_LONG_FOR_CONTEXT = 'This selection, with the earlier text it builds on, is too long for the model\'s context window, so it was not sent. Select less text, or switch to a model with a larger context window.';
+
// 0 means the prompt alone leaves the answer less than its floor in the provider's
// context window, so the card should hand the selection to the side panel.
export function inlineSelectionMaxTokens(selectionText, action, providerCeiling, { contextWindow, messages } = {}) {
@@ -315,11 +331,7 @@ export function inlineSelectionMaxTokens(selectionText, action, providerCeiling,
: Math.max(INLINE_ANSWER_MIN_TOKENS, Math.min(INLINE_ANSWER_MAX_TOKENS, chars * 2));
const limit = Math.floor(Number(providerCeiling));
const budget = Math.min(wanted, limit > 0 ? limit : INLINE_FALLBACK_OUTPUT_CEILING);
- const window = Math.floor(Number(contextWindow));
- if (!(window > 0)) return budget;
- const room = window - estimateInlinePromptTokens(messages);
- if (room < Math.min(INLINE_MIN_ANSWER_TOKENS, budget)) return 0;
- return Math.min(budget, room);
+ return fitAnswerToContext({ contextWindow, messages, wanted: budget });
}
// Whether the model stopped because it ran out of output budget. Providers report
diff --git a/test/run.js b/test/run.js
index 47bfb7f5d..c17dd4682 100644
--- a/test/run.js
+++ b/test/run.js
@@ -696,6 +696,8 @@ const {
inlineAnswerWasCut: inlineAnswerWasCutCh,
inlineErrorWasCut: inlineErrorWasCutCh,
inlineSelectionMaxTokens: inlineSelectionMaxTokensCh,
+ fitAnswerToContext: fitAnswerToContextCh,
+ SELECTION_TOO_LONG_FOR_CONTEXT: SELECTION_TOO_LONG_FOR_CONTEXT_CH,
buildContinuationSelectionPrompt: buildContinuationSelectionPromptCh,
INLINE_CONTINUE_PARTIAL_MAX_CHARS: INLINE_CONTINUE_PARTIAL_MAX_CHARS_CH,
buildFollowUpSelectionText: buildFollowUpSelectionTextCh,
@@ -721,6 +723,8 @@ const {
inlineAnswerWasCut: inlineAnswerWasCutFx,
inlineErrorWasCut: inlineErrorWasCutFx,
inlineSelectionMaxTokens: inlineSelectionMaxTokensFx,
+ fitAnswerToContext: fitAnswerToContextFx,
+ SELECTION_TOO_LONG_FOR_CONTEXT: SELECTION_TOO_LONG_FOR_CONTEXT_FX,
buildContinuationSelectionPrompt: buildContinuationSelectionPromptFx,
INLINE_CONTINUE_PARTIAL_MAX_CHARS: INLINE_CONTINUE_PARTIAL_MAX_CHARS_FX,
buildFollowUpSelectionText: buildFollowUpSelectionTextFx,
@@ -54324,6 +54328,57 @@ test('an OpenAI Responses call that ends at the output limit is still metered',
}
});
+test('a side-panel selection reply is sized to the room the context window leaves', async () => {
+ for (const [label, AgentClass, grounding, fit, tooLong] of [
+ ['chrome', AgentCh, SELECTION_ONLY_SOURCE_GROUNDING_CH, fitAnswerToContextCh, SELECTION_TOO_LONG_FOR_CONTEXT_CH],
+ ['firefox', AgentFx, SELECTION_ONLY_SOURCE_GROUNDING_FX, fitAnswerToContextFx, SELECTION_TOO_LONG_FOR_CONTEXT_FX],
+ ]) {
+ // The helper: an unknown window changes nothing, a roomy one allows the ceiling, a tight one is clamped, none refuses.
+ const prompt = (chars) => [{ role: 'user', content: 'x'.repeat(chars) }];
+ assert.equal(fit({ contextWindow: undefined, messages: prompt(9000), wanted: 8192 }), 8192, `${label}: an unknown window is not guessed at`);
+ assert.equal(fit({ contextWindow: 131_072, messages: prompt(9000), wanted: 8192 }), 8192, `${label}: a large window allows the full ceiling`);
+ assert.equal(fit({ contextWindow: 8192, messages: prompt(6000), wanted: 8192 }), 8192 - 6000 - 128, `${label}: a tight window is clamped to what the prompt leaves`);
+ assert.equal(fit({ contextWindow: 8192, messages: prompt(8000), wanted: 8192 }), 0, `${label}: a prompt that leaves under the floor is refused`);
+ assert.equal(fit({ contextWindow: 8192, messages: prompt(8000), wanted: 100 }), 0, `${label}: and so is one that cannot fit even a small ceiling`);
+
+ const run = async ({ contextWindow, chars, maxOutputTokens = 8192, runOptions = { sourceGrounding: grounding }, phase = 'response_only' }) => {
+ const requested = [];
+ const provider = {
+ supportsTools: false, supportsVision: false, promptTier: 'full', contextWindow, maxOutputTokens,
+ model: 'test-model', name: 'test-provider',
+ chat: async (_messages, options) => { requested.push(options.maxTokens); return { content: 'The rest.', toolCalls: null }; },
+ };
+ const agent = new AgentClass({ getActive: () => provider, getVisionProvider: async () => null });
+ agent._checkCostAllowance = async () => null;
+ agent._recordCostUsage = async () => null;
+ const messages = [{ role: 'user', content: 'y'.repeat(chars) }];
+ const text = await agent._generateContextOnlyResponse(
+ 1, messages, provider, agent._newCostRunState(), null, { phase, step: 1, runOptions, currentUserMessage: messages[0] },
+ );
+ return { text, requested };
+ };
+
+ // An 8k local model asked to resume a long answer: the reply may not ask for the 8k ceiling.
+ let result = await run({ contextWindow: 8192, chars: 5000 });
+ assert.equal(result.requested.length, 1, `${label}: the model is asked`);
+ assert.ok(result.requested[0] <= 8192 - 5000 && result.requested[0] >= 256, `${label}: the reply is limited to what the prompt leaves: ${result.requested[0]}`);
+ assert.equal(result.text, 'The rest.');
+ // Too long to leave any room: nothing is sent, and the person is told why.
+ result = await run({ contextWindow: 8192, chars: 9000 });
+ assert.deepEqual(result.requested, [], `${label}: a prompt that fills the window is not sent`);
+ assert.equal(result.text, tooLong, `${label}: the reply says the selection is too long for the model`);
+ // A large window is unaffected, and so is a run that is not selection-only.
+ result = await run({ contextWindow: 131_072, chars: 9000 });
+ assert.equal(result.requested[0], 8192, `${label}: a roomy window keeps the provider's ceiling`);
+ result = await run({ contextWindow: 8192, chars: 9000, runOptions: {} });
+ assert.equal(result.requested[0], 8192, `${label}: only selection-only runs, which skip context management, are clamped`);
+ result = await run({ contextWindow: undefined, chars: 9000 });
+ assert.equal(result.requested[0], 8192, `${label}: an unknown window is not guessed at`);
+ result = await run({ contextWindow: 8192, chars: 9000, phase: 'step_limit_recovery' });
+ assert.equal(result.requested[0], 8192, `${label}: only the reply itself is clamped, not a recovery phase`);
+ }
+});
+
test('ordinary attachments leave selection grounding and remain usable', async () => {
for (const [label, AgentClass, buildSelectionPrompt, sourceGrounding] of [
['chrome', AgentCh, buildSelectionPromptCh, SELECTION_ONLY_SOURCE_GROUNDING_CH],