Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Budget non-ASCII prompts at four tokens; stop a resumed Proofread restarting
Claudedca2611680f3
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex aaf1550e0..e45af94fd 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,7 +10,7 @@ This changelog was generated from the repository Git history and release tags. V- Selecting text on a page now shows a small bar, **Translate | Explain | Proofread | More…**, instead of the single `?` button. Translate, Explain and Proofread answer in a card next to the selection: Translate shows the detected source language, the original text and a target-language picker, and every card has an "Ask a follow-up" box that, once the answer is there, continues in the side panel with the card's answer as context. **More…** opens the same popup the `?` button opened (Summarize, Quiz me, Humanize, a question box and Hide this; Translate, Explain and Proofread are on the bar, so they are not repeated there). A card sends the selected text to your configured model provider, like any other selection action; selections over 4,000 characters go to the side panel instead of a card. A card answer that reaches the model's output limit says it was cut off and offers to continue in the side panel, which picks up from the unfinished text instead of starting over (a side-panel reply that is cut off again says so, including when an OpenAI Responses model reports the limit as an error, in which case the text it wrote is kept and the call still counts toward your cloud-cost allowance), and closing or redoing a card, or reloading or leaving the page, cancels the request it no longer needs. Chrome and Firefox.### Fixed-- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now, taking the new window from the tab-attached event rather than from a lookup a quick click could outrun. A card answer whose request is the one to reach the cloud cost allowance now says so under the answer instead of showing it as an ordinary success. The room a card leaves for the prompt in a small context window is now estimated as a worst case (a token per ASCII character, two for anything else), so dense text such as minified code or hashes is handed to the side panel on a 4k local model instead of being sent a request the backend rejects.+- Chrome: sending a follow-up or a selection action to WebBrain while the side panel is already open no longer closes it. Each side panel now tells the background whether it is on screen, and the background skips re-opening a panel that is already showing and hands it the prompt directly A panel that was open while the background restarted tells the new worker about itself at once instead of after a delay, so a request in that moment does not close it. The same holds for actions chosen on a selection in WebBrain's PDF viewer, where a panel that was closed is opened once rather than twice. A side panel that the browser moves to another window (its tab dragged out) now reports its new window, so an action in that window no longer closes it and an action in the old window opens a panel again. The PDF viewer keeps its own window up to date the same way, so a viewer tab dragged to another window reports where it is now, taking the new window from the tab-attached event rather than from a lookup a quick click could outrun. A card answer whose request is the one to reach the cloud cost allowance now says so under the answer instead of showing it as an ordinary success. The room a card leaves for the prompt in a small context window is now estimated as a worst case (a token per ASCII character, four for anything else), so dense text such as minified code or hashes is handed to the side panel on a 4k local model instead of being sent a request the backend rejects. Continuing a cut-off Proofread in the side panel no longer asks for the whole corrected version again alongside the request for only the remainder.- Stopping a task while WebBrain was waiting for page navigation to settle now stops it immediately. The pending settle step previously kept waiting and could let the run continue after you chose Stop.- Mastodon: replying to toots in a narrow window no longer posts the same reply several times and stops before the next toot. There, a toot's Reply button opens the composer at /publish, and after posting Mastodon goes back to the previous column by itself. WebBrain took both page changes for something you did, so it reported each click's outcome as unknown; the agent then concluded its reply had not been posted, wrote it again, and never reached the next toot. In Chrome, a page's own Back is now told apart from your browser's Back button: the page a click opens counts as that click's verified effect, and a click after which the page goes back by itself counts as delivered, to be checked on the page. The Mastodon notes now cover replies (start with the prefilled @mentions once, check the reply under the toot before any retry) and stay active on Favourites, Bookmarks and the other columns. Firefox still treats a page's own Back as yours until its Back button reporting is verified.- Finishing a form you asked WebBrain not to submit no longer takes most of a minute. The completion check removed every line that mentioned a password, so "fill out this form with a random password … don't submit" reached its judge as "[credential omitted]", and so did the approved plan and the agent's summary. Seeing only an unsubmitted signup form, the judge answered "pending" again and again; in the trace it took four attempts and about 50 seconds, ending only when a slow screenshot check ran out of time. The active model's check, which already holds these texts in the conversation, now receives every line of them, with only addresses shortened and bearer tokens and entered passwords hidden, as in all judge evidence. A decision model on this device, or WebBrain Cloud's own while WebBrain Cloud is also your provider, receives that same copy; any other decision model keeps whole-line redaction, and when that leaves it without the task, it is no longer asked while the active model can judge. A password WebBrain typed into a password field during the run is now hidden in the text judges receive, such as the agent's summary or a field a "show password" control revealed, unless the typing tool could not tell what kind of field it was; other entered values stay visible, so a judge can still catch a mistyped one. The judges are told that a task asking not to submit is complete once the staged values match, that a hidden value neither proves nor disproves an entry, and, for each recorded entry, whether its field read back what was entered, whether it is a credential field and which field it targeted; a credential field's line in the page evidence keeps its role and reference. Staged field edits are judged from the accessibility tree first: a screenshot shows passwords only as dots, and the focused field's blinking caret made each one stale before its verdict returned.diff --git a/docs/architecture.md b/docs/architecture.mdindex 98649822d..23a0f64a7 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -522,7 +522,7 @@ cost allowance (90 s timeout) and returns the text. The output budget is sized tselection (at least 2,000 tokens; Translate and Proofread grow to about two tokens percharacter, up to 8,000) and never exceeds the provider's own output ceiling or the room itscontext window leaves after the prompt (estimated on the high side, since the provider's tokenizer-is not available: a whole token per ASCII character, as dense text such as minified code or hashes can need, two per other character such as accented letters or CJK,+is not available: a whole token per ASCII character, as dense text such as minified code or hashes can need, four per other character, the most a byte-level fallback spends on one code point (accented letters, CJK, emoji),plus a reserve), so a small local window is not sent a request it would reject. When the promptleaves less than 256 tokens for the answer, the background makes no call and answers with thecut-off shape, so the card hands the selection to the side panel. If the reply@@ -537,7 +537,7 @@ cut-off, the error), and the card shows it under that answer, since the next carfail with no warning. Continue sends the unfinished text as`continueFrom`; the background wraps it inside the selection's untrusted block (whole, up to themost a card can produce, because the side panel shows that block to the user) with a trusted request to resume from where it stops, because running the action again-would meet the same output limit. A side-panel reply that is cut off again says so (the+would meet the same output limit. A resumed Proofread gets its own instruction, without the ordinary request for one complete corrected version, so a model asked for only the remainder is not also asked to write the corrected text again. A side-panel reply that is cut off again says so (theresponse-only path appends a notice when the provider reports a length finish, WebGPUreports `outputTruncated`, or OpenAI Responses raises the limit as an error carrying`partialContent`, whose text is then shown), so a long remainder never passes as completediff --git a/src/chrome/src/context-menu-storage.js b/src/chrome/src/context-menu-storage.jsindex a1794372d..ee0387a8c 100644--- a/src/chrome/src/context-menu-storage.js+++ b/src/chrome/src/context-menu-storage.js@@ -283,12 +283,12 @@ const INLINE_FALLBACK_OUTPUT_CEILING = 4096;// provider's tokenizer is not available here, so the prompt is estimated per// character as a worst case, not as prose: an ASCII character at one whole token (minified// code, identifiers, hashes and other high-entropy text can take that, and prose takes far-// less), anything else (accented Latin, CJK, Indic, Arabic, Cyrillic, emoji) at two, which-// covers byte-level fallbacks. Over-counting only makes a tight window end in a flagged-// cut-off or a hand-off to the side panel instead of a refused request. Chat-template-// tokens are covered by the reserve.+// less), anything else (accented Latin, CJK, Indic, Arabic, Cyrillic, emoji) at four, the+// most a byte-level fallback spends on one code point (its UTF-8 bytes). Over-counting only+// makes a tight window end in a flagged cut-off or a hand-off to the side panel instead of a+// refused request. Chat-template tokens are covered by the reserve.const INLINE_ASCII_TOKENS_PER_CHAR = 1;-const INLINE_OTHER_TOKENS_PER_CHAR = 2;+const INLINE_OTHER_TOKENS_PER_CHAR = 4;const INLINE_ASCII_RANGE_END = 0x80;const INLINE_PROMPT_TOKEN_RESERVE = 128;const INLINE_MIN_ANSWER_TOKENS = 256;@@ -390,13 +390,23 @@ export function buildFollowUpSelectionText(selectionText, priorAction, priorAnswexport const INLINE_CONTINUE_PARTIAL_MAX_CHARS = INLINE_CARD_ANSWER_MAX_CHARS;const INLINE_CONTINUE_NOTE = ' An earlier attempt at this stopped partway; the text it produced is included after the selection. Continue it from exactly where it stops and give only the missing remainder, without repeating anything already written.';+// Proofread's own instruction asks for one complete corrected version. Sent beside a request+// to give only the missing remainder, a model can satisfy the first by starting the corrected+// text over, which leaves the person with it twice. A resumed run therefore gets an instruction+// that describes the work without the whole-deliverable demand.+const INLINE_CONTINUATION_INSTRUCTIONS = Object.freeze({+ proofread: 'Proofread this selected text, tying every claimed error to exact selected wording and distinguishing actual errors from optional style suggestions. If the earlier text stopped inside the corrected version, carry that version on rather than starting it again.',+});+export function buildContinuationSelectionPrompt(selectionText, action, partialAnswer, language = '') {const actionId = String(action || '').trim();const label = INLINE_FOLLOW_UP_LABELS[actionId];const selection = String(selectionText || '').trim();const partial = String(partialAnswer || '').trim();if (!label || !selection || !partial) return '';- const instruction = selectionActionInstruction(actionId, '', language);+ const instruction = Object.prototype.hasOwnProperty.call(INLINE_CONTINUATION_INSTRUCTIONS, actionId)+ ? `${INLINE_CONTINUATION_INSTRUCTIONS[actionId]}${responseLanguageInstruction(language)}`+ : selectionActionInstruction(actionId, '', language);if (!instruction) return '';const kept = partial.length > INLINE_CONTINUE_PARTIAL_MAX_CHARS? `…${partial.slice(-INLINE_CONTINUE_PARTIAL_MAX_CHARS)}`diff --git a/src/firefox/src/context-menu-storage.js b/src/firefox/src/context-menu-storage.jsindex 729bd3ab9..460dd2934 100644--- a/src/firefox/src/context-menu-storage.js+++ b/src/firefox/src/context-menu-storage.js@@ -283,12 +283,12 @@ const INLINE_FALLBACK_OUTPUT_CEILING = 4096;// provider's tokenizer is not available here, so the prompt is estimated per// character as a worst case, not as prose: an ASCII character at one whole token (minified// code, identifiers, hashes and other high-entropy text can take that, and prose takes far-// less), anything else (accented Latin, CJK, Indic, Arabic, Cyrillic, emoji) at two, which-// covers byte-level fallbacks. Over-counting only makes a tight window end in a flagged-// cut-off or a hand-off to the side panel instead of a refused request. Chat-template-// tokens are covered by the reserve.+// less), anything else (accented Latin, CJK, Indic, Arabic, Cyrillic, emoji) at four, the+// most a byte-level fallback spends on one code point (its UTF-8 bytes). Over-counting only+// makes a tight window end in a flagged cut-off or a hand-off to the side panel instead of a+// refused request. Chat-template tokens are covered by the reserve.const INLINE_ASCII_TOKENS_PER_CHAR = 1;-const INLINE_OTHER_TOKENS_PER_CHAR = 2;+const INLINE_OTHER_TOKENS_PER_CHAR = 4;const INLINE_ASCII_RANGE_END = 0x80;const INLINE_PROMPT_TOKEN_RESERVE = 128;const INLINE_MIN_ANSWER_TOKENS = 256;@@ -390,13 +390,23 @@ export function buildFollowUpSelectionText(selectionText, priorAction, priorAnswexport const INLINE_CONTINUE_PARTIAL_MAX_CHARS = INLINE_CARD_ANSWER_MAX_CHARS;const INLINE_CONTINUE_NOTE = ' An earlier attempt at this stopped partway; the text it produced is included after the selection. Continue it from exactly where it stops and give only the missing remainder, without repeating anything already written.';+// Proofread's own instruction asks for one complete corrected version. Sent beside a request+// to give only the missing remainder, a model can satisfy the first by starting the corrected+// text over, which leaves the person with it twice. A resumed run therefore gets an instruction+// that describes the work without the whole-deliverable demand.+const INLINE_CONTINUATION_INSTRUCTIONS = Object.freeze({+ proofread: 'Proofread this selected text, tying every claimed error to exact selected wording and distinguishing actual errors from optional style suggestions. If the earlier text stopped inside the corrected version, carry that version on rather than starting it again.',+});+export function buildContinuationSelectionPrompt(selectionText, action, partialAnswer, language = '') {const actionId = String(action || '').trim();const label = INLINE_FOLLOW_UP_LABELS[actionId];const selection = String(selectionText || '').trim();const partial = String(partialAnswer || '').trim();if (!label || !selection || !partial) return '';- const instruction = selectionActionInstruction(actionId, '', language);+ const instruction = Object.prototype.hasOwnProperty.call(INLINE_CONTINUATION_INSTRUCTIONS, actionId)+ ? `${INLINE_CONTINUATION_INSTRUCTIONS[actionId]}${responseLanguageInstruction(language)}`+ : selectionActionInstruction(actionId, '', language);if (!instruction) return '';const kept = partial.length > INLINE_CONTINUE_PARTIAL_MAX_CHARS? `…${partial.slice(-INLINE_CONTINUE_PARTIAL_MAX_CHARS)}`diff --git a/test/run.js b/test/run.jsindex 9ae5b91fd..47bfb7f5d 100644--- a/test/run.js+++ b/test/run.js@@ -52207,6 +52207,18 @@ test('a cut-off card answer is resumed in the side panel, not run again', () =>assert.doesNotMatch(longPrompt, /START-/, `${label}: the head of a long unfinished answer is dropped`);assert.match(longPrompt, /\[Unfinished proofreading result of the text above\./);+ // Resuming a Proofread must not also ask for the whole corrected version again.+ const resumedProofread = buildContinuation('Some text with an eror.', 'proofread', 'Errors: "eror" should be "error". Corrected: Some text with an', 'en');+ const resumedTrusted = resumedProofread.slice(0, resumedProofread.indexOf('<untrusted_page_content id="'));+ assert.doesNotMatch(resumedTrusted, /one complete corrected version|First verify|select the complete passage/, `${label}: a resumed Proofread must not be told to produce the whole corrected version`);+ assert.match(resumedTrusted, /^Proofread this selected text, tying every claimed error to exact selected wording/, `${label}: it is still a proofreading task`);+ assert.match(resumedTrusted, /carry that version on rather than starting it again/, `${label}: a stop inside the corrected version is picked up there`);+ assert.match(resumedTrusted, /Continue it from exactly where it stops and give only the missing remainder, without repeating anything already written\./, `${label}: the remainder is all that is asked for`);+ assert.match(buildContinuation('Some text.', 'proofread', 'Partial.', 'de').slice(0, 600), /Respond in German/, `${label}: the reply language still follows the card`);+ assert.equal(buildContinuation('Some text.', 'proofread', 'Partial.', 'xx').includes('Respond in'), false, `${label}: an unknown language adds no language rule`);+ assert.match(buildSelectionPrompt('Some text.', 'proofread', '', 'en'), /provide one complete corrected version/, `${label}: the ordinary Proofread request is unchanged`);+ assert.match(buildContinuation('Some text.', 'explain', 'Partial.', 'en'), /^Explain this selected text in plain language\./, `${label}: other actions keep their ordinary instruction`);+// Nothing to resume, or nothing it can resume: no prompt, so the background rejects the request.assert.equal(buildContinuation('Some text.', 'translate', ' ', 'de'), '');assert.equal(buildContinuation('', 'translate', 'partial', 'de'), '');@@ -52468,11 +52480,13 @@ test('inline card token budget follows the selection and never exceeds the proviconst cjk = (chars) => [{ role: 'system', content: 'x'.repeat(500) }, { role: 'user', content: '漢'.repeat(chars) }];assert.equal(budget('漢'.repeat(4000), 'translate', 16_000, { contextWindow: 4096, messages: cjk(4000) }), 0,`${label}: a 4,000-character CJK selection does not fit a 4k window and is handed off`);- assert.equal(budget('漢'.repeat(1000), 'translate', 16_000, { contextWindow: 4096, messages: cjk(1000) }), 4096 - 128 - 500 - 2000,- `${label}: a shorter one gets the window minus a two-token-per-character prompt`);+ assert.equal(budget('漢'.repeat(500), 'translate', 16_000, { contextWindow: 4096, messages: cjk(500) }), 4096 - 128 - 500 - 2000,+ `${label}: a shorter one gets the window minus a four-token-per-character prompt`);+ assert.equal(budget('漢'.repeat(1000), 'translate', 16_000, { contextWindow: 4096, messages: cjk(1000) }), 0,+ `${label}: a thousand CJK characters can need four tokens each and leave nothing for the answer`);assert.equal(budget('漢'.repeat(4000), 'translate', 16_000, { contextWindow: 131_072, messages: cjk(4000) }), 8000,`${label}: a large window still allows the full answer`);- assert.equal(budget('Привет'.repeat(300), 'translate', 16_000, { contextWindow: 4096, messages: [{ role: 'user', content: 'Привет'.repeat(300) }] }), 4096 - 128 - 3600,+ assert.equal(budget('Привет'.repeat(150), 'translate', 16_000, { contextWindow: 4096, messages: [{ role: 'user', content: 'Привет'.repeat(150) }] }), 4096 - 128 - 3600,`${label}: Cyrillic is counted as non-Latin text too`);// High-entropy ASCII (hashes, minified code, identifiers) is not prose: a token per character.const hashes = '0123456789abcdef'.repeat(250);@@ -52480,9 +52494,12 @@ test('inline card token budget follows the selection and never exceeds the provi`${label}: 4,000 characters of hex do not fit a 4k window, so the card hands them to the side panel`);assert.equal(budget(hashes, 'translate', 16_000, { contextWindow: 8192, messages: [{ role: 'user', content: hashes }] }), 8192 - 128 - 4000,`${label}: an 8k window keeps back a token per character of the prompt`);- assert.equal(budget('é'.repeat(300), 'translate', 16_000, { contextWindow: 1000, messages: [{ role: 'user', content: 'é'.repeat(300) }] }), 1000 - 128 - 600,- `${label}: accented letters can take two tokens, so they are counted as such`);- assert.equal(budget('😀'.repeat(10), 'translate', 16_000, { contextWindow: 500, messages: [{ role: 'user', content: '😀'.repeat(10) }] }), 500 - 128 - 20,+ assert.equal(budget('é'.repeat(150), 'translate', 16_000, { contextWindow: 1000, messages: [{ role: 'user', content: 'é'.repeat(150) }] }), 1000 - 128 - 600,+ `${label}: accented letters can fall back to bytes, so they are counted at the most a code point can take`);+ const emoji = '😀'.repeat(1000);+ assert.equal(budget(emoji, 'translate', 16_000, { contextWindow: 4096, messages: [{ role: 'user', content: emoji }] }), 0,+ `${label}: a thousand emoji or rare characters can fill a 4k window by themselves and are handed to the side panel`);+ assert.equal(budget('😀'.repeat(10), 'translate', 16_000, { contextWindow: 500, messages: [{ role: 'user', content: '😀'.repeat(10) }] }), 500 - 128 - 40,`${label}: an astral character counts once, not as two UTF-16 units`);assert.equal(budget(longText, 'translate', 16_000, { contextWindow: undefined, messages: prompt(3500) }), 6000,`${label}: an unknown window is not guessed at`);