Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

tiny Ask: look again for a visual follow-up to a screenshot answer

Claudec0210a9928a1
diff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.md
index 87df983f2..c68ac0e34 100644
--- a/docs/privacy-and-data-flow.md
+++ b/docs/privacy-and-data-flow.md
@@ -57,7 +57,8 @@ The user chooses their provider in Settings. Options include:
Worker with no API key, base URL, localhost server, or model endpoint. In Ask,
a model that reads images also gets a screenshot of the visible page when its
own on-device readout says the question needs it (not in Strict Secret Mode);
- it stays in the browser and is not saved with the chat
+ it stays in the browser and is not saved with the chat (the stored question
+ only notes that one was used, so a follow-up can look again)
- **Local OpenAI-compatible Proxy**: WebBrain connects only to the configured
local gateway, but the gateway may forward the request context to an upstream
account. Its configuration and privacy policy determine where data goes.
diff --git a/docs/webgpu-models.md b/docs/webgpu-models.md
index 972f216c9..dc9d4e5cb 100644
--- a/docs/webgpu-models.md
+++ b/docs/webgpu-models.md
@@ -190,6 +190,15 @@ Use the tools in `scripts/models/bonsai/`:
- Misses lean towards no screenshot, which is the old behaviour; most are
colour-of-a-button questions in unseen languages. A wrong "yes" only
costs a screenshot, about 0.5 s.
+ - Follow-ups: the screenshot is gone from history, so "And the button?"
+ after a screenshot answer needs another look. The stored question is
+ marked (`webbrainTinyAskScreenshot`, never the image), and the next turn
+ uses a lower bar, 0.35 instead of 0.5. The readout still sees only the
+ new message: with the earlier question added it said yes to 11–13 of 30
+ text follow-ups such as "Thanks!". On 60 follow-ups in 14 languages the
+ lower bar caught 28/30 visual ones (22/30 at 0.5) with 1/30 false "yes".
+ On 52 others in 16 languages, chosen afterwards, it caught 21/26 (18/26)
+ with 0/26 (`check-tiny-ask-screenshot-gate.mjs --follow-ups`).
- A page with almost no text gets a note to answer from the image. A
screenshot alone never suppresses offline references. With an attached
image both images can be sent, when the readout says the page is needed
diff --git a/scripts/check-tiny-ask-screenshot-gate.mjs b/scripts/check-tiny-ask-screenshot-gate.mjs
index 9ac8981aa..c2da851c0 100644
--- a/scripts/check-tiny-ask-screenshot-gate.mjs
+++ b/scripts/check-tiny-ask-screenshot-gate.mjs
@@ -13,11 +13,16 @@
// the built-in cases ({label, language, question, pageChars, ...})
// --return-hidden also record each row's margin and hidden state, for
// scripts/models/bonsai/train_gate_probe.py
+// --follow-ups short follow-ups to a turn answered from a screenshot, scored
+// at TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD (the readout sees
+// only the follow-up; the earlier question is for the reader)
import fs from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { chromium } from 'playwright';
-import { tinyAskScreenshotGate, tinyAskScreenshotGateProbability, TINY_ASK_SCREENSHOT_GATE_THRESHOLD } from '../src/chrome/src/agent/tiny-ask.js';
+import {
+ tinyAskScreenshotGate, tinyAskScreenshotGateProbability, TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD, TINY_ASK_SCREENSHOT_GATE_THRESHOLD,
+} from '../src/chrome/src/agent/tiny-ask.js';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const args = process.argv.slice(2);
@@ -26,7 +31,8 @@ const extension = path.resolve(option('--extension', path.join(root, 'src/chrome
const profile = path.resolve(option('--profile', '.build/tiny-ask-gate/profile'));
const output = path.resolve(option('--output', '.build/tiny-ask-gate/result.json'));
const model = 'webbrain-one/Ternary-Bonsai-2-2B-ONNX-WebGPU-Vision';
-const threshold = Number(option('--threshold', TINY_ASK_SCREENSHOT_GATE_THRESHOLD));
+const followUps = args.includes('--follow-ups');
+const threshold = Number(option('--threshold', followUps ? TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD : TINY_ASK_SCREENSHOT_GATE_THRESHOLD));
// [expected, language, question, context]. Expected Yes: the answer depends on
// what the current page visibly shows. Pages have text unless `pageChars` says
@@ -97,11 +103,63 @@ const CASES = [
[false, 'es', 'Resume https://news.example/launch en una frase.', { ...TEXT_PAGE, linkedPage: true }],
];
+// Follow-ups: [language, earlier question answered from a screenshot,
+// [follow-ups that need another look], [follow-ups that do not]]. The
+// threshold was chosen on the first set and checked on the second, which
+// adds languages (sv, id, vi, hi) and phrasings the first never uses.
+const FOLLOW_UPS_CHOSEN_ON = [
+ ['en', 'What colour is the banner?', ['And the button?', 'What about the logo?', 'Is it darker than the header?', 'Which one is bigger?'],
+ ['How much is the Team plan?', 'Who wrote this article?', 'Thanks!', 'Who was Alan Turing?']],
+ ['en', 'Describe the photo at the top.', ['What colour is it?', 'How many people are in it?', 'And the second picture?', 'Is anyone smiling?'],
+ ['Summarise the page.', 'When was this published?', 'What is the phone number here?', 'Tell me a short joke.']],
+ ['tr', 'Afiş ne renk?', ['Peki buton?', 'Ya logo?', 'Daha koyu mu?'], ['Ekip paketi ne kadar?', 'Teşekkürler!', 'Alan Turing kimdir?']],
+ ['tr', 'Fotoğrafta ne var?', ['Kaç kişi var?', 'Sağdaki ne?'], ['Bu sayfayı özetle.', 'Makale ne zaman yayımlandı?']],
+ ['de', 'Welche Farbe hat das Logo?', ['Und der Knopf?', 'Ist es dunkler als der Hintergrund?'], ['Wie viel kostet der Team-Tarif?', 'Danke!']],
+ ['es', '¿De qué color es el logo?', ['¿Y el botón?', '¿Y la foto de abajo?'], ['¿Cuánto cuesta el plan Team?', '¡Gracias!']],
+ ['fr', 'Que montre la photo ?', ["Et l'autre ?", 'De quelle couleur ?'], ["Combien coûte l'offre Team ?", 'Merci !']],
+ ['it', 'Di che colore è il logo?', ['E il pulsante?'], ['Quanto costa il piano Team?']],
+ ['pt', 'O que mostra a imagem?', ['E a de baixo?'], ['Quanto custa o plano Team?']],
+ ['ru', 'Какого цвета логотип?', ['А кнопка?'], ['Сколько стоит тариф Team?']],
+ ['ja', 'この写真には何が写っていますか?', ['背景は何色?', '右のは?'], ['チームプランはいくら?', 'ありがとう!']],
+ ['zh', '这张图片里有什么?', ['那背景呢?'], ['团队版多少钱?']],
+ ['ar', 'ما لون الشعار؟', ['والزر؟'], ['كم سعر خطة الفريق؟']],
+ ['ko', '로고는 무슨 색이에요?', ['버튼은요?', '배경은요?'], ['팀 요금제는 얼마예요?', '고마워요!']],
+ ['nl', 'Welke kleur heeft het logo?', ['En de knop?'], ['Hoeveel kost het Team-abonnement?']],
+ ['pl', 'Jakiego koloru jest logo?', ['A przycisk?'], ['Ile kosztuje plan Team?']],
+];
+const FOLLOW_UPS_CHECKED_ON = [
+ ['en', 'Is the logo round?', ['And the icon next to it?', 'Is the text on it white?', 'What shape is the button below?', 'How about on the right?'],
+ ['What does the refund policy say?', 'Can you translate the intro?', 'What is the capital of Peru?', 'Great, thank you.']],
+ ['en', 'What does the chart show?', ['Which bar is the tallest?', 'What colour is the line?', 'Is the trend going up?'],
+ ['Who is the author?', 'What are the opening hours?', 'OK.']],
+ ['tr', 'Logo yuvarlak mı?', ['Yanındaki simge?', 'Üstündeki yazı beyaz mı?', 'Grafikteki en uzun çubuk hangisi?'],
+ ['İade politikası ne diyor?', "Peru'nun başkenti neresi?", 'Tamam, sağ ol.']],
+ ['de', 'Was zeigt das Bild oben?', ['Und unten links?', 'Welche Farbe hat das Auto darin?'], ['Wer hat den Artikel geschrieben?', 'Was ist die Hauptstadt von Peru?']],
+ ['es', '¿Qué muestra el gráfico?', ['¿Cuál es la barra más alta?', '¿Y la línea roja?'], ['¿Quién es el autor?', 'Vale, gracias.']],
+ ['fr', 'Le logo est-il rond ?', ["Et l'icône à côté ?"], ['Que dit la politique de remboursement ?']],
+ ['it', 'Cosa mostra la foto?', ['Quante persone ci sono?'], ["Chi è l'autore?"]],
+ ['pt', 'De que cor é o botão?', ['E o fundo?'], ['Qual é o horário de funcionamento?']],
+ ['ru', 'Что на фотографии?', ['А справа?'], ['Кто автор статьи?']],
+ ['ja', 'ロゴは丸いですか?', ['隣のアイコンは?'], ['返金ポリシーは?']],
+ ['zh', '图表显示了什么?', ['哪个柱子最高?'], ['作者是谁?']],
+ ['ar', 'ماذا تظهر الصورة؟', ['وعلى اليمين؟'], ['من هو الكاتب؟']],
+ ['sv', 'Vilken färg har knappen?', ['Och bakgrunden?'], ['Vad kostar Team-planen?']],
+ ['id', 'Apa warna tombolnya?', ['Kalau logonya?'], ['Berapa harga paket Team?']],
+ ['vi', 'Nút có màu gì?', ['Còn nền thì sao?'], ['Gói Team giá bao nhiêu?']],
+ ['hi', 'बटन किस रंग का है?', ['और लोगो?'], ['टीम प्लान की कीमत क्या है?']],
+ ['ko', '사진에 뭐가 있어요?', ['오른쪽은요?'], ['작성자가 누구예요?']],
+];
+const followUpRows = (pairs, set) => pairs.flatMap(([language, earlier, visual, text]) => [
+ ...visual.map(question => [true, language, question, { ...TEXT_PAGE, set, earlier }]),
+ ...text.map(question => [false, language, question, { ...TEXT_PAGE, set, earlier }]),
+]);
+
const returnHidden = args.includes('--return-hidden');
const rows = option('--dataset', '')
? JSON.parse(await fs.readFile(option('--dataset'), 'utf8')).map(row => [Boolean(row.label), row.language, row.question,
{ pageChars: row.pageChars, imageAttachment: row.imageAttachment === true, linkedPage: row.linkedPage === true }])
- : CASES;
+ : followUps ? [...followUpRows(FOLLOW_UPS_CHOSEN_ON, 'chosen on'), ...followUpRows(FOLLOW_UPS_CHECKED_ON, 'checked on')]
+ : CASES;
const result = { model, extension, threshold, dataset: option('--dataset', ''), startedAt: new Date().toISOString(), cases: [] };
let context;
try {
@@ -124,7 +182,8 @@ try {
const status = await message('get_webgpu_download_status', { model });
if (!status?.ready) throw new Error(`${model} is not cached in this profile`);
for (const [index, [expected, language, question, facts]] of rows.entries()) {
- const gate = tinyAskScreenshotGate({ question, ...facts });
+ const { set: _set, earlier: _earlier, ...gateFacts } = facts;
+ const gate = tinyAskScreenshotGate({ question, ...gateFacts });
const started = Date.now();
const reply = await message('webgpu_decide_fixture', { fixture: { ...gate, promptVariant: 'gate', returnHidden } });
const seconds = (Date.now() - started) / 1000;
@@ -152,6 +211,13 @@ try {
latencySeconds: { p50: latency[Math.floor(latency.length / 2)], p95: latency[Math.floor(latency.length * 0.95)] },
misses: all.filter(entry => !entry.correct).map(entry => ({ expected: entry.expected, p: entry.probability, question: entry.question })),
};
+ if (followUps) {
+ result.summary.bySet = Object.fromEntries(['chosen on', 'checked on'].map(set => {
+ const rows = all.filter(entry => entry.set === set);
+ return [set, { yesRecall: share(rows.filter(entry => entry.expected), entry => entry.predicted),
+ falseYes: share(rows.filter(entry => !entry.expected), entry => entry.predicted) }];
+ }));
+ }
console.log(JSON.stringify(result.summary, null, 1));
} catch (error) {
result.error = String(error?.stack || error);
diff --git a/scripts/check-webgpu-ask.mjs b/scripts/check-webgpu-ask.mjs
index 59e333693..e83501287 100644
--- a/scripts/check-webgpu-ask.mjs
+++ b/scripts/check-webgpu-ask.mjs
@@ -118,6 +118,10 @@ const tasks = [
// A vision model also sees the visible page when its own readout says the
// question needs it; text questions (price above) stay text-only.
{ name: 'image-color', page: '/image.html', text: 'What color is the mug in the photo?', expect: /green/i, screenshot: true },
+ // The screenshot is not kept in history: a short visual follow-up looks
+ // again (a lower bar after a screenshot turn), a text one does not.
+ { name: 'image-follow-up', followUp: true, text: 'And the background?', expect: /white/i, screenshot: true },
+ { name: 'image-text-follow-up', followUp: true, text: 'How much does it hold?', expect: /350/, screenshot: false },
{ name: 'image-turkish', page: '/image.html', text: 'Fotoğraftaki kupa ne renk?', expect: /yeşil|green/i, screenshot: true },
{ name: 'image-only-page', page: '/poster.html', text: 'What is on this page?', expect: /orange|circle|sun/i, screenshot: true },
// The decision to look is the model's own readout, so it works in any language.
diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js
index 80d369962..46e07cb12 100644
--- a/src/chrome/src/agent/agent.js
+++ b/src/chrome/src/agent/agent.js
@@ -208,7 +208,7 @@ import { shouldAutoGroupTabs } from '../tab-group-preference.js';
import {
TINY_ASK_ACT_BLOCKED_MESSAGE, TINY_ASK_COMPACT_LINKED_MAX_CHARS, TINY_ASK_COMPACT_PAGE_MAX_CHARS,
TINY_ASK_COMPACT_SELECTION_MAX_CHARS, TINY_ASK_READ_MAX_CHARS, TINY_ASK_READ_WINDOW_CHARS, TINY_ASK_SYSTEM_PROMPT,
- TINY_ASK_SCREENSHOT_GATE_THRESHOLD, TINY_ASK_SCREENSHOT_GATE_TIMEOUT_MS, tinyAskLinkedUrl, tinyAskScreenshotGate, tinyAskScreenshotGateProbability, tinyAskScreenshotNote, tinyAskPageContext, tinyAskPageReadable, tinyAskRuntimeLine,
+ TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD, TINY_ASK_SCREENSHOT_GATE_THRESHOLD, TINY_ASK_SCREENSHOT_GATE_TIMEOUT_MS, tinyAskLinkedUrl, tinyAskScreenshotGate, tinyAskScreenshotGateProbability, tinyAskScreenshotNote, tinyAskPageContext, tinyAskPageReadable, tinyAskRuntimeLine,
} from './tiny-ask.js';
import { webgpuModelDisplayName } from '../providers/webgpu.js';
@@ -11999,7 +11999,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// A tiny Ask turn carries a one-line trusted clock and the page (or PDF)
// text and selection, read by WebBrain before the single model call. Those
// blocks are ephemeral: the persisted turn keeps only the user's question.
- async _enrichTinyAskUserMessage(tabId, userMessage, userAuthoredMetadata = {}, onUpdate = null, { imageAttachment = false } = {}) {
+ async _enrichTinyAskUserMessage(tabId, userMessage, userAuthoredMetadata = {}, onUpdate = null, { imageAttachment = false, followsScreenshot = false } = {}) {
const { tabUrl: url, tabTitle: title } = await this._getTabUrlTitle(tabId);
let page = null;
let selection = '';
@@ -12040,7 +12040,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (linked.unavailable) onUpdate?.('thinking', { step: 0, note: 'Could not read the linked page.' });
}
const screenshot = await this._tinyAskScreenshot(tabId, {
- question: questionText, page, pageReadable, imageAttachment, linkedPage: !!linkedUrl, onUpdate,
+ question: questionText, page, pageReadable, imageAttachment, linkedPage: !!linkedUrl, followsScreenshot, onUpdate,
});
let modelLabel = '';
try {
@@ -12079,6 +12079,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
role: 'user',
content: [block, ...screenshotBlocks, ...question],
...userAuthoredMetadata,
+ // Kept on the stored question (never the image): the next turn's gate
+ // reads it as a follow-up. Model-facing copies drop it.
+ ...(screenshot ? { webbrainTinyAskScreenshot: true } : {}),
};
// A screenshot alone does not ground the turn: offline references are
// still looked up when the question asks for them.
@@ -12127,7 +12130,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// tiny Ask stays on the user's device. Same capture as the agent's
// automatic screenshots (redaction, size budget, per-turn screenshot limit).
// Any failure leaves the turn text-only, as before.
- async _tinyAskScreenshot(tabId, { question = '', page = null, pageReadable = false, imageAttachment = false, linkedPage = false, onUpdate = null } = {}) {
+ async _tinyAskScreenshot(tabId, { question = '', page = null, pageReadable = false, imageAttachment = false, linkedPage = false, followsScreenshot = false, onUpdate = null } = {}) {
// Redaction covers form fields, not text drawn in images or canvases.
// Browser-internal pages cannot be captured; a page whose text read
// failed still can, and is offered to the gate as having no text.
@@ -12153,14 +12156,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
} catch (error) {
if (error?.name === 'AbortError' || this._checkAbort(tabId)) throw error;
}
+ const threshold = followsScreenshot ? TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD : TINY_ASK_SCREENSHOT_GATE_THRESHOLD;
const gateNote = {
probability: Number.isFinite(probability) ? Math.round(probability * 1000) / 1000 : null,
+ threshold,
ms: Date.now() - started,
};
const runId = this.currentRunId.get(tabId);
if (runId) trace.recordNote(runId, null, 'tiny_ask_screenshot_gate', gateNote);
else this.pendingTinyAskGateTraces.set(tabId, gateNote);
- if (!(probability >= TINY_ASK_SCREENSHOT_GATE_THRESHOLD)) return null;
+ if (!(probability >= threshold)) return null;
onUpdate?.('thinking', { step: 0, note: 'Looking at the page…' });
try {
const shot = await this._captureBudgetedAutoScreenshot(tabId);
@@ -12181,7 +12186,12 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const userAuthoredMetadata = runOptions?.scheduledResume === true
? {} : { webbrainUserAuthoredText: this._messageText(userMessage) };
if (this._isTinyAskRun(runOptions)) {
- return this._enrichTinyAskUserMessage(tabId, userMessage, userAuthoredMetadata, onUpdate, { imageAttachment });
+ // The previous turn is the last stored user message (this one is
+ // stored after enrichment).
+ const previous = [...messages].reverse().find(message => message?.role === 'user');
+ return this._enrichTinyAskUserMessage(tabId, userMessage, userAuthoredMetadata, onUpdate, {
+ imageAttachment, followsScreenshot: previous?.webbrainTinyAskScreenshot === true,
+ });
}
// Standalone is intentionally plain chat regardless of provider.
// Do not add runtime state, recording/API guidance, tab metadata, adapter
@@ -12813,6 +12823,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const {
webbrainStandaloneChat: _standalone,
webbrainTinyAsk: _tinyAsk,
+ webbrainTinyAskScreenshot: _tinyAskScreenshot,
tool_calls: _toolCalls,
tool_call_id: _toolCallId,
responseItems: _responseItems,
diff --git a/src/chrome/src/agent/tiny-ask.js b/src/chrome/src/agent/tiny-ask.js
index f64306c3a..6b4b504d1 100644
--- a/src/chrome/src/agent/tiny-ask.js
+++ b/src/chrome/src/agent/tiny-ask.js
@@ -58,6 +58,13 @@ export const TINY_ASK_READ_MAX_CHARS = 60000;
// steer the decision.
export const TINY_ASK_SCREENSHOT_MAX_PAGE_CHARS = 200;
export const TINY_ASK_SCREENSHOT_GATE_THRESHOLD = 0.5;
+// The turn right after one answered from a screenshot: the screenshot is gone
+// from history, and a short follow-up ("And the button?", "Ya logo?") is a
+// visual question more often than a fresh one. The readout still sees only the
+// new message (with the earlier one prepended, it says yes to "Thanks!" too);
+// a lower bar is the prior shift. Chosen on 60 follow-ups in 14 languages and
+// checked on 52 others in 16 (docs/webgpu-models.md).
+export const TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD = 0.35;
// The readout takes ~0.2 s, plus a model load on the first turn after a
// browser restart; past this the turn goes ahead text-only.
export const TINY_ASK_SCREENSHOT_GATE_TIMEOUT_MS = 10_000;
diff --git a/test/run.js b/test/run.js
index 8807c748f..e8e37a99b 100644
--- a/test/run.js
+++ b/test/run.js
@@ -41709,6 +41709,34 @@ test('tiny Ask shows a vision-capable in-browser model the visible page when its
assert.equal(captures.length, 0, 'a No takes no screenshot');
assert.deepEqual(imagesOf(requests[0]), []);
}
+ {
+ // The turn after one answered from a screenshot: the stored question is
+ // marked (never the image), and a short follow-up clears a lower bar. The
+ // readout still sees only the new message. A text-only turn resets it.
+ const { TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD, TINY_ASK_SCREENSHOT_GATE_THRESHOLD } = TinyAskCh;
+ assert.ok(TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD < 0.4 && TINY_ASK_SCREENSHOT_GATE_THRESHOLD > 0.4);
+ const values = [0.9, 0.4, 0.4, 0.1, 0.4];
+ const { agent, tabId, requests, decisions, captures, run } = setup({ gate: () => values[decisions.length - 1] });
+ await run('What colour is the banner?');
+ assert.equal(captures.length, 1);
+ const stored = () => agent.conversations.get(tabId).filter(message => message.role === 'user');
+ assert.equal(stored()[0].webbrainTinyAskScreenshot, true, 'the stored question remembers the screenshot');
+ await run('And the button?');
+ assert.doesNotMatch(decisions[1].state, /banner/, 'the readout sees only the new message');
+ assert.equal(captures.length, 2, 'a follow-up at 0.4 looks again');
+ assert.equal(agent.pendingTinyAskGateTraces.get(tabId)?.threshold, TINY_ASK_SCREENSHOT_FOLLOW_UP_THRESHOLD, 'the trace shows the bar used');
+ assert.deepEqual(imagesOf(requests[1]), ['data:image/jpeg;base64,c2NyZWVu'], 'only this turn\'s screenshot reaches the model');
+ assert.doesNotMatch(JSON.stringify(requests[1]), /webbrainTinyAskScreenshot/, 'the marker never reaches the model');
+ await run('Ya logo?');
+ assert.equal(captures.length, 3, 'a chain of visual follow-ups keeps looking');
+ await run('How much is the blue plan?');
+ assert.equal(captures.length, 3);
+ assert.equal(stored()[3].webbrainTinyAskScreenshot, undefined);
+ await run('And the yellow one?');
+ assert.equal(captures.length, 3, 'after a text-only turn the usual bar applies');
+ assert.equal(agent.pendingTinyAskGateTraces.get(tabId)?.threshold, TINY_ASK_SCREENSHOT_GATE_THRESHOLD);
+ assert.doesNotMatch(JSON.stringify(agent.conversations.get(tabId)), /data:image|c2NyZWVu/, 'no screenshot is persisted');
+ }
{
const { requests, captures, run } = setup({ gate: () => new Error('worker busy') });
assert.equal(await run('What color is the Buy button?'), 'Answer.', 'a failed readout still answers');