Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
fix(webgpu): keep special-token text plain for Compass VLD Tiny v5
Claude5f5490ea1f88
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 34c0286ce..b49bf64ab 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,7 +7,7 @@ This changelog was generated from the repository Git history and release tags. V## [Unreleased]### Added-- Compass VLD Tiny v5, a new in-browser model for Chrome: one 2.2 GB WebGPU download that chats, reads screenshots and answers WebBrain's quick decision questions on your device, with no network once it is downloaded. It is WebBrain's fine-tune of MiniCPM-V-4.7-1B and the new default local model: the first-install guide's **Download local models** button downloads it, instead of D1 and Ternary Bonsai 2 2B Vision, and Apocalypse Mode's Text Model card and Providers → WebGPU list it first. A WebGPU provider that was never set up moves to it when Compass Tiny v2.1, the previous default, was never downloaded on that device. Its files are fetched at a fixed revision and each is checked against a pinned size and SHA-256 before it is used; a dropped connection retries that file. Compass Tiny v2.1 and Ternary Bonsai 2 2B Vision stay available.+- Compass VLD Tiny v5, a new in-browser model for Chrome: one 2.2 GB WebGPU download that chats, reads screenshots and answers WebBrain's quick decision questions on your device, with no network once it is downloaded. It is WebBrain's fine-tune of MiniCPM-V-4.7-1B and the new default local model: the first-install guide's **Download local models** button downloads it, instead of D1 and Ternary Bonsai 2 2B Vision, and Apocalypse Mode's Text Model card and Providers → WebGPU list it first. Once, on the first start after the update, a WebGPU provider that was never set up moves to it when Compass Tiny v2.1, the previous default, was never downloaded on that device. Its files are fetched at a fixed revision and each is checked against a pinned size and SHA-256 before it is used; a dropped connection retries that file. Compass Tiny v2.1 and Ternary Bonsai 2 2B Vision stay available.### Changed- Decision models, Auto (Chrome): once Compass VLD Tiny v5 is downloaded, it checks whether tasks and scheduled runs are finished, before a downloaded D1 and in place of WebBrain Cloud's decision model, which still answers what a request asks for. As with D1, a "finished" from it is confirmed by the chat model. On d1's synthetic test, through the extension, it answered 120 of 126 questions correctly and called none of the 48 unfinished views finished; D1 answered 92 and called one finished. **In-browser chat model** under On can be chosen again and uses it. Ternary Bonsai 2 2B Vision no longer answers decisions, and a saved selection of it now uses Compass VLD Tiny v5. Firefox is unchanged.diff --git a/docs/providers-and-models.md b/docs/providers-and-models.mdindex 1a496c159..739cf6d70 100644--- a/docs/providers-and-models.md+++ b/docs/providers-and-models.md@@ -564,9 +564,12 @@ browser actions. The standalone chat window keeps its own page-free profile.processor ported from PyTorch. It runs greedy with thinking disabled andemits Qwen3.5 XML tool calls (`<tool_call><function=...><parameter=...>`),which the local fallback parser accepts. Files are fetched at a pinned- revision and each is checked against its pinned size and SHA-256. A profile- that never set up WebGPU moves from the previous default, Compass Tiny v2.1,- to it when Compass Tiny v2.1 was never downloaded there.+ revision and each is checked against its pinned size and SHA-256. Message+ text that spells a special token (`<|image_pad|>`, `<|im_end|>`) is encoded+ as ordinary text. On the first start after the update, a profile that never+ set up WebGPU moves from the previous default, Compass Tiny v2.1, to it when+ Compass Tiny v2.1 was never downloaded there; choosing Compass Tiny v2.1+ later stays.- [`webbrain-one/webbrain-compass-tiny-v2.1`](https://huggingface.co/webbrain-one/webbrain-compass-tiny-v2.1)(`q4f16`, about 1.87 GB across two external-data shards), WebBrain's CompassTiny v2.1 fine-tune of MiniCPM5-2B for Compact tool routing. It runs greedydiff --git a/docs/webgpu-models.md b/docs/webgpu-models.mdindex 5233cd741..05177f052 100644--- a/docs/webgpu-models.md+++ b/docs/webgpu-models.md@@ -189,13 +189,19 @@ Ask) and decisions. Its training data and evaluations are on the model card.`tokenizer_config.json` at revision `main`: the pinned download nevercaches that, so the lookup would go to Hugging Face on every load and failoffline. With the published files it renders the reference's prompts and- produces its token ids exactly. Decoding is greedy with thinking off; tool- calls use Qwen3.5's XML format.+ produces its token ids exactly. Message text, tool descriptions and decision+ questions that spell one of its special tokens get a zero-width space after+ the first character, so a page quoting `<|image_pad|>` or `<|im_end|>`+ cannot add an image slot or end a turn. Decoding is greedy with thinking+ off; tool calls use Qwen3.5's XML format.- **Download.** Files are read at a pinned commit (`COMPASS_VLD_REVISION`),one at a time. Each is stored only once it arrived whole and is thenchecked against its pinned size and SHA-256; a mismatch removes it andfails the download. A dropped connection or a server error retries that- file twice. Inference reads only the cache.+ file twice. Inference reads only the cache. The ready marker does not name+ the revision, so readiness also checks that every pinned file is cached;+ after a new pin the model reads as not downloaded until Download fetches+ the new files, and a finished download removes the earlier revision's.- **Checks on the published files** (RTX 5090, Chromium through Playwright):- `audit_webgpu_onnx.py` passes for every graph: no `Scan` or `Loop`, anddecoder logits of shape `[batch, 1, 248094]`.diff --git a/src/chrome/src/offscreen/compass-vld-runtime.js b/src/chrome/src/offscreen/compass-vld-runtime.jsindex 03a6743bb..4a79aa477 100644--- a/src/chrome/src/offscreen/compass-vld-runtime.js+++ b/src/chrome/src/offscreen/compass-vld-runtime.js@@ -151,6 +151,20 @@ export async function downloadCompassVld({}onProgress({ status: 'done', file: file.path, loaded: file.bytes, total: file.bytes });}+ await removeOtherCompassVldRevisions(cache, modelId);+}++/** Delete this model's files from any other revision, left by an earlier pin. */+export async function removeOtherCompassVldRevisions(cache, modelId, revision = COMPASS_VLD_REVISION) {+ if (typeof cache.keys !== 'function') return 0;+ const base = `https://huggingface.co/${modelId}/resolve/`;+ const current = `${base}${encodeURIComponent(revision)}/`;+ let removed = 0;+ for (const request of await cache.keys()) {+ const url = typeof request === 'string' ? request : request.url;+ if (url.startsWith(base) && !url.startsWith(current) && await cache.delete(request)) removed += 1;+ }+ return removed;}async function fetchIntoCache(cache, fetchImpl, url, file, onProgress) {@@ -640,6 +654,23 @@ export function argmax(values) {return best;}+/**+ * Text that spells one of the tokenizer's special tokens (`<|image_pad|>`,+ * `<|im_end|>`, `<slice>`, …) would be encoded as that token: an image slot+ * with no image behind it, or a turn boundary that page text could inject.+ * The returned function puts a zero-width space after the first character of+ * each such spelling, so it stays readable and is encoded as ordinary text.+ * It is for message text before the chat template renders it; the template's+ * own special tokens are added afterwards.+ */+export function plainTextFor(specialTokens = []) {+ const tokens = [...new Set(Array.from(specialTokens, String))].filter(token => token.length > 1)+ .sort((a, b) => b.length - a.length);+ if (!tokens.length) return text => String(text ?? '');+ const pattern = new RegExp(tokens.map(token => token.replace(/[\\^$.*+?()[\]{}|/-]/g, '\\$&')).join('|'), 'g');+ return text => String(text ?? '').replace(pattern, token => `${token[0]}\u200b${token.slice(1)}`);+}+export class CompassVldRuntime {/*** @param ort onnxruntime-web (the WebGPU build Transformers.js uses)@@ -654,6 +685,17 @@ export class CompassVldRuntime {this.decoder = sessions.decoder_model_merged;this.cfg = cfg;this.layerKinds = Array.from({ length: cfg.layers }, (_, i) => ((i + 1) % cfg.fullAttentionInterval === 0 ? 'attn' : 'gdn'));+ this.plainText = plainTextFor(tokenizer?.all_special_tokens || []);+ }++ /** A copy of messages or tools with every string passed through plainText. */+ plainStrings(value) {+ if (typeof value === 'string') return this.plainText(value);+ if (Array.isArray(value)) return value.map(item => this.plainStrings(item));+ if (value && typeof value === 'object' && Object.getPrototypeOf(value) === Object.prototype) {+ return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, this.plainStrings(item)]));+ }+ return value;}async dispose() {@@ -706,6 +748,12 @@ export class CompassVldRuntime {async buildInputs(promptText, images = [], { beforeImage = null } = {}) {const text = images.length ? expandImagePlaceholders(promptText, images) : promptText;const ids = this.encode(text);+ // Message text must reach here through plainText; an image token it spelled+ // would otherwise be taken for an image patch.+ const div = this.cfg.mergeFactor * this.cfg.mergeFactor;+ const patches = images.reduce((sum, image) => sum + image.crops.reduce((n, crop) => n + (crop.h * crop.w) / div, 0), 0);+ const pads = ids.reduce((sum, id) => sum + (id === this.cfg.imageTokenId ? 1 : 0), 0);+ if (pads !== patches) throw new Error(`The prompt has ${pads} image tokens for ${patches} image patches.`);const embeds = await this.embedIds(ids);if (images.length) {const features = [];@@ -775,19 +823,24 @@ export class CompassVldRuntime {const data = tensor.location === 'gpu-buffer' ? await tensor.getData() : tensor.data;return data instanceof Uint16Array ? f16ToF32(data) : (data instanceof Float32Array ? data : Float32Array.from(data));};+ const present = Object.entries(out).filter(([name]) => name.startsWith('present'));+ let kept = false;try {const logits = await read(out.logits);const hidden = wantHidden && out.decision_hidden ? await read(out.decision_hidden) : null;const next = { feeds: {}, length: total };- for (const [name, tensor] of Object.entries(out)) {- if (!name.startsWith('present')) continue;- if (keepCache) next.feeds[name.replace('present_conv', 'past_conv').replace('present_recurrent', 'past_recurrent').replace('present.', 'past_key_values.')] = tensor;- else tensor.dispose?.();+ if (keepCache) {+ for (const [name, tensor] of present) {+ next.feeds[name.replace('present_conv', 'past_conv').replace('present_recurrent', 'past_recurrent').replace('present.', 'past_key_values.')] = tensor;+ }+ kept = true;}return { logits, hidden, cache: next };} finally {out.logits?.dispose?.();out.decision_hidden?.dispose?.();+ // The new cache is released unless it was handed back, also when a read failed.+ if (!kept) for (const [, tensor] of present) tensor.dispose?.();}}@@ -858,6 +911,7 @@ export class CompassVldRuntime {/** Logits (and decision_hidden) after appending `text` to a prefilled prefix, leaving the prefix cache as it was. */async continuation(prefix, prefixCache, text) {const suffix = this.encode(text);+ if (suffix.includes(this.cfg.imageTokenId)) throw new Error('The continuation text has an image token; pass it through plainText.');const pos = canvasPositions(prefix.ids.concat(suffix), prefix.sizes, this.cfg);const P = prefix.ids.length;const embeds = await this.embedIds(suffix);diff --git a/src/chrome/src/offscreen/inference-worker.js b/src/chrome/src/offscreen/inference-worker.jsindex a39de1de8..63db4d4cf 100644--- a/src/chrome/src/offscreen/inference-worker.js+++ b/src/chrome/src/offscreen/inference-worker.js@@ -660,6 +660,13 @@ async function isTextModelReady(modelId, dtype) {const cache = await caches.open(TRANSFORMERS_CACHE_NAME);const marker = await cache.match(textReadyMarkerUrl(modelId, dtype));if (!marker) return false;+ // The marker does not name Compass VLD Tiny v5's pinned revision. When its+ // files are not all cached (a newer pin), drop the marker so the model+ // reads as not downloaded and Download fetches the missing files.+ if (isCompassVldModel(modelId) && !await compassVldFilesCached(cache, modelId, dtype)) {+ await cache.delete(textReadyMarkerUrl(modelId, dtype));+ return false;+ }readyTextModelKeys.add(key);return true;} catch {@@ -1913,11 +1920,12 @@ async function runCompassVldText(payload) {if (payload?.requireTools === true) assertToolCapableTextRuntime(runtime, modelId);const tools = Array.isArray(payload?.options?.tools) ? payload.options.tools : [];const { messages, imageUrls } = prepareMultimodalMessages(payload?.messages);- const prompt = runtime.tokenizer.apply_chat_template(messages, {+ // Page text and tool descriptions stay text even where they spell a special token.+ const prompt = runtime.tokenizer.apply_chat_template(runtime.compassVld.plainStrings(messages), {add_generation_prompt: true,tokenize: false,enable_thinking: false,- tools: tools.length ? tools : undefined,+ tools: tools.length ? runtime.compassVld.plainStrings(tools) : undefined,});const stopIfCancelled = () => throwIfTextCancelled(requestKey);const images = await loadCompassVldImages(runtime, imageUrls, stopIfCancelled);@@ -2089,11 +2097,15 @@ async function runCompassVldDecision(payload, { isCancelled = () => false } = {}if (variant === 'probe' && modelId !== COMPASS_VLD_DECISION_PROBE.model) {throw new Error(`No decision probe is trained for ${modelId}.`);}+ const engine = runtime.compassVld;+ // The page state, the user's message and the questions stay text even where+ // they spell a special token.+ const plain = text => engine.plainText(text);const messages = [{ role: 'system', content: [{ type: 'text', text: decisionSystemPrompt(payload?.promptVariant) }] },{ role: 'user', content: [...request.images.map(() => ({ type: 'image' })),- { type: 'text', text: `${variant === 'gate' ? String(request.state || '') : decisionStateText(request.state)}\n\n${DECISION_QUESTION_MARKER}` },+ { type: 'text', text: `${plain(variant === 'gate' ? String(request.state || '') : decisionStateText(request.state))}\n\n${DECISION_QUESTION_MARKER}` },] },];const rendered = runtime.tokenizer.apply_chat_template(messages, { add_generation_prompt: true, enable_thinking: false, tokenize: false });@@ -2103,7 +2115,6 @@ async function runCompassVldDecision(payload, { isCancelled = () => false } = {}}const prefixText = rendered.slice(0, split);const tailText = rendered.slice(split + DECISION_QUESTION_MARKER.length);- const engine = runtime.compassVld;const codeIds = new Map();const codeId = code => {if (!codeIds.has(code)) {@@ -2139,7 +2150,7 @@ async function runCompassVldDecision(payload, { isCancelled = () => false } = {}const count = decisionOptions(question).names.length;for (let index = 0; index < count; index++) {stopIfCancelled();- const output = await engine.continuation(prefix, prefill.cache, `${decisionVerifyText(question, index)}${tailText}`);+ const output = await engine.continuation(prefix, prefill.cache, `${plain(decisionVerifyText(question, index))}${tailText}`);suffixTokens += output.tokens;margins.push(output.logits[yesId] - output.logits[noId]);hidden.push(output.hidden);@@ -2155,7 +2166,7 @@ async function runCompassVldDecision(payload, { isCancelled = () => false } = {}if (variant === 'gate') {if (question.type !== 'noul') throw new Error('The screenshot gate takes Yes/No questions only.');stopIfCancelled();- const output = await engine.continuation(prefix, prefill.cache, `${decisionQuestionText(question)}${tailText}`);+ const output = await engine.continuation(prefix, prefill.cache, `${plain(decisionQuestionText(question))}${tailText}`);suffixTokens += output.tokens;const margin = output.logits[yesId] - output.logits[noId];const probability = tinyAskGateProbability(modelId, margin, output.hidden);@@ -2173,7 +2184,7 @@ async function runCompassVldDecision(payload, { isCancelled = () => false } = {}continue;}stopIfCancelled();- const output = await engine.continuation(prefix, prefill.cache, `${decisionQuestionText(question)}${tailText}`);+ const output = await engine.continuation(prefix, prefill.cache, `${plain(decisionQuestionText(question))}${tailText}`);suffixTokens += output.tokens;const logits = output.logits;const ids = decisionOptions(question).codes.map(codeId);diff --git a/src/chrome/src/providers/manager.js b/src/chrome/src/providers/manager.jsindex c71d24fe4..4a1120f4f 100644--- a/src/chrome/src/providers/manager.js+++ b/src/chrome/src/providers/manager.js@@ -104,6 +104,8 @@ const WEBBRAIN_CLOUD_CONTEXT_WINDOW = 1000000;const WEBBRAIN_CLOUD_LEGACY_CONTEXT_WINDOW = 256000;const WEBBRAIN_DEVICE_GUID_KEY = 'webbrainDeviceGuid';const HELP_IMPROVE_WEBBRAIN_KEY = 'helpImproveWebBrain';+// The WebGPU default the untouched-default migration last decided for.+const WEBGPU_DEFAULT_MIGRATION_KEY = 'webgpuDefaultMigration';const OPENROUTER_DEFAULT_MODEL = 'openrouter/free';const OPENROUTER_LEGACY_DEFAULT_MODEL = 'stepfun/step-3.7-flash';const OPENAI_DEFAULT_MODEL = 'gpt-5.6-terra';@@ -2004,21 +2006,33 @@ export class ProviderManager {* Older releases stored every built-in provider's defaults, so a stored* Compass Tiny v2.1 is not a choice by itself. Only a configuration nobody* set up moves: not marked configured, not the active provider, and Compass- * Tiny v2.1 never downloaded here (an unreadable cache keeps it). Returns- * whether it moved.+ * Tiny v2.1 never downloaded here. This is decided once; an unreadable cache+ * leaves it for a later start. Choosing Compass Tiny v2.1 afterwards stays,+ * even where that choice is saved without marking the provider configured+ * (Apocalypse Mode's model list). Returns whether it moved.*/async migrateUntouchedWebgpuDefault() {+ const stored = await chrome.storage.local.get(WEBGPU_DEFAULT_MIGRATION_KEY);+ if (stored?.[WEBGPU_DEFAULT_MIGRATION_KEY] === WEBGPU_DEFAULT_MODEL_ID) return false;const config = this.providers.get('webgpu')?.config;- if (!config || config.configured === true || this.activeProviderId === 'webgpu') return false;- if (String(config.model || '').trim() !== WEBGPU_COMPASS_TINY_V2_MODEL_ID) return false;- if (await hasWebgpuTextModelCache(WEBGPU_COMPASS_TINY_V2_MODEL_ID) !== false) return false;- await this.updateProvider('webgpu', {- model: WEBGPU_DEFAULT_MODEL_ID,- dtype: webgpuModelDtype(WEBGPU_DEFAULT_MODEL_ID),- contextWindow: 32768,- supportsVision: true,- }, { markConfigured: false });- return true;+ if (!config) return false;+ let moved = false;+ if (config.configured !== true && this.activeProviderId !== 'webgpu'+ && String(config.model || '').trim() === WEBGPU_COMPASS_TINY_V2_MODEL_ID) {+ const cached = await hasWebgpuTextModelCache(WEBGPU_COMPASS_TINY_V2_MODEL_ID);+ if (cached === null) return false;+ if (cached === false) {+ await this.updateProvider('webgpu', {+ model: WEBGPU_DEFAULT_MODEL_ID,+ dtype: webgpuModelDtype(WEBGPU_DEFAULT_MODEL_ID),+ contextWindow: 32768,+ supportsVision: true,+ }, { markConfigured: false });+ moved = true;+ }+ }+ await chrome.storage.local.set({ [WEBGPU_DEFAULT_MIGRATION_KEY]: WEBGPU_DEFAULT_MODEL_ID });+ return moved;}/** Start the selected shipped Apocalypse chat model, defaulting to Compass VLD Tiny v5. */diff --git a/test/compass-vld-runtime.mjs b/test/compass-vld-runtime.mjsindex 4905b2382..3bf4cafd9 100644--- a/test/compass-vld-runtime.mjs+++ b/test/compass-vld-runtime.mjs@@ -87,7 +87,8 @@ function memoryCache() {entries,async match(url) { const bytes = entries.get(url); return bytes ? new Response(bytes, { headers: { 'content-length': String(bytes.byteLength) } }) : undefined; },async put(url, response) { entries.set(url, new Uint8Array(await response.arrayBuffer())); },- async delete(url) { return entries.delete(url); },+ async delete(key) { return entries.delete(typeof key === 'string' ? key : key.url); },+ async keys() { return [...entries.keys()].map(url => new Request(url)); },};}const fileOf = (path, text) => {@@ -111,8 +112,15 @@ test('a download stores each file only after its size and SHA-256 match, and skiassert.deepEqual(events.filter(event => event.status === 'done').map(event => event.file), files.map(file => file.path));assert.ok(events.some(event => event.status === 'progress' && event.loaded > 0 && event.total === files[1].bytes), 'progress counts bytes as they arrive');fetched.length = 0;+ const earlier = runtime.compassVldFileUrl(MODEL, 'onnx/model.onnx_data', '0'.repeat(40));+ const otherModel = 'https://huggingface.co/owner/other/resolve/main/config.json';+ cache.entries.set(earlier, new Uint8Array([1]));+ cache.entries.set(otherModel, new Uint8Array([1]));await runtime.downloadCompassVld({ cache, fetchImpl, modelId: MODEL, files });assert.equal(fetched.length, 0, 'cached files are not fetched again');+ assert.equal(cache.entries.has(earlier), false, 'a download removes the files of an earlier pinned revision');+ assert.equal(cache.entries.has(otherModel), true, "other models' files stay");+ assert.equal(await runtime.compassVldFilesCached(cache, MODEL, undefined, files), true);const corrupt = memoryCache();await assert.rejects(runtime.downloadCompassVld({ cache: corrupt, modelId: MODEL, files,@@ -216,3 +224,81 @@ test('image pixels become RGB, alpha dropped and grey expanded, before the proceassert.deepEqual(Array.from(runtime.f16ToF32(new Uint16Array([0x3c00, 0xc000, 0x0001, 0x7c00, 0x0000]))), [1, -2, 2 ** -24, Infinity, 0]);assert.equal(runtime.argmax([0.1, 3, -2, 3]), 1, 'the first of equal maxima, as greedy decoding expects');});++// A byte-level BPE tokenizer with no merges and two special tokens, enough to+// encode any text the way the model's tokenizer splits special tokens out.+async function byteLevelTokenizer() {+ const library = await import('../src/chrome/vendor/transformers/transformers.web.js');+ const bytes = [];+ for (let b = 33; b <= 126; b++) bytes.push(b);+ for (let b = 161; b <= 172; b++) bytes.push(b);+ for (let b = 174; b <= 255; b++) bytes.push(b);+ const chars = bytes.slice();+ for (let b = 0, n = 0; b < 256; b++) if (!bytes.includes(b)) { bytes.push(b); chars.push(256 + n++); }+ const vocab = Object.fromEntries(chars.map((code, index) => [String.fromCharCode(code), index]));+ const special = (id, content) => ({ id, content, single_word: false, lstrip: false, rstrip: false, normalized: false, special: true });+ return new library.Qwen2Tokenizer({+ version: '1.0', truncation: null, padding: null, normalizer: null, post_processor: null,+ added_tokens: [special(256, '<|image_pad|>'), special(257, '<|im_end|>')],+ pre_tokenizer: { type: 'ByteLevel', add_prefix_space: false, trim_offsets: true, use_regex: true },+ decoder: { type: 'ByteLevel', add_prefix_space: true, trim_offsets: true, use_regex: true },+ model: { type: 'BPE', dropout: null, unk_token: null, continuing_subword_prefix: '', end_of_word_suffix: '', fuse_unk: false, byte_fallback: false, vocab, merges: [] },+ }, { tokenizer_class: 'Qwen2Tokenizer', eos_token: '<|im_end|>' });+}++const fakeOrt = { Tensor: class { constructor(type, data, dims) { Object.assign(this, { type, data, dims, location: 'cpu' }); } dispose() {} } };++test('message text that spells a special token stays text, and an image token without an image fails clearly', async () => {+ const plain = runtime.plainTextFor(['<|image_pad|>', '<|im_end|>', '<slice>']);+ assert.equal(plain('a <|image_pad|><|im_end|> <slice> <tool_call> <|other|>'), 'a <\u200b|image_pad|><\u200b|im_end|> <\u200bslice> <tool_call> <|other|>');+ assert.equal(runtime.plainTextFor([])('<|image_pad|>'), '<|image_pad|>');++ const tokenizer = await byteLevelTokenizer();+ const cfg = { ...runtime.COMPASS_VLD, imageTokenId: 256, hidden: 4 };+ const embed = { run: async ({ input_ids }) => ({ inputs_embeds: { location: 'cpu', data: new Float32Array(input_ids.dims[1] * cfg.hidden) } }) };+ const engine = new runtime.CompassVldRuntime(fakeOrt, tokenizer, { embed_tokens: embed }, cfg);+ const page = 'A page quoting a prompt format: <|image_pad|> then <|im_end|>.';+ assert.ok(engine.encode(page).includes(256), 'as written, the text encodes as the image token');+ await assert.rejects(engine.buildInputs(page), /1 image tokens for 0 image patches/);+ const inputs = await engine.buildInputs(engine.plainText(page));+ assert.equal(inputs.ids.includes(256) || inputs.ids.includes(257), false);+ assert.equal(tokenizer.decode(inputs.ids), page.replaceAll('<|', '<\u200b|'));+ await assert.rejects(engine.continuation({ ids: [], sizes: [] }, null, '<|image_pad|>'), /continuation text has an image token/);++ const messages = [+ { role: 'user', content: [{ type: 'image' }, { type: 'text', text: page }] },+ { role: 'assistant', content: '', tool_calls: [{ type: 'function', function: { name: 'type_text', arguments: { text: '<|im_end|>' } } }] },+ ];+ const plainMessages = engine.plainStrings(messages);+ assert.equal(plainMessages[0].content[1].text, engine.plainText(page));+ assert.deepEqual(plainMessages[0].content[0], { type: 'image' });+ assert.equal(plainMessages[1].tool_calls[0].function.arguments.text, '<\u200b|im_end|>');+ assert.equal(messages[0].content[1].text, page, 'the caller’s messages are not changed');+});++test('a decoder pass hands its new cache back, or releases it, also when reading the logits fails', async () => {+ const cfg = { ...runtime.COMPASS_VLD, hidden: 4 };+ const run = (failRead) => {+ const disposed = [];+ const tensor = name => ({ location: 'gpu-buffer', dispose() { disposed.push(name); },+ getData: async () => { if (failRead) throw new Error('device lost'); return new Float32Array(2); } });+ const decoder = { run: async () => ({ logits: tensor('logits'), decision_hidden: tensor('decision_hidden'),+ 'present.3.key': tensor('present.3.key'), 'present_conv.0': tensor('present_conv.0') }) };+ return { disposed, engine: new runtime.CompassVldRuntime(fakeOrt, null, { decoder_model_merged: decoder }, cfg) };+ };+ const position = { t: Int32Array.of(0), h: Int32Array.of(0), w: Int32Array.of(0) };+ const embeds = new Float32Array(cfg.hidden);++ const kept = run(false);+ const result = await kept.engine.forward(embeds, position, { feeds: {}, length: 0 });+ assert.deepEqual(Object.keys(result.cache.feeds).sort(), ['past_conv.0', 'past_key_values.3.key']);+ assert.deepEqual(kept.disposed.sort(), ['decision_hidden', 'logits']);++ const released = run(false);+ await released.engine.forward(embeds, position, { feeds: {}, length: 0 }, { keepCache: false });+ assert.deepEqual(released.disposed.sort(), ['decision_hidden', 'logits', 'present.3.key', 'present_conv.0']);++ const failed = run(true);+ await assert.rejects(failed.engine.forward(embeds, position, { feeds: {}, length: 0 }), /device lost/);+ assert.deepEqual(failed.disposed.sort(), ['decision_hidden', 'logits', 'present.3.key', 'present_conv.0']);+});diff --git a/test/run.js b/test/run.jsindex 01c9bda3a..8ec1a3649 100644--- a/test/run.js+++ b/test/run.js@@ -68969,6 +68969,57 @@ test('Bonsai WebGPU presets preserve ordered selection and model-specific imageassert.equal(manager.getAll().webgpu.promptTier, 'mid');});+test('an untouched Compass Tiny v2.1 WebGPU setting moves to Compass VLD Tiny v5 once, and a later choice of v2.1 stays', async () => {+ const previousChrome = globalThis.chrome;+ const previousCaches = globalThis.caches;+ const local = {};+ const cachedModels = new Set();+ globalThis.chrome = { storage: { local: {+ get: async key => (Object.hasOwn(local, key) ? { [key]: local[key] } : {}),+ set: async patch => { Object.assign(local, patch); },+ } } };+ const readableCaches = { match: async url => ([...cachedModels].some(id => decodeURIComponent(url).includes(`/${id}|`)) ? new Response('{}') : undefined) };+ // A profile whose WebGPU provider holds Compass Tiny v2.1, never marked configured.+ const profile = async () => {+ const manager = new ProviderManagerCh();+ manager.save = async () => {};+ manager.providers.set('webgpu', new WebGPUProvider(manager._defaultConfigs().webgpu));+ await manager.updateProvider('webgpu', { model: WEBGPU_COMPASS_TINY_V2_MODEL_ID }, { markConfigured: false });+ assert.notEqual(manager.getAll().webgpu.configured, true);+ return manager;+ };+ try {+ globalThis.caches = undefined;+ const unreadable = await profile();+ assert.equal(await unreadable.migrateUntouchedWebgpuDefault(), false);+ assert.equal(unreadable.getAll().webgpu.model, WEBGPU_COMPASS_TINY_V2_MODEL_ID);+ assert.deepEqual(local, {}, 'an unreadable cache leaves the decision for a later start');++ globalThis.caches = readableCaches;+ const untouched = await profile();+ assert.equal(await untouched.migrateUntouchedWebgpuDefault(), true);+ assert.equal(untouched.getAll().webgpu.model, WEBGPU_COMPASS_VLD_TINY_V5_MODEL_ID);+ assert.deepEqual(untouched.getAll().webgpu.dtype, WEBGPU_COMPASS_VLD_DTYPE);+ assert.notEqual(untouched.getAll().webgpu.configured, true, 'the move does not mark the provider configured');++ // Picking Compass Tiny v2.1 in Apocalypse Mode saves it unconfigured; a+ // service worker restart must not move it again.+ const chosen = await profile();+ assert.equal(await chosen.migrateUntouchedWebgpuDefault(), false);+ assert.equal(chosen.getAll().webgpu.model, WEBGPU_COMPASS_TINY_V2_MODEL_ID);++ for (const key of Object.keys(local)) delete local[key];+ cachedModels.add(WEBGPU_COMPASS_TINY_V2_MODEL_ID);+ const downloaded = await profile();+ assert.equal(await downloaded.migrateUntouchedWebgpuDefault(), false, 'a downloaded Compass Tiny v2.1 stays');+ assert.equal(downloaded.getAll().webgpu.model, WEBGPU_COMPASS_TINY_V2_MODEL_ID);+ assert.equal(local.webgpuDefaultMigration, WEBGPU_COMPASS_VLD_TINY_V5_MODEL_ID, 'that decision is final too');+ } finally {+ globalThis.chrome = previousChrome;+ if (previousCaches === undefined) delete globalThis.caches; else globalThis.caches = previousCaches;+ }+});+test('Apocalypse hydrates every selectable Bonsai preset and inspects sibling cache state without changing selection', async () => {const source = fs.readFileSync(path.join(ROOT, 'src/chrome/src/ui/apocalypse-mode.js'), 'utf8');const functions = source.slice(source.indexOf('async function refreshWebgpuDownloadStatus('), source.indexOf('async function onWebgpuTextPresetChange('));diff --git a/test/webgpu-decisions.mjs b/test/webgpu-decisions.mjsindex 297d18a5c..a46653853 100644--- a/test/webgpu-decisions.mjs+++ b/test/webgpu-decisions.mjs@@ -455,4 +455,13 @@ test('the screenshot gate probes never fall back to the untrained readout', asynconst compassVld = worker.slice(worker.indexOf('async function runCompassVldDecision'), worker.indexOf('async function runMultimodalDecision'));assert.match(compassVld, /const probability = tinyAskGateProbability\(modelId, margin, output\.hidden\);\n[\s\S]*?if \(probability === null && !\(returnHidden && output\.hidden\)\) \{\n\s*throw new Error\(`No screenshot gate probe is trained for/,'Compass VLD Tiny v5 uses its own probe the same way');+ // Page text, the user's message and the questions that spell a special token+ // (`<|image_pad|>`, `<|im_end|>`) must reach Compass VLD Tiny v5 as text.+ assert.match(compassVld, /text: `\$\{plain\(variant === 'gate' \? String\(request\.state \|\| ''\) : decisionStateText\(request\.state\)\)\}\\n\\n\$\{DECISION_QUESTION_MARKER\}`/);+ assert.equal((compassVld.match(/engine\.continuation\(prefix, prefill\.cache, `\$\{plain\(decision(?:Verify|Question)Text\(question(?:, index)?\)\)\}\$\{tailText\}`\)/g) || []).length, 3,+ 'every question continuation is plain text');+ assert.doesNotMatch(compassVld, /engine\.continuation\(prefix, prefill\.cache, `\$\{decision/);+ const compassVldText = worker.slice(worker.indexOf('async function runCompassVldText'), worker.indexOf('async function runMultimodalText'));+ assert.match(compassVldText, /apply_chat_template\(runtime\.compassVld\.plainStrings\(messages\), \{[\s\S]*?tools: tools\.length \? runtime\.compassVld\.plainStrings\(tools\) : undefined,/,+ 'chat messages and tool descriptions are plain text too');});