Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
d1: pin the FP16-weight package (0.95 GB) and accept the 128 MiB default binding limit
Claudef18499f2602b
diff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.mdindex 24d343f50..1a46ae0e9 100644--- a/docs/privacy-and-data-flow.md+++ b/docs/privacy-and-data-flow.md@@ -121,13 +121,13 @@ a decision model the turn starts in Ask; only when the active model asks tohand the turn to Act or Dev is the same page-free question sent to the activemodel to confirm it.-The optional experimental **D1 FP32 WebGPU** decision provider has no inference+The optional experimental **D1 WebGPU** decision provider has no inferenceendpoint or API key. Its bounded text/JSON state, questions and inline screenshotsstay in the local decision Worker. Explicit download consent permits only pinned-ONNX/config/tokenizer downloads (about 1.9 GB) from Hugging Face, verified against-the packaged manifest and checksums. The roughly 3.84 GB published package also-contains a full checkpoint for external reproducibility, but the browser never-downloads `model.safetensors`. JS/WASM stays bundled with the extension. Cache and+ONNX/config/tokenizer downloads (about 0.95 GB) from Hugging Face, verified against+the packaged manifest and checksums. The full FP32 checkpoint stays in the+separate FP32 repository for external reproducibility; the browser never+downloads it. JS/WASM stays bundled with the extension. Cache andconsent are local and device-bound; no download is triggered by a decision call.Usage traces use the existing bounded model/provider/token/cost metadata, not anadditional copy of D1 state or pixels. Existing separately configured trace anddiff --git a/docs/providers-and-models.md b/docs/providers-and-models.mdindex cb1a48e91..dbba608f6 100644--- a/docs/providers-and-models.md+++ b/docs/providers-and-models.md@@ -71,20 +71,30 @@ Image support follows available model capabilities or the custom override. TestConnection checks typed answers and, when enabled, two independent image-onlyfacts. A successful HTTP response alone does not establish image support.-**D1 FP32 WebGPU (experimental)** is a separate, optional on-device decision+**D1 WebGPU (experimental)** is a separate, optional on-device decisionprovider. It does not replace the conversational/planning model and does notgenerate tokens. It supports named choice, noul (P(yes)) and ordinal scorequestions over bounded text/JSON state and inline screenshots. Selecting itdoes not download anything. Explicit consent and **Download and load** fetch-about 1.9 GB of pinned FP32 model data and tokenizer/configuration files from-the [Hugging Face model repository](https://huggingface.co/webbrain-one/d1-browser-decision-fp32)-into OPFS. The repository name is not a production-readiness claim; this provider+about 0.95 GB of pinned model data and tokenizer/configuration files from+the [Hugging Face model repository](https://huggingface.co/webbrain-one/d1-browser-decision-fp16)+into OPFS. The package stores FP16 weights and computes in FP32; it is the+[FP32 release](https://huggingface.co/webbrain-one/d1-browser-decision-fp32)+with its weights rounded once, and answers the same on its held-out test. A+cached FP32 package from an earlier version is removed on the next download. The repository name is not a production-readiness claim; this providerremains experimental. Executable JS/WASM is packaged in the extension, notdownloaded from the model repository. **Load cached model** verifies the filesagain; inference cannot trigger a model download.This experimental path needs hardware WebGPU, OPFS and substantial GPU/host-memory. Software adapters are rejected. Chrome uses the shared MV3 offscreen+memory. Software adapters are rejected. The encoder's attention is dense, so+memory grows with the square of each row's length: rows are batched under a+256 MiB attention budget (never more than the adapter's storage-binding+limit), text-only rows are capped at 2,048 tokens on WebGPU (the state is+truncated, as the source does at its context limit), and a request whose+screenshots alone exceed the adapter limit fails with a clear error instead of+losing the GPU device. Screenshot features stay on the GPU between the vision,+projector and decision graphs. Chrome uses the shared MV3 offscreendocument with a dedicated decision Worker; Firefox uses its background-pageWorker when the corresponding browser capabilities are available. Mirroredcode is not evidence that every browser/GPU is supported. CPU/WASM control anddiff --git a/src/chrome/src/agent/decision-config.js b/src/chrome/src/agent/decision-config.jsindex 2a77a6e87..eb9616e3f 100644--- a/src/chrome/src/agent/decision-config.js+++ b/src/chrome/src/agent/decision-config.js@@ -49,7 +49,7 @@ export function resolveDecisionConfig(stored = {}, compass = null) {}export async function listDecisionModels(config, fetchImpl = globalThis.fetch, signal) {- if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP32 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];+ if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP16 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];return withCompletionTimeout(requestSignal => discoverDecisionModels(config, fetchImpl, requestSignal), signal, 5000);}diff --git a/src/chrome/src/providers/d1-config.js b/src/chrome/src/providers/d1-config.jsindex b9ea69ceb..635368344 100644--- a/src/chrome/src/providers/d1-config.js+++ b/src/chrome/src/providers/d1-config.js@@ -1,9 +1,9 @@-export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp32';+export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp16';export const D1_CONSENT_VERSION = 1;export const D1_CONSENT_KEY = 'd1WebgpuConsentVersion';-export const D1_CACHE_NAME = 'webbrain-d1-fp32-v1';+export const D1_CACHE_NAME = 'webbrain-d1-fp16-v1';// Immutable release package; downloaded data is verified before local loading.-export const D1_RELEASE = Object.freeze({ revision: '71866f0ccab0b9fa92c6df64d1c34eacf2a7b2b8', manifestSha256: '47b3a9efa02ce2cacf6da8c19607cc1179a8a2b7a339aebf3db05f2237151dbb' });+export const D1_RELEASE = Object.freeze({ revision: 'd83c32b1db90861948fa5ce2bf926b46445fc7bd', manifestSha256: '3e0f61917b8e21cb2b7603cbc37680d225934cabb35e9dbd0db0c23c1aebaf1c' });export const D1_REQUIRED_FILES = Object.freeze(['config.json', 'tokenizer.json', 'tokenizer_config.json', 'tiling-ratios.json', 'onnx/decision.onnx', 'onnx/decision.data', 'onnx/vision.onnx', 'onnx/vision.data', 'onnx/projector.onnx', 'onnx/projector.data']);export function d1BaseURL() {@@ -20,7 +20,8 @@ export function assertD1Adapter(adapter) {if (!adapter) throw new Error('Hardware WebGPU is unavailable.');const info = d1AdapterInfo(adapter);if (info.isFallbackAdapter === true || /swiftshader|llvmpipe|software|lavapipe/i.test(Object.values(info).join(' '))) throw new Error('D1 requires hardware WebGPU, not a software adapter.');- // The FP32 [65536,1024] embedding is one 256 MiB Gather storage input.- if (adapter.limits.maxBufferSize < 268435456 || adapter.limits.maxStorageBufferBindingSize < 268435456) throw new Error('D1 requires at least 256 MiB buffer and storage-binding limits for its FP32 embedding.');+ // The FP16 [65536,1024] embedding is one 128 MiB Gather storage input (the+ // WebGPU default limit). Attention batches adapt to the binding limit.+ if (adapter.limits.maxBufferSize < 134217728 || adapter.limits.maxStorageBufferBindingSize < 134217728) throw new Error('D1 requires at least 128 MiB buffer and storage-binding limits for its FP16 embedding.');return info;}diff --git a/src/chrome/src/ui/locales/decision-copy.mjs b/src/chrome/src/ui/locales/decision-copy.mjsindex bb49bb955..51a9e4a74 100644--- a/src/chrome/src/ui/locales/decision-copy.mjs+++ b/src/chrome/src/ui/locales/decision-copy.mjs@@ -1,6 +1,6 @@export const decisionEnglish = {'st.d1.provider': 'Local decision model - Experimental',- 'st.d1.warning': 'Experimental on-device decision model, about 1.9 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',+ 'st.d1.warning': 'Experimental on-device decision model, about 0.95 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.','st.d1.consent': 'I accept the experimental local model download from Hugging Face and its memory requirements.','st.d1.need_consent': 'Accept the experimental model download consent first.','st.d1.download': 'Download and load', 'st.d1.load': 'Load cached model', 'st.d1.stop': 'Stop download',@@ -25,7 +25,7 @@ export const decisionEnglish = {export const decisionTranslations = {tr: {'st.d1.provider': 'Yerel karar modeli — Deneysel',- 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 1,9 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',+ 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 0,95 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.','st.d1.consent': 'Hugging Face üzerinden deneysel yerel model indirmesini ve bellek gereksinimlerini kabul ediyorum.','st.d1.need_consent': 'Önce deneysel model indirme onayını kabul edin.','st.d1.download': 'İndir ve yükle', 'st.d1.load': 'Önbellekten yükle', 'st.d1.stop': 'İndirmeyi durdur',diff --git a/src/firefox/src/agent/decision-config.js b/src/firefox/src/agent/decision-config.jsindex 2a77a6e87..eb9616e3f 100644--- a/src/firefox/src/agent/decision-config.js+++ b/src/firefox/src/agent/decision-config.js@@ -49,7 +49,7 @@ export function resolveDecisionConfig(stored = {}, compass = null) {}export async function listDecisionModels(config, fetchImpl = globalThis.fetch, signal) {- if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP32 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];+ if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP16 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];return withCompletionTimeout(requestSignal => discoverDecisionModels(config, fetchImpl, requestSignal), signal, 5000);}diff --git a/src/firefox/src/providers/d1-config.js b/src/firefox/src/providers/d1-config.jsindex b9ea69ceb..635368344 100644--- a/src/firefox/src/providers/d1-config.js+++ b/src/firefox/src/providers/d1-config.js@@ -1,9 +1,9 @@-export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp32';+export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp16';export const D1_CONSENT_VERSION = 1;export const D1_CONSENT_KEY = 'd1WebgpuConsentVersion';-export const D1_CACHE_NAME = 'webbrain-d1-fp32-v1';+export const D1_CACHE_NAME = 'webbrain-d1-fp16-v1';// Immutable release package; downloaded data is verified before local loading.-export const D1_RELEASE = Object.freeze({ revision: '71866f0ccab0b9fa92c6df64d1c34eacf2a7b2b8', manifestSha256: '47b3a9efa02ce2cacf6da8c19607cc1179a8a2b7a339aebf3db05f2237151dbb' });+export const D1_RELEASE = Object.freeze({ revision: 'd83c32b1db90861948fa5ce2bf926b46445fc7bd', manifestSha256: '3e0f61917b8e21cb2b7603cbc37680d225934cabb35e9dbd0db0c23c1aebaf1c' });export const D1_REQUIRED_FILES = Object.freeze(['config.json', 'tokenizer.json', 'tokenizer_config.json', 'tiling-ratios.json', 'onnx/decision.onnx', 'onnx/decision.data', 'onnx/vision.onnx', 'onnx/vision.data', 'onnx/projector.onnx', 'onnx/projector.data']);export function d1BaseURL() {@@ -20,7 +20,8 @@ export function assertD1Adapter(adapter) {if (!adapter) throw new Error('Hardware WebGPU is unavailable.');const info = d1AdapterInfo(adapter);if (info.isFallbackAdapter === true || /swiftshader|llvmpipe|software|lavapipe/i.test(Object.values(info).join(' '))) throw new Error('D1 requires hardware WebGPU, not a software adapter.');- // The FP32 [65536,1024] embedding is one 256 MiB Gather storage input.- if (adapter.limits.maxBufferSize < 268435456 || adapter.limits.maxStorageBufferBindingSize < 268435456) throw new Error('D1 requires at least 256 MiB buffer and storage-binding limits for its FP32 embedding.');+ // The FP16 [65536,1024] embedding is one 128 MiB Gather storage input (the+ // WebGPU default limit). Attention batches adapt to the binding limit.+ if (adapter.limits.maxBufferSize < 134217728 || adapter.limits.maxStorageBufferBindingSize < 134217728) throw new Error('D1 requires at least 128 MiB buffer and storage-binding limits for its FP16 embedding.');return info;}diff --git a/src/firefox/src/ui/locales/decision-copy.mjs b/src/firefox/src/ui/locales/decision-copy.mjsindex bb49bb955..51a9e4a74 100644--- a/src/firefox/src/ui/locales/decision-copy.mjs+++ b/src/firefox/src/ui/locales/decision-copy.mjs@@ -1,6 +1,6 @@export const decisionEnglish = {'st.d1.provider': 'Local decision model - Experimental',- 'st.d1.warning': 'Experimental on-device decision model, about 1.9 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',+ 'st.d1.warning': 'Experimental on-device decision model, about 0.95 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.','st.d1.consent': 'I accept the experimental local model download from Hugging Face and its memory requirements.','st.d1.need_consent': 'Accept the experimental model download consent first.','st.d1.download': 'Download and load', 'st.d1.load': 'Load cached model', 'st.d1.stop': 'Stop download',@@ -25,7 +25,7 @@ export const decisionEnglish = {export const decisionTranslations = {tr: {'st.d1.provider': 'Yerel karar modeli — Deneysel',- 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 1,9 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',+ 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 0,95 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.','st.d1.consent': 'Hugging Face üzerinden deneysel yerel model indirmesini ve bellek gereksinimlerini kabul ediyorum.','st.d1.need_consent': 'Önce deneysel model indirme onayını kabul edin.','st.d1.download': 'İndir ve yükle', 'st.d1.load': 'Önbellekten yükle', 'st.d1.stop': 'İndirmeyi durdur',diff --git a/test/d1-decision.mjs b/test/d1-decision.mjsindex d32b8c00a..b901d72e8 100644--- a/test/d1-decision.mjs+++ b/test/d1-decision.mjs@@ -122,11 +122,13 @@ for (const build of ['chrome', 'firefox']) {assert.equal(imported.decisionProvider, 'webgpu_d1'); assert.equal(config.resolveDecisionConfig(imported).enabled, false);});test(`${build}: hardware guards reject software and inadequate buffers, not other capable GPUs`, () => {- const adapter = { info: { description: 'Some capable GPU', isFallbackAdapter: false }, limits: { maxBufferSize: 268435456, maxStorageBufferBindingSize: 268435456 } };+ // The WebGPU default limits (128 MiB) now hold the FP16 embedding.+ const adapter = { info: { description: 'Some capable GPU', isFallbackAdapter: false }, limits: { maxBufferSize: 134217728, maxStorageBufferBindingSize: 134217728 } };assert.doesNotThrow(() => pin.assertD1Adapter(adapter));assert.throws(() => pin.assertD1Adapter({ ...adapter, info: { description: 'SwiftShader' } }), /software/);assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { maxBufferSize: 64 } }), /buffer/);- assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { ...adapter.limits, maxStorageBufferBindingSize: 134217728 } }), /storage-binding/);+ assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { ...adapter.limits, maxStorageBufferBindingSize: 67108864 } }), /storage-binding/);+ assert.equal(pin.D1_MODEL_ID, 'webbrain-one/d1-browser-decision-fp16');});test(`${build}: media remains inline data and separate from state, remote network media is rejected`, () => {const url = 'data:image/png;base64,YQ==';