Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

d1: pin the FP16-weight package (0.95 GB) and accept the 128 MiB default binding limit

Claudef18499f2602b
diff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.md
index 24d343f50..1a46ae0e9 100644
--- a/docs/privacy-and-data-flow.md
+++ b/docs/privacy-and-data-flow.md
@@ -121,13 +121,13 @@ a decision model the turn starts in Ask; only when the active model asks to
hand the turn to Act or Dev is the same page-free question sent to the active
model to confirm it.
-The optional experimental **D1 FP32 WebGPU** decision provider has no inference
+The optional experimental **D1 WebGPU** decision provider has no inference
endpoint or API key. Its bounded text/JSON state, questions and inline screenshots
stay in the local decision Worker. Explicit download consent permits only pinned
-ONNX/config/tokenizer downloads (about 1.9 GB) from Hugging Face, verified against
-the packaged manifest and checksums. The roughly 3.84 GB published package also
-contains a full checkpoint for external reproducibility, but the browser never
-downloads `model.safetensors`. JS/WASM stays bundled with the extension. Cache and
+ONNX/config/tokenizer downloads (about 0.95 GB) from Hugging Face, verified against
+the packaged manifest and checksums. The full FP32 checkpoint stays in the
+separate FP32 repository for external reproducibility; the browser never
+downloads it. JS/WASM stays bundled with the extension. Cache and
consent are local and device-bound; no download is triggered by a decision call.
Usage traces use the existing bounded model/provider/token/cost metadata, not an
additional copy of D1 state or pixels. Existing separately configured trace and
diff --git a/docs/providers-and-models.md b/docs/providers-and-models.md
index cb1a48e91..dbba608f6 100644
--- a/docs/providers-and-models.md
+++ b/docs/providers-and-models.md
@@ -71,20 +71,30 @@ Image support follows available model capabilities or the custom override. Test
Connection checks typed answers and, when enabled, two independent image-only
facts. A successful HTTP response alone does not establish image support.
-**D1 FP32 WebGPU (experimental)** is a separate, optional on-device decision
+**D1 WebGPU (experimental)** is a separate, optional on-device decision
provider. It does not replace the conversational/planning model and does not
generate tokens. It supports named choice, noul (P(yes)) and ordinal score
questions over bounded text/JSON state and inline screenshots. Selecting it
does not download anything. Explicit consent and **Download and load** fetch
-about 1.9 GB of pinned FP32 model data and tokenizer/configuration files from
-the [Hugging Face model repository](https://huggingface.co/webbrain-one/d1-browser-decision-fp32)
-into OPFS. The repository name is not a production-readiness claim; this provider
+about 0.95 GB of pinned model data and tokenizer/configuration files from
+the [Hugging Face model repository](https://huggingface.co/webbrain-one/d1-browser-decision-fp16)
+into OPFS. The package stores FP16 weights and computes in FP32; it is the
+[FP32 release](https://huggingface.co/webbrain-one/d1-browser-decision-fp32)
+with its weights rounded once, and answers the same on its held-out test. A
+cached FP32 package from an earlier version is removed on the next download. The repository name is not a production-readiness claim; this provider
remains experimental. Executable JS/WASM is packaged in the extension, not
downloaded from the model repository. **Load cached model** verifies the files
again; inference cannot trigger a model download.
This experimental path needs hardware WebGPU, OPFS and substantial GPU/host
-memory. Software adapters are rejected. Chrome uses the shared MV3 offscreen
+memory. Software adapters are rejected. The encoder's attention is dense, so
+memory grows with the square of each row's length: rows are batched under a
+256 MiB attention budget (never more than the adapter's storage-binding
+limit), text-only rows are capped at 2,048 tokens on WebGPU (the state is
+truncated, as the source does at its context limit), and a request whose
+screenshots alone exceed the adapter limit fails with a clear error instead of
+losing the GPU device. Screenshot features stay on the GPU between the vision,
+projector and decision graphs. Chrome uses the shared MV3 offscreen
document with a dedicated decision Worker; Firefox uses its background-page
Worker when the corresponding browser capabilities are available. Mirrored
code is not evidence that every browser/GPU is supported. CPU/WASM control and
diff --git a/src/chrome/src/agent/decision-config.js b/src/chrome/src/agent/decision-config.js
index 2a77a6e87..eb9616e3f 100644
--- a/src/chrome/src/agent/decision-config.js
+++ b/src/chrome/src/agent/decision-config.js
@@ -49,7 +49,7 @@ export function resolveDecisionConfig(stored = {}, compass = null) {
}
export async function listDecisionModels(config, fetchImpl = globalThis.fetch, signal) {
- if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP32 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];
+ if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP16 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];
return withCompletionTimeout(requestSignal => discoverDecisionModels(config, fetchImpl, requestSignal), signal, 5000);
}
diff --git a/src/chrome/src/providers/d1-config.js b/src/chrome/src/providers/d1-config.js
index b9ea69ceb..635368344 100644
--- a/src/chrome/src/providers/d1-config.js
+++ b/src/chrome/src/providers/d1-config.js
@@ -1,9 +1,9 @@
-export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp32';
+export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp16';
export const D1_CONSENT_VERSION = 1;
export const D1_CONSENT_KEY = 'd1WebgpuConsentVersion';
-export const D1_CACHE_NAME = 'webbrain-d1-fp32-v1';
+export const D1_CACHE_NAME = 'webbrain-d1-fp16-v1';
// Immutable release package; downloaded data is verified before local loading.
-export const D1_RELEASE = Object.freeze({ revision: '71866f0ccab0b9fa92c6df64d1c34eacf2a7b2b8', manifestSha256: '47b3a9efa02ce2cacf6da8c19607cc1179a8a2b7a339aebf3db05f2237151dbb' });
+export const D1_RELEASE = Object.freeze({ revision: 'd83c32b1db90861948fa5ce2bf926b46445fc7bd', manifestSha256: '3e0f61917b8e21cb2b7603cbc37680d225934cabb35e9dbd0db0c23c1aebaf1c' });
export const D1_REQUIRED_FILES = Object.freeze(['config.json', 'tokenizer.json', 'tokenizer_config.json', 'tiling-ratios.json', 'onnx/decision.onnx', 'onnx/decision.data', 'onnx/vision.onnx', 'onnx/vision.data', 'onnx/projector.onnx', 'onnx/projector.data']);
export function d1BaseURL() {
@@ -20,7 +20,8 @@ export function assertD1Adapter(adapter) {
if (!adapter) throw new Error('Hardware WebGPU is unavailable.');
const info = d1AdapterInfo(adapter);
if (info.isFallbackAdapter === true || /swiftshader|llvmpipe|software|lavapipe/i.test(Object.values(info).join(' '))) throw new Error('D1 requires hardware WebGPU, not a software adapter.');
- // The FP32 [65536,1024] embedding is one 256 MiB Gather storage input.
- if (adapter.limits.maxBufferSize < 268435456 || adapter.limits.maxStorageBufferBindingSize < 268435456) throw new Error('D1 requires at least 256 MiB buffer and storage-binding limits for its FP32 embedding.');
+ // The FP16 [65536,1024] embedding is one 128 MiB Gather storage input (the
+ // WebGPU default limit). Attention batches adapt to the binding limit.
+ if (adapter.limits.maxBufferSize < 134217728 || adapter.limits.maxStorageBufferBindingSize < 134217728) throw new Error('D1 requires at least 128 MiB buffer and storage-binding limits for its FP16 embedding.');
return info;
}
diff --git a/src/chrome/src/ui/locales/decision-copy.mjs b/src/chrome/src/ui/locales/decision-copy.mjs
index bb49bb955..51a9e4a74 100644
--- a/src/chrome/src/ui/locales/decision-copy.mjs
+++ b/src/chrome/src/ui/locales/decision-copy.mjs
@@ -1,6 +1,6 @@
export const decisionEnglish = {
'st.d1.provider': 'Local decision model - Experimental',
- 'st.d1.warning': 'Experimental on-device decision model, about 1.9 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',
+ 'st.d1.warning': 'Experimental on-device decision model, about 0.95 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',
'st.d1.consent': 'I accept the experimental local model download from Hugging Face and its memory requirements.',
'st.d1.need_consent': 'Accept the experimental model download consent first.',
'st.d1.download': 'Download and load', 'st.d1.load': 'Load cached model', 'st.d1.stop': 'Stop download',
@@ -25,7 +25,7 @@ export const decisionEnglish = {
export const decisionTranslations = {
tr: {
'st.d1.provider': 'Yerel karar modeli — Deneysel',
- 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 1,9 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',
+ 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 0,95 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',
'st.d1.consent': 'Hugging Face üzerinden deneysel yerel model indirmesini ve bellek gereksinimlerini kabul ediyorum.',
'st.d1.need_consent': 'Önce deneysel model indirme onayını kabul edin.',
'st.d1.download': 'İndir ve yükle', 'st.d1.load': 'Önbellekten yükle', 'st.d1.stop': 'İndirmeyi durdur',
diff --git a/src/firefox/src/agent/decision-config.js b/src/firefox/src/agent/decision-config.js
index 2a77a6e87..eb9616e3f 100644
--- a/src/firefox/src/agent/decision-config.js
+++ b/src/firefox/src/agent/decision-config.js
@@ -49,7 +49,7 @@ export function resolveDecisionConfig(stored = {}, compass = null) {
}
export async function listDecisionModels(config, fetchImpl = globalThis.fetch, signal) {
- if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP32 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];
+ if (config.provider === 'webgpu_d1') return [{ id: D1_MODEL_ID, name: 'D1 FP16 (experimental, on-device)', supportsVision: true, inputRate: 0, outputRate: 0 }];
return withCompletionTimeout(requestSignal => discoverDecisionModels(config, fetchImpl, requestSignal), signal, 5000);
}
diff --git a/src/firefox/src/providers/d1-config.js b/src/firefox/src/providers/d1-config.js
index b9ea69ceb..635368344 100644
--- a/src/firefox/src/providers/d1-config.js
+++ b/src/firefox/src/providers/d1-config.js
@@ -1,9 +1,9 @@
-export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp32';
+export const D1_MODEL_ID = 'webbrain-one/d1-browser-decision-fp16';
export const D1_CONSENT_VERSION = 1;
export const D1_CONSENT_KEY = 'd1WebgpuConsentVersion';
-export const D1_CACHE_NAME = 'webbrain-d1-fp32-v1';
+export const D1_CACHE_NAME = 'webbrain-d1-fp16-v1';
// Immutable release package; downloaded data is verified before local loading.
-export const D1_RELEASE = Object.freeze({ revision: '71866f0ccab0b9fa92c6df64d1c34eacf2a7b2b8', manifestSha256: '47b3a9efa02ce2cacf6da8c19607cc1179a8a2b7a339aebf3db05f2237151dbb' });
+export const D1_RELEASE = Object.freeze({ revision: 'd83c32b1db90861948fa5ce2bf926b46445fc7bd', manifestSha256: '3e0f61917b8e21cb2b7603cbc37680d225934cabb35e9dbd0db0c23c1aebaf1c' });
export const D1_REQUIRED_FILES = Object.freeze(['config.json', 'tokenizer.json', 'tokenizer_config.json', 'tiling-ratios.json', 'onnx/decision.onnx', 'onnx/decision.data', 'onnx/vision.onnx', 'onnx/vision.data', 'onnx/projector.onnx', 'onnx/projector.data']);
export function d1BaseURL() {
@@ -20,7 +20,8 @@ export function assertD1Adapter(adapter) {
if (!adapter) throw new Error('Hardware WebGPU is unavailable.');
const info = d1AdapterInfo(adapter);
if (info.isFallbackAdapter === true || /swiftshader|llvmpipe|software|lavapipe/i.test(Object.values(info).join(' '))) throw new Error('D1 requires hardware WebGPU, not a software adapter.');
- // The FP32 [65536,1024] embedding is one 256 MiB Gather storage input.
- if (adapter.limits.maxBufferSize < 268435456 || adapter.limits.maxStorageBufferBindingSize < 268435456) throw new Error('D1 requires at least 256 MiB buffer and storage-binding limits for its FP32 embedding.');
+ // The FP16 [65536,1024] embedding is one 128 MiB Gather storage input (the
+ // WebGPU default limit). Attention batches adapt to the binding limit.
+ if (adapter.limits.maxBufferSize < 134217728 || adapter.limits.maxStorageBufferBindingSize < 134217728) throw new Error('D1 requires at least 128 MiB buffer and storage-binding limits for its FP16 embedding.');
return info;
}
diff --git a/src/firefox/src/ui/locales/decision-copy.mjs b/src/firefox/src/ui/locales/decision-copy.mjs
index bb49bb955..51a9e4a74 100644
--- a/src/firefox/src/ui/locales/decision-copy.mjs
+++ b/src/firefox/src/ui/locales/decision-copy.mjs
@@ -1,6 +1,6 @@
export const decisionEnglish = {
'st.d1.provider': 'Local decision model - Experimental',
- 'st.d1.warning': 'Experimental on-device decision model, about 1.9 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',
+ 'st.d1.warning': 'Experimental on-device decision model, about 0.95 GB download plus tokenizer. Requires hardware WebGPU and substantial GPU memory. Not a chat model or sole completion authority. CPU/WASM control operators may run. Browser support and model decisions are not universally validated.',
'st.d1.consent': 'I accept the experimental local model download from Hugging Face and its memory requirements.',
'st.d1.need_consent': 'Accept the experimental model download consent first.',
'st.d1.download': 'Download and load', 'st.d1.load': 'Load cached model', 'st.d1.stop': 'Stop download',
@@ -25,7 +25,7 @@ export const decisionEnglish = {
export const decisionTranslations = {
tr: {
'st.d1.provider': 'Yerel karar modeli — Deneysel',
- 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 1,9 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',
+ 'st.d1.warning': 'Deneysel yerel karar modeli; tokenizer ile birlikte yaklaşık 0,95 GB indirme. Donanım WebGPU ve yüksek GPU belleği gerekir. Sohbet modeli veya tek başına tamamlanma yetkilisi değildir. CPU/WASM kontrol işlemleri çalışabilir. Her tarayıcı, GPU ve model kararı doğrulanmış değildir.',
'st.d1.consent': 'Hugging Face üzerinden deneysel yerel model indirmesini ve bellek gereksinimlerini kabul ediyorum.',
'st.d1.need_consent': 'Önce deneysel model indirme onayını kabul edin.',
'st.d1.download': 'İndir ve yükle', 'st.d1.load': 'Önbellekten yükle', 'st.d1.stop': 'İndirmeyi durdur',
diff --git a/test/d1-decision.mjs b/test/d1-decision.mjs
index d32b8c00a..b901d72e8 100644
--- a/test/d1-decision.mjs
+++ b/test/d1-decision.mjs
@@ -122,11 +122,13 @@ for (const build of ['chrome', 'firefox']) {
assert.equal(imported.decisionProvider, 'webgpu_d1'); assert.equal(config.resolveDecisionConfig(imported).enabled, false);
});
test(`${build}: hardware guards reject software and inadequate buffers, not other capable GPUs`, () => {
- const adapter = { info: { description: 'Some capable GPU', isFallbackAdapter: false }, limits: { maxBufferSize: 268435456, maxStorageBufferBindingSize: 268435456 } };
+ // The WebGPU default limits (128 MiB) now hold the FP16 embedding.
+ const adapter = { info: { description: 'Some capable GPU', isFallbackAdapter: false }, limits: { maxBufferSize: 134217728, maxStorageBufferBindingSize: 134217728 } };
assert.doesNotThrow(() => pin.assertD1Adapter(adapter));
assert.throws(() => pin.assertD1Adapter({ ...adapter, info: { description: 'SwiftShader' } }), /software/);
assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { maxBufferSize: 64 } }), /buffer/);
- assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { ...adapter.limits, maxStorageBufferBindingSize: 134217728 } }), /storage-binding/);
+ assert.throws(() => pin.assertD1Adapter({ ...adapter, limits: { ...adapter.limits, maxStorageBufferBindingSize: 67108864 } }), /storage-binding/);
+ assert.equal(pin.D1_MODEL_ID, 'webbrain-one/d1-browser-decision-fp16');
});
test(`${build}: media remains inline data and separate from state, remote network media is rejected`, () => {
const url = 'data:image/png;base64,YQ==';