3D engine that AI agents drive
JavaScript 3d3d-engineagentic-aiai-agentsblenderclaude-codegame-enginegaussian-splattinggltfjavascriptmcpmcp-servermodel-context-protocolpath-tracerpath-tracingpbrrenderingthreejs-alternativewebgl2webgpu
Report why the native path tracer fell back to gpu (#2227)
Nguyen Xuan An2f18ccd55b95
diff --git a/changes/2227.md b/changes/2227.mdnew file mode 100644index 00000000..54d64be7--- /dev/null+++ b/changes/2227.md@@ -0,0 +1 @@+- A path traced job that asks for `renderer.backend "native"` without the native tracer now reports `backendFallback` (`requestedBackend`, `usedBackend`, `fallbackReason`) and adds that reason to a later failure such as the gpu triangle limit.diff --git a/docs/path-tracer.md b/docs/path-tracer.mdindex 0e8c11b4..7116c4ef 100644--- a/docs/path-tracer.md+++ b/docs/path-tracer.md@@ -141,6 +141,12 @@ The 1920 by 1080 still of the demo room at 64 samples took 58.15 s on the GPU (R`backend: 'webgpu'` (issue #625) runs the same algorithm as WGSL compute (`src/path-tracer-webgpu.js` and `src/path-tracer-webgpu-kernel.js`, loaded only for this backend). It takes the same packed scene tables as storage buffers, draws the same random numbers in the same order and returns the same buffers, so the denoiser and tone mapping are unchanged. It needs `navigator.gpu` and an adapter with 10 storage buffers per shader stage; without them it throws an error that names `gpu` and `cpu` (the MCP `cpu` runtime has no WebGPU). Volume fog and subsurface scattering are traced too (`src/path-tracer-webgpu-medium.js`, the WGSL form of the WebGL2 pieces with the same random numbers): they are compiled into the megakernel only for a scene that uses them, and the subsurface table rides in the materials buffer, so a scene without them compiles the kernel it always did. The wavefront kernel does not trace them and throws a `RangeError` that names the megakernel. The same kernel starts paths at lightmap texels for the light bake (`bakeLightmapsAsync(..., { backend: 'webgpu' })`, see `docs/light-bake.md`). Measured on the RTX 3060 (2026-11-09, `docs/webgpu-decision.md`): the kernel compiles in 0.5 s cold against 29 s for the WebGL2 kernel, the demo room at 1920 by 1080 and 64 samples traces in 8.0 to 8.5 s against 15.2 s (three alternating runs), and FLIP against the WebGL2 backend is 0.0000 to 0.0019 on 15 of the 16 reference scenes (the sixteenth, `fog`, was not supported then; with fog and subsurface ported, 480 by 270 at 32 samples without the denoiser, RTX 3060, one run on a loaded machine, 2026-11-09: `fog` 0.0000 FLIP against WebGL2 in 1.35 s against 33.8 s, `subsurface` 0.0039 in 2.28 s against 174 s; the WebGL2 numbers are mostly the 17 and 106 s kernel compile, which is 1.0 and 1.9 s on WebGPU). The default kernel is a megakernel; a wavefront kernel (generate, extend, shade, shadow, with optional sorting by material) is built and was slower on this scene (17.3 to 18.1 s), so it is opt-in through `traceOnWebGpu` only. The default backend stays `cpu`.+### The native backend++The native backend needs the optional Cook package (`@codepawlhq/cook`) with its built binary and a Vulkan ray query adapter. An npm install of Crucible does not bring it, `COOK_DISABLE=1` and `JS67_NATIVE_TRACER=off` switch it off, and a tracer or adapter failure marks it unusable for the rest of the run.++When a job asks for `backend: "native"` and it is not available, the job runs on the gpu backend (cpu on a runtime without a GPU) and the report says so: `pathTracer.backendNote` has the sentence, and `pathTracer.backendFallback` has `{ requestedBackend, usedBackend, fallbackReason }`. If that fallback then fails, for example on the gpu limit of 2000000 triangles, the error ends with the same reason, what to install and what to pass instead. A job that ran on native, or asked for gpu or cpu itself, has neither field.+## SpeedWhere the time went on the 181,708 triangle demo room (issue #129; one thread, CPU time, 640 by 360 band, measured while another session loaded the machine):diff --git a/scripts/agent-native-trace.mjs b/scripts/agent-native-trace.mjsindex 0281e135..3a0045f3 100644--- a/scripts/agent-native-trace.mjs+++ b/scripts/agent-native-trace.mjs@@ -108,7 +108,7 @@ export async function chooseTraceBackend({ requested, runtimeBackend, locator })const found = await locator.find();if (found.tracer) return { backend: 'native', tracer: found.tracer };const fallback = runtimeBackend === 'cpu' ? 'cpu' : 'gpu';- return { backend: fallback, note: `${found.reason} The ${fallback} backend ran instead.` };+ return { backend: fallback, reason: found.reason, note: `${found.reason} The ${fallback} backend ran instead.` };}const V2_TABLE_TYPES = { Uint8Array, Uint32Array, Int32Array, Float32Array };diff --git a/scripts/agent-runtime.mjs b/scripts/agent-runtime.mjsindex 98d3f737..d5beda04 100644--- a/scripts/agent-runtime.mjs+++ b/scripts/agent-runtime.mjs@@ -22,6 +22,7 @@ import { chooseDenoiser, finishWithOidn } from './agent-path-trace-denoise.mjs';import { createNativeSequence } from './agent-native-sequence.mjs';import { chooseNativeTemporal, createTemporalAccumulator, temporalFrameSeed } from './agent-native-temporal.mjs';import { readNativeCapacity } from './native-capacity.mjs';+import { describeBackendFallback, explainFailureAfterFallback } from '../src/agent-backend-fallback.js';import { chooseNativeInstancing, chooseNativeKernel, chooseTraceBackend, createNativeTracerLocator, fetchNativeExport, isNativeTimeLimitError, traceFrameNative, traceTexelsNative } from './agent-native-trace.mjs';import { addonCapabilities, addonFileFor, addonImportMap } from './mcp-addons.mjs';import { createAgentAssetCatalog } from './agent-asset-catalog.mjs';@@ -979,6 +980,7 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', browconst traceOptions = { width, height, ...tracerOptions, backend: trace.backend === 'native' ? 'gpu' : trace.backend, denoise: choice.denoise, threads: PATH_TRACER_THREADS };let traced;let backendNote = trace.note;+ let fallbackReason = trace.reason;if (trace.backend === 'native') {let exportedScene = false;nativeMemoryAllowance.grantForFrame({ width, height, memoryMiB: memory });@@ -1009,20 +1011,29 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brownativeTemporal?.restart('a frame rendered on another backend');// Only a tracer or adapter failure makes the tracer unusable; a scene that failed to export says so for this job alone.if (exportedScene) nativeTracerLocator.markUnusable(error.message);- backendNote = exportedScene ? (await nativeTracerLocator.find()).reason : `renderer.backend "native" could not export the scene (${error.message}).`;+ fallbackReason = exportedScene ? (await nativeTracerLocator.find()).reason : `renderer.backend "native" could not export the scene (${error.message}).`;+ backendNote = fallbackReason;traceOptions.backend = backend === 'cpu' ? 'cpu' : 'gpu';choice = await chooseDenoiser(settings.renderer.denoise);traceOptions.denoise = choice.denoise;backendNote += ' The ' + traceOptions.backend + ' backend ran instead.';}}+ const fallback = settings.renderer.backend === 'native' && traceOptions.backend !== 'native'+ ? describeBackendFallback({ requestedBackend: 'native', usedBackend: traceOptions.backend, fallbackReason })+ : undefined;if (!traced) {- traced = await jobPage.evaluate(input => window.__agentPreview.pathTrace(input), { ...frameInput, options: traceOptions });+ try {+ traced = await jobPage.evaluate(input => window.__agentPreview.pathTrace(input), { ...frameInput, options: traceOptions });+ } catch (error) {+ throw explainFailureAfterFallback(error, fallback);+ }if (traced.features) {traced = await finishWithOidn({ traced, denoiser: choice.denoiser, drawLinear: input => jobPage.evaluate(drawInput => window.__agentPreview.finishPathTrace(drawInput), input) });}}if (backendNote) traced.report.backendNote = backendNote;+ if (fallback) traced.report.backendFallback = fallback;const denoiserNote = [choice.note, traced.report.denoiserNote].filter(Boolean).join(' ');if (denoiserNote) traced.report.denoiserNote = denoiserNote;metrics.renderSeconds += (performance.now() - renderStarted) / 1000;diff --git a/src/agent-backend-fallback.js b/src/agent-backend-fallback.jsnew file mode 100644index 00000000..8cd530b6--- /dev/null+++ b/src/agent-backend-fallback.js@@ -0,0 +1,20 @@+// When a path traced job asks for renderer.backend "native" and the native tracer is not available, the job runs on the gpu+// or cpu backend. These helpers say so in the report and in any later error, so an agent sees why the scene hit a gpu limit.++/** The fields a job report carries when the backend it ran on is not the one it asked for. */+export function describeBackendFallback({ requestedBackend, usedBackend, fallbackReason }) {+ return { requestedBackend, usedBackend, fallbackReason };+}++/**+ * The error a fallen back job fails with, now naming the fallback. The original message stays first, so its own advice+ * (such as the triangle limit) still reads the same; the added sentence says why native was not used and what to pass.+ */+export function explainFailureAfterFallback(error, fallback) {+ if (!fallback || !(error instanceof Error)) return error;+ const { requestedBackend, usedBackend, fallbackReason } = fallback;+ error.message = `${error.message} renderer.backend "${requestedBackend}" was requested but the ${usedBackend} backend ran instead: ${fallbackReason} `+ + 'Make the native tracer available (see "The native backend" in docs/path-tracer.md), or simplify the scene to fit the '+ + `${usedBackend} tracer, or pass renderer.backend "${usedBackend}" to stop asking for native.`;+ return error;+}diff --git a/tests/native-fallback-job.test.js b/tests/native-fallback-job.test.jsindex 58252ec2..80c4093d 100644--- a/tests/native-fallback-job.test.js+++ b/tests/native-fallback-job.test.js@@ -65,6 +65,6 @@ test('a native job without the native tracer runs the cpu backend with the sameassert.equal(cpu.report.backendNote, undefined);assert.ok(native.png.equals(cpu.png), 'the fallback image is byte-identical to the cpu backend');for (const field of DETERMINISTIC_FIELDS) assert.deepEqual(native.report[field], cpu.report[field], field);- const reportKeys = report => Object.keys(report).filter(key => key !== 'backendNote').sort();+ const reportKeys = report => Object.keys(report).filter(key => key !== 'backendNote' && key !== 'backendFallback').sort();assert.deepEqual(reportKeys(native.report), reportKeys(cpu.report));});diff --git a/tests/native-fallback-reason.test.js b/tests/native-fallback-reason.test.jsnew file mode 100644index 00000000..cf98442a--- /dev/null+++ b/tests/native-fallback-reason.test.js@@ -0,0 +1,36 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { chooseTraceBackend, createNativeTracerLocator } from '../scripts/agent-native-trace.mjs';+import { describeBackendFallback, explainFailureAfterFallback } from '../src/agent-backend-fallback.js';++const TRIANGLE_LIMIT_MESSAGE = 'This GPU path trace takes at most 2000000 triangles in total, got 2500000.';++async function withCookDisabled(run) {+ const previous = process.env.COOK_DISABLE;+ process.env.COOK_DISABLE = '1';+ try {+ return await run();+ } finally {+ if (previous === undefined) delete process.env.COOK_DISABLE;+ else process.env.COOK_DISABLE = previous;+ }+}++test('a native request without the addon records why it fell back and repeats it in the triangle limit error', async () => {+ const trace = await withCookDisabled(() => chooseTraceBackend({ requested: 'native', runtimeBackend: 'gpu', locator: createNativeTracerLocator() }));+ assert.equal(trace.backend, 'gpu');+ assert.match(trace.reason, /needs @codepawlhq\/cook/);++ const fallback = describeBackendFallback({ requestedBackend: 'native', usedBackend: trace.backend, fallbackReason: trace.reason });+ assert.deepEqual(fallback, { requestedBackend: 'native', usedBackend: 'gpu', fallbackReason: trace.reason });++ const error = explainFailureAfterFallback(new RangeError(TRIANGLE_LIMIT_MESSAGE), fallback);+ assert.match(error.message, /at most 2000000 triangles/);+ assert.match(error.message, /renderer\.backend "native" was requested but the gpu backend ran instead: renderer\.backend "native" needs @codepawlhq\/cook/);+ assert.match(error.message, /pass renderer\.backend "gpu"/);+});++test('a job that ran on the backend it asked for keeps its error unchanged', () => {+ const error = new RangeError(TRIANGLE_LIMIT_MESSAGE);+ assert.equal(explainFailureAfterFallback(error, undefined).message, TRIANGLE_LIMIT_MESSAGE);+});