Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Trace render jobs on the GPU and bake recipes on the GPU (issue #622)

Nguyen Xuan Anbeda1e4f133e
diff --git a/changes/622.md b/changes/622.md
new file mode 100644
index 00000000..697f93bd
--- /dev/null
+++ b/changes/622.md
@@ -0,0 +1 @@
+- Render jobs take `settings.renderer.backend: "gpu"` to path trace on the GPU, film and stills jobs can path trace every frame with one kernel compile, and recipe bakes run on the GPU when the page can, with `scene_inspect` naming the backend (#622)
diff --git a/docs/agent-render-jobs.md b/docs/agent-render-jobs.md
index f5dc3425..f70a6505 100644
--- a/docs/agent-render-jobs.md
+++ b/docs/agent-render-jobs.md
@@ -127,14 +127,14 @@ A seek to any frame gives the same pixels: a job samples each frame from its tim
## Path tracer
-An `image` job can ask for the offline [path tracer](https://github.com/codepawl/67.js/blob/main/docs/path-tracer.md) with `settings.renderer`:
+An `image`, `stills` or `video` job can ask for the offline [path tracer](https://github.com/codepawl/67.js/blob/main/docs/path-tracer.md) with `settings.renderer`:
```json
{ "kind": "image", "width": 960, "height": 540,
- "renderer": { "type": "pathTracer", "samples": 64, "bounces": 8, "denoise": true, "seed": 0, "maxSeconds": 120 } }
+ "renderer": { "type": "pathTracer", "backend": "gpu", "samples": 64, "bounces": 8, "denoise": true, "seed": 0, "maxSeconds": 120 } }
```
-Every field after `type` is optional. Limits: `samples` 1 to 1024, `bounces` 1 to 16, `seed` 0 to 2147483647, `maxSeconds` 1 to 600 (the job's deadline is `maxSeconds` plus 60 seconds; a 1920 by 1080 still of a 180,000 triangle room at 64 samples took 246 s, so give a large still `maxSeconds` 600). `passes` and the other kinds are refused with it. The tracer loads in the job page on demand, uses eight threads at below-normal priority, stops pixels that have converged (`report.averageSamples`), and writes `render.png`; `scene_render_result` metrics carry `pathTracer` with the samples, threads, seconds and what the scene asked for that was not drawn. The same recipe and seed give the same PNG. A job that passes `maxSeconds` fails and writes no image.
+Every field after `type` is optional. `backend` is `cpu` (the default, byte-identical on SwiftShader smokes) or `gpu`, which traces in a WebGL2 shader and needs a runtime started on the GPU backend (a `--backend cpu` runtime refuses it). The report in `metrics.pathTracer` names the `backend` that ran, the `gpu`, `gpuCompileSeconds` and `gpuKernelCompiles`. A `stills` or `video` job traces every frame with the same options: the kernel compiles once per job page and each frame reuses it, so a film shows `gpuKernelCompiles: 1`. A video of traced frames goes through the ffmpeg encoder (the browser H.264 encoder has no drawn frame to read), `maxSeconds` limits each frame, and the job deadline is `maxSeconds` times the frame count plus 60 seconds, up to the job ceiling. Limits: `samples` 1 to 1024, `bounces` 1 to 16, `seed` 0 to 2147483647, `maxSeconds` 1 to 600 (the job's deadline is `maxSeconds` plus 60 seconds; a 1920 by 1080 still of a 180,000 triangle room at 64 samples took 246 s, so give a large still `maxSeconds` 600). `passes` are refused with it. The tracer loads in the job page on demand, uses eight threads at below-normal priority, stops pixels that have converged (`report.averageSamples`), and writes `render.png`; `scene_render_result` metrics carry `pathTracer` with the samples, threads, seconds and what the scene asked for that was not drawn. The same recipe and seed give the same PNG. A job that passes `maxSeconds` fails and writes no image.
## Controller API
diff --git a/docs/light-bake.md b/docs/light-bake.md
index 0313b1c6..b5c69f45 100644
--- a/docs/light-bake.md
+++ b/docs/light-bake.md
@@ -56,6 +56,10 @@ Spot lights take the cone and soft edge the renderer uses (`angle`, `penumbra`,
Not baked: the environment map (it already lights every surface at run time) and specular reflection. A metal part reflects no baked light. Fresnel and sheen are ignored.
+### Recipe bakes choose the backend themselves
+
+A recipe `bake` built in a runtime on the GPU backend (the MCP server's default, `scene_inspect`, render jobs) traces its lightmaps on the GPU when the page has WebGL2 with `EXT_color_buffer_float` and `EXT_float_blend`, and on the CPU worker otherwise: SwiftShader runtimes (`--backend cpu`), a recipe whose lights use `affects`, or a GPU bake that fails. `scene_inspect` reports `bake.backend` (`cpu` or `gpu`) and, for the GPU, `bake.gpu` and `bake.gpuCompileSeconds`. The CPU path is unchanged, so the same recipe gives the same bytes there. The irradiance grid and probes still trace on the CPU.
+
## How a part draws it
A baked part is a copy with new geometry and a new material. A plain part (no maps, pattern, layers or emission) gets lightmap UVs from the `uv_unwrap` unwrap in channel 0, and its material gets `emissiveMap` (the lightmap, `srgb`, linear filter, clamped), `emissiveColor [1, 1, 1]` and `emission` set to the brightest texel, so the renderer, its shaders and every scene without a bake stay as they are. The input parts are not changed.
diff --git a/docs/path-tracer.md b/docs/path-tracer.md
index d641c14e..1f50d143 100644
--- a/docs/path-tracer.md
+++ b/docs/path-tracer.md
@@ -62,7 +62,7 @@ The same scene and options give the same bytes. Random numbers come from a hash
`backend: 'gpu'` (issue #302) runs the same algorithm in a WebGL2 fragment shader (`src/path-tracer-gpu.js`, loaded only for this backend, so a CPU render and the core bundle do not carry it). The BVH, triangles, materials, maps, lights and the environment table are packed into float textures (`src/path-tracer-gpu-pack.js`), the random numbers are the same integer hash and PCG stream as the CPU tracer and are drawn in the same order, and each sample adds into four float targets (colour and depth, albedo, normal, luminance and its square) that feed the same denoiser. Float32 maths and a 24-bit random value make the pixels differ a little from the CPU tracer: FLIP (LDR) of GPU against CPU at the same seed was 0.0001 to 0.0018 on the 16 path tracer reference scenes at 480 by 270 and 32 samples. It needs a page with WebGL2, `EXT_color_buffer_float` and `EXT_float_blend`, and throws without them, so choose it only where those exist. It traces everything the CPU tracer does (lights with linking, all materials and maps including emissive maps, glass, volume fog, the sky); it does not use `noiseThreshold` (every pixel draws every sample) or `threads`, and a sample that produced a NaN or an infinity counts as black. The first still of a page compiles the kernel (13 to 30 s), later ones reuse it.
-The 1920 by 1080 still of the demo room at 64 samples took 58.15 s on the GPU (RTX 3060, clean image, including 10.3 s of compiling with a warm shader cache, the BVH build and the denoiser; a cold compile measured 30 s on an earlier version of the kernel, which would have put it over 60 s; remeasured 2026-11-04 below) against 585.9 s on the CPU tracer (measured 2026-11-03; remeasured 2026-11-04 at 38.7 s for a first compile of a kernel text the driver had not seen, 7.0 s of it compiling, and 41.0 s for the same kernel again, 7.3 s compiling, each in a fresh headless Chrome profile, so the 60 s bound holds cold: the compile no longer differs between cold and warm on this machine, and `gpuCompileSeconds` in the job report records it; a scene with a subsurface part compiles a larger kernel, 42.8 s cold in the same setup, which adds that to its still; see [the WebGPU decision](webgpu-decision.md) for the choice of WebGL2 over WebGPU). Against Cycles (FLIP, 960 by 540, 64 samples, 16 scenes, `bench/reference/path-tracer.json` for the CPU and `path-tracer-gpu.json` for the GPU), the two backends scored within 0.0002 of each other on every scene (largest gap 0.0001 to 0.0002 on `foliage`, `glass`, `character`), so each keeps its score within the 0.005 bound (measured 2026-11-04). The GPU run took 4 to 13 s per scene after the first scene's compile. `node bench/reference/gpu-compare.mjs <directory> <scene or recipe.json> --backend both` draws a scene with both backends, and `bench/reference/flip-compare.py` scores them against each other and against Cycles. Render jobs do not select the GPU backend yet.
+The 1920 by 1080 still of the demo room at 64 samples took 58.15 s on the GPU (RTX 3060, clean image, including 10.3 s of compiling with a warm shader cache, the BVH build and the denoiser; a cold compile measured 30 s on an earlier version of the kernel, which would have put it over 60 s; remeasured 2026-11-04 below) against 585.9 s on the CPU tracer (measured 2026-11-03; remeasured 2026-11-04 at 38.7 s for a first compile of a kernel text the driver had not seen, 7.0 s of it compiling, and 41.0 s for the same kernel again, 7.3 s compiling, each in a fresh headless Chrome profile, so the 60 s bound holds cold: the compile no longer differs between cold and warm on this machine, and `gpuCompileSeconds` in the job report records it; a scene with a subsurface part compiles a larger kernel, 42.8 s cold in the same setup, which adds that to its still; see [the WebGPU decision](webgpu-decision.md) for the choice of WebGL2 over WebGPU). Against Cycles (FLIP, 960 by 540, 64 samples, 16 scenes, `bench/reference/path-tracer.json` for the CPU and `path-tracer-gpu.json` for the GPU), the two backends scored within 0.0002 of each other on every scene (largest gap 0.0001 to 0.0002 on `foliage`, `glass`, `character`), so each keeps its score within the 0.005 bound (measured 2026-11-04). The GPU run took 4 to 13 s per scene after the first scene's compile. `node bench/reference/gpu-compare.mjs <directory> <scene or recipe.json> --backend both` draws a scene with both backends, and `bench/reference/flip-compare.py` scores them against each other and against Cycles. A render job selects it with `settings.renderer.backend: "gpu"` (see `docs/agent-render-jobs.md`); the job default stays `cpu`.
## Speed
diff --git a/scripts/agent-preview-browser.js b/scripts/agent-preview-browser.js
index c064fd13..3685ed9b 100644
--- a/scripts/agent-preview-browser.js
+++ b/scripts/agent-preview-browser.js
@@ -498,14 +498,22 @@ function useKernelWorkers() {
return kernelWorkersAvailable() && globalThis.__67jsKernelWorkers !== 'off';
}
-function buildSceneOffMainThread(recipe, options) {
+// A recipe bake runs on the GPU when the page has float render targets; the CPU bake stays the default everywhere else.
+async function pageKernelsFor(recipe, backend) {
+ if (!recipe.bake || backend !== 'gpu') return undefined;
+ const { gpuBakeKernels } = await import('../src/agent-bake-gpu.js');
+ return gpuBakeKernels();
+}
+
+async function buildSceneOffMainThread(recipe, options, backend) {
+ const pageKernels = await pageKernelsFor(recipe, backend);
return buildWithKernels(() => {
const scene = createAgentScene(recipe, {}, options);
for (const part of scene.parts) {
void part.material.map;
}
return scene;
- });
+ }, { pageKernels });
}
function seekForExport(record, motionTime) {
@@ -817,7 +825,7 @@ window.__agentPreview = {
finishBuildReport(startedAt);
return createPreview(scene, input, undefined, undefined, sequenceFrames);
}
- return buildSceneOffMainThread(input.recipe, { sequenceFrames }).then(scene => {
+ return buildSceneOffMainThread(input.recipe, { sequenceFrames }, input.backend).then(scene => {
finishBuildReport(startedAt);
return createPreview(scene, input, undefined, undefined, sequenceFrames);
});
diff --git a/scripts/agent-render-jobs.mjs b/scripts/agent-render-jobs.mjs
index 660e6987..b4edb394 100644
--- a/scripts/agent-render-jobs.mjs
+++ b/scripts/agent-render-jobs.mjs
@@ -25,12 +25,12 @@ const PATH_TRACE_OVERHEAD_MS = 60 * 1000;
/**
* 60 s plus 2 s per frame, scaled up for frames larger than 1024x576, never above the ceiling. A path trace may run for
- * its own maxSeconds, so its deadline also reaches that plus the overhead and the trace stops itself first.
+ * its own maxSeconds on every frame, so its deadline also reaches that plus the overhead and the trace stops itself first.
*/
export function jobDeadlineMs({ frameCount, width, height, renderer }, maximumMs = MAXIMUM_JOB_DEADLINE_MS) {
const sizeScale = Math.max(1, (width * height) / REFERENCE_FRAME_PIXELS);
const frameDeadline = Math.ceil(BASE_DEADLINE_MS + frameCount * FRAME_ALLOWANCE_MS * sizeScale);
- const traceDeadline = renderer?.type === 'pathTracer' ? Math.ceil(renderer.maxSeconds * 1000) + PATH_TRACE_OVERHEAD_MS : 0;
+ const traceDeadline = renderer?.type === 'pathTracer' ? Math.ceil(renderer.maxSeconds * 1000 * frameCount) + PATH_TRACE_OVERHEAD_MS : 0;
return Math.min(maximumMs, Math.max(frameDeadline, traceDeadline));
}
const MAXIMUM_JSON_BYTES = 1024 ** 2;
@@ -96,7 +96,7 @@ function settingsFrom(input, clips, recipe) {
if (frameCount > 900) throw new RangeError('Job exceeds 900 frames.');
const result = { kind: input.kind, width, height, fps, duration, frameCount };
if (input.passes !== undefined) result.passes = passesFrom(input.passes, input.kind);
- if (input.renderer !== undefined) result.renderer = rendererFrom(input.renderer, input.kind, input.passes);
+ if (input.renderer !== undefined) result.renderer = rendererFrom(input.renderer, input.passes);
const lastSampleTime = input.kind === 'image' ? 0 : input.kind === 'stills' ? duration : (frameCount - 1) / fps;
if (timelineStart !== undefined) {
if (input.motion !== undefined || input.animation !== undefined) throw new TypeError('A timeline owns the motion clock; leave motion and animation out.');
@@ -148,14 +148,15 @@ function settingsFrom(input, clips, recipe) {
const FRACTIONAL_RENDERER_FIELDS = new Set(['maxSeconds', 'noiseThreshold']);
-function rendererFrom(renderer, kind, passes) {
- objectKeys(renderer, ['type', 'samples', 'bounces', 'denoise', 'seed', 'maxSeconds', 'noiseThreshold'], 'renderer');
+function rendererFrom(renderer, passes) {
+ objectKeys(renderer, ['type', 'backend', 'samples', 'bounces', 'denoise', 'seed', 'maxSeconds', 'noiseThreshold'], 'renderer');
if (renderer.type !== 'pathTracer') throw new TypeError('renderer.type must be pathTracer.');
- if (kind !== 'image') throw new RangeError('The path tracer renders one image: use kind image.');
if (passes !== undefined) throw new RangeError('The path tracer writes the lit image only: leave passes out.');
const denoise = renderer.denoise ?? true;
if (typeof denoise !== 'boolean' && !PATH_TRACER_DENOISERS.includes(denoise)) throw new TypeError(`renderer.denoise must be a boolean or one of ${PATH_TRACER_DENOISERS.join(', ')}.`);
- const result = { type: 'pathTracer', denoise };
+ const backend = renderer.backend ?? 'cpu';
+ if (backend !== 'cpu' && backend !== 'gpu') throw new TypeError('renderer.backend must be cpu or gpu.');
+ const result = { type: 'pathTracer', backend, denoise };
for (const name of Object.keys(PATH_TRACER_JOB_LIMITS)) {
const { minimum, maximum, default: fallback } = PATH_TRACER_JOB_LIMITS[name];
const defaultValue = name === 'samples' && denoise === 'oidn' ? OIDN_DEFAULT_SAMPLES : fallback;
diff --git a/scripts/agent-runtime.mjs b/scripts/agent-runtime.mjs
index 8deba9e9..365c160e 100644
--- a/scripts/agent-runtime.mjs
+++ b/scripts/agent-runtime.mjs
@@ -408,13 +408,14 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brow
frameReceiver = createFrameReceiver({ maximumBytes: width * height * 4 });
activeFrameReceiver = frameReceiver;
const frameSocketUrl = `ws://127.0.0.1:${httpServer.address().port}${frameReceiver.path}`;
- if (settings.encoder === 'nvenc') {
+ // A path traced frame is a PNG made outside the page's drawing buffer, so the browser encoder has nothing to encode.
+ if (settings.encoder === 'nvenc' && !settings.renderer) {
const pageEncoder = await startVideoEncoderInPage(jobPage, frameSocketUrl,
{ width, height, fps: settings.fps, quantizer: DEFAULT_PAGE_ENCODER_QUANTIZER });
pageEncodes = pageEncoder.supported;
}
encoder = createFrameEncoder({ executable: ffmpeg, output: join(directory, 'render.mp4.partial'),
- fps: settings.fps, format: pageEncodes ? 'h264' : 'jpeg', width, height, signal, maximumFrameBytes: MAXIMUM_JOB_FRAME_BYTES, backend: settings.encoder === 'nvenc' ? 'gpu' : 'cpu',
+ fps: settings.fps, format: pageEncodes ? 'h264' : settings.renderer ? 'png' : 'jpeg', width, height, signal, maximumFrameBytes: MAXIMUM_JOB_FRAME_BYTES, backend: settings.encoder === 'nvenc' ? 'gpu' : 'cpu',
audioArguments: settings.audio ? audioInputArguments(audio.path, settings.audio.offset) : [],
audioCodec: settings.audio?.track === 'pcm' ? 'pcm' : 'aac' });
if (pageEncodes) frameReceiver.onChunk(chunk => encoder.writeChunk(chunk));
@@ -438,8 +439,10 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brow
...(settings.animation ? { animation: { clip: settings.animation.clip, time: settings.animation.start + elapsed } } : {}) };
const renderStarted = performance.now();
if (settings.renderer) {
+ const { type, ...tracerOptions } = settings.renderer;
+ if (settings.renderer.backend === 'gpu' && backend === 'cpu') throw new Error('renderer.backend "gpu" needs a runtime on the GPU backend, and this one runs on SwiftShader.');
const choice = await chooseDenoiser(settings.renderer.denoise);
- let traced = await jobPage.evaluate(input => window.__agentPreview.pathTrace(input), { ...frameInput, options: { width, height, ...settings.renderer, denoise: choice.denoise, threads: PATH_TRACER_THREADS } });
+ let traced = await jobPage.evaluate(input => window.__agentPreview.pathTrace(input), { ...frameInput, options: { width, height, ...tracerOptions, denoise: choice.denoise, threads: PATH_TRACER_THREADS } });
if (traced.features) {
traced = await finishWithOidn({ traced, denoiser: choice.denoiser, drawLinear: input => jobPage.evaluate(drawInput => window.__agentPreview.finishPathTrace(drawInput), input) });
}
@@ -449,10 +452,15 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brow
metrics.height = height;
metrics.pathTracer = traced.report;
const tracedBytes = Buffer.from(traced.data, 'base64');
- if (tracedBytes.length > MAXIMUM_JOB_FRAME_BYTES) throw new Error('Path traced image exceeded 12 MiB.');
- await writeFile(join(directory, 'render.png.partial'), tracedBytes, { flag: 'wx' });
- artifacts.push({ temporary: 'render.png.partial', filename: 'render.png', mimeType: 'image/png' });
- await progress({ frames: 1, maximumPendingFrames: 1 });
+ if (tracedBytes.length > MAXIMUM_JOB_FRAME_BYTES) throw new Error('Path traced frame exceeded 12 MiB.');
+ if (encoder) {
+ await encoder.writeFrame(tracedBytes);
+ } else {
+ const filename = settings.kind === 'image' ? 'render.png' : `frame-${frame}.png`;
+ await writeFile(join(directory, `${filename}.partial`), tracedBytes, { flag: 'wx' });
+ artifacts.push({ temporary: `${filename}.partial`, filename, mimeType: 'image/png' });
+ }
+ await progress({ frames: frame + 1, maximumPendingFrames: 1 });
continue;
}
// A video frame is read in the same task as its draw, because the browser clears the drawing buffer after presenting.
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index 5c32c0e3..0fd41016 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -689,7 +689,7 @@ const renderSettings = z.strictObject({
animation: z.strictObject({ clip: z.number().int().min(0), start: z.number().min(0).max(3600).optional() }).optional(),
motion: z.strictObject({ start: z.number().min(0).max(3600).optional() }).optional(),
rendererTime: z.strictObject({ start: z.number().min(0).max(3600).optional(), rate: z.number().min(-16).max(16).optional() }).optional(),
- renderer: z.strictObject({ type: z.literal('pathTracer'), samples: z.number().optional(), bounces: z.number().optional(), denoise: z.union([z.boolean(), z.enum(['wavelet', 'oidn', 'none'])]).optional(), seed: z.number().optional(), maxSeconds: z.number().optional(), noiseThreshold: z.number().optional() }).optional().describe('image only: path tracer; see scene_capabilities pathTracer.'),
+ renderer: z.strictObject({ type: z.literal('pathTracer'), backend: z.enum(['cpu', 'gpu']).optional(), samples: z.number().optional(), bounces: z.number().optional(), denoise: z.union([z.boolean(), z.enum(['wavelet', 'oidn', 'none'])]).optional(), seed: z.number().optional(), maxSeconds: z.number().optional(), noiseThreshold: z.number().optional() }).optional().describe('path tracer, backend gpu is faster; see scene_capabilities pathTracer.'),
timeline: z.strictObject({ start: z.number().min(0).max(60).optional() }).optional().describe('Play the recipe timeline from start (default 0); duration defaults to the rest of it. It owns the motion clock, so leave motion and animation out.'),
audio: z.strictObject({ file: z.string().max(68), offset: z.number().min(0).max(3600).optional(), track: z.enum(['aac', 'pcm']).optional() }).optional().describe('Video only: a file name in the --audio folder (.wav, .mp3, .ogg, .flac) mixed in from offset seconds.'),
encoder: z.enum(VIDEO_ENCODERS).optional().describe('Video only: cpu (default) or nvenc.'),
diff --git a/scripts/smoke-path-tracer.mjs b/scripts/smoke-path-tracer.mjs
index b5325a2d..0354fbf4 100644
--- a/scripts/smoke-path-tracer.mjs
+++ b/scripts/smoke-path-tracer.mjs
@@ -75,6 +75,7 @@ try {
const first = await renderedBytes(sessionId, { ...base, renderer: tracer });
assert.equal(first.metrics.pathTracer.samples, 4);
assert.equal(first.metrics.pathTracer.threads, 8);
+ assert.equal(first.metrics.pathTracer.backend, 'cpu');
const second = await renderedBytes(sessionId, { ...base, renderer: tracer });
assert.deepEqual(second.bytes, first.bytes, 'The same recipe and seed must give the same PNG.');
const reseeded = await renderedBytes(sessionId, { ...base, renderer: { ...tracer, seed: 5 } });
@@ -98,13 +99,18 @@ try {
[{ ...base, renderer: { ...tracer, samples: 99999 } }, /samples/],
[{ ...base, renderer: { ...tracer, extra: 1 } }, /extra|Unrecognized|Unknown/],
[{ ...base, renderer: tracer, passes: ['albedo'] }, /passes/],
- [{ ...base, kind: 'stills', renderer: tracer }, /image/],
+ [{ ...base, renderer: { ...tracer, backend: 'metal' } }, /backend/],
]) {
const refused = await submit(sessionId, settings);
assert.equal(refused.isError, true, JSON.stringify(settings));
assert.match(refused.content[0].text, message);
}
+ // A SwiftShader runtime keeps the CPU tracer: a job that asks for the GPU backend fails instead of drawing something else.
+ const gpuOnSwiftShader = await submit(sessionId, { ...base, renderer: { ...tracer, backend: 'gpu' } });
+ assert.equal(gpuOnSwiftShader.isError, undefined, gpuOnSwiftShader.content?.[0]?.text);
+ assert.equal((await finished(gpuOnSwiftShader.structuredContent.jobId)).state, 'failed', 'backend gpu must fail on a cpu runtime.');
+
const slow = await submit(sessionId, { kind: 'image', width: 512, height: 512, renderer: { type: 'pathTracer', samples: 1024, maxSeconds: 1 } });
assert.equal(slow.isError, undefined, slow.content?.[0]?.text);
const slowRecord = await finished(slow.structuredContent.jobId);
diff --git a/src/agent-bake-gpu.js b/src/agent-bake-gpu.js
new file mode 100644
index 00000000..24eb800d
--- /dev/null
+++ b/src/agent-bake-gpu.js
@@ -0,0 +1,43 @@
+import { planLightmaps } from './light-bake.js';
+import { BAKE_WORKER_MAXIMUM_SECONDS } from './agent-scene-capabilities.js';
+import { changedPartsOf, fitSettingsToBudget } from './agent-bake.js';
+import { isGpuTracerAvailable } from './path-tracer-gpu.js';
+import { traceLightmapsAsync } from './light-bake-gpu.js';
+
+// The recipe light bake on the GPU. The scene build is synchronous, so a build that meets a `lightBake` kernel defers it
+// (src/kernel-offload.js) and this module answers it on the page: same inputs, same result shape as the worker's CPU
+// kernel, with `report.backend` naming the tracer. Anything the GPU kernel cannot trace returns undefined, and the CPU
+// worker bakes it instead.
+
+function lightsOf(lighting) {
+ return [...(lighting.pointLights ?? []), ...(lighting.spotLights ?? []), ...(lighting.areaLights ?? [])];
+}
+
+/** Whether a `lightBake` call can run on the GPU: the kernel has no light linking (`affects`). */
+export function canBakeOnGpu(lighting) {
+ return lightsOf(lighting).every(light => light.affects === undefined || light.affects === null);
+}
+
+/** The `lightBake` answer traced on the GPU, in the shape `bakeChangedParts` returns. */
+export async function bakeChangedPartsOnGpu(snapshots, lighting, settings) {
+ const plan = planLightmaps(snapshots, lighting, { ...settings, maxSeconds: BAKE_WORKER_MAXIMUM_SECONDS, backend: 'gpu' });
+ const { used, reduced, reduction } = fitSettingsToBudget(plan, settings);
+ const { parts, report } = await traceLightmapsAsync(plan, { ...used, backend: 'gpu' });
+ return { changed: changedPartsOf(parts, snapshots), report, used, reduced, reduction };
+}
+
+/**
+ * Page kernels for `buildWithKernels`: the GPU bake when this page has WebGL2 float targets, and nothing otherwise, so
+ * the worker keeps the CPU bake.
+ */
+export function gpuBakeKernels() {
+ if (!isGpuTracerAvailable()) {
+ return undefined;
+ }
+ return {
+ lightBake: {
+ accepts: ([, lighting]) => canBakeOnGpu(lighting),
+ compute: ([snapshots, lighting, settings]) => bakeChangedPartsOnGpu(snapshots, lighting, settings),
+ },
+ };
+}
diff --git a/src/agent-bake.js b/src/agent-bake.js
index 1df31bb1..69b9d964 100644
--- a/src/agent-bake.js
+++ b/src/agent-bake.js
@@ -195,13 +195,18 @@ export function bakeChangedParts(snapshots, lighting, settings) {
const plan = planLightmaps(snapshots, lighting, { ...settings, maxSeconds: bakeSecondsLimit() });
const { used, reduced, reduction } = fitSettingsToBudget(plan, settings);
const { parts, report } = traceWithAdvice(plan, used);
+ return { changed: changedPartsOf(parts, snapshots), report, used, reduced, reduction };
+}
+
+/** The [index, part] pairs of the parts a bake replaced; a part left alone is the caller's own object. */
+export function changedPartsOf(bakedParts, snapshots) {
const changed = [];
- parts.forEach((part, index) => {
+ bakedParts.forEach((part, index) => {
if (part !== snapshots[index]) {
changed.push([index, part]);
}
});
- return { changed, report, used, reduced, reduction };
+ return changed;
}
/**
@@ -251,7 +256,7 @@ export function bakeAgentParts(parts, worldMatrices, environment, settings, gene
// Probes are placed here, on the page, because they build maps and materials; their trace ran above.
const placed = probeTrace ? placeProbes(bakedParts, probeTrace) : null;
return { parts: Object.freeze(placed ? placed.parts : bakedParts), report: { ...settings, ...(placed ? { probes: placed.report } : {}), used, reduced, reduction, texels: report.texels, rayWork: report.rayWork, seconds: report.seconds,
- bakedParts: report.parts.length, skipped: report.skipped } };
+ backend: report.backend, ...(report.gpu ? { gpu: report.gpu, gpuCompileSeconds: report.gpuCompileSeconds } : {}), bakedParts: report.parts.length, skipped: report.skipped } };
}
// The grid is a few thousand nodes of six gathers each, so its samples are a quarter of the lightmap preset's: about 32 at standard.
diff --git a/src/kernel-offload.js b/src/kernel-offload.js
index 5c997d20..8b40b12e 100644
--- a/src/kernel-offload.js
+++ b/src/kernel-offload.js
@@ -23,15 +23,16 @@ let workerStartFailure = null;
// What ran where, since the last reset: a kernel served by a worker, a kernel computed inline, and why a kernel that
// should have gone to a worker did not. A silent fallback would otherwise look the same as success.
const MAXIMUM_RECORDED_FALLBACKS = 16;
-const statistics = { worker: 0, inline: 0, fallbacks: [] };
+const statistics = { worker: 0, page: 0, inline: 0, fallbacks: [] };
/** Counts of kernels served by a worker and run inline, with the reason for each fallback. */
export function kernelStats() {
- return { worker: statistics.worker, inline: statistics.inline, fallbacks: statistics.fallbacks.map(fallback => ({ ...fallback })) };
+ return { worker: statistics.worker, page: statistics.page, inline: statistics.inline, fallbacks: statistics.fallbacks.map(fallback => ({ ...fallback })) };
}
export function resetKernelStats() {
statistics.worker = 0;
+ statistics.page = 0;
statistics.inline = 0;
statistics.fallbacks = [];
}
@@ -202,7 +203,7 @@ function requestKernel(worker, job, signal) {
* built from identical bytes. `createWorker` replaces the browser Worker (tests, other hosts); `signal` cancels the
* build and ends the worker. Hosts without workers just run the build.
*/
-export async function buildWithKernels(build, { createWorker = defaultWorkerFactory, signal, available = kernelWorkersAvailable() } = {}) {
+export async function buildWithKernels(build, { createWorker = defaultWorkerFactory, signal, available = kernelWorkersAvailable(), pageKernels } = {}) {
if (!available || activeSession) {
return build();
}
@@ -227,6 +228,9 @@ export async function buildWithKernels(build, { createWorker = defaultWorkerFact
session.inlineAll = true;
recordFallback(error.job.name, 'the build asked for more kernels than any scene holds');
}
+ if (await computeOnPage(session, pageKernels, error.job)) {
+ continue;
+ }
worker ??= createWorkerOrNull(createWorker);
await computeOnWorker(session, worker, error.job, signal);
} finally {
@@ -238,6 +242,23 @@ export async function buildWithKernels(build, { createWorker = defaultWorkerFact
}
}
+// A page kernel answers a job on the page itself, where the GPU is, instead of on the worker. It takes a job only when its
+// `accepts` says so; if it fails, the worker still does the job on the CPU, so a GPU fault never fails a build.
+async function computeOnPage(session, pageKernels, job) {
+ const kernel = pageKernels?.[job.name];
+ if (!kernel?.accepts(job.args)) {
+ return false;
+ }
+ try {
+ session.results.set(job.key, { value: await kernel.compute(job.args) });
+ statistics.page++;
+ return true;
+ } catch (error) {
+ recordFallback(job.name, `the page kernel failed: ${error.message}`);
+ return false;
+ }
+}
+
function createWorkerOrNull(createWorker) {
try {
return createWorker();
diff --git a/src/path-tracer-capabilities.js b/src/path-tracer-capabilities.js
index 8a50f0b0..d2886ec7 100644
--- a/src/path-tracer-capabilities.js
+++ b/src/path-tracer-capabilities.js
@@ -16,7 +16,7 @@ export const PATH_TRACER_DENOISERS = Object.freeze(['wavelet', 'oidn', 'none']);
export const OIDN_DEFAULT_SAMPLES = 16;
export const PATH_TRACER_CAPABILITIES = Object.freeze({
- setting: 'settings.renderer { type: "pathTracer", samples, bounces, denoise (true | false | "wavelet" | "oidn" | "none"), seed, maxSeconds, noiseThreshold }; image jobs only, no passes. Past maxSeconds the job fails.',
+ setting: 'settings.renderer { type: "pathTracer", backend ("cpu" default | "gpu"), samples, bounces, denoise (true | false | "wavelet" | "oidn" | "none"), seed, maxSeconds, noiseThreshold }; image, stills and video jobs, no passes. backend gpu needs a runtime on the GPU backend; the report names backend, gpuCompileSeconds and gpuKernelCompiles (1 for a whole film). Past maxSeconds on one frame the job fails.',
noiseThreshold: 'Default 0 draws every sample. Above 0 a pixel stops once its standard error relative to its mean is under the value, checked from 16 samples on: faster and noisier, and it cost quality on spot-area (score 0.0658 at 0, 0.0900 at 0.03). docs/path-tracer.md has the table.',
traces: 'emissive maps and strength; volume fog as a uniform medium (density, colour, anisotropy, box; noise, drift and ambient are not used); the material subsurface block as a bounded random walk on the cpu and gpu backends (at most 32 scattering events, mean free path radius times scale in metres), and a card with nothing behind it as thin translucency.',
skips: 'occlusion maps, liquids, anisotropy, iridescence, sheen, caustics, depth of field, bloom; metrics.pathTracer.ignored in the job result lists what the scene asked for.',
diff --git a/src/path-tracer-gpu.js b/src/path-tracer-gpu.js
index 7962d618..54738d9f 100644
--- a/src/path-tracer-gpu.js
+++ b/src/path-tracer-gpu.js
@@ -191,6 +191,13 @@ function getContext() {
return session;
}
+let kernelCompileCount = 0;
+
+/** How many path tracing kernels (the light bake's texel kernel not counted) this page has compiled so far: a job of many frames compiles one, because every frame reuses the cached program. */
+export function gpuKernelCompileCount() {
+ return kernelCompileCount;
+}
+
/** The kernel for a scene, compiled once per variant: a scene with a subsurface part gets the random walk, any other scene the plain kernel. */
function getKernel(hasSubsurface, texelBake = false) {
const { gl, programs } = getContext();
@@ -200,6 +207,7 @@ function getKernel(hasSubsurface, texelBake = false) {
const program = createProgram(gl, { hasSubsurface, texelBake });
gl.getProgramParameter(program, gl.LINK_STATUS);
programs.set(key, { program, compileSeconds: (performance.now() - started) / 1000 });
+ if (!texelBake) kernelCompileCount++;
}
return programs.get(key);
}
diff --git a/src/path-tracer.js b/src/path-tracer.js
index 4b7a44f7..ffcd7229 100644
--- a/src/path-tracer.js
+++ b/src/path-tracer.js
@@ -1043,11 +1043,13 @@ export async function renderPathTracedImage({ parts, environment = {}, camera },
const threads = typeof Worker === 'function' ? options.threads : 1;
let gpuName = null;
let gpuCompileSeconds = 0;
+ let gpuKernelCompiles = 0;
if (options.backend === 'gpu') {
- const { traceOnGpu } = await import('./path-tracer-gpu.js');
+ const { traceOnGpu, gpuKernelCompileCount } = await import('./path-tracer-gpu.js');
const traced = await traceOnGpu(scene, view, options, shouldStop);
gpuName = traced.gpu;
gpuCompileSeconds = traced.compileSeconds;
+ gpuKernelCompiles = gpuKernelCompileCount();
copyBand(buffers, traced, 0, width);
} else if (threads > 1) {
await renderBandsInWorkers(scene, view, { ...options, threads }, buffers, shouldStop);
@@ -1064,7 +1066,7 @@ export async function renderPathTracedImage({ parts, environment = {}, camera },
report: {
width, height, samples: options.samples, bounces: options.bounces, seed: options.seed, denoised: options.denoiser === 'wavelet',
denoiser: options.denoiser, threads,
- backend: options.backend, ...(gpuName ? { gpu: gpuName, gpuCompileSeconds } : {}),
+ backend: options.backend, ...(gpuName ? { gpu: gpuName, gpuCompileSeconds, gpuKernelCompiles } : {}),
averageSamples: totalSamples(samplesTaken) / pixelCount, noiseThreshold: options.noiseThreshold,
triangles: scene.indices.length / 3, tracedSeconds, seconds: (performance.now() - started) / 1000,
ignored: describeIgnoredFeatures(parts, environment),
diff --git a/tests/agent-bake-gpu.test.js b/tests/agent-bake-gpu.test.js
new file mode 100644
index 00000000..271fe8e5
--- /dev/null
+++ b/tests/agent-bake-gpu.test.js
@@ -0,0 +1,48 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { canBakeOnGpu, gpuBakeKernels } from '../src/agent-bake-gpu.js';
+import { buildWithKernels, kernelStats, resetKernelStats, runKernel } from '../src/kernel-offload.js';
+
+function workerAnswering(value, workerCalls = []) {
+ const worker = {
+ postMessage(message) {
+ workerCalls.push(message.name);
+ setTimeout(() => worker.onmessage({ data: { value } }), 0);
+ },
+ terminate() {},
+ };
+ return worker;
+}
+
+test('a light with affects keeps the bake on the CPU, because the GPU kernel has no light linking', () => {
+ assert.equal(canBakeOnGpu({ pointLights: [{ position: [0, 1, 0] }], spotLights: [], areaLights: [] }), true);
+ assert.equal(canBakeOnGpu({ pointLights: [{ affects: null }] }), true);
+ assert.equal(canBakeOnGpu({ pointLights: [{ affects: ['floor'] }] }), false);
+ assert.equal(canBakeOnGpu({ areaLights: [{ affects: ['floor'] }] }), false);
+});
+
+test('a page without WebGL2 offers no GPU bake kernels, so the worker bakes on the CPU', () => {
+ assert.equal(gpuBakeKernels(), undefined);
+});
+
+test('a page kernel answers a deferred kernel when it accepts the call, and the worker answers the rest', async () => {
+ resetKernelStats();
+ const workerCalls = [];
+ const worker = workerAnswering('from the worker', workerCalls);
+ const pageKernels = { lightBake: { accepts: ([flag]) => flag === 'gpu', compute: async () => 'from the page' } };
+ const build = flag => () => runKernel('lightBake', [flag], () => 'inline');
+ assert.equal(await buildWithKernels(build('gpu'), { createWorker: () => worker, available: true, pageKernels }), 'from the page');
+ assert.deepEqual(workerCalls, []);
+ assert.equal(await buildWithKernels(build('cpu'), { createWorker: () => worker, available: true, pageKernels }), 'from the worker');
+ assert.deepEqual(workerCalls, ['lightBake']);
+ assert.equal(kernelStats().page, 1);
+});
+
+test('a page kernel that fails leaves the kernel to the worker', async () => {
+ resetKernelStats();
+ const worker = workerAnswering('from the worker');
+ const pageKernels = { lightBake: { accepts: () => true, compute: async () => { throw new Error('context lost'); } } };
+ const result = await buildWithKernels(() => runKernel('lightBake', ['x'], () => 'inline'), { createWorker: () => worker, available: true, pageKernels });
+ assert.equal(result, 'from the worker');
+ assert.match(kernelStats().fallbacks[0].reason, /page kernel failed: context lost/);
+});
diff --git a/tests/agent-render-jobs.test.js b/tests/agent-render-jobs.test.js
index 741935b7..9b369bca 100644
--- a/tests/agent-render-jobs.test.js
+++ b/tests/agent-render-jobs.test.js
@@ -5,7 +5,7 @@ import { watch, writeFileSync } from 'node:fs';
import { tmpdir, constants, setPriority } from 'node:os';
import { join } from 'node:path';
import { randomUUID, createHash } from 'node:crypto';
-import { createAgentRenderJobs } from '../scripts/agent-render-jobs.mjs';
+import { createAgentRenderJobs, jobDeadlineMs } from '../scripts/agent-render-jobs.mjs';
setPriority(0, constants.priority.PRIORITY_BELOW_NORMAL);
const limits = { maxOutputBytes: 1024, maxFrameBytes: 128, maxStoreBytes: 1024 ** 2, cleanupMs: 100 };
@@ -418,3 +418,32 @@ test('a path tracer job draws every sample unless it passes a noise threshold, w
for (const noiseThreshold of [-0.01, 0.11, Number.NaN]) await assert.rejects(submitWith({ noiseThreshold }), /noiseThreshold/);
await controller.close();
});
+
+test('a path tracer job runs on the cpu unless it asks for the gpu, and film jobs may trace every frame', async context => {
+ const root = await temporary(context);
+ const assetBytes = Buffer.from('bounded embedded GLB fixture');
+ const assetId = createHash('sha256').update(assetBytes).digest('hex');
+ const snapshot = { recipe: { format: '67js.asset', assetId }, assetBytes, clips: [{ duration: 5 }] };
+ const controller = await createAgentRenderJobs({ root, limits, async run({ directory }) {
+ await writeFile(join(directory, 'render.png.partial'), Buffer.alloc(32, 1));
+ return { artifacts: [{ temporary: 'render.png.partial', filename: 'render.png', mimeType: 'image/png' }], metrics: {} };
+ } });
+ const submitWith = (renderer, kind = 'image') => controller.submit({ sourceRevision: 1, snapshot, settings: { kind, width: 64, height: 64, duration: 1, fps: 2, renderer: { type: 'pathTracer', ...renderer } } });
+ const defaulted = await submitWith({});
+ await terminal(controller, defaulted.jobId);
+ assert.equal(controller.status(defaulted.jobId).settings.renderer.backend, 'cpu');
+ const onGpu = await submitWith({ backend: 'gpu' });
+ await terminal(controller, onGpu.jobId);
+ assert.equal(controller.status(onGpu.jobId).settings.renderer.backend, 'gpu');
+ await assert.rejects(submitWith({ backend: 'metal' }), /backend must be cpu or gpu/);
+ const film = await submitWith({ backend: 'gpu', samples: 16 }, 'video');
+ assert.equal(controller.status(film.jobId).settings.renderer.samples, 16);
+ await controller.close();
+});
+
+test('the deadline of a traced film covers every frame', () => {
+ const renderer = { type: 'pathTracer', maxSeconds: 10 };
+ const single = jobDeadlineMs({ frameCount: 1, width: 64, height: 64, renderer });
+ const film = jobDeadlineMs({ frameCount: 5, width: 64, height: 64, renderer });
+ assert.equal(film - single, 4 * 10 * 1000);
+});