Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add user post-process passes and a shared GLSL snippet validator

Nguyen Xuan Anea36e06463ea
diff --git a/README.md b/README.md
index 3f052523..808756a8 100644
--- a/README.md
+++ b/README.md
@@ -116,6 +116,7 @@ Each addon is a separate import, so a page loads only what it uses.
| Physics | `67js/addons/physics` | [Rapier contacts and joints](docs/physics.md) recorded into a timeline |
| Modeling | `67js/addons/mesh`, `67js/addons/modeling`, `67js/addons/sculpt` | [Mesh editing](docs/mesh.md), [modeling](docs/modeling.md) and [sculpt brushes](https://67js.codepawl.com/sculpt.md) |
| Offline light | `67js/addons/path-tracer`, `67js/addons/light-bake` | [Path-traced stills](https://67js.codepawl.com/path-tracer.md) and [baked lightmaps](https://67js.codepawl.com/light-bake.md) |
+| Post passes | `67js/addons/user-pass` | [Your own full-screen GLSL passes](docs/user-passes.md): dither, pixel downscale, quantise |
| Agents | `67js/addons/agent-scene`, `67js/addons/agent-asset` | [Scene sessions](docs/agent-scene.md) with revisions and checkpoints |
The [documentation page](https://67js.codepawl.com/docs.html) lists every guide.
diff --git a/changes/405.md b/changes/405.md
new file mode 100644
index 00000000..df76f3c2
--- /dev/null
+++ b/changes/405.md
@@ -0,0 +1 @@
+- Add user post-process passes (part 1 of issue #405). A scene can run its own full-screen GLSL ES 3.00 passes on the finished frame, through the `67js/addons/user-pass` addon (`installUserPasses`, `renderer.userPasses.set`), the `postPasses` recipe section and the `set-post-passes` operation. A pass reads `sceneColor`, `sceneDepth`, `resolution`, `time` and `worldNormalAt`, takes named float, vec2 to vec4 and int uniforms, and can render at a `scale` below 1 and enlarge with nearest filtering for a pixel-art look. MCP accepts no host code; a snippet passes `src/shader-snippet-validator.js` first (at most 4 passes of 4 KB, no `#` lines, no `while` or `do`, only counted `for` loops over integer literals up to 64 iterations, no `gl_` names but `gl_FragCoord`, no `u67_` names, at most 16 numeric uniforms) and every refusal carries the line, the column and a fix. A compile failure comes back as `renderError` with the driver log mapped to snippet lines, and the session stays usable. `scene_capabilities` lists the limits under `postPasses`, `scene_inspect` lists the passes, and `docs/user-passes.md` has the API and a dither plus pixel downscale example. The validator is a separate module so the material fragment hook (part 2) can reuse it.
diff --git a/changes/405.size.md b/changes/405.size.md
new file mode 100644
index 00000000..a1678709
--- /dev/null
+++ b/changes/405.size.md
@@ -0,0 +1,7 @@
+- entry:agent-scene+lazy: the user pass addon and the shader snippet validator (about 7.7 KB gzip) load only for a recipe with `postPasses`; the scene entry itself changes by -2 bytes because the repeated nullable-section operations share one branch.
+- entry:agent-asset+lazy: the same lazy addon, reachable from the asset entry's scene modules.
+- scene:agent-scene+lazy: the same lazy addon and validator, loaded on demand.
+- tool-total: three short additions an agent needs to use the feature: the postPasses field, the set-post-passes operation and the postPasses section of scene_capabilities.
+- tool:scene_create: the recipe schema gains a one-line postPasses field that points at scene_capabilities for the rules.
+- tool:scene_apply: the operation schema gains set-post-passes.
+- result:scene_capabilities: the new postPasses section holds the field list, the inputs, the rules and the limits (about 1 KB), which keeps that detail out of the tool schemas.
diff --git a/docs/mcp.md b/docs/mcp.md
index 6256fda0..d709cd24 100644
--- a/docs/mcp.md
+++ b/docs/mcp.md
@@ -178,6 +178,8 @@ Nested `volumeFog` and `bloom` fields merge. `pointLights` replaces the complete
`environment.lines` turns on outlines, toon light and polylines, and `lines: null` removes them. A material takes `toon` for its own light steps. The code loads only for a recipe that uses either. See [line art](line-art.md).
+`postPasses` (set with `set-post-passes`, `null` clears) adds full-screen GLSL passes after the finished frame. See [user passes](user-passes.md) for the rules. A compile failure comes back as `renderError` with the log mapped to the snippet's lines, and the session stays usable.
+
The preview allocates depth support at creation so enabling fog does not recreate render targets.
Active volume fog requires HDR support. Unsupported previews reject that edit before committing.
@@ -240,7 +242,7 @@ The [asset scene contract](agent-asset.md) defines root transforms, material ove
Imported scenes decode once, and `mesh_edit` and `mesh_inspect` take a part ID such as `mesh-10` to edit or check the topology of one imported part (see [the asset scene contract](agent-asset.md)); `mesh_stack` stays procedural.
Their bounds describe authored base vertices in the sampled world transforms, without claiming deformed skin or morph bounds.
-The tools expose no arbitrary JavaScript execution, remote URLs or arbitrary caller-selected file paths.
+The tools expose no arbitrary JavaScript execution, remote URLs or arbitrary caller-selected file paths. MCP accepts no host code at all. The one text that reaches a shader is a bounded GLSL fragment snippet in `postPasses`: at most 4 passes of 4 KB, with no preprocessor lines, no `while` or `do`, only counted `for` loops over integer literals (64 iterations at most), no `gl_` or `u67_` names beyond `gl_FragCoord`, and 16 declared numeric uniforms per pass. A strict validator refuses anything else, with the line, the column and a fix, before any compile. See [user passes](user-passes.md).
Release `0.8.0` supports [imported source-node tracks](agent-character-motion.md) through the bounded motion recipe.
Facial morph track authoring and rig creation remain unsupported.
Render jobs use a configured private store and fixed output filenames.
diff --git a/docs/user-passes.md b/docs/user-passes.md
new file mode 100644
index 00000000..02cc29c5
--- /dev/null
+++ b/docs/user-passes.md
@@ -0,0 +1,114 @@
+# User post-process passes
+
+A scene can add its own full-screen passes: a GLSL fragment that reads the finished frame, the depth and a normal, and writes a new frame. Use it for a pixel-art downscale, an ordered dither, a colour quantise, an outline of your own or a PS2-style look.
+
+The same field works from three places: the `67js/addons/user-pass` addon, the `postPasses` section of a recipe, and MCP (`scene_create`, `scene_apply` with `set-post-passes`). MCP accepts no host JavaScript. It accepts GLSL fragment snippets that pass a strict check before anything compiles; the rules are below.
+
+## Where a pass runs
+
+Passes run in order on the finished frame: after depth of field, motion blur, line art, bloom, grading and the view transform, as the last step before the canvas. The first pass reads the frame as it would have been shown. Each next pass reads the one before. Colours are display-referred (0 to 1), so quantising and dithering work on the values people see. The path tracer does not apply passes. Render passes (`pass: 'depth'` and the others) and the water mirror are unaffected.
+
+## A pass
+
+```json
+{ "name": "pixel-dither", "scale": 0.25, "uniforms": { "levels": 4 }, "fragment": "vec4 userPass(vec2 uv) { ... }" }
+```
+
+| Field | Meaning |
+| --- | --- |
+| `name` | Lowercase letter, then letters, digits or hyphens, 32 characters at most. Unique in the scene. |
+| `fragment` | GLSL ES 3.00 text that defines `vec4 userPass(vec2 uv)`. Helper functions and `const` values may come before it. 4096 bytes at most. |
+| `uniforms` | Named values. A number is a `float`; `[x, y]`, `[x, y, z]` and `[x, y, z, w]` are `vec2`, `vec3` and `vec4`; an int is `{ "type": "int", "value": 4 }`. |
+| `scale` | 0.05 to 1 (default 1). The pass renders at this fraction of the width and height, and whatever reads it enlarges it with nearest filtering. A `scale` of 0.25 on the last pass is a pixel-art downscale. |
+
+You do not declare uniforms or the version line; the engine writes them. A snippet sees these names:
+
+| Name | Type | Meaning |
+| --- | --- | --- |
+| `sceneColor` | `sampler2D` | The frame, or the previous pass. |
+| `sceneDepth` | `sampler2D` | Depth of the scene, at full size. It needs `depth: true` on the renderer (the MCP preview has it); without it every sample reads 1. |
+| `resolution` | `vec2` | Size of this pass in pixels. |
+| `time` | `float` | Frame time in seconds. |
+| `depthAt(uv)`, `worldPointAt(uv)` | functions | Raw depth, and the world position of the pixel. |
+| `worldNormalAt(uv)` | function | The surface normal, found from neighbouring depth samples. It is exact on flat faces and wrong at silhouettes and in the sky. |
+| `gl_FragCoord` | `vec4` | The pixel position inside this pass. |
+
+## Limits and rules
+
+The check runs on the text before any compile. A snippet that breaks a rule is refused with the line, the column and a fix.
+
+- At most 4 passes, 4096 bytes per snippet, 16 uniforms per pass. Uniform names match `^[a-z][A-Za-z0-9]{0,31}$`, are not GLSL keywords and do not reuse an input name. Values are finite numbers.
+- No `#` lines (no `#define`, `#extension`, `#version`). No backslash, and ASCII text only.
+- No `while` or `do`. A `for` loop has the exact form `for (int i = 0; i < 8; i++) { ... }`: a counter that starts at an integer literal, a test against an integer literal, a step of `i++`, `++i`, `i--`, `--i`, `i += N` or `i -= N`, and a body in braces. It runs 64 iterations at most, and the body does not change the counter.
+- Loop work stays under 4096 iterations per pixel: nested loops multiply, and a call to your own helper counts that helper's loops again.
+- No `uniform`, `layout`, `attribute`, `varying` or `buffer` declarations.
+- No name that starts with `gl_`, except `gl_FragCoord`. No name that starts with `u67_`; the engine keeps that prefix.
+
+## JavaScript
+
+```js
+import { installUserPasses } from '@codepawlhq/67js/addons/user-pass';
+
+installUserPasses(renderer); // needs depth: true for sceneDepth
+renderer.userPasses.set([{ name: 'soft', uniforms: { amount: 0.5 }, fragment: `
+ vec4 userPass(vec2 uv) {
+ vec3 color = texture(sceneColor, uv).rgb;
+ return vec4(mix(color, vec3(dot(color, vec3(0.299, 0.587, 0.114))), amount), 1.0);
+ }` }]);
+renderer.userPasses.set([]); // remove them
+```
+
+`set` can run every frame. New uniform values do not recompile; a new `fragment` compiles on the next frame. A compile or link error throws from that frame's `render` as an `Error` that carries the driver's log with line numbers mapped to your snippet. The same checks apply as for a recipe, so a snippet that works here works in a recipe.
+
+## Recipe and MCP
+
+```json
+{ "postPasses": [ { "name": "pixel-dither", "scale": 0.25, "uniforms": { "levels": 4, "spread": 1 }, "fragment": "..." } ] }
+```
+
+`scene_create` takes `postPasses`. `scene_apply` takes `{ "type": "set-post-passes", "postPasses": [...] }`; `null` or `[]` clears it. `scene_inspect` lists the passes (name, scale, uniform names with types, snippet size). `scene_capabilities` reports the same limits under `postPasses`. `scene_look` and `scene_render_submit` draw the passes.
+
+## Errors
+
+Each message names the pass, the position and a fix:
+
+```text
+Post pass "pixel-dither" fragment line 4, column 3: "while" loops are not allowed. Fix: Use a for loop with integer literal bounds of at most 64, for example for (int i = 0; i < 8; i++) { ... }.
+Post pass "pixel-dither" fragment line 2, column 1: Preprocessor lines (#) are not allowed. Fix: Remove the # line; write constants as const float name = value; and use plain code instead of #define.
+Post pass "pixel-dither" fragment line 3, column 25: This for loop runs more than 64 iterations. Fix: Lower the bound or raise the step so the loop runs at most 64 times.
+```
+
+A snippet that passes the check can still fail the compiler. In MCP the edit is kept, the result has `renderSucceeded: false` and `renderError` holds the log, and the session stays usable: send `set-post-passes` again with the fix.
+
+```text
+Post pass "pixel-dither" failed to compile as GLSL ES 3.00. Line numbers refer to your fragment snippet.
+ERROR: snippet line 7: '=' : dimension mismatch
+ | vec3 wrong = 1.0;
+ERROR: snippet line 7: '=' : cannot convert from 'const float' to 'highp 3-component vector of float'
+ | vec3 wrong = 1.0;
+```
+
+## Worked example: ordered dither with a pixel downscale
+
+The pass renders at a quarter of the width and height, mixes a 4x4 Bayer threshold into the colour, rounds each channel to `levels` steps, and is enlarged with nearest filtering. The blocks are 4x4 screen pixels and the frame has at most `levels` values per channel.
+
+```glsl
+// Ordered 4x4 dither, then colour steps. The pass renders at a quarter size, so one dither cell is a chunky pixel.
+const float bayer[16] = float[16] (0.0, 8.0, 2.0, 10.0, 12.0, 4.0, 14.0, 6.0, 3.0, 11.0, 1.0, 9.0, 15.0, 7.0, 13.0, 5.0);
+
+vec4 userPass(vec2 uv) {
+ vec3 color = texture(sceneColor, uv).rgb;
+ ivec2 cell = ivec2(gl_FragCoord.xy) & 3;
+ float threshold = (bayer[cell.y * 4 + cell.x] + 0.5) / 16.0 - 0.5;
+ vec3 stepped = floor(color * (levels - 1.0) + 0.5 + threshold * spread) / (levels - 1.0);
+ return vec4(clamp(stepped, 0.0, 1.0), 1.0);
+}
+```
+
+As a recipe section:
+
+```json
+{"postPasses":[{"name":"pixel-dither","scale":0.25,"uniforms":{"levels":4,"spread":1},"fragment":"// Ordered 4x4 dither, then colour steps. The pass renders at a quarter size, so one dither cell is a chunky pixel.\nconst float bayer[16] = float[16] (0.0, 8.0, 2.0, 10.0, 12.0, 4.0, 14.0, 6.0, 3.0, 11.0, 1.0, 9.0, 15.0, 7.0, 13.0, 5.0);\n\nvec4 userPass(vec2 uv) {\n vec3 color = texture(sceneColor, uv).rgb;\n ivec2 cell = ivec2(gl_FragCoord.xy) & 3;\n float threshold = (bayer[cell.y * 4 + cell.x] + 0.5) / 16.0 - 0.5;\n vec3 stepped = floor(color * (levels - 1.0) + 0.5 + threshold * spread) / (levels - 1.0);\n return vec4(clamp(stepped, 0.0, 1.0), 1.0);\n}"}]}
+```
+
+`levels` of 2 gives a one-bit look, 8 a soft one. `spread` of 0 turns the dither off and leaves plain steps.
diff --git a/package.json b/package.json
index 219748a1..b888f8b4 100644
--- a/package.json
+++ b/package.json
@@ -39,6 +39,7 @@
"docs/mesh.md",
"docs/modeling.md",
"docs/physics.md",
+ "docs/user-passes.md",
"docs/skeleton-pose.md",
"docs/verification.md"
],
@@ -87,6 +88,7 @@
"./addons/camera-effects": "./build/camera-effects.js",
"./addons/water-reflection": "./build/water-reflection.js",
"./addons/render-passes": "./build/render-passes.js",
+ "./addons/user-pass": "./build/user-pass.js",
"./addons/line-art": "./build/line-art.js",
"./addons/coplanar-overlap": "./build/coplanar-overlap.js",
"./addons/reversed-depth": "./build/reversed-depth.js",
@@ -132,6 +134,7 @@
"smoke:agent-assets": "node scripts/smoke-agent-assets.mjs",
"smoke:agent-vegetation": "node scripts/smoke-agent-vegetation.mjs",
"smoke:agent-render-jobs": "node scripts/smoke-agent-render-jobs.mjs",
+ "smoke:agent-post-pass": "node scripts/smoke-agent-post-pass.mjs",
"smoke:agent-texture-sequence": "node scripts/smoke-agent-texture-sequence.mjs",
"smoke:render-passes": "node scripts/smoke-render-passes.mjs",
"smoke:path-tracer": "node scripts/smoke-path-tracer.mjs",
diff --git a/scripts/agent-preview-browser.js b/scripts/agent-preview-browser.js
index e6317ea4..8dc4726b 100644
--- a/scripts/agent-preview-browser.js
+++ b/scripts/agent-preview-browser.js
@@ -229,6 +229,7 @@ function draw(record, time = 0, camera = record.scene.camera, frameOptions = {})
record.shadowKey = shadowKey;
}
record.renderer.world = environment.world;
+ record.renderer.userPasses?.set(record.scene.postPasses);
applyRecipeWindHeading(record.renderer, environment);
record.renderer.render({ ...camera, environment: litEnvironment(record.scene.parts, environment, Boolean(record.renderer.groundPart)), effects, exposure: effects.exposure, time, lines: environment.lines, ...windFrame(environment), ...frameOptions });
const error = record.renderer.context.getError();
@@ -323,8 +324,8 @@ function initializeRenderer(scene, { memoryBudgetMiB, backend }) {
samples: backend === 'cpu' ? 1 : 4,
memoryBudgetMiB,
}))));
- // Outlines, toon light and polylines draw only for a frame that carries environment.lines.
- return agentSceneModule('lineArt').installLineArt(renderer);
+ // Outlines, toon light and polylines draw only for a frame that carries environment.lines; user passes run last, on the finished frame.
+ return agentSceneModule('userPass').installUserPasses(agentSceneModule('lineArt').installLineArt(renderer));
}
// Camera fields from a timeline shot on top of the sampled camera, checked with the scene's own camera rules.
diff --git a/scripts/agent-runtime.mjs b/scripts/agent-runtime.mjs
index 69a601aa..93e9bf57 100644
--- a/scripts/agent-runtime.mjs
+++ b/scripts/agent-runtime.mjs
@@ -8,6 +8,7 @@ import { fileURLToPath } from 'node:url';
import packageMetadata from '../package.json' with { type: 'json' };
import { VEGETATION_CAPABILITY_NOTE } from '../src/agent-vegetation-notes.js';
import { CROWD_CAPABILITY_NOTE } from '../src/agent-crowd-notes.js';
+import { USER_PASS_CAPABILITIES } from '../src/user-pass.js';
import { AGENT_SCENE_CAPABILITIES, AGENT_SCENE_DEFAULT_LIMITS, RENDER_PASS_NAMES, RENDER_PASS_DEPTH_RANGE, VIDEO_ENCODERS, MAXIMUM_VIDEO_SECONDS, MAXIMUM_JOB_SIDE, MAXIMUM_JOB_PIXELS, MAXIMUM_MEMORY_MIB, BAKE_WORKER_MAXIMUM_SECONDS } from '../src/agent-scene-capabilities.js';
import { PATH_TRACER_CAPABILITIES } from '../src/path-tracer-capabilities.js';
import { chooseDenoiser, finishWithOidn } from './agent-path-trace-denoise.mjs';
@@ -574,7 +575,7 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brow
return Object.freeze({
async capabilities() {
return {
- scene: { ...AGENT_SCENE_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
+ scene: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
sceneLimits: SCENE_LIMITS,
preview: { backend, memoryBudgetMiB: memory, maximumWidth: 1024, maximumHeight: 1024, maximumImageBytes: MAXIMUM_IMAGE_BYTES, maximumActiveScenes: 1, maximumQueuedCalls: MAXIMUM_QUEUED_CALLS, evaluationTimeoutMs: EVALUATION_TIMEOUT_MS },
review: { ...REVIEW_LIMITS, backend, memoryBudgetMiB: memory, evaluationTimeoutMs: EVALUATION_TIMEOUT_MS,
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index a3b5e5fe..5cf22b9f 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -575,6 +575,7 @@ const recipe = z.strictObject({
timeline: timelineReference.nullable().optional(),
particles: z.record(z.string(), z.unknown()).nullable().optional().describe('Particle emitters and hair, seeked by motionTime. Fields: scene_capabilities particles.'),
bake: z.record(z.string(), z.unknown()).nullable().optional().describe('Bounced light baked at build time. Fields: scene_capabilities bake.'),
+ postPasses: z.array(z.record(z.string(), z.unknown())).nullable().optional().describe('Full-screen GLSL passes, [{name, fragment, uniforms, scale}]. Rules: scene_capabilities postPasses.'),
}).describe('Bounded scene recipe; the backend validates it.');
const assetRecipe = z.strictObject({
format: z.literal('67js.asset'),
@@ -614,6 +615,7 @@ const otherOperations = [
z.strictObject({ type: z.literal('set-timeline'), timeline: z.record(z.string(), z.unknown()).nullable() }),
z.strictObject({ type: z.literal('set-particles'), particles: z.record(z.string(), z.unknown()).nullable() }),
z.strictObject({ type: z.literal('set-bake'), bake: z.record(z.string(), z.unknown()).nullable() }),
+ z.strictObject({ type: z.literal('set-post-passes'), postPasses: z.array(z.record(z.string(), z.unknown())).nullable() }),
z.strictObject({ type: z.literal('set-environment'), environment }),
z.strictObject({ type: z.literal('set-effects'), effects }),
z.strictObject({ type: z.literal('set-shadow'), shadow }),
diff --git a/scripts/measure-tool-cost.mjs b/scripts/measure-tool-cost.mjs
index 8f488802..1059badb 100644
--- a/scripts/measure-tool-cost.mjs
+++ b/scripts/measure-tool-cost.mjs
@@ -5,6 +5,7 @@ import { InMemoryTransport } from '@modelcontextprotocol/server';
import { createAgentMcpServer } from './mcp-server.mjs';
import { VEGETATION_CAPABILITY_NOTE } from '../src/agent-vegetation-notes.js';
import { CROWD_CAPABILITY_NOTE } from '../src/agent-crowd-notes.js';
+import { USER_PASS_CAPABILITIES } from '../src/user-pass.js';
import { createAgentScene, AGENT_SCENE_CAPABILITIES } from '../src/agent-scene.js';
import { TEXTURE_SEQUENCE_CAPABILITIES } from '../src/texture-sequence-limits.js';
import { loadAgentSceneModules } from '../src/agent-scene-modules.js';
@@ -71,7 +72,7 @@ async function typicalResults(recipe) {
const meshReport = scene.inspectMeshes({ geometries: ['seat'] });
const { inspection, ...statusWithoutInspection } = status;
return {
- scene_capabilities: { ...AGENT_SCENE_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
+ scene_capabilities: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
scene_create: status,
scene_inspect: { sessionId: SAMPLE_SESSION, ...scene.inspect() },
mesh_inspect: { sessionId: SAMPLE_SESSION, ...meshReport },
diff --git a/scripts/smoke-agent-post-pass.mjs b/scripts/smoke-agent-post-pass.mjs
new file mode 100644
index 00000000..495730fd
--- /dev/null
+++ b/scripts/smoke-agent-post-pass.mjs
@@ -0,0 +1,152 @@
+import assert from 'node:assert/strict';
+import { mkdtemp, mkdir, readFile, writeFile } from 'node:fs/promises';
+import { tmpdir, constants, setPriority } from 'node:os';
+import { join, resolve } from 'node:path';
+import { setTimeout as delay } from 'node:timers/promises';
+import sharp from 'sharp';
+import { Client } from '@modelcontextprotocol/client';
+import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
+import { createAgentPreviewFixture } from './agent-preview-fixture.mjs';
+
+// Proves the user post pass through the MCP server: a scene with an ordered dither and a quarter-size downscale looks
+// quantised and blocky, a broken snippet returns the mapped compile log and leaves the session usable, a snippet that
+// breaks a validator rule is refused before any compile, and a render job draws the pass too.
+
+setPriority(0, constants.priority.PRIORITY_BELOW_NORMAL);
+const WIDTH = 320;
+const HEIGHT = 240;
+const SCALE = 0.25;
+const LEVELS = 4;
+const BLOCK = 4;
+const directory = await mkdtemp(join(tmpdir(), '67js-mcp-post-pass-'));
+const jobs = join(directory, 'jobs');
+const cli = resolve(process.argv[2] ?? 'bin/67js-mcp.mjs');
+let diagnostics = '';
+
+const transport = new StdioClientTransport({ command: process.execPath,
+ args: [cli, '--backend', 'cpu', '--memory', '128', '--jobs', jobs], cwd: directory, stderr: 'pipe' });
+transport.stderr.on('data', bytes => { diagnostics = (diagnostics + bytes).slice(-8192); });
+const client = new Client({ name: '67js-post-pass-proof', version: '1' });
+
+async function call(name, argumentsValue = {}) {
+ const result = await client.callTool({ name, arguments: argumentsValue }, { timeout: 60000 });
+ assert.equal(result.isError, undefined, result.content?.find(item => item.type === 'text')?.text);
+ return result;
+}
+
+async function callExpectingError(name, argumentsValue) {
+ const result = await client.callTool({ name, arguments: argumentsValue }, { timeout: 60000 });
+ assert.equal(result.isError, true, `${name} should have been refused`);
+ return result.content.find(item => item.type === 'text').text;
+}
+
+// Distinct colours, and the share of aligned BLOCK x BLOCK cells whose pixels are all the same colour.
+async function measure(png) {
+ const { data, info } = await sharp(png).raw().toBuffer({ resolveWithObject: true });
+ assert.equal(info.width, WIDTH);
+ const colorAt = (x, y) => data.readUIntBE((y * info.width + x) * info.channels, 3);
+ const colors = new Set();
+ let uniformBlocks = 0;
+ let blocks = 0;
+ for (let y = 0; y + BLOCK <= info.height; y += BLOCK) {
+ for (let x = 0; x + BLOCK <= info.width; x += BLOCK) {
+ const first = colorAt(x, y);
+ let uniform = true;
+ for (let offsetY = 0; offsetY < BLOCK; offsetY++) {
+ for (let offsetX = 0; offsetX < BLOCK; offsetX++) {
+ const color = colorAt(x + offsetX, y + offsetY);
+ colors.add(color);
+ if (color !== first) uniform = false;
+ }
+ }
+ blocks++;
+ if (uniform) uniformBlocks++;
+ }
+ }
+ return { colors: colors.size, uniformBlockShare: uniformBlocks / blocks };
+}
+
+const ditherFragment = `const float bayer[16] = float[16] (0.0, 8.0, 2.0, 10.0, 12.0, 4.0, 14.0, 6.0, 3.0, 11.0, 1.0, 9.0, 15.0, 7.0, 13.0, 5.0);
+vec4 userPass(vec2 uv) {
+ vec3 color = texture(sceneColor, uv).rgb;
+ ivec2 cell = ivec2(gl_FragCoord.xy) & 3;
+ float threshold = (bayer[cell.y * 4 + cell.x] + 0.5) / 16.0 - 0.5;
+ vec3 stepped = floor(color * (levels - 1.0) + 0.5 + threshold * spread) / (levels - 1.0);
+ return vec4(clamp(stepped, 0.0, 1.0), 1.0);
+}`;
+const ditherPass = { name: 'pixel-dither', scale: SCALE, uniforms: { levels: LEVELS, spread: 1 }, fragment: ditherFragment };
+const brokenPass = { ...ditherPass, fragment: ditherFragment.replace('return vec4(clamp(stepped, 0.0, 1.0), 1.0);', 'vec3 wrong = 1.0;\n return vec4(clamp(stepped, 0.0, 1.0), 1.0);') };
+
+async function look(sessionId) {
+ return Buffer.from((await call('scene_look', { sessionId, width: WIDTH, height: HEIGHT })).content[0].data, 'base64');
+}
+
+async function waitForJob(jobId) {
+ const deadline = Date.now() + 90000;
+ while (Date.now() < deadline) {
+ const record = (await call('scene_render_status', { jobId })).structuredContent;
+ if (['completed', 'cancelled', 'failed'].includes(record.state)) return record;
+ await delay(100);
+ }
+ throw new Error('The render job did not finish within 90 seconds.');
+}
+
+try {
+ await client.connect(transport);
+ assert.equal((await client.listTools()).tools.length, 22);
+ const capabilities = (await call('scene_capabilities')).structuredContent;
+ assert.equal(capabilities.scene.postPasses.limits.maxPasses, 4);
+ assert.ok(capabilities.scene.operations.includes('set-post-passes'));
+
+ const created = (await call('scene_create', { recipe: createAgentPreviewFixture() })).structuredContent;
+ const sessionId = created.sessionId;
+ const plain = await measure(await look(sessionId));
+ assert.ok(plain.colors > 400, `the plain scene should have many colours, has ${plain.colors}`);
+ assert.ok(plain.uniformBlockShare < 0.9, `the plain scene should not be blocky, share ${plain.uniformBlockShare}`);
+
+ // A rule the validator enforces refuses the edit before anything compiles, and nothing changes.
+ const refused = await callExpectingError('scene_apply', { sessionId, expectedRevision: 0,
+ operations: [{ type: 'set-post-passes', postPasses: [{ ...ditherPass, fragment: `${ditherFragment}\n#define X 1` }] }] });
+ assert.match(refused, /Preprocessor lines \(#\) are not allowed/);
+ assert.match(refused, /line 9, column 1/);
+ assert.match(refused, /Fix: /);
+
+ // A snippet the validator accepts but the compiler rejects: committed, with the log mapped to snippet lines.
+ const broken = (await call('scene_apply', { sessionId, expectedRevision: 0, operations: [{ type: 'set-post-passes', postPasses: [brokenPass] }] })).structuredContent;
+ assert.equal(broken.committed, true);
+ assert.equal(broken.renderSucceeded, false);
+ assert.match(broken.renderError, /Post pass "pixel-dither" failed to compile/);
+ assert.match(broken.renderError, /snippet line 7/);
+ assert.match(broken.renderError, /vec3 wrong = 1\.0;/);
+ const brokenRevision = broken.revision;
+
+ // The session is still usable: the fix draws, and the result is quantised and blocky.
+ const fixed = (await call('scene_apply', { sessionId, expectedRevision: brokenRevision, operations: [{ type: 'set-post-passes', postPasses: [ditherPass] }] })).structuredContent;
+ assert.equal(fixed.renderSucceeded, true, fixed.renderError);
+ assert.deepEqual(fixed.inspection.postPasses, [{ name: 'pixel-dither', scale: SCALE, uniforms: ['levels:float', 'spread:float'], fragmentBytes: ditherFragment.length }]);
+ const dithered = await measure(await look(sessionId));
+ assert.ok(dithered.colors <= LEVELS ** 3, `the dithered scene should have at most ${LEVELS ** 3} colours, has ${dithered.colors}`);
+ assert.ok(dithered.uniformBlockShare > 0.99, `the dithered scene should be ${BLOCK}x${BLOCK} blocks, share ${dithered.uniformBlockShare}`);
+
+ // A render job applies the pass too.
+ const submitted = (await call('scene_render_submit', { sessionId, expectedRevision: fixed.revision,
+ settings: { kind: 'image', width: WIDTH, height: HEIGHT } })).structuredContent;
+ assert.equal((await waitForJob(submitted.jobId)).state, 'completed');
+ const artifact = (await call('scene_render_result', { jobId: submitted.jobId })).structuredContent.artifacts[0];
+ const rendered = await measure(await readFile(artifact.path));
+ assert.ok(rendered.colors <= LEVELS ** 3 && rendered.uniformBlockShare > 0.99, `the render job should apply the pass: ${JSON.stringify(rendered)}`);
+ await call('scene_render_release', { jobId: submitted.jobId });
+
+ // Clearing the pass restores the plain image.
+ await call('scene_apply', { sessionId, expectedRevision: fixed.revision, operations: [{ type: 'set-post-passes', postPasses: null }] });
+ const cleared = await measure(await look(sessionId));
+ assert.ok(cleared.colors > 400 && cleared.uniformBlockShare < 0.9);
+
+ await mkdir('artifacts', { recursive: true });
+ const proof = { compileLog: broken.renderError, plain, dithered, rendered, cleared, levels: LEVELS, scale: SCALE, refusedBeforeCompile: true, mappedCompileLog: true, sessionUsableAfterError: true };
+ await writeFile('artifacts/mcp-post-pass-proof.json', JSON.stringify(proof, null, 2));
+ console.log(JSON.stringify(proof));
+} finally {
+ await client.close();
+ if (diagnostics.trim()) console.error(diagnostics.trim().split('\n').slice(-5).join('\n'));
+}
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index 34b6b201..7eb92cf3 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -177,7 +177,7 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
operations: Object.freeze([
'set-node', 'set-material', 'set-camera', 'add-node', 'remove-node',
'add-geometry', 'remove-geometry', 'add-material', 'remove-material',
- 'set-environment', 'set-effects', 'set-shadow', 'set-motion', 'set-armatures', 'set-physics', 'set-cloth', 'set-timeline', 'set-particles', 'set-bake', 'edit-mesh', 'set-mesh-stack',
+ 'set-environment', 'set-effects', 'set-shadow', 'set-motion', 'set-armatures', 'set-physics', 'set-cloth', 'set-timeline', 'set-particles', 'set-bake', 'set-post-passes', 'edit-mesh', 'set-mesh-stack',
]),
materialTags: Object.freeze(['surface', 'unlit', 'liquid', 'foam', 'foliage', 'grass', 'water']),
// A flat water sheet; the largest sets the level. A bed d metres down keeps exp(-absorption * d) per channel (RGB per metre).
diff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.js
index 800c42d0..90da1272 100644
--- a/src/agent-scene-modules.js
+++ b/src/agent-scene-modules.js
@@ -31,6 +31,7 @@ const MODULE_LOADERS = {
lineArt: () => import('./line-art.js'),
viewTransform: () => import('./view-transform-shader.js'),
coplanar: () => import('./coplanar-overlap.js'),
+ userPass: () => import('./user-pass.js'),
};
const loadedModules = {};
@@ -70,6 +71,7 @@ const MODULE_MARKERS = {
viewTransform: ['"viewTransform":"agx"', '"viewTransform":"neutral"', '"look":"'],
// The coplanar-overlap warning in scene_inspect appears only once this is loaded, which loadAllAgentSceneModules does; a recipe with resolveCoplanar needs it.
coplanar: ['"resolveCoplanar"'],
+ userPass: ['"postPasses"'],
};
/** Whether a recipe, operation list or edit list contains a boolean edit, which needs loadMeshBooleans() first. */
diff --git a/src/agent-scene.js b/src/agent-scene.js
index a3d4f394..cc67ea10 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -32,6 +32,11 @@ const LOOK_OPERATION_FIELDS = Object.freeze({
'set-effects': 'effects',
'set-shadow': 'shadow',
});
+// Each of these replaces one recipe section whole; null clears it.
+const NULLABLE_SECTION_OPERATIONS = Object.freeze({
+ 'set-motion': 'motion', 'set-physics': 'physics', 'set-timeline': 'timeline', 'set-cloth': 'cloth',
+ 'set-bake': 'bake', 'set-particles': 'particles', 'set-post-passes': 'postPasses', 'set-armatures': 'armatures',
+});
const ADDITION_FIELDS = Object.freeze({
'add-geometry': ['geometries', 'geometry'],
'add-material': ['materials', 'material'],
@@ -661,7 +666,7 @@ function linkedMaterial(liveMaterial, lightMask, wrappers) {
function normalizeRecipe(input, limits) {
let recipe = copiedJson(input, limits.maxRecipeBytes);
- knownKeys(recipe, ['format', 'schemaVersion', 'geometries', 'materials', 'nodes', 'armatures', 'camera', 'environment', 'effects', 'shadow', 'motion', 'physics', 'cloth', 'timeline', 'particles', 'bake'], 'recipe');
+ knownKeys(recipe, ['format', 'schemaVersion', 'geometries', 'materials', 'nodes', 'armatures', 'camera', 'environment', 'effects', 'shadow', 'motion', 'physics', 'cloth', 'timeline', 'particles', 'bake', 'postPasses'], 'recipe');
if (recipe.format !== '67js.scene' || recipe.schemaVersion !== 1) {
throw new RangeError('Expected 67js.scene schemaVersion 1.');
}
@@ -732,6 +737,7 @@ function normalizeRecipe(input, limits) {
if (Object.hasOwn(recipe, 'bake')) {
recipe.bake = recipe.bake === null ? null : agentSceneModule('bake').normalizeAgentBake(recipe.bake);
}
+ if (Object.hasOwn(recipe, 'postPasses')) recipe.postPasses = agentSceneModule('userPass').normalizeUserPasses(recipe.postPasses ?? []);
recipe = copiedJson(recipe, limits.maxRecipeBytes);
return recipe;
}
@@ -932,6 +938,19 @@ function authoredRecipe(recipe) {
return authored;
}
+// The mesh geometry an edit names; `remedy` tells what to do when it is another type.
+function meshGeometryTarget(candidate, operation, remedy) {
+ identifier(operation.geometry);
+ const target = candidate.geometries.find(geometry => geometry.id === operation.geometry);
+ if (!target) {
+ throw new RangeError(`Geometry ID was not found: ${operation.geometry}. Known geometry IDs: ${[...candidate.geometries.map(known => known.id)].join(', ')}.`);
+ }
+ if (target.type !== 'mesh') {
+ throw new TypeError(`Geometry ${operation.geometry} is a ${target.type}. ${remedy(target.type)}`);
+ }
+ return target;
+}
+
function removeNode(candidate, operation) {
knownKeys(operation, ['type', 'id', 'recursive'], 'operation');
identifier(operation.id);
@@ -1351,6 +1370,9 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
get environment() {
return structuredClone(state.recipe.environment);
},
+ get postPasses() {
+ return structuredClone(state.recipe.postPasses ?? []);
+ },
get effects() {
return structuredClone(state.recipe.effects);
},
@@ -1379,64 +1401,14 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
const removedNodes = new Set();
const removedMaterials = new Set();
for (const operation of operations) {
- if (operation.type === 'set-motion') {
- knownKeys(operation, ['type', 'motion'], 'operation');
- if (!Object.hasOwn(operation, 'motion')) {
- throw new TypeError('set-motion requires motion; use null to clear it.');
- }
- candidate.motion = operation.motion;
- continue;
- }
- if (operation.type === 'set-armatures') {
- knownKeys(operation, ['type', 'armatures'], 'operation');
- if (!Object.hasOwn(operation, 'armatures')) {
- throw new TypeError('set-armatures requires armatures; use null to clear them.');
- }
- if (operation.armatures === null) {
- delete candidate.armatures;
- } else {
- candidate.armatures = operation.armatures;
- }
- continue;
- }
- if (operation.type === 'set-physics') {
- knownKeys(operation, ['type', 'physics'], 'operation');
- if (!Object.hasOwn(operation, 'physics')) {
- throw new TypeError('set-physics requires physics; use null to clear it.');
- }
- candidate.physics = operation.physics;
- continue;
- }
- if (operation.type === 'set-timeline') {
- knownKeys(operation, ['type', 'timeline'], 'operation');
- if (!Object.hasOwn(operation, 'timeline')) {
- throw new TypeError('set-timeline requires timeline; use null to clear it.');
- }
- candidate.timeline = operation.timeline;
- continue;
- }
- if (operation.type === 'set-cloth') {
- knownKeys(operation, ['type', 'cloth'], 'operation');
- if (!Object.hasOwn(operation, 'cloth')) {
- throw new TypeError('set-cloth requires cloth; use null to clear it.');
- }
- candidate.cloth = operation.cloth;
- continue;
- }
- if (operation.type === 'set-bake') {
- knownKeys(operation, ['type', 'bake'], 'operation');
- if (!Object.hasOwn(operation, 'bake')) {
- throw new TypeError('set-bake requires bake; use null to clear it.');
+ const section = Object.hasOwn(NULLABLE_SECTION_OPERATIONS, operation.type) ? NULLABLE_SECTION_OPERATIONS[operation.type] : null;
+ if (section) {
+ knownKeys(operation, ['type', section], 'operation');
+ if (!Object.hasOwn(operation, section)) {
+ throw new TypeError(`${operation.type} requires ${section}; use null to clear it.`);
}
- candidate.bake = operation.bake;
- continue;
- }
- if (operation.type === 'set-particles') {
- knownKeys(operation, ['type', 'particles'], 'operation');
- if (!Object.hasOwn(operation, 'particles')) {
- throw new TypeError('set-particles requires particles; use null to clear it.');
- }
- candidate.particles = operation.particles;
+ candidate[section] = operation[section];
+ if (candidate.armatures === null) delete candidate.armatures;
continue;
}
const lookField = Object.hasOwn(LOOK_OPERATION_FIELDS, operation.type) ? LOOK_OPERATION_FIELDS[operation.type] : null;
@@ -1476,14 +1448,7 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
}
if (operation.type === 'edit-mesh') {
knownKeys(operation, ['type', 'geometry', 'edits', 'replace'], 'operation');
- identifier(operation.geometry);
- const target = candidate.geometries.find(geometry => geometry.id === operation.geometry);
- if (!target) {
- throw new RangeError(`Geometry ID was not found: ${operation.geometry}. Known geometry IDs: ${[...candidate.geometries.map(known => known.id)].join(', ')}.`);
- }
- if (target.type !== 'mesh') {
- throw new TypeError(`Geometry ${operation.geometry} is a ${target.type}. Add a mesh geometry with base: { type: '${target.type}', ... } to edit it.`);
- }
+ const target = meshGeometryTarget(candidate, operation, type => `Add a mesh geometry with base: { type: '${type}', ... } to edit it.`);
if (operation.replace !== undefined && typeof operation.replace !== 'boolean') {
throw new TypeError('replace must be boolean.');
}
@@ -1496,14 +1461,7 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
}
if (operation.type === 'set-mesh-stack') {
knownKeys(operation, ['type', 'geometry', 'stack', 'apply'], 'operation');
- identifier(operation.geometry);
- const target = candidate.geometries.find(geometry => geometry.id === operation.geometry);
- if (!target) {
- throw new RangeError(`Geometry ID was not found: ${operation.geometry}. Known geometry IDs: ${[...candidate.geometries.map(known => known.id)].join(', ')}.`);
- }
- if (target.type !== 'mesh') {
- throw new TypeError(`Geometry ${operation.geometry} is a ${target.type}. Only mesh geometries have a modifier stack.`);
- }
+ const target = meshGeometryTarget(candidate, operation, () => 'Only mesh geometries have a modifier stack.');
const stack = operation.stack ?? target.stack ?? [];
agentSceneModule('mesh').normalizeMeshStack(stack);
const applied = operation.apply ?? 0;
@@ -1605,6 +1563,7 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
...(state.assetRuntime.placements.length ? { assets: agentSceneModule('assets').describeAssetPlacements(state.assetRuntime, state.graph, geometry => state.localBounds.get(geometry)) } : {}),
...(state.sequences.length ? { textureSequences: state.sequences.map(sequence => sequence.describe()) } : {}),
...(state.bakeReport ? { bake: state.bakeReport } : {}),
+ ...(state.recipe.postPasses?.length ? { postPasses: agentSceneModule('userPass').describeUserPasses(state.recipe.postPasses) } : {}),
...(state.reflectionReport ? { reflections: state.reflectionReport } : {}),
...(state.recipe.armatures ? { rig: rigReport(state.recipe, state.skinBindings) } : {}),
boundsSpace: 'authored-base-world',
diff --git a/src/shader-snippet-validator.js b/src/shader-snippet-validator.js
new file mode 100644
index 00000000..a2bb9d70
--- /dev/null
+++ b/src/shader-snippet-validator.js
@@ -0,0 +1,389 @@
+// Strict checks for GLSL snippets that an agent sends as data. Nothing here touches WebGL: the checks run on the text
+// before any compile, so a snippet that breaks a rule never reaches a driver. The post-process pass (user-pass.js) is the
+// first user; a material fragment hook can reuse validateShaderSnippet, validateSnippetUniforms and mapShaderInfoLog.
+//
+// A snippet is one GLSL ES 3.00 fragment of at most 4 KB. It declares no uniforms (they come from a separate object),
+// has no preprocessor, no `while` or `do`, and only counted `for` loops over integer literals, so every snippet ends.
+
+export const SNIPPET_LIMITS = Object.freeze({
+ maxBytes: 4096,
+ maxPasses: 4,
+ maxUniforms: 16,
+ maxLoopIterations: 64,
+ // Loop iterations of one call, with nested loops multiplied and helper function calls counted in.
+ maxLoopWork: 4096,
+});
+
+export const UNIFORM_NAME_PATTERN = /^[a-z][A-Za-z0-9]{0,31}$/;
+export const UNIFORM_TYPES = Object.freeze(['float', 'vec2', 'vec3', 'vec4', 'int']);
+const VECTOR_LENGTHS = Object.freeze({ vec2: 2, vec3: 3, vec4: 4 });
+// The engine's own names start with this; a snippet may not write it, so it cannot reach or shadow engine internals.
+export const ENGINE_PREFIX = 'u67_';
+const ALLOWED_BUILTIN_VARIABLE = 'gl_FragCoord';
+
+// GLSL ES 3.00 keywords and reserved words that fit the uniform name pattern.
+const RESERVED_WORDS = new Set(`attribute const uniform varying break continue do for while if else in out inout float int void bool
+ true false discard return mat2 mat3 mat4 vec2 vec3 vec4 ivec2 ivec3 ivec4 uvec2 uvec3 uvec4 bvec2 bvec3 bvec4 struct precision highp
+ mediump lowp layout centroid flat smooth patch sample buffer shared coherent volatile restrict readonly writeonly resource common
+ partition active asm class union enum typedef template this packed goto inline noinline public static extern external interface
+ long short double half fixed unsigned superp input output filter namespace using sizeof cast switch case default uint
+ sampler2D sampler3D samplerCube`.split(/\s+/));
+
+export class ShaderSnippetError extends Error {
+ constructor(label, problem, hint, position) {
+ const where = position ? ` line ${position.line}, column ${position.column}` : '';
+ super(`${label}${where}: ${problem} Fix: ${hint}`);
+ this.name = 'ShaderSnippetError';
+ this.line = position?.line ?? null;
+ this.column = position?.column ?? null;
+ this.hint = hint;
+ }
+}
+
+/** Rejects an object that carries a field outside `allowed`, naming the allowed ones. */
+export function requireKnownFields(value, allowed, label) {
+ if (!value || typeof value !== 'object' || Array.isArray(value)) throw new TypeError(`${label} must be an object.`);
+ for (const key of Object.keys(value)) {
+ if (!allowed.includes(key)) throw new TypeError(`Unknown ${label} field: ${key}. Allowed fields: ${allowed.join(', ')}.`);
+ }
+}
+
+function positionOf(text, index) {
+ let line = 1;
+ let lineStart = 0;
+ for (let cursor = 0; cursor < index; cursor++) {
+ if (text[cursor] === '\n') {
+ line++;
+ lineStart = cursor + 1;
+ }
+ }
+ return { line, column: index - lineStart + 1 };
+}
+
+// Comments become spaces (newlines stay), so the text that was checked is the text the driver compiles and line numbers hold.
+function blankComments(source, label) {
+ let result = '';
+ let index = 0;
+ while (index < source.length) {
+ const pair = source.slice(index, index + 2);
+ if (pair === '//') {
+ while (index < source.length && source[index] !== '\n') {
+ result += ' ';
+ index++;
+ }
+ } else if (pair === '/*') {
+ const end = source.indexOf('*/', index + 2);
+ if (end === -1) {
+ throw new ShaderSnippetError(label, 'This block comment never closes.', 'Close it with */ or use // comments.', positionOf(source, index));
+ }
+ for (; index < end + 2; index++) result += source[index] === '\n' ? '\n' : ' ';
+ } else {
+ result += source[index];
+ index++;
+ }
+ }
+ return result;
+}
+
+function checkCharacters(source, label) {
+ for (let index = 0; index < source.length; index++) {
+ const code = source.charCodeAt(index);
+ const printable = code >= 0x20 && code <= 0x7e;
+ if (source[index] === '\\') {
+ throw new ShaderSnippetError(label, 'A backslash is not allowed.', 'Remove it; a line continuation can hide code from the checks.', positionOf(source, index));
+ }
+ if (!printable && code !== 0x0a && code !== 0x0d && code !== 0x09) {
+ throw new ShaderSnippetError(label, `Character code ${code} is not allowed.`, 'Use plain ASCII text and newlines only.', positionOf(source, index));
+ }
+ }
+}
+
+const TOKEN_PATTERN = /[A-Za-z_][A-Za-z0-9_]*|\d[\w.]*|\.\d[\w.]*|\+\+|--|<<=|>>=|<=|>=|==|!=|&&|\|\||\^\^|<<|>>|[-+*/%&|^]=|\S/g;
+
+function tokenize(text) {
+ return [...text.matchAll(TOKEN_PATTERN)].map(match => ({ text: match[0], index: match.index }));
+}
+
+const isIdentifier = token => token !== undefined && /^[A-Za-z_]/.test(token.text);
+
+function checkIdentifiers(tokens, text, label) {
+ for (const token of tokens) {
+ if (!isIdentifier(token)) continue;
+ const position = positionOf(text, token.index);
+ const word = token.text;
+ if (word === 'while' || word === 'do') {
+ throw new ShaderSnippetError(label, `"${word}" loops are not allowed.`,
+ `Use a for loop with integer literal bounds of at most ${SNIPPET_LIMITS.maxLoopIterations}, for example for (int i = 0; i < 8; i++) { ... }.`, position);
+ }
+ if (['uniform', 'layout', 'attribute', 'varying', 'buffer', 'shared'].includes(word)) {
+ throw new ShaderSnippetError(label, `"${word}" declarations are not allowed in a snippet.`,
+ 'Declare values in the uniforms object of the pass; the engine writes their declarations. Use the provided inputs for the rest.', position);
+ }
+ if (word.startsWith('gl_') && word !== ALLOWED_BUILTIN_VARIABLE) {
+ throw new ShaderSnippetError(label, `"${word}" is not allowed; only ${ALLOWED_BUILTIN_VARIABLE} may start with gl_.`,
+ `Take the pixel position from ${ALLOWED_BUILTIN_VARIABLE} or the uv argument.`, position);
+ }
+ if (word.startsWith(ENGINE_PREFIX)) {
+ throw new ShaderSnippetError(label, `"${word}" starts with ${ENGINE_PREFIX}, which the engine keeps for its own names.`,
+ `Rename it without the ${ENGINE_PREFIX} prefix.`, position);
+ }
+ }
+}
+
+function matchingIndex(tokens, openIndex, open, close) {
+ let depth = 0;
+ for (let index = openIndex; index < tokens.length; index++) {
+ if (tokens[index].text === open) depth++;
+ if (tokens[index].text === close && --depth === 0) return index;
+ }
+ return -1;
+}
+
+const INTEGER_LITERAL = /^(0|[1-9]\d*)$/;
+const FOR_HEADER_HINT = `Write the header as for (int i = 0; i < 8; i++) with integer literals, at most ${SNIPPET_LIMITS.maxLoopIterations} iterations.`;
+
+// for ( int NAME = N ; NAME <op> N ; step ) { ... } with decimal literals; returns the variable and its iteration count.
+function parseForLoop(tokens, forIndex, text, label) {
+ const position = positionOf(text, tokens[forIndex].index);
+ const fail = (problem, at = forIndex) => {
+ throw new ShaderSnippetError(label, problem, FOR_HEADER_HINT, positionOf(text, tokens[Math.min(at, tokens.length - 1)].index));
+ };
+ const texts = tokens.slice(forIndex).map(token => token.text);
+ if (texts[1] !== '(' || texts[2] !== 'int' || !isIdentifier(tokens[forIndex + 3]) || texts[4] !== '=') {
+ fail('This for loop must declare one counter: int NAME = literal.');
+ }
+ const variable = texts[3];
+ if (!INTEGER_LITERAL.test(texts[5] ?? '') || texts[6] !== ';') fail('The counter must start at an integer literal.', forIndex + 5);
+ if (texts[7] !== variable || !['<', '<=', '>', '>=', '!='].includes(texts[8]) || !INTEGER_LITERAL.test(texts[9] ?? '') || texts[10] !== ';') {
+ fail('The loop test must compare the counter with an integer literal.', forIndex + 7);
+ }
+ let stepDelta;
+ let headerEnd;
+ if (texts[11] === variable && (texts[12] === '++' || texts[12] === '--') && texts[13] === ')') {
+ stepDelta = texts[12] === '++' ? 1 : -1;
+ headerEnd = 13;
+ } else if ((texts[11] === '++' || texts[11] === '--') && texts[12] === variable && texts[13] === ')') {
+ stepDelta = texts[11] === '++' ? 1 : -1;
+ headerEnd = 13;
+ } else if (texts[11] === variable && (texts[12] === '+=' || texts[12] === '-=') && INTEGER_LITERAL.test(texts[13] ?? '') && texts[14] === ')') {
+ stepDelta = Number(texts[13]) * (texts[12] === '+=' ? 1 : -1);
+ headerEnd = 14;
+ } else {
+ fail('The loop step must be i++, ++i, i--, --i, i += N or i -= N.', forIndex + 11);
+ }
+ if (texts[headerEnd + 1] !== '{') fail('The loop body must be inside { }.', forIndex + headerEnd + 1);
+ let counter = Number(texts[5]);
+ const limit = Number(texts[9]);
+ const compare = { '<': (a, b) => a < b, '<=': (a, b) => a <= b, '>': (a, b) => a > b, '>=': (a, b) => a >= b, '!=': (a, b) => a !== b }[texts[8]];
+ let iterations = 0;
+ while (compare(counter, limit) && iterations <= SNIPPET_LIMITS.maxLoopIterations) {
+ counter += stepDelta;
+ iterations++;
+ }
+ if (iterations > SNIPPET_LIMITS.maxLoopIterations) {
+ throw new ShaderSnippetError(label, `This for loop runs more than ${SNIPPET_LIMITS.maxLoopIterations} iterations.`,
+ `Lower the bound or raise the step so the loop runs at most ${SNIPPET_LIMITS.maxLoopIterations} times.`, position);
+ }
+ const bodyStart = forIndex + headerEnd + 1;
+ const bodyEnd = matchingIndex(tokens, bodyStart, '{', '}');
+ if (bodyEnd === -1) fail('The loop body never closes.', bodyStart);
+ return { variable, iterations, bodyStart, bodyEnd };
+}
+
+const ASSIGNMENTS = new Set(['=', '+=', '-=', '*=', '/=', '%=', '++', '--', '<<=', '>>=', '&=', '|=', '^=']);
+
+function checkCounterUntouched(tokens, loop, text, label) {
+ for (let index = loop.bodyStart + 1; index < loop.bodyEnd; index++) {
+ if (tokens[index].text !== loop.variable) continue;
+ const touched = ASSIGNMENTS.has(tokens[index + 1]?.text) || ['++', '--'].includes(tokens[index - 1]?.text);
+ if (touched) {
+ throw new ShaderSnippetError(label, `The loop counter "${loop.variable}" is changed inside its loop body.`,
+ 'Leave the counter alone in the body; copy it to another variable if you need to change it.', positionOf(text, tokens[index].index));
+ }
+ }
+}
+
+// Function definitions at depth 0: name -> token range of the body.
+function findFunctions(tokens) {
+ const functions = new Map();
+ let depth = 0;
+ for (let index = 0; index < tokens.length; index++) {
+ if (tokens[index].text === '{') {
+ if (depth === 0 && tokens[index - 1]?.text === ')') {
+ let open = index - 1;
+ let parentheses = 0;
+ for (; open >= 0; open--) {
+ if (tokens[open].text === ')') parentheses++;
+ if (tokens[open].text === '(' && --parentheses === 0) break;
+ }
+ const end = matchingIndex(tokens, index, '{', '}');
+ if (open > 0 && end !== -1 && isIdentifier(tokens[open - 1])) functions.set(tokens[open - 1].text, { start: index, end });
+ }
+ depth++;
+ }
+ if (tokens[index].text === '}') depth--;
+ }
+ return functions;
+}
+
+// Loop iterations of one run of tokens[start..end]: a loop multiplies its body, a call to a function of the snippet adds that function's work.
+function loopWork(tokens, start, end, loopsByIndex, functions, working, workOfFunction) {
+ let total = 0;
+ for (let index = start; index < end; index++) {
+ const loop = loopsByIndex.get(index);
+ if (loop) {
+ total += loop.iterations * loopWork(tokens, loop.bodyStart + 1, loop.bodyEnd, loopsByIndex, functions, working, workOfFunction);
+ index = loop.bodyEnd;
+ } else if (functions.has(tokens[index].text) && tokens[index + 1]?.text === '(' && !working.has(tokens[index].text)) {
+ total += workOfFunction(tokens[index].text);
+ }
+ }
+ return Math.max(1, total);
+}
+
+function checkLoopWork(tokens, loops, text, label) {
+ const loopsByIndex = new Map(loops.map(loop => [loop.forIndex, loop]));
+ const functions = findFunctions(tokens);
+ const memo = new Map();
+ const working = new Set();
+ function workOfFunction(name) {
+ if (!memo.has(name)) {
+ working.add(name);
+ const { start, end } = functions.get(name);
+ memo.set(name, loopWork(tokens, start + 1, end, loopsByIndex, functions, working, workOfFunction));
+ working.delete(name);
+ }
+ return memo.get(name);
+ }
+ let worst = 1;
+ let worstName = null;
+ for (const name of functions.keys()) {
+ const work = workOfFunction(name);
+ if (work > worst) {
+ worst = work;
+ worstName = name;
+ }
+ }
+ if (worst > SNIPPET_LIMITS.maxLoopWork) {
+ const start = functions.get(worstName).start;
+ throw new ShaderSnippetError(label, `Function ${worstName} can run ${worst} loop iterations per pixel, over the limit of ${SNIPPET_LIMITS.maxLoopWork}.`,
+ 'Shorten the loops or the nesting; nested loops multiply and every call to a helper counts its loops again.', positionOf(text, tokens[start].index));
+ }
+ return worst;
+}
+
+/**
+ * Checks one snippet and returns { source, loopWork }, where `source` is the snippet with comments blanked: compile that
+ * text, not the original. Throws ShaderSnippetError (line, column, hint) at the first rule it breaks.
+ *
+ * options.label: names the snippet in messages. options.requiredFunction: { pattern, example } when the snippet must
+ * define an entry function. options.maxBytes: lower than the default 4 KB to tighten the limit.
+ */
+export function validateShaderSnippet(source, options = {}) {
+ const label = options.label ?? 'Shader snippet';
+ const maxBytes = Math.min(options.maxBytes ?? SNIPPET_LIMITS.maxBytes, SNIPPET_LIMITS.maxBytes);
+ if (typeof source !== 'string' || source.trim() === '') {
+ throw new ShaderSnippetError(label, 'The fragment is empty or not a string.', 'Send the GLSL text as a string.');
+ }
+ if (source.length > maxBytes) {
+ throw new ShaderSnippetError(label, `The fragment is ${source.length} bytes; the limit is ${maxBytes}.`, 'Shorten the snippet or move constants into uniforms.');
+ }
+ checkCharacters(source, label);
+ const text = blankComments(source, label);
+ const preprocessor = text.indexOf('#');
+ if (preprocessor !== -1) {
+ throw new ShaderSnippetError(label, 'Preprocessor lines (#) are not allowed.', 'Remove the # line; write constants as const float name = value; and use plain code instead of #define.', positionOf(text, preprocessor));
+ }
+ const tokens = tokenize(text);
+ checkIdentifiers(tokens, text, label);
+ const loops = [];
+ tokens.forEach((token, index) => {
+ if (token.text !== 'for') return;
+ const loop = { ...parseForLoop(tokens, index, text, label), forIndex: index };
+ checkCounterUntouched(tokens, loop, text, label);
+ loops.push(loop);
+ });
+ const work = checkLoopWork(tokens, loops, text, label);
+ if (options.requiredFunction && !options.requiredFunction.pattern.test(text)) {
+ throw new ShaderSnippetError(label, 'The entry function is missing.', `Define ${options.requiredFunction.example}.`);
+ }
+ return { source: text, loopWork: work };
+}
+
+function checkFiniteNumber(value, label) {
+ if (typeof value !== 'number' || !Number.isFinite(Math.fround(value))) {
+ throw new RangeError(`${label} must be a finite number. Fix: send plain JSON numbers, not strings, NaN or Infinity.`);
+ }
+ return value;
+}
+
+function inferType(value) {
+ if (typeof value === 'number') return 'float';
+ if (Array.isArray(value)) return `vec${value.length}`;
+ return null;
+}
+
+/**
+ * Checks a pass's uniforms and returns them as { name: { type, value } }. A bare number means float and an array of 2 to 4
+ * numbers means vec2 to vec4; an int is written { type: 'int', value: 4 }. `reservedNames` are inputs the engine provides.
+ */
+export function validateSnippetUniforms(uniforms, { label = 'Shader snippet', reservedNames = [] } = {}) {
+ if (uniforms === undefined) return {};
+ if (!uniforms || typeof uniforms !== 'object' || Array.isArray(uniforms)) {
+ throw new TypeError(`${label} uniforms must be an object of name: value. Fix: for example { "levels": 4 }.`);
+ }
+ const names = Object.keys(uniforms);
+ if (names.length > SNIPPET_LIMITS.maxUniforms) {
+ throw new RangeError(`${label} has ${names.length} uniforms; the limit is ${SNIPPET_LIMITS.maxUniforms}. Fix: pack values into vec4 uniforms.`);
+ }
+ const result = {};
+ for (const name of names) {
+ const where = `${label} uniform "${name}"`;
+ if (!UNIFORM_NAME_PATTERN.test(name)) {
+ throw new RangeError(`${where}: the name must match ${UNIFORM_NAME_PATTERN}. Fix: start with a lowercase letter, then letters and digits, 32 characters at most.`);
+ }
+ if (RESERVED_WORDS.has(name) || reservedNames.includes(name)) {
+ throw new RangeError(`${where}: the name is a GLSL keyword or an input the engine provides. Fix: pick another name.`);
+ }
+ const entry = uniforms[name];
+ const explicit = entry && typeof entry === 'object' && !Array.isArray(entry);
+ const type = explicit ? entry.type : inferType(entry);
+ const value = explicit ? entry.value : entry;
+ if (explicit) requireKnownFields(entry, ['type', 'value'], where);
+ if (!UNIFORM_TYPES.includes(type)) {
+ throw new TypeError(`${where}: type must be one of ${UNIFORM_TYPES.join(', ')}. Fix: send a number, an array of 2 to 4 numbers, or { "type": "int", "value": 4 }.`);
+ }
+ if (type === 'float') {
+ checkFiniteNumber(value, where);
+ } else if (type === 'int') {
+ if (!Number.isInteger(value) || Math.abs(value) > 2 ** 31 - 1) throw new RangeError(`${where}: an int value must be a whole number within 32 bits.`);
+ } else if (!Array.isArray(value) || value.length !== VECTOR_LENGTHS[type]) {
+ throw new RangeError(`${where}: a ${type} value is an array of ${VECTOR_LENGTHS[type]} numbers.`);
+ } else {
+ value.forEach((component, index) => checkFiniteNumber(component, `${where}[${index}]`));
+ }
+ result[name] = { type, value: Array.isArray(value) ? [...value] : value };
+ }
+ return result;
+}
+
+/** The `uniform` lines for validated uniforms, one per line, in name order. */
+export function snippetUniformDeclarations(uniforms) {
+ return Object.keys(uniforms).sort().map(name => `uniform ${uniforms[name].type} ${name};`).join('\n');
+}
+
+/**
+ * Rewrites a driver info log so every line number refers to the snippet. `headerLines` counts the lines the engine put
+ * before the snippet in the compiled source; `snippet` is the checked text, used to quote the offending line.
+ */
+export function mapShaderInfoLog(log, { headerLines, snippet }) {
+ const snippetLines = snippet.split('\n');
+ return String(log).replace(/\0/g, '').trim().split('\n').map(line => {
+ const match = /^(ERROR|WARNING):\s*\d+:(\d+):\s*(.*)$/.exec(line.trim());
+ if (!match) return line;
+ const absolute = Number(match[2]);
+ const mapped = absolute - headerLines;
+ if (mapped < 1 || mapped > snippetLines.length) return `${match[1]}: engine wrapper line ${absolute}: ${match[3]}`;
+ return `${match[1]}: snippet line ${mapped}: ${match[3]}\n | ${snippetLines[mapped - 1].trim()}`;
+ }).join('\n');
+}
diff --git a/src/user-pass.js b/src/user-pass.js
new file mode 100644
index 00000000..aeb17a17
--- /dev/null
+++ b/src/user-pass.js
@@ -0,0 +1,308 @@
+import { compileProgram, fullscreenVertex } from './gl-program.js';
+import {
+ SNIPPET_LIMITS, requireKnownFields, validateShaderSnippet, validateSnippetUniforms, snippetUniformDeclarations, mapShaderInfoLog,
+} from './shader-snippet-validator.js';
+
+// User post-process passes: fragment snippets that run on the finished frame, after the stock effects, the grade and the
+// view transform, as the last step before the canvas. The core renderer carries nothing for them. installUserPasses wraps
+// resolveFrame, copies the finished frame into a texture, runs each pass in order and draws the last one to the canvas.
+// A snippet is checked by shader-snippet-validator.js before it is ever compiled.
+
+/** Names the engine provides to a snippet; a uniform cannot take one. */
+export const USER_PASS_INPUTS = Object.freeze(['sceneColor', 'sceneDepth', 'resolution', 'time', 'depthAt', 'worldPointAt', 'worldNormalAt', 'userPass']);
+export const USER_PASS_SCALE = Object.freeze({ min: 0.05, max: 1 });
+const PASS_NAME_PATTERN = /^[a-z][A-Za-z0-9-]{0,31}$/;
+const PASS_FIELDS = ['name', 'fragment', 'uniforms', 'scale'];
+const ENTRY_FUNCTION = Object.freeze({ pattern: /\bvec4\s+userPass\s*\(\s*vec2\s+[A-Za-z_]\w*\s*\)\s*\{/, example: 'vec4 userPass(vec2 uv) { return texture(sceneColor, uv); }' });
+
+/**
+ * Checks a list of passes and returns it normalized: uniforms as { name: { type, value } }, scale filled in. Throws on the
+ * first problem with a message that names the pass, the line and column, and a fix.
+ */
+export function normalizeUserPasses(input) {
+ if (!Array.isArray(input)) throw new TypeError('postPasses must be an array of { name, fragment, uniforms, scale }. Fix: send [] to clear it.');
+ if (input.length > SNIPPET_LIMITS.maxPasses) {
+ throw new RangeError(`postPasses holds ${input.length} passes; the limit is ${SNIPPET_LIMITS.maxPasses}. Fix: merge passes, or draw several effects in one fragment.`);
+ }
+ const names = new Set();
+ return input.map((entry, index) => {
+ requireKnownFields(entry, PASS_FIELDS, `postPasses[${index}]`);
+ if (typeof entry.name !== 'string' || !PASS_NAME_PATTERN.test(entry.name)) {
+ throw new RangeError(`postPasses[${index}].name must match ${PASS_NAME_PATTERN}. Fix: a lowercase letter, then letters, digits or hyphens, 32 characters at most.`);
+ }
+ if (names.has(entry.name)) throw new RangeError(`Pass name "${entry.name}" is used twice. Fix: give each pass its own name.`);
+ names.add(entry.name);
+ const scale = entry.scale ?? 1;
+ if (typeof scale !== 'number' || !(scale >= USER_PASS_SCALE.min && scale <= USER_PASS_SCALE.max)) {
+ throw new RangeError(`Post pass "${entry.name}" scale must be between ${USER_PASS_SCALE.min} and ${USER_PASS_SCALE.max}. Fix: 0.25 renders at a quarter of the width and height.`);
+ }
+ validateShaderSnippet(entry.fragment, { label: `Post pass "${entry.name}" fragment`, requiredFunction: ENTRY_FUNCTION });
+ const uniforms = validateSnippetUniforms(entry.uniforms, { label: `Post pass "${entry.name}"`, reservedNames: USER_PASS_INPUTS });
+ return { name: entry.name, fragment: entry.fragment, uniforms, scale };
+ });
+}
+
+/** What scene_capabilities reports under postPasses; it lives here so the scene entry does not carry the text. */
+export const USER_PASS_CAPABILITIES = Object.freeze({
+ fields: 'name, fragment (defines vec4 userPass(vec2 uv)), uniforms, scale 0.05-1 (1)',
+ inputs: 'sceneColor, sceneDepth (sampler2D), resolution (vec2, this pass), time (s), depthAt(uv), worldPointAt(uv), worldNormalAt(uv, from depth); gl_FragCoord',
+ uniforms: 'a number is float, [x,y(,z(,w))] is vec2-4, {type:"int",value} is int; name ^[a-z][A-Za-z0-9]{0,31}$',
+ rules: 'no # lines, while, do, uniform/layout declarations, gl_ names but gl_FragCoord, or u67_ names; for loops only as for (int i = 0; i < N; i++) { } with literal bounds, 64 iterations at most, loop work under 4096',
+ limits: Object.freeze({ maxPasses: SNIPPET_LIMITS.maxPasses, maxBytes: SNIPPET_LIMITS.maxBytes, maxUniforms: SNIPPET_LIMITS.maxUniforms, maxLoopIterations: SNIPPET_LIMITS.maxLoopIterations, maxLoopWork: SNIPPET_LIMITS.maxLoopWork }),
+ notes: 'Passes run in order on the finished frame; scale under 1 renders smaller and enlarges with nearest filtering. A compile error returns the log with snippet line numbers (renderError); the scene stays. Stills and render jobs apply them; the path tracer does not. docs/user-passes.md',
+});
+
+/** What scene_inspect reports for each pass. */
+export function describeUserPasses(passes) {
+ return passes.map(pass => ({
+ name: pass.name,
+ scale: pass.scale,
+ uniforms: Object.entries(pass.uniforms).map(([name, { type }]) => `${name}:${type}`),
+ fragmentBytes: pass.fragment.length,
+ }));
+}
+
+const FRAGMENT_HEADER = `#version 300 es
+precision highp float;
+precision highp int;
+in vec2 textureCoordinate;
+uniform sampler2D sceneColor;
+uniform sampler2D sceneDepth;
+uniform mat4 u67_inverseViewProjection;
+uniform vec2 resolution;
+uniform float time;
+out vec4 u67_output;
+float depthAt(vec2 uv) { return texture(sceneDepth, uv).r; }
+vec3 worldPointAt(vec2 uv) {
+ vec4 point = u67_inverseViewProjection * vec4(uv * 2.0 - 1.0, depthAt(uv) * 2.0 - 1.0, 1.0);
+ return point.xyz / point.w;
+}
+// Normal from the depth buffer: the cross product of the smaller step in x and in y, facing the camera.
+vec3 worldNormalAt(vec2 uv) {
+ vec2 texel = 1.0 / vec2(textureSize(sceneDepth, 0));
+ vec3 center = worldPointAt(uv);
+ vec3 right = worldPointAt(uv + vec2(texel.x, 0.0)) - center;
+ vec3 left = center - worldPointAt(uv - vec2(texel.x, 0.0));
+ vec3 up = worldPointAt(uv + vec2(0.0, texel.y)) - center;
+ vec3 down = center - worldPointAt(uv - vec2(0.0, texel.y));
+ vec3 normal = cross(dot(right, right) < dot(left, left) ? right : left, dot(up, up) < dot(down, down) ? up : down);
+ float size = length(normal);
+ return size > 1e-12 ? normal / size : vec3(0.0, 1.0, 0.0);
+}
+`;
+const FRAGMENT_FOOTER = '\nvoid main() { u67_output = userPass(textureCoordinate); }\n';
+
+/** The full fragment source of a normalized pass, and how many lines come before the snippet. */
+export function buildPassFragment(pass) {
+ const { source: snippet } = validateShaderSnippet(pass.fragment, { label: `Post pass "${pass.name}" fragment`, requiredFunction: ENTRY_FUNCTION });
+ const header = `${FRAGMENT_HEADER}${snippetUniformDeclarations(pass.uniforms)}\n`;
+ return { source: `${header}${snippet}${FRAGMENT_FOOTER}`, headerLines: header.split('\n').length - 1, snippet };
+}
+
+const copyFragment = `#version 300 es
+precision highp float;
+in vec2 textureCoordinate;
+uniform sampler2D sourceTexture;
+out vec4 outputColor;
+void main() { outputColor = texture(sourceTexture, textureCoordinate); }`;
+
+function createTexture(context, width, height, filter, withAlpha = true) {
+ const texture = context.createTexture();
+ context.bindTexture(context.TEXTURE_2D, texture);
+ context.texImage2D(context.TEXTURE_2D, 0, withAlpha ? context.RGBA8 : context.RGB8, width, height, 0, withAlpha ? context.RGBA : context.RGB, context.UNSIGNED_BYTE, null);
+ for (const parameter of [context.TEXTURE_MIN_FILTER, context.TEXTURE_MAG_FILTER]) context.texParameteri(context.TEXTURE_2D, parameter, filter);
+ context.texParameteri(context.TEXTURE_2D, context.TEXTURE_WRAP_S, context.CLAMP_TO_EDGE);
+ context.texParameteri(context.TEXTURE_2D, context.TEXTURE_WRAP_T, context.CLAMP_TO_EDGE);
+ return texture;
+}
+
+function createTarget(context, width, height) {
+ // Nearest, so a pass rendered at a lower scale is enlarged into hard pixels by whatever reads it.
+ const texture = createTexture(context, width, height, context.NEAREST);
+ const framebuffer = context.createFramebuffer();
+ context.bindFramebuffer(context.FRAMEBUFFER, framebuffer);
+ context.framebufferTexture2D(context.FRAMEBUFFER, context.COLOR_ATTACHMENT0, context.TEXTURE_2D, texture, 0);
+ if (context.checkFramebufferStatus(context.FRAMEBUFFER) !== context.FRAMEBUFFER_COMPLETE) {
+ throw new Error('Post pass framebuffer allocation failed. Fix: lower the pass scale or the render size.');
+ }
+ return { texture, framebuffer, width, height };
+}
+
+function setUniformValue(context, location, { type, value }) {
+ if (type === 'float') context.uniform1f(location, value);
+ else if (type === 'int') context.uniform1i(location, value);
+ else if (type === 'vec2') context.uniform2fv(location, value);
+ else if (type === 'vec3') context.uniform3fv(location, value);
+ else context.uniform4fv(location, value);
+}
+
+function compilationError(pass, error, built) {
+ const log = error.message.replace(/^Shader (compilation|linking) failed: /, '');
+ const mapped = mapShaderInfoLog(log, { headerLines: built.headerLines, snippet: built.snippet });
+ return new Error(`Post pass "${pass.name}" failed to compile as GLSL ES 3.00. Line numbers refer to your fragment snippet.\n${mapped}`);
+}
+
+/**
+ * Adds user passes to a renderer. `renderer.userPasses.set(passes)` takes the list `normalizeUserPasses` accepts (a bare
+ * number is a float, an array a vec2 to vec4); `set([])` removes them. A pass reads `sceneColor` (the finished frame, or the
+ * previous pass), `sceneDepth` (needs depth: true on the renderer; reads 1 without it), `resolution` (this pass's size in
+ * pixels), `time`, and the helpers depthAt, worldPointAt and worldNormalAt (a normal derived from the depth). Setting new
+ * uniform values does not recompile; a new fragment does, on the next frame, and a compile failure throws from that frame.
+ */
+export function installUserPasses(renderer) {
+ if (renderer.userPasses) return renderer;
+ const state = { passes: [], key: '[]', entries: [], capture: null, copy: null };
+ const builtByPass = new WeakMap();
+
+ function releaseEntryTarget(context, entry) {
+ if (!entry.target) return;
+ context.deleteTexture(entry.target.texture);
+ context.deleteFramebuffer(entry.target.framebuffer);
+ entry.target = null;
+ }
+
+ function releaseEntry(context, entry) {
+ releaseEntryTarget(context, entry);
+ context.deleteProgram(entry.program);
+ }
+
+ function releaseTargets(context) {
+ if (state.capture) context.deleteTexture(state.capture.texture);
+ state.capture = null;
+ state.entries.forEach(entry => releaseEntryTarget(context, entry));
+ }
+
+ function dispose(context) {
+ releaseTargets(context);
+ state.entries.forEach(entry => context.deleteProgram(entry.program));
+ state.entries = [];
+ if (state.copy) context.deleteProgram(state.copy.program);
+ state.copy = null;
+ }
+
+ // One compiled program per pass, rebuilt only when its fragment or its uniform declarations change.
+ function compiledEntries() {
+ const context = renderer.context;
+ state.entries.slice(state.passes.length).forEach(entry => releaseEntry(context, entry));
+ state.entries.length = Math.min(state.entries.length, state.passes.length);
+ state.passes.forEach((pass, index) => {
+ if (!builtByPass.has(pass)) builtByPass.set(pass, buildPassFragment(pass));
+ const built = builtByPass.get(pass);
+ const entry = state.entries[index];
+ if (entry?.source === built.source) {
+ entry.pass = pass;
+ return;
+ }
+ let program;
+ try {
+ program = compileProgram(context, fullscreenVertex, built.source);
+ } catch (error) {
+ throw compilationError(pass, error, built);
+ }
+ if (entry) releaseEntry(context, entry);
+ const names = ['sceneColor', 'sceneDepth', 'u67_inverseViewProjection', 'resolution', 'time', ...Object.keys(pass.uniforms)];
+ state.entries[index] = {
+ pass, source: built.source, program, target: null,
+ locations: Object.fromEntries(names.map(name => [name, context.getUniformLocation(program, name)])),
+ };
+ });
+ return state.entries;
+ }
+
+ function captureFinishedFrame(width, height) {
+ const context = renderer.context;
+ if (state.capture && (state.capture.width !== width || state.capture.height !== height)) {
+ context.deleteTexture(state.capture.texture);
+ state.capture = null;
+ }
+ // A copy from the canvas needs a texture format the canvas has: without an alpha channel it has none either.
+ const withAlpha = Boolean(context.getContextAttributes()?.alpha);
+ state.capture ??= { texture: createTexture(context, width, height, context.LINEAR, withAlpha), width, height };
+ context.bindFramebuffer(context.FRAMEBUFFER, null);
+ context.bindTexture(context.TEXTURE_2D, state.capture.texture);
+ context.copyTexSubImage2D(context.TEXTURE_2D, 0, 0, 0, 0, 0, width, height);
+ return state.capture.texture;
+ }
+
+ function targetFor(entry, width, height) {
+ if (entry.target && (entry.target.width !== width || entry.target.height !== height)) releaseEntryTarget(renderer.context, entry);
+ entry.target ??= createTarget(renderer.context, width, height);
+ return entry.target;
+ }
+
+ function drawPass(entry, inputTexture, width, height, frame) {
+ const context = renderer.context;
+ const { locations, pass } = entry;
+ context.viewport(0, 0, width, height);
+ context.useProgram(entry.program);
+ context.activeTexture(context.TEXTURE0);
+ context.bindTexture(context.TEXTURE_2D, inputTexture);
+ context.uniform1i(locations.sceneColor, 0);
+ context.activeTexture(context.TEXTURE1);
+ context.bindTexture(context.TEXTURE_2D, renderer.depthTexture ?? renderer.whiteTexture);
+ context.uniform1i(locations.sceneDepth, 1);
+ context.uniformMatrix4fv(locations.u67_inverseViewProjection, false, renderer.inverseViewProjection);
+ context.uniform2f(locations.resolution, width, height);
+ context.uniform1f(locations.time, frame.time ?? 0);
+ for (const [name, uniform] of Object.entries(pass.uniforms)) setUniformValue(context, locations[name], uniform);
+ context.drawArrays(context.TRIANGLES, 0, 3);
+ }
+
+ function copyToCanvas(texture, width, height) {
+ const context = renderer.context;
+ if (!state.copy) {
+ const program = compileProgram(context, fullscreenVertex, copyFragment);
+ state.copy = { program, texture: context.getUniformLocation(program, 'sourceTexture') };
+ }
+ context.bindFramebuffer(context.FRAMEBUFFER, null);
+ context.viewport(0, 0, width, height);
+ context.useProgram(state.copy.program);
+ context.activeTexture(context.TEXTURE0);
+ context.bindTexture(context.TEXTURE_2D, texture);
+ context.uniform1i(state.copy.texture, 0);
+ context.drawArrays(context.TRIANGLES, 0, 3);
+ }
+
+ function runPasses(frame) {
+ const context = renderer.context;
+ const { width, height } = renderer.canvas;
+ const entries = compiledEntries();
+ context.disable(context.DEPTH_TEST);
+ context.disable(context.CULL_FACE);
+ context.disable(context.BLEND);
+ context.bindVertexArray(null);
+ let input = captureFinishedFrame(width, height);
+ let drewToCanvas = false;
+ entries.forEach((entry, index) => {
+ const directToCanvas = index === entries.length - 1 && entry.pass.scale === 1;
+ const passWidth = directToCanvas ? width : Math.max(1, Math.round(width * entry.pass.scale));
+ const passHeight = directToCanvas ? height : Math.max(1, Math.round(height * entry.pass.scale));
+ const target = directToCanvas ? null : targetFor(entry, passWidth, passHeight);
+ context.bindFramebuffer(context.FRAMEBUFFER, target?.framebuffer ?? null);
+ drawPass(entry, input, passWidth, passHeight, frame);
+ input = target?.texture;
+ drewToCanvas = directToCanvas;
+ });
+ if (!drewToCanvas) copyToCanvas(input, width, height);
+ }
+
+ const resolveFrame = renderer.resolveFrame;
+ renderer.resolveFrame = function resolveFrameWithUserPasses(frame, ...rest) {
+ resolveFrame.call(this, frame, ...rest);
+ if (state.passes.length > 0 && !this.benchmark) runPasses(frame);
+ };
+ // Only the lifecycle: the passes run after the final draw, not between the stock stages.
+ renderer.postStages.push({ name: 'User passes', apply: () => null, releaseTargets, dispose });
+ renderer.userPasses = {
+ set(passes) {
+ const key = JSON.stringify(passes);
+ if (key === state.key) return;
+ state.passes = normalizeUserPasses(passes);
+ state.key = key;
+ },
+ get passes() {
+ return structuredClone(state.passes);
+ },
+ };
+ return renderer;
+}
diff --git a/tests/shader-snippet-validator.test.js b/tests/shader-snippet-validator.test.js
new file mode 100644
index 00000000..3d4bea18
--- /dev/null
+++ b/tests/shader-snippet-validator.test.js
@@ -0,0 +1,148 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import {
+ SNIPPET_LIMITS, ShaderSnippetError, validateShaderSnippet, validateSnippetUniforms, snippetUniformDeclarations, mapShaderInfoLog,
+} from '../src/shader-snippet-validator.js';
+import { normalizeUserPasses, buildPassFragment, describeUserPasses, USER_PASS_INPUTS, USER_PASS_CAPABILITIES } from '../src/user-pass.js';
+import { AGENT_SCENE_CAPABILITIES } from '../src/agent-scene-capabilities.js';
+import { createAgentScene, loadAgentSceneModules } from '../src/agent-scene.js';
+
+const ENTRY = 'vec4 userPass(vec2 uv) { return texture(sceneColor, uv); }';
+const wrap = body => `vec4 userPass(vec2 uv) {\n${body}\n return texture(sceneColor, uv);\n}`;
+
+function rejects(source, pattern, position) {
+ assert.throws(() => validateShaderSnippet(source, { label: 'Test' }), error => {
+ assert.ok(error instanceof ShaderSnippetError, String(error));
+ assert.match(error.message, pattern);
+ assert.match(error.message, /Fix: /);
+ assert.ok(error.hint.length > 0);
+ if (position) assert.deepEqual([error.line, error.column], position);
+ return true;
+ });
+}
+
+test('the validator refuses each forbidden pattern and says where', () => {
+ rejects(wrap(' #define X 1'), /Preprocessor lines \(#\)/, [2, 3]);
+ rejects(wrap(' while (true) {}'), /"while" loops are not allowed/, [2, 3]);
+ rejects(wrap(' do {} while (false);'), /"do" loops are not allowed/, [2, 3]);
+ rejects(wrap(' for (int i = 0; i < 65; i++) {}'), /more than 64 iterations/, [2, 3]);
+ rejects(wrap(' for (int i = 0; i < 8; i += 0) {}'), /more than 64 iterations/);
+ rejects(wrap(' for (int i = 0; i < 8; i++) { i = 0; }'), /loop counter "i" is changed/);
+ rejects(wrap(' for (int i = 0; i < n; i++) {}'), /integer literal/);
+ rejects(wrap(' for (float f = 0.0; f < 8.0; f += 1.0) {}'), /declare one counter/);
+ rejects(wrap(' for (int i = 0; i < 8; i++) x += 1.0;'), /inside \{ \}/);
+ rejects(wrap(' float a = gl_FragDepth;'), /"gl_FragDepth" is not allowed/, [2, 13]);
+ rejects(wrap(' vec4 p = gl_Position;'), /"gl_Position" is not allowed/);
+ rejects(wrap(' float u67_x = 1.0;'), /starts with u67_/);
+ rejects(wrap(' float y = u67_output.x;'), /starts with u67_/);
+ rejects(`uniform float strength;\n${ENTRY}`, /"uniform" declarations are not allowed/, [1, 1]);
+ rejects(`layout(location = 0) out vec4 c;\n${ENTRY}`, /"layout" declarations/);
+ rejects(`${ENTRY}\n/* never closes`, /block comment never closes/, [2, 1]);
+ rejects(`${ENTRY}\n// note \\\n`, /backslash/);
+ rejects(`${ENTRY} é`, /not allowed/);
+ rejects('x'.repeat(SNIPPET_LIMITS.maxBytes + 1), /limit is 4096/);
+ rejects(' ', /empty/);
+});
+
+test('loops nest and call helpers within the work limit', () => {
+ const nested = 'void f() {\n for (int a = 0; a < 64; a++) {\n for (int b = 0; b < 64; b++) {\n for (int c = 0; c < 2; c++) {}\n }\n }\n}';
+ rejects(nested, /can run 8192 loop iterations/);
+ const viaHelper = 'float blur() { float s = 0.0; for (int a = 0; a < 64; a++) { s += 1.0; } return s; }\nvoid g() { for (int b = 0; b < 64; b++) { blur(); } }\nvoid h() { for (int c = 0; c < 2; c++) { g(); } }';
+ rejects(viaHelper, /Function h can run 8192/);
+ const fine = validateShaderSnippet('void f() { for (int a = 0; a < 64; a++) { for (int b = 8; b > 0; b -= 2) {} } }');
+ assert.equal(fine.loopWork, 256);
+});
+
+test('comments are blanked and never hide a rule', () => {
+ const checked = validateShaderSnippet(`// # while\n/* uniform */ ${ENTRY}`);
+ assert.ok(!checked.source.includes('while'));
+ assert.equal(checked.source.split('\n').length, 2);
+ rejects(`/* ok */ ${wrap(' while (a) {}')}`, /"while"/);
+});
+
+test('an entry function can be required', () => {
+ const requiredFunction = { pattern: /\bvec4\s+userPass\s*\(/, example: 'vec4 userPass(vec2 uv)' };
+ assert.throws(() => validateShaderSnippet('float x = 1.0;', { requiredFunction }), /entry function is missing.*Define vec4 userPass/);
+});
+
+test('uniforms: names, types, values and counts', () => {
+ const options = { label: 'Pass', reservedNames: USER_PASS_INPUTS };
+ const checked = validateSnippetUniforms({ amount: 0.5, tint: [1, 0, 0.5], level: { type: 'int', value: 3 }, offset: { type: 'vec2', value: [1, 2] } }, options);
+ assert.deepEqual(checked.tint, { type: 'vec3', value: [1, 0, 0.5] });
+ assert.equal(checked.level.type, 'int');
+ assert.equal(snippetUniformDeclarations(checked), 'uniform float amount;\nuniform int level;\nuniform vec2 offset;\nuniform vec3 tint;');
+ for (const [uniforms, pattern] of [
+ [{ Bad: 1 }, /must match/], [{ '1a': 1 }, /must match/], [{ a_b: 1 }, /must match/], [{ ['x'.repeat(33)]: 1 }, /must match/],
+ [{ float: 1 }, /keyword/], [{ time: 1 }, /keyword or an input/], [{ sceneColor: 1 }, /keyword or an input/],
+ [{ a: Infinity }, /finite number/], [{ a: NaN }, /finite number/], [{ a: '1' }, /type must be one of/], [{ a: [1] }, /type must be one of/],
+ [{ a: [1, 2, 3, 4, 5] }, /type must be one of/], [{ a: [1, NaN] }, /finite number/], [{ a: { type: 'int', value: 1.5 } }, /whole number/],
+ [{ a: { type: 'mat4', value: [] } }, /type must be one of/], [{ a: { type: 'vec3', value: [1, 2] } }, /array of 3/],
+ [{ a: { type: 'float', value: 1, extra: 1 } }, /Unknown/], [{ a: 1e39 }, /finite number/],
+ [Object.fromEntries(Array.from({ length: 17 }, (_, index) => [`u${index}`, 1])), /limit is 16/],
+ ]) {
+ assert.throws(() => validateSnippetUniforms(uniforms, options), pattern, JSON.stringify(uniforms).slice(0, 60));
+ }
+ assert.deepEqual(validateSnippetUniforms(undefined), {});
+});
+
+test('info log lines map to the snippet', () => {
+ const pass = normalizeUserPasses([{ name: 'broken', fragment: wrap(' vec3 v = 1.0;'), uniforms: { amount: 1 } }])[0];
+ const built = buildPassFragment(pass);
+ const absolute = built.headerLines + 2;
+ const mapped = mapShaderInfoLog(`ERROR: 0:${absolute}: '=' : cannot convert from 'const float' to 'vec3'\nERROR: 0:3: 'x' : engine\n`, built);
+ assert.match(mapped, /ERROR: snippet line 2: '=' : cannot convert/);
+ assert.match(mapped, /\| vec3 v = 1\.0;/);
+ assert.match(mapped, /ERROR: engine wrapper line 3: 'x' : engine/);
+ assert.equal(built.source.split('\n')[built.headerLines].startsWith('vec4 userPass'), true);
+ assert.ok(built.source.includes('uniform float amount;'));
+});
+
+test('the documented limits match the validator', () => {
+ const { limits } = USER_PASS_CAPABILITIES;
+ assert.deepEqual({ ...limits }, { maxPasses: 4, maxBytes: 4096, maxUniforms: 16, maxLoopIterations: 64, maxLoopWork: 4096 });
+ assert.match(USER_PASS_CAPABILITIES.rules, /64 iterations at most, loop work under 4096/);
+ assert.equal(limits.maxBytes, SNIPPET_LIMITS.maxBytes);
+ assert.equal(limits.maxUniforms, SNIPPET_LIMITS.maxUniforms);
+ assert.equal(limits.maxLoopIterations, SNIPPET_LIMITS.maxLoopIterations);
+ assert.equal(limits.maxLoopWork, SNIPPET_LIMITS.maxLoopWork);
+ assert.ok(AGENT_SCENE_CAPABILITIES.operations.includes('set-post-passes'));
+});
+
+test('passes: count, names, fields, scale and the entry function', () => {
+ const pass = (overrides = {}) => ({ name: 'one', fragment: ENTRY, ...overrides });
+ assert.equal(normalizeUserPasses([pass()])[0].scale, 1);
+ assert.throws(() => normalizeUserPasses(Array.from({ length: 5 }, (_, index) => pass({ name: `p${index}` }))), /limit is 4/);
+ assert.throws(() => normalizeUserPasses([pass(), pass()]), /used twice/);
+ assert.throws(() => normalizeUserPasses([pass({ name: 'Bad Name' })]), /name must match/);
+ assert.throws(() => normalizeUserPasses([pass({ scale: 0 })]), /scale must be between/);
+ assert.throws(() => normalizeUserPasses([pass({ scale: 2 })]), /scale must be between/);
+ assert.throws(() => normalizeUserPasses([pass({ shader: 'x' })]), /Unknown postPasses\[0\] field: shader/);
+ assert.throws(() => normalizeUserPasses([pass({ fragment: 'float x = 1.0;' })]), /Define vec4 userPass/);
+ assert.throws(() => normalizeUserPasses('x'), /must be an array/);
+ assert.deepEqual(describeUserPasses(normalizeUserPasses([pass({ uniforms: { amount: 1 } })])), [{ name: 'one', scale: 1, uniforms: ['amount:float'], fragmentBytes: ENTRY.length }]);
+});
+
+test('the documented example passes the validator and builds a scene', async () => {
+ const documentation = readFileSync(new URL('../docs/user-passes.md', import.meta.url), 'utf8');
+ const recipeText = documentation.match(/```json\n(\{"postPasses".*)\n```/)[1];
+ const { postPasses } = JSON.parse(recipeText);
+ const [pass] = normalizeUserPasses(postPasses);
+ assert.equal(pass.scale, 0.25);
+ const glsl = documentation.match(/```glsl\n([\s\S]*?)\n```/)[1];
+ assert.equal(glsl, postPasses[0].fragment);
+ const recipe = { format: '67js.scene', schemaVersion: 1, geometries: [], materials: [], nodes: [], postPasses };
+ await loadAgentSceneModules(recipe);
+ const scene = createAgentScene(recipe);
+ assert.equal(scene.postPasses[0].name, 'pixel-dither');
+ assert.equal(scene.inspect().postPasses[0].uniforms.join(), 'levels:float,spread:float');
+ scene.apply([{ type: 'set-post-passes', postPasses: null }], { expectedRevision: scene.revision });
+ assert.deepEqual(scene.postPasses, []);
+ assert.equal(scene.inspect().postPasses, undefined);
+ scene.apply([{ type: 'set-post-passes', postPasses }], { expectedRevision: scene.revision });
+ assert.equal(scene.postPasses.length, 1);
+ assert.throws(() => scene.apply([{ type: 'set-post-passes', postPasses: [{ ...postPasses[0], fragment: '#define A 1\n' + postPasses[0].fragment }] }], { expectedRevision: scene.revision }), /Preprocessor lines/);
+ assert.equal(scene.postPasses.length, 1);
+ assert.throws(() => scene.apply([{ type: 'set-post-passes' }], { expectedRevision: scene.revision }), /requires postPasses; use null to clear it/);
+ assert.deepEqual(scene.exportRecipe().postPasses, scene.postPasses);
+});
diff --git a/tests/support/preview-harness.mjs b/tests/support/preview-harness.mjs
index cbe57b19..9a9d4e30 100644
--- a/tests/support/preview-harness.mjs
+++ b/tests/support/preview-harness.mjs
@@ -77,7 +77,7 @@ export function previewHarness() {
parseGltf, Uint8Array, atob, sessionProblem, assetCopyKey, recipeAssetNeeds,
applyProjection, mat4, resolveAgentShadow, agentEnvironmentMap, BRANCH_LIMITS,
// Line art needs a real renderer with render passes, so the fault-injected one gets a no-op install.
- agentSceneModule: name => (name === 'lineArt' ? { installLineArt: renderer => renderer } : agentSceneModule(name)), agentSceneModuleLoaded, createBranchStore, printableLabel,
+ agentSceneModule: name => (name === 'lineArt' ? { installLineArt: renderer => renderer } : name === 'userPass' ? { ...agentSceneModule(name), installUserPasses: renderer => renderer } : agentSceneModule(name)), agentSceneModuleLoaded, createBranchStore, printableLabel,
Renderer67: FaultInjectedRenderer, skyOpenness: () => 1, fogSkyVisibility: () => null,
// The fault-injected renderer has no floor part; the automatic floor is covered by agent-ground.test.js and the live smokes.
agentGroundPlacement: () => null, createAgentGround: () => null, placeAgentGround: () => {}, FLOOR_COLOR: [0.22, 0.22, 0.21], groundBounceEnvironment: (parts, environment) => environment,
diff --git a/vite.lib.config.js b/vite.lib.config.js
index bd1c54a9..ced0d2a0 100644
--- a/vite.lib.config.js
+++ b/vite.lib.config.js
@@ -55,6 +55,7 @@ export default defineConfig({
'camera-effects': 'src/camera-effects.js',
'water-reflection': 'src/water-reflection.js',
'render-passes': 'src/render-passes.js',
+ 'user-pass': 'src/user-pass.js',
'line-art': 'src/line-art.js',
'coplanar-overlap': 'src/coplanar-overlap.js',
'reversed-depth': 'src/reversed-depth.js',