Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add a per-material fragment hook for colour, alpha and discard

Nguyen Xuan An58d7f7f42e5a
diff --git a/changes/405-material.md b/changes/405-material.md
new file mode 100644
index 00000000..f1786834
--- /dev/null
+++ b/changes/405-material.md
@@ -0,0 +1 @@
+- Add the material fragment hook (part 2 of issue #405). A recipe material gets an optional `snippet`: `{ fragment, uniforms, mask }`, where `fragment` defines `void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition)` and may change the colour and alpha or `discard`. It works from `scene_create` and `set-material` (`snippet: null` clears it); no new MCP tool and no host code. The hook runs in the colour pass, in the shadow pass (a discarded fragment casts no shadow) and in the ID pass. `mask: "map"` lets the snippet sample `snippetMask`, the material's own map or pattern. The snippet passes the same validator as a post pass (4 KB, no `#` lines, no `while` or `do`, counted `for` loops, no `gl_` or `u67_` names, at most 16 finite uniforms), every refusal names the line, the column and a fix, and a compile failure returns `renderError` with the log mapped to snippet lines before the running renderer is touched. `scene_capabilities` lists the limits under `materialSnippet`, and `docs/user-passes.md` has a dissolve example. The code lives in the lazy `src/material-snippet.js`, loaded when a recipe contains a `"snippet":{`. Closes #405.
diff --git a/changes/405-material.size.md b/changes/405-material.size.md
new file mode 100644
index 00000000..2c81d0c2
--- /dev/null
+++ b/changes/405-material.size.md
@@ -0,0 +1,15 @@
+- entry:core: +114 B gzip over the +0 target, the seams a material hook needs in the shared renderer: a shader feature per snippet, a `surfaceHook` slot in the colour shader before the alpha cutoff (so the ID pass sees the discard), a `withDepthCode` rewrite of the shadow shaders (so a discarded fragment casts no shadow), one binding call in the shadow pass and the snippet in the batching key. The snippet code, validator and binding are in the lazy `src/material-snippet.js`.
+- entry:agent-scene: +89 B gzip: the `snippet` material field, its lazy-module call, the snippet in the batch signature and the loader table entry.
+- entry:agent-asset: +45 B gzip: the same scene glue.
+- entry:agent-scene+lazy: the lazy material snippet module (about 3.4 KB gzip) loads only for a recipe with a `"snippet":{`.
+- scene:lit-box: +101 B gzip from the core seams above.
+- scene:gltf-box: +107 B gzip from the core seams above.
+- scene:gltf-box+lazy: +107 B gzip from the core seams above.
+- scene:agent-scene: +168 B gzip, the core seams plus the scene glue above.
+- scene:agent-scene+lazy: the lazy material snippet module, loaded on demand.
+- archive: the new lazy chunk, the smoke script and the docs section.
+- tool-total: the `snippet` field repeats in three tool schemas and the scene_capabilities section grows.
+- tool:scene_create: the recipe material schema gains one short `snippet` field.
+- tool:scene_apply: set-material takes the same `snippet` field in two operation shapes.
+- tool:scene_create_asset: it shares the material schema.
+- result:scene_capabilities: the new materialSnippet section lists the function signature, the inputs, the limits and the notes; the rules point at postPasses.
diff --git a/docs/mcp.md b/docs/mcp.md
index 65e1a0f9..2250ff24 100644
--- a/docs/mcp.md
+++ b/docs/mcp.md
@@ -242,7 +242,7 @@ The [asset scene contract](agent-asset.md) defines root transforms, material ove
Imported scenes decode once, and `mesh_edit` and `mesh_inspect` take a part ID such as `mesh-10` to edit or check the topology of one imported part (see [the asset scene contract](agent-asset.md)); `mesh_stack` stays procedural.
Their bounds describe authored base vertices in the sampled world transforms, without claiming deformed skin or morph bounds.
-The tools expose no arbitrary JavaScript execution, remote URLs or arbitrary caller-selected file paths. MCP accepts no host code at all. The one text that reaches a shader is a bounded GLSL fragment snippet in `postPasses`: at most 4 passes of 4 KB, with no preprocessor lines, no `while` or `do`, only counted `for` loops over integer literals (64 iterations at most), no `gl_` or `u67_` names beyond `gl_FragCoord`, and 16 declared numeric uniforms per pass. A strict validator refuses anything else, with the line, the column and a fix, before any compile. See [user passes](user-passes.md).
+The tools expose no arbitrary JavaScript execution, remote URLs or arbitrary caller-selected file paths. MCP accepts no host code at all. The one text that reaches a shader is a bounded GLSL fragment snippet in `postPasses`: at most 4 passes of 4 KB, with no preprocessor lines, no `while` or `do`, only counted `for` loops over integer literals (64 iterations at most), no `gl_` or `u67_` names beyond `gl_FragCoord`, and 16 declared numeric uniforms per pass. A strict validator refuses anything else, with the line, the column and a fix, before any compile. The same validator and limits cover the fragment hook of a material (`snippet`, one function of at most 4 KB, 16 uniforms, `mask` only the material's own map). See [user passes](user-passes.md).
Release `0.8.0` supports [imported source-node tracks](agent-character-motion.md) through the bounded motion recipe.
Facial morph track authoring and rig creation remain unsupported.
Render jobs use a configured private store and fixed output filenames.
diff --git a/docs/user-passes.md b/docs/user-passes.md
index 02cc29c5..84003eba 100644
--- a/docs/user-passes.md
+++ b/docs/user-passes.md
@@ -112,3 +112,52 @@ As a recipe section:
```
`levels` of 2 gives a one-bit look, 8 a soft one. `spread` of 0 turns the dither off and leaves plain steps.
+
+# Material fragment hook
+
+A material can carry its own fragment code: a GLSL function that changes the surface colour and alpha, or discards the fragment. Use it for a dissolve, a shoreline cut from a mask, a leaf cutout or a stripe pattern. It works from the `snippet` field of a recipe material and from MCP (`scene_create`, `scene_apply` with `set-material`; `snippet: null` clears it). It follows the same rules as a post pass, and MCP accepts no host code.
+
+```json
+{ "id": "stone", "color": [0.5, 0.5, 0.5], "snippet": { "fragment": "...", "uniforms": { "threshold": 0.4 }, "mask": "map" } }
+```
+
+| Field | Meaning |
+| --- | --- |
+| `fragment` | GLSL ES 3.00 text that defines `void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition)`. Helper functions and `const` values may come before it. 4096 bytes at most. |
+| `uniforms` | Named values, as for a post pass: a number is a `float`, `[x, y]` to `[x, y, z, w]` are vectors, `{ "type": "int", "value": 4 }` is an int. At most 16. |
+| `mask` | Optional, `"map"`: the snippet may sample `snippetMask` (a `sampler2D`), which is the material's own map or pattern. Load the image as the material `map`; there are no remote URLs. |
+
+Inputs: `color` is the base colour after the map and the vertex tint, `alpha` is the map alpha before the material opacity, `uv` is the texture coordinate and `worldPosition` the position in the world. `discard` removes the fragment in every alpha mode. A lowered `alpha` cuts only when the material has `alphaMode: "mask"`.
+
+## Where the hook runs
+
+The hook runs in the colour pass, in the shadow pass and in the ID pass. A discarded fragment therefore casts no shadow and is not picked. In the shadow pass `color` starts white and `alpha` is the map alpha, so base a discard on `alpha`, `uv`, `worldPosition` and your uniforms, not on `color`. The path tracer and exports ignore the hook. Changing a snippet makes the MCP preview build a new renderer, so edit it between looks, not in a loop.
+
+The rules are the ones of [a pass](#limits-and-rules): no `#` lines, no `while` or `do`, counted `for` loops only, no `gl_` names but `gl_FragCoord`, no `u67_` names. Errors name the material's parts, the line, the column and a fix. A snippet that passes the check but fails the compiler comes back as `renderError` with the log mapped to snippet lines, and the session stays usable:
+
+```text
+Material snippet (used by caster) failed to compile as GLSL ES 3.00. Line numbers refer to your fragment snippet.
+ERROR: snippet line 3: '=' : dimension mismatch
+ | vec3 wrong = 1.0;
+```
+
+## Worked example: dissolve a wall with a noise threshold
+
+The wall breaks into hash cells; a cell is removed when its noise value is below `threshold`, and cells near the edge glow orange. Moving `threshold` from 0 to 1 dissolves the wall, and the shadow of the wall dissolves with it.
+
+```glsl
+float cellNoise(vec2 cell) { return fract(sin(dot(cell, vec2(12.9898, 78.233))) * 43758.5453); }
+
+void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {
+ float value = cellNoise(floor(worldPosition.xy * 6.0));
+ if (value < threshold) discard;
+ float edge = 1.0 - smoothstep(0.0, 0.08, value - threshold);
+ color = mix(color, glow, edge);
+}
+```
+
+```json
+{ "type": "set-material", "id": "wall", "snippet": { "uniforms": { "threshold": 0.35, "glow": [1.0, 0.45, 0.1] }, "fragment": "float cellNoise(vec2 cell) { return fract(sin(dot(cell, vec2(12.9898, 78.233))) * 43758.5453); }\n\nvoid materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n float value = cellNoise(floor(worldPosition.xy * 6.0));\n if (value < threshold) discard;\n float edge = 1.0 - smoothstep(0.0, 0.08, value - threshold);\n color = mix(color, glow, edge);\n}" } }
+```
+
+For a leaf or shoreline cutout, give the material a `map` whose alpha or red channel holds the mask, set `"mask": "map"` and write `if (texture(snippetMask, uv).r < 0.5) discard;`.
diff --git a/package.json b/package.json
index 6b26803b..6b1e733c 100644
--- a/package.json
+++ b/package.json
@@ -138,6 +138,7 @@
"smoke:agent-vegetation": "node scripts/smoke-agent-vegetation.mjs",
"smoke:agent-render-jobs": "node scripts/smoke-agent-render-jobs.mjs",
"smoke:agent-post-pass": "node scripts/smoke-agent-post-pass.mjs",
+ "smoke:agent-material-snippet": "node scripts/smoke-agent-material-snippet.mjs",
"smoke:agent-texture-sequence": "node scripts/smoke-agent-texture-sequence.mjs",
"smoke:render-passes": "node scripts/smoke-render-passes.mjs",
"smoke:path-tracer": "node scripts/smoke-path-tracer.mjs",
diff --git a/scripts/agent-preview-browser.js b/scripts/agent-preview-browser.js
index e856a8d1..3db15695 100644
--- a/scripts/agent-preview-browser.js
+++ b/scripts/agent-preview-browser.js
@@ -149,7 +149,8 @@ function syncSequenceTextures(record) {
}
function sceneIsTextured(scene) {
- return hasPatternTextures(scene.parts) || (scene.sequenceTextures?.().length ?? 0) > 0;
+ // A material snippet adds code to the colour and shadow programs, which only a new renderer compiles.
+ return hasPatternTextures(scene.parts) || (scene.sequenceTextures?.().length ?? 0) > 0 || scene.parts.some(part => part.material.snippet);
}
function hasPatternTextures(parts) {
@@ -157,7 +158,13 @@ function hasPatternTextures(parts) {
}
// replaceParts streams only untextured parts, so a topology change around pattern textures takes a new renderer.
+// A material snippet is compiled alone first, so a compile error names the snippet's own lines and the running renderer stays.
+function probeSnippets(scene) {
+ if (agentSceneModuleLoaded('materialSnippet')) agentSceneModule('materialSnippet').probeMaterialSnippets(scene.parts);
+}
+
function rebuildRenderer(record) {
+ probeSnippets(record.scene);
record.renderer.dispose();
record.renderer = null;
resetCanvas();
@@ -309,6 +316,7 @@ function cascadesFor(scene) {
}
function initializeRenderer(scene, { memoryBudgetMiB, backend }) {
+ probeSnippets(scene);
const placement = agentGroundPlacement(scene.parts, scene.environment);
const renderer = installDepthPrecision(installRenderPasses(installCameraEffects(new Renderer67(canvas, scene.parts, {
ground: placement ? createAgentGround(placement) : false,
diff --git a/scripts/agent-runtime.mjs b/scripts/agent-runtime.mjs
index 93e9bf57..c5e5664b 100644
--- a/scripts/agent-runtime.mjs
+++ b/scripts/agent-runtime.mjs
@@ -9,6 +9,7 @@ import packageMetadata from '../package.json' with { type: 'json' };
import { VEGETATION_CAPABILITY_NOTE } from '../src/agent-vegetation-notes.js';
import { CROWD_CAPABILITY_NOTE } from '../src/agent-crowd-notes.js';
import { USER_PASS_CAPABILITIES } from '../src/user-pass.js';
+import { MATERIAL_SNIPPET_CAPABILITIES } from '../src/material-snippet.js';
import { AGENT_SCENE_CAPABILITIES, AGENT_SCENE_DEFAULT_LIMITS, RENDER_PASS_NAMES, RENDER_PASS_DEPTH_RANGE, VIDEO_ENCODERS, MAXIMUM_VIDEO_SECONDS, MAXIMUM_JOB_SIDE, MAXIMUM_JOB_PIXELS, MAXIMUM_MEMORY_MIB, BAKE_WORKER_MAXIMUM_SECONDS } from '../src/agent-scene-capabilities.js';
import { PATH_TRACER_CAPABILITIES } from '../src/path-tracer-capabilities.js';
import { chooseDenoiser, finishWithOidn } from './agent-path-trace-denoise.mjs';
@@ -575,7 +576,7 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', brow
return Object.freeze({
async capabilities() {
return {
- scene: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
+ scene: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, materialSnippet: MATERIAL_SNIPPET_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
sceneLimits: SCENE_LIMITS,
preview: { backend, memoryBudgetMiB: memory, maximumWidth: 1024, maximumHeight: 1024, maximumImageBytes: MAXIMUM_IMAGE_BYTES, maximumActiveScenes: 1, maximumQueuedCalls: MAXIMUM_QUEUED_CALLS, evaluationTimeoutMs: EVALUATION_TIMEOUT_MS },
review: { ...REVIEW_LIMITS, backend, memoryBudgetMiB: memory, evaluationTimeoutMs: EVALUATION_TIMEOUT_MS,
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index 5cf22b9f..fff32a13 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -121,6 +121,7 @@ const materialFields = {
...waterFields,
...materialLayerFields,
subsurface: subsurface.optional(),
+ snippet: z.record(z.string(), z.unknown()).nullable().optional().describe('GLSL fragment hook; scene_capabilities materialSnippet.'),
pattern: materialPattern.optional(),
map: z.strictObject({ sequence: z.strictObject({
frames: z.array(z.string().regex(/^[a-f0-9]{64}$/)).min(2).max(64),
diff --git a/scripts/measure-tool-cost.mjs b/scripts/measure-tool-cost.mjs
index 1059badb..2572e254 100644
--- a/scripts/measure-tool-cost.mjs
+++ b/scripts/measure-tool-cost.mjs
@@ -6,6 +6,7 @@ import { createAgentMcpServer } from './mcp-server.mjs';
import { VEGETATION_CAPABILITY_NOTE } from '../src/agent-vegetation-notes.js';
import { CROWD_CAPABILITY_NOTE } from '../src/agent-crowd-notes.js';
import { USER_PASS_CAPABILITIES } from '../src/user-pass.js';
+import { MATERIAL_SNIPPET_CAPABILITIES } from '../src/material-snippet.js';
import { createAgentScene, AGENT_SCENE_CAPABILITIES } from '../src/agent-scene.js';
import { TEXTURE_SEQUENCE_CAPABILITIES } from '../src/texture-sequence-limits.js';
import { loadAgentSceneModules } from '../src/agent-scene-modules.js';
@@ -72,7 +73,7 @@ async function typicalResults(recipe) {
const meshReport = scene.inspectMeshes({ geometries: ['seat'] });
const { inspection, ...statusWithoutInspection } = status;
return {
- scene_capabilities: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
+ scene_capabilities: { ...AGENT_SCENE_CAPABILITIES, postPasses: USER_PASS_CAPABILITIES, materialSnippet: MATERIAL_SNIPPET_CAPABILITIES, vegetation: VEGETATION_CAPABILITY_NOTE, crowd: CROWD_CAPABILITY_NOTE, textureSequence: TEXTURE_SEQUENCE_CAPABILITIES },
scene_create: status,
scene_inspect: { sessionId: SAMPLE_SESSION, ...scene.inspect() },
mesh_inspect: { sessionId: SAMPLE_SESSION, ...meshReport },
diff --git a/scripts/smoke-agent-material-snippet.mjs b/scripts/smoke-agent-material-snippet.mjs
new file mode 100644
index 00000000..65f40fc2
--- /dev/null
+++ b/scripts/smoke-agent-material-snippet.mjs
@@ -0,0 +1,182 @@
+import assert from 'node:assert/strict';
+import { mkdtemp, mkdir, writeFile } from 'node:fs/promises';
+import { tmpdir, constants, setPriority } from 'node:os';
+import { join, resolve } from 'node:path';
+import sharp from 'sharp';
+import { Client } from '@modelcontextprotocol/client';
+import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
+
+// Proves the material fragment hook through the MCP server: a valid hook changes pixels and takes new uniform values, a
+// discarded caster casts no shadow, a snippet that breaks a validator rule is refused before any compile, a snippet the
+// compiler rejects returns the log mapped to snippet lines and leaves the session usable, and null clears the hook.
+
+setPriority(0, constants.priority.PRIORITY_BELOW_NORMAL);
+const WIDTH = 320;
+const HEIGHT = 240;
+const directory = await mkdtemp(join(tmpdir(), '67js-mcp-material-snippet-'));
+const jobs = join(directory, 'jobs');
+const cli = resolve(process.argv[2] ?? 'bin/67js-mcp.mjs');
+let diagnostics = '';
+
+const transport = new StdioClientTransport({ command: process.execPath,
+ args: [cli, '--backend', 'cpu', '--memory', '128', '--jobs', jobs], cwd: directory, stderr: 'pipe' });
+transport.stderr.on('data', bytes => { diagnostics = (diagnostics + bytes).slice(-8192); });
+const client = new Client({ name: '67js-material-snippet-proof', version: '1' });
+
+async function call(name, argumentsValue = {}) {
+ const result = await client.callTool({ name, arguments: argumentsValue }, { timeout: 60000 });
+ assert.equal(result.isError, undefined, result.content?.find(item => item.type === 'text')?.text);
+ return result;
+}
+
+async function callExpectingError(name, argumentsValue) {
+ const result = await client.callTool({ name, arguments: argumentsValue }, { timeout: 60000 });
+ assert.equal(result.isError, true, `${name} should have been refused`);
+ return result.content.find(item => item.type === 'text').text;
+}
+
+async function look(sessionId) {
+ return Buffer.from((await call('scene_look', { sessionId, width: WIDTH, height: HEIGHT })).content[0].data, 'base64');
+}
+
+async function measure(png) {
+ const { data, info } = await sharp(png).raw().toBuffer({ resolveWithObject: true });
+ let red = 0;
+ let green = 0;
+ const luminance = [];
+ for (let index = 0; index < data.length; index += info.channels) {
+ red += data[index];
+ green += data[index + 1];
+ luminance.push(0.299 * data[index] + 0.587 * data[index + 1] + 0.114 * data[index + 2]);
+ }
+ const pixels = luminance.length;
+ const sorted = [...luminance].sort((first, second) => first - second);
+ const lit = sorted[Math.floor(pixels * 0.9)];
+ return { redShare: red / (red + green), darkPixels: luminance.filter(value => value < lit * 0.6).length, pixels, data };
+}
+
+function differenceBetween(first, second) {
+ let total = 0;
+ for (let index = 0; index < first.data.length; index++) total += Math.abs(first.data[index] - second.data[index]);
+ return total / first.data.length;
+}
+
+// A wide floor, and a slab high above it. The camera looks straight down at the shadow, and the slab itself is outside the
+// frame, so any change in the shadow comes from the shadow pass and not from the slab's own pixels.
+const recipe = {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'block', type: 'box' }],
+ materials: [{ id: 'floor', color: [0.7, 0.7, 0.7], roughness: 0.9 }, { id: 'slab', color: [0.4, 0.5, 0.8], roughness: 0.6 },
+ { id: 'wood', pattern: { type: 'wood', size: 0.6, axis: 'z' }, roughness: 0.8 }],
+ nodes: [
+ { id: 'ground', geometry: 'block', material: 'floor', scale: [40, 0.2, 40], translation: [0, -0.1, 0] },
+ { id: 'caster', geometry: 'block', material: 'slab', scale: [1.2, 0.3, 1.2], translation: [0, 3, 0] },
+ { id: 'panel', geometry: 'block', material: 'wood', scale: [1.4, 0.05, 1.4], translation: [5.5, 0.1, 0] },
+ ],
+ camera: { eye: [4, 20, 0], target: [4, 0, 0], up: [0, 0, -1], fov: 15, near: 1, far: 60 },
+ environment: { world: true, lightDirection: [-1, 1, 0], keyColor: [3, 2.8, 2.5], ambientIntensity: 0.25, ground: false },
+ shadow: { distance: 20, extent: 8, near: 0.1, far: 40 },
+};
+
+const tint = amount => ({ fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n color = mix(color, vec3(1.0, 0.1, 0.1), amount);\n}', uniforms: { amount } });
+const discardAll = { fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n discard;\n}' };
+const broken = { fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n color = vec3(1.0);\n vec3 wrong = 1.0;\n}' };
+
+try {
+ await client.connect(transport);
+ assert.equal((await client.listTools()).tools.length, 22);
+ const capabilities = (await call('scene_capabilities')).structuredContent;
+ assert.equal(capabilities.scene.materialSnippet.limits.maxUniforms, 16);
+
+ const created = (await call('scene_create', { recipe })).structuredContent;
+ const sessionId = created.sessionId;
+ let revision = created.revision;
+ const apply = async (operations) => {
+ const result = (await call('scene_apply', { sessionId, expectedRevision: revision, operations })).structuredContent;
+ revision = result.revision;
+ return result;
+ };
+
+ const plain = await measure(await look(sessionId));
+ assert.ok(plain.darkPixels > 400, `the plain scene should show the slab's shadow, dark pixels ${plain.darkPixels}`);
+
+ // A valid hook changes pixels, and a new uniform value takes effect without breaking anything.
+ const none = await apply([{ type: 'set-material', id: 'floor', snippet: tint(0) }]);
+ assert.equal(none.renderSucceeded, true, none.renderError);
+ const unchanged = await measure(await look(sessionId));
+ assert.ok(differenceBetween(plain, unchanged) < 1, `amount 0 should leave the picture as it was, difference ${differenceBetween(plain, unchanged)}`);
+ const reddened = await apply([{ type: 'set-material', id: 'floor', snippet: tint(0.9) }]);
+ assert.equal(reddened.renderSucceeded, true, reddened.renderError);
+ const red = await measure(await look(sessionId));
+ assert.ok(red.redShare > plain.redShare + 0.1, `the hook should redden the floor: ${plain.redShare} to ${red.redShare}`);
+ assert.ok(red.darkPixels > 400, 'the shadow is still cast while the floor is tinted');
+
+ // A discard in the colour pass cuts holes in the floor: the stripes show the background and the picture changes a lot.
+ const striped = await apply([{ type: 'set-material', id: 'floor', snippet: { fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n if (fract(worldPosition.x) < 0.5) discard;\n}' } }]);
+ assert.equal(striped.renderSucceeded, true, striped.renderError);
+ assert.ok(differenceBetween(plain, await measure(await look(sessionId))) > 5, 'a discard in the colour pass should cut the floor');
+
+ // The dissolve of docs/user-passes.md compiles in the colour and shadow programs.
+ const dissolveFragment = `float cellNoise(vec2 cell) { return fract(sin(dot(cell, vec2(12.9898, 78.233))) * 43758.5453); }
+
+void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {
+ float value = cellNoise(floor(worldPosition.xy * 6.0));
+ if (value < threshold) discard;
+ float edge = 1.0 - smoothstep(0.0, 0.08, value - threshold);
+ color = mix(color, glow, edge);
+}`;
+ const dissolved = await apply([{ type: 'set-material', id: 'floor', snippet: { uniforms: { threshold: 0.35, glow: [1.0, 0.45, 0.1] }, fragment: dissolveFragment } }]);
+ assert.equal(dissolved.renderSucceeded, true, dissolved.renderError);
+ await look(sessionId);
+
+ // The mask is the material's own map: the hook reads it in the colour pass and in the shadow pass, so both programs compile.
+ const mapped = await measure(await look(sessionId));
+ const masked = await apply([{ type: 'set-material', id: 'wood', snippet: { mask: 'map', fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n color = vec3(texture(snippetMask, uv).r, 0.0, 0.0);\n}' } }]);
+ assert.equal(masked.renderSucceeded, true, masked.renderError);
+ const maskedLook = await measure(await look(sessionId));
+ assert.ok(differenceBetween(mapped, maskedLook) > 0.2, 'the mask hook should recolour the wooden panel');
+ assert.ok(maskedLook.darkPixels > 400, 'the shadow is still cast with a mask hook in the shadow program');
+ await apply([{ type: 'set-material', id: 'wood', snippet: null }]);
+
+ // The slab's hook discards every fragment: it draws nothing, and the shadow pass runs the hook, so its shadow is gone too.
+ await apply([{ type: 'set-material', id: 'floor', snippet: null }]);
+ const removed = await apply([{ type: 'set-material', id: 'slab', snippet: discardAll }]);
+ assert.equal(removed.renderSucceeded, true, removed.renderError);
+ const unshadowed = await measure(await look(sessionId));
+ assert.ok(unshadowed.darkPixels < plain.darkPixels * 0.1, `a discarded caster should cast no shadow: ${plain.darkPixels} dark pixels before, ${unshadowed.darkPixels} after`);
+
+ // A rule the validator enforces refuses the edit before anything compiles, and nothing changes.
+ const refused = await callExpectingError('scene_apply', { sessionId, expectedRevision: revision,
+ operations: [{ type: 'set-material', id: 'slab', snippet: { ...discardAll, fragment: `#define X 1\n${discardAll.fragment}` } }] });
+ assert.match(refused, /Preprocessor lines \(#\) are not allowed/);
+ assert.match(refused, /line 1, column 1/);
+ assert.match(refused, /Fix: /);
+
+ // A snippet the validator accepts but the compiler rejects: committed, with the log mapped to snippet lines.
+ const rejected = await apply([{ type: 'set-material', id: 'slab', snippet: broken }]);
+ assert.equal(rejected.committed, true);
+ assert.equal(rejected.renderSucceeded, false);
+ assert.match(rejected.renderError, /Material snippet \(used by caster\) failed to compile/);
+ assert.match(rejected.renderError, /snippet line 3/);
+ assert.match(rejected.renderError, /vec3 wrong = 1\.0;/);
+
+ // The session is still usable: the fix draws, and null clears the hook so the shadow is back.
+ const fixed = await apply([{ type: 'set-material', id: 'slab', snippet: tint(0.5) }]);
+ assert.equal(fixed.renderSucceeded, true, fixed.renderError);
+ const cleared = await apply([{ type: 'set-material', id: 'slab', snippet: null }]);
+ assert.equal(cleared.renderSucceeded, true, cleared.renderError);
+ const restored = await measure(await look(sessionId));
+ assert.ok(Math.abs(restored.darkPixels - plain.darkPixels) < plain.darkPixels * 0.05, `clearing the hook should restore the shadow: ${plain.darkPixels} against ${restored.darkPixels}`);
+
+ await mkdir('artifacts', { recursive: true });
+ const proof = {
+ darkPixels: { plain: plain.darkPixels, discardedCaster: unshadowed.darkPixels, restored: restored.darkPixels },
+ redShare: { plain: plain.redShare, tinted: red.redShare },
+ compileLog: rejected.renderError, refusedBeforeCompile: true, sessionUsableAfterError: true,
+ };
+ await writeFile('artifacts/mcp-material-snippet-proof.json', JSON.stringify(proof, null, 2));
+ console.log(JSON.stringify(proof));
+} finally {
+ await client.close();
+ if (diagnostics.trim()) console.error(diagnostics.trim().split('\n').slice(-5).join('\n'));
+}
diff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.js
index 90da1272..91812311 100644
--- a/src/agent-scene-modules.js
+++ b/src/agent-scene-modules.js
@@ -32,6 +32,7 @@ const MODULE_LOADERS = {
viewTransform: () => import('./view-transform-shader.js'),
coplanar: () => import('./coplanar-overlap.js'),
userPass: () => import('./user-pass.js'),
+ materialSnippet: () => import('./material-snippet.js'),
};
const loadedModules = {};
@@ -72,6 +73,7 @@ const MODULE_MARKERS = {
// The coplanar-overlap warning in scene_inspect appears only once this is loaded, which loadAllAgentSceneModules does; a recipe with resolveCoplanar needs it.
coplanar: ['"resolveCoplanar"'],
userPass: ['"postPasses"'],
+ materialSnippet: ['"snippet":{'],
};
/** Whether a recipe, operation list or edit list contains a boolean edit, which needs loadMeshBooleans() first. */
diff --git a/src/agent-scene.js b/src/agent-scene.js
index cc67ea10..8aee8f42 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -24,7 +24,7 @@ const MAX_MESH_REPORTS = 16;
const TERRAIN_SKY_VISIBILITY_BYTES = 4;
const CAMERA_DEFAULTS = Object.freeze({ eye: [4, 3, 5], target: [0, 0, 0], up: [0, 1, 0], fov: 45, near: 0.1, far: 1000 });
const NODE_FIELDS = ['translation', 'rotation', 'scale', 'parent', 'label', 'geometry', 'material', 'skin', 'relations', 'asset', 'vegetation', 'decal', 'depthLayer'];
-const MATERIAL_FIELDS = ['color', 'metallic', 'roughness', 'emission', 'emissiveColor', 'opacity', 'alphaMode', 'doubleSided', 'pattern', 'map', 'subsurface', 'waterPreset', 'absorption', 'turbidity', ...MATERIAL_LAYER_FIELDS];
+const MATERIAL_FIELDS = ['color', 'metallic', 'roughness', 'emission', 'emissiveColor', 'opacity', 'alphaMode', 'doubleSided', 'pattern', 'map', 'subsurface', 'snippet', 'waterPreset', 'absorption', 'turbidity', ...MATERIAL_LAYER_FIELDS];
const PATTERN_TEXTURE_FIELDS = ['map', 'normalMap', 'metallicRoughnessMap', 'emissiveMap', 'boxMapScale', 'boxMapAxis'];
const CAMERA_FIELDS = Object.keys(CAMERA_DEFAULTS);
const LOOK_OPERATION_FIELDS = Object.freeze({
@@ -199,6 +199,7 @@ function copyMaterialLayers(value, result) {
}
validateMaterialLayers(result);
if (value.subsurface !== undefined) result.subsurface = agentSceneModule('subsurfaceShader').copySubsurface(value.subsurface, result.tag);
+ if (value.snippet) result.snippet = agentSceneModule('materialSnippet').normalizeMaterialSnippet(value.snippet, { hasMap: Boolean(value.map || value.pattern) });
if (result.anisotropy > 0 || result.iridescence > 0) agentSceneModule('layerShaders');
if (result.toon > 0) agentSceneModule('lineArt');
}
@@ -763,7 +764,7 @@ function geometryStats(records) {
function batchSignature(material) {
// Textures are fixed per renderer, so a changed pattern rebuilds it like a changed tag does.
// Transmission moves a material to the pass that sees the opaque scene, which is fixed per renderer.
- return `${material.tag}|${material.alphaMode}|${material.opacity}|${material.transmission > 0}|${material.doubleSided}|${JSON.stringify(material.pattern ?? material.map ?? null)}`;
+ return `${material.tag}|${material.alphaMode}|${material.opacity}|${material.transmission > 0}|${material.doubleSided}|${JSON.stringify(material.pattern ?? material.map ?? null)}|${JSON.stringify(material.snippet)}`;
}
function materialHandle(snapshot, sequence) {
diff --git a/src/material-snippet.js b/src/material-snippet.js
new file mode 100644
index 00000000..0ca2235a
--- /dev/null
+++ b/src/material-snippet.js
@@ -0,0 +1,232 @@
+import { registerShaderFeatureCode, layerTextureSupport } from './shader-registry.js';
+import {
+ SNIPPET_LIMITS, requireKnownFields, validateShaderSnippet, validateSnippetUniforms, snippetUniformDeclarations, mapShaderInfoLog,
+} from './shader-snippet-validator.js';
+
+// Material fragment hook: a bounded GLSL function that changes colour and alpha, or discards, for the surfaces of one
+// material. The core renderer carries three small seams for it (a feature per distinct snippet, the `surfaceHook` slot of
+// the main fragment shader, and the `depthVertex` and `depthFragment` rewrites of the shadow shader). Everything else lives
+// here, in a lazy chunk. The main pass runs the hook before the render-pass output, so the ID pass sees the discard, and the
+// shadow pass runs the same hook, so a discarded fragment casts no shadow.
+
+/** Names the engine provides to a snippet; a uniform cannot take one. */
+export const MATERIAL_SNIPPET_INPUTS = Object.freeze(['materialHook', 'snippetMask']);
+const SNIPPET_FIELDS = ['fragment', 'uniforms', 'mask', 'feature'];
+const ENTRY_FUNCTION = Object.freeze({
+ pattern: /\bvoid\s+materialHook\s*\(\s*inout\s+vec3\s+[A-Za-z_]\w*\s*,\s*inout\s+float\s+[A-Za-z_]\w*\s*,\s*vec2\s+[A-Za-z_]\w*\s*,\s*vec3\s+[A-Za-z_]\w*\s*\)\s*\{/,
+ example: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) { }',
+});
+
+/** What scene_capabilities reports under materialSnippet; it lives here so the scene entry does not carry the text. */
+export const MATERIAL_SNIPPET_CAPABILITIES = Object.freeze({
+ fields: 'material.snippet { fragment (defines void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition)), uniforms, mask: "map" }; null clears it',
+ inputs: 'color (albedo after the map), alpha (map alpha), uv, worldPosition; snippetMask (the material map, with mask "map"); discard',
+ uniforms: 'as postPasses.uniforms',
+ rules: 'as postPasses.rules',
+ limits: Object.freeze({ maxBytes: SNIPPET_LIMITS.maxBytes, maxUniforms: SNIPPET_LIMITS.maxUniforms }),
+ notes: 'Runs in the colour, shadow (a discarded fragment casts no shadow; color starts white there) and ID passes. alpha cuts only with alphaMode "mask"; discard always. Rasterizer only. A compile error returns renderError with snippet line numbers. docs/user-passes.md',
+});
+
+/**
+ * Checks a material's snippet and returns it normalized and frozen: uniforms as { name: { type, value } }, plus the `feature`
+ * the renderer compiles it under. `hasMap` says whether the material has a map or a pattern for `mask` to read.
+ */
+export function normalizeMaterialSnippet(input, { hasMap = false } = {}) {
+ requireKnownFields(input, SNIPPET_FIELDS, 'snippet');
+ const source = validateShaderSnippet(input.fragment, { label: 'Material snippet fragment', requiredFunction: ENTRY_FUNCTION }).source;
+ const uniforms = validateSnippetUniforms(input.uniforms, { label: 'Material snippet', reservedNames: MATERIAL_SNIPPET_INPUTS });
+ if (input.mask !== undefined && input.mask !== 'map') {
+ throw new RangeError('Material snippet mask must be "map". Fix: the mask is the material\'s own map or pattern; load the image as the material map.');
+ }
+ if (input.mask !== undefined && !hasMap) {
+ throw new RangeError('Material snippet mask needs a map. Fix: give the material a map or a pattern, or remove mask.');
+ }
+ const mask = input.mask === 'map';
+ if (!mask && /(?<![.\w])snippetMask(?!\w)/.test(source)) {
+ throw new RangeError('Material snippet fragment uses snippetMask without a mask. Fix: add "mask": "map" to the snippet and give the material a map.');
+ }
+ // The recipe keeps a normalized snippet as it is, so it normalizes again: `feature` is derived, and an incoming one is replaced.
+ const snippet = Object.freeze({ feature: featureName(source, uniforms, mask), fragment: source, uniforms: Object.freeze(uniforms), ...(mask ? { mask: 'map' } : {}) });
+ registerSnippetFeature(snippet);
+ return snippet;
+}
+
+// FNV-1a over the code and the declared uniform types: two materials with the same code share one compiled function.
+function featureName(fragment, uniforms, mask) {
+ let hash = 0x811c9dc5;
+ for (const character of `${fragment}\n${snippetUniformDeclarations(uniforms)}\n${mask}`) {
+ hash = Math.imul(hash ^ character.codePointAt(0), 0x01000193) >>> 0;
+ }
+ return `s${hash.toString(16).padStart(8, '0')}`;
+}
+
+// Names declared at the top of the snippet (functions and constants), found by reading the text outside braces and parentheses.
+function topLevelNames(code) {
+ const outside = [];
+ let depth = 0;
+ for (const character of code) {
+ if (character === '{') {
+ depth++;
+ outside.push('\n');
+ } else if (character === '}') {
+ depth--;
+ } else if (depth === 0) {
+ outside.push(character);
+ }
+ }
+ const text = outside.join('').replace(/\([^()]*\)/g, ' ').replace(/\([^()]*\)/g, ' ');
+ const declaration = /(?:^|[;\s])(?:const\s+)?(?:float|int|bool|void|[ibu]?vec[234]|mat[234])(?:\[\d*\])?\s+([A-Za-z_]\w*)(?:\s*\[\d*\])?\s*(?=[=;,]|\s*$|\n)/gm;
+ return [...text.matchAll(declaration)].map(match => match[1]);
+}
+
+function replaceName(code, name, replacement) {
+ return code.replace(new RegExp(`(?<![.\\w])${name}(?!\\w)`, 'g'), () => replacement);
+}
+
+// The snippet's functions, constants and uniforms get the feature's prefix, so two materials can use the same names.
+function prefixedCode(snippet) {
+ const prefix = `u67_${snippet.feature}_`;
+ let code = snippet.fragment;
+ const names = new Set([...topLevelNames(code), ...Object.keys(snippet.uniforms)]);
+ for (const name of names) code = replaceName(code, name, `${prefix}${name}`);
+ const declarations = snippetUniformDeclarations(Object.fromEntries(Object.entries(snippet.uniforms).map(([name, uniform]) => [`${prefix}${name}`, uniform])));
+ return { code, declarations, hook: `${prefix}materialHook` };
+}
+
+const DEPTH_FRAGMENT_HEAD = 'void main() {\n if (alphaMasked < 0.5) return;\n float alpha = materialOpacity;\n if (hasBaseMap > 0.5) alpha *= texture(baseMap, textureUV).a;\n';
+const DEPTH_CODE_MARK = '// u67 material hook code\n';
+const DEPTH_CALL_MARK = ' // u67 material hook calls\n';
+// Same inputs as the colour pass: white colour, the map alpha, the uv and the world position the vertex shader passes on.
+const DEPTH_FRAGMENT_BODY = `void main() {
+ vec3 hookColor = vec3(1.0);
+ float hookAlpha = hasBaseMap > 0.5 ? texture(baseMap, textureUV).a : 1.0;
+${DEPTH_CALL_MARK} if (alphaMasked < 0.5) return;
+ float alpha = materialOpacity * hookAlpha;
+`;
+
+function insertAfter(text, marker, addition) {
+ return text.replace(marker, () => marker + addition);
+}
+
+// The shadow vertex shader passes the world position on, which the hook takes as its fourth input.
+function patchDepthVertex(source) {
+ if (source.includes('snippetWorld')) return source;
+ return source
+ .replace('out vec2 textureUV;', () => 'out vec2 textureUV;\nout vec3 snippetWorld;')
+ .replace('textureUV = uv;', () => 'textureUV = uv;\n snippetWorld = world.xyz;');
+}
+
+function registerSnippetFeature(snippet) {
+ const { code, declarations, hook } = prefixedCode(snippet);
+ const flag = `u67_${snippet.feature}`;
+ const mainCode = replaceName(`uniform float ${flag};\n${declarations}\n${code}\n`, 'snippetMask', 'baseMap');
+ const call = `if (${flag} > 0.5) ${hook}(hookColor, hookAlpha, textureUV, snippetWorld);\n`;
+ registerShaderFeatureCode(snippet.feature, Object.freeze({
+ libraries: mainCode,
+ surfaceHook: ` if (${flag} > 0.5) ${hook}(color, textureAlpha, textureUV, worldPosition);\n`,
+ depth: (source) => {
+ if (source.includes('lightProjection * world')) return patchDepthVertex(source);
+ let text = source;
+ if (!text.includes(DEPTH_CALL_MARK)) {
+ text = text.replace('in vec2 textureUV;', () => 'in vec2 textureUV;\nin vec3 snippetWorld;')
+ .replace(DEPTH_FRAGMENT_HEAD, () => `${DEPTH_CODE_MARK}${DEPTH_FRAGMENT_BODY}`);
+ }
+ return insertAfter(insertAfter(text, DEPTH_CODE_MARK, mainCode), DEPTH_CALL_MARK, ` ${call}`);
+ },
+ }));
+ registered.set(snippet.feature, { flag, snippet });
+}
+
+const registered = new Map();
+
+function setUniformValue(context, location, { type, value }) {
+ if (location === null) return;
+ if (type === 'float') context.uniform1f(location, value);
+ else if (type === 'int') context.uniform1i(location, value);
+ else if (type === 'vec2') context.uniform2fv(location, value);
+ else if (type === 'vec3') context.uniform3fv(location, value);
+ else context.uniform4fv(location, value);
+}
+
+// Sets the on/off flag of every snippet this renderer compiled, and the uniforms of the one the batch uses.
+function bindSnippets(renderer, program, locations, batch) {
+ const context = renderer.context;
+ const active = batch.parts[0].material.snippet?.feature;
+ for (const feature of renderer.shaderFeatures) {
+ const entry = registered.get(feature);
+ if (!entry) continue;
+ const locationOf = (name) => {
+ locations[name] ??= context.getUniformLocation(program, name);
+ return locations[name];
+ };
+ context.uniform1f(locationOf(entry.flag), feature === active ? 1 : 0);
+ if (feature !== active) continue;
+ for (const [name, uniform] of Object.entries(entry.snippet.uniforms)) {
+ setUniformValue(context, locationOf(`u67_${feature}_${name}`), uniform);
+ }
+ }
+}
+
+// Other optional features also bind per batch, so this keeps whatever was registered first.
+const previousBind = layerTextureSupport.bind;
+layerTextureSupport.bind = (renderer, mapState, layerState, batch) => {
+ previousBind?.(renderer, mapState, layerState, batch);
+ bindSnippets(renderer, renderer.program, renderer.uniforms, batch);
+};
+
+// The shadow pass: the same flags and uniforms on the depth program, and the map on unit 1 when the snippet reads it.
+layerTextureSupport.bindDepth = (renderer, batch) => {
+ renderer.depthSnippetUniforms ??= {};
+ bindSnippets(renderer, renderer.depthProgram, renderer.depthSnippetUniforms, batch);
+ if (batch.parts[0].material.snippet?.mask) {
+ renderer.context.activeTexture(renderer.context.TEXTURE0 + 1);
+ renderer.context.bindTexture(renderer.context.TEXTURE_2D, batch.textures[0]);
+ }
+};
+
+const PROBE_HEADER = `#version 300 es
+precision highp float;
+precision highp int;
+uniform sampler2D snippetMask;
+out vec4 u67_output;
+`;
+const PROBE_FOOTER = '\nvoid main() { vec3 color = vec3(0.5); float alpha = 1.0; materialHook(color, alpha, vec2(0.5), vec3(0.0)); u67_output = vec4(color, alpha); }\n';
+
+function newProbeContext() {
+ const canvas = typeof OffscreenCanvas === 'function' ? new OffscreenCanvas(1, 1) : document.createElement('canvas');
+ return canvas.getContext('webgl2');
+}
+
+/**
+ * Compiles each distinct snippet of the parts alone, before the renderer builds its programs, so a compile error names the
+ * snippet's own line and the running renderer stays untouched. Throws an Error that carries the mapped log.
+ */
+export function probeMaterialSnippets(parts, createContext = newProbeContext) {
+ const snippets = new Map();
+ for (const part of parts) {
+ const { snippet } = part.material;
+ if (snippet && !snippets.has(snippet.feature)) snippets.set(snippet.feature, { snippet, parts: [] });
+ if (snippet) snippets.get(snippet.feature).parts.push(part.name);
+ }
+ if (snippets.size === 0) return;
+ const context = createContext();
+ if (!context) return;
+ try {
+ for (const { snippet, parts: users } of snippets.values()) {
+ const header = `${PROBE_HEADER}${snippetUniformDeclarations(snippet.uniforms)}\n`;
+ const shader = context.createShader(context.FRAGMENT_SHADER);
+ context.shaderSource(shader, `${header}${snippet.fragment}${PROBE_FOOTER}`);
+ context.compileShader(shader);
+ const compiled = context.getShaderParameter(shader, context.COMPILE_STATUS);
+ const log = compiled ? '' : context.getShaderInfoLog(shader);
+ context.deleteShader(shader);
+ if (!compiled) {
+ const mapped = mapShaderInfoLog(log, { headerLines: header.split('\n').length - 1, snippet: snippet.fragment });
+ const names = users.slice(0, 3).join(', ');
+ throw new Error(`Material snippet (used by ${names}${users.length > 3 ? ', ...' : ''}) failed to compile as GLSL ES 3.00. Line numbers refer to your fragment snippet.\n${mapped}`);
+ }
+ }
+ } finally {
+ context.getExtension('WEBGL_lose_context')?.loseContext();
+ }
+}
diff --git a/src/renderer.js b/src/renderer.js
index 797c3c3a..2e7f624b 100644
--- a/src/renderer.js
+++ b/src/renderer.js
@@ -1,6 +1,6 @@
import { materialTags } from './shader-registry.js';
import { mat4, mat3 } from 'gl-matrix';
-import { SPLAT_UNITS, requireSplatSupport, layerTextureSupport, LIGHT_SHADING_FEATURE, SHADING_MODES, LIGHT_SHADING_MAXIMUM_SAMPLES, createVertexShader, vertexShader, liquidFragmentSource, createFragmentShader, liquidPassDrawer, depthFragmentShader, benchmarkVertexShader, benchmarkFragmentShader, viewTransformShader, createDepthVertexShader } from './shaders.js';
+import { SPLAT_UNITS, requireSplatSupport, layerTextureSupport, LIGHT_SHADING_FEATURE, SHADING_MODES, LIGHT_SHADING_MAXIMUM_SAMPLES, createVertexShader, vertexShader, liquidFragmentSource, createFragmentShader, liquidPassDrawer, depthFragmentShader, withDepthCode, benchmarkVertexShader, benchmarkFragmentShader, viewTransformShader, createDepthVertexShader } from './shaders.js';
import { postFragment, createPostFragment, POST_STEPS, skyFragment, bloomFragment } from './post.js';
import { fullscreenVertex, startProgram, finishProgram, compileProgram } from './gl-program.js';
import { ambientOcclusionFragment } from './ambient-occlusion.js';
@@ -89,6 +89,7 @@ function shaderFeaturesOfParts(parts) {
if (material.lightMask !== undefined) features.add('lightLinking');
if ((material.alphaMode ?? 'opaque') !== 'opaque') features.add('alphaMode');
if (material.subsurface?.weight > 0) features.add('subsurface');
+ if (material.snippet) features.add(material.snippet.feature);
if (materialLayerValue(material, 'clearcoat') > 0) features.add('clearcoat');
if (materialLayerValue(material, 'sheenColor').some(channel => channel !== 0)) features.add('sheen');
if (TAG_SHADER_FEATURES.has(material.tag)) features.add(material.tag);
@@ -339,7 +340,7 @@ export function groupParts(parts, batching = true) {
const key = batching ? part.geometry : part;
if (!groups.has(key)) groups.set(key, new Map());
let maps = groups.get(key);
- for (const value of [...materialMapState(part.material), ...materialLayerState(part.material), part.skin ?? null, part.crowd ?? null, part.morph ?? null, castsShadow(part), depthLayerOf(part), renderPass(part.material), part.lods ?? null, part.shadowLods ?? null, ...materialAlphaState(part.material)]) {
+ for (const value of [...materialMapState(part.material), ...materialLayerState(part.material), part.skin ?? null, part.material.snippet, part.crowd ?? null, part.morph ?? null, castsShadow(part), depthLayerOf(part), renderPass(part.material), part.lods ?? null, part.shadowLods ?? null, ...materialAlphaState(part.material)]) {
if (!maps.has(value)) maps.set(value, new Map());
maps = maps.get(value);
}
@@ -602,7 +603,7 @@ export class Renderer67 {
if (shadowCascades) this.shaderFeatures.add(shadowCascades.shaderFeature);
const pending = {
main: this.startMainProgram(),
- depth: startProgram(this.context, createDepthVertexShader(this.shaderFeatures), depthFragmentShader),
+ depth: startProgram(this.context, createDepthVertexShader(this.shaderFeatures), withDepthCode(depthFragmentShader, this.shaderFeatures)),
};
if (!benchmark) {
pending.post = startProgram(this.context, fullscreenVertex, this.withViewTransform(postFragment, 'post'));
@@ -1878,6 +1879,7 @@ export class Renderer67 {
context.uniform1f(uniforms.alphaCutoff, batch.alphaCutoff);
context.uniform1f(uniforms.materialOpacity, batch.opacity);
context.uniform1f(uniforms.hasBaseMap, batch.mapState[0] ? 1 : 0);
+ layerTextureSupport.bindDepth?.(this, batch);
if (masked) {
context.activeTexture(context.TEXTURE0 + 1);
context.bindTexture(context.TEXTURE_2D, batch.textures[0]);
@@ -2027,7 +2029,7 @@ export class Renderer67 {
this.context.uniform1f(this.uniforms.boxMapScale, batch.mapState[12]);
this.context.uniform1i(this.uniforms.boxMapAxis, batch.mapState[13]);
if (this.shaderFeatures?.has('splat')) requireSplatSupport().bind(this, batch.splat);
- layerTextureSupport.bind?.(this, batch.mapState, layers);
+ layerTextureSupport.bind?.(this, batch.mapState, layers, batch);
this.context.uniform1f(this.uniforms.hasLightMap, batch.lightMapTexture ? 1 : 0);
if (batch.lightMapTexture) {
this.context.activeTexture(this.context.TEXTURE0 + LIGHT_MAP_UNIT);
diff --git a/src/shaders.js b/src/shaders.js
index b7bf4d74..3a8f5c2a 100644
--- a/src/shaders.js
+++ b/src/shaders.js
@@ -122,8 +122,13 @@ void main() {
}
// Built when a renderer starts, so the crowd addon can install its skinning code first.
+// A feature may rewrite the shadow shaders (the material snippet hook discards there too).
+export function withDepthCode(source, features) {
+ return [...features].reduce((text, feature) => registeredFeatureCode.get(feature)?.depth?.(text) ?? text, source);
+}
+
export function createDepthVertexShader(features = []) {
- return `#version 300 es
+ return withDepthCode(`#version 300 es
${vertexDefinitions(new Set(features))}
precision highp float;
layout(location=0) in vec3 position;
@@ -149,7 +154,7 @@ void main() {
world.xyz += windOffset(position, world.xyz, uv, instanceSurface.w, time, wind);
textureUV = uv;
gl_Position = lightProjection * world;
-}`;
+}`, features);
}
export const depthVertexShader = createDepthVertexShader();
@@ -543,7 +548,7 @@ void main() {
color *= sampledBase.rgb;
textureAlpha = sampledBase.a;
}
- float alpha = surfaceAlpha(textureAlpha);
+${featureCode("surfaceHook")} float alpha = surfaceAlpha(textureAlpha);
vec3 emission = emissionColor * surface.z;
if (USES_EMISSIVE_MAP) emission *= (BOX_MAPPED ? sampleBox(emissiveMap) : texture(emissiveMap, mapUV(hasEmissiveMap))).rgb;
if (USES_RENDER_PASS) {
diff --git a/tests/material-snippet.test.js b/tests/material-snippet.test.js
new file mode 100644
index 00000000..71b5fce3
--- /dev/null
+++ b/tests/material-snippet.test.js
@@ -0,0 +1,150 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { normalizeMaterialSnippet, probeMaterialSnippets, MATERIAL_SNIPPET_CAPABILITIES } from '../src/material-snippet.js';
+import { createFragmentShader, createVertexShader, createDepthVertexShader, withDepthCode, depthFragmentShader } from '../src/shaders.js';
+import { groupParts } from '../src/renderer.js';
+import { createAgentScene, loadAgentSceneModules } from '../src/agent-scene.js';
+
+const DISSOLVE = `float hash(vec2 p) { return fract(sin(dot(p, vec2(12.9898, 78.233))) * 43758.5453); }
+void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {
+ if (hash(floor(worldPosition.xz * 8.0)) < threshold) discard;
+ color *= tint;
+}`;
+const dissolve = (overrides = {}) => ({ fragment: DISSOLVE, uniforms: { threshold: 0.4, tint: [1, 0.8, 0.6] }, ...overrides });
+
+function refuses(snippet, pattern, options) {
+ assert.throws(() => normalizeMaterialSnippet(snippet, options), error => {
+ assert.match(error.message, pattern);
+ assert.match(error.message, /Fix: /);
+ return true;
+ });
+}
+
+test('a valid snippet is normalized, frozen and registered as a shader feature', () => {
+ const snippet = normalizeMaterialSnippet(dissolve());
+ assert.match(snippet.feature, /^s[0-9a-f]{8}$/);
+ assert.deepEqual(snippet.uniforms, { threshold: { type: 'float', value: 0.4 }, tint: { type: 'vec3', value: [1, 0.8, 0.6] } });
+ assert.ok(Object.isFrozen(snippet));
+ assert.equal(normalizeMaterialSnippet(dissolve()).feature, snippet.feature);
+ assert.notEqual(normalizeMaterialSnippet(dissolve({ fragment: `${DISSOLVE}\n` })).feature, snippet.feature);
+});
+
+test('the snippet follows the part 1 validation rules and names a fix', () => {
+ refuses(dissolve({ fragment: `#define X 1\n${DISSOLVE}` }), /line 1, column 1.*Preprocessor/s);
+ refuses(dissolve({ fragment: DISSOLVE.replace('discard;', 'discard; while (true) { }') }), /"while" loops are not allowed/);
+ refuses(dissolve({ fragment: DISSOLVE.replace('threshold', 'gl_Position') }), /gl_/);
+ refuses(dissolve({ fragment: DISSOLVE.replace('color *= tint', 'color *= u67_secret') }), /u67_/);
+ refuses(dissolve({ fragment: 'void materialHook(inout vec3 color) { }' }), /entry function is missing.*materialHook/s);
+ refuses(dissolve({ uniforms: { materialHook: 1 } }), /keyword or an input the engine provides/);
+ refuses(dissolve({ uniforms: { threshold: Infinity } }), /finite/);
+ refuses(dissolve({ uniforms: Object.fromEntries(Array.from({ length: 17 }, (_, index) => [`value${index}`, 1])) }), /17 uniforms; the limit is 16/);
+ refuses(dissolve({ fragment: 'x'.repeat(5000) }), /4096|bytes/i);
+ assert.throws(() => normalizeMaterialSnippet({ ...dissolve(), extra: 1 }), /Unknown snippet field: extra\. Allowed fields: fragment, uniforms, mask/);
+});
+
+test('mask reads the material map and needs one', () => {
+ const masked = { fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) { if (texture(snippetMask, uv).r < 0.5) discard; }', mask: 'map' };
+ refuses(masked, /mask needs a map/, { hasMap: false });
+ refuses({ ...masked, mask: 'remote.png' }, /mask must be "map"/, { hasMap: true });
+ refuses({ fragment: masked.fragment }, /uses snippetMask without a mask/, { hasMap: true });
+ assert.equal(normalizeMaterialSnippet(masked, { hasMap: true }).mask, 'map');
+});
+
+test('the main shader runs the hook before the render-pass output, with prefixed names', () => {
+ const snippet = normalizeMaterialSnippet(dissolve());
+ const source = createFragmentShader(['studioLighting', snippet.feature]);
+ const prefix = `u67_${snippet.feature}`;
+ assert.match(source, new RegExp(`uniform float ${prefix};`));
+ assert.match(source, new RegExp(`uniform float ${prefix}_threshold;`));
+ assert.match(source, new RegExp(`float ${prefix}_hash\\(vec2 p\\)`));
+ assert.match(source, new RegExp(`void ${prefix}_materialHook\\(inout vec3 color`));
+ const call = source.indexOf(`${prefix}_materialHook(color, textureAlpha, textureUV, worldPosition)`);
+ assert.ok(call > 0);
+ assert.ok(call < source.indexOf('float alpha = surfaceAlpha(textureAlpha)'), 'before the cutoff, so alpha can cut');
+ assert.ok(call < source.indexOf('if (USES_RENDER_PASS)'), 'before the ID pass output');
+ assert.doesNotMatch(createFragmentShader(['studioLighting']), /u67_s[0-9a-f]{8}/);
+});
+
+test('two materials with the same helper names share one shader without clashing', () => {
+ const first = normalizeMaterialSnippet(dissolve());
+ const second = normalizeMaterialSnippet(dissolve({ uniforms: { threshold: 0.4, tint: [1, 1, 1], extra: 2 } }));
+ assert.notEqual(first.feature, second.feature);
+ const source = createFragmentShader(['studioLighting', first.feature, second.feature]);
+ assert.match(source, new RegExp(`float u67_${first.feature}_hash`));
+ assert.match(source, new RegExp(`float u67_${second.feature}_hash`));
+});
+
+test('the shadow shaders run the same hook so a discarded fragment casts no shadow', () => {
+ const snippet = normalizeMaterialSnippet(dissolve());
+ const features = new Set(['studioLighting', snippet.feature]);
+ const vertex = createDepthVertexShader(features);
+ assert.match(vertex, /out vec3 snippetWorld;/);
+ assert.match(vertex, /snippetWorld = world\.xyz;/);
+ const fragment = withDepthCode(depthFragmentShader, features);
+ assert.match(fragment, /in vec3 snippetWorld;/);
+ assert.match(fragment, new RegExp(`u67_${snippet.feature}_materialHook\\(hookColor, hookAlpha, textureUV, snippetWorld\\)`));
+ assert.ok(fragment.indexOf('materialHook(hookColor') < fragment.indexOf('if (alphaMasked < 0.5) return;'), 'the hook runs for opaque casters too');
+ assert.match(fragment, /float alpha = materialOpacity \* hookAlpha;/);
+ const other = normalizeMaterialSnippet(dissolve({ uniforms: { threshold: 0.1, tint: [1, 1, 1], extra: 2 } }));
+ const both = withDepthCode(depthFragmentShader, new Set([snippet.feature, other.feature]));
+ assert.equal(both.split('in vec3 snippetWorld;').length, 2, 'declared once');
+ assert.equal(both.split('(hookColor, hookAlpha').length, 3, 'one call per snippet');
+ assert.equal(withDepthCode(depthFragmentShader, new Set(['studioLighting'])), depthFragmentShader);
+ assert.equal(createDepthVertexShader(['studioLighting']), createDepthVertexShader([]));
+});
+
+test('materials with different snippets land in different batches, and a stored snippet normalizes again', async () => {
+ const snippet = normalizeMaterialSnippet(dissolve());
+ const recipe = {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'box', type: 'box' }],
+ materials: [{ id: 'hooked', snippet: dissolve() }, { id: 'plain' }],
+ nodes: ['a', 'b', 'c'].map((id, index) => ({ id, geometry: 'box', material: id === 'b' ? 'plain' : 'hooked', translation: [index * 2, 0, 0] })),
+ };
+ await loadAgentSceneModules(recipe);
+ const batches = groupParts(createAgentScene(recipe).parts);
+ assert.deepEqual(batches.map(batch => batch.map(entry => entry.name)), [['a', 'c'], ['b']]);
+ assert.equal(normalizeMaterialSnippet(JSON.parse(JSON.stringify(snippet))).feature, snippet.feature);
+ assert.equal(normalizeMaterialSnippet(structuredClone(snippet)).feature, snippet.feature);
+});
+
+test('probing a snippet maps the driver log to snippet lines and names the parts', () => {
+ const snippet = normalizeMaterialSnippet({ fragment: 'void materialHook(inout vec3 color, inout float alpha, vec2 uv, vec3 worldPosition) {\n vec3 wrong = 1.0;\n}' });
+ const header = 6;
+ const context = {
+ FRAGMENT_SHADER: 1, COMPILE_STATUS: 2,
+ createShader: () => ({}), shaderSource() {}, compileShader() {}, deleteShader() {},
+ getShaderParameter: () => false,
+ getShaderInfoLog: () => `ERROR: 0:${header + 2}: '=' : cannot convert from 'const float' to 'highp 3-component vector of float'`,
+ getExtension: () => null,
+ };
+ const parts = [{ name: 'wall', material: { snippet } }, { name: 'floor', material: {} }];
+ assert.throws(() => probeMaterialSnippets(parts, () => context), error => {
+ assert.match(error.message, /Material snippet \(used by wall\) failed to compile as GLSL ES 3\.00/);
+ assert.match(error.message, /snippet line 2: '=' : cannot convert/);
+ assert.match(error.message, /vec3 wrong = 1\.0;/);
+ return true;
+ });
+ assert.doesNotThrow(() => probeMaterialSnippets([{ name: 'floor', material: {} }], () => { throw new Error('no context needed'); }));
+});
+
+test('a recipe sets, edits and clears the snippet and scene_capabilities reports the limits', async () => {
+ const recipe = {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'box', type: 'box' }],
+ materials: [{ id: 'stone', color: [0.5, 0.5, 0.5], snippet: dissolve() }, { id: 'plain', color: [0.2, 0.2, 0.2] }],
+ nodes: [{ id: 'a', geometry: 'box', material: 'stone' }, { id: 'b', geometry: 'box', material: 'plain', translation: [2, 0, 0] }],
+ };
+ await loadAgentSceneModules(recipe);
+ const scene = createAgentScene(recipe);
+ const hooked = scene.parts.find(part => part.name === 'a').material;
+ assert.match(hooked.snippet.feature, /^s[0-9a-f]{8}$/);
+ assert.equal(scene.parts.find(part => part.name === 'b').material.snippet, undefined);
+ scene.apply([{ type: 'set-material', id: 'stone', snippet: dissolve({ uniforms: { threshold: 0.7, tint: [1, 1, 1] } }) }], { expectedRevision: scene.revision });
+ assert.equal(scene.parts.find(part => part.name === 'a').material.snippet.uniforms.threshold.value, 0.7);
+ assert.throws(() => scene.apply([{ type: 'set-material', id: 'stone', snippet: dissolve({ fragment: '#define A 1\n' + DISSOLVE }) }], { expectedRevision: scene.revision }), /Preprocessor lines/);
+ assert.equal(scene.parts.find(part => part.name === 'a').material.snippet.uniforms.threshold.value, 0.7, 'a refused edit changes nothing');
+ scene.apply([{ type: 'set-material', id: 'stone', snippet: null }], { expectedRevision: scene.revision });
+ assert.equal(scene.parts.find(part => part.name === 'a').material.snippet, undefined);
+ assert.equal(MATERIAL_SNIPPET_CAPABILITIES.limits.maxUniforms, 16);
+});