Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add shadowOnly nodes that cast a shadow but stay out of the picture (#926)

Nguyen Xuan Anadd9b48013c9
diff --git a/changes/926-shadow-only-nodes.md b/changes/926-shadow-only-nodes.md
new file mode 100644
index 00000000..b17fc4e7
--- /dev/null
+++ b/changes/926-shadow-only-nodes.md
@@ -0,0 +1 @@
+- A node takes `shadowOnly: true` to cast a shadow while staying out of the picture, for a gobo such as a window frame outside the shot (#926)
diff --git a/docs/agent-scene.md b/docs/agent-scene.md
index 84f1e011..472e5b6a 100644
--- a/docs/agent-scene.md
+++ b/docs/agent-scene.md
@@ -345,7 +345,9 @@ Five node fields change how a node renders; `add-node` and `set-node` take them,
- `castShadow: false` and `receiveShadow: false` (mesh nodes) stop a mesh casting or receiving shadows. Both default to true; glass casts none unless `castShadow` is set.
- `holdout: true` writes zero colour and zero alpha where the mesh covers, and still hides what is behind it and still casts its shadow. `shadowCatcher: true` draws only the shadow it receives, black with the shadow as premultiplied alpha, and casts none. Both are for compositing over footage and need a transparent canvas: `new Renderer67(canvas, parts, { transparent: true })`. A node is never both. The shadow catcher shows the sun shadow only.
-Parts with different `castShadow`, `receiveShadow`, `holdout` or `shadowCatcher` values batch apart; `visible` never splits a batch. `receiveShadow`, `holdout` and `shadowCatcher` changes need `replaceParts`, not `updateParts`.
+- `shadowOnly: true` (mesh nodes) keeps the node in the shadow passes and leaves it out of the picture, the same as Blender's Ray Visibility > Camera off. Use it for a gobo, such as a window frame outside the shot that throws bars of sun onto a wall. It is also left out of depth, object ID and wireframe passes. It defaults to false, so no scene changes. A node cannot set `shadowOnly` together with `castShadow: false`, `holdout` or `shadowCatcher`. The raster renderer honours it; the Cook path tracer does not yet, and traces the node as visible geometry.
+
+Parts with different `castShadow`, `receiveShadow`, `holdout`, `shadowCatcher` or `shadowOnly` values batch apart; `visible` never splits a batch. `receiveShadow`, `holdout`, `shadowCatcher` and `shadowOnly` changes need `replaceParts`, not `updateParts`.
### Relations between parts
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index 73968822..a7e744de 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -281,7 +281,7 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
choose: 'agx for lit scenes and Blender comparisons, neutral for product and brand colour, aces for contrast; needs loadAgentSceneModules',
}),
// Flush surfaces flicker (z-fighting). They are ranked at draw time by default; coplanar-overlap warns about what is left.
- renderFlags: Object.freeze({ nodeFields: Object.freeze(['visible', 'castShadow', 'receiveShadow', 'holdout', 'shadowCatcher']), motion: 'node track property visible: 1 or 0, STEP', note: 'visible:false hides a node and children; holdout and shadowCatcher need a transparent canvas (Renderer67 transparent:true); flags split batches. Blended and transmissive parts cast a sun shadow weighted by opacity and tinted by colour (1024 map, one extra draw of those parts only); castShadow:false turns it off, and cascades or splat/3-layer-map receivers skip the tint.' }),
+ renderFlags: Object.freeze({ nodeFields: Object.freeze(['visible', 'castShadow', 'receiveShadow', 'holdout', 'shadowCatcher', 'shadowOnly']), motion: 'node track property visible: 1 or 0, STEP', note: 'visible:false hides a node and children; holdout and shadowCatcher need a transparent canvas (Renderer67 transparent:true); flags split batches. Blended and transmissive parts cast a sun shadow weighted by opacity and tinted by colour (1024 map, one extra draw of those parts only); shadowOnly:true keeps a node in the shadow passes and out of the picture (a gobo, such as a window frame outside the shot); castShadow:false turns it off, and cascades or splat/3-layer-map receivers skip the tint.' }),
depthBias: Object.freeze({ nodeFields: Object.freeze(['decal', 'depthLayer']), maxDepthLayer: 4, environment: 'environment.resolveCoplanar', modes: Object.freeze(['offset', false]), default: 'offset', warning: 'coplanar-overlap', note: 'Automatic: flush or near-flush faces, inside one mesh or across parts, are ranked per triangle at draw time, the front face wins, geometry never moves; no field needed. resolveCoplanar false opts out. decal wins over the surface below; depthLayer stacks decals. scene_inspect flush.ranked counts ranked parts. A coplanar-overlap warning carries reason rank-overflow (over 15 stacked faces: remove duplicates), double-sided-gap, unchecked or collapsed-node.' }),
proceduralMotion: true,
motionIk: Object.freeze(['two-bone', 'aim', 'chain', 'spline']),
diff --git a/src/agent-scene.js b/src/agent-scene.js
index 5ebef196..bf5b158b 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -41,7 +41,7 @@ const PENDING_CUTTER_SCOPE = Object.freeze({ definitions: null, chain: Object.fr
const TERRAIN_SKY_VISIBILITY_BYTES = 4;
const CAMERA_DEFAULTS = Object.freeze({ eye: [4, 3, 5], target: [0, 0, 0], up: [0, 1, 0], fov: 45, near: 0.1, far: 1000 });
// Per-node render flags: `visible` hides a node and its children; the rest become renderer part flags of a mesh node.
-const RENDER_FLAG_FIELDS = ['visible', 'castShadow', 'receiveShadow', 'holdout', 'shadowCatcher'];
+const RENDER_FLAG_FIELDS = ['visible', 'castShadow', 'receiveShadow', 'holdout', 'shadowCatcher', 'shadowOnly'];
const PART_FLAG_FIELDS = RENDER_FLAG_FIELDS.slice(1);
const NODE_FIELDS = ['translation', 'rotation', 'scale', 'parent', 'label', 'geometry', 'material', 'skin', 'relations', 'asset', 'vegetation', 'decal', 'depthLayer', 'blend', ...RENDER_FLAG_FIELDS];
const MATERIAL_FIELDS = ['color', 'metallic', 'roughness', 'emission', 'emissiveColor', 'opacity', 'alphaMode', 'doubleSided', 'pattern', 'map', 'subsurface', 'snippet', 'waterPreset', 'absorption', 'turbidity', 'aov', 'wear', ...MATERIAL_LAYER_FIELDS];
@@ -343,6 +343,9 @@ function nodeFrom(value) {
if (flag !== 'visible' && value[flag] !== undefined && value.geometry === undefined) throw invalidField(flag, value[flag], 'a flag on a mesh node with geometry and material; remove it from group nodes', TypeError);
}
if (value.holdout === true && value.shadowCatcher === true) throw invalidField('shadowCatcher', value.shadowCatcher, 'false when holdout is true; a node is a holdout or a shadow catcher, not both', TypeError);
+ if (value.shadowOnly === true && (value.castShadow === false || value.holdout === true || value.shadowCatcher === true)) {
+ throw invalidField('shadowOnly', value.shadowOnly, 'false when castShadow is false, or holdout or shadowCatcher is true; a shadow-only node casts a shadow and is not drawn', TypeError);
+ }
if (value.skin !== undefined) {
if (value.geometry === undefined) {
throw invalidField('skin', value.skin, 'a skin on a mesh node with geometry and material; remove skin from group nodes', TypeError);
diff --git a/src/renderer.js b/src/renderer.js
index afbfb126..eba078ae 100644
--- a/src/renderer.js
+++ b/src/renderer.js
@@ -152,6 +152,11 @@ function partNeedsFullUpload(part, texturesReplaceable = Boolean(texturedReplace
return Boolean(part.skin || part.morph || part.geometry.morphTargets || part.geometry.skinIndices || hasMaterialMap(part.material) && !texturesReplaceable);
}
+// A shadow-only part joins the shadow passes and none of the colour, depth or ID passes, so the camera never sees it.
+function partPass(part) {
+ return part.shadowOnly === true ? 'shadowOnly' : renderPass(part.material);
+}
+
function batchPass(batch) {
return batch.pass ?? 'opaque';
}
@@ -279,13 +284,14 @@ const receivesShadow = part => part.receiveShadow !== false;
// Uniform value of a batch: 0 draws normally, 1 is a holdout (zero colour, zero alpha), 2 is a shadow catcher (black, alpha = shadow).
const compositeModeOf = part => (part.holdout === true ? 1 : part.shadowCatcher === true ? 2 : 0);
-const BOOLEAN_PART_FLAGS = ['castShadow', 'receiveShadow', 'holdout', 'shadowCatcher', 'visible'];
+const BOOLEAN_PART_FLAGS = ['castShadow', 'receiveShadow', 'holdout', 'shadowCatcher', 'shadowOnly', 'visible'];
function validatePartFlags(part) {
for (const flag of BOOLEAN_PART_FLAGS) {
if (part[flag] !== undefined && typeof part[flag] !== 'boolean') throw new TypeError(`${flag} must be a boolean`);
}
if (part.holdout === true && part.shadowCatcher === true) throw new Error(`A part is a holdout or a shadow catcher, not both: ${part.name}`);
+ if (part.shadowOnly === true && (part.castShadow === false || part.holdout === true || part.shadowCatcher === true)) throw new Error(`A shadow-only part casts a shadow and is not drawn, so it cannot also set castShadow false, holdout or shadowCatcher: ${part.name}`);
}
/** A hidden part keeps its slot in the batch but draws and casts nothing: its instance matrices are packed as zeros. */
@@ -501,7 +507,7 @@ function createPartGrouper(batching) {
materialTail = [renderPass(part.material), ...materialAlphaState(part.material), part.material.aovMap ?? null, part.material.transmissionMap ?? null, ...(part.material.aov ?? ZERO_VECTOR)];
materialKeyTails.set(part.material, materialTail);
}
- const values = [batching ? part.geometry : part, ...materialValues, part.skin ?? null, part.material.snippet, part.crowd ?? null, part.morph ?? null, castsShadow(part), receivesShadow(part), compositeModeOf(part), depthLayerOf(part), flushOrderOf(part), part.lods ?? null, part.shadowLods ?? null, isMirroredMatrix(part.matrix), ...materialTail];
+ const values = [batching ? part.geometry : part, ...materialValues, part.skin ?? null, part.material.snippet, part.crowd ?? null, part.morph ?? null, castsShadow(part), receivesShadow(part), compositeModeOf(part), part.shadowOnly === true, depthLayerOf(part), flushOrderOf(part), part.lods ?? null, part.shadowLods ?? null, isMirroredMatrix(part.matrix), ...materialTail];
const key = values.map(numberOf).join(',');
let batch = batchesByKey.get(key);
if (!batch) {
@@ -1327,7 +1333,7 @@ export class Renderer67 {
const [alphaMode, alphaCutoff, opacity, doubleSided] = materialAlphaState(parts[0].material);
const layerState = materialLayerState(parts[0].material);
const culling = this.createCullingState(parts);
- const batch = { ...culling, alphaMode, alphaCutoff, opacity, doubleSided, mirrored: isMirroredMatrix(parts[0].matrix), layerState, wear: parts[0].material.wear ?? null, parts: [...parts], vertexArray, instanceBuffer, instanceData, textures, mapState, splat, lightMapTexture, texture: textures[0], map, skin, skinTexture, influenceTexture, crowd, crowdTexture, morph, morphTexture, morphWeights, indexCount: flushOrder?.indices.length ?? geometry.indices.length, instanceCount: parts.length, floor: parts[0].material.tag === 'floor', castShadow: castsShadow(parts[0]), tintedShadow: castsTintedShadow(parts[0]), tintBlind: Boolean(splat) || Object.keys(parts[0].material.layerMaps ?? {}).length >= 3, receivesShadow: receivesShadow(parts[0]), compositeMode: compositeModeOf(parts[0]), depthLayer: depthLayerOf(parts[0]), flushOrder, flushIndexBuffer, pass: renderPass(parts[0].material) };
+ const batch = { ...culling, alphaMode, alphaCutoff, opacity, doubleSided, mirrored: isMirroredMatrix(parts[0].matrix), layerState, wear: parts[0].material.wear ?? null, parts: [...parts], vertexArray, instanceBuffer, instanceData, textures, mapState, splat, lightMapTexture, texture: textures[0], map, skin, skinTexture, influenceTexture, crowd, crowdTexture, morph, morphTexture, morphWeights, indexCount: flushOrder?.indices.length ?? geometry.indices.length, instanceCount: parts.length, floor: parts[0].material.tag === 'floor', castShadow: castsShadow(parts[0]), tintedShadow: castsTintedShadow(parts[0]), tintBlind: Boolean(splat) || Object.keys(parts[0].material.layerMaps ?? {}).length >= 3, receivesShadow: receivesShadow(parts[0]), compositeMode: compositeModeOf(parts[0]), depthLayer: depthLayerOf(parts[0]), flushOrder, flushIndexBuffer, pass: partPass(parts[0]) };
if (parts[0].lods) culling.levelGeometries.forEach((geometryOfLevel, level) => this.levelDraw(batch, level));
culling.shadowLevels?.levelGeometries.forEach((geometryOfLevel, level) => this.levelDraw(batch, level, batch.shadowLevels));
parts.forEach((part, index) => this.partLocations.set(part, { batch, index, geometry: part.geometry, map, mapState, layerState, skin, jointMatrices: skin?.jointMatrices, skinIndices: geometry.skinIndices, skinWeights: geometry.skinWeights, morph, morphWeights: morph?.weights, morphTargets: geometry.morphTargets, normalMatrix: mat3.create(), committedMatrix: batch.instanceData.subarray(index * INSTANCE_STRIDE, index * INSTANCE_STRIDE + 16) }));
@@ -1359,7 +1365,7 @@ export class Renderer67 {
const sameParts = batch?.parts.length === group.length && batch.parts.every((part, index) => part === group[index]);
const alphaState = materialAlphaState(first.material);
const sameState = location?.geometry === first.geometry && batch?.castShadow === castsShadow(first) && batch.receivesShadow === receivesShadow(first) && batch.compositeMode === compositeModeOf(first) && (batch.depthLayer ?? 0) === depthLayerOf(first) && (batch.flushOrder ?? null) === flushOrderOf(first) &&
- batch.floor === (first.material.tag === 'floor') && batchPass(batch) === renderPass(first.material) &&
+ batch.floor === (first.material.tag === 'floor') && batchPass(batch) === partPass(first) &&
[batch.alphaMode, batch.alphaCutoff, batch.opacity, batch.doubleSided].every((value, index) => value === alphaState[index]) &&
batch.mapState.every((value, index) => value === mapState[index]) &&
batch.layerState.every((value, index) => value === materialLayerState(first.material)[index]);
@@ -1523,7 +1529,7 @@ export class Renderer67 {
if (part.skin || part.morph) this.updatePartIdentities.add(part);
validatePartFlags(part);
if (castsShadow(part) !== location.batch.castShadow) throw new Error(`Shadow casting changes require a new renderer: ${part.name}`);
- if (receivesShadow(part) !== location.batch.receivesShadow || compositeModeOf(part) !== location.batch.compositeMode) throw new Error(`receiveShadow, holdout and shadowCatcher changes require replaceParts: ${part.name}`);
+ if (receivesShadow(part) !== location.batch.receivesShadow || (part.shadowOnly === true) !== (batchPass(location.batch) === 'shadowOnly') || compositeModeOf(part) !== location.batch.compositeMode) throw new Error(`receiveShadow, holdout, shadowCatcher and shadowOnly changes require replaceParts: ${part.name}`);
if (depthLayerOf(part) !== (location.batch.depthLayer ?? 0)) throw new Error(`depthLayer changes require replaceParts: ${part.name}`);
// A material whose fields still equal the ones that passed the full checks below needs none of them again.
const materialUnchanged = location.materialSnapshot?.matches(part.material) === true;
@@ -1534,7 +1540,7 @@ export class Renderer67 {
}
if (part.geometry !== location.geometry) throw new Error(`Geometry changes require a new renderer: ${part.name}`);
if (isMirroredMatrix(part.matrix) !== Boolean(location.batch.mirrored)) mirrorChanged = true;
- if (!materialUnchanged && renderPass(part.material) !== batchPass(location.batch)) throw new Error(`Moving a part into or out of the liquid or foam pass requires replaceParts: ${part.name}`);
+ if (!materialUnchanged && partPass(part) !== batchPass(location.batch)) throw new Error(`Moving a part into or out of the liquid or foam pass requires replaceParts: ${part.name}`);
if ((part.skin ?? null) !== location.skin || part.skin?.jointMatrices !== location.jointMatrices || part.geometry.skinIndices !== location.skinIndices || part.geometry.skinWeights !== location.skinWeights) throw new Error(`Skin identity changes require a new renderer: ${part.name}`);
if (part.skin) {
if (PROCEDURAL_MATERIAL_TAGS.has(part.material.tag)) throw new Error('Skinned parts require a standard material tag.');
diff --git a/tests/render-flags-scene.test.js b/tests/render-flags-scene.test.js
index 7b3a8520..57edc150 100644
--- a/tests/render-flags-scene.test.js
+++ b/tests/render-flags-scene.test.js
@@ -82,6 +82,25 @@ test('shadow and composite flags reach the part and split batches', () => {
assert.ok(batches.some(names => names.join() === 'other,plain'));
});
+test('shadowOnly reaches the part, splits batches and conflicts with the flags that remove the shadow or the picture', () => {
+ const scene = createAgentScene(recipe({ nodes: [
+ { id: 'plain', geometry: 'box', material: 'finish' },
+ { id: 'gobo', geometry: 'box', material: 'finish', shadowOnly: true, translation: [2, 0, 0] },
+ { id: 'gobo2', geometry: 'box', material: 'finish', shadowOnly: true, translation: [4, 0, 0] },
+ ] }));
+ const byName = new Map(scene.parts.map(part => [part.name, part]));
+ assert.equal(byName.get('gobo').shadowOnly, true);
+ assert.equal(byName.get('plain').shadowOnly, undefined);
+ const batches = groupParts(scene.parts).map(batch => batch.map(part => part.name).sort());
+ assert.equal(batches.length, 2);
+ assert.ok(batches.some(names => names.join() === 'gobo,gobo2'));
+ const node = fields => recipe({ nodes: [{ id: 'one', geometry: 'box', material: 'finish', ...fields }] });
+ assert.throws(() => createAgentScene(node({ shadowOnly: 1 })), /nodes\[0\]\.shadowOnly: got 1; expected true or false/);
+ assert.throws(() => createAgentScene(node({ shadowOnly: true, castShadow: false })), /nodes\[0\]\.shadowOnly: got true; expected false when castShadow is false/);
+ assert.throws(() => createAgentScene(node({ shadowOnly: true, holdout: true })), /nodes\[0\]\.shadowOnly: got true/);
+ assert.throws(() => createAgentScene(recipe({ nodes: [{ id: 'g', shadowOnly: true }] })), /nodes\[0\]\.shadowOnly: got true; expected a flag on a mesh node/);
+});
+
test('flags must be booleans on mesh nodes and a node is not both holdout and catcher', () => {
const node = fields => recipe({ nodes: [{ id: 'one', geometry: 'box', material: 'finish', ...fields }] });
assert.throws(() => createAgentScene(node({ castShadow: 'no' })), /nodes\[0\]\.castShadow: got "no"; expected true or false/);
diff --git a/tests/render-flags.test.js b/tests/render-flags.test.js
index ba4450e2..1da4042e 100644
--- a/tests/render-flags.test.js
+++ b/tests/render-flags.test.js
@@ -91,7 +91,7 @@ test('visible, castShadow, receiveShadow, holdout and shadowCatcher draw as docu
}
const page = await browser.newPage({ viewport: { width: 96, height: 96 } });
await page.goto(`http://127.0.0.1:${port}/`);
- const names = ['plain', 'noCaster', 'hiddenCaster', 'holdoutCaster', 'holdoutNoCast', 'holdoutBlindFloor', 'catcherLit', 'catcherNoCaster', 'hiddenAll'];
+ const names = ['plain', 'noCaster', 'hiddenCaster', 'holdoutCaster', 'holdoutNoCast', 'holdoutBlindFloor', 'catcherLit', 'catcherNoCaster', 'hiddenAll', 'shadowOnlyCaster'];
const images = await drawScenarios(page, [
{ floor: {}, caster: {} },
{ floor: {}, caster: { holdout: true } },
@@ -102,6 +102,7 @@ test('visible, castShadow, receiveShadow, holdout and shadowCatcher draw as docu
{ floor: { shadowCatcher: true }, caster: { holdout: true } },
{ floor: { shadowCatcher: true }, caster: { visible: false } },
{ floor: { visible: false }, caster: { visible: false } },
+ { floor: {}, caster: { shadowOnly: true } },
]);
const image = Object.fromEntries(names.map((name, index) => [name, images[index]]));
assert.ok(opaquePixels(image.plain) > 3000, 'the plain scene draws the floor and the caster');
@@ -116,6 +117,9 @@ test('visible, castShadow, receiveShadow, holdout and shadowCatcher draw as docu
// A hidden caster casts no shadow either: its scene equals the one whose caster is a holdout that does not cast.
const hiddenNoShadow = differingPixels(image.hiddenCaster, image.holdoutNoCast);
assert.ok(hiddenNoShadow > 150, 'holdout still occludes while a hidden caster leaves the floor whole');
+ // Shadow only (issue 926): the caster leaves the picture but its shadow stays on the floor.
+ assert.ok(differingPixels(image.shadowOnlyCaster, image.plain) > 150, 'a shadow-only caster is not drawn');
+ assert.ok(differingPixels(image.shadowOnlyCaster, image.hiddenCaster) > 30, 'a shadow-only caster still shades the floor, which a hidden one does not');
// Shadow catcher: black pixels whose alpha is the shadow, and nothing without a caster.
assert.equal(opaquePixels(image.catcherNoCaster), 0, 'a lit shadow catcher is fully transparent');
assert.ok(opaquePixels(image.catcherLit) > 30, 'the received shadow is written as alpha');