Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Animate lights, sun, sky, exposure and focus distance with motion tracks (issue #605)

Nguyen Xuan An67be8932065b
diff --git a/changes/605.md b/changes/605.md
new file mode 100644
index 00000000..f523336c
--- /dev/null
+++ b/changes/605.md
@@ -0,0 +1 @@
+- Motion tracks animate lights, the sun, sky and fog, exposure and focus distance, so a sunset, a lamp flicker or a rack focus plays from a recipe (#605)
diff --git a/changes/605.size.md b/changes/605.size.md
new file mode 100644
index 00000000..aa093d27
--- /dev/null
+++ b/changes/605.size.md
@@ -0,0 +1,6 @@
+- scene:agent-scene: light, sun, environment and effects tracks need target checks, ranges, a patch per kind and two getters on the scene (about 330 bytes gzip); the code that applies them loads with the motion module (#605).
+- scene:agent-scene+lazy: same code, in the lazy motion module (#605).
+- goal:agent-scene: a sunset, lamp flicker or rack focus could not be authored before; the core change is the two getters and the pose publish line, the rest sits in the lazy motion module (#605).
+- tool-total: one track schema covers four kinds, with the properties of each kind in scene_capabilities motionLook instead of the tool text.
+- tool:scene_create: the single lookTrack schema in the motion tracks union.
+- result:scene_capabilities: motionLook lists the kinds, ids and properties once, so the tool schemas stay small.
diff --git a/docs/agent-motion.md b/docs/agent-motion.md
index c9caa2a7..f7db7666 100644
--- a/docs/agent-motion.md
+++ b/docs/agent-motion.md
@@ -201,6 +201,36 @@ uniform scale. `true` takes the defaults:
The lock reads the layers only: tracks and `ik` are applied after it. `scene_inspect` returns `footLock.feet` with the
planted phases per foot. The hips do not move, so a ground far from the character's floor can be out of reach.
+### Light, sun, sky and exposure tracks
+
+On a scene recipe (not an imported asset) four more kinds animate the look, with the keyframes, interpolations and `easing` of
+any other track. `light` takes an `id` of `point:N`, `spot:N` or `area:N` (the index in `environment.pointLights` and so on) and
+the properties `position`, `color`, `intensity`, plus `direction` on spot and area lights. `sun` (no ID) takes `direction`,
+`color` and `intensity`, which set `lightDirection`, `keyColor` and `keyIntensity`. `environment` takes `skyTop`, `skyBottom`,
+`fogColor`, `fogDensity`, `volumeFogDensity`, `ambientIntensity`, `fillIntensity`, `environmentIntensity` and
+`environmentRotation` (radians; these dim and turn the sky map shown when `environment.world` is true). `effects` takes `exposure`
+and `focusDistance`, which needs `effects.depthOfField.blur` above 0 (a camera with an `fStop` focuses on its target, so move the
+camera `target` track for a rack focus). Values keep the range of the environment field, and an unknown light, property or kind
+fails with the valid ones. Untracked values keep their authored value, and a scene with no such track renders as before.
+
+After `scene.sampleMotion(time)`, `scene.environment` and `scene.effects` return the sampled values (the exported recipe keeps
+the authored ones), so a renderer that reads them per frame follows the tracks without a rebuild. Baked light (`bake`) is computed
+once and does not follow a moving light.
+
+```js
+// A sunset: the sun drops to the horizon, the sky map turns and dims, the exposure rises to compensate.
+const sunset = {
+ duration: 4,
+ tracks: [
+ { kind: 'sun', property: 'direction', keyframes: [{ time: 0, value: [-1, 4, 1] }, { time: 4, value: [-4, 0.3, 1] }] },
+ { kind: 'sun', property: 'color', keyframes: [{ time: 0, value: [3, 2.8, 2.5] }, { time: 4, value: [3.4, 1.4, 0.6] }] },
+ { kind: 'environment', property: 'environmentIntensity', keyframes: [{ time: 0, value: 1 }, { time: 4, value: 0.4 }] },
+ { kind: 'light', id: 'point:0', property: 'intensity', keyframes: [{ time: 0, value: 0 }, { time: 4, value: 3 }] },
+ { kind: 'effects', property: 'exposure', keyframes: [{ time: 0, value: 1 }, { time: 4, value: 1.4 }] },
+ ],
+};
+```
+
### Morph weight tracks
A track `{ "kind": "morph", "id": "eyeBlinkLeft", "property": "weight", "keyframes": [...] }` keys the weight (0 to 1) of every
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index 5c32c0e3..d8986d7d 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -442,6 +442,14 @@ function motionTrack(kind, property, value, tangent) {
})).min(1).max(64),
});
}
+// Lights, sun, sky and effects share one track shape: the backend names the valid light IDs and properties of each kind when one is wrong.
+const lookTrack = z.strictObject({
+ kind: z.enum(['light', 'sun', 'environment', 'effects']).describe('Properties per kind: scene_capabilities motionLook.'),
+ id: z.string().max(16).optional().describe('Light only: point:0, spot:0, area:0.'),
+ property: z.string().max(24),
+ interpolation: z.enum(['LINEAR', 'STEP', 'SMOOTH']).optional(),
+ keyframes: z.array(z.strictObject({ time: z.number().min(0).max(30), value: z.union([finite, vector3]), easing: easingField })).min(1).max(64),
+});
const ikKeyframes = (value, description) => z.strictObject({
interpolation: z.enum(['LINEAR', 'STEP', 'SMOOTH']).optional(),
keyframes: z.array(z.strictObject({ time: z.number().min(0).max(30), value })).min(1).max(64),
@@ -525,6 +533,7 @@ const motion = z.strictObject({
motionTrack('camera', 'target', vector3, vector3),
motionTrack('camera', 'fov', finite.gt(0).lt(180), finite),
motionTrack('camera', 'roll', finite, finite),
+ lookTrack,
])).max(128),
}).describe('One absolute-time procedural clip. Local node transforms propagate through parents. Cubic tangents are derivatives per second. Backend checks references, duplicate tracks, 2048 total keys and every sampled pose.');
const physicsBody = z.strictObject({
diff --git a/src/agent-motion.js b/src/agent-motion.js
index fe064588..b80da211 100644
--- a/src/agent-motion.js
+++ b/src/agent-motion.js
@@ -15,6 +15,29 @@ const PROPERTIES = {
camera: { eye: 3, target: 3, fov: 1, roll: 1 },
// An imported asset's morph target weight, addressed by the target's name.
morph: { weight: 1 },
+ // A light of the scene, addressed as point:N, spot:N or area:N (its index in environment.pointLights and so on). Spot and area lights also take direction.
+ light: { position: 3, direction: 3, color: 3, intensity: 1 },
+ // The key light of the environment: lightDirection, keyColor, keyIntensity.
+ sun: { direction: 3, color: 3, intensity: 1 },
+ // Sky, fog and ambient values of the environment. volumeFogDensity is environment.volumeFog.density; environmentIntensity and
+ // environmentRotation (radians) dim and turn the sky map that shows when environment.world is true.
+ environment: { skyTop: 3, skyBottom: 3, fogColor: 3, fogDensity: 1, volumeFogDensity: 1, ambientIntensity: 1, fillIntensity: 1, environmentIntensity: 1, environmentRotation: 1 },
+ // Frame effects: exposure, and the focus distance of the depth of field (needs effects.depthOfField.blur above 0).
+ effects: { exposure: 1, focusDistance: 1 },
+};
+// Kinds with one target in the scene, so a track names no ID.
+const SINGLETON_KINDS = new Set(['camera', 'sun', 'environment', 'effects']);
+const LOOK_KINDS = new Set(['light', 'sun', 'environment', 'effects']);
+const LIGHT_LISTS = { point: 'pointLights', spot: 'spotLights', area: 'areaLights' };
+const LIGHT_DIRECTION_TYPES = new Set(['spot', 'area']);
+const LOOK_RANGES = {
+ light: { position: [-1e6, 1e6], direction: [-1e6, 1e6], color: [0, 1], intensity: [0, 1e5] },
+ sun: { direction: [-1e6, 1e6], color: [0, 1000], intensity: [0, 100] },
+ environment: {
+ skyTop: [0, 1], skyBottom: [0, 1], fogColor: [0, 1], fogDensity: [0, 10], volumeFogDensity: [0, 1000],
+ ambientIntensity: [0, 100], fillIntensity: [0, 100], environmentIntensity: [0, 100], environmentRotation: [-100, 100],
+ },
+ effects: { exposure: [0.01, 16], focusDistance: [0.01, 100000] },
};
const INTERPOLATIONS = ['LINEAR', 'STEP', 'SMOOTH', 'CUBICSPLINE'];
const MAX_IK_SOLVERS = 16;
@@ -43,6 +66,13 @@ function validateValue(kind, property, value) {
if (kind === 'morph' && components.some(component => component < 0 || component > 1)) {
throw new RangeError('Motion morph weight must stay within zero and one.');
}
+ const range = LOOK_RANGES[kind]?.[property];
+ if (range && components.some(component => component < range[0] || component > range[1])) {
+ throw new RangeError(`Motion ${kind} ${property} must stay within ${range[0]} and ${range[1]}.`);
+ }
+ if (property === 'direction' && Math.hypot(...components) < 1e-6) {
+ throw new RangeError(`Motion ${kind} direction needs a length of at least 0.000001; a blend between opposite directions passes through zero.`);
+ }
if (kind === 'camera' && property === 'fov' && (Math.fround(value) <= 0 || Math.fround(value) >= 180)) {
throw new RangeError('Motion fov must stay between zero and 180 at Float32 precision.');
}
@@ -90,6 +120,32 @@ function normalizedKeyframe(keyframe, track, duration, interpolation) {
return result;
}
+/** The light IDs a recipe's environment has, such as point:0, spot:0 and area:1. */
+function lightIds(environment) {
+ return Object.entries(LIGHT_LISTS).flatMap(([type, field]) => (environment[field] ?? []).map((_, index) => `${type}:${index}`));
+}
+
+/** Checks that a light, sun, environment or effects track names something the scene has. */
+function checkLookTarget(track, recipe) {
+ const { kind, id, property } = track;
+ if (!LOOK_KINDS.has(kind)) return;
+ if (!recipe.environment || !recipe.effects) {
+ throw new RangeError(`Motion ${kind} tracks move the lights, sky and effects of a scene recipe; an imported asset has none.`);
+ }
+ if (kind === 'light') {
+ const ids = lightIds(recipe.environment);
+ if (!ids.includes(id)) {
+ throw new RangeError(`Motion light was not found: ${JSON.stringify(id)}. Lights of this scene: ${ids.join(', ') || 'none'}. The sun is a sun track.`);
+ }
+ if (property === 'direction' && !LIGHT_DIRECTION_TYPES.has(id.split(':')[0])) {
+ throw new RangeError(`Motion light ${id} has no direction; only spot and area lights do. A point light takes position, color and intensity.`);
+ }
+ }
+ if (kind === 'effects' && property === 'focusDistance' && !(recipe.effects.depthOfField?.blur > 0)) {
+ throw new RangeError('Motion focusDistance needs effects.depthOfField.blur above 0. A camera with an fStop focuses on its target, so move the camera target track instead.');
+ }
+}
+
function nodeIdFrom(value, nodes, label) {
if (typeof value !== 'string' || !nodes.has(value)) {
throw new RangeError(`${label} must be a node ID in this recipe; got ${JSON.stringify(value)}.`);
@@ -318,22 +374,24 @@ export function normalizeAgentMotion(input, recipe) {
const tracks = inputTracks.map((track, index) => {
knownKeys(track, ['kind', 'id', 'property', 'interpolation', 'keyframes'], 'motion track');
if (!Object.hasOwn(PROPERTIES, track.kind) || !Object.hasOwn(PROPERTIES[track.kind], track.property)) {
- throw new RangeError('Unsupported motion target or property.');
+ const known = Object.hasOwn(PROPERTIES, track.kind) ? ` ${track.kind} properties: ${Object.keys(PROPERTIES[track.kind]).join(', ')}.` : ` Kinds: ${Object.keys(PROPERTIES).join(', ')}.`;
+ throw new RangeError(`Unsupported motion target or property.${known}`);
}
- if (track.kind === 'camera') {
+ if (SINGLETON_KINDS.has(track.kind)) {
if (Object.hasOwn(track, 'id')) {
- throw new TypeError('Camera tracks cannot have an ID.');
+ throw new TypeError(`${track.kind[0].toUpperCase()}${track.kind.slice(1)} tracks cannot have an ID.`);
}
} else {
if (typeof track.id !== 'string' || !track.id.trim() || track.id.length > 128) {
throw new TypeError('Motion target IDs need 1 to 128 characters.');
}
const references = track.kind === 'node' ? nodes : track.kind === 'morph' ? morphTargets : materials;
- if (!references.has(track.id)) {
+ if (track.kind !== 'light' && !references.has(track.id)) {
const hint = track.kind === 'morph' ? ` Morph targets of this asset: ${[...morphTargets].slice(0, 24).join(', ') || 'none'}.` : '';
throw new RangeError(`Motion target was not found: ${track.id}.${hint}`);
}
}
+ checkLookTarget(track, recipe);
const identity = JSON.stringify([track.kind, track.id ?? null, track.property]);
if (identities.has(identity)) {
throw new RangeError('Duplicate motion target property.');
@@ -359,7 +417,7 @@ export function normalizeAgentMotion(input, recipe) {
interpolation,
keyframes: clip.tracks[0].keyframes.map(keyframe => copiedKeyframe(keyframe, interpolation)),
};
- if (track.kind !== 'camera') {
+ if (!SINGLETON_KINDS.has(track.kind)) {
result.id = track.id;
}
return result;
@@ -521,6 +579,43 @@ function constrainedNodeIds(motion) {
return new Set([...driven, ...(motion.constraints ?? []).map(constraint => constraint.node)]);
}
+/** An environment (a clone the caller may change) with the light, sun and sky values a sampled motion sets. */
+export function motionEnvironment(environment, sampled) {
+ for (const [id, patch] of sampled.lights) {
+ const [type, index] = id.split(':');
+ Object.assign(environment[LIGHT_LISTS[type]][Number(index)], copiedPatch(patch));
+ }
+ if (sampled.sun) {
+ const { direction, color, intensity } = copiedPatch(sampled.sun);
+ if (direction) environment.lightDirection = direction;
+ if (color) environment.keyColor = color;
+ if (intensity !== undefined) environment.keyIntensity = intensity;
+ }
+ if (sampled.environment) {
+ const { volumeFogDensity, ...fields } = copiedPatch(sampled.environment);
+ Object.assign(environment, fields);
+ if (volumeFogDensity !== undefined) environment.volumeFog.density = volumeFogDensity;
+ }
+ return environment;
+}
+
+/** Effects (a clone the caller may change) with the exposure and focus distance a sampled motion sets. */
+export function motionEffects(effects, sampled) {
+ const { exposure, focusDistance } = sampled.effects;
+ if (exposure !== undefined) effects.exposure = exposure;
+ if (focusDistance !== undefined) effects.depthOfField.focusDistance = focusDistance;
+ return effects;
+}
+
+/** True when a sample moves a light, the sky or the effects, which the scene then reads through the two functions above. */
+export function motionMovesLook(sampled) {
+ return Boolean(sampled && (sampled.lights.size || sampled.sun || sampled.environment || sampled.effects));
+}
+
+function copiedPatch(patch) {
+ return Object.fromEntries(Object.entries(patch).map(([property, value]) => [property, typeof value === 'number' ? value : Array.from(value)]));
+}
+
/** Retains published maps and vectors. A failed sample leaves them unchanged. */
export function compileAgentMotion(input, recipe) {
const motion = normalizeAgentMotion(input, recipe);
@@ -528,6 +623,7 @@ export function compileAgentMotion(input, recipe) {
return null;
}
const ordinaryTracks = motion.tracks.filter(track => track.kind !== 'camera');
+ // Light, sun, environment and effects patches are allocated once here; the scene reads them through lookAt functions below.
const clip = createAnimationClip({
duration: motion.duration,
loop: motion.loop,
@@ -539,14 +635,20 @@ export function compileAgentMotion(input, recipe) {
})),
});
const sampler = compileAnimationSampler(clip);
- const output = { nodes: new Map(), materials: new Map(), morphs: new Map(), camera: null };
- const patchesByKind = { node: output.nodes, material: output.materials, morph: output.morphs };
+ const output = { nodes: new Map(), materials: new Map(), morphs: new Map(), lights: new Map(), sun: null, environment: null, effects: null, camera: null };
+ const patchesByKind = { node: output.nodes, material: output.materials, morph: output.morphs, light: output.lights };
const bindings = ordinaryTracks.map((track, index) => {
- const patches = patchesByKind[track.kind];
- if (!patches.has(track.id)) {
- patches.set(track.id, {});
+ let patch;
+ if (LOOK_KINDS.has(track.kind) && track.kind !== 'light') {
+ output[track.kind] ??= {};
+ patch = output[track.kind];
+ } else {
+ const patches = patchesByKind[track.kind];
+ if (!patches.has(track.id)) {
+ patches.set(track.id, {});
+ }
+ patch = patches.get(track.id);
}
- const patch = patches.get(track.id);
const size = PROPERTIES[track.kind][track.property];
patch[track.property] = size === 1 ? 0 : new Float64Array(size);
return { track, patch, staged: sampler.tracks[index].output };
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index abc317cf..16796e3b 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -280,6 +280,13 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
notes: 'Bounced light from point, spot, area and key lights. Reruns when parts, lights, materials or bake change. Skips alpha, skin, morph, hair, cloth (bake.skipped). A preset shrinks to fit, never refuses (bake.used, bake.reduction); explicit settings fail over maxPathWork or past maxSeconds. No environment map or specular. Probes: specular only, none in glTF.',
limits: Object.freeze({ maxSeconds: BAKE_MAXIMUM_SECONDS, workerMaxSeconds: BAKE_WORKER_MAXIMUM_SECONDS, maxPathWork: BAKE_EXPLICIT_PATH_WORK_LIMIT, maxPointLights: MAX_POINT_LIGHTS }),
}),
+ motionLook: Object.freeze({
+ light: 'id point:N, spot:N or area:N (index in its list): position, color, intensity; direction on spot and area',
+ sun: 'direction, color, intensity (lightDirection, keyColor, keyIntensity)',
+ environment: 'skyTop, skyBottom, fogColor, fogDensity, volumeFogDensity, ambientIntensity, fillIntensity, environmentIntensity, environmentRotation',
+ effects: 'exposure, focusDistance (needs effects.depthOfField.blur above 0; an fStop camera focuses on its target)',
+ note: 'Same keyframes and easing as other tracks. After sampleMotion, scene.environment and scene.effects hold the sampled values; the recipe keeps its authored ones. Baked light (bake) does not follow.',
+ }),
motionLimits: Object.freeze({ maxTracks: 128, maxKeyframesPerTrack: 64, maxKeyframes: 2048, maxDuration: 30 }),
});
diff --git a/src/agent-scene.js b/src/agent-scene.js
index bf27cf27..39e1e296 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -939,7 +939,8 @@ function stagePose(stagedState, time) {
const clothGeometries = sampledGeometries(stagedState, time);
// Staged last: a pose pending on a decoded asset must be committed or discarded, so nothing above may throw after it.
const assetClips = stagedState.assetRuntime.animators.length ? agentSceneModule('assets').stageAssetClips(stagedState.assetRuntime, graph, time) : null;
- return { graph, materialUpdates, camera, time, clothGeometries, assetClips, hiddenNodes };
+ const movedLook = stagedState.motionSampler && agentSceneModule('motion').motionMovesLook(sampled) ? sampled : null;
+ return { graph, materialUpdates, camera, time, clothGeometries, assetClips, hiddenNodes, movedLook };
}
// Cloth and particles replace the geometry of their own nodes at a sampled time; no node belongs to both.
@@ -1018,6 +1019,7 @@ function publishPose(stagedState, pose) {
}
for (const sequence of stagedState.sequences) sequence.seek(pose.time);
stagedState.renderCamera = pose.camera;
+ stagedState.movedLook = pose.movedLook;
stagedState.motionTime = pose.time;
const nextParts = partsWithCloth(stagedState.restParts, pose.clothGeometries, stagedState.parts);
const partsChanged = nextParts !== stagedState.parts;
@@ -1472,14 +1474,17 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
}
return state.motionTime;
},
+ // After sampleMotion, the lights, sun and sky the motion tracks set at that time; the recipe keeps its authored values.
get environment() {
- return structuredClone(state.recipe.environment);
+ const environment = structuredClone(state.recipe.environment);
+ return state.movedLook ?agentSceneModule('motion').motionEnvironment(environment, state.movedLook) : environment;
},
get postPasses() {
return structuredClone(state.recipe.postPasses ?? []);
},
get effects() {
- return structuredClone(state.recipe.effects);
+ const effects = structuredClone(state.recipe.effects);
+ return state.movedLook?.effects ?agentSceneModule('motion').motionEffects(effects, state.movedLook) : effects;
},
get shadow() {
return structuredClone(state.recipe.shadow);
diff --git a/tests/agent-motion-look.test.js b/tests/agent-motion-look.test.js
new file mode 100644
index 00000000..fc9f1d45
--- /dev/null
+++ b/tests/agent-motion-look.test.js
@@ -0,0 +1,164 @@
+// Motion tracks for lights, the sun, sky and fog, exposure and focus distance (issue 605).
+import './support/load-agent-scene-modules.mjs';
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { spawn } from 'node:child_process';
+import { createAgentScene } from '../src/agent-scene.js';
+import { freePort } from './helpers/free-port.js';
+
+function sceneRecipe(tracks, extra = {}) {
+ return {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'box', type: 'box', bevel: 0 }],
+ materials: [{ id: 'finish', color: [0.6, 0.6, 0.6], emission: 0 }],
+ nodes: [{ id: 'block', geometry: 'box', material: 'finish' }],
+ camera: { eye: [4, 3, 5], target: [0, 0, 0] },
+ environment: {
+ pointLights: [{ position: [0, 3, 0], color: [1, 0.9, 0.7], intensity: 2 }],
+ spotLights: [{ position: [2, 3, 0], direction: [0, -1, 0] }],
+ areaLights: [{ position: [0, 4, 0], direction: [0, -1, 0], width: 1, height: 1 }],
+ },
+ effects: { depthOfField: { blur: 0.4, focusDistance: 4 } },
+ motion: { duration: 2, tracks },
+ ...extra,
+ };
+}
+const keys = (start, end) => [{ time: 0, value: start }, { time: 2, value: end }];
+
+test('light tracks set color, intensity and position of one light and leave the rest authored', () => {
+ const scene = createAgentScene(sceneRecipe([
+ { kind: 'light', id: 'point:0', property: 'intensity', keyframes: keys(2, 0) },
+ { kind: 'light', id: 'point:0', property: 'position', keyframes: keys([0, 3, 0], [4, 3, 0]) },
+ { kind: 'light', id: 'spot:0', property: 'direction', keyframes: keys([0, -1, 0], [-1, -1, 0]) },
+ { kind: 'light', id: 'area:0', property: 'color', keyframes: keys([1, 1, 1], [1, 0, 0]) },
+ ]));
+ scene.sampleMotion(1);
+ const environment = scene.environment;
+ assert.equal(environment.pointLights[0].intensity, 1);
+ assert.deepEqual(environment.pointLights[0].position, [2, 3, 0]);
+ assert.deepEqual(environment.pointLights[0].color, [1, 0.9, 0.7]);
+ assert.deepEqual(environment.spotLights[0].direction, [-0.5, -1, 0]);
+ assert.deepEqual(environment.areaLights[0].color, [1, 0.5, 0.5]);
+ scene.sampleMotion(null);
+ assert.equal(scene.environment.pointLights[0].intensity, 2);
+ assert.equal(scene.exportRecipe().environment.pointLights[0].intensity, 2);
+});
+
+test('sun and environment tracks set the key light, sky, fog and volume fog', () => {
+ const scene = createAgentScene(sceneRecipe([
+ { kind: 'sun', property: 'direction', keyframes: keys([-1, 4, 0], [-1, 0.2, 0]) },
+ { kind: 'sun', property: 'intensity', keyframes: keys(1, 0.2) },
+ { kind: 'environment', property: 'skyTop', keyframes: keys([0.2, 0.4, 0.9], [0.9, 0.3, 0.1]) },
+ { kind: 'environment', property: 'fogDensity', keyframes: keys(0, 0.4) },
+ { kind: 'environment', property: 'volumeFogDensity', keyframes: keys(0, 0.2) },
+ ]));
+ scene.sampleMotion(2);
+ const environment = scene.environment;
+ assert.deepEqual(environment.lightDirection, [-1, 0.2, 0]);
+ assert.equal(environment.keyIntensity, 0.2);
+ assert.deepEqual(environment.skyTop, [0.9, 0.3, 0.1]);
+ assert.equal(environment.fogDensity, 0.4);
+ assert.equal(environment.volumeFog.density, 0.2);
+ assert.notDeepEqual(scene.exportRecipe().environment.skyTop, environment.skyTop);
+});
+
+test('effects tracks set exposure and the focus distance of the depth of field', () => {
+ const scene = createAgentScene(sceneRecipe([
+ { kind: 'effects', property: 'exposure', keyframes: keys(1, 2) },
+ { kind: 'effects', property: 'focusDistance', interpolation: 'SMOOTH', keyframes: keys(2, 8) },
+ ]));
+ scene.sampleMotion(1);
+ assert.equal(scene.effects.exposure, 1.5);
+ assert.equal(scene.effects.depthOfField.focusDistance, 5);
+ assert.equal(scene.effects.depthOfField.blur, 0.4);
+});
+
+test('a scene with no look tracks reads the authored environment and effects', () => {
+ const scene = createAgentScene(sceneRecipe([{ kind: 'material', id: 'finish', property: 'emission', keyframes: keys(0, 1) }]));
+ const authored = JSON.stringify([scene.environment, scene.effects]);
+ scene.sampleMotion(1);
+ assert.equal(JSON.stringify([scene.environment, scene.effects]), authored);
+});
+
+test('unknown lights, properties and ill-fitting targets name the valid ones', () => {
+ const failure = (track, pattern) => assert.throws(() => createAgentScene(sceneRecipe([track])), pattern);
+ failure({ kind: 'light', id: 'point:3', property: 'color', keyframes: keys([1, 1, 1], [0, 0, 0]) }, /point:3.*point:0, spot:0, area:0/);
+ failure({ kind: 'light', id: 'point:0', property: 'direction', keyframes: keys([0, -1, 0], [0, 1, 0]) }, /point light has no direction|point:0 has no direction/);
+ failure({ kind: 'light', id: 'point:0', property: 'radius', keyframes: keys(1, 2) }, /light properties: position, direction, color, intensity/);
+ failure({ kind: 'sun', property: 'angle', keyframes: keys(1, 2) }, /sun properties: direction, color, intensity/);
+ failure({ kind: 'sun', id: 'x', property: 'intensity', keyframes: keys(1, 2) }, /Sun tracks cannot have an ID/);
+ failure({ kind: 'environment', property: 'skyTop', keyframes: keys([0, 0, 0], [2, 0, 0]) }, /skyTop must stay within 0 and 1/);
+ failure({ kind: 'sun', property: 'direction', keyframes: keys([0, 0, 0], [0, 1, 0]) }, /direction needs a length/);
+ failure({ kind: 'effects', property: 'exposure', keyframes: keys(1, 99) }, /exposure must stay within/);
+ assert.throws(() => createAgentScene(sceneRecipe([{ kind: 'effects', property: 'focusDistance', keyframes: keys(1, 2) }],
+ { effects: {} })), /focusDistance needs effects.depthOfField.blur above 0/);
+});
+
+test('a sunset changes the sun and sky between two frames of a render', async (context) => {
+ let chromium;
+ try {
+ ({ chromium } = await import('playwright'));
+ } catch {
+ context.skip('playwright is not installed');
+ return;
+ }
+ const port = await freePort();
+ const server = spawn(process.execPath, ['node_modules/vite/bin/vite.js', '--host', '127.0.0.1', '--port', String(port), '--strictPort'], { windowsHide: true, stdio: ['ignore', 'pipe', 'pipe'] });
+ await new Promise((resolve, reject) => {
+ const timeout = setTimeout(() => reject(new Error('The dev server did not start.')), 30000);
+ server.stdout.on('data', data => { if (/ready in/.test(data.toString())) { clearTimeout(timeout); resolve(); } });
+ server.on('exit', code => { clearTimeout(timeout); reject(new Error(`The dev server exited ${code}.`)); });
+ });
+ let browser;
+ try {
+ try {
+ browser = await chromium.launch({ headless: true, args: ['--use-angle=swiftshader', '--enable-unsafe-swiftshader', '--enable-webgl', '--ignore-gpu-blocklist'] });
+ } catch (error) {
+ context.skip(`Chrome did not start: ${error.message.split('\n')[0]}`);
+ return;
+ }
+ const page = await browser.newPage({ viewport: { width: 64, height: 64 } });
+ await page.goto(`http://127.0.0.1:${port}/`);
+ const recipe = sceneRecipe([
+ { kind: 'sun', property: 'direction', keyframes: keys([-1, 4, 1], [-4, 0.3, 1]) },
+ { kind: 'environment', property: 'environmentRotation', keyframes: keys(0, 2) },
+ { kind: 'environment', property: 'environmentIntensity', keyframes: keys(1, 0.4) },
+ ], { environment: { world: true } });
+ const frames = await page.evaluate(async (input) => {
+ const { createAgentScene } = await import('/src/agent-scene.js');
+ const { loadAllAgentSceneModules } = await import('/src/agent-scene.js');
+ await loadAllAgentSceneModules?.();
+ const { Renderer67 } = await import('/src/renderer.js');
+ const { agentEnvironmentMap } = await import('/src/agent-look.js');
+ const canvas = document.createElement('canvas');
+ canvas.width = 64;
+ canvas.height = 64;
+ document.body.append(canvas);
+ const copy = document.createElement('canvas');
+ copy.width = 64;
+ copy.height = 64;
+ const copyContext = copy.getContext('2d', { willReadFrequently: true });
+ const scene = createAgentScene(input);
+ const renderer = new Renderer67(canvas, scene.parts, { ground: false, world: true, samples: 1, pixelRatio: 1, environment: scene.environment });
+ const results = [];
+ for (const time of [0, 2]) {
+ scene.sampleMotion(time);
+ const environment = scene.environment;
+ renderer.render({ ...scene.camera, environment: { ...environment, map: agentEnvironmentMap(environment) } });
+ copyContext.clearRect(0, 0, 64, 64);
+ copyContext.drawImage(canvas, 0, 0, 64, 64);
+ results.push(Array.from(copyContext.getImageData(0, 0, 64, 64).data));
+ }
+ return results;
+ }, recipe);
+ let differing = 0;
+ for (let index = 0; index < frames[0].length; index += 4) {
+ const change = Math.abs(frames[0][index] - frames[1][index]) + Math.abs(frames[0][index + 1] - frames[1][index + 1]) + Math.abs(frames[0][index + 2] - frames[1][index + 2]);
+ if (change > 12) differing++;
+ }
+ assert.ok(differing > 1500, `the sunset changed ${differing} of 4096 pixels`);
+ } finally {
+ await browser?.close();
+ server.kill();
+ }
+});