3D engine that AI agents drive
Add F-curve modifiers and extrapolation to motion tracks (issue #690)
Nguyen Xuan An6405ac7b3a52
diff --git a/changes/690-fcurve-modifiers.md b/changes/690-fcurve-modifiers.mdnew file mode 100644index 00000000..bd986ac8--- /dev/null+++ b/changes/690-fcurve-modifiers.md@@ -0,0 +1 @@+- Motion tracks take `extrapolation` (`constant` or `linear`) and `modifiers`: `cycles` (repeat, offset, mirror, with counts), `stepped`, `noise` (seeded), `generator`, `limits` and `envelope`, sampled the same in sampleMotion, camera paths and render jobs (#690)diff --git a/changes/690-fcurve-modifiers.size.md b/changes/690-fcurve-modifiers.size.mdnew file mode 100644index 00000000..e389732e--- /dev/null+++ b/changes/690-fcurve-modifiers.size.md@@ -0,0 +1,2 @@+- entry:agent-asset: track modifiers and extrapolation (noise, cycles, envelope and four more) load with the motion code, which agents use for camera shake and looping walks+- scene:agent-scene+lazy: the same lazy motion code; the eager agent-scene bundle stays within its allowancediff --git a/docs/animation-curves.md b/docs/animation-curves.mdindex d9f8a780..bd9e5a9f 100644--- a/docs/animation-curves.md+++ b/docs/animation-curves.md@@ -82,3 +82,31 @@ Transform-only targets retain their original material objects.Explicit tangent data keeps the existing track model and permits deterministic sampling without external callback state.This unit does not add a mixer, skeletal animation, morph targets or glTF importer changes.++## Extrapolation and modifiers++A track in a recipe `motion` takes `extrapolation` and `modifiers`, after Blender's F-curve modifiers. Every sampler reads them the same way: `sampleMotion`, camera paths and render jobs. Each result depends only on the track and the time, so a seek backwards and a render give the same values. A clip that does not loop passes times past its duration to the tracks, so extrapolation and cycles work there; a looping clip wraps the time first.++- `extrapolation`: `constant` (default) holds the first and last value. `linear` continues the slope of the first or last segment. Not for rotation.+- Time modifiers run first, in listed order. `cycles` takes `before` and `after` (`repeat`, `offset`, `mirror` or `none`, default `repeat`) and `beforeCount` and `afterCount` (0 is forever; after the count the curve holds). `offset` adds the change from the first to the last key each cycle. `stepped` takes `step` seconds and `offset` and holds the curve for each step.+- Value modifiers run next, in listed order, on every component. `noise` adds smooth seeded noise: `scale` seconds per wobble, `strength`, `phase`, `seed`; each component gets its own noise. `generator` takes `coefficients` of a polynomial of time, lowest power first, and replaces the value, or adds with `additive: true`. `limits` clamps to `min` and `max`. `envelope` maps `referenceMin` to `referenceMax` (default -1 to 1) onto a `min` and `max` that change over `points: [{time, min, max}]`.+- Rotation tracks take only `cycles` (not `offset`) and `stepped`. At most 8 modifiers a track. Bad fields name the accepted ones.+- Library use: `createAnimationClip` accepts the same fields once `animation-modifiers.js` is imported; GLB export writes the keyframes only.++Camera shake: hold a camera still and add noise to its eye, with a faster, weaker noise on the target.++```json+{+ "duration": 4,+ "tracks": [+ { "kind": "camera", "property": "eye",+ "keyframes": [{ "time": 0, "value": [0, 1.6, 5] }],+ "modifiers": [{ "type": "noise", "scale": 0.12, "strength": 0.03, "seed": 11 }] },+ { "kind": "camera", "property": "target",+ "keyframes": [{ "time": 0, "value": [0, 1.5, 0] }],+ "modifiers": [{ "type": "noise", "scale": 0.07, "strength": 0.015, "seed": 12 }] }+ ]+}+```++A walk that travels: two keys on `translation` with `{ "type": "cycles", "before": "offset", "after": "offset" }` carry on forward past the last key by one stride per cycle. Stop motion: `{ "type": "stepped", "step": 0.0833 }` samples at 12 frames a second.diff --git a/docs/parity.md b/docs/parity.mdindex 8e15b374..a45e9ae0 100644--- a/docs/parity.md+++ b/docs/parity.md@@ -33,6 +33,7 @@ To move to a new Blender release, run `node scripts/parity-manual-snapshot.mjs`.| Manual path | Status | What exists, or why not | Evidence | Issues || --- | --- | --- | --- | --- || User Interface | out of scope | Interface for a person with a mouse. Agents use MCP tools. | | |+| Editors > Graph Editor > F-Curves | partial | Per track extrapolation `constant` or `linear` (slope of the end segment, not for rotation) and up to 8 modifiers: `cycles` (repeat, offset, mirror, counts), `stepped`, `noise`, `generator`, `limits` and `envelope`, seeded and the same at every sample time. Noise is smooth value noise, not Blender's, so values differ; no blend modes, no frame range or influence per modifier, no Bezier-handle slope, and glTF export keeps only the keyframes. The editor itself is out of scope. | [animation-modifiers](../tests/animation-modifiers.test.js) | #690 || Editors | out of scope | Interface for a person with a mouse. Agents use MCP tools, and people review images. | | || Scenes & Objects > Scenes | partial | One recipe holds one scene with its camera and environment. There are no scene units or linked scenes. | | || Scenes & Objects > Objects > Object Types | partial | Mesh nodes and empty nodes exist, and `curve` and `text` geometry make curve-like and text objects as meshes. Lights are environment settings. Metaballs and NURBS surfaces are out of scope. | [curve-text](../tests/curve-text.test.js), [Blender comparison](../tests/blender-curve-text.test.js) | #45 |diff --git a/src/agent-motion.js b/src/agent-motion.jsindex 285e7fc2..6ef4a62b 100644--- a/src/agent-motion.js+++ b/src/agent-motion.js@@ -1,4 +1,5 @@import { createAnimationClip, compileAnimationSampler } from './animation.js';+import { trackEffectsDescription } from './animation-modifiers.js';import { invalidField, withFieldPrefix } from './field-error.js';import { createCameraPath } from './camera-path.js';import { CAMERA_LENS_FIELDS } from './camera.js';@@ -374,7 +375,7 @@ export function normalizeAgentMotion(input, recipe) {const identities = new Set();let keyframeCount = 0;const tracks = inputTracks.map((track, index) => {- knownKeys(track, ['kind', 'id', 'property', 'interpolation', 'keyframes'], 'motion track');+ knownKeys(track, ['kind', 'id', 'property', 'interpolation', 'keyframes', 'extrapolation', 'modifiers'], 'motion track');if (!Object.hasOwn(PROPERTIES, track.kind) || !Object.hasOwn(PROPERTIES[track.kind], track.property)) {const known = Object.hasOwn(PROPERTIES, track.kind) ? ` ${track.kind} properties: ${Object.keys(PROPERTIES[track.kind]).join(', ')}.` : ` Kinds: ${Object.keys(PROPERTIES).join(', ')}.`;throw invalidField(`motion.tracks[${index}].property`, `${track.kind}.${track.property}`, `a kind and property pair.${known}`);@@ -411,7 +412,7 @@ export function normalizeAgentMotion(input, recipe) {const keyframes = track.keyframes.map((keyframe, keyframeIndex) => withFieldPrefix(`motion.tracks[${index}].keyframes[${keyframeIndex}]`, () => normalizedKeyframe(keyframe, track, duration, interpolation)));const clip = createAnimationClip({duration, loop,- tracks: [{ target: String(index), property: coreProperty(track), interpolation, keyframes }],+ tracks: [{ target: String(index), property: coreProperty(track), interpolation, keyframes, extrapolation: track.extrapolation, modifiers: track.modifiers }],});const result = {kind: track.kind,@@ -419,6 +420,7 @@ export function normalizeAgentMotion(input, recipe) {interpolation,keyframes: clip.tracks[0].keyframes.map(keyframe => copiedKeyframe(keyframe, interpolation)),};+ if (clip.tracks[0].effects) Object.assign(result, trackEffectsDescription(clip.tracks[0].effects));if (!SINGLETON_KINDS.has(track.kind)) {result.id = track.id;}@@ -634,6 +636,8 @@ export function compileAgentMotion(input, recipe) {property: coreProperty(track),interpolation: track.interpolation,keyframes: track.keyframes,+ extrapolation: track.extrapolation,+ modifiers: track.modifiers,})),});const sampler = compileAnimationSampler(clip);@@ -706,7 +710,7 @@ export function compileAgentMotion(input, recipe) {loop: motion.loop,};for (const track of cameraTracks) {- cameraInput[track.property] = { interpolation: track.interpolation, keyframes: track.keyframes };+ cameraInput[track.property] = { interpolation: track.interpolation, keyframes: track.keyframes, extrapolation: track.extrapolation, modifiers: track.modifiers };}const cameraPath = cameraTracks.length ? createCameraPath(cameraInput) : null;const stagedCamera = cameraPath ? { eye: new Float32Array(3), target: new Float32Array(3), up: new Float32Array(3) } : null;diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.jsindex 72952b94..17c97350 100644--- a/src/agent-scene-capabilities.js+++ b/src/agent-scene-capabilities.js@@ -167,22 +167,22 @@ export const MESH_ATTRIBUTE_NOTES = Object.freeze({// Shape keys on a recipe mesh. Details stay here so the tool schemas stay small.export const MESH_SHAPE_KEY_NOTES = Object.freeze({- shapeKeys: 'On a mesh geometry, at most 16: { name, offsets, value } or { name, edits, value }. offsets is a list of { select, offset: [x,y,z], falloff } moves like the translate edit; edits are mesh edits that only move vertices run on the finished mesh, after the stack. value (0, from 0 to 1) is the weight without motion. Each key becomes a glTF morph target. Animate it with a motion track { kind: "morph", id: <key name>, property: "weight", keyframes }; a GLB export writes targets, names (mesh.extras.targetNames) and weights.',+ shapeKeys: 'On a mesh geometry, at most 16: { name, offsets, value } or { name, edits, value }. offsets is a list of { select, offset: [x,y,z], falloff } moves like the translate edit; edits are vertex-moving mesh edits run after the stack. value (0, 0 to 1) is the weight without motion. Each key becomes a glTF morph target. Animate it with a motion track { kind: "morph", id: <key name>, property: "weight", keyframes }.',});// Organic shaping edits. Details stay here so the tool schemas stay small.export const MESH_SCULPT_NOTES = Object.freeze({- sculpt: 'strokes run in order on the vertices of the selected faces; other vertices stay, so a tag confines them. A stroke has type inflate (along the vertex normal), displace (along direction, needed) or smooth (strength 0 to 1), center and radius, strength in mesh units (negative pushes in). Falloff is smooth to zero at the radius.',- shrinkwrap: 'target is a primitive record placed by translation, rotation and scale. mode nearest (default) moves each selected vertex to the closest point; project moves it along direction (needed) to the first hit. offset (default 0) stands it off along the normal, on the side it came from. maxDistance limits the search; onMiss error (default) or keep. For a decal, use enough grid cells to follow the curve, then solidify.',+ sculpt: 'strokes run in order on the vertices of the selected faces; a tag confines them. A stroke has type inflate (along the vertex normal), displace (along direction, needed) or smooth (strength 0 to 1), center and radius, strength in mesh units (negative pushes in). Falloff is smooth to zero at the radius.',+ shrinkwrap: 'target is a primitive record placed by translation, rotation and scale. mode nearest (default) moves each selected vertex to the closest point; project moves it along direction (needed) to the first hit. offset (0) stands it off along the normal. maxDistance limits the search; onMiss error (default) or keep. For a decal, use enough grid cells, then solidify.',});export const AGENT_SCENE_CAPABILITIES = Object.freeze({format: '67js.scene', schemaVersion: 1,geometryTypes: Object.freeze(['box', 'cylinder', 'sphere', 'cone', 'icoSphere', 'torus', 'lathe', 'extrude', 'tube', 'curve', 'text', 'grid', 'plane', 'circle', 'line', 'terrain', 'mesh']),- camera: 'eye, target, up, near, far, then fov (vertical degrees) or focalLength (mm, with sensorWidth mm, default 36; not both). projection orthographic with orthoScale (view height in m, default 4). shift [x, y] moves the frame by a fraction of its width and height. fStop (0.7 to 64) turns on depth of field focused on target; 1.4 or lower is full blur, and effects.depthOfField.blur must stay 0. Orthographic and shifted cameras render with the rasterizer, not the path tracer.',- terrain: 'One mesh, x and z from -size/2 to size/2, hills about y = 0. size 40 m (1-2000), resolution 64 cells (2-256), seed 1, amplitude 1 m (0-200), holes up to 16 of {center:[x,z],radius} with no surface inside; radius a few cells or more.',- placedAssets: 'Node field asset:{id,materials?,clip?,drive?} turns a group node into a catalog GLB (id from asset_catalog). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start on a decoded copy of its own; set-motion tracks move the node and scene time seeks the clip. drive:{path | legs, cruiseSpeed | speed, wheels, lights} drives a car along a path in the placement frame on a decoded copy (fields as motion.drive in an asset recipe, docs/vehicle.md). scene_inspect assets adds wheels, rejectedNames and light signals. At most 512 placements, 16 assets, 16 playing a clip or driving.',- navigation: 'scene_inspect path:{ground,from:[x,z],to:[x,z]} returns a route over the ground node around its holes, steep slopes and the nodes in avoid:[ids] (box footprints), as keyframes [{time,value:[x,y,z]}] for a translation track with interpolation LINEAR. radius 0.3 m keeps the walker off holes and avoided nodes; maxSlope 40 degrees; cellSize defaults to radius; give seconds for the whole walk or speed (1.4 m/s); lift raises y.',+ camera: 'eye, target, up, near, far, then fov (vertical degrees) or focalLength (mm, with sensorWidth mm, default 36; not both). projection orthographic with orthoScale (view height in m, default 4). shift [x, y] moves the frame by a fraction of its size. fStop (0.7 to 64) turns on depth of field focused on target; 1.4 or lower is full blur, and effects.depthOfField.blur must stay 0. Orthographic and shifted cameras render with the rasterizer, not the path tracer.',+ terrain: 'One mesh, x and z from -size/2 to size/2, hills about y = 0. size 40 m (1-2000), resolution 64 cells (2-256), seed 1, amplitude 1 m (0-200), holes up to 16 of {center:[x,z],radius}, no surface inside; radius a few cells or more.',+ placedAssets: 'Node field asset:{id,materials?,clip?,drive?} turns a group node into a catalog GLB (id from asset_catalog). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start; set-motion tracks move the node and scene time seeks the clip. drive:{path | legs, cruiseSpeed | speed, wheels, lights} drives a car along a path in the placement frame (docs/vehicle.md). At most 512 placements, 16 assets, 16 animated.',+ navigation: 'scene_inspect path:{ground,from:[x,z],to:[x,z]} returns a route over the ground node around its holes, steep slopes and the nodes in avoid:[ids] (box footprints), as keyframes [{time,value:[x,y,z]}] for a translation track with interpolation LINEAR. radius 0.3 m; maxSlope 40 degrees; cellSize defaults to radius; seconds or speed (1.4 m/s); lift adds y.',meshEdits: Object.freeze({ operations: Object.freeze(Object.keys(MESH_EDIT_OPERATIONS)), maxEdits: MAX_MESH_EDITS }),meshStack: Object.freeze({ operations: STACK_OPERATIONS, maxNodes: MAX_STACK_NODES }),meshModifiers: MESH_MODIFIER_NOTES,@@ -308,8 +308,9 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({sun: 'direction, color, intensity (lightDirection, keyColor, keyIntensity)',environment: 'skyTop, skyBottom, fogColor, fogDensity, volumeFogDensity, ambientIntensity, fillIntensity, environmentIntensity, environmentRotation',effects: 'exposure, focusDistance (needs effects.depthOfField.blur above 0; an fStop camera focuses on its target)',- note: 'Same keyframes and easing as other tracks. After sampleMotion, scene.environment and scene.effects hold the sampled values; the recipe keeps its authored ones. Baked light (bake) does not follow.',+ note: 'Same keyframes and easing as other tracks. After sampleMotion, scene.environment and scene.effects hold the sampled values.',}),+ motionModifiers: 'Track extrapolation constant|linear; modifiers [{type: cycles|stepped|noise|generator|limits|envelope, ...}], 8 at most; cycles and stepped remap time first.',motionLimits: Object.freeze({ maxTracks: 128, maxKeyframesPerTrack: 64, maxKeyframes: 2048, maxDuration: 30 }),});diff --git a/src/animation-modifiers.js b/src/animation-modifiers.jsnew file mode 100644index 00000000..5d65ebdf--- /dev/null+++ b/src/animation-modifiers.js@@ -0,0 +1,250 @@+/**+ * Track modifiers and extrapolation, after Blender's F-curve modifiers (fmodifier.cc) and extrapolation (fcurve.cc).+ *+ * A track samples in three steps. Time modifiers (`cycles`, `stepped`) run in listed order and remap the time. The keyframes+ * are sampled at the remapped time, extended past the first and last keyframe by the track `extrapolation`. Value modifiers+ * (`noise`, `generator`, `limits`, `envelope`) then run in listed order on the sampled value. Every step is a pure function of+ * the track and the time, so seeking backwards and rendering give the same bytes.+ */++import { registerTrackEffects } from './animation.js';++const CYCLE_MODES = ['none', 'repeat', 'offset', 'mirror'];+const TIME_TYPES = new Set(['cycles', 'stepped']);+const MAXIMUM_MODIFIERS = 8;+const MAXIMUM_POLYNOMIAL_TERMS = 8;+const MAXIMUM_ENVELOPE_POINTS = 16;++function finiteNumber(value, field, fallback) {+ const result = value ?? fallback;+ if (!Number.isFinite(result)) throw new RangeError(`Modifier ${field} must be a finite number, for example ${fallback}.`);+ return result;+}++function checkKeys(modifier, allowed) {+ for (const key of Object.keys(modifier)) {+ if (key !== 'type' && !allowed.includes(key)) throw new RangeError(`Modifier ${modifier.type} has no field ${key}. Fields: ${allowed.join(', ')}.`);+ }+}++function cycleCount(value, field) {+ const count = value ?? 0;+ if (!Number.isInteger(count) || count < 0 || count > 1000) throw new RangeError(`Modifier ${field} must be an integer from 0 (forever) to 1000.`);+ return count;+}++function normalizeCycles(modifier, property) {+ checkKeys(modifier, ['before', 'after', 'beforeCount', 'afterCount']);+ const before = modifier.before ?? 'repeat';+ const after = modifier.after ?? 'repeat';+ for (const [field, mode] of [['before', before], ['after', after]]) {+ if (!CYCLE_MODES.includes(mode)) throw new RangeError(`Modifier cycles ${field} must be one of ${CYCLE_MODES.join(', ')}, not ${mode}.`);+ if (mode === 'offset' && property === 'rotation') throw new RangeError('Modifier cycles offset does not apply to rotation. Use repeat or mirror.');+ }+ return { type: 'cycles', before, after, beforeCount: cycleCount(modifier.beforeCount, 'cycles beforeCount'), afterCount: cycleCount(modifier.afterCount, 'cycles afterCount') };+}++function normalizeStepped(modifier) {+ checkKeys(modifier, ['step', 'offset']);+ const step = finiteNumber(modifier.step, 'stepped step', 0.1);+ if (step <= 0) throw new RangeError('Modifier stepped step must be above 0 seconds, for example 0.1.');+ return { type: 'stepped', step, offset: finiteNumber(modifier.offset, 'stepped offset', 0) };+}++function normalizeNoise(modifier) {+ checkKeys(modifier, ['scale', 'strength', 'phase', 'seed']);+ const scale = finiteNumber(modifier.scale, 'noise scale', 1);+ if (scale <= 0) throw new RangeError('Modifier noise scale is seconds per wobble and must be above 0, for example 0.25.');+ const seed = modifier.seed ?? 0;+ if (!Number.isInteger(seed) || Math.abs(seed) > 2 ** 31) throw new RangeError('Modifier noise seed must be an integer, for example 7.');+ return { type: 'noise', scale, strength: finiteNumber(modifier.strength, 'noise strength', 1), phase: finiteNumber(modifier.phase, 'noise phase', 0), seed };+}++function normalizeGenerator(modifier) {+ checkKeys(modifier, ['coefficients', 'additive']);+ const coefficients = modifier.coefficients ?? [0, 1];+ if (!Array.isArray(coefficients) || !coefficients.length || coefficients.length > MAXIMUM_POLYNOMIAL_TERMS || !coefficients.every(Number.isFinite)) {+ throw new RangeError(`Modifier generator coefficients are 1 to ${MAXIMUM_POLYNOMIAL_TERMS} finite numbers, lowest power first, for example [0, 1].`);+ }+ if (modifier.additive !== undefined && typeof modifier.additive !== 'boolean') throw new RangeError('Modifier generator additive must be true or false.');+ return { type: 'generator', coefficients: [...coefficients], additive: modifier.additive ?? false };+}++function normalizeLimits(modifier) {+ checkKeys(modifier, ['min', 'max']);+ if (modifier.min === undefined && modifier.max === undefined) throw new RangeError('Modifier limits needs min, max or both, for example {min:0,max:1}.');+ const result = { type: 'limits' };+ if (modifier.min !== undefined) result.min = finiteNumber(modifier.min, 'limits min', 0);+ if (modifier.max !== undefined) result.max = finiteNumber(modifier.max, 'limits max', 1);+ if (result.min !== undefined && result.max !== undefined && result.min > result.max) throw new RangeError('Modifier limits min must not exceed max.');+ return result;+}++function normalizeEnvelope(modifier) {+ checkKeys(modifier, ['points', 'referenceMin', 'referenceMax']);+ const points = modifier.points;+ if (!Array.isArray(points) || !points.length || points.length > MAXIMUM_ENVELOPE_POINTS) {+ throw new RangeError(`Modifier envelope points are 1 to ${MAXIMUM_ENVELOPE_POINTS} of {time,min,max}, for example [{time:0,min:-1,max:1}].`);+ }+ const normalized = points.map(point => ({+ time: finiteNumber(point?.time, 'envelope point time', 0),+ min: finiteNumber(point?.min, 'envelope point min', -1),+ max: finiteNumber(point?.max, 'envelope point max', 1),+ })).sort((first, second) => first.time - second.time);+ for (let index = 1; index < normalized.length; index++) {+ if (normalized[index].time === normalized[index - 1].time) throw new RangeError('Modifier envelope point times must be distinct.');+ }+ const referenceMin = finiteNumber(modifier.referenceMin, 'envelope referenceMin', -1);+ const referenceMax = finiteNumber(modifier.referenceMax, 'envelope referenceMax', 1);+ if (referenceMax <= referenceMin) throw new RangeError('Modifier envelope referenceMax must be above referenceMin.');+ return { type: 'envelope', points: normalized, referenceMin, referenceMax };+}++const NORMALIZERS = {+ cycles: normalizeCycles,+ stepped: normalizeStepped,+ noise: normalizeNoise,+ generator: normalizeGenerator,+ limits: normalizeLimits,+ envelope: normalizeEnvelope,+};++/** Returns `{ extrapolation, timeModifiers, valueModifiers }` frozen, or null when the track has neither. */+export function normalizeTrackEffects(track) {+ const extrapolation = track.extrapolation ?? 'constant';+ if (extrapolation !== 'constant' && extrapolation !== 'linear') throw new RangeError(`Track extrapolation must be constant or linear, not ${extrapolation}.`);+ if (extrapolation === 'linear' && track.property === 'rotation') throw new RangeError('Rotation tracks cannot extrapolate linearly. Use constant, or a cycles modifier.');+ const modifiers = track.modifiers ?? [];+ if (!Array.isArray(modifiers) || modifiers.length > MAXIMUM_MODIFIERS) throw new RangeError(`Track modifiers is a list of at most ${MAXIMUM_MODIFIERS} objects like {type:'noise'}.`);+ if (extrapolation === 'constant' && !modifiers.length) return null;+ const timeModifiers = [];+ const valueModifiers = [];+ for (const modifier of modifiers) {+ const normalizer = NORMALIZERS[modifier?.type];+ if (!normalizer) throw new RangeError(`Unknown modifier type ${modifier?.type}. Types: ${Object.keys(NORMALIZERS).join(', ')}.`);+ if (!TIME_TYPES.has(modifier.type) && track.property === 'rotation') throw new RangeError(`Rotation tracks accept only cycles and stepped modifiers, not ${modifier.type}.`);+ (TIME_TYPES.has(modifier.type) ? timeModifiers : valueModifiers).push(Object.freeze(normalizer(modifier, track.property)));+ }+ return Object.freeze({ extrapolation, timeModifiers: Object.freeze(timeModifiers), valueModifiers: Object.freeze(valueModifiers) });+}++/** The modifiers back in authored form, for recipes: time modifiers first, as they run. */+export function trackEffectsDescription(effects) {+ const description = {};+ if (effects.extrapolation !== 'constant') description.extrapolation = effects.extrapolation;+ const modifiers = [...effects.timeModifiers, ...effects.valueModifiers];+ if (modifiers.length) description.modifiers = modifiers.map(modifier => structuredClone(modifier));+ return description;+}++/** Remaps time through `cycles` and `stepped`. Returns the time and adds the per-cycle value offset multiples to `offsets`. */+export function remapTime(effects, keyframes, time, offsets) {+ const firstTime = keyframes[0].time;+ const length = keyframes[keyframes.length - 1].time - firstTime;+ let remapped = time;+ for (const modifier of effects.timeModifiers) {+ if (modifier.type === 'stepped') {+ remapped = Math.floor((remapped - modifier.offset) / modifier.step) * modifier.step + modifier.offset;+ continue;+ }+ if (length <= 0 || remapped >= firstTime && remapped <= firstTime + length) continue;+ const isBefore = remapped < firstTime;+ const mode = isBefore ? modifier.before : modifier.after;+ if (mode === 'none') continue;+ const limit = isBefore ? modifier.beforeCount : modifier.afterCount;+ let cycle = Math.floor((remapped - firstTime) / length);+ let local = remapped - firstTime - cycle * length;+ const passed = isBefore ? -cycle : cycle;+ if (limit > 0 && passed > limit) {+ cycle = isBefore ? -limit : limit;+ local = isBefore ? 0 : length;+ }+ if (mode === 'mirror' && Math.abs(cycle) % 2 === 1) local = length - local;+ if (mode === 'offset') offsets.push(cycle);+ remapped = firstTime + local;+ }+ return remapped;+}++function hashUnit(index, seed) {+ let state = Math.imul(index ^ 0x9e3779b9, 0x85ebca6b) ^ Math.imul(seed | 0, 0xc2b2ae35);+ state = Math.imul(state ^ (state >>> 15), 0x2c1b3c6d);+ state = Math.imul(state ^ (state >>> 12), 0x297a2d39);+ state ^= state >>> 15;+ return (state >>> 0) / 2147483648 - 1;+}++/** Smooth value noise in [-1, 1]; the same input always gives the same output. */+export function valueNoise(position, seed) {+ const cell = Math.floor(position);+ const fraction = position - cell;+ const blend = fraction * fraction * (3 - 2 * fraction);+ const start = hashUnit(cell, seed);+ return start + (hashUnit(cell + 1, seed) - start) * blend;+}++function envelopeBounds(points, time) {+ const last = points[points.length - 1];+ if (time <= points[0].time) return [points[0].min, points[0].max];+ if (time >= last.time) return [last.min, last.max];+ let upper = 1;+ while (points[upper].time < time) upper++;+ const start = points[upper - 1];+ const end = points[upper];+ const progress = (time - start.time) / (end.time - start.time);+ return [start.min + (end.min - start.min) * progress, start.max + (end.max - start.max) * progress];+}++/** Runs the value modifiers over `output` in place. `time` is the original, unremapped time. */+export function applyValueModifiers(effects, time, output) {+ for (const modifier of effects.valueModifiers) {+ for (let axis = 0; axis < output.length; axis++) {+ const value = output[axis];+ if (modifier.type === 'noise') {+ output[axis] = value + valueNoise((time + modifier.phase) / modifier.scale, modifier.seed + axis * 7919) * modifier.strength;+ } else if (modifier.type === 'generator') {+ let polynomial = 0;+ for (let term = modifier.coefficients.length - 1; term >= 0; term--) polynomial = polynomial * time + modifier.coefficients[term];+ output[axis] = modifier.additive ? value + polynomial : polynomial;+ } else if (modifier.type === 'limits') {+ output[axis] = Math.min(modifier.max ?? Infinity, Math.max(modifier.min ?? -Infinity, value));+ } else {+ const [minimum, maximum] = envelopeBounds(modifier.points, time);+ const progress = (value - modifier.referenceMin) / (modifier.referenceMax - modifier.referenceMin);+ output[axis] = minimum + (maximum - minimum) * progress;+ }+ }+ }+}++function componentOf(value, axis) {+ return typeof value === 'number' ? value : value[axis];+}++/** Secant slope per second between two keyframes on one axis. */+function secantSlope(from, to, axis) {+ return (componentOf(to.value, axis) - componentOf(from.value, axis)) / (to.time - from.time);+}++function sampleTrackWithEffects(track, time, output, sampleKeyframes, constrainSample) {+ const { effects, keyframes } = track;+ const offsets = [];+ const remapped = remapTime(effects, keyframes, time, offsets);+ const first = keyframes[0];+ const last = keyframes[keyframes.length - 1];+ sampleKeyframes(track, remapped, output);+ if (effects.extrapolation === 'linear' && keyframes.length > 1) {+ const slopeFrom = remapped < first.time ? first : keyframes[keyframes.length - 2];+ const slopeTo = remapped < first.time ? keyframes[1] : last;+ const distance = remapped < first.time ? remapped - first.time : remapped > last.time ? remapped - last.time : 0;+ for (let axis = 0; axis < output.length; axis++) output[axis] += secantSlope(slopeFrom, slopeTo, axis) * distance;+ }+ for (const cycle of offsets) {+ for (let axis = 0; axis < output.length; axis++) output[axis] += cycle * (componentOf(last.value, axis) - componentOf(first.value, axis));+ }+ applyValueModifiers(effects, time, output);+ for (let axis = 0; axis < output.length; axis++) output[axis] = constrainSample(track.property, output[axis]);+ return output;+}++registerTrackEffects({ normalize: normalizeTrackEffects, sample: sampleTrackWithEffects });diff --git a/src/animation.js b/src/animation.jsindex 82e5f00c..150cb89d 100644--- a/src/animation.js+++ b/src/animation.js@@ -85,7 +85,9 @@ export function createAnimationClip({ duration, loop = false, tracks = [] } = {}for (let index = 1; index < keyframes.length; index++) {if (keyframes[index].time === keyframes[index - 1].time) throw new RangeError('Keyframe times must be distinct.');}- return Object.freeze({ target: track.target, property: track.property, interpolation, keyframes: Object.freeze(keyframes) });+ const effects = track.extrapolation === undefined && track.modifiers === undefined ? null : trackEffects?.normalize(track);+ if (effects === undefined) throw new Error('Import animation-modifiers.js.');+ return Object.freeze({ target: track.target, property: track.property, interpolation, keyframes: Object.freeze(keyframes), ...(effects ? { effects } : {}) });});return Object.freeze({ duration, loop, tracks: Object.freeze(compiled) });}@@ -157,6 +159,17 @@ function sampleTrackInto(track, time, output) {return output;}+let trackEffects = null;++/** Called by animation-modifiers.js when it loads, so the modifier code stays out of bundles that never use it. */+export function registerTrackEffects(handlers) {+ trackEffects = handlers;+}++function sampleTrack(track, time, output) {+ return track.effects ? trackEffects.sample(track, time, output, sampleTrackInto, constrainSample) : sampleTrackInto(track, time, output);+}+/*** Linear keyframes that trace a track with eased segments, for formats with no easing (glTF). Each segment gets* `stepsPerSegment` linear pieces; a track without easing comes back unchanged.@@ -181,7 +194,8 @@ export function bakeEasedKeyframes(track, stepsPerSegment) {function normalizedTime(clip, time) {if (!Number.isFinite(time)) throw new TypeError('Animation time must be finite.');- if (!clip.loop) return Math.max(0, Math.min(clip.duration, time));+ // A clip that does not loop passes the time through: keyframes lie within the duration, so tracks hold their ends anyway, and extrapolation and cycles need the time past them.+ if (!clip.loop) return time;const remainder = time % clip.duration;return remainder < 0 ? remainder + clip.duration : remainder;}@@ -189,7 +203,7 @@ function normalizedTime(clip, time) {export function sampleAnimationClip(clip, time) {const clipTime = normalizedTime(clip, time);return clip.tracks.map(track => {- const output = sampleTrackInto(track, clipTime, new Float64Array(PROPERTY_SIZES[track.property]));+ const output = sampleTrack(track, clipTime, new Float64Array(PROPERTY_SIZES[track.property]));const value = output.length === 1 ? output[0] : Array.from(output);return { target: track.target, property: track.property, value };});@@ -207,7 +221,7 @@ export function compileAnimationSampler(clip) {sample(time) {const clipTime = normalizedTime(clip, time);for (let index = 0; index < tracks.length; index++) {- sampleTrackInto(clip.tracks[index], clipTime, tracks[index].output);+ sampleTrack(clip.tracks[index], clipTime, tracks[index].output);}return tracks;},@@ -263,7 +277,7 @@ export function createAnimationPlayer(clip, parts) {apply(time) {const clipTime = normalizedTime(clip, time);for (const compiled of compiledTracks) {- sampleTrackInto(compiled.track, clipTime, compiled.output);+ sampleTrack(compiled.track, clipTime, compiled.output);}for (const binding of compiledBindings) {if (binding.hasTransform) {diff --git a/tests/animation-modifiers.test.js b/tests/animation-modifiers.test.jsnew file mode 100644index 00000000..2413569e--- /dev/null+++ b/tests/animation-modifiers.test.js@@ -0,0 +1,166 @@+import assert from 'node:assert/strict';+import test from 'node:test';+import { createAnimationClip, sampleAnimationClip } from '../src/animation.js';+import { compileAgentMotion, normalizeAgentMotion } from '../src/agent-motion.js';+import { valueNoise } from '../src/animation-modifiers.js';++const rampKeyframes = [{ time: 1, value: 0 }, { time: 3, value: 4 }];++function sampler(extra, keyframes = rampKeyframes, loop = false) {+ const clip = createAnimationClip({ duration: 4, loop, tracks: [{ target: 'part', property: 'value', keyframes, ...extra }] });+ return time => sampleAnimationClip(clip, time)[0].value;+}++test('extrapolation constant holds ends and linear continues the end slopes', () => {+ const constant = sampler({ extrapolation: 'constant' });+ assert.equal(constant(-5), 0);+ assert.equal(constant(10), 4);+ const linear = sampler({ extrapolation: 'linear' });+ assert.equal(linear(0), -2);+ assert.equal(linear(5), 8);+ assert.equal(linear(2), 2);+ const single = sampler({ extrapolation: 'linear' }, [{ time: 1, value: 3 }]);+ assert.equal(single(9), 3);+ const vector = createAnimationClip({ duration: 2, tracks: [{ target: 'part', property: 'translation', extrapolation: 'linear', keyframes: [{ time: 0, value: [0, 1, 2] }, { time: 1, value: [1, 1, 0] }] }] });+ assert.deepEqual(sampleAnimationClip(vector, 3)[0].value, [3, 1, -4]);+});++test('cycles repeat, offset and mirror before and after the keys, with finite counts', () => {+ const repeat = sampler({ modifiers: [{ type: 'cycles' }] });+ assert.equal(repeat(4), 2);+ assert.equal(repeat(7), 0);+ assert.equal(repeat(-0.5), 1);+ const offset = sampler({ modifiers: [{ type: 'cycles', before: 'offset', after: 'offset' }] });+ assert.equal(offset(4), 6);+ assert.equal(offset(6), 10);+ assert.equal(offset(0), -2);+ assert.equal(offset(-1), -4);+ const mirror = sampler({ modifiers: [{ type: 'cycles', before: 'mirror', after: 'mirror' }] });+ assert.equal(mirror(3.5), 3);+ assert.equal(mirror(5), 0);+ assert.equal(mirror(7), 4);+ const counted = sampler({ modifiers: [{ type: 'cycles', after: 'offset', afterCount: 1 }] });+ assert.equal(counted(4), 6);+ assert.equal(counted(4.5), 7);+ assert.equal(counted(5.9), 8);+ assert.equal(counted(50), 8);+ const none = sampler({ modifiers: [{ type: 'cycles', before: 'none', after: 'none' }] });+ assert.equal(none(10), 4);+});++test('cycles keep going past the clip duration', () => {+ const value = sampler({ modifiers: [{ type: 'cycles', after: 'offset' }] });+ assert.equal(value(11), 20);+});++test('stepped holds the curve for each step', () => {+ const stepped = sampler({ modifiers: [{ type: 'stepped', step: 0.5 }] });+ assert.equal(stepped(1.9), 1);+ assert.equal(stepped(2.49), 2);+ assert.equal(stepped(2.5), 3);+ const shifted = sampler({ modifiers: [{ type: 'stepped', step: 1, offset: 0.25 }] });+ assert.equal(shifted(2.2), 0.5);+});++test('generator replaces or adds a polynomial of time', () => {+ const replace = sampler({ modifiers: [{ type: 'generator', coefficients: [1, 2, 3] }] });+ assert.equal(replace(2), 1 + 2 * 2 + 3 * 4);+ const additive = sampler({ modifiers: [{ type: 'generator', coefficients: [10, 0, 1], additive: true }] });+ assert.equal(additive(2), 2 + 10 + 4);+});++test('limits clamp and envelope remaps a value range over time', () => {+ const limits = sampler({ modifiers: [{ type: 'limits', min: 1, max: 3 }] });+ assert.equal(limits(0), 1);+ assert.equal(limits(2), 2);+ assert.equal(limits(3), 3);+ const wave = [{ time: 0, value: -1 }, { time: 4, value: 1 }];+ const envelope = sampler({ modifiers: [{ type: 'envelope', points: [{ time: 0, min: 0, max: 2 }, { time: 4, min: -2, max: 6 }] }] }, wave);+ assert.equal(envelope(0), 0);+ assert.equal(envelope(4), 6);+ assert.equal(envelope(2), 1.5);+});++test('noise is seeded, bounded, smooth and independent of the order of seeks', () => {+ const noisy = sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 7 }] });+ const forward = [0, 0.1, 0.37, 1, 2.2, 3, 3.9].map(noisy);+ const backward = [3.9, 3, 2.2, 1, 0.37, 0.1, 0].map(noisy).reverse();+ assert.deepEqual(forward, backward);+ const plain = sampler({});+ for (const time of [0.1, 1.5, 2.2, 3.5]) assert.ok(Math.abs(noisy(time) - plain(time)) <= 0.5);+ const reseeded = sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 8 }] });+ assert.notEqual(noisy(2.2), reseeded(2.2));+ assert.equal(valueNoise(3, 1), valueNoise(3, 1));+ const step = Math.abs(valueNoise(1.001, 5) - valueNoise(1, 5));+ assert.ok(step < 0.01);+ assert.equal(noisy(2.2), sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 7 }] })(2.2));+});++test('noise differs per axis and a pinned sample stays byte stable', () => {+ const clip = createAnimationClip({ duration: 1, tracks: [{ target: 'part', property: 'translation', modifiers: [{ type: 'noise', scale: 0.1, strength: 1, seed: 3 }], keyframes: [{ time: 0, value: [0, 0, 0] }] }] });+ const [x, y, z] = sampleAnimationClip(clip, 0.33)[0].value;+ assert.notEqual(x, y);+ assert.notEqual(y, z);+ assert.deepEqual(sampleAnimationClip(clip, 0.33)[0].value, [x, y, z]);+});++test('modifiers run time steps first, then value steps, and compose with extrapolation', () => {+ const composed = sampler({ extrapolation: 'linear', modifiers: [{ type: 'limits', max: 6 }, { type: 'stepped', step: 1 }] });+ assert.equal(composed(5.9), 6);+ assert.equal(composed(5.4), 6);+ assert.equal(composed(3.9), 4);+});++test('rotation tracks take only repeat or mirror cycles and stepped', () => {+ const rotation = { target: 'part', property: 'rotation', keyframes: [{ time: 0, value: [0, 0, 0, 1] }, { time: 1, value: [0, 1, 0, 1] }] };+ const clip = createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'cycles', after: 'mirror' }] }] });+ const first = sampleAnimationClip(clip, 0.25)[0].value;+ const mirrored = sampleAnimationClip(clip, 1.75)[0].value;+ first.forEach((component, index) => assert.ok(Math.abs(component - mirrored[index]) < 1e-12));+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, extrapolation: 'linear' }] }), /Rotation tracks cannot extrapolate/);+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'noise' }] }] }), /only cycles and stepped/);+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'cycles', after: 'offset' }] }] }), /does not apply to rotation/);+});++test('invalid modifiers name the field and an accepted example', () => {+ const bad = modifiers => () => sampler({ modifiers });+ assert.throws(bad([{ type: 'wiggle' }]), /Unknown modifier type wiggle/);+ assert.throws(bad([{ type: 'noise', scale: 0 }]), /noise scale/);+ assert.throws(bad([{ type: 'noise', speed: 1 }]), /no field speed/);+ assert.throws(bad([{ type: 'stepped', step: 0 }]), /stepped step/);+ assert.throws(bad([{ type: 'limits' }]), /min, max or both/);+ assert.throws(bad([{ type: 'limits', min: 2, max: 1 }]), /must not exceed/);+ assert.throws(bad([{ type: 'generator', coefficients: [] }]), /coefficients/);+ assert.throws(bad([{ type: 'envelope', points: [] }]), /envelope points/);+ assert.throws(bad([{ type: 'cycles', after: 'spin' }]), /cycles after/);+ assert.throws(() => sampler({ extrapolation: 'cubic' }), /constant or linear/);+});++const recipe = { nodes: [{ id: 'cube' }], camera: { eye: [0, 0, 5], target: [0, 0, 0], up: [0, 1, 0], fov: 45, near: 0.1, far: 100 } };++test('recipe motion tracks keep modifiers, sample them at the same times in every sampler and seek backwards', () => {+ const input = {+ duration: 2,+ tracks: [+ { kind: 'node', id: 'cube', property: 'translation', extrapolation: 'linear',+ modifiers: [{ type: 'noise', scale: 0.2, strength: 0.1, seed: 4 }],+ keyframes: [{ time: 0, value: [0, 0, 0] }, { time: 1, value: [1, 0, 0] }] },+ { kind: 'camera', property: 'eye', modifiers: [{ type: 'cycles', after: 'offset' }],+ keyframes: [{ time: 0, value: [0, 0, 5] }, { time: 1, value: [1, 0, 5] }] },+ ],+ };+ const normalized = normalizeAgentMotion(input, recipe);+ assert.equal(normalized.tracks[0].extrapolation, 'linear');+ assert.deepEqual(normalized.tracks[0].modifiers[0], { type: 'noise', scale: 0.2, strength: 0.1, phase: 0, seed: 4 });+ assert.deepEqual(normalizeAgentMotion(normalized, recipe), normalized);+ const compiled = compileAgentMotion(input, recipe);+ const read = time => {+ const output = compiled.sample(time);+ return { cube: Array.from(output.nodes.get('cube').translation), eye: Array.from(output.camera.eye) };+ };+ const forward = [0.3, 1.5, 2.5].map(read);+ const backward = [2.5, 1.5, 0.3].map(read).reverse();+ assert.deepEqual(forward, backward);+ assert.ok(Math.abs(forward[1].cube[0] - 1.5) <= 0.1);+ assert.ok(Math.abs(forward[2].eye[0] - 2.5) < 1e-6);+});