Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add F-curve modifiers and extrapolation to motion tracks (issue #690)

Nguyen Xuan An6405ac7b3a52
diff --git a/changes/690-fcurve-modifiers.md b/changes/690-fcurve-modifiers.md
new file mode 100644
index 00000000..bd986ac8
--- /dev/null
+++ b/changes/690-fcurve-modifiers.md
@@ -0,0 +1 @@
+- Motion tracks take `extrapolation` (`constant` or `linear`) and `modifiers`: `cycles` (repeat, offset, mirror, with counts), `stepped`, `noise` (seeded), `generator`, `limits` and `envelope`, sampled the same in sampleMotion, camera paths and render jobs (#690)
diff --git a/changes/690-fcurve-modifiers.size.md b/changes/690-fcurve-modifiers.size.md
new file mode 100644
index 00000000..e389732e
--- /dev/null
+++ b/changes/690-fcurve-modifiers.size.md
@@ -0,0 +1,2 @@
+- entry:agent-asset: track modifiers and extrapolation (noise, cycles, envelope and four more) load with the motion code, which agents use for camera shake and looping walks
+- scene:agent-scene+lazy: the same lazy motion code; the eager agent-scene bundle stays within its allowance
diff --git a/docs/animation-curves.md b/docs/animation-curves.md
index d9f8a780..bd9e5a9f 100644
--- a/docs/animation-curves.md
+++ b/docs/animation-curves.md
@@ -82,3 +82,31 @@ Transform-only targets retain their original material objects.
Explicit tangent data keeps the existing track model and permits deterministic sampling without external callback state.
This unit does not add a mixer, skeletal animation, morph targets or glTF importer changes.
+
+## Extrapolation and modifiers
+
+A track in a recipe `motion` takes `extrapolation` and `modifiers`, after Blender's F-curve modifiers. Every sampler reads them the same way: `sampleMotion`, camera paths and render jobs. Each result depends only on the track and the time, so a seek backwards and a render give the same values. A clip that does not loop passes times past its duration to the tracks, so extrapolation and cycles work there; a looping clip wraps the time first.
+
+- `extrapolation`: `constant` (default) holds the first and last value. `linear` continues the slope of the first or last segment. Not for rotation.
+- Time modifiers run first, in listed order. `cycles` takes `before` and `after` (`repeat`, `offset`, `mirror` or `none`, default `repeat`) and `beforeCount` and `afterCount` (0 is forever; after the count the curve holds). `offset` adds the change from the first to the last key each cycle. `stepped` takes `step` seconds and `offset` and holds the curve for each step.
+- Value modifiers run next, in listed order, on every component. `noise` adds smooth seeded noise: `scale` seconds per wobble, `strength`, `phase`, `seed`; each component gets its own noise. `generator` takes `coefficients` of a polynomial of time, lowest power first, and replaces the value, or adds with `additive: true`. `limits` clamps to `min` and `max`. `envelope` maps `referenceMin` to `referenceMax` (default -1 to 1) onto a `min` and `max` that change over `points: [{time, min, max}]`.
+- Rotation tracks take only `cycles` (not `offset`) and `stepped`. At most 8 modifiers a track. Bad fields name the accepted ones.
+- Library use: `createAnimationClip` accepts the same fields once `animation-modifiers.js` is imported; GLB export writes the keyframes only.
+
+Camera shake: hold a camera still and add noise to its eye, with a faster, weaker noise on the target.
+
+```json
+{
+ "duration": 4,
+ "tracks": [
+ { "kind": "camera", "property": "eye",
+ "keyframes": [{ "time": 0, "value": [0, 1.6, 5] }],
+ "modifiers": [{ "type": "noise", "scale": 0.12, "strength": 0.03, "seed": 11 }] },
+ { "kind": "camera", "property": "target",
+ "keyframes": [{ "time": 0, "value": [0, 1.5, 0] }],
+ "modifiers": [{ "type": "noise", "scale": 0.07, "strength": 0.015, "seed": 12 }] }
+ ]
+}
+```
+
+A walk that travels: two keys on `translation` with `{ "type": "cycles", "before": "offset", "after": "offset" }` carry on forward past the last key by one stride per cycle. Stop motion: `{ "type": "stepped", "step": 0.0833 }` samples at 12 frames a second.
diff --git a/docs/parity.md b/docs/parity.md
index 8e15b374..a45e9ae0 100644
--- a/docs/parity.md
+++ b/docs/parity.md
@@ -33,6 +33,7 @@ To move to a new Blender release, run `node scripts/parity-manual-snapshot.mjs`.
| Manual path | Status | What exists, or why not | Evidence | Issues |
| --- | --- | --- | --- | --- |
| User Interface | out of scope | Interface for a person with a mouse. Agents use MCP tools. | | |
+| Editors > Graph Editor > F-Curves | partial | Per track extrapolation `constant` or `linear` (slope of the end segment, not for rotation) and up to 8 modifiers: `cycles` (repeat, offset, mirror, counts), `stepped`, `noise`, `generator`, `limits` and `envelope`, seeded and the same at every sample time. Noise is smooth value noise, not Blender's, so values differ; no blend modes, no frame range or influence per modifier, no Bezier-handle slope, and glTF export keeps only the keyframes. The editor itself is out of scope. | [animation-modifiers](../tests/animation-modifiers.test.js) | #690 |
| Editors | out of scope | Interface for a person with a mouse. Agents use MCP tools, and people review images. | | |
| Scenes & Objects > Scenes | partial | One recipe holds one scene with its camera and environment. There are no scene units or linked scenes. | | |
| Scenes & Objects > Objects > Object Types | partial | Mesh nodes and empty nodes exist, and `curve` and `text` geometry make curve-like and text objects as meshes. Lights are environment settings. Metaballs and NURBS surfaces are out of scope. | [curve-text](../tests/curve-text.test.js), [Blender comparison](../tests/blender-curve-text.test.js) | #45 |
diff --git a/src/agent-motion.js b/src/agent-motion.js
index 285e7fc2..6ef4a62b 100644
--- a/src/agent-motion.js
+++ b/src/agent-motion.js
@@ -1,4 +1,5 @@
import { createAnimationClip, compileAnimationSampler } from './animation.js';
+import { trackEffectsDescription } from './animation-modifiers.js';
import { invalidField, withFieldPrefix } from './field-error.js';
import { createCameraPath } from './camera-path.js';
import { CAMERA_LENS_FIELDS } from './camera.js';
@@ -374,7 +375,7 @@ export function normalizeAgentMotion(input, recipe) {
const identities = new Set();
let keyframeCount = 0;
const tracks = inputTracks.map((track, index) => {
- knownKeys(track, ['kind', 'id', 'property', 'interpolation', 'keyframes'], 'motion track');
+ knownKeys(track, ['kind', 'id', 'property', 'interpolation', 'keyframes', 'extrapolation', 'modifiers'], 'motion track');
if (!Object.hasOwn(PROPERTIES, track.kind) || !Object.hasOwn(PROPERTIES[track.kind], track.property)) {
const known = Object.hasOwn(PROPERTIES, track.kind) ? ` ${track.kind} properties: ${Object.keys(PROPERTIES[track.kind]).join(', ')}.` : ` Kinds: ${Object.keys(PROPERTIES).join(', ')}.`;
throw invalidField(`motion.tracks[${index}].property`, `${track.kind}.${track.property}`, `a kind and property pair.${known}`);
@@ -411,7 +412,7 @@ export function normalizeAgentMotion(input, recipe) {
const keyframes = track.keyframes.map((keyframe, keyframeIndex) => withFieldPrefix(`motion.tracks[${index}].keyframes[${keyframeIndex}]`, () => normalizedKeyframe(keyframe, track, duration, interpolation)));
const clip = createAnimationClip({
duration, loop,
- tracks: [{ target: String(index), property: coreProperty(track), interpolation, keyframes }],
+ tracks: [{ target: String(index), property: coreProperty(track), interpolation, keyframes, extrapolation: track.extrapolation, modifiers: track.modifiers }],
});
const result = {
kind: track.kind,
@@ -419,6 +420,7 @@ export function normalizeAgentMotion(input, recipe) {
interpolation,
keyframes: clip.tracks[0].keyframes.map(keyframe => copiedKeyframe(keyframe, interpolation)),
};
+ if (clip.tracks[0].effects) Object.assign(result, trackEffectsDescription(clip.tracks[0].effects));
if (!SINGLETON_KINDS.has(track.kind)) {
result.id = track.id;
}
@@ -634,6 +636,8 @@ export function compileAgentMotion(input, recipe) {
property: coreProperty(track),
interpolation: track.interpolation,
keyframes: track.keyframes,
+ extrapolation: track.extrapolation,
+ modifiers: track.modifiers,
})),
});
const sampler = compileAnimationSampler(clip);
@@ -706,7 +710,7 @@ export function compileAgentMotion(input, recipe) {
loop: motion.loop,
};
for (const track of cameraTracks) {
- cameraInput[track.property] = { interpolation: track.interpolation, keyframes: track.keyframes };
+ cameraInput[track.property] = { interpolation: track.interpolation, keyframes: track.keyframes, extrapolation: track.extrapolation, modifiers: track.modifiers };
}
const cameraPath = cameraTracks.length ? createCameraPath(cameraInput) : null;
const stagedCamera = cameraPath ? { eye: new Float32Array(3), target: new Float32Array(3), up: new Float32Array(3) } : null;
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index 72952b94..17c97350 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -167,22 +167,22 @@ export const MESH_ATTRIBUTE_NOTES = Object.freeze({
// Shape keys on a recipe mesh. Details stay here so the tool schemas stay small.
export const MESH_SHAPE_KEY_NOTES = Object.freeze({
- shapeKeys: 'On a mesh geometry, at most 16: { name, offsets, value } or { name, edits, value }. offsets is a list of { select, offset: [x,y,z], falloff } moves like the translate edit; edits are mesh edits that only move vertices run on the finished mesh, after the stack. value (0, from 0 to 1) is the weight without motion. Each key becomes a glTF morph target. Animate it with a motion track { kind: "morph", id: <key name>, property: "weight", keyframes }; a GLB export writes targets, names (mesh.extras.targetNames) and weights.',
+ shapeKeys: 'On a mesh geometry, at most 16: { name, offsets, value } or { name, edits, value }. offsets is a list of { select, offset: [x,y,z], falloff } moves like the translate edit; edits are vertex-moving mesh edits run after the stack. value (0, 0 to 1) is the weight without motion. Each key becomes a glTF morph target. Animate it with a motion track { kind: "morph", id: <key name>, property: "weight", keyframes }.',
});
// Organic shaping edits. Details stay here so the tool schemas stay small.
export const MESH_SCULPT_NOTES = Object.freeze({
- sculpt: 'strokes run in order on the vertices of the selected faces; other vertices stay, so a tag confines them. A stroke has type inflate (along the vertex normal), displace (along direction, needed) or smooth (strength 0 to 1), center and radius, strength in mesh units (negative pushes in). Falloff is smooth to zero at the radius.',
- shrinkwrap: 'target is a primitive record placed by translation, rotation and scale. mode nearest (default) moves each selected vertex to the closest point; project moves it along direction (needed) to the first hit. offset (default 0) stands it off along the normal, on the side it came from. maxDistance limits the search; onMiss error (default) or keep. For a decal, use enough grid cells to follow the curve, then solidify.',
+ sculpt: 'strokes run in order on the vertices of the selected faces; a tag confines them. A stroke has type inflate (along the vertex normal), displace (along direction, needed) or smooth (strength 0 to 1), center and radius, strength in mesh units (negative pushes in). Falloff is smooth to zero at the radius.',
+ shrinkwrap: 'target is a primitive record placed by translation, rotation and scale. mode nearest (default) moves each selected vertex to the closest point; project moves it along direction (needed) to the first hit. offset (0) stands it off along the normal. maxDistance limits the search; onMiss error (default) or keep. For a decal, use enough grid cells, then solidify.',
});
export const AGENT_SCENE_CAPABILITIES = Object.freeze({
format: '67js.scene', schemaVersion: 1,
geometryTypes: Object.freeze(['box', 'cylinder', 'sphere', 'cone', 'icoSphere', 'torus', 'lathe', 'extrude', 'tube', 'curve', 'text', 'grid', 'plane', 'circle', 'line', 'terrain', 'mesh']),
- camera: 'eye, target, up, near, far, then fov (vertical degrees) or focalLength (mm, with sensorWidth mm, default 36; not both). projection orthographic with orthoScale (view height in m, default 4). shift [x, y] moves the frame by a fraction of its width and height. fStop (0.7 to 64) turns on depth of field focused on target; 1.4 or lower is full blur, and effects.depthOfField.blur must stay 0. Orthographic and shifted cameras render with the rasterizer, not the path tracer.',
- terrain: 'One mesh, x and z from -size/2 to size/2, hills about y = 0. size 40 m (1-2000), resolution 64 cells (2-256), seed 1, amplitude 1 m (0-200), holes up to 16 of {center:[x,z],radius} with no surface inside; radius a few cells or more.',
- placedAssets: 'Node field asset:{id,materials?,clip?,drive?} turns a group node into a catalog GLB (id from asset_catalog). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start on a decoded copy of its own; set-motion tracks move the node and scene time seeks the clip. drive:{path | legs, cruiseSpeed | speed, wheels, lights} drives a car along a path in the placement frame on a decoded copy (fields as motion.drive in an asset recipe, docs/vehicle.md). scene_inspect assets adds wheels, rejectedNames and light signals. At most 512 placements, 16 assets, 16 playing a clip or driving.',
- navigation: 'scene_inspect path:{ground,from:[x,z],to:[x,z]} returns a route over the ground node around its holes, steep slopes and the nodes in avoid:[ids] (box footprints), as keyframes [{time,value:[x,y,z]}] for a translation track with interpolation LINEAR. radius 0.3 m keeps the walker off holes and avoided nodes; maxSlope 40 degrees; cellSize defaults to radius; give seconds for the whole walk or speed (1.4 m/s); lift raises y.',
+ camera: 'eye, target, up, near, far, then fov (vertical degrees) or focalLength (mm, with sensorWidth mm, default 36; not both). projection orthographic with orthoScale (view height in m, default 4). shift [x, y] moves the frame by a fraction of its size. fStop (0.7 to 64) turns on depth of field focused on target; 1.4 or lower is full blur, and effects.depthOfField.blur must stay 0. Orthographic and shifted cameras render with the rasterizer, not the path tracer.',
+ terrain: 'One mesh, x and z from -size/2 to size/2, hills about y = 0. size 40 m (1-2000), resolution 64 cells (2-256), seed 1, amplitude 1 m (0-200), holes up to 16 of {center:[x,z],radius}, no surface inside; radius a few cells or more.',
+ placedAssets: 'Node field asset:{id,materials?,clip?,drive?} turns a group node into a catalog GLB (id from asset_catalog). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start; set-motion tracks move the node and scene time seeks the clip. drive:{path | legs, cruiseSpeed | speed, wheels, lights} drives a car along a path in the placement frame (docs/vehicle.md). At most 512 placements, 16 assets, 16 animated.',
+ navigation: 'scene_inspect path:{ground,from:[x,z],to:[x,z]} returns a route over the ground node around its holes, steep slopes and the nodes in avoid:[ids] (box footprints), as keyframes [{time,value:[x,y,z]}] for a translation track with interpolation LINEAR. radius 0.3 m; maxSlope 40 degrees; cellSize defaults to radius; seconds or speed (1.4 m/s); lift adds y.',
meshEdits: Object.freeze({ operations: Object.freeze(Object.keys(MESH_EDIT_OPERATIONS)), maxEdits: MAX_MESH_EDITS }),
meshStack: Object.freeze({ operations: STACK_OPERATIONS, maxNodes: MAX_STACK_NODES }),
meshModifiers: MESH_MODIFIER_NOTES,
@@ -308,8 +308,9 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
sun: 'direction, color, intensity (lightDirection, keyColor, keyIntensity)',
environment: 'skyTop, skyBottom, fogColor, fogDensity, volumeFogDensity, ambientIntensity, fillIntensity, environmentIntensity, environmentRotation',
effects: 'exposure, focusDistance (needs effects.depthOfField.blur above 0; an fStop camera focuses on its target)',
- note: 'Same keyframes and easing as other tracks. After sampleMotion, scene.environment and scene.effects hold the sampled values; the recipe keeps its authored ones. Baked light (bake) does not follow.',
+ note: 'Same keyframes and easing as other tracks. After sampleMotion, scene.environment and scene.effects hold the sampled values.',
}),
+ motionModifiers: 'Track extrapolation constant|linear; modifiers [{type: cycles|stepped|noise|generator|limits|envelope, ...}], 8 at most; cycles and stepped remap time first.',
motionLimits: Object.freeze({ maxTracks: 128, maxKeyframesPerTrack: 64, maxKeyframes: 2048, maxDuration: 30 }),
});
diff --git a/src/animation-modifiers.js b/src/animation-modifiers.js
new file mode 100644
index 00000000..5d65ebdf
--- /dev/null
+++ b/src/animation-modifiers.js
@@ -0,0 +1,250 @@
+/**
+ * Track modifiers and extrapolation, after Blender's F-curve modifiers (fmodifier.cc) and extrapolation (fcurve.cc).
+ *
+ * A track samples in three steps. Time modifiers (`cycles`, `stepped`) run in listed order and remap the time. The keyframes
+ * are sampled at the remapped time, extended past the first and last keyframe by the track `extrapolation`. Value modifiers
+ * (`noise`, `generator`, `limits`, `envelope`) then run in listed order on the sampled value. Every step is a pure function of
+ * the track and the time, so seeking backwards and rendering give the same bytes.
+ */
+
+import { registerTrackEffects } from './animation.js';
+
+const CYCLE_MODES = ['none', 'repeat', 'offset', 'mirror'];
+const TIME_TYPES = new Set(['cycles', 'stepped']);
+const MAXIMUM_MODIFIERS = 8;
+const MAXIMUM_POLYNOMIAL_TERMS = 8;
+const MAXIMUM_ENVELOPE_POINTS = 16;
+
+function finiteNumber(value, field, fallback) {
+ const result = value ?? fallback;
+ if (!Number.isFinite(result)) throw new RangeError(`Modifier ${field} must be a finite number, for example ${fallback}.`);
+ return result;
+}
+
+function checkKeys(modifier, allowed) {
+ for (const key of Object.keys(modifier)) {
+ if (key !== 'type' && !allowed.includes(key)) throw new RangeError(`Modifier ${modifier.type} has no field ${key}. Fields: ${allowed.join(', ')}.`);
+ }
+}
+
+function cycleCount(value, field) {
+ const count = value ?? 0;
+ if (!Number.isInteger(count) || count < 0 || count > 1000) throw new RangeError(`Modifier ${field} must be an integer from 0 (forever) to 1000.`);
+ return count;
+}
+
+function normalizeCycles(modifier, property) {
+ checkKeys(modifier, ['before', 'after', 'beforeCount', 'afterCount']);
+ const before = modifier.before ?? 'repeat';
+ const after = modifier.after ?? 'repeat';
+ for (const [field, mode] of [['before', before], ['after', after]]) {
+ if (!CYCLE_MODES.includes(mode)) throw new RangeError(`Modifier cycles ${field} must be one of ${CYCLE_MODES.join(', ')}, not ${mode}.`);
+ if (mode === 'offset' && property === 'rotation') throw new RangeError('Modifier cycles offset does not apply to rotation. Use repeat or mirror.');
+ }
+ return { type: 'cycles', before, after, beforeCount: cycleCount(modifier.beforeCount, 'cycles beforeCount'), afterCount: cycleCount(modifier.afterCount, 'cycles afterCount') };
+}
+
+function normalizeStepped(modifier) {
+ checkKeys(modifier, ['step', 'offset']);
+ const step = finiteNumber(modifier.step, 'stepped step', 0.1);
+ if (step <= 0) throw new RangeError('Modifier stepped step must be above 0 seconds, for example 0.1.');
+ return { type: 'stepped', step, offset: finiteNumber(modifier.offset, 'stepped offset', 0) };
+}
+
+function normalizeNoise(modifier) {
+ checkKeys(modifier, ['scale', 'strength', 'phase', 'seed']);
+ const scale = finiteNumber(modifier.scale, 'noise scale', 1);
+ if (scale <= 0) throw new RangeError('Modifier noise scale is seconds per wobble and must be above 0, for example 0.25.');
+ const seed = modifier.seed ?? 0;
+ if (!Number.isInteger(seed) || Math.abs(seed) > 2 ** 31) throw new RangeError('Modifier noise seed must be an integer, for example 7.');
+ return { type: 'noise', scale, strength: finiteNumber(modifier.strength, 'noise strength', 1), phase: finiteNumber(modifier.phase, 'noise phase', 0), seed };
+}
+
+function normalizeGenerator(modifier) {
+ checkKeys(modifier, ['coefficients', 'additive']);
+ const coefficients = modifier.coefficients ?? [0, 1];
+ if (!Array.isArray(coefficients) || !coefficients.length || coefficients.length > MAXIMUM_POLYNOMIAL_TERMS || !coefficients.every(Number.isFinite)) {
+ throw new RangeError(`Modifier generator coefficients are 1 to ${MAXIMUM_POLYNOMIAL_TERMS} finite numbers, lowest power first, for example [0, 1].`);
+ }
+ if (modifier.additive !== undefined && typeof modifier.additive !== 'boolean') throw new RangeError('Modifier generator additive must be true or false.');
+ return { type: 'generator', coefficients: [...coefficients], additive: modifier.additive ?? false };
+}
+
+function normalizeLimits(modifier) {
+ checkKeys(modifier, ['min', 'max']);
+ if (modifier.min === undefined && modifier.max === undefined) throw new RangeError('Modifier limits needs min, max or both, for example {min:0,max:1}.');
+ const result = { type: 'limits' };
+ if (modifier.min !== undefined) result.min = finiteNumber(modifier.min, 'limits min', 0);
+ if (modifier.max !== undefined) result.max = finiteNumber(modifier.max, 'limits max', 1);
+ if (result.min !== undefined && result.max !== undefined && result.min > result.max) throw new RangeError('Modifier limits min must not exceed max.');
+ return result;
+}
+
+function normalizeEnvelope(modifier) {
+ checkKeys(modifier, ['points', 'referenceMin', 'referenceMax']);
+ const points = modifier.points;
+ if (!Array.isArray(points) || !points.length || points.length > MAXIMUM_ENVELOPE_POINTS) {
+ throw new RangeError(`Modifier envelope points are 1 to ${MAXIMUM_ENVELOPE_POINTS} of {time,min,max}, for example [{time:0,min:-1,max:1}].`);
+ }
+ const normalized = points.map(point => ({
+ time: finiteNumber(point?.time, 'envelope point time', 0),
+ min: finiteNumber(point?.min, 'envelope point min', -1),
+ max: finiteNumber(point?.max, 'envelope point max', 1),
+ })).sort((first, second) => first.time - second.time);
+ for (let index = 1; index < normalized.length; index++) {
+ if (normalized[index].time === normalized[index - 1].time) throw new RangeError('Modifier envelope point times must be distinct.');
+ }
+ const referenceMin = finiteNumber(modifier.referenceMin, 'envelope referenceMin', -1);
+ const referenceMax = finiteNumber(modifier.referenceMax, 'envelope referenceMax', 1);
+ if (referenceMax <= referenceMin) throw new RangeError('Modifier envelope referenceMax must be above referenceMin.');
+ return { type: 'envelope', points: normalized, referenceMin, referenceMax };
+}
+
+const NORMALIZERS = {
+ cycles: normalizeCycles,
+ stepped: normalizeStepped,
+ noise: normalizeNoise,
+ generator: normalizeGenerator,
+ limits: normalizeLimits,
+ envelope: normalizeEnvelope,
+};
+
+/** Returns `{ extrapolation, timeModifiers, valueModifiers }` frozen, or null when the track has neither. */
+export function normalizeTrackEffects(track) {
+ const extrapolation = track.extrapolation ?? 'constant';
+ if (extrapolation !== 'constant' && extrapolation !== 'linear') throw new RangeError(`Track extrapolation must be constant or linear, not ${extrapolation}.`);
+ if (extrapolation === 'linear' && track.property === 'rotation') throw new RangeError('Rotation tracks cannot extrapolate linearly. Use constant, or a cycles modifier.');
+ const modifiers = track.modifiers ?? [];
+ if (!Array.isArray(modifiers) || modifiers.length > MAXIMUM_MODIFIERS) throw new RangeError(`Track modifiers is a list of at most ${MAXIMUM_MODIFIERS} objects like {type:'noise'}.`);
+ if (extrapolation === 'constant' && !modifiers.length) return null;
+ const timeModifiers = [];
+ const valueModifiers = [];
+ for (const modifier of modifiers) {
+ const normalizer = NORMALIZERS[modifier?.type];
+ if (!normalizer) throw new RangeError(`Unknown modifier type ${modifier?.type}. Types: ${Object.keys(NORMALIZERS).join(', ')}.`);
+ if (!TIME_TYPES.has(modifier.type) && track.property === 'rotation') throw new RangeError(`Rotation tracks accept only cycles and stepped modifiers, not ${modifier.type}.`);
+ (TIME_TYPES.has(modifier.type) ? timeModifiers : valueModifiers).push(Object.freeze(normalizer(modifier, track.property)));
+ }
+ return Object.freeze({ extrapolation, timeModifiers: Object.freeze(timeModifiers), valueModifiers: Object.freeze(valueModifiers) });
+}
+
+/** The modifiers back in authored form, for recipes: time modifiers first, as they run. */
+export function trackEffectsDescription(effects) {
+ const description = {};
+ if (effects.extrapolation !== 'constant') description.extrapolation = effects.extrapolation;
+ const modifiers = [...effects.timeModifiers, ...effects.valueModifiers];
+ if (modifiers.length) description.modifiers = modifiers.map(modifier => structuredClone(modifier));
+ return description;
+}
+
+/** Remaps time through `cycles` and `stepped`. Returns the time and adds the per-cycle value offset multiples to `offsets`. */
+export function remapTime(effects, keyframes, time, offsets) {
+ const firstTime = keyframes[0].time;
+ const length = keyframes[keyframes.length - 1].time - firstTime;
+ let remapped = time;
+ for (const modifier of effects.timeModifiers) {
+ if (modifier.type === 'stepped') {
+ remapped = Math.floor((remapped - modifier.offset) / modifier.step) * modifier.step + modifier.offset;
+ continue;
+ }
+ if (length <= 0 || remapped >= firstTime && remapped <= firstTime + length) continue;
+ const isBefore = remapped < firstTime;
+ const mode = isBefore ? modifier.before : modifier.after;
+ if (mode === 'none') continue;
+ const limit = isBefore ? modifier.beforeCount : modifier.afterCount;
+ let cycle = Math.floor((remapped - firstTime) / length);
+ let local = remapped - firstTime - cycle * length;
+ const passed = isBefore ? -cycle : cycle;
+ if (limit > 0 && passed > limit) {
+ cycle = isBefore ? -limit : limit;
+ local = isBefore ? 0 : length;
+ }
+ if (mode === 'mirror' && Math.abs(cycle) % 2 === 1) local = length - local;
+ if (mode === 'offset') offsets.push(cycle);
+ remapped = firstTime + local;
+ }
+ return remapped;
+}
+
+function hashUnit(index, seed) {
+ let state = Math.imul(index ^ 0x9e3779b9, 0x85ebca6b) ^ Math.imul(seed | 0, 0xc2b2ae35);
+ state = Math.imul(state ^ (state >>> 15), 0x2c1b3c6d);
+ state = Math.imul(state ^ (state >>> 12), 0x297a2d39);
+ state ^= state >>> 15;
+ return (state >>> 0) / 2147483648 - 1;
+}
+
+/** Smooth value noise in [-1, 1]; the same input always gives the same output. */
+export function valueNoise(position, seed) {
+ const cell = Math.floor(position);
+ const fraction = position - cell;
+ const blend = fraction * fraction * (3 - 2 * fraction);
+ const start = hashUnit(cell, seed);
+ return start + (hashUnit(cell + 1, seed) - start) * blend;
+}
+
+function envelopeBounds(points, time) {
+ const last = points[points.length - 1];
+ if (time <= points[0].time) return [points[0].min, points[0].max];
+ if (time >= last.time) return [last.min, last.max];
+ let upper = 1;
+ while (points[upper].time < time) upper++;
+ const start = points[upper - 1];
+ const end = points[upper];
+ const progress = (time - start.time) / (end.time - start.time);
+ return [start.min + (end.min - start.min) * progress, start.max + (end.max - start.max) * progress];
+}
+
+/** Runs the value modifiers over `output` in place. `time` is the original, unremapped time. */
+export function applyValueModifiers(effects, time, output) {
+ for (const modifier of effects.valueModifiers) {
+ for (let axis = 0; axis < output.length; axis++) {
+ const value = output[axis];
+ if (modifier.type === 'noise') {
+ output[axis] = value + valueNoise((time + modifier.phase) / modifier.scale, modifier.seed + axis * 7919) * modifier.strength;
+ } else if (modifier.type === 'generator') {
+ let polynomial = 0;
+ for (let term = modifier.coefficients.length - 1; term >= 0; term--) polynomial = polynomial * time + modifier.coefficients[term];
+ output[axis] = modifier.additive ? value + polynomial : polynomial;
+ } else if (modifier.type === 'limits') {
+ output[axis] = Math.min(modifier.max ?? Infinity, Math.max(modifier.min ?? -Infinity, value));
+ } else {
+ const [minimum, maximum] = envelopeBounds(modifier.points, time);
+ const progress = (value - modifier.referenceMin) / (modifier.referenceMax - modifier.referenceMin);
+ output[axis] = minimum + (maximum - minimum) * progress;
+ }
+ }
+ }
+}
+
+function componentOf(value, axis) {
+ return typeof value === 'number' ? value : value[axis];
+}
+
+/** Secant slope per second between two keyframes on one axis. */
+function secantSlope(from, to, axis) {
+ return (componentOf(to.value, axis) - componentOf(from.value, axis)) / (to.time - from.time);
+}
+
+function sampleTrackWithEffects(track, time, output, sampleKeyframes, constrainSample) {
+ const { effects, keyframes } = track;
+ const offsets = [];
+ const remapped = remapTime(effects, keyframes, time, offsets);
+ const first = keyframes[0];
+ const last = keyframes[keyframes.length - 1];
+ sampleKeyframes(track, remapped, output);
+ if (effects.extrapolation === 'linear' && keyframes.length > 1) {
+ const slopeFrom = remapped < first.time ? first : keyframes[keyframes.length - 2];
+ const slopeTo = remapped < first.time ? keyframes[1] : last;
+ const distance = remapped < first.time ? remapped - first.time : remapped > last.time ? remapped - last.time : 0;
+ for (let axis = 0; axis < output.length; axis++) output[axis] += secantSlope(slopeFrom, slopeTo, axis) * distance;
+ }
+ for (const cycle of offsets) {
+ for (let axis = 0; axis < output.length; axis++) output[axis] += cycle * (componentOf(last.value, axis) - componentOf(first.value, axis));
+ }
+ applyValueModifiers(effects, time, output);
+ for (let axis = 0; axis < output.length; axis++) output[axis] = constrainSample(track.property, output[axis]);
+ return output;
+}
+
+registerTrackEffects({ normalize: normalizeTrackEffects, sample: sampleTrackWithEffects });
diff --git a/src/animation.js b/src/animation.js
index 82e5f00c..150cb89d 100644
--- a/src/animation.js
+++ b/src/animation.js
@@ -85,7 +85,9 @@ export function createAnimationClip({ duration, loop = false, tracks = [] } = {}
for (let index = 1; index < keyframes.length; index++) {
if (keyframes[index].time === keyframes[index - 1].time) throw new RangeError('Keyframe times must be distinct.');
}
- return Object.freeze({ target: track.target, property: track.property, interpolation, keyframes: Object.freeze(keyframes) });
+ const effects = track.extrapolation === undefined && track.modifiers === undefined ? null : trackEffects?.normalize(track);
+ if (effects === undefined) throw new Error('Import animation-modifiers.js.');
+ return Object.freeze({ target: track.target, property: track.property, interpolation, keyframes: Object.freeze(keyframes), ...(effects ? { effects } : {}) });
});
return Object.freeze({ duration, loop, tracks: Object.freeze(compiled) });
}
@@ -157,6 +159,17 @@ function sampleTrackInto(track, time, output) {
return output;
}
+let trackEffects = null;
+
+/** Called by animation-modifiers.js when it loads, so the modifier code stays out of bundles that never use it. */
+export function registerTrackEffects(handlers) {
+ trackEffects = handlers;
+}
+
+function sampleTrack(track, time, output) {
+ return track.effects ? trackEffects.sample(track, time, output, sampleTrackInto, constrainSample) : sampleTrackInto(track, time, output);
+}
+
/**
* Linear keyframes that trace a track with eased segments, for formats with no easing (glTF). Each segment gets
* `stepsPerSegment` linear pieces; a track without easing comes back unchanged.
@@ -181,7 +194,8 @@ export function bakeEasedKeyframes(track, stepsPerSegment) {
function normalizedTime(clip, time) {
if (!Number.isFinite(time)) throw new TypeError('Animation time must be finite.');
- if (!clip.loop) return Math.max(0, Math.min(clip.duration, time));
+ // A clip that does not loop passes the time through: keyframes lie within the duration, so tracks hold their ends anyway, and extrapolation and cycles need the time past them.
+ if (!clip.loop) return time;
const remainder = time % clip.duration;
return remainder < 0 ? remainder + clip.duration : remainder;
}
@@ -189,7 +203,7 @@ function normalizedTime(clip, time) {
export function sampleAnimationClip(clip, time) {
const clipTime = normalizedTime(clip, time);
return clip.tracks.map(track => {
- const output = sampleTrackInto(track, clipTime, new Float64Array(PROPERTY_SIZES[track.property]));
+ const output = sampleTrack(track, clipTime, new Float64Array(PROPERTY_SIZES[track.property]));
const value = output.length === 1 ? output[0] : Array.from(output);
return { target: track.target, property: track.property, value };
});
@@ -207,7 +221,7 @@ export function compileAnimationSampler(clip) {
sample(time) {
const clipTime = normalizedTime(clip, time);
for (let index = 0; index < tracks.length; index++) {
- sampleTrackInto(clip.tracks[index], clipTime, tracks[index].output);
+ sampleTrack(clip.tracks[index], clipTime, tracks[index].output);
}
return tracks;
},
@@ -263,7 +277,7 @@ export function createAnimationPlayer(clip, parts) {
apply(time) {
const clipTime = normalizedTime(clip, time);
for (const compiled of compiledTracks) {
- sampleTrackInto(compiled.track, clipTime, compiled.output);
+ sampleTrack(compiled.track, clipTime, compiled.output);
}
for (const binding of compiledBindings) {
if (binding.hasTransform) {
diff --git a/tests/animation-modifiers.test.js b/tests/animation-modifiers.test.js
new file mode 100644
index 00000000..2413569e
--- /dev/null
+++ b/tests/animation-modifiers.test.js
@@ -0,0 +1,166 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { createAnimationClip, sampleAnimationClip } from '../src/animation.js';
+import { compileAgentMotion, normalizeAgentMotion } from '../src/agent-motion.js';
+import { valueNoise } from '../src/animation-modifiers.js';
+
+const rampKeyframes = [{ time: 1, value: 0 }, { time: 3, value: 4 }];
+
+function sampler(extra, keyframes = rampKeyframes, loop = false) {
+ const clip = createAnimationClip({ duration: 4, loop, tracks: [{ target: 'part', property: 'value', keyframes, ...extra }] });
+ return time => sampleAnimationClip(clip, time)[0].value;
+}
+
+test('extrapolation constant holds ends and linear continues the end slopes', () => {
+ const constant = sampler({ extrapolation: 'constant' });
+ assert.equal(constant(-5), 0);
+ assert.equal(constant(10), 4);
+ const linear = sampler({ extrapolation: 'linear' });
+ assert.equal(linear(0), -2);
+ assert.equal(linear(5), 8);
+ assert.equal(linear(2), 2);
+ const single = sampler({ extrapolation: 'linear' }, [{ time: 1, value: 3 }]);
+ assert.equal(single(9), 3);
+ const vector = createAnimationClip({ duration: 2, tracks: [{ target: 'part', property: 'translation', extrapolation: 'linear', keyframes: [{ time: 0, value: [0, 1, 2] }, { time: 1, value: [1, 1, 0] }] }] });
+ assert.deepEqual(sampleAnimationClip(vector, 3)[0].value, [3, 1, -4]);
+});
+
+test('cycles repeat, offset and mirror before and after the keys, with finite counts', () => {
+ const repeat = sampler({ modifiers: [{ type: 'cycles' }] });
+ assert.equal(repeat(4), 2);
+ assert.equal(repeat(7), 0);
+ assert.equal(repeat(-0.5), 1);
+ const offset = sampler({ modifiers: [{ type: 'cycles', before: 'offset', after: 'offset' }] });
+ assert.equal(offset(4), 6);
+ assert.equal(offset(6), 10);
+ assert.equal(offset(0), -2);
+ assert.equal(offset(-1), -4);
+ const mirror = sampler({ modifiers: [{ type: 'cycles', before: 'mirror', after: 'mirror' }] });
+ assert.equal(mirror(3.5), 3);
+ assert.equal(mirror(5), 0);
+ assert.equal(mirror(7), 4);
+ const counted = sampler({ modifiers: [{ type: 'cycles', after: 'offset', afterCount: 1 }] });
+ assert.equal(counted(4), 6);
+ assert.equal(counted(4.5), 7);
+ assert.equal(counted(5.9), 8);
+ assert.equal(counted(50), 8);
+ const none = sampler({ modifiers: [{ type: 'cycles', before: 'none', after: 'none' }] });
+ assert.equal(none(10), 4);
+});
+
+test('cycles keep going past the clip duration', () => {
+ const value = sampler({ modifiers: [{ type: 'cycles', after: 'offset' }] });
+ assert.equal(value(11), 20);
+});
+
+test('stepped holds the curve for each step', () => {
+ const stepped = sampler({ modifiers: [{ type: 'stepped', step: 0.5 }] });
+ assert.equal(stepped(1.9), 1);
+ assert.equal(stepped(2.49), 2);
+ assert.equal(stepped(2.5), 3);
+ const shifted = sampler({ modifiers: [{ type: 'stepped', step: 1, offset: 0.25 }] });
+ assert.equal(shifted(2.2), 0.5);
+});
+
+test('generator replaces or adds a polynomial of time', () => {
+ const replace = sampler({ modifiers: [{ type: 'generator', coefficients: [1, 2, 3] }] });
+ assert.equal(replace(2), 1 + 2 * 2 + 3 * 4);
+ const additive = sampler({ modifiers: [{ type: 'generator', coefficients: [10, 0, 1], additive: true }] });
+ assert.equal(additive(2), 2 + 10 + 4);
+});
+
+test('limits clamp and envelope remaps a value range over time', () => {
+ const limits = sampler({ modifiers: [{ type: 'limits', min: 1, max: 3 }] });
+ assert.equal(limits(0), 1);
+ assert.equal(limits(2), 2);
+ assert.equal(limits(3), 3);
+ const wave = [{ time: 0, value: -1 }, { time: 4, value: 1 }];
+ const envelope = sampler({ modifiers: [{ type: 'envelope', points: [{ time: 0, min: 0, max: 2 }, { time: 4, min: -2, max: 6 }] }] }, wave);
+ assert.equal(envelope(0), 0);
+ assert.equal(envelope(4), 6);
+ assert.equal(envelope(2), 1.5);
+});
+
+test('noise is seeded, bounded, smooth and independent of the order of seeks', () => {
+ const noisy = sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 7 }] });
+ const forward = [0, 0.1, 0.37, 1, 2.2, 3, 3.9].map(noisy);
+ const backward = [3.9, 3, 2.2, 1, 0.37, 0.1, 0].map(noisy).reverse();
+ assert.deepEqual(forward, backward);
+ const plain = sampler({});
+ for (const time of [0.1, 1.5, 2.2, 3.5]) assert.ok(Math.abs(noisy(time) - plain(time)) <= 0.5);
+ const reseeded = sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 8 }] });
+ assert.notEqual(noisy(2.2), reseeded(2.2));
+ assert.equal(valueNoise(3, 1), valueNoise(3, 1));
+ const step = Math.abs(valueNoise(1.001, 5) - valueNoise(1, 5));
+ assert.ok(step < 0.01);
+ assert.equal(noisy(2.2), sampler({ modifiers: [{ type: 'noise', scale: 0.25, strength: 0.5, seed: 7 }] })(2.2));
+});
+
+test('noise differs per axis and a pinned sample stays byte stable', () => {
+ const clip = createAnimationClip({ duration: 1, tracks: [{ target: 'part', property: 'translation', modifiers: [{ type: 'noise', scale: 0.1, strength: 1, seed: 3 }], keyframes: [{ time: 0, value: [0, 0, 0] }] }] });
+ const [x, y, z] = sampleAnimationClip(clip, 0.33)[0].value;
+ assert.notEqual(x, y);
+ assert.notEqual(y, z);
+ assert.deepEqual(sampleAnimationClip(clip, 0.33)[0].value, [x, y, z]);
+});
+
+test('modifiers run time steps first, then value steps, and compose with extrapolation', () => {
+ const composed = sampler({ extrapolation: 'linear', modifiers: [{ type: 'limits', max: 6 }, { type: 'stepped', step: 1 }] });
+ assert.equal(composed(5.9), 6);
+ assert.equal(composed(5.4), 6);
+ assert.equal(composed(3.9), 4);
+});
+
+test('rotation tracks take only repeat or mirror cycles and stepped', () => {
+ const rotation = { target: 'part', property: 'rotation', keyframes: [{ time: 0, value: [0, 0, 0, 1] }, { time: 1, value: [0, 1, 0, 1] }] };
+ const clip = createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'cycles', after: 'mirror' }] }] });
+ const first = sampleAnimationClip(clip, 0.25)[0].value;
+ const mirrored = sampleAnimationClip(clip, 1.75)[0].value;
+ first.forEach((component, index) => assert.ok(Math.abs(component - mirrored[index]) < 1e-12));
+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, extrapolation: 'linear' }] }), /Rotation tracks cannot extrapolate/);
+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'noise' }] }] }), /only cycles and stepped/);
+ assert.throws(() => createAnimationClip({ duration: 1, tracks: [{ ...rotation, modifiers: [{ type: 'cycles', after: 'offset' }] }] }), /does not apply to rotation/);
+});
+
+test('invalid modifiers name the field and an accepted example', () => {
+ const bad = modifiers => () => sampler({ modifiers });
+ assert.throws(bad([{ type: 'wiggle' }]), /Unknown modifier type wiggle/);
+ assert.throws(bad([{ type: 'noise', scale: 0 }]), /noise scale/);
+ assert.throws(bad([{ type: 'noise', speed: 1 }]), /no field speed/);
+ assert.throws(bad([{ type: 'stepped', step: 0 }]), /stepped step/);
+ assert.throws(bad([{ type: 'limits' }]), /min, max or both/);
+ assert.throws(bad([{ type: 'limits', min: 2, max: 1 }]), /must not exceed/);
+ assert.throws(bad([{ type: 'generator', coefficients: [] }]), /coefficients/);
+ assert.throws(bad([{ type: 'envelope', points: [] }]), /envelope points/);
+ assert.throws(bad([{ type: 'cycles', after: 'spin' }]), /cycles after/);
+ assert.throws(() => sampler({ extrapolation: 'cubic' }), /constant or linear/);
+});
+
+const recipe = { nodes: [{ id: 'cube' }], camera: { eye: [0, 0, 5], target: [0, 0, 0], up: [0, 1, 0], fov: 45, near: 0.1, far: 100 } };
+
+test('recipe motion tracks keep modifiers, sample them at the same times in every sampler and seek backwards', () => {
+ const input = {
+ duration: 2,
+ tracks: [
+ { kind: 'node', id: 'cube', property: 'translation', extrapolation: 'linear',
+ modifiers: [{ type: 'noise', scale: 0.2, strength: 0.1, seed: 4 }],
+ keyframes: [{ time: 0, value: [0, 0, 0] }, { time: 1, value: [1, 0, 0] }] },
+ { kind: 'camera', property: 'eye', modifiers: [{ type: 'cycles', after: 'offset' }],
+ keyframes: [{ time: 0, value: [0, 0, 5] }, { time: 1, value: [1, 0, 5] }] },
+ ],
+ };
+ const normalized = normalizeAgentMotion(input, recipe);
+ assert.equal(normalized.tracks[0].extrapolation, 'linear');
+ assert.deepEqual(normalized.tracks[0].modifiers[0], { type: 'noise', scale: 0.2, strength: 0.1, phase: 0, seed: 4 });
+ assert.deepEqual(normalizeAgentMotion(normalized, recipe), normalized);
+ const compiled = compileAgentMotion(input, recipe);
+ const read = time => {
+ const output = compiled.sample(time);
+ return { cube: Array.from(output.nodes.get('cube').translation), eye: Array.from(output.camera.eye) };
+ };
+ const forward = [0.3, 1.5, 2.5].map(read);
+ const backward = [2.5, 1.5, 0.3].map(read).reverse();
+ assert.deepEqual(forward, backward);
+ assert.ok(Math.abs(forward[1].cube[0] - 1.5) <= 0.1);
+ assert.ok(Math.abs(forward[2].eye[0] - 2.5) < 1e-6);
+});