Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Bake physics, constraints and IK to keyframes, and simplify baked curves

Nguyen Xuan An05ea5e611686
diff --git a/changes/697.md b/changes/697.md
new file mode 100644
index 00000000..d46bdf2f
--- /dev/null
+++ b/changes/697.md
@@ -0,0 +1 @@
+- `scene_apply` takes `bake-motion`: it samples nodes over a range into keyframe tracks (tracks, IK, constraints, drivers, ragdoll and rigid bodies), optionally drops keys within a position and angle tolerance, and removes the sources it replaced, so the motion exports as a clip (#697)
diff --git a/changes/697.size.md b/changes/697.size.md
new file mode 100644
index 00000000..249d19bb
--- /dev/null
+++ b/changes/697.size.md
@@ -0,0 +1,5 @@
+- tool-total: one scene_apply operation lets an agent hand a physics, IK or constraint motion to export as keys; its detail sits in scene_capabilities motionBake
+- tool:scene_apply: the bake-motion operation has five flat fields (nodes, start, end, step, tolerance) with no descriptions
+- scene:agent-scene: the core grows only by the bake hook, the shared node-pose helper and a short capabilities note; the sampler and the simplifier load on demand
+- goal:agent-scene: the core growth is the hook and the note; the bake code itself is a lazy module
+- scene:agent-scene+lazy: the lazy bake module (sampling, Ramer-Douglas-Peucker simplification, source removal) is the feature
diff --git a/docs/agent-motion.md b/docs/agent-motion.md
index 3cebbcf6..8602fa71 100644
--- a/docs/agent-motion.md
+++ b/docs/agent-motion.md
@@ -415,7 +415,7 @@ It can contain at most 128 tracks, 64 keys per track, and 2048 keys in total.
Values and tangents must remain finite at Float32 precision.
The scene integration must also enforce its total recipe byte budget.
-1. Run `node --test tests/agent-motion.test.js`.
+1. Run `node --test tests/agent-motion.test.js tests/motion-bake.test.js`.
The CPU tests cover curves, looping, references, bounds, backward seeks, retained buffers, and failed sample publication.
Native rendering and published MCP verification belong to the integration layer.
diff --git a/docs/parity.md b/docs/parity.md
index 93d55012..eb1891b4 100644
--- a/docs/parity.md
+++ b/docs/parity.md
@@ -133,7 +133,7 @@ To move to a new Blender release, run `node scripts/parity-manual-snapshot.mjs`.
| Sculpting & Painting > Curves Sculpting | missing | No hair curves. | | #63 |
| Grease Pencil | out of scope | Decided in `docs/goal.md`. | | |
| Animation & Rigging > Animation Editors | out of scope | Interface for a mouse. | | |
-| Animation & Rigging > Keyframes | partial | Linear, step, smooth and Hermite curves with deterministic seeks, plus per-keyframe named easings (Sine, Quad, Cubic, Quart, Quint, Expo, Circ, Back, Bounce and Elastic, each in, out or in-out) and `cubic-bezier` (#286). Each easing matches Blender 5.2 within 1e-4 at 20 sample times. Back and Elastic take overshoot, amplitude and period. There are no Bezier handles or keying sets. | [animation](../tests/animation.test.js), [easing](../tests/easing.test.js) | #286 |
+| Animation & Rigging > Keyframes | partial | Linear, step, smooth and Hermite curves with deterministic seeks, plus per-keyframe named easings (Sine, Quad, Cubic, Quart, Quint, Expo, Circ, Back, Bounce and Elastic, each in, out or in-out) and `cubic-bezier` (#286). Each easing matches Blender 5.2 within 1e-4 at 20 sample times. Back and Elastic take overshoot, amplitude and period. `bake-motion` in `scene_apply` samples a node's local transform (tracks, IK, constraints, drivers, ragdoll and rigid bodies) into LINEAR keys and drops keys within a position and angle tolerance, as Blender's Bake Action and Clean Channels do together; no Bezier handles, keying sets, world-space bake or bake of imported assets. | [animation](../tests/animation.test.js), [easing](../tests/easing.test.js), [motion-bake](../tests/motion-bake.test.js) | #286, #697 |
| Animation & Rigging > Armatures > Bones | partial | Recipe `armatures` list bones with a name, parent, head and tail, and each bone is a recipe node that tracks, `motion.ik` and a physics ragdoll (#96, `tests/ragdoll.test.js`: a capsule body per bone, spherical joints without swing limits, a seekable baked fall, at most 48 bones) address. No roll, bone shapes or custom properties. | [rigging](../tests/rigging.test.js) | #46 |
| Animation & Rigging > Armatures > Properties | partial | An armature has an id, a parent node, a transform, an optional bone-ID prefix and a `humanoid` or `quadruped` preset fitted to a mesh node or to bounds. No display settings or bone collections. | [rigging](../tests/rigging.test.js) | #46 |
| Animation & Rigging > Armatures > Structure | partial | Imported rigs keep their hierarchy, and recipes author one. A rig exports to GLB with its bones and a baked walk clip; Blender 5.2 opens it as 1 armature, 21 bones, 14 vertex-grouped meshes and 1 action. | [gltf](../tests/gltf.test.js), [rigging](../tests/rigging.test.js) | #46 |
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index 30927342..34a4b944 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -719,6 +719,7 @@ const otherOperations = [
materialListed(z.strictObject({ type: z.literal('set-material'), id: identifier, ...materialFields }, { error: flatFieldsHint('set-material', 'material') }), 'Change a material.', { type: z.literal('set-material'), id: identifier }),
z.strictObject({ type: z.literal('set-camera'), ...cameraFields }),
z.strictObject({ type: z.literal('set-motion'), motion: recipeMotionReference.nullable() }),
+ z.strictObject({ type: z.literal('bake-motion'), nodes: z.array(identifier).min(1).max(32), start: nonnegative.optional(), end: nonnegative.optional(), step: positive.optional(), tolerance: z.record(z.string(), z.number()).optional() }),
z.strictObject({ type: z.literal('set-armatures'), armatures: armaturesReference.nullable() }),
z.strictObject({ type: z.literal('set-physics'), physics: z.record(z.string(), z.unknown()).nullable().describe('Same object as recipe.physics in scene_create.') }),
z.strictObject({ type: z.literal('set-cloth'), cloth: z.record(z.string(), z.unknown()).nullable() }),
diff --git a/src/agent-apply-report.js b/src/agent-apply-report.js
index 85b8769b..d6238933 100644
--- a/src/agent-apply-report.js
+++ b/src/agent-apply-report.js
@@ -108,6 +108,9 @@ function describeOperation(working, operation, maxText) {
else working[name] = structuredClone(next);
return { changed: operation[name] === null ? `cleared ${name}` : `set ${name}` };
}
+ if (type === 'bake-motion') {
+ return { changed: `baked ${shortList(operation.nodes ?? [], maxText)} to keys` };
+ }
if (type === 'edit-mesh') {
const count = Array.isArray(operation.edits) ? operation.edits.length : 0;
if (count === 0 && !operation.replace) return { warning: 'no change, no edits given' };
diff --git a/src/agent-motion-bake.js b/src/agent-motion-bake.js
new file mode 100644
index 00000000..1e097dc2
--- /dev/null
+++ b/src/agent-motion-bake.js
@@ -0,0 +1,249 @@
+import { quat } from 'gl-matrix';
+import { invalidField } from './field-error.js';
+import { bounded, knownKeys } from './motion-validation.js';
+
+// The bake-motion operation: sample the live local transform of chosen nodes (tracks, IK, constraints, drivers and
+// physics all included), write it back as keyframe tracks and drop the live sources, so the motion plays from keys alone.
+
+export const BAKE_LIMITS = Object.freeze({ maxNodes: 32, maxSamples: 512, maxKeysPerTrack: 64, defaultStep: 1 / 30, minimumStep: 1 / 120 });
+const DEFAULT_TOLERANCE = Object.freeze({ position: 0.001, angle: 0.1 });
+const MAX_MOTION_DURATION = 30;
+const BAKED_PROPERTIES = ['translation', 'rotation', 'scale'];
+const REST_VALUES = { translation: [0, 0, 0], rotation: [0, 0, 0, 1], scale: [1, 1, 1] };
+// Float noise below this is no movement: a node the bake leaves where it was gets no track.
+const STEADY_EPSILON = 1e-6;
+const RADIANS_PER_DEGREE = Math.PI / 180;
+
+function roundTime(seconds) {
+ return Math.round(seconds * 1e6) / 1e6;
+}
+
+function normalizedTolerance(input) {
+ if (input === undefined) return null;
+ knownKeys(input, ['position', 'angle', 'scale'], 'tolerance');
+ const position = bounded(input.position ?? DEFAULT_TOLERANCE.position, 0, 10, 'tolerance.position');
+ const angleDegrees = bounded(input.angle ?? DEFAULT_TOLERANCE.angle, 0, 45, 'tolerance.angle');
+ const scale = bounded(input.scale ?? position, 0, 10, 'tolerance.scale');
+ return { translation: position, rotation: angleDegrees * RADIANS_PER_DEGREE, scale };
+}
+
+function normalizedNodeIds(input, candidate) {
+ const known = new Map(candidate.nodes.map(node => [node.id, node]));
+ const example = JSON.stringify([candidate.nodes.find(node => !node.assetPart)?.id ?? 'crate']);
+ if (!Array.isArray(input) || input.length < 1 || input.length > BAKE_LIMITS.maxNodes) {
+ throw invalidField('nodes', Array.isArray(input) ? `${input.length} IDs` : input, `a list of 1 to ${BAKE_LIMITS.maxNodes} node IDs, such as ${example}`, Array.isArray(input) ? RangeError : TypeError);
+ }
+ input.forEach((id, position) => {
+ const node = known.get(id);
+ if (!node || node.assetPart) {
+ throw invalidField(`nodes[${position}]`, id, `the ID of a node you placed, such as ${example}; scene_inspect lists them`);
+ }
+ if (input.indexOf(id) !== position) {
+ throw invalidField(`nodes[${position}]`, id, 'each node ID once');
+ }
+ });
+ return input;
+}
+
+/** Validate a bake-motion operation against the scene. `source` describes what the committed scene can sample. */
+function normalizedBake(operation, candidate, source) {
+ knownKeys(operation, ['type', 'nodes', 'start', 'end', 'step', 'tolerance'], 'operation');
+ const ids = normalizedNodeIds(operation.nodes, candidate);
+ if (!source.animated) {
+ throw invalidField('nodes', ids, 'a scene that already moves: add motion or physics first, since there is nothing to sample yet');
+ }
+ const naturalEnd = Math.max(source.motionDuration, source.physicsDuration);
+ const start = bounded(operation.start ?? 0, 0, MAX_MOTION_DURATION, 'start');
+ const end = roundTime(bounded(operation.end ?? Math.min(naturalEnd, MAX_MOTION_DURATION), 0, MAX_MOTION_DURATION, 'end'));
+ if (end <= start) {
+ throw invalidField('end', end, `a time after start ${start}, up to ${MAX_MOTION_DURATION} seconds, such as ${roundTime(start + 1)}`);
+ }
+ const step = bounded(operation.step ?? BAKE_LIMITS.defaultStep, BAKE_LIMITS.minimumStep, 1, 'step');
+ const sampleCount = Math.ceil((end - start) / step - 1e-9) + 1;
+ if (sampleCount > BAKE_LIMITS.maxSamples) {
+ throw invalidField('step', step, `a step that gives at most ${BAKE_LIMITS.maxSamples} samples over ${roundTime(end - start)} seconds, so at least ${roundTime((end - start) / (BAKE_LIMITS.maxSamples - 1))}, or a shorter start to end range`);
+ }
+ return { ids, start, end, step, sampleCount, tolerance: normalizedTolerance(operation.tolerance), toleranceInput: operation.tolerance };
+}
+
+function sampleTimes({ start, end, step, sampleCount }) {
+ return Array.from({ length: sampleCount }, (_, index) => (index === sampleCount - 1 ? end : roundTime(start + index * step)));
+}
+
+/** Per node and property, the values at every sample time. Quaternions keep one sign so interpolation takes the short way. */
+function sampledValues(options, source) {
+ const times = sampleTimes(options);
+ const series = new Map(options.ids.map(id => [id, { translation: [], rotation: [], scale: [] }]));
+ for (const time of times) {
+ const transforms = source.sampleLocalTransforms(time, options.ids);
+ for (const id of options.ids) {
+ const transform = transforms.get(id);
+ const target = series.get(id);
+ for (const property of BAKED_PROPERTIES) {
+ const value = Array.from(transform[property]);
+ const previous = target[property].at(-1);
+ if (property === 'rotation' && previous && quat.dot(previous, value) < 0) {
+ value.forEach((component, index) => { value[index] = -component; });
+ }
+ target[property].push(value);
+ }
+ }
+ }
+ return { times, series };
+}
+
+function distance(first, second) {
+ return Math.hypot(...first.map((component, index) => component - second[index]));
+}
+
+function rotationAngle(first, second) {
+ return 2 * Math.acos(Math.min(1, Math.abs(quat.dot(first, second))));
+}
+
+function difference(property, first, second) {
+ return property === 'rotation' ? rotationAngle(first, second) : distance(first, second);
+}
+
+function interpolated(property, first, second, progress) {
+ if (property === 'rotation') return Array.from(quat.slerp(new Float64Array(4), first, second, progress));
+ return first.map((component, index) => component + (second[index] - component) * progress);
+}
+
+/** Ramer-Douglas-Peucker over time: keep the sample the kept neighbours miss most until none misses by more than the tolerance. */
+function keptIndices(property, times, values, tolerance) {
+ const kept = new Set([0, values.length - 1]);
+ const pending = [[0, values.length - 1]];
+ while (pending.length) {
+ const [first, last] = pending.pop();
+ let worstIndex = -1;
+ let worstError = tolerance;
+ for (let index = first + 1; index < last; index++) {
+ const progress = (times[index] - times[first]) / (times[last] - times[first]);
+ const error = difference(property, interpolated(property, values[first], values[last], progress), values[index]);
+ if (error > worstError) {
+ worstError = error;
+ worstIndex = index;
+ }
+ }
+ if (worstIndex >= 0) {
+ kept.add(worstIndex);
+ pending.push([first, worstIndex], [worstIndex, last]);
+ }
+ }
+ return [...kept].sort((left, right) => left - right);
+}
+
+/** The keyframes of one property, or null when it stays at the node's rest value. */
+function keyframesFor(property, times, values, restValue, tolerance) {
+ const limit = Math.max(tolerance?.[property] ?? 0, STEADY_EPSILON);
+ const steady = values.every(value => difference(property, value, values[0]) <= limit);
+ if (steady) {
+ return difference(property, values[0], restValue) <= limit ? null : [{ time: times[0], value: values[0] }];
+ }
+ const indices = tolerance ? keptIndices(property, times, values, limit) : values.map((_, index) => index);
+ return indices.map(index => ({ time: times[index], value: values[index] }));
+}
+
+function bakedTracks(options, series, times, candidate) {
+ const nodes = new Map(candidate.nodes.map(node => [node.id, node]));
+ const tracks = [];
+ for (const id of options.ids) {
+ for (const property of BAKED_PROPERTIES) {
+ const restValue = nodes.get(id)[property] ?? REST_VALUES[property];
+ const keyframes = keyframesFor(property, times, series.get(id)[property], restValue, options.tolerance);
+ if (!keyframes) continue;
+ if (keyframes.length > BAKE_LIMITS.maxKeysPerTrack) {
+ const advice = options.tolerance ? 'a larger tolerance' : 'a tolerance such as { "position": 0.005, "angle": 0.5 }';
+ throw invalidField('tolerance', options.toleranceInput ?? null, `${advice}, or a larger step, since a track holds at most ${BAKE_LIMITS.maxKeysPerTrack} keys and ${id} ${property} needs ${keyframes.length}`);
+ }
+ tracks.push({ kind: 'node', id, property, interpolation: 'LINEAR', keyframes });
+ }
+ }
+ return tracks;
+}
+
+function solverJoints(solver, nodes) {
+ if (solver.type === 'two-bone') return [solver.root, solver.middle, solver.end];
+ if (solver.type === 'aim') return solver.joints;
+ if (solver.chain) return solver.chain;
+ const path = [];
+ for (let id = solver.end; id !== null && id !== undefined; id = nodes.get(id)?.parent) {
+ path.push(id);
+ if (id === solver.root) break;
+ }
+ return path;
+}
+
+function physicsNodeIds(physics, candidate) {
+ const ids = (physics.bodies ?? []).map(body => body.node);
+ if (physics.ragdoll) {
+ ids.push(...candidate.nodes.filter(node => node.armature === physics.ragdoll.armature && node.id !== node.armature).map(node => node.id));
+ }
+ return ids;
+}
+
+/** A solver or a physics world moves its nodes together, so a bake that takes some of them must take all. */
+function requireWholeGroup(group, kind, remedy, ids) {
+ const missing = group.filter(id => !ids.has(id));
+ if (missing.length) {
+ throw invalidField('nodes', [...ids], `every node the ${kind} moves, so also ${missing.slice(0, 6).join(', ')}${missing.length > 6 ? ' and more' : ''}; ${remedy}`);
+ }
+}
+
+/** Remove what drove the baked nodes, so the keys are not applied on top of the live source. */
+function dropLiveSources(candidate, idList) {
+ const ids = new Set(idList);
+ const nodes = new Map(candidate.nodes.map(node => [node.id, node]));
+ const { motion, physics } = candidate;
+ if (physics) {
+ const group = physicsNodeIds(physics, candidate);
+ if (group.some(id => ids.has(id))) {
+ requireWholeGroup(group, 'physics world', 'bodies collide, so they bake together', ids);
+ delete candidate.physics;
+ }
+ }
+ if (!motion) return;
+ const solvers = (motion.ik ?? []).filter(solver => {
+ const joints = solverJoints(solver, nodes);
+ if (!joints.some(id => ids.has(id))) return true;
+ requireWholeGroup(joints, `${solver.type} IK solver`, 'its joints bake together', ids);
+ return false;
+ });
+ const remaining = {
+ ik: solvers,
+ constraints: (motion.constraints ?? []).filter(constraint => !ids.has(constraint.node)),
+ drivers: (motion.drivers ?? []).filter(driver => !(driver.target.kind === 'node' && ids.has(driver.target.id))),
+ };
+ for (const [field, list] of Object.entries(remaining)) {
+ if (list.length) motion[field] = list;
+ else delete motion[field];
+ }
+}
+
+function motionHasContent(motion) {
+ return Boolean(motion.tracks.length || motion.layers?.length || motion.ik?.length || motion.constraints?.length || motion.drivers?.length || motion.drive);
+}
+
+/**
+ * Apply one bake-motion operation to a candidate recipe. `source` gives { animated, motionDuration, physicsDuration,
+ * sampleLocalTransforms(time, ids) } for the committed scene. Returns { tracks, keyframes } counts for the report.
+ */
+export function bakeMotion(candidate, operation, source) {
+ const options = normalizedBake(operation, candidate, source);
+ if (candidate.motion?.loop && options.end > candidate.motion.duration) {
+ throw invalidField('end', options.end, `at most the looping motion duration ${candidate.motion.duration}, since a longer bake would change the loop; set the motion loop to false first`);
+ }
+ dropLiveSources(candidate, options.ids);
+ const { times, series } = sampledValues(options, source);
+ const tracks = bakedTracks(options, series, times, candidate);
+ const baked = new Set(options.ids);
+ const motion = candidate.motion ?? { duration: Math.max(options.end, 0.1), loop: false, tracks: [] };
+ motion.duration = Math.max(motion.duration, options.end);
+ motion.tracks = [
+ ...motion.tracks.filter(track => !(track.kind === 'node' && baked.has(track.id) && BAKED_PROPERTIES.includes(track.property))),
+ ...tracks,
+ ];
+ if (motionHasContent(motion)) candidate.motion = motion;
+ else delete candidate.motion;
+ return { tracks: tracks.length, keyframes: tracks.reduce((total, track) => total + track.keyframes.length, 0) };
+}
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index fba4eb7a..8155493d 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -212,7 +212,7 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
operations: Object.freeze([
'set-node', 'set-material', 'set-camera', 'add-node', 'remove-node',
'add-geometry', 'remove-geometry', 'add-material', 'remove-material',
- 'set-environment', 'set-effects', 'set-shadow', 'set-motion', 'set-armatures', 'set-physics', 'set-cloth', 'set-timeline', 'set-particles', 'set-bake', 'set-post-passes', 'edit-mesh', 'set-mesh-stack',
+ 'set-environment', 'set-effects', 'set-shadow', 'set-motion', 'set-armatures', 'set-physics', 'set-cloth', 'set-timeline', 'set-particles', 'set-bake', 'set-post-passes', 'edit-mesh', 'set-mesh-stack', 'bake-motion',
]),
materialTags: Object.freeze(['surface', 'unlit', 'liquid', 'foam', 'foliage', 'grass', 'water']),
// A flat water sheet; the largest sets the level. A bed d metres down keeps exp(-absorption * d) per channel (RGB per metre).
@@ -326,6 +326,7 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
}),
motionModifiers: 'Track extrapolation constant|linear; modifiers [{type: cycles|stepped|noise|generator|limits|envelope, ...}], 8 at most; cycles and stepped remap time first.',
motionLimits: Object.freeze({ maxTracks: 128, maxKeyframesPerTrack: 64, maxKeyframes: 2048, maxDuration: 30 }),
+ motionBake: 'scene_apply bake-motion { nodes (32 at most), start 0, end (motion or physics duration), step 1/30 (512 samples), tolerance { position m, angle deg, scale } }: samples tracks, IK, constraints, drivers and physics into LINEAR tracks and removes those sources (an IK solver or the physics world only when all its nodes are baked). tolerance drops keys within it; without it a track keeps every sample, 64 keys at most. Procedural scenes only.',
});
/** Degrees between faces above which a mesh geometry keeps a hard edge, as Blender's auto smooth does. */
diff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.js
index 82d96445..22a7bad9 100644
--- a/src/agent-scene-modules.js
+++ b/src/agent-scene-modules.js
@@ -8,6 +8,7 @@ const MODULE_LOADERS = {
modeling: () => import('./modeling.js'),
text: () => import('./text-geometry.js'),
motion: () => import('./agent-motion.js'),
+ motionBake: () => import('./agent-motion-bake.js'),
drive: () => import('./agent-drive.js'),
physics: () => import('./agent-physics.js'),
cloth: () => import('./agent-cloth.js'),
@@ -50,7 +51,8 @@ const MODULE_MARKERS = {
mesh: ['"type":"mesh"', '"instances"'],
modeling: ['"type":"lathe"', '"type":"extrude"', '"type":"tube"', '"type":"curve"'],
text: ['"type":"text"'],
- motion: ['"motion":{'],
+ motion: ['"motion":{', '"bake-motion"'],
+ motionBake: ['"bake-motion"'],
drive: ['"drive":{'],
physics: ['"physics":{'],
cloth: ['"cloth":{'],
diff --git a/src/agent-scene.js b/src/agent-scene.js
index dc17eb25..0332b1b7 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -955,20 +955,46 @@ function cameraPoseFrom(pose) {
return camera;
}
+// Physics holds its last pose after its duration, so a seek past the end is the resting state.
+function simulatedPose(stagedState, time) {
+ return time === null ? null : stagedState.physicsSampler?.sample(Math.min(time, stagedState.physicsSampler.duration));
+}
+
+/** The local transform of a node at a sampled motion and simulated pose: a body, then a track, then the rest value. */
+function localTransformOf(node, sampled, simulated) {
+ const patch = sampled?.nodes.get(node.id);
+ const bodyPatch = simulated?.get(node.id);
+ return {
+ translation: bodyPatch?.translation ?? patch?.translation ?? node.translation,
+ rotation: bodyPatch?.rotation ?? patch?.rotation ?? node.rotation,
+ scale: patch?.scale ?? node.scale,
+ };
+}
+
+/** What a bake-motion operation samples: the committed scene's local node transforms at any time, without touching the pose. */
+function motionBakeSource(stagedState) {
+ const nodesById = new Map(stagedState.recipe.nodes.map(node => [node.id, node]));
+ return {
+ animated: Boolean(stagedState.motionSampler || stagedState.physicsSampler),
+ motionDuration: stagedState.recipe.motion?.duration ?? 0,
+ physicsDuration: stagedState.physicsSampler?.duration ?? 0,
+ sampleLocalTransforms(time, ids) {
+ const sampled = stagedState.motionSampler?.sample(time) ?? null;
+ const simulated = simulatedPose(stagedState, time);
+ return new Map(ids.map(id => [id, localTransformOf(nodesById.get(id), sampled, simulated)]));
+ },
+ };
+}
+
// Candidate buffers are private. A failed seek cannot publish half a pose to render handles.
function stagePose(stagedState, time) {
const sampled = time === null ? null : stagedState.motionSampler?.sample(time);
const hiddenNodes = hiddenNodeIds(stagedState.recipe, sampled);
- // Physics holds its last pose after its duration, so a seek past the end is the resting state.
- const simulated = time === null ? null : stagedState.physicsSampler?.sample(Math.min(time, stagedState.physicsSampler.duration));
+ const simulated = simulatedPose(stagedState, time);
const graph = stagedState.poseGraph ?? stagedState.graph;
if (stagedState.poseGraph) {
for (const node of stagedState.recipe.nodes) {
- const patch = sampled?.nodes.get(node.id);
- const bodyPatch = simulated?.get(node.id);
- const translation = bodyPatch?.translation ?? patch?.translation ?? node.translation;
- const rotation = bodyPatch?.rotation ?? patch?.rotation ?? node.rotation;
- const scale = patch?.scale ?? node.scale;
+ const { translation, rotation, scale } = localTransformOf(node, sampled, simulated);
if (Array.from(scale).some(component => !Number.isFinite(Math.fround(component)) || Math.fround(component) <= 0)) {
throw new RangeError('Motion scale must remain positive at Float32 precision.');
}
@@ -1696,6 +1722,10 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
candidate[collection].push(record);
continue;
}
+ if (operation.type === 'bake-motion') {
+ agentSceneModule('motionBake').bakeMotion(candidate, operation, motionBakeSource(state));
+ continue;
+ }
if (operation.type === 'edit-mesh') {
knownKeys(operation, ['type', 'geometry', 'edits', 'replace'], 'operation');
const target = meshGeometryTarget(candidate, operation, type => `Add a mesh geometry with base: { type: '${type}', ... } to edit it.`);
diff --git a/tests/motion-bake.test.js b/tests/motion-bake.test.js
new file mode 100644
index 00000000..0e1d1011
--- /dev/null
+++ b/tests/motion-bake.test.js
@@ -0,0 +1,193 @@
+import './support/load-agent-scene-modules.mjs';
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { AnimationMixer, Vector3 } from 'three';
+import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';
+import { createAgentScene, loadPhysicsEngine } from '../src/agent-scene.js';
+import { exportGlb } from '../src/export.js';
+import { walkerRecipe } from '../bench/agent-modeling/walker.mjs';
+
+await loadPhysicsEngine();
+
+const STEP = 0.05;
+
+function pathRecipe() {
+ return {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'box', type: 'box', bevel: 0 }],
+ materials: [{ id: 'paint', color: [0.6, 0.6, 0.6] }],
+ nodes: [
+ { id: 'rider', geometry: 'box', material: 'paint' },
+ { id: 'still', geometry: 'box', material: 'paint', translation: [3, 0, 0] },
+ ],
+ motion: {
+ duration: 2, loop: false, tracks: [],
+ constraints: [{ type: 'follow_path', node: 'rider', points: [[0, 0, 0], [2, 1, 0], [3, 0, 2], [0, 2, 3]], followCurve: true, forwardAxis: '+z', upAxis: 'y', duration: 2 }],
+ },
+ };
+}
+
+function frameTimes(duration, step) {
+ return Array.from({ length: Math.round(duration / step) + 1 }, (_, index) => Math.min(duration, Math.round(index * step * 1e6) / 1e6));
+}
+
+function matrixAt(scene, name, time) {
+ scene.sampleMotion(time);
+ return Array.from(scene.parts.find(part => part.name === name).matrix);
+}
+
+function largestDifference(first, second) {
+ return Math.max(...first.map((value, index) => Math.abs(value - second[index])));
+}
+
+function applyBake(scene, operation) {
+ return scene.apply([{ type: 'bake-motion', ...operation }], { expectedRevision: scene.revision });
+}
+
+test('baked keys reproduce the live constraint sample at every frame', () => {
+ const live = createAgentScene(pathRecipe());
+ const baked = createAgentScene(pathRecipe());
+ applyBake(baked, { nodes: ['rider'], step: STEP });
+ const motion = baked.exportRecipe().motion;
+ assert.equal(motion.constraints, undefined, 'the constraint is gone once its result is keys');
+ assert.deepEqual(motion.tracks.map(track => `${track.id}.${track.property}`).sort(), ['rider.rotation', 'rider.translation']);
+ for (const time of frameTimes(2, STEP)) {
+ assert.ok(largestDifference(matrixAt(baked, 'rider', time), matrixAt(live, 'rider', time)) < 1e-5, `rider at ${time}`);
+ }
+});
+
+test('simplifying keeps every frame within the tolerance and drops keys', () => {
+ const live = createAgentScene(pathRecipe());
+ const baked = createAgentScene(pathRecipe());
+ const tolerance = { position: 0.01, angle: 1 };
+ applyBake(baked, { nodes: ['rider'], step: 0.02, tolerance });
+ const tracks = baked.exportRecipe().motion.tracks;
+ const keys = tracks.reduce((total, track) => total + track.keyframes.length, 0);
+ assert.ok(keys < 2 * 101 / 2, `${keys} keys instead of 202`);
+ assert.ok(tracks.every(track => track.keyframes.length <= 64));
+ for (const time of frameTimes(2, 0.02)) {
+ const difference = largestDifference(matrixAt(baked, 'rider', time), matrixAt(live, 'rider', time));
+ assert.ok(difference < 0.05, `rider at ${time} is ${difference} off`);
+ }
+});
+
+test('a bake repeats byte for byte', () => {
+ const recipes = [0, 1].map(() => {
+ const scene = createAgentScene(pathRecipe());
+ applyBake(scene, { nodes: ['rider', 'still'], step: 0.04, tolerance: {} });
+ return JSON.stringify(scene.exportRecipe());
+ });
+ assert.equal(recipes[0], recipes[1]);
+});
+
+test('a node at its rest value gets no track, and a steady one gets a single key', () => {
+ const scene = createAgentScene(pathRecipe());
+ applyBake(scene, { nodes: ['still'], step: 0.5 });
+ assert.equal(scene.exportRecipe().motion.constraints.length, 1, 'the rider keeps its constraint');
+ assert.deepEqual(scene.exportRecipe().motion.tracks, []);
+ const withAnchor = pathRecipe();
+ withAnchor.nodes.push({ id: 'anchor', geometry: 'box', material: 'paint', translation: [0, 5, 0] });
+ const moved = createAgentScene({ ...withAnchor, motion: { duration: 1, tracks: [], constraints: [{ type: 'copy_location', node: 'still', target: 'anchor' }] } });
+ applyBake(moved, { nodes: ['still'], start: 0, end: 1, step: 0.5, tolerance: {} });
+ assert.equal(moved.exportRecipe().motion.tracks.find(track => track.property === 'translation').keyframes.length, 1);
+});
+
+test('a bake that cannot be honest names the field and what to pass', () => {
+ const scene = createAgentScene(pathRecipe());
+ const fails = (operation, pattern) => assert.throws(() => applyBake(scene, operation), pattern);
+ fails({ nodes: ['ghost'] }, /nodes\[0\].*ghost.*scene_inspect/);
+ fails({ nodes: [] }, /nodes.*1 to 32/);
+ fails({ nodes: ['rider', 'rider'] }, /nodes\[1\].*each node ID once/);
+ fails({ nodes: ['rider'], end: 0, start: 1 }, /end.*after start/);
+ fails({ nodes: ['rider'], step: 0.001 }, /step.*0.0083/);
+ fails({ nodes: ['rider'], step: 0.001 * 20, end: 30 }, /step.*512 samples/);
+ fails({ nodes: ['rider'], step: 0.01 }, /tolerance.*64 keys/);
+ fails({ nodes: ['rider'], tolerance: { position: -1 } }, /tolerance.position/);
+ fails({ nodes: ['rider'], speed: 2 }, /Unknown.*speed/);
+ assert.equal(scene.revision, 0, 'a failed bake changes nothing');
+ const motionless = pathRecipe();
+ delete motionless.motion;
+ const still = createAgentScene(motionless);
+ assert.throws(() => applyBake(still, { nodes: ['rider'] }), /nodes.*add motion or physics first/);
+});
+
+function ragdollRecipe() {
+ const recipe = walkerRecipe();
+ delete recipe.motion;
+ recipe.geometries.push({ id: 'slab', type: 'box' });
+ recipe.nodes.push({ id: 'floor', geometry: 'slab', material: 'cloth', translation: [0, -0.25, 0], scale: [8, 0.5, 8] });
+ recipe.armatures[0].translation = [0, 1, 0];
+ recipe.physics = { duration: 3, bodies: [{ node: 'floor', type: 'fixed' }], ragdoll: { armature: 'rig' } };
+ return recipe;
+}
+
+function jointWorlds(scene, time) {
+ scene.sampleMotion(time);
+ return scene.parts.filter(part => part.skin).flatMap(part => Array.from(part.skin.jointWorldMatrices));
+}
+
+function worldBoneIds(recipe) {
+ return [...recipe.nodes.filter(node => node.armature === 'rig' && node.id !== 'rig').map(node => node.id), 'floor'];
+}
+
+test('a ragdoll fall bakes to keys, simplifies and exports as a clip a viewer plays the same', async () => {
+ const live = createAgentScene(ragdollRecipe());
+ const baked = createAgentScene(ragdollRecipe());
+ const ids = worldBoneIds(baked.exportRecipe());
+ assert.throws(() => applyBake(baked, { nodes: ids.slice(1) }), /every node the physics world moves, so also/);
+ applyBake(baked, { nodes: ids, tolerance: { position: 0.01, angle: 1 } });
+ const recipe = baked.exportRecipe();
+ assert.equal(recipe.physics, undefined, 'the recording is replaced by keys');
+ const keys = recipe.motion.tracks.reduce((total, track) => total + track.keyframes.length, 0);
+ assert.ok(keys > 0 && keys < ids.length * 2 * 91 / 2, `${keys} keys`);
+ for (const time of frameTimes(3, 1 / 6)) {
+ const difference = largestDifference(jointWorlds(baked, time), jointWorlds(live, time));
+ assert.ok(difference < 0.1, `joints at ${time} are ${difference} off`);
+ }
+ const gltf = await new GLTFLoader().parseAsync(exportGlb(baked.parts, baked.rigExport()), '');
+ assert.equal(gltf.animations.length, 1);
+ const skinned = [];
+ gltf.scene.traverse(object => {
+ if (object.isSkinnedMesh) skinned.push(object);
+ });
+ const mixer = new AnimationMixer(gltf.scene);
+ mixer.clipAction(gltf.animations[0]).play();
+ const hips = skinned[0].skeleton.bones.find(bone => bone.name === 'hips');
+ const hipsIndex = skinned[0].skeleton.bones.indexOf(hips);
+ for (const time of [0.5, 1.5, 2.9]) {
+ mixer.setTime(time);
+ gltf.scene.updateMatrixWorld(true);
+ const liveMatrix = jointWorlds(live, time).slice(hipsIndex * 16, hipsIndex * 16 + 16);
+ const viewerPosition = hips.getWorldPosition(new Vector3());
+ const difference = Math.hypot(viewerPosition.x - liveMatrix[12], viewerPosition.y - liveMatrix[13], viewerPosition.z - liveMatrix[14]);
+ assert.ok(difference < 0.05, `hips at ${time} are ${difference} off in the viewer`);
+ }
+});
+
+function armRecipe() {
+ return {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'box', type: 'box', bevel: 0 }],
+ materials: [{ id: 'paint', color: [0.6, 0.6, 0.6] }],
+ nodes: [
+ { id: 'shoulder', translation: [0, 1, 0] },
+ { id: 'elbow', parent: 'shoulder', translation: [0.5, 0, 0] },
+ { id: 'tip', parent: 'elbow', translation: [0.4, 0, 0], geometry: 'box', material: 'paint', scale: [0.03, 0.03, 0.03] },
+ ],
+ motion: {
+ duration: 1, tracks: [],
+ ik: [{ type: 'two-bone', root: 'shoulder', middle: 'elbow', end: 'tip', pole: [0, 3, 0], target: { keyframes: [{ time: 0, value: [0.4, 1.5, 0.3] }, { time: 1, value: [0.2, 1.1, -0.3] }] } }],
+ },
+ };
+}
+
+test('IK bakes whole: the arm joints become rotation keys, and a partial bake is refused', () => {
+ const live = createAgentScene(armRecipe());
+ const baked = createAgentScene(armRecipe());
+ assert.throws(() => applyBake(baked, { nodes: ['elbow'] }), /nodes.*every node the two-bone IK solver moves, so also shoulder, tip/);
+ applyBake(baked, { nodes: ['shoulder', 'elbow', 'tip'], step: 0.1 });
+ assert.equal(baked.exportRecipe().motion.ik, undefined);
+ for (const time of frameTimes(1, 0.1)) {
+ assert.ok(largestDifference(matrixAt(baked, 'tip', time), matrixAt(live, 'tip', time)) < 1e-4, `tip at ${time}`);
+ }
+});