Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Drive cars through MCP with motion.drive and asset.drive

Nguyen Xuan An487d78011974
diff --git a/changes/322-mcp.md b/changes/322-mcp.md
new file mode 100644
index 00000000..ea3adf58
--- /dev/null
+++ b/changes/322-mcp.md
@@ -0,0 +1 @@
+- Drive cars through MCP (issue #322): `motion.drive` on an imported asset recipe and `drive` on a placed asset run the vehicle kit from a path or legs, with Ackermann steering, reverse legs, wheels found with `wheels: 'auto'`, headlight poses and tail lights that follow the brake. `scene_inspect` reports `motionDrive` with the wheels found, the names rejected and the light signals, and `scene_capabilities` documents the entry. Spin times radius matches the distance driven within 2 percent through a curve and a stop.
diff --git a/docs/agent-motion.md b/docs/agent-motion.md
index f3715716..d2b9786a 100644
--- a/docs/agent-motion.md
+++ b/docs/agent-motion.md
@@ -176,6 +176,12 @@ matched), `unmapped` (joints no bone claimed, up to 24, with `unmappedCount`) an
"motion": { "duration": 1, "loop": true, "layers": [{ "from": { "asset": "<id of the walking file>", "clip": 0 }, "scale": 1.1 }] }
```
+### Drive
+
+`motion.drive` (imported assets only; a placed asset takes `asset.drive`) drives a car GLB along a path: wheels found by name and geometry roll and steer with Ackermann angles, the body leans with the speed profile, legs can reverse, and tail lights glow with the brake.
+`motion.duration` may be left out and becomes the drive time.
+Fields, the `motionDrive` report of `scene_inspect` and the limits are in [vehicle.md](vehicle.md#driving-through-mcp).
+
### Foot lock
`motion.footLock` (opt-in, needs `layers`) plants the feet. The layers are sampled at 30 frames per second once, and a foot
diff --git a/docs/vehicle.md b/docs/vehicle.md
index c7d3e152..e582e273 100644
--- a/docs/vehicle.md
+++ b/docs/vehicle.md
@@ -137,11 +137,62 @@ Use the headlight poses to place spot lights; the pose carries no renderer state
For imported GLBs, wheel and body nodes must be editable (positive translation, rotation, scale), and children of a moved node follow it.
Skinned vehicles are not driven by this addon.
+## Driving through MCP
+
+An agent drives a car GLB without code. The same entry works in two places.
+
+- An asset recipe (`scene_create_asset`) takes `motion.drive`. `motion.duration` may be left out and becomes the drive time; the car holds the end of its path afterwards.
+- A placed asset in a recipe scene takes `asset: { id, drive }`, with the path in the placement's own frame and scene time as drive time. The placement gets its own decoded copy, like one that plays a clip. Tail-light glow is not available there.
+
+```json
+{
+ "motion": {
+ "drive": {
+ "legs": [
+ { "path": [[0, 0, 0], [0, 0, 10], [4, 0, 16]] },
+ { "path": [[4, 0, 16], [4, 0, 12]], "direction": "reverse", "pause": 0.5 }
+ ],
+ "cruiseSpeed": 6, "acceleration": 3, "braking": 4,
+ "wheels": "auto",
+ "lights": {
+ "headlights": [{ "id": "left", "position": [0.5, 0.7, 2] }],
+ "tailLights": [{ "material": "material-1", "signal": "brake", "off": 0.2, "on": 6 }]
+ }
+ }
+ }
+}
+```
+
+| Field | Meaning |
+| --- | --- |
+| `path` or `legs` | Exactly one. A path is `[x, y, z]` points or a camera path `{ duration, eye: { keyframes } }`. A leg is `{ path, direction: 'forward' \| 'reverse', pause }` |
+| `speed` or `cruiseSpeed` | As in `driveAlongPath`, with `acceleration`, `braking`, `startSpeed`, `endSpeed`, `maxLateralAcceleration`, `startTime`, `fullBrakeDeceleration` |
+| `wheels` | `'auto'` (default) or `[{ id, node \| nodes, side, axle, radius, center, steerable, spinSign }]`, where nodes are labels or `node-N` ids |
+| `body`, `forwardAxis`, `wheelbase`, `maxSteerAngle`, `steering`, `rollGain`, `pitchGain` | As in `createVehicleRig` and `driveAlongPath` (angles in radians) |
+| `lights.headlights` | `[{ id, position, direction }]` in model space; poses are reported, place a spot light from them |
+| `lights.tailLights` | `[{ material, signal: 'brake' \| 'reverse', off, on }]`: the material id (the lamp needs an `emissiveColor`) gets `emission` from `off` to `on` with the signal |
+
+Nodes that the drive moves (the body roots and the wheels) must not also be keyed by `tracks`.
+Sampling is a pure function of time, so `scene_review`, renders and seeks in any order show the same pose.
+
+`scene_inspect kind: "hierarchy"` on an asset recipe returns `motionDrive`; on a recipe scene each entry of `assets` has `drive`:
+
+```json
+{ "wheels": [{ "id": "front-left", "side": "left", "axle": "front", "radius": 0.35, "steerable": true, "nodes": ["Wheel_FL"] }],
+ "rejectedNames": [{ "index": 7, "name": "Steering_Wheel_Cover", "reason": "contains \"steering\", which does not roll with the wheel" }],
+ "wheelbase": 2.4, "trackWidth": 1.4, "duration": 6.67, "length": 34.7,
+ "lights": { "brake": 0, "reverse": 0, "headlights": [{ "id": "left", "position": [0.5, 0.7, 2], "direction": [0, 0, 1] }] },
+ "speed": 0, "direction": 1 }
+```
+
+`wheels`, `rejectedNames`, `wheelbase`, `trackWidth`, `duration` and `length` describe the rig; `lights`, `speed` and `direction` are read at the current motion time.
+Check `wheels` and `rejectedNames` before trusting `'auto'` on an unfamiliar model.
+
## Limits
- Kinematic only: no collisions, tire slip or jumps. Heading follows the path tangent (plus a half turn while reversing).
- Ground is sampled under the wheels as a plane fit, so sharp bumps narrower than the wheelbase show as suspension offsets, clamped to `suspensionTravel`.
- Rear-steered wheels get no Ackermann angle: the split is measured from the rear axle.
-- The MCP `motion.drive` field is not wired yet; the addon is code-only for now.
+- Through MCP the tail lights drive materials of an asset recipe only (placed assets share their materials), and `groundHeight` is not available: the car follows the path heights.
- Nodes with a nonzero scene-graph `pivot` are rejected.
- Wheel radius from geometry assumes the wheel is round in the plane across its axle.
diff --git a/scripts/mcp-server.mjs b/scripts/mcp-server.mjs
index 3724319a..de45676a 100644
--- a/scripts/mcp-server.mjs
+++ b/scripts/mcp-server.mjs
@@ -28,7 +28,7 @@ const skinSpec = z.strictObject({
maxDistance: positive.optional().describe('Metres from the nearest bone beyond which a vertex counts as unweighted in scene_inspect rig. Defaults to a quarter of the rig size.'),
}).describe('Bind this mesh node to an armature with automatic weights: each vertex follows its nearest bones, by distance to the bone segments. Needs the armature at rest where the mesh is.');
const relationList = z.array(z.record(z.string(), z.unknown())).max(8).describe('Relations against other mesh nodes; fields in scene_capabilities relations. [] removes them.').meta({ id: 'Relations' });
-const placedAsset = z.record(z.string(), z.unknown()).describe('Catalog GLB placed at this group node: {id,materials,clip}; see scene_capabilities placedAssets.').meta({ id: 'PlacedAsset' });
+const placedAsset = z.record(z.string(), z.unknown()).describe('Catalog GLB placed at this group node: {id,materials,clip,drive}; see scene_capabilities placedAssets.').meta({ id: 'PlacedAsset' });
const nodeFields = {
relations: relationList.optional(),
asset: placedAsset.optional(),
@@ -512,7 +512,7 @@ const cloth = z.strictObject({
}).describe('Baked at 60 fps, seeked by motionTime. Units, defaults: scene_capabilities cloth.');
// The full motion schema (tracks, ik, constraints, drivers) is in scene_create's recipe. scene_apply and asset recipes
// take the same object, validated by the backend, without repeating about 35 KB of schema in each tool.
-const recipeMotionReference = z.record(z.string(), z.unknown()).describe('The same object as recipe.motion in scene_create: duration, loop, tracks, ik, constraints, drivers. Read that tool for the fields. On an imported asset it also takes layers, up to 8 clips played at motion time before tracks and ik: { clip (scene_inspect animations id), mask: { joints, descendants, exclude } of node IDs, weight, speed, offset, loop, additive, reference, bones: { clip node: asset node } to retarget by name, morph } or from: { asset, clip } for a clip of another catalog asset retargeted onto this humanoid (scale, rootMotion). motion also takes footLock (true or { contactHeight, contactSpeed, blend, ground }) and { kind: "morph", id: target name, property: "weight" } tracks. A masked layer waves while another walks.');
+const recipeMotionReference = z.record(z.string(), z.unknown()).describe('The same object as recipe.motion in scene_create: duration, loop, tracks, ik, constraints, drivers. Read that tool for the fields. On an imported asset it also takes layers, up to 8 clips played at motion time before tracks and ik: { clip (scene_inspect animations id), mask: { joints, descendants, exclude } of node IDs, weight, speed, offset, loop, additive, reference, bones: { clip node: asset node } to retarget by name, morph } or from: { asset, clip } for a clip of another catalog asset retargeted onto this humanoid (scale, rootMotion). motion also takes drive (a car along a path with Ackermann steering, reversing legs and lights; see scene_capabilities motionDrive; scene_inspect motionDrive) and footLock (true or { contactHeight, contactSpeed, blend, ground }) and { kind: "morph", id: target name, property: "weight" } tracks. A masked layer waves while another walks.');
const boneRecord = z.strictObject({
name: z.string().min(1).max(64).describe('Unique within the armature. Humanoid rigs use the canonical names (hips, spine, chest, neck, head, leftUpperArm, leftLowerArm, leftHand, leftUpperLeg, leftLowerLeg, leftFoot, leftToes, and right...) so retarget works.'),
parent: z.string().min(1).max(64).nullable().optional().describe('Parent bone name, or null for a root bone.'),
diff --git a/scripts/smoke-agent-drive.mjs b/scripts/smoke-agent-drive.mjs
new file mode 100644
index 00000000..6fc07110
--- /dev/null
+++ b/scripts/smoke-agent-drive.mjs
@@ -0,0 +1,92 @@
+// Through MCP on the CPU backend: a four-wheel car GLB driven along a curve with a reverse leg, once as an asset recipe
+// (motion.drive) and once placed in a scene (asset.drive). Checks the wheels found, the brake signal and that a seek
+// reproduces its frame, and writes stills for a person to look at.
+import assert from 'node:assert/strict';
+import { mkdir, writeFile } from 'node:fs/promises';
+import { constants, setPriority } from 'node:os';
+import { resolve } from 'node:path';
+import { Client } from '@modelcontextprotocol/client';
+import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
+import { carAsset } from '../tests/fixtures/car-glb.js';
+
+setPriority(0, constants.priority.PRIORITY_BELOW_NORMAL);
+await mkdir('artifacts/drive-proof', { recursive: true });
+await writeFile('artifacts/drive-proof/car.glb', Buffer.from(carAsset()));
+const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve('scripts/mcp-server.mjs'), '--backend', 'cpu', '--memory', '192', '--assets', resolve('artifacts/drive-proof')],
+ stderr: 'pipe', cwd: process.cwd(),
+});
+let diagnostics = '';
+transport.stderr.on('data', chunk => { diagnostics += chunk.toString(); });
+const client = new Client({ name: '67js-drive-proof', version: '1' }, {
+ versionNegotiation: { mode: { pin: '2026-07-28' }, probe: { timeoutMs: 180000 } },
+});
+async function call(name, argumentsValue = {}) {
+ const result = await client.callTool({ name, arguments: argumentsValue }, { timeout: 180000 });
+ assert.equal(result.isError, undefined, result.content?.find(item => item.type === 'text')?.text);
+ return result;
+}
+async function look(sessionId, label, motionTime) {
+ const result = await call('scene_look', { sessionId, width: 768, height: 512, motionTime });
+ const bytes = Buffer.from(result.content.find(item => item.type === 'image').data, 'base64');
+ await writeFile(`artifacts/drive-${label}.png`, bytes);
+ return bytes;
+}
+
+const drive = {
+ legs: [
+ { path: [[0, 0, -6], [0, 0, 2], [3, 0, 7], [8, 0, 9]] },
+ { path: [[8, 0, 9], [5, 0, 8]], direction: 'reverse', pause: 0.4 },
+ ],
+ cruiseSpeed: 6, acceleration: 3, braking: 4,
+ lights: { headlights: [{ id: 'left', position: [0.5, 0.7, 2] }], tailLights: [{ material: 'material-1', signal: 'brake', off: 0.2, on: 6 }] },
+};
+const camera = { eye: [-6, 7, -9], target: [4, 0, 3], fov: 50 };
+
+try {
+ await client.connect(transport, { timeout: 180000 });
+ assert.equal((await client.listTools()).tools.length, 22);
+ const catalog = (await call('scene_asset_catalog')).structuredContent;
+ const car = catalog.assets.find(asset => asset.name === 'car.glb').assetId;
+
+ const created = (await call('scene_create_asset', { recipe: { format: '67js.asset', schemaVersion: 1, assetId: car, motion: { drive }, camera } })).structuredContent;
+ const sessionId = created.sessionId;
+ const duration = (await call('scene_export_recipe', { sessionId })).structuredContent.recipe.motion.duration;
+ const report = (await call('scene_inspect', { sessionId, kind: 'hierarchy', limit: 1 })).structuredContent.motionDrive;
+ assert.equal(report.wheels.length, 4);
+ assert.deepEqual(report.rejectedNames.map(entry => entry.name), ['Steering_Wheel_Cover']);
+ const times = { start: 0, middle: duration * 0.35, stopped: duration * 0.62, reversing: duration - 0.5, end: duration };
+ const frames = {};
+ for (const [label, time] of Object.entries(times)) frames[label] = await look(sessionId, `asset-${label}`, time);
+ assert.notDeepEqual(frames.start, frames.middle, 'the car moves');
+ assert.deepEqual(await look(sessionId, 'asset-middle-again', times.middle), frames.middle, 'a seek reproduces its frame');
+ await call('scene_dispose', { sessionId });
+
+ const placed = (await call('scene_create', { recipe: {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'ground', type: 'box' }],
+ materials: [{ id: 'asphalt', color: [0.12, 0.12, 0.13], roughness: 0.9 }],
+ nodes: [
+ { id: 'road', geometry: 'ground', material: 'asphalt', translation: [4, -0.05, 2], scale: [30, 0.1, 30] },
+ { id: 'car', translation: [0, 0, 0], asset: { id: car, drive: { ...drive, lights: { headlights: drive.lights.headlights } } } },
+ ],
+ camera,
+ } })).structuredContent;
+ const placedId = placed.sessionId;
+ const first = await look(placedId, 'placed-start', 0);
+ const later = await look(placedId, 'placed-middle', duration * 0.35);
+ assert.notDeepEqual(first, later, 'the placed car moves');
+ const inspection = (await call('scene_inspect', { sessionId: placedId })).structuredContent;
+ assert.equal(inspection.assets[0].drive.wheels.length, 4);
+ await call('scene_dispose', { sessionId: placedId });
+
+ const proof = { backend: 'cpu', wheels: report.wheels.length, rejected: report.rejectedNames.length, duration, seekExact: true };
+ await writeFile('artifacts/drive-cpu.json', JSON.stringify(proof, null, 2));
+ console.log(JSON.stringify(proof));
+} catch (error) {
+ console.error(diagnostics);
+ throw error;
+} finally {
+ await client.close();
+}
diff --git a/src/agent-asset.js b/src/agent-asset.js
index d5cd47ad..59f46fcd 100644
--- a/src/agent-asset.js
+++ b/src/agent-asset.js
@@ -36,6 +36,7 @@ export const AGENT_ASSET_CAPABILITIES = Object.freeze({
'set-camera', 'set-environment', 'set-effects', 'set-shadow',
'set-root', 'set-animation', 'set-material', 'set-asset-node', 'set-motion', 'set-physics', 'edit-mesh',
]),
+ motionDrive: 'drive {path | legs [{path, direction forward|reverse, pause}], speed | cruiseSpeed (+ acceleration, braking, maxLateralAcceleration), wheels auto | [{id,node|nodes,side,axle,radius,steerable,spinSign}], body, forwardAxis, wheelbase, steering ackermann|shared, lights {headlights [{id,position,direction}], tailLights [{material,signal brake|reverse,off,on}]}}: a car driven along a path (points or a camera path), wheels rolling and steering, tail lights glowing with the brake. duration may be left out: it becomes the drive time. scene_inspect motionDrive lists the wheels found, rejectedNames and the light signals at the motion time. See docs/vehicle.md',
ragdoll: 'physics {duration, floor?, ragdoll {}}: joints fall limp onto a floor at world height floor (default 0)',
boundsKind: 'authored-base-world',
});
@@ -507,6 +508,16 @@ export function createAgentAssetScene(asset, inputRecipe, { sources } = {}) {
return copiedJson(result);
}
+ /** The drive of motion.drive for this asset. Built once per distinct entry: the rig and the speed profile cost a simulation. */
+ const builtDrives = new Map();
+ function buildDrive(drive) {
+ const key = JSON.stringify(drive);
+ if (!builtDrives.has(key)) {
+ builtDrives.set(key, agentSceneModule('drive').createAssetDrive(asset, drive, { materialIds: new Set(materialRecords.keys()) }));
+ }
+ return builtDrives.get(key);
+ }
+
function motionRecipe(candidate) {
const baselines = new Map([...materialRecords].map(([id, record]) => [id, { id, ...record.baseline }]));
for (const override of candidate.materialOverrides) {
@@ -516,7 +527,7 @@ export function createAgentAssetScene(asset, inputRecipe, { sources } = {}) {
return {
nodes: [...sourceNodes.values()].filter(node => node.editable),
materials: [...baselines.values()], camera: candidate.camera,
- ikHierarchy: [...sourceNodes.values()], clipLayerContext, morphTargets: [...morphTargetIndex.keys()],
+ buildDrive, ikHierarchy: [...sourceNodes.values()], clipLayerContext, morphTargets: [...morphTargetIndex.keys()],
ikRootMatrix: Array.from(mat4.fromRotationTranslationScale(mat4.create(), root.rotation, root.translation, [root.scale[0], root.scale[0], root.scale[0]])),
};
}
@@ -781,6 +792,13 @@ export function createAgentAssetScene(asset, inputRecipe, { sources } = {}) {
};
}
+ /** scene_inspect's motionDrive: the wheels found, the names rejected and the light signals at the current motion time. */
+ function motionDriveReport() {
+ if (!recipe.motion?.drive) return {};
+ const drive = buildDrive(recipe.motion.drive);
+ return { motionDrive: drive.describe(motionTime === null ? undefined : drive.sample(motionTime).pose) };
+ }
+
function motionIkReport() {
if (!recipe.motion?.ik?.length) return {};
const sweeper = agentMotion.compileAgentMotion(recipe.motion, motionRecipe(recipe));
@@ -971,7 +989,7 @@ export function createAgentAssetScene(asset, inputRecipe, { sources } = {}) {
return sourceNodes.get(id);
});
const nodes = structuredClone(selected.slice(offset, offset + limit));
- return { revision, topologyRevision, assetId: canonicalAssetId, kind: 'hierarchy', ...motionIkReport(), ...motionLayerReport(), ...diagnosticsReport(),
+ return { revision, topologyRevision, assetId: canonicalAssetId, kind: 'hierarchy', ...motionIkReport(), ...motionDriveReport(), ...motionLayerReport(), ...diagnosticsReport(),
counts: { nodes: sourceNodes.size, joints: [...sourceNodes.values()].filter(node => node.joint).length },
nodes, offset, truncated: offset + nodes.length < selected.length,
motion: recipe.motion ? { duration: recipe.motion.duration, loop: recipe.motion.loop, tracks: recipe.motion.tracks.length, time: motionTime } : null };
diff --git a/src/agent-drive-schema.js b/src/agent-drive-schema.js
new file mode 100644
index 00000000..8610e463
--- /dev/null
+++ b/src/agent-drive-schema.js
@@ -0,0 +1,127 @@
+// Shape checks for the `drive` entry of an asset recipe's motion and of a placed asset. Pure validation with no vehicle
+// code, so the recipe normalizers and the MCP server can use it without loading the drive (agent-drive.js).
+
+export const DRIVE_LIMITS = Object.freeze({ maxPathPoints: 512, maxLegs: 32, maxWheels: 8, maxNodeReferences: 64, maxLights: 16 });
+
+const DRIVE_FIELDS = [
+ 'path', 'legs', 'speed', 'cruiseSpeed', 'acceleration', 'braking', 'startSpeed', 'endSpeed', 'maxLateralAcceleration',
+ 'startTime', 'steering', 'maxSteerAngle', 'fullBrakeDeceleration', 'rollGain', 'pitchGain',
+ 'wheels', 'body', 'forwardAxis', 'wheelbase', 'lights',
+];
+const NUMBER_OPTIONS = DRIVE_FIELDS.slice(2, 9).concat(['startTime', 'maxSteerAngle', 'fullBrakeDeceleration', 'rollGain', 'pitchGain', 'wheelbase']);
+const WHEEL_FIELDS = ['id', 'node', 'nodes', 'side', 'axle', 'radius', 'center', 'steerable', 'spinSign'];
+const LIGHT_FIELDS = ['headlights', 'tailLights'];
+const HEADLIGHT_FIELDS = ['id', 'position', 'direction'];
+const TAIL_LIGHT_FIELDS = ['id', 'material', 'signal', 'off', 'on'];
+const CAMERA_PATH_FIELDS = ['duration', 'loop', 'eye'];
+
+function knownKeys(value, allowed, label) {
+ if (!value || typeof value !== 'object' || Array.isArray(value)) throw new TypeError(`${label} must be an object.`);
+ for (const key of Object.keys(value)) {
+ if (!allowed.includes(key)) throw new TypeError(`Unknown ${label} field: ${key}. Allowed fields: ${allowed.join(', ')}.`);
+ }
+}
+
+function vector(value, label) {
+ if (!Array.isArray(value) || value.length !== 3 || value.some(component => !Number.isFinite(component))) {
+ throw new TypeError(`${label} must be three finite numbers.`);
+ }
+ return [...value];
+}
+
+function nodeReference(value, label) {
+ if (typeof value !== 'string' || !value || value.length > 256) throw new TypeError(`${label} must be a node label or id of 1 to 256 characters.`);
+ return value;
+}
+
+function normalizePath(path, label) {
+ if (Array.isArray(path)) {
+ if (path.length < 2 || path.length > DRIVE_LIMITS.maxPathPoints) {
+ throw new RangeError(`${label} needs 2 to ${DRIVE_LIMITS.maxPathPoints} points.`);
+ }
+ return path.map((point, index) => vector(point, `${label} point ${index}`));
+ }
+ knownKeys(path, CAMERA_PATH_FIELDS, label);
+ if (!path.eye || !Array.isArray(path.eye.keyframes) || path.eye.keyframes.length > DRIVE_LIMITS.maxPathPoints) {
+ throw new TypeError(`${label} is an array of [x, y, z] points or a camera path { duration, eye: { keyframes } }.`);
+ }
+ return structuredClone(path);
+}
+
+function normalizeWheels(wheels) {
+ if (wheels === 'auto') return wheels;
+ if (!Array.isArray(wheels) || wheels.length < 2 || wheels.length > DRIVE_LIMITS.maxWheels) {
+ throw new RangeError(`wheels is 'auto' or a list of 2 to ${DRIVE_LIMITS.maxWheels} wheels.`);
+ }
+ return wheels.map((wheel, index) => {
+ knownKeys(wheel, WHEEL_FIELDS, `wheel ${index}`);
+ const references = wheel.nodes ?? [wheel.node];
+ if (!Array.isArray(references) || !references.length || references.length > DRIVE_LIMITS.maxNodeReferences) {
+ throw new TypeError(`wheel ${index} needs a node, or nodes listing tire, hub and bolts.`);
+ }
+ references.forEach(reference => nodeReference(reference, `wheel ${index} node`));
+ return structuredClone(wheel);
+ });
+}
+
+function normalizeLights(lights) {
+ knownKeys(lights, LIGHT_FIELDS, 'drive lights');
+ const result = {};
+ if (lights.headlights !== undefined) {
+ if (!Array.isArray(lights.headlights) || lights.headlights.length > DRIVE_LIMITS.maxLights) throw new RangeError(`headlights lists at most ${DRIVE_LIMITS.maxLights} lamps.`);
+ result.headlights = lights.headlights.map((light, index) => {
+ knownKeys(light, HEADLIGHT_FIELDS, `headlight ${index}`);
+ const lamp = { position: vector(light.position, `headlight ${index} position`) };
+ if (light.id !== undefined) lamp.id = String(light.id);
+ if (light.direction !== undefined) lamp.direction = vector(light.direction, `headlight ${index} direction`);
+ return lamp;
+ });
+ }
+ if (lights.tailLights !== undefined) {
+ if (!Array.isArray(lights.tailLights) || lights.tailLights.length > DRIVE_LIMITS.maxLights) throw new RangeError(`tailLights lists at most ${DRIVE_LIMITS.maxLights} lamps.`);
+ result.tailLights = lights.tailLights.map((light, index) => {
+ knownKeys(light, TAIL_LIGHT_FIELDS, `tail light ${index}`);
+ if (typeof light.material !== 'string' || !light.material) throw new TypeError(`tail light ${index} needs the material id whose emission it drives.`);
+ if (light.signal !== 'brake' && light.signal !== 'reverse') throw new RangeError(`tail light ${index} signal must be brake or reverse.`);
+ return structuredClone(light);
+ });
+ }
+ return result;
+}
+
+/** Validate the shape of a drive entry and return its canonical JSON. Numbers are checked when the drive is built. */
+export function normalizeDrive(input) {
+ knownKeys(input, DRIVE_FIELDS, 'drive');
+ if ((input.path === undefined) === (input.legs === undefined)) throw new TypeError('A drive needs exactly one of path or legs.');
+ const result = {};
+ if (input.path !== undefined) result.path = normalizePath(input.path, 'drive path');
+ if (input.legs !== undefined) {
+ if (!Array.isArray(input.legs) || !input.legs.length || input.legs.length > DRIVE_LIMITS.maxLegs) throw new RangeError(`legs lists 1 to ${DRIVE_LIMITS.maxLegs} legs.`);
+ result.legs = input.legs.map((leg, index) => {
+ knownKeys(leg, ['path', 'direction', 'pause'], `leg ${index}`);
+ if (leg.direction !== undefined && leg.direction !== 'forward' && leg.direction !== 'reverse') throw new RangeError(`leg ${index} direction must be forward or reverse.`);
+ return { ...structuredClone(leg), path: normalizePath(leg.path, `leg ${index} path`) };
+ });
+ }
+ for (const field of NUMBER_OPTIONS) {
+ if (input[field] !== undefined) {
+ if (!Number.isFinite(input[field])) throw new TypeError(`drive ${field} must be a finite number.`);
+ result[field] = input[field];
+ }
+ }
+ if (input.steering !== undefined) {
+ if (input.steering !== 'ackermann' && input.steering !== 'shared') throw new RangeError("drive steering must be 'ackermann' or 'shared'.");
+ result.steering = input.steering;
+ }
+ if (input.forwardAxis !== undefined) {
+ if (!['+x', '-x', '+z', '-z'].includes(input.forwardAxis)) throw new RangeError('drive forwardAxis must be +x, -x, +z or -z.');
+ result.forwardAxis = input.forwardAxis;
+ }
+ if (input.wheels !== undefined) result.wheels = normalizeWheels(input.wheels);
+ if (input.body !== undefined) {
+ if (!Array.isArray(input.body) || !input.body.length || input.body.length > DRIVE_LIMITS.maxNodeReferences) throw new RangeError(`drive body lists 1 to ${DRIVE_LIMITS.maxNodeReferences} nodes.`);
+ result.body = input.body.map(reference => nodeReference(reference, 'drive body node'));
+ }
+ if (input.lights !== undefined) result.lights = normalizeLights(input.lights);
+ return result;
+}
diff --git a/src/agent-drive.js b/src/agent-drive.js
new file mode 100644
index 00000000..39e54129
--- /dev/null
+++ b/src/agent-drive.js
@@ -0,0 +1,67 @@
+import { createCameraPath } from './camera-path.js';
+import { createVehicleRig, driveAlongLegs, createVehicleNodeOverrides } from './vehicle.js';
+
+// The `drive` entry of an asset recipe's motion and of a placed asset: a car driven along a path by the vehicle kit.
+// Loaded on demand (module 'drive'); sampling is a pure function of time, so frames render in any order.
+
+function resolvePath(path) {
+ return Array.isArray(path) ? path : createCameraPath({ loop: false, ...path });
+}
+
+function rounded(value) {
+ return Number(value.toFixed(5));
+}
+
+/**
+ * Builds the rig and the drive for a decoded asset. `materialIds` are the ids a tail light may drive.
+ * `sample(time)` returns the pose, the node overrides that put the asset in it, and the tail-light emission per material.
+ */
+export function createAssetDrive(asset, drive, { materialIds = null } = {}) {
+ const { path, legs, wheels, body, forwardAxis, wheelbase, lights = {}, ...driveOptions } = drive;
+ const rig = createVehicleRig({
+ asset, wheels, body, forwardAxis, wheelbase,
+ maxSteerAngle: driveOptions.maxSteerAngle,
+ lights: { headlights: lights.headlights },
+ });
+ for (const tail of lights.tailLights ?? []) {
+ if (materialIds && !materialIds.has(tail.material)) throw new RangeError(`Tail light material was not found: ${tail.material}.`);
+ }
+ const vehicle = driveAlongLegs(rig, (legs ?? [{ path }]).map(leg => ({ ...leg, path: resolvePath(leg.path) })), driveOptions);
+ const tailLights = lights.tailLights ?? [];
+ return Object.freeze({
+ rig,
+ duration: vehicle.duration,
+ length: vehicle.length,
+ sample(time) {
+ const pose = vehicle.sample(time);
+ return {
+ pose,
+ nodeOverrides: createVehicleNodeOverrides(rig, pose),
+ emissions: tailLights.map(tail => ({ material: tail.material, emission: (tail.off ?? 0) + ((tail.on ?? 4) - (tail.off ?? 0)) * pose.lights[tail.signal] })),
+ };
+ },
+ /** What scene_inspect reports: the wheels found, the names turned down, and the light signals of a pose. */
+ describe(pose) {
+ return {
+ wheels: rig.wheels.map(wheel => ({
+ id: wheel.id, side: wheel.side, axle: wheel.axle, radius: rounded(wheel.radius), steerable: wheel.steerable,
+ nodes: wheel.entries.map(entry => entry.label),
+ })),
+ rejectedNames: rig.rejectedNames,
+ wheelbase: rounded(rig.wheelbase),
+ trackWidth: rounded(rig.trackWidth),
+ duration: rounded(vehicle.duration),
+ length: rounded(vehicle.length),
+ ...(pose ? {
+ lights: {
+ brake: rounded(pose.lights.brake),
+ reverse: pose.lights.reverse,
+ headlights: pose.lights.headlights.map(light => ({ id: light.id, position: light.position.map(rounded), direction: light.direction.map(rounded) })),
+ },
+ speed: rounded(pose.speed),
+ direction: pose.direction,
+ } : {}),
+ };
+ },
+ });
+}
diff --git a/src/agent-motion.js b/src/agent-motion.js
index a82ea539..4a29cbd6 100644
--- a/src/agent-motion.js
+++ b/src/agent-motion.js
@@ -5,6 +5,7 @@ import { createNodeConstraintEvaluator, normalizeConstraints, normalizeDrivers }
import { normalizeClipLayers } from './agent-clip-layers.js';
import { normalizeKeyframeEasing } from './easing.js';
import { normalizeFootLock } from './foot-lock.js';
+import { normalizeDrive } from './agent-drive-schema.js';
import { bounded, denseArray, finite, knownKeys, valueFrom } from './motion-validation.js';
const PROPERTIES = {
@@ -279,20 +280,31 @@ function ikKeyframeCount(ik) {
}
/** Normalize bounded authored data without storing bindings or sampled poses. */
+/** The canonical drive entry and the drive built from it, which proves the wheels, path and speeds are usable. */
+function normalizedDrive(input, recipe) {
+ if (!recipe.buildDrive) {
+ throw new RangeError('motion.drive moves an imported asset recipe. For an asset placed in a scene, give the placement a drive: { asset: { id, drive } }.');
+ }
+ const canonical = normalizeDrive(input);
+ return { canonical, built: recipe.buildDrive(canonical) };
+}
+
export function normalizeAgentMotion(input, recipe) {
if (input === null || input === undefined) {
return null;
}
- knownKeys(input, ['duration', 'loop', 'tracks', 'ik', 'constraints', 'drivers', 'layers', 'footLock'], 'motion');
- const duration = bounded(input.duration, 0.1, 30, 'Motion duration');
+ knownKeys(input, ['duration', 'loop', 'tracks', 'ik', 'constraints', 'drivers', 'layers', 'footLock', 'drive'], 'motion');
+ const drive = input.drive === undefined ? null : normalizedDrive(input.drive, recipe);
+ // A drive knows how long its path takes, so the motion may leave the duration out.
+ const duration = bounded(input.duration === undefined && drive ? Math.ceil(drive.built.duration * 1000) / 1000 : input.duration, 0.1, 30, 'Motion duration');
const loop = input.loop === undefined ? false : input.loop;
if (typeof loop !== 'boolean') {
throw new TypeError('Motion loop must be boolean.');
}
const inputTracks = input.tracks ?? [];
denseArray(inputTracks, 128, 'Motion tracks');
- if (!inputTracks.length && !input.layers?.length && !input.ik?.length && !input.constraints?.length && !input.drivers?.length) {
- throw new RangeError('Motion needs at least one track, IK solver, constraint or driver. Use null to clear it.');
+ if (!inputTracks.length && !input.layers?.length && !input.ik?.length && !input.constraints?.length && !input.drivers?.length && !drive) {
+ throw new RangeError('Motion needs at least one track, IK solver, constraint, driver or drive. Use null to clear it.');
}
const nodes = new Set((recipe.nodes ?? []).map(node => node.id));
const materials = new Set((recipe.materials ?? []).map(material => material.id));
@@ -349,6 +361,9 @@ export function normalizeAgentMotion(input, recipe) {
return result;
});
const result = { duration, loop, tracks };
+ if (drive) {
+ result.drive = drive.canonical;
+ }
if (input.layers !== undefined) {
result.layers = normalizeClipLayers(input.layers, recipe.clipLayerContext);
}
@@ -566,6 +581,14 @@ export function compileAgentMotion(input, recipe) {
if (!output.materials.has(target.id)) output.materials.set(target.id, {});
output.materials.get(target.id)[target.property] = target.component === undefined ? 0 : new Float64Array(3);
}
+ const driveSampler = motion.drive ? recipe.buildDrive(motion.drive) : null;
+ if (driveSampler) {
+ // The drive touches the same nodes and tail-light materials at every time, so its patches are allocated once.
+ for (const { id } of driveSampler.sample(0).nodeOverrides) {
+ output.nodes.set(id, { translation: new Float64Array(3), rotation: new Float64Array(4), scale: new Float64Array(3) });
+ }
+ for (const { material } of motion.drive.lights?.tailLights ?? []) output.materials.set(material, { emission: 0 });
+ }
const baselines = new Map((recipe.materials ?? []).map(material => [material.id, structuredClone(material)]));
const cameraTracks = motion.tracks.filter(track => track.kind === 'camera');
const cameraInput = {
@@ -596,6 +619,7 @@ export function compileAgentMotion(input, recipe) {
}
return Object.freeze({
ikUsesStagedPose,
+ driveSampler,
/**
* Solve IK, then drivers and constraints, over a pose an asset staged for this time ({ id, translation, rotation,
* scale } per node). Returns the node overrides to add: rotations for IK nodes, whole transforms for constrained ones.
@@ -656,6 +680,7 @@ export function compileAgentMotion(input, recipe) {
materialSamples.get(track.id)[track.property] = value;
}
}
+ const driven = driveSampler?.sample(time);
if (!ikUsesStagedPose) ikEvaluator?.solve(time);
const evaluated = constraintEvaluator?.evaluate(time, ikUsesStagedPose ? trackBases() : poseBases());
const drivenValues = [];
@@ -697,6 +722,13 @@ export function compileAgentMotion(input, recipe) {
nodePatch.scale.set(frame.scale);
}
}
+ for (const override of driven?.nodeOverrides ?? []) {
+ const nodePatch = output.nodes.get(override.id);
+ nodePatch.translation.set(override.translation);
+ nodePatch.rotation.set(override.rotation);
+ nodePatch.scale.set(override.scale);
+ }
+ for (const { material, emission } of driven?.emissions ?? []) output.materials.get(material).emission = emission;
for (const { target, value } of drivenValues) {
const materialPatch = output.materials.get(target.id);
if (typeof value === 'number') materialPatch[target.property] = value;
diff --git a/src/agent-scene-assets.js b/src/agent-scene-assets.js
index b11a0a0c..17a10332 100644
--- a/src/agent-scene-assets.js
+++ b/src/agent-scene-assets.js
@@ -1,4 +1,5 @@
import { mat4 } from 'gl-matrix';
+import { normalizeDrive } from './agent-drive-schema.js';
// Placing decoded glTF assets inside a procedural recipe scene. A node with an `asset` field is a group: the scene
// turns each mesh part of the asset into a generated child node, so terrain, lights, procedural meshes and any number of
@@ -9,7 +10,7 @@ import { mat4 } from 'gl-matrix';
export const ASSET_PLACEMENT_LIMITS = Object.freeze({ maxPlacements: 512, maxAssets: 16, maxAnimatedPlacements: 16, maxOverrides: 32 });
const ASSET_ID = /^[a-f0-9]{64}$/;
-const PLACEMENT_FIELDS = ['id', 'materials', 'clip'];
+const PLACEMENT_FIELDS = ['id', 'materials', 'clip', 'drive'];
const CLIP_FIELDS = ['index', 'start', 'speed', 'loop'];
const COLOR_FIELDS = ['color', 'emissiveColor'];
const UNIT_FIELDS = ['metallic', 'roughness', 'opacity'];
@@ -85,6 +86,13 @@ export function normalizeAssetPlacement(value) {
if (typeof loop !== 'boolean') throw new TypeError('asset clip loop must be boolean.');
result.clip = { index, start, speed, loop };
}
+ if (value.drive !== undefined) {
+ if (value.clip !== undefined) throw new TypeError('An asset placement plays a clip or drives along a path, not both.');
+ result.drive = normalizeDrive(value.drive);
+ if (result.drive.lights?.tailLights) {
+ throw new RangeError('Placed assets share their materials, so tail lights glow only in an asset recipe (motion.drive). Headlight and brake signals are still reported by scene_inspect.');
+ }
+ }
return result;
}
@@ -114,7 +122,7 @@ export function recipeAssetNeeds(value) {
for (const placement of placements) {
const id = placement.id.toLowerCase();
const need = needs.get(id) ?? { id, copies: 1 };
- if (placement.clip) need.copies++;
+ if (placement.clip || placement.drive) need.copies++;
needs.set(id, need);
}
return [...needs.values()];
@@ -157,9 +165,9 @@ function materialOutput(source, fields) {
/**
* Generate the geometries, materials and child nodes of every placement into a normalized recipe, in place.
- * `assets` maps assetCopyKey(id, copy) to a decoded parseGltf asset. Returns what the scene keeps outside the recipe JSON.
+ * `assets` maps assetCopyKey(id, copy) to a decoded parseGltf asset; `createAssetDrive` is the loaded drive module's builder. Returns what the scene keeps outside the recipe JSON.
*/
-export function expandAssetPlacements(recipe, assets, limits, outputCache = new Map()) {
+export function expandAssetPlacements(recipe, assets, limits, outputCache = new Map(), createAssetDrive = null) {
const placed = recipe.nodes.filter(node => node.asset);
const runtime = { placements: [], geometryParts: new Map(), materialOutputs: new Map(), partExtras: new Map(), animators: [] };
if (!placed.length) return runtime;
@@ -167,7 +175,7 @@ export function expandAssetPlacements(recipe, assets, limits, outputCache = new
throw new Error('This recipe places catalog assets, which a host loads and decodes first. Pass them as the assets option of createAgentScene; MCP does this for scene_create.');
}
const distinct = new Set(placed.map(node => node.asset.id));
- const animated = placed.filter(node => node.asset.clip);
+ const animated = placed.filter(node => node.asset.clip || node.asset.drive);
if (placed.length > ASSET_PLACEMENT_LIMITS.maxPlacements || distinct.size > ASSET_PLACEMENT_LIMITS.maxAssets ||
animated.length > ASSET_PLACEMENT_LIMITS.maxAnimatedPlacements) {
throw new RangeError(`A scene places at most ${ASSET_PLACEMENT_LIMITS.maxPlacements} assets, ${ASSET_PLACEMENT_LIMITS.maxAssets} different ones, ${ASSET_PLACEMENT_LIMITS.maxAnimatedPlacements} of them playing a clip; this recipe places ${placed.length}, ${distinct.size} and ${animated.length}.`);
@@ -177,9 +185,9 @@ export function expandAssetPlacements(recipe, assets, limits, outputCache = new
const materialIds = new Map();
const generatedNodes = [];
for (const node of placed) {
- const { id: assetId, materials: overrides = [], clip } = node.asset;
+ const { id: assetId, materials: overrides = [], clip, drive } = node.asset;
let copy = 0;
- if (clip) {
+ if (clip || drive) {
copy = (copyCounter.get(assetId) ?? 0) + 1;
copyCounter.set(assetId, copy);
}
@@ -230,9 +238,9 @@ export function expandAssetPlacements(recipe, assets, limits, outputCache = new
...(part.castShadow !== undefined ? { castShadow: part.castShadow } : {}),
});
});
- const record = { node: node.id, assetId, copy, asset, partNodeIds, clip: clip ?? null };
+ const record = { node: node.id, assetId, copy, asset, partNodeIds, clip: clip ?? null, drive: drive ? createDrive(node.id, createAssetDrive, asset, drive) : null, drivePose: null };
runtime.placements.push(record);
- if (clip) runtime.animators.push(record);
+ if (clip || drive) runtime.animators.push(record);
}
if (recipe.nodes.length + generatedNodes.length > limits.maxNodes) {
throw new RangeError(`Placed assets expand to ${generatedNodes.length} part nodes, which with the ${recipe.nodes.length} authored nodes exceeds maxNodes ${limits.maxNodes}. Place fewer copies or simpler assets.`);
@@ -241,6 +249,16 @@ export function expandAssetPlacements(recipe, assets, limits, outputCache = new
return runtime;
}
+function createDrive(nodeId, createAssetDrive, asset, drive) {
+ if (!createAssetDrive) throw new Error('This recipe drives a placed asset, which needs the drive code: await loadAgentSceneModules(recipe) before createAgentScene or apply.');
+ try {
+ return createAssetDrive(asset, drive);
+ } catch (error) {
+ error.message = `Node ${nodeId} drive: ${error.message}`;
+ throw error;
+ }
+}
+
/** The time inside a clip for a scene time: held at the first frame before `start`, then looped or held at the end. */
export function clipTimeAt(clip, duration, time) {
const elapsed = Math.max(0, ((time ?? 0) - clip.start) * clip.speed);
@@ -257,8 +275,15 @@ export function stageAssetClips(runtime, graph, time) {
const matrices = new Map();
try {
for (const record of runtime.animators) {
- const duration = record.asset.animations[record.clip.index].duration;
- const transaction = record.asset.stagePose({ animation: { clip: record.clip.index, time: clipTimeAt(record.clip, duration, time) }, nodeOverrides: [] });
+ let transaction;
+ if (record.drive) {
+ const driven = record.drive.sample(time ?? 0);
+ record.drivePose = driven.pose;
+ transaction = record.asset.stagePose({ nodeOverrides: driven.nodeOverrides });
+ } else {
+ const duration = record.asset.animations[record.clip.index].duration;
+ transaction = record.asset.stagePose({ animation: { clip: record.clip.index, time: clipTimeAt(record.clip, duration, time) }, nodeOverrides: [] });
+ }
transactions.push(transaction);
const placement = graph.getNode(record.node).worldMatrix;
record.partNodeIds.forEach((id, index) => {
@@ -300,6 +325,7 @@ export function describeAssetPlacements(runtime, graph, geometryBounds) {
clips: record.asset.animations.map((animation, index) => ({ index, name: animation.name ?? `clip ${index}`, duration: animation.duration })),
materials: [...new Set(record.asset.parts.map(part => part.material.name).filter(Boolean))],
...(record.clip ? { playing: record.clip } : {}),
+ ...(record.drive ? { drive: record.drive.describe(record.drivePose ?? undefined) } : {}),
};
});
}
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index 15eb2cb0..8bde13bc 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -155,7 +155,7 @@ export const AGENT_SCENE_CAPABILITIES = Object.freeze({
format: '67js.scene', schemaVersion: 1,
geometryTypes: Object.freeze(['box', 'cylinder', 'sphere', 'torus', 'lathe', 'extrude', 'tube', 'curve', 'text', 'grid', 'terrain', 'mesh']),
terrain: 'One mesh, x and z from -size/2 to size/2, hills about y = 0. size 40 m (1-2000), resolution 64 cells (2-256), seed 1, amplitude 1 m (0-200), holes up to 16 of {center:[x,z],radius} with no surface inside; radius a few cells or more.',
- placedAssets: 'Node field asset:{id,materials?,clip?} turns a group node into a catalog GLB (id from asset_catalog; scene_create_asset is one asset alone). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start, on a decoded copy of its own; scene_apply set-motion tracks move the node as usual and scene time seeks the clip. Copies of one asset share geometry and materials and draw as one batch. scene_inspect assets lists id, bounds, clips and material names. At most 512 placements, 16 assets, 16 playing a clip. Render jobs do not draw placed assets yet; scene_look does.',
+ placedAssets: 'Node field asset:{id,materials?,clip?,drive?} turns a group node into a catalog GLB (id from asset_catalog; scene_create_asset is one asset alone). Its transform places it. materials:[{name,color,metallic,roughness,opacity,emission,emissiveColor}] override source materials by name. clip:{index,start=0,speed=1,loop=true} plays that clip from scene time start, on a decoded copy of its own; scene_apply set-motion tracks move the node as usual and scene time seeks the clip. Copies of one asset share geometry and materials and draw as one batch. drive:{path | legs, cruiseSpeed | speed, wheels (auto or a list), lights:{headlights}} drives a car along a path in the placement frame (scene time is drive time; same fields as an asset recipe motion.drive, see docs/vehicle.md), also on its own copy; tail-light glow needs an asset recipe. scene_inspect assets lists id, bounds, clips, material names and the wheels of a drive, rejectedNames and light signals. At most 512 placements, 16 assets, 16 playing a clip or driving. Render jobs do not draw placed assets yet; scene_look does.',
navigation: 'scene_inspect path:{ground,from:[x,z],to:[x,z]} returns a route over the ground node around its holes, steep slopes and the nodes in avoid:[ids] (box footprints), as keyframes [{time,value:[x,y,z]}] for a node translation track with interpolation LINEAR. radius 0.3 m is how far the walker position stays from holes and avoided nodes; maxSlope 40 degrees; cellSize defaults to radius; give seconds for the whole walk or speed (1.4 m/s); lift raises y, such as half the walker height. Ground and avoided nodes are read at their authored pose. Errors name an unwalkable start or goal, or one that cannot be reached.',
meshEdits: Object.freeze({ operations: Object.freeze(Object.keys(MESH_EDIT_OPERATIONS)), maxEdits: MAX_MESH_EDITS }),
meshStack: Object.freeze({ operations: STACK_OPERATIONS, maxNodes: MAX_STACK_NODES }),
diff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.js
index 7c8b355d..bbf99bd9 100644
--- a/src/agent-scene-modules.js
+++ b/src/agent-scene-modules.js
@@ -8,6 +8,7 @@ const MODULE_LOADERS = {
modeling: () => import('./modeling.js'),
text: () => import('./text-geometry.js'),
motion: () => import('./agent-motion.js'),
+ drive: () => import('./agent-drive.js'),
physics: () => import('./agent-physics.js'),
cloth: () => import('./agent-cloth.js'),
particles: () => import('./agent-particles.js'),
@@ -40,6 +41,7 @@ const MODULE_MARKERS = {
modeling: ['"type":"lathe"', '"type":"extrude"', '"type":"tube"', '"type":"curve"'],
text: ['"type":"text"'],
motion: ['"motion":{'],
+ drive: ['"drive":{'],
physics: ['"physics":{'],
cloth: ['"cloth":{'],
particles: ['"particles":{'],
diff --git a/src/agent-scene.js b/src/agent-scene.js
index e255d4d9..9692f0bd 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -1016,7 +1016,7 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
function stage(inputRecipe, preferred = null, removedNodes = new Set(), removedMaterials = new Set()) {
const recipe = normalizeRecipe(inputRecipe, limits);
const assetRuntime = recipe.nodes.some(node => node.asset)
- ? agentSceneModule('assets').expandAssetPlacements(recipe, options.assets, limits, assetOutputCache)
+ ? agentSceneModule('assets').expandAssetPlacements(recipe, options.assets, limits, assetOutputCache, recipe.nodes.some(node => node.asset?.drive) ? agentSceneModule('drive').createAssetDrive : null)
: NO_ASSETS;
if (state) {
retainedMatrices(state.parts);
diff --git a/tests/agent-drive.test.js b/tests/agent-drive.test.js
new file mode 100644
index 00000000..68108ed7
--- /dev/null
+++ b/tests/agent-drive.test.js
@@ -0,0 +1,189 @@
+import './support/load-agent-scene-modules.mjs';
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { mat3, quat, vec3 } from 'gl-matrix';
+import { parseGltf } from '../src/gltf.js';
+import { createAgentAssetScene } from '../src/agent-asset.js';
+import { createAgentScene } from '../src/agent-scene.js';
+import { createAssetDrive } from '../src/agent-drive.js';
+import { assetCopyKey, recipeAssetNeeds } from '../src/agent-scene-assets.js';
+import { carAsset, CAR_WHEEL_RADIUS, CAR_WHEELBASE, CAR_TRACK } from './fixtures/car-glb.js';
+
+const CURVE = [[0, 0, 0], [0, 0, 12], [6, 0, 22], [16, 0, 26]];
+const ASSET_ID = 'ca'.repeat(32);
+// Part order of the fixture: body, tail lamp, then the front left, front right, rear left and rear right wheels.
+const REAR_WHEEL_PARTS = [4, 5];
+const TAIL_LAMP_MATERIAL = 'material-1';
+
+const assetRecipe = drive => ({ format: '67js.asset', schemaVersion: 1, assetId: ASSET_ID, motion: { drive } });
+const motionDrive = scene => scene.inspect({ kind: 'hierarchy', limit: 1 }).motionDrive;
+
+async function withCar(run) {
+ const asset = await parseGltf(carAsset());
+ try {
+ return await run(asset);
+ } finally {
+ asset.dispose();
+ }
+}
+
+/** Rotation of a part's world matrix about its own x axle between two poses: how far the wheel rolled. */
+function rolledAngle(before, after) {
+ const rotationBefore = mat3.fromMat4(mat3.create(), before);
+ const rotationAfter = mat3.fromMat4(mat3.create(), after);
+ const change = mat3.multiply(mat3.create(), rotationAfter, mat3.transpose(mat3.create(), rotationBefore));
+ const rotation = quat.fromMat3(quat.create(), change);
+ const axis = vec3.create();
+ let angle = quat.getAxisAngle(axis, rotation);
+ if (angle > Math.PI) angle -= 2 * Math.PI;
+ return angle * vec3.dot(axis, [before[0], before[1], before[2]]);
+}
+
+/** Total roll of the rear wheels over [from, to], summed in small steps, as metres rolled. */
+function metresRolled(scene, from, to, step = 1 / 60) {
+ scene.sampleMotion(from);
+ let previous = REAR_WHEEL_PARTS.map(index => Float64Array.from(scene.parts[index].matrix));
+ let rolled = 0;
+ for (let time = from + step; time < to + step / 2; time += step) {
+ scene.sampleMotion(Math.min(time, to));
+ const current = REAR_WHEEL_PARTS.map(index => Float64Array.from(scene.parts[index].matrix));
+ rolled += REAR_WHEEL_PARTS.reduce((total, _, index) => total + rolledAngle(previous[index], current[index]), 0) / REAR_WHEEL_PARTS.length;
+ previous = current;
+ }
+ return rolled * CAR_WHEEL_RADIUS;
+}
+
+test('the drive finds the wheels, reports the names it rejected and sets the motion duration from the path', async () => {
+ await withCar(asset => {
+ const scene = createAgentAssetScene(asset, assetRecipe({ path: CURVE, cruiseSpeed: 8 }));
+ const report = motionDrive(scene);
+ assert.deepEqual(report.wheels.map(wheel => wheel.nodes[0]).sort(), ['Wheel_FL', 'Wheel_FR', 'Wheel_RL', 'Wheel_RR']);
+ assert.deepEqual(report.rejectedNames.map(entry => entry.name), ['Steering_Wheel_Cover']);
+ assert.ok(Math.abs(report.wheelbase - CAR_WHEELBASE) < 0.01);
+ assert.ok(Math.abs(report.trackWidth - CAR_TRACK) < 0.01);
+ assert.ok(Math.abs(report.wheels[0].radius - CAR_WHEEL_RADIUS) < 0.01);
+ assert.equal(scene.exportRecipe().motion.duration, Math.ceil(report.duration * 1000) / 1000);
+ });
+});
+
+test('through a curve and a stop the rear wheels roll the distance driven, and any sampling order gives one pose', async () => {
+ await withCar(asset => {
+ const drive = { path: CURVE, cruiseSpeed: 8, acceleration: 3, braking: 4 };
+ const scene = createAgentAssetScene(asset, assetRecipe(drive));
+ const reference = createAssetDrive(asset, drive);
+ const duration = scene.exportRecipe().motion.duration;
+ const rolled = metresRolled(scene, 0, duration);
+ const driven = reference.sample(duration).pose.distance;
+ assert.ok(driven > 30, `the path is ${driven} m`);
+ assert.ok(Math.abs(rolled - driven) / driven < 0.02, `rolled ${rolled} m for ${driven} m driven`);
+ const snapshot = time => {
+ scene.sampleMotion(time);
+ return scene.parts.map(part => Array.from(part.matrix));
+ };
+ const middle = snapshot(duration * 0.4);
+ snapshot(duration);
+ snapshot(0);
+ assert.deepEqual(snapshot(duration * 0.4), middle);
+ scene.sampleMotion(duration);
+ const end = scene.parts[0].matrix;
+ assert.ok(Math.hypot(end[12] - 16, end[14] - 26) < 1.5, 'the car ends at the end of the path');
+ });
+});
+
+test('a reverse leg backs up with the wheels spinning backwards', async () => {
+ await withCar(asset => {
+ const drive = {
+ cruiseSpeed: 4, acceleration: 2, braking: 2,
+ legs: [{ path: [[0, 0, 0], [0, 0, 10]] }, { path: [[0, 0, 10], [0, 0, 4]], direction: 'reverse', pause: 0.5 }],
+ };
+ const scene = createAgentAssetScene(asset, assetRecipe(drive));
+ const reference = createAssetDrive(asset, drive);
+ const duration = scene.exportRecipe().motion.duration;
+ scene.sampleMotion(duration);
+ assert.equal(motionDrive(scene).direction, -1);
+ assert.equal(motionDrive(scene).lights.reverse, 0, 'the reverse light is off once the car has stopped');
+ assert.ok(reference.sample(duration - 0.9).pose.speed < 0);
+ scene.sampleMotion(duration - 0.9);
+ assert.equal(motionDrive(scene).direction, -1);
+ assert.equal(motionDrive(scene).lights.reverse, 1);
+ const backed = metresRolled(scene, duration - 3, duration);
+ const distance = reference.sample(duration).pose.distance - reference.sample(duration - 3).pose.distance;
+ assert.ok(distance < -1 && backed < -1, `drove ${distance} m, rolled ${backed} m`);
+ assert.ok(Math.abs(backed - distance) / Math.abs(distance) < 0.02);
+ const finalPose = scene.parts[0].matrix;
+ assert.ok(Math.abs(finalPose[14] - 4) < 0.3, `ends at z 4, not ${finalPose[14]}`);
+ });
+});
+
+test('the brake signal is on only while the car slows, and the tail lamp glows with it', async () => {
+ await withCar(asset => {
+ const drive = {
+ path: CURVE, cruiseSpeed: 8, acceleration: 3, braking: 4,
+ lights: { tailLights: [{ material: TAIL_LAMP_MATERIAL, signal: 'brake', off: 0.2, on: 6 }], headlights: [{ id: 'left', position: [0.5, 0.6, 2], direction: [0, 0, 1] }] },
+ };
+ const scene = createAgentAssetScene(asset, assetRecipe(drive));
+ const reference = createAssetDrive(asset, drive);
+ const duration = scene.exportRecipe().motion.duration;
+ const lamp = scene.parts[1].material;
+ let braking = 0;
+ let coasting = 0;
+ for (let time = 0.1; time < duration - 0.1; time += 0.05) {
+ const slowing = reference.sample(time + 0.01).pose.speed - reference.sample(time - 0.01).pose.speed;
+ scene.sampleMotion(time);
+ const signal = motionDrive(scene).lights.brake;
+ if (slowing < -0.01) {
+ braking++;
+ assert.ok(signal > 0, `braking at ${time} s shows ${signal}`);
+ assert.ok(Math.abs(lamp.emission - (0.2 + 5.8 * signal)) < 1e-3);
+ } else if (slowing > -1e-6) {
+ coasting++;
+ assert.equal(signal, 0, `not braking at ${time} s shows ${signal}`);
+ assert.equal(lamp.emission, 0.2);
+ }
+ }
+ assert.ok(braking > 5 && coasting > 20, `${braking} braking and ${coasting} other samples`);
+ assert.equal(motionDrive(scene).lights.headlights[0].id, 'left');
+ });
+});
+
+test('a bad drive names the problem', async () => {
+ await withCar(asset => {
+ assert.throws(() => createAgentAssetScene(asset, assetRecipe({ path: CURVE })), /exactly one of speed/);
+ assert.throws(() => createAgentAssetScene(asset, assetRecipe({ path: CURVE, speed: 3, banana: 1 })), /Unknown drive field: banana/);
+ assert.throws(() => createAgentAssetScene(asset, assetRecipe({ path: CURVE, speed: 3, wheels: [{ node: 'Wheel_FL' }, { node: 'Nope' }] })), /was not found: Nope/);
+ assert.throws(() => createAgentAssetScene(asset, assetRecipe({ path: CURVE, speed: 3, lights: { tailLights: [{ material: 'material-99', signal: 'brake' }] } })), /Tail light material was not found/);
+ assert.throws(() => createAgentAssetScene(asset, assetRecipe({ path: CURVE, speed: 3 })).apply([{ type: 'set-motion', motion: { duration: 5, drive: { legs: [{ path: CURVE, direction: 'sideways' }], speed: 3 } } }], { expectedRevision: 0 }), /direction must be forward or reverse/);
+ });
+});
+
+const sceneRecipe = (drive, translation = [10, 0, 0]) => ({
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'ground', type: 'box' }],
+ materials: [{ id: 'grass', color: [0.2, 0.5, 0.2] }],
+ nodes: [
+ { id: 'floor', geometry: 'ground', material: 'grass', scale: [60, 0.1, 60] },
+ { id: 'car', translation, asset: { id: ASSET_ID, drive } },
+ ],
+});
+
+test('a placed car drives its path in the placement frame, with its own decoded copy', async () => {
+ const drive = { path: [[0, 0, 0], [0, 0, 8], [4, 0, 14]], cruiseSpeed: 6 };
+ const needs = recipeAssetNeeds(sceneRecipe(drive));
+ assert.deepEqual(needs, [{ id: ASSET_ID, copies: 2 }]);
+ const assets = new Map([[assetCopyKey(ASSET_ID, 0), await parseGltf(carAsset())], [assetCopyKey(ASSET_ID, 1), await parseGltf(carAsset())]]);
+ try {
+ const scene = createAgentScene(sceneRecipe(drive), {}, { assets });
+ const body = () => scene.parts.find(part => part.name === 'car/0').matrix;
+ scene.sampleMotion(0);
+ assert.ok(Math.abs(body()[12] - 10) < 0.2, 'the car starts at its placement');
+ scene.sampleMotion(60);
+ assert.ok(Math.abs(body()[12] - 14) < 0.5 && Math.abs(body()[14] - 14) < 0.5, `ends at ${body()[12]}, ${body()[14]}`);
+ const placed = scene.inspect({}).assets[0];
+ assert.equal(placed.drive.wheels.length, 4);
+ assert.deepEqual(placed.drive.rejectedNames.map(entry => entry.name), ['Steering_Wheel_Cover']);
+ assert.equal(placed.drive.lights.brake, 0);
+ assert.throws(() => createAgentScene(sceneRecipe({ ...drive, lights: { tailLights: [{ material: 'm', signal: 'brake' }] } }), {}, { assets }), /tail lights glow only in an asset recipe/);
+ } finally {
+ for (const asset of assets.values()) asset.dispose();
+ }
+});
diff --git a/tests/fixtures/car-glb.js b/tests/fixtures/car-glb.js
new file mode 100644
index 00000000..2e1724a3
--- /dev/null
+++ b/tests/fixtures/car-glb.js
@@ -0,0 +1,139 @@
+/**
+ * A minimal four-wheel car as a GLB: a body box, a tail-light box with its own emissive material, and four
+ * 16-sided wheels (radius 0.35 m) named like a real export. Forward is +z, wheelbase 2.4 m, track 1.4 m.
+ */
+export const CAR_WHEEL_RADIUS = 0.35;
+export const CAR_WHEELBASE = 2.4;
+export const CAR_TRACK = 1.4;
+
+const WHEEL_SEGMENTS = 16;
+const WHEEL_HALF_WIDTH = 0.12;
+
+function boxGeometry(minimum, maximum) {
+ const faces = [
+ { normal: [1, 0, 0], corners: [[1, 0, 0], [1, 1, 0], [1, 1, 1], [1, 0, 1]] },
+ { normal: [-1, 0, 0], corners: [[0, 0, 1], [0, 1, 1], [0, 1, 0], [0, 0, 0]] },
+ { normal: [0, 1, 0], corners: [[0, 1, 0], [0, 1, 1], [1, 1, 1], [1, 1, 0]] },
+ { normal: [0, -1, 0], corners: [[0, 0, 1], [0, 0, 0], [1, 0, 0], [1, 0, 1]] },
+ { normal: [0, 0, 1], corners: [[1, 0, 1], [1, 1, 1], [0, 1, 1], [0, 0, 1]] },
+ { normal: [0, 0, -1], corners: [[0, 0, 0], [0, 1, 0], [1, 1, 0], [1, 0, 0]] },
+ ];
+ const positions = [];
+ const normals = [];
+ const indices = [];
+ for (const face of faces) {
+ const first = positions.length / 3;
+ for (const corner of face.corners) {
+ positions.push(...corner.map((flag, axis) => (flag ? maximum[axis] : minimum[axis])));
+ normals.push(...face.normal);
+ }
+ indices.push(first, first + 1, first + 2, first, first + 2, first + 3);
+ }
+ return { positions, normals, indices };
+}
+
+/** A cylinder whose axis is x, so a wheel at a car's side rolls about x when the car drives along z. */
+function wheelGeometry() {
+ const positions = [];
+ const normals = [];
+ const indices = [];
+ for (let segment = 0; segment < WHEEL_SEGMENTS; segment++) {
+ const angle = (segment / WHEEL_SEGMENTS) * Math.PI * 2;
+ const y = Math.cos(angle) * CAR_WHEEL_RADIUS;
+ const z = Math.sin(angle) * CAR_WHEEL_RADIUS;
+ for (const x of [-WHEEL_HALF_WIDTH, WHEEL_HALF_WIDTH]) {
+ positions.push(x, y, z);
+ normals.push(0, Math.cos(angle), Math.sin(angle));
+ }
+ }
+ for (let segment = 0; segment < WHEEL_SEGMENTS; segment++) {
+ const next = (segment + 1) % WHEEL_SEGMENTS;
+ const a = segment * 2;
+ const b = next * 2;
+ indices.push(a, b, a + 1, a + 1, b, b + 1);
+ }
+ return { positions, normals, indices };
+}
+
+function boundsOf(positions) {
+ const minimum = [Infinity, Infinity, Infinity];
+ const maximum = [-Infinity, -Infinity, -Infinity];
+ for (let index = 0; index < positions.length; index += 3) {
+ for (let axis = 0; axis < 3; axis++) {
+ minimum[axis] = Math.min(minimum[axis], positions[index + axis]);
+ maximum[axis] = Math.max(maximum[axis], positions[index + axis]);
+ }
+ }
+ return { minimum, maximum };
+}
+
+function packGlb(document, chunks, byteLength) {
+ const json = new TextEncoder().encode(JSON.stringify(document));
+ const jsonLength = Math.ceil(json.length / 4) * 4;
+ const result = new ArrayBuffer(28 + jsonLength + byteLength);
+ const header = new DataView(result);
+ header.setUint32(0, 0x46546c67, true);
+ header.setUint32(4, 2, true);
+ header.setUint32(8, result.byteLength, true);
+ header.setUint32(12, jsonLength, true);
+ header.setUint32(16, 0x4e4f534a, true);
+ const bytes = new Uint8Array(result);
+ bytes.fill(32, 20, 20 + jsonLength);
+ bytes.set(json, 20);
+ header.setUint32(20 + jsonLength, byteLength, true);
+ header.setUint32(24 + jsonLength, 0x004e4942, true);
+ chunks.forEach((chunk, index) => {
+ bytes.set(new Uint8Array(chunk.buffer, chunk.byteOffset, chunk.byteLength), 28 + jsonLength + document.bufferViews[index].byteOffset);
+ });
+ return result;
+}
+
+export function carAsset() {
+ const chunks = [];
+ const document = {
+ asset: { version: '2.0' }, scene: 0, scenes: [{ nodes: [0] }],
+ nodes: [], meshes: [], materials: [
+ { name: 'Paint', pbrMetallicRoughness: { baseColorFactor: [0.1, 0.25, 0.7, 1], metallicFactor: 0.2, roughnessFactor: 0.4 } },
+ { name: 'Rubber', pbrMetallicRoughness: { baseColorFactor: [0.03, 0.03, 0.03, 1], metallicFactor: 0, roughnessFactor: 0.9 } },
+ { name: 'TailLamp', pbrMetallicRoughness: { baseColorFactor: [0.5, 0, 0, 1] }, emissiveFactor: [1, 0.05, 0.05] },
+ ],
+ accessors: [], bufferViews: [], buffers: [{ byteLength: 0 }],
+ };
+ let offset = 0;
+ function append(data, type, componentType = 5126) {
+ const components = { SCALAR: 1, VEC3: 3 }[type];
+ const accessor = document.accessors.length;
+ document.bufferViews.push({ buffer: 0, byteOffset: offset, byteLength: data.byteLength });
+ chunks.push(data);
+ offset += Math.ceil(data.byteLength / 4) * 4;
+ document.buffers[0].byteLength = offset;
+ document.accessors.push({ bufferView: chunks.length - 1, componentType, count: data.length / components, type });
+ return accessor;
+ }
+ function addMesh(geometry, material) {
+ const positionAccessor = append(new Float32Array(geometry.positions), 'VEC3');
+ const { minimum, maximum } = boundsOf(geometry.positions);
+ document.accessors[positionAccessor].min = minimum;
+ document.accessors[positionAccessor].max = maximum;
+ const attributes = { POSITION: positionAccessor, NORMAL: append(new Float32Array(geometry.normals), 'VEC3') };
+ const indices = append(new Uint16Array(geometry.indices), 'SCALAR', 5123);
+ document.meshes.push({ primitives: [{ attributes, indices, material }] });
+ return document.meshes.length - 1;
+ }
+ const bodyMesh = addMesh(boxGeometry([-0.8, 0.4, -2.0], [0.8, 1.1, 2.0]), 0);
+ const lampMesh = addMesh(boxGeometry([-0.6, 0.6, -0.04], [0.6, 0.8, 0.04]), 2);
+ const wheelMesh = addMesh(wheelGeometry(), 1);
+ const halfBase = CAR_WHEELBASE / 2;
+ const halfTrack = CAR_TRACK / 2;
+ document.nodes.push(
+ { name: 'Car', children: [1, 2, 3, 4, 5, 6, 7] },
+ { name: 'Body', mesh: bodyMesh },
+ { name: 'TailLamp', mesh: lampMesh, translation: [0, 0, -2.04] },
+ { name: 'Wheel_FL', mesh: wheelMesh, translation: [halfTrack, CAR_WHEEL_RADIUS, halfBase] },
+ { name: 'Wheel_FR', mesh: wheelMesh, translation: [-halfTrack, CAR_WHEEL_RADIUS, halfBase] },
+ { name: 'Wheel_RL', mesh: wheelMesh, translation: [halfTrack, CAR_WHEEL_RADIUS, -halfBase] },
+ { name: 'Wheel_RR', mesh: wheelMesh, translation: [-halfTrack, CAR_WHEEL_RADIUS, -halfBase] },
+ { name: 'Steering_Wheel_Cover', mesh: bodyMesh, translation: [0, 0.9, 0.5], scale: [0.1, 0.1, 0.1] },
+ );
+ return packGlb(document, chunks, offset);
+}