Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(modeling): found weights can be corrected, shapes make expressions, and a controller can reach and look at goals in the game

yhh8edc175a27a1
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d30beec1d..d7c32e129 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,7 +22,7 @@ published separately; it ships inside the editor.
- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.
- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.
-**AI agents can build 3D models in the editor from steps.** The `modeling` tool group writes a `.esmodel`, the steps that build a model (boxes, cylinders and floor plans, then extrude, inset, bevel, slice, array, mirror, paint), and bakes the `.esmesh` a scene draws beside it. Changing one number rebuilds it and whatever draws it follows, Undo takes both files back, and `model_views` shows every side with its silhouette, so an agent can check a shape without a scene. Blockouts, buildings and hard-surface props; and with the `organic` step, creatures, plants and rocks, as one skin over parts that melt into each other, carve, mirror and roughen. `lathe` and `loft` make curved solids from a few points: vases, pipes, handles, hulls. A `rig` of joints gets its weights found over the surface and `animations` of a few keys play as ordinary clips, so a creature can walk, with feet told where to land and a `controller` choosing which clip plays when. `subdivide` and a rounded `bevel` smooth what polygons leave hard. A `boolean` step cuts real holes through solids, and a `uv` step or `uvAtlas` lays out how a texture lands.
+**AI agents can build 3D models in the editor from steps.** The `modeling` tool group writes a `.esmodel`, the steps that build a model (boxes, cylinders and floor plans, then extrude, inset, bevel, slice, array, mirror, paint), and bakes the `.esmesh` a scene draws beside it. Changing one number rebuilds it and whatever draws it follows, Undo takes both files back, and `model_views` shows every side with its silhouette, so an agent can check a shape without a scene. Blockouts, buildings and hard-surface props; and with the `organic` step, creatures, plants and rocks, as one skin over parts that melt into each other, carve, mirror and roughen. `lathe` and `loft` make curved solids from a few points: vases, pipes, handles, hulls. A `rig` of joints gets its weights found over the surface and `animations` of a few keys play as ordinary clips, so a creature can walk, with feet told where to land and a `controller` choosing which clip plays when. `rig.adjust` corrects a weight, `shapes` make expressions a clip can move, and `controller.ik` reaches and looks at goals as the game runs. `subdivide` and a rounded `bevel` smooth what polygons leave hard. A `boolean` step cuts real holes through solids, and a `uv` step or `uvAtlas` lays out how a texture lands.
- **A model of many parts that moves: `CompositeBatch`.** Put it on the root of a ship, a vehicle or a moving platform assembled from many meshes, and while the game runs its parts are drawn in a few draws parented to the root: a mesh repeated four times or more (a deck plank, a rail post) as one instanced set that copies no vertices, every other part merged by draw state into chunks of the root's own space. Moving, turning or scaling the root moves the whole model and merges nothing again. Adding, removing, re-meshing or re-placing a part re-merges only its chunk within half a second (`CompositeBatchingRes.refresh(root)` looks at once), and a part that keeps moving inside the model is let go to draw itself. The root reports what it drew, and so does `get_stats`. On a model of 2,004 parts, the meshes handed to the renderer went from 2,003 to 5 and the CPU frame from 3.3 ms to 1.2–1.4 ms; a model of about a hundred parts costs the same either way. A part under a composite is never also statically batched.
diff --git a/desktop b/desktop
index f5d864851..f3705594c 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit f5d864851ecee37520f1992e50bac62ce9c55e55
+Subproject commit f3705594c843b4a6e38ad2ef7d2c53e8635d1feb
diff --git a/pipeline/src/modeling/index.ts b/pipeline/src/modeling/index.ts
index e6f7e1a88..81499d19d 100644
--- a/pipeline/src/modeling/index.ts
+++ b/pipeline/src/modeling/index.ts
@@ -13,4 +13,4 @@ export { needsPreparing, prepareModeling } from './booleans';
export { bakeModel, type BakedModel, type ModelReport } from './modelBake';
export type { FaceSelector } from './polyMesh';
export { DEFAULT_MODEL_VIEWS, MODEL_VIEWS, renderModelSheet, renderRigSheet, type ModelSheet, type ModelViewName, type RigViewOptions } from './modelViews';
-export { controllerOf, jointPositions, posedPositions, prefabOf, timelineOf, type AnimationSource, type ControllerSource, type RigSource } from './rig';
+export { controllerOf, jointPositions, morphedPositions, posedPositions, prefabOf, timelineOf, type AnimationSource, type ControllerSource, type RigSource, type ShapeSource } from './rig';
diff --git a/pipeline/src/modeling/modelBake.ts b/pipeline/src/modeling/modelBake.ts
index baefbac48..5e18c8c5a 100644
--- a/pipeline/src/modeling/modelBake.ts
+++ b/pipeline/src/modeling/modelBake.ts
@@ -8,10 +8,10 @@
* "2 open edges" or "this shell is inside out" gets fixed on the next call,
* where the same fault in a lit picture reads as a slightly odd shadow.
*/
-import { MeshChannel, MeshChannelType, packChannels, unwrapLightmapUV, type MeshData } from 'esengine';
+import { MeshChannel, MeshChannelType, packChannels, unwrapLightmapUV, type MeshData, type MeshMorphTargets } from 'esengine';
import { triangulateFace } from './triangulate';
import { faceUvs } from './modelUv';
-import { bindMatrices, computeWeights, resolveJoints, type RigJoint, type SkinWeights } from './rig';
+import { adjustWeights, bindMatrices, computeWeights, resolveJoints, shapeDeltas, type RigJoint, type SkinWeights } from './rig';
import { buildPolyMesh, validateModelSource, type ModelSource, type StepReport } from './modelSource';
import {
boundsOf, collectEdges, cross, dot, faceArea, faceNormal, length, normalize, sub,
@@ -52,6 +52,8 @@ export interface BakedModel {
report: ModelReport;
/** Present on a rigged model. */
rig?: { joints: RigJoint[]; skin: SkinWeights };
+ /** Present where a source has shapes: three offsets per surface vertex, for each. */
+ shapes?: { names: string[]; deltas: Float32Array[] };
/** One triangle per entry: its three positions' indices, and the face it came from. */
triangles: { v: [number, number, number]; face: number }[];
}
@@ -112,7 +114,7 @@ function cornerNormals(mesh: PolyMesh, smoothAngle: number): Vec3[][] {
}));
}
-function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number][][], source: ModelSource, rig?: BakedModel['rig']): MeshData {
+function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number][][], source: ModelSource, rig?: BakedModel['rig'], shapes?: BakedModel['shapes']): MeshData {
const { channels, vertexStride } = packChannels([
{ semantic: MeshChannel.Position, components: 3, type: MeshChannelType.Float32 },
{ semantic: MeshChannel.TexCoord0, components: 2, type: MeshChannelType.Float32 },
@@ -126,7 +128,7 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
const normals = cornerNormals(mesh, source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE);
const uvScale = source.uvScale ?? 1;
const seen = new Map<string, number>();
- const pos: number[] = [], nrm: number[] = [], uv: number[] = [], col: number[] = [], source_: number[] = [];
+ const pos: number[] = [], nrm: number[] = [], uv: number[] = [], col: number[] = [], source_: number[] = [], cornerOf: [number, number][] = [];
const indices: number[] = [];
const uvs = new Map<number, Map<number, readonly [number, number]>>();
const vertexOf = (f: number, v: number): number => {
@@ -147,6 +149,7 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
uv.push(tu, tv);
col.push(...color);
source_.push(v);
+ cornerOf.push([f, face.v.indexOf(v)]);
}
return at;
};
@@ -185,9 +188,30 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
aabbMin: [...min] as [number, number, number],
aabbMax: [...max] as [number, number, number],
...(rig ? { inverseBindMatrices: bindMatrices(rig.joints) } : {}),
+ ...(shapes ? { morph: morphOf(mesh, shapes, source_, cornerOf, normals, source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE) } : {}),
};
}
+/** Offsets for the baked vertices: positions from the surface's, normals from what the surface's corners become in that shape. */
+function morphOf(mesh: PolyMesh, shapes: NonNullable<BakedModel['shapes']>, sourceOf: readonly number[], cornerOf: readonly [number, number][], rest: Vec3[][], smoothAngle: number): MeshMorphTargets {
+ const count = sourceOf.length;
+ const deltas = new Float32Array(shapes.names.length * count * 6);
+ shapes.deltas.forEach((d, s) => {
+ const moved: PolyMesh = { ...mesh, positions: mesh.positions.map((p, v) => [p[0] + d[v * 3]!, p[1] + d[v * 3 + 1]!, p[2] + d[v * 3 + 2]!]) };
+ const normals = cornerNormals(moved, smoothAngle);
+ for (let i = 0; i < count; i++) {
+ const at = (s * count + i) * 6;
+ const v = sourceOf[i]!;
+ const [f, c] = cornerOf[i]!;
+ for (let k = 0; k < 3; k++) {
+ deltas[at + k] = d[v * 3 + k]!;
+ deltas[at + 3 + k] = normals[f]![c]![k]! - rest[f]![c]![k]!;
+ }
+ }
+ });
+ return { names: shapes.names, hasNormals: true, deltas };
+}
+
/** Gives the base UV the unwrap's: no two triangles share texels, so a texture can be painted for this one model. */
function copyLightmapUvToBase(mesh: MeshData): void {
const from = mesh.channels.find((c) => c.semantic === MeshChannel.TexCoord1)!.offset;
@@ -210,11 +234,12 @@ export function bakeModel(raw: unknown): BakedModel {
let rig: BakedModel['rig'];
if (source.rig) {
const joints = resolveJoints(source.rig);
- rig = { joints, skin: computeWeights(mesh, joints) };
+ rig = { joints, skin: adjustWeights(mesh, joints, computeWeights(mesh, joints), source.rig.adjust ?? []) };
}
- // A skinned mesh takes no lightmap: bones move it, and a bake holds still.
- const baked = toMeshData(mesh, triangles, source, rig);
- const unwrapped = rig ? baked : unwrapLightmapUV(baked).mesh;
+ const shapes = source.shapes ? { names: source.shapes.map((s) => s.name), deltas: source.shapes.map((s) => shapeDeltas(mesh, s)) } : undefined;
+ // A mesh that moves takes no lightmap: a bake holds still.
+ const baked = toMeshData(mesh, triangles, source, rig, shapes);
+ const unwrapped = rig || shapes ? baked : unwrapLightmapUV(baked).mesh;
if (source.uvAtlas) copyLightmapUvToBase(unwrapped);
const edges = [...collectEdges(mesh).values()];
@@ -260,6 +285,7 @@ export function bakeModel(raw: unknown): BakedModel {
poly: mesh,
data: unwrapped,
...(rig ? { rig } : {}),
+ ...(shapes ? { shapes } : {}),
triangles: triangles.flatMap((tris, face) => tris.map((v) => ({ v, face }))),
report: {
vertices: unwrapped.vertexCount,
diff --git a/pipeline/src/modeling/modelSource.ts b/pipeline/src/modeling/modelSource.ts
index 8bfbe67b0..fcee7c803 100644
--- a/pipeline/src/modeling/modelSource.ts
+++ b/pipeline/src/modeling/modelSource.ts
@@ -18,7 +18,7 @@ import { booleanOp } from './booleans';
import { uvStep } from './modelUv';
import { lathe, loft } from './surfaces';
import { subdivide } from './subdivide';
-import { checkAnimations, checkController, checkRig, type AnimationSource, type ControllerSource, type RigSource } from './rig';
+import { checkAnimations, checkController, checkRig, checkShapes, type AnimationSource, type ControllerSource, type RigSource, type ShapeSource } from './rig';
import { compact, type Face, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
export const MODEL_SOURCE_VERSION = 1;
@@ -43,6 +43,8 @@ export interface ModelSource {
rig?: RigSource;
/** Clips moving the rig's joints, each written as an `.estimeline`. */
animations?: AnimationSource[];
+ /** Shapes the mesh can blend toward: expressions, a mouth. */
+ shapes?: ShapeSource[];
/** Which clip plays when: an `.esanimator` the prefab's Animator runs. */
controller?: ControllerSource;
steps: ModelStep[];
@@ -256,8 +258,8 @@ export function validateModelSource(raw: unknown): ModelSource {
if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) throw new ModelBuildError('a model source must be an object with `steps`', null, null, null);
const source = raw as Record<string, unknown>;
for (const key of Object.keys(source)) {
- if (!['version', 'smoothAngle', 'uvScale', 'uvAtlas', 'rig', 'animations', 'controller', 'steps'].includes(key)) {
- throw new ModelBuildError(`unknown field "${key}"; a model source has version, smoothAngle, uvScale, uvAtlas, rig, animations, controller, steps`, null, null, null);
+ if (!['version', 'smoothAngle', 'uvScale', 'uvAtlas', 'rig', 'shapes', 'animations', 'controller', 'steps'].includes(key)) {
+ throw new ModelBuildError(`unknown field "${key}"; a model source has version, smoothAngle, uvScale, uvAtlas, rig, shapes, animations, controller, steps`, null, null, null);
}
}
if (source.version !== undefined && source.version !== MODEL_SOURCE_VERSION) {
@@ -277,12 +279,17 @@ export function validateModelSource(raw: unknown): ModelSource {
if (bad) throw new ModelBuildError(bad, null, null, null);
if (source.uvAtlas) throw new ModelBuildError('uvAtlas lays out with the lightmap unwrap, which a rigged mesh does not have: bones move it. Use uv steps', null, null, null);
}
+ if (source.shapes !== undefined) {
+ const bad = checkShapes(source.shapes);
+ if (bad) throw new ModelBuildError(bad, null, null, null);
+ if (source.uvAtlas) throw new ModelBuildError('uvAtlas lays out with the lightmap unwrap, which a morphing mesh does not have. Use uv steps', null, null, null);
+ }
if (source.animations !== undefined) {
- const bad = checkAnimations(source.animations, source.rig);
+ const bad = checkAnimations(source.animations, source.rig, source.shapes);
if (bad) throw new ModelBuildError(bad, null, null, null);
}
if (source.controller !== undefined) {
- const bad = checkController(source.controller, source.animations);
+ const bad = checkController(source.controller, source.animations, source.rig as RigSource | undefined);
if (bad) throw new ModelBuildError(bad, null, null, null);
}
if (!Array.isArray(source.steps)) throw new ModelBuildError('`steps` must be an array', null, null, null);
@@ -346,7 +353,7 @@ export type ModelEdit =
| { op: 'set'; step: string | number; fields: Record<string, unknown> }
| { op: 'insert'; step: ModelStep; before?: string | number; after?: string | number }
| { op: 'remove'; step: string | number }
- | { op: 'settings'; fields: Pick<ModelSource, 'smoothAngle' | 'uvScale' | 'uvAtlas' | 'rig' | 'animations' | 'controller'> };
+ | { op: 'settings'; fields: Pick<ModelSource, 'smoothAngle' | 'uvScale' | 'uvAtlas' | 'rig' | 'shapes' | 'animations' | 'controller'> };
function indexOfStep(source: ModelSource, ref: string | number): number {
const i = typeof ref === 'number' ? ref : source.steps.findIndex((s) => s.id === ref);
diff --git a/pipeline/src/modeling/modelViews.ts b/pipeline/src/modeling/modelViews.ts
index 3ae02f9fd..e98405107 100644
--- a/pipeline/src/modeling/modelViews.ts
+++ b/pipeline/src/modeling/modelViews.ts
@@ -15,8 +15,8 @@
* What the model looks like in the game is what `screenshot` is for.
*/
import type { BakedModel } from './modelBake';
-import { boneLines, jointColor, posedPositions, type AnimationSource } from './rig';
-import { add, boundsOf, collectEdges, cross, dot, faceNormal, normalize, selectFaces, sub, type FaceSelector, type Vec3 } from './polyMesh';
+import { boneLines, jointColor, morphedPositions, posedPositions, shapeWeightsAt, type AnimationSource } from './rig';
+import { add, boundsOf, collectEdges, cross, dot, faceNormal, normalize, selectFaces, sub, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
export const MODEL_VIEWS = {
front: { right: [1, 0, 0], up: [0, 1, 0], toward: [0, 0, 1] },
@@ -193,6 +193,8 @@ export interface RigViewOptions {
/** Draws this clip. With `times`, a strip of one view at each. */
animation?: string;
times?: readonly number[];
+ /** Shows this shape at full weight. */
+ shape?: string;
/** Colours each face by the joint moving it most, or fades all but one joint's reach. */
weights?: boolean | string;
/** Draws the bones over the shaded row. Default true on a rigged model. */
@@ -201,16 +203,17 @@ export interface RigViewOptions {
wire?: boolean;
}
-/** One sheet for a model's rig: the bind pose, a clip at some moments, or who moves what. */
+/** One sheet for a model that moves: at rest, a clip at some moments, a shape, or who moves what. */
export function renderRigSheet(model: BakedModel, animations: readonly AnimationSource[], o: RigViewOptions = {}): ModelSheet {
const panel = o.panel ?? 384;
- if (!model.rig) {
- if (o.animation || o.weights) throw new Error('this model has no rig');
- return renderModelSheet(model, o.views ?? DEFAULT_MODEL_VIEWS, panel, o.focus, o.wire === true);
- }
- const { joints, skin } = model.rig;
+ const rig = model.rig;
+ const names = model.shapes?.names ?? [];
+ if (!rig && o.weights) throw new Error('this model has no rig');
+ if (o.shape !== undefined && !names.includes(o.shape)) throw new Error(`no shape "${o.shape}"; there are ${names.join(', ') || '(none)'}`);
+ if (!rig && !names.length && o.animation) throw new Error('this model has no rig');
let drawn = model;
- if (o.weights !== undefined && o.weights !== false) {
+ if (rig && o.weights !== undefined && o.weights !== false) {
+ const { joints, skin } = rig;
const only = typeof o.weights === 'string' ? joints.findIndex((j) => j.name === o.weights) : -1;
if (typeof o.weights === 'string' && only < 0) throw new Error(`no joint "${o.weights}"; there are ${joints.map((j) => j.name).join(', ')}`);
const faces = model.poly.faces.map((face) => {
@@ -226,10 +229,14 @@ export function renderRigSheet(model: BakedModel, animations: readonly Animation
}
const anim = o.animation === undefined ? null : animations.find((a) => a.name === o.animation);
if (o.animation !== undefined && !anim) throw new Error(`no animation "${o.animation}"; there are ${animations.map((a) => a.name).join(', ') || '(none)'}`);
- const skeleton = o.skeleton !== false;
- const posed = (time: number): BakedModel => ({ ...drawn, poly: { ...drawn.poly, positions: posedPositions(model.poly, joints, skin, anim ?? null, time) } });
+ const skeleton = rig !== undefined && o.skeleton !== false;
+ const weightsAt = (time: number): number[] => (o.shape !== undefined ? names.map((n) => (n === o.shape ? 1 : 0)) : shapeWeightsAt(names, anim ?? null, time));
+ const posed = (time: number): BakedModel => {
+ const shaped: PolyMesh = model.shapes ? { ...model.poly, positions: morphedPositions(model.poly, model.shapes.deltas, weightsAt(time)) } : model.poly;
+ return { ...drawn, poly: { ...drawn.poly, positions: rig ? posedPositions(shaped, rig.joints, rig.skin, anim ?? null, time) : shaped.positions } };
+ };
if (!anim) {
- return renderModelSheet(drawn, o.views ?? DEFAULT_MODEL_VIEWS, panel, o.focus, o.wire === true, skeleton ? { overlay: boneLines(joints, null, 0) } : {});
+ return renderModelSheet(posed(0), o.views ?? DEFAULT_MODEL_VIEWS, panel, o.focus, o.wire === true, skeleton ? { overlay: boneLines(rig!.joints, null, 0) } : {});
}
const times = o.times ?? Array.from({ length: 4 }, (_, i) => (i / 4) * anim.duration);
if (times.length === 0) throw new Error('`times` names no moment');
@@ -237,10 +244,12 @@ export function renderRigSheet(model: BakedModel, animations: readonly Animation
const view = o.views?.[0] ?? 'iso';
// One frame for all of them, so what moves in the strip is the model and not the camera.
const poses = times.map(posed);
- const all = poses.flatMap((m) => m.poly.positions);
- const frame = boundsOf({ positions: all, faces: [] });
+ const framedBy = o.focus === undefined ? poses.flatMap((m) => m.poly.positions)
+ : poses.flatMap((m) => selectFaces(m.poly, o.focus!).flatMap((f) => m.poly.faces[f]!.v.map((v) => m.poly.positions[v]!)));
+ if (framedBy.length === 0) throw new Error(`focus ${JSON.stringify(o.focus)} matched no face`);
+ const frame = boundsOf({ positions: framedBy, faces: [] });
const sheets = poses.map((m, i) => renderModelSheet(m, [view], panel, undefined, o.wire === true, {
- frame, ...(skeleton ? { overlay: boneLines(joints, anim, times[i]!) } : {}),
+ frame, ...(skeleton ? { overlay: boneLines(rig!.joints, anim, times[i]!) } : {}),
}));
const width = times.length * panel + (times.length - 1);
const height = sheets[0]!.height;
diff --git a/pipeline/src/modeling/rig.ts b/pipeline/src/modeling/rig.ts
index c232f82f0..00d3cae36 100644
--- a/pipeline/src/modeling/rig.ts
+++ b/pipeline/src/modeling/rig.ts
@@ -9,7 +9,7 @@
* whether one should drag the other.
*/
import { ANIMATOR_FORMAT_VERSION, PREFAB_FORMAT_VERSION, TIMELINE_FORMAT_VERSION } from 'esengine';
-import { add, boundsOf, cross, dot, length, normalize, scale, sub, type PolyMesh, type Vec3 } from './polyMesh';
+import { add, boundsOf, cross, dot, length, normalize, scale, selectFaces, sub, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
export const MAX_RIG_JOINTS = 64;
const DEG = Math.PI / 180;
@@ -23,8 +23,28 @@ export interface RigJointSource {
mirror?: 'x' | 'y' | 'z';
}
+/** Sets how much one joint moves some of the surface, where the weights found are wrong. Later ones win. */
+export interface WeightAdjust {
+ joint: string;
+ /** 0 frees the surface of this joint, 1 binds it wholly. */
+ weight: number;
+ /** These faces. */
+ faces?: FaceSelector;
+ /** Or a ball of surface, fading out to its edge. */
+ near?: { at: Vec3; radius: number };
+}
+
+/** An entity in the rig that bends nothing: what an IK constraint reaches for or looks at, moved by a game. */
+export interface RigGoal {
+ name: string;
+ at: Vec3;
+ parent?: string;
+}
+
export interface RigSource {
joints: readonly RigJointSource[];
+ adjust?: readonly WeightAdjust[];
+ goals?: readonly RigGoal[];
}
export interface AnimationTrack {
@@ -52,7 +72,9 @@ export interface AnimationSource {
duration: number;
/** Default true. */
loop?: boolean;
- tracks: readonly AnimationTrack[];
+ tracks?: readonly AnimationTrack[];
+ /** Weights of the model's `shapes` over the clip: a blink, a mouth. `value` is [time, weight]. */
+ shapes?: readonly { name: string; value: readonly (readonly number[])[]; phase?: number; linear?: boolean }[];
}
export interface RigJoint {
@@ -264,6 +286,195 @@ export function computeWeights(mesh: PolyMesh, joints: readonly RigJoint[]): Ski
return { joints: outJoints, weights: outWeights, dominant };
}
+/** @p skin with @p adjusts applied, the rest of each touched vertex rescaled to still sum to one. */
+export function adjustWeights(mesh: PolyMesh, joints: readonly RigJoint[], skin: SkinWeights, adjusts: readonly WeightAdjust[]): SkinWeights {
+ if (adjusts.length === 0) return skin;
+ const outJoints = skin.joints.slice();
+ const outWeights = skin.weights.slice();
+ for (const adjust of adjusts) {
+ const j = joints.findIndex((x) => x.name === adjust.joint);
+ const reach = new Float32Array(mesh.positions.length);
+ if (adjust.faces !== undefined) {
+ const hit = selectFaces(mesh, adjust.faces);
+ if (hit.length === 0) throw new Error(`rig.adjust on "${adjust.joint}": \`faces\` ${JSON.stringify(adjust.faces)} matched no face`);
+ for (const f of hit) for (const v of mesh.faces[f]!.v) reach[v] = 1;
+ } else {
+ const { at, radius } = adjust.near!;
+ let any = false;
+ mesh.positions.forEach((p, v) => {
+ const t = 1 - length(sub(p, at)) / radius;
+ if (t > 0) { reach[v] = t * t * (3 - 2 * t); any = true; }
+ });
+ if (!any) throw new Error(`rig.adjust on "${adjust.joint}": no surface within ${radius} of ${JSON.stringify(at)}`);
+ }
+ for (let v = 0; v < mesh.positions.length; v++) {
+ const m = reach[v]!;
+ if (m <= 0) continue;
+ const held = new Map<number, number>();
+ for (let k = 0; k < 4; k++) if (outWeights[v * 4 + k]! > 0) held.set(outJoints[v * 4 + k]!, outWeights[v * 4 + k]!);
+ const old = held.get(j) ?? 0;
+ const next = old + (adjust.weight - old) * m;
+ const others = 1 - old;
+ held.delete(j);
+ // With nothing else holding it, what this joint gives up goes up the chain rather than vanishing.
+ if (others <= 1e-6 && next < 1) {
+ const heir = joints[j]!.parent >= 0 ? joints[j]!.parent : joints[j]!.children[0] ?? j;
+ held.set(heir, 1);
+ }
+ const total = [...held.values()].reduce((a, b) => a + b, 0);
+ const ranked = [...held.entries()].map(([joint, w]) => ({ joint, w: total > 0 ? (w / total) * (1 - next) : 0 }));
+ if (next > 0) ranked.push({ joint: j, w: next });
+ ranked.sort((a, b) => b.w - a.w);
+ const keep = ranked.slice(0, 4);
+ const sum = keep.reduce((n, r) => n + r.w, 0) || 1;
+ for (let k = 0; k < 4; k++) {
+ outJoints[v * 4 + k] = keep[k]?.joint ?? 0;
+ outWeights[v * 4 + k] = keep[k] ? keep[k]!.w / sum : 0;
+ }
+ }
+ }
+ const dominant = joints.map(() => 0);
+ for (let v = 0; v < mesh.positions.length; v++) dominant[outJoints[v * 4]!]!++;
+ return { joints: outJoints, weights: outWeights, dominant };
+}
+
+// ---- Shapes ---------------------------------------------------------------------
+
+/** One way a part of the surface moves toward a shape. */
+export interface ShapeMove {
+ faces?: FaceSelector;
+ /** A ball of surface, fading out to its edge: a smile's corner, a raised brow. */
+ near?: { at: Vec3; radius: number };
+ translate?: Vec3;
+ /** Along each vertex's own outward direction: a swelling, a pucker. */
+ push?: number;
+ /** Toward or away from the centre of what is chosen. */
+ scale?: number;
+}
+
+/** A shape the mesh can blend toward, for a face or a muscle: morph target `name`, moved by `weights` in a clip or a game. */
+export interface ShapeSource {
+ name: string;
+ moves: readonly ShapeMove[];
+}
+
+/** Each shape's offsets, three per surface vertex. */
+export function shapeDeltas(mesh: PolyMesh, shape: ShapeSource): Float32Array {
+ const deltas = new Float32Array(mesh.positions.length * 3);
+ // Outward by the surface around each vertex, which `push` needs and nothing else does.
+ let normals: Vec3[] | null = null;
+ const outward = (): Vec3[] => {
+ if (normals) return normals;
+ const sum: Vec3[] = mesh.positions.map(() => [0, 0, 0]);
+ for (const face of mesh.faces) {
+ const n = faceNormalOf(mesh, face.v);
+ for (const v of face.v) sum[v] = add(sum[v]!, n);
+ }
+ normals = sum.map(normalize);
+ return normals;
+ };
+ for (const move of shape.moves) {
+ const reach = new Float32Array(mesh.positions.length);
+ if (move.faces !== undefined) {
+ const hit = selectFaces(mesh, move.faces);
+ if (hit.length === 0) throw new Error(`shape "${shape.name}": \`faces\` ${JSON.stringify(move.faces)} matched no face`);
+ for (const f of hit) for (const v of mesh.faces[f]!.v) reach[v] = 1;
+ } else {
+ const { at, radius } = move.near!;
+ let any = false;
+ mesh.positions.forEach((p, v) => {
+ const t = 1 - length(sub(p, at)) / radius;
+ if (t > 0) { reach[v] = t * t * (3 - 2 * t); any = true; }
+ });
+ if (!any) throw new Error(`shape "${shape.name}": no surface within ${radius} of ${JSON.stringify(at)}`);
+ }
+ let centre: Vec3 = [0, 0, 0];
+ if (move.scale !== undefined) {
+ let total = 0;
+ mesh.positions.forEach((p, v) => { centre = add(centre, scale(p, reach[v]!)); total += reach[v]!; });
+ centre = scale(centre, 1 / total);
+ }
+ mesh.positions.forEach((p, v) => {
+ const m = reach[v]!;
+ if (m <= 0) return;
+ let d: Vec3 = [0, 0, 0];
+ if (move.translate) d = add(d, move.translate);
+ if (move.push !== undefined) d = add(d, scale(outward()[v]!, move.push));
+ if (move.scale !== undefined) d = add(d, scale(sub(p, centre), move.scale - 1));
+ deltas[v * 3] = deltas[v * 3]! + d[0] * m;
+ deltas[v * 3 + 1] = deltas[v * 3 + 1]! + d[1] * m;
+ deltas[v * 3 + 2] = deltas[v * 3 + 2]! + d[2] * m;
+ });
+ }
+ return deltas;
+}
+
+function faceNormalOf(mesh: PolyMesh, loop: readonly number[]): Vec3 {
+ let n: Vec3 = [0, 0, 0];
+ for (let i = 0; i < loop.length; i++) {
+ const a = mesh.positions[loop[i]!]!, b = mesh.positions[loop[(i + 1) % loop.length]!]!;
+ n = add(n, cross(a, b));
+ }
+ return n;
+}
+
+/** @p mesh's positions blended toward its shapes by @p weights. */
+export function morphedPositions(mesh: PolyMesh, deltas: readonly Float32Array[], weights: readonly number[]): Vec3[] {
+ return mesh.positions.map((p, v) => {
+ let out = p;
+ deltas.forEach((d, s) => {
+ const w = weights[s] ?? 0;
+ if (w !== 0) out = [out[0] + d[v * 3]! * w, out[1] + d[v * 3 + 1]! * w, out[2] + d[v * 3 + 2]! * w];
+ });
+ return out;
+ });
+}
+
+/** Each shape's weight at @p time in a clip. */
+export function shapeWeightsAt(names: readonly string[], animation: AnimationSource | null, time: number): number[] {
+ const loop = animation?.loop !== false;
+ return names.map((name) => {
+ const track = animation?.shapes?.find((s) => s.name === name);
+ if (!track) return 0;
+ const keys = track.value.map((k) => [k[0]!, k[1]!, 0, 0]);
+ return sample(keys, time, animation!.duration, loop, track.phase ?? 0, track.linear === true)[0];
+ });
+}
+
+/** What is wrong with a source's `shapes`, or null. */
+export function checkShapes(raw: unknown): string | null {
+ if (!Array.isArray(raw) || raw.length === 0) return 'shapes must be an array of { name, moves }';
+ const seen = new Set<string>();
+ for (const [i, s] of (raw as Record<string, unknown>[]).entries()) {
+ if (typeof s !== 'object' || s === null) return `shape ${i} must be an object`;
+ for (const key of Object.keys(s)) if (!['name', 'moves'].includes(key)) return `shape ${i} has an unknown field "${key}"; it takes name, moves`;
+ if (typeof s.name !== 'string' || !s.name) return `shape ${i} needs a name`;
+ if (seen.has(s.name)) return `there are two shapes called "${s.name}"`;
+ seen.add(s.name);
+ if (!Array.isArray(s.moves) || s.moves.length === 0) return `shape "${s.name}" needs moves`;
+ for (const m of s.moves as Record<string, unknown>[]) {
+ for (const key of Object.keys(m)) if (!['faces', 'near', 'translate', 'push', 'scale'].includes(key)) return `shape "${s.name}": a move has an unknown field "${key}"; it takes faces, near, translate, push, scale`;
+ const bad = checkRegion(m, `shape "${s.name}"`);
+ if (bad) return bad;
+ if (m.translate === undefined && m.push === undefined && m.scale === undefined) return `shape "${s.name}": a move needs one of translate, push, scale`;
+ if (m.translate !== undefined && !vec3(m.translate)) return `shape "${s.name}": translate must be [x, y, z]`;
+ if (m.push !== undefined && !isNum(m.push)) return `shape "${s.name}": push must be a number`;
+ if (m.scale !== undefined && !isNum(m.scale)) return `shape "${s.name}": scale must be a number`;
+ }
+ }
+ return null;
+}
+
+/** Faces or a ball, exactly one. */
+function checkRegion(m: Record<string, unknown>, where: string): string | null {
+ if ((m.faces === undefined) === (m.near === undefined)) return `${where}: give exactly one of faces, near`;
+ if (m.near !== undefined) {
+ const n = m.near as Record<string, unknown>;
+ if (typeof n !== 'object' || n === null || !vec3(n.at) || !isNum(n.radius) || (n.radius as number) <= 0) return `${where}: near must be { at: [x, y, z], radius }, with a positive radius`;
+ }
+ return null;
+}
+
// ---- Validation -----------------------------------------------------------------
const isNum = (v: unknown): v is number => typeof v === 'number' && Number.isFinite(v);
@@ -273,7 +484,7 @@ const vec3 = (v: unknown): boolean => Array.isArray(v) && v.length === 3 && v.ev
export function checkRig(raw: unknown): string | null {
const r = raw as { joints?: unknown };
if (typeof raw !== 'object' || raw === null || !Array.isArray(r.joints)) return 'rig must be { joints: [{ name, at, parent?, mirror? }] }';
- for (const key of Object.keys(raw)) if (key !== 'joints') return `rig has an unknown field "${key}"; it takes joints`;
+ for (const key of Object.keys(raw)) if (!['joints', 'adjust', 'goals'].includes(key)) return `rig has an unknown field "${key}"; it takes joints, adjust, goals`;
for (const [i, j] of (r.joints as Record<string, unknown>[]).entries()) {
if (typeof j !== 'object' || j === null) return `rig joint ${i} must be an object`;
for (const key of Object.keys(j)) if (!['name', 'at', 'parent', 'mirror'].includes(key)) return `rig joint ${i} has an unknown field "${key}"; a joint takes name, at, parent, mirror`;
@@ -283,11 +494,35 @@ export function checkRig(raw: unknown): string | null {
if (j.parent !== undefined && typeof j.parent !== 'string') return `rig joint "${j.name}": parent must be a joint's name`;
if (j.mirror !== undefined && !['x', 'y', 'z'].includes(j.mirror as string)) return `rig joint "${j.name}": mirror must be x, y or z`;
}
+ let names: Set<string>;
try {
- resolveJoints(raw as RigSource);
+ names = new Set(resolveJoints(raw as RigSource).map((j) => j.name));
} catch (e) {
return (e as Error).message;
}
+ const rig = raw as RigSource;
+ if (rig.adjust !== undefined) {
+ if (!Array.isArray(rig.adjust)) return 'rig.adjust must be an array';
+ for (const a of rig.adjust as unknown as Record<string, unknown>[]) {
+ for (const key of Object.keys(a)) if (!['joint', 'weight', 'faces', 'near'].includes(key)) return `rig.adjust has an unknown field "${key}"; it takes joint, weight, faces, near`;
+ if (!names.has(a.joint as string)) return `rig.adjust names no joint "${String(a.joint)}"; there are ${[...names].join(', ')}`;
+ if (!isNum(a.weight) || a.weight < 0 || a.weight > 1) return `rig.adjust on "${String(a.joint)}": weight must be 0..1`;
+ const bad = checkRegion(a, `rig.adjust on "${String(a.joint)}"`);
+ if (bad) return bad;
+ }
+ }
+ if (rig.goals !== undefined) {
+ if (!Array.isArray(rig.goals)) return 'rig.goals must be an array';
+ const taken = new Set(names);
+ for (const g of rig.goals as unknown as Record<string, unknown>[]) {
+ for (const key of Object.keys(g)) if (!['name', 'at', 'parent'].includes(key)) return `a rig goal has an unknown field "${key}"; it takes name, at, parent`;
+ if (typeof g.name !== 'string' || !g.name || /[/\\]/.test(g.name)) return 'a rig goal needs a name without a slash';
+ if (taken.has(g.name)) return `the name "${g.name}" is already taken in this rig`;
+ taken.add(g.name);
+ if (!vec3(g.at)) return `rig goal "${g.name}": at must be [x, y, z]`;
+ if (g.parent !== undefined && !names.has(g.parent as string)) return `rig goal "${g.name}": parent must be a joint; there are ${[...names].join(', ')}`;
+ }
+ }
return null;
}
@@ -305,23 +540,38 @@ function checkKeys(keys: unknown, duration: number, label: string): string | nul
}
/** What is wrong with a source's `animations`, or null. */
-export function checkAnimations(raw: unknown, rig: unknown): string | null {
+export function checkAnimations(raw: unknown, rig: unknown, shapes?: unknown): string | null {
if (!Array.isArray(raw)) return 'animations must be an array';
- if (!rig) return 'animations move joints, and this source has no rig';
- const jointList = resolveJoints(rig as RigSource);
+ if (!rig && !shapes) return 'animations move joints or shapes, and this source has no rig and no shapes';
+ const jointList = rig ? resolveJoints(rig as RigSource) : [];
+ const shapeNames = new Set(((shapes ?? []) as { name: string }[]).map((s) => s.name));
const names = new Set(jointList.map((j) => j.name));
const seen = new Set<string>();
for (const [i, a] of (raw as Record<string, unknown>[]).entries()) {
if (typeof a !== 'object' || a === null) return `animation ${i} must be an object`;
- for (const key of Object.keys(a)) if (!['name', 'duration', 'loop', 'tracks'].includes(key)) return `animation ${i} has an unknown field "${key}"; it takes name, duration, loop, tracks`;
+ for (const key of Object.keys(a)) if (!['name', 'duration', 'loop', 'tracks', 'shapes'].includes(key)) return `animation ${i} has an unknown field "${key}"; it takes name, duration, loop, tracks, shapes`;
if (typeof a.name !== 'string' || !/^[\w-]+$/.test(a.name)) return `animation ${i} needs a name of letters, digits, _ or -: it becomes a file name`;
if (seen.has(a.name)) return `there are two animations called "${a.name}"`;
seen.add(a.name);
if (!isNum(a.duration) || a.duration <= 0) return `animation "${a.name}": duration must be positive seconds`;
if (a.loop !== undefined && typeof a.loop !== 'boolean') return `animation "${a.name}": loop must be true or false`;
- if (!Array.isArray(a.tracks) || a.tracks.length === 0) return `animation "${a.name}" needs tracks`;
+ const trackList = (a.tracks ?? []) as Record<string, unknown>[];
+ const shapeList = (a.shapes ?? []) as Record<string, unknown>[];
+ if (!Array.isArray(trackList) || !Array.isArray(shapeList) || trackList.length + shapeList.length === 0) return `animation "${a.name}" needs tracks or shapes`;
+ for (const t of shapeList) {
+ for (const key of Object.keys(t)) if (!['name', 'value', 'phase', 'linear'].includes(key)) return `animation "${a.name}": a shape track has an unknown field "${key}"; it takes name, value, phase, linear`;
+ if (!shapeNames.has(t.name as string)) return `animation "${a.name}": no shape "${String(t.name)}"; there are ${[...shapeNames].join(', ') || '(none)'}`;
+ if (!Array.isArray(t.value) || t.value.length === 0) return `animation "${a.name}": shape "${t.name as string}" needs value keys of [time, weight]`;
+ let last = -Infinity;
+ for (const k of t.value as unknown[]) {
+ if (!Array.isArray(k) || k.length !== 2 || !k.every(isNum)) return `animation "${a.name}": shape "${t.name as string}": every key is [time, weight], not ${JSON.stringify(k)}`;
+ if ((k[0] as number) < 0 || (k[0] as number) > a.duration) return `animation "${a.name}": shape "${t.name as string}": time ${k[0] as number} is outside the clip's 0..${a.duration}`;
+ if ((k[0] as number) <= last) return `animation "${a.name}": shape "${t.name as string}": keys must be in time order`;
+ last = k[0] as number;
+ }
+ }
const tracked = new Set<string>();
- for (const t of a.tracks as Record<string, unknown>[]) {
+ for (const t of trackList) {
for (const key of Object.keys(t)) if (!['joint', 'rotate', 'move', 'reach', 'pole', 'linear', 'phase'].includes(key)) return `animation "${a.name}": a track has an unknown field "${key}"; it takes joint, rotate, move, reach, pole, linear, phase`;
if (!names.has(t.joint as string)) return `animation "${a.name}": no joint "${String(t.joint)}"; there are ${[...names].join(', ')}`;
if (tracked.has(t.joint as string)) return `animation "${a.name}": two tracks for joint "${t.joint as string}"; put their keys in one`;
@@ -343,12 +593,12 @@ export function checkAnimations(raw: unknown, rig: unknown): string | null {
if (t.phase !== undefined && !isNum(t.phase)) return `animation "${a.name}": phase must be a number`;
}
// A reach decides its limb, so a rotate on the two joints it turns would be silently thrown away.
- for (const t of a.tracks as AnimationTrack[]) {
+ for (const t of trackList as unknown as AnimationTrack[]) {
if (!t.reach) continue;
const foot = jointList.find((j) => j.name === t.joint)!;
const lower = jointList[foot.parent]!, upper = jointList[lower.parent]!;
for (const taken of [lower, upper]) {
- const clash = (a.tracks as AnimationTrack[]).find((o) => o.joint === taken.name && o.rotate);
+ const clash = (trackList as unknown as AnimationTrack[]).find((o) => o.joint === taken.name && o.rotate);
if (clash) return `animation "${a.name}": "${taken.name}" has rotate keys, but the reach on "${t.joint}" decides how it turns`;
}
}
@@ -460,7 +710,7 @@ function poseAt(joints: readonly RigJoint[], animation: AnimationSource | null,
const loop = animation?.loop !== false;
const at = (keys: readonly (readonly number[])[], t: AnimationTrack): Vec3 => sample(keys, time, animation!.duration, loop, t.phase ?? 0, t.linear === true);
const local: Pose['local'] = joints.map((j) => {
- const track = animation?.tracks.find((t) => t.joint === j.name);
+ const track = animation?.tracks?.find((t) => t.joint === j.name);
const euler = track?.rotate && !track.reach ? at(track.rotate, track) : [0, 0, 0] as Vec3;
const moved = track?.move ? at(track.move, track) : [0, 0, 0] as Vec3;
const rest = j.parent >= 0 ? sub(j.at, joints[j.parent]!.at) : j.at;
@@ -535,7 +785,7 @@ export function boneLines(joints: readonly RigJoint[], animation: AnimationSourc
interface Keyframe { time: number; value: number; inTangent: 0; outTangent: 0; interpolation: 'linear' }
/** A clip as an `.estimeline`: sampled, since the engine interpolates each rotation component on its own and a few keys would cut corners. */
-export function timelineOf(animation: AnimationSource, joints: readonly RigJoint[]): Record<string, unknown> {
+export function timelineOf(animation: AnimationSource, joints: readonly RigJoint[], shapeNames: readonly string[] = []): Record<string, unknown> {
const loop = animation.loop !== false;
const frames = Math.max(2, Math.ceil(animation.duration * SAMPLE_RATE) + 1);
const times = Array.from({ length: frames }, (_, i) => (i / (frames - 1)) * animation.duration);
@@ -547,7 +797,7 @@ export function timelineOf(animation: AnimationSource, joints: readonly RigJoint
// Every joint the clip turns, a reach's two bones among them: they have no keys of their own, only the solved pose.
const turned = new Set<number>();
const moved = new Set<number>();
- for (const track of animation.tracks) {
+ for (const track of animation.tracks ?? []) {
const index = joints.findIndex((j) => j.name === track.joint);
if (track.rotate || track.reach) turned.add(index);
if (track.move) moved.add(index);
@@ -579,23 +829,33 @@ export function timelineOf(animation: AnimationSource, joints: readonly RigJoint
}
return [{ type: 'property', name: joint.name, childPath: pathOf(index), component: 'Transform', channels }];
});
- return { version: TIMELINE_FORMAT_VERSION, type: 'timeline', duration: animation.duration, wrapMode: loop ? 'loop' : 'once', tracks };
+ const shaped = animation.shapes?.length ? [{
+ type: 'property', name: 'shapes', childPath: '', component: 'MeshMorph',
+ channels: animation.shapes.map((track) => ({
+ // By index: the component holds a list, and a name lives with the geometry.
+ property: `weights.${shapeNames.indexOf(track.name)}`,
+ keyframes: times.map((t) => ({ time: t, value: shapeWeightsAt([track.name], animation, t)[0]!, inTangent: 0, outTangent: 0, interpolation: 'linear' })),
+ })),
+ }] : [];
+ return { version: TIMELINE_FORMAT_VERSION, type: 'timeline', duration: animation.duration, wrapMode: loop ? 'loop' : 'once', tracks: [...tracks, ...shaped] };
}
/**
* The prefab that draws and moves it: the mesh on the root, the joints below it as
* entities named so a clip's paths find them, and a player for the first clip.
*/
-export function prefabOf(name: string, meshPath: string, joints: readonly RigJoint[], cullBackfaces: boolean, firstClip: string | null, controller: string | null = null): Record<string, unknown> {
+export function prefabOf(name: string, meshPath: string, joints: readonly RigJoint[], cullBackfaces: boolean, firstClip: string | null, controller: string | null = null, extra: { goals?: readonly RigGoal[]; shapes?: number } = {}): Record<string, unknown> {
const idOf = (i: number): string => `j${i}`;
+ const goals = extra.goals ?? [];
const entities: Record<string, unknown>[] = [{
prefabEntityId: 'root', name, parent: null,
- children: joints.flatMap((j, i) => (j.parent < 0 ? [idOf(i)] : [])),
+ children: [...joints.flatMap((j, i) => (j.parent < 0 ? [idOf(i)] : [])), ...goals.flatMap((g, i) => (g.parent === undefined ? [`g${i}`] : []))],
visible: true,
components: [
{ type: 'Transform', data: {} },
{ type: 'MeshRenderer', data: { mesh: meshPath, lit: true, opaque: true, cullBackfaces, enabled: true } },
- { type: 'MeshSkin', data: { joints: joints.map((_, i) => idOf(i)) } },
+ ...(joints.length ? [{ type: 'MeshSkin', data: { joints: joints.map((_, i) => idOf(i)) } }] : []),
+ ...(extra.shapes ? [{ type: 'MeshMorph', data: { weights: Array.from({ length: extra.shapes }, () => 0) } }] : []),
// A controller decides what plays; a player beside it would fight it for the same joints.
...(controller ? [{ type: 'Animator', data: { controller, enabled: true } }]
: firstClip ? [{ type: 'TimelinePlayer', data: { timeline: firstClip, playing: true } }] : []),
@@ -605,7 +865,15 @@ export function prefabOf(name: string, meshPath: string, joints: readonly RigJoi
const rest = j.parent >= 0 ? sub(j.at, joints[j.parent]!.at) : j.at;
entities.push({
prefabEntityId: idOf(i), name: j.name, parent: j.parent >= 0 ? idOf(j.parent) : 'root',
- children: j.children.map(idOf), visible: true,
+ children: [...j.children.map(idOf), ...goals.flatMap((g, k) => (g.parent === j.name ? [`g${k}`] : []))], visible: true,
+ components: [{ type: 'Transform', data: { position: { x: rest[0], y: rest[1], z: rest[2] } } }],
+ });
+ });
+ goals.forEach((g, i) => {
+ const parent = g.parent === undefined ? -1 : joints.findIndex((j) => j.name === g.parent);
+ const rest = parent >= 0 ? sub(g.at, joints[parent]!.at) : g.at;
+ entities.push({
+ prefabEntityId: `g${i}`, name: g.name, parent: parent >= 0 ? idOf(parent) : 'root', children: [], visible: true,
components: [{ type: 'Transform', data: { position: { x: rest[0], y: rest[1], z: rest[2] } } }],
});
});
@@ -640,13 +908,29 @@ export interface ControllerSource {
states: readonly ControllerState[];
/** Checked from every state first: a hit reaction, a death. */
any?: readonly ControllerTransition[];
+ /** Run on what the clips say, every frame: a hand reaching for a goal, a head turning to it. */
+ ik?: readonly ControllerIk[];
+}
+
+export interface ControllerIk {
+ kind: 'two-bone' | 'look-at';
+ /** The joint the chain ends at. */
+ tip: string;
+ /** A joint or a rig goal it reaches or looks at. */
+ target: string;
+ pole?: string;
+ weight?: number;
+ /** A float parameter scaling the weight, to fade it in. */
+ parameter?: string;
+ /** look-at: which of the tip's own axes points at the target. */
+ axis?: 'x' | 'y' | 'z';
}
/** What is wrong with a source's `controller`, or null. */
-export function checkController(raw: unknown, animations: unknown): string | null {
+export function checkController(raw: unknown, animations: unknown, rig?: RigSource): string | null {
const c = raw as Record<string, unknown>;
if (typeof raw !== 'object' || raw === null) return 'controller must be { parameters?, initial, states, any? }';
- for (const key of Object.keys(c)) if (!['parameters', 'initial', 'states', 'any'].includes(key)) return `controller has an unknown field "${key}"; it takes parameters, initial, states, any`;
+ for (const key of Object.keys(c)) if (!['parameters', 'initial', 'states', 'any', 'ik'].includes(key)) return `controller has an unknown field "${key}"; it takes parameters, initial, states, any, ik`;
if (!Array.isArray(animations) || animations.length === 0) return 'a controller plays animations, and this source has none';
const clips = new Set((animations as AnimationSource[]).map((a) => a.name));
const params = new Map<string, string>();
@@ -693,11 +977,42 @@ export function checkController(raw: unknown, animations: unknown): string | nul
const bad = checkTransitions(s.transitions, `state "${s.name}"`);
if (bad) return bad;
}
- return checkTransitions(c.any, 'any');
+ const anyBad = checkTransitions(c.any, 'any');
+ if (anyBad) return anyBad;
+ if (c.ik !== undefined) {
+ if (!Array.isArray(c.ik)) return 'controller.ik must be an array';
+ const list = rig ? resolveJoints(rig) : [];
+ const jointNames = new Set(list.map((j) => j.name));
+ const known = new Set([...jointNames, ...(rig?.goals ?? []).map((g) => g.name)]);
+ const every = [...known].join(', ') || '(none)';
+ for (const k of c.ik as ControllerIk[]) {
+ for (const key of Object.keys(k)) if (!['kind', 'tip', 'target', 'pole', 'weight', 'parameter', 'axis'].includes(key)) return `an ik constraint has an unknown field "${key}"; it takes kind, tip, target, pole, weight, parameter, axis`;
+ if (k.kind !== 'two-bone' && k.kind !== 'look-at') return `an ik constraint's kind ${JSON.stringify(k.kind)} must be two-bone or look-at`;
+ if (!jointNames.has(k.tip)) return `an ik constraint's tip "${k.tip}" is not a joint; there are ${[...jointNames].join(', ')}`;
+ if (!known.has(k.target)) return `an ik constraint's target "${k.target}" is not a joint or a goal; there are ${every}`;
+ if (k.pole !== undefined && !known.has(k.pole)) return `an ik constraint's pole "${k.pole}" is not a joint or a goal; there are ${every}`;
+ if (k.kind === 'two-bone') {
+ const tip = list.find((j) => j.name === k.tip)!;
+ if (tip.parent < 0 || list[tip.parent]!.parent < 0) return `two-bone ik needs two bones above "${k.tip}"`;
+ }
+ if (k.weight !== undefined && !isNum(k.weight)) return 'an ik constraint\'s weight must be a number';
+ if (k.parameter !== undefined && params.get(k.parameter) !== 'float') return `an ik constraint's parameter "${k.parameter}" must be a float parameter`;
+ if (k.axis !== undefined && !['x', 'y', 'z'].includes(k.axis)) return 'an ik constraint\'s axis must be x, y or z';
+ }
+ }
+ return null;
}
/** A controller as an `.esanimator`. @p clipFile turns an animation's name into the file it plays, as written beside this one. */
-export function controllerOf(source: ControllerSource, animations: readonly AnimationSource[], clipFile: (name: string) => string): Record<string, unknown> {
+export function controllerOf(source: ControllerSource, animations: readonly AnimationSource[], clipFile: (name: string) => string, rig?: RigSource): Record<string, unknown> {
+ const joints = rig ? resolveJoints(rig) : [];
+ const pathTo = (name: string): string => {
+ const goal = rig?.goals?.find((g) => g.name === name);
+ const owner = goal ? goal.parent : name;
+ const parts = goal ? [goal.name] : [];
+ for (let at = owner === undefined ? -1 : joints.findIndex((j) => j.name === owner); at >= 0; at = joints[at]!.parent) parts.unshift(joints[at]!.name);
+ return parts.join('/');
+ };
const motionOf = (clip: string, speed?: number): Record<string, unknown> => ({
kind: 'timeline', clip: clipFile(clip),
loop: animations.find((a) => a.name === clip)?.loop !== false,
@@ -714,6 +1029,8 @@ export function controllerOf(source: ControllerSource, animations: readonly Anim
parameters: (source.parameters ?? []).map((p) => ({ ...p })),
initialState: source.initial,
...(source.any ? { anyStateTransitions: source.any.map(transition) } : {}),
+ // An engine constraint names joints the way a clip does: by path from the root.
+ ...(source.ik ? { ik: source.ik.map((k) => ({ ...k, tip: pathTo(k.tip), target: pathTo(k.target), ...(k.pole ? { pole: pathTo(k.pole) } : {}) })) } : {}),
states: source.states.map((s, i) => ({
name: s.name, x: 80 + (i % 4) * 220, y: 80 + Math.floor(i / 4) * 140,
motion: s.blend
diff --git a/pipeline/tests/modeling-shapes-ik.test.ts b/pipeline/tests/modeling-shapes-ik.test.ts
new file mode 100644
index 000000000..995af55ea
--- /dev/null
+++ b/pipeline/tests/modeling-shapes-ik.test.ts
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/** Weights put right by hand, shapes a face can make, and the goals a controller's IK reaches for. */
+import { describe, expect, it } from 'vitest';
+import { decodeMesh, encodeMesh } from 'esengine';
+import { bakeModel, controllerOf, morphedPositions, posedPositions, prefabOf, renderRigSheet, timelineOf, validateModelSource, type AnimationSource, type BakedModel, type ModelStep, type RigSource } from '../src/modeling';
+
+const limb: ModelStep = { op: 'organic', resolution: 48, tag: 'limb', parts: [{ shape: 'chain', points: [[0, 0, 0, 0.2], [0, 2, 0, 0.2]], tag: 'limb' }, { shape: 'sphere', radius: 0.25, at: [0, 2.2, 0], tag: 'knob', blend: 0.05 }] };
+const joints: RigSource['joints'] = [
+ { name: 'base', at: [0, 0, 0] },
+ { name: 'mid', at: [0, 1, 0], parent: 'base' },
+ { name: 'tip', at: [0, 2, 0], parent: 'mid' },
+];
+const rigged = (adjust?: unknown, extra: Record<string, unknown> = {}): BakedModel => bakeModel({ version: 1, rig: { joints, ...(adjust ? { adjust } : {}) }, steps: [limb], ...extra });
+const held = (m: BakedModel, name: string, v: number): number => {
+ const j = m.rig!.joints.findIndex((x) => x.name === name);
+ let w = 0;
+ for (let k = 0; k < 4; k++) if (m.rig!.skin.joints[v * 4 + k] === j) w += m.rig!.skin.weights[v * 4 + k]!;
+ return w;
+};
+const sums = (m: BakedModel): void => {
+ for (let v = 0; v < m.poly.positions.length; v++) {
+ expect([...m.rig!.skin.weights.subarray(v * 4, v * 4 + 4)].reduce((a, b) => a + b, 0)).toBeCloseTo(1, 4);
+ }
+};
+
+describe('rig.adjust', () => {
+ it('binds faces wholly to a joint, and leaves the rest alone', () => {
+ const before = rigged();
+ const m = rigged([{ joint: 'base', weight: 1, faces: 'knob' }]);
+ sums(m);
+ const knob = new Set(m.poly.faces.filter((f) => f.tag === 'knob').flatMap((f) => f.v));
+ expect(knob.size).toBeGreaterThan(50);
+ for (const v of knob) expect(held(m, 'base', v)).toBeCloseTo(1, 5);
+ m.poly.positions.forEach((_, v) => {
+ if (!knob.has(v)) expect(held(m, 'mid', v)).toBeCloseTo(held(before, 'mid', v), 5);
+ });
+ });
+
+ it('frees faces of a joint, sharing what it held among the others', () => {
+ const m = rigged([{ joint: 'mid', weight: 0, faces: 'knob' }]);
+ sums(m);
+ const knob = new Set(m.poly.faces.filter((f) => f.tag === 'knob').flatMap((f) => f.v));
+ for (const v of knob) expect(held(m, 'mid', v)).toBeCloseTo(0, 5);
+ });
+
+ it('a sole holder freed passes its weight up the chain instead of leaving the vertex to nothing', () => {
+ const m = rigged([{ joint: 'mid', weight: 1, faces: 'knob' }, { joint: 'mid', weight: 0, faces: 'knob' }]);
+ sums(m);
+ const v = m.poly.faces.find((f) => f.tag === 'knob')!.v[0]!;
+ expect(held(m, 'base', v)).toBeCloseTo(1, 5);
+ });
+
+ it('`near` fades to its edge', () => {
+ const m = rigged([{ joint: 'tip', weight: 1, near: { at: [0, 0.5, 0.2], radius: 0.6 } }]);
+ sums(m);
+ const dist = (v: number): number => Math.hypot(...m.poly.positions[v]!.map((c, k) => c - [0, 0.5, 0.2][k]!));
+ const count = m.poly.positions.length;
+ const close = Array.from({ length: count }, (_, v) => v).filter((v) => dist(v) < 0.1);
+ const edge = Array.from({ length: count }, (_, v) => v).filter((v) => dist(v) > 0.45 && dist(v) < 0.55);
+ expect(close.length).toBeGreaterThan(0);
+ for (const v of close) expect(held(m, 'tip', v)).toBeGreaterThan(0.9);
+ for (const v of edge) expect(held(m, 'tip', v)).toBeLessThan(0.3);
+ });
+
+ it('moves the surface accordingly in a pose, and the report counts it', () => {
+ const a: AnimationSource = { name: 'turn', duration: 1, loop: false, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 90]] }] };
+ const free = rigged([{ joint: 'base', weight: 1, faces: 'knob' }], { animations: [a] });
+ const stuck = free.poly.faces.find((f) => f.tag === 'knob')!.v[0]!;
+ const after = posedPositions(free.poly, free.rig!.joints, free.rig!.skin, a, 0)[stuck]!;
+ expect(after).toEqual(free.poly.positions[stuck]);
+ expect(free.report.joints!.base!).toBeGreaterThan(rigged().report.joints!.base!);
+ });
+
+ it('refuses what it cannot apply', () => {
+ expect(() => rigged([{ joint: 'mid', weight: 1, faces: 'nope' }])).toThrow(/matched no face/);
+ expect(() => rigged([{ joint: 'mid', weight: 1, near: { at: [50, 0, 0], radius: 1 } }])).toThrow(/no surface within 1/);
+ const check = (adjust: unknown) => validateModelSource({ version: 1, rig: { joints, adjust }, steps: [limb] });
+ expect(() => check([{ joint: 'nope', weight: 1, faces: 'knob' }])).toThrow(/no joint "nope"; there are base, mid, tip/);
+ expect(() => check([{ joint: 'mid', weight: 2, faces: 'knob' }])).toThrow(/weight must be 0..1/);
+ expect(() => check([{ joint: 'mid', weight: 1 }])).toThrow(/exactly one of faces, near/);
+ expect(() => check([{ joint: 'mid', weight: 1, faces: 'knob', near: { at: [0, 0, 0], radius: 1 } }])).toThrow(/exactly one of faces, near/);
+ });
+});
+
+const ball: ModelStep = { op: 'organic', resolution: 40, tag: 'ball', parts: [{ shape: 'sphere', radius: 1, tag: 'ball' }] };
+const faced = (shapes: unknown, extra: Record<string, unknown> = {}): BakedModel => bakeModel({ version: 1, shapes, steps: [ball], ...extra });
+
+describe('shapes', () => {
+ it('a whole-face move carries every vertex by that much at weight 1, and half at half', () => {
+ const m = faced([{ name: 'up', moves: [{ faces: 'all', translate: [0, 1, 0] }] }]);
+ const full = morphedPositions(m.poly, m.shapes!.deltas, [1]);
+ const half = morphedPositions(m.poly, m.shapes!.deltas, [0.5]);
+ m.poly.positions.forEach((p, v) => {
+ expect(full[v]![1]).toBeCloseTo(p[1] + 1, 5);
+ expect(half[v]![1]).toBeCloseTo(p[1] + 0.5, 5);
+ expect(full[v]![0]).toBeCloseTo(p[0], 5);
+ });
+ });
+
+ it('`near` moves its ball and fades, leaving the far side', () => {
+ const m = faced([{ name: 'bump', moves: [{ near: { at: [0, 0, 1], radius: 0.5 }, translate: [0, 0, 0.3] }] }]);
+ const after = morphedPositions(m.poly, m.shapes!.deltas, [1]);
+ const front = m.poly.positions.findIndex((p) => p[2] > 0.99);
+ expect(after[front]![2] - m.poly.positions[front]![2]).toBeGreaterThan(0.25);
+ m.poly.positions.forEach((p, v) => {
+ if (p[2] < 0) expect(after[v]).toEqual(p);
+ });
+ });
+
+ it('`push` moves along the surface: a ball swells evenly', () => {
+ const m = faced([{ name: 'fat', moves: [{ faces: 'all', push: 0.2 }] }]);
+ const after = morphedPositions(m.poly, m.shapes!.deltas, [1]);
+ const radii = after.map((p) => Math.hypot(...p));
+ expect(Math.min(...radii)).toBeGreaterThan(1.17);
+ expect(Math.max(...radii)).toBeLessThan(1.23);
+ });
+
+ it('`scale` closes toward the middle of what is chosen', () => {
+ const m = faced([{ name: 'small', moves: [{ faces: 'all', scale: 0.5 }] }]);
+ const after = morphedPositions(m.poly, m.shapes!.deltas, [1]);
+ expect(Math.max(...after.map((p) => Math.hypot(...p)))).toBeLessThan(0.55);
+ });
+
+ it('reaches the file, per baked vertex, with normals, and takes no lightmap', () => {
+ const m = faced([{ name: 'a', moves: [{ faces: 'all', translate: [1, 0, 0] }] }, { name: 'b', moves: [{ faces: 'all', push: 0.1 }] }]);
+ const back = decodeMesh(encodeMesh(m.data));
+ expect(back.morph!.names).toEqual(['a', 'b']);
+ expect(back.morph!.hasNormals).toBe(true);
+ expect(back.morph!.deltas.length).toBe(2 * back.vertexCount * 6);
+ expect(back.morph!.deltas[0]).toBeCloseTo(1, 5);
+ // Moving a whole ball sideways turns no surface.
+ expect(Math.abs(back.morph!.deltas[3]!)).toBeLessThan(1e-5);
+ });
+
+ it('moves in a clip, by index, on the mesh\'s root', () => {
+ const a: AnimationSource = { name: 'blink', duration: 1, shapes: [{ name: 'b', value: [[0, 0], [0.5, 1]] }] };
+ const doc = timelineOf(a, [], ['a', 'b']) as { tracks: { childPath: string; component: string; channels: { property: string; keyframes: { value: number }[] }[] }[] };
+ expect(doc.tracks[0]).toMatchObject({ childPath: '', component: 'MeshMorph' });
+ expect(doc.tracks[0]!.channels[0]!.property).toBe('weights.1');
+ const values = doc.tracks[0]!.channels[0]!.keyframes.map((k) => k.value);
+ expect(Math.max(...values)).toBeCloseTo(1, 2);
+ });
+
+ it('works beside a rig: both reach the prefab, and bones carry the shape', () => {
+ const a: AnimationSource = { name: 'turn', duration: 1, loop: false, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 90]] }], shapes: [{ name: 'up', value: [[0, 1]] }] };
+ const m = bakeModel({ version: 1, rig: { joints }, shapes: [{ name: 'up', moves: [{ faces: 'knob', translate: [0, 0.5, 0] }] }], animations: [a], steps: [limb] });
+ const doc = prefabOf('X', 'x.esmesh', m.rig!.joints, true, null, null, { shapes: 1 }) as { entities: { components: { type: string; data: Record<string, unknown> }[] }[] };
+ const root = doc.entities[0]!.components;
+ expect(root.find((c) => c.type === 'MeshMorph')!.data.weights).toEqual([0]);
+ expect(root.find((c) => c.type === 'MeshSkin')).toBeDefined();
+ const decoded = decodeMesh(encodeMesh(m.data));
+ expect(decoded.morph).toBeDefined();
+ expect(decoded.inverseBindMatrices).toBeDefined();
+ });
+
+ it('stands without a rig: a prefab with no skin', () => {
+ const doc = prefabOf('X', 'x.esmesh', [], true, 'c.estimeline', null, { shapes: 2 }) as { entities: { components: { type: string; data: Record<string, unknown> }[] }[] };
+ expect(doc.entities.length).toBe(1);
+ const root = doc.entities[0]!.components;
+ expect(root.find((c) => c.type === 'MeshSkin')).toBeUndefined();
+ expect(root.find((c) => c.type === 'MeshMorph')!.data.weights).toEqual([0, 0]);
+ expect(root.find((c) => c.type === 'TimelinePlayer')).toBeDefined();
+ });
+
+ it('can be looked at, at full weight and through a clip', () => {
+ const a: AnimationSource = { name: 'pulse', duration: 1, shapes: [{ name: 'fat', value: [[0, 0], [0.5, 1]] }] };
+ const m = faced([{ name: 'fat', moves: [{ faces: 'all', push: 0.5 }] }], { animations: [a] });
+ const dark = (s: { width: number; rgba: Uint8Array }, col: number, panel: number): number => {
+ let n = 0;
+ for (let y = 0; y < panel; y++) for (let x = 0; x < panel; x++) if (s.rgba[((panel + 1 + y) * s.width + col * (panel + 1) + x) * 4]! < 60) n++;
+ return n;
+ };
+ const still = renderRigSheet(m, [a], { views: ['front'], panel: 64 });
+ expect(still.width).toBe(64);
+ const strip = renderRigSheet(m, [a], { animation: 'pulse', times: [0, 0.5], views: ['front'], panel: 64 });
+ expect(dark(strip, 1, 64)).toBeGreaterThan(dark(strip, 0, 64) * 1.5); // same frame, so the swollen one is bigger
+ expect(() => renderRigSheet(m, [a], { shape: 'nope' })).toThrow(/no shape "nope"; there are fat/);
+ });
+
+ it('refuses what it cannot make', () => {
+ const check = (shapes: unknown, extra = {}) => validateModelSource({ version: 1, shapes, steps: [ball], ...extra });
+ expect(() => check([{ name: 'a', moves: [] }])).toThrow(/needs moves/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all' }] }])).toThrow(/one of translate, push, scale/);
+ expect(() => check([{ name: 'a', moves: [{ translate: [1, 0, 0] }] }])).toThrow(/exactly one of faces, near/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all', push: 1, shove: 1 }] }])).toThrow(/unknown field "shove"/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all', push: 1 }] }, { name: 'a', moves: [{ faces: 'all', push: 1 }] }])).toThrow(/two shapes called "a"/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all', push: 1 }] }], { uvAtlas: true })).toThrow(/Use uv steps/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all', push: 1 }] }], { animations: [{ name: 'c', duration: 1, shapes: [{ name: 'b', value: [[0, 1]] }] }] })).toThrow(/no shape "b"; there are a/);
+ expect(() => check([{ name: 'a', moves: [{ faces: 'all', push: 1 }] }], { animations: [{ name: 'c', duration: 1 }] })).toThrow(/needs tracks or shapes/);
+ expect(() => faced([{ name: 'a', moves: [{ faces: 'nope', push: 1 }] }])).toThrow(/shape "a".*matched no face/);
+ });
+});
+
+describe('goals and ik', () => {
+ const rig: RigSource = { joints, goals: [{ name: 'aim', at: [1, 3, 0] }, { name: 'hold', at: [0.5, 1, 0], parent: 'mid' }] };
+ const clip: AnimationSource = { name: 'c', duration: 1, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 5]] }] };
+ const controller = { initial: 's', states: [{ name: 's', clip: 'c' }], parameters: [{ name: 'fade', type: 'float', default: 1 }], ik: [
+ { kind: 'look-at', tip: 'tip', target: 'aim', axis: 'y' },
+ { kind: 'two-bone', tip: 'tip', target: 'hold', pole: 'aim', parameter: 'fade' },
+ ] };
+
+ it('goals become entities that bend nothing: under the root, or under a joint', () => {
+ const m = rigged();
+ const doc = prefabOf('X', 'x.esmesh', m.rig!.joints, true, null, null, { goals: rig.goals! }) as { entities: { prefabEntityId: string; name: string; parent: string; children: string[]; components: { type: string; data: Record<string, unknown> }[] }[] };
+ const aim = doc.entities.find((e) => e.name === 'aim')!;
+ const hold = doc.entities.find((e) => e.name === 'hold')!;
+ expect(aim.parent).toBe('root');
+ expect(aim.components[0]!.data.position).toEqual({ x: 1, y: 3, z: 0 });
+ expect(doc.entities.find((e) => e.prefabEntityId === hold.parent)!.name).toBe('mid');
+ expect(hold.components[0]!.data.position).toEqual({ x: 0.5, y: 0, z: 0 }); // relative to mid
+ expect(doc.entities.find((e) => e.name === 'mid')!.children).toContain(hold.prefabEntityId);
+ const skin = doc.entities[0]!.components.find((c) => c.type === 'MeshSkin')!.data.joints as string[];
+ expect(skin).not.toContain(aim.prefabEntityId);
+ });
+
+ it('a controller\'s ik names joints by their path, as the engine looks them up', () => {
+ validateModelSource({ version: 1, rig, animations: [clip], controller, steps: [limb] });
+ const doc = controllerOf(controller as never, [clip], (n) => `${n}.estimeline`, rig) as { ik: Record<string, unknown>[] };
+ expect(doc.ik[0]).toEqual({ kind: 'look-at', tip: 'base/mid/tip', target: 'aim', axis: 'y' });
+ expect(doc.ik[1]).toEqual({ kind: 'two-bone', tip: 'base/mid/tip', target: 'base/mid/hold', pole: 'aim', parameter: 'fade' });
+ });
+
+ it('refuses constraints that name what is not there', () => {
+ const check = (ik: unknown, r: RigSource = rig) => validateModelSource({ version: 1, rig: r, animations: [clip], controller: { ...controller, ik }, steps: [limb] });
+ expect(() => check([{ kind: 'look-at', tip: 'aim', target: 'aim' }])).toThrow(/tip "aim" is not a joint/);
+ expect(() => check([{ kind: 'look-at', tip: 'tip', target: 'nope' }])).toThrow(/target "nope" is not a joint or a goal; there are base, mid, tip, aim, hold/);
+ expect(() => check([{ kind: 'spin', tip: 'tip', target: 'aim' }])).toThrow(/must be two-bone or look-at/);
+ expect(() => check([{ kind: 'two-bone', tip: 'mid', target: 'aim' }])).toThrow(/two bones above "mid"/);
+ expect(() => check([{ kind: 'look-at', tip: 'tip', target: 'aim', parameter: 'nope' }])).toThrow(/must be a float parameter/);
+ const dup = { joints, goals: [{ name: 'mid', at: [0, 0, 0] }] };
+ expect(() => validateModelSource({ version: 1, rig: dup, steps: [limb] })).toThrow(/"mid" is already taken/);
+ const badParent = { joints, goals: [{ name: 'g', at: [0, 0, 0], parent: 'nope' }] };
+ expect(() => validateModelSource({ version: 1, rig: badParent, steps: [limb] })).toThrow(/parent must be a joint/);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index e2e62fe65..7cf288b84 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -3866,6 +3866,10 @@ export const NOTED = {
{"note":"A sea seen from above is no longer scored with straight lines."},
"feat(mcp): an agent builds a 3D model in the editor from steps, changes it by one number, and sees every side of it":
{"note":"AI agents can build 3D models in the editor from steps."},
+ "feat(modeling): found weights can be corrected, shapes make expressions, and a controller can reach and look at goals in the game":
+ {"note":"AI agents can build 3D models in the editor from steps."},
+ "feat(mcp): model_write corrects weights, makes shapes and wires ik goals":
+ {"note":"AI agents can build 3D models in the editor from steps."},
"feat(modeling): a foot can be told where to land, a controller says which clip plays when, and surfaces can be subdivided and edges rounded":
{"note":"AI agents can build 3D models in the editor from steps."},
"feat(mcp): model_write plants feet, writes an animator, subdivides and rounds":