Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(modeling): a model can have a rig with weights found over its surface, and clips of a few keys, so what an agent builds can move

yhh37467df2e3ca
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5e25c2c25..1b102b5b9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,15 +16,36 @@ published separately; it ships inside the editor.
### Added
+- **An Ambient light can be two colours, sky and ground.** A hemisphere of light without baking a panorama: the ground colour fills what faces down, and a scene with none set looks as it did.
+- **A material can hold more than seven textures, and Water gains a foam texture.** Past seven, a material takes the texture units below 8 its shader never reads. Water's `u_foamMap` reads as coverage: thin foam keeps only the texture's densest marks and thick foam fills it.
+- **Every row of Project Settings can be read and written by id over MCP.** A partial write keeps what it did not name.
- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.
- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.
-**AI agents can build 3D models in the editor from steps.** The `modeling` tool group writes a `.esmodel`, the steps that build a model (boxes, cylinders and floor plans, then extrude, inset, bevel, slice, array, mirror, paint), and bakes the `.esmesh` a scene draws beside it. Changing one number rebuilds it and whatever draws it follows, Undo takes both files back, and `model_views` shows every side with its silhouette, so an agent can check a shape without a scene. Blockouts, buildings and hard-surface props; and with the `organic` step, creatures, plants and rocks, as one skin over parts that melt into each other, carve, mirror and roughen. `lathe` and `loft` make curved solids from a few points: vases, pipes, handles, hulls. A `boolean` step cuts real holes through solids, and a `uv` step or `uvAtlas` lays out how a texture lands.
+**AI agents can build 3D models in the editor from steps.** The `modeling` tool group writes a `.esmodel`, the steps that build a model (boxes, cylinders and floor plans, then extrude, inset, bevel, slice, array, mirror, paint), and bakes the `.esmesh` a scene draws beside it. Changing one number rebuilds it and whatever draws it follows, Undo takes both files back, and `model_views` shows every side with its silhouette, so an agent can check a shape without a scene. Blockouts, buildings and hard-surface props; and with the `organic` step, creatures, plants and rocks, as one skin over parts that melt into each other, carve, mirror and roughen. `lathe` and `loft` make curved solids from a few points: vases, pipes, handles, hulls. A `rig` of joints gets its weights found over the surface and `animations` of a few keys play as ordinary clips, so a creature can walk. A `boolean` step cuts real holes through solids, and a `uv` step or `uvAtlas` lays out how a texture lands.
- **A model of many parts that moves: `CompositeBatch`.** Put it on the root of a ship, a vehicle or a moving platform assembled from many meshes, and while the game runs its parts are drawn in a few draws parented to the root: a mesh repeated four times or more (a deck plank, a rail post) as one instanced set that copies no vertices, every other part merged by draw state into chunks of the root's own space. Moving, turning or scaling the root moves the whole model and merges nothing again. Adding, removing, re-meshing or re-placing a part re-merges only its chunk within half a second (`CompositeBatchingRes.refresh(root)` looks at once), and a part that keeps moving inside the model is let go to draw itself. The root reports what it drew, and so does `get_stats`. On a model of 2,004 parts, the meshes handed to the renderer went from 2,003 to 5 and the CPU frame from 3.3 ms to 1.2–1.4 ms; a model of about a hundred parts costs the same either way. A part under a composite is never also statically batched.
+### Changed
+
+- **Phones and tablets take Medium when a project names no level.** Full-screen water on a Snapdragon 8 Elite is 8.6 ms a frame there against 25.8 as authored, mostly by leaving four samples behind. Naming a level still wins, Off included, and the setting gains Default for naming none. Mini games keep their own defaults. Such a project on a handheld also draws without multisampling, with Medium's particle limit of 5000, three shadow cascades and streamed textures: name a level to keep what it had.
+- **The temporal smoothing of a quality level leaves orthographic cameras alone.** Medium and any level set to `temporal` put it on every camera, which softened a 2D game's sprites and text and gave it a full-screen chain. A `taa` an author lists on an orthographic camera still runs.
+- **The blank template starts in the linear pipeline.** A lit mesh in gamma clips and washes out, and blank is where a project with no other plan starts. Only the template changes: a project that says nothing stays gamma.
+
### Fixed
+- **Water costs a phone half of what it did.** A full-screen sea on a Snapdragon 8 Elite went from 45 ms to 21.5 at full quality, with the same picture. Every lit material on a phone gains part of it: the light loop read all sixteen slots for every pixel, and now stops at the lights there are. A surface can also declare `#pragma lighting direct` to compile out what it never reaches. Water does, so water takes the first sixteen lights only.
+- **A quality level can skip every effect there is.** The list was cut at thirty-two and there are forty effects now, so the last ones in name order kept running while the row showed them off.
+- **A new material no longer shows the inside of a sphere.** An opaque mesh tests and writes depth whatever its material says, and a new surface material tests depth.
+- **A scene view no longer goes black for good after a heightmap loads.** A texture made or rewritten from JS left the device's unit bound differently from what the renderer believed.
+- **A blended pane in front of water is drawn over it.** With no post-process chain the water follows the camera, sorted nearest and covered what stood in front of it.
+- **A restarted scene draws its images again instead of white.** A reload released a handle the scene still named.
+- **A saved layout of empty groups no longer leaves an editor with no viewport.** And a plugin writing its cache is no longer reloaded on Windows.
+- **A setting the manifest would refuse is not written.** One bad field can no longer leave a project that does not open.
+- **A mesh's colour draws as picked.** A script-built mesh's colour stays as typed, the graph and terrain pickers show the true colour, and a lit mesh in a gamma project offers to switch to linear.
+- **One box of a vector row edits a material or a clip, and Rotation boxes behave.** Typing one box of a Rotation row turns that axis, where it stored NaN in all four components, and a Rotation scrub survives the pole.
+- **Undo after a save takes back the edit over MCP.** A save was an undo step, so Undo put the scene file back instead.
+- **A lightmap bake's bounces start at 8 and go to 16 in the panel.** As the SDK default now does.
- **A sea seen from above is no longer scored with straight lines.** Its waves were straight sines a few directions apart, whose crests add up to long parallel strokes that turn with the wind. Each wave is now read at a place bent by a slow sine, in the shader and in the CPU's formula alike, so what floats still sits on what is drawn, and the ripples spread over 74 degrees either side of the wind rather than 41. It adds nothing measurable to a frame on a phone. The detail map in the Terrain & Water example was mostly one wave and drew its own fine lines over all that: it has waves in every direction now, from `tools/render-lab/water_detail.py`. A project that copied the old map should take the new one.
- **Quality high no longer costs a phone 40 ms a frame.** The render target pool keeps at most 256 MB of targets nobody has borrowed, and it counted what the frame had just given back as nobody's. A frame's own targets past that size were destroyed and allocated again every frame: a 4096 shadow atlas and a 4x MSAA scene target at 2670x1200 come to 320 MB. On a Snapdragon 8 Elite `high` went from 50 to 10.5 ms with GI off, level with no quality setting at all. Any platform with targets that large paid the same.
- **A linear project on a native build is no longer far too dark.** A native host read only half of the packaged config. The colour space, output transform, MSAA, quality, y-sort and depth layers and camera fit never reached it, with no error. A linear game ran as gamma and showed its linear values with no encode: a grey of 0.2 read 51 instead of 124, on Android, Windows and every other native build. Web and native now apply these through one function, and the colour space is read before the first shader compiles. A native build matches the web to about one level in the test room. Linear costs a phone 2.7 ms a frame at 2670x1200 with nothing else on. A project that declares a quality level now gets it on a device.
diff --git a/desktop b/desktop
index ff845a3a6..4ea260fc4 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit ff845a3a6b467e141cfdf2703a27d3656e5e56d1
+Subproject commit 4ea260fc471392d4f191ea3ecb060d55f41b1e9b
diff --git a/pipeline/src/modeling/index.ts b/pipeline/src/modeling/index.ts
index 9423a34ac..b527f3772 100644
--- a/pipeline/src/modeling/index.ts
+++ b/pipeline/src/modeling/index.ts
@@ -12,4 +12,5 @@ export { organic, ORGANIC_DEFAULT_RESOLUTION, ORGANIC_MAX_RESOLUTION, type Organ
export { needsPreparing, prepareModeling } from './booleans';
export { bakeModel, type BakedModel, type ModelReport } from './modelBake';
export type { FaceSelector } from './polyMesh';
-export { DEFAULT_MODEL_VIEWS, MODEL_VIEWS, renderModelSheet, type ModelSheet, type ModelViewName } from './modelViews';
+export { DEFAULT_MODEL_VIEWS, MODEL_VIEWS, renderModelSheet, renderRigSheet, type ModelSheet, type ModelViewName, type RigViewOptions } from './modelViews';
+export { posedPositions, prefabOf, timelineOf, type AnimationSource, type RigSource } from './rig';
diff --git a/pipeline/src/modeling/modelBake.ts b/pipeline/src/modeling/modelBake.ts
index 83c8c845d..baefbac48 100644
--- a/pipeline/src/modeling/modelBake.ts
+++ b/pipeline/src/modeling/modelBake.ts
@@ -11,6 +11,7 @@
import { MeshChannel, MeshChannelType, packChannels, unwrapLightmapUV, type MeshData } from 'esengine';
import { triangulateFace } from './triangulate';
import { faceUvs } from './modelUv';
+import { bindMatrices, computeWeights, resolveJoints, type RigJoint, type SkinWeights } from './rig';
import { buildPolyMesh, validateModelSource, type ModelSource, type StepReport } from './modelSource';
import {
boundsOf, collectEdges, cross, dot, faceArea, faceNormal, length, normalize, sub,
@@ -39,6 +40,8 @@ export interface ModelReport {
/** Closed shells whose faces point inward, which render as their own inside. */
insideOutShells: number;
steps: StepReport[];
+ /** For a rigged model: each joint, and how many vertices it moves the most. */
+ joints?: Record<string, number>;
/** The findings above that are worth stopping for, as sentences. */
warnings: string[];
}
@@ -47,6 +50,8 @@ export interface BakedModel {
poly: PolyMesh;
data: MeshData;
report: ModelReport;
+ /** Present on a rigged model. */
+ rig?: { joints: RigJoint[]; skin: SkinWeights };
/** One triangle per entry: its three positions' indices, and the face it came from. */
triangles: { v: [number, number, number]; face: number }[];
}
@@ -107,17 +112,21 @@ function cornerNormals(mesh: PolyMesh, smoothAngle: number): Vec3[][] {
}));
}
-function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number][][], source: ModelSource): MeshData {
+function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number][][], source: ModelSource, rig?: BakedModel['rig']): MeshData {
const { channels, vertexStride } = packChannels([
{ semantic: MeshChannel.Position, components: 3, type: MeshChannelType.Float32 },
{ semantic: MeshChannel.TexCoord0, components: 2, type: MeshChannelType.Float32 },
{ semantic: MeshChannel.Color, components: 4, type: MeshChannelType.UNorm8 },
{ semantic: MeshChannel.Normal, components: 3, type: MeshChannelType.Float32 },
+ ...(rig ? [
+ { semantic: MeshChannel.Joints, components: 4, type: MeshChannelType.UInt16 },
+ { semantic: MeshChannel.Weights, components: 4, type: MeshChannelType.Float32 },
+ ] : []),
]);
const normals = cornerNormals(mesh, source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE);
const uvScale = source.uvScale ?? 1;
const seen = new Map<string, number>();
- const pos: number[] = [], nrm: number[] = [], uv: number[] = [], col: number[] = [];
+ const pos: number[] = [], nrm: number[] = [], uv: number[] = [], col: number[] = [], source_: number[] = [];
const indices: number[] = [];
const uvs = new Map<number, Map<number, readonly [number, number]>>();
const vertexOf = (f: number, v: number): number => {
@@ -137,6 +146,7 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
nrm.push(n[0], n[1], n[2]);
uv.push(tu, tv);
col.push(...color);
+ source_.push(v);
}
return at;
};
@@ -159,6 +169,14 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
view.setFloat32(at + uvAt, uv[i * 2]!, true);
view.setFloat32(at + uvAt + 4, uv[i * 2 + 1]!, true);
for (let k = 0; k < 4; k++) view.setUint8(at + colorAt + k, col[i * 4 + k]!);
+ if (rig) {
+ const from = source_[i]! * 4;
+ const [jointAt, weightAt] = [offset(MeshChannel.Joints), offset(MeshChannel.Weights)];
+ for (let k = 0; k < 4; k++) {
+ view.setUint16(at + jointAt + k * 2, rig.skin.joints[from + k]!, true);
+ view.setFloat32(at + weightAt + k * 4, rig.skin.weights[from + k]!, true);
+ }
+ }
}
const { min, max } = boundsOf(mesh);
return {
@@ -166,6 +184,7 @@ function toMeshData(mesh: PolyMesh, triangles: readonly [number, number, number]
indices: new Uint32Array(indices),
aabbMin: [...min] as [number, number, number],
aabbMax: [...max] as [number, number, number],
+ ...(rig ? { inverseBindMatrices: bindMatrices(rig.joints) } : {}),
};
}
@@ -188,7 +207,14 @@ export function bakeModel(raw: unknown): BakedModel {
if (mesh.faces.length === 0) throw new Error('the steps leave no faces; a model starts with a box, cylinder, cone, sphere, plane or prism');
const triangles = mesh.faces.map((f) => triangulateFace(mesh, f));
- const unwrapped = unwrapLightmapUV(toMeshData(mesh, triangles, source)).mesh;
+ let rig: BakedModel['rig'];
+ if (source.rig) {
+ const joints = resolveJoints(source.rig);
+ rig = { joints, skin: computeWeights(mesh, joints) };
+ }
+ // A skinned mesh takes no lightmap: bones move it, and a bake holds still.
+ const baked = toMeshData(mesh, triangles, source, rig);
+ const unwrapped = rig ? baked : unwrapLightmapUV(baked).mesh;
if (source.uvAtlas) copyLightmapUvToBase(unwrapped);
const edges = [...collectEdges(mesh).values()];
@@ -222,10 +248,18 @@ export function bakeModel(raw: unknown): BakedModel {
if (degenerateFaces) warnings.push(`${degenerateFaces} face${degenerateFaces > 1 ? 's' : ''} with no area; \`weld\` removes them`);
if (insideOutShells) warnings.push(`${insideOutShells} closed shell${insideOutShells > 1 ? 's are' : ' is'} inside out`);
+ const jointReport = rig ? Object.fromEntries(rig.joints.map((j, i) => [j.name, rig!.skin.dominant[i]!])) : undefined;
+ if (rig) {
+ // A tip often moves nothing: it marks where the last bone ends.
+ const idle = rig.joints.filter((j, i) => j.children.length > 0 && rig!.skin.dominant[i] === 0).map((j) => j.name);
+ if (idle.length) warnings.push(`joint${idle.length > 1 ? 's' : ''} ${idle.join(', ')} move${idle.length > 1 ? '' : 's'} no vertex the most: too near another, or outside the surface`);
+ }
+
const roundVec = (v: readonly number[]): Vec3 => v.map((x) => Math.round(x * 1e5) / 1e5) as unknown as Vec3;
return {
poly: mesh,
data: unwrapped,
+ ...(rig ? { rig } : {}),
triangles: triangles.flatMap((tris, face) => tris.map((v) => ({ v, face }))),
report: {
vertices: unwrapped.vertexCount,
@@ -237,7 +271,7 @@ export function bakeModel(raw: unknown): BakedModel {
size: roundVec(sub(max, min)),
center: roundVec([(min[0] + max[0]) / 2, (min[1] + max[1]) / 2, (min[2] + max[2]) / 2]),
},
- tags, openEdges, nonManifoldEdges, flippedEdges, degenerateFaces, insideOutShells, steps, warnings,
+ tags, openEdges, nonManifoldEdges, flippedEdges, degenerateFaces, insideOutShells, steps, ...(jointReport ? { joints: jointReport } : {}), warnings,
},
};
}
diff --git a/pipeline/src/modeling/modelSource.ts b/pipeline/src/modeling/modelSource.ts
index 225c48c4b..1074dfa9e 100644
--- a/pipeline/src/modeling/modelSource.ts
+++ b/pipeline/src/modeling/modelSource.ts
@@ -17,6 +17,7 @@ import { organic } from './organic';
import { booleanOp } from './booleans';
import { uvStep } from './modelUv';
import { lathe, loft } from './surfaces';
+import { checkAnimations, checkRig, type AnimationSource, type RigSource } from './rig';
import { compact, type Face, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
export const MODEL_SOURCE_VERSION = 1;
@@ -37,6 +38,10 @@ export interface ModelSource {
uvScale?: number;
/** Lay the base UV out so no two triangles share texels: for a texture painted for this one model, where tiling is not wanted. */
uvAtlas?: boolean;
+ /** Joints: the mesh is skinned to them, with weights found from the surface. */
+ rig?: RigSource;
+ /** Clips moving the rig's joints, each written as an `.estimeline`. */
+ animations?: AnimationSource[];
steps: ModelStep[];
}
@@ -247,8 +252,8 @@ export function validateModelSource(raw: unknown): ModelSource {
if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) throw new ModelBuildError('a model source must be an object with `steps`', null, null, null);
const source = raw as Record<string, unknown>;
for (const key of Object.keys(source)) {
- if (!['version', 'smoothAngle', 'uvScale', 'uvAtlas', 'steps'].includes(key)) {
- throw new ModelBuildError(`unknown field "${key}"; a model source has version, smoothAngle, uvScale, uvAtlas, steps`, null, null, null);
+ if (!['version', 'smoothAngle', 'uvScale', 'uvAtlas', 'rig', 'animations', 'steps'].includes(key)) {
+ throw new ModelBuildError(`unknown field "${key}"; a model source has version, smoothAngle, uvScale, uvAtlas, rig, animations, steps`, null, null, null);
}
}
if (source.version !== undefined && source.version !== MODEL_SOURCE_VERSION) {
@@ -263,6 +268,15 @@ export function validateModelSource(raw: unknown): ModelSource {
if (source.uvAtlas !== undefined && typeof source.uvAtlas !== 'boolean') {
throw new ModelBuildError('uvAtlas must be true or false', null, null, null);
}
+ if (source.rig !== undefined) {
+ const bad = checkRig(source.rig);
+ if (bad) throw new ModelBuildError(bad, null, null, null);
+ if (source.uvAtlas) throw new ModelBuildError('uvAtlas lays out with the lightmap unwrap, which a rigged mesh does not have: bones move it. Use uv steps', null, null, null);
+ }
+ if (source.animations !== undefined) {
+ const bad = checkAnimations(source.animations, source.rig);
+ if (bad) throw new ModelBuildError(bad, null, null, null);
+ }
if (!Array.isArray(source.steps)) throw new ModelBuildError('`steps` must be an array', null, null, null);
const ids = new Set<string>();
source.steps.forEach((step: Partial<ModelStep> | null, index: number) => {
@@ -324,7 +338,7 @@ export type ModelEdit =
| { op: 'set'; step: string | number; fields: Record<string, unknown> }
| { op: 'insert'; step: ModelStep; before?: string | number; after?: string | number }
| { op: 'remove'; step: string | number }
- | { op: 'settings'; fields: Pick<ModelSource, 'smoothAngle' | 'uvScale' | 'uvAtlas'> };
+ | { op: 'settings'; fields: Pick<ModelSource, 'smoothAngle' | 'uvScale' | 'uvAtlas' | 'rig' | 'animations'> };
function indexOfStep(source: ModelSource, ref: string | number): number {
const i = typeof ref === 'number' ? ref : source.steps.findIndex((s) => s.id === ref);
diff --git a/pipeline/src/modeling/modelViews.ts b/pipeline/src/modeling/modelViews.ts
index d7fc2c653..3ae02f9fd 100644
--- a/pipeline/src/modeling/modelViews.ts
+++ b/pipeline/src/modeling/modelViews.ts
@@ -15,6 +15,7 @@
* What the model looks like in the game is what `screenshot` is for.
*/
import type { BakedModel } from './modelBake';
+import { boneLines, jointColor, posedPositions, type AnimationSource } from './rig';
import { add, boundsOf, collectEdges, cross, dot, faceNormal, normalize, selectFaces, sub, type FaceSelector, type Vec3 } from './polyMesh';
export const MODEL_VIEWS = {
@@ -55,8 +56,16 @@ const SILHOUETTE = [20, 20, 20] as const;
const WIRE = [30, 30, 30] as const;
const GUTTER = [120, 120, 120] as const;
+/** What a rig's views add: lines drawn over the shaded row, and the box to frame so a strip of poses shares one scale. */
+export interface SheetExtras {
+ overlay?: readonly (readonly [Vec3, Vec3])[];
+ frame?: { min: Vec3; max: Vec3 };
+}
+
+const BONE = [255, 140, 0] as const;
+
/** Lays the views out as columns over two rows, each panel `panel` pixels square. */
-export function renderModelSheet(model: BakedModel, views: readonly ModelViewName[] = DEFAULT_MODEL_VIEWS, panel = 384, focus?: FaceSelector, wire = false): ModelSheet {
+export function renderModelSheet(model: BakedModel, views: readonly ModelViewName[] = DEFAULT_MODEL_VIEWS, panel = 384, focus?: FaceSelector, wire = false, extra: SheetExtras = {}): ModelSheet {
if (views.length === 0) throw new Error('name at least one view');
for (const v of views) {
if (!(v in MODEL_VIEWS)) throw new Error(`unknown view "${v}"; there are ${Object.keys(MODEL_VIEWS).join(', ')}`);
@@ -68,7 +77,7 @@ export function renderModelSheet(model: BakedModel, views: readonly ModelViewNam
if (hit.length === 0) throw new Error(`focus ${JSON.stringify(focus)} matched no face`);
framed = new Set(hit.flatMap((f) => poly.faces[f]!.v));
}
- const { min, max } = boundsOf(poly, framed);
+ const { min, max } = extra.frame ?? boundsOf(poly, framed);
const center: Vec3 = [(min[0] + max[0]) / 2, (min[1] + max[1]) / 2, (min[2] + max[2]) / 2];
// The bounding sphere's diameter fits every view, the iso one included.
const diameter = Math.hypot(max[0] - min[0], max[1] - min[1], max[2] - min[2]) || 1;
@@ -161,6 +170,85 @@ export function renderModelSheet(model: BakedModel, views: readonly ModelViewNam
put(ox + x, y, WIRE);
}
}
+ // On top of everything: a bone is inside the body, which is the point of showing it.
+ for (const [from, to] of extra.overlay ?? []) {
+ const a = toScreen(from), b = toScreen(to);
+ const steps = Math.max(1, Math.ceil(Math.max(Math.abs(b[0] - a[0]), Math.abs(b[1] - a[1]))));
+ for (let s = 0; s <= steps; s++) {
+ const t = s / steps;
+ for (const [dx, dy] of [[0, 0], [1, 0], [0, 1]] as const) {
+ const x = Math.floor(a[0] + (b[0] - a[0]) * t) + dx, y = Math.floor(a[1] + (b[1] - a[1]) * t) + dy;
+ if (x >= 0 && y >= 0 && x < panel && y < panel) put(ox + x, y, BONE);
+ }
+ }
+ }
});
return { width, height, rgba, views, unitsPerPixel };
}
+
+/** The ways to look at a rigged model. */
+export interface RigViewOptions {
+ views?: readonly ModelViewName[];
+ panel?: number;
+ /** Draws this clip. With `times`, a strip of one view at each. */
+ animation?: string;
+ times?: readonly number[];
+ /** Colours each face by the joint moving it most, or fades all but one joint's reach. */
+ weights?: boolean | string;
+ /** Draws the bones over the shaded row. Default true on a rigged model. */
+ skeleton?: boolean;
+ focus?: FaceSelector;
+ wire?: boolean;
+}
+
+/** One sheet for a model's rig: the bind pose, a clip at some moments, or who moves what. */
+export function renderRigSheet(model: BakedModel, animations: readonly AnimationSource[], o: RigViewOptions = {}): ModelSheet {
+ const panel = o.panel ?? 384;
+ if (!model.rig) {
+ if (o.animation || o.weights) throw new Error('this model has no rig');
+ return renderModelSheet(model, o.views ?? DEFAULT_MODEL_VIEWS, panel, o.focus, o.wire === true);
+ }
+ const { joints, skin } = model.rig;
+ let drawn = model;
+ if (o.weights !== undefined && o.weights !== false) {
+ const only = typeof o.weights === 'string' ? joints.findIndex((j) => j.name === o.weights) : -1;
+ if (typeof o.weights === 'string' && only < 0) throw new Error(`no joint "${o.weights}"; there are ${joints.map((j) => j.name).join(', ')}`);
+ const faces = model.poly.faces.map((face) => {
+ const totals = joints.map(() => 0);
+ for (const v of face.v) for (let k = 0; k < 4; k++) totals[skin.joints[v * 4 + k]!]! += skin.weights[v * 4 + k]!;
+ if (only >= 0) {
+ const w = totals[only]! / face.v.length;
+ return { ...face, color: [0.15 + w * 0.85, 0.15, 0.15 + (1 - w) * 0.5, 1] as const };
+ }
+ return { ...face, color: jointColor(totals.indexOf(Math.max(...totals))) };
+ });
+ drawn = { ...model, poly: { ...model.poly, faces } };
+ }
+ const anim = o.animation === undefined ? null : animations.find((a) => a.name === o.animation);
+ if (o.animation !== undefined && !anim) throw new Error(`no animation "${o.animation}"; there are ${animations.map((a) => a.name).join(', ') || '(none)'}`);
+ const skeleton = o.skeleton !== false;
+ const posed = (time: number): BakedModel => ({ ...drawn, poly: { ...drawn.poly, positions: posedPositions(model.poly, joints, skin, anim ?? null, time) } });
+ if (!anim) {
+ return renderModelSheet(drawn, o.views ?? DEFAULT_MODEL_VIEWS, panel, o.focus, o.wire === true, skeleton ? { overlay: boneLines(joints, null, 0) } : {});
+ }
+ const times = o.times ?? Array.from({ length: 4 }, (_, i) => (i / 4) * anim.duration);
+ if (times.length === 0) throw new Error('`times` names no moment');
+ if (times.length * panel > 2048) throw new Error(`${times.length} panels of ${panel} is too wide a strip; use fewer or a smaller size`);
+ const view = o.views?.[0] ?? 'iso';
+ // One frame for all of them, so what moves in the strip is the model and not the camera.
+ const poses = times.map(posed);
+ const all = poses.flatMap((m) => m.poly.positions);
+ const frame = boundsOf({ positions: all, faces: [] });
+ const sheets = poses.map((m, i) => renderModelSheet(m, [view], panel, undefined, o.wire === true, {
+ frame, ...(skeleton ? { overlay: boneLines(joints, anim, times[i]!) } : {}),
+ }));
+ const width = times.length * panel + (times.length - 1);
+ const height = sheets[0]!.height;
+ const rgba = new Uint8Array(width * height * 4);
+ sheets.forEach((s, i) => {
+ for (let y = 0; y < height; y++) {
+ rgba.set(s.rgba.subarray(y * s.width * 4, (y + 1) * s.width * 4), (y * width + i * (panel + 1)) * 4);
+ }
+ });
+ return { width, height, rgba, views: times.map(() => view), unitsPerPixel: sheets[0]!.unitsPerPixel };
+}
diff --git a/pipeline/src/modeling/rig.ts b/pipeline/src/modeling/rig.ts
new file mode 100644
index 000000000..8b4cb4d1c
--- /dev/null
+++ b/pipeline/src/modeling/rig.ts
@@ -0,0 +1,521 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file rig.ts — a model that moves: joints, the weights found for them, and clips.
+ *
+ * A caller states where the joints are and how a clip turns them. Weights are
+ * found, measured along the surface from each bone: two legs side by side are
+ * close through the air and far apart over the skin, and only the second is
+ * whether one should drag the other.
+ */
+import { PREFAB_FORMAT_VERSION, TIMELINE_FORMAT_VERSION } from 'esengine';
+import { add, boundsOf, cross, dot, length, scale, sub, type PolyMesh, type Vec3 } from './polyMesh';
+
+export const MAX_RIG_JOINTS = 64;
+const DEG = Math.PI / 180;
+const SAMPLE_RATE = 30;
+
+export interface RigJointSource {
+ name: string;
+ at: Vec3;
+ parent?: string;
+ /** Also exists reflected across this axis: `arm` becomes `arm.L` on the positive side and `arm.R` on the other. */
+ mirror?: 'x' | 'y' | 'z';
+}
+
+export interface RigSource {
+ joints: readonly RigJointSource[];
+}
+
+export interface AnimationTrack {
+ joint: string;
+ /** [time, x, y, z] in degrees, X then Y then Z, from the bind pose. */
+ rotate?: readonly (readonly number[])[];
+ /** [time, x, y, z] from where the joint rests. */
+ move?: readonly (readonly number[])[];
+ /** Shifts the keys by this fraction of the clip: the other leg, half a stride behind. */
+ phase?: number;
+}
+
+export interface AnimationSource {
+ name: string;
+ duration: number;
+ /** Default true. */
+ loop?: boolean;
+ tracks: readonly AnimationTrack[];
+}
+
+export interface RigJoint {
+ name: string;
+ at: Vec3;
+ parent: number;
+ children: number[];
+}
+
+/** Joints flattened, mirrored ones expanded, parents always before their children. */
+export function resolveJoints(rig: RigSource): RigJoint[] {
+ const out: { name: string; at: Vec3; parentName: string | undefined }[] = [];
+ const mirrored = new Set(rig.joints.filter((j) => j.mirror).map((j) => j.name));
+ for (const j of rig.joints) {
+ if (!j.mirror) {
+ out.push({ name: j.name, at: j.at, parentName: j.parent });
+ continue;
+ }
+ const k = { x: 0, y: 1, z: 2 }[j.mirror];
+ const flip = [...j.at] as [number, number, number];
+ flip[k] = -flip[k];
+ const positive = j.at[k]! >= 0;
+ const side = (s: string): string | undefined => (j.parent === undefined ? undefined : mirrored.has(j.parent) ? `${j.parent}.${s}` : j.parent);
+ out.push({ name: `${j.name}.L`, at: positive ? j.at : flip, parentName: side('L') });
+ out.push({ name: `${j.name}.R`, at: positive ? flip : j.at, parentName: side('R') });
+ }
+ if (out.length === 0) throw new Error('a rig needs at least one joint');
+ if (out.length > MAX_RIG_JOINTS) throw new Error(`this rig has ${out.length} joints; a mesh can be posed by at most ${MAX_RIG_JOINTS}`);
+ const index = new Map<string, number>();
+ out.forEach((j, i) => {
+ if (index.has(j.name)) throw new Error(`there are two joints called "${j.name}"`);
+ index.set(j.name, i);
+ });
+ const parents = out.map((j) => {
+ if (j.parentName === undefined) return -1;
+ const p = index.get(j.parentName);
+ if (p === undefined) throw new Error(`joint "${j.name}" has parent "${j.parentName}", which is not a joint; there are ${out.map((o) => o.name).join(', ')}`);
+ return p;
+ });
+ if (parents.filter((p) => p < 0).length !== 1) {
+ throw new Error(`a rig has one root; these have no parent: ${out.filter((_, i) => parents[i]! < 0).map((j) => j.name).join(', ')}`);
+ }
+ // Parents first, which also proves there is no loop.
+ const order: number[] = [];
+ const placed = new Set<number>();
+ while (order.length < out.length) {
+ const before = order.length;
+ out.forEach((_, i) => {
+ if (!placed.has(i) && (parents[i]! < 0 || placed.has(parents[i]!))) { order.push(i); placed.add(i); }
+ });
+ if (order.length === before) throw new Error('the joints\' parents go round in a loop');
+ }
+ const newIndex = new Map(order.map((old, i) => [old, i]));
+ const joints: RigJoint[] = order.map((old) => ({
+ name: out[old]!.name, at: out[old]!.at, parent: parents[old]! < 0 ? -1 : newIndex.get(parents[old]!)!, children: [],
+ }));
+ joints.forEach((j, i) => { if (j.parent >= 0) joints[j.parent]!.children.push(i); });
+ return joints;
+}
+
+// ---- Weights --------------------------------------------------------------------
+
+function pointSegment(p: Vec3, a: Vec3, b: Vec3): number {
+ const ab = sub(b, a);
+ const len2 = dot(ab, ab);
+ const t = len2 > 0 ? Math.min(1, Math.max(0, dot(sub(p, a), ab) / len2)) : 0;
+ return length(sub(p, add(a, scale(ab, t))));
+}
+
+/** A min-heap of [distance, vertex], for Dijkstra over the surface. */
+class Heap {
+ private d: number[] = [];
+ private v: number[] = [];
+ get size(): number { return this.d.length; }
+ push(dist: number, vertex: number): void {
+ let i = this.d.length;
+ this.d.push(dist); this.v.push(vertex);
+ while (i > 0) {
+ const up = (i - 1) >> 1;
+ if (this.d[up]! <= dist) break;
+ this.d[i] = this.d[up]!; this.v[i] = this.v[up]!;
+ i = up;
+ }
+ this.d[i] = dist; this.v[i] = vertex;
+ }
+ pop(): [number, number] {
+ const [topD, topV] = [this.d[0]!, this.v[0]!];
+ const [lastD, lastV] = [this.d.pop()!, this.v.pop()!];
+ const n = this.d.length;
+ if (n > 0) {
+ let i = 0;
+ for (;;) {
+ let c = 2 * i + 1;
+ if (c >= n) break;
+ if (c + 1 < n && this.d[c + 1]! < this.d[c]!) c++;
+ if (this.d[c]! >= lastD) break;
+ this.d[i] = this.d[c]!; this.v[i] = this.v[c]!;
+ i = c;
+ }
+ this.d[i] = lastD; this.v[i] = lastV;
+ }
+ return [topD, topV];
+ }
+}
+
+export interface SkinWeights {
+ /** Four per vertex. */
+ joints: Uint16Array;
+ weights: Float32Array;
+ /** How many vertices each joint moves the most. */
+ dominant: number[];
+}
+
+/** How sharply influence falls off between bones: higher keeps a limb rigid and leaves the bend to the joint. */
+const WEIGHT_POWER = 6;
+const MIN_WEIGHT = 0.02;
+const SEED_SAMPLES = 6;
+
+const pointToBones = (p: Vec3, from: Vec3, ends: readonly Vec3[]): number => {
+ let d = Infinity;
+ for (const end of ends) d = Math.min(d, pointSegment(p, from, end));
+ return d;
+};
+
+export function computeWeights(mesh: PolyMesh, joints: readonly RigJoint[]): SkinWeights {
+ const count = mesh.positions.length;
+ const neighbours: number[][] = Array.from({ length: count }, () => []);
+ for (const face of mesh.faces) {
+ face.v.forEach((v, i) => {
+ const w = face.v[(i + 1) % face.v.length]!;
+ neighbours[v]!.push(w);
+ neighbours[w]!.push(v);
+ });
+ }
+ // Pieces of the surface: each is seeded on its own, so a separate eyeball still follows the bone nearest it.
+ const shell = new Int32Array(count).fill(-1);
+ let shells = 0;
+ for (let v = 0; v < count; v++) {
+ if (shell[v]! >= 0) continue;
+ const stack = [v];
+ shell[v] = shells;
+ while (stack.length) {
+ for (const w of neighbours[stack.pop()!]!) {
+ if (shell[w]! < 0) { shell[w] = shells; stack.push(w); }
+ }
+ }
+ shells++;
+ }
+
+ const { min, max } = boundsOf(mesh);
+ const soft = length(sub(max, min)) * 0.004;
+ const distances = joints.map((j) => {
+ // A joint owns the bones running to its children; a tip owns only its point.
+ const ends = j.children.length ? j.children.map((c) => joints[c]!.at) : [j.at];
+ // Seeded where the bone is nearest the skin, sample by sample along it: one seed band for the whole bone would miss a
+ // limb's thick end, and one drawn wide enough to catch it reaches the next leg through the air.
+ const samples: Vec3[] = [];
+ for (const end of ends) {
+ for (let i = 0; i < SEED_SAMPLES; i++) samples.push(add(j.at, scale(sub(end, j.at), i / (SEED_SAMPLES - 1))));
+ }
+ const dist = new Float64Array(count).fill(Infinity);
+ const heap = new Heap();
+ const air = new Float64Array(count);
+ for (const sample of samples) {
+ const nearest = new Float64Array(shells).fill(Infinity);
+ for (let v = 0; v < count; v++) {
+ const d = length(sub(mesh.positions[v]!, sample));
+ air[v] = d;
+ if (d < nearest[shell[v]!]!) nearest[shell[v]!] = d;
+ }
+ for (let v = 0; v < count; v++) {
+ if (air[v]! > nearest[shell[v]!]! * 1.1 + soft) continue;
+ const d = pointToBones(mesh.positions[v]!, j.at, ends);
+ if (d < dist[v]!) { dist[v] = d; heap.push(d, v); }
+ }
+ }
+ while (heap.size) {
+ const [d, v] = heap.pop();
+ if (d > dist[v]!) continue;
+ for (const w of neighbours[v]!) {
+ const next = d + length(sub(mesh.positions[v]!, mesh.positions[w]!));
+ if (next < dist[w]!) { dist[w] = next; heap.push(next, w); }
+ }
+ }
+ return dist;
+ });
+
+ const outJoints = new Uint16Array(count * 4);
+ const outWeights = new Float32Array(count * 4);
+ const dominant = joints.map(() => 0);
+ const ranked: { j: number; w: number }[] = [];
+ for (let v = 0; v < count; v++) {
+ let least = Infinity;
+ for (const d of distances) if (d[v]! < least) least = d[v]!;
+ ranked.length = 0;
+ distances.forEach((d, j) => {
+ const w = ((least + soft) / (d[v]! + soft)) ** WEIGHT_POWER;
+ if (w >= MIN_WEIGHT) ranked.push({ j, w });
+ });
+ ranked.sort((a, b) => b.w - a.w);
+ const keep = ranked.slice(0, 4);
+ const total = keep.reduce((n, r) => n + r.w, 0);
+ keep.forEach((r, i) => {
+ outJoints[v * 4 + i] = r.j;
+ outWeights[v * 4 + i] = r.w / total;
+ });
+ dominant[keep[0]!.j]!++;
+ }
+ return { joints: outJoints, weights: outWeights, dominant };
+}
+
+// ---- Validation -----------------------------------------------------------------
+
+const isNum = (v: unknown): v is number => typeof v === 'number' && Number.isFinite(v);
+const vec3 = (v: unknown): boolean => Array.isArray(v) && v.length === 3 && v.every(isNum);
+
+/** What is wrong with a source's `rig`, or null. */
+export function checkRig(raw: unknown): string | null {
+ const r = raw as { joints?: unknown };
+ if (typeof raw !== 'object' || raw === null || !Array.isArray(r.joints)) return 'rig must be { joints: [{ name, at, parent?, mirror? }] }';
+ for (const key of Object.keys(raw)) if (key !== 'joints') return `rig has an unknown field "${key}"; it takes joints`;
+ for (const [i, j] of (r.joints as Record<string, unknown>[]).entries()) {
+ if (typeof j !== 'object' || j === null) return `rig joint ${i} must be an object`;
+ for (const key of Object.keys(j)) if (!['name', 'at', 'parent', 'mirror'].includes(key)) return `rig joint ${i} has an unknown field "${key}"; a joint takes name, at, parent, mirror`;
+ if (typeof j.name !== 'string' || !j.name) return `rig joint ${i} needs a name`;
+ if (/[/\\]/.test(j.name)) return `rig joint "${j.name}": a name cannot hold a slash, which is how a clip addresses a joint`;
+ if (!vec3(j.at)) return `rig joint "${j.name}": at must be [x, y, z]`;
+ if (j.parent !== undefined && typeof j.parent !== 'string') return `rig joint "${j.name}": parent must be a joint's name`;
+ if (j.mirror !== undefined && !['x', 'y', 'z'].includes(j.mirror as string)) return `rig joint "${j.name}": mirror must be x, y or z`;
+ }
+ try {
+ resolveJoints(raw as RigSource);
+ } catch (e) {
+ return (e as Error).message;
+ }
+ return null;
+}
+
+function checkKeys(keys: unknown, duration: number, label: string): string | null {
+ if (!Array.isArray(keys) || keys.length === 0) return `${label} must be an array of [time, x, y, z]`;
+ let last = -Infinity;
+ for (const k of keys as unknown[]) {
+ if (!Array.isArray(k) || k.length !== 4 || !k.every(isNum)) return `${label}: every key is [time, x, y, z], not ${JSON.stringify(k)}`;
+ const t = k[0] as number;
+ if (t < 0 || t > duration) return `${label}: time ${t} is outside the clip's 0..${duration}`;
+ if (t <= last) return `${label}: keys must be in time order, and ${t} comes after ${last}`;
+ last = t;
+ }
+ return null;
+}
+
+/** What is wrong with a source's `animations`, or null. */
+export function checkAnimations(raw: unknown, rig: unknown): string | null {
+ if (!Array.isArray(raw)) return 'animations must be an array';
+ if (!rig) return 'animations move joints, and this source has no rig';
+ const names = new Set(resolveJoints(rig as RigSource).map((j) => j.name));
+ const seen = new Set<string>();
+ for (const [i, a] of (raw as Record<string, unknown>[]).entries()) {
+ if (typeof a !== 'object' || a === null) return `animation ${i} must be an object`;
+ for (const key of Object.keys(a)) if (!['name', 'duration', 'loop', 'tracks'].includes(key)) return `animation ${i} has an unknown field "${key}"; it takes name, duration, loop, tracks`;
+ if (typeof a.name !== 'string' || !/^[\w-]+$/.test(a.name)) return `animation ${i} needs a name of letters, digits, _ or -: it becomes a file name`;
+ if (seen.has(a.name)) return `there are two animations called "${a.name}"`;
+ seen.add(a.name);
+ if (!isNum(a.duration) || a.duration <= 0) return `animation "${a.name}": duration must be positive seconds`;
+ if (a.loop !== undefined && typeof a.loop !== 'boolean') return `animation "${a.name}": loop must be true or false`;
+ if (!Array.isArray(a.tracks) || a.tracks.length === 0) return `animation "${a.name}" needs tracks`;
+ const tracked = new Set<string>();
+ for (const t of a.tracks as Record<string, unknown>[]) {
+ for (const key of Object.keys(t)) if (!['joint', 'rotate', 'move', 'phase'].includes(key)) return `animation "${a.name}": a track has an unknown field "${key}"; it takes joint, rotate, move, phase`;
+ if (!names.has(t.joint as string)) return `animation "${a.name}": no joint "${String(t.joint)}"; there are ${[...names].join(', ')}`;
+ if (tracked.has(t.joint as string)) return `animation "${a.name}": two tracks for joint "${t.joint as string}"; put their keys in one`;
+ tracked.add(t.joint as string);
+ if (t.rotate === undefined && t.move === undefined) return `animation "${a.name}": the track for "${t.joint as string}" has neither rotate nor move`;
+ for (const kind of ['rotate', 'move'] as const) {
+ if (t[kind] === undefined) continue;
+ const bad = checkKeys(t[kind], a.duration, `animation "${a.name}", ${t.joint as string}.${kind}`);
+ if (bad) return bad;
+ }
+ if (t.phase !== undefined && !isNum(t.phase)) return `animation "${a.name}": phase must be a number`;
+ }
+ }
+ return null;
+}
+
+// ---- Motion ---------------------------------------------------------------------
+
+type Quat = readonly [number, number, number, number];
+
+const quatMul = (a: Quat, b: Quat): Quat => [
+ a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1],
+ a[3] * b[1] - a[0] * b[2] + a[1] * b[3] + a[2] * b[0],
+ a[3] * b[2] + a[0] * b[1] - a[1] * b[0] + a[2] * b[3],
+ a[3] * b[3] - a[0] * b[0] - a[1] * b[1] - a[2] * b[2],
+];
+
+/** X, then Y, then Z. */
+function eulerToQuat(x: number, y: number, z: number): Quat {
+ const axis = (i: number, deg: number): Quat => {
+ const h = (deg * DEG) / 2;
+ const q = [0, 0, 0, Math.cos(h)] as [number, number, number, number];
+ q[i] = Math.sin(h);
+ return q;
+ };
+ return quatMul(axis(2, z), quatMul(axis(1, y), axis(0, x)));
+}
+
+function rotateBy(q: Quat, v: Vec3): Vec3 {
+ const u: Vec3 = [q[0], q[1], q[2]];
+ const t = scale(cross(u, v), 2);
+ return add(add(v, scale(t, q[3])), cross(u, t));
+}
+
+/** A key track's value at @p time: a smooth line through the keys, passing through each exactly. */
+function sample(keys: readonly (readonly number[])[], time: number, duration: number, loop: boolean, phase: number): Vec3 {
+ const shifted = keys.map((k) => [k[0]! + phase * duration, k[1]!, k[2]!, k[3]!] as const);
+ const wrap = (t: number): number => ((t % duration) + duration) % duration;
+ const list = (loop ? shifted.map((k) => [wrap(k[0]), k[1], k[2], k[3]] as const) : shifted).slice().sort((a, b) => a[0] - b[0]);
+ const n = list.length;
+ if (n === 1) return [list[0]![1], list[0]![2], list[0]![3]];
+ if (!loop && time <= list[0]![0]) return [list[0]![1], list[0]![2], list[0]![3]];
+ if (!loop && time >= list[n - 1]![0]) return [list[n - 1]![1], list[n - 1]![2], list[n - 1]![3]];
+ const at = (i: number): readonly number[] => {
+ if (loop) {
+ const turns = Math.floor(i / n);
+ const k = list[((i % n) + n) % n]!;
+ return [k[0] + turns * duration, k[1], k[2], k[3]];
+ }
+ return list[Math.max(0, Math.min(n - 1, i))]!;
+ };
+ let i = loop ? -1 : 0;
+ const t = loop ? wrap(time) : time;
+ while (at(i + 1)[0]! <= t) i++;
+ const [p0, p1, p2, p3] = [at(i - 1), at(i), at(i + 1), at(i + 2)];
+ const span = p2[0]! - p1[0]!;
+ const u = span > 0 ? (t - p1[0]!) / span : 0;
+ const [u2, u3] = [u * u, u * u * u];
+ const v = [1, 2, 3].map((c) => {
+ const [a, b] = [p1[c]!, p2[c]!];
+ const m1 = ((b - p0[c]!) / Math.max(p2[0]! - p0[0]!, 1e-6)) * span;
+ const m2 = ((p3[c]! - a) / Math.max(p3[0]! - p1[0]!, 1e-6)) * span;
+ return (2 * u3 - 3 * u2 + 1) * a + (u3 - 2 * u2 + u) * m1 + (-2 * u3 + 3 * u2) * b + (u3 - u2) * m2;
+ });
+ return v as unknown as Vec3;
+}
+
+/** Each joint's world rotation and origin at @p time; the bind pose with no animation. */
+function poseAt(joints: readonly RigJoint[], animation: AnimationSource | null, time: number): { q: Quat; origin: Vec3 }[] {
+ const out: { q: Quat; origin: Vec3 }[] = [];
+ joints.forEach((j, i) => {
+ const track = animation?.tracks.find((t) => t.joint === j.name);
+ const loop = animation?.loop !== false;
+ const euler = track?.rotate ? sample(track.rotate, time, animation!.duration, loop, track.phase ?? 0) : [0, 0, 0] as Vec3;
+ const moved = track?.move ? sample(track.move, time, animation!.duration, loop, track.phase ?? 0) : [0, 0, 0] as Vec3;
+ const local = eulerToQuat(euler[0], euler[1], euler[2]);
+ const rest = j.parent >= 0 ? sub(j.at, joints[j.parent]!.at) : j.at;
+ const offset = add(rest, moved);
+ if (j.parent < 0) out[i] = { q: local, origin: offset };
+ else {
+ const p = out[j.parent]!;
+ out[i] = { q: quatMul(p.q, local), origin: add(p.origin, rotateBy(p.q, offset)) };
+ }
+ });
+ return out;
+}
+
+/** The surface's positions in a pose, skinned as the engine skins it. */
+export function posedPositions(mesh: PolyMesh, joints: readonly RigJoint[], skin: SkinWeights, animation: AnimationSource | null, time: number): Vec3[] {
+ const pose = poseAt(joints, animation, time);
+ return mesh.positions.map((p, v) => {
+ let out: Vec3 = [0, 0, 0];
+ for (let k = 0; k < 4; k++) {
+ const w = skin.weights[v * 4 + k]!;
+ if (w <= 0) continue;
+ const j = skin.joints[v * 4 + k]!;
+ const moved = add(rotateBy(pose[j]!.q, sub(p, joints[j]!.at)), pose[j]!.origin);
+ out = add(out, scale(moved, w));
+ }
+ return out;
+ });
+}
+
+/** The bones as lines in a pose, for drawing over a view. */
+export function boneLines(joints: readonly RigJoint[], animation: AnimationSource | null, time: number): [Vec3, Vec3][] {
+ const pose = poseAt(joints, animation, time);
+ return joints.flatMap((j, i) => (j.parent >= 0 ? [[pose[j.parent]!.origin, pose[i]!.origin] as [Vec3, Vec3]] : []));
+}
+
+// ---- Products -------------------------------------------------------------------
+
+interface Keyframe { time: number; value: number; inTangent: 0; outTangent: 0; interpolation: 'linear' }
+
+/** A clip as an `.estimeline`: sampled, since the engine interpolates each rotation component on its own and a few keys would cut corners. */
+export function timelineOf(animation: AnimationSource, joints: readonly RigJoint[]): Record<string, unknown> {
+ const loop = animation.loop !== false;
+ const frames = Math.max(2, Math.ceil(animation.duration * SAMPLE_RATE) + 1);
+ const times = Array.from({ length: frames }, (_, i) => (i / (frames - 1)) * animation.duration);
+ const pathOf = (i: number): string => {
+ const parts: string[] = [];
+ for (let at = i; at >= 0; at = joints[at]!.parent) parts.unshift(joints[at]!.name);
+ return parts.join('/');
+ };
+ const tracks = animation.tracks.map((track) => {
+ const index = joints.findIndex((j) => j.name === track.joint);
+ const joint = joints[index]!;
+ const channels: { property: string; keyframes: Keyframe[] }[] = [];
+ const push = (property: string, values: number[]): void => {
+ channels.push({ property, keyframes: values.map((value, f) => ({ time: times[f]!, value, inTangent: 0, outTangent: 0, interpolation: 'linear' })) });
+ };
+ if (track.rotate) {
+ let last: Quat | null = null;
+ const qs = times.map((t) => {
+ const e = sample(track.rotate!, t, animation.duration, loop, track.phase ?? 0);
+ let q = eulerToQuat(e[0], e[1], e[2]);
+ // The short way round between samples: the two signs of a quaternion are one rotation, and interpolated apart they spin the long way.
+ if (last && q[0] * last[0] + q[1] * last[1] + q[2] * last[2] + q[3] * last[3] < 0) q = [-q[0], -q[1], -q[2], -q[3]];
+ last = q;
+ return q;
+ });
+ ['x', 'y', 'z', 'w'].forEach((c, k) => push(`rotation.${c}`, qs.map((q) => q[k]!)));
+ }
+ if (track.move) {
+ const rest = joint.parent >= 0 ? sub(joint.at, joints[joint.parent]!.at) : joint.at;
+ const ps = times.map((t) => add(rest, sample(track.move!, t, animation.duration, loop, track.phase ?? 0)));
+ ['x', 'y', 'z'].forEach((c, k) => push(`position.${c}`, ps.map((p) => p[k]!)));
+ }
+ return { type: 'property', name: joint.name, childPath: pathOf(index), component: 'Transform', channels };
+ });
+ return { version: TIMELINE_FORMAT_VERSION, type: 'timeline', duration: animation.duration, wrapMode: loop ? 'loop' : 'once', tracks };
+}
+
+/**
+ * The prefab that draws and moves it: the mesh on the root, the joints below it as
+ * entities named so a clip's paths find them, and a player for the first clip.
+ */
+export function prefabOf(name: string, meshPath: string, joints: readonly RigJoint[], cullBackfaces: boolean, firstClip: string | null): Record<string, unknown> {
+ const idOf = (i: number): string => `j${i}`;
+ const entities: Record<string, unknown>[] = [{
+ prefabEntityId: 'root', name, parent: null,
+ children: joints.flatMap((j, i) => (j.parent < 0 ? [idOf(i)] : [])),
+ visible: true,
+ components: [
+ { type: 'Transform', data: {} },
+ { type: 'MeshRenderer', data: { mesh: meshPath, lit: true, opaque: true, cullBackfaces, enabled: true } },
+ { type: 'MeshSkin', data: { joints: joints.map((_, i) => idOf(i)) } },
+ ...(firstClip ? [{ type: 'TimelinePlayer', data: { timeline: firstClip, playing: true } }] : []),
+ ],
+ }];
+ joints.forEach((j, i) => {
+ const rest = j.parent >= 0 ? sub(j.at, joints[j.parent]!.at) : j.at;
+ entities.push({
+ prefabEntityId: idOf(i), name: j.name, parent: j.parent >= 0 ? idOf(j.parent) : 'root',
+ children: j.children.map(idOf), visible: true,
+ components: [{ type: 'Transform', data: { position: { x: rest[0], y: rest[1], z: rest[2] } } }],
+ });
+ });
+ return { version: PREFAB_FORMAT_VERSION, name, rootEntityId: 'root', entities };
+}
+
+/** One bind matrix per joint: a joint's bind pose is unrotated, so each is only the way back to its rest position. */
+export function bindMatrices(joints: readonly RigJoint[]): Float32Array {
+ const out = new Float32Array(joints.length * 16);
+ joints.forEach((j, i) => {
+ out.set([1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, -j.at[0], -j.at[1], -j.at[2], 1], i * 16);
+ });
+ return out;
+}
+
+/** A colour for a joint, far from its neighbours in the list. */
+export function jointColor(index: number): readonly [number, number, number, number] {
+ const h = (index * 0.618034) % 1;
+ const c = (n: number): number => {
+ const k = (n + h * 6) % 6;
+ return 0.85 - 0.7 * Math.max(0, Math.min(k, 4 - k, 1));
+ };
+ return [c(5), c(3), c(1), 1];
+}
diff --git a/pipeline/tests/modeling-rig.test.ts b/pipeline/tests/modeling-rig.test.ts
new file mode 100644
index 000000000..199c21893
--- /dev/null
+++ b/pipeline/tests/modeling-rig.test.ts
@@ -0,0 +1,283 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/** A rig held to what moving proves: rest is rest, a joint turns what it should and nothing else, and the files say the same. */
+import { describe, expect, it } from 'vitest';
+import { decodeMesh, encodeMesh, MeshChannel } from 'esengine';
+import { bakeModel, posedPositions, prefabOf, renderRigSheet, timelineOf, validateModelSource, type AnimationSource, type BakedModel, type ModelStep, type RigSource } from '../src/modeling';
+
+/** An upright limb in three bones, base at the origin. */
+const limb: ModelStep = { op: 'organic', resolution: 48, blend: 0.05, tag: 'limb', parts: [{ shape: 'chain', points: [[0, 0, 0, 0.2], [0, 2, 0, 0.2]] }] };
+const limbRig: RigSource = { joints: [
+ { name: 'base', at: [0, 0, 0] },
+ { name: 'mid', at: [0, 1, 0], parent: 'base' },
+ { name: 'tip', at: [0, 2, 0], parent: 'mid' },
+] };
+const bake = (animations: AnimationSource[] = [], rig: RigSource = limbRig, steps: ModelStep[] = [limb]): BakedModel =>
+ bakeModel({ version: 1, rig, ...(animations.length ? { animations } : {}), steps });
+const bend = (deg: number): AnimationSource => ({ name: 'bend', duration: 1, loop: false, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 0], [1, 0, 0, deg]] }] });
+
+const pose = (m: BakedModel, a: AnimationSource | null, t: number) => posedPositions(m.poly, m.rig!.joints, m.rig!.skin, a, t);
+
+describe('weights', () => {
+ it('sum to one on every vertex, with the strongest first', () => {
+ const { skin } = bake().rig!;
+ for (let v = 0; v * 4 < skin.weights.length; v++) {
+ const w = [...skin.weights.subarray(v * 4, v * 4 + 4)];
+ expect(w.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 5);
+ expect(w[0]!).toBeGreaterThanOrEqual(w[1]!);
+ }
+ });
+
+ it('a vertex well along a bone is held by that bone alone', () => {
+ const m = bake();
+ const { joints, weights } = m.rig!.skin;
+ const baseIndex = m.rig!.joints.findIndex((j) => j.name === 'base');
+ m.poly.positions.forEach((p, v) => {
+ if (Math.abs(p[1] - 0.3) > 0.1) return;
+ expect(joints[v * 4]).toBe(baseIndex);
+ expect(weights[v * 4]!).toBeGreaterThan(0.95);
+ });
+ });
+
+ it('two legs side by side do not take each other: they are far apart over the skin', () => {
+ // Hips with two legs a hair apart through the air.
+ const legs: ModelStep = { op: 'organic', resolution: 96, parts: [
+ { shape: 'sphere', radius: 0.4, at: [0, 2, 0], tag: 'hips' },
+ { shape: 'chain', mirror: 'x', points: [[0.17, 2, 0, 0.15], [0.17, 0, 0, 0.15]], tag: 'leg' },
+ ] };
+ const rig: RigSource = { joints: [
+ { name: 'hips', at: [0, 2, 0] },
+ { name: 'leg', at: [0.17, 1.8, 0], parent: 'hips', mirror: 'x' },
+ { name: 'foot', at: [0.17, 0, 0], parent: 'leg', mirror: 'x' },
+ ] };
+ const m = bake([], rig, [legs]);
+ const left = m.rig!.joints.findIndex((j) => j.name === 'leg.L');
+ const right = m.rig!.joints.findIndex((j) => j.name === 'leg.R');
+ let crossed = 0, checked = 0;
+ m.poly.positions.forEach((p, v) => {
+ if (p[1] > 1.3 || p[1] < 0.3) return; // well down the legs
+ checked++;
+ const wrong = p[0] > 0 ? right : left;
+ for (let k = 0; k < 4; k++) if (m.rig!.skin.joints[v * 4 + k] === wrong && m.rig!.skin.weights[v * 4 + k]! > 0.01) crossed++;
+ });
+ expect(checked).toBeGreaterThan(100);
+ expect(crossed).toBe(0);
+ });
+
+ it('a separate shell follows the bone nearest it, rigidly', () => {
+ const m = bake([], limbRig, [limb, { op: 'sphere', radius: 0.1, at: [0.4, 1.6, 0], tag: 'eye' }]);
+ const midIndex = m.rig!.joints.findIndex((j) => j.name === 'mid');
+ m.poly.faces.filter((f) => f.tag === 'eye').flatMap((f) => f.v).forEach((v) => {
+ expect(m.rig!.skin.joints[v * 4]).toBe(midIndex);
+ });
+ });
+
+ it('the report says what each joint moves, and which move nothing', () => {
+ const m = bake();
+ expect(m.report.joints!.base).toBeGreaterThan(100);
+ expect(m.report.warnings).toEqual([]);
+ const far = bake([], { joints: [{ name: 'base', at: [0, 1, 0] }, { name: 'stray', at: [9, 0, 0], parent: 'base' }, { name: 'strayEnd', at: [10, 0, 0], parent: 'stray' }, { name: 'end', at: [0, 2, 0], parent: 'base' }] });
+ expect(far.report.warnings.join()).toMatch(/stray moves no vertex/);
+ });
+});
+
+describe('poses', () => {
+ it('the bind pose is where the mesh already is', () => {
+ const m = bake();
+ pose(m, null, 0).forEach((p, v) => {
+ const q = m.poly.positions[v]!;
+ expect(Math.hypot(p[0] - q[0], p[1] - q[1], p[2] - q[2])).toBeLessThan(1e-5);
+ });
+ });
+
+ it('turning a joint moves what is past it and leaves what is before it', () => {
+ const a = bend(90);
+ const m = bake([a]);
+ const after = pose(m, a, 1);
+ m.poly.positions.forEach((q, v) => {
+ const p = after[v]!;
+ if (q[1] < 0.6) expect(Math.hypot(p[0] - q[0], p[1] - q[1], p[2] - q[2])).toBeLessThan(0.01);
+ });
+ // 90 degrees about Z takes the tip from straight up to out along -X, from the joint at height 1.
+ const tip = after[m.poly.positions.reduce((best, q, v) => (q[1] > m.poly.positions[best]![1] ? v : best), 0)]!;
+ expect(tip[0]).toBeLessThan(-1);
+ expect(tip[1]).toBeCloseTo(1, 0);
+ });
+
+ it('the joints stack: turning two adds up', () => {
+ const a: AnimationSource = { name: 'curl', duration: 1, loop: false, tracks: [
+ { joint: 'base', rotate: [[0, 0, 0, 0], [1, 0, 0, 90]] },
+ { joint: 'mid', rotate: [[0, 0, 0, 0], [1, 0, 0, 90]] },
+ ] };
+ const m = bake([a]);
+ const after = pose(m, a, 1);
+ const top = m.poly.positions.reduce((best, q, v) => (q[1] > m.poly.positions[best]![1] ? v : best), 0);
+ // Base: out along -X for 1. Then a further 90: on down -Y for 1.
+ expect(after[top]![0]).toBeCloseTo(-1, 0);
+ expect(after[top]![1]).toBeLessThan(-0.5);
+ });
+
+ it('a clip passes through its keys exactly, and a loop joins up', () => {
+ const a: AnimationSource = { name: 'wave', duration: 2, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 0], [0.5, 0, 0, 40], [1.5, 0, 0, -40]] }] };
+ const m = bake([a]);
+ const near = (x: readonly (readonly number[])[], y: readonly (readonly number[])[]): number =>
+ Math.max(...x.map((p, i) => Math.hypot(p[0]! - y[i]![0]!, p[1]! - y[i]![1]!, p[2]! - y[i]![2]!)));
+ expect(near(pose(m, a, 0), pose(m, a, 2))).toBeLessThan(1e-4);
+ const key: AnimationSource = { name: 'k', duration: 2, loop: false, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 40]] }] };
+ expect(near(pose(m, a, 0.5), pose(m, key, 0))).toBeLessThan(1e-4);
+ });
+
+ it('phase puts a second joint behind the first', () => {
+ const keys = [[0, 0, 0, 30], [0.5, 0, 0, -30]];
+ const a: AnimationSource = { name: 'p', duration: 1, tracks: [{ joint: 'mid', rotate: keys, phase: 0.5 }] };
+ const b: AnimationSource = { name: 'q', duration: 1, tracks: [{ joint: 'mid', rotate: keys }] };
+ const m = bake([a, b]);
+ const [x, y] = [pose(m, a, 0.5), pose(m, b, 0)];
+ expect(Math.hypot(x[0]![0] - y[0]![0], x[0]![1] - y[0]![1])).toBeLessThan(1e-4);
+ });
+
+ it('move shifts a joint: a whole body bobs', () => {
+ const a: AnimationSource = { name: 'bob', duration: 1, loop: false, tracks: [{ joint: 'base', move: [[0, 0, 0, 0], [1, 0, 1, 0]] }] };
+ const m = bake([a]);
+ pose(m, a, 1).forEach((p, v) => expect(p[1]).toBeCloseTo(m.poly.positions[v]![1] + 1, 3));
+ });
+
+ it('mirror makes both sides, each with its own parent side', () => {
+ const rig: RigSource = { joints: [
+ { name: 'hips', at: [0, 1, 0] },
+ { name: 'leg', at: [0.3, 1, 0], parent: 'hips', mirror: 'x' },
+ { name: 'foot', at: [0.3, 0, 0], parent: 'leg', mirror: 'x' },
+ ] };
+ const m = bake([], rig, [{ op: 'box', size: 2 }]);
+ const names = m.rig!.joints.map((j) => j.name);
+ expect(names).toEqual(expect.arrayContaining(['leg.L', 'leg.R', 'foot.L', 'foot.R']));
+ const footR = m.rig!.joints.find((j) => j.name === 'foot.R')!;
+ expect(m.rig!.joints[footR.parent]!.name).toBe('leg.R');
+ expect(footR.at[0]).toBeCloseTo(-0.3, 6);
+ });
+});
+
+describe('the files', () => {
+ it('the mesh carries the joints, the weights and one bind matrix each, and no lightmap', () => {
+ const m = bake();
+ const back = decodeMesh(encodeMesh(m.data));
+ expect(back.channels.find((c) => c.semantic === MeshChannel.Joints)).toBeDefined();
+ expect(back.channels.find((c) => c.semantic === MeshChannel.Weights)).toBeDefined();
+ expect(back.channels.find((c) => c.semantic === MeshChannel.TexCoord1)).toBeUndefined();
+ expect(back.inverseBindMatrices!.length).toBe(3 * 16);
+ const mid = m.rig!.joints.findIndex((j) => j.name === 'mid');
+ expect(back.inverseBindMatrices![mid * 16 + 13]).toBeCloseTo(-1, 6);
+ });
+
+ it('a clip addresses each joint by its path under the root, and keeps to the short way round', () => {
+ const m = bake([bend(350)]);
+ const doc = timelineOf(bend(350), m.rig!.joints) as { duration: number; wrapMode: string; tracks: { childPath: string; component: string; channels: { property: string; keyframes: { value: number }[] }[] }[] };
+ expect(doc.duration).toBe(1);
+ expect(doc.wrapMode).toBe('once');
+ expect(doc.tracks[0]!.childPath).toBe('base/mid');
+ expect(doc.tracks[0]!.component).toBe('Transform');
+ expect(doc.tracks[0]!.channels.map((c) => c.property)).toEqual(['rotation.x', 'rotation.y', 'rotation.z', 'rotation.w']);
+ const [z, w] = [doc.tracks[0]!.channels[2]!.keyframes, doc.tracks[0]!.channels[3]!.keyframes];
+ for (let i = 1; i < z.length; i++) {
+ const dot = z[i]!.value * z[i - 1]!.value + w[i]!.value * w[i - 1]!.value;
+ expect(dot).toBeGreaterThan(0);
+ }
+ });
+
+ it('a looping clip says so', () => {
+ const a: AnimationSource = { name: 'w', duration: 1, tracks: [{ joint: 'mid', rotate: [[0, 0, 0, 30]] }] };
+ expect((timelineOf(a, bake([a]).rig!.joints) as { wrapMode: string }).wrapMode).toBe('loop');
+ });
+
+ it('the prefab has the mesh on its root, the joints under it by name, the skin naming them in bind order', () => {
+ const m = bake();
+ const doc = prefabOf('Limb', 'assets/Limb.esmesh', m.rig!.joints, true, 'assets/Limb_bend.estimeline') as { rootEntityId: string; entities: { prefabEntityId: string; name: string; parent: string | null; children: string[]; components: { type: string; data: Record<string, unknown> }[] }[] };
+ const root = doc.entities.find((e) => e.prefabEntityId === doc.rootEntityId)!;
+ const comp = (t: string) => root.components.find((c) => c.type === t)!.data;
+ expect(comp('MeshRenderer')).toMatchObject({ mesh: 'assets/Limb.esmesh', lit: true, opaque: true, cullBackfaces: true });
+ expect(comp('TimelinePlayer')).toMatchObject({ timeline: 'assets/Limb_bend.estimeline', playing: true });
+ const ids = comp('MeshSkin').joints as string[];
+ expect(ids.map((id) => doc.entities.find((e) => e.prefabEntityId === id)!.name)).toEqual(m.rig!.joints.map((j) => j.name));
+ const mid = doc.entities.find((e) => e.name === 'mid')!;
+ expect(mid.components[0]!.data.position).toEqual({ x: 0, y: 1, z: 0 });
+ expect(doc.entities.find((e) => e.prefabEntityId === mid.parent)!.name).toBe('base');
+ expect(root.children.length).toBe(1);
+ });
+});
+
+describe('looking at it', () => {
+ const a = bend(90);
+ const m = bake([a]);
+ const darkCount = (s: { width: number; height: number; rgba: Uint8Array }): number => {
+ let n = 0;
+ for (let i = 0; i < s.width * s.height; i++) if (s.rgba[i * 4 + 2]! === 0 && s.rgba[i * 4]! === 255) n++;
+ return n;
+ };
+
+ it('draws the bones over the body by default', () => {
+ expect(darkCount(renderRigSheet(m, [a], { views: ['front'], panel: 128 }))).toBeGreaterThan(20);
+ expect(darkCount(renderRigSheet(m, [a], { views: ['front'], panel: 128, skeleton: false }))).toBe(0);
+ });
+
+ it('a strip shows a clip over time, at one scale', () => {
+ const s = renderRigSheet(m, [a], { animation: 'bend', times: [0, 1], views: ['front'], panel: 96 });
+ expect(s.width).toBe(96 * 2 + 1);
+ const silhouette = (col: number): number => {
+ let n = 0;
+ for (let y = 0; y < 96; y++) for (let x = 0; x < 96; x++) if (s.rgba[((97 + y) * s.width + col * 97 + x) * 4]! < 60) n++;
+ return n;
+ };
+ // Different shapes, and close to the same amount of body: a bend, not a zoom.
+ expect(Math.abs(silhouette(0) - silhouette(1)) / silhouette(0)).toBeLessThan(0.2);
+ let differ = 0;
+ for (let y = 0; y < 96; y++) for (let x = 0; x < 96; x++) if (s.rgba[((97 + y) * s.width + x) * 4] !== s.rgba[((97 + y) * s.width + 97 + x) * 4]) differ++;
+ expect(differ).toBeGreaterThan(100);
+ });
+
+ it('shows who moves what, or how far one joint reaches', () => {
+ const all = renderRigSheet(m, [a], { weights: true, views: ['front'], panel: 96, skeleton: false });
+ expect(all.width).toBe(96);
+ const one = renderRigSheet(m, [a], { weights: 'mid', views: ['front'], panel: 96, skeleton: false });
+ expect(one.rgba).not.toEqual(all.rgba);
+ });
+
+ it('refuses what it cannot show', () => {
+ expect(() => renderRigSheet(m, [a], { animation: 'nope' })).toThrow(/no animation "nope"; there are bend/);
+ expect(() => renderRigSheet(m, [a], { weights: 'nope' })).toThrow(/no joint "nope"/);
+ const still = bakeModel({ version: 1, steps: [{ op: 'box', size: 1 }] });
+ expect(() => renderRigSheet(still, [], { weights: true })).toThrow(/no rig/);
+ });
+});
+
+describe('refusals name what to change', () => {
+ const source = (extra: Record<string, unknown>) => ({ version: 1, steps: [limb], ...extra });
+
+ it('a rig that does not hold together', () => {
+ expect(() => validateModelSource(source({ rig: { joints: [] } }))).toThrow(/at least one joint/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a', at: [0, 0, 0] }, { name: 'b', at: [0, 1, 0] }] } }))).toThrow(/one root; these have no parent: a, b/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a', at: [0, 0, 0] }, { name: 'b', at: [0, 1, 0], parent: 'c' }] } }))).toThrow(/parent "c", which is not a joint/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a', at: [0, 0, 0], parent: 'b' }, { name: 'b', at: [0, 1, 0], parent: 'a' }] } }))).toThrow(/one root/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a', at: [0, 0, 0] }, { name: 'a', at: [0, 1, 0], parent: 'a' }] } }))).toThrow(/two joints called "a"/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a/b', at: [0, 0, 0] }] } }))).toThrow(/cannot hold a slash/);
+ expect(() => validateModelSource(source({ rig: { joints: [{ name: 'a', at: [0, 0, 0], bend: 1 }] } }))).toThrow(/unknown field "bend"/);
+ const many = Array.from({ length: 70 }, (_, i) => ({ name: `j${i}`, at: [0, i, 0], ...(i ? { parent: `j${i - 1}` } : {}) }));
+ expect(() => validateModelSource(source({ rig: { joints: many } }))).toThrow(/at most 64/);
+ });
+
+ it('clips that do not fit the rig', () => {
+ const rig = limbRig;
+ const anim = (tracks: unknown[], extra = {}) => source({ rig, animations: [{ name: 'a', duration: 1, tracks, ...extra }] });
+ expect(() => validateModelSource(source({ animations: [] }))).toThrow(/no rig/);
+ expect(() => validateModelSource(anim([{ joint: 'elbow', rotate: [[0, 0, 0, 0]] }]))).toThrow(/no joint "elbow"; there are base, mid, tip/);
+ expect(() => validateModelSource(anim([{ joint: 'mid' }]))).toThrow(/neither rotate nor move/);
+ expect(() => validateModelSource(anim([{ joint: 'mid', rotate: [[2, 0, 0, 0]] }]))).toThrow(/time 2 is outside/);
+ expect(() => validateModelSource(anim([{ joint: 'mid', rotate: [[0.5, 0, 0, 0], [0.2, 0, 0, 0]] }]))).toThrow(/time order/);
+ expect(() => validateModelSource(anim([{ joint: 'mid', rotate: [[0, 0, 0]] }]))).toThrow(/every key is \[time, x, y, z\]/);
+ expect(() => validateModelSource(anim([{ joint: 'mid', rotate: [[0, 0, 0, 0]] }, { joint: 'mid', move: [[0, 0, 0, 0]] }]))).toThrow(/two tracks for joint/);
+ expect(() => validateModelSource(source({ rig, animations: [{ name: 'a b', duration: 1, tracks: [] }] }))).toThrow(/file name/);
+ });
+
+ it('an atlas on a rigged mesh, with what to use', () => {
+ expect(() => validateModelSource(source({ rig: limbRig, uvAtlas: true }))).toThrow(/Use uv steps/);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index deef83c4c..ffecd8d1b 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -3831,9 +3831,9 @@ export const NOTED = {
"perf(render): Voxel GI reads its cache plainly where nothing else writes it, and skips empty slots when it ages them, so a phone spends far less on its upkeep and its rays":
{"note":"Voxel GI costs less on mobile GPUs."},
"fix(native): every native host draws from one pacer, its present's own block and the game's cap, so an Android frame near its budget holds 60 fps instead of dropping to 30":
- {"note":"Android frames no longer drop to 30 fps when they come close to the frame budget."},
+ {"note":"Android frames no longer drop to 30 fps when they come close to the frame budget"},
"fix(native): a platform only names the font face a glyph needs, and the rasterizer maps each face's file once, so a new glyph no longer reads a whole system font":
- {"note":"A new glyph on a native build no longer stalls the frame reading a whole system font."},
+ {"note":"A new glyph on a native build no longer stalls the frame reading a whole system font"},
"fix(native): Android draws a face the shared rasterizer cannot read with its own, as Apple does, so a native build shows the system's CJK text":
{"note":"A native Android build shows CJK text from the system font."},
"perf(text): a new glyph is drawn and distance-transformed at 2x rather than 4x, and the transform skips what cannot change its bytes, so a phone pays a third of what it did":
@@ -3866,6 +3866,10 @@ export const NOTED = {
{"note":"A sea seen from above is no longer scored with straight lines."},
"feat(mcp): an agent builds a 3D model in the editor from steps, changes it by one number, and sees every side of it":
{"note":"AI agents can build 3D models in the editor from steps."},
+ "feat(modeling): a model can have a rig with weights found over its surface, and clips of a few keys, so what an agent builds can move":
+ {"note":"AI agents can build 3D models in the editor from steps."},
+ "feat(mcp): model_write rigs and animates a model, and model_views shows the bones, the weights and a clip through time":
+ {"note":"AI agents can build 3D models in the editor from steps."},
"feat(modeling): lathe and loft make curved solids from a few points, so an agent can build vases, pipes, handles and hulls":
{"note":"AI agents can build 3D models in the editor from steps."},
"feat(mcp): model_write turns outlines and joins sections into curved solids":
@@ -3880,4 +3884,48 @@ export const NOTED = {
{"note":"AI agents can build 3D models in the editor from steps."},
"feat(modeling): a model is the steps that build it, baked to the mesh a scene draws, so one number changes the whole shape":
{"note":"AI agents can build 3D models in the editor from steps."},
+ "feat(quality): a phone or tablet takes Medium when a project names no level, and an orthographic camera is left out of its temporal smoothing":
+ {"note": "Phones and tablets take Medium when a project names no level."},
+ "feat(settings): the quality level can be left at Default, which holds a phone to Medium, and Off is a choice that holds there":
+ {"note": "Phones and tablets take Medium when a project names no level."},
+ "perf(water): a sea on a phone costs half of what it did, 45 ms to 21.5, with the same picture":
+ {"note": "Water costs a phone half of what it did."},
+ "feat(templates): the blank template starts in the linear pipeline":
+ {"note": "The blank template starts in the linear pipeline."},
+ "fix(quality): a level can skip every effect there is, where past thirty-two the rest quietly kept running":
+ {"note": "A quality level can skip every effect there is."},
+ "feat(render): an Ambient light can be two colours, sky and ground, without baking a panorama":
+ {"note": "An Ambient light can be two colours, sky and ground."},
+ "fix(render): an opaque mesh tests and writes depth whatever its material says, so a new material no longer shows a sphere's inside":
+ {"note": "A new material no longer shows the inside of a sphere."},
+ "fix(asset): a texture made or rewritten from JS leaves the device's unit bound as it was, so a scene view no longer goes black for good after a heightmap loads":
+ {"note": "A scene view no longer goes black for good after a heightmap loads."},
+ "fix(render): with no post-process chain, a blended pane in front of water is drawn over it instead of under it":
+ {"note": "A blended pane in front of water is drawn over it."},
+ "feat(render): a material past seven textures takes the units below 8 its shader never reads, and Water gains a foam texture":
+ {"note": "A material can hold more than seven textures, and Water gains a foam texture."},
+ "fix(runtime): a restarted scene draws its images again instead of white":
+ {"note": "A restarted scene draws its images again instead of white."},
+ "fix(editor): a saved layout of empty groups no longer leaves an editor with no viewport, and a plugin writing its cache is not reloaded on Windows":
+ {"note": "A saved layout of empty groups no longer leaves an editor with no viewport."},
+ "fix(project): a setting the manifest would refuse is not written, so one bad field can no longer leave a project that does not open":
+ {"note": "A setting the manifest would refuse is not written."},
+ "feat(mcp): every row of Project Settings reads and writes by id, and a partial write keeps what it did not name":
+ {"note": "Every row of Project Settings can be read and written by id over MCP."},
+ "fix(editor): a script-built mesh's colour stays as typed, the graph and terrain pickers show the true colour, and a lit mesh in gamma offers linear":
+ {"note": "A mesh's colour draws as picked."},
+ "fix(editor): a picked colour draws as its swatch on a mesh, and a new surface material tests depth":
+ {"note": "A mesh's colour draws as picked."},
+ "fix(inspector): one box of a vector row edits a material or a clip, and a Rotation scrub survives the pole":
+ {"note": "One box of a vector row edits a material or a clip, and Rotation boxes behave."},
+ "fix(inspector): typing one box of a Rotation row turns that axis, where it stored NaN in all four components":
+ {"note": "One box of a vector row edits a material or a clip, and Rotation boxes behave."},
+ "fix(mcp): a save is no undo step, so Undo after it takes back the edit instead of the save":
+ {"note": "Undo after a save takes back the edit over MCP."},
+ "fix(lighting): the bake's bounces start at 8 and go to 16 in the panel, as the SDK default now does":
+ {"note": "A lightmap bake's bounces start at 8 and go to 16 in the panel."},
+ "fix(ci): the WebGL2 Voxel GI scene is not launched on a runner with no GPU, where it kills the GPU process before it draws":
+ {"internal": "CI only: which scenes a runner launches."},
+ "fix(templates): the 3D templates' camera no longer names a field Camera dropped, and the debug channel's test knows the world the stats read":
+ {"internal": "The stale field was dropped on load either way, so no frame changes; the rest is a test."},
};