3D engine that AI agents drive
Merge main into the soft body film tests
Nguyen Xuan An779253ababc8
diff --cc docs/physics.mdindex 87d3fe78,62733c51..b8cfef00--- a/docs/physics.md+++ b/docs/physics.md@@@ -430,8 -430,9 +430,9 @@@ A recipe `cloth` section records deform- `{ "attach": "leftHand", "shape": "ball", "radius": 0.08, "offset": [0, 0, 0] }` is a ball on a node. `{ "attach": "upperArm", "shape": "capsule", "to": "lowerArm", "radius": 0.05 }` runs a capsule from one node (plus `offset`) to another (plus `toOffset`). Radius 0.001 to 10 m.- `{ "armature": "rig", "radiusFraction": 0.2, "exclude": ["head"] }` adds one capsule per bone, head to tail, the radius a fraction (0.02 to 0.5, default 0.2) of the bone length as the ragdoll fits it. A bone with no length is skipped.- A sheet's `stiffnessGroup: "sleeve"` names a vertex group whose weights scale stretch and bend stiffness per vertex; a constraint is as stiff as the softer of its two ends, and a weight under 0.05 counts as 0.05 so the vertex does not drift off.- - `selfCollision: true` makes the sheet's own particles push each other apart, so cloth folded over itself stays in layers instead of passing through. The contact radius is 0.9 of half the shortest edge, a spatial hash keeps it near linear, two particles joined by an edge are skipped, and a sheet over 2000 particles fails with the count and the fix. It is per particle, so a layer can still slip between particles on a mesh much coarser than the fold. The fold test (`tests/soft-body.test.js`) keeps every unjoined pair at least 0.8 of the contact distance apart; without it the same fold interpenetrates.+ - `selfCollision: true` makes the sheet's own particles push each other apart, so cloth folded over itself stays in layers instead of passing through. The contact radius is 0.9 of half the shortest edge, or the sheet's `thickness` when the sheet sets one (never above half the shortest edge), so a thin cloth on a coarse grid stacks its layers 2 x thickness apart instead of about 4 cm. This changes existing scenes that set both `selfCollision: true` and `thickness`: they used to ignore the thickness and now stack layers 2 x thickness apart, so their baked frames differ; pass `selfCollision: { radius }` to keep a fixed radius. A scene that sets no thickness keeps the old radius (`tests/cloth-self-collision-thickness.test.js`). a spatial hash keeps it near linear, two particles joined by an edge are skipped, and a sheet over 2000 particles fails with the count and the fix. It is per particle, so a layer can still slip between particles on a mesh much coarser than the fold. The fold test (`tests/soft-body.test.js`) keeps every unjoined pair at least 0.8 of the contact distance apart; without it the same fold interpenetrates.- `softBody: { "shape": 0.2, "pressure": 1 }` turns the sheet node into a jelly: `shape` (0 to 1) pulls every particle toward the best rigid fit of the rest shape (shape matching, rotation extracted with a warm-started iteration), and `pressure` (0 to 3) is a volume constraint that holds that multiple of the rest volume. Use 1 for a jelly that keeps its volume and squashes on impact; below 1 the body deflates to that volume before it lands (0.85 is a body 15 percent smaller from the first frame), above 1 it inflates. Pressure needs a closed mesh (every edge in two triangles) and fails naming the open edge otherwise; set `pressure: 0` to keep only shape matching. A box or sphere geometry is closed. It rests on the colliders like cloth and bakes to the same frames every run.+ - `softBody.bounce` (0 to 1, default 0) sets how much of its landing speed the jelly keeps when it turns around: the body remembers the speed its centre arrived with, and when the centre stops moving into the collider it adds `bounce` times that speed to every free particle along the contact normal. It never slows a body that already leaves faster, and approach speeds under 0.5 m/s are ignored so a resting jelly does not buzz. Leaving it out bakes the same frames as before. How soft the jelly is needs no new field: lower `softBody.shape` for a looser body and lower the sheet's `stiffness` for a body that squashes further (the table below). `damping` is air drag on every particle, so a high `damping` also takes height off the bounce (with `damping: 0` and `bounce: 1` the cube returns to 1.85 m of its 1.95 m drop).- A sheet on a mesh geometry can hold chosen vertices with `pinGroup: "hem"`, the name of a vertex group made by a `vertex_group` mesh edit; vertices of weight 0.5 or more are pinned, on their own or together with `pin`. An unknown name fails and lists the groups the mesh has.- A sheet's `pinTo: { "node": "chest", "weight": 1 }` makes the pinned edges (`pin`) ride that node, keeping their rest offset in its space. Weight 1 holds them on the node; 0.01 to under 1 pulls them toward it by that fraction on every pass, so they sag and lag.- Up to 64 moving balls and capsules, counting every bone. The node poses come from the recipe's motion tracks and IK; a physics ragdoll or body is not followed. Points are pushed out of the capsules, not whole triangles, so a coarse sheet can still cut a corner of a thin limb: use a finer grid than the limb is wide. A cape on the reference walker (24 by 24 grid, 5 s, about 2.5 s to bake on the dev machine) stays outside every limb capsule in `tests/cloth-moving.test.js`.@@@ -440,14 -441,29 +441,37 @@@- A seek replaces the sheet's part, so the renderer sees a topology change (`topologyRevision`); the MCP preview uses `replaceParts` and `scene_look` with `motionTime` repeats the same PNG bytes (`npm run smoke:agent-cloth`).- `scene_inspect` returns a `cloth` report: particles, triangles, pinned count and the final bounds of each sheet.+Soft body films (`scripts/soft-body-film.mjs`, `tests/soft-body-film.test.js`): a jelly cube landing and a flag lowered into folds on itself, each one recipe that the test bakes twice and the preview script renders.++- Jelly cube: a rounded box 0.5 m across (384 particles) dropped from 1.95 m, `softBody { shape: 0.15, pressure: 1 }`. It first rests on the floor at frame 48 (0.8 s) and is dented 0.0503 m (10 percent of its height) at frame 49, one step later, while its width grows from 0.5 m to 0.547 m; the frame before contact has no dent.+- Flag: a 38 by 38 grid (1444 particles, the most that fits the cloth work limit for a 3 s film), 1 m tall and 1.6 m wide, held by its two top corners (`pin: ['topLeft', 'topRight']`, `pinTo` a pole) so the free edge sags and billows instead of falling as a flat card. The pole whips 0.5 m from side to side and 0.4 m front to back while it drops 2 m in 0.8 s, so the sheet ripples in the air, then zig-zags 0.3 m from side to side while it moves 2 m across the floor, which pulls the cloth into a thin crumpled layer. The sheet sets `bend: 0.0002`, `stiffness: 0.6`, `damping: 1.5`, `friction: 0.4` and `thickness: 0.002`, so its self collision layers stack 4 mm apart (earlier rounds: 3.7 cm from the mesh-based radius and a 0.547 m block, then a flat card with a smooth sheet at rest, then one smooth ridge). Measured in the test: first touch at step 39; normal spread (0 flat, toward 1 scattered) is 0 at the start, 0.480 mid fall (step 27, the test fails below 0.25) and 0.674 at rest (fails below 0.5), the sheet bends 0.81 m out of its plane mid fall, and at the final step (179) the highest vertex is 0.0855 m above the floor (a small fold; the test fails above 0.1 m), half of the cloth lies under 0.0042 m and nine tenths of it under 0.0308 m (the test fails above 0.05 m). With `selfCollision: true` the closest pair of particles not joined by an edge is never under 1.8 mm apart in any sampled frame after touch (0.0021 m); the same film without it comes down to 0.2 mm. Those numbers come from the same bake the preview renders, and the film JSON (`soft-body-film.json`) records the highest, median and 90th percentile height at rest. The bake takes 7 to 17 s on the loaded dev machine (about 30 to 40 ms a step with self collision); a 44 by 44 grid exceeds the 2000 particle self collision limit and a 4 s film at 38 by 38 exceeds the cloth work limit. The look is a red and cream striped cloth (a `graph` pattern, so every fold bends the stripes) on a mid grey floor under a low grazing key light (`environment.lightDirection`, fill and ambient lowered), framed from 5.4 m so the cloth fills about a fifth of the picture width.+- Both films bake to the same bytes in every frame across two runs; the test compares every frame's vertex buffer.+- Frame time, in milliseconds per solver step (the bake is one call, so this is its total over the steps, on a machine shared with other sessions, so read it as a range): jelly 6 to 9 ms a step over 120 steps, flag 30 to 40 ms a step over 180 steps (29 ms a step without self collision). Sampling a recorded frame for playback costs about 1.5 to 2 ms.+- `node scripts/render-soft-body-film.mjs <folder> [--flag-only]` (run it through gpu-lock) writes a 960 by 540, 60 fps preview MP4 of each film (one video frame per simulation step, so frame N is step N), stills at contact and at the deepest dent for the jelly and at start, mid fall, first touch, settling and final rest for the flag (`--flag-only` skips the jelly cube), `jelly-cube-compare.png` (both side by side with the measured width and height drawn on it) and `soft-body-film.json` with every number, its unit and its step. It needs `npm run build:lib` first. No default changes.++ Jelly feel (`tests/soft-body-bounce.test.js`, measured): the 0.5 m rounded cube of `tests/support/jelly-cube.mjs`, dropped from 1.95 m onto the floor, `damping: 2`, `bend: 0`, `stiffness: 0.9` unless listed, `pressure: 1`. It lands at step 48. Dent is the rest height minus the shortest height (rest height 0.5 m); rebound is how far the lowest vertex rises above the floor before the cube falls back.++ | `shape` | `stiffness` | Dent | Rebound at `bounce` 0 | at 0.5 | at 1 |+ | --- | --- | --- | --- | --- | --- |+ | 1 | 0.9 | 0.026 m | 0.102 m | 0.192 m | 0.516 m |+ | 0.15 (default) | 0.9 | 0.050 m | 0.072 m | 0.149 m | 0.469 m |+ | 0.02 | 0.9 | 0.065 m | 0.000 m | 0.093 m | 0.441 m |+ | 0.02 | 0.1 | 0.099 m | 0.000 m | 0.100 m | 0.447 m |++ Before `bounce` existed, `shape` and `damping` were the only knobs that changed the rebound, and none gave a controlled one. Lowering `shape` or `stiffness` deepens the dent (0.026 to 0.099 m), so softness needed no new field. A very low `shape` has a price: with `shape` 0.02 and `stiffness` 0.1, 132 of the cube's 588 triangles turn inside out in the worst frame. That is a real surface fold, and `selfCollision`, `bend`, more solver iterations and a gentler volume step do not remove it (measured). A separate cause made a clean dent look ragged: the film's box has a 0.12 bevel, so each flat side is a single quad between two rim rows, the bulge can only sit on the rims, and at the dent the top rim juts past the sunken top face (a ledge and a stepped outline, with no fold and no inverted triangle). A box with a 0.3 bevel has rows of vertices up the sides, and the bulge becomes a belly.++ Three presets that step clearly, drawn on a box with `bevel` 0.3 and `subdivisions` 4 (`JELLY_PRESETS` and `JELLY_LOOK` in `tests/support/jelly-cube.mjs`, previewed by `scripts/render-jelly-bounce.mjs` and labelled by `scripts/label-jelly-bounce.py`; the preview is 960 by 540 at 60 fps, 4 seconds, from a long lens so the cubes carry no perspective skew, and each cube has a name label and its own colour):++ | Preset (cube colour in the preview) | `shape` | `stiffness` | `bounce` | Dent | Sideways bulge | Rebound | Stops moving at step |+ | --- | --- | --- | --- | --- | --- | --- | --- |+ | firm (blue) | 1 | 0.9 | 0 | 0.029 m | 0.022 m | 0.147 m | 110 |+ | medium (orange) | 0.2 | 0.05 | 0.5 | 0.054 m | 0.068 m | 0.223 m | 156 |+ | soft (pink) | 0.06 | 0.02 | 0.9 | 0.085 m | 0.106 m | 0.349 m | 210 |++ The dent steps 1.9 and 1.6 times and the rebound 1.5 and 1.6 times, so each step shows in the dent and in the air under the cube. Every triangle keeps facing out (0 inverted in the bake, 0 folded at the dent), the top face is level within 0.03 degrees at the deepest dent and at the end, and the widest row of the soft cube at its dent sits at 5 percent of its height (a belly low on the body), not at the top rim. In flight the soft cube is 43.3 to 53.1 cm tall and 48.3 to 58.1 cm wide (the firm one 48.3 to 50.6 cm tall), so it stretches and wobbles while it is airborne.++ With `damping: 0` and the default shape, `bounce` 1 sends the cube back to 1.85 m of its 1.95 m drop.+Blender comparison (`scripts/blender-cloth.py`, Blender 5.2.2, default stiffness, mass set from the same kg/m2; `tests/cloth-blender.test.js`):| Case | Result |diff --cc src/agent-scene-capabilities.jsindex 75de70a7,a93e0943..80f453d1--- a/src/agent-scene-capabilities.js+++ b/src/agent-scene-capabilities.js@@@ -312,7 -312,7 +312,7 @@@ export const AGENT_SCENE_CAPABILITIEScloth: Object.freeze({sheetGeometry: 'grid { segments 1-64 }: unit sheet in XY facing +Z, sized by node scale; any triangle mesh works',defaults: Object.freeze({ stiffness: 0.9, bend: 0.1, density: 0.2, damping: 0.3, friction: 0.3, thickness: 0.01, colliderShape: 'box' }),- notes: 'density kg/m2, damping 1/s, thickness m, wind m/s steady. Colliders { node, shape ball|box } fit geometry bounds; like sheets they are root nodes without a motion track or body. Moving: { attach, shape ball|capsule, radius, offset, to, toOffset } or { armature, radiusFraction, exclude }. Sheet pinGroup: a vertex group whose vertices (weight 0.5 or more) are held. Sheet pinTo { node, weight } rides pinned edges on a node. Sheet stiffnessGroup: vertex group whose weights scale stiffness (floor 0.05). Sheet softBody { shape 0-1 (0.2), pressure 0-3 (1) }: shape matching plus a volume term; pressure needs a closed mesh, a node with an open mesh fails and says to set pressure 0. Sheet selfCollision true: particles of the sheet push apart (radius 0.9 of half the shortest edge, or the sheet thickness when set, up to 2000 particles); cloth does not collide with other sheets.',- notes: 'density kg/m2, damping 1/s, thickness m, wind m/s steady. Colliders { node, shape ball|box } fit geometry bounds; like sheets they are root nodes without a motion track or body. Moving: { attach, shape ball|capsule, radius, offset, to, toOffset } or { armature, radiusFraction, exclude }. Sheet pinGroup: a vertex group whose vertices (weight 0.5 or more) are held. Sheet pinTo { node, weight } rides pinned edges on a node. Sheet stiffnessGroup: vertex group whose weights scale stiffness (floor 0.05). Sheet softBody { shape 0-1 (0.2), pressure 0-3 (1), bounce 0-1 (0) }: shape matching plus a volume term; pressure needs a closed mesh, a node with an open mesh fails and says to set pressure 0. bounce is the share of its landing speed the body keeps when it turns around; lower shape or sheet stiffness makes a softer, deeper dent. Sheet selfCollision true: particles of the sheet push apart (radius 0.9 of half the shortest edge, up to 2000 particles); cloth does not collide with other sheets.',++ notes: 'density kg/m2, damping 1/s, thickness m, wind m/s steady. Colliders { node, shape ball|box } fit geometry bounds; like sheets they are root nodes without a motion track or body. Moving: { attach, shape ball|capsule, radius, offset, to, toOffset } or { armature, radiusFraction, exclude }. Sheet pinGroup: a vertex group whose vertices (weight 0.5 or more) are held. Sheet pinTo { node, weight } rides pinned edges on a node. Sheet stiffnessGroup: vertex group whose weights scale stiffness (floor 0.05). Sheet softBody { shape 0-1 (0.2), pressure 0-3 (1), bounce 0-1 (0) }: shape matching plus a volume term; pressure needs a closed mesh, a node with an open mesh fails and says to set pressure 0. bounce is the share of its landing speed the body keeps when it turns around; lower shape or sheet stiffness makes a softer, deeper dent. Sheet selfCollision true: particles of the sheet push apart (radius 0.9 of half the shortest edge, or the sheet thickness when set, up to 2000 particles); cloth does not collide with other sheets.',limits: Object.freeze({ maxSheets: 8, maxColliders: 16, maxMovingColliders: 64, maxDuration: 10, maxParticles: 5000, framesPerSecond: 60, substeps: 4, iterations: 8 }),}),// Recipe particles: closed-form from seeded launch values, so any time is computed directly and a seek never replays.