A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(animation): a rig's hands held in a named pose that fits any skeleton
yhhdd4644cefaee
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex d71d8b0bd..6b3a97b65 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,6 +16,7 @@ published separately; it ships inside the editor.### Added+- **Hands that hold things.** A `GripPose` holds a rig's hands in a named pose: `pistol`, `handle`, `cup`, `fist` and others, or one a game registers. A pose says how far each finger bends and nothing about bones, so it fits any rig with fingers; a change of pose eases over, a trigger moves the index finger alone, and it goes over what a clip does with the fingers by its weight.- **Clouds that keep time with the game.** Where a Sky Atmosphere's clouds are is now what the wind has done so far, so a wind that changes carries them on instead of moving the whole sky; a Weather's wind reaches them, their shapes change as they go, and a paused game has a still sky. They can have a second layer with its own height, grain and wind, and `skyClockState()` keeps their place in a save. Self-shadowing reaches through a deep cloud, edges wear into billows, and the lit edge near the sun is narrower.- **Ground that is a volume.** The optional `esengine/voxel` module adds `VoxelTerrain`, beside the height-field Terrain: it holds caves and overhangs, and can be carved, filled and painted with a sphere, a box or a capsule while the game runs. Only the chunks an edit reached are made again, with their collision, and a save holds only what was changed, chunk by chunk. A sleeping body is now woken when the collision under it changes, on a Terrain as well, where before it stayed in the air.- **What stands on the ground can blend into it.** The `builtin:ground-blend` material fades a rock or a foundation into whatever is behind it, on both backends, with nothing baked, and can bring a second layer up from the ground or paint one with vertex colours. `RoadDecal` lays a mesh along a Spline over a terrain with soft edges. The `snowCover` post effect and a Snow weather lay snow on whatever faces the sky, terrain and meshes alike.diff --git a/desktop b/desktopindex 5d08f5a40..bf5e07137 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit 5d08f5a4011d8aa11e54f80898585ade7421faba+Subproject commit bf5e0713710930cf93d79ddf8c72ab01d473c245diff --git a/docs/astro/astro.config.mjs b/docs/astro/astro.config.mjsindex 2e08db611..8a027583f 100644--- a/docs/astro/astro.config.mjs+++ b/docs/astro/astro.config.mjs@@ -241,6 +241,7 @@ export default defineConfig({{ label: 'Overview', translations: { 'zh-CN': '总览' }, slug: 'animation/overview' },{ label: 'Timeline', translations: { 'zh-CN': '时间轴' }, slug: 'animation/timeline' },{ label: 'Spring bones', translations: { 'zh-CN': '弹簧骨骼' }, slug: 'animation/spring-bones' },+ { label: 'Grip Pose', translations: { 'zh-CN': '握持姿势' }, slug: 'animation/grip-pose' },{ label: '2D Rigs', translations: { 'zh-CN': '2D 骨骼' }, slug: 'animation/rig-2d' },{ label: 'Spine Animation', translations: { 'zh-CN': 'Spine 动画' }, slug: 'animation/spine' },{ label: 'DragonBones Animation', translations: { 'zh-CN': 'DragonBones 动画' }, slug: 'animation/dragonbones' },diff --git a/docs/astro/src/content/docs/animation/grip-pose.mdx b/docs/astro/src/content/docs/animation/grip-pose.mdxnew file mode 100644index 000000000..26f94d8a5--- /dev/null+++ b/docs/astro/src/content/docs/animation/grip-pose.mdx@@ -0,0 +1,80 @@+---+title: Grip Pose+description: Hold a character's hands in a named pose, a pistol grip, a handle, a cup, that fits any rig with fingers.+---++A `GripPose` holds a rig's hands the way you name: `pistol`, `handle`, `cup`. A pose says only how far each finger bends, how far they fan out and how far the thumb comes across, and nothing in it names a bone, so one pose fits any rig. Which way those bends go is worked out from each rig's own hand, the first time it is held.++It shapes the hand. Putting the hand on an object is up to whatever attaches it, usually an entity under the hand joint and two-bone IK for the arm.++## Hold a hand++Put a `GripPose` on the rig's root and name a pose for either hand:++```ts+import { GripPose } from 'esengine';++world.insert(hero, GripPose, { rightPose: 'pistol', leftPose: 'support' });++// Squeeze: the index finger goes from resting to pressing, and nothing else moves.+world.update(hero, GripPose, (grip) => { grip.rightTrigger = 1; });+```++| Field | What it does |+|---|---|+| `rightPose`, `leftPose` | The pose each hand takes. Empty fades that hand out and leaves it to whatever else moves it. |+| `rightWeight`, `leftWeight` | How much of the pose applies, over what a clip is doing with the fingers. |+| `rightTrigger`, `leftTrigger` | How far the index finger has gone from resting to pressing, in a pose that has both. |+| `blendSeconds` | About how long a change of pose takes. It starts from still and comes to rest, with no jump. |+| `rightHand`, `leftHand` | The hand joint, by path from this entity. Empty looks for it by name. |++It runs after the animator, so a walk cycle goes on moving the arm while the pose holds the fingers, and works with no animator at all. It runs while the game plays, not while a scene is laid out.++## The poses++| Name | For |+|---|---|+| `open` | A flat hand, as the rig rests. |+| `relaxed` | A hand hanging at rest. |+| `fist` | Closed, the thumb over the fingers. |+| `pistol` | A pistol's grip, the index finger along the frame. `trigger` presses it. |+| `support` | A hand under a rifle's foregrip. |+| `handle` | A hammer, a sword, a tool's handle. |+| `pinch` | Thumb and index finger together. |+| `cup` | Round a mug or a ball, fingers apart. |++To add your own, or replace one by name:++```ts+import { registerGripPose } from 'esengine';++registerGripPose('claw', {+ thumb: { curl: 0.6, opposition: 0.7 },+ index: { curl: 0.9, base: 0.2 },+ middle: { curl: 0.9, base: 0.2 },+ ring: { curl: 0.9, base: 0.2 },+ pinky: { curl: 0.9, base: 0.2 },+});+```++| Number | What it says |+|---|---|+| `curl` | How far the finger bends, 0 as the rig rests to 1 closed. |+| `base` | How far its first joint bends, where that differs: a finger laid along a frame, or hooked. Defaults to `curl`. |+| `spread` | How far the fingers fan out, -1 to 1. |+| `opposition` | The thumb only: how far it comes across the palm, 0 beside the hand to 1 facing the fingers. |+| `trigger` | On the pose: the index finger as `rightTrigger` takes it to 1. |++## On your rig++The hand is found by name: a joint called something like `RightHand` or `hand_r`, with fingers named `thumb`, `index`, `middle`, `ring`, and `pinky` or `little` under it. Bones in the palm (`index_metacarpal_r`) are passed through. Where a rig's names say nothing of the kind, name the joint in `rightHand`. If its fingers are not named either, it cannot be held.++A rig with fewer fingers does without the rest, and says which in the log. `gripPoseStatus(app, entity)` answers what was found on each hand.++## Limits++- **It does not touch what is held.** A pose is a shape, not a fit: fingers can sink into a wide grip or stop short of a narrow one. Use `curl` for the object, or a pose of its own.+- **A hand's numbers are worked out once,** from how the rig stands when it is first held. A rig whose hand is mid-animation by then has its axes taken from that. In practice they are close.+- **Rigs with a negative scale on an arm are not handled.** A mirrored bone turns the wrong way.+- **Checked on a real rig of one kind.** The poses were tuned on a Mixamo-named character, and the same hand built with bones facing other ways and under other names gives the same fingertips, but no second real rig has been looked at.+- **Not measured on a device.** It is a few quaternions for each of up to 30 joints a frame, and nothing once settled on a rig no clip moves.diff --git a/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdxnew file mode 100644index 000000000..c0eb5f57a--- /dev/null+++ b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx@@ -0,0 +1,80 @@+---+title: 握持姿势+description: 让角色的手保持一个命名姿势,比如手枪握、手柄握、捧杯,适用于任何带手指骨骼的角色。+---++`GripPose` 让角色的手保持你指定的姿势:`pistol`、`handle`、`cup`。姿势只描述每根手指弯多少、张开多少、拇指对掌多少,不涉及任何骨骼名字,所以同一个姿势可以用在任何骨架上。弯曲的方向在第一次握持时,根据每个角色自己的手算出来。++它只负责手型。把手放到物体上由挂点负责,通常是手部关节下的子实体,加上手臂的 two-bone IK。++## 使用++把 `GripPose` 放在角色的根实体上,给左右手各指定一个姿势:++```ts+import { GripPose } from 'esengine';++world.insert(hero, GripPose, { rightPose: 'pistol', leftPose: 'support' });++// 扣扳机:食指从搭放变成扣下,其他手指不动。+world.update(hero, GripPose, (grip) => { grip.rightTrigger = 1; });+```++| 字段 | 作用 |+|---|---|+| `rightPose`、`leftPose` | 每只手的姿势。留空时这只手淡出,交还给其他驱动它的东西。 |+| `rightWeight`、`leftWeight` | 姿势应用多少,叠加在动画片段对手指的控制之上。 |+| `rightTrigger`、`leftTrigger` | 食指从搭放到扣下的程度,要姿势同时定义了这两个状态才有效。 |+| `blendSeconds` | 切换姿势大约花多久。从静止起步,平稳停下,没有跳变。 |+| `rightHand`、`leftHand` | 手部关节,写从这个实体出发的路径。留空时按名字查找。 |++它在动画系统之后运行,所以走路动画继续带动手臂,姿势负责手指;没有动画系统时也能用。它只在游戏运行时生效,布置场景时不生效。++## 内置姿势++| 名字 | 用途 |+|---|---|+| `open` | 张开的手,即骨架的静止状态。 |+| `relaxed` | 自然垂放的手。 |+| `fist` | 握拳,拇指压在手指上。 |+| `pistol` | 手枪握,食指沿枪身伸直。`trigger` 让它扣下。 |+| `support` | 托在步枪前握下方的手。 |+| `handle` | 锤子、剑、工具的手柄。 |+| `pinch` | 拇指和食指捏合。 |+| `cup` | 捧杯或握球,手指张开。 |++要添加自己的姿势,或者按名字替换内置姿势:++```ts+import { registerGripPose } from 'esengine';++registerGripPose('claw', {+ thumb: { curl: 0.6, opposition: 0.7 },+ index: { curl: 0.9, base: 0.2 },+ middle: { curl: 0.9, base: 0.2 },+ ring: { curl: 0.9, base: 0.2 },+ pinky: { curl: 0.9, base: 0.2 },+});+```++| 数值 | 含义 |+|---|---|+| `curl` | 手指弯多少,0 是骨架静止状态,1 是完全握紧。 |+| `base` | 第一个关节弯多少,用于和其余关节不同的情况,比如手指沿枪身伸直或者勾起。默认等于 `curl`。 |+| `spread` | 手指张开多少,-1 到 1。 |+| `opposition` | 仅拇指:拇指对掌多少,0 贴在手边,1 正对手指。 |+| `trigger` | 写在姿势上:`rightTrigger` 到 1 时食指的状态。 |++## 适配你的骨架++手部按名字查找:名字类似 `RightHand` 或 `hand_r` 的关节,下面的手指名字包含 `thumb`、`index`、`middle`、`ring`、`pinky` 或 `little`。手掌内的骨骼(`index_metacarpal_r`)会被跳过。如果骨架名字完全对不上,就在 `rightHand` 里写明关节路径;如果手指也没有这样的名字,就无法使用。++手指较少的骨架会跳过缺失的手指,并在日志里说明。`gripPoseStatus(app, entity)` 可以查询每只手找到了什么。++## 限制++- **不接触被握的物体。** 姿势只是手型,不做贴合:握把太粗时手指可能陷进去,太细时可能碰不到。可以调 `curl`,或者给这个物体单独做一个姿势。+- **手的参数只计算一次,** 基于第一次握持时骨架的状态。如果那时手正处在动画中,轴就按当时的状态算,实际差别不大。+- **不支持手臂带负缩放的骨架。** 镜像骨骼会转错方向。+- **只在一种真实骨架上检查过。** 姿势是在一个 Mixamo 命名的角色上调的。用骨骼朝向不同、名字不同的同一只手测试,指尖位置一致,但还没有看过第二套真实骨架。+- **没有在真机上测过。** 每帧最多 30 个关节,每个几次四元数运算;在没有动画片段驱动的骨架上,手稳定后就不再有开销。diff --git a/docs/astro/src/data/apiStability.generated.json b/docs/astro/src/data/apiStability.generated.jsonindex 301ec372b..b1a1c27cc 100644--- a/docs/astro/src/data/apiStability.generated.json+++ b/docs/astro/src/data/apiStability.generated.json@@ -3,10 +3,10 @@"counts": {"public": 115,"beta": 178,- "experimental": 2968,+ "experimental": 2978,"deprecated": 3},- "total": 3261,+ "total": 3271,"subsystems": [{"id": "mobile-controls",diff --git a/docs/astro/src/data/componentDocs.ts b/docs/astro/src/data/componentDocs.tsindex 0c0d7c57f..b519b38ed 100644--- a/docs/astro/src/data/componentDocs.ts+++ b/docs/astro/src/data/componentDocs.ts@@ -207,6 +207,7 @@ export const COMPONENT_DOCS: Record<string, ComponentDoc> = {// ── Animation ──────────────────────────────────────────────────────────────Animator: { category: 'animation', summary: 'Runs an animator controller — states, transitions and blend trees — over the entity.', doc: 'animation/overview', anchor: 'animator-state-machine', anchorZh: '动画状态机' },AnimatorRootMotion: { category: 'animation', summary: 'Publishes how far the active animation asks the character to move, for gameplay to hand to a character controller — a request, never a result.', doc: 'animation/overview', anchor: 'root-motion', anchorZh: '根运动' },+ GripPose: { category: 'animation', summary: 'Holds a rig\'s hands in a named pose, a pistol grip, a handle, a cup, that fits any rig with fingers.', doc: 'animation/grip-pose' },SpringBone: { category: 'animation', summary: 'Makes every bone below this root a spring: the chain trails the animated pose by inertia, falls with gravity and is pulled back toward it (hair, tails, cords).', doc: 'animation/spring-bones', anchor: 'the-component', anchorZh: '组件' },SpringBoneCollider: { category: 'animation', summary: 'A sphere or capsule that the spring-bone chains naming it are kept out of.', doc: 'animation/spring-bones', anchor: 'the-component', anchorZh: '组件' },SpriteAnimator: { category: 'animation', summary: 'Plays a flipbook clip by swapping the sprite frame, with speed, looping and frame events.', doc: 'animation/overview', anchor: 'sprite-animation', anchorZh: '精灵动画' },diff --git a/docs/astro/src/data/components.generated.json b/docs/astro/src/data/components.generated.jsonindex 2cd9543c8..5422d34e5 100644--- a/docs/astro/src/data/components.generated.json+++ b/docs/astro/src/data/components.generated.json@@ -2740,6 +2740,80 @@"rippleTexture"]},+ {+ "name": "GripPose",+ "source": "ts",+ "fields": [+ {+ "key": "rightPose",+ "type": "string",+ "default": "empty",+ "doc": "open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone."+ },+ {+ "key": "leftPose",+ "type": "string",+ "default": "empty",+ "doc": "open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone."+ },+ {+ "key": "rightWeight",+ "type": "number",+ "default": "1",+ "doc": "How much of the pose applies, over what a clip is doing with the fingers.",+ "range": "0–1, step 0.01"+ },+ {+ "key": "leftWeight",+ "type": "number",+ "default": "1",+ "doc": "How much of the pose applies, over what a clip is doing with the fingers.",+ "range": "0–1, step 0.01"+ },+ {+ "key": "rightTrigger",+ "type": "number",+ "default": "0",+ "doc": "How far the index finger has gone from resting to pressing, in a pose that has both.",+ "range": "0–1, step 0.01"+ },+ {+ "key": "leftTrigger",+ "type": "number",+ "default": "0",+ "doc": "How far the index finger has gone from resting to pressing, in a pose that has both.",+ "range": "0–1, step 0.01"+ },+ {+ "key": "blendSeconds",+ "type": "number",+ "default": "0.15",+ "doc": "About how long a change of pose takes.",+ "range": "≥ 0, step 0.01"+ },+ {+ "key": "rightHand",+ "type": "string",+ "default": "empty",+ "doc": "The hand joint, by path from this entity. Empty looks for it by name.",+ "advanced": true+ },+ {+ "key": "leftHand",+ "type": "string",+ "default": "empty",+ "doc": "The hand joint, by path from this entity. Empty looks for it by name.",+ "advanced": true+ }+ ],+ "animatable": [+ "rightWeight",+ "leftWeight",+ "rightTrigger",+ "leftTrigger",+ "blendSeconds"+ ]+ },{"name": "GroupOpacity","source": "cpp",diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.mdindex 27f63ac00..23750b6f2 100644--- a/sdk/etc/index.api.md+++ b/sdk/etc/index.api.md@@ -1,7 +1,7 @@# API surface — esengine/index<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->-Symbols: 115 public · 166 beta · 2099 experimental · 3 deprecated+Symbols: 115 public · 166 beta · 2109 experimental · 3 deprecated## ABI_LAYOUT_HASH — const @experimental```@@ -5029,6 +5029,58 @@ static new (opts: GridLayoutOptions): GridLayoutProviderstatic prototype: GridLayoutProvider```+## GripFinger — interface @experimental+```+base: number | undefined+curl: number+spread: number | undefined+```++## GripHandStatus — interface @experimental+```+hand: number | null+joints: number+missing: string[]+pose: string+weight: number+```++## GripPose — const @experimental+```+ComponentDef<GripPoseData>+```++## GripPoseData — interface @experimental+```+blendSeconds: number+leftHand: string+leftPose: string+leftTrigger: number+leftWeight: number+rightHand: string+rightPose: string+rightTrigger: number+rightWeight: number+```++## GripPoseDef — interface @experimental+```+index: GripFinger+middle: GripFinger+pinky: GripFinger+ring: GripFinger+thumb: GripThumb+trigger: GripFinger | undefined+```++## GripThumb — interface @experimental+```+base: number | undefined+curl: number+opposition: number+spread: number | undefined+```+## Ground — type @experimental```(x: number, z: number) => { y: number; up: number } | null@@ -15727,6 +15779,21 @@ Plugin(preserveDrawingBuffer?: boolean): WebGLContextAttributes```+## gripPose — function @experimental+```+(name: string): GripPoseDef | undefined+```++## gripPoseNames — function @experimental+```+(): string[]+```++## gripPoseStatus — function @experimental+```+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null+```+## hasPlayableCta — function @experimental```(): boolean@@ -16852,6 +16919,11 @@ PrefabsPlugin(def: EffectDef): void```+## registerGripPose — function @experimental+```+(name: string, pose: GripPoseDef): void+```+## registerPackagedSideModules — function @experimental```(config: Pick<PackagedGameConfig, "sideModules">): voiddiff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.mdindex 41a76da29..c43254dad 100644--- a/sdk/etc/index.native.api.md+++ b/sdk/etc/index.native.api.md@@ -1,7 +1,7 @@# API surface — esengine/index.native<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->-Symbols: 115 public · 171 beta · 2118 experimental · 3 deprecated+Symbols: 115 public · 171 beta · 2128 experimental · 3 deprecated## ABI_LAYOUT_HASH — const @experimental```@@ -5037,6 +5037,58 @@ static new (opts: GridLayoutOptions): GridLayoutProviderstatic prototype: GridLayoutProvider```+## GripFinger — interface @experimental+```+base: number | undefined+curl: number+spread: number | undefined+```++## GripHandStatus — interface @experimental+```+hand: number | null+joints: number+missing: string[]+pose: string+weight: number+```++## GripPose — const @experimental+```+ComponentDef<GripPoseData>+```++## GripPoseData — interface @experimental+```+blendSeconds: number+leftHand: string+leftPose: string+leftTrigger: number+leftWeight: number+rightHand: string+rightPose: string+rightTrigger: number+rightWeight: number+```++## GripPoseDef — interface @experimental+```+index: GripFinger+middle: GripFinger+pinky: GripFinger+ring: GripFinger+thumb: GripThumb+trigger: GripFinger | undefined+```++## GripThumb — interface @experimental+```+base: number | undefined+curl: number+opposition: number+spread: number | undefined+```+## Ground — type @experimental```(x: number, z: number) => { y: number; up: number } | null@@ -15983,6 +16035,21 @@ Plugin(preserveDrawingBuffer?: boolean): WebGLContextAttributes```+## gripPose — function @experimental+```+(name: string): GripPoseDef | undefined+```++## gripPoseNames — function @experimental+```+(): string[]+```++## gripPoseStatus — function @experimental+```+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null+```+## hasPlayableCta — function @experimental```(): boolean@@ -17118,6 +17185,11 @@ PrefabsPlugin(def: EffectDef): void```+## registerGripPose — function @experimental+```+(name: string, pose: GripPoseDef): void+```+## registerPackagedSideModules — function @experimental```(config: Pick<PackagedGameConfig, "sideModules">): voiddiff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.mdindex 2bd9a4687..a7893554a 100644--- a/sdk/etc/index.node.api.md+++ b/sdk/etc/index.node.api.md@@ -1,7 +1,7 @@# API surface — esengine/index.node<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->-Symbols: 115 public · 168 beta · 2098 experimental · 3 deprecated+Symbols: 115 public · 168 beta · 2108 experimental · 3 deprecated## ANCHOR_AXES — const @experimental```@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProviderstatic prototype: GridLayoutProvider```+## GripFinger — interface @experimental+```+base: number | undefined+curl: number+spread: number | undefined+```++## GripHandStatus — interface @experimental+```+hand: number | null+joints: number+missing: string[]+pose: string+weight: number+```++## GripPose — const @experimental+```+ComponentDef<GripPoseData>+```++## GripPoseData — interface @experimental+```+blendSeconds: number+leftHand: string+leftPose: string+leftTrigger: number+leftWeight: number+rightHand: string+rightPose: string+rightTrigger: number+rightWeight: number+```++## GripPoseDef — interface @experimental+```+index: GripFinger+middle: GripFinger+pinky: GripFinger+ring: GripFinger+thumb: GripThumb+trigger: GripFinger | undefined+```++## GripThumb — interface @experimental+```+base: number | undefined+curl: number+opposition: number+spread: number | undefined+```+## Ground — type @experimental```(x: number, z: number) => { y: number; up: number } | null@@ -15722,6 +15774,21 @@ Plugin(preserveDrawingBuffer?: boolean): WebGLContextAttributes```+## gripPose — function @experimental+```+(name: string): GripPoseDef | undefined+```++## gripPoseNames — function @experimental+```+(): string[]+```++## gripPoseStatus — function @experimental+```+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null+```+## hasPlayableCta — function @experimental```(): boolean@@ -16857,6 +16924,11 @@ PrefabsPlugin(def: EffectDef): void```+## registerGripPose — function @experimental+```+(name: string, pose: GripPoseDef): void+```+## registerPackagedSideModules — function @experimental```(config: Pick<PackagedGameConfig, "sideModules">): voiddiff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.mdindex 4e7774220..22346145f 100644--- a/sdk/etc/minigame.api.md+++ b/sdk/etc/minigame.api.md@@ -1,7 +1,7 @@# API surface — esengine/minigame<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->-Symbols: 115 public · 166 beta · 2120 experimental · 3 deprecated+Symbols: 115 public · 166 beta · 2130 experimental · 3 deprecated## ANCHOR_AXES — const @experimental```@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProviderstatic prototype: GridLayoutProvider```+## GripFinger — interface @experimental+```+base: number | undefined+curl: number+spread: number | undefined+```++## GripHandStatus — interface @experimental+```+hand: number | null+joints: number+missing: string[]+pose: string+weight: number+```++## GripPose — const @experimental+```+ComponentDef<GripPoseData>+```++## GripPoseData — interface @experimental+```+blendSeconds: number+leftHand: string+leftPose: string+leftTrigger: number+leftWeight: number+rightHand: string+rightPose: string+rightTrigger: number+rightWeight: number+```++## GripPoseDef — interface @experimental+```+index: GripFinger+middle: GripFinger+pinky: GripFinger+ring: GripFinger+thumb: GripThumb+trigger: GripFinger | undefined+```++## GripThumb — interface @experimental+```+base: number | undefined+curl: number+opposition: number+spread: number | undefined+```+## Ground — type @experimental```(x: number, z: number) => { y: number; up: number } | null@@ -16029,6 +16081,21 @@ Plugin(preserveDrawingBuffer?: boolean): WebGLContextAttributes```+## gripPose — function @experimental+```+(name: string): GripPoseDef | undefined+```++## gripPoseNames — function @experimental+```+(): string[]+```++## gripPoseStatus — function @experimental+```+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null+```+## hasPlayableCta — function @experimental```(): boolean@@ -17164,6 +17231,11 @@ PrefabsPlugin(def: EffectDef): void```+## registerGripPose — function @experimental+```+(name: string, pose: GripPoseDef): void+```+## registerPackagedSideModules — function @experimental```(config: Pick<PackagedGameConfig, "sideModules">): voiddiff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.mdindex aeb2e70ae..e7321de4d 100644--- a/sdk/etc/wechat.api.md+++ b/sdk/etc/wechat.api.md@@ -1,7 +1,7 @@# API surface — esengine/wechat<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->-Symbols: 115 public · 166 beta · 2111 experimental · 3 deprecated+Symbols: 115 public · 166 beta · 2121 experimental · 3 deprecated## ANCHOR_AXES — const @experimental```@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProviderstatic prototype: GridLayoutProvider```+## GripFinger — interface @experimental+```+base: number | undefined+curl: number+spread: number | undefined+```++## GripHandStatus — interface @experimental+```+hand: number | null+joints: number+missing: string[]+pose: string+weight: number+```++## GripPose — const @experimental+```+ComponentDef<GripPoseData>+```++## GripPoseData — interface @experimental+```+blendSeconds: number+leftHand: string+leftPose: string+leftTrigger: number+leftWeight: number+rightHand: string+rightPose: string+rightTrigger: number+rightWeight: number+```++## GripPoseDef — interface @experimental+```+index: GripFinger+middle: GripFinger+pinky: GripFinger+ring: GripFinger+thumb: GripThumb+trigger: GripFinger | undefined+```++## GripThumb — interface @experimental+```+base: number | undefined+curl: number+opposition: number+spread: number | undefined+```+## Ground — type @experimental```(x: number, z: number) => { y: number; up: number } | null@@ -15755,6 +15807,21 @@ Plugin(preserveDrawingBuffer?: boolean): WebGLContextAttributes```+## gripPose — function @experimental+```+(name: string): GripPoseDef | undefined+```++## gripPoseNames — function @experimental+```+(): string[]+```++## gripPoseStatus — function @experimental+```+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null+```+## hasPlayableCta — function @experimental```(): boolean@@ -16890,6 +16957,11 @@ PrefabsPlugin(def: EffectDef): void```+## registerGripPose — function @experimental+```+(name: string, pose: GripPoseDef): void+```+## registerPackagedSideModules — function @experimental```(config: Pick<PackagedGameConfig, "sideModules">): voiddiff --git a/sdk/src/animation/AnimationPlugin.ts b/sdk/src/animation/AnimationPlugin.tsindex 4a716070e..b47468f3e 100644--- a/sdk/src/animation/AnimationPlugin.ts+++ b/sdk/src/animation/AnimationPlugin.ts@@ -29,6 +29,7 @@ import { installFogOfWar } from '../render/fogOfWar';import { installSwayPush } from '../render/swayPush';import { installFootsteps } from '../render/footsteps';import { SystemLabel } from '../ecs/systemLabels';+import { installGripPose } from './gripPose';export class AnimationPlugin implements Plugin {name = 'animation';@@ -107,6 +108,8 @@ export class AnimationPlugin implements Plugin {},{ name: 'SpriteAnimatorSystem' }), { runAfter: [SystemLabel.Tween, SystemLabel.Animator], runIf: playModeOnly });++ installGripPose(app);}cleanup(app?: App): void {diff --git a/sdk/src/animation/gripPose.ts b/sdk/src/animation/gripPose.tsnew file mode 100644index 000000000..dbc8ab301--- /dev/null+++ b/sdk/src/animation/gripPose.ts@@ -0,0 +1,494 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file gripPose.ts+ * @brief How a hand is held: a pistol grip, a hammer's handle, a cup, said once and fitting any hand.+ *+ * @details A pose stored as bone rotations belongs to the skeleton it was made on, since the bones of two rigs do not face the same way.+ * So a GripPose says only what is true of a hand: how far each finger bends, how far it fans out, how far the thumb comes across.+ * The axes are worked out from each rig's own hand, once, from where its joints stand: which way the palm faces, which way each+ * finger points. What runs each frame is then a few quaternions a joint, and nothing at all once a hand has settled on a rig no clip moves.+ *+ * It does not know what is held. Putting the hand on an object is the attachment's business, and fitting the fingers to its+ * surface a later step's.+ */+import type { App } from '../app/app';+import { defineComponent, getComponent, Children, Name, Transform, type ComponentDef, type TransformData } from '../ecs/component';+import { defineSystem, GetWorld, Schedule } from '../ecs/system';+import { Res, Time, type TimeData } from '../ecs/resource';+import type { World } from '../ecs/world';+import { resolveChildEntity } from '../ecs/childPath';+import { playModeOnly } from '../ecs/env';+import { SystemLabel } from '../ecs/systemLabels';+import { log } from '../util/logger';+import { q } from '../math/quat';+import type { Entity } from '../types';+import { leanQuat } from './quatMix';++/** How one finger is held. @experimental */+export interface GripFinger {+ /** How far it bends, 0 as the rig rests to 1 closed. */+ curl: number;+ /** How far its first joint bends, where that differs from the rest of it: a finger laid along a frame, or hooked. Defaults to `curl`. */+ base?: number;+ /** How far the fingers fan out from one another, -1 to 1. */+ spread?: number;+}++/** How the thumb is held. @experimental */+export interface GripThumb extends GripFinger {+ /** How far it comes across the palm, 0 beside the hand to 1 facing the fingers. */+ opposition: number;+}++/** How a whole hand is held. Nothing in it names a bone, so it fits any rig. @experimental */+export interface GripPoseDef {+ thumb: GripThumb;+ index: GripFinger;+ middle: GripFinger;+ ring: GripFinger;+ pinky: GripFinger;+ /** The index finger as `trigger` takes it to 1: pressing, where the finger above is resting. Absent, `trigger` does nothing. */+ trigger?: GripFinger;+}++const f = (curl: number, base?: number, spread?: number): GripFinger => ({ curl, ...(base !== undefined ? { base } : {}), ...(spread !== undefined ? { spread } : {}) });+const thumb = (curl: number, opposition: number, base?: number): GripThumb => ({ curl, opposition, ...(base !== undefined ? { base } : {}) });++const poses = new Map<string, GripPoseDef>(Object.entries({+ open: { thumb: thumb(0, 0), index: f(0), middle: f(0), ring: f(0), pinky: f(0) },+ relaxed: { thumb: thumb(0.15, 0.25), index: f(0.18, 0.15), middle: f(0.25, 0.2), ring: f(0.3, 0.25), pinky: f(0.35, 0.3) },+ fist: { thumb: thumb(0.7, 0.85), index: f(1), middle: f(1), ring: f(1), pinky: f(1) },+ pistol: {+ thumb: thumb(0.45, 0.8), index: f(0.1, 0.32), middle: f(0.78, 0.72), ring: f(0.82, 0.76), pinky: f(0.85, 0.8),+ trigger: f(0.55, 0.42),+ },+ support: { thumb: thumb(0.3, 0.65), index: f(0.4, 0.35), middle: f(0.45, 0.4), ring: f(0.48, 0.42), pinky: f(0.5, 0.45) },+ handle: { thumb: thumb(0.6, 0.85), index: f(0.78, 0.7), middle: f(0.82, 0.75), ring: f(0.85, 0.78), pinky: f(0.88, 0.8) },+ pinch: { thumb: thumb(0.5, 0.8), index: f(0.55, 0.5), middle: f(0.25, 0.2), ring: f(0.25, 0.2), pinky: f(0.25, 0.2) },+ cup: { thumb: thumb(0.25, 0.7), index: f(0.45, 0.25, 0.3), middle: f(0.45, 0.25, 0.3), ring: f(0.45, 0.25, 0.3), pinky: f(0.45, 0.25, 0.3) },+} satisfies Record<string, GripPoseDef>));++/** Adds a pose of the game's own, or replaces one by name. @experimental */+export function registerGripPose(name: string, pose: GripPoseDef): void {+ poses.set(name, pose);+}++/** The names a `GripPose` component can take. @experimental */+export function gripPoseNames(): string[] {+ return [...poses.keys()];+}++/** The pose of that name, or undefined. @experimental */+export function gripPose(name: string): GripPoseDef | undefined {+ return poses.get(name);+}++/** The fields of the `GripPose` component. @experimental */+export interface GripPoseData {+ /** The pose each hand takes; empty leaves that hand to whatever else moves it. */+ rightPose: string;+ leftPose: string;+ /** How much of it applies, over what a clip is doing with the fingers. */+ rightWeight: number;+ leftWeight: number;+ /** How far the index finger has gone from resting to pressing. */+ rightTrigger: number;+ leftTrigger: number;+ /** About how long a change of pose takes, in seconds. */+ blendSeconds: number;+ /** The hand joint, by path from this entity. Empty looks for it by name. */+ rightHand: string;+ leftHand: string;+}++/**+ * Holds a rig's hands in a named pose. Goes on the rig's root.+ *+ * @experimental+ */+export const GripPose: ComponentDef<GripPoseData> = /* @__PURE__ */ defineComponent<GripPoseData>('GripPose', {+ rightPose: '', leftPose: '', rightWeight: 1, leftWeight: 1, rightTrigger: 0, leftTrigger: 0, blendSeconds: 0.15, rightHand: '', leftHand: '',+}, {+ fields: {+ rightPose: { category: 'Right Hand', tooltip: 'open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone.' },+ rightWeight: { min: 0, max: 1, step: 0.01, category: 'Right Hand', tooltip: 'How much of the pose applies, over what a clip is doing with the fingers.' },+ rightTrigger: { min: 0, max: 1, step: 0.01, category: 'Right Hand', tooltip: 'How far the index finger has gone from resting to pressing, in a pose that has both.' },+ rightHand: { category: 'Right Hand', advanced: true, tooltip: 'The hand joint, by path from this entity. Empty looks for it by name.' },+ leftPose: { category: 'Left Hand', tooltip: 'open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone.' },+ leftWeight: { min: 0, max: 1, step: 0.01, category: 'Left Hand', tooltip: 'How much of the pose applies, over what a clip is doing with the fingers.' },+ leftTrigger: { min: 0, max: 1, step: 0.01, category: 'Left Hand', tooltip: 'How far the index finger has gone from resting to pressing, in a pose that has both.' },+ leftHand: { category: 'Left Hand', advanced: true, tooltip: 'The hand joint, by path from this entity. Empty looks for it by name.' },+ blendSeconds: { min: 0, step: 0.01, unit: 's', tooltip: 'About how long a change of pose takes.' },+ },+});++interface Quat { w: number; x: number; y: number; z: number }+interface Vec3 { x: number; y: number; z: number }++/** The fingers, in the order a hand's numbers are kept. */+const FINGERS = ['thumb', 'index', 'middle', 'ring', 'pinky'] as const;+type FingerName = (typeof FINGERS)[number];++const NAME_OF: Record<FingerName, RegExp> = {+ thumb: /thumb/i, index: /index|pointer/i, middle: /middle/i, ring: /(?<![a-z])ring|[a-z]ring(?=[\d_.:-]|$)/i, pinky: /pinky|pinkie|little/i,+};+/** A bone inside the palm: nothing a pose bends, and what the first joint of a finger may hang from. */+const INSIDE_PALM = /metacarpal|palm/i;++/** How far each joint of a finger bends when it is closed: first, second, third. In radians. */+const CLOSED = [Math.PI / 2, (100 * Math.PI) / 180, (70 * Math.PI) / 180];+/** The thumb's are far less: its root barely bends, and what it does there is opposition's. */+const THUMB_CLOSED = [(15 * Math.PI) / 180, (60 * Math.PI) / 180, (80 * Math.PI) / 180];+const SPREAD = (15 * Math.PI) / 180;+const FAN: Record<FingerName, number> = { thumb: 0, index: 1, middle: 0.15, ring: -0.6, pinky: -1 };++const add = (a: Vec3, b: Vec3): Vec3 => ({ x: a.x + b.x, y: a.y + b.y, z: a.z + b.z });+const sub = (a: Vec3, b: Vec3): Vec3 => ({ x: a.x - b.x, y: a.y - b.y, z: a.z - b.z });+const dot = (a: Vec3, b: Vec3): number => a.x * b.x + a.y * b.y + a.z * b.z;+const scale = (a: Vec3, k: number): Vec3 => ({ x: a.x * k, y: a.y * k, z: a.z * k });+const cross = (a: Vec3, b: Vec3): Vec3 => ({ x: a.y * b.z - a.z * b.y, y: a.z * b.x - a.x * b.z, z: a.x * b.y - a.y * b.x });+const norm = (a: Vec3): Vec3 => { const l = Math.hypot(a.x, a.y, a.z); return l < 1e-9 ? { x: 0, y: 0, z: 0 } : scale(a, 1 / l); };+const axisAngle = (axis: Vec3, radians: number): Quat => {+ const s = Math.sin(radians / 2);+ return { w: Math.cos(radians / 2), x: axis.x * s, y: axis.y * s, z: axis.z * s };+};++const nameOf = (world: World, e: Entity): string => (world.tryGet(e, Name) as { value?: string } | null)?.value ?? '';+const childrenOf = (world: World, e: Entity): Entity[] => (world.tryGet(e, Children) as { entities?: Entity[] } | null)?.entities ?? [];++/** One joint a hand moves: where it rests, and the turns a pose asks of it, in its own frame. */+interface Joint {+ entity: Entity;+ rest: Quat;+ curlAxis: Vec3;+ /** On a finger's first joint. */+ spreadAxis?: Vec3;+ /** On the thumb's first joint: the turn that faces it to the fingers at full opposition, as an axis and an angle. */+ opposeAxis?: Vec3;+ opposeAngle?: number;+ /** What this wrote last, to tell it from what a clip has written since. */+ written: Quat;+}++/** A finger's joints, knuckle first. Those it has no joint for are left out. */+type Finger = { name: FingerName; joints: Joint[]; fan: number };++/** Numbers a hand is held by, kept flat so two poses can be eased between: curl, base, spread for each finger, then the thumb's opposition. */+const NUMBERS = FINGERS.length * 3 + 1;++function numbersOf(pose: GripPoseDef | null, out: Float64Array): void {+ let i = 0;+ for (const name of FINGERS) {+ const finger = pose?.[name];+ out[i++] = clamp01(finger?.curl ?? 0);+ out[i++] = clamp01(finger?.base ?? finger?.curl ?? 0);+ out[i++] = Math.max(-1, Math.min(1, finger?.spread ?? 0));+ }+ out[i] = clamp01(pose?.thumb.opposition ?? 0);+}++const clamp01 = (v: number): number => (v > 0 ? (v < 1 ? v : 1) : 0);++/** What is known of one hand on one rig. */+interface HandState {+ hand: Entity;+ fingers: Finger[];+ /** Which fingers it could not find, to say so once. */+ missing: FingerName[];+ /** The numbers now, and how much of them is applied now: both eased. */+ current: Float64Array;+ wanted: Float64Array;+ weight: number;+ /** How fast each number is moving, and the weight after them. */+ speed: Float64Array;+ /** Whether `current` and `weight` have arrived and everything written still stands. */+ settled: boolean;+ pose: string;+ trigger: number;+}++/** A finger's bones in order, from the hand: those named for it, knuckle first, those in the palm passed through. */+function fingerChain(world: World, hand: Entity, name: FingerName): Entity[] {+ const pattern = NAME_OF[name];+ const search = (at: Entity, depth: number): Entity | null => {+ if (depth > 3) return null;+ for (const child of childrenOf(world, at)) {+ if (pattern.test(nameOf(world, child)) && !INSIDE_PALM.test(nameOf(world, child))) return child;+ }+ for (const child of childrenOf(world, at)) {+ const named = nameOf(world, child);+ // Through the palm's own bones, and not into something held.+ if (INSIDE_PALM.test(named)) { const found = search(child, depth + 1); if (found !== null) return found; }+ }+ return null;+ };+ const first = search(hand, 0);+ if (first === null) return [];+ const chain = [first];+ for (let at = first, guard = 0; guard < 6; guard++) {+ const next = childrenOf(world, at).find((c) => pattern.test(nameOf(world, c))) ?? null;+ if (next === null) break;+ chain.push(next);+ at = next;+ }+ return chain;+}++/** Where a joint stands from the hand, and its turn from it. */+function fromHand(world: World, chain: Entity[]): { at: Vec3[]; turn: Quat[] } {+ let position: Vec3 = { x: 0, y: 0, z: 0 };+ let rotation: Quat = { w: 1, x: 0, y: 0, z: 0 };+ const at: Vec3[] = [], turn: Quat[] = [];+ for (const joint of chain) {+ const local = world.get(joint, Transform) as TransformData;+ position = add(position, q.rotate(rotation, local.position));+ rotation = q.normalize(q.mul(rotation, local.rotation));+ at.push(position);+ turn.push(rotation);+ }+ return { at, turn };+}++/** Which way the palm faces, and toward the thumb across it, in the hand's own frame. Null with too few fingers to tell. */+function palmOf(world: World, side: 'right' | 'left', chains: Partial<Record<FingerName, Entity[]>>): { palm: Vec3; across: Vec3; forward: Vec3 } | null {+ const knuckle = (name: FingerName): Vec3 | null => {+ const chain = chains[name];+ return chain && chain.length ? fromHand(world, chain).at[0]! : null;+ };+ const outer = knuckle('index') ?? knuckle('middle');+ const inner = knuckle('pinky') ?? knuckle('ring') ?? knuckle('middle');+ const middle = knuckle('middle') ?? outer;+ if (!outer || !inner || !middle) return null;+ const across = norm(sub(outer, inner));+ if (Math.hypot(across.x, across.y, across.z) < 0.5) return null;+ // The way the fingers themselves point, and square to the knuckles: a wrist is rarely straight behind its middle finger.+ const long = chains.middle && chains.middle.length > 1 ? chains.middle : chains.index && chains.index.length > 1 ? chains.index : null;+ const placed = long ? fromHand(world, long).at : null;+ const along = placed ? sub(placed[1]!, placed[0]!) : middle;+ const forward = norm(sub(along, scale(across, dot(along, across))));+ // A right hand with its fingers ahead and its thumb to the left has its palm down: which is the cross product, one way round for each hand.+ const palm = norm(scale(cross(forward, across), side === 'right' ? -1 : 1));+ return Math.hypot(palm.x, palm.y, palm.z) < 0.5 ? null : { palm, across, forward };+}++/** The hand's joints, with the turns a pose asks of each. They are worked out from the rig as it stands now, once. */+function buildHand(world: World, hand: Entity, side: 'right' | 'left'): HandState | null {+ const chains: Partial<Record<FingerName, Entity[]>> = {};+ for (const name of FINGERS) chains[name] = fingerChain(world, hand, name);+ const shape = palmOf(world, side, chains);+ if (!shape) return null;+ const { palm, across, forward } = shape;+ const fingers: Finger[] = [];+ const missing: FingerName[] = [];+ for (const name of FINGERS) {+ const chain = chains[name]!;+ if (chain.length === 0) { missing.push(name); continue; }+ const { at, turn } = fromHand(world, chain);+ // Its first joint to its second is the way it points; a finger of one joint points as the joint does from the hand.+ const pointing = norm(chain.length > 1 ? sub(at[1]!, at[0]!) : at[0]!);+ // The thumb's way to the fingers is a direction and not a turn about some axis: where it starts from differs from rig to rig, and where it has to get to does not.+ const facing = name === 'thumb' ? norm(add(add(scale(forward, 0.85), scale(palm, 0.45)), scale(across, 0.25))) : pointing;+ const oppose = q.rotationTo(pointing, facing);+ // A closing thumb lies over the first two fingers, its tip going on toward the little finger.+ const towards = name === 'thumb' ? norm(add(add(scale(palm, 0.15), scale(across, -1)), scale(forward, -0.1))) : palm;+ // Worked out where it faces once opposed, and taken back to where it rests, since that turn comes first.+ const bend = q.rotate(q.conjugate(oppose), norm(cross(facing, towards)));+ // The last bone of a chain that goes on past three is the tip's end, which nothing bends.+ const moved = chain.slice(0, 3);+ const joints: Joint[] = moved.map((entity, i) => {+ const toJoint = q.conjugate(turn[i]!);+ const rest = q.normalize((world.get(entity, Transform) as TransformData).rotation);+ const joint: Joint = { entity, rest: { ...rest }, curlAxis: norm(q.rotate(toJoint, bend)), written: { ...rest } };+ if (i === 0) {+ joint.spreadAxis = norm(q.rotate(toJoint, palm));+ // Toward the index finger for a positive spread.+ if (dot(cross(palm, pointing), across) < 0) joint.spreadAxis = scale(joint.spreadAxis, -1);+ if (name === 'thumb') {+ const half = Math.min(1, Math.abs(oppose.w));+ const axis = norm({ x: oppose.x, y: oppose.y, z: oppose.z });+ joint.opposeAxis = norm(q.rotate(toJoint, oppose.w < 0 ? scale(axis, -1) : axis));+ joint.opposeAngle = 2 * Math.acos(half);+ }+ }+ return joint;+ });+ fingers.push({ name, joints, fan: FAN[name] });+ }+ return { hand, fingers, missing, current: new Float64Array(NUMBERS), wanted: new Float64Array(NUMBERS), weight: 0, speed: new Float64Array(NUMBERS + 1), settled: false, pose: '', trigger: 0 };+}++const NAMED_HAND = /hand|wrist/i;+const NOT_A_HAND = /thumb|index|middle|ring|pinky|pinkie|little|finger|metacarpal|palm/i;+const IS_RIGHT = /right|(^|[^a-z])r($|[^a-z])/i;+const IS_LEFT = /left|(^|[^a-z])l($|[^a-z])/i;++/** The joint named for that hand under `root`, nearest first. */+function findHand(world: World, root: Entity, side: 'right' | 'left'): Entity | null {+ const wanted = side === 'right' ? IS_RIGHT : IS_LEFT;+ const queue: Entity[] = [root];+ for (let i = 0; i < queue.length && i < 512; i++) {+ const at = queue[i]!;+ const name = nameOf(world, at);+ if (at !== root && NAMED_HAND.test(name) && !NOT_A_HAND.test(name) && wanted.test(name) && fingerChain(world, at, 'middle').length + fingerChain(world, at, 'index').length > 0) return at;+ queue.push(...childrenOf(world, at));+ }+ return null;+}++/** What one rig's hands are doing. */+interface RigState { right: HandState | null; left: HandState | null; rightHand: string; leftHand: string; tried: { right: boolean; left: boolean } }++const SAME = 1e-6;+const same = (a: Quat, b: Quat): boolean => Math.abs(a.w - b.w) < SAME && Math.abs(a.x - b.x) < SAME && Math.abs(a.y - b.y) < SAME && Math.abs(a.z - b.z) < SAME;++/**+ * A number moved toward where it is wanted, critically damped: it starts from still and comes to rest without passing it. Going a fixed share of the+ * gap each frame would be at its fastest on the first, and a finger with far to go visibly jumps.+ */+function ease(value: number, wanted: number, state: HandState, slot: number, blend: number, dt: number): number {+ if (blend <= 0) { state.speed[slot] = 0; return wanted; }+ const omega = 6 / blend;+ const gap = value - wanted;+ const carry = (state.speed[slot]! + omega * gap) * dt;+ const fade = Math.exp(-omega * dt);+ const speed = (state.speed[slot]! - omega * carry) * fade;+ const next = wanted + (gap + carry) * fade;+ if (Math.abs(next - wanted) < 1e-3 && Math.abs(speed) < 1e-2) { state.speed[slot] = 0; return wanted; }+ state.speed[slot] = speed;+ return next;+}++/** Holds one hand a frame: eases its numbers on, and turns each joint to them over what is there. */+function holdHand(world: World, state: HandState, poseName: string, weight: number, trigger: number, blend: number, dt: number, moved: boolean): void {+ const pose = poseName ? poses.get(poseName) ?? null : null;+ if (poseName !== state.pose) {+ if (poseName && !pose) log.warn('animation', `there is no grip pose called "${poseName}": the hand is left alone`);+ state.pose = poseName;+ // A new pose eases on from where the hand is; a hand let go keeps its numbers and fades by its weight.+ if (pose) numbersOf(pose, state.wanted);+ state.settled = false;+ }+ const wantedWeight = pose ? clamp01(weight) : 0;+ const press = pose?.trigger ? clamp01(trigger) : 0;+ if (press !== state.trigger) { state.trigger = press; state.settled = false; }+ if (wantedWeight !== state.weight) state.settled = false;+ if (state.settled && !moved) return;++ let arrived = true;+ for (let i = 0; i < NUMBERS; i++) {+ const next = ease(state.current[i]!, state.wanted[i]!, state, i, blend, dt);+ state.current[i] = next;+ if (next !== state.wanted[i]) arrived = false;+ }+ state.weight = ease(state.weight, wantedWeight, state, NUMBERS, blend, dt);+ if (state.weight !== wantedWeight) arrived = false;++ const opposition = state.current[NUMBERS - 1]!;+ for (const finger of state.fingers) {+ const at = FINGERS.indexOf(finger.name) * 3;+ let curl = state.current[at]!, base = state.current[at + 1]!;+ const spread = state.current[at + 2]!;+ if (finger.name === 'index' && press > 0 && pose?.trigger) {+ curl += (clamp01(pose.trigger.curl) - curl) * press;+ base += (clamp01(pose.trigger.base ?? pose.trigger.curl) - base) * press;+ }+ for (let i = 0; i < finger.joints.length; i++) {+ const joint = finger.joints[i]!;+ let turn = axisAngle(joint.curlAxis, (i === 0 ? base : curl) * (finger.name === 'thumb' ? THUMB_CLOSED : CLOSED)[i]!);+ if (joint.spreadAxis && spread !== 0 && finger.fan !== 0) turn = q.mul(axisAngle(joint.spreadAxis, spread * finger.fan * SPREAD), turn);+ if (joint.opposeAxis && opposition > 0) turn = q.mul(axisAngle(joint.opposeAxis, opposition * joint.opposeAngle!), turn);+ const target = q.normalize(q.mul(joint.rest, turn));+ const local = world.get(joint.entity, Transform) as TransformData;+ // Moved since this wrote it, a clip is speaking and the pose goes over that; otherwise over how the rig rests.+ const from = same(local.rotation, joint.written) ? joint.rest : local.rotation;+ const out = { w: from.w, x: from.x, y: from.y, z: from.z };+ if (state.weight >= 1) { out.w = target.w; out.x = target.x; out.y = target.y; out.z = target.z; }+ else leanQuat(out, target, state.weight);+ const written = q.normalize(out);+ world.update(joint.entity, Transform, (t: TransformData) => { t.rotation = { w: written.w, x: written.x, y: written.y, z: written.z }; });+ joint.written = { ...((world.get(joint.entity, Transform) as TransformData).rotation) };+ }+ }+ state.settled = arrived;+}++function letGo(world: World, state: HandState | null): void {+ if (!state) return;+ for (const finger of state.fingers) {+ for (const joint of finger.joints) {+ if (world.valid(joint.entity) && world.has(joint.entity, Transform)) {+ world.update(joint.entity, Transform, (t: TransformData) => { t.rotation = { w: joint.rest.w, x: joint.rest.x, y: joint.rest.y, z: joint.rest.z }; });+ }+ }+ }+}++/** What a rig's hand was found to have. @experimental */+export interface GripHandStatus {+ /** The joint it took for the hand, or null where none could be found. */+ hand: Entity | null;+ /** How many joints a pose moves on it. */+ joints: number;+ /** The fingers it has no bones for: a pose simply does without them. */+ missing: string[];+ /** The pose it is in. */+ pose: string;+ /** How much of it applies now. */+ weight: number;+}++const rigs = new WeakMap<App, Map<Entity, RigState>>();++/** What each hand of a rig with a `GripPose` was found to have, once the system has seen it. @experimental */+export function gripPoseStatus(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus } | null {+ const state = rigs.get(app)?.get(rig);+ if (!state) return null;+ const of = (h: HandState | null): GripHandStatus => ({+ hand: h?.hand ?? null, joints: h ? h.fingers.reduce((n, finger) => n + finger.joints.length, 0) : 0, missing: h ? [...h.missing] : [...FINGERS], pose: h?.pose ?? '', weight: h?.weight ?? 0,+ });+ return { right: of(state.right), left: of(state.left) };+}++/** Holds every rig's hands. Runs after the animator, since what a clip says of the fingers is what a pose goes over. */+export function installGripPose(app: App): void {+ const known = new Map<Entity, RigState>();+ rigs.set(app, known);+ // By name: what moves the bones is the animator's or the timeline's, which this does not need to carry.+ const hasClipOn = (world: World, rig: Entity): boolean => ['Animator', 'TimelinePlayer'].some((name) => { const def = getComponent(name); return !!def && world.has(rig, def); });+ // system-access: the hand's joints are found by walking the rig, so which entities it writes is known only then, and only their Transforms.+ app.addSystemToSchedule(Schedule.Update, defineSystem([Res(Time), GetWorld()], (time: TimeData, world: World) => {+ for (const [rig, state] of known) {+ if (world.valid(rig) && world.has(rig, GripPose)) continue;+ letGo(world, state.right);+ letGo(world, state.left);+ known.delete(rig);+ }+ for (const rig of world.getEntitiesWithComponents([GripPose])) {+ const data = world.get(rig, GripPose) as GripPoseData;+ if (!data.rightPose && !data.leftPose && !known.has(rig)) continue;+ let state = known.get(rig);+ if (!state || state.rightHand !== data.rightHand || state.leftHand !== data.leftHand) {+ if (state) { letGo(world, state.right); letGo(world, state.left); }+ state = { right: null, left: null, rightHand: data.rightHand, leftHand: data.leftHand, tried: { right: false, left: false } };+ known.set(rig, state);+ }+ const moved = hasClipOn(world, rig);+ for (const side of ['right', 'left'] as const) {+ const poseName = side === 'right' ? data.rightPose : data.leftPose;+ if (!state[side] && poseName && !state.tried[side]) {+ state.tried[side] = true;+ const path = side === 'right' ? data.rightHand : data.leftHand;+ const hand = path ? resolveChildEntity(world, rig, path) : findHand(world, rig, side);+ state[side] = hand === null ? null : buildHand(world, hand, side);+ if (!state[side]) log.warn('animation', `a GripPose found no ${side} hand with fingers on "${nameOf(world, rig)}"${path ? ` at "${path}"` : ''}: name its joint in ${side}Hand`);+ else if (state[side]!.missing.length) log.warn('animation', `the ${side} hand of "${nameOf(world, rig)}" has no bones for its ${state[side]!.missing.join(', ')}: a pose does without them`);+ }+ const hand = state[side];+ if (hand) {+ holdHand(world, hand, poseName, side === 'right' ? data.rightWeight : data.leftWeight, side === 'right' ? data.rightTrigger : data.leftTrigger, data.blendSeconds, time.delta, moved);+ }+ }+ }+ }, { name: 'GripPoseSystem' }), { runAfter: [SystemLabel.Animator], runIf: playModeOnly });+}diff --git a/sdk/src/animation/index.ts b/sdk/src/animation/index.tsindex e52d8c676..46ac457a4 100644--- a/sdk/src/animation/index.ts+++ b/sdk/src/animation/index.ts@@ -81,6 +81,10 @@ export {type AnimatorEvalResult,type AnimatorPathEvalResult,} from './Animator';+export {+ GripPose, registerGripPose, gripPose, gripPoseNames, gripPoseStatus,+ type GripPoseData, type GripPoseDef, type GripFinger, type GripThumb, type GripHandStatus,+} from './gripPose';export {emptyAnimatorController,diff --git a/sdk/src/core-content.ts b/sdk/src/core-content.tsindex 0e845299a..20d7b5d42 100644--- a/sdk/src/core-content.ts+++ b/sdk/src/core-content.ts@@ -650,6 +650,16 @@ export {TweenSequence,type Completable,type TweenFactory,+ GripPose,+ registerGripPose,+ gripPose,+ gripPoseNames,+ gripPoseStatus,+ type GripPoseData,+ type GripPoseDef,+ type GripFinger,+ type GripThumb,+ type GripHandStatus,} from './animation';// =============================================================================diff --git a/sdk/tests/grip-pose-wasm.test.ts b/sdk/tests/grip-pose-wasm.test.tsnew file mode 100644index 000000000..a8518883c--- /dev/null+++ b/sdk/tests/grip-pose-wasm.test.ts@@ -0,0 +1,446 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * A hand held in a named pose, on the real engine. What makes a pose worth having is that it fits any rig, so the claims are about where the+ * fingertips end up in the world: the same for two rigs whose bones face different ways, and mirrored for the other hand.+ */+import { afterEach, beforeAll, describe, expect, it, vi } from 'vitest';+import { App } from '../src/app/app';+import { Animator } from '../src/animation';+import { AnimationPlugin } from '../src/animation/AnimationPlugin';+import { GripPose, gripPoseNames, gripPoseStatus, registerGripPose, type GripPoseData } from '../src/animation/gripPose';+import { Name, Transform, type TransformData } from '../src/ecs/component';+import { setEditorMode, setPlayMode } from '../src/ecs/env';+import { q } from '../src/math/quat';+import type { Entity } from '../src/types';+import type { ESEngineModule } from '../src/wasm';+import { HAS_WASM, loadWasmModule } from './helpers/loadWasm';++interface V { x: number; y: number; z: number }+interface Q { w: number; x: number; y: number; z: number }+const IDENTITY: Q = { w: 1, x: 0, y: 0, z: 0 };++let core: ESEngineModule;+const apps: App[] = [];+afterEach(() => { while (apps.length) apps.pop()!.quit?.(); setEditorMode(false); setPlayMode(false); vi.restoreAllMocks(); });++function scene() {+ core.setActiveContext(null);+ const app = App.new(); apps.push(app); app.connectCpp(new core.Registry(), core, { ownedRegistry: true });+ setPlayMode(true);+ app.addPlugin(new AnimationPlugin());+ const tick = async (n = 1): Promise<void> => { for (let i = 0; i < n; i++) { await app.tick(1 / 60); await Promise.resolve(); } };+ return { app, world: app.world, tick };+}+type World = App['world'];++/** Where a finger's joints stand, seen from the hand with the fingers ahead along +Y, the palm toward -Z and, on a right hand, the thumb toward -X. */+const LAYOUT: Record<string, { from: V; along: V; lengths: number[] }> = {+ thumb: { from: { x: -0.03, y: 0.02, z: -0.01 }, along: { x: -0.7, y: 0.7, z: 0 }, lengths: [0.035, 0.03, 0.025] },+ index: { from: { x: -0.025, y: 0.09, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.04, 0.025, 0.02] },+ middle: { from: { x: -0.005, y: 0.095, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.045, 0.03, 0.02] },+ ring: { from: { x: 0.015, y: 0.09, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.04, 0.028, 0.02] },+ pinky: { from: { x: 0.033, y: 0.08, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.03, 0.02, 0.018] },+};++interface RigOptions {+ side?: 'right' | 'left';+ /** The turn each bone is given from its parent. The picture of the hand stays the same, since the positions are written to match. */+ frame?: (depth: number) => Q;+ names?: (finger: string, i: number) => string;+ handName?: string;+ without?: string[];+ /** Bones in the palm between the hand and each finger. */+ metacarpal?: boolean;+ /** A turn over the whole rig. */+ root?: Q;+}++/** A hand built so its world shape does not depend on how its bones face. Answers the bones by finger, tips included. */+function buildRig(world: World, options: RigOptions = {}) {+ const mirror = options.side === 'left' ? -1 : 1;+ const frame = options.frame ?? (() => IDENTITY);+ const names = options.names ?? ((finger, i) => `mixamorig:${options.side === 'left' ? 'Left' : 'Right'}Hand${finger[0]!.toUpperCase()}${finger.slice(1)}${i + 1}`);+ const spawn = (name: string, parent: Entity | null, position: V, rotation: Q): Entity => {+ const e = world.spawn(name);+ world.insert(e, Name, { value: name });+ world.insert(e, Transform, { position, rotation });+ if (parent !== null) world.setParent(e, parent);+ return e;+ };+ const root = spawn('Rig', null, { x: 0, y: 0, z: 0 }, options.root ?? IDENTITY);+ const armature = spawn('Armature', root, { x: 0, y: 0, z: 0 }, IDENTITY);+ const handTurn = frame(0);+ const hand = spawn(options.handName ?? `mixamorig:${options.side === 'left' ? 'Left' : 'Right'}Hand`, armature, { x: 0, y: 0, z: 0 }, handTurn);+ const bones: Record<string, Entity[]> = {};+ for (const [finger, layout] of Object.entries(LAYOUT)) {+ if (options.without?.includes(finger)) continue;+ let parent = hand, parentTurn = handTurn, at: V = { x: 0, y: 0, z: 0 };+ const chain: Entity[] = [];+ const placeChild = (name: string, worldAt: V, depth: number): Entity => {+ const turn = frame(depth);+ const local = q.rotate(q.conjugate(parentTurn), { x: worldAt.x - at.x, y: worldAt.y - at.y, z: worldAt.z - at.z });+ const e = spawn(name, parent, local, q.normalize(q.mul(q.conjugate(parentTurn), turn)));+ parent = e; parentTurn = turn; at = worldAt;+ return e;+ };+ const start: V = { x: layout.from.x * mirror, y: layout.from.y, z: layout.from.z };+ if (options.metacarpal && finger !== 'thumb') placeChild(`${finger}_metacarpal`, { x: start.x * 0.5, y: start.y * 0.4, z: start.z }, 1);+ let p = start;+ chain.push(placeChild(names(finger, 0), p, 2));+ const along = { x: layout.along.x * mirror, y: layout.along.y, z: layout.along.z };+ const unit = Math.hypot(along.x, along.y, along.z);+ layout.lengths.forEach((l, i) => {+ p = { x: p.x + along.x / unit * l, y: p.y + along.y / unit * l, z: p.z + along.z / unit * l };+ chain.push(placeChild(names(finger, i + 1), p, 3 + i));+ });+ bones[finger] = chain;+ }+ return { root, hand, bones };+}++/** A bone's place in the world, from the locals: the world's own record is last frame's. */+function placeOf(world: World, e: Entity, stop: Entity): V {+ const links: Entity[] = [];+ for (let at = e; ; at = (world.get(at, getParent(world)) as { entity: Entity }).entity) { links.unshift(at); if (at === stop) break; }+ let p: V = { x: 0, y: 0, z: 0 }, r: Q = IDENTITY;+ for (const link of links) {+ const t = world.get(link, Transform) as TransformData;+ p = (() => { const v = q.rotate(r, t.position); return { x: p.x + v.x, y: p.y + v.y, z: p.z + v.z }; })();+ r = q.normalize(q.mul(r, t.rotation));+ }+ return p;+}+import { Parent } from '../src/ecs/component';+const getParent = (_world: World) => Parent;++const tips = (world: World, rig: ReturnType<typeof buildRig>): Record<string, V> => Object.fromEntries(Object.entries(rig.bones).map(([finger, chain]) => [finger, placeOf(world, chain[chain.length - 1]!, rig.hand)]));+const hold = (world: World, rig: { root: Entity }, data: Partial<GripPoseData>): void => { world.insert(rig.root, GripPose, { blendSeconds: 0, ...data }); };+const close = (a: V, b: V, digits = 3): void => { expect(a.x).toBeCloseTo(b.x, digits); expect(a.y).toBeCloseTo(b.y, digits); expect(a.z).toBeCloseTo(b.z, digits); };+const dist = (a: V, b: V): number => Math.hypot(a.x - b.x, a.y - b.y, a.z - b.z);++/** A turn that has nothing to do with how a hand bends: bones in a real rig face wherever the exporter left them. */+const scrambled = (depth: number): Q => q.normalize({ w: 0.3 + depth * 0.17, x: Math.sin(depth * 1.9 + 0.4), y: Math.cos(depth * 2.7), z: Math.sin(depth * 0.8 - 1) });++describe.skipIf(!HAS_WASM)('a grip pose', () => {+ beforeAll(async () => { if (HAS_WASM) core = await loadWasmModule(); });++ it('closes the fingers toward the palm, not away from it', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig);+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ const shut = tips(s.world, rig);+ for (const f of ['index', 'middle', 'ring', 'pinky']) {+ // A closed finger rolls back toward the palm, so its tip ends near it, on the palm's side.+ expect(shut[f]!.z, f).toBeLessThan(-0.005);+ expect(shut[f]!.y, f).toBeLessThan(open[f]!.y - 0.03);+ }+ });++ it('brings the thumb across the palm toward the fingers', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig);+ hold(s.world, rig, { rightPose: 'handle' });+ await s.tick(2);+ const held = tips(s.world, rig);+ expect(held.thumb!.z).toBeLessThan(-0.01);+ expect(dist(held.thumb!, held.index!)).toBeLessThan(dist(open.thumb!, open.index!));+ });++ it('does not turn the thumb back over the wrist, however closed', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ // Ahead of where the fingers start from the hand, and on the palm's side: a thumb that has gone past the wrist is behind both.+ const tip = tips(s.world, rig).thumb!;+ expect(tip.y).toBeGreaterThan(0);+ expect(tip.z).toBeLessThan(0);+ // And over the fingers, not out past the little one on the far side.+ const knuckles = LAYOUT.pinky!.from.x;+ expect(tip.x).toBeLessThan(knuckles);+ expect(tip.x).toBeGreaterThan(LAYOUT.index!.from.x - 0.02);+ });++ it('turns the thumb toward the fingers by its opposition, whatever else it does', async () => {+ const none = { curl: 0 };+ registerGripPose('thumb-beside', { thumb: { curl: 0, opposition: 0 }, index: none, middle: none, ring: none, pinky: none });+ registerGripPose('thumb-across', { thumb: { curl: 0, opposition: 1 }, index: none, middle: none, ring: none, pinky: none });+ const s = scene();+ const beside = buildRig(s.world);+ hold(s.world, beside, { rightPose: 'thumb-beside' });+ const t = scene();+ const across = buildRig(t.world);+ hold(t.world, across, { rightPose: 'thumb-across' });+ await s.tick(2);+ await t.tick(2);+ const a = tips(s.world, beside).thumb!, b = tips(t.world, across).thumb!;+ expect(b.z).toBeLessThan(a.z - 0.01);+ expect(b.y).toBeGreaterThan(a.y);+ expect(tips(t.world, across).index!.z).toBeCloseTo(tips(s.world, beside).index!.z, 5);+ });++ it('makes the same hand on rigs whose bones face different ways', async () => {+ for (const pose of ['fist', 'pistol', 'cup', 'pinch']) {+ const s = scene();+ const a = buildRig(s.world);+ hold(s.world, a, { rightPose: pose });+ const t = scene();+ const b = buildRig(t.world, { frame: scrambled });+ hold(t.world, b, { rightPose: pose });+ await s.tick(2);+ await t.tick(2);+ const ta = tips(s.world, a), tb = tips(t.world, b);+ for (const f of Object.keys(ta)) close(ta[f]!, tb[f]!);+ }+ });++ it('mirrors for the other hand', async () => {+ const s = scene();+ const right = buildRig(s.world);+ hold(s.world, right, { rightPose: 'pistol' });+ const t = scene();+ const left = buildRig(t.world, { side: 'left' });+ hold(t.world, left, { leftPose: 'pistol' });+ await s.tick(2);+ await t.tick(2);+ const r = tips(s.world, right), l = tips(t.world, left);+ for (const f of Object.keys(r)) close({ x: -r[f]!.x, y: r[f]!.y, z: r[f]!.z }, l[f]!);+ });++ it('holds both hands of one rig, each its own', async () => {+ const s = scene();+ const root = s.world.spawn('Rig');+ s.world.insert(root, Transform, {});+ const r = buildRig(s.world), l = buildRig(s.world, { side: 'left' });+ s.world.setParent(r.root, root); s.world.setParent(l.root, root);+ const openLeft = tips(s.world, l);+ hold(s.world, { root }, { rightPose: 'fist', leftPose: 'open' });+ await s.tick(2);+ expect(tips(s.world, r).index!.z).toBeLessThan(-0.015);+ close(tips(s.world, l).index!, openLeft.index!);+ });++ it('does not depend on the turn of the rig it is on', async () => {+ const s = scene();+ const a = buildRig(s.world);+ hold(s.world, a, { rightPose: 'handle' });+ const t = scene();+ const b = buildRig(t.world, { root: q.normalize({ w: 0.5, x: 0.5, y: -0.5, z: 0.5 }) });+ hold(t.world, b, { rightPose: 'handle' });+ await s.tick(2);+ await t.tick(2);+ const ta = tips(s.world, a), tb = tips(t.world, b);+ for (const f of Object.keys(ta)) close(ta[f]!, tb[f]!);+ });++ it('finds the hand and fingers under other rigs\' names, through the bones in the palm', async () => {+ const s = scene();+ const rig = buildRig(s.world, { handName: 'hand_r', metacarpal: true, names: (finger, i) => `${finger}_0${i + 1}_r` });+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ const status = gripPoseStatus(s.app, rig.root)!;+ expect(status.right.hand).toBe(rig.hand);+ expect(status.right.joints).toBe(15);+ expect(status.right.missing).toEqual([]);+ expect(tips(s.world, rig).middle!.z).toBeLessThan(-0.015);+ });++ it('does not take a finger for the hand', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(1);+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBe(rig.hand);+ });++ it('does without the fingers a rig has no bones for, and says so', async () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ const s = scene();+ const rig = buildRig(s.world, { without: ['ring', 'pinky'] });+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ const status = gripPoseStatus(s.app, rig.root)!;+ expect(status.right.missing).toEqual(['ring', 'pinky']);+ expect(status.right.joints).toBe(9);+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.015);+ warn.mockRestore();+ });++ it('takes the hand from a path where the name would not find it', async () => {+ const s = scene();+ const rig = buildRig(s.world, { handName: 'bone_17' });+ hold(s.world, rig, { rightPose: 'fist', rightHand: 'Armature/bone_17' });+ await s.tick(2);+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBe(rig.hand);+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.015);+ });++ it('leaves a hand it cannot find alone, and goes on', async () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ const s = scene();+ const rig = buildRig(s.world, { handName: 'bone_17' });+ const before = tips(s.world, rig);+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBeNull();+ close(tips(s.world, rig).index!, before.index!);+ warn.mockRestore();+ });++ it('stays put however long it is held: nothing piles up', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'pistol', rightWeight: 0.5 });+ await s.tick(2);+ const once = tips(s.world, rig);+ await s.tick(120);+ const later = tips(s.world, rig);+ for (const f of Object.keys(once)) close(once[f]!, later[f]!, 5);+ });++ it('goes part of the way by its weight', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig).index!;+ hold(s.world, rig, { rightPose: 'fist', rightWeight: 1 });+ await s.tick(2);+ const shut = tips(s.world, rig).index!;+ const t = scene();+ const half = buildRig(t.world);+ hold(t.world, half, { rightPose: 'fist', rightWeight: 0.5 });+ await t.tick(2);+ const between = tips(t.world, half).index!;+ expect(dist(between, open)).toBeGreaterThan(0.01);+ expect(dist(between, shut)).toBeGreaterThan(0.01);+ });++ it('eases from one pose to the next, with no jump', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'open', blendSeconds: 0.2 });+ await s.tick(30);+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightPose = 'fist'; });+ let last = tips(s.world, rig).index!, biggest = 0, first = -1;+ for (let i = 0; i < 40; i++) {+ await s.tick(1);+ const now = tips(s.world, rig).index!;+ if (first < 0) first = dist(now, last);+ biggest = Math.max(biggest, dist(now, last));+ last = now;+ }+ const t = scene();+ const direct = buildRig(t.world);+ const open = tips(t.world, direct).index!;+ hold(t.world, direct, { rightPose: 'fist' });+ await t.tick(2);+ const whole = dist(tips(t.world, direct).index!, open);+ expect(dist(last, tips(t.world, direct).index!)).toBeLessThan(0.002);+ // It starts from still: a share of the gap each frame would be at its fastest on the first.+ expect(first).toBeLessThan(whole * 0.25);+ expect(biggest).toBeLessThan(whole * 0.4);+ });++ it('presses the trigger with the index finger alone, in a pose that has one', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'pistol', rightTrigger: 0 });+ await s.tick(2);+ const resting = tips(s.world, rig);+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightTrigger = 1; });+ await s.tick(2);+ const pressed = tips(s.world, rig);+ expect(dist(resting.index!, pressed.index!)).toBeGreaterThan(0.01);+ close(resting.middle!, pressed.middle!, 5);+ close(resting.thumb!, pressed.thumb!, 5);++ const t = scene();+ const plain = buildRig(t.world);+ hold(t.world, plain, { rightPose: 'fist', rightTrigger: 0 });+ await t.tick(2);+ const shut = tips(t.world, plain).index!;+ t.world.update(plain.root, GripPose, (d: GripPoseData) => { d.rightTrigger = 1; });+ await t.tick(2);+ close(shut, tips(t.world, plain).index!, 5);+ });++ it('goes over what a clip says of the fingers, by its weight', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ s.world.insert(rig.root, Animator, {});+ const finger = rig.bones.index![0]!;+ const clip = q.normalize({ w: 0.9, x: 0.3, y: 0.1, z: 0 });+ const speak = (): void => { s.world.update(finger, Transform, (d: TransformData) => { d.rotation = clip; }); };+ hold(s.world, rig, { rightPose: 'fist', rightWeight: 0 });+ speak();+ await s.tick(2);+ const left = (s.world.get(finger, Transform) as TransformData).rotation;+ expect(left.x).toBeCloseTo(clip.x, 4);+ expect(left.w).toBeCloseTo(clip.w, 4);+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightWeight = 1; });+ speak();+ await s.tick(2);+ const over = (s.world.get(finger, Transform) as TransformData).rotation;+ expect(Math.abs(over.x - clip.x) + Math.abs(over.w - clip.w)).toBeGreaterThan(0.05);+ });++ it('fades away when its pose is taken off, and leaves the hand as it found it when it goes', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig);+ hold(s.world, rig, { rightPose: 'fist', blendSeconds: 0.05 });+ await s.tick(30);+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightPose = ''; });+ await s.tick(30);+ close(tips(s.world, rig).index!, open.index!);++ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ expect(dist(tips(s.world, rig).index!, open.index!)).toBeGreaterThan(0.02);+ s.world.remove(rig.root, GripPose);+ await s.tick(2);+ close(tips(s.world, rig).index!, open.index!, 4);+ });++ it('writes nothing once a hand has settled on a rig no clip moves', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ hold(s.world, rig, { rightPose: 'pistol' });+ await s.tick(3);+ const spy = vi.spyOn(s.world, 'update');+ await s.tick(10);+ expect(spy.mock.calls.filter((c) => c[1] === Transform).length).toBe(0);+ });++ it('takes a pose of the game\'s own, and leaves a name it does not know alone', async () => {+ registerGripPose('claw', { thumb: { curl: 1, opposition: 1 }, index: { curl: 1, base: 0 }, middle: { curl: 1, base: 0 }, ring: { curl: 1, base: 0 }, pinky: { curl: 1, base: 0 } });+ expect(gripPoseNames()).toContain('claw');+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig);+ hold(s.world, rig, { rightPose: 'claw' });+ await s.tick(2);+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.01);++ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ const t = scene();+ const other = buildRig(t.world);+ hold(t.world, other, { rightPose: 'nothing like this' });+ await t.tick(2);+ close(tips(t.world, other).index!, open.index!);+ warn.mockRestore();+ });++ it('does nothing while not playing', async () => {+ const s = scene();+ const rig = buildRig(s.world);+ const open = tips(s.world, rig);+ setEditorMode(true); setPlayMode(false);+ hold(s.world, rig, { rightPose: 'fist' });+ await s.tick(2);+ close(tips(s.world, rig).index!, open.index!, 5);+ });+});diff --git a/tools/check-corpus-authoring.mjs b/tools/check-corpus-authoring.mjsindex 2a623041b..84f67eef1 100644--- a/tools/check-corpus-authoring.mjs+++ b/tools/check-corpus-authoring.mjs@@ -125,6 +125,7 @@ const DISPOSITION = {Planet: ['owed', 'authored by a game today (star-neighbors, a round planet from its own generator); an example that registers a planet provider is owed'],MeshInstances: ['derived', 'the foliage and world generation systems write it, one entity a kind and part of a cell; its places are data a system computes, not something a scene authors'],ParticleCollider: ['owed', 'no example has a GPU emitter bouncing off a shape yet; the particle-gpu scenes are where the first one goes'],+ GripPose: ['owed', 'no example holds a tool in a named pose yet; a character with something in its hand is where the first one goes'],VoxelTerrain: ['owed', 'no example digs in a volume yet; a small mine is where the first one goes'],RoadDecal: ['owed', 'no example lays a road over a terrain yet; wilds is where the first one goes'],PlantProxy: ['derived', 'the plant interaction makes it for a plant near a contact, and takes it away again; a scene never holds one'],diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 6fb6810d8..5de2699fa 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -4232,6 +4232,8 @@ export const NOTED = {{"note": "Particles and plain sprites in a Canvas paint by their place in its tree."},"fix(text): stopping lets go of the labels the engine holds, so the next Play cannot draw them on other entities":{"note": "Labels no longer come back as black boxes after a change between Plays."},+ "feat(animation): a rig's hands held in a named pose that fits any skeleton":+ {"note": "Hands that hold things."},"feat(sky): clouds that keep time with the game, follow the weather's wind, and have a second layer":{"note": "Clouds that keep time with the game."},"feat(voxel): ground that is a volume, carved while the game runs, with collision that follows and saves by chunk":