Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(animation): a rig's hands held in a named pose that fits any skeleton

yhhdd4644cefaee
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d71d8b0bd..6b3a97b65 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- **Hands that hold things.** A `GripPose` holds a rig's hands in a named pose: `pistol`, `handle`, `cup`, `fist` and others, or one a game registers. A pose says how far each finger bends and nothing about bones, so it fits any rig with fingers; a change of pose eases over, a trigger moves the index finger alone, and it goes over what a clip does with the fingers by its weight.
- **Clouds that keep time with the game.** Where a Sky Atmosphere's clouds are is now what the wind has done so far, so a wind that changes carries them on instead of moving the whole sky; a Weather's wind reaches them, their shapes change as they go, and a paused game has a still sky. They can have a second layer with its own height, grain and wind, and `skyClockState()` keeps their place in a save. Self-shadowing reaches through a deep cloud, edges wear into billows, and the lit edge near the sun is narrower.
- **Ground that is a volume.** The optional `esengine/voxel` module adds `VoxelTerrain`, beside the height-field Terrain: it holds caves and overhangs, and can be carved, filled and painted with a sphere, a box or a capsule while the game runs. Only the chunks an edit reached are made again, with their collision, and a save holds only what was changed, chunk by chunk. A sleeping body is now woken when the collision under it changes, on a Terrain as well, where before it stayed in the air.
- **What stands on the ground can blend into it.** The `builtin:ground-blend` material fades a rock or a foundation into whatever is behind it, on both backends, with nothing baked, and can bring a second layer up from the ground or paint one with vertex colours. `RoadDecal` lays a mesh along a Spline over a terrain with soft edges. The `snowCover` post effect and a Snow weather lay snow on whatever faces the sky, terrain and meshes alike.
diff --git a/desktop b/desktop
index 5d08f5a40..bf5e07137 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 5d08f5a4011d8aa11e54f80898585ade7421faba
+Subproject commit bf5e0713710930cf93d79ddf8c72ab01d473c245
diff --git a/docs/astro/astro.config.mjs b/docs/astro/astro.config.mjs
index 2e08db611..8a027583f 100644
--- a/docs/astro/astro.config.mjs
+++ b/docs/astro/astro.config.mjs
@@ -241,6 +241,7 @@ export default defineConfig({
{ label: 'Overview', translations: { 'zh-CN': '总览' }, slug: 'animation/overview' },
{ label: 'Timeline', translations: { 'zh-CN': '时间轴' }, slug: 'animation/timeline' },
{ label: 'Spring bones', translations: { 'zh-CN': '弹簧骨骼' }, slug: 'animation/spring-bones' },
+ { label: 'Grip Pose', translations: { 'zh-CN': '握持姿势' }, slug: 'animation/grip-pose' },
{ label: '2D Rigs', translations: { 'zh-CN': '2D 骨骼' }, slug: 'animation/rig-2d' },
{ label: 'Spine Animation', translations: { 'zh-CN': 'Spine 动画' }, slug: 'animation/spine' },
{ label: 'DragonBones Animation', translations: { 'zh-CN': 'DragonBones 动画' }, slug: 'animation/dragonbones' },
diff --git a/docs/astro/src/content/docs/animation/grip-pose.mdx b/docs/astro/src/content/docs/animation/grip-pose.mdx
new file mode 100644
index 000000000..26f94d8a5
--- /dev/null
+++ b/docs/astro/src/content/docs/animation/grip-pose.mdx
@@ -0,0 +1,80 @@
+---
+title: Grip Pose
+description: Hold a character's hands in a named pose, a pistol grip, a handle, a cup, that fits any rig with fingers.
+---
+
+A `GripPose` holds a rig's hands the way you name: `pistol`, `handle`, `cup`. A pose says only how far each finger bends, how far they fan out and how far the thumb comes across, and nothing in it names a bone, so one pose fits any rig. Which way those bends go is worked out from each rig's own hand, the first time it is held.
+
+It shapes the hand. Putting the hand on an object is up to whatever attaches it, usually an entity under the hand joint and two-bone IK for the arm.
+
+## Hold a hand
+
+Put a `GripPose` on the rig's root and name a pose for either hand:
+
+```ts
+import { GripPose } from 'esengine';
+
+world.insert(hero, GripPose, { rightPose: 'pistol', leftPose: 'support' });
+
+// Squeeze: the index finger goes from resting to pressing, and nothing else moves.
+world.update(hero, GripPose, (grip) => { grip.rightTrigger = 1; });
+```
+
+| Field | What it does |
+|---|---|
+| `rightPose`, `leftPose` | The pose each hand takes. Empty fades that hand out and leaves it to whatever else moves it. |
+| `rightWeight`, `leftWeight` | How much of the pose applies, over what a clip is doing with the fingers. |
+| `rightTrigger`, `leftTrigger` | How far the index finger has gone from resting to pressing, in a pose that has both. |
+| `blendSeconds` | About how long a change of pose takes. It starts from still and comes to rest, with no jump. |
+| `rightHand`, `leftHand` | The hand joint, by path from this entity. Empty looks for it by name. |
+
+It runs after the animator, so a walk cycle goes on moving the arm while the pose holds the fingers, and works with no animator at all. It runs while the game plays, not while a scene is laid out.
+
+## The poses
+
+| Name | For |
+|---|---|
+| `open` | A flat hand, as the rig rests. |
+| `relaxed` | A hand hanging at rest. |
+| `fist` | Closed, the thumb over the fingers. |
+| `pistol` | A pistol's grip, the index finger along the frame. `trigger` presses it. |
+| `support` | A hand under a rifle's foregrip. |
+| `handle` | A hammer, a sword, a tool's handle. |
+| `pinch` | Thumb and index finger together. |
+| `cup` | Round a mug or a ball, fingers apart. |
+
+To add your own, or replace one by name:
+
+```ts
+import { registerGripPose } from 'esengine';
+
+registerGripPose('claw', {
+ thumb: { curl: 0.6, opposition: 0.7 },
+ index: { curl: 0.9, base: 0.2 },
+ middle: { curl: 0.9, base: 0.2 },
+ ring: { curl: 0.9, base: 0.2 },
+ pinky: { curl: 0.9, base: 0.2 },
+});
+```
+
+| Number | What it says |
+|---|---|
+| `curl` | How far the finger bends, 0 as the rig rests to 1 closed. |
+| `base` | How far its first joint bends, where that differs: a finger laid along a frame, or hooked. Defaults to `curl`. |
+| `spread` | How far the fingers fan out, -1 to 1. |
+| `opposition` | The thumb only: how far it comes across the palm, 0 beside the hand to 1 facing the fingers. |
+| `trigger` | On the pose: the index finger as `rightTrigger` takes it to 1. |
+
+## On your rig
+
+The hand is found by name: a joint called something like `RightHand` or `hand_r`, with fingers named `thumb`, `index`, `middle`, `ring`, and `pinky` or `little` under it. Bones in the palm (`index_metacarpal_r`) are passed through. Where a rig's names say nothing of the kind, name the joint in `rightHand`. If its fingers are not named either, it cannot be held.
+
+A rig with fewer fingers does without the rest, and says which in the log. `gripPoseStatus(app, entity)` answers what was found on each hand.
+
+## Limits
+
+- **It does not touch what is held.** A pose is a shape, not a fit: fingers can sink into a wide grip or stop short of a narrow one. Use `curl` for the object, or a pose of its own.
+- **A hand's numbers are worked out once,** from how the rig stands when it is first held. A rig whose hand is mid-animation by then has its axes taken from that. In practice they are close.
+- **Rigs with a negative scale on an arm are not handled.** A mirrored bone turns the wrong way.
+- **Checked on a real rig of one kind.** The poses were tuned on a Mixamo-named character, and the same hand built with bones facing other ways and under other names gives the same fingertips, but no second real rig has been looked at.
+- **Not measured on a device.** It is a few quaternions for each of up to 30 joints a frame, and nothing once settled on a rig no clip moves.
diff --git a/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx
new file mode 100644
index 000000000..c0eb5f57a
--- /dev/null
+++ b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx
@@ -0,0 +1,80 @@
+---
+title: 握持姿势
+description: 让角色的手保持一个命名姿势,比如手枪握、手柄握、捧杯,适用于任何带手指骨骼的角色。
+---
+
+`GripPose` 让角色的手保持你指定的姿势:`pistol`、`handle`、`cup`。姿势只描述每根手指弯多少、张开多少、拇指对掌多少,不涉及任何骨骼名字,所以同一个姿势可以用在任何骨架上。弯曲的方向在第一次握持时,根据每个角色自己的手算出来。
+
+它只负责手型。把手放到物体上由挂点负责,通常是手部关节下的子实体,加上手臂的 two-bone IK。
+
+## 使用
+
+把 `GripPose` 放在角色的根实体上,给左右手各指定一个姿势:
+
+```ts
+import { GripPose } from 'esengine';
+
+world.insert(hero, GripPose, { rightPose: 'pistol', leftPose: 'support' });
+
+// 扣扳机:食指从搭放变成扣下,其他手指不动。
+world.update(hero, GripPose, (grip) => { grip.rightTrigger = 1; });
+```
+
+| 字段 | 作用 |
+|---|---|
+| `rightPose`、`leftPose` | 每只手的姿势。留空时这只手淡出,交还给其他驱动它的东西。 |
+| `rightWeight`、`leftWeight` | 姿势应用多少,叠加在动画片段对手指的控制之上。 |
+| `rightTrigger`、`leftTrigger` | 食指从搭放到扣下的程度,要姿势同时定义了这两个状态才有效。 |
+| `blendSeconds` | 切换姿势大约花多久。从静止起步,平稳停下,没有跳变。 |
+| `rightHand`、`leftHand` | 手部关节,写从这个实体出发的路径。留空时按名字查找。 |
+
+它在动画系统之后运行,所以走路动画继续带动手臂,姿势负责手指;没有动画系统时也能用。它只在游戏运行时生效,布置场景时不生效。
+
+## 内置姿势
+
+| 名字 | 用途 |
+|---|---|
+| `open` | 张开的手,即骨架的静止状态。 |
+| `relaxed` | 自然垂放的手。 |
+| `fist` | 握拳,拇指压在手指上。 |
+| `pistol` | 手枪握,食指沿枪身伸直。`trigger` 让它扣下。 |
+| `support` | 托在步枪前握下方的手。 |
+| `handle` | 锤子、剑、工具的手柄。 |
+| `pinch` | 拇指和食指捏合。 |
+| `cup` | 捧杯或握球,手指张开。 |
+
+要添加自己的姿势,或者按名字替换内置姿势:
+
+```ts
+import { registerGripPose } from 'esengine';
+
+registerGripPose('claw', {
+ thumb: { curl: 0.6, opposition: 0.7 },
+ index: { curl: 0.9, base: 0.2 },
+ middle: { curl: 0.9, base: 0.2 },
+ ring: { curl: 0.9, base: 0.2 },
+ pinky: { curl: 0.9, base: 0.2 },
+});
+```
+
+| 数值 | 含义 |
+|---|---|
+| `curl` | 手指弯多少,0 是骨架静止状态,1 是完全握紧。 |
+| `base` | 第一个关节弯多少,用于和其余关节不同的情况,比如手指沿枪身伸直或者勾起。默认等于 `curl`。 |
+| `spread` | 手指张开多少,-1 到 1。 |
+| `opposition` | 仅拇指:拇指对掌多少,0 贴在手边,1 正对手指。 |
+| `trigger` | 写在姿势上:`rightTrigger` 到 1 时食指的状态。 |
+
+## 适配你的骨架
+
+手部按名字查找:名字类似 `RightHand` 或 `hand_r` 的关节,下面的手指名字包含 `thumb`、`index`、`middle`、`ring`、`pinky` 或 `little`。手掌内的骨骼(`index_metacarpal_r`)会被跳过。如果骨架名字完全对不上,就在 `rightHand` 里写明关节路径;如果手指也没有这样的名字,就无法使用。
+
+手指较少的骨架会跳过缺失的手指,并在日志里说明。`gripPoseStatus(app, entity)` 可以查询每只手找到了什么。
+
+## 限制
+
+- **不接触被握的物体。** 姿势只是手型,不做贴合:握把太粗时手指可能陷进去,太细时可能碰不到。可以调 `curl`,或者给这个物体单独做一个姿势。
+- **手的参数只计算一次,** 基于第一次握持时骨架的状态。如果那时手正处在动画中,轴就按当时的状态算,实际差别不大。
+- **不支持手臂带负缩放的骨架。** 镜像骨骼会转错方向。
+- **只在一种真实骨架上检查过。** 姿势是在一个 Mixamo 命名的角色上调的。用骨骼朝向不同、名字不同的同一只手测试,指尖位置一致,但还没有看过第二套真实骨架。
+- **没有在真机上测过。** 每帧最多 30 个关节,每个几次四元数运算;在没有动画片段驱动的骨架上,手稳定后就不再有开销。
diff --git a/docs/astro/src/data/apiStability.generated.json b/docs/astro/src/data/apiStability.generated.json
index 301ec372b..b1a1c27cc 100644
--- a/docs/astro/src/data/apiStability.generated.json
+++ b/docs/astro/src/data/apiStability.generated.json
@@ -3,10 +3,10 @@
"counts": {
"public": 115,
"beta": 178,
- "experimental": 2968,
+ "experimental": 2978,
"deprecated": 3
},
- "total": 3261,
+ "total": 3271,
"subsystems": [
{
"id": "mobile-controls",
diff --git a/docs/astro/src/data/componentDocs.ts b/docs/astro/src/data/componentDocs.ts
index 0c0d7c57f..b519b38ed 100644
--- a/docs/astro/src/data/componentDocs.ts
+++ b/docs/astro/src/data/componentDocs.ts
@@ -207,6 +207,7 @@ export const COMPONENT_DOCS: Record<string, ComponentDoc> = {
// ── Animation ──────────────────────────────────────────────────────────────
Animator: { category: 'animation', summary: 'Runs an animator controller — states, transitions and blend trees — over the entity.', doc: 'animation/overview', anchor: 'animator-state-machine', anchorZh: '动画状态机' },
AnimatorRootMotion: { category: 'animation', summary: 'Publishes how far the active animation asks the character to move, for gameplay to hand to a character controller — a request, never a result.', doc: 'animation/overview', anchor: 'root-motion', anchorZh: '根运动' },
+ GripPose: { category: 'animation', summary: 'Holds a rig\'s hands in a named pose, a pistol grip, a handle, a cup, that fits any rig with fingers.', doc: 'animation/grip-pose' },
SpringBone: { category: 'animation', summary: 'Makes every bone below this root a spring: the chain trails the animated pose by inertia, falls with gravity and is pulled back toward it (hair, tails, cords).', doc: 'animation/spring-bones', anchor: 'the-component', anchorZh: '组件' },
SpringBoneCollider: { category: 'animation', summary: 'A sphere or capsule that the spring-bone chains naming it are kept out of.', doc: 'animation/spring-bones', anchor: 'the-component', anchorZh: '组件' },
SpriteAnimator: { category: 'animation', summary: 'Plays a flipbook clip by swapping the sprite frame, with speed, looping and frame events.', doc: 'animation/overview', anchor: 'sprite-animation', anchorZh: '精灵动画' },
diff --git a/docs/astro/src/data/components.generated.json b/docs/astro/src/data/components.generated.json
index 2cd9543c8..5422d34e5 100644
--- a/docs/astro/src/data/components.generated.json
+++ b/docs/astro/src/data/components.generated.json
@@ -2740,6 +2740,80 @@
"rippleTexture"
]
},
+ {
+ "name": "GripPose",
+ "source": "ts",
+ "fields": [
+ {
+ "key": "rightPose",
+ "type": "string",
+ "default": "empty",
+ "doc": "open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone."
+ },
+ {
+ "key": "leftPose",
+ "type": "string",
+ "default": "empty",
+ "doc": "open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone."
+ },
+ {
+ "key": "rightWeight",
+ "type": "number",
+ "default": "1",
+ "doc": "How much of the pose applies, over what a clip is doing with the fingers.",
+ "range": "0–1, step 0.01"
+ },
+ {
+ "key": "leftWeight",
+ "type": "number",
+ "default": "1",
+ "doc": "How much of the pose applies, over what a clip is doing with the fingers.",
+ "range": "0–1, step 0.01"
+ },
+ {
+ "key": "rightTrigger",
+ "type": "number",
+ "default": "0",
+ "doc": "How far the index finger has gone from resting to pressing, in a pose that has both.",
+ "range": "0–1, step 0.01"
+ },
+ {
+ "key": "leftTrigger",
+ "type": "number",
+ "default": "0",
+ "doc": "How far the index finger has gone from resting to pressing, in a pose that has both.",
+ "range": "0–1, step 0.01"
+ },
+ {
+ "key": "blendSeconds",
+ "type": "number",
+ "default": "0.15",
+ "doc": "About how long a change of pose takes.",
+ "range": "≥ 0, step 0.01"
+ },
+ {
+ "key": "rightHand",
+ "type": "string",
+ "default": "empty",
+ "doc": "The hand joint, by path from this entity. Empty looks for it by name.",
+ "advanced": true
+ },
+ {
+ "key": "leftHand",
+ "type": "string",
+ "default": "empty",
+ "doc": "The hand joint, by path from this entity. Empty looks for it by name.",
+ "advanced": true
+ }
+ ],
+ "animatable": [
+ "rightWeight",
+ "leftWeight",
+ "rightTrigger",
+ "leftTrigger",
+ "blendSeconds"
+ ]
+ },
{
"name": "GroupOpacity",
"source": "cpp",
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 27f63ac00..23750b6f2 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2099 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2109 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -5029,6 +5029,58 @@ static new (opts: GridLayoutOptions): GridLayoutProvider
static prototype: GridLayoutProvider
```
+## GripFinger — interface @experimental
+```
+base: number | undefined
+curl: number
+spread: number | undefined
+```
+
+## GripHandStatus — interface @experimental
+```
+hand: number | null
+joints: number
+missing: string[]
+pose: string
+weight: number
+```
+
+## GripPose — const @experimental
+```
+ComponentDef<GripPoseData>
+```
+
+## GripPoseData — interface @experimental
+```
+blendSeconds: number
+leftHand: string
+leftPose: string
+leftTrigger: number
+leftWeight: number
+rightHand: string
+rightPose: string
+rightTrigger: number
+rightWeight: number
+```
+
+## GripPoseDef — interface @experimental
+```
+index: GripFinger
+middle: GripFinger
+pinky: GripFinger
+ring: GripFinger
+thumb: GripThumb
+trigger: GripFinger | undefined
+```
+
+## GripThumb — interface @experimental
+```
+base: number | undefined
+curl: number
+opposition: number
+spread: number | undefined
+```
+
## Ground — type @experimental
```
(x: number, z: number) => { y: number; up: number } | null
@@ -15727,6 +15779,21 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripPose — function @experimental
+```
+(name: string): GripPoseDef | undefined
+```
+
+## gripPoseNames — function @experimental
+```
+(): string[]
+```
+
+## gripPoseStatus — function @experimental
+```
+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
@@ -16852,6 +16919,11 @@ PrefabsPlugin
(def: EffectDef): void
```
+## registerGripPose — function @experimental
+```
+(name: string, pose: GripPoseDef): void
+```
+
## registerPackagedSideModules — function @experimental
```
(config: Pick<PackagedGameConfig, "sideModules">): void
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 41a76da29..c43254dad 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.native
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 171 beta · 2118 experimental · 3 deprecated
+Symbols: 115 public · 171 beta · 2128 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -5037,6 +5037,58 @@ static new (opts: GridLayoutOptions): GridLayoutProvider
static prototype: GridLayoutProvider
```
+## GripFinger — interface @experimental
+```
+base: number | undefined
+curl: number
+spread: number | undefined
+```
+
+## GripHandStatus — interface @experimental
+```
+hand: number | null
+joints: number
+missing: string[]
+pose: string
+weight: number
+```
+
+## GripPose — const @experimental
+```
+ComponentDef<GripPoseData>
+```
+
+## GripPoseData — interface @experimental
+```
+blendSeconds: number
+leftHand: string
+leftPose: string
+leftTrigger: number
+leftWeight: number
+rightHand: string
+rightPose: string
+rightTrigger: number
+rightWeight: number
+```
+
+## GripPoseDef — interface @experimental
+```
+index: GripFinger
+middle: GripFinger
+pinky: GripFinger
+ring: GripFinger
+thumb: GripThumb
+trigger: GripFinger | undefined
+```
+
+## GripThumb — interface @experimental
+```
+base: number | undefined
+curl: number
+opposition: number
+spread: number | undefined
+```
+
## Ground — type @experimental
```
(x: number, z: number) => { y: number; up: number } | null
@@ -15983,6 +16035,21 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripPose — function @experimental
+```
+(name: string): GripPoseDef | undefined
+```
+
+## gripPoseNames — function @experimental
+```
+(): string[]
+```
+
+## gripPoseStatus — function @experimental
+```
+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
@@ -17118,6 +17185,11 @@ PrefabsPlugin
(def: EffectDef): void
```
+## registerGripPose — function @experimental
+```
+(name: string, pose: GripPoseDef): void
+```
+
## registerPackagedSideModules — function @experimental
```
(config: Pick<PackagedGameConfig, "sideModules">): void
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index 2bd9a4687..a7893554a 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.node
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 168 beta · 2098 experimental · 3 deprecated
+Symbols: 115 public · 168 beta · 2108 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProvider
static prototype: GridLayoutProvider
```
+## GripFinger — interface @experimental
+```
+base: number | undefined
+curl: number
+spread: number | undefined
+```
+
+## GripHandStatus — interface @experimental
+```
+hand: number | null
+joints: number
+missing: string[]
+pose: string
+weight: number
+```
+
+## GripPose — const @experimental
+```
+ComponentDef<GripPoseData>
+```
+
+## GripPoseData — interface @experimental
+```
+blendSeconds: number
+leftHand: string
+leftPose: string
+leftTrigger: number
+leftWeight: number
+rightHand: string
+rightPose: string
+rightTrigger: number
+rightWeight: number
+```
+
+## GripPoseDef — interface @experimental
+```
+index: GripFinger
+middle: GripFinger
+pinky: GripFinger
+ring: GripFinger
+thumb: GripThumb
+trigger: GripFinger | undefined
+```
+
+## GripThumb — interface @experimental
+```
+base: number | undefined
+curl: number
+opposition: number
+spread: number | undefined
+```
+
## Ground — type @experimental
```
(x: number, z: number) => { y: number; up: number } | null
@@ -15722,6 +15774,21 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripPose — function @experimental
+```
+(name: string): GripPoseDef | undefined
+```
+
+## gripPoseNames — function @experimental
+```
+(): string[]
+```
+
+## gripPoseStatus — function @experimental
+```
+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
@@ -16857,6 +16924,11 @@ PrefabsPlugin
(def: EffectDef): void
```
+## registerGripPose — function @experimental
+```
+(name: string, pose: GripPoseDef): void
+```
+
## registerPackagedSideModules — function @experimental
```
(config: Pick<PackagedGameConfig, "sideModules">): void
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index 4e7774220..22346145f 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/minigame
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2120 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2130 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProvider
static prototype: GridLayoutProvider
```
+## GripFinger — interface @experimental
+```
+base: number | undefined
+curl: number
+spread: number | undefined
+```
+
+## GripHandStatus — interface @experimental
+```
+hand: number | null
+joints: number
+missing: string[]
+pose: string
+weight: number
+```
+
+## GripPose — const @experimental
+```
+ComponentDef<GripPoseData>
+```
+
+## GripPoseData — interface @experimental
+```
+blendSeconds: number
+leftHand: string
+leftPose: string
+leftTrigger: number
+leftWeight: number
+rightHand: string
+rightPose: string
+rightTrigger: number
+rightWeight: number
+```
+
+## GripPoseDef — interface @experimental
+```
+index: GripFinger
+middle: GripFinger
+pinky: GripFinger
+ring: GripFinger
+thumb: GripThumb
+trigger: GripFinger | undefined
+```
+
+## GripThumb — interface @experimental
+```
+base: number | undefined
+curl: number
+opposition: number
+spread: number | undefined
+```
+
## Ground — type @experimental
```
(x: number, z: number) => { y: number; up: number } | null
@@ -16029,6 +16081,21 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripPose — function @experimental
+```
+(name: string): GripPoseDef | undefined
+```
+
+## gripPoseNames — function @experimental
+```
+(): string[]
+```
+
+## gripPoseStatus — function @experimental
+```
+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
@@ -17164,6 +17231,11 @@ PrefabsPlugin
(def: EffectDef): void
```
+## registerGripPose — function @experimental
+```
+(name: string, pose: GripPoseDef): void
+```
+
## registerPackagedSideModules — function @experimental
```
(config: Pick<PackagedGameConfig, "sideModules">): void
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index aeb2e70ae..e7321de4d 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/wechat
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2111 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2121 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5024,6 +5024,58 @@ static new (opts: GridLayoutOptions): GridLayoutProvider
static prototype: GridLayoutProvider
```
+## GripFinger — interface @experimental
+```
+base: number | undefined
+curl: number
+spread: number | undefined
+```
+
+## GripHandStatus — interface @experimental
+```
+hand: number | null
+joints: number
+missing: string[]
+pose: string
+weight: number
+```
+
+## GripPose — const @experimental
+```
+ComponentDef<GripPoseData>
+```
+
+## GripPoseData — interface @experimental
+```
+blendSeconds: number
+leftHand: string
+leftPose: string
+leftTrigger: number
+leftWeight: number
+rightHand: string
+rightPose: string
+rightTrigger: number
+rightWeight: number
+```
+
+## GripPoseDef — interface @experimental
+```
+index: GripFinger
+middle: GripFinger
+pinky: GripFinger
+ring: GripFinger
+thumb: GripThumb
+trigger: GripFinger | undefined
+```
+
+## GripThumb — interface @experimental
+```
+base: number | undefined
+curl: number
+opposition: number
+spread: number | undefined
+```
+
## Ground — type @experimental
```
(x: number, z: number) => { y: number; up: number } | null
@@ -15755,6 +15807,21 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripPose — function @experimental
+```
+(name: string): GripPoseDef | undefined
+```
+
+## gripPoseNames — function @experimental
+```
+(): string[]
+```
+
+## gripPoseStatus — function @experimental
+```
+(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
@@ -16890,6 +16957,11 @@ PrefabsPlugin
(def: EffectDef): void
```
+## registerGripPose — function @experimental
+```
+(name: string, pose: GripPoseDef): void
+```
+
## registerPackagedSideModules — function @experimental
```
(config: Pick<PackagedGameConfig, "sideModules">): void
diff --git a/sdk/src/animation/AnimationPlugin.ts b/sdk/src/animation/AnimationPlugin.ts
index 4a716070e..b47468f3e 100644
--- a/sdk/src/animation/AnimationPlugin.ts
+++ b/sdk/src/animation/AnimationPlugin.ts
@@ -29,6 +29,7 @@ import { installFogOfWar } from '../render/fogOfWar';
import { installSwayPush } from '../render/swayPush';
import { installFootsteps } from '../render/footsteps';
import { SystemLabel } from '../ecs/systemLabels';
+import { installGripPose } from './gripPose';
export class AnimationPlugin implements Plugin {
name = 'animation';
@@ -107,6 +108,8 @@ export class AnimationPlugin implements Plugin {
},
{ name: 'SpriteAnimatorSystem' }
), { runAfter: [SystemLabel.Tween, SystemLabel.Animator], runIf: playModeOnly });
+
+ installGripPose(app);
}
cleanup(app?: App): void {
diff --git a/sdk/src/animation/gripPose.ts b/sdk/src/animation/gripPose.ts
new file mode 100644
index 000000000..dbc8ab301
--- /dev/null
+++ b/sdk/src/animation/gripPose.ts
@@ -0,0 +1,494 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file gripPose.ts
+ * @brief How a hand is held: a pistol grip, a hammer's handle, a cup, said once and fitting any hand.
+ *
+ * @details A pose stored as bone rotations belongs to the skeleton it was made on, since the bones of two rigs do not face the same way.
+ * So a GripPose says only what is true of a hand: how far each finger bends, how far it fans out, how far the thumb comes across.
+ * The axes are worked out from each rig's own hand, once, from where its joints stand: which way the palm faces, which way each
+ * finger points. What runs each frame is then a few quaternions a joint, and nothing at all once a hand has settled on a rig no clip moves.
+ *
+ * It does not know what is held. Putting the hand on an object is the attachment's business, and fitting the fingers to its
+ * surface a later step's.
+ */
+import type { App } from '../app/app';
+import { defineComponent, getComponent, Children, Name, Transform, type ComponentDef, type TransformData } from '../ecs/component';
+import { defineSystem, GetWorld, Schedule } from '../ecs/system';
+import { Res, Time, type TimeData } from '../ecs/resource';
+import type { World } from '../ecs/world';
+import { resolveChildEntity } from '../ecs/childPath';
+import { playModeOnly } from '../ecs/env';
+import { SystemLabel } from '../ecs/systemLabels';
+import { log } from '../util/logger';
+import { q } from '../math/quat';
+import type { Entity } from '../types';
+import { leanQuat } from './quatMix';
+
+/** How one finger is held. @experimental */
+export interface GripFinger {
+ /** How far it bends, 0 as the rig rests to 1 closed. */
+ curl: number;
+ /** How far its first joint bends, where that differs from the rest of it: a finger laid along a frame, or hooked. Defaults to `curl`. */
+ base?: number;
+ /** How far the fingers fan out from one another, -1 to 1. */
+ spread?: number;
+}
+
+/** How the thumb is held. @experimental */
+export interface GripThumb extends GripFinger {
+ /** How far it comes across the palm, 0 beside the hand to 1 facing the fingers. */
+ opposition: number;
+}
+
+/** How a whole hand is held. Nothing in it names a bone, so it fits any rig. @experimental */
+export interface GripPoseDef {
+ thumb: GripThumb;
+ index: GripFinger;
+ middle: GripFinger;
+ ring: GripFinger;
+ pinky: GripFinger;
+ /** The index finger as `trigger` takes it to 1: pressing, where the finger above is resting. Absent, `trigger` does nothing. */
+ trigger?: GripFinger;
+}
+
+const f = (curl: number, base?: number, spread?: number): GripFinger => ({ curl, ...(base !== undefined ? { base } : {}), ...(spread !== undefined ? { spread } : {}) });
+const thumb = (curl: number, opposition: number, base?: number): GripThumb => ({ curl, opposition, ...(base !== undefined ? { base } : {}) });
+
+const poses = new Map<string, GripPoseDef>(Object.entries({
+ open: { thumb: thumb(0, 0), index: f(0), middle: f(0), ring: f(0), pinky: f(0) },
+ relaxed: { thumb: thumb(0.15, 0.25), index: f(0.18, 0.15), middle: f(0.25, 0.2), ring: f(0.3, 0.25), pinky: f(0.35, 0.3) },
+ fist: { thumb: thumb(0.7, 0.85), index: f(1), middle: f(1), ring: f(1), pinky: f(1) },
+ pistol: {
+ thumb: thumb(0.45, 0.8), index: f(0.1, 0.32), middle: f(0.78, 0.72), ring: f(0.82, 0.76), pinky: f(0.85, 0.8),
+ trigger: f(0.55, 0.42),
+ },
+ support: { thumb: thumb(0.3, 0.65), index: f(0.4, 0.35), middle: f(0.45, 0.4), ring: f(0.48, 0.42), pinky: f(0.5, 0.45) },
+ handle: { thumb: thumb(0.6, 0.85), index: f(0.78, 0.7), middle: f(0.82, 0.75), ring: f(0.85, 0.78), pinky: f(0.88, 0.8) },
+ pinch: { thumb: thumb(0.5, 0.8), index: f(0.55, 0.5), middle: f(0.25, 0.2), ring: f(0.25, 0.2), pinky: f(0.25, 0.2) },
+ cup: { thumb: thumb(0.25, 0.7), index: f(0.45, 0.25, 0.3), middle: f(0.45, 0.25, 0.3), ring: f(0.45, 0.25, 0.3), pinky: f(0.45, 0.25, 0.3) },
+} satisfies Record<string, GripPoseDef>));
+
+/** Adds a pose of the game's own, or replaces one by name. @experimental */
+export function registerGripPose(name: string, pose: GripPoseDef): void {
+ poses.set(name, pose);
+}
+
+/** The names a `GripPose` component can take. @experimental */
+export function gripPoseNames(): string[] {
+ return [...poses.keys()];
+}
+
+/** The pose of that name, or undefined. @experimental */
+export function gripPose(name: string): GripPoseDef | undefined {
+ return poses.get(name);
+}
+
+/** The fields of the `GripPose` component. @experimental */
+export interface GripPoseData {
+ /** The pose each hand takes; empty leaves that hand to whatever else moves it. */
+ rightPose: string;
+ leftPose: string;
+ /** How much of it applies, over what a clip is doing with the fingers. */
+ rightWeight: number;
+ leftWeight: number;
+ /** How far the index finger has gone from resting to pressing. */
+ rightTrigger: number;
+ leftTrigger: number;
+ /** About how long a change of pose takes, in seconds. */
+ blendSeconds: number;
+ /** The hand joint, by path from this entity. Empty looks for it by name. */
+ rightHand: string;
+ leftHand: string;
+}
+
+/**
+ * Holds a rig's hands in a named pose. Goes on the rig's root.
+ *
+ * @experimental
+ */
+export const GripPose: ComponentDef<GripPoseData> = /* @__PURE__ */ defineComponent<GripPoseData>('GripPose', {
+ rightPose: '', leftPose: '', rightWeight: 1, leftWeight: 1, rightTrigger: 0, leftTrigger: 0, blendSeconds: 0.15, rightHand: '', leftHand: '',
+}, {
+ fields: {
+ rightPose: { category: 'Right Hand', tooltip: 'open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone.' },
+ rightWeight: { min: 0, max: 1, step: 0.01, category: 'Right Hand', tooltip: 'How much of the pose applies, over what a clip is doing with the fingers.' },
+ rightTrigger: { min: 0, max: 1, step: 0.01, category: 'Right Hand', tooltip: 'How far the index finger has gone from resting to pressing, in a pose that has both.' },
+ rightHand: { category: 'Right Hand', advanced: true, tooltip: 'The hand joint, by path from this entity. Empty looks for it by name.' },
+ leftPose: { category: 'Left Hand', tooltip: 'open, relaxed, fist, pistol, support, handle, pinch, cup, or one the game registers. Empty leaves the hand alone.' },
+ leftWeight: { min: 0, max: 1, step: 0.01, category: 'Left Hand', tooltip: 'How much of the pose applies, over what a clip is doing with the fingers.' },
+ leftTrigger: { min: 0, max: 1, step: 0.01, category: 'Left Hand', tooltip: 'How far the index finger has gone from resting to pressing, in a pose that has both.' },
+ leftHand: { category: 'Left Hand', advanced: true, tooltip: 'The hand joint, by path from this entity. Empty looks for it by name.' },
+ blendSeconds: { min: 0, step: 0.01, unit: 's', tooltip: 'About how long a change of pose takes.' },
+ },
+});
+
+interface Quat { w: number; x: number; y: number; z: number }
+interface Vec3 { x: number; y: number; z: number }
+
+/** The fingers, in the order a hand's numbers are kept. */
+const FINGERS = ['thumb', 'index', 'middle', 'ring', 'pinky'] as const;
+type FingerName = (typeof FINGERS)[number];
+
+const NAME_OF: Record<FingerName, RegExp> = {
+ thumb: /thumb/i, index: /index|pointer/i, middle: /middle/i, ring: /(?<![a-z])ring|[a-z]ring(?=[\d_.:-]|$)/i, pinky: /pinky|pinkie|little/i,
+};
+/** A bone inside the palm: nothing a pose bends, and what the first joint of a finger may hang from. */
+const INSIDE_PALM = /metacarpal|palm/i;
+
+/** How far each joint of a finger bends when it is closed: first, second, third. In radians. */
+const CLOSED = [Math.PI / 2, (100 * Math.PI) / 180, (70 * Math.PI) / 180];
+/** The thumb's are far less: its root barely bends, and what it does there is opposition's. */
+const THUMB_CLOSED = [(15 * Math.PI) / 180, (60 * Math.PI) / 180, (80 * Math.PI) / 180];
+const SPREAD = (15 * Math.PI) / 180;
+const FAN: Record<FingerName, number> = { thumb: 0, index: 1, middle: 0.15, ring: -0.6, pinky: -1 };
+
+const add = (a: Vec3, b: Vec3): Vec3 => ({ x: a.x + b.x, y: a.y + b.y, z: a.z + b.z });
+const sub = (a: Vec3, b: Vec3): Vec3 => ({ x: a.x - b.x, y: a.y - b.y, z: a.z - b.z });
+const dot = (a: Vec3, b: Vec3): number => a.x * b.x + a.y * b.y + a.z * b.z;
+const scale = (a: Vec3, k: number): Vec3 => ({ x: a.x * k, y: a.y * k, z: a.z * k });
+const cross = (a: Vec3, b: Vec3): Vec3 => ({ x: a.y * b.z - a.z * b.y, y: a.z * b.x - a.x * b.z, z: a.x * b.y - a.y * b.x });
+const norm = (a: Vec3): Vec3 => { const l = Math.hypot(a.x, a.y, a.z); return l < 1e-9 ? { x: 0, y: 0, z: 0 } : scale(a, 1 / l); };
+const axisAngle = (axis: Vec3, radians: number): Quat => {
+ const s = Math.sin(radians / 2);
+ return { w: Math.cos(radians / 2), x: axis.x * s, y: axis.y * s, z: axis.z * s };
+};
+
+const nameOf = (world: World, e: Entity): string => (world.tryGet(e, Name) as { value?: string } | null)?.value ?? '';
+const childrenOf = (world: World, e: Entity): Entity[] => (world.tryGet(e, Children) as { entities?: Entity[] } | null)?.entities ?? [];
+
+/** One joint a hand moves: where it rests, and the turns a pose asks of it, in its own frame. */
+interface Joint {
+ entity: Entity;
+ rest: Quat;
+ curlAxis: Vec3;
+ /** On a finger's first joint. */
+ spreadAxis?: Vec3;
+ /** On the thumb's first joint: the turn that faces it to the fingers at full opposition, as an axis and an angle. */
+ opposeAxis?: Vec3;
+ opposeAngle?: number;
+ /** What this wrote last, to tell it from what a clip has written since. */
+ written: Quat;
+}
+
+/** A finger's joints, knuckle first. Those it has no joint for are left out. */
+type Finger = { name: FingerName; joints: Joint[]; fan: number };
+
+/** Numbers a hand is held by, kept flat so two poses can be eased between: curl, base, spread for each finger, then the thumb's opposition. */
+const NUMBERS = FINGERS.length * 3 + 1;
+
+function numbersOf(pose: GripPoseDef | null, out: Float64Array): void {
+ let i = 0;
+ for (const name of FINGERS) {
+ const finger = pose?.[name];
+ out[i++] = clamp01(finger?.curl ?? 0);
+ out[i++] = clamp01(finger?.base ?? finger?.curl ?? 0);
+ out[i++] = Math.max(-1, Math.min(1, finger?.spread ?? 0));
+ }
+ out[i] = clamp01(pose?.thumb.opposition ?? 0);
+}
+
+const clamp01 = (v: number): number => (v > 0 ? (v < 1 ? v : 1) : 0);
+
+/** What is known of one hand on one rig. */
+interface HandState {
+ hand: Entity;
+ fingers: Finger[];
+ /** Which fingers it could not find, to say so once. */
+ missing: FingerName[];
+ /** The numbers now, and how much of them is applied now: both eased. */
+ current: Float64Array;
+ wanted: Float64Array;
+ weight: number;
+ /** How fast each number is moving, and the weight after them. */
+ speed: Float64Array;
+ /** Whether `current` and `weight` have arrived and everything written still stands. */
+ settled: boolean;
+ pose: string;
+ trigger: number;
+}
+
+/** A finger's bones in order, from the hand: those named for it, knuckle first, those in the palm passed through. */
+function fingerChain(world: World, hand: Entity, name: FingerName): Entity[] {
+ const pattern = NAME_OF[name];
+ const search = (at: Entity, depth: number): Entity | null => {
+ if (depth > 3) return null;
+ for (const child of childrenOf(world, at)) {
+ if (pattern.test(nameOf(world, child)) && !INSIDE_PALM.test(nameOf(world, child))) return child;
+ }
+ for (const child of childrenOf(world, at)) {
+ const named = nameOf(world, child);
+ // Through the palm's own bones, and not into something held.
+ if (INSIDE_PALM.test(named)) { const found = search(child, depth + 1); if (found !== null) return found; }
+ }
+ return null;
+ };
+ const first = search(hand, 0);
+ if (first === null) return [];
+ const chain = [first];
+ for (let at = first, guard = 0; guard < 6; guard++) {
+ const next = childrenOf(world, at).find((c) => pattern.test(nameOf(world, c))) ?? null;
+ if (next === null) break;
+ chain.push(next);
+ at = next;
+ }
+ return chain;
+}
+
+/** Where a joint stands from the hand, and its turn from it. */
+function fromHand(world: World, chain: Entity[]): { at: Vec3[]; turn: Quat[] } {
+ let position: Vec3 = { x: 0, y: 0, z: 0 };
+ let rotation: Quat = { w: 1, x: 0, y: 0, z: 0 };
+ const at: Vec3[] = [], turn: Quat[] = [];
+ for (const joint of chain) {
+ const local = world.get(joint, Transform) as TransformData;
+ position = add(position, q.rotate(rotation, local.position));
+ rotation = q.normalize(q.mul(rotation, local.rotation));
+ at.push(position);
+ turn.push(rotation);
+ }
+ return { at, turn };
+}
+
+/** Which way the palm faces, and toward the thumb across it, in the hand's own frame. Null with too few fingers to tell. */
+function palmOf(world: World, side: 'right' | 'left', chains: Partial<Record<FingerName, Entity[]>>): { palm: Vec3; across: Vec3; forward: Vec3 } | null {
+ const knuckle = (name: FingerName): Vec3 | null => {
+ const chain = chains[name];
+ return chain && chain.length ? fromHand(world, chain).at[0]! : null;
+ };
+ const outer = knuckle('index') ?? knuckle('middle');
+ const inner = knuckle('pinky') ?? knuckle('ring') ?? knuckle('middle');
+ const middle = knuckle('middle') ?? outer;
+ if (!outer || !inner || !middle) return null;
+ const across = norm(sub(outer, inner));
+ if (Math.hypot(across.x, across.y, across.z) < 0.5) return null;
+ // The way the fingers themselves point, and square to the knuckles: a wrist is rarely straight behind its middle finger.
+ const long = chains.middle && chains.middle.length > 1 ? chains.middle : chains.index && chains.index.length > 1 ? chains.index : null;
+ const placed = long ? fromHand(world, long).at : null;
+ const along = placed ? sub(placed[1]!, placed[0]!) : middle;
+ const forward = norm(sub(along, scale(across, dot(along, across))));
+ // A right hand with its fingers ahead and its thumb to the left has its palm down: which is the cross product, one way round for each hand.
+ const palm = norm(scale(cross(forward, across), side === 'right' ? -1 : 1));
+ return Math.hypot(palm.x, palm.y, palm.z) < 0.5 ? null : { palm, across, forward };
+}
+
+/** The hand's joints, with the turns a pose asks of each. They are worked out from the rig as it stands now, once. */
+function buildHand(world: World, hand: Entity, side: 'right' | 'left'): HandState | null {
+ const chains: Partial<Record<FingerName, Entity[]>> = {};
+ for (const name of FINGERS) chains[name] = fingerChain(world, hand, name);
+ const shape = palmOf(world, side, chains);
+ if (!shape) return null;
+ const { palm, across, forward } = shape;
+ const fingers: Finger[] = [];
+ const missing: FingerName[] = [];
+ for (const name of FINGERS) {
+ const chain = chains[name]!;
+ if (chain.length === 0) { missing.push(name); continue; }
+ const { at, turn } = fromHand(world, chain);
+ // Its first joint to its second is the way it points; a finger of one joint points as the joint does from the hand.
+ const pointing = norm(chain.length > 1 ? sub(at[1]!, at[0]!) : at[0]!);
+ // The thumb's way to the fingers is a direction and not a turn about some axis: where it starts from differs from rig to rig, and where it has to get to does not.
+ const facing = name === 'thumb' ? norm(add(add(scale(forward, 0.85), scale(palm, 0.45)), scale(across, 0.25))) : pointing;
+ const oppose = q.rotationTo(pointing, facing);
+ // A closing thumb lies over the first two fingers, its tip going on toward the little finger.
+ const towards = name === 'thumb' ? norm(add(add(scale(palm, 0.15), scale(across, -1)), scale(forward, -0.1))) : palm;
+ // Worked out where it faces once opposed, and taken back to where it rests, since that turn comes first.
+ const bend = q.rotate(q.conjugate(oppose), norm(cross(facing, towards)));
+ // The last bone of a chain that goes on past three is the tip's end, which nothing bends.
+ const moved = chain.slice(0, 3);
+ const joints: Joint[] = moved.map((entity, i) => {
+ const toJoint = q.conjugate(turn[i]!);
+ const rest = q.normalize((world.get(entity, Transform) as TransformData).rotation);
+ const joint: Joint = { entity, rest: { ...rest }, curlAxis: norm(q.rotate(toJoint, bend)), written: { ...rest } };
+ if (i === 0) {
+ joint.spreadAxis = norm(q.rotate(toJoint, palm));
+ // Toward the index finger for a positive spread.
+ if (dot(cross(palm, pointing), across) < 0) joint.spreadAxis = scale(joint.spreadAxis, -1);
+ if (name === 'thumb') {
+ const half = Math.min(1, Math.abs(oppose.w));
+ const axis = norm({ x: oppose.x, y: oppose.y, z: oppose.z });
+ joint.opposeAxis = norm(q.rotate(toJoint, oppose.w < 0 ? scale(axis, -1) : axis));
+ joint.opposeAngle = 2 * Math.acos(half);
+ }
+ }
+ return joint;
+ });
+ fingers.push({ name, joints, fan: FAN[name] });
+ }
+ return { hand, fingers, missing, current: new Float64Array(NUMBERS), wanted: new Float64Array(NUMBERS), weight: 0, speed: new Float64Array(NUMBERS + 1), settled: false, pose: '', trigger: 0 };
+}
+
+const NAMED_HAND = /hand|wrist/i;
+const NOT_A_HAND = /thumb|index|middle|ring|pinky|pinkie|little|finger|metacarpal|palm/i;
+const IS_RIGHT = /right|(^|[^a-z])r($|[^a-z])/i;
+const IS_LEFT = /left|(^|[^a-z])l($|[^a-z])/i;
+
+/** The joint named for that hand under `root`, nearest first. */
+function findHand(world: World, root: Entity, side: 'right' | 'left'): Entity | null {
+ const wanted = side === 'right' ? IS_RIGHT : IS_LEFT;
+ const queue: Entity[] = [root];
+ for (let i = 0; i < queue.length && i < 512; i++) {
+ const at = queue[i]!;
+ const name = nameOf(world, at);
+ if (at !== root && NAMED_HAND.test(name) && !NOT_A_HAND.test(name) && wanted.test(name) && fingerChain(world, at, 'middle').length + fingerChain(world, at, 'index').length > 0) return at;
+ queue.push(...childrenOf(world, at));
+ }
+ return null;
+}
+
+/** What one rig's hands are doing. */
+interface RigState { right: HandState | null; left: HandState | null; rightHand: string; leftHand: string; tried: { right: boolean; left: boolean } }
+
+const SAME = 1e-6;
+const same = (a: Quat, b: Quat): boolean => Math.abs(a.w - b.w) < SAME && Math.abs(a.x - b.x) < SAME && Math.abs(a.y - b.y) < SAME && Math.abs(a.z - b.z) < SAME;
+
+/**
+ * A number moved toward where it is wanted, critically damped: it starts from still and comes to rest without passing it. Going a fixed share of the
+ * gap each frame would be at its fastest on the first, and a finger with far to go visibly jumps.
+ */
+function ease(value: number, wanted: number, state: HandState, slot: number, blend: number, dt: number): number {
+ if (blend <= 0) { state.speed[slot] = 0; return wanted; }
+ const omega = 6 / blend;
+ const gap = value - wanted;
+ const carry = (state.speed[slot]! + omega * gap) * dt;
+ const fade = Math.exp(-omega * dt);
+ const speed = (state.speed[slot]! - omega * carry) * fade;
+ const next = wanted + (gap + carry) * fade;
+ if (Math.abs(next - wanted) < 1e-3 && Math.abs(speed) < 1e-2) { state.speed[slot] = 0; return wanted; }
+ state.speed[slot] = speed;
+ return next;
+}
+
+/** Holds one hand a frame: eases its numbers on, and turns each joint to them over what is there. */
+function holdHand(world: World, state: HandState, poseName: string, weight: number, trigger: number, blend: number, dt: number, moved: boolean): void {
+ const pose = poseName ? poses.get(poseName) ?? null : null;
+ if (poseName !== state.pose) {
+ if (poseName && !pose) log.warn('animation', `there is no grip pose called "${poseName}": the hand is left alone`);
+ state.pose = poseName;
+ // A new pose eases on from where the hand is; a hand let go keeps its numbers and fades by its weight.
+ if (pose) numbersOf(pose, state.wanted);
+ state.settled = false;
+ }
+ const wantedWeight = pose ? clamp01(weight) : 0;
+ const press = pose?.trigger ? clamp01(trigger) : 0;
+ if (press !== state.trigger) { state.trigger = press; state.settled = false; }
+ if (wantedWeight !== state.weight) state.settled = false;
+ if (state.settled && !moved) return;
+
+ let arrived = true;
+ for (let i = 0; i < NUMBERS; i++) {
+ const next = ease(state.current[i]!, state.wanted[i]!, state, i, blend, dt);
+ state.current[i] = next;
+ if (next !== state.wanted[i]) arrived = false;
+ }
+ state.weight = ease(state.weight, wantedWeight, state, NUMBERS, blend, dt);
+ if (state.weight !== wantedWeight) arrived = false;
+
+ const opposition = state.current[NUMBERS - 1]!;
+ for (const finger of state.fingers) {
+ const at = FINGERS.indexOf(finger.name) * 3;
+ let curl = state.current[at]!, base = state.current[at + 1]!;
+ const spread = state.current[at + 2]!;
+ if (finger.name === 'index' && press > 0 && pose?.trigger) {
+ curl += (clamp01(pose.trigger.curl) - curl) * press;
+ base += (clamp01(pose.trigger.base ?? pose.trigger.curl) - base) * press;
+ }
+ for (let i = 0; i < finger.joints.length; i++) {
+ const joint = finger.joints[i]!;
+ let turn = axisAngle(joint.curlAxis, (i === 0 ? base : curl) * (finger.name === 'thumb' ? THUMB_CLOSED : CLOSED)[i]!);
+ if (joint.spreadAxis && spread !== 0 && finger.fan !== 0) turn = q.mul(axisAngle(joint.spreadAxis, spread * finger.fan * SPREAD), turn);
+ if (joint.opposeAxis && opposition > 0) turn = q.mul(axisAngle(joint.opposeAxis, opposition * joint.opposeAngle!), turn);
+ const target = q.normalize(q.mul(joint.rest, turn));
+ const local = world.get(joint.entity, Transform) as TransformData;
+ // Moved since this wrote it, a clip is speaking and the pose goes over that; otherwise over how the rig rests.
+ const from = same(local.rotation, joint.written) ? joint.rest : local.rotation;
+ const out = { w: from.w, x: from.x, y: from.y, z: from.z };
+ if (state.weight >= 1) { out.w = target.w; out.x = target.x; out.y = target.y; out.z = target.z; }
+ else leanQuat(out, target, state.weight);
+ const written = q.normalize(out);
+ world.update(joint.entity, Transform, (t: TransformData) => { t.rotation = { w: written.w, x: written.x, y: written.y, z: written.z }; });
+ joint.written = { ...((world.get(joint.entity, Transform) as TransformData).rotation) };
+ }
+ }
+ state.settled = arrived;
+}
+
+function letGo(world: World, state: HandState | null): void {
+ if (!state) return;
+ for (const finger of state.fingers) {
+ for (const joint of finger.joints) {
+ if (world.valid(joint.entity) && world.has(joint.entity, Transform)) {
+ world.update(joint.entity, Transform, (t: TransformData) => { t.rotation = { w: joint.rest.w, x: joint.rest.x, y: joint.rest.y, z: joint.rest.z }; });
+ }
+ }
+ }
+}
+
+/** What a rig's hand was found to have. @experimental */
+export interface GripHandStatus {
+ /** The joint it took for the hand, or null where none could be found. */
+ hand: Entity | null;
+ /** How many joints a pose moves on it. */
+ joints: number;
+ /** The fingers it has no bones for: a pose simply does without them. */
+ missing: string[];
+ /** The pose it is in. */
+ pose: string;
+ /** How much of it applies now. */
+ weight: number;
+}
+
+const rigs = new WeakMap<App, Map<Entity, RigState>>();
+
+/** What each hand of a rig with a `GripPose` was found to have, once the system has seen it. @experimental */
+export function gripPoseStatus(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus } | null {
+ const state = rigs.get(app)?.get(rig);
+ if (!state) return null;
+ const of = (h: HandState | null): GripHandStatus => ({
+ hand: h?.hand ?? null, joints: h ? h.fingers.reduce((n, finger) => n + finger.joints.length, 0) : 0, missing: h ? [...h.missing] : [...FINGERS], pose: h?.pose ?? '', weight: h?.weight ?? 0,
+ });
+ return { right: of(state.right), left: of(state.left) };
+}
+
+/** Holds every rig's hands. Runs after the animator, since what a clip says of the fingers is what a pose goes over. */
+export function installGripPose(app: App): void {
+ const known = new Map<Entity, RigState>();
+ rigs.set(app, known);
+ // By name: what moves the bones is the animator's or the timeline's, which this does not need to carry.
+ const hasClipOn = (world: World, rig: Entity): boolean => ['Animator', 'TimelinePlayer'].some((name) => { const def = getComponent(name); return !!def && world.has(rig, def); });
+ // system-access: the hand's joints are found by walking the rig, so which entities it writes is known only then, and only their Transforms.
+ app.addSystemToSchedule(Schedule.Update, defineSystem([Res(Time), GetWorld()], (time: TimeData, world: World) => {
+ for (const [rig, state] of known) {
+ if (world.valid(rig) && world.has(rig, GripPose)) continue;
+ letGo(world, state.right);
+ letGo(world, state.left);
+ known.delete(rig);
+ }
+ for (const rig of world.getEntitiesWithComponents([GripPose])) {
+ const data = world.get(rig, GripPose) as GripPoseData;
+ if (!data.rightPose && !data.leftPose && !known.has(rig)) continue;
+ let state = known.get(rig);
+ if (!state || state.rightHand !== data.rightHand || state.leftHand !== data.leftHand) {
+ if (state) { letGo(world, state.right); letGo(world, state.left); }
+ state = { right: null, left: null, rightHand: data.rightHand, leftHand: data.leftHand, tried: { right: false, left: false } };
+ known.set(rig, state);
+ }
+ const moved = hasClipOn(world, rig);
+ for (const side of ['right', 'left'] as const) {
+ const poseName = side === 'right' ? data.rightPose : data.leftPose;
+ if (!state[side] && poseName && !state.tried[side]) {
+ state.tried[side] = true;
+ const path = side === 'right' ? data.rightHand : data.leftHand;
+ const hand = path ? resolveChildEntity(world, rig, path) : findHand(world, rig, side);
+ state[side] = hand === null ? null : buildHand(world, hand, side);
+ if (!state[side]) log.warn('animation', `a GripPose found no ${side} hand with fingers on "${nameOf(world, rig)}"${path ? ` at "${path}"` : ''}: name its joint in ${side}Hand`);
+ else if (state[side]!.missing.length) log.warn('animation', `the ${side} hand of "${nameOf(world, rig)}" has no bones for its ${state[side]!.missing.join(', ')}: a pose does without them`);
+ }
+ const hand = state[side];
+ if (hand) {
+ holdHand(world, hand, poseName, side === 'right' ? data.rightWeight : data.leftWeight, side === 'right' ? data.rightTrigger : data.leftTrigger, data.blendSeconds, time.delta, moved);
+ }
+ }
+ }
+ }, { name: 'GripPoseSystem' }), { runAfter: [SystemLabel.Animator], runIf: playModeOnly });
+}
diff --git a/sdk/src/animation/index.ts b/sdk/src/animation/index.ts
index e52d8c676..46ac457a4 100644
--- a/sdk/src/animation/index.ts
+++ b/sdk/src/animation/index.ts
@@ -81,6 +81,10 @@ export {
type AnimatorEvalResult,
type AnimatorPathEvalResult,
} from './Animator';
+export {
+ GripPose, registerGripPose, gripPose, gripPoseNames, gripPoseStatus,
+ type GripPoseData, type GripPoseDef, type GripFinger, type GripThumb, type GripHandStatus,
+} from './gripPose';
export {
emptyAnimatorController,
diff --git a/sdk/src/core-content.ts b/sdk/src/core-content.ts
index 0e845299a..20d7b5d42 100644
--- a/sdk/src/core-content.ts
+++ b/sdk/src/core-content.ts
@@ -650,6 +650,16 @@ export {
TweenSequence,
type Completable,
type TweenFactory,
+ GripPose,
+ registerGripPose,
+ gripPose,
+ gripPoseNames,
+ gripPoseStatus,
+ type GripPoseData,
+ type GripPoseDef,
+ type GripFinger,
+ type GripThumb,
+ type GripHandStatus,
} from './animation';
// =============================================================================
diff --git a/sdk/tests/grip-pose-wasm.test.ts b/sdk/tests/grip-pose-wasm.test.ts
new file mode 100644
index 000000000..a8518883c
--- /dev/null
+++ b/sdk/tests/grip-pose-wasm.test.ts
@@ -0,0 +1,446 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * A hand held in a named pose, on the real engine. What makes a pose worth having is that it fits any rig, so the claims are about where the
+ * fingertips end up in the world: the same for two rigs whose bones face different ways, and mirrored for the other hand.
+ */
+import { afterEach, beforeAll, describe, expect, it, vi } from 'vitest';
+import { App } from '../src/app/app';
+import { Animator } from '../src/animation';
+import { AnimationPlugin } from '../src/animation/AnimationPlugin';
+import { GripPose, gripPoseNames, gripPoseStatus, registerGripPose, type GripPoseData } from '../src/animation/gripPose';
+import { Name, Transform, type TransformData } from '../src/ecs/component';
+import { setEditorMode, setPlayMode } from '../src/ecs/env';
+import { q } from '../src/math/quat';
+import type { Entity } from '../src/types';
+import type { ESEngineModule } from '../src/wasm';
+import { HAS_WASM, loadWasmModule } from './helpers/loadWasm';
+
+interface V { x: number; y: number; z: number }
+interface Q { w: number; x: number; y: number; z: number }
+const IDENTITY: Q = { w: 1, x: 0, y: 0, z: 0 };
+
+let core: ESEngineModule;
+const apps: App[] = [];
+afterEach(() => { while (apps.length) apps.pop()!.quit?.(); setEditorMode(false); setPlayMode(false); vi.restoreAllMocks(); });
+
+function scene() {
+ core.setActiveContext(null);
+ const app = App.new(); apps.push(app); app.connectCpp(new core.Registry(), core, { ownedRegistry: true });
+ setPlayMode(true);
+ app.addPlugin(new AnimationPlugin());
+ const tick = async (n = 1): Promise<void> => { for (let i = 0; i < n; i++) { await app.tick(1 / 60); await Promise.resolve(); } };
+ return { app, world: app.world, tick };
+}
+type World = App['world'];
+
+/** Where a finger's joints stand, seen from the hand with the fingers ahead along +Y, the palm toward -Z and, on a right hand, the thumb toward -X. */
+const LAYOUT: Record<string, { from: V; along: V; lengths: number[] }> = {
+ thumb: { from: { x: -0.03, y: 0.02, z: -0.01 }, along: { x: -0.7, y: 0.7, z: 0 }, lengths: [0.035, 0.03, 0.025] },
+ index: { from: { x: -0.025, y: 0.09, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.04, 0.025, 0.02] },
+ middle: { from: { x: -0.005, y: 0.095, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.045, 0.03, 0.02] },
+ ring: { from: { x: 0.015, y: 0.09, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.04, 0.028, 0.02] },
+ pinky: { from: { x: 0.033, y: 0.08, z: 0 }, along: { x: 0, y: 1, z: 0 }, lengths: [0.03, 0.02, 0.018] },
+};
+
+interface RigOptions {
+ side?: 'right' | 'left';
+ /** The turn each bone is given from its parent. The picture of the hand stays the same, since the positions are written to match. */
+ frame?: (depth: number) => Q;
+ names?: (finger: string, i: number) => string;
+ handName?: string;
+ without?: string[];
+ /** Bones in the palm between the hand and each finger. */
+ metacarpal?: boolean;
+ /** A turn over the whole rig. */
+ root?: Q;
+}
+
+/** A hand built so its world shape does not depend on how its bones face. Answers the bones by finger, tips included. */
+function buildRig(world: World, options: RigOptions = {}) {
+ const mirror = options.side === 'left' ? -1 : 1;
+ const frame = options.frame ?? (() => IDENTITY);
+ const names = options.names ?? ((finger, i) => `mixamorig:${options.side === 'left' ? 'Left' : 'Right'}Hand${finger[0]!.toUpperCase()}${finger.slice(1)}${i + 1}`);
+ const spawn = (name: string, parent: Entity | null, position: V, rotation: Q): Entity => {
+ const e = world.spawn(name);
+ world.insert(e, Name, { value: name });
+ world.insert(e, Transform, { position, rotation });
+ if (parent !== null) world.setParent(e, parent);
+ return e;
+ };
+ const root = spawn('Rig', null, { x: 0, y: 0, z: 0 }, options.root ?? IDENTITY);
+ const armature = spawn('Armature', root, { x: 0, y: 0, z: 0 }, IDENTITY);
+ const handTurn = frame(0);
+ const hand = spawn(options.handName ?? `mixamorig:${options.side === 'left' ? 'Left' : 'Right'}Hand`, armature, { x: 0, y: 0, z: 0 }, handTurn);
+ const bones: Record<string, Entity[]> = {};
+ for (const [finger, layout] of Object.entries(LAYOUT)) {
+ if (options.without?.includes(finger)) continue;
+ let parent = hand, parentTurn = handTurn, at: V = { x: 0, y: 0, z: 0 };
+ const chain: Entity[] = [];
+ const placeChild = (name: string, worldAt: V, depth: number): Entity => {
+ const turn = frame(depth);
+ const local = q.rotate(q.conjugate(parentTurn), { x: worldAt.x - at.x, y: worldAt.y - at.y, z: worldAt.z - at.z });
+ const e = spawn(name, parent, local, q.normalize(q.mul(q.conjugate(parentTurn), turn)));
+ parent = e; parentTurn = turn; at = worldAt;
+ return e;
+ };
+ const start: V = { x: layout.from.x * mirror, y: layout.from.y, z: layout.from.z };
+ if (options.metacarpal && finger !== 'thumb') placeChild(`${finger}_metacarpal`, { x: start.x * 0.5, y: start.y * 0.4, z: start.z }, 1);
+ let p = start;
+ chain.push(placeChild(names(finger, 0), p, 2));
+ const along = { x: layout.along.x * mirror, y: layout.along.y, z: layout.along.z };
+ const unit = Math.hypot(along.x, along.y, along.z);
+ layout.lengths.forEach((l, i) => {
+ p = { x: p.x + along.x / unit * l, y: p.y + along.y / unit * l, z: p.z + along.z / unit * l };
+ chain.push(placeChild(names(finger, i + 1), p, 3 + i));
+ });
+ bones[finger] = chain;
+ }
+ return { root, hand, bones };
+}
+
+/** A bone's place in the world, from the locals: the world's own record is last frame's. */
+function placeOf(world: World, e: Entity, stop: Entity): V {
+ const links: Entity[] = [];
+ for (let at = e; ; at = (world.get(at, getParent(world)) as { entity: Entity }).entity) { links.unshift(at); if (at === stop) break; }
+ let p: V = { x: 0, y: 0, z: 0 }, r: Q = IDENTITY;
+ for (const link of links) {
+ const t = world.get(link, Transform) as TransformData;
+ p = (() => { const v = q.rotate(r, t.position); return { x: p.x + v.x, y: p.y + v.y, z: p.z + v.z }; })();
+ r = q.normalize(q.mul(r, t.rotation));
+ }
+ return p;
+}
+import { Parent } from '../src/ecs/component';
+const getParent = (_world: World) => Parent;
+
+const tips = (world: World, rig: ReturnType<typeof buildRig>): Record<string, V> => Object.fromEntries(Object.entries(rig.bones).map(([finger, chain]) => [finger, placeOf(world, chain[chain.length - 1]!, rig.hand)]));
+const hold = (world: World, rig: { root: Entity }, data: Partial<GripPoseData>): void => { world.insert(rig.root, GripPose, { blendSeconds: 0, ...data }); };
+const close = (a: V, b: V, digits = 3): void => { expect(a.x).toBeCloseTo(b.x, digits); expect(a.y).toBeCloseTo(b.y, digits); expect(a.z).toBeCloseTo(b.z, digits); };
+const dist = (a: V, b: V): number => Math.hypot(a.x - b.x, a.y - b.y, a.z - b.z);
+
+/** A turn that has nothing to do with how a hand bends: bones in a real rig face wherever the exporter left them. */
+const scrambled = (depth: number): Q => q.normalize({ w: 0.3 + depth * 0.17, x: Math.sin(depth * 1.9 + 0.4), y: Math.cos(depth * 2.7), z: Math.sin(depth * 0.8 - 1) });
+
+describe.skipIf(!HAS_WASM)('a grip pose', () => {
+ beforeAll(async () => { if (HAS_WASM) core = await loadWasmModule(); });
+
+ it('closes the fingers toward the palm, not away from it', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig);
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ const shut = tips(s.world, rig);
+ for (const f of ['index', 'middle', 'ring', 'pinky']) {
+ // A closed finger rolls back toward the palm, so its tip ends near it, on the palm's side.
+ expect(shut[f]!.z, f).toBeLessThan(-0.005);
+ expect(shut[f]!.y, f).toBeLessThan(open[f]!.y - 0.03);
+ }
+ });
+
+ it('brings the thumb across the palm toward the fingers', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig);
+ hold(s.world, rig, { rightPose: 'handle' });
+ await s.tick(2);
+ const held = tips(s.world, rig);
+ expect(held.thumb!.z).toBeLessThan(-0.01);
+ expect(dist(held.thumb!, held.index!)).toBeLessThan(dist(open.thumb!, open.index!));
+ });
+
+ it('does not turn the thumb back over the wrist, however closed', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ // Ahead of where the fingers start from the hand, and on the palm's side: a thumb that has gone past the wrist is behind both.
+ const tip = tips(s.world, rig).thumb!;
+ expect(tip.y).toBeGreaterThan(0);
+ expect(tip.z).toBeLessThan(0);
+ // And over the fingers, not out past the little one on the far side.
+ const knuckles = LAYOUT.pinky!.from.x;
+ expect(tip.x).toBeLessThan(knuckles);
+ expect(tip.x).toBeGreaterThan(LAYOUT.index!.from.x - 0.02);
+ });
+
+ it('turns the thumb toward the fingers by its opposition, whatever else it does', async () => {
+ const none = { curl: 0 };
+ registerGripPose('thumb-beside', { thumb: { curl: 0, opposition: 0 }, index: none, middle: none, ring: none, pinky: none });
+ registerGripPose('thumb-across', { thumb: { curl: 0, opposition: 1 }, index: none, middle: none, ring: none, pinky: none });
+ const s = scene();
+ const beside = buildRig(s.world);
+ hold(s.world, beside, { rightPose: 'thumb-beside' });
+ const t = scene();
+ const across = buildRig(t.world);
+ hold(t.world, across, { rightPose: 'thumb-across' });
+ await s.tick(2);
+ await t.tick(2);
+ const a = tips(s.world, beside).thumb!, b = tips(t.world, across).thumb!;
+ expect(b.z).toBeLessThan(a.z - 0.01);
+ expect(b.y).toBeGreaterThan(a.y);
+ expect(tips(t.world, across).index!.z).toBeCloseTo(tips(s.world, beside).index!.z, 5);
+ });
+
+ it('makes the same hand on rigs whose bones face different ways', async () => {
+ for (const pose of ['fist', 'pistol', 'cup', 'pinch']) {
+ const s = scene();
+ const a = buildRig(s.world);
+ hold(s.world, a, { rightPose: pose });
+ const t = scene();
+ const b = buildRig(t.world, { frame: scrambled });
+ hold(t.world, b, { rightPose: pose });
+ await s.tick(2);
+ await t.tick(2);
+ const ta = tips(s.world, a), tb = tips(t.world, b);
+ for (const f of Object.keys(ta)) close(ta[f]!, tb[f]!);
+ }
+ });
+
+ it('mirrors for the other hand', async () => {
+ const s = scene();
+ const right = buildRig(s.world);
+ hold(s.world, right, { rightPose: 'pistol' });
+ const t = scene();
+ const left = buildRig(t.world, { side: 'left' });
+ hold(t.world, left, { leftPose: 'pistol' });
+ await s.tick(2);
+ await t.tick(2);
+ const r = tips(s.world, right), l = tips(t.world, left);
+ for (const f of Object.keys(r)) close({ x: -r[f]!.x, y: r[f]!.y, z: r[f]!.z }, l[f]!);
+ });
+
+ it('holds both hands of one rig, each its own', async () => {
+ const s = scene();
+ const root = s.world.spawn('Rig');
+ s.world.insert(root, Transform, {});
+ const r = buildRig(s.world), l = buildRig(s.world, { side: 'left' });
+ s.world.setParent(r.root, root); s.world.setParent(l.root, root);
+ const openLeft = tips(s.world, l);
+ hold(s.world, { root }, { rightPose: 'fist', leftPose: 'open' });
+ await s.tick(2);
+ expect(tips(s.world, r).index!.z).toBeLessThan(-0.015);
+ close(tips(s.world, l).index!, openLeft.index!);
+ });
+
+ it('does not depend on the turn of the rig it is on', async () => {
+ const s = scene();
+ const a = buildRig(s.world);
+ hold(s.world, a, { rightPose: 'handle' });
+ const t = scene();
+ const b = buildRig(t.world, { root: q.normalize({ w: 0.5, x: 0.5, y: -0.5, z: 0.5 }) });
+ hold(t.world, b, { rightPose: 'handle' });
+ await s.tick(2);
+ await t.tick(2);
+ const ta = tips(s.world, a), tb = tips(t.world, b);
+ for (const f of Object.keys(ta)) close(ta[f]!, tb[f]!);
+ });
+
+ it('finds the hand and fingers under other rigs\' names, through the bones in the palm', async () => {
+ const s = scene();
+ const rig = buildRig(s.world, { handName: 'hand_r', metacarpal: true, names: (finger, i) => `${finger}_0${i + 1}_r` });
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ const status = gripPoseStatus(s.app, rig.root)!;
+ expect(status.right.hand).toBe(rig.hand);
+ expect(status.right.joints).toBe(15);
+ expect(status.right.missing).toEqual([]);
+ expect(tips(s.world, rig).middle!.z).toBeLessThan(-0.015);
+ });
+
+ it('does not take a finger for the hand', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(1);
+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBe(rig.hand);
+ });
+
+ it('does without the fingers a rig has no bones for, and says so', async () => {
+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
+ const s = scene();
+ const rig = buildRig(s.world, { without: ['ring', 'pinky'] });
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ const status = gripPoseStatus(s.app, rig.root)!;
+ expect(status.right.missing).toEqual(['ring', 'pinky']);
+ expect(status.right.joints).toBe(9);
+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.015);
+ warn.mockRestore();
+ });
+
+ it('takes the hand from a path where the name would not find it', async () => {
+ const s = scene();
+ const rig = buildRig(s.world, { handName: 'bone_17' });
+ hold(s.world, rig, { rightPose: 'fist', rightHand: 'Armature/bone_17' });
+ await s.tick(2);
+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBe(rig.hand);
+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.015);
+ });
+
+ it('leaves a hand it cannot find alone, and goes on', async () => {
+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
+ const s = scene();
+ const rig = buildRig(s.world, { handName: 'bone_17' });
+ const before = tips(s.world, rig);
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ expect(gripPoseStatus(s.app, rig.root)!.right.hand).toBeNull();
+ close(tips(s.world, rig).index!, before.index!);
+ warn.mockRestore();
+ });
+
+ it('stays put however long it is held: nothing piles up', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'pistol', rightWeight: 0.5 });
+ await s.tick(2);
+ const once = tips(s.world, rig);
+ await s.tick(120);
+ const later = tips(s.world, rig);
+ for (const f of Object.keys(once)) close(once[f]!, later[f]!, 5);
+ });
+
+ it('goes part of the way by its weight', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig).index!;
+ hold(s.world, rig, { rightPose: 'fist', rightWeight: 1 });
+ await s.tick(2);
+ const shut = tips(s.world, rig).index!;
+ const t = scene();
+ const half = buildRig(t.world);
+ hold(t.world, half, { rightPose: 'fist', rightWeight: 0.5 });
+ await t.tick(2);
+ const between = tips(t.world, half).index!;
+ expect(dist(between, open)).toBeGreaterThan(0.01);
+ expect(dist(between, shut)).toBeGreaterThan(0.01);
+ });
+
+ it('eases from one pose to the next, with no jump', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'open', blendSeconds: 0.2 });
+ await s.tick(30);
+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightPose = 'fist'; });
+ let last = tips(s.world, rig).index!, biggest = 0, first = -1;
+ for (let i = 0; i < 40; i++) {
+ await s.tick(1);
+ const now = tips(s.world, rig).index!;
+ if (first < 0) first = dist(now, last);
+ biggest = Math.max(biggest, dist(now, last));
+ last = now;
+ }
+ const t = scene();
+ const direct = buildRig(t.world);
+ const open = tips(t.world, direct).index!;
+ hold(t.world, direct, { rightPose: 'fist' });
+ await t.tick(2);
+ const whole = dist(tips(t.world, direct).index!, open);
+ expect(dist(last, tips(t.world, direct).index!)).toBeLessThan(0.002);
+ // It starts from still: a share of the gap each frame would be at its fastest on the first.
+ expect(first).toBeLessThan(whole * 0.25);
+ expect(biggest).toBeLessThan(whole * 0.4);
+ });
+
+ it('presses the trigger with the index finger alone, in a pose that has one', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'pistol', rightTrigger: 0 });
+ await s.tick(2);
+ const resting = tips(s.world, rig);
+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightTrigger = 1; });
+ await s.tick(2);
+ const pressed = tips(s.world, rig);
+ expect(dist(resting.index!, pressed.index!)).toBeGreaterThan(0.01);
+ close(resting.middle!, pressed.middle!, 5);
+ close(resting.thumb!, pressed.thumb!, 5);
+
+ const t = scene();
+ const plain = buildRig(t.world);
+ hold(t.world, plain, { rightPose: 'fist', rightTrigger: 0 });
+ await t.tick(2);
+ const shut = tips(t.world, plain).index!;
+ t.world.update(plain.root, GripPose, (d: GripPoseData) => { d.rightTrigger = 1; });
+ await t.tick(2);
+ close(shut, tips(t.world, plain).index!, 5);
+ });
+
+ it('goes over what a clip says of the fingers, by its weight', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ s.world.insert(rig.root, Animator, {});
+ const finger = rig.bones.index![0]!;
+ const clip = q.normalize({ w: 0.9, x: 0.3, y: 0.1, z: 0 });
+ const speak = (): void => { s.world.update(finger, Transform, (d: TransformData) => { d.rotation = clip; }); };
+ hold(s.world, rig, { rightPose: 'fist', rightWeight: 0 });
+ speak();
+ await s.tick(2);
+ const left = (s.world.get(finger, Transform) as TransformData).rotation;
+ expect(left.x).toBeCloseTo(clip.x, 4);
+ expect(left.w).toBeCloseTo(clip.w, 4);
+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightWeight = 1; });
+ speak();
+ await s.tick(2);
+ const over = (s.world.get(finger, Transform) as TransformData).rotation;
+ expect(Math.abs(over.x - clip.x) + Math.abs(over.w - clip.w)).toBeGreaterThan(0.05);
+ });
+
+ it('fades away when its pose is taken off, and leaves the hand as it found it when it goes', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig);
+ hold(s.world, rig, { rightPose: 'fist', blendSeconds: 0.05 });
+ await s.tick(30);
+ s.world.update(rig.root, GripPose, (d: GripPoseData) => { d.rightPose = ''; });
+ await s.tick(30);
+ close(tips(s.world, rig).index!, open.index!);
+
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ expect(dist(tips(s.world, rig).index!, open.index!)).toBeGreaterThan(0.02);
+ s.world.remove(rig.root, GripPose);
+ await s.tick(2);
+ close(tips(s.world, rig).index!, open.index!, 4);
+ });
+
+ it('writes nothing once a hand has settled on a rig no clip moves', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ hold(s.world, rig, { rightPose: 'pistol' });
+ await s.tick(3);
+ const spy = vi.spyOn(s.world, 'update');
+ await s.tick(10);
+ expect(spy.mock.calls.filter((c) => c[1] === Transform).length).toBe(0);
+ });
+
+ it('takes a pose of the game\'s own, and leaves a name it does not know alone', async () => {
+ registerGripPose('claw', { thumb: { curl: 1, opposition: 1 }, index: { curl: 1, base: 0 }, middle: { curl: 1, base: 0 }, ring: { curl: 1, base: 0 }, pinky: { curl: 1, base: 0 } });
+ expect(gripPoseNames()).toContain('claw');
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig);
+ hold(s.world, rig, { rightPose: 'claw' });
+ await s.tick(2);
+ expect(tips(s.world, rig).index!.z).toBeLessThan(-0.01);
+
+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
+ const t = scene();
+ const other = buildRig(t.world);
+ hold(t.world, other, { rightPose: 'nothing like this' });
+ await t.tick(2);
+ close(tips(t.world, other).index!, open.index!);
+ warn.mockRestore();
+ });
+
+ it('does nothing while not playing', async () => {
+ const s = scene();
+ const rig = buildRig(s.world);
+ const open = tips(s.world, rig);
+ setEditorMode(true); setPlayMode(false);
+ hold(s.world, rig, { rightPose: 'fist' });
+ await s.tick(2);
+ close(tips(s.world, rig).index!, open.index!, 5);
+ });
+});
diff --git a/tools/check-corpus-authoring.mjs b/tools/check-corpus-authoring.mjs
index 2a623041b..84f67eef1 100644
--- a/tools/check-corpus-authoring.mjs
+++ b/tools/check-corpus-authoring.mjs
@@ -125,6 +125,7 @@ const DISPOSITION = {
Planet: ['owed', 'authored by a game today (star-neighbors, a round planet from its own generator); an example that registers a planet provider is owed'],
MeshInstances: ['derived', 'the foliage and world generation systems write it, one entity a kind and part of a cell; its places are data a system computes, not something a scene authors'],
ParticleCollider: ['owed', 'no example has a GPU emitter bouncing off a shape yet; the particle-gpu scenes are where the first one goes'],
+ GripPose: ['owed', 'no example holds a tool in a named pose yet; a character with something in its hand is where the first one goes'],
VoxelTerrain: ['owed', 'no example digs in a volume yet; a small mine is where the first one goes'],
RoadDecal: ['owed', 'no example lays a road over a terrain yet; wilds is where the first one goes'],
PlantProxy: ['derived', 'the plant interaction makes it for a plant near a contact, and takes it away again; a scene never holds one'],
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 6fb6810d8..5de2699fa 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -4232,6 +4232,8 @@ export const NOTED = {
{"note": "Particles and plain sprites in a Canvas paint by their place in its tree."},
"fix(text): stopping lets go of the labels the engine holds, so the next Play cannot draw them on other entities":
{"note": "Labels no longer come back as black boxes after a change between Plays."},
+ "feat(animation): a rig's hands held in a named pose that fits any skeleton":
+ {"note": "Hands that hold things."},
"feat(sky): clouds that keep time with the game, follow the weather's wind, and have a second layer":
{"note": "Clouds that keep time with the game."},
"feat(voxel): ground that is a volume, carved while the game runs, with collision that follows and saves by chunk":