Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(animation): fingers close onto what they hold, a finger joint is dragged in the viewport, and a clip plays under the grip in edit mode

yhh1a69b102a2dd
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ae379aa6e..de74dc497 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,6 +19,7 @@ published separately; it ships inside the editor.
- **Multi-touch can be turned off.** `Input.setMultiTouch(false)` handles one finger at a time: the first to go down until it lifts, with any finger pressed meanwhile ignored by the UI and the game alike, so it cannot press a button or move a joystick under the first. Turning it off with several fingers down keeps the first and cancels the rest. `isMultiTouchEnabled()` reads it. It is on by default; **Project Settings → Game → Runtime → Multi-touch** sets what a game starts with, and a call to `setMultiTouch` wins over it. It works the same on the web, mini-games and native apps (#117).
- **A Spine skeleton can be part of a UI.** A `SpineAnimation` under a Canvas paints by its place in the Canvas tree, between the controls around it, and takes the UI tree's opacity, hidden state and masks. Under a screen-space Canvas it is drawn with the UI in the screen overlay instead of by the cameras, where before it showed up in the world at the Canvas' pixel coordinates and never in the UI (#116). A non-zero `layer` still holds it over or under the whole UI, and a skeleton outside any Canvas is unchanged. DragonBones armatures under a screen Canvas are drawn in the overlay too.
- **A grip is edited where it is held.** Select an object with a `GripProfile` and its Details section edits the grip while a character in the scene holds it in the edit viewport, without Play: pick the character (it is given a `GripHolder`), create and pick the two palm anchors, see each palm drawn where it goes, switch the leading hand, and change every finger joint by joint in degrees, starting from a grip pose. Anchors are placed once and mirrored in the object's own plane for the other hand (`GripProfile.hand`, `mirror`); fingers said joint by joint (`GripFinger.bend`, `fan`, kept in `mainFingers`/`supportFingers`) are measured on the hand's own axes, so they mirror and fit any rig. The preview is the viewport's Preview FX and gives the bones back when it is turned off.
+- **Fingers close onto what they hold.** A `GripSurface` on a held object, or on anything under it, is a cylinder, capsule, sphere or box sized like a collider; with the `GripProfile`'s new `wrap` (on by default) each finger closes from open toward its pose and stops where a bone meets one, the joints after it closing on, and the thumb stops on it the same way. No physics and no mesh: a distance to a shape, the same every time. In the editor, the grip section lists and creates surfaces and says which fingers touch, the viewport draws them, every finger joint of the previewing character has a handle to drag its bend (one undo step), and a timeline can be put under the grip in the edit viewport, scrubbed or played, the hold going over it as in Play.
- **A material reads its maps through Tiling, Offset and Rotation.** The Model shader's `u_uvTiling`, `u_uvOffset` and `u_uvRotation` move where every map is read (colour, normal, occlusion, metal-roughness, emission), as Unity's Tiling and Offset do; Ground Blend takes them too. A glTF's `KHR_texture_transform` is imported into them, turned into the engine's V-up uv, where before it was dropped with a warning and a tiled texture set was stretched once over each mesh: a three.js port of a container port lost 224 of them. Maps of one material that disagree are all read through the first map's, and the import says so. A cut-out's shadow and depth prepass read through the same transform, so a tiled fence casts the shadow of the fence drawn.
diff --git a/desktop b/desktop
index a0c86c9c1..67d4139e9 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit a0c86c9c14b95d618d9617dbe05661628a55c43f
+Subproject commit 67d4139e9ad01b945e1d955ca1745c074b79cfeb
diff --git a/docs/astro/src/content/docs/animation/grip-pose.mdx b/docs/astro/src/content/docs/animation/grip-pose.mdx
index 64e5b3c31..233d5f5da 100644
--- a/docs/astro/src/content/docs/animation/grip-pose.mdx
+++ b/docs/astro/src/content/docs/animation/grip-pose.mdx
@@ -114,6 +114,7 @@ world.insert(hero, GripHolder, { held: rifle, placement: GripPlacement.Object })
| `mainFingers`, `supportFingers` | Each hand's fingers joint by joint, as JSON of a grip pose (the editor writes it). Over the pose by name where set. |
| `hand` | The hand the anchors were placed for. Held with the other hand leading, both are mirrored. |
| `mirror` | The object's own plane that mirrors them: the one square to its X (the default), Y or Z. |
+| `wrap` | Whether the fingers close onto the object's `GripSurface` shapes, each stopping where it meets one. On by default; with no surface it does nothing. |
It runs after the animator, in one order every frame, so nothing can drive itself round a loop: the clip's pose; then the object, on the leading hand or where the game put it; then the supporting anchor, read off the object as it now stands; then that arm; then the fingers of both. The object never reads the supporting hand. Each frame solves from the clip's pose, never from its own last answer, and nothing reads a clock but the game's: the same inputs and steps give the same bones, bit for bit.
@@ -121,6 +122,22 @@ It runs after the animator, in one order every frame, so nothing can drive itsel
`captureFingerPose(world, handJoint)` takes the turn of every named joint under a hand as it stands, as plain JSON; `registerFingerPose(name, pose)` keeps it for a `GripProfile` to name. Pose the hand with a `GripPose`, a clip or by hand, capture it, and save the JSON with the game. A captured pose names its rig's bones, so it belongs to that rig; a grip pose by name fits any.
+### Closing onto the object
+
+A pose says how far the fingers close; what they close onto is the object's. Put a `GripSurface` on the object, or on anything under it, for each part a hand goes round: a cylinder or capsule for a handle or a barrel, a sphere, a box for a block grip. It is sized in its entity's own units, as a collider is:
+
+| `GripSurface` field | What it does |
+|---|---|
+| `shape` | Cylinder, Capsule, Sphere or Box. |
+| `direction` | The axis a cylinder or capsule runs along: X, Y or Z of the entity. |
+| `center` | Its middle, in the entity's own units. |
+| `radius`, `height` | A cylinder's, capsule's or sphere's radius, and a cylinder's or capsule's whole length. |
+| `size` | A box's size. |
+
+With `wrap` on, each finger closes from open toward the pose's angles, all its joints together; the bone that first meets a surface stops there with every joint before it, and the joints after it go on closing. The thumb comes in along the way it reaches and stops where a bone of it meets one. A finger stops a little short of the shape, by about its own thickness (8% of the index finger's length), and never goes further than the pose: a finger the pose leaves open stays open. A bone that starts inside a surface (a handle pressed into the palm) may go on, but no deeper. It is a distance from a point to a shape, a few hundred a hand: no physics and no mesh, and the same every time.
+
+So a `fist` with `wrap` on closes every finger onto the handle, whatever its thickness, and a `pistol` keeps its index finger out along the frame.
+
### The other hand
Anchors are placed once, for the hand `hand` names. A holder leading with the other hand gets both anchors mirrored in the object's `mirror` plane: their places reflected and their turns made the other hand's palm. Fingers by name, and fingers joint by joint, are a hand's shape rather than its bones, so they mirror too. A captured pose does not: it names one hand's bones.
@@ -131,7 +148,9 @@ Select an object with a `GripProfile`. Its section in Details edits the grip, an
- **Preview.** Pick the character. One without a `GripHolder` gets one, holding this object and placed by it, so it reaches for the object where it stands in the scene. The leading-hand switch sets the holder's `mainHand`, which is how to see the mirrored grip. A palm that cannot reach its anchor says how far short it stops.
- **Anchors.** Pick an entity, or Create one under the object. Select moves to it, for the move and rotate gizmos. The viewport draws each palm where it goes, mirrored when the other hand leads: a frame for the palm, an arrow the way the fingers point, a dashed line the way the palm faces, a stub on the thumb's side; amber for the leading hand, blue for the supporting one.
-- **Fingers.** Start from a grip pose, then change any joint's bend or any finger's lean in degrees. The first change writes the pose out joint by joint into `mainFingers` or `supportFingers`; Reset goes back to the pose itself.
+- **Clip.** Pick a timeline and drag the time, or play it: the character is posed by it in the viewport and the grip goes over it, as in Play. Only the viewport is posed; taking the clip off, turning the preview off or leaving the section gives the pose back.
+- **Fingers.** Start from a grip pose, then change any joint's bend or any finger's lean in degrees. The first change writes the pose out joint by joint into `mainFingers` or `supportFingers`; Reset goes back to the pose itself. In the viewport, each finger joint of the previewing character has a handle (amber on the leading hand, blue on the supporting one): drag it to bend that joint, one undo step a drag.
+- **Close onto surfaces.** The `wrap` switch, which fingers touch a surface now, and the object's surfaces. Create adds a cylinder across the leading palm under its anchor, sized by the character's palm, to fit to the handle with the gizmos and its fields. The viewport draws the surfaces in green with the palms.
The preview is the viewport's **Preview FX**, the same switch spring bones use. It moves only the viewport's bones and the held object, never the scene, and turning it off gives them back as the scene has them.
@@ -141,15 +160,15 @@ Anchors are where a palm goes, not where a hand joint goes, and the arm is solve
### Limits of holding
-- **The fingers do not fit the object.** A pose is set, not found on the surface: they can sink into a thick grip or stop short of a thin one. Fitting them to it is not built.
-- **Fingers are edited as numbers.** There is no dragging a finger joint in the viewport, and a captured pose is not edited in the section.
-- **A preview with no clip.** In the editor the character holds from the pose it stands in in the scene; how the grip sits over a clip is seen in Play.
+- **Fingers close onto simple shapes, not the mesh.** An object's surfaces are what it says they are; a grip of an odd shape is a few of them.
+- **The thumb has no handle in the viewport.** It is reached for, not bent, so it is set by its numbers; a captured pose is not edited in the section.
+- **The editor plays a timeline, not an Animator's state machine,** under the grip; the machine itself runs in Play.
- **Only the supporting arm is bent with `Hand` placement,** and only the leading one is first with `Object`; a shoulder or the spine does not lean to help an arm reach.
- **One object a holder.**
## Limits
-- **It does not touch what is held.** A pose is a shape, not a fit: fingers can sink into a wide grip or stop short of a narrow one. Use `curl` for the object, or a pose of its own.
+- **A `GripPose` on its own does not touch anything.** A pose is a shape, not a fit: fingers can sink into a wide grip or stop short of a narrow one. Holding the object with a `GripHolder` and putting `GripSurface`s on it closes them onto it.
- **A hand's numbers are worked out once,** from how the rig stands when it is first held. A rig whose hand is mid-animation by then has its axes taken from that. In practice they are close.
- **Rigs with a negative scale on an arm are not handled.** A mirrored bone turns the wrong way.
- **Checked on a real rig of one kind.** The poses were tuned on a Mixamo-named character, and the same hand built with bones facing other ways and under other names gives the same fingertips, but no second real rig has been looked at.
diff --git a/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx
index 427080e85..260741015 100644
--- a/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx
+++ b/docs/astro/src/content/docs/zh-cn/animation/grip-pose.mdx
@@ -114,6 +114,7 @@ world.insert(hero, GripHolder, { held: rifle, placement: GripPlacement.Object })
| `mainFingers`、`supportFingers` | 每只手逐个关节的手指,GripPose 的 JSON(编辑器写入)。设置后优先于名字。 |
| `hand` | 锚点是照哪只手摆的。另一只手主握时,两个锚点都会镜像。 |
| `mirror` | 用物体自己的哪个平面镜像:垂直于 X(默认)、Y 或 Z 的那个。 |
+| `wrap` | 手指是否合拢到物体的 `GripSurface` 形状上,每根碰到就停。默认打开;没有表面时不起作用。 |
它在动画系统之后运行,每帧按固定顺序单向计算,所以不会形成循环:动画姿态;然后物体(跟着主手,或者在游戏放的位置);然后从物体当前位置读出副手锚点;然后副手手臂;最后两只手的手指。物体从不读取副手。每帧都从动画姿态开始求解,从不在自己上一帧的结果上继续;除了游戏时钟不读任何时钟。所以同样的输入和步进得到逐位相同的骨骼。
@@ -121,6 +122,22 @@ world.insert(hero, GripHolder, { held: rifle, placement: GripPlacement.Object })
`captureFingerPose(world, handJoint)` 把手下面每个有名字的关节当前的旋转截取成普通 JSON;`registerFingerPose(name, pose)` 保存它,供 `GripProfile` 引用。用 GripPose、动画片段或手动摆好手型后截取下来,把 JSON 和游戏一起保存。截取的姿势用的是骨骼名字,所以属于那套骨架;GripPose 名字则适用于任何骨架。
+### 贴合物体
+
+姿势决定手指合拢多少,合拢到什么上由物体决定。在物体上、或它下面的任何实体上,为手要握住的每个部分加一个 `GripSurface`:握把或枪管用圆柱或胶囊,球形用球,方块握把用盒。尺寸按所在实体自己的单位,和碰撞体一样:
+
+| `GripSurface` 字段 | 作用 |
+|---|---|
+| `shape` | 圆柱、胶囊、球或盒。 |
+| `direction` | 圆柱或胶囊沿哪根轴:实体的 X、Y 或 Z。 |
+| `center` | 中心,按实体自己的单位。 |
+| `radius`、`height` | 圆柱、胶囊或球的半径,以及圆柱或胶囊的总长。 |
+| `size` | 盒的尺寸。 |
+
+`wrap` 打开时,每根手指从张开往姿势的角度合拢,所有关节一起动;最先碰到表面的那节骨头连同它之前的关节停在那里,之后的关节继续合拢。拇指沿它伸出的路径合拢,同样碰到就停。手指停在离形状大约一个手指粗细的地方(食指长度的 8%),也不会比姿势弯得更多:姿势里张开的手指保持张开。一开始就在表面里面的骨头(比如压在掌心的握把)可以继续动,但不会更深。算的只是点到形状的距离,每只手几百次:不用物理、不读网格,每次结果都一样。
+
+所以 `fist` 加上 `wrap`,不管握把多粗,每根手指都会合拢到握把上;`pistol` 则让食指仍然沿着枪身伸直。
+
### 另一只手
锚点只摆一次,按 `hand` 指定的那只手。持有者用另一只手主握时,两个锚点都按物体的 `mirror` 平面镜像:位置反射,朝向变成另一只手的掌心。按名字的手指和逐关节的手指描述的是手的形状而不是骨骼,所以同样会镜像。截取的姿势不会:它记录的是一只手的骨骼。
@@ -131,7 +148,9 @@ world.insert(hero, GripHolder, { held: rifle, placement: GripPlacement.Object })
- **预览。** 选一个角色。没有 `GripHolder` 的会被加上一个,握住这个物体、摆放方式为物体,于是它会去够场景里物体所在的位置。主手开关改的是持有者的 `mainHand`,用它查看镜像后的握持。掌心够不着锚点时,会显示差多远。
- **锚点。** 选一个实体,或者在物体下新建一个。「选中」会切到它,用移动、旋转 gizmo 摆放。视口按实际使用的位置画出每个掌心(另一只手主握时画镜像后的):一个框表示掌心,箭头是手指方向,虚线是掌心朝向,短线在拇指一侧;主手黄色,副手蓝色。
-- **手指。** 从一个 GripPose 开始,再按度数改任意关节的弯曲或任意手指的张开。第一次修改会把姿势逐关节写进 `mainFingers` 或 `supportFingers`;「重置」回到姿势本身。
+- **动画。** 选一个时间轴动画,拖时间条或播放:视口里角色按它摆出姿势,握持叠在上面,和 Play 里一样。只动视口;去掉动画、关掉预览或离开这个分区时姿势复原。
+- **手指。** 从一个 GripPose 开始,再按度数改任意关节的弯曲或任意手指的张开。第一次修改会把姿势逐关节写进 `mainFingers` 或 `supportFingers`;「重置」回到姿势本身。视口里预览角色的每个手指关节上有一个点(主手黄色,副手蓝色),拖动它就弯曲那个关节,一次拖动一步撤销。
+- **贴合表面。** `wrap` 开关、当前贴住表面的手指,以及物体上的表面列表。「新建」会在主手锚点下加一个横过掌心的圆柱,按角色手掌定大小,再用 gizmo 和字段调到握把上。视口里表面用绿色线框和掌心标记一起画出。
预览用的是视口的 **预览特效** 开关,和弹簧骨同一个。它只动视口里的骨骼和被握的物体,不写场景;关掉后它们回到场景里的样子。
@@ -141,15 +160,15 @@ world.insert(hero, GripHolder, { held: rifle, placement: GripPlacement.Object })
### 握持的限制
-- **手指不贴合物体。** 姿势是设定好的,不是在表面上找到的:握把太粗时手指会陷进去,太细时碰不到。贴合还没有做。
-- **手指用数值编辑。** 不能在视口里直接拖手指关节,截取的姿势也不能在面板里编辑。
-- **预览没有动画。** 编辑器里角色从它在场景里的姿势出发去握;握持叠在动画上的效果要进 Play 看。
+- **手指贴合的是简单形状,不是网格。** 物体的表面就是它声明的那些形状;形状奇怪的握把用几个拼起来。
+- **拇指在视口里没有拖动点。** 拇指是按「够向」求的,不是按角度弯的,所以用数值设置;截取的姿势也不能在面板里编辑。
+- **编辑器在握持下面播放的是时间轴,不是 Animator 状态机;** 状态机本身在 Play 里运行。
- **`Hand` 模式只弯副手手臂,`Object` 模式先弯主手;** 肩膀和脊柱不会为了够到而倾斜。
- **一个持有者只能握一个物体。**
## 限制
-- **不接触被握的物体。** 姿势只是手型,不做贴合:握把太粗时手指可能陷进去,太细时可能碰不到。可以调 `curl`,或者给这个物体单独做一个姿势。
+- **单独的 `GripPose` 不接触任何东西。** 姿势只是手型:握把太粗时手指可能陷进去,太细时可能碰不到。用 `GripHolder` 握住物体并在物体上加 `GripSurface`,手指就会贴合上去。
- **手的参数只计算一次,** 基于第一次握持时骨架的状态。如果那时手正处在动画中,轴就按当时的状态算,实际差别不大。
- **不支持手臂带负缩放的骨架。** 镜像骨骼会转错方向。
- **只在一种真实骨架上检查过。** 姿势是在一个 Mixamo 命名的角色上调的。用骨骼朝向不同、名字不同的同一只手测试,指尖位置一致,但还没有看过第二套真实骨架。
diff --git a/docs/astro/src/data/apiStability.generated.json b/docs/astro/src/data/apiStability.generated.json
index 9562e1f8c..7342ffff4 100644
--- a/docs/astro/src/data/apiStability.generated.json
+++ b/docs/astro/src/data/apiStability.generated.json
@@ -3,10 +3,10 @@
"counts": {
"public": 115,
"beta": 178,
- "experimental": 3009,
+ "experimental": 3025,
"deprecated": 3
},
- "total": 3302,
+ "total": 3318,
"subsystems": [
{
"id": "mobile-controls",
diff --git a/docs/astro/src/data/componentDocs.ts b/docs/astro/src/data/componentDocs.ts
index cd114ca55..05bb45c12 100644
--- a/docs/astro/src/data/componentDocs.ts
+++ b/docs/astro/src/data/componentDocs.ts
@@ -208,6 +208,7 @@ export const COMPONENT_DOCS: Record<string, ComponentDoc> = {
Animator: { category: 'animation', summary: 'Runs an animator controller — states, transitions and blend trees — over the entity.', doc: 'animation/overview', anchor: 'animator-state-machine', anchorZh: '动画状态机' },
AnimatorRootMotion: { category: 'animation', summary: 'Publishes how far the active animation asks the character to move, for gameplay to hand to a character controller — a request, never a result.', doc: 'animation/overview', anchor: 'root-motion', anchorZh: '根运动' },
GripProfile: { category: 'animation', summary: 'Where on an object each palm sits, and how its fingers are held there.', doc: 'animation/grip-pose', anchor: 'holding-an-object' },
+ GripSurface: { category: 'animation', summary: 'A cylinder, capsule, sphere or box on a held object that its holder's fingers close onto and stop at.', doc: 'animation/grip-pose', anchor: 'closing-onto-the-object' },
GripHolder: { category: 'animation', summary: 'Holds an object in its GripProfile\'s grip: the arms bent to its anchors, the fingers posed, in one order that cannot loop.', doc: 'animation/grip-pose', anchor: 'holding-an-object' },
GripPose: { category: 'animation', summary: 'Holds a rig\'s hands in a named pose, a pistol grip, a handle, a cup, that fits any rig with fingers.', doc: 'animation/grip-pose' },
SpringBone: { category: 'animation', summary: 'Makes every bone below this root a spring: the chain trails the animated pose by inertia, falls with gravity and is pulled back toward it (hair, tails, cords).', doc: 'animation/spring-bones', anchor: 'the-component', anchorZh: '组件' },
diff --git a/docs/astro/src/data/components.generated.json b/docs/astro/src/data/components.generated.json
index a794a3741..5e4ba42b7 100644
--- a/docs/astro/src/data/components.generated.json
+++ b/docs/astro/src/data/components.generated.json
@@ -2951,6 +2951,12 @@
"XZ (Y)",
"XY (Z)"
]
+ },
+ {
+ "key": "wrap",
+ "type": "boolean",
+ "default": "true",
+ "doc": "Close the fingers onto the GripSurface shapes on this object and under it, each stopping where it meets one. The pose says how far they may close."
}
],
"animatable": [
@@ -2960,6 +2966,73 @@
"mirror"
]
},
+ {
+ "name": "GripSurface",
+ "source": "ts",
+ "fields": [
+ {
+ "key": "shape",
+ "type": "enum",
+ "default": "Cylinder",
+ "doc": "What the fingers close onto.",
+ "values": [
+ "Cylinder",
+ "Capsule",
+ "Sphere",
+ "Box"
+ ]
+ },
+ {
+ "key": "direction",
+ "type": "enum",
+ "default": "Y",
+ "doc": "The axis a cylinder or capsule runs along.",
+ "values": [
+ "X",
+ "Y",
+ "Z"
+ ]
+ },
+ {
+ "key": "center",
+ "type": "Vec3",
+ "default": "(0, 0, 0)",
+ "doc": "Its middle, in this entity's own units."
+ },
+ {
+ "key": "radius",
+ "type": "number",
+ "default": "0.5",
+ "doc": "A cylinder's, capsule's or sphere's radius.",
+ "range": "≥ 0, step 0.001"
+ },
+ {
+ "key": "height",
+ "type": "number",
+ "default": "1",
+ "doc": "A cylinder's or capsule's whole length.",
+ "range": "≥ 0, step 0.001"
+ },
+ {
+ "key": "size",
+ "type": "Vec3",
+ "default": "(1, 1, 1)",
+ "doc": "A box's size."
+ }
+ ],
+ "animatable": [
+ "shape",
+ "direction",
+ "center.x",
+ "center.y",
+ "center.z",
+ "radius",
+ "height",
+ "size.x",
+ "size.y",
+ "size.z"
+ ]
+ },
{
"name": "GroupOpacity",
"source": "cpp",
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 9e7f1af19..0b58b5fe9 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2134 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2137 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -5074,6 +5074,15 @@ supportWeight: number
weight: number
```
+## GripJoint — interface @experimental
+```
+axis: { x: number; y: number; z: number; }
+bone: { x: number; y: number; z: number; }
+entity: number
+finger: string
+joint: number
+```
+
## GripMirror — const @experimental
```
{ readonly X: 0; readonly Y: 1; readonly Z: 2; }
@@ -5127,6 +5136,27 @@ mirror: number
supportAnchor: number
supportFingers: string
supportPose: string
+wrap: boolean
+```
+
+## GripSurface — const @experimental
+```
+ComponentDef<GripSurfaceData>
+```
+
+## GripSurfaceData — interface @experimental
+```
+center: Vec3
+direction: number
+height: number
+radius: number
+shape: number
+size: Vec3
+```
+
+## GripSurfaceShape — const @experimental
+```
+{ readonly Cylinder: 0; readonly Capsule: 1; readonly Sphere: 2; readonly Box: 3; }
```
## GripThumb — interface @experimental
@@ -15940,6 +15970,11 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripHandJoints — function @experimental
+```
+(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[]; } | null
+```
+
## gripHolderStatus — function @experimental
```
(app: App, rig: Entity): { right: boolean; left: boolean; amount: number; held: Entity; short: { main: number; support: number; }; mirrored: boolean; palm: number; } | null
@@ -15970,6 +16005,11 @@ Plugin
(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
```
+## gripSurfaceLines — function @experimental
+```
+(world: World, object: Entity): { x: number; y: number; z: number; }[][]
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 30915ae92..f78c5d966 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.native
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 171 beta · 2154 experimental · 3 deprecated
+Symbols: 115 public · 171 beta · 2157 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -5082,6 +5082,15 @@ supportWeight: number
weight: number
```
+## GripJoint — interface @experimental
+```
+axis: { x: number; y: number; z: number; }
+bone: { x: number; y: number; z: number; }
+entity: number
+finger: string
+joint: number
+```
+
## GripMirror — const @experimental
```
{ readonly X: 0; readonly Y: 1; readonly Z: 2; }
@@ -5135,6 +5144,27 @@ mirror: number
supportAnchor: number
supportFingers: string
supportPose: string
+wrap: boolean
+```
+
+## GripSurface — const @experimental
+```
+ComponentDef<GripSurfaceData>
+```
+
+## GripSurfaceData — interface @experimental
+```
+center: Vec3
+direction: number
+height: number
+radius: number
+shape: number
+size: Vec3
+```
+
+## GripSurfaceShape — const @experimental
+```
+{ readonly Cylinder: 0; readonly Capsule: 1; readonly Sphere: 2; readonly Box: 3; }
```
## GripThumb — interface @experimental
@@ -16201,6 +16231,11 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripHandJoints — function @experimental
+```
+(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[]; } | null
+```
+
## gripHolderStatus — function @experimental
```
(app: App, rig: Entity): { right: boolean; left: boolean; amount: number; held: Entity; short: { main: number; support: number; }; mirrored: boolean; palm: number; } | null
@@ -16231,6 +16266,11 @@ Plugin
(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
```
+## gripSurfaceLines — function @experimental
+```
+(world: World, object: Entity): { x: number; y: number; z: number; }[][]
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index 13087635b..711d0bfda 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.node
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 168 beta · 2133 experimental · 3 deprecated
+Symbols: 115 public · 168 beta · 2136 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5069,6 +5069,15 @@ supportWeight: number
weight: number
```
+## GripJoint — interface @experimental
+```
+axis: { x: number; y: number; z: number; }
+bone: { x: number; y: number; z: number; }
+entity: number
+finger: string
+joint: number
+```
+
## GripMirror — const @experimental
```
{ readonly X: 0; readonly Y: 1; readonly Z: 2; }
@@ -5122,6 +5131,27 @@ mirror: number
supportAnchor: number
supportFingers: string
supportPose: string
+wrap: boolean
+```
+
+## GripSurface — const @experimental
+```
+ComponentDef<GripSurfaceData>
+```
+
+## GripSurfaceData — interface @experimental
+```
+center: Vec3
+direction: number
+height: number
+radius: number
+shape: number
+size: Vec3
+```
+
+## GripSurfaceShape — const @experimental
+```
+{ readonly Cylinder: 0; readonly Capsule: 1; readonly Sphere: 2; readonly Box: 3; }
```
## GripThumb — interface @experimental
@@ -15935,6 +15965,11 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripHandJoints — function @experimental
+```
+(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[]; } | null
+```
+
## gripHolderStatus — function @experimental
```
(app: App, rig: Entity): { right: boolean; left: boolean; amount: number; held: Entity; short: { main: number; support: number; }; mirrored: boolean; palm: number; } | null
@@ -15965,6 +16000,11 @@ Plugin
(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
```
+## gripSurfaceLines — function @experimental
+```
+(world: World, object: Entity): { x: number; y: number; z: number; }[][]
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index 1562b738f..dbb61de73 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/minigame
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2155 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2158 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5069,6 +5069,15 @@ supportWeight: number
weight: number
```
+## GripJoint — interface @experimental
+```
+axis: { x: number; y: number; z: number; }
+bone: { x: number; y: number; z: number; }
+entity: number
+finger: string
+joint: number
+```
+
## GripMirror — const @experimental
```
{ readonly X: 0; readonly Y: 1; readonly Z: 2; }
@@ -5122,6 +5131,27 @@ mirror: number
supportAnchor: number
supportFingers: string
supportPose: string
+wrap: boolean
+```
+
+## GripSurface — const @experimental
+```
+ComponentDef<GripSurfaceData>
+```
+
+## GripSurfaceData — interface @experimental
+```
+center: Vec3
+direction: number
+height: number
+radius: number
+shape: number
+size: Vec3
+```
+
+## GripSurfaceShape — const @experimental
+```
+{ readonly Cylinder: 0; readonly Capsule: 1; readonly Sphere: 2; readonly Box: 3; }
```
## GripThumb — interface @experimental
@@ -16242,6 +16272,11 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripHandJoints — function @experimental
+```
+(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[]; } | null
+```
+
## gripHolderStatus — function @experimental
```
(app: App, rig: Entity): { right: boolean; left: boolean; amount: number; held: Entity; short: { main: number; support: number; }; mirrored: boolean; palm: number; } | null
@@ -16272,6 +16307,11 @@ Plugin
(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
```
+## gripSurfaceLines — function @experimental
+```
+(world: World, object: Entity): { x: number; y: number; z: number; }[][]
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index f3717475e..fd29c5198 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/wechat
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2146 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2149 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -5069,6 +5069,15 @@ supportWeight: number
weight: number
```
+## GripJoint — interface @experimental
+```
+axis: { x: number; y: number; z: number; }
+bone: { x: number; y: number; z: number; }
+entity: number
+finger: string
+joint: number
+```
+
## GripMirror — const @experimental
```
{ readonly X: 0; readonly Y: 1; readonly Z: 2; }
@@ -5122,6 +5131,27 @@ mirror: number
supportAnchor: number
supportFingers: string
supportPose: string
+wrap: boolean
+```
+
+## GripSurface — const @experimental
+```
+ComponentDef<GripSurfaceData>
+```
+
+## GripSurfaceData — interface @experimental
+```
+center: Vec3
+direction: number
+height: number
+radius: number
+shape: number
+size: Vec3
+```
+
+## GripSurfaceShape — const @experimental
+```
+{ readonly Cylinder: 0; readonly Capsule: 1; readonly Sphere: 2; readonly Box: 3; }
```
## GripThumb — interface @experimental
@@ -15968,6 +15998,11 @@ Plugin
(preserveDrawingBuffer?: boolean): WebGLContextAttributes
```
+## gripHandJoints — function @experimental
+```
+(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[]; } | null
+```
+
## gripHolderStatus — function @experimental
```
(app: App, rig: Entity): { right: boolean; left: boolean; amount: number; held: Entity; short: { main: number; support: number; }; mirrored: boolean; palm: number; } | null
@@ -15998,6 +16033,11 @@ Plugin
(app: App, rig: Entity): { right: GripHandStatus; left: GripHandStatus; } | null
```
+## gripSurfaceLines — function @experimental
+```
+(world: World, object: Entity): { x: number; y: number; z: number; }[][]
+```
+
## hasPlayableCta — function @experimental
```
(): boolean
diff --git a/sdk/src/animation/gripContact.ts b/sdk/src/animation/gripContact.ts
new file mode 100644
index 000000000..c252c853a
--- /dev/null
+++ b/sdk/src/animation/gripContact.ts
@@ -0,0 +1,225 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file gripContact.ts
+ * @brief What a held object is to the fingers: simple shapes they close onto, and stop at.
+ *
+ * @details A mesh is too much to ask of every finger every frame, and the part of an object a hand closes around is nearly always a
+ * handle, a barrel, a ball or a block. So the object says so: a `GripSurface` on it, or on anything under it, is a cylinder,
+ * a capsule, a sphere or a box, sized in its own units as a collider is. Fingers close joint by joint until a bone meets one,
+ * and the thumb's reach is moved out of them. Nothing here needs physics: it is a distance from a point to a shape.
+ */
+import { defineComponent, Children, Transform, type ComponentDef, type TransformData } from '../ecs/component';
+import type { World } from '../ecs/world';
+import { q } from '../math/quat';
+import type { Entity } from '../types';
+
+interface Vec3 { x: number; y: number; z: number }
+interface Quat { w: number; x: number; y: number; z: number }
+
+/** The shape a grip surface is. @experimental */
+export const GripSurfaceShape = { Cylinder: 0, Capsule: 1, Sphere: 2, Box: 3 } as const;
+
+/** The fields of the `GripSurface` component. @experimental */
+export interface GripSurfaceData {
+ shape: number;
+ /** The axis a cylinder or capsule runs along, of the entity's own: 0 X, 1 Y, 2 Z. */
+ direction: number;
+ /** Its middle, in the entity's own units. */
+ center: Vec3;
+ /** A cylinder's, capsule's or sphere's radius. */
+ radius: number;
+ /** A cylinder's or capsule's whole length, ends included. */
+ height: number;
+ /** A box's size. */
+ size: Vec3;
+}
+
+/**
+ * A shape on a held object that fingers close onto and stop at: a handle, a barrel, a mug's body. Goes on the object or anything under it;
+ * a `GripProfile` with `wrap` on closes each hand's fingers onto every one there.
+ *
+ * @experimental
+ */
+export const GripSurface: ComponentDef<GripSurfaceData> = /* @__PURE__ */ defineComponent<GripSurfaceData>('GripSurface', {
+ shape: GripSurfaceShape.Cylinder, direction: 1, center: { x: 0, y: 0, z: 0 }, radius: 0.5, height: 1, size: { x: 1, y: 1, z: 1 },
+}, {
+ fields: {
+ shape: { enum: [{ label: 'Cylinder', value: 0 }, { label: 'Capsule', value: 1 }, { label: 'Sphere', value: 2 }, { label: 'Box', value: 3 }], tooltip: 'What the fingers close onto.' },
+ direction: { enum: [{ label: 'X', value: 0 }, { label: 'Y', value: 1 }, { label: 'Z', value: 2 }], tooltip: 'The axis a cylinder or capsule runs along.' },
+ center: { tooltip: 'Its middle, in this entity\'s own units.' },
+ radius: { min: 0, step: 0.001, tooltip: 'A cylinder\'s, capsule\'s or sphere\'s radius.' },
+ height: { min: 0, step: 0.001, tooltip: 'A cylinder\'s or capsule\'s whole length.' },
+ size: { tooltip: 'A box\'s size.' },
+ },
+});
+
+/** A surface as this frame has it, in world units: a shape about `center`, turned by `turn`. @internal */
+export interface PlacedSurface {
+ shape: number;
+ center: Vec3;
+ turn: Quat;
+ /** For a cylinder or capsule, the axis; the other numbers by shape: radius, half its length (a capsule's between its ends' centres), a box's half sizes. */
+ axis: number;
+ radius: number;
+ half: number;
+ halves: Vec3;
+}
+
+const childrenOf = (world: World, e: Entity): Entity[] => (world.tryGet(e, Children) as { entities?: Entity[] } | null)?.entities ?? [];
+const pick = (v: Vec3, axis: number): number => (axis === 0 ? v.x : axis === 2 ? v.z : v.y);
+
+/** Every grip surface on `root` and under it, placed where its transform now is. Transforms must be composed. @internal */
+export function surfacesUnder(world: World, root: Entity): PlacedSurface[] {
+ const out: PlacedSurface[] = [];
+ const queue: Entity[] = [root];
+ for (let i = 0; i < queue.length && i < 256; i++) {
+ const at = queue[i]!;
+ if (world.has(at, GripSurface) && world.has(at, Transform)) {
+ const s = world.get(at, GripSurface) as GripSurfaceData;
+ const t = world.get(at, Transform) as TransformData;
+ const k = { x: Math.abs(t.worldScale.x), y: Math.abs(t.worldScale.y), z: Math.abs(t.worldScale.z) };
+ const turn = q.normalize(t.worldRotation);
+ const c = s.center ?? { x: 0, y: 0, z: 0 };
+ const center = { x: t.worldPosition.x, y: t.worldPosition.y, z: t.worldPosition.z };
+ const off = q.rotate(turn, { x: c.x * k.x, y: c.y * k.y, z: c.z * k.z });
+ center.x += off.x; center.y += off.y; center.z += off.z;
+ const axis = s.direction === 0 || s.direction === 2 ? s.direction : 1;
+ // Round across, so its radius takes the larger of the two scales square to it.
+ const across = axis === 0 ? Math.max(k.y, k.z) : axis === 2 ? Math.max(k.x, k.y) : Math.max(k.x, k.z);
+ const radius = Math.max(0, s.radius) * (s.shape === GripSurfaceShape.Sphere ? Math.max(k.x, k.y, k.z) : across);
+ const length = Math.max(0, s.height) * pick(k, axis);
+ const half = s.shape === GripSurfaceShape.Capsule ? Math.max(0, length / 2 - radius) : length / 2;
+ const size = s.size ?? { x: 1, y: 1, z: 1 };
+ out.push({ shape: s.shape, center, turn, axis, radius, half, halves: { x: Math.abs(size.x) * k.x / 2, y: Math.abs(size.y) * k.y / 2, z: Math.abs(size.z) * k.z / 2 } });
+ }
+ queue.push(...childrenOf(world, at));
+ }
+ return out;
+}
+
+/**
+ * Every grip surface on `object` and under it as lines in the world, where the fingers meet them: what an editor draws. Transforms must be
+ * composed.
+ * @experimental
+ */
+export function gripSurfaceLines(world: World, object: Entity): { x: number; y: number; z: number }[][] {
+ const out: Vec3[][] = [];
+ const N = 24;
+ for (const s of surfacesUnder(world, object)) {
+ const place = (v: Vec3): Vec3 => { const w = q.rotate(s.turn, v); return { x: s.center.x + w.x, y: s.center.y + w.y, z: s.center.z + w.z }; };
+ // A local point with `a` along the shape's axis and (u, v) round it.
+ const onAxis = (a: number, u: number, v: number): Vec3 => (s.axis === 0 ? { x: a, y: u, z: v } : s.axis === 2 ? { x: u, y: v, z: a } : { x: u, y: a, z: v });
+ const ring = (a: number, r: number, from = 0, to = Math.PI * 2, n = N): Vec3[] => Array.from({ length: n + 1 }, (_, i) => {
+ const t = from + ((to - from) * i) / n;
+ return place(onAxis(a, Math.cos(t) * r, Math.sin(t) * r));
+ });
+ if (s.shape === GripSurfaceShape.Box) {
+ const h = s.halves;
+ const c = (x: number, y: number, z: number): Vec3 => place({ x: x * h.x, y: y * h.y, z: z * h.z });
+ out.push([c(-1, -1, -1), c(1, -1, -1), c(1, -1, 1), c(-1, -1, 1), c(-1, -1, -1)], [c(-1, 1, -1), c(1, 1, -1), c(1, 1, 1), c(-1, 1, 1), c(-1, 1, -1)]);
+ for (const [x, z] of [[-1, -1], [1, -1], [1, 1], [-1, 1]] as const) out.push([c(x, -1, z), c(x, 1, z)]);
+ } else if (s.shape === GripSurfaceShape.Sphere) {
+ const r = s.radius;
+ const circle = (f: (t: number) => Vec3): Vec3[] => Array.from({ length: N + 1 }, (_, i) => place(f((Math.PI * 2 * i) / N)));
+ out.push(circle((t) => ({ x: Math.cos(t) * r, y: Math.sin(t) * r, z: 0 })), circle((t) => ({ x: Math.cos(t) * r, y: 0, z: Math.sin(t) * r })), circle((t) => ({ x: 0, y: Math.cos(t) * r, z: Math.sin(t) * r })));
+ } else {
+ out.push(ring(s.half, s.radius), ring(-s.half, s.radius));
+ for (let k = 0; k < 4; k++) {
+ const t = (k * Math.PI) / 2, u = Math.cos(t) * s.radius, v = Math.sin(t) * s.radius;
+ out.push([place(onAxis(-s.half, u, v)), place(onAxis(s.half, u, v))]);
+ }
+ if (s.shape === GripSurfaceShape.Capsule) {
+ // The two ends' half circles, across both ways.
+ for (const end of [1, -1]) {
+ for (const side of [0, 1]) {
+ out.push(Array.from({ length: N / 2 + 1 }, (_, i) => {
+ const t = (Math.PI * i) / (N / 2), a = end * (s.half + Math.sin(t) * s.radius), r = Math.cos(t) * s.radius;
+ return place(side ? onAxis(a, 0, r) : onAxis(a, r, 0));
+ }));
+ }
+ }
+ }
+ }
+ }
+ return out;
+}
+
+/** How far a point in the world is from the surface, outside positive, inside negative. */
+function distanceTo(s: PlacedSurface, p: Vec3): number {
+ const l = q.rotate(q.conjugate(s.turn), { x: p.x - s.center.x, y: p.y - s.center.y, z: p.z - s.center.z });
+ if (s.shape === GripSurfaceShape.Sphere) return Math.hypot(l.x, l.y, l.z) - s.radius;
+ if (s.shape === GripSurfaceShape.Box) {
+ const dx = Math.abs(l.x) - s.halves.x, dy = Math.abs(l.y) - s.halves.y, dz = Math.abs(l.z) - s.halves.z;
+ return Math.hypot(Math.max(dx, 0), Math.max(dy, 0), Math.max(dz, 0)) + Math.min(Math.max(dx, dy, dz), 0);
+ }
+ const along = pick(l, s.axis);
+ const round = s.axis === 0 ? Math.hypot(l.y, l.z) : s.axis === 2 ? Math.hypot(l.x, l.y) : Math.hypot(l.x, l.z);
+ if (s.shape === GripSurfaceShape.Capsule) {
+ const past = Math.abs(along) - s.half;
+ return Math.hypot(round, Math.max(past, 0)) - s.radius;
+ }
+ const a = round - s.radius, b = Math.abs(along) - s.half;
+ return Math.min(Math.max(a, b), 0) + Math.hypot(Math.max(a, 0), Math.max(b, 0));
+}
+
+/** How far a point is from the nearest of the surfaces. Infinite with none. @internal */
+export function surfaceDistance(surfaces: readonly PlacedSurface[], p: Vec3): number {
+ let d = Infinity;
+ for (const s of surfaces) { const v = distanceTo(s, p); if (v < d) d = v; }
+ return d;
+}
+
+/** The way out of the nearest surface at a point, by the change of distance across `step`. @internal */
+export function surfaceOutward(surfaces: readonly PlacedSurface[], p: Vec3, step: number): Vec3 {
+ const at = (x: number, y: number, z: number): number => surfaceDistance(surfaces, { x: p.x + x, y: p.y + y, z: p.z + z });
+ const g = { x: at(step, 0, 0) - at(-step, 0, 0), y: at(0, step, 0) - at(0, -step, 0), z: at(0, 0, step) - at(0, 0, -step) };
+ const l = Math.hypot(g.x, g.y, g.z);
+ return l < 1e-12 ? { x: 0, y: 0, z: 0 } : { x: g.x / l, y: g.y / l, z: g.z / l };
+}
+
+/** How near a bone from `a` to `b` comes to any surface, its line measured at a few points along it. @internal */
+export function boneDistance(surfaces: readonly PlacedSurface[], a: Vec3, b: Vec3): number {
+ let d = Infinity;
+ for (let i = 0; i <= 4; i++) {
+ const t = i / 4;
+ const v = surfaceDistance(surfaces, { x: a.x + (b.x - a.x) * t, y: a.y + (b.y - a.y) * t, z: a.z + (b.z - a.z) * t });
+ if (v < d) d = v;
+ }
+ return d;
+}
+
+/**
+ * A finger closed from `start` toward `limit`, every free joint together, until its bones meet a surface: a bone that touches holds itself
+ * and every joint before it where it touched, and those after it go on closing. `touching(bends, from)` answers the first bone from `from` on
+ * that touches with these bends, or -1. Answers the bends, and whether any bone touched.
+ * @internal
+ */
+export function closeFinger(start: readonly number[], limit: readonly number[], touching: (bends: readonly number[], from: number) => number): { bends: number[]; touched: boolean } {
+ const n = start.length;
+ const bends = [...start];
+ let free = 0, touched = false;
+ const STEPS = 40;
+ let t = 0;
+ const set = (to: number): void => { for (let j = free; j < n; j++) bends[j] = start[j]! + (limit[j]! - start[j]!) * to; };
+ while (free < n && t < 1) {
+ const next = Math.min(1, t + 1 / STEPS);
+ set(next);
+ if (touching(bends, free) < 0) { t = next; continue; }
+ // Between the last clear step and this one: halve to where a bone meets it.
+ let lo = t, hi = next;
+ for (let i = 0; i < 10; i++) {
+ const mid = (lo + hi) / 2;
+ set(mid);
+ if (touching(bends, free) >= 0) hi = mid; else lo = mid;
+ }
+ set(hi);
+ const at = touching(bends, free);
+ set(lo);
+ touched = true;
+ // The bone that touched holds, and so does every joint before it; those after it go on from here.
+ free = Math.max(free, at) + 1;
+ t = lo;
+ }
+ return { bends, touched };
+}
diff --git a/sdk/src/animation/gripHold.ts b/sdk/src/animation/gripHold.ts
index 971e28860..326f8577e 100644
--- a/sdk/src/animation/gripHold.ts
+++ b/sdk/src/animation/gripHold.ts
@@ -44,6 +44,8 @@ export interface GripProfileData {
hand: number;
/** The plane of the object's own that mirrors them: the one square to its X, Y or Z. */
mirror: number;
+ /** Whether the fingers close onto the object's grip surfaces (GripSurface, on it or under it), each stopping where it meets one. */
+ wrap: boolean;
}
/** The plane an object's anchors are mirrored in, by the axis square to it. @experimental */
@@ -55,7 +57,7 @@ export const GripMirror = { X: 0, Y: 1, Z: 2 } as const;
* @experimental
*/
export const GripProfile: ComponentDef<GripProfileData> = /* @__PURE__ */ defineComponent<GripProfileData>('GripProfile', {
- mainAnchor: 0 as Entity, supportAnchor: 0 as Entity, mainPose: 'handle', supportPose: 'support', mainFingers: '', supportFingers: '', hand: 0, mirror: 0,
+ mainAnchor: 0 as Entity, supportAnchor: 0 as Entity, mainPose: 'handle', supportPose: 'support', mainFingers: '', supportFingers: '', hand: 0, mirror: 0, wrap: true,
}, {
entityFields: ['mainAnchor', 'supportAnchor'],
fields: {
@@ -67,6 +69,7 @@ export const GripProfile: ComponentDef<GripProfileData> = /* @__PURE__ */ define
supportAnchor: { tooltip: 'Where the supporting palm sits. Empty holds this in one hand.' },
mainPose: { tooltip: 'The leading hand\'s fingers: a captured finger pose, or a grip pose (pistol, handle, support, ...).' },
supportPose: { tooltip: 'The supporting hand\'s fingers.' },
+ wrap: { tooltip: 'Close the fingers onto the GripSurface shapes on this object and under it, each stopping where it meets one. The pose says how far they may close.' },
},
});
@@ -469,6 +472,7 @@ function hold(app: App, world: World, known: Map<Entity, HolderState>, rig: Enti
// The fingers last: a pose said joint by joint on the profile, a captured one by joint name, or a grip pose by name; the last two kinds are handed
// to the grip-pose system, which eases them on over whatever else holds that hand.
const now: HolderState['asked'] = {};
+ const wrapOn = held && (world.get(held, GripProfile) as GripProfileData).wrap !== false ? { wrap: held } : {};
for (const { arm, pose, inline, weight } of fingerNames) {
const side = arm === state.arms.right ? 'right' : 'left';
const joints = inlineFingers(inline);
@@ -476,9 +480,9 @@ function hold(app: App, world: World, known: Map<Entity, HolderState>, rig: Enti
if (captured) {
for (const [joint, turn] of captured) { wanted.set(joint, turn); supportWeights.set(joint, weight); }
} else if (joints) {
- now[side] = { key: inline, pose: joints, weight: 0, share: weight };
+ now[side] = { key: inline, pose: joints, weight: 0, share: weight, ...wrapOn };
} else if (gripPose(pose)) {
- now[side] = { key: pose, pose: gripPose(pose)!, weight: 0, share: weight };
+ now[side] = { key: pose, pose: gripPose(pose)!, weight: 0, share: weight, ...wrapOn };
}
}
// Let go of, the last grip fades out with the hold; held, what is asked now.
@@ -487,7 +491,7 @@ function hold(app: App, world: World, known: Map<Entity, HolderState>, rig: Enti
const hands = { blendSeconds: data.blendSeconds, rightHand: data.rightHand, leftHand: data.leftHand } as Parameters<typeof askHeldHands>[2];
for (const side of ['right', 'left'] as const) {
const a = state.asked[side];
- if (a) hands[side] = { key: a.key, pose: a.pose, weight: state.amount * a.share };
+ if (a) hands[side] = { key: a.key, pose: a.pose, weight: state.amount * a.share, ...(a.wrap !== undefined ? { wrap: a.wrap } : {}) };
}
if (hands.right || hands.left) askHeldHands(app, rig, hands);
} else {
diff --git a/sdk/src/animation/gripPose.ts b/sdk/src/animation/gripPose.ts
index 32114d20b..0588423bb 100644
--- a/sdk/src/animation/gripPose.ts
+++ b/sdk/src/animation/gripPose.ts
@@ -9,11 +9,11 @@
* The axes are worked out from each rig's own hand, once, from where its joints stand: which way the palm faces, which way each
* finger points. What runs each frame is then a few quaternions a joint, and nothing at all once a hand has settled on a rig no clip moves.
*
- * It does not know what is held. Putting the hand on an object is the attachment's business, and fitting the fingers to its
- * surface a later step's.
+ * It does not know what is held. Putting the hand on an object is the attachment's business; a holder can ask for the fingers
+ * to close onto the object's grip surfaces, which each finger then does joint by joint, the pose saying how far it may go.
*/
import type { App } from '../app/app';
-import { defineComponent, getComponent, Children, Name, Transform, type ComponentDef, type TransformData } from '../ecs/component';
+import { defineComponent, getComponent, Children, Name, Parent, Transform, type ComponentDef, type TransformData } from '../ecs/component';
import { defineSystem, GetWorld, Schedule } from '../ecs/system';
import { Res, Time, type TimeData } from '../ecs/resource';
import type { World } from '../ecs/world';
@@ -24,6 +24,7 @@ import { log } from '../util/logger';
import { q } from '../math/quat';
import type { Entity } from '../types';
import { leanQuat } from './quatMix';
+import { boneDistance, closeFinger, surfacesUnder, type PlacedSurface } from './gripContact';
/** How one finger is held. @experimental */
export interface GripFinger {
@@ -167,6 +168,8 @@ const childrenOf = (world: World, e: Entity): Entity[] => (world.tryGet(e, Child
interface Joint {
entity: Entity;
rest: Quat;
+ /** Where it stands from its parent as the rig rests. */
+ at: Vec3;
curlAxis: Vec3;
/** On a finger's first joint. */
spreadAxis?: Vec3;
@@ -175,7 +178,13 @@ interface Joint {
}
/** A finger's joints, knuckle first. Those it has no joint for are left out. */
-type Finger = { name: FingerName; joints: Joint[]; fan: number };
+type Finger = {
+ name: FingerName; joints: Joint[]; fan: number;
+ /** Where its first joint's parent stands in the hand, and its turn: the hand itself, or a bone in the palm. */
+ root: { p: Vec3; r: Quat };
+ /** Its tip, from the last joint a pose bends. */
+ tip: Vec3;
+};
/** Numbers a hand is held by, kept flat so two poses can be eased between: each finger's three joint bends and its lean (radians), then the thumb's reach, how much of it is for the index tip, and its gap. */
const PER_FINGER = 4;
@@ -270,6 +279,13 @@ interface HandState {
/** The numbers now, and how much of them is applied now: both eased. */
current: Float64Array;
wanted: Float64Array;
+ /** The pose's own numbers: what `wanted` is, until fingers closing onto a surface stop short of them. */
+ posed: Float64Array;
+ /** The fingers whose bones touch a surface, and the surfaces and the hand as this frame has them while the hand closes onto any. */
+ touching: FingerName[];
+ thumbTouching: boolean;
+ surfaces: PlacedSurface[] | null;
+ world: { p: Vec3; r: Quat; s: number };
weight: number;
/** How fast each number is moving, and the weight after them. */
speed: Float64Array;
@@ -385,7 +401,8 @@ function buildHand(world: World, hand: Entity, side: 'right' | 'left'): HandStat
const joints: Joint[] = moved.map((entity, i) => {
const toJoint = q.conjugate(turn[i]!);
const rest = q.normalize((world.get(entity, Transform) as TransformData).rotation);
- const joint: Joint = { entity, rest: { ...rest }, curlAxis: norm(q.rotate(toJoint, bend)), written: { ...rest } };
+ const at0 = (world.get(entity, Transform) as TransformData).position;
+ const joint: Joint = { entity, rest: { ...rest }, at: { x: at0.x, y: at0.y, z: at0.z }, curlAxis: norm(q.rotate(toJoint, bend)), written: { ...rest } };
if (i === 0) {
joint.spreadAxis = norm(q.rotate(toJoint, palm));
// Toward the index finger for a positive spread.
@@ -393,7 +410,7 @@ function buildHand(world: World, hand: Entity, side: 'right' | 'left'): HandStat
}
return joint;
});
- fingers.push({ name, joints, fan: FAN[name] });
+ fingers.push({ name, joints, fan: FAN[name], root: rootOf(world, hand, chain[0]!), tip: tipOf(world, chain) });
}
const index = chains.index ?? [], middle = chains.middle ?? [];
const lengthOf = (chain: Entity[]): number => {
@@ -404,11 +421,37 @@ function buildHand(world: World, hand: Entity, side: 'right' | 'left'): HandStat
};
const reachUnit = lengthOf(index) || lengthOf(middle) || 0;
return {
- hand, fingers, missing, current: new Float64Array(NUMBERS), wanted: new Float64Array(NUMBERS), weight: 0, speed: new Float64Array(NUMBERS + 1), settled: false, pose: '', trigger: 0,
+ hand, fingers, missing, current: new Float64Array(NUMBERS), wanted: new Float64Array(NUMBERS), posed: new Float64Array(NUMBERS), touching: [], thumbTouching: false, surfaces: null,
+ world: { p: { x: 0, y: 0, z: 0 }, r: { w: 1, x: 0, y: 0, z: 0 }, s: 1 }, weight: 0, speed: new Float64Array(NUMBERS + 1), settled: false, pose: '', trigger: 0,
thumb: thumbReach(world, chains.thumb ?? []), palm, across, reachUnit, index, middle,
};
}
+/** Where a finger's first joint's parent stands in the hand as the rig rests, and its turn: through any bone in the palm between. */
+function rootOf(world: World, hand: Entity, first: Entity): { p: Vec3; r: Quat } {
+ const up: Entity[] = [];
+ for (let at = (world.tryGet(first, Parent) as { entity?: Entity } | null)?.entity, guard = 0; at !== undefined && at !== hand && guard < 8; guard++) {
+ up.unshift(at);
+ at = (world.tryGet(at, Parent) as { entity?: Entity } | null)?.entity;
+ }
+ let p: Vec3 = { x: 0, y: 0, z: 0 }, r: Quat = { w: 1, x: 0, y: 0, z: 0 };
+ for (const e of up) {
+ const local = world.get(e, Transform) as TransformData;
+ p = add(p, q.rotate(r, local.position));
+ r = q.normalize(q.mul(r, local.rotation));
+ }
+ return { p, r };
+}
+
+/** A finger's tip from its last bent joint: the next bone where the chain has one, else most of the bone before, as fingers end. */
+function tipOf(world: World, chain: Entity[]): Vec3 {
+ if (chain.length > 3) return { ...(world.get(chain[3]!, Transform) as TransformData).position };
+ const last = chain[chain.length - 1]!;
+ const local = world.get(last, Transform) as TransformData;
+ // The bone before, seen from the last joint: its own place from its parent, turned into its frame.
+ return scale(q.rotate(q.conjugate(q.normalize(local.rotation)), local.position), 0.8);
+}
+
/** The thumb's chain as it rests: null with fewer than three bones, since two cannot both reach and curl. */
function thumbReach(world: World, chain: Entity[]): ThumbReach | null {
if (chain.length < 3) return null;
@@ -434,9 +477,9 @@ function chainTip(lengths: readonly number[], bend: number): [number, number] {
/**
* The thumb's joints turned so its tip lands on `goal`, its bones curling toward `inward`: the bend found by halving, the plane by where the goal is.
- * Answers each joint's turn from the hand.
+ * Answers each joint's turn from the hand; `points`, where given, gets where its root, its joints and its tip then stand in the hand.
*/
-function reachThumb(reach: ThumbReach, goal: Vec3, inward: Vec3): [Quat, Quat, Quat] {
+function reachThumb(reach: ThumbReach, goal: Vec3, inward: Vec3, points?: Vec3[]): [Quat, Quat, Quat] {
const toGoal = sub(goal, reach.root);
const total = reach.lengths[0] + reach.lengths[1] + reach.lengths[2];
const want = Math.min(Math.max(Math.hypot(...xyz(toGoal)), total * 0.35), total - 1e-6);
@@ -454,6 +497,11 @@ function reachThumb(reach: ThumbReach, goal: Vec3, inward: Vec3): [Quat, Quat, Q
const v = Math.hypot(...xyz(off)) > 1e-6 ? norm(off) : norm(cross(u, { x: 0, y: 0, z: 1 }));
const along = (angle: number): Vec3 => add(scale(u, Math.cos(angle)), scale(v, Math.sin(angle)));
const dirs = [along(-lean), along(bend - lean), along(2 * bend - lean)];
+ if (points) {
+ let at = reach.root;
+ points.push(at);
+ for (let i = 0; i < 3; i++) { at = add(at, scale(dirs[i]!, reach.lengths[i]!)); points.push(at); }
+ }
// Each bone turned the least way from where its parent left it: no twist added along it.
const turns: Quat[] = [];
let parentNow = reach.parentTurn, parentRest = reach.parentTurn;
@@ -544,21 +592,99 @@ function ease(value: number, wanted: number, state: HandState, slot: number, ble
return next;
}
-/** Holds one hand a frame: eases its numbers on, and turns each joint to them over what is there. `poseName` tells one pose from the next; `def` is it. */
-function holdHand(world: World, state: HandState, poseName: string, def: GripPoseDef | null, weight: number, trigger: number, blend: number, dt: number, moved: boolean): void {
+/** A finger's joints and tip in the world, its joints bent by `bends` and its first leaning by `lean`, from the hand as this frame has it. */
+function fingerInWorld(state: HandState, finger: Finger, bends: readonly number[], lean: number): Vec3[] {
+ let p = finger.root.p, r = finger.root.r;
+ const out: Vec3[] = [];
+ const w = state.world;
+ const toWorld = (v: Vec3): Vec3 => add(w.p, q.rotate(w.r, scale(v, w.s)));
+ for (let i = 0; i < finger.joints.length; i++) {
+ const joint = finger.joints[i]!;
+ p = add(p, q.rotate(r, joint.at));
+ let turn = axisAngle(joint.curlAxis, bends[i] ?? 0);
+ if (joint.spreadAxis && lean !== 0) turn = q.mul(axisAngle(joint.spreadAxis, lean), turn);
+ r = q.normalize(q.mul(r, q.mul(joint.rest, turn)));
+ out.push(toWorld(p));
+ }
+ out.push(toWorld(add(p, q.rotate(r, finger.tip))));
+ return out;
+}
+
+/** A finger's bone from the line through it to its skin: a share of the index finger's length, as fingers are. */
+const THICKNESS = 0.08;
+
+/**
+ * Each finger closed from open toward the pose, joint by joint, until it meets a surface: what `wanted` asks of its bends then. Answers whether
+ * that changed anything. The thumb is moved out of the surfaces where it reaches (`thumbTurns`).
+ */
+function closeOnto(world: World, state: HandState, surfaces: PlacedSurface[]): boolean {
+ const t = world.get(state.hand, Transform) as TransformData;
+ state.world = { p: { ...t.worldPosition }, r: q.normalize(t.worldRotation), s: Math.abs(t.worldScale.x) || 1 };
+ state.surfaces = surfaces;
+ const skin = state.reachUnit * THICKNESS * state.world.s;
+ let changed = false;
+ const touching: FingerName[] = [];
+ for (const finger of state.fingers) {
+ if (finger.name === 'thumb') continue;
+ const at = FINGERS.indexOf(finger.name) * PER_FINGER;
+ const n = finger.joints.length;
+ const limit = Array.from({ length: n }, (_, i) => state.posed[at + i]!);
+ const lean = state.posed[at + 3]!;
+ const start = limit.map((b) => Math.min(0, b));
+ // A bone already within its thickness of a surface as the finger opens (a handle pressed into the palm) may go on, but no deeper than it started.
+ const open = fingerInWorld(state, finger, start, lean);
+ const reach = Array.from({ length: n }, (_, k) => Math.min(skin, boneDistance(surfaces, open[k]!, open[k + 1]!) - 1e-6));
+ const { bends, touched } = closeFinger(start, limit, (b, from) => {
+ const points = fingerInWorld(state, finger, b, lean);
+ for (let k = from; k < n; k++) if (boneDistance(surfaces, points[k]!, points[k + 1]!) <= reach[k]!) return k;
+ return -1;
+ });
+ if (touched) touching.push(finger.name);
+ for (let i = 0; i < n; i++) {
+ if (Math.abs(state.wanted[at + i]! - bends[i]!) > 1e-5) { state.wanted[at + i] = bends[i]!; changed = true; }
+ }
+ }
+ state.touching = touching;
+ return changed;
+}
+
+/** Whether every joint still stands as this last wrote it: something else (a clip, an editor sampling one) may have turned them since. */
+function standsAsWritten(world: World, state: HandState): boolean {
+ for (const finger of state.fingers) {
+ for (const joint of finger.joints) {
+ if (!world.valid(joint.entity) || !same((world.get(joint.entity, Transform) as TransformData).rotation, joint.written)) return false;
+ }
+ }
+ return true;
+}
+
+/**
+ * Holds one hand a frame: eases its numbers on, and turns each joint to them over what is there. `poseName` tells one pose from the next; `def` is it.
+ * `surfaces`, where there are any, are what the fingers close onto.
+ */
+function holdHand(world: World, state: HandState, poseName: string, def: GripPoseDef | null, weight: number, trigger: number, blend: number, dt: number, moved: boolean, surfaces: PlacedSurface[] | null): void {
const pose = poseName ? def : null;
if (poseName !== state.pose) {
if (poseName && !pose) log.warn('animation', `there is no grip pose called "${poseName}": the hand is left alone`);
state.pose = poseName;
// A new pose eases on from where the hand is; a hand let go keeps its numbers and fades by its weight.
- if (pose) numbersOf(pose, state.wanted);
+ if (pose) { numbersOf(pose, state.posed); state.wanted.set(state.posed); }
+ state.settled = false;
+ }
+ if (pose && surfaces && surfaces.length) {
+ if (closeOnto(world, state, surfaces)) state.settled = false;
+ } else if (state.surfaces) {
+ // Nothing to close onto any more: back to the pose itself.
+ state.surfaces = null;
+ state.touching = [];
+ if (pose) state.wanted.set(state.posed);
state.settled = false;
}
const wantedWeight = pose ? clamp01(weight) : 0;
const press = pose?.trigger ? clamp01(trigger) : 0;
if (press !== state.trigger) { state.trigger = press; state.settled = false; }
if (wantedWeight !== state.weight) state.settled = false;
- if (state.settled && !moved) return;
+ if (state.settled && !moved && standsAsWritten(world, state)) return;
let arrived = true;
for (let i = 0; i < NUMBERS; i++) {
@@ -628,8 +754,28 @@ function thumbTurns(world: World, state: HandState): [Quat, Quat, Quat] | null {
const on = add(scale(fingers, 1 - toIndex), scale(index, toIndex));
const thickness = state.reachUnit * 0.16;
const contact = add(on, scale(state.palm, thickness + state.current[GAP]! * state.reachUnit));
- const goal = add(scale(reach.restTip, 1 - amount), scale(contact, amount));
- return reachThumb(reach, goal, norm(add(scale(state.across, -0.8), scale(state.palm, 0.6))));
+ const inward = norm(add(scale(state.across, -0.8), scale(state.palm, 0.6)));
+ const goalAt = (a: number): Vec3 => add(scale(reach.restTip, 1 - a), scale(contact, a));
+ state.thumbTouching = false;
+ if (!state.surfaces) return reachThumb(reach, goalAt(amount), inward);
+ // Over a surface, the thumb comes in along the same way and stops where a bone of it meets one: a goal on the far side of a handle, past
+ // where the thumb can reach, would otherwise straighten it through the handle.
+ const w = state.world, surfaces = state.surfaces;
+ const skin = state.reachUnit * THICKNESS * w.s;
+ const bonesAt = (a: number): Vec3[] => {
+ const points: Vec3[] = [];
+ reachThumb(reach, goalAt(a), inward, points);
+ return points.map((p) => add(w.p, q.rotate(w.r, scale(p, w.s))));
+ };
+ const open = bonesAt(0);
+ const limit = [0, 1, 2].map((k) => Math.min(skin, boneDistance(surfaces, open[k]!, open[k + 1]!) - 1e-6));
+ const { bends, touched } = closeFinger([0], [amount], (a, from) => {
+ if (from > 0) return -1;
+ const points = bonesAt(a[0]!);
+ return [0, 1, 2].some((k) => boneDistance(surfaces, points[k]!, points[k + 1]!) <= limit[k]!) ? 0 : -1;
+ });
+ state.thumbTouching = touched;
+ return reachThumb(reach, goalAt(bends[0]!), inward);
}
function letGo(world: World, state: HandState | null): void {
@@ -647,7 +793,11 @@ function letGo(world: World, state: HandState | null): void {
* A hand pose something else asks of a rig each frame: a holder handing the fingers its grip. Over the rig's own GripPose for that hand, if it has
* one. `key` tells one pose from the next. @internal
*/
-export interface HeldHand { key: string; pose: GripPoseDef | null; weight: number }
+export interface HeldHand {
+ key: string; pose: GripPoseDef | null; weight: number;
+ /** An object whose grip surfaces, on it and under it, the fingers close onto, the pose saying how far they may. */
+ wrap?: Entity;
+}
/** @internal */
export interface HeldHands { right?: HeldHand; left?: HeldHand; blendSeconds: number; rightHand: string; leftHand: string }
@@ -682,6 +832,29 @@ export interface GripHandStatus {
pose: string;
/** How much of it applies now. */
weight: number;
+ /** The fingers whose bones touch a surface they were closed onto. */
+ touching: string[];
+}
+
+/** One joint a pose bends, and the axis it bends about in its own frame: a positive bend turns it that way. @experimental */
+export interface GripJoint {
+ finger: string;
+ /** 0 the knuckle, 1 the middle joint, 2 the last. */
+ joint: number;
+ entity: Entity;
+ axis: { x: number; y: number; z: number };
+ /** The bone it turns, to the next joint or the fingertip, in its own frame: where a handle shows which way a bend moves it. */
+ bone: { x: number; y: number; z: number };
+}
+
+/** The joints each hand of a rig bends, once the grip-pose system has seen it: what an editor puts a handle on. @experimental */
+export function gripHandJoints(app: App, rig: Entity): { right: GripJoint[]; left: GripJoint[] } | null {
+ const state = rigs.get(app)?.get(rig);
+ if (!state) return null;
+ const of = (h: HandState | null): GripJoint[] => (h ? h.fingers.filter((f) => f.name !== 'thumb').flatMap((f) => f.joints.map((j, i) => ({
+ finger: f.name, joint: i, entity: j.entity, axis: { ...j.curlAxis }, bone: { ...(f.joints[i + 1]?.at ?? f.tip) },
+ }))) : []);
+ return { right: of(state.right), left: of(state.left) };
}
const rigs = new WeakMap<App, Map<Entity, RigState>>();
@@ -692,6 +865,7 @@ export function gripPoseStatus(app: App, rig: Entity): { right: GripHandStatus;
if (!state) return null;
const of = (h: HandState | null): GripHandStatus => ({
hand: h?.hand ?? null, joints: h ? h.fingers.reduce((n, finger) => n + finger.joints.length, 0) : 0, missing: h ? [...h.missing] : [...FINGERS], pose: h?.pose ?? '', weight: h?.weight ?? 0,
+ touching: h ? [...(h.thumbTouching && h.surfaces ? ['thumb'] : []), ...h.touching] : [],
});
return { right: of(state.right), left: of(state.left) };
}
@@ -714,14 +888,24 @@ export function installGripPose(app: App): void {
if (state.byComponent) { letGo(world, state.right); letGo(world, state.left); known.delete(rig); }
}
const rigsNow = new Set<Entity>([...comps, ...wanted.keys(), ...known.keys()]);
+ // Closing onto a surface measures where the hand and the object are this frame, after the hold has moved both.
+ const placed = new Map<Entity, PlacedSurface[]>();
+ let composed = false;
+ const surfacesOf = (object: Entity | undefined): PlacedSurface[] | null => {
+ if (object === undefined || !world.valid(object)) return null;
+ if (!composed) { world.ensureTransformsComposed(); composed = true; }
+ let found = placed.get(object);
+ if (!found) { found = surfacesUnder(world, object); placed.set(object, found); }
+ return found;
+ };
for (const rig of rigsNow) {
if (!world.valid(rig)) continue;
const data = comps.has(rig) ? world.get(rig, GripPose) as GripPoseData : null;
const ask = wanted.get(rig);
const rightHandPath = data?.rightHand || ask?.rightHand || '', leftHandPath = data?.leftHand || ask?.leftHand || '';
- const handOf = (side: 'right' | 'left'): { key: string; def: GripPoseDef | null; weight: number; trigger: number } => {
+ const handOf = (side: 'right' | 'left'): { key: string; def: GripPoseDef | null; weight: number; trigger: number; wrap?: Entity } => {
const held = ask?.[side];
- if (held) return { key: held.key, def: held.pose, weight: held.weight, trigger: data ? (side === 'right' ? data.rightTrigger : data.leftTrigger) : 0 };
+ if (held) return { key: held.key, def: held.pose, weight: held.weight, trigger: data ? (side === 'right' ? data.rightTrigger : data.leftTrigger) : 0, wrap: held.wrap };
if (data) {
const name = side === 'right' ? data.rightPose : data.leftPose;
return { key: name, def: poses.get(name) ?? null, weight: side === 'right' ? data.rightWeight : data.leftWeight, trigger: side === 'right' ? data.rightTrigger : data.leftTrigger };
@@ -752,7 +936,7 @@ export function installGripPose(app: App): void {
else if (state[side]!.missing.length) log.warn('animation', `the ${side} hand of "${nameOf(world, rig)}" has no bones for its ${state[side]!.missing.join(', ')}: a pose does without them`);
}
const hand = state[side];
- if (hand) holdHand(world, hand, poseName, hands[side].def, hands[side].weight, hands[side].trigger, blend, time.delta, moved);
+ if (hand) holdHand(world, hand, poseName, hands[side].def, hands[side].weight, hands[side].trigger, blend, time.delta, moved, surfacesOf(hands[side].wrap));
}
// Hands no one asks for any more, faded all the way out: nothing is left to hold.
if (!data && !ask && (!state.right || state.right.weight === 0) && (!state.left || state.left.weight === 0)) known.delete(rig);
diff --git a/sdk/src/animation/index.ts b/sdk/src/animation/index.ts
index 7bb3063cd..828d80f27 100644
--- a/sdk/src/animation/index.ts
+++ b/sdk/src/animation/index.ts
@@ -82,9 +82,10 @@ export {
type AnimatorPathEvalResult,
} from './Animator';
export {
- GripPose, registerGripPose, gripPose, gripPoseNames, gripPoseStatus, gripPoseByJoint, gripPoseFromJson,
- type GripPoseData, type GripPoseDef, type GripFinger, type GripThumb, type GripHandStatus,
+ GripPose, registerGripPose, gripPose, gripPoseNames, gripPoseStatus, gripPoseByJoint, gripPoseFromJson, gripHandJoints,
+ type GripPoseData, type GripPoseDef, type GripFinger, type GripThumb, type GripHandStatus, type GripJoint,
} from './gripPose';
+export { GripSurface, GripSurfaceShape, gripSurfaceLines, type GripSurfaceData } from './gripContact';
export {
GripProfile, GripHolder, GripHand, GripPlacement, GripMirror, releaseGripHolds, registerFingerPose, fingerPose, captureFingerPose, gripHolderStatus, slerpQuat,
type GripProfileData, type GripHolderData, type FingerPoseDef,
diff --git a/sdk/src/core-content.ts b/sdk/src/core-content.ts
index 9b9acadf9..c9b2cae34 100644
--- a/sdk/src/core-content.ts
+++ b/sdk/src/core-content.ts
@@ -662,6 +662,7 @@ export {
gripHandJoints,
GripSurface,
GripSurfaceShape,
+ gripSurfaceLines,
type GripSurfaceData,
type GripJoint,
type GripPoseData,
diff --git a/sdk/tests/grip-hold-wasm.test.ts b/sdk/tests/grip-hold-wasm.test.ts
index 59486792d..7dd3d40d6 100644
--- a/sdk/tests/grip-hold-wasm.test.ts
+++ b/sdk/tests/grip-hold-wasm.test.ts
@@ -8,7 +8,8 @@ import { afterEach, beforeAll, describe, expect, it } from 'vitest';
import { App } from '../src/app/app';
import { AnimationPlugin } from '../src/animation/AnimationPlugin';
import { GripHand, GripHolder, GripMirror, GripPlacement, GripProfile, captureFingerPose, gripHolderStatus, registerFingerPose, releaseGripHolds, slerpQuat, type GripHolderData, type GripProfileData } from '../src/animation/gripHold';
-import { GripPose, gripPose, gripPoseByJoint, gripPoseFromJson, gripPoseStatus, palmFrameOf, registerGripPose } from '../src/animation/gripPose';
+import { GripPose, gripHandJoints, gripPose, gripPoseByJoint, gripPoseFromJson, gripPoseStatus, palmFrameOf, registerGripPose } from '../src/animation/gripPose';
+import { GripSurface, GripSurfaceShape, closeFinger, gripSurfaceLines, surfaceDistance, surfacesUnder } from '../src/animation/gripContact';
import { Children, Name, Parent, Transform, type TransformData } from '../src/ecs/component';
import { setEditorMode, setFxEditPreview, setPlayMode } from '../src/ecs/env';
import { q } from '../src/math/quat';
@@ -532,3 +533,160 @@ function descendant(world: World, root: Entity, name: string): Entity {
throw new Error(`no ${name}`);
}
+
+/** How far a point is from a line through `on` along `along` (a unit vector). */
+function fromLine(p: V, on: V, along: V): number {
+ const d = { x: p.x - on.x, y: p.y - on.y, z: p.z - on.z };
+ const k = d.x * along.x + d.y * along.y + d.z * along.z;
+ return Math.hypot(d.x - along.x * k, d.y - along.y * k, d.z - along.z * k);
+}
+
+describe.skipIf(!HAS_WASM)('closing the fingers onto what is held', () => {
+ beforeAll(async () => { if (HAS_WASM) core = await loadWasmModule(); });
+
+ const FOUR = ['Index', 'Middle', 'Ring', 'Pinky'] as const;
+ /** The index finger's length, and a finger's thickness from its line to its skin: what the fingers stop short of a surface by. */
+ const SKIN = (0.04 + 0.025 + 0.02) * 0.08;
+
+ /**
+ * A right hand held on a gun by its anchor, a handle running across the palm under the knuckles: a cylinder on the anchor, along its X,
+ * `radius` round. Its axis in the world, and the hand's finger bones, come back.
+ */
+ async function onHandle(radius: number, options: { pose?: string; wrap?: boolean; shape?: number; size?: V; fingers?: string; center?: V } = {}) {
+ const s = scene();
+ const b = body(s.world);
+ const g = gun(s.world, { x: -0.15, y: 1.25, z: 0.45 });
+ s.world.update(g.e, GripProfile, (p: GripProfileData) => { p.mainPose = options.pose ?? 'fist'; p.wrap = options.wrap ?? true; p.mainFingers = options.fingers ?? ''; });
+ // The palm faces the anchor's +Y: the handle lies that way from it, a little ahead, under where the fingers start.
+ const center = options.center ?? { x: 0, y: radius + 0.012, z: 0.05 };
+ s.world.insert(g.main, GripSurface, { shape: options.shape ?? GripSurfaceShape.Cylinder, direction: 0, center, radius, height: 0.2, size: options.size ?? { x: 0.2, y: 0.03, z: 0.03 } });
+ holder(s.world, b.root, { held: g.e, placement: GripPlacement.Object, supportWeight: 0 });
+ await s.tick(4);
+ const anchor = worldPose(s.world, g.main);
+ const axis = { on: (() => { const v = q.rotate(anchor.r, center); return { x: anchor.p.x + v.x, y: anchor.p.y + v.y, z: anchor.p.z + v.z }; })(), along: q.rotate(anchor.r, { x: 1, y: 0, z: 0 }) };
+ const bones = (finger: string): V[] => [1, 2, 3, 4].map((i) => worldPose(s.world, descendant(s.world, b.arms.Right!.hand, `mixamorig:RightHand${finger}${i}`)).p);
+ /** The nearest any point of a finger's bones comes to the handle's axis. */
+ const nearest = (finger: string): number => {
+ const at = bones(finger);
+ let best = Infinity;
+ for (let k = 0; k < 3; k++) for (let i = 0; i <= 8; i++) {
+ const t = i / 8, a = at[k]!, c = at[k + 1]!;
+ best = Math.min(best, fromLine({ x: a.x + (c.x - a.x) * t, y: a.y + (c.y - a.y) * t, z: a.z + (c.z - a.z) * t }, axis.on, axis.along));
+ }
+ return best;
+ };
+ const bendOf = (finger: string): number => [1, 2, 3].reduce((n, i) => n + angle((s.world.get(descendant(s.world, b.arms.Right!.hand, `mixamorig:RightHand${finger}${i}`), Transform) as TransformData).rotation, I), 0);
+ return { s, b, g, axis, bones, nearest, bendOf, status: () => gripPoseStatus(s.app, b.root)!.right };
+ }
+
+ it('closes each finger until its bones meet the handle, and no further: on its skin, never inside it', async () => {
+ const R = 0.02;
+ const h = await onHandle(R);
+ expect(h.status().touching.filter((f) => f !== 'thumb').sort()).toEqual(['index', 'middle', 'pinky', 'ring']);
+ for (const finger of FOUR) {
+ // Within a ten-thousandth of the skin: the bisection's own step, and float32 bones.
+ expect(h.nearest(finger)).toBeGreaterThan(R + SKIN - 3e-4);
+ expect(h.nearest(finger)).toBeLessThan(R + SKIN + 1e-3);
+ }
+ });
+
+ it('closes further round a thinner handle, and less round a thicker one', async () => {
+ const thin = await onHandle(0.012), thick = await onHandle(0.03);
+ for (const finger of FOUR) expect(thin.bendOf(finger)).toBeGreaterThan(thick.bendOf(finger) + 0.1);
+ });
+
+ it('closes no further than the pose does, where the pose stops short of the handle', async () => {
+ const open = await onHandle(0.02, { pose: 'relaxed' });
+ const unwrapped = await onHandle(0.02, { pose: 'relaxed', wrap: false });
+ // The relaxed index finger is the least curled: it stops short, and is exactly as the pose has it.
+ expect(open.status().touching).not.toContain('index');
+ for (const finger of FOUR) {
+ if (open.status().touching.includes(finger.toLowerCase())) expect(open.bendOf(finger)).toBeLessThan(unwrapped.bendOf(finger));
+ else expect(Math.abs(open.bendOf(finger) - unwrapped.bendOf(finger))).toBeLessThan(1e-4);
+ }
+ });
+
+ it('goes through the handle with wrap off, as the pose says', async () => {
+ const h = await onHandle(0.02, { wrap: false });
+ expect(h.status().touching).toEqual([]);
+ expect(Math.min(...FOUR.map((f) => h.nearest(f)))).toBeLessThan(0.02);
+ });
+
+ it('rests the thumb on the handle rather than inside it', async () => {
+ const R = 0.02;
+ const h = await onHandle(R);
+ const tip = worldPose(h.s.world, descendant(h.s.world, h.b.arms.Right!.hand, 'mixamorig:RightHandThumb4')).p;
+ expect(fromLine(tip, h.axis.on, h.axis.along)).toBeGreaterThan(R + SKIN - 3e-4);
+ expect(h.nearest('Thumb')).toBeGreaterThan(R + SKIN - 3e-4);
+ expect(h.status().touching).toContain('thumb');
+ });
+
+ it('closes onto a box and a sphere the same way, on their skin', async () => {
+ const box = await onHandle(0.02, { shape: GripSurfaceShape.Box, size: { x: 0.2, y: 0.04, z: 0.04 } });
+ expect(box.status().touching.length).toBeGreaterThan(2);
+ const ball = await onHandle(0.03, { shape: GripSurfaceShape.Sphere });
+ expect(ball.status().touching.length).toBeGreaterThan(0);
+ });
+
+ it('holds every joint before the bone that touched: a fingertip meeting the object first stops the whole finger there', () => {
+ // Three joints closing to 1 together; the tip's bone (2) touches once the last joint passes 0.5.
+ const { bends, touched } = closeFinger([0, 0, 0], [1, 1, 1], (b, from) => (from <= 2 && b[2]! >= 0.5 ? 2 : -1));
+ expect(touched).toBe(true);
+ for (const b of bends) expect(b).toBeCloseTo(0.5, 2);
+ // A knuckle's bone touching holds only the knuckle: the rest go on to the pose.
+ const knuckle = closeFinger([0, 0, 0], [1, 1, 1], (b, from) => (from <= 0 && b[0]! >= 0.3 ? 0 : -1));
+ expect(knuckle.bends[0]).toBeCloseTo(0.3, 2);
+ expect(knuckle.bends[1]).toBe(1);
+ expect(knuckle.bends[2]).toBe(1);
+ });
+
+ it('fits a finger as it leans: each finger leaning out, closed onto a ball, rests on its skin', async () => {
+ const lean = (fan: number) => ({ curl: 1, bend: [90, 100, 70], fan });
+ const fingers = JSON.stringify({ thumb: { opposition: 0 }, index: lean(25), middle: lean(25), ring: lean(25), pinky: lean(25) });
+ const h = await onHandle(0.03, { shape: GripSurfaceShape.Sphere, fingers, center: { x: 0, y: 0.045, z: 0.06 } });
+ const placed = surfacesUnder(h.s.world, h.g.e);
+ const touching = h.status().touching.filter((f) => f !== 'thumb');
+ expect(touching.length).toBeGreaterThan(1);
+ for (const finger of touching) {
+ const at = h.bones(finger.charAt(0).toUpperCase() + finger.slice(1));
+ let near = Infinity;
+ for (let k = 0; k < 3; k++) for (let i = 0; i <= 8; i++) {
+ const t = i / 8, a = at[k]!, c = at[k + 1]!;
+ near = Math.min(near, surfaceDistance(placed, { x: a.x + (c.x - a.x) * t, y: a.y + (c.y - a.y) * t, z: a.z + (c.z - a.z) * t }));
+ }
+ expect(near, finger).toBeGreaterThan(SKIN - 3e-4);
+ expect(near, finger).toBeLessThan(SKIN + 1e-3);
+ }
+ });
+
+ it('goes back over fingers something else turned after the hand settled, with no clip component to say so: an editor sampling a clip', async () => {
+ const h = await onHandle(0.02);
+ const knuckle = descendant(h.s.world, h.b.arms.Right!.hand, 'mixamorig:RightHandIndex1');
+ const held = { ...(h.s.world.get(knuckle, Transform) as TransformData).rotation };
+ h.s.world.update(knuckle, Transform, (t: TransformData) => { t.rotation = q.normalize({ w: 0.9, x: 0.3, y: 0.2, z: 0.1 }); });
+ await h.s.tick(1);
+ expect(angle((h.s.world.get(knuckle, Transform) as TransformData).rotation, held)).toBeLessThan(1e-4);
+ });
+
+ it('says which joints each hand bends and the axis a positive bend turns each about', async () => {
+ const h = await onHandle(0.02);
+ const joints = gripHandJoints(h.s.app, h.b.root)!;
+ expect(joints.right.map((j) => `${j.finger}${j.joint}`)).toEqual(['index0', 'index1', 'index2', 'middle0', 'middle1', 'middle2', 'ring0', 'ring1', 'ring2', 'pinky0', 'pinky1', 'pinky2']);
+ const knuckle = joints.right[0]!;
+ expect(knuckle.entity).toBe(descendant(h.s.world, h.b.arms.Right!.hand, 'mixamorig:RightHandIndex1'));
+ expect(Math.hypot(knuckle.axis.x, knuckle.axis.y, knuckle.axis.z)).toBeCloseTo(1, 5);
+ // The knuckle's bone runs to the next joint; the last joint's to the fingertip.
+ expect(dist(knuckle.bone, (h.s.world.get(descendant(h.s.world, h.b.arms.Right!.hand, 'mixamorig:RightHandIndex2'), Transform) as TransformData).position)).toBeLessThan(1e-6);
+ expect(dist(joints.right[2]!.bone, (h.s.world.get(descendant(h.s.world, h.b.arms.Right!.hand, 'mixamorig:RightHandIndex4'), Transform) as TransformData).position)).toBeLessThan(1e-6);
+ });
+
+ it('draws each surface where the fingers meet it: every point of its lines on it', async () => {
+ for (const shape of [GripSurfaceShape.Cylinder, GripSurfaceShape.Capsule, GripSurfaceShape.Sphere, GripSurfaceShape.Box]) {
+ const h = await onHandle(0.02, { shape, size: { x: 0.1, y: 0.04, z: 0.03 } });
+ const lines = gripSurfaceLines(h.s.world, h.g.e);
+ expect(lines.length).toBeGreaterThan(2);
+ const placed = surfacesUnder(h.s.world, h.g.e);
+ for (const line of lines) for (const p of line) expect(Math.abs(surfaceDistance(placed, p))).toBeLessThan(1e-5);
+ }
+ });
+});
diff --git a/tools/check-gizmo-coverage.mjs b/tools/check-gizmo-coverage.mjs
index cbbb33e30..147f308c3 100644
--- a/tools/check-gizmo-coverage.mjs
+++ b/tools/check-gizmo-coverage.mjs
@@ -66,6 +66,9 @@ const DISPOSITION = {
'WaterBody.sizeZ': 'extent',
'SpringBoneCollider.radius': 'extent',
'SpringBoneCollider.height': 'extent',
+ 'GripSurface.radius': 'extent',
+ 'GripSurface.height': 'extent',
+ 'GripSurface.size': 'extent',
'Hunter.attackRange': 'extent',
'Perceiver.range': 'extent',
'ThirdPersonCamera.distance': 'extent',
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 54ff6cf78..fa679eddc 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -4262,6 +4262,8 @@ export const NOTED = {
{"note": "Labels no longer come back as black boxes after a change between Plays."},
"feat(editor): a grip is edited where it is held, the character holding it live in the edit viewport":
{"note": "A grip is edited where it is held."},
+ "feat(animation): fingers close onto what they hold, a finger joint is dragged in the viewport, and a clip plays under the grip in edit mode":
+ {"note": "Fingers close onto what they hold."},
"feat(animation): a character holds an object by anchors on it, the arms bent to them in one order that cannot loop":
{"note": "Holding an object in both hands."},
"feat(animation): a rig's hands held in a named pose that fits any skeleton":