A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(ui): a Spine skeleton in a Canvas paints by its place in the tree, with the UI's opacity, visibility and masks
yhh85f3835ea6fe
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4fa7e48b9..9e1dc63e3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,6 +16,7 @@ published separately; it ships inside the editor.### Added+- **A Spine skeleton can be part of a UI.** A `SpineAnimation` under a Canvas paints by its place in the Canvas tree, between the controls around it, and takes the UI tree's opacity, hidden state and masks. Under a screen-space Canvas it is drawn with the UI in the screen overlay instead of by the cameras, where before it showed up in the world at the Canvas' pixel coordinates and never in the UI (#116). A non-zero `layer` still holds it over or under the whole UI, and a skeleton outside any Canvas is unchanged. DragonBones armatures under a screen Canvas are drawn in the overlay too.- **A grip is edited where it is held.** Select an object with a `GripProfile` and its Details section edits the grip while a character in the scene holds it in the edit viewport, without Play: pick the character (it is given a `GripHolder`), create and pick the two palm anchors, see each palm drawn where it goes, switch the leading hand, and change every finger joint by joint in degrees, starting from a grip pose. Anchors are placed once and mirrored in the object's own plane for the other hand (`GripProfile.hand`, `mirror`); fingers said joint by joint (`GripFinger.bend`, `fan`, kept in `mainFingers`/`supportFingers`) are measured on the hand's own axes, so they mirror and fit any rig. The preview is the viewport's Preview FX and gives the bones back when it is turned off.- **A material reads its maps through Tiling, Offset and Rotation.** The Model shader's `u_uvTiling`, `u_uvOffset` and `u_uvRotation` move where every map is read (colour, normal, occlusion, metal-roughness, emission), as Unity's Tiling and Offset do; Ground Blend takes them too. A glTF's `KHR_texture_transform` is imported into them, turned into the engine's V-up uv, where before it was dropped with a warning and a tiled texture set was stretched once over each mesh: a three.js port of a container port lost 224 of them. Maps of one material that disagree are all read through the first map's, and the import says so. A cut-out's shadow and depth prepass read through the same transform, so a tiled fence casts the shadow of the fence drawn.diff --git a/desktop b/desktopindex cd9a1c9a4..af0091328 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit cd9a1c9a412b1a18cfe37820ff77cff59aa2d024+Subproject commit af0091328c3636cac558f28272ace21fb46e531cdiff --git a/docs/astro/src/content/docs/animation/spine.mdx b/docs/astro/src/content/docs/animation/spine.mdxindex 94a14dfa5..7040a286d 100644--- a/docs/astro/src/content/docs/animation/spine.mdx+++ b/docs/astro/src/content/docs/animation/spine.mdx@@ -109,6 +109,24 @@ const skins = spine.getSkins(entity); // string[]const bounds = spine.getBounds(entity); // { x, y, width, height } | null```+## Spine in a UI++A skeleton can be part of a Canvas. Put the entity with `SpineAnimation` under+the Canvas, with or without a `UINode` to place it, and it paints by its place+in the Canvas tree, like the controls around it. A skeleton listed after a+panel is drawn over it, and one listed before is drawn under it. It also takes+on the rest of the UI tree's rules:++- the `opacity` of the UI nodes above it fades it;+- a hidden parent hides it;+- a `UIMask` above it cuts it.++A skeleton under a screen-space Canvas is drawn with the UI in the screen+overlay, after every camera. A non-zero `layer` keeps the old behaviour and+holds the skeleton over or under the whole UI. A skeleton outside any Canvas+draws as before. DragonBones armatures under a screen Canvas are drawn in the+overlay too.+## Animation eventsThe `SpineEvents` resource publishes track events once per frame — read it the samediff --git a/docs/astro/src/content/docs/zh-cn/animation/spine.mdx b/docs/astro/src/content/docs/zh-cn/animation/spine.mdxindex 65e102c92..e3d3d5f31 100644--- a/docs/astro/src/content/docs/zh-cn/animation/spine.mdx+++ b/docs/astro/src/content/docs/zh-cn/animation/spine.mdx@@ -101,6 +101,20 @@ const skins = spine.getSkins(entity); // string[]const bounds = spine.getBounds(entity); // { x, y, width, height } | null```+## 在 UI 中使用 Spine++骨骼可以作为 Canvas 的一部分。把带 `SpineAnimation` 的实体放到 Canvas 下面(可以带 `UINode` 来定位,也可以不带),+它就会和周围的控件一样,按在 Canvas 节点树里的位置绘制:排在某个面板后面就画在面板上面,排在前面就被面板挡住。+它同时遵循 UI 树的其他规则:++- 上方 UI 节点的 `opacity` 会让它一起变淡;+- 父节点隐藏时它也隐藏;+- 上方的 `UIMask` 会裁剪它。++屏幕空间 Canvas 下的骨骼和 UI 一起在屏幕叠加层里绘制,也就是在所有相机之后。+把 `layer` 设为非 0 会保留原来的行为,让骨骼整体压在 UI 上面或下面。不在任何 Canvas 下的骨骼绘制方式不变。+屏幕 Canvas 下的 DragonBones 骨架同样在叠加层里绘制。+## 动画事件`SpineEvents` 资源每帧发布一次轨道事件——在同一帧读取:diff --git a/sdk/src/dragonbones/DragonBonesPlugin.ts b/sdk/src/dragonbones/DragonBonesPlugin.tsindex 9e8b8e9ef..d5db3c958 100644--- a/sdk/src/dragonbones/DragonBonesPlugin.ts+++ b/sdk/src/dragonbones/DragonBonesPlugin.ts@@ -22,6 +22,7 @@ import { DragonBonesManager } from './DragonBonesManager';import { DragonBonesModuleController } from './DragonBonesController';import { wrapDragonBonesModule, type DragonBonesWasmModule } from './DragonBonesModuleLoader';import { log } from '../util/logger';+import { addSkeletalSubmit } from '../skeletal/submitDomains';/** Reach the manager from a system or from gameplay code. Null until acquired. */export const DragonBones = defineResource<DragonBonesManager | null>(null, 'DragonBones');@@ -98,10 +99,11 @@ export class DragonBonesPlugin implements Plugin {if (!this.submitWired_) {this.submitWired_ = true;const core = engineApi(app);- app.pipeline?.addPreFlushCallback(registry => {+ // The cameras' armatures, then a screen Canvas' in the overlay (skeletal/submitDomains).+ addSkeletalSubmit(app, (registry, include) => {if (core) this.manager_?.submitMeshes(core, registry._cpp, (e) =>- (app.world.tryGet(e, DragonBonesAnimation)?.material as number | undefined) ?? 0, isActive);- });+ (app.world.tryGet(e, DragonBonesAnimation)?.material as number | undefined) ?? 0, include);+ }, isActive);}// Carries DragonBonesAnimation.enabled across before advancing — the flagdiff --git a/sdk/src/skeletal/submitDomains.ts b/sdk/src/skeletal/submitDomains.tsnew file mode 100644index 000000000..266b0a3a0--- /dev/null+++ b/sdk/src/skeletal/submitDomains.ts@@ -0,0 +1,35 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file skeletal/submitDomains.ts+ * @brief Where a skeletal runtime submits its posed meshes: the cameras, and the screen overlay.+ *+ * @details A skeleton under a screen-space Canvas is UI: the screen overlay draws it, in+ * its place in the Canvas' tree, and no camera does (the core partitions the two+ * collects the same way — RenderFrame::submitSkeletalBatch). So each runtime+ * submits twice a frame, each time only what that pass owns: the cameras get the+ * world's skeletons, the overlay the screen's. Asking first keeps a skeleton from+ * being posed into the heap for a pass that would drop it.+ */+import type { App } from '../app/app';+import type { Entity } from '../types';+import type { CppRegistry } from '../wasm';+import { engineApi } from '../ecs/bridge/engineApi';++/** Submits one pass's skeletons; `include` says which entities this pass owns. */+export type SkeletalDomainSubmit = (registry: { _cpp: CppRegistry }, include: (entity: Entity) => boolean) => void;++/**+ * Register `submit` for the cameras and for the screen overlay. A core that cannot+ * say what the screen owns has partitioned nothing, so everything goes to the cameras.+ */+export function addSkeletalSubmit(app: App, submit: SkeletalDomainSubmit, isActive: (entity: Entity) => boolean): void {+ const api = engineApi(app) as { ui_isScreenDomain?(registry: unknown, entity: number): boolean } | null;+ const isScreen = (registry: { _cpp: CppRegistry }, entity: Entity): boolean =>+ api?.ui_isScreenDomain?.(registry._cpp, entity) ?? false;+ app.pipeline?.addPreFlushCallback((registry) =>+ submit(registry, (e) => isActive(e) && !isScreen(registry, e)));+ if (!api?.ui_isScreenDomain) return;+ app.pipeline?.addScreenOverlayCallback((registry) =>+ submit(registry, (e) => isActive(e) && isScreen(registry, e)));+}diff --git a/sdk/src/spine/SpinePlugin.ts b/sdk/src/spine/SpinePlugin.tsindex 34d1db4af..80767f3a7 100644--- a/sdk/src/spine/SpinePlugin.ts+++ b/sdk/src/spine/SpinePlugin.ts@@ -13,6 +13,7 @@ import { engineApi } from '../ecs/bridge/engineApi';import { AnimatorController } from '../animation/Animator';import { SpineAnimation } from '../ecs/component';import { SkeletalEnableMirror } from '../skeletal/enableSync';+import { addSkeletalSubmit } from '../skeletal/submitDomains';export type SpineEventType = 'start' | 'interrupt' | 'end' | 'complete' | 'event';@@ -96,10 +97,12 @@ export class SpinePlugin implements Plugin {if (this.submitWired_) return;this.submitWired_ = true;const isActive = (e: Entity) => app.world.isEntityActive(e);- app.pipeline?.addPreFlushCallback((registry) => {+ // Twice a frame: the cameras' skeletons, then a screen Canvas' in the overlay,+ // where they paint by their place in its tree (skeletal/submitDomains).+ addSkeletalSubmit(app, (registry, include) => {this.spineManager_?.submitMeshes(registry._cpp, (e: Entity) =>- (app.world.tryGet(e, SpineAnimation)?.material as number | undefined) ?? 0, isActive);- });+ (app.world.tryGet(e, SpineAnimation)?.material as number | undefined) ?? 0, include);+ }, isActive);}build(app: App): void {diff --git a/sdk/tests/skeletal-activity-wasm.test.ts b/sdk/tests/skeletal-activity-wasm.test.tsindex 64fff1af4..22dbdb21c 100644--- a/sdk/tests/skeletal-activity-wasm.test.ts+++ b/sdk/tests/skeletal-activity-wasm.test.ts@@ -31,7 +31,9 @@ describe.skipIf(!HAS_WASM || !hasSideModule('spine38') || !hasSideModule('dragoncore.renderer_submitSkeletalBatchByEntityEx = core.renderer_submitSkeletalBatchByEntity;const app = App.new(); app.connectCpp(registry, core);const callbacks: Array<(r: { _cpp: CppRegistry }) => void> = [];- app.setPipeline({ addPreFlushCallback: (cb: (r: { _cpp: CppRegistry }) => void) => callbacks.push(cb) } as unknown as RenderPipeline);+ // The cameras' pass only: the screen overlay's submits nothing in a world with no Canvas.+ app.setPipeline({ addPreFlushCallback: (cb: (r: { _cpp: CppRegistry }) => void) => callbacks.push(cb),+ addScreenOverlayCallback: () => {} } as unknown as RenderPipeline);const world = app.world;const parent = world.spawn('parent'); world.insert(parent, Transform, {});const child = world.spawn('skeleton'); world.insert(child, Transform, {}); world.setParent(child, parent);diff --git a/src/esengine/bindings/RendererBindings.cpp b/src/esengine/bindings/RendererBindings.cppindex fab736743..a25fb4612 100644--- a/src/esengine/bindings/RendererBindings.cpp+++ b/src/esengine/bindings/RendererBindings.cpp@@ -684,6 +684,17 @@ u32 probe_volume_create(i32 resX, i32 resY, i32 resZ, uintptr_t shPtr) {}/** @brief Releases a probe volume. */+void probe_volume_setSkyVisibility(u32 volumeHandle, uintptr_t visibilityPtr) {+ auto* rm = ctx().tryGet<resource::ResourceManager>();+ if (!rm) return;+ ProbeVolume* grid = rm->getProbeVolume(resource::ProbeVolumeHandle(volumeHandle));+ if (!grid || !grid->isValid()) return;+ const u32 want = grid->probeCount() * 9;+ const f32* v = boundarySpan<f32>(visibilityPtr, want, "probe_volume_setSkyVisibility.visibility");+ if (!v) return;+ grid->skyVisibility.assign(v, v + want);+}+void probe_volume_release(u32 volumeHandle) {if (auto* rm = ctx().tryGet<resource::ResourceManager>()) {rm->releaseProbeVolume(resource::ProbeVolumeHandle(volumeHandle));diff --git a/src/esengine/renderer/frame/RenderFrameSubmit.cpp b/src/esengine/renderer/frame/RenderFrameSubmit.cppindex cb111787c..e391a55c6 100644--- a/src/esengine/renderer/frame/RenderFrameSubmit.cpp+++ b/src/esengine/renderer/frame/RenderFrameSubmit.cpp@@ -93,10 +93,19 @@ void RenderFrame::submitSkeletalBatch(const u16* indices, i32 indexCount,u32 textureId, i32 blendMode,const f32* transform16,- Entity entity, i32 layer, f32 depth, u32 materialId, bool premultipliedColor+ Entity entity, i32 layer, f32 depth, u32 materialId, bool premultipliedColor,+ f32 treeAlpha, u32 cullBit) {- if (vertexCount <= 0 || indexCount <= 0- || !batchEntityVisible(draw_list_, clip_state_, entity, layer)) return;+ // The screen overlay and the cameras partition the world (RenderCollectContext::accepts):+ // a skeleton under a screen Canvas is the overlay's, and a camera drawing it too is how it+ // showed in the world at the Canvas' pixel coordinates while missing from the UI.+ const bool screen = screen_domain_ && screen_domain_->count(entity.id()) != 0;+ if (screen != overlay_active_) return;+ if (layer == 0 && ui_tree_layers_) {+ if (const auto it = ui_tree_layers_->find(entity.id()); it != ui_tree_layers_->end()) layer = it->second;+ }+ if (vertexCount <= 0 || indexCount <= 0 || treeAlpha <= 0.0f+ || !batchEntityVisible(draw_list_, clip_state_, entity, layer, cullBit)) return;constexpr i32 FLOATS_PER_VERTEX = 8;glm::mat4 model = glm::make_mat4(transform16);@@ -114,7 +123,8 @@ void RenderFrame::submitSkeletalBatch(for (i32 i = 0; i < vertexCount; ++i) {const f32* v = vertices + i * FLOATS_PER_VERTEX;glm::vec4 worldPos = model * glm::vec4(v[0], v[1], 0.0f, 1.0f);- glm::vec4 color(v[4], v[5], v[6], v[7]);+ // The UI tree's opacity, as a control's quad takes it: premultiplied colour scales whole.+ glm::vec4 color = glm::vec4(v[4], v[5], v[6], v[7]) * (premultipliedColor ? glm::vec4(treeAlpha) : glm::vec4(1.0f, 1.0f, 1.0f, treeAlpha));if (premultipliedColor && builtinColor) {// Recover straight color before 8-bit packing and the shader EOTF.color = color.a > 0.0f ? glm::vec4(glm::vec3(color) / color.a, color.a)@@ -133,6 +143,7 @@ void RenderFrame::submitSkeletalBatch(.blend = blend,.textureId = textureId,.depth = depth,+ .cullBit = cullBit,.entity = entity,.type = RenderType::Skeletal,};diff --git a/src/esengine/ui/UIRenderOrderSystem.hpp b/src/esengine/ui/UIRenderOrderSystem.hppindex 6c2e3d493..37fdbf8fe 100644--- a/src/esengine/ui/UIRenderOrderSystem.hpp+++ b/src/esengine/ui/UIRenderOrderSystem.hpp@@ -9,6 +9,7 @@#include "../ecs/components/UIVisual.hpp"#include "../ecs/components/Canvas.hpp"#include "../ecs/components/ParticleEmitter.hpp"+#include "../ecs/components/SpineAnimation.hpp"#include "CanvasRoots.hpp"#include <unordered_map>@@ -17,7 +18,7 @@namespace esengine::ecs {/**- * @brief The draw layer of what a Canvas holds that has no uiOrder of its own: an emitter, and a Sprite with no UINode.+ * @brief The draw layer of what a Canvas holds that has no uiOrder of its own: an emitter, a Spine skeleton, and a Sprite with no UINode.* @details They paint by their place in the tree like the controls round them, so one listed after a panel is over it and one listed* before is under it. Written to this table and not to their `layer`, which stays what its author said.*/@@ -58,7 +59,8 @@ inline i32 assignRenderOrder(Registry& registry, Entity entity, i32 counter, u32}}if (tree && !registry.has<UIVisual>(entity)- && (registry.has<ParticleEmitter>(entity) || (registry.has<Sprite>(entity) && !registry.has<UINode>(entity)))) {+ && (registry.has<ParticleEmitter>(entity) || registry.has<SpineAnimation>(entity)+ || (registry.has<Sprite>(entity) && !registry.has<UINode>(entity)))) {tree->layers[entity.id()] = tree->base + counter;counter++;}diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 13cf95ce3..7ae56eb87 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -28,6 +28,8 @@ export const NOTES_FLOOR = 'v0.65.0';* the reader recognises anyway.*/export const NOTED = {+ "feat(ui): a Spine skeleton in a Canvas paints by its place in the tree, with the UI's opacity, visibility and masks":+ { note: "A Spine skeleton can be part of a UI." },"fix(export): an Android package carries its Spine runtimes on every architecture, not the first only":{note: "An Android package built for the emulator carries its Spine runtimes on x86_64 too."},"fix(docs): a component summary with an apostrophe no longer breaks the docs build":