Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(examples): video-puzzle — one stream, many live tiles; refresh video-playback

esengine34a368f0f9bc
diff --git a/build-tools/tasks/check-examples.js b/build-tools/tasks/check-examples.js
index f83d714e6..cb8e945af 100644
--- a/build-tools/tasks/check-examples.js
+++ b/build-tools/tasks/check-examples.js
@@ -46,11 +46,16 @@ function setupTypeLinks(exampleDir, rootDir) {
mkdirSync(esengineDir, { recursive: true });
- if (!existsSync(sdkLink)) {
- symlinkSync(sdkDist, sdkLink, 'dir');
- }
-
- if (!existsSync(editorLink) && existsSync(editorDist)) {
+ // Always relink to the LIVE dist: the editor stages a real-directory type
+ // mirror at the same path when a project is opened, and a stale mirror
+ // would silently shadow current SDK types (checks would fail — or pass —
+ // against an old API). The mirror is a disposable cache; the editor
+ // re-stages it on the next open.
+ rmSync(sdkLink, { recursive: true, force: true });
+ symlinkSync(sdkDist, sdkLink, 'dir');
+
+ if (existsSync(editorDist)) {
+ rmSync(editorLink, { recursive: true, force: true });
symlinkSync(editorDist, editorLink, 'dir');
}
}
diff --git a/examples/README.md b/examples/README.md
index d7ef434d8..813766746 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -29,6 +29,7 @@ for your own game.
| **postprocess-effects** | Effects | Full-screen post-processing: bloom, vignette, color grading. |
| **lighting-2d** | Rendering | Dynamic 2D lights and soft shadow casters over `Sprite.lit` surfaces. |
| **effects-gallery** | Rendering | The built-in material effect templates — hit flash, outline, dissolve, pixelate, UV scroll — with params driven from code. |
+| **video-playback** | Rendering | Play a video on a Sprite with the declarative `Video` component — "a Sprite whose texture is alive". |
| **tilemap-demo** | Tilemap | Two scenes: a Tiled `.tmj` import and an engine-native `.estileset` — multi-tileset, parallax, animated water, per-tile collision. |
| **ui-controls** | UI | Buttons, sliders, toggles and progress bars. |
| **ui-interaction** | UI | Dragging, focus and pointer interaction. |
@@ -37,6 +38,7 @@ for your own game.
| **enemy-ai** | AI | A state machine drives enemies to patrol, sense the player, and chase via A* navigation. |
| **multiplayer-arena** | Networking | Server-authoritative multiplayer with client prediction: each player steers a replicated pawn, own-pawn input applies instantly ('2 Players' in the Play dropdown runs listen server + client). |
| **platformer** | Game | A tiny platformer — run, jump and collect coins. |
+| **video-puzzle** | Game | A tile-swap puzzle whose tiles are live regions of ONE playing video — texture-handle sharing + per-piece `uvOffset`/`uvScale`. |
| **space-shooter** | Game | A vertical shmup with prefabs, a HUD and a difficulty ramp. |
## Art & assets
diff --git a/examples/video-playback/README.md b/examples/video-playback/README.md
index b4fa12981..c5b03cb19 100644
--- a/examples/video-playback/README.md
+++ b/examples/video-playback/README.md
@@ -41,9 +41,13 @@ handle.onEnded = () => console.log('finished');
// once handle.isReady, sample handle.textureHandle from any Sprite/material.
```
-## Status
+## Platforms
-This is the **M0** milestone: the runtime playback path (web / desktop). It is
-verified on both WebGL2 and WebGPU (`desktop/scripts/headless-verify.mjs` →
-`verify:render:video`). The editor asset-slot picker, `.mp4` import, and the
-WeChat backend land in the following milestones.
+Web / desktop decode through the platform `<video>` element (hardware,
+zero-copy on WebGL2); WeChat MiniGame decodes through the engine's own wasm
+MPEG-1 decoder — the export cook transcodes the clip to `.esv` and demuxes its
+audio track automatically, so the same scene ships everywhere unchanged. Both
+paths are pixel-verified headless (`verify:render:video[:ui]`).
+
+For many surfaces sharing ONE stream (e.g. a puzzle of live tiles), see the
+**video-puzzle** example.
diff --git a/examples/video-playback/project.esproject b/examples/video-playback/project.esproject
index 62616576a..0c0165165 100644
--- a/examples/video-playback/project.esproject
+++ b/examples/video-playback/project.esproject
@@ -1,7 +1,7 @@
{
"formatVersion": "1",
"name": "Video Playback",
- "description": "Play a video on a Sprite via the Video component (M0).",
+ "description": "Play a video on a Sprite via the Video component.",
"tag": "Rendering",
"version": "0.1.0",
"defaultScene": "assets/scenes/main.esscene",
diff --git a/examples/video-puzzle/.gitignore b/examples/video-puzzle/.gitignore
new file mode 100644
index 000000000..359472ac2
--- /dev/null
+++ b/examples/video-puzzle/.gitignore
@@ -0,0 +1,7 @@
+node_modules/
+dist/
+.vscode/
+.idea/
+.DS_Store
+Thumbs.db
+.esengine/cache/
diff --git a/examples/video-puzzle/README.md b/examples/video-puzzle/README.md
new file mode 100644
index 000000000..a6e8d7c48
--- /dev/null
+++ b/examples/video-puzzle/README.md
@@ -0,0 +1,51 @@
+# Video Puzzle
+
+A tile-swap puzzle whose tiles are live regions of **one playing video**.
+
+![tag:Gameplay](https://img.shields.io/badge/tag-Gameplay-green)
+
+## What it shows
+
+The mechanism, more than the game: **one decoded video stream feeding many
+surfaces**. The scene's `Preview` entity carries `Sprite + Video` — the engine
+decodes `assets/video/clip.mp4` into a live GPU texture and keeps that
+sprite's `texture` handle current. Every puzzle piece simply **shares the same
+texture handle** and picks its region with the Sprite's `uvOffset`/`uvScale`
+(the same convention atlas frames use: `v = 0` is the image's bottom row).
+
+No per-piece decoding, no extra API: the video system updates the one texture
+in place each frame, so a handle written once keeps every piece live.
+
+```
+Preview (Sprite + Video) Piece (slot r,c)
+ texture ── decoded frame ──▶ texture = preview.texture
+ uvOffset = { x: c/N, y: 1 − (r+1)/N }
+ uvScale = { x: 1/N, y: 1/N }
+```
+
+## How to play
+
+Click a tile to select it (gold tint), click another to swap which video
+region each shows. When every tile is back in its home slot the board turns
+green — click anywhere to deal a new shuffle. The corner preview always shows
+the complete video for reference.
+
+## Why the Preview entity matters
+
+Beyond UX, it is the architectural anchor: the `Video` component's `source`
+ref makes the clip **reachable** to the asset cook (script-only string refs
+are culled from builds), and its presence tells the WeChat exporter to ship
+the `videodec` wasm decoder. Declarative data drives the pipeline; the script
+only arranges who samples the result.
+
+## Systems
+
+| system | job |
+|--------|-----|
+| `SpawnBoardSystem` | one-time: spawn N×N pieces with a solvable shuffle |
+| `ShareVideoTextureSystem` | mirror the preview's live texture handle onto every piece |
+| `SwapTilesSystem` | click-to-select, click-to-swap, win detection, re-deal |
+
+Works identically on web/desktop (`<video>` hardware decode) and WeChat
+(engine-owned MPEG-1 wasm decoder) — the texture-handle contract is the same
+on every backend.
diff --git a/examples/video-puzzle/assets/scenes/main.esscene b/examples/video-puzzle/assets/scenes/main.esscene
new file mode 100644
index 000000000..31e1d0d85
--- /dev/null
+++ b/examples/video-puzzle/assets/scenes/main.esscene
@@ -0,0 +1,48 @@
+{
+ "version": "1.0",
+ "name": "Main",
+ "entities": [
+ {
+ "id": 0,
+ "name": "Camera",
+ "parent": null,
+ "children": [],
+ "components": [
+ { "type": "Transform", "data": { "position": { "x": 0, "y": 0, "z": 10 } } },
+ { "type": "Camera", "data": { "projectionType": 1, "orthoSize": 240, "isActive": true, "priority": 0 } }
+ ],
+ "visible": true
+ },
+ {
+ "id": 1,
+ "name": "Preview",
+ "parent": null,
+ "children": [],
+ "components": [
+ { "type": "Transform", "data": { "position": { "x": 232, "y": 156, "z": 0 } } },
+ {
+ "type": "Sprite",
+ "data": {
+ "size": { "x": 128, "y": 96 },
+ "color": { "r": 1, "g": 1, "b": 1, "a": 1 },
+ "layer": 1
+ }
+ },
+ {
+ "type": "Video",
+ "data": {
+ "source": "assets/video/clip.mp4",
+ "autoplay": true,
+ "loop": true,
+ "muted": true,
+ "volume": 1,
+ "playbackRate": 1,
+ "fitSize": false,
+ "enabled": true
+ }
+ }
+ ],
+ "visible": true
+ }
+ ]
+}
diff --git a/examples/video-puzzle/assets/scenes/main.esscene.meta b/examples/video-puzzle/assets/scenes/main.esscene.meta
new file mode 100644
index 000000000..930fd334c
--- /dev/null
+++ b/examples/video-puzzle/assets/scenes/main.esscene.meta
@@ -0,0 +1,6 @@
+{
+ "uuid": "e926e1ba-d3fe-4760-b6c3-cc88262f395e",
+ "version": "2.0",
+ "type": "scene",
+ "importer": {}
+}
diff --git a/examples/video-puzzle/assets/video/clip.mp4 b/examples/video-puzzle/assets/video/clip.mp4
new file mode 100644
index 000000000..a4e72b662
Binary files /dev/null and b/examples/video-puzzle/assets/video/clip.mp4 differ
diff --git a/examples/video-puzzle/assets/video/clip.mp4.meta b/examples/video-puzzle/assets/video/clip.mp4.meta
new file mode 100644
index 000000000..2a58c1476
--- /dev/null
+++ b/examples/video-puzzle/assets/video/clip.mp4.meta
@@ -0,0 +1,6 @@
+{
+ "uuid": "9fa233c3-a560-4280-9416-a6ba482b7c05",
+ "version": "2.0",
+ "type": "video",
+ "importer": {}
+}
diff --git a/examples/video-puzzle/project.esproject b/examples/video-puzzle/project.esproject
new file mode 100644
index 000000000..582eaf92a
--- /dev/null
+++ b/examples/video-puzzle/project.esproject
@@ -0,0 +1,25 @@
+{
+ "formatVersion": "1",
+ "name": "Video Puzzle",
+ "description": "A sliding-tile puzzle whose tiles are live regions of ONE playing video.",
+ "tag": "Gameplay",
+ "version": "0.1.0",
+ "defaultScene": "assets/scenes/main.esscene",
+ "designResolution": {
+ "width": 640,
+ "height": 480
+ },
+ "spineVersion": "none",
+ "packaging": {
+ "platform": "wechat",
+ "config": "shipping",
+ "sourceMaps": false,
+ "openFolder": true,
+ "outDir": {
+ "wechat": "dist-wechat"
+ },
+ "compressTextures": false,
+ "atlasTextures": false,
+ "compressAudio": false
+ }
+}
diff --git a/examples/video-puzzle/src/components.ts b/examples/video-puzzle/src/components.ts
new file mode 100644
index 000000000..56826d2c3
--- /dev/null
+++ b/examples/video-puzzle/src/components.ts
@@ -0,0 +1,12 @@
+import { defineComponent } from 'esengine';
+
+/**
+ * One board tile. `slot` is the fixed board cell this entity sits in (never
+ * changes); `tile` is which region of the video it currently SHOWS. Swapping
+ * two pieces swaps their `tile`s — the entities and their transforms stay put,
+ * only the sampled video region moves. Solved = every piece shows its own slot.
+ */
+export const PuzzlePiece = defineComponent('PuzzlePiece', {
+ slot: 0,
+ tile: 0,
+});
diff --git a/examples/video-puzzle/src/main.ts b/examples/video-puzzle/src/main.ts
new file mode 100644
index 000000000..d002e0918
--- /dev/null
+++ b/examples/video-puzzle/src/main.ts
@@ -0,0 +1,171 @@
+// SPDX-License-Identifier: Apache-2.0
+// Video Puzzle — N×N tiles, each a live REGION of one playing video.
+//
+// The mechanism is the point of this example: there is exactly ONE decoded
+// video stream. The scene's "Preview" entity carries Sprite + Video, so the
+// engine's video system decodes the clip and keeps that sprite's `texture`
+// handle alive. Every puzzle piece simply SHARES that texture handle and picks
+// its region with the Sprite's `uvOffset`/`uvScale` — no per-piece decoding,
+// no extra API. Click two tiles to swap which region each one shows.
+import {
+ addSystemToSchedule, defineSystem, Schedule,
+ Query, Mut, Res, Commands, Input, UICameraInfo,
+ Transform, Sprite, Video,
+} from 'esengine';
+import type { UICameraData } from 'esengine';
+import { PuzzlePiece } from './components';
+
+const GRID = 3; // GRID × GRID tiles
+const TILE = 96; // world size of one tile
+const GAP = 4; // spacing so the seams read as pieces
+const BOARD_X = -96; // board center (preview sits to the right)
+const BOARD_Y = 0;
+
+const SELECTED_TINT = { r: 1, g: 0.85, b: 0.4, a: 1 };
+const NORMAL_TINT = { r: 1, g: 1, b: 1, a: 1 };
+const SOLVED_TINT = { r: 0.65, g: 1, b: 0.7, a: 1 };
+
+const slotX = (slot: number) => BOARD_X + ((slot % GRID) - (GRID - 1) / 2) * (TILE + GAP);
+const slotY = (slot: number) => BOARD_Y - (Math.floor(slot / GRID) - (GRID - 1) / 2) * (TILE + GAP);
+
+/**
+ * The video texture samples like any flipY-uploaded image: v = 0 is the
+ * image's BOTTOM row (the same convention atlas frames use). Tile 0 is the
+ * top-left region, so its v-offset is the top band: 1 - 1/GRID.
+ */
+function tileUv(tile: number): { offset: { x: number; y: number }; scale: { x: number; y: number } } {
+ const col = tile % GRID;
+ const row = Math.floor(tile / GRID);
+ return {
+ offset: { x: col / GRID, y: 1 - (row + 1) / GRID },
+ scale: { x: 1 / GRID, y: 1 / GRID },
+ };
+}
+
+/** Fisher–Yates over an even number of transpositions keeps a swap puzzle
+ * solvable by construction; reshuffle away the already-solved deal. */
+function shuffledTiles(): number[] {
+ const tiles = Array.from({ length: GRID * GRID }, (_, i) => i);
+ do {
+ for (let i = tiles.length - 1; i > 0; i--) {
+ const j = Math.floor(Math.random() * (i + 1));
+ [tiles[i], tiles[j]] = [tiles[j], tiles[i]];
+ }
+ } while (tiles.every((t, i) => t === i));
+ return tiles;
+}
+
+let selectedSlot = -1;
+let solved = false;
+
+const spawnBoardSystem = defineSystem(
+ [Query(PuzzlePiece), Commands()],
+ (pieces, cmds) => {
+ if (!pieces.isEmpty()) return;
+ selectedSlot = -1;
+ solved = false;
+ const tiles = shuffledTiles();
+ for (let slot = 0; slot < GRID * GRID; slot++) {
+ const uv = tileUv(tiles[slot]);
+ cmds.spawn()
+ .insert(Transform, { position: { x: slotX(slot), y: slotY(slot), z: 0 } })
+ .insert(Sprite, {
+ size: { x: TILE, y: TILE },
+ color: NORMAL_TINT,
+ uvOffset: uv.offset,
+ uvScale: uv.scale,
+ })
+ .insert(PuzzlePiece, { slot, tile: tiles[slot] });
+ }
+ },
+ { name: 'SpawnBoardSystem' },
+);
+
+/**
+ * Mirror the live video texture onto every piece. The Video component keeps
+ * the Preview sprite's `texture` current; pieces adopt the same handle — the
+ * GPU texture updates in place each frame, so one write per piece per stream
+ * start is all the sharing costs.
+ */
+const shareVideoTextureSystem = defineSystem(
+ [Query(Sprite, Video), Query(Mut(Sprite), PuzzlePiece)],
+ (preview, pieces) => {
+ let texture = 0;
+ for (const [, sprite] of preview) {
+ texture = sprite.texture;
+ break;
+ }
+ if (!texture) return; // first frame not decoded yet
+ for (const [, sprite] of pieces) {
+ if (sprite.texture !== texture) sprite.texture = texture;
+ }
+ },
+ { name: 'ShareVideoTextureSystem' },
+);
+
+const swapTilesSystem = defineSystem(
+ [Res(Input), Res(UICameraInfo), Query(Mut(Sprite), Mut(PuzzlePiece))],
+ (input, camera: UICameraData, pieces) => {
+ if (!camera.valid || !input.isMouseButtonPressed(0)) return;
+
+ if (solved) { // any click deals a new board
+ const tiles = shuffledTiles();
+ for (const [, sprite, piece] of pieces) {
+ piece.tile = tiles[piece.slot];
+ const uv = tileUv(piece.tile);
+ sprite.uvOffset = uv.offset;
+ sprite.uvScale = uv.scale;
+ sprite.color = NORMAL_TINT;
+ }
+ solved = false;
+ return;
+ }
+
+ const half = (TILE + GAP) / 2;
+ let clickedSlot = -1;
+ for (const [, , piece] of pieces) {
+ if (Math.abs(camera.worldMouseX - slotX(piece.slot)) <= half &&
+ Math.abs(camera.worldMouseY - slotY(piece.slot)) <= half) {
+ clickedSlot = piece.slot;
+ break;
+ }
+ }
+ if (clickedSlot < 0) return;
+
+ if (selectedSlot < 0) {
+ selectedSlot = clickedSlot;
+ } else if (selectedSlot !== clickedSlot) {
+ // Swap which region the two pieces sample.
+ let a: { tile: number } | null = null;
+ let b: { tile: number } | null = null;
+ for (const [, , piece] of pieces) {
+ if (piece.slot === selectedSlot) a = piece;
+ if (piece.slot === clickedSlot) b = piece;
+ }
+ if (a && b) {
+ [a.tile, b.tile] = [b.tile, a.tile];
+ }
+ selectedSlot = -1;
+ } else {
+ selectedSlot = -1; // clicking the selection again deselects
+ }
+
+ let allHome = true;
+ for (const [, sprite, piece] of pieces) {
+ const uv = tileUv(piece.tile);
+ sprite.uvOffset = uv.offset;
+ sprite.uvScale = uv.scale;
+ if (piece.tile !== piece.slot) allHome = false;
+ sprite.color = piece.slot === selectedSlot ? SELECTED_TINT : NORMAL_TINT;
+ }
+ if (allHome) {
+ solved = true;
+ for (const [, sprite] of pieces) sprite.color = SOLVED_TINT;
+ }
+ },
+ { name: 'SwapTilesSystem' },
+);
+
+addSystemToSchedule(Schedule.Update, spawnBoardSystem);
+addSystemToSchedule(Schedule.Update, shareVideoTextureSystem);
+addSystemToSchedule(Schedule.Update, swapTilesSystem);
diff --git a/examples/video-puzzle/tsconfig.json b/examples/video-puzzle/tsconfig.json
new file mode 100644
index 000000000..0a8528b29
--- /dev/null
+++ b/examples/video-puzzle/tsconfig.json
@@ -0,0 +1,22 @@
+{
+ "compilerOptions": {
+ "target": "ES2020",
+ "module": "ESNext",
+ "moduleResolution": "bundler",
+ "strict": true,
+ "esModuleInterop": true,
+ "skipLibCheck": true,
+ "forceConsistentCasingInFileNames": true,
+ "declaration": false,
+ "outDir": "./dist",
+ "rootDir": ".",
+ "baseUrl": ".",
+ "paths": {
+ "esengine": ["./.esengine/sdk/index.d.ts"],
+ "esengine/wasm": ["./.esengine/sdk/wasm.d.ts"],
+ "@esengine/editor": ["./.esengine/editor/index.d.ts"]
+ }
+ },
+ "include": ["src/**/*"],
+ "exclude": ["node_modules"]
+}