A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(examples): video-puzzle — one stream, many live tiles; refresh video-playback
esengine34a368f0f9bc
diff --git a/build-tools/tasks/check-examples.js b/build-tools/tasks/check-examples.jsindex f83d714e6..cb8e945af 100644--- a/build-tools/tasks/check-examples.js+++ b/build-tools/tasks/check-examples.js@@ -46,11 +46,16 @@ function setupTypeLinks(exampleDir, rootDir) {mkdirSync(esengineDir, { recursive: true });- if (!existsSync(sdkLink)) {- symlinkSync(sdkDist, sdkLink, 'dir');- }-- if (!existsSync(editorLink) && existsSync(editorDist)) {+ // Always relink to the LIVE dist: the editor stages a real-directory type+ // mirror at the same path when a project is opened, and a stale mirror+ // would silently shadow current SDK types (checks would fail — or pass —+ // against an old API). The mirror is a disposable cache; the editor+ // re-stages it on the next open.+ rmSync(sdkLink, { recursive: true, force: true });+ symlinkSync(sdkDist, sdkLink, 'dir');++ if (existsSync(editorDist)) {+ rmSync(editorLink, { recursive: true, force: true });symlinkSync(editorDist, editorLink, 'dir');}}diff --git a/examples/README.md b/examples/README.mdindex d7ef434d8..813766746 100644--- a/examples/README.md+++ b/examples/README.md@@ -29,6 +29,7 @@ for your own game.| **postprocess-effects** | Effects | Full-screen post-processing: bloom, vignette, color grading. || **lighting-2d** | Rendering | Dynamic 2D lights and soft shadow casters over `Sprite.lit` surfaces. || **effects-gallery** | Rendering | The built-in material effect templates — hit flash, outline, dissolve, pixelate, UV scroll — with params driven from code. |+| **video-playback** | Rendering | Play a video on a Sprite with the declarative `Video` component — "a Sprite whose texture is alive". || **tilemap-demo** | Tilemap | Two scenes: a Tiled `.tmj` import and an engine-native `.estileset` — multi-tileset, parallax, animated water, per-tile collision. || **ui-controls** | UI | Buttons, sliders, toggles and progress bars. || **ui-interaction** | UI | Dragging, focus and pointer interaction. |@@ -37,6 +38,7 @@ for your own game.| **enemy-ai** | AI | A state machine drives enemies to patrol, sense the player, and chase via A* navigation. || **multiplayer-arena** | Networking | Server-authoritative multiplayer with client prediction: each player steers a replicated pawn, own-pawn input applies instantly ('2 Players' in the Play dropdown runs listen server + client). || **platformer** | Game | A tiny platformer — run, jump and collect coins. |+| **video-puzzle** | Game | A tile-swap puzzle whose tiles are live regions of ONE playing video — texture-handle sharing + per-piece `uvOffset`/`uvScale`. || **space-shooter** | Game | A vertical shmup with prefabs, a HUD and a difficulty ramp. |## Art & assetsdiff --git a/examples/video-playback/README.md b/examples/video-playback/README.mdindex b4fa12981..c5b03cb19 100644--- a/examples/video-playback/README.md+++ b/examples/video-playback/README.md@@ -41,9 +41,13 @@ handle.onEnded = () => console.log('finished');// once handle.isReady, sample handle.textureHandle from any Sprite/material.```-## Status+## Platforms-This is the **M0** milestone: the runtime playback path (web / desktop). It is-verified on both WebGL2 and WebGPU (`desktop/scripts/headless-verify.mjs` →-`verify:render:video`). The editor asset-slot picker, `.mp4` import, and the-WeChat backend land in the following milestones.+Web / desktop decode through the platform `<video>` element (hardware,+zero-copy on WebGL2); WeChat MiniGame decodes through the engine's own wasm+MPEG-1 decoder — the export cook transcodes the clip to `.esv` and demuxes its+audio track automatically, so the same scene ships everywhere unchanged. Both+paths are pixel-verified headless (`verify:render:video[:ui]`).++For many surfaces sharing ONE stream (e.g. a puzzle of live tiles), see the+**video-puzzle** example.diff --git a/examples/video-playback/project.esproject b/examples/video-playback/project.esprojectindex 62616576a..0c0165165 100644--- a/examples/video-playback/project.esproject+++ b/examples/video-playback/project.esproject@@ -1,7 +1,7 @@{"formatVersion": "1","name": "Video Playback",- "description": "Play a video on a Sprite via the Video component (M0).",+ "description": "Play a video on a Sprite via the Video component.","tag": "Rendering","version": "0.1.0","defaultScene": "assets/scenes/main.esscene",diff --git a/examples/video-puzzle/.gitignore b/examples/video-puzzle/.gitignorenew file mode 100644index 000000000..359472ac2--- /dev/null+++ b/examples/video-puzzle/.gitignore@@ -0,0 +1,7 @@+node_modules/+dist/+.vscode/+.idea/+.DS_Store+Thumbs.db+.esengine/cache/diff --git a/examples/video-puzzle/README.md b/examples/video-puzzle/README.mdnew file mode 100644index 000000000..a6e8d7c48--- /dev/null+++ b/examples/video-puzzle/README.md@@ -0,0 +1,51 @@+# Video Puzzle++A tile-swap puzzle whose tiles are live regions of **one playing video**.++++## What it shows++The mechanism, more than the game: **one decoded video stream feeding many+surfaces**. The scene's `Preview` entity carries `Sprite + Video` — the engine+decodes `assets/video/clip.mp4` into a live GPU texture and keeps that+sprite's `texture` handle current. Every puzzle piece simply **shares the same+texture handle** and picks its region with the Sprite's `uvOffset`/`uvScale`+(the same convention atlas frames use: `v = 0` is the image's bottom row).++No per-piece decoding, no extra API: the video system updates the one texture+in place each frame, so a handle written once keeps every piece live.++```+Preview (Sprite + Video) Piece (slot r,c)+ texture ── decoded frame ──▶ texture = preview.texture+ uvOffset = { x: c/N, y: 1 − (r+1)/N }+ uvScale = { x: 1/N, y: 1/N }+```++## How to play++Click a tile to select it (gold tint), click another to swap which video+region each shows. When every tile is back in its home slot the board turns+green — click anywhere to deal a new shuffle. The corner preview always shows+the complete video for reference.++## Why the Preview entity matters++Beyond UX, it is the architectural anchor: the `Video` component's `source`+ref makes the clip **reachable** to the asset cook (script-only string refs+are culled from builds), and its presence tells the WeChat exporter to ship+the `videodec` wasm decoder. Declarative data drives the pipeline; the script+only arranges who samples the result.++## Systems++| system | job |+|--------|-----|+| `SpawnBoardSystem` | one-time: spawn N×N pieces with a solvable shuffle |+| `ShareVideoTextureSystem` | mirror the preview's live texture handle onto every piece |+| `SwapTilesSystem` | click-to-select, click-to-swap, win detection, re-deal |++Works identically on web/desktop (`<video>` hardware decode) and WeChat+(engine-owned MPEG-1 wasm decoder) — the texture-handle contract is the same+on every backend.diff --git a/examples/video-puzzle/assets/scenes/main.esscene b/examples/video-puzzle/assets/scenes/main.esscenenew file mode 100644index 000000000..31e1d0d85--- /dev/null+++ b/examples/video-puzzle/assets/scenes/main.esscene@@ -0,0 +1,48 @@+{+ "version": "1.0",+ "name": "Main",+ "entities": [+ {+ "id": 0,+ "name": "Camera",+ "parent": null,+ "children": [],+ "components": [+ { "type": "Transform", "data": { "position": { "x": 0, "y": 0, "z": 10 } } },+ { "type": "Camera", "data": { "projectionType": 1, "orthoSize": 240, "isActive": true, "priority": 0 } }+ ],+ "visible": true+ },+ {+ "id": 1,+ "name": "Preview",+ "parent": null,+ "children": [],+ "components": [+ { "type": "Transform", "data": { "position": { "x": 232, "y": 156, "z": 0 } } },+ {+ "type": "Sprite",+ "data": {+ "size": { "x": 128, "y": 96 },+ "color": { "r": 1, "g": 1, "b": 1, "a": 1 },+ "layer": 1+ }+ },+ {+ "type": "Video",+ "data": {+ "source": "assets/video/clip.mp4",+ "autoplay": true,+ "loop": true,+ "muted": true,+ "volume": 1,+ "playbackRate": 1,+ "fitSize": false,+ "enabled": true+ }+ }+ ],+ "visible": true+ }+ ]+}diff --git a/examples/video-puzzle/assets/scenes/main.esscene.meta b/examples/video-puzzle/assets/scenes/main.esscene.metanew file mode 100644index 000000000..930fd334c--- /dev/null+++ b/examples/video-puzzle/assets/scenes/main.esscene.meta@@ -0,0 +1,6 @@+{+ "uuid": "e926e1ba-d3fe-4760-b6c3-cc88262f395e",+ "version": "2.0",+ "type": "scene",+ "importer": {}+}diff --git a/examples/video-puzzle/assets/video/clip.mp4 b/examples/video-puzzle/assets/video/clip.mp4new file mode 100644index 000000000..a4e72b662Binary files /dev/null and b/examples/video-puzzle/assets/video/clip.mp4 differdiff --git a/examples/video-puzzle/assets/video/clip.mp4.meta b/examples/video-puzzle/assets/video/clip.mp4.metanew file mode 100644index 000000000..2a58c1476--- /dev/null+++ b/examples/video-puzzle/assets/video/clip.mp4.meta@@ -0,0 +1,6 @@+{+ "uuid": "9fa233c3-a560-4280-9416-a6ba482b7c05",+ "version": "2.0",+ "type": "video",+ "importer": {}+}diff --git a/examples/video-puzzle/project.esproject b/examples/video-puzzle/project.esprojectnew file mode 100644index 000000000..582eaf92a--- /dev/null+++ b/examples/video-puzzle/project.esproject@@ -0,0 +1,25 @@+{+ "formatVersion": "1",+ "name": "Video Puzzle",+ "description": "A sliding-tile puzzle whose tiles are live regions of ONE playing video.",+ "tag": "Gameplay",+ "version": "0.1.0",+ "defaultScene": "assets/scenes/main.esscene",+ "designResolution": {+ "width": 640,+ "height": 480+ },+ "spineVersion": "none",+ "packaging": {+ "platform": "wechat",+ "config": "shipping",+ "sourceMaps": false,+ "openFolder": true,+ "outDir": {+ "wechat": "dist-wechat"+ },+ "compressTextures": false,+ "atlasTextures": false,+ "compressAudio": false+ }+}diff --git a/examples/video-puzzle/src/components.ts b/examples/video-puzzle/src/components.tsnew file mode 100644index 000000000..56826d2c3--- /dev/null+++ b/examples/video-puzzle/src/components.ts@@ -0,0 +1,12 @@+import { defineComponent } from 'esengine';++/**+ * One board tile. `slot` is the fixed board cell this entity sits in (never+ * changes); `tile` is which region of the video it currently SHOWS. Swapping+ * two pieces swaps their `tile`s — the entities and their transforms stay put,+ * only the sampled video region moves. Solved = every piece shows its own slot.+ */+export const PuzzlePiece = defineComponent('PuzzlePiece', {+ slot: 0,+ tile: 0,+});diff --git a/examples/video-puzzle/src/main.ts b/examples/video-puzzle/src/main.tsnew file mode 100644index 000000000..d002e0918--- /dev/null+++ b/examples/video-puzzle/src/main.ts@@ -0,0 +1,171 @@+// SPDX-License-Identifier: Apache-2.0+// Video Puzzle — N×N tiles, each a live REGION of one playing video.+//+// The mechanism is the point of this example: there is exactly ONE decoded+// video stream. The scene's "Preview" entity carries Sprite + Video, so the+// engine's video system decodes the clip and keeps that sprite's `texture`+// handle alive. Every puzzle piece simply SHARES that texture handle and picks+// its region with the Sprite's `uvOffset`/`uvScale` — no per-piece decoding,+// no extra API. Click two tiles to swap which region each one shows.+import {+ addSystemToSchedule, defineSystem, Schedule,+ Query, Mut, Res, Commands, Input, UICameraInfo,+ Transform, Sprite, Video,+} from 'esengine';+import type { UICameraData } from 'esengine';+import { PuzzlePiece } from './components';++const GRID = 3; // GRID × GRID tiles+const TILE = 96; // world size of one tile+const GAP = 4; // spacing so the seams read as pieces+const BOARD_X = -96; // board center (preview sits to the right)+const BOARD_Y = 0;++const SELECTED_TINT = { r: 1, g: 0.85, b: 0.4, a: 1 };+const NORMAL_TINT = { r: 1, g: 1, b: 1, a: 1 };+const SOLVED_TINT = { r: 0.65, g: 1, b: 0.7, a: 1 };++const slotX = (slot: number) => BOARD_X + ((slot % GRID) - (GRID - 1) / 2) * (TILE + GAP);+const slotY = (slot: number) => BOARD_Y - (Math.floor(slot / GRID) - (GRID - 1) / 2) * (TILE + GAP);++/**+ * The video texture samples like any flipY-uploaded image: v = 0 is the+ * image's BOTTOM row (the same convention atlas frames use). Tile 0 is the+ * top-left region, so its v-offset is the top band: 1 - 1/GRID.+ */+function tileUv(tile: number): { offset: { x: number; y: number }; scale: { x: number; y: number } } {+ const col = tile % GRID;+ const row = Math.floor(tile / GRID);+ return {+ offset: { x: col / GRID, y: 1 - (row + 1) / GRID },+ scale: { x: 1 / GRID, y: 1 / GRID },+ };+}++/** Fisher–Yates over an even number of transpositions keeps a swap puzzle+ * solvable by construction; reshuffle away the already-solved deal. */+function shuffledTiles(): number[] {+ const tiles = Array.from({ length: GRID * GRID }, (_, i) => i);+ do {+ for (let i = tiles.length - 1; i > 0; i--) {+ const j = Math.floor(Math.random() * (i + 1));+ [tiles[i], tiles[j]] = [tiles[j], tiles[i]];+ }+ } while (tiles.every((t, i) => t === i));+ return tiles;+}++let selectedSlot = -1;+let solved = false;++const spawnBoardSystem = defineSystem(+ [Query(PuzzlePiece), Commands()],+ (pieces, cmds) => {+ if (!pieces.isEmpty()) return;+ selectedSlot = -1;+ solved = false;+ const tiles = shuffledTiles();+ for (let slot = 0; slot < GRID * GRID; slot++) {+ const uv = tileUv(tiles[slot]);+ cmds.spawn()+ .insert(Transform, { position: { x: slotX(slot), y: slotY(slot), z: 0 } })+ .insert(Sprite, {+ size: { x: TILE, y: TILE },+ color: NORMAL_TINT,+ uvOffset: uv.offset,+ uvScale: uv.scale,+ })+ .insert(PuzzlePiece, { slot, tile: tiles[slot] });+ }+ },+ { name: 'SpawnBoardSystem' },+);++/**+ * Mirror the live video texture onto every piece. The Video component keeps+ * the Preview sprite's `texture` current; pieces adopt the same handle — the+ * GPU texture updates in place each frame, so one write per piece per stream+ * start is all the sharing costs.+ */+const shareVideoTextureSystem = defineSystem(+ [Query(Sprite, Video), Query(Mut(Sprite), PuzzlePiece)],+ (preview, pieces) => {+ let texture = 0;+ for (const [, sprite] of preview) {+ texture = sprite.texture;+ break;+ }+ if (!texture) return; // first frame not decoded yet+ for (const [, sprite] of pieces) {+ if (sprite.texture !== texture) sprite.texture = texture;+ }+ },+ { name: 'ShareVideoTextureSystem' },+);++const swapTilesSystem = defineSystem(+ [Res(Input), Res(UICameraInfo), Query(Mut(Sprite), Mut(PuzzlePiece))],+ (input, camera: UICameraData, pieces) => {+ if (!camera.valid || !input.isMouseButtonPressed(0)) return;++ if (solved) { // any click deals a new board+ const tiles = shuffledTiles();+ for (const [, sprite, piece] of pieces) {+ piece.tile = tiles[piece.slot];+ const uv = tileUv(piece.tile);+ sprite.uvOffset = uv.offset;+ sprite.uvScale = uv.scale;+ sprite.color = NORMAL_TINT;+ }+ solved = false;+ return;+ }++ const half = (TILE + GAP) / 2;+ let clickedSlot = -1;+ for (const [, , piece] of pieces) {+ if (Math.abs(camera.worldMouseX - slotX(piece.slot)) <= half &&+ Math.abs(camera.worldMouseY - slotY(piece.slot)) <= half) {+ clickedSlot = piece.slot;+ break;+ }+ }+ if (clickedSlot < 0) return;++ if (selectedSlot < 0) {+ selectedSlot = clickedSlot;+ } else if (selectedSlot !== clickedSlot) {+ // Swap which region the two pieces sample.+ let a: { tile: number } | null = null;+ let b: { tile: number } | null = null;+ for (const [, , piece] of pieces) {+ if (piece.slot === selectedSlot) a = piece;+ if (piece.slot === clickedSlot) b = piece;+ }+ if (a && b) {+ [a.tile, b.tile] = [b.tile, a.tile];+ }+ selectedSlot = -1;+ } else {+ selectedSlot = -1; // clicking the selection again deselects+ }++ let allHome = true;+ for (const [, sprite, piece] of pieces) {+ const uv = tileUv(piece.tile);+ sprite.uvOffset = uv.offset;+ sprite.uvScale = uv.scale;+ if (piece.tile !== piece.slot) allHome = false;+ sprite.color = piece.slot === selectedSlot ? SELECTED_TINT : NORMAL_TINT;+ }+ if (allHome) {+ solved = true;+ for (const [, sprite] of pieces) sprite.color = SOLVED_TINT;+ }+ },+ { name: 'SwapTilesSystem' },+);++addSystemToSchedule(Schedule.Update, spawnBoardSystem);+addSystemToSchedule(Schedule.Update, shareVideoTextureSystem);+addSystemToSchedule(Schedule.Update, swapTilesSystem);diff --git a/examples/video-puzzle/tsconfig.json b/examples/video-puzzle/tsconfig.jsonnew file mode 100644index 000000000..0a8528b29--- /dev/null+++ b/examples/video-puzzle/tsconfig.json@@ -0,0 +1,22 @@+{+ "compilerOptions": {+ "target": "ES2020",+ "module": "ESNext",+ "moduleResolution": "bundler",+ "strict": true,+ "esModuleInterop": true,+ "skipLibCheck": true,+ "forceConsistentCasingInFileNames": true,+ "declaration": false,+ "outDir": "./dist",+ "rootDir": ".",+ "baseUrl": ".",+ "paths": {+ "esengine": ["./.esengine/sdk/index.d.ts"],+ "esengine/wasm": ["./.esengine/sdk/wasm.d.ts"],+ "@esengine/editor": ["./.esengine/editor/index.d.ts"]+ }+ },+ "include": ["src/**/*"],+ "exclude": ["node_modules"]+}