Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(native): a native build's frame can be captured and replayed

esenginebdf40ea305c5
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fe95a024d..00a9779d5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- The Frame Debugger captures and replays the frame of a connected native build (desktop and Android) as it does a web or mini-game one.
- A native development build names its device to the editor — `macOS · Mac16,10`, `Android · Pixel 8` — as web and mini-game builds already did.
- A desktop (native) development build connects to the editor like a web or mini-game one — log, profiler, the world, content status, pause and step; its renderer cannot capture frames yet and says so. The native host gained a `ws://` client for it.
- Profiling a connected development build, the Profiler shows which content revision it runs, the one its next launch starts on, and a hot update it has found but not applied; `device_update_status` answers the same for an agent.
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 5d7ebbf81..17e699813 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -10145,7 +10145,7 @@ MotionDriver<AnimatorBlend2DMotion>
## captureFrame — function @experimental
```
-(module: ESEngineModule, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
+(module: CaptureEngine, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
```
## captureFramePixels — function @beta
@@ -12000,7 +12000,7 @@ PrefabsPlugin
## replayDraw — function @experimental
```
-(module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
+(module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
```
## replayFrameDraw — function @experimental
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 3f477783e..b709944ba 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -10361,7 +10361,7 @@ MotionDriver<AnimatorBlend2DMotion>
## captureFrame — function @experimental
```
-(module: ESEngineModule, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
+(module: CaptureEngine, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
```
## captureFramePixels — function @beta
@@ -12251,7 +12251,7 @@ PrefabsPlugin
## replayDraw — function @experimental
```
-(module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
+(module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
```
## replayFrameDraw — function @experimental
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index 634456b3f..e8ffc2ddc 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -10140,7 +10140,7 @@ MotionDriver<AnimatorBlend2DMotion>
## captureFrame — function @experimental
```
-(module: ESEngineModule, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
+(module: CaptureEngine, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
```
## captureFramePixels — function @beta
@@ -12005,7 +12005,7 @@ PrefabsPlugin
## replayDraw — function @experimental
```
-(module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
+(module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
```
## replayFrameDraw — function @experimental
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index e52cdc23e..77fb91427 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -10416,7 +10416,7 @@ MotionDriver<AnimatorBlend2DMotion>
## captureFrame — function @experimental
```
-(module: ESEngineModule, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
+(module: CaptureEngine, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
```
## captureFramePixels — function @beta
@@ -12286,7 +12286,7 @@ PrefabsPlugin
## replayDraw — function @experimental
```
-(module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
+(module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
```
## replayFrameDraw — function @experimental
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index 342edf4b0..c5bf5d782 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -10171,7 +10171,7 @@ MotionDriver<AnimatorBlend2DMotion>
## captureFrame — function @experimental
```
-(module: ESEngineModule, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
+(module: CaptureEngine, nextFrame: NextFrame, maxFrames?: number): Promise<FrameCaptureData | null>
```
## captureFramePixels — function @beta
@@ -12036,7 +12036,7 @@ PrefabsPlugin
## replayDraw — function @experimental
```
-(module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
+(module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames?: number): Promise<ReplaySnapshot | null>
```
## replayFrameDraw — function @experimental
diff --git a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
index 6a864d982..9383ed597 100644
--- a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
+++ b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
@@ -125,6 +125,10 @@ export interface NativeEngineApi {
renderer_beginScreenOverlay?(projectionPtr: number, vpX: number, vpY: number, vpW: number, vpH: number): void;
renderer_captureFrame?(w: number, h: number): number;
renderer_captureNextFrame?(): void;
+ renderer_copyCapturedEntities?(dest: number, destSize: number): boolean;
+ renderer_copyCapturedRecords?(dest: number, destSize: number): boolean;
+ renderer_copyCapturedTextures?(dest: number, destSize: number): boolean;
+ renderer_copySnapshot?(dest: number, destSize: number): boolean;
renderer_createSkeletalPreview?(w: number, h: number): number;
renderer_createTarget?(width: number, height: number, flags: number): number;
renderer_destroySkeletalPreview?(preview: number): void;
@@ -397,6 +401,10 @@ export function createNativeEngineApi(
bind('renderer_beginScreenOverlay', 'es_renderer_beginScreenOverlay', false);
bind('renderer_captureFrame', 'es_renderer_captureFrame', false);
bind('renderer_captureNextFrame', 'es_renderer_captureNextFrame', false);
+ bind('renderer_copyCapturedEntities', 'es_renderer_copyCapturedEntities', false);
+ bind('renderer_copyCapturedRecords', 'es_renderer_copyCapturedRecords', false);
+ bind('renderer_copyCapturedTextures', 'es_renderer_copyCapturedTextures', false);
+ bind('renderer_copySnapshot', 'es_renderer_copySnapshot', false);
bind('renderer_createSkeletalPreview', 'es_renderer_createSkeletalPreview', false);
bind('renderer_createTarget', 'es_renderer_createTarget', false);
bind('renderer_destroySkeletalPreview', 'es_renderer_destroySkeletalPreview', false);
diff --git a/sdk/src/render/frameCapture.ts b/sdk/src/render/frameCapture.ts
index 51f18215b..bfd9ef508 100644
--- a/sdk/src/render/frameCapture.ts
+++ b/sdk/src/render/frameCapture.ts
@@ -1,6 +1,5 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
-import type { ESEngineModule } from '../wasm';
import { awaitReadback, READBACK_READY } from './readback';
/**
@@ -105,22 +104,84 @@ export interface FrameCaptureData {
/** sizeof(DrawCallRecord) in FrameCapture.hpp, which asserts the offsets read here. */
const RECORD_SIZE_BYTES = 84;
-export function decodeFrameCapture(module: ESEngineModule): FrameCaptureData | null {
+/**
+ * What capture asks of the engine: the wasm module, which hands out addresses in
+ * its heap, or a native host's surface, which copies into a buffer allocated in
+ * its heap arena — its script cannot read engine memory directly.
+ */
+export interface CaptureEngine {
+ HEAPU8: Uint8Array;
+ _malloc?(bytes: number): number;
+ _free?(offset: number): void;
+ renderer_captureNextFrame(): void;
+ renderer_hasCapturedData(): boolean;
+ renderer_getCapturedFrameSize(): number;
+ renderer_getCapturedEntityCount(): number;
+ renderer_getCapturedTextureCount(): number;
+ renderer_getCapturedPassCount(): number;
+ renderer_getCapturedFrameData?(): number;
+ renderer_getCapturedEntities?(): number;
+ renderer_getCapturedTextures?(): number;
+ renderer_copyCapturedRecords?(dest: number, destSize: number): boolean;
+ renderer_copyCapturedEntities?(dest: number, destSize: number): boolean;
+ renderer_copyCapturedTextures?(dest: number, destSize: number): boolean;
+ renderer_replayToDrawCall(drawCallIndex: number): void;
+ renderer_snapshotMatchesCapture(): boolean;
+ renderer_pollSnapshotReadback(): number;
+ renderer_getSnapshotPtr?(): number;
+ renderer_getSnapshotSize(): number;
+ renderer_getSnapshotWidth(): number;
+ renderer_getSnapshotHeight(): number;
+ renderer_copySnapshot?(dest: number, destSize: number): boolean;
+}
+
+/** @p bytes of engine data as a JS-owned copy: read at its address where the
+ * engine gives one, else copied through a scratch buffer in the heap. */
+function readOut(
+ engine: CaptureEngine, bytes: number,
+ address: (() => number) | undefined,
+ copy: ((dest: number, destSize: number) => boolean) | undefined,
+): Uint8Array | null {
+ if (bytes === 0) return new Uint8Array(0);
+ if (address) {
+ const at = address();
+ return at ? engine.HEAPU8.slice(at, at + bytes) : null;
+ }
+ if (!copy || !engine._malloc || !engine._free) return null;
+ const scratch = engine._malloc(bytes);
+ if (!scratch) return null;
+ try {
+ return copy(scratch, bytes) ? engine.HEAPU8.slice(scratch, scratch + bytes) : null;
+ } finally {
+ engine._free(scratch);
+ }
+}
+
+export function decodeFrameCapture(module: CaptureEngine): FrameCaptureData | null {
if (!module.renderer_hasCapturedData()) return null;
const count = module.renderer_getCapturedFrameSize();
if (count === 0) return null;
- const dataPtr = module.renderer_getCapturedFrameData();
- const entitiesPtr = module.renderer_getCapturedEntities();
const entityCount = module.renderer_getCapturedEntityCount();
const passCount = module.renderer_getCapturedPassCount();
-
- const heap = module.HEAPU8;
- const view = new DataView(heap.buffer, dataPtr, count * RECORD_SIZE_BYTES);
- const entityHeap = new Uint32Array(heap.buffer, entitiesPtr, entityCount);
const textureCount = module.renderer_getCapturedTextureCount();
- const textureHeap = new Uint32Array(heap.buffer, module.renderer_getCapturedTextures(), textureCount);
+
+ const m = module;
+ const records = readOut(m, count * RECORD_SIZE_BYTES,
+ m.renderer_getCapturedFrameData && (() => m.renderer_getCapturedFrameData!()),
+ m.renderer_copyCapturedRecords && ((d, n) => m.renderer_copyCapturedRecords!(d, n)));
+ const entityBytes = readOut(m, entityCount * 4,
+ m.renderer_getCapturedEntities && (() => m.renderer_getCapturedEntities!()),
+ m.renderer_copyCapturedEntities && ((d, n) => m.renderer_copyCapturedEntities!(d, n)));
+ const textureBytes = readOut(m, textureCount * 4,
+ m.renderer_getCapturedTextures && (() => m.renderer_getCapturedTextures!()),
+ m.renderer_copyCapturedTextures && ((d, n) => m.renderer_copyCapturedTextures!(d, n)));
+ if (!records || !entityBytes || !textureBytes) return null;
+
+ const view = new DataView(records.buffer);
+ const entityHeap = new Uint32Array(entityBytes.buffer);
+ const textureHeap = new Uint32Array(textureBytes.buffer);
const drawCalls: DrawCallInfo[] = [];
for (let i = 0; i < count; i++) {
@@ -172,13 +233,13 @@ export function decodeFrameCapture(module: ESEngineModule): FrameCaptureData | n
* them on a later frame, right after the pass that drew them, so frames have to
* keep rendering until {@link getSnapshotImageData} resolves.
*/
-export function replayToDrawCall(module: ESEngineModule, drawCallIndex: number): void {
+export function replayToDrawCall(module: CaptureEngine, drawCallIndex: number): void {
module.renderer_replayToDrawCall(drawCallIndex);
}
/** Whether the replayed pass made as many draws as the capture did; false means
* the scene changed since, and the snapshot shows the later frame. */
-export function snapshotMatchesCapture(module: ESEngineModule): boolean {
+export function snapshotMatchesCapture(module: CaptureEngine): boolean {
return module.renderer_snapshotMatchesCapture();
}
@@ -187,7 +248,7 @@ export function snapshotMatchesCapture(module: ESEngineModule): boolean {
* seam: immediate on GL, a later event-loop turn on WebGPU). Null when no
* snapshot is in flight or the readback failed.
*/
-export async function getSnapshotImageData(module: ESEngineModule): Promise<ImageData | null> {
+export async function getSnapshotImageData(module: CaptureEngine): Promise<ImageData | null> {
if (await awaitReadback(() => module.renderer_pollSnapshotReadback()) !== READBACK_READY) {
return null;
}
@@ -198,9 +259,12 @@ export async function getSnapshotImageData(module: ESEngineModule): Promise<Imag
const h = module.renderer_getSnapshotHeight();
if (w === 0 || h === 0) return null;
- const ptr = module.renderer_getSnapshotPtr();
- const heap = module.HEAPU8;
- const pixels = new Uint8ClampedArray(heap.buffer, ptr, size);
+ const m = module;
+ const bytes = readOut(m, size,
+ m.renderer_getSnapshotPtr && (() => m.renderer_getSnapshotPtr!()),
+ m.renderer_copySnapshot && ((d, n) => m.renderer_copySnapshot!(d, n)));
+ if (!bytes) return null;
+ const pixels = new Uint8ClampedArray(bytes.buffer);
const flipped = new Uint8ClampedArray(size);
const rowBytes = w * 4;
@@ -224,7 +288,7 @@ export type NextFrame = () => Promise<void>;
* rendered and none of them drew anything.
*/
export async function captureFrame(
- module: ESEngineModule, nextFrame: NextFrame, maxFrames = 8,
+ module: CaptureEngine, nextFrame: NextFrame, maxFrames = 8,
): Promise<FrameCaptureData | null> {
module.renderer_captureNextFrame();
for (let i = 0; i < maxFrames; i++) {
@@ -243,7 +307,7 @@ export interface ReplaySnapshot {
/** The captured frame's pass, drawn up to and including draw @p drawCallIndex. */
export async function replayDraw(
- module: ESEngineModule, drawCallIndex: number, nextFrame: NextFrame, maxFrames = 8,
+ module: CaptureEngine, drawCallIndex: number, nextFrame: NextFrame, maxFrames = 8,
): Promise<ReplaySnapshot | null> {
if (drawCallIndex < 0 || drawCallIndex >= module.renderer_getCapturedFrameSize()) return null;
module.renderer_replayToDrawCall(drawCallIndex);
diff --git a/sdk/src/render/frameDebugReport.ts b/sdk/src/render/frameDebugReport.ts
index 64b818a71..06e2b5487 100644
--- a/sdk/src/render/frameDebugReport.ts
+++ b/sdk/src/render/frameDebugReport.ts
@@ -12,7 +12,8 @@
import type { App } from '../app/app';
import { Assets } from '../asset/AssetPlugin';
import { Name } from '../ecs/component';
-import { captureFrame, replayDraw, type DrawCallInfo, type FrameCaptureData, type NextFrame } from './frameCapture';
+import { captureFrame, replayDraw, type CaptureEngine, type DrawCallInfo, type FrameCaptureData, type NextFrame } from './frameCapture';
+import { engineApi } from '../ecs/bridge/engineApi';
export interface FrameDebugReport {
passCount: number;
@@ -71,7 +72,7 @@ export function frameDebugReport(
export async function captureFrameReport(
app: App, nextFrame: NextFrame, pathOf: (path: string) => string | null = (p) => p,
): Promise<FrameDebugReport | null> {
- const m = app.wasmModule;
+ const m = captureEngineOf(app);
if (!m) return null;
const cap = await captureFrame(m, nextFrame);
if (!cap) return null;
@@ -85,9 +86,16 @@ export async function captureFrameReport(
});
}
+/** The engine surface capture runs on — the wasm module, or a native host's — or
+ * null on a host without one. */
+export function captureEngineOf(app: App): CaptureEngine | null {
+ const e = engineApi(app) as Partial<CaptureEngine> | null;
+ return e && typeof e.renderer_captureNextFrame === 'function' && e.HEAPU8 ? e as CaptureEngine : null;
+}
+
/** The last capture's pass in @p app, drawn up to and including @p drawIndex. */
export async function replayFrameDraw(app: App, drawIndex: number, nextFrame: NextFrame): Promise<FrameReplayImage | null> {
- const m = app.wasmModule;
+ const m = captureEngineOf(app);
const snap = m ? await replayDraw(m, drawIndex, nextFrame) : null;
return snap && {
width: snap.image.width, height: snap.image.height,
diff --git a/sdk/src/runtime/debugChannel.ts b/sdk/src/runtime/debugChannel.ts
index b88a813bb..71157d519 100644
--- a/sdk/src/runtime/debugChannel.ts
+++ b/sdk/src/runtime/debugChannel.ts
@@ -15,7 +15,7 @@ import type { App } from '../app/app';
import { getPlatform, platformNow } from '../platform/base';
import type { PlatformSocket } from '../platform/types';
import type { NextFrame } from '../render/frameCapture';
-import { captureFrameReport, replayFrameDraw, type FrameReplayImage } from '../render/frameDebugReport';
+import { captureFrameReport, captureEngineOf, replayFrameDraw, type FrameReplayImage } from '../render/frameDebugReport';
import { log } from '../util/logger';
import { Assets } from '../asset/AssetPlugin';
import { Lifecycle } from '../ecs/lifecycle';
@@ -133,8 +133,8 @@ export function startDebugChannel(config: DebugChannelConfig): void {
if ((q.kind === 'frameCapture' || q.kind === 'frameReplay') && !drawsFrames) {
throw new Error('the game is in the background, where it draws no frames: bring it to the front');
}
- if ((q.kind === 'frameCapture' || q.kind === 'frameReplay') && !game.wasmModule) {
- throw new Error('this build cannot capture frames: a native host has no frame capture yet');
+ if ((q.kind === 'frameCapture' || q.kind === 'frameReplay') && !captureEngineOf(game)) {
+ throw new Error('this build cannot capture frames: its host has no frame capture');
}
if (q.kind === 'frameCapture') {
const report = await captureFrameReport(game, frame);
diff --git a/sdk/tests/debug-channel.test.ts b/sdk/tests/debug-channel.test.ts
index 06dcfbf5f..ec2e57cf3 100644
--- a/sdk/tests/debug-channel.test.ts
+++ b/sdk/tests/debug-channel.test.ts
@@ -76,7 +76,7 @@ describe('a build whose renderer cannot capture', () => {
channel.attachDebugChannel({ hasResource: () => false, onFrameEnd: () => () => {}, wasmModule: null } as unknown as App);
s.receive({ t: 'query', reqId: 12, kind: 'frameCapture' });
await flush();
- expect(s.sent.find((m) => (m as { reqId?: number }).reqId === 12)).toMatchObject({ error: expect.stringMatching(/native host/) });
+ expect(s.sent.find((m) => (m as { reqId?: number }).reqId === 12)).toMatchObject({ error: expect.stringMatching(/no frame capture/) });
});
});
diff --git a/sdk/tests/frame-capture-copy.test.ts b/sdk/tests/frame-capture-copy.test.ts
new file mode 100644
index 000000000..84c8e734a
--- /dev/null
+++ b/sdk/tests/frame-capture-copy.test.ts
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file A native host hands its captured frame out by copying into a buffer in
+ * its heap, where the wasm module hands out addresses; the frame decoded
+ * either way is the same frame.
+ */
+import { describe, it, expect } from 'vitest';
+import { decodeFrameCapture, getSnapshotImageData, type CaptureEngine } from '../src/render/frameCapture';
+
+const RECORD = 84;
+
+function engine(byAddress: boolean): CaptureEngine {
+ const heap = new Uint8Array(4096);
+ const recordsAt = 64, entitiesAt = 512, texturesAt = 768, pixelsAt = 1024;
+ const view = new DataView(heap.buffer);
+ for (let i = 0; i < 2; i++) {
+ const o = recordsAt + i * RECORD;
+ view.setUint32(o, i, true);
+ view.setUint32(o + 4, 1, true);
+ view.setUint32(o + 24, 6 * (i + 1), true);
+ view.setUint32(o + 32, 1, true);
+ view.setUint32(o + 36, i, true);
+ view.setUint8(o + 44, 6);
+ view.setUint8(o + 72, 1);
+ view.setUint32(o + 80, i, true);
+ }
+ new Uint32Array(heap.buffer, entitiesAt, 2).set([4194305, 4194306]);
+ new Uint32Array(heap.buffer, texturesAt, 2).set([7, 9]);
+ heap.set([10, 20, 30, 255, 40, 50, 60, 255], pixelsAt);
+ let next = 2048;
+ const copier = (at: number, bytes: number) => (dest: number, size: number) => {
+ if (size < bytes) return false;
+ heap.copyWithin(dest, at, at + bytes);
+ return true;
+ };
+ const base = {
+ HEAPU8: heap,
+ renderer_captureNextFrame: () => {},
+ renderer_hasCapturedData: () => true,
+ renderer_getCapturedFrameSize: () => 2,
+ renderer_getCapturedEntityCount: () => 2,
+ renderer_getCapturedTextureCount: () => 2,
+ renderer_getCapturedPassCount: () => 1,
+ renderer_replayToDrawCall: () => {},
+ renderer_snapshotMatchesCapture: () => true,
+ renderer_pollSnapshotReadback: () => 1,
+ renderer_getSnapshotSize: () => 8,
+ renderer_getSnapshotWidth: () => 1,
+ renderer_getSnapshotHeight: () => 2,
+ };
+ return byAddress
+ ? {
+ ...base,
+ renderer_getCapturedFrameData: () => recordsAt,
+ renderer_getCapturedEntities: () => entitiesAt,
+ renderer_getCapturedTextures: () => texturesAt,
+ renderer_getSnapshotPtr: () => pixelsAt,
+ }
+ : {
+ ...base,
+ _malloc: (bytes: number) => { const at = next; next += bytes; return at; },
+ _free: () => {},
+ renderer_copyCapturedRecords: copier(recordsAt, 2 * RECORD),
+ renderer_copyCapturedEntities: copier(entitiesAt, 8),
+ renderer_copyCapturedTextures: copier(texturesAt, 8),
+ renderer_copySnapshot: copier(pixelsAt, 8),
+ };
+}
+
+describe('a captured frame', () => {
+ it('decodes the same whether the engine gives addresses or copies', () => {
+ const byAddress = decodeFrameCapture(engine(true));
+ const byCopy = decodeFrameCapture(engine(false));
+ expect(byCopy).toEqual(byAddress);
+ expect(byCopy?.drawCalls.map((d) => [d.index, d.indexCount, d.entities, d.textures])).toEqual([
+ [0, 6, [4194305], [7]],
+ [1, 12, [4194306], [9]],
+ ]);
+ });
+
+ it('replays to the same pixels either way', async () => {
+ const a = await getSnapshotImageData(engine(true));
+ const b = await getSnapshotImageData(engine(false));
+ expect(Array.from(b!.data)).toEqual(Array.from(a!.data));
+ expect(Array.from(b!.data)).toEqual([40, 50, 60, 255, 10, 20, 30, 255]);
+ });
+
+ it('is nothing, not garbage, on a host that can neither give addresses nor copy', () => {
+ const e = engine(false);
+ delete e.renderer_copyCapturedRecords;
+ expect(decodeFrameCapture(e)).toBeNull();
+ });
+});
diff --git a/sdk/tests/frame-capture.test.ts b/sdk/tests/frame-capture.test.ts
index c2d61f88e..55dea6f4e 100644
--- a/sdk/tests/frame-capture.test.ts
+++ b/sdk/tests/frame-capture.test.ts
@@ -42,13 +42,14 @@ function buildModule(opts: {
const drawCallsSize = drawCallCount * RECORD_SIZE;
// Layout: [snapshot | entities | drawCalls]
- const totalSize = snapshotSize + entitiesSize + drawCallsSize + 16;
+ const totalSize = 16 + snapshotSize + entitiesSize + drawCallsSize + 16;
const buffer = new ArrayBuffer(totalSize);
const heap = new Uint8Array(buffer);
- const snapshotPtr = 0;
- const entitiesPtr = snapshotSize;
- const dataPtr = snapshotSize + entitiesSize;
+ // Nothing at address 0: the engine answers 0 for "no data", as wasm never allocates there.
+ const snapshotPtr = 16;
+ const entitiesPtr = snapshotPtr + snapshotSize;
+ const dataPtr = entitiesPtr + entitiesSize;
// Fill snapshot with a recognizable pattern: each pixel is (i % 256).
for (let i = 0; i < snapshotSize; i++) {
diff --git a/src/esengine/bindings/RendererBindings.cpp b/src/esengine/bindings/RendererBindings.cpp
index 351f0c34c..dcc32536a 100644
--- a/src/esengine/bindings/RendererBindings.cpp
+++ b/src/esengine/bindings/RendererBindings.cpp
@@ -1248,6 +1248,37 @@ i32 renderer_pollSnapshotReadback() {
return g_renderFrame ? g_renderFrame->pollSnapshotReadback() : 2;
}
+namespace {
+bool copyOut(const void* src, usize bytes, uintptr_t dest, u32 destSize) {
+ if (!src || !dest || bytes == 0 || destSize < bytes) return false;
+ std::memcpy(reinterpret_cast<void*>(dest), src, bytes);
+ return true;
+}
+} // namespace
+
+bool renderer_copyCapturedRecords(uintptr_t dest, u32 destSize) {
+ if (!g_renderFrame) return false;
+ const auto& c = g_renderFrame->frameCapture();
+ return copyOut(c.getRecords(), usize(c.getRecordCount()) * sizeof(DrawCallRecord), dest, destSize);
+}
+
+bool renderer_copyCapturedEntities(uintptr_t dest, u32 destSize) {
+ if (!g_renderFrame) return false;
+ const auto& c = g_renderFrame->frameCapture();
+ return copyOut(c.getEntities(), usize(c.getEntityCount()) * sizeof(Entity), dest, destSize);
+}
+
+bool renderer_copyCapturedTextures(uintptr_t dest, u32 destSize) {
+ if (!g_renderFrame) return false;
+ const auto& c = g_renderFrame->frameCapture();
+ return copyOut(c.getTextures(), usize(c.getTextureCount()) * sizeof(u32), dest, destSize);
+}
+
+bool renderer_copySnapshot(uintptr_t dest, u32 destSize) {
+ if (!g_renderFrame) return false;
+ return copyOut(g_renderFrame->getSnapshotPixels(), g_renderFrame->getSnapshotSize(), dest, destSize);
+}
+
uintptr_t renderer_getSnapshotPtr() {
if (!g_renderFrame) return 0;
return reinterpret_cast<uintptr_t>(g_renderFrame->getSnapshotPixels());
diff --git a/src/esengine/bindings/RendererBindings.hpp b/src/esengine/bindings/RendererBindings.hpp
index 64a6508dd..0a2c96064 100644
--- a/src/esengine/bindings/RendererBindings.hpp
+++ b/src/esengine/bindings/RendererBindings.hpp
@@ -256,6 +256,14 @@ u32 renderer_getSnapshotSize();
u32 renderer_getSnapshotWidth();
u32 renderer_getSnapshotHeight();
+/** The captured frame and the replay snapshot copied into the caller's buffer,
+ * for a host whose script cannot read engine memory (native). False when there
+ * is nothing to copy or @p destSize is too small for it. */
+bool renderer_copyCapturedRecords(uintptr_t dest, u32 destSize);
+bool renderer_copyCapturedEntities(uintptr_t dest, u32 destSize);
+bool renderer_copyCapturedTextures(uintptr_t dest, u32 destSize);
+bool renderer_copySnapshot(uintptr_t dest, u32 destSize);
+
// A skeletal preview is owned by whoever created it: every call names the
// handle, so two of them cannot reach each other and there is no "current" one.
u32 renderer_createSkeletalPreview(i32 w, i32 h);
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 312221a57..526646d74 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -1140,4 +1140,6 @@ export const NOTED = {
{ internal: 'A test of the generated Android Studio project; nothing a creator sees changes.' },
'fix(android): every Android package fills its manifest from one list':
{ note: 'A native Android app can reach the network: its manifest never asked for it, so a remote group\'s assets, hot updates and the debug channel all failed on Android. A development build also allows plain http (a LAN CDN); a shipping build is held to https, and the export warns when its CDN is http.' },
+ 'feat(native): a native build\'s frame can be captured and replayed':
+ { note: 'The Frame Debugger captures and replays the frame of a connected native build (desktop and Android) as it does a web or mini-game one.' },
};