A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(editor): prefab instance expand/collapse round-trip core (REARCH_PREFABS PF2)
esengineb84b7197f43b
diff --git a/desktop/src/engine/PrefabInstance.ts b/desktop/src/engine/PrefabInstance.tsnew file mode 100644index 000000000..643b1034a--- /dev/null+++ b/desktop/src/engine/PrefabInstance.ts@@ -0,0 +1,121 @@+import { flattenPrefab, diffAgainstSource } from 'esengine';+import type { PrefabData, PrefabOverride, ProcessedEntity, PrefabEntityId } from 'esengine';++/**+ * Prefab instance expand/collapse (REARCH_PREFABS.md §3-4, PF2 core).+ *+ * A prefab instance is persisted as a DELTA over its prefab asset and lives in+ * the editor model as EXPANDED entities. This module is the lossless boundary+ * between the two, built on the engine's own prefab data layer+ * (`flattenPrefab` / `diffAgainstSource`):+ *+ * expand: prefab asset + delta → flat instance entities (for the SceneModel)+ * collapse: flat instance entities → delta (for the scene file)+ *+ * A `{prefab, overrides}` pair alone is NOT enough — `PrefabOverride` cannot+ * express structural edits — so the delta also carries `added` (entities added+ * under the instance) and `removed` (prefab children deleted in the instance),+ * sourced from `diffAgainstSource`'s `untracked` / `orphanedSourceIds`. Both+ * structural buckets reference entities by stable `prefabEntityId`, so the round+ * trip survives the id reallocation that `flattenPrefab` performs.+ *+ * Pure data (no World / wasm) — unit-tested as the data-loss safety net before+ * the save path is wired.+ */++/** An entity added under a prefab instance (not part of the prefab asset). */+export interface AddedEntity {+ /** Stable instance-local identity (its prefabEntityId — absent from the asset). */+ prefabEntityId: PrefabEntityId;+ name: string;+ components: ProcessedEntity['components'];+ visible: boolean;+ /** Parent by stable id — a prefab entity OR another added entity; null = under the instance root. */+ parentId: PrefabEntityId | null;+}++/** The persisted form of a prefab instance: a minimal delta over the asset. */+export interface PrefabInstanceDelta {+ /** `@uuid:` ref to the prefab asset. */+ prefab: string;+ /** Property / component / name / visibility / metadata edits. */+ overrides: PrefabOverride[];+ /** Entities added under the instance (not in the asset). */+ added: AddedEntity[];+ /** Prefab children deleted in this instance (by their asset prefabEntityId). */+ removed: PrefabEntityId[];+}++/**+ * Expand a prefab asset + instance delta into flat instance entities, allocating+ * ids via `allocateId`. Reuses `flattenPrefab` (which applies the overrides),+ * drops `removed` prefab entities, and appends `added` entities re-linked by+ * their stable parent id. The returned entities are ready to map into the model.+ */+export function expandInstance(+ prefab: PrefabData,+ delta: PrefabInstanceDelta,+ allocateId: () => number,+): ProcessedEntity[] {+ const { entities, rootId } = flattenPrefab(prefab, delta.overrides, {+ allocateId,+ loadPrefab: () => null, // nested prefabs: PF2-later+ });++ const removed = new Set(delta.removed);+ const kept = entities.filter((e) => !removed.has(e.prefabEntityId));++ // prefabEntityId → runtime id, for both kept prefab entities and added ones.+ const idByPrefabId = new Map<PrefabEntityId, number>();+ for (const e of kept) idByPrefabId.set(e.prefabEntityId, e.id);++ // Allocate added ids first so parent refs resolve regardless of order.+ const added: ProcessedEntity[] = delta.added.map((a) => {+ const id = allocateId();+ idByPrefabId.set(a.prefabEntityId, id);+ return {+ id,+ prefabEntityId: a.prefabEntityId,+ name: a.name,+ parent: rootId,+ children: [],+ components: a.components,+ visible: a.visible,+ };+ });+ delta.added.forEach((a, i) => {+ added[i].parent = a.parentId != null ? (idByPrefabId.get(a.parentId) ?? rootId) : rootId;+ });++ const all = [...kept, ...added];+ // Rebuild children arrays from parent links so the subtree stays consistent+ // (kept entities' children may reference removed ids).+ const byId = new Map(all.map((e) => [e.id, e]));+ for (const e of all) e.children = [];+ for (const e of all) {+ if (e.parent != null) byId.get(e.parent)?.children.push(e.id);+ }+ return all;+}++/**+ * Collapse expanded instance entities back to a delta. `diffAgainstSource`+ * yields the override list + the structural buckets (`untracked` → added,+ * `orphanedSourceIds` → removed), with added parents recorded by stable id.+ */+export function collapseInstance(+ prefab: PrefabData,+ prefabRef: string,+ expanded: readonly ProcessedEntity[],+): PrefabInstanceDelta {+ const { overrides, untracked, orphanedSourceIds } = diffAgainstSource(prefab, expanded);+ const byId = new Map(expanded.map((e) => [e.id, e]));+ const added: AddedEntity[] = untracked.map((u) => ({+ prefabEntityId: u.prefabEntityId,+ name: u.name,+ components: u.components,+ visible: u.visible,+ parentId: u.parent != null ? (byId.get(u.parent)?.prefabEntityId ?? null) : null,+ }));+ return { prefab: prefabRef, overrides, added, removed: orphanedSourceIds };+}diff --git a/desktop/tests/prefab-instance.test.ts b/desktop/tests/prefab-instance.test.tsnew file mode 100644index 000000000..6f5da7121--- /dev/null+++ b/desktop/tests/prefab-instance.test.ts@@ -0,0 +1,113 @@+/**+ * @file Prefab instance expand/collapse round-trip (REARCH_PREFABS.md PF2 core).+ * The data-loss safety net: a prefab instance is saved as a DELTA+ * (overrides + added + removed) and loaded by expanding the prefab asset.+ * This pins `collapse(expand(delta)) === delta` over every bucket —+ * property/name/visibility overrides AND structural add/remove — so the+ * diff-based save path can't silently drop an edit. Pure data (no World).+ */+import { describe, it, expect } from 'vitest';+import { migratePrefabData } from 'esengine';+import type { PrefabData } from 'esengine';+import { expandInstance, collapseInstance, type PrefabInstanceDelta } from '@/engine/PrefabInstance';++function turretPrefab(): PrefabData {+ return migratePrefabData({+ version: '1.0',+ name: 'Turret',+ rootEntityId: 'root',+ entities: [+ {+ prefabEntityId: 'root',+ name: 'Turret',+ parent: null,+ children: ['barrel', 'scope'],+ components: [{ type: 'Transform', data: { position: { x: 0, y: 0, z: 0 } } }],+ visible: true,+ },+ {+ prefabEntityId: 'barrel',+ name: 'Barrel',+ parent: 'root',+ children: [],+ components: [{ type: 'Sprite', data: { color: { r: 1, g: 1, b: 1, a: 1 } } }],+ visible: true,+ },+ {+ prefabEntityId: 'scope',+ name: 'Scope',+ parent: 'root',+ children: [],+ components: [],+ visible: true,+ },+ ],+ }).data as PrefabData;+}++const REF = '@uuid:turret-0000';++describe('Prefab instance expand/collapse round-trip', () => {+ it('round-trips overrides + added + removed (collapse∘expand = identity)', () => {+ const prefab = turretPrefab();+ const delta: PrefabInstanceDelta = {+ prefab: REF,+ overrides: [+ { prefabEntityId: 'root', type: 'property', componentType: 'Transform', propertyName: 'position', value: { x: 5, y: 0, z: 0 } },+ { prefabEntityId: 'barrel', type: 'name', value: 'BigBarrel' },+ { prefabEntityId: 'barrel', type: 'visibility', value: false },+ ],+ added: [+ {+ prefabEntityId: 'addon1',+ name: 'Muzzle',+ components: [{ type: 'Sprite', data: { color: { r: 0, g: 1, b: 0, a: 1 } } }],+ visible: true,+ parentId: 'root',+ },+ ],+ removed: ['scope'],+ };++ let nid = 0;+ const expanded = expandInstance(prefab, delta, () => nid++);+ const back = collapseInstance(prefab, REF, expanded);++ expect(back.prefab).toBe(REF);+ expect(back.removed.sort()).toEqual(['scope']);++ expect(back.added).toHaveLength(1);+ expect(back.added[0].prefabEntityId).toBe('addon1');+ expect(back.added[0].name).toBe('Muzzle');+ expect(back.added[0].parentId).toBe('root');++ const propOv = back.overrides.find(+ (o) => o.type === 'property' && o.prefabEntityId === 'root' && o.propertyName === 'position',+ );+ expect(propOv?.value).toEqual({ x: 5, y: 0, z: 0 });+ expect(back.overrides.find((o) => o.type === 'name' && o.prefabEntityId === 'barrel')?.value).toBe('BigBarrel');+ expect(back.overrides.find((o) => o.type === 'visibility' && o.prefabEntityId === 'barrel')?.value).toBe(false);+ });++ it('expands to the right entities: kept prefab entities (minus removed) + added', () => {+ const prefab = turretPrefab();+ const delta: PrefabInstanceDelta = { prefab: REF, overrides: [], added: [], removed: ['scope'] };+ let nid = 0;+ const expanded = expandInstance(prefab, delta, () => nid++);+ // root + barrel kept; scope removed; nothing added.+ expect(expanded.map((e) => e.prefabEntityId).sort()).toEqual(['barrel', 'root']);+ const root = expanded.find((e) => e.prefabEntityId === 'root')!;+ // root's children no longer reference the removed scope.+ expect(root.children).toHaveLength(1);+ });++ it('a clean instance (no edits) collapses to an empty delta', () => {+ const prefab = turretPrefab();+ const delta: PrefabInstanceDelta = { prefab: REF, overrides: [], added: [], removed: [] };+ let nid = 0;+ const back = collapseInstance(prefab, REF, expandInstance(prefab, delta, () => nid++));+ expect(back.overrides).toEqual([]);+ expect(back.added).toEqual([]);+ expect(back.removed).toEqual([]);+ });+});diff --git a/docs/REARCH_PREFABS.md b/docs/REARCH_PREFABS.mdnew file mode 100644index 000000000..9d4325ece--- /dev/null+++ b/docs/REARCH_PREFABS.md@@ -0,0 +1,195 @@+# REARCH — Prefabs (model-authoritative instances over the engine's prefab module)++Status: **design / proposed**. The payoff layer of the model-authoritative editor+(`REARCH_EDITOR_MODEL.md`) sitting on the asset pipeline (`REARCH_ASSETS.md`).+**Key fact: the engine already implements the entire prefab DATA system** — this+doc wires the editor onto it and closes the one scene-format gap. Not a rewrite.++## 1. What the engine already provides (the big reuse)++`sdk/src/prefab/` is a complete, tested prefab data layer:++- **`PrefabData`** (`.esprefab`): `{ version, name, rootEntityId, entities:+ PrefabEntityData[], basePrefab?, overrides? }`. Entities use a stable string+ `prefabEntityId`, carry `components/parent/children/visible/metadata?` and an+ optional `nestedPrefab`. `basePrefab` gives **variants** (inherit + override).+- **`flattenPrefab(prefab, overrides, ctx)` → `{ entities: ProcessedEntity[],+ rootId }`** — EXPANDS a prefab + its overrides (+ nested + variants) into flat+ entities, allocating runtime ids while preserving each `prefabEntityId`.+- **`diffAgainstSource(source, instance, opts)` → `{ overrides, untracked,+ orphanedSourceIds }`** — COLLAPSES an instance subtree back to a minimal+ `PrefabOverride[]` (float-epsilon aware, ignore-lists, etc.).+- **`PrefabOverride`** covers everything: `property`, `component_added`,+ `component_replaced`, `component_removed`, `name`, `visibility`,+ `metadata_set`, `metadata_removed`.+- Plus `migratePrefabData`, `bucketOverridesByEntity`, `validateOverrides`,+ `cloneComponentData`, `preloadNestedPrefabs`, and the runtime+ `instantiatePrefab(world, prefab, {overrides, parent, assets})` /+ `PrefabServer.instantiate(path)`.++So **expand / collapse / overrides / nesting / variants are DONE**. The architecture+below is the natural one *because the engine already speaks it.*++## 2. Problem: the editor doesn't use it, and scenes can't persist instances++- **No editor prefab workflow.** No "instantiate prefab" command, no editing a+ prefab instance, no open-`.esprefab`-as-document, no override UI. The asset+ pipeline (`REARCH_ASSETS.md` A2/A4) already indexes `.esprefab` + tracks its+ deps, but nothing instantiates one into a scene.+- **The scene format can't persist a prefab instance.** `SceneData` entities are+ `{ id, name, parent, children, components, visible? }`; `loadSceneData`+ (`sdk/src/scene.ts`) has **no prefab handling**. Prefabs can only be+ instantiated programmatically (`PrefabServer.instantiate`), never authored into+ a saved scene with overrides. So there's no `play == ship` for prefab content.++## 3. Principle: instance = a document delta in the FILE, expanded entities in the MODEL++A prefab instance is, **in the scene file, a minimal delta** — `{ prefab: @uuid,+overrides }` — and, **in the open model, the fully-expanded real entities**.+**Load expands** (the engine's `flattenPrefab`); **save collapses** (the engine's+`diffAgainstSource`). The model-authoritative spine *is* the prefab editing engine.++```+.esprefab asset (PrefabData; @uuid-addressed; A2 dep graph already tracks it)+ │ load: flattenPrefab(asset, overrides) → ProcessedEntity[] → SceneModel entities+ │ (real source ids; each tagged with its prefabEntityId + instance root)+ ▼+SceneModel (ordinary entities — Reconciler / SceneQuery / selection / commands /+ undo all operate UNCHANGED; prefab-ness is inert tag metadata)+ │ save: group instance subtree → diffAgainstSource(asset, subtree) → overrides+ ▼+scene file: { id, prefab:@uuid, parent, overrides:[…] } — minimal, propagatable+```++**The core decision — expand-in-model, not keep-collapsed.** The alternative+(store one `{prefab,overrides}` record and expand only at read/projection time)+forces a *second* addressing scheme (instanceRoot + prefabEntityId) through+SceneQuery / Reconciler / selection / commands — a parallel system fighting the+clean single-source-id model. Expand-in-model keeps **one** entity model and+**one** addressing scheme; prefab-ness becomes metadata + a load/save transform.+This is what Unity does internally (instances are real objects + a modification+list; the file stores the delta) and is the cleanest fit for our spine.++### 3.1 A `{prefab, overrides}` delta is NOT enough — structural edits++`PrefabOverride` covers `property` / `component_*` / `name` / `visibility` /+`metadata_*` — but it has **no `entity_added` / `entity_removed`**. By design,+`diffAgainstSource` returns *structural* instance edits separately:++- a child entity **added** to the instance → `untracked: ProcessedEntity[]`+- a prefab child **deleted** in the instance → `orphanedSourceIds: PrefabEntityId[]`++(`sdk/src/prefab/diff.ts`: *"Does NOT yet emit entity_removed overrides … →+`orphanedSourceIds`"*.) So a pure `{prefab, overrides}` entry persists property/+component edits losslessly but **silently loses structural instance edits**.++The complete scene instance entry is therefore:++```+{ id, prefab: @uuid, parent,+ overrides: PrefabOverride[], // diffAgainstSource().overrides+ added: SceneEntity[], // .untracked — entities added under the instance+ removed: PrefabEntityId[] } // .orphanedSourceIds — prefab children deleted here+```++On load: `flattenPrefab(asset, overrides)` → minus `removed` → plus `added`. On+save: write all three buckets from `diffAgainstSource`. This is the modern-Unity+model (added objects + removed components/objects are stored alongside the+modification list). The earlier "minimal delta" framing was incomplete — structural+fidelity requires these buckets (or, the restrictive alternative, disallow+structural edits on instances until the user "unpacks").++## 4. Target architecture (mapped to our components)++1. **Prefab asset** = `.esprefab` (`PrefabData`), already an `@uuid` asset. **Editing+ a prefab = open it as a document in the SAME editor** (it's just entities) —+ zero new authoring UI.+2. **Scene-format gap (the one engine change).** A `SceneData` entity may carry+ `{ prefab: <ref>, overrides: PrefabOverride[] }` (the instance root). `loadSceneData`+ detects it, loads the prefab asset, calls `flattenPrefab`, and spawns the+ subtree. This is a small, clean completion of the existing prefab module — and+ gives `play == ship` for prefab scenes (the runtime loads the same delta).+3. **Editor load — expand.** `ProjectStore.loadCurrentScene` (which already owns+ asset access) expands each prefab-instance entry via `flattenPrefab` into+ SceneModel entities with fresh source ids, tagging each with its+ `prefabEntityId` + the instance-root source id (in `entity.metadata`, the field+ the engine designed to survive round-trips). Then `Reconciler.adopt` as usual.+4. **Editor save — collapse.** `serializeCurrent` groups each instance subtree (by+ the instance-root tag), maps it to `ProcessedEntity[]`, and calls+ `diffAgainstSource(prefabAsset, subtree)` → overrides → writes a single+ `{ id, prefab, parent, overrides }` entry; non-prefab entities serialize normally.+5. **Instantiate command.** Drag a `.esprefab` from the Content Browser into the+ scene → `SceneCommands.instantiatePrefab(ref, parent)` → `flattenPrefab` → add+ the tagged entities to the model (ordinary `addEntity` sequence) → the+ Reconciler spawns them. Undoable — and it reuses the recursive-delete subtree+ path already built.+6. **Overrides are free.** Any ordinary edit inside an instance (setField /+ addComponent / reparent / rename / visibility) is captured at save by+ `diffAgainstSource`. No per-edit override bookkeeping.+7. **Nesting + variants are free** — `flattenPrefab`/`diffAgainstSource` already+ handle `nestedPrefab` + `basePrefab`; the A2 dep graph already records nested+ prefab refs.+8. **Propagation.** On a prefab-asset change (the asset DB watch, `REARCH_ASSETS.md`+ A2c), re-expand the open scene's instances of that prefab: re-`flattenPrefab`+ with the instance's current overrides, preserving them. Closed scenes pick up+ the new prefab on next load.++The Reconciler, SceneQuery, selection, commands, and undo need **no changes** —+they see ordinary entities. Only the load/save boundary (ProjectStore) and one+new command know prefabs exist.++## 5. How each goal is met (reuse-first)++| Goal | How |+|---|---|+| Expand / collapse / overrides | The engine's `flattenPrefab` / `diffAgainstSource` / `PrefabOverride` — reused verbatim. |+| Compact, portable scene files | Instances persist as a delta — `{ prefab:@uuid, overrides, added, removed }` (§3.1) — not the expanded subtree. |+| Edit instances naturally | Expanded entities are ordinary model entities — the whole editor works unchanged. |+| `play == ship` | Both editor + runtime expand the same delta via `flattenPrefab` (needs the §4.2 loader gap). |+| Nesting + variants | Already in the prefab module. |+| Prefab authoring UI | Open `.esprefab` as a scene document — reuses the editor. |+| Asset integration | Prefab = `@uuid` asset; A2 dep graph + A4 cook already include prefabs. |++## 6. Phased plan++- **PF1 — scene-format + `loadSceneData` prefab-instance support** (the one engine+ change; `sdk/src/scene.ts`, reusing `sdk/src/prefab/flatten`). A scene entity may+ be `{ prefab, overrides }`; load expands it. Unit-testable in node. Unblocks+ `play == ship` for prefab scenes.+- **PF2 — editor instantiate + load-expand + save-collapse** on the SceneModel:+ `SceneCommands.instantiatePrefab`, ProjectStore expand-on-load / collapse-on-save+ via `flattenPrefab` / `diffAgainstSource`, persisting all three buckets+ (`overrides` + `added` + `removed`, §3.1), instance-tag metadata. Drag-from-+ Content-Browser to instantiate. **Ship the round-trip safety test FIRST** (see §7).+- **PF3 — override UX**: highlight overridden fields in the Inspector (diff vs the+ prefab), Revert-override, Apply-instance-to-prefab (writes the asset),+ open-`.esprefab`-as-document, and propagation on asset change (A2c).++Prereq: PF2/3 need the asset DB (A2, done) to read prefab assets for+expand/diff; PF1 needs the prefab module (done).++## 7. Open decisions++- **Round-trip is the safety net (REQUIRED, do first).** Save persists via+ `diffAgainstSource`, so a diff/flatten gap = silent data loss. Before wiring+ save, land a test asserting `expand(collapse(instance)) === instance` for a+ scene of instances exercising every override type + structural `added`/`removed`+ (§3.1). The engine module is well-covered (`sdk/tests/prefab*.test.ts`, ~82),+ but the editor's collapse→file→expand round trip is new and must be pinned.+- **Structural instance edits** are handled by the three-bucket entry (§3.1) —+ decided, not open; the only sub-choice is allow-now vs. require-unpack-first.+- **Delta-in-scene (optimal) vs baked-in-scene (no engine change).** §4.2 (delta ++ `loadSceneData` expansion) is optimal — minimal files, propagation, `play==ship`+ — but touches `sdk/src/scene.ts`. A no-engine-change fallback BAKES the expanded+ subtree into the scene with `entity.metadata` prefab tags (the runtime ignores+ metadata; the editor re-expands on propagation): larger files, fully editor-side.+ **Recommend delta** — it's the right architecture and the prefab module makes the+ loader change small; the sdk touch is a legitimate engine completion.+- **source id ↔ `prefabEntityId` mapping.** The editor allocates scene source ids+ on expand and stores the origin `prefabEntityId` (+ instance-root id) in+ `entity.metadata`; `diffAgainstSource` matches by `prefabEntityId` at save.+- **Propagation timing.** Re-expand open instances eagerly on the asset DB change+ event (A2c), preserving overrides; vs. lazy on next load only. Eager is the+ better UX once A2c lands.+- **Variants in the editor** — `basePrefab` works at the data layer; surfacing+ "create variant" in the UI is a PF3+ nicety, not required for PF1/2.