A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(editor): unify asset resolution onto the engine Assets (REARCH_ASSETS A1)
esengine4a00c3de3cdc
diff --git a/desktop/src/project/ProjectStore.ts b/desktop/src/project/ProjectStore.tsindex 493be8d57..a59dc8c4b 100644--- a/desktop/src/project/ProjectStore.ts+++ b/desktop/src/project/ProjectStore.ts@@ -17,16 +17,20 @@ import { resolveLayout, WORKSPACE_DIR, type OpenedProject, type ProjectLayout, t* directory, loads its scene into the live engine World via `resetWorldTo`, and* saves back. The bridge sandboxes every fs path to the open project root.*- * Texture `@uuid:` refs resolve through the `estella://` custom protocol- * (electron/main) which serves the project's files — so project textures- * actually render. On save, texture handles are mapped back to `@uuid:` using- * each component's asset fields, keeping saved scenes portable.+ * Assets resolve through the engine's own asset system (REARCH_ASSETS.md A1):+ * the editor builds a uuid→path registry from `.meta` sidecars, points the+ * engine `Assets` loader at the `estella://` transport (electron/main serves+ * project files), and preloads EVERY referenced asset type — not just textures.+ * The lossless model keeps `@uuid:` refs verbatim, so save stays portable.*/const UUID_PREFIX = '@uuid:';-interface AssetsLike {- loadTexture(ref: string): Promise<{ handle: number }>;+/** The subset of the engine's SceneAssetResult the Reconciler resolver reads. */+interface PreloadResult {+ textureHandles: Map<string, number>;+ materialHandles: Map<string, number>;+ fontHandles: Map<string, number>;}interface ProjectState {@@ -51,41 +55,14 @@ function unknownComponentTypes(data: SceneData): string[] {return [...unknown];}-/** Collect every `@uuid:<id>` referenced anywhere in the scene. */-function collectUuids(value: unknown, into: Set<string>): void {- if (typeof value === 'string') {- if (value.startsWith(UUID_PREFIX)) into.add(value.slice(UUID_PREFIX.length));- return;- }- if (Array.isArray(value)) {- for (const v of value) collectUuids(v, into);- return;- }- if (value && typeof value === 'object') {- for (const v of Object.values(value as Record<string, unknown>)) collectUuids(v, into);- }-}--/** Replace `@uuid:<id>` refs with a resolved asset handle (or 0 if unresolved). */-function mapAssetRefs(value: unknown, resolve: (uuid: string) => number): unknown {- if (typeof value === 'string') {- return value.startsWith(UUID_PREFIX) ? resolve(value.slice(UUID_PREFIX.length)) : value;- }- if (Array.isArray(value)) return value.map((v) => mapAssetRefs(v, resolve));- if (value && typeof value === 'object') {- const out: Record<string, unknown> = {};- for (const [k, v] of Object.entries(value as Record<string, unknown>)) out[k] = mapAssetRefs(v, resolve);- return out;- }- return value;-}-class ProjectStoreImpl {private readonly store = createStore<{ project: ProjectState | null }>(() => ({ project: null }));- /** Resolved `@uuid:` → GL handle map for the current scene's textures; the- * Reconciler reads it (via the registered resolver) when projecting entities- * recreated incrementally (duplicate, undo-of-delete) back into the World. */- private readonly uuidToHandle = new Map<string, number>();+ /** uuid → project-relative path, scanned from `.meta` sidecars — the editor's+ * asset registry. The engine `Assets` loader resolves refs through it. */+ private readonly uuidToPath = new Map<string, string>();+ /** The latest scene preload result; the Reconciler resolver reads handles from+ * it for entities recreated incrementally (duplicate / undo / play-stop). */+ private lastAssetResult: PreloadResult | null = null;/** Read accessor so existing `this.state` reads stay unchanged after the move. */private get state(): ProjectState | null {return this.store.getState().project;@@ -186,7 +163,20 @@ class ProjectStoreImpl {`are preserved verbatim in the source-of-truth model and on save (JSON-first).`,);}- const data = await this.resolveTextures(raw);+ // Build the uuid→path registry + point the engine Assets loader at the+ // estella:// transport, then preload EVERY referenced asset type (textures,+ // materials, fonts, audio, spine, …) through the engine's own system, and+ // resolve a COPY of the scene (refs → handles) for the World.+ await this.buildAssetRegistry();+ const assets = EngineHost.getResource(Assets);+ let resolved: SceneData = raw;+ if (assets) {+ const result = await assets.preloadSceneAssets(raw);+ resolved = JSON.parse(JSON.stringify(raw)) as SceneData; // resolveSceneAssetPaths mutates+ assets.resolveSceneAssetPaths(resolved, result);+ this.lastAssetResult = result; // narrowed to the handle maps the resolver reads+ }+// A scene is a session document: replacing it clears the editor history +// selection so undo closures can't reference the previous scene's entities// (REARCH_EDITOR_MODEL.md §6). The Reconciler bulk path then builds the World@@ -195,60 +185,73 @@ class ProjectStoreImpl {// truth. The World is a lossy projection; the model is what save() serializes.EditorHistory.clear();useSelection.getState().select(null);- Reconciler.setAssetResolver((uuid) => this.uuidToHandle.get(uuid) ?? 0);- Reconciler.adopt(raw, data);+ // Incremental recreate (duplicate / undo / play-stop) re-resolves @uuid:→handle+ // from the same preload result — for all types, not just textures.+ Reconciler.setAssetResolver((uuid) => this.handleForUuid(uuid));+ Reconciler.adopt(raw, resolved);EngineHost.syncEditorViewToScene();this.store.setState({ project: { ...st, currentScene: rel } });}/**- * Resolve the scene's `@uuid:` texture refs to live GL handles by loading the- * files through the `estella://` protocol; records handle→uuid for save.- * Unresolved refs blank to 0 (solid color). Textures only — material/font- * refs aren't served yet.+ * Scan `.meta` sidecars under the project's asset roots into a uuid→path+ * registry, then point the engine `Assets` loader at it + the `estella://`+ * transport. This is the ONE asset-resolution path: `Assets.resolveRef` turns+ * `@uuid:` → path, the backend fetches `estella://project/<path>` (REARCH_ASSETS.md A1).*/- private async resolveTextures(raw: SceneData): Promise<SceneData> {+ private async buildAssetRegistry(): Promise<void> {const st = this.state;- const bridge = window.estella;- this.uuidToHandle.clear();- if (!st) return raw;+ this.uuidToPath.clear();+ if (!st) return;+ // The conventional asset root(s): the top-level dir of the declared scenes /+ // textures dirs (e.g. `assets`), which also holds prefabs/audio/etc.+ const roots = new Set([st.layout.scenes, st.layout.textures].map((d) => d.split('/')[0]));+ for (const root of roots) await this.scanMetaDir(root);- const referenced = new Set<string>();- collectUuids(raw, referenced);- if (referenced.size === 0) return raw;+ const assets = EngineHost.getResource(Assets);+ if (assets) {+ assets.baseUrl = 'estella://project';+ assets.setAssetRefResolver((ref) => this.resolveRef(ref));+ }+ }- // uuid → texture path, from the `.meta` sidecars in the textures dir.- const uuidToPath = new Map<string, string>();+ /** Recursively collect `<file>.meta` → {uuid} under `dir` into uuidToPath. */+ private async scanMetaDir(dir: string): Promise<void> {+ let entries;try {- for (const entry of await bridge.fs.readDir(st.layout.textures)) {- if (entry.isDir || !entry.name.endsWith('.meta')) continue;- try {- const meta = JSON.parse(- await bridge.fs.read(`${st.layout.textures}/${entry.name}`),- ) as { uuid?: string };- if (meta.uuid && referenced.has(meta.uuid)) {- uuidToPath.set(meta.uuid, `${st.layout.textures}/${entry.name.replace(/\.meta$/, '')}`);- }- } catch {- // skip a malformed .meta- }- }+ entries = await window.estella.fs.readDir(dir);} catch {- // no textures dir — every ref blanks to 0+ return; // dir absent}-- const assets = EngineHost.getResource(Assets) as unknown as AssetsLike | undefined;- if (assets) {- for (const [uuid, rel] of uuidToPath) {- try {- const { handle } = await assets.loadTexture(`estella://project/${rel}`);- this.uuidToHandle.set(uuid, handle);- } catch (err) {- console.warn('[project] texture load failed', rel, err);- }+ for (const e of entries) {+ const p = `${dir}/${e.name}`;+ if (e.isDir) {+ await this.scanMetaDir(p);+ continue;+ }+ if (!e.name.endsWith('.meta')) continue;+ try {+ const meta = JSON.parse(await window.estella.fs.read(p)) as { uuid?: string };+ if (meta.uuid) this.uuidToPath.set(meta.uuid.toLowerCase(), p.replace(/\.meta$/, ''));+ } catch {+ // skip a malformed .meta}}- return mapAssetRefs(raw, (uuid) => this.uuidToHandle.get(uuid) ?? 0) as SceneData;+ }++ /** Resolve a serialized asset ref to a project-relative path for the engine+ * loader: `@uuid:` → path (null if unknown); a plain path passes through. */+ private resolveRef(ref: string): string | null {+ if (!ref.startsWith(UUID_PREFIX)) return ref;+ return this.uuidToPath.get(ref.slice(UUID_PREFIX.length).toLowerCase()) ?? null;+ }++ /** The live GL handle for a uuid, from the latest preload result (any type). */+ private handleForUuid(uuid: string): number {+ const path = this.uuidToPath.get(uuid.toLowerCase());+ const r = this.lastAssetResult;+ if (!path || !r) return 0;+ return r.textureHandles.get(path) ?? r.materialHandles.get(path) ?? r.fontHandles.get(path) ?? 0;}/**diff --git a/docs/REARCH_ASSETS.md b/docs/REARCH_ASSETS.mdnew file mode 100644index 000000000..c0748aa35--- /dev/null+++ b/docs/REARCH_ASSETS.md@@ -0,0 +1,200 @@+# REARCH — Unified Asset Pipeline (AssetDatabase)++Status: **design / proposed**. Authoritative plan for the engine's second spine —+**content/assets** — complementing the document/scene spine built in+`REARCH_EDITOR_MODEL.md` (model-authoritative editor) and+`REARCH_SERIALIZATION.md` (JSON-first scenes). It is the foundation the Content+Browser, prefabs (`REARCH_PREFABS.md`, future), materials/fonts/audio in the+editor, hot-reload, and the ship/cook path all sit on.++## 1. Problem: a strong runtime asset system the editor doesn't use++The **runtime** already has a capable asset system — it is not the gap:++- `sdk/src/asset/Assets.ts` loads every type (`loadTexture`/`loadMaterial`/+ `loadFont`/`loadAudio`/`loadSpine`/`loadAnimClip`/`loadTilemap`/`loadTimeline`/+ `loadPrefab`), with ref-counted caches, hot-reload (`invalidate` / `onInvalidate`),+ a pluggable `setAssetRefResolver` / `setAssetRegistry`, and scene-level+ `preloadSceneAssets` + `resolveSceneAssetPaths`.+- `sdk/src/asset/AssetRegistry.ts` resolves `@uuid:` → path (`resolveRef` /+ `uuidToPath`); `sdk/src/runtimeLoader.ts` abstracts fetching per platform.+- `tools/asset-meta.js` generates per-asset `.meta` sidecars+ (`{uuid, version, type, importer}`) for ~all types and can `--emit-manifest`.+- `.esprefab` files + `Assets.loadPrefab` already exist (e.g.+ `examples/space-shooter/assets/prefabs/*`).++The gap is **the editor and the project layer**:++- **The editor re-implements a texture-only subset.** `ProjectStore.resolveTextures`+ scans `.meta`, loads **only textures** via `estella://`, and writes its own+ `uuidToHandle` map — bypassing the engine `Assets`/`AssetRegistry`. Its own+ comment: *"material/font refs aren't served yet."* So in the viewport,+ materials/fonts/audio/spine/tilemap refs resolve to handle `0` (blank).+- **Three duplicate resolution paths.** `ProjectStore.resolveTextures`,+ `SceneLoader.loadTextures`, and the P1 `Reconciler.setAssetResolver` each+ collect uuids → load → map to handles independently. They drift.+- **No project-wide AssetDatabase.** `.meta` generation is a one-shot CLI;+ `AssetRegistry` is a runtime manifest. Nothing in the editor *lives*: scans the+ project, watches for changes, tracks dependencies, and serves one index to the+ Content Browser + the registry + the cook.+- **Content Browser is read-only.** `ContentBrowser.tsx` lists/searches/previews+ but Import/Add are disabled and tiles are `draggable={false}` — no import, no+ drag-assign to inspector fields.+- **No cook/ship step.** `asset-meta.js --emit-manifest` exists, but+ `build-tools/cli.js` does not dedupe / transcode / bundle assets per target+ (web / wechat / native).+- **Asset-field knowledge is split** across `AssetFieldRegistry` (builtins),+ `component.generated.ts`, and `extractSchemas` (user components).++**One root cause:** there is no single project-level **AssetDatabase** that the+editor, the runtime loader, and the cook all read — so the editor improvised a+texture-only slice instead of feeding the engine's existing loader.++## 2. Principle: one AssetDatabase, the content source of truth++Mirror the model-authoritative principle on the **content axis**: a single+**AssetDatabase** owns *what assets exist* (guid / type / importer / dependencies);+the editor, the engine `Assets` loader, and the cook are all **derived consumers**+of it. The editor stops resolving assets itself — it builds an `AssetRegistry`+from the DB, hands it to the engine `Assets` (`setAssetRegistry`), and reuses+`preloadSceneAssets` / `resolveSceneAssetPaths`. `@uuid:` stays the portable ref+in the lossless model; the DB is what turns a `@uuid:` into a live handle, for+**every** type, in editor and runtime alike.++```+ ┌────────── AssetDatabase (main process, single index) ─────────┐+ Importers ──write│ scan project → .meta {guid, type, importer, deps} │── change events ─┐+ (texture/audio/ │ dependency graph guid→[guid]; cooked/preview cache by guid │ │+ font/spine/...) │ artifact: .esengine/cache/assets.json (like schemas.json) │ ▼+ └────────────────────────────────────────────────────────────────┘ ┌──────────────────┐+ │ serve (guid|path) │ registry │ cook │ │ hot-reload │+ ▼ ▼ ▼ │ │ Assets.invalidate│+ estella:// AssetServer engine Assets.setAssetRegistry build-tools → │ └──────────────────┘+ (editor preview + runtime) + preloadSceneAssets + bundles ++ resolveSceneAssetPaths assets.manifest+ │+ ▼+ the ONE resolution path: editor + runtime + cook+ all turn @uuid: → handle through engine Assets+```++This is the pattern mature engines converge on: Unity's AssetDatabase + import+pipeline + addressables, UE's Asset Registry + Cooker. The **runtime** half+already exists here; this rearch builds the **project/editor** half and unifies+the two.++## 3. Target architecture (6 layers)++1. **AssetDatabase** (main process, alongside `extractSchemas`/`buildScripts`) —+ scans the project, reads/writes `.meta` (the git-friendly per-asset truth),+ builds the guid↔path index + dependency graph, watches the fs, and emits+ `.esengine/cache/assets.json` (the fast index artifact). One IPC surface+ (`project:assets*`) to the renderer.+2. **Importers** — per-type, producing import settings + (where needed) a cooked+ or preview artifact keyed by guid. Reuse `tools/asset-meta.js`'s extension→type+ map and default importer settings as the seed.+3. **AssetServer** — generalize the `estella://` handler (already serves project+ files by path) to also serve **by guid** (via the DB index) and to serve the+ manifest. One transport for editor preview AND runtime fetch.+4. **Editor↔engine unification** — on project open the editor builds an+ `AssetRegistry` from the DB and calls `Assets.setAssetRegistry`; scene load+ uses `Assets.preloadSceneAssets` + `resolveSceneAssetPaths`; the P1+ `Reconciler` asset resolver delegates to the registry/`Assets` for **all**+ types. The three duplicate paths collapse to one. Hot-reload: DB fs-watch →+ `Assets.invalidate(guid)` → Reconciler re-projects.+5. **Content workflow** — Content Browser **import** (copy file in + write `.meta`+ via the DB), **drag-assign** (drag a guid onto an inspector asset field →+ `SceneCommands.setField('@uuid:'+guid)`; model-authoritative, undoable), and+ **inspector asset-field controls** per type, resolved from a single asset-field+ source. The asset ref edit flows through the same model→Reconciler→Assets path.+6. **Cook / ship** — `build-tools` walks the DB dependency graph from the entry+ scene(s), transcodes/compresses/dedupes per target, emits bundles ++ `assets.manifest`. The isolated play realm (`REARCH_EDITOR_MODEL` P3) loads the+ **same** manifest → `play == ship` for assets too.++## 4. How each problem dissolves (structural, not patched)++| Problem | Why it goes away |+|---|---|+| Editor resolves textures only | The editor feeds the DB's registry to engine `Assets` and reuses `preloadSceneAssets`; every type resolves the way the runtime already loads it. |+| 3 duplicate resolution paths | One path: `@uuid:` → DB registry → `Assets` handle. `ProjectStore.resolveTextures` + `SceneLoader.loadTextures` are deleted; the Reconciler resolver delegates to `Assets`. |+| No project-wide asset index | The AssetDatabase is the live index (scan + watch + deps), the single source the Content Browser, registry, and cook read. |+| Content Browser read-only | Import writes file + `.meta` through the DB; drag-assign emits a model `setField` of the guid. |+| No ship/cook | The cook walks the DB dep graph → per-target bundles + manifest; the play realm loads the same manifest. |+| Asset-field knowledge split | One asset-field source (folded into the DB/schema layer) drives builtin + user components alike. |+| Non-texture assets uneditable in inspector | They resolve now (layer 4) and get per-type controls (layer 5). |++## 5. Phased migration (reuse-heavy, not a rewrite)++- **A1 — Unify the editor onto the engine `Assets`; collapse the 3 paths**+ (highest ROI). The editor builds an `AssetRegistry` from `.meta` and uses+ `Assets.setAssetRegistry` + `preloadSceneAssets` + `resolveSceneAssetPaths`;+ the Reconciler resolver delegates to it. Outcome: **every** asset type resolves+ and previews in the viewport; pure wiring over existing engine code.+- **A2 — AssetDatabase service.** Live scan + fs-watch + dependency graph ++ `.esengine/cache/assets.json` artifact + `project:assets*` IPC; it becomes the+ registry source (replacing the ad-hoc `.meta` scan) and the Content Browser's+ data source. Hot-reload wired through `Assets.invalidate`.+- **A3 — Content workflow.** Content Browser import + drag-assign; inspector+ asset-field controls for all types; a single asset-field source.+- **A4 — Cook / bundle.** Per-target transcode/dedupe/bundle from the dep graph ++ `assets.manifest`, integrated into the build CLI; closes `play == ship` with the+ play realm.+- **A5 — Prefabs** (own doc `REARCH_PREFABS.md`): the `.esprefab` format ++ `Assets.loadPrefab` exist; the editor authoring — prefab **instance + overrides**+ on the model-authoritative `SceneModel` (an override *is* a document delta) — is+ the payoff layer this pipeline unlocks.++## 6. A1 — detailed implementation checklist++Goal: one resolution path; all asset types resolve in the editor viewport;+`@uuid:` stays lossless in the model.++**Editor registry from `.meta`**+- On project open, build an `AssetRegistry` from the project's `.meta` files+ (reuse `tools/asset-meta.js`'s scan, or `AssetRegistry` + a manifest emitted by+ it) and call `EngineHost.getResource(Assets).setAssetRegistry(registry)`.+- Set the `estella://`-based fetch path on `Assets` (the runtime loader) so it+ fetches project files through the protocol (it already serves them).++**`ProjectStore.loadCurrentScene`** (replace texture-only resolution)+- Replace `resolveTextures` with `Assets.preloadSceneAssets(raw)` (async, all+ types) → `resolveSceneAssetPaths` to produce the resolved scene for+ `resetWorldTo`. Keep the **raw** `@uuid:` scene as the model (lossless, P1).+- Delete `mapAssetRefs`/`collectUuids`/`uuidToHandle` from ProjectStore.++**`Reconciler` resolver** (P1 `setAssetResolver`)+- Delegate `@uuid:` → handle to the registry/`Assets` for **all** types (not the+ texture-only map). Keep it a **sync** lookup of already-preloaded handles (see+ Open decisions: async). For incremental recreate (duplicate/undo) the refs were+ preloaded at scene load, so the lookup hits.++**`SceneLoader.loadInto`** — route through `Assets` + the registry (delete its+private `loadTextures`); editor + runtime now share one loader.++**Hot-reload** — DB/file change → `Assets.invalidate(guid)` → Reconciler+re-projects the affected components.++**Tests**+- Load a scene referencing a material/font/spine asset → handles are non-zero →+ viewport renders (not blank). Round-trip keeps `@uuid:` in the model.+- Duplicate/undo an entity with an asset ref → the recreated World entity keeps+ the resolved asset (no regression vs the P1 texture path).+- One-path assertion: ProjectStore no longer holds a uuid→handle map.++## 7. Open decisions++- **Async loads vs the synchronous Reconciler.** Asset loads are async; the+ World projection is sync. Pattern: **preload at the boundary** (scene load,+ drag-assign) so projection-time lookups are sync hits. A NEW ref assigned via+ the inspector must `await Assets.load*` before/within `setField` so the handle+ is ready when the Reconciler projects. (Alternative: project a placeholder+ handle, patch on resolve — more complex; defer.)+- **`.meta` (per-asset, git-friendly) vs `assets.json` (fast index).** Keep+ `.meta` as the authored truth; the DB derives `assets.json` as a cache (rebuilt+ on scan), the same artifact pattern as `schemas.json`/`scripts.mjs`.+- **guid scheme** stays UUID v4 (already established; do not change).+- **DB location** — main process (fs + watch), with an IPC index to the renderer,+ consistent with `extractSchemas`/`buildScripts`.+- **Where importers run** — pure-node in main (like `extractSchemas`), so cook is+ headless-capable.