Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(editor): unify asset resolution onto the engine Assets (REARCH_ASSETS A1)

esengine4a00c3de3cdc
diff --git a/desktop/src/project/ProjectStore.ts b/desktop/src/project/ProjectStore.ts
index 493be8d57..a59dc8c4b 100644
--- a/desktop/src/project/ProjectStore.ts
+++ b/desktop/src/project/ProjectStore.ts
@@ -17,16 +17,20 @@ import { resolveLayout, WORKSPACE_DIR, type OpenedProject, type ProjectLayout, t
* directory, loads its scene into the live engine World via `resetWorldTo`, and
* saves back. The bridge sandboxes every fs path to the open project root.
*
- * Texture `@uuid:` refs resolve through the `estella://` custom protocol
- * (electron/main) which serves the project's files — so project textures
- * actually render. On save, texture handles are mapped back to `@uuid:` using
- * each component's asset fields, keeping saved scenes portable.
+ * Assets resolve through the engine's own asset system (REARCH_ASSETS.md A1):
+ * the editor builds a uuid→path registry from `.meta` sidecars, points the
+ * engine `Assets` loader at the `estella://` transport (electron/main serves
+ * project files), and preloads EVERY referenced asset type — not just textures.
+ * The lossless model keeps `@uuid:` refs verbatim, so save stays portable.
*/
const UUID_PREFIX = '@uuid:';
-interface AssetsLike {
- loadTexture(ref: string): Promise<{ handle: number }>;
+/** The subset of the engine's SceneAssetResult the Reconciler resolver reads. */
+interface PreloadResult {
+ textureHandles: Map<string, number>;
+ materialHandles: Map<string, number>;
+ fontHandles: Map<string, number>;
}
interface ProjectState {
@@ -51,41 +55,14 @@ function unknownComponentTypes(data: SceneData): string[] {
return [...unknown];
}
-/** Collect every `@uuid:<id>` referenced anywhere in the scene. */
-function collectUuids(value: unknown, into: Set<string>): void {
- if (typeof value === 'string') {
- if (value.startsWith(UUID_PREFIX)) into.add(value.slice(UUID_PREFIX.length));
- return;
- }
- if (Array.isArray(value)) {
- for (const v of value) collectUuids(v, into);
- return;
- }
- if (value && typeof value === 'object') {
- for (const v of Object.values(value as Record<string, unknown>)) collectUuids(v, into);
- }
-}
-
-/** Replace `@uuid:<id>` refs with a resolved asset handle (or 0 if unresolved). */
-function mapAssetRefs(value: unknown, resolve: (uuid: string) => number): unknown {
- if (typeof value === 'string') {
- return value.startsWith(UUID_PREFIX) ? resolve(value.slice(UUID_PREFIX.length)) : value;
- }
- if (Array.isArray(value)) return value.map((v) => mapAssetRefs(v, resolve));
- if (value && typeof value === 'object') {
- const out: Record<string, unknown> = {};
- for (const [k, v] of Object.entries(value as Record<string, unknown>)) out[k] = mapAssetRefs(v, resolve);
- return out;
- }
- return value;
-}
-
class ProjectStoreImpl {
private readonly store = createStore<{ project: ProjectState | null }>(() => ({ project: null }));
- /** Resolved `@uuid:` → GL handle map for the current scene's textures; the
- * Reconciler reads it (via the registered resolver) when projecting entities
- * recreated incrementally (duplicate, undo-of-delete) back into the World. */
- private readonly uuidToHandle = new Map<string, number>();
+ /** uuid → project-relative path, scanned from `.meta` sidecars — the editor's
+ * asset registry. The engine `Assets` loader resolves refs through it. */
+ private readonly uuidToPath = new Map<string, string>();
+ /** The latest scene preload result; the Reconciler resolver reads handles from
+ * it for entities recreated incrementally (duplicate / undo / play-stop). */
+ private lastAssetResult: PreloadResult | null = null;
/** Read accessor so existing `this.state` reads stay unchanged after the move. */
private get state(): ProjectState | null {
return this.store.getState().project;
@@ -186,7 +163,20 @@ class ProjectStoreImpl {
`are preserved verbatim in the source-of-truth model and on save (JSON-first).`,
);
}
- const data = await this.resolveTextures(raw);
+ // Build the uuid→path registry + point the engine Assets loader at the
+ // estella:// transport, then preload EVERY referenced asset type (textures,
+ // materials, fonts, audio, spine, …) through the engine's own system, and
+ // resolve a COPY of the scene (refs → handles) for the World.
+ await this.buildAssetRegistry();
+ const assets = EngineHost.getResource(Assets);
+ let resolved: SceneData = raw;
+ if (assets) {
+ const result = await assets.preloadSceneAssets(raw);
+ resolved = JSON.parse(JSON.stringify(raw)) as SceneData; // resolveSceneAssetPaths mutates
+ assets.resolveSceneAssetPaths(resolved, result);
+ this.lastAssetResult = result; // narrowed to the handle maps the resolver reads
+ }
+
// A scene is a session document: replacing it clears the editor history +
// selection so undo closures can't reference the previous scene's entities
// (REARCH_EDITOR_MODEL.md §6). The Reconciler bulk path then builds the World
@@ -195,60 +185,73 @@ class ProjectStoreImpl {
// truth. The World is a lossy projection; the model is what save() serializes.
EditorHistory.clear();
useSelection.getState().select(null);
- Reconciler.setAssetResolver((uuid) => this.uuidToHandle.get(uuid) ?? 0);
- Reconciler.adopt(raw, data);
+ // Incremental recreate (duplicate / undo / play-stop) re-resolves @uuid:→handle
+ // from the same preload result — for all types, not just textures.
+ Reconciler.setAssetResolver((uuid) => this.handleForUuid(uuid));
+ Reconciler.adopt(raw, resolved);
EngineHost.syncEditorViewToScene();
this.store.setState({ project: { ...st, currentScene: rel } });
}
/**
- * Resolve the scene's `@uuid:` texture refs to live GL handles by loading the
- * files through the `estella://` protocol; records handle→uuid for save.
- * Unresolved refs blank to 0 (solid color). Textures only — material/font
- * refs aren't served yet.
+ * Scan `.meta` sidecars under the project's asset roots into a uuid→path
+ * registry, then point the engine `Assets` loader at it + the `estella://`
+ * transport. This is the ONE asset-resolution path: `Assets.resolveRef` turns
+ * `@uuid:` → path, the backend fetches `estella://project/<path>` (REARCH_ASSETS.md A1).
*/
- private async resolveTextures(raw: SceneData): Promise<SceneData> {
+ private async buildAssetRegistry(): Promise<void> {
const st = this.state;
- const bridge = window.estella;
- this.uuidToHandle.clear();
- if (!st) return raw;
+ this.uuidToPath.clear();
+ if (!st) return;
+ // The conventional asset root(s): the top-level dir of the declared scenes /
+ // textures dirs (e.g. `assets`), which also holds prefabs/audio/etc.
+ const roots = new Set([st.layout.scenes, st.layout.textures].map((d) => d.split('/')[0]));
+ for (const root of roots) await this.scanMetaDir(root);
- const referenced = new Set<string>();
- collectUuids(raw, referenced);
- if (referenced.size === 0) return raw;
+ const assets = EngineHost.getResource(Assets);
+ if (assets) {
+ assets.baseUrl = 'estella://project';
+ assets.setAssetRefResolver((ref) => this.resolveRef(ref));
+ }
+ }
- // uuid → texture path, from the `.meta` sidecars in the textures dir.
- const uuidToPath = new Map<string, string>();
+ /** Recursively collect `<file>.meta` → {uuid} under `dir` into uuidToPath. */
+ private async scanMetaDir(dir: string): Promise<void> {
+ let entries;
try {
- for (const entry of await bridge.fs.readDir(st.layout.textures)) {
- if (entry.isDir || !entry.name.endsWith('.meta')) continue;
- try {
- const meta = JSON.parse(
- await bridge.fs.read(`${st.layout.textures}/${entry.name}`),
- ) as { uuid?: string };
- if (meta.uuid && referenced.has(meta.uuid)) {
- uuidToPath.set(meta.uuid, `${st.layout.textures}/${entry.name.replace(/\.meta$/, '')}`);
- }
- } catch {
- // skip a malformed .meta
- }
- }
+ entries = await window.estella.fs.readDir(dir);
} catch {
- // no textures dir — every ref blanks to 0
+ return; // dir absent
}
-
- const assets = EngineHost.getResource(Assets) as unknown as AssetsLike | undefined;
- if (assets) {
- for (const [uuid, rel] of uuidToPath) {
- try {
- const { handle } = await assets.loadTexture(`estella://project/${rel}`);
- this.uuidToHandle.set(uuid, handle);
- } catch (err) {
- console.warn('[project] texture load failed', rel, err);
- }
+ for (const e of entries) {
+ const p = `${dir}/${e.name}`;
+ if (e.isDir) {
+ await this.scanMetaDir(p);
+ continue;
+ }
+ if (!e.name.endsWith('.meta')) continue;
+ try {
+ const meta = JSON.parse(await window.estella.fs.read(p)) as { uuid?: string };
+ if (meta.uuid) this.uuidToPath.set(meta.uuid.toLowerCase(), p.replace(/\.meta$/, ''));
+ } catch {
+ // skip a malformed .meta
}
}
- return mapAssetRefs(raw, (uuid) => this.uuidToHandle.get(uuid) ?? 0) as SceneData;
+ }
+
+ /** Resolve a serialized asset ref to a project-relative path for the engine
+ * loader: `@uuid:` → path (null if unknown); a plain path passes through. */
+ private resolveRef(ref: string): string | null {
+ if (!ref.startsWith(UUID_PREFIX)) return ref;
+ return this.uuidToPath.get(ref.slice(UUID_PREFIX.length).toLowerCase()) ?? null;
+ }
+
+ /** The live GL handle for a uuid, from the latest preload result (any type). */
+ private handleForUuid(uuid: string): number {
+ const path = this.uuidToPath.get(uuid.toLowerCase());
+ const r = this.lastAssetResult;
+ if (!path || !r) return 0;
+ return r.textureHandles.get(path) ?? r.materialHandles.get(path) ?? r.fontHandles.get(path) ?? 0;
}
/**
diff --git a/docs/REARCH_ASSETS.md b/docs/REARCH_ASSETS.md
new file mode 100644
index 000000000..c0748aa35
--- /dev/null
+++ b/docs/REARCH_ASSETS.md
@@ -0,0 +1,200 @@
+# REARCH — Unified Asset Pipeline (AssetDatabase)
+
+Status: **design / proposed**. Authoritative plan for the engine's second spine —
+**content/assets** — complementing the document/scene spine built in
+`REARCH_EDITOR_MODEL.md` (model-authoritative editor) and
+`REARCH_SERIALIZATION.md` (JSON-first scenes). It is the foundation the Content
+Browser, prefabs (`REARCH_PREFABS.md`, future), materials/fonts/audio in the
+editor, hot-reload, and the ship/cook path all sit on.
+
+## 1. Problem: a strong runtime asset system the editor doesn't use
+
+The **runtime** already has a capable asset system — it is not the gap:
+
+- `sdk/src/asset/Assets.ts` loads every type (`loadTexture`/`loadMaterial`/
+ `loadFont`/`loadAudio`/`loadSpine`/`loadAnimClip`/`loadTilemap`/`loadTimeline`/
+ `loadPrefab`), with ref-counted caches, hot-reload (`invalidate` / `onInvalidate`),
+ a pluggable `setAssetRefResolver` / `setAssetRegistry`, and scene-level
+ `preloadSceneAssets` + `resolveSceneAssetPaths`.
+- `sdk/src/asset/AssetRegistry.ts` resolves `@uuid:` → path (`resolveRef` /
+ `uuidToPath`); `sdk/src/runtimeLoader.ts` abstracts fetching per platform.
+- `tools/asset-meta.js` generates per-asset `.meta` sidecars
+ (`{uuid, version, type, importer}`) for ~all types and can `--emit-manifest`.
+- `.esprefab` files + `Assets.loadPrefab` already exist (e.g.
+ `examples/space-shooter/assets/prefabs/*`).
+
+The gap is **the editor and the project layer**:
+
+- **The editor re-implements a texture-only subset.** `ProjectStore.resolveTextures`
+ scans `.meta`, loads **only textures** via `estella://`, and writes its own
+ `uuidToHandle` map — bypassing the engine `Assets`/`AssetRegistry`. Its own
+ comment: *"material/font refs aren't served yet."* So in the viewport,
+ materials/fonts/audio/spine/tilemap refs resolve to handle `0` (blank).
+- **Three duplicate resolution paths.** `ProjectStore.resolveTextures`,
+ `SceneLoader.loadTextures`, and the P1 `Reconciler.setAssetResolver` each
+ collect uuids → load → map to handles independently. They drift.
+- **No project-wide AssetDatabase.** `.meta` generation is a one-shot CLI;
+ `AssetRegistry` is a runtime manifest. Nothing in the editor *lives*: scans the
+ project, watches for changes, tracks dependencies, and serves one index to the
+ Content Browser + the registry + the cook.
+- **Content Browser is read-only.** `ContentBrowser.tsx` lists/searches/previews
+ but Import/Add are disabled and tiles are `draggable={false}` — no import, no
+ drag-assign to inspector fields.
+- **No cook/ship step.** `asset-meta.js --emit-manifest` exists, but
+ `build-tools/cli.js` does not dedupe / transcode / bundle assets per target
+ (web / wechat / native).
+- **Asset-field knowledge is split** across `AssetFieldRegistry` (builtins),
+ `component.generated.ts`, and `extractSchemas` (user components).
+
+**One root cause:** there is no single project-level **AssetDatabase** that the
+editor, the runtime loader, and the cook all read — so the editor improvised a
+texture-only slice instead of feeding the engine's existing loader.
+
+## 2. Principle: one AssetDatabase, the content source of truth
+
+Mirror the model-authoritative principle on the **content axis**: a single
+**AssetDatabase** owns *what assets exist* (guid / type / importer / dependencies);
+the editor, the engine `Assets` loader, and the cook are all **derived consumers**
+of it. The editor stops resolving assets itself — it builds an `AssetRegistry`
+from the DB, hands it to the engine `Assets` (`setAssetRegistry`), and reuses
+`preloadSceneAssets` / `resolveSceneAssetPaths`. `@uuid:` stays the portable ref
+in the lossless model; the DB is what turns a `@uuid:` into a live handle, for
+**every** type, in editor and runtime alike.
+
+```
+ ┌────────── AssetDatabase (main process, single index) ─────────┐
+ Importers ──write│ scan project → .meta {guid, type, importer, deps} │── change events ─┐
+ (texture/audio/ │ dependency graph guid→[guid]; cooked/preview cache by guid │ │
+ font/spine/...) │ artifact: .esengine/cache/assets.json (like schemas.json) │ ▼
+ └────────────────────────────────────────────────────────────────┘ ┌──────────────────┐
+ │ serve (guid|path) │ registry │ cook │ │ hot-reload │
+ ▼ ▼ ▼ │ │ Assets.invalidate│
+ estella:// AssetServer engine Assets.setAssetRegistry build-tools → │ └──────────────────┘
+ (editor preview + runtime) + preloadSceneAssets + bundles +
+ resolveSceneAssetPaths assets.manifest
+ │
+ ▼
+ the ONE resolution path: editor + runtime + cook
+ all turn @uuid: → handle through engine Assets
+```
+
+This is the pattern mature engines converge on: Unity's AssetDatabase + import
+pipeline + addressables, UE's Asset Registry + Cooker. The **runtime** half
+already exists here; this rearch builds the **project/editor** half and unifies
+the two.
+
+## 3. Target architecture (6 layers)
+
+1. **AssetDatabase** (main process, alongside `extractSchemas`/`buildScripts`) —
+ scans the project, reads/writes `.meta` (the git-friendly per-asset truth),
+ builds the guid↔path index + dependency graph, watches the fs, and emits
+ `.esengine/cache/assets.json` (the fast index artifact). One IPC surface
+ (`project:assets*`) to the renderer.
+2. **Importers** — per-type, producing import settings + (where needed) a cooked
+ or preview artifact keyed by guid. Reuse `tools/asset-meta.js`'s extension→type
+ map and default importer settings as the seed.
+3. **AssetServer** — generalize the `estella://` handler (already serves project
+ files by path) to also serve **by guid** (via the DB index) and to serve the
+ manifest. One transport for editor preview AND runtime fetch.
+4. **Editor↔engine unification** — on project open the editor builds an
+ `AssetRegistry` from the DB and calls `Assets.setAssetRegistry`; scene load
+ uses `Assets.preloadSceneAssets` + `resolveSceneAssetPaths`; the P1
+ `Reconciler` asset resolver delegates to the registry/`Assets` for **all**
+ types. The three duplicate paths collapse to one. Hot-reload: DB fs-watch →
+ `Assets.invalidate(guid)` → Reconciler re-projects.
+5. **Content workflow** — Content Browser **import** (copy file in + write `.meta`
+ via the DB), **drag-assign** (drag a guid onto an inspector asset field →
+ `SceneCommands.setField('@uuid:'+guid)`; model-authoritative, undoable), and
+ **inspector asset-field controls** per type, resolved from a single asset-field
+ source. The asset ref edit flows through the same model→Reconciler→Assets path.
+6. **Cook / ship** — `build-tools` walks the DB dependency graph from the entry
+ scene(s), transcodes/compresses/dedupes per target, emits bundles +
+ `assets.manifest`. The isolated play realm (`REARCH_EDITOR_MODEL` P3) loads the
+ **same** manifest → `play == ship` for assets too.
+
+## 4. How each problem dissolves (structural, not patched)
+
+| Problem | Why it goes away |
+|---|---|
+| Editor resolves textures only | The editor feeds the DB's registry to engine `Assets` and reuses `preloadSceneAssets`; every type resolves the way the runtime already loads it. |
+| 3 duplicate resolution paths | One path: `@uuid:` → DB registry → `Assets` handle. `ProjectStore.resolveTextures` + `SceneLoader.loadTextures` are deleted; the Reconciler resolver delegates to `Assets`. |
+| No project-wide asset index | The AssetDatabase is the live index (scan + watch + deps), the single source the Content Browser, registry, and cook read. |
+| Content Browser read-only | Import writes file + `.meta` through the DB; drag-assign emits a model `setField` of the guid. |
+| No ship/cook | The cook walks the DB dep graph → per-target bundles + manifest; the play realm loads the same manifest. |
+| Asset-field knowledge split | One asset-field source (folded into the DB/schema layer) drives builtin + user components alike. |
+| Non-texture assets uneditable in inspector | They resolve now (layer 4) and get per-type controls (layer 5). |
+
+## 5. Phased migration (reuse-heavy, not a rewrite)
+
+- **A1 — Unify the editor onto the engine `Assets`; collapse the 3 paths**
+ (highest ROI). The editor builds an `AssetRegistry` from `.meta` and uses
+ `Assets.setAssetRegistry` + `preloadSceneAssets` + `resolveSceneAssetPaths`;
+ the Reconciler resolver delegates to it. Outcome: **every** asset type resolves
+ and previews in the viewport; pure wiring over existing engine code.
+- **A2 — AssetDatabase service.** Live scan + fs-watch + dependency graph +
+ `.esengine/cache/assets.json` artifact + `project:assets*` IPC; it becomes the
+ registry source (replacing the ad-hoc `.meta` scan) and the Content Browser's
+ data source. Hot-reload wired through `Assets.invalidate`.
+- **A3 — Content workflow.** Content Browser import + drag-assign; inspector
+ asset-field controls for all types; a single asset-field source.
+- **A4 — Cook / bundle.** Per-target transcode/dedupe/bundle from the dep graph +
+ `assets.manifest`, integrated into the build CLI; closes `play == ship` with the
+ play realm.
+- **A5 — Prefabs** (own doc `REARCH_PREFABS.md`): the `.esprefab` format +
+ `Assets.loadPrefab` exist; the editor authoring — prefab **instance + overrides**
+ on the model-authoritative `SceneModel` (an override *is* a document delta) — is
+ the payoff layer this pipeline unlocks.
+
+## 6. A1 — detailed implementation checklist
+
+Goal: one resolution path; all asset types resolve in the editor viewport;
+`@uuid:` stays lossless in the model.
+
+**Editor registry from `.meta`**
+- On project open, build an `AssetRegistry` from the project's `.meta` files
+ (reuse `tools/asset-meta.js`'s scan, or `AssetRegistry` + a manifest emitted by
+ it) and call `EngineHost.getResource(Assets).setAssetRegistry(registry)`.
+- Set the `estella://`-based fetch path on `Assets` (the runtime loader) so it
+ fetches project files through the protocol (it already serves them).
+
+**`ProjectStore.loadCurrentScene`** (replace texture-only resolution)
+- Replace `resolveTextures` with `Assets.preloadSceneAssets(raw)` (async, all
+ types) → `resolveSceneAssetPaths` to produce the resolved scene for
+ `resetWorldTo`. Keep the **raw** `@uuid:` scene as the model (lossless, P1).
+- Delete `mapAssetRefs`/`collectUuids`/`uuidToHandle` from ProjectStore.
+
+**`Reconciler` resolver** (P1 `setAssetResolver`)
+- Delegate `@uuid:` → handle to the registry/`Assets` for **all** types (not the
+ texture-only map). Keep it a **sync** lookup of already-preloaded handles (see
+ Open decisions: async). For incremental recreate (duplicate/undo) the refs were
+ preloaded at scene load, so the lookup hits.
+
+**`SceneLoader.loadInto`** — route through `Assets` + the registry (delete its
+private `loadTextures`); editor + runtime now share one loader.
+
+**Hot-reload** — DB/file change → `Assets.invalidate(guid)` → Reconciler
+re-projects the affected components.
+
+**Tests**
+- Load a scene referencing a material/font/spine asset → handles are non-zero →
+ viewport renders (not blank). Round-trip keeps `@uuid:` in the model.
+- Duplicate/undo an entity with an asset ref → the recreated World entity keeps
+ the resolved asset (no regression vs the P1 texture path).
+- One-path assertion: ProjectStore no longer holds a uuid→handle map.
+
+## 7. Open decisions
+
+- **Async loads vs the synchronous Reconciler.** Asset loads are async; the
+ World projection is sync. Pattern: **preload at the boundary** (scene load,
+ drag-assign) so projection-time lookups are sync hits. A NEW ref assigned via
+ the inspector must `await Assets.load*` before/within `setField` so the handle
+ is ready when the Reconciler projects. (Alternative: project a placeholder
+ handle, patch on resolve — more complex; defer.)
+- **`.meta` (per-asset, git-friendly) vs `assets.json` (fast index).** Keep
+ `.meta` as the authored truth; the DB derives `assets.json` as a cache (rebuilt
+ on scan), the same artifact pattern as `schemas.json`/`scripts.mjs`.
+- **guid scheme** stays UUID v4 (already established; do not change).
+- **DB location** — main process (fs + watch), with an IPC index to the renderer,
+ consistent with `extractSchemas`/`buildScripts`.
+- **Where importers run** — pure-node in main (like `extractSchemas`), so cook is
+ headless-capable.