Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(assets): dependency graph learns path refs + bare uuids - real projects cook correctly

yhhbc0c96ebabe1
diff --git a/desktop/electron/assetDb.ts b/desktop/electron/assetDb.ts
index 7bc9ed401..6961b323f 100644
--- a/desktop/electron/assetDb.ts
+++ b/desktop/electron/assetDb.ts
@@ -72,24 +72,62 @@ async function* walkMeta(root: string, rel = ''): AsyncGenerator<string> {
}
}
-/** Collect every `@uuid:<id>` (or bare uuid) referenced in a scene/prefab JSON. */
-function collectRefs(value: unknown, into: Set<string>): void {
+/**
+ * Asset types whose JSON documents can reference other assets — these are
+ * dependency-scanned so the cook's reachability closure includes what they
+ * pull in (a material's shader + textures, a tileset's atlas, …).
+ */
+const JSON_REF_TYPES = new Set([
+ 'scene', 'prefab', 'material', 'tileset', 'animclip', 'statemachine', 'behaviortree',
+]);
+
+/**
+ * Collect every asset reference in a JSON document:
+ * - `@uuid:<id>` refs (editor-serialized stable refs)
+ * - BARE uuid strings, only when they name a real asset (anim-clip flipbook
+ * frames serialize these — the runtime's extractUuid form; entity ids are
+ * uuid-shaped too, so index membership is the filter)
+ * - PATH refs, resolved through @p resolvePath (real content references
+ * materials/textures by project path, not uuid — a dep graph that only
+ * reads `@uuid:` sees an empty project and the cook culls everything)
+ */
+function collectRefs(
+ value: unknown,
+ into: Set<string>,
+ resolvePath?: (ref: string) => string | null,
+ knownUuid?: (id: string) => boolean,
+): void {
if (typeof value === 'string') {
if (value.startsWith(UUID_PREFIX)) {
const id = value.slice(UUID_PREFIX.length).toLowerCase();
if (UUID_V4.test(id)) into.add(id);
+ return;
}
+ if (UUID_V4.test(value)) {
+ const id = value.toLowerCase();
+ if (knownUuid?.(id)) into.add(id);
+ return;
+ }
+ const uuid = resolvePath?.(value);
+ if (uuid) into.add(uuid);
return;
}
if (Array.isArray(value)) {
- for (const v of value) collectRefs(v, into);
+ for (const v of value) collectRefs(v, into, resolvePath, knownUuid);
return;
}
if (value && typeof value === 'object') {
- for (const v of Object.values(value as Record<string, unknown>)) collectRefs(v, into);
+ for (const v of Object.values(value as Record<string, unknown>)) {
+ collectRefs(v, into, resolvePath, knownUuid);
+ }
}
}
+/** Normalize a ref for path-index lookup (forward slashes, no ./ or leading /). */
+function normalizeRefPath(ref: string): string {
+ return ref.replace(/\\/g, '/').replace(/^\.\//, '').replace(/^\//, '');
+}
+
/**
* Scan `root` for `.meta` sidecars → build the asset index (registry + dep
* graph) and (unless `write: false`) write `.esengine/cache/assets.json`.
@@ -123,14 +161,29 @@ export async function scanAssetDatabase(
entries.sort((a, b) => a.path.localeCompare(b.path));
- // Dependency graph: scenes/prefabs reference assets by uuid.
+ // Dependency graph: any JSON asset can reference others — by uuid or by path
+ // (project-relative like a scene's "assets/materials/x.esmaterial", or
+ // relative to the referencing document's own directory like a material's
+ // "shader": "x.esshader"). Path refs resolve against the index, so only
+ // strings naming a real asset create edges — no false positives.
+ const byPath = new Map(entries.map((e) => [e.path, e]));
+ const uuids = new Set(entries.map((e) => e.uuid));
const deps: Record<string, string[]> = {};
for (const entry of entries) {
- if (entry.type !== 'scene' && entry.type !== 'prefab') continue;
+ if (!JSON_REF_TYPES.has(entry.type)) continue;
try {
const json = JSON.parse(await readFile(path.join(root, entry.path), 'utf8'));
+ const dir = entry.path.includes('/') ? entry.path.slice(0, entry.path.lastIndexOf('/')) : '';
const refs = new Set<string>();
- collectRefs(json, refs);
+ collectRefs(json, refs, (ref) => {
+ if (ref.includes('://')) return null;
+ const norm = normalizeRefPath(ref);
+ const direct = byPath.get(norm);
+ if (direct) return direct.uuid;
+ const relative = dir ? byPath.get(normalizeRefPath(`${dir}/${norm}`)) : undefined;
+ return relative?.uuid ?? null;
+ }, (id) => uuids.has(id));
+ refs.delete(entry.uuid);
if (refs.size > 0) deps[entry.uuid] = [...refs].sort();
} catch (err) {
warnings.push(`${entry.path}: ${err instanceof Error ? err.message : String(err)}`);
diff --git a/desktop/electron/cookAssets.ts b/desktop/electron/cookAssets.ts
index 446dfaa07..d1ea0079c 100644
--- a/desktop/electron/cookAssets.ts
+++ b/desktop/electron/cookAssets.ts
@@ -38,6 +38,14 @@ export interface CookManifestEntry extends AssetEntry {
contentHash: string;
/** Staged byte length. */
size: number;
+ /**
+ * The asset's LOGICAL project-relative path — its addressable identity, kept
+ * even when content-addressed staging renames the physical file (`path`) or
+ * texture encoding swaps its extension. This is what path-style references
+ * (a scene's "assets/x.esmaterial", a material's relative "x.esshader")
+ * name at runtime; a host maps it to `path` to serve them from a cooked build.
+ */
+ sourcePath: string;
/** GPU formats the staged KTX2 can transcode to, when the asset was compressed. */
compressedFormats?: string[];
/**
@@ -162,6 +170,18 @@ export async function cookAssets(
ext = '.ktx2';
compressedFormats = COMPRESSED_TARGETS;
}
+ // Shaders ship as-authored; a WGSL twin is what makes one run on the
+ // WebGPU backend, so a twin-less shader is worth a cook-time warning
+ // (generation is a dev-time step — tools/gen-shader-twins.mjs writes the
+ // twins into the source file, keeping the cook deterministic and free of
+ // converter-toolchain dependencies).
+ if (ext.toLowerCase() === '.esshader' &&
+ !Buffer.from(data).toString('utf8').includes('#pragma fragment wgsl')) {
+ warnings.push(
+ `${entry.path}: no WGSL twin — this shader will not render on the WebGPU backend ` +
+ '(run tools/gen-shader-twins.mjs to generate one)',
+ );
+ }
const hash = contentHashHex(data);
// Content-addressed naming: leaf assets ship as assets/<hash><ext>, so
// byte-identical assets collapse to one file (dedup) and the URL is immutable
@@ -184,6 +204,7 @@ export async function cookAssets(
manifestEntries.push({
uuid: entry.uuid,
path: outRel,
+ sourcePath: entry.path,
type: entry.type,
importer: entry.importer,
contentHash: hash,
diff --git a/desktop/tests/cook-assets.test.ts b/desktop/tests/cook-assets.test.ts
index 93190cdcf..1cb0d226f 100644
--- a/desktop/tests/cook-assets.test.ts
+++ b/desktop/tests/cook-assets.test.ts
@@ -213,4 +213,82 @@ describe('cookAssets (A4)', () => {
rmSync(r, { recursive: true, force: true });
}
}, 30_000);
+
+ it('follows PATH refs: scene → material (project path) → shader + texture (dir-relative)', async () => {
+ const r = mkdtempSync(path.join(tmpdir(), 'estella-cook-pathref-'));
+ try {
+ const SC = 'aaaa1111-1111-4111-8111-111111111111';
+ const MAT = 'aaaa2222-2222-4222-8222-222222222222';
+ const SHADER = 'aaaa3333-3333-4333-8333-333333333333';
+ const TEX = 'aaaa4444-4444-4444-8444-444444444444';
+ const ORPHAN = 'aaaa5555-5555-4555-8555-555555555555';
+ const wa = (rel: string, type: string, uuid: string, body: string): void => {
+ const abs = path.join(r, rel);
+ mkdirSync(path.dirname(abs), { recursive: true });
+ writeFileSync(abs, body);
+ writeFileSync(`${abs}.meta`, JSON.stringify({ uuid, version: '2.0', type, importer: {} }));
+ };
+ // The real-content shape: the scene names the material by PROJECT PATH (no
+ // @uuid:), and the material names its shader + texture RELATIVE to itself.
+ wa('assets/materials/m.esmaterial', 'material', MAT, JSON.stringify({
+ version: '1.0', type: 'material', shader: 'm.esshader', properties: { u_mask: 'green.png' },
+ }));
+ wa('assets/materials/m.esshader', 'shader', SHADER,
+ '#pragma shader "M"\n#pragma fragment\nvoid main() {}\n#pragma end\n');
+ wa('assets/materials/green.png', 'texture', TEX, 'G');
+ wa('assets/materials/orphan.png', 'texture', ORPHAN, 'O');
+ wa('s/main.esscene', 'scene', SC, JSON.stringify({
+ version: '1.0', name: 's', entities: [
+ { id: 1, name: 'E', parent: null, children: [], components: [{ type: 'Sprite', data: { material: 'assets/materials/m.esmaterial' } }] },
+ ],
+ }));
+
+ const res = await cookAssets(r, { entryScenes: ['s/main.esscene'], outDir: 'out' });
+ expect(res.included.sort()).toEqual([SC, MAT, SHADER, TEX].sort());
+ expect(res.unused).toEqual([ORPHAN]);
+ // A twin-less shader ships GL-only: the cook says so.
+ expect(res.warnings.some((w) => w.includes('no WGSL twin'))).toBe(true);
+
+ // Content addressing renames the physical file but keeps the LOGICAL
+ // identity in sourcePath — what path refs name at runtime.
+ const ca = await cookAssets(r, { entryScenes: ['s/main.esscene'], outDir: 'out-ca', contentAddressed: true });
+ const m = JSON.parse(readFileSync(ca.manifestPath!, 'utf8')) as AssetManifest & {
+ entries: Array<{ uuid: string; path: string; sourcePath: string }>;
+ };
+ const mat = m.entries.find((e) => e.uuid === MAT)!;
+ expect(mat.sourcePath).toBe('assets/materials/m.esmaterial');
+ expect(mat.path).toMatch(/^assets\/[0-9a-f]{16}\.esmaterial$/);
+ } finally {
+ rmSync(r, { recursive: true, force: true });
+ }
+ });
+
+ it('bare uuid refs (anim-frame form) create edges only when they name a real asset', async () => {
+ const r = mkdtempSync(path.join(tmpdir(), 'estella-cook-bareuuid-'));
+ try {
+ const SC = 'bbbb1111-1111-4111-8111-111111111111';
+ const CLIP = 'bbbb2222-2222-4222-8222-222222222222';
+ const FRAME = 'bbbb3333-3333-4333-8333-333333333333';
+ const NOT_AN_ASSET = 'bbbb9999-9999-4999-8999-999999999999'; // uuid-shaped entity id
+ const wa = (rel: string, type: string, uuid: string, body: string): void => {
+ const abs = path.join(r, rel);
+ mkdirSync(path.dirname(abs), { recursive: true });
+ writeFileSync(abs, body);
+ writeFileSync(`${abs}.meta`, JSON.stringify({ uuid, version: '2.0', type, importer: {} }));
+ };
+ wa('a/frame.png', 'texture', FRAME, 'F');
+ wa('a/idle.esanim', 'animclip', CLIP, JSON.stringify({
+ version: '1.0', frames: [FRAME], sourceId: NOT_AN_ASSET,
+ }));
+ wa('s/main.esscene', 'scene', SC, JSON.stringify({
+ version: '1.0', name: 's', entities: [
+ { id: 1, name: 'E', parent: null, children: [], components: [{ type: 'SpriteAnimator', data: { clip: 'a/idle.esanim' } }] },
+ ],
+ }));
+ const res = await cookAssets(r, { entryScenes: ['s/main.esscene'], outDir: 'out' });
+ expect(res.included.sort()).toEqual([SC, CLIP, FRAME].sort());
+ } finally {
+ rmSync(r, { recursive: true, force: true });
+ }
+ });
});
diff --git a/docs/REARCHITECTURE.md b/docs/REARCHITECTURE.md
index b1ab992af..234cf632c 100644
--- a/docs/REARCHITECTURE.md
+++ b/docs/REARCHITECTURE.md
@@ -191,7 +191,8 @@ TS 驱动的主 pass clear 退出边界,`clear` 家族退出公共 RHI:
**Custom-draw/Draw API 现代化(2026-07-08,最后一个 GL-only 通路闭合)**:新绑定 `draw_meshWithMaterial`——反射材质(compileEsshader)经 `MaterialStore::bindForDraw` 走 MaterialConstants(binding 1)+ 材质记录供给全部管线态,两后端同路零散装 uniform;TS 侧 18 名字 loose 表退为 raw-GLSL `createShader` 兼容路径。编辑器网格迁双语 `.esshader`(`u_model`→`u_rect` 参数 + authored vertex 声明 FrameConstants)——**WebGPU 下编辑器网格首次渲染**,GL/WebGPU 帧一致;`ESTELLA_VERIFY_GRID` 双后端 on/off 像素 diff 断言入 harness。
**WebGPU 异步 readback seam(2026-07-08,后端不对称性收口)**:同步 `readPixels` 退出 RHI,全接口统一为异步四段 `requestReadback / pollReadback / takeReadback / discardReadback`(GL = 请求时同步读、首 poll 即 Ready;WebGPU = `copyTextureToBuffer` → 256 对齐 staging + `mapAsync(AllowSpontaneous)`,跨事件循环 turn 落地;行序统一 bottom-up 契约在 take 时翻转)。RenderFrame 的两个消费者(材质预览 renderToTarget、帧调试器 replayToDrawCall)改为 pass 关闭后请求 + `pollPreview/SnapshotReadback` 落地(顺带修 snapshot 尺寸 getter 读当前帧而非快照时尺寸的隐患);TS 侧 `Material.renderPreview`/`Renderer.getSnapshotImageData` 转 Promise(`readback.ts` 单一 poll 循环,双后端同码)。离屏色纹理补 `CopySrc` usage。**材质预览断言在 WebGPU 下点亮**:mat-tint 场景(补了手写 WGSL 孪生的用户 `.esshader` 资产)双后端 EXPECT + PREVIEW 像素精确 PASS——顺带证明场景资产 shader 带孪生即可整链上 WebGPU。
**Phase 4 转换管线(2026-07-08)**:用户 `.esshader` 的 GLSL→WGSL 自动孪生落地——`tools/gen-shader-twins.mjs`:引擎 wasm 新绑定 `esshader_cookInfo`(web glue 可在纯 Node 加载)吐出与运行时逐字节一致的装配后 GLSL + 纹理单元反射(零漂移),经方言适配(310 es、precision 全 highp、显式引擎绑定注入、combined sampler 拆对、函数采样器参数/实参拆分、varying location 按名对齐)→ `glslangValidator -V` → `naga` → 自包含 `#pragma vertex|fragment wgsl full` 孪生段(ShaderParser 新语义:full 段跳过全部注入头)。9 个 fixture shader 批量生成后,mat-lit 系/mat-tex/mat-dissolve/mat-graph/mat-builtin-lit/mat-instance **在 WebGPU 全部从全黑转绿,lit 像素与 GL 逐点一致**(15 场景 battery)。带 `#pragma switch` 的 shader 跳过(排列需逐集合,手写孪生)。
-**剩余**:cook 管线集成(cookAssets 挂钩 + 转换器 vendored wasm 交付,前置 = RC6 资产依赖图两个存量缺口:assetDb 不扫 .esmaterial 引用、collectRefs 只认 @uuid: 不认路径引用);CI 无 WebGPU 适配器(webgpu verify 仅本地)。
+**cook 集成第一层(2026-07-08)——资产依赖图现代化**:assetDb 依赖图从"仅 scene/prefab 的 @uuid:"扩展为**任意 JSON 资产 × 三种引用形态**——`@uuid:`、裸 uuid(anim 帧的 extractUuid 形态,仅命中索引才建边,防实体 id 误伤)、**路径引用**(项目路径 + 相对本文档目录,只有真实资产命中才建边零误报);扫描类型 scene/prefab/material/tileset/animclip/statemachine/behaviortree。真实项目(路径引用内容)的材质→shader→纹理链自此进入 cook 闭包(此前 deps 全空,整链被静默剔除)。cook manifest 新增 `sourcePath`(逻辑身份,内容寻址改名后路径引用的运行时映射依据);`.esshader` 无 WGSL 孪生 → cook 告警(生成是开发期步骤,保 cook 确定性 + 零转换器依赖)。
+**剩余**:CA 构建的运行时路径解析(gameHost 用 sourcePath 喂 Catalog buildPath + resolvePlayAssetRef 路径分支;**注意存量缺口**——材质内相对引用按 staged 目录解析,在 CA 构建下对 @uuid: 引用同样破损,需 loader 侧以逻辑路径做相对解析,牵 Assets 内部,单独切片);转换器 vendored wasm 交付(glslang wasm + naga wasm32-wasi,仿 build-tools/basis);CI 无 WebGPU 适配器(webgpu verify 仅本地)。
见 `REARCH_WGSL.md`(本地 gitignore,同 RC6/2D_PARITY 约定):着色器版图盘点(uniform 数据已全后端中立,剩程序文本)、岔路拍板(有限集人工孪生 / 图材质 AST 双发射 / 用户材质 cook 期转换——拒绝运行时翻译器的 MB 级体积代价)、四期计划(源语言接缝 → WebGPUDevice 原型 → 双语发射 → cook 期用户材质)。**Phase 0 已落地**:最后 3 个 GLSL ES 1.00 残留(SDK outline/drop-shadow/color-matrix 滤镜)重写为 fragment-only `.esshader` 反射路径——顺带修复其设计性破损(raw 形态无 `u_projection` 来源、无属性绑定、mat4 超 `setMaterialUniform` 的 vec4 上限**从未能传入**;色彩矩阵改为 4×vec4 行 + 偏移列,新增 `Filters.colorMatrixUniforms` 助手)。全仓每个程序自此都在现代缝上(300 es + UBO 反射)。
**Phase 1(源语言接缝)已落地**:RHI 补上最后一块后端中立拼图——`createProgram` 收语言标签的 `GfxShaderSource`,新增 `supportsShaderLanguage` 能力查询(GL = 仅 GLSL ES 300;`Shader::compile` 在任何 GPU 调用前对不支持的语言快速失败);语言贯穿 Shader/ResourceManager(DrawParams 重写仅对 GLSL 生效);`ShaderParser::assembleStage` 获得 `ShaderTargetLanguage` 参数(WGSL 发射 = Phase 3)。MockGfxDevice 新增 `wgslSupported` 开关 + 语言门用例;行为零变化(8 harness + 场景抽样等价)。
**Phase 2 切片 1(WebGPU 后端启动)已落地**:emdawnwebgpu 端口(Dawn)集成为测试目标独立依赖(主构建零影响);`renderer/webgpu/` 新增纯映射层(GfxEnums→WGPU 全翻译,blend/模板表镜像 GLDevice 语义,load-op 规则与 RenderPassDesc 契约一致)+ 空设备安全的 `WebGPUDevice` 骨架(缓冲/纹理/WGSL 模块真实现,布局/管线描述符保留待惰性构建,pass/draw 为切片 2 结构化桩);`test_webgpu_device` 用真 Dawn 头编译验证全部描述符拼写,入 CI。