Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(editor): opening a large project no longer spends seconds on strings that cannot be paths, a model texture the project lacks is left off instead of dangling, and an unlit mesh variant with tangents links

yhh4945d0795861
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ef72c4499..c5ce883dc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,9 @@ published separately; it ships inside the editor.
- **Scroll areas can be pulled past an end and spring back.** A scroll area whose movement is Elastic now does what the setting always said: a drag past an end is stiffer the further out it goes, letting go returns the content, and a flick that reaches an end bounces off it. It is opt-in per area (`ScrollMovement.Elastic`; Clamped, the default, behaves as before), and the gesture model (`ScrollMotion`, with `ScrollContainer.dragTo` and `getOverscroll`) is exported for scrolling anything else the same way.
- **More easing curves.** Sine, quart, quint, expo and circ (each in, out and in-out), bounce in and in-out, and smoothstep and smootherstep join the quad, cubic, back, elastic and out-bounce curves. They are new `EasingType` members after the existing ones, so a saved tween keeps its curve, and they ease the same in a C++ tween and a script value tween.
- **World content has a rectangle: `ContentRect`.** A size and the pivot the entity's transform sits on, which the `Sprite` and `Text` on the entity follow and which a `UIMask` on it clips to — no `UINode` needed, so a clipping region or a scroll viewport can stand in the world. `contentRectContains` tests a world point against it.
+- **Large projects open faster.** The asset dependency scan no longer asks the path index about every string in every document (names, types and labels cannot be paths), which took the open of a 5000-document project from 17 s to under 5 s, and the Cocos script rewrite makes two passes over a file instead of one per declaration.
+- **A model texture the project does not have is left off.** A model whose material names a texture file that is not there is imported without it, with a warning, instead of keeping a reference that fails to load on every use.
+- **An unlit mesh variant with tangents links.** The fragment stage declared the tangent varying wherever a mesh had tangents, though the vertex stage wrote it only when lit, so an unlit variant failed to link.
- **Cocos scripts load one by one.** Each script of a converted project loads on its own, so one whose module throws no longer stops the rest from registering their components. A part of the `cc` module the layer does not provide is a stand-in that survives being read, called and constructed (and says once that it is missing) rather than `undefined`. `Director` is a class scripts can patch, `AssetManager.Cache` exists, and `input._registerEventDispatcher` is accepted.
- **A converted Cocos project links its images.** The source images, fonts and spine files are hard-linked into the converted project (copied only across drives), so a multi-gigabyte art folder is neither copied nor stored twice, and a re-import skips what is already linked. The project reads those files and never writes into them.
- **A Cocos conversion shows its progress.** The importer reports the stage it is in and how far through it is, and the editor runs the conversion on a thread of its own with a progress bar over the window, instead of freezing until it ends.
diff --git a/pipeline/src/assets/assetDb.ts b/pipeline/src/assets/assetDb.ts
index 4f1394f8d..c36430dd6 100644
--- a/pipeline/src/assets/assetDb.ts
+++ b/pipeline/src/assets/assetDb.ts
@@ -281,6 +281,13 @@ function collectRefs(
}
}
+/**
+ * Whether a string could name a file: it ends in an extension, written in one case. Almost every string in a document is a name, a type or
+ * a label (`Transform`, `Cocos.Widget`, `ui_layer`), and each one asked of the path index costs a normalisation and a lookup; in a project of
+ * thousands of documents those are most of the scan.
+ */
+const PATH_LIKE = /\.(?:[a-z0-9]{1,16}|[A-Z0-9]{1,16})$/;
+
/** Normalize a ref for path-index lookup (forward slashes, no ./ or leading /). */
function normalizeRefPath(ref: string): string {
return ref.replace(/\\/g, '/').replace(/^\.\//, '').replace(/^\//, '');
@@ -359,7 +366,7 @@ async function computeDeps(
} else {
const json = JSON.parse(await readTextInRoot(path.join(root, entry.path)));
collectRefs(json, refs, (ref) => {
- if (ref.includes('://')) return null;
+ if (ref.includes('://') || !PATH_LIKE.test(ref)) return null;
const direct = byPath.get(normalizeRefPath(ref));
if (direct) return direct.uuid;
// Fall back to a path RELATIVE to the referencing document, resolved the way
diff --git a/pipeline/src/assets/importModelFile.ts b/pipeline/src/assets/importModelFile.ts
index 9020b8112..680477665 100644
--- a/pipeline/src/assets/importModelFile.ts
+++ b/pipeline/src/assets/importModelFile.ts
@@ -172,6 +172,23 @@ export async function importModelFile(opts: ImportModelOptions): Promise<Importe
externalRefs.set(uri, rel);
}
+ // A texture the model names whose file is not there is left off the material, and said: a reference that resolves to nothing fails to
+ // load every time the model is used, and the material is no better for it (a model exported from a tool that kept its own paths names a
+ // `.psd` the project never had).
+ const lost = new Set(externalFiles.filter((uri) => !externalRefs.has(uri) && !existsSync(path.join(sourceDir, uri))));
+ if (lost.size > 0) {
+ const slots = ['baseColorTexture', 'normalTexture', 'emissiveTexture', 'occlusionTexture', 'metallicRoughnessTexture', 'flowTexture', 'shiftTexture'] as const;
+ for (const mesh of meshes) {
+ const material = mesh.material as Record<string, unknown> | undefined;
+ if (!material) continue;
+ for (const slot of slots) {
+ const image = material[slot] as { external?: boolean; file?: string } | undefined;
+ if (image?.external && image.file !== undefined && lost.has(image.file)) delete material[slot];
+ }
+ }
+ for (const uri of lost) warnings.push(`${uri} is not there; the material is left without that texture`);
+ }
+
const meshFiles: string[] = [];
for (const mesh of meshes) {
meshFiles.push(await write(`${mesh.name}.esmesh`, encodeImportedMesh(mesh)));
diff --git a/pipeline/src/bundle/cocosCompat.ts b/pipeline/src/bundle/cocosCompat.ts
index 76406f8cf..723b641fe 100644
--- a/pipeline/src/bundle/cocosCompat.ts
+++ b/pipeline/src/bundle/cocosCompat.ts
@@ -50,7 +50,8 @@ function packageFileFrom(dir: string, specifier: string): string | undefined {
}
const CONST_DECL = /\bconst\s+([A-Za-z_$][\w$]*)\s*(?::[^=;\n]+)?=/g;
-const escapeRe = (name: string): string => name.replace(/[$]/g, '\\$&');
+/** An assignment statement: the name at the start of a statement, then `=` that is not `==` or `=>`, or a compound operator. */
+const ASSIGNMENT = /(?:^|[;{}\n])\s*([A-Za-z_$][\w$]*)\s*(?:[-+*/%&|^]|\*\*|<<|>>>?|&&|\|\||\?\?)?=(?![=>])/g;
const NAMED_CC_IMPORT = /^[ \t]*import\s*\{([^}]*)\}\s*from\s*(['"])(cc(?:\/env)?)\2\s*;?/gm;
@@ -78,18 +79,17 @@ export function readNamedImportsThroughNamespace(source: string): string {
return reads.length > 0 ? reads.join(' ') + ' ' + rest : rest;
}
-/** The source with every `const` that the script assigns to again read as `let`; the source itself when there is none. */
+/**
+ * The source with every `const` that the script assigns to again read as `let`; the source itself when there is none. Two passes over the text, not
+ * one per declaration: a project's scripts are thousands of files, and this runs on every build, on the editor's main thread.
+ */
export function letTheReassignedConsts(source: string): string {
if (!source.includes('const ')) return source;
- let out = source;
- for (const m of source.matchAll(CONST_DECL)) {
- const name = escapeRe(m[1]!);
- // an assignment statement: the name at the start of a statement, then `=` that is not `==` or `=>`, or a compound operator
- const assigned = new RegExp(`(?:^|[;{}\\n])\\s*${name}\\s*(?:[-+*/%&|^]|\\*\\*|<<|>>>?|&&|\\|\\||\\?\\?)?=(?![=>])`, 'm');
- if (!assigned.test(source.slice((m.index ?? 0) + m[0].length))) continue;
- out = out.replace(new RegExp(`\\bconst(\\s+${name}\\b)`), 'let$1');
- }
- return out;
+ const assigned = new Set<string>();
+ for (const m of source.matchAll(ASSIGNMENT)) assigned.add(m[1]!);
+ if (assigned.size === 0) return source;
+ // a name assigned anywhere makes every const of that name a let: harmless where the assignment was to another variable of the same name
+ return source.replace(CONST_DECL, (all: string, name: string) => (assigned.has(name) ? all.replace('const', 'let') : all));
}
export function cocosCompatPlugin(): Plugin {
diff --git a/pipeline/tests/model-missing-texture.test.ts b/pipeline/tests/model-missing-texture.test.ts
new file mode 100644
index 000000000..d01c14546
--- /dev/null
+++ b/pipeline/tests/model-missing-texture.test.ts
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file A model whose material names a texture file that is not there is imported without that texture, and says so — instead of
+ * leaving a reference that fails to load every time the model is used.
+ */
+import { mkdtempSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+import { afterAll, describe, expect, it } from 'vitest';
+import { importModelFile } from '../src/assets/importModelFile';
+
+function triangleGltf(image: string): string {
+ const positions = new Float32Array([0, 0, 0, 1, 0, 0, 0, 1, 0]);
+ const indices = new Uint16Array([0, 1, 2, 0]);
+ const bytes = Buffer.concat([Buffer.from(positions.buffer), Buffer.from(indices.buffer)]);
+ return JSON.stringify({
+ asset: { version: '2.0' },
+ scene: 0, scenes: [{ nodes: [0] }], nodes: [{ mesh: 0, name: 'tri' }],
+ meshes: [{ primitives: [{ attributes: { POSITION: 0 }, indices: 1, material: 0 }] }],
+ materials: [{ name: 'wall', pbrMetallicRoughness: { baseColorTexture: { index: 0 } } }],
+ textures: [{ source: 0 }], images: [{ uri: image }],
+ buffers: [{ byteLength: bytes.length, uri: `data:application/octet-stream;base64,${bytes.toString('base64')}` }],
+ bufferViews: [{ buffer: 0, byteOffset: 0, byteLength: 36, target: 34962 }, { buffer: 0, byteOffset: 36, byteLength: 6, target: 34963 }],
+ accessors: [
+ { bufferView: 0, componentType: 5126, count: 3, type: 'VEC3', min: [0, 0, 0], max: [1, 1, 0] },
+ { bufferView: 1, componentType: 5123, count: 3, type: 'SCALAR' },
+ ],
+ });
+}
+
+describe('a model that names a texture the project does not have', () => {
+ const roots: string[] = [];
+ afterAll(() => { for (const r of roots) rmSync(r, { recursive: true, force: true, maxRetries: 10, retryDelay: 50 }); });
+
+ async function run(image: string, withFile: boolean) {
+ const root = mkdtempSync(path.join(tmpdir(), 'model-missing-'));
+ roots.push(root);
+ const src = path.join(root, 'assets', 'models');
+ mkdirSync(src, { recursive: true });
+ writeFileSync(path.join(src, 'tri.gltf'), triangleGltf(image));
+ if (withFile) writeFileSync(path.join(src, image), Buffer.from('89504e470d0a1a0a', 'hex'));
+ const result = await importModelFile({ root, destDir: 'assets/models', absSource: path.join(src, 'tri.gltf') });
+ const text = readdirSync(src).map((f) => (/\.(esprefab|esmaterial)$/.test(f) ? readFileSync(path.join(src, f), 'utf8') : '')).join('\n');
+ return { result, text };
+ }
+
+ it('leaves the texture off and says it is not there', async () => {
+ const { result, text } = await run('wall.psd', false);
+ expect(result.warnings.some((w) => w.includes('wall.psd'))).toBe(true);
+ expect(text).not.toContain('wall.psd');
+ });
+
+ it('keeps a texture that is there', async () => {
+ const { result, text } = await run('wall.png', true);
+ expect(result.warnings.some((w) => w.includes('wall.png'))).toBe(false);
+ expect(text).toContain('wall.png');
+ });
+});
diff --git a/src/esengine/resource/ShaderParser.cpp b/src/esengine/resource/ShaderParser.cpp
index 2bae39372..5cc276360 100644
--- a/src/esengine/resource/ShaderParser.cpp
+++ b/src/esengine/resource/ShaderParser.cpp
@@ -2940,12 +2940,14 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
// Tangent frame from screen-space derivatives, so a tangent-space normal
// lands in world space without the geometry carrying a tangent channel —
// which most exported models do not. Degenerate UVs return N unchanged.
- "#ifdef MESH_TANGENTS\n"
+ // Only where the vertex stage writes it: that is LIT with MESH_TANGENTS. A fragment stage that names it elsewhere (an unlit variant of
+ // a mesh that carries tangents) fails to link, though nothing calls the function.
+ "#if defined(MESH_TANGENTS) && defined(LIT)\n"
"in highp vec4 v_worldTangent;\n"
"#endif\n"
"highp vec3 perturbNormal(in highp vec3 N, in highp vec3 worldPos,\n"
" in highp vec2 uv, in highp vec3 tangentNormal) {\n"
- "#ifdef MESH_TANGENTS\n"
+ "#if defined(MESH_TANGENTS) && defined(LIT)\n"
// The frame the normal map was baked in: the tangent made square to the interpolated normal.
" highp vec3 Tg = normalize(v_worldTangent.xyz - N * dot(N, v_worldTangent.xyz));\n"
" highp vec3 Bg = cross(N, Tg) * v_worldTangent.w;\n"
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index bfe462725..66557614c 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -28,6 +28,8 @@ export const NOTES_FLOOR = 'v0.65.0';
* the reader recognises anyway.
*/
export const NOTED = {
+ 'fix(editor): opening a large project no longer spends seconds on strings that cannot be paths, a model texture the project lacks is left off instead of dangling, and an unlit mesh variant with tangents links':
+ {"note":"Large projects open faster, and a model texture the project lacks is left off."},
'feat(cocos): one script that fails to load no longer takes the others with it, a part of cc the layer lacks is a stand-in that survives use, and Director, AssetManager.Cache and the input dispatcher hook exist':
{"note":"Cocos scripts load one by one."},
'feat(cocos): a converted project links its source images instead of copying them, so conversion does not store a multi-gigabyte art folder twice':