Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(texture): data maps are sampled as data under the linear pipeline

esengineee71d45da5db
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 57f8fa2d5..68b8a303c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -39,6 +39,7 @@ published separately; it ships inside the editor.
### Fixed
+- Normal, roughness/metalness and occlusion maps are sampled as data, not colour, under the linear pipeline. A KTX2 texture took sRGB decode from the project's colour space alone and ignored its own sRGB setting, and a model import never turned that setting off for the maps that hold numbers — so every normal map leaned and every roughness read glossier than authored. Imports now mark those maps not sRGB, and every upload path follows the setting. A model imported before keeps its textures' settings: turn sRGB off on its normal and packed maps.
- A post-process effect placed before bloom or SSAO takes effect. Their last pass laid its result over the untouched scene instead of over what the chain had made so far, so a volume's colour grade, exposure or grayscale — and SSAO, under bloom — vanished whenever bloom or SSAO came after it. They now composite over their own input.
- A double-sided mesh is lit on the side the camera sees. Its back faces took the front's normal, so a leaf, a flag or an awning turned away from the light went black on the side the light actually reached; the normal now flips for a back face, as glTF specifies, in the engine's mesh shader and in the material shaders an imported model uses.
- Normal maps stored in two channels light correctly. The shader rebuilds Z from X and Y instead of reading blue, so a BC5 map, or one a converter wrote with blue at 0 — every normal map of Bistro's glTF conversion — no longer points its surface's normals inward, which left normal-mapped surfaces unlit by their sun.
diff --git a/desktop b/desktop
index a942eb4d7..7653e5c1a 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit a942eb4d7009096ecf77efff1de34b00f4dffd85
+Subproject commit 7653e5c1a0354af5f095625cbce146116dcd126a
diff --git a/pipeline/src/assets/modelImages.ts b/pipeline/src/assets/modelImages.ts
index c8c72dbe7..dfcc1f176 100644
--- a/pipeline/src/assets/modelImages.ts
+++ b/pipeline/src/assets/modelImages.ts
@@ -18,6 +18,9 @@ type ImageSlot = 'baseColorTexture' | 'normalTexture' | 'emissiveTexture'
const SLOTS: ImageSlot[] = ['baseColorTexture', 'normalTexture', 'emissiveTexture',
'occlusionTexture', 'metallicRoughnessTexture'];
+/** Slots whose texels are numbers, not colour: sampled through sRGB decode they bend. */
+const DATA_SLOTS = new Set<ImageSlot>(['normalTexture', 'occlusionTexture', 'metallicRoughnessTexture']);
+
interface Pixels { width: number; height: number; rgba: Uint8Array }
/** Above this many texels a single colour is worth saying: the file costs what a constant would not. */
@@ -126,7 +129,8 @@ export function prepareModelImages(result: ModelImportResult,
continue;
}
// The sampler settings ride the reference, not the file.
- material[slot] = ref.settings ? { ...converted, settings: ref.settings } : converted;
+ const settings = DATA_SLOTS.has(slot) ? { ...ref.settings, sRGB: false } : ref.settings;
+ material[slot] = settings ? { ...converted, settings } : converted;
if (!pixels) continue;
// A format with no alpha mode (FBX) leaves the base colour's alpha to say it.
if (slot === 'baseColorTexture' && options.alphaFromBaseColor
diff --git a/pipeline/src/assets/modelImport.ts b/pipeline/src/assets/modelImport.ts
index f7a016439..63616fd7d 100644
--- a/pipeline/src/assets/modelImport.ts
+++ b/pipeline/src/assets/modelImport.ts
@@ -41,7 +41,7 @@ export interface ImportedImageRef {
external: boolean;
/** Import settings the source's sampler asks for, in the engine's own words.
* Absent where the source names no sampler (its defaults are the engine's). */
- settings?: { filterMode?: 'nearest' | 'linear'; wrapMode?: 'repeat' | 'clamp' | 'mirror' };
+ settings?: { filterMode?: 'nearest' | 'linear'; wrapMode?: 'repeat' | 'clamp' | 'mirror'; sRGB?: boolean };
}
/**
diff --git a/pipeline/tests/model-images.test.ts b/pipeline/tests/model-images.test.ts
index c491e78ac..4e31a3b38 100644
--- a/pipeline/tests/model-images.test.ts
+++ b/pipeline/tests/model-images.test.ts
@@ -85,6 +85,22 @@ describe('preparing a model\'s images', () => {
expect(run(i => (i % 4 === 0 ? 0 : 255), false)).toEqual({ cutoff: undefined, opaque: true });
});
+ it('marks the maps that hold numbers as not sRGB, and leaves colour alone', () => {
+ const result = model({
+ baseColorTexture: { file: 'base.jpg', external: true },
+ emissiveTexture: { file: 'glow.jpg', external: true },
+ normalTexture: { file: 'n.jpg', external: true, settings: { wrapMode: 'clamp' } },
+ metallicRoughnessTexture: { file: 'mr.jpg', external: true },
+ occlusionTexture: { file: 'ao.jpg', external: true },
+ });
+ prepareModelImages(result, () => null);
+ const m = result.meshes[0]!.material!;
+ expect([m.baseColorTexture?.settings, m.emissiveTexture?.settings]).toEqual([undefined, undefined]);
+ expect(m.normalTexture?.settings).toEqual({ wrapMode: 'clamp', sRGB: false });
+ expect([m.metallicRoughnessTexture?.settings, m.occlusionTexture?.settings])
+ .toEqual([{ sRGB: false }, { sRGB: false }]);
+ });
+
it('skips a DDS it cannot read rather than referencing a file nothing loads', () => {
const result = model({ normalTexture: { file: 'gone.dds', external: true } }, ['gone.dds']);
prepareModelImages(result, () => null);
diff --git a/sdk/src/asset/glTextureUpload.ts b/sdk/src/asset/glTextureUpload.ts
index 393706eba..d907013cf 100644
--- a/sdk/src/asset/glTextureUpload.ts
+++ b/sdk/src/asset/glTextureUpload.ts
@@ -18,7 +18,7 @@
* Everything here acts on the CURRENTLY BOUND `TEXTURE_2D`, which is what lets
* one function serve both a first upload and a re-upload into a live texture.
*/
-import { linearColorSpace } from '../ecs/env';
+import { samplesAsSrgb } from '../ecs/env';
import { glWrapMode, type TextureWrap } from './glTexParams';
import type { PlatformCanvas, PlatformImage } from '../platform/types';
import { requireResourceManager } from '../wasm/resourceManager';
@@ -59,7 +59,7 @@ export function uploadBoundTextureImage(
gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, 0);
// Linear pipeline: color textures store sRGB-encoded — the sampler
// linearizes in hardware. Data textures (normal maps) opt out.
- const internalFormat = linearColorSpace() && (srgb ?? true) ? gl.SRGB8_ALPHA8 : gl.RGBA;
+ const internalFormat = samplesAsSrgb(srgb) ? gl.SRGB8_ALPHA8 : gl.RGBA;
gl.texImage2D(gl.TEXTURE_2D, 0, internalFormat, gl.RGBA, gl.UNSIGNED_BYTE, source as TexImageSource);
// Left where every other upload expects it, not where this one wanted it.
gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, 0);
diff --git a/sdk/src/asset/loaders/TextureLoader.ts b/sdk/src/asset/loaders/TextureLoader.ts
index 1360a3128..cc617bffc 100644
--- a/sdk/src/asset/loaders/TextureLoader.ts
+++ b/sdk/src/asset/loaders/TextureLoader.ts
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
import type { AssetLoader, LoadContext, TextureResult } from '../AssetLoader';
-import { linearColorSpace } from '../../ecs/env';
+import { linearColorSpace, samplesAsSrgb } from '../../ecs/env';
import { platformCreateImage } from '../../platform/base';
import type { PlatformImage } from '../../platform/types';
import { decodeImageBitmap, readImagePixels } from '../imageDecode';
@@ -238,10 +238,10 @@ export class TextureLoader implements AssetLoader<TextureResult> {
if (!isKtx2(bytes)) throw new Error(`TextureLoader: ${path} is not a KTX2 file`);
// Native (embedded Dawn): the ResourceManager transcodes the KTX2 with the
// host's basis library and uploads the compressed blocks — no WebGL2, no
- // wasm transcoder. sRGB follows the color pipeline, like the web path below.
+ // wasm transcoder. sRGB follows the texture's settings, as on every path.
const rm = requireResourceManager();
if (rm.createTextureFromKTX2) {
- const r = rm.createTextureFromKTX2(bytes, linearColorSpace());
+ const r = rm.createTextureFromKTX2(bytes, samplesAsSrgb(settings?.srgb));
if (!r) throw new Error(`TextureLoader: KTX2 transcode failed for ${path}`);
// A host that reports no format says nothing rather than claiming
// success: -1 with no block refusal reads as "this device offered
@@ -258,14 +258,12 @@ export class TextureLoader implements AssetLoader<TextureResult> {
// No GL context (the WebGPU backend): ask the ENGINE which format it
// samples and upload through it. Sampler state is what the GL path
// below still has and this one does not — here it takes the defaults.
- return this.loadCompressedViaEngine(path, transcoder, bytes);
+ return this.loadCompressedViaEngine(path, transcoder, bytes, samplesAsSrgb(settings?.srgb));
}
- // KTX2 payloads are color, like the PNG path: linear mode wants the
- // sRGB variant of whatever compressed format the device supports.
// A WebGL2 context implies a wasm module (native has neither, and threw
// above on the missing gl); the KTX2 path is web-only.
const r = await loadCompressedTexture(gl, this.module_!, transcoder, bytes,
- { ...settings, srgb: linearColorSpace() });
+ { ...settings, srgb: samplesAsSrgb(settings?.srgb) });
this.lastDecision_ = r.decision;
return { handle: r.handle, width: r.width, height: r.height };
}
@@ -276,11 +274,10 @@ export class TextureLoader implements AssetLoader<TextureResult> {
* transcode to it fails — the same bargain the GL path makes.
*/
private async loadCompressedViaEngine(
- path: string, transcoder: BasisTranscoder, bytes: Uint8Array,
+ path: string, transcoder: BasisTranscoder, bytes: Uint8Array, srgb: boolean,
): Promise<TextureResult> {
const rm = requireResourceManager();
const module = this.module_;
- const srgb = linearColorSpace();
if (module && rm.supportsCompressedFormat && rm.createCompressedTexture) {
const supports = rm.supportsCompressedFormat.bind(rm);
const target = chooseEngineTargetFormat((code) => supports(code), srgb);
@@ -427,7 +424,7 @@ export class TextureLoader implements AssetLoader<TextureResult> {
const rm = requireResourceManager();
// Format 2 = sRGB color under the linear pipeline (see rm_createTexture).
- const format = linearColorSpace() && (settings?.srgb ?? true) ? 2 : 1;
+ const format = samplesAsSrgb(settings?.srgb) ? 2 : 1;
const handle = withMalloc(module, pixels.length, ptr => {
module.HEAPU8.set(pixels, ptr);
return rm.createTexture(width, height, ptr, pixels.length, format, flip, TextureContent.Asset);
diff --git a/sdk/src/ecs/env.ts b/sdk/src/ecs/env.ts
index 66d6a1e6a..f98efb7c2 100644
--- a/sdk/src/ecs/env.ts
+++ b/sdk/src/ecs/env.ts
@@ -80,3 +80,12 @@ export function setLinearColorSpace(linear: boolean): void {
export function linearColorSpace(): boolean {
return linearColor;
}
+
+/**
+ * Whether a texture is sampled through sRGB decode: colour is, under the linear
+ * pipeline, unless its import settings say it holds data (a normal map, packed
+ * roughness), which decoded as colour would bend every value below one.
+ */
+export function samplesAsSrgb(srgb?: boolean): boolean {
+ return linearColor && (srgb ?? true);
+}
diff --git a/sdk/src/runtime/runtimeAssets.ts b/sdk/src/runtime/runtimeAssets.ts
index 7f6616962..bb2518ba0 100644
--- a/sdk/src/runtime/runtimeAssets.ts
+++ b/sdk/src/runtime/runtimeAssets.ts
@@ -7,10 +7,9 @@
* builder runtime loader and the spine scene loader (and any editor that
* drives the same load path). Kept in its own module so neither the
* runtime loader nor the spine loader has to import the other.
-import { linearColorSpace } from '../ecs/env';
*/
import { TextureContent, type ESEngineModule } from '../wasm';
-import { linearColorSpace } from '../ecs/env';
+import { samplesAsSrgb } from '../ecs/env';
import type { Backend } from '../asset/Backend';
import type { ParsedTextureImportSettings } from '../asset/textureImportSettings';
import { provideTextureContent, requireResourceManager } from '../wasm/resourceManager';
@@ -121,7 +120,7 @@ async function uploadAtlasPage(
const bytes = new Uint8Array(await fetchBinary(staged));
const rm = requireResourceManager();
if (rm.createTextureFromKTX2) {
- const r = rm.createTextureFromKTX2(bytes, linearColorSpace());
+ const r = rm.createTextureFromKTX2(bytes, samplesAsSrgb());
if (!r) throw new Error(`KTX2 transcode failed: ${staged}`);
return r;
}
@@ -142,7 +141,7 @@ export function createTextureFromPixels(
): number {
const rm = requireResourceManager();
// Format code 2 = sRGB-encoded color (linear pipeline); 1 = plain RGBA8.
- const format = linearColorSpace() && (params?.srgb ?? true) ? 2 : 1;
+ const format = samplesAsSrgb(params?.srgb) ? 2 : 1;
// Native path: no wasm heap — the ResourceManager uploads the bytes itself.
// The wasm embind object has no createTextureFromBytes, so web falls through
// to the heap path below unchanged.
diff --git a/sdk/tests/texture-srgb-setting.test.ts b/sdk/tests/texture-srgb-setting.test.ts
new file mode 100644
index 000000000..c6b93f81b
--- /dev/null
+++ b/sdk/tests/texture-srgb-setting.test.ts
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file A KTX2 texture is sampled as colour or as data by its own sRGB setting.
+ */
+import { describe, it, expect, vi, afterEach } from 'vitest';
+
+const { rm } = vi.hoisted(() => ({
+ rm: { createTextureFromKTX2: vi.fn(() => ({ handle: 3, width: 4, height: 4 })) },
+}));
+vi.mock('../src/wasm/resourceManager', () => ({
+ requireResourceManager: () => rm,
+ provideTextureContent: () => {},
+}));
+
+import { TextureLoader } from '../src/asset/loaders/TextureLoader';
+import { setLinearColorSpace } from '../src/ecs/env';
+
+const KTX2 = new Uint8Array([0xab, 0x4b, 0x54, 0x58, 0x20, 0x32, 0x30, 0xbb, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 0]);
+const ctx = {
+ backend: { fetchBinary: async () => KTX2.buffer.slice(0) },
+ catalog: { getBuildPath: (p: string) => p },
+} as never;
+
+async function srgbFor(settings?: { srgb?: boolean }): Promise<boolean> {
+ rm.createTextureFromKTX2.mockClear();
+ await new TextureLoader(null).loadDetached('a.ktx2', ctx, false, settings);
+ return (rm.createTextureFromKTX2.mock.calls[0] as unknown[])[1] as boolean;
+}
+
+describe('a KTX2 texture under the linear pipeline', () => {
+ afterEach(() => setLinearColorSpace(false));
+
+ it('is decoded as colour unless its settings say it holds data', async () => {
+ setLinearColorSpace(true);
+ expect(await srgbFor()).toBe(true);
+ expect(await srgbFor({ srgb: true })).toBe(true);
+ // A normal map decoded as colour would bend every texel below one.
+ expect(await srgbFor({ srgb: false })).toBe(false);
+ });
+
+ it('is never decoded as colour when the pipeline is not linear', async () => {
+ expect(await srgbFor()).toBe(false);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 62f4cfaf7..e3031c8f4 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -1242,6 +1242,10 @@ export const NOTED = {
{ note: 'A post-process effect placed before bloom or SSAO takes effect' },
'feat(import): an FBX material takes its alpha from its base colour, and .DoubleSided draws both sides':
{ note: 'An FBX material takes its alpha from its base colour image' },
+ 'fix(texture): data maps are sampled as data under the linear pipeline':
+ { note: 'Normal, roughness/metalness and occlusion maps are sampled as data' },
+ 'fix(import): a model\'s data maps reach their .meta as not sRGB':
+ { note: 'Normal, roughness/metalness and occlusion maps are sampled as data' },
'fix(eht): the generator writes LF on every platform':
{ internal: 'The engine header tool is a repository generator; on Windows it rewrote every generated file with CRLF, which only a contributor there ever saw.' },
};