Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): texture streaming holds each mesh texture at the size its views need, measured per draw from the mesh's UV density, and moves it behind the same handle

yhhf81f313b5a42
diff --git a/desktop b/desktop
index 29a39d682..5710a5d2d 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 29a39d682c11cd66843dc04ab5c45b2cca7088d5
+Subproject commit 5710a5d2d939137b0e77107b5b23715bff604911
diff --git a/docs/astro/src/content/docs/performance/quality-levels.mdx b/docs/astro/src/content/docs/performance/quality-levels.mdx
index 3a670efb4..4afff5549 100644
--- a/docs/astro/src/content/docs/performance/quality-levels.mdx
+++ b/docs/astro/src/content/docs/performance/quality-levels.mdx
@@ -29,7 +29,7 @@ to.
| Group | Settings |
|---|---|
-| Rendering | Render scale, Min render scale, Anti-aliasing (MSAA or Temporal), MSAA, Depth pre-pass, Detail distance, Draw call budget, 2D rain shelter mask |
+| Rendering | Render scale, Min render scale, Anti-aliasing (MSAA or Temporal), MSAA, Depth pre-pass, Detail distance, Draw call budget, Texture streaming and its budget, 2D rain shelter mask |
| Shadows | Atlas size, Cell size, Cascades, Distance, 2D soft-shadow samples |
| Reflections and particles | Planar reflection scale, max size and updates per second, Particle limit |
| Water | Water waves (1 to 4), Water ripple cells, and the Displacement, Mirror and Caustics switches. Shown once the level is not Off and the water module is in the build |
@@ -62,6 +62,27 @@ it. A draw costs the CPU and the driver more than the GPU, and on a phone or in
first. To bring it down, see [Static batching](/docs/performance/static-batching/) for many different meshes and fewer shadow
cascades for the shadow pass.
+**Texture streaming** holds each texture that only meshes draw at the size its views need, and grows it back as the camera
+comes close. For every mesh a view draws, the renderer works out how many texels across its textures need for a texel to be
+about a pixel there, from the mesh's own UV density, its size and its distance; four times a second the textures that are the
+wrong size are decoded again from their files at the next power of two above that, and put behind the same handle. A texture
+grows as soon as it is needed bigger and shrinks only after it has been needed smaller for three seconds; one no view draws for
+five seconds goes down to 64 texels. It is on at Low and Medium and off at High; **Texture streaming budget** caps what the
+streamed textures take together (128 MB at Low, 256 MB at Medium, 0 is no cap), and past it they are held a level softer.
+
+Measured in the editor's Play at Low, in the `texture-streaming` editor check:
+
+| Example | GPU memory, streaming off | On, ten seconds in | The two pictures |
+|---|---|---|---|
+| `dusk-fort` | 285.6 MB | 31.0 MB (12 textures, 2048 held at 64 to 256) | mean difference 0.28 of 255 |
+| `bistro` | 1293.5 MB | 980.7 MB (228 textures, 62 moved so far) | mean difference 1.8 of 255; 0.05% of pixels differ visibly |
+
+What it does not do: it saves GPU memory, not the download or the decode, since a file is read and decoded whole the first time
+and again for each size it is moved to; a move costs a decode and an upload on the frame it lands, which shows as a frame or two
+of tens of milliseconds while a scene settles. A texture a sprite, a UI element, a tilemap, a particle or a trail also draws is left
+at its size. A KTX2 file is cut at a level without decoding; where the host transcodes KTX2 itself (the native builds) it cannot
+be, and the texture is left at its size.
+
## Leaving effects out
Every post-process effect the engine has is in the list, ticked where the level runs it. An unticked effect is skipped
diff --git a/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx b/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
index abd47a345..db6e3d8ae 100644
--- a/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
+++ b/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
@@ -27,7 +27,7 @@ import { Aside } from '@astrojs/starlight/components';
| 分组 | 设置 |
|---|---|
-| 渲染 | 渲染比例、最小渲染比例、抗锯齿方式(MSAA 或时间性)、MSAA、深度预通道、细节距离、Draw call 预算、2D 雨遮罩尺寸 |
+| 渲染 | 渲染比例、最小渲染比例、抗锯齿方式(MSAA 或时间性)、MSAA、深度预通道、细节距离、Draw call 预算、贴图流式加载及其预算、2D 雨遮罩尺寸 |
| 阴影 | 图集尺寸、单元尺寸、级联数、距离、2D 软阴影采样数 |
| 反射与粒子 | 平面反射的比例、最大尺寸和每秒更新次数,粒子上限 |
| 水体 | 水体波数(1 到 4)、水体涟漪格数,以及位移、镜面、焦散三个开关。档位不是关闭、且构建里带水体模块时才显示 |
@@ -46,6 +46,17 @@ import { Aside } from '@astrojs/starlight/components';
**Draw call 预算**是这一档每帧最多应向 GPU 发出的 draw 数:低档 150、中档 400、高档 1500。所有 pass 都算,不只是场景自己的绘制列表:每个阴影级联都要把投影物再画一次,深度预通道要把不透明场景再画一次,后处理每个 pass 也要一次 draw。不会为了达标而少画什么。Profiler 的**渲染**模块显示这一帧的 GPU draw 数和预算,并按 pass 拆开;统计浮层在超出时把数字标红。一次 draw 花在 CPU 和驱动上的比花在 GPU 上的多,在手机和小游戏里,3D 场景最先用完的就是它。要降下来,不同网格很多时看[静态合并](/docs/zh-cn/performance/static-batching/),阴影那部分则减少级联数。
+**贴图流式加载**把只被网格使用的贴图按视图实际需要的尺寸常驻,相机靠近时再变大。渲染器对视图里画的每个网格,根据网格自身的 UV 密度、尺寸和距离,算出它的贴图需要多少纹素宽才能做到一个纹素约等于一个像素;每秒四次,尺寸不对的贴图会从文件按刚好够用的 2 的幂重新解码,放回同一个句柄后面。需要变大时立刻变大;需要变小时,要持续三秒才会缩;五秒没有被任何视图画到的贴图缩到 64 纹素。低档和中档默认开、高档默认关;**贴图流式预算**限制被流式管理的贴图合计占用(低档 128 MB,中档 256 MB,0 表示不设上限),超出时整体降一级清晰度。
+
+在编辑器 Play 里用低档实测(`texture-streaming` 编辑器检查):
+
+| 示例 | 关流式时的显存 | 开流式十秒后 | 两次画面 |
+|---|---|---|---|
+| `dusk-fort` | 285.6 MB | 31.0 MB(12 张贴图,2048 的缩到 64 到 256) | 平均差 0.28/255 |
+| `bistro` | 1293.5 MB | 980.7 MB(228 张贴图,十秒内完成 62 次缩放) | 平均差 1.8/255;0.05% 的像素有明显变化 |
+
+它做不到的:省的是显存,不是下载和解码。文件第一次仍然完整读取和解码,每换一次尺寸还要再解码一次;一次缩放要在它落地的那一帧付出解码和上传,场景稳定下来之前会有一两帧几十毫秒。同时被精灵、UI、瓦片地图、粒子或拖尾使用的贴图保持原尺寸。KTX2 文件直接从某一级截取,不用重新解码;宿主自己转码 KTX2 的平台(原生打包)做不到这一点,这类贴图保持原尺寸。
+
## 去掉一些效果
引擎有的每个后处理效果都在列表里,该档位运行的打勾。没勾选的效果在这个档位被跳过,不管相机的体积是否要求它——而且是整个跳过:
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 52db98825..b168086f5 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 164 beta · 2014 experimental · 3 deprecated
+Symbols: 115 public · 164 beta · 2025 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -1076,6 +1076,7 @@ releaseTilemap: (ref: string) => void
releaseTimeline: (ref: string) => void
rematerializeMeshesAfterDeviceLoss: () => Promise<number>
remoteRoot: string | undefined
+resizeTexture: (handle: number, maxEdge: number) => Promise<{ width: number; height: number; } | null>
resolveLoadPath: (ref: string) => string
resolveRef: (ref: string) => string | null
resolveSceneAssetPaths: (sceneData: SceneData, result: SceneAssetResult) => void
@@ -1088,6 +1089,7 @@ setRefCounter: (counter: AssetRefCounter) => void
setRemoteRoot: (url: string | undefined) => void
setTextureImportSettingsResolver: (resolver: TextureImportSettingsResolver | null) => void
textureFormatReport: () => TextureFormatReport
+textureResizable: (handle: number) => boolean
texturesAwaitingReupload: () => { handle: number; content: TextureContent; path: string; }[]
trackSceneAssetReferences: (scene: SceneData, entities: ReadonlyMap<number, number>) => void
updateStatus: () => UpdateStatus
@@ -8093,6 +8095,8 @@ shadowSoftSamples: number
shelterMaskSize: number
skipEffects: readonly string[]
ssao: boolean
+textureStreaming: boolean
+textureStreamingBudget: number
waterCaustics: boolean
waterDisplacement: boolean
waterMirror: boolean
@@ -10524,6 +10528,12 @@ toActivate: string[]
toDeactivate: string[]
```
+## StreamMove — interface @experimental
+```
+edge: number
+handle: number
+```
+
## StreamPlane — enum @experimental
```
Ground = 0
@@ -10535,6 +10545,17 @@ Screen = 1
'unload' | 'sleep'
```
+## StreamedTexture — interface @experimental
+```
+edge: number
+fullHeight: number
+fullWidth: number
+handle: number
+seenAt: number
+smallerSince: number
+wanted: number
+```
+
## StreamedWorld — const @experimental
```
ComponentDef<StreamedWorldData>
@@ -10740,6 +10761,11 @@ reads: readonly string[] | undefined
writes: readonly string[] | undefined
```
+## TEXTURE_STREAMING_DEFAULTS — const @experimental
+```
+TextureStreamingOptions
+```
+
## THEME_COLOR_ROLES — const @experimental
```
readonly (keyof ThemeColors)[]
@@ -11335,6 +11361,41 @@ slot: number | undefined
textureId: number
```
+## TextureStreaming — class @experimental
+```
+bytes: () => { now: number; full: number; }
+enabled: boolean
+failures: number
+held: Set<number>
+inFlight: number
+moves: number
+refused: Set<number>
+textures: Map<number, StreamedTexture>
+static new (): TextureStreaming
+static prototype: TextureStreaming
+```
+
+## TextureStreamingOptions — interface @experimental
+```
+budgetBytes: number
+minEdge: number
+shrinkAfter: number
+unseenAfter: number
+```
+
+## TextureStreamingPlugin — class @experimental
+```
+build: (app: App) => void
+name: string
+static new (): TextureStreamingPlugin
+static prototype: TextureStreamingPlugin
+```
+
+## TextureStreamingRes — const @experimental
+```
+ResourceDef<TextureStreaming>
+```
+
## TextureUploadDecision — interface @experimental
```
effective: UploadedGpuFormat
@@ -15844,6 +15905,11 @@ ParticlePlugin
(candidates: readonly StaticBatchCandidate[], limits?: StaticBatchLimits): StaticBatchPlan
```
+## planStreaming — function @experimental
+```
+(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[]
+```
+
## planarReflectionStatus — function @experimental
```
(app: App, entity: Entity, view?: Entity): PlanarReflectionStatus
@@ -16934,6 +17000,16 @@ StatsPlugin
(world: World, entity: Entity): void
```
+## streamTargetEdge — function @experimental
+```
+(fullEdge: number, wanted: number, minEdge: number): number
+```
+
+## streamedBytes — function @experimental
+```
+(fullWidth: number, fullHeight: number, edge: number): number
+```
+
## streamingSourcesOf — function @experimental
```
(app: App): ChunkSource[]
@@ -17024,6 +17100,11 @@ TextPlugin
(importer: Record<string, unknown> | undefined | null): ParsedTextureImportSettings | undefined
```
+## textureStreamingPlugin — const @experimental
+```
+Plugin
+```
+
## themeButtonStates — function @experimental
```
(): Record<string, ButtonStateVisual>
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 115740b2f..4a13b3383 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.native
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 169 beta · 2033 experimental · 3 deprecated
+Symbols: 115 public · 169 beta · 2044 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -1076,6 +1076,7 @@ releaseTilemap: (ref: string) => void
releaseTimeline: (ref: string) => void
rematerializeMeshesAfterDeviceLoss: () => Promise<number>
remoteRoot: string | undefined
+resizeTexture: (handle: number, maxEdge: number) => Promise<{ width: number; height: number; } | null>
resolveLoadPath: (ref: string) => string
resolveRef: (ref: string) => string | null
resolveSceneAssetPaths: (sceneData: SceneData, result: SceneAssetResult) => void
@@ -1088,6 +1089,7 @@ setRefCounter: (counter: AssetRefCounter) => void
setRemoteRoot: (url: string | undefined) => void
setTextureImportSettingsResolver: (resolver: TextureImportSettingsResolver | null) => void
textureFormatReport: () => TextureFormatReport
+textureResizable: (handle: number) => boolean
texturesAwaitingReupload: () => { handle: number; content: TextureContent; path: string; }[]
trackSceneAssetReferences: (scene: SceneData, entities: ReadonlyMap<number, number>) => void
updateStatus: () => UpdateStatus
@@ -8296,6 +8298,8 @@ shadowSoftSamples: number
shelterMaskSize: number
skipEffects: readonly string[]
ssao: boolean
+textureStreaming: boolean
+textureStreamingBudget: number
waterCaustics: boolean
waterDisplacement: boolean
waterMirror: boolean
@@ -10737,6 +10741,12 @@ toActivate: string[]
toDeactivate: string[]
```
+## StreamMove — interface @experimental
+```
+edge: number
+handle: number
+```
+
## StreamPlane — enum @experimental
```
Ground = 0
@@ -10748,6 +10758,17 @@ Screen = 1
'unload' | 'sleep'
```
+## StreamedTexture — interface @experimental
+```
+edge: number
+fullHeight: number
+fullWidth: number
+handle: number
+seenAt: number
+smallerSince: number
+wanted: number
+```
+
## StreamedWorld — const @experimental
```
ComponentDef<StreamedWorldData>
@@ -10953,6 +10974,11 @@ reads: readonly string[] | undefined
writes: readonly string[] | undefined
```
+## TEXTURE_STREAMING_DEFAULTS — const @experimental
+```
+TextureStreamingOptions
+```
+
## THEME_COLOR_ROLES — const @experimental
```
readonly (keyof ThemeColors)[]
@@ -11548,6 +11574,41 @@ slot: number | undefined
textureId: number
```
+## TextureStreaming — class @experimental
+```
+bytes: () => { now: number; full: number; }
+enabled: boolean
+failures: number
+held: Set<number>
+inFlight: number
+moves: number
+refused: Set<number>
+textures: Map<number, StreamedTexture>
+static new (): TextureStreaming
+static prototype: TextureStreaming
+```
+
+## TextureStreamingOptions — interface @experimental
+```
+budgetBytes: number
+minEdge: number
+shrinkAfter: number
+unseenAfter: number
+```
+
+## TextureStreamingPlugin — class @experimental
+```
+build: (app: App) => void
+name: string
+static new (): TextureStreamingPlugin
+static prototype: TextureStreamingPlugin
+```
+
+## TextureStreamingRes — const @experimental
+```
+ResourceDef<TextureStreaming>
+```
+
## TextureUploadDecision — interface @experimental
```
effective: UploadedGpuFormat
@@ -16107,6 +16168,11 @@ ParticlePlugin
(candidates: readonly StaticBatchCandidate[], limits?: StaticBatchLimits): StaticBatchPlan
```
+## planStreaming — function @experimental
+```
+(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[]
+```
+
## planarReflectionStatus — function @experimental
```
(app: App, entity: Entity, view?: Entity): PlanarReflectionStatus
@@ -17197,6 +17263,16 @@ StatsPlugin
(world: World, entity: Entity): void
```
+## streamTargetEdge — function @experimental
+```
+(fullEdge: number, wanted: number, minEdge: number): number
+```
+
+## streamedBytes — function @experimental
+```
+(fullWidth: number, fullHeight: number, edge: number): number
+```
+
## streamingSourcesOf — function @experimental
```
(app: App): ChunkSource[]
@@ -17287,6 +17363,11 @@ TextPlugin
(importer: Record<string, unknown> | undefined | null): ParsedTextureImportSettings | undefined
```
+## textureStreamingPlugin — const @experimental
+```
+Plugin
+```
+
## themeButtonStates — function @experimental
```
(): Record<string, ButtonStateVisual>
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index ae5fb4ae4..24d1995ac 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.node
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2013 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2024 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1071,6 +1071,7 @@ releaseTilemap: (ref: string) => void
releaseTimeline: (ref: string) => void
rematerializeMeshesAfterDeviceLoss: () => Promise<number>
remoteRoot: string | undefined
+resizeTexture: (handle: number, maxEdge: number) => Promise<{ width: number; height: number; } | null>
resolveLoadPath: (ref: string) => string
resolveRef: (ref: string) => string | null
resolveSceneAssetPaths: (sceneData: SceneData, result: SceneAssetResult) => void
@@ -1083,6 +1084,7 @@ setRefCounter: (counter: AssetRefCounter) => void
setRemoteRoot: (url: string | undefined) => void
setTextureImportSettingsResolver: (resolver: TextureImportSettingsResolver | null) => void
textureFormatReport: () => TextureFormatReport
+textureResizable: (handle: number) => boolean
texturesAwaitingReupload: () => { handle: number; content: TextureContent; path: string; }[]
trackSceneAssetReferences: (scene: SceneData, entities: ReadonlyMap<number, number>) => void
updateStatus: () => UpdateStatus
@@ -8088,6 +8090,8 @@ shadowSoftSamples: number
shelterMaskSize: number
skipEffects: readonly string[]
ssao: boolean
+textureStreaming: boolean
+textureStreamingBudget: number
waterCaustics: boolean
waterDisplacement: boolean
waterMirror: boolean
@@ -10519,6 +10523,12 @@ toActivate: string[]
toDeactivate: string[]
```
+## StreamMove — interface @experimental
+```
+edge: number
+handle: number
+```
+
## StreamPlane — enum @experimental
```
Ground = 0
@@ -10530,6 +10540,17 @@ Screen = 1
'unload' | 'sleep'
```
+## StreamedTexture — interface @experimental
+```
+edge: number
+fullHeight: number
+fullWidth: number
+handle: number
+seenAt: number
+smallerSince: number
+wanted: number
+```
+
## StreamedWorld — const @experimental
```
ComponentDef<StreamedWorldData>
@@ -10735,6 +10756,11 @@ reads: readonly string[] | undefined
writes: readonly string[] | undefined
```
+## TEXTURE_STREAMING_DEFAULTS — const @experimental
+```
+TextureStreamingOptions
+```
+
## THEME_COLOR_ROLES — const @experimental
```
readonly (keyof ThemeColors)[]
@@ -11330,6 +11356,41 @@ slot: number | undefined
textureId: number
```
+## TextureStreaming — class @experimental
+```
+bytes: () => { now: number; full: number; }
+enabled: boolean
+failures: number
+held: Set<number>
+inFlight: number
+moves: number
+refused: Set<number>
+textures: Map<number, StreamedTexture>
+static new (): TextureStreaming
+static prototype: TextureStreaming
+```
+
+## TextureStreamingOptions — interface @experimental
+```
+budgetBytes: number
+minEdge: number
+shrinkAfter: number
+unseenAfter: number
+```
+
+## TextureStreamingPlugin — class @experimental
+```
+build: (app: App) => void
+name: string
+static new (): TextureStreamingPlugin
+static prototype: TextureStreamingPlugin
+```
+
+## TextureStreamingRes — const @experimental
+```
+ResourceDef<TextureStreaming>
+```
+
## TextureUploadDecision — interface @experimental
```
effective: UploadedGpuFormat
@@ -15849,6 +15910,11 @@ ParticlePlugin
(candidates: readonly StaticBatchCandidate[], limits?: StaticBatchLimits): StaticBatchPlan
```
+## planStreaming — function @experimental
+```
+(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[]
+```
+
## planarReflectionStatus — function @experimental
```
(app: App, entity: Entity, view?: Entity): PlanarReflectionStatus
@@ -16939,6 +17005,16 @@ StatsPlugin
(world: World, entity: Entity): void
```
+## streamTargetEdge — function @experimental
+```
+(fullEdge: number, wanted: number, minEdge: number): number
+```
+
+## streamedBytes — function @experimental
+```
+(fullWidth: number, fullHeight: number, edge: number): number
+```
+
## streamingSourcesOf — function @experimental
```
(app: App): ChunkSource[]
@@ -17029,6 +17105,11 @@ TextPlugin
(importer: Record<string, unknown> | undefined | null): ParsedTextureImportSettings | undefined
```
+## textureStreamingPlugin — const @experimental
+```
+Plugin
+```
+
## themeButtonStates — function @experimental
```
(): Record<string, ButtonStateVisual>
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index bae4a31e7..645c382d3 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/minigame
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 164 beta · 2035 experimental · 3 deprecated
+Symbols: 115 public · 164 beta · 2046 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1071,6 +1071,7 @@ releaseTilemap: (ref: string) => void
releaseTimeline: (ref: string) => void
rematerializeMeshesAfterDeviceLoss: () => Promise<number>
remoteRoot: string | undefined
+resizeTexture: (handle: number, maxEdge: number) => Promise<{ width: number; height: number; } | null>
resolveLoadPath: (ref: string) => string
resolveRef: (ref: string) => string | null
resolveSceneAssetPaths: (sceneData: SceneData, result: SceneAssetResult) => void
@@ -1083,6 +1084,7 @@ setRefCounter: (counter: AssetRefCounter) => void
setRemoteRoot: (url: string | undefined) => void
setTextureImportSettingsResolver: (resolver: TextureImportSettingsResolver | null) => void
textureFormatReport: () => TextureFormatReport
+textureResizable: (handle: number) => boolean
texturesAwaitingReupload: () => { handle: number; content: TextureContent; path: string; }[]
trackSceneAssetReferences: (scene: SceneData, entities: ReadonlyMap<number, number>) => void
updateStatus: () => UpdateStatus
@@ -8383,6 +8385,8 @@ shadowSoftSamples: number
shelterMaskSize: number
skipEffects: readonly string[]
ssao: boolean
+textureStreaming: boolean
+textureStreamingBudget: number
waterCaustics: boolean
waterDisplacement: boolean
waterMirror: boolean
@@ -10814,6 +10818,12 @@ toActivate: string[]
toDeactivate: string[]
```
+## StreamMove — interface @experimental
+```
+edge: number
+handle: number
+```
+
## StreamPlane — enum @experimental
```
Ground = 0
@@ -10825,6 +10835,17 @@ Screen = 1
'unload' | 'sleep'
```
+## StreamedTexture — interface @experimental
+```
+edge: number
+fullHeight: number
+fullWidth: number
+handle: number
+seenAt: number
+smallerSince: number
+wanted: number
+```
+
## StreamedWorld — const @experimental
```
ComponentDef<StreamedWorldData>
@@ -11030,6 +11051,11 @@ reads: readonly string[] | undefined
writes: readonly string[] | undefined
```
+## TEXTURE_STREAMING_DEFAULTS — const @experimental
+```
+TextureStreamingOptions
+```
+
## THEME_COLOR_ROLES — const @experimental
```
readonly (keyof ThemeColors)[]
@@ -11625,6 +11651,41 @@ slot: number | undefined
textureId: number
```
+## TextureStreaming — class @experimental
+```
+bytes: () => { now: number; full: number; }
+enabled: boolean
+failures: number
+held: Set<number>
+inFlight: number
+moves: number
+refused: Set<number>
+textures: Map<number, StreamedTexture>
+static new (): TextureStreaming
+static prototype: TextureStreaming
+```
+
+## TextureStreamingOptions — interface @experimental
+```
+budgetBytes: number
+minEdge: number
+shrinkAfter: number
+unseenAfter: number
+```
+
+## TextureStreamingPlugin — class @experimental
+```
+build: (app: App) => void
+name: string
+static new (): TextureStreamingPlugin
+static prototype: TextureStreamingPlugin
+```
+
+## TextureStreamingRes — const @experimental
+```
+ResourceDef<TextureStreaming>
+```
+
## TextureUploadDecision — interface @experimental
```
effective: UploadedGpuFormat
@@ -16155,6 +16216,11 @@ ParticlePlugin
(candidates: readonly StaticBatchCandidate[], limits?: StaticBatchLimits): StaticBatchPlan
```
+## planStreaming — function @experimental
+```
+(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[]
+```
+
## planarReflectionStatus — function @experimental
```
(app: App, entity: Entity, view?: Entity): PlanarReflectionStatus
@@ -17245,6 +17311,16 @@ StatsPlugin
(world: World, entity: Entity): void
```
+## streamTargetEdge — function @experimental
+```
+(fullEdge: number, wanted: number, minEdge: number): number
+```
+
+## streamedBytes — function @experimental
+```
+(fullWidth: number, fullHeight: number, edge: number): number
+```
+
## streamingSourcesOf — function @experimental
```
(app: App): ChunkSource[]
@@ -17335,6 +17411,11 @@ TextPlugin
(importer: Record<string, unknown> | undefined | null): ParsedTextureImportSettings | undefined
```
+## textureStreamingPlugin — const @experimental
+```
+Plugin
+```
+
## themeButtonStates — function @experimental
```
(): Record<string, ButtonStateVisual>
diff --git a/sdk/etc/wasm.api.md b/sdk/etc/wasm.api.md
index 2c85293a4..ab33de0e4 100644
--- a/sdk/etc/wasm.api.md
+++ b/sdk/etc/wasm.api.md
@@ -569,6 +569,7 @@ renderer_submitSkeletalPreviewBatch: (preview: number, verticesPtr: number, vert
renderer_submitSkeletalPreviewBatchEx: (preview: number, verticesPtr: number, vertexCount: number, indicesPtr: number, indexCount: number, textureId: number, blendMode: number, transformPtr: number, layer: number, depth: number, materialId: number, premultipliedColor: boolean) => number
renderer_submitTextBatch: ((verticesPtr: number, vertexCount: number, indicesPtr: number, indexCount: number, textureId: number, transformPtr: number, entity: number, layer: number, depth: number, glyphMode: number, cullBit: number) => void) | undefined
renderer_takeFrameCapture: ((handle: number, dest: number, destSize: number) => boolean) | undefined
+renderer_takeTextureWants: (() => string) | undefined
renderer_updateTransforms: (registry: CppRegistry) => void
sdfFromAlpha: ((alphaPtr: number, outPtr: number, width: number, height: number, spread: number) => void) | undefined
setActiveContext: (ctx: CppEngineContext | null) => void
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index 5771f2703..7bfc66078 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/wechat
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 164 beta · 2026 experimental · 3 deprecated
+Symbols: 115 public · 164 beta · 2037 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1071,6 +1071,7 @@ releaseTilemap: (ref: string) => void
releaseTimeline: (ref: string) => void
rematerializeMeshesAfterDeviceLoss: () => Promise<number>
remoteRoot: string | undefined
+resizeTexture: (handle: number, maxEdge: number) => Promise<{ width: number; height: number; } | null>
resolveLoadPath: (ref: string) => string
resolveRef: (ref: string) => string | null
resolveSceneAssetPaths: (sceneData: SceneData, result: SceneAssetResult) => void
@@ -1083,6 +1084,7 @@ setRefCounter: (counter: AssetRefCounter) => void
setRemoteRoot: (url: string | undefined) => void
setTextureImportSettingsResolver: (resolver: TextureImportSettingsResolver | null) => void
textureFormatReport: () => TextureFormatReport
+textureResizable: (handle: number) => boolean
texturesAwaitingReupload: () => { handle: number; content: TextureContent; path: string; }[]
trackSceneAssetReferences: (scene: SceneData, entities: ReadonlyMap<number, number>) => void
updateStatus: () => UpdateStatus
@@ -8095,6 +8097,8 @@ shadowSoftSamples: number
shelterMaskSize: number
skipEffects: readonly string[]
ssao: boolean
+textureStreaming: boolean
+textureStreamingBudget: number
waterCaustics: boolean
waterDisplacement: boolean
waterMirror: boolean
@@ -10526,6 +10530,12 @@ toActivate: string[]
toDeactivate: string[]
```
+## StreamMove — interface @experimental
+```
+edge: number
+handle: number
+```
+
## StreamPlane — enum @experimental
```
Ground = 0
@@ -10537,6 +10547,17 @@ Screen = 1
'unload' | 'sleep'
```
+## StreamedTexture — interface @experimental
+```
+edge: number
+fullHeight: number
+fullWidth: number
+handle: number
+seenAt: number
+smallerSince: number
+wanted: number
+```
+
## StreamedWorld — const @experimental
```
ComponentDef<StreamedWorldData>
@@ -10742,6 +10763,11 @@ reads: readonly string[] | undefined
writes: readonly string[] | undefined
```
+## TEXTURE_STREAMING_DEFAULTS — const @experimental
+```
+TextureStreamingOptions
+```
+
## THEME_COLOR_ROLES — const @experimental
```
readonly (keyof ThemeColors)[]
@@ -11337,6 +11363,41 @@ slot: number | undefined
textureId: number
```
+## TextureStreaming — class @experimental
+```
+bytes: () => { now: number; full: number; }
+enabled: boolean
+failures: number
+held: Set<number>
+inFlight: number
+moves: number
+refused: Set<number>
+textures: Map<number, StreamedTexture>
+static new (): TextureStreaming
+static prototype: TextureStreaming
+```
+
+## TextureStreamingOptions — interface @experimental
+```
+budgetBytes: number
+minEdge: number
+shrinkAfter: number
+unseenAfter: number
+```
+
+## TextureStreamingPlugin — class @experimental
+```
+build: (app: App) => void
+name: string
+static new (): TextureStreamingPlugin
+static prototype: TextureStreamingPlugin
+```
+
+## TextureStreamingRes — const @experimental
+```
+ResourceDef<TextureStreaming>
+```
+
## TextureUploadDecision — interface @experimental
```
effective: UploadedGpuFormat
@@ -15881,6 +15942,11 @@ ParticlePlugin
(candidates: readonly StaticBatchCandidate[], limits?: StaticBatchLimits): StaticBatchPlan
```
+## planStreaming — function @experimental
+```
+(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[]
+```
+
## planarReflectionStatus — function @experimental
```
(app: App, entity: Entity, view?: Entity): PlanarReflectionStatus
@@ -16971,6 +17037,16 @@ StatsPlugin
(world: World, entity: Entity): void
```
+## streamTargetEdge — function @experimental
+```
+(fullEdge: number, wanted: number, minEdge: number): number
+```
+
+## streamedBytes — function @experimental
+```
+(fullWidth: number, fullHeight: number, edge: number): number
+```
+
## streamingSourcesOf — function @experimental
```
(app: App): ChunkSource[]
@@ -17061,6 +17137,11 @@ TextPlugin
(importer: Record<string, unknown> | undefined | null): ParsedTextureImportSettings | undefined
```
+## textureStreamingPlugin — const @experimental
+```
+Plugin
+```
+
## themeButtonStates — function @experimental
```
(): Record<string, ButtonStateVisual>
diff --git a/sdk/src/app/pluginSets.ts b/sdk/src/app/pluginSets.ts
index e9f6e71c8..f7ef5c2b8 100644
--- a/sdk/src/app/pluginSets.ts
+++ b/sdk/src/app/pluginSets.ts
@@ -30,6 +30,7 @@ import { springBonePlugin } from '../springbone';
import { meshRendererPlugin } from '../render/meshRenderer';
import { staticBatchPlugin } from '../render/staticBatchPlugin';
import { impostorPlugin } from '../render/impostor';
+import { textureStreamingPlugin } from '../render/textureStreaming';
import { postProcessPlugin } from '../postprocess';
import { timerPlugin } from '../ecs/timer';
import { velocityPlugin } from '../velocity';
@@ -58,7 +59,7 @@ export const simulationBasePlugins = (): Plugin[] => [
/** What exists to be seen. Every entry drives the engine through `engineApi`. */
export const presentationBasePlugins = (): Plugin[] => [
- animationPlugin, videoPlugin, particlePlugin, trailPlugin, springBonePlugin, meshRendererPlugin, staticBatchPlugin, impostorPlugin,
+ animationPlugin, videoPlugin, particlePlugin, trailPlugin, springBonePlugin, meshRendererPlugin, staticBatchPlugin, impostorPlugin, textureStreamingPlugin,
postProcessPlugin,
];
@@ -69,7 +70,7 @@ export const presentationBasePlugins = (): Plugin[] => [
*/
export const PLUGIN_BUILD_ORDER: readonly string[] = [
'Diagnostics', 'timer', 'velocity', 'lifecycle', 'animation', 'audio', 'video',
- 'particle', 'trail', 'springBone', 'meshRenderer', 'staticBatch', 'impostor', 'terrain', 'foliage', 'spline', 'water', 'floating', 'underwater', 'tilemap', 'postProcess', 'timeline',
+ 'particle', 'trail', 'springBone', 'meshRenderer', 'staticBatch', 'impostor', 'textureStreaming', 'terrain', 'foliage', 'spline', 'water', 'floating', 'underwater', 'tilemap', 'postProcess', 'timeline',
'gameplay',
'perception', 'fsm', 'bt', 'scriptGraph', 'nav', 'eventBinding', 'replication',
'services',
@@ -95,7 +96,7 @@ export function inBuildOrder(plugins: readonly Plugin[]): Plugin[] {
export const webBasePlugins = (): Plugin[] => [
diagnosticsPlugin,
timerPlugin, velocityPlugin, lifecyclePlugin, animationPlugin, audioPlugin, videoPlugin,
- particlePlugin, trailPlugin, springBonePlugin, meshRendererPlugin, staticBatchPlugin, impostorPlugin, postProcessPlugin,
+ particlePlugin, trailPlugin, springBonePlugin, meshRendererPlugin, staticBatchPlugin, impostorPlugin, textureStreamingPlugin, postProcessPlugin,
eventBindingPlugin,
servicesPlugin,
];
diff --git a/sdk/src/asset/Assets.ts b/sdk/src/asset/Assets.ts
index 5d16c9ec4..f9f81bcd1 100644
--- a/sdk/src/asset/Assets.ts
+++ b/sdk/src/asset/Assets.ts
@@ -547,6 +547,21 @@ export class Assets {
});
}
+ /**
+ * Decode the texture behind @p handle again, no larger than @p maxEdge texels on a side (0 is its full size), behind the same handle:
+ * what texture streaming moves a texture between sizes with. Null for a texture no file of this realm's stands behind.
+ */
+ async resizeTexture(handle: number, maxEdge: number): Promise<{ width: number; height: number } | null> {
+ const size = await this.textureLoader_.resize(handle, maxEdge, this.getLoadContext_());
+ if (size) evictTextureDimensions(handle);
+ return size;
+ }
+
+ /** Whether {@link resizeTexture} can move @p handle between sizes. */
+ textureResizable(handle: number): boolean {
+ return this.textureLoader_.resizable(handle);
+ }
+
/**
* Refills every owed texture someone can: an asset texture is loaded again and
* moved behind its handle, any other goes to the refill its creator provided.
diff --git a/sdk/src/asset/compressed.ts b/sdk/src/asset/compressed.ts
index a9e620e45..5fbc8a4e2 100644
--- a/sdk/src/asset/compressed.ts
+++ b/sdk/src/asset/compressed.ts
@@ -224,6 +224,21 @@ export interface CompressedUploadOptions {
readonly srgb?: boolean;
/** False keeps level 0 alone, however many levels the file carries. */
readonly mipmaps?: boolean;
+ /** Leave out the levels larger than this many texels on a side (texture streaming); 0 keeps them all. */
+ readonly maxEdge?: number;
+}
+
+/**
+ * The chain from the first level no larger than @p maxEdge on a side: what a texture held smaller than its file needs, without decoding
+ * or encoding anything. A file whose smallest level is still larger keeps its smallest; 0 keeps the chain as it is.
+ */
+export function levelsFrom<T extends TranscodeResult | RgbaResult>(r: T, maxEdge: number | undefined): T {
+ if (!maxEdge || maxEdge <= 0 || Math.max(r.width, r.height) <= maxEdge) return r;
+ const levels = r.levels && r.levels.length > 0 ? r.levels : [{ width: r.width, height: r.height, data: r.data }];
+ let from = levels.findIndex((l) => Math.max(l.width, l.height) <= maxEdge);
+ if (from < 0) from = levels.length - 1;
+ const kept = levels.slice(from);
+ return { ...r, width: kept[0]!.width, height: kept[0]!.height, data: kept[0]!.data, levels: kept };
}
export interface UploadedTexture {
@@ -395,7 +410,8 @@ export async function loadCompressedTexture(
): Promise<LoadedCompressedTexture> {
const support = detectCompressedTextureSupport(gl);
const target = chooseTargetFormat(support, opts?.srgb ?? false);
- const t = target !== null ? await transcoder.transcode(bytes, target) : null;
+ const whole = target !== null ? await transcoder.transcode(bytes, target) : null;
+ const t = whole ? levelsFrom(whole, opts?.maxEdge) : null;
const aligned = target !== null && t !== null && webglCompressedLevelsSupported(target, levelsOf(t, opts));
const decision: TextureUploadDecision = target !== null && t && !aligned
? { payload: 'ktx2', target, effective: 'rgba8', reason: 'not-block-aligned' }
@@ -403,7 +419,8 @@ export async function loadCompressedTexture(
if (target !== null && t && aligned) {
return { ...uploadCompressedTexture(gl, module, support, target, t, opts), decision };
}
- const rgba = await transcoder.transcodeToRgba(bytes);
- if (!rgba) throw new Error('BasisTranscoder failed to decode KTX2 (compressed and RGBA paths both failed)');
+ const decoded = await transcoder.transcodeToRgba(bytes);
+ if (!decoded) throw new Error('BasisTranscoder failed to decode KTX2 (compressed and RGBA paths both failed)');
+ const rgba = levelsFrom(decoded, opts?.maxEdge);
return { ...uploadRgbaTexture(gl, module, rgba, opts), decision };
}
diff --git a/sdk/src/asset/loaders/TextureLoader.ts b/sdk/src/asset/loaders/TextureLoader.ts
index 979dee785..869d5feab 100644
--- a/sdk/src/asset/loaders/TextureLoader.ts
+++ b/sdk/src/asset/loaders/TextureLoader.ts
@@ -9,7 +9,7 @@ import { requireResourceManager } from '../../wasm/resourceManager';
import { TextureContent, type ESEngineModule } from '../../wasm';
import { withMalloc } from '../../wasm/wasmScratch';
import {
- isKtx2, isKtx2Path, loadCompressedTexture, chooseEngineTargetFormat, engineFormatCode,
+ isKtx2, isKtx2Path, loadCompressedTexture, levelsFrom, chooseEngineTargetFormat, engineFormatCode,
compressedUploadDecision, detectCompressedTextureSupport, glInternalFormat,
supportedTargetFormats, hostUploadDecision, RAW_PAYLOAD_UPLOAD,
CompressedTextureFormat,
@@ -131,6 +131,8 @@ export class TextureLoader implements AssetLoader<TextureResult> {
* took the decisions and cannot outlive the textures it describes.
*/
private readonly formats_ = new TextureFormatLog();
+ /** Where each texture this loader decoded from a file came from: what a resize decodes again. */
+ private readonly sources_ = new Map<number, { path: string; flip: boolean; settings?: TextureImportSettings }>();
/** The decision the in-flight upload came to. Set by the one entry point that
* chooses a path (decodeAndUpload_) and read by the one that records. */
private lastDecision_: TextureUploadDecision = RAW_PAYLOAD_UPLOAD;
@@ -200,6 +202,7 @@ export class TextureLoader implements AssetLoader<TextureResult> {
}
unload(asset: TextureResult): void {
+ this.sources_.delete(asset.handle);
const rm = requireResourceManager();
rm.releaseTexture(asset.handle);
}
@@ -216,10 +219,29 @@ export class TextureLoader implements AssetLoader<TextureResult> {
return this.decodeAndUpload_(path, ctx, flip, settings);
}
+ /**
+ * Decode the texture behind @p handle again, no larger than @p maxEdge texels on a side (0 is its full size), and move the new
+ * pixels behind the handle: everything that holds the handle — a material, a renderer, a batch key — sees the new size at once.
+ * The texture streamer's one move. False for a handle this loader did not decode from a file, or one the host cannot re-point.
+ */
+ async resize(handle: number, maxEdge: number, ctx: LoadContext): Promise<{ width: number; height: number } | null> {
+ const source = this.sources_.get(handle);
+ const rm = requireResourceManager();
+ if (!source || !rm.adoptTextureContent) return null;
+ const fresh = await this.decodeAndUpload_(source.path, ctx, source.flip, source.settings, maxEdge);
+ const moved = rm.adoptTextureContent(handle, fresh.handle);
+ rm.releaseTexture(fresh.handle);
+ return moved ? { width: fresh.width, height: fresh.height } : null;
+ }
+
+ /** Whether @p handle was decoded from a file by this loader, so it can be decoded again at another size. */
+ resizable(handle: number): boolean { return this.sources_.has(handle); }
+
private async loadWithFlip(
path: string, ctx: LoadContext, flip: boolean, settings?: TextureImportSettings,
): Promise<TextureResult> {
const result = await this.decodeAndUpload_(path, ctx, flip, settings);
+ this.sources_.set(result.handle, { path, flip, settings });
// Recorded HERE, once, for every path into the loader — the compressed
// paths, the pixel decoder and the <img> upload alike. A record kept only
// where a fallback happens cannot say that everything else was fine.
@@ -234,29 +256,31 @@ export class TextureLoader implements AssetLoader<TextureResult> {
}
private async decodeAndUpload_(
- path: string, ctx: LoadContext, flip: boolean, settings?: TextureImportSettings,
+ path: string, ctx: LoadContext, flip: boolean, settings?: TextureImportSettings, maxEdge = 0,
): Promise<TextureResult> {
this.lastDecision_ = RAW_PAYLOAD_UPLOAD;
if (isKtx2Path(path)) {
const buf = await ctx.backend.fetchBinary(ctx.catalog.getBuildPath(path));
- return this.uploadKtx2_(path, new Uint8Array(buf), settings);
+ return this.uploadKtx2_(path, new Uint8Array(buf), settings, maxEdge);
}
let preview = false;
if (this.compressedSource_) {
const compressed = await this.compressedSource_(path);
- if (compressed instanceof Uint8Array) return this.uploadKtx2_(path, compressed, settings);
+ if (compressed instanceof Uint8Array) return this.uploadKtx2_(path, compressed, settings, maxEdge);
preview = compressed === 'pending';
}
+ // The smaller of a preview's edge and the one asked for: a texture held small is decoded small.
+ const edge = preview ? (maxEdge > 0 ? Math.min(maxEdge, TEXTURE_PREVIEW_EDGE) : TEXTURE_PREVIEW_EDGE) : maxEdge;
// A platform pixel decoder (runtime scene loader) pre-decodes to RGBA and
// uploads through the shared createTexture path — the same code the old
// runtimeLoader.loadTextures used — instead of a URL-based <img>.
if (this.pixelDecoder_) {
- const result = await this.pixelDecoder_(path, flip, preview ? TEXTURE_PREVIEW_EDGE : 0);
+ const result = await this.pixelDecoder_(path, flip, edge);
const handle = createTextureFromPixels(this.module_, result, TextureContent.Asset, flip, pixelParams(settings));
return { handle, width: result.width, height: result.height };
}
const url = ctx.backend.resolveUrl(ctx.catalog.getBuildPath(path));
- const img = await this.loadImage(url, flip, preview ? TEXTURE_PREVIEW_EDGE : 0, this.imageRevisions_.get(path));
+ const img = await this.loadImage(url, flip, edge, this.imageRevisions_.get(path));
return this.createTextureFromImage(img, flip, settings);
}
@@ -266,7 +290,7 @@ export class TextureLoader implements AssetLoader<TextureResult> {
* the `flip` flag does not apply.
*/
private async uploadKtx2_(
- path: string, bytes: Uint8Array, settings?: TextureImportSettings,
+ path: string, bytes: Uint8Array, settings?: TextureImportSettings, maxEdge = 0,
): Promise<TextureResult> {
if (!isKtx2(bytes)) throw new Error(`TextureLoader: ${path} is not a KTX2 file`);
// Native (embedded Dawn): the ResourceManager transcodes the KTX2 with the
@@ -291,12 +315,12 @@ export class TextureLoader implements AssetLoader<TextureResult> {
// No GL context (the WebGPU backend): ask the ENGINE which format it
// samples and upload through it. Sampler state is what the GL path
// below still has and this one does not — here it takes the defaults.
- return this.loadCompressedViaEngine(path, transcoder, bytes, samplesAsSrgb(settings?.srgb));
+ return this.loadCompressedViaEngine(path, transcoder, bytes, samplesAsSrgb(settings?.srgb), maxEdge);
}
// A WebGL2 context implies a wasm module (native has neither, and threw
// above on the missing gl); the KTX2 path is web-only.
const r = await loadCompressedTexture(gl, this.module_!, transcoder, bytes,
- { ...settings, srgb: samplesAsSrgb(settings?.srgb) });
+ { ...settings, srgb: samplesAsSrgb(settings?.srgb), maxEdge });
this.lastDecision_ = r.decision;
return { handle: r.handle, width: r.width, height: r.height };
}
@@ -307,14 +331,15 @@ export class TextureLoader implements AssetLoader<TextureResult> {
* transcode to it fails — the same bargain the GL path makes.
*/
private async loadCompressedViaEngine(
- path: string, transcoder: BasisTranscoder, bytes: Uint8Array, srgb: boolean,
+ path: string, transcoder: BasisTranscoder, bytes: Uint8Array, srgb: boolean, maxEdge = 0,
): Promise<TextureResult> {
const rm = requireResourceManager();
const module = this.module_;
if (module && rm.supportsCompressedFormat && rm.createCompressedTexture) {
const supports = rm.supportsCompressedFormat.bind(rm);
const target = chooseEngineTargetFormat((code) => supports(code), srgb);
- const t = target ? await transcoder.transcode(bytes, target) : null;
+ const whole = target ? await transcoder.transcode(bytes, target) : null;
+ const t = whole ? levelsFrom(whole, maxEdge) : null;
this.lastDecision_ = compressedUploadDecision(target, t !== null);
if (target && t) {
const code = engineFormatCode(target, srgb);
@@ -338,8 +363,9 @@ export class TextureLoader implements AssetLoader<TextureResult> {
} else {
this.lastDecision_ = compressedUploadDecision(null, false);
}
- const rgba = await transcoder.transcodeToRgba(bytes);
- if (!rgba) throw new Error(`TextureLoader: KTX2 decode failed for ${path}`);
+ const decoded = await transcoder.transcodeToRgba(bytes);
+ if (!decoded) throw new Error(`TextureLoader: KTX2 decode failed for ${path}`);
+ const rgba = levelsFrom(decoded, maxEdge);
return this.loadFromPixels(rgba.width, rgba.height, rgba.data, false, TextureContent.Asset);
}
diff --git a/sdk/src/core-content.ts b/sdk/src/core-content.ts
index f39f061eb..6478c8b18 100644
--- a/sdk/src/core-content.ts
+++ b/sdk/src/core-content.ts
@@ -857,6 +857,11 @@ export {
mergeMeshes, planStaticBatches, STATIC_BATCH_DEFAULTS,
type StaticBatchGroup, type StaticBatchLimits, type StaticBatchRefusal, type StaticBatchReport,
} from './render/staticBatch';
+export {
+ TextureStreamingPlugin, textureStreamingPlugin, TextureStreaming, TextureStreamingRes, TEXTURE_STREAMING_DEFAULTS,
+ planStreaming, streamTargetEdge, streamedBytes,
+ type StreamedTexture, type StreamMove, type TextureStreamingOptions,
+} from './render/textureStreaming';
export {
ImpostorPlugin, impostorPlugin, ImpostorAtlases, ImpostorAtlasesRes, IMPOSTOR_BAKE_LAYER,
impostorEncode, impostorDecode, impostorFrameDirection, impostorPictureUp, impostorKindKey,
diff --git a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
index c8133a120..83423dffc 100644
--- a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
+++ b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
@@ -248,6 +248,7 @@ export interface NativeEngineApi {
renderer_submitSkeletalPreviewBatchEx?(preview: number, verticesPtr: number, vertexCount: number, indicesPtr: number, indexCount: number, textureId: number, blendMode: number, transformPtr: number, layer: number, depth: number, materialId: number, premultipliedColor: boolean): number;
renderer_submitTextBatch?(verticesPtr: number, vertexCount: number, indicesPtr: number, indexCount: number, textureId: number, transformPtr: number, entity: number, layer: number, depth: number, glyphMode: number, cullBit: number): void;
renderer_takeFrameCapture?(handle: number, dest: number, destSize: number): boolean;
+ renderer_takeTextureWants?(): string;
renderer_updateTransforms?(registry: unknown): void;
rm_acquireTextureByPath?(rm: unknown, path: string): number;
rm_adoptTextureContent?(rm: unknown, target: number, source: number): boolean;
@@ -588,6 +589,7 @@ export function createNativeEngineApi(
bind('renderer_submitSkeletalPreviewBatchEx', 'es_renderer_submitSkeletalPreviewBatchEx', false);
bind('renderer_submitTextBatch', 'es_renderer_submitTextBatch', false);
bind('renderer_takeFrameCapture', 'es_renderer_takeFrameCapture', false);
+ bind('renderer_takeTextureWants', 'es_renderer_takeTextureWants', false);
bind('renderer_updateTransforms', 'es_renderer_updateTransforms', true);
bind('rm_acquireTextureByPath', 'es_rm_acquireTextureByPath', true);
bind('rm_adoptTextureContent', 'es_rm_adoptTextureContent', true);
diff --git a/sdk/src/render/quality.ts b/sdk/src/render/quality.ts
index 5f0c78c2a..cf5a432a5 100644
--- a/sdk/src/render/quality.ts
+++ b/sdk/src/render/quality.ts
@@ -84,6 +84,13 @@ export interface QualityProfile {
* went. Draw calls cost the CPU and the driver more than the GPU, and on a phone or in a mini game they are what a 3D scene runs out of first.
*/
drawCallBudget: number;
+ /**
+ * Hold each texture only meshes draw at the size its views need (see `textureStreamingPlugin`): a texture far away takes a
+ * fraction of its memory, and grows back as the camera comes close. Its file is still read and decoded whole the first time.
+ */
+ textureStreaming: boolean;
+ /** The megabytes the streamed textures may take together; past it they are held a level softer. 0 is no cap. */
+ textureStreamingBudget: number;
}
export interface DeviceQualityRule {
@@ -137,7 +144,7 @@ const PROFILES: Record<QualityLevel, QualityProfile> = {
shadowDistance: 0, detailScale: 0.5, particleLimit: 1000, postProcess: true, ssao: false,
waterWaves: 2, waterRippleCells: 32, waterDisplacement: false, waterMirror: false, waterCaustics: false,
skipEffects: ['ssr', 'taa', 'depthOfField', 'lightShafts', 'motionBlur', 'lensFlare', 'gtao', 'subsurface', 'voxelGI'],
- shelterMaskSize: 256, shadowSoftSamples: 8, drawCallBudget: 150,
+ shelterMaskSize: 256, shadowSoftSamples: 8, drawCallBudget: 150, textureStreaming: true, textureStreamingBudget: 128,
},
medium: {
renderScale: 1, minRenderScale: 0.65, msaaSamples: 2, antialiasing: 'temporal',
@@ -146,7 +153,7 @@ const PROFILES: Record<QualityLevel, QualityProfile> = {
shadowDistance: 0, detailScale: 0.8, particleLimit: 5000, postProcess: true, ssao: true,
waterWaves: 3, waterRippleCells: 64, waterDisplacement: true, waterMirror: true, waterCaustics: true,
skipEffects: ['ssr', 'depthOfField'],
- shelterMaskSize: 512, shadowSoftSamples: 12, drawCallBudget: 400,
+ shelterMaskSize: 512, shadowSoftSamples: 12, drawCallBudget: 400, textureStreaming: true, textureStreamingBudget: 256,
},
high: {
renderScale: 1, minRenderScale: 0.75, msaaSamples: 4, antialiasing: 'msaa',
@@ -157,7 +164,7 @@ const PROFILES: Record<QualityLevel, QualityProfile> = {
shadowDistance: 0, detailScale: 1, particleLimit: 20000, postProcess: true, ssao: true,
waterWaves: 4, waterRippleCells: 128, waterDisplacement: true, waterMirror: true, waterCaustics: true,
skipEffects: [],
- shelterMaskSize: 1024, shadowSoftSamples: 16, drawCallBudget: 1500,
+ shelterMaskSize: 1024, shadowSoftSamples: 16, drawCallBudget: 1500, textureStreaming: false, textureStreamingBudget: 0,
},
};
@@ -263,6 +270,8 @@ function resolveProfile(level: QualityLevel, value: unknown): QualityProfile {
shelterMaskSize: Math.round(finite(r.shelterMaskSize, base.shelterMaskSize, 64, 2048)),
shadowSoftSamples: Math.round(finite(r.shadowSoftSamples, base.shadowSoftSamples, 1, 32)),
drawCallBudget: Math.round(finite(r.drawCallBudget, base.drawCallBudget, 1, 100000)),
+ textureStreaming: typeof r.textureStreaming === 'boolean' ? r.textureStreaming : base.textureStreaming,
+ textureStreamingBudget: Math.round(finite(r.textureStreamingBudget, base.textureStreamingBudget, 0, 65536)),
};
}
diff --git a/sdk/src/render/textureStreaming.ts b/sdk/src/render/textureStreaming.ts
new file mode 100644
index 000000000..38cd70705
--- /dev/null
+++ b/sdk/src/render/textureStreaming.ts
@@ -0,0 +1,247 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file textureStreaming.ts
+ * @brief Holds each mesh texture at the size its views need, within the quality level's budget.
+ *
+ * @details The renderer says, for every texture a scene view drew a mesh with, how many texels across it needs for a texel to be about
+ * a pixel there (MeshPlugin, read through `renderer_takeTextureWants`). Four times a second this turns that into a size per
+ * texture — the next power of two above the need, never below `minEdge` nor above the file — and moves the textures that are
+ * the wrong size: decoded again from their file at the new size and put behind the same handle, so every material and
+ * renderer holding the handle sees it at once. A texture grows as soon as it is needed bigger and shrinks only after it has
+ * been needed smaller for a while, so a camera passing by does not make it go back and forth.
+ *
+ * What it streams: textures decoded from a file that only meshes draw. A texture a sprite, a UI element, a tilemap, a particle
+ * or a trail also draws is left at its size, since nothing measures what those need. What it saves is GPU memory; a texture is
+ * still read and decoded whole from its file the first time, so it saves neither the download nor the decode. A KTX2 file
+ * is cut at a level and not decoded again.
+ */
+import type { App, Plugin } from '../app/app';
+import { defineSystem, Schedule } from '../ecs/system';
+import { defineResource } from '../ecs/resource';
+import { getComponent } from '../ecs/component';
+import { playModeOnly } from '../ecs/env';
+import { Assets } from '../asset/AssetPlugin';
+import { engineApi } from '../ecs/bridge/engineApi';
+import { getTextureDimensions } from '../wasm/resourceManager';
+import { platformNow } from '../platform';
+import { Quality } from './qualityRuntime';
+
+/** What the streamer keeps of one texture. @experimental */
+export interface StreamedTexture {
+ handle: number;
+ /** The file's own size. */
+ fullWidth: number;
+ fullHeight: number;
+ /** The longer side it is held at now. */
+ edge: number;
+ /** The most texels across a view asked for lately; 0 when none has drawn it for `unseenAfter`. */
+ wanted: number;
+ /** When a view last drew it, ms. */
+ seenAt: number;
+ /** Since when the size it should be has been smaller than the one it is, ms; -1 when it has not. */
+ smallerSince: number;
+}
+
+/** @experimental */
+export interface TextureStreamingOptions {
+ /** No texture is held smaller than this on its longer side. */
+ minEdge: number;
+ /** The bytes the streamed textures may hold together; 0 is no cap. */
+ budgetBytes: number;
+ /** How long a texture must be needed smaller before it shrinks, ms. */
+ shrinkAfter: number;
+ /** How long a texture no view draws keeps the size it was needed at, ms; then it goes to `minEdge`. */
+ unseenAfter: number;
+}
+
+/** @experimental */
+export const TEXTURE_STREAMING_DEFAULTS: TextureStreamingOptions = { minEdge: 64, budgetBytes: 0, shrinkAfter: 3000, unseenAfter: 5000 };
+
+const nextPow2 = (n: number): number => 2 ** Math.ceil(Math.log2(Math.max(1, n)));
+
+/** The longer side a texture should be held at: the next power of two above what is wanted, between `minEdge` and the file. @experimental */
+export function streamTargetEdge(fullEdge: number, wanted: number, minEdge: number): number {
+ if (!(wanted > 0)) return Math.min(fullEdge, minEdge);
+ return Math.min(fullEdge, Math.max(minEdge, nextPow2(wanted)));
+}
+
+/** The bytes a texture held at `edge` on its longer side takes, with its chain, at four bytes a texel. @experimental */
+export function streamedBytes(fullWidth: number, fullHeight: number, edge: number): number {
+ const k = Math.min(1, edge / Math.max(fullWidth, fullHeight));
+ const w = Math.max(1, Math.round(fullWidth * k)), h = Math.max(1, Math.round(fullHeight * k));
+ return Math.round(w * h * 4 * 4 / 3);
+}
+
+/** A size to move a texture to. @experimental */
+export interface StreamMove {
+ handle: number;
+ edge: number;
+}
+
+/**
+ * What to move now: every texture whose size should change, the growing ones first and the most starved of those first, then the
+ * shrinking ones. With a budget, the sizes are first brought within it by halving the largest, so a scene that needs more than the
+ * budget is drawn a level softer rather than over it. Writes `smallerSince` on the entries it judges.
+ *
+ * @experimental
+ */
+export function planStreaming(entries: readonly StreamedTexture[], now: number, options: TextureStreamingOptions): StreamMove[] {
+ const target = new Map<number, number>();
+ for (const e of entries) {
+ const wanted = now - e.seenAt > options.unseenAfter ? 0 : e.wanted;
+ target.set(e.handle, streamTargetEdge(Math.max(e.fullWidth, e.fullHeight), wanted, options.minEdge));
+ }
+ if (options.budgetBytes > 0) {
+ let total = 0;
+ for (const e of entries) total += streamedBytes(e.fullWidth, e.fullHeight, target.get(e.handle)!);
+ while (total > options.budgetBytes) {
+ let worst: StreamedTexture | null = null, worstBytes = 0;
+ for (const e of entries) {
+ const edge = target.get(e.handle)!;
+ if (edge <= options.minEdge) continue;
+ const b = streamedBytes(e.fullWidth, e.fullHeight, edge);
+ if (b > worstBytes) { worst = e; worstBytes = b; }
+ }
+ if (!worst) break;
+ const halved = Math.max(options.minEdge, target.get(worst.handle)! / 2);
+ total += streamedBytes(worst.fullWidth, worst.fullHeight, halved) - worstBytes;
+ target.set(worst.handle, halved);
+ }
+ }
+ const grow: Array<StreamMove & { starved: number }> = [];
+ const shrink: StreamMove[] = [];
+ for (const e of entries) {
+ const edge = target.get(e.handle)!;
+ if (edge > e.edge) {
+ e.smallerSince = -1;
+ grow.push({ handle: e.handle, edge, starved: edge / e.edge });
+ } else if (edge < e.edge) {
+ if (e.smallerSince < 0) e.smallerSince = now;
+ if (now - e.smallerSince >= options.shrinkAfter) shrink.push({ handle: e.handle, edge });
+ } else {
+ e.smallerSince = -1;
+ }
+ }
+ grow.sort((a, b) => b.starved - a.starved);
+ return [...grow.map(({ handle, edge }) => ({ handle, edge })), ...shrink];
+}
+
+/** What streaming is doing, for a profiler and a check. @experimental */
+export class TextureStreaming {
+ readonly textures = new Map<number, StreamedTexture>();
+ /** Textures something other than a mesh draws, left at their size. */
+ readonly held = new Set<number>();
+ /** Textures that came back larger than asked (a host that cannot cut a file at a level): left alone from then on. */
+ readonly refused = new Set<number>();
+ moves = 0;
+ failures = 0;
+ inFlight = 0;
+ enabled = false;
+
+ /** The bytes the streamed textures take now, and would at their files' sizes. */
+ bytes(): { now: number; full: number } {
+ let now = 0, full = 0;
+ for (const t of this.textures.values()) {
+ now += streamedBytes(t.fullWidth, t.fullHeight, t.edge);
+ full += streamedBytes(t.fullWidth, t.fullHeight, Math.max(t.fullWidth, t.fullHeight));
+ }
+ return { now, full };
+ }
+}
+
+/** @experimental */
+export const TextureStreamingRes = defineResource<TextureStreaming>(new TextureStreaming(), 'TextureStreaming');
+
+/** Components whose `texture` field draws a texture no view measures the need of. */
+const UNMEASURED = ['Sprite', 'UIVisual', 'TilemapLayer', 'ParticleEmitter', 'TrailRenderer', 'SpriteMask'];
+
+/** @experimental */
+export class TextureStreamingPlugin implements Plugin {
+ name = 'textureStreaming';
+
+ build(app: App): void {
+ const state = new TextureStreaming();
+ app.insertResource(TextureStreamingRes, state);
+ const world = app.world;
+ let last = 0, lastHeld = -Infinity;
+ const MAX_IN_FLIGHT = 2;
+
+ const parse = (raw: string): Map<number, number> => {
+ const out = new Map<number, number>();
+ if (!raw) return out;
+ for (const pair of raw.split(',')) {
+ const cut = pair.indexOf(':');
+ if (cut > 0) out.set(Number(pair.slice(0, cut)), Number(pair.slice(cut + 1)));
+ }
+ return out;
+ };
+
+ const refreshHeld = (): void => {
+ state.held.clear();
+ for (const name of UNMEASURED) {
+ const def = getComponent(name);
+ if (!def) continue;
+ for (const e of world.getEntitiesWithComponents([def])) {
+ const t = (world.get(e, def) as { texture?: number }).texture;
+ if (typeof t === 'number' && t > 0) state.held.add(t);
+ }
+ }
+ };
+
+ app.addSystemToSchedule(Schedule.Last, defineSystem([], () => {
+ const quality = app.hasResource(Quality) ? app.getResource(Quality) : null;
+ const profile = quality?.profile;
+ state.enabled = !!profile?.textureStreaming;
+ const api = engineApi(app);
+ if (!state.enabled || !api?.renderer_takeTextureWants || !app.hasResource(Assets)) return;
+ const now = platformNow();
+ if (now - last < 250) return;
+ last = now;
+ if (now - lastHeld > 2000) { refreshHeld(); lastHeld = now; }
+
+ const assets = app.getResource(Assets);
+ for (const [handle, side] of parse(api.renderer_takeTextureWants())) {
+ if (state.held.has(handle)) { state.textures.delete(handle); continue; }
+ if (state.refused.has(handle)) continue;
+ let entry = state.textures.get(handle);
+ if (!entry) {
+ if (!assets.textureResizable(handle)) continue;
+ const size = getTextureDimensions(handle);
+ if (!size || size.width <= 0 || size.height <= 0) continue;
+ entry = { handle, fullWidth: size.width, fullHeight: size.height, edge: Math.max(size.width, size.height), wanted: 0, seenAt: now, smallerSince: -1 };
+ state.textures.set(handle, entry);
+ }
+ entry.wanted = entry.seenAt === now ? Math.max(entry.wanted, side) : side;
+ entry.seenAt = now;
+ }
+ for (const handle of state.held) state.textures.delete(handle);
+
+ const options: TextureStreamingOptions = {
+ ...TEXTURE_STREAMING_DEFAULTS,
+ budgetBytes: Math.max(0, profile!.textureStreamingBudget) * 1024 * 1024,
+ };
+ for (const move of planStreaming([...state.textures.values()], now, options)) {
+ if (state.inFlight >= MAX_IN_FLIGHT) break;
+ const entry = state.textures.get(move.handle)!;
+ const full = Math.max(entry.fullWidth, entry.fullHeight);
+ state.inFlight++;
+ const was = entry.edge;
+ entry.edge = move.edge; // claimed now, so the next pass does not ask again while this one decodes
+ void assets.resizeTexture(move.handle, move.edge >= full ? 0 : move.edge).then((size) => {
+ if (size && Math.max(size.width, size.height) > move.edge && move.edge < full) {
+ // It could not be made that small, and asking again would decode it again for nothing.
+ entry.edge = Math.max(size.width, size.height);
+ state.refused.add(move.handle);
+ state.textures.delete(move.handle);
+ } else if (size) { entry.edge = Math.max(size.width, size.height); state.moves++; }
+ else { entry.edge = was; state.failures++; state.textures.delete(move.handle); }
+ }, () => { entry.edge = was; state.failures++; state.textures.delete(move.handle); })
+ .finally(() => { state.inFlight--; });
+ }
+ }, { name: 'TextureStreamingSystem' }), { runIf: playModeOnly });
+ }
+}
+
+/** @experimental */
+export const textureStreamingPlugin: Plugin = new TextureStreamingPlugin();
diff --git a/sdk/src/wasm.ts b/sdk/src/wasm.ts
index e05411c79..0edb72a7f 100644
--- a/sdk/src/wasm.ts
+++ b/sdk/src/wasm.ts
@@ -688,6 +688,8 @@ export interface ESEngineModule {
renderer_getGpuDrawCalls?(): number;
/** Hand an entity's Impostor the atlas it draws from (0 takes it away). Optional: older engine builds do not carry it. */
renderer_setImpostorAtlas?(entity: number, texture: number, frames: number): void;
+ /** What size each texture was asked for since the last call, as `handle:side` pairs; texture streaming reads it. */
+ renderer_takeTextureWants?(): string;
/**
* GPU objects the device has created and not destroyed, for the resource
* census. Null when no device is initialized — a census must record those
diff --git a/sdk/tests/texture-streaming.test.ts b/sdk/tests/texture-streaming.test.ts
new file mode 100644
index 000000000..9d63926cf
--- /dev/null
+++ b/sdk/tests/texture-streaming.test.ts
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+import { describe, expect, it } from 'vitest';
+import {
+ planStreaming, streamTargetEdge, streamedBytes, TEXTURE_STREAMING_DEFAULTS, type StreamedTexture,
+} from '../src/render/textureStreaming';
+import { defaultQualityProfile, resolveQualityProfile } from '../src/render/quality';
+
+const tex = (handle: number, edge: number, wanted: number, seenAt = 0, full = 2048): StreamedTexture =>
+ ({ handle, fullWidth: full, fullHeight: full, edge, wanted, seenAt, smallerSince: -1 });
+
+describe('the size a texture is held at', () => {
+ it('is the next power of two above what is wanted, between the floor and the file', () => {
+ expect(streamTargetEdge(2048, 300, 64)).toBe(512);
+ expect(streamTargetEdge(2048, 512, 64)).toBe(512);
+ expect(streamTargetEdge(2048, 5000, 64)).toBe(2048);
+ expect(streamTargetEdge(2048, 3, 64)).toBe(64);
+ expect(streamTargetEdge(2048, 0, 64)).toBe(64);
+ expect(streamTargetEdge(32, 0, 64)).toBe(32);
+ });
+
+ it('costs a quarter for each level it drops', () => {
+ expect(streamedBytes(2048, 2048, 1024) / streamedBytes(2048, 2048, 2048)).toBeCloseTo(0.25, 3);
+ expect(streamedBytes(2048, 1024, 512)).toBe(Math.round(512 * 256 * 4 * 4 / 3));
+ });
+});
+
+describe('what streaming moves', () => {
+ const opts = { ...TEXTURE_STREAMING_DEFAULTS };
+
+ it('grows a texture at once, the most starved first', () => {
+ const a = tex(1, 256, 1000), b = tex(2, 64, 1000);
+ const moves = planStreaming([a, b], 100, opts);
+ expect(moves).toEqual([{ handle: 2, edge: 1024 }, { handle: 1, edge: 1024 }]);
+ });
+
+ it('shrinks one only once it has been needed smaller for a while', () => {
+ const a = tex(1, 2048, 100);
+ expect(planStreaming([a], 0, opts)).toEqual([]);
+ expect(planStreaming([a], opts.shrinkAfter - 1, opts)).toEqual([]);
+ expect(planStreaming([a], opts.shrinkAfter, opts)).toEqual([{ handle: 1, edge: 128 }]);
+ // Needed full size again before then, and the wait starts over.
+ const b = tex(2, 2048, 100);
+ planStreaming([b], 0, opts);
+ b.wanted = 4000;
+ expect(planStreaming([b], 10, opts)).toEqual([]);
+ expect(b.smallerSince).toBe(-1);
+ });
+
+ it('lets a texture no view draws go to the floor', () => {
+ const a = tex(1, 2048, 4000, 0);
+ planStreaming([a], opts.unseenAfter + 1, opts);
+ expect(planStreaming([a], opts.unseenAfter + 1 + opts.shrinkAfter, opts)).toEqual([{ handle: 1, edge: 64 }]);
+ });
+
+ it('halves the largest until the textures fit the budget', () => {
+ const big = tex(1, 64, 4000), small = tex(2, 64, 300);
+ const budget = streamedBytes(2048, 2048, 1024) + streamedBytes(2048, 2048, 512);
+ const moves = planStreaming([big, small], 0, { ...opts, budgetBytes: budget });
+ expect(moves).toEqual([{ handle: 1, edge: 1024 }, { handle: 2, edge: 512 }]);
+ });
+});
+
+describe('the quality levels', () => {
+ it('stream on the weaker two, with a budget, and not on high', () => {
+ expect(defaultQualityProfile('low').textureStreaming).toBe(true);
+ expect(defaultQualityProfile('medium').textureStreamingBudget).toBeGreaterThan(defaultQualityProfile('low').textureStreamingBudget);
+ expect(defaultQualityProfile('high').textureStreaming).toBe(false);
+ expect(resolveQualityProfile('high', { textureStreaming: true, textureStreamingBudget: 512 }).textureStreamingBudget).toBe(512);
+ });
+});
diff --git a/src/esengine/bindings/RendererBindings.cpp b/src/esengine/bindings/RendererBindings.cpp
index 5190d95d7..b343c3a75 100644
--- a/src/esengine/bindings/RendererBindings.cpp
+++ b/src/esengine/bindings/RendererBindings.cpp
@@ -1338,6 +1338,11 @@ i32 renderer_lodInspect(u32 view, u32 entity, uintptr_t outPtr) {
return 1;
}
+std::string renderer_takeTextureWants() {
+ if (!g_renderFrame) return {};
+ return g_renderFrame->takeTextureWants();
+}
+
void renderer_setImpostorAtlas(u32 entity, u32 texture, u32 frames) {
if (!g_renderFrame) return;
g_renderFrame->setImpostorAtlas(Entity::fromRaw(entity), texture, frames);
diff --git a/src/esengine/bindings/RendererBindings.hpp b/src/esengine/bindings/RendererBindings.hpp
index a6f5a391f..923177c06 100644
--- a/src/esengine/bindings/RendererBindings.hpp
+++ b/src/esengine/bindings/RendererBindings.hpp
@@ -303,6 +303,13 @@ void renderer_setLodPreview(u32 view, u32 entity, i32 level);
*/
void renderer_setImpostorAtlas(u32 entity, u32 texture, u32 frames);
+/**
+ * @brief What size each texture was asked for since the last call, as `handle:side` pairs joined by commas, and the record starts again.
+ * @details `side` is the texels across a view needs for a texel to be about a pixel where it drew a mesh with that texture (texture
+ * streaming reads it). Empty when nothing was drawn with a mesh texture.
+ */
+std::string renderer_takeTextureWants();
+
/**
* @brief Shade a view's meshes as authored (0), as neutral clay (1) or as flat albedo (2).
* @details View state, like the LOD preview beside it: the same scene drawn through
diff --git a/src/esengine/bindings/WebSDKEntry.cpp b/src/esengine/bindings/WebSDKEntry.cpp
index 1eb077d8d..84cdd86fe 100644
--- a/src/esengine/bindings/WebSDKEntry.cpp
+++ b/src/esengine/bindings/WebSDKEntry.cpp
@@ -808,6 +808,7 @@ EMSCRIPTEN_BINDINGS(esengine_renderer) {
emscripten::function("renderer_lodInspect", &esengine::renderer_lodInspect);
emscripten::function("renderer_setLodPreview", &esengine::renderer_setLodPreview);
emscripten::function("renderer_setImpostorAtlas", &esengine::renderer_setImpostorAtlas);
+ emscripten::function("renderer_takeTextureWants", &esengine::renderer_takeTextureWants);
emscripten::function("renderer_setViewShading", &esengine::renderer_setViewShading);
emscripten::function("renderer_setMeshEdgeBuffers", &esengine::renderer_setMeshEdgeBuffers);
emscripten::function("renderer_setViewSkinWeights", &esengine::renderer_setViewSkinWeights);
diff --git a/src/esengine/renderer/frame/RenderFrame.cpp b/src/esengine/renderer/frame/RenderFrame.cpp
index c1f21c0ee..f2555d997 100644
--- a/src/esengine/renderer/frame/RenderFrame.cpp
+++ b/src/esengine/renderer/frame/RenderFrame.cpp
@@ -1427,6 +1427,19 @@ void RenderFrame::buildSpriteMasks(ecs::Registry& registry) {
mask_entities_scratch_.clear();
}
+std::string RenderFrame::takeTextureWants() {
+ std::string out;
+ out.reserve(texture_wants_.size() * 12);
+ for (const auto& [texture, side] : texture_wants_) {
+ if (!out.empty()) out += ',';
+ out += std::to_string(texture);
+ out += ':';
+ out += std::to_string(static_cast<u32>(std::ceil(side)));
+ }
+ texture_wants_.clear();
+ return out;
+}
+
RenderFrameContext RenderFrame::makeContext() {
RenderFrameContext ctx{
context_,
diff --git a/src/esengine/renderer/frame/RenderFrame.hpp b/src/esengine/renderer/frame/RenderFrame.hpp
index 23347070d..78336c451 100644
--- a/src/esengine/renderer/frame/RenderFrame.hpp
+++ b/src/esengine/renderer/frame/RenderFrame.hpp
@@ -120,6 +120,20 @@ public:
/// (a tiled one), the second pass is geometry paid for twice.
void setDepthPrepass(bool on) { depth_prepass_ = on; }
void setPlanarQuality(f32 scale, u32 maxSize, f32 updatesPerSecond);
+ /**
+ * @brief Note that a view drew a mesh with texture @p texture (a resource handle) where it needs @p side texels across for a texel to
+ * be about a pixel. The largest asked since the last take is kept.
+ */
+ void wantTextureSide(u32 texture, f32 side) {
+ if (texture == 0 || !(side > 0.0f)) return;
+ f32& held = texture_wants_[texture];
+ if (side > held) held = side;
+ }
+ /** @brief What each texture was asked for since the last take, as `handle:side` pairs joined by commas; the record starts again. */
+ std::string takeTextureWants();
+ /** @brief The pixel height of the view being collected: what a texel is held to. */
+ f32 viewHeightPx() const { return static_cast<f32>(scene_viewport_.h > 0 ? scene_viewport_.h : height_); }
+
/** @brief An impostor's pictures: the atlas texture and how many frames along its side. */
struct ImpostorAtlas {
u32 texture = 0;
@@ -755,6 +769,8 @@ private:
u32 planar_texture_ = 0;
/// Each impostor's atlas, by raw entity id: set by the SDK's bake, read by the mesh collect.
std::unordered_map<u32, ImpostorAtlas> impostor_atlases_;
+ /// The largest side each texture was asked for since the SDK last took the record (texture streaming).
+ std::unordered_map<u32, f32> texture_wants_;
PlanarConstants planar_constants_;
glm::vec4 scene_clear_{0.0f, 0.0f, 0.0f, 1.0f};
rg::ResourceId planar_resource_ = rg::kNoResource;
diff --git a/src/esengine/renderer/plugins/MeshPlugin.cpp b/src/esengine/renderer/plugins/MeshPlugin.cpp
index 2cfdb23ae..db2e0fda3 100644
--- a/src/esengine/renderer/plugins/MeshPlugin.cpp
+++ b/src/esengine/renderer/plugins/MeshPlugin.cpp
@@ -883,6 +883,26 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
}
tick(2);
+ // How big a texture this mesh needs here: a texel about a pixel at the nearest point of its sphere. The scene's own views
+ // only — a shadow map's texels are not anyone's pixels — and only where the mesh says how much texture its surface holds.
+ if (ctx.purpose == RenderPurpose::Scene && ctx.frame && resident && resident->uvDensity > 0.0f && !placements) {
+ glm::vec3 centre(0.0f);
+ f32 radius = 0.0f;
+ lod::boundingSphere(position, rotation, scale, resident->localMin, resident->localMax, centre, radius);
+ const glm::mat4& vp = ctx.view_projection;
+ // NDC y per world unit is the length of the view-projection's second row under both projections; half the view's
+ // pixels span one unit of NDC.
+ const f32 ndcPerUnit = glm::length(glm::vec3(vp[0][1], vp[1][1], vp[2][1]));
+ const glm::vec4 eye = collect_ctx.camera.eye;
+ const f32 distance = eye.w > 0.5f ? std::max(glm::length(glm::vec3(eye) - centre) - radius, 1e-3f) : 1.0f;
+ const f32 pixelsPerUnit = ndcPerUnit * 0.5f * ctx.frame->viewHeightPx() / distance;
+ const f32 grow = std::max({std::abs(scale.x), std::abs(scale.y), std::abs(scale.z)});
+ const f32 side = pixelsPerUnit * grow / resident->uvDensity;
+ if (mesh.texture.isValid()) ctx.frame->wantTextureSide(mesh.texture.id(), side);
+ if (mesh.normalMap.isValid()) ctx.frame->wantTextureSide(mesh.normalMap.id(), side);
+ if (material) for (const auto& b : material->textures) if (b.texture.isValid()) ctx.frame->wantTextureSide(b.texture.id(), side);
+ }
+
u32 textureId = ctx.white_texture_id;
if (mesh.texture.isValid()) {
if (Texture* tex = ctx.resources.getTexture(mesh.texture)) {
diff --git a/src/esengine/resource/Mesh.hpp b/src/esengine/resource/Mesh.hpp
index 1f6dcda3e..517f3cf98 100644
--- a/src/esengine/resource/Mesh.hpp
+++ b/src/esengine/resource/Mesh.hpp
@@ -92,6 +92,10 @@ public:
* normal map was baked in. Without one the fragment derives a frame from screen-space derivatives. */
bool hasTangents = false;
+ /** UV units per world unit of the mesh's own surface (see MeshUvDensity.hpp): what a view asks a texture's size by. 0 without
+ * float positions and a float first UV set. */
+ f32 uvDensity = 0.0f;
+
/** Local-space bounds of the vertices, for the frustum cull. */
glm::vec3 localMin{0.0f, 0.0f, 0.0f};
glm::vec3 localMax{0.0f, 0.0f, 0.0f};
diff --git a/src/esengine/resource/MeshUvDensity.hpp b/src/esengine/resource/MeshUvDensity.hpp
new file mode 100644
index 000000000..e61e045b6
--- /dev/null
+++ b/src/esengine/resource/MeshUvDensity.hpp
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file MeshUvDensity.hpp
+ * @brief How much texture a mesh's surface holds: UV units per world unit of its own surface.
+ *
+ * @details The square root of the mesh's UV area over its surface area, both summed over its triangles. A texture `side` texels
+ * across, drawn on it, puts `side * density` texels on a world unit of the mesh at its own size — which is what a view
+ * compares with the pixels a world unit covers to say how big a texture it needs. An average over the whole mesh: a mesh
+ * that packs one part of its surface much denser than another is answered for the mean.
+ *
+ * Header-only: vertex bytes, their layout and the triangle list are the whole input. 0 for a mesh without float positions
+ * and a float first UV set, which no view then asks for a size of.
+ */
+#pragma once
+
+#include "../core/Types.hpp"
+#include "../renderer/rhi/GfxEnums.hpp"
+
+#include <cmath>
+#include <cstring>
+
+namespace esengine::resource {
+
+inline f32 meshUvDensity(const u8* vertexBytes, usize vertexByteCount, u32 vertexStride,
+ u32 positionOffset, u32 uvOffset, const u32* indices, usize indexCount) {
+ if (!vertexBytes || vertexStride == 0 || indexCount < 3) return 0.0f;
+ const usize vertexCount = vertexByteCount / vertexStride;
+ const auto read = [&](u32 vertex, u32 offset, u32 k) {
+ f32 value = 0.0f;
+ std::memcpy(&value, vertexBytes + static_cast<usize>(vertex) * vertexStride + offset + k * 4u, sizeof(f32));
+ return value;
+ };
+ f64 world = 0.0, tex = 0.0;
+ for (usize i = 0; i + 2 < indexCount; i += 3) {
+ const u32 a = indices[i], b = indices[i + 1], c = indices[i + 2];
+ if (a >= vertexCount || b >= vertexCount || c >= vertexCount) continue;
+ const f32 ux = read(b, positionOffset, 0) - read(a, positionOffset, 0);
+ const f32 uy = read(b, positionOffset, 1) - read(a, positionOffset, 1);
+ const f32 uz = read(b, positionOffset, 2) - read(a, positionOffset, 2);
+ const f32 vx = read(c, positionOffset, 0) - read(a, positionOffset, 0);
+ const f32 vy = read(c, positionOffset, 1) - read(a, positionOffset, 1);
+ const f32 vz = read(c, positionOffset, 2) - read(a, positionOffset, 2);
+ const f64 cx = uy * vz - uz * vy, cy = uz * vx - ux * vz, cz = ux * vy - uy * vx;
+ world += std::sqrt(cx * cx + cy * cy + cz * cz) * 0.5;
+ const f32 su = read(b, uvOffset, 0) - read(a, uvOffset, 0), sv = read(b, uvOffset, 1) - read(a, uvOffset, 1);
+ const f32 tu = read(c, uvOffset, 0) - read(a, uvOffset, 0), tv = read(c, uvOffset, 1) - read(a, uvOffset, 1);
+ tex += std::abs(static_cast<f64>(su) * tv - static_cast<f64>(sv) * tu) * 0.5;
+ }
+ return world > 0.0 ? static_cast<f32>(std::sqrt(tex / world)) : 0.0f;
+}
+
+} // namespace esengine::resource
diff --git a/src/esengine/resource/ResourceManager.cpp b/src/esengine/resource/ResourceManager.cpp
index eb17ea783..528ad49cf 100644
--- a/src/esengine/resource/ResourceManager.cpp
+++ b/src/esengine/resource/ResourceManager.cpp
@@ -13,6 +13,7 @@
#include "../core/Half.hpp"
#include "ResourceManager.hpp"
+#include "MeshUvDensity.hpp"
#include "MeshEdges.hpp"
#include "ShaderParser.hpp"
#include "../text/BitmapFont.hpp"
@@ -675,6 +676,17 @@ bool ResourceManager::realizeMesh(Mesh& mesh, ConstSpan<u8> vertexBytes, ConstSp
mesh.hasTangents = tangentChannel && hasNormals;
mesh.localMin = localMin;
mesh.localMax = localMax;
+ {
+ const GfxVertexAttribute* position = nullptr;
+ const GfxVertexAttribute* uv = nullptr;
+ for (const GfxVertexAttribute& c : channels) {
+ if (c.location == static_cast<u32>(MeshChannel::Position) && c.type == GfxDataType::Float && c.components >= 3) position = &c;
+ if (c.location == static_cast<u32>(MeshChannel::TexCoord0) && c.type == GfxDataType::Float && c.components >= 2) uv = &c;
+ }
+ mesh.uvDensity = position && uv
+ ? meshUvDensity(vertexBytes.data(), vertexBytes.size(), vertexStride, position->offset, uv->offset, indices.data(), indices.size())
+ : 0.0f;
+ }
mesh.inverseBind.clear();
if (skinned) {
const usize joints = inverseBind.size() / 16;
diff --git a/tools/check-project-settings.mjs b/tools/check-project-settings.mjs
index 9a057a311..28af24036 100644
--- a/tools/check-project-settings.mjs
+++ b/tools/check-project-settings.mjs
@@ -300,6 +300,8 @@ const BLOCKS = [
shelterMaskSize: { ui: 'project.rendering.quality.shelterMaskSize' },
shadowSoftSamples: { ui: 'project.rendering.quality.shadowSoftSamples' },
drawCallBudget: { ui: 'project.rendering.quality.drawCallBudget' },
+ textureStreaming: { ui: 'project.rendering.quality.textureStreaming' },
+ textureStreamingBudget: { ui: 'project.rendering.quality.textureStreamingBudget' },
skipEffects: { ui: 'project.rendering.quality.effects' },
},
},