3D engine that AI agents drive
Parallax-correct the object probe lookup (part of #167)
Nguyen Xuan Anb411b8267cd5
diff --git a/bench/reference/README.md b/bench/reference/README.mdindex d9598371..cb3644c0 100644--- a/bench/reference/README.md+++ b/bench/reference/README.md@@ -970,3 +970,21 @@ Scores against Cycles, GPU, `--skip-blender` on the stored Cycles and EEVEE arraNo scene got worse by more than 0.0002 (`spot-area`, `fog-valley` 0.1855 to 0.1859). Frame cost on the RTX 3060 (`frame-time.mjs`, median of three runs): `brick-flat` 0.375 to 0.39 ms before and 0.375 to 0.38 after, `metal` 0.44 to 0.45 before and 0.44 after, so the extra lookup is below the timer's resolution. SwiftShader `transparency` in `bench/runtime`: 345.9 ms before, 347.5 after (15 frames; that scene has no environment map, so the shader path is the same).`metal` is still 0.0046 above EEVEE, so the first box of #167 stays open. Where the rest is, by 6 by 8 block of the FLIP map (measured): the largest error is the lower half of the chrome sphere, where 67js is 0.062 brighter in display luminance than Cycles (EEVEE 0.022). Cycles shows the floor there darkened by the sphere's own contact shadow, and the sphere's probe traces the scene without the sphere, so the floor it shows is lit all the way to its base (inferred; the "seeing through the part itself" try above, 0.1117, was the attempt at that and was not kept). The rest of the frame is 67js 0.01 to 0.03 darker than Cycles, and EEVEE is darker still, so there is little left to gain from the direct specular. The floor holds over half of the error (0.0639 of 0.1147 in the table above) and is not a reflector.++## Parallax-corrected object probes (2026-11-03, part of issue #167)++An object probe sits at its part's centre, so a ray reflected at a point on the surface was looked up as if it had left the centre. The probe now records the level floor under it (one ray straight down at trace time, kept only where the surface normal is within 0.99 of up), and the shader sends a reflected ray that heads down toward the point where it meets that plane, seen from the probe: `normalize(hit - probeCentre)`, the plane-projected cube map correction. A probe with no level floor, and every room probe, keeps the plain lookup. The code is compiled in only with the `probe` shader feature, so a scene with no reflective part has the same shader code as before.++Main reproduced `baseline.json` to four digits on the stored arrays first. Scores against Cycles, GPU:++| Variant | `metal` | `car-paint` | `brushed-film` | `material-spheres` | `metal-room` |+| --- | --- | --- | --- | --- | --- |+| Main | 0.0962 | 0.1028 | 0.1026 | 0.1185 | 0.5071 |+| Parallax | 0.0957 | 0.1021 | 0.1016 | 0.1180 | 0.5071 |+| Parallax, plus the part's shadow on the probe's floor (full sun) | 0.0984 | 0.1020 | 0.1000 | 0.1178 | 0.5072 |++Every other scene is unchanged (it has no object probe). The shadow on top of the parallax lookup helps the other three scenes and makes `metal` worse again (0.0957 to 0.0984), so it was not kept; the correction alone is. `metal` is still 0.0041 above EEVEE (0.0916), so the first box of #167 stays open. Parallax moved the sphere's bottom edge less than hoped, which points at the floor term and the bounce light from the other metal parts, not the lookup, as what is left (guessed, not measured).++Frame cost, RTX 3060 (`frame-time.mjs`, 200 frames): `brick-flat` plain 0.385 ms (0.375 to 0.39 before), `metal` plain 0.43 ms (0.44 to 0.45 before). SwiftShader `transparency` in `bench/runtime`: 350.3 ms (345.9 and 347.5 before; 15 frames, machine under load).++Bundle size: the floor trace and the lookup correction add about 250 bytes gzip to the three bundles that carry reflection probes (`entry:agent-scene+lazy`, `entry:agent-asset+lazy`, `scene:agent-scene+lazy`), so each budget in `bench/size/budgets.json` is raised by 280 bytes. Measured: they were over by 244, 239 and 253 bytes.diff --git a/bench/reference/baseline.json b/bench/reference/baseline.jsonindex a58bf0ad..f9edcf2a 100644--- a/bench/reference/baseline.json+++ b/bench/reference/baseline.json@@ -125,18 +125,18 @@"gpu-vs-cpu": 0.0071},"brushed-film": {- "gpu-vs-cycles": 0.1026,- "gpu-vs-eevee": 0.0841,- "cpu-vs-cycles": 0.1024,- "cpu-vs-eevee": 0.0842,+ "gpu-vs-cycles": 0.1016,+ "gpu-vs-eevee": 0.0835,+ "cpu-vs-cycles": 0.1014,+ "cpu-vs-eevee": 0.0835,"eevee-vs-cycles": 0.0985,"gpu-vs-cpu": 0.0061},"car-paint": {- "gpu-vs-cycles": 0.1028,- "gpu-vs-eevee": 0.0641,- "cpu-vs-cycles": 0.1025,- "cpu-vs-eevee": 0.0642,+ "gpu-vs-cycles": 0.1021,+ "gpu-vs-eevee": 0.0633,+ "cpu-vs-cycles": 0.1018,+ "cpu-vs-eevee": 0.0634,"eevee-vs-cycles": 0.0844,"gpu-vs-cpu": 0.0058},@@ -229,10 +229,10 @@"gpu-vs-cpu": 0.0066},"material-spheres": {- "gpu-vs-cycles": 0.1185,- "gpu-vs-eevee": 0.0686,- "cpu-vs-cycles": 0.1184,- "cpu-vs-eevee": 0.0686,+ "gpu-vs-cycles": 0.118,+ "gpu-vs-eevee": 0.0681,+ "cpu-vs-cycles": 0.1179,+ "cpu-vs-eevee": 0.0681,"eevee-vs-cycles": 0.1053,"gpu-vs-cpu": 0.0082},@@ -245,18 +245,18 @@"gpu-vs-cpu": 0.0276},"metal": {- "gpu-vs-cycles": 0.0962,- "gpu-vs-eevee": 0.0821,- "cpu-vs-cycles": 0.096,- "cpu-vs-eevee": 0.0823,+ "gpu-vs-cycles": 0.0957,+ "gpu-vs-eevee": 0.0806,+ "cpu-vs-cycles": 0.0955,+ "cpu-vs-eevee": 0.0808,"eevee-vs-cycles": 0.0916,- "gpu-vs-cpu": 0.0063+ "gpu-vs-cpu": 0.0064},"metal-room": {"gpu-vs-cycles": 0.5071,- "gpu-vs-eevee": 0.1089,+ "gpu-vs-eevee": 0.109,"cpu-vs-cycles": 0.5067,- "cpu-vs-eevee": 0.1095,+ "cpu-vs-eevee": 0.1097,"eevee-vs-cycles": 0.5262,"gpu-vs-cpu": 0.0062},diff --git a/bench/reference/images/brushed-film.jpg b/bench/reference/images/brushed-film.jpgindex ef9453bf..ee78a795 100644Binary files a/bench/reference/images/brushed-film.jpg and b/bench/reference/images/brushed-film.jpg differdiff --git a/bench/reference/images/car-paint.jpg b/bench/reference/images/car-paint.jpgindex 33c9d628..8fc3f65d 100644Binary files a/bench/reference/images/car-paint.jpg and b/bench/reference/images/car-paint.jpg differdiff --git a/bench/reference/images/material-spheres.jpg b/bench/reference/images/material-spheres.jpgindex 6b177877..8b02dc20 100644Binary files a/bench/reference/images/material-spheres.jpg and b/bench/reference/images/material-spheres.jpg differdiff --git a/bench/reference/images/metal-room.jpg b/bench/reference/images/metal-room.jpgindex 7d0efd3f..6779c043 100644Binary files a/bench/reference/images/metal-room.jpg and b/bench/reference/images/metal-room.jpg differdiff --git a/bench/reference/images/metal.jpg b/bench/reference/images/metal.jpgindex dccd696f..c3526134 100644Binary files a/bench/reference/images/metal.jpg and b/bench/reference/images/metal.jpg differdiff --git a/bench/size/budgets.json b/bench/size/budgets.jsonindex 8ed69987..839c22b8 100644--- a/bench/size/budgets.json+++ b/bench/size/budgets.json@@ -34,9 +34,9 @@"entry:path-tracer": 14600,"entry:texture-set": 900,"entry:agent-scene": 77780,- "entry:agent-scene+lazy": 259700,+ "entry:agent-scene+lazy": 259980,"entry:agent-asset": 109130,- "entry:agent-asset+lazy": 271100,+ "entry:agent-asset+lazy": 271380,"entry:hdri": 3300,"entry:camera-effects": 8600,"entry:water-reflection": 26000,@@ -48,6 +48,6 @@"scene:gltf-box": 84330,"scene:gltf-box+lazy": 87650,"scene:agent-scene": 105110,- "scene:agent-scene+lazy": 287020+ "scene:agent-scene+lazy": 287300}}diff --git a/changes/167-parallax.md b/changes/167-parallax.mdnew file mode 100644index 00000000..1f5728bc--- /dev/null+++ b/changes/167-parallax.md@@ -0,0 +1 @@+- Look up an object probe along the ray from its centre to where the reflected ray meets the floor under it, so chrome shows the floor next to its contact point where the real ray lands (part of issue #167). `metal` goes from 0.0962 to 0.0957 against Cycles; `car-paint`, `brushed-film` and `material-spheres` also improve and no scene gets worse.diff --git a/src/environment.js b/src/environment.jsindex aa7bfdba..56693388 100644--- a/src/environment.js+++ b/src/environment.js@@ -294,12 +294,12 @@ vec3 multipleScatterGain(vec2 lookup, vec3 baseReflectance) {return 1.0 / (1.0 - missingEnergy * averageFresnel);}// specularWeight scales only the reflected sky; leaf cards use less, since their bent normals face the view at grazing angles.-vec3 environmentLighting(vec3 normal, vec3 specularNormal, vec3 baseReflectance, vec3 viewDirection, vec3 color, float metallic, float roughness, float occlusion, float ambient, float specularWeight) {+vec3 environmentLighting(vec3 normal, vec3 specularNormal, vec3 baseReflectance, vec3 viewDirection, vec3 reflected, vec3 color, float metallic, float roughness, float occlusion, float ambient, float specularWeight) {float normalView = clamp(dot(normal, viewDirection), 0.001, 1.0);vec2 lookup = texture(environmentBrdf, vec2(normalView, roughness)).rg;vec3 singleScatter = baseReflectance * lookup.x + lookup.y;vec3 specularColor = singleScatter * multipleScatterGain(lookup, baseReflectance) * specularWeight;- vec3 specular = environmentRadiance(reflect(-viewDirection, specularNormal), roughness * environmentLevelMax) * specularColor;+ vec3 specular = environmentRadiance(reflected, roughness * environmentLevelMax) * specularColor;vec3 diffuse = environmentDiffuse(normal) * color * (1.0 - metallic) * (1.0 - specularColor);return (specular + diffuse) * occlusion * ambient;}diff --git a/src/probe-trace.js b/src/probe-trace.jsindex cad5639d..652e34c7 100644--- a/src/probe-trace.js+++ b/src/probe-trace.js@@ -16,6 +16,7 @@ const PROBE_SAMPLES = 32;const PROBE_BOUNCES = 2;const MAX_PROBE_PIXELS = 65536;+const LEVEL_FLOOR_NORMAL = 0.99;// Object probes: a sharp mirror needs a finer map than a room probe, and one bounce is enough for what a reflection shows.export const MAXIMUM_OBJECT_PROBES = 8;@@ -99,10 +100,20 @@ export function traceProbeMaps(parts, lighting, positions, { width = 64, samples}}}- return { data, width, height, hitFraction: hits / (width * height) };+ return { data, width, height, hitFraction: hits / (width * height), floorHeight: floorBelow(world, tracer, position) };});}+// The height of the horizontal surface straight under a probe, or null when the ground there is not level. A reflected ray+// that heads down meets this plane, which is how the lookup is corrected for the probe sitting away from the shaded point.+function floorBelow(world, tracer, position) {+ const hit = tracer.trace(position[0], position[1], position[2], 0, -1, 0, Infinity, false);+ if (hit.nearestTriangle < 0) return null;+ const normal = [0, 0, 0];+ hitNormal(world, hit.nearestTriangle, [0, -1, 0], normal);+ return normal[1] > LEVEL_FLOOR_NORMAL ? position[1] - hit.nearestDistance : null;+}+/** Validates the probe list of a recipe: up to MAXIMUM_PROBES of `{ position, radius? }`. */export function normalizeProbeList(list) {if (!Array.isArray(list) || list.length > MAXIMUM_PROBES) {diff --git a/src/reflection-probes.js b/src/reflection-probes.jsindex edd63aa9..6ef2d22f 100644--- a/src/reflection-probes.js+++ b/src/reflection-probes.js@@ -43,9 +43,12 @@ function probedMaterial(liveMaterial, probeMap, wrappers) {/** Gives each part the nearest probe that reaches it, as a material wrapper that keeps reading the live material. */export function placeProbes(parts, probeTrace, { rotation = 0, wrappers = new Map() } = {}) {- const probes = probeTrace.traced.map(({ position, radius, width, height, data, hitFraction, part }) => ({- position, radius, hitFraction, part, map: createEnvironmentMap({ width, height, data, intensity: 1, rotation }),- }));+ const probes = probeTrace.traced.map(({ position, radius, width, height, data, hitFraction, part, floorHeight = null }) => {+ const map = createEnvironmentMap({ width, height, data, intensity: 1, rotation });+ // An object probe knows the floor under it, so the lookup can follow the ray from the shaded point to that floor.+ const parallax = part !== undefined && floorHeight !== null ? [...position, floorHeight] : null;+ return { position, radius, hitFraction, part, map: parallax ? Object.freeze({ ...map, parallax }) : map };+ });const probesByPart = new Map(probes.filter(probe => probe.part !== undefined).map(probe => [probe.part, probe]));const sharedProbes = probes.filter(probe => probe.part === undefined);const probedParts = parts.map((part, index) => {@@ -69,9 +72,17 @@ export function applyReflectionProbes(parts, lighting, specs, options = {}) {}registerShaderFeatureCode('probe', Object.freeze({- declarations: 'uniform float probeEnabled;\n',+ declarations: 'uniform float probeEnabled;\nuniform vec4 probeFloor;\n',// Divides out the world-mode weight the surroundings get below, because a probe's radiance is already the real light.probeSurroundings: ' if (probeEnabled > 0.5) surroundings = environmentRadiance(reflected, roughness * environmentLevelMax) / max(mix(1.0, 0.5 * daylight, worldMode), 0.05);\n',+ // probeEnabled is 2 when the probe knows its floor: a ray heading down is looked up toward where it meets that floor,+ // seen from the probe, as a box-projected cube map does, because the probe sits at the part's centre and not at the shaded point.+ probeReflection: [+ ' if (probeEnabled > 1.5 && reflected.y < -0.01 && worldPosition.y > probeFloor.w) {',+ ' reflected = normalize(worldPosition + reflected * ((probeFloor.w - worldPosition.y) / reflected.y) - probeFloor.xyz);',+ ' }',+ '',+ ].join('\n'),}));const rendererStates = new WeakMap();@@ -79,7 +90,8 @@ const rendererStates = new WeakMap();function stateOf(renderer) {if (!rendererStates.has(renderer)) {const enabledLocation = renderer.context.getUniformLocation(renderer.program, 'probeEnabled');- rendererStates.set(renderer, { resources: new Map(), enabledLocation, bound: false });+ const floorLocation = renderer.context.getUniformLocation(renderer.program, 'probeFloor');+ rendererStates.set(renderer, { resources: new Map(), enabledLocation, floorLocation, bound: false });}return rendererStates.get(renderer);}@@ -150,7 +162,8 @@ function bindProbe(renderer, mapState) {// The environment's irradiance still lights the diffuse term, and the intensity uniform is now 1.context.uniform3fv(uniforms['environmentIrradiance[0]'], renderer.environmentResources.irradiance.map(value => value * globalMap.intensity));}- context.uniform1f(state.enabledLocation, 1);+ if (probeMap.parallax) context.uniform4fv(state.floorLocation, probeMap.parallax);+ context.uniform1f(state.enabledLocation, probeMap.parallax ? 2 : 1);}const previousBind = layerTextureSupport.bind;diff --git a/src/shaders.js b/src/shaders.jsindex 1423c841..6b75f4f4 100644--- a/src/shaders.js+++ b/src/shaders.js@@ -877,11 +877,11 @@ ${featureCode("linkArea")} vec3 lightNormal = areaLightNormals[index].xyz;}// Leaf and grass cards keep a faint sheen; full sky reflection washed grazing cards grey-blue.float skyReflection = canopyShaded ? 0.3 : 1.0;- if (ENVIRONMENT_ENABLED) {- radiance += environmentLighting(normal, specularNormal, baseReflectance, viewDirection, color, metallic, roughness, occlusion, ambientIntensity, skyReflection);+ vec3 reflected = reflect(-viewDirection, specularNormal);+${featureCode("probeReflection")} if (ENVIRONMENT_ENABLED) {+ radiance += environmentLighting(normal, specularNormal, baseReflectance, viewDirection, reflected, color, metallic, roughness, occlusion, ambientIntensity, skyReflection);} else {// Outdoors the surroundings are sky and haze; reflecting the studio softboxes there lit grazing leaves white and cyan.- vec3 reflected = reflect(-viewDirection, specularNormal);vec3 surroundings = worldMode > 0.5 ? mix(fogColor * 0.8, vec3(0.20, 0.38, 0.52) * 1.6, smoothstep(-0.15, 0.7, reflected.y)) : studio(reflected, roughness);${featureCode("probeSurroundings")} radiance += surroundings * reflectance * skyReflection * (1.0 - roughness * 0.55) * mix(1.0, 0.5 * daylight, worldMode) * occlusion * ambientIntensity;radiance += color * (1.0 - metallic) * mix(vec3(0.16, 0.20, 0.22), vec3(0.19, 0.25, 0.30) * max(daylight, 0.08), worldMode) * (0.45 + 0.55 * max(normal.y, 0.0)) * occlusion * ambientIntensity;diff --git a/src/water-reflection.js b/src/water-reflection.jsindex 19d588d6..0e9e8289 100644--- a/src/water-reflection.js+++ b/src/water-reflection.js@@ -125,7 +125,7 @@ uniform float waterReflectionEnabled;}mirrored = (hdrEnabled > 0.5 ? mirrored / ${BLUR_TAPS}.0 : pow(mirrored / ${BLUR_TAPS}.0, vec3(2.2))) / max(exposure, 1e-4);float fresnelWeight = ${FRESNEL_BASE} + (1.0 - ${FRESNEL_BASE}) * pow(1.0 - max(viewDirection.y, 0.0), 5.0);- vec3 skyLight = ENVIRONMENT_ENABLED ? environmentLighting(normal, specularNormal, baseReflectance, viewDirection, vec3(0.0), metallic, roughness, occlusion, ambientIntensity, skyReflection) : vec3(0.0);+ vec3 skyLight = ENVIRONMENT_ENABLED ? environmentLighting(normal, specularNormal, baseReflectance, viewDirection, reflect(-viewDirection, specularNormal), vec3(0.0), metallic, roughness, occlusion, ambientIntensity, skyReflection) : vec3(0.0);radiance = ENVIRONMENT_ENABLED ? max(radiance - skyLight, vec3(0.0)) + mirrored * ${REFLECTION_GAIN} * fresnelWeight : mix(radiance, mirrored * ${REFLECTION_GAIN}, fresnelWeight);}`,diff --git a/tests/reflection-probes.test.js b/tests/reflection-probes.test.jsindex 61753baa..23a41d6c 100644--- a/tests/reflection-probes.test.js+++ b/tests/reflection-probes.test.js@@ -138,6 +138,21 @@ test('the probe of a part never hits the part itself and sees the floor and theassert.ok(probe.hitFraction > 0.45 && probe.hitFraction < 0.65, `floor and block fill about half the view: ${probe.hitFraction}`);});+test('an object probe records the level floor under it so the lookup can follow the ray to the floor', () => {+ const parts = objectScene();+ const trace = traceObjectProbes(parts, openAir, { width: 32 });+ assert.ok(Math.abs(trace.traced[0].floorHeight) < 1e-3, `floor top is at 0: ${trace.traced[0].floorHeight}`);+ const { probes } = placeProbes(parts, trace);+ assert.deepEqual(probes[0].map.parallax.map(value => Math.round(value * 100) / 100), [-1, 0.5, 0, 0]);+ assert.match(createFragmentShader(['probe']), /probeFloor/);+});++test('a room probe keeps the plain lookup', () => {+ const parts = objectScene();+ const trace = traceProbes(parts, openAir, [{ position: [0, 1, 0], radius: 9 }]);+ assert.equal(placeProbes(parts, trace).probes[0].map.parallax, undefined);+});+test('a scene with no reflective part, or too many, traces nothing', () => {const plain = objectScene().map(part => ({ ...part, material: { ...part.material, metallic: 0 } }));assert.equal(traceObjectProbes(plain, openAir).traced.length, 0);