Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add an image node to the material graph (#860)

Nguyen Xuan An8d045cf76dc8
diff --git a/changes/860-image-node.md b/changes/860-image-node.md
new file mode 100644
index 00000000..631a122d
--- /dev/null
+++ b/changes/860-image-node.md
@@ -0,0 +1 @@
+- A material graph has an `image` node that reads a catalog image with the graph's coordinates and mapping (#860)
diff --git a/changes/860-image-node.size.md b/changes/860-image-node.size.md
new file mode 100644
index 00000000..8911d46c
--- /dev/null
+++ b/changes/860-image-node.size.md
@@ -0,0 +1,3 @@
+- scene:agent-scene: the image node of the material graph (bilinear sampler, asset check) lets a graph read a catalog image.
+- scene:agent-scene+lazy: the same node, counted again in the lazy build.
+- goal:agent-scene: the image node is an agent-facing capability (textures from catalog images in a graph) that needs a sampler in the baked graph; 139 bytes gzip.
diff --git a/docs/agent-scene.md b/docs/agent-scene.md
index 4a6254b7..a73dbdb0 100644
--- a/docs/agent-scene.md
+++ b/docs/agent-scene.md
@@ -189,10 +189,11 @@ Placement and pattern nodes:
- `brick` (`columns` 8, `rows` 8 and even, `mortar` 0.05 of a brick, `offset` 0.5 for every other row, `seed`, `output`: `mortar` is 1 in the joint, `edge` is 0 at a joint and 1 at the brick centre, `cell` is one random value per brick) lays a running bond and costs 4. Whole columns and even rows keep it tiling.
- `whiteNoise` (`scale` 16, `seed`, `output`: `value` or `color` for three independent values) is one random value per cell of a `scale` by `scale` grid. It tiles and is the same for the same seed.
+- `image` (`asset`, the 64 hexadecimal characters of a catalog image ID from `scene_asset_catalog`; `output`: `color` is linear RGB decoded from sRGB, or `alpha`; `at` as for the other textures) reads a catalog image with bilinear filtering and repeats it across the tile, and costs 4. The host decodes the image into the same pool as texture sequence frames, so it follows their limits (1024 pixels a side, 64 MiB decoded in all); remote URLs are not accepted. Row 0 of the image is the top of the tile, as with a baked graph. A graph that names an image the host did not load fails when the material is baked, naming the asset.
- `coord` (`"generated"`, `"object"` or `"world"`) is a vector for `at`. `generated` is the unit tile, `object` and `world` are metres across the tile (texture u and v times the pattern `size`). A baked texture is projected onto the mesh, so it knows only the two axes of that projection and the third is 0; `object` and `world` are the same because box mapping reads the placed, scaled position.
- `mapping` (`vector`, default `{ "coord": "generated" }`; `scale` a number or `[x, y, z]`; `rotate` radians around Z; `offset` `[x, y, z]`) scales, turns and moves a vector, as Blender's Mapping node does. Feed it to `at`.
-These make a material work on a mesh without UVs: box mapping (the default) projects the baked graph along the object axes, so a brick wall or a noise pattern needs no unwrap. `vertex_color` on a mesh already multiplies the baked colour. Not in graphs yet: ambient occlusion, bevel and curvature masks, fresnel and an image texture node (they need per-vertex or per-pixel data that a bake from `u` and `v` alone does not have).
+These make a material work on a mesh without UVs: box mapping (the default) projects the baked graph along the object axes, so a brick wall or a noise pattern needs no unwrap. `vertex_color` on a mesh already multiplies the baked colour. Not in graphs yet: ambient occlusion, bevel and curvature masks, fresnel and a vertex group node (they need per-vertex or per-pixel data that a bake from `u` and `v` alone does not have).
Limits, checked while the recipe is read: at most 64 nodes in all outputs, depth 12 (the field expression limits), at most 8 ramp
stops, `resolution` of 256, 512 or 1024, and a work budget of 30 million units, where a plain node costs 1, a `noise` octave 3
diff --git a/src/agent-scene-capabilities.js b/src/agent-scene-capabilities.js
index 08677bed..ee720e2e 100644
--- a/src/agent-scene-capabilities.js
+++ b/src/agent-scene-capabilities.js
@@ -52,7 +52,7 @@ export const MAX_MATERIAL_PATTERNS = 16;
// expression limits; maxWork is work units per pixel times pixels, so a costly graph needs a smaller texture.
export const MATERIAL_GRAPH_OUTPUTS = Object.freeze(['color', 'roughness', 'metallic', 'emission', 'height', 'opacity', 'clearcoat', 'transmission', 'aov']);
export const MATERIAL_GRAPH_OPERATORS = Object.freeze([
- 'uv', 'coord', 'mapping', 'vec', 'ramp', 'noise', 'whiteNoise', 'brick', 'voronoi', 'wave', 'checker', 'gradient',
+ 'uv', 'coord', 'mapping', 'vec', 'ramp', 'image', 'noise', 'whiteNoise', 'brick', 'voronoi', 'wave', 'checker', 'gradient',
'add', 'sub', 'mul', 'div', 'min', 'max', 'pow', 'abs', 'floor', 'fract', 'sqrt', 'invert', 'clamp', 'smoothstep', 'mix',
'layer', 'hue', 'curve',
]);
@@ -79,11 +79,12 @@ export const MATERIAL_GRAPH_NODES = Object.freeze({
checker: '{"checker":{"count":8}} (even)',
gradient: '{"gradient":{"type":"u"|"v"|"radial"}}',
whiteNoise: '{"whiteNoise":{"scale":16,"seed":1,"output":"value"|"color"}} one random value per cell, tiles.',
+ image: '{"image":{"asset":"<64 hex catalog image asset ID>","output":"color"|"alpha"}} bilinear, repeats; color is linear RGB, decoded from sRGB. The host decodes catalog images (scene_asset_catalog); no URLs.',
brick: '{"brick":{"columns":8,"rows":4,"mortar":0.05,"offset":0.5,"seed":1,"output":"mortar"|"edge"|"cell"}} running bond; rows even.',
note: 'Each takes "at", a vector such as {"uv":"both"}. Scales are whole numbers 1-64.',
}),
uv: '{"uv":"u"|"v"|"both"} coordinates.',
- coordinates: '{"coord":"generated"|"object"|"world"} vector of where the texel sits: generated is the unit tile, object and world are metres across the tile (u * size); the third axis is 0 because a baked texture is projected. {"mapping":{"vector":node,"scale":n|[x,y,z],"offset":[x,y,z],"rotate":radians}} scales, turns around Z and moves a vector (default: generated). Feed either to "at". Meshes without UVs use box mapping; vertex_color already multiplies the baked colour. Not in graphs yet: AO, bevel, curvature, fresnel, image.',
+ coordinates: '{"coord":"generated"|"object"|"world"} vector of where the texel sits: generated is the unit tile, object and world are metres across the tile (u * size); the third axis is 0 because a baked texture is projected. {"mapping":{"vector":node,"scale":n|[x,y,z],"offset":[x,y,z],"rotate":radians}} scales, turns around Z and moves a vector (default: generated). Feed either to "at". Meshes without UVs use box mapping; vertex_color already multiplies the baked colour. Not in graphs yet: AO, bevel, curvature, fresnel, vertexGroup (per-pixel or per-vertex data a bake does not have).',
ramp: '{"ramp":{"value":node,"stops":[[0,[r,g,b]],[1,[r,g,b]]]}} 2-8 stops rising from 0 to 1.',
mixing: '{"layer":{"base","top","mask","edge","softness"}}; {"hue":{"color","shift","saturation","value"}}; {"curve":{"value","points":[[x,y]]}}.',
maths: 'add sub mul div min max pow take [a,b]; abs floor fract sqrt invert take one node; clamp smoothstep mix take [a,b,c]; {"vec":[r,g,b]}.',
diff --git a/src/agent-scene.js b/src/agent-scene.js
index 998bd7fe..72350221 100644
--- a/src/agent-scene.js
+++ b/src/agent-scene.js
@@ -989,8 +989,8 @@ function batchSignature(material) {
return `${material.tag}|${material.alphaMode}|${material.opacity}|${material.transmission > 0}|${material.doubleSided}|${JSON.stringify(material.pattern ?? material.map ?? null)}|${JSON.stringify(material.snippet)}`;
}
-function materialHandle(snapshot, sequence) {
- const holder = { snapshot, sequence };
+function materialHandle(snapshot, sequence, images) {
+ const holder = { snapshot, sequence, images };
const output = {};
for (const field of ['id', 'tag', ...MATERIAL_FIELDS]) {
Object.defineProperty(output, field, {
@@ -1007,7 +1007,7 @@ function materialHandle(snapshot, sequence) {
// Baked on first read, so validating or inspecting a recipe never pays for it.
get() {
if (field === 'map' && holder.sequence) return holder.sequence.descriptor;
- return holder.snapshot.pattern ? agentSceneModule('patterns').materialPatternTextures(holder.snapshot.pattern)[field] : undefined;
+ return holder.snapshot.pattern ? agentSceneModule('patterns').materialPatternTextures(holder.snapshot.pattern, holder.images)[field] : undefined;
},
});
}
@@ -1479,7 +1479,7 @@ export function createAgentScene(input, inputLimits = {}, options = {}) {
// The renderer keeps tag, blending, opacity and sidedness per batch, so a handle is reused only when all match.
const existing = [preferredHandle, activeHandle].find(handle => handle && batchSignature(handle.holder.snapshot) === batchSignature(material));
const assetOutput = assetRuntime.materialOutputs.get(material.id);
- const handle = assetOutput ? assetMaterialHandle(snapshot, assetOutput, preferredHandle, activeHandle) : existing ?? materialHandle(snapshot, material.map && agentSceneModule('assets').createTextureSequence(material.id, material.map, options.sequenceFrames));
+ const handle = assetOutput ? assetMaterialHandle(snapshot, assetOutput, preferredHandle, activeHandle) : existing ?? materialHandle(snapshot, material.map && agentSceneModule('assets').createTextureSequence(material.id, material.map, options.sequenceFrames), options.sequenceFrames);
materialHandles.set(material.id, handle);
}
const activeParts = new Map(state?.sourceRestParts.map(part => [part.name, part]) ?? []);
diff --git a/src/agent-texture-sequence.js b/src/agent-texture-sequence.js
index 0154cf46..8a550115 100644
--- a/src/agent-texture-sequence.js
+++ b/src/agent-texture-sequence.js
@@ -1,6 +1,7 @@
import { createImage } from './material-texture-kit.js';
import { TEXTURE_SEQUENCE_LIMITS } from './texture-sequence-limits.js';
import { invalidField } from './field-error.js';
+import { graphImageAssets } from './material-graph.js';
// An image sequence as the colour map of a recipe material: `map: { sequence: { frames, fps, loop } }`. The frames are
// catalog image assets that the host decodes before createAgentScene. The scene picks a frame from its motion time, so a
@@ -49,15 +50,20 @@ export function sequenceFrameIndex(sequence, time) {
return sequence.loop ? raw % count : Math.min(raw, count - 1);
}
-/** The catalog image IDs a recipe, or a list of edit operations, uses as sequence frames. */
+/** The catalog image IDs a recipe, or a list of edit operations, uses as sequence frames or as `image` nodes of a material graph. */
export function sequenceFrameIds(value) {
const ids = new Set();
const visit = (node, depth) => {
- if (!node || typeof node !== 'object' || depth > 8) return;
+ if (!node || typeof node !== 'object' || depth > 24) return;
if (Array.isArray(node)) {
for (const child of node) visit(child, depth + 1);
return;
}
+ if (node.type === 'graph' && node.graph) {
+ for (const asset of graphImageAssets(node.graph)) {
+ if (ASSET_ID.test(asset.toLowerCase())) ids.add(asset.toLowerCase());
+ }
+ }
const frames = node.map?.sequence?.frames;
if (Array.isArray(frames)) {
for (const frame of frames) {
diff --git a/src/material-graph.js b/src/material-graph.js
index b5c94251..32184531 100644
--- a/src/material-graph.js
+++ b/src/material-graph.js
@@ -520,8 +520,65 @@ function compileWhiteNoise(node, state, depth) {
return output === 'color' ? vectorValue(channel(1), channel(2), channel(3)) : scalarValue(channel(0));
}
+const IMAGE_OUTPUTS = ['color', 'alpha'];
+const IMAGE_COST = 4;
+const IMAGE_ASSET_PATTERN = /^[a-f0-9]{64}$/;
+
+const srgbToLinear = byte => {
+ const encoded = byte / 255;
+ return encoded <= 0.04045 ? encoded / 12.92 : ((encoded + 0.055) / 1.055) ** 2.4;
+};
+
+// One channel of an RGBA image read with bilinear filtering; the image repeats, so it tiles. Row 0 is the top, as is
+// row 0 of a baked graph, so v needs no flip. Colour channels are decoded from sRGB to linear before they are blended.
+function sampleImageChannel(image, positionU, positionV, channel, decode) {
+ const { width, height, data } = image;
+ const texelU = fract(positionU) * width - 0.5;
+ const texelV = fract(positionV) * height - 0.5;
+ const leftColumn = Math.floor(texelU);
+ const topRow = Math.floor(texelV);
+ const shareU = texelU - leftColumn;
+ const shareV = texelV - topRow;
+ const read = (column, row) => decode(data[(wrapCell(row, height) * width + wrapCell(column, width)) * 4 + channel]);
+ const top = mixNumbers(read(leftColumn, topRow), read(leftColumn + 1, topRow), shareU);
+ const bottom = mixNumbers(read(leftColumn, topRow + 1), read(leftColumn + 1, topRow + 1), shareU);
+ return mixNumbers(top, bottom, shareV);
+}
+
+/**
+ * The catalog image asset IDs a graph reads with `image` nodes, in the order they first appear. The host decodes these
+ * (the same pool as texture sequence frames) before the graph is baked.
+ */
+export function graphImageAssets(graph) {
+ const assets = new Set();
+ const visit = (value, depth) => {
+ if (!value || typeof value !== 'object' || depth > MATERIAL_GRAPH_LIMITS.maxDepth * 3) return;
+ if (!Array.isArray(value) && typeof value.image?.asset === 'string') assets.add(value.image.asset);
+ for (const child of Array.isArray(value) ? value : Object.values(value)) visit(child, depth + 1);
+ };
+ visit(graph, 0);
+ return [...assets];
+}
+
+// An image from the catalog, decoded by the host. Validating a graph needs no pixels, so without a pool only the
+// asset ID is checked; baking without the image throws.
+function compileImage(node, state, depth) {
+ const settings = textureSettings(node, 'image', ['asset', 'output', 'at']);
+ const asset = typeof settings.asset === 'string' ? settings.asset.toLowerCase() : settings.asset;
+ if (typeof asset !== 'string' || !IMAGE_ASSET_PATTERN.test(asset)) throw graphError('image asset must be the 64 hexadecimal characters of a catalog image asset ID; scene_asset_catalog lists them. Remote URLs are not accepted.');
+ const output = oneOf(settings.output ?? 'color', IMAGE_OUTPUTS, 'image output');
+ state.cost += IMAGE_COST;
+ const [positionU, positionV] = compilePosition(settings, state, depth);
+ const image = state.images ? state.images.get(asset) : null;
+ if (state.images && !image) throw graphError(`image ${asset.slice(0, 12)} was not loaded. The host decodes catalog images before the scene is created.`);
+ if (!image) return output === 'alpha' ? scalarValue(() => 0) : vectorValue(() => 0, () => 0, () => 0);
+ const at = (channel, decode) => (u, v) => sampleImageChannel(image, positionU(u, v), positionV(u, v), channel, decode);
+ if (output === 'alpha') return scalarValue(at(3, byte => byte / 255));
+ return vectorValue(at(0, srgbToLinear), at(1, srgbToLinear), at(2, srgbToLinear));
+}
+
const TEXTURE_COMPILERS = new Map([
- ['noise', compileNoise], ['whiteNoise', compileWhiteNoise], ['brick', compileBrick], ['voronoi', compileVoronoi], ['wave', compileWave], ['checker', compileChecker], ['gradient', compileGradient],
+ ['image', compileImage], ['noise', compileNoise], ['whiteNoise', compileWhiteNoise], ['brick', compileBrick], ['voronoi', compileVoronoi], ['wave', compileWave], ['checker', compileChecker], ['gradient', compileGradient],
]);
function compileNode(node, state, depth) {
@@ -561,12 +618,12 @@ const SCALAR_OUTPUTS = ['roughness', 'metallic', 'height', 'opacity', 'clearcoat
* Check a graph and compile each output. `size` is the metres one texture tile covers, which object and world coordinates report. Limits on nodes, depth and work per pixel are enforced here, before any
* pixel is evaluated. Returns { outputs: { name: compiled }, nodeCount, cost }.
*/
-export function compileMaterialGraph(graph, { size = 1 } = {}) {
+export function compileMaterialGraph(graph, { size = 1, images } = {}) {
if (!isPlainObject(graph)) throw graphError(`graph must be an object with outputs: ${MATERIAL_GRAPH_OUTPUTS.join(', ')}.`);
onlyKeys(graph, MATERIAL_GRAPH_OUTPUTS, 'graph');
const names = Object.keys(graph);
if (names.length === 0) throw graphError(`graph needs at least one output: ${MATERIAL_GRAPH_OUTPUTS.join(', ')}.`);
- const state = { nodeCount: 0, cost: 0, size };
+ const state = { nodeCount: 0, cost: 0, size, images };
const outputs = {};
for (const name of names) {
outputs[name] = compileNode(graph[name], state, 0);
@@ -615,8 +672,8 @@ function writeColor(data, offset, channels) {
}
/** Bake a normalized graph into texture descriptors, one for each output the graph has. Deterministic. */
-export function bakeMaterialGraph(graph, { resolution, bump, size = 1 }) {
- const compiled = compileMaterialGraph(graph, { size });
+export function bakeMaterialGraph(graph, { resolution, bump, size = 1 }, images) {
+ const compiled = compileMaterialGraph(graph, { size, images: images ?? new Map() });
checkMaterialGraphWork(compiled, resolution);
const { color, roughness, metallic, emission, height, opacity, clearcoat, transmission, aov } = compiled.outputs;
const pixelCount = resolution * resolution;
diff --git a/src/material-patterns.js b/src/material-patterns.js
index 78618e42..6532c3f6 100644
--- a/src/material-patterns.js
+++ b/src/material-patterns.js
@@ -1,5 +1,5 @@
import { MATERIAL_PATTERN_TYPES, MATERIAL_MAPPINGS } from './agent-scene-capabilities.js';
-import { bakeMaterialGraph, normalizeMaterialGraph } from './material-graph.js';
+import { bakeMaterialGraph, graphImageAssets, normalizeMaterialGraph } from './material-graph.js';
import { runKernel } from './kernel-offload.js';
// Procedural surface patterns for recipe materials: seeded, tileable and baked on the CPU into the same texture
@@ -222,12 +222,12 @@ export function normalizeMaterialPattern(value) {
/** Bake a normalized pattern into { map, normalMap, metallicRoughnessMap } texture descriptors. Deterministic. */
-export function bakeMaterialPattern(pattern) {
+export function bakeMaterialPattern(pattern, images) {
const placement = {
boxMapScale: pattern.mapping === 'uv' ? 0 : 1 / pattern.size,
boxMapAxis: ['x', 'y', 'z'].indexOf(pattern.axis),
};
- if (pattern.type === GRAPH_TYPE) return { ...bakeMaterialGraph(pattern.graph, pattern), ...placement };
+ if (pattern.type === GRAPH_TYPE) return { ...bakeMaterialGraph(pattern.graph, pattern, images), ...placement };
const { resolution, type, seed, count, colors, bump } = pattern;
if (!Object.hasOwn(PATTERN_SAMPLERS, type)) throw patternError(`type must be one of ${MATERIAL_PATTERN_TYPES.join(', ')}.`);
const sampler = PATTERN_SAMPLERS[type];
@@ -272,7 +272,8 @@ const MAX_BAKED_PATTERNS = 32;
const bakedByPatternObject = new WeakMap();
/** The baked textures of a pattern, shared by every material with the same definition. */
-export function materialPatternTextures(pattern) {
+export function materialPatternTextures(pattern, imagePool) {
+ const images = pattern.graph && graphImageAssets(pattern.graph).length ? imagePool : undefined;
const remembered = bakedByPatternObject.get(pattern);
if (remembered) return remembered;
const key = JSON.stringify(pattern);
@@ -280,7 +281,7 @@ export function materialPatternTextures(pattern) {
if (baked) {
bakedPatterns.delete(key);
} else {
- baked = runKernel('pattern', [pattern], bakeMaterialPattern);
+ baked = runKernel('pattern', images ? [pattern, images] : [pattern], bakeMaterialPattern);
if (bakedPatterns.size >= MAX_BAKED_PATTERNS) bakedPatterns.delete(bakedPatterns.keys().next().value);
}
bakedPatterns.set(key, baked);
diff --git a/tests/material-graph-image.test.js b/tests/material-graph-image.test.js
new file mode 100644
index 00000000..6bbc99bd
--- /dev/null
+++ b/tests/material-graph-image.test.js
@@ -0,0 +1,81 @@
+import './support/load-agent-scene-modules.mjs';
+import assert from 'node:assert/strict';
+import { createHash } from 'node:crypto';
+import test from 'node:test';
+import { bakeMaterialPattern, normalizeMaterialPattern } from '../src/material-patterns.js';
+import { compileMaterialGraph, graphImageAssets } from '../src/material-graph.js';
+import { sequenceFrameIds } from '../src/agent-texture-sequence.js';
+import { createAgentScene } from '../src/agent-scene.js';
+
+const IMAGE_ASSET = 'a'.repeat(64);
+const OTHER_ASSET = 'b'.repeat(64);
+
+function imagePool(width, height, pixel) {
+ const data = new Uint8ClampedArray(width * height * 4);
+ for (let row = 0; row < height; row++) {
+ for (let column = 0; column < width; column++) data.set(pixel(column, row), (row * width + column) * 4);
+ }
+ return new Map([[IMAGE_ASSET, { width, height, data }]]);
+}
+
+const srgbByteToLinear = byte => {
+ const encoded = byte / 255;
+ return encoded <= 0.04045 ? encoded / 12.92 : ((encoded + 0.055) / 1.055) ** 2.4;
+};
+
+const imagePattern = () => normalizeMaterialPattern({ type: 'graph', resolution: 256, graph: { color: { image: { asset: IMAGE_ASSET } } } });
+
+const colorHash = baked => createHash('sha256').update(Buffer.from(baked.map.image.data)).digest('hex');
+
+test('an image node matches a bilinear reference at texel centres and halfway between them', () => {
+ const pool = imagePool(2, 2, (column, row) => [column * 200, row * 100, 50, 255 - column * 100]);
+ const colorNode = compileMaterialGraph({ color: { image: { asset: IMAGE_ASSET } } }, { images: pool }).outputs.color;
+ const alphaNode = compileMaterialGraph({ opacity: { image: { asset: IMAGE_ASSET, output: 'alpha' } } }, { images: pool }).outputs.opacity;
+ // Texel centres sit at 0.25 and 0.75 on a 2 by 2 image.
+ assert.ok(Math.abs(colorNode.parts[0](0.75, 0.25) - srgbByteToLinear(200)) < 1e-9);
+ assert.ok(Math.abs(colorNode.parts[1](0.25, 0.75) - srgbByteToLinear(100)) < 1e-9);
+ assert.ok(Math.abs(alphaNode.evaluate(0.75, 0.25) - 155 / 255) < 1e-9);
+ const halfway = (srgbByteToLinear(0) + srgbByteToLinear(200)) / 2;
+ assert.ok(Math.abs(colorNode.parts[0](0.5, 0.25) - halfway) < 1e-9);
+ assert.ok(Math.abs(colorNode.parts[0](0.6, 0.3) - colorNode.parts[0](1.6, -0.7)) < 1e-9);
+});
+
+test('an image node validates its asset and output, and needs the pixels only when baking', () => {
+ assert.throws(() => compileMaterialGraph({ color: { image: { asset: 'https://example.com/a.png' } } }), /catalog image asset ID.*Remote URLs/);
+ assert.throws(() => compileMaterialGraph({ color: { image: { asset: IMAGE_ASSET, output: 'red' } } }), /image output must be one of color, alpha/);
+ assert.throws(() => compileMaterialGraph({ color: { image: { asset: IMAGE_ASSET, extra: 1 } } }), /image does not take "extra"/);
+ assert.throws(() => bakeMaterialPattern(imagePattern(), new Map()), /image aaaaaaaaaaaa was not loaded/);
+});
+
+test('a baked image graph is deterministic and its colour map carries the image', () => {
+ const pool = imagePool(4, 4, (column, row) => [column * 60, row * 60, 128, 255]);
+ const first = bakeMaterialPattern(imagePattern(), pool);
+ assert.equal(colorHash(first), colorHash(bakeMaterialPattern(imagePattern(), pool)));
+ const offset = (32 * 256 + 32) * 4;
+ assert.ok(first.map.image.data[offset] < 60);
+ assert.ok(first.map.image.data[offset + 2] > 100);
+ assert.notEqual(colorHash(first), colorHash(bakeMaterialPattern(imagePattern(), imagePool(4, 4, () => [255, 255, 255, 255]))));
+});
+
+test('graphImageAssets lists the images a graph reads and the host finds them in a recipe', () => {
+ const graph = { color: { mix: [{ image: { asset: IMAGE_ASSET } }, { image: { asset: OTHER_ASSET, at: { mapping: { scale: 2 } } } }, 0.5] } };
+ assert.deepEqual(graphImageAssets(graph), [IMAGE_ASSET, OTHER_ASSET]);
+ assert.deepEqual(graphImageAssets({ color: { noise: { scale: 4 } } }), []);
+ const recipe = { materials: [{ id: 'wall', tag: 'surface', pattern: { type: 'graph', graph } }] };
+ assert.deepEqual(sequenceFrameIds(recipe).sort(), [IMAGE_ASSET, OTHER_ASSET]);
+});
+
+test('a scene material with an image graph bakes through the frame pool the host decodes', () => {
+ const pool = imagePool(2, 2, column => [column * 255, 0, 0, 255]);
+ const recipe = {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [{ id: 'block', type: 'grid', segments: 1 }],
+ materials: [{ id: 'photo', tag: 'surface', pattern: { type: 'graph', resolution: 256, mapping: 'uv', graph: { color: { image: { asset: IMAGE_ASSET } } } } }],
+ nodes: [{ id: 'a', geometry: 'block', material: 'photo' }],
+ };
+ const scene = createAgentScene(recipe, {}, { sequenceFrames: pool });
+ const material = scene.parts[0].material;
+ const data = material.map.image.data;
+ assert.equal(data.length, 256 * 256 * 4);
+ assert.ok(data[(128 * 256 + 255) * 4] > data[(128 * 256) * 4]);
+});
diff --git a/tests/material-graph.test.js b/tests/material-graph.test.js
index c70332bb..68a0ffcc 100644
--- a/tests/material-graph.test.js
+++ b/tests/material-graph.test.js
@@ -177,7 +177,7 @@ test('graph and preset fields do not mix', () => {
test('every capability operator compiles, and no other operator does', () => {
const samples = {
- uv: { uv: 'u' }, coord: { coord: 'object' }, mapping: { mapping: {} }, whiteNoise: { whiteNoise: {} }, brick: { brick: {} }, vec: { vec: [1, 2, 3] }, ramp: { ramp: { value: 0.5, stops: [[0, 0], [1, 1]] } },
+ uv: { uv: 'u' }, image: { image: { asset: 'a'.repeat(64) } }, coord: { coord: 'object' }, mapping: { mapping: {} }, whiteNoise: { whiteNoise: {} }, brick: { brick: {} }, vec: { vec: [1, 2, 3] }, ramp: { ramp: { value: 0.5, stops: [[0, 0], [1, 1]] } },
noise: { noise: {} }, voronoi: { voronoi: {} }, wave: { wave: {} }, checker: { checker: {} }, gradient: { gradient: {} },
layer: { layer: { base: [0, 0, 0], top: [1, 1, 1], mask: 0.5 } }, hue: { hue: { color: [1, 0, 0], shift: 0.1 } }, curve: { curve: { value: 0.5, points: [[0, 0], [1, 1]] } },
};