Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(sound): layers struck together, sounds that differ, and a recording to be like - a group is several layers as one event to repeat or scatter, variants makes up to eight files that are the same sound and not the same samples, and reference draws a recording above the made sound to one scale of time

yhhb774af8e3432
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c4b257aa6..29bc278e1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,7 +20,7 @@ published separately; it ships inside the editor.
- **A flower turns to face the eye.** A flat flower faces the sky, so a field of them is seen edge on from a few metres up and has no colour a field away; tipping each far plant's head at random (`farTilt`) showed half of them from behind. `Foliage.faceEyeNear` and `Foliage.faceEye` turn the head of every mesh plant toward the eye, by the first close to and by the second from half the draw distance away, as a clump of flowers shows its colour from the side. It is done the way a pivot painted into a uv is elsewhere: the copies carry in a second uv where each vertex of a head sits from its plant's axis, and a material's vertex hook now reads a mesh's second uv as `es_vertexUv1` (zero on a mesh with none), beside `es_vertexColor`. At 0 and 0, the default, no channel is added and nothing turns.
- **A far field of flowers shows its colour.** A mesh plant kept in a far version grows across the ground to stand for those left out. A flat flower grown that way is a disc facing the sky: from an eye a few metres up it is seen edge on, and a meadow that is all flowers at your feet was plain green a field away. `Foliage.farTilt` tips the top of each far plant over by so many degrees (the last tenth of its height: the flower, not its stem), each plant its own way round, so half the field shows its face. 0, the default, leaves a far plant as it was.
- **A layer with a grain is read without repeat and without hexagons.** Near the eye a terrain layer is read in hexagonal cells, each shifted and turned at random, so no tile shows. A layer with a direction to it (sand in ripples, rock in beds) turned cell by cell has its normal map lit cell by cell, and the ground showed a mosaic of light and dark hexagons a metre wide. `u_antiTileTurn` on the Terrain and Terrain Layers materials is how far round a cell may be turned: 1, the default, is any way as before; 0 shifts the cell and does not turn it, which is the setting for a layer with a grain.
-- **An AI can make a sound effect.** The engine played recordings and nothing here made one: a game built by an agent had pictures and models of its own and no sounds. A sound is now layers, as a picture is: `sound_write` takes a source of tones (sine, triangle, saw, square), noise (white, pink, brown) and `modes`, the tones a struck thing rings with, each placed in time and shaped by a level, a decay, filters whose corner can move, a second tone bending it, drive, pan and repeats, with an echo and a room over the whole, and writes an `.essound` and the `.wav` a scene plays. Whoever writes one cannot hear it, so the reply is a picture: the wave on top and pitch against time under it, a thump a blot low on the left, a hiss a haze across the top, a falling tone a falling line, with the level and the share of the energy that is low, middle and high. `sound_edit` changes a number and makes it again; `loop` runs the end into the start for an engine or a wind. A gunshot, footsteps, breaking glass, a pickup, an explosion, an engine loop and a laser are 150 to 500 bytes of source each and are made in under a tenth of a second. The same source is the same file on every machine. The first sounds made this way were listened to and did not pass: one noise through one filter with one decay is heard at once as a machine. So a layer can now be what a real sound is made of: noise cut to a `band` and brought back to full level, several of which die at different rates behind an `impulse`; a `tail` that lets part of a decay linger; `scatter`, which strikes a layer hundreds of times at random in time, pitch, loudness and side, for glass, gravel and rain; `repeat` with `faster`, for a thing bouncing to rest; a `peak` filter for the note a body adds; and `reverb` as a place, `room` or `open`, made by laying the sound over what that place answers to a click, with distinct first echoes and a wash whose top dies before its bottom. The tool text gives the method and the numbers for a shot, glass, a step and an explosion.
+- **An AI can make a sound effect.** The engine played recordings and nothing here made one: a game built by an agent had pictures and models of its own and no sounds. A sound is now layers, as a picture is: `sound_write` takes a source of tones (sine, triangle, saw, square), noise (white, pink, brown) and `modes`, the tones a struck thing rings with, each placed in time and shaped by a level, a decay, filters whose corner can move, a second tone bending it, drive, pan and repeats, with an echo and a room over the whole, and writes an `.essound` and the `.wav` a scene plays. Whoever writes one cannot hear it, so the reply is a picture: the wave on top and pitch against time under it, a thump a blot low on the left, a hiss a haze across the top, a falling tone a falling line, with the level and the share of the energy that is low, middle and high. `sound_edit` changes a number and makes it again; `loop` runs the end into the start for an engine or a wind. A gunshot, footsteps, breaking glass, a pickup, an explosion, an engine loop and a laser are 150 to 500 bytes of source each and are made in under a tenth of a second. The same source is the same file on every machine. The first sounds made this way were listened to and did not pass: one noise through one filter with one decay is heard at once as a machine. So a layer can now be what a real sound is made of: noise cut to a `band` and brought back to full level, several of which die at different rates behind an `impulse`; a `tail` that lets part of a decay linger; `scatter`, which strikes a layer hundreds of times at random in time, pitch, loudness and side, for glass, gravel and rain; `repeat` with `faster`, for a thing bouncing to rest; a `peak` filter for the note a body adds; and `reverb` as a place, `room` or `open`, made by laying the sound over what that place answers to a click, with distinct first echoes and a wash whose top dies before its bottom. The tool text gives the method and the numbers for a shot, glass, a step and an explosion. A `group` is several layers struck together as one event, so four footsteps are one step with `repeat` (787 bytes of source where it was 24 layers), each striking with its own grit. `variants` makes up to eight files that are the same sound and not the same samples, their pitch, loudness, decay and timing off by `vary`, with a sound event beside them that plays one after another without repeating: a game that plays one file for every shot is heard as a machine however good the file. And `reference` names a recording to be like, which is drawn above the made sound in every reply, the same way and to one scale of time, and measured the same way; `sound_read` reads a recording on its own.
- **An AI can draw a surface: a picture that repeats, with what the light does on it.** `image_write` drew one picture, and a picture laid beside itself shows its edges; nor is a wall only its colour. `tile: true` on an image source makes it repeat: a shape, a line or a brush stroke that leaves one edge comes in at the opposite one, a shadow and a blur go round with it, and a noise fill meets itself. `material: { relief, height, roughness, metal }` makes it a surface: each of height, roughness and metal is layers drawn as a grey picture of their own (light is high, rough, metal), or the colour's own lightness stands in for the relief, and a normal map, a map of roughness and metal and an `.esmaterial` are written beside the picture. The reply shows the picture laid beside itself two by two, where a join would run through the middle, and the same under a low light, and gives the fields to put on a mesh. A brick wall, a plank floor and a riveted plate, 256 pixels each, drew in under a third of a second apiece and tiled three across in the engine with no join. A `filter` still stops at the edges, and a noise turned or stretched by `transform` does not come round.
- **A model is written once for all its like parts, and a leaf can be a picture.** A model's source was a flat list of steps with plain numbers: a plant of ten fronds wrote out a frond ten times, and every leaf was a thin solid in one colour. Three things change that. `params` names numbers and points, and a value written `"=length * 0.4"` is a sum over them, with points that add and scale, angles in degrees, and makers for what shapes keep asking for: `polar` for a point round the upright axis, `arc` for a line that bends from one lean to another, `part` for a share of a line. `{ op: "repeat", over: [rows], steps: [...] }` runs its steps once for each row, the row's fields being names inside it, and `let` names a value for the steps after it; what is saved and edited is the source as written, so one edit to a name changes every part. The `strip` step carries a band with no thickness along a path, shaped by `widths`, folded by `cup`, with a picture laid along it from root to tip. `material.texture` names that picture, and with `alpha: "cutout"` its alpha is the outline: a leaf with veins and a torn edge, drawn once. `loft` takes an `up` for the way its section stands. The palm that was 386 steps is six steps written in two models of 2.9 and 2.4 KB, and 9,150 triangles where it was 26,814. Drawn in the engine it cost a little more GPU than the solid one all the same (about 1.1 ms against 0.8 for the frame at 1024 by 1024 on an RTX 3060): cut-out leaves draw both sides and test every pixel.
- **An AI can lay leaflets along a stem in one step.** `scatter` along a `path` pointed every copy the way the path runs, turned at random, so the leaflets of a frond, the barbs of a feather or the legs of a centipede had to be written out one step each: a palm was 386 steps and a 749 KB model file. `rows` (1 or 2) stands the copies off the path instead, in a row or in pairs, `sweep` degrees from the way it runs and `lift` out of the plane, each as one number or as a start and an end. A copy's +z faces `up`, so a flat blade keeps its face to the sky whichever way it points. `sizes` shapes them along the path, widest in the middle if you say so. The same palm is 40 steps and 38 KB, and bakes to the same triangle count. A scatter without `rows` is as it was.
diff --git a/desktop b/desktop
index 0fdd3ac6f..89ddfa988 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 0fdd3ac6fb990dc35960662dda3e67dcbdc993a5
+Subproject commit 89ddfa9889b57ea04c34a0dbaec1b96c5e3c85d8
diff --git a/pipeline/src/sound/soundPicture.ts b/pipeline/src/sound/soundPicture.ts
index 0ce9ab539..eccd67d9a 100644
--- a/pipeline/src/sound/soundPicture.ts
+++ b/pipeline/src/sound/soundPicture.ts
@@ -135,3 +135,25 @@ export function soundPicture(sound: RenderedSound, width = 768): SoundPicture {
for (let x = 0; x < width; x++) put(x, waveH, 90, 96, 110);
return { width, height, rgba };
}
+
+/**
+ * A recording over the sound made to be like it, drawn the same way and to one scale of time: a second is as wide in
+ * the one as in the other, so a front that is too slow, a tail that is too short and a band that is missing all show
+ * as differences in shape. The recording is on top, under a pale bar; the made sound is below, under a blue one.
+ */
+export function comparePicture(reference: RenderedSound, made: RenderedSound, width = 768): SoundPicture {
+ const longest = Math.max(reference.seconds, made.seconds, 1e-6);
+ const drawn = [reference, made].map((s) => soundPicture(s, Math.max(48, Math.round((width * s.seconds) / longest))));
+ const [bar, gap] = [6, 10];
+ const height = drawn[0]!.height + drawn[1]!.height + bar * 2 + gap;
+ const rgba = new Uint8Array(width * height * 4);
+ for (let i = 0; i < width * height; i++) rgba.set([8, 8, 14, 255], i * 4);
+ let top = 0;
+ drawn.forEach((p, k) => {
+ const [r, g, b] = k === 0 ? [236, 226, 196] : [120, 214, 255];
+ for (let y = 0; y < bar; y++) for (let x = 0; x < p.width; x++) rgba.set([r, g, b, 255], ((top + y) * width + x) * 4);
+ for (let y = 0; y < p.height; y++) rgba.set(p.rgba.subarray(y * p.width * 4, (y + 1) * p.width * 4), ((top + bar + y) * width) * 4);
+ top += bar + p.height + gap;
+ });
+ return { width, height, rgba };
+}
diff --git a/pipeline/src/sound/soundRender.ts b/pipeline/src/sound/soundRender.ts
index 692cbd3b6..90391b9e2 100644
--- a/pipeline/src/sound/soundRender.ts
+++ b/pipeline/src/sound/soundRender.ts
@@ -343,8 +343,18 @@ function strikesOf(layer: SoundLayer, start: number, seed: number): Strike[] {
return out;
}
-/** The samples a validated source makes. */
-export function renderSound(source: SoundSource): RenderedSound {
+/** Where and how an event is struck: what a group hands down to the layers in it. */
+interface Placement { offset: number; pitch: number; loud: number; pan: number; seed: number }
+
+/** How a variant of a layer strays from the layer as written. */
+interface Stray { pitch: number; loud: number; decay: number; late: number; seed: number }
+
+/**
+ * The samples a validated source makes. @p variant 0 is the sound as written; another is the same sound a little
+ * unlike it, as a second shot from the same gun is: other noise, and each layer's pitch, loudness, decay and timing
+ * off by up to the source's `vary`.
+ */
+export function renderSound(source: SoundSource, variant = 0): RenderedSound {
const rate = source.rate ?? 44100;
const count = Math.max(1, Math.round(source.duration * rate));
// A loop is rendered a little long and its extra folded back over its start, so the end runs into the beginning.
@@ -355,43 +365,94 @@ export function renderSound(source: SoundSource): RenderedSound {
const reports: RenderedSound['layers'] = [];
let stereo = source.reverb !== undefined;
- source.layers.forEach((layer, index) => {
- if (layer.mute) return;
- const start = layer.start ?? 0;
+ const vary = variant > 0 ? source.vary ?? 0.06 : 0;
+ const strayRandom = noiseOf(variant * 7717 + 13);
+ const strays = new Map<SoundLayer, Stray>();
+ const strayOf = (layer: SoundLayer): Stray => {
+ let s = strays.get(layer);
+ if (!s) {
+ s = vary > 0
+ ? { pitch: 1 + vary * strayRandom(), loud: 1 + vary * 2 * strayRandom(), decay: 1 + vary * 1.5 * strayRandom(), late: 1 + vary * strayRandom(), seed: variant * 1009 }
+ : { pitch: 1, loud: 1, decay: 1, late: 1, seed: 0 };
+ strays.set(layer, s);
+ }
+ return s;
+ };
+
+ /** One layer, or a group of them, struck where @p place says. Answers how loud it was and when it ended. */
+ const play = (layer: SoundLayer, index: number, where: string, place: Placement, top: boolean): { peak: number; last: number; heard: number; strikes: number } => {
+ const stray = strayOf(layer);
+ // A layer that comes after the start of its event comes a little sooner or later in another variant.
+ const start = place.offset + (layer.start ?? 0) * stray.late;
+ const seed = (layer.seed ?? index + 1) + place.seed + stray.seed;
+ if (layer.source === 'group') {
+ const strikes = strikesOf(layer, start, seed);
+ const loud = (typeof layer.gain === 'number' ? layer.gain : 1) * stray.loud;
+ let [peak, last, heard] = [0, start, 0];
+ for (const s of strikes) {
+ if (s.at * rate >= total) continue;
+ heard++;
+ layer.layers!.forEach((child, j) => {
+ if (child.mute) return;
+ const made = play(child, j, `${where} > layer ${j}${child.id ? ` "${child.id}"` : ''}`,
+ { offset: s.at, pitch: place.pitch * s.pitch, loud: place.loud * s.loud * loud, pan: Math.max(-1, Math.min(1, place.pan + s.pan)), seed: s.seed + j * 977 }, false);
+ peak = Math.max(peak, made.peak);
+ last = Math.max(last, made.last);
+ });
+ }
+ return { peak, last, heard, strikes: strikes.length };
+ }
const space = total / rate - start;
const seconds = Math.min(layer.length ?? space, space);
- const where = `layer ${index}${layer.id ? ` "${layer.id}"` : ''}`;
- if (!(seconds > 0)) { notes.push(`${where} starts at the end of the sound and is not heard`); return; }
- if (layer.length !== undefined && layer.length > space + 1e-6 && !layer.repeat && !layer.scatter) notes.push(`${where} is cut short: it runs ${layer.length} s from ${start} s in a sound of ${source.duration} s`);
+ if (!(seconds > 0)) {
+ if (top) notes.push(`${where} starts at the end of the sound and is not heard`);
+ return { peak: 0, last: start, heard: 0, strikes: 0 };
+ }
+ if (top && layer.length !== undefined && layer.length > space + 1e-6 && !layer.repeat && !layer.scatter) notes.push(`${where} is cut short: it runs ${layer.length} s from ${Math.round(start * 1000) / 1000} s in a sound of ${source.duration} s`);
const wide = layer.source === 'noise' ? layer.wide ?? 0 : 0;
- const strikes = strikesOf(layer, start, layer.seed ?? index + 1);
- if (wide > 0 || strikes.some((s) => s.pan !== 0)) stereo = true;
+ // The variant's own layer: the same in everything but how long it rings.
+ const made = stray.decay !== 1 && layer.decay
+ ? { ...layer, decay: layer.decay * stray.decay, ...(layer.tail ? { tail: { ...layer.tail, decay: layer.tail.decay * stray.decay } } : {}) }
+ : layer;
+ const strikes = strikesOf(layer, start, seed);
+ if (wide > 0 || place.pan !== 0 || strikes.some((s) => s.pan !== 0)) stereo = true;
let [peak, last, heard] = [0, start, 0];
for (const s of strikes) {
const from = Math.round(Math.max(0, s.at) * rate);
if (from >= total) continue;
heard++;
const length = Math.min(seconds, (total - from) / rate);
+ const pitch = s.pitch * place.pitch * stray.pitch;
+ const loud = s.loud * place.loud * stray.loud;
// Each striking has its own noise, as no two footsteps are the same grit.
- const hit = strike(layer, length, rate, s.pitch, s.seed);
+ const hit = strike(made, length, rate, pitch, s.seed);
// The other ear's noise is another noise, by as much as the layer is wide.
- const other = wide > 0 ? strike(layer, length, rate, s.pitch, s.seed + 50021) : null;
+ const other = wide > 0 ? strike(made, length, rate, pitch, s.seed + 50021) : null;
// Equal power: a sound sent to one side is as loud as one in the middle.
- const [gl, gr] = [Math.cos(((s.pan + 1) * Math.PI) / 4), Math.sin(((s.pan + 1) * Math.PI) / 4)];
+ const pan = Math.max(-1, Math.min(1, s.pan + place.pan));
+ const [gl, gr] = [Math.cos(((pan + 1) * Math.PI) / 4), Math.sin(((pan + 1) * Math.PI) / 4)];
for (let n = 0; n < hit.length && from + n < total; n++) {
- const v = hit[n]! * s.loud;
+ const v = hit[n]! * loud;
left[from + n]! += v * gl;
- right[from + n]! += (other ? v * (1 - wide) + other[n]! * s.loud * wide : v) * gr;
+ right[from + n]! += (other ? v * (1 - wide) + other[n]! * loud * wide : v) * gr;
if (Math.abs(v) > peak) peak = Math.abs(v);
}
last = Math.max(last, Math.min(total / rate, s.at + hit.length / rate));
}
+ return { peak, last, heard, strikes: strikes.length };
+ };
+
+ source.layers.forEach((layer, index) => {
+ if (layer.mute) return;
+ const where = `layer ${index}${layer.id ? ` "${layer.id}"` : ''}`;
+ const made = play(layer, index, where, { offset: 0, pitch: 1, loud: 1, pan: 0, seed: 0 }, true);
+ const start = layer.start ?? 0;
reports.push({
- ...(layer.id ? { id: layer.id } : {}), source: layer.source, from: Math.round(start * 1000) / 1000, to: Math.round(last * 1000) / 1000,
- peak: Math.round(peak * 1000) / 1000, ...(strikes.length > 1 ? { strikes: heard } : {}),
+ ...(layer.id ? { id: layer.id } : {}), source: layer.source, from: Math.round(start * 1000) / 1000, to: Math.round(made.last * 1000) / 1000,
+ peak: Math.round(made.peak * 1000) / 1000, ...(made.strikes > 1 ? { strikes: made.heard } : {}),
});
- if (heard < strikes.length) notes.push(`${where}: ${strikes.length - heard} of its ${strikes.length} strikes fall past the end of the sound and are not heard`);
- if (peak < 1e-4) notes.push(`${where} makes no sound: its level is nothing throughout`);
+ if (made.heard < made.strikes) notes.push(`${where}: ${made.strikes - made.heard} of its ${made.strikes} strikes fall past the end of the sound and are not heard`);
+ if (made.strikes > 0 && made.peak < 1e-4) notes.push(`${where} makes no sound: its level is nothing throughout`);
});
const channels = stereo ? [left, right] : [left.map((v, i) => (v + right[i]!) * Math.SQRT1_2)];
@@ -474,3 +535,54 @@ export function encodeWav(sound: Pick<RenderedSound, 'rate' | 'channels'>): Uint
}
return bytes;
}
+
+/** Every variant a source asks for, the first being the sound as written. */
+export function renderSoundVariants(source: SoundSource): RenderedSound[] {
+ return Array.from({ length: source.variants ?? 1 }, (_, k) => renderSound(source, k));
+}
+
+/** A recording as this file's sounds are held, so it can be drawn and measured as they are. */
+export function soundFromSamples(rate: number, channels: readonly Float32Array[]): RenderedSound {
+ const kept = channels.slice(0, 2).map((c) => Float32Array.from(c));
+ let [peak, squares] = [0, 0];
+ for (const channel of kept) for (const v of channel) { if (Math.abs(v) > peak) peak = Math.abs(v); squares += v * v; }
+ const frames = kept[0]?.length ?? 0;
+ return { rate, channels: kept, seconds: frames / rate, peak, rms: Math.sqrt(squares / Math.max(1, frames * kept.length)), layers: [], notes: [] };
+}
+
+/** A `.wav` read: PCM of 8, 16, 24 or 32 bits, or 32-bit floats. Throws on anything else, saying what it found. */
+export function decodeWav(bytes: Uint8Array): RenderedSound {
+ const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
+ const tag = (at: number): string => String.fromCharCode(bytes[at]!, bytes[at + 1]!, bytes[at + 2]!, bytes[at + 3]!);
+ if (bytes.length < 44 || tag(0) !== 'RIFF' || tag(8) !== 'WAVE') throw new Error('not a .wav: it does not begin RIFF....WAVE');
+ let [format, channels, rate, bits] = [0, 0, 0, 0];
+ let data: { at: number; size: number } | null = null;
+ for (let at = 12; at + 8 <= bytes.length;) {
+ const size = view.getUint32(at + 4, true);
+ if (tag(at) === 'fmt ') {
+ format = view.getUint16(at + 8, true);
+ channels = view.getUint16(at + 10, true);
+ rate = view.getUint32(at + 12, true);
+ bits = view.getUint16(at + 22, true);
+ // An extensible header says its real format further in.
+ if (format === 0xfffe && size >= 26) format = view.getUint16(at + 32, true);
+ } else if (tag(at) === 'data') data = { at: at + 8, size: Math.min(size, bytes.length - at - 8) };
+ at += 8 + size + (size & 1);
+ }
+ if (!data || !channels || !rate) throw new Error('a .wav with no format or no samples in it');
+ if (!((format === 1 && [8, 16, 24, 32].includes(bits)) || (format === 3 && bits === 32))) throw new Error(`a .wav of a kind not read here (format ${format}, ${bits} bits); PCM of 8 to 32 bits and 32-bit floats are`);
+ const step = bits / 8;
+ const frames = Math.floor(data.size / (step * channels));
+ const out = Array.from({ length: channels }, () => new Float32Array(frames));
+ for (let n = 0; n < frames; n++) {
+ for (let c = 0; c < channels; c++) {
+ const at = data.at + (n * channels + c) * step;
+ out[c]![n] = format === 3 ? view.getFloat32(at, true)
+ : bits === 8 ? (bytes[at]! - 128) / 128
+ : bits === 16 ? view.getInt16(at, true) / 32768
+ : bits === 24 ? ((bytes[at]! | (bytes[at + 1]! << 8) | (view.getInt8(at + 2) << 16))) / 8388608
+ : view.getInt32(at, true) / 2147483648;
+ }
+ }
+ return soundFromSamples(rate, out);
+}
diff --git a/pipeline/src/sound/soundSource.ts b/pipeline/src/sound/soundSource.ts
index 98cfc25a2..17a2da3c4 100644
--- a/pipeline/src/sound/soundSource.ts
+++ b/pipeline/src/sound/soundSource.ts
@@ -20,7 +20,8 @@ export const SOUND_RATES = [22050, 44100, 48000] as const;
/** One value for the whole of a layer, or [[seconds, value], ...] from the layer's start: a pitch that falls, a filter that opens. */
export type SoundCurve = number | readonly (readonly [number, number])[];
-export const SOUND_SOURCES = ['sine', 'triangle', 'saw', 'square', 'noise', 'modes', 'impulse'] as const;
+export const SOUND_SOURCES = ['sine', 'triangle', 'saw', 'square', 'noise', 'modes', 'impulse', 'group'] as const;
+export const MAX_SOUND_VARIANTS = 8;
export type SoundSourceKind = (typeof SOUND_SOURCES)[number];
export const SOUND_FILTERS = ['lowpass', 'highpass', 'bandpass', 'notch', 'peak'] as const;
export const REVERB_KINDS = ['room', 'open'] as const;
@@ -39,6 +40,13 @@ export interface SoundFilter {
export interface SoundLayer {
id?: string;
source: SoundSourceKind;
+ /**
+ * group: the layers that make one event, struck together. A footstep is a thud, a scrape and two clicks, and four
+ * footsteps are that one group with `repeat`, each striking with its own grit, in place of the layers written out
+ * four times. The group's `start`, `gain`, `pan`, `repeat` and `scatter` are the event's; each layer in it keeps its
+ * own start, counted from the event's.
+ */
+ layers?: readonly SoundLayer[];
/** A tone's pitch in Hz; a curve glides between its points as the ear hears pitch. */
freq?: SoundCurve;
/** square: the share of each cycle it is high, 0.05..0.95. Default 0.5. impulse: seconds its one spike lasts, 0.00002..0.02; a shot's is 0.0002. */
@@ -101,6 +109,16 @@ export interface SoundSource {
rate?: number;
/** Its end runs into its start with no click, for an engine, a wind, a hum. */
loop?: boolean;
+ /**
+ * How many of it to make, 1..8, each a little unlike the others: other noise, and pitch, loudness, decay and timing
+ * off by up to `vary`. A game that plays the same file for every shot or step is heard as a machine however good the
+ * file; with variants a sound event is written beside them that plays one after another without repeating.
+ */
+ variants?: number;
+ /** How far a variant strays, 0..0.3. Default 0.06. */
+ vary?: number;
+ /** A recording to make it like, by project path. It is shown above the sound in every reply, drawn the same way to the same scale, and measured the same way. Nothing is taken from it. */
+ reference?: string;
layers: readonly SoundLayer[];
/** Over the whole sound, after the layers are summed: */
highpass?: number;
@@ -138,17 +156,25 @@ function curveProblem(v: unknown, lo: number, hi: number, what: string): string
const LAYER_FIELDS = ['id', 'source', 'freq', 'width', 'band', 'wide', 'color', 'modes', 'seed', 'start', 'length', 'gain', 'attack', 'release', 'decay', 'tail', 'vibrato', 'tremolo', 'fm', 'filters', 'drive', 'pan', 'repeat', 'scatter', 'mute'];
-function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>, duration: number): void {
+const GROUP_FIELDS = ['id', 'source', 'layers', 'start', 'gain', 'pan', 'repeat', 'scatter', 'mute'];
+
+function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>, duration: number, depth = 0): void {
const fail = (what: string): never => { throw new Error(`${where}: ${what}`); };
if (typeof l !== 'object' || l === null || Array.isArray(l)) fail('must be an object');
- for (const key of Object.keys(l)) if (!LAYER_FIELDS.includes(key)) fail(`unknown field "${key}"; a layer takes ${LAYER_FIELDS.join(', ')}`);
+ if (l.source === 'group') {
+ for (const key of Object.keys(l)) if (!GROUP_FIELDS.includes(key)) fail(`a group takes ${GROUP_FIELDS.join(', ')}; "${key}" belongs on the layers in it`);
+ if (depth >= 3) fail('groups go three deep and no further');
+ if (!Array.isArray(l.layers) || l.layers.length === 0) fail('a group needs `layers`: one or more');
+ if (l.gain !== undefined && !between(l.gain, 0, 4)) fail('a group\'s gain is one number, 0..4');
+ } else if (l.layers !== undefined) fail('`layers` goes with source "group"');
+ if (l.source !== 'group') for (const key of Object.keys(l)) if (!LAYER_FIELDS.includes(key)) fail(`unknown field "${key}"; a layer takes ${LAYER_FIELDS.join(', ')}`);
if (l.id !== undefined) {
if (typeof l.id !== 'string' || !l.id) fail('id must be a name');
if (ids.has(l.id as string)) fail(`the id "${String(l.id)}" is already used`);
ids.add(l.id as string);
}
if (!SOUND_SOURCES.includes(l.source as SoundSourceKind)) fail(`source ${JSON.stringify(l.source)} must be one of ${SOUND_SOURCES.join(', ')}`);
- const tone = l.source !== 'noise' && l.source !== 'modes' && l.source !== 'impulse';
+ const tone = l.source !== 'noise' && l.source !== 'modes' && l.source !== 'impulse' && l.source !== 'group';
if (tone) {
if (l.freq === undefined) fail(`a ${String(l.source)} needs freq, its pitch in Hz`);
const bad = curveProblem(l.freq, 10, 20000, 'freq');
@@ -181,7 +207,7 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>,
if (l.seed !== undefined && !Number.isInteger(l.seed)) fail('seed must be a whole number');
if (l.start !== undefined && !between(l.start, 0, duration)) fail(`start must be 0..${duration}, inside the sound`);
if (l.length !== undefined && !(isNum(l.length) && l.length > 0)) fail('length must be above 0');
- if (l.gain !== undefined) { const bad = curveProblem(l.gain, 0, 4, 'gain'); if (bad) fail(bad); }
+ if (l.gain !== undefined && l.source !== 'group') { const bad = curveProblem(l.gain, 0, 4, 'gain'); if (bad) fail(bad); }
for (const key of ['attack', 'release']) if (l[key] !== undefined && !between(l[key], 0, MAX_SOUND_SECONDS)) fail(`${key} must be seconds, 0 or more`);
if (l.decay !== undefined && !(isNum(l.decay) && l.decay > 0)) fail('decay must be seconds above 0');
const wobble = (key: string, deepest: number): void => {
@@ -241,9 +267,10 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>,
for (const key of ['pan', 'fade']) if (c![key] !== undefined && !between(c![key], 0, 1)) fail(`scatter.${key} must be 0..1`);
}
if (l.mute !== undefined && typeof l.mute !== 'boolean') fail('mute must be true or false');
+ if (l.source === 'group') (l.layers as Record<string, unknown>[]).forEach((c, i) => checkLayer(c, `${where} > layer ${i}${c?.id ? ` "${String(c.id)}"` : ''}`, ids, duration, depth + 1));
}
-const SOURCE_FIELDS = ['version', 'duration', 'rate', 'loop', 'layers', 'highpass', 'lowpass', 'echo', 'reverb', 'gain', 'normalize'];
+const SOURCE_FIELDS = ['version', 'duration', 'rate', 'loop', 'variants', 'vary', 'reference', 'layers', 'highpass', 'lowpass', 'echo', 'reverb', 'gain', 'normalize'];
/** The source if it is well formed; otherwise throws, naming the layer and the field. */
export function validateSoundSource(raw: unknown): SoundSource {
@@ -254,6 +281,9 @@ export function validateSoundSource(raw: unknown): SoundSource {
if (!(isNum(s.duration) && s.duration > 0 && s.duration <= MAX_SOUND_SECONDS)) throw new Error(`duration must be seconds, above 0 and up to ${MAX_SOUND_SECONDS}`);
if (s.rate !== undefined && !SOUND_RATES.includes(s.rate as never)) throw new Error(`rate must be one of ${SOUND_RATES.join(', ')}`);
if (s.loop !== undefined && typeof s.loop !== 'boolean') throw new Error('loop must be true or false');
+ if (s.variants !== undefined && !(Number.isInteger(s.variants) && (s.variants as number) >= 1 && (s.variants as number) <= MAX_SOUND_VARIANTS)) throw new Error(`variants must be a whole number, 1..${MAX_SOUND_VARIANTS}`);
+ if (s.vary !== undefined && !between(s.vary, 0, 0.3)) throw new Error('vary must be 0..0.3');
+ if (s.reference !== undefined && !(typeof s.reference === 'string' && /\.(wav|ogg|mp3|flac|m4a|aac|webm)$/i.test(s.reference))) throw new Error('reference must be a project path to a recording (.wav, .ogg, .mp3, .flac, .m4a)');
if (s.normalize !== undefined && typeof s.normalize !== 'boolean') throw new Error('normalize must be true or false');
if (s.gain !== undefined && !between(s.gain, 0, 1)) throw new Error('gain must be 0..1');
for (const key of ['highpass', 'lowpass']) if (s[key] !== undefined && !between(s[key], 20, 20000)) throw new Error(`${key} must be Hz, 20..20000`);
@@ -280,24 +310,35 @@ export function validateSoundSource(raw: unknown): SoundSource {
export type SoundEdit =
| { op: 'set'; layer: string | number; fields: Record<string, unknown> }
- | { op: 'insert'; layer: SoundLayer; before?: string | number; after?: string | number }
+ | { op: 'insert'; layer: SoundLayer; before?: string | number; after?: string | number; into?: string }
| { op: 'remove'; layer: string | number }
| { op: 'settings'; fields: Partial<Omit<SoundSource, 'layers' | 'version'>> };
/** The source with @p edits applied, in order. A `null` field removes it. Refuses without changing anything. */
export function applySoundEdits(source: SoundSource, edits: readonly SoundEdit[]): SoundSource {
const next = JSON.parse(JSON.stringify(source)) as SoundSource & { layers: (SoundLayer & Record<string, unknown>)[] };
- const find = (ref: string | number): number => {
- const i = typeof ref === 'number' ? ref : next.layers.findIndex((l) => l.id === ref);
- if (!Number.isInteger(i) || i < 0 || i >= next.layers.length) {
- throw new Error(`no layer ${JSON.stringify(ref)}; ${next.layers.length} layers, ids: ${next.layers.map((l) => l.id).filter(Boolean).join(', ') || '(none)'}`);
- }
- return i;
+ type Row = SoundLayer & Record<string, unknown>;
+ const allIds = (list: readonly Row[]): string[] => list.flatMap((l) => [...(l.id ? [l.id] : []), ...allIds((l.layers ?? []) as Row[])]);
+ /** The list a layer is in and its place there: by index among the top layers, or by id anywhere, in a group too. */
+ const locate = (ref: string | number): { list: Row[]; index: number } => {
+ const within = (list: Row[]): { list: Row[]; index: number } | null => {
+ for (let i = 0; i < list.length; i++) {
+ if (list[i]!.id === ref) return { list, index: i };
+ const hit = list[i]!.layers ? within(list[i]!.layers as Row[]) : null;
+ if (hit) return hit;
+ }
+ return null;
+ };
+ const top = next.layers as Row[];
+ const hit = typeof ref === 'number' ? (Number.isInteger(ref) && ref >= 0 && ref < top.length ? { list: top, index: ref } : null) : within(top);
+ if (!hit) throw new Error(`no layer ${JSON.stringify(ref)}; ${top.length} layers, ids: ${allIds(top).join(', ') || '(none)'}`);
+ return hit;
};
for (const edit of edits) {
switch (edit?.op) {
case 'set': {
- const layer = next.layers[find(edit.layer)]!;
+ const at = locate(edit.layer);
+ const layer = at.list[at.index]!;
for (const [key, value] of Object.entries(edit.fields ?? {})) {
if (value === null) delete layer[key];
else layer[key] = value;
@@ -305,13 +346,23 @@ export function applySoundEdits(source: SoundSource, edits: readonly SoundEdit[]
break;
}
case 'insert': {
- const at = edit.before !== undefined ? find(edit.before) : edit.after !== undefined ? find(edit.after) + 1 : next.layers.length;
- next.layers.splice(at, 0, edit.layer as never);
+ // Beside the layer named, in whichever list that is; or, with `into`, at the end of a group.
+ if (edit.into !== undefined) {
+ const group = locate(edit.into);
+ const row = group.list[group.index]!;
+ if (row.source !== 'group') throw new Error(`${JSON.stringify(edit.into)} is not a group`);
+ (row.layers as Row[]).push(edit.layer as Row);
+ } else if (edit.before !== undefined || edit.after !== undefined) {
+ const at = locate((edit.before ?? edit.after)!);
+ at.list.splice(at.index + (edit.before !== undefined ? 0 : 1), 0, edit.layer as Row);
+ } else (next.layers as Row[]).push(edit.layer as Row);
break;
}
- case 'remove':
- next.layers.splice(find(edit.layer), 1);
+ case 'remove': {
+ const at = locate(edit.layer);
+ at.list.splice(at.index, 1);
break;
+ }
case 'settings':
for (const [key, value] of Object.entries(edit.fields ?? {})) {
if (key === 'layers' || key === 'version') throw new Error(`settings does not take ${key}; use set, insert and remove for layers`);
diff --git a/pipeline/tests/sound.test.ts b/pipeline/tests/sound.test.ts
index 886acca30..812130b00 100644
--- a/pipeline/tests/sound.test.ts
+++ b/pipeline/tests/sound.test.ts
@@ -6,8 +6,8 @@
*/
import { describe, expect, it } from 'vitest';
import { applySoundEdits, validateSoundSource, type SoundLayer, type SoundSource } from '../src/sound/soundSource';
-import { encodeWav, renderSound } from '../src/sound/soundRender';
-import { soundBands, soundPicture } from '../src/sound/soundPicture';
+import { decodeWav, encodeWav, renderSound, renderSoundVariants } from '../src/sound/soundRender';
+import { comparePicture, soundBands, soundPicture } from '../src/sound/soundPicture';
const make = (layers: SoundLayer[], rest: Partial<SoundSource> = {}) => renderSound(validateSoundSource({ duration: 0.5, layers, ...rest }));
/** Times a second the wave crosses nothing going up, between two shares of the sound: its pitch, for a plain tone. */
@@ -314,3 +314,138 @@ describe('what makes a made sound pass for a real one', () => {
expect(bad({ source: 'noise' }, { reverb: { kind: 'cave' } })).toThrow(/reverb\.kind must be one of room, open/);
});
});
+
+describe('layers struck together, sounds that differ, and a recording to be like', () => {
+ const raw = (layers: SoundLayer[], rest: Partial<SoundSource> = {}) => make(layers, { normalize: false, ...rest });
+ // A knock: a spike, and a ring that starts a hundredth of a second after it.
+ const knock: SoundLayer[] = [
+ { source: 'impulse', width: 0.0002, length: 0.002 },
+ { source: 'sine', freq: 1000, start: 0.01, length: 0.03, decay: 0.01 },
+ ];
+
+ it('is the same sound as its layers written out, when a group is struck once', () => {
+ const flat = raw(knock.map((l) => ({ ...l, start: (l.start ?? 0) + 0.1 }))).channels[0]!;
+ const grouped = raw([{ source: 'group', start: 0.1, layers: knock }]).channels[0]!;
+ let worst = 0;
+ for (let n = 0; n < flat.length; n++) worst = Math.max(worst, Math.abs(flat[n]! - grouped[n]!));
+ expect(worst).toBeLessThan(1e-6);
+ });
+
+ it('strikes the whole group again, each of its layers keeping its place in the event', () => {
+ const s = raw([{ id: 'knocks', source: 'group', layers: knock, repeat: { count: 3, every: 0.15, gain: 0.5 } }]);
+ const ring = (k: number): number => level(s.channels[0]!, (k * 0.15 + 0.012) / 0.5, (k * 0.15 + 0.03) / 0.5);
+ expect(ring(0)).toBeGreaterThan(0.05);
+ expect(ring(1)).toBeCloseTo(ring(0) * 0.5, 1);
+ expect(ring(2)).toBeCloseTo(ring(0) * 0.25, 1);
+ // Between one knock and the next there is nothing.
+ expect(level(s.channels[0]!, 0.1 / 0.5, 0.14 / 0.5)).toBeLessThan(1e-3);
+ expect(s.layers).toEqual([{ id: 'knocks', source: 'group', from: 0, to: expect.any(Number), peak: 1, strikes: 3 }]);
+ });
+
+ it('gives each striking of a group its own noise', () => {
+ const s = raw([{ source: 'group', layers: [{ source: 'noise', length: 0.05 }], repeat: { count: 2, every: 0.2 } }]).channels[0]!;
+ const [first, second] = [s.subarray(0, 2000), s.subarray(8820, 10820)];
+ let same = 0;
+ for (let n = 100; n < 1900; n++) if (Math.abs(first[n]! - second[n]!) < 1e-6) same++;
+ expect(same).toBeLessThan(20);
+ });
+
+ it('scatters a group, and hands its pitch, loudness and side down to its layers', () => {
+ const s = raw([{ source: 'group', gain: 0.5, pan: -1, layers: [{ source: 'sine', freq: 500, length: 0.02 }], scatter: { count: 12, over: 0.4, pitch: [2, 2] } }]);
+ expect(s.channels).toHaveLength(2);
+ expect(level(s.channels[1]!)).toBeLessThan(1e-6);
+ expect(s.layers[0]!.strikes).toBe(12);
+ expect(s.layers[0]!.peak).toBeLessThanOrEqual(0.5);
+ expect(soundBands(s).mid).toBeGreaterThan(0.9);
+ });
+
+ it('finds a layer inside a group by its id, to change it, add beside it or put one in', () => {
+ const source = validateSoundSource({ duration: 1, layers: [{ id: 'step', source: 'group', layers: [{ id: 'heel', source: 'noise', length: 0.01 }] }] });
+ const edited = applySoundEdits(source, [
+ { op: 'set', layer: 'heel', fields: { band: [3800, 16000] } },
+ { op: 'insert', layer: { id: 'toe', source: 'noise', length: 0.01, start: 0.008 }, after: 'heel' },
+ { op: 'insert', layer: { id: 'thud', source: 'noise', color: 'brown', length: 0.05 }, into: 'step' },
+ ]);
+ expect(edited.layers[0]!.layers!.map((l) => l.id)).toEqual(['heel', 'toe', 'thud']);
+ expect(edited.layers[0]!.layers![0]!.band).toEqual([3800, 16000]);
+ expect(applySoundEdits(edited, [{ op: 'remove', layer: 'toe' }]).layers[0]!.layers).toHaveLength(2);
+ expect(() => applySoundEdits(source, [{ op: 'insert', layer: { source: 'noise' }, into: 'heel' }])).toThrow(/"heel" is not a group/);
+ expect(() => applySoundEdits(source, [{ op: 'set', layer: 'ankle', fields: {} }])).toThrow(/no layer "ankle"; 1 layers, ids: step, heel/);
+ });
+
+ it('refuses what a group cannot be', () => {
+ const bad = (layer: unknown) => () => validateSoundSource({ duration: 1, layers: [layer] });
+ expect(bad({ source: 'group' })).toThrow(/a group needs `layers`/);
+ expect(bad({ source: 'group', layers: [{ source: 'noise' }], decay: 0.1 })).toThrow(/"decay" belongs on the layers in it/);
+ expect(bad({ source: 'noise', layers: [] })).toThrow(/`layers` goes with source "group"/);
+ expect(bad({ source: 'group', layers: [{ source: 'sine' }] })).toThrow(/layer 0 > layer 0: a sine needs freq/);
+ expect(bad({ source: 'group', id: 'a', layers: [{ id: 'a', source: 'noise' }] })).toThrow(/the id "a" is already used/);
+ });
+
+ it('makes variants that are the same sound and not the same samples', () => {
+ const source = validateSoundSource({
+ duration: 0.5, variants: 4,
+ layers: [{ source: 'noise', band: [800, 2500], decay: 0.03 }, { source: 'sine', freq: 400, decay: 0.05 }, { source: 'sine', freq: 1200, start: 0.2, length: 0.05 }],
+ });
+ const all = renderSoundVariants(source);
+ expect(all).toHaveLength(4);
+ // The first is the sound as written, the same as asking for no variants at all.
+ expect(Buffer.from(all[0]!.channels[0]!.buffer).equals(Buffer.from(renderSound({ ...source, variants: 1 }).channels[0]!.buffer))).toBe(true);
+ for (let k = 1; k < 4; k++) {
+ expect(Buffer.from(all[k]!.channels[0]!.buffer).equals(Buffer.from(all[0]!.channels[0]!.buffer))).toBe(false);
+ // Alike in what can be measured: as long, as loud at its loudest, its energy in the same place.
+ expect(all[k]!.seconds).toBe(all[0]!.seconds);
+ expect(all[k]!.peak).toBeCloseTo(all[0]!.peak, 5);
+ expect(Math.abs(soundBands(all[k]!).mid - soundBands(all[0]!).mid)).toBeLessThan(0.2);
+ }
+ expect(Buffer.from(all[2]!.channels[0]!.buffer).equals(Buffer.from(renderSoundVariants(source)[2]!.channels[0]!.buffer))).toBe(true);
+ expect(Buffer.from(all[1]!.channels[0]!.buffer).equals(Buffer.from(all[2]!.channels[0]!.buffer))).toBe(false);
+ });
+
+ it('strays no further than `vary` says', () => {
+ const pitches = (vary: number): number[] => renderSoundVariants(validateSoundSource({ duration: 0.5, variants: 6, vary, layers: [{ source: 'sine', freq: 1000 }] }))
+ .map((s) => pitchOf(s.channels[0]!, s.rate, 0.1, 0.9));
+ const spread = (v: number[]): number => Math.max(...v) - Math.min(...v);
+ expect(spread(pitches(0))).toBeLessThan(3);
+ expect(spread(pitches(0.2))).toBeGreaterThan(spread(pitches(0.03)) * 2);
+ for (const p of pitches(0.03)) expect(Math.abs(p - 1000)).toBeLessThan(35);
+ expect(() => validateSoundSource({ duration: 1, variants: 9, layers: [{ source: 'noise' }] })).toThrow(/variants must be a whole number, 1\.\.8/);
+ expect(() => validateSoundSource({ duration: 1, vary: 0.5, layers: [{ source: 'noise' }] })).toThrow(/vary must be 0\.\.0\.3/);
+ });
+
+ it('reads back the file it writes, and other kinds of .wav', () => {
+ const s = make([{ source: 'sine', freq: 300, pan: 0.5 }], { rate: 22050 });
+ const back = decodeWav(encodeWav(s));
+ expect(back.rate).toBe(22050);
+ expect(back.channels).toHaveLength(2);
+ expect(back.seconds).toBeCloseTo(0.5, 3);
+ let worst = 0;
+ for (let n = 0; n < s.channels[0]!.length; n++) worst = Math.max(worst, Math.abs(back.channels[0]![n]! - s.channels[0]![n]!));
+ expect(worst).toBeLessThan(1 / 16000);
+ // Floats, with a chunk before the format that a reader has to step over.
+ const floats = new Uint8Array(12 + 12 + 24 + 8 + 16);
+ const view = new DataView(floats.buffer);
+ const text = (at: number, t: string): void => { for (let i = 0; i < t.length; i++) floats[at + i] = t.charCodeAt(i); };
+ text(0, 'RIFF'); view.setUint32(4, floats.length - 8, true); text(8, 'WAVE');
+ text(12, 'LIST'); view.setUint32(16, 4, true);
+ text(24, 'fmt '); view.setUint32(28, 16, true); view.setUint16(32, 3, true); view.setUint16(34, 1, true); view.setUint32(36, 48000, true); view.setUint16(46, 32, true);
+ text(48, 'data'); view.setUint32(52, 16, true);
+ [0.25, -0.5, 1, 0].forEach((v, i) => view.setFloat32(56 + i * 4, v, true));
+ expect([...decodeWav(floats).channels[0]!]).toEqual([0.25, -0.5, 1, 0]);
+ expect(() => decodeWav(new Uint8Array(64))).toThrow(/not a \.wav/);
+ });
+
+ it('draws a recording over the made sound, a second as wide in the one as in the other', () => {
+ const long = make([{ source: 'sine', freq: 300 }], { duration: 1 });
+ const short = make([{ source: 'sine', freq: 3000 }], { duration: 0.25 });
+ const p = comparePicture(long, short, 400);
+ expect(p.width).toBe(400);
+ expect(p.height).toBe(384 * 2 + 22);
+ const lit = (x: number, y: number): boolean => p.rgba[(y * p.width + x) * 4]! + p.rgba[(y * p.width + x) * 4 + 1]! + p.rgba[(y * p.width + x) * 4 + 2]! > 60;
+ // The pale bar runs the whole width over the long one; the blue bar a quarter of it over the short one.
+ expect(lit(390, 2)).toBe(true);
+ expect(lit(90, 384 + 6 + 10 + 2)).toBe(true);
+ expect(lit(110, 384 + 6 + 10 + 2)).toBe(false);
+ expect(lit(300, 384 + 6 + 10 + 60)).toBe(false);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 6c4b3708e..4e6abfa02 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -4594,6 +4594,10 @@ export const NOTED = {
{"note": "An AI can make a sound effect."},
"feat(sound): sound_write teaches the method - bands of noise behind an impulse, strikes scattered by the hundred, two clicks to a step, and what a real bang looks like in the picture":
{"note": "An AI can make a sound effect."},
+ "feat(sound): layers struck together, sounds that differ, and a recording to be like - a group is several layers as one event to repeat or scatter, variants makes up to eight files that are the same sound and not the same samples, and reference draws a recording above the made sound to one scale of time":
+ {"note": "An AI can make a sound effect."},
+ "feat(sound): variants write a sound event, a reference is drawn above the sound, and sound_read reads a recording - the model tools say groups, variants and references":
+ {"note": "An AI can make a sound effect."},
"feat(sky): a sky of your own shades the ground under its clouds - defineSkyEffect takes groundShadow and a second function, skyShadow(world, sun), what is left of the light at a world position, and the sky pass multiplies every surface it finds by it, as Sky Atmosphere does with its own clouds":
{"note": "A sky of your own shades the ground under its clouds."},
"fix(terrain): a colour map and an occlusion map lie the way the heightmap does - Terrain.colorMap, Terrain.occlusionMap and Foliage.colorMap were sampled with their rows turned over, so a picture painted to match a heightmap, a splatmap or a density map came out mirrored north to south under them; an image over an area now has its first row along the least z whoever reads it":