Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(audio): a sound anything authored can start

esenginead8643e73f82
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b319cbb7d..7659a5892 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -74,8 +74,38 @@ published separately; it ships inside the editor.
player moves, releases it and they stop, dodges and the score rises, chases a
block and the run ends, presses Space and it begins again on a clear board.
+- **A sound anything authored can start.** An `AudioSource` could only ever be
+ played by `playOnAwake`. Once the entity existed, nothing an author can reach
+ — a state machine's hook, a behaviour tree's leaf, an event wire, a graph
+ node, the inspector — could make it sound again, and the engine's own answer
+ to "play a sound" was a system holding `Res(Audio)`, which is code.
+
+ It now carries the flag contract `TimelinePlayer` and `SpriteAnimator` already
+ keep: raise `playing` and the clip sounds, lower it and the voice stops, and
+ the engine lowers it again when a non-looping clip ends — so the field reads
+ as "is this sounding" rather than "was it ever asked to". `finished` latches
+ beside it, which is what a state lasting exactly as long as its sound needs.
+ `playOnAwake` is now one trigger expressed through that same switch rather
+ than a second road into the backend.
+
+ `audio.play`, `audio.stop` and `audio.finished` are that flag as vocabulary,
+ registered the way every other verb is, so one name reaches all four authored
+ surfaces at once. `audio.play` takes the clip as a DECLARED parameter, which
+ is what gives it an input pin in a graph instead of only a hand-typed
+ argument — and `spriteAnim.play` and `spriteAnim.restart` declare theirs now
+ too, so the three play verbs are one shape. Naming a different clip while a
+ voice is in flight is a second sound, not a no-op: the voice is a play OF a
+ clip, so the old one stops and the new one starts, which is what a character
+ with a hurt clip and a death clip is.
+
### Fixed
+- **A sound stops when whatever owned it goes away.** Switching an AudioSource
+ off, clearing its clip, or removing the component dropped the handle to the
+ voice WITHOUT stopping it. The sound kept playing with nothing anywhere able
+ to reach it again — a looping one until the session ended. The pass that
+ notices a source left now stops its voice rather than forgetting it.
+
- **Run works from wherever you are standing.** The running game's frame is
parented by whichever panel hosts it — a Game tab, or the Viewport itself in
play-in-viewport mode. Opening any asset document puts it in the tab over the
diff --git a/desktop b/desktop
index caed5b386..0a12d1587 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit caed5b3860b84d43b69dec0671603734a5eec868
+Subproject commit 0a12d1587a0ec3621b20f100a1fa94077e7d7752
diff --git a/docs/astro/src/data/components.generated.json b/docs/astro/src/data/components.generated.json
index 20002bacb..4ccacfc51 100644
--- a/docs/astro/src/data/components.generated.json
+++ b/docs/astro/src/data/components.generated.json
@@ -122,6 +122,19 @@
"type": "boolean",
"default": "false"
},
+ {
+ "key": "playing",
+ "type": "boolean",
+ "default": "false",
+ "doc": "Raise to play, lower to stop. The engine lowers it when a non-looping clip ends."
+ },
+ {
+ "key": "finished",
+ "type": "boolean",
+ "default": "false",
+ "doc": "Clip completed (runtime, read-only). Raise Playing to play it again.",
+ "advanced": true
+ },
{
"key": "playOnAwake",
"type": "boolean",
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index a3711e333..f4648667d 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -1298,11 +1298,13 @@ attenuationModel: number
bus: string
clip: string
enabled: boolean
+finished: boolean
loop: boolean
maxDistance: number
minDistance: number
pitch: number
playOnAwake: boolean
+playing: boolean
priority: number
rolloff: number
spatial: boolean
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 43fbaf30a..1e8173363 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -1298,11 +1298,13 @@ attenuationModel: number
bus: string
clip: string
enabled: boolean
+finished: boolean
loop: boolean
maxDistance: number
minDistance: number
pitch: number
playOnAwake: boolean
+playing: boolean
priority: number
rolloff: number
spatial: boolean
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index f5799da99..a74ec0fd4 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -1293,11 +1293,13 @@ attenuationModel: number
bus: string
clip: string
enabled: boolean
+finished: boolean
loop: boolean
maxDistance: number
minDistance: number
pitch: number
playOnAwake: boolean
+playing: boolean
priority: number
rolloff: number
spatial: boolean
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index 9a686af98..d2564eef1 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -1293,11 +1293,13 @@ attenuationModel: number
bus: string
clip: string
enabled: boolean
+finished: boolean
loop: boolean
maxDistance: number
minDistance: number
pitch: number
playOnAwake: boolean
+playing: boolean
priority: number
rolloff: number
spatial: boolean
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index 42d793ed3..4a774e03a 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -1293,11 +1293,13 @@ attenuationModel: number
bus: string
clip: string
enabled: boolean
+finished: boolean
loop: boolean
maxDistance: number
minDistance: number
pitch: number
playOnAwake: boolean
+playing: boolean
priority: number
rolloff: number
spatial: boolean
diff --git a/sdk/src/ai/builtins.ts b/sdk/src/ai/builtins.ts
index 422642dc4..0fc18d58b 100644
--- a/sdk/src/ai/builtins.ts
+++ b/sdk/src/ai/builtins.ts
@@ -14,14 +14,19 @@
*/
import { aiRegistry, type AiContext } from './fsm/AiContext';
-import type { AiAction, AiActionInput, AiCondition, AiParamValue, AiTouches } from './fsm/registry';
+import type { AiAction, AiActionInput, AiCondition, AiParamDef, AiParamValue, AiTouches } from './fsm/registry';
import { TimelinePlayer } from '../timeline/TimelinePlayerComponent';
import { SpriteAnimator } from '../animation/SpriteAnimator';
+import { AudioSource } from '../audio/AudioComponents';
import { setEntityProperty } from '../ecs/propertyPath';
+import type { AnyComponentDef, ComponentData } from '../ecs/component';
import type { Entity } from '../types';
const TIMELINE: AiTouches = { reads: [TimelinePlayer._name], writes: [TimelinePlayer._name] };
const SPRITE_ANIM: AiTouches = { reads: [SpriteAnimator._name], writes: [SpriteAnimator._name] };
+const AUDIO: AiTouches = { reads: [AudioSource._name], writes: [AudioSource._name] };
+/** The one parameter the play verbs take: which clip, empty for the current one. */
+const CLIP_PARAM: readonly AiParamDef[] = [{ name: 'clip', type: 'string', tooltip: 'Leave empty to replay the current one' }];
/**
* Register the engine's built-in actions/conditions. Idempotent (and safe after
@@ -57,16 +62,14 @@ export function ensureBuiltinAiRegistrations(): void {
return player.finished && !player.playing;
}, { reads: [TimelinePlayer._name] });
- // `arg` is the clip to play (a `.esanim` ref/path); without it the action
- // resumes/replays the animator's current clip. Same-clip play while already
- // playing is a no-op, so `onUpdate: spriteAnim.play` is safe.
- action('spriteAnim.play', (ctx, _bb, arg) => {
- if (!ctx.has(SpriteAnimator)) return;
- const sp = ctx.get(SpriteAnimator);
- const switching = !!arg && arg !== sp.clip;
- if (!switching && sp.playing) return;
+ // `clip` is a `.esanim` ref/path; without it the action resumes/replays the
+ // animator's current clip. Same-clip play while already playing is a no-op,
+ // so `onUpdate: spriteAnim.play` is safe.
+ playVerb('spriteAnim.play', SpriteAnimator, SPRITE_ANIM, (sp, clip) => {
+ const switching = !!clip && clip !== sp.clip;
+ if (!switching && sp.playing) return false;
if (switching) {
- sp.clip = arg!;
+ sp.clip = clip;
sp.currentFrame = 0;
sp.frameTimer = 0;
sp.finished = false;
@@ -74,21 +77,19 @@ export function ensureBuiltinAiRegistrations(): void {
// Raising the flag on a finished one-shot replays it from the top —
// the SpriteAnimator flag contract (mirrors TimelinePlayer's).
sp.playing = true;
- ctx.set(SpriteAnimator, sp);
- }, SPRITE_ANIM);
+ return true;
+ });
// Unconditional rewind + play (re-trigger a one-shot mid-flight), with the
- // same optional clip arg.
- action('spriteAnim.restart', (ctx, _bb, arg) => {
- if (!ctx.has(SpriteAnimator)) return;
- const sp = ctx.get(SpriteAnimator);
- if (arg) sp.clip = arg;
+ // same optional clip.
+ playVerb('spriteAnim.restart', SpriteAnimator, SPRITE_ANIM, (sp, clip) => {
+ if (clip) sp.clip = clip;
sp.currentFrame = 0;
sp.frameTimer = 0;
sp.finished = false;
sp.playing = true;
- ctx.set(SpriteAnimator, sp);
- }, SPRITE_ANIM);
+ return true;
+ });
action('spriteAnim.stop', ctx => {
if (!ctx.has(SpriteAnimator)) return;
@@ -108,6 +109,40 @@ export function ensureBuiltinAiRegistrations(): void {
return sp.finished && !sp.playing;
}, { reads: [SpriteAnimator._name] });
+ // — Sound. An AudioSource is played by raising its flag, the same contract
+ // the timeline and the sprite animator keep, so every authored surface
+ // starts a sound through the component it can already see. —
+ playVerb('audio.play', AudioSource, AUDIO, (source, clip) => {
+ const switching = !!clip && clip !== source.clip;
+ if (switching) {
+ source.clip = clip;
+ source.finished = false;
+ }
+ // Already sounding THIS clip: raising a raised flag is not a replay, and
+ // `onUpdate: audio.play` must not machine-gun the voice. Naming ANOTHER
+ // clip is a second sound, so it goes through even mid-voice.
+ if (!switching && source.playing) return false;
+ source.playing = true;
+ return true;
+ });
+
+ action('audio.stop', ctx => {
+ if (!ctx.has(AudioSource)) return;
+ const source = ctx.get(AudioSource);
+ if (!source.playing) return;
+ source.playing = false;
+ ctx.set(AudioSource, source);
+ }, AUDIO);
+
+ // Latched only when a non-looping clip ends — the same shape as
+ // `timeline.finished`, so `onEnter: audio.play` plus an `audio.finished`
+ // transition is a state that lasts exactly as long as its sound.
+ condition('audio.finished', ctx => {
+ if (!ctx.has(AudioSource)) return false;
+ const source = ctx.get(AudioSource);
+ return source.finished && !source.playing;
+ }, { reads: [AudioSource._name] });
+
// The general-purpose write, through the engine's reflection writer — the
// same addressing a UIGear binding and a Timeline track use ("Component" +
// a dot path). One verb instead of a growing family of setters, and it
@@ -163,6 +198,31 @@ function parseValue(raw: AiParamValue): unknown {
}
}
+/**
+ * A `play`-shaped verb: one optional `clip` parameter, declared — which is what
+ * gives it an input pin in a graph instead of only a hand-typed `arg` — and a
+ * component edit that says whether anything changed.
+ */
+function playVerb<C extends AnyComponentDef>(
+ name: string,
+ component: C,
+ touches: AiTouches,
+ edit: (data: ComponentData<C>, clip: string) => boolean,
+): void {
+ if (aiRegistry.hasAction(name)) return;
+ aiRegistry.registerAction(name, {
+ params: CLIP_PARAM,
+ touches,
+ run: (ctx, _bb, _arg, params) => {
+ if (!ctx.has(component)) return;
+ const data = ctx.get(component);
+ if (edit(data, typeof params?.clip === 'string' ? params.clip.trim() : '')) {
+ ctx.set(component, data);
+ }
+ },
+ });
+}
+
function action(name: string, fn: AiAction<AiContext>, touches?: AiTouches): void {
if (!aiRegistry.hasAction(name)) aiRegistry.registerAction(name, { run: fn, touches });
}
diff --git a/sdk/src/audio/AudioComponents.ts b/sdk/src/audio/AudioComponents.ts
index 43760fbe5..154839723 100644
--- a/sdk/src/audio/AudioComponents.ts
+++ b/sdk/src/audio/AudioComponents.ts
@@ -9,6 +9,17 @@ export interface AudioSourceData {
volume: number;
pitch: number;
loop: boolean;
+ /**
+ * Raise it and the clip sounds; lower it and the voice stops. The engine
+ * lowers it again when a non-looping clip ends, so it reads as "is this
+ * sounding" — the flag contract TimelinePlayer and SpriteAnimator keep.
+ */
+ playing: boolean;
+ /**
+ * Latched true when a non-looping clip reaches its end; cleared when
+ * `playing` is raised again. Runtime-observable — don't author it.
+ */
+ finished: boolean;
playOnAwake: boolean;
spatial: boolean;
minDistance: number;
@@ -25,6 +36,8 @@ export const AudioSource = defineComponent<AudioSourceData>('AudioSource', {
volume: 1.0,
pitch: 1.0,
loop: false,
+ playing: false,
+ finished: false,
playOnAwake: false,
spatial: false,
minDistance: 100,
@@ -35,6 +48,10 @@ export const AudioSource = defineComponent<AudioSourceData>('AudioSource', {
enabled: true,
}, {
assetFields: [{ field: 'clip', type: 'audio' }],
+ fields: {
+ playing: { tooltip: 'Raise to play, lower to stop. The engine lowers it when a non-looping clip ends.' },
+ finished: { advanced: true, tooltip: 'Clip completed (runtime, read-only). Raise Playing to play it again.' },
+ },
});
export interface AudioListenerData {
diff --git a/sdk/src/audio/AudioPlugin.ts b/sdk/src/audio/AudioPlugin.ts
index 0a26a6825..b4024c0c9 100644
--- a/sdk/src/audio/AudioPlugin.ts
+++ b/sdk/src/audio/AudioPlugin.ts
@@ -24,6 +24,16 @@ function worldPositionOf(t: WorldTransformData): Vec3 {
return t.worldPosition ?? t.position;
}
+/**
+ * The voice a source currently has, and the clip it is a play OF. A source whose
+ * clip changed is no longer described by the sound in flight, and telling them
+ * apart is what lets one AudioSource play a second sound.
+ */
+interface ActiveVoice {
+ handle: AudioHandle;
+ clip: string;
+}
+
export interface AudioPluginConfig {
initialPoolSize?: number;
masterVolume?: number;
@@ -34,7 +44,7 @@ export interface AudioPluginConfig {
export class AudioPlugin implements Plugin {
name = 'audio';
private config_: AudioPluginConfig;
- private activeSourceHandles_: Map<number, AudioHandle> | null = null;
+ private activeSourceHandles_: Map<number, ActiveVoice> | null = null;
private playedEntities_: Set<number> | null = null;
private audio_: AudioAPI | null = null;
private offMemoryWarning_: (() => void) | null = null;
@@ -70,19 +80,25 @@ export class AudioPlugin implements Plugin {
if (config.sfxVolume !== undefined) mixer.sfx.volume = config.sfxVolume;
}
- const activeSourceHandles = new Map<number, AudioHandle>();
+ const activeSourceHandles = new Map<number, ActiveVoice>();
this.activeSourceHandles_ = activeSourceHandles;
const playedEntities = new Set<number>();
this.playedEntities_ = playedEntities;
this.offDespawn_ = app.world.onDespawn((entity: Entity) => {
- const handle = activeSourceHandles.get(entity);
- if (handle) {
- handle.stop();
+ const voice = activeSourceHandles.get(entity);
+ if (voice) {
+ voice.handle.stop();
activeSourceHandles.delete(entity);
}
playedEntities.delete(entity);
});
+ const stopVoice = (id: number): void => {
+ const voice = activeSourceHandles.get(id);
+ if (!voice) return;
+ voice.handle.stop();
+ activeSourceHandles.delete(id);
+ };
const liveEntities = new Set<number>();
/** Clips an authored playOnAwake is waiting on, and ones that will never arrive. */
const loadingClips = new Set<string>();
@@ -135,28 +151,63 @@ export class AudioPlugin implements Plugin {
for (const entity of sources) {
const source = world.get(entity, AudioSource) as AudioSourceData;
- if (!source.enabled || !source.clip) continue;
const id = entity as number;
+
+ if (!source.enabled) {
+ // Switching a source off has to silence the voice it
+ // already has: dropping the handle alone leaves a sound
+ // playing that nothing can reach any more.
+ stopVoice(id);
+ continue;
+ }
+ if (!source.clip) continue;
liveEntities.add(id);
- if (source.playOnAwake && !playedEntities.has(id) && !activeSourceHandles.has(id) && backend.isReady) {
+ // playOnAwake is one trigger expressed through the switch
+ // everything else reads, so a scene that starts a sound and
+ // an FSM hook that starts one take the same path.
+ let playing = source.playing;
+ if (source.playOnAwake && !playedEntities.has(id)) {
+ playedEntities.add(id);
+ if (!playing) {
+ playing = true;
+ world.update(entity, AudioSource, draft => { draft.playing = true; });
+ }
+ }
+
+ let voice = activeSourceHandles.get(id);
+ // A source that names a different clip now is not the one
+ // this voice is playing, so the voice goes and the raised
+ // flag plays what the source says today.
+ if (voice && voice.clip !== source.clip) {
+ stopVoice(id);
+ voice = undefined;
+ }
+
+ if (playing && !voice && backend.isReady) {
const buffer = audioAPI.getBufferHandle(source.clip);
if (buffer) {
// Through the API, not straight at the backend: the
// voice cap and the soft bus gain are decided there,
// and a sound outside that door gets neither.
- const handle = audioAPI.playBuffer(buffer, {
- bus: source.bus,
- volume: source.volume,
- loop: source.loop,
- playbackRate: source.pitch,
- priority: source.priority,
+ activeSourceHandles.set(id, {
+ clip: source.clip,
+ handle: audioAPI.playBuffer(buffer, {
+ bus: source.bus,
+ volume: source.volume,
+ loop: source.loop,
+ playbackRate: source.pitch,
+ priority: source.priority,
+ }),
});
- activeSourceHandles.set(id, handle);
- playedEntities.add(id);
+ // Whoever raised the flag is asking for this clip
+ // from the top, so the latch from the last one goes.
+ if (source.finished) {
+ world.update(entity, AudioSource, draft => { draft.finished = false; });
+ }
} else if (!unplayableClips.has(source.clip)) {
- // A scene setting playOnAwake has already said to play
- // it; demanding a preload call beside it is a component
+ // A source asking to play has already said so;
+ // demanding a preload call beside it is a component
// that silently does nothing. Unplayed until it lands.
if (!loadingClips.has(source.clip)) {
const clip = source.clip;
@@ -164,20 +215,26 @@ export class AudioPlugin implements Plugin {
void audioAPI.preload(clip)
.catch((err) => {
unplayableClips.add(clip);
- log.warn('audio', `playOnAwake: cannot load clip "${clip}"`, err);
+ log.warn('audio', `cannot load clip "${clip}"`, err);
})
.finally(() => loadingClips.delete(clip));
}
}
+ } else if (!playing && voice) {
+ stopVoice(id);
+ } else if (playing && voice && !voice.handle.isPlaying) {
+ // The voice ended by itself. Lowering the flag here is
+ // what makes it readable as "is this sounding" rather
+ // than "was it ever asked to sound".
+ activeSourceHandles.delete(id);
+ world.update(entity, AudioSource, draft => {
+ draft.playing = false;
+ draft.finished = true;
+ });
}
- if (source.spatial && activeSourceHandles.has(id)) {
- const handle = activeSourceHandles.get(id)!;
- if (!handle.isPlaying) {
- activeSourceHandles.delete(id);
- continue;
- }
-
+ const sounding = activeSourceHandles.get(id);
+ if (source.spatial && sounding) {
if (!hasListener && !spatialListenerWarned) {
log.warn('audio', 'spatial audio used but no AudioListener entity found');
spatialListenerWarned = true;
@@ -197,15 +254,16 @@ export class AudioPlugin implements Plugin {
const attenuation = calculateAttenuation(distance, spatialConfig);
const pan = calculatePanning(sourceAt, listenerAt, listenerRight, source.maxDistance);
- handle.setVolume(source.volume * attenuation);
- handle.setPan(pan);
+ sounding.handle.setVolume(source.volume * attenuation);
+ sounding.handle.setPan(pan);
}
}
- for (const [entityId, handle] of activeSourceHandles) {
- if (!liveEntities.has(entityId) || !handle.isPlaying) {
- activeSourceHandles.delete(entityId);
- }
+ // Sources the pass above did not reach at all. Their voices are
+ // STOPPED, not just forgotten: a dropped handle is a sound
+ // nothing can reach any more.
+ for (const entityId of [...activeSourceHandles.keys()]) {
+ if (!liveEntities.has(entityId)) stopVoice(entityId);
}
},
{ name: 'AudioUpdateSystem' }
@@ -215,8 +273,8 @@ export class AudioPlugin implements Plugin {
stopAllSources(): void {
if (this.activeSourceHandles_) {
- for (const handle of this.activeSourceHandles_.values()) {
- handle.stop();
+ for (const voice of this.activeSourceHandles_.values()) {
+ voice.handle.stop();
}
this.activeSourceHandles_.clear();
}
diff --git a/sdk/tests/ai-builtin-verbs.test.ts b/sdk/tests/ai-builtin-verbs.test.ts
index 3ea7ae402..4a9d3a086 100644
--- a/sdk/tests/ai-builtin-verbs.test.ts
+++ b/sdk/tests/ai-builtin-verbs.test.ts
@@ -3,8 +3,9 @@
/**
* @file ai-builtin-verbs.test.ts
* @brief The engine verbs an authored wire can reach with no game code:
- * `property.set` (the generic reflection write) and `ui.setVisible`
- * (show/hide, on the switch the renderer actually honours).
+ * `property.set` (the generic reflection write), `ui.setVisible`
+ * (show/hide, on the switch the renderer actually honours), and the
+ * audio verbs (the AudioSource flag, from a hook or a graph node).
*/
import { beforeEach, describe, expect, it } from 'vitest';
import { aiRegistry } from '../src/ai/fsm/AiContext';
@@ -12,6 +13,7 @@ import { ensureBuiltinAiRegistrations } from '../src/ai/builtins';
import { ensureControllerAiRegistrations } from '../src/ui/controller/ai-builtins';
import { Blackboard } from '../src/ai/fsm/Blackboard';
import { UINode } from '../src/ui/core/ui-node';
+import { AudioSource } from '../src/audio/AudioComponents';
import { UIDisplay } from '../src/wasm/wasm.generated';
import { registerComponent, defineComponent } from '../src/ecs/component';
import type { AnyComponentDef } from '../src/ecs/component';
@@ -42,7 +44,10 @@ function ctxFor(world: ReturnType<typeof makeWorld>, entity = E) {
entity,
world,
dt: 0,
- get: (c: AnyComponentDef) => world.get(entity, c),
+ // A COPY, the way an engine-backed component reads: a verb that edits
+ // the draft and forgets to hand it back changes nothing, and a test
+ // holding the stored object would confirm the write that never happened.
+ get: (c: AnyComponentDef) => ({ ...(world.get(entity, c) as object) }),
set: (c: AnyComponentDef, d: unknown) => world.set(entity, c, d),
has: (c: AnyComponentDef) => world.has(entity, c),
} as never;
@@ -120,3 +125,51 @@ describe('ui.setVisible', () => {
expect(aiRegistry.getActionParams('ui.setVisible')).toEqual([{ name: 'visible', type: 'bool' }]);
});
});
+
+describe('the audio verbs', () => {
+ /** A source as the scene authored it: a clip, silent. */
+ function sourced() {
+ const world = makeWorld();
+ world.insert(E, AudioSource, { ...AudioSource._default, clip: 'boom.wav' });
+ return world;
+ }
+
+ it('starts the source this entity carries', () => {
+ const world = sourced();
+ aiRegistry.getAction('audio.play')!(ctxFor(world), new Blackboard());
+ expect((world.get(E, AudioSource) as { playing: boolean }).playing).toBe(true);
+ });
+
+ it('takes the clip as a parameter, so a graph can wire one in', () => {
+ const world = sourced();
+ // Through the canonical string an .esfsm carries AND through the named
+ // parameter a graph pin supplies — one declaration, both forms.
+ aiRegistry.getAction('audio.play')!(ctxFor(world), new Blackboard(), 'hurt.wav');
+ expect(world.get(E, AudioSource)).toMatchObject({ clip: 'hurt.wav', playing: true });
+
+ aiRegistry.getAction('audio.play')!(ctxFor(world), new Blackboard(), undefined, { clip: 'die.wav' });
+ expect(world.get(E, AudioSource)).toMatchObject({ clip: 'die.wav', playing: true });
+ });
+
+ it('stops it, and answers whether the clip finished', () => {
+ const world = sourced();
+ world.set(E, AudioSource, { ...AudioSource._default, clip: 'boom.wav', finished: true });
+
+ expect(aiRegistry.getCondition('audio.finished')!(ctxFor(world), new Blackboard())).toBe(true);
+
+ // Both flags up — the latch still standing as a new play begins, before
+ // the system that clears it has run. A sound that is going has not
+ // finished.
+ world.set(E, AudioSource, { ...AudioSource._default, clip: 'boom.wav', playing: true, finished: true });
+ expect(aiRegistry.getCondition('audio.finished')!(ctxFor(world), new Blackboard())).toBe(false);
+
+ aiRegistry.getAction('audio.stop')!(ctxFor(world), new Blackboard());
+ expect((world.get(E, AudioSource) as { playing: boolean }).playing).toBe(false);
+ });
+
+ it('does nothing at all to an entity with no AudioSource', () => {
+ const world = makeWorld();
+ aiRegistry.getAction('audio.play')!(ctxFor(world), new Blackboard(), 'boom.wav');
+ expect(world.has(E, AudioSource)).toBe(false);
+ });
+});
diff --git a/sdk/tests/audio-plugin.test.ts b/sdk/tests/audio-plugin.test.ts
index a3b59a1e1..7d1ddee0e 100644
--- a/sdk/tests/audio-plugin.test.ts
+++ b/sdk/tests/audio-plugin.test.ts
@@ -96,9 +96,11 @@ describe('AudioPlugin', () => {
const handle1 = createMockHandle({ id: 1 });
const handle2 = createMockHandle({ id: 2 });
- const handles = new Map<number, AudioHandle>();
- handles.set(10, handle1);
- handles.set(20, handle2);
+ // The plugin records a voice as the handle AND the clip it is a play
+ // of, which is how one source can be given a second sound.
+ const handles = new Map<number, { handle: AudioHandle; clip: string }>();
+ handles.set(10, { handle: handle1, clip: 'a.wav' });
+ handles.set(20, { handle: handle2, clip: 'b.wav' });
(plugin as any).activeSourceHandles_ = handles;
plugin.stopAllSources();
diff --git a/sdk/tests/audio-source-flag.test.ts b/sdk/tests/audio-source-flag.test.ts
new file mode 100644
index 000000000..79d80c091
--- /dev/null
+++ b/sdk/tests/audio-source-flag.test.ts
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file audio-source-flag.test.ts
+ * @brief Starting and stopping a sound through the component — the flag
+ * contract an FSM hook, a graph node and the inspector all write.
+ *
+ * Held here: that raising the flag sounds the clip and lowering it
+ * stops the voice, and the two ways such a switch goes wrong — a voice
+ * that ends and starts itself over forever, and a sound nothing can
+ * reach any more once its source is switched off, re-clipped or
+ * un-clipped.
+ */
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+import type { AudioHandle, PlatformAudioBackend } from '../src/audio/PlatformAudioBackend';
+
+const backend = {
+ name: 'Mock',
+ mixer: null,
+ isReady: true,
+ initialize: vi.fn().mockResolvedValue(undefined),
+ ensureResumed: vi.fn().mockResolvedValue(undefined),
+ loadBuffer: vi.fn().mockResolvedValue({ id: 1, duration: 1 }),
+ unloadBuffer: vi.fn(),
+ play: vi.fn(),
+ suspend: vi.fn(), resume: vi.fn(), dispose: vi.fn(),
+} as unknown as PlatformAudioBackend;
+
+vi.mock('../src/platform/base', async (importOriginal) => ({
+ ...(await importOriginal<typeof import('../src/platform/base')>()),
+ platformCreateAudioBackend: () => backend,
+ platformOnMemoryWarning: () => () => {},
+}));
+
+const { App } = await import('../src/app/app');
+const { AudioPlugin } = await import('../src/audio/AudioPlugin');
+const { Audio } = await import('../src/audio/Audio');
+const { AudioSource } = await import('../src/audio/AudioComponents');
+
+/** A voice whose end the test decides, the way a real one ends on its own. */
+function voice(): AudioHandle & { isPlaying: boolean } {
+ return {
+ id: 1, stop: vi.fn(), pause: vi.fn(), resume: vi.fn(),
+ setVolume: vi.fn(), setPan: vi.fn(), setLoop: vi.fn(), setPlaybackRate: vi.fn(),
+ isPlaying: true, currentTime: 0, duration: 1,
+ };
+}
+
+/** An app with the plugin built and two clips already resident. */
+async function appWithClips() {
+ const app = App.new();
+ app.addPlugin(new AudioPlugin());
+ const audio = app.getResource(Audio);
+ await audio.preload('boom.wav');
+ await audio.preload('ping.wav');
+ return app;
+}
+
+let voices: ReturnType<typeof voice>[] = [];
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ voices = [];
+ (backend.play as ReturnType<typeof vi.fn>).mockImplementation(() => {
+ const v = voice();
+ voices.push(v);
+ return v;
+ });
+});
+
+describe('the AudioSource playing flag', () => {
+ it('sounds a source that was never playOnAwake, and silences it again', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true });
+
+ await app.tick(1 / 60);
+ expect(backend.play).toHaveBeenCalledTimes(1);
+
+ app.world.update(e, AudioSource, d => { d.playing = false; });
+ await app.tick(1 / 60);
+ expect(voices[0]!.stop).toHaveBeenCalledTimes(1);
+ });
+
+ it('lowers the flag and latches finished when the clip ends by itself', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true });
+ await app.tick(1 / 60);
+
+ voices[0]!.isPlaying = false;
+ await app.tick(1 / 60);
+
+ const source = app.world.get(e, AudioSource);
+ expect(source.playing).toBe(false);
+ expect(source.finished).toBe(true);
+ });
+
+ it('does not start the clip over once it has ended', async () => {
+ // The failure this exists for: retiring the handle in one place and
+ // reading the flag in another leaves a raised flag with no voice, and
+ // the next frame plays the sound again — forever, once a frame.
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true });
+ await app.tick(1 / 60);
+ voices[0]!.isPlaying = false;
+
+ for (let i = 0; i < 5; i++) await app.tick(1 / 60);
+
+ expect(backend.play).toHaveBeenCalledTimes(1);
+ });
+
+ it('plays it again when the flag is raised after it finished', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true });
+ await app.tick(1 / 60);
+ voices[0]!.isPlaying = false;
+ await app.tick(1 / 60);
+
+ app.world.update(e, AudioSource, d => { d.playing = true; });
+ await app.tick(1 / 60);
+
+ expect(backend.play).toHaveBeenCalledTimes(2);
+ // The latch describes the LAST play, so a replay clears it — otherwise
+ // `audio.finished` answers yes about a sound that is currently going.
+ expect(app.world.get(e, AudioSource).finished).toBe(false);
+ });
+
+ it('keeps a looping voice playing, with nothing to latch', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true, loop: true });
+
+ for (let i = 0; i < 5; i++) await app.tick(1 / 60);
+
+ expect(backend.play).toHaveBeenCalledTimes(1);
+ expect(app.world.get(e, AudioSource).playing).toBe(true);
+ expect(app.world.get(e, AudioSource).finished).toBe(false);
+ });
+
+ it('drops the voice in flight when the source names a different clip', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true });
+ await app.tick(1 / 60);
+
+ app.world.update(e, AudioSource, d => { d.clip = 'ping.wav'; });
+ await app.tick(1 / 60);
+
+ // One AudioSource playing a second sound is what a character with a hurt
+ // clip and a death clip is: the voice is a play OF a clip, not of the
+ // source, so the old one goes and the new one starts.
+ expect(voices[0]!.stop).toHaveBeenCalledTimes(1);
+ expect(backend.play).toHaveBeenCalledTimes(2);
+ });
+
+ it('silences a source that is switched off', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true, loop: true });
+ await app.tick(1 / 60);
+
+ app.world.update(e, AudioSource, d => { d.enabled = false; });
+ await app.tick(1 / 60);
+
+ // A dropped handle would leave the loop sounding with nothing able to
+ // reach it.
+ expect(voices[0]!.stop).toHaveBeenCalledTimes(1);
+ });
+
+ it('silences a source whose clip is taken away', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playing: true, loop: true });
+ await app.tick(1 / 60);
+
+ app.world.update(e, AudioSource, d => { d.clip = ''; });
+ await app.tick(1 / 60);
+
+ // A source with no clip is skipped entirely, so the voice has to be
+ // stopped where it is noticed missing — forgetting the handle is how a
+ // loop outlives everything that could ever reach it.
+ expect(voices[0]!.stop).toHaveBeenCalledTimes(1);
+ });
+
+ it('raises the flag once for playOnAwake, and leaves it lowered after', async () => {
+ const app = await appWithClips();
+ const e = app.world.spawn();
+ app.world.insert(e, AudioSource, { clip: 'boom.wav', playOnAwake: true });
+
+ await app.tick(1 / 60);
+ expect(app.world.get(e, AudioSource).playing).toBe(true);
+ expect(backend.play).toHaveBeenCalledTimes(1);
+
+ voices[0]!.isPlaying = false;
+ for (let i = 0; i < 3; i++) await app.tick(1 / 60);
+
+ expect(app.world.get(e, AudioSource).playing).toBe(false);
+ expect(backend.play).toHaveBeenCalledTimes(1);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 653812667..c2a041dbf 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -28,6 +28,12 @@ export const NOTES_FLOOR = 'v0.65.0';
* the reader recognises anyway.
*/
export const NOTED = {
+ 'feat(audio): a sound anything authored can start':
+ { note: 'A sound anything authored can start.' },
+
+ 'feat(audio): the audio verbs describe themselves in the palettes':
+ { note: 'A sound anything authored can start.' },
+
'feat(logic): a graph can spawn a prefab, and the node says which one':
{ note: 'Gameplay you can draw.' },