the ultimate vocal synthesizer - sing, speak, beatbox, and more
The engine in Rust
Design, 2026-11-05. Written before the port; the "Status" section at the end records what shipped.
Why
- JDAW. JDAW (
~/jdaw/VISION.md) is a native Rust engine with a web UI. CHIMERA joins it as a source module "once its DSP is ported to Rust" (principles 5, 9, 14). JDAW needs a headless, deterministic, real-time-safe library with no browser in it. Offline renders must equal real-time output (6), the engine must be able to use every core (7), and macOS and Linux must both work, with WASM for the web (8). - Audio-thread headroom. The TS engine uses 30–45% of one fast core in the AudioWorklet (male voice at 2× oversampling, the creatures at a similar cost). A slower laptop has little headroom left.
The UI lag is a separate problem. Measured in Chromium, it comes from the anatomy view's canvas
blur() filter (applied to every fill, every frame), not from the engine. It gets its own small
fix.
Scope
Everything under src/engine/ moves to Rust, and the TS engine is deleted, so there is one
source of truth and no drift. That covers the airway FDTD, the frequency-domain oracle, the human
anatomy and articulators, the gestural score, the glottis/valves/syrinx sources, the creatures,
the phonetics (IPA, CMUdict G2P, syllables, durations, planner, singing, beatbox, demo songs) and
the analysis code (FFT, LPC, YIN). The worklet's state machine (src/worklet/processor.ts: body
switches, creature switches, beat player, ordered replay) moves too, because JDAW needs that
behaviour as well.
These stay in TS:
- UI-only code: React, the canvas renderers, the creature drawing data (silhouettes and tube
paths), the zustand stores, Web MIDI, and the live Sing controller (
ui/singing.ts). The controller calls the Rust planners. - The browser audio graph: the analysers and the close-mic chain, which use WebAudio nodes.
Layout
Cargo.toml workspace
crates/chimera/ the engine (lib). deps: libm, serde(+derive), serde_json. No I/O, no globals but OnceLock tables.
crates/chimera-wasm/ cdylib for wasm32-unknown-unknown: a C ABI over chimera, no wasm-bindgen
crates/chimera-cli/ `chimera render …`: headless renders to WAV (replaces tools/render-*.ts) + calibration bins
src/engine/ (TS) thin typed facade over the wasm module: same names/types the UI imports today
Module tree of chimera mirrors src/engine 1:1 (airway::network, anatomy::human, …), so the
port can be reviewed file against file. chimera::engine::Engine is the old processor.
Real-time contract (what JDAW needs)
Engine,HumanVoiceandCreatureVoiceareSendand share no mutable state, so each can render on its own thread.Steady-state rendering allocates nothing. That covers
render(&mut [f32]), control steps, geometry frames, noise-source lists, beat-sequencer gesture builds and telemetry snapshots, which all reuse buffers held in their structs.Bounded state with defined overflow. The gesture score holds at most
GestureScore::CAPACITY(16 384) gestures. A gesture added past that is dropped and counted (the TS score had no bound). Eachschedulecommand takes at most 4 000 gestures, as the TS worklet did. Pruning compacts in place and keeps insertion order, which matters because blends sum in that order.Commands that allocate. These are meant for when nothing is sounding, or for a host that builds things off the audio thread:
- a body switch (it builds a new airway network);
- a creature switch;
- the first
Engine::new(it also loads the phonetics tables, so nothing initialises lazily on the audio thread later).
Scheduling, releases, artic/source/gate changes and beat edits don't allocate.
Planners allocate (text → gestures) and belong on a non-real-time thread: the main thread in the browser, an edit thread in JDAW.
plan_note_onfor live sung notes is small but allocates too; making it allocation-free is left for the JDAW integration.Block-size independence.
Engine::renderadvances in fixed 128-sample sub-blocks, and the per-block work (beat-sequencer ticks) happens at those sub-block boundaries, which sit at absolute sample positions. The same commands at the same absolute sample positions therefore give the same samples whatever the host's block size. The TS worklet ticked once per host block, so a 48 000-sample offline block would have lost beats. Commands apply at the start of the nextrendercall; a host that needs sample accuracy splits its block at the event.Events out. Responses go into a queue that the host drains after
handle/render:Ready { body, creature, n_ph, n_or }after a voice switch, andPlanned { id, t0, end }for a schedule that carries an id (the UI's timelines uset0).Telemetry. A snapshot is fixed-size and written into caller-owned buffers. Interval measures (RMS levels, the glottal scope's cycle capture) reset on each read; the struct docs mark which fields are which. Load measurement and message cadence stay in JS.
The human clock stops while a creature plays, as before.
Determinism
- Repeatability (promised). A given build renders the same samples every time, for any block size.
- Cross-target identity (tested where possible). All transcendental math goes through
chimera::math, which wraps the pure-Rustlibmcrate at a pinned version.sqrtand the basic operations are IEEE-exact, and Rust never contracts to FMA on its own. A test renders fixed scenarios natively and through the wasm build and requires identical samples on x86-64 Linux. aarch64/macOS is untested here, so identity there is a goal, not a claim. - JS semantics.
Math.min/Math.maxbecomejs_min/js_max, which propagate NaN from either argument and order −0 below +0. Ternaries are ported as written.Math.roundbecomesjs_round, which rounds halves up. The PRNG uses explicit wrappingu32arithmetic. - Precision. State and gesture values are
f64throughout; only output and telemetry aref32, as the TSFloat32Arrays were. - No FTZ/DAZ. Hosts must not enable flush-to-zero or denormals-are-zero, because those change the samples. The engine already flushes silent state itself.
Port fidelity
The port is literal. It keeps the same equations, the same operation order, the same constants, the same PRNG seeds, and the same container order where order decides random-number consumption or sums (noise sources, gestures). It retunes nothing. The checks:
- Golden comparison against the TS engine, before the TS engine is deleted. Each
tools/golden/<area>.tsruns the TS engine on fixed scenarios and writes fixtures, and the Rust tests compare against them.- Numeric outputs (geometry, area functions, plans, G2P, syllables) must agree to 1e-9 relative, with an absolute floor near zero. Plans and G2P must agree exactly in structure.
- Audio is compared on scenarios with events throughout (artic changes, releases, cancels, calls, body switches), not just at the start. Renders must agree sample for sample (≲1e-6 of peak) until a discrete decision flips on a last-ulp difference between V8's libm and Rust's. After that, every 10 ms window's RMS must agree to 0.05 dB, and onset/release timing, silence, pitch and closures are covered by the ported behavioural tests. Any divergence before an event is investigated, not tolerated.
- With the TS engine gone, the fixtures record what it did (
crates/chimera/tests/golden/README.md). The human-voice audio can be re-blessed from the Rust engine (CHIMERA_BLESS=1).
- The existing bun tests (resonances against analytic values, FDTD against the chain
matrix, stability, formants, closures, singing behaviour) are ported to
cargo testwith the same thresholds. - The app's e2e smoke test (
tools/e2e.ts) runs against the wasm build and stays green. - Benchmarks measure native and wasm real-time factors against the TS numbers taken before the port.
The browser boundary
One wasm module. The main thread compiles it once and passes the
WebAssembly.Moduleto the worklet inprocessorOptions(structured clone). Each side instantiates it synchronously. The import object is empty: no WASI and no JS glue. A build check asserts the artifact has no imports.Startup ordering. The wasm module must be ready before any UI module that calls into it runs, because
ui/singing.tsbuilds syllables when it loads.main.tsxawaits the module, then imports the app. A failure shows a retryable error screen.Audio thread (worklet). The exports are C ABI with numbers and pointers only, over one engine handle.
Engine-owned buffers have a stable address for the engine's lifetime: an output buffer of up to 4 096 frames, a command scratch area, and the telemetry block. JS re-creates its typed-array views whenever
memory.bufferhas changed, and copies telemetry before the next call.Gestures cross as fixed 256-byte little-endian records, layout version 1, specified in
crates/chimera-wasm/src/lib.rsand mirrored bysrc/engine/wire.ts:- eight
f64s: t0, t1, rampIn, rampOut, strength, speed, releaseHold, tag; - 22
f64targets inControlKeyorder; - a u8 label length and 24 bytes of UTF-8;
- padding.
NaN marks an absent optional or target. Labels are cut at a character boundary.
- eight
The worklet scope has no
TextEncoder/TextDecoder, so a tiny UTF-8 helper handles labels and ids.
Main thread. Rich, non-real-time calls are JSON in and out through
serde_json:- planning, G2P and syllables;
- the anatomy model: frame, transfer function and formants, the
P/unPtemplate maps, scale, tube node positions, and branch → drawing position; - the creature, beatbox and vowel catalogues.
The TS facade gives the UI the types it uses today. Creature drawing data stays in TS.
Faults. The crate is built with
panic = "abort", so a panic in the worklet traps. The processor then outputs silence for good and posts one error. The main thread may build a new node, which restarts playback state: score, clocks and beat. There's no attempt at seamless recovery.
Build
bun run wasmrunscargo build -p chimera-wasm --release --target wasm32-unknown-unknownand copies the result tosrc/wasm/chimera.wasm, which Vite imports with?url.dev,buildandtestrun it first.bun testkeeps the TS-side tests (the codec, the facade, and the e2e run against the built app).cargo testholds the engine tests.
What JDAW gets
chimera as a path or git dependency:
Engine::new(sample_rate)with:- commands as methods:
set_body,set_artic,set_source,set_gate,schedule,release,stop_speech,set_beat_*,beat_hit,set_creature,set_creature_params,play_callandstop_call; render(&mut [f32]);telemetry/creature_telemetry.
- commands as methods:
- The voices directly:
HumanVoice,CreatureVoice. - The planners:
plan_phrases,plan_song,plan_note_on/off,lyrics_to_syllables, andEnglishG2Pwith a dictionary given as&str.
JDAW's eventual module trait (note-on/off with lyrics and expression read from the notes) is an adapter over these, written when JDAW's module interface exists. Designing it now would be speculative (JDAW principle: generality must earn its place).
Tools
render-speech,render-text,render-song,render-beatbox,render-creatureandrender-vowelsbecomechimera render <kind> …, which writes a WAV. The spectrogram PNG stays a TS tool (tools/spectrogram.ts in.wav out.png, reusingtools/lib).calibrate-vowelsandcalibrate-consonantsbecomechimera calibrate vowels|consonants, because they regenerate committed data.gen-syrinx-tableandgen-ipa-nfcstay TS, because neither drives the engine. The NFC data comes from the JS engine's ICU.speech-diagnostics(nasal leakage, formant undershoot, bandwidths, pitch) becomeschimera diagnose, because it's a measurement tool for future tuning.- The one-off explorations are deleted:
proto-vowel,explore-syrinx,measure-closures,measure-consonantsanddraw-anatomy. Their findings are already in the code and the commit history.
Review
A design review (Codex, 2026-11-05) changed several points: block-size independence of the beat sequencer, the bounded score and the list of allocating commands, latching faults instead of rebuilding on the audio thread, determinism claimed only as tested, the event and telemetry contract, the ABI layout, facade gaps and startup ordering.
It also suggested keeping G2P, planning and songs in TS for now. That suggestion was declined:
- the beat sequencer runs in the engine and needs most of the planner core (inventory, oral gestures, vowel targets);
- JDAW's headless core (principle 4) has to sing lyrics on notes (principle 15);
- phonetics split across two languages would drift.
The worklet's per-block replay budget stays out of Rust, because commands apply at the next render call and so it isn't needed.
Status (what shipped)
Crates
crates/chimera: the engine.crates/chimera-wasm: the browser ABI (about 1 MB of wasm, no imports).crates/chimera-cli: thechimeraCLI.
Browser
src/wasm/holds the facade and codec.src/worklet/processor.tsruns on the wasm engine.src/engine/(the TS engine) is gone.
Verification
- Golden fixtures from the TS engine (
crates/chimera/tests/golden/README.md):- human voice: within 1e-9 of peak over every render;
- creatures: within 2e-10, including the chaotic and self-oscillating modes;
- plans and beat sequences: bit-exact apart from 1-ulp
f0values; - G2P, IPA and syllables: exact on about 3 200 cases;
- geometry: within 8e-14 relative; analysis (transfer functions, formants, FFT, LPC): within 1e-10 relative (the UI's dB curve 9.7e-11, formants 1.1e-12).
- The bun tests that were about the engine are now
cargo tests with the same thresholds. The UI's live paths (Sing keys, worklet ordering, creature telemetry layout) run on the wasm build. - Native and wasm render bit-identical samples on x86-64.
- Block size: a full session (sequencer, body switch, held note) is bit-identical in blocks of 1 to 48 000 samples.
- Allocations: counted through the audio-thread ABI, from idle into labelled speech, the sequencer and a creature. There are none.
- CLI renders match the old TS tools to under 1e-6 of peak.
Speed (Ryzen 7 8745HS)
- Male voice at 2× oversampling, rendered offline: 46% of a core for TS in Chromium, 32% for wasm in Chromium, 19–21% native.
- Engine load in the running app: human 40% → 29%, creatures roughly halved (seal 56% → 28%).
Where it differs from the design above
- Commands are methods applied at once, not a
Commandenum. The worklet's body/creature pending queues and per-block replay budget are gone, since ordering is the call order. - Schedules return their clock offset, and the worklet posts
ready/planneditself, so there is no event queue. - The FDTD step uses checked indexing.
get_uncheckedgained nothing measurable, and it relied on an invariant that safe callers could break. - Only the human-voice audio fixtures can be re-blessed. The others record the TS engine.
- Vowel calibration isn't bit-reproducible.
chimera calibrate vowelsmoves 7 posture values by ±0.001–0.002: last-ulp math differences, amplified by Nelder–Mead on flat minima. The committed table is kept, so the voice is unchanged.
Open
- Determinism on aarch64/macOS is untested.
- Performance beyond TS parity. Exactness with the TS engine ruled out reciprocal multiplies and similar changes in the FDTD step. That's possible now, with a re-bless.
- Real-time planning.
plan_note_onallocates; a JDAW live-input path will want a non-allocating form.