Skip to content

A Claude Code plugin that turns a story into a finished anime

Ruby

Replace Stable Audio SFX with ElevenLabs v3

Francesco 'makevoid' Canessaadc1a76eb4e7
diff --git a/README.md b/README.md
index 7a8df06..59a58dc 100644
--- a/README.md
+++ b/README.md
@@ -82,7 +82,7 @@ Each stage reads prompts from `prompts/<run>/` and writes to `output/<run>/`, so
3. **Keyframes.** GPT Image 2.6 paints the first frame of each shot from the sheets.
4. **Shots.** MiniMax H3 animates each keyframe as one continuous take, with the camera move written into the prompt and the speaker's voice as target audio so the mouth follows the line. Held stills and colour inserts cost nothing. Existing takes can be reused across scenes with `from_take`. For moments that need exact placement, [layered scenes](skills/make-an-anime/references/layered-scenes.md) composite green-screen sprites over a painted plate.
5. **Overlay.** A p5 script (`05_overlay.js`) draws titles, subtitles, fades and any composite work.
-6. **Sound.** `sfx:gen` creates sound effects with Stable Audio, and `sfx:mix` places them from a cue list and levels them against the mix.
+6. **Sound.** `sfx:gen` creates sound effects with ElevenLabs Sound Effects v3, and `sfx:mix` places them from a cue list and levels them against the mix.
7. **Finish.** `vfx:render` adds grain, glow or impact effects with Swift Core Image, on frame-accurate cues. The optional `vfx:bake` pass preserves the plain picture so repeating the finish does not accumulate effects.
For detailed staging and playback review, see [Staging and review](skills/make-an-anime/references/staging-and-review.md). Local graphics tools include [deterministic particles and light-only glow](skills/make-an-anime/references/particles-and-glow.md), plus [animation asset preparation](skills/make-an-anime/references/animation-assets.md).
diff --git a/skills/make-an-anime/examples/01-sample-seven/docs/PLAN.md b/skills/make-an-anime/examples/01-sample-seven/docs/PLAN.md
index 6b42dad..e05dff9 100644
--- a/skills/make-an-anime/examples/01-sample-seven/docs/PLAN.md
+++ b/skills/make-an-anime/examples/01-sample-seven/docs/PLAN.md
@@ -14,7 +14,7 @@ plans that grew through review: `examples/02-final-inspection`, `03-riverbank-bo
| MiniMax H3 Max 768P | 5 takes, 13 s generated | 3 more |
| ElevenLabs Eleven v4 | 3 lines | 3 more |
| ElevenLabs Music v2.5 | 1 score of 14 s | 1 more |
-| Stable Audio SFX | 4 sounds | 2 more |
+| ElevenLabs Sound Effects v3 | 4 sounds | 2 more |
| Whisper | 3 line checks | 3 more |
## Creative direction
diff --git a/skills/make-an-anime/examples/01-sample-seven/prompts/lab-s01-sfx/sfx.yml b/skills/make-an-anime/examples/01-sample-seven/prompts/lab-s01-sfx/sfx.yml
index dbcf569..67db951 100644
--- a/skills/make-an-anime/examples/01-sample-seven/prompts/lab-s01-sfx/sfx.yml
+++ b/skills/make-an-anime/examples/01-sample-seven/prompts/lab-s01-sfx/sfx.yml
@@ -7,10 +7,10 @@ peak_db: -3
max_gain_db: 18
voice_dip_db: 5
sounds:
- room: {prompt: "A quiet empty laboratory at night, a low steady refrigerator hum and a faint ticking wall clock, no music, no voices", duration: 14, seed: 71}
- chair: {prompt: "A wheeled stool shoved back hard on a tiled floor, one quick scrape, isolated, no music, no voices", duration: 1, seed: 72}
- steps: {prompt: "One person sprinting in sneakers on a tiled floor, fast light footsteps, isolated, no music, no voices", duration: 2, seed: 73}
- knock: {prompt: "A glass flask set down hard on a wooden desk, one solid knock with a small glassy ring, isolated, no music, no voices", duration: 1, seed: 74}
+ room: {prompt: "A quiet empty laboratory at night, a low steady refrigerator hum and a faint ticking wall clock, no music, no voices", duration: 14}
+ chair: {prompt: "A wheeled stool shoved back hard on a tiled floor, one quick scrape, isolated, no music, no voices", duration: 1}
+ steps: {prompt: "One person sprinting in sneakers on a tiled floor, fast light footsteps, isolated, no music, no voices", duration: 2}
+ knock: {prompt: "A glass flask set down hard on a wooden desk, one solid knock with a small glassy ring, isolated, no music, no voices", duration: 1}
cues:
- {at: 0.0, sound: room, rel_db: -10, len: 14.0, fade: 0.6}
- {at: 7.9, sound: chair, rel_db: 2, len: 0.5, fade: 0.05}
diff --git a/skills/make-an-anime/examples/02-final-inspection/prompts/fi-s01-sfx/sfx.yml b/skills/make-an-anime/examples/02-final-inspection/prompts/fi-s01-sfx/sfx.yml
index 63a1cf8..1571ab0 100644
--- a/skills/make-an-anime/examples/02-final-inspection/prompts/fi-s01-sfx/sfx.yml
+++ b/skills/make-an-anime/examples/02-final-inspection/prompts/fi-s01-sfx/sfx.yml
@@ -6,14 +6,14 @@ peak_db: -3
max_gain_db: 18
voice_dip_db: 5
sounds:
- wind: {prompt: "Cold steady wind over an empty snowfield and around large rusted metal, low and hollow, no music, no voices", duration: 40, seed: 61}
- surf: {prompt: "Distant slow waves on a cold shore heard from far away across snow, soft and low, no music, no voices, no birds", duration: 40, seed: 62}
- creak: {prompt: "One long slow low creak of huge old rusted steel settling in the cold, distant and heavy, isolated, no music, no voices", duration: 4, seed: 63}
- brush: {prompt: "A leather glove brushing dry powder snow off metal, two soft strokes, close up, isolated, no music, no voices", duration: 2, seed: 64}
- pour: {prompt: "Hot tea poured from a thermos into a small enamel cup, a thin steady stream, close up, outdoors, no music, no voices", duration: 2, seed: 65}
- cup_set: {prompt: "A small enamel cup set down carefully on thick steel plate, one small soft clink, isolated, no music, no voices", duration: 1, seed: 66}
- boots: {prompt: "Slow heavy boots walking in dry fresh snow, a soft regular crunch, an old man's short steps, no music, no voices", duration: 6, seed: 67}
- hum: {prompt: "A very low almost subsonic electrical hum slowly waking up inside a huge old machine, deep and steady, faint, no music, no voices", duration: 16, seed: 68}
+ wind: {prompt: "Cold steady wind over an empty snowfield and around large rusted metal, low and hollow, no music, no voices", duration: 40, loop: true}
+ surf: {prompt: "Distant slow waves on a cold shore heard from far away across snow, soft and low, no music, no voices, no birds", duration: 40, loop: true}
+ creak: {prompt: "One long slow low creak of huge old rusted steel settling in the cold, distant and heavy, isolated, no music, no voices", duration: 4}
+ brush: {prompt: "A leather glove brushing dry powder snow off metal, two soft strokes, close up, isolated, no music, no voices", duration: 2}
+ pour: {prompt: "Hot tea poured from a thermos into a small enamel cup, a thin steady stream, close up, outdoors, no music, no voices", duration: 2}
+ cup_set: {prompt: "A small enamel cup set down carefully on thick steel plate, one small soft clink, isolated, no music, no voices", duration: 1}
+ boots: {prompt: "Slow heavy boots walking in dry fresh snow, a soft regular crunch, an old man's short steps, no music, no voices", duration: 6}
+ hum: {prompt: "A very low almost subsonic electrical hum slowly waking up inside a huge old machine, deep and steady, faint, no music, no voices", duration: 16}
cues:
- {at: 0.0, sound: wind, rel_db: -3, len: 34.0, fade: 1.0}
- {at: 0.0, sound: surf, rel_db: -13, len: 34.0, fade: 1.0}
diff --git a/skills/make-an-anime/examples/03-riverbank-bout/docs/PLAN.md b/skills/make-an-anime/examples/03-riverbank-bout/docs/PLAN.md
index 7624cca..a091cd0 100644
--- a/skills/make-an-anime/examples/03-riverbank-bout/docs/PLAN.md
+++ b/skills/make-an-anime/examples/03-riverbank-bout/docs/PLAN.md
@@ -10,7 +10,7 @@ A shonen fight of about 17 s, 24 fps, 16:9, 1080p, requested after a first shone
| MiniMax H3 Max 768P | 8 takes (2 panels, 6 shots) | 5 more |
| ElevenLabs Eleven v4 | 5 lines | 6 more |
| ElevenLabs Music v2.5 | 1 score | 1 more |
-| Stable Audio SFX | up to 8 sounds | 4 more |
+| ElevenLabs Sound Effects v3 | up to 8 sounds | 4 more |
| Whisper | 5 line checks | 6 more |
## What changes from the first attempt
diff --git a/skills/make-an-anime/examples/03-riverbank-bout/prompts/bout-s01-sfx/sfx.yml b/skills/make-an-anime/examples/03-riverbank-bout/prompts/bout-s01-sfx/sfx.yml
index c851b3c..688741a 100644
--- a/skills/make-an-anime/examples/03-riverbank-bout/prompts/bout-s01-sfx/sfx.yml
+++ b/skills/make-an-anime/examples/03-riverbank-bout/prompts/bout-s01-sfx/sfx.yml
@@ -6,13 +6,13 @@ peak_db: -3
max_gain_db: 18
voice_dip_db: 5
sounds:
- river: {prompt: "Evening riverbank ambience, gentle wind through tall grass and soft water lapping, steady, no music, no voices", duration: 18, seed: 41}
- train: {prompt: "A distant commuter train crossing a steel railway bridge far away, soft rhythmic clatter, no music, no voices", duration: 4, seed: 42}
- run: {prompt: "One person sprinting barefoot on grass and packed earth, fast heavy footsteps, isolated, no music, no voices", duration: 1.5, seed: 48}
- whoosh: {prompt: "One short fast punch whoosh through the air, anime fight sound effect, isolated, no music, no voices", duration: 1, seed: 44}
- block: {prompt: "One hard punch landing on a raised forearm, a tight meaty smack, anime fight sound effect, isolated, no music, no voices", duration: 1, seed: 45}
- impact: {prompt: "One huge heavy punch impact with a deep boom and a sharp crack, anime finishing blow sound effect, isolated, no music, no voices", duration: 1.5, seed: 46}
- thud: {prompt: "A body slamming flat onto grass and earth, one heavy thud with a short dusty skid, isolated, no music, no voices", duration: 1, seed: 47}
+ river: {prompt: "Evening riverbank ambience, gentle wind through tall grass and soft water lapping, steady, no music, no voices", duration: 18}
+ train: {prompt: "A distant commuter train crossing a steel railway bridge far away, soft rhythmic clatter, no music, no voices", duration: 4}
+ run: {prompt: "One person sprinting barefoot on grass and packed earth, fast heavy footsteps, isolated, no music, no voices", duration: 1.5}
+ whoosh: {prompt: "One short fast punch whoosh through the air, anime fight sound effect, isolated, no music, no voices", duration: 1}
+ block: {prompt: "One hard punch landing on a raised forearm, a tight meaty smack, anime fight sound effect, isolated, no music, no voices", duration: 1}
+ impact: {prompt: "One huge heavy punch impact with a deep boom and a sharp crack, anime finishing blow sound effect, isolated, no music, no voices", duration: 1.5}
+ thud: {prompt: "A body slamming flat onto grass and earth, one heavy thud with a short dusty skid, isolated, no music, no voices", duration: 1}
cues:
- {at: 0.0, sound: river, rel_db: -10, len: 17.08, fade: 0.4}
- {at: 0.8, sound: train, rel_db: -6, len: 4.0, fade: 0.8}
diff --git a/skills/make-an-anime/examples/04-too-salty/docs/PLAN.md b/skills/make-an-anime/examples/04-too-salty/docs/PLAN.md
index 652e5e8..63b2f24 100644
--- a/skills/make-an-anime/examples/04-too-salty/docs/PLAN.md
+++ b/skills/make-an-anime/examples/04-too-salty/docs/PLAN.md
@@ -10,7 +10,7 @@ The user's script, `docs/BRIEF.md`, is the source. About 70 s (73.5 s as cut), 2
| MiniMax H3 Max 768P | about 18 takes | 10 more |
| ElevenLabs Eleven v4 | 6 voice tests, 13 lines | 10 more |
| ElevenLabs Music v2.5 | 1 score | 1 more |
-| Stable Audio SFX | about 9 sounds | 6 more |
+| ElevenLabs Sound Effects v3 | about 9 sounds | 6 more |
| Whisper | line checks | as needed |
## Creative direction
diff --git a/skills/make-an-anime/examples/04-too-salty/prompts/salty-s01-sfx/sfx.yml b/skills/make-an-anime/examples/04-too-salty/prompts/salty-s01-sfx/sfx.yml
index f24d2a7..8f78622 100644
--- a/skills/make-an-anime/examples/04-too-salty/prompts/salty-s01-sfx/sfx.yml
+++ b/skills/make-an-anime/examples/04-too-salty/prompts/salty-s01-sfx/sfx.yml
@@ -6,15 +6,15 @@ peak_db: -3
max_gain_db: 18
voice_dip_db: 5
sounds:
- rain: {prompt: "Heavy steady rain falling on a corrugated tin awning, heard from inside a small quiet room, metallic pattering, no music, no voices, no thunder", duration: 47, seed: 51}
- sizzle_loud: {prompt: "Beaten egg sizzling loudly and angrily in a very hot copper pan, spitting and crackling, close up, no music, no voices", duration: 5, seed: 52}
- sizzle_soft: {prompt: "Egg sizzling very softly and steadily in a copper pan on a low flame, a gentle whisper, close up, no music, no voices", duration: 12, seed: 53}
- ring: {prompt: "One light tap of a bamboo chopstick on the rim of a copper pan, a single clean metallic ring that fades, isolated, no music, no voices", duration: 2, seed: 54}
- click: {prompt: "Bamboo chopsticks set down on a small ceramic plate, one soft dry click, isolated, no music, no voices", duration: 1, seed: 55}
- wipe: {prompt: "A folded cloth slowly wiping the inside of a metal pan, soft quiet rubbing, no music, no voices", duration: 3, seed: 56}
- roll_slide: {prompt: "A rolled heavy cloth on a wooden pole sliding along a wooden counter, one soft low slide, isolated, no music, no voices", duration: 1.5, seed: 57}
- dawn: {prompt: "A quiet Tokyo street at dawn after rain, slow water drips from an awning, a few distant sparrows, no traffic, no music, no voices", duration: 12, seed: 58}
- cloth: {prompt: "A heavy cotton curtain falling open and swaying on a wooden pole, soft cloth flap, isolated, no music, no voices", duration: 1.5, seed: 59}
+ rain: {prompt: "Heavy steady rain falling on a corrugated tin awning, heard from inside a small quiet room, metallic pattering, no music, no voices, no thunder", duration: 47, loop: true}
+ sizzle_loud: {prompt: "Beaten egg sizzling loudly and angrily in a very hot copper pan, spitting and crackling, close up, no music, no voices", duration: 5}
+ sizzle_soft: {prompt: "Egg sizzling very softly and steadily in a copper pan on a low flame, a gentle whisper, close up, no music, no voices", duration: 12}
+ ring: {prompt: "One light tap of a bamboo chopstick on the rim of a copper pan, a single clean metallic ring that fades, isolated, no music, no voices", duration: 2}
+ click: {prompt: "Bamboo chopsticks set down on a small ceramic plate, one soft dry click, isolated, no music, no voices", duration: 1}
+ wipe: {prompt: "A folded cloth slowly wiping the inside of a metal pan, soft quiet rubbing, no music, no voices", duration: 3}
+ roll_slide: {prompt: "A rolled heavy cloth on a wooden pole sliding along a wooden counter, one soft low slide, isolated, no music, no voices", duration: 1.5}
+ dawn: {prompt: "A quiet Tokyo street at dawn after rain, slow water drips from an awning, a few distant sparrows, no traffic, no music, no voices", duration: 12}
+ cloth: {prompt: "A heavy cotton curtain falling open and swaying on a wooden pole, soft cloth flap, isolated, no music, no voices", duration: 1.5}
cues:
- {at: 0.0, sound: rain, rel_db: -2, len: 39.5, fade: 0.8}
- {at: 3.0, sound: sizzle_soft, rel_db: -8, len: 3.0, fade: 0.2}
diff --git a/skills/make-an-anime/references/animation-audio-vfx.md b/skills/make-an-anime/references/animation-audio-vfx.md
index bf5d895..0c9e015 100644
--- a/skills/make-an-anime/references/animation-audio-vfx.md
+++ b/skills/make-an-anime/references/animation-audio-vfx.md
@@ -92,13 +92,17 @@ rel_db: -6
peak_db: -6
max_gain_db: 18
sounds:
- pop: {prompt: "One short dry cartoon cork pop, isolated, no music or voice", duration: 2, seed: 7}
+ pop: {prompt: "One short dry cartoon cork pop, isolated, no music or voice", duration: 2}
flatline: {tone: 1000, duration: 1}
cues:
- {at: 1.25, sound: pop, rel_db: -5, len: 0.25, fade: 0.03}
```
-Stable Audio SFX is paid; a steady tone is synthesized locally. SFX are cached by prompt, duration, seed and negative prompt. Mix trims leading silence, sets cue loudness relative to music in that window, caps gain and peak-limits. Use exact song times, not cut-relative times. Check dialogue intelligibility, silence jokes and audible but subordinate effects. The final mux can add codec peaks, so listen and measure the delivered file if close to full scale.
+ElevenLabs Sound Effects v3 is paid; a steady tone is synthesized locally. SFX use `elevenlabs/sound-effects/v3` and are cached by model, prompt, duration, `loop` (default false) and `prompt_influence` (default 0.3). Duration is 0.5–30 seconds; for longer ambience set `loop: true` to generate up to 30 seconds and repeat it locally to the requested length. The downloaded MP3 is retained and decoded to a 48 kHz stereo WAV for mixing.
+
+Legacy `seed` is ignored with a warning; legacy `negative` is appended to the prompt as “Avoid: …” (combined prompt limit: 450 characters). For a fresh take of identical input use `FORCE=1 NEW_REQUEST=1`.
+
+Mix trims leading silence, sets cue loudness relative to music in that window, caps gain and peak-limits. Use exact song times, not cut-relative times. Check dialogue intelligibility, silence jokes and audible but subordinate effects. The final mux can add codec peaks, so listen and measure the delivered file if close to full scale.
## Swift VFX and p5 light layers
diff --git a/skills/make-an-anime/references/testing.md b/skills/make-an-anime/references/testing.md
index f10414a..c75eaa8 100644
--- a/skills/make-an-anime/references/testing.md
+++ b/skills/make-an-anime/references/testing.md
@@ -40,6 +40,8 @@ LIVE_FAL=1 LIVE_SONG=/absolute/short-song.wav LIVE_LYRICS='known sung words' PRO
Supply `FAL_AI_API_KEY` via the environment for this explicit developer CLI test. The plugin's sensitive key is scoped to its MCP server, which does not expose the live test runner. Budget per run: one Sunburst generation, one Sunburst edit, one five-second H3 768P clip (a harness-only saving; real projects keep the 1080P default), one Whisper transcription, one Demucs separation and one SFX generation. Local composition/VFX adds no model charges. No automatic paid rerolls. Run only after accepting those costs. Output is retained for inspection. A provider failure ends the test with request evidence; resume/recover before paying again.
+For an SFX-only paid smoke, run `LIVE_FAL=1 PROFILE=sfx_live ruby scripts/mv.rb test` with `FAL_AI_API_KEY` injected into the environment. This submits one two-second ElevenLabs Sound Effects v3 request (approximately $0.004), fetches its OpenAPI schema, exercises the SFX download and WAV conversion, checks duration and non-silent PCM, and verifies cache reuse. Artifacts and request receipts remain in `scripts/tmp/sfx-live/`; repeated runs reuse them. This does not run the image/video live suite. Listen to the retained `output/pop/sounds/pop.wav` to assess whether the sound matches the prompt.
+
## Behavioral rehearsal for the Markdown workflow
Executable tests check document links/interfaces; they do not prove an agent will obey creative prose. Rehearse these scenarios using the skill with a disposable project. Paid calls require authorization; preserve authorization already given for a bounded regeneration:
diff --git a/skills/make-an-anime/scripts/lib/fal/models/elevenlabs_sfx.rb b/skills/make-an-anime/scripts/lib/fal/models/elevenlabs_sfx.rb
new file mode 100644
index 0000000..d0defe2
--- /dev/null
+++ b/skills/make-an-anime/scripts/lib/fal/models/elevenlabs_sfx.rb
@@ -0,0 +1,20 @@
+require_relative "base"
+
+module Fal
+ module Models
+ # https://fal.ai/models/elevenlabs/sound-effects/v3/api
+ class ElevenlabsSfx < Base
+ ENDPOINT = "elevenlabs/sound-effects/v3".freeze
+
+ def generate(prompt:, duration: 2, prompt_influence: 0.3, loop: false)
+ raise ArgumentError, "SFX prompt must contain 1–450 characters" unless prompt.is_a?(String) && !prompt.strip.empty? && prompt.length <= 450
+ raise ArgumentError, "SFX duration must be 0.5–30 seconds" unless duration.is_a?(Numeric) && (0.5..30).cover?(duration)
+ raise ArgumentError, "SFX prompt_influence must be 0–1" unless prompt_influence.is_a?(Numeric) && (0..1).cover?(prompt_influence)
+ raise ArgumentError, "SFX loop must be true or false" unless [true, false].include?(loop)
+
+ call(text: prompt, duration_seconds: duration, prompt_influence: prompt_influence,
+ loop: loop, output_format: "mp3_44100_128")
+ end
+ end
+ end
+end
diff --git a/skills/make-an-anime/scripts/lib/fal/models/stable_audio_sfx.rb b/skills/make-an-anime/scripts/lib/fal/models/stable_audio_sfx.rb
deleted file mode 100644
index d460884..0000000
--- a/skills/make-an-anime/scripts/lib/fal/models/stable_audio_sfx.rb
+++ /dev/null
@@ -1,14 +0,0 @@
-require_relative "base"
-
-module Fal
- module Models
- # https://fal.ai/models/fal-ai/stable-audio-3/small/sfx/text-to-audio: short sound effects from a text prompt.
- class StableAudioSfx < Base
- ENDPOINT = "fal-ai/stable-audio-3/small/sfx/text-to-audio".freeze
-
- def generate(prompt:, **opts)
- call(prompt: prompt, **opts)
- end
- end
- end
-end
diff --git a/skills/make-an-anime/scripts/lib/media/sfx.rb b/skills/make-an-anime/scripts/lib/media/sfx.rb
index d399c8c..6ddfd15 100644
--- a/skills/make-an-anime/scripts/lib/media/sfx.rb
+++ b/skills/make-an-anime/scripts/lib/media/sfx.rb
@@ -8,12 +8,12 @@ module Media
# Sound effects over a finished video (the full-song preview). prompts/<name>/sfx.yml has
# source: / out: the video to add sounds to (only read) and the new file
# rel_db: default level of a cue, in dB relative to the music's loudness under it (−4 ≈ clearly audible, still under it)
-# peak_db / max_gain_db: caps: a cue never peaks above peak_db (−6) nor gets boosted more than max_gain_db (+18)
- # sounds: { name => { prompt:, duration:, seed:, negative: } } generated with fal stable-audio-3 SFX (Fal::Models::StableAudioSfx),
+ # peak_db / max_gain_db: caps: a cue never peaks above peak_db (−6) nor gets boosted more than max_gain_db (+18)
+ # sounds: { name => { prompt:, duration:, loop:, prompt_influence: } } generated with ElevenLabs Sound Effects v3,
# or { tone: Hz, duration: }: a steady sine synthesized with ffmpeg (a monitor flatline; the model only makes rhythmic beeps)
# cues: [{ at: song s, sound:, rel_db:, rate: (pitch/speed), len: (s, cut + fade), fade: (s) }, ...]
# Sounds are cached in output/<name>/sounds/<sound>.wav with a sidecar json; a sound is generated again only when its prompt,
- # duration or seed changed (or FORCE=1 with ONLY=a,b). scripts/sfx_mix.py trims their leading silence, sets each cue's level
+ # model, duration, loop or prompt influence changed (or FORCE=1 with ONLY=a,b). scripts/sfx_mix.py trims their leading silence, sets each cue's level
# against the music in its window and limits the sum; the video stream is copied untouched.
class Sfx < Shell
ROOT = File.expand_path("../..", __dir__)
@@ -41,8 +41,13 @@ module Media
want = if spec["tone"]
{ "tone" => spec["tone"], "duration" => spec.fetch("duration", 2) }
else
- { "prompt" => spec.fetch("prompt"), "duration" => spec.fetch("duration", 2), "seed" => spec["seed"],
- "negative_prompt" => spec["negative"] || config["negative"] }
+ prompt = spec.fetch("prompt")
+ negative = spec["negative"] || config["negative"]
+ prompt = "#{prompt}. Avoid: #{negative}" unless negative.to_s.strip.empty?
+ warn "[sfx] #{sound}: seed is ignored by ElevenLabs SFX v3" if spec.key?("seed")
+ { "model" => Fal::Models::ElevenlabsSfx.endpoint, "prompt" => prompt,
+ "duration" => spec.fetch("duration", 2), "loop" => spec.fetch("loop", false),
+ "prompt_influence" => spec.fetch("prompt_influence", 0.3) }
end
meta = File.join(dir, "sounds", "#{sound}.json")
have = File.exist?(meta) && File.exist?(wav(sound)) ? JSON.parse(File.read(meta)) : {}
@@ -57,12 +62,26 @@ module Media
end
warn "[sfx] #{sound} generating…"
client ||= Fal::Client.new
- model ||= Fal::Models::StableAudioSfx.new(client: client)
- res = model.generate(prompt: want["prompt"], duration: want["duration"], seed: want["seed"],
- negative_prompt: want["negative_prompt"], output_format: "wav")
- client.download(res.output.dig("audio", "url"), wav(sound))
- File.write(meta, JSON.pretty_generate(want.merge("request_id" => res.request_id, "used_seed" => res.output["seed"],
- "url" => res.output.dig("audio", "url"))))
+ model ||= Fal::Models::ElevenlabsSfx.new(client: client)
+ duration = want["duration"]
+ generated_duration = want["loop"] == true && duration.is_a?(Numeric) ? [duration, 30].min : duration
+ res = model.generate(prompt: want["prompt"], duration: generated_duration,
+ loop: want["loop"], prompt_influence: want["prompt_influence"])
+ url = res.output.fetch("audio").fetch("url")
+ original = File.join(dir, "sounds", "#{sound}.mp3")
+ client.download(url, original)
+ # Decode before looping: repeated MP3 encoder padding otherwise shortens the final PCM duration.
+ pending = File.join(dir, "sounds", "#{sound}.pending.wav")
+ run("ffmpeg", "-y", "-v", "error", "-i", original, "-t", generated_duration,
+ "-ar", "48000", "-ac", "2", "-c:a", "pcm_s16le", pending)
+ if want["loop"]
+ looped = File.join(dir, "sounds", "#{sound}.looped.wav")
+ run("ffmpeg", "-y", "-v", "error", "-stream_loop", "-1", "-i", pending,
+ "-t", duration, "-c:a", "pcm_s16le", looped)
+ File.rename(looped, pending)
+ end
+ File.rename(pending, wav(sound))
+ File.write(meta, JSON.pretty_generate(want.merge("request_id" => res.request_id, "url" => url)))
sound
end
end
diff --git a/skills/make-an-anime/scripts/lib/toolkit/episode.rb b/skills/make-an-anime/scripts/lib/toolkit/episode.rb
index d54f729..ad1f91e 100644
--- a/skills/make-an-anime/scripts/lib/toolkit/episode.rb
+++ b/skills/make-an-anime/scripts/lib/toolkit/episode.rb
@@ -229,14 +229,14 @@ module Toolkit
def sfx
<<~YAML
- # #{@run} sound effects over the finished scene (Stable Audio SFX, paid; tones are local). Times in scene seconds.
+ # #{@run} sound effects over the finished scene (ElevenLabs Sound Effects v3, paid; tones are local). Times in scene seconds.
source: output/#{@run}/final_overlay.mp4
out: output/#{@run}/final_sfx.mp4
rel_db: -8
peak_db: -6
max_gain_db: 18
sounds:
- ambience: {prompt: "PLACE ambience, no music, no voices", duration: 10, seed: 1}
+ ambience: {prompt: "PLACE ambience, no music, no voices", duration: 10}
cues:
- {at: 0.0, sound: ambience, rel_db: -12, len: #{seconds}, fade: 0.8}
YAML
diff --git a/skills/make-an-anime/scripts/lib/toolkit/test_runner.rb b/skills/make-an-anime/scripts/lib/toolkit/test_runner.rb
index afda1c3..ffb9696 100644
--- a/skills/make-an-anime/scripts/lib/toolkit/test_runner.rb
+++ b/skills/make-an-anime/scripts/lib/toolkit/test_runner.rb
@@ -3,19 +3,22 @@ module Toolkit
class TestRunner
def run
profile = ENV.fetch("PROFILE", "all")
- raise "PROFILE must be all, core, media, swift, live or seedance_live" unless %w[all core media swift live seedance_live].include?(profile)
+ raise "PROFILE must be all, core, media, swift, live, seedance_live or sfx_live" unless %w[all core media swift live seedance_live sfx_live].include?(profile)
if profile == "live"
raise "Paid tests require LIVE_FAL=1, LIVE_SONG=/path/to/song.wav and LIVE_LYRICS='recognizable words'" unless ENV["LIVE_FAL"] == "1" && File.file?(ENV["LIVE_SONG"].to_s) && !ENV["LIVE_LYRICS"].to_s.strip.empty?
end
if profile == "seedance_live"
raise "Paid Seedance smoke requires LIVE_FAL=1 and LIVE_IMAGE=/path/to/reference.png" unless ENV["LIVE_FAL"] == "1" && File.file?(ENV["LIVE_IMAGE"].to_s)
end
+ if profile == "sfx_live"
+ raise "Paid SFX smoke requires LIVE_FAL=1" unless ENV["LIVE_FAL"] == "1"
+ end
# Specs chdir into temporary workspaces; pin the toolkit's own venv so Python helpers keep numpy/PIL.
venv = File.expand_path(".venv/bin/python3")
ENV["MV_PYTHON"] ||= venv if File.file?(venv)
args = [RbConfig.ruby, "-S", "rspec", "spec", "--format", "documentation"]
args += profile == "all" ? ["--tag", "~live"] : ["--tag", profile]
- args += ["--tag", "~seedance_live"] if profile == "live"
+ args += ["--tag", "~seedance_live", "--tag", "~sfx_live"] if profile == "live"
ok = system(*args)
raise "RSpec failed for PROFILE=#{profile}; see failures above" unless ok
end
diff --git a/skills/make-an-anime/scripts/spec/live_spec.rb b/skills/make-an-anime/scripts/spec/live_spec.rb
index a1bc20c..16f8dd7 100644
--- a/skills/make-an-anime/scripts/spec/live_spec.rb
+++ b/skills/make-an-anime/scripts/spec/live_spec.rb
@@ -35,8 +35,8 @@ RSpec.describe "Paid Fal and local finishing", :live do
expect(animation.input[:resolution]).to eq("768P")
expect(ff.summary(clip).dig(:video,:h)).to be >= 700
expect(ff.duration(clip)).to be_within(0.3).of(5)
- sfx = Fal::Models::StableAudioSfx.new(client: client).generate(prompt: "One short cartoon pop, isolated, no voice or music", duration: 2)
- sound = client.download(sfx.output.fetch("audio").fetch("url"), file("pop.wav"))
+ sfx = Fal::Models::ElevenlabsSfx.new(client: client).generate(prompt: "One short cartoon pop, isolated, no voice or music", duration: 2)
+ sound = client.download(sfx.output.fetch("audio").fetch("url"), file("pop.mp3"))
expect(ff.duration(sound)).to be > 0
json(file("requests.json"), [generated, edited, transcript, stem, animation, sfx].map { |r| {request_id: r.request_id, input: r.input} })
# Key the actual model clip, place at a controlled location, overlay and finish through Swift.
diff --git a/skills/make-an-anime/scripts/spec/sfx_live_spec.rb b/skills/make-an-anime/scripts/spec/sfx_live_spec.rb
new file mode 100644
index 0000000..af86829
--- /dev/null
+++ b/skills/make-an-anime/scripts/spec/sfx_live_spec.rb
@@ -0,0 +1,32 @@
+require_relative "spec_helper"
+
+RSpec.describe "Paid ElevenLabs SFX smoke", :live, :sfx_live do
+ it "generates a real effect through Media::Sfx, decodes audible WAV and reuses its cache" do
+ raise "Set LIVE_FAL=1 to authorize one two-second SFX generation" unless ENV["LIVE_FAL"] == "1"
+ root = File.join(RT, "tmp", "sfx-live")
+ FileUtils.mkdir_p(root)
+ stub_const("Media::Sfx::ROOT", root)
+ Dir.chdir(root) do
+ FileUtils.mkdir_p(["docs", "prompts/pop"])
+ File.write("docs/PLAN.md", "SFX smoke: one two-second ElevenLabs Sound Effects v3 cartoon pop, approximately $0.004. Decode to WAV, measure signal and verify cache reuse. No paid rerolls.\n")
+ Workflow::Approval.new.record!("LIVE_FAL=1 supplied by test operator")
+ json_config = {"sounds" => {"pop" => {"prompt" => "One short dry cartoon cork pop, isolated, no voice or music", "duration" => 2}},
+ "cues" => [{"sound" => "pop", "at" => 0}]}
+ File.write("prompts/pop/sfx.yml", YAML.dump(json_config))
+ Fal::OpenAPI.fetch(Fal::Models::ElevenlabsSfx.endpoint)
+ sfx = Media::Sfx.new("pop")
+ sfx.generate
+ summary = ff.summary(sfx.wav("pop"))
+ expect(File.binread(sfx.wav("pop"), 4)).to eq("RIFF")
+ expect(summary[:duration]).to be_within(0.15).of(2)
+ expect(summary[:audio]).to eq(codec: "pcm_s16le", rate: 48000, channels: 2)
+ pcm = ff.run("ffmpeg", "-v", "error", "-i", sfx.wav("pop"), "-f", "s16le", "-ac", "1", "-", quiet: true).unpack("s<*")
+ rms = Math.sqrt(pcm.sum { |v| v.to_f ** 2 } / pcm.length) / 32768
+ expect(rms).to be > 0.0001
+ expect(sfx.generate(client: double("cache must not call provider"))).to eq([])
+ json("verification.json", summary: summary, rms: rms, cached: true,
+ generation: JSON.parse(File.read(File.join(sfx.dir, "sounds/pop.json"))))
+ puts "SFX artifacts: #{root}"
+ end
+ end
+end
diff --git a/skills/make-an-anime/scripts/spec/sfx_spec.rb b/skills/make-an-anime/scripts/spec/sfx_spec.rb
new file mode 100644
index 0000000..d7e0045
--- /dev/null
+++ b/skills/make-an-anime/scripts/spec/sfx_spec.rb
@@ -0,0 +1,61 @@
+require_relative "spec_helper"
+
+RSpec.describe Fal::Models::ElevenlabsSfx, :core do
+ it "maps toolkit arguments to the v3 schema without unsupported legacy fields" do
+ client = double("Fal client")
+ expect(client).to receive(:run).with("elevenlabs/sound-effects/v3", {
+ text: "One dry pop", duration_seconds: 2, prompt_influence: 0.3,
+ loop: false, output_format: "mp3_44100_128"
+ }).and_return(["request-1", {"audio" => {"url" => "https://fixture.test/pop.mp3"}}])
+ expect(described_class.new(client: client).generate(prompt: "One dry pop").request_id).to eq("request-1")
+ end
+
+ it "rejects invalid prompts, durations and controls before a paid call" do
+ model = described_class.new(client: double("unused client"))
+ [{prompt: ""}, {prompt: "a" * 451}, {duration: 0.4}, {duration: 31},
+ {prompt_influence: 1.1}, {loop: "true"}].each do |invalid|
+ expect { model.generate(**{prompt: "A bell"}.merge(invalid)) }.to raise_error(ArgumentError)
+ end
+ end
+end
+
+RSpec.describe Media::Sfx, :media do
+ it "decodes MP3 into WAV, replaces legacy cache, caches v3 output and repeats long ambience" do
+ with_workspace do |root|
+ stub_const("Media::Sfx::ROOT", root)
+ FileUtils.mkdir_p("prompts/test")
+ spec = {"prompt" => "Rain", "duration" => 2, "seed" => 7, "negative" => "voices"}
+ config = {"sounds" => {"rain" => spec}, "cues" => [{"sound" => "rain", "at" => 0}]}
+ File.write("prompts/test/sfx.yml", YAML.dump(config))
+ sfx = described_class.new("test")
+ tone(sfx.wav("rain"), duration: 2)
+ meta = File.join(sfx.dir, "sounds/rain.json")
+ json(meta, prompt: "Rain", duration: 2, seed: 7, negative_prompt: "voices")
+ encoded = File.join(root, "fixture.mp3")
+ ff.run("ffmpeg", "-y", "-v", "error", "-i", sfx.wav("rain"), encoded)
+ inputs = []
+ client = double("Fal client")
+ allow(client).to receive(:run) do |endpoint, input|
+ expect(endpoint).to eq("elevenlabs/sound-effects/v3")
+ inputs << input
+ ["request-#{inputs.length}", {"audio" => {"url" => "https://fixture.test/rain.mp3"}}]
+ end
+ allow(client).to receive(:download) { |_url, path| FileUtils.cp(encoded, path); path }
+
+ expect(sfx.generate(client: client)).to eq(["rain"])
+ expect(inputs.first[:text]).to eq("Rain. Avoid: voices")
+ expect(inputs.first.keys).not_to include(:seed, :negative_prompt, :duration, :prompt)
+ expect(File.binread(sfx.wav("rain"), 4)).to eq("RIFF")
+ expect(ff.summary(sfx.wav("rain"))[:audio]).to eq(codec: "pcm_s16le", rate: 48000, channels: 2)
+ expect(sfx.generate(client: client)).to eq([])
+ expect(inputs.length).to eq(1)
+ expect(JSON.parse(File.read(meta))["model"]).to eq("elevenlabs/sound-effects/v3")
+
+ sfx.sounds["rain"].merge!("duration" => 47, "loop" => true, "prompt_influence" => 0.5)
+ expect(sfx.generate(client: client)).to eq(["rain"])
+ expect(inputs.last).to include(duration_seconds: 30, loop: true, prompt_influence: 0.5)
+ expect(ff.duration(sfx.wav("rain"))).to be_within(0.01).of(47)
+ expect(sfx.generate(client: client)).to eq([])
+ end
+ end
+end