Ears for coding agents: let AI hear, review and make sound. CPU-only, open source.
Launch profile + critique: evaluation
Generated by eval/launch/run_eval.py (numbers only; the films are local reference files of the author, never copied). Profile = earshot/data/launch_profile.json, built from the 7 references below.
Commands
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_1,ref_2,ref_3 --fresh --measure-only
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_4,ref_5,ref_6,ref_7,clawd_real,clawd_synth --fresh --measure-only
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --build # profile, LOO, Clawd, this file
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg base,quiet,hot,mono,muffled,late,deadair,fadeout,hole,squashed
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sfx_only,amb_only,synth_bed
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sine,pink
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py # rewrite this file from the caches
What was tuned on what
- Detector thresholds (salient hit: >= 6 dB jump of the 100 ms level within 80 ms, landing within 8 LU of integrated, one hit per run, timed at the attack; visual event: frame-difference peak >= 4x the median of the surrounding second or >= 0.15, now earshot.video.diff_events; stop-down: >= 10 dB dip for 0.15-1.5 s) were chosen by looking at plots of all 9 films (7 refs + 2 Clawd), at the Clawd mix-report placements (hits) and at ffmpeg scene cuts (visual events). The Clawd real soundtrack is therefore partly tuning data for the hit detector.
- Music threshold (AudioSet Music >= 0.3 per 5 s window) was set after looking at the per-window probabilities of all 9 films (music under voice or SFX scored 0.4-0.7, one sound-design stretch 0.1-0.15).
- Changed after the first leave-one-out run (first build; not a blind test of these choices): true peak went from critical to major and is judged against 0 dBTP for decoded lossy files (the -1 dBTP master rule failed all 7 AAC-decoded references); a critical rule missed by <= 2 tolerances counts as major; the metric max_gap_s was added; the music_frac tolerance went from 0.1 to 0.05 after an SFX-only stem (music in 42 % of windows) came out as only a near miss.
- Changed after the independent review, before the known-bad set below was first run: p_real uses the realness model's own thresholds (ok at its 'real or pro-produced' threshold, critical below 0.5, minor in between; no near-miss downgrade); a density range that would reach 0 keeps a floor at half the sparsest reference (salient hits >= 0.05 /s); a mono mix (identical channels) counts as -40 dB stereo width instead of n.a.; files under 10 s get no loudness-shape rows; brightness_hz replaces the power-weighted centroid (info only). The known-bad expectations were written into run_eval.py before its first run; nothing was changed after seeing its results.
- Not adjusted after seeing results: the other tolerances, the weights of the other metrics and the verdict thresholds (90 / 75 / 50 %).
- Ranges are data: min-max of the reference films widened by each metric's tolerance. Loudness and true peak use web-delivery practice (-16..-9 LUFS; <= -1 dBTP for a master, <= 0 dBTP for decoded AAC / MP3) instead, so references can fail them.
Per-film measurements
| film | dur s | LUFS | dBTP | LRA | music | speech | onsets/s | hits/s | vis events/s | events w/ sound (chance) | audio pass s (cold) | picture pass s | AudioSet pass s (cold, first build) | warm s |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ref_1 | 93.1 | -12.1 | 0.1 | 1.7 | 100% | 95% | 5.87 | 1.56 | 0.3 | 89% (90%) | 49.8 | 0.0 | 144.4 | 0.4 |
| ref_2 | 28.9 | -14.8 | 1.0 | 3.6 | 100% | 83% | 5.98 | 1.21 | 0.31 | 67% (89%) | 9.7 | 0.0 | 37.0 | 0.7 |
| ref_3 | 20.5 | -10.4 | 0.6 | 6.4 | 100% | 100% | 3.81 | 1.66 | 0.83 | 88% (68%) | 5.6 | 0.1 | 23.0 | 0.7 |
| ref_4 | 32.3 | -14.8 | -0.8 | 7.0 | 100% | 0% | 8.22 | 1.18 | 1.3 | 100% (99%) | 18.1 | 0.0 | 58.6 | 0.3 |
| ref_5 | 29.5 | -22.2 | 1.2 | 12.2 | 67% | 0% | 4.03 | 1.25 | 0.54 | 50% (48%) | 4.0 | 1.9 | 41.6 | 0.2 |
| ref_6 | 49.3 | -17.4 | 0.1 | 5.5 | 100% | 100% | 5.66 | 1.3 | 0.63 | 81% (86%) | 5.8 | 14.2 | 54.5 | 0.2 |
| ref_7 | 41.8 | -7.3 | 2.0 | 3.3 | 89% | 0% | 2.46 | 0.1 | 1.6 | 60% (44%) | 5.8 | 5.1 | 98.4 | 0.3 |
| clawd_real | 60.0 | -15.0 | -1.4 | 10.4 | 100% | 0% | 2.48 | 0.27 | 0.43 | 62% (37%) | 8.0 | 0.0 | 78.9 | 0.2 |
| clawd_synth | 60.0 | -14.0 | -2.1 | 7.2 | 100% | 0% | 2.1 | 0.38 | 0.43 | 54% (43%) | 8.1 | 25.4 | 76.3 | 0.6 |
Audio pass = decode + features.measure(light) + librosa onsets / beats + loudness curves, timed cold in this round (this module's cache deleted; the core decode cache was cold too after its v2 bump). Picture pass = earshot.video.frame_diff (core cache, shared with timeline / sheet): 0.0 means another module had already decoded that film; ref_6 (4K) and clawd_synth were cold. AudioSet = 5 s windows through AST, measured cold in the first build (same code and cache key, so not re-run): ~4-9 s per 5 s window on this laptop under load from 3-4 other model jobs; earshot critique --fast skips it and realness. Warm = everything cached. EARSHOT_THREADS=2.
Leave-one-out: each reference vs a profile of the other six
| film | verdict | weighted score | scored metrics in range | reference-range metrics in range | out of range |
|---|---|---|---|---|---|
| ref_1 | close | 75% | 17/22 | 16/20 | true_peak high, lra low, first_sound_s high, end_silence_s high, side_to_mid_db low |
| ref_2 | launch-grade | 94% | 21/22 | 20/20 | true_peak high |
| ref_3 | close | 83% | 17/22 | 16/20 | true_peak high, end_lu low, band_sub low, band_lowmid high, band_air low |
| ref_4 | close | 89% | 19/22 | 17/20 | onset_rate high, intro_lu low, band_mid low |
| ref_5 | needs work | 56% | 14/22 | 14/20 | lufs_i low, true_peak high, music_frac low, cuts_with_sound low, lra high, max_gap_s high, band_air high, side_to_mid_db high |
| ref_6 | close | 86% | 19/22 | 19/20 | lufs_i low, true_peak high, intro_lu high |
| ref_7 | close | 77% | 18/22 | 18/20 | lufs_i high, true_peak high, onset_rate low, hit_rate low |
Total: 125/154 scored metric checks in range (81 %); reference-range metrics only: 120/140 (86 %). For exchangeable data a held-out film falls outside the min-max of six others with probability 2/7 (29 %) per continuous metric before tolerance, so ~71 % is the no-overfit baseline; the tolerance adds a little. p_real is not part of this check (the realness model was trained on these films).
In-sample (each reference vs the full 7-film profile): every reference-range metric is in range by construction; the only failures are web-practice rules: ref_1: true_peak high; ref_2: true_peak high; ref_3: true_peak high; ref_5: lufs_i low, true_peak high; ref_6: lufs_i low, true_peak high; ref_7: lufs_i high, true_peak high.
Per metric (7 leave-one-out checks each)
| metric | weight | ok | low | high | n.a. |
|---|---|---|---|---|---|
| lufs_i | critical | 4 | 2 | 1 | 0 |
| true_peak | major | 1 | 0 | 6 | 0 |
| music_frac | critical | 6 | 1 | 0 | 0 |
| cuts_with_sound | major | 6 | 1 | 0 | 0 |
| lra | major | 5 | 1 | 1 | 0 |
| crest_lu | major | 7 | 0 | 0 | 0 |
| first_sound_s | major | 6 | 0 | 1 | 0 |
| end_silence_s | major | 6 | 0 | 1 | 0 |
| max_gap_s | major | 6 | 0 | 1 | 0 |
| final_lu | major | 7 | 0 | 0 | 0 |
| onset_rate | major | 5 | 1 | 1 | 0 |
| hit_rate | major | 6 | 1 | 0 | 0 |
| intro_lu | minor | 5 | 1 | 1 | 0 |
| end_lu | minor | 6 | 1 | 0 | 0 |
| fade_s | minor | 7 | 0 | 0 | 0 |
| band_sub | minor | 6 | 1 | 0 | 0 |
| band_low | minor | 7 | 0 | 0 | 0 |
| band_lowmid | minor | 6 | 0 | 1 | 0 |
| band_mid | minor | 6 | 1 | 0 | 0 |
| band_presence | minor | 7 | 0 | 0 | 0 |
| band_air | minor | 5 | 1 | 1 | 0 |
| side_to_mid_db | minor | 5 | 1 | 1 | 0 |
Sensitivity: known-bad mixes
Each row is one defect made on purpose (ffmpeg, scratch files in out/launch/neg/), critiqued against the 7-film profile. 'Should flag' was written down before the first run; 'caught' = at least one of those metrics came out low / high. Rows without models skip AudioSet and realness (partial verdicts); the control is the untouched real soundtrack.
| mix | what | should flag | caught | verdict | weighted score | all flagged rows | p_real | s (critique, cold) |
|---|---|---|---|---|---|---|---|---|
| base | control: the real Clawd soundtrack untouched | - | - | close (partial) | 87% | crest_lu high (major); first_sound_s high (major) | - | 27.5 |
| quiet | unmastered: 12 dB quieter | lufs_i | lufs_i | needs work (partial) | 77% | lufs_i low (critical); crest_lu high (major); first_sound_s high (major) | - | 9.0 |
| hot | 12 dB hotter, clipped by the 16-bit file | lufs_i, true_peak | lufs_i, true_peak | needs work (partial) | 77% | lufs_i high (critical); true_peak high (major); first_sound_s high (major) | - | 8.8 |
| mono | folded to mono | side_to_mid_db | side_to_mid_db | close (partial) | 77% | lufs_i low (major); crest_lu high (major); first_sound_s high (major); side_to_mid_db low (minor) | - | 9.5 |
| muffled | low-passed at 3 kHz (12 dB/oct) | band_presence, band_air | band_presence, band_air | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); band_presence low (minor); band_air low (minor) | - | 10.0 |
| late | 2.5 s of silence before the start | first_sound_s | first_sound_s | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); onset_rate low (major) | - | 13.8 |
| deadair | 4 s of silence after the end | end_silence_s | end_silence_s | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); end_silence_s high (major) | - | 14.4 |
| fadeout | 12 s fade-out | fade_s, final_lu, end_lu | fade_s, final_lu | close (partial) | 77% | crest_lu high (major); first_sound_s high (major); final_lu low (major); fade_s high (minor) | - | 12.6 |
| hole | 2.5 s muted mid-film | max_gap_s | max_gap_s | needs work (partial) | 74% | crest_lu high (major); first_sound_s high (major); max_gap_s high (major); onset_rate low (major) | - | 13.2 |
| squashed | over-compressed: 20:1 from -36 dB + limiter | lra, crest_lu | no | launch-grade (partial) | 90% | true_peak high (major); side_to_mid_db high (minor) | - | 14.6 |
| sfx_only | the SFX stem alone, with the picture | music_frac | music_frac | needs work | 77% | lufs_i low (critical); music_frac low (critical); band_sub low (minor); band_low low (minor); band_mid high (minor) | 0.787 | 194.8 |
| amb_only | the ambience stem alone | music_frac, onset_rate, hit_rate | music_frac | needs work | 68% | lufs_i low (critical); music_frac low (critical); crest_lu high (major); band_sub low (minor); band_low low (minor); band_mid high (minor); band_presence high (minor) | 0.882 | 90.6 |
| synth_bed | the retired code-synthesized music bed | p_real | p_real | not launch-grade | 49% | lufs_i low (critical); p_real low (critical); music_frac low (critical); crest_lu high (major); first_sound_s high (major); onset_rate low (major); band_sub low (minor); band_low low (minor); band_mid high (minor); side_to_mid_db high (minor) | 0.095 | 43.7 |
| sine | 30 s 440 Hz sine | p_real, music_frac, onset_rate | p_real, music_frac, onset_rate | not launch-grade | 35% | lufs_i low (critical); p_real low (critical); music_frac low (critical); lra low (major); crest_lu low (major); onset_rate low (major); hit_rate low (major); band_sub low (minor); band_low low (minor); band_lowmid high (minor); band_mid low (minor); band_presence low (minor); band_air low (minor); side_to_mid_db low (minor) | 0.03 | 47.5 |
| pink | 30 s pink noise | p_real, music_frac, onset_rate | p_real, music_frac, onset_rate | not launch-grade | 46% | lufs_i low (critical); p_real low (critical); music_frac low (critical); lra low (major); crest_lu low (major); onset_rate low (major); hit_rate low (major); band_mid high (minor); band_presence high (minor); side_to_mid_db low (minor) | 0.158 | 22.7 |
Caught: 13 of 14 known-bad mixes have the expected metric flagged (missed: squashed). Verdicts: launch-grade 1, close 5, needs work 5, not launch-grade 3 (control: close). A single structural defect on a mix that is already close usually stays close: the verdict is coarse, the flagged rows and FIX FIRST actions are the signal. Over-compression is not a defect by these references: two of them (LRA 1.7 / 3.3 LU, loudest 3 s only 1.1-1.2 LU above integrated) are as dense as the squashed mix.
The Clawd film: real soundtrack vs the retired synthesized soundtrack
Reading: the scorecard rates the real soundtrack close (90%) and the synthesized one needs work (80%). The structural metrics separate them only a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has one loudness peak that sticks out. The difference a listener hears most - synthesized timbre - is not structural; it is what p_real measures: real 0.94 vs synth 0.33 (earshot.realness, verdict thresholds lo 0.4 / hi 0.77 at this run); below 0.5 the p_real row is critical, which is what moves the synth version's verdict. Caveat: the realness model (another module) was trained partly on these 7 launch films, on the Clawd project's real asset library and on the retired synthesized stems (eval/realness/), so p_real on the Clawd films is not an independent test.
clawd_real: Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)
Verdict close, weighted score 90%, 21/23 in range, 0 critical / 2 major issues; critique call 4.6 s with the measurement passes cached (realness runs inside it).
- [major] nothing reaches body level until 0:01.01 (before that the sound sits around -42 LUFS, 27 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00
- [major] the loudest 3 s (around 0:41.50, -10.4 LUFS) stand 4.6 LU above the integrated level (refs 1.1 LU .. 3.5 LU): the rest of the film plays quiet next to it; lift the body (quietest stretch 0:06.20-0:09.20 at -24.5 LUFS short-term) or tame that peak
- tip: 5 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.30, 0:00.80, 0:01.30, 0:30.00, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down
- tip: no stop-downs: 4 of 7 references drop the sound out for 0.2-1 s right before a big hit; try one before the biggest visual change at 0:20.00
- realness: Clawd_1080p60.mp4: real or pro-produced (p_real 0.94, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 0 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score +2.9. No window leans synthesized (lowest p_real 0.76); that does not rule out a code-made layer under a louder real mix (most are missed): judge the source files and stems.
clawd_synth: the same film with the retired code-synthesized soundtrack
Verdict needs work, weighted score 80%, 19/23 in range, 1 critical / 2 major issues; critique call 4.6 s with the measurement passes cached (realness runs inside it).
- [critical] realness: likely code-synthesized (p_real 0.33, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:56.00-1:00.00 (0.05): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with
earshot realness stem.wav(a finished mix is harder to judge than its sources) - [major] the final moment sits 7.5 LU below the body: end on a logo sting / hit at the last visual event (0:59.00) instead of fading away (refs -2.2 LU .. 5.6 LU)
- [major] sparse: 2.1 onsets/s (refs 2.46 /s .. 8.22 /s): use music with more rhythmic drive, or add UI foley (clicks, typing, ticks) on the on-screen actions
- [minor] 4.4 s fade-out (refs 0.08 s .. 2.54 s): end on a sting with a short ring-out, or cut hard on the last picture cut
- tip: 8 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.80, 0:01.30, 0:28.33, 0:28.98, 0:30.00, 0:33.00, 0:40.88, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down
- realness: Clawd_1080p60_synth-audio.mp4: likely code-synthesized (p_real 0.33, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 14 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score -0.5. Leaning synthesized: 0:00.00-0:48.00, 0:52.00-1:00.00; 1 at or below the 4 s-window threshold 0.17 (code-synthesized): 0:56.00-1:00.00.
Hit / onset timing vs ground truth (tuning data: the Clawd real soundtrack)
The mix report lists 264 sample placements (every key tick, pop, whoosh, hit, music motif note ...). earshot found 16 salient hits and 149 onsets in the mixed film audio; 11 of the 16 hits sit within 2 ms of a placement (median distance 1.0 ms).
| tolerance | hits on a placement (chance) | onsets on a placement | placements with an onset |
|---|---|---|---|
| +-20 ms | 75% (10%) | 61% | 62% |
| +-50 ms | 88% (25%) | 75% | 78% |
| +-100 ms | 88% (39%) | 79% | 83% |
Chance = the same hits shifted off the placements by fixed offsets. Hit and onset times are moved to the steepest rise of the 2 ms level (attack), which is how a mixer aligns a sample.
Visual-event detector (earshot.video frame difference) vs ffmpeg scene cuts
Scene cuts = video.cuts(path, method='scene') (ffmpeg scene score 0.3). 'Same as first build' = the core video.frame_diff signal is bit-identical to this module's first-build picture pass, so moving the detector into core changed no number.
| film | scene cuts | visual events | share of scene cuts also found (+-1 frame) | same as first build |
|---|---|---|---|---|
| ref_1 | 0 | 28 | n.a. (< 5 scene cuts) | True |
| ref_2 | 2 | 9 | n.a. (< 5 scene cuts) | True |
| ref_3 | 15 | 17 | 1.0 | True |
| ref_4 | 17 | 42 | 0.82 | True |
| ref_5 | 2 | 16 | n.a. (< 5 scene cuts) | True |
| ref_6 | 11 | 31 | 1.0 | True |
| ref_7 | 40 | 67 | 0.8 | True |
| clawd_real | 10 | 26 | 1.0 | True |
| clawd_synth | 10 | 26 | 1.0 | True |