Ears for coding agents: let AI hear, review and make sound. CPU-only, open source.
earshot 0.1.0: ears for coding agents
Leonxlnx108763c62648
diff --git a/.gitattributes b/.gitattributesnew file mode 100644index 0000000..6c4ae40--- /dev/null+++ b/.gitattributes@@ -0,0 +1,3 @@+* text=auto eol=lf+*.png binary+*.npz binarydiff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..7abc409--- /dev/null+++ b/.gitignore@@ -0,0 +1,33 @@+.venv/+.cache/+__pycache__/+*.pyc+*.egg-info/+build/+dist/+out/+.pytest_cache/+# never commit media: reference videos, downloaded recordings, renders+*.mp4+*.mov+*.wav+*.flac+*.ogg+*.mp3+*.m4a+*.aac+!tests/fixtures/*.wav+# local secrets+.env++# local evaluation data locations (see eval/README.md) and data+eval/local_paths.json+eval/data/++# demo media and scratch (the demo READMEs and scripts may be committed; renders, plots and logs stay local)+demo/**/*.png+demo/**/*.jpg+demo/**/*.npy+demo/**/*.log+demo/**/*.err+*.npydiff --git a/LICENSE b/LICENSEnew file mode 100644index 0000000..cfa96d6--- /dev/null+++ b/LICENSE@@ -0,0 +1,21 @@+MIT License++Copyright (c) 2026 Leon Lin++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to deal+in the Software without restriction, including without limitation the rights+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell+copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in all+copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE+SOFTWARE.diff --git a/README.md b/README.mdnew file mode 100644index 0000000..32b81eb--- /dev/null+++ b/README.md@@ -0,0 +1,201 @@+# earshot++**Ears for coding agents.**++<!-- launch video -->+[**Watch the 22 s launch film**](https://github.com/Leonxlnx/earshot/releases/tag/v0.1.0) - every sound in it was made+in code by `earshot forge`; no recordings, no music track.++++AI coding agents can now write the code that renders a film, a product demo or a game. They still cannot hear what+they made. They cannot tell a premium key press from a cheap one, notice that a hit lands 80 ms after the cut, or+hear that the music swells in the wrong place.++earshot turns sound into evidence an agent can reason about: what a file sounds like in words, how well it fits a+description, the best moment inside a long recording, whether it is a real recording or code-synthesized, a+second-by-second listening transcript of a whole film, an annotated picture of it, and a scorecard against+professional launch films. It runs locally on CPU, as a CLI, a Python API and an MCP server.++With ears, an agent can also **make its own sounds in code** and improve them by listening (`earshot forge`, below).++```+$ earshot describe enter.ogg+ 0.56 s short transient (attack 3 ms, decays in 190 ms). Sounds most like: a single mechanical keyboard key press+ (z +5.4), a soft key release click (z +3.6) ... [clear reading]. peak -0.1 dBFS, brightness 1.5 kHz ...++$ earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch keyboard" "a mouse click"+ -> strong (z +3.7; would rank #2 among 322 vocabulary descriptions) ...++$ earshot find keys.ogg "a single mechanical key press" --top 4+ 0:03.94-0:04.19 (0.24 s stroke, z +4.1 strong); 0:10.60-0:10.86 ...++$ earshot critique film.mp4+ VERDICT: close - 21 of 23 scored metrics in range ... FIX FIRST: 1. [major] nothing reaches body level until 0:01.01 ...+```++## What it does++| command | what an agent gets |+|---|---|+| `describe FILE` | what it sounds like: calibrated tags (z) with the closest alternative readings, AudioSet labels, measurements |+| `match FILE QUERY [--alts ...]` | how well a description fits: cosine, z among the vocabulary, a verdict (strong / plausible / weak / no match) with held-out evidence |+| `rank QUERY PATHS...` | many candidate files sorted by a description (pick the best download) |+| `find FILE QUERY` | the best moments inside a long recording, as sample trims (start / end seconds) |+| `compare FILE --refs DIR` / `ab A B` | nearest reference recordings and typicality; A/B two sounds or two trims, the differences in words |+| `realness FILE...` | real recording / pro-produced vs code-synthesized, with evidence and per-window scores |+| `timeline FILE [--video V] [--cues C]` | the listening transcript: events (time, level, tag), sections, loudness, beat grid, sync of sound to picture cuts and cue frames |+| `sheet FILE -o out.png` | an annotated PNG (video frames, sections, labelled events, spectrogram, waveform, loudness, sync markers) for agents that read images |+| `critique FILE [--video V]` | a launch-video scorecard against a profile of 7 professional launch films, with timestamped fixes |+| `profile REFS... -o p.json` | build your own reference profile from films you admire |+| `measure FILE` | objective measurements only (no models): LUFS, true peak, LRA, envelope, spectrum, tempo / key |+| `forge RECIPE --refs DIR` | make a sound in code and improve it by listening (see below) |+| `llm FILE --yes-upload` | optional audio-LLM critique (Gemini), only with your own key and explicit consent |+| `mcp` | the same tools as an MCP server for Claude Code and other agents |+| `doctor [--load]` | check the install, models, data files and CPU settings |++Every command prints a plain-language summary first; `--json` gives the full result, `--out f.json` saves it. Errors+are one line on stderr (`earshot <command>: error: ...`, exit 1). Each CLI call is a new process and loads the models+(~20-40 s) before it answers: for more than a few calls use the MCP server (models stay loaded) or one Python script+with `from earshot import api`.++What an agent sees when it listens to that film with `earshot sheet` (frames, labelled sound events, spectrogram,+loudness, sync to the beat):++++## Make sounds: earshot forge++Listening makes iteration possible. The agent writes a sound as code: a **recipe** of physical synthesis (modal+resonators, contact excitations, filtered noise, a computed room and a microphone; no recorded audio inside). The+forge renders candidates from the recipe's parameters, and a **critic** built from earshot scores each one:+description match against named confusions, realness, closeness to real reference recordings, measured shape, and a+model-free tonal-balance and tail check against the references. An evolutionary **search** keeps the best and mutates+them. Then the agent reads the critic's words ("sounds like: a cork pop, a glass ting", "decay +2.4 outside the+refs"), looks at spectrograms next to the references, rewrites the recipe, and runs it again.++```bash+earshot forge # list the recipes: click, impact, keypress, pop, whoosh+earshot forge keypress --refs path/to/real/keypresses/ --budget 144 --reps 2 --tone 1.5 --keep 4 --robust 8 --dir made/keypress+earshot forge whoosh --target "a fast crisp air whoosh" --alts "a wind gust" "a sword swing" \+ --refs path/to/real/whooshes/ --tone 1.5 --dir made/whoosh+```++Output: the best diverse sounds as WAVs plus `report.json` with every candidate's parameters and scores, the same+critic's scores of the real references (leave-one-out), held-out AudioSet tags, and with `--robust N` each pick+re-rendered on N new seeds. Every score also carries model-free measurements (`earshot/forge/tone.py`): the+1/3-octave tonal balance against the references' median ("+16 dB at 500-630 Hz (above every ref)"), the tail and the+length. In Python: `from earshot.forge import forge`.++**Blind test.** A key press, a UI click, a pop, a fast whoosh and an impact were made from scratch in code and+played blind next to real recordings of the same kind of sound. The author picked the made one correctly **8 times out+of 15**: a coin flip. Only the made impact was spotted (its tail dies too fast). One listener and 15 trials is a small+test, not a proof; it says the made sounds are no longer easy to tell from real ones.++**Honest limits (Goodhart).** The search optimises earshot's own models, and those can be pleased without the sound+getting better. After a search the made sounds pass every model check, held-out ones included, so the model checks+stop telling made from real; the 1/3-octave measurement still can. The realness model is easy to fool (a modal+resonator plus a room and a noise floor reads "real"). Judge a made sound by the model-free measurements,+spectrograms next to the references, robust re-renders, and a human blind test.++## Install++Python 3.10+ and ffmpeg / ffprobe on PATH. CPU only; no GPU is used.++```bash+git clone https://github.com/Leonxlnx/earshot && cd earshot+python -m venv .venv+.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu # Windows: .venv/Scripts/python+.venv/bin/python -m pip install -e ".[mcp,llm,dev]"+.venv/bin/python -m earshot doctor --load+```++The first model use downloads two open models (~1.4 GB) into `.cache/hf` inside the repo (or `EARSHOT_HOME`): CLAP+`laion/larger_clap_general` (text-audio embeddings) and the AudioSet tagger `MIT/ast-finetuned-audioset-10-10-0.4593`.+After that earshot works offline.++Settings (environment variables): `EARSHOT_THREADS` (torch CPU threads, default min(4, cores/2)), `EARSHOT_HOME`+(cache root), `EARSHOT_PRECISION` (`bf16` by default, used only on CPUs with native bf16 such as AVX512-BF16 / AMX,+fp32 everywhere else; `fp32` forces fp32), `EARSHOT_OFFLINE=1`. `earshot doctor --load` reports what is in use.++### MCP server for Claude Code++```bash+claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp+```++Or in any client's `.mcp.json`:++```json+{"mcpServers": {"earshot": {"command": "/path/to/earshot/.venv/bin/python", "args": ["-m", "earshot", "mcp"],+ "env": {"EARSHOT_THREADS": "3"}}}}+```++Models load on the first call and stay warm. On a slow machine start Claude Code with `MCP_TIMEOUT=60000`. The+optional `llm_critique` tool reads `GEMINI_API_KEY` from the environment; never write a literal key into a file.++### Python++```python+import earshot+earshot.describe('whoosh.wav')+earshot.find('typing.ogg', 'a single isolated key press')+earshot.timeline('soundtrack.wav', video='film.mp4')+from earshot import api # everything in one namespace: api.rank, api.critique, api.realness_many, ...+```++## Agent skills++Two skills ship in `skills/`. Copy or symlink them into `~/.claude/skills/` (or a project's `.claude/skills/`).++- **`skills/earshot`**: how an agent should listen with earshot: which command answers which question, how to read+ the verdicts, when not to trust them.+- **`skills/real-sound`**: a full process for premium, realistic soundtracks for code-made films from real licensed+ recordings: sourcing with licence checks, a music edit, a JSON sound design, a deterministic mixer, measured QA, a+ multi-lens critique, mastering, credits and mux. earshot chooses, trims and verifies the sounds.++## How good is it?++Every number below comes with the command that measured it in [docs/VALIDATION.md](docs/VALIDATION.md). What+earshot cannot judge is in [docs/LIMITS.md](docs/LIMITS.md).++- **Matching descriptions** (a held-out library of 125 real recordings, nothing tuned on it): in a library search a+ "strong" verdict is the right kind of sound 98% of the time and the exact variant 61% of the time.+- **Finding the moment**: on a recording of 17 single key presses the found trims land on real key presses; in a+ 300 s scene every planted event is found as a tight trim.+- **Real vs code-synthesized**: held-out ROC-AUC 0.92 with whole sound families held out. Firm verdicts call 6.7% of+ real clips synthesized and 5.5% of synthesized clips real; a fifth of clips get "uncertain".+- **Listening through a soundtrack**: 93% of the sound effects placed in a 60 s film mix are detected as events (89%+ of the quiet ones under music). A 60-90 s film takes about 1.5 minutes on a laptop CPU.+- **Launch scorecard**: each reference film critiqued against the other six falls in range on 86% of the metrics+ (the no-overfit baseline is about 71%; some rules changed after the first run, so it is not a blind test).++## Limits++earshot hears through two open models and a set of measurements. Trustworthy: loudness, timing and sync to picture,+"is this the kind of sound I asked for" on single exposed sounds, picking among candidates, obvious code-synthesized+sources. Hints only: tags of events inside a busy mix, music genre and mood. It cannot judge taste or emotion,+provenance or licences, speech content, or anything above 14 kHz from the models. The launch scorecard describes the+style of the 7 films behind it, not every good film. Details in [docs/LIMITS.md](docs/LIMITS.md).++## Repo map++```+earshot/ the package: CLI, Python API, MCP server, CPU model wrappers+ forge/ make sounds in code: DSP building blocks, critic, model-free tone checks, search, recipes/+ data/ vocabulary + calibration stats, realness model weights, launch profile (numbers only, no media)+skills/earshot/ agent skill: how to listen with earshot+skills/real-sound/ agent skill: real-recordings soundtracks (scripts, templates, prompts, references)+docs/ VALIDATION, LIMITS, launch-video sound playbook, integration notes, design notes+eval/ evaluation scripts and results per area (see eval/README.md)+tests/ pytest (fast by default; --slow loads the models)+```++No audio or video is stored in this repo. The evaluation data (licensed recordings and third-party films) is not+redistributed; only derived numbers are committed.++## Licence++MIT for the code (see [LICENSE](LICENSE)). The models are downloaded from their authors under their own licences:+CLAP (`laion/larger_clap_general`) Apache-2.0, AST (`MIT/ast-finetuned-audioset-10-10-0.4593`) BSD-3-Clause.+`skills/real-sound/scripts/master.py` is vendored from the Cinetic skill by the same author.diff --git a/docs/DESIGN.md b/docs/DESIGN.mdnew file mode 100644index 0000000..d716f61--- /dev/null+++ b/docs/DESIGN.md@@ -0,0 +1,102 @@+# earshot: design notes++earshot gives coding agents "ears": an agent cannot hear, so earshot turns sound into things an agent can reason+about: calibrated text-audio match scores, the best moment inside a long recording, similarity to real reference+recordings, real-vs-synthesized judgement, a second-by-second listening transcript of a soundtrack, an annotated+picture of it (agents can look at images), a launch-video scorecard against professional references, and (only with+the user's own key) an audio-LLM critique. Everything runs on CPU; no GPU is ever used.++The repo also ships `skills/real-sound/`: the agent skill for building premium soundtracks for code-made films from+real licensed recordings (sourcing + licence audit + design JSON + mixer + mastering + credits), which uses earshot to+pick and verify sounds.++## Principles+1. **Honest numbers.** Every score says what it is (cosine, z vs background, probability) and how much to trust it.+ Never present CLAP cosine as a probability. Validation results live in `docs/VALIDATION.md` with the exact commands.+2. **Agent-readable first.** Every command prints a short plain-language `summary` first, then details; `--json` for+ the full dict. Timestamps as seconds (3 decimals) and m:ss.ss in text. Levels in dBFS / LUFS / dBTP.+3. **Deterministic and cached.** No random crops; fixed windows; embeddings cached on disk (EARSHOT_HOME/emb).+4. **Gentle on the machine.** CPU only, bf16 by default, `EARSHOT_THREADS` (default min(4, cores/2)). Batch work.+ This laptop (Ryzen 7 260, 8 cores / 16 threads) is shared with other jobs: when developing set `EARSHOT_THREADS=3`,+ keep any single job under ~10 minutes, run jobs in the foreground, one heavy job at a time per agent.+5. **No media in git.** Never commit audio/video (reference films, downloaded recordings, renders). Derived numbers+ (profiles, trained weights, eval tables) are fine.+6. **No synthesized deliverables.** earshot may synthesize test signals for tests/evaluation only.++## Layout and ownership (one owner per file during the build)+| file | owner | purpose |+|---|---|---|+| `earshot/config.py`, `audio.py`, `features.py`, `video.py`, `report.py`, `cli.py`, `doctor.py`, `__init__.py`, `models/clap.py`, `models/ast.py` | core (lead) | shared base: read-only for builders. Need a change? Work around it in your module and report it. |+| `earshot/data/vocab.json`, `earshot/vocab.py`, `describe.py`, `match.py`, `search.py`, `compare.py` | semantics | describe / match / rank / find / compare / ab |+| `earshot/timeline.py`, `earshot/sheet.py` | timeline | listening transcript, sync analysis, annotated PNG |+| `earshot/realness.py`, `earshot/data/realness*.json`, `eval/realness/*` | realness | real vs synthesized detector + its evaluation |+| `earshot/profile.py`, `earshot/critique.py`, `earshot/data/launch_profile.json`, `docs/launch-video-sound.md` | launch | reference profile from pro launch films + scorecard |+| `earshot/llm.py`, `earshot/mcp_server.py`, `skills/earshot/SKILL.md` | interfaces | optional audio-LLM critique, MCP server, agent skill for using earshot |+| `skills/real-sound/**`, `docs/INTEGRATION.md` | real-sound | the soundtrack skill (moved from the Clawd project) + Cinetic integration notes |+| `tests/test_<module>.py` | that module's owner | fast tests (synthetic fixtures, no network). Model tests marked `@pytest.mark.slow`. |+| `README.md`, `docs/VALIDATION.md`, `docs/LIMITS.md` | lead (integration) | builders send their numbers in their final report |++## Core API (already built; use it)+- `earshot.audio`: `load(path, sr=48000, mono=False, start=None, end=None)` -> float32 (n,2)|(n,), any ffmpeg-readable+ file incl. the audio of mp4; `probe(path)`; `windows(x, win, hop)`; `list_media(paths)`; `resample`; `to_mono`;+ `file_id(path)`; `SR = 48000`.+- `earshot.features`: `measure(path_or_array, music=False, light=False)` -> dict (see module doc: lufs, true peak,+ noise floor, digital_silence_frac, attack_ms, decay_ms, centroid, bands_db, flatness, tonal_peak_ratio_db,+ harmonic_ratio, f0, onsets, tempo/beats/key); `loudness_curve(x, win, hop)`; `integrated_lufs`; `lra`; `onsets`.+- `earshot.models.clap.get()` -> `Clap`: `embed_text(list)` (n,512) unit; `embed_audio(list_of_arrays, pad=+ 'repeatpad'|'zero', per_window=False)`; `embed_file(path, start, end, pad)` (cached); `embed_files(paths)`;+ `logit_scale` (~38.7). ~0.3-1.2 s per 10 s window on 4 threads (bf16). Every input costs one 10 s window (short+ sounds are padded), so batch and avoid needless tiny windows.+- `earshot.models.ast.get()` -> `AST`: `probs(list_of_arrays, reduce='max'|'mean')` (n,527) sigmoid; `probs_file`;+ `top(p, k)`; `labels`; `prob_of(p, label)`. AudioSet ontology labels. Hears up to 8 kHz (16 kHz input).+- `earshot.video`: `cuts(path, threshold=0.3)`, `frame(path, t, width)`, `frames`, `info`, `has_video`.+- `earshot.report`: `emit(result, args)`, `fmt_t`, `to_json`. `earshot.cli.common(parser)` adds `--json --out`.++## Public functions (the contract other modules and the MCP server rely on)+All return JSON-serialisable dicts with a `summary` string. Paths may be audio or video files.+- `describe(path, start=None, end=None, top=8)` -> tags per vocab category with calibrated scores, AST top labels,+ key measurements, one-paragraph `summary`.+- `match(path, query, alts=None, start=None, end=None)` -> `{'query', 'cosine', 'z', 'percentile', 'rank_in_vocab',+ 'p_vs_alts' (softmax over query+alts with logit_scale), 'verdict' ('strong'|'plausible'|'weak'|'no match'), ...}`.+- `rank(paths, query, top=None)` -> files sorted by calibrated match. `find(path, query, win=None, hop=None, top=5,+ start=None, end=None, kind='auto')` -> best segments inside a long recording `[{'start','end','score','z','kind',+ 'head_db','tail_db','cut'}]`: sample trims, except one marked 'head cut' / 'tail cut'.+- `compare(path, refs, k=5, start=None, end=None)` -> nearest references + typicality vs the reference set;+ `ab(a, b, query=None, refs=None, k=5, a_start=None, a_end=None, b_start=None, b_end=None)` -> which is closer to the+ query / refs and how they differ (feature deltas in words); the trims judge only part of a file.+- `realness(path)` -> `{'p_real', 'verdict', 'evidence': [...]}` (+ per-segment for long files).+- `timeline(path, video=None, cues=None, ...)` -> events (time, level, tags, AST labels), windows tags, loudness+ curve, cuts, sync stats (event-to-cut / event-to-cue offsets), `text` = the listening transcript.+- `sheet(path, out_png, video=None, ...)` -> writes an annotated PNG an agent can open; returns its path + summary.+- `critique(path, profile=None, video=None)` -> scorecard: each metric, the soundtrack's value, the reference range,+ status (ok/low/high), what to do.+- `profile` CLI builds a profile JSON from a folder of reference videos/soundtracks.+- `llm.critique(path, prompt=None, provider='gemini')` -> only when GEMINI_API_KEY / GOOGLE_API_KEY is set and the+ caller passes consent (`--yes-upload`); otherwise a clear message how to enable it. Never hardcode keys.++CLI: each module implements `add_cli(sub)` registering its subcommand(s) (names in `earshot/cli.py COMMANDS`; `search`+registers `rank` and `find`; `compare` registers `compare` and `ab`). Use `from .cli import common` for `--json/--out`.++## Evaluation data (not redistributed)+Evaluation data: licensed recordings and third-party films, not redistributed. The evaluation scripts in `eval/`+read their locations from environment variables or a gitignored `eval/local_paths.json` (see `eval/README.md`).+What was used:+- A library of real licensed recordings (Freesound / Mixkit) in 8 categories (ambience, foley, garden, keyboard,+ magic, motion, music, toy): 219 files used for tuning, plus 126 real recordings rejected for quality, used as the+ held-out set. Labels come from per-file manifests (category, title, need, notes).+- Code-synthesized audio: a retired synthesized film score with its stems, and one-shots rendered by an open synth+ engine (kick, sub_boom, clap, hat, tick, tock, key_click, thock, ui_click, snap, blip, fm_bell, whoosh, riser,+ swell, pluck, bass, drone, keys ...).+- A finished 60 s real-recordings film soundtrack with ground truth: stems (sfx, amb, music), every placement with+ time and sample in a mix report, picture cue events at 60 fps, and the same film with the old synthesized+ soundtrack.+- 7 professional launch films as the quality bar (measured only, never copied), referred to as ref_1..ref_7 with+ neutral style descriptions in `eval/launch/refs.json`.++## Environment+- Python 3.10+ with torch (CPU), transformers, librosa, scikit-learn, mcp. Run from the repo root (package importable+ as `earshot`), e.g. `.venv/bin/python -m earshot describe file.wav`.+- Models are cached in `.cache/hf` inside the repo or `EARSHOT_HOME` (CLAP laion/larger_clap_general, AST+ MIT/ast-finetuned-audioset).+- Scratch output: `out/<area>/` (gitignored).+- ffmpeg / ffprobe on PATH.diff --git a/docs/INTEGRATION.md b/docs/INTEGRATION.mdnew file mode 100644index 0000000..e258783--- /dev/null+++ b/docs/INTEGRATION.md@@ -0,0 +1,113 @@+# Integration: real-sound + earshot inside a Cinetic film (without changing Cinetic)++Cinetic (`~/.claude/skills/cinetic`) directs and renders launch films from code. Its Step 5 synthesizes the whole+soundtrack (`scripts/audio/score.py` + `synth.py`) from a beat-locked timeline. This document shows how a Cinetic film+gets a real-recordings soundtrack from `skills/real-sound` instead, and how earshot gives Cinetic's review loop ears,+using Cinetic exactly as it is today. The last section lists the small changes a future Cinetic could make so this+becomes one flag.++## 1. What Cinetic produces that the soundtrack needs++| Cinetic output | content | used by real-sound / earshot as |+|---|---|---|+| `out/cues.json` (`npx tsx scripts/export-cues.ts`) | `{fps, bpm, total, acts, cue, events: [{f, kind, weight, pan, pitch?, apexFrac?, dur?, variant?, id?}]}`, sync frames computed from the picture code | **the cue file, unchanged**: `mix.py --cues out/cues.json` (mix.py's cue format is this format). Rules match `kind` (regex), `id` (regex), `variant`, a time window; `weight` drives `weight_db`; `pan` drives `pan: "cue"` |+| `src/timeline.ts` (`FPS`, `BPM`, `TOTAL`, `ACT`, `CUE`) | the grid | `TOTAL / FPS` = the design's `duration`; `BPM` = `arrange.py --film-bpm`; `ACT` (exported as `acts`) = qa.py's per-act loudness |+| `audio/score.json` | key, bpm, one chord per bar, sections (`intro / build / groove / drive / breakdown / resolve / hold`), drops + silences, `payoff`, signature tick/tock/motif, typing mode, mix, `master.lufs` | the musical intent: sections -> `arc.json` for `arrange.py`; `payoff` / drops -> `drop_film_bar`; key and chords -> which tuned recordings to source and whether a splice keeps the harmony; `master` -> the design's `master` |+| `TREATMENT.md` / `BRIEF.md` | the brand's three adjectives, the personality row (`references/sound.md` §5.5a), the beat sheet with a sound column | the SOUND_BRIEF's concept and picture map |+| `public/audio/soundtrack.wav` (written by `npm run audio`) | the master Cinetic's `render.sh --audio` muxes | **replaced by `mix.py`'s `out/audio/soundtrack.wav`**, passed to `render.sh --audio` (render.sh takes any WAV; keeping it out of `public/` keeps the premaster and stems out of the Remotion bundle) |+| `out/stems/` | stems for `av-audit.py --stems` | `mix.py --stems out/stems` writes `sfx.wav`, `amb.wav`, `music.wav`: av-audit picks `sfx*.wav` as the effects and `music.wav` as the music bus |++### score.json sections -> an arrange.py arc+One number per film bar (dB relative to the track's loudest sections): `intro` -14 (rising to -10), `build` -10 -> -4+across its bars, `groove` -6, `drive` -2..0, `breakdown` -10, `resolve` -8 -> -14 over its bars, `hold` / `tail` -20.+`payoff` (usually the drop's bar) becomes `drop_film_bar`; a drop's `silence` becomes a deliberate hole in the music+edit (the segment before the drop ends `silence` early; qa.py lists it as a note, not a defect).++### Synth voices -> real recordings++| Cinetic kind | synthesized today | real-sound replacement (category / source / rule) |+|---|---|---|+| music stems: drums, bass, pad, arp, bells, reverb | generated per chord and section | ONE licensed track edited at bar lines (`arrange.py` draft, music supervisor); its own drop lands on `payoff` |+| `tick` / `tock` (signature clock) | tuned wooden mallet | a real wood block / soft mechanical switch, two takes a fourth apart, resampled within +-2 st to the key |+| `key` (typing) | dark modal tock per key, thinned to ~12/s | slices of one real keyboard recording (`slices` + `mode: cycle`); `variant: "space"/"return"` -> a deeper spacebar / enter take; per-word style -> a rule on `variant: "word"` only |+| `click` | bright 2 kHz voice + release | a real mouse / trackpad press + release (two rules, `offset` ~0.085) |+| `pop` | pitched blip | a real cork / mouth pop / plastic tick, `pitch_ramp` up across a run |+| `snap` | click + body on the chord root | a real latch / Lego / magnet clack, layered with a low body |+| `land` | wooden knock + sub | real foley thumps by weight (`weight_db`), light taps for `variant: "light"` |+| `hit` | sub boom + clap + burst | a real drum / taiko / kick hit + a short top transient (two rules) |+| `drop` | long sub boom | the music's own drop (edit), optionally a real low impact under it |+| `whoosh` | noise band sweep | a real swish (bamboo, stick, cloth), `align: "peak"` |+| `riser` / `swell` | noise + saw glide / reverse chord | a real cymbal roll or orchestral swell, `align: "end"` (or the track's own build) |+| `suck` | reverse whoosh | a real air-suck recording, or nothing (mix.py does not reverse recordings) |+| `bell` (`variant: "motif"`) | FM bell / the motif | a real celesta / bell / glass note in the key (one sample per note; the Clawd film built its motif this way) |++Cinetic's sound rules stay valid (the sound map, levels, the drop on the hero reveal, the resolved logo, restraint);+`skills/real-sound/references/sound-design.md` restates them for recordings.++## 2. The bridge, step by step (Cinetic untouched)++Inside a Cinetic film project (`films/<name>/`), with `<rs>` = `skills/real-sound` of this repo:++1. Cinetic Steps 0-4 as usual. In `BRIEF.md` write the spec line with `audio: real recordings (real-sound)`.+2. Cues: `npx tsx scripts/export-cues.ts` -> `out/cues.json`.+3. Brief: copy `<rs>/templates/SOUND_BRIEF.md` to `audio/SOUND_BRIEF.md`; take the picture map from the treatment's beat+ sheet (its sound column) and `out/cues.json`; take the personality row and the arc from `audio/score.json`+ (write it even though score.py will not run: it is the musical intent).+4. Sourcing, licence audit, music edit, design fragments, critique and fix: the real-sound process+ (`<rs>/SKILL.md`, prompts in `<rs>/templates/prompts/`), with every path relative to the film project.+5. Render the soundtrack:+ ```+ python -I <rs>/scripts/mix.py --root . --design audio/sounddesign.json --cues out/cues.json \+ --out out/audio/soundtrack.wav --stems out/stems --report out/qa/mix_report.json --trace out/qa/adds.json+ ```+ Set the design's `master.lufs` to `audio/score.json`'s `master.lufs` (-14 for launch films). Do not run+ `npm run audio` for this film (it writes the synthesized score).+6. Cinetic Step 6 with the real soundtrack: `bash scripts/render.sh Film out/preview.mp4 --preview --audio+ out/audio/soundtrack.wav` (and `--blur` for the master; never `--build-audio`, which re-runs score.py). Its `check-sync.py` gates lag and true peak on the decoded+ MP4 as before: `python3 scripts/check-sync.py out/film.mp4 out/audio/soundtrack.wav`.+7. Cinetic Step 7 (review loop) with ears (section 3).+8. Credits: `audit.py --report out/qa/mix_report.json` and `credits.py`; put `out/CREDITS_description.txt` into the+ delivery's README / video description.++`real-sound`'s own `mux.mjs` is not needed in a Cinetic film: `render.sh` muxes. Use `mux.mjs` for films rendered+outside Cinetic.++## 3. Where earshot slots into Cinetic's self-critique loop++Cinetic's loop (§8): render -> measure (probe, forensics, av-audit) -> sheets -> critic lenses -> verify -> fix.+earshot adds hearing to the measure and lens steps:++| Cinetic step | add | what it gives the critics |+|---|---|---|+| Step 5 gate (after the soundtrack is written) | `python -I <rs>/scripts/qa.py --layers --stems out/stems` | clicks / cuts with the guilty sample, masked events, harsh 2-6 kHz peaks, chopped music splices, sync per event, partial coverage, loudness and LRA |+| Step 7 measure | `python -m earshot timeline out/preview.mp4 --cues out/qa/cues_sounded.json` | a second-by-second listening transcript of the decoded film: events with tags, sections, loudness curve, beat grid, sound-to-cut and sound-to-cue offsets |+| Step 7 measure | `python -m earshot sheet out/audio/soundtrack.wav -o out/qa/sound_sheet.png --video out/preview.mp4 --cues out/qa/cues_sounded.json` | one PNG (frames, sections, tagged events, spectrogram, loudness, cue sync) the director and sound lenses look at next to the contact sheets |+| Step 7 measure | `python -m earshot critique out/audio/soundtrack.wav --video out/preview.mp4` | the scorecard against the profile of professional launch films (metric, value, reference range, what to do) |+| av-audit | `python3 scripts/av-audit.py out/preview.mp4 --cues out/qa/cues_sounded.json --stems out/stems` | Cinetic's own onset/sync audit on the real soundtrack; `cues_sounded.json` (written by mix.py) lists only events that received a sound, so deliberate silences are not counted as misses |+| sound-sync lens (`assets/critics/sound-sync.md`) | hand the lens `qa.json`, the earshot transcript, sheet and scorecard with `av.json` | the lens can name the exact rule / sample to change; with real recordings its fixes are design operations (`<rs>/templates/prompts/fixer.md`), not score.py parameters |+| verifier (`assets/critics/verifier.md`) | refute a sound issue with earshot evidence (`earshot match` on the placed sample, the transcript at that time) | fewer false P1s from reading code alone |+| a synthesized score already exists | `python -m earshot realness public/audio/soundtrack.wav` (score.py's output) | shows (per 4 s window) where the synthesized score reads as code-made: the case for switching |++## 4. A minimal change list for a future Cinetic++Each item is small and keeps the synthesized path as the default.++1. **One switch in `audio/score.json`:** `"engine": "synth" | "real"` and `"design": "audio/sounddesign.json"`. With+ `real`, `npm run audio` runs real-sound's `mix.py` with the same `--cues/--stems`, `--out` outside `public/`+ (render.sh's `--audio` default and `--build-audio` follow it) and `--json` mapped to `--report`; `score.py` stays+ for `synth`.+2. **`export-cues.ts --thin-typing`:** apply `plan_typing()`'s thinning (<= 12 keys/s, per-word above 25 chars/s) at+ export, marking skipped keys `variant: "silent"`, so a recordings design sounds exactly the keys score.py would.+3. **`av-audit.py`:** when `engine` is `real`, read `out/qa/cues_sounded.json` and skip the score.py typing replay.+4. **`references/sound.md`:** a §1b "Real recordings" pointing at real-sound, and a "real palette" column in the+ personality table (calm: felt piano, soft wood; fast: tight drum hits, metal latches; bright: marimba, claps, cork+ pops; precise: sparse piano, glass; raw: switches, latches; warm: plucked strings, soft taps).+5. **`assets/critics/sound-sync.md`:** with `engine: real`, replace "Never propose stock or sampled sounds" by+ "propose recordings from the manifests or a new sourcing need; never synthesized sounds", and read `qa.json` and+ the earshot transcript / scorecard.+6. **`references/review-loop.md` Measure step:** add `qa.py --layers` and `earshot timeline / sheet / critique` to the+ scripts that write `out/qa/`.+7. **`scripts/new-film.sh --audio real`:** scaffold `audio/SOUND_BRIEF.md`, `audio/sounddesign.json`, `audio/design/`+ and `audio/assets/manifests/` from real-sound's templates, and write `audio: real` in the BRIEF spec line.+8. **One `master.py`:** real-sound vendors Cinetic's `master.py` byte for byte (sha1+ 2af9044a923cdbe01a81bb1d7922b97d41641479 in both today); keep a single source when the two ship together.diff --git a/docs/LIMITS.md b/docs/LIMITS.mdnew file mode 100644index 0000000..33933ce--- /dev/null+++ b/docs/LIMITS.md@@ -0,0 +1,69 @@+# What earshot can and cannot judge++earshot gives an agent measured evidence about sound, not taste. Read this before trusting a verdict.++## Trustworthy+- **Measurements**: loudness (LUFS, true peak, LRA), timing (onsets, offsets to cuts and cue frames, to the+ millisecond), levels, spectrum, envelope, clipping, silence, tempo of steady music. These are meters, not opinions.+- **"Is this the kind of sound I asked for?"** on single, exposed sounds (a key press, a whoosh, rain, a glass ting):+ a "strong" match is very reliable on held-out recordings; "plausible" and "weak" are hints worth a second check.+- **Picking among candidates** (`rank`) and **finding the clean moment** (`find`) in real recordings.+- **Obvious code-synthesized sources** (noise risers, simple FM bells, steady test tones): flagged with ~92% precision+ when the verdict is firm (6.7% of real clips get called synthesized, 5.5% of synthesized clips real). Bare beeps+ are less reliable than that list suggests: with the beep family held out, beeps were among the most-missed types+ (accuracy 0.71).+- **Whether sounds land on the picture**: offsets of hard hits to cuts / cue frames (median 0 ms on Clawd, 96% within+ a frame).++## Hints only+- **Tags of events inside a busy mix** (what each hit "is"): right about half the time for exposed events, a quarter for+ events under the music. Times and levels are right; the label is a guess.+- **Music genre and mood**: often wrong (an upbeat track's top mood was a dark one 18% of the time); describe keeps+ them apart as unvalidated hints. Instruments, rain, birds and voice are usually right.+- **"Uncertain" from realness** on designed library sounds ("magic" chimes/celesta, "motion" whooshes) and on real+ electronic music: they sit between real and synthesized. Read it as "sounds designed", not "fake".+- **The launch scorecard ranges** come from 7 films the user picked. They describe that style (modern SaaS / AI+ launch films, kinetic type, UI demos), not every good film. A cinematic short or a quiet piano piece will break+ rules that do not apply to it. Build your own profile from films you admire (`earshot profile`).+- **The "cuts without a hit" list** (timeline, sheet, critique tips): soft build-ups (whooshes, risers, swells) can+ still be missed - on Clawd both cues it named were sounded whooshes. Look at the sheet before adding a sound.+- **critique's "onset on visual events" share** counts only new attacks within +-100 ms (the definition the reference+ ranges were measured with), so a whoosh swelling through a cut does not count; its fixes and tips name only the cuts+ with no sound at all (timeline's matcher). critique's stop-downs (short drop-outs before a hit, per minute) and+ timeline's gaps (silences of any kind) are different counts.+- **realness verdicts depend on length**: a sound up to 6 s is judged with the clip thresholds (lo 0.17), a longer file+ by the mean of its 4 s windows with the weakly calibrated file thresholds (lo 0.40), so a lower p_real can get the+ milder verdict. Judge the trim you will actually use (`--start / --end`), not a multi-take file.+- **AudioSet labels on one-shots under ~0.5 s**: often 'unsure' or a near class (Zipper for a swish, Finger snapping+ for a click). Treat those as neutral; only an outright different class (Speech, Music, Engine) is a red flag.++## Cannot judge+- **Taste and emotion**: whether a sound is beautiful, whether the music fits the brand, whether a moment feels+ satisfying. earshot can say "a strong match for a deep thocky key press, 3 ms attack, lands 4 ms after the cut"; it+ cannot say the film feels premium.+- **A code-made layer hidden under a louder real mix**: usually missed. Judge source files and stems, not only the+ final mix.+- **Provenance**: realness hears synthesis, not licences. A professionally made, licensed synthetic UI sound can read+ "likely code-synthesized"; code-made noise beds that imitate nature (rain, wind) or telephone tones can read real.+- **Anything above 14 kHz from the models** (CLAP's front end stops there; the AudioSet tagger at 8 kHz). Use+ `measure` (bandwidth, bands) for air, hiss and codec cut-offs.+- **Speech content**: no transcription; voice is detected, words are not.+- **Licences**: earshot never decides whether a file may be used. The real-sound skill's `audit.py` checks manifests;+ the licence itself must be read on the item's page.++## Known engineering limits+- Cold start: every CLI call is a fresh process that loads the models (~20-40 s on a busy laptop), even for a cached+ file; the MCP server or one Python script (`from earshot import api`) loads them once. `doctor` without `--load`+ loads nothing heavy.+- bf16 is used only on CPUs with native bf16 (AVX512-BF16 / AMX); elsewhere earshot runs fp32 (bf16 emulation was+ measured 16x slower). The bundled calibration was measured with bf16 embeddings (fp32 agrees to cosine >= 0.9993 on+ 12 real clips, `eval/semantics/precision.json`).+- Caches are keyed on path, size, modification time and 16 sampled 4 KB blocks of the file: a re-render that differs+ only between those blocks and keeps the old size and timestamp would still read stale. Use `timeline --fresh` or+ touch the file when in doubt.+- The models were evaluated on one sound library (Freesound / Mixkit picks for one film), 7 launch films and three+ synthesis code bases. Other domains (orchestral recording sessions, field recordings in other climates, other synth+ engines) are untested.+- The optional Gemini critique has not been validated against real answers (it needs the user's key and consent to+ upload).+- Cut detection marks big visual changes (a blinking cursor counts); slow crossfades are missed.diff --git a/docs/VALIDATION.md b/docs/VALIDATION.mdnew file mode 100644index 0000000..aebf107--- /dev/null+++ b/docs/VALIDATION.md@@ -0,0 +1,113 @@+# Validation++Every number here comes from a command in this repo, run on a Ryzen 7 260 laptop CPU (bf16, `EARSHOT_THREADS` 2-3,+other jobs running). Full tables, data descriptions and "what was tuned on what" live in each area's results file;+this page is the summary. Data: the Clawd film's licensed recordings (219 used + 125 rejected Freesound/Mixkit files),+its finished 60 s soundtrack with every placement logged, the retired code-synthesized score the user rejected, renders+of the Cinetic synth engine, independent numpy generators, and 7 professional launch films (measured locally, never+committed).++## 1. Matching descriptions, ranking, finding moments ([eval/semantics/RESULTS.md](../eval/semantics/RESULTS.md))+`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py analyze` (tuning library) and `... heldout`.++- Calibration (shipped: 5 prompt templates, term-bias removal + per-sound z): pooled AUC of (recording, description)+ pairs 0.966 on the tuning library (raw cosine 0.951). Tuning-library numbers are optimistic: 67 of the 322+ vocabulary terms were added after seeing its classes.+- **Held-out library (125 files labelled from file names before any scoring, nothing tuned on it)**, per verdict band:++| verdict | z | true : wrong (lenient) | precision at a 3% base rate | at even odds | strict (close variant = wrong) |+|---|---|---|---|---|---|+| strong | >= 3 | ~1300 : 1 (126 true vs 3 wrong pairs) | 0.98 | 1.00 | 51 : 1 (precision 0.61 at 3%) |+| plausible | 2.5-3 | 21 : 1 | 0.41 | 0.95 | 9.1 : 1 |+| weak | 2-2.5 | 12 : 1 | 0.28 | 0.92 | 6.6 : 1 |+| no match | < 2 | 0.25 : 1 | 0.008 | 0.20 | 0.25 : 1 |++ Half of the true matches reach "strong" (recall 0.51), 76% reach at least "weak". The strong band's ratio rests on+ only 3 wrong pairs: with that count's 95% Poisson range (0.6-8.8) it lies anywhere from ~440:1 to ~6000:1, and the+ tuning library gave 170:1. In a library search (~3% of candidate pairs true) a "strong" hit is the right kind of+ sound 98% of the time but the exact variant only 61% of the time (an Enter key for a spacebar counts as wrong).+- Concrete needs, **tuning library only** (24 queries written from its manifests, such as "a deep thocky keyboard+ press", "a glass ting", "rain on a window"; optimistic, never scored on the held-out library): a correct file in the+ top 3 for 23/24 (miss: "a cork pop", mouth pops rank first), rank 1 for 19/24.+- `find`: on a recording of 17 single key presses, 16 usable trims on 16 distinct presses (the 17th is a false positive+ on a double hit the curator marked to skip), each starting 5-54 ms before the attack, never after; 9/10 on a 32-press+ board; in a 44 s scene of 8 real one-shots the right event is top-1 for 8/8 (7 tight trims, one 5 s window); in a+ 300 s scene every planted event is found as a tight trim (the first build never looked past ~95 s; fixed with+ gap-free windows and a two-stage search). Re-run after the final review's find changes (snapped windows for+ one-shot queries, no events across edit gaps, `--kind event` = onset-based trims only): the same numbers.+- Speed (`eval/semantics/timing.json`, models already loaded, nothing cached): model load 27 s once per process;+ `describe` of a 0.56 s key 17 s; `match`, new query on a cached file 2.0 s; `find` on an 18 s file 29.9 s for a new+ file, 1.2 s for a new query; `rank` 2.8 s per new file. Every CLI call is a new process and pays the model load, so+ for more than a few calls use the MCP server or one Python script (`from earshot import api`).++## 2. Real vs code-synthesized ([eval/realness/RESULTS.md](../eval/realness/RESULTS.md))+`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage report` (data and training scripts in+`eval/realness/`; `train.py` reproduces the shipped weights bit for bit).++- 1484 clean clips (691 real / pro, 793 synthesized), grouped cross-validation that never tests on a trained group:++| scheme | ROC-AUC | [email protected] |+|---|---|---|+| leave one real category / synth family out (headline) | 0.922 | 0.860 |+| leave one synth generator type out | 0.958 | 0.902 |+| retired (user-rejected) score never trained on: real vs retired | **0.948** | 0.862 |+| retired score + the whole engine that made it never trained on: real vs retired | 0.909 | 0.839 |+| zero-shot CLAP prompts (no training, baseline) | 0.858 | 0.749 |++- Verdict bands (held-out): "real or pro-produced" p_real >= 0.77 and "likely code-synthesized" <= 0.17 are each right+ ~92% of the time on a pool that is 47% real / 53% synthesized, with both thresholds chosen on these same held-out+ predictions (so the 92% is in-sample for that choice); 79% of clips get a firm verdict, the rest "uncertain". The+ base-rate-free error rates: 6.7% of real clips are called synthesized, 5.5% of synthesized clips are called real.+ Long files use file-level thresholds (lo 0.40) on the mean of their 4 s windows, read from only 23 synthesized long+ files (weak calibration; the tool says so).+- Pro-produced music held out by provider: mean p_real 0.86, 44 of 52 files called real, none called synthesized.+- A deterministic rule catches pure, dead-steady test tones: a 30 s 440 Hz sine gets 0.48 from the model alone and 0.03+ with the rule; the rule fired on none of 4741 audible clips of the 345 real recordings+ (`eval/realness/tone_guard.py` -> `tone_guard.json`).++## 3. Listening through a soundtrack ([eval/timeline/RESULTS.md](../eval/timeline/RESULTS.md))+`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py` (ground truth: the Clawd mixer's placement log).++- Event detection vs the 147 placed sound effects: recall 93.2% (96.4% of those louder than the bed, 89.1% of those+ under it; 64.7% of the ones more than 6 dB under the music).+- Sync against the picture cue frames: for the 140 of the 153 cues with an attack-aligned placement, measured offsets+ within 5 ms of the true placement offset for 84%, within one frame for 96%. The "without a hit" list is the weak+ part: both cues it named on Clawd were sounded whooshes. Beat grid of the 120 BPM score found at 120.0 BPM.+- Event tags are hints: right kind for about 1 in 2 exposed events and 1 in 4 events buried in a bed; whooshes 56%,+ keyboard presses 58-82%, grass footsteps under music 0%.+- Time: 60 s soundtrack 81 s, 93 s film 93 s (detail normal, 2 threads, 2-4 other processes busy).++## 4. Launch scorecard ([eval/launch/RESULTS.md](../eval/launch/RESULTS.md))+`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py`++- Leave-one-out: each of the 7 reference films vs a profile of the other six: 120/140 reference-range metric checks in+ range (86%; the no-overfit baseline for min-max ranges of six is ~71%). Not a blind test: some rules were changed+ after the first leave-one-out run (listed in the results file). `critique` on one of these 7 films now detects it+ and scores it leave-one-out automatically.+- The Clawd film: with its real-recordings soundtrack "close" (21/23 metrics, p_real 0.94; flags a quiet first second+ and one loud stretch), with the old code-synthesized soundtrack "needs work" (critical: p_real 0.33, the+ code-synthesized windows listed; major: fades away instead of ending on a hit, sparse).+- Known-bad variants written before the run (quiet, hot, mono, muffled, late start, dead air, long fade, hole,+ squashed, SFX-only, ambience-only, synth bed, sine, pink noise) and how each is flagged: in the results file.++## 5. Interfaces+`.venv/Scripts/python.exe -m pytest tests/test_interfaces.py -q` (+ `--slow`)++- All 13 MCP tools exercised over a real stdio connection on real files (silence, clicks, mono/stereo, ogg, mp3, the+ 60 s soundtrack, an mp4 film); bad input returns readable errors, never a crash; every JSON part valid.+- Models stay loaded between MCP calls (the stdio handshake takes a few seconds; not benchmarked).+- The Gemini critique has never been called live (no key on this machine, and a live call uploads audio): every path+ is tested against a fake HTTP layer, including the key never being printed and uploads deleted after use.++## 6. The real-sound skill reproduces the Clawd soundtrack exactly+```+.venv/Scripts/python.exe -I skills/real-sound/scripts/mix.py --root "<film project>" --design audio/sounddesign.json \+ --cues out/cues_all.json --out out/repro/soundtrack.wav --stems out/repro/stems --report out/repro/mix_report.json \+ --cues-out out/repro/cues_sounded.json --cache out/repro/cache+```+Bit-identical to the shipped `out/audio/soundtrack.wav` (sha1 `b48c5e8872fe96765624f88db18a0b49a0a05d68`), with an empty+decode cache, and `mix_report.json` byte-identical. (`--cache` must point to a writable folder.)++## 7. Tests+`.venv/Scripts/python.exe -m pytest -q` (fast; synthetic fixtures, no network) and `--slow` (loads the models / long+end-to-end runs). At release: 128 passed, 14 skipped (slow) in ~65 s; the 14 slow tests also pass (~3 min).diff --git a/docs/launch-video-sound.md b/docs/launch-video-sound.mdnew file mode 100644index 0000000..6c4d6be--- /dev/null+++ b/docs/launch-video-sound.md@@ -0,0 +1,220 @@+# Launch-film sound: the playbook for coding agents++You cannot hear the film you are making. This page tells you what professional launch films do with sound, how to+build that from real recordings, and how to check your result with earshot until the numbers say it is there.++How to read it:++- **[measured]** = measured by earshot on the 7 professional launch films the user chose as the quality bar (modern+ SaaS / AI product launches and motion reels, web-delivered MP4s with 128 kbps AAC audio). Numbers come from+ `earshot/data/launch_profile.json`; the commands and the leave-one-out check are in `eval/launch/RESULTS.md`.+ Seven films are a small sample: treat the numbers as "what pros did here", not as laws.+- **[practice]** = established professional practice for web video and trailer / motion-design sound. earshot did+ not measure these claims.++The references (anonymised as in the profile):++| id | style |+|---|---|+| ref_1 | AI video-search SaaS launch: white UI walkthrough, chat prompts and cursor, kinetic type, logo end card; voice-over (93 s) |+| ref_2 | AI workspace explainer: white minimal kinetic type, app icons, chat UI, photographic logo reveal; voice-over (29 s) |+| ref_3 | editorial kinetic collage: grainy paper, typewriter text, photo cut-outs, pixel icons, fast cuts; voice-over (20 s) |+| ref_4 | fintech brand film: bold black / green kinetic typography, grids, numerals, flicker cuts, logo stings (32 s) |+| ref_5 | AI image-generation launch: gradients, typed prompt, photo collage, sound design with hard silences (30 s) |+| ref_6 | creator-commerce SaaS: blue UI walkthrough, notifications, cursor clicks, tagline end card; voice-over (49 s) |+| ref_7 | gaming motion-design reel: logo stings, 3D explosions, very fast cuts, brickwall-loud music (42 s) |++## The short version++1. **Music leads.** Music runs under (nearly) the whole film: AudioSet hears it in 89-100 % of 5 s windows in 6 of 7+ films [measured]. Pick the track first and cut the picture to it [practice].+2. **Loud, steady body.** Median -14.8 LUFS integrated; the loudest 3 s sit only 1.1-3.5 LU above the integrated+ level (median 2.3 LU); loudness range 1.7-12.2 LU, median 5.5 [measured].+3. **Start at once.** First sound within 0.04-0.48 s (median 0.13 s); 6 of 7 open within -1.8..+2.2 LU of the+ integrated level [measured]. No silent cold open.+4. **Land the ending.** The last 3 s of sound reach body level (median +0.2 LU vs integrated, range -2.2..+5.6);+ fades are short (0.08-2.54 s, median 0.27 s); dead air after the sound is short (median 0.04 s) [measured].+5. **Dense rhythm, a few big accents.** 2.5-8.2 onsets per second (median 5.7) but only ~1.2-1.7 salient hits per+ second (6 of 7 films) [measured]: a busy groove with a handful of moments that jump out.+6. **Not a hit on every cut.** Sound on visual events is no better than chance in most references (details below)+ [measured]. Dense music covers the cuts; deliberate accents go on the moments that matter (reveal, logo, key UI+ action) [practice].+7. **Real recordings, produced music.** Synthesized beeps and oscillator whooshes read as cheap next to films like+ these [practice]. Build from licensed recordings (`skills/real-sound/`) and check with `earshot realness`.+8. **Master for the web.** -14 LUFS integrated, true peak <= -1 dBTP on the master [practice]. 6 of 7 references+ clip after AAC decoding (true peak +0.1..+2.0 dBTP) [measured]: a common flaw, do not copy it.++## 1. Music leads the edit++- [measured] Music presence (AudioSet 'Music' >= 0.3 per 5 s window; music under voice or SFX scores 0.4-0.7, a+ bare track ~0.9): 100 % in 5 films, 89 % in the reel (all but its last, 1.8 s window), 67 % in ref_5, whose+ 5-15 s stretch is sound design with hard silences.+- [measured] Tempo 108-136 BPM, median 123 (librosa beat tracker; double / half tempo errors are possible).+- [practice] Choose the track before animating; place section changes, the reveal and the logo on downbeats or bar+ lines; edit music on bar lines with short crossfades, never mid-phrase. If the film is a different length than the+ track, cut whole bars out of the middle, and keep the track's own ending.+- [practice] One track, or one composed edit of a track. Hopping between unrelated cues breaks the energy.++## 2. Loudness and peaks for web delivery++- [measured] Integrated loudness -22.2 .. -7.3 LUFS, median -14.8 (4 of 7 inside -16..-9). The quiet outlier (ref_5,+ -22.2) is the sound-design film with silences; the loud one (ref_7, -7.3, LRA 3.3) is a brickwalled reel.+- [measured] True peak of the decoded AAC: -0.8 .. +2.0 dBTP; 6 of 7 above 0 dBTP, i.e. they clip on playback.+- [practice] YouTube normalises playback to about -14 LUFS and only turns loud uploads down (a quiet master stays+ quiet); Apple's players aim at about -16; X, LinkedIn and website autoplay players do not normalise at all. So: about -14 LUFS integrated (up to about -10+ for a punchy social cut), true peak <= -1 dBTP on the master (AAC / MP3 encoding adds up to ~1 dB of overshoot),+ limiter last in the chain.+- earshot: `lufs_i` (web window -16..-9 LUFS), `true_peak` (<= -1 dBTP for a lossless master, <= 0 dBTP for an already+ decoded MP4 / MP3), `lra`, `crest_lu`.++## 3. The arc: open strong, hold the body, land the ending++- [measured] Opening level (first 10 % of the film vs integrated): -1.8 .. +2.2 LU in 6 films; ref_4 builds from+ -17.5 LU over ~4 s. Where the loudest 3 s sit varies (0.03-0.88 of the duration); 5 of 7 put it after 60 %.+- [measured] Body: loudness range median 5.5 LU, loudest 3 s median +2.3 LU over integrated. The references stay+ loud; they do not make you wait for a climax.+- [measured] The median loudness arc (LU vs integrated at 0 %, 10 % .. 100 % of the film) is flat: -1.3, -0.8, +0.3,+ -0.3, -0.2, +0.2, 0.0, -0.4, +0.9, -0.3, then -7.4 at the very end (ring-out / fade); between 10 % and 90 % the+ p10-p90 band is -5.8..+2.0 LU. A story-film arc (quiet start, climax at 70-80 %) is the opposite: the Clawd film's+ real soundtrack starts at -9.8 LU and peaks at +3.9 LU. `earshot critique` prints your arc next to the median.+- [measured] Silent holes: none longer than 0.04 s except ref_5's deliberate silences (1.66 s).+- [practice] Shape the energy with arrangement (layers in / out, filter sweeps, a stop-down before the reveal) rather+ than with big level changes.+- earshot: `intro_lu`, `end_lu`, `peak_pos`, `first_sound_s`, `max_gap_s`.++## 4. Picture and sound: what the references actually sync++earshot detects visual events (hard cuts and big picture changes: a frame-difference spike >= 4x its surroundings, or a+mean change >= 15 % of the grey scale) and compares them with audio onsets and salient hits. "Chance" = the same events+shifted off the picture by fixed offsets, so it is what random edit points would score with that music.++| [measured] share of visual events with ... | median of 7 | chance median | films >= 10 points above their chance |+|---|---|---|---|+| an onset within +-100 ms | 81 % | 86 % | ref_3 88 vs 68, ref_7 60 vs 44 |+| a salient hit within +-100 ms | 29 % | 27 % | ref_1 46 vs 30, ref_3 53 vs 35 |+| an onset within +-1 frame | 44 % | 44 % | ref_3 47 vs 36, ref_5 38 vs 24 |+| a tracked beat within +-70 ms | 22 % | 28 % | ref_4 40 vs 28, ref_5 38 vs 25, ref_6 52 vs 31 |++For comparison, the code-made Clawd film (its sound placed on picture cues by a script) scores 38 % vs 10 % chance+within +-1 frame and 69 % vs 25 % on the beat grid: earshot does see tight sync when it is there.++What that means: in these films the music is so dense (median 5.7 onsets/s) that almost every cut has *some* sound+under it, and they do **not** put a measurable hit on every cut. Some lock part of the edit to the beat. Caveat:+earshot measures the final mix; quiet SFX under loud music can be present without being separable.++- [measured] 4 of 7 use stop-downs: the sound drops >= 10 dB for 0.15-1.5 s and comes back on a hit (1.2-18.3 per+ minute).+- [measured] Visual events: 0.3-1.6 per second (median 0.63).+- [practice] **Whoosh into transitions**: the whoosh peaks on the cut (or on the fastest frame of the move) and starts+ 0.2-0.6 s before it; match its length to the move.+- [practice] **Riser -> stop-down -> impact** for the big reveal: a 1-4 s riser, cut everything for 0.1-0.5 s, then an+ impact (with a sub drop) exactly on the reveal frame. Use it once or twice per film, not on every scene.+- [practice] **Accents with priority**: the reveal, the logo, the key UI action and the section changes get a+ designed sound; small motion rides on the music.+- [practice] For a sparse soundtrack (no dense music) every cut needs sound: `earshot critique` scores the share of+ visual events with an onset (a new attack) within +-100 ms (`cuts_with_sound`, the references' definition) and+ lists the events with no sound at all - no onset, no whoosh / riser peak, no level peak - by the same matcher as+ `earshot timeline` (an action when `cuts_with_sound` fails, a tip otherwise). Soft build-ups can still be missed:+ look at the sheet before adding a sound. Align samples on their attack, to the frame.+- earshot: `cuts_with_sound` (scored), `cut_hit_frac`, `cut_frame_sync`, `cuts_on_beat`, `stopdowns_min` (context).++## 5. UI foley (clicks, typing, notifications)++- [practice] One click per on-screen click, on the frame the cursor presses (not when the UI reacts). Typing: one key+ sound per typed character, from a multi-take recording so no two consecutive keys are the same sample; slightly+ irregular timing. Notifications: short, soft, tonal; avoid alarm-like beeps.+- [practice] Keep UI foley short, bright (2-6 kHz) and low: it should read as detail under the music, ~10-15 dB below+ the music peaks, panned toward where it happens on screen.+- earshot: `earshot find typing.wav "a single soft key press"` to cut takes; `earshot match` to check a sample is what+ its name says; the critique's `onset_rate` tells you if the track is too sparse.++## 6. Logo sting and endings++- [measured] The final moment lands at body level (median +0.2 LU); fade / ring-out median 0.27 s, longest 2.54 s;+ the strongest hit of the last 4 s rings out 2.0-3.6 s in 5 films; ref_7 ends without a final hit.+- [practice] End on a designed sting: a hit plus a tonal tail (2-4 s), with the music resolving into it or stopping+ for it, finishing within ~0.5 s of the last frame. Avoid long fade-outs and dead air.+- earshot: `final_lu`, `fade_s`, `end_silence_s` (scored), `tail_s` (context).++## 7. Frequency space++- [measured] Energy share per band (median, dB of the total): sub 20-60 Hz -6.6, low 60-250 Hz -4.2, low-mid+ 250 Hz-1 kHz -5.3, mid 1-4 kHz -12.4, presence 4-8 kHz -17.8, air 8-16 kHz -23.6; brightness (median+ magnitude-weighted spectral centroid) 1169-3633 Hz, median 1966 Hz. Bass-led mixes, as expected for music-led films.+ (Above 16 kHz is not compared: AAC cuts it.)+- [measured] Stereo: side 14.5-5.0 dB below mid (median -8.5 dB), L/R correlation 0.52-0.93.+- [practice] Give every layer its own place: high-pass SFX and ambience (unless they are impacts), keep one element+ in charge of the sub at a time, dip the music 2-5 kHz and duck it 3-6 dB under voice, keep UI sounds out of the+ music's lead range. Check the mono fold-down; phones are mono.+- earshot: `band_*`, `side_to_mid_db` (scored minor; a mono mix, identical left and right, counts as -40 dB and is+ flagged), `brightness_hz`, `correlation` (context).++## 8. Density and restraint++- [measured] 2.5-8.2 onsets per second; salient hits ~1.2-1.7 per second in 6 films (the brickwalled reel 0.1).+- [practice] More sounds is not better. Sound the moments that carry meaning, let the music carry the rest, leave+ room (a stop-down, a breath before the logo). A cluttered SFX layer on top of busy music turns into noise.+- earshot: `onset_rate`, `hit_rate` (scored both ways: too sparse and too busy; salient hits need at least 0.05 /s,+ half the sparsest reference).++## 9. Voice-over++- [measured] 4 of 7 have a voice-over (AudioSet hears speech in 83-100 % of windows); 3 are music-only.+- [practice] If you use VO: duck the music 3-6 dB under it (sidechain or automation), keep SFX out of 1-4 kHz while it+ speaks, and never put a big hit on top of a key word.++## 10. Real recordings, not synthesis++- [practice] Code-synthesized "random" sounds (oscillator beeps, filtered-noise whooshes) are the fastest way to+ sound amateur next to films like these. Use licensed real recordings and produced music, with a licence audit and+ credits: `skills/real-sound/`.+- [measured] The Clawd film scores p_real 0.94 with its real-recordings soundtrack and 0.33 with the retired+ synthesized one (eval/realness/RESULTS.md, application files; eval/launch/RESULTS.md); that line is what separates them in the critique: the real one+ reads "close" (90 %), the synthesized one "needs work" (80 %, 1 critical). The structural metrics alone rate both+ close. Not an independent test: the realness model was trained on related material (eval/launch/RESULTS.md).+- earshot: `earshot realness mix.wav` gives p_real, a verdict and the 4 s windows that sound synthesized;+ `earshot critique` uses the same thresholds: the film's mean p_real against the file thresholds (ok at "real or+ pro-produced", critical below 0.5, where synthesized is the more likely reading, minor in between; a minor miss+ with no window leaning synthesized is only a tip), and its listed 4 s windows against the window threshold, as+ `earshot realness` lists them. A finished mix is harder to judge than its sources:+ run `earshot realness` on each stem.++## Checklist (run it, read it, fix, repeat)++```+earshot critique film.mp4 # scorecard + concrete fixes with timestamps (start here)+earshot critique film.mp4 --fast # quick pass, no AudioSet / realness: a PARTIAL verdict+earshot critique mix.wav --video film.mp4 # a separate soundtrack checked against its picture+earshot timeline film.mp4 # listening transcript: what sounds when, next to the cuts+earshot sheet film.mp4 -o out/sheet.png # annotated picture of the soundtrack (open the PNG)+earshot realness mix.wav # which 4 s windows sound code-synthesized+earshot profile my_refs/ -o my_refs.json # measure your own reference films ...+earshot critique film.mp4 --profile my_refs.json # ... and score against them instead of the bundled 7+```++1. Pick the music first; cut the picture to it. Check `music_frac` (whole film) and `tempo`.+2. Lay the accents: reveal (riser -> stop-down -> impact), logo sting, key UI actions, section changes. Check+ `hit_rate`, `stopdowns_min` and the critique's list of the biggest visual events without a hit.+3. Add UI foley on the on-screen interactions, aligned to the frame. Check `cuts_with_sound` and `onset_rate`.+4. Shape the arc: sound from 0:00, steady body, ending at body level. Check `first_sound_s`, `intro_lu`, `lra`,+ `crest_lu`, `final_lu`, `fade_s`, `end_silence_s`, `max_gap_s`.+5. Mix: balance bands and width. Check `band_*`, `side_to_mid_db`.+6. Master: about -14 LUFS, true peak <= -1 dBTP. Check `lufs_i`, `true_peak`.+7. Run `earshot critique` (without `--fast`) on the final MP4 and fix every critical and major line, not just the+ verdict: one structural defect on an otherwise good mix usually keeps it "close" (eval/launch/RESULTS.md, known-bad+ mixes: a 12 s fade-out, 4 s of dead air or a mono fold each stay "close" but are flagged with the fix). The+ verdict is coarse; the FIX FIRST lines are the signal. "launch-grade" or "close" with no open critical / major+ line is the goal; the 7 references score launch-grade (1), close (5) and needs work (1: the -22 LUFS+ sound-design film) in the leave-one-out check.++## What the numbers cannot tell you++- Taste: whether the music fits the brand, whether a whoosh sounds expensive. Use `earshot describe` / `match` /+ `compare` against good reference recordings and listen-proxy with `earshot sheet`.+- Sync of individual layers inside a dense mix (see section 4).+- Over-compression: a mix crushed by a 20:1 compressor and a limiter is not flagged, because two references are as+ dense (LRA 1.7 / 3.3 LU, loudest 3 s 1.1-1.2 LU above integrated). Listen-proxy with `earshot sheet` if in doubt.+- Clips under 10 s: loudness range, crest and opening / closing / final levels are not scored (they would measure the+ clip, not a film); the references are 20-93 s films.+- Styles outside the profile: for a cinematic trailer or a calm explainer, build your own profile from references+ of that style (`earshot profile`).diff --git a/docs/media/launch_frames.png b/docs/media/launch_frames.pngnew file mode 100644index 0000000..b13121bBinary files /dev/null and b/docs/media/launch_frames.png differdiff --git a/docs/media/launch_sheet.png b/docs/media/launch_sheet.pngnew file mode 100644index 0000000..5562a76Binary files /dev/null and b/docs/media/launch_sheet.png differdiff --git a/earshot/__init__.py b/earshot/__init__.pynew file mode 100644index 0000000..e1a76d3--- /dev/null+++ b/earshot/__init__.py@@ -0,0 +1,30 @@+"""earshot: ears for coding agents.++Python API (models load lazily on first use and stay warm):++ import earshot+ earshot.describe('whoosh.wav')+ earshot.match('key.ogg', 'a deep thocky mechanical keyboard press', alts=['a clicky blue switch'])+ earshot.find('typing.ogg', 'a single isolated key press')+ earshot.timeline('soundtrack.wav', video='film.mp4')++or explicitly: `from earshot import api` / `from earshot.describe import describe`.+"""+import importlib++__version__ = '0.1.0'++# modules that are callable as their main function (see _callable.py)+_CALLABLE = {'describe', 'match', 'compare', 'realness', 'timeline', 'sheet', 'critique', 'profile'}+# functions whose name is not a module name+_FUNCS = {'rank': ('search', 'rank'), 'find': ('search', 'find'), 'ab': ('compare', 'ab'),+ 'measure': ('features', 'measure'), 'llm_critique': ('llm', 'critique')}+++def __getattr__(name):+ if name in _CALLABLE:+ return importlib.import_module(f'earshot.{name}')+ if name in _FUNCS:+ mod, fn = _FUNCS[name]+ return getattr(importlib.import_module(f'earshot.{mod}'), fn)+ raise AttributeError(name) # anything else: normal submodule importdiff --git a/earshot/__main__.py b/earshot/__main__.pynew file mode 100644index 0000000..56dada6--- /dev/null+++ b/earshot/__main__.py@@ -0,0 +1,4 @@+import sys+from .cli import main++sys.exit(main())diff --git a/earshot/_callable.py b/earshot/_callable.pynew file mode 100644index 0000000..2878db5--- /dev/null+++ b/earshot/_callable.py@@ -0,0 +1,15 @@+"""Make a module callable as its main function, so `earshot.describe(path)` works whether or not the submodule+`earshot.describe` has been imported (Python binds a submodule over a package attribute of the same name)."""+import sys+import types+++def callable_module(name, fn_name=None):+ mod = sys.modules[name]+ fn = fn_name or name.rsplit('.', 1)[-1]++ class CallableModule(types.ModuleType):+ def __call__(self, *args, **kwargs):+ return getattr(self, fn)(*args, **kwargs)++ mod.__class__ = CallableModulediff --git a/earshot/api.py b/earshot/api.pynew file mode 100644index 0000000..8fda9a7--- /dev/null+++ b/earshot/api.py@@ -0,0 +1,31 @@+"""The Python API in one namespace (models load lazily on first use and stay warm in this process).++ from earshot import api+ api.describe('whoosh.wav')+ api.match('key.ogg', 'a deep thocky mechanical keyboard press', alts=['a clicky blue switch'])+ api.find('typing.ogg', 'a single isolated key press')+ api.timeline('soundtrack.wav', video='film.mp4')+"""+import importlib++_FUNCS = {+ 'describe': ('describe', 'describe'), 'match': ('match', 'match'), 'rank': ('search', 'rank'),+ 'find': ('search', 'find'), 'compare': ('compare', 'compare'), 'ab': ('compare', 'ab'),+ 'realness': ('realness', 'realness'), 'realness_many': ('realness', 'realness_many'), 'timeline': ('timeline', 'timeline'), 'sheet': ('sheet', 'sheet'),+ 'critique': ('critique', 'critique'), 'profile': ('profile', 'profile'), 'measure': ('features', 'measure'),+ 'llm_critique': ('llm', 'critique'), 'doctor': ('doctor', 'check'),+}+++def _fn(name):+ mod, fn = _FUNCS[name]+ return getattr(importlib.import_module(f'earshot.{mod}'), fn)+++def __getattr__(name):+ if name in _FUNCS:+ return _fn(name)+ raise AttributeError(name)+++__all__ = sorted(_FUNCS)diff --git a/earshot/audio.py b/earshot/audio.pynew file mode 100644index 0000000..6836067--- /dev/null+++ b/earshot/audio.py@@ -0,0 +1,247 @@+"""Audio I/O shared by every earshot module.++Any file ffmpeg can read works (wav/flac/ogg/mp3/m4a and the audio track of mp4/mov/webm). Audio is decoded once to+float32 at 48 kHz and cached under EARSHOT_HOME/decoded (keyed on path, size, mtime and 16 sampled 4 KB blocks of the+content, so a same-size replacement that kept its timestamp is still noticed), so repeated calls are fast.++ x = load('kick.ogg') # (n, 2) float32 at 48 kHz+ m = load('kick.ogg', mono=True) # (n,)+ seg = load('long.wav', start=1.2, end=1.8, mono=True)++Levels: the stereo decode keeps true sample levels (a mono file is duplicated to both channels at unity gain), so+measure peaks and loudness on it (features.measure does). mono=True is ffmpeg's downmix, which sums a stereo source+at -3 dB per channel: correlated stereo reads up to +3 dB hotter than mean(L, R). It is the models' input convention+(their calibration was measured with it), so keep it for model input and timing, not for level readings.+"""+import hashlib+import json+import os+import subprocess++import numpy as np++from . import config++SR = 48000+AUDIO_EXTS = {'.wav', '.flac', '.ogg', '.oga', '.opus', '.mp3', '.m4a', '.aac', '.aif', '.aiff', '.wma'}+VIDEO_EXTS = {'.mp4', '.mov', '.m4v', '.webm', '.mkv', '.avi'}+MEDIA_EXTS = AUDIO_EXTS | VIDEO_EXTS+LOSSY = {'aac', 'mp3', 'mp2', 'opus', 'vorbis', 'wmav2', 'ac3', 'eac3'}+++class AudioError(RuntimeError):+ pass+++DECODE_VERSION = 2 # bump when decoded samples change (derived caches may key on it)+++def _sample_hash(path, size, blocks=16, block=4096):+ """sha1 of `blocks` evenly spaced 4 KB blocks (first and last included): cheap, and catches a same-size file+ swapped in with its old timestamp kept (cp -p, robocopy, unzip)"""+ h = hashlib.sha1()+ try:+ with open(path, 'rb') as fh:+ for off in sorted({int(i * max(0, size - block) / max(1, blocks - 1)) for i in range(blocks)}):+ fh.seek(off)+ h.update(fh.read(block))+ except OSError:+ return 'unreadable'+ return h.hexdigest()[:12]+++def _state(path):+ st = os.stat(path)+ return f'{os.path.abspath(path)}|{st.st_size}|{st.st_mtime_ns}|{_sample_hash(path, st.st_size)}'+++def _key(path, sr, ch):+ return hashlib.sha1(f'{_state(path)}|{sr}|{ch}|v{DECODE_VERSION}'.encode()).hexdigest()[:20]+++def replace_quietly(tmp, dst, tries=5):+ """os.replace(tmp, dst) for a cache write: retried briefly (Windows: another process may hold dst for a moment),+ then given up (tmp removed) - a cache write must never fail the call"""+ import time+ for attempt in range(tries):+ try:+ os.replace(tmp, dst)+ return True+ except OSError:+ time.sleep(0.05 * (attempt + 1))+ try:+ os.remove(tmp)+ except OSError:+ pass+ return False+++def _src_channels(path):+ try:+ out = subprocess.run(['ffprobe', '-v', 'error', '-select_streams', 'a:0', '-show_entries', 'stream=channels',+ '-of', 'csv=p=0', path], capture_output=True, text=True, timeout=60).stdout.strip()+ return int(out.splitlines()[0]) if out else 0+ except Exception: # noqa: BLE001+ return 0+++def file_id(path):+ """stable id of a file's content state (path + size + mtime + sampled content); used as a cache key by the models"""+ return hashlib.sha1(_state(path).encode()).hexdigest()[:20]+++def array_id(x):+ x = np.ascontiguousarray(x, dtype=np.float32)+ return hashlib.sha1(x.tobytes() + str(x.shape).encode()).hexdigest()[:20]+++def _ffmpeg_reason(err):+ """the last meaningful line of ffmpeg's stderr, without its [component @ 0x...] prefix, on one line"""+ import re+ lines = [re.sub(r'^\[[^\]]*@ [0-9a-fA-Fx]+\]\s*', '', ln).strip() for ln in err.splitlines()]+ lines = [ln for ln in lines if ln and not ln.lower().startswith(('error opening output', 'conversion failed'))]+ return (lines[-1] if lines else 'ffmpeg failed')[:200]+++def _decode(path, sr, ch):+ up = ch == 2 and _src_channels(path) == 1 # ffmpeg would upmix mono at -3 dB; duplicate at unity instead+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-vn', '-f', 'f32le', '-acodec', 'pcm_f32le',+ '-ac', '1' if up else str(ch), '-ar', str(sr), '-'], capture_output=True)+ err = p.stderr.decode(errors='replace')+ if p.returncode != 0 or not p.stdout:+ pr = probe(path)+ if not pr.get('probe_error') and pr.get('has_video') and not pr.get('codec'):+ raise AudioError(f'{path} has no audio track (a silent render? mux the soundtrack in first)')+ if p.returncode == 0 or 'does not contain any stream' in err:+ raise AudioError(f'{path}: no audio stream')+ raise AudioError(f'cannot decode {path}: {_ffmpeg_reason(err)}')+ x = np.frombuffer(p.stdout, np.float32).copy()+ if up:+ return np.repeat(x[:, None], 2, axis=1)+ return x.reshape(-1, ch) if ch > 1 else x+++def load(path, sr=SR, mono=False, start=None, end=None, cache=True):+ """decode `path` to float32 (n, 2) or (n,) when mono; optional [start, end) in seconds"""+ if not os.path.exists(path):+ raise AudioError(f'no such file: {path}')+ ch = 1 if mono else 2+ x = None+ npy = None+ if cache:+ config.ensure_dirs()+ npy = os.path.join(config.DEC_CACHE, _key(path, sr, ch) + '.npy')+ if os.path.exists(npy):+ try:+ x = np.load(npy)+ except Exception:+ x = None+ if x is None:+ x = _decode(path, sr, ch)+ if npy:+ tmp = f'{npy}.{os.getpid()}.tmp.npy'+ np.save(tmp, x)+ replace_quietly(tmp, npy)+ if start is not None or end is not None:+ a = max(0, int(round((start or 0.0) * sr)))+ b = len(x) if end is None else min(len(x), int(round(end * sr)))+ x = x[a:b]+ return x+++def to_mono(x):+ return x.mean(axis=1) if x.ndim == 2 else x+++def resample(x, sr_from, sr_to):+ if sr_from == sr_to:+ return x+ import soxr+ return soxr.resample(x, sr_from, sr_to, quality='HQ').astype(np.float32)+++def duration(path):+ return len(load(path, mono=True)) / SR+++def probe(path):+ """ffprobe summary: codec, sample rate, channels, kbps, duration, has_video"""+ try:+ out = subprocess.run(['ffprobe', '-v', 'error', '-show_entries',+ 'stream=codec_type,codec_name,sample_rate,channels,bit_rate,width,height,r_frame_rate:'+ 'stream_disposition=attached_pic:'+ 'format=duration,bit_rate', '-of', 'json', path],+ capture_output=True, text=True, timeout=60).stdout+ j = json.loads(out or '{}')+ except Exception as e: # noqa: BLE001+ return {'probe_error': str(e)[:200]}+ streams = j.get('streams') or []+ a = next((s for s in streams if s.get('codec_type') == 'audio'), {})+ v = next((s for s in streams if s.get('codec_type') == 'video'+ and not (s.get('disposition') or {}).get('attached_pic')), None) # cover art is not a picture track+ fmt = j.get('format') or {}+ br = a.get('bit_rate')+ if not (br and str(br).isdigit() and int(br) < 10_000_000): # Vorbis can report 2^32-1: use the container's rate+ br = fmt.get('bit_rate') if not v else None+ res = {'codec': a.get('codec_name'), 'src_rate': int(a.get('sample_rate') or 0), 'channels': int(a.get('channels') or 0),+ 'kbps': round(int(br) / 1000) if br and str(br).isdigit() else None, 'lossy': a.get('codec_name') in LOSSY,+ 'duration': round(float(fmt.get('duration') or 0), 3), 'has_video': v is not None}+ if v is not None:+ num, _, den = (v.get('r_frame_rate') or '0/1').partition('/')+ res['video'] = {'width': v.get('width'), 'height': v.get('height'),+ 'fps': round(float(num) / float(den or 1), 3) if float(den or 1) else None}+ return res+++def windows(x, win, hop, sr=SR, pad_last=False):+ """[(t0_seconds, segment)] over the first axis"""+ n, w, h = len(x), int(round(win * sr)), int(round(hop * sr))+ out = []+ if n <= w:+ return [(0.0, x)]+ for s in range(0, n - w + 1, h):+ out.append((s / sr, x[s:s + w]))+ if pad_last and (n - w) % h:+ out.append(((n - w) / sr, x[n - w:]))+ return out+++def list_media(paths, recursive=True, exts=MEDIA_EXTS):+ """expand files / folders / glob results into a sorted list of media files"""+ if isinstance(paths, str):+ paths = [paths]+ out = []+ for p in paths:+ if os.path.isdir(p):+ for root, dirs, files in os.walk(p):+ dirs.sort()+ for f in sorted(files):+ if os.path.splitext(f)[1].lower() in exts:+ out.append(os.path.join(root, f))+ if not recursive:+ break+ elif os.path.isfile(p):+ out.append(p)+ else:+ import glob+ out += sorted(g for g in glob.glob(p, recursive=True) if os.path.splitext(g)[1].lower() in exts)+ seen, res = set(), []+ for p in out:+ k = os.path.normcase(os.path.abspath(p))+ if k not in seen:+ seen.add(k)+ res.append(p)+ return res+++def check_paths(paths, what='paths'):+ """raise AudioError naming every given path that yields no media file (a typo must not vanish from a batch)"""+ paths = [paths] if isinstance(paths, str) else list(paths or [])+ bad = [f"{p} ({'no such file or folder' if not os.path.exists(p) else 'no audio or video files in it'})"+ for p in paths if not list_media([p])]+ if bad:+ raise AudioError(f'{what}: nothing playable in ' + ', '.join(bad[:5]) + (' ...' if len(bad) > 5 else ''))+++def db(v, floor=1e-12):+ return float(20 * np.log10(max(float(v), floor)))diff --git a/earshot/cli.py b/earshot/cli.pynew file mode 100644index 0000000..2f8468c--- /dev/null+++ b/earshot/cli.py@@ -0,0 +1,100 @@+"""earshot command line. Every subcommand lives in its own module, which exposes add_cli(subparsers).++ earshot [--threads N] <command> [args] [--json] [--out result.json] [--threads N]++Run `earshot -h` for the list and `earshot <command> -h` for details. Only the module of the requested command is+imported (fast start); `-h` imports them all to list them. Errors: one line on stderr, 'earshot <command>: error: ...',+exit 1 (with --json, describe / match / rank / find / compare / ab also print it as JSON on stdout).+"""+import argparse+import contextlib+import importlib+import os+import sys++# command -> module (order = order in --help)+COMMANDS = {+ 'describe': 'describe', # what does this sound like? tags + measurements+ 'match': 'match', # how well does a sound match a text description (calibrated)+ 'rank': 'search', # rank many files by a description+ 'find': 'search', # find the best moments inside a long recording (sample trims)+ 'compare': 'compare', # similarity to reference recordings+ 'ab': 'compare', # A/B two sounds+ 'realness': 'realness', # real recording vs code-synthesized sound+ 'timeline': 'timeline', # listen through a whole soundtrack or film: events, tags, sync, loudness+ 'sheet': 'sheet', # annotated picture of a soundtrack (spectrogram + events + frames) for agents with eyes+ 'critique': 'critique', # launch-video sound scorecard against a profile of professional references+ 'profile': 'profile', # build a reference profile from your own reference films+ 'forge': 'forge', # make sounds in code and improve them by listening+ 'measure': 'features', # objective measurements only (no models)+ 'llm': 'llm', # optional audio-LLM critique (only with your own API key + consent)+ 'mcp': 'mcp_server', # run earshot as an MCP server for coding agents+ 'doctor': 'doctor', # check the install, models and CPU settings+}+++def _parser():+ ap = argparse.ArgumentParser(prog='earshot', description='Ears for coding agents: describe, match, search, compare, '+ 'time-align and critique sound files you cannot hear.')+ ap.add_argument('--threads', type=int, help='torch CPU threads (default: EARSHOT_THREADS or min(4, cores/2))')+ return ap+++def _register(sub, modules):+ failed = {}+ done = set()+ for mod in modules:+ if mod in done:+ continue+ done.add(mod)+ try:+ importlib.import_module(f'earshot.{mod}').add_cli(sub)+ except Exception as e: # a broken module must not break the others+ failed[mod] = f'{type(e).__name__}: {e}'+ return failed+++def main(argv=None):+ argv = sys.argv[1:] if argv is None else list(argv)+ ap = _parser()+ sub = ap.add_subparsers(dest='command', metavar='<command>')+ cmd = next((a for a in argv if not a.startswith('-')), None)+ if cmd in COMMANDS:+ failed = _register(sub, [COMMANDS[cmd]])+ if failed:+ print(f'earshot: the "{cmd}" command is unavailable: {failed[COMMANDS[cmd]]}', file=sys.stderr)+ return 2+ else:+ failed = _register(sub, list(COMMANDS.values()))+ for m, e in failed.items():+ print(f'(earshot: module {m} unavailable: {e})', file=sys.stderr)+ args = ap.parse_args(argv)+ if args.threads:+ os.environ['EARSHOT_THREADS'] = str(args.threads)+ if not getattr(args, 'func', None):+ ap.print_help()+ return 0+ if args.command != 'mcp': # a file name outside the console code page must never crash a print+ with contextlib.suppress(Exception):+ sys.stdout.reconfigure(errors='backslashreplace')+ try:+ return args.func(args) or 0+ except Exception as e: # readable one-line errors for agents; EARSHOT_DEBUG=1 shows the traceback+ if os.environ.get('EARSHOT_DEBUG'):+ raise+ from .report import error_line+ error_line(args.command, f'{type(e).__name__}: {e}')+ return 1+++def common(p):+ """flags shared by every command"""+ p.add_argument('--json', action='store_true', help='print the full result as JSON')+ p.add_argument('--out', help='also write the JSON result to this file')+ p.add_argument('--threads', type=int, default=argparse.SUPPRESS,+ help='torch CPU threads (also accepted before the command; default EARSHOT_THREADS or min(4, cores/2))')+ return p+++if __name__ == '__main__':+ sys.exit(main())diff --git a/earshot/compare.py b/earshot/compare.pynew file mode 100644index 0000000..fb25c00--- /dev/null+++ b/earshot/compare.py@@ -0,0 +1,313 @@+"""Compare sounds with each other and with reference recordings: nearest references, typicality, A/B in words.++ earshot compare candidate.ogg --refs refs/approved_keys/ [--start 1.2 --end 1.6]+ earshot ab take1.ogg take2.ogg --query "a deep thocky key press"+ earshot ab pack.ogg pack.ogg --a-start 1.217 --a-end 1.615 --b-start 7.881 --b-end 8.285 # two trims (find output)+ earshot ab mine.wav pro.wav --refs refs/launch_whooshes/+ from earshot.compare import compare, ab++compare: CLAP cosine to every reference, the k nearest, and typicality = how close the file is to its k nearest+references compared with how close each reference is to its own k nearest neighbours in the set (z and percentile:+z >= -1 fits the set, z < -2 is an outlier). Feature level: robust z of each measurement (loudness, brightness,+attack, tail, low end, noise floor, ...) against the references' median / spread, in words.+ab: which of two sounds is closer to a description (calibrated z of each) and/or to the references, how similar they+are to each other, and how B differs from A in measurable terms (only differences big enough to hear; 'closer' is+None when the z difference is below 0.3).+Trims: compare --start/--end and ab --a-start/--a-end/--b-start/--b-end judge only that part of a file (both the+CLAP embedding and the measurements), so a take found by `earshot find` in a multi-take pack can be A/B-ed alone;+without them the whole file is compared (every take plus the silence between them).+Measurements are trim-proof, so two trims of the same sample read as the same: loudness = gated integrated LUFS of+the sound padded to at least 0.4 s (silence before / after it does not count), sounding length = first to last+10 ms frame within 40 dB of the loudest, noise floor only for sounds of 1 s or more (floored at -120 dBFS), width+floored at -40 dB side/mid (= mono). Unreadable files are skipped and listed under 'skipped'.+"""+import math+import os++import numpy as np++from . import audio, features, vocab+from .cli import common++LOG = {'brightness_hz', 'bandwidth_hz', 'attack_ms', 'decay_ms', 'length_s'}+MIN_SCALE = {'loudness_lufs': 1.0, 'peak_dbfs': 1.0, 'noise_floor_dbfs': 2.0, 'low_end_db': 1.0, 'width_db': 2.0,+ 'brightness_hz': 0.05, 'bandwidth_hz': 0.05, 'attack_ms': 0.1, 'decay_ms': 0.1, 'length_s': 0.05}+# (key, threshold, kind) in order of how much they matter to the ear+DIFFS = [('loudness_lufs', 1.5, 'db'), ('brightness_hz', 1.15, 'ratio'), ('attack_ms', 1.5, 'ratio'),+ ('decay_ms', 1.5, 'ratio'), ('low_end_db', 3.0, 'db'), ('length_s', 1.25, 'ratio'),+ ('noise_floor_dbfs', 6.0, 'db'), ('width_db', 6.0, 'db'), ('bandwidth_hz', 1.4, 'ratio')]+WORDS = {'loudness_lufs': ('louder', 'quieter'), 'brightness_hz': ('brighter', 'darker'),+ 'bandwidth_hz': ('more top end', 'less top end'), 'attack_ms': ('a softer attack', 'a sharper attack'),+ 'decay_ms': ('a longer tail', 'a shorter tail'), 'noise_floor_dbfs': ('noisier', 'cleaner'),+ 'length_s': ('longer', 'shorter'), 'low_end_db': ('more low end', 'less low end'),+ 'width_db': ('wider', 'narrower'), 'peak_dbfs': ('hotter peaks', 'lower peaks')}+WITH = {'attack_ms', 'decay_ms', 'low_end_db', 'bandwidth_hz'}+LABEL = {'brightness_hz': 'brightness', 'bandwidth_hz': 'bandwidth', 'noise_floor_dbfs': 'noise floor',+ 'width_db': 'side/mid', 'low_end_db': 'energy below 250 Hz', 'loudness_lufs': 'loudness',+ 'length_s': 'sounding length'}+MONO_DB = -40.0+CLOSE_DZ = 0.3+++def _hz(f):+ return f'{f / 1000:.1f} kHz' if f >= 1000 else f'{f:.0f} Hz'+++def _ms(v):+ return f'{v / 1000:.1f} s' if v >= 2000 else f'{v:.0f} ms'+++def active_lufs(x):+ """gated integrated loudness of the sound padded to >= 0.4 s: trailing / leading silence does not change it"""+ x = np.asarray(x, np.float32)+ n = int(0.4 * audio.SR)+ if len(x) < n:+ x = np.concatenate([x, np.zeros((n - len(x),) + x.shape[1:], np.float32)])+ return round(features.integrated_lufs(x), 1)+++def active_length(x, floor_db=40.0):+ """seconds from the first to the last 10 ms frame within floor_db of the loudest frame"""+ e = features.frame_rms(audio.to_mono(np.asarray(x, np.float32)), int(0.01 * audio.SR))+ on = np.nonzero(20 * np.log10(e) >= 20 * np.log10(e.max()) - floor_db)[0]+ return round(min(len(x) / audio.SR, (on[-1] - on[0] + 1) * 0.01), 3)+++def feats(m, x):+ """the comparable, trim-proof measurements of a features.measure() dict and its audio x"""+ b = m.get('bands_db') or {}+ sm, nf = m.get('side_to_mid_db'), m.get('noise_floor_dbfs')+ f = {'loudness_lufs': active_lufs(x), 'peak_dbfs': m.get('peak_dbfs'), 'brightness_hz': m.get('brightness_hz'),+ 'bandwidth_hz': m.get('bandwidth_hz'), 'attack_ms': m.get('attack_ms'), 'decay_ms': m.get('decay_ms'),+ 'length_s': active_length(x), 'duration': m.get('duration'),+ 'noise_floor_dbfs': max(nf, -120.0) if nf is not None and m['duration'] >= 1.0 else None,+ 'width_db': MONO_DB if sm is None else max(sm, MONO_DB)}+ if 'sub' in b and 'low' in b:+ f['low_end_db'] = round(10 * math.log10(10 ** (b['sub'] / 10) + 10 ** (b['low'] / 10)), 1)+ return {k: v for k, v in f.items() if v is not None}+++def _val(k, v):+ return math.log(max(v, 1e-3)) if k in LOG else v+++def _fmt(k, v):+ if k in ('brightness_hz', 'bandwidth_hz'):+ return _hz(v)+ if k in ('attack_ms', 'decay_ms'):+ return _ms(v)+ if k in ('length_s', 'duration'):+ return f'{v:.2f} s'+ if k == 'width_db' and v <= MONO_DB:+ return 'mono'+ return f'{v:.0f} ' + ('LUFS' if k == 'loudness_lufs' else 'dBFS' if k in ('noise_floor_dbfs', 'peak_dbfs') else 'dB')+++def _phrase(k, up, a_txt, b_txt):+ lab = LABEL.get(k)+ return ('with ' if k in WITH else '') + WORDS[k][0 if up else 1] + f" ({(lab + ' ') if lab else ''}{b_txt} vs {a_txt})"+++def differences(fa, fb, limit=4):+ """how B differs from A in words, from two feats() dicts: the `limit` most important differences big enough to+ hear (level, brightness, attack, tail, low end, length, noise, width, top end)"""+ out = []+ for k, thr, kind in DIFFS:+ a, b = fa.get(k), fb.get(k)+ if a is None or b is None:+ continue+ if kind == 'db':+ if abs(b - a) < thr:+ continue+ out.append((k, f'{abs(b - a):.0f} dB ' + WORDS[k][0 if b > a else 1] if k == 'loudness_lufs'+ else _phrase(k, b > a, _fmt(k, a), _fmt(k, b))))+ elif min(a, b) > 0 and max(a, b) / min(a, b) >= thr and not (k == 'attack_ms' and abs(a - b) < 3):+ out.append((k, _phrase(k, b > a, _fmt(k, a), _fmt(k, b))))+ out = out[:limit]+ parts = [p for _, p in out if not p.startswith('with ')] + [p for _, p in out if p.startswith('with ')]+ if not parts:+ return [], 'no measurable difference big enough to hear (level, brightness, envelope, low end, noise)'+ return [k for k, _ in out], parts[0] if len(parts) == 1 else ', '.join(parts[:-1]) + ' and ' + parts[-1]+++def measure_path(p, start=None, end=None):+ x = audio.load(p, start=start, end=end)+ return feats(features.measure(x, light=True), x)+++def _name(p, start=None, end=None):+ """file name plus the trim, when one is given"""+ from .report import fmt_t+ rng = f' [{fmt_t(start or 0.0)}-{fmt_t(end) if end is not None else "end"}]' if start is not None or end is not None \+ else ''+ return os.path.basename(p) + rng+++def _typicality(a_x, A_r, k):+ """(mean cos of x to its k nearest refs, z, percentile) vs the refs' own k-nearest-neighbour similarity"""+ S = A_r @ A_r.T+ np.fill_diagonal(S, -np.inf)+ kk = max(1, min(k, len(A_r) - 1))+ ref_s = np.sort(S, axis=1)[:, ::-1][:, :kk].mean(1)+ sx = float(np.sort(A_r @ a_x)[::-1][:kk].mean())+ sd = float(ref_s.std()) + 1e-6+ return sx, (sx - float(ref_s.mean())) / sd, 100 * float((ref_s <= sx).mean())+++def _ordinal(v):+ n = int(round(v))+ return f"{n}{'th' if 10 <= n % 100 <= 20 else {1: 'st', 2: 'nd', 3: 'rd'}.get(n % 10, 'th')}"+++def _fit_words(z):+ if z >= -1:+ return 'fits the set'+ if z >= -2:+ return 'somewhat unusual for the set'+ return 'an outlier: unlike the references'+++def _refs(refs, exclude=()):+ """(readable reference files minus `exclude`, [{path, error}] skipped)"""+ audio.check_paths(refs, 'refs')+ ex = {os.path.normcase(os.path.abspath(p)) for p in exclude}+ return vocab.readable([p for p in audio.list_media(refs) if os.path.normcase(os.path.abspath(p)) not in ex])+++def _skipped_text(bad):+ return f' Skipped {len(bad)} unreadable file(s): ' + ', '.join(os.path.basename(b['path']) for b in bad[:3]) + \+ ('...' if len(bad) > 3 else '') + '.' if bad else ''+++def compare(path, refs, k=5, start=None, end=None):+ """nearest references, typicality vs the reference set, feature-level differences; start / end (s) = judge only+ that part of `path`"""+ vocab.check(path, start, end)+ R, bad = _refs(refs, exclude=[path])+ if len(R) < 2:+ return {'summary': f'need at least 2 readable reference files, found {len(R)} in {refs}.' + _skipped_text(bad),+ 'path': path, 'n_refs': len(R), 'nearest': [], 'skipped': bad}+ A = vocab.embed_files([path] + R, ranges=[(start, end)] + [(None, None)] * len(R))+ ax, AR = A[0], A[1:]+ cos = AR @ ax+ order = np.argsort(-cos)+ nearest = [{'path': R[i], 'name': os.path.basename(R[i]), 'cosine': round(float(cos[i]), 3)} for i in order[:k]]+ sx, z, pct = _typicality(ax, AR, k)+ fx = measure_path(path, start, end)+ FR = [measure_path(p) for p in R]+ fz = {}+ for key, fxv in fx.items():+ vals = np.array([_val(key, f[key]) for f in FR if key in f])+ if len(vals) < 2 or key == 'duration':+ continue+ med = float(np.median(vals))+ scale = max(1.4826 * float(np.median(np.abs(vals - med))), MIN_SCALE.get(key, 0.1))+ fz[key] = {'value': fxv, 'refs_median': round(math.exp(med) if key in LOG else med, 3),+ 'robust_z': round((_val(key, fxv) - med) / scale, 2),+ 'refs_p10_p90': [round(float(np.percentile([f[key] for f in FR if key in f], q)), 3) for q in (10, 90)]}+ order_k = [d[0] for d in DIFFS]+ outl = sorted([(kk, d) for kk, d in fz.items() if abs(d['robust_z']) >= 2 and kk in order_k],+ key=lambda t: order_k.index(t[0]))[:4]+ words = []+ for kk, d in outl:+ up = d['robust_z'] > 0+ if kk == 'loudness_lufs':+ words.append(f"{abs(d['value'] - d['refs_median']):.0f} dB {WORDS[kk][0 if up else 1]} than their median")+ else:+ words.append(_phrase(kk, up, 'median ' + _fmt(kk, d['refs_median']), _fmt(kk, d['value'])))+ fdist = float(np.median([abs(d['robust_z']) for d in fz.values()])) if fz else None+ s = (f"{_name(path, start, end)} vs {len(R)} references. Nearest: " ++ ', '.join(f"{n['name']} (cos {n['cosine']:.2f})" for n in nearest[:3]) ++ f". Typicality z {vocab.zf(z)} ({_ordinal(pct)} percentile of how close the references are to each other): "+ f"{_fit_words(z)}.")+ s += (' Measured against the references it is ' + '; '.join(words) + '.') if words else \+ " Every measurement is within the references' spread."+ return {'summary': s + _skipped_text(bad), 'path': path, 'range': [start, end], 'n_refs': len(R), 'nearest': nearest,+ 'typicality': {'mean_cos_knn': round(sx, 3), 'z': round(z, 2), 'percentile': round(pct, 1), 'k': k,+ 'verdict': _fit_words(z)},+ 'features': fz, 'feature_distance': None if fdist is None else round(fdist, 2), 'skipped': bad}+++def _closer(za, zb):+ """('a' | 'b' | None, words): None below CLOSE_DZ"""+ d = abs(za - zb)+ if d < CLOSE_DZ:+ return None, 'Neither is closer'+ w = 'B' if zb > za else 'A'+ return w.lower(), f"{w} is {'clearly' if d >= 1 else 'slightly'} closer"+++def ab(a, b, query=None, refs=None, k=5, a_start=None, a_end=None, b_start=None, b_end=None):+ """which of two sounds is closer to a description and/or reference set, and how B differs from A; *_start /+ *_end (s) = use only that part of a file (e.g. two trims from `find`, even of the same file)"""+ vocab.check(a, a_start, a_end)+ vocab.check(b, b_start, b_end)+ A = vocab.embed_files([a, b], ranges=[(a_start, a_end), (b_start, b_end)])+ res = {'a': a, 'b': b, 'a_range': [a_start, a_end], 'b_range': [b_start, b_end],+ 'cosine_ab': round(float(A[0] @ A[1]), 3)}+ s = [f'A = {_name(a, a_start, a_end)}, B = {_name(b, b_start, b_end)}.']+ if query:+ sc = vocab.get().scores(A, [query])+ za, zb = sc[0][0]['z'], sc[1][0]['z']+ w, txt = _closer(za, zb)+ res['query'] = {'query': query, 'a': sc[0][0], 'b': sc[1][0], 'closer': w, 'dz': round(abs(za - zb), 2)}+ s.append(f"{txt} to '{query}' (z A {vocab.zf(za)}, B {vocab.zf(zb)}).")+ if refs:+ R, bad = _refs(refs, exclude=[a, b])+ if len(R) >= 2:+ AR = vocab.embed_files(R)+ ta, tb = _typicality(A[0], AR, k), _typicality(A[1], AR, k)+ w, txt = _closer(ta[1], tb[1])+ res['refs'] = {'n_refs': len(R), 'a': {'mean_cos_knn': round(ta[0], 3), 'z': round(ta[1], 2)},+ 'b': {'mean_cos_knn': round(tb[0], 3), 'z': round(tb[1], 2)}, 'closer': w, 'skipped': bad}+ s.append(f'{txt} to the {len(R)} references (typicality z A {vocab.zf(ta[1])}, B {vocab.zf(tb[1])}; '+ f'>= -1 fits the set).' + _skipped_text(bad))+ else:+ s.append(f'Need at least 2 readable reference files, found {len(R)}.' + _skipped_text(bad))+ sim = res['cosine_ab']+ same = ('nearly the same sound' if sim >= 0.9 else 'similar' if sim >= 0.75 else 'related' if sim >= 0.5+ else 'different sounds')+ s.append(f'CLAP hears them as {same} (cosine {sim:.2f}).')+ fa, fb = measure_path(a, a_start, a_end), measure_path(b, b_start, b_end)+ keys, txt = differences(fa, fb)+ res['features'] = {'a': fa, 'b': fb}+ res['differences'] = keys+ s.append(f'B is {txt}.' if keys else f'Measured: {txt}.')+ res['summary'] = ' '.join(s)+ return res+++def _compare_text(r):+ lines = [r['summary'], '', 'nearest:'] + [f" cos {n['cosine']:.3f} {n['name']}" for n in r.get('nearest', [])]+ if r.get('features'):+ lines.append('features (value / refs median / robust z):')+ for k, d in r['features'].items():+ lines.append(f" {k:<17} {d['value']:>9} / {d['refs_median']:>9} / {d['robust_z']:+.1f}")+ return '\n'.join(lines)+++def add_cli(sub):+ p = sub.add_parser('compare', help='nearest reference recordings, typicality and feature differences')+ p.add_argument('path')+ p.add_argument('--refs', nargs='+', required=True, help='reference files / folders')+ p.add_argument('-k', type=int, default=5, help='how many nearest references to list (default 5)')+ p.add_argument('--start', type=float, help='judge only from this time of PATH (s), e.g. a trim from find')+ p.add_argument('--end', type=float, help='judge only up to this time of PATH (s)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(lambda: compare(a.path, a.refs, a.k, a.start, a.end), a, _compare_text))+ p = sub.add_parser('ab', help='A/B two sounds (or two trims): closer to a description / references, differences '+ 'in words')+ p.add_argument('a', help='sound A (e.g. the current version)')+ p.add_argument('b', help='sound B (e.g. the candidate / new render; may be the same file as A with other trims)')+ p.add_argument('--query', help='a description both should match: which one is closer (calibrated z)')+ p.add_argument('--refs', nargs='+', help='reference files / folders both should resemble')+ for w in ('a', 'b'):+ p.add_argument(f'--{w}-start', type=float, help=f'use {w.upper()} only from this time (s)')+ p.add_argument(f'--{w}-end', type=float, help=f'use {w.upper()} only up to this time (s)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(lambda: ab(a.a, a.b, a.query, a.refs, a_start=a.a_start, a_end=a.a_end,+ b_start=a.b_start, b_end=a.b_end), a))+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/config.py b/earshot/config.pynew file mode 100644index 0000000..73b31b7--- /dev/null+++ b/earshot/config.py@@ -0,0 +1,46 @@+"""Paths, model ids and CPU limits. Everything can be overridden with environment variables.++EARSHOT_HOME cache root (models, embeddings, decoded audio). Default: <repo>/.cache in a source checkout,+ else ~/.cache/earshot.+EARSHOT_THREADS torch CPU threads per process (default: min(4, cores // 2)); earshot never uses a GPU.+EARSHOT_CLAP CLAP model id (default laion/larger_clap_general)+EARSHOT_AST AudioSet tagger id (default MIT/ast-finetuned-audioset-10-10-0.4593)+EARSHOT_OFFLINE 1 = never touch the network (models must already be in the cache)+EARSHOT_PRECISION bf16 (default; used only on CPUs with native bf16, AVX512-BF16 / AMX, else fp32: on the dev laptop+ 1.4x faster, embeddings within cosine 0.9993 of fp32, eval/semantics/precision.json) | fp32+"""+import os++PKG_DIR = os.path.dirname(os.path.abspath(__file__))+REPO_DIR = os.path.dirname(PKG_DIR)+DATA_DIR = os.path.join(PKG_DIR, 'data')+++def _default_home():+ if os.environ.get('EARSHOT_HOME'):+ return os.path.abspath(os.environ['EARSHOT_HOME'])+ if os.path.exists(os.path.join(REPO_DIR, 'pyproject.toml')) and os.access(REPO_DIR, os.W_OK):+ return os.path.join(REPO_DIR, '.cache')+ return os.path.join(os.path.expanduser('~'), '.cache', 'earshot')+++HOME = _default_home()+HF_CACHE = os.path.join(HOME, 'hf')+EMB_CACHE = os.path.join(HOME, 'emb')+DEC_CACHE = os.path.join(HOME, 'decoded')++CLAP_MODEL = os.environ.get('EARSHOT_CLAP', 'laion/larger_clap_general')+AST_MODEL = os.environ.get('EARSHOT_AST', 'MIT/ast-finetuned-audioset-10-10-0.4593')+OFFLINE = os.environ.get('EARSHOT_OFFLINE', '0') == '1'+PRECISION = os.environ.get('EARSHOT_PRECISION', 'bf16').lower()+++def threads():+ if os.environ.get('EARSHOT_THREADS'):+ return max(1, int(os.environ['EARSHOT_THREADS']))+ return max(1, min(4, (os.cpu_count() or 2) // 2))+++def ensure_dirs():+ for d in (HOME, HF_CACHE, EMB_CACHE, DEC_CACHE):+ os.makedirs(d, exist_ok=True)diff --git a/earshot/critique.py b/earshot/critique.pynew file mode 100644index 0000000..4a486df--- /dev/null+++ b/earshot/critique.py@@ -0,0 +1,651 @@+"""Launch-video sound scorecard: compare a soundtrack or film with professional launch films and say what to fix.++ earshot critique film.mp4 # audio + its own picture (cuts and big visual changes)+ earshot critique mix.wav --video film.mp4 # a soundtrack checked against the picture it is for+ earshot critique film.mp4 --profile my_refs.json # your own references (earshot profile ...)+ earshot critique film.mp4 --fast # skip the AudioSet and realness passes (seconds, not minutes)++ from earshot.critique import critique+ r = critique('film.mp4')+ r['verdict'], r['score'], r['actions'], r['metrics']++Each metric row: value, reference range, typical band (p10-p90), median, status ok / low / high / n.a., weight+(critical / major / minor; info rows are style choices that are shown but not scored), why it matters and concrete+actions with timestamps. Ranges come from the profile: the min-max of the reference films widened by a per-metric+tolerance (measurement precision; a density whose range would reach 0 keeps a floor at half the sparsest reference),+except loudness and true peak, which follow web-delivery practice (-16..-9 LUFS integrated; true peak <= -1 dBTP for a+master, <= 0 dBTP for audio decoded from AAC / MP3, which already contains the codec overshoot) and show the references+for context. p_real comes from earshot.realness (when installed) and uses its verdict thresholds: ok at 'real or+pro-produced', critical below 0.5 (synthesized is the more likely reading), minor in between; a minor realness miss with no 4 s+window leaning synthesized is a tip, not a fix (nothing to replace was found). The 4 s windows it lists use the+window (clip) thresholds, as `earshot realness` does. A mono mix (identical channels) scores as stereo width -40 dB. The verdict weighs critical 3, major 2, minor 1 (a critical reference / web+rule missed by <= 2 tolerances counts as major): launch-grade (>= 90 % and no critical issue), close (>= 75 %, no+critical issue), needs work (>= 50 %), else not launch-grade; --fast (no realness, no music presence) marks the verdict+partial. Files under 3 s or silent are not scored; under 10 s the loudness-shape rows are n.a.+Sound on cuts: the scored share counts visual events with an onset (a new attack) within +-100 ms, the definition the+reference ranges were measured with; a whoosh swelling through a cut or music already playing is not an onset. The+fixes and tips instead name the events that get no sound at all by earshot.timeline's matcher (no onset within+100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms), so critique and timeline list the same+cuts. A file that is one of the profile's reference films (same duration and measurements) is scored leave-one-out,+against the other references, and says so. r['events'] lists the visual events, those without an onset within++-100 ms, those with no sound at all (timeline matcher), those without a salient hit, the hits and the stop-downs+(seconds). Missing files, files without audio, a --video without picture and a bad --profile raise+earshot.profile.InputError with a plain message. How the references fare against leave-one-out ranges and which+known-bad mixes get caught: eval/launch/RESULTS.md; the playbook behind the metrics: docs/launch-video-sound.md.+"""+import os+import sys+import time+from functools import lru_cache++import numpy as np++from . import audio, profile as P+from .cli import common+from .report import emit, error_line, fmt_t++WEIGHT = {3: 'critical', 2: 'major', 1: 'minor', 0: 'info'}+W = {v: k for k, v in WEIGHT.items()}+# key: (weight, tolerance, sides checked, practice window or None, why it matters)+SPEC = {+ 'lufs_i': (3, 1.0, 'both', (-16.0, -9.0),+ 'how loud the film plays next to other launch videos: X, LinkedIn and site autoplay do not normalise, '+ 'YouTube turns masters above -14 LUFS down'),+ 'true_peak': (2, 0.5, 'upper', (None, -1.0),+ 'a master above -1 dBTP clips once encoded to AAC / MP3 (a decoded lossy file above 0 dBTP already '+ 'clips on playback): crackle on the biggest hits'),+ 'p_real': (3, 0.0, 'lower', None,+ 'code-synthesized sound reads as cheap and generic; launch films use real recordings and produced music'),+ 'music_frac': (3, 0.05, 'lower', None,+ 'the references are music-led: one continuous track carries the energy and hides edit seams'),+ 'cuts_with_sound': (2, 0.05, 'lower', None,+ 'a cut or big visual change with no sound under it feels dead; viewers expect sound on motion'),+ 'lra': (2, 1.0, 'both', None, 'how much the loudness swings: launch films keep the body consistently strong'),+ 'crest_lu': (2, 0.5, 'both', None, 'how far the loudest 3 s stick out above the average level'),+ 'first_sound_s': (2, 0.1, 'upper', None, 'silence at the start loses autoplay viewers: the references sound at once'),+ 'end_silence_s': (2, 0.25, 'upper', None, 'dead air after the sound stops makes the ending feel unfinished'),+ 'max_gap_s': (2, 0.25, 'upper', None, 'a silent hole mid-film reads as a mistake unless it is a deliberate stop-down'),+ 'final_lu': (2, 1.5, 'both', None, 'the logo / end-card moment should land at body level or above, not fade away'),+ 'onset_rate': (2, 0.3, 'both', None, 'rhythmic activity: too sparse feels empty, too dense feels cluttered'),+ 'hit_rate': (2, 0.2, 'both', None, 'punch: salient hits mark the key moments; without them nothing lands'),+ 'intro_lu': (1, 1.5, 'both', None, 'how the film opens: the references open close to body level (one builds up)'),+ 'end_lu': (1, 1.5, 'both', None, 'how the last 10 % sit against the body'),+ 'fade_s': (1, 0.5, 'both', None, 'launch films end on a sting or a hard out; long fade-outs lose the button'),+ 'band_sub': (1, 1.5, 'both', None, 'sub (20-60 Hz) weight under hits and music'),+ 'band_low': (1, 1.5, 'both', None, 'low end body (60-250 Hz)'),+ 'band_lowmid': (1, 1.5, 'both', None, 'low-mids (250 Hz-1 kHz): too much = muddy, too little = hollow'),+ 'band_mid': (1, 1.5, 'both', None, 'mids (1-4 kHz): where clicks and voice cut through phone speakers'),+ 'band_presence': (1, 1.5, 'both', None, 'presence (4-8 kHz): bite of transients, clicks and typing'),+ 'band_air': (1, 1.5, 'both', None, 'air (8-16 kHz): openness; missing air sounds muffled or low-bitrate'),+ 'side_to_mid_db': (1, 1.0, 'both', None, 'stereo width: near-mono sounds small, too wide collapses on phones'),+ 'speech_frac': (0, 0, 'both', None, 'voice share (4 of 7 references have a voice-over, 3 are music-only)'),+ 'tempo': (0, 0, 'both', None, 'music tempo'),+ 'peak_pos': (0, 0, 'both', None, 'where the loudest moment sits'),+ 'tail_s': (0, 0, 'both', None, 'how long the final hit rings out'),+ 'stopdowns_min': (0, 0, 'both', None, 'short drop-outs before hits (4 of 7 references use them)'),+ 'cut_rate': (0, 0, 'both', None, 'edit pace (visual events per second)'),+ 'cut_hit_frac': (0, 0, 'both', None, 'visual events with a salient hit (references: near chance)'),+ 'cut_frame_sync': (0, 0, 'both', None, 'visual events with an onset within one frame (references: near chance)'),+ 'cuts_on_beat': (0, 0, 'both', None, 'visual events on the beat grid (references: near chance)'),+ 'brightness_hz': (0, 0, 'both', None, 'perceived brightness of the mix'),+ 'correlation': (0, 0, 'both', None, 'L/R correlation (1 = mono)'),+}+BAND_FIX = {+ 'band_sub': ('thin bottom: add a sub layer (sub drop or kick) under reveals, or pick music with real low end',+ 'too much 20-60 Hz rumble eats headroom and distorts small speakers: high-pass SFX and ambience at '+ '30-40 Hz and cut the sub of the music by 2-4 dB'),+ 'band_low': ('weak low end: the music has no body; choose a fuller track or add a low layer (bass, low hits)',+ 'boomy: cut 100-200 Hz on the music and on heavy SFX by 2-3 dB'),+ 'band_lowmid': ('hollow: the 250 Hz-1 kHz body is scooped; lift it on the music or use fuller recordings',+ 'muddy / boxy: cut 300-500 Hz on the music bed and on roomy SFX by 2-3 dB'),+ 'band_mid': ('recessed mids: clicks and voice will not cut through phone speakers; lift 1-3 kHz on SFX / VO',+ 'honky or harsh mids: cut 1-3 kHz on the loudest layer by 2 dB'),+ 'band_presence': ('dull: transients, clicks and typing lack bite; brighten SFX or use brighter recordings',+ 'harsh / sibilant: cut 5-7 kHz or de-ess the brightest layer'),+ 'band_air': ('no air: it sounds muffled or low-bitrate; use full-band recordings (44.1 / 48 kHz, no low-pass)',+ 'hissy / fizzy: low-pass or de-noise the noisiest layer'),+}+++def _times(ts, k=8):+ ts = list(ts)+ s = ', '.join(fmt_t(t) for t in ts[:k])+ return s + (f' (+{len(ts) - k} more)' if len(ts) > k else '')+++@lru_cache(maxsize=2)+def real_thresholds(file=True):+ """earshot.realness verdict thresholds {'lo', 'hi'}: the file-level ones for a long file's mean p_real (what+ critique scores), the clip-level ones for a single short clip; 0.5 / 0.5 when the model is not installed"""+ try:+ from .realness import load_model, thresholds_of+ t = thresholds_of(load_model(), file=file)+ return {'lo': float(t['lo']), 'hi': float(t['hi'])}+ except Exception: # noqa: BLE001 (realness is optional)+ return {'lo': 0.5, 'hi': 0.5}+++def _window(key, stats, lossy=False):+ """(lo, hi, basis) for a metric: practice window, or the reference hull widened by the tolerance"""+ w, tol, sides, practice, _ = SPEC[key]+ if key == 'p_real':+ return real_thresholds()['hi'], None, 'realness model'+ if key == 'true_peak' and lossy:+ return None, 0.0, 'decoded lossy'+ if practice:+ return practice[0], practice[1], 'web'+ if not stats or not stats.get('n'):+ return None, None, 'no reference data'+ lo, hi = stats['min'] - tol, stats['max'] + tol+ unit = P.METRICS[key][2]+ if lo <= 0 and unit in ('s', '/s', '/min', 'share'):+ lo = round(stats['min'] / 2, 3) if unit == '/s' and stats['min'] > 0 else None+ return (lo if sides in ('both', 'lower') else None), (hi if sides in ('both', 'upper') else None), 'references'+++def _status(v, lo, hi):+ if v is None:+ return 'n.a.'+ if lo is not None and v < lo:+ return 'low'+ if hi is not None and v > hi:+ return 'high'+ return 'ok'+++def _missing(events, targets, tol=0.1):+ targets = np.sort(np.asarray(targets, float))+ out = []+ for t in events:+ if not len(targets) or np.min(np.abs(targets - t)) > tol:+ out.append(t)+ return out+++def _ranges(spans):+ """merge [(t0, t1)] that touch"""+ out = []+ for a, b in spans:+ if out and a <= out[-1][1] + 1e-6:+ out[-1] = (out[-1][0], b)+ else:+ out.append((a, b))+ return out+++def _p(v, t):+ """p_real for text: 2 decimals, 3 when 2 would round onto a threshold it is actually below / above"""+ s = f'{v:.2f}'+ return f'{v:.3f}' if any(float(s) == x and abs(v - x) > 1e-9 for x in (t['lo'], t['hi'])) else s+++def _realness_windows(f):+ """(4 s windows with a p_real, those at or below the window threshold sorted lowest first, window thresholds)"""+ rl = f.get('realness') or {}+ t = real_thresholds(file=False) # each 4 s window is judged like a clip, as `earshot realness` does+ segs = [q for q in rl.get('segments') or [] if q.get('p_real') is not None]+ return segs, sorted((q for q in segs if q['p_real'] <= t['lo']), key=lambda q: q['p_real']), t+++def _quiet_realness_miss(f, v):+ """a minor realness miss with nothing to replace: p_real between 0.5 and the 'real' line and no 4 s window+ leaning synthesized"""+ segs, _, _ = _realness_windows(f)+ return v is not None and v >= 0.5 and not any(q['p_real'] < 0.5 for q in segs)+++def _unanswered(f):+ """(no sound at all, only a level peak) visual-event times, by earshot.timeline's matcher (None: not computed)"""+ sy = f.get('cut_sync')+ if not sy:+ return None, None+ items = sy['items']+ return ([i['t'] for i in items if i['status'] in ('no sound', 'off')], [i['t'] for i in items if i['status'] == 'peak'])+++def _actions(key, st, v, f, a, refs):+ """concrete fixes for one out-of-range metric"""+ ev, dur, m = f['events'], f['duration'], f['metrics']+ rng = f"refs {P.fmt_value(key, refs.get('min'))} .. {P.fmt_value(key, refs.get('max'))}" if refs.get('n') else ''+ if key == 'lufs_i':+ if st == 'low':+ return [f'raise the master by {-14 - v:+.1f} dB to about -14 LUFS integrated (web) with a true-peak limiter '+ f'(ceiling -1 dBTP); if the limiter works harder than ~3 dB, compress the music bus first ({rng})']+ return [f'turn the master down by {v + 12:.1f} dB to about -12 LUFS: above -9 LUFS normalising platforms turn it '+ f'down anyway and the limiter flattens every hit ({rng})']+ if key == 'true_peak':+ clip = ' (this decoded lossy file already clips on playback)' if f['lossy'] else ''+ return [f'true peak {v:+.1f} dBTP{clip}: end the master chain with a true-peak limiter, ceiling -1.0 dBTP, '+ 'then re-encode']+ if key == 'p_real':+ rl = f.get('realness') or {}+ t = real_thresholds(file=bool(rl.get('segments'))) # the mean of a long file's windows: file-level thresholds+ segs, flagged, tw = _realness_windows(f)+ if _quiet_realness_miss(f, v):+ return [] # nothing synthetic found to replace: a tip (see _tips), not a fix+ if v <= t['lo']:+ head = f"realness: likely code-synthesized (p_real {_p(v, t)}, at or below the model's {t['lo']:.2f} threshold)"+ elif v < 0.5:+ head = (f"realness leans code-synthesized: p_real {_p(v, t)} (below 0.5 the model rates synthesized as more "+ f"likely; realness calls {t['lo']:.2f}-{t['hi']:.2f} 'uncertain')")+ else:+ head = f"realness is uncertain (p_real {_p(v, t)}; real or pro-produced sound scores >= {t['hi']:.2f})"+ pick = []+ for q in flagged or sorted(segs, key=lambda q: q['p_real']): # lowest first, so a cut list keeps the worst+ if all(abs(q['start'] - x['start']) >= 2.0 for x in pick):+ pick.append(q)+ pick = sorted(pick[:6 if flagged else 3], key=lambda q: q['start'])+ spans = ', '.join(f"{fmt_t(q['start'])}-{fmt_t(q['end'])} ({q['p_real']:.2f})" for q in pick)+ more = f' (+{len(flagged) - len(pick)} more)' if flagged and len(flagged) > len(pick) else ''+ where = (f"; 4 s windows at or below the window threshold {tw['lo']:.2f} (code-synthesized): {spans}{more}"+ if flagged else f'; lowest-scoring 4 s windows: {spans}' if spans else '')+ fix = ('replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the '+ 'sourcing and licence workflow)' if v < 0.5 else 'find and replace the layer that sounds synthetic')+ return [f'{head}{where}: {fix}; judge each stem with `earshot realness stem.wav` (a finished mix is harder to '+ 'judge than its sources)']+ if key == 'music_frac':+ gaps = _ranges([(t, min(dur, t + P.AST_WIN)) for t, mu, _ in ev['music_windows'] or [] if mu < P.MUSIC_P])+ where = ', '.join(f'{fmt_t(x)}-{fmt_t(y)}' for x, y in gaps[:6])+ return [f'music is heard in only {100 * v:.0f}% of 5 s windows ({rng}); no music at {where}: run one '+ 'continuous track under the whole film and duck it under VO instead of stopping it']+ if key == 'cuts_with_sound':+ miss = _missing(ev['visual'], a['onsets'])+ none, peak = _unanswered(f)+ if none is None:+ return [f'{len(miss)} of {len(ev["visual"])} visual events have no onset (new attack) within +-100 ms ({rng}): '+ f'add a whoosh into the transition (peak on the cut), a hit or a UI click at {_times(miss)}']+ where = (f'no sound at all at {_times(none)}: add a whoosh into the transition (peak on the cut), a hit or a UI '+ 'click there' if none else 'every one has some sound under it')+ thru = len(miss) - len(none)+ return [f'{len(miss)} of {len(ev["visual"])} visual events have no onset (new attack) within +-100 ms ({rng}); '+ f'{where}' + (f'; {thru} have sound playing through them (a swell, music or a level peak) but no new '+ 'attack: add an accent on the ones that matter' if thru > 0 else '')]+ if key in ('lra', 'crest_lu'):+ ts, Ls = a['ts'], a['Ls']+ body = (ts > 0.1 * dur) & (ts < 0.9 * dur)+ if st == 'high' and body.any():+ iq = np.where(body)[0][int(np.argmin(Ls[body]))]+ il = int(np.argmax(Ls))+ quiet = f'{fmt_t(max(0, ts[iq] - 1.5))}-{fmt_t(ts[iq] + 1.5)} at {Ls[iq]:.1f} LUFS short-term'+ if key == 'lra':+ return [f'the loudness swings {v:.1f} LU (LRA; {rng}): the quietest body stretch is {quiet}, the loudest 3 s '+ f'sit around {fmt_t(ts[il])} ({Ls[il]:.1f} LUFS); lift the quiet parts or compress the music bus '+ f'2-4 dB so the body holds']+ return [f'the loudest 3 s (around {fmt_t(ts[il])}, {Ls[il]:.1f} LUFS) stand {v:.1f} LU above the integrated '+ f'level ({rng}): the rest of the film plays quiet next to it; lift the body (quietest stretch {quiet}) '+ f'or tame that peak']+ if key == 'lra':+ return [f'loudness range (LRA) only {v:.1f} LU ({rng}): the level never moves; give the film sections (a '+ 'calmer VO or UI passage, a lift into the reveal) or ease the limiter']+ return [f'the loudest 3 s are only {v:.1f} LU above integrated ({rng}): no moment stands out; let the main reveal '+ '/ logo hit rise 2-3 LU above the body or ease the limiter']+ if key == 'first_sound_s':+ pre = a['Lf'][a['tf'] < v]+ lvl = P._energy_mean(pre) if len(pre) else None+ if lvl is not None and lvl > -60:+ return [f'nothing reaches body level until {fmt_t(v)} (before that the sound sits around {lvl:.0f} LUFS, '+ f"{m['lufs_i'] - lvl:.0f} LU under the body; {rng}): raise the opening or start the music at 0:00"]+ return [f'silence until {fmt_t(v)} ({rng}): start the music, ambience or a riser at 0:00']+ if key == 'end_silence_s':+ return [f'{v:.1f} s of dead air after the sound ends at {fmt_t(dur - v)}: trim the picture or let the final sting '+ f'ring out over the end card ({rng})']+ if key == 'max_gap_s':+ Lf, tf, I = a['Lf'], a['tf'], m['lufs_i']+ h = float(tf[1] - tf[0])+ gaps, run = [], []+ for t, q in zip(tf, Lf < I - 30):+ if q:+ run.append(t)+ elif run:+ if len(run) * h >= 0.5 and run[0] > (m['first_sound_s'] or 0) and t < dur - (m['end_silence_s'] or 0):+ gaps.append((run[0] - 0.05, run[-1] + 0.05))+ run = []+ where = ', '.join(f'{fmt_t(x)}-{fmt_t(y)}' for x, y in gaps[:6])+ return [f'silent holes inside the film (longest {v:.1f} s; {rng}): {where}; keep a music bed or room tone under '+ 'them, or make each one a deliberate stop-down that ends on a hit']+ if key == 'final_lu':+ last = ev['visual'][-1] if ev['visual'] else None+ if st == 'low':+ at = f' at the last visual event ({fmt_t(last)})' if last is not None else ''+ return [f'the final moment sits {-v:.1f} LU below the body: end on a logo sting / hit{at} instead of fading '+ f'away ({rng})']+ return [f'the final hit is {v:.1f} LU above the body: pull it down 2-3 dB so it does not jump out ({rng})']+ if key == 'onset_rate':+ if st == 'low':+ return [f'sparse: {v:.1f} onsets/s ({rng}): use music with more rhythmic drive, or add UI foley (clicks, '+ f'typing, ticks) on the on-screen actions']+ return [f'busy: {v:.1f} onsets/s ({rng}): thin out the SFX layer and keep accents for the important moments']+ if key == 'hit_rate':+ if st == 'low':+ big = sorted(zip(ev['visual_strength'], ev['visual']), reverse=True)+ cand = _missing([t for _, t in big], ev['hits'])[:6]+ at = f' - the biggest visual events without one: {_times(sorted(cand))}' if cand else ''+ return [f"{'no' if v == 0 else 'few'} salient hits ({v:.2f}/s, {rng}): add impacts / accents (a hit, a sub "+ f'drop, a snappy UI click) on the key moments{at}']+ return [f'too many salient hits ({v:.2f}/s, {rng}): keep the biggest for reveals, soften the rest by 3-6 dB']+ if key in ('intro_lu', 'end_lu'):+ span = f'0:00-{fmt_t(0.1 * dur)}' if key == 'intro_lu' else f'{fmt_t(0.9 * dur)}-{fmt_t(dur)}'+ part = 'opening' if key == 'intro_lu' else 'ending'+ if st == 'low':+ return [f'the {part} ({span}) sits {-v:.1f} LU below the body ({rng}): ' ++ ('start closer to full level or shorten the build-up' if key == 'intro_lu' else+ 'keep the energy up into the end card and finish on a sting')]+ return [f'the {part} ({span}) is {v:.1f} LU louder than the body ({rng}): bring it down or lift the body']+ if key == 'fade_s':+ if st == 'high':+ return [f'{v:.1f} s fade-out ({rng}): end on a sting with a short ring-out, or cut hard on the last picture cut']+ return [f'the end is cut hard; let the last hit ring out briefly ({rng})']+ if key in BAND_FIX:+ return [BAND_FIX[key][0 if st == 'low' else 1] + f' ({P.fmt_value(key, v)} share, {rng})']+ if key == 'side_to_mid_db':+ if st == 'low':+ what = ('the mix is mono (identical left and right channels' if f.get('mono') else+ f'nearly mono (side {v:.1f} dB vs mid')+ return [f'{what}; {rng}): use stereo music and ambience, pan SFX with the motion']+ return [f'very wide (side {v:.1f} dB vs mid; {rng}): check the mono fold-down, narrow wide ambience / reverbs']+ return []+++def _tips(f, a, flagged, p_real=None):+ """not scored: craft suggestions drawn from what the references do"""+ ev, m, out = f['events'], f['metrics'], []+ none, peak = _unanswered(f)+ if none and m.get('cuts_with_sound') is not None and 'cuts_with_sound' not in flagged:+ out.append(f'{len(none)} of {len(ev["visual"])} visual events get no sound at all (no onset within 100 ms, no '+ f'whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): '+ f'{_times(none)}; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as '+ 'a deliberate stop-down' + (f'; {len(peak)} more are met only by a level peak (a soft swell, or just '+ f'the bed: check {_times(peak, 4)} on the sheet)' if peak else ''))+ if p_real is not None and 'p_real' not in flagged and p_real < real_thresholds()['hi'] and \+ _quiet_realness_miss(f, p_real):+ segs, _, _ = _realness_windows(f)+ low = min((q['p_real'] for q in segs), default=None)+ t = real_thresholds()+ out.append(f"realness not confirmed (p_real {_p(p_real, t)}, under the {t['hi']:.2f} 'real or pro-produced' line) "+ 'but nothing synthetic was found' + (f' (no 4 s window leans synthesized; lowest {low:.2f})' if segs+ else '') ++ ': no layer to replace on this evidence; a finished mix is harder to judge than its sources, so '+ 'check the stems with `earshot realness stem.wav` if you have them')+ if m.get('stopdowns_min') == 0 and ev['visual']:+ big = max(zip(ev['visual_strength'], ev['visual']))[1]+ out.append(f'no stop-downs: 4 of 7 references drop the sound out for 0.2-1 s right before a big hit; try one '+ f'before the biggest visual change at {fmt_t(big)}')+ if m.get('speech_frac') is not None and m['speech_frac'] >= 0.5:+ out.append(f'voice detected in {100 * m["speech_frac"]:.0f}% of windows: duck music 3-6 dB under the voice and '+ 'keep SFX out of 1-4 kHz while it speaks')+ return out+++def _load(profile):+ if isinstance(profile, dict):+ return profile+ if profile and not os.path.isfile(profile):+ raise P.InputError(f'profile not found: {profile}')+ try:+ prof = P.load_profile(profile)+ except (ValueError, UnicodeDecodeError) as e:+ raise P.InputError(f'{profile} is not a profile JSON (make one with `earshot profile`): {e}') from None+ if not isinstance(prof, dict) or not isinstance(prof.get('metrics'), dict):+ raise P.InputError(f'{profile} is not a profile JSON (no "metrics"; make one with `earshot profile`)')+ return prof+++def _na_reason(key, f, use_ast, use_realness):+ if key == 'p_real':+ return 'skipped by --fast' if not use_realness else 'realness unavailable'+ if key == 'music_frac':+ return 'skipped by --fast' if not use_ast else 'no AudioSet windows'+ if key == 'cuts_with_sound':+ return 'no picture: pass --video film.mp4' if not f['video'] else 'fewer than 3 visual events'+ if key in P.SHAPE and f['short']:+ return f'clip under {P.SHORT_S:.0f} s; the references are 20-93 s films'+ return 'not measurable on this file'+++def critique(path, profile=None, video=None, use_ast=True, use_realness=True):+ """scorecard of a soundtrack / film against a reference profile (default: the bundled 7 launch films)"""+ t0 = time.time()+ if video and os.path.abspath(video) == os.path.abspath(path) and not audio.probe(path).get('has_video'):+ video = None+ for p_, need in ((path, 'audio'), (video, 'video')):+ if p_ and (why := P.media_problem(p_, need)):+ raise P.InputError(why)+ prof = _load(profile)+ dur, why = P.unscorable(path)+ if why:+ msg = (f'{os.path.basename(path)} {why}: nothing to score against launch films. For a single sound use '+ '`earshot describe` / `earshot realness`.')+ return {'path': path, 'video': video, 'duration': round(dur, 3), 'verdict': 'n.a.', 'score': None,+ 'in_range': 0, 'scored': 0, 'critical_issues': 0, 'major_issues': 0, 'partial': [], 'actions': [],+ 'tips': [], 'metrics': [], 'summary': msg, 'text': msg}+ f = P.film_metrics(path, video=video, use_ast=use_ast)+ a = P._audio_pass(path)+ ref = reference_match(f, prof)+ if ref: # one of the profile's own films: score it against the others (its values must not set the ranges)+ prof = leave_one_out(prof, ref)+ f['cut_sync'] = cut_sync(path, f)+ vals = dict(f['metrics'])+ vals['p_real'] = None+ if use_realness:+ try:+ from .realness import realness+ rl = realness(path)+ f['realness'] = {k: rl.get(k) for k in ('p_real', 'verdict', 'synth_windows', 'segments', 'summary')}+ vals['p_real'] = rl.get('p_real')+ except Exception as e: # noqa: BLE001 (realness is optional)+ f['realness'] = {'error': f'{type(e).__name__}: {e}'}+ sc = scorecard(vals, prof, f['chance'], f['lossy'])+ actions = []+ for row in sc['metrics']:+ if row['status'] in ('low', 'high'):+ w, tol = W[row['weight']], SPEC[row['metric']][1]+ lo, hi = row['range']+ edge = lo if row['status'] == 'low' else hi+ row['actions'] = _actions(row['metric'], row['status'], row['value'], f, a,+ prof['metrics'].get(row['metric'], {}))+ for act in row['actions']:+ actions.append({'metric': row['metric'], 'weight': WEIGHT[w], 'w': w,+ 'dist': abs(row['value'] - edge) / max(tol, 1e-3), 'action': act})+ actions.sort(key=lambda x: (-x['w'], -x['dist']))+ ev = f['events']+ partial = [n for n, k in (('realness', 'p_real'), ('music presence', 'music_frac')) if vals.get(k) is None]+ res = {'path': path, 'video': f['video'], 'duration': f['duration'], 'profile_n': prof.get('n'),+ 'lossy': f['lossy'],+ 'profile': 'bundled launch films' if profile is None else+ (profile if isinstance(profile, str) else 'custom')}+ res.update({k: v for k, v in sc.items() if k != 'metrics'})+ res.update(partial=partial,+ actions=[{'weight': x['weight'], 'metric': x['metric'], 'action': x['action']} for x in actions],+ tips=_tips(f, a, {x['metric'] for x in actions}, vals.get('p_real')), metrics=sc['metrics'],+ reference_match=ref and {'id': ref, 'scored_against': prof.get('n')},+ events={'visual': ev['visual'], 'visual_without_onset': _missing(ev['visual'], a['onsets']),+ 'visual_without_any_sound': _unanswered(f)[0],+ 'visual_without_hit': _missing(ev['visual'], ev['hits']), 'hits': ev['hits'],+ 'stopdowns': ev['stopdowns']},+ arc={'film': f.get('arc'), **{f'refs_{q}': v for q, v in (prof.get('arc') or {}).items()}},+ realness=f.get('realness'), seconds=round(time.time() - t0, 1))+ na = {}+ for r in sc['metrics']:+ if r['status'] == 'n.a.' and r['weight'] != 'info':+ na.setdefault(_na_reason(r['metric'], f, use_ast, use_realness), []).append(r['metric'])+ res['not_measured'] = na+ first = actions[0]['action'] if actions else None+ res['summary'] = (f"{os.path.basename(path)} vs {prof.get('n')} reference films ({res['profile']}): "+ + (f'{_ref_note(res)} ' if ref else '') + f"{_verdict_line(res)}" ++ (f" Fix first: {first}{'' if first.endswith('.') else '.'}" if first else+ ' Nothing to fix against the references.') ++ (' Not measured: ' + '; '.join(f"{', '.join(k)} ({why})" for why, k in na.items()) + '.'+ if na else ''))+ res['text'] = critique_text(res)+ return res+++def reference_match(f, prof):+ """id of the profile film this file is (same duration within 50 ms and the same loudness, true peak, brightness+ and mid-band share), else None"""+ m = f['metrics']+ tol = {'lufs_i': 0.15, 'true_peak': 0.15, 'brightness_hz': 15.0, 'band_mid': 0.15}+ for g in prof.get('films') or []:+ if abs(float(g.get('duration') or -1) - float(f['duration'])) > 0.05:+ continue+ both = [k for k in tol if m.get(k) is not None and (g.get('metrics') or {}).get(k) is not None]+ if len(both) >= 3 and all(abs(m[k] - g['metrics'][k]) <= tol[k] for k in both):+ return g.get('id') or 'a reference'+ return None+++def leave_one_out(prof, ref):+ """the profile rebuilt from every film except `ref` (needs at least 2 others; else the profile unchanged)"""+ others = [dict(g, video=g.get('has_video', True), style=g.get('style', '')) for g in prof.get('films') or []+ if g.get('id') != ref]+ if len(others) < 2:+ return prof+ return dict(P.aggregate(others), loo_of=ref, n_full=prof.get('n'))+++def cut_sync(path, f):+ """each visual event matched to the sound that answers it with earshot.timeline's matcher (onset, whoosh / riser+ peak, level peak), so critique and timeline name the same unsounded cuts; None without picture events"""+ vis = f['events'].get('visual') or []+ if not vis:+ return None+ try:+ from . import timeline as T+ x = audio.load(path)+ fps = f.get('fps')+ return T.sync(vis, T.detect_events(x), None, 1.0 / fps if fps else None, T.level_envs(x))+ except Exception: # noqa: BLE001 (the onset-based list is the fallback)+ return None+++def _ref_note(res):+ rm = res.get('reference_match') or {}+ return (f"NOTE: this file is {rm.get('id')} of the profile (same duration and measurements), so it is scored "+ f"against the other {rm.get('scored_against')} references (leave-one-out): its own values do not set the "+ 'ranges.')+++def _verdict_line(res):+ part = f" (partial: {' and '.join(res['partial'])} not checked)" if res.get('partial') else ''+ return (f"{res['verdict']}{part} - {res['in_range']} of {res['scored']} scored metrics in range (weighted "+ f"{100 * (res['score'] or 0):.0f}%), {res['critical_issues']} critical and {res['major_issues']} major "+ 'issues.')+++def scorecard(vals, prof, chance=None, lossy=False):+ """status of every metric in `vals` against a profile + weighted score and verdict (no actions; used by+ critique() and by the leave-one-out evaluation). lossy=True: the audio was decoded from AAC / MP3 / Opus, so true+ peak is judged against 0 dBTP (codec overshoot is already in it) instead of the -1 dBTP master rule."""+ rows = []+ for key, (w, tol, sides, practice, why) in SPEC.items():+ stats = prof['metrics'].get(key, {})+ lo, hi, basis = _window(key, stats, lossy)+ v = vals.get(key)+ st = 'n.a.' if basis == 'no reference data' else _status(v, lo, hi) if w else ('n.a.' if v is None else 'info')+ if key == 'p_real' and st == 'low':+ w = 3 if v < 0.5 else 1+ elif w == 3 and st in ('low', 'high') and abs(v - (lo if st == 'low' else hi)) <= 2 * tol:+ w = 2 # a near miss of a critical rule counts as major+ real = key == 'p_real'+ row = {'metric': key, 'label': 'realness (p_real)' if real else P.METRICS[key][1],+ 'group': 'realness' if real else P.METRICS[key][0], 'value': v, 'unit': 'p' if real else P.METRICS[key][2],+ 'range': [lo, hi], 'basis': basis, 'median': stats.get('median'),+ 'typical': [stats.get('p10'), stats.get('p90')] if stats.get('n') else None,+ 'status': st, 'weight': WEIGHT[w], 'why': why}+ if chance and key in chance:+ row['chance'] = chance[key]+ rows.append(row)+ scored = [(W[r['weight']], r) for r in rows if r['weight'] != 'info' and r['status'] != 'n.a.']+ wsum = sum(w for w, _ in scored)+ score = sum(w for w, r in scored if r['status'] == 'ok') / wsum if wsum else None+ n_crit = sum(1 for w, r in scored if w == 3 and r['status'] != 'ok')+ n_major = sum(1 for w, r in scored if w == 2 and r['status'] != 'ok')+ verdict = ('n.a.' if score is None else 'launch-grade' if score >= 0.9 and not n_crit else+ 'close' if score >= 0.75 and not n_crit else 'needs work' if score >= 0.5 else 'not launch-grade')+ return {'verdict': verdict, 'score': None if score is None else round(score, 3),+ 'in_range': sum(1 for _, r in scored if r['status'] == 'ok'), 'scored': len(scored),+ 'critical_issues': n_crit, 'major_issues': n_major, 'metrics': rows}+++def _range_txt(r):+ lo, hi = r['range']+ k = r['metric']+ fv = (lambda x: f'{x:.2f}') if k == 'p_real' else (lambda x: P.fmt_value(k, x))+ if r['weight'] == 'info':+ t = r['typical']+ return f"refs p10-p90 {fv(t[0])} .. {fv(t[1])}" if t and t[0] is not None else ''+ if lo is None and hi is None:+ return r['basis']+ s = f'>= {fv(lo)}' if hi is None else f'<= {fv(hi)}' if lo is None else f'{fv(lo)} .. {fv(hi)}'+ return s + (f" ({r['basis']})" if r['basis'] != 'references' else '')+++def critique_text(res):+ lines = [f"earshot critique: {os.path.basename(res['path'])} ({res['duration']:.1f} s) vs {res['profile_n']} "+ f"reference films ({res['profile']})"] + ([_ref_note(res)] if res.get('reference_match') else []) + \+ [f'VERDICT: {_verdict_line(res)}']+ if res.get('partial'):+ lines.append(' (run without --fast before shipping: realness and music presence are critical checks)'+ if len(res['partial']) == 2 else f" ({' and '.join(res['partial'])} could not be measured)")+ if res['actions']:+ lines.append('FIX FIRST (most important first):')+ for i, x in enumerate(res['actions'], 1):+ lines.append(f" {i}. [{x['weight']}] {x['action']}")+ for tip in res['tips']:+ lines.append(f' tip: {tip}')+ if res.get('not_measured'):+ lines.append('NOT MEASURED: ' + '; '.join(f"{', '.join(k)} ({why})" for why, k in res['not_measured'].items()))+ lines.append(f"SCORECARD {'':30s} {'value':>11s} {'ok range':30s} {'ref median':>11s} status")+ for r in res['metrics']:+ if r['weight'] == 'info':+ continue+ v = ('n.a.' if r['value'] is None else _p(r['value'], real_thresholds()) if r['metric'] == 'p_real' else+ P.fmt_value(r['metric'], r['value']))+ med = '' if r['median'] is None else P.fmt_value(r['metric'], r['median'])+ lines.append(f" {r['group'][:8]:8s} {r['label'][:30]:30s} {v:>11s} {_range_txt(r)[:30]:30s} {med:>11s} "+ f"{r['status']}{'' if r['status'] in ('ok', 'n.a.') else ' (' + r['weight'] + ')'}")+ arc = res.get('arc') or {}+ if arc.get('film') and arc.get('refs_median'):+ fa = lambda xs: ' '.join(' - ' if x is None else f'{x:+6.1f}' for x in xs)+ lines.append('LOUDNESS ARC (LU vs integrated at 0%, 10% .. 100% of the film; not scored):')+ lines.append(f" this film {fa(arc['film'])}")+ lines.append(f" refs median {fa(arc['refs_median'])}")+ lines.append('NOT SCORED (style choices, shown for context):')+ for r in res['metrics']:+ if r['weight'] != 'info' or r['value'] is None:+ continue+ ch = f" (chance {P.fmt_value(r['metric'], r['chance'])})" if r.get('chance') is not None else ''+ lines.append(f" {r['label'][:44]:44s} {P.fmt_value(r['metric'], r['value']) + ch:>24s} {_range_txt(r)}")+ rl = res.get('realness') or {}+ if rl.get('summary'):+ lines.append('realness: ' + rl['summary'])+ elif rl.get('error'):+ lines.append('realness: not available (' + rl['error'][:120] + ')')+ return '\n'.join(lines)+++def _cmd(args):+ try:+ r = critique(args.file, profile=args.profile, video=args.video, use_ast=not args.fast,+ use_realness=not args.fast)+ except (P.InputError, audio.AudioError) as e:+ error_line('critique', e)+ return 1+ emit(r, args)+++def add_cli(sub):+ p = sub.add_parser('critique', help='launch-video sound scorecard against professional reference films',+ description='Measure a soundtrack or film (loudness, true peak, arc, ending, music presence, '+ 'onset / hit density, sound on cuts, spectrum, stereo, realness) and compare it with a profile '+ 'of professional launch films: status per metric and concrete fixes with timestamps.')+ p.add_argument('file', help='soundtrack or film')+ p.add_argument('--video', help='the picture the soundtrack is for (default: the file itself if it is a video)')+ p.add_argument('--profile', help='profile JSON from `earshot profile` (default: the bundled 7 launch films)')+ p.add_argument('--fast', action='store_true', help='skip the AudioSet (music / speech) and realness passes')+ common(p)+ p.set_defaults(func=_cmd)+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/data/launch_profile.json b/earshot/data/launch_profile.jsonnew file mode 100644index 0000000..642c98e--- /dev/null+++ b/earshot/data/launch_profile.json@@ -0,0 +1,1102 @@+{+ "version": 2,+ "n": 7,+ "metrics": {+ "lufs_i": {+ "n": 7,+ "median": -14.8,+ "p10": -19.32,+ "p90": -9.16,+ "min": -22.2,+ "max": -7.3,+ "values": [+ -12.1,+ -14.8,+ -10.4,+ -14.8,+ -22.2,+ -17.4,+ -7.3+ ]+ },+ "true_peak": {+ "n": 7,+ "median": 0.6,+ "p10": -0.26,+ "p90": 1.52,+ "min": -0.8,+ "max": 2.0,+ "values": [+ 0.1,+ 1.0,+ 0.6,+ -0.8,+ 1.2,+ 0.1,+ 2.0+ ]+ },+ "lra": {+ "n": 7,+ "median": 5.5,+ "p10": 2.66,+ "p90": 9.08,+ "min": 1.7,+ "max": 12.2,+ "values": [+ 1.7,+ 3.6,+ 6.4,+ 7.0,+ 12.2,+ 5.5,+ 3.3+ ]+ },+ "crest_lu": {+ "n": 7,+ "median": 2.3,+ "p10": 1.16,+ "p90": 3.32,+ "min": 1.1,+ "max": 3.5,+ "values": [+ 1.1,+ 2.4,+ 2.3,+ 2.0,+ 3.2,+ 3.5,+ 1.2+ ]+ },+ "intro_lu": {+ "n": 7,+ "median": -1.1,+ "p10": -8.08,+ "p90": 0.58,+ "min": -17.5,+ "max": 2.2,+ "values": [+ -1.1,+ -1.8,+ -0.5,+ -17.5,+ -1.0,+ 2.2,+ -1.3+ ]+ },+ "end_lu": {+ "n": 7,+ "median": -3.0,+ "p10": -11.96,+ "p90": -0.66,+ "min": -14.3,+ "max": -0.3,+ "values": [+ -2.5,+ -3.0,+ -14.3,+ -5.5,+ -10.4,+ -0.3,+ -0.9+ ]+ },+ "peak_pos": {+ "n": 7,+ "median": 0.75,+ "p10": 0.096,+ "p90": 0.826,+ "min": 0.03,+ "max": 0.88,+ "values": [+ 0.62,+ 0.75,+ 0.79,+ 0.78,+ 0.14,+ 0.03,+ 0.88+ ]+ },+ "first_sound_s": {+ "n": 7,+ "median": 0.13,+ "p10": 0.052,+ "p90": 0.402,+ "min": 0.04,+ "max": 0.48,+ "values": [+ 0.48,+ 0.13,+ 0.11,+ 0.35,+ 0.34,+ 0.04,+ 0.06+ ]+ },+ "max_gap_s": {+ "n": 7,+ "median": 0.02,+ "p10": 0.0,+ "p90": 0.688,+ "min": 0.0,+ "max": 1.66,+ "values": [+ 0.04,+ 0.0,+ 0.0,+ 0.04,+ 1.66,+ 0.02,+ 0.0+ ]+ },+ "end_silence_s": {+ "n": 7,+ "median": 0.04,+ "p10": 0.02,+ "p90": 1.162,+ "min": 0.02,+ "max": 1.9,+ "values": [+ 1.9,+ 0.04,+ 0.02,+ 0.02,+ 0.67,+ 0.28,+ 0.02+ ]+ },+ "fade_s": {+ "n": 7,+ "median": 0.27,+ "p10": 0.092,+ "p90": 2.288,+ "min": 0.08,+ "max": 2.54,+ "values": [+ 2.12,+ 0.27,+ 2.54,+ 1.64,+ 0.25,+ 0.1,+ 0.08+ ]+ },+ "final_lu": {+ "n": 7,+ "median": 0.2,+ "p10": -1.42,+ "p90": 5.18,+ "min": -2.2,+ "max": 5.6,+ "values": [+ 1.1,+ 0.2,+ 5.6,+ -0.9,+ -2.2,+ 4.9,+ -0.0+ ]+ },+ "tail_s": {+ "n": 6,+ "median": 2.99,+ "p10": 1.065,+ "p90": 3.58,+ "min": 0.13,+ "max": 3.58,+ "values": [+ 3.58,+ 2.44,+ 3.54,+ 0.13,+ 3.58,+ 2.0,+ null+ ]+ },+ "music_frac": {+ "n": 7,+ "median": 1.0,+ "p10": 0.802,+ "p90": 1.0,+ "min": 0.67,+ "max": 1.0,+ "values": [+ 1.0,+ 1.0,+ 1.0,+ 1.0,+ 0.67,+ 1.0,+ 0.89+ ]+ },+ "speech_frac": {+ "n": 7,+ "median": 0.83,+ "p10": 0.0,+ "p90": 1.0,+ "min": 0.0,+ "max": 1.0,+ "values": [+ 0.95,+ 0.83,+ 1.0,+ 0.0,+ 0.0,+ 1.0,+ 0.0+ ]+ },+ "tempo": {+ "n": 7,+ "median": 123.0,+ "p10": 115.2,+ "p90": 136.0,+ "min": 108.0,+ "max": 136.0,+ "values": [+ 136.0,+ 120.0,+ 108.0,+ 123.0,+ 136.0,+ 133.0,+ 123.0+ ]+ },+ "onset_rate": {+ "n": 7,+ "median": 5.66,+ "p10": 3.27,+ "p90": 6.876,+ "min": 2.46,+ "max": 8.22,+ "values": [+ 5.87,+ 5.98,+ 3.81,+ 8.22,+ 4.03,+ 5.66,+ 2.46+ ]+ },+ "hit_rate": {+ "n": 7,+ "median": 1.25,+ "p10": 0.748,+ "p90": 1.6,+ "min": 0.1,+ "max": 1.66,+ "values": [+ 1.56,+ 1.21,+ 1.66,+ 1.18,+ 1.25,+ 1.3,+ 0.1+ ]+ },+ "stopdowns_min": {+ "n": 7,+ "median": 1.2,+ "p10": 0.0,+ "p90": 9.06,+ "min": 0.0,+ "max": 18.3,+ "values": [+ 1.3,+ 0.0,+ 2.9,+ 0.0,+ 18.3,+ 1.2,+ 0.0+ ]+ },+ "cut_rate": {+ "n": 7,+ "median": 0.63,+ "p10": 0.306,+ "p90": 1.42,+ "min": 0.3,+ "max": 1.6,+ "values": [+ 0.3,+ 0.31,+ 0.83,+ 1.3,+ 0.54,+ 0.63,+ 1.6+ ]+ },+ "cuts_with_sound": {+ "n": 7,+ "median": 0.81,+ "p10": 0.56,+ "p90": 0.934,+ "min": 0.5,+ "max": 1.0,+ "values": [+ 0.89,+ 0.67,+ 0.88,+ 1.0,+ 0.5,+ 0.81,+ 0.6+ ]+ },+ "cut_hit_frac": {+ "n": 7,+ "median": 0.29,+ "p10": 0.078,+ "p90": 0.488,+ "min": 0.03,+ "max": 0.53,+ "values": [+ 0.46,+ 0.11,+ 0.53,+ 0.24,+ 0.31,+ 0.29,+ 0.03+ ]+ },+ "cut_frame_sync": {+ "n": 7,+ "median": 0.44,+ "p10": 0.34,+ "p90": 0.592,+ "min": 0.28,+ "max": 0.67,+ "values": [+ 0.54,+ 0.44,+ 0.47,+ 0.67,+ 0.38,+ 0.39,+ 0.28+ ]+ },+ "cuts_on_beat": {+ "n": 7,+ "median": 0.22,+ "p10": 0.198,+ "p90": 0.448,+ "min": 0.18,+ "max": 0.52,+ "values": [+ 0.21,+ 0.22,+ 0.18,+ 0.4,+ 0.38,+ 0.52,+ 0.22+ ]+ },+ "band_sub": {+ "n": 7,+ "median": -6.6,+ "p10": -12.36,+ "p90": -4.58,+ "min": -16.2,+ "max": -4.1,+ "values": [+ -9.8,+ -6.8,+ -16.2,+ -6.6,+ -4.1,+ -6.4,+ -4.9+ ]+ },+ "band_low": {+ "n": 7,+ "median": -4.2,+ "p10": -5.92,+ "p90": -2.34,+ "min": -6.7,+ "max": -2.1,+ "values": [+ -3.5,+ -5.4,+ -6.7,+ -2.1,+ -4.2,+ -4.4,+ -2.5+ ]+ },+ "band_lowmid": {+ "n": 7,+ "median": -5.3,+ "p10": -16.38,+ "p90": -3.2,+ "min": -16.8,+ "max": -1.7,+ "values": [+ -4.9,+ -4.2,+ -1.7,+ -8.4,+ -16.8,+ -5.3,+ -16.1+ ]+ },+ "band_mid": {+ "n": 7,+ "median": -12.4,+ "p10": -15.92,+ "p90": -10.68,+ "min": -19.1,+ "max": -10.5,+ "values": [+ -11.0,+ -10.5,+ -10.8,+ -19.1,+ -13.6,+ -13.8,+ -12.4+ ]+ },+ "band_presence": {+ "n": 7,+ "median": -17.8,+ "p10": -30.24,+ "p90": -13.52,+ "min": -30.3,+ "max": -13.1,+ "values": [+ -14.6,+ -17.8,+ -30.2,+ -30.3,+ -13.1,+ -13.8,+ -17.8+ ]+ },+ "band_air": {+ "n": 7,+ "median": -23.6,+ "p10": -31.86,+ "p90": -14.74,+ "min": -34.2,+ "max": -11.2,+ "values": [+ -23.7,+ -21.8,+ -34.2,+ -30.3,+ -11.2,+ -17.1,+ -23.6+ ]+ },+ "brightness_hz": {+ "n": 7,+ "median": 1966.0,+ "p10": 1183.4,+ "p90": 3030.0,+ "min": 1169.0,+ "max": 3633.0,+ "values": [+ 1966.0,+ 1193.0,+ 1169.0,+ 1220.0,+ 3633.0,+ 1976.0,+ 2628.0+ ]+ },+ "side_to_mid_db": {+ "n": 7,+ "median": -8.5,+ "p10": -11.8,+ "p90": -5.78,+ "min": -14.5,+ "max": -5.0,+ "values": [+ -14.5,+ -8.5,+ -7.1,+ -6.3,+ -5.0,+ -10.0,+ -9.1+ ]+ },+ "correlation": {+ "n": 7,+ "median": 0.75,+ "p10": 0.58,+ "p90": 0.864,+ "min": 0.52,+ "max": 0.93,+ "values": [+ 0.93,+ 0.75,+ 0.68,+ 0.62,+ 0.52,+ 0.82,+ 0.78+ ]+ }+ },+ "chance": {+ "cuts_with_sound": {+ "n": 7,+ "median": 0.86,+ "p10": 0.464,+ "p90": 0.936,+ "min": 0.44,+ "max": 0.99,+ "values": [+ 0.9,+ 0.89,+ 0.68,+ 0.99,+ 0.48,+ 0.86,+ 0.44+ ]+ },+ "cut_hit_frac": {+ "n": 7,+ "median": 0.27,+ "p10": 0.152,+ "p90": 0.32,+ "min": 0.02,+ "max": 0.35,+ "values": [+ 0.3,+ 0.26,+ 0.35,+ 0.27,+ 0.24,+ 0.28,+ 0.02+ ]+ },+ "cut_frame_sync": {+ "n": 7,+ "median": 0.44,+ "p10": 0.22,+ "p90": 0.622,+ "min": 0.19,+ "max": 0.7,+ "values": [+ 0.57,+ 0.47,+ 0.36,+ 0.7,+ 0.24,+ 0.44,+ 0.19+ ]+ },+ "cuts_on_beat": {+ "n": 7,+ "median": 0.28,+ "p10": 0.242,+ "p90": 0.314,+ "min": 0.23,+ "max": 0.32,+ "values": [+ 0.32,+ 0.28,+ 0.23,+ 0.28,+ 0.25,+ 0.31,+ 0.27+ ]+ }+ },+ "arc": {+ "median": [+ -1.3,+ -0.8,+ 0.3,+ -0.3,+ -0.2,+ 0.2,+ 0.0,+ -0.4,+ 0.9,+ -0.3,+ -7.4+ ],+ "p10": [+ -12.4,+ -5.1,+ -4.8,+ -2.9,+ -3.0,+ -0.8,+ -2.7,+ -2.4,+ -3.3,+ -5.8,+ -12.2+ ],+ "p90": [+ 1.3,+ -0.1,+ 1.0,+ 0.7,+ 0.4,+ 1.3,+ 0.8,+ 1.4,+ 2.0,+ 0.7,+ -2.5+ ]+ },+ "films": [+ {+ "id": "ref_1",+ "style": "AI video-search SaaS launch: white UI walkthrough with chat prompts and cursor, kinetic type, thumbnail-wall reveals, logo end card; voice-over",+ "duration": 93.12,+ "has_video": true,+ "metrics": {+ "lufs_i": -12.1,+ "true_peak": 0.1,+ "lra": 1.7,+ "crest_lu": 1.1,+ "intro_lu": -1.1,+ "end_lu": -2.5,+ "peak_pos": 0.62,+ "first_sound_s": 0.48,+ "max_gap_s": 0.04,+ "end_silence_s": 1.9,+ "fade_s": 2.12,+ "final_lu": 1.1,+ "tail_s": 3.58,+ "music_frac": 1.0,+ "speech_frac": 0.95,+ "tempo": 136,+ "onset_rate": 5.87,+ "hit_rate": 1.56,+ "stopdowns_min": 1.3,+ "cut_rate": 0.3,+ "cuts_with_sound": 0.89,+ "cut_hit_frac": 0.46,+ "cut_frame_sync": 0.54,+ "cuts_on_beat": 0.21,+ "band_sub": -9.8,+ "band_low": -3.5,+ "band_lowmid": -4.9,+ "band_mid": -11.0,+ "band_presence": -14.6,+ "band_air": -23.7,+ "brightness_hz": 1966,+ "side_to_mid_db": -14.5,+ "correlation": 0.93+ },+ "chance": {+ "cuts_with_sound": 0.9,+ "cut_hit_frac": 0.3,+ "cut_frame_sync": 0.57,+ "cuts_on_beat": 0.32+ },+ "arc": [+ -1.3,+ -0.5,+ 0.3,+ -0.0,+ 0.4,+ -0.1,+ 0.4,+ -0.4,+ 0.1,+ -0.1,+ -11.4+ ]+ },+ {+ "id": "ref_2",+ "style": "AI workspace explainer: white minimal kinetic type, app-icon clusters, chat UI, photographic logo reveal, gradient end card; voice-over",+ "duration": 28.909,+ "has_video": true,+ "metrics": {+ "lufs_i": -14.8,+ "true_peak": 1.0,+ "lra": 3.6,+ "crest_lu": 2.4,+ "intro_lu": -1.8,+ "end_lu": -3.0,+ "peak_pos": 0.75,+ "first_sound_s": 0.13,+ "max_gap_s": 0.0,+ "end_silence_s": 0.04,+ "fade_s": 0.27,+ "final_lu": 0.2,+ "tail_s": 2.44,+ "music_frac": 1.0,+ "speech_frac": 0.83,+ "tempo": 120,+ "onset_rate": 5.98,+ "hit_rate": 1.21,+ "stopdowns_min": 0.0,+ "cut_rate": 0.31,+ "cuts_with_sound": 0.67,+ "cut_hit_frac": 0.11,+ "cut_frame_sync": 0.44,+ "cuts_on_beat": 0.22,+ "band_sub": -6.8,+ "band_low": -5.4,+ "band_lowmid": -4.2,+ "band_mid": -10.5,+ "band_presence": -17.8,+ "band_air": -21.8,+ "brightness_hz": 1193,+ "side_to_mid_db": -8.5,+ "correlation": 0.75+ },+ "chance": {+ "cuts_with_sound": 0.89,+ "cut_hit_frac": 0.26,+ "cut_frame_sync": 0.47,+ "cuts_on_beat": 0.28+ },+ "arc": [+ -1.0,+ -2.2,+ 0.3,+ 0.7,+ -0.2,+ 0.4,+ -0.0,+ 1.2,+ 1.0,+ -1.6,+ -4.1+ ]+ },+ {+ "id": "ref_3",+ "style": "editorial kinetic collage: grainy paper texture, typewriter text, photo cut-outs, pixel icons, hand-drawn strokes, fast cuts; voice-over",+ "duration": 20.459,+ "has_video": true,+ "metrics": {+ "lufs_i": -10.4,+ "true_peak": 0.6,+ "lra": 6.4,+ "crest_lu": 2.3,+ "intro_lu": -0.5,+ "end_lu": -14.3,+ "peak_pos": 0.79,+ "first_sound_s": 0.11,+ "max_gap_s": 0.0,+ "end_silence_s": 0.02,+ "fade_s": 2.54,+ "final_lu": 5.6,+ "tail_s": 3.54,+ "music_frac": 1.0,+ "speech_frac": 1.0,+ "tempo": 108,+ "onset_rate": 3.81,+ "hit_rate": 1.66,+ "stopdowns_min": 2.9,+ "cut_rate": 0.83,+ "cuts_with_sound": 0.88,+ "cut_hit_frac": 0.53,+ "cut_frame_sync": 0.47,+ "cuts_on_beat": 0.18,+ "band_sub": -16.2,+ "band_low": -6.7,+ "band_lowmid": -1.7,+ "band_mid": -10.8,+ "band_presence": -30.2,+ "band_air": -34.2,+ "brightness_hz": 1169,+ "side_to_mid_db": -7.1,+ "correlation": 0.68+ },+ "chance": {+ "cuts_with_sound": 0.68,+ "cut_hit_frac": 0.35,+ "cut_frame_sync": 0.36,+ "cuts_on_beat": 0.23+ },+ "arc": [+ -0.5,+ -3.0,+ -7.4,+ -0.3,+ -1.7,+ -1.8,+ 0.2,+ -1.0,+ 2.4,+ -1.3,+ -13.5+ ]+ },+ {+ "id": "ref_4",+ "style": "fintech brand film: bold black and green kinetic typography, grids and numerals, flicker cuts, logo stings; music only",+ "duration": 32.256,+ "has_video": true,+ "metrics": {+ "lufs_i": -14.8,+ "true_peak": -0.8,+ "lra": 7.0,+ "crest_lu": 2.0,+ "intro_lu": -17.5,+ "end_lu": -5.5,+ "peak_pos": 0.78,+ "first_sound_s": 0.35,+ "max_gap_s": 0.04,+ "end_silence_s": 0.02,+ "fade_s": 1.64,+ "final_lu": -0.9,+ "tail_s": 0.13,+ "music_frac": 1.0,+ "speech_frac": 0.0,+ "tempo": 123,+ "onset_rate": 8.22,+ "hit_rate": 1.18,+ "stopdowns_min": 0.0,+ "cut_rate": 1.3,+ "cuts_with_sound": 1.0,+ "cut_hit_frac": 0.24,+ "cut_frame_sync": 0.67,+ "cuts_on_beat": 0.4,+ "band_sub": -6.6,+ "band_low": -2.1,+ "band_lowmid": -8.4,+ "band_mid": -19.1,+ "band_presence": -30.3,+ "band_air": -30.3,+ "brightness_hz": 1220,+ "side_to_mid_db": -6.3,+ "correlation": 0.62+ },+ "chance": {+ "cuts_with_sound": 0.99,+ "cut_hit_frac": 0.27,+ "cut_frame_sync": 0.7,+ "cuts_on_beat": 0.28+ },+ "arc": [+ -19.8,+ -8.2,+ -1.8,+ -1.1,+ -1.4,+ 0.2,+ 1.4,+ 1.6,+ 1.8,+ -0.3,+ -10.2+ ]+ },+ {+ "id": "ref_5",+ "style": "AI image-generation launch: soft gradients, typed prompt, collage of generated photos; sound design with hard silences",+ "duration": 29.525,+ "has_video": true,+ "metrics": {+ "lufs_i": -22.2,+ "true_peak": 1.2,+ "lra": 12.2,+ "crest_lu": 3.2,+ "intro_lu": -1.0,+ "end_lu": -10.4,+ "peak_pos": 0.14,+ "first_sound_s": 0.34,+ "max_gap_s": 1.66,+ "end_silence_s": 0.67,+ "fade_s": 0.25,+ "final_lu": -2.2,+ "tail_s": 3.58,+ "music_frac": 0.67,+ "speech_frac": 0.0,+ "tempo": 136,+ "onset_rate": 4.03,+ "hit_rate": 1.25,+ "stopdowns_min": 18.3,+ "cut_rate": 0.54,+ "cuts_with_sound": 0.5,+ "cut_hit_frac": 0.31,+ "cut_frame_sync": 0.38,+ "cuts_on_beat": 0.38,+ "band_sub": -4.1,+ "band_low": -4.2,+ "band_lowmid": -16.8,+ "band_mid": -13.6,+ "band_presence": -13.1,+ "band_air": -11.2,+ "brightness_hz": 3633,+ "side_to_mid_db": -5.0,+ "correlation": 0.52+ },+ "chance": {+ "cuts_with_sound": 0.48,+ "cut_hit_frac": 0.24,+ "cut_frame_sync": 0.24,+ "cuts_on_beat": 0.25+ },+ "arc": [+ -7.5,+ -0.8,+ 1.8,+ -5.5,+ -5.0,+ 0.9,+ -0.9,+ -4.4,+ -5.5,+ -12.0,+ -7.4+ ]+ },+ {+ "id": "ref_6",+ "style": "creator-commerce SaaS: blue UI walkthrough, notifications, cursor clicks, data-flow diagrams, tagline end card; voice-over",+ "duration": 49.28,+ "has_video": true,+ "metrics": {+ "lufs_i": -17.4,+ "true_peak": 0.1,+ "lra": 5.5,+ "crest_lu": 3.5,+ "intro_lu": 2.2,+ "end_lu": -0.3,+ "peak_pos": 0.03,+ "first_sound_s": 0.04,+ "max_gap_s": 0.02,+ "end_silence_s": 0.28,+ "fade_s": 0.1,+ "final_lu": 4.9,+ "tail_s": 2.0,+ "music_frac": 1.0,+ "speech_frac": 1.0,+ "tempo": 133,+ "onset_rate": 5.66,+ "hit_rate": 1.3,+ "stopdowns_min": 1.2,+ "cut_rate": 0.63,+ "cuts_with_sound": 0.81,+ "cut_hit_frac": 0.29,+ "cut_frame_sync": 0.39,+ "cuts_on_beat": 0.52,+ "band_sub": -6.4,+ "band_low": -4.4,+ "band_lowmid": -5.3,+ "band_mid": -13.8,+ "band_presence": -13.8,+ "band_air": -17.1,+ "brightness_hz": 1976,+ "side_to_mid_db": -10.0,+ "correlation": 0.82+ },+ "chance": {+ "cuts_with_sound": 0.86,+ "cut_hit_frac": 0.28,+ "cut_frame_sync": 0.44,+ "cuts_on_beat": 0.31+ },+ "arc": [+ 3.9,+ -0.8,+ -3.1,+ -0.8,+ -0.1,+ 1.8,+ -0.6,+ -1.0,+ -1.8,+ 0.8,+ -2.4+ ]+ },+ {+ "id": "ref_7",+ "style": "gaming motion-design reel: logo stings, 3D explosions, saturated colour, very fast cuts, brickwall-loud music",+ "duration": 41.792,+ "has_video": true,+ "metrics": {+ "lufs_i": -7.3,+ "true_peak": 2.0,+ "lra": 3.3,+ "crest_lu": 1.2,+ "intro_lu": -1.3,+ "end_lu": -0.9,+ "peak_pos": 0.88,+ "first_sound_s": 0.06,+ "max_gap_s": 0.0,+ "end_silence_s": 0.02,+ "fade_s": 0.08,+ "final_lu": -0.0,+ "tail_s": null,+ "music_frac": 0.89,+ "speech_frac": 0.0,+ "tempo": 123,+ "onset_rate": 2.46,+ "hit_rate": 0.1,+ "stopdowns_min": 0.0,+ "cut_rate": 1.6,+ "cuts_with_sound": 0.6,+ "cut_hit_frac": 0.03,+ "cut_frame_sync": 0.28,+ "cuts_on_beat": 0.22,+ "band_sub": -4.9,+ "band_low": -2.5,+ "band_lowmid": -16.1,+ "band_mid": -12.4,+ "band_presence": -17.8,+ "band_air": -23.6,+ "brightness_hz": 2628,+ "side_to_mid_db": -9.1,+ "correlation": 0.78+ },+ "chance": {+ "cuts_with_sound": 0.44,+ "cut_hit_frac": 0.02,+ "cut_frame_sync": 0.19,+ "cuts_on_beat": 0.27+ },+ "arc": [+ -4.4,+ 0.5,+ 0.4,+ 0.7,+ 0.5,+ 0.2,+ -5.4,+ 0.6,+ 0.9,+ 0.7,+ -2.6+ ]+ }+ ],+ "warnings": [],+ "summary": "profile of 7 reference films (medians): integrated -14.8 LUFS (p10-p90 -19.3 LUFS .. -9.2 LUFS), LRA 5.5 LU, onsets 5.66 /s, salient hits 1.25 /s, music in 100% of 5 s windows, visual events 0.63 /s."+}\ No newline at end of filediff --git a/earshot/data/realness_model.json b/earshot/data/realness_model.jsonnew file mode 100644index 0000000..edf8bbc--- /dev/null+++ b/earshot/data/realness_model.json@@ -0,0 +1,2931 @@+{+ "version": 1,+ "trained": "0ddd1e75a342",+ "about": "p_real = sigmoid(fusion over head logits); heads are standardised logistic regressions. clap head: laion/larger_clap_general audio embedding (repeat-padded, one window per clip <= 6 s, else 4 s windows). signal head: earshot.realness.signal_features, coefficients sign-constrained to their physical direction. Positives: real recordings (Freesound/Mixkit), pro launch-film soundtracks, pro music; negatives: code-synthesized audio (cinetic synth engine, a retired synthesized film score, generic numpy generators).",+ "data": {+ "clips": 2198,+ "clean": 1484,+ "augmented": 714,+ "real": 691,+ "synth": 793,+ "groups": [+ "ambience",+ "cinetic",+ "film",+ "foley",+ "garden",+ "generic",+ "keyboard",+ "magic",+ "motion",+ "music:ch",+ "music:mk",+ "music:other",+ "music:sb",+ "retired",+ "toy"+ ]+ },+ "long_s": 6.0,+ "win_s": 4.0,+ "silent_db": -60.0,+ "heads": {+ "clap": {+ "mean": [+ -0.052052,+ 0.010976,+ -0.004569,+ -0.003044,+ -0.005411,+ 0.026436,+ -0.017017,+ 0.028811,+ 0.019686,+ -0.021169,+ -0.022143,+ 0.02707,+ -0.029073,+ 0.00241,+ -0.016344,+ -0.000995,+ 0.003438,+ -0.002563,+ 0.014137,+ -0.011145,+ 0.014577,+ 0.014147,+ 0.011586,+ 0.02613,+ -0.019365,+ 0.001896,+ -0.002179,+ 0.014947,+ -0.03729,+ -0.006841,+ 0.041042,+ 0.000769,+ -0.001643,+ 0.00586,+ -0.009046,+ 0.002283,+ 0.025049,+ 0.018162,+ -0.012405,+ -0.03027,+ -0.012354,+ -0.027211,+ -0.012911,+ -0.007307,+ -0.027262,+ 0.010318,+ 0.013678,+ 0.020977,+ -0.008354,+ 0.044987,+ -0.0451,+ -0.017171,+ -0.030465,+ -0.004234,+ -0.008269,+ 0.016205,+ 0.011159,+ -0.000114,+ -0.011801,+ -0.013,+ 0.021533,+ -0.01395,+ -0.036966,+ -0.028105,+ 0.020855,+ -0.000518,+ -0.008804,+ 0.017363,+ 0.035451,+ -0.00591,+ 0.018849,+ 0.009891,+ -0.021618,+ 0.035145,+ 0.008708,+ -0.000333,+ 0.016824,+ 0.021497,+ 0.01148,+ 0.00495,+ -0.026559,+ -0.001794,+ -0.009823,+ 0.002874,+ -0.029703,+ 0.008199,+ -0.010536,+ 0.033249,+ 0.029878,+ 0.006114,+ -0.02597,+ 0.002441,+ -0.035933,+ -0.02593,+ 0.005504,+ 0.030967,+ -0.012626,+ -0.056255,+ 0.003912,+ 0.005915,+ -0.007281,+ -0.008202,+ 0.013946,+ 0.011283,+ -0.011522,+ -0.011447,+ -0.029182,+ 0.007981,+ 0.029232,+ 0.00321,+ -0.058353,+ 0.013941,+ -0.001954,+ 0.000442,+ 0.006888,+ 0.008312,+ -0.007807,+ -0.03804,+ 0.026441,+ -0.015196,+ 0.015464,+ 0.015224,+ -0.009339,+ 0.000496,+ -0.006727,+ -0.001686,+ -0.006997,+ 0.011305,+ 0.033801,+ -0.055747,+ -0.003654,+ 0.002984,+ -0.006481,+ -0.035892,+ 0.002861,+ 0.022777,+ 0.001311,+ 0.00738,+ 0.017051,+ -0.006361,+ -0.05056,+ 7.2e-05,+ 0.016366,+ 0.020521,+ 0.018378,+ 0.017469,+ 0.011309,+ -0.008491,+ -0.033222,+ -0.000619,+ -0.004876,+ 0.000657,+ -0.011388,+ 0.030756,+ -0.0122,+ -0.049721,+ -0.010717,+ 0.001908,+ 0.062097,+ 0.006526,+ 0.010608,+ 0.017763,+ -0.031974,+ 0.016324,+ 0.006162,+ -0.014149,+ 0.027702,+ 0.040241,+ 0.020436,+ 0.012412,+ -0.029644,+ 0.002578,+ -0.00075,+ -0.012359,+ -0.009838,+ -0.05347,+ 0.013857,+ 0.005574,+ -0.039945,+ -0.013577,+ 0.014435,+ -0.016033,+ 0.015408,+ 0.004547,+ -0.001405,+ 0.004249,+ -0.038841,+ -0.010332,+ -0.008464,+ -0.020923,+ -0.005048,+ 0.007335,+ 0.016746,+ -0.005693,+ 0.030425,+ 0.010702,+ -0.028092,+ 0.028576,+ -0.013736,+ -0.01955,+ 0.025166,+ -0.024728,+ 0.022122,+ 0.033136,+ 0.017173,+ 0.000796,+ 0.021941,+ 0.04024,+ 0.0065,+ -0.026371,+ 0.019036,+ -0.017052,+ 0.001036,+ 0.005257,+ 0.000459,+ 0.013542,+ 0.008417,+ -0.004983,+ 0.005982,+ -0.005807,+ 0.07088,+ -0.021949,+ 0.000402,+ 0.031062,+ -0.024443,+ -0.037339,+ -0.011588,+ 0.016751,+ 0.040752,+ -0.03953,+ -0.009752,+ -0.00782,+ -0.027718,+ -0.003333,+ 0.016281,+ 0.018963,+ 0.008869,+ -0.029852,+ -0.004513,+ -0.008626,+ -0.011971,+ 0.002944,+ -0.010338,+ -0.010721,+ 0.033495,+ 0.001736,+ 0.002409,+ 0.028071,+ -0.003041,+ 0.018811,+ -0.022422,+ 0.012639,+ -0.020539,+ -0.01886,+ -0.015742,+ 0.030108,+ 0.005355,+ 0.008012,+ 0.008047,+ 0.011044,+ -0.007858,+ -0.031354,+ 0.005827,+ -0.007449,+ 0.022778,+ -0.030285,+ 0.009578,+ -0.006494,+ 0.015044,+ -0.017281,+ -0.020953,+ 0.034053,+ 0.001021,+ 0.019848,+ -0.026488,+ -0.001908,+ -0.011668,+ -0.042836,+ -0.018499,+ -0.052762,+ -0.024251,+ 0.021304,+ 0.050344,+ -0.018274,+ 0.008663,+ -0.031931,+ -0.008471,+ 0.004718,+ -0.054488,+ 0.012766,+ 0.011084,+ -0.00229,+ -0.02377,+ 0.003266,+ -0.016413,+ 0.010888,+ 0.001345,+ 0.001521,+ 0.000979,+ -0.013347,+ -0.011841,+ -0.007964,+ 0.031646,+ 0.003424,+ -0.007488,+ 0.031762,+ -0.01289,+ 0.026183,+ 0.009796,+ 0.022316,+ -0.011907,+ 0.024812,+ -0.007521,+ -0.00585,+ 0.029919,+ -0.031959,+ -0.013494,+ -0.057312,+ -0.001964,+ -0.004058,+ 0.001409,+ 0.027736,+ -0.013119,+ 0.00425,+ -0.027138,+ -0.03019,+ -0.018986,+ 0.02076,+ -0.020836,+ 0.030703,+ 0.028342,+ -0.006096,+ -0.020485,+ 0.04031,+ 0.007613,+ 0.005876,+ 0.01988,+ 0.001644,+ 0.036376,+ 0.04949,+ 0.049277,+ 0.016384,+ 0.028957,+ -0.005255,+ -0.021786,+ -0.014548,+ -0.017334,+ -0.003681,+ 0.015463,+ -0.053005,+ 0.016081,+ -0.026686,+ -0.016345,+ -0.01626,+ 0.053603,+ 0.001245,+ -0.01297,+ 0.012214,+ -0.007412,+ 0.020754,+ 0.00095,+ -0.024119,+ -0.006635,+ -0.000203,+ -0.047805,+ -0.029228,+ -0.010981,+ 0.02146,+ -0.032135,+ 0.01565,+ -0.014084,+ 0.011774,+ -0.002722,+ -0.035301,+ -0.047962,+ 0.009619,+ 7e-05,+ 0.019724,+ -0.018379,+ -0.020813,+ 0.006211,+ -0.046887,+ -0.033982,+ 0.006263,+ 0.004226,+ -0.019837,+ -0.013674,+ 0.006539,+ -0.027232,+ 0.023231,+ -0.010683,+ 0.00457,+ -0.028055,+ 0.031922,+ 0.016797,+ 0.026962,+ 0.014091,+ 0.005545,+ -0.027142,+ -0.050815,+ 0.0164,+ -0.007391,+ 0.016524,+ -1.2e-05,+ -0.004784,+ -0.009855,+ 0.033591,+ 0.004799,+ 0.031913,+ 0.009923,+ -0.022216,+ 0.026065,+ -0.019847,+ -0.013039,+ 0.028232,+ 0.002344,+ -0.014404,+ 0.04857,+ 0.056599,+ -0.037344,+ 0.004882,+ -0.018246,+ 0.022998,+ -0.006521,+ 0.005804,+ 0.01657,+ 0.023362,+ 0.0135,+ 0.005162,+ 0.032901,+ 0.016672,+ 0.006357,+ 0.028973,+ 0.015682,+ -0.004169,+ -0.006586,+ 0.013674,+ -0.013177,+ 0.002819,+ 0.006835,+ -0.002713,+ -0.03448,+ 0.013748,+ 0.010378,+ -0.005356,+ -0.004288,+ 0.011127,+ 0.017186,+ -0.00875,+ -0.025657,+ -0.014734,+ 0.000794,+ -0.04212,+ 0.014893,+ -0.033825,+ -0.001013,+ -0.001951,+ -0.019484,+ 0.019202,+ -0.010559,+ 0.03127,+ -0.005421,+ 0.02027,+ 0.000176,+ 0.010505,+ -0.002503,+ 0.025661,+ 0.006047,+ -0.030755,+ 0.017296,+ -0.013436,+ 0.017607,+ 0.009469,+ -0.001727,+ 0.040798,+ -0.026485,+ 0.014848,+ 0.022048,+ -0.008641,+ -0.009585,+ -0.021602,+ -0.010853,+ 0.017525,+ -0.006726,+ -0.006435,+ 0.01254,+ 0.010467,+ -0.009174,+ 0.010279,+ 0.028514,+ -0.006701,+ 0.000576,+ 0.018453,+ -0.003924,+ -0.038595,+ 0.015499,+ 0.022411,+ 0.024945,+ 0.01519,+ -0.044208,+ -0.023382,+ 0.012422,+ -0.030343,+ -0.014023,+ -0.051741,+ 0.022914,+ 0.036537,+ 0.018642,+ 0.034063,+ 0.007402,+ 0.013189,+ 0.004047+ ],+ "scale": [+ 0.050475,+ 0.030355,+ 0.030734,+ 0.041901,+ 0.042426,+ 0.035508,+ 0.049647,+ 0.044821,+ 0.041865,+ 0.038287,+ 0.040085,+ 0.032945,+ 0.03235,+ 0.044179,+ 0.03354,+ 0.033994,+ 0.041557,+ 0.036535,+ 0.055282,+ 0.035124,+ 0.0399,+ 0.040092,+ 0.032845,+ 0.03827,+ 0.037326,+ 0.035933,+ 0.037232,+ 0.031767,+ 0.038341,+ 0.033341,+ 0.034773,+ 0.057302,+ 0.042033,+ 0.037878,+ 0.032621,+ 0.046161,+ 0.042344,+ 0.035722,+ 0.033958,+ 0.038071,+ 0.038686,+ 0.037831,+ 0.035801,+ 0.041091,+ 0.030907,+ 0.038416,+ 0.04196,+ 0.035343,+ 0.043432,+ 0.035406,+ 0.038895,+ 0.046694,+ 0.032564,+ 0.038585,+ 0.035491,+ 0.040398,+ 0.043199,+ 0.046062,+ 0.033173,+ 0.042178,+ 0.03362,+ 0.033383,+ 0.038654,+ 0.037204,+ 0.032981,+ 0.041449,+ 0.039267,+ 0.040797,+ 0.043258,+ 0.035577,+ 0.043506,+ 0.037379,+ 0.03809,+ 0.030944,+ 0.03446,+ 0.032168,+ 0.029313,+ 0.036597,+ 0.038882,+ 0.031058,+ 0.032652,+ 0.039514,+ 0.046626,+ 0.042047,+ 0.034482,+ 0.044053,+ 0.040404,+ 0.029954,+ 0.045133,+ 0.03848,+ 0.038589,+ 0.032041,+ 0.036364,+ 0.033058,+ 0.040434,+ 0.031063,+ 0.036078,+ 0.032255,+ 0.037057,+ 0.044724,+ 0.041868,+ 0.038451,+ 0.040233,+ 0.033884,+ 0.033202,+ 0.032467,+ 0.040384,+ 0.043215,+ 0.033238,+ 0.03621,+ 0.041257,+ 0.044428,+ 0.034612,+ 0.040448,+ 0.03873,+ 0.033927,+ 0.038093,+ 0.037996,+ 0.033715,+ 0.038321,+ 0.044019,+ 0.039179,+ 0.034655,+ 0.031389,+ 0.035118,+ 0.034516,+ 0.033945,+ 0.043769,+ 0.03594,+ 0.052615,+ 0.037794,+ 0.033044,+ 0.03894,+ 0.040379,+ 0.034066,+ 0.036633,+ 0.03692,+ 0.038144,+ 0.039939,+ 0.035316,+ 0.038615,+ 0.040902,+ 0.03034,+ 0.036421,+ 0.038342,+ 0.042669,+ 0.040354,+ 0.032451,+ 0.03211,+ 0.036199,+ 0.044707,+ 0.030202,+ 0.035593,+ 0.04137,+ 0.037812,+ 0.03609,+ 0.03647,+ 0.042413,+ 0.034291,+ 0.036104,+ 0.042988,+ 0.040692,+ 0.042657,+ 0.034828,+ 0.032891,+ 0.043075,+ 0.040305,+ 0.04091,+ 0.035763,+ 0.036976,+ 0.039645,+ 0.038172,+ 0.040072,+ 0.042214,+ 0.037706,+ 0.037094,+ 0.043967,+ 0.041344,+ 0.037228,+ 0.038288,+ 0.034784,+ 0.040158,+ 0.041158,+ 0.040505,+ 0.042259,+ 0.038182,+ 0.040854,+ 0.033817,+ 0.045729,+ 0.043845,+ 0.046013,+ 0.037788,+ 0.034547,+ 0.040028,+ 0.038065,+ 0.041534,+ 0.034516,+ 0.035128,+ 0.038067,+ 0.038339,+ 0.036049,+ 0.043899,+ 0.040407,+ 0.038741,+ 0.037339,+ 0.03781,+ 0.037834,+ 0.03789,+ 0.038931,+ 0.037309,+ 0.033144,+ 0.032126,+ 0.035256,+ 0.038085,+ 0.035256,+ 0.04266,+ 0.030869,+ 0.035088,+ 0.036373,+ 0.051991,+ 0.031688,+ 0.033326,+ 0.038573,+ 0.041712,+ 0.036409,+ 0.036883,+ 0.035696,+ 0.033397,+ 0.037225,+ 0.033829,+ 0.045618,+ 0.032804,+ 0.035596,+ 0.035724,+ 0.0331,+ 0.047186,+ 0.03684,+ 0.040029,+ 0.040028,+ 0.037858,+ 0.031107,+ 0.038923,+ 0.032551,+ 0.031358,+ 0.039447,+ 0.046948,+ 0.036609,+ 0.038397,+ 0.03684,+ 0.039072,+ 0.032447,+ 0.031801,+ 0.030642,+ 0.039247,+ 0.040608,+ 0.031025,+ 0.035592,+ 0.034422,+ 0.043454,+ 0.031033,+ 0.035426,+ 0.048045,+ 0.037981,+ 0.03416,+ 0.034631,+ 0.037765,+ 0.042798,+ 0.038598,+ 0.03849,+ 0.036844,+ 0.042166,+ 0.043283,+ 0.039815,+ 0.040415,+ 0.03622,+ 0.02928,+ 0.034667,+ 0.032242,+ 0.037878,+ 0.040883,+ 0.042603,+ 0.030796,+ 0.042815,+ 0.02834,+ 0.036631,+ 0.043182,+ 0.034833,+ 0.03652,+ 0.042146,+ 0.03739,+ 0.034958,+ 0.042228,+ 0.035229,+ 0.0405,+ 0.03686,+ 0.035127,+ 0.030196,+ 0.042866,+ 0.030123,+ 0.031729,+ 0.034795,+ 0.034523,+ 0.033364,+ 0.038607,+ 0.031988,+ 0.037,+ 0.040281,+ 0.039046,+ 0.035313,+ 0.04812,+ 0.02721,+ 0.033333,+ 0.036908,+ 0.050888,+ 0.031833,+ 0.042354,+ 0.031525,+ 0.044494,+ 0.045115,+ 0.040026,+ 0.038404,+ 0.03947,+ 0.033215,+ 0.044282,+ 0.03382,+ 0.03798,+ 0.035794,+ 0.031426,+ 0.036671,+ 0.044447,+ 0.044607,+ 0.037376,+ 0.036966,+ 0.035676,+ 0.04398,+ 0.033731,+ 0.039948,+ 0.034818,+ 0.032359,+ 0.031594,+ 0.040349,+ 0.03411,+ 0.033323,+ 0.045682,+ 0.045198,+ 0.040016,+ 0.036476,+ 0.037545,+ 0.038921,+ 0.028101,+ 0.040995,+ 0.037305,+ 0.031369,+ 0.033649,+ 0.039405,+ 0.03684,+ 0.033454,+ 0.041701,+ 0.036824,+ 0.040053,+ 0.034858,+ 0.042779,+ 0.032208,+ 0.040953,+ 0.037752,+ 0.043975,+ 0.03428,+ 0.041225,+ 0.03994,+ 0.033665,+ 0.03425,+ 0.041842,+ 0.041535,+ 0.038547,+ 0.040358,+ 0.034244,+ 0.039516,+ 0.032776,+ 0.028902,+ 0.038216,+ 0.045054,+ 0.037248,+ 0.036345,+ 0.037755,+ 0.039019,+ 0.040382,+ 0.040918,+ 0.036306,+ 0.039894,+ 0.039636,+ 0.041751,+ 0.035421,+ 0.037292,+ 0.041197,+ 0.040023,+ 0.036578,+ 0.046061,+ 0.031567,+ 0.037953,+ 0.048426,+ 0.029053,+ 0.039384,+ 0.041911,+ 0.045268,+ 0.035752,+ 0.040228,+ 0.042409,+ 0.038704,+ 0.037661,+ 0.045791,+ 0.04274,+ 0.032824,+ 0.033087,+ 0.035633,+ 0.041514,+ 0.034815,+ 0.035811,+ 0.048718,+ 0.03492,+ 0.035392,+ 0.044479,+ 0.041038,+ 0.038268,+ 0.032366,+ 0.033521,+ 0.042063,+ 0.04107,+ 0.037106,+ 0.036158,+ 0.043677,+ 0.026654,+ 0.039715,+ 0.037282,+ 0.040378,+ 0.043463,+ 0.038118,+ 0.051552,+ 0.043058,+ 0.032864,+ 0.036951,+ 0.038027,+ 0.046852,+ 0.032633,+ 0.040532,+ 0.031789,+ 0.033557,+ 0.03519,+ 0.038072,+ 0.042621,+ 0.033436,+ 0.039479,+ 0.030863,+ 0.035105,+ 0.033249,+ 0.043198,+ 0.034535,+ 0.038509,+ 0.032617,+ 0.036008,+ 0.045002,+ 0.042516,+ 0.032211,+ 0.034732,+ 0.032679,+ 0.042359,+ 0.035122,+ 0.031985,+ 0.042429,+ 0.038994,+ 0.035323,+ 0.036689,+ 0.040421,+ 0.036748,+ 0.040329,+ 0.039536,+ 0.034769,+ 0.03572,+ 0.036426,+ 0.037192,+ 0.036854,+ 0.034756,+ 0.039218,+ 0.03792,+ 0.043236,+ 0.043663,+ 0.0397,+ 0.03806,+ 0.032802,+ 0.038904,+ 0.037331,+ 0.0371,+ 0.039063,+ 0.0361,+ 0.035807,+ 0.033277,+ 0.039928,+ 0.041693,+ 0.036071,+ 0.040468,+ 0.040419,+ 0.032793,+ 0.044584,+ 0.038091,+ 0.041416,+ 0.0414,+ 0.038636,+ 0.042853,+ 0.036945,+ 0.035011,+ 0.034862,+ 0.039037,+ 0.03525+ ],+ "clip": [+ [+ -0.154729,+ -0.067196,+ -0.081139,+ -0.115281,+ -0.106854,+ -0.077683,+ -0.118854,+ -0.075252,+ -0.086352,+ -0.115735,+ -0.123217,+ -0.05358,+ -0.118662,+ -0.109033,+ -0.099481,+ -0.083356,+ -0.098264,+ -0.097384,+ -0.104927,+ -0.090233,+ -0.100858,+ -0.081891,+ -0.073235,+ -0.067505,+ -0.112022,+ -0.091521,+ -0.096442,+ -0.064542,+ -0.133976,+ -0.100851,+ -0.051151,+ -0.132017,+ -0.120786,+ -0.084384,+ -0.092173,+ -0.121797,+ -0.088504,+ -0.063516,+ -0.10117,+ -0.119639,+ -0.101489,+ -0.143214,+ -0.099455,+ -0.109628,+ -0.111521,+ -0.086326,+ -0.091467,+ -0.064046,+ -0.126712,+ -0.051396,+ -0.157442,+ -0.118085,+ -0.110938,+ -0.101131,+ -0.110475,+ -0.081783,+ -0.087546,+ -0.116834,+ -0.101887,+ -0.116964,+ -0.064424,+ -0.092412,+ -0.150675,+ -0.112902,+ -0.067852,+ -0.10495,+ -0.102316,+ -0.097421,+ -0.071371,+ -0.102623,+ -0.078136,+ -0.086325,+ -0.121495,+ -0.050934,+ -0.08796,+ -0.079462,+ -0.049633,+ -0.079523,+ -0.090721,+ -0.072739,+ -0.098372,+ -0.120345,+ -0.131441,+ -0.111814,+ -0.117963,+ -0.10246,+ -0.124294,+ -0.047636,+ -0.085196,+ -0.094332,+ -0.124842,+ -0.072352,+ -0.126688,+ -0.10803,+ -0.097473,+ -0.053719,+ -0.109209,+ -0.134328,+ -0.094296,+ -0.119174,+ -0.100351,+ -0.118926,+ -0.093287,+ -0.071752,+ -0.083458,+ -0.096832,+ -0.130691,+ -0.085562,+ -0.053476,+ -0.118072,+ -0.153924,+ -0.111254,+ -0.126232,+ -0.090013,+ -0.090334,+ -0.084356,+ -0.10395,+ -0.12709,+ -0.05262,+ -0.10025,+ -0.097082,+ -0.089695,+ -0.095783,+ -0.075906,+ -0.10006,+ -0.091723,+ -0.086574,+ -0.118014,+ -0.064266,+ -0.195084,+ -0.090847,+ -0.07997,+ -0.105387,+ -0.133492,+ -0.084612,+ -0.064574,+ -0.097656,+ -0.084206,+ -0.083779,+ -0.096455,+ -0.155618,+ -0.101971,+ -0.065402,+ -0.081234,+ -0.070908,+ -0.094354,+ -0.09627,+ -0.083589,+ -0.109974,+ -0.095011,+ -0.104911,+ -0.071249,+ -0.097282,+ -0.070452,+ -0.101844,+ -0.131982,+ -0.098625,+ -0.105099,+ -0.023381,+ -0.104114,+ -0.095226,+ -0.068288,+ -0.147339,+ -0.067423,+ -0.078765,+ -0.106795,+ -0.076801,+ -0.084125,+ -0.054405,+ -0.077112,+ -0.135902,+ -0.099291,+ -0.098413,+ -0.14162,+ -0.095763,+ -0.144614,+ -0.090573,+ -0.102509,+ -0.122354,+ -0.112475,+ -0.072155,+ -0.102343,+ -0.086363,+ -0.0947,+ -0.091428,+ -0.103039,+ -0.128216,+ -0.096768,+ -0.114634,+ -0.123786,+ -0.122414,+ -0.071258,+ -0.075045,+ -0.097695,+ -0.057023,+ -0.0971,+ -0.114063,+ -0.069344,+ -0.119098,+ -0.114151,+ -0.065989,+ -0.136529,+ -0.085744,+ -0.064909,+ -0.087737,+ -0.087184,+ -0.083651,+ -0.060697,+ -0.105601,+ -0.126203,+ -0.070272,+ -0.097579,+ -0.088482,+ -0.091711,+ -0.079112,+ -0.099536,+ -0.07087,+ -0.093035,+ -0.100184,+ -0.134544,+ -0.014582,+ -0.104701,+ -0.081824,+ -0.081528,+ -0.134312,+ -0.130773,+ -0.095729,+ -0.075339,+ -0.055382,+ -0.110621,+ -0.103011,+ -0.093356,+ -0.122941,+ -0.096558,+ -0.075069,+ -0.092544,+ -0.085923,+ -0.116721,+ -0.122231,+ -0.10598,+ -0.089938,+ -0.100689,+ -0.086998,+ -0.083348,+ -0.067417,+ -0.119592,+ -0.108836,+ -0.069967,+ -0.090978,+ -0.080142,+ -0.096308,+ -0.074693,+ -0.08952,+ -0.127223,+ -0.113109,+ -0.052171,+ -0.086613,+ -0.077031,+ -0.102729,+ -0.080439,+ -0.108057,+ -0.136857,+ -0.117531,+ -0.084275,+ -0.060768,+ -0.116319,+ -0.100277,+ -0.094894,+ -0.076647,+ -0.102609,+ -0.120046,+ -0.067125,+ -0.092937,+ -0.086557,+ -0.119191,+ -0.077652,+ -0.100318,+ -0.115073,+ -0.111237,+ -0.145347,+ -0.124399,+ -0.066492,+ -0.053889,+ -0.089071,+ -0.083611,+ -0.127805,+ -0.091393,+ -0.089332,+ -0.163292,+ -0.081796,+ -0.092099,+ -0.101976,+ -0.104206,+ -0.093392,+ -0.11162,+ -0.0713,+ -0.08136,+ -0.106571,+ -0.071535,+ -0.086608,+ -0.108721,+ -0.099728,+ -0.058129,+ -0.090685,+ -0.086311,+ -0.06298,+ -0.125396,+ -0.061751,+ -0.086295,+ -0.085308,+ -0.082485,+ -0.065494,+ -0.098443,+ -0.106266,+ -0.061116,+ -0.128097,+ -0.08404,+ -0.154385,+ -0.139156,+ -0.108081,+ -0.095942,+ -0.06923,+ -0.095913,+ -0.099237,+ -0.107576,+ -0.11428,+ -0.104177,+ -0.057521,+ -0.105446,+ -0.077709,+ -0.071922,+ -0.099354,+ -0.105643,+ -0.049067,+ -0.108242,+ -0.080386,+ -0.078483,+ -0.080571,+ -0.044779,+ -0.042236,+ -0.05514,+ -0.061125,+ -0.054196,+ -0.119121,+ -0.118578,+ -0.109125,+ -0.109114,+ -0.096601,+ -0.092461,+ -0.121802,+ -0.078365,+ -0.112157,+ -0.090953,+ -0.097063,+ -0.049947,+ -0.086781,+ -0.104832,+ -0.095201,+ -0.089875,+ -0.093423,+ -0.115753,+ -0.121912,+ -0.079761,+ -0.10365,+ -0.140171,+ -0.137888,+ -0.092587,+ -0.077294,+ -0.134598,+ -0.077849,+ -0.102629,+ -0.100619,+ -0.102775,+ -0.128647,+ -0.137687,+ -0.078639,+ -0.090751,+ -0.080392,+ -0.098153,+ -0.116412,+ -0.100155,+ -0.136882,+ -0.127946,+ -0.098264,+ -0.099186,+ -0.112681,+ -0.116715,+ -0.088263,+ -0.126162,+ -0.080494,+ -0.106397,+ -0.086487,+ -0.126219,+ -0.073568,+ -0.097209,+ -0.065049,+ -0.106141,+ -0.078243,+ -0.114842,+ -0.147591,+ -0.055729,+ -0.110585,+ -0.081338,+ -0.118838,+ -0.083864,+ -0.108034,+ -0.073454,+ -0.095305,+ -0.05939,+ -0.093432,+ -0.114747,+ -0.065255,+ -0.094766,+ -0.109978,+ -0.074246,+ -0.088068,+ -0.105679,+ -0.067524,+ -0.046488,+ -0.130666,+ -0.10819,+ -0.117754,+ -0.07277,+ -0.084923,+ -0.084838,+ -0.089862,+ -0.091907,+ -0.08396,+ -0.081425,+ -0.086436,+ -0.05124,+ -0.087414,+ -0.06095,+ -0.092328,+ -0.107714,+ -0.102203,+ -0.119702,+ -0.128213,+ -0.081603,+ -0.090024,+ -0.094431,+ -0.143053,+ -0.080963,+ -0.08258,+ -0.085894,+ -0.091869,+ -0.070993,+ -0.072807,+ -0.118887,+ -0.105026,+ -0.099949,+ -0.075051,+ -0.130118,+ -0.072661,+ -0.138942,+ -0.090125,+ -0.105244,+ -0.108157,+ -0.068367,+ -0.134512,+ -0.059377,+ -0.093035,+ -0.069844,+ -0.075598,+ -0.094145,+ -0.089992,+ -0.053814,+ -0.089214,+ -0.127714,+ -0.06837,+ -0.105258,+ -0.084896,+ -0.079653,+ -0.114469,+ -0.055332,+ -0.12575,+ -0.072065,+ -0.067321,+ -0.104106,+ -0.093455,+ -0.107464,+ -0.108287,+ -0.090794,+ -0.115549,+ -0.112991,+ -0.094918,+ -0.08693,+ -0.093811,+ -0.098518,+ -0.066858,+ -0.098187,+ -0.103786,+ -0.078658,+ -0.09032,+ -0.119197,+ -0.087004,+ -0.099045,+ -0.071262,+ -0.087602,+ -0.136919,+ -0.103317,+ -0.084987,+ -0.117434,+ -0.128615,+ -0.143389,+ -0.080124,+ -0.065305,+ -0.073078,+ -0.063511,+ -0.089806,+ -0.081585,+ -0.089154+ ],+ [+ 0.072373,+ 0.086475,+ 0.068243,+ 0.10308,+ 0.095035,+ 0.102194,+ 0.108352,+ 0.13945,+ 0.116626,+ 0.078373,+ 0.069801,+ 0.108306,+ 0.052137,+ 0.096047,+ 0.06221,+ 0.084963,+ 0.094774,+ 0.082905,+ 0.172698,+ 0.08242,+ 0.106582,+ 0.12338,+ 0.088863,+ 0.111141,+ 0.085764,+ 0.094299,+ 0.08834,+ 0.095628,+ 0.057585,+ 0.081413,+ 0.11777,+ 0.137714,+ 0.09169,+ 0.103571,+ 0.077026,+ 0.118481,+ 0.119469,+ 0.129742,+ 0.079908,+ 0.064958,+ 0.094097,+ 0.060445,+ 0.076791,+ 0.119891,+ 0.054774,+ 0.091488,+ 0.119884,+ 0.106219,+ 0.099814,+ 0.127145,+ 0.075208,+ 0.097581,+ 0.055848,+ 0.095633,+ 0.083456,+ 0.108935,+ 0.12109,+ 0.125582,+ 0.070573,+ 0.084651,+ 0.103227,+ 0.074154,+ 0.064192,+ 0.073689,+ 0.09923,+ 0.103955,+ 0.112639,+ 0.107431,+ 0.139768,+ 0.077136,+ 0.126266,+ 0.101024,+ 0.083906,+ 0.110754,+ 0.092899,+ 0.101782,+ 0.096059,+ 0.102456,+ 0.121282,+ 0.074744,+ 0.061246,+ 0.100853,+ 0.103648,+ 0.104181,+ 0.065048,+ 0.111998,+ 0.101734,+ 0.106475,+ 0.129031,+ 0.098926,+ 0.070228,+ 0.083917,+ 0.050537,+ 0.058979,+ 0.105064,+ 0.111632,+ 0.092161,+ 0.034142,+ 0.089322,+ 0.102227,+ 0.108333,+ 0.099756,+ 0.10763,+ 0.099984,+ 0.079242,+ 0.077411,+ 0.090352,+ 0.128038,+ 0.115598,+ 0.092329,+ 0.045313,+ 0.11437,+ 0.080718,+ 0.128817,+ 0.100844,+ 0.085907,+ 0.09001,+ 0.056954,+ 0.119423,+ 0.083109,+ 0.128768,+ 0.105695,+ 0.078284,+ 0.079503,+ 0.084893,+ 0.081261,+ 0.077014,+ 0.109799,+ 0.125756,+ 0.059974,+ 0.090709,+ 0.081217,+ 0.096,+ 0.069726,+ 0.089971,+ 0.112135,+ 0.09602,+ 0.091759,+ 0.11809,+ 0.078429,+ 0.047035,+ 0.093753,+ 0.095624,+ 0.098029,+ 0.106432,+ 0.133872,+ 0.103,+ 0.07604,+ 0.05194,+ 0.090932,+ 0.093213,+ 0.079678,+ 0.078172,+ 0.120268,+ 0.08386,+ 0.046606,+ 0.07727,+ 0.093338,+ 0.145726,+ 0.093336,+ 0.12579,+ 0.130649,+ 0.08641,+ 0.103099,+ 0.101546,+ 0.106692,+ 0.124036,+ 0.136143,+ 0.119406,+ 0.114933,+ 0.069371,+ 0.10421,+ 0.109721,+ 0.08112,+ 0.089629,+ 0.039085,+ 0.109513,+ 0.100191,+ 0.072614,+ 0.080919,+ 0.097477,+ 0.092409,+ 0.112519,+ 0.108892,+ 0.113523,+ 0.099712,+ 0.068738,+ 0.07573,+ 0.109937,+ 0.097434,+ 0.112601,+ 0.111462,+ 0.095996,+ 0.099334,+ 0.120899,+ 0.103942,+ 0.062342,+ 0.111624,+ 0.075541,+ 0.083055,+ 0.116143,+ 0.081513,+ 0.11583,+ 0.117452,+ 0.104083,+ 0.098491,+ 0.11433,+ 0.125843,+ 0.095322,+ 0.063691,+ 0.101016,+ 0.075103,+ 0.089269,+ 0.097348,+ 0.087017,+ 0.110054,+ 0.080901,+ 0.08661,+ 0.088302,+ 0.112859,+ 0.138884,+ 0.059943,+ 0.101197,+ 0.126002,+ 0.061415,+ 0.052987,+ 0.074794,+ 0.094437,+ 0.133651,+ 0.054561,+ 0.100284,+ 0.076724,+ 0.061472,+ 0.083505,+ 0.096821,+ 0.161064,+ 0.102735,+ 0.080861,+ 0.092108,+ 0.080617,+ 0.059182,+ 0.103691,+ 0.078574,+ 0.068402,+ 0.117709,+ 0.106664,+ 0.094523,+ 0.118086,+ 0.09051,+ 0.133742,+ 0.0606,+ 0.100495,+ 0.05367,+ 0.081163,+ 0.080935,+ 0.100509,+ 0.093665,+ 0.08916,+ 0.12794,+ 0.081498,+ 0.070528,+ 0.093986,+ 0.084184,+ 0.074163,+ 0.111211,+ 0.073201,+ 0.100549,+ 0.102875,+ 0.123028,+ 0.072745,+ 0.095665,+ 0.134611,+ 0.097713,+ 0.114911,+ 0.06497,+ 0.077184,+ 0.082284,+ 0.052802,+ 0.076175,+ 0.044908,+ 0.089936,+ 0.105357,+ 0.148603,+ 0.052483,+ 0.102814,+ 0.072539,+ 0.07478,+ 0.093021,+ 0.048671,+ 0.109254,+ 0.101317,+ 0.108484,+ 0.071931,+ 0.100777,+ 0.080765,+ 0.095699,+ 0.07716,+ 0.107708,+ 0.071968,+ 0.082016,+ 0.073945,+ 0.084418,+ 0.111959,+ 0.117458,+ 0.089963,+ 0.119222,+ 0.096036,+ 0.124233,+ 0.10056,+ 0.154893,+ 0.059502,+ 0.103339,+ 0.080142,+ 0.106786,+ 0.121357,+ 0.092106,+ 0.062945,+ 0.06535,+ 0.106856,+ 0.090606,+ 0.103215,+ 0.120348,+ 0.082009,+ 0.108652,+ 0.056609,+ 0.07275,+ 0.077691,+ 0.098517,+ 0.074827,+ 0.135653,+ 0.142386,+ 0.092003,+ 0.078187,+ 0.129751,+ 0.118577,+ 0.090994,+ 0.110434,+ 0.096612,+ 0.117997,+ 0.122195,+ 0.137814,+ 0.117806,+ 0.108414,+ 0.114627,+ 0.095554,+ 0.089797,+ 0.067142,+ 0.090288,+ 0.105822,+ 0.021106,+ 0.137842,+ 0.074083,+ 0.07223,+ 0.067159,+ 0.137939,+ 0.099568,+ 0.06686,+ 0.111102,+ 0.10855,+ 0.114443,+ 0.079504,+ 0.084126,+ 0.088244,+ 0.100214,+ 0.046776,+ 0.079059,+ 0.088758,+ 0.134618,+ 0.071311,+ 0.096868,+ 0.064537,+ 0.105556,+ 0.110626,+ 0.062634,+ 0.055046,+ 0.096994,+ 0.102627,+ 0.096635,+ 0.048846,+ 0.083157,+ 0.128077,+ 0.056314,+ 0.054102,+ 0.083316,+ 0.112634,+ 0.082715,+ 0.083871,+ 0.101438,+ 0.071286,+ 0.109593,+ 0.103188,+ 0.098695,+ 0.063536,+ 0.144343,+ 0.111042,+ 0.126029,+ 0.135449,+ 0.084455,+ 0.07559,+ 0.079103,+ 0.092589,+ 0.071416,+ 0.115691,+ 0.104155,+ 0.08772,+ 0.092864,+ 0.139092,+ 0.103329,+ 0.128007,+ 0.114191,+ 0.095072,+ 0.103772,+ 0.067931,+ 0.074601,+ 0.139078,+ 0.086658,+ 0.068988,+ 0.173502,+ 0.151934,+ 0.063897,+ 0.117727,+ 0.084164,+ 0.121849,+ 0.08997,+ 0.089012,+ 0.115055,+ 0.109736,+ 0.105555,+ 0.096203,+ 0.131208,+ 0.082019,+ 0.099824,+ 0.122403,+ 0.105143,+ 0.112152,+ 0.093097,+ 0.1282,+ 0.08748,+ 0.08969,+ 0.097879,+ 0.107827,+ 0.072189,+ 0.096072,+ 0.109974,+ 0.089053,+ 0.076054,+ 0.100995,+ 0.108274,+ 0.103558,+ 0.065825,+ 0.092602,+ 0.072802,+ 0.050506,+ 0.102944,+ 0.063859,+ 0.099863,+ 0.091281,+ 0.062084,+ 0.10353,+ 0.096683,+ 0.139849,+ 0.081561,+ 0.104618,+ 0.085879,+ 0.120288,+ 0.090438,+ 0.105954,+ 0.102162,+ 0.069838,+ 0.105076,+ 0.078246,+ 0.126751,+ 0.098166,+ 0.090261,+ 0.133086,+ 0.056007,+ 0.108987,+ 0.114942,+ 0.087719,+ 0.085794,+ 0.065325,+ 0.094817,+ 0.106168,+ 0.104212,+ 0.103687,+ 0.114557,+ 0.103102,+ 0.076533,+ 0.096605,+ 0.112124,+ 0.096341,+ 0.087511,+ 0.105264,+ 0.095985,+ 0.053186,+ 0.098866,+ 0.130073,+ 0.114807,+ 0.110924,+ 0.056492,+ 0.069483,+ 0.13471,+ 0.058112,+ 0.085304,+ 0.065335,+ 0.122806,+ 0.141465,+ 0.10711,+ 0.122459,+ 0.089813,+ 0.119309,+ 0.084103+ ]+ ],+ "coef": [+ 0.05924,+ -0.00699,+ -0.034455,+ 0.010189,+ 0.015235,+ 0.009063,+ -0.010381,+ -0.020696,+ -0.011051,+ 0.024126,+ -0.016943,+ 0.005169,+ 0.021843,+ 0.033003,+ 0.022349,+ 0.027824,+ -0.026869,+ -0.010709,+ -0.020683,+ 0.015361,+ 0.018543,+ -0.010009,+ -0.041406,+ -0.031161,+ -0.014887,+ 0.011872,+ -0.013186,+ 0.036435,+ -0.00926,+ 0.001697,+ -0.017356,+ -0.021725,+ 0.00767,+ 0.053511,+ -0.010702,+ -0.03082,+ -0.017088,+ 0.027766,+ -0.004229,+ -0.015708,+ 0.022437,+ 0.001998,+ 0.02396,+ 0.028479,+ 0.022588,+ -0.010984,+ -0.036845,+ -0.02693,+ 0.040769,+ -0.031716,+ -0.044573,+ 0.031104,+ -0.000112,+ -0.016139,+ 0.023215,+ 0.012559,+ 0.009451,+ 0.050127,+ -0.010196,+ -0.036672,+ 0.008955,+ -6e-06,+ -0.007574,+ 0.041252,+ -0.026359,+ 0.019338,+ 0.024673,+ 0.007631,+ -0.059603,+ -0.005256,+ 0.022911,+ -0.026228,+ 0.04361,+ 0.001042,+ -0.022519,+ -0.001413,+ 0.018295,+ 0.036154,+ -0.005427,+ -0.007098,+ -0.011775,+ 0.037229,+ 0.02344,+ 0.017562,+ -0.002499,+ -0.012895,+ -0.025663,+ -0.02301,+ -0.053134,+ 9.5e-05,+ 0.023825,+ 0.029256,+ 0.004078,+ 0.008986,+ -0.00769,+ -0.028023,+ 0.000283,+ 0.02428,+ -0.001951,+ -0.023687,+ -0.005593,+ -0.007224,+ -0.008693,+ -0.034957,+ 0.005234,+ 0.026756,+ 0.060947,+ 0.011303,+ -0.01682,+ -0.019694,+ 0.028171,+ 0.002053,+ -0.031216,+ 0.013995,+ -0.022303,+ -0.035202,+ -0.007784,+ 0.037799,+ 0.028181,+ -0.018375,+ 0.005488,+ -0.007101,+ 0.030691,+ -0.003298,+ 0.0206,+ -0.044727,+ 0.009851,+ -0.052274,+ -0.009091,+ 0.067048,+ 0.015639,+ -0.01757,+ 0.031162,+ 0.009329,+ 0.018869,+ -0.018739,+ 0.025078,+ 0.037833,+ 0.022058,+ 0.004288,+ 0.01336,+ -0.007082,+ -0.002535,+ -0.063531,+ 0.004176,+ -0.014881,+ -0.002708,+ 0.003254,+ -0.003854,+ 0.013757,+ 0.002488,+ -0.022428,+ -0.000554,+ -0.0019,+ 0.045924,+ 0.016259,+ -0.029482,+ -0.00642,+ -0.032835,+ -0.013424,+ -0.027276,+ -0.000559,+ -0.023628,+ 0.012883,+ 0.016813,+ 0.06005,+ -0.034864,+ 0.037062,+ 0.040739,+ -0.015157,+ -0.004034,+ 0.003871,+ 0.017532,+ 0.011031,+ -0.000206,+ -0.005617,+ -0.003741,+ 0.00208,+ -0.002241,+ 0.016971,+ 0.033999,+ 0.049909,+ -0.013196,+ -0.024671,+ -0.016538,+ 0.021074,+ 0.019778,+ 0.020509,+ 0.001333,+ 0.039808,+ -0.01477,+ -0.007635,+ -0.048126,+ 0.015084,+ -0.008951,+ -0.046556,+ -0.038684,+ -0.004614,+ -0.070564,+ 0.035197,+ -0.036123,+ 0.010838,+ -0.011326,+ -0.02202,+ -0.018276,+ 0.035811,+ 0.007861,+ -0.017308,+ -0.001593,+ -0.006772,+ -0.059866,+ -0.001945,+ -0.031247,+ 0.01115,+ 0.006367,+ 0.001057,+ -0.009032,+ 0.024076,+ -0.017677,+ 0.011211,+ -0.006223,+ -0.027263,+ 0.01821,+ 0.003422,+ 0.016652,+ -0.002246,+ 0.019419,+ -0.011716,+ -0.003477,+ 0.044237,+ -0.031376,+ 0.013054,+ 0.018015,+ -0.017493,+ -0.038355,+ -0.006102,+ -0.020786,+ 0.049602,+ 0.055344,+ 0.018271,+ 0.006311,+ -0.016881,+ -0.015835,+ 0.022737,+ -0.040864,+ -0.027434,+ 0.010209,+ 0.010558,+ 0.022241,+ 0.00243,+ 0.037626,+ 0.001195,+ 0.056002,+ -0.033321,+ 0.005616,+ -0.028252,+ -0.019787,+ -0.006723,+ -0.032572,+ -0.049894,+ -0.002147,+ 0.024247,+ -0.011935,+ 0.051404,+ 0.00641,+ 0.002558,+ 0.015276,+ 0.001992,+ 0.019293,+ 0.009177,+ 0.006856,+ -0.00739,+ -0.027067,+ -0.015647,+ 0.007719,+ -0.019544,+ 0.008249,+ 0.013082,+ -0.006722,+ -0.00285,+ 0.016991,+ -0.033859,+ -0.020252,+ 0.001593,+ -0.045824,+ 0.013449,+ 0.029819,+ 0.039589,+ -0.045504,+ -0.027453,+ -0.034208,+ -0.034143,+ -0.015265,+ -0.047337,+ 0.019082,+ -0.01645,+ -0.009176,+ -0.020236,+ 0.007476,+ 0.018381,+ -0.043247,+ 0.028845,+ 0.036598,+ -0.024589,+ 0.007133,+ -0.06043,+ 0.046638,+ -0.003205,+ -0.001137,+ -0.037769,+ 0.014055,+ -0.00512,+ -0.039879,+ 0.009445,+ 0.001856,+ -0.027325,+ 0.011825,+ 0.022229,+ 0.034174,+ 0.009826,+ 0.007098,+ -0.003228,+ 0.039834,+ -0.00625,+ 0.059165,+ 0.043994,+ 0.030412,+ -0.001859,+ 0.029628,+ 0.000178,+ -0.006955,+ 0.003871,+ 0.035204,+ -0.024515,+ 0.029763,+ -0.072262,+ -0.061947,+ -0.01961,+ -0.011117,+ -0.034192,+ 0.019356,+ -0.028628,+ 0.000135,+ -0.023375,+ 0.007633,+ -0.031802,+ -0.013816,+ -0.007903,+ -0.000812,+ 0.055063,+ 0.016734,+ -0.017371,+ -0.005936,+ 0.012725,+ -0.034471,+ 0.017609,+ 0.040121,+ 0.012427,+ 0.015454,+ -0.025469,+ 0.006379,+ 0.015673,+ 0.029066,+ -0.004657,+ 0.028907,+ -0.009821,+ 0.029359,+ 0.04493,+ -0.011109,+ 0.003567,+ -0.006612,+ -0.037995,+ -0.014886,+ 0.007557,+ 0.043739,+ -0.009283,+ 0.018757,+ 0.014872,+ -0.025522,+ 0.013312,+ -0.025225,+ 0.025535,+ 0.052581,+ 0.000286,+ 0.034085,+ 0.02561,+ -0.034815,+ -0.018954,+ 0.007773,+ -0.060559,+ 0.011573,+ -0.04462,+ 0.008991,+ -0.021888,+ -0.012652,+ 0.001385,+ 0.012112,+ 0.010426,+ 0.00891,+ 0.044867,+ -0.022068,+ -0.004518,+ 0.024281,+ -0.007653,+ -0.003857,+ 0.053683,+ -0.007193,+ -0.051435,+ -0.021273,+ -0.012483,+ 0.015293,+ -0.033573,+ 0.023027,+ 0.037866,+ 0.002631,+ 0.009463,+ 0.001998,+ -0.025985,+ 0.005456,+ 0.008689,+ 0.018609,+ -0.011313,+ 0.023685,+ -0.004156,+ 0.011172,+ -0.009468,+ -0.003314,+ 0.019422,+ -0.007117,+ 0.000544,+ -0.011943,+ -0.007962,+ 0.005544,+ 0.014037,+ 0.025574,+ -0.000414,+ 0.000643,+ -0.038295,+ -0.032301,+ -0.026311,+ -0.028277,+ 0.023698,+ 0.004483,+ -0.002129,+ 0.016133,+ -0.007148,+ -0.026121,+ 0.013065,+ 0.00353,+ 0.000833,+ 0.030757,+ 0.027981,+ 0.026868,+ 0.003657,+ 0.024915,+ -0.025465,+ -0.058383,+ 0.023877,+ -0.042678,+ -0.013076,+ 0.024197,+ -0.002932,+ -0.007783,+ 0.014123,+ 0.033223,+ -0.012829,+ 0.019866,+ -0.024978,+ 0.032858,+ 0.001142,+ -0.024346,+ 0.012786,+ 0.025857,+ -0.027984,+ 0.009047,+ -0.012072,+ 0.003251,+ -0.004397,+ -0.0056,+ 0.037969,+ 0.035465,+ -0.033393,+ -0.004819,+ 0.003293,+ 0.022821,+ -0.010973,+ 0.014243,+ -0.025472,+ -0.01028,+ 0.000436,+ 0.021206,+ -0.030076,+ -0.011386,+ 0.02245,+ 0.045114,+ 0.018883,+ 0.001034,+ 0.011526,+ 0.00562,+ 0.00954,+ -0.007781,+ -0.013924,+ 0.000598,+ -0.011761,+ 0.019526,+ 0.001724,+ -0.019377,+ -0.009443,+ -0.029839,+ 0.05367,+ 0.041604,+ -0.021492+ ],+ "intercept": -0.041353,+ "inputs": "clap_embedding_512"+ },+ "signal": {+ "mean": [+ 0.046988,+ -35.819096,+ -77.296944,+ -2.946889,+ 52.209124,+ 0.591578,+ 11.829969,+ -3.073923,+ 0.56807,+ 1.589956,+ -0.232734,+ 1.312882,+ 16.506316+ ],+ "scale": [+ 0.150139,+ 36.75158,+ 37.418658,+ 2.342378,+ 29.434968,+ 0.324279,+ 4.76627,+ 2.78057,+ 0.377947,+ 1.197701,+ 0.660996,+ 0.498763,+ 6.792149+ ],+ "clip": [+ [+ 0.0,+ -140.0,+ -166.867731,+ -11.340768,+ 7.562637,+ 0.033342,+ 0.577656,+ -11.812673,+ 0.0,+ -1.99963,+ -2.0,+ 0.459907,+ 3.016251+ ],+ [+ 0.905215,+ -0.06664,+ -21.402317,+ -0.081025,+ 150.150704,+ 0.999964,+ 15.0,+ -0.018219,+ 1.0,+ 3.510156,+ 1.193301,+ 3.18157,+ 37.219513+ ]+ ],+ "coef": [+ -0.106636,+ 0.317801,+ 0.0,+ 0.0,+ 0.0,+ 0.0,+ 0.0,+ 0.898494,+ 0.0,+ 0.199091,+ 0.007039,+ 0.0,+ 0.755868+ ],+ "intercept": -0.052934,+ "inputs": [+ "silence_frac",+ "floor_db",+ "spec_floor_db",+ "log_flatness",+ "peakiness_db",+ "line_share",+ "bandwidth_khz",+ "quiet_flat",+ "tonal_frames",+ "pitch_wobble",+ "env_resid_db",+ "texture_kurt",+ "crest_db"+ ]+ }+ },+ "fusion": {+ "heads": [+ "clap",+ "signal"+ ],+ "coef": [+ 1.212409,+ 0.0+ ],+ "intercept": -0.153358+ },+ "C": {+ "clap": 0.001,+ "signal": 0.1+ },+ "thresholds": {+ "hi": 0.77,+ "lo": 0.17+ },+ "file_thresholds": {+ "hi": 0.77,+ "lo": 0.4,+ "files": 119,+ "synth_files": 23,+ "how": "lo: 92.5% precision on held-out long files (scheme M); hi: max(that, the clip hi)"+ },+ "prompts": {+ "real": [+ "a real recording made with a microphone",+ "a field recording of a real place",+ "a real foley recording of an object",+ "a professionally produced music track",+ "a professional cinematic sound effect from a sound library",+ "a natural acoustic sound"+ ],+ "synth": [+ "a cheap synthesized beep",+ "a pure sine wave test tone",+ "a simple electronic synthesizer tone made by code",+ "an 8-bit video game sound effect",+ "filtered white noise",+ "a basic drum machine loop"+ ]+ },+ "facts": [+ {+ "feature": "spec_floor_db",+ "side": "synth",+ "op": "<=",+ "value": -128.123,+ "precision": 0.901,+ "support": 223+ },+ {+ "feature": "log_flatness",+ "side": "synth",+ "op": "<=",+ "value": -6.6345,+ "precision": 0.919,+ "support": 149+ },+ {+ "feature": "peakiness_db",+ "side": "synth",+ "op": ">=",+ "value": 93.2532,+ "precision": 0.904,+ "support": 178+ },+ {+ "feature": "quiet_flat",+ "side": "synth",+ "op": "<=",+ "value": -4.9503,+ "precision": 0.901,+ "support": 282+ },+ {+ "feature": "pitch_wobble",+ "side": "synth",+ "op": "<=",+ "value": -0.765,+ "precision": 0.913,+ "support": 103+ },+ {+ "feature": "crest_db",+ "side": "synth",+ "op": "<=",+ "value": 7.6265,+ "precision": 0.91,+ "support": 89+ },+ {+ "feature": "crest_db",+ "side": "real",+ "op": ">=",+ "value": 29.1703,+ "precision": 0.904,+ "support": 104+ }+ ],+ "silence_base": [+ {+ "at_least": 0.005,+ "real": 0.194,+ "synth": 0.289+ },+ {+ "at_least": 0.02,+ "real": 0.117,+ "synth": 0.201+ },+ {+ "at_least": 0.05,+ "real": 0.094,+ "synth": 0.148+ },+ {+ "at_least": 0.1,+ "real": 0.075,+ "synth": 0.115+ },+ {+ "at_least": 0.2,+ "real": 0.054,+ "synth": 0.095+ },+ {+ "at_least": 0.3,+ "real": 0.033,+ "synth": 0.071+ },+ {+ "at_least": 0.5,+ "real": 0.013,+ "synth": 0.037+ }+ ],+ "clap_direction": {+ "toward_real": [+ "a high quality studio recording",+ "a metal scrape",+ "an old typewriter",+ "a match being struck",+ "a knife chopping on a board",+ "a small object dropping on a table",+ "pizzicato strings",+ "a can opening"+ ],+ "toward_synth": [+ "a square wave chiptune blip",+ "a simple electronic synthesizer tone made by code",+ "a warm analog synth pad",+ "a cheap synthesized beep",+ "a computer-generated electronic tone",+ "filtered white noise",+ "an 8-bit video game sound effect",+ "an 8-bit sound effect"+ ],+ "how": "cosine between each vocabulary text embedding and the CLAP head weight direction (coef / scale)"+ },+ "eval": {+ "scheme_A": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9219,+ "[email protected]": 0.8598,+ "brier": 0.1088,+ "logloss": 0.36+ },+ "scheme_M": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.92,+ "[email protected]": 0.8801,+ "brier": 0.0984,+ "logloss": 0.3549+ },+ "band": {+ "coverage": 0.7904,+ "precision_real": 0.9241,+ "precision_synth": 0.9224,+ "real_called_synth": 0.0666,+ "synth_called_real": 0.0555,+ "uncertain_frac": 0.2096+ },+ "retired_check": {+ "C: real vs retired": {+ "auc": 0.9475,+ "[email protected]": 0.862,+ "synth_mean_p": 0.166+ },+ "C: real recordings only (no music/films) vs retired": {+ "auc": 0.9418+ },+ "D: real vs retired": {+ "auc": 0.9085,+ "[email protected]": 0.8389,+ "synth_mean_p": 0.185+ },+ "D: real vs cinetic": {+ "auc": 0.921,+ "[email protected]": 0.8484,+ "synth_mean_p": 0.145+ },+ "D: real recordings only (no music/films) vs retired": {+ "auc": 0.9031+ }+ },+ "layers": "code-synthesized layers at -6..+6 dB inside real beds: 0.30 of windows lean synthesized (p_real < 0.5), 0.14 at or below lo; the beds alone: 0.114",+ "file_level_M": {+ "n": 119,+ "auc": 0.9538,+ "[email protected]": 0.9244,+ "band": {+ "coverage": 0.8655,+ "precision_real": 0.9884,+ "precision_synth": 1.0,+ "real_called_synth": 0.0,+ "synth_called_real": 0.0435,+ "uncertain_frac": 0.1345+ }+ },+ "see": "eval/realness/RESULTS.md"+ }+}\ No newline at end of filediff --git a/earshot/data/vocab.json b/earshot/data/vocab.jsonnew file mode 100644index 0000000..61a8560--- /dev/null+++ b/earshot/data/vocab.json@@ -0,0 +1,114 @@+{+ "_doc": "earshot's listening vocabulary: short caption-like phrases CLAP scores sounds against. 'templates' are prompt ensembles (each term is embedded through every template and averaged). Every term in every category also forms the background distribution used to calibrate scores (z = how far a match stands out from what an arbitrary description would score). History (for honest validation): 217 terms were written before any evaluation (eval/semantics/vocab_preregistered.json); 67 were added after seeing the 219-file tuning library's classes (tuned to it); 38 general everyday terms (vehicles, household, tools, animals, people) were added from general knowledge; the list was then frozen before the held-out library (eval/semantics/heldout_labels.json) was scored. Extending it changes every z a little: re-run eval/semantics/run_eval.py analyze + heldout afterwards. data/vocab_stats.json holds the reference mean embedding (independent of the vocabulary) and a derived table of each term's mean / std cosine over the reference recordings.",+ "templates": ["{}", "the sound of {}", "a recording of {}", "{} sound effect", "an audio clip of {}"],+ "categories": {+ "transitions": [+ "a fast whoosh", "a soft air whoosh", "a deep cinematic whoosh", "a short swish", "a swoosh passing by",+ "a stick swung fast through the air", "cloth whipping through the air", "a gust of wind rushing past",+ "a rising riser building tension", "an uplifter sweep", "a reverse cymbal swell", "a downlifter falling sweep",+ "a tape stop slowing down", "a record scratch", "a quick swipe", "a whip pan transition",+ "a stutter glitch transition"+ ],+ "impacts": [+ "a deep cinematic boom", "a sub bass drop", "a punchy impact hit", "a big braam", "a heavy thud", "a soft thump",+ "a cinematic hit with a long tail", "a big drum hit", "a metallic clang", "a door slam", "a snare hit", "a kick drum",+ "a hand clap", "a gunshot", "an explosion", "a low rumble", "a body falling on the ground", "a wooden knock"+ ],+ "ui": [+ "a mouse click", "a soft UI button tap", "a crisp interface click", "a notification ding", "a chat message pop",+ "a phone vibration buzz", "a camera shutter click", "a toggle switch click", "a light switch flicked",+ "a scroll wheel tick", "a success chime", "an error buzz", "a digital glitch", "a typing cursor blip",+ "a touchscreen tap", "a button being pressed"+ ],+ "keyboard": [+ "a keyboard typing fast", "a single mechanical keyboard key press", "a deep thocky mechanical keyboard",+ "a clicky mechanical keyboard", "a laptop keyboard typing", "a spacebar press", "an enter key press",+ "a soft key release click", "slow typing with separate key presses", "an old typewriter"+ ],+ "foley": [+ "a plastic click", "a plastic toy snapping together", "lego bricks clicking", "a plastic case snapping shut",+ "a latch clicking shut", "a box lid closing", "a ratchet clicking", "a fidget toy clicking", "a pen click",+ "a cork pop", "a mouth pop", "a bubble pop", "a balloon squeak", "a rubber stretch", "a suction cup pop",+ "a finger squeaking on glass", "a finger tapping on wood", "a finger tapping on glass", "glass clinking",+ "coins dropping", "paper rustling", "a page turning", "a zipper", "cloth rustling", "footsteps on wood",+ "footsteps on gravel", "high heels walking", "a small object dropping on a table",+ "a plastic object falling on a table", "dominoes falling", "a wooden block hit", "a wood creak", "a creaking door",+ "a metal scrape", "a drawer closing", "scissors cutting", "a knife chopping on a board", "a can opening",+ "a squishy wet squelch"+ ],+ "water": [+ "water splashing", "a water drop", "a liquid bloop", "pouring water", "ice cubes in a glass",+ "an object plunging into water", "bubbles underwater", "being underwater", "a stream flowing", "a waterfall",+ "a dripping tap"+ ],+ "nature": [+ "footsteps on grass", "walking through dry leaves", "leaves rustling", "a tree shaking", "a branch snapping",+ "an apple falling on grass", "something falling into bushes", "digging in soil",+ "a small animal running through grass", "insects buzzing", "a bee flying by", "a bird flapping its wings",+ "a horse galloping"+ ],+ "chimes": [+ "a small bell", "a glass ting", "a wine glass ring", "a triangle hit", "wind chimes", "a glissando on bar chimes",+ "a celesta note", "a kalimba note", "a glockenspiel run", "a singing bowl", "a sustained glass harp tone",+ "a music box melody", "a doorbell", "church bells", "a magic sparkle chime"+ ],+ "ambience": [+ "rain on a window", "heavy rain", "rain on a roof", "a thunderstorm", "wind blowing", "a gentle breeze in tall grass",+ "wind in pine trees", "birds singing in the morning", "skylarks over a meadow", "a forest ambience",+ "a quiet countryside morning", "a summer meadow with insects", "a city street with traffic", "a busy cafe",+ "an office ambience", "a quiet room tone", "a quiet room at night", "a computer fan hum", "an electrical hum",+ "ocean waves", "a crackling fire", "crickets at night", "an airplane flying overhead", "a crowd cheering",+ "a dog barking", "people talking in the background"+ ],+ "everyday": [+ "a car horn honking", "a car engine starting", "a car driving past", "a motorcycle revving", "a train passing",+ "a helicopter flying overhead", "a police siren", "a bicycle bell", "a knock on a door", "a phone ringing",+ "an alarm clock beeping", "a microwave beep", "a kettle whistling", "a vacuum cleaner", "a toilet flushing",+ "a clock ticking", "a cash register", "a printer printing", "keys jingling", "a match being struck",+ "a spray can hissing", "a hammer hitting a nail", "a power drill", "a saw cutting wood", "a sword being drawn",+ "a basketball bouncing", "fireworks exploding", "a thunder clap", "a heartbeat", "applause", "a cat meowing",+ "a bird chirping", "a rooster crowing", "a cow mooing", "a horse neighing"+ ],+ "music_genre": [+ "minimal electronic music", "future bass music", "lo-fi hip hop beat", "cinematic orchestral music",+ "uplifting orchestral film score", "epic trailer music", "ambient synth pad music", "upbeat corporate pop music",+ "inspirational corporate background music", "synthwave music", "house music", "tech house music",+ "drum and bass music", "trap beat", "glitch hop music", "indie pop music", "solo piano music",+ "playful children's piano music", "cheerful ukulele music", "acoustic guitar music", "jazz music",+ "rock music with electric guitars", "hip hop beat with heavy bass", "chiptune video game music", "dubstep music",+ "classical string quartet", "funk music", "techno music"+ ],+ "music_mood": [+ "uplifting and hopeful music", "playful and quirky music", "dark and tense music", "sad and melancholic music",+ "calm and relaxing music", "energetic and driving music", "dreamy and ethereal music", "triumphant and heroic music",+ "mysterious music", "romantic music", "aggressive music", "warm and cozy music", "futuristic music",+ "confident and cool music", "suspenseful build-up music", "celebratory and joyful music"+ ],+ "instruments": [+ "a deep sub bass", "a punchy kick drum pattern", "crisp hi-hats", "a snare roll", "a synthesizer lead",+ "a warm analog synth pad", "a plucked synth arpeggio", "a grand piano", "an electric piano", "strings ensemble",+ "pizzicato strings", "brass section", "a celesta", "a glockenspiel", "bells", "a marimba", "an acoustic guitar",+ "an electric guitar", "a bass guitar", "a ukulele", "a choir", "a vocal chop", "orchestral percussion", "a music box",+ "a cymbal roll"+ ],+ "voice": [+ "a man speaking", "a woman speaking", "a voice-over narration", "someone whispering", "singing vocals",+ "a crowd talking", "children laughing", "a person breathing", "a person coughing", "people laughing", "a baby crying",+ "a man shouting"+ ],+ "quality": [+ "a deep sound", "a bright sound", "a thin sound", "a warm sound", "a harsh sound", "a muffled sound", "a crisp sound",+ "a punchy sound", "a soft sound", "a loud sound", "a quiet sound", "a short sound", "a long sound",+ "a dry close sound", "a reverberant sound in a large hall", "a distant sound", "a tight snappy sound", "a boomy sound",+ "an airy sound", "a metallic sound", "a hollow sound", "a satisfying sound"+ ],+ "production": [+ "a real recording of a physical object", "a close-miked foley recording", "a field recording outdoors",+ "a professionally mixed and mastered music track", "a high quality studio recording", "a cheap synthesized beep",+ "a pure sine wave tone", "a square wave chiptune blip", "a computer-generated electronic tone", "white noise",+ "pink noise", "a retro video game sound effect", "an 8-bit sound effect", "a robotic synthetic sound",+ "a low quality recording with hiss", "a distorted clipped recording", "a low bitrate mp3 with artifacts",+ "a recording made on a phone", "silence"+ ]+ }+}diff --git a/earshot/data/vocab_stats.json b/earshot/data/vocab_stats.jsonnew file mode 100644index 0000000..de08681--- /dev/null+++ b/earshot/data/vocab_stats.json@@ -0,0 +1,336 @@+{+ "max_win": "tuning files 6, others 3",+ "reference_set": "the 219 tuning recordings of the Clawd project + 7 professional launch films + Clawd soundtrack/stems + retired code-synthesized stems (eval/semantics/run_eval.py ship_refs); disjoint from the held-out library",+ "model": "laion/larger_clap_general",+ "precision": "bf16",+ "pad": "repeatpad",+ "templates": ["{}", "the sound of {}", "a recording of {}", "{} sound effect", "an audio clip of {}"],+ "logit_scale": 38.75,+ "n_ref": 241,+ "n_folders": 13,+ "mean": [-0.04439, 0.0169, -0.01226, 0.00216, 0.01, 0.04588, -0.03536, 0.01163, 0.0079, -0.01316, -0.01773, 0.0088, -0.03454, 0.03178, -0.00433, 0.01824, -0.00746, 0.00536, 0.01336, -0.00446, 0.01802, -0.00451, 0.00884, 0.01414, -0.02244, 0.00112, -0.01648, 0.02348, -0.00941, -0.02099, 0.04904, 0.0134, -0.00836, 0.01828, -0.00175, -0.00218, 0.00594, 0.00689, -0.00055, -0.04398, 0.00311, 0.00011, -0.01998, 0.0077, -0.01628, -0.00388, -0.00897, 0.01948, -0.00891, 0.03264, -0.05593, -0.02529, -0.0277, 0.01669, 0.01029, 0.00558, 0.02096, 0.02243, -0.03078, -0.03826, 0.02416, -0.01714, -0.03, -0.02399, 0.00732, 0.00419, -0.00468, 0.00867, 0.01126, -0.01408, 0.03181, 0.00121, 0.01665, 0.03412, 0.00223, 0.00435, 0.01022, 0.01049, -0.02426, -0.01182, -0.03835, 0.01789, 0.00096, -0.00482, 0.0018, -0.02217, -0.02959, 0.01915, 0.03103, 0.00248, -0.03474, -0.00514, -0.04087, -0.04522, 0.02357, 0.02043, -0.0024, -0.04176, 0.00949, -0.00302, -0.00959, -0.01351, 0.00745, -0.00058, 0.01151, -0.01235, -0.02495, 0.02623, 0.01021, -0.01118, -0.06419, -0.01321, -0.00421, 0.00122, 0.0301, -0.00461, -0.01634, -0.0273, 0.01056, -0.03997, 0.01483, 0.00221, -0.012, -0.02885, -0.01406, 0.00633, 0.01673, -0.00292, 0.02534, -0.03161, -0.02825, 0.01518, 0.00701, -0.01807, 0.02357, 0.00938, 0.01745, 0.0013, 0.00424, -0.027, -0.03514, -0.01568, 0.0134, 0.02795, 0.0055, 0.01186, 0.00691, -0.00788, -0.02819, 0.00702, -0.01227, 0.01158, -0.01094, 0.02822, -0.02201, -0.06372, -0.02814, -0.01162, 0.0596, 0.01128, 0.00598, -0.0008, -0.05768, -0.00495, 0.01498, -0.01482, 0.01164, 0.03885, 0.01351, 0.03613, -0.03559, -0.00791, 0.04268, -0.0054, -0.01577, -0.04854, 0.03815, -0.01684, -0.05427, 0.00768, 0.03516, -0.02643, 0.0096, 0.01343, 0.00939, -0.00549, -0.0228, -0.00709, -0.02582, -0.02923, -0.02889, 0.00519, -0.00257, 0.00801, 0.04024, 0.01144, -0.04957, 0.01843, -0.0231, -0.01413, 0.02386, -0.01536, 0.01121, 0.05282, 0.01775, -0.00327, 0.03345, 0.02468, 0.00462, -0.01736, 0.02052, -0.02754, -0.02544, 0.0005, 0.00409, 0.00302, 0.00983, 0.00427, -0.01003, 0.01538, 0.06139, -0.02466, -0.01444, 0.02602, -0.0224, -0.03263, -0.0019, 0.00692, 0.01812, -0.02256, -0.01252, -0.00159, -0.03673, -0.00776, -0.00499, 0.02816, -0.00385, -0.02855, 0.00637, 0.00921, -0.01227, -0.0106, -0.00806, -0.01758, 0.02461, -0.02916, -0.00539, 0.0062, 0.00791, 0.01348, -0.00718, 0.02718, -0.01782, -0.03336, -0.00235, 0.01651, 0.0079, 0.00022, 0.01107, 0.00821, 0.0009, -0.02677, -0.01111, 0.00773, 0.02634, -0.01605, 0.02201, 0.01722, 0.0165, -0.0106, -0.01345, 0.03958, -0.01502, 0.00046, -0.01204, 0.02199, -0.02863, -0.04619, -0.01905, -0.07297, -0.00684, 0.00083, 0.02664, -0.01634, -0.00367, -0.0193, -0.0134, -0.00082, -0.04689, -0.00302, 0.00058, 0.00289, -0.02715, -0.02295, -0.00868, -0.00625, 0.01361, 0.02362, 0.01493, 0.01426, -0.00845, 0.01023, 0.01557, -0.00156, -0.0123, 0.0156, 0.015, 0.01965, -0.00165, -0.00195, 0.00027, 0.03488, -0.0037, 0.00496, 0.01256, -0.0097, 0.0042, -0.0567, 0.00553, -0.00255, 0.01169, 0.019, -0.00817, 0.01589, -0.04003, -0.04164, -0.01443, 0.01822, 0.00071, 0.04494, -0.00234, -0.02439, -0.00115, 0.0419, 0.03303, 0.00287, 0.0066, 0.00669, 0.02157, 0.05161, 0.03715, 0.01549, 0.02643, -0.0305, -0.00841, -0.00459, -0.03466, -0.00862, -0.00334, -0.03173, 0.03868, -0.01025, -0.0103, -0.02418, 0.03445, 0.0032, 0.00291, 0.03319, -0.00047, -0.01122, -0.00044, -0.02792, -0.02111, 0.01741, -0.04329, -0.03536, -0.02473, 0.02594, -0.04518, 0.01997, 0.00266, 0.01985, 0.00324, -0.01738, -0.03642, -0.00284, 0.00597, 0.02794, -0.02295, -0.01927, -0.00772, -0.03671, -0.03674, 0.01707, -0.00816, 0.00866, -0.00542, -0.01613, -0.01535, 0.01964, 0.01041, 0.00324, -0.00052, 0.00304, 0.01772, 0.03032, 0.00984, 0.02491, -0.01055, -0.05347, 0.00146, -0.00876, 0.02007, -0.00805, -0.01074, 0.0184, 0.02281, 0.01999, 0.03915, 0.01346, -0.02115, 0.01746, -0.01026, -0.0095, 0.00608, 0.0274, -0.03774, 0.01563, 0.05524, -0.00686, 0.01033, -0.04068, 0.02438, 0.01067, -0.00681, -0.00416, 0.00234, 0.02551, 0.01158, 0.02857, -0.00122, 0.01368, 0.00698, 0.0173, -0.01528, 0.00683, -0.00138, -0.04089, -0.00141, -0.02674, -0.01841, -0.03526, 0.01128, 0.00788, 0.00965, -0.00783, 0.01087, -0.00381, -0.01793, -0.0068, -0.01981, -0.00959, -0.03578, 0.01143, -0.03364, -0.00748, -0.00904, -0.02734, 0.00314, -0.02949, 0.02308, -0.01226, 0.00172, -0.00334, 0.0187, -0.02007, 0.02552, -0.00057, -0.03231, 0.01378, -0.00212, -0.00585, 0.01964, -0.0323, 0.04676, -0.03517, 0.04131, 0.0118, -0.01056, 0.01973, -0.00706, 0.00815, 0.04572, 0.00109, 0.00726, 0.00552, 0.04224, -0.01879, 0.01108, 0.02366, -0.02875, 0.00582, 0.00553, 0.00279, -0.03479, 0.009, 0.01082, 0.02314, 0.01302, -0.04155, -0.00982, 0.00063, -0.01865, -0.02634, -0.0294, 0.00417, 0.03912, 0.02337, 0.03002, 0.0269, 0.01449, 0.01886],+ "term_cos": {+ "a fast whoosh": [0.0159, 0.1165],+ "a soft air whoosh": [0.0059, 0.0957],+ "a deep cinematic whoosh": [0.0081, 0.0932],+ "a short swish": [0.0704, 0.1064],+ "a swoosh passing by": [0.0144, 0.094],+ "a stick swung fast through the air": [0.0165, 0.1194],+ "cloth whipping through the air": [0.0062, 0.1137],+ "a gust of wind rushing past": [0.011, 0.1104],+ "a rising riser building tension": [0.0516, 0.1226],+ "an uplifter sweep": [0.0497, 0.1342],+ "a reverse cymbal swell": [0.0366, 0.104],+ "a downlifter falling sweep": [0.0371, 0.1235],+ "a tape stop slowing down": [0.0618, 0.0876],+ "a record scratch": [0.0445, 0.0834],+ "a quick swipe": [0.0546, 0.0972],+ "a whip pan transition": [0.0717, 0.0932],+ "a stutter glitch transition": [0.0806, 0.0947],+ "a deep cinematic boom": [-0.0067, 0.0977],+ "a sub bass drop": [0.0194, 0.1054],+ "a punchy impact hit": [-0.0455, 0.1247],+ "a big braam": [0.0385, 0.1],+ "a heavy thud": [-0.0164, 0.1068],+ "a soft thump": [-0.021, 0.1067],+ "a cinematic hit with a long tail": [-0.019, 0.1043],+ "a big drum hit": [-0.0125, 0.0982],+ "a metallic clang": [0.0069, 0.0942],+ "a door slam": [-0.009, 0.0917],+ "a snare hit": [-0.0262, 0.0796],+ "a kick drum": [-0.0045, 0.0963],+ "a hand clap": [0.0677, 0.0797],+ "a gunshot": [0.0097, 0.0831],+ "an explosion": [-0.031, 0.0888],+ "a low rumble": [-0.0067, 0.0898],+ "a body falling on the ground": [-0.0124, 0.0874],+ "a wooden knock": [0.0142, 0.0981],+ "a mouse click": [0.0469, 0.1107],+ "a soft UI button tap": [0.0247, 0.1102],+ "a crisp interface click": [0.0783, 0.0924],+ "a notification ding": [0.0413, 0.1084],+ "a chat message pop": [0.1046, 0.1013],+ "a phone vibration buzz": [-0.0014, 0.0779],+ "a camera shutter click": [0.0524, 0.0987],+ "a toggle switch click": [0.075, 0.1086],+ "a light switch flicked": [0.0631, 0.1132],+ "a scroll wheel tick": [0.0679, 0.113],+ "a success chime": [0.0859, 0.1351],+ "an error buzz": [0.0113, 0.0916],+ "a digital glitch": [0.0616, 0.0835],+ "a typing cursor blip": [0.0545, 0.1212],+ "a touchscreen tap": [0.0548, 0.1042],+ "a button being pressed": [0.038, 0.1105],+ "a keyboard typing fast": [0.0455, 0.1102],+ "a single mechanical keyboard key press": [0.065, 0.1547],+ "a deep thocky mechanical keyboard": [0.0794, 0.116],+ "a clicky mechanical keyboard": [0.0462, 0.1238],+ "a laptop keyboard typing": [0.0589, 0.1169],+ "a spacebar press": [0.0817, 0.118],+ "an enter key press": [0.0969, 0.1287],+ "a soft key release click": [0.0233, 0.1385],+ "slow typing with separate key presses": [0.0619, 0.1223],+ "an old typewriter": [0.021, 0.1162],+ "a plastic click": [0.0647, 0.1048],+ "a plastic toy snapping together": [0.0795, 0.1087],+ "lego bricks clicking": [0.0448, 0.103],+ "a plastic case snapping shut": [0.036, 0.1012],+ "a latch clicking shut": [0.0262, 0.1142],+ "a box lid closing": [0.0526, 0.1019],+ "a ratchet clicking": [0.0381, 0.1102],+ "a fidget toy clicking": [0.053, 0.1059],+ "a pen click": [0.0687, 0.1148],+ "a cork pop": [0.0752, 0.1218],+ "a mouth pop": [0.0605, 0.1206],+ "a bubble pop": [0.0927, 0.1106],+ "a balloon squeak": [-0.0091, 0.0888],+ "a rubber stretch": [0.0143, 0.0847],+ "a suction cup pop": [0.0797, 0.1176],+ "a finger squeaking on glass": [-0.0242, 0.0955],+ "a finger tapping on wood": [0.0293, 0.1072],+ "a finger tapping on glass": [0.0469, 0.1112],+ "glass clinking": [-0.0098, 0.1023],+ "coins dropping": [0.0155, 0.0864],+ "paper rustling": [0.023, 0.1166],+ "a page turning": [0.0414, 0.0859],+ "a zipper": [0.0351, 0.0805],+ "cloth rustling": [0.0287, 0.1118],+ "footsteps on wood": [-0.0371, 0.0993],+ "footsteps on gravel": [-0.0651, 0.1078],+ "high heels walking": [-0.0484, 0.0821],+ "a small object dropping on a table": [0.0136, 0.1131],+ "a plastic object falling on a table": [0.0505, 0.0959],+ "dominoes falling": [0.0195, 0.0896],+ "a wooden block hit": [0.0303, 0.1032],+ "a wood creak": [0.0198, 0.0972],+ "a creaking door": [-0.016, 0.0882],+ "a metal scrape": [-0.0201, 0.0962],+ "a drawer closing": [0.0558, 0.0758],+ "scissors cutting": [0.0291, 0.1082],+ "a knife chopping on a board": [0.0439, 0.1199],+ "a can opening": [0.0698, 0.0971],+ "a squishy wet squelch": [-0.0128, 0.1017],+ "water splashing": [0.0105, 0.0881],+ "a water drop": [0.0748, 0.1082],+ "a liquid bloop": [0.0651, 0.1015],+ "pouring water": [-0.0029, 0.0921],+ "ice cubes in a glass": [0.0203, 0.1013],+ "an object plunging into water": [0.0412, 0.087],+ "bubbles underwater": [0.0083, 0.1144],+ "being underwater": [0.0231, 0.1062],+ "a stream flowing": [0.0465, 0.0947],+ "a waterfall": [0.0508, 0.1169],+ "a dripping tap": [0.0503, 0.1135],+ "footsteps on grass": [-0.0248, 0.1088],+ "walking through dry leaves": [0.0009, 0.1403],+ "leaves rustling": [0.0161, 0.1335],+ "a tree shaking": [0.0304, 0.1101],+ "a branch snapping": [0.0331, 0.0974],+ "an apple falling on grass": [0.0129, 0.095],+ "something falling into bushes": [0.0162, 0.0974],+ "digging in soil": [-0.0421, 0.1151],+ "a small animal running through grass": [0.0099, 0.1069],+ "insects buzzing": [0.0096, 0.0878],+ "a bee flying by": [0.0037, 0.0883],+ "a bird flapping its wings": [-0.0109, 0.1014],+ "a horse galloping": [-0.0008, 0.0829],+ "a small bell": [0.0427, 0.1089],+ "a glass ting": [0.0499, 0.1254],+ "a wine glass ring": [-0.0081, 0.1074],+ "a triangle hit": [-0.0118, 0.1157],+ "wind chimes": [0.0417, 0.1294],+ "a glissando on bar chimes": [0.0547, 0.1421],+ "a celesta note": [0.0849, 0.1069],+ "a kalimba note": [0.0385, 0.1337],+ "a glockenspiel run": [0.09, 0.1481],+ "a singing bowl": [-0.0291, 0.1024],+ "a sustained glass harp tone": [-0.0263, 0.1095],+ "a music box melody": [0.0623, 0.1405],+ "a doorbell": [0.0598, 0.107],+ "church bells": [0.0097, 0.1214],+ "a magic sparkle chime": [0.0776, 0.1405],+ "rain on a window": [0.0488, 0.1291],+ "heavy rain": [0.0612, 0.1303],+ "rain on a roof": [0.0407, 0.1297],+ "a thunderstorm": [0.0429, 0.1019],+ "wind blowing": [-0.0236, 0.0956],+ "a gentle breeze in tall grass": [-0.008, 0.1017],+ "wind in pine trees": [0.0177, 0.1127],+ "birds singing in the morning": [-0.048, 0.1007],+ "skylarks over a meadow": [0.0163, 0.1128],+ "a forest ambience": [0.0122, 0.0982],+ "a quiet countryside morning": [-0.0312, 0.1031],+ "a summer meadow with insects": [0.0054, 0.0994],+ "a city street with traffic": [-0.0628, 0.0791],+ "a busy cafe": [-0.0978, 0.0718],+ "an office ambience": [-0.0413, 0.0785],+ "a quiet room tone": [-0.0362, 0.1002],+ "a quiet room at night": [-0.0378, 0.1013],+ "a computer fan hum": [-0.013, 0.1019],+ "an electrical hum": [-0.0219, 0.0821],+ "ocean waves": [0.0629, 0.1064],+ "a crackling fire": [0.0469, 0.0976],+ "crickets at night": [-0.0271, 0.0922],+ "an airplane flying overhead": [-0.0008, 0.0867],+ "a crowd cheering": [-0.0543, 0.074],+ "a dog barking": [-0.0407, 0.0701],+ "people talking in the background": [-0.1115, 0.0735],+ "a car horn honking": [-0.0438, 0.078],+ "a car engine starting": [0.0227, 0.0851],+ "a car driving past": [-0.0249, 0.0779],+ "a motorcycle revving": [0.0055, 0.0776],+ "a train passing": [-0.0085, 0.0755],+ "a helicopter flying overhead": [0.0241, 0.0794],+ "a police siren": [-0.0274, 0.0769],+ "a bicycle bell": [0.0124, 0.0994],+ "a knock on a door": [-0.0204, 0.0886],+ "a phone ringing": [-0.0012, 0.0845],+ "an alarm clock beeping": [0.0023, 0.0886],+ "a microwave beep": [0.013, 0.0783],+ "a kettle whistling": [-0.0508, 0.0934],+ "a vacuum cleaner": [-0.0036, 0.0993],+ "a toilet flushing": [-0.0099, 0.0853],+ "a clock ticking": [0.026, 0.1032],+ "a cash register": [0.0206, 0.0833],+ "a printer printing": [0.016, 0.1018],+ "keys jingling": [0.007, 0.0909],+ "a match being struck": [-0.013, 0.0951],+ "a spray can hissing": [-0.0039, 0.0901],+ "a hammer hitting a nail": [0.0461, 0.1057],+ "a power drill": [-0.0143, 0.0825],+ "a saw cutting wood": [0.0084, 0.0861],+ "a sword being drawn": [-0.0047, 0.0876],+ "a basketball bouncing": [0.0463, 0.0877],+ "fireworks exploding": [-0.0316, 0.0979],+ "a thunder clap": [0.0289, 0.0969],+ "a heartbeat": [0.0222, 0.0848],+ "applause": [-0.0047, 0.0842],+ "a cat meowing": [-0.0342, 0.0711],+ "a bird chirping": [-0.0398, 0.0885],+ "a rooster crowing": [0.0147, 0.0893],+ "a cow mooing": [-0.0344, 0.087],+ "a horse neighing": [-0.046, 0.0733],+ "minimal electronic music": [0.101, 0.1174],+ "future bass music": [0.0905, 0.1204],+ "lo-fi hip hop beat": [0.0747, 0.1153],+ "cinematic orchestral music": [0.0519, 0.1335],+ "uplifting orchestral film score": [0.0475, 0.1401],+ "epic trailer music": [0.0813, 0.1365],+ "ambient synth pad music": [0.018, 0.1402],+ "upbeat corporate pop music": [0.1169, 0.1356],+ "inspirational corporate background music": [0.0893, 0.1503],+ "synthwave music": [0.0725, 0.1447],+ "house music": [0.0902, 0.1141],+ "tech house music": [0.1018, 0.1209],+ "drum and bass music": [0.0162, 0.1197],+ "trap beat": [0.0766, 0.1092],+ "glitch hop music": [0.0781, 0.1162],+ "indie pop music": [0.0929, 0.1076],+ "solo piano music": [0.0592, 0.1109],+ "playful children's piano music": [0.0687, 0.1124],+ "cheerful ukulele music": [0.0568, 0.115],+ "acoustic guitar music": [0.0703, 0.0971],+ "jazz music": [0.0318, 0.0938],+ "rock music with electric guitars": [0.028, 0.1091],+ "hip hop beat with heavy bass": [0.0124, 0.094],+ "chiptune video game music": [0.0676, 0.1062],+ "dubstep music": [0.0569, 0.1187],+ "classical string quartet": [0.0142, 0.0953],+ "funk music": [0.0596, 0.0871],+ "techno music": [0.043, 0.129],+ "uplifting and hopeful music": [0.0926, 0.1452],+ "playful and quirky music": [0.0897, 0.0966],+ "dark and tense music": [0.0692, 0.1371],+ "sad and melancholic music": [0.0748, 0.1436],+ "calm and relaxing music": [0.0395, 0.1205],+ "energetic and driving music": [0.052, 0.1247],+ "dreamy and ethereal music": [0.0452, 0.1556],+ "triumphant and heroic music": [0.0391, 0.1263],+ "mysterious music": [0.0888, 0.1486],+ "romantic music": [0.114, 0.1308],+ "aggressive music": [0.0485, 0.1358],+ "warm and cozy music": [0.1014, 0.1256],+ "futuristic music": [0.0855, 0.1344],+ "confident and cool music": [0.0819, 0.1299],+ "suspenseful build-up music": [0.0813, 0.1398],+ "celebratory and joyful music": [0.0446, 0.097],+ "a deep sub bass": [0.0306, 0.0929],+ "a punchy kick drum pattern": [-0.031, 0.103],+ "crisp hi-hats": [0.0347, 0.0853],+ "a snare roll": [-0.0108, 0.0827],+ "a synthesizer lead": [0.0694, 0.1138],+ "a warm analog synth pad": [0.0239, 0.136],+ "a plucked synth arpeggio": [0.0898, 0.1201],+ "a grand piano": [0.0388, 0.1113],+ "an electric piano": [0.0809, 0.1072],+ "strings ensemble": [0.0595, 0.1244],+ "pizzicato strings": [0.0499, 0.0976],+ "brass section": [0.0379, 0.0896],+ "a celesta": [0.1095, 0.1],+ "a glockenspiel": [0.1137, 0.1576],+ "bells": [0.064, 0.1288],+ "a marimba": [0.0948, 0.1244],+ "an acoustic guitar": [0.0725, 0.076],+ "an electric guitar": [0.0569, 0.0838],+ "a bass guitar": [0.0439, 0.0781],+ "a ukulele": [0.0569, 0.0979],+ "a choir": [-0.0064, 0.1112],+ "a vocal chop": [0.091, 0.087],+ "orchestral percussion": [0.0676, 0.118],+ "a music box": [0.0777, 0.1299],+ "a cymbal roll": [0.0247, 0.1023],+ "a man speaking": [-0.1071, 0.0829],+ "a woman speaking": [-0.0931, 0.0793],+ "a voice-over narration": [-0.0124, 0.0809],+ "someone whispering": [-0.0379, 0.0769],+ "singing vocals": [0.0006, 0.0853],+ "a crowd talking": [-0.1111, 0.0701],+ "children laughing": [-0.0811, 0.0757],+ "a person breathing": [-0.014, 0.0823],+ "a person coughing": [-0.0419, 0.0808],+ "people laughing": [-0.1184, 0.0795],+ "a baby crying": [-0.044, 0.0757],+ "a man shouting": [-0.1108, 0.0798],+ "a deep sound": [0.0492, 0.0783],+ "a bright sound": [0.0608, 0.1055],+ "a thin sound": [-0.0028, 0.0805],+ "a warm sound": [0.09, 0.0969],+ "a harsh sound": [0.029, 0.0714],+ "a muffled sound": [-0.0462, 0.0887],+ "a crisp sound": [0.0342, 0.0836],+ "a punchy sound": [0.0255, 0.1018],+ "a soft sound": [-0.0066, 0.0849],+ "a loud sound": [-0.0247, 0.0765],+ "a quiet sound": [-0.0208, 0.0896],+ "a short sound": [0.0744, 0.0849],+ "a long sound": [0.0759, 0.0731],+ "a dry close sound": [0.1289, 0.0994],+ "a reverberant sound in a large hall": [-0.0292, 0.0882],+ "a distant sound": [-0.0381, 0.0906],+ "a tight snappy sound": [0.04, 0.0875],+ "a boomy sound": [0.0568, 0.0805],+ "an airy sound": [0.0024, 0.0995],+ "a metallic sound": [-0.0213, 0.0933],+ "a hollow sound": [0.0214, 0.089],+ "a satisfying sound": [0.0861, 0.1045],+ "a real recording of a physical object": [0.0704, 0.1014],+ "a close-miked foley recording": [0.0458, 0.0889],+ "a field recording outdoors": [-0.0131, 0.0881],+ "a professionally mixed and mastered music track": [0.1832, 0.1042],+ "a high quality studio recording": [0.1746, 0.0621],+ "a cheap synthesized beep": [-0.0023, 0.0869],+ "a pure sine wave tone": [0.0334, 0.0915],+ "a square wave chiptune blip": [0.0202, 0.0874],+ "a computer-generated electronic tone": [0.0345, 0.0908],+ "white noise": [-0.0047, 0.1225],+ "pink noise": [0.03, 0.1229],+ "a retro video game sound effect": [0.0476, 0.0891],+ "an 8-bit sound effect": [0.0424, 0.094],+ "a robotic synthetic sound": [0.1072, 0.0828],+ "a low quality recording with hiss": [-0.0056, 0.0971],+ "a distorted clipped recording": [0.0074, 0.0738],+ "a low bitrate mp3 with artifacts": [0.0538, 0.0883],+ "a recording made on a phone": [0.0171, 0.0732],+ "silence": [0.0602, 0.1076]+ }+}diff --git a/earshot/describe.py b/earshot/describe.pynew file mode 100644index 0000000..236730c--- /dev/null+++ b/earshot/describe.py@@ -0,0 +1,269 @@+"""What does this sound sound like? Calibrated vocabulary tags, AudioSet labels and key measurements in one paragraph.++ earshot describe key.ogg+ earshot describe ambience.ogg --start 30 --end 60 --json+ from earshot.describe import describe+ describe('enter.ogg')['summary']+ # '0.56 s short transient (attack 3 ms, decays in 190 ms). Sounds most like: a single mechanical keyboard key press+ # (z +5.4), a soft key release click (z +3.6), a clicky mechanical keyboard (z +3.2) [clear reading]. ...'++Tags: CLAP against earshot/data/vocab.json, calibrated per sound and per term (earshot/vocab.py). Reading of the best+tag: z >= 3 clear, 2.5-3 plausible, 2-2.5 weak, below 2 tentative (match.VERDICTS; held-out evidence in+match.VERDICT_EVIDENCE and eval/semantics/RESULTS.md). 'Other readings' = the best tags of other categories: the+closest alternative interpretations. Music genre / mood, instrument and voice tags were never validated: they are+listed separately as hints (on 45 upbeat library tracks the top mood tag was a dark / sad / tense one for 18%).+AudioSet labels come from the AST tagger (independent per-class probabilities, hears up to 8 kHz only). Measurements+come from earshot.features (no model; brightness = median magnitude-weighted spectral centroid). Files longer than+20 s are described as a whole (mean of up to 6 evenly spaced 10 s windows, the same ones rank and match use) plus up+to 3 non-overlapping windows that read differently from each other. Near-silent input (peak below -70 dBFS) and+input shorter than 5 ms are reported as such, untagged.+"""+import warnings++import numpy as np++from . import audio, features, vocab+from .cli import common+from .match import SKIP_CATS, VERDICTS, verdict+from .report import fmt_t++LONG_S = 20.0+SILENT_DBFS = -70.0+MONO_DB = -40.0+MUSIC_P = 0.5+NOTE = ['C', 'C#', 'D', 'Eb', 'E', 'F', 'F#', 'G', 'Ab', 'A', 'Bb', 'B']+++def _hz(f):+ return f'{f / 1000:.1f} kHz' if f >= 1000 else f'{f:.0f} Hz'+++def _note(f):+ m = int(round(69 + 12 * np.log2(f / 440.0)))+ return f'{NOTE[m % 12]}{m // 12 - 1}'+++def _ms(v):+ return f'{v / 1000:.1f} s' if v >= 2000 else f'{v:.0f} ms'+++def shape(m):+ """a few words on the envelope / texture from the measurements"""+ d, att, dec, ons = m['duration'], m.get('attack_ms'), m.get('decay_ms'), m.get('onset_count')+ rate = m.get('onsets_per_s', 0)+ if d > LONG_S:+ words = ['long recording' if ons is None else f'busy, {rate:.1f} onsets/s' if rate >= 1.5 else+ 'steady, few onsets' if ons <= d / 10 else f'{ons} separate events']+ if m.get('lra_lu') is not None:+ words.append(f'loudness range {m["lra_lu"]:.0f} LU')+ return ', '.join(words)+ if d < 2.5 and (ons is None or ons <= 3):+ kind = 'transient' if att is not None and att <= 15 else 'swell' if att is not None and att >= 150 else 'one-shot'+ words = [f'short {kind}']+ elif rate >= 1.5:+ words = [f'busy, {rate:.1f} onsets/s']+ elif (ons or 0) >= 2:+ words = [f'{ons} separate events']+ else:+ words = ['sustained']+ if m.get('f0_hz'):+ words.append(f'tonal, pitch ~{m["f0_hz"]:.0f} Hz ({_note(m["f0_hz"])})')+ elif m.get('flatness', 0) >= 0.3:+ words.append('noise-like')+ env = ([f'attack {att:.0f} ms'] if att is not None else []) + ([f'decays in {_ms(dec)}'] if dec is not None else [])+ return ', '.join(words) + (f' ({", ".join(env)})' if env else '')+++MEAS_KEYS = ['duration', 'peak_dbfs', 'true_peak_dbtp', 'rms_dbfs', 'lufs', 'lra_lu', 'attack_ms', 'decay_ms',+ 'brightness_hz', 'bandwidth_hz', 'noise_floor_dbfs', 'flatness', 'harmonic_ratio', 'f0_hz',+ 'onset_count', 'onsets_per_s', 'width', 'digital_silence_frac']+++def measurements(m):+ """the measurements worth showing: no noise floor under 1 s (it would be the padding), width 'mono' below -40 dB"""+ out = dict(m)+ sm = m.get('side_to_mid_db')+ out['width'] = 'mono' if sm is None or sm < MONO_DB else f'side/mid {sm:.0f} dB'+ if m['duration'] < 1.0:+ out.pop('noise_floor_dbfs', None)+ return {k: out[k] for k in MEAS_KEYS if out.get(k) is not None}+++def _meas_text(m):+ s = [f'peak {m["peak_dbfs"]:.1f} dBFS']+ if m.get('lufs') is not None:+ s.append(f'{m["lufs"]:.1f} LUFS')+ if m.get('brightness_hz'):+ s.append(f'brightness {_hz(m["brightness_hz"])}')+ if m.get('bandwidth_hz'):+ s.append(f'bandwidth {_hz(m["bandwidth_hz"])}')+ nf = m.get('noise_floor_dbfs')+ if nf is not None and m['duration'] <= LONG_S:+ s.append('digital silence between sounds' if nf < -120 else f'noise floor {nf:.0f} dBFS')+ s.append(m['width'])+ return ', '.join(s)+++def _ast_top(p, t, k=5, min_p=0.05):+ top = t.top(p, k)+ return [{'label': l, 'p': pr} for i, (l, pr) in enumerate(top) if pr >= min_p or i == 0]+++def _ast_text(top):+ labels = ', '.join(f"{d['label']} {d['p']:.2f}" for d in top[:3])+ return f'AudioSet: {labels}.' if top[0]['p'] >= 0.2 else f'AudioSet tagger unsure ({labels}).'+++def reading(z):+ v = verdict(z)+ return {'strong': 'clear reading', 'plausible': 'plausible reading', 'weak': 'weak reading'}.get(+ v, 'tentative: nothing in the vocabulary stands out')+++def _tags(v, a, top):+ """(validated top tags, unvalidated hints, best 3 per category)"""+ rows = v.top_terms(a, k=len(v.names), exclude=SKIP_CATS)+ best = [dict(term=t, category=c, z=z, cosine=cos, verdict=verdict(z))+ for t, c, z, cos in rows if c not in vocab.UNVALIDATED][:top]+ hints = [dict(term=t, category=c, z=z, cosine=cos) for t, c, z, cos in rows if c in vocab.UNVALIDATED][:top]+ per = {cat: [dict(term=t, z=z, cosine=cos) for t, _, z, cos in v.top_terms(a, k=3, cats=(cat,))] for cat in v.cats}+ return best, hints, per+++def distinct_windows(spans, W, tags, n=3):+ """up to n non-overlapping windows, farthest from the file's mean embedding first, each with a different top tag"""+ if len(spans) <= 1:+ return []+ mean = W.mean(0)+ d0 = 1 - W @ (mean / (np.linalg.norm(mean) + 1e-12))+ picked = []+ for i in np.argsort(-d0):+ if len(picked) >= n:+ break+ if all(spans[i][1] <= spans[j][0] + 1e-6 or spans[i][0] >= spans[j][1] - 1e-6 for j in picked) and \+ tags[i] not in {tags[j] for j in picked}:+ picked.append(int(i))+ return sorted(picked) if len(picked) > 1 else []+++def _ast():+ from .models import ast as ast_mod+ with warnings.catch_warnings():+ warnings.simplefilter('ignore')+ return ast_mod.get()+++def _untagged(res, why):+ res.update(shape=why, top_tags=[], music_hints=[], tags={}, audioset=[], character=[], summary=why)+ return res+++def describe(path, start=None, end=None, top=8):+ """tags (calibrated), AudioSet labels and measurements of a sound or a [start, end) segment of it"""+ vocab.check(path, start, end)+ x = audio.load(path, start=start, end=end)+ dur = len(x) / audio.SR+ s0 = start or 0.0+ m = features.measure(x, light=dur > 60)+ res = {'path': path, 'start': start, 'end': end, 'duration': round(dur, 3)}+ if m.get('error'):+ res['measurements'] = {'duration': round(dur, 3)}+ return _untagged(res, f'{dur * 1000:.1f} ms of audio: too short to describe or measure (needs at least 5 ms).')+ meas = measurements(m)+ res['measurements'] = meas+ if m['peak_dbfs'] < SILENT_DBFS:+ return _untagged(res, f'{dur:.2f} s of near-silence (peak {m["peak_dbfs"]:.0f} dBFS): nothing to describe; '+ f'model tags of silence are meaningless.')+ v = vocab.get()+ t = _ast()+ xm = audio.to_mono(x)+ if dur > LONG_S:+ spans = vocab.spans_for(s0, s0 + dur)+ W = vocab.embed_spans([(path, a, b) for a, b in spans])+ a = vocab._norm(W.mean(0))+ with warnings.catch_warnings():+ warnings.simplefilter('ignore')+ pw = t.probs([xm[int((a_ - s0) * audio.SR):int((b_ - s0) * audio.SR)] for a_, b_ in spans])+ p = pw.max(0)+ else:+ a = vocab.embed_file(path, start, end)+ with warnings.catch_warnings():+ warnings.simplefilter('ignore')+ p = t.probs_file(path, start, end)+ best, hints, per = _tags(v, a, top)+ ast_top = _ast_top(p, t)+ p_music = t.prob_of(p, 'Music')+ other, seen = [], {b['category'] for b in best[:3]}+ for b in best[3:]:+ if b['category'] not in seen and len(other) < 2:+ other.append(b)+ seen.add(b['category'])+ char = [d['term'].split(' ', 1)[1].removesuffix(' sound') for d in per.get('quality') or []+ if d['z'] >= VERDICTS[1][0]][:2]+ res.update(shape=shape(m), top_tags=best, music_hints=hints, tags=per, audioset=ast_top, character=char)+ s = f'{dur:.2f} s {res["shape"]}.'+ if p_music >= MUSIC_P:+ gen = [h for h in hints if h['category'] == 'music_genre'][:2]+ mood = [h for h in hints if h['category'] == 'music_mood'][:2]+ s += (f' Music (AudioSet Music {p_music:.2f}). Unvalidated hints - genre: '+ + ', '.join(f"{h['term']} (z {vocab.zf(h['z'])})" for h in gen)+ + '; mood (wrong for ~1 in 5 tracks in a test): ' + ', '.join(f"{h['term']} (z {vocab.zf(h['z'])})"+ for h in mood) + '.')+ if best[0]['z'] >= VERDICTS[-1][0]:+ s += (' Sounds most like: ' + ', '.join(f"{b['term']} (z {vocab.zf(b['z'])})" for b in best[:3])+ + f' [{reading(best[0]["z"])}].')+ if other:+ s += ' Other readings: ' + ', '.join(f"{b['term']} ({b['category']}, z {vocab.zf(b['z'])})"+ for b in other) + '.'+ else:+ s += (' No sound-effect description stands out (closest: ' + ', '.join(+ f"{b['term']} z {vocab.zf(b['z'])}" for b in best[:2]) + '; tentative).')+ if p_music < MUSIC_P and hints and hints[0]['z'] >= VERDICTS[0][0]:+ s += f" Unvalidated hint: {hints[0]['term']} (z {vocab.zf(hints[0]['z'])})."+ if char:+ s += ' Character: ' + ', '.join(char) + '.'+ s += ' ' + _ast_text(ast_top) + ' ' + _meas_text(meas) + '.'+ if dur > LONG_S:+ wt = [_tags(v, W[i], 1)[0][0] for i in range(len(spans))]+ wins = []+ for i in distinct_windows(spans, W, [w['term'] for w in wt]):+ wins.append({'start': round(spans[i][0], 3), 'end': round(spans[i][1], 3), 'top_tag': wt[i],+ 'audioset': _ast_top(pw[i], t, k=3, min_p=0.1)})+ res['distinct_windows'] = wins+ if wins:+ s += ' Windows that read differently: ' + '; '.join(+ f"{fmt_t(w['start'])}-{fmt_t(w['end'])} {w['top_tag']['term']} (z {vocab.zf(w['top_tag']['z'])}; "+ f"AudioSet {w['audioset'][0]['label']} {w['audioset'][0]['p']:.2f})" for w in wins) + '.'+ else:+ s += f" Uniform: all {len(spans)} analysed 10 s windows read as {wt[0]['term']}."+ res['summary'] = s+ return res+++def _text(r):+ lines = [r['summary']]+ if r['top_tags']:+ lines += ['', 'top tags (z calibrated: >= 3 clear, 2.5-3 plausible, 2-2.5 weak, < 2 tentative):']+ lines += [f" {vocab.zf(b['z']):>5} {b['term']} [{b['category']}] cos {b['cosine']:.3f}" for b in r['top_tags']]+ if r['music_hints']:+ lines.append('unvalidated hints (music genre / mood, instruments, voice): ' + ', '.join(+ f"{h['term']} ({vocab.zf(h['z'])})" for h in r['music_hints'][:5]))+ lines.append('best per category: ' + '; '.join(f"{c}: {d[0]['term']} ({vocab.zf(d[0]['z'])})"+ for c, d in r['tags'].items() if d))+ lines.append('audioset: ' + ', '.join(f"{d['label']} {d['p']:.2f}" for d in r['audioset']))+ lines.append('measurements: ' + ', '.join(f'{k} {v}' for k, v in r['measurements'].items()))+ return '\n'.join(lines)+++def add_cli(sub):+ p = sub.add_parser('describe', help='what does a sound sound like: calibrated tags, AudioSet labels, measurements')+ p.add_argument('path', help='audio file or video')+ p.add_argument('--start', type=float, help='describe only from this time (s), e.g. a trim from find')+ p.add_argument('--end', type=float, help='describe only up to this time (s)')+ p.add_argument('--top', type=int, default=8, help='tags per category (default 8)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(lambda: describe(a.path, a.start, a.end, a.top), a, _text))+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/doctor.py b/earshot/doctor.pynew file mode 100644index 0000000..5272789--- /dev/null+++ b/earshot/doctor.py@@ -0,0 +1,85 @@+"""earshot doctor: check the install (python packages, ffmpeg, model files, CPU settings, optional extras).++ earshot doctor [--load] [--json]++Without --load it imports nothing heavy (a few seconds). --load also loads both models, times one embedding (first+load can take ~20-40 s) and reports the precision in use (bf16 only on CPUs with native bf16, else fp32).+"""+import importlib+import importlib.metadata+import importlib.util+import os+import shutil+import sys+import time++from . import config+from .report import emit++PACKAGES = [('numpy', 'numpy'), ('scipy', 'scipy'), ('soundfile', 'soundfile'), ('soxr', 'soxr'),+ ('librosa', 'librosa'), ('pyloudnorm', 'pyloudnorm'), ('sklearn', 'scikit-learn'), ('PIL', 'pillow'),+ ('matplotlib', 'matplotlib'), ('torch', 'torch'), ('transformers', 'transformers'), ('mcp', 'mcp'),+ ('requests', 'requests')]+++def check(load=False):+ r = {'python': sys.version.split()[0], 'earshot_home': config.HOME, 'threads': config.threads(),+ 'precision': config.PRECISION, 'clap_model': config.CLAP_MODEL, 'ast_model': config.AST_MODEL}+ pk = {}+ for name, dist in PACKAGES: # installed versions from package metadata: no import (torch alone takes ~10-20 s)+ try:+ if importlib.util.find_spec(name) is None:+ raise ImportError(name)+ try:+ pk[name] = importlib.metadata.version(dist)+ except importlib.metadata.PackageNotFoundError:+ pk[name] = 'ok'+ except Exception as e: # noqa: BLE001+ pk[name] = f'MISSING ({type(e).__name__})'+ r['packages'] = pk+ r['ffmpeg'] = shutil.which('ffmpeg') or 'MISSING (install ffmpeg and put it on PATH)'+ r['ffprobe'] = shutil.which('ffprobe') or 'MISSING'+ r['torch_cuda_used'] = False++ def cached(mid):+ d = os.path.join(config.HF_CACHE, 'models--' + mid.replace('/', '--'))+ return os.path.isdir(d)+ r['models_cached'] = {config.CLAP_MODEL: cached(config.CLAP_MODEL), config.AST_MODEL: cached(config.AST_MODEL)}+ r['llm_keys_present'] = {k: bool(os.environ.get(k)) for k in ('GEMINI_API_KEY', 'GOOGLE_API_KEY')}+ data = {f: os.path.exists(os.path.join(config.DATA_DIR, f)) for f in+ ('vocab.json', 'vocab_stats.json', 'realness_model.json', 'launch_profile.json')}+ r['data_files'] = data+ problems = [f'{k}: {v}' for k, v in pk.items() if str(v).startswith('MISSING') and k not in ('mcp', 'requests')]+ if 'MISSING' in str(r['ffmpeg']):+ problems.append('ffmpeg missing')+ problems += [f'data file earshot/data/{f} missing' for f, ok in data.items() if not ok]+ if load:+ import numpy as np+ from .models import ast, clap+ t = time.time()+ c = clap.get()+ r['clap_load_s'] = round(time.time() - t, 1)+ t = time.time()+ c.embed_audio([np.zeros(48000, np.float32)])+ r['clap_one_window_s'] = round(time.time() - t, 2)+ r['clap_precision'] = c.precision+ t = time.time()+ a = ast.get()+ r['ast_load_s'] = round(time.time() - t, 1)+ t = time.time()+ a.probs([np.zeros(48000, np.float32)])+ r['ast_one_window_s'] = round(time.time() - t, 2)+ import torch+ r['native_bf16'] = clap.native_bf16(torch)+ r['problems'] = problems+ r['summary'] = ('earshot looks healthy.' if not problems else 'Problems: ' + '; '.join(problems)) + (+ '' if all(r['models_cached'].values()) else ' Models are not cached yet: the first call downloads them '+ '(~1.4 GB) into ' + config.HF_CACHE + '.')+ return r+++def add_cli(sub):+ from .cli import common+ p = common(sub.add_parser('doctor', help='check the install, models and CPU settings'))+ p.add_argument('--load', action='store_true', help='also load both models and time one window')+ p.set_defaults(func=lambda a: emit(check(a.load), a) and 0)diff --git a/earshot/features.py b/earshot/features.pynew file mode 100644index 0000000..0b03162--- /dev/null+++ b/earshot/features.py@@ -0,0 +1,313 @@+"""Objective measurements of a sound (no models): what a meter, a spectrum analyser and an oscilloscope would tell you.++ from earshot.features import measure+ m = measure('hit.wav') # or an array (n,) / (n, 2) at 48 kHz+ m['lufs'], m['attack_ms'], m['centroid_hz'], m['noise_floor_dbfs'], m['digital_silence_frac']++Fields (all optional: short sounds skip what needs length):+ duration, peak_dbfs, true_peak_dbtp (4x oversampled), rms_dbfs, lufs (BS.1770 integrated), lra_lu, short_term_max_lufs,+ clipped_samples, dc, noise_floor_dbfs (10th pct of 20 ms RMS), dynamic_range_db, digital_silence_frac (samples at+ exactly zero, |x| < 1e-6: code-generated audio often has long stretches of it, but so do trimmed, gated or+ noise-reduced real recordings (59% of one licensed real-recording library) - weak evidence on its own),+ side_to_mid_db + correlation (stereo width; None for mono-identical channels), attack_ms (10% -> 90% rise of the+ first event on a 5 ms RMS envelope: ~4 ms floor for an instant onset), decay_ms (overall peak -> 30 dB down on a+ 10 ms RMS envelope, staying down for 20 ms; see envelope_times), brightness_hz (median over active frames of the magnitude-weighted spectral+ centroid: tracks perceived brightness), centroid_hz (power-weighted over the whole spectrum: dominated by the low end,+ kept for compatibility), rolloff95_hz, bandwidth_hz (highest 1/3-oct band within 60 dB of the loudest), bands_db+ (energy share per band: sub 20-60, low 60-250, lowmid 250-1k, mid 1-4k, presence 4-8k, air 8-16k, top 16k+), flatness+ (0 = pure tone .. 1 = white noise), tonal_peak_ratio_db (strongest spectral peak vs median: sine-like when huge),+ harmonic_ratio (HPSS harmonic share, first 30 s), f0_hz (median pitch, only when harmonic_ratio >= 0.8), onsets+ (times) + onsets_per_s, and with music=True: tempo, beats, key.+"""+import math+import os++import numpy as np+from scipy import ndimage, signal++from . import audio++SR = audio.SR+BANDS = {'sub': (20, 60), 'low': (60, 250), 'lowmid': (250, 1000), 'mid': (1000, 4000), 'presence': (4000, 8000),+ 'air': (8000, 16000), 'top': (16000, 24000)}+KK_MAJ = np.array([6.35, 2.23, 3.48, 2.33, 4.38, 4.09, 2.52, 5.19, 2.39, 3.66, 2.29, 2.88])+KK_MIN = np.array([6.33, 2.68, 3.52, 5.38, 2.60, 3.53, 2.54, 4.75, 3.98, 2.69, 3.34, 3.17])+NOTE = ['C', 'C#', 'D', 'Eb', 'E', 'F', 'F#', 'G', 'Ab', 'A', 'Bb', 'B']+++def db(v):+ return audio.db(v)+++def _stereo(x):+ x = np.asarray(x, np.float32)+ return np.stack([x, x], axis=1) if x.ndim == 1 else x+++def frame_rms(m, win):+ n = len(m) // win+ if n == 0:+ return np.array([math.sqrt(float(np.mean(m.astype(np.float64) ** 2)) + 1e-20)])+ f = m[: n * win].reshape(n, win).astype(np.float64)+ return np.sqrt(np.mean(f ** 2, axis=1) + 1e-20)+++def smooth_env(m, win_ms):+ """RMS envelope with one value per millisecond, each over a centred `win_ms` window (truncated at the file edges).+ 1 ms frames are useless for envelopes: they dip at every zero crossing below ~500 Hz and flicker on noise"""+ m = np.asarray(m, np.float64)+ n, h, w2 = len(m), SR // 1000, max(1, int(win_ms * SR / 2000))+ c = np.concatenate([[0.0], np.cumsum(m * m)])+ mid = np.arange(0, n, h) + h // 2+ a, b = np.clip(mid - w2, 0, n), np.clip(mid + w2, 0, n)+ return np.sqrt((c[b] - c[a]) / np.maximum(b - a, 1) + 1e-20)+++def envelope_times(m):+ """attack_at / peak_at (s), attack_ms and decay_ms of a mono 48 kHz array.+ attack: the first event (first ms within 20 dB of the loudest), its 10% -> 90% rise to its own maximum within 300 ms,+ on a 5 ms RMS envelope (an instant onset reads ~4 ms, a sound that starts at full level ~0 ms);+ decay: from the overall peak until the 10 ms RMS envelope is 30 dB down and stays there for 20 ms (None when it+ never gets there: a sustained sound, or one cut off / buried in its own noise floor)"""+ e5, e10 = smooth_env(m, 5), smooth_env(m, 10)+ i0 = int(np.argmax(e5 >= e5.max() * 0.1))+ loc = e5[i0: i0 + 300]+ ip = int(np.argmax(e10))+ after = e10[ip:]+ ahead = ndimage.maximum_filter1d(after, size=21, origin=-10, mode='nearest') # max over [i, i + 20 ms]+ below = np.nonzero(ahead < e10[ip] * 10 ** (-30 / 20))[0]+ return {'attack_at': round(i0 / 1000, 3), 'peak_at': round(ip / 1000, 3),+ 'attack_ms': float(int(np.argmax(loc >= 0.9 * loc.max()))) if len(loc) else None,+ 'decay_ms': float(below[0]) if len(below) else None}+++def true_peak_dbtp(x):+ y = signal.resample_poly(_stereo(x), 4, 1, axis=0)+ return db(np.abs(y).max())+++def _kw(x):+ """BS.1770 K-weighting at 48 kHz"""+ b1, a1 = [1.53512485958697, -2.69169618940638, 1.19839281085285], [1.0, -1.69065929318241, 0.73248077421585]+ b2, a2 = [1.0, -2.0, 1.0], [1.0, -1.99004745483398, 0.99007225036621]+ return signal.lfilter(b2, a2, signal.lfilter(b1, a1, x, axis=0), axis=0)+++def loudness_curve(x, win=0.4, hop=0.1):+ """(t_mid, LUFS) of sliding blocks: win 0.4 = momentary, 3.0 = short-term"""+ y = _kw(_stereo(x).astype(np.float64))+ p = np.cumsum(np.concatenate([np.zeros((1, 2)), y ** 2]), axis=0)+ w, h = int(win * SR), int(hop * SR)+ if len(y) < w:+ ms = (p[-1] - p[0]) / max(1, len(y))+ return np.array([len(y) / SR / 2]), np.array([-0.691 + 10 * np.log10(ms.sum() + 1e-20)])+ st = np.arange(0, len(y) - w + 1, h)+ ms = (p[st + w] - p[st]) / w+ return (st + w / 2) / SR, -0.691 + 10 * np.log10(ms.sum(axis=1) + 1e-20)+++def integrated_lufs(x):+ t, L = loudness_curve(x, 0.4, 0.1)+ L = L[L > -70]+ if not len(L):+ return -70.0+ rel = 10 * np.log10(np.mean(10 ** (L / 10))) - 10+ L2 = L[L > rel]+ return float(10 * np.log10(np.mean(10 ** (L2 / 10)))) if len(L2) else float(rel + 10)+++def lra(x):+ t, S = loudness_curve(x, 3.0, 0.1)+ S = S[S > -70]+ if len(S) < 5:+ return None+ rel = 10 * np.log10(np.mean(10 ** (S / 10))) - 20+ S = S[S > rel]+ return float(np.percentile(S, 95) - np.percentile(S, 10)) if len(S) > 4 else None+++def onsets(m, sr=SR, delta=0.07):+ import librosa+ y = audio.resample(m, sr, 22050)+ o = librosa.onset.onset_detect(y=y, sr=22050, hop_length=256, units='time', backtrack=True, delta=delta)+ return [round(float(v), 3) for v in o]+++def key_estimate(m, sr=SR):+ import librosa+ y = audio.resample(m, sr, 22050)+ c = librosa.feature.chroma_cqt(y=y, sr=22050, hop_length=512).mean(axis=1)+ best = []+ for mode, prof in (('major', KK_MAJ), ('minor', KK_MIN)):+ for k in range(12):+ best.append((float(np.corrcoef(c, np.roll(prof, k))[0, 1]), f'{NOTE[k]} {mode}'))+ best.sort(reverse=True)+ return [{'key': b[1], 'r': round(b[0], 3)} for b in best[:3]]+++def brightness_hz(m, sr=SR):+ """median magnitude-weighted spectral centroid over frames within 30 dB of the loudest (perceived brightness)"""+ m = np.asarray(m, np.float32)+ if m.ndim == 2:+ m = m.mean(axis=1)+ if len(m) < 1024:+ return None+ f, t, Z = signal.stft(m, sr, nperseg=1024, noverlap=768, boundary=None)+ A = np.abs(Z)+ e = A.sum(axis=0)+ act = e >= e.max() * 10 ** (-30 / 20)+ if not act.any():+ return None+ c = (f[:, None] * A[:, act]).sum(axis=0) / (A[:, act].sum(axis=0) + 1e-20)+ return int(np.median(c))+++def measure(src, music=False, start=None, end=None, light=False):+ """objective measurements of a file path or a 48 kHz array; light=True skips the slow parts (onsets, HPSS, f0);+ a negative, reversed or out-of-file range raises ValueError"""+ if start is not None and start < 0:+ raise ValueError(f'start must be >= 0 s (got {start:g})')+ if end is not None and end <= (start or 0.0):+ raise ValueError(f'end ({end:g} s) must be greater than start ({start or 0.0:g} s)')+ if isinstance(src, str):+ if start:+ total = audio.probe(src).get('duration') or 0+ if total and start >= total:+ raise ValueError(f'start {start:g} s is at or beyond the end of {os.path.basename(src)} ({total:g} s long)')+ x = audio.load(src, start=start, end=end)+ else:+ x = _stereo(src)+ m = x.mean(axis=1)+ n = len(m)+ r = {'duration': round(n / SR, 3)}+ if n < int(0.005 * SR):+ r['error'] = 'too short to measure'+ return r+ pk = float(np.abs(x).max())+ r['peak_dbfs'] = round(db(pk), 1)+ r['true_peak_dbtp'] = round(true_peak_dbtp(x), 1)+ r['rms_dbfs'] = round(db(math.sqrt(float(np.mean(x.astype(np.float64) ** 2)))), 1)+ if n >= int(0.4 * SR):+ r['lufs'] = round(integrated_lufs(x), 1)+ t, S = loudness_curve(x, 3.0, 0.5) if n >= 3 * SR else (None, None)+ if S is not None:+ r['short_term_max_lufs'] = round(float(S.max()), 1)+ v = lra(x) if n >= 6 * SR else None+ if v is not None:+ r['lra_lu'] = round(v, 1)+ r['clipped_samples'] = int(np.sum(np.abs(x) >= 0.999))+ r['dc'] = round(float(np.mean(m)), 5)+ r20 = frame_rms(m, int(0.02 * SR))+ r['noise_floor_dbfs'] = round(db(np.percentile(r20, 10)), 1)+ r['dynamic_range_db'] = round(db(r20.max()) - r['noise_floor_dbfs'], 1)+ r['digital_silence_frac'] = round(float(np.mean(np.abs(m) < 1e-6)), 4)+ mid, side = (x[:, 0] + x[:, 1]) / 2, (x[:, 0] - x[:, 1]) / 2+ sm = float(np.sqrt(np.mean(side ** 2)))+ r['side_to_mid_db'] = round(db(sm) - db(np.sqrt(np.mean(mid ** 2)) + 1e-12), 1) if sm > 1e-7 else None+ if np.std(x[:, 0]) > 1e-9 and np.std(x[:, 1]) > 1e-9:+ r['correlation'] = round(float(np.corrcoef(x[:, 0], x[:, 1])[0, 1]), 3)+ r.update(envelope_times(m))+ # spectrum (whole file)+ seg = m.astype(np.float64)+ nfft = 1 << int(min(16, max(10, math.ceil(math.log2(max(1024, min(len(seg), 65536)))))))+ f, P = signal.welch(seg, SR, nperseg=min(nfft, len(seg)), noverlap=None)+ P = P + 1e-30+ tot = float(P.sum())+ r['centroid_hz'] = int(float((f * P).sum() / tot))+ c = np.cumsum(P) / tot+ r['rolloff95_hz'] = int(f[min(len(f) - 1, int(np.searchsorted(c, 0.95)))])+ r['bands_db'] = {k: round(10 * math.log10(float(P[(f >= a) & (f < b)].sum()) / tot + 1e-12), 1) for k, (a, b) in BANDS.items()}+ # bandwidth: highest third-octave band within 60 dB of the loudest band+ edges = 20 * 2 ** (np.arange(0, 31) / 3)+ lv = []+ for a, b in zip(edges[:-1], edges[1:]):+ s = P[(f >= a) & (f < b)]+ lv.append(10 * math.log10(float(s.mean())) if len(s) else -300)+ lv = np.array(lv)+ ok = np.where(lv >= lv.max() - 60)[0]+ r['bandwidth_hz'] = int(edges[ok[-1] + 1]) if len(ok) else None+ r['flatness'] = round(float(np.exp(np.mean(np.log(P))) / np.mean(P)), 4)+ r['brightness_hz'] = brightness_hz(m)+ r['tonal_peak_ratio_db'] = round(10 * math.log10(float(P.max() / np.median(P))), 1)+ if light:+ return r+ import librosa+ y22 = audio.resample(m, SR, 22050)+ yt = y22[: 30 * 22050] # tonality: the first 30 s are plenty and keep long files fast+ if len(yt) >= 2048:+ H, Pp = librosa.effects.hpss(yt)+ eh, ep = float(np.sum(H ** 2)), float(np.sum(Pp ** 2))+ r['harmonic_ratio'] = round(eh / (eh + ep + 1e-20), 3)+ try:+ f0 = librosa.yin(yt, fmin=50, fmax=2000, sr=22050, frame_length=2048)+ rms = librosa.feature.rms(y=yt, frame_length=2048, hop_length=512)[0][: len(f0)]+ voiced = f0[: len(rms)][rms > rms.max() * 0.1]+ if len(voiced) and r.get('harmonic_ratio', 0) >= 0.8:+ r['f0_hz'] = round(float(np.median(voiced)), 1)+ except Exception:+ pass+ on = onsets(m)+ r['onsets'] = on[:200]+ r['onset_count'] = len(on)+ r['onsets_per_s'] = round(len(on) / (n / SR), 2)+ if music and n >= 5 * SR:+ oenv = librosa.onset.onset_strength(y=y22, sr=22050, hop_length=256)+ tempo, beats = librosa.beat.beat_track(onset_envelope=oenv, sr=22050, hop_length=256, units='time')+ r['tempo'] = round(float(np.atleast_1d(tempo)[0]), 1)+ r['beats'] = [round(float(b), 3) for b in beats]+ if len(beats) > 4:+ r['beat_ibi_std_ms'] = round(float(np.std(np.diff(beats))) * 1000, 1)+ r['key'] = key_estimate(m)+ return r+++def _text(r, path):+ if r.get('error'):+ return f'{path}: {r["error"]}'+ if r.get('peak_dbfs', 0) < -70:+ what = 'digitally silent' if r.get('digital_silence_frac', 0) >= 0.999 else 'near-silent'+ return (f'{path}: {r["duration"]:.3f} s, {what} (peak {r["peak_dbfs"]:.0f} dBFS): nothing to measure; the loudness, '+ 'spectrum and envelope values of silence are meaningless.')+ b = r.get('bands_db') or {}+ parts = [f'{path}: {r["duration"]:.3f} s, peak {r["peak_dbfs"]} dBFS (true peak {r["true_peak_dbtp"]} dBTP)']+ if r.get('lufs') is not None:+ parts.append(f'{r["lufs"]} LUFS' + (f', LRA {r["lra_lu"]} LU' if r.get('lra_lu') is not None else ''))+ env = f'attack {r["attack_ms"]:g} ms' if r.get('attack_ms') is not None else ''+ env += (', ' if env else '') + (f'decay {r["decay_ms"]:g} ms' if r.get('decay_ms') is not None else+ 'no -30 dB decay (sustained or cut off)')+ parts.append(f'noise floor {r["noise_floor_dbfs"]} dBFS, {env}, '+ f'brightness {r.get("brightness_hz")} Hz, bandwidth {r.get("bandwidth_hz")} Hz')+ if r.get('clipped_samples'):+ parts.append(f'{r["clipped_samples"]} clipped samples')+ if r.get('tempo'):+ parts.append(f'tempo {r["tempo"]} BPM, key {r["key"][0]["key"] if r.get("key") else "?"}')+ if b:+ parts.append('bands ' + ' '.join(f'{k} {v}' for k, v in b.items()) + ' dB')+ return '; '.join(parts) + '.'+++def add_cli(sub):+ from .cli import common+ from .report import emit+ p = common(sub.add_parser('measure', help='objective measurements only (loudness, peaks, envelope, spectrum; no models)'))+ p.add_argument('files', nargs='+')+ p.add_argument('--music', action='store_true', help='also tempo, beats and key')+ p.add_argument('--light', action='store_true', help='skip the slow parts (onsets, HPSS, pitch)')+ p.add_argument('--start', type=float, help='measure only from this time (s)')+ p.add_argument('--end', type=float, help='measure only up to this time (s)')++ def run(a):+ res = []+ audio.check_paths(a.files, 'files')+ files = audio.list_media(a.files)+ if not files:+ raise audio.AudioError('no audio/video files found in: ' + ', '.join(a.files))+ for f in files:+ r = measure(f, music=a.music, start=a.start, end=a.end, light=a.light)+ r['file'] = f+ r['summary'] = _text(r, os.path.basename(f))+ res.append(r)+ out = res[0] if len(res) == 1 else {'summary': f'{len(res)} files measured.', 'files': res}+ emit(out, a, text='\n'.join(r['summary'] for r in res))+ return 0 if res else 1+ p.set_defaults(func=run)diff --git a/earshot/forge/__init__.py b/earshot/forge/__init__.pynew file mode 100644index 0000000..0deac4c--- /dev/null+++ b/earshot/forge/__init__.py@@ -0,0 +1,145 @@+"""earshot forge: an agent makes its own sounds in code and improves them by listening with earshot.++ earshot forge keypress --refs kit/keyboard --budget 160 --keep 4 --dir made/keypress+ earshot forge whoosh --target "a fast crisp air whoosh" --alts "a wind gust" "a sword swing" --refs kit/whoosh++Loop: a recipe (earshot/forge/recipes/<name>.py: physical synthesis written in code, no recorded audio) renders+candidates from parameters; the Critic scores each one (description match z, real-or-synthetic p_real, closeness to+real reference recordings, measured shape); an evolutionary search keeps the best and mutates them; repeat. The agent's+outer loop reads the critic's words ("sounds like: a wooden knock", "decay +2.4 outside the refs") and rewrites the+recipe itself. Output: the best diverse sounds as WAVs + report.json (scores, parameters, the whole history) + the same+critic's scores of the real references (leave-one-out, so a reference is not its own neighbour) and their held-out+AudioSet tags, so the gap to real is always visible.++Honest limits: the critic's models can be pleased without the sound getting better (Goodhart). earshot's realness model+is easy to fool with modal resonators, a room and a mic noise floor, so a high p_real proves little. Worse, once a+search has tuned against them, every model check saturates: the forge test's made sounds land inside or above the real+recordings on held-out phrasings, the AudioSet tagger and independent real sets too, while a plain 1/3-octave+measurement still finds them 8-13 dB off the real balance. So the critic always reports the model-free tone check+(earshot/forge/tone.py: tonal balance against the refs, the bands outside every ref, the tail), and `tone=W` (CLI+--tone W, 1.5 is a good start) puts it into the objective. Judge finished sounds by those measurements, robust+re-renders (`robust=N`), spectrograms and, last, human ears. AudioSet tags are a held-out check only while nobody picks+or steers by them: once they decide which candidate is kept, they are part of the objective.+"""+import importlib+import json+import os+import pkgutil++import numpy as np++from .. import audio+from ..report import emit+++def recipes():+ pkg = importlib.import_module(f'{__name__}.recipes') # not `from . import recipes`: this function shadows it+ return sorted(m.name for m in pkgutil.iter_modules(pkg.__path__) if not m.name.startswith('_'))+++def load_recipe(name):+ return importlib.import_module(f'earshot.forge.recipes.{name}')+++def forge(recipe, target=None, alts=None, refs=None, budget=160, pop=16, keep=4, seed=0, out=None, log=print,+ init=None, holdout=True, robust=0, space=None, sigma0=0.22, sigma1=0.04, critic=None, tone=0.0,+ tone_target=None, reps=1):+ """run the loop for one recipe; writes WAVs + report.json into `out` when given; returns the report dict.+ robust=N re-renders every pick on N new seeds (mean / min / pass rate in pick['robust']); space= a narrower+ {name: spec} family; sigma0/sigma1 the first / last mutation step; critic= a prebuilt Critic (reuse one across+ several forge() calls in a round script, or pass one with an `extra` check); tone= the weight of the model-free+ tonal-balance penalty (Critic tone=, tone_target=); reps= renders per candidate inside the search (ranked by+ their mean, against lucky single renders)"""+ import soundfile as sf+ from .critic import Critic+ from .search import diverse, optimize, render+ from .search import robust as robust_stats+ rec = load_recipe(recipe) if isinstance(recipe, str) else recipe+ name = rec.__name__.rsplit('.', 1)[-1]+ if critic is None:+ target = target or rec.TARGET+ alts = list(alts if alts is not None else getattr(rec, 'ALTS', []))+ critic = Critic(target, alts, refs, tone=tone, tone_target=tone_target)+ else:+ target, alts = critic.target, list(critic.alts)+ res = optimize(rec, critic, budget=budget, pop=pop, seed=seed, init=init, log=log, space=space, sigma0=sigma0,+ sigma1=sigma1, reps=reps)+ picks = diverse(res['best'] + sorted(res['history'], key=lambda h: -h['score']['total'])[:64], rec, keep)+ ref_scores = critic.score_refs(loo=True) if critic.refs else []+ report = {'recipe': name, 'target': target, 'alts': alts, 'refs': critic.refs, 'budget': budget, 'seed': seed,+ 'tone_weight': getattr(critic, 'tone', 0.0), 'tone_target': getattr(critic, 'tone_target', None),+ 'reps': reps,+ 'generations': res['generations'], 'picks': [], 'ref_scores': [], 'ref_scores_loo': True}+ if space is not None:+ report['space'] = space+ xs = [render(rec, h['params'], h['seed']) for h in picks]+ hold = critic.holdout([x for x in xs if x is not None]) if holdout and xs else []+ ref_hold = critic.holdout([audio.load(p) for p in critic.refs]) if holdout and critic.refs else []+ if out:+ os.makedirs(out, exist_ok=True)+ for i, (h, x) in enumerate(zip(picks, xs)):+ item = {'name': f'{name}_{i + 1:02d}', 'params': h['params'], 'seed': h['seed'],+ 'score': {k: v for k, v in h['score'].items() if k != 'i'}}+ if i < len(hold):+ item['holdout'] = hold[i]+ if robust:+ item['robust'] = robust_stats(rec, critic, h['params'], robust)+ if out and x is not None:+ item['file'] = os.path.join(out, item['name'] + '.wav').replace(chr(92), '/')+ sf.write(item['file'], x, 48000, subtype='PCM_24')+ report['picks'].append(item)+ for i, (p, s) in enumerate(zip(critic.refs, ref_scores)):+ report['ref_scores'].append({'file': os.path.basename(p), **{k: s.get(k) for k in+ ('total', 'passes', 'match_z', 'alt_margin', 'alt_zs', 'p_real', 'ref_typ_z',+ 'feat_pen', 'tone_db', 'tone_ratio', 'sounds_like')}})+ if i < len(ref_hold):+ report['ref_scores'][-1]['holdout'] = ref_hold[i]+ best_ref = max((r['total'] for r in report['ref_scores'] if r['total'] is not None and np.isfinite(r['total'])),+ default=None)+ n_pass = sum(1 for p in report['picks'] if p['score'].get('passes'))+ top = report['picks'][0]['score'] if report['picks'] else {}+ report['summary'] = (f"forge {name}: '{target}' - {n_pass} of {len(report['picks'])} kept sounds pass "+ f"(best total {top.get('total')}, match z {top.get('match_z')}, p_real {top.get('p_real')})"+ + (f"; the real references score up to {best_ref:.2f} with the same critic" if best_ref is not None+ else '') + ' (references scored leave-one-out).')+ report['history'] = [{'gen': h['gen'], 'params': h['params'], 'seed': h['seed'],+ 'score': {k: h['score'].get(k) for k in ('total', 'passes', 'match_z', 'alt_margin', 'p_real',+ 'ref_typ_z', 'feat_pen', 'tone_db', 'tone_ratio',+ 'sounds_like')}}+ for h in res['history']]+ if out:+ with open(os.path.join(out, 'report.json'), 'w', encoding='utf-8') as fh:+ json.dump(report, fh, indent=1, default=lambda o: o.tolist() if hasattr(o, 'tolist') else str(o))+ return report+++def add_cli(sub):+ from ..cli import common+ p = common(sub.add_parser('forge', help='make sounds in code and improve them by listening (recipe + critic + search)'))+ p.add_argument('recipe', nargs='?', help='recipe name (earshot/forge/recipes/); omit to list them')+ p.add_argument('--target', help="description to aim for (default: the recipe's)")+ p.add_argument('--alts', nargs='*', help='confusions it must sound less like')+ p.add_argument('--refs', nargs='*', help='real reference recordings (files / folders) to compare with')+ p.add_argument('--budget', type=int, default=160, help='candidates to render and score (default 160)')+ p.add_argument('--pop', type=int, default=16)+ p.add_argument('--keep', type=int, default=4, help='diverse best sounds to write')+ p.add_argument('--seed', type=int, default=0)+ p.add_argument('--dir', dest='outdir', help='folder for the WAVs + report.json')+ p.add_argument('--robust', type=int, default=0, help='re-render every kept sound on N new seeds and report the mean')+ p.add_argument('--tone', type=float, default=0.0,+ help='weight of the model-free tonal-balance penalty vs the refs (0 = report only; 1.5 is a good start)')+ p.add_argument('--reps', type=int, default=1,+ help='renders per candidate inside the search, ranked by their mean (2 = half as lucky, twice as slow)')+ p.add_argument('--tone-target', type=float, default=None,+ help="dB distance from the refs' median balance that counts as typical (default: the refs' own spread)")++ def run(a):+ if not a.recipe:+ print('recipes: ' + ', '.join(recipes()))+ return 0+ r = forge(a.recipe, a.target, a.alts, a.refs, a.budget, a.pop, a.keep, a.seed, a.outdir, robust=a.robust,+ tone=a.tone, tone_target=a.tone_target, reps=a.reps)+ lines = [r['summary']] + [f" {p['name']}: {p['score'].get('summary')}" for p in r['picks']]+ emit({k: v for k, v in r.items() if k != 'history'}, a, text='\n'.join(lines))+ return 0+ p.set_defaults(func=run)diff --git a/earshot/forge/critic.py b/earshot/forge/critic.pynew file mode 100644index 0000000..e24d4e5--- /dev/null+++ b/earshot/forge/critic.py@@ -0,0 +1,221 @@+"""The ears of the forge: scores sounds made in code against a description and real reference recordings.++ from earshot.forge.critic import Critic+ c = Critic('a single deep thocky mechanical keyboard key press', alts=['a wooden knock', 'a mouse click'],+ refs='refs/keyboard/')+ rows = c.score([x1, x2]) # 48 kHz arrays (mono or stereo)+ rows[0]['total'], rows[0]['passes'], rows[0]['summary']++One CLAP embedding per sound feeds every model judgement (the vocabulary calibration and the realness head use the+same padding), so a batch of 16 sounds costs about one batch of CLAP windows. Components:+ match_z calibrated z of the description among earshot's vocabulary (strong >= 3, as `earshot match`)+ alt_margin match_z minus the best alternative's z (must be > 0: it sounds more like the target than the confusions)+ p_real earshot realness (clip thresholds: real >= 0.77; the pure-tone rule applies)+ ref_typ_z how close it sits to the nearest real references compared with how close the references sit to each other+ feat_pen how far its measurements (attack, decay, brightness, length, low end, bandwidth) fall outside the+ references' spread (robust z beyond 1.5, averaged)+ total min(match_z, 6) + 2 * (2 * p_real - 1) + 0.5 * clip(ref_typ_z, -3, 2) - 0.7 * feat_pen+ - max(0, 0.5 - alt_margin)+ passes match_z >= 3 and alt_margin > 0 and p_real >= 0.77 and ref_typ_z >= -2 and feat_pen <= 1+Optimising against earshot can find sounds that please its models without sounding better (Goodhart). `holdout()`+uses the AudioSet tagger, which is not in the objective, but it stays held out only while nobody picks or steers by+its tags (in the forge test it did steer four of five sounds, which biased their tags). Once a search has run, the+model checks saturate: held-out phrasings and independent real sets agree too. The model-free `tone` check below is+the signal that still separates made from real. The realness+model in particular is easy to fool: modal resonators plus a room and a mic noise floor read "real" (p_real > 0.9)+whether or not they sound like the target, so trust reference closeness, the held-out tags and spectrograms more.++Fair comparison with the references: `score_refs()` scores every reference leave-one-out (its typicality and the+feature spread are computed from the OTHER references), so a real recording is not its own nearest neighbour. With+fewer than 3 other references there is no spread to compare with, and ref_typ_z is None. Each row also carries+`alt_zs` (the z of every alternative, so you can see which confusion is closest). `extra=fn(x, row) -> float` adds a+recipe-specific check (a model-free distance to the refs, say) to the total; its value is kept in row['extra'].++Model-free ears (`tone=`): every row also carries row['tone'] from earshot.forge.tone (1/3-octave tonal balance against+the refs' median, which bands fall outside every ref, the tail's t40), measured from the samples, no model. The model+checks above saturate once a search has tuned against them (made sounds land inside or above the real recordings on+every one, held-out ones included), while these measurements still show the gap. tone=W (default 0, the scores the+forge test reported) subtracts W * ToneRef.penalty() from the total (0 while the sound is no farther from the refs'+median balance than a typical real recording, then linear) and, with W > 0, `passes` also needs tone_db no larger+than the farthest real reference's. tone_target= sets that "typical" distance in dB (and the pass limit to 1.6x it,+the median : farthest proportion of the six real key presses) when the refs are too few or too different from each+other for their own spread to mean anything (the four kit pops sit 15-24 dB apart).+"""+import os++import numpy as np++from .. import audio, compare, features, realness, vocab+from .tone import ToneRef++FEATS = ('attack_ms', 'decay_ms', 'brightness_hz', 'length_s', 'low_end_db', 'bandwidth_hz')+LOG = ('attack_ms', 'decay_ms', 'brightness_hz', 'length_s', 'bandwidth_hz')+++def _mono(x):+ x = np.asarray(x, np.float32)+ return x.mean(axis=1) if x.ndim == 2 else x+++def _stereo(x):+ x = np.asarray(x, np.float32)+ return np.stack([x, x], axis=1) if x.ndim == 1 else x+++def _fv(k, v):+ return np.log(max(v, 1e-3)) if k in LOG else float(v)+++class Critic:+ def __init__(self, target, alts=(), refs=None, k=3, extra=None, tone=0.0, tone_target=None):+ self.target, self.alts = target, list(alts or [])+ self.extra = extra+ self.tone, self.tone_target = float(tone), tone_target+ self.v = vocab.get()+ self.Q = self.v.embed_text([target] + self.alts)+ self.m = realness.load_model()+ self.k = k+ self.refs = audio.list_media(refs, exts=audio.AUDIO_EXTS) if refs else []+ self.A_r, self.ref_feats, self.ref_stats, self.tone_ref = None, [], {}, None+ if self.refs:+ self.A_r = vocab.embed_files(self.refs)+ xs = []+ for p in self.refs:+ x = audio.load(p)+ xs.append(x)+ self.ref_feats.append(compare.feats(features.measure(x, light=True), x))+ self.ref_stats = self._stats(self.ref_feats)+ self.tone_ref = ToneRef(xs)++ @staticmethod+ def _stats(ref_feats):+ st = {}+ for key in FEATS:+ vals = np.array([_fv(key, f[key]) for f in ref_feats if f.get(key) is not None])+ if len(vals) >= 2:+ q1, med, q3 = np.percentile(vals, [25, 50, 75])+ st[key] = (float(med), max(float(q3 - q1) / 1.35, 0.15 if key in LOG else 1.5))+ return st++ def embed(self, xs, batch=16):+ from ..models import clap+ return clap.get().embed_audio([_mono(x) for x in xs], pad=vocab.PAD, batch=batch)++ def score(self, xs, E=None, exclude=None):+ """one result dict per sound (silent or broken sounds get total -inf). exclude: per sound, the index of a+ reference to leave out of its comparison (leave-one-out scoring of the references themselves) or None"""+ xs = list(xs)+ E = self.embed(xs) if E is None else E+ cal = self.v.calibrate(E, self.Q)+ hs = self.m['heads']+ rows = []+ exclude = list(exclude) if exclude is not None else [None] * len(xs)+ loo = {}+ for i, x in enumerate(xs):+ ex = exclude[i]+ if ex is None:+ A_r, refs, stats, min_refs = self.A_r, self.refs, self.ref_stats, 2+ else:+ if ex not in loo:+ keep = [j for j in range(len(self.refs)) if j != ex]+ loo[ex] = (self.A_r[keep] if self.A_r is not None else None, [self.refs[j] for j in keep],+ self._stats([self.ref_feats[j] for j in keep]))+ (A_r, refs, stats), min_refs = loo[ex], 3+ mono = _mono(x)+ row = {'i': i}+ if not len(mono) or not realness.audible(mono):+ rows.append({**row, 'total': -np.inf, 'passes': False, 'summary': 'silent'})+ continue+ z = cal['z_q'][i]+ mz = float(z[0])+ alt = float(z[1:].max()) if len(z) > 1 else -np.inf+ f = realness.signal_features(mono)+ lc, _ = realness.head_logit(hs['clap'], E[i])+ ls, _ = realness.head_logit(hs['signal'], realness._sig_vector(hs['signal'], f)) if f else (0.0, None)+ p, _ = realness.fuse(self.m['fusion'], {'clap': lc, 'signal': ls})+ if f and realness.pure_tone(f):+ p = min(p, 0.03)+ st = _stereo(x)+ fx = compare.feats(features.measure(st, light=True), st)+ pen, dev = 0.0, {}+ for key, (med, sd) in stats.items():+ if fx.get(key) is None:+ continue+ zz = (_fv(key, fx[key]) - med) / sd+ dev[key] = round(float(zz), 2)+ pen += max(0.0, abs(zz) - 1.5)+ pen = pen / max(1, len(dev))+ typ, near = None, None+ if A_r is not None and len(A_r) >= min_refs:+ _, typ, _ = compare._typicality(E[i], A_r, self.k)+ near = os.path.basename(refs[int(np.argmax(A_r @ E[i]))])+ margin = mz - alt if np.isfinite(alt) else 9.0+ total = (min(mz, 6.0) + 2.0 * (2 * p - 1) + (0.5 * float(np.clip(typ, -3, 2)) if typ is not None else 0.0)+ - 0.7 * pen - max(0.0, 0.5 - margin))+ passes = bool(mz >= 3 and margin > 0 and p >= 0.77 and (typ is None or typ >= -2) and pen <= 1.0)+ top = self.v.top_terms(E[i], k=3)+ row.update(match_z=round(mz, 2), alt_z=round(alt, 2) if np.isfinite(alt) else None,+ alt_zs={a: round(float(z[j + 1]), 2) for j, a in enumerate(self.alts)},+ alt_margin=round(margin, 2), p_real=round(float(p), 3),+ ref_typ_z=None if typ is None else round(float(typ), 2), nearest_ref=near,+ feat_pen=round(pen, 2), feat_dev=dev, feats={k: fx.get(k) for k in FEATS if fx.get(k) is not None},+ total=round(float(total), 3), passes=passes, sounds_like=[t[0] for t in top])+ if ex is not None:+ row['left_out'] = os.path.basename(self.refs[ex])+ if self.tone_ref is not None and self.tone_ref.n:+ tm = self.tone_ref.check(x, exclude=ex)+ if self.tone_target is not None:+ tm['tone_ratio'] = round(tm['tone_db'] / float(self.tone_target), 2)+ row['tone'] = {k: tm[k] for k in ('tone_db', 'tone_ratio', 'outside', 't40_ms', 't40_ref', 'length_s',+ 'length_ref', 'shares', 'shares_ref_med', 'words')}+ row['tone_db'], row['tone_ratio'] = tm['tone_db'], tm['tone_ratio']+ if self.tone > 0:+ tpen = self.tone_ref.penalty(tm)+ row['tone_pen'] = round(tpen, 3)+ row['total'] = round(row['total'] - self.tone * tpen, 3)+ far = max(tm['tone_ref']) if tm['tone_ref'] else None+ if self.tone_target is not None: # median : farthest of a typical real set is ~1 : 1.6+ far = 1.6 * float(self.tone_target)+ if far is not None and tm['tone_db'] > far:+ row['passes'] = False+ if self.extra is not None:+ adj = float(self.extra(x, row))+ row['extra'] = round(adj, 3)+ row['total'] = round(float(row['total']) + adj, 3)+ row['summary'] = self._summary(row)+ rows.append(row)+ return rows++ def _summary(self, r):+ bits = [f"match z {r['match_z']:+.2f}" + (f" (best alternative {r['alt_z']:+.2f})" if r['alt_z'] is not None else ''),+ f"p_real {r['p_real']:.2f}"]+ if r['ref_typ_z'] is not None:+ bits.append(f"typicality vs refs z {r['ref_typ_z']:+.2f} (nearest {r['nearest_ref']})")+ off = [f'{k} {v:+.1f}' for k, v in r['feat_dev'].items() if abs(v) > 1.5]+ if off:+ bits.append('outside the refs: ' + ', '.join(off))+ bits.append('sounds like: ' + ', '.join(r['sounds_like']))+ if r.get('tone'):+ bits.append(r['tone']['words'])+ return ('PASS ' if r['passes'] else '') + f"total {r['total']:.2f}: " + '; '.join(bits)++ def score_files(self, paths):+ return self.score([audio.load(p) for p in paths])++ def score_refs(self, loo=True):+ """the same critic's scores of the real references, in self.refs order. loo=True (fair): each reference is+ compared with the OTHER references only; loo=False counts it as its own nearest neighbour (inflated)"""+ xs = [audio.load(p) for p in self.refs]+ return self.score(xs, exclude=list(range(len(xs))) if loo else None)++ def holdout(self, xs):+ """checks that are NOT in the objective: AudioSet top labels and its synth-class probability (held out only+ while they decide nothing: fix the selection rule before reading them)"""+ from ..models import ast+ t = ast.get()+ P = t.probs([_mono(x) for x in xs])+ out = []+ for p in P:+ synth = max(t.prob_of(p, lab) for lab in realness.AST_SYNTH if lab in t.index)+ out.append({'audioset_top': t.top(p, 3), 'audioset_synth_max': round(float(synth), 3)})+ return outdiff --git a/earshot/forge/dsp.py b/earshot/forge/dsp.pynew file mode 100644index 0000000..196b1eb--- /dev/null+++ b/earshot/forge/dsp.py@@ -0,0 +1,269 @@+"""Building blocks for sounds written in code (48 kHz float). Physical first: real objects ring in modes, air moves+as filtered turbulence, every recording has a room and a microphone. Recipes in earshot/forge/recipes/ combine these.++ from earshot.forge import dsp+ r = dsp.rng(7)+ hit = dsp.modal([620, 1490, 2710], [0.035, 0.02, 0.012], [1, .5, .3], 0.3, r) # a small plastic body ringing+ x = dsp.excite(hit, dsp.contact(0.0015, r)) # struck for 1.5 ms+ x = dsp.mic(dsp.room(x, r, rt60=0.18, wet=0.08), r) # in a room, through a mic++No recorded audio is used anywhere: every sample is computed.+"""+import numpy as np+from scipy import signal++SR = 48000+++def rng(seed):+ return np.random.default_rng(int(seed) & 0xFFFFFFFF)+++def n_of(seconds):+ return max(1, int(round(seconds * SR)))+++def t_of(n):+ return np.arange(n) / SR+++# ------------------------------------------------------------------------------------------------ noise+def noise(seconds, r, color='white'):+ """white / pink (1/f) / brown (1/f^2) noise, unit RMS"""+ n = n_of(seconds)+ w = r.standard_normal(n)+ if color != 'white':+ F = np.fft.rfft(w)+ f = np.fft.rfftfreq(n, 1 / SR)+ f[0] = f[1]+ F /= f ** (0.5 if color == 'pink' else 1.0)+ w = np.fft.irfft(F, n)+ return (w / (np.sqrt(np.mean(w ** 2)) + 1e-12)).astype(np.float64)+++# ------------------------------------------------------------------------------------------------ envelopes+def env_exp(seconds, tau, attack=0.0005):+ """fast linear attack then exponential decay with time constant tau (s)"""+ n = n_of(seconds)+ t = t_of(n)+ e = np.exp(-t / max(tau, 1e-5))+ a = n_of(attack)+ if a > 1:+ e[:a] *= np.linspace(0, 1, a)+ return e+++def env_curve(points, seconds, shape='lin'):+ """piecewise envelope through [(t, value), ...]; shape 'lin' or 'exp' (interpolated in dB, values > 0)"""+ n = n_of(seconds)+ t = t_of(n)+ pts = sorted(points)+ tx, vx = [p[0] for p in pts], [p[1] for p in pts]+ if shape == 'exp':+ return 10 ** (np.interp(t, tx, 20 * np.log10(np.maximum(vx, 1e-6))) / 20)+ return np.interp(t, tx, vx)+++def contact(seconds, r, bright=0.7):+ """the excitation of a strike: a short raised-cosine force pulse plus a little contact noise (bright 0..1)"""+ n = max(2, n_of(seconds))+ pulse = 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, n))+ grit = r.standard_normal(n) * pulse * bright+ return pulse + grit+++# ------------------------------------------------------------------------------------------------ resonators+def modal(freqs, decays, amps, seconds, r, jitter=0.003, glide=0.0):+ """sum of exponentially damped sinusoids (a body's modes): freqs Hz, decays = time constants s, amps linear.+ jitter: random detune per mode (fraction); glide: pitch drop per mode over the decay (fraction, e.g. 0.01)"""+ n = n_of(seconds)+ t = t_of(n)+ out = np.zeros(n)+ for f, d, a in zip(freqs, decays, amps):+ if f >= SR / 2 * 0.95 or a == 0:+ continue+ f = f * (1 + jitter * r.standard_normal())+ inst = f * (1 - glide * (1 - np.exp(-t / max(d, 1e-4))))+ ph = 2 * np.pi * np.cumsum(inst) / SR + r.uniform(0, 2 * np.pi)+ out += a * np.exp(-t / max(d, 1e-5)) * np.sin(ph)+ return out+++def modal_cloud(f_lo, f_hi, count, decay_lo, decay_hi, seconds, r, tilt=-3.0):+ """many modes spread log-uniformly between f_lo and f_hi (dense bodies: plastic shells, wood, cases); higher modes+ decay faster (decay interpolated log from decay_lo at f_lo to decay_hi at f_hi), tilt dB/octave"""+ f = np.sort(np.exp(r.uniform(np.log(f_lo), np.log(f_hi), int(count))))+ u = np.log(f / f_lo) / max(1e-9, np.log(f_hi / f_lo))+ d = np.exp(np.log(decay_lo) + u * (np.log(decay_hi) - np.log(decay_lo)))+ a = 10 ** (tilt * np.log2(f / f_lo) / 20) * r.uniform(0.3, 1.0, len(f))+ return modal(f, d, a, seconds, r, jitter=0.0)+++def excite(resonance, excitation):+ """drive a resonator response (its impulse response) with an excitation signal"""+ y = signal.fftconvolve(excitation, resonance)[: len(resonance)]+ return y+++# ------------------------------------------------------------------------------------------------ filters+def butter(x, kind, f, order=2):+ """kind 'low' | 'high' | 'band' (f = (lo, hi)) | 'bandstop'"""+ f = np.clip(f, 10, SR / 2 * 0.95)+ sos = signal.butter(order, f, kind if kind != 'band' else 'bandpass', fs=SR, output='sos')+ return signal.sosfilt(sos, x, axis=0)+++def peq(x, f0, gain_db, q=0.9):+ A = 10 ** (gain_db / 40)+ w0 = 2 * np.pi * min(f0, SR / 2 * 0.95) / SR+ al = np.sin(w0) / (2 * q)+ c = np.cos(w0)+ b = np.array([1 + al * A, -2 * c, 1 - al * A])+ a = np.array([1 + al / A, -2 * c, 1 - al / A])+ return signal.lfilter(b / a[0], a / a[0], x, axis=0)+++def svf(x, cutoff, q=0.7, mode='band'):+ """time-varying state-variable filter (Cytomic TPT): cutoff and q may be arrays (one value per sample); mode+ 'low' | 'band' | 'high' | 'bandn'. Use it for moving resonances: whooshes, swells, sweeps.+ Gains: 'low' and 'high' are 1 in their pass bands (q above 0.707 adds a peak of about q at the cutoff); 'band' peaks+ at q (not 1) at the cutoff, so a q of 8 is +18 dB; 'bandn' is the same band-pass normalised to a peak gain of 1"""+ n = len(x)+ fc = np.broadcast_to(np.clip(np.asarray(cutoff, float), 20, SR * 0.45), (n,))+ qq = np.broadcast_to(np.asarray(q, float), (n,))+ return _svf(np.asarray(x, np.float64), np.ascontiguousarray(fc), np.ascontiguousarray(qq),+ {'low': 0, 'band': 1, 'high': 2, 'bandn': 3}[mode])+++try:+ from numba import njit+except Exception: # pragma: no cover+ def njit(*a, **k):+ return (lambda f: f) if not (a and callable(a[0])) else a[0]+++@njit(cache=True)+def _svf(x, fc, q, mode):+ y = np.empty_like(x)+ ic1 = 0.0+ ic2 = 0.0+ for i in range(len(x)):+ g = np.tan(np.pi * fc[i] / 48000.0)+ k = 1.0 / max(q[i], 0.05)+ a1 = 1.0 / (1.0 + g * (g + k))+ a2 = g * a1+ a3 = g * a2+ v3 = x[i] - ic2+ v1 = a1 * ic1 + a2 * v3+ v2 = ic2 + a2 * ic1 + a3 * v3+ ic1 = 2.0 * v1 - ic1+ ic2 = 2.0 * v2 - ic2+ if mode == 0:+ y[i] = v2+ elif mode == 1:+ y[i] = v1+ elif mode == 3:+ y[i] = k * v1+ else:+ y[i] = x[i] - k * v1 - v2+ return y+++# ------------------------------------------------------------------------------------------------ space + mic+def room_ir(r, rt60=0.25, size=0.5, damp=0.6, seconds=None):+ """a small procedural room impulse response: a few early reflections (size 0..1 sets their spread) and an+ exponentially decaying noise tail whose highs die faster (damp 0..1)"""+ seconds = seconds or min(2.0, rt60 * 1.2 + 0.05)+ n = n_of(seconds)+ t = t_of(n)+ tail = r.standard_normal(n) * np.exp(-6.91 * t / max(rt60, 0.02))+ lo = butter(tail, 'low', 1500 + (1 - damp) * 9000, order=1)+ mix = np.interp(t, [0, seconds], [0.0, 1.0]) ** 0.7+ tail = (1 - mix * damp) * tail + mix * damp * lo+ ir = tail * 0.3+ ir[0] += 1.0+ for _ in range(6):+ d = int(r.uniform(0.002, 0.004 + 0.02 * size) * SR)+ if d < n:+ ir[d] += r.uniform(0.15, 0.45) * r.choice([-1, 1])+ return ir+++def room(x, r, rt60=0.25, size=0.5, damp=0.6, wet=0.12):+ """place x in a small room (wet 0..1 = reverb share); output is as long as x plus the tail. x may be mono (n,)+ or stereo (n, 2); a stereo x gets the same room on both channels (for a wide room, build one IR per channel)"""+ ir = room_ir(r, rt60, size, damp)+ ir[0] = 0.0+ x = np.asarray(x, np.float64)+ if x.ndim == 2:+ y = signal.fftconvolve(x, ir[:, None], axes=0)+ out = np.zeros(y.shape)+ out[: len(x)] += x+ return out + wet * y / (np.sqrt(np.sum(ir ** 2)) + 1e-9)+ y = signal.fftconvolve(x, ir)+ out = np.zeros(len(y))+ out[: len(x)] += x+ return out + wet * y / (np.sqrt(np.sum(ir ** 2)) + 1e-9)+++def mic(x, r, low_cut=35.0, air_db=-2.0, floor_db=-66.0, color=None):+ """a microphone and preamp: gentle low cut, a slightly soft top (air_db at 10 kHz), an optional presence colour+ (color=(f0, gain_db)), and a pink noise floor at floor_db relative to the peak (real recordings are never silent).+ x may be mono (n,) or stereo (n, 2); a stereo x gets an independent noise floor per channel"""+ y = butter(x, 'high', low_cut, order=1)+ y = peq(y, 10000, air_db, 0.6)+ if color:+ y = peq(y, color[0], color[1], 0.8)+ pk = np.abs(y).max() + 1e-12+ if y.ndim == 2:+ fl = np.stack([noise(len(y) / SR, r, 'pink')[:len(y)] for _ in range(y.shape[1])], axis=1)+ return y + fl * pk * 10 ** (floor_db / 20)+ y = y + noise(len(y) / SR, r, 'pink') * pk * 10 ** (floor_db / 20)+ return y+++def stereo(x, r, width=0.3, pan=0.0):+ """mono -> stereo with a decorrelated side (two short random all-pass-like delays) and constant-power pan"""+ d1, d2 = int(r.uniform(0.2, 0.9) * 1e-3 * SR), int(r.uniform(0.2, 0.9) * 1e-3 * SR)+ side = np.zeros_like(x)+ side[d1:] += x[: len(x) - d1]+ side[d2:] -= x[: len(x) - d2]+ side = butter(side, 'high', 300)+ L, R = x + width * side, x - width * side+ th = (np.clip(pan, -1, 1) + 1) * np.pi / 4+ return np.stack([L * np.cos(th) * np.sqrt(2), R * np.sin(th) * np.sqrt(2)], axis=1)+++# ------------------------------------------------------------------------------------------------ finishing+def fade(x, fin=0.001, fout=0.02):+ x = np.array(x, dtype=np.float64)+ a, b = min(len(x), n_of(fin)), min(len(x), n_of(fout))+ if a > 1:+ w = 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, a))+ x[:a] *= w if x.ndim == 1 else w[:, None]+ if b > 1:+ w = 0.5 + 0.5 * np.cos(np.linspace(0, np.pi, b))+ x[-b:] *= w if x.ndim == 1 else w[:, None]+ return x+++def place(dst, x, t, gain=1.0):+ """add x into dst starting at t seconds (both mono or both stereo)"""+ i = n_of(t) if t > 0 else 0+ if i >= len(dst):+ return dst+ m = min(len(x), len(dst) - i)+ dst[i:i + m] += gain * x[:m]+ return dst+++def trim_tail(x, floor_db=-70.0, keep=0.01):+ """cut trailing samples quieter than floor_db (relative to the peak), keeping `keep` s after the last loud one"""+ m = np.abs(x if x.ndim == 1 else x.max(axis=1))+ loud = np.nonzero(m >= m.max() * 10 ** (floor_db / 20))[0]+ end = min(len(x), (loud[-1] if len(loud) else len(x)) + n_of(keep))+ return x[:end]+++def normalize(x, peak_db=-1.0):+ return x * (10 ** (peak_db / 20) / (np.abs(x).max() + 1e-12))diff --git a/earshot/forge/recipes/__init__.py b/earshot/forge/recipes/__init__.pynew file mode 100644index 0000000..a4f6335--- /dev/null+++ b/earshot/forge/recipes/__init__.py@@ -0,0 +1,5 @@+"""Sound recipes for earshot forge: one module per sound type (SPACE, DEFAULTS, TARGET, ALTS, render)."""+import os++# where the recipes' default real reference recordings live (critic references only; pass --refs to use your own)+REF_ROOT = os.environ.get('EARSHOT_REFS', 'refs').replace(chr(92), '/').rstrip('/')diff --git a/earshot/forge/recipes/click.py b/earshot/forge/recipes/click.pynew file mode 100644index 0000000..26fb01e--- /dev/null+++ b/earshot/forge/recipes/click.py@@ -0,0 +1,202 @@+"""A soft premium UI button click, built from physics (final: the round-14 version).++What a real button click is, physically:+ 1. the fingertip lands on the cap: a soft, slow (2-10 ms) force pulse into a heavily damped low mass-spring (finger+ pad + cap + travel) -> a dull low "thump" you mostly feel, heard as a low pressure bump; the actuator touching+ the dome / latch often makes a tiny precursor tick 1-6 ms before the snap;+ 2. the dome / latch snaps: the force changes almost instantly (tens to hundreds of microseconds) -> a broadband+ impulse. A small sound source radiates pressure proportional to its acceleration, so the snap itself reaches the+ mic as a sharp click, and it rings the light hollow plastic shell. Close-up recordings of real clicks show ONE+ dominant, very damped shell mode (2-7 kHz, a handful of cycles: gone in 0.2-2 ms) over weaker modes, and the+ case / housing adds a few mid modes that are also gone within a few ms (a long-ringing ~1 kHz housing reads+ as a wooden block, not a button). A small object radiates low frequencies poorly (6 dB/oct below a+ size-dependent corner), so the click is light in the low end unless a finger thump is part of it;+ 3. the contact chatters: the latch / dome micro-bounces once to three times within a few milliseconds (each a+ smaller impulse on the same bodies), and a millisecond or so later the stem bottoms out: a softer contact;+ lossy plastic also has a dense crowd of high, very damped modes that read as a short grainy texture, not tones;+ 4. optionally the release: 20-90 ms later the dome springs back, a thinner, quieter click;+ 5. friction and spring rattle leave a short band of noise; a small room and a close microphone (low cut, soft top)+ frame it, and a quiet click recorded close is never far above the preamp hiss: a broadband floor (white to pink)+ only 40-80 dB under the peak, with some low room tone (rumble) under it.+Two ears disagree here: the critic's description match prefers a duller, thumpier, tonal sound with a precursor tick,+a loud release and more room, while a held-out model (AudioSet) and the measured spectra put the real clicks+elsewhere (dense damped shell modes, a strong direct pulse and bottom-out, little room, quiet release). The ranges+below keep the search inside the region where the real recordings live.+(Round 9 tried a statistical body - 60-220 overlapping modes with a smooth envelope - instead of the hand-placed shell+and housing modes: smoother spectra, but the held-out tagger heard it as "crunch" and the critic liked it less, so+round 10 returns to this structure.) The refs' envelopes fall fast to about -40 dB and then hold a slow, bright tail+(small reflective room / case ring-down) for 50-150 ms, so the room may be wetter than the tagger alone suggested.+Round 10 measured close to the refs in envelope and tail balance, but its shell modes still drew clean lines+lasting 30-40 ms (heard as "coin dropping" / "ice cubes in a glass") and its finger thump sat at 60-250 Hz, where the+refs are quiet, leaving a hole at 300-600 Hz where they are not: round 11 damps the shell harder and stiffens the+finger / cap contact. Round 12 also lets the small-source radiation corner act on the finger thump (the cap+radiates it), which takes the remaining excess out of 50-250 Hz.+Round 13 passed the critic, but a diagnostic re-render showed WHY: its release click (40-45 ms, -18 dB) is what the+description match rewards ("a soft key release click"), and the same release made the held-out tagger place the+sounds nearer real mechanical keyboard presses than real UI clicks. The refs' second events come at 25-35 ms and+-25 to -30 dB, or not at all, so round 14 keeps the release early and quiet. Round 14 then measured as close to the+refs as the refs are to each other on every held-out check except the tail (-46 to -51 dB at 30-80 ms where the refs+sit near -40 dB for 50-150 ms). Round 15 forced more room and longer clips: the tails barely rose and the critic+dropped, so these round-14 ranges are the final ones (the finals in demo/forge-test/click/final come from them).+Real close-up clicks are BRIGHT and broadband: their first 30 ms have a plateau from 2 to 10 kHz and little below+150 Hz; a dull top, a strong sub thump or a single peaky mid mode is what made earlier rounds read as a key press, a+wooden block or a finger snap.+Every sample is computed; no recorded audio is used.+"""+import numpy as np++from .. import dsp+from . import REF_ROOT++TARGET = 'a soft premium UI button click'+ALTS = ['a mechanical keyboard key press', 'a wooden knock', 'a camera shutter click', 'a finger snap']+_K = REF_ROOT + '/ui/'+REFS = [_K + f for f in ('ui_click_01.wav', 'ui_click_02.wav', 'ui_click_03.wav', 'ui_click_04.wav',+ 'ui_toggle_01.wav', 'ui_toggle_02.wav')]++SPACE = {+ 'thump_amp': (0.003, 0.12, 'log'), # fingertip landing level (relative to the snap)+ 'thump_f': (250, 600, 'log'), # finger pad on a small hard cap: k ~ 1e4 N/m, m ~ 2 g -> ~350 Hz+ 'thump_w': (0.0015, 0.005, 'log'), # fingertip force pulse width+ 'pre': (0.001, 0.012, 'log'), # fingertip landing -> snap+ 'snap_w': (0.00002, 0.00012, 'log'), # snap force pulse width: a raised cosine of width w has its first+ # spectral null at 2/w, so > 0.12 ms leaves the 6-12 kHz plateau dull+ 'snap_bright': (0.0, 0.5), # contact grit in the snap+ 'direct': (0.4, 3.0, 'log'), # directly radiated pulse vs body ringing+ 'main_f': (2500, 8000, 'log'), # the dominant, very damped shell mode+ 'main_decay': (0.00015, 0.0008, 'log'),+ 'main_amp': (0.3, 4.0, 'log'), # its level against the other shell modes+ 'tick_lvl': (0.0, 0.08), # precursor tick (actuator meets the dome)+ 'tick_dt': (0.001, 0.006, 'log'), # ... this long before the snap+ 'radiate_hz': (60, 1500, 'log'), # small-source radiation corner (6 dB/oct below)+ 'shell_f': (2000, 8000, 'log'), # lowest of the other shell modes+ 'shell_n': (7, 14, 'int'), # how many shell modes+ 'shell_decay': (0.0004, 0.0025, 'log'),+ 'grain': (0.05, 1.2), # dense, very damped high mode crowd (lossy plastic texture)+ 'body_f': (400, 2500, 'log'), # housing modes+ 'body_decay': (0.0005, 0.003, 'log'),+ 'body_amp': (0.02, 0.5, 'log'),+ 'chatter': (0, 2, 'int'), # micro-bounces of the latch after the snap+ 'chatter_dt': (0.0006, 0.003, 'log'),+ 'bottom_dt': (0.0003, 0.0015, 'log'),+ 'bottom_lvl': (0.2, 0.6),+ 'rel_dt': (0.02, 0.036),+ 'rel_lvl': (0.0, 0.05),+ 'tail_lvl': (0.005, 0.2, 'log'), # friction / spring rattle / case ring-down noise+ 'tail_tau': (0.004, 0.08, 'log'),+ 'tail_f': (2000, 10000, 'log'),+ 'tail_bw': (2.0, 6.0, 'log'), # tail band half-width (ratio)+ 'room_wet': (0.0, 0.3),+ 'room_damp': (0.1, 0.8),+ 'rt60': (0.05, 0.35, 'log'),+ 'floor_db': (-72.0, -38.0), # preamp hiss under the peak+ 'hiss_tilt': (0.0, 1.0), # 0 white .. 1 pink+ 'rumble_db': (-80.0, -45.0), # room tone: low rumble (traffic, HVAC) under the peak+ 'clip_len': (0.07, 0.16), # the edited clip: room tone runs on to here, then a short fade+}+DEFAULTS = {'main_f': 4500, 'main_decay': 0.0006, 'main_amp': 2.5, 'tick_lvl': 0.08, 'tick_dt': 0.003, 'radiate_hz': 300,+ 'tail_bw': 1.8, 'room_damp': 0.6, 'thump_amp': 0.15, 'thump_f': 140, 'thump_w': 0.003, 'pre': 0.0015, 'snap_w': 0.0002, 'snap_bright': 0.5,+ 'direct': 0.8, 'shell_f': 4200, 'shell_n': 10, 'shell_decay': 0.0018, 'grain': 0.5, 'body_f': 1500,+ 'body_decay': 0.002, 'body_amp': 0.3, 'chatter': 1, 'chatter_dt': 0.0008, 'bottom_dt': 0.0015,+ 'bottom_lvl': 0.5, 'rel_dt': 0.035, 'rel_lvl': 0.12, 'tail_lvl': 0.03, 'tail_tau': 0.012, 'tail_f': 1500,+ 'room_wet': 0.1, 'rt60': 0.15, 'floor_db': -66.0, 'hiss_tilt': 0.5, 'rumble_db': -60.0, 'clip_len': 0.12}++PRE_ROLL = 0.003+++def _pulse(width, r, bright=0.0):+ """a force pulse: raised cosine of `width` s, with optional contact grit (micro-roughness of the surfaces)"""+ n = max(2, dsp.n_of(width))+ p = 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, n))+ if bright > 0:+ p = p + bright * r.standard_normal(n) * p+ return p / (p.sum() + 1e-12) * n # unit-ish peak, energy scales with width+++def _bodies(p, r, f_scale=1.0, seconds=0.12):+ """impulse response of the shell (bright, few, damped modes) + housing (low-mid modes)"""+ n = int(p['shell_n'])+ # plastic shell: modes roughly following a plate-like spacing, higher ones die faster+ ratios = np.array([1.0, 1.52, 2.03, 2.61, 3.18, 3.9, 4.57, 5.3, 6.1, 6.9, 7.8, 8.7, 9.7, 10.8])[:n]+ ratios = ratios * np.exp(0.06 * r.standard_normal(n))+ fs = p['shell_f'] * f_scale * ratios+ dec = p['shell_decay'] * ratios ** -0.7+ amps = r.uniform(0.4, 1.0, n) * ratios ** -0.5+ shell = dsp.modal(fs, dec, amps, seconds, r, jitter=0.0)+ main = dsp.modal([p['main_f'] * f_scale], [p['main_decay']], [1.0], seconds, r, jitter=0.01)+ shell = shell / (np.abs(shell).max() + 1e-12) + p['main_amp'] * main+ hf = p['body_f'] * f_scale * np.array([1.0, 1.71, 2.47])+ body = dsp.modal(hf, p['body_decay'] * np.array([1.0, 0.6, 0.4]), np.array([1.0, 0.5, 0.3]) * p['body_amp'],+ seconds, r)+ out = shell + body * (np.abs(shell).max() / (np.abs(body).max() + 1e-12)) * min(1.0, p['body_amp'])+ if p['grain'] > 0:+ cloud = dsp.modal_cloud(p['shell_f'] * f_scale * 0.8, 16000, 40, 0.0015, 0.0003, seconds, r, tilt=-2.0)+ out = out + p['grain'] * cloud / (np.abs(cloud).max() + 1e-12) * np.abs(shell).max()+ return dsp.butter(out, 'high', p['radiate_hz'], order=1) # small source: poor low-frequency radiation+++def _snap(p, r, level=1.0, f_scale=1.0, width_scale=1.0):+ """the snap + bottom-out on the bodies, plus the directly radiated pulse"""+ seconds = 0.12+ ir = _bodies(p, r, f_scale, seconds)+ exc = _pulse(p['snap_w'] * width_scale, r, p['snap_bright'])+ x = dsp.excite(ir, exc)+ x = x / (np.abs(x).max() + 1e-12)+ d = np.zeros_like(x)+ d[:len(exc)] += exc / (np.abs(exc).max() + 1e-12)+ d = dsp.butter(d, 'high', max(250, p['radiate_hz']), order=1) # small source: pressure ~ acceleration+ out = x + p['direct'] * d+ t, lvl = 0.0, 1.0+ for _ in range(int(p['chatter'])): # micro-bounces: each later and weaker than the last+ t += p['chatter_dt'] * r.uniform(0.6, 1.4)+ lvl *= r.uniform(0.25, 0.6)+ dsp.place(out, lvl * x, t)+ dsp.place(out, lvl * p['direct'] * d, t)+ if p['bottom_lvl'] > 0:+ exc2 = _pulse(p['snap_w'] * width_scale * 2.5, r, p['snap_bright'] * 0.5)+ y = dsp.excite(_bodies(p, r, f_scale * 0.92, seconds), exc2)+ y = y / (np.abs(y).max() + 1e-12)+ dsp.place(out, p['bottom_lvl'] * y, p['bottom_dt'])+ return level * out+++def _thump(p, r):+ """fingertip landing: slow force pulse into a heavily damped low mass-spring, heard as a low pressure bump"""+ exc = _pulse(p['thump_w'], r, 0.0)+ ir = dsp.modal([p['thump_f']], [1.0 / (np.pi * p['thump_f'] * 1.2)], [1.0], 0.06, r, jitter=0.0)+ y = dsp.excite(ir, exc)+ return y / (np.abs(y).max() + 1e-12)+++def _tail(p, r):+ """friction and spring rattle after the snap: band-passed noise with an exponential decay"""+ sec = min(0.25, p['tail_tau'] * 7)+ nz = dsp.noise(sec, r, 'white') * dsp.env_exp(sec, p['tail_tau'], 0.0003)+ lo, hi = p['tail_f'] / p['tail_bw'], min(21000, p['tail_f'] * p['tail_bw'])+ return dsp.butter(nz, 'band', (lo, hi), order=2)+++def render(p, r):+ x = np.zeros(dsp.n_of(0.35))+ t0 = PRE_ROLL + p['pre']+ th = dsp.butter(_thump(p, r), 'high', p['radiate_hz'], order=1) # the cap is a small radiator too+ dsp.place(x, p['thump_amp'] * th / (np.abs(th).max() + 1e-12), PRE_ROLL)+ if p['tick_lvl'] > 0.005:+ dsp.place(x, _snap(dict(p, chatter=0, bottom_lvl=0.0), r, p['tick_lvl'], f_scale=1.07, width_scale=1.5),+ max(0.0, t0 - p['tick_dt']))+ s = _snap(p, r)+ dsp.place(x, s, t0)+ dsp.place(x, p['tail_lvl'] * _tail(p, r), t0)+ if p['rel_lvl'] > 0.01:+ dsp.place(x, _snap(p, r, p['rel_lvl'], f_scale=1.15, width_scale=0.7), t0 + p['rel_dt'])+ x = dsp.room(x, r, rt60=p['rt60'], size=0.2, damp=p['room_damp'], wet=p['room_wet'])+ x = dsp.trim_tail(x, -60, 0.02) # the sound ends where it falls under the floor ...+ if len(x) < dsp.n_of(p['clip_len']): # ... but the recording (room tone, hiss) runs on to the edit point+ x = np.pad(x, (0, dsp.n_of(p['clip_len']) - len(x)))+ x = dsp.mic(x, r, low_cut=40.0, air_db=-1.0, floor_db=-120.0)+ pk = np.abs(x).max() # ... then the preamp adds its own hiss+ h = (1 - p['hiss_tilt']) * dsp.noise(len(x) / dsp.SR, r, 'white') + p['hiss_tilt'] * dsp.noise(len(x) / dsp.SR, r, 'pink')+ x = x + h / (np.sqrt(np.mean(h ** 2)) + 1e-12) * pk * 10 ** (p['floor_db'] / 20)+ rb = dsp.butter(dsp.noise(len(x) / dsp.SR, r, 'brown'), 'band', (30, 220), order=2)+ x = x + rb / (np.sqrt(np.mean(rb ** 2)) + 1e-12) * pk * 10 ** (p['rumble_db'] / 20)+ return dsp.fade(x, 0.0003, 0.02)diff --git a/earshot/forge/recipes/impact.py b/earshot/forge/recipes/impact.pynew file mode 100644index 0000000..0735793--- /dev/null+++ b/earshot/forge/recipes/impact.py@@ -0,0 +1,523 @@+"""A deep cinematic boom impact (trailer hit), built from physics and studio practice; every sample is computed.++What a trailer boom is, physically:+ * a big, slack drum-like body (a large membrane / a heavy panel) struck hard: its fundamental starts high and SLIDES+ DOWN as the amplitude dies (tension modulation: a big displacement stretches the skin and raises the pitch; as it+ rings down the pitch settles), with a few inharmonic membrane overtones (Bessel ratios 1, 1.59, 2.14, 2.30 ...)+ that die faster than the fundamental;+ * the contact: a broadband crack of a few ms to a few tens of ms (the striker, the air shock, splintering), much+ quieter than the body but carrying the bandwidth;+ * the rest of the struck structure (drum shell, stage, a big panel): a dense cloud of low-mid modes, 70 Hz to ~1.5 kHz,+ the higher ones dying faster;+ * a sub layer: the long low tail (a sine at 30-60 Hz with a slow decay) that you feel more than hear;+ * rumble: low turbulent energy (brown noise through a falling low-pass) from the structure and the air settling;+ * a whoosh-down: air displaced by the hit, heard as resonant noise whose band sweeps down after the contact;+ * a flanged tail: trailer hits are often processed with a comb whose delay drifts (a jet flanger on the reverb):+ the comb's peaks at k / delay slide down together as the delay grows, a fan of descending partials in the tail;+ * distant echoes: the boom comes back from far walls / buildings, later, quieter and darker each time;+ * a large hall: frequency-dependent decay (lows ring longest, highs die early), diffuse early reflections,+ pre-delay, a decorrelated stereo tail;+ * a reverse swell into the hit (the trailer "suck-in"): a broadband hall tail of the contact computed, reversed,+ filtered and faded in so it rises into the contact;+ * an air riser before it (air pulled in: band-passed noise rising in pitch and level into the contact);+ * console saturation that only bites on the loud part (tanh, small signals pass clean) followed by the console's+ high-frequency roll-off, then a microphone with a noise floor.+No recorded audio is loaded, sampled or convolved: the impulse responses are computed noise.++History (the forge agent's listening notes): round 1 had every layer but the highs hung on as a flat broadband+curtain for seconds (hard-clipped sub = square-wave harmonics), the swell was a dark low blob, bandwidth was ~2 kHz+against 5-16 kHz in the refs. Round 2: the low layers are rendered at 6 kHz and up-sampled (speed), saturation is+followed by a roll-off, the swell is broadband, echoes were added, the hall's early reflections are diffuse.+Round 3: the crack's single noise burst printed static stripes through the hall tail (its speckled spectrum) -> a+smooth contact pulse + a micro-impulse crackle; a flanged tail (time-varying feedback comb) for the descending+partial lines heard and seen in the refs' tails; low bounds extended where the search sat on them.+Round 4: the stripes were still there (the impulse cloud was only a few ms long) and a dark line sat at 6 kHz (the+hall's crossover notch): subtractive hall bands; contact = smooth pulse + damped striker mode + debris spread over+30-500 ms; the direct sound stays centred and the stereo comes from the hall and from diffuse layers rendered+partly independent per channel (the old mid/side widener combed each channel).+Round 5: the tails were a near-pure 38 Hz tone (the <50 Hz band held the energy, 200-500 Hz sat 10 dB below the refs)+and the striker mode (up to 9 ms) printed a thin line at ~2 kHz through the hall: a low-mid shell mode cloud, striker+damping limited to 3 ms.+Round 6: the swells stayed dark low blobs (the search sat on pre_bright's lower bound) while refs 2 and 3 rise into+the hit with a broadband air swell: an air riser (rising band-passed noise), pre_bright allowed lower.+Round 7: the search kept pushing the debris burst to its 30 ms floor with the striker mode at its 3 ms ceiling,+which brings faint static stripes back into the tail (the critic likes them; the refs have none): debris >= 60 ms.+Round 8 (launch film, after a human blind test): of 15 forged sounds the listener could pick out only the impact, and+the reason was its tail: it falls -22 dB by 2 s and dies, while real booms keep rolling at about -10..-14 dB for 2-3 s+before they fall (kit impact_03 holds within -6 dB for 2.5 s). A new layer, the ROLL: the low energy of a big space+settling after a hit (distant reflections arriving and overlapping, the structure and the air still moving), brown+noise under a low-pass with a slow, irregular swell-and-recede (a roll, as thunder rolls), building over ~0.15 s,+holding for roll_hold s with a slow decline, then falling (roll_fall). It is diffuse (partly independent per+channel). It draws its noise only when roll_level > 0 and after every other layer, so the round-1-7 finals re-render+sample-exact from their parameters.+"""+import numpy as np+from scipy import signal++from .. import dsp+from . import REF_ROOT++SR = dsp.SR+LO = 6000 # sample rate of the low layers (body, sub, rumble), up-sampled x8+TARGET = 'a deep cinematic boom impact'+ALTS = ['a kick drum', 'an explosion', 'a door slam', 'thunder']+REFS = [f'{REF_ROOT}/impact/impact_0{i}.wav' for i in (1, 2, 3)]++MEMBRANE = [1.0, 1.594, 2.136, 2.296, 2.653, 2.918]++SPACE = {+ # reverse swell into the hit+ 'pre_len': (0.08, 1.6, 'log'),+ 'pre_level': (0.02, 1.0, 'log'),+ 'pre_bright': (200, 16000, 'log'),+ 'pre_low': (0.0, 1.0),+ 'riser_level': (0.0, 0.6),+ 'riser_f': (800, 12000, 'log'),+ # contact crack+ 'crack_level': (0.01, 0.8, 'log'),+ 'crack_ms': (0.2, 3.0, 'log'),+ 'crack_f': (600, 9000, 'log'),+ 'contact_ms': (0.15, 3.0, 'log'),+ 'crackle': (0.0, 1.5),+ 'debris_ms': (60, 500, 'log'),+ # the body: pitch-dropping membrane+ 'thump_f0': (70, 400, 'log'),+ 'thump_f1': (22, 70, 'log'),+ 'drop_tau': (0.006, 0.3, 'log'),+ 'thump_tau': (0.12, 2.0, 'log'),+ 'modes_level': (0.0, 0.6),+ # the struck structure's low-mid mode cloud+ 'shell_level': (0.0, 0.8),+ 'shell_lo': (70, 250, 'log'),+ 'shell_span': (2.0, 10.0, 'log'),+ 'shell_tau': (0.05, 1.2, 'log'),+ # sub tail+ 'sub_f': (24, 62, 'log'),+ 'sub_tau': (0.15, 3.0, 'log'),+ 'sub_level': (0.0, 1.0),+ # rumble+ 'rumble_level': (0.02, 1.2, 'log'),+ 'rumble_cut': (80, 1500, 'log'),+ 'rumble_tau': (0.3, 3.5, 'log'),+ # whoosh-down+ 'sweep_level': (0.005, 0.5, 'log'),+ 'sweep_hi': (250, 8000, 'log'),+ 'sweep_tau': (0.08, 1.2, 'log'),+ # distant echoes+ 'echo_level': (0.0, 0.6),+ 'echo_delay': (0.08, 0.6, 'log'),+ # hall + chain+ # flanged tail (time-varying feedback comb on the hall)+ 'comb_mix': (0.0, 0.8),+ 'comb_d0': (0.15, 2.0, 'log'),+ 'comb_ratio': (1.2, 6.0, 'log'),+ 'comb_fb': (0.2, 0.85),+ 'hall_wet': (0.05, 1.2, 'log'),+ 'hall_rt': (0.5, 8.0, 'log'),+ 'hall_damp': (0.0, 1.0),+ 'predelay': (0.0, 0.06),+ 'drive': (0.0, 1.0),+ 'sat_tone': (1500, 14000, 'log'),+ 'width': (0.0, 1.0),+ 'floor_db': (-95.0, -66.0),+ # round 8: the roll (the long low tail of a big space settling)+ 'roll_level': (0.0, 1.5),+ 'roll_cut': (60, 400, 'log'),+ 'roll_hold': (0.8, 3.2),+ 'roll_fall': (0.25, 1.5, 'log'),+ 'roll_rate': (0.3, 3.0, 'log'),+ 'roll_depth': (0.0, 0.8),+}+# round 8 pick 1 (launch film forge: the launch-film forge run (impact), tone=1.5): the forge-test final impact_01 + the roll+DEFAULTS = {'pre_len': 0.7875, 'pre_level': 0.133, 'pre_bright': 600.0, 'pre_low': 0.5775, 'riser_level': 0.0,+ 'riser_f': 3063.0, 'crack_level': 0.72, 'crack_ms': 1.907, 'crack_f': 1133.0, 'contact_ms': 0.6308,+ 'crackle': 1.035, 'debris_ms': 66.93, 'thump_f0': 141.9, 'thump_f1': 30.16, 'drop_tau': 0.03454,+ 'thump_tau': 0.2087, 'modes_level': 0.2633, 'shell_level': 0.2035, 'shell_lo': 121.7, 'shell_span': 3.57,+ 'shell_tau': 0.3507, 'sub_f': 60.13, 'sub_tau': 0.8391, 'sub_level': 0.0, 'rumble_level': 0.1937,+ 'rumble_cut': 103.9, 'rumble_tau': 1.658, 'sweep_level': 0.1123, 'sweep_hi': 365.9, 'sweep_tau': 0.5692,+ 'echo_level': 0.485, 'echo_delay': 0.1192, 'comb_mix': 0.1677, 'comb_d0': 0.2453, 'comb_ratio': 3.23,+ 'comb_fb': 0.7521, 'hall_wet': 0.8017, 'hall_rt': 3.897, 'hall_damp': 0.2887, 'predelay': 0.0,+ 'drive': 0.3529, 'sat_tone': 8003.0, 'width': 0.3363, 'floor_db': -88.67, 'roll_level': 0.4072,+ 'roll_cut': 141.7, 'roll_hold': 2.26, 'roll_fall': 0.5904, 'roll_rate': 0.6466, 'roll_depth': 0.5458}++TAIL_S = 5.0+NEUTRAL = {'comb_mix': 0.0, 'riser_level': 0.0, 'roll_level': 0.0} # values that switch a newer layer off (to seed from older rounds)+++# ------------------------------------------------------------------------------------------------ helpers+def _noise(n, r, color='white'):+ """white / pink / brown noise of n samples, unit RMS (IIR colouring: cheap at any length)"""+ w = r.standard_normal(n)+ if color == 'pink': # Paul Kellet's economy pink filter (-3 dB/oct within 0.5 dB above ~10 Hz at 48 kHz)+ w = signal.lfilter([0.049922035, -0.095993537, 0.050612699, -0.004408786],+ [1, -2.494956002, 2.017265875, -0.522189400], w)+ elif color == 'brown': # leaky integrator (-6 dB/oct above a few Hz)+ w = signal.lfilter([1.0], [1.0, -0.997], w)+ return w / (np.sqrt(np.mean(w ** 2)) + 1e-12)+++def _mic(y, r, floor_db, low_cut=20.0, air_db=-1.0):+ """a microphone + preamp on a stereo array: low cut, soft top, independent pink noise floor per channel"""+ y = dsp.butter(y, 'high', low_cut, order=1)+ y = dsp.peq(y, 10000, air_db, 0.6)+ pk = np.abs(y).max() + 1e-12+ for c in range(y.shape[1]):+ y[:, c] += _noise(len(y), r, 'pink') * pk * 10 ** (floor_db / 20)+ return y+++def _t(n, sr=SR):+ return np.arange(n) / sr+++def _glide(t, f_start, f_end, tau, r):+ """a sine whose frequency slides exponentially from f_start to f_end (time constant tau), closed-form phase"""+ ph = 2 * np.pi * (f_end * t + (f_start - f_end) * tau * (1 - np.exp(-t / max(tau, 1e-4))))+ return np.sin(ph + r.uniform(-0.3, 0.3))+++def _env(t, attack, tau):+ e = np.exp(-t / max(tau, 1e-4))+ if attack > 0:+ a = t < attack+ e[a] *= 0.5 - 0.5 * np.cos(np.pi * t[a] / attack)+ return e+++def _lp(x, f, order=2, sr=SR):+ return signal.sosfilt(signal.butter(order, float(np.clip(f, 10, sr * 0.45)), 'low', fs=sr, output='sos'), x, axis=0)+++def _hp(x, f, order=2, sr=SR):+ return signal.sosfilt(signal.butter(order, float(np.clip(f, 10, sr * 0.45)), 'high', fs=sr, output='sos'), x, axis=0)+++def _up(x, n):+ """6 kHz -> 48 kHz, cut / padded to n samples"""+ y = signal.resample_poly(x, SR // LO, 1)+ return y[:n] if len(y) >= n else np.pad(y, (0, n - len(y)))+++_XO = None+++def hall_ir(r, rt60, damp, size=0.8, predelay=0.02, seconds=None):+ """a computed hall impulse response: ~40 diffuse early reflections over the first ~(20 + 80 size) ms, then a noise+ tail that builds up and decays with a frequency-dependent RT60 (lows longer, highs shorter: air + wall absorption).+ The 4 decay bands are split subtractively (low-pass, remainder, low-pass, remainder ...) so they sum back exactly:+ summed Butterworth band-passes left phase notches at the crossovers (a dark line at 6 kHz through every tail)."""+ global _XO+ if _XO is None:+ _XO = [signal.butter(2, f, 'low', fs=SR, output='sos') for f in (250, 1500, 6000)]+ seconds = seconds or min(5.0, rt60 * 1.1 + 0.1)+ n = dsp.n_of(seconds)+ t = _t(n)+ w = r.standard_normal(n)+ rts = [rt60 * 1.25, rt60, rt60 * (1 - 0.5 * damp), rt60 * (1 - 0.8 * damp)]+ tail = np.zeros(n)+ rest = w+ for k, rt in enumerate(rts):+ if k < 3:+ band = signal.sosfilt(_XO[k], rest)+ rest = rest - band+ else:+ band = rest+ tail += band * np.exp(-6.91 * t / max(rt, 0.05))+ tail *= 1 - np.exp(-t / (0.008 + 0.03 * size)) # diffuse build-up+ er = np.zeros(n)+ for _ in range(40):+ d = int(r.uniform(0.003, 0.02 + 0.08 * size) * SR)+ if d < n:+ er[d] += r.uniform(0.05, 0.2) * r.choice([-1, 1]) * np.exp(-d / SR / 0.06)+ ir = tail * 0.25 + _lp(er, 7000, 1)+ ir /= np.sqrt(np.sum(ir ** 2)) + 1e-12+ pd = dsp.n_of(predelay) if predelay > 0 else 0+ return np.concatenate([np.zeros(pd), ir])+++def _sat(x, drive, tone):+ """soft clipping that only bites on the loud part (small signals pass at unity), then the console's top roll-off"""+ if drive <= 1e-3:+ return x+ g = 1 + 7 * drive+ pk = np.abs(x).max() + 1e-12+ y = pk * np.tanh(g * x / pk) / g * (1 + 0.15 * drive)+ return _lp(y, tone, 2)+++try:+ from numba import njit+except Exception: # pragma: no cover+ def njit(*a, **k):+ return (lambda f: f) if not (a and callable(a[0])) else a[0]+++@njit(cache=True)+def _vcomb_core(x, d, fb):+ """feedback comb with a time-varying fractional delay d (samples): y[i] = x[i] + fb * y[i - d[i]]"""+ n = len(x)+ y = np.empty(n)+ for i in range(n):+ v = x[i]+ q = i - d[i]+ if q >= 1.0:+ j = int(q)+ fr = q - j+ v += fb * ((1.0 - fr) * y[j] + fr * y[j + 1])+ y[i] = v+ return y+++def _flange(x, p):+ """the comb's peaks sit at k / delay: a delay that grows over the tail makes a fan of partials sliding down (the+ descending lines in the refs' tails). Mixed in parallel, so the plain hall stays underneath."""+ if p['comb_mix'] < 0.01:+ return x+ t = _t(len(x))+ d = p['comb_d0'] / 1000 * SR * (1 + (p['comb_ratio'] - 1) * (1 - np.exp(-t / 1.2)))+ y = _vcomb_core(np.ascontiguousarray(x), np.ascontiguousarray(d), p['comb_fb'])+ y *= (1 - p['comb_fb'])+ return x + p['comb_mix'] * (y - x)+++# ------------------------------------------------------------------------------------------------ layers+def _body(p, t, r):+ """the struck membrane at LO rate: fundamental + inharmonic overtones, all sliding down together"""+ x = _glide(t, p['thump_f0'], p['thump_f1'], p['drop_tau'], r) * _env(t, 0.0015, p['thump_tau'])+ ml = p['modes_level']+ if ml > 0.005:+ for j, ratio in enumerate(MEMBRANE[1:]):+ f0, f1 = p['thump_f0'] * ratio, p['thump_f1'] * ratio+ if f0 > LO * 0.45:+ continue+ a = ml * (0.75 ** j) * r.uniform(0.6, 1.0)+ x += a * _glide(t, f0, f1, p['drop_tau'], r) * _env(t, 0.001, p['thump_tau'] / (1.5 + 0.8 * j))+ return x+++def _shell(p, t, r):+ """the rest of the struck structure at LO rate: a dense cloud of low-mid modes (drum shell, the floor / stage, a+ big panel) between shell_lo and shell_lo * shell_span, higher modes decaying faster. Fills the 100-600 Hz region+ that a lone sliding fundamental leaves empty (the refs' tails carry it at -7..-10 dB, round 4 at -20 dB)."""+ lo = p['shell_lo']+ hi = min(lo * p['shell_span'], LO * 0.42)+ cnt = 28+ f = np.sort(np.exp(r.uniform(np.log(lo), np.log(hi), cnt)))+ u = np.log(f / lo) / max(1e-9, np.log(hi / lo))+ tau = p['shell_tau'] * np.exp(-1.6 * u) # top modes ring ~5x shorter+ a = 10 ** (-4.0 * np.log2(f / lo) / 20) * r.uniform(0.3, 1.0, cnt)+ x = np.zeros(len(t))+ for fi, ti, ai in zip(f, tau, a):+ x += ai * np.sin(2 * np.pi * fi * t + r.uniform(0, 2 * np.pi)) * np.exp(-t / ti)+ x *= 1 - np.exp(-t / 0.0015)+ return x / (np.abs(x).max() + 1e-12)+++def _rumble(p, t, r):+ """low turbulent energy at LO rate: brown noise through a low-pass that falls as the energy drains, rolling a bit"""+ n = len(t)+ w = _noise(n, r, 'brown')+ c = min(p['rumble_cut'], LO * 0.42)+ a = _lp(w, c, 2, LO)+ b = _lp(w, c * 0.3, 2, LO)+ k = np.exp(-t / max(p['rumble_tau'] * 0.5, 0.05))+ y = k * a + (1 - k) * b * 1.4+ y = _hp(y, 22, 2, LO)+ mod = np.ones(n)+ for _ in range(3):+ mod += 0.12 * np.sin(2 * np.pi * r.uniform(1.5, 6.0) * t + r.uniform(0, 2 * np.pi))+ y = y * mod * _env(t, 0.012, p['rumble_tau'])+ return y / (np.abs(y).max() + 1e-12)+++def _crack(p, n, r):+ """the contact: a smooth force pulse (contact_ms long: its spectrum rolls off smoothly above ~1/contact_ms), the+ striker's heavily damped high mode at crack_f ringing for crack_ms (a smooth spectral bump), and debris: a crackle+ of micro-impulses spread over debris_ms. History: a single few-ms noise burst (round 2) and a few-ms impulse cloud+ (round 3) both have a speckled spectrum (random deep notches every ~1/duration Hz) that the hall prints as static+ stripes through the whole tail; spread over >= 30 ms the speckle is finer than the ear's (and the spectrogram's)+ resolution."""+ m = min(n, dsp.n_of(0.6))+ out = np.zeros(n)+ k = max(2, dsp.n_of(p['contact_ms'] / 1000))+ out[:k] += 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, k))+ t = _t(m)+ tau = p['crack_ms'] / 1000+ out[:m] += 0.8 * np.sin(2 * np.pi * p['crack_f'] * t) * np.exp(-t / tau) * (1 - np.exp(-t / 0.0002))+ out[:m] = _hp(out[:m], 30, 1)+ if p['crackle'] > 0.01:+ td = p['debris_ms'] / 1000+ cnt = int(60 + 3000 * td)+ ts = r.exponential(td * 0.5, cnt)+ ts = ts[ts < 0.58]+ cl = np.zeros(m)+ np.add.at(cl, (ts * SR).astype(int), r.standard_normal(len(ts)) * np.exp(-ts / td))+ f = p['crack_f']+ cl = dsp.butter(cl, 'band', (f * 0.45, min(f * 4.0, SR * 0.45)), 2)+ cl /= np.abs(cl).max() + 1e-12+ out[:m] += p['crackle'] * cl+ return out / (np.abs(out).max() + 1e-12)+++def _sweep(p, n, r):+ t = _t(n)+ w = _noise(n, r, 'pink')+ lo = 70.0+ fc = lo + (p['sweep_hi'] - lo) * np.exp(-t / p['sweep_tau'])+ y = dsp.svf(w, fc, 2.5, 'band')+ y = y * _env(t, 0.004, p['sweep_tau'] * 1.8)+ return y / (np.abs(y).max() + 1e-12)+++def _riser(p, n, r):+ """air pulled in before the hit (the trailer suck-in as air, not reverb): noise through a band-pass whose centre+ rises exponentially to riser_f at the contact while the level grows (a reversed whoosh, steepest at the end)"""+ t = _t(n)+ u = t / max(t[-1], 1e-6) # 0 .. 1 over the swell+ fc = 150.0 * (p['riser_f'] / 150.0) ** (u ** 1.5)+ w = _noise(n, r, 'pink')+ y = dsp.svf(w, fc, 1.2, 'band')+ y *= (np.exp(3.5 * u) - 1) / (np.exp(3.5) - 1)+ return y / (np.abs(y).max() + 1e-12)+++def _roll(p, n, r):+ """round 8: the long low roll of a big space settling (see the history above), 48 kHz, peak 1. The noise is+ rendered at LO rate and up-sampled; the swell-and-recede is a smooth random log-amplitude (roll_rate Hz)"""+ m = int(n * LO / SR) + 8+ t = _t(m, LO)+ w = _lp(_noise(m, r, 'brown'), p['roll_cut'], 2, LO)+ w += 0.5 * _lp(_noise(m, r, 'pink'), p['roll_cut'] * 1.8, 2, LO) # a little body above the cut+ w = _hp(w, 24, 2, LO)+ k = max(2, int(LO / max(p['roll_rate'], 0.05)))+ lf = _lp(_noise(m + k, r), p['roll_rate'], 2, LO)[k:]+ lf /= np.sqrt(np.mean(lf ** 2)) + 1e-12+ hold, fall = p['roll_hold'], p['roll_fall']+ e = (1 - np.exp(-t / 0.15)) * np.exp(-t / (hold * 2.5))+ e *= np.where(t > hold, np.exp(-(t - hold) / fall), 1.0)+ y = w * e * np.exp(p['roll_depth'] * lf - 0.5 * p['roll_depth'] ** 2)+ y = _up(y, n)+ return y / (np.abs(y).max() + 1e-12)+++def _echoes(x, p, r):+ """2-3 distant reflections: later, quieter, darker each time"""+ out = np.zeros_like(x)+ if p['echo_level'] < 0.01:+ return out+ d = 0.0+ for k in range(3):+ d += p['echo_delay'] * r.uniform(0.8, 1.3)+ i = dsp.n_of(d)+ if i >= len(x):+ break+ e = _lp(x[:len(x) - i], 4000 * 0.5 ** k, 2)+ out[i:] += p['echo_level'] * 0.6 ** k * e+ return out+++def render(p, r):+ pre = p['pre_len']+ n_hit = dsp.n_of(TAIL_S)+ # centred direct sound: the body + sub (6 kHz, up-sampled, saturated) and the contact+ tl = _t(int(TAIL_S * LO), LO)+ low = (_body(p, tl, r) + p['sub_level'] * np.sin(2 * np.pi * p['sub_f'] * tl + r.uniform(0, 1)) * _env(tl, 0.02, p['sub_tau'])+ + p['shell_level'] * _shell(p, tl, r))+ low = _sat(_up(low, n_hit), p['drive'], p['sat_tone'])+ crack = _crack(p, n_hit, r)+ direct = low + p['crack_level'] * crack+ # diffuse layers (rumble, whoosh-down): partly independent per channel (width), like a diffuse sound field+ w = p['width']+ rum_c, swp_c = _up(_rumble(p, tl, r), n_hit), _sweep(p, n_hit, r)+ n_pre = dsp.n_of(pre)+ irs = [hall_ir(r, p['hall_rt'], p['hall_damp'], 0.8, p['predelay']),+ hall_ir(r, p['hall_rt'], p['hall_damp'], 0.8, p['predelay'] + r.uniform(0.001, 0.006))]+ n_tot = n_pre + n_hit + max(len(i) for i in irs)+ out = np.zeros((n_tot, 2))+ for c in range(2):+ rum = (1 - w) * rum_c + w * _up(_rumble(p, tl, r), n_hit) if w > 0.01 else rum_c+ swp = (1 - w) * swp_c + w * _sweep(p, n_hit, r) if w > 0.01 else swp_c+ dry = direct + p['rumble_level'] * rum + p['sweep_level'] * swp+ dry = dry + _echoes(dry, p, r)+ dry = dsp.fade(dry, 0.0, 0.4)+ out[n_pre:n_pre + n_hit, c] += dry+ wet = _flange(signal.fftconvolve(dry, irs[c]), p)+ out[n_pre:n_pre + len(wet), c] += p['hall_wet'] * wet+ # reverse swell: a hall tail of the contact (plus some of the low body), reversed into the hit+ m = dsp.n_of(0.2)+ head = crack[:m] / (np.abs(crack[:m]).max() + 1e-12) + p['pre_low'] * low[:m] / (np.abs(low[:m]).max() + 1e-12)+ pk = np.abs(out).max()+ for c in range(2):+ sw_ir = hall_ir(r, max(0.3, pre * 1.6), 0.4, 0.6, 0.0, seconds=pre * 1.2 + 0.25)+ sw = signal.fftconvolve(head, sw_ir)[:n_pre][::-1]+ sw = _lp(sw, p['pre_bright'], 2)+ sw = dsp.fade(sw, min(pre * 0.25, 0.1), 0.003)+ out[:n_pre, c] += sw * p['pre_level'] * pk / (np.abs(sw).max() + 1e-12)+ if p['riser_level'] > 0.005 and n_pre > 64:+ out[:n_pre, c] += p['riser_level'] * pk * dsp.fade(_riser(p, n_pre, r), 0.01, 0.002)+ if p.get('roll_level', 0.0) > 0.005: # round 8: drawn last, so older params re-render exactly+ n_roll = min(len(out) - n_pre, dsp.n_of(p['roll_hold'] + 6 * p['roll_fall'] + 0.5))+ rc = _roll(p, n_roll, r)+ w = p['width']+ for c in range(2):+ rl = (1 - w) * rc + w * _roll(p, n_roll, r) if w > 0.01 else rc+ out[n_pre:n_pre + n_roll, c] += p['roll_level'] * pk * dsp.fade(rl, 0.0, 0.3)+ y = _mic(out, r, p['floor_db'])+ y = dsp.trim_tail(y, -66, 0.05)+ return dsp.fade(y, 0.002, 0.25)+++def swell(seconds, r, f_start=150.0, f_end=6000.0, q=1.2, curve=1.5, grow=3.5, width=0.6, room_wet=0.25,+ tube=0.5, tube_lo=90.0, tube_hi=600.0, flutter=0.25, rev=0.5):+ """the air riser on its own (launch film): a rising swell that ENDS on its loudest sample, for a stop-down cut+ into a hit. Layers, all computed:+ * air pulled in: pink noise through a band-pass whose centre rises from f_start to f_end+ (f = f_start * (f_end / f_start) ** u**curve, steepest at the end) while the level grows exponentially (grow);+ * a shortening air column (tube): the same air driving a pipe whose resonances k * f_tube (k = 1..6, Q ~ 9,+ amplitude 1 / k) rise from tube_lo to tube_hi: the tonal, harmonic sweep of a tension riser;+ * turbulence: a slow random log-normal flutter of the level (flutter) - moving air is never steady;+ * a reverse swell (rev): the computed hall tail of a broadband strike, reversed, so a diffuse wash grows into+ the end (the trailer reverse-cymbal);+ partly independent per channel (width), in a short hall, through a microphone. Stereo 48 kHz, peak 1, a 6 ms+ fade at the end. History: round 1 (band-passed noise alone) read 'filtered white noise', p_real 0.12."""+ n = dsp.n_of(seconds)+ t = _t(n)+ u = t / max(t[-1], 1e-6)+ fc = f_start * (f_end / f_start) ** (u ** curve)+ ft = tube_lo * (tube_hi / tube_lo) ** (u ** curve)+ lvl = (np.exp(grow * u) - 1) / (np.exp(grow) - 1)+ ir = hall_ir(r, 0.9, 0.5, 0.6, 0.004)++ def one():+ w = _noise(n, r, 'pink')+ y = dsp.svf(w, fc, q, 'band')+ y /= np.sqrt(np.mean(y ** 2)) + 1e-12+ if tube > 0:+ pipe = np.zeros(n)+ for k in range(1, 7):+ pipe += dsp.svf(w, np.minimum(ft * k, SR * 0.4), 9.0, 'bandn') / k+ y += tube * pipe / (np.sqrt(np.mean(pipe ** 2)) + 1e-12)+ if flutter > 0:+ k = int(SR / 12)+ lf = _lp(_noise(n + k, r), 12.0, 2)[k:]+ y *= np.exp(flutter * lf / (np.sqrt(np.mean(lf ** 2)) + 1e-12) - 0.5 * flutter ** 2)+ return y * lvl++ c = one()+ out = np.zeros((n, 2))+ if rev > 0:+ strike = np.zeros(dsp.n_of(0.02))+ strike[:dsp.n_of(0.004)] = _noise(dsp.n_of(0.004), r)+ rv_ir = hall_ir(r, seconds * 1.4, 0.45, 0.7, 0.0, seconds=seconds + 0.1)+ for ch in range(2):+ y = (1 - width) * c + width * one()+ y /= np.abs(y).max() + 1e-12+ if rev > 0:+ tail = signal.fftconvolve(strike, rv_ir)[:n][::-1]+ tail = _lp(tail, f_end * 0.8, 2)+ y += rev * tail / (np.abs(tail).max() + 1e-12) * (0.3 + 0.7 * u)+ wet = signal.fftconvolve(y, ir)[:n]+ out[:, ch] = y + room_wet * wet / (np.sqrt(np.sum(ir ** 2)) + 1e-9)+ out = _mic(out, r, -80.0)+ out = dsp.fade(out, 0.03, 0.006)+ return out / (np.abs(out).max() + 1e-12)diff --git a/earshot/forge/recipes/keypress.py b/earshot/forge/recipes/keypress.pynew file mode 100644index 0000000..b38af73--- /dev/null+++ b/earshot/forge/recipes/keypress.py@@ -0,0 +1,339 @@+"""A deep "thocky" mechanical keyboard key press, built from physics (the model after 15 forge rounds; demo/forge-test/+keypress/LOG.md has the round-by-round story and what earshot heard). Rounds 13-15 (after the judge) fixed the tonal+balance measured model-free against the real keys: the case body moved below 260 Hz (it was a +16 dB boxy 'tok' at+500-630 Hz), the 8-10 kHz spring ping came back, the high shell modes start above the refs' 4-6 kHz dip, the finger's+push is shorter and a session high-pass removes the 15-45 Hz shove: 8-13 dB from the real keys' median balance before,+5.4-5.6 dB after (the real keys sit 3.1-6.0 dB from each other).++What the real Keychron references show (measured in demo/forge-test/keypress/analyze_refs.py, lowwave.py, and seen on+their waveforms / spectrograms), and how each part is modelled here:++1. Bottom-out contact (t = 0). The stem / keycap slams into the bottom housing and plate in well under a millisecond.+ - an instantaneous full-band click (the refs' onset is one vertical line up to 20 kHz; 7-14 kHz is 20 dB down after+ ~2 ms): a broadband crack plus a dense high shell-mode cloud, driven by a half-sine Hertzian contact pulse whose+ length (click_tc) sets how far up the spectrum it reaches; a direct pressure snap (the first big asymmetric spike);+ - the keycap / plate modes: a main mode near 1.25-1.6 kHz plus a cluster up to ~2.3x, mixed with band noise (a+ cluster too dense to resolve). The finger keeps the stem pressed, so the cap rings freely for ~1 ms and is then+ clamped to a residual (cap_fast / cap_res: time-varying damping). Longer cap rings (tau > ~4.5 ms) read as a+ dropped coin to AudioSet, so cap_tau is capped there;+ - the case body (110-260 Hz since round 13; the refs dip at 250-630 Hz) rings a little longer.+ Part levels are energy ratios over the first 5 ms (as the refs' band energies are measured), not peak ratios.+2. The "thock": a 58-77 Hz low wave that swells to its maximum 20-35 ms after the hit, beats (dips near 45 and 85 ms)+ and dies with tau ~50-60 ms - the board + desk moving on its feet (thock_f) beating with the case's own low mode+ (case_f), driven by the finger's smooth push (thock_rise). The mic hears pressure, which follows acceleration, so+ the quasi-static push itself is high-passed away.+3. The release 70-125 ms later: two small high ticks (the stem leaving the bottom housing / the spring snapping off its+ seat, tick_dt before the top-out), then the top-out - a smaller, brighter contact with the cap modes shifted, and a+ small kick to the low wave. A probe showed the critic hears the ticks as essential (without them typicality -4).+4. Pre-travel friction (a faint swell before the hit), a spaced stereo pair in a small room (removing the room drops+ typicality to -10: the refs are close-mic'd but not dry; a sparse-echo or a diffuse room model), a noise floor (low+ rumble + hiss rolled off near 18-20 kHz) and the kit's edit: 0.2-0.4 s after the hit, faded.+The modal layout and the mic / room geometry belong to the board and the session, so they come from the `key`+parameter (fixed per key); each press only varies force, timing and noise - the same key sounds the same twice.++No recorded audio is used: every sample is computed.+"""+import numpy as np+from scipy import signal++from .. import dsp+from . import REF_ROOT++SR = dsp.SR+TARGET = 'a single deep thocky mechanical keyboard key press'+ALTS = ['a wooden knock', 'a computer mouse click', 'a typewriter key', 'a cork pop']+_K = REF_ROOT + '/keyboard/'+REFS = [_K + f for f in ('kb_thock_01.wav', 'kb_thock_02.wav', 'kb_thock_03.wav', 'kb_thock_04.wav',+ 'kb_enter_01.wav', 'kb_space_01.wav')]++# ranges marked (refs) are limits measured on the references, set after the search exploited regions no real+# recording occupies (LOG.md rounds 5, 6, 12)+SPACE = {+ 'click_tc': (0.00003, 0.0004, 'log'), # contact time of the bottom-out (sets how high the click reaches)+ 'click_tau': (0.00015, 0.001, 'log'), # high shell modes' decay (refs: 7-14 kHz 20 dB down in ~2 ms)+ 'click_lvl': (0.1, 2.0, 'log'), # click energy relative to the cap modes (first 5 ms)+ 'click_lo': (6500, 9500, 'log'), # lowest of the high shell modes (round 13: the refs dip at 4-6.3 kHz)+ 'click_tilt': (-3.0, 3.0), # their spectral tilt, dB / octave+ 'crack': (0.2, 2.0, 'log'), # broadband crack relative to the high modes+ 'snap': (0.0, 1.0), # direct pressure spike+ 'cap_f': (900, 1800, 'log'), # main keycap / plate mode+ 'cap_tau': (0.0015, 0.0045, 'log'), # (refs: 1.2-3 kHz 20 dB down in 6-10 ms)+ 'cap_noise': (0.0, 1.0), # dense-cluster share (band noise) vs discrete modes+ 'cap_n': (6, 30, 'int'),+ 'cap_cl': (0.1, 1.0, 'log'), # the cluster's level under the main cap mode+ 'cap_fast': (0.0005, 0.003, 'log'), # free ring before the stem clamps the cap+ 'cap_res': (0.05, 1.0, 'log'), # residual ring after clamping (1 = no clamping)+ 'mid_f': (110, 260, 'log'), # case body mode (rounds 13-15: the refs dip at 250-630 Hz and are full+ 'mid_tau': (0.003, 0.016, 'log'), # at 100-200 Hz; at 337-615 Hz it was a +16 dB boxy 'tok', at 200-340+ 'mid_lvl': (0.01, 0.2, 'log'), # Hz still +10 dB at 250 Hz). Energy relative to the cap+ 'thock_f': (55, 90, 'log'), # the low board + desk mode (refs: 58-77 Hz)+ 'thock_tau': (0.045, 0.1, 'log'), # (refs: tau ~50-60 ms)+ 'thock_lvl': (0.05, 0.8, 'log'), # its amplitude relative to the click peak+ 'thock_rise': (0.001, 0.025, 'log'), # length of the finger's push (round 13: longer pushes put -9..-12 dB+ # at 15-45 Hz, a sub-sonic shove the refs do not have)+ 'case_f': (70, 220, 'log'), # the case's own low mode (beats with the board mode)+ 'case_lvl': (0.0, 0.8),+ 'case_dec': (0.3, 1.0), # its decay relative to the board mode's+ 'up_delay': (0.07, 0.125), # (refs: top-out 75-120 ms after the hit)+ 'up_lvl': (0.08, 0.35, 'log'), # (refs: 10-22 dB under the bottom-out)+ 'up_pitch': (0.9, 1.5),+ 'tick_lvl': (0.0, 1.0), # release ticks before the top-out+ 'tick_dt': (0.004, 0.025, 'log'), # how long before the top-out they come+ 'pre_len': (0.004, 0.025, 'log'), # key travel before the bottom-out (friction swell)+ 'pre_lvl': (0.001, 0.06, 'log'),+ 'ping_f': (7500, 11000, 'log'), # spring ping (round 13: the refs' 8-10 kHz line, 10-40 ms; it was+ 'ping_tau': (0.004, 0.03, 'log'), # muted or at 6-7 kHz, -14 dB at 10 kHz)+ 'ping_lvl': (0.04, 0.3),+ 'room_wet': (0.0, 0.4),+ 'rt60': (0.05, 0.4, 'log'),+ 'room_size': (0.0, 1.0),+ 'room_damp': (0.2, 0.95),+ 'room_model': ['sparse', 'diffuse'], # dsp.room (6 discrete early echoes + tail) or a diffuse field+ 'floor_db': (-78.0, -55.0), # (refs: tails at -53..-77 dB re peak)+ 'rumble': (0.0, 0.7), # low-frequency share of the noise floor+ 'clip': (0.2, 0.4), # clip length after the hit (the kit's edits: 0.2-0.39 s)+ 'hpf': (36, 46, 'log'), # the session's high-pass (round 13: the refs are -38 / -47 dB at 39 /+ 'key': (0, 999, 'int'), # 31 Hz; the slow push and the rumble put +16-39 dB there). key: see below+}+# the most reliable sound after the judge (round 15 pick 4 = final keypress_01, parameters rounded to 6 digits): 8 new+# seeds -> critic mean 6.74 with the model-free tone check in the total (Critic tone=1.5), 7/8 pass, 5.6 dB from the+# real keys' median balance+DEFAULTS = {'click_tc': 7.62977e-05, 'click_tau': 0.000400034, 'click_lvl': 0.195527, 'click_lo': 7323.18,+ 'click_tilt': -1.25571, 'crack': 0.395643, 'snap': 0.0, 'cap_f': 1489.18, 'cap_tau': 0.00399449,+ 'cap_noise': 1.0, 'cap_n': 7, 'cap_cl': 0.357156, 'cap_fast': 0.000995191, 'cap_res': 0.749554,+ 'mid_f': 205.793, 'mid_tau': 0.00918634, 'mid_lvl': 0.0902506, 'thock_f': 86.8567,+ 'thock_tau': 0.0549253, 'thock_lvl': 0.226067, 'thock_rise': 0.00834742, 'case_f': 70.0,+ 'case_lvl': 0.581134, 'case_dec': 0.736866, 'up_delay': 0.124575, 'up_lvl': 0.0969741,+ 'up_pitch': 1.04402, 'tick_lvl': 0.581597, 'tick_dt': 0.0111911, 'pre_len': 0.0128195,+ 'pre_lvl': 0.0207414, 'ping_f': 9083.43, 'ping_tau': 0.00578368, 'ping_lvl': 0.122274,+ 'room_wet': 0.261726, 'rt60': 0.113685, 'room_size': 0.212985, 'room_damp': 0.95, 'room_model': 'sparse',+ 'floor_db': -55.0, 'rumble': 0.0177183, 'clip': 0.38353, 'hpf': 44.5337, 'key': 999}+++# ------------------------------------------------------------------------------------------------ helpers+def _t(n):+ return np.arange(n) / SR+++def _key_rng(p, part):+ """the key's (and the session's) fixed physical layout: mode frequencies and mic reflections belong to this board,+ not to one press, so they come from the `key` parameter; the per-press generator r only adds natural variation"""+ return dsp.rng(int(p['key']) * 7919 + part * 104729 + 17)+++def _ring(freqs, taus, amps, n, r):+ """damped sinusoids, each computed only over ~9 time constants (cheap for short plastic modes)"""+ out = np.zeros(n)+ t = _t(n)+ for f, d, a in zip(freqs, taus, amps):+ if a == 0 or f >= SR * 0.47:+ continue+ m = min(n, int(9 * d * SR) + 2)+ out[:m] += a * np.exp(-t[:m] / d) * np.sin(2 * np.pi * f * t[:m] + r.uniform(0, 2 * np.pi))+ return out+++def _cloud(f_lo, f_hi, count, tau_lo, tau_hi, n, kr, r, tilt=-3.0, detune=0.002):+ """a dense set of modes, log-spread (layout from kr: the object's own modes); higher ones decay faster; tilt+ dB/oct; each press detunes them a hair (contact point, temperature)"""+ f = np.sort(np.exp(kr.uniform(np.log(f_lo), np.log(f_hi), int(count))))+ a = 10 ** (tilt * np.log2(f / f_lo) / 20) * kr.uniform(0.3, 1.0, len(f))+ f = f * (1 + detune * r.standard_normal(len(f)))+ u = np.log(f / f_lo) / max(1e-9, np.log(f_hi / f_lo))+ d = np.exp(np.log(tau_lo) + np.clip(u, 0, 1) * (np.log(tau_hi) - np.log(tau_lo)))+ return _ring(f, d, a, n, r)+++def _hiss(lo, hi, tau, n, r, order=2):+ """band noise with an exponential decay (a mode cluster too dense to resolve)"""+ m = min(n, int(9 * tau * SR) + 2)+ w = dsp.butter(r.standard_normal(m), 'band', (lo, min(hi, SR * 0.45)), order)+ out = np.zeros(n)+ out[:m] = w * np.exp(-_t(m) / tau)+ return out+++def _pulse(tc):+ """half-sine contact force of duration tc, unit area"""+ k = max(1, int(round(tc * SR)))+ q = np.sin(np.pi * (np.arange(k) + 0.5) / k)+ return q / q.sum()+++def _drive(ir, tc):+ return np.convolve(ir, _pulse(tc))[: len(ir)]+++def _unit(x):+ return x / (np.abs(x).max() + 1e-12)+++def _en(x, ms=5.0):+ """scale to unit RMS over the first `ms` milliseconds (levels between the parts of a hit are energy ratios, as+ measured in the refs' bands, not peak ratios: a noise burst's peak says little about how loud it sounds)"""+ m = max(8, dsp.n_of(ms / 1000))+ return x / (np.sqrt(np.mean(x[:m] ** 2)) + 1e-12)+++# ------------------------------------------------------------------------------------------------ the parts+def _crack(r, n, tau, tc):+ """the impact's own pressure transient: an extremely short broadband burst, flat up to 20 kHz (the refs' onset is+ one vertical line across the whole spectrum)"""+ m = min(n, max(dsp.n_of(9 * tau), 8))+ w = r.standard_normal(m) * np.exp(-_t(m) / tau)+ out = np.zeros(n)+ out[:m] = dsp.butter(w, 'high', 1200, 1)+ return _drive(out, tc)+++def _contact(p, r, n, part, pitch=1.0, bright=1.0):+ """one plastic contact (part 0 bottom-out, 1 top-out): crack + high shell modes + snap + cap modes + mid body"""+ kr = _key_rng(p, part)+ tc = p['click_tc'] * float(np.exp(0.15 * r.standard_normal())) # a press is never exactly the same+ hf = _cloud(p['click_lo'], 18000, 40, p['click_tau'] * 1.6, p['click_tau'] * 0.6, n, kr, r, tilt=p['click_tilt'])+ hf = _drive(_unit(hf), tc)+ crack = _unit(_crack(r, n, p['click_tau'] * 0.4, tc * 0.5))+ cf = p['cap_f'] * pitch+ modes = _ring([cf], [p['cap_tau'] * 1.2], [1.0], n, r) + p['cap_cl'] * \+ _unit(_cloud(cf * 1.08, cf * 2.3, p['cap_n'], p['cap_tau'] * 0.9, p['cap_tau'] * 0.45, n, kr, r))+ band = _hiss(cf * 0.75, cf * 2.4, p['cap_tau'] * 0.8, n, r)+ cap = (1 - p['cap_noise']) * _en(modes) + p['cap_noise'] * _en(band)+ # the finger keeps the stem pressed on the bottom housing: the cap rings freely for a millisecond or two, then it+ # is clamped and only a residual rings on (refs: the dense 1-3 kHz burst dies within ~3 ms, a softer ring follows;+ # measured attack 1 ms = the onset energy is front-loaded). Time-varying damping = a gain envelope on the cap.+ cap = cap * (p['cap_res'] + (1 - p['cap_res']) * np.exp(-_t(n) / p['cap_fast']))+ cap = _drive(cap, max(tc, 0.00012))+ mf = p['mid_f'] * pitch+ ratios = [1.0, kr.uniform(1.35, 1.6), kr.uniform(1.9, 2.4)]+ mid = _ring([mf * q for q in ratios], [p['mid_tau'], p['mid_tau'] * 0.7, p['mid_tau'] * 0.5], [1, .5, .3], n, r)+ k = max(3, int(kr.uniform(0.25, 0.6) * 1e-3 * SR))+ snap = np.zeros(n)+ snap[:k] = -np.hanning(k)+ x = bright * p['click_lvl'] * _en(_unit(hf) + p['crack'] * crack) + _en(cap) + p['mid_lvl'] * _en(mid)+ return _unit(_unit(x) + p['snap'] * snap)+++def _ping(p, r, n, f_mul=1.0):+ """the spring's ping / the plate's high ring: a few close high modes (8-10 kHz line in the refs, 10-40 ms long)"""+ f = p['ping_f'] * f_mul+ return _unit(_ring([f, f * 1.013, f * 1.62], [p['ping_tau'], p['ping_tau'] * 0.8, p['ping_tau'] * 0.4],+ [1.0, 0.5, 0.1], n, r))+++def _low(p, r, n):+ """the board + desk moving as a mass on its feet (thock_f) plus the case's own low mode (case_f), driven by the+ finger's push: a smooth force of thock_rise seconds, so the low wave swells after the click instead of starting at+ full size. The refs' low wave (lowwave.py) is 58-77 Hz, swells to its maximum 20-35 ms after the hit and then beats+ (dips near 45 and 85 ms): two close low modes, both dying with tau ~50 ms. The mic hears pressure, which follows+ the surface's acceleration, not its displacement: the quasi-static push itself is inaudible (high-pass)."""+ t = _t(n)+ f = p['thock_f'] * (1 + 0.01 * r.standard_normal())+ ir = np.exp(-t / p['thock_tau']) * np.sin(2 * np.pi * f * t)+ ir += p['case_lvl'] * np.exp(-t / (p['thock_tau'] * p['case_dec'])) * np.sin(2 * np.pi * p['case_f'] * t + 0.5)+ k = max(2, dsp.n_of(p['thock_rise']))+ force = 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, k))+ y = np.convolve(ir, force / force.sum())[:n]+ return _unit(dsp.butter(y, 'high', 0.75 * min(f, p['case_f']), order=2))+++def _pretravel(p, r, n):+ """the finger riding the cap down and the stem sliding in its housing: a faint friction noise that swells over+ the travel and stops at the bottom-out (refs sit at -45..-28 dB in the 5-25 ms before the hit)"""+ m = dsp.n_of(p['pre_len'])+ t = np.linspace(0, 1, m)+ w = dsp.butter(r.standard_normal(m), 'band', (700, 9000), 2)+ grit = (r.uniform(0, 1, m) < 0.004) * r.standard_normal(m) * 4.0 # stick-slip micro clicks+ w = (w + dsp.butter(grit, 'high', 2000, 1)) * t ** 2+ out = np.zeros(n)+ out[:m] = w / (np.abs(w).max() + 1e-12)+ return out+++def _diffuse_room(x, kr, p):+ """a small furnished room as a diffuse field: no sparse discrete echoes (those comb-filter into a metallic, boxy+ ring: AudioSet heard a dropped coin in the sparse dsp.room version), but one desk reflection, then a smooth+ build-up of dense reflections (room_size sets how slowly it builds) and a tail whose highs die faster than its+ lows (room_damp: soft furnishings absorb highs)"""+ rt = p['rt60']+ m = dsp.n_of(min(0.6, rt * 1.1 + 0.03))+ t = _t(m)+ w = kr.standard_normal(m)+ ramp = 0.0015 + 0.008 * p['room_size']+ onset = (1 - np.exp(-t / ramp)) * (t > 0.0008)+ ir = np.zeros(m)+ for band, k in ((dsp.butter(w, 'low', 700, 2), 1.0), (dsp.butter(w, 'band', (700, 4000), 2), 0.75),+ (dsp.butter(w, 'high', 4000, 2), 0.25)):+ rtb = max(0.02, rt * (1 - p['room_damp'] * (1 - k)))+ ir += band * np.exp(-6.91 * t / rtb)+ ir *= onset+ d = int(kr.uniform(0.8e-3, 2.5e-3) * SR) # the desk under the keyboard+ ir[d] += 0.35 * np.sqrt(np.sum(ir ** 2))+ ir = dsp.butter(ir, 'low', 9000, 1) * 0.6 + ir * 0.4+ y = signal.fftconvolve(x, ir)[: len(x)]+ return x + p['room_wet'] * y / (np.sqrt(np.sum(ir ** 2)) + 1e-9)+++def _spaced_pair(x, p, r):+ """two close mics a few cm apart in a small room (all fixed for the session, so from the key rng): each sees the+ source with its own tiny delay and weak early reflections (the desk, the case), plus the room's own response.+ The probe (probe.py) showed the room is not optional: the same press without it lands at typicality z -10."""+ kr = _key_rng(p, 9)+ chans = []+ for c in range(2):+ ir = np.zeros(int(0.007 * SR))+ ir[int(kr.uniform(0, 0.25e-3) * SR)] = 1.0+ refl = np.zeros_like(ir)+ for _ in range(6):+ refl[int(kr.uniform(0.3e-3, 6.5e-3) * SR)] += kr.uniform(0.03, 0.12) * kr.choice([-1, 1])+ ir += dsp.butter(refl, 'low', 12000, order=1)+ y = np.convolve(x, ir)[: len(x)]+ if p['room_model'] == 'diffuse':+ y = _diffuse_room(y, _key_rng(p, 10 + c), p)+ else:+ y = dsp.room(y, _key_rng(p, 10 + c), rt60=p['rt60'], size=p['room_size'], damp=p['room_damp'],+ wet=p['room_wet'])[: len(x)]+ chans.append(y)+ return np.stack(chans, axis=1)+++def _floor(n, r, rumble, rum_lo=25):+ """the recording's own noise: low rumble + near-white hiss rolled off around 18-20 kHz (unit RMS)"""+ hiss = dsp.noise(n / SR, r, 'pink') + dsp.noise(n / SR, r, 'white')+ hiss = dsp.butter(hiss, 'low', r.uniform(17000, 20000), order=4)+ rum = dsp.butter(dsp.noise(n / SR, r, 'brown'), 'band', (rum_lo, 120), order=2)+ y = (1 - rumble) * hiss / (np.std(hiss) + 1e-12) + rumble * 2.0 * rum / (np.std(rum) + 1e-12)+ return y / (np.std(y) + 1e-12)+++# ------------------------------------------------------------------------------------------------ render+def render(p, r):+ pre = p['pre_len'] + r.uniform(0.003, 0.02)+ dur = pre + p['clip'] * (1 + 0.05 * r.standard_normal()) # the kit's clips are edited and faded+ n = dsp.n_of(dur + 0.05)+ nh = dsp.n_of(0.12)+ force = float(np.exp(0.1 * r.standard_normal())) # this press's strength (brightness follows force)+ x = np.zeros(n)+ dsp.place(x, _pretravel(p, r, dsp.n_of(p['pre_len'])), pre - p['pre_len'], p['pre_lvl'])+ dsp.place(x, _contact(p, r, nh, 0, bright=force), pre)+ dsp.place(x, _ping(p, r, nh), pre, p['ping_lvl'])+ dsp.place(x, _low(p, r, n - dsp.n_of(pre)), pre, p['thock_lvl'] * force)+ tu = pre + p['up_delay'] * (1 + 0.04 * r.standard_normal())+ if p['tick_lvl'] > 0.005: # the stem leaving the bottom housing / the spring snapping off its seat: two+ kr = _key_rng(p, 5) # small high ticks 5-25 ms before the top-out (refs: ~80 + ~88 ms, up at ~97)+ t1 = tu - p['tick_dt'] * (1 + 0.08 * r.standard_normal())+ for j, (dt, g) in enumerate([(0.0, 1.0), (r.uniform(0.003, 0.009), r.uniform(0.4, 1.0))]):+ tick = _drive(_unit(_cloud(3000, 14000, 14, 0.0008, 0.0003, nh, kr, r)), 0.00005)+ dsp.place(x, tick, min(t1 + dt, tu - 0.001), g * p['tick_lvl'] * p['up_lvl'])+ dsp.place(x, _contact(p, r, nh, 1, pitch=p['up_pitch'], bright=1.2), tu, p['up_lvl'])+ dsp.place(x, _ping(p, r, nh, 1.0 + 0.01 * r.standard_normal()), tu, p['ping_lvl'] * p['up_lvl'] * 0.7)+ dsp.place(x, _low(p, r, n - dsp.n_of(tu)), tu, p['thock_lvl'] * p['up_lvl'] * 0.3)+ st = _spaced_pair(x, p, r)+ st = dsp.butter(st, 'high', 22, order=2)+ pk = np.abs(st).max()+ hpf = p.get('hpf') # absent in the forge test's round 1-12 params: their sound is unchanged+ for c in range(2):+ st[:, c] += _floor(n, r, p['rumble'], 25 if hpf is None else hpf) * pk * 10 ** (p['floor_db'] / 20)+ if hpf is not None: # the session's high-pass, as on the kit's edits (order 8: steep under ~45 Hz)+ st = dsp.butter(st, 'high', hpf, order=8)+ st = st[: dsp.n_of(dur)]+ return dsp.fade(st, 0.002, r.uniform(0.025, 0.05))diff --git a/earshot/forge/recipes/pluck.py b/earshot/forge/recipes/pluck.pynew file mode 100644index 0000000..4a090b6--- /dev/null+++ b/earshot/forge/recipes/pluck.py@@ -0,0 +1,235 @@+"""A kalimba (thumb piano) tine pluck, built from physics; every sample is computed, no recorded audio anywhere.++ from earshot.forge import dsp+ from earshot.forge.recipes import pluck+ x = pluck.render({**pluck.DEFAULTS, 'f0': pluck.note('C5')}, dsp.rng(1)) # 48 kHz stereo, one C5 pluck++What a kalimba note is, physically, and how each part is modelled:+ * The tine is a steel cantilever (clamped at the bridge, free at the tip). A clamped-free bar's bending modes are+ strongly inharmonic: Euler-Bernoulli gives 1 : 6.27 : 17.55 : 34.39 (real tines, with the tuning mass of their+ length and a curved tip, sit at ~5-7x for the second mode). The first mode is the note; the second is the bright+ metallic "ping" in the attack, and it dies far faster than the fundamental (air and bridge losses grow with+ frequency): taus fall roughly as f^-0.9.+ * Two polarisations: the tine bends up / down and, slightly stiffer, sideways. A pluck is never perfectly vertical,+ so the note is two close modes (beat_hz apart): the slow shimmer of a real kalimba note.+ * Geometric nonlinearity: a large deflection stretches the tine, giving a weak 2nd harmonic that follows the+ SQUARED envelope (it decays twice as fast as the fundamental).+ * The thumb: the flesh pushes the tine down for a few ms, then the tine slips off. The release is soft (the contact+ has a width thumb_ms), so high modes are excited less (a low-pass on the excitation, ~1 / (1 + (f T)^2)), plus a+ little skin / nail noise on the slip.+ * The box: the tine drives the bridge, the bridge drives a wooden box: a Helmholtz resonance (the sound hole) and a+ cloud of short wooden plate modes (150 Hz - 2.5 kHz, 3-30 ms) that colour the attack (the "wooden" knock in it).+ * Sympathetic tines: every other tine on the bridge is driven a little (strongest where its note coincides with a+ partial of the played note: octaves, fifths); they ring up slowly and keep a faint chord under the note (clearly+ visible in real kalimba recordings as the other tines' lines).+ * A small room, a close microphone with a noise floor, a little stereo from the room.+Pitch: `f0` (Hz) renders the note at an exact pitch (note('C5') = 523.25 Hz, A4 = 440). Without f0 a forge run picks+one of the reference recordings' pitches per render (G4 A4 B4 D5 E5) so the critic compares like with like.+"""+import numpy as np+from scipy import signal++from .. import dsp+from . import REF_ROOT++SR = dsp.SR+TARGET = 'a kalimba note, a thumb piano pluck'+ALTS = ['a music box', 'a xylophone note', 'a glockenspiel', 'an electric piano note', 'a synthesizer pluck']+_K = REF_ROOT + '/kalimba/'+REFS = [_K + f for f in ('fs-681787_kalimba-G.ogg', 'fs-681788_kalimba-A.ogg', 'fs-681789_kalimba-B.ogg',+ 'fs-681791_kalimba-D-high.ogg', 'fs-681792_kalimba-E-high.ogg')]+REF_F0 = [392.0, 440.0, 493.9, 587.3, 659.3] # the references' notes (measured: G4 A4 B4 D5 E5)+# a 17-tine kalimba in C (C4 .. E6): the tines that can ring sympathetically+_SCALE = [0, 2, 4, 5, 7, 9, 11]+TINES = [261.63 * 2 ** ((12 * o + s) / 12) for o in range(3) for s in _SCALE][:17]++SPACE = {+ 'p2_ratio': (5.0, 7.0), # second bending mode / fundamental (ideal bar 6.27)+ 'p2_lvl': (0.02, 0.6, 'log'), # its level at the pluck (before the thumb's low-pass)+ 'p3_lvl': (0.0, 0.25), # third bending mode (2.8 x the second)+ 'tau1': (0.35, 2.4, 'log'), # fundamental decay at C5 (higher notes shorter, ~ f^-0.5)+ 'tau_exp': (0.5, 1.4), # upper modes: tau ~ tau1 * (f / f0)^-tau_exp+ 'beat_hz': (0.2, 4.0, 'log'), # polarisation split+ 'beat_lvl': (0.0, 0.6),+ 'h2_lvl': (0.0, 0.25), # 2nd harmonic from the large-deflection nonlinearity+ 'thumb_ms': (0.1, 4.0, 'log'), # contact width of the release (soft flesh = long = darker)+ 'zing_lvl': (0.0, 0.4), # round 2: the release slap + bridge rattle: a short metallic buzz at f0's period+ 'zing_f': (2500, 12000, 'log'),+ 'zing_tau': (0.004, 0.06, 'log'),+ 'skin_lvl': (0.0, 0.25), # skin / nail slip noise+ 'skin_f': (1500, 9000, 'log'),+ 'helm_f': (160, 420, 'log'), # box Helmholtz resonance+ 'helm_tau': (0.008, 0.06, 'log'),+ 'box_lvl': (0.0, 0.8), # the wooden box's share (drives through the bridge)+ 'box_lo': (150, 600, 'log'),+ 'box_tau': (0.003, 0.03, 'log'),+ 'knock_lvl': (0.0, 0.8), # the thumb's push on the instrument: a low thud of the box / table+ 'knock_f': (40, 160, 'log'),+ 'knock_tau': (0.008, 0.08, 'log'),+ 'symp_lvl': (0.0, 0.25), # sympathetic tines+ 'symp_rise': (0.005, 0.08, 'log'),+ 'room_wet': (0.0, 0.35),+ 'rt60': (0.15, 0.9, 'log'),+ 'width': (0.0, 0.5),+ 'air_db': (-9.0, 0.0),+ 'floor_db': (-88.0, -58.0),+ 'rumble': (0.0, 0.8), # low-frequency share of the room tone (real recordings are never silent)+}+# round 2 pick 1 (launch film forge: the launch-film forge run (pluck), tone=1.5)+DEFAULTS = {'p2_ratio': 5.0, 'p2_lvl': 0.386, 'p3_lvl': 0.2038, 'tau1': 0.3529, 'tau_exp': 1.309, 'beat_hz': 2.759,+ 'beat_lvl': 0.01985, 'h2_lvl': 0.1751, 'thumb_ms': 0.3572, 'zing_lvl': 0.09041, 'zing_f': 6908.0,+ 'zing_tau': 0.006356, 'skin_lvl': 0.2027, 'skin_f': 1500.0, 'helm_f': 197.2, 'helm_tau': 0.01108,+ 'box_lvl': 0.1924, 'box_lo': 545.6, 'box_tau': 0.01542, 'knock_lvl': 0.5643, 'knock_f': 77.94,+ 'knock_tau': 0.03731, 'symp_lvl': 0.02239, 'symp_rise': 0.07959, 'room_wet': 0.2094, 'rt60': 0.4881,+ 'width': 0.0, 'air_db': -7.11, 'floor_db': -64.71, 'rumble': 0.3376}+LEN = 3.0+_NOTES = {'C': 0, 'D': 2, 'E': 4, 'F': 5, 'G': 7, 'A': 9, 'B': 11}+++def note(name, a4=440.0):+ """'C5' -> 523.25 Hz ('F#4', 'Bb3' work too)"""+ s = _NOTES[name[0].upper()]+ i = 1+ while name[i] in '#b':+ s += 1 if name[i] == '#' else -1+ i += 1+ octave = int(name[i:])+ return a4 * 2 ** ((s + 12 * (octave + 1) - 69) / 12)+++def _t(n):+ return np.arange(n) / SR+++def _mode(t, f, tau, a, r, attack):+ """one damped mode; its onset follows the thumb's release (raised-cosine over `attack` s)"""+ ph = r.uniform(0, 2 * np.pi)+ y = a * np.exp(-t / tau) * np.sin(2 * np.pi * f * t + ph * 0.15) # released from rest: near-sine phase+ k = max(2, dsp.n_of(attack))+ if k < len(y):+ y[:k] *= 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, k))+ return y+++def _tine(p, f0, n, r):+ t = _t(n)+ T = p['thumb_ms'] / 1000+ lp = lambda f: 1.0 / (1.0 + f * T * 0.5) # the soft release excites high modes less+ tau1 = p['tau1'] * (f0 / 523.25) ** -0.5+ att = max(T * 0.35, 0.0002)+ y = _mode(t, f0, tau1, 1.0, r, att)+ if p['beat_lvl'] > 0.005: # the second polarisation+ y += _mode(t, f0 + p['beat_hz'], tau1 * 0.85, p['beat_lvl'], r, att)+ f2 = f0 * p['p2_ratio']+ y += _mode(t, f2, tau1 * (f2 / f0) ** -p['tau_exp'], p['p2_lvl'] * lp(f2) / lp(f0), r, att)+ f3 = f2 * 2.8+ if p['p3_lvl'] > 0.005 and f3 < SR * 0.45:+ y += _mode(t, f3, tau1 * (f3 / f0) ** -p['tau_exp'], p['p3_lvl'] * lp(f3) / lp(f0), r, att)+ f4 = f2 * 5.5 # fourth bending mode (34.4 / 6.27)+ if p['p3_lvl'] > 0.005 and f4 < SR * 0.45:+ y += _mode(t, f4, tau1 * (f4 / f0) ** -p['tau_exp'], 0.5 * p['p3_lvl'] * lp(f4) / lp(f0), r, att)+ if p['h2_lvl'] > 0.005: # nonlinearity: squared envelope+ y += _mode(t, 2 * f0, tau1 / 2, p['h2_lvl'], r, att)+ return y+++def _symp(p, f0, n, r):+ """the other tines, driven through the bridge: strongest where a tine coincides with a partial of the note"""+ out = np.zeros(n)+ if p['symp_lvl'] < 0.003:+ return out+ t = _t(n)+ rise = 1 - np.exp(-t / p['symp_rise'])+ for ft in TINES:+ if abs(ft / f0 - 1) < 0.03:+ continue+ cents = min(abs(1200 * np.log2(ft / (f0 * h))) for h in (0.5, 1, 1.5, 2, 3, 4))+ a = p['symp_lvl'] * (0.15 + np.exp(-(cents / 30.0) ** 2)) * r.uniform(0.5, 1.0)+ tau = p['tau1'] * (ft / 523.25) ** -0.5 * 1.2+ out += a * np.sin(2 * np.pi * ft * (1 + 0.0015 * r.standard_normal()) * t + r.uniform(0, 6.3)) \+ * np.exp(-t / tau) * rise+ return out+++def _box_ir(p, r):+ """the wooden box seen from the bridge: Helmholtz mode + short plate modes"""+ n = dsp.n_of(0.25)+ t = _t(n)+ ir = np.zeros(n)+ ir[0] = 1.0+ ir += 0.9 * np.sin(2 * np.pi * p['helm_f'] * t) * np.exp(-t / p['helm_tau']) * 2 * np.pi * p['helm_f'] / SR * 6+ cnt = 18+ f = np.exp(r.uniform(np.log(p['box_lo']), np.log(2500.0), cnt))+ for fi in f:+ tau = p['box_tau'] * (fi / p['box_lo']) ** -0.5+ ir += r.uniform(0.2, 1.0) * np.sin(2 * np.pi * fi * t + r.uniform(0, 6.3)) * np.exp(-t / tau) \+ * 2 * np.pi * fi / SR * 2+ return ir+++def _zing(p, f0, n, r):+ """the tine slapping free of the thumb and rattling on the bridge saddle: band-passed noise, gated at the period+ of the note (a buzz, not a hiss), dying in tens of ms"""+ m = min(n, dsp.n_of(0.4))+ t = _t(m)+ w = dsp.butter(r.standard_normal(m), 'band', (p['zing_f'] * 0.5, min(p['zing_f'] * 1.8, SR * 0.45)), 2)+ gate = np.maximum(0.0, np.sin(2 * np.pi * f0 * t)) ** 4+ y = w * (0.35 + gate) * np.exp(-t / p['zing_tau'])+ out = np.zeros(n)+ out[:m] = y / (np.abs(y).max() + 1e-12)+ return out+++def _skin(p, n, r):+ m = dsp.n_of(0.02)+ w = r.standard_normal(m) * np.exp(-_t(m) / 0.003)+ w = dsp.butter(w, 'band', (p['skin_f'] * 0.5, min(p['skin_f'] * 2.0, SR * 0.45)), 2)+ out = np.zeros(n)+ out[:m] = w / (np.abs(w).max() + 1e-12)+ return out+++def _knock(p, n, r):+ """the thumb pushes the whole instrument: a low, heavily damped thud of the box on its support"""+ m = min(n, dsp.n_of(0.4))+ t = _t(m)+ f = p['knock_f'] * (1 + 0.05 * r.standard_normal())+ y = np.sin(2 * np.pi * f * t) * np.exp(-t / p['knock_tau']) * (1 - np.exp(-t / 0.004))+ y += 0.5 * np.sin(2 * np.pi * f * 2.3 * t) * np.exp(-t / (p['knock_tau'] * 0.5)) * (1 - np.exp(-t / 0.002))+ out = np.zeros(n)+ out[:m] = y / (np.abs(y).max() + 1e-12)+ return out+++def _room_tone(n, r, rumble):+ w = dsp.noise(n / SR, r, 'pink')[:n]+ lo = dsp.butter(dsp.noise(n / SR, r, 'brown')[:n], 'low', 120, 2)+ lo /= np.sqrt(np.mean(lo ** 2)) + 1e-12+ return (1 - rumble) * w + rumble * 2.0 * lo+++def render(p, r):+ f0 = p.get('f0') or float(r.choice(REF_F0))+ n = dsp.n_of(LEN)+ pre = dsp.n_of(0.004)+ tine = _tine(p, f0, n - pre, r)+ pk = np.abs(tine).max() + 1e-12+ # the bridge drives the box; the tine's own radiation is weak at low notes, the box carries the body+ body = signal.fftconvolve(tine, _box_ir(p, r))[: len(tine)]+ body *= pk / (np.abs(body).max() + 1e-12)+ x = (1 - 0.5 * p['box_lvl']) * tine + p['box_lvl'] * body+ x += _symp(p, f0, len(x), r) * pk+ x += p['skin_lvl'] * pk * _skin(p, len(x), r)+ x += p.get('knock_lvl', 0.0) * pk * _knock(p, len(x), r)+ if p.get('zing_lvl', 0.0) > 0.003:+ x += p['zing_lvl'] * pk * _zing(p, f0, len(x), r)+ x = np.concatenate([np.zeros(pre), x])+ x = dsp.stereo(x, r, width=p['width'], pan=0.0)+ x = dsp.room(x, r, rt60=p['rt60'], size=0.3, damp=0.6, wet=p['room_wet'])+ x = dsp.mic(x, r, low_cut=20.0, air_db=p['air_db'], floor_db=-140.0)+ x = x[:n]+ pk = np.abs(x).max()+ for c in range(x.shape[1]):+ x[:, c] += _room_tone(len(x), r, p.get('rumble', 0.0)) * pk * 10 ** (p['floor_db'] / 20)+ x = dsp.trim_tail(x, p['floor_db'] - 2, 0.05)+ return dsp.fade(x, 0.0005, min(0.4, len(x) / SR * 0.3))diff --git a/earshot/forge/recipes/pop.py b/earshot/forge/recipes/pop.pynew file mode 100644index 0000000..4aabfa9--- /dev/null+++ b/earshot/forge/recipes/pop.py@@ -0,0 +1,240 @@+"""A small bubble pop, built from physics (no recorded audio anywhere: every sample is computed).++What happens when a small bubble pops, and how each part is modelled:+ 0. Leak before the release. A lip / film seal does not open in zero time: for a few ms before the main release+ air already leaks through a growing gap - turbulence shaped by the same cavity, swelling up to the burst (real+ pops rise over several ms; a perfect step onset reads as a digital tick).+ 1. Rupture. The thin liquid wall tears and the hole opens at the Taylor-Culick speed (~10 m/s for a micron film),+ so a millimetre cap is gone in 0.1-3 ms - not in one clean step: the film tears in a few jerks (a short crackle+ of micro-ruptures). Each jerk releases part of the trapped overpressure (4 sigma / R): a positive-first pressure+ pulse with a smaller negative recovery lobe (zero net pressure). Plus film-shred noise (droplets and fragments+ thrown off) that dies within a millisecond or two. A low-pass sets how bright the tear reads at the mic.+ 2. Cavity ring (Minnaert / Helmholtz). The gas pocket left behind breathes at f = (1 / 2 pi R) sqrt(3 gamma p0 / rho)+ = 3.26 Hz*m / R: 0.5-3.5 kHz for millimetre bubbles. Damping (thermal + radiation + viscous) gives Q ~ 5-40, so it+ rings for a few ms. The pitch RISES as the pocket shrinks and its neck opens (+10-120 % within 3-80 ms). The+ amplitude builds over a fraction of a millisecond or longer (the wall must accelerate; radiation grows as the+ neck opens). Breathing is slightly nonlinear: a weak 2nd harmonic that follows the squared envelope.+ 3. Second cavity formant. A real cavity (the pocket and its neck, like a mouth) has a higher resonance too; it is+ excited by the same release, has its own (lower) Q and, being a mode of the same shrinking / opening cavity, it+ follows most of the same pitch rise (a fixed-pitch higher mode reads as a glass 'ting', not as a cavity).+ Real cavities are never perfectly still: the wall shape fluctuates, so the pitch wobbles by a few per cent.+ Air rushing through the opening also drives the same cavity with turbulence for the first milliseconds: a+ breathy band of noise that follows the ring's pitch (this is what makes a mouth / lip pop sound wet and soft).+ 4. Daughter bubble. A burst often pinches off a smaller bubble a few ms later: a second, higher, shorter ring.+ 5. Air release. The escaping gas jet and the retracting film rim give a few ms of band-limited hiss.+ 6. Puff. The net volume change is a monopole pulse: a soft low half-cycle under the tick.+ 7. Room tone before the event, a small room (size, damping of highs) and a close microphone (tilt, noise floor).++FAMILIES narrows SPACE for the kinds of pop the forge test tried (demo/forge-test/pop/): the main mouth / bubble pop,+the slow bloop (an entrained bubble: no film to tear, rings up while its pitch doubles), the small high pop (a+physically small bubble; round 7, read as a finger snap) and, from round 8, the lip pop (a cavity that opens without+the bubble's big pitch glide: little rise, lower Q, breathier). Apply one with {**SPACE, **FAMILIES[name]}.++ 8. The edit (round 8, after the judge's measurements). The kit's pops are mid-band events, edited clean: 0.19-0.41 s+ long, 40 dB down 35-145 ms after the peak, almost nothing under 150 Hz or above 4 kHz (energy shares median+ -26 / -37 / -43 dB below 150 Hz / 4-12 kHz / above 12 kHz). Rounds 1-7 ran 0.7-0.85 s over a -55 dB hiss floor+ (the realness model liked the floor) with a low puff and a full-band tick and hiss: +15-18 dB at 40-80 Hz and+ +12-19 dB at 2-16 kHz against the kit's median. So: a recording chain high-pass (hp) and gentle low-pass (lp), a+ quieter floor, and an edit cut `clip` s after the release with a short fade, never before the ring has fallen+ `cut_db` under its peak. These keys are absent in the round 1-7 parameters, which therefore still render exactly+ as before (and round 8's picks, which have no cut_db, render as round 8 wrote them).+"""+import numpy as np++from .. import dsp+from . import REF_ROOT++TARGET = 'a small bubble pop'+ALTS = ['a cork pop', 'a water drop', 'a finger snap', 'a mouse click']+REFS = [REF_ROOT + '/ui/ui_pop_0%d.wav' % i for i in (1, 2, 3, 4)]++SPACE = {+ 'f0': (400, 3500, 'log'), # cavity (Minnaert / Helmholtz) breathing frequency at the burst+ 'q': (4, 40, 'log'), # its quality factor (decay tau = q / (pi f0))+ 'rise': (-0.1, 1.4), # pitch rise as the cavity shrinks / opens (fraction of f0)+ 'rise_tau': (0.002, 0.08, 'log'), # how fast the pitch rises+ 'att': (0.0001, 0.03, 'log'), # build-up time of the ring+ 'h2': (0.0, 0.3), # nonlinear 2nd harmonic+ 'wobble': (0.0, 0.05), # random pitch fluctuation of the cavity (fraction)+ 'breath_amp': (0.0, 1.0), # turbulent air driving the cavity+ 'breath_tau': (0.001, 0.03, 'log'),+ 'f2_ratio': (1.6, 4.5), # second cavity formant / f0+ 'f2_amp': (0.0, 0.8),+ 'f2_q': (3, 25, 'log'),+ 'f2_follow': (0.5, 1.0), # share of the pitch rise the formant follows (same cavity)+ 'sub_amp': (0.0, 0.6), # daughter bubble+ 'sub_ratio': (1.2, 2.6),+ 'sub_delay': (0.002, 0.03, 'log'),+ 'click_amp': (0.0, 1.5), # rupture pressure pulses+ 'click_w': (0.00005, 0.0006, 'log'),+ 'tear': (0.0001, 0.003, 'log'), # time over which the film tears (spread of the micro-ruptures)+ 'jerks': (1, 6, 'int'),+ 'click_lp': (2500, 20000, 'log'),+ 'shred_amp': (0.0, 1.0), # film-shred noise at the rupture+ 'shred_tau': (0.0002, 0.003, 'log'),+ 'shred_hp': (500, 8000, 'log'),+ 'hiss_amp': (0.0, 0.6), # air release+ 'hiss_f': (1500, 10000, 'log'),+ 'hiss_q': (0.5, 3.0),+ 'hiss_tau': (0.001, 0.03, 'log'),+ 'puff_amp': (0.0, 1.0), # low monopole puff+ 'puff_f': (60, 400, 'log'),+ 'ring_delay': (0.0, 0.003), # ring starts slightly after the first rupture+ 'rt60': (0.08, 0.9, 'log'),+ 'wet': (0.0, 0.6),+ 'damp': (0.3, 0.95), # how much faster the room's highs die+ 'size': (0.05, 0.8),+ 'air_db': (-12.0, 2.0), # mic / distance tilt at 10 kHz+ 'floor_db': (-80, -55),+ 'swell_amp': (0.0, 0.6), # leak before the release+ 'swell_t': (0.0005, 0.012, 'log'),+ 'pre': (0.004, 0.035), # room tone before the event (a recording starts before the pop)+ 'hp': (40, 160, 'log'), # 8. recording chain high-pass (order 4)+ 'lp': (4000, 14000, 'log'), # and a gentle low-pass (order 2): a close mic on a mid-band event+ 'clip': (0.14, 0.38), # edit point after the release (the kit: 0.19-0.41 s files)+ 'cut_fade': (0.004, 0.03, 'log'), # the edit's fade+ 'cut_db': (-56, -42), # never cut before the ring has fallen this far under its peak (the kit's+} # pops are cut at -46..-62 dB; round 8's bloops were cut at -26 dB)+# round 8 limits (measured on the kit's pops; see 8. above): no -55 dB hiss bed, no big low puff, no full-band tick+SPACE.update({'floor_db': (-86, -64), 'puff_amp': (0.0, 0.35), 'puff_f': (120, 400, 'log'), 'click_lp': (2000, 9000, 'log'),+ 'shred_amp': (0.0, 0.5), 'shred_hp': (500, 5000, 'log'), 'hiss_f': (1500, 6000, 'log'),+ 'rt60': (0.05, 0.45, 'log'), 'wet': (0.0, 0.45)})+# the main pop after the judge (round 8 main pick 2 = final pop_01, parameters rounded to 6 digits): 8 new seeds ->+# critic mean 6.76 with the tone and region checks in the total, 7/8 pass; 0.27 s, cut clean, band-limited like the+# kit's pops+DEFAULTS = {'f0': 440.159, 'q': 22.1207, 'rise': 0.991892, 'rise_tau': 0.0159722, 'att': 0.000357098,+ 'h2': 0.00847939, 'wobble': 0.0112906, 'breath_amp': 0.0, 'breath_tau': 0.00848874, 'f2_ratio': 3.82514,+ 'f2_amp': 0.8, 'f2_q': 25.0, 'f2_follow': 0.948223, 'sub_amp': 0.00328062, 'sub_ratio': 2.08913,+ 'sub_delay': 0.0111736, 'click_amp': 0.685048, 'click_w': 7.72837e-05, 'tear': 0.000171419, 'jerks': 1,+ 'click_lp': 4881.31, 'shred_amp': 0.200779, 'shred_tau': 0.00153019, 'shred_hp': 5000.0,+ 'hiss_amp': 0.0659931, 'hiss_f': 2418.88, 'hiss_q': 2.86992, 'hiss_tau': 0.00113073, 'puff_amp': 0.0,+ 'puff_f': 121.063, 'ring_delay': 0.00105542, 'rt60': 0.249181, 'wet': 0.291252, 'damp': 0.490701,+ 'size': 0.05, 'air_db': -0.476997, 'floor_db': -64.4367, 'swell_amp': 0.160452, 'swell_t': 0.00252584,+ 'pre': 0.0269276, 'hp': 92.6866, 'lp': 10181.4, 'clip': 0.274425, 'cut_fade': 0.0153059}++_SOFT = {'swell_amp': (0.15, 0.6), 'swell_t': (0.002, 0.012, 'log')}+FAMILIES = {+ 'main': {**_SOFT, 'click_amp': (0.0, 0.7)},+ 'bloop': {'f0': (350, 900, 'log'), 'rise': (0.7, 1.4), 'rise_tau': (0.025, 0.08, 'log'), 'att': (0.006, 0.04, 'log'),+ 'q': (18, 40, 'log'), 'click_amp': (0.0, 0.35), 'shred_amp': (0.0, 0.2), 'ring_delay': (0.0, 0.025)},+ 'small': {**_SOFT, 'f0': (1400, 3500, 'log'), 'rise': (0.0, 1.0), 'click_amp': (0.0, 0.8), 'shred_amp': (0.0, 0.45)},+ 'lip': {'f0': (450, 1300, 'log'), 'rise': (-0.1, 0.35), 'rise_tau': (0.005, 0.05, 'log'), 'q': (4, 14, 'log'),+ 'breath_amp': (0.2, 1.0), 'att': (0.0003, 0.004, 'log'), 'click_amp': (0.0, 0.6), 'sub_amp': (0.0, 0.2)},+}++DUR = 0.35+++def rupture_pulse(w, n):+ """positive raised-cosine pulse of width w followed by a negative lobe of 2.5 w at matching area (zero DC)"""+ a, b = max(2, dsp.n_of(w)), max(3, dsp.n_of(2.5 * w))+ pos = 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, a))+ neg = 0.5 - 0.5 * np.cos(np.linspace(0, 2 * np.pi, b))+ neg *= -pos.sum() / neg.sum()+ x = np.zeros(n)+ m = min(n, a + b)+ x[:m] = np.concatenate([pos, neg])[:m]+ return x+++def tear_crackle(w, tear, jerks, n, r):+ """the film tears in `jerks` steps spread over `tear` s; the first is strongest, later ones smaller"""+ x = np.zeros(n)+ one = rupture_pulse(w, n)+ times = np.sort(np.concatenate([[0.0], r.uniform(0, tear, max(0, jerks - 1))]))+ for k, tk in enumerate(times):+ amp = (1.0 if k == 0 else r.uniform(0.25, 0.8) * 0.8 ** k) * r.choice([1, 1, 1, -1]) ** (k > 0)+ dsp.place(x, one * amp, tk)+ return x+++def wobble_curve(depth, n, r, rate=60.0):+ """slow random pitch fluctuation (low-passed noise, unit RMS scaled to depth)"""+ if depth <= 0:+ return np.zeros(n)+ w = dsp.butter(r.standard_normal(n), 'low', rate, order=2)+ return depth * w / (np.sqrt(np.mean(w ** 2)) + 1e-12)+++def pitch_track(f0, rise, rise_tau, n, wob):+ t = dsp.t_of(n)+ return f0 * (1 + rise * (1 - np.exp(-t / rise_tau))) * (1 + wob)+++def gliding_ring(f0, q, rise, rise_tau, att, h2, n, r, wob=None):+ """damped breathing mode whose pitch rises f0 -> f0 (1 + rise); 2nd harmonic follows the squared envelope"""+ t = dsp.t_of(n)+ f = pitch_track(f0, rise, rise_tau, n, 0.0 if wob is None else wob)+ ph = 2 * np.pi * np.cumsum(f) / dsp.SR + r.uniform(-0.3, 0.3)+ tau = q / (np.pi * f0)+ env = (1 - np.exp(-t / att)) * np.exp(-t / tau)+ return env * np.sin(ph) + h2 * env ** 2 * np.sin(2 * ph + 0.5)+++def render(p, r):+ n = dsp.n_of(DUR)+ t = dsp.t_of(n)+ x = np.zeros(n)+ # 1. rupture: torn-film pressure pulses (band-limited at the mic) + film shred noise+ tick = tear_crackle(p['click_w'], p['tear'], p['jerks'], n, r)+ if p['click_lp'] < 19000:+ tick = dsp.butter(tick, 'low', p['click_lp'], order=2)+ x += p['click_amp'] * tick+ shred = dsp.butter(r.standard_normal(n), 'high', p['shred_hp']) * np.exp(-t / p['shred_tau'])+ x += p['shred_amp'] * 0.5 * shred+ # 2. cavity ring + 3. its second formant (start a little after the first rupture)+ wob = wobble_curve(p['wobble'], n, r)+ body = gliding_ring(p['f0'], p['q'], p['rise'], p['rise_tau'], p['att'], p['h2'], n, r, wob)+ f2 = p['f0'] * p['f2_ratio']+ body += p['f2_amp'] * gliding_ring(f2, p['f2_q'], p['rise'] * p['f2_follow'], p['rise_tau'], p['att'], 0.0, n, r,+ wob)+ # turbulent air through the opening drives the same (gliding) cavity: a breathy band following the pitch+ if p['breath_amp'] > 0.01:+ fc = pitch_track(p['f0'], p['rise'], p['rise_tau'], n, wob)+ br = dsp.svf(r.standard_normal(n), fc, max(2.0, p['q'] * 0.5), 'band')+ br *= (1 - np.exp(-t / max(p['att'], 0.0003))) * np.exp(-t / p['breath_tau'])+ body += p['breath_amp'] * br / (np.abs(br).max() + 1e-9)+ # 4. daughter bubble: higher, shorter, less glide+ if p['sub_amp'] > 0.01:+ sub = gliding_ring(p['f0'] * p['sub_ratio'], p['q'] * 0.7, p['rise'] * 0.4, p['rise_tau'] * 0.6,+ max(p['att'], 0.0002), p['h2'], n, r)+ dsp.place(body, p['sub_amp'] * sub, p['sub_delay'])+ dsp.place(x, body, p['ring_delay'])+ # 5. air release hiss+ hiss = dsp.svf(r.standard_normal(n), p['hiss_f'], p['hiss_q'], 'band')+ hiss *= (1 - np.exp(-t / 0.0003)) * np.exp(-t / p['hiss_tau'])+ x += p['hiss_amp'] * hiss / (np.abs(hiss).max() + 1e-9)+ # 6. low monopole puff: one damped low cycle+ puff = np.sin(2 * np.pi * p['puff_f'] * t) * np.exp(-t * p['puff_f'] * 2.5) * (1 - np.exp(-t / 0.0004))+ x += p['puff_amp'] * puff+ # 0. leak before the release: cavity-shaped turbulence swelling up to the burst (-43 dB -> 0 dB)+ if p['swell_amp'] > 0.01:+ m = dsp.n_of(p['swell_t'])+ w = r.standard_normal(m)+ leak = dsp.svf(w, p['f0'], 3.0, 'band') + 0.6 * dsp.svf(w, p['f0'] * p['f2_ratio'], 2.0, 'band')+ leak += 0.3 * dsp.butter(w, 'high', 3000) * 3.0+ leak *= np.exp(5.0 * (np.arange(m) / m - 1.0))+ x = np.concatenate([p['swell_amp'] * leak / (np.abs(leak).max() + 1e-9), x])+ # 7. room tone before the event, room, mic+ lead = len(x) - n # the leak swell, if any, comes before the release+ x = np.concatenate([np.zeros(dsp.n_of(p['pre'])), x])+ x = dsp.room(x, r, rt60=p['rt60'], size=p['size'], damp=p['damp'], wet=p['wet'])+ if p.get('clip') is None: # rounds 1-7: unchanged+ x = dsp.mic(x, r, air_db=p['air_db'], floor_db=p['floor_db'])+ x = dsp.trim_tail(x, -62, 0.02)+ return dsp.fade(x, 0.001, 0.02)+ # 8. the recording chain and the edit+ x = dsp.butter(x, 'high', p['hp'], order=4)+ x = dsp.butter(x, 'low', p['lp'], order=2)+ x = dsp.mic(x, r, air_db=p['air_db'], floor_db=p['floor_db'])+ end = dsp.n_of(p['pre']) + lead + dsp.n_of(p['clip'])+ if p.get('cut_db') is not None: # an editor cuts once the ring has died away+ h = dsp.n_of(0.005)+ mono = x if x.ndim == 1 else x.mean(axis=1)+ k = len(mono) // h+ e = 20 * np.log10(np.sqrt((mono[:k * h].reshape(k, h) ** 2).mean(1)) + 1e-12)+ i = int(np.argmax(e))+ below = np.nonzero(e[i:] < e[i] + p['cut_db'])[0]+ died = (i + (below[0] if len(below) else k - i)) * h + dsp.n_of(p['cut_fade'])+ end = min(len(x), max(end, died), dsp.n_of(p['pre']) + lead + dsp.n_of(0.42))+ return dsp.fade(x[:end], 0.001, p['cut_fade'])diff --git a/earshot/forge/recipes/whoosh.py b/earshot/forge/recipes/whoosh.pynew file mode 100644index 0000000..75e1a71--- /dev/null+++ b/earshot/forge/recipes/whoosh.py@@ -0,0 +1,230 @@+"""A fast air whoosh: an object (stick, cane, arm, cloth edge) swung fast past a close microphone, from aeroacoustics.++Every sample is computed; no recorded audio is loaded, sampled or convolved. Physics modelled:+ * Speed profile. The swing accelerates, peaks at the bottom of the arc (near the mic) and decelerates: U(t) is an+ asymmetric generalised bell exp(-|dt/s|^beta / 2) around the pass time tc (rise width s_rise, fall width s_fall;+ beta > 2 is a flatter top with sharper edges, beta < 2 a pointed peak).+ * Vortex shedding (the body of a whoosh). A bluff object sheds vortices at f = St * U / d (Strouhal ~0.2); at high+ Reynolds numbers the Aeolian tone smears into a band of width f/Q. A swung object has a speed gradient along its+ length (the tip moves fastest), so each strip sheds at its own frequency: K strips from ROOT to the tip give K bands+ at f_peak * U(t)^f_track * s_k (f_track < 1 lets the ridge hold steadier, as a body / cavity resonance does). A+ dipole radiates intensity ~ U^6, so each strip's amplitude goes ~ (s_k U)^3 and the tip band dominates: energy+ piles up below a resonant ridge at the tip frequency, as in real recordings. Loudness ~ U^a_pow / r(t). Each band+ falls steeply above its peak (`steep`, a second low-pass stage), as a turbulent cylinder wake's spectrum does+ above its Strouhal peak: the real recordings show a -23..-31 dB/oct cliff above the ridge.+ * Wake. Behind the object the shed vortices keep turning over as a decaying turbulent wake: a narrow noise band+ (wake_q) that starts at the pass, stays where the object passed (no Doppler), decays with wake_tau and darkens as+ it decays (large eddies outlive small ones, so the highs die first and the centre sinks: f ~ (1 + dt /+ wake_tau)^-wake_dark). This is the flat ridge that outlasts the hiss in real whooshes. It is bounded below the+ shedding level (a wake cannot out-radiate the shedding at full speed).+ * Proximity and Doppler. Distance r(t) = sqrt(1 + ((t - tc) / prox_w)^2) (prox_w = closest distance / pass speed);+ amplitude ~ 1/r; frequencies scaled by 1 - doppler * radial speed (higher on approach, lower after the pass).+ * Hiss. Small-scale trailing-edge turbulence: white + pink noise above a steep edge (hiss_hp) under a low-pass that+ opens with U; level ~ U^hiss_pow (a steep power makes the crisp, short top of a fast pass). Real recordings keep+ a dark valley between the ridge and ~2 kHz, so the hiss starts above it.+ * Wake hiss tail. The small eddies of the wake die fastest but not instantly: a broadband tail above the same edge,+ decaying with 0.4 * wake_tau, at wake_hiss_db.+ * Push. A close pass shoves the air at the diaphragm: a short sub-bass pressure gust centred on the pass (low-passed+ turbulence under a window of width push_w plus the smooth bipolar pressure pulse of the displaced air).+ * Buffet. The passing pressure field and the wake gust hit the diaphragm: random low-frequency pressure+ (< buffet_f), arriving buffet_delay after the pass and dying with buffet_tau.+ * Turbulence flutter. Random log-normal amplitude modulation (bandwidth flut_f) on the shedding and hiss.+ * Whip tone. A thin, smooth object keeps a narrower Aeolian tone (Q whip_q) at whip_ratio * tip frequency.+ * Stereo. The source moves across the mic: a mono-compatible balance pan follows tanh((t - tc) / prox_w) plus a+ little decorrelated side signal (the mono sum is unchanged, so stereo cannot game a mono critic); ranges follow+ the real references (balance swings of a few dB, L/R correlation ~0.9).+ * A small room (independent reflections per channel) and a microphone (low cut, top roll-off air_db, noise floor,+ recorder low-pass at 19 kHz).++ * The edit (round 12, after the judge). The kit's fast whooshes are edited files of 0.32-0.5 s that fall 40 dB in+ 90-180 ms after the pass; rounds 1-11 ran 0.68-0.9 s with t40 230-285 ms ('medium' whooshes: their critic also+ compared them with the three medium refs). `cut_db`: the file is cut once the envelope has fallen that far under+ its peak, with a `cut_fade` fade. Parameters without these keys (rounds 1-11) render exactly as before.++What the listening loop taught (demo/forge-test/whoosh/LOG.md): the critic's description match rewards longer,+fuller tails than the real fast whooshes have; when the wake was unbounded it grew as loud as the pass itself and+doubled the tail (found by muting components one at a time), hence the physical bounds on wake level and Q.+"""+import numpy as np++from .. import dsp+from . import REF_ROOT++TARGET = 'a fast crisp air whoosh'+ALTS = ['a wind gust', 'rain', 'a sword swing', 'a jet engine']+# round 12: the fast references only (the target is a FAST whoosh; rounds 1-11 also used whoosh_med_01-03, whose+# longer tails the critic then rewarded). REFS_ROUNDS_1_11 is the old set, kept for reproducing those reports.+REFS = [f'{REF_ROOT}/whoosh/whoosh_fast_0{i}.wav' for i in range(1, 5)]+REFS_ROUNDS_1_11 = REFS + [f'{REF_ROOT}/whoosh/whoosh_med_0{i}.wav' for i in range(1, 4)]++SPACE = {+ 'tc': (0.06, 0.2), # pass time (s)+ 's_rise': (0.012, 0.08, 'log'), # speed bell widths (s)+ 's_fall': (0.015, 0.1, 'log'),+ 'beta': (1.2, 3.5), # bell shape+ 'prox_w': (0.004, 0.06, 'log'), # closest distance / speed (s)+ 'f_peak': (150, 1500, 'log'), # tip shedding frequency at full speed (Hz)+ 'f_track': (0.0, 1.0), # ridge frequency ~ U^f_track (1: pure Strouhal, 0: fixed resonance)+ 'q_shed': (1.5, 12.0, 'log'),+ 'steep': (0.0, 1.0), # 4-pole skirt above the shedding peak (0: plain 2-pole band)+ 'a_pow': (1.0, 4.0), # loudness ~ U^a_pow+ 'doppler': (0.0, 0.08), # pass speed / speed of sound+ 'wake_db': (-24, -6), # round 9: a wake cannot out-radiate the shedding at full speed+ 'wake_tau': (0.03, 0.15, 'log'),+ 'wake_q': (4.0, 16.0, 'log'), # the wake ridge is narrow and steady in real recordings (Q 5-10)+ 'wake_dark': (0.0, 1.0),+ 'wake_hiss_db': (-36, -16), # broadband wake tail vs shedding+ 'hiss_db': (-38, -14),+ 'hiss_hp': (800, 3000, 'log'), # lower edge of the small-scale hiss+ 'hiss_f': (1500, 14000, 'log'),+ 'hiss_pow': (2.0, 7.0),+ 'push_db': (-24, 6), # close-pass pressure gust vs shedding+ 'push_w': (0.006, 0.04, 'log'),+ 'push_f': (40, 250, 'log'),+ 'buffet_db': (-30, 3),+ 'buffet_f': (60, 500, 'log'),+ 'buffet_delay': (0.0, 0.04),+ 'buffet_tau': (0.015, 0.08, 'log'),+ 'flut_depth': (0.0, 0.4),+ 'flut_f': (8, 60, 'log'),+ 'whip_db': (-45, -15),+ 'whip_q': (8, 60, 'log'),+ 'whip_ratio': (1.0, 3.0),+ 'pan_w': (0.05, 0.25), # real refs: balance swings of 1-6 dB, L/R correlation mostly 0.9-1.0+ 'width': (0.0, 0.12),+ 'room_wet': (0.0, 0.35),+ 'rt60': (0.1, 0.5, 'log'),+ 'air_db': (-12, 0), # mic / recorder top roll-off at 10 kHz+ 'floor_db': (-95, -70), # mic / preamp noise floor relative to the peak+ 'cut_db': (-52, -38), # round 12: the edit, once the envelope has fallen this far under its peak+ 'cut_fade': (0.015, 0.06, 'log'), # (the kit's fast whooshes: 0.32-0.5 s files, t40 90-180 ms)+}+# = final/whoosh_01 after the judge (round 12 pick 1, parameters rounded to 6 digits): fast refs only, tone check in+# the objective; 8 new seeds -> critic mean 8.02, 8/8 pass; t40 125 ms, 0.31 s+DEFAULTS = {'tc': 0.0706346, 's_rise': 0.0380857, 's_fall': 0.0578346, 'beta': 1.49847, 'prox_w': 0.0243735,+ 'f_peak': 398.391, 'f_track': 0.108498, 'q_shed': 7.05767, 'steep': 0.23086, 'a_pow': 2.90371,+ 'doppler': 0.00143736, 'wake_db': -14.0966, 'wake_tau': 0.0532322, 'wake_q': 4.45435,+ 'wake_dark': 0.613911, 'wake_hiss_db': -16.0, 'hiss_db': -19.345, 'hiss_hp': 1671.88, 'hiss_f': 9908.19,+ 'hiss_pow': 2.57151, 'push_db': -4.56985, 'push_w': 0.0251241, 'push_f': 125.898, 'buffet_db': -9.18542,+ 'buffet_f': 95.0096, 'buffet_delay': 0.0219106, 'buffet_tau': 0.0262978, 'flut_depth': 0.0178753,+ 'flut_f': 30.2817, 'whip_db': -16.8109, 'whip_q': 22.6043, 'whip_ratio': 1.75668, 'pan_w': 0.102279,+ 'width': 0.0730709, 'room_wet': 0.158579, 'rt60': 0.138233, 'air_db': -12.0, 'floor_db': -75.3278,+ 'cut_db': -46.4213, 'cut_fade': 0.0281062}++LEN = 0.8+K = 6+ROOT = 0.45 # innermost shedding strip (fraction of tip speed); no effect on the score in round 2, fixed+WAKE_RATIO = 0.9 # wake centre / tip shedding frequency at the pass+HISS_PINK = 0.5+++def _lvl(x):+ """peak of a 10 ms RMS envelope (component level for mixing)"""+ n = dsp.n_of(0.01)+ k = len(x) // n+ return float(np.sqrt((x[:k * n].reshape(k, n) ** 2).mean(1)).max()) + 1e-12+++def _lfnoise(n, fc, r):+ """unit-std random control signal band-limited to fc (for flutter / jitter)"""+ w = dsp.butter(r.standard_normal(n), 'low', fc, order=2)+ return w / (w.std() + 1e-12)+++def _band(n, fc, q, r, steep=0.0):+ """white noise through a moving band-pass, scaled to unity peak gain (power ~ bandwidth). steep 0..1 blends in a+ second, low-pass stage tuned just above fc: the skirt above the peak falls at 18-24 dB/oct instead of 6, as the+ spectrum of a turbulent cylinder wake does above its Strouhal peak"""+ y = dsp.svf(r.standard_normal(n), fc, q, 'band') / q+ if steep > 0:+ y = (1 - steep) * y + steep * dsp.svf(y, np.asarray(fc) * 1.08, 0.9, 'low')+ return y+++def render(p, r):+ n = dsp.n_of(LEN)+ t = dsp.t_of(n)+ tc = p['tc']+ dt = t - tc+ s = np.where(dt < 0, p['s_rise'], p['s_fall'])+ U = np.exp(-0.5 * np.abs(dt / s) ** p['beta']) # speed / max speed+ rr = np.sqrt(1 + (dt / p['prox_w']) ** 2) # distance / closest distance+ v_rad = (dt / p['prox_w']) / rr * U # radial speed fraction (+ receding)+ dop = 1.0 - p['doppler'] * v_rad+ amp = U ** p['a_pow'] / rr+ flut = np.exp(p['flut_depth'] * _lfnoise(n, p['flut_f'], r) - 0.5 * p['flut_depth'] ** 2)++ # vortex shedding along the span+ f_tip = np.maximum(p['f_peak'] * np.maximum(U, 1e-3) ** p['f_track'] * dop, 30.0)+ shed = np.zeros(n)+ for k in range(1, K + 1):+ sk = ROOT + (1 - ROOT) * k / K+ shed += sk ** 3 * _band(n, np.maximum(f_tip * sk, 25.0), p['q_shed'], r, p['steep'])+ shed *= amp * flut++ # decaying, darkening wake behind the object+ after = np.maximum(dt, 0.0)+ w_on = 0.5 + 0.5 * np.tanh(dt / max(0.25 * p['s_rise'], 0.003))+ w_env = w_on * np.exp(-after / p['wake_tau'])+ f_w = np.maximum(p['f_peak'] * WAKE_RATIO * (1 + after / p['wake_tau']) ** (-p['wake_dark']), 40.0)+ wake = _band(n, f_w, p['wake_q'], r, p['steep']) * w_env # stationary in space: no Doppler++ # small-scale wake hiss+ w = (1 - HISS_PINK) * r.standard_normal(n) + HISS_PINK * dsp.noise(LEN, r, 'pink')[:n] * 3.0+ hiss = dsp.svf(w, np.maximum(p['hiss_f'] * U ** 0.7 * dop, 400.0), 0.6, 'low')+ hiss = dsp.butter(hiss, 'high', p['hiss_hp'], order=4) * U ** p['hiss_pow'] / rr * flut++ # wake hiss tail+ wh = dsp.butter(dsp.butter(dsp.noise(LEN, r, 'pink')[:n], 'high', p['hiss_hp'], order=4), 'low', 9000, order=2)+ wh = wh * w_on * np.exp(-after / (0.4 * p['wake_tau']))++ # push: the close pass shoves air at the diaphragm+ pw = p['push_w']+ win = np.exp(-0.5 * (dt / pw) ** 2)+ ricker = (1 - (dt / (0.35 * pw)) ** 2) * np.exp(-0.5 * (dt / (0.35 * pw)) ** 2)+ push = dsp.butter(dsp.noise(LEN, r, 'brown')[:n], 'low', p['push_f'], order=2) * win+ push = push / _lvl(push) + 0.7 * dsp.butter(ricker, 'low', p['push_f'] * 2, order=2) / _lvl(ricker)++ # buffet: wake gust on the diaphragm+ tb = t - (tc + p['buffet_delay'])+ benv = np.where(tb < 0, np.exp(-0.5 * (tb / max(p['s_rise'] * 0.5, 0.004)) ** 2), np.exp(-tb / p['buffet_tau']))+ buf = dsp.butter(dsp.noise(LEN, r, 'brown')[:n], 'low', p['buffet_f'], order=2) * benv++ # whip tone+ whip = _band(n, np.minimum(f_tip * p['whip_ratio'], 16000), p['whip_q'], r) * U ** 4 / rr++ x = shed / _lvl(shed)+ x += 10 ** (p['wake_db'] / 20) * wake / _lvl(wake)+ x += 10 ** (p['hiss_db'] / 20) * hiss / _lvl(hiss)+ x += 10 ** (p['wake_hiss_db'] / 20) * wh / _lvl(wh)+ x += 10 ** (p['push_db'] / 20) * push / _lvl(push)+ x += 10 ** (p['buffet_db'] / 20) * buf / _lvl(buf)+ x += 10 ** (p['whip_db'] / 20) * whip / _lvl(whip)++ # moving source -> stereo+ side_dir = 1.0 if r.uniform() < 0.5 else -1.0+ pan = side_dir * p['pan_w'] * np.tanh(dt / p['prox_w'])+ d1 = int(r.uniform(0.2, 0.9) * 1e-3 * dsp.SR)+ side = np.zeros(n)+ side[d1:] = x[:n - d1]+ side = dsp.butter(side - x, 'high', 300) * p['width']+ L = x * (1 + pan) + side # L + R = 2x: mono sum unchanged+ R = x * (1 - pan) - side+ out = []+ for ch in (L, R):+ y = dsp.room(ch, r, rt60=p['rt60'], size=0.4, wet=p['room_wet'])+ y = dsp.mic(y, r, low_cut=25.0, air_db=p['air_db'], floor_db=p['floor_db'])+ y = dsp.butter(y, 'low', 19000, order=4)+ out.append(y[:n + dsp.n_of(0.1)])+ m = min(len(o) for o in out)+ y = np.stack([o[:m] for o in out], axis=1)+ if p.get('cut_db') is None: # rounds 1-11: unchanged+ return dsp.fade(dsp.trim_tail(y, -62, 0.03), 0.002, 0.06)+ h = dsp.n_of(0.005) # round 12: the edit+ mono = y.mean(axis=1)+ k = len(mono) // h+ e = 20 * np.log10(np.sqrt((mono[:k * h].reshape(k, h) ** 2).mean(1)) + 1e-12)+ i = int(np.argmax(e))+ below = np.nonzero(e[i:] < e[i] + p['cut_db'])[0]+ end = min(len(y), (i + (below[0] if len(below) else k - i)) * h + dsp.n_of(p['cut_fade']))+ return dsp.fade(y[:end], 0.002, p['cut_fade'])diff --git a/earshot/forge/search.py b/earshot/forge/search.pynew file mode 100644index 0000000..e98bbdf--- /dev/null+++ b/earshot/forge/search.py@@ -0,0 +1,171 @@+"""The listen-and-improve loop: an evolutionary search over a recipe's parameters, scored by the Critic.++ from earshot.forge.search import optimize+ res = optimize(recipe, critic, budget=160, pop=16, seed=0)+ res['best'][0]['params'], res['best'][0]['score']['summary'], res['history']++A recipe is a module (or object) with+ SPACE {name: (lo, hi)} | {name: (lo, hi, 'log')} | {name: (lo, hi, 'int')} | {name: [choice, ...]}+ DEFAULTS {name: value} (a sensible starting sound; optional)+ render(params, r) -> np.ndarray (48 kHz mono or stereo float; r = numpy Generator for natural variation)+Each generation renders `pop` candidates (elites mutated with a shrinking step, plus a few fresh random ones), peak-+normalises them to -1 dBFS (so level never decides), scores them in one batch, and keeps the best. Deterministic for a+given seed. Every candidate is kept in the history, so an agent can read what helped.++`space=` restricts the search to a family (a narrower {name: spec} dict; parameters it leaves out keep the recipe's+DEFAULTS). `robust()` re-renders a result on new noise seeds: the best single render of a search is a winner's curse+(the forge makers measured robust means 0.1-2.2 below the reported best), so judge finals by their robust mean.+`reps=N` fights that inside the search: every candidate is rendered on N seeds and ranked by its mean total (it passes+when at least half of its renders pass); the row kept is the first render's, with 'total_reps' listing all N.+"""+import math+import sys+import traceback++import numpy as np++from . import dsp+++def _decode(space, u):+ p = {}+ for (k, spec), v in zip(space.items(), u):+ v = float(np.clip(v, 0.0, 1.0))+ if isinstance(spec, list):+ p[k] = spec[min(len(spec) - 1, int(v * len(spec)))]+ else:+ lo, hi = spec[0], spec[1]+ kind = spec[2] if len(spec) > 2 else 'lin'+ if kind == 'log':+ p[k] = float(math.exp(math.log(lo) + v * (math.log(hi) - math.log(lo))))+ elif kind == 'int':+ p[k] = int(round(lo + v * (hi - lo)))+ else:+ p[k] = float(lo + v * (hi - lo))+ return p+++def _encode(space, params):+ u = []+ for k, spec in space.items():+ v = params.get(k)+ if v is None:+ u.append(0.5)+ elif isinstance(spec, list):+ u.append((spec.index(v) + 0.5) / len(spec) if v in spec else 0.5)+ else:+ lo, hi = spec[0], spec[1]+ kind = spec[2] if len(spec) > 2 else 'lin'+ if kind == 'log':+ u.append((math.log(max(v, 1e-12)) - math.log(lo)) / (math.log(hi) - math.log(lo)))+ else:+ u.append((v - lo) / (hi - lo))+ return np.clip(np.array(u, float), 0, 1)+++_failed = set()+++def render(recipe, params, seed):+ """render one candidate, peak-normalised to -1 dBFS; None if it fails or is silent. The first exception of each+ recipe is printed to stderr (a recipe bug must not look like 'render failed or silent' candidates)."""+ try:+ x = np.asarray(recipe.render(dict(params), dsp.rng(seed)), np.float64)+ except Exception as e:+ name = getattr(recipe, '__name__', type(recipe).__name__)+ if name not in _failed:+ _failed.add(name)+ tb = traceback.extract_tb(e.__traceback__)[-1]+ sys.stderr.write(f'forge: recipe {name} raised {type(e).__name__}: {e} ({tb.filename}:{tb.lineno}); '+ 'later failures of this recipe are not printed' + chr(10))+ return None+ if not len(x) or not np.isfinite(x).all() or np.abs(x).max() < 1e-7:+ return None+ return dsp.normalize(x, -1.0).astype(np.float32)+++def optimize(recipe, critic, budget=160, pop=16, elite=4, seed=0, init=None, sigma0=0.22, sigma1=0.04,+ fresh=0.15, log=None, space=None, reps=1):+ """returns {'best': [top results], 'history': [every result], 'generations': n}; a result = {params, seed, score}.+ space: optional narrower {name: spec} (a family); parameters it leaves out keep the recipe's DEFAULTS.+ reps: renders per candidate (ranked by the mean total; budget counts candidates, so reps=2 costs twice the time)"""+ base = {}+ if space is None:+ space = recipe.SPACE+ else:+ base = dict(getattr(recipe, 'DEFAULTS', {}) or {})+ R = np.random.default_rng(seed)+ d = len(space)+ starts = [dict(getattr(recipe, 'DEFAULTS', {}) or {})] + [dict(p) for p in (init or [])]+ gens = max(1, budget // pop)+ history, elites = [], []+ for g in range(gens):+ sig = sigma0 * (sigma1 / sigma0) ** (g / max(1, gens - 1))+ U, seeds = [], []+ for j in range(pop):+ if g == 0:+ u = _encode(space, starts[j]) if j < len(starts) else R.uniform(0, 1, d)+ elif R.uniform() < fresh or not elites:+ u = R.uniform(0, 1, d)+ else:+ parent = elites[min(len(elites) - 1, int(R.integers(0, max(1, min(elite, len(elites))))))]+ u = np.clip(parent['u'] + R.normal(0, sig, d), 0, 1)+ U.append(u)+ seeds.append(int(R.integers(0, 2 ** 31)))+ params = [{**base, **_decode(space, u)} for u in U]+ jobs = [(j, s + k * 7919) for j, s in enumerate(seeds) for k in range(max(1, reps))]+ xs = [render(recipe, params[j], s) for j, s in jobs]+ live = [i for i, x in enumerate(xs) if x is not None]+ scores = critic.score([xs[i] for i in live]) if live else []+ got = {}+ for i, sc in zip(live, scores):+ got.setdefault(jobs[i][0], []).append(sc)+ by = {}+ for j, rows in got.items():+ if len(rows) < max(1, reps): # a failed render among the reps: the candidate is unreliable+ continue+ if reps > 1:+ tot = [float(r['total']) for r in rows]+ row = dict(rows[0])+ row['total_reps'] = [round(t, 3) for t in tot]+ row['total'] = round(float(np.mean(tot)), 3) if all(np.isfinite(tot)) else -np.inf+ row['passes'] = sum(1 for r in rows if r.get('passes')) * 2 >= len(rows)+ by[j] = row+ else:+ by[j] = rows[0]+ for j in range(pop):+ sc = by.get(j, {'total': -np.inf, 'passes': False, 'summary': 'render failed or silent'})+ history.append({'gen': g, 'u': U[j], 'params': params[j], 'seed': seeds[j], 'score': sc})+ ranked = sorted([h for h in history if np.isfinite(h['score']['total'])], key=lambda h: -h['score']['total'])+ elites = ranked[:elite]+ if log and elites:+ b = elites[0]['score']+ log(f"gen {g + 1}/{gens}: best total {b['total']:.2f} ({'PASS' if b['passes'] else 'no pass'}) "+ f"z {b.get('match_z')} p_real {b.get('p_real')} typ {b.get('ref_typ_z')} pen {b.get('feat_pen')}")+ ranked = sorted([h for h in history if np.isfinite(h['score']['total'])], key=lambda h: -h['score']['total'])+ return {'best': ranked[:max(elite, 8)], 'history': history, 'generations': gens}+++def robust(recipe, critic, params, seeds=4, seed0=5000):+ """re-render `params` on `seeds` new noise seeds (seed0, seed0+1, ...) and score them: {mean, min, max, sd,+ pass_rate, n, totals}. A sound that only scores well on one seed is a lucky render, not a good recipe."""+ ss = list(range(seed0, seed0 + seeds)) if isinstance(seeds, int) else list(seeds)+ xs = [render(recipe, params, s) for s in ss]+ live = [x for x in xs if x is not None]+ rows = critic.score(live) if live else []+ fin = [float(r['total']) for r in rows if np.isfinite(r['total'])]+ return {'n': len(ss), 'seeds': ss, 'totals': [round(t, 3) for t in fin], 'failed': len(ss) - len(fin),+ 'mean': round(float(np.mean(fin)), 3) if fin else None, 'min': round(float(min(fin)), 3) if fin else None,+ 'max': round(float(max(fin)), 3) if fin else None, 'sd': round(float(np.std(fin)), 3) if fin else None,+ 'pass_rate': round(sum(1 for r in rows if r.get('passes')) / len(ss), 3)}+++def diverse(results, recipe, k=4, min_dist=0.08):+ """pick up to k strong results whose parameter vectors differ (normalised distance >= min_dist), passing first"""+ out = []+ for h in sorted(results, key=lambda h: (not h['score'].get('passes'), -h['score']['total'])):+ if all(np.linalg.norm(h['u'] - o['u']) / math.sqrt(len(h['u'])) >= min_dist for o in out):+ out.append(h)+ if len(out) == k:+ break+ return outdiff --git a/earshot/forge/tone.py b/earshot/forge/tone.pynew file mode 100644index 0000000..b045dda--- /dev/null+++ b/earshot/forge/tone.py@@ -0,0 +1,172 @@+"""Model-free ears for the forge: tonal balance and tail, measured straight from the samples.++ from earshot.forge.tone import ToneRef+ tr = ToneRef(['real_01.wav', 'real_02.wav', ...]) # files or 48 kHz arrays+ m = tr.check(x)+ m['tone_db'], m['tone_ratio'], m['words'] # e.g. 'tone 11.2 dB from the refs (they sit 3.1-6.0 apart):+ # +16 dB at 500-630 Hz (above every ref), -14 dB at 10 kHz (below ...)'++Why this exists: earshot's model signals saturate. After a search tunes a sound against CLAP (match z, typicality) and+the realness head, the sound lands inside or above the real recordings on every model check, including held-out+phrasings, the AudioSet tagger and independent real sets (demo/forge-test, the judge's re-scoring). The models can no+longer tell made from real, but a 1/3-octave measurement still can (a key press 8-13 dB off the real keys' balance+while the real keys sit 3-6 dB from each other). These numbers cannot be pleased by a room and a mic noise floor.++Measures (mono sum, the whole file):+ profile power per 1/3-octave band, 31.5 Hz - 16 kHz (28 bands), in dB relative to the file's total power+ (so level never decides), floored at -60 dB+ tone_db RMS over bands of (profile - the refs' median profile)+ tone_ref the refs' own distances, each to the median of the OTHER refs (leave-one-out): what "real" spreads to+ tone_ratio tone_db / median(tone_ref): 1 = as far from the median as a typical real recording+ outside bands where the sound is beyond the refs' min / max by more than `margin` dB (the words name the worst)+ t40_ms time from the peak until the 5 ms RMS envelope first falls 40 dB under it (tail length), with the refs'+ range; length_s the file length+ shares energy share of 5 broad regions (<150 Hz, 150 Hz-1 kHz, 1-4, 4-12, >12 kHz) in dB of the total, with the+ refs' median share; region_penalty() counts the excess beyond median + 6 dB (a low puff, a bright hiss)+"""+import numpy as np++SR = 48000+CEN = 1000 * 2 ** (np.arange(-15, 13) / 3) # 31.5 Hz .. 15.9 kHz+EDGE = 2 ** (1 / 6)+REGIONS = {'<150': (0, 150), '150-1k': (150, 1000), '1-4k': (1000, 4000), '4-12k': (4000, 12000), '>12k': (12000, 24000)}+++def _mono(x):+ x = np.asarray(x, np.float64)+ return x.mean(axis=1) if x.ndim == 2 else x+++def _load(x):+ if isinstance(x, str):+ from .. import audio+ return audio.load(x)+ return x+++def label(c):+ return f'{c / 1000:.3g} kHz' if c >= 1000 else f'{c:.0f} Hz'+++def profile(x):+ """1/3-octave power profile in dB relative to the total power (28 bands, floored at -60)"""+ m = _mono(x)+ F = np.abs(np.fft.rfft(m)) ** 2+ f = np.fft.rfftfreq(len(m), 1 / SR)+ p = np.array([F[(f >= c / EDGE) & (f < c * EDGE)].sum() for c in CEN]) + 1e-20+ return np.maximum(10 * np.log10(p / p.sum()), -60.0)+++def shares(x):+ """energy share of each broad region (REGIONS) in dB of the total"""+ m = _mono(x)+ F = np.abs(np.fft.rfft(m)) ** 2+ f = np.fft.rfftfreq(len(m), 1 / SR)+ tot = F.sum() + 1e-20+ return {k: round(float(10 * np.log10(F[(f >= a) & (f < b)].sum() / tot + 1e-12)), 1) for k, (a, b) in REGIONS.items()}+++def t40_ms(x, db=40.0, ms=5.0):+ """ms from the peak of the 5 ms RMS envelope until it first falls `db` under it"""+ m = _mono(x)+ n = int(SR * ms / 1000)+ k = len(m) // n+ if k < 2:+ return 0.0+ e = 20 * np.log10(np.sqrt((m[:k * n].reshape(k, n) ** 2).mean(1)) + 1e-12)+ i = int(np.argmax(e))+ after = e[i:] - e[i]+ idx = np.nonzero(after < -db)[0]+ return float((idx[0] if len(idx) else len(after)) * ms)+++class ToneRef:+ """the real references' tonal balance and tail, and a check of any sound against them"""++ def __init__(self, refs, margin=3.0):+ xs = [_load(r) for r in refs]+ self.n = len(xs)+ self.margin = margin+ self.P = np.stack([profile(x) for x in xs]) if xs else np.zeros((0, len(CEN)))+ self.med = np.median(self.P, 0) if xs else None+ self.lo, self.hi = (self.P.min(0), self.P.max(0)) if xs else (None, None)+ self.t40 = np.array([t40_ms(x) for x in xs])+ self.S = [shares(x) for x in xs]+ self.len_s = np.array([len(_mono(x)) / SR for x in xs])+ self.ref_dist = np.array([self._loo(i) for i in range(self.n)]) if self.n >= 3 else np.zeros(0)++ def _loo(self, i):+ keep = [j for j in range(self.n) if j != i]+ return float(np.sqrt(np.mean((self.P[i] - np.median(self.P[keep], 0)) ** 2)))++ @property+ def spread(self):+ """(median, max) of the refs' leave-one-out distances, or None with fewer than 3 refs"""+ if not len(self.ref_dist):+ return None+ return float(np.median(self.ref_dist)), float(self.ref_dist.max())++ def check(self, x, exclude=None):+ """measure one sound against the refs. exclude = index of a ref to leave out (scoring a ref fairly)"""+ x = _load(x)+ if self.n == 0:+ return {}+ keep = [j for j in range(self.n) if j != exclude]+ P = self.P[keep]+ med, lo, hi = np.median(P, 0), P.min(0), P.max(0)+ p = profile(x)+ dev = p - med+ dist = float(np.sqrt(np.mean(dev ** 2)))+ sp = self.ref_dist[keep] if len(self.ref_dist) else np.zeros(0)+ ratio = dist / max(float(np.median(sp)), 0.5) if len(sp) >= 2 else None+ over = np.maximum(p - hi - self.margin, 0) - np.maximum(lo - p - self.margin, 0)+ outside = [(label(c), round(float(d), 1), 'above' if o > 0 else 'below')+ for c, d, o in zip(CEN, dev, over) if o != 0]+ t40 = t40_ms(x)+ t40r = self.t40[keep]+ out = {'tone_db': round(dist, 2), 'tone_ratio': None if ratio is None else round(ratio, 2),+ 'tone_ref': [round(float(v), 2) for v in sp], 'band_dev': [round(float(v), 1) for v in dev],+ 'outside': outside, 't40_ms': t40, 't40_ref': (float(t40r.min()), float(t40r.max())),+ 'length_s': round(len(_mono(x)) / SR, 3), 'shares': shares(x),+ 'shares_ref_med': {k: round(float(np.median([self.S[j][k] for j in keep])), 1) for k in REGIONS},+ 'length_ref': (round(float(self.len_s[keep].min()), 3), round(float(self.len_s[keep].max()), 3))}+ out['words'] = self.words(out)+ return out++ @staticmethod+ def words(m):+ sp = m['tone_ref']+ s = f"tone {m['tone_db']:.1f} dB from the refs' median"+ if sp:+ s += f" (the refs sit {min(sp):.1f}-{max(sp):.1f} apart)"+ worst = sorted(m['outside'], key=lambda o: -abs(o[1]))[:3]+ if worst:+ s += ': ' + ', '.join(f'{d:+.0f} dB at {b} ({w} every ref)' for b, d, w in worst)+ a, b = m['t40_ref']+ if m['t40_ms'] > 1.5 * b + 10 or m['t40_ms'] < 0.6 * a - 5:+ s += f"; tail t40 {m['t40_ms']:.0f} ms (refs {a:.0f}-{b:.0f})"+ loud = [f"{k} {m['shares'][k] - m['shares_ref_med'][k]:+.0f} dB" for k in REGIONS+ if abs(m['shares'][k] - m['shares_ref_med'][k]) > 9]+ if loud:+ s += '; energy share vs the refs: ' + ', '.join(loud)+ lo, hi = m['length_ref']+ if m['length_s'] > 1.3 * hi:+ s += f"; {m['length_s']:.2f} s long (refs {lo:.2f}-{hi:.2f})"+ return s++ @staticmethod+ def region_penalty(m, regions=('<150', '4-12k', '>12k'), margin=6.0, scale=6.0):+ """model-free check of broad regions: 1 per `scale` dB that a region's energy share exceeds the refs' median+ share by more than `margin` dB (a low puff or a bright hiss / tick on a mid-band sound)"""+ return sum(max(0.0, m['shares'][k] - m['shares_ref_med'][k] - margin) / scale for k in regions)++ def penalty(self, m):+ """0 while the sound is no farther from the refs' median than a typical real recording, then linear in the+ excess (tone_ratio - 1), plus a share of the tail excess beyond the refs' longest t40"""+ if not m or m.get('tone_ratio') is None:+ return 0.0+ pen = max(0.0, m['tone_ratio'] - 1.0)+ a, b = m['t40_ref']+ if b > 0 and m['t40_ms'] > b:+ pen += 0.5 * min(2.0, (m['t40_ms'] - b) / b)+ return pendiff --git a/earshot/llm.py b/earshot/llm.pynew file mode 100644index 0000000..17c5761--- /dev/null+++ b/earshot/llm.py@@ -0,0 +1,518 @@+"""Optional audio-LLM critique: a second opinion in words from a hosted audio model (Google Gemini).++ earshot llm film.mp4 --yes-upload # film-sound rubric: sync, realism, mix, loudness, fixes+ earshot llm mix.wav --yes-upload --with-timeline # also send earshot's own listening transcript as context+ earshot llm mix.wav --yes-upload --prompt "Does the whoosh at 0:12 feel late?"++ from earshot import llm+ r = llm.critique('mix.wav', consent=True) # r['summary'], r['text'], r['critique']['scores' | 'issues' | ...]+ r['critique']['issues'] # [{'start', 'end', 'severity', 'category', 'problem', 'fix'}] in file seconds++Off unless BOTH hold: GEMINI_API_KEY (or GOOGLE_API_KEY) is set in the environment, and the caller consents to the+upload (--yes-upload / consent=True), because the audio leaves this machine and goes to Google. Without them it+returns a message explaining how to enable it; the rest of earshot never needs it. Only audio is sent (never the+picture of a video): mono 64 kbps MP3, trimmed to --max-seconds (default 600). Requests up to ~18 MB of base64 go+inline; larger ones use the Files API and the uploaded copy is deleted right after the answer (or the failure). The+key is read from the environment, sent only in the x-goog-api-key header and never printed, logged or stored.++Model: EARSHOT_GEMINI_MODEL (default gemini-3.8-flash). Gemini API docs used (checked 2026-10-14):+ https://ai.google.dev/gemini-api/docs/audio audio input, 32 tokens/s, downsampled to 16 kbps mono+ https://ai.google.dev/gemini-api/docs/file-input-methods inline limit (100 MB; the audio page still says 20 MB)+ https://ai.google.dev/gemini-api/docs/files Files API resumable upload, 48 h retention, DELETE+ https://ai.google.dev/api/generate-content v1beta models/{model}:generateContent, JSON mode++Trust: an opinion from a general model, not a measurement, and not deterministic. Gemini hears a reduced copy (its docs:+downsampled to 16 kbps, one channel), so fine top-end detail and stereo are beyond it. Timestamps are approximate (a+few seconds off is common); loudness numbers in the answer come from earshot's own measurements, passed in as context.+Verify every suggested fix with earshot timeline / ab before and after.+"""+import base64+import json+import os+import re+import subprocess+import time++from . import audio+from .report import emit, fmt_t++DEFAULT_MODEL = 'gemini-3.8-flash'+BASE = 'https://generativelanguage.googleapis.com'+INLINE_MAX_B64 = 18 * 1024 * 1024 # stay under the strictest documented inline request limit (20 MB)+KBPS = 64+RETRY_STATUS = (429, 500, 502, 503, 504)+RETRY_WAIT = (4, 12)+KEY_URL = 'https://aistudio.google.com/apikey'++RUBRIC = """You are a senior re-recording mixer and sound supervisor reviewing the soundtrack of a short product launch+film (or one stem of it). The bar is a premium launch film from a top studio: real recorded foley and ambience, music+that is produced and mixed, sounds that land exactly on picture events, a clean, controlled master that is loud but+not crushed. Listen to the whole clip. Be specific and blunt; praise only what is genuinely good.++Judge:+1. sync: do hits, whooshes and accents land where they seem to belong? You cannot see the picture: use the picture+ cut / cue times in the context if given, otherwise judge rhythmic tightness and whether sounds feel late, early or+ smeared. Use null for the sync score when there is nothing to judge it against.+2. realism: which sounds are real recordings and which sound code-synthesized, stock-cheesy or generic (pure sine+ beeps, plain filtered-noise whooshes, chiptune blips, preset risers, the identical sample machine-gunned)?+3. mix balance: music vs effects vs ambience, masking, elements too loud or buried, low-end mud, harsh 2-5 kHz+ (the clip is a mono reduction: do not judge stereo width).+4. loudness and dynamics: use the measured numbers in the context, never guess numbers; squashed, too quiet, peaky,+ pumping, clipped?+5. what is cheap: the moments that would make a professional wince.+6. concrete fixes, each tied to a time range and saying exactly what to change (for example: replace the beep with a+ recorded UI tap, -4 dB on the music under 12.0-15.0, move the hit 80 ms earlier, high-pass the pad at 150 Hz).++Scores 1-10: 10 = indistinguishable from the premium reference, 7 = good professional work with small flaws,+5 = competent amateur, 3 = obviously cheap or synthetic, 1 = broken.+Times are seconds from the start of THIS clip, as numbers (12.4, not "0:12"). Reply with ONLY this JSON object:+{"overall": {"score": <1-10>, "verdict": "<one sentence>"},+ "scores": {"sync": <1-10 or null>, "realism": <1-10>, "mix_balance": <1-10>, "loudness": <1-10>,+ "music": <1-10 or null when there is no music>},+ "notes": {"sync": "", "realism": "", "mix_balance": "", "loudness": "", "music": ""},+ "synthetic_or_cheap": [{"start": <s>, "end": <s>, "what": "<the sound>", "why": "<what gives it away>"}],+ "issues": [{"start": <s>, "end": <s>, "severity": "high|medium|low",+ "category": "sync|realism|mix|loudness|music|noise|edit", "problem": "", "fix": ""}],+ "strengths": [""],+ "top_fixes": ["<most important first, at most 5>"]%s}"""++MEASURED = [('integrated loudness', 'lufs', ' LUFS'), ('true peak', 'true_peak_dbtp', ' dBTP'),+ ('loudness range', 'lra_lu', ' LU'), ('short-term max', 'short_term_max_lufs', ' LUFS'),+ ('quietest 20 ms level (10th percentile)', 'noise_floor_dbfs', ' dBFS'),+ ('clipped samples', 'clipped_samples', '')]+SEV = {'high': 0, 'medium': 1, 'low': 2}+SCORE_KEYS = ('sync', 'realism', 'mix_balance', 'loudness', 'music')+++class LLMError(RuntimeError):+ pass+++class Provider:+ """one hosted audio model: subclasses implement generate(audio_bytes, mime, prompt, model, key) -> dict"""+ name = ''+ env_keys = ()+ default_model = ''++ def key(self):+ """the first key set in the environment; an unexpanded "${VAR}" placeholder from an MCP config counts as none"""+ return next((v for v in (os.environ.get(k, '').strip() for k in self.env_keys)+ if v and not v.startswith('${')), None)++ def model(self, model=None):+ return (model or self.default_model).removeprefix('models/')++ def generate(self, data, mime, prompt, model, key):+ raise NotImplementedError+++class Gemini(Provider):+ name = 'gemini'+ env_keys = ('GEMINI_API_KEY', 'GOOGLE_API_KEY')++ @property+ def default_model(self):+ return os.environ.get('EARSHOT_GEMINI_MODEL') or DEFAULT_MODEL++ def generate(self, data, mime, prompt, model, key):+ rq = _requests()+ b64 = base64.b64encode(data).decode()+ stored = {} # the Files API name, as soon as Google holds a copy: deleted whatever happens next+ try:+ if len(b64) <= INLINE_MAX_B64:+ part, method = {'inline_data': {'mime_type': mime, 'data': b64}}, 'inline'+ else:+ f = self._upload(rq, data, mime, key, stored)+ part, method = {'file_data': {'mime_type': mime, 'file_uri': f['uri']}}, 'files_api'+ body = {'contents': [{'role': 'user', 'parts': [part, {'text': prompt}]}],+ 'generationConfig': {'responseMimeType': 'application/json', 'maxOutputTokens': 16384}}+ j = _json(_post(rq, f'{BASE}/v1beta/models/{model}:generateContent', key, json=body, timeout=(20, 600)))+ except Exception as e:+ if not stored:+ raise+ gone = self._delete(rq, stored, key)+ raise LLMError(f'{_msg(e)} (the uploaded copy was ' ++ ('deleted)' if gone else 'NOT deleted: Google removes it after 48 h)')) from None+ deleted = self._delete(rq, stored, key) if stored else None+ cands = j.get('candidates') or []+ if not cands:+ fb = j.get('promptFeedback') or {}+ raise LLMError(f'Gemini returned no answer (blockReason: {fb.get("blockReason", "unknown")}).')+ c = cands[0]+ parts = (c.get('content') or {}).get('parts') or []+ text = ''.join(p.get('text', '') for p in parts if not p.get('thought'))+ if not text.strip():+ raise LLMError(f'Gemini returned an empty answer (finishReason: {c.get("finishReason", "unknown")}).')+ u = j.get('usageMetadata') or {}+ return {'text': text, 'finish_reason': c.get('finishReason'), 'method': method, 'files_api_deleted': deleted,+ 'model_version': j.get('modelVersion') or model,+ 'usage': {k: u[k] for k in ('promptTokenCount', 'candidatesTokenCount', 'thoughtsTokenCount',+ 'totalTokenCount') if k in u}}++ def _upload(self, rq, data, mime, key, stored):+ h = {'X-Goog-Upload-Protocol': 'resumable', 'X-Goog-Upload-Command': 'start',+ 'X-Goog-Upload-Header-Content-Length': str(len(data)), 'X-Goog-Upload-Header-Content-Type': mime}+ r = _post(rq, f'{BASE}/upload/v1beta/files', key, headers=h, json={'file': {'display_name': 'earshot-review'}},+ timeout=(20, 120))+ url = r.headers.get('x-goog-upload-url')+ if not url:+ raise LLMError('Gemini Files API did not return an upload URL.')+ r = _post(rq, url, None, headers={'Content-Length': str(len(data)), 'X-Goog-Upload-Offset': '0',+ 'X-Goog-Upload-Command': 'upload, finalize'}, data=data, timeout=(20, 900))+ f = (_json(r) or {}).get('file') or {}+ if f.get('name'):+ stored['name'] = f['name']+ for _ in range(60):+ if f.get('state') != 'PROCESSING':+ break+ time.sleep(2)+ g = rq.get(f'{BASE}/v1beta/{f["name"]}', headers={'x-goog-api-key': key}, timeout=(20, 60))+ f = _json(_check(g))+ if f.get('state') not in (None, 'ACTIVE') or not f.get('uri'):+ raise LLMError(f'Gemini Files API upload did not become ACTIVE (state {f.get("state")}).')+ return f++ def _delete(self, rq, f, key):+ try:+ return rq.delete(f'{BASE}/v1beta/{f["name"]}', headers={'x-goog-api-key': key}, timeout=(20, 60)).ok+ except Exception: # noqa: BLE001+ return False+++PROVIDERS = {'gemini': Gemini}+++def _requests():+ try:+ import requests+ except ImportError as e:+ raise LLMError('the requests package is missing: pip install "earshot[llm]" (or pip install requests)') from e+ return requests+++def _msg(e):+ return str(e) if isinstance(e, LLMError) else f'{type(e).__name__}: {e}'+++def _json(r):+ try:+ return r.json()+ except ValueError:+ raise LLMError(f'Gemini answered HTTP {r.status_code} with something that is not JSON (a proxy or firewall '+ f'page?): {(r.text or "")[:120]!r}') from None+++def _check(r):+ if r.status_code < 400:+ return r+ try:+ msg = (r.json().get('error') or {}).get('message') or ''+ except Exception: # noqa: BLE001+ msg = (r.text or '')[:300]+ hint = {400: 'bad request (often: the model does not accept audio, or the request is malformed)',+ 401: 'the API key was rejected', 403: 'the key has no access to this API / model (or the API is disabled)',+ 404: 'model not found: set EARSHOT_GEMINI_MODEL or --model to a current model id',+ 413: 'request too large: use --max-seconds or --start/--end', 429: 'rate limit or quota exhausted'}+ raise LLMError(f'Gemini HTTP {r.status_code}: {hint.get(r.status_code, "request failed")}. {msg}'.strip())+++def _post(rq, url, key, headers=None, timeout=(20, 300), **kw):+ h = dict(headers or {})+ if key:+ h['x-goog-api-key'] = key+ if 'json' in kw:+ h.setdefault('Content-Type', 'application/json')+ for i in range(len(RETRY_WAIT) + 1):+ try:+ r = rq.post(url, headers=h, timeout=timeout, **kw)+ except rq.Timeout:+ raise LLMError(f'Gemini did not answer within {timeout[1]} s (try a shorter clip)') from None+ except rq.RequestException as e:+ if i == len(RETRY_WAIT):+ raise LLMError(f'network error talking to Gemini: {type(e).__name__}') from None+ time.sleep(RETRY_WAIT[i])+ continue+ if r.status_code in RETRY_STATUS and i < len(RETRY_WAIT):+ time.sleep(RETRY_WAIT[i])+ continue+ return _check(r)+++def encode(path, start=None, end=None, max_seconds=600.0, kbps=KBPS):+ """the audio of `path` (or [start, end)) as small mono MP3 bytes -> (bytes, mime, info)"""+ dur = float(audio.probe(path).get('duration') or 0) or audio.duration(path)+ a = max(0.0, float(start or 0.0))+ b = min(dur, float(end)) if end is not None else dur+ if b <= a:+ raise LLMError(f'empty range {a:.3f}-{b:.3f} s (file is {dur:.3f} s)')+ trimmed = b - a > max_seconds+ if trimmed:+ b = a + max_seconds+ cmd = ['ffmpeg', '-v', 'error', '-nostdin', '-ss', f'{a:.3f}', '-i', path, '-t', f'{b - a:.3f}', '-vn', '-ac', '1',+ '-ar', '44100', '-c:a', 'libmp3lame', '-b:a', f'{kbps}k', '-f', 'mp3', '-']+ p = subprocess.run(cmd, capture_output=True)+ if p.returncode != 0 or not p.stdout:+ raise LLMError(f'ffmpeg could not encode {path}: {p.stderr.decode(errors="replace")[:300]}')+ return p.stdout, 'audio/mp3', {'start': round(a, 3), 'end': round(b, 3), 'seconds': round(b - a, 3),+ 'file_duration': round(dur, 3), 'trimmed': trimmed, 'bytes': len(p.stdout),+ 'codec': f'mp3 mono {kbps} kbps'}+++def context(path, clip, video=None, with_timeline=False, cuts=True):+ """what earshot itself measured, as plain text for the prompt (numbers the model must not guess)"""+ from . import features+ lines = [f'Clip under review: {clip["seconds"]:.1f} s (file seconds {clip["start"]:.1f}-{clip["end"]:.1f} of a '+ f'{clip["file_duration"]:.1f} s file), sent as {clip["codec"]} for this review.']+ m = features.measure(path, start=clip['start'], end=clip['end'], light=True)+ got = [f'{label} {m[k]}{unit}' for label, k, unit in MEASURED if k in m]+ if got:+ lines.append('Measured by earshot (exact; use these, never estimate loudness): ' + ', '.join(got) + '.')+ vid = video or (path if os.path.splitext(path)[1].lower() in audio.VIDEO_EXTS else None)+ notes = []+ if with_timeline:+ try:+ from .timeline import timeline+ tl = timeline(path, video=vid, start=clip['start'], end=clip['end'], detail='low')+ txt = (tl.get('text') or tl.get('summary') or '')[:16000]+ lines.append('earshot listening transcript (machine-made; file times' ++ (f', clip starts at file second {clip["start"]:.1f}' if clip['start'] else '') + '):\n' + txt)+ vid = None+ except Exception as e: # noqa: BLE001+ notes.append(f'the earshot timeline context was NOT sent: timeline failed ({type(e).__name__}: {e})')+ if vid and cuts:+ try:+ from . import video as vmod+ cs = [round(c - clip['start'], 2) for c in vmod.cuts(vid) if clip['start'] <= c < clip['end']]+ if cs:+ lines.append('Picture cuts / big visual changes, clip seconds (frame-difference detector: hard cuts '+ 'and whips reliable, slow crossfades missed or late): ' ++ ', '.join(f'{c:g}' for c in cs[:200]) + '.')+ except Exception as e: # noqa: BLE001+ notes.append(f'picture cut times were NOT sent: cut detection failed ({type(e).__name__})')+ return '\n'.join(lines), notes+++def build_prompt(ctx_text, question=None):+ rubric = RUBRIC % (',\n "answer": "<direct answer to the editor\'s question>"' if question else '')+ q = (f'\n\nThe editor\'s specific question (answer it in "answer"; still fill the rest):\n{question}'+ if question else '')+ return f'{rubric}\n\nContext:\n{ctx_text}{q}'+++def parse_json(text):+ """the JSON object inside a model answer (tolerates code fences, prose around it, trailing commas) or None"""+ s = re.sub(r'^\s*```(?:json)?\s*|\s*```\s*$', '', text or '')+ for cand in (s, _outer_object(s)):+ if not cand:+ continue+ for c in (cand, re.sub(r',\s*([}\]])', r'\1', cand)):+ try:+ j = json.loads(c)+ if isinstance(j, dict):+ return j+ except ValueError:+ pass+ return None+++def _outer_object(s):+ i = s.find('{')+ if i < 0:+ return None+ depth, instr, esc = 0, False, False+ for k in range(i, len(s)):+ ch = s[k]+ if instr:+ esc = ch == '\\' and not esc+ if ch == '"' and not esc:+ instr = False+ continue+ if ch == '"':+ instr = True+ elif ch == '{':+ depth += 1+ elif ch == '}':+ depth -= 1+ if depth == 0:+ return s[i:k + 1]+ return s[i:]+++def _sec(v):+ if isinstance(v, (int, float)) and not isinstance(v, bool):+ return float(v)+ if isinstance(v, str):+ m = re.fullmatch(r'\s*(?:(\d+):)?(\d+(?:\.\d+)?)\s*s?\s*', v)+ if m:+ return int(m.group(1) or 0) * 60 + float(m.group(2))+ return None+++def _score(v):+ if isinstance(v, str):+ m = re.match(r'\s*(\d+(?:\.\d+)?)', v)+ v = float(m.group(1)) if m else None+ if isinstance(v, (int, float)) and not isinstance(v, bool):+ return round(min(10.0, max(0.0, float(v))), 1)+ return None+++def _times(items, offset, clip_len, keys):+ out = []+ for it in items if isinstance(items, list) else []:+ if not isinstance(it, dict):+ continue+ a, b = _sec(it.get('start')), _sec(it.get('end'))+ if a is not None:+ a = min(max(0.0, a), clip_len)+ b = a if b is None else min(max(a, b), clip_len)+ d = {'start': None if a is None else round(a + offset, 2), 'end': None if b is None else round(b + offset, 2)}+ d.update({k: str(it.get(k, '')).strip() for k in keys})+ out.append(d)+ return out+++def normalize(j, offset=0.0, clip_len=1e9):+ """make a parsed answer predictable: numeric scores, file-time seconds, issues sorted by severity then time"""+ ov = j.get('overall') if isinstance(j.get('overall'), dict) else {'score': j.get('overall')}+ sc = j.get('scores') if isinstance(j.get('scores'), dict) else {}+ notes = j.get('notes') if isinstance(j.get('notes'), dict) else {}+ issues = _times(j.get('issues'), offset, clip_len, ('severity', 'category', 'problem', 'fix'))+ for it in issues:+ it['severity'] = it['severity'].lower() if it['severity'].lower() in SEV else 'medium'+ issues.sort(key=lambda d: (SEV[d['severity']], d['start'] if d['start'] is not None else 1e9))+ r = {'overall': {'score': _score(ov.get('score')), 'verdict': str(ov.get('verdict') or '').strip()},+ 'scores': {k: _score(sc.get(k)) for k in SCORE_KEYS},+ 'notes': {k: str(notes.get(k) or '').strip() for k in SCORE_KEYS if notes.get(k)},+ 'synthetic_or_cheap': _times(j.get('synthetic_or_cheap'), offset, clip_len, ('what', 'why')),+ 'issues': issues,+ 'strengths': [str(s) for s in j.get('strengths') or [] if str(s).strip()][:8],+ 'top_fixes': [str(s) for s in j.get('top_fixes') or [] if str(s).strip()][:5]}+ if j.get('answer'):+ r['answer'] = str(j['answer']).strip()+ return r+++def _span(it):+ if it['start'] is None:+ return '(no time)'+ return fmt_t(it['start']) if it['end'] in (None, it['start']) else f'{fmt_t(it["start"])}-{fmt_t(it["end"])}'+++def render(r):+ """agent-readable text of a critique result"""+ lines = [r['summary']] + [f'(note: {n})' for n in r.get('notes') or []]+ c = r.get('critique')+ if not c:+ return '\n'.join(lines + ([r['raw'][:3000]] if r.get('raw') else []))+ if c.get('answer'):+ lines.append(f'answer: {c["answer"]}')+ lines.append('scores (1-10, LLM opinion): ' + ', '.join(f'{k} {v:g}' for k, v in c['scores'].items()+ if v is not None))+ for k, v in c['notes'].items():+ lines.append(f' {k}: {v}')+ if c['issues']:+ lines.append('issues (most severe first; times = file seconds, approximate):')+ lines += [f' {_span(i)} {i["severity"]} {i["category"]}: {i["problem"]} -> fix: {i["fix"]}'+ for i in c['issues']]+ if c['synthetic_or_cheap']:+ lines.append('sounds that read as synthetic / cheap:')+ lines += [f' {_span(i)} {i["what"]}: {i["why"]}' for i in c['synthetic_or_cheap']]+ if c['strengths']:+ lines.append('strengths: ' + '; '.join(c['strengths']))+ if c['top_fixes']:+ lines.append('top fixes: ' + ' | '.join(f'{n}. {f}' for n, f in enumerate(c['top_fixes'], 1)))+ return '\n'.join(lines)+++def _scrub(s, key):+ return s.replace(key, '***') if key and isinstance(s, str) else s+++def _off(path, provider, why, steps):+ r = {'enabled': False, 'ok': False, 'path': path, 'provider': provider.name, 'model': provider.model(),+ 'summary': why, 'how_to_enable': steps}+ r['text'] = why + '\n' + '\n'.join(f' {n}. {s}' for n, s in enumerate(steps, 1))+ return r+++def critique(path, prompt=None, provider='gemini', model=None, consent=False, video=None, with_timeline=False,+ start=None, end=None, max_seconds=600.0, cuts=True):+ """audio-LLM critique of a soundtrack against the film-sound rubric (only with an API key AND consent)"""+ if provider not in PROVIDERS:+ raise ValueError(f'unknown provider {provider!r}; available: {", ".join(PROVIDERS)}')+ if not os.path.isfile(path):+ raise audio.AudioError(f'no such file: {path}')+ pv = PROVIDERS[provider]()+ key = pv.key()+ if not key:+ return _off(path, pv, 'LLM critique is off: no GEMINI_API_KEY or GOOGLE_API_KEY in the environment. Everything '+ 'else in earshot works without it.',+ [f'Create a Gemini API key at {KEY_URL} (the user\'s own account; usage may be billed).',+ 'Set it in the shell that runs earshot: PowerShell $env:GEMINI_API_KEY="..." / bash export '+ 'GEMINI_API_KEY=... (never paste a key into code, files or chat).',+ 'Rerun with --yes-upload (consent=True), only if the user agrees that the audio is uploaded '+ 'to Google.'])+ if not consent:+ return _off(path, pv, f'Not uploaded: this would send the audio of {os.path.basename(path)} to Google Gemini '+ f'({pv.model(model)}). It needs explicit consent.',+ ['Ask the user whether this audio may be uploaded to Google for an AI critique.',+ 'If yes, rerun with --yes-upload (CLI) or consent=True (Python / MCP).'])+ m = pv.model(model)+ t0 = time.time()+ clip = None+ try:+ data, mime, clip = encode(path, start, end, max_seconds)+ ctx_text, notes = context(path, clip, video, with_timeline, cuts)+ out = pv.generate(data, mime, build_prompt(ctx_text, prompt), m, key)+ except Exception as e: # noqa: BLE001 - any failure becomes a readable result, never a traceback with the key+ msg = _scrub(_msg(e), key)+ return {'enabled': True, 'ok': False, 'path': path, 'provider': pv.name, 'model': m, 'uploaded': clip,+ 'summary': f'LLM critique failed: {msg}', 'text': f'LLM critique failed: {msg}', 'error': msg}+ j = parse_json(out['text'])+ r = {'enabled': True, 'ok': j is not None, 'path': path, 'provider': pv.name, 'model': out['model_version'],+ 'uploaded': dict(clip, method=out['method']), 'seconds': round(time.time() - t0, 1), 'usage': out['usage'],+ 'finish_reason': out['finish_reason'], 'context_sent': ctx_text, 'notes': notes}+ if out.get('files_api_deleted') is not None:+ r['uploaded']['files_api_deleted'] = out['files_api_deleted']+ trim = (f' (trimmed to the first {clip["seconds"]:g} s; use --start/--end for other parts)'+ if clip['trimmed'] else '')+ if j is None:+ r['raw'] = _scrub(out['text'], key)[:20000]+ r['summary'] = (f'{pv.name} answered but not in parseable JSON (finish {out["finish_reason"]}); raw answer '+ f'attached{trim}.')+ else:+ c = r['critique'] = normalize(j, clip['start'], clip['seconds'])+ high = sum(1 for i in c['issues'] if i['severity'] == 'high')+ ov = c['overall']+ lead = f'{ov["score"]:g}/10' if ov['score'] is not None else 'unscored'+ verdict = (ov['verdict'] or 'no verdict').rstrip()+ verdict += '' if verdict[-1] in '.!?' else '.'+ r['summary'] = (f'{pv.name} ({r["model"]}) rates the soundtrack {lead}: {verdict} '+ f'{len(c["issues"])} issues ({high} high).' ++ (f' Top fix: {c["top_fixes"][0]}' if c['top_fixes'] else '') ++ f' LLM opinion, timestamps approximate; verify with earshot timeline/ab{trim}.')+ r['text'] = render(r)+ return r+++def add_cli(sub):+ from .cli import common+ p = common(sub.add_parser('llm', help='optional audio-LLM critique (Gemini; needs your key + --yes-upload)',+ description='Second opinion from a hosted audio model. Uploads the audio to Google: '+ 'needs GEMINI_API_KEY/GOOGLE_API_KEY and --yes-upload.'))+ p.add_argument('file', help='soundtrack, stem or video (only its audio is sent)')+ p.add_argument('--prompt', help='a specific question to answer in addition to the rubric')+ p.add_argument('--yes-upload', action='store_true', help='consent: upload this audio to Google Gemini')+ p.add_argument('--model', help=f'Gemini model id (default EARSHOT_GEMINI_MODEL or {DEFAULT_MODEL})')+ p.add_argument('--video', help='the film, for picture-cut context (only cut times are sent, never frames)')+ p.add_argument('--with-timeline', action='store_true',+ help="also send earshot's listening transcript (loads models)")+ p.add_argument('--no-cuts', action='store_true', help='do not compute picture cuts for a video')+ p.add_argument('--start', type=float, help='send only from here (s)')+ p.add_argument('--end', type=float, help='send only up to here (s)')+ p.add_argument('--max-seconds', type=float, default=600.0, help='trim longer audio to this (default 600)')+ p.set_defaults(func=lambda a: 0 if emit(critique(a.file, a.prompt, model=a.model, consent=a.yes_upload,+ video=a.video, with_timeline=a.with_timeline, start=a.start,+ end=a.end, max_seconds=a.max_seconds, cuts=not a.no_cuts),+ a)['ok'] else 1)diff --git a/earshot/match.py b/earshot/match.pynew file mode 100644index 0000000..d91a659--- /dev/null+++ b/earshot/match.py@@ -0,0 +1,149 @@+"""How well does a sound match a text description? Calibrated scores and a verdict whose evidence was measured.++ earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch" "a typewriter"+ earshot match whoosh.wav "a fast whoosh" "a soft air whoosh" "a deep cinematic whoosh" # several queries+ from earshot.match import match+ match('key.ogg', 'a deep thock', alts=['a clicky switch'])++Fields per query: cosine (raw CLAP similarity, NOT a probability; only comparable for the same sound and query), z+(calibrated: how far the description stands out among the vocabulary descriptions for this sound after removing the+description's own bias, see earshot/vocab.py), percentile + rank_in_vocab (where the query would rank among the+vocabulary descriptions of this sound), p_vs_alts (softmax of logit_scale * cosine over the query and its alternatives:+a forced choice, only as meaningful as the alternatives; with several queries and no alts the queries compete with+each other), verdict and the closest vocabulary descriptions (what else it could be).++Verdicts are bands of z (not cumulative). Thresholds chosen on 219 labelled real recordings by a fixed likelihood-ratio+rule; the evidence below was measured on a separate held-out library of 125 real recordings, labelled from file names+before scoring, that nothing was tuned on (eval/semantics/RESULTS.md). LR = how much more often true matches land in+the band than wrong ones; precision_library = share of pairs in the band that are true when ~3% of candidate pairs are+true (searching a library), precision_even_odds = when you already half-expect it to fit. 'lenient' leaves out pairs+of related classes (an Enter key for 'a spacebar press', wind for grass steps); 'strict' counts them as wrong, so it is+the number to use when the exact variant matters. 'weak' is evidence (LR ~7-12), not proof; below 2 there is none.+The strong band's LR (1289) rests on only 3 wrong pairs (95% range roughly 440-6200; 170 on the tuning library), so+the text output quotes precision, not the ratio.+Music genre / mood, instrument and voice descriptions were never validated: treat their verdicts as hints.+"""+import os++import numpy as np++from . import audio, vocab+from .cli import common++VERDICTS = [(3.0, 'strong'), (2.5, 'plausible'), (2.0, 'weak')]+# held-out library, shipped configuration (eval/semantics/heldout.json): 248 true / 7614 wrong pairs (lenient),+# 8222 wrong (strict); base rate 3.2%+VERDICT_EVIDENCE = {+ 'strong': {'z': '>= 3', 'lr': 1289, 'precision_library': 0.98, 'precision_even_odds': 1.0,+ 'lr_strict': 51, 'precision_library_strict': 0.61, 'recall_at_or_above': 0.51},+ 'plausible': {'z': '2.5 - 3', 'lr': 21, 'precision_library': 0.41, 'precision_even_odds': 0.95,+ 'lr_strict': 9.1, 'precision_library_strict': 0.22, 'recall_at_or_above': 0.63},+ 'weak': {'z': '2 - 2.5', 'lr': 12, 'precision_library': 0.28, 'precision_even_odds': 0.92,+ 'lr_strict': 6.6, 'precision_library_strict': 0.17, 'recall_at_or_above': 0.76},+ 'no match': {'z': '< 2', 'lr': 0.25, 'precision_library': 0.008, 'precision_even_odds': 0.2,+ 'lr_strict': 0.25, 'precision_library_strict': 0.008},+}+SKIP_CATS = ('quality', 'production')+SILENT_DBFS = -70.0+++def verdict(z):+ for t, name in VERDICTS:+ if z >= t:+ return name+ return 'no match'+++def softmax(x, scale):+ e = np.exp(scale * (np.asarray(x, np.float64) - np.max(x)))+ return e / e.sum()+++def _p(p):+ return '>99%' if p > 0.995 else '<1%' if p < 0.005 else f'{100 * p:.0f}%'+++def score_queries(a, queries, alts=None, v=None):+ """per-query score dicts for one audio embedding a (512,)"""+ v = v or vocab.get()+ rows = v.scores(a[None], queries)[0]+ scale = vocab.logit_scale()+ alt_rows = v.scores(a[None], list(alts))[0] if alts else []+ multi = softmax([r['cosine'] for r in rows], scale) if len(rows) > 1 and not alts else None+ kinds = v.nearest_category(v.embed_text([r['query'] for r in rows]))+ for i, r in enumerate(rows):+ r['verdict'] = verdict(r['z'])+ r['validated'] = kinds[i] not in vocab.UNVALIDATED+ if alts:+ p = softmax([r['cosine']] + [x['cosine'] for x in alt_rows], scale)+ r['p_vs_alts'] = round(float(p[0]), 3)+ r['alts'] = [dict(query=x['query'], p=round(float(pi), 3), z=x['z'], cosine=x['cosine'])+ for x, pi in zip(alt_rows, p[1:])]+ else:+ r['p_vs_alts'] = None if multi is None else round(float(multi[i]), 3)+ return rows+++def peak_dbfs(path, start=None, end=None):+ """sample peak of the file as it plays (stereo decode; mono files are duplicated at unity gain)"""+ x = audio.load(path, start=start, end=end)+ return audio.db(np.abs(x).max() if len(x) else 0.0)+++def match(path, query, alts=None, start=None, end=None):+ """calibrated match of one or several text descriptions against a sound (or a [start, end) segment of it)"""+ queries = [query] if isinstance(query, str) else list(query or [])+ if not queries or not all(isinstance(q, str) and q.strip() for q in queries):+ raise ValueError('no query given: pass one or more non-empty descriptions')+ vocab.check(path, start, end)+ v = vocab.get()+ a = vocab.embed_file(path, start, end)+ rows = sorted(score_queries(a, queries, alts, v), key=lambda r: -r['z'])+ best = rows[0]+ near = [dict(term=t, category=c, z=z, cosine=cos) for t, c, z, cos in v.top_terms(a, k=4, exclude=SKIP_CATS)]+ seg = f' [{start or 0:.3f}-{end:.3f} s]' if end is not None else (f' [from {start:.3f} s]' if start else '')+ unval = ' (unvalidated: music / instrument / voice description)'+ parts = [f"'{r['query']}' -> {r['verdict']}{'' if r['validated'] else unval} (z {vocab.zf(r['z'])}; would rank "+ f"#{r['rank_in_vocab']} among {r['vocab_size']} vocabulary descriptions; cosine {r['cosine']:.2f})"+ for r in rows]+ s = f'{os.path.basename(path)}{seg}: ' + '; '.join(parts) + '.'+ if best.get('alts'):+ s += f" Forced choice vs your alternatives: '{best['query']}' {_p(best['p_vs_alts'])}, " + ', '.join(+ f"'{x['query']}' {_p(x['p'])}" for x in best['alts']) + '.'+ elif len(rows) > 1:+ s += ' Forced choice among your queries: ' + ', '.join(f"'{r['query']}' {_p(r['p_vs_alts'])}" for r in rows) + '.'+ alt_txt = ', '.join(f"'{x['term']}' (z {vocab.zf(x['z'])})" for x in near if x['term'] not in queries)+ if alt_txt:+ s += f' It sounds most like: {alt_txt}.'+ ev = VERDICT_EVIDENCE[best['verdict']]+ if best['verdict'] != 'no match' and best['validated']:+ s += (f" Evidence for '{best['verdict']}' (held-out library search, ~3% of candidate pairs true): the right kind "+ f"of sound {100 * ev['precision_library']:.0f}% of the time, the exact variant "+ f"{100 * ev['precision_library_strict']:.0f}% (a close variant such as an Enter key for a spacebar counts as "+ 'wrong then).')+ pk = peak_dbfs(path, start, end)+ if pk < SILENT_DBFS:+ s = f'WARNING: near-silent input (peak {pk:.0f} dBFS), scores are meaningless. ' + s+ res = {'summary': s, 'path': path, 'start': start, 'end': end, 'peak_dbfs': round(pk, 1)}+ res.update({k: best[k] for k in ('query', 'cosine', 'z', 'percentile', 'rank_in_vocab', 'p_vs_alts', 'verdict',+ 'validated')})+ if best.get('alts'):+ res['alts'] = best['alts']+ res.update(results=rows, closest_vocab=near, verdict_thresholds_z={n: t for t, n in VERDICTS},+ verdict_evidence=VERDICT_EVIDENCE)+ return res+++def add_cli(sub):+ p = sub.add_parser('match', help='how well does a sound match a text description (calibrated z + verdict)')+ p.add_argument('path', help='audio file or video')+ p.add_argument('query', nargs='+', help='one or more descriptions')+ p.add_argument('--alts', nargs='+', help='alternative descriptions for a forced choice (p_vs_alts)')+ p.add_argument('--start', type=float, help='judge only from this time (s)')+ p.add_argument('--end', type=float, help='judge only up to this time (s)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(lambda: match(a.path, a.query, a.alts, a.start, a.end), a))+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/mcp_server.py b/earshot/mcp_server.pynew file mode 100644index 0000000..edb6a6c--- /dev/null+++ b/earshot/mcp_server.py@@ -0,0 +1,628 @@+"""earshot as an MCP server: coding agents call earshot's ears as tools (stdio transport).++ earshot mcp # started by the MCP client, not by hand; speaks JSON-RPC on stdin/stdout+ earshot mcp --warm # also load the models in the background at startup (the first model call is then fast)+ earshot mcp --tools # list the tools and exit+ python -m earshot.mcp_server [--warm] # the same server without the CLI front end++Register with Claude Code (user scope = every project; -e passes environment variables to the server):+ claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp+Any MCP client (project .mcp.json, Claude Desktop, Cursor ...):+ {"mcpServers": {"earshot": {"command": "/path/to/earshot/.venv/bin/python", "args": ["-m", "earshot", "mcp"],+ "env": {"EARSHOT_THREADS": "3", "GEMINI_API_KEY": "${GEMINI_API_KEY:-}"}}}}+GEMINI_API_KEY is only for the optional llm_critique tool: keep the key in the user's own environment and reference it+as ${GEMINI_API_KEY:-} (Claude Code expands it at startup); never write a literal key into .mcp.json or any project+file. The server answers the handshake within a few seconds, longer on a cold disk or a busy machine; Claude Code waits+MCP_TIMEOUT ms for it, so on a slow machine start Claude Code with MCP_TIMEOUT=60000 (PowerShell:+$env:MCP_TIMEOUT=60000; claude) or the tools are missing for the whole session.++Tools: measure, describe, match, rank, find, compare, ab, realness, timeline, sheet, critique, doctor, llm_critique.+Models load on the first call that needs them (~20-60 s) and stay warm for the life of the server; calls run one at a+time. Each result is text: the plain-language summary first (plus the listening transcript for timeline / the+critique text for llm_critique / one row per file for realness on a folder), then compact JSON (always valid JSON;+full=true for everything). sheet also returns the picture as an image. A tool whose module is missing from this+install answers with a clear message instead of failing the server.+"""+from __future__ import annotations++import contextlib+import functools+import importlib.util+import inspect+import io+import json+import os+import sys+import threading+import time+from typing import Annotated++from . import audio, config+from .report import _clean, emit++INSTRUCTIONS = """earshot gives you ears: you cannot hear, so use these tools whenever you must judge, choose or check+sound (sound effects, music, ambience, a mix, the audio of a video). Use absolute paths.+Loops: choose an asset = rank (many files) -> find (best moment in the winner) -> describe + realness (verify).+Review a mix or film = timeline (what happens when, sync to cuts/cues) -> sheet (look at the picture) -> critique+(scorecard vs professional launch films). A/B a change = ab on before/after renders, then timeline on the new one.+measure = exact meters (LUFS, dBTP, noise floor), no models. llm_critique = optional Gemini opinion, uploads audio:+only with the user's explicit consent.+Reading numbers: z = how far a description stands out among the few hundred vocabulary descriptions scored on the+same sound (>= 3 strong, 2.5-3 plausible, 2-2.5 weak, < 2 no match). p_real and AudioSet values are probabilities;+p_vs_alts is an uncalibrated forced choice among the alternatives you give (sharp, only as good as those+alternatives); cosine is raw CLAP similarity (not a probability, only comparable for the same sound and query). Trust+measurements most, verdicts and z next, cosine least. If realness is unavailable, say real-vs-synthetic was not+checked; describe's AudioSet synth classes (Sine wave, Beep, Synthesizer) are weaker evidence. The first call that+needs models takes ~20-60 s; later calls are fast (models stay loaded)."""++IMG_INLINE_MAX = 1600 # px, long edge of the inline copy (sent as JPEG q88: ~1/3 of the PNG bytes)+JSON_MAX = 7000+SILENT_PEAK_DBFS = -70.0 # same near-silence line as describe+TEXT_MAX = 24000+_LOCK = threading.RLock() # one tool call (or the warm-up) at a time: no import races, gentle on the CPU+++class ToolFailure(Exception):+ """a readable error for the agent (becomes an MCP error result)"""+++def _field_types():+ """tool signatures are string annotations resolved when the server starts: only `earshot mcp` imports pydantic"""+ from pydantic import Field+ globals().update(+ Field=Field,+ Path=Annotated[str, Field(description='absolute path to an audio or video file (wav, flac, ogg, mp3, m4a, or '+ 'the audio track of mp4/mov/webm)')],+ Paths=Annotated[list[str] | str, Field(description='files, folders (searched recursively) or globs: one '+ 'string or a list')],+ Full=Annotated[bool, Field(description='return the complete JSON instead of the compact one')],+ Start=Annotated[float | None, Field(description='optional start of the segment to analyse, seconds')],+ End=Annotated[float | None, Field(description='optional end of the segment, seconds')],+ Query=Annotated[str, Field(description='a plain description of the sound, e.g. "a deep thocky mechanical '+ 'keyboard key press" (describe the sound itself, not its use)')],+ Video=Annotated[str | None, Field(description='optional path to the film (mp4/mov) for picture cuts and '+ 'frames')],+ Cues=Annotated[list[float] | str | None, Field(description='optional picture event times: a list of seconds, '+ '"1.2,3.4", or an absolute path to a cues JSON '+ 'file ({fps, events: [{f or t, kind}]})')])+++def _file(path, what='file'):+ if not path or not isinstance(path, str):+ raise ToolFailure(f'{what}: a file path is required')+ p = os.path.abspath(os.path.expanduser(path.strip().strip('"')))+ if not os.path.exists(p):+ raise ToolFailure(f'{what} not found: {p} (pass an absolute path; the server runs in {os.getcwd()})')+ if os.path.isdir(p):+ raise ToolFailure(f'{what} is a folder, expected a file: {p}')+ return p+++def _opt_file(path, what):+ return _file(path, what) if path else None+++def _many(paths, what='paths'):+ """files of every given file / folder / glob; a given path that yields nothing is an error, not silently skipped"""+ if isinstance(paths, str):+ paths = [paths]+ if not paths:+ raise ToolFailure(f'{what}: give at least one file, folder or glob')+ expanded = [os.path.abspath(os.path.expanduser(str(p).strip().strip('"'))) for p in paths]+ empty = [p for p in expanded if not audio.list_media([p])]+ if empty:+ raise ToolFailure(f'{what}: no audio/video files found in {", ".join(empty[:5])} (no such file or folder, or '+ f'nothing playable in it; the server runs in {os.getcwd()}). Fix or remove '+ f'{"it" if len(empty) == 1 else "them"}.')+ return audio.list_media(expanded)+++def _api(name):+ """the public earshot function `name`, imported lazily, or a ToolFailure saying it is not available yet"""+ from earshot import api+ try:+ return api._fn(name)+ except Exception as e: # noqa: BLE001+ mod = api._FUNCS.get(name, (name, name))[0]+ if importlib.util.find_spec(f'earshot.{mod}') is None:+ present = [k for k in (f.__name__[5:] for f, _, _ in TOOLS)+ if k not in api._FUNCS or importlib.util.find_spec(f'earshot.{api._FUNCS[k][0]}')]+ raise ToolFailure(f'earshot.{name} is not part of this install yet. Tools with their module present: '+ f'{", ".join(present)}.') from None+ raise ToolFailure(f'earshot.{name} could not be loaded ({type(e).__name__}: {e}). If earshot is being '+ 'updated, retry in a moment; `earshot doctor` checks the install.') from None+++def _call(fn, kw):+ """call fn with the non-None keyword arguments it accepts; note the ones this earshot version does not take"""+ kw = {k: v for k, v in kw.items() if v is not None}+ params = inspect.signature(fn).parameters+ if any(p.kind == p.VAR_KEYWORD for p in params.values()):+ return fn(**kw), []+ return fn(**{k: v for k, v in kw.items() if k in params}), [k for k in kw if k not in params]+++def _shrink(o, n, depth=0):+ if isinstance(o, float):+ return round(o, 3)+ if isinstance(o, str):+ return o if len(o) <= 400 else o[:400] + ' ...'+ if isinstance(o, dict):+ return {k: _shrink(v, n, depth + 1) for k, v in o.items() if not (depth == 0 and k in ('summary', 'text'))}+ if isinstance(o, list):+ out = [_shrink(v, n, depth + 1) for v in o[:n]]+ return out + ([f'... +{len(o) - n} more'] if len(o) > n else [])+ return o+++def _dumps(o):+ return json.dumps(o, ensure_ascii=False, separators=(',', ':'))+++def compact(r, limit=JSON_MAX):+ """short, always valid JSON for an agent's context: floats to 3 decimals, long lists cut, summary/text left out;+ if that is still too long the largest top-level fields are left out and named in 'cut_fields'"""+ c = _clean(r)+ for n in (24, 12, 6, 3, 1):+ d = _shrink(c, n)+ s = _dumps(d)+ if len(s) <= limit:+ return s+ if not isinstance(d, dict):+ return _dumps({'cut': 'too long for the compact view; call again with full=true'})+ cut = []+ for k in sorted(d, key=lambda k: -len(_dumps(d[k]))):+ cut.append(k)+ s = _dumps({**{k2: v for k2, v in d.items() if k2 not in cut}, 'cut_fields': cut,+ 'note': 'fields left out to stay short; call again with full=true'})+ if len(s) <= limit:+ break+ return s+++def _format(r, full=False, ignored=()):+ head = str(r.get('summary') or '').strip()+ body = str(r.get('text') or '').strip()+ if body.startswith(head):+ head = ''+ if len(body) > TEXT_MAX and not full:+ body = body[:TEXT_MAX] + f'\n... (text cut at {TEXT_MAX} characters; full=true gives all of it)'+ js = json.dumps(_clean(r), ensure_ascii=False) if full else compact(r)+ note = f'(this earshot version ignored: {", ".join(ignored)})' if ignored else ''+ return '\n\n'.join(x for x in (head, body, note, ('JSON: ' if full else 'JSON (compact): ') + js) if x)+++def _result(name, kw, fn=None):+ """(result dict, ignored kwargs) of one earshot call; any failure becomes a readable ToolFailure"""+ with _LOCK, contextlib.redirect_stdout(sys.stderr):+ fn = fn or _api(name)+ try:+ r, ignored = _call(fn, kw)+ except ToolFailure:+ raise+ except Exception as e: # noqa: BLE001+ raise ToolFailure(f'{name} failed: {type(e).__name__}: {e}') from None+ return (r if isinstance(r, dict) else {'summary': str(r)}), ignored+++def _run(name, kw, full=False, fn=None):+ r, ignored = _result(name, kw, fn)+ return _format(r, full, ignored)+++def _cues(cues):+ """cue times as given (list / "1.2,3.4" / JSON text) or an absolute path to a cues JSON file"""+ if isinstance(cues, str) and cues.strip() and cues.strip()[:1] not in '[{0123456789.-':+ return _file(cues, 'cues')+ return cues or None+++def _detail(detail):+ if detail not in ('low', 'normal', 'high'):+ raise ToolFailure('detail must be "low", "normal" or "high"')+++def measure_summary(m):+ """one plain sentence from a features.measure() dict"""+ if m.get('error'):+ return f'{m["duration"]:.3f} s: {m["error"]}.'+ if m.get('peak_dbfs', 0) < SILENT_PEAK_DBFS:+ what = 'digitally silent' if m.get('digital_silence_frac', 0) >= 0.999 else 'near-silent'+ return (f'{m["duration"]:.3f} s, {what} (peak {m["peak_dbfs"]:.0f} dBFS): nothing to measure; the loudness, '+ 'spectrum and envelope values of silence are meaningless.')+ parts = [f'{m["duration"]:.3f} s']+ if 'lufs' in m:+ parts.append(f'{m["lufs"]} LUFS integrated' + (f' (short-term max {m["short_term_max_lufs"]})'+ if 'short_term_max_lufs' in m else ''))+ else:+ parts.append(f'RMS {m["rms_dbfs"]} dBFS (too short for LUFS)')+ parts.append(f'true peak {m["true_peak_dbtp"]} dBTP' + (f', {m["clipped_samples"]} clipped samples'+ if m.get('clipped_samples') else ''))+ if 'lra_lu' in m:+ parts.append(f'LRA {m["lra_lu"]} LU')+ parts.append(f'noise floor {m["noise_floor_dbfs"]} dBFS')+ if m.get('brightness_hz'):+ parts.append(f'brightness {m["brightness_hz"] / 1000:.1f} kHz')+ if m.get('bandwidth_hz'):+ parts.append(f'bandwidth {m["bandwidth_hz"] / 1000:.1f} kHz')+ if m.get('attack_ms') is not None:+ parts.append(f'attack {m["attack_ms"]:g} ms' + (f', decay {m["decay_ms"]:g} ms' if m.get('decay_ms') else ''))+ if 'onset_count' in m:+ parts.append(f'{m["onset_count"]} onsets')+ if 'tempo' in m:+ parts.append(f'tempo ~{m["tempo"]:g} BPM, key {m["key"][0]["key"] if m.get("key") else "?"}')+ s = ', '.join(parts) + '.'+ if m.get('digital_silence_frac', 0) > 0.01:+ s += (f' {100 * m["digital_silence_frac"]:.1f}% exact digital zeros (usually edit, fade or gate padding; weak '+ 'evidence on its own: use realness to judge real vs synthesized).')+ return s+++# ---------------------------------------------------------------------------------------------------------- tools+def tool_measure(path: Path, start: Start = None, end: End = None,+ music: Annotated[bool, Field(description='also estimate tempo, beats and key (music only)')] = False,+ light: Annotated[bool, Field(description='skip the slower parts (onsets, harmonic ratio, pitch): '+ 'levels and spectrum only')] = False,+ full: Full = False) -> str:+ """Exact meter readings of one file, no AI models, fast: duration, integrated LUFS, true peak dBTP, loudness+ range, noise floor, clipped samples, exact digital zeros, attack/decay ms, brightness, bandwidth, band energy,+ onsets (music=true adds tempo, beats, key). Use for loudness / peak / clipping / silence checks, to compare levels+ before and after a mix change, or to find where events start. These numbers are measurements: trust them."""+ from . import features+ p = _file(path)++ def run(path, start=None, end=None, music=False, light=False):+ m = features.measure(path, music=music, start=start, end=end, light=light)+ return dict(summary=measure_summary(m), **m)+ return _run('measure', dict(path=p, start=start, end=end, music=music, light=light), full, run)+++def tool_describe(path: Path, start: Start = None, end: End = None,+ top: Annotated[int, Field(description='tags per category', ge=1, le=30)] = 8,+ full: Full = False) -> str:+ """What does this sound like? Calibrated tags per category (source, material, action, character ...), AudioSet+ top labels and key measurements, with a one-paragraph summary. Use first on any unknown file, or to check that a+ downloaded asset really is what its title claims. Loads the CLAP + AST models (first call ~20-60 s)."""+ return _run('describe', dict(path=_file(path), start=start, end=end, top=top), full)+++def tool_match(path: Path, query: Query,+ alts: Annotated[list[str] | None, Field(description='1-5 plausible alternative descriptions to '+ 'compete with the query, e.g. ["a clicky blue '+ 'switch", "a typewriter key"]')] = None,+ start: Start = None, end: End = None, full: Full = False) -> str:+ """How well one sound matches a text description, calibrated: z (how far the description stands out among the+ vocabulary descriptions scored on this sound; vocab_size gives their count), verdict (z >= 3 strong, 2.5-3+ plausible, 2-2.5 weak, below 2 no match), percentile, p_vs_alts (an uncalibrated, sharp forced choice of the query+ among your alternatives), raw cosine (not a probability) and the closest vocabulary descriptions (what else it+ could be). Use to verify a specific claim about one file ("is this a deep thocky keyboard press?"). Judge by z+ and verdict; p_vs_alts only says which of the given wordings fits best, so give plausible alternatives."""+ return _run('match', dict(path=_file(path), query=query, alts=alts, start=start, end=end), full)+++def tool_rank(paths: Paths, query: Query,+ top: Annotated[int | None, Field(description='return only the best N', ge=1)] = 20,+ full: Full = False) -> str:+ """Sort many candidate files by how well they match a description (calibrated). Use to choose the best asset from+ a folder of downloads or a sample library; then run find on the winner for the exact moment, and describe ++ realness to verify it. Embeddings are cached, so re-ranking the same folder with a new query is fast."""+ return _run('rank', dict(paths=_many(paths), query=query, top=top), full)+++def tool_find(path: Path, query: Query,+ top: Annotated[int, Field(description='number of segments', ge=1, le=50)] = 5,+ kind: Annotated[str, Field(description='"event" = onset-based trims only (hits / clicks / steps), '+ '"texture" = plain windows only (beds, ambiences), "auto" reads '+ 'the query and scores both')] = 'auto',+ win: Annotated[float | None, Field(description='texture window length in seconds (default: duration / '+ '6, clamped to 1-5 s)')] = None,+ hop: Annotated[float | None, Field(description='hop between texture windows in seconds')] = None,+ start: Start = None, end: End = None, full: Full = False) -> str:+ """Find the best moments inside one long recording that match a description: segments [{start, end, z, verdict+ kind, head_db, tail_db, cut ...}] in seconds. kind stroke / event = onset-based trims (5 ms before the attack,+ after the decay); window = a stretch whose edges are snapped to the sound for one-shot queries (plain, to be+ crossfaded, for beds). cut = 'head cut' / 'tail cut' when the trim starts or ends inside a sound (ranked lower:+ check before using it). Use when a sound pack, ambience bed or typing take holds many events and you need the+ single cleanest one; then describe / measure / realness / ab that segment with its start and end."""+ if kind not in ('auto', 'event', 'texture'):+ raise ToolFailure('kind must be "auto", "event" or "texture"')+ return _run('find', dict(path=_file(path), query=query, top=top, kind=kind, win=win, hop=hop, start=start,+ end=end), full)+++def tool_compare(path: Path,+ refs: Annotated[list[str] | str, Field(description='reference files, folders or globs: one string '+ 'or a list')],+ k: Annotated[int, Field(description='nearest references to list', ge=1, le=50)] = 5,+ start: Start = None, end: End = None, full: Full = False) -> str:+ """Similarity of one sound to a set of reference recordings: the nearest references, typicality (z >= -1 fits+ the set, z < -2 is an outlier) and which measurements differ from the references, in words. Use to check that a+ candidate fits an established palette, or sounds like a reference the user loves. start / end judge only that+ part of the file (e.g. a trim from find)."""+ return _run('compare', dict(path=_file(path), refs=_many(refs, 'refs'), k=k, start=start, end=end), full)+++def tool_ab(a: Annotated[str, Field(description='path of sound A (e.g. the current version)')],+ b: Annotated[str, Field(description='path of sound B (e.g. the candidate / new render)')],+ query: Annotated[str | None, Field(description='optional description both should match')] = None,+ refs: Annotated[list[str] | str | None, Field(description='optional reference files, folders or globs '+ 'both should resemble')] = None,+ a_start: Annotated[float | None, Field(description='use A only from this time (s), e.g. a find trim',+ ge=0)] = None,+ a_end: Annotated[float | None, Field(description='use A only up to this time (s)', gt=0)] = None,+ b_start: Annotated[float | None, Field(description='use B only from this time (s)', ge=0)] = None,+ b_end: Annotated[float | None, Field(description='use B only up to this time (s)', gt=0)] = None,+ full: Full = False) -> str:+ """A/B two sounds: which is closer to a description and/or a reference set, and how they differ in measured+ terms (louder, brighter, longer decay ...). Use to decide between two candidates, or to confirm a mix change did+ what you intended (before vs after render). For multi-take files pass the trims (a_start / a_end, b_start /+ b_end; A and B may be the same file): a whole-file A/B compares every take plus the silence between them."""+ return _run('ab', dict(a=_file(a, 'a'), b=_file(b, 'b'), query=query,+ refs=_many(refs, 'refs') if refs else None, a_start=a_start, a_end=a_end,+ b_start=b_start, b_end=b_end), full)+++def tool_realness(path: Annotated[str, Field(description='an audio/video file, or a folder / glob of candidates (one '+ 'row per file)')],+ start: Start = None, end: End = None,+ ast: Annotated[bool, Field(description='also run the AudioSet tagger for synth-class evidence '+ '(slower)')] = False,+ full: Full = False) -> str:+ """Real recording / professionally produced audio, or cheap code-synthesized sound (oscillators, filtered noise,+ simple envelopes, beeps)? p_real (a probability) with a verdict and the evidence in words; files over 6 s also get+ per-window verdicts; a folder or glob gives one row per file. Electronic music and pro sound design count as real:+ it flags amateur procedural audio. Use on every sound before it goes into a film and on finished mixes: the user+ rejects synthetic-sounding audio. Strong evidence, not proof: read the evidence list."""+ fn = _api('realness')+ mod = sys.modules.get('earshot.realness')+ model = getattr(mod, 'MODEL_PATH', None)+ if model and not os.path.isfile(model):+ raise ToolFailure('realness is unavailable in this install (its model file is missing). Tell the user that '+ 'real-vs-synthesized was NOT checked; as weaker evidence use describe (AudioSet classes such '+ 'as Sine wave, Beep, Synthesizer) and measure.')+ p = os.path.abspath(os.path.expanduser(path.strip().strip('"'))) if isinstance(path, str) else ''+ if os.path.isfile(p):+ return _run('realness', dict(path=p, start=start, end=end, ast=ast or None), full, fn)+ if not os.path.isdir(p) and not any(ch in p for ch in '*?['):+ _file(path) # a plain path that does not exist: the usual "not found" message+ files = _many(path)+ many = getattr(mod, 'realness_many', None)+ if many is None:+ raise ToolFailure('this earshot version judges one file per call: pass a file')+ r, ignored = _result('realness', dict(paths=files, ast=ast or None, start=start, end=end), many)+ rows = [r.get('summary', ''), 'p_real verdict file: main reason']+ for f in r.get('files', []):+ pr = '-' if f.get('p_real') is None else f'{f["p_real"]:.2f}'+ rows.append(f'{pr:>6} {f.get("verdict", ""):<24} {os.path.basename(f.get("path", ""))}: '+ f'{f.get("why") or f.get("summary", "")}')+ r['text'] = '\n'.join(rows)+ return _format(r, full, ignored)+++def tool_timeline(path: Path, video: Video = None,+ cues: Cues = None, start: Start = None, end: End = None,+ detail: Annotated[str, Field(description='"low" (fast), "normal" or "high" (every event tagged, '+ 'slow)')] = 'normal',+ full: Full = False) -> str:+ """Listen through a whole soundtrack or film: a listening transcript (events with time, level, tags, AudioSet+ labels), loudness curve, picture cuts (with video) and sync stats (event-to-cut / event-to-cue offsets). Use to+ review a mix or a film's audio, to learn what happens when, and to check that sounds land on picture. For a video+ file the picture is read from the same file unless video= is given."""+ p = _file(path)+ _detail(detail)+ vid = _opt_file(video, 'video') or (p if os.path.splitext(p)[1].lower() in audio.VIDEO_EXTS else None)+ return _run('timeline', dict(path=p, video=vid, cues=_cues(cues), start=start, end=end, detail=detail), full)+++def tool_sheet(path: Path, video: Video = None, cues: Cues = None, start: Start = None, end: End = None,+ detail: Annotated[str, Field(description='model budget of the underlying timeline: "low" (fast), '+ '"normal" or "high" (every event tagged, slow)')] = 'normal',+ out_png: Annotated[str | None, Field(description='where to write the PNG (default: a sheets folder in '+ 'the earshot cache)')] = None,+ include_image: Annotated[bool, Field(description='also return the picture inline (needs a client '+ 'that shows images)')] = True,+ full: Full = False):+ """Draw an annotated picture of a soundtrack or film on one time axis: video frames, sections, labelled events, a+ spectrogram, waveform with loudness, beats, and cue markers coloured by sync (green = hit within a frame, amber =+ within 100 ms, red = none); cuts are dashed lines. Returns the image plus its PNG path. Use when you can look at+ images: structure, density, gaps and sync at a glance; zoom with start/end, confirm details with timeline."""+ p = _file(path)+ _detail(detail)+ vid = _opt_file(video, 'video') or (p if os.path.splitext(p)[1].lower() in audio.VIDEO_EXTS else None)+ if out_png:+ out = os.path.abspath(os.path.expanduser(out_png))+ else:+ span = '' if start is None and end is None else f'_{start or 0:g}-{"end" if end is None else f"{end:g}"}'+ out = os.path.join(config.HOME, 'sheets',+ f'{os.path.splitext(os.path.basename(p))[0]}_{audio.file_id(p)[:8]}{span}.png')+ os.makedirs(os.path.dirname(out), exist_ok=True)+ r, ignored = _result('sheet', dict(path=p, out_png=out, video=vid, cues=_cues(cues), start=start, end=end,+ detail=detail))+ png, text = r.get('png') or out, _format(r, full, ignored)+ if not os.path.isfile(png):+ return f'{text}\n\n(no PNG found at {png})'+ text = f'PNG: {png}\n\n{text}'+ return (text, _image_for_agent(png)) if include_image else text+++def _image_for_agent(path):+ from PIL import Image+ im = Image.open(path)+ im.thumbnail((IMG_INLINE_MAX, IMG_INLINE_MAX))+ buf = io.BytesIO()+ im.convert('RGB').save(buf, 'JPEG', quality=88, optimize=True)+ return buf.getvalue()+++def tool_critique(path: Path, video: Video = None,+ profile: Annotated[str | None, Field(description='optional reference profile JSON from `earshot '+ 'profile` (default: the bundled launch films)')]+ = None,+ fast: Annotated[bool, Field(description='skip the AudioSet and realness passes (seconds instead of '+ 'minutes)')] = False,+ full: Full = False) -> str:+ """Launch-film sound scorecard: each metric of the soundtrack (loudness, true peak, dynamics, spectral balance,+ event density, sync to picture, realness ...) against the range measured on professional launch films, with+ status ok / low / high, weight, why it matters and concrete actions with timestamps, plus an overall verdict+ (launch-grade / close / needs work / not launch-grade). Use on a finished soundtrack or film before delivery, and+ again after each round of fixes."""+ p = _file(path)+ vid = _opt_file(video, 'video') or (p if os.path.splitext(p)[1].lower() in audio.VIDEO_EXTS else None)+ kw = dict(path=p, profile=_opt_file(profile, 'profile'), video=vid)+ if fast:+ kw.update(use_ast=False, use_realness=False)+ return _run('critique', kw, full)+++def tool_doctor(load: Annotated[bool, Field(description='also load both models and time one window (~20-60 s)')]+ = False, full: Full = False) -> str:+ """Check the earshot install: python packages, ffmpeg, cached models, CPU threads, whether an LLM key is present.+ Use when a tool fails, or once before the first real use."""+ from . import doctor+ return _run('doctor', dict(load=load), full, doctor.check)+++def tool_llm_critique(path: Path,+ consent: Annotated[bool, Field(description='true ONLY if the user explicitly agreed that this '+ 'audio is uploaded to Google')] = False,+ prompt: Annotated[str | None, Field(description='optional specific question to answer as '+ 'well')] = None,+ video: Video = None,+ with_timeline: Annotated[bool, Field(description="also send earshot's listening transcript as "+ 'context (loads models)')] = False,+ model: Annotated[str | None, Field(description='Gemini model id (default '+ 'EARSHOT_GEMINI_MODEL or gemini-3.8-flash)')]+ = None,+ start: Start = None, end: End = None, full: Full = False) -> str:+ """OPTIONAL second opinion from Google Gemini (an audio LLM): rubric critique of sync, realism, mix balance,+ loudness, cheap / synthetic moments, and concrete fixes with timestamps. Use after timeline / critique when a+ judgement in words would help (what sounds cheap, what a mixer would change). It UPLOADS the audio to Google: call+ with consent=true only after the user explicitly agreed, and it needs GEMINI_API_KEY in the server's environment.+ Without key or consent it returns how to enable it (nothing is sent). An opinion, not a measurement: timestamps+ are approximate; verify fixes with timeline / ab."""+ from . import llm+ p = _file(path)+ kw = dict(path=p, consent=consent, prompt=prompt, video=_opt_file(video, 'video'), with_timeline=with_timeline,+ model=model, start=start, end=end)+ return _run('llm_critique', kw, full, llm.critique)+++TOOLS = [ # (function, read-only, talks to the internet)+ (tool_measure, True, False), (tool_describe, True, False), (tool_match, True, False), (tool_rank, True, False),+ (tool_find, True, False), (tool_compare, True, False), (tool_ab, True, False), (tool_realness, True, False),+ (tool_timeline, True, False), (tool_sheet, False, False), (tool_critique, True, False),+ (tool_doctor, True, False), (tool_llm_critique, False, True), # uploads audio: clients should ask first+]+++def _sdk():+ try:+ from mcp.server.mcpserver import Image, MCPServer as Server+ from mcp.server.mcpserver.exceptions import ToolError+ except ImportError: # mcp 1.x+ from mcp.server.fastmcp import FastMCP as Server, Image+ from mcp.server.fastmcp.exceptions import ToolError+ from mcp.types import ToolAnnotations+ return Server, Image, ToolError, ToolAnnotations+++def _adapt(fn, Image, ToolError):+ @functools.wraps(fn)+ def run(**kw):+ try:+ with _LOCK:+ out = fn(**kw)+ except ToolFailure as e:+ raise ToolError(str(e)) from None+ if isinstance(out, tuple):+ return [out[0], Image(data=out[1], format='jpeg')]+ return out+ return run+++def build():+ """the configured MCP server object (not running)"""+ from . import __version__+ Server, Image, ToolError, ToolAnnotations = _sdk()+ _field_types()+ try:+ srv = Server(name='earshot', instructions=INSTRUCTIONS, log_level='WARNING', version=__version__)+ except TypeError: # mcp 1.x FastMCP: no version argument+ srv = Server(name='earshot', instructions=INSTRUCTIONS, log_level='WARNING') # stderr stays readable+ low = getattr(srv, '_mcp_server', None)+ if low is not None:+ low.version = __version__+ for fn, ro, net in TOOLS:+ name = fn.__name__[len('tool_'):]+ kw = dict(name=name, description=_doc(fn),+ annotations=ToolAnnotations(title=f'earshot {name}', readOnlyHint=ro, destructiveHint=False,+ openWorldHint=net))+ try:+ srv.add_tool(_adapt(fn, Image, ToolError), structured_output=False, **kw)+ except TypeError: # older SDKs without structured_output+ srv.add_tool(_adapt(fn, Image, ToolError), **kw)+ srv.earshot_version = __version__+ return srv+++def warm():+ """load the models in a background thread, holding the tool lock (two threads importing torch / scipy for the first+ time at once can see half-initialised modules). No stdout redirect here: it raced with the stdio transport, which+ picks up sys.stdout when it starts (the initialize answer then went to stderr and the client hung)."""+ def load():+ t = time.time()+ with _LOCK:+ try:+ import numpy as np+ from . import features+ from .models import ast, clap+ clap.get()+ ast.get()+ features.measure(np.random.RandomState(0).randn(48000).astype(np.float32) * 0.1) # librosa import + JIT+ if importlib.util.find_spec('earshot.vocab'):+ from . import vocab+ vocab.get()+ print(f'earshot mcp: models warm ({time.time() - t:.0f} s)', file=sys.stderr)+ except Exception as e: # noqa: BLE001+ print(f'earshot mcp: warm-up failed: {type(e).__name__}: {e}', file=sys.stderr)+ threading.Thread(target=load, daemon=True).start()+++def _doc(fn):+ return ' '.join(fn.__doc__.split())+++def serve(args):+ if getattr(args, 'tools', False):+ rows = [{'name': f.__name__[5:], 'read_only': ro, 'network': net, 'description': _doc(f)}+ for f, ro, net in TOOLS]+ head = f'{len(rows)} MCP tools. Register: claude mcp add earshot -- {sys.executable} -m earshot mcp'+ lines = [f'{r["name"]:13s} {r["description"].split(". ")[0]}.' for r in rows]+ return emit({'summary': head, 'tools': rows}, args, text='\n'.join([head] + lines)) and 0+ try:+ srv = build()+ except ImportError as e:+ print(f'earshot mcp needs the MCP SDK: pip install "earshot[mcp]" ({e})', file=sys.stderr)+ return 1+ if args.warm:+ warm()+ print(f'earshot mcp {srv.earshot_version}: serving {len(TOOLS)} tools on stdio (threads={config.threads()})',+ file=sys.stderr)+ srv.run('stdio')+ return 0+++def add_cli(sub):+ from .cli import common+ p = common(sub.add_parser('mcp', help='run earshot as an MCP server (stdio) for coding agents',+ description='MCP stdio server. Register: claude mcp add earshot -- '+ '<python> -m earshot mcp'))+ p.add_argument('--warm', action='store_true', help='load CLAP + AST in the background at startup')+ p.add_argument('--tools', action='store_true', help='list the tools and exit (does not start the server)')+ p.set_defaults(func=serve)+ return p+++if __name__ == '__main__': # python -m earshot.mcp_server [--warm]: the same server without loading the other commands+ import argparse+ sys.exit(serve(add_cli(argparse.ArgumentParser(prog='earshot').add_subparsers()).parse_args()))diff --git a/earshot/models/__init__.py b/earshot/models/__init__.pynew file mode 100644index 0000000..e69de29diff --git a/earshot/models/ast.py b/earshot/models/ast.pynew file mode 100644index 0000000..d6bcc2d--- /dev/null+++ b/earshot/models/ast.py@@ -0,0 +1,111 @@+"""AudioSet tagger (Audio Spectrogram Transformer) on CPU: 527 sound classes with independent probabilities.++ from earshot.models import ast+ t = ast.get()+ t.top_file('beep.wav', k=5) # [('Beep, bleep', 0.71), ('Sine wave', 0.33), ...]++A second, independent "ear" next to CLAP: it was trained supervised on AudioSet, so its probabilities are calibrated+per class (CLAP's cosine scores are not). Useful classes for film sound: 'Synthesizer', 'Sine wave', 'Beep, bleep',+'Whoosh, swoosh, swish', 'Typing', 'Computer keyboard', 'Click', 'Boom', 'Music', 'Speech', 'Silence' ...+Input is resampled to 16 kHz (so it hears up to 8 kHz only) and read in 10.24 s windows; longer audio is split into+windows and the per-class maximum (or mean) is taken.+"""+import hashlib+import os+import threading+import warnings++import numpy as np++from .. import audio, config+from .clap import _load_hf, _set_precision, _slug, offline_if_cached++SR_AST = 16000+WIN_S = 10.24+_lock = threading.Lock()+_inst = None+++class AST:+ def __init__(self, model_id=None):+ config.ensure_dirs()+ offline_if_cached(model_id or config.AST_MODEL)+ import torch+ from transformers import ASTFeatureExtractor, ASTForAudioClassification+ torch.set_num_threads(config.threads())+ self.torch = torch+ self.model_id = model_id or config.AST_MODEL+ self.model = _load_hf(ASTForAudioClassification, self.model_id).eval()+ self.fe = _load_hf(ASTFeatureExtractor, self.model_id)+ self.dtype = _set_precision(self.model, torch)+ self.precision = 'bf16' if self.dtype == torch.bfloat16 else 'fp32'+ self.labels = [self.model.config.id2label[i] for i in range(len(self.model.config.id2label))]+ self.index = {l: i for i, l in enumerate(self.labels)}+ self.dir = os.path.join(config.EMB_CACHE, 'ast', _slug(self.model_id) + '--' + self.precision)+ os.makedirs(self.dir, exist_ok=True)++ def _forward(self, segs16, batch=8):+ torch = self.torch+ out = []+ for i in range(0, len(segs16), batch):+ chunk = [np.pad(s, (0, max(0, 1600 - len(s)))) for s in segs16[i:i + batch]]+ with warnings.catch_warnings():+ warnings.filterwarnings('ignore', message='.*mel filter.*')+ inp = self.fe(chunk, sampling_rate=SR_AST, return_tensors='pt')+ inp = {k: (v.to(self.dtype) if v.is_floating_point() else v) for k, v in inp.items()}+ with torch.inference_mode():+ logits = self.model(**inp).logits+ out.append(torch.sigmoid(logits.float()).numpy().astype(np.float32))+ return np.concatenate(out) if out else np.zeros((0, len(self.labels)), np.float32)++ def probs(self, arrays, reduce='max', batch=8):+ """48 kHz arrays -> (n, 527) class probabilities (windows of 10.24 s reduced by max or mean)"""+ if isinstance(arrays, np.ndarray) and arrays.ndim == 1:+ arrays = [arrays]+ segs, owner = [], []+ n = int(WIN_S * SR_AST)+ for j, x in enumerate(arrays):+ y = audio.resample(audio.to_mono(np.asarray(x, np.float32)), audio.SR, SR_AST)+ if len(y) <= n:+ segs.append(y); owner.append(j)+ else:+ for s in range(0, len(y) - n + 1, n // 2):+ segs.append(y[s:s + n]); owner.append(j)+ if (len(y) - n) % (n // 2):+ segs.append(y[-n:]); owner.append(j)+ p = self._forward(segs, batch)+ owner = np.array(owner)+ red = np.max if reduce == 'max' else np.mean+ return np.stack([red(p[owner == j], axis=0) for j in range(len(arrays))])++ def probs_file(self, path, start=None, end=None, reduce='max'):+ k = hashlib.sha1(f'{audio.file_id(path)}|{start}|{end}|{reduce}'.encode()).hexdigest()[:24]+ f = os.path.join(self.dir, 'p_' + k + '.npy')+ if os.path.exists(f):+ try:+ return np.load(f)+ except Exception:+ pass+ p = self.probs([audio.load(path, mono=True, start=start, end=end)], reduce=reduce)[0]+ tmp = f'{f}.{os.getpid()}.tmp.npy'+ np.save(tmp, p)+ audio.replace_quietly(tmp, f)+ return p++ def top(self, p, k=10, min_p=0.0):+ idx = np.argsort(p)[::-1][:k]+ return [(self.labels[i], round(float(p[i]), 3)) for i in idx if p[i] >= min_p]++ def top_file(self, path, k=10, start=None, end=None):+ return self.top(self.probs_file(path, start, end), k)++ def prob_of(self, p, label):+ return float(p[self.index[label]])+++def get(model_id=None):+ global _inst+ with _lock:+ if _inst is None or (model_id and _inst.model_id != model_id):+ _inst = AST(model_id)+ return _instdiff --git a/earshot/models/clap.py b/earshot/models/clap.pynew file mode 100644index 0000000..aa53579--- /dev/null+++ b/earshot/models/clap.py@@ -0,0 +1,285 @@+"""CLAP (contrastive language-audio pretraining) on CPU: one embedding space for sounds and text.++ from earshot.models import clap+ c = clap.get()+ a = c.embed_file('whoosh.wav') # (512,) unit vector, cached on disk+ t = c.embed_text(['a fast air whoosh']) # (1, 512)+ cos = a @ t.T++Model: laion/larger_clap_general (Apache-2.0) by default. Its mel front end covers 50 Hz - 14 kHz at 48 kHz, so it is+deaf to "air" above 14 kHz (use earshot.features for bandwidth / lowpass checks). Inputs longer than 10 s are embedded+as deterministic 10 s windows (hop 5 s) and averaged; the model's own random crop is never used, so results are+reproducible. Short sounds are repeat-padded to 10 s like in CLAP's training (pad='repeatpad'); pad='zero' pads with+silence instead (better for judging a single hit's character in isolation).+"""+import hashlib+import os+import threading++import numpy as np++from .. import audio, config++WIN_S = 10.0+HOP_S = 5.0+_lock = threading.Lock()+_inst = None+++def _slug(model_id):+ return model_id.replace('/', '--')+++def _read_npz(path):+ """dict of arrays from an .npz, closing the file (an open NpzFile blocks os.replace on Windows)"""+ if not os.path.exists(path):+ return {}+ try:+ with np.load(path, allow_pickle=False) as z:+ return {k: z[k] for k in z.files}+ except Exception:+ return {}+++def offline_if_cached(*model_ids):+ """when every model is already in the cache, keep huggingface_hub / transformers fully offline (no update checks)"""+ ok = all(os.path.isdir(os.path.join(config.HF_CACHE, 'models--' + _slug(m))) for m in model_ids)+ if ok or config.OFFLINE:+ os.environ.setdefault('HF_HUB_OFFLINE', '1')+ os.environ.setdefault('TRANSFORMERS_OFFLINE', '1')+ return ok+++def _quiet_transformers():+ try:+ from transformers.utils import logging as tlog+ tlog.set_verbosity_error()+ tlog.disable_progress_bar()+ except Exception: # noqa: BLE001+ pass+++def _load_hf(cls, model_id):+ import warnings+ _quiet_transformers()+ kw = dict(cache_dir=config.HF_CACHE)+ with warnings.catch_warnings():+ warnings.filterwarnings('ignore', message='.*mel filter.*')+ warnings.filterwarnings('ignore', category=FutureWarning)+ try:+ return cls.from_pretrained(model_id, local_files_only=True, **kw)+ except Exception:+ if config.OFFLINE:+ raise+ os.environ.pop('HF_HUB_OFFLINE', None)+ os.environ.pop('TRANSFORMERS_OFFLINE', None)+ return cls.from_pretrained(model_id, **kw)+++def native_bf16(torch):+ """True when the CPU has native bf16 matrix instructions (AVX512-BF16 or AMX). Without them torch still runs+ bf16, but emulated: measured 16x slower than fp32 on an AVX2-only CPU"""+ try: # kernels capped below AVX512 (ATEN_CPU_CAPABILITY / ONEDNN_MAX_CPU_ISA) cannot use the bf16 instructions+ if torch.backends.cpu.get_cpu_capability() in ('DEFAULT', 'NO AVX', 'AVX', 'AVX2', 'ZVECTOR', 'VSX'):+ return False+ except Exception: # noqa: BLE001+ pass+ if os.environ.get('ONEDNN_MAX_CPU_ISA', '').upper() in ('SSE41', 'AVX', 'AVX2', 'AVX2_VNNI', 'AVX512_CORE',+ 'AVX512_CORE_VNNI'):+ return False+ cpu = getattr(torch, 'cpu', None)+ for name in ('_is_avx512_bf16_supported', '_is_amx_tile_supported'):+ try:+ if getattr(cpu, name)():+ return True+ except Exception: # noqa: BLE001 (older torch: no probe -> assume no native bf16)+ pass+ return False+++def _set_precision(model, torch):+ """bf16 weights when the CPU has native bf16 (AVX512-BF16 / AMX; on the Zen 4 dev laptop 1.4x faster than fp32,+ embeddings within cosine 0.9993 of it: eval/semantics/precision.json); fp32 otherwise, and always with+ EARSHOT_PRECISION=fp32"""+ if config.PRECISION != 'bf16' or not native_bf16(torch):+ return torch.float32+ try:+ model.to(torch.bfloat16)+ with torch.inference_mode():+ torch.ones(2, 2, dtype=torch.bfloat16) @ torch.ones(2, 2, dtype=torch.bfloat16)+ return torch.bfloat16+ except Exception:+ model.to(torch.float32)+ return torch.float32+++class Clap:+ def __init__(self, model_id=None):+ config.ensure_dirs()+ offline_if_cached(model_id or config.CLAP_MODEL)+ import torch+ from transformers import ClapModel, ClapProcessor+ torch.set_num_threads(config.threads())+ self.torch = torch+ self.model_id = model_id or config.CLAP_MODEL+ self.model = _load_hf(ClapModel, self.model_id).eval()+ self.proc = _load_hf(ClapProcessor, self.model_id)+ self.dtype = _set_precision(self.model, torch)+ self.precision = 'bf16' if self.dtype == torch.bfloat16 else 'fp32'+ self.dir = os.path.join(config.EMB_CACHE, 'clap', _slug(self.model_id) + '--' + self.precision)+ os.makedirs(self.dir, exist_ok=True)+ self._text = {}+ self._text_path = os.path.join(self.dir, 'text.npz')+ self._text = _read_npz(self._text_path)+ self._text_dirty = False+ try:+ self.logit_scale = float(self.model.logit_scale_a.exp().item())+ except Exception:+ self.logit_scale = 100.0++ # ------------------------------------------------------------------ text+ @staticmethod+ def _tkey(t):+ return hashlib.sha1(t.encode('utf-8')).hexdigest()[:24]++ def embed_text(self, texts, batch=64):+ """(n, 512) unit vectors; cached per model on disk"""+ if isinstance(texts, str):+ texts = [texts]+ keys = [self._tkey(t) for t in texts]+ todo = [(k, t) for k, t in dict(zip(keys, texts)).items() if k not in self._text]+ torch = self.torch+ for i in range(0, len(todo), batch):+ chunk = todo[i:i + batch]+ inp = self.proc(text=[t for _, t in chunk], return_tensors='pt', padding=True)+ with torch.inference_mode():+ e = self.model.get_text_features(**inp)+ e = getattr(e, 'pooler_output', e)+ e = e.float().numpy().astype(np.float32)+ e /= np.linalg.norm(e, axis=1, keepdims=True) + 1e-12+ for (k, _), v in zip(chunk, e):+ self._text[k] = v+ self._text_dirty = True+ if self._text_dirty:+ self._save_text()+ return np.stack([self._text[k] for k in keys]) if keys else np.zeros((0, 512), np.float32)++ def _save_text(self):+ """merge with what other processes saved meanwhile, then replace atomically"""+ import time+ with _lock:+ merged = _read_npz(self._text_path)+ merged.update(self._text)+ self._text = merged+ tmp = self._text_path + f'.{os.getpid()}.tmp.npz'+ np.savez(tmp, **merged)+ for attempt in range(5): # Windows: another process may hold the file for a moment+ try:+ os.replace(tmp, self._text_path)+ break+ except OSError:+ time.sleep(0.05 * (attempt + 1))+ else: # a cache write must never fail the call: keep the in-memory cache+ try:+ os.remove(tmp)+ except OSError:+ pass+ self._text_dirty = False++ # ------------------------------------------------------------------ audio+ def _prep(self, x, pad):+ x = np.asarray(x, np.float32)+ if x.ndim == 2:+ x = x.mean(axis=1)+ n = int(WIN_S * audio.SR)+ if len(x) < 480: # < 10 ms: pad so the front end has something to read+ x = np.pad(x, (0, 480 - len(x)))+ if len(x) < n and pad == 'zero':+ x = np.pad(x, (0, n - len(x)))+ return x[:n]++ def _forward(self, segs, pad, batch):+ torch = self.torch+ out = []+ for i in range(0, len(segs), batch):+ chunk = [self._prep(s, pad) for s in segs[i:i + batch]]+ inp = self.proc(audio=chunk, sampling_rate=audio.SR, return_tensors='pt')+ inp = {k: (v.to(self.dtype) if v.is_floating_point() else v) for k, v in inp.items()}+ with torch.inference_mode():+ e = self.model.get_audio_features(**inp)+ e = getattr(e, 'pooler_output', e)+ e = e.float().numpy().astype(np.float32)+ out.append(e / (np.linalg.norm(e, axis=1, keepdims=True) + 1e-12))+ return np.concatenate(out) if out else np.zeros((0, 512), np.float32)++ def embed_audio(self, arrays, pad='repeatpad', batch=8, per_window=False):+ """mono (or stereo) 48 kHz arrays -> (n, 512) unit vectors. Arrays longer than 10 s are split into 10 s+ windows (hop 5 s) and averaged; per_window=True returns a list of (k_i, 512) arrays instead."""+ if isinstance(arrays, np.ndarray) and arrays.ndim == 1:+ arrays = [arrays]+ segs, owner = [], []+ n = int(WIN_S * audio.SR)+ for j, x in enumerate(arrays):+ x = audio.to_mono(np.asarray(x, np.float32))+ if len(x) <= n:+ segs.append(x); owner.append(j)+ else:+ for _, w in audio.windows(x, WIN_S, HOP_S, pad_last=True):+ segs.append(w); owner.append(j)+ e = self._forward(segs, pad, batch)+ owner = np.array(owner)+ groups = [e[owner == j] for j in range(len(arrays))]+ if per_window:+ return groups+ m = np.stack([g.mean(axis=0) for g in groups])+ return m / (np.linalg.norm(m, axis=1, keepdims=True) + 1e-12)++ def embed_file(self, path, start=None, end=None, pad='repeatpad'):+ """(512,) for a file or a [start, end) segment of it; cached on disk"""+ k = hashlib.sha1(f'{audio.file_id(path)}|{start}|{end}|{pad}'.encode()).hexdigest()[:24]+ f = os.path.join(self.dir, 'a_' + k + '.npy')+ if os.path.exists(f):+ try:+ return np.load(f)+ except Exception:+ pass+ x = audio.load(path, mono=True, start=start, end=end)+ e = self.embed_audio([x], pad=pad)[0]+ tmp = f'{f}.{os.getpid()}.tmp.npy'+ np.save(tmp, e)+ audio.replace_quietly(tmp, f)+ return e++ def embed_files(self, paths, pad='repeatpad', batch=8):+ """(n, 512) for many files: cached ones are read, the rest are embedded in batches"""+ out = [None] * len(paths)+ todo = []+ for i, p in enumerate(paths):+ k = hashlib.sha1(f'{audio.file_id(p)}|None|None|{pad}'.encode()).hexdigest()[:24]+ f = os.path.join(self.dir, 'a_' + k + '.npy')+ if os.path.exists(f):+ try:+ out[i] = np.load(f)+ continue+ except Exception:+ pass+ todo.append((i, p, f))+ for s in range(0, len(todo), batch):+ chunk = todo[s:s + batch]+ xs = [audio.load(p, mono=True) for _, p, _ in chunk]+ es = self.embed_audio(xs, pad=pad, batch=batch)+ for (i, _, f), e in zip(chunk, es):+ out[i] = e+ tmp = f'{f}.{os.getpid()}.tmp.npy'+ np.save(tmp, e)+ audio.replace_quietly(tmp, f)+ return np.stack(out) if out else np.zeros((0, 512), np.float32)+++def get(model_id=None):+ """process-wide singleton (the MCP server keeps it warm between calls)"""+ global _inst+ with _lock:+ if _inst is None or (model_id and _inst.model_id != model_id):+ _inst = Clap(model_id)+ return _instdiff --git a/earshot/profile.py b/earshot/profile.pynew file mode 100644index 0000000..36f3d02--- /dev/null+++ b/earshot/profile.py@@ -0,0 +1,603 @@+"""Launch-film sound profile: what professional launch films do with sound, measured the same way for every film.++ earshot profile refs/ more_ref.mp4 -o my_profile.json [--styles styles.json] [--no-ast]++ from earshot.profile import film_metrics, profile, load_profile+ f = film_metrics('film.mp4') # one film: metrics + event lists (cached under EARSHOT_HOME/emb/launch)+ f = film_metrics('mix.wav', video='film.mp4')+ p = profile(['a.mp4', 'b.mp4'], styles=['UI walkthrough', 'kinetic type']) # median, p10-p90, min-max per metric+ load_profile() # the bundled profile of 7 professional launch films (data/launch_profile.json)++Per-film metrics (keys of METRICS; None when a film is too short or has no picture):+ loudness lufs_i (BS.1770 integrated), true_peak (dBTP, 4x oversampled), lra (LU), crest_lu (loudest 3 s minus+ integrated)+ arc arc (11 points: loudness at 0 %, 10 % .. 100 % of the film in LU vs integrated; shown, not scored),+ intro_lu / end_lu (energy mean of the first / last 10 % of the film relative to integrated), peak_pos+ (where the loudest 3 s sit, 0 = start, 1 = end), first_sound_s (first 100 ms block within 20 LU of the+ integrated level), max_gap_s (longest stretch between first and last sound where the 100 ms level stays+ 30 LU below integrated)+ ending end_silence_s (dead air after the last block within 30 LU of integrated), fade_s (last moment at full+ level -> end of sound; < 0.5 s = hard out), final_lu (loudest 0.4 s of the last 3 s of sound relative to+ integrated: is there a final hit?), tail_s (strongest onset of the last 4 s -> end of sound, when it is a+ real hit: how long the final sting rings out; None = no final hit)+ music music_frac (share of 5 s windows with AudioSet 'Music' >= 0.3: music under voice or SFX scores 0.4-0.7,+ a bare music track ~0.9, no music ~0.1), speech_frac (share with 'Speech' >= 0.5), tempo (BPM)+ density onset_rate (onsets / s, librosa spectral flux), hit_rate (salient hits / s: the 100 ms loudness jumps+ >= 6 dB within 80 ms to within 8 LU of the integrated level), stopdowns_min (dips >= 10 dB below the+ preceding 2 s for 0.15-1.5 s that snap back to full level within 0.3 s, per minute)+ sync cut_rate (visual events / s: hard cuts and big picture changes, see below),+ cuts_with_sound (share of visual events with an onset within +-100 ms), cut_hit_frac (salient hit+ within +-100 ms), cut_frame_sync (onset within +-1 frame), cuts_on_beat (within +-70 ms of the tracked+ beat). Each sync share also has a 'chance' value: what the same events shifted off the picture score.+ stereo side_to_mid_db (side vs mid energy: higher = wider; identical channels = a mono mix count as -40 dB),+ correlation (L/R)+ spectrum band_sub .. band_air (energy share per band in dB over the whole file: sub 20-60, low 60-250, lowmid 250-1k,+ mid 1-4k, presence 4-8k, air 8-16k Hz; > 16 kHz is left out: AAC web encodes cut it), brightness_hz+ (features.brightness_hz: median magnitude-weighted spectral centroid)+Files shorter than 10 s get no loudness-shape metrics (lra, crest_lu, intro_lu, end_lu, final_lu): on a short clip+they measure the clip, not a film. profile() skips files that are missing, have no audio, are shorter than 3 s or+silent (with a warning) and refuses an empty result.++Visual events are earshot.video.cuts' frame-difference detector (video.frame_diff + video.diff_events: 96x54 grey+frames, a peak >= 4x the median of the surrounding second or a change of >= 15 % mean grey level), which also catches+cuts on white-background motion graphics where ffmpeg's scene score finds almost nothing (eval/launch/RESULTS.md).+"""+import hashlib+import json+import os+import sys+import time++import numpy as np++from . import audio, config, features+from .cli import common+from .report import emit, error_line+from .video import diff_events, frame_diff++VERSION = 2+AUDIO_V = 'a3'+PROFILE_PATH = os.path.join(config.DATA_DIR, 'launch_profile.json')+AST_WIN = 5.0+MUSIC_P = 0.3+HR = 22050+HOP = 256+E2 = 0.002+MIN_S, SHORT_S = 3.0, 10.0+MONO_DB = -40.0+SHAPE = ('lra', 'crest_lu', 'intro_lu', 'end_lu', 'final_lu')+SHIFTS = (0.23, 0.31, 0.43, 0.57, 0.71, 0.87, 1.13, 1.37, -0.29, -0.41, -0.53, -0.67, -0.79, -0.97, -1.19, -1.43)++# key: (group, label, unit, decimals)+METRICS = {+ 'lufs_i': ('loudness', 'integrated loudness', 'LUFS', 1),+ 'true_peak': ('loudness', 'true peak', 'dBTP', 1),+ 'lra': ('loudness', 'loudness range', 'LU', 1),+ 'crest_lu': ('loudness', 'loudest 3 s above integrated', 'LU', 1),+ 'intro_lu': ('arc', 'opening level (first 10%)', 'LU', 1),+ 'end_lu': ('arc', 'closing level (last 10%)', 'LU', 1),+ 'peak_pos': ('arc', 'position of the loudest 3 s', '0-1', 2),+ 'first_sound_s': ('arc', 'time to first sound', 's', 2),+ 'max_gap_s': ('arc', 'longest silent gap inside', 's', 2),+ 'end_silence_s': ('ending', 'dead air at the end', 's', 2),+ 'fade_s': ('ending', 'final fade / ring-out length', 's', 2),+ 'final_lu': ('ending', 'final moment vs integrated', 'LU', 1),+ 'tail_s': ('ending', 'final hit ring-out', 's', 2),+ 'music_frac': ('music', 'music presence', 'share', 2),+ 'speech_frac': ('music', 'voice / speech presence', 'share', 2),+ 'tempo': ('music', 'tempo', 'BPM', 0),+ 'onset_rate': ('density', 'onset density', '/s', 2),+ 'hit_rate': ('density', 'salient hit density', '/s', 2),+ 'stopdowns_min': ('density', 'stop-downs', '/min', 1),+ 'cut_rate': ('sync', 'visual events (cuts, big changes)', '/s', 2),+ 'cuts_with_sound': ('sync', 'onset on visual events +-100ms', 'share', 2),+ 'cut_hit_frac': ('sync', 'visual events with a salient hit +-100 ms', 'share', 2),+ 'cut_frame_sync': ('sync', 'visual events with an onset +-1 frame', 'share', 2),+ 'cuts_on_beat': ('sync', 'visual events on the beat grid +-70 ms', 'share', 2),+ 'band_sub': ('spectrum', 'sub 20-60 Hz share', 'dB', 1),+ 'band_low': ('spectrum', 'low 60-250 Hz share', 'dB', 1),+ 'band_lowmid': ('spectrum', 'low-mid 250-1k share', 'dB', 1),+ 'band_mid': ('spectrum', 'mid 1-4 kHz share', 'dB', 1),+ 'band_presence': ('spectrum', 'presence 4-8 kHz share', 'dB', 1),+ 'band_air': ('spectrum', 'air 8-16 kHz share', 'dB', 1),+ 'brightness_hz': ('spectrum', 'brightness (median spectral centroid)', 'Hz', 0),+ 'side_to_mid_db': ('stereo', 'stereo width (side vs mid)', 'dB', 1),+ 'correlation': ('stereo', 'L/R correlation', 'r', 2),+}+++class InputError(ValueError):+ """a file that cannot be measured: missing, a folder, no audio / picture stream"""+++def media_problem(path, need='audio'):+ """None when `path` is a media file with an audio (need='audio') or picture (need='video') stream, else why not"""+ if not os.path.exists(path):+ return f'file not found: {path}'+ if os.path.isdir(path):+ return f'{path} is a folder, not a media file'+ pr = audio.probe(path)+ if pr.get('probe_error') or not (pr.get('codec') or pr.get('has_video')):+ return f'{path} is not a readable audio or video file'+ if need == 'audio' and not pr.get('codec'):+ return f'{path} has no audio track'+ if need == 'video' and not pr.get('has_video'):+ return f'{path} has no picture (no video stream)'+ return None+++def unscorable(path):+ """(duration, reason) - reason is None unless the sound is shorter than MIN_S or silent (cheap: no librosa)"""+ if os.path.exists(cache_paths(path)['audio']):+ m = _audio_pass(path)['meas']+ dur, lufs = float(m['duration']), m.get('lufs', -70.0)+ else:+ x = audio.load(path)+ dur = len(x) / audio.SR+ lufs = features.integrated_lufs(x) if dur >= MIN_S else -70.0+ if dur < MIN_S:+ return dur, f'is shorter than {MIN_S:.0f} s ({dur:.2f} s)'+ if lufs is None or lufs <= -69.5:+ return dur, 'is silent (nothing above -70 LUFS)'+ return dur, None+++def _cache_file(kind, *ids):+ d = os.path.join(config.EMB_CACHE, 'launch')+ os.makedirs(d, exist_ok=True)+ k = hashlib.sha1('|'.join(map(str, ids)).encode()).hexdigest()[:20]+ return os.path.join(d, f'{kind}_{k}')+++def cache_paths(path):+ """this module's cache files for one media file (audio pass, AudioSet pass); the audio key follows the core decode+ version, so a decoder change re-measures; the picture pass is cached by earshot.video.frame_diff"""+ return {'audio': _cache_file('audio', audio._key(path, audio.SR, 2), AUDIO_V) + '.npz',+ 'ast': _cache_file('ast', audio.file_id(path), AST_WIN, 1) + '.npz'}+++def _energy_mean(L):+ L = np.asarray(L, np.float64)+ return float(10 * np.log10(np.mean(10 ** (L / 10)))) if len(L) else None+++def _audio_pass(path):+ """light measurements + onset envelope, onsets, beats and loudness curves (cached)"""+ f = cache_paths(path)['audio']+ if os.path.exists(f):+ try:+ z = np.load(f, allow_pickle=False)+ r = {k: z[k] for k in z.files if k != 'meas'}+ r['meas'] = json.loads(str(z['meas']))+ return r+ except Exception:+ pass+ import librosa+ x = audio.load(path)+ meas = features.measure(x, light=True)+ y = audio.resample(x.mean(axis=1), audio.SR, HR)+ env = librosa.onset.onset_strength(y=y, sr=HR, hop_length=HOP)+ on = librosa.onset.onset_detect(onset_envelope=env, sr=HR, hop_length=HOP, units='time')+ tempo, beats = librosa.beat.beat_track(onset_envelope=env, sr=HR, hop_length=HOP, units='time')+ tf, Lf = features.loudness_curve(x, 0.1, 0.02)+ tm, Lm = features.loudness_curve(x, 0.4, 0.1)+ ts, Ls = features.loudness_curve(x, 3.0, 0.1)+ e2 = 20 * np.log10(features.frame_rms(x.mean(axis=1), int(E2 * audio.SR)) + 1e-9).astype(np.float32)+ r = dict(env=env.astype(np.float32), e2=e2, onsets=attack_times(on, e2), beats=np.asarray(beats, np.float64),+ tempo=np.atleast_1d(np.asarray(tempo, np.float64))[:1], tf=tf, Lf=Lf, tm=tm, Lm=Lm, ts=ts, Ls=Ls)+ tmp = f'{f}.{os.getpid()}.tmp.npz'+ np.savez(tmp, meas=json.dumps(meas), **r)+ audio.replace_quietly(tmp, f)+ r['meas'] = meas+ return r+++def attack_times(times, e2, h=E2):+ """move onset / hit times to the attack: the steepest rise of the 2 ms level within 60 ms before .. 20 ms after+ (spectral-flux peaks lag the attack by 10-30 ms; an editor aligns on the attack)"""+ out = []+ for t in np.asarray(times, np.float64):+ a, b = max(1, int((t - 0.06) / h)), min(len(e2) - 1, int((t + 0.02) / h) + 1)+ if b - a < 2:+ out.append(float(t))+ continue+ d = e2[a:b + 1] - e2[a - 1:b]+ out.append(round((a + int(np.argmax(d))) * h, 4))+ return np.array(out, np.float64)+++def _ast_pass(path):+ """AudioSet probabilities per 5 s window (cached): (window starts, (n, 527) float16)"""+ f = cache_paths(path)['ast']+ if os.path.exists(f):+ try:+ z = np.load(f, allow_pickle=False)+ return z['t'], z['p'].astype(np.float32), [str(v) for v in z['labels']]+ except Exception:+ pass+ from .models import ast+ A = ast.get()+ x = audio.load(path, mono=True)+ w = int(AST_WIN * audio.SR)+ starts = [s for s in range(0, len(x), w) if len(x) - s >= audio.SR or s == 0]+ p = A.probs([x[s:s + w] for s in starts])+ t = np.array(starts, np.float64) / audio.SR+ tmp = f'{f}.{os.getpid()}.tmp.npz'+ np.savez(tmp, t=t, p=p.astype(np.float16), labels=np.array(A.labels))+ audio.replace_quietly(tmp, f)+ return t, p, A.labels+++def _windows_min_max(L, a, b):+ from numpy.lib.stride_tricks import sliding_window_view as sw+ n = len(L)+ pre = sw(np.concatenate([np.full(a, L[0]), L]), a).min(axis=1)[:n]+ post = sw(np.concatenate([L, np.full(b - 1, L[-1])]), b).max(axis=1)[:n]+ return pre, post+++def salient_hits(tf, Lf, lufs, env, e2=None, jump=6.0, rel=-8.0):+ """times of salient hits: the 100 ms loudness jumps >= jump dB within 80 ms (vs the quietest block of the 200 ms+ before) and lands within `rel` LU of the integrated level; one hit per run of such blocks, timed at the+ strongest onset-envelope frame of the run, moved to its attack when the 2 ms level `e2` is given"""+ if len(Lf) < 20:+ return np.zeros(0)+ h = float(tf[1] - tf[0])+ pre, post = _windows_min_max(Lf, int(round(0.2 / h)), int(round(0.08 / h)))+ idx = np.where((post - pre >= jump) & (post >= lufs + rel))[0]+ if not len(idx):+ return np.zeros(0)+ te = np.arange(len(env)) * HOP / HR+ out = []+ for run in np.split(idx, np.where(np.diff(idx) > 2)[0] + 1):+ sel = (te >= tf[run[0]] - 0.05) & (te <= tf[run[-1]] + 0.1)+ t = float(te[sel][int(np.argmax(env[sel]))]) if sel.any() else float(tf[run[0]]) + 0.03+ if not out or t - out[-1] > 0.1:+ out.append(t)+ return attack_times(out, e2) if e2 is not None else np.array(out)+++def stopdowns(tf, Lf, drop=10.0, back=3.0, min_s=0.15, max_s=1.5):+ """[(start, length, depth_db)] of short dips before a hit: >= drop dB below the median of the preceding 2 s for+ min_s..max_s, back to within `back` dB of that median within 0.3 s"""+ if len(Lf) < 3:+ return []+ h = float(tf[1] - tf[0])+ w = int(round(2.0 / h))+ if len(Lf) <= w:+ return []+ from numpy.lib.stride_tricks import sliding_window_view as sw+ ref = np.concatenate([np.full(w, np.nan), np.median(sw(Lf, w), axis=1)[:len(Lf) - w]])+ out, i, n = [], w, len(Lf)+ while i < n:+ r = ref[i]+ if Lf[i] < r - drop:+ j = i+ while j < n and Lf[j] < r - drop:+ j += 1+ ln = (j - i) * h+ if min_s <= ln <= max_s and j < n and Lf[j: j + int(round(0.3 / h)) + 1].max() >= r - back:+ out.append((round(float(tf[i]) - 0.05, 3), round(ln, 3), round(float(r - Lf[i:j].min()), 1)))+ i = j + int(round(0.5 / h))+ else:+ i += 1+ return out+++def near_frac(events, targets, tol):+ """share of events with a target within +-tol seconds"""+ events, targets = np.asarray(events, float), np.sort(np.asarray(targets, float))+ if not len(events):+ return None+ if not len(targets):+ return 0.0+ i = np.searchsorted(targets, events)+ d = np.minimum(np.abs(targets[np.clip(i, 0, len(targets) - 1)] - events),+ np.abs(targets[np.clip(i - 1, 0, len(targets) - 1)] - events))+ return float(np.mean(d <= tol))+++def chance_frac(events, targets, tol, dur):+ """the same share for the events shifted off the picture (circularly, by fixed offsets): what chance scores"""+ events = np.asarray(events, float)+ if not len(events) or dur <= 0:+ return None+ return float(np.mean([near_frac((events + s) % dur, targets, tol) for s in SHIFTS]))+++def _edge(e2, t, first=True, h=E2):+ """refine the start (first=True) or end of sound found with a 100 ms block centred at t: the first / last 2 ms+ frame of that block within 20 dB of the loudest 2 ms of the next / previous 150 ms"""+ a, b = max(0, int((t - 0.05) / h)), min(len(e2), int((t + 0.05) / h) + 1)+ if b <= a:+ return max(0.0, t)+ ref = e2[a:min(len(e2), a + int(0.15 / h))].max() if first else e2[max(0, b - int(0.15 / h)):b].max()+ ok = np.where(e2[a:b] >= ref - 20)[0]+ if not len(ok):+ return max(0.0, t)+ return round((a + (ok[0] if first else ok[-1] + 1)) * h, 3)+++def _arc(tm, Lm, I, dur):+ """loudness arc: energy-mean momentary loudness around 0 %, 10 % .. 100 % of the duration, in LU vs integrated"""+ w = max(1.5, 0.05 * dur)+ out = []+ for k in range(11):+ sel = np.abs(tm - k / 10 * dur) <= w+ out.append(round(_energy_mean(Lm[sel]) - I, 1) if sel.any() else None)+ return out+++def _longest_run(mask):+ best = cur = 0+ for b in mask:+ cur = cur + 1 if b else 0+ best = max(best, cur)+ return best+++def _final_hit_tail(env, onsets, t_end, look=4.0):+ """seconds from the strongest onset of the last `look` s of sound to the end of sound, if that onset is a real+ hit (onset strength in the film's top quarter); None when the film just stops or fades without one"""+ if not len(onsets):+ return None+ st = env[np.clip(np.round(onsets * HR / HOP).astype(int), 0, len(env) - 1)]+ sel = np.where((onsets >= t_end - look) & (onsets < t_end))[0]+ if not len(sel):+ return None+ j = sel[int(np.argmax(st[sel]))]+ return t_end - float(onsets[j]) if st[j] >= np.percentile(st, 75) else None+++def _r(v, nd):+ return None if v is None or (isinstance(v, float) and not np.isfinite(v)) else round(float(v), nd)+++def film_metrics(path, video=None, use_ast=True):+ """launch metrics + event lists for one soundtrack or film (picture from `video`, else from `path` if it has one)"""+ t0 = time.time()+ a = _audio_pass(path)+ timing = {'audio': round(time.time() - t0, 1)}+ m = a['meas']+ dur = float(m['duration'])+ I = m.get('lufs', -70.0)+ tf, Lf, tm, Lm, ts, Ls = a['tf'], a['Lf'], a['tm'], a['Lm'], a['ts'], a['Ls']+ onsets, beats = a['onsets'], a['beats']+ hits = salient_hits(tf, Lf, I, a['env'], a['e2'])+ sd = stopdowns(tf, Lf)+ v = {k: None for k in METRICS}+ mono = m.get('side_to_mid_db') is None and I > -69.5+ v.update(lufs_i=m.get('lufs'), true_peak=m.get('true_peak_dbtp'), lra=m.get('lra_lu'), correlation=m.get('correlation'),+ side_to_mid_db=MONO_DB if mono else m.get('side_to_mid_db'), brightness_hz=m.get('brightness_hz'))+ for b in ('sub', 'low', 'lowmid', 'mid', 'presence', 'air'):+ v['band_' + b] = (m.get('bands_db') or {}).get(b)+ if m.get('short_term_max_lufs') is not None and m.get('lufs') is not None:+ v['crest_lu'] = m['short_term_max_lufs'] - m['lufs']+ if len(Lm) > 2:+ v['intro_lu'] = _energy_mean(Lm[tm <= max(0.1 * dur, 0.5)]) - I+ v['end_lu'] = _energy_mean(Lm[tm >= dur - max(0.1 * dur, 0.5)]) - I+ if len(Ls) > 1 and dur >= 6:+ v['peak_pos'] = float(ts[int(np.argmax(Ls))]) / dur+ arc = _arc(tm, Lm, I, dur) if len(Lm) > 2 else None+ loud = np.where(Lf > I - 20)[0]+ audible = np.where(Lf > I - 30)[0]+ t_end = dur+ if len(loud):+ v['first_sound_s'] = _edge(a['e2'], float(tf[loud[0]]), first=True)+ if len(audible):+ t_end = min(dur, _edge(a['e2'], float(tf[audible[-1]]), first=False))+ v['end_silence_s'] = max(0.0, dur - t_end)+ full = np.where((Lm >= I - 6) & (tm <= t_end))[0]+ if len(full):+ v['fade_s'] = max(0.0, t_end - float(tm[full[-1]]) - 0.2)+ inside = (tf >= (v['first_sound_s'] or 0.0)) & (tf <= t_end)+ v['max_gap_s'] = _longest_run(Lf[inside] < I - 30) * float(tf[1] - tf[0]) if inside.sum() > 1 else 0.0+ last = (tm >= t_end - 3.0) & (tm <= t_end)+ if last.any():+ v['final_lu'] = float(Lm[last].max()) - I+ v['tail_s'] = _final_hit_tail(a['env'], onsets, t_end)+ v['onset_rate'] = len(onsets) / dur if dur > 0 else None+ v['hit_rate'] = len(hits) / dur if dur > 0 else None+ v['stopdowns_min'] = len(sd) / dur * 60 if dur >= 6 else None+ v['tempo'] = float(a['tempo'][0]) if len(beats) > 4 else None+ if dur < SHORT_S:+ v.update({k: None for k in SHAPE})+ music = None+ if use_ast:+ t1 = time.time()+ tw, P, labels = _ast_pass(path)+ timing['audioset'] = round(time.time() - t1, 1)+ ix = {l: i for i, l in enumerate(labels)}+ music = [(round(float(t), 2), round(float(P[i, ix['Music']]), 2), round(float(P[i, ix['Speech']]), 2))+ for i, t in enumerate(tw)]+ v['music_frac'] = float(np.mean(P[:, ix['Music']] >= MUSIC_P))+ v['speech_frac'] = float(np.mean(P[:, ix['Speech']] >= 0.5))+ pr = audio.probe(path)+ vid = video or (path if pr.get('has_video') else None)+ ev_t, ev_d, chance, fps = [], [], {}, None+ if vid:+ t1 = time.time()+ fps, d = frame_diff(vid)+ timing['picture'] = round(time.time() - t1, 1)+ ev = diff_events(d, fps)+ ev_t = [round(float(i / fps), 3) for i in ev if i / fps < dur]+ ev_d = [round(float(d[i]), 3) for i in ev if i / fps < dur]+ v['cut_rate'] = len(ev_t) / dur+ if len(ev_t) >= 3:+ tol_f = 1.0 / fps + 0.006+ for key, targets, tol in (('cuts_with_sound', onsets, 0.1), ('cut_hit_frac', hits, 0.1),+ ('cut_frame_sync', onsets, tol_f), ('cuts_on_beat', beats, 0.07)):+ v[key] = near_frac(ev_t, targets, tol)+ chance[key] = _r(chance_frac(ev_t, targets, tol, dur), 2)+ vals = {k: _r(x, METRICS[k][3] if METRICS[k][3] else 0) for k, x in v.items()}+ for k in ('tempo', 'brightness_hz'):+ if vals[k] is not None:+ vals[k] = int(vals[k])+ res = {'path': path, 'video': vid, 'duration': round(dur, 3), 'fps': fps, 'codec': pr.get('codec'),+ 'lossy': bool(pr.get('lossy')), 'mono': mono, 'short': dur < SHORT_S, 'metrics': vals, 'chance': chance, 'arc': arc,+ 'events': {'hits': [round(float(t), 3) for t in hits], 'visual': ev_t, 'visual_strength': ev_d,+ 'stopdowns': sd, 'beats': [round(float(b), 3) for b in beats], 'music_windows': music,+ 'onsets_n': len(onsets)},+ 'seconds': round(time.time() - t0, 1), 'timing': timing}+ res['summary'] = _film_summary(res)+ return res+++def _film_summary(r):+ m = r['metrics']+ s = f"{os.path.basename(r['path'])}: {r['duration']:.1f} s, {m['lufs_i']} LUFS, true peak {m['true_peak']} dBTP"+ if m['lra'] is not None:+ s += f", LRA {m['lra']} LU"+ if m['music_frac'] is not None:+ s += f", music in {100 * m['music_frac']:.0f}% of 5 s windows"+ s += f", {m['onset_rate']} onsets/s, {m['hit_rate']} salient hits/s"+ if m['cuts_with_sound'] is not None:+ s += f", {len(r['events']['visual'])} visual events ({100 * m['cuts_with_sound']:.0f}% with an onset +-100 ms)"+ return s + '.'+++def _stats(vals):+ x = np.array([v for v in vals if v is not None], np.float64)+ if not len(x):+ return {'n': 0}+ q = lambda p: round(float(np.percentile(x, p)), 3)+ return {'n': int(len(x)), 'median': q(50), 'p10': q(10), 'p90': q(90), 'min': round(float(x.min()), 3),+ 'max': round(float(x.max()), 3), 'values': [None if v is None else round(float(v), 3) for v in vals]}+++def aggregate(films):+ """profile dict from a list of film_metrics() results (ids/styles already set on each)"""+ stats = {k: _stats([f['metrics'].get(k) for f in films]) for k in METRICS}+ chance = {k: _stats([f.get('chance', {}).get(k) for f in films])+ for k in ('cuts_with_sound', 'cut_hit_frac', 'cut_frame_sync', 'cuts_on_beat')}+ arcs = [f['arc'] for f in films if f.get('arc')]+ arc = {q: [None if all(a[i] is None for a in arcs) else round(float(np.percentile(+ [a[i] for a in arcs if a[i] is not None], pq)), 1) for i in range(11)] for q, pq in+ (('median', 50), ('p10', 10), ('p90', 90))} if arcs else None+ return {'version': VERSION, 'n': len(films), 'metrics': stats, 'chance': chance, 'arc': arc,+ 'films': [{'id': f['id'], 'style': f.get('style', ''), 'duration': f['duration'], 'has_video': bool(f['video']),+ 'metrics': f['metrics'], 'chance': f.get('chance', {}), 'arc': f.get('arc')} for f in films]}+++def _warn(msg):+ print(f' warning: {msg}', file=sys.stderr, flush=True)+++def profile(paths, styles=None, use_ast=True, out=None):+ """build a reference profile from reference films / soundtracks (numbers only: no media, no file names); files+ that are missing, have no audio, are shorter than 3 s or silent are skipped with a warning"""+ paths = [paths] if isinstance(paths, str) else list(paths)+ skipped = [f'{p}: not found (or no media files in it)' for p in paths if not audio.list_media([p])]+ for w in skipped:+ _warn(f'skipped {w}')+ files = audio.list_media(paths)+ films = []+ for i, p in enumerate(files):+ try:+ why = media_problem(p)+ if not why:+ r = unscorable(p)[1]+ why = r and f'{p} {r}'+ if not why:+ f = film_metrics(p, use_ast=use_ast)+ except audio.AudioError as e:+ why = f'{p}: cannot decode ({str(e)[:160]})'+ if why:+ skipped.append(why)+ _warn(f'skipped {why}')+ continue+ f['id'] = f'ref_{len(films) + 1}'+ f['style'] = (styles.get(os.path.basename(p), '') if isinstance(styles, dict) else+ styles[i] if styles and i < len(styles) else '')+ films.append(f)+ print(f" {f['id']}: {f['duration']:.1f} s analysed in {f['seconds']:.1f} s", file=sys.stderr, flush=True)+ if not films:+ raise InputError('no usable reference films: ' + ('; '.join(skipped) if skipped else 'no media files given'))+ prof = aggregate(films)+ prof['warnings'] = [f'{len(skipped)} file(s) skipped (missing, no audio, under {MIN_S:.0f} s or silent)'] if skipped else []+ if len(films) < 3:+ prof['warnings'].append(f'only {len(films)} reference film(s): the ranges are the min-max of {len(films)} '+ 'value(s) plus a small tolerance, far too narrow to judge by; use 5 or more references')+ _warn(prof['warnings'][-1])+ prof['summary'] = profile_summary(prof) + ''.join(f' Warning: {w}.' for w in prof['warnings'])+ if out:+ os.makedirs(os.path.dirname(os.path.abspath(out)), exist_ok=True)+ with open(out, 'w', encoding='utf-8') as fh:+ json.dump(prof, fh, indent=1)+ return prof+++def profile_summary(p):+ m = p['metrics']+ med = lambda k: fmt_value(k, (m.get(k) or {}).get('median')) # n.a. when no film measured it (LRA needs >= 10 s)+ li = m.get('lufs_i') or {}+ s = (f"profile of {p['n']} reference films (medians): integrated {med('lufs_i')} (p10-p90 "+ f"{fmt_value('lufs_i', li.get('p10'))} .. {fmt_value('lufs_i', li.get('p90'))}), LRA {med('lra')}, "+ f"onsets {med('onset_rate')}, salient hits {med('hit_rate')}")+ if m['music_frac'].get('n'):+ s += f", music in {med('music_frac')} of 5 s windows"+ if m['cut_rate'].get('n'):+ s += f", visual events {med('cut_rate')}"+ return s + '.'+++def load_profile(path=None):+ with open(path or PROFILE_PATH, encoding='utf-8') as fh:+ return json.load(fh)+++def fmt_value(k, v):+ if v is None:+ return 'n.a.'+ unit, nd = METRICS[k][2], METRICS[k][3]+ if unit == 'share':+ return f'{100 * v:.0f}%'+ return f'{v:.{nd}f} {unit}' if unit not in ('0-1', 'r') else f'{v:.{nd}f}'+++def profile_text(p):+ lines = [p['summary'], '']+ for f in p['films']:+ lines.append(f" {f['id']} ({f['duration']:.0f} s): {f['style'] or '-'}")+ if p.get('arc'):+ lines += ['', 'loudness arc, median (LU vs integrated at 0%, 10% .. 100%): ' ++ ' '.join('-' if x is None else f'{x:+.1f}' for x in p['arc']['median'])]+ lines += ['', f"{'metric':40s} {'key':15s} {'median':>11s} {'p10 .. p90':24s} min .. max"]+ for k, st in p['metrics'].items():+ if st.get('n'):+ lines.append(f"{METRICS[k][1][:40]:40s} {k:15s} {fmt_value(k, st['median']):>11s} "+ f"{fmt_value(k, st['p10']) + ' .. ' + fmt_value(k, st['p90']):24s} "+ f"{fmt_value(k, st['min'])} .. {fmt_value(k, st['max'])}")+ return '\n'.join(lines)+++def _cmd(args):+ styles = None+ try:+ if args.styles:+ if not os.path.isfile(args.styles):+ raise InputError(f'styles file not found: {args.styles}')+ with open(args.styles, encoding='utf-8') as fh:+ styles = json.load(fh)+ p = profile(args.refs, styles=styles, use_ast=not args.no_ast)+ except (InputError, audio.AudioError, json.JSONDecodeError) as e:+ error_line('profile', e)+ return 1+ emit(p, args, text=profile_text(p))+++def add_cli(sub):+ p = sub.add_parser('profile', help='build a launch-sound reference profile from your own reference films',+ description='Measure reference launch films / soundtracks (loudness, arc, ending, music and '+ 'voice presence, onset / hit density, picture sync, spectrum, stereo) and write their median, '+ 'p10-p90 and min-max as a profile for `earshot critique --profile`.')+ p.add_argument('refs', nargs='+', help='reference videos / soundtracks or folders')+ p.add_argument('-o', dest='out', help='write the profile JSON here (same as --out)')+ p.add_argument('--styles', help='JSON: list of style descriptions in file order, or {basename: description}')+ p.add_argument('--no-ast', action='store_true', help='skip the AudioSet pass (no music / speech presence; faster)')+ common(p)+ p.set_defaults(func=_cmd)+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/realness.py b/earshot/realness.pynew file mode 100644index 0000000..5989d51--- /dev/null+++ b/earshot/realness.py@@ -0,0 +1,612 @@+"""Is this a real recording / professionally produced audio, or a cheap code-synthesized sound?++ earshot realness hit.wav # p_real, verdict, the evidence behind it+ earshot realness soundtrack.wav # files over 6 s also get per-4 s-window verdicts (segments)+ earshot realness film.mp4 --start 20 --end 32 # one part of a file (segment times stay file times)+ earshot realness assets/ more.ogg # several files / folders: one table row per file+ from earshot.realness import realness+ realness('pop.ogg') # {'p_real': 0.96, 'verdict': 'real or pro-produced', 'why': ..., 'evidence': [...], 'summary': ...}++What it judges: NOT acoustic vs electronic. Real recordings AND professionally produced audio (electronic music, pro+sound design such as risers and sub drops in launch films) count as real; what it flags is amateur procedural audio+written in code: oscillators, filtered random noise, simple envelopes, algorithmic reverb, beeps.++p_real is a probability from a small transparent model (earshot/data/realness_model.json, plain JSON, no pickle):+standardised logistic heads over two evidence families, fused by a logistic layer fitted on held-out head outputs:+ clap the CLAP embedding of the clip (what it sounds like)+ signal measurements a synthesizer leaves behind: no noise floor, energy in a few pure spectral lines, dead-steady+ pitch, mathematically smooth envelopes, Gaussian (non-crackly) noise ... and exact digital silence, which is+ weak evidence only (edited library recordings have it too) and is never reported as a verdict reason+The fitted fusion gives the signal head weight 0 (it added no held-out accuracy), so the CLAP head decides and the+measurements are evidence in words (each with the share of training clips past that value that were of that side).+One rule overrides the model: a pure, dead-steady tone (one spectral line in every frame, crest <= 4.5 dB, no pitch+drift: a test tone or bare oscillator, outside the training data) gets p_real <= 0.03, and its evidence says so.+ast=True (--ast, slower: one AudioSet pass per clip) adds the AudioSet synth classes (Sine wave, Beep, bleep ...).+Verdict: 'real or pro-produced' (p_real >= hi), 'likely code-synthesized' (p_real <= lo), else 'uncertain'; hi / lo hold+~92% precision on held-out clips. A long file's p_real is the mean of its windows and its verdict uses file-level+thresholds read from held-out long files (few synthesized ones: weak calibration), so the same p_real can get a+different verdict as a short clip (lo 0.17) and as a long file (lo 0.40): every summary names the thresholds it used.+For a multi-take file, judge the trim you will use (--start / --end): the mean over takes and silence is not it. The windows find stretches made+entirely of code synthesis; a code-made layer under a louder real mix is usually missed, so judge stems and source files.+It hears how a sound sounds, not where it came from: clean professionally synthesized tones (UI sounds, chimes) also+read as code-synthesized. Sources quieter than -30 dBFS RMS are raised to -30 dBFS RMS first (no level cliff).+Training data, held-out accuracy and what fools it: eval/realness/RESULTS.md.+"""+import hashlib+import json+import math+import os+import sys++import numpy as np++from . import audio, config+from .cli import common+from .report import emit, fmt_t++MODEL_PATH = os.path.join(config.DATA_DIR, 'realness_model.json')+CACHE_V = 6+LONG_S, WIN_S = 6.0, 4.0+SILENT_DB = -60.0+LIFT_BELOW_DB, QUIET_MIN_PEAK_DB = -30.0, -80.0+VERDICTS = ('real or pro-produced', 'uncertain', 'likely code-synthesized', 'silent', 'error')+SR32 = 32000+REAL_PROMPTS = ['a real recording made with a microphone', 'a field recording of a real place',+ 'a real foley recording of an object', 'a professionally produced music track',+ 'a professional cinematic sound effect from a sound library', 'a natural acoustic sound']+SYNTH_PROMPTS = ['a cheap synthesized beep', 'a pure sine wave test tone',+ 'a simple electronic synthesizer tone made by code', 'an 8-bit video game sound effect',+ 'filtered white noise', 'a basic drum machine loop']+AST_SYNTH = ['Synthesizer', 'Sine wave', 'Beep, bleep', 'Electronic tuner', 'Chirp tone', 'Dial tone', 'Pink noise',+ 'White noise', 'Static', 'Buzz', 'Hum', 'Theremin', 'Video game music', 'Ringtone', 'Sound effect']+_model = None+++# ---------------------------------------------------------------------------------------------- signal measurements+def _db(v):+ return 10 * math.log10(max(float(v), 1e-30))+++def _frames(y, n, hop=None):+ hop = hop or n+ k = 1 + (len(y) - n) // hop if len(y) >= n else 0+ if k <= 0:+ return np.pad(y, (0, n - len(y)))[None, :]+ idx = np.arange(n)[None, :] + hop * np.arange(k)[:, None]+ return y[idx]+++def audible(x):+ """RMS above SILENT_DB: quieter clips are not judged (and were not used in training)"""+ return len(x) > 0 and audio.db(math.sqrt(float(np.mean(np.square(x, dtype=np.float64))))) > SILENT_DB+++def lift_quiet(x):+ """a whole sound quieter than LIFT_BELOW_DB RMS that still has content (peak over -80 dBFS, e.g. a raw orchestral+ sample) is raised to LIFT_BELOW_DB RMS (peak at most -1 dBFS): very quiet input bends the CLAP features, and a+ gain that grows smoothly as the level drops has no cliff. Returns (x, gain_db, rms_db, peak_db)."""+ if not len(x):+ return x, 0.0, -200.0, -200.0+ rdb = audio.db(math.sqrt(float(np.mean(np.square(x, dtype=np.float64)))))+ pdb = audio.db(float(np.abs(x).max()))+ g = min(LIFT_BELOW_DB - rdb, -1.0 - pdb)+ if pdb < QUIET_MIN_PEAK_DB or g <= 0:+ return x, 0.0, rdb, pdb+ return (x * 10 ** (g / 20)).astype(np.float32), g, rdb, pdb+++def signal_features(x):+ """measurements of a mono 48 kHz clip that separate synthesized from recorded audio (None when silent).+ Everything but silence_frac runs on a 32 kHz copy normalised to peak 1, so the source sample rate, codec cut-off+ above 16 kHz and level do not matter."""+ from scipy import signal as sg+ m = np.asarray(audio.to_mono(np.asarray(x, np.float32)), np.float64)+ if len(m) < int(0.02 * audio.SR) or np.abs(m).max() < 1e-7:+ return None+ f = {'silence_frac': float(np.mean(np.abs(m) < 1e-6))}+ y = audio.resample(m.astype(np.float32), audio.SR, SR32).astype(np.float64)+ y /= np.abs(y).max() + 1e-12+ r = np.sqrt(np.mean(_frames(y, 640) ** 2, axis=1) + 1e-30)+ rdb = 20 * np.log10(r)+ f['floor_db'] = float(max(-140.0, np.percentile(rdb, 10) - rdb.max()))+ fr, P = sg.welch(y, SR32, nperseg=min(2048, len(y)))+ band = (fr >= 40) & (fr <= 15000)+ Pb = P[band] + 1e-30+ f['log_flatness'] = float(np.log10(np.exp(np.mean(np.log(Pb))) / np.mean(Pb)))+ f['peakiness_db'] = _db(Pb.max() / np.median(Pb))+ k = max(1, int(0.01 * len(Pb)))+ f['line_share'] = float(np.sort(Pb)[-k:].sum() / Pb.sum())+ lv = 10 * np.log10(Pb)+ f['bandwidth_khz'] = float(fr[band][np.where(lv >= lv.max() - 60)[0][-1]] / 1000)+ # short-time spectra of the active frames (within 50 dB of the loudest)+ win = np.hanning(1024)+ S = np.abs(np.fft.rfft(_frames(y, 1024, 256) * win, axis=1)) ** 2 + 1e-30+ fb = np.fft.rfftfreq(1024, 1 / SR32)+ sb = (fb >= 40) & (fb <= 15000)+ S = S[:, sb]+ e = 10 * np.log10(S.sum(axis=1))+ loud = e > e.max() - 30+ L = 10 * np.log10(S[loud])+ f['spec_floor_db'] = float(np.median(np.percentile(L, 10, axis=1) - L.max(axis=1)))+ act = e > e.max() - 50+ quiet = act & (e <= np.percentile(e[act], 25)) if act.sum() >= 8 else act+ qs = S[quiet].mean(axis=0) if quiet.any() else S.mean(axis=0)+ f['quiet_flat'] = float(np.log10(np.exp(np.mean(np.log(qs))) / np.mean(qs)))+ # pure-tone frames: the strongest peak (+-2 bins, the Hann main lobe) holds over half of the frame's energy+ i = np.clip(S.argmax(axis=1), 2, S.shape[1] - 3)+ rows = np.arange(len(S))+ share = sum(S[rows, i + o] for o in (-2, -1, 0, 1, 2)) / S.sum(axis=1)+ tonal = act & (share > 0.5)+ f['tonal_frames'] = float(tonal.sum() / max(1, act.sum()))+ f['pitch_wobble'] = float('nan')+ trip = tonal[:-2] & tonal[1:-1] & tonal[2:]+ if trip.sum() >= 4:+ a, b, c = (np.log(S[rows, i + o]) for o in (-1, 0, 1))+ cents = 1200 * np.log2(np.maximum(fb[sb][i] + 0.5 * (a - c) / (a - 2 * b + c - 1e-12) * (fb[1] - fb[0]), 20))+ d2 = np.abs(cents[2:] - 2 * cents[1:-1] + cents[:-2])[trip]+ f['pitch_wobble'] = float(np.log10(np.median(d2) + 0.01))+ # envelope smoothness: 5 ms log-energy residual around a 35 ms median trend, on active frames+ e5 = 10 * np.log10(np.mean(_frames(y, 160) ** 2, axis=1) + 1e-30)+ ok = e5 > e5.max() - 40+ if ok.sum() >= 9:+ tr = sg.medfilt(e5, 7)+ f['env_resid_db'] = float(np.log10(np.median(np.abs(e5 - tr)[ok]) + 0.01))+ else:+ f['env_resid_db'] = float('nan')+ # texture: kurtosis of the 2-8 kHz band after removing its 10 ms envelope (3 = Gaussian noise, 1.5 = a sine)+ z = sg.sosfilt(sg.butter(4, [2000, 8000], 'band', fs=SR32, output='sos'), y)+ env = np.sqrt(np.convolve(z ** 2, np.ones(320) / 320, 'same')) + 1e-9+ zz = (z / env)[np.abs(z) > 0][env[np.abs(z) > 0] > env.max() * 10 ** (-40 / 20)] if np.any(z) else z+ f['texture_kurt'] = (float(np.log(np.mean(zz ** 4) / (np.mean(zz ** 2) ** 2 + 1e-30) + 1e-9)) if len(zz) > 100+ else float('nan'))+ ya = y[np.abs(y) > 0]+ f['crest_db'] = float(20 * np.log10(1.0 / (np.sqrt(np.mean(ya ** 2)) + 1e-12))) if len(ya) else 0.0+ return f+++# ---------------------------------------------------------------------------------------------- model+def load_model(path=None):+ global _model+ if path:+ return json.load(open(path, encoding='utf-8'))+ if _model is None:+ if not os.path.exists(MODEL_PATH):+ raise FileNotFoundError(f'{MODEL_PATH} missing: train it with eval/realness/train.py')+ _model = json.load(open(MODEL_PATH, encoding='utf-8'))+ return _model+++def head_logit(h, v):+ v = np.asarray(v, np.float64)+ v = np.where(np.isfinite(v), v, np.asarray(h['mean']))+ if 'clip' in h:+ v = np.clip(v, h['clip'][0], h['clip'][1])+ z = (v - np.asarray(h['mean'])) / np.asarray(h['scale'])+ return float(z @ np.asarray(h['coef']) + h['intercept']), z+++def fuse(fusion, logits):+ s = fusion['intercept'] + sum(w * logits[k] for k, w in zip(fusion['heads'], fusion['coef']))+ return 1 / (1 + math.exp(-s)), s+++def thresholds_of(m, file=False):+ """verdict thresholds: per clip, or for the mean of a long file's windows (file-level held-out thresholds)"""+ return m.get('file_thresholds', m['thresholds']) if file else m['thresholds']+++def verdict_of(p, m, file=False):+ t = thresholds_of(m, file)+ return 'real or pro-produced' if p >= t['hi'] else 'likely code-synthesized' if p <= t['lo'] else 'uncertain'+++def _sig_vector(h, f):+ return [f.get(k, float('nan')) for k in h['inputs']]+++# ---------------------------------------------------------------------------------------------- evidence in words+def _pct(v):+ return f'{100 * v:.1f}%' if 0 < v < 0.095 else f'{100 * v:.0f}%'+++def _phrase(name, v, toward_real):+ """one measured reason in words; toward_real = the model reads this value as evidence of a real recording"""+ if name == 'floor_db':+ return (f'quietest parts {v:.0f} dB under the peak: cleaner than a microphone usually records' if v < -95 else+ f'noise floor {v:.0f} dB under the peak, like a microphone or room')+ if name == 'spec_floor_db':+ return (f'between its partials the spectrum falls {-v:.0f} dB under the peak (no measurable mic or room noise)'+ if not toward_real else f'a broadband noise floor {-v:.0f} dB under the spectral peak, as microphones leave')+ if name == 'line_share':+ return (f'{_pct(v)} of the energy sits in a few pure spectral lines (oscillator-like)' if not toward_real else+ f'energy spread over the spectrum (top 1% of bins hold {_pct(v)})')+ if name == 'pitch_wobble':+ return ('pitch is dead steady like an oscillator (no natural drift)' if not toward_real else+ 'pitch drifts naturally from frame to frame')+ if name == 'env_resid_db':+ return ('envelope is mathematically smooth (exponential / linear ramps)' if not toward_real else+ 'envelope has the micro-fluctuations of a recorded source')+ if name == 'texture_kurt':+ return ('its noise is Gaussian like filtered random numbers, not the crackle of a real texture' if not toward_real+ else 'high-frequency texture is crackly / impulsive like a real source')+ if name == 'quiet_flat':+ return ('the quiet parts are tonal tails, not broadband room noise' if not toward_real else+ 'the quiet parts are broadband room / mic noise')+ if name == 'tonal_frames':+ return (f'{_pct(v)} of frames are dominated by one pure tone' if not toward_real else+ f'few pure-tone frames ({_pct(v)})')+ if name == 'log_flatness':+ return ('spectrum is a few pure lines over near-silence' if not toward_real else+ 'spectrum is dense and noise-rich')+ if name == 'peakiness_db':+ return (f'one spectral line stands {v:.0f} dB above the median bin' if not toward_real else+ f'no dominant spectral line ({v:.0f} dB peak/median)')+ if name == 'bandwidth_khz':+ return f'content reaches {v:.1f} kHz'+ if name == 'crest_db':+ return (f'crest factor only {v:.0f} dB: a steady, oscillator-like level' if not toward_real else+ f'crest factor {v:.0f} dB: sharp natural transients')+ return f'{name} = {v:.3g}'+++def pure_tone(f):+ """a pure, dead-steady tone (test tone / bare oscillator): nearly all energy in one spectral line in every frame,+ a flat level (crest <= 4.5 dB) and no pitch drift. Out of the models' training distribution, so it is decided by+ rule; it fired on none of 4741 audible clips of the 345 real licensed recordings (eval/realness/tone_guard.json;+ closest: a wine-glass tap, tonal but crest 20 dB)"""+ w = f.get('pitch_wobble', float('nan'))+ return (f.get('tonal_frames', 0) >= 0.95 and f.get('line_share', 0) >= 0.9 and f.get('crest_db', 99) <= 4.5+ and (not np.isfinite(w) or w <= -1.0))+++def _facts(m, f):+ """signal facts that fire for these measurements: (side, precision, text). Each fact is a threshold measured on the+ training clips (train.py): past it, `precision` of the clips were of that side."""+ out = []+ for q in m.get('facts', []):+ v = f.get(q['feature'], float('nan'))+ if np.isfinite(v) and (v <= q['value'] if q['op'] == '<=' else v >= q['value']):+ who = 'real/pro' if q['side'] == 'real' else 'synthesized'+ out.append((q['side'], q['precision'], f"{_phrase(q['feature'], v, q['side'] == 'real')} "+ f"({100 * q['precision']:.0f}% of training clips past this were {who})"))+ return sorted(out, key=lambda t: -t[1])+++def _silence_line(m, f):+ """exact digital silence in words, with how common it was in each class of the training clips: edited and gated+ library recordings have it too (56% of the real training clips contain some), so it is context, never a verdict"""+ v = f.get('silence_frac', 0.0)+ rows = [r for r in m.get('silence_base', []) if v >= r['at_least']]+ if v < 0.02 or not rows:+ return None+ r = rows[-1]+ return (f"{_pct(v)} of samples are exact digital silence (edited, gated or synthesized; weak evidence: "+ f"{_pct(r['real'])} of real/pro and {_pct(r['synth'])} of synthesized training clips have "+ f">= {_pct(r['at_least'])})")+++def _clap_evidence(emb, prompt_emb, m, score):+ """the CLAP head's score in words, plus the closest of the 12 reference prompts (weak context: zero-shot prompts+ alone rank clips worse than the trained head, and their cosines are small)"""+ cos = prompt_emb @ emb+ n = len(m['prompts']['real'])+ names = m['prompts']['real'] + m['prompts']['synth']+ o = [int(k) for k in np.argsort(-cos)]+ i = o[0]+ j = next(k for k in o if (k < n) != (i < n))+ side = 'real' if i < n else 'synth'+ line = (f"CLAP head (decides p_real): score {score:+.1f} ({'toward real' if score >= 0 else 'toward synthesized'}); "+ f"closest reference prompt '{names[i]}' ({side} side, cos {cos[i]:.2f}; best on the other side "+ f"'{names[j]}', cos {cos[j]:.2f}; prompt cosines are weak context, not the decision)")+ return names[i], line+++# ---------------------------------------------------------------------------------------------- scoring+def _clips(x):+ """[(t0, t1, clip)] : the whole sound up to LONG_S, else 4 s windows (the last one aligned to the end, unless that+ would add under 0.25 s of new audio)"""+ n = len(x)+ if n <= LONG_S * audio.SR:+ return [(0.0, n / audio.SR, x)]+ w = audio.windows(x, WIN_S, WIN_S, pad_last=True)+ if len(w) > 2 and w[-1][0] - w[-2][0] < 0.25:+ w = w[:-1]+ return [(t0, t0 + WIN_S, c) for t0, c in w]+++def score_clips(clips, m=None, use_ast=False):+ """p_real + head logits + evidence parts for a list of mono 48 kHz arrays (silent ones -> None)"""+ from .models import clap as clap_mod+ m = m or load_model()+ rows = [None] * len(clips)+ live = []+ for i, x in enumerate(clips):+ f = signal_features(x) if audible(x) else None+ if f is not None:+ live.append(i)+ rows[i] = {'features': f}+ if not live:+ return rows+ c = clap_mod.get()+ E = c.embed_audio([clips[i] for i in live])+ P = c.embed_text(m['prompts']['real'] + m['prompts']['synth'])+ A = None+ if use_ast:+ from .models import ast as ast_mod+ t = ast_mod.get()+ A = (t, t.probs([clips[i] for i in live]))+ hs = m['heads']+ for j, i in enumerate(live):+ r = rows[i]+ lc, _ = head_logit(hs['clap'], E[j])+ ls, _ = head_logit(hs['signal'], _sig_vector(hs['signal'], r['features']))+ p, _ = fuse(m['fusion'], {'clap': lc, 'signal': ls})+ near, line = _clap_evidence(E[j], P, m, lc)+ facts = _facts(m, r['features'])+ tone = pure_tone(r['features'])+ if tone:+ p = min(p, 0.03)+ facts.insert(0, ('synth', 0.99, 'a pure, dead-steady tone: one spectral line, flat level, no pitch drift '+ '(a test tone or bare oscillator; real sources drift and breathe)'))+ line = (f"decided by the pure-tone rule, not the CLAP head (the CLAP head alone scores {lc:+.1f}, "+ f"{'toward real' if lc >= 0 else 'toward synthesized'}: a bare tone is outside its training data)")+ r.update(p_real=p, logits={'clap': lc, 'signal': ls}, near=near, clap_line=line, facts=facts, tone=tone)+ if A is not None:+ t, pr = A+ top = sorted(((lab, t.prob_of(pr[j], lab)) for lab in AST_SYNTH), key=lambda v: -v[1])[:3]+ r['ast_line'] = 'AST: ' + ', '.join(f'{lab} {v:.2f}' for lab, v in top) + \+ ' | top: ' + ', '.join(f'{lab} {v:.2f}' for lab, v in t.top(pr[j], 3))+ return rows+++def _cache_path(path, start, end, use_ast, m):+ k = hashlib.sha1(f'{audio.file_id(path)}|{start}|{end}|{use_ast}|{m["version"]}|{m.get("trained", "")}|{CACHE_V}'+ .encode()).hexdigest()[:24]+ d = os.path.join(config.EMB_CACHE, 'realness')+ os.makedirs(d, exist_ok=True)+ return os.path.join(d, k + '.json')+++def _check_range(path, start, end):+ """clear errors for a reversed range or one that starts after the end of the file"""+ if start is not None and start < 0:+ raise ValueError(f'start must be >= 0 s (got {start:g})')+ s0 = start or 0.0+ if end is not None and end <= s0:+ raise ValueError(f'end ({end:g} s) must be greater than start ({s0:g} s)')+ total = audio.probe(path).get('duration') or 0+ if total and s0 >= total:+ raise ValueError(f'start {s0:g} s is at or beyond the end of {os.path.basename(path)} ({total:g} s long)')+++def _spans(segs):+ """merge touching windows into 'm:ss.ss-m:ss.ss' spans"""+ out = []+ for s in segs:+ if out and s['start'] <= out[-1][1] + 1e-6:+ out[-1][1] = max(out[-1][1], s['end'])+ else:+ out.append([s['start'], s['end']])+ return ', '.join(f'{fmt_t(a)}-{fmt_t(b)}' for a, b in out[:6]) + (' ...' if len(out) > 6 else '')+++def realness(path, start=None, end=None, ast=False, cache=True):+ """p_real (probability the sound is a real recording or professionally produced, not code-synthesized), verdict,+ evidence in words and, for sounds over 6 s, per-4 s-window segments (times in seconds from the file start)"""+ m = load_model()+ if start is not None or end is not None:+ _check_range(path, start, end)+ cp = _cache_path(path, start, end, ast, m) if cache else None+ if cp and os.path.exists(cp):+ try:+ with open(cp, encoding='utf-8') as fh:+ return dict(json.load(fh), path=path) # the path as given now, not as the first caller spelled it+ except Exception: # noqa: BLE001 (a broken cache entry is recomputed)+ pass+ off = float(start or 0.0)+ x, gain, rdb, pdb = lift_quiet(audio.load(path, mono=True, start=start, end=end))+ dur = len(x) / audio.SR+ spans = _clips(x)+ rows = score_clips([w for _, _, w in spans], m, ast)+ res = {'path': path, 'duration': round(dur, 3)}+ name = os.path.basename(path)+ if start is not None or end is not None:+ res['range'] = [round(off, 3), round(off + dur, 3)]+ name += f' [{fmt_t(off)}-{fmt_t(off + dur)}]'+ scored = [(s, r) for s, r in zip(spans, rows) if r]+ if not scored:+ peak = 'exact digital silence' if pdb <= -199 else f'peak {pdb:.0f} dBFS'+ why = f'silent (no part above {SILENT_DB:.0f} dBFS RMS; {peak}), not judged'+ res.update(p_real=None, verdict='silent', why=why, evidence=['the sound is (near) silent: nothing to judge'],+ summary=f'{name}: {why}.')+ return res+ long_ = len(spans) > 1+ ps = np.array([r['p_real'] for _, r in scored])+ p = float(ps.mean())+ v = verdict_of(p, m, long_)+ real_side = p >= 0.5+ s0, r0 = (max if real_side else min)(scored, key=lambda sr: sr[1]['p_real'])+ side = 'real' if real_side else 'synth'+ pro = [t for sd, _, t in r0['facts'] if sd == side][:3]+ con = [t for sd, _, t in r0['facts'] if sd != side][:1]+ fu = m['fusion']+ sil = _silence_line(m, r0['features'])+ ev = ([r0['clap_line']] + pro + ['but: ' + t for t in con] + ([sil] if sil else []) ++ ([r0['ast_line']] if 'ast_line' in r0 else []))+ if gain:+ res['gain_db'] = round(gain, 1)+ if gain >= 3:+ ev.append(f'quiet source (RMS {rdb:.0f} dBFS, peak {pdb:.0f} dBFS): judged after +{gain:.0f} dB of gain (very '+ 'quiet input bends the CLAP features; level otherwise barely moves p_real, see RESULTS.md)')+ if long_:+ ev.insert(0, f"evidence below is from the most {'real' if real_side else 'synthesized'}-sounding window, "+ f"{fmt_t(off + s0[0])}-{fmt_t(off + min(s0[1], dur))} (p_real {r0['p_real']:.2f})")+ if fu['coef'][1] != 0:+ ev.append('fusion weights: CLAP head {:.2f}, signal head {:.2f} (signal head score {:+.1f})'.format(+ *fu['coef'], r0['logits']['signal']))+ elif pro or con or sil:+ ev.append('the signal measurements above are context, not votes: the signal head added no held-out accuracy, '+ 'so its fusion weight is 0')+ t = thresholds_of(m)+ if not long_:+ lc = r0['logits']['clap']+ why = (f"pure-tone rule (the CLAP head alone: {lc:+.1f})" if r0.get('tone') else+ f"CLAP head score {lc:+.1f} ({'toward real' if lc >= 0 else 'toward synthesized'})") + \+ (f'; {pro[0]}' if pro and not r0.get('tone') else '')+ res.update(p_real=round(p, 3), verdict=v, why=why, evidence=ev,+ summary=f"{name}: {v} (p_real {_pf(p, t)}; clip thresholds lo {t['lo']} / hi {t['hi']}): {why}.",+ note=f"p_real is calibrated on held-out clips (lo {t['lo']} / hi {t['hi']} hold ~92% precision "+ 'there). It hears how a sound sounds, not where it came from: clean professionally synthesized '+ 'tones (UI sounds, chimes, synth stabs) also read as code-synthesized. Trained on Freesound/'+ 'Mixkit recordings, pro launch films and music vs numpy-synthesized sounds: see '+ 'eval/realness/RESULTS.md for accuracy and what fools it')+ else:+ segs = []+ for (a, b, _), r in zip(spans, rows):+ seg = {'start': round(off + a, 3), 'end': round(off + min(b, dur), 3)}+ if not r:+ seg.update(p_real=None, verdict='silent')+ else:+ sd = 'real' if r['p_real'] >= 0.5 else 'synth'+ seg.update(p_real=round(r['p_real'], 3), verdict=verdict_of(r['p_real'], m))+ fx = [q for s_, _, q in r['facts'] if s_ == sd]+ if fx:+ seg['why'] = fx[0]+ segs.append(seg)+ live = [s for s in segs if s['p_real'] is not None]+ lean = [s for s in live if s['p_real'] < 0.5]+ flagged = [s for s in live if s['p_real'] <= t['lo']]+ ft = thresholds_of(m, True)+ mlc = float(np.mean([r['logits']['clap'] for _, r in scored]))+ tones = sum(1 for _, r in scored if r.get('tone'))+ why = (f'{len(lean)} of {len(live)} windows lean synthesized (p_real < 0.5); mean CLAP head score {mlc:+.1f}' ++ (f'; {tones} window(s) decided by the pure-tone rule' if tones else ''))+ if lean:+ tail = f' Leaning synthesized: {_spans(lean)}' + (+ f"; {len(flagged)} at or below the 4 s-window threshold {t['lo']} (code-synthesized): {_spans(flagged)}."+ if flagged else f"; none at or below the 4 s-window threshold {t['lo']}.")+ else:+ tail = (f' No window leans synthesized (lowest p_real {ps.min():.2f}); that does not rule out a code-made '+ 'layer under a louder real mix (most are missed): judge the source files and stems.')+ res.update(p_real=round(p, 3), verdict=v, why=why, evidence=ev, segments=segs, synth_windows=len(flagged),+ lean_synth_windows=len(lean), p_real_min=round(float(ps.min()), 3),+ summary=(f"{name}: {v} (p_real {_pf(p, ft)}, mean of {len(live)} 4 s windows; file thresholds lo "+ f"{ft['lo']} / hi {ft['hi']}, weak calibration): {why}." + tail),+ note=f"a long file's p_real is the mean of its window probabilities; its verdict uses file-level "+ f"thresholds lo {ft['lo']} / hi {ft['hi']} read from held-out long files "+ f"({ft.get('files', '?')} files, only {ft.get('synth_files', '?')} synthesized: weak "+ f"calibration). Each window is calibrated on held-out 4 s clips (lo {t['lo']} / hi {t['hi']}). "+ 'Windows find stretches made entirely of code synthesis; a code-made layer under a louder real '+ 'mix is usually missed, so judge the source files and stems too. It hears how a sound sounds, '+ 'not where it came from (clean pro-synthesized UI tones read as code-synthesized): see '+ 'eval/realness/RESULTS.md')+ if cp:+ tmp = f'{cp}.{os.getpid()}.tmp'+ with open(tmp, 'w', encoding='utf-8') as fh:+ json.dump(res, fh)+ audio.replace_quietly(tmp, cp)+ return res+++def _pf(p, t):+ """p_real for text: 2 decimals, 3 when 2 would round onto a threshold it is actually below / above"""+ s = f'{p:.2f}'+ return f'{p:.3f}' if any(float(s) == v and abs(p - v) > 1e-9 for v in (t['lo'], t['hi'])) else s+++def _missing(paths):+ """[(path, reason)] for given paths that name no media file"""+ return [(q, 'no such file or folder' if not os.path.exists(q) else 'no audio or video files in it')+ for q in paths if not audio.list_media([q])]+++def _error_row(path, why):+ return {'path': path, 'p_real': None, 'verdict': 'error', 'why': why,+ 'summary': f'{os.path.basename(path) or path}: error: {why}'}+++def realness_many(paths, ast=False, start=None, end=None):+ """realness of every media file in paths (files, folders, globs); the range applies to each file. Paths that+ name nothing and files that fail to decode are 'error' rows, silent files 'silent' rows: neither counts as real."""+ paths = [paths] if isinstance(paths, str) else list(paths)+ rows = []+ for f in audio.list_media(paths):+ try:+ rows.append(realness(f, start, end, ast=ast))+ except Exception as e: # noqa: BLE001 (one bad file must not stop the batch)+ msg = (str(e).strip().splitlines() or [''])[0]+ rows.append(_error_row(f, f'{type(e).__name__}: {msg}'[:200]))+ rows += [_error_row(q, why) for q, why in _missing(paths)]+ n = {k: sum(r.get('verdict') == k for r in rows) for k in VERDICTS}+ parts = [f"{n['real or pro-produced']} real or pro-produced", f"{n['uncertain']} uncertain",+ f"{n['likely code-synthesized']} likely code-synthesized"]+ if n['silent']:+ parts.append(f"{n['silent']} silent (not judged)")+ if n['error']:+ parts.append(f"{n['error']} could not be read (not judged: check them)")+ rng = f" Range {fmt_t(start or 0)}-{fmt_t(end) if end is not None else 'end'} of each file." \+ if start is not None or end is not None else ''+ return {'files': rows, 'counts': n, 'summary': f'{len(rows)} files: ' + ', '.join(parts) + '.' + rng}+++def _text(r):+ lines = [r['summary']]+ for e in r.get('evidence', []):+ lines.append(' - ' + e)+ for s in r.get('segments', []):+ pr = ' - ' if s['p_real'] is None else f"{s['p_real']:.2f}"+ lean = ', leans synthesized' if s['verdict'] == 'uncertain' and s['p_real'] < 0.5 else ''+ lines.append(f" {fmt_t(s['start'])}-{fmt_t(s['end'])} p_real {pr} {s['verdict']}{lean}" ++ (f" ({s['why']})" if s.get('why') and s['verdict'] != 'real or pro-produced' else ''))+ if r.get('note'):+ lines.append(' note: ' + r['note'])+ return '\n'.join(lines)+++def _table(res):+ try:+ m = load_model()+ t, ft = thresholds_of(m), thresholds_of(m, True)+ rule = (f"Verdict thresholds: sounds up to {LONG_S:.0f} s lo {t['lo']} / hi {t['hi']}; longer files (mean of 4 s "+ f"windows) lo {ft['lo']} / hi {ft['hi']}, weakly calibrated. [n low windows] = 4 s windows at or below "+ f"{t['lo']}. For a multi-take file judge the trim you will use (--start / --end).")+ except Exception: # noqa: BLE001+ rule = ''+ lines = [res['summary']] + ([rule] if rule else []) + [f"{'p_real':>6} {'verdict':<24} file: main reason"]+ for r in res['files']:+ p = ' - ' if r.get('p_real') is None else f"{r['p_real']:.2f}"+ extra = f" [{r['synth_windows']} low window(s)]" if r.get('synth_windows') else ''+ long_ = ' (file)' if r.get('segments') else ''+ lines.append(f"{p:>6} {r.get('verdict', '') + long_:<24} {os.path.basename(r['path']) or r['path']}{extra}: "+ f"{r.get('why') or r.get('summary', '')}")+ return '\n'.join(lines)+++def add_cli(sub):+ p = sub.add_parser('realness', help='real recording / pro-produced audio vs code-synthesized sound')+ p.add_argument('paths', nargs='+', help='files or folders (audio or video)')+ p.add_argument('--start', type=float, help='judge only from this time (seconds from the file start; with several '+ 'files it applies to each)')+ p.add_argument('--end', type=float, help='judge only up to this time (seconds from the file start)')+ p.add_argument('--ast', action='store_true', help='also run the AudioSet tagger for evidence (slower)')+ common(p)++ def run(a):+ bad = _missing(a.paths)+ if bad:+ for q, why in bad:+ print(f'earshot realness: error: {q}: {why}', file=sys.stderr)+ return 1+ files = audio.list_media(a.paths)+ if len(files) == 1:+ r = realness(files[0], a.start, a.end, a.ast)+ emit(r, a, _text(r))+ return 0+ r = realness_many(files, a.ast, a.start, a.end)+ emit(r, a, _table(r))+ return 1 if r['counts']['error'] else 0+ p.set_defaults(func=run)+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/report.py b/earshot/report.pynew file mode 100644index 0000000..2f881b3--- /dev/null+++ b/earshot/report.py@@ -0,0 +1,86 @@+"""Output helpers: every command returns a dict with a plain-language "summary" plus details.++Agents read the text form by default; --json prints the full dict; --out FILE also writes it to disk.+Conventions: times in seconds (3 decimals), levels in dBFS / LUFS / dBTP (1 decimal), similarities as cosine (3+decimals), calibrated scores as z (standard deviations above a background) or 0..1 probabilities.+"""+import json+import math+import os+import sys++import numpy as np+++def _clean(o):+ if isinstance(o, dict):+ return {str(k): _clean(v) for k, v in o.items()}+ if isinstance(o, (list, tuple)):+ return [_clean(v) for v in o]+ if isinstance(o, (np.floating,)):+ o = float(o)+ if isinstance(o, (np.integer,)):+ return int(o)+ if isinstance(o, np.ndarray):+ return _clean(o.tolist())+ if isinstance(o, float):+ if math.isnan(o) or math.isinf(o):+ return None+ return o+ return o+++def to_json(result, indent=1):+ return json.dumps(_clean(result), indent=indent, ensure_ascii=False)+++def render_text(result):+ """generic text rendering: the summary first, then compact key: value lines"""+ lines = []+ if result.get('summary'):+ lines.append(str(result['summary']))+ for k, v in result.items():+ if k in ('summary', 'text'):+ continue+ if isinstance(v, (dict, list)):+ s = json.dumps(_clean(v), ensure_ascii=False)+ if len(s) > 600:+ s = s[:600] + ' ...'+ else:+ s = str(v)+ lines.append(f'{k}: {s}')+ return '\n'.join(lines)+++def emit(result, args, text=None):+ """print the result for a CLI call (text unless --json) and write --out if given"""+ if getattr(args, 'out', None):+ d = os.path.dirname(os.path.abspath(args.out))+ os.makedirs(d, exist_ok=True)+ with open(args.out, 'w', encoding='utf-8') as fh:+ fh.write(to_json(result))+ if getattr(args, 'json', False):+ # ASCII escapes: valid JSON whatever the console code page (a piped Windows stdout is often cp1252)+ sys.stdout.write(json.dumps(_clean(result), indent=1, ensure_ascii=True) + '\n')+ else:+ t = text if text is not None else (result.get('text') or render_text(result))+ try:+ sys.stdout.write(t + '\n')+ except UnicodeEncodeError:+ enc = sys.stdout.encoding or 'ascii'+ sys.stdout.write(t.encode(enc, 'backslashreplace').decode(enc, 'replace') + '\n')+ return result+++def error_line(cmd, msg):+ """the one-line error every command prints on stderr: 'earshot <cmd>: error: <msg>'"""+ msg = ' '.join(str(msg).split()) or 'unknown error'+ print(f'earshot {cmd}: error: {msg}', file=sys.stderr)+++def fmt_t(t):+ """seconds -> m:ss.ss (agent-readable timestamps)"""+ if t is None:+ return '-'+ m = int(t // 60)+ return f'{m}:{t - 60 * m:05.2f}'diff --git a/earshot/search.py b/earshot/search.pynew file mode 100644index 0000000..31aa0b1--- /dev/null+++ b/earshot/search.py@@ -0,0 +1,476 @@+"""Find the best sound for a description: rank many files, or find the best moments inside one long recording.++ earshot rank "a deep thocky key press" sfx/keyboard/ --top 5+ earshot find typing.ogg "a single mechanical key press" --top 5+ earshot find meadow.ogg "a skylark singing" --kind texture --win 4+ from earshot.search import rank, find+ find('keys.ogg', 'a single key press')['segments'][0] # {'start': 1.119, 'end': 1.734, 'z': 3.1, ...}++rank: every readable file is embedded once (cached; long files = mean of up to 6 evenly spaced 10 s windows) and sorted+by the calibrated z of the description (earshot/vocab.py), with the verdict of earshot.match. Unreadable files are+skipped (listed under 'skipped'), near-silent files are flagged and ranked last. vocab_percentile = where the query+ranks among the vocabulary descriptions for that file (not a rank among the files).+find: candidate segments are scored with CLAP and the best non-overlapping ones returned. Candidate kinds:+ stroke - one onset until it has decayed by 35 dB (never into the next onset)+ event - strokes that belong together: pairs (gap < 0.25 s: a key's down- and up-stroke; never led by a stroke 15 dB+ weaker than the rest) and longer chains, at most 2 s; never across an edit gap (>= 10 ms of exact digital+ silence: separate takes in a sample pack). Up to 60 s: at most 48 strokes / events, picked round-robin over+ 2 s stretches (isolated, prominent ones first). Longer files: two stages - the windows are scored first,+ then the strokes / events inside the 6 best windows, so a strong moment anywhere in the file is found.+ window - sliding windows covering the whole range with no gaps (default win = duration / 6 clamped to 1..5 s,+ hop = win / 2; long files grow the window up to 10 s, then the hop up to the window, to stay near 40).+ For one-shot queries each window's edges are snapped to the sound before scoring: a start+ inside a rising sound moves back to where it is quiet, a start inside the tail of an earlier sound moves+ forward past it, an end inside a decay moves on until it has decayed (each by at most 1 s), an end inside+ a new attack moves back before it. For in-between and continuous queries windows are plain stretches+ (crossfade them).+kind='auto' reads the query (CLAP text similarity to generic one-shot vs continuous phrases; thresholds from labelled+query texts, eval/semantics/kind.json): one-shot queries score all three kinds, never return a window that holds+two or more separate sounds (a quiet gap >= 0.25 s or an edit gap between them) and replace a winning window by the+best stroke / event inside it (+-50 ms) when that one scores within 0.5 z (a tighter trim of the same moment); clearly+continuous queries (rain, room tone, music) use windows only; anything in between scores all kinds and lets the+scores decide. kind='event' returns strokes / events only (windows only when the recording has no onsets).+start/end are sample trims in seconds: strokes / events start 5 ms before the measured attack, snapped windows 5 ms+before their sound and end 20 ms after it has decayed; head_db / tail_db say how loud the first 5 ms / last 10 ms+are against the segment's peak (tail -30 dB or lower = decayed cleanly; head or tail+above -15 dB = cut into a sound, flagged 'head cut' / 'tail cut' and ranked 0.5 z lower; plain windows of in-between+and continuous queries are not judged on their edges, they are cut anywhere by design). With --start/--end only segments whose attack lies inside the+range are returned (the audio is read from 0.1 s earlier so an attack at the boundary is seen whole).+Speed: every candidate costs one CLAP window (~0.3-1 s on CPU); candidates are query-independent and cached, so+more queries on the same file are fast.+"""+import os++import numpy as np++from . import audio, features, vocab+from .cli import common+from .match import SILENT_DBFS, VERDICT_EVIDENCE, VERDICTS, verdict+from .report import fmt_t++FRAME = 0.005+PRE_ROLL = 0.005+MARGIN = 0.1+CAP_S = 2.0+MERGE_GAP = 0.25+DECAY_DB = 35.0+MAX_EVENTS = 48+MAX_WINDOWS = 40+MAX_WIN_S = 10.0+SINGLE_STAGE_S = 60.0+TOP_WINDOWS = 6+BUCKET_S = 2.0+REFINE_DZ = 0.5+WEAK_DB = 15.0+CUT_DB = -15.0+TAIL_PENALTY = 0.5+EDIT_DB, EDIT_S = -100.0, 0.01 # an edit gap: >= 10 ms at or below -100 dBFS (exact digital silence between takes)+SNAP_S = 1.0 # how far a window edge may move to reach the sound's start / decay+QUIET_DB = 40.0 # 'quiet' = 40 dB under the window's peak (or 6 dB over the floor, if higher)+EVENT_ANCHORS = ['a single short sound effect', 'one short hit', 'a click', 'a single impact', 'one short pop',+ 'a quick one-shot sound']+TEXTURE_ANCHORS = ['a continuous sound', 'a long background atmosphere', 'a steady drone', 'a piece of music',+ 'ambient room noise', 'a soundscape']+# from the labelled tuning phrasings (eval/semantics/kind_queries.json "tune"): event = the lowest event score+# (rounded down), texture = the median texture score (only clearly continuous descriptions skip the onset candidates)+KIND_EVENT, KIND_TEXTURE = -0.08, -0.27+++KIND_WORDS = {'event': 'read as a one-shot query', 'texture': 'read as a continuous-sound query (windows only)',+ 'either': 'read as either a one-shot or a continuous sound'}+++def _weakly(v):+ return 'not a match' if v == 'no match' else f'only a {v} match'+++# ---------------------------------------------------------------------------------------------------- rank+def rank(paths, query, top=None):+ """files (or folders / globs) sorted by how well they match `query` (calibrated z); a given path that yields no+ media file raises audio.AudioError"""+ audio.check_paths(paths)+ files, bad = vocab.readable(audio.list_media(paths))+ skipped = f" Skipped {len(bad)} unreadable file(s): {', '.join(os.path.basename(b['path']) for b in bad[:3])}." \+ if bad else ''+ if not files:+ return {'summary': f'no readable media files found in {paths}.' + skipped, 'query': query, 'n_files': 0,+ 'results': [], 'skipped': bad}+ v = vocab.get()+ A = vocab.embed_files(files)+ r = v.calibrate(A, v.embed_text([query]))+ z, cos, pct = np.round(r['z_q'][:, 0], 2), r['cos_q'][:, 0], r['pct_q'][:, 0]+ silent = np.array([vocab.peak_db(p) < SILENT_DBFS for p in files])+ order = sorted(range(len(files)), key=lambda i: (silent[i], -z[i]))+ res = []+ for i in order[:top] if top else order:+ bt = v.top_terms(A[i], k=1, exclude=('quality', 'production'))[0]+ res.append({'path': files[i], 'name': os.path.basename(files[i]), 'z': round(float(z[i]), 2),+ 'cosine': round(float(cos[i]), 3), 'vocab_percentile': round(100 * float(pct[i]), 1),+ 'verdict': 'no match' if silent[i] else verdict(float(z[i])), 'near_silent': bool(silent[i]),+ 'sounds_most_like': f'{bt[0]} (z {vocab.zf(bt[2])})'})+ counts = {n: sum(1 for i in order if not silent[i] and verdict(float(z[i])) == n) for _, n in VERDICTS}+ head = ', '.join(f"{x['name']} (z {vocab.zf(x['z'])}, {x['verdict']})" for x in res[:3] if not x['near_silent'])+ s = (f"Ranked {len(files)} files for '{query}'. Best: {head or 'none (all near-silent)'}. "+ f"{counts['strong']} strong, {counts['plausible']} plausible, {counts['weak']} weak match(es).")+ if res and not res[0]['near_silent'] and res[0]['verdict'] in ('weak', 'no match'):+ s += f" Even the best is {_weakly(res[0]['verdict'])}: it sounds more like {res[0]['sounds_most_like']}."+ if counts['strong'] > 1:+ ev = VERDICT_EVIDENCE['strong']+ s += (f" In a held-out test 'strong' was right {100 * ev['precision_library']:.0f}% of the time, "+ f"{100 * ev['precision_library_strict']:.0f}% when a close variant (a mouth pop for a cork pop) counts as wrong.")+ if silent.any():+ s += f' {int(silent.sum())} near-silent file(s) (peak < {SILENT_DBFS:.0f} dBFS) ranked last: their scores mean nothing.'+ return {'summary': s + skipped, 'query': query, 'n_files': len(files), 'results': res, 'skipped': bad}+++# ---------------------------------------------------------------------------------------------------- find+def _env_db(x, frame=FRAME):+ w = max(1, int(round(frame * audio.SR)))+ return 20 * np.log10(np.maximum(features.frame_rms(x, w), 1e-12))+++def _refine_start(env1, t, floor):+ """first 1 ms frame of the attack near onset t (s), so the trim does not clip it"""+ i0, i1 = max(0, int(t * 1000) - 30), min(len(env1), int(t * 1000) + 30)+ seg = env1[i0:i1]+ if not len(seg):+ return t+ k = int(np.argmax(seg >= max(floor + 10, seg.max() - 30)))+ return (i0 + k) / 1000+++def query_kind(v, query):+ """('event' | 'texture' | 'either', score): CLAP text similarity to one-shot vs continuous anchor phrases"""+ q = v.embed_text([query])[0]+ d = float(q @ v.embed_text(EVENT_ANCHORS).mean(0) - q @ v.embed_text(TEXTURE_ANCHORS).mean(0))+ return ('event' if d >= KIND_EVENT else 'texture' if d <= KIND_TEXTURE else 'either'), round(d, 3)+++def _tail_db(env, a, b):+ """level of the last 10 ms of [a, b) relative to the segment's peak (5 ms frames)"""+ seg = env[int(a / FRAME):max(int(a / FRAME) + 2, int(b / FRAME))]+ return float(seg[-2:].max() - seg.max()) if len(seg) else 0.0+++def _head_db(env, a, b):+ """level of the first 5 ms frame of [a, b) (the pre-roll of a stroke / event) relative to the segment's peak"""+ seg = env[int(a / FRAME):max(int(a / FRAME) + 2, int(b / FRAME))]+ return float(seg[0] - seg.max()) if len(seg) else 0.0+++def _edit_gaps(env, a, b):+ """True when [a, b) holds an edit gap: >= 10 ms of exact digital silence (5 ms frames)"""+ seg = (env[int(a / FRAME):int(b / FRAME)] <= EDIT_DB).astype(float)+ n = int(round(EDIT_S / FRAME))+ return bool(len(seg) >= n and np.convolve(seg, np.ones(n), 'valid').max() >= n)+++def _quiet_level(env, floor, i0, i1):+ seg = env[i0:i1]+ return max(floor + 6.0, float(seg.max()) - QUIET_DB) if len(seg) else floor + 6.0+++def snap_window(env, floor, a, b, lo, hi):+ """(a, b) of a window moved onto the sound's own edges (see the module docstring); lo / hi bound the search"""+ i0, i1 = int(round(a / FRAME)), max(int(round(a / FRAME)) + 1, int(round(b / FRAME)))+ i1 = min(i1, len(env))+ if i0 >= i1 - 1:+ return a, b+ q = _quiet_level(env, floor, i0, i1)+ q_end = max(floor + 4.0, float(env[i0:i1].max()) - DECAY_DB)+ w, n_lo, n_hi = 6, max(0, int(round(lo / FRAME))), min(len(env), int(round(hi / FRAME)))+ step = int(SNAP_S / FRAME)+ if env[i0] > q:+ falling = i0 - w >= n_lo and env[i0 - w:i0].mean() > env[i0:i0 + w].mean() + 1.0+ if falling: # the window starts inside the tail of an earlier sound: start after that tail+ k = next((k for k in range(i0, min(i1 - 1, i0 + step)) if env[k] <= q), None)+ if k is not None and (i1 - k) * FRAME >= 0.1:+ i0 = k+ else: # it starts inside a rising / sounding stretch: go back to where it is quiet+ k = next((k for k in range(i0, max(n_lo, i0 - step) - 1, -1) if env[k] <= q), None)+ if k is not None:+ i0 = k+ last = i1 - 1+ if env[last] > q_end:+ rising = last + 1 + w <= n_hi and env[last + 1:last + 1 + w].mean() > env[max(i0, last - w):last + 1].mean() + 1.0+ if rising: # a new sound starts at the end: stop before its attack+ k = next((k for k in range(last, i0, -1) if env[k] <= q), None)+ if k is not None and (k - i0) * FRAME >= 0.1:+ i1 = k + 1+ else: # still decaying (or sounding): follow it until it has decayed, never into a new attack+ k, low = last, env[last]+ while k + 1 < min(n_hi, last + step) and env[k] > q_end and env[k + 1] <= low + 6.0:+ k += 1+ low = min(low, env[k])+ i1 = k + 1+ seg = env[i0:i1] # tighten: from one frame (5 ms) before the sound to its decay, no leading / trailing silence+ loud = np.nonzero(seg > q)[0]+ if len(loud):+ i0 = i0 + max(0, int(loud[0]) - 1)+ dec = np.nonzero(env[i0:i1] > q_end)[0]+ if len(dec):+ i1 = i0 + int(dec[-1]) + 1+ a2, b2 = i0 * FRAME, i1 * FRAME+ return max(lo, a2), min(hi, b2 + (0.02 if b2 < hi else 0.0))+++def multi_sound(env, floor, a, b, gap=MERGE_GAP):+ """True when [a, b) holds two or more separate sounds: a quiet stretch >= gap, or an edit gap, with sound on both+ sides"""+ i0, i1 = int(round(a / FRAME)), int(round(b / FRAME))+ seg = env[i0:i1]+ if len(seg) < 3:+ return False+ q = _quiet_level(env, floor, i0, i1)+ loud = np.nonzero(seg > q)[0]+ if len(loud) < 2:+ return False+ if _edit_gaps(env, a + loud[0] * FRAME, a + (loud[-1] + 1) * FRAME):+ return True+ quiet = (seg[loud[0]:loud[-1] + 1] <= q).astype(float)+ n = int(round(gap / FRAME))+ return bool(len(quiet) >= n and np.convolve(quiet, np.ones(n), 'valid').max() >= n)+++def _select(cands, n=MAX_EVENTS, bucket=BUCKET_S):+ """at most n candidates, round-robin over time buckets, best priority first within each bucket"""+ by = {}+ for c in sorted(cands, key=lambda c: -c[3]):+ by.setdefault(int(c[0] // bucket), []).append(c)+ out = []+ while len(out) < n and any(by.values()):+ for k in sorted(by):+ if by[k] and len(out) < n:+ out.append(by[k].pop(0))+ return out+++def event_candidates(x, cap=CAP_S, n=MAX_EVENTS):+ """[(start, end, kind, priority)] onset-based segments of a mono 48 kHz array (priority = prominence over the+ noise floor in dB, +10 when the event is isolated: quiet before it and decayed by 25 dB at its end); at most n+ (round-robin over time), all of them when n is None"""+ dur = len(x) / audio.SR+ on = features.onsets(x)+ if not on:+ return []+ env = _env_db(x)+ env1 = _env_db(x, 0.001)+ floor = float(np.percentile(env, 10))+ strokes = []+ for i, o in enumerate(on):+ nxt = on[i + 1] if i + 1 < len(on) else dur+ f0, f1 = int(o / FRAME), max(int(o / FRAME) + 1, int(nxt / FRAME))+ seg = env[f0:f1]+ if not len(seg):+ continue+ pk = f0 + int(np.argmax(seg[:int(0.06 / FRAME) + 1]))+ below = np.nonzero(env[pk:f1] < max(env[pk] - DECAY_DB, floor + 4))[0]+ end = (pk + below[0]) * FRAME + 0.02 if len(below) else nxt+ st = max(0.0, _refine_start(env1, o, floor) - PRE_ROLL)+ if strokes:+ st = max(st, strokes[-1][1])+ end = min(end, nxt - 0.002 if i + 1 < len(on) else dur, st + cap)+ if end - st >= 0.02:+ strokes.append((st, end, float(env[pk] - floor)))+ out = [(s, e, 'stroke', p) for s, e, p in strokes]+ joins = [b[0] - a[1] < MERGE_GAP and not _edit_gaps(env, a[0], b[0]) for a, b in zip(strokes, strokes[1:])]+ for k, (a, b) in enumerate(zip(strokes, strokes[1:])):+ if joins[k] and b[1] - a[0] <= cap and a[2] >= b[2] - WEAK_DB:+ out.append((a[0], b[1], 'event', max(a[2], b[2])))+ i = 0+ while i < len(strokes):+ j = i+ while j + 1 < len(strokes) and joins[j] and strokes[j + 1][1] - strokes[i][0] <= cap:+ j += 1+ ch = strokes[i:j + 1]+ top = max(s[2] for s in ch)+ while ch and ch[0][2] < top - WEAK_DB:+ ch = ch[1:]+ if len(ch) > 2:+ out.append((ch[0][0], ch[-1][1], 'event', top))+ i = j + 1+ res = []+ for s, e, k, p in out:+ pre = env[max(0, int(s / FRAME) - 6):int(s / FRAME)]+ iso = (not len(pre) or pre.mean() <= floor + 10) and _tail_db(env, s, e) <= -25+ res.append((s, e, k, p + 10 * iso))+ return res if n is None else _select(res, n)+++def window_candidates(dur, win=None, hop=None, t0=0.0):+ """sliding windows covering [t0, dur) without gaps; with defaults about MAX_WINDOWS of them at most (the window+ grows up to 10 s, then the hop up to the window length, before the count grows)"""+ span = dur - t0+ if not win:+ win = float(np.clip(span / 6, 1.0, 5.0))+ while win < MAX_WIN_S and (span - win) / (win / 2) + 1 > MAX_WINDOWS:+ win = min(MAX_WIN_S, win * 1.25)+ if span <= win:+ return [(t0, dur, 'window', 0.0)]+ if not hop:+ hop = win / 2+ if (span - win) / hop + 1 > MAX_WINDOWS:+ hop = min(win, (span - win) / (MAX_WINDOWS - 1))+ st = list(np.arange(t0, dur - win + 1e-6, hop))+ if dur - win - st[-1] > 1e-3:+ st.append(dur - win)+ return [(float(s), float(s + win), 'window', 0.0) for s in st]+++def _dedupe(cands, iou=0.9):+ out = []+ for c in cands:+ if all(min(c[1], d[1]) - max(c[0], d[0]) < iou * (max(c[1], d[1]) - min(c[0], d[0])) for d in out):+ out.append(c)+ return out+++def _score(v, path, m0, cands, Qe):+ """(z, cosine) of every candidate (local times + m0 = file times)"""+ if not cands:+ return np.zeros(0), np.zeros(0)+ r = v.calibrate(vocab.embed_spans([(path, m0 + a, m0 + b) for a, b, _, _ in cands]), Qe)+ return r['z_q'][:, 0], r['cos_q'][:, 0]+++def find(path, query, win=None, hop=None, top=5, start=None, end=None, kind='auto'):+ """best non-overlapping segments of a recording for `query`; start/end are sample trims in seconds"""+ if not query or not query.strip():+ raise ValueError('no query given')+ full = vocab.check(path, start, end)+ s0, s1 = start or 0.0, min(end, full) if end is not None else full+ m0 = max(0.0, s0 - MARGIN)+ x = audio.load(path, mono=True, start=m0, end=s1)+ off, dur = s0 - m0, len(x) / audio.SR+ v = vocab.get()+ Qe = v.embed_text([query])+ qkind, qscore = query_kind(v, query)+ use_ev = kind == 'event' or (kind == 'auto' and qkind != 'texture')+ prefer_ev = kind == 'event' or (kind == 'auto' and qkind == 'event')+ env = _env_db(x)+ floor = float(np.percentile(env, 10))+ wins = window_candidates(dur, win, hop, off)+ n_win, win_s = len(wins), wins[0][1] - wins[0][0]+ if prefer_ev: # one-shot queries: window edges on the sound, not on a fixed grid+ wins = _dedupe([snap_window(env, floor, a, b, off, dur) + (k, p) for a, b, k, p in wins])+ zw, cw = _score(v, path, m0, wins, Qe)+ evs, stage = [], 'one stage'+ if use_ev:+ allev = [c for c in event_candidates(x, n=None) if c[0] + PRE_ROLL >= off - 1e-3]+ if s1 - s0 <= SINGLE_STAGE_S:+ evs = _select(allev)+ else:+ best = [wins[i] for i in np.argsort(-zw)[:TOP_WINDOWS]]+ evs = _select([c for c in allev if any(c[0] < b and c[1] > a for a, b, _, _ in best)])+ stage = f'two stages: events inside the {len(best)} best windows'+ evs = _dedupe(evs)+ ze, ce = _score(v, path, m0, evs, Qe)+ if kind == 'event' and evs: # onset-based segments only+ wins, zw, cw = [], zw[:0], cw[:0]+ cands = evs + wins+ z, cos = np.round(np.concatenate([ze, zw]), 2), np.concatenate([ce, cw])+ tails = np.array([_tail_db(env, a, b) for a, b, _, _ in cands])+ heads = np.array([_head_db(env, a, b) for a, b, _, _ in cands])+ judged = np.array([use_ev and (prefer_ev or c[2] != 'window') for c in cands], bool) # whose edges must be clean+ cut = ((tails > CUT_DB) | (heads > CUT_DB)) & judged+ score = z - TAIL_PENALTY * cut+ rank_z = score.copy()+ if prefer_ev:+ for i, (a, b, k, _) in enumerate(cands):+ if k != 'window':+ continue+ inside = [z[j] for j, c in enumerate(cands) if c[2] != 'window' and c[0] >= a - 0.05 and c[1] <= b + 0.05]+ if (inside and max(inside) >= z[i] - REFINE_DZ) or (evs and multi_sound(env, floor, a, b)):+ rank_z[i] = -np.inf+ xs = audio.load(path, start=m0, end=s1)+ loud_file = audio.db(np.abs(xs).max() if len(xs) else 0.0) >= SILENT_DBFS+ picked = []+ for i in np.argsort(-rank_z):+ if not np.isfinite(rank_z[i]):+ break+ a, b = float(cands[i][0]), float(cands[i][1])+ if all(b <= c['_a'] + 1e-3 or a >= c['_b'] - 1e-3 for c in picked):+ seg = xs[int(a * audio.SR):int(b * audio.SR)]+ if loud_file and audio.db(np.abs(seg).max() if len(seg) else 0) < SILENT_DBFS:+ continue # a near-silent stretch of a sounding file is never an answer+ picked.append({'_a': a, '_b': b, 'start': round(m0 + a, 3), 'end': round(m0 + b, 3),+ 'duration': round(b - a, 3), 'score': round(float(score[i]), 2), 'z': round(float(z[i]), 2),+ 'cosine': round(float(cos[i]), 3), 'verdict': verdict(float(z[i])), 'kind': cands[i][2],+ 'peak_dbfs': round(audio.db(np.abs(seg).max() if len(seg) else 0), 1),+ 'head_db': round(float(heads[i]), 1), 'tail_db': round(float(tails[i]), 1),+ 'cut': ', '.join(w for w, c in (('head cut', heads[i] > CUT_DB), ('tail cut', tails[i] > CUT_DB))+ if c and judged[i])})+ if len(picked) >= top:+ break+ for p in picked:+ del p['_a'], p['_b']+ rng = f'{fmt_t(s0)}-{fmt_t(s1)} ({s1 - s0:.1f} s)' if start is not None or end is not None else f'all {s1 - s0:.1f} s'+ nw = len(wins) if wins else n_win+ how = (f"{KIND_WORDS[qkind] if kind == 'auto' else 'kind=' + kind}; searched {rng}: {nw} windows of ~{win_s:.1f} s"+ + (' (edges snapped to the sound)' if prefer_ev else '') + (' to locate events, none returned' if not wins else '')+ + (f" + {len(evs)} onset-based candidates ({stage})" if use_ev else ''))+ if picked:+ segs = '; '.join(f"{fmt_t(p['start'])}-{fmt_t(p['end'])} ({p['duration']:.2f} s {p['kind']}, z {vocab.zf(p['z'])} "+ f"{p['verdict']}" + (f", {p['cut']}" if p['cut'] else '') + ')' for p in picked)+ s = (f"Best moments for '{query}' in {os.path.basename(path)} ({how}): {segs}. start/end are sample trims in "+ 'seconds (strokes / events start 5 ms before the attack' ++ ('; windows are snapped to the sound' if prefer_ev else '; windows are plain stretches: crossfade them') ++ "; a 'head cut' / 'tail cut' trim cuts into a sound).")+ if picked[0]['verdict'] in ('weak', 'no match'):+ s += f" Even the best segment is {_weakly(picked[0]['verdict'])}: the recording may not contain this sound."+ else:+ s = f'No candidate segments in {os.path.basename(path)} ({how}).'+ pk = audio.db(np.abs(xs).max() if len(xs) else 0.0)+ if pk < SILENT_DBFS:+ s = f'WARNING: near-silent input (peak {pk:.0f} dBFS), scores are meaningless. ' + s+ return {'summary': s, 'path': path, 'query': query, 'query_kind': qkind, 'query_kind_score': qscore,+ 'duration': round(s1 - s0, 3), 'n_candidates': len(cands), 'n_windows': len(wins), 'n_events': len(evs),+ 'segments': picked}+++# ---------------------------------------------------------------------------------------------------- cli+def _label(x, dup):+ """file name, with its parent folder when another result has the same name"""+ return os.path.join(os.path.basename(os.path.dirname(x['path'])), x['name']) if x['name'] in dup else x['name']+++def _rank_text(r):+ lines = [r['summary'], '']+ names = [x['name'] for x in r['results']]+ dup = {n for n in names if names.count(n) > 1}+ for i, x in enumerate(r['results'], 1):+ lines.append(f"{i:3d}. z {vocab.zf(x['z']):>5} {x['verdict']:<9} cos {x['cosine']:.3f} {_label(x, dup)} "+ f"(sounds most like {x['sounds_most_like']})" + (' NEAR-SILENT' if x['near_silent'] else ''))+ return '\n'.join(lines)+++def _find_text(r):+ lines = [r['summary'], '']+ for i, p in enumerate(r['segments'], 1):+ lines.append(f"{i}. {p['start']:.3f}-{p['end']:.3f} s ({fmt_t(p['start'])}, {p['duration']:.3f} s) z {vocab.zf(p['z'])} "+ f"{p['verdict']:<9} {p['kind']:<6} peak {p['peak_dbfs']:.1f} dBFS head {p['head_db']:.0f} dB "+ f"tail {p['tail_db']:.0f} dB" + (f" (ranked {vocab.zf(p['score'])}: {p['cut']})" if p.get('cut') else ''))+ return '\n'.join(lines)+++def add_cli(sub):+ p = sub.add_parser('rank', help='rank sound files by how well they match a description')+ p.add_argument('query')+ p.add_argument('paths', nargs='+', help='files, folders or globs')+ p.add_argument('--top', type=int, help='list only the best N files (default: all)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(lambda: rank(a.paths, a.query, a.top), a, _rank_text))+ p = sub.add_parser('find', help='find the best moments for a description inside a long recording (sample trims)')+ p.add_argument('path')+ p.add_argument('query')+ p.add_argument('--top', type=int, default=5, help='number of non-overlapping segments (default 5)')+ p.add_argument('--win', type=float, help='window length in s (default: duration/6 clamped to 1..5)')+ p.add_argument('--hop', type=float, help='hop between windows in s (default: win/2)')+ p.add_argument('--kind', choices=['auto', 'event', 'texture'], default='auto',+ help='event = onset-based strokes / events only (hits, clicks, steps); texture = sliding windows '+ 'only (beds, ambiences); auto = read the query and score both')+ p.add_argument('--start', type=float, help='search from this time (s)')+ p.add_argument('--end', type=float, help='search up to this time (s)')+ common(p)+ p.set_defaults(func=lambda a: vocab.run_cli(+ lambda: find(a.path, a.query, a.win, a.hop, a.top, a.start, a.end, a.kind), a, _find_text))diff --git a/earshot/sheet.py b/earshot/sheet.pynew file mode 100644index 0000000..fc89fbd--- /dev/null+++ b/earshot/sheet.py@@ -0,0 +1,366 @@+"""An annotated picture of a soundtrack or film for agents that can look at images.++ earshot sheet soundtrack.wav -o sheet.png [--video film.mp4] [--cues cues.json] [--from 10 --to 25]+ earshot sheet film.mp4 -o film_sheet.png # a video: frames along the top, its cuts as lines++ from earshot.sheet import sheet+ sheet('film.mp4', 'film_sheet.png') # {'png': ..., 'summary': ...}++Top to bottom, on one shared time axis (m:ss): video frames (one per slot, taken just after a cut when the slot has+one; thin ticks show each frame's exact time), sections with their tags, event labels (kind + best CLAP tag,+staggered so they never overlap; pale with '?' = the event is under 12 dB over its bed, so the tag is a weak hint;+untagged onsets are short ticks), a log-frequency spectrogram (30 Hz - 16 kHz; digital silence draws black), the+waveform with momentary / short-term loudness (LUFS), the integrated level and gaps (yellow), then beats (tall =+downbeat guess) and cue / cut markers coloured by sync (green = an onset within a frame, amber = within 100 ms, blue =+a whoosh / riser peak, violet = only a level peak, red = none). Picture cuts are dashed lines through every lane. The+header repeats the key numbers (loudness, tempo, one sync line each for cuts and cues with their chance levels).+Built from earshot.timeline (its cached result is reused); --from/--to zooms in. An audio file's cover art is not a+video; a picture-only video draws its frames and cuts over silence.+"""+import os++import numpy as np++from . import audio+from .report import emit, fmt_t+from .timeline import DETAIL, timeline++SR = audio.SR+KIND_COLOR = {'WHOOSH': '#1f77b4', 'RISER': '#17becf', 'HIT': '#d62728', 'KEY': '#2ca02c', 'UI': '#8c564b',+ 'STEP': '#bcbd22', 'FOLEY': '#e377c2', 'NOTE': '#9467bd', 'DRUM': '#ff7f0e', 'VOICE': '#0b6e4f',+ '-': '#7f7f7f'}+SYNC_COLOR = {'OK': '#2ca02c', 'near': '#ff9f1c', 'swell': '#1f77b4', 'peak': '#9467bd', 'off': '#d62728',+ 'no sound': '#d62728'}+CUT_COLOR = '#00a6d6'+FONT = 'DejaVu Sans'+MONO = 'DejaVu Sans Mono'+++def _ticks(t0, t1, max_n=24):+ span = max(t1 - t0, 1e-3)+ step = next((s for s in (0.1, 0.25, 0.5, 1, 2, 5, 10, 15, 30, 60, 120, 300) if span / s <= max_n), 600)+ first = np.ceil(t0 / step) * step+ return np.arange(first, t1 + 1e-9, step), step+++def _tick_label(t, step):+ m = int(t // 60)+ s = t - 60 * m+ if step >= 1:+ return f'{m}:{int(round(s)):02d}'+ nd = 1 if abs(step * 10 - round(step * 10)) < 1e-9 else 2 # 0.25 s ticks need two decimals+ return f'{m}:{s:0{3 + nd}.{nd}f}'+++def _logspec(m, t0, n_cols, n_rows=240, fmin=30.0, fmax=16000.0):+ """log-frequency magnitude spectrogram (dB) with about n_cols time columns"""+ n_fft = 4096+ hop = max(64, int(len(m) / n_cols))+ y = np.pad(np.asarray(m, np.float32), (n_fft // 2, n_fft // 2))+ n = max(1, 1 + (len(y) - n_fft) // hop)+ fr = np.lib.stride_tricks.as_strided(y, (n, n_fft), (y.strides[0] * hop, y.strides[0]))+ f = np.fft.rfftfreq(n_fft, 1 / SR)+ edges = np.geomspace(fmin, fmax, n_rows + 2)+ lo, c, hi = edges[:-2, None], edges[1:-1, None], edges[2:, None]+ fb = np.maximum(0, np.minimum((f - lo) / (c - lo), (hi - f) / (hi - c)))+ fb[fb.sum(axis=1) == 0, np.argmin(np.abs(f[None, :] - c[fb.sum(axis=1) == 0]), axis=1)] = 1.0+ fb /= fb.sum(axis=1, keepdims=True)+ win = np.hanning(n_fft).astype(np.float32)+ out = np.empty((n_rows, n), np.float32)+ for a in range(0, n, 1024):+ P = np.abs(np.fft.rfft(fr[a:a + 1024] * win, axis=1)) ** 2+ out[:, a:a + 1024] = 10 * np.log10(fb.astype(np.float32) @ P.T + 1e-14)+ return out, edges[1:-1]+++def _thumb_times(t0, t1, cuts, n_slots):+ """one frame per slot: just after the first cut inside the slot, else the slot centre"""+ w = (t1 - t0) / n_slots+ out = []+ for k in range(n_slots):+ a, b = t0 + k * w, t0 + (k + 1) * w+ inside = [c for c in cuts if a <= c < b - 0.2]+ out.append(min(inside[0] + 0.15, b - 0.05) if inside else (a + b) / 2)+ return out+++def _place_labels(items, t0, t1, plot_px, char_px, n_rows):+ """greedy staggering: items [(priority, t, text)] -> [(row, t, text)] with no horizontal overlap in a row"""+ px_s = plot_px / max(t1 - t0, 1e-6)+ rows = [[] for _ in range(n_rows)]+ out = []+ for _, t, text in sorted(items, key=lambda q: q[0]):+ w = (len(text) * char_px + 16) / px_s+ a, b = t - 0.15 * w, t + 0.85 * w+ if b > t1:+ a, b = t1 - w, t1+ a = max(a, t0)+ for r, used in enumerate(rows):+ if all(b <= u0 or a >= u1 for u0, u1 in used):+ used.append((a, b))+ out.append((r, t, a, text))+ break+ return out+++def sheet(path, out_png, video=None, cues=None, start=None, end=None, detail='normal', width=2000, tl=None):+ """draw the annotated sheet PNG; returns {'png', 'summary', ...}. tl: a timeline() result to reuse"""+ import matplotlib+ matplotlib.use('Agg')+ import matplotlib.pyplot as plt+ from matplotlib import patches+ r = tl or timeline(path, video=video, cues=cues, start=start, end=end, detail=detail)+ video = r.get('video')+ t0, t1 = r['start'], r['end']+ has_audio = r.get('has_audio', True)+ m = audio.to_mono(audio.load(path, start=t0, end=t1)) if has_audio else np.zeros(int((t1 - t0) * SR), np.float32)+ width = int(min(max(width, 1000), 2400))+ dpi = 100+ fs = max(11.0, width / 125) # base font size (pt)+ left_px, right_px = max(92, int(6.4 * fs) + 8), 64+ plot_px = width - left_px - right_px+ lf = fs * 0.85 # event label font size (pt, monospace)+ char_px = 0.602 * lf * dpi / 72+ lanes = []+ if video:+ lanes.append(('frames', 1.55))+ lanes += [('sections', 0.6), ('labels', 1.55), ('spec', 3.6), ('wave', 0.95), ('loud', 1.75), ('beats', 0.75)]+ lo, mu = r['loudness'], r['music']+ sub = []+ if lo.get('integrated_lufs') is not None:+ sub.append(f"{lo['integrated_lufs']:.1f} LUFS TP {lo['true_peak_dbtp']:.1f} dBTP" ++ (f" LRA {lo['lra_lu']:.1f} LU" if lo.get('lra_lu') is not None else ''))+ elif lo.get('digital_silence'):+ sub.append('digital silence' if r.get('has_audio', True) else 'no audio stream')+ if mu.get('tempo') and mu['confidence'] != 'none':+ sub.append(f"~{mu['tempo']:.0f}" + (f" (or {mu['tempo_alt']:.0f})" if mu.get('tempo_alt') else '') ++ f" BPM ({mu['confidence']})")+ sub.append(f"{len(r['events'])} events")+ lines = [(os.path.basename(path) + f" {fmt_t(t0)}-{fmt_t(t1)}", fs * 1.25, 'bold', '#000000'),+ (' '.join(sub), fs * 0.95, 'normal', '#333333')]+ for kind in ('cuts', 'cues'):+ st = (r['sync'].get(kind) or {}).get('stats') or {}+ if st.get('n'):+ k1 = 'hit_within_1_frame' if 'hit_within_1_frame' in st else 'hit_within_40ms'+ ka = k1.replace('hit', 'accent')+ acc = (f", {100 * st[ka]:.0f}% on a strong accent (chance "+ f"{100 * st[ka.replace('accent', 'accent_chance')]:.0f}%)" if ka in st else '')+ pk = f", {st['answered_by_level_peak']} only a level peak" if st.get('answered_by_level_peak') else ''+ lines.append((f"sync {st['n']} {kind}: {100 * st[k1]:.0f}% onset within "+ f"{'1 frame' if 'frame' in k1 else '40 ms'} (chance "+ f"{100 * st.get(k1.replace('hit', 'chance'), 0):.0f}%){acc}{pk}, "+ f"{len(st['no_sound'])} without a hit", fs * 0.95, 'normal', '#333333'))+ if r['sync']:+ shapes = ', '.join(n for k, n in (('cues', 'v cue'), ('cuts', 'diamond cut')) if r['sync'].get(k))+ lines.append((f'markers ({shapes}): green = an onset within 1 frame, amber = within 100 ms, blue = a whoosh / '+ 'riser peak, violet = only a level peak (soft whoosh?), red = no hit.' ++ (' Dashed lines = picture cuts.' if r.get('cuts') else ''), fs * 0.72, 'normal', '#555555'))+ if any(e.get('label') for e in r['events']):+ lines.append(("event labels: kind + CLAP tag, hints only; pale + '?' = under 12 dB over the bed (mostly bed in "+ 'the excerpt): a weak hint.', fs * 0.72, 'normal', '#555555'))+ head = 0.22 + sum(sz * 1.45 / 72 for _, sz, _, _ in lines)+ fig_h = round((head + sum(h for _, h in lanes) + 0.55) * dpi) / dpi+ fig = plt.figure(figsize=(width / dpi, fig_h), dpi=dpi)+ fig.patch.set_facecolor('white')+ axes, y = {}, fig_h - head+ for name, h in lanes:+ y -= h+ axes[name] = fig.add_axes([left_px / width, y / fig_h, plot_px / width, (h - 0.06) / fig_h])+ axes[name].set_xlim(t0, t1)+ ticks, step = _ticks(t0, t1)+ for name, ax in axes.items():+ ax.set_xticks(ticks)+ ax.tick_params(labelsize=fs * 0.85, length=3)+ ax.set_xticklabels([] if name != 'beats' else [_tick_label(t, step) for t in ticks], fontfamily=FONT)+ for sp in ax.spines.values():+ sp.set_linewidth(0.6)+ sp.set_color('#999999')++ def lane_title(ax, text, inside=False):+ if inside:+ ax.text(0.004, 0.94, text, transform=ax.transAxes, ha='left', va='top', fontsize=fs * 0.78, zorder=6,+ fontfamily=FONT, color='#222222', bbox=dict(boxstyle='round,pad=0.2', fc='white', ec='none', alpha=0.8))+ else:+ ax.text(-0.006, 0.5, text, transform=ax.transAxes, ha='right', va='center', fontsize=fs * 0.82,+ fontfamily=FONT, color='#333333')++ yy = 0.12+ for text, size, weight, color in lines:+ yy += size * 0.725 / 72+ fig.text(left_px / width, 1 - yy / fig_h, text, fontsize=size, fontfamily=FONT, weight=weight, va='center',+ color=color)+ yy += size * 0.725 / 72+ n_thumbs = 0+ if video:+ from . import video as vid+ ax = axes['frames']+ ax.set_yticks([])+ ax.set_ylim(0, 1)+ info = vid.info(video)+ aspect = (info.get('width') or 16) / float(info.get('height') or 9)+ th_px = 1.55 * dpi - 18+ tw_px = th_px * aspect+ n_slots = max(1, int(plot_px // (tw_px + 6)))+ times = _thumb_times(t0, t1, r.get('cuts') or [], n_slots)+ slot = (t1 - t0) / n_slots+ for k, t in enumerate(times):+ im = vid.frame(video, t, width=int(tw_px * 1.5))+ if im is None:+ continue+ a = t0 + k * slot + (slot - tw_px / plot_px * (t1 - t0)) / 2+ ax.imshow(np.asarray(im), extent=(a, a + tw_px / plot_px * (t1 - t0), 0.12, 1.0), aspect='auto',+ zorder=1)+ ax.plot([t, t], [0, 0.12], color='#333333', lw=1.2, zorder=2)+ n_thumbs += 1+ ax.set_xlim(t0, t1)+ lane_title(ax, 'frames')++ ax = axes['sections']+ ax.set_yticks([])+ ax.set_ylim(0, 1)+ for k, s in enumerate(r['sections']):+ a, b = max(s['start'], t0), min(s['end'], t1)+ ax.add_patch(patches.Rectangle((a, 0.05), b - a, 0.9, fc=('#e8eef7', '#f4ead9')[k % 2], ec='#8899aa', lw=0.8))+ n_chars = int((b - a) / (t1 - t0) * plot_px / (0.55 * fs * 0.9 * dpi / 72)) - 1+ lab = s['label'] if len(s['label']) <= n_chars else s['label'][:max(0, n_chars - 2)] + '..'+ if n_chars >= 4:+ ax.text(a + (t1 - t0) * 4 / plot_px, 0.5, lab, va='center', ha='left', fontsize=fs * 0.9, fontfamily=FONT,+ clip_on=True)+ lane_title(ax, 'sections')++ ax = axes['labels']+ ax.set_ylim(0, 1)+ ax.set_yticks([])+ ax.axis('off')+ n_rows = 4+ near = {it.get('event_t') for k in ('cuts', 'cues') for it in (r['sync'].get(k) or {}).get('items', [])}+ items = []+ for e in r['events']:+ if e.get('label'):+ pri = (0 if e['t'] in near else 1, -e['prominence_db'])+ items.append((pri, e['t'], f"{e['kind']}{'' if e.get('exposed') else '?'} {e['label']}"))+ placed = _place_labels(items, t0, t1, plot_px, char_px, n_rows)+ for row, t, a, text in placed:+ yy = 1 - (row + 0.5) / n_rows+ kind = text.split(' ', 1)[0]+ col, alpha = KIND_COLOR.get(kind.rstrip('?'), '#777777'), 0.55 if kind.endswith('?') else 1.0+ ax.text(a, yy, text, fontsize=lf, fontfamily=MONO, va='center', ha='left', color='white',+ bbox=dict(boxstyle='square,pad=0.15', fc=col, ec='none', alpha=alpha), clip_on=False, zorder=3)+ ax.plot([t, t], [0, yy - 0.5 / n_rows + 0.02], color=col, lw=1.0, alpha=alpha, zorder=2)+ lane_title(axes['labels'], 'events')+ xk = 1 - right_px / width+ for k in sorted({text.split(' ', 1)[0].rstrip('?') for _, _, _, text in placed}, reverse=True):+ fig.text(xk, 1 - 0.32 / fig_h, k, fontsize=fs * 0.85, fontfamily=MONO, ha='right', va='center',+ color='white', bbox=dict(boxstyle='round,pad=0.25', fc=KIND_COLOR.get(k, '#777777'), ec='none'))+ xk -= (len(k) * char_px * 0.9 + 22) / width++ ax = axes['spec']+ S, fc = _logspec(m, t0, plot_px)+ top = max(float(np.percentile(S, 99.7)), -40.0) # -40 dB here ~ a -100 dBFS tone: silence draws black+ ax.imshow(S, origin='lower', aspect='auto', cmap='magma', vmin=top - 75, vmax=top,+ extent=(t0, t1, 0, len(fc)), interpolation='bilinear')+ yt = [50, 100, 200, 500, 1000, 2000, 5000, 10000]+ ax.set_yticks([float(np.interp(np.log(f), np.log(fc), np.arange(len(fc)))) for f in yt])+ ax.set_yticklabels(['50', '100', '200', '500', '1k', '2k', '5k', '10k'], fontsize=fs * 0.8, fontfamily=FONT)+ for e in r['events']:+ h = len(fc) * (0.07 if e.get('label') else 0.035)+ ax.plot([e['t'], e['t']], [len(fc) - h, len(fc)], color='white', lw=1.0 if e.get('label') else 0.6,+ alpha=0.9 if e.get('label') else 0.55)+ ax.set_ylim(0, len(fc))+ lane_title(ax, 'spectrogram (Hz, log)', inside=True)++ ax = axes['wave']+ n = len(m)+ edges = np.linspace(0, n, min(plot_px, max(1, n)) + 1).astype(int)+ seg_max = np.maximum.reduceat(m, edges[:-1]) if n else np.zeros(1)+ seg_min = np.minimum.reduceat(m, edges[:-1]) if n else np.zeros(1)+ pk = float(np.abs(m).max()) if n else 0.0+ sc = max(float(np.percentile(np.abs(m), 99.9)) if n else 0.0, 1e-9)+ ax.fill_between(t0 + edges[:-1] / SR, np.clip(seg_min / sc, -1, 1), np.clip(seg_max / sc, -1, 1), color='#6b7a90',+ lw=0, step='post')+ ax.set_ylim(-1.08, 1.08)+ ax.set_yticks([])+ lane_title(ax, f'waveform (full height = {audio.db(sc):.0f} dBFS, peak {audio.db(pk):.1f} dBFS)' if pk > 0 else+ 'waveform (digital silence)', inside=True)++ ax = axes['loud']+ lo_y = -60.0+ if lo.get('momentary'):+ mm = lo['momentary']+ tt = mm['t0'] + mm['dt'] * np.arange(len(mm['lufs']))+ ax.plot(tt, np.maximum(mm['lufs'], lo_y), color='#3d6fb6', lw=1.0, label='momentary')+ if lo.get('short_term'):+ ss = lo['short_term']+ tt = ss['t0'] + ss['dt'] * np.arange(len(ss['lufs']))+ ax.plot(tt, np.maximum(ss['lufs'], lo_y), color='#0b2e66', lw=2.4, label='short-term')+ if lo.get('integrated_lufs') is not None and lo['integrated_lufs'] > lo_y:+ ax.axhline(lo['integrated_lufs'], color='#d1495b', lw=1.2, ls='--')+ ax.text(t1, lo['integrated_lufs'], f" {lo['integrated_lufs']:.1f}", color='#d1495b', fontsize=fs * 0.8,+ va='center', ha='left', fontfamily=FONT, clip_on=False)+ for g in lo.get('silences', []):+ ax.axvspan(g['start'], g['end'], color='#ffdd57', alpha=0.45, lw=0)+ for c in lo['clipping']['times']:+ ax.plot([c, c], [-3, 0], color='red', lw=2)+ ax.set_ylim(lo_y, 0)+ ax.set_yticks([-50, -40, -30, -20, -10])+ ax.tick_params(axis='y', labelsize=fs * 0.8)+ ax.legend(loc='lower left', frameon=False, ncol=2, prop={'family': FONT, 'size': fs * 0.78})+ lane_title(ax, 'loudness (LUFS)', inside=True)++ ax = axes['beats']+ ax.set_ylim(0, 1)+ ax.set_yticks([])+ beats = mu.get('beats') or []+ if beats and mu.get('confidence') != 'none':+ db = set(mu.get('downbeats') or [])+ for b in beats:+ if t0 <= b <= t1:+ ax.plot([b, b], [0.0, 0.45 if b in db else 0.28], color='#222222' if b in db else '#666666', lw=1.0)+ for it in (r['sync'].get('cues') or {}).get('items', []):+ ax.plot([it['t']], [0.84], marker='v', ms=fs * 0.55, color=SYNC_COLOR.get(it['status'], '#777777'), mew=0)+ for it in (r['sync'].get('cuts') or {}).get('items', []):+ ax.plot([it['t']], [0.6], marker='D', ms=fs * 0.45, color=SYNC_COLOR.get(it['status'], '#777777'), mew=0)+ lane_title(ax, 'sync /\nbeats', inside=False)+ ax.set_xlabel('time (m:ss)', fontsize=fs * 0.85, fontfamily=FONT)++ for c in r.get('cuts') or []:+ for name, a in axes.items():+ if name != 'labels' and t0 <= c <= t1:+ a.axvline(c, color=CUT_COLOR, lw=1.1, ls=(0, (4, 3)), alpha=0.9, zorder=0.5 if name == 'frames' else 4)+ d = os.path.dirname(os.path.abspath(out_png))+ os.makedirs(d, exist_ok=True)+ fig.savefig(out_png, dpi=dpi, facecolor='white')+ plt.close(fig)+ w_px, h_px = int(width), int(round(fig_h * dpi))+ res = {'png': os.path.abspath(out_png), 'width': w_px, 'height': h_px, 'start': t0, 'end': t1,+ 'thumbnails': n_thumbs, 'labelled_events': len(placed), 'tagged_events': len(items),+ 'events': len(r['events']), 'cuts': len(r.get('cuts') or []), 'timeline_summary': r['summary']}+ res['summary'] = (f"Wrote {res['png']} ({w_px}x{h_px}): {fmt_t(t0)}-{fmt_t(t1)}, {n_thumbs} frames, "+ f"{len(placed)} of {len(items)} tagged events labelled, {len(r['events'])} event ticks, "+ f"{res['cuts']} cut lines. Open it with an image viewer / Read tool."+ ).encode('ascii', 'backslashreplace').decode()+ res['text'] = res['summary'] + '\n\nTimeline: ' + r['summary']+ return res+++def add_cli(sub):+ from .cli import common+ p = sub.add_parser('sheet', help='annotated PNG of a soundtrack / film: frames, spectrogram, events, loudness, sync')+ p.add_argument('file', help='audio file or video')+ p.add_argument('-o', dest='png', help='output PNG (default: <file stem>_sheet.png in the current folder)')+ p.add_argument('--video', help='video for frames and cuts (default: FILE itself when it is a video)')+ p.add_argument('--cues', help='cue events: JSON file {fps, events:[{f, kind, id}]}, a JSON list, or "1.5,3.2,..." s')+ p.add_argument('--from', '--start', dest='start', type=float, help='zoom: start (s)')+ p.add_argument('--to', '--end', dest='end', type=float, help='zoom: end (s)')+ p.add_argument('--detail', choices=list(DETAIL), default='normal', help='timeline model budget')+ p.add_argument('--width', type=int, default=2000, help='image width in px (1000-2400)')+ common(p)++ def run(a):+ png = a.png or os.path.splitext(os.path.basename(a.file))[0] + '_sheet.png'+ return emit(sheet(a.file, png, a.video, a.cues, a.start, a.end, a.detail, a.width), a) and 0+ p.set_defaults(func=run)+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/timeline.py b/earshot/timeline.pynew file mode 100644index 0000000..90377b1--- /dev/null+++ b/earshot/timeline.py@@ -0,0 +1,1460 @@+"""Listen through a whole soundtrack or film like a sound supervisor: events, sections, loudness, beat grid, sync.++ earshot timeline soundtrack.wav [--video film.mp4] [--cues cues.json] [--from 10 --to 40] [--detail low|normal|high]+ earshot timeline film.mp4 # a video: its audio is analysed and its own picture cuts are used++ from earshot.timeline import timeline+ r = timeline('film.mp4', cues='cues.json')+ print(r['text']) # the listening transcript (ASCII, chronological)+ r['events'], r['sections'], r['loudness'], r['music'], r['sync']++What the numbers mean (and how far to trust them; measured in eval/timeline/RESULTS.md):+ events onsets from a spectral-flux detector (5 ms frames), refined to the attack on a 1 ms envelope (the first+ millisecond within 20 dB of the event's peak above the bed, like an editor aligning a sample). On a+ finished film mix (Clawd) 93% of placed sound effects are found within 30 ms (65% of those more than+ 6 dB under the music + ambience), median |timing error| 0.5 ms. Of the detections 51% are placed+ effects, 81% placed effects or music / ambience onsets, 96% counting the extra hits inside multi-hit+ effect samples. A file that starts with sound has an event at its first millisecond.+ prominence_db = rise of the K-weighted level over the bed just before (broadband, perceptual; bed_db =+ that bed, ~LUFS, floored at -80, so a hit after silence shows a huge prominence: the transcript prints+ 'silent' in its 'over bed' column when the bed is below -60, and the summary's strongest accents only+ count events over a running bed, each also listed in the transcript);+ contrast_db = the same in the band where it stands out most (band: full / low < 200 Hz / high > 2 kHz);+ z = onset novelty in robust SDs vs the surrounding 2 s. 'swell' events are build-ups timed at their peak.+ tags CLAP guesses for the event (compact event vocabulary, repeat-padded excerpt; phrases that score high on+ any short sound are damped by their mean over 536 real background events). They are hints: on the+ tuning mixes the right kind came first for 1 in 4 events masked by their bed and 1 in 2 events at least+ 12 dB over it ('exposed'; the transcript marks the others with '?'), and no score we tried predicts a+ right tag better. On Clawd: key presses 58%, whooshes 56%, grass footsteps 0%; false KEY / UI tags on+ its ambience stem 35%, music stem 5%. Where AST runs (the most important events), AST speech >= 0.5+ overrides the kind with VOICE.+ sections CLAP windows merged while they sound alike, split at strong timbre / level changes (Foote novelty);+ genre / mood / instrument tags from CLAP ('?' = weak genre; a mood only when it stands out, z >= 2.5:+ busy sound effects over music push weaker moods toward 'dark / aggressive'), extra sources and Music /+ Speech from AST; neighbours with the same genre and sources are merged.+ loudness BS.1770: momentary (0.4 s) / short-term (3 s) LUFS (floored at the -70 LUFS gate), integrated LUFS (None+ when all is gated out), LRA, true peak (4x oversampled; None for digital silence); gaps touching the+ start / end of the file are a lead-in / tail, not a stop-down.+ music tempo + beats (own autocorrelation / dynamic-programming tracker), downbeat guess, pulse clarity;+ tempo_alt = the double tempo when it pulses nearly as clearly (the felt tempo may be either).+ sync each cut / cue -> the onset that answers it: offset_ms = sound - picture (negative = sound early);+ OK = within a frame (or 40 ms), near = within 100 ms, swell = a detected build-up peaks within 300 ms,+ peak = no onset, only a level peak within 150 ms (a soft whoosh / swell, or just the bed: check it),+ off / no sound = 'without a hit'. Build-ups are hard: on Clawd 2 sounded whooshes are still listed+ without a hit and 1 unsounded cue is answered by a level peak. 'chance' = what random edit times would+ score with this event density, so a 60% hit rate means little when chance is 50%. accent_* = the same+ for strong accents only (top 20% by prominence and >= 6 dB over the bed): the number to read for a+ dense mix, where some onset is always near.+--from/--to (or --start/--end) zooms in: the zoom is analysed inside +-30 s of context, so its verdicts match the full+analysis. True peak on decoded lossy audio (AAC / MP3 / Opus) includes the codec's overshoot: up to 0 dBTP is normal for+a delivered file (a master should leave the mixer at <= -1 dBTP before encoding); above 0 dBTP it clips on playback.+Model time is budgeted by --detail (CLAP ~0.3-1 s and AST several s per window on CPU): events near a cut / cue+first, spread over the timeline, one per burst, then the most prominent. Results are cached (EARSHOT_HOME/emb/timeline,+keyed on the file, the options, this module and the core decode / features / cut code); --fresh recomputes+everything and stores the new result.+"""+import hashlib+import json+import os+import time++import numpy as np+from scipy import ndimage, signal++from . import audio, config, features+from .report import emit, fmt_t, to_json++SR = audio.SR+SPAN_VERSION = 1 # bump when an excerpt / embedding definition changes (per-span model cache)+CODE_ID = hashlib.sha1(open(__file__, 'rb').read()).hexdigest()[:12] # cached results never outlive a code change+HOP = 240 # onset-detector frame hop: 5 ms+DETAIL = { # events CLAP-tagged / AST-labelled; section window, hop (s, grows to keep <= max_win+ # windows on long files); AST section crops+ 'low': {'tag': 12, 'ast': 0, 'win': 4.0, 'hop': 4.0, 'max_win': 30, 'ast_sec': 0},+ 'normal': {'tag': 20, 'ast': 1, 'win': 3.0, 'hop': 2.0, 'max_win': 32, 'ast_sec': 3},+ 'high': {'tag': 96, 'ast': 16, 'win': 3.0, 'hop': 1.5, 'max_win': 200, 'ast_sec': 12},+}+EVENT_VOCAB = { # compact event vocabulary: kind -> caption-like phrases (tuned on synthetic mixes)+ 'WHOOSH': ['a fast air whoosh', 'a swoosh passing by', 'a short swish', 'a whip pan whoosh transition'],+ 'RISER': ['a rising riser building tension', 'a reverse cymbal swell', 'an uplifter sweep'],+ 'HIT': ['a deep cinematic boom', 'a punchy impact hit', 'a heavy thud', 'a sub bass drop', 'a soft thump'],+ 'KEY': ['a mechanical keyboard key press', 'typing on a computer keyboard', 'a laptop key click',+ 'a spacebar press'],+ 'UI': ['a mouse click', 'a soft UI button tap', 'a notification ding', 'a chat message pop',+ 'a toggle switch click', 'a camera shutter click'],+ 'STEP': ['footsteps on grass', 'footsteps on wood', 'a small soft footstep'],+ 'FOLEY': ['a plastic click', 'lego bricks clicking', 'a wooden block hit', 'glass clinking', 'paper rustling',+ 'leaves rustling', 'a cork pop', 'a bubble pop', 'a water drop', 'a small object dropping on a table'],+ 'NOTE': ['a piano note', 'a bell chime', 'a plucked string note', 'an orchestral stab', 'a celesta note',+ 'a glockenspiel note'],+ 'DRUM': ['a kick drum', 'a snare drum hit', 'a hi-hat', 'a drum hit'],+ 'VOICE': ['a spoken word', 'a person talking', 'a breath', 'a short vocal sound'],+}+TAG_BG = { # mean cosine of each event phrase over 536 real background events (tag_bg.py)+ 'a fast air whoosh': -0.0137, 'a swoosh passing by': -0.0271, 'a short swish': 0.0538,+ 'a whip pan whoosh transition': 0.0173, 'a rising riser building tension': -0.0403,+ 'a reverse cymbal swell': -0.0354, 'an uplifter sweep': -0.0522, 'a deep cinematic boom': -0.0294,+ 'a punchy impact hit': 0.0361, 'a heavy thud': 0.0383, 'a sub bass drop': -0.0194, 'a soft thump': -0.0344,+ 'a mechanical keyboard key press': 0.1109, 'typing on a computer keyboard': -0.0065,+ 'a laptop key click': 0.1327, 'a spacebar press': 0.1456, 'a mouse click': 0.0743,+ 'a soft UI button tap': 0.0578, 'a notification ding': 0.0487, 'a chat message pop': 0.1634,+ 'a toggle switch click': 0.1030, 'a camera shutter click': 0.0964, 'footsteps on grass': -0.0284,+ 'footsteps on wood': -0.0025, 'a small soft footstep': 0.0282, 'a plastic click': 0.0729,+ 'lego bricks clicking': -0.0168, 'a wooden block hit': 0.0335, 'glass clinking': -0.0419,+ 'paper rustling': -0.0469, 'leaves rustling': -0.0228, 'a cork pop': 0.0633, 'a bubble pop': 0.0977,+ 'a water drop': 0.1012, 'a small object dropping on a table': 0.0316, 'a piano note': 0.0143,+ 'a bell chime': -0.0081, 'a plucked string note': 0.0687, 'an orchestral stab': 0.0640,+ 'a celesta note': 0.0378, 'a glockenspiel note': 0.0350, 'a kick drum': 0.0659, 'a snare drum hit': 0.0322,+ 'a hi-hat': 0.0719, 'a drum hit': 0.0758, 'a spoken word': -0.0140, 'a person talking': -0.1026,+ 'a breath': 0.0520, 'a short vocal sound': 0.0318}+TAG_TEMPLATES = ('{}', 'the sound of {}', 'a recording of {}', '{} sound effect', 'an audio clip of {}')+TAG_CENTER = 0.5 # share of each phrase's background mean removed before ranking (hub phrases)+EXPOSED_DB = 12.0 # an event this far over its bed dominates its excerpt: its tag is right ~2x as often+BURST_S = 0.5 # one CLAP tag per burst: same-band events this close share the budget+CONTEXT_S = 30.0 # a zoom is analysed inside +-30 s of context (sync verdicts, accents, beat grid)+SECTION_CATS = ('music_genre', 'music_mood', 'instruments', 'ambience', 'voice')+Z_OK, Z_WEAK = 2.5, 1.5+NEAR_S = 0.10 # an event within 100 ms of a cut / cue is "near" it and always tagged+SYNC_S = 0.25 # search radius for the event that answers a cut / cue+ON_BEAT_S = 0.05+ACCENT_DB = 6.0 # an accent (deliberate hit) is >= 6 dB over its bed and in the file's top 20%+SWELL_RISE_DB = 8.0 # whoosh / riser build-up: minimum rise (dB)+PEAK_S, PEAK_RISE_DB = 0.15, 3.0 # a cut / cue without an onset: level peak within 150 ms rising >= 3 dB (swell_tune)+NOVELTY_Z = 3.0 # section change: Foote novelty robust z (tuned on synthetic bed switches)+CLEAR_PULSE, WEAK_PULSE = 0.30, 0.15 # pulse clarity thresholds (calibrated on music vs ambience beds)+BED_FLOOR = 10 ** (-80 / 20) # beds quieter than -80 dBFS count as -80 (keeps prominence finite)+SILENT_BED_DB = -60.0 # a bed below this is a gap: the rise over it ('over bed') is not shown as a number+++# ------------------------------------------------------------------ events (no models)+def _band_envs(x):+ """1 ms max-abs envelopes of the full signal, its lows (< 200 Hz) and highs (> 2 kHz); the band filters run on a+ reflected 100 ms lead-in so they start without a transient"""+ m = audio.to_mono(x).astype(np.float64)+ w = SR // 1000+ n = len(m) // w+ pad = min(len(m) - 1, SR // 10)+ mp = np.concatenate([m[pad:0:-1], m])++ def env(y):+ return np.abs(y[: n * w]).reshape(n, w).max(axis=1) + 1e-7+ bands = {'full': np.abs(x[: n * w]).reshape(n, w, -1).max(axis=(1, 2)) + 1e-7 if x.ndim == 2 else env(m)}+ for kind, f in (('low', 200), ('high', 2000)):+ bands[kind] = env(signal.sosfilt(signal.butter(4, f, kind, fs=SR, output='sos'), mp)[pad:])+ return bands+++def _mel_fb(n_fft, n_mels, fmin, fmax):+ """triangular mel filterbank (HTK mel scale), (n_mels, n_fft // 2 + 1)"""+ def mel(f):+ return 2595 * np.log10(1 + f / 700.0)+ edges = 700 * (10 ** (np.linspace(mel(fmin), mel(fmax), n_mels + 2) / 2595) - 1)+ f = np.fft.rfftfreq(n_fft, 1 / SR)+ lo, c, hi = edges[:-2, None], edges[1:-1, None], edges[2:, None]+ return np.maximum(0, np.minimum((f - lo) / (c - lo), (hi - f) / (hi - c)))+++def log_mel(m, n_fft=1024, hop=HOP, n_mels=96, fmin=30, fmax=16000):+ """log-mel power spectrogram (dB), frame i centred at i * hop / SR"""+ y = np.pad(np.asarray(m, np.float32), (n_fft // 2, n_fft // 2), mode='reflect' if len(m) > n_fft else 'constant')+ n = 1 + (len(y) - n_fft) // hop+ fr = np.lib.stride_tricks.as_strided(y, (n, n_fft), (y.strides[0] * hop, y.strides[0]))+ out = np.empty((n_mels, n), np.float32)+ fb, win = _mel_fb(n_fft, n_mels, fmin, fmax).astype(np.float32), np.hanning(n_fft).astype(np.float32)+ for a in range(0, n, 4096):+ P = np.abs(np.fft.rfft(fr[a:a + 4096] * win, axis=1)) ** 2+ out[:, a:a + 4096] = 10 * np.log10(fb @ P.T + 1e-12)+ return out+++def onset_curve(m):+ """superflux onset strength (mean positive log-mel increase, dB per frame) at 5 ms frames"""+ L = log_mel(m)+ L = np.maximum(L, L.max() - 80)+ d = L[:, 2:] - ndimage.maximum_filter1d(L, 3, axis=0)[:, :-2]+ return np.concatenate([np.zeros(2), np.maximum(d, 0).mean(axis=0)])+++def _robust_z(v, size):+ med = ndimage.median_filter(v, size, mode='nearest')+ mad = ndimage.median_filter(np.abs(v - med), size, mode='nearest') * 1.4826+ pos = v[v > 0]+ floor = 0.02 * float(np.percentile(pos, 99)) if len(pos) else 1e-6+ return (v - med) / (mad + floor + 1e-9)+++def _attack(envs, tc):+ """refine a detector frame time to the attack: find the band where the event stands out most above its bed,+ then walk back from the event's peak to the first millisecond within 20 dB of it (above the bed); the timing+ comes from the full-band envelope whenever the event rises 12 dB there too (filtered bands smear time)"""+ ic = int(round(tc * 1000))+ res = {}+ for band, e in envs.items():+ a, b = max(0, ic - 15), min(len(e), ic + 50)+ if b - a < 5:+ continue+ ip = a + int(np.argmax(e[a:b]))+ pre = e[max(0, ic - 160): max(0, ic - 40)]+ bed = max(float(np.median(pre if len(pre) >= 20 else e[: ic + 250])), BED_FLOOR)+ thr = bed + 0.1 * (e[ip] - bed)+ j, lo = ip, max(0, ic - 40)+ while j > lo and e[j - 1] >= thr:+ j -= 1+ res[band] = (e[ip] / bed, band, j, ip, bed)+ if not res:+ return None+ best = max(res.values())+ timing = res['full'] if 'full' in res and res['full'][0] >= 4.0 else best+ return best[0], best[1], timing+++def _length_ms(e, i0, ip, bed):+ """from the attack until the band envelope falls 20 dB below the event peak or back to 3 dB over its bed+ (and stays there for 30 ms); capped at 2 s (a sustained note or a bed change)"""+ floor = max(e[ip] * 0.1, bed * 1.41)+ seg = e[ip: ip + 2000] < floor+ run = np.convolve(seg, np.ones(30, int), 'valid') == 30+ k = int(np.argmax(run)) if run.any() else len(seg)+ return int(min(ip + k - i0, 2000))+++def _excess_centroid(m, t, length_s):+ """spectral centroid of what the event adds over the bed just before it (Hz)"""+ a, n = int(t * SR), int(min(max(length_s, 0.03), 0.12) * SR)+ seg = m[a: a + n]+ pre = m[max(0, a - int(0.15 * SR)): max(0, a - int(0.01 * SR))]+ if len(seg) < 256:+ return None+ nf = 4096+ f = np.fft.rfftfreq(nf, 1 / SR)+ P = np.abs(np.fft.rfft(seg * np.hanning(len(seg)), nf)) ** 2 / len(seg)+ if len(pre) >= 256:+ Q = np.abs(np.fft.rfft(pre * np.hanning(len(pre)), nf)) ** 2 / len(pre)+ X = np.maximum(P - Q, 0)+ if X.sum() > 0.2 * P.sum():+ P = X+ k = (f >= 20) & (f <= 20000)+ return int((f[k] * P[k]).sum() / (P[k].sum() + 1e-30))+++def _kw_env_db(x):+ """K-weighted 10 ms power (dB) at 1 ms steps: index i = window starting at i ms"""+ y = features._kw(x if x.ndim == 2 else x[:, None]).astype(np.float64)+ p = np.concatenate([[0.0], np.cumsum((y ** 2).sum(axis=1))])+ st = np.arange(0, max(1, len(y) - 480), 48)+ return 10 * np.log10((p[np.minimum(st + 480, len(y))] - p[st]) / 480 + 1e-12)+++def detect_events(x, zthr=3.0, min_contrast_db=3.0, min_gap=0.03, lead=True):+ """transient onsets + loudness swells of a 48 kHz array -> list of event dicts (times relative to x); lead: x+ starts where the file does, so sound from the first millisecond is an event too (nothing precedes it)"""+ m = audio.to_mono(x).astype(np.float32)+ if len(m) < int(0.1 * SR):+ return []+ odf = onset_curve(m)+ z = _robust_z(odf, 401)+ loc = (odf == ndimage.maximum_filter1d(odf, 11)) & (z >= zthr) & (odf > 0)+ envs = _band_envs(x)+ lm_t, lm = features.loudness_curve(x, 0.4, 0.05) if len(m) > 0.4 * SR else (np.array([0.0]), np.array([-70.0]))+ out = []+ for i in np.where(loc)[0]:+ att = _attack(envs, i * HOP / SR)+ if att is None or 20 * np.log10(att[0]) < min_contrast_db:+ continue+ ratio, band, (_, tband, j, ip, bed) = att+ out.append({'t': j / 1000, 'shape': 'hit', 'band': band, 'contrast_db': min(20 * np.log10(ratio), 60.0),+ 'z': min(float(z[i]), 99.0), 'length_ms': _length_ms(envs[tband], j, ip, bed)})+ out.sort(key=lambda e: e['t'])+ merged = []+ for e in out:+ if merged and e['t'] - merged[-1]['t'] < min_gap:+ if e['contrast_db'] > merged[-1]['contrast_db']:+ merged[-1] = e+ continue+ merged.append(e)+ full = envs['full']+ if lead and len(full) > 5 and full[:50].max() >= 1e-3 and not any(e['t'] < 0.05 for e in merged):+ ip = int(np.argmax(full[:50]))+ j = int(np.argmax(full[:ip + 1] >= 0.1 * full[ip]))+ merged.insert(0, {'t': j / 1000, 'shape': 'hit', 'band': 'full', 'contrast_db': 60.0, 'z': 0.0,+ 'length_ms': _length_ms(full, j, ip, BED_FLOOR), 'starts_file': True})+ kw = _kw_env_db(x)+ for e in merged:+ j = int(round(e['t'] * 1000))+ pre = kw[max(0, j - 300): max(0, j - 20)]+ bed = -80.0 if e.get('starts_file') else max(float(np.median(pre)) if len(pre) >= 20 else+ float(np.percentile(kw[: j + 2000], 10)), -80.0)+ e['prominence_db'] = max(0.0, float(kw[j: j + 80].max()) - bed) if j < len(kw) else 0.0+ e['bed_db'] = round(bed, 1)+ merged += _swells(x, merged)+ merged.sort(key=lambda e: e['t'])+ xs = x if x.ndim == 2 else x[:, None]+ for e in merged:+ a = int(e['t'] * SR)+ b = a + int(max(0.05, min(e['length_ms'] / 1000, 0.3)) * SR)+ e['peak_dbfs'] = round(audio.db(np.abs(xs[a:max(b, a + 1)]).max()), 1)+ e['local_lufs'] = round(float(np.interp(e['t'], lm_t, lm)), 1)+ e['centroid_hz'] = _excess_centroid(m, e['t'], e['length_ms'] / 1000)+ e['t'] = round(float(e['t']), 3)+ e['prominence_db'] = round(float(e['prominence_db']), 1)+ e['contrast_db'] = round(float(e['contrast_db']), 1)+ e['z'] = round(float(e['z']), 1)+ return merged+++def level_envs(x):+ """10 ms RMS (dB, floored at -80) of the full signal and of its highs (> 2 kHz, where a whoosh under music+ shows), smoothed over 90 ms: {'full': s, 'high': s}, frame i = i * 10 ms"""+ m = audio.to_mono(x).astype(np.float64)+ n = len(m) // 480+ if n < 3:+ return {}+ pad = min(len(m) - 1, SR // 10)+ hi = signal.sosfilt(signal.butter(4, 2000, 'high', fs=SR, output='sos'), np.concatenate([m[pad:0:-1], m]))[pad:]+ out = {}+ for band, y in (('full', m), ('high', hi)):+ r = np.maximum(20 * np.log10(np.sqrt(np.mean(y[: n * 480].reshape(n, 480) ** 2, axis=1)) + 1e-9), -80.0)+ out[band] = ndimage.uniform_filter1d(r, 9)+ return out+++def level_peak(envs, t, radius=PEAK_S, min_rise_db=PEAK_RISE_DB, min_fall_db=2.0, local=0.3):+ """a level peak near t that a transient detector misses (a whoosh / riser / swell peaking on a cut): the+ loudest smoothed 10 ms frame within +-radius, a local maximum over +-local s, rising >= min_rise_db over the+ 0.6 s before it and falling >= min_fall_db within 0.3 s after -> {'t', 'band', 'rise_db'} or None"""+ best = None+ for band, s in envs.items():+ c = int(round(t * 100))+ a, b = max(0, c - int(radius * 100)), min(len(s), c + int(radius * 100) + 1)+ if b - a < 3:+ continue+ i = a + int(np.argmax(s[a:b]))+ w = int(local * 100)+ if s[i] < s[max(0, i - w): i + w + 1].max() or i < 5:+ continue+ rise = float(s[i] - s[max(0, i - 60): i].min())+ after = s[i: i + 31]+ if rise < min_rise_db or len(after) < 5 or s[i] - after.min() < min_fall_db:+ continue+ if best is None or rise > best['rise_db']:+ best = {'t': round(i / 100, 3), 'band': band, 'rise_db': round(rise, 1)}+ return best+++def _swells(x, hits, min_rise_db=SWELL_RISE_DB, mono_frac=0.6, min_dur=0.25, envs=None):+ """build-ups (whooshes, risers, swells) timed at their peak: the level_envs rising >= min_rise_db over 0.25-2 s,+ mostly monotonically, then falling; a peak that is just a hit's attack is skipped"""+ envs = level_envs(x) if envs is None else envs+ if len(envs.get('full', ())) < 60:+ return []+ hit_t = np.array([h['t'] for h in hits]) if hits else np.zeros(0)+ out = []+ for band, s in envs.items():+ for i in np.where((s == ndimage.maximum_filter1d(s, 31)) & (s > -70))[0]:+ a = max(0, i - 200)+ if i - a < 25:+ continue+ j = a + int(np.argmin(s[a: i - 15]))+ rise, dur, after = s[i] - s[j], (i - j) / 100, s[i: i + 100]+ if rise < min_rise_db or dur < min_dur or len(after) < 10 or s[i] - after.min() < 4:+ continue+ if np.mean(np.diff(s[j: i + 1]) > 0) < mono_frac:+ continue+ t = i / 100+ if len(hit_t) and np.any(np.abs(hit_t - (t - 0.03)) < 0.05):+ continue+ if any(abs(o['t'] - t) < 0.15 for o in out):+ continue+ fall = np.where(after < s[i] - 10)[0]+ out.append({'t': t, 'shape': 'swell', 'band': band, 'contrast_db': float(rise), 'prominence_db': float(rise),+ 'bed_db': round(float(s[j]), 1), 'z': 0.0, 'rise_ms': int(dur * 1000),+ 'length_ms': int(fall[0] * 10) if len(fall) else 1000})+ return out+++# ------------------------------------------------------------------ loudness+def loudness(x, at_start=True, at_end=True):+ """BS.1770 loudness picture: curves (floored at the -70 LUFS absolute gate), integrated (None when everything is+ gated out), LRA, true peak (None for digital silence), loudest moments, stop-downs, clipping. at_start / at_end:+ x begins / ends where the file does (gaps touching it are a lead-in / tail, not a stop-down)"""+ dur = len(x) / SR+ pk = float(np.abs(x).max()) if len(x) else 0.0+ r = {'integrated_lufs': None, 'true_peak_dbtp': round(features.true_peak_dbtp(x), 1) if pk > 0 else None,+ 'digital_silence': pk == 0.0}+ if dur >= 0.4:+ tm, M = features.loudness_curve(x, 0.4, 0.1)+ M = np.maximum(M, -70.0)+ if M.max() > -70.0:+ r['integrated_lufs'] = round(features.integrated_lufs(x), 1)+ r['max_momentary'] = {'t': round(float(tm[np.argmax(M)]), 2), 'lufs': round(float(M.max()), 1)}+ r['momentary'] = {'t0': round(float(tm[0]), 3), 'dt': 0.1, 'lufs': [round(float(v), 1) for v in M]}+ if dur >= 3.0:+ ts, S = features.loudness_curve(x, 3.0, 0.5)+ S = np.maximum(S, -70.0)+ k = int(np.argmax(S))+ if S[k] > -70.0:+ r['max_short_term'] = {'start': round(float(ts[k]) - 1.5, 2), 'end': round(float(ts[k]) + 1.5, 2),+ 'lufs': round(float(S[k]), 1)}+ r['short_term'] = {'t0': round(float(ts[0]), 3), 'dt': 0.5, 'lufs': [round(float(v), 1) for v in S]}+ v = features.lra(x) if dur >= 6 else None+ r['lra_lu'] = round(v, 1) if v is not None else None+ r['silences'] = _silences(x, r['integrated_lufs'], at_start, at_end)+ a = np.abs(x).max(axis=1) if x.ndim == 2 else np.abs(x)+ clip = a >= 0.999+ runs = _runs(clip, 3)+ r['clipping'] = {'samples': int(clip.sum()), 'runs': len(runs),+ 'times': [round(s / SR, 3) for s, _ in runs[:20]]}+ return r+++def _runs(mask, min_len):+ d = np.diff(np.concatenate([[0], mask.astype(np.int8), [0]]))+ st, en = np.where(d == 1)[0], np.where(d == -1)[0]+ return [(int(s), int(e)) for s, e in zip(st, en) if e - s >= min_len]+++def _silences(x, integ, at_start=True, at_end=True):+ """gaps >= 0.25 s where the fast (0.1 s) loudness drops 20 LU under the integrated level (or below -60 LUFS);+ 'where' = lead-in / tail when the gap touches the start / end of the file"""+ if len(x) < SR // 2:+ return []+ t, F = features.loudness_curve(x, 0.1, 0.05)+ F = np.maximum(F, -70.0)+ thr = -70.0 if integ is None or integ <= -60 else max(-60.0, integ - 20.0)+ low = (F <= thr) if integ is None else (F < thr)+ low = ndimage.binary_closing(np.pad(low, 3, mode='edge'), np.ones(7, bool))[3:-3] & (F < thr + 6) # edge pad+ out, dur = [], len(x) / SR+ for s, e in _runs(low, 5):+ a, b = max(0.0, t[s] - 0.05), min(dur, t[e - 1] + 0.05)+ a, b = (0.0 if s == 0 else a), (dur if e == len(F) else b)+ seg = x[int(a * SR): int(b * SR)]+ pk = float(np.abs(seg).max()) if len(seg) else 0.0+ kind = 'digital silence' if pk < 1e-5 else ('near silence' if float(F[s:e].max()) < -60 else 'drop')+ g = {'start': round(a, 2), 'end': round(b, 2), 'dur': round(b - a, 2), 'kind': kind,+ 'level_lufs': round(float(F[s:e].mean()), 1)}+ head, tail = a <= 0.05 and at_start, b >= dur - 0.05 and at_end+ if head or tail:+ g['where'] = 'whole file' if head and tail else ('lead-in' if head else 'tail')+ out.append(g)+ return out+++# ------------------------------------------------------------------ music grid+def _pulse_envs(m):+ """beat-tracking novelty at 10 ms frames, broadband and low band (30-200 Hz) log-mel flux, local mean removed"""+ L = log_mel(m, n_fft=2048, hop=480, n_mels=64, fmin=30, fmax=8000)+ L = np.maximum(L, L.max() - 80)+ d = np.maximum(np.diff(L, axis=1, prepend=L[:, :1]), 0)+ out = []+ for env in (d.mean(axis=0), d[:6].mean(axis=0)):+ env = np.maximum(env - ndimage.uniform_filter1d(env, 101), 0)+ out.append(env / (env.std() + 1e-9))+ return out+++def _period(env, min_bpm, max_bpm):+ """(beat period in 10 ms frames, pulse clarity, score, autocorrelation) with a soft 120 BPM prior"""+ e0 = env - env.mean()+ f = np.fft.rfft(e0, 2 * len(e0))+ ac = np.fft.irfft(np.abs(f) ** 2)[: len(e0)]+ ac /= ac[0] + 1e-12+ lags = np.arange(int(6000 / max_bpm), min(int(6000 / min_bpm), len(ac) // 2))+ if not len(lags):+ return None+ prior = np.exp(-0.5 * (np.log2(6000 / lags / 120.0) / 0.9) ** 2)+ sc = (ac[lags] + 0.5 * ac[np.minimum(2 * lags, len(ac) - 1)]) * prior+ k = int(np.argmax(sc))+ P = float(lags[k])+ if 0 < k < len(lags) - 1:+ y0, y1, y2 = sc[k - 1], sc[k], sc[k + 1]+ P += 0.5 * (y0 - y2) / (y0 - 2 * y1 + y2 + 1e-12)+ return P, float(ac[int(round(P))]), float(sc[k]), ac+++def music_grid(m, min_bpm=60, max_bpm=200):+ """tempo (autocorrelation with a soft 120 BPM prior, on the broadband or the low-band flux, whichever pulses+ more clearly), beats (dynamic programming, Ellis 2007), a downbeat guess and how clear the pulse is+ (pulse_clarity = normalised autocorrelation at the beat period); tempo_alt = the double tempo when it pulses at+ least half as clearly (the felt tempo may be either: eval/timeline/tempo_tune.py)"""+ none = {'tempo': None, 'beats': [], 'downbeats': [], 'confidence': 'none'}+ if len(m) < 5 * SR:+ return dict(none, note='too short for a beat grid')+ cands = [(p, env) for env in _pulse_envs(m) for p in [_period(env, min_bpm, max_bpm)] if p]+ if not cands:+ return dict(none, note='too short for a beat grid')+ (P, clarity, _, ac), env = max(cands, key=lambda c: c[0][2])+ win = np.arange(max(1, int(P / 2)), int(2 * P) + 1)+ pen = 100.0 * np.log(win / P) ** 2+ score, back = env.copy(), np.full(len(env), -1)+ for t in range(int(P / 2), len(env)):+ prev = t - win+ ok = prev >= 0+ if not ok.any():+ continue+ c = score[prev[ok]] - pen[ok]+ j = int(np.argmax(c))+ if c[j] > 0:+ score[t] += c[j]+ back[t] = prev[ok][j]+ t = len(env) - int(P) + int(np.argmax(score[-int(P):]))+ beats = []+ while t >= 0:+ beats.append(t)+ t = back[t]+ beats = np.array(beats[::-1])+ if len(beats) < 4:+ return dict(none, note='no steady pulse found')+ ibi = np.diff(beats)+ reg = float(np.std(ibi) / np.mean(ibi))+ conf = 'clear' if clarity >= CLEAR_PULSE and reg < 0.1 else ('weak' if clarity >= WEAK_PULSE else 'none')+ ph = _downbeat_phase(m, beats)+ bt = beats * 0.01+ h = int(round(P / 2))+ alt = round(12000.0 / P, 1) if 12000.0 / P <= max_bpm and ac[h] >= 0.5 * clarity else None+ return {'tempo': round(6000.0 / P, 1), 'tempo_alt': alt, 'beats': [round(float(b), 3) for b in bt],+ 'downbeats': [round(float(b), 3) for b in bt[ph::4]], 'pulse_clarity': round(clarity, 3),+ 'beat_regularity': round(reg, 3), 'confidence': conf,+ 'note': 'beats at 10 ms resolution; downbeats are a guess (4/4 assumed); tempo may be half/double the felt '+ 'tempo'}+++def _downbeat_phase(m, beats):+ """which of 4 beat phases starts the bar: harmony changes (pitch-class profile between beats) plus low-band+ onsets (kick) are strongest on downbeats"""+ n_fft, hop = 4096, 480+ y = np.pad(np.asarray(m, np.float32), (n_fft // 2, n_fft // 2))+ n = 1 + (len(y) - n_fft) // hop+ f = np.fft.rfftfreq(n_fft, 1 / SR)+ sel = (f >= 55) & (f <= 2000)+ pc = (np.round(12 * np.log2(f[sel] / 440.0)) % 12).astype(int)+ fr = np.lib.stride_tricks.as_strided(y, (n, n_fft), (y.strides[0] * hop, y.strides[0]))+ C = np.zeros((n, 12), np.float32)+ low = np.zeros(n, np.float32)+ win = np.hanning(n_fft).astype(np.float32)+ for a in range(0, n, 2048):+ M = np.abs(np.fft.rfft(fr[a:a + 2048] * win, axis=1))+ for k in range(12):+ C[a:a + 2048, k] = M[:, sel][:, pc == k].sum(axis=1)+ low[a:a + 2048] = np.log1p(M[:, f < 150].sum(axis=1))+ b = np.minimum(beats, n - 1)+ seg = np.array([C[p:q].mean(axis=0) if q > p else C[p] for p, q in zip(b[:-1], b[1:])] + [C[b[-1]]])+ seg /= np.linalg.norm(seg, axis=1, keepdims=True) + 1e-9+ harm = np.concatenate([[0.0], 1 - np.sum(seg[1:] * seg[:-1], axis=1)])+ kick = np.maximum(np.diff(low, prepend=low[0]), 0)+ kick = np.array([kick[max(0, i - 3): i + 4].max() for i in b])+ acc = harm / (harm.std() + 1e-9) + kick / (kick.std() + 1e-9)+ return int(np.argmax([acc[p::4].mean() for p in range(4)]))+++# ------------------------------------------------------------------ cues and sync+def read_cues(cues):+ """cue events from a JSON path / JSON text / 'a,b,c' seconds / list / {fps, events:[{t|f, kind, id}]}+ (also accepted: time / sec / seconds for t, frame for f) -> (cues, fps)"""+ if cues is None:+ return [], None+ if isinstance(cues, str):+ s = cues.strip()+ if os.path.exists(s):+ cues = json.load(open(s, encoding='utf-8'))+ elif s[:1] in '[{':+ cues = json.loads(s)+ else:+ try:+ cues = [float(v) for v in s.replace(';', ',').split(',') if v.strip()]+ except ValueError:+ raise ValueError('cues: expected a JSON file, JSON text or comma-separated seconds, '+ f'got "{s[:60]}"') from None+ fps = None+ if isinstance(cues, dict):+ fps = cues.get('fps')+ cues = cues.get('events') or cues.get('cues') or []+ out = []+ for c in cues:+ if isinstance(c, (int, float)):+ out.append({'t': float(c), 'kind': 'cue', 'id': None})+ continue+ tk = next((k for k in ('t', 'time', 'sec', 'seconds') if k in c), None)+ fk = next((k for k in ('f', 'frame') if k in c), None)+ if tk:+ t = float(c[tk])+ elif fk and fps:+ t = float(c[fk]) / float(fps)+ elif fk:+ raise ValueError('cues have frame numbers but no fps: pass {"fps": 60, "events": [{"f": 120, ...}]}')+ else:+ raise ValueError(f'cue without a time: {json.dumps(c)[:80]} (give "t" in seconds or "f" with "fps")')+ out.append({'t': round(t, 4), 'kind': c.get('kind', 'cue'), 'id': c.get('id')})+ out.sort(key=lambda c: c['t'])+ return out, fps+++def _strong(events):+ if not events:+ return []+ p = np.array([e['prominence_db'] for e in events])+ thr = min(6.0, float(np.percentile(p, 40)))+ return [e for e in events if e['prominence_db'] >= thr]+++def _nearest(t, evs, ts, radius):+ if not len(ts):+ return None+ k = int(np.searchsorted(ts, t))+ best = None+ for j in (k - 1, k, k + 1):+ if 0 <= j < len(ts) and abs(ts[j] - t) <= radius and (best is None or abs(ts[j] - t) < abs(ts[best] - t)):+ best = j+ return None if best is None else evs[best]+++def _answer(t, events, ts, strong, st, swells, sw, tol1, levels=None):+ """the event that answers a cut / cue at t: any onset within tol1 (OK), else the nearest strong one within+ 100 ms, else any within 100 ms (near), else a detected build-up (whoosh / riser) peaking within 300 ms (swell),+ else a level peak within 150 ms that the onset detector misses (peak: a soft whoosh / swell?), else a strong+ onset within 250 ms (off); None = no sound"""+ for evs, arr, rad, status in ((events, ts, tol1, 'OK'), (strong, st, 0.1, 'near'), (events, ts, 0.1, 'near'),+ (swells, sw, 0.3, 'swell')):+ e = _nearest(t, evs, arr, rad)+ if e is not None:+ return e, status+ pk = level_peak(levels, t) if levels else None+ if pk is not None:+ return {'t': pk['t'], 'prominence_db': pk['rise_db'], 'shape': 'peak', 'band': pk['band']}, 'peak'+ e = _nearest(t, strong, st, SYNC_S)+ return (e, 'off') if e is not None else (None, 'no sound')+++def _accents(events):+ """events that read as a deliberate hit: the top 20% by prominence and >= 6 dB over the bed -> (events, dB)"""+ if not events:+ return [], None+ thr = max(ACCENT_DB, float(np.percentile([e['prominence_db'] for e in events], 80)))+ return [e for e in events if e['prominence_db'] >= thr], round(thr, 1)+++def _chance(ts, lo, hi, tol):+ """fraction of random times in [lo, hi] that fall within tol of one of ts (sorted)"""+ if not len(ts):+ return 0.0+ grid = np.arange(lo, hi, 0.005)+ k = np.clip(np.searchsorted(ts, grid), 1, max(1, len(ts) - 1))+ d = np.minimum(np.abs(grid - ts[k - 1]), np.abs(grid - ts[np.minimum(k, len(ts) - 1)]))+ return round(float(np.mean(d <= tol)), 3)+++def sync(times, events, beats=None, frame_s=None, levels=None):+ """for each cut / cue time: the audio event that answers it (offset ms, negative = sound early) and stats;+ levels = level_envs(x) also looks for level peaks (soft whooshes / swells) where no onset answers"""+ strong = _strong(events)+ swells = [e for e in events if e['shape'] == 'swell']+ st, sw = np.array([e['t'] for e in strong]), np.array([e['t'] for e in swells])+ ts = np.array([e['t'] for e in events])+ acc, acc_db = _accents(events)+ at = np.array([e['t'] for e in acc])+ tol1 = frame_s or 0.04+ bt = np.asarray(beats) if beats is not None and len(beats) else None+ items = []+ for c in times:+ t = c['t'] if isinstance(c, dict) else float(c)+ e, status = _answer(t, events, ts, strong, st, swells, sw, tol1, levels)+ it = {'t': round(t, 3)}+ if isinstance(c, dict):+ it.update({k: c[k] for k in ('kind', 'id') if c.get(k) is not None})+ it.update({'offset_ms': None if e is None else round((e['t'] - t) * 1000, 1), 'status': status})+ if e is not None:+ it.update({'event_t': e['t'], 'event_prominence_db': e['prominence_db']})+ if e.get('label'):+ it['event'] = e['label']+ a = _nearest(t, acc, at, 0.1)+ it['accent_offset_ms'] = None if a is None else round((a['t'] - t) * 1000, 1)+ if bt is not None:+ it['beat_offset_ms'] = round((t - float(bt[int(np.argmin(np.abs(bt - t)))])) * 1000, 1)+ items.append(it)+ n = len(items)+ stats = {'n': n}+ if not n:+ return {'items': items, 'stats': stats}+ offs = np.array([abs(i['offset_ms']) for i in items if i['offset_ms'] is not None])+ k1 = 'within_1_frame' if frame_s else 'within_40ms'+ stats.update({f'hit_{k1}': round(float(np.mean([i['status'] == 'OK' for i in items])), 3),+ 'hit_within_100ms': round(float(np.mean([i['status'] in ('OK', 'near') for i in items])), 3),+ 'median_abs_offset_ms': round(float(np.median(offs)), 1) if len(offs) else None,+ 'answered_by_swell': sum(i['status'] == 'swell' for i in items),+ 'answered_by_level_peak': sum(i['status'] == 'peak' for i in items),+ 'no_sound': [i['t'] for i in items if i['status'] in ('no sound', 'off')],+ ('frame_ms' if frame_s else 'tolerance_ms'): round(tol1 * 1000, 1)})+ lo, hi = min(i['t'] for i in items) - 1, max(i['t'] for i in items) + 1+ if len(ts):+ stats[f'chance_{k1}'] = _chance(ts, lo, hi, tol1)+ stats['chance_within_100ms'] = _chance(ts, lo, hi, 0.1)+ if len(at):+ ao = np.array([abs(i['accent_offset_ms']) if i['accent_offset_ms'] is not None else 1e9 for i in items])+ stats.update({'accent_min_db': acc_db, f'accent_{k1}': round(float(np.mean(ao <= tol1 * 1000 + 1e-6)), 3),+ f'accent_chance_{k1}': _chance(at, lo, hi, tol1),+ 'accent_within_100ms': round(float(np.mean(ao <= 100)), 3),+ 'accent_chance_within_100ms': _chance(at, lo, hi, 0.1)})+ if bt is not None and len(bt) > 1:+ bo = np.array([abs(i['beat_offset_ms']) for i in items])+ stats['on_beat'] = round(float(np.mean(bo <= ON_BEAT_S * 1000)), 3)+ stats['on_beat_chance'] = round(min(1.0, 2 * ON_BEAT_S / float(np.median(np.diff(bt)))), 3)+ return {'items': items, 'stats': stats}+++# ------------------------------------------------------------------ models (CLAP tags, AST labels)+def _embed_text(c, texts):+ """c.embed_text, surviving a failed save of the shared text cache (Windows: os.replace over an open file); the+ embeddings are then kept in memory for this process"""+ try:+ return c.embed_text(texts)+ except PermissionError:+ c._text_dirty = False+ return c.embed_text(texts)+++def _phrase_matrix(c, terms, tpl=None):+ """[(kind, phrase)] embedded through prompt templates (default: earshot's vocab.json ones; mean, unit rows)"""+ if tpl is None:+ v = json.load(open(os.path.join(config.DATA_DIR, 'vocab.json'), encoding='utf-8'))+ tpl = v.get('templates') or ['{}']+ E = _embed_text(c, [p.format(t) for _, t in terms for p in tpl]).reshape(len(terms), len(tpl), -1).mean(axis=1)+ return E / (np.linalg.norm(E, axis=1, keepdims=True) + 1e-12)+++def _vocab_matrix(c):+ cats = json.load(open(os.path.join(config.DATA_DIR, 'vocab.json'), encoding='utf-8'))['categories']+ terms = [(k, t) for k, ts in cats.items() for t in ts]+ return [t for _, t in terms], np.array([k for k, _ in terms]), _phrase_matrix(c, terms)+++def _event_matrix(c):+ """compact event vocabulary: (terms, kinds, E), embedded through timeline's own templates (TAG_BG is+ measured with them)"""+ terms = [(k, t) for k, ts in EVENT_VOCAB.items() for t in ts]+ return [t for _, t in terms], np.array([k for k, _ in terms]), _phrase_matrix(c, terms, TAG_TEMPLATES)+++def tag_scores(A, E, terms):+ """CLAP audio embeddings A (n, 512) vs the event phrases E -> (cos, z): z within each event after removing+ TAG_CENTER of each phrase's background mean, so phrases that score high on any short sound ('a chat+ message pop', 'a spacebar press') no longer win everywhere (eval/timeline/tag_bg.py)"""+ cos = A @ E.T+ d = cos - TAG_CENTER * np.array([TAG_BG[t] for t in terms])+ return cos, (d - d.mean(axis=1, keepdims=True)) / (d.std(axis=1, keepdims=True) + 1e-9)+++def _zscores(A, E):+ cos = A @ E.T+ return cos, (cos - cos.mean(axis=1, keepdims=True)) / (cos.std(axis=1, keepdims=True) + 1e-9)+++def _top(terms, cats, cos, z, allowed, k=3):+ idx = [i for i in np.argsort(-z) if cats[i] in allowed][:k]+ return [{'tag': terms[i], 'cat': str(cats[i]), 'z': round(float(z[i]), 2), 'cos': round(float(cos[i]), 3)}+ for i in idx]+++def _short(term):+ for a in ('a ', 'an ', 'the '):+ if term.startswith(a):+ return term[len(a):]+ return term+++def _cached(model, fid, spans, kind, compute, reuse=True):+ """per-span model outputs cached on disk next to the model's own cache (reuse=False recomputes and rewrites)"""+ keys = [hashlib.sha1(f'{fid}|{a:.3f}|{b:.3f}|{kind}|tl{SPAN_VERSION}'.encode()).hexdigest()[:24] for a, b in spans]+ files = [os.path.join(model.dir, f'tl_{k}.npy') for k in keys]+ out, todo = [None] * len(spans), []+ for i, f in enumerate(files):+ if reuse and os.path.exists(f):+ try:+ out[i] = np.load(f)+ continue+ except Exception:+ pass+ todo.append(i)+ if todo:+ res = compute([spans[i] for i in todo])+ for i, v in zip(todo, res):+ out[i] = v+ tmp = f'{files[i]}.{os.getpid()}.tmp.npy'+ np.save(tmp, v)+ audio.replace_quietly(tmp, files[i])+ return np.stack(out) if out else None+++def span_id(path):+ """file id for the per-span model cache; mono sources decode at unity gain since core decode v2"""+ return audio.file_id(path) + ('|mono2' if audio.probe(path).get('channels') == 1 else '')+++def _seg(x, a, b):+ return audio.to_mono(x[max(0, int(a * SR)): max(int(a * SR) + 480, int(b * SR))])+++def _event_span(e, dur):+ if e['shape'] == 'swell':+ a, b = e['t'] - e.get('rise_ms', 500) / 1000, e['t'] + 0.3+ else:+ a, b = e['t'] - 0.02, e['t'] + min(max(e['length_ms'] / 1000, 0.3), 1.5)+ return round(max(0.0, a), 3), round(min(dur, b), 3)+++def _spread(order, times, hi, n, lo=0.0):+ """choose n indices from a priority order so the whole range [lo, hi] is covered: the best event of each of+ n // 2 equal time bins first, then the rest by priority"""+ nb = max(1, n // 2)+ first, seen = [], set()+ for i in order:+ b = min(nb - 1, max(0, int((times[i] - lo) / max(hi - lo, 1e-6) * nb)))+ if b not in seen:+ seen.add(b)+ first.append(i)+ rest = [i for i in order if i not in set(first)]+ return sorted((first + rest)[:n])+++def _pick(events, n_tag, near, lo, hi):+ """events to tag: those near a cut / cue first, then by prominence, spread over the timeline; later events of+ a burst (same band within BURST_S of a picked one) go last, so one typing run does not take the whole budget"""+ near = np.asarray(sorted(near))+ is_near = [bool(len(near) and np.min(np.abs(near - e['t'])) <= NEAR_S) for e in events]+ order = sorted(range(len(events)),+ key=lambda i: (-is_near[i], -events[i]['prominence_db'], -events[i]['contrast_db']))+ kept, dup = [], []+ for i in order:+ e = events[i]+ same = any(abs(events[j]['t'] - e['t']) < BURST_S and events[j]['band'] == e['band'] for j in kept)+ (dup if same else kept).append(i)+ return _spread(kept + dup, [e['t'] for e in events], hi, n_tag, lo), kept + dup+++def tag_events(x, fid, t0, events, n_tag, n_ast, near=(), reuse=True, win=None):+ """CLAP character tags for up to n_tag events (see _pick): the event excerpt, repeat-padded to one 10 s window,+ scored against the compact event vocabulary (tag_scores); 'exposed' = the event is >= EXPOSED_DB over its bed,+ so it dominates the excerpt. AST labels for the n_ast most important tagged events; AST speech >= 0.5 overrides+ the CLAP kind with VOICE. Repeat padding beat zero padding and bed subtraction on the tuning mixes+ (eval/timeline/tag_tune.py) at half the cost of the latter. Times are relative to x, which starts at file time+ t0 (span cache keys); win = the analysed window inside x (default: all of it)"""+ from .models import clap+ if not events or n_tag <= 0:+ return {'tagged': 0, 'ast': 0, 'clap_windows': 0}+ dur = len(x) / SR+ pick, order = _pick(events, n_tag, near, *(win or (0.0, dur)))+ c = clap.get()+ terms, kinds, E = _event_matrix(c)+ spans = [_event_span(events[i], dur) for i in pick]+ A = _cached(c, fid, [(a + t0, b + t0) for a, b in spans], 'clap-repeat',+ lambda q: c.embed_audio([_seg(x, a - t0, b - t0) for a, b in q], pad='repeatpad'), reuse)+ _, z = tag_scores(A, E, terms)+ for j, i in enumerate(pick):+ idx = np.argsort(-z[j])[:3]+ e = events[i]+ e['tags'] = [{'tag': terms[k], 'kind': str(kinds[k]), 'z': round(float(z[j][k]), 2)} for k in idx]+ e['kind'] = e['tags'][0]['kind']+ e['label'] = _short(terms[idx[0]])+ e['exposed'] = bool(e['prominence_db'] >= EXPOSED_DB)+ picked = set(pick)+ ranked = [i for i in order if i in picked][:n_ast]+ if ranked:+ from .models import ast+ a = ast.get()+ sp = [_event_span(events[i], dur) for i in ranked]+ P = _cached(a, fid, [(s + t0, b + t0) for s, b in sp], 'ast',+ lambda q: a.probs([_seg(x, s - t0, b - t0) for s, b in q]), reuse)+ for i, p in zip(ranked, P):+ e = events[i]+ e['ast'] = a.top(p, 4, min_p=0.05)+ sp_p = a.prob_of(p, 'Speech')+ if sp_p >= 0.5 and e['kind'] != 'VOICE':+ e.update(kind='VOICE', label=f'speech (AST {sp_p:.2f})', exposed=True)+ return {'tagged': len(pick), 'ast': len(ranked), 'clap_windows': len(pick)}+++# ------------------------------------------------------------------ sections+def _novelty_refine(m, t, radius):+ """move a coarse boundary to the strongest spectral change within +-radius (1 s log-mel halves)"""+ a, b = max(0.0, t - radius - 1.0), min(len(m) / SR, t + radius + 1.0)+ seg = m[int(a * SR): int(b * SR)]+ if len(seg) < 2 * SR:+ return t+ L = log_mel(seg, n_fft=2048, hop=2400, n_mels=40)+ best, bt = -1.0, t+ for k in range(10, L.shape[1] - 10):+ tt = a + k * 0.05+ if abs(tt - t) > radius:+ continue+ d = float(np.linalg.norm(L[:, k - 10:k].mean(axis=1) - L[:, k:k + 10].mean(axis=1)))+ if d > best:+ best, bt = d, tt+ return round(bt, 2)+++def novelty_bounds(m, kernel_s=4.0, zthr=NOVELTY_Z):+ """strong timbre / level changes: Foote novelty (checkerboard kernel, +-kernel_s) on 0.5 s log-mel frames;+ returns [(time_s, z)] of peaks at least kernel_s apart with robust z >= zthr"""+ L = log_mel(m, n_fft=2048, hop=2400, n_mels=40, fmin=30, fmax=16000)+ n, w = L.shape[1] // 10, int(kernel_s / 0.5)+ if n < 4 * w:+ return []+ F = L[:, : n * 10].reshape(L.shape[0], n, 10).mean(axis=2)+ F = (F - F.mean(axis=1, keepdims=True)) / (F.std(axis=1, keepdims=True) + 1e-6)+ F /= np.linalg.norm(F, axis=0, keepdims=True) + 1e-9+ S = F.T @ F+ g = np.exp(-0.5 * (np.arange(-w, w) + 0.5) ** 2 / (0.5 * w) ** 2)+ K = np.outer(np.r_[-np.ones(w), np.ones(w)], np.r_[-np.ones(w), np.ones(w)]) * np.outer(g, g)+ nov = np.zeros(n)+ for i in range(w, n - w):+ nov[i] = float(np.sum(K * S[i - w:i + w, i - w:i + w]))+ z = _robust_z(nov, n)+ pk = np.where((nov == ndimage.maximum_filter1d(nov, 2 * w + 1)) & (z >= zthr))[0]+ return [(round(i * 0.5, 2), round(float(z[i]), 1)) for i in pk if w <= i < n - w]+++def sections(x, fid, t0, win, hop, ast_budget=4, merge_cos=0.84, min_len=4.0, reuse=True, nov_z=None):+ """CLAP windows merged into contiguous sections; each section tagged (genre, mood, instruments, ambience,+ voice) plus AST Music / Speech / Silence"""+ from .models import clap+ dur = len(x) / SR+ if dur < 1.0:+ return [], {'windows': 0, 'ast': 0}+ c = clap.get()+ terms, cats, E = _vocab_matrix(c)+ starts = list(np.arange(0.0, max(dur - win, 0.0) + 1e-6, hop))+ if not starts or starts[-1] + win < dur - 0.5:+ starts.append(max(0.0, dur - win))+ spans = [(round(s, 3), round(min(dur, s + win), 3)) for s in starts]+ A = _cached(c, fid, [(a + t0, b + t0) for a, b in spans], 'clap-repeat',+ lambda sp: c.embed_audio([_seg(x, a - t0, b - t0) for a, b in sp], pad='repeatpad'), reuse)+ segs = [[i] for i in range(len(spans))]++ def mean(seg):+ v = A[seg].mean(axis=0)+ return v / (np.linalg.norm(v) + 1e-12)++ def length(seg):+ return spans[seg[-1]][1] - spans[seg[0]][0]+ m = audio.to_mono(x).astype(np.float32)+ nov = [t for t, _ in novelty_bounds(m, zthr=NOVELTY_Z if nov_z is None else nov_z)]++ def blocked(a, b):+ j = spans[b[0]][0] + (win if spans[b[0]][0] >= spans[a[-1]][1] else (spans[a[-1]][1] - spans[b[0]][0])) / 2+ return any(abs(t - j) <= hop for t in nov)+ max_n = max(2, int(np.ceil(dur / 8.0)))+ while len(segs) > 1:+ sims = np.array([float(mean(a) @ mean(b)) - blocked(a, b) for a, b in zip(segs[:-1], segs[1:])])+ short = [j for j in range(len(sims)) if length(segs[j]) < min_len or length(segs[j + 1]) < min_len]+ j = int(np.argmax(sims))+ if sims[j] < merge_cos and len(segs) <= max_n:+ if not short:+ break+ j = max(short, key=lambda k: sims[k])+ segs[j] = segs[j] + segs.pop(j + 1)+ bounds = [0.0]+ for a, b in zip(segs[:-1], segs[1:]):+ coarse = (spans[a[-1]][0] + spans[b[0]][0] + win) / 2 if spans[b[0]][0] < spans[a[-1]][1] else spans[b[0]][0]+ bounds.append(_novelty_refine(m, coarse, hop))+ bounds.append(round(dur, 2))+ k = 1+ while k < len(bounds) - 1: # a refined boundary may leave a sliver: merge it away+ if bounds[k] - bounds[k - 1] < min_len / 2 or bounds[k + 1] - bounds[k] < min_len / 2:+ segs[k - 1] = segs[k - 1] + segs.pop(k)+ bounds.pop(k)+ else:+ k += 1+ out = []+ for k, seg in enumerate(segs):+ v = mean(seg)+ cos, z = _zscores(v[None], E)+ s = {'start': round(bounds[k], 2), 'end': round(bounds[k + 1], 2), 'windows': len(seg),+ 'coherence': round(float(np.mean(A[seg] @ v)), 3)}+ for cat in SECTION_CATS:+ s[cat] = _top(terms, cats, cos[0], z[0], (cat,), k=2)+ _section_level(x, s)+ out.append(s)+ n_ast = _section_ast(x, fid, t0, out, ast_budget, reuse)+ for s in out:+ s['label'] = _section_label(s)+ def key(s): # genre + other sources (instruments / mood vary in a track)+ parts = s['label'].split(' | ')+ return parts[0].split(',')[0], parts[1:]+ k = 1+ while k < len(out): # neighbours that read alike, no strong change between: merge+ a, b = out[k - 1], out[k]+ if key(a) != key(b) or any(abs(t - b['start']) <= 1.0 for t in nov):+ k += 1+ continue+ keep = a if a['end'] - a['start'] >= b['end'] - b['start'] else b+ out[k - 1] = dict(keep, start=a['start'], end=b['end'], windows=a['windows'] + b['windows'])+ _section_level(x, out[k - 1])+ out.pop(k)+ return out, {'windows': len(spans), 'ast': n_ast}+++def _section_level(x, s):+ xs = x[int(s['start'] * SR): int(s['end'] * SR)]+ v = features.integrated_lufs(xs) if len(xs) > 0.4 * SR else None+ s['lufs'] = round(v, 1) if v is not None and v > -70 else None+ s['digital_silence'] = bool(len(xs) and float(np.abs(xs).max()) == 0.0)+++def _section_ast(x, fid, t0, secs, budget, reuse=True):+ """AST Music / Speech / Silence and top labels per section from centred 10.24 s crops: one per section (longest+ first) while the budget lasts, a second one for sections over 15 s if crops are left"""+ if budget <= 0 or not secs:+ return 0+ from .models import ast+ a = ast.get()+ order = sorted(range(len(secs)), key=lambda k: -(secs[k]['end'] - secs[k]['start']))+ plan = {k: 1 for k in order[:budget]}+ for k in order:+ if sum(plan.values()) < budget and k in plan and secs[k]['end'] - secs[k]['start'] > 15:+ plan[k] = 2+ spans, owner = [], []+ for k, n in sorted(plan.items()):+ s0, d = secs[k]['start'], secs[k]['end'] - secs[k]['start']+ w = min(d, 10.24)+ for j in range(n):+ c0 = s0 + (d - w) * (j + 0.5) / n if n > 1 else s0 + (d - w) / 2+ spans.append((round(c0, 3), round(c0 + w, 3)))+ owner.append(k)+ P = _cached(a, fid, [(p + t0, q + t0) for p, q in spans], 'ast',+ lambda sp: a.probs([_seg(x, p - t0, q - t0) for p, q in sp]), reuse)+ owner = np.array(owner)+ for k in plan:+ p = P[owner == k].mean(axis=0)+ secs[k]['ast'] = {lab: round(a.prob_of(p, lab), 3) for lab in ('Music', 'Speech', 'Silence')}+ secs[k]['ast_top'] = a.top(p, 5, min_p=0.05)+ return len(spans)+++AST_GENERIC = {'Music', 'Musical instrument', 'Speech', 'Silence', 'Sound effect', 'Inside, small room',+ 'Outside, rural or natural', 'Outside, urban or manmade', 'Inside, large room or hall', 'Animal'}+++def _section_label(s):+ """'cinematic orchestral music, strings ensemble, uplifting | also: rain, typing (AST) | voice: ...' -- genre+ tags under z 2.5 carry a '?'"""+ if s.get('digital_silence'):+ return 'digital silence'+ if s.get('lufs') is None or s['lufs'] < -55:+ return 'near silence'+ ap = s.get('ast') or {}+ parts = []+ music = ap['Music'] >= 0.3 if ap else s['music_genre'][0]['z'] >= s['ambience'][0]['z']+ if music:+ g, md, ins = s['music_genre'][0], s['music_mood'][0], s['instruments'][0]+ bits = [_short(g['tag']) + ('' if g['z'] >= Z_OK else '?')]+ if ins['z'] >= Z_WEAK:+ bits.append(_short(ins['tag']))+ if md['z'] >= Z_OK: # weaker moods were wrong ~45% under sfx (label_tune.py)+ bits.append(_short(md['tag']).replace(' music', ''))+ parts.append(', '.join(bits))+ extra = [lab.lower() for lab, p in s.get('ast_top') or [] if lab not in AST_GENERIC and p >= 0.08][:3]+ if extra:+ parts.append('also: ' + ', '.join(extra) + ' (AST)')+ elif s['ambience'][0]['z'] >= (Z_OK if music else Z_WEAK):+ parts.append('amb: ' + _short(s['ambience'][0]['tag']))+ if ap.get('Speech', 0) >= 0.3 or (not ap and s['voice'][0]['z'] >= Z_OK):+ parts.append('voice: ' + _short(s['voice'][0]['tag']))+ return ' | '.join(parts) or 'unclear (no confident music / ambience / voice tag)'+++# ------------------------------------------------------------------ transcript+def _ascii(s):+ return str(s).encode('ascii', 'backslashreplace').decode()+++def _hz(f):+ return '-' if not f else (f'{f / 1000:.1f} kHz' if f >= 1000 else f'{f} Hz')+++BUILDUP_WORDS = ('whoosh', 'swoosh', 'swish', 'riser', 'rise', 'swell', 'sweep', 'wave', 'uplift')+++def _event_line(e, marks):+ """one transcript line: kind ('?' = the event is masked by its bed, so the tag is a weak hint), best tag /+ runner-up of another kind, level, prominence, length, centroid, AST, sync marks"""+ if e.get('label'):+ alt = next((t['tag'] for t in e['tags'] if t['kind'] != e['kind']), None)+ what = e['label'] + (f' / {_short(alt)}' if alt else '')+ k = e['kind'] + ('' if e.get('exposed') else '?')+ else:+ what, k = f"{e['band']} {'swell' if e['shape'] == 'swell' else 'onset'}", '-'+ over = f"{e['prominence_db']:+5.1f} dB" if e.get('bed_db', 0.0) > SILENT_BED_DB else f"{'silent':>8}"+ ln = (f"{fmt_t(e['t']):>8} {k:<6} {what[:46]:<46} {e['peak_dbfs']:6.1f} dBFS {over} "+ f"{e['length_ms']:>5} ms {_hz(e.get('centroid_hz')):>8}")+ lab = next((f'{l} {p:.2f}' for l, p in e.get('ast') or [] if l != 'Music' or len(e['ast']) == 1), None)+ if lab:+ ln += f' [AST {lab}]'+ if marks:+ ln += ' | ' + '; '.join(marks)+ return _ascii(ln)+++def top_accents(ev, n=3):+ """the n most prominent events over a running bed (bed above -60), at least 0.5 s apart"""+ acc = []+ for e in sorted([e for e in ev if e.get('bed_db', -80) > SILENT_BED_DB], key=lambda e: -e['prominence_db']):+ if all(abs(e['t'] - a['t']) >= 0.5 for a in acc):+ acc.append(e)+ if len(acc) == n:+ break+ return acc+++def _unanswered_line(kind, it):+ who = ' '.join(v for v in (it.get('kind') if it.get('kind') != 'cue' else '', it.get('id') or '') if v)+ who = f' {who}' if who and kind == 'cues' else ''+ if it['status'] == 'peak':+ why = (f"no onset; level peak {it['offset_ms']:+.0f} ms, rising {it['event_prominence_db']:.0f} dB "+ '(a soft whoosh / swell, or just the bed?) <- CHECK')+ else:+ why = ('no onset or level peak within 250 ms' if it['status'] == 'no sound' else+ f"no onset within 100 ms (strong sound at {it['offset_ms']:+.0f} ms)")+ build = any(w in f"{it.get('kind') or ''} {it.get('id') or ''}".lower() for w in BUILDUP_WORDS)+ why += ' <- NO HIT' + (' (a soft build-up can still be missed: look at the sheet)' if build else '')+ return _ascii(f"{fmt_t(it['t']):>8} {kind[:3].upper():<6}{who} {why}")+++def transcript(r):+ """the listening transcript: summary, sections, then one chronological line per tagged event, unanswered cut /+ cue, gap and section change"""+ L = [r['summary'], '']+ secs = r.get('sections') or []+ if secs:+ L.append('SECTIONS')+ for s in secs:+ ap = s.get('ast') or {}+ extra = f" [AST music {ap['Music']:.2f}, speech {ap['Speech']:.2f}]" if ap else ''+ lu = f" {s['lufs']:.0f} LUFS" if s.get('lufs') is not None else ''+ L.append(_ascii(f" {fmt_t(s['start'])}-{fmt_t(s['end'])} {s['label']}{lu}{extra}"))+ L.append('')+ ev, sy = r.get('events') or [], r.get('sync') or {}+ marks = {}+ for kind in ('cuts', 'cues'):+ for it in (sy.get(kind) or {}).get('items', []):+ if it.get('event_t') is not None and it['status'] not in ('off', 'peak'):+ tag = 'cut' if kind == 'cuts' else (it.get('kind') or 'cue')+ marks.setdefault(it['event_t'], []).append(f"{tag} {it['offset_ms']:+.0f} ms {it['status']}")+ acc = {e['t'] for e in top_accents(ev)}+ for t in acc:+ marks.setdefault(t, []).append('strong accent (summary)')+ rows = [(e['t'], 1, _event_line(e, marks.get(e['t'], []))) for e in ev if e.get('label') or e['t'] in marks]+ for kind in ('cuts', 'cues'):+ for it in (sy.get(kind) or {}).get('items', []):+ if it['status'] in ('no sound', 'off', 'peak'):+ rows.append((it['t'], 2, _unanswered_line(kind, it)))+ bounds = [s['start'] for s in secs] + [r['end']]+ for k, s in enumerate(secs):+ rest = [e for e in ev if s['start'] <= e['t'] < bounds[k + 1] and not e.get('label') and e['t'] not in marks]+ more = f' (+{len(rest)} untagged onsets, {len(rest) / max(s["end"] - s["start"], 1e-6):.1f}/s)' if rest else ''+ rows.append((s['start'], 0, _ascii(f"== {fmt_t(s['start'])}-{fmt_t(s['end'])} {s['label']}{more}")))+ for g in (r.get('loudness') or {}).get('silences', []):+ lvl = '< -70' if g['level_lufs'] <= -70 else f"{g['level_lufs']:.0f}"+ rows.append((g['start'], 1, f"{fmt_t(g['start']):>8} {'GAP':<6} {g['kind']} {g['dur']:.2f} s ({lvl} LUFS)"+ + (f" = {g['where']}" if g.get('where') else '')))+ rows.sort(key=lambda q: (q[0], q[1]))+ L.append('TRANSCRIPT time kind what (CLAP tag / runner-up) peak over bed length '+ 'centroid | sync')+ L += [q[2] for q in rows]+ L += ['', 'NOTES times, levels, loudness and sync offsets are measurements. Event kinds / tags are CLAP guesses: '+ f"'?' = the event is less than {EXPOSED_DB:.0f} dB over its bed, so its excerpt is mostly bed and the tag a "+ 'weak hint (right kind about 1 in 4 on the tuning mixes, 1 in 2 for exposed events; often wrong on music '+ "and voice). Where AST ran, its speech overrides the kind with VOICE. Section genres marked ? are weak. "+ "'over bed' = how far the event rises over the level just before it; 'silent' = that bed was below -60 "+ '(after a gap the rise means nothing). Full data: --json.']+ return '\n'.join(L)+++# ------------------------------------------------------------------ main entry+def _src_hash():+ h = hashlib.sha1()+ for f in ('audio.py', 'features.py', 'video.py', 'config.py'):+ h.update(open(os.path.join(os.path.dirname(os.path.abspath(__file__)), f), 'rb').read())+ return h.hexdigest()[:12]+++CORE_ID = _src_hash() # cached results never outlive a change of decode, features or cut detection+++def _cue_key(cues):+ if cues is None:+ return 'none'+ if isinstance(cues, str) and os.path.exists(cues):+ return audio.file_id(cues)+ return hashlib.sha1(json.dumps(cues, sort_keys=True, default=str).encode()).hexdigest()[:16]+++def _is_video(path, pr):+ """a real picture stream: not the cover art of an audio file (one attached picture at 90000 'fps')"""+ fps = (pr.get('video') or {}).get('fps') or 0+ return bool(pr.get('has_video')) and 0 < fps <= 240 and os.path.splitext(path)[1].lower() not in audio.AUDIO_EXTS+++def timeline(path, video=None, cues=None, start=None, end=None, detail='normal', cache=True, fresh=False):+ """listen through a soundtrack / film: events with tags, sections, loudness, beat grid, cut / cue sync,+ transcript. cache: read / write cached results and model outputs; fresh: recompute everything and store it"""+ if detail not in DETAIL:+ raise ValueError(f'detail must be one of {list(DETAIL)}')+ for f in (path, video):+ if f is not None and not os.path.isfile(f):+ raise FileNotFoundError(f'no such file: {f}')+ pr = audio.probe(path)+ if video is None and _is_video(path, pr):+ video = path+ elif video is not None and not _is_video(video, pr if video == path else audio.probe(video)):+ raise ValueError(f'--video {os.path.basename(video)} has no picture stream (an audio file?)')+ if not pr.get('codec') and not video:+ raise ValueError(f'{os.path.basename(path)} has no audio stream')+ cue_list, cue_fps = read_cues(cues)+ key = hashlib.sha1(f'{audio.file_id(path)}|{audio.file_id(video) if video else None}|{_cue_key(cues)}|{start}|'+ f'{end}|{detail}|{CODE_ID}|{CORE_ID}'.encode()).hexdigest()[:24]+ cfile = os.path.join(config.EMB_CACHE, 'timeline', f'{key}.json')+ if cache and not fresh and os.path.exists(cfile):+ try:+ with open(cfile, encoding='utf-8') as fh:+ return dict(json.load(fh), cached=True, file=path) # the path as given now+ except Exception:+ pass+ r = _analyse(path, pr, video, cue_list, cue_fps, start, end, DETAIL[detail], cache and not fresh)+ r['detail'] = detail+ r['summary'] = _summary(r)+ r['text'] = transcript(r)+ if cache or fresh:+ os.makedirs(os.path.dirname(cfile), exist_ok=True)+ tmp = f'{cfile}.{os.getpid()}.tmp'+ with open(tmp, 'w', encoding='utf-8') as fh:+ fh.write(to_json(r))+ audio.replace_quietly(tmp, cfile)+ return dict(r, cached=False)+++def _load_range(path, pr, start, end):+ """(x_context, c0, t0, t1, file_duration): a zoom [start, end] is analysed inside +-CONTEXT_S of context, so+ verdicts near its edges and the accent threshold match the full analysis"""+ name = os.path.basename(path)+ if not pr.get('codec'): # picture only: analyse silence of the same length+ fdur = float(pr.get('duration') or 0)+ t0, t1 = max(0.0, float(start or 0)), min(fdur, float(end)) if end is not None else fdur+ return np.zeros((int(round(fdur * SR)), 2), np.float32), 0.0, t0, t1, fdur+ if start is None and end is None:+ x = audio.load(path)+ fdur = len(x) / SR+ if fdur < 0.1:+ raise ValueError(f'{name} is {fdur:.2f} s long: too short for a timeline (>= 0.1 s); for a one-shot use '+ 'earshot measure / describe')+ return x, 0.0, 0.0, fdur, fdur+ fdur = float(pr.get('duration') or 0) or audio.duration(path)+ t0, t1 = max(0.0, float(start or 0)), min(fdur, float(end)) if end is not None else fdur+ if t1 - t0 < 0.1:+ raise ValueError(f"nothing to analyse between {t0:g} s and {'the end' if end is None else f'{end:g} s'}: "+ f'{name} is {fdur:.2f} s long')+ c0, c1 = max(0.0, t0 - CONTEXT_S), min(fdur, t1 + CONTEXT_S)+ return audio.load(path, start=c0, end=c1), c0, t0, t1, fdur+++def _analyse(path, pr, video, cue_list, cue_fps, start, end, cfg, reuse):+ clock, t_all = {}, time.time()++ def lap(name, t):+ clock[name] = round(time.time() - t, 2)+ return time.time()+ t = time.time()+ xc, c0, t0, t1, fdur = _load_range(path, pr, start, end)+ wa, wb = t0 - c0, min(t1 - c0, len(xc) / SR) # the window inside the context (context times)+ x = xc[int(round(wa * SR)): int(round(wb * SR))]+ dur = len(x) / SR+ t = lap('decode_s', t)+ loud = loudness(x, at_start=t0 <= 0.01, at_end=t0 + dur >= fdur - 0.01)+ t = lap('loudness_s', t)+ silent = bool(loud['digital_silence'])+ ev_c = [] if silent else detect_events(xc, lead=c0 == 0)+ lv = level_envs(xc)+ t = lap('events_s', t)+ grid = music_grid(audio.to_mono(xc))+ t = lap('beats_s', t)+ cuts, fps = [], None+ if video:+ from . import video as vid+ fps = vid.info(video).get('fps')+ cuts = [round(c - c0, 3) for c in vid.cuts(video) if t0 + 0.05 < c < t0 + dur - 0.05]+ cues_w = [dict(c, t=round(c['t'] - c0, 4)) for c in cue_list if t0 - 1e-6 <= c['t'] <= t0 + dur + 1e-6]+ t = lap('cuts_s', t)+ events = [e for e in ev_c if wa - 1e-6 <= e['t'] < wb]+ sid = span_id(path)+ tg = tag_events(xc, sid, c0, events, cfg['tag'], cfg['ast'], cuts + [c['t'] for c in cues_w], reuse, (wa, wb))+ t = lap('tag_events_s', t)+ hop = max(cfg['hop'], round((dur - cfg['win']) / max(1, cfg['max_win'] - 1), 2))+ if silent:+ secs, sn = ([{'start': 0.0, 'end': round(dur, 2), 'windows': 0, 'label': 'digital silence', 'lufs': None,+ 'digital_silence': True}] if dur >= 1.0 else []), {'windows': 0, 'ast': 0}+ else:+ secs, sn = sections(x, sid, t0, cfg['win'], hop, cfg['ast_sec'], reuse=reuse)+ t = lap('sections_s', t)+ beats = grid['beats'] if grid.get('confidence') in ('clear', 'weak') else None+ if beats:+ bt = np.asarray(beats)+ for e in events:+ e['beat_offset_ms'] = round((e['t'] - float(bt[int(np.argmin(np.abs(bt - e['t'])))])) * 1000, 1)+ sy = {}+ if cuts:+ sy['cuts'] = sync(cuts, ev_c, beats, 1.0 / fps if fps else None, lv)+ if cues_w:+ sy['cues'] = sync(cues_w, ev_c, beats, 1.0 / float(cue_fps) if cue_fps else None, lv)+ lap('total_s', t_all)+ _shift(loud, secs, t0)+ for e in events:+ e['t'] = round(e['t'] + c0, 3)+ for k in ('beats', 'downbeats'):+ grid[k] = [round(b + c0, 3) for b in grid[k] if wa - 1e-6 <= b <= wb + 1e-6]+ for v in sy.values():+ for it in v['items']:+ it['t'] = round(it['t'] + c0, 3)+ if it.get('event_t') is not None:+ it['event_t'] = round(it['event_t'] + c0, 3)+ v['stats']['no_sound'] = [round(s + c0, 3) for s in v['stats'].get('no_sound', [])]+ return {'file': path, 'video': video, 'start': round(t0, 3), 'end': round(t0 + dur, 3), 'duration': round(dur, 3),+ 'file_duration': round(fdur, 3), 'context': [round(c0, 3), round(c0 + len(xc) / SR, 3)],+ 'has_audio': bool(pr.get('codec')), 'lossy': bool(pr.get('lossy')), 'loudness': loud, 'music': grid,+ 'events': events, 'sections': secs,+ 'sync': sy, 'cuts': [round(c + c0, 3) for c in cuts],+ 'cut_method': 'frame difference (earshot.video.cuts)' if video else None, 'fps': fps,+ 'model_calls': {'clap_event_windows': tg['clap_windows'], 'ast_event_windows': tg['ast'],+ 'clap_section_windows': sn['windows'], 'ast_section_windows': sn['ast']},+ 'timing_s': clock}+++def _shift(loud, secs, t0):+ """window-relative loudness and section times -> file times"""+ def sh(v, nd=3):+ return round(v + t0, nd)+ for g in loud['silences']:+ g['start'], g['end'] = sh(g['start'], 2), sh(g['end'], 2)+ if 'max_momentary' in loud:+ loud['max_momentary']['t'] = sh(loud['max_momentary']['t'], 2)+ if 'max_short_term' in loud:+ ms = loud['max_short_term']+ ms['start'], ms['end'] = sh(ms['start'], 2), sh(ms['end'], 2)+ for k in ('momentary', 'short_term'):+ if k in loud:+ loud[k]['t0'] = sh(loud[k]['t0'])+ loud['clipping']['times'] = [sh(v) for v in loud['clipping']['times']]+ for s in secs:+ s['start'], s['end'] = sh(s['start'], 2), sh(s['end'], 2)+++def _sync_sentence(kind, st, zoomed):+ k1 = 'hit_within_1_frame' if 'hit_within_1_frame' in st else 'hit_within_40ms'+ c1 = st.get(k1.replace('hit', 'chance'))+ lab = '1 frame' if 'frame' in k1 else '40 ms'+ s = (f"Sync to {st['n']} {kind}: {100 * st[k1]:.0f}% have an onset within {lab}"+ + (f" (chance {100 * c1:.0f}%)" if c1 is not None else '')+ + f", {100 * st['hit_within_100ms']:.0f}% within 100 ms")+ if st.get('median_abs_offset_ms') is not None:+ s += f", median |offset| {st['median_abs_offset_ms']:.0f} ms"+ ka = k1.replace('hit', 'accent')+ if ka in st:+ s += (f"; {100 * st[ka]:.0f}% land on a strong accent (>= {st['accent_min_db']:.0f} dB over the bed"+ + (', ranked over the +-30 s context' if zoomed else '') + f") within {lab} "+ f"(chance {100 * st[ka.replace('accent', 'accent_chance')]:.0f}%)")+ if 'on_beat' in st:+ s += f"; {100 * st['on_beat']:.0f}% on the beat grid (chance {100 * st['on_beat_chance']:.0f}%)"+ if st.get('answered_by_swell'):+ s += f"; {st['answered_by_swell']} met by a whoosh / riser peak"+ if st.get('answered_by_level_peak'):+ s += f"; {st['answered_by_level_peak']} only by a level peak (soft whoosh / swell?)"+ if st.get('no_sound'):+ s += f"; {len(st['no_sound'])} without a hit: " + ', '.join(fmt_t(v) for v in st['no_sound'][:6])+ return s + '.'+++def _summary(r):+ lo, mu, ev = r['loudness'], r['music'], r['events']+ p = [f"{os.path.basename(r['file'])}: {fmt_t(r['start'])}-{fmt_t(r['end'])} ({r['duration']:.1f} s)."]+ if not r.get('has_audio', True):+ p.append(f"NO AUDIO STREAM: picture only ({len(r['cuts'])} picture cuts; every cut / cue is unsounded).")+ elif lo.get('digital_silence'):+ p.append('DIGITAL SILENCE: every sample is zero.')+ elif lo.get('integrated_lufs') is not None:+ s = f"Loudness {lo['integrated_lufs']:.1f} LUFS integrated, true peak {lo['true_peak_dbtp']:.1f} dBTP"+ if lo.get('lra_lu') is not None:+ s += f", LRA {lo['lra_lu']:.1f} LU"+ if lo.get('max_short_term'):+ ms = lo['max_short_term']+ s += f"; loudest {fmt_t(ms['start'])}-{fmt_t(ms['end'])} ({ms['lufs']:.1f} LUFS short-term)"+ p.append(s + '.')+ else:+ p.append(f"True peak {lo['true_peak_dbtp']:.1f} dBTP"+ + (' (below the -70 LUFS gate: near silence).' if r['duration'] >= 0.4 else+ ' (too short for a loudness measurement).'))+ if lo['clipping']['runs']:+ p.append(f"CLIPPING: {lo['clipping']['runs']} runs of clipped samples "+ f"(first at {fmt_t(lo['clipping']['times'][0])}).")+ elif lo.get('true_peak_dbtp') is not None and r.get('lossy') and lo['true_peak_dbtp'] > 0.0:+ p.append(f"True peak {lo['true_peak_dbtp']:+.1f} dBTP on decoded lossy audio: above 0 dBTP it clips on playback "+ '(the master should leave the mixer at <= -1 dBTP before encoding).')+ elif lo.get('true_peak_dbtp') is not None and r.get('lossy') and lo['true_peak_dbtp'] > -1.0:+ p.append(f"True peak {lo['true_peak_dbtp']:+.1f} dBTP on decoded lossy audio (codec overshoot included: fine for "+ 'a delivered file; a master should be <= -1 dBTP before encoding).')+ elif lo.get('true_peak_dbtp') is not None and lo['true_peak_dbtp'] > -1.0:+ p.append(f"True peak {lo['true_peak_dbtp']:.1f} dBTP is above -1 dBTP (risk of overs after encoding).")+ gaps = [g for g in lo['silences'] if not g.get('where')]+ edges = [g for g in lo['silences'] if g.get('where') and g['where'] != 'whole file']+ if gaps:+ p.append(f"{len(gaps)} stop-down(s)/gap(s): " + ', '.join(f"{fmt_t(g['start'])} {g['dur']:.1f} s {g['kind']}"+ for g in gaps[:5]) + '.')+ if edges:+ p.append('Quiet ' + ' and '.join(f"{g['where']} {g['dur']:.1f} s ({g['kind']})" for g in edges) + '.')+ if mu.get('tempo') and mu['confidence'] != 'none':+ alt = f" or {mu['tempo_alt']:.0f}" if mu.get('tempo_alt') else ''+ p.append(f"Beat grid: ~{mu['tempo']:.0f}{alt} BPM ({mu['confidence']} pulse, clarity "+ f"{mu['pulse_clarity']:.2f}, {len(mu['beats'])} beats).")+ elif not lo.get('digital_silence'):+ p.append('No steady beat grid' + (f" (pulse clarity {max(0.0, mu['pulse_clarity']):.2f} < {WEAK_PULSE})"+ if mu.get('pulse_clarity') is not None else '') + '.')+ if r['sections'] and not lo.get('digital_silence'):+ p.append(f"{len(r['sections'])} section(s): " + '; '.join(+ f"{fmt_t(s['start'])}-{fmt_t(s['end'])} {s['label']}" for s in r['sections'][:6]) + '.')+ tagged = [e for e in ev if e.get('kind')]+ if ev:+ kinds = {}+ for e in tagged:+ kinds[e['kind']] = kinds.get(e['kind'], 0) + 1+ ks = ', '.join(f'{v} {k.lower()}' for k, v in sorted(kinds.items(), key=lambda kv: -kv[1]))+ p.append(f"{len(ev)} events ({len(ev) / max(r['duration'], 1e-6):.1f}/s); {len(tagged)} tagged"+ + (f" ({ks}; tags are hints)" if tagged else '') + '.')+ acc = top_accents(ev)+ if acc:+ p.append('Strongest accents over a running bed: ' + ', '.join(+ f"{fmt_t(e['t'])} +{e['prominence_db']:.0f} dB" + (f" ({e['label']})" if e.get('label') else '')+ for e in acc) + '.')+ zoomed = r['context'] != [r['start'], r['end']]+ for kind in ('cuts', 'cues'):+ st = (r['sync'].get(kind) or {}).get('stats')+ if st and st.get('n'):+ p.append(_sync_sentence(kind, st, zoomed))+ return _ascii(' '.join(p))+++def add_cli(sub):+ from .cli import common+ p = sub.add_parser('timeline', help='listen through a soundtrack / film: events, sections, loudness, beats, sync')+ p.add_argument('file', help='audio file or video (its audio is analysed, its picture cuts are used)')+ p.add_argument('--video', help='video for picture cuts (default: FILE itself when it is a video)')+ p.add_argument('--cues', help='cue events: JSON file {fps, events:[{f, kind, id}]}, a JSON list, or "1.5,3.2,..." s')+ p.add_argument('--from', '--start', dest='start', type=float, help='zoom: start (s)')+ p.add_argument('--to', '--end', dest='end', type=float, help='zoom: end (s)')+ p.add_argument('--detail', choices=list(DETAIL), default='normal',+ help='model budget: low = 12 CLAP-tagged events, no AST (fastest); normal = 20 events + AST on the '+ 'top event and 3 section crops; high = up to 96 events (most important first), 16 AST event '+ 'labels, finer sections')+ p.add_argument('--fresh', action='store_true', help='recompute everything (no cached result or model outputs) and '+ 'store the new result')+ common(p)+ p.set_defaults(func=lambda a: emit(timeline(a.file, a.video, a.cues, a.start, a.end, a.detail, fresh=a.fresh), a)+ and 0)+++from ._callable import callable_module # noqa: E402+callable_module(__name__)diff --git a/earshot/video.py b/earshot/video.pynew file mode 100644index 0000000..dc4447e--- /dev/null+++ b/earshot/video.py@@ -0,0 +1,119 @@+"""Video helpers (ffmpeg, CPU): picture cuts, frames for contact strips, basic stream info.++ from earshot import video+ video.cuts('film.mp4') # [0.0, 1.83, 2.5, ...] hard cuts / big visual changes (seconds)+ video.frames('film.mp4', [0.5, 1.0]) # [PIL.Image, ...]++Cut detection (method='diff', default) reads 96x54 grey frames at the native frame rate and marks frame-difference+peaks that stand >= 4x above the median of the surrounding second (and above 1.2 % grey change), or any change of+>= 15 %, at most one per 0.12 s. Unlike ffmpeg's scene score (method='scene', threshold 0.3), it also catches cuts and+whip transitions on white-background motion graphics, where the scene score finds almost nothing. Slow crossfades are+missed or found late; treat the result as "big visual change" markers, not an edit decision list.+"""+import hashlib+import json+import os+import re+import subprocess++import numpy as np++from . import audio, config++FRAME_W, FRAME_H = 96, 54+++def has_video(path):+ return bool(audio.probe(path).get('has_video'))+++def info(path):+ p = audio.probe(path)+ v = p.get('video') or {}+ return {'duration': p.get('duration'), 'fps': v.get('fps'), 'width': v.get('width'), 'height': v.get('height'),+ 'has_audio': bool(p.get('codec')), 'has_video': p.get('has_video', False)}+++def _cache(name):+ config.ensure_dirs()+ d = os.path.join(config.EMB_CACHE, 'video')+ os.makedirs(d, exist_ok=True)+ return os.path.join(d, name)+++def frame_diff(path):+ """(fps, d): mean absolute difference (0..1) between consecutive 96x54 grey frames (d[0] = 0), cached"""+ f = _cache('diff_' + hashlib.sha1(f'{audio.file_id(path)}|{FRAME_W}x{FRAME_H}|v1'.encode()).hexdigest()[:20] + '.npz')+ if os.path.exists(f):+ try:+ with np.load(f, allow_pickle=False) as z:+ return float(z['fps']), np.array(z['d'])+ except Exception:+ pass+ fps = (audio.probe(path).get('video') or {}).get('fps') or 30.0+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-an', '-vf',+ f'scale={FRAME_W}:{FRAME_H}:flags=area,format=gray', '-f', 'rawvideo', '-'], capture_output=True)+ if p.returncode != 0 or not p.stdout:+ raise audio.AudioError(f'cannot decode the picture of {path}: {p.stderr.decode(errors="replace")[:200]}')+ fr = np.frombuffer(p.stdout, np.uint8)+ fr = fr[: len(fr) // (FRAME_W * FRAME_H) * FRAME_W * FRAME_H].reshape(-1, FRAME_H, FRAME_W).astype(np.float32)+ d = np.concatenate([[0.0], np.abs(np.diff(fr, axis=0)).mean(axis=(1, 2)) / 255]).astype(np.float32)+ tmp = f + f'.{os.getpid()}.tmp.npz'+ np.savez(tmp, fps=fps, d=d)+ audio.replace_quietly(tmp, f)+ return float(fps), d+++def diff_events(d, fps, ratio=4.0, floor=0.012, big=0.15, gap=0.12):+ """frame indices of cuts / big picture changes in a frame-difference signal (see module doc)"""+ from scipy.ndimage import maximum_filter1d, median_filter+ if len(d) < 3:+ return np.zeros(0, int)+ med = median_filter(d, size=int(round(fps)) | 1, mode='nearest')+ loc = maximum_filter1d(d, size=3, mode='nearest')+ out = []+ for i in np.where((d == loc) & (((d > floor) & (d > ratio * med + 0.004)) | (d >= big)))[0]:+ if i == 0:+ continue+ if out and i - out[-1] < gap * fps:+ if d[i] > d[out[-1]]:+ out[-1] = int(i)+ continue+ out.append(int(i))+ return np.array(out, int)+++def cuts(path, method='diff', threshold=0.3, min_gap=0.2, cache=True):+ """times (s) of picture cuts / big visual changes; method 'diff' (default) or 'scene' (ffmpeg scene score)"""+ if method == 'diff':+ fps, d = frame_diff(path)+ return [round(float(i) / fps, 3) for i in diff_events(d, fps)]+ k = hashlib.sha1(f'{audio.file_id(path)}|{threshold}|{min_gap}|v1'.encode()).hexdigest()[:20]+ f = _cache(f'cuts_{k}.json')+ if cache and os.path.exists(f):+ return json.load(open(f))+ p = subprocess.run(['ffmpeg', '-hide_banner', '-nostdin', '-i', path, '-an', '-vf',+ f"scale=320:-2,select='gt(scene,{threshold})',showinfo", '-f', 'null', '-'],+ capture_output=True, text=True, errors='replace')+ ts = [float(m) for m in re.findall(r'pts_time:([0-9.]+)', p.stderr)]+ out = []+ for t in ts:+ if not out or t - out[-1] >= min_gap:+ out.append(round(t, 3))+ json.dump(out, open(f, 'w'))+ return out+++def frame(path, t, width=320):+ """one frame at time t as a PIL image"""+ from PIL import Image+ import io+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-ss', f'{max(0.0, t):.3f}', '-i', path, '-frames:v', '1',+ '-vf', f'scale={width}:-2', '-f', 'image2pipe', '-vcodec', 'png', '-'], capture_output=True)+ if p.returncode != 0 or not p.stdout:+ return None+ return Image.open(io.BytesIO(p.stdout)).convert('RGB')+++def frames(path, times, width=320):+ return [frame(path, t, width) for t in times]diff --git a/earshot/vocab.py b/earshot/vocab.pynew file mode 100644index 0000000..08ed471--- /dev/null+++ b/earshot/vocab.py@@ -0,0 +1,379 @@+"""Listening vocabulary and score calibration: raw CLAP cosines -> numbers that mean the same for every sound and query.++ from earshot import vocab+ v = vocab.get() # data/vocab.json terms, embedded once (cached on disk)+ a = vocab.embed_file('hit.ogg') # (512,) audio embedding, cached+ s = v.scores(a[None], ['a glass ting']) # cosine, z, percentile, rank of the query among the vocab terms+ v.top_terms(a, k=8) # best vocab terms for this sound, calibrated++Why calibrate: a raw CLAP cosine is not comparable across sounds (dense, busy sounds score high on every text) or+across descriptions (some phrases score high for every sound). Every score here is+ u = cos(a, q) - m.q term bias removed: m.q = mean cosine of description q over 241 reference+ recordings (the 219-file tuning library, 7 pro launch films, a film soundtrack ++ stems, synth stems; every source folder weighted equally). Same as centring audio+ embeddings on the reference mean m (data/vocab_stats.json), so it works for ANY+ query, not only vocabulary terms.+ z = (u - mean_t u_t) / std_t u_t per-sound z over all 322 vocab terms t: how far q stands out among the+ vocabulary descriptions for this very sound.+Chosen by measurement (eval/semantics/RESULTS.md). Tuning library (219 labelled real recordings): the per-sound z is+the big win (pooled AUC of (recording, description) pairs 0.951 -> 0.963), centring adds a little (AUC 0.966,+category MAP 0.770 -> 0.778; class MAP 0.650 -> 0.642); dividing by each description's std over the references hurt+and was dropped; 5 prompt templates beat 1 (top-1 tagging 0.553 -> 0.594). Held-out library (125 real recordings,+nothing tuned on it): pooled AUC 0.980 (raw cosine 0.978), class MAP 0.646 (raw cosine 0.700: the z helps absolute+verdicts more than ranking files). Audio: 10 s or shorter = one CLAP window, repeat-padded (PAD; zero padding: AUC+0.964, class top-3 0.843 vs 0.886); longer = mean of 10 s windows (hop 5 s), at most MAX_WIN evenly spaced.++Rebuild the reference statistics after changing the model or the reference set (vocab edits need no rebuild):+ python -m earshot.vocab --build-stats REF_FOLDERS_OR_FILES...+"""+import hashlib+import json+import os+import sys+import threading++import numpy as np++from . import audio, config++VOCAB_PATH = os.path.join(config.DATA_DIR, 'vocab.json')+STATS_PATH = os.path.join(config.DATA_DIR, 'vocab_stats.json')+PAD = 'repeatpad'+PADS = ('repeatpad', 'zero', 'spaced')+SPACED_S = 1.0+MAX_WIN = 6+WIN, HOP = 10.0, 5.0+METHODS = ('raw', 'sound', 'center', 'both')+# categories whose tags were never validated (the evaluation has no labelled music genre / mood / instrument / speech+# data): reported as hints, never with a verdict+UNVALIDATED = ('music_genre', 'music_mood', 'instruments', 'voice')+_lock = threading.Lock()+_inst = None+++def load_vocab(path=VOCAB_PATH):+ with open(path, encoding='utf-8') as fh:+ return json.load(fh)+++def load_stats(path=STATS_PATH):+ if not os.path.exists(path):+ return None+ with open(path, encoding='utf-8') as fh:+ return json.load(fh)+++def _key(*parts):+ return hashlib.sha1('|'.join(map(str, parts)).encode()).hexdigest()[:24]+++def _clap():+ from .models import clap+ return clap.get()+++def _cache_dir():+ d = os.path.join(config.EMB_CACHE, 'semantics', config.CLAP_MODEL.replace('/', '--') + '--' + config.PRECISION)+ os.makedirs(d, exist_ok=True)+ return d+++# ---------------------------------------------------------------------------------------------------- input checks+def check(path, start=None, end=None):+ """duration of `path` after validating [start, end); raises audio.AudioError (missing / undecodable) or ValueError"""+ dur = audio.duration(path)+ name = os.path.basename(path)+ if start is not None and start < 0:+ raise ValueError(f'start {start:.3f} s is negative')+ if start is not None and start >= dur:+ raise ValueError(f'start {start:.3f} s is at or past the end of {name} ({dur:.3f} s long)')+ if end is not None and end <= (start or 0.0):+ raise ValueError(f'end {end:.3f} s must be after start {start or 0.0:.3f} s')+ return dur+++def readable(paths):+ """(decodable paths, [{path, error}] for the rest): one bad file must not sink a whole batch"""+ ok, bad = [], []+ for p in paths:+ try:+ audio.duration(p)+ ok.append(p)+ except (audio.AudioError, OSError) as e:+ bad.append({'path': p, 'error': str(e).strip().splitlines()[0][:200]})+ return ok, bad+++def zf(z):+ """z for text, floored to 0.1 so it can never contradict a verdict threshold (2.96 -> '+2.9', not '+3.0')"""+ return f'{np.floor(float(z) * 10 + 1e-9) / 10:+.1f}'+++def peak_db(path, start=None, end=None):+ """sample peak of the mono decode in dB (for the near-silence guard; levels proper come from the stereo decode)"""+ x = audio.load(path, mono=True, start=start, end=end)+ return audio.db(float(np.abs(x).max()) if len(x) else 0.0)+++def run_cli(fn, a, text=None):+ """run a CLI handler: expected problems (missing / undecodable file, bad range) as one plain line, also as JSON+ with --json, exit 1; anything else is left to earshot.cli"""+ from .report import emit, error_line+ try:+ r = fn()+ except (audio.AudioError, ValueError, OSError) as e:+ msg = ' '.join(str(e).split()) or type(e).__name__+ error_line(getattr(a, 'command', None) or '', msg)+ if getattr(a, 'json', False): # the JSON error stays on stdout for a caller parsing --json+ sys.stdout.write(json.dumps({'summary': f'error: {msg}', 'error': msg}) + '\n')+ return 1+ emit(r, a, text(r) if text else r['summary'])+ return 0+++# ---------------------------------------------------------------------------------------------------- audio embedding+def spans_for(start, end, max_win=MAX_WIN):+ """CLAP windows [(s, e)] covering [start, end): one span if <= 10 s, else 10 s windows hop 5 s (last one flush with+ the end), at most max_win of them evenly spaced"""+ n, w, h = int(round((end - start) * audio.SR)), int(WIN * audio.SR), int(HOP * audio.SR)+ if n <= w:+ return [(start, end)]+ st = list(range(0, n - w + 1, h))+ if (n - w) % h:+ st.append(n - w)+ if max_win and len(st) > max_win:+ st = [st[int(round(i))] for i in np.linspace(0, len(st) - 1, max_win)]+ return [(start + s / audio.SR, start + (s + w) / audio.SR) for s in st]+++def _pad_key(pad, dur):+ """cache key of the padding actually applied to a span of `dur` seconds"""+ if dur >= WIN - 1e-3:+ return '-'+ return 'repeatpad' if pad == 'spaced' and dur >= SPACED_S - 1e-3 else pad+++def embed_spans(items, pad=PAD, batch=8):+ """items = [(path, start, end)] -> (n, 512): every span is one CLAP window (<= 10 s), padded to 10 s with `pad`:+ 'repeatpad' (tile the sound), 'zero' (silence after it) or 'spaced' (silence up to 1 s, then tile: one hit per+ second); cached per span on disk and computed in batches"""+ d = _cache_dir()+ out = [None] * len(items)+ todo = []+ for i, (p, s, e) in enumerate(items):+ f = os.path.join(d, _key(audio.file_id(p), round(s, 4), round(e, 4), _pad_key(pad, e - s)) + '.npy')+ if os.path.exists(f):+ try:+ out[i] = np.load(f)+ continue+ except Exception:+ pass+ todo.append((i, p, s, e, f))+ loaded = {}+ for k in range(0, len(todo), batch):+ chunk = todo[k:k + batch]+ segs = []+ for _, p, s, e, _ in chunk:+ if p not in loaded:+ loaded = {p: audio.load(p, mono=True)}+ seg = loaded[p][int(round(s * audio.SR)):int(round(e * audio.SR))]+ if pad == 'spaced' and len(seg) < SPACED_S * audio.SR:+ seg = np.pad(seg, (0, int(SPACED_S * audio.SR) - len(seg)))+ segs.append(seg)+ es = _clap().embed_audio(segs, pad='repeatpad' if pad == 'spaced' else pad, batch=batch)+ for (i, _, _, _, f), v in zip(chunk, es):+ out[i] = v+ tmp = f'{f}.{os.getpid()}.tmp.npy'+ np.save(tmp, v)+ audio.replace_quietly(tmp, f)+ return np.stack(out) if out else np.zeros((0, 512), np.float32)+++def _norm(m):+ return m / (np.linalg.norm(m, axis=-1, keepdims=True) + 1e-12)+++def embed_files(paths, pad=PAD, max_win=MAX_WIN, ranges=None):+ """(n, 512) file (or [start, end) range) embeddings = mean of their CLAP windows; ranges: [(start, end)] or None"""+ items, owner = [], []+ for j, p in enumerate(paths):+ s, e = (ranges[j] if ranges else (None, None))+ dur = audio.duration(p)+ s = max(0.0, s or 0.0)+ e = dur if e is None else min(dur, e)+ for sp in spans_for(s, max(e, s + 0.01), max_win):+ items.append((p, sp[0], sp[1]))+ owner.append(j)+ E = embed_spans(items, pad)+ owner = np.array(owner)+ return _norm(np.stack([E[owner == j].mean(axis=0) for j in range(len(paths))]))+++def embed_file(path, start=None, end=None, pad=PAD, max_win=MAX_WIN):+ return embed_files([path], pad, max_win, [(start, end)])[0]+++# ---------------------------------------------------------------------------------------------------- vocabulary+class Vocab:+ def __init__(self, templates=None, stats=None, categories=None):+ d = load_vocab()+ self.templates = list(templates or d['templates'])+ self.cats = {c: list(ts) for c, ts in (categories or d['categories']).items()}+ self.names = [t for ts in self.cats.values() for t in ts]+ self.cat_of = {t: c for c, ts in self.cats.items() for t in ts}+ self.E = self.embed_text(self.names)+ self.stats = load_stats() if stats is None else (stats or None)+ if self.stats and self.stats.get('model') != config.CLAP_MODEL:+ self.stats = None+ self.m = np.array(self.stats['mean'], np.float32) if self.stats else None+ self.mu_E = self.bias(self.E)++ def embed_text(self, texts):+ """(n, 512): each phrase through every template, averaged and renormalised. Cached on disk (one file per+ phrase list > 16, else one per phrase), so a repeated query needs no model load"""+ texts = [texts] if isinstance(texts, str) else list(texts)+ if not texts:+ return np.zeros((0, 512), np.float32)+ d = _cache_dir()+ keys = ([_key(json.dumps([self.templates, texts]))] if len(texts) > 16 else+ [_key(json.dumps([self.templates, [t]])) for t in texts])+ files = [os.path.join(d, 't_' + k + '.npy') for k in keys]+ got = []+ for f in files:+ try:+ got.append(np.load(f) if os.path.exists(f) else None)+ except Exception:+ got.append(None)+ if all(g is not None for g in got):+ v = np.concatenate([np.atleast_2d(g) for g in got])+ if v.shape == (len(texts), 512):+ return v+ T = len(self.templates)+ e = _clap().embed_text([tp.format(t) for t in texts for tp in self.templates])+ v = _norm(e.reshape(len(texts), T, -1).mean(axis=1)).astype(np.float32)+ for f, part in zip(files, [v] if len(files) == 1 and len(texts) > 16 else v[:, None, :]):+ tmp = f'{f}.{os.getpid()}.tmp.npy'+ np.save(tmp, part if len(texts) > 16 else part[0])+ audio.replace_quietly(tmp, f)+ return v++ def nearest_category(self, Q):+ """category of the vocabulary term whose text is closest to each query embedding (what kind of description)"""+ return [self.cat_of[self.names[i]] for i in np.argmax(np.atleast_2d(Q) @ self.E.T, axis=1)]++ def bias(self, Q):+ """term bias = mean cosine of each row of Q (n, 512) over the reference recordings (= Q . their mean embedding)"""+ Q = np.atleast_2d(Q)+ return Q @ self.m if self.m is not None else np.zeros(len(Q), np.float32)++ def calibrate(self, A, Q=None, method='both'):+ """A (n, 512) audio, Q (m, 512) query embeddings -> dict of (n, T) term arrays and (n, m) query arrays:+ cos, u (bias removed when method is 'center' / 'both'), z (per-sound z when 'sound' / 'both', else u),+ pct (share of vocab terms the query beats)"""+ A = np.atleast_2d(A)+ C = A @ self.E.T+ U = C - self.mu_E if method in ('center', 'both') else C+ r = {'cos_t': C, 'u_t': U}+ if method in ('sound', 'both'):+ mu, sd = U.mean(1, keepdims=True), U.std(1, keepdims=True) + 1e-9+ else:+ mu, sd = 0.0, 1.0+ r['z_t'] = (U - mu) / sd+ if Q is not None:+ Q = np.atleast_2d(Q)+ Cq = A @ Q.T+ Uq = Cq - self.bias(Q) if method in ('center', 'both') else Cq+ r['cos_q'], r['u_q'], r['z_q'] = Cq, Uq, (Uq - mu) / sd+ r['pct_q'] = (U[:, None, :] < Uq[:, :, None]).mean(-1)+ return r++ def scores(self, A, queries, method='both'):+ """per sound (rows of A) and query: cosine, z, percentile among vocab terms, rank_in_vocab"""+ Q = self.embed_text(list(queries))+ r = self.calibrate(A, Q, method)+ out = []+ for i in range(len(np.atleast_2d(A))):+ row = []+ for j, q in enumerate(queries):+ beats = int((r['u_t'][i] > r['u_q'][i, j]).sum())+ row.append({'query': q, 'cosine': round(float(r['cos_q'][i, j]), 3), 'z': round(float(r['z_q'][i, j]), 2),+ 'percentile': round(100 * float(r['pct_q'][i, j]), 1), 'rank_in_vocab': beats + 1,+ 'vocab_size': len(self.names)})+ out.append(row)+ return out++ def top_terms(self, a, k=8, cats=None, method='both', exclude=()):+ """[(term, category, z, cosine)] best calibrated vocab terms for one sound"""+ r = self.calibrate(a, method=method)+ z, c = r['z_t'][0], r['cos_t'][0]+ idx = [i for i in np.argsort(-z) if (cats is None or self.cat_of[self.names[i]] in cats)+ and self.cat_of[self.names[i]] not in exclude]+ return [(self.names[i], self.cat_of[self.names[i]], round(float(z[i]), 2), round(float(c[i]), 3)) for i in idx[:k]]+++def logit_scale():+ """CLAP's softmax temperature (stored with the reference stats, so no model load is needed)"""+ st = load_stats()+ return float(st['logit_scale']) if st and st.get('logit_scale') else float(_clap().logit_scale)+++def get(templates=None):+ global _inst+ with _lock:+ if _inst is None or (templates and list(templates) != _inst.templates):+ _inst = Vocab(templates)+ return _inst+++# ---------------------------------------------------------------------------------------------------- reference stats+def group_weights(paths):+ """weight 1 / (files in the same folder): every source folder counts equally, whatever its size"""+ folders = [os.path.normcase(os.path.dirname(os.path.abspath(p))) for p in paths]+ n = {f: folders.count(f) for f in set(folders)}+ return np.array([1.0 / n[f] for f in folders])+++def build_stats(A, paths, meta=None, path=STATS_PATH, templates=None):+ """reference statistics from audio embeddings A (n, 512) of `paths`: the folder-balanced mean embedding (all the+ calibration uses) + the resulting mean / std cosine of every vocab term (derived, for inspection); written to+ `path` unless it is None"""+ A = np.asarray(A, np.float64)+ w = group_weights(paths)+ w /= w.sum()+ m = (w[:, None] * A).sum(0)+ v = Vocab(templates, stats=False)+ C = A @ v.E.T.astype(np.float64)+ mu = (w[:, None] * C).sum(0)+ sd = np.sqrt((w[:, None] * (C - mu) ** 2).sum(0))+ res = dict(meta or {}, model=config.CLAP_MODEL, precision=config.PRECISION, pad=PAD, templates=v.templates,+ logit_scale=round(float(_clap().logit_scale), 4), n_ref=len(A),+ n_folders=len({os.path.dirname(os.path.abspath(p)) for p in paths}),+ mean=[round(float(x), 5) for x in m],+ term_cos={t: [round(float(a), 4), round(float(b), 4)] for t, a, b in zip(v.names, mu, sd)})+ if path:+ lines = [f' {json.dumps(k)}: {json.dumps(x)}' for k, x in res.items() if k != 'term_cos']+ lines.append(' "term_cos": {' + ','.join(f'\n {json.dumps(t)}: {json.dumps(x)}'+ for t, x in res['term_cos'].items()) + '\n }')+ with open(path, 'w', encoding='utf-8') as fh:+ fh.write('{\n' + ',\n'.join(lines) + '\n}\n')+ return res+++def main(argv=None):+ import argparse+ ap = argparse.ArgumentParser(prog='python -m earshot.vocab', description=__doc__.splitlines()[0])+ ap.add_argument('--build-stats', nargs='+', metavar='REF', help='reference recordings (files / folders)')+ a = ap.parse_args(argv)+ if not a.build_stats:+ v = get()+ print(f'{len(v.names)} terms in {len(v.cats)} categories; reference stats: '+ f'{"%d recordings" % v.stats["n_ref"] if v.stats else "none"}')+ return 0+ paths = audio.list_media(a.build_stats)+ r = build_stats(embed_files(paths, max_win=3), paths, {'reference_set': ' '.join(a.build_stats), 'max_win': 3})+ print(f'wrote {STATS_PATH}: {r["n_ref"]} reference recordings from {r["n_folders"]} folders')+ return 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/eval/README.md b/eval/README.mdnew file mode 100644index 0000000..e168883--- /dev/null+++ b/eval/README.md@@ -0,0 +1,56 @@+# earshot evaluations++Each folder holds the scripts that measured one part of earshot, the raw numbers (`*.json`) and a written report+(`RESULTS.md`) with the exact commands. The headline numbers in the main README and in `docs/VALIDATION.md` come from+here.++| folder | what it measures | report |+|---|---|---|+| `semantics/` | text-audio matching (`describe`, `match`, `rank`, `find`): verdict precision, likelihood ratios, top-k retrieval on a tuning library and a held-out library labelled before scoring, query-kind reading, find trims, bf16 vs fp32 precision, timing | [RESULTS.md](semantics/RESULTS.md) |+| `timeline/` | the listening transcript and the sheet: event recall on a real 60 s film mix with known placements (quiet events under music too), tags, sections, tempo / beat grid, sync of hits to picture cuts and cue frames, run time on 30-90 s films | [RESULTS.md](timeline/RESULTS.md) |+| `realness/` | the real-vs-code-synthesized detector: held-out ROC-AUC with whole sound families held out, robustness to noise, room, gating, length and mp3, firm-verdict error rates, a tone guard on real recordings | [RESULTS.md](realness/RESULTS.md) |+| `launch/` | the launch profile and `critique` scorecard: leave-one-out over the 7 reference films, a film with a real vs a synthesized soundtrack, known-bad mixes (quiet, hot, mono, muffled, late, dead air, fade-out, hole, over-compressed ...), the visual-event detector vs scene cuts | [RESULTS.md](launch/RESULTS.md) |++## The data is not redistributed++The evaluations ran on licensed recordings and third-party films. None of it is in this repo, and none of it may be+redistributed from here. Only derived numbers are committed. What was used:++- **A library of real licensed recordings** (Freesound / Mixkit) in 8 categories: ambience, foley, garden, keyboard,+ magic, motion, music, toy. 219 files were used for tuning; 126 real recordings that were rejected for quality form+ the held-out set. Each file has a manifest entry (category, title, need, notes).+- **A finished short film project** with a real-recordings soundtrack: the 60 s mix, its stems (sfx, ambience, music),+ a mix report with every placement (time and sample), picture cue events at 60 fps, and the same film with an older+ code-synthesized soundtrack and its stems.+- **Code-synthesized one-shots** rendered with an open synth engine (kick, clap, tick, key click, UI click, FM bell,+ whoosh, riser, drone ...).+- **7 professional launch films** as the quality bar, measured only. They are called `ref_1` .. `ref_7` everywhere,+ with a neutral style line each in `launch/refs.json`. The shipped profile (`earshot/data/launch_profile.json`) holds+ their numbers only: no media, no file names.++## Running the scripts on your own data++The scripts read the data locations from environment variables, or from a gitignored `eval/local_paths.json`+(environment variables win). See `evalpaths.py` for the defaults.++| variable | `local_paths.json` key | what it points to |+|---|---|---|+| `EARSHOT_EVAL_ASSETS` | `assets` | a library of real recordings: `<category>/`, `_rejected/<category>/`, `manifests/*.json` |+| `EARSHOT_EVAL_FILM` | `film_project` | a film project: `out/audio/soundtrack.wav`, `out/audio/stems_pro/{sfx,amb,music}.wav`, `out/qa/mix_report.json`, `out/cues_all.json`, the film as `out/Clawd_1080p60.mp4`, the synthesized versions under `audio/_retired_synth/`, `out/audio/stems/` and `out/superseded/` |+| `EARSHOT_EVAL_REFS` | `refs_dir`, or `refs: {"ref_1": "/any/file.mp4", ...}` | your reference launch films, named `ref_1.mp4` .. `ref_7.mp4` |+| `EARSHOT_EVAL_SYNTH` | `synth_scripts` | a folder with the synth engine scripts (`synth.py`, `score.py`), only for `realness/build_data.py` |++Example `eval/local_paths.json`:++```json+{"assets": "/data/sfx-library",+ "film_project": "/data/my-film",+ "refs": {"ref_1": "/data/films/launch_a.mp4", "ref_2": "/data/films/launch_b.mp4"}}+```++To build a launch profile from films you admire you do not need any of this: `earshot profile a.mp4 b.mp4 ... -o+my_profile.json`, then `earshot critique film.mp4 --profile my_profile.json`.++Run every script from the repo root with the repo's Python, e.g.+`EARSHOT_THREADS=2 .venv/bin/python eval/launch/run_eval.py --help`. Scratch audio goes to `out/` (gitignored). Your+numbers will differ from the committed ones unless your data matches the original.diff --git a/eval/evalpaths.py b/eval/evalpaths.pynew file mode 100644index 0000000..273f0cd--- /dev/null+++ b/eval/evalpaths.py@@ -0,0 +1,52 @@+"""Where the evaluation data lives on your machine.++The evaluation data (licensed recordings, a film project with its stems and mix report, and 7 third-party launch+films) is not redistributed. Each script reads the locations from environment variables or from a gitignored+`eval/local_paths.json`; see eval/README.md. Environment variables win over the file, the file wins over the defaults.++ EARSHOT_EVAL_FILM film project root: audio/assets/, audio/_retired_synth/, out/audio/, out/qa/mix_report.json,+ out/cues_all.json, out/Clawd_1080p60.mp4 ... (default eval/data/film)+ EARSHOT_EVAL_ASSETS library of real recordings: <category>/, _rejected/<category>/, manifests/+ (default <film>/audio/assets)+ EARSHOT_EVAL_REFS folder with the reference launch films ref_1.mp4 .. ref_7.mp4 (default eval/data/refs)+ EARSHOT_EVAL_SYNTH folder of the synth engine scripts (synth.py, score.py) (default eval/data/synth)++local_paths.json may also map single reference films to any file: {"refs": {"ref_1": "/films/a.mp4", ...}}.+"""+import json+import os++HERE = os.path.dirname(os.path.abspath(__file__))+LOCAL = os.path.join(HERE, 'local_paths.json')+try:+ with open(LOCAL, encoding='utf-8') as fh:+ _CFG = json.load(fh)+except FileNotFoundError:+ _CFG = {}+++def _get(key, env, default):+ return (os.environ.get(env) or _CFG.get(key) or default).replace('\\', '/').rstrip('/')+++FILM = _get('film_project', 'EARSHOT_EVAL_FILM', os.path.join(HERE, 'data', 'film'))+ASSETS = _get('assets', 'EARSHOT_EVAL_ASSETS', FILM + '/audio/assets')+REFS_DIR = _get('refs_dir', 'EARSHOT_EVAL_REFS', os.path.join(HERE, 'data', 'refs'))+SYNTH = _get('synth_scripts', 'EARSHOT_EVAL_SYNTH', os.path.join(HERE, 'data', 'synth'))+REF_IDS = tuple(f'ref_{i}' for i in range(1, 8))+++def ref(rid):+ """path of reference launch film `rid` ('ref_1' .. 'ref_7')"""+ if not os.environ.get('EARSHOT_EVAL_REFS') and rid in (_CFG.get('refs') or {}):+ return _CFG['refs'][rid].replace('\\', '/')+ return f'{REFS_DIR}/{rid}.mp4'+++def anon(path):+ """a reference film's local path -> 'refs/ref_N.mp4' (for anything written into committed results)"""+ p = str(path).replace('\\', '/')+ for rid in REF_IDS:+ if p == ref(rid):+ return f'refs/{rid}.mp4'+ return pathdiff --git a/eval/launch/RESULTS.md b/eval/launch/RESULTS.mdnew file mode 100644index 0000000..61e71ec--- /dev/null+++ b/eval/launch/RESULTS.md@@ -0,0 +1,160 @@+# Launch profile + critique: evaluation++Generated by `eval/launch/run_eval.py` (numbers only; the films are the user's local reference files, never copied). Profile = `earshot/data/launch_profile.json`, built from the 7 references below.++## Commands++```+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_1,ref_2,ref_3 --fresh --measure-only+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_4,ref_5,ref_6,ref_7,clawd_real,clawd_synth --fresh --measure-only+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --build # profile, LOO, Clawd, this file+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg base,quiet,hot,mono,muffled,late,deadair,fadeout,hole,squashed+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sfx_only,amb_only,synth_bed+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sine,pink+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py # rewrite this file from the caches+```++## What was tuned on what++- **Detector thresholds** (salient hit: >= 6 dB jump of the 100 ms level within 80 ms, landing within 8 LU of integrated, one hit per run, timed at the attack; visual event: frame-difference peak >= 4x the median of the surrounding second or >= 0.15, now earshot.video.diff_events; stop-down: >= 10 dB dip for 0.15-1.5 s) were chosen by looking at plots of all 9 films (7 refs + 2 Clawd), at the Clawd mix-report placements (hits) and at ffmpeg scene cuts (visual events). The Clawd real soundtrack is therefore partly tuning data for the hit detector.+- **Music threshold** (AudioSet Music >= 0.3 per 5 s window) was set after looking at the per-window probabilities of all 9 films (music under voice or SFX scored 0.4-0.7, one sound-design stretch 0.1-0.15).+- **Changed after the first leave-one-out run** (first build; not a blind test of these choices): true peak went from critical to major and is judged against 0 dBTP for decoded lossy files (the -1 dBTP master rule failed all 7 AAC-decoded references); a critical rule missed by <= 2 tolerances counts as major; the metric max_gap_s was added; the music_frac tolerance went from 0.1 to 0.05 after an SFX-only stem (music in 42 % of windows) came out as only a near miss.+- **Changed after the independent review, before the known-bad set below was first run:** p_real uses the realness model's own thresholds (ok at its 'real or pro-produced' threshold, critical below 0.5, minor in between; no near-miss downgrade); a density range that would reach 0 keeps a floor at half the sparsest reference (salient hits >= 0.05 /s); a mono mix (identical channels) counts as -40 dB stereo width instead of n.a.; files under 10 s get no loudness-shape rows; brightness_hz replaces the power-weighted centroid (info only). The known-bad expectations were written into run_eval.py before its first run; nothing was changed after seeing its results.+- **Not adjusted after seeing results:** the other tolerances, the weights of the other metrics and the verdict thresholds (90 / 75 / 50 %).+- **Ranges** are data: min-max of the reference films widened by each metric's tolerance. Loudness and true peak use web-delivery practice (-16..-9 LUFS; <= -1 dBTP for a master, <= 0 dBTP for decoded AAC / MP3) instead, so references can fail them.++## Per-film measurements++| film | dur s | LUFS | dBTP | LRA | music | speech | onsets/s | hits/s | vis events/s | events w/ sound (chance) | audio pass s (cold) | picture pass s | AudioSet pass s (cold, first build) | warm s |+|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|+| ref_1 | 93.1 | -12.1 | 0.1 | 1.7 | 100% | 95% | 5.87 | 1.56 | 0.3 | 89% (90%) | 49.8 | 0.0 | 144.4 | 0.4 |+| ref_2 | 28.9 | -14.8 | 1.0 | 3.6 | 100% | 83% | 5.98 | 1.21 | 0.31 | 67% (89%) | 9.7 | 0.0 | 37.0 | 0.7 |+| ref_3 | 20.5 | -10.4 | 0.6 | 6.4 | 100% | 100% | 3.81 | 1.66 | 0.83 | 88% (68%) | 5.6 | 0.1 | 23.0 | 0.7 |+| ref_4 | 32.3 | -14.8 | -0.8 | 7.0 | 100% | 0% | 8.22 | 1.18 | 1.3 | 100% (99%) | 18.1 | 0.0 | 58.6 | 0.3 |+| ref_5 | 29.5 | -22.2 | 1.2 | 12.2 | 67% | 0% | 4.03 | 1.25 | 0.54 | 50% (48%) | 4.0 | 1.9 | 41.6 | 0.2 |+| ref_6 | 49.3 | -17.4 | 0.1 | 5.5 | 100% | 100% | 5.66 | 1.3 | 0.63 | 81% (86%) | 5.8 | 14.2 | 54.5 | 0.2 |+| ref_7 | 41.8 | -7.3 | 2.0 | 3.3 | 89% | 0% | 2.46 | 0.1 | 1.6 | 60% (44%) | 5.8 | 5.1 | 98.4 | 0.3 |+| clawd_real | 60.0 | -15.0 | -1.4 | 10.4 | 100% | 0% | 2.48 | 0.27 | 0.43 | 62% (37%) | 8.0 | 0.0 | 78.9 | 0.2 |+| clawd_synth | 60.0 | -14.0 | -2.1 | 7.2 | 100% | 0% | 2.1 | 0.38 | 0.43 | 54% (43%) | 8.1 | 25.4 | 76.3 | 0.6 |++Audio pass = decode + features.measure(light) + librosa onsets / beats + loudness curves, timed cold in this round (this module's cache deleted; the core decode cache was cold too after its v2 bump). Picture pass = earshot.video.frame_diff (core cache, shared with timeline / sheet): 0.0 means another module had already decoded that film; ref_6 (4K) and clawd_synth were cold. AudioSet = 5 s windows through AST, measured cold in the first build (same code and cache key, so not re-run): ~4-9 s per 5 s window on this laptop under load from 3-4 other model jobs; `earshot critique --fast` skips it and realness. Warm = everything cached. EARSHOT_THREADS=2.++## Leave-one-out: each reference vs a profile of the other six++| film | verdict | weighted score | scored metrics in range | reference-range metrics in range | out of range |+|---|---|---|---|---|---|+| ref_1 | close | 75% | 17/22 | 16/20 | true_peak high, lra low, first_sound_s high, end_silence_s high, side_to_mid_db low |+| ref_2 | launch-grade | 94% | 21/22 | 20/20 | true_peak high |+| ref_3 | close | 83% | 17/22 | 16/20 | true_peak high, end_lu low, band_sub low, band_lowmid high, band_air low |+| ref_4 | close | 89% | 19/22 | 17/20 | onset_rate high, intro_lu low, band_mid low |+| ref_5 | needs work | 56% | 14/22 | 14/20 | lufs_i low, true_peak high, music_frac low, cuts_with_sound low, lra high, max_gap_s high, band_air high, side_to_mid_db high |+| ref_6 | close | 86% | 19/22 | 19/20 | lufs_i low, true_peak high, intro_lu high |+| ref_7 | close | 77% | 18/22 | 18/20 | lufs_i high, true_peak high, onset_rate low, hit_rate low |++Total: 125/154 scored metric checks in range (81 %); reference-range metrics only: 120/140 (86 %). For exchangeable data a held-out film falls outside the min-max of six others with probability 2/7 (29 %) per continuous metric before tolerance, so ~71 % is the no-overfit baseline; the tolerance adds a little. p_real is not part of this check (the realness model was trained on these films).++In-sample (each reference vs the full 7-film profile): every reference-range metric is in range by construction; the only failures are web-practice rules: ref_1: true_peak high; ref_2: true_peak high; ref_3: true_peak high; ref_5: lufs_i low, true_peak high; ref_6: lufs_i low, true_peak high; ref_7: lufs_i high, true_peak high.++### Per metric (7 leave-one-out checks each)++| metric | weight | ok | low | high | n.a. |+|---|---|---|---|---|---|+| lufs_i | critical | 4 | 2 | 1 | 0 |+| true_peak | major | 1 | 0 | 6 | 0 |+| music_frac | critical | 6 | 1 | 0 | 0 |+| cuts_with_sound | major | 6 | 1 | 0 | 0 |+| lra | major | 5 | 1 | 1 | 0 |+| crest_lu | major | 7 | 0 | 0 | 0 |+| first_sound_s | major | 6 | 0 | 1 | 0 |+| end_silence_s | major | 6 | 0 | 1 | 0 |+| max_gap_s | major | 6 | 0 | 1 | 0 |+| final_lu | major | 7 | 0 | 0 | 0 |+| onset_rate | major | 5 | 1 | 1 | 0 |+| hit_rate | major | 6 | 1 | 0 | 0 |+| intro_lu | minor | 5 | 1 | 1 | 0 |+| end_lu | minor | 6 | 1 | 0 | 0 |+| fade_s | minor | 7 | 0 | 0 | 0 |+| band_sub | minor | 6 | 1 | 0 | 0 |+| band_low | minor | 7 | 0 | 0 | 0 |+| band_lowmid | minor | 6 | 0 | 1 | 0 |+| band_mid | minor | 6 | 1 | 0 | 0 |+| band_presence | minor | 7 | 0 | 0 | 0 |+| band_air | minor | 5 | 1 | 1 | 0 |+| side_to_mid_db | minor | 5 | 1 | 1 | 0 |++## Sensitivity: known-bad mixes++Each row is one defect made on purpose (ffmpeg, scratch files in out/launch/neg/), critiqued against the 7-film profile. 'Should flag' was written down before the first run; 'caught' = at least one of those metrics came out low / high. Rows without models skip AudioSet and realness (partial verdicts); the control is the untouched real soundtrack.++| mix | what | should flag | caught | verdict | weighted score | all flagged rows | p_real | s (critique, cold) |+|---|---|---|---|---|---|---|---|---|+| base | control: the real Clawd soundtrack untouched | - | - | close (partial) | 87% | crest_lu high (major); first_sound_s high (major) | - | 27.5 |+| quiet | unmastered: 12 dB quieter | lufs_i | lufs_i | needs work (partial) | 77% | lufs_i low (critical); crest_lu high (major); first_sound_s high (major) | - | 9.0 |+| hot | 12 dB hotter, clipped by the 16-bit file | lufs_i, true_peak | lufs_i, true_peak | needs work (partial) | 77% | lufs_i high (critical); true_peak high (major); first_sound_s high (major) | - | 8.8 |+| mono | folded to mono | side_to_mid_db | side_to_mid_db | close (partial) | 77% | lufs_i low (major); crest_lu high (major); first_sound_s high (major); side_to_mid_db low (minor) | - | 9.5 |+| muffled | low-passed at 3 kHz (12 dB/oct) | band_presence, band_air | band_presence, band_air | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); band_presence low (minor); band_air low (minor) | - | 10.0 |+| late | 2.5 s of silence before the start | first_sound_s | first_sound_s | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); onset_rate low (major) | - | 13.8 |+| deadair | 4 s of silence after the end | end_silence_s | end_silence_s | close (partial) | 81% | crest_lu high (major); first_sound_s high (major); end_silence_s high (major) | - | 14.4 |+| fadeout | 12 s fade-out | fade_s, final_lu, end_lu | fade_s, final_lu | close (partial) | 77% | crest_lu high (major); first_sound_s high (major); final_lu low (major); fade_s high (minor) | - | 12.6 |+| hole | 2.5 s muted mid-film | max_gap_s | max_gap_s | needs work (partial) | 74% | crest_lu high (major); first_sound_s high (major); max_gap_s high (major); onset_rate low (major) | - | 13.2 |+| squashed | over-compressed: 20:1 from -36 dB + limiter | lra, crest_lu | no | launch-grade (partial) | 90% | true_peak high (major); side_to_mid_db high (minor) | - | 14.6 |+| sfx_only | the SFX stem alone, with the picture | music_frac | music_frac | needs work | 77% | lufs_i low (critical); music_frac low (critical); band_sub low (minor); band_low low (minor); band_mid high (minor) | 0.787 | 194.8 |+| amb_only | the ambience stem alone | music_frac, onset_rate, hit_rate | music_frac | needs work | 68% | lufs_i low (critical); music_frac low (critical); crest_lu high (major); band_sub low (minor); band_low low (minor); band_mid high (minor); band_presence high (minor) | 0.882 | 90.6 |+| synth_bed | the retired code-synthesized music bed | p_real | p_real | not launch-grade | 49% | lufs_i low (critical); p_real low (critical); music_frac low (critical); crest_lu high (major); first_sound_s high (major); onset_rate low (major); band_sub low (minor); band_low low (minor); band_mid high (minor); side_to_mid_db high (minor) | 0.095 | 43.7 |+| sine | 30 s 440 Hz sine | p_real, music_frac, onset_rate | p_real, music_frac, onset_rate | not launch-grade | 35% | lufs_i low (critical); p_real low (critical); music_frac low (critical); lra low (major); crest_lu low (major); onset_rate low (major); hit_rate low (major); band_sub low (minor); band_low low (minor); band_lowmid high (minor); band_mid low (minor); band_presence low (minor); band_air low (minor); side_to_mid_db low (minor) | 0.03 | 47.5 |+| pink | 30 s pink noise | p_real, music_frac, onset_rate | p_real, music_frac, onset_rate | not launch-grade | 46% | lufs_i low (critical); p_real low (critical); music_frac low (critical); lra low (major); crest_lu low (major); onset_rate low (major); hit_rate low (major); band_mid high (minor); band_presence high (minor); side_to_mid_db low (minor) | 0.158 | 22.7 |++Caught: 13 of 14 known-bad mixes have the expected metric flagged (missed: squashed). Verdicts: launch-grade 1, close 5, needs work 5, not launch-grade 3 (control: close). A single structural defect on a mix that is already close usually stays close: the verdict is coarse, the flagged rows and FIX FIRST actions are the signal. Over-compression is not a defect by these references: two of them (LRA 1.7 / 3.3 LU, loudest 3 s only 1.1-1.2 LU above integrated) are as dense as the squashed mix.++## The Clawd film: real soundtrack vs the retired synthesized soundtrack++Reading: the scorecard rates the real soundtrack close (90%) and the synthesized one needs work (80%). The structural metrics separate them only a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has one loudness peak that sticks out. The difference the user hears most - synthesized timbre - is not structural; it is what p_real measures: real 0.94 vs synth 0.33 (earshot.realness, verdict thresholds lo 0.4 / hi 0.77 at this run); below 0.5 the p_real row is critical, which is what moves the synth version's verdict. Caveat: the realness model (another module) was trained partly on these 7 launch films, on the Clawd project's real asset library and on the retired synthesized stems (eval/realness/), so p_real on the Clawd films is not an independent test.++### clawd_real: Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)++Verdict **close**, weighted score 90%, 21/23 in range, 0 critical / 2 major issues; critique call 4.6 s with the measurement passes cached (realness runs inside it).++- [major] nothing reaches body level until 0:01.01 (before that the sound sits around -42 LUFS, 27 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00+- [major] the loudest 3 s (around 0:41.50, -10.4 LUFS) stand 4.6 LU above the integrated level (refs 1.1 LU .. 3.5 LU): the rest of the film plays quiet next to it; lift the body (quietest stretch 0:06.20-0:09.20 at -24.5 LUFS short-term) or tame that peak+- tip: 5 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.30, 0:00.80, 0:01.30, 0:30.00, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down+- tip: no stop-downs: 4 of 7 references drop the sound out for 0.2-1 s right before a big hit; try one before the biggest visual change at 0:20.00+- realness: Clawd_1080p60.mp4: real or pro-produced (p_real 0.94, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 0 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score +2.9. No window leans synthesized (lowest p_real 0.76); that does not rule out a code-made layer under a louder real mix (most are missed): judge the source files and stems.++### clawd_synth: the same film with the retired code-synthesized soundtrack++Verdict **needs work**, weighted score 80%, 19/23 in range, 1 critical / 2 major issues; critique call 4.6 s with the measurement passes cached (realness runs inside it).++- [critical] realness: likely code-synthesized (p_real 0.33, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:56.00-1:00.00 (0.05): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with `earshot realness stem.wav` (a finished mix is harder to judge than its sources)+- [major] the final moment sits 7.5 LU below the body: end on a logo sting / hit at the last visual event (0:59.00) instead of fading away (refs -2.2 LU .. 5.6 LU)+- [major] sparse: 2.1 onsets/s (refs 2.46 /s .. 8.22 /s): use music with more rhythmic drive, or add UI foley (clicks, typing, ticks) on the on-screen actions+- [minor] 4.4 s fade-out (refs 0.08 s .. 2.54 s): end on a sting with a short ring-out, or cut hard on the last picture cut+- tip: 8 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.80, 0:01.30, 0:28.33, 0:28.98, 0:30.00, 0:33.00, 0:40.88, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down+- realness: Clawd_1080p60_synth-audio.mp4: likely code-synthesized (p_real 0.33, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 14 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score -0.5. Leaning synthesized: 0:00.00-0:48.00, 0:52.00-1:00.00; 1 at or below the 4 s-window threshold 0.17 (code-synthesized): 0:56.00-1:00.00.++## Hit / onset timing vs ground truth (tuning data: the Clawd real soundtrack)++The mix report lists 264 sample placements (every key tick, pop, whoosh, hit, music motif note ...). earshot found 16 salient hits and 149 onsets in the mixed film audio; 11 of the 16 hits sit within 2 ms of a placement (median distance 1.0 ms).++| tolerance | hits on a placement (chance) | onsets on a placement | placements with an onset |+|---|---|---|---|+| +-20 ms | 75% (10%) | 61% | 62% |+| +-50 ms | 88% (25%) | 75% | 78% |+| +-100 ms | 88% (39%) | 79% | 83% |++Chance = the same hits shifted off the placements by fixed offsets. Hit and onset times are moved to the steepest rise of the 2 ms level (attack), which is how a mixer aligns a sample.++## Visual-event detector (earshot.video frame difference) vs ffmpeg scene cuts++Scene cuts = `video.cuts(path, method='scene')` (ffmpeg scene score 0.3). 'Same as first build' = the core video.frame_diff signal is bit-identical to this module's first-build picture pass, so moving the detector into core changed no number.++| film | scene cuts | visual events | share of scene cuts also found (+-1 frame) | same as first build |+|---|---|---|---|---|+| ref_1 | 0 | 28 | n.a. (< 5 scene cuts) | True |+| ref_2 | 2 | 9 | n.a. (< 5 scene cuts) | True |+| ref_3 | 15 | 17 | 1.0 | True |+| ref_4 | 17 | 42 | 0.82 | True |+| ref_5 | 2 | 16 | n.a. (< 5 scene cuts) | True |+| ref_6 | 11 | 31 | 1.0 | True |+| ref_7 | 40 | 67 | 0.8 | True |+| clawd_real | 10 | 26 | 1.0 | True |+| clawd_synth | 10 | 26 | 1.0 | True |diff --git a/eval/launch/films.json b/eval/launch/films.jsonnew file mode 100644index 0000000..de0f346--- /dev/null+++ b/eval/launch/films.json@@ -0,0 +1,650 @@+{+ "ref_1": {+ "id": "ref_1",+ "style": "AI video-search SaaS launch: white UI walkthrough with chat prompts and cursor, kinetic type, thumbnail-wall reveals, logo end card; voice-over",+ "duration": 93.12,+ "fps": 23.976,+ "lossy": true,+ "metrics": {+ "lufs_i": -12.1,+ "true_peak": 0.1,+ "lra": 1.7,+ "crest_lu": 1.1,+ "intro_lu": -1.1,+ "end_lu": -2.5,+ "peak_pos": 0.62,+ "first_sound_s": 0.48,+ "max_gap_s": 0.04,+ "end_silence_s": 1.9,+ "fade_s": 2.12,+ "final_lu": 1.1,+ "tail_s": 3.58,+ "music_frac": 1.0,+ "speech_frac": 0.95,+ "tempo": 136,+ "onset_rate": 5.87,+ "hit_rate": 1.56,+ "stopdowns_min": 1.3,+ "cut_rate": 0.3,+ "cuts_with_sound": 0.89,+ "cut_hit_frac": 0.46,+ "cut_frame_sync": 0.54,+ "cuts_on_beat": 0.21,+ "band_sub": -9.8,+ "band_low": -3.5,+ "band_lowmid": -4.9,+ "band_mid": -11.0,+ "band_presence": -14.6,+ "band_air": -23.7,+ "brightness_hz": 1966,+ "side_to_mid_db": -14.5,+ "correlation": 0.93+ },+ "chance": {+ "cuts_with_sound": 0.9,+ "cut_hit_frac": 0.3,+ "cut_frame_sync": 0.57,+ "cuts_on_beat": 0.32+ },+ "arc": [+ -1.3,+ -0.5,+ 0.3,+ -0.0,+ 0.4,+ -0.1,+ 0.4,+ -0.4,+ 0.1,+ -0.1,+ -11.4+ ],+ "video": true,+ "n_visual": 28,+ "n_hits": 145,+ "seconds_cold": 50.0,+ "seconds_warm": 0.4,+ "timing_cold": {+ "audio": 49.8,+ "audioset": 0.0,+ "picture": 0.0+ },+ "audioset_cold_s": 144.4+ },+ "ref_2": {+ "id": "ref_2",+ "style": "AI workspace explainer: white minimal kinetic type, app-icon clusters, chat UI, photographic logo reveal, gradient end card; voice-over",+ "duration": 28.909,+ "fps": 30.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -14.8,+ "true_peak": 1.0,+ "lra": 3.6,+ "crest_lu": 2.4,+ "intro_lu": -1.8,+ "end_lu": -3.0,+ "peak_pos": 0.75,+ "first_sound_s": 0.13,+ "max_gap_s": 0.0,+ "end_silence_s": 0.04,+ "fade_s": 0.27,+ "final_lu": 0.2,+ "tail_s": 2.44,+ "music_frac": 1.0,+ "speech_frac": 0.83,+ "tempo": 120,+ "onset_rate": 5.98,+ "hit_rate": 1.21,+ "stopdowns_min": 0.0,+ "cut_rate": 0.31,+ "cuts_with_sound": 0.67,+ "cut_hit_frac": 0.11,+ "cut_frame_sync": 0.44,+ "cuts_on_beat": 0.22,+ "band_sub": -6.8,+ "band_low": -5.4,+ "band_lowmid": -4.2,+ "band_mid": -10.5,+ "band_presence": -17.8,+ "band_air": -21.8,+ "brightness_hz": 1193,+ "side_to_mid_db": -8.5,+ "correlation": 0.75+ },+ "chance": {+ "cuts_with_sound": 0.89,+ "cut_hit_frac": 0.26,+ "cut_frame_sync": 0.47,+ "cuts_on_beat": 0.28+ },+ "arc": [+ -1.0,+ -2.2,+ 0.3,+ 0.7,+ -0.2,+ 0.4,+ -0.0,+ 1.2,+ 1.0,+ -1.6,+ -4.1+ ],+ "video": true,+ "n_visual": 9,+ "n_hits": 35,+ "seconds_cold": 10.2,+ "seconds_warm": 0.7,+ "timing_cold": {+ "audio": 9.7,+ "audioset": 0.0,+ "picture": 0.0+ },+ "audioset_cold_s": 37.0+ },+ "ref_3": {+ "id": "ref_3",+ "style": "editorial kinetic collage: grainy paper texture, typewriter text, photo cut-outs, pixel icons, hand-drawn strokes, fast cuts; voice-over",+ "duration": 20.459,+ "fps": 23.976,+ "lossy": true,+ "metrics": {+ "lufs_i": -10.4,+ "true_peak": 0.6,+ "lra": 6.4,+ "crest_lu": 2.3,+ "intro_lu": -0.5,+ "end_lu": -14.3,+ "peak_pos": 0.79,+ "first_sound_s": 0.11,+ "max_gap_s": 0.0,+ "end_silence_s": 0.02,+ "fade_s": 2.54,+ "final_lu": 5.6,+ "tail_s": 3.54,+ "music_frac": 1.0,+ "speech_frac": 1.0,+ "tempo": 108,+ "onset_rate": 3.81,+ "hit_rate": 1.66,+ "stopdowns_min": 2.9,+ "cut_rate": 0.83,+ "cuts_with_sound": 0.88,+ "cut_hit_frac": 0.53,+ "cut_frame_sync": 0.47,+ "cuts_on_beat": 0.18,+ "band_sub": -16.2,+ "band_low": -6.7,+ "band_lowmid": -1.7,+ "band_mid": -10.8,+ "band_presence": -30.2,+ "band_air": -34.2,+ "brightness_hz": 1169,+ "side_to_mid_db": -7.1,+ "correlation": 0.68+ },+ "chance": {+ "cuts_with_sound": 0.68,+ "cut_hit_frac": 0.35,+ "cut_frame_sync": 0.36,+ "cuts_on_beat": 0.23+ },+ "arc": [+ -0.5,+ -3.0,+ -7.4,+ -0.3,+ -1.7,+ -1.8,+ 0.2,+ -1.0,+ 2.4,+ -1.3,+ -13.5+ ],+ "video": true,+ "n_visual": 17,+ "n_hits": 34,+ "seconds_cold": 6.2,+ "seconds_warm": 0.7,+ "timing_cold": {+ "audio": 5.6,+ "audioset": 0.0,+ "picture": 0.1+ },+ "audioset_cold_s": 23.0+ },+ "ref_4": {+ "id": "ref_4",+ "style": "fintech brand film: bold black and green kinetic typography, grids and numerals, flicker cuts, logo stings; music only",+ "duration": 32.256,+ "fps": 25.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -14.8,+ "true_peak": -0.8,+ "lra": 7.0,+ "crest_lu": 2.0,+ "intro_lu": -17.5,+ "end_lu": -5.5,+ "peak_pos": 0.78,+ "first_sound_s": 0.35,+ "max_gap_s": 0.04,+ "end_silence_s": 0.02,+ "fade_s": 1.64,+ "final_lu": -0.9,+ "tail_s": 0.13,+ "music_frac": 1.0,+ "speech_frac": 0.0,+ "tempo": 123,+ "onset_rate": 8.22,+ "hit_rate": 1.18,+ "stopdowns_min": 0.0,+ "cut_rate": 1.3,+ "cuts_with_sound": 1.0,+ "cut_hit_frac": 0.24,+ "cut_frame_sync": 0.67,+ "cuts_on_beat": 0.4,+ "band_sub": -6.6,+ "band_low": -2.1,+ "band_lowmid": -8.4,+ "band_mid": -19.1,+ "band_presence": -30.3,+ "band_air": -30.3,+ "brightness_hz": 1220,+ "side_to_mid_db": -6.3,+ "correlation": 0.62+ },+ "chance": {+ "cuts_with_sound": 0.99,+ "cut_hit_frac": 0.27,+ "cut_frame_sync": 0.7,+ "cuts_on_beat": 0.28+ },+ "arc": [+ -19.8,+ -8.2,+ -1.8,+ -1.1,+ -1.4,+ 0.2,+ 1.4,+ 1.6,+ 1.8,+ -0.3,+ -10.2+ ],+ "video": true,+ "n_visual": 42,+ "n_hits": 38,+ "seconds_cold": 18.3,+ "seconds_warm": 0.3,+ "timing_cold": {+ "audio": 18.1,+ "audioset": 0.0,+ "picture": 0.0+ },+ "audioset_cold_s": 58.6+ },+ "ref_5": {+ "id": "ref_5",+ "style": "AI image-generation launch: soft gradients, typed prompt, collage of generated photos; sound design with hard silences",+ "duration": 29.525,+ "fps": 25.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -22.2,+ "true_peak": 1.2,+ "lra": 12.2,+ "crest_lu": 3.2,+ "intro_lu": -1.0,+ "end_lu": -10.4,+ "peak_pos": 0.14,+ "first_sound_s": 0.34,+ "max_gap_s": 1.66,+ "end_silence_s": 0.67,+ "fade_s": 0.25,+ "final_lu": -2.2,+ "tail_s": 3.58,+ "music_frac": 0.67,+ "speech_frac": 0.0,+ "tempo": 136,+ "onset_rate": 4.03,+ "hit_rate": 1.25,+ "stopdowns_min": 18.3,+ "cut_rate": 0.54,+ "cuts_with_sound": 0.5,+ "cut_hit_frac": 0.31,+ "cut_frame_sync": 0.38,+ "cuts_on_beat": 0.38,+ "band_sub": -4.1,+ "band_low": -4.2,+ "band_lowmid": -16.8,+ "band_mid": -13.6,+ "band_presence": -13.1,+ "band_air": -11.2,+ "brightness_hz": 3633,+ "side_to_mid_db": -5.0,+ "correlation": 0.52+ },+ "chance": {+ "cuts_with_sound": 0.48,+ "cut_hit_frac": 0.24,+ "cut_frame_sync": 0.24,+ "cuts_on_beat": 0.25+ },+ "arc": [+ -7.5,+ -0.8,+ 1.8,+ -5.5,+ -5.0,+ 0.9,+ -0.9,+ -4.4,+ -5.5,+ -12.0,+ -7.4+ ],+ "video": true,+ "n_visual": 16,+ "n_hits": 37,+ "seconds_cold": 6.1,+ "seconds_warm": 0.2,+ "timing_cold": {+ "audio": 4.0,+ "audioset": 0.0,+ "picture": 1.9+ },+ "audioset_cold_s": 41.6+ },+ "ref_6": {+ "id": "ref_6",+ "style": "creator-commerce SaaS: blue UI walkthrough, notifications, cursor clicks, data-flow diagrams, tagline end card; voice-over",+ "duration": 49.28,+ "fps": 30.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -17.4,+ "true_peak": 0.1,+ "lra": 5.5,+ "crest_lu": 3.5,+ "intro_lu": 2.2,+ "end_lu": -0.3,+ "peak_pos": 0.03,+ "first_sound_s": 0.04,+ "max_gap_s": 0.02,+ "end_silence_s": 0.28,+ "fade_s": 0.1,+ "final_lu": 4.9,+ "tail_s": 2.0,+ "music_frac": 1.0,+ "speech_frac": 1.0,+ "tempo": 133,+ "onset_rate": 5.66,+ "hit_rate": 1.3,+ "stopdowns_min": 1.2,+ "cut_rate": 0.63,+ "cuts_with_sound": 0.81,+ "cut_hit_frac": 0.29,+ "cut_frame_sync": 0.39,+ "cuts_on_beat": 0.52,+ "band_sub": -6.4,+ "band_low": -4.4,+ "band_lowmid": -5.3,+ "band_mid": -13.8,+ "band_presence": -13.8,+ "band_air": -17.1,+ "brightness_hz": 1976,+ "side_to_mid_db": -10.0,+ "correlation": 0.82+ },+ "chance": {+ "cuts_with_sound": 0.86,+ "cut_hit_frac": 0.28,+ "cut_frame_sync": 0.44,+ "cuts_on_beat": 0.31+ },+ "arc": [+ 3.9,+ -0.8,+ -3.1,+ -0.8,+ -0.1,+ 1.8,+ -0.6,+ -1.0,+ -1.8,+ 0.8,+ -2.4+ ],+ "video": true,+ "n_visual": 31,+ "n_hits": 64,+ "seconds_cold": 20.2,+ "seconds_warm": 0.2,+ "timing_cold": {+ "audio": 5.8,+ "audioset": 0.0,+ "picture": 14.2+ },+ "audioset_cold_s": 54.5+ },+ "ref_7": {+ "id": "ref_7",+ "style": "gaming motion-design reel: logo stings, 3D explosions, saturated colour, very fast cuts, brickwall-loud music",+ "duration": 41.792,+ "fps": 30.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -7.3,+ "true_peak": 2.0,+ "lra": 3.3,+ "crest_lu": 1.2,+ "intro_lu": -1.3,+ "end_lu": -0.9,+ "peak_pos": 0.88,+ "first_sound_s": 0.06,+ "max_gap_s": 0.0,+ "end_silence_s": 0.02,+ "fade_s": 0.08,+ "final_lu": -0.0,+ "tail_s": null,+ "music_frac": 0.89,+ "speech_frac": 0.0,+ "tempo": 123,+ "onset_rate": 2.46,+ "hit_rate": 0.1,+ "stopdowns_min": 0.0,+ "cut_rate": 1.6,+ "cuts_with_sound": 0.6,+ "cut_hit_frac": 0.03,+ "cut_frame_sync": 0.28,+ "cuts_on_beat": 0.22,+ "band_sub": -4.9,+ "band_low": -2.5,+ "band_lowmid": -16.1,+ "band_mid": -12.4,+ "band_presence": -17.8,+ "band_air": -23.6,+ "brightness_hz": 2628,+ "side_to_mid_db": -9.1,+ "correlation": 0.78+ },+ "chance": {+ "cuts_with_sound": 0.44,+ "cut_hit_frac": 0.02,+ "cut_frame_sync": 0.19,+ "cuts_on_beat": 0.27+ },+ "arc": [+ -4.4,+ 0.5,+ 0.4,+ 0.7,+ 0.5,+ 0.2,+ -5.4,+ 0.6,+ 0.9,+ 0.7,+ -2.6+ ],+ "video": true,+ "n_visual": 67,+ "n_hits": 4,+ "seconds_cold": 11.1,+ "seconds_warm": 0.3,+ "timing_cold": {+ "audio": 5.8,+ "audioset": 0.0,+ "picture": 5.1+ },+ "audioset_cold_s": 98.4+ },+ "clawd_real": {+ "id": "clawd_real",+ "style": "Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)",+ "duration": 60.011,+ "fps": 60.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -15.0,+ "true_peak": -1.4,+ "lra": 10.4,+ "crest_lu": 4.6,+ "intro_lu": -7.3,+ "end_lu": -1.7,+ "peak_pos": 0.69,+ "first_sound_s": 1.01,+ "max_gap_s": 0.0,+ "end_silence_s": 0.09,+ "fade_s": 0.23,+ "final_lu": 0.3,+ "tail_s": 2.4,+ "music_frac": 1.0,+ "speech_frac": 0.0,+ "tempo": 120,+ "onset_rate": 2.48,+ "hit_rate": 0.27,+ "stopdowns_min": 0.0,+ "cut_rate": 0.43,+ "cuts_with_sound": 0.62,+ "cut_hit_frac": 0.19,+ "cut_frame_sync": 0.38,+ "cuts_on_beat": 0.69,+ "band_sub": -13.8,+ "band_low": -3.8,+ "band_lowmid": -3.9,+ "band_mid": -9.5,+ "band_presence": -18.7,+ "band_air": -19.6,+ "brightness_hz": 2088,+ "side_to_mid_db": -4.3,+ "correlation": 0.46+ },+ "chance": {+ "cuts_with_sound": 0.37,+ "cut_hit_frac": 0.05,+ "cut_frame_sync": 0.1,+ "cuts_on_beat": 0.25+ },+ "arc": [+ -9.8,+ -7.1,+ -3.8,+ -0.0,+ -0.8,+ -3.0,+ -4.9,+ 3.9,+ 3.2,+ 1.3,+ -2.5+ ],+ "video": true,+ "n_visual": 26,+ "n_hits": 16,+ "seconds_cold": 8.2,+ "seconds_warm": 0.2,+ "timing_cold": {+ "audio": 8.0,+ "audioset": 0.0,+ "picture": 0.0+ },+ "audioset_cold_s": 78.9+ },+ "clawd_synth": {+ "id": "clawd_synth",+ "style": "the same film with the retired code-synthesized soundtrack",+ "duration": 60.011,+ "fps": 60.0,+ "lossy": true,+ "metrics": {+ "lufs_i": -14.0,+ "true_peak": -2.1,+ "lra": 7.2,+ "crest_lu": 3.3,+ "intro_lu": -4.7,+ "end_lu": -7.4,+ "peak_pos": 0.77,+ "first_sound_s": 0.0,+ "max_gap_s": 0.04,+ "end_silence_s": 0.51,+ "fade_s": 4.4,+ "final_lu": -7.5,+ "tail_s": 1.47,+ "music_frac": 1.0,+ "speech_frac": 0.0,+ "tempo": 120,+ "onset_rate": 2.1,+ "hit_rate": 0.38,+ "stopdowns_min": 1.0,+ "cut_rate": 0.43,+ "cuts_with_sound": 0.54,+ "cut_hit_frac": 0.23,+ "cut_frame_sync": 0.31,+ "cuts_on_beat": 0.62,+ "band_sub": -3.1,+ "band_low": -5.0,+ "band_lowmid": -8.4,+ "band_mid": -13.7,+ "band_presence": -23.4,+ "band_air": -29.3,+ "brightness_hz": 1756,+ "side_to_mid_db": -12.2,+ "correlation": 0.89+ },+ "chance": {+ "cuts_with_sound": 0.43,+ "cut_hit_frac": 0.05,+ "cut_frame_sync": 0.1,+ "cuts_on_beat": 0.27+ },+ "arc": [+ -5.5,+ -3.9,+ 0.2,+ 0.5,+ 1.0,+ -0.9,+ -1.5,+ 2.4,+ 2.5,+ -2.8,+ -10.0+ ],+ "video": true,+ "n_visual": 26,+ "n_hits": 23,+ "seconds_cold": 33.8,+ "seconds_warm": 0.6,+ "timing_cold": {+ "audio": 8.1,+ "audioset": 0.0,+ "picture": 25.4+ },+ "audioset_cold_s": 76.3+ }+}\ No newline at end of filediff --git a/eval/launch/negatives.json b/eval/launch/negatives.jsonnew file mode 100644index 0000000..65a820d--- /dev/null+++ b/eval/launch/negatives.json@@ -0,0 +1,437 @@+{+ "base": {+ "what": "control: the real Clawd soundtrack untouched",+ "models": false,+ "picture": false,+ "expect": [],+ "caught": [],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.871,+ "critical": 0,+ "major": 2,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)"+ ],+ "first_action": "nothing reaches body level until 0:01.01 (before that the sound sits around -42 LUFS, 27 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00",+ "p_real": null,+ "seconds": 27.5+ },+ "quiet": {+ "what": "unmastered: 12 dB quieter",+ "models": false,+ "picture": false,+ "expect": [+ "lufs_i"+ ],+ "caught": [+ "lufs_i"+ ],+ "verdict": "needs work",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.774,+ "critical": 1,+ "major": 2,+ "flagged": [+ "lufs_i low (critical)",+ "crest_lu high (major)",+ "first_sound_s high (major)"+ ],+ "first_action": "raise the master by +13.0 dB to about -14 LUFS integrated (web) with a true-peak limiter (ceiling -1 dBTP); if the limiter works harder than ~3 dB, compress the music bus first (refs -22.2 LUFS .. -7.3 LUFS)",+ "p_real": null,+ "seconds": 9.0+ },+ "hot": {+ "what": "12 dB hotter, clipped by the 16-bit file",+ "models": false,+ "picture": false,+ "expect": [+ "lufs_i",+ "true_peak"+ ],+ "caught": [+ "lufs_i",+ "true_peak"+ ],+ "verdict": "needs work",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.774,+ "critical": 1,+ "major": 2,+ "flagged": [+ "lufs_i high (critical)",+ "true_peak high (major)",+ "first_sound_s high (major)"+ ],+ "first_action": "turn the master down by 7.8 dB to about -12 LUFS: above -9 LUFS normalising platforms turn it down anyway and the limiter flattens every hit (refs -22.2 LUFS .. -7.3 LUFS)",+ "p_real": null,+ "seconds": 8.8+ },+ "mono": {+ "what": "folded to mono",+ "models": false,+ "picture": false,+ "expect": [+ "side_to_mid_db"+ ],+ "caught": [+ "side_to_mid_db"+ ],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.767,+ "critical": 0,+ "major": 3,+ "flagged": [+ "lufs_i low (major)",+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "side_to_mid_db low (minor)"+ ],+ "first_action": "nothing reaches body level until 0:01.01 (before that the sound sits around -43 LUFS, 26 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00",+ "p_real": null,+ "seconds": 9.5+ },+ "muffled": {+ "what": "low-passed at 3 kHz (12 dB/oct)",+ "models": false,+ "picture": false,+ "expect": [+ "band_presence",+ "band_air"+ ],+ "caught": [+ "band_presence",+ "band_air"+ ],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.806,+ "critical": 0,+ "major": 2,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "band_presence low (minor)",+ "band_air low (minor)"+ ],+ "first_action": "nothing reaches body level until 0:00.94 (before that the sound sits around -44 LUFS, 29 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00",+ "p_real": null,+ "seconds": 10.0+ },+ "late": {+ "what": "2.5 s of silence before the start",+ "models": false,+ "picture": false,+ "expect": [+ "first_sound_s"+ ],+ "caught": [+ "first_sound_s"+ ],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.806,+ "critical": 0,+ "major": 3,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "onset_rate low (major)"+ ],+ "first_action": "nothing reaches body level until 0:03.51 (before that the sound sits around -46 LUFS, 32 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00",+ "p_real": null,+ "seconds": 13.8+ },+ "deadair": {+ "what": "4 s of silence after the end",+ "models": false,+ "picture": false,+ "expect": [+ "end_silence_s"+ ],+ "caught": [+ "end_silence_s"+ ],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.806,+ "critical": 0,+ "major": 3,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "end_silence_s high (major)"+ ],+ "first_action": "4.1 s of dead air after the sound ends at 0:59.93: trim the picture or let the final sting ring out over the end card (refs 0.02 s .. 1.90 s)",+ "p_real": null,+ "seconds": 14.4+ },+ "fadeout": {+ "what": "12 s fade-out",+ "models": false,+ "picture": false,+ "expect": [+ "fade_s",+ "final_lu",+ "end_lu"+ ],+ "caught": [+ "fade_s",+ "final_lu"+ ],+ "verdict": "close",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.774,+ "critical": 0,+ "major": 3,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "final_lu low (major)",+ "fade_s high (minor)"+ ],+ "first_action": "the final moment sits 11.9 LU below the body: end on a logo sting / hit instead of fading away (refs -2.2 LU .. 5.6 LU)",+ "p_real": null,+ "seconds": 12.6+ },+ "hole": {+ "what": "2.5 s muted mid-film",+ "models": false,+ "picture": false,+ "expect": [+ "max_gap_s"+ ],+ "caught": [+ "max_gap_s"+ ],+ "verdict": "needs work",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.742,+ "critical": 0,+ "major": 4,+ "flagged": [+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "max_gap_s high (major)",+ "onset_rate low (major)"+ ],+ "first_action": "nothing reaches body level until 0:01.01 (before that the sound sits around -42 LUFS, 27 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00",+ "p_real": null,+ "seconds": 13.2+ },+ "squashed": {+ "what": "over-compressed: 20:1 from -36 dB + limiter",+ "models": false,+ "picture": false,+ "expect": [+ "lra",+ "crest_lu"+ ],+ "caught": [],+ "verdict": "launch-grade",+ "partial": [+ "realness",+ "music presence"+ ],+ "score": 0.903,+ "critical": 0,+ "major": 1,+ "flagged": [+ "true_peak high (major)",+ "side_to_mid_db high (minor)"+ ],+ "first_action": "true peak +0.3 dBTP: end the master chain with a true-peak limiter, ceiling -1.0 dBTP, then re-encode",+ "p_real": null,+ "seconds": 14.6+ },+ "sfx_only": {+ "what": "the SFX stem alone, with the picture",+ "models": true,+ "picture": true,+ "expect": [+ "music_frac"+ ],+ "caught": [+ "music_frac"+ ],+ "verdict": "needs work",+ "partial": [],+ "score": 0.769,+ "critical": 2,+ "major": 0,+ "flagged": [+ "lufs_i low (critical)",+ "music_frac low (critical)",+ "band_sub low (minor)",+ "band_low low (minor)",+ "band_mid high (minor)"+ ],+ "first_action": "raise the master by +13.6 dB to about -14 LUFS integrated (web) with a true-peak limiter (ceiling -1 dBTP); if the limiter works harder than ~3 dB, compress the music bus first (refs -22.2 LUFS .. -7.3 LUFS)",+ "p_real": 0.787,+ "seconds": 194.8+ },+ "amb_only": {+ "what": "the ambience stem alone",+ "models": true,+ "picture": false,+ "expect": [+ "music_frac",+ "onset_rate",+ "hit_rate"+ ],+ "caught": [+ "music_frac"+ ],+ "verdict": "needs work",+ "partial": [],+ "score": 0.676,+ "critical": 2,+ "major": 1,+ "flagged": [+ "lufs_i low (critical)",+ "music_frac low (critical)",+ "crest_lu high (major)",+ "band_sub low (minor)",+ "band_low low (minor)",+ "band_mid high (minor)",+ "band_presence high (minor)"+ ],+ "first_action": "raise the master by +25.5 dB to about -14 LUFS integrated (web) with a true-peak limiter (ceiling -1 dBTP); if the limiter works harder than ~3 dB, compress the music bus first (refs -22.2 LUFS .. -7.3 LUFS)",+ "p_real": 0.882,+ "seconds": 90.6+ },+ "synth_bed": {+ "what": "the retired code-synthesized music bed",+ "models": true,+ "picture": false,+ "expect": [+ "p_real"+ ],+ "caught": [+ "p_real"+ ],+ "verdict": "not launch-grade",+ "partial": [],+ "score": 0.486,+ "critical": 3,+ "major": 3,+ "flagged": [+ "lufs_i low (critical)",+ "p_real low (critical)",+ "music_frac low (critical)",+ "crest_lu high (major)",+ "first_sound_s high (major)",+ "onset_rate low (major)",+ "band_sub low (minor)",+ "band_low low (minor)",+ "band_mid high (minor)",+ "side_to_mid_db high (minor)"+ ],+ "first_action": "realness: likely code-synthesized (p_real 0.10, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:08.00-0:12.00 (0.01), 0:12.00-0:16.00 (0.03), 0:16.00-0:20.00 (0.01), 0:28.00-0:32.00 (0.04), 0:52.00-0:56.00 (0.02), 0:56.00-1:00.00 (0.02) (+6 more): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with `earshot realness stem.wav` (a finished mix is harder to judge than its sources)",+ "p_real": 0.095,+ "seconds": 43.7+ },+ "sine": {+ "what": "30 s 440 Hz sine",+ "models": true,+ "picture": false,+ "expect": [+ "p_real",+ "music_frac",+ "onset_rate"+ ],+ "caught": [+ "p_real",+ "music_frac",+ "onset_rate"+ ],+ "verdict": "not launch-grade",+ "partial": [],+ "score": 0.351,+ "critical": 3,+ "major": 4,+ "flagged": [+ "lufs_i low (critical)",+ "p_real low (critical)",+ "music_frac low (critical)",+ "lra low (major)",+ "crest_lu low (major)",+ "onset_rate low (major)",+ "hit_rate low (major)",+ "band_sub low (minor)",+ "band_low low (minor)",+ "band_lowmid high (minor)",+ "band_mid low (minor)",+ "band_presence low (minor)",+ "band_air low (minor)",+ "side_to_mid_db low (minor)"+ ],+ "first_action": "realness: likely code-synthesized (p_real 0.03, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:00.00-0:04.00 (0.03), 0:04.00-0:08.00 (0.03), 0:08.00-0:12.00 (0.03), 0:12.00-0:16.00 (0.03), 0:16.00-0:20.00 (0.03), 0:20.00-0:24.00 (0.03) (+2 more): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with `earshot realness stem.wav` (a finished mix is harder to judge than its sources)",+ "p_real": 0.03,+ "seconds": 47.5+ },+ "pink": {+ "what": "30 s pink noise",+ "models": true,+ "picture": false,+ "expect": [+ "p_real",+ "music_frac",+ "onset_rate"+ ],+ "caught": [+ "p_real",+ "music_frac",+ "onset_rate"+ ],+ "verdict": "not launch-grade",+ "partial": [],+ "score": 0.459,+ "critical": 3,+ "major": 4,+ "flagged": [+ "lufs_i low (critical)",+ "p_real low (critical)",+ "music_frac low (critical)",+ "lra low (major)",+ "crest_lu low (major)",+ "onset_rate low (major)",+ "hit_rate low (major)",+ "band_mid high (minor)",+ "band_presence high (minor)",+ "side_to_mid_db low (minor)"+ ],+ "first_action": "realness: likely code-synthesized (p_real 0.16, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:00.00-0:04.00 (0.12), 0:08.00-0:12.00 (0.16), 0:12.00-0:16.00 (0.15), 0:20.00-0:24.00 (0.17), 0:24.00-0:28.00 (0.11), 0:26.00-0:30.00 (0.15): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with `earshot realness stem.wav` (a finished mix is harder to judge than its sources)",+ "p_real": 0.158,+ "seconds": 22.7+ }+}\ No newline at end of filediff --git a/eval/launch/refs.json b/eval/launch/refs.jsonnew file mode 100644index 0000000..ec23a16--- /dev/null+++ b/eval/launch/refs.json@@ -0,0 +1,25 @@+{+ "note": "The 7 professional launch films used as the quality bar (third-party, measured only, never copied or redistributed), as ref_1..ref_7 with neutral style lines. Local file locations come from eval/local_paths.json or EARSHOT_EVAL_REFS (see eval/README.md); test files are relative to the film project (EARSHOT_EVAL_FILM).",+ "refs": [+ {"id": "ref_1",+ "style": "AI video-search SaaS launch: white UI walkthrough with chat prompts and cursor, kinetic type, thumbnail-wall reveals, logo end card; voice-over"},+ {"id": "ref_2",+ "style": "AI workspace explainer: white minimal kinetic type, app-icon clusters, chat UI, photographic logo reveal, gradient end card; voice-over"},+ {"id": "ref_3",+ "style": "editorial kinetic collage: grainy paper texture, typewriter text, photo cut-outs, pixel icons, hand-drawn strokes, fast cuts; voice-over"},+ {"id": "ref_4",+ "style": "fintech brand film: bold black and green kinetic typography, grids and numerals, flicker cuts, logo stings; music only"},+ {"id": "ref_5",+ "style": "AI image-generation launch: soft gradients, typed prompt, collage of generated photos; sound design with hard silences"},+ {"id": "ref_6",+ "style": "creator-commerce SaaS: blue UI walkthrough, notifications, cursor clicks, data-flow diagrams, tagline end card; voice-over"},+ {"id": "ref_7",+ "style": "gaming motion-design reel: logo stings, 3D explosions, saturated colour, very fast cuts, brickwall-loud music"}+ ],+ "tests": [+ {"id": "clawd_real", "file": "out/Clawd_1080p60.mp4",+ "style": "Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)"},+ {"id": "clawd_synth", "file": "out/superseded/Clawd_1080p60_synth-audio.mp4",+ "style": "the same film with the retired code-synthesized soundtrack"}+ ]+}diff --git a/eval/launch/results.json b/eval/launch/results.jsonnew file mode 100644index 0000000..0003730--- /dev/null+++ b/eval/launch/results.json@@ -0,0 +1,702 @@+{+ "loo": [+ {+ "id": "ref_1",+ "verdict": "close",+ "score": 0.75,+ "in_range": 17,+ "scored": 22,+ "critical": 0,+ "major": 4,+ "out": [+ "true_peak high",+ "lra low",+ "first_sound_s high",+ "end_silence_s high",+ "side_to_mid_db low"+ ],+ "refs_basis_in_range": 16,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "true_peak high"+ ]+ },+ {+ "id": "ref_2",+ "verdict": "launch-grade",+ "score": 0.944,+ "in_range": 21,+ "scored": 22,+ "critical": 0,+ "major": 1,+ "out": [+ "true_peak high"+ ],+ "refs_basis_in_range": 20,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "true_peak high"+ ]+ },+ {+ "id": "ref_3",+ "verdict": "close",+ "score": 0.833,+ "in_range": 17,+ "scored": 22,+ "critical": 0,+ "major": 1,+ "out": [+ "true_peak high",+ "end_lu low",+ "band_sub low",+ "band_lowmid high",+ "band_air low"+ ],+ "refs_basis_in_range": 16,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "true_peak high"+ ]+ },+ {+ "id": "ref_4",+ "verdict": "close",+ "score": 0.889,+ "in_range": 19,+ "scored": 22,+ "critical": 0,+ "major": 1,+ "out": [+ "onset_rate high",+ "intro_lu low",+ "band_mid low"+ ],+ "refs_basis_in_range": 17,+ "refs_basis_scored": 20,+ "in_sample_out": []+ },+ {+ "id": "ref_5",+ "verdict": "needs work",+ "score": 0.556,+ "in_range": 14,+ "scored": 22,+ "critical": 2,+ "major": 4,+ "out": [+ "lufs_i low",+ "true_peak high",+ "music_frac low",+ "cuts_with_sound low",+ "lra high",+ "max_gap_s high",+ "band_air high",+ "side_to_mid_db high"+ ],+ "refs_basis_in_range": 14,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "lufs_i low",+ "true_peak high"+ ]+ },+ {+ "id": "ref_6",+ "verdict": "close",+ "score": 0.857,+ "in_range": 19,+ "scored": 22,+ "critical": 0,+ "major": 2,+ "out": [+ "lufs_i low",+ "true_peak high",+ "intro_lu high"+ ],+ "refs_basis_in_range": 19,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "lufs_i low",+ "true_peak high"+ ]+ },+ {+ "id": "ref_7",+ "verdict": "close",+ "score": 0.771,+ "in_range": 18,+ "scored": 22,+ "critical": 0,+ "major": 4,+ "out": [+ "lufs_i high",+ "true_peak high",+ "onset_rate low",+ "hit_rate low"+ ],+ "refs_basis_in_range": 18,+ "refs_basis_scored": 20,+ "in_sample_out": [+ "lufs_i high",+ "true_peak high"+ ]+ }+ ],+ "per_metric": {+ "lufs_i": {+ "ok": 4,+ "low": 2,+ "high": 1,+ "na": 0+ },+ "true_peak": {+ "ok": 1,+ "low": 0,+ "high": 6,+ "na": 0+ },+ "music_frac": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "cuts_with_sound": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "lra": {+ "ok": 5,+ "low": 1,+ "high": 1,+ "na": 0+ },+ "crest_lu": {+ "ok": 7,+ "low": 0,+ "high": 0,+ "na": 0+ },+ "first_sound_s": {+ "ok": 6,+ "low": 0,+ "high": 1,+ "na": 0+ },+ "end_silence_s": {+ "ok": 6,+ "low": 0,+ "high": 1,+ "na": 0+ },+ "max_gap_s": {+ "ok": 6,+ "low": 0,+ "high": 1,+ "na": 0+ },+ "final_lu": {+ "ok": 7,+ "low": 0,+ "high": 0,+ "na": 0+ },+ "onset_rate": {+ "ok": 5,+ "low": 1,+ "high": 1,+ "na": 0+ },+ "hit_rate": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "intro_lu": {+ "ok": 5,+ "low": 1,+ "high": 1,+ "na": 0+ },+ "end_lu": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "fade_s": {+ "ok": 7,+ "low": 0,+ "high": 0,+ "na": 0+ },+ "band_sub": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "band_low": {+ "ok": 7,+ "low": 0,+ "high": 0,+ "na": 0+ },+ "band_lowmid": {+ "ok": 6,+ "low": 0,+ "high": 1,+ "na": 0+ },+ "band_mid": {+ "ok": 6,+ "low": 1,+ "high": 0,+ "na": 0+ },+ "band_presence": {+ "ok": 7,+ "low": 0,+ "high": 0,+ "na": 0+ },+ "band_air": {+ "ok": 5,+ "low": 1,+ "high": 1,+ "na": 0+ },+ "side_to_mid_db": {+ "ok": 5,+ "low": 1,+ "high": 1,+ "na": 0+ }+ },+ "clawd": {+ "clawd_real": {+ "style": "Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)",+ "verdict": "close",+ "score": 0.897,+ "in_range": 21,+ "scored": 23,+ "critical": 0,+ "major": 2,+ "actions": [+ {+ "weight": "major",+ "metric": "first_sound_s",+ "action": "nothing reaches body level until 0:01.01 (before that the sound sits around -42 LUFS, 27 LU under the body; refs 0.04 s .. 0.48 s): raise the opening or start the music at 0:00"+ },+ {+ "weight": "major",+ "metric": "crest_lu",+ "action": "the loudest 3 s (around 0:41.50, -10.4 LUFS) stand 4.6 LU above the integrated level (refs 1.1 LU .. 3.5 LU): the rest of the film plays quiet next to it; lift the body (quietest stretch 0:06.20-0:09.20 at -24.5 LUFS short-term) or tame that peak"+ }+ ],+ "tips": [+ "5 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.30, 0:00.80, 0:01.30, 0:30.00, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down",+ "no stop-downs: 4 of 7 references drop the sound out for 0.2-1 s right before a big hit; try one before the biggest visual change at 0:20.00"+ ],+ "status": {+ "lufs_i": [+ -15.0,+ "ok"+ ],+ "true_peak": [+ -1.4,+ "ok"+ ],+ "p_real": [+ 0.942,+ "ok"+ ],+ "music_frac": [+ 1.0,+ "ok"+ ],+ "cuts_with_sound": [+ 0.62,+ "ok"+ ],+ "lra": [+ 10.4,+ "ok"+ ],+ "crest_lu": [+ 4.6,+ "high"+ ],+ "first_sound_s": [+ 1.01,+ "high"+ ],+ "end_silence_s": [+ 0.09,+ "ok"+ ],+ "max_gap_s": [+ 0.0,+ "ok"+ ],+ "final_lu": [+ 0.3,+ "ok"+ ],+ "onset_rate": [+ 2.48,+ "ok"+ ],+ "hit_rate": [+ 0.27,+ "ok"+ ],+ "intro_lu": [+ -7.3,+ "ok"+ ],+ "end_lu": [+ -1.7,+ "ok"+ ],+ "fade_s": [+ 0.23,+ "ok"+ ],+ "band_sub": [+ -13.8,+ "ok"+ ],+ "band_low": [+ -3.8,+ "ok"+ ],+ "band_lowmid": [+ -3.9,+ "ok"+ ],+ "band_mid": [+ -9.5,+ "ok"+ ],+ "band_presence": [+ -18.7,+ "ok"+ ],+ "band_air": [+ -19.6,+ "ok"+ ],+ "side_to_mid_db": [+ -4.3,+ "ok"+ ],+ "speech_frac": [+ 0.0,+ "info"+ ],+ "tempo": [+ 120,+ "info"+ ],+ "peak_pos": [+ 0.69,+ "info"+ ],+ "tail_s": [+ 2.4,+ "info"+ ],+ "stopdowns_min": [+ 0.0,+ "info"+ ],+ "cut_rate": [+ 0.43,+ "info"+ ],+ "cut_hit_frac": [+ 0.19,+ "info"+ ],+ "cut_frame_sync": [+ 0.38,+ "info"+ ],+ "cuts_on_beat": [+ 0.69,+ "info"+ ],+ "brightness_hz": [+ 2088,+ "info"+ ],+ "correlation": [+ 0.46,+ "info"+ ]+ },+ "realness": "Clawd_1080p60.mp4: real or pro-produced (p_real 0.94, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 0 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score +2.9. No window leans synthesized (lowest p_real 0.76); that does not rule out a code-made layer under a louder real mix (most are missed): judge the source files and stems.",+ "seconds_critique": 4.6+ },+ "clawd_synth": {+ "style": "the same film with the retired code-synthesized soundtrack",+ "verdict": "needs work",+ "score": 0.795,+ "in_range": 19,+ "scored": 23,+ "critical": 1,+ "major": 2,+ "actions": [+ {+ "weight": "critical",+ "metric": "p_real",+ "action": "realness: likely code-synthesized (p_real 0.33, at or below the model's 0.40 threshold); 4 s windows at or below the window threshold 0.17 (code-synthesized): 0:56.00-1:00.00 (0.05): replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the sourcing and licence workflow); judge each stem with `earshot realness stem.wav` (a finished mix is harder to judge than its sources)"+ },+ {+ "weight": "major",+ "metric": "final_lu",+ "action": "the final moment sits 7.5 LU below the body: end on a logo sting / hit at the last visual event (0:59.00) instead of fading away (refs -2.2 LU .. 5.6 LU)"+ },+ {+ "weight": "major",+ "metric": "onset_rate",+ "action": "sparse: 2.1 onsets/s (refs 2.46 /s .. 8.22 /s): use music with more rhythmic drive, or add UI foley (clicks, typing, ticks) on the on-screen actions"+ },+ {+ "weight": "minor",+ "metric": "fade_s",+ "action": "4.4 s fade-out (refs 0.08 s .. 2.54 s): end on a sting with a short ring-out, or cut hard on the last picture cut"+ }+ ],+ "tips": [+ "8 of 26 visual events get no sound at all (no onset within 100 ms, no whoosh / riser peak within 300 ms, no level peak within 150 ms; the matcher earshot timeline uses): 0:00.80, 0:01.30, 0:28.33, 0:28.98, 0:30.00, 0:33.00, 0:40.88, 0:51.85; give the ones that matter a whoosh into the cut, a hit or a UI click, or keep one as a deliberate stop-down"+ ],+ "status": {+ "lufs_i": [+ -14.0,+ "ok"+ ],+ "true_peak": [+ -2.1,+ "ok"+ ],+ "p_real": [+ 0.327,+ "low"+ ],+ "music_frac": [+ 1.0,+ "ok"+ ],+ "cuts_with_sound": [+ 0.54,+ "ok"+ ],+ "lra": [+ 7.2,+ "ok"+ ],+ "crest_lu": [+ 3.3,+ "ok"+ ],+ "first_sound_s": [+ 0.0,+ "ok"+ ],+ "end_silence_s": [+ 0.51,+ "ok"+ ],+ "max_gap_s": [+ 0.04,+ "ok"+ ],+ "final_lu": [+ -7.5,+ "low"+ ],+ "onset_rate": [+ 2.1,+ "low"+ ],+ "hit_rate": [+ 0.38,+ "ok"+ ],+ "intro_lu": [+ -4.7,+ "ok"+ ],+ "end_lu": [+ -7.4,+ "ok"+ ],+ "fade_s": [+ 4.4,+ "high"+ ],+ "band_sub": [+ -3.1,+ "ok"+ ],+ "band_low": [+ -5.0,+ "ok"+ ],+ "band_lowmid": [+ -8.4,+ "ok"+ ],+ "band_mid": [+ -13.7,+ "ok"+ ],+ "band_presence": [+ -23.4,+ "ok"+ ],+ "band_air": [+ -29.3,+ "ok"+ ],+ "side_to_mid_db": [+ -12.2,+ "ok"+ ],+ "speech_frac": [+ 0.0,+ "info"+ ],+ "tempo": [+ 120,+ "info"+ ],+ "peak_pos": [+ 0.77,+ "info"+ ],+ "tail_s": [+ 1.47,+ "info"+ ],+ "stopdowns_min": [+ 1.0,+ "info"+ ],+ "cut_rate": [+ 0.43,+ "info"+ ],+ "cut_hit_frac": [+ 0.23,+ "info"+ ],+ "cut_frame_sync": [+ 0.31,+ "info"+ ],+ "cuts_on_beat": [+ 0.62,+ "info"+ ],+ "brightness_hz": [+ 1756,+ "info"+ ],+ "correlation": [+ 0.89,+ "info"+ ]+ },+ "realness": "Clawd_1080p60_synth-audio.mp4: likely code-synthesized (p_real 0.33, mean of 15 4 s windows; file thresholds lo 0.4 / hi 0.77, weak calibration): 14 of 15 windows lean synthesized (p_real < 0.5); mean CLAP head score -0.5. Leaning synthesized: 0:00.00-0:48.00, 0:52.00-1:00.00; 1 at or below the 4 s-window threshold 0.17 (code-synthesized): 0:56.00-1:00.00.",+ "seconds_critique": 4.6+ }+ },+ "detector": [+ {+ "id": "ref_1",+ "scene_cuts": 0,+ "visual_events": 28,+ "same_as_first_build": true+ },+ {+ "id": "ref_2",+ "scene_cuts": 2,+ "visual_events": 9,+ "same_as_first_build": true+ },+ {+ "id": "ref_3",+ "scene_cuts": 15,+ "visual_events": 17,+ "recall_of_scene_cuts": 1.0,+ "same_as_first_build": true+ },+ {+ "id": "ref_4",+ "scene_cuts": 17,+ "visual_events": 42,+ "recall_of_scene_cuts": 0.82,+ "same_as_first_build": true+ },+ {+ "id": "ref_5",+ "scene_cuts": 2,+ "visual_events": 16,+ "same_as_first_build": true+ },+ {+ "id": "ref_6",+ "scene_cuts": 11,+ "visual_events": 31,+ "recall_of_scene_cuts": 1.0,+ "same_as_first_build": true+ },+ {+ "id": "ref_7",+ "scene_cuts": 40,+ "visual_events": 67,+ "recall_of_scene_cuts": 0.8,+ "same_as_first_build": true+ },+ {+ "id": "clawd_real",+ "scene_cuts": 10,+ "visual_events": 26,+ "recall_of_scene_cuts": 1.0,+ "same_as_first_build": true+ },+ {+ "id": "clawd_synth",+ "scene_cuts": 10,+ "visual_events": 26,+ "recall_of_scene_cuts": 1.0,+ "same_as_first_build": true+ }+ ],+ "hit_timing": {+ "placements": 264,+ "hits": 16,+ "onsets": 149,+ "hit_offset_median_ms": 1.0,+ "hits_within_2ms": 11,+ "hit_precision_20ms": 0.75,+ "hit_chance_20ms": 0.1,+ "onset_precision_20ms": 0.61,+ "placement_recall_20ms": 0.62,+ "hit_precision_50ms": 0.88,+ "hit_chance_50ms": 0.25,+ "onset_precision_50ms": 0.75,+ "placement_recall_50ms": 0.78,+ "hit_precision_100ms": 0.88,+ "hit_chance_100ms": 0.39,+ "onset_precision_100ms": 0.79,+ "placement_recall_100ms": 0.83+ }+}\ No newline at end of filediff --git a/eval/launch/run_eval.py b/eval/launch/run_eval.pynew file mode 100644index 0000000..bc289cc--- /dev/null+++ b/eval/launch/run_eval.py@@ -0,0 +1,439 @@+"""Evaluate the launch profile and the critique scorecard.++ .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_1,ref_2 --fresh # measure films (cold), in chunks+ .venv/Scripts/python.exe eval/launch/run_eval.py --neg quiet,hot,mono # known-bad mixes, in chunks+ .venv/Scripts/python.exe eval/launch/run_eval.py --build # + write earshot/data/launch_profile.json+ .venv/Scripts/python.exe eval/launch/run_eval.py # LOO + Clawd + RESULTS.md (cached films)++1. per film: earshot.profile.film_metrics for the 7 references and the two Clawd films; --fresh first deletes this+ module's audio-pass cache for the film (--fresh-ast also the AudioSet pass) so that pass is timed cold. Times and+ metrics go to eval/launch/films.json (numbers only).+2. --build: the shipped profile (earshot/data/launch_profile.json) from the 7 references, ids and styles from refs.json.+3. leave-one-out: each reference scored against a profile of the other six (do the ranges generalise?), plus the+ in-sample check against all seven.+4. the Clawd film with its real soundtrack vs the same film with the retired synthesized soundtrack: full critique+ against the 7-film profile (with earshot.realness when its model is installed).+5. sensitivity: known-bad mixes (the Clawd real soundtrack degraded one way at a time, its SFX / ambience stems alone,+ the retired synthesized bed, a sine, pink noise; scratch audio in out/launch/neg/, never committed): is the metric+ the defect should move flagged, and what is the verdict? Results go to eval/launch/negatives.json.+6. the visual-event detector (earshot.video frame difference) vs ffmpeg scene cuts (video.cuts(method='scene')), and+ a check that the core detector reproduces this module's first-build picture pass exactly.+Writes eval/launch/results.json and eval/launch/RESULTS.md.+"""+import hashlib+import argparse+import json+import os+import sys+import time++import subprocess # noqa: E402++import numpy as np # noqa: E402++HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.path.dirname(os.path.dirname(HERE))+sys.path.insert(0, ROOT)+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++from earshot import audio, config, video # noqa: E402+from earshot import profile as P # noqa: E402+from earshot.critique import SPEC, critique, real_thresholds, scorecard # noqa: E402++REFS = json.load(open(os.path.join(HERE, 'refs.json'), encoding='utf-8'))+for _r in REFS['refs']:+ _r['path'] = EP.ref(_r['id']) # local files, see eval/README.md+for _r in REFS['tests']:+ _r['path'] = f"{EP.FILM}/{_r['file']}"+FILMS = os.path.join(HERE, 'films.json')+NEGS = os.path.join(HERE, 'negatives.json')+RESULTS = os.path.join(HERE, 'results.json')+NEG_DIR = os.path.join(ROOT, 'out', 'launch', 'neg')+ST = f'{EP.FILM}/out/audio/soundtrack.wav'+STEMS = f'{EP.FILM}/out/audio/stems_pro/'+FILM = f'{EP.FILM}/out/Clawd_1080p60.mp4'+# id: (source or lavfi graph, ffmpeg -af filter, models (AudioSet + realness), picture, metrics the defect should flag,+# what it is). Expectations were written down before the first run.+NEGATIVES = {+ 'base': (ST, None, False, None, (), 'control: the real Clawd soundtrack untouched'),+ 'quiet': (ST, 'volume=-12dB', False, None, ('lufs_i',), 'unmastered: 12 dB quieter'),+ 'hot': (ST, 'volume=12dB', False, None, ('lufs_i', 'true_peak'), '12 dB hotter, clipped by the 16-bit file'),+ 'mono': (ST, 'pan=stereo|c0=0.5*c0+0.5*c1|c1=0.5*c0+0.5*c1', False, None, ('side_to_mid_db',), 'folded to mono'),+ 'muffled': (ST, 'lowpass=f=3000,lowpass=f=3000', False, None, ('band_presence', 'band_air'),+ 'low-passed at 3 kHz (12 dB/oct)'),+ 'late': (ST, 'adelay=2500:all=1,atrim=0:60', False, None, ('first_sound_s',), '2.5 s of silence before the start'),+ 'deadair': (ST, 'apad=pad_dur=4', False, None, ('end_silence_s',), '4 s of silence after the end'),+ 'fadeout': (ST, 'afade=t=out:st=48:d=12', False, None, ('fade_s', 'final_lu', 'end_lu'), '12 s fade-out'),+ 'hole': (ST, "volume=enable='between(t,25,27.5)':volume=0", False, None, ('max_gap_s',), '2.5 s muted mid-film'),+ 'squashed': (ST, 'acompressor=threshold=-36dB:ratio=20:attack=1:release=60:makeup=12,alimiter=limit=0.7', False,+ None, ('lra', 'crest_lu'), 'over-compressed: 20:1 from -36 dB + limiter'),+ 'sfx_only': (STEMS + 'sfx.wav', None, True, FILM, ('music_frac',), 'the SFX stem alone, with the picture'),+ 'amb_only': (STEMS + 'amb.wav', None, True, None, ('music_frac', 'onset_rate', 'hit_rate'), 'the ambience stem alone'),+ 'synth_bed': (f'{EP.FILM}/audio/_retired_synth/bed.wav', None, True, None, ('p_real',),+ 'the retired code-synthesized music bed'),+ 'sine': ('lavfi:sine=f=440:d=30:sample_rate=48000', 'volume=0.3', True, None, ('p_real', 'music_frac', 'onset_rate'),+ '30 s 440 Hz sine'),+ 'pink': ('lavfi:anoisesrc=color=pink:d=30:a=0.3:r=48000', None, True, None, ('p_real', 'music_frac', 'onset_rate'),+ '30 s pink noise'),+}+++def _all():+ return REFS['refs'] + REFS['tests']+++def _clear(path, ast=False):+ for k, f in P.cache_paths(path).items():+ if os.path.exists(f) and (ast or k != 'ast'):+ os.remove(f)+++def measure(only, fresh, fresh_ast=False):+ films = json.load(open(FILMS)) if os.path.exists(FILMS) else {}+ for r in _all():+ if only and r['id'] not in only:+ continue+ if fresh:+ _clear(r['path'], fresh_ast)+ t0 = time.time()+ f = P.film_metrics(r['path'])+ cold = round(time.time() - t0, 1)+ t0 = time.time()+ P.film_metrics(r['path'])+ warm = round(time.time() - t0, 1)+ old = films.get(r['id'], {})+ films[r['id']] = {'id': r['id'], 'style': r['style'], 'duration': f['duration'], 'fps': f['fps'],+ 'lossy': f['lossy'],+ 'metrics': f['metrics'], 'chance': f['chance'], 'arc': f['arc'],+ 'video': bool(f['video']),+ 'n_visual': len(f['events']['visual']), 'n_hits': len(f['events']['hits']),+ 'seconds_cold': cold if fresh else old.get('seconds_cold'), 'seconds_warm': warm,+ 'timing_cold': f['timing'] if fresh else old.get('timing_cold'),+ 'audioset_cold_s': (f['timing'].get('audioset') if fresh_ast else+ old.get('audioset_cold_s', (old.get('timing_cold') or {}).get('audioset')))}+ print(f"{r['id']}: {f['duration']:.1f} s film, {'cold ' + str(cold) + ' s, ' if fresh else ''}warm {warm} s | "+ f"{f['summary']}", flush=True)+ json.dump(films, open(FILMS, 'w'), indent=1)+ return films+++def loo(films):+ refs = [films[r['id']] for r in REFS['refs']]+ full = P.aggregate(refs)+ out = []+ for i, f in enumerate(refs):+ others = refs[:i] + refs[i + 1:]+ sc = scorecard(f['metrics'], P.aggregate(others), f['chance'], f['lossy'])+ ins = scorecard(f['metrics'], full, f['chance'], f['lossy'])+ rows = [r for r in sc['metrics'] if r['weight'] != 'info' and r['status'] != 'n.a.']+ out.append({'id': f['id'], 'verdict': sc['verdict'], 'score': sc['score'], 'in_range': sc['in_range'],+ 'scored': sc['scored'], 'critical': sc['critical_issues'], 'major': sc['major_issues'],+ 'out': [f"{r['metric']} {r['status']}" for r in rows if r['status'] != 'ok'],+ 'refs_basis_in_range': sum(1 for r in rows if r['basis'] == 'references' and r['status'] == 'ok'),+ 'refs_basis_scored': sum(1 for r in rows if r['basis'] == 'references'),+ 'in_sample_out': [f"{r['metric']} {r['status']}" for r in ins['metrics']+ if r['weight'] != 'info' and r['status'] in ('low', 'high')]})+ per_metric = {}+ for k in SPEC:+ if SPEC[k][0] == 0 or k == 'p_real':+ continue+ st = []+ for i, f in enumerate(refs):+ sc = scorecard(f['metrics'], P.aggregate(refs[:i] + refs[i + 1:]), lossy=f['lossy'])+ st.append(next(r['status'] for r in sc['metrics'] if r['metric'] == k))+ per_metric[k] = {'ok': st.count('ok'), 'low': st.count('low'), 'high': st.count('high'), 'na': st.count('n.a.')}+ return out, per_metric+++def clawd():+ res = {}+ for r in REFS['tests']:+ t0 = time.time()+ c = critique(r['path'])+ res[r['id']] = {'style': r['style'], 'verdict': c['verdict'], 'score': c['score'], 'in_range': c['in_range'],+ 'scored': c['scored'], 'critical': c['critical_issues'], 'major': c['major_issues'],+ 'actions': c['actions'], 'tips': c['tips'],+ 'status': {m['metric']: [m['value'], m['status']] for m in c['metrics']},+ 'realness': (c.get('realness') or {}).get('summary') or (c.get('realness') or {}).get('error'),+ 'seconds_critique': round(time.time() - t0, 1)}+ print(f"{r['id']}: {c['summary']}", flush=True)+ return res+++def detector_check():+ out = []+ for r in _all():+ fps, d = video.frame_diff(r['path'])+ ev = video.diff_events(d, fps) / fps+ hc = video.cuts(r['path'], method='scene')+ row = {'id': r['id'], 'scene_cuts': len(hc), 'visual_events': int(len(ev))}+ if len(hc) >= 5:+ row['recall_of_scene_cuts'] = round(P.near_frac(hc, ev, 1.0 / fps + 1e-3), 2)+ st = os.stat(r['path']) # the first build's cache key used the file id of that time (path, size, mtime only)+ fid = hashlib.sha1(f"{os.path.abspath(r['path'])}|{st.st_size}|{st.st_mtime_ns}".encode()).hexdigest()[:20]+ k = hashlib.sha1('|'.join(map(str, (fid, 96, 54, 1))).encode()).hexdigest()[:20]+ old = os.path.join(config.EMB_CACHE, 'launch', f'frames_{k}.npz')+ if os.path.exists(old):+ with np.load(old) as z:+ row['same_as_first_build'] = bool(np.array_equal(z['d'], d))+ out.append(row)+ return out+++def _make_neg(nid):+ src, af = NEGATIVES[nid][:2]+ if not af and not src.startswith('lavfi:'):+ return src+ os.makedirs(NEG_DIR, exist_ok=True)+ out = os.path.join(NEG_DIR, f'{nid}.wav')+ if not os.path.exists(out):+ inp = ['-f', 'lavfi', '-i', src[6:]] if src.startswith('lavfi:') else ['-i', src]+ subprocess.run(['ffmpeg', '-v', 'error', '-y', *inp, *(['-af', af] if af else []), '-ac', '2', '-ar', '48000',+ '-c:a', 'pcm_s16le', out], check=True)+ return out+++def negatives(ids):+ res = json.load(open(NEGS)) if os.path.exists(NEGS) else {}+ for nid in ids:+ src, af, models, vid, expect, what = NEGATIVES[nid]+ path = _make_neg(nid)+ t0 = time.time()+ c = critique(path, video=vid, use_ast=models, use_realness=models)+ st = {m['metric']: m['status'] for m in c['metrics']}+ flagged = [f"{m['metric']} {m['status']} ({m['weight']})" for m in c['metrics']+ if m['status'] in ('low', 'high')]+ res[nid] = {'what': what, 'models': models, 'picture': bool(vid), 'expect': list(expect),+ 'caught': [k for k in expect if st.get(k) in ('low', 'high')], 'verdict': c['verdict'],+ 'partial': c.get('partial'), 'score': c['score'], 'critical': c['critical_issues'],+ 'major': c['major_issues'], 'flagged': flagged,+ 'first_action': c['actions'][0]['action'] if c['actions'] else None,+ 'p_real': next((m['value'] for m in c['metrics'] if m['metric'] == 'p_real'), None),+ 'seconds': round(time.time() - t0, 1)}+ print(f"{nid}: {c['summary'][:300]}", flush=True)+ json.dump(res, open(NEGS, 'w'), indent=1)+ return res+++def hit_timing_check():+ """tuning data: detected hits / onsets of the Clawd real soundtrack vs the sample placements in its mix report"""+ mr = f'{EP.FILM}/out/qa/mix_report.json'+ film = next(r['path'] for r in REFS['tests'] if r['id'] == 'clawd_real')+ if not os.path.exists(mr):+ return None+ pl = sorted(p['t'] for p in json.load(open(mr))['placements'])+ f = P.film_metrics(film)+ on = P._audio_pass(film)['onsets']+ hits = f['events']['hits']+ off = [1000 * min(abs(p - t) for p in pl) for t in hits]+ row = {'placements': len(pl), 'hits': len(hits), 'onsets': len(on),+ 'hit_offset_median_ms': round(float(sorted(off)[len(off) // 2]), 1) if off else None,+ 'hits_within_2ms': sum(1 for o in off if o <= 2.0)}+ for tol in (0.02, 0.05, 0.1):+ row[f'hit_precision_{int(tol * 1000)}ms'] = round(P.near_frac(hits, pl, tol), 2)+ row[f'hit_chance_{int(tol * 1000)}ms'] = round(P.chance_frac(hits, pl, tol, f['duration']), 2)+ row[f'onset_precision_{int(tol * 1000)}ms'] = round(P.near_frac(on, pl, tol), 2)+ row[f'placement_recall_{int(tol * 1000)}ms'] = round(P.near_frac(pl, on, tol), 2)+ return row+++def _fmt(k, v):+ return 'n.a.' if v is None else P.fmt_value(k, v) if k in P.METRICS else str(v)+++def report(films, loo_rows, per_metric, cl, det, ht, neg=None):+ L = ['# Launch profile + critique: evaluation', '',+ 'Generated by `eval/launch/run_eval.py` (numbers only; the films are the user\'s local reference files, '+ 'never copied). Profile = `earshot/data/launch_profile.json`, built from the 7 references below.', '',+ '## Commands', '',+ '```',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_1,ref_2,ref_3 --fresh --measure-only',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --only ref_4,ref_5,ref_6,ref_7,clawd_real,'+ 'clawd_synth --fresh --measure-only',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --build # profile, LOO, Clawd, this file',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg base,quiet,hot,mono,muffled,late,deadair,'+ 'fadeout,hole,squashed',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sfx_only,amb_only,synth_bed',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py --neg sine,pink',+ 'EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/launch/run_eval.py # rewrite this file from the caches',+ '```', '',+ '## What was tuned on what', '',+ '- **Detector thresholds** (salient hit: >= 6 dB jump of the 100 ms level within 80 ms, landing within 8 LU '+ 'of integrated, one hit per run, timed at the attack; visual event: frame-difference peak >= 4x the median '+ 'of the surrounding second or >= 0.15, now earshot.video.diff_events; stop-down: >= 10 dB dip for 0.15-1.5 s) '+ 'were chosen by looking at plots of all 9 films (7 refs + 2 Clawd), at the Clawd mix-report placements (hits) '+ 'and at ffmpeg scene cuts (visual events). The Clawd real soundtrack is therefore partly tuning data for the '+ 'hit detector.',+ '- **Music threshold** (AudioSet Music >= 0.3 per 5 s window) was set after looking at the per-window '+ 'probabilities of all 9 films (music under voice or SFX scored 0.4-0.7, one sound-design stretch 0.1-0.15).',+ '- **Changed after the first leave-one-out run** (first build; not a blind test of these choices): true peak '+ 'went from critical to major and is judged against 0 dBTP for decoded lossy files (the -1 dBTP master rule '+ 'failed all 7 AAC-decoded references); a critical rule missed by <= 2 tolerances counts as major; the metric '+ 'max_gap_s was added; the music_frac tolerance went from 0.1 to 0.05 after an SFX-only stem (music in 42 % of '+ 'windows) came out as only a near miss.',+ '- **Changed after the independent review, before the known-bad set below was first run:** p_real uses the '+ "realness model's own thresholds (ok at its 'real or pro-produced' threshold, critical below 0.5, minor in "+ 'between; no near-miss downgrade); a density range that would reach 0 keeps a floor at half the sparsest '+ 'reference (salient hits >= 0.05 /s); a mono mix (identical channels) counts as -40 dB stereo width instead '+ 'of n.a.; files under 10 s get no loudness-shape rows; brightness_hz replaces the power-weighted centroid '+ '(info only). The known-bad expectations were written into run_eval.py before its first run; nothing was '+ 'changed after seeing its results.',+ '- **Not adjusted after seeing results:** the other tolerances, the weights of the other metrics and the '+ 'verdict thresholds (90 / 75 / 50 %).',+ "- **Ranges** are data: min-max of the reference films widened by each metric's tolerance. Loudness and true "+ 'peak use web-delivery practice (-16..-9 LUFS; <= -1 dBTP for a master, <= 0 dBTP for decoded AAC / MP3) '+ 'instead, so references can fail them.', '',+ '## Per-film measurements', '',+ '| film | dur s | LUFS | dBTP | LRA | music | speech | onsets/s | hits/s | vis events/s | events w/ sound '+ '(chance) | audio pass s (cold) | picture pass s | AudioSet pass s (cold, first build) | warm s |',+ '|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|']+ for r in _all():+ f = films.get(r['id'])+ if not f:+ continue+ m, c, tc = f['metrics'], f['chance'], f.get('timing_cold') or {}+ L.append(f"| {r['id']} | {f['duration']:.1f} | {m['lufs_i']} | {m['true_peak']} | {m['lra']} | "+ f"{_fmt('music_frac', m['music_frac'])} | {_fmt('speech_frac', m['speech_frac'])} | {m['onset_rate']} | "+ f"{m['hit_rate']} | {m['cut_rate']} | {_fmt('cuts_with_sound', m['cuts_with_sound'])} "+ f"({_fmt('cuts_with_sound', c.get('cuts_with_sound'))}) | {tc.get('audio')} | {tc.get('picture')} | "+ f"{f.get('audioset_cold_s')} | {f.get('seconds_warm')} |")+ L += ['', 'Audio pass = decode + features.measure(light) + librosa onsets / beats + loudness curves, timed cold in '+ "this round (this module's cache deleted; the core decode cache was cold too after its v2 bump). Picture "+ 'pass = earshot.video.frame_diff (core cache, shared with timeline / sheet): 0.0 means another module had '+ 'already decoded that film; ref_6 (4K) and clawd_synth were cold. AudioSet = 5 s windows through AST, '+ 'measured cold in the first build (same code and cache key, so not re-run): ~4-9 s per 5 s window on this '+ 'laptop under load from 3-4 other model jobs; `earshot critique --fast` skips it and realness. Warm = '+ 'everything cached. EARSHOT_THREADS=2.', '',+ '## Leave-one-out: each reference vs a profile of the other six', '',+ '| film | verdict | weighted score | scored metrics in range | reference-range metrics in range | out of range |',+ '|---|---|---|---|---|---|']+ n_ok = n_all = r_ok = r_all = 0+ for x in loo_rows:+ n_ok += x['in_range']; n_all += x['scored']; r_ok += x['refs_basis_in_range']; r_all += x['refs_basis_scored']+ L.append(f"| {x['id']} | {x['verdict']} | {100 * x['score']:.0f}% | {x['in_range']}/{x['scored']} | "+ f"{x['refs_basis_in_range']}/{x['refs_basis_scored']} | {', '.join(x['out']) or '-'} |")+ L += ['', f'Total: {n_ok}/{n_all} scored metric checks in range ({100 * n_ok / max(1, n_all):.0f} %); '+ f'reference-range metrics only: {r_ok}/{r_all} ({100 * r_ok / max(1, r_all):.0f} %). For exchangeable data '+ 'a held-out film falls outside the min-max of six others with probability 2/7 (29 %) per continuous metric '+ 'before tolerance, so ~71 % is the no-overfit baseline; the tolerance adds a little. p_real is not part of '+ 'this check (the realness model was trained on these films).', '',+ 'In-sample (each reference vs the full 7-film profile): every reference-range metric is in range by '+ 'construction; the only failures are web-practice rules: ' ++ '; '.join(f"{x['id']}: {', '.join(x['in_sample_out'])}" for x in loo_rows if x['in_sample_out']) + '.', '',+ '### Per metric (7 leave-one-out checks each)', '', '| metric | weight | ok | low | high | n.a. |',+ '|---|---|---|---|---|---|']+ for k, st in per_metric.items():+ L.append(f"| {k} | {['info', 'minor', 'major', 'critical'][SPEC[k][0]]} | {st['ok']} | {st['low']} | "+ f"{st['high']} | {st['na']} |")+ if neg:+ bad = {k: v for k, v in neg.items() if v['expect']}+ caught = sum(1 for v in bad.values() if v['caught'])+ L += ['', '## Sensitivity: known-bad mixes', '',+ 'Each row is one defect made on purpose (ffmpeg, scratch files in out/launch/neg/), critiqued against the '+ "7-film profile. 'Should flag' was written down before the first run; 'caught' = at least one of those "+ 'metrics came out low / high. Rows without models skip AudioSet and realness (partial verdicts); the '+ 'control is the untouched real soundtrack.', '',+ '| mix | what | should flag | caught | verdict | weighted score | all flagged rows | p_real | s (critique, cold) |',+ '|---|---|---|---|---|---|---|---|---|']+ for k, v in neg.items():+ verdict = v['verdict'] + (' (partial)' if v.get('partial') else '')+ L.append(f"| {k} | {v['what']} | {', '.join(v['expect']) or '-'} | {', '.join(v['caught']) or ('no' if v['expect'] else '-')} | "+ f"{verdict} | {100 * (v['score'] or 0):.0f}% | {'; '.join(v['flagged']) or '-'} | "+ f"{'-' if v['p_real'] is None else v['p_real']} | {v['seconds']} |")+ miss = [k for k, v in bad.items() if not v['caught']]+ L += ['', f'Caught: {caught} of {len(bad)} known-bad mixes have the expected metric flagged' ++ (f" (missed: {', '.join(miss)})" if miss else '') + '. Verdicts: ' ++ ', '.join(f"{n} {sum(1 for v in bad.values() if v['verdict'] == n)}" for n in+ ('launch-grade', 'close', 'needs work', 'not launch-grade')) ++ f" (control: {neg.get('base', {}).get('verdict', 'n.a.')}). A single structural defect on a mix that "+ 'is already close usually stays close: the verdict is coarse, the flagged rows and FIX FIRST actions '+ 'are the signal. Over-compression is not a defect by these references: two of them (LRA 1.7 / 3.3 LU, '+ 'loudest 3 s only 1.1-1.2 LU above integrated) are as dense as the squashed mix.']+ L += ['', '## The Clawd film: real soundtrack vs the retired synthesized soundtrack', '']+ r, s_ = cl.get('clawd_real'), cl.get('clawd_synth')+ if r and s_:+ pr, ps = r['status']['p_real'][0], s_['status']['p_real'][0]+ L += [f"Reading: the scorecard rates the real soundtrack {r['verdict']} ({100 * r['score']:.0f}%) and the "+ f"synthesized one {s_['verdict']} ({100 * s_['score']:.0f}%). The structural metrics separate them only "+ 'a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, '+ 'is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has '+ 'one loudness peak that sticks out. The difference the user hears most - synthesized timbre - is not '+ 'structural; it is what p_real measures' ++ (f": real {pr:.2f} vs synth {ps:.2f} (earshot.realness, verdict thresholds lo {real_thresholds()['lo']} "+ f"/ hi {real_thresholds()['hi']} at this run); below 0.5 the p_real row is critical, which is what "+ "moves the synth version's verdict." if pr is not None and ps is not None else+ '; the realness model was not installed when this ran.') ++ ' Caveat: the realness model (another module) was trained partly on these 7 launch films, on the Clawd '+ "project's real asset library and on the retired synthesized stems (eval/realness/), so p_real on the "+ 'Clawd films is not an independent test.', '']+ for k, c in cl.items():+ L += [f"### {k}: {c['style']}", '',+ f"Verdict **{c['verdict']}**, weighted score {100 * c['score']:.0f}%, {c['in_range']}/{c['scored']} in range, "+ f"{c['critical']} critical / {c['major']} major issues; critique call {c['seconds_critique']} s with the "+ f"measurement passes cached (realness runs inside it).", '']+ for a in c['actions']:+ L.append(f"- [{a['weight']}] {a['action']}")+ for t in c['tips']:+ L.append(f'- tip: {t}')+ L.append(f"- realness: {c['realness']}")+ L.append('')+ if ht:+ L += ['## Hit / onset timing vs ground truth (tuning data: the Clawd real soundtrack)', '',+ f"The mix report lists {ht['placements']} sample placements (every key tick, pop, whoosh, hit, music "+ f"motif note ...). earshot found {ht['hits']} salient hits and {ht['onsets']} onsets in the mixed film "+ f"audio; {ht['hits_within_2ms']} of the {ht['hits']} hits sit within 2 ms of a placement (median distance "+ f"{ht['hit_offset_median_ms']} ms).", '',+ '| tolerance | hits on a placement (chance) | onsets on a placement | placements with an onset |',+ '|---|---|---|---|']+ for ms in (20, 50, 100):+ L.append(f"| +-{ms} ms | {100 * ht[f'hit_precision_{ms}ms']:.0f}% ({100 * ht[f'hit_chance_{ms}ms']:.0f}%) | "+ f"{100 * ht[f'onset_precision_{ms}ms']:.0f}% | {100 * ht[f'placement_recall_{ms}ms']:.0f}% |")+ L += ['', 'Chance = the same hits shifted off the placements by fixed offsets. Hit and onset times are moved to '+ 'the steepest rise of the 2 ms level (attack), which is how a mixer aligns a sample.', '']+ L += ['## Visual-event detector (earshot.video frame difference) vs ffmpeg scene cuts', '',+ "Scene cuts = `video.cuts(path, method='scene')` (ffmpeg scene score 0.3). 'Same as first build' = the core "+ "video.frame_diff signal is bit-identical to this module's first-build picture pass, so moving the detector "+ 'into core changed no number.', '',+ '| film | scene cuts | visual events | share of scene cuts also found (+-1 frame) | same as first build |',+ '|---|---|---|---|---|']+ for d in det:+ rec = d.get('recall_of_scene_cuts', 'n.a. (< 5 scene cuts)')+ L.append(f"| {d['id']} | {d['scene_cuts']} | {d['visual_events']} | {rec} | "+ f"{d.get('same_as_first_build', 'n.a.')} |")+ return '\n'.join(L) + '\n'+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--only', help='comma-separated ids to (re)measure')+ ap.add_argument('--fresh', action='store_true', help='delete the audio-pass cache first (cold timings)')+ ap.add_argument('--fresh-ast', action='store_true', help='with --fresh: also delete the AudioSet cache')+ ap.add_argument('--neg', help='comma-separated known-bad mixes to critique (or "all"): ' + ','.join(NEGATIVES))+ ap.add_argument('--build', action='store_true', help='write earshot/data/launch_profile.json')+ ap.add_argument('--measure-only', action='store_true')+ a = ap.parse_args()+ only = set(a.only.split(',')) if a.only else None+ if a.neg:+ negatives(list(NEGATIVES) if a.neg == 'all' else a.neg.split(','))+ return+ films = (measure(only, a.fresh, a.fresh_ast) if (only or a.fresh or not os.path.exists(FILMS)) else+ json.load(open(FILMS)))+ if a.measure_only:+ return+ if a.build:+ P.profile([r['path'] for r in REFS['refs']], styles=[r['style'] for r in REFS['refs']], out=P.PROFILE_PATH)+ loo_rows, per_metric = loo(films)+ cl = clawd()+ det = detector_check()+ ht = hit_timing_check()+ neg = json.load(open(NEGS)) if os.path.exists(NEGS) else {}+ json.dump({'loo': loo_rows, 'per_metric': per_metric, 'clawd': cl, 'detector': det, 'hit_timing': ht},+ open(RESULTS, 'w'), indent=1)+ open(os.path.join(HERE, 'RESULTS.md'), 'w', encoding='utf-8').write(+ report(films, loo_rows, per_metric, cl, det, ht, neg))+ print('wrote', RESULTS, 'and RESULTS.md')+++if __name__ == '__main__':+ main()diff --git a/eval/realness/RESULTS.md b/eval/realness/RESULTS.mdnew file mode 100644index 0000000..c53b8f7--- /dev/null+++ b/eval/realness/RESULTS.md@@ -0,0 +1,330 @@+# Realness detector: held-out evaluation++Generated by `eval/realness/run_eval.py --stage report`; every number below comes from that run (`results.json` holds+all of them). The question: is a sound a real recording / professionally produced audio (positive), or amateur audio+synthesized in code: oscillators, filtered noise, simple envelopes, algorithmic reverb, beeps (negative)?+Professionally produced electronic music and pro sound design count as positive.++## Data+1484 clean clips (691 real/pro, 793 synthesized) plus 1863 perturbed copies+(robustness tests and training-only augmentation). A clip is a whole sound up to 6 s, else a 4 s window: exactly the+clips `earshot.realness` scores at inference. Audio stays in `out/realness/` (never committed).++| family | clean clips | what |+|---|---|---|+| cinetic | 318 | cinetic synth engine: generator types x seeds / parameters, short mixes, 6 full rendered scores |+| generic | 260 | independent numpy generators (14 types) + 6 generic code-made mixes |+| pro-film | 72 | audio of the 7 professional launch films (4 s windows) |+| pro-music | 119 | professionally produced music (assets/music, kept and rejected) |+| real-asset | 366 | Freesound / Mixkit recordings used in the Clawd film (7 categories) |+| real-rejected | 134 | real recordings rejected for quality (still real) |+| retired | 215 | the retired code-synthesized Clawd score: stems, beds, melody, events |++Tuning vs evaluation: model choices (regularisation C of the CLAP head, fusion weights) are fitted inside each training+fold by inner grouped CV; nothing is tuned on the held-out group. The verdict thresholds (lo 0.17 / hi 0.77)+are read from the scheme-M held-out predictions (>= 92.5% precision on each side), so the band numbers are in-sample+for that choice (two numbers fitted on 1484 clips); the file thresholds for long files likewise come from held-out+long files. Design choices made by looking at held-out results across the three build rounds: augmentation on+(robustness below), the precision target, music weighted as four sources (scheme M), the feature set, the file+threshold rule, and not training on layered clips (layer test below).+The final model (`earshot/data/realness_model.json`) is trained on all groups with the scheme-M grouping. Robustness copies are only ever scored, never trained+on; augmentation copies (synthetic noise floor, gating, short excerpts, codec round trips, with randomised parameters+different from the robustness set, never the real room tone) are only trained on.++## Held-out results+Every clip is scored by a model that never saw its group (nor augmented copies of its clips).++| scheme | clips | ROC-AUC | [email protected] | Brier | log loss |+|---|---|---|---|---|---|+| A: leave one real category / synth family out (headline) | 1484 | 0.9219 | 0.8598 | 0.1088 | 0.36 |+| A without augmented training copies | 1484 | 0.9314 | 0.8693 | 0.1023 | 0.3387 |+| A, CLAP head only | 1484 | 0.9257 | 0.8612 | 0.1069 | 0.3514 |+| A, signal-measurement head only | 1484 | 0.721 | 0.7123 | 0.2448 | 0.7195 |+| B: leave one synth generator type out | 1484 | 0.9576 | 0.9016 | 0.0764 | 0.2599 |+| C: retired score never trained on | 1484 | 0.8854 | 0.8389 | 0.1298 | 0.4509 |+| D: retired score and cinetic engine never trained on | 1224 | 0.916 | 0.8554 | 0.1147 | 0.401 |+| M: as A, but music held out by provider (mk / sb / ch / other) | 1484 | 0.92 | 0.8801 | 0.0984 | 0.3549 |++D is scored on real vs the retired score and the cinetic engine only (its sole negatives were the generic generators).++### The required check: real recordings rank above the retired synthesized score+The retired score (the synthesized soundtrack the user rejected) is held out of every training set (scheme C), or held+out together with the whole cinetic engine that made it (scheme D):++| check | ROC-AUC | [email protected] | mean p_real of the synth side |+|---|---|---|---|+| C: real vs retired | 0.9475 | 0.862 | 0.166 |+| C: real recordings only (no music/films) vs retired | 0.9418 | - | - |+| D: real vs retired | 0.9085 | 0.8389 | 0.185 |+| D: real vs cinetic | 0.921 | 0.8484 | 0.145 |+| D: real recordings only (no music/films) vs retired | 0.9031 | - | - |++### Verdict bands (scheme M held-out predictions)+`real or pro-produced` when p_real >= 0.77, `likely code-synthesized` when p_real <= 0.17, else `uncertain`.+Coverage 79% (the rest is uncertain); precision of the real verdict 0.9241, of the+synth verdict 0.9224; 6.7% of real clips are called synthesized and+5.5% of synthesized clips are called real. Other precision targets:++| precision target | lo | hi | coverage | real called synth | synth called real |+|---|---|---|---|---|---|+| 0.95 | 0.05 | 0.88 | 0.5195 | 0.0405 | 0.0277 |+| 0.925 | 0.17 | 0.77 | 0.7904 | 0.0666 | 0.0555 |+| 0.9 | 0.47 | 0.68 | 0.9407 | 0.11 | 0.0794 |++Calibration (scheme M, clean clips):++| p_real bin | clips | mean p_real | fraction real |+|---|---|---|---|+| 0.0-0.1 | 494 | 0.041 | 0.073 |+| 0.1-0.2 | 119 | 0.141 | 0.118 |+| 0.2-0.3 | 67 | 0.242 | 0.179 |+| 0.3-0.4 | 62 | 0.347 | 0.129 |+| 0.4-0.5 | 37 | 0.455 | 0.324 |+| 0.5-0.6 | 42 | 0.555 | 0.571 |+| 0.6-0.7 | 44 | 0.66 | 0.477 |+| 0.7-0.8 | 62 | 0.756 | 0.742 |+| 0.8-0.9 | 134 | 0.856 | 0.866 |+| 0.9-1.0 | 423 | 0.96 | 0.95 |++### Pro music+Scheme A holds out all music at once, so the model scoring it has never heard real music while its negatives include+synthesized scores: a pessimistic estimate (music mean p_real 0.595). Scheme M holds out one+provider at a time (the model has heard other providers' music): 119 windows of 52 files, mean+p_real 0.859, window [email protected] 0.9244, file [email protected] 0.9615; files (mean of windows)+called real: 44, called code-synthesized: 0. The price of weighting music+as four sources, on everything except music: scheme A AUC 0.9292 / Brier+0.1002 vs scheme M AUC 0.9123 / Brier 0.1022.++| provider fold | windows | mean p_real | [email protected] |+|---|---|---|---|+| music:ch | 30 | 0.83 | 0.9 |+| music:mk | 45 | 0.887 | 0.9333 |+| music:other | 8 | 0.77 | 0.75 |+| music:sb | 36 | 0.867 | 0.9722 |++### Per group (A: the whole group held out; M: the same, music held out by provider)++| group | label | clips | A mean p_real | A [email protected] | A AUC vs the other class | M mean p_real | M [email protected] |+|---|---|---|---|---|---|---|---|+| ambience | real/pro | 77 | 0.78 | 0.974 | 0.9542 | 0.868 | 0.987 |+| cinetic | synth | 318 | 0.081 | 0.9686 | 0.9753 | 0.101 | 0.978 |+| film | real/pro | 72 | 0.75 | 0.9722 | 0.9449 | 0.886 | 0.9722 |+| foley | real/pro | 73 | 0.869 | 0.9726 | 0.9721 | 0.905 | 0.9726 |+| garden | real/pro | 78 | 0.835 | 0.9487 | 0.9572 | 0.857 | 0.9103 |+| generic | synth | 260 | 0.329 | 0.6731 | 0.8355 | 0.322 | 0.6846 |+| keyboard | real/pro | 77 | 0.969 | 1.0 | 0.9925 | 0.963 | 1.0 |+| magic | real/pro | 68 | 0.451 | 0.4853 | 0.753 | 0.405 | 0.4559 |+| motion | real/pro | 66 | 0.649 | 0.6667 | 0.8794 | 0.622 | 0.6667 |+| music | real/pro | 119 | 0.595 | 0.6891 | 0.8867 | 0.859 | 0.9244 |+| retired | synth | 215 | 0.166 | 0.9674 | 0.9474 | 0.157 | 0.9674 |+| toy | real/pro | 61 | 0.838 | 0.9672 | 0.9627 | 0.878 | 0.9672 |++### Synthesized generator types (A: the whole family held out; B: only that type held out)++| type | clips | A mean p_real | A acc | B mean p_real | B acc |+|---|---|---|---|---|---|+| cinetic:bass | 10 | 0.025 | 1.0 | 0.021 | 1.0 |+| cinetic:blip | 10 | 0.036 | 1.0 | 0.035 | 1.0 |+| cinetic:clap | 10 | 0.016 | 1.0 | 0.013 | 1.0 |+| cinetic:crisp_tick | 10 | 0.045 | 1.0 | 0.015 | 1.0 |+| cinetic:drone | 10 | 0.108 | 1.0 | 0.144 | 1.0 |+| cinetic:fm_bell | 10 | 0.067 | 1.0 | 0.118 | 1.0 |+| cinetic:hat | 10 | 0.141 | 1.0 | 0.234 | 0.9 |+| cinetic:key_click | 10 | 0.069 | 1.0 | 0.058 | 1.0 |+| cinetic:keys | 10 | 0.131 | 1.0 | 0.203 | 0.8 |+| cinetic:kick | 10 | 0.026 | 1.0 | 0.066 | 1.0 |+| cinetic:marker | 10 | 0.016 | 1.0 | 0.01 | 1.0 |+| cinetic:mix | 16 | 0.094 | 1.0 | 0.298 | 0.875 |+| cinetic:motif | 10 | 0.012 | 1.0 | 0.016 | 1.0 |+| cinetic:pluck | 10 | 0.007 | 1.0 | 0.012 | 1.0 |+| cinetic:riser | 10 | 0.033 | 1.0 | 0.037 | 1.0 |+| cinetic:score | 42 | 0.038 | 1.0 | 0.148 | 1.0 |+| cinetic:snap | 10 | 0.132 | 0.9 | 0.246 | 0.8 |+| cinetic:sub_boom | 10 | 0.04 | 1.0 | 0.083 | 1.0 |+| cinetic:suck | 10 | 0.176 | 1.0 | 0.184 | 1.0 |+| cinetic:supersaw | 10 | 0.004 | 1.0 | 0.004 | 1.0 |+| cinetic:swell | 10 | 0.023 | 1.0 | 0.027 | 1.0 |+| cinetic:thock | 10 | 0.092 | 0.9 | 0.078 | 0.9 |+| cinetic:tick | 10 | 0.155 | 0.9 | 0.162 | 0.8 |+| cinetic:tock | 10 | 0.271 | 0.8 | 0.188 | 1.0 |+| cinetic:typing | 10 | 0.022 | 1.0 | 0.035 | 1.0 |+| cinetic:ui_click | 10 | 0.165 | 0.8 | 0.167 | 0.8 |+| cinetic:whoosh | 10 | 0.363 | 0.7 | 0.502 | 0.5 |+| cinetic:word_blip | 10 | 0.097 | 1.0 | 0.081 | 1.0 |+| generic:beep | 14 | 0.299 | 0.7143 | 0.126 | 0.9286 |+| generic:chiptune | 14 | 0.19 | 0.9286 | 0.019 | 1.0 |+| generic:chord | 14 | 0.335 | 0.7143 | 0.177 | 0.9286 |+| generic:click | 14 | 0.289 | 0.7143 | 0.2 | 1.0 |+| generic:drums | 14 | 0.009 | 1.0 | 0.027 | 1.0 |+| generic:dtmf | 14 | 0.713 | 0.0714 | 0.384 | 1.0 |+| generic:fm | 14 | 0.276 | 0.7857 | 0.155 | 0.8571 |+| generic:impact | 14 | 0.178 | 0.8571 | 0.19 | 0.8571 |+| generic:lfsr | 14 | 0.354 | 0.7143 | 0.079 | 1.0 |+| generic:mix | 36 | 0.609 | 0.3611 | 0.588 | 0.3889 |+| generic:pad | 14 | 0.3 | 0.7857 | 0.132 | 0.9286 |+| generic:pluck | 14 | 0.087 | 0.9286 | 0.077 | 0.9286 |+| generic:rain | 14 | 0.664 | 0.2143 | 0.61 | 0.2143 |+| generic:riser | 14 | 0.043 | 1.0 | 0.039 | 1.0 |+| generic:siren | 14 | 0.67 | 0.1429 | 0.177 | 0.8571 |+| generic:whoosh | 14 | 0.103 | 1.0 | 0.082 | 1.0 |+| generic:wind | 14 | 0.035 | 1.0 | 0.05 | 1.0 |+| retired:ambience | 15 | 0.224 | 0.9333 | 0.216 | 0.8667 |+| retired:bed | 15 | 0.285 | 0.9333 | 0.256 | 0.8667 |+| retired:melody | 12 | 0.151 | 1.0 | 0.1 | 1.0 |+| retired:melody-event | 25 | 0.174 | 0.96 | 0.147 | 0.92 |+| retired:stem_arp | 12 | 0.045 | 1.0 | 0.011 | 1.0 |+| retired:stem_bass | 13 | 0.059 | 1.0 | 0.027 | 1.0 |+| retired:stem_bed | 15 | 0.292 | 0.9333 | 0.265 | 0.8 |+| retired:stem_bells | 2 | 0.096 | 1.0 | 0.049 | 1.0 |+| retired:stem_bells-event | 1 | 0.028 | 1.0 | 0.004 | 1.0 |+| retired:stem_drums | 11 | 0.097 | 1.0 | 0.056 | 1.0 |+| retired:stem_drums-event | 25 | 0.089 | 1.0 | 0.084 | 1.0 |+| retired:stem_pad | 14 | 0.045 | 1.0 | 0.014 | 1.0 |+| retired:stem_reverb | 15 | 0.239 | 1.0 | 0.3 | 0.7333 |+| retired:stem_sfx | 15 | 0.241 | 0.9333 | 0.339 | 0.8 |+| retired:stem_sfx-event | 25 | 0.196 | 0.92 | 0.221 | 0.84 |++### File level (long files: the mean of their window p_real)+Sources with >= 2 windows (films, music, ambience, retired stems, full synthesized mixes): 119 files+(23 synthesized). Scheme A: AUC 0.9022, [email protected] 0.8235. Scheme M (the final model's+grouping): AUC 0.9538, [email protected] 0.9244. File verdict thresholds (lo: 92.5% precision on held-out long files (scheme M); hi: max(that, the clip hi)): lo 0.4 /+hi 0.77; with them 0.8655 of the files get a firm verdict, real files called+synthesized 0.0, synthesized files called real 0.0435.+Only 23 synthesized long files from three code bases: weak calibration. Scheme M synthesized files,+lowest to highest: stems/bass.wav 0.053, stems/arp.wav 0.069, stems/pad.wav 0.072, cinetic_score_2/soundtrack.wav 0.081, stems/bells.wav 0.086, stems/drums.wav 0.089, cinetic_score_1/soundtrack.wav 0.105, cinetic_score_0/soundtrack.wav 0.106, cinetic_score_4/soundtrack.wav 0.129, cinetic_score_5/soundtrack.wav 0.13, audio/melody.wav 0.131, audio/ambience.wav 0.191, cinetic_score_3/soundtrack.wav 0.197, stems/sfx.wav 0.207, _retired_synth/bed.wav 0.254, stems/bed.wav 0.256, stems/reverb.wav 0.316, mixes/generic_mix_4.wav 0.513, mixes/generic_mix_0.wav 0.561, mixes/generic_mix_1.wav 0.614, mixes/generic_mix_2.wav 0.662, mixes/generic_mix_3.wav 0.769, mixes/generic_mix_5.wav 0.953.+Scheme M real files under 0.6: music/ch-59068_sakuragirl-yay.mp3 0.401, music/mk-1142_gummies.mp3 0.435, ambience/fs-639459_summer-meadow-wind.ogg 0.499.++### Robustness (perturbed copies, scored by the fold model that never saw their group)+noise70 = pink noise at -70 dBFS RMS added to synthesized clips; room70 = a real room tone (quietest 20 s of a rain+recording) at -70 dBFS; gated = real clips gated to exact digital silence below -45 dB re peak; short = 0.15 s around+the peak; mp3 = 128 kbps round trip. The final model is trained with augmentation (A, M); A_noaug shows the trade-off.++| perturbation | label | clips | acc perturbed (A) | acc unperturbed (A) | acc perturbed (A_noaug) | acc perturbed (M) |+|---|---|---|---|---|---|---|+| noise70 | synth | 115 | 0.8 | 0.9304 | 0.687 | 0.8348 |+| room70 | synth | 115 | 0.7391 | 0.8609 | 0.7217 | 0.7739 |+| gated | real/pro | 106 | 0.7453 | 0.8113 | 0.7264 | 0.7547 |+| short | synth | 115 | 0.8957 | 0.9043 | 0.9565 | 0.9043 |+| short | real/pro | 107 | 0.7383 | 0.8972 | 0.7196 | 0.729 |+| mp3 | synth | 115 | 0.8609 | 0.887 | 0.8696 | 0.8957 |+| mp3 | real/pro | 107 | 0.8505 | 0.8505 | 0.8411 | 0.8505 |++Level (final model, so in-sample clips; it measures sensitivity, not accuracy): the same clips at their own level, 20 dB quieter, normalised to -25 dBFS RMS, after the inference lift (sources under -30 dBFS RMS are raised to -30 dBFS RMS, peak <= -1 dBFS), and moved to -56 dBFS RMS without / with that lift (the first build only lifted sources under -60 dBFS RMS, so -56 dBFS input was scored as it was).++| clips | n | median RMS dBFS | lifted at inference | mean p_real own / -20dB / norm-25 / lifted / at-56 / at-56_lifted | mean abs change -20dB / norm-25 / lifted / at-56 / at-56_lifted | acc own / -20dB / norm-25 / lifted / at-56 / at-56_lifted |+|---|---|---|---|---|---|---|+| real | 34 | -30.9 | 18 | 0.887 / 0.867 / 0.86 / 0.876 / 0.864 / 0.863 | 0.046 / 0.033 / 0.012 / 0.063 / 0.032 | 0.941 / 0.941 / 0.912 / 0.912 / 0.941 / 0.912 |+| cinetic | 31 | -24.3 | 8 | 0.062 / 0.08 / 0.068 / 0.062 / 0.082 / 0.072 | 0.037 / 0.012 / 0.001 / 0.059 / 0.017 | 1.0 / 0.968 / 1.0 / 1.0 / 0.968 / 1.0 |+| generic | 28 | -24.6 | 7 | 0.042 / 0.05 / 0.043 / 0.042 / 0.064 / 0.042 | 0.015 / 0.006 / 0.0 / 0.03 / 0.007 | 1.0 / 1.0 / 1.0 / 1.0 / 1.0 / 1.0 |++### Baselines (no training)+- CLAP zero-shot (log-sum-exp over 6 'real' prompts minus 6 'synth' prompts): AUC 0.8579,+ [email protected] 0.7487 on all clean clips.+- Single signal measurements (AUC, physical sign): silence_frac 0.5435, floor_db 0.4436, spec_floor_db 0.7546, log_flatness 0.7366, peakiness_db 0.7232, line_share 0.6516, bandwidth_khz 0.6971, quiet_flat 0.7859, tonal_frames 0.599, pitch_wobble 0.6744, env_resid_db 0.594, texture_kurt 0.5601, crest_db 0.7385.+- AST, on a stratified subset of 244 clean clips (AST costs ~5x CLAP here; the subset predates+ the full synthesized mixes): max over the synth classes (Synthesizer, Sine wave, Beep, bleep, Electronic tuner, Chirp tone, Dial tone, Pink noise, White noise, Static, Buzz, Hum, Theremin, Video game music, Ringtone, Sound effect)+ AUC 0.7555; single classes (high = real):+ Synthesizer 0.8465, Sine wave 0.8253, Beep, bleep 0.7498, Electronic music 0.7551, Music 0.6642.+ On the same subset: scheme A 0.951, CLAP zero-shot 0.8778,+ clap+signal heads refit on the subset 0.9566, plus an AST head 0.9533,+ AST head alone 0.7618. An AST head adds nothing, so AST is evidence only (`--ast`).++## What carries the decision+- Fusion weights (fitted on inner held-out head outputs): {'clap': 1.212409, 'signal': 0.0}. The CLAP head carries+ the decision; the signal head (0.721 AUC alone vs 0.9257 for CLAP alone) gets weight+ 0.0, so its measurements are reported as evidence (each with the share of training clips past that+ threshold that were of that side), not votes.+- The CLAP head's weight direction is closest to these vocabulary texts. Toward real:+ a high quality studio recording, a metal scrape, an old typewriter, a match being struck, a knife chopping on a board, a small object dropping on a table, pizzicato strings, a can opening. Toward synthesized:+ a square wave chiptune blip, a simple electronic synthesizer tone made by code, a warm analog synth pad, a cheap synthesized beep, a computer-generated electronic tone, filtered white noise, an 8-bit video game sound effect, an 8-bit sound effect.+- Signal-head coefficients (standardised, sign-constrained to their physics): {'silence_frac': -0.106636, 'floor_db': 0.317801, 'spec_floor_db': 0.0, 'log_flatness': 0.0, 'peakiness_db': 0.0, 'line_share': 0.0, 'bandwidth_khz': 0.0, 'quiet_flat': 0.898494, 'tonal_frames': 0.0, 'pitch_wobble': 0.199091, 'env_resid_db': 0.007039, 'texture_kurt': 0.0, 'crest_db': 0.755868}.+- Exact digital silence is weak evidence: over half of the clips of both classes contain some (edited and gated library+ recordings too), so it is never a verdict reason, only context quoted with its training base rates.+- One rule overrides the model at inference (earshot.realness.pure_tone): a pure, dead-steady tone (one spectral line in+ every frame, crest <= 4.5 dB, no pitch drift) gets p_real <= 0.03, because bare test tones lie outside the training+ data. Checked by `eval/realness/tone_guard.py` (`tone_guard.json`): a 30 s 440 Hz sine gets 0.48 from the model+ alone and 0.03 with the rule; the rule fires on none of 4741 audible clips of the 345 real recordings.++## Application files (whole soundtracks, 4 s windows)+| file | held-out model: p_real (mean) / verdict | windows < 0.5 / <= lo / all (held-out) | final model (in-sample): p_real / verdict |+|---|---|---|---|+| clawd_film_real_soundtrack | 0.91 / real or pro-produced | 0 / 0 / 15 | 0.942 / real or pro-produced |+| clawd_film_synth_soundtrack | 0.334 / likely code-synthesized | 13 / 1 / 15 | 0.327 / likely code-synthesized |+| clawd_soundtrack_wav | 0.901 / real or pro-produced | 0 / 0 / 15 | 0.933 / real or pro-produced |+| clawd_stem_pro_sfx | 0.646 / uncertain | 5 / 0 / 15 | 0.787 / real or pro-produced |+| clawd_stem_pro_amb | 0.84 / real or pro-produced | 0 / 0 / 15 | 0.882 / real or pro-produced |+| clawd_stem_pro_music | 0.811 / real or pro-produced | 0 / 0 / 15 | 0.915 / real or pro-produced |++The held-out model (1273 training clips) never saw accepted Clawd assets (incl. assets/music) and the retired score: the real soundtrack is built from+83 of those accepted recordings, and the synthesized soundtrack is the retired score, so this column is out of sample+on both sides. The final model trained on both (in-sample, optimistic). Verdicts use the file thresholds for files+over 6 s.++### Professionally made synthetic sounds (Windows UI / alert sounds, never trained on)+70 top-level wav files of C:/Windows/Media (chimes, alarms, ringtones, notification tones; made by sound+designers, mostly synthesized): 32 called 'likely code-synthesized', 32+'uncertain', 6 'real or pro-produced' (median p_real 0.238). Lowest:+Windows Hardware Insert.wav 0.006, Windows Hardware Remove.wav 0.007, Alarm10.wav 0.009, Windows Notify System Generic.wav 0.009, Windows Background.wav 0.013. Highest: Ring08.wav 0.861, Speech Off.wav 0.885, ding.wav 0.893, Windows Recycle.wav 0.898, Windows Default.wav 0.946.+The detector hears synthesis, not provenance: a clean, simple synthesized tone reads as code-made even when a+professional made it and it is licensed. An agent using a licensed UI-sound pack must not treat that verdict as proof.++### Layer test: a code-synthesized layer inside a real mix+369 clips: 184 real 4 s beds (ambience, films, music, foley ...; mean p_real alone+0.804, 0.114 of them under 0.5 alone) with one code-synthesized layer each (a+cinetic / generic / retired clip: one-shots 1-4 times, 4 s windows throughout), at a layer-to-bed RMS of -12 / -6 / 0 /++6 dB (layer RMS over the span it plays). Every clip is scored by a model that never saw its bed's group nor its+layer's family (`--stage layers`). 'base' = the shipped recipe; 'train' = also trained on the layered clips of the+other groups.++| layer level | clips | base: mean p_real | base: < 0.5 | base: <= lo | base: >= hi (called real) | train: mean p_real | train: < 0.5 | train: <= lo |+|---|---|---|---|---|---|---|---|---|+| -12 dB | 93 | 0.673 | 0.269 | 0.172 | 0.538 | 0.591 | 0.344 | 0.204 |+| -6 dB | 91 | 0.621 | 0.33 | 0.121 | 0.429 | 0.536 | 0.418 | 0.132 |+| +0 dB | 94 | 0.633 | 0.298 | 0.149 | 0.479 | 0.524 | 0.457 | 0.17 |+| +6 dB | 91 | 0.622 | 0.286 | 0.143 | 0.396 | 0.512 | 0.484 | 0.154 |++Most layers are missed: the windows lean synthesized for only about a third of the clips, almost independent of the+layer level, and many are still called real. Per bed group (base, -6..+6 dB, share < 0.5): {'ambience': 0.167, 'film': 0.167, 'foley': 0.214, 'garden': 0.133, 'keyboard': 0.133, 'magic': 0.833, 'motion': 0.682, 'music': 0.222, 'toy': 0.227}+(magic and motion beds already lean synthesized on their own). Per layer family: {'cinetic': 0.375, 'generic': 0.221, 'retired': 0.295}.+Training on layered clips (scheme M_layers: M plus the layered clips of the other groups) raised the held-out clip AUC+to 0.9281 (acc 0.8935, Brier 0.0926; M: 0.92 / 0.8801 /+0.0984) and the layer detection above, but compressed the synthesized scores: at the same 92.5% precision+target its thresholds are lo 0.05 / hi 0.68 with 0.5081 firm-verdict+coverage (M: 0.7904). It was not adopted (decided after seeing these held-out numbers): judging source files+firmly is the main use, and even trained, most layers stay unflagged. A dedicated layer head (CLAP logistic, bed vs bed++ layer, same pair folds; a scratch experiment, not in results.json) reached AUC 0.82-0.84 with 40-46% of the layers+at -6..+6 dB found at 5% false alarms on beds, but 11-17% false alarms on film and music beds: not shipped either.+So `earshot realness` says plainly that a clean window does not rule out a code-made layer under a real mix.+++## Limits: what fools it+- Synthesized types it misses most (scheme A acc < 0.75, lowest first; acc / mean p_real): generic:dtmf (0.0714 / 0.713), generic:siren (0.1429 / 0.67), generic:rain (0.2143 / 0.664), generic:mix (0.3611 / 0.609), cinetic:whoosh (0.7 / 0.363), generic:beep (0.7143 / 0.299), generic:chord (0.7143 / 0.335), generic:click (0.7143 / 0.289), generic:lfsr (0.7143 / 0.354).+ Code-made noise textures that imitate nature (rain, wind beds) and telephone-like tones (DTMF, sirens) sound like+ recordings of real things to CLAP, and a generic code-made mix built on such beds can score real.+- Real groups it doubts (scheme A acc < 0.8; acc / mean p_real): magic (0.4853 / 0.451), motion (0.6667 / 0.649), music (0.6891 / 0.595).+ Library "magic" and "motion" sounds are themselves designed (often synthesized) by sound designers, and real+ electronic music sits near synthesized scores: read 'uncertain' there as "sounds designed", not "fake".+- Finished mixes are harder than sources, and a code-made layer under a louder real mix is usually missed (layer+ test). Judge the source files and stems, not only the final mix.+- Professionally made synthetic tones (UI sounds, chimes, alarms) are called code-synthesized: it hears synthesis, not+ provenance (Windows UI sounds above).+- A -70 dBFS noise floor added to synthesized audio lowers accuracy (robustness table): the detector mostly hears what+ a sound is, not microphone physics. Very short clips (< 0.2 s) of real sounds are harder.+- Training data: one sound library (Freesound / Mixkit picks for one film), 7 launch films, ~120 music windows; the+ negatives come from three code bases (cinetic, the retired score made with it, generic numpy generators). Synthesis+ from other tools (pro soft synths, granular, sample-based) was never seen: pro-produced synth music is meant to score+ real, and amateur code using techniques outside these three code bases may also score real.++## Reproduce+```+.venv/Scripts/python.exe eval/realness/build_data.py # clips -> out/realness/+.venv/Scripts/python.exe eval/realness/build_data.py --only layers # layer-test clips+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/extract.py --ast 240 # repeat until "done"+OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage cv # until "cv done"+OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage layers # twice+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage apps+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/train.py+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage level # needs the final model+.venv/Scripts/python.exe eval/realness/run_eval.py --stage report # results.json + RESULTS.md+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/train.py # embeds the summary in the model+```diff --git a/eval/realness/build_data.py b/eval/realness/build_data.pynew file mode 100644index 0000000..a7fe822--- /dev/null+++ b/eval/realness/build_data.py@@ -0,0 +1,617 @@+"""Build the realness dataset: clips of real / professionally produced audio and of code-synthesized audio.++ .venv/Scripts/python.exe eval/realness/build_data.py [--only real,rejected,films,retired,cinetic,generic,mixes,robust,augment]+ .venv/Scripts/python.exe eval/realness/build_data.py --only layers # real beds + one code-synthesized layer++Writes mono float32 48 kHz clips to out/realness/clips/<family>/ (scratch, gitignored, never committed) and the+manifest out/realness/manifest.json: one entry per clip with its label (1 = real or pro-produced, 0 = synthesized),+family, CV group, type and provenance (source file + seconds). Deterministic: same inputs -> same clips.++Clip policy (the same for both classes, and the same as earshot.realness at inference): a source up to 6 s is one+clip; a longer source is cut into 4 s windows (evenly spaced picks for long real recordings, every window for the+retired score and the films); one-shot events are cut from long one-shot recordings with the same onset cutter for+real and synthesized sources. Near-silent windows (RMS < -60 dBFS) are skipped. Full synthesized mixes (part 'mixes'):+6 random cinetic soundtracks rendered by its score.py and 6 generic code-made mixes (generic_synth voices, Schroeder+reverb, limiter), cut into 4 s windows. The current robust / augment parts were built before the mixes (no perturbed+copies of mixes); rebuilding them would add some and change the robustness set. Part 'layers' (build_layers): real+4 s beds with one code-synthesized layer mixed in at -12 / -6 / 0 / +6 dB, label 0 (the layer-in-a-mix test).+"""+import argparse+import json+import os+import subprocess+import sys++import numpy as np+import soundfile as sf++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+sys.path.insert(0, HERE)+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402+from earshot import audio # noqa: E402++SR = audio.SR+OUT = os.path.join(os.path.dirname(os.path.dirname(HERE)), 'out', 'realness')+CLIPS = os.path.join(OUT, 'clips')+ASSETS = EP.ASSETS # local data, see eval/README.md+CINETIC = EP.SYNTH+FILMS = {rid: EP.ref(rid) for rid in EP.REF_IDS} # the 7 reference launch films+RETIRED = {'bed': f'{EP.FILM}/audio/_retired_synth/bed.wav',+ 'melody': f'{EP.FILM}/out/audio/melody.wav', 'ambience': f'{EP.FILM}/out/audio/ambience.wav',+ **{f'stem_{s}': f'{EP.FILM}/out/audio/stems/{s}.wav'+ for s in ('arp', 'bass', 'bed', 'bells', 'drums', 'pad', 'reverb', 'sfx')}}+CATS = ('ambience', 'foley', 'garden', 'keyboard', 'magic', 'motion', 'music', 'toy')+LONG_S, WIN_S = 6.0, 4.0+++def rms_db(x):+ return audio.db(np.sqrt(np.mean(np.square(x, dtype=np.float64)) + 1e-30))+++def cached(family, name):+ p = os.path.join(CLIPS, family, name + '.wav')+ return (os.path.relpath(p, OUT).replace('\\', '/'), sf.info(p).frames / SR) if os.path.exists(p) else None+++def write(x, family, name):+ d = os.path.join(CLIPS, family)+ os.makedirs(d, exist_ok=True)+ p = os.path.join(d, name + '.wav')+ sf.write(p, np.asarray(x, np.float32), SR, subtype='FLOAT')+ return os.path.relpath(p, OUT).replace('\\', '/')+++def pick_windows(n, k, margin=0.0):+ """k evenly spaced 4 s window starts (seconds) inside [margin, dur - margin]"""+ dur = n / SR+ lo, hi = min(margin, max(0.0, dur - WIN_S) / 2), max(0.0, dur - WIN_S - min(margin, max(0.0, dur - WIN_S) / 2))+ if k <= 1:+ return [round((lo + hi) / 2, 3)]+ return [round(lo + (hi - lo) * i / (k - 1), 3) for i in range(k)]+++def events(x, k, max_len=1.0, min_gap=0.25):+ """up to k one-shot events: onset -> next onset (<= max_len), loudest first, chronological output"""+ from earshot.features import onsets+ ons = [o for o in onsets(x) if o * SR < len(x) - 0.03 * SR]+ cand = []+ for i, o in enumerate(ons):+ if i and o - ons[i - 1] < min_gap:+ continue+ a = max(0, int((o - 0.005) * SR))+ nxt = next((v for v in ons[i + 1:] if v - o >= min_gap), None)+ b = min(len(x), int((min(o + max_len, nxt - 0.01) if nxt else o + max_len) * SR))+ if b - a > 0.03 * SR and rms_db(x[a:b]) > -50:+ cand.append((float(np.abs(x[a:b]).max()), a, b))+ cand.sort(reverse=True)+ return sorted((a, b) for _, a, b in cand[:k])+++def clip_entry(cid, rel, label, family, group, typ, src, start, end):+ return {'id': cid, 'wav': rel, 'label': label, 'family': family, 'group': group, 'type': typ, 'src': src,+ 'start': round(start, 3), 'end': round(end, 3)}+++def build_real(kept=True):+ out = []+ for cat in CATS:+ files = audio.list_media(os.path.join(ASSETS, cat) if kept else os.path.join(ASSETS, '_rejected', cat))+ fam = ('pro-music' if cat == 'music' else 'real-asset') if kept else ('pro-music' if cat == 'music' else 'real-rejected')+ for f in files:+ x = audio.load(f, mono=True, cache=False)+ base = os.path.splitext(os.path.basename(f))[0][:40]+ tag = ('k_' if kept else 'r_') + base+ if len(x) <= LONG_S * SR:+ out.append(clip_entry(tag, write(x, fam, tag), 1, fam, cat, cat, f, 0, len(x) / SR))+ continue+ k = (4 if kept else 2) if cat == 'music' else 2 if cat == 'ambience' else 1+ for s in pick_windows(len(x), k, margin=5.0 if cat == 'music' else 0.5):+ seg = x[int(s * SR): int((s + WIN_S) * SR)]+ if rms_db(seg) > -60:+ cid = f'{tag}@{s:.1f}'+ out.append(clip_entry(cid, write(seg, fam, cid.replace('@', '_')), 1, fam, cat, cat, f, s, s + WIN_S))+ if cat not in ('music', 'ambience'):+ for a, b in events(x, 2):+ cid = f'{tag}#ev{a / SR:.2f}'+ out.append(clip_entry(cid, write(x[a:b], fam, cid.replace('#', '_')), 1, fam, cat, cat + '-event',+ f, a / SR, b / SR))+ return out+++def build_films():+ out = []+ for name, f in FILMS.items():+ x = audio.load(f, mono=True, cache=False)+ for i, s in enumerate(np.arange(0, len(x) / SR - WIN_S + 1e-6, WIN_S)):+ seg = x[int(s * SR): int((s + WIN_S) * SR)]+ if rms_db(seg) > -60:+ cid = f'film_{name}@{s:.0f}'+ out.append(clip_entry(cid, write(seg, 'pro-film', cid.replace('@', '_')), 1, 'pro-film', 'film', name, f,+ float(s), float(s + WIN_S)))+ return out+++def build_retired():+ out = []+ for name, f in RETIRED.items():+ x = audio.load(f, mono=True, cache=False)+ for s in np.arange(0, len(x) / SR - WIN_S + 1e-6, WIN_S):+ seg = x[int(s * SR): int((s + WIN_S) * SR)]+ if rms_db(seg) > -60:+ cid = f'retired_{name}@{s:.0f}'+ out.append(clip_entry(cid, write(seg, 'retired', cid.replace('@', '_')), 0, 'retired', 'retired', name, f,+ float(s), float(s + WIN_S)))+ if name in ('stem_sfx', 'stem_drums', 'melody', 'stem_bells'):+ for a, b in events(x, 25):+ cid = f'retired_{name}#ev{a / SR:.2f}'+ out.append(clip_entry(cid, write(x[a:b], 'retired', cid.replace('#', '_')), 0, 'retired', 'retired',+ name + '-event', f, a / SR, b / SR))+ return out+++def cinetic_voice(S, kind, r, seed):+ note = lambda lo, hi: S.mtof(int(r.integers(lo, hi))) # noqa: E731+ u = r.uniform+ if kind == 'kick':+ return S.kick(note(28, 40), u(0.4, 1.2), u(0.6, 1.4), seed)+ if kind == 'sub_boom':+ f1 = note(26, 40)+ return S.sub_boom(f1 * u(1.5, 3), f1, u(0.05, 0.3), u(1.0, 3.0), seed, u(0.4, 1.2))+ if kind == 'clap':+ return S.clap(u(0.25, 0.6), seed)+ if kind == 'hat':+ return S.hat(u(0.05, 0.3), u(0.008, 0.08), u(5000, 9000), seed)+ if kind == 'tick':+ return S.tick(note(72, 90), u(0.05, 0.1), u(0.006, 0.015), u(0.6, 1.0), seed, r.random() < 0.4)+ if kind == 'tock':+ return S.tock(u(600, 1600), u(0.08, 0.2), u(0.015, 0.05), u(0.2, 0.6), seed)+ if kind == 'key_click':+ return S.key_click(int(r.integers(0, 60)), u(0.98, 1.02), None, str(r.choice(S.KEY_STYLES)),+ str(r.choice(['letter', 'letter', 'back', 'space', 'return'])), seed, u(0.6, 1.3),+ 0.09 if r.random() < 0.2 else None, second=0.03 if r.random() < 0.5 else None)+ if kind == 'thock':+ return S.thock(u(800, 1100), int(r.integers(0, 60)), str(r.choice(S.KEY_STYLES)), str(r.choice(['space', 'return'])), seed)+ if kind == 'ui_click':+ return S.ui_click(note(88, 100), 0.14, int(r.integers(0, 40)), 0.085 if r.random() < 0.6 else None,+ note(76, 86) if r.random() < 0.3 else None, seed)+ if kind == 'crisp_tick':+ return S.crisp_tick(seed, u(0.03, 0.08), note(90, 100) if r.random() < 0.3 else None)+ if kind == 'snap':+ return S.snap(note(28, 40), u(0.3, 0.8), u(0.5, 1.2), seed)+ if kind == 'blip':+ return S.blip(note(70, 92), r.random() < 0.5, u(0.08, 0.2))+ if kind == 'word_blip':+ return S.word_blip(note(64, 86), u(0.036, 0.06), u(1.0, 4.0), seed)+ if kind == 'fm_bell':+ return S.fm_bell(note(72, 92), u(1.5, 3.0), u(1.0, 3.5), float(r.choice([3.5, 2.0, 1.4, 2.76])), u(0.4, 1.2))+ if kind == 'marker':+ return S.marker(u(0.15, 0.5), seed)+ if kind == 'whoosh':+ return S.whoosh(u(0.4, 1.4), u(80, 250), u(500, 1500), u(150, 400), u(0.3, 0.7), u(-1, 1), u(0.01, 0.05), seed)+ if kind == 'riser':+ return S.riser(u(1.0, 3.0), u(150, 400), u(4000, 9000), note(48, 72) if r.random() < 0.6 else None, seed)+ if kind == 'suck':+ return S.suck(u(0.2, 0.5), seed)+ if kind == 'swell':+ root = int(r.integers(55, 67))+ return S.swell([S.mtof(root + i) for i in (0, 4, 7)], u(0.6, 2.0), seed)+ if kind == 'supersaw':+ return S.supersaw(note(48, 66), u(1.0, 3.0), int(r.integers(3, 8)), u(0.05, 0.2), u(900, 3500), u(0.05, 0.4), u(0.3, 1.0), seed)+ if kind == 'pluck':+ return S.pluck(note(55, 80), u(0.3, 1.0), u(2500, 6000), u(300, 800), u(0.04, 0.15), u(0.1, 0.4))+ if kind == 'bass':+ return S.bass(note(28, 45), u(0.4, 1.5), u(1.0, 2.0), u(0.1, 0.35))+ if kind == 'drone':+ f = note(33, 48)+ return S.drone([f, f * 1.5], u(2.0, 4.0), u(300, 900), u(0.5, 2.0))+ if kind == 'keys':+ return S.keys(note(55, 76), u(1.0, 2.5), u(0.8, 2.2), u(0.4, 1.2))+ if kind == 'typing':+ bus = S.Bus(seconds=4.0)+ t, key = 0.1, int(r.integers(0, 30))+ style = str(r.choice(S.KEY_STYLES))+ while t < 3.6:+ bus.place(S.key_click(key + int(r.integers(0, 26)), u(0.985, 1.015), None, style,+ 'space' if r.random() < 0.15 else 'letter', int(t * 1000), u(0.6, 1.2)), t)+ t += u(0.06, 0.2)+ return bus.x+ if kind == 'motif':+ voice = str(r.choice(['pluck', 'keys', 'fm_bell', 'blip', 'tick']))+ root = int(r.integers(60, 76))+ bus = S.Bus(seconds=4.0)+ for k in range(int(r.integers(3, 7))):+ f = S.mtof(root + int(r.choice([0, 2, 4, 7, 9, 12])))+ v = {'pluck': lambda: S.pluck(f), 'keys': lambda: S.keys(f, 1.2), 'fm_bell': lambda: S.fm_bell(f, 2.0),+ 'blip': lambda: S.blip(f), 'tick': lambda: S.tick(f)}[voice]()+ bus.place(v[: S.ns(3.0)] * 0.5, 0.1 + 0.45 * k)+ return S.pingpong(bus.x, 0.3, 0.35, 0.3) if r.random() < 0.5 else bus.x+ if kind == 'mix':+ return cinetic_mix(S, r, seed)+ raise ValueError(kind)+++def cinetic_mix(S, r, seed):+ """four bars of a cinetic-style arrangement (drums, bass, pad, arp, rooms): like score.py output, not the film's"""+ bpm = r.uniform(96, 132)+ beat = 60 / bpm+ bus = S.Bus(seconds=4.0)+ root = int(r.integers(36, 48))+ third = 3 if r.random() < 0.5 else 4+ layers = {k: r.random() < p for k, p in (('drums', 0.8), ('bass', 0.8), ('pad', 0.7), ('arp', 0.7), ('bells', 0.3))}+ if not any(layers.values()):+ layers['pad'] = True+ for b in range(int(4.0 / beat) + 1):+ t = b * beat+ if layers['drums']:+ bus.place(S.kick(S.mtof(root - 12)) * 0.8, t)+ bus.place(S.hat(0.08, 0.012, seed=seed + b) * 0.3, t + beat / 2)+ if b % 2:+ bus.place(S.clap(seed=seed + b) * 0.4, t)+ if layers['bass'] and b % 2 == 0:+ bus.place(S.bass(S.mtof(root), beat * 1.8) * 0.5, t)+ if layers['arp']:+ for k in range(2):+ bus.place(S.pluck(S.mtof(root + 24 + int(r.choice([0, third, 7, 12])))) * 0.25, t + k * beat / 2)+ if layers['pad']:+ for i in (0, third, 7):+ bus.place(S.supersaw(S.mtof(root + 12 + i), 4.0, seed=seed + i) * 0.2, 0.0)+ if layers['bells']:+ bus.place(S.fm_bell(S.mtof(root + 36), 2.0) * 0.3, r.uniform(0, 2))+ return bus.x + S.reverb(bus.x, 'hall') * r.uniform(0.15, 0.4)+++CINETIC_TYPES = ('kick', 'sub_boom', 'clap', 'hat', 'tick', 'tock', 'key_click', 'thock', 'ui_click', 'crisp_tick',+ 'snap', 'blip', 'word_blip', 'fm_bell', 'marker', 'whoosh', 'riser', 'suck', 'swell', 'supersaw',+ 'pluck', 'bass', 'drone', 'keys', 'typing', 'motif', 'mix')+++def build_cinetic(per_type=10):+ sys.path.insert(0, CINETIC)+ import synth as S+ out = []+ for kind in CINETIC_TYPES:+ for k in range(per_type + (6 if kind == 'mix' else 0)):+ seed = 100 + k+ cid = f'cinetic_{kind}_{k}'+ hit = cached('cinetic', cid)+ if hit:+ out.append(clip_entry(cid, hit[0], 0, 'cinetic', 'cinetic', kind, 'synth.py', 0, hit[1]))+ continue+ r = np.random.default_rng(10007 * (CINETIC_TYPES.index(kind) + 1) + k)+ x = np.asarray(cinetic_voice(S, kind, r, seed), np.float64)+ if kind not in ('mix', 'typing', 'motif') and r.random() < 0.35:+ x = S.stereo(x) + S.reverb(S.filt(S.stereo(x), S.hp(250)), str(r.choice(['hall', 'room', 'small']))) * r.uniform(0.15, 0.5)+ x = audio.to_mono(x) if x.ndim == 2 else x+ x = x[: int(LONG_S * SR)]+ x = x / (np.abs(x).max() + 1e-12) * 10 ** (r.uniform(-20, -1) / 20)+ out.append(clip_entry(cid, write(x, 'cinetic', cid), 0, 'cinetic', 'cinetic', kind, 'synth.py', 0, len(x) / SR))+ return out+++def build_generic(per_type=14):+ import generic_synth as G+ out = []+ for kind in G.TYPES:+ for k in range(per_type):+ cid = f'generic_{kind}_{k}'+ hit = cached('generic', cid)+ if hit:+ out.append(clip_entry(cid, hit[0], 0, 'generic', 'generic', kind, 'generic_synth.py', 0, hit[1]))+ continue+ x = G.render(kind, k)+ r = np.random.default_rng(5003 * k + 17)+ x = audio.to_mono(x)[: int(LONG_S * SR)]+ x = x / (np.abs(x).max() + 1e-12) * 10 ** (r.uniform(-20, -1) / 20)+ out.append(clip_entry(cid, write(x, 'generic', cid), 0, 'generic', 'generic', kind, 'generic_synth.py', 0,+ len(x) / SR))+ return out+++CHORDS = ['', 'm', 'maj7', 'm7', '7', 'sus2', 'sus4', 'add9', 'm9', 'maj9', '6']+NOTES = ['C', 'Db', 'D', 'Eb', 'E', 'F', 'Gb', 'G', 'Ab', 'A', 'Bb', 'B']+EVENT_KINDS = ['tick', 'tock', 'hit', 'land', 'pop', 'whoosh', 'suck', 'riser', 'drop', 'key', 'click', 'snap', 'swell', 'bell']+++def cinetic_score(k, seconds=28.0):+ """render a random cinetic soundtrack (score.py: music from a random score.json, effects from random cue events,+ mastered) - a full synthesized film mix like the retired score, but with other cues, keys, tempi and arrangements"""+ r = np.random.default_rng(777 + k)+ d = os.path.join(OUT, 'mixes', f'cinetic_score_{k}')+ os.makedirs(d, exist_ok=True)+ wav = os.path.join(d, 'soundtrack.wav')+ if os.path.exists(wav):+ return wav+ bpm, fps = float(r.choice([96, 100, 108, 110, 116, 120, 124, 128])), 60+ total = int(seconds * fps)+ nbars = int(np.ceil(seconds / (240.0 / bpm)))+ events, f = [], 30.0+ while f < total - 60:+ kind = str(r.choice(EVENT_KINDS, p=np.r_[[0.08, 0.05, 0.08, 0.08, 0.08, 0.1, 0.03, 0.04, 0.03, 0.25, 0.08, 0.05, 0.03, 0.02]]))+ n = int(r.integers(4, 30)) if kind == 'key' else 1+ for j in range(n):+ events.append({'f': round(f, 2), 'kind': kind, 'weight': round(float(r.uniform(0.2, 1.0)), 2), 'id': f'e{len(events)}',+ 'pan': round(float(r.uniform(-0.5, 0.5)), 2)})+ f += float(r.uniform(4, 9)) if kind == 'key' else 0+ f += float(r.uniform(15, 120))+ cues = {'fps': fps, 'bpm': bpm, 'total': total, 'cue': {'mid': total // 2}, 'events': events}+ tonic = str(r.choice(NOTES))+ mode = str(r.choice(['major', 'minor', 'dorian', 'mixolydian', 'lydian']))+ styles = ['intro', 'build', 'groove', 'drive', 'breakdown', 'resolve']+ secs, b = [], 1+ while b <= nbars:+ ln = int(r.integers(1, 4))+ secs.append({'bars': [b, min(nbars, b + ln - 1)], 'style': str(r.choice(styles)), 'level': round(float(r.uniform(0.7, 1.0)), 2)})+ b += ln+ score = {'key': f'{tonic} {mode}', 'bpm': bpm, 'sections': secs,+ 'chords': [NOTES[(NOTES.index(tonic) + int(r.choice([0, 2, 5, 7, 9]))) % 12] + str(r.choice(CHORDS)) for _ in range(nbars)],+ 'typing': str(r.choice(['auto', 'soft', 'mechanical', 'pitched', 'muted', 'word'])),+ 'drops': [{'at': '@mid', 'silence': '1/8', 'suck': 0.2}] if r.random() < 0.5 else [],+ 'master': {'lufs': float(r.choice([-16, -14, -12]))}}+ json.dump(cues, open(os.path.join(d, 'cues.json'), 'w'))+ json.dump(score, open(os.path.join(d, 'score.json'), 'w'))+ subprocess.run([sys.executable, os.path.join(CINETIC, 'score.py'), '--cues', os.path.join(d, 'cues.json'), '--score',+ os.path.join(d, 'score.json'), '--out', wav, '--quiet'], capture_output=True, cwd=d)+ return wav if os.path.exists(wav) else None+++def generic_mix(k, seconds=24.0):+ """a generic code-made soundtrack: generic_synth voices layered on a timeline, a Schroeder reverb send, a tanh+ limiter and loudness normalisation (no cinetic code)"""+ import generic_synth as G+ r = np.random.default_rng(999 + k)+ n = int(seconds * SR)+ bus = np.zeros(n + 6 * SR)+ beds = [t for t in ('pad', 'chord', 'wind', 'rain', 'drums', 'chiptune') if r.random() < 0.5] or ['pad']+ for t in beds:+ pos = 0+ while pos < n:+ x = audio.to_mono(G.render(t, int(r.integers(0, 10 ** 6)))).astype(np.float64)+ bus[pos: pos + len(x)] += x * r.uniform(0.2, 0.5)+ pos += max(len(x) - int(0.1 * SR), int(0.5 * SR))+ for _ in range(int(r.integers(6, 25))):+ t = str(r.choice(['beep', 'fm', 'pluck', 'whoosh', 'riser', 'impact', 'click', 'siren', 'lfsr', 'dtmf']))+ x = audio.to_mono(G.render(t, int(r.integers(0, 10 ** 6)))).astype(np.float64)+ i = int(r.integers(0, n))+ bus[i: i + len(x)] += x[: len(bus) - i] * r.uniform(0.2, 0.7)+ bus = bus[:n]+ if r.random() < 0.7:+ bus = G.schroeder(bus, r, mix=r.uniform(0.15, 0.35))[:n]+ bus = np.tanh(bus / (np.percentile(np.abs(bus), 99.5) + 1e-9) * r.uniform(0.8, 1.6))+ return bus / (np.abs(bus).max() + 1e-12) * 10 ** (-1 / 20)+++def build_mixes(n=6):+ out = []+ for fam, maker in (('cinetic', cinetic_score), ('generic', generic_mix)):+ for k in range(n):+ if fam == 'cinetic':+ path = maker(k)+ if not path:+ print(f'cinetic score {k}: render failed')+ continue+ x = audio.load(path, mono=True, cache=False)+ src = path+ else:+ cached_wav = os.path.join(OUT, 'mixes', f'generic_mix_{k}.wav')+ if not os.path.exists(cached_wav):+ os.makedirs(os.path.dirname(cached_wav), exist_ok=True)+ sf.write(cached_wav, maker(k).astype(np.float32), SR, subtype='FLOAT')+ x, _ = sf.read(cached_wav, dtype='float32')+ src = cached_wav+ for s in np.arange(0, len(x) / SR - WIN_S + 1e-6, WIN_S):+ seg = x[int(s * SR): int((s + WIN_S) * SR)]+ if rms_db(seg) > -60:+ cid = f'{fam}_mix{k}@{s:.0f}'+ out.append(clip_entry(cid, write(seg, fam, cid.replace('@', '_')), 0, fam, fam,+ 'score' if fam == 'cinetic' else 'mix', src, float(s), float(s + WIN_S)))+ return out+++def _mp3(x):+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-f', 'f32le', '-ar', str(SR), '-ac', '1', '-i', '-',+ '-c:a', 'libmp3lame', '-b:a', '128k', '-f', 'mp3', '-'], input=np.asarray(x, np.float32).tobytes(),+ capture_output=True, check=True)+ q = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-f', 'mp3', '-i', '-', '-f', 'f32le', '-ac', '1', '-ar',+ str(SR), '-'], input=p.stdout, capture_output=True, check=True)+ return np.frombuffer(q.stdout, np.float32)+++def _pinkish(n, rng):+ X = np.fft.rfft(rng.standard_normal(n))+ f = np.fft.rfftfreq(n, 1 / SR)+ X[1:] /= np.sqrt(f[1:] / 100.0)+ X[0] = 0+ y = np.fft.irfft(X, n)+ return y / (np.std(y) + 1e-12)+++def perturb(kind, x, rng, room=None):+ if kind == 'noise70':+ return x + _pinkish(len(x), rng) * 10 ** (-70 / 20)+ if kind == 'room70':+ s = int(rng.integers(0, len(room) - len(x))) if len(room) > len(x) else 0+ r = np.resize(room[s:], len(x))+ return x + r / (np.sqrt(np.mean(r ** 2)) + 1e-12) * 10 ** (-70 / 20)+ if kind == 'gated':+ k = int(0.01 * SR)+ e = np.repeat(np.sqrt(np.convolve(x ** 2, np.ones(k) / k, 'same')[::k] + 1e-20), k)[: len(x)]+ y = x.copy()+ y[: len(e)][e < np.abs(x).max() * 10 ** (-45 / 20)] = 0.0+ return y+ if kind == 'short':+ i = int(np.argmax(np.abs(x)))+ a = max(0, min(i - int(0.02 * SR), len(x) - int(0.15 * SR)))+ return x[a: a + int(0.15 * SR)]+ if kind == 'mp3':+ return _mp3(x)+ raise ValueError(kind)+++ROBUST = {'noise70': 0, 'room70': 0, 'gated': 1, 'short': None, 'mp3': None}+++def build_robust(clips, per_group=12):+ """perturbed copies of clips spread over every CV group (they keep the group, so held-out predictions stay+ held out): synthesized + pink noise floor / real room tone at -70 dBFS RMS, real gated to digital silence below+ -45 dB re peak, 0.15 s excerpts around the peak (both classes), MP3 128 kbps round trip (both classes)"""+ room = audio.load(os.path.join(ASSETS, 'ambience', 'fs-346642_rain-on-windows-interior.ogg'), mono=True, cache=False)+ room = room[np.argsort([rms_db(room[i:i + SR]) for i in range(0, len(room) - SR, SR)])[0] * SR:][: 20 * SR]+ out = []+ for kind, lab in ROBUST.items():+ rng = np.random.default_rng(len(kind))+ by = {}+ for c in clips:+ if lab is None or c['label'] == lab:+ key = (c['group'], c['type']) if c['label'] == 0 else c['group']+ by.setdefault(key, []).append(c)+ for key in sorted(by, key=str):+ pool = by[key]+ n = max(1, per_group // (6 if isinstance(key, tuple) else 1))+ for i in sorted(rng.choice(len(pool), min(n, len(pool)), replace=False)):+ c = pool[i]+ x, _ = sf.read(os.path.join(OUT, c['wav']), dtype='float32')+ y = perturb(kind, x, rng, room)+ cid = f'{kind}:{c["id"]}'+ e = dict(c, id=cid, wav=write(y, 'robust', cid.replace(':', '_').replace('@', '_').replace('#', '_')),+ perturb=kind, base=c['id'])+ out.append(e)+ return out+++def _codec(x, rng):+ c = [('libmp3lame', 'mp3', ['-b:a', f'{int(rng.choice([96, 128, 192]))}k'], 'mp3'),+ ('libvorbis', 'ogg', ['-q:a', str(int(rng.integers(3, 7)))], 'ogg'),+ ('aac', 'adts', ['-b:a', '128k'], 'aac')][int(rng.integers(0, 3))]+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-f', 'f32le', '-ar', str(SR), '-ac', '1', '-i', '-',+ '-c:a', c[0], *c[2], '-f', c[1], '-'], input=np.asarray(x, np.float32).tobytes(),+ capture_output=True, check=True)+ q = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-f', c[3], '-i', '-', '-f', 'f32le', '-ac', '1', '-ar',+ str(SR), '-'], input=p.stdout, capture_output=True, check=True)+ y = np.frombuffer(q.stdout, np.float32)+ return y[: len(x)] if len(y) >= len(x) else np.pad(y, (0, len(x) - len(y)))+++def augment(kind, x, rng):+ """training-only variants with randomised parameters (different from the fixed robustness set): a synthetic+ coloured noise floor (-85..-50 dBFS RMS; never a real room tone), gating to digital silence (-55..-30 dB re peak),+ a 0.08-0.4 s excerpt, a lossy codec round trip (mp3 96-192k / vorbis q3-6 / aac 128k)"""+ if kind == 'floor':+ X = np.fft.rfft(rng.standard_normal(len(x)))+ f = np.fft.rfftfreq(len(x), 1 / SR)+ X[1:] /= (f[1:] / 100.0) ** (float(rng.choice([0.0, 1.0, 2.0])) / 2)+ X[0] = 0+ nz = np.fft.irfft(X, len(x))+ return x + nz / (np.std(nz) + 1e-12) * 10 ** (rng.uniform(-85, -50) / 20)+ if kind == 'gate':+ k = int(rng.uniform(0.005, 0.02) * SR)+ e = np.repeat(np.sqrt(np.convolve(x ** 2, np.ones(k) / k, 'same')[::k] + 1e-20), k)[: len(x)]+ y = x.copy()+ y[: len(e)][e < np.abs(x).max() * 10 ** (rng.uniform(-55, -30) / 20)] = 0.0+ return y+ if kind == 'short':+ n = int(rng.uniform(0.08, 0.4) * SR)+ i = int(np.argmax(np.abs(x))) - int(rng.uniform(0.0, 0.05) * SR)+ a = max(0, min(i, len(x) - n))+ return x[a: a + n]+ return _codec(x, rng)+++def build_augment(clips):+ """one randomised training variant for about half of the clean clips (chosen by id hash)"""+ import hashlib+ out = []+ for c in clips:+ h = int(hashlib.sha1(c['id'].encode()).hexdigest()[:8], 16)+ rng = np.random.default_rng(h)+ if rng.random() >= 0.5:+ continue+ kinds, probs = (['floor', 'short', 'codec'], [0.45, 0.25, 0.3]) if c['label'] == 0 else (['gate', 'short', 'codec'], [0.35, 0.35, 0.3])+ kind = str(rng.choice(kinds, p=probs))+ x, _ = sf.read(os.path.join(OUT, c['wav']), dtype='float32')+ y = augment(kind, x.astype(np.float64), rng)+ cid = f'aug-{kind}:{c["id"]}'+ out.append(dict(c, id=cid, wav=write(y, 'augment', cid.replace(':', '_').replace('@', '_').replace('#', '_')),+ perturb='aug-' + kind, base=c['id'], augment=True))+ return out+++LAYER_LEVELS = (-12, -6, 0, 6)+++def build_layers(clips, per_group=24):+ """real 4 s beds (windows of ambience, music, films ... up to per_group per CV group) with one code-synthesized+ layer mixed in, label 0: does a code-made layer inside a real mix get found? Each bed gets two different layers at+ two of LAYER_LEVELS (layer RMS over the span it plays vs the bed RMS, dB). A one-shot layer plays 1-4 times at+ random offsets, a 4 s synth window plays throughout. The clip keeps the layer's family / group / type and records+ the bed's (bed_group, bed_src): run_eval keeps both groups out of a model before it scores the clip."""+ import hashlib+ beds = {}+ for c in clips:+ if c['label'] == 1 and c['end'] - c['start'] >= 3.5:+ beds.setdefault(c['group'], []).append(c)+ syn = sorted((c for c in clips if c['label'] == 0), key=lambda c: c['id'])+ out = []+ for g in sorted(beds):+ pool = sorted(beds[g], key=lambda c: hashlib.sha1(c['id'].encode()).hexdigest())[:per_group]+ for j, b in enumerate(pool):+ rng = np.random.default_rng(int(hashlib.sha1(b['id'].encode()).hexdigest()[:8], 16))+ x, _ = sf.read(os.path.join(OUT, b['wav']), dtype='float64')+ x = x[: int(WIN_S * SR)]+ for k in range(2):+ lc = syn[int(rng.integers(0, len(syn)))]+ lev = LAYER_LEVELS[(j + 2 * k) % len(LAYER_LEVELS)]+ y, _ = sf.read(os.path.join(OUT, lc['wav']), dtype='float64')+ lay, act = np.zeros(len(x)), np.zeros(len(x), bool)+ if len(y) >= 0.75 * len(x):+ lay[: min(len(x), len(y))] = y[: len(x)]+ act[:] = True+ else:+ for _ in range(int(rng.integers(1, 5))):+ a = int(rng.integers(0, len(x) - len(y)))+ lay[a: a + len(y)] += y+ act[a: a + len(y)] = True+ gain = 10 ** ((lev + rms_db(x) - rms_db(lay[act])) / 20)+ mix = x + gain * lay+ mix *= min(1.0, 0.98 / (np.abs(mix).max() + 1e-12))+ cid = f'layer{lev:+d}:{lc["id"]}+{b["id"]}'+ out.append(dict(clip_entry(cid, write(mix, 'layers', f'L{len(out):04d}'), 0, lc['family'], lc['group'],+ lc['type'], lc['src'], b['start'], b['end']),+ perturb='layer', level_db=lev, layer=lc['id'], bed=b['id'], bed_group=b['group'],+ bed_src=b['src'], bed_family=b['family']))+ return out+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])+ ap.add_argument('--only', default='real,rejected,films,retired,cinetic,generic,mixes,robust,augment',+ help='parts to (re)build; also: layers (real beds + a code-synthesized layer, label 0)')+ a = ap.parse_args(argv)+ os.makedirs(OUT, exist_ok=True)+ mpath = os.path.join(OUT, 'manifest.json')+ man = json.load(open(mpath)) if os.path.exists(mpath) else {}+ builders = {'real': build_real, 'rejected': lambda: build_real(False), 'films': build_films, 'retired': build_retired,+ 'cinetic': build_cinetic, 'generic': build_generic, 'mixes': build_mixes}+ for part in a.only.split(','):+ if part in builders:+ man[part] = builders[part]()+ print(f'{part}: {len(man[part])} clips')+ json.dump(man, open(mpath, 'w'), indent=1)+ base = [c for k in builders for c in man.get(k, [])]+ if 'robust' in a.only.split(','):+ man['robust'] = build_robust(base)+ print(f'robust: {len(man["robust"])} clips')+ if 'augment' in a.only.split(','):+ man['augment'] = build_augment(base)+ print(f'augment: {len(man["augment"])} clips')+ if 'layers' in a.only.split(','):+ man['layers'] = build_layers(base)+ print(f'layers: {len(man["layers"])} clips')+ json.dump(man, open(mpath, 'w'), indent=1)+ n = sum(len(v) for v in man.values())+ print(f'manifest: {n} clips -> {mpath}')+++if __name__ == '__main__':+ main()diff --git a/eval/realness/common.py b/eval/realness/common.pynew file mode 100644index 0000000..7ecf164--- /dev/null+++ b/eval/realness/common.py@@ -0,0 +1,214 @@+"""Shared training code for the realness detector: dataset loading, the two logistic heads, the fusion layer and+grouped cross-validation. Used by train.py (final model) and run_eval.py (held-out evaluation).++Model: head_clap = standardised L2 logistic regression on the 512-d CLAP embedding; head_signal = the same on the+signal measurements of earshot.realness.signal_features; fusion = logistic regression on the two head logits, fitted on+inner out-of-group head predictions (so its weights reflect how well each head generalises to unseen groups).+Sample weights: each class weighs 1 in total, split equally over its CV groups, equally over the clips of a group.+Layered clips (a real bed + one code-synthesized layer, label 0) belong to two groups: the layer's and the bed's. They+are never test clips of the clip-level schemes, and a model never trains on one whose layer group or bed group is held+out.+"""+import json+import os+import sys++import numpy as np+from scipy.optimize import minimize++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+OUT = os.path.join(os.path.dirname(os.path.dirname(HERE)), 'out', 'realness')+FEATS = os.path.join(OUT, 'feats')+SIG_INPUTS = ['silence_frac', 'floor_db', 'spec_floor_db', 'log_flatness', 'peakiness_db', 'line_share', 'bandwidth_khz',+ 'quiet_flat', 'tonal_frames', 'pitch_wobble', 'env_resid_db', 'texture_kurt', 'crest_db']+# physical direction of each measurement (+1: larger reads as a recording): the signal head's coefficients are+# constrained to these signs, so a measurement can never vote against its physics because of a dataset quirk+SIGNS = {'silence_frac': -1, 'floor_db': 1, 'spec_floor_db': 1, 'log_flatness': 1, 'peakiness_db': -1, 'line_share': -1,+ 'bandwidth_khz': 1, 'quiet_flat': 1, 'tonal_frames': -1, 'pitch_wobble': 1, 'env_resid_db': 1, 'texture_kurt': 1,+ 'crest_db': 1}+C_CLAP = (0.001, 0.003, 0.01)+C_SIGNAL = 0.1+C_AST = 0.01+AUG_WEIGHT = 0.35+++class Data:+ """clips with features, aligned arrays; .sub(mask) for a subset"""++ def __init__(self, clips, clap, sig, ast=None):+ self.clips = clips+ self.ids = np.array([c['id'] for c in clips])+ self.y = np.array([c['label'] for c in clips])+ self.family = np.array([c['family'] for c in clips])+ self.group = np.array([c['group'] for c in clips])+ self.type = np.array([c['type'] for c in clips])+ self.src = np.array([c['src'] for c in clips])+ self.perturb = np.array([c.get('perturb', '') for c in clips])+ self.augment = np.array([bool(c.get('augment')) for c in clips])+ self.base = np.array([c.get('base', c['id']) for c in clips])+ self.is_layer = self.perturb == 'layer'+ self.bed_group = np.array([c.get('bed_group', c['group']) for c in clips])+ self.bed_src = np.array([c.get('bed_src', c['src']) for c in clips])+ self.Xc = np.stack([clap[i] for i in self.ids]) if len(clips) else np.zeros((0, 512))+ self.Xs = np.array([[np.nan if sig[i] is None else sig[i].get(k, np.nan) for k in SIG_INPUTS] for i in self.ids], float)+ self.Xa = np.stack([ast[i] for i in self.ids]) if ast is not None and len(clips) else None++ def sub(self, mask):+ d = object.__new__(Data)+ for k, v in self.__dict__.items():+ d.__dict__[k] = (v[mask] if isinstance(v, np.ndarray) else [c for c, m in zip(v, mask) if m]) if v is not None else None+ return d++ def __len__(self):+ return len(self.ids)+++def load(require_ast=False, robust=True):+ man = json.load(open(os.path.join(OUT, 'manifest.json')))+ clips = [c for k, part in man.items() for c in part if robust or k != 'robust']+ # a few recordings sit in two asset folders (e.g. foley and toy): keep them in the first group only, so no+ # recording is ever on both sides of a held-out split+ owner, seen, drop = {}, set(), set()+ for c in clips:+ if c['label'] == 1 and 'perturb' not in c:+ key = os.path.basename(c['src']).split('_')[0]+ if owner.setdefault(key, c['group']) != c['group'] or c['id'] in seen:+ drop.add(c['id'])+ seen.add(c['id'])+ clips = [c for c in clips if c['id'] not in drop and c.get('base') not in drop]+ sig = json.load(open(os.path.join(FEATS, 'signal.json')))+ z = np.load(os.path.join(FEATS, 'clap.npz'), allow_pickle=False)+ clap = dict(zip(z['ids'], z['emb']))+ ast = None+ if require_ast:+ za = np.load(os.path.join(FEATS, 'ast.npz'), allow_pickle=False)+ ast = dict(zip(za['ids'], za['emb']))+ keep = [c for c in clips if c['id'] in clap and c['id'] in sig and sig[c['id']] is not None and (ast is None or c['id'] in ast)]+ return Data(keep, clap, sig, ast)+++def weights(d, groups, aug_weight=AUG_WEIGHT):+ """each class weighs 1, split equally over its groups; inside a group an augmented or layered clip weighs+ aug_weight of a clean clip"""+ w = np.zeros(len(d))+ u = np.where(d.augment | d.is_layer, aug_weight, 1.0)+ for lab in (0, 1):+ gs = sorted(set(groups[d.y == lab]))+ for g in gs:+ m = (d.y == lab) & (groups == g)+ w[m] = u[m] / (len(gs) * u[m].sum())+ return w * len(d) / w.sum()+++def logreg(X, y, w, C, signs=None):+ """weighted L2 logistic regression (sklearn's objective: C * sum w*logloss + |b|^2 / 2, free intercept);+ signs (+1 / -1 / 0 per column) bound coefficients to that sign (0 = free)"""+ n, k = X.shape+ s = 2 * y - 1.0++ def f(b):+ m = s * (X @ b[:k] + b[k])+ lo = np.logaddexp(0, -m)+ g = -s * w / (1 + np.exp(np.clip(m, -50, 50)))+ return C * np.sum(w * lo) + 0.5 * b[:k] @ b[:k], np.r_[C * (X.T @ g) + b[:k], C * g.sum()]+ bounds = None+ if signs is not None:+ bounds = [(0, None) if v > 0 else (None, 0) if v < 0 else (None, None) for v in signs] + [(None, None)]+ r = minimize(f, np.zeros(k + 1), jac=True, method='L-BFGS-B', bounds=bounds, options={'maxiter': 3000})+ return r.x[:k], float(r.x[k])+++def fit_head(X, y, w, C, signs=None):+ mean = np.nanmean(X, axis=0)+ X = np.where(np.isfinite(X), X, mean)+ lo, hi = np.percentile(X, 0.5, axis=0), np.percentile(X, 99.5, axis=0)+ X = np.clip(X, lo, hi)+ scale = X.std(axis=0) + 1e-6+ coef, b0 = logreg((X - mean) / scale, y, w, C, signs)+ return {'mean': mean.tolist(), 'scale': scale.tolist(), 'clip': [lo.tolist(), hi.tolist()],+ 'coef': coef.tolist(), 'intercept': b0}+++def head_logits(h, X):+ mean = np.asarray(h['mean'])+ X = np.where(np.isfinite(X), X, mean)+ X = np.clip(X, np.asarray(h['clip'][0]), np.asarray(h['clip'][1]))+ return ((X - mean) / np.asarray(h['scale'])) @ np.asarray(h['coef']) + h['intercept']+++def head_signs(name):+ return [SIGNS[k] for k in SIG_INPUTS] if name == 'signal' else None+++def head_inputs(d, name):+ return {'clap': d.Xc, 'signal': d.Xs, 'ast': None if d.Xa is None else np.log(np.clip(d.Xa, 1e-4, 1))}[name]+++def inner_folds(groups, k=5):+ """split the distinct groups into k folds (deterministic, round-robin over sorted names)"""+ gs = sorted(set(groups))+ return [np.isin(groups, gs[i::k]) for i in range(min(k, len(gs)))]+++def fit_model(d, groups, heads=('clap', 'signal'), c_clap=None, inner=5, aug_weight=AUG_WEIGHT):+ """heads + fusion on data d. c_clap: fixed C for the CLAP head, else chosen by inner grouped CV (log loss on+ clean clips: augmented copies help training but are not what the model is scored on)."""+ w = weights(d, groups, aug_weight)+ clean = ~d.augment & ~d.is_layer+ folds = inner_folds(groups, inner)+ cs = {'signal': [C_SIGNAL], 'ast': [C_AST], 'clap': [c_clap] if c_clap else list(C_CLAP)}+ oof = {}+ chosen = {}+ for name in heads:+ X = head_inputs(d, name)+ best = None+ for C in cs[name]:+ z = np.zeros(len(d))+ for te in folds:+ tr = ~te+ if len(set(d.y[tr])) < 2:+ z[te] = 0.0+ continue+ z[te] = head_logits(fit_head(X[tr], d.y[tr], w[tr], C, head_signs(name)), X[te])+ p = 1 / (1 + np.exp(-np.clip(z, -30, 30)))+ ll = -np.sum((w * (d.y * np.log(p + 1e-9) + (1 - d.y) * np.log(1 - p + 1e-9)))[clean]) / w[clean].sum()+ if best is None or ll < best[0]:+ best = (ll, C, z)+ chosen[name], oof[name] = best[1], best[2]+ F = np.stack([oof[h] for h in heads], axis=1)+ coef, b0 = logreg(F[clean], d.y[clean], w[clean], 1.0, [1] * len(heads))+ model = {'heads': {}, 'fusion': {'heads': list(heads), 'coef': coef.tolist(), 'intercept': b0}, 'C': chosen}+ for name in heads:+ model['heads'][name] = fit_head(head_inputs(d, name), d.y, w, chosen[name], head_signs(name))+ return model+++def predict(model, d, return_heads=False):+ fu = model['fusion']+ L = {h: head_logits(model['heads'][h], head_inputs(d, h)) for h in fu['heads']}+ s = fu['intercept'] + sum(c * L[h] for h, c in zip(fu['heads'], fu['coef']))+ p = 1 / (1 + np.exp(-np.clip(s, -30, 30)))+ return (p, L) if return_heads else p+++def logo(d, groups, train_mask=None, augment=True, layers=False, bed_groups=None, **kw):+ """leave-one-group-out predictions: every clip is scored by a model that never saw its group (nor augmented+ copies of its clips). train_mask limits which clips may ever be used for training (e.g. never the retired score);+ augment=False trains on clean clips only; layers=True also trains on layered clips whose layer group and bed group+ (bed_groups) are both outside the held-out group. Returns p (n,) and head logits (augmented and layered clips: NaN)."""+ p = np.full(len(d), np.nan)+ L = {}+ tm = np.ones(len(d), bool) if train_mask is None else train_mask+ bg = groups if bed_groups is None else bed_groups+ usable = (d.perturb == '') | (d.augment & augment) | (d.is_layer & layers)+ for g in sorted(set(groups[~d.is_layer])):+ te = (groups == g) & ~d.augment & ~d.is_layer+ tr = (groups != g) & (bg != g) & tm & usable+ sub = d.sub(tr)+ m = fit_model(sub, groups[tr], **kw)+ pt, Lt = predict(m, d.sub(te), True)+ p[te] = pt+ for h, v in Lt.items():+ L.setdefault(h, np.full(len(d), np.nan))[te] = v+ return p, Ldiff --git a/eval/realness/extract.py b/eval/realness/extract.pynew file mode 100644index 0000000..16066ed--- /dev/null+++ b/eval/realness/extract.py@@ -0,0 +1,113 @@+"""Compute the realness features of every clip in out/realness/manifest.json (resumable and time-boxed).++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/extract.py [--budget 480] [--what signal,clap] [--ast N]++Writes out/realness/feats/signal.json (id -> earshot.realness.signal_features, None when silent), clap.npz (ids,+emb: the same CLAP call as earshot.realness at inference) and, with --ast N, ast.npz for a fixed stratified subset of+about N clips (AudioSet probabilities; AST costs ~5x CLAP on this CPU, so it is a baseline / optional head only).+Run it again until it prints "done": each run stops after --budget seconds and keeps what it computed.+"""+import argparse+import hashlib+import json+import os+import sys+import time++import numpy as np+import soundfile as sf++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+from earshot import realness # noqa: E402++OUT = os.path.join(os.path.dirname(os.path.dirname(HERE)), 'out', 'realness')+FEATS = os.path.join(OUT, 'feats')+++def all_clips():+ man = json.load(open(os.path.join(OUT, 'manifest.json')))+ return [c for part in man.values() for c in part]+++def load_clip(c):+ x, _ = sf.read(os.path.join(OUT, c['wav']), dtype='float32')+ return x if x.ndim == 1 else x.mean(axis=1)+++def ast_subset(clips, n):+ """deterministic stratified subset: the same share of every (label, group, type) cell, picked by id hash"""+ cells = {}+ for c in clips:+ if 'perturb' not in c:+ cells.setdefault((c['label'], c['group'], c['type']), []).append(c)+ frac = n / max(1, sum(len(v) for v in cells.values()))+ out = []+ for k in sorted(cells, key=str):+ v = sorted(cells[k], key=lambda c: hashlib.sha1(c['id'].encode()).hexdigest())+ out += v[: max(1, int(round(frac * len(v))))]+ return out+++def save_npz(path, ids, arr):+ tmp = path + '.tmp.npz'+ np.savez(tmp, ids=np.array(ids), emb=np.asarray(arr, np.float32))+ os.replace(tmp, path)+++def load_npz(path):+ if not os.path.exists(path):+ return [], []+ z = np.load(path, allow_pickle=False)+ return list(z['ids']), list(z['emb'])+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])+ ap.add_argument('--budget', type=float, default=480)+ ap.add_argument('--what', default='signal,clap')+ ap.add_argument('--ast', type=int, default=0, help='AST probabilities for a stratified subset of ~N clips')+ ap.add_argument('--batch', type=int, default=16)+ a = ap.parse_args(argv)+ t0 = time.time()+ os.makedirs(FEATS, exist_ok=True)+ clips = all_clips()+ what = a.what.split(',')+ if 'signal' in what:+ sp = os.path.join(FEATS, 'signal.json')+ sig = json.load(open(sp)) if os.path.exists(sp) else {}+ todo = [c for c in clips if c['id'] not in sig]+ for c in todo:+ x = load_clip(c)+ sig[c['id']] = realness.signal_features(x) if realness.audible(x) else None+ json.dump(sig, open(sp, 'w'))+ print(f'signal: {len(todo)} new, {len(sig)} total ({time.time() - t0:.0f} s)')+ jobs = []+ if 'clap' in what:+ jobs.append(('clap', clips))+ if a.ast:+ jobs.append(('ast', ast_subset(clips, a.ast)))+ left = 0+ for name, pool in jobs:+ path = os.path.join(FEATS, f'{name}.npz')+ ids, arr = load_npz(path)+ todo = [c for c in pool if c['id'] not in set(ids)]+ if todo and time.time() - t0 < a.budget:+ from earshot.models import ast, clap+ model = clap.get() if name == 'clap' else ast.get()+ fn = model.embed_audio if name == 'clap' else model.probs+ if name == 'ast':+ json.dump(model.labels, open(os.path.join(FEATS, 'ast_labels.json'), 'w'))+ print(f'{name}: {len(todo)} to do, model ready after {time.time() - t0:.0f} s', flush=True)+ while todo and time.time() - t0 < a.budget:+ chunk, todo = todo[:a.batch], todo[a.batch:]+ ids += [c['id'] for c in chunk]+ arr += list(fn([load_clip(c) for c in chunk]))+ save_npz(path, ids, arr)+ left += len(todo)+ print(f'{name}: {len(ids)} stored, {len(todo)} left ({time.time() - t0:.0f} s)', flush=True)+ print('done' if not left else 'run again to continue')+++if __name__ == '__main__':+ main()diff --git a/eval/realness/generic_synth.py b/eval/realness/generic_synth.pynew file mode 100644index 0000000..60d42d7--- /dev/null+++ b/eval/realness/generic_synth.py@@ -0,0 +1,278 @@+"""Generic procedural sounds written from scratch (independent of the cinetic engine) as negatives for the realness+detector: the kind of audio a coding agent writes when it "synthesizes a sound effect" in numpy.++ from generic_synth import render, TYPES+ x = render('beep', seed=3) # float32 mono or (n, 2) at 48 kHz, peak 1++Every type draws its parameters from `seed`, so a (type, seed) pair always renders the same samples. Types cover+tones (beeps, chords, chiptune arps, FM bells, additive pads, Karplus-Strong plucks, sirens, DTMF), noise designs+(whooshes, risers, impacts, wind, rain, clicks, LFSR noise), a drum-machine loop and Schroeder reverb applied to some.+"""+import numpy as np+from scipy import signal++SR = 48000+++def _n(d):+ return int(round(d * SR))+++def _t(d):+ return np.arange(_n(d)) / SR+++def _osc(kind, f, d, rng):+ """naive (aliasing) oscillators as code usually writes them; f may be an array (glide)"""+ t = _t(d)+ f = np.resize(np.asarray(f, float), t.shape)+ ph = np.cumsum(f) / SR + rng.random()+ if kind == 'sine':+ return np.sin(2 * np.pi * ph)+ if kind == 'square':+ return np.sign(np.sin(2 * np.pi * ph))+ if kind == 'saw':+ return 2 * (ph % 1.0) - 1+ return 2 * np.abs(2 * (ph % 1.0) - 1) - 1+++def _env(d, rng, kind=None):+ t = _t(d)+ kind = kind or rng.choice(['exp', 'lin', 'adsr', 'gate'])+ if kind == 'exp':+ return np.exp(-t / (d * rng.uniform(0.1, 0.4)))+ if kind == 'lin':+ return np.clip(1 - t / d, 0, 1)+ if kind == 'gate':+ e = np.ones_like(t)+ k = int(0.002 * SR)+ e[:k] = np.linspace(0, 1, k)+ e[-k:] = np.linspace(1, 0, k)+ return e+ a, dc, s, r = rng.uniform(0.002, 0.05), rng.uniform(0.02, 0.2), rng.uniform(0.3, 0.8), rng.uniform(0.05, 0.3)+ return np.interp(t, [0, a, a + dc, max(a + dc, d - r), d], [0, 1, s, s, 0])+++def _bandpass_sweep(x, f_lo, f_hi, q=2.0, block=512):+ """block-wise band-pass whose centre follows f_lo -> f_hi (arrays allowed)"""+ n = len(x)+ fc = np.geomspace(f_lo, f_hi, n) if np.isscalar(f_lo) else np.asarray(f_lo)+ y = np.zeros(n)+ zi = None+ for s in range(0, n, block):+ c = float(np.clip(fc[min(n - 1, s + block // 2)], 60, SR * 0.4))+ sos = signal.butter(2, [c / (1 + 0.5 / q), min(c * (1 + 0.5 / q), SR * 0.45)], 'band', fs=SR, output='sos')+ if zi is None:+ zi = np.zeros((sos.shape[0], 2))+ y[s:s + block], zi = signal.sosfilt(sos, x[s:s + block], zi=zi)+ return y+++def _pink(n, rng):+ X = np.fft.rfft(rng.standard_normal(n))+ f = np.fft.rfftfreq(n, 1 / SR)+ X[1:] /= np.sqrt(f[1:])+ X[0] = 0+ y = np.fft.irfft(X, n)+ return y / (np.std(y) + 1e-12)+++def schroeder(x, rng, rt=None, mix=None):+ """classic Schroeder reverb: 4 parallel combs + 2 allpasses (the textbook algorithmic reverb)"""+ rt = rt or rng.uniform(0.6, 2.5)+ mix = mix if mix is not None else rng.uniform(0.2, 0.5)+ x = np.concatenate([x, np.zeros(int(rt * SR))])+ wet = np.zeros_like(x)+ for dl in (1557, 1617, 1491, 1422):+ g = 10 ** (-3 * dl / SR / rt)+ wet += signal.lfilter([1.0], np.r_[1.0, np.zeros(dl - 1), -g], x)+ for dl, g in ((225, 0.5), (556, 0.5)):+ wet = signal.lfilter(np.r_[-g, np.zeros(dl - 1), 1.0], np.r_[1.0, np.zeros(dl - 1), -g], wet)+ return (1 - mix) * x + mix * wet / (np.abs(wet).max() + 1e-12) * np.abs(x).max()+++def beep(rng):+ d = rng.uniform(0.05, 0.6)+ f = rng.choice([rng.uniform(300, 3000), 440.0, 880.0, 1000.0, 1200.0])+ if rng.random() < 0.3:+ f = np.geomspace(f, f * rng.choice([0.5, 1.5, 2.0]), _n(d))+ return _osc(rng.choice(['sine', 'sine', 'square', 'saw', 'tri']), f, d, rng) * _env(d, rng)+++def chord(rng):+ d = rng.uniform(0.5, 3.0)+ root = 110 * 2 ** (rng.integers(0, 24) / 12)+ kind = rng.choice(['sine', 'saw', 'square', 'tri'])+ ivs = [[0, 4, 7], [0, 3, 7], [0, 4, 7, 11], [0, 7, 12, 16]][rng.integers(0, 4)]+ x = sum(_osc(kind, root * 2 ** (iv / 12), d, rng) for iv in ivs)+ if kind != 'sine':+ x = signal.sosfilt(signal.butter(2, rng.uniform(800, 5000), 'low', fs=SR, output='sos'), x)+ return x * _env(d, rng, 'adsr')+++def chiptune(rng):+ bpm = rng.uniform(100, 180)+ step = 60 / bpm / rng.choice([2, 4])+ root = 220 * 2 ** (rng.integers(0, 12) / 12)+ scale = [0, 2, 4, 5, 7, 9, 11, 12]+ kind = rng.choice(['square', 'square', 'tri', 'saw', 'sine'])+ out = []+ for _ in range(int(rng.uniform(1.5, 4.0) / step)):+ n = _osc(kind, root * 2 ** (scale[rng.integers(0, 8)] / 12), step, rng) * _env(step, rng, rng.choice(['gate', 'exp']))+ out.append(n * 0.6)+ return np.concatenate(out)+++def fm(rng):+ d = rng.uniform(0.8, 3.5)+ t = _t(d)+ fc = 200 * 2 ** (rng.uniform(0, 4))+ ratio, idx, tau = rng.choice([1.0, 1.4, 2.0, 3.5, 7.0]), rng.uniform(1, 8), rng.uniform(0.2, 1.5)+ x = np.sin(2 * np.pi * fc * t + idx * np.exp(-t / (tau * 0.5)) * np.sin(2 * np.pi * fc * ratio * t))+ return x * np.exp(-t / tau) * np.minimum(1, t / 0.002)+++def pad(rng):+ d = rng.uniform(2.0, 4.0)+ t = _t(d)+ f0 = 80 * 2 ** rng.uniform(0, 3)+ x = sum((1 / k ** rng.uniform(0.8, 2)) * np.sin(2 * np.pi * f0 * k * t + rng.random() * 6) *+ (1 + 0.3 * np.sin(2 * np.pi * rng.uniform(0.1, 1.5) * t)) for k in range(1, 12))+ return x * _env(d, rng, 'adsr')+++def pluck(rng):+ """Karplus-Strong string"""+ f = 110 * 2 ** rng.uniform(0, 3)+ d = rng.uniform(0.6, 2.0)+ n, p = _n(d), int(SR / f)+ buf = rng.uniform(-1, 1, p)+ out = np.zeros(n)+ k = rng.uniform(0.990, 0.999)+ for i in range(n):+ out[i] = buf[i % p]+ buf[i % p] = k * 0.5 * (buf[i % p] + buf[(i + 1) % p])+ return out+++def siren(rng):+ d = rng.uniform(1.0, 3.0)+ t = _t(d)+ f = rng.uniform(500, 1200) + rng.uniform(100, 500) * np.sin(2 * np.pi * rng.uniform(0.5, 6) * t)+ return _osc(rng.choice(['sine', 'square', 'saw']), f, d, rng) * _env(d, rng, 'gate')+++def dtmf(rng):+ rows, cols = [697, 770, 852, 941], [1209, 1336, 1477]+ out = []+ for _ in range(rng.integers(3, 9)):+ d = rng.uniform(0.08, 0.2)+ t = _t(d)+ out += [np.sin(2 * np.pi * rows[rng.integers(0, 4)] * t) + np.sin(2 * np.pi * cols[rng.integers(0, 3)] * t),+ np.zeros(int(rng.uniform(0.05, 0.12) * SR))]+ return np.concatenate(out)+++def whoosh(rng):+ d = rng.uniform(0.4, 2.0)+ n = _n(d)+ src = _pink(n, rng) if rng.random() < 0.5 else rng.standard_normal(n)+ f = np.geomspace(rng.uniform(150, 600), rng.uniform(1500, 6000), n)+ if rng.random() < 0.5:+ f = np.concatenate([f[: n // 2], f[: n - n // 2][::-1]])+ return _bandpass_sweep(src, f, None, q=rng.uniform(1, 4)) * np.hanning(n) ** rng.uniform(0.7, 2)+++def riser(rng):+ d = rng.uniform(1.0, 4.0)+ n = _n(d)+ u = np.arange(n) / n+ x = _bandpass_sweep(rng.standard_normal(n), np.geomspace(200, rng.uniform(4000, 10000), n), None) * u ** 2+ if rng.random() < 0.6:+ x += 0.5 * _osc(rng.choice(['saw', 'sine', 'square']), np.geomspace(rng.uniform(80, 200), rng.uniform(800, 2000), n), d, rng) * u ** 2+ return x+++def impact(rng):+ d = rng.uniform(0.5, 2.5)+ t = _t(d)+ f = rng.uniform(35, 60) + rng.uniform(60, 200) * np.exp(-t / rng.uniform(0.03, 0.2))+ x = _osc('sine', f, d, rng) * np.exp(-t / rng.uniform(0.2, 0.8))+ x += rng.uniform(0.2, 0.8) * rng.standard_normal(len(t)) * np.exp(-t / rng.uniform(0.01, 0.08))+ return np.tanh(x * rng.uniform(1, 3))+++def wind(rng):+ d = rng.uniform(2.0, 4.0)+ n = _n(d)+ t = _t(d)+ gust = 0.6 + 0.4 * np.sin(2 * np.pi * rng.uniform(0.1, 0.5) * t + rng.random() * 6)+ fc = rng.uniform(300, 900) * (0.7 + 0.6 * gust)+ return _bandpass_sweep(_pink(n, rng), fc, None, q=rng.uniform(0.7, 2)) * gust+++def rain(rng):+ d = rng.uniform(2.0, 4.0)+ n = _n(d)+ x = _pink(n, rng) * 0.05+ for _ in range(int(d * rng.uniform(30, 300))):+ i = rng.integers(0, n - 2000)+ k = np.arange(int(rng.uniform(100, 1500)))+ x[i:i + len(k)] += rng.uniform(0.1, 1) * np.sin(2 * np.pi * rng.uniform(1500, 6000) * k / SR) * np.exp(-k / rng.uniform(50, 400))+ return signal.sosfilt(signal.butter(2, rng.uniform(300, 1000), 'high', fs=SR, output='sos'), x)+++def click(rng):+ d = rng.uniform(0.02, 0.15)+ t = _t(d)+ x = rng.standard_normal(len(t)) * np.exp(-t / rng.uniform(0.001, 0.01))+ if rng.random() < 0.5:+ x = signal.sosfilt(signal.butter(2, sorted(rng.uniform(500, 8000, 2)), 'band', fs=SR, output='sos'), x)+ return x+++def lfsr(rng):+ """8-bit console noise channel: a 15-bit LFSR clocked at a low rate, gated in bursts"""+ d = rng.uniform(0.2, 1.5)+ n = _n(d)+ hold = int(SR / rng.uniform(2000, 16000))+ reg, bits = 1, []+ for _ in range(n // hold + 1):+ b = (reg ^ (reg >> 1)) & 1+ reg = (reg >> 1) | (b << 14)+ bits.append(1.0 if reg & 1 else -1.0)+ return np.repeat(bits, hold)[:n] * _env(d, rng, rng.choice(['exp', 'lin']))+++def drums(rng):+ bpm = rng.uniform(90, 140)+ step = 60 / bpm / 4+ n = int(16 * step * SR)+ out = np.zeros(n + SR)+ t = _t(0.5)+ kick = np.sin(2 * np.pi * np.cumsum(50 + 120 * np.exp(-t / 0.03)) / SR) * np.exp(-t / 0.15)+ snare = rng.standard_normal(len(t)) * np.exp(-t / 0.06) * 0.6 + np.sin(2 * np.pi * 190 * t) * np.exp(-t / 0.05) * 0.4+ hat = signal.sosfilt(signal.butter(2, 7000, 'high', fs=SR, output='sos'), rng.standard_normal(len(t))) * np.exp(-t / 0.02) * 0.4+ for s in range(16):+ i = int(s * step * SR)+ for hit, on in ((kick, s % 4 == 0), (snare, s % 8 == 4), (hat, s % 2 == 0 or rng.random() < 0.3)):+ if on:+ out[i:i + len(hit)] += hit+ return out[:n]+++TYPES = {f.__name__: f for f in (beep, chord, chiptune, fm, pad, pluck, siren, dtmf, whoosh, riser, impact, wind, rain,+ click, lfsr, drums)}+++def render(kind, seed=0):+ """one sound of `kind` from `seed`: float32, peak 1, mono (stereo for a random third, as code often widens)"""+ rng = np.random.default_rng(1000 * (list(TYPES).index(kind) + 1) + seed)+ x = TYPES[kind](rng).astype(np.float64)+ if kind not in ('wind', 'rain', 'drums') and rng.random() < 0.3:+ x = schroeder(x, rng)+ x = x / (np.abs(x).max() + 1e-12)+ if rng.random() < 0.33:+ dl = int(rng.uniform(0.0003, 0.02) * SR)+ x = np.stack([x, np.r_[np.zeros(dl), x[:-dl]]], axis=1)+ return x.astype(np.float32)diff --git a/eval/realness/results.json b/eval/realness/results.jsonnew file mode 100644index 0000000..aecea54--- /dev/null+++ b/eval/realness/results.json@@ -0,0 +1,3720 @@+{+ "data": {+ "clips": 1484,+ "real": 691,+ "synth": 793,+ "by_family": {+ "cinetic": 318,+ "generic": 260,+ "pro-film": 72,+ "pro-music": 119,+ "real-asset": 366,+ "real-rejected": 134,+ "retired": 215+ },+ "real_groups": [+ "ambience",+ "film",+ "foley",+ "garden",+ "keyboard",+ "magic",+ "motion",+ "music",+ "toy"+ ],+ "perturbed_clips": 1863+ },+ "schemes": {+ "A": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9219,+ "[email protected]": 0.8598,+ "brier": 0.1088,+ "logloss": 0.36+ },+ "A_noaug": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9314,+ "[email protected]": 0.8693,+ "brier": 0.1023,+ "logloss": 0.3387+ },+ "A_clap": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9257,+ "[email protected]": 0.8612,+ "brier": 0.1069,+ "logloss": 0.3514+ },+ "A_signal": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.721,+ "[email protected]": 0.7123,+ "brier": 0.2448,+ "logloss": 0.7195+ },+ "B": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9576,+ "[email protected]": 0.9016,+ "brier": 0.0764,+ "logloss": 0.2599+ },+ "C": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.8854,+ "[email protected]": 0.8389,+ "brier": 0.1298,+ "logloss": 0.4509+ },+ "D": {+ "n": 1224,+ "n_real": 691,+ "auc": 0.916,+ "[email protected]": 0.8554,+ "brier": 0.1147,+ "logloss": 0.401,+ "note": "real vs retired + cinetic, neither ever trained on (generic generators were the only negatives)"+ },+ "M": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.92,+ "[email protected]": 0.8801,+ "brier": 0.0984,+ "logloss": 0.3549+ }+ },+ "baselines": {+ "clap_zero_shot": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.8579,+ "[email protected]": 0.7487,+ "brier": 0.1887,+ "logloss": 0.6675,+ "prompts": {+ "real": [+ "a real recording made with a microphone",+ "a field recording of a real place",+ "a real foley recording of an object",+ "a professionally produced music track",+ "a professional cinematic sound effect from a sound library",+ "a natural acoustic sound"+ ],+ "synth": [+ "a cheap synthesized beep",+ "a pure sine wave test tone",+ "a simple electronic synthesizer tone made by code",+ "an 8-bit video game sound effect",+ "filtered white noise",+ "a basic drum machine loop"+ ]+ }+ },+ "signal_univariate_auc": {+ "silence_frac": 0.5435,+ "floor_db": 0.4436,+ "spec_floor_db": 0.7546,+ "log_flatness": 0.7366,+ "peakiness_db": 0.7232,+ "line_share": 0.6516,+ "bandwidth_khz": 0.6971,+ "quiet_flat": 0.7859,+ "tonal_frames": 0.599,+ "pitch_wobble": 0.6744,+ "env_resid_db": 0.594,+ "texture_kurt": 0.5601,+ "crest_db": 0.7385+ }+ },+ "thresholds": {+ "hi": 0.77,+ "lo": 0.17,+ "scheme": "M",+ "how": ">= 92.5% precision on each side"+ },+ "band": {+ "coverage": 0.7904,+ "precision_real": 0.9241,+ "precision_synth": 0.9224,+ "real_called_synth": 0.0666,+ "synth_called_real": 0.0555,+ "uncertain_frac": 0.2096+ },+ "band_alt": {+ "0.95": {+ "hi": 0.88,+ "lo": 0.05,+ "coverage": 0.5195,+ "precision_real": 0.9519,+ "precision_synth": 0.9108,+ "real_called_synth": 0.0405,+ "synth_called_real": 0.0277,+ "uncertain_frac": 0.4805+ },+ "0.925": {+ "hi": 0.77,+ "lo": 0.17,+ "coverage": 0.7904,+ "precision_real": 0.9241,+ "precision_synth": 0.9224,+ "real_called_synth": 0.0666,+ "synth_called_real": 0.0555,+ "uncertain_frac": 0.2096+ },+ "0.9": {+ "hi": 0.68,+ "lo": 0.47,+ "coverage": 0.9407,+ "precision_real": 0.9002,+ "precision_synth": 0.9007,+ "real_called_synth": 0.11,+ "synth_called_real": 0.0794,+ "uncertain_frac": 0.0593+ }+ },+ "calibration": [+ {+ "bin": "0.0-0.1",+ "n": 494,+ "mean_p": 0.041,+ "frac_real": 0.073+ },+ {+ "bin": "0.1-0.2",+ "n": 119,+ "mean_p": 0.141,+ "frac_real": 0.118+ },+ {+ "bin": "0.2-0.3",+ "n": 67,+ "mean_p": 0.242,+ "frac_real": 0.179+ },+ {+ "bin": "0.3-0.4",+ "n": 62,+ "mean_p": 0.347,+ "frac_real": 0.129+ },+ {+ "bin": "0.4-0.5",+ "n": 37,+ "mean_p": 0.455,+ "frac_real": 0.324+ },+ {+ "bin": "0.5-0.6",+ "n": 42,+ "mean_p": 0.555,+ "frac_real": 0.571+ },+ {+ "bin": "0.6-0.7",+ "n": 44,+ "mean_p": 0.66,+ "frac_real": 0.477+ },+ {+ "bin": "0.7-0.8",+ "n": 62,+ "mean_p": 0.756,+ "frac_real": 0.742+ },+ {+ "bin": "0.8-0.9",+ "n": 134,+ "mean_p": 0.856,+ "frac_real": 0.866+ },+ {+ "bin": "0.9-1.0",+ "n": 423,+ "mean_p": 0.96,+ "frac_real": 0.95+ }+ ],+ "per_group_A": [+ {+ "group": "ambience",+ "label": "real/pro",+ "n": 77,+ "mean_p_real": 0.78,+ "[email protected]": 0.974,+ "auc_vs_other_class": 0.9542+ },+ {+ "group": "cinetic",+ "label": "synth",+ "n": 318,+ "mean_p_real": 0.081,+ "[email protected]": 0.9686,+ "auc_vs_other_class": 0.9753+ },+ {+ "group": "film",+ "label": "real/pro",+ "n": 72,+ "mean_p_real": 0.75,+ "[email protected]": 0.9722,+ "auc_vs_other_class": 0.9449+ },+ {+ "group": "foley",+ "label": "real/pro",+ "n": 73,+ "mean_p_real": 0.869,+ "[email protected]": 0.9726,+ "auc_vs_other_class": 0.9721+ },+ {+ "group": "garden",+ "label": "real/pro",+ "n": 78,+ "mean_p_real": 0.835,+ "[email protected]": 0.9487,+ "auc_vs_other_class": 0.9572+ },+ {+ "group": "generic",+ "label": "synth",+ "n": 260,+ "mean_p_real": 0.329,+ "[email protected]": 0.6731,+ "auc_vs_other_class": 0.8355+ },+ {+ "group": "keyboard",+ "label": "real/pro",+ "n": 77,+ "mean_p_real": 0.969,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9925+ },+ {+ "group": "magic",+ "label": "real/pro",+ "n": 68,+ "mean_p_real": 0.451,+ "[email protected]": 0.4853,+ "auc_vs_other_class": 0.753+ },+ {+ "group": "motion",+ "label": "real/pro",+ "n": 66,+ "mean_p_real": 0.649,+ "[email protected]": 0.6667,+ "auc_vs_other_class": 0.8794+ },+ {+ "group": "music",+ "label": "real/pro",+ "n": 119,+ "mean_p_real": 0.595,+ "[email protected]": 0.6891,+ "auc_vs_other_class": 0.8867+ },+ {+ "group": "retired",+ "label": "synth",+ "n": 215,+ "mean_p_real": 0.166,+ "[email protected]": 0.9674,+ "auc_vs_other_class": 0.9474+ },+ {+ "group": "toy",+ "label": "real/pro",+ "n": 61,+ "mean_p_real": 0.838,+ "[email protected]": 0.9672,+ "auc_vs_other_class": 0.9627+ }+ ],+ "per_group_M": [+ {+ "group": "ambience",+ "label": "real/pro",+ "n": 77,+ "mean_p_real": 0.868,+ "[email protected]": 0.987,+ "auc_vs_other_class": 0.966+ },+ {+ "group": "cinetic",+ "label": "synth",+ "n": 318,+ "mean_p_real": 0.101,+ "[email protected]": 0.978,+ "auc_vs_other_class": 0.9537+ },+ {+ "group": "film",+ "label": "real/pro",+ "n": 72,+ "mean_p_real": 0.886,+ "[email protected]": 0.9722,+ "auc_vs_other_class": 0.9695+ },+ {+ "group": "foley",+ "label": "real/pro",+ "n": 73,+ "mean_p_real": 0.905,+ "[email protected]": 0.9726,+ "auc_vs_other_class": 0.9726+ },+ {+ "group": "garden",+ "label": "real/pro",+ "n": 78,+ "mean_p_real": 0.857,+ "[email protected]": 0.9103,+ "auc_vs_other_class": 0.9506+ },+ {+ "group": "generic",+ "label": "synth",+ "n": 260,+ "mean_p_real": 0.322,+ "[email protected]": 0.6846,+ "auc_vs_other_class": 0.8642+ },+ {+ "group": "keyboard",+ "label": "real/pro",+ "n": 77,+ "mean_p_real": 0.963,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.987+ },+ {+ "group": "magic",+ "label": "real/pro",+ "n": 68,+ "mean_p_real": 0.405,+ "[email protected]": 0.4559,+ "auc_vs_other_class": 0.6337+ },+ {+ "group": "motion",+ "label": "real/pro",+ "n": 66,+ "mean_p_real": 0.622,+ "[email protected]": 0.6667,+ "auc_vs_other_class": 0.8263+ },+ {+ "group": "music",+ "label": "real/pro",+ "n": 119,+ "mean_p_real": 0.859,+ "[email protected]": 0.9244,+ "auc_vs_other_class": 0.9569+ },+ {+ "group": "retired",+ "label": "synth",+ "n": 215,+ "mean_p_real": 0.157,+ "[email protected]": 0.9674,+ "auc_vs_other_class": 0.9374+ },+ {+ "group": "toy",+ "label": "real/pro",+ "n": 61,+ "mean_p_real": 0.878,+ "[email protected]": 0.9672,+ "auc_vs_other_class": 0.9649+ }+ ],+ "non_music": {+ "A": {+ "n": 1365,+ "n_real": 572,+ "auc": 0.9292,+ "[email protected]": 0.8747,+ "brier": 0.1002,+ "logloss": 0.3375+ },+ "M": {+ "n": 1365,+ "n_real": 572,+ "auc": 0.9123,+ "[email protected]": 0.8762,+ "brier": 0.1022,+ "logloss": 0.3683+ }+ },+ "per_type_A": [+ {+ "group": "cinetic:bass",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.025,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9974+ },+ {+ "group": "cinetic:blip",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.036,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9912+ },+ {+ "group": "cinetic:clap",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.016,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9984+ },+ {+ "group": "cinetic:crisp_tick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.045,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9894+ },+ {+ "group": "cinetic:drone",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.108,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9667+ },+ {+ "group": "cinetic:fm_bell",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.067,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.981+ },+ {+ "group": "cinetic:hat",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.141,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9569+ },+ {+ "group": "cinetic:key_click",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.069,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9838+ },+ {+ "group": "cinetic:keys",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.131,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9625+ },+ {+ "group": "cinetic:kick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.026,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9962+ },+ {+ "group": "cinetic:marker",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.016,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9988+ },+ {+ "group": "cinetic:mix",+ "label": "synth",+ "n": 16,+ "mean_p_real": 0.094,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9688+ },+ {+ "group": "cinetic:motif",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.012,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9999+ },+ {+ "group": "cinetic:pluck",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.007,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9996+ },+ {+ "group": "cinetic:riser",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.033,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9949+ },+ {+ "group": "cinetic:score",+ "label": "synth",+ "n": 42,+ "mean_p_real": 0.038,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9922+ },+ {+ "group": "cinetic:snap",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.132,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9547+ },+ {+ "group": "cinetic:sub_boom",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.04,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9907+ },+ {+ "group": "cinetic:suck",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.176,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.942+ },+ {+ "group": "cinetic:supersaw",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.004,+ "[email protected]": 1.0,+ "auc_vs_other_class": 1.0+ },+ {+ "group": "cinetic:swell",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.023,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9978+ },+ {+ "group": "cinetic:thock",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.092,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9713+ },+ {+ "group": "cinetic:tick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.155,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9381+ },+ {+ "group": "cinetic:tock",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.271,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9135+ },+ {+ "group": "cinetic:typing",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.022,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9978+ },+ {+ "group": "cinetic:ui_click",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.165,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9236+ },+ {+ "group": "cinetic:whoosh",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.363,+ "[email protected]": 0.7,+ "auc_vs_other_class": 0.8795+ },+ {+ "group": "cinetic:word_blip",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.097,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9726+ },+ {+ "group": "generic:beep",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.299,+ "[email protected]": 0.7143,+ "auc_vs_other_class": 0.83+ },+ {+ "group": "generic:chiptune",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.19,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.9426+ },+ {+ "group": "generic:chord",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.335,+ "[email protected]": 0.7143,+ "auc_vs_other_class": 0.8453+ },+ {+ "group": "generic:click",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.289,+ "[email protected]": 0.7143,+ "auc_vs_other_class": 0.9031+ },+ {+ "group": "generic:drums",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.009,+ "[email protected]": 1.0,+ "auc_vs_other_class": 1.0+ },+ {+ "group": "generic:dtmf",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.713,+ "[email protected]": 0.0714,+ "auc_vs_other_class": 0.6431+ },+ {+ "group": "generic:fm",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.276,+ "[email protected]": 0.7857,+ "auc_vs_other_class": 0.8704+ },+ {+ "group": "generic:impact",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.178,+ "[email protected]": 0.8571,+ "auc_vs_other_class": 0.9415+ },+ {+ "group": "generic:lfsr",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.354,+ "[email protected]": 0.7143,+ "auc_vs_other_class": 0.8778+ },+ {+ "group": "generic:mix",+ "label": "synth",+ "n": 36,+ "mean_p_real": 0.609,+ "[email protected]": 0.3611,+ "auc_vs_other_class": 0.624+ },+ {+ "group": "generic:pad",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.3,+ "[email protected]": 0.7857,+ "auc_vs_other_class": 0.8947+ },+ {+ "group": "generic:pluck",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.087,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.9295+ },+ {+ "group": "generic:rain",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.664,+ "[email protected]": 0.2143,+ "auc_vs_other_class": 0.6718+ },+ {+ "group": "generic:riser",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.043,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.99+ },+ {+ "group": "generic:siren",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.67,+ "[email protected]": 0.1429,+ "auc_vs_other_class": 0.6116+ },+ {+ "group": "generic:whoosh",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.103,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9687+ },+ {+ "group": "generic:wind",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.035,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9924+ },+ {+ "group": "retired:ambience",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.224,+ "[email protected]": 0.9333,+ "auc_vs_other_class": 0.9312+ },+ {+ "group": "retired:bed",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.285,+ "[email protected]": 0.9333,+ "auc_vs_other_class": 0.9142+ },+ {+ "group": "retired:melody",+ "label": "synth",+ "n": 12,+ "mean_p_real": 0.151,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.951+ },+ {+ "group": "retired:melody-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.174,+ "[email protected]": 0.96,+ "auc_vs_other_class": 0.9415+ },+ {+ "group": "retired:stem_arp",+ "label": "synth",+ "n": 12,+ "mean_p_real": 0.045,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9913+ },+ {+ "group": "retired:stem_bass",+ "label": "synth",+ "n": 13,+ "mean_p_real": 0.059,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9851+ },+ {+ "group": "retired:stem_bed",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.292,+ "[email protected]": 0.9333,+ "auc_vs_other_class": 0.9097+ },+ {+ "group": "retired:stem_bells",+ "label": "synth",+ "n": 2,+ "mean_p_real": 0.096,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9696+ },+ {+ "group": "retired:stem_bells-event",+ "label": "synth",+ "n": 1,+ "mean_p_real": 0.028,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9971+ },+ {+ "group": "retired:stem_drums",+ "label": "synth",+ "n": 11,+ "mean_p_real": 0.097,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9658+ },+ {+ "group": "retired:stem_drums-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.089,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9719+ },+ {+ "group": "retired:stem_pad",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.045,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9909+ },+ {+ "group": "retired:stem_reverb",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.239,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9274+ },+ {+ "group": "retired:stem_sfx",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.241,+ "[email protected]": 0.9333,+ "auc_vs_other_class": 0.9223+ },+ {+ "group": "retired:stem_sfx-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.196,+ "[email protected]": 0.92,+ "auc_vs_other_class": 0.9293+ }+ ],+ "per_type_B": [+ {+ "group": "cinetic:bass",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.021,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9959+ },+ {+ "group": "cinetic:blip",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.035,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.988+ },+ {+ "group": "cinetic:clap",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.013,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9978+ },+ {+ "group": "cinetic:crisp_tick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.015,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9974+ },+ {+ "group": "cinetic:drone",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.144,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9616+ },+ {+ "group": "cinetic:fm_bell",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.118,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9715+ },+ {+ "group": "cinetic:hat",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.234,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9444+ },+ {+ "group": "cinetic:key_click",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.058,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9891+ },+ {+ "group": "cinetic:keys",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.203,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9585+ },+ {+ "group": "cinetic:kick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.066,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.978+ },+ {+ "group": "cinetic:marker",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.01,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.999+ },+ {+ "group": "cinetic:mix",+ "label": "synth",+ "n": 16,+ "mean_p_real": 0.298,+ "[email protected]": 0.875,+ "auc_vs_other_class": 0.9361+ },+ {+ "group": "cinetic:motif",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.016,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9968+ },+ {+ "group": "cinetic:pluck",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.012,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9959+ },+ {+ "group": "cinetic:riser",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.037,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9887+ },+ {+ "group": "cinetic:score",+ "label": "synth",+ "n": 42,+ "mean_p_real": 0.148,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9563+ },+ {+ "group": "cinetic:snap",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.246,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9466+ },+ {+ "group": "cinetic:sub_boom",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.083,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9735+ },+ {+ "group": "cinetic:suck",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.184,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9489+ },+ {+ "group": "cinetic:supersaw",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.004,+ "[email protected]": 1.0,+ "auc_vs_other_class": 1.0+ },+ {+ "group": "cinetic:swell",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.027,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9929+ },+ {+ "group": "cinetic:thock",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.078,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9848+ },+ {+ "group": "cinetic:tick",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.162,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9664+ },+ {+ "group": "cinetic:tock",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.188,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.952+ },+ {+ "group": "cinetic:typing",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.035,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9896+ },+ {+ "group": "cinetic:ui_click",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.167,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9656+ },+ {+ "group": "cinetic:whoosh",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.502,+ "[email protected]": 0.5,+ "auc_vs_other_class": 0.8935+ },+ {+ "group": "cinetic:word_blip",+ "label": "synth",+ "n": 10,+ "mean_p_real": 0.081,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9797+ },+ {+ "group": "generic:beep",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.126,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.9686+ },+ {+ "group": "generic:chiptune",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.019,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9959+ },+ {+ "group": "generic:chord",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.177,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.9559+ },+ {+ "group": "generic:click",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.2,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.946+ },+ {+ "group": "generic:drums",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.027,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9938+ },+ {+ "group": "generic:dtmf",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.384,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9177+ },+ {+ "group": "generic:fm",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.155,+ "[email protected]": 0.8571,+ "auc_vs_other_class": 0.9657+ },+ {+ "group": "generic:impact",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.19,+ "[email protected]": 0.8571,+ "auc_vs_other_class": 0.9513+ },+ {+ "group": "generic:lfsr",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.079,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9765+ },+ {+ "group": "generic:mix",+ "label": "synth",+ "n": 36,+ "mean_p_real": 0.588,+ "[email protected]": 0.3889,+ "auc_vs_other_class": 0.7929+ },+ {+ "group": "generic:pad",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.132,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.966+ },+ {+ "group": "generic:pluck",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.077,+ "[email protected]": 0.9286,+ "auc_vs_other_class": 0.9799+ },+ {+ "group": "generic:rain",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.61,+ "[email protected]": 0.2143,+ "auc_vs_other_class": 0.8701+ },+ {+ "group": "generic:riser",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.039,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.989+ },+ {+ "group": "generic:siren",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.177,+ "[email protected]": 0.8571,+ "auc_vs_other_class": 0.9567+ },+ {+ "group": "generic:whoosh",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.082,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9741+ },+ {+ "group": "generic:wind",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.05,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9846+ },+ {+ "group": "retired:ambience",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.216,+ "[email protected]": 0.8667,+ "auc_vs_other_class": 0.9517+ },+ {+ "group": "retired:bed",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.256,+ "[email protected]": 0.8667,+ "auc_vs_other_class": 0.9429+ },+ {+ "group": "retired:melody",+ "label": "synth",+ "n": 12,+ "mean_p_real": 0.1,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9708+ },+ {+ "group": "retired:melody-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.147,+ "[email protected]": 0.92,+ "auc_vs_other_class": 0.9629+ },+ {+ "group": "retired:stem_arp",+ "label": "synth",+ "n": 12,+ "mean_p_real": 0.011,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9987+ },+ {+ "group": "retired:stem_bass",+ "label": "synth",+ "n": 13,+ "mean_p_real": 0.027,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.993+ },+ {+ "group": "retired:stem_bed",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.265,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9419+ },+ {+ "group": "retired:stem_bells",+ "label": "synth",+ "n": 2,+ "mean_p_real": 0.049,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9819+ },+ {+ "group": "retired:stem_bells-event",+ "label": "synth",+ "n": 1,+ "mean_p_real": 0.004,+ "[email protected]": 1.0,+ "auc_vs_other_class": 1.0+ },+ {+ "group": "retired:stem_drums",+ "label": "synth",+ "n": 11,+ "mean_p_real": 0.056,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.98+ },+ {+ "group": "retired:stem_drums-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.084,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.977+ },+ {+ "group": "retired:stem_pad",+ "label": "synth",+ "n": 14,+ "mean_p_real": 0.014,+ "[email protected]": 1.0,+ "auc_vs_other_class": 0.9981+ },+ {+ "group": "retired:stem_reverb",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.3,+ "[email protected]": 0.7333,+ "auc_vs_other_class": 0.9363+ },+ {+ "group": "retired:stem_sfx",+ "label": "synth",+ "n": 15,+ "mean_p_real": 0.339,+ "[email protected]": 0.8,+ "auc_vs_other_class": 0.9216+ },+ {+ "group": "retired:stem_sfx-event",+ "label": "synth",+ "n": 25,+ "mean_p_real": 0.221,+ "[email protected]": 0.84,+ "auc_vs_other_class": 0.9377+ }+ ],+ "music_M": {+ "windows": 119,+ "mean_p_real": 0.859,+ "[email protected]": 0.9244,+ "files": 52,+ "[email protected]": 0.9615,+ "files_called_synth": 0,+ "files_called_real": 44,+ "scheme_A_mean_p_real": 0.595,+ "per_provider": [+ {+ "group": "music:ch",+ "label": "real/pro",+ "n": 30,+ "mean_p_real": 0.83,+ "[email protected]": 0.9,+ "auc_vs_other_class": 0.9407+ },+ {+ "group": "music:mk",+ "label": "real/pro",+ "n": 45,+ "mean_p_real": 0.887,+ "[email protected]": 0.9333,+ "auc_vs_other_class": 0.9672+ },+ {+ "group": "music:other",+ "label": "real/pro",+ "n": 8,+ "mean_p_real": 0.77,+ "[email protected]": 0.75,+ "auc_vs_other_class": 0.9232+ },+ {+ "group": "music:sb",+ "label": "real/pro",+ "n": 36,+ "mean_p_real": 0.867,+ "[email protected]": 0.9722,+ "auc_vs_other_class": 0.9651+ }+ ]+ },+ "retired_check": {+ "C: real vs retired": {+ "auc": 0.9475,+ "[email protected]": 0.862,+ "synth_mean_p": 0.166+ },+ "C: real recordings only (no music/films) vs retired": {+ "auc": 0.9418+ },+ "D: real vs retired": {+ "auc": 0.9085,+ "[email protected]": 0.8389,+ "synth_mean_p": 0.185+ },+ "D: real vs cinetic": {+ "auc": 0.921,+ "[email protected]": 0.8484,+ "synth_mean_p": 0.145+ },+ "D: real recordings only (no music/films) vs retired": {+ "auc": 0.9031+ }+ },+ "file_level_A": {+ "n": 119,+ "n_real": 96,+ "auc": 0.9022,+ "[email protected]": 0.8235,+ "brier": 0.1254,+ "logloss": 0.4123,+ "note": "sources with >= 2 windows (films, music, ambience, retired stems, synthesized mixes): mean of window p_real",+ "files": [+ {+ "src": "cinetic_score_2/soundtrack.wav",+ "label": 0,+ "p_real": 0.021,+ "windows": 7+ },+ {+ "src": "cinetic_score_0/soundtrack.wav",+ "label": 0,+ "p_real": 0.029,+ "windows": 7+ },+ {+ "src": "cinetic_score_1/soundtrack.wav",+ "label": 0,+ "p_real": 0.03,+ "windows": 7+ },+ {+ "src": "cinetic_score_5/soundtrack.wav",+ "label": 0,+ "p_real": 0.036,+ "windows": 7+ },+ {+ "src": "cinetic_score_4/soundtrack.wav",+ "label": 0,+ "p_real": 0.039,+ "windows": 7+ },+ {+ "src": "stems/arp.wav",+ "label": 0,+ "p_real": 0.045,+ "windows": 12+ },+ {+ "src": "stems/pad.wav",+ "label": 0,+ "p_real": 0.045,+ "windows": 14+ },+ {+ "src": "stems/bass.wav",+ "label": 0,+ "p_real": 0.059,+ "windows": 13+ },+ {+ "src": "cinetic_score_3/soundtrack.wav",+ "label": 0,+ "p_real": 0.07,+ "windows": 7+ },+ {+ "src": "stems/bells.wav",+ "label": 0,+ "p_real": 0.096,+ "windows": 2+ },+ {+ "src": "stems/drums.wav",+ "label": 0,+ "p_real": 0.097,+ "windows": 11+ },+ {+ "src": "audio/melody.wav",+ "label": 0,+ "p_real": 0.151,+ "windows": 12+ },+ {+ "src": "music/mk-1142_gummies.mp3",+ "label": 1,+ "p_real": 0.172,+ "windows": 2+ },+ {+ "src": "music/ch-59068_sakuragirl-yay.mp3",+ "label": 1,+ "p_real": 0.217,+ "windows": 4+ },+ {+ "src": "audio/ambience.wav",+ "label": 0,+ "p_real": 0.224,+ "windows": 15+ },+ {+ "src": "music/ch-25837_ahimitsu-adventures.mp3",+ "label": 1,+ "p_real": 0.231,+ "windows": 2+ },+ {+ "src": "stems/reverb.wav",+ "label": 0,+ "p_real": 0.239,+ "windows": 15+ },+ {+ "src": "stems/sfx.wav",+ "label": 0,+ "p_real": 0.241,+ "windows": 15+ },+ {+ "src": "_retired_synth/bed.wav",+ "label": 0,+ "p_real": 0.285,+ "windows": 15+ },+ {+ "src": "stems/bed.wav",+ "label": 0,+ "p_real": 0.292,+ "windows": 15+ },+ {+ "src": "music/kmac-enchanted-valley_celesta-orchestral.mp3",+ "label": 1,+ "p_real": 0.327,+ "windows": 2+ },+ {+ "src": "music/mk-727_pop-track-01.mp3",+ "label": 1,+ "p_real": 0.338,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_4.wav",+ "label": 0,+ "p_real": 0.408,+ "windows": 6+ },+ {+ "src": "music/mk-746_relaxation-02.mp3",+ "label": 1,+ "p_real": 0.425,+ "windows": 2+ },+ {+ "src": "music/sb-soar_family-adventure.mp3",+ "label": 1,+ "p_real": 0.431,+ "windows": 2+ },+ {+ "src": "music/sb-racethesun_hybrid-orchestral.mp3",+ "label": 1,+ "p_real": 0.445,+ "windows": 2+ },+ {+ "src": "music/ss-building-inspiration_orch-hybrid.mp3",+ "label": 1,+ "p_real": 0.456,+ "windows": 4+ },+ {+ "src": "music/ch-59608_sakuragirl-stars.mp3",+ "label": 1,+ "p_real": 0.456,+ "windows": 2+ },+ {+ "src": "music/mk-4_delightful.mp3",+ "label": 1,+ "p_real": 0.456,+ "windows": 2+ },+ {+ "src": "ambience/fs-639459_summer-meadow-wind.ogg",+ "label": 1,+ "p_real": 0.462,+ "windows": 2+ },+ {+ "src": "music/sb-discovery_hybrid-orchestral.mp3",+ "label": 1,+ "p_real": 0.462,+ "windows": 2+ },+ {+ "src": "music/sb-bornofthesky_bright-orchestral.mp3",+ "label": 1,+ "p_real": 0.465,+ "windows": 4+ },+ {+ "src": "music/mk-1037_my-little-star.mp3",+ "label": 1,+ "p_real": 0.483,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_1.wav",+ "label": 0,+ "p_real": 0.484,+ "windows": 6+ },+ {+ "src": "music/mk-506_little-bells.mp3",+ "label": 1,+ "p_real": 0.488,+ "windows": 2+ },+ {+ "src": "music/sb-quirkylane_whimsical-winds.mp3",+ "label": 1,+ "p_real": 0.493,+ "windows": 2+ },+ {+ "src": "music/sb-lifeinsilico_piano-strings.mp3",+ "label": 1,+ "p_real": 0.518,+ "windows": 2+ },+ {+ "src": "music/mk-175_digital-clouds.mp3",+ "label": 1,+ "p_real": 0.548,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_0.wav",+ "label": 0,+ "p_real": 0.549,+ "windows": 6+ },+ {+ "src": "music/sb-clearskies_uplifting-orchestral.mp3",+ "label": 1,+ "p_real": 0.557,+ "windows": 4+ },+ {+ "src": "music/ch-53234_roa-daybreak.mp3",+ "label": 1,+ "p_real": 0.563,+ "windows": 2+ },+ {+ "src": "music/sb-felicity_hopeful-piano-pop.mp3",+ "label": 1,+ "p_real": 0.57,+ "windows": 2+ },+ {+ "src": "music/mk-828_dont-worry.mp3",+ "label": 1,+ "p_real": 0.572,+ "windows": 4+ },+ {+ "src": "mixes/generic_mix_2.wav",+ "label": 0,+ "p_real": 0.59,+ "windows": 6+ },+ {+ "src": "music/ch-25942_keysofmoon-cheer-up.mp3",+ "label": 1,+ "p_real": 0.595,+ "windows": 2+ },+ {+ "src": "music/sb-echoesofhome_warm-orchestral.mp3",+ "label": 1,+ "p_real": 0.613,+ "windows": 2+ },+ {+ "src": "music/mk-32_driving-ambition.mp3",+ "label": 1,+ "p_real": 0.616,+ "windows": 2+ },+ {+ "src": "music/ch-38454_roa-bloom.mp3",+ "label": 1,+ "p_real": 0.617,+ "windows": 2+ },+ {+ "src": "music/ch-60852_roa-stay-a-little.mp3",+ "label": 1,+ "p_real": 0.625,+ "windows": 2+ },+ {+ "src": "music/ch-58303_roa-delighted.mp3",+ "label": 1,+ "p_real": 0.634,+ "windows": 2+ },+ {+ "src": "music/sb-bringmethesky_uplifting-orchestral.mp3",+ "label": 1,+ "p_real": 0.635,+ "windows": 4+ },+ {+ "src": "music/ch-60528_roa-dream-up.mp3",+ "label": 1,+ "p_real": 0.636,+ "windows": 2+ },+ {+ "src": "refs/ref_5.mp4",+ "label": 1,+ "p_real": 0.659,+ "windows": 7+ },+ {+ "src": "ambience/fs-331999_reed-grass-wind-close.ogg",+ "label": 1,+ "p_real": 0.672,+ "windows": 2+ },+ {+ "src": "ambience/fs-511758_early-morning-birds-countryside.ogg",+ "label": 1,+ "p_real": 0.684,+ "windows": 2+ },+ {+ "src": "ambience/mk-2464_morning-garden.wav",+ "label": 1,+ "p_real": 0.684,+ "windows": 2+ },+ {+ "src": "music/mk-158_happy-times.mp3",+ "label": 1,+ "p_real": 0.684,+ "windows": 2+ },+ {+ "src": "music/mk-1026_el-embustero.mp3",+ "label": 1,+ "p_real": 0.689,+ "windows": 2+ },+ {+ "src": "music/sb-tomorrow_quirky-orchestral.mp3",+ "label": 1,+ "p_real": 0.689,+ "windows": 2+ },+ {+ "src": "ambience/mk-2472_morning-birds.wav",+ "label": 1,+ "p_real": 0.698,+ "windows": 2+ },+ {+ "src": "ambience/fs-466242_dawn-chorus-hampshire.ogg",+ "label": 1,+ "p_real": 0.699,+ "windows": 2+ },+ {+ "src": "music/sb-icecream_childlike-piano.mp3",+ "label": 1,+ "p_real": 0.699,+ "windows": 4+ },+ {+ "src": "ambience/fs-409143_summer-meadow-donegal.ogg",+ "label": 1,+ "p_real": 0.701,+ "windows": 2+ },+ {+ "src": "refs/ref_1.mp4",+ "label": 1,+ "p_real": 0.704,+ "windows": 23+ },+ {+ "src": "refs/ref_2.mp4",+ "label": 1,+ "p_real": 0.707,+ "windows": 7+ },+ {+ "src": "mixes/generic_mix_3.wav",+ "label": 0,+ "p_real": 0.707,+ "windows": 6+ },+ {+ "src": "ambience/fs-346533_birds-spring-woods.ogg",+ "label": 1,+ "p_real": 0.709,+ "windows": 2+ },+ {+ "src": "music/mk-535_happy-spring.mp3",+ "label": 1,+ "p_real": 0.715,+ "windows": 2+ },+ {+ "src": "ambience/fs-677535_woodlark-field-spring.ogg",+ "label": 1,+ "p_real": 0.716,+ "windows": 2+ },+ {+ "src": "ambience/fs-565058_blackbird-garden-rustling-trees.ogg",+ "label": 1,+ "p_real": 0.72,+ "windows": 2+ },+ {+ "src": "music/mk-595_secret-garden.mp3",+ "label": 1,+ "p_real": 0.724,+ "windows": 2+ },+ {+ "src": "music/mk-855_pop-parapop.mp3",+ "label": 1,+ "p_real": 0.725,+ "windows": 2+ },+ {+ "src": "ambience/mk-2474_light-rain-atmosphere.wav",+ "label": 1,+ "p_real": 0.73,+ "windows": 2+ },+ {+ "src": "music/ch-61388_sakuragirl-sunflower.mp3",+ "label": 1,+ "p_real": 0.73,+ "windows": 2+ },+ {+ "src": "ambience/fs-271229_skylark-meadow-calm.ogg",+ "label": 1,+ "p_real": 0.735,+ "windows": 2+ },+ {+ "src": "ambience/fs-81819_rain-window-reverb-room.ogg",+ "label": 1,+ "p_real": 0.738,+ "windows": 2+ },+ {+ "src": "music/kmac-wholesome_cello-marimba.mp3",+ "label": 1,+ "p_real": 0.74,+ "windows": 2+ },+ {+ "src": "refs/ref_4.mp4",+ "label": 1,+ "p_real": 0.746,+ "windows": 8+ },+ {+ "src": "ambience/fs-751473_soft-wind-leaves-rustle.ogg",+ "label": 1,+ "p_real": 0.75,+ "windows": 2+ },+ {+ "src": "ambience/fs-440807_meadow-birds-nt3.ogg",+ "label": 1,+ "p_real": 0.753,+ "windows": 2+ },+ {+ "src": "refs/ref_6.mp4",+ "label": 1,+ "p_real": 0.756,+ "windows": 12+ },+ {+ "src": "music/mk-12_playground-fun.mp3",+ "label": 1,+ "p_real": 0.756,+ "windows": 2+ },+ {+ "src": "ambience/mk-1249_light-rain-looping.wav",+ "label": 1,+ "p_real": 0.766,+ "windows": 2+ },+ {+ "src": "music/sb-highgames_bouncy-comedy.mp3",+ "label": 1,+ "p_real": 0.768,+ "windows": 2+ },+ {+ "src": "music/sb-jubilee_buoyant-adventure.mp3",+ "label": 1,+ "p_real": 0.77,+ "windows": 2+ },+ {+ "src": "music/mk-31_dreaming-big.mp3",+ "label": 1,+ "p_real": 0.771,+ "windows": 2+ },+ {+ "src": "music/mk-1146_im-not-the-only-one.mp3",+ "label": 1,+ "p_real": 0.772,+ "windows": 2+ },+ {+ "src": "ambience/mk-2467_morning-birds-singing.wav",+ "label": 1,+ "p_real": 0.774,+ "windows": 2+ },+ {+ "src": "ambience/fs-333510_light-rain-skylight.ogg",+ "label": 1,+ "p_real": 0.777,+ "windows": 2+ },+ {+ "src": "ambience/fs-215414_wind-tall-grass-prairie.ogg",+ "label": 1,+ "p_real": 0.783,+ "windows": 2+ },+ {+ "src": "music/mk-587_discover.mp3",+ "label": 1,+ "p_real": 0.789,+ "windows": 2+ },+ {+ "src": "ambience/fs-669486_rain-window-interior.ogg",+ "label": 1,+ "p_real": 0.79,+ "windows": 2+ },+ {+ "src": "music/ch-58955_roa-aurora.mp3",+ "label": 1,+ "p_real": 0.792,+ "windows": 2+ },+ {+ "src": "ambience/fs-788338_rain-inside-apartment.ogg",+ "label": 1,+ "p_real": 0.793,+ "windows": 2+ },+ {+ "src": "ambience/fs-181801_breeze-pines.ogg",+ "label": 1,+ "p_real": 0.794,+ "windows": 2+ },+ {+ "src": "ambience/fs-847380_spring-meadow-birds-soft-wind.ogg",+ "label": 1,+ "p_real": 0.794,+ "windows": 2+ },+ {+ "src": "music/ch-58136_sakuragirl-daisy.mp3",+ "label": 1,+ "p_real": 0.797,+ "windows": 4+ },+ {+ "src": "ambience/fs-539750_raindrops-window-long.ogg",+ "label": 1,+ "p_real": 0.809,+ "windows": 2+ },+ {+ "src": "ambience/fs-575645_large-leaves-rustle-wind.ogg",+ "label": 1,+ "p_real": 0.814,+ "windows": 2+ },+ {+ "src": "ambience/fs-455828_roomtone-computer-fans.ogg",+ "label": 1,+ "p_real": 0.823,+ "windows": 2+ },+ {+ "src": "ambience/fs-346642_rain-on-windows-interior.ogg",+ "label": 1,+ "p_real": 0.831,+ "windows": 2+ },+ {+ "src": "ambience/fs-869851_rain-hitting-window.ogg",+ "label": 1,+ "p_real": 0.831,+ "windows": 2+ },+ {+ "src": "ambience/fs-349044_meadow-forest-spring.ogg",+ "label": 1,+ "p_real": 0.846,+ "windows": 2+ },+ {+ "src": "ambience/fs-502692_rain-bedroom-indoors.ogg",+ "label": 1,+ "p_real": 0.847,+ "windows": 2+ },+ {+ "src": "ambience/fs-243781_rain-against-window-intense.ogg",+ "label": 1,+ "p_real": 0.849,+ "windows": 2+ },+ {+ "src": "music/mk-2_comical.mp3",+ "label": 1,+ "p_real": 0.853,+ "windows": 2+ },+ {+ "src": "ambience/fs-360756_raindrops-window-night.ogg",+ "label": 1,+ "p_real": 0.861,+ "windows": 2+ },+ {+ "src": "ambience/fs-456766_birds-field-light-breeze.ogg",+ "label": 1,+ "p_real": 0.87,+ "windows": 2+ },+ {+ "src": "refs/ref_3.mp4",+ "label": 1,+ "p_real": 0.871,+ "windows": 5+ },+ {+ "src": "music/ch-61389_sakuragirl-lemonade.mp3",+ "label": 1,+ "p_real": 0.871,+ "windows": 2+ },+ {+ "src": "ambience/fs-664152_roomtone-basement-hifi.ogg",+ "label": 1,+ "p_real": 0.874,+ "windows": 2+ },+ {+ "src": "ambience/fs-637808_roomtone-small-quiet.ogg",+ "label": 1,+ "p_real": 0.874,+ "windows": 2+ },+ {+ "src": "music/mk-6_fun-and-games.mp3",+ "label": 1,+ "p_real": 0.875,+ "windows": 2+ },+ {+ "src": "music/mk-10_playful.mp3",+ "label": 1,+ "p_real": 0.877,+ "windows": 2+ },+ {+ "src": "refs/ref_7.mp4",+ "label": 1,+ "p_real": 0.888,+ "windows": 10+ },+ {+ "src": "ambience/fs-85145_rain-on-my-window.ogg",+ "label": 1,+ "p_real": 0.888,+ "windows": 2+ },+ {+ "src": "ambience/fs-381258_computer-hum-gentle.ogg",+ "label": 1,+ "p_real": 0.907,+ "windows": 2+ },+ {+ "src": "ambience/fs-543757_computer-hum.ogg",+ "label": 1,+ "p_real": 0.907,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_5.wav",+ "label": 0,+ "p_real": 0.915,+ "windows": 6+ }+ ]+ },+ "file_level_M": {+ "n": 119,+ "n_real": 96,+ "auc": 0.9538,+ "[email protected]": 0.9244,+ "brier": 0.0527,+ "logloss": 0.2148,+ "note": "sources with >= 2 windows (films, music, ambience, retired stems, synthesized mixes): mean of window p_real",+ "files": [+ {+ "src": "stems/bass.wav",+ "label": 0,+ "p_real": 0.053,+ "windows": 13+ },+ {+ "src": "stems/arp.wav",+ "label": 0,+ "p_real": 0.069,+ "windows": 12+ },+ {+ "src": "stems/pad.wav",+ "label": 0,+ "p_real": 0.072,+ "windows": 14+ },+ {+ "src": "cinetic_score_2/soundtrack.wav",+ "label": 0,+ "p_real": 0.081,+ "windows": 7+ },+ {+ "src": "stems/bells.wav",+ "label": 0,+ "p_real": 0.086,+ "windows": 2+ },+ {+ "src": "stems/drums.wav",+ "label": 0,+ "p_real": 0.089,+ "windows": 11+ },+ {+ "src": "cinetic_score_1/soundtrack.wav",+ "label": 0,+ "p_real": 0.105,+ "windows": 7+ },+ {+ "src": "cinetic_score_0/soundtrack.wav",+ "label": 0,+ "p_real": 0.106,+ "windows": 7+ },+ {+ "src": "cinetic_score_4/soundtrack.wav",+ "label": 0,+ "p_real": 0.129,+ "windows": 7+ },+ {+ "src": "cinetic_score_5/soundtrack.wav",+ "label": 0,+ "p_real": 0.13,+ "windows": 7+ },+ {+ "src": "audio/melody.wav",+ "label": 0,+ "p_real": 0.131,+ "windows": 12+ },+ {+ "src": "audio/ambience.wav",+ "label": 0,+ "p_real": 0.191,+ "windows": 15+ },+ {+ "src": "cinetic_score_3/soundtrack.wav",+ "label": 0,+ "p_real": 0.197,+ "windows": 7+ },+ {+ "src": "stems/sfx.wav",+ "label": 0,+ "p_real": 0.207,+ "windows": 15+ },+ {+ "src": "_retired_synth/bed.wav",+ "label": 0,+ "p_real": 0.254,+ "windows": 15+ },+ {+ "src": "stems/bed.wav",+ "label": 0,+ "p_real": 0.256,+ "windows": 15+ },+ {+ "src": "stems/reverb.wav",+ "label": 0,+ "p_real": 0.316,+ "windows": 15+ },+ {+ "src": "music/ch-59068_sakuragirl-yay.mp3",+ "label": 1,+ "p_real": 0.401,+ "windows": 4+ },+ {+ "src": "music/mk-1142_gummies.mp3",+ "label": 1,+ "p_real": 0.435,+ "windows": 2+ },+ {+ "src": "ambience/fs-639459_summer-meadow-wind.ogg",+ "label": 1,+ "p_real": 0.499,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_4.wav",+ "label": 0,+ "p_real": 0.513,+ "windows": 6+ },+ {+ "src": "mixes/generic_mix_0.wav",+ "label": 0,+ "p_real": 0.561,+ "windows": 6+ },+ {+ "src": "mixes/generic_mix_1.wav",+ "label": 0,+ "p_real": 0.614,+ "windows": 6+ },+ {+ "src": "music/ch-25837_ahimitsu-adventures.mp3",+ "label": 1,+ "p_real": 0.62,+ "windows": 2+ },+ {+ "src": "music/mk-727_pop-track-01.mp3",+ "label": 1,+ "p_real": 0.654,+ "windows": 2+ },+ {+ "src": "music/mk-4_delightful.mp3",+ "label": 1,+ "p_real": 0.656,+ "windows": 2+ },+ {+ "src": "music/kmac-enchanted-valley_celesta-orchestral.mp3",+ "label": 1,+ "p_real": 0.659,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_2.wav",+ "label": 0,+ "p_real": 0.662,+ "windows": 6+ },+ {+ "src": "refs/ref_5.mp4",+ "label": 1,+ "p_real": 0.672,+ "windows": 7+ },+ {+ "src": "ambience/fs-331999_reed-grass-wind-close.ogg",+ "label": 1,+ "p_real": 0.684,+ "windows": 2+ },+ {+ "src": "music/ss-building-inspiration_orch-hybrid.mp3",+ "label": 1,+ "p_real": 0.717,+ "windows": 4+ },+ {+ "src": "music/mk-506_little-bells.mp3",+ "label": 1,+ "p_real": 0.72,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_3.wav",+ "label": 0,+ "p_real": 0.769,+ "windows": 6+ },+ {+ "src": "music/sb-quirkylane_whimsical-winds.mp3",+ "label": 1,+ "p_real": 0.781,+ "windows": 2+ },+ {+ "src": "music/sb-soar_family-adventure.mp3",+ "label": 1,+ "p_real": 0.783,+ "windows": 2+ },+ {+ "src": "ambience/mk-2464_morning-garden.wav",+ "label": 1,+ "p_real": 0.786,+ "windows": 2+ },+ {+ "src": "ambience/fs-409143_summer-meadow-donegal.ogg",+ "label": 1,+ "p_real": 0.792,+ "windows": 2+ },+ {+ "src": "music/sb-bornofthesky_bright-orchestral.mp3",+ "label": 1,+ "p_real": 0.8,+ "windows": 4+ },+ {+ "src": "music/sb-racethesun_hybrid-orchestral.mp3",+ "label": 1,+ "p_real": 0.803,+ "windows": 2+ },+ {+ "src": "ambience/mk-2472_morning-birds.wav",+ "label": 1,+ "p_real": 0.813,+ "windows": 2+ },+ {+ "src": "ambience/mk-2474_light-rain-atmosphere.wav",+ "label": 1,+ "p_real": 0.817,+ "windows": 2+ },+ {+ "src": "ambience/fs-215414_wind-tall-grass-prairie.ogg",+ "label": 1,+ "p_real": 0.822,+ "windows": 2+ },+ {+ "src": "music/ch-59608_sakuragirl-stars.mp3",+ "label": 1,+ "p_real": 0.826,+ "windows": 2+ },+ {+ "src": "ambience/fs-271229_skylark-meadow-calm.ogg",+ "label": 1,+ "p_real": 0.827,+ "windows": 2+ },+ {+ "src": "ambience/fs-751473_soft-wind-leaves-rustle.ogg",+ "label": 1,+ "p_real": 0.828,+ "windows": 2+ },+ {+ "src": "music/sb-clearskies_uplifting-orchestral.mp3",+ "label": 1,+ "p_real": 0.831,+ "windows": 4+ },+ {+ "src": "ambience/fs-511758_early-morning-birds-countryside.ogg",+ "label": 1,+ "p_real": 0.834,+ "windows": 2+ },+ {+ "src": "ambience/fs-81819_rain-window-reverb-room.ogg",+ "label": 1,+ "p_real": 0.837,+ "windows": 2+ },+ {+ "src": "ambience/fs-669486_rain-window-interior.ogg",+ "label": 1,+ "p_real": 0.838,+ "windows": 2+ },+ {+ "src": "ambience/fs-181801_breeze-pines.ogg",+ "label": 1,+ "p_real": 0.84,+ "windows": 2+ },+ {+ "src": "music/mk-175_digital-clouds.mp3",+ "label": 1,+ "p_real": 0.84,+ "windows": 2+ },+ {+ "src": "music/sb-jubilee_buoyant-adventure.mp3",+ "label": 1,+ "p_real": 0.842,+ "windows": 2+ },+ {+ "src": "refs/ref_2.mp4",+ "label": 1,+ "p_real": 0.843,+ "windows": 7+ },+ {+ "src": "ambience/fs-440807_meadow-birds-nt3.ogg",+ "label": 1,+ "p_real": 0.844,+ "windows": 2+ },+ {+ "src": "music/ch-53234_roa-daybreak.mp3",+ "label": 1,+ "p_real": 0.845,+ "windows": 2+ },+ {+ "src": "ambience/fs-677535_woodlark-field-spring.ogg",+ "label": 1,+ "p_real": 0.852,+ "windows": 2+ },+ {+ "src": "ambience/mk-1249_light-rain-looping.wav",+ "label": 1,+ "p_real": 0.863,+ "windows": 2+ },+ {+ "src": "ambience/fs-346533_birds-spring-woods.ogg",+ "label": 1,+ "p_real": 0.868,+ "windows": 2+ },+ {+ "src": "music/sb-discovery_hybrid-orchestral.mp3",+ "label": 1,+ "p_real": 0.872,+ "windows": 2+ },+ {+ "src": "ambience/fs-333510_light-rain-skylight.ogg",+ "label": 1,+ "p_real": 0.875,+ "windows": 2+ },+ {+ "src": "ambience/fs-539750_raindrops-window-long.ogg",+ "label": 1,+ "p_real": 0.876,+ "windows": 2+ },+ {+ "src": "music/mk-1037_my-little-star.mp3",+ "label": 1,+ "p_real": 0.876,+ "windows": 2+ },+ {+ "src": "refs/ref_1.mp4",+ "label": 1,+ "p_real": 0.877,+ "windows": 23+ },+ {+ "src": "music/ch-25942_keysofmoon-cheer-up.mp3",+ "label": 1,+ "p_real": 0.877,+ "windows": 2+ },+ {+ "src": "ambience/fs-466242_dawn-chorus-hampshire.ogg",+ "label": 1,+ "p_real": 0.879,+ "windows": 2+ },+ {+ "src": "ambience/fs-565058_blackbird-garden-rustling-trees.ogg",+ "label": 1,+ "p_real": 0.889,+ "windows": 2+ },+ {+ "src": "music/sb-felicity_hopeful-piano-pop.mp3",+ "label": 1,+ "p_real": 0.889,+ "windows": 2+ },+ {+ "src": "ambience/fs-788338_rain-inside-apartment.ogg",+ "label": 1,+ "p_real": 0.89,+ "windows": 2+ },+ {+ "src": "music/ch-60852_roa-stay-a-little.mp3",+ "label": 1,+ "p_real": 0.893,+ "windows": 2+ },+ {+ "src": "ambience/fs-847380_spring-meadow-birds-soft-wind.ogg",+ "label": 1,+ "p_real": 0.894,+ "windows": 2+ },+ {+ "src": "refs/ref_6.mp4",+ "label": 1,+ "p_real": 0.9,+ "windows": 12+ },+ {+ "src": "music/mk-828_dont-worry.mp3",+ "label": 1,+ "p_real": 0.9,+ "windows": 4+ },+ {+ "src": "music/sb-echoesofhome_warm-orchestral.mp3",+ "label": 1,+ "p_real": 0.9,+ "windows": 2+ },+ {+ "src": "ambience/fs-455828_roomtone-computer-fans.ogg",+ "label": 1,+ "p_real": 0.902,+ "windows": 2+ },+ {+ "src": "music/ch-61388_sakuragirl-sunflower.mp3",+ "label": 1,+ "p_real": 0.903,+ "windows": 2+ },+ {+ "src": "ambience/fs-575645_large-leaves-rustle-wind.ogg",+ "label": 1,+ "p_real": 0.904,+ "windows": 2+ },+ {+ "src": "music/ch-38454_roa-bloom.mp3",+ "label": 1,+ "p_real": 0.904,+ "windows": 2+ },+ {+ "src": "music/sb-bringmethesky_uplifting-orchestral.mp3",+ "label": 1,+ "p_real": 0.906,+ "windows": 4+ },+ {+ "src": "ambience/mk-2467_morning-birds-singing.wav",+ "label": 1,+ "p_real": 0.906,+ "windows": 2+ },+ {+ "src": "ambience/fs-349044_meadow-forest-spring.ogg",+ "label": 1,+ "p_real": 0.907,+ "windows": 2+ },+ {+ "src": "ambience/fs-869851_rain-hitting-window.ogg",+ "label": 1,+ "p_real": 0.914,+ "windows": 2+ },+ {+ "src": "music/sb-icecream_childlike-piano.mp3",+ "label": 1,+ "p_real": 0.915,+ "windows": 4+ },+ {+ "src": "ambience/fs-346642_rain-on-windows-interior.ogg",+ "label": 1,+ "p_real": 0.92,+ "windows": 2+ },+ {+ "src": "ambience/fs-360756_raindrops-window-night.ogg",+ "label": 1,+ "p_real": 0.922,+ "windows": 2+ },+ {+ "src": "ambience/fs-664152_roomtone-basement-hifi.ogg",+ "label": 1,+ "p_real": 0.932,+ "windows": 2+ },+ {+ "src": "music/sb-highgames_bouncy-comedy.mp3",+ "label": 1,+ "p_real": 0.932,+ "windows": 2+ },+ {+ "src": "music/mk-746_relaxation-02.mp3",+ "label": 1,+ "p_real": 0.933,+ "windows": 2+ },+ {+ "src": "refs/ref_4.mp4",+ "label": 1,+ "p_real": 0.934,+ "windows": 8+ },+ {+ "src": "ambience/fs-243781_rain-against-window-intense.ogg",+ "label": 1,+ "p_real": 0.935,+ "windows": 2+ },+ {+ "src": "ambience/fs-637808_roomtone-small-quiet.ogg",+ "label": 1,+ "p_real": 0.936,+ "windows": 2+ },+ {+ "src": "music/sb-lifeinsilico_piano-strings.mp3",+ "label": 1,+ "p_real": 0.937,+ "windows": 2+ },+ {+ "src": "music/mk-1026_el-embustero.mp3",+ "label": 1,+ "p_real": 0.94,+ "windows": 2+ },+ {+ "src": "music/mk-12_playground-fun.mp3",+ "label": 1,+ "p_real": 0.942,+ "windows": 2+ },+ {+ "src": "music/mk-32_driving-ambition.mp3",+ "label": 1,+ "p_real": 0.942,+ "windows": 2+ },+ {+ "src": "ambience/fs-502692_rain-bedroom-indoors.ogg",+ "label": 1,+ "p_real": 0.943,+ "windows": 2+ },+ {+ "src": "ambience/fs-85145_rain-on-my-window.ogg",+ "label": 1,+ "p_real": 0.944,+ "windows": 2+ },+ {+ "src": "music/mk-158_happy-times.mp3",+ "label": 1,+ "p_real": 0.946,+ "windows": 2+ },+ {+ "src": "music/ch-60528_roa-dream-up.mp3",+ "label": 1,+ "p_real": 0.953,+ "windows": 2+ },+ {+ "src": "mixes/generic_mix_5.wav",+ "label": 0,+ "p_real": 0.953,+ "windows": 6+ },+ {+ "src": "music/ch-58303_roa-delighted.mp3",+ "label": 1,+ "p_real": 0.955,+ "windows": 2+ },+ {+ "src": "music/mk-855_pop-parapop.mp3",+ "label": 1,+ "p_real": 0.957,+ "windows": 2+ },+ {+ "src": "music/sb-tomorrow_quirky-orchestral.mp3",+ "label": 1,+ "p_real": 0.958,+ "windows": 2+ },+ {+ "src": "ambience/fs-456766_birds-field-light-breeze.ogg",+ "label": 1,+ "p_real": 0.962,+ "windows": 2+ },+ {+ "src": "refs/ref_3.mp4",+ "label": 1,+ "p_real": 0.963,+ "windows": 5+ },+ {+ "src": "ambience/fs-381258_computer-hum-gentle.ogg",+ "label": 1,+ "p_real": 0.963,+ "windows": 2+ },+ {+ "src": "music/mk-535_happy-spring.mp3",+ "label": 1,+ "p_real": 0.966,+ "windows": 2+ },+ {+ "src": "music/mk-31_dreaming-big.mp3",+ "label": 1,+ "p_real": 0.967,+ "windows": 2+ },+ {+ "src": "music/ch-58136_sakuragirl-daisy.mp3",+ "label": 1,+ "p_real": 0.969,+ "windows": 4+ },+ {+ "src": "music/ch-61389_sakuragirl-lemonade.mp3",+ "label": 1,+ "p_real": 0.969,+ "windows": 2+ },+ {+ "src": "music/ch-58955_roa-aurora.mp3",+ "label": 1,+ "p_real": 0.971,+ "windows": 2+ },+ {+ "src": "ambience/fs-543757_computer-hum.ogg",+ "label": 1,+ "p_real": 0.972,+ "windows": 2+ },+ {+ "src": "music/mk-587_discover.mp3",+ "label": 1,+ "p_real": 0.973,+ "windows": 2+ },+ {+ "src": "music/mk-1146_im-not-the-only-one.mp3",+ "label": 1,+ "p_real": 0.974,+ "windows": 2+ },+ {+ "src": "music/mk-595_secret-garden.mp3",+ "label": 1,+ "p_real": 0.984,+ "windows": 2+ },+ {+ "src": "music/mk-10_playful.mp3",+ "label": 1,+ "p_real": 0.985,+ "windows": 2+ },+ {+ "src": "music/kmac-wholesome_cello-marimba.mp3",+ "label": 1,+ "p_real": 0.987,+ "windows": 2+ },+ {+ "src": "music/mk-2_comical.mp3",+ "label": 1,+ "p_real": 0.987,+ "windows": 2+ },+ {+ "src": "music/mk-6_fun-and-games.mp3",+ "label": 1,+ "p_real": 0.987,+ "windows": 2+ },+ {+ "src": "refs/ref_7.mp4",+ "label": 1,+ "p_real": 0.988,+ "windows": 10+ }+ ],+ "thresholds": {+ "hi": 0.77,+ "lo": 0.4,+ "files": 119,+ "synth_files": 23,+ "how": "lo: 92.5% precision on held-out long files (scheme M); hi: max(that, the clip hi)"+ },+ "band": {+ "coverage": 0.8655,+ "precision_real": 0.9884,+ "precision_synth": 1.0,+ "real_called_synth": 0.0,+ "synth_called_real": 0.0435,+ "uncertain_frac": 0.1345+ },+ "calibration": [+ {+ "bin": "0.0-0.2",+ "n": 13,+ "mean_p": 0.111,+ "frac_real": 0.0+ },+ {+ "bin": "0.2-0.4",+ "n": 4,+ "mean_p": 0.258,+ "frac_real": 0.0+ },+ {+ "bin": "0.4-0.6",+ "n": 5,+ "mean_p": 0.482,+ "frac_real": 0.6+ },+ {+ "bin": "0.6-0.8",+ "n": 15,+ "mean_p": 0.705,+ "frac_real": 0.8+ },+ {+ "bin": "0.8-1.0",+ "n": 82,+ "mean_p": 0.907,+ "frac_real": 0.988+ }+ ]+ },+ "robustness_A": [+ {+ "perturbation": "noise70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8,+ "acc_unperturbed": 0.9304,+ "mean_p_real": 0.281,+ "mean_p_unperturbed": 0.154+ },+ {+ "perturbation": "room70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.7391,+ "acc_unperturbed": 0.8609,+ "mean_p_real": 0.313,+ "mean_p_unperturbed": 0.172+ },+ {+ "perturbation": "gated",+ "label": "real/pro",+ "n": 106,+ "[email protected]": 0.7453,+ "acc_unperturbed": 0.8113,+ "mean_p_real": 0.656,+ "mean_p_unperturbed": 0.715+ },+ {+ "perturbation": "short",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8957,+ "acc_unperturbed": 0.9043,+ "mean_p_real": 0.156,+ "mean_p_unperturbed": 0.145+ },+ {+ "perturbation": "short",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.7383,+ "acc_unperturbed": 0.8972,+ "mean_p_real": 0.675,+ "mean_p_unperturbed": 0.772+ },+ {+ "perturbation": "mp3",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8609,+ "acc_unperturbed": 0.887,+ "mean_p_real": 0.21,+ "mean_p_unperturbed": 0.159+ },+ {+ "perturbation": "mp3",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.8505,+ "acc_unperturbed": 0.8505,+ "mean_p_real": 0.744,+ "mean_p_unperturbed": 0.754+ }+ ],+ "level_sensitivity": {+ "real": {+ "clips": 34,+ "rms_db_median": -30.9,+ "lifted_clips": 18,+ "mean_p_orig": 0.887,+ "mean_p_-20dB": 0.867,+ "mean_p_norm-25": 0.86,+ "mean_p_lifted": 0.876,+ "mean_p_at-56": 0.864,+ "mean_p_at-56_lifted": 0.863,+ "acc_orig": 0.941,+ "acc_-20dB": 0.941,+ "acc_norm-25": 0.912,+ "acc_lifted": 0.912,+ "acc_at-56": 0.941,+ "acc_at-56_lifted": 0.912,+ "mean_abs_dp_-20dB": 0.046,+ "mean_abs_dp_norm-25": 0.033,+ "mean_abs_dp_lifted": 0.012,+ "mean_abs_dp_at-56": 0.063,+ "mean_abs_dp_at-56_lifted": 0.032+ },+ "cinetic": {+ "clips": 31,+ "rms_db_median": -24.3,+ "lifted_clips": 8,+ "mean_p_orig": 0.062,+ "mean_p_-20dB": 0.08,+ "mean_p_norm-25": 0.068,+ "mean_p_lifted": 0.062,+ "mean_p_at-56": 0.082,+ "mean_p_at-56_lifted": 0.072,+ "acc_orig": 1.0,+ "acc_-20dB": 0.968,+ "acc_norm-25": 1.0,+ "acc_lifted": 1.0,+ "acc_at-56": 0.968,+ "acc_at-56_lifted": 1.0,+ "mean_abs_dp_-20dB": 0.037,+ "mean_abs_dp_norm-25": 0.012,+ "mean_abs_dp_lifted": 0.001,+ "mean_abs_dp_at-56": 0.059,+ "mean_abs_dp_at-56_lifted": 0.017+ },+ "generic": {+ "clips": 28,+ "rms_db_median": -24.6,+ "lifted_clips": 7,+ "mean_p_orig": 0.042,+ "mean_p_-20dB": 0.05,+ "mean_p_norm-25": 0.043,+ "mean_p_lifted": 0.042,+ "mean_p_at-56": 0.064,+ "mean_p_at-56_lifted": 0.042,+ "acc_orig": 1.0,+ "acc_-20dB": 1.0,+ "acc_norm-25": 1.0,+ "acc_lifted": 1.0,+ "acc_at-56": 1.0,+ "acc_at-56_lifted": 1.0,+ "mean_abs_dp_-20dB": 0.015,+ "mean_abs_dp_norm-25": 0.006,+ "mean_abs_dp_lifted": 0.0,+ "mean_abs_dp_at-56": 0.03,+ "mean_abs_dp_at-56_lifted": 0.007+ }+ },+ "robustness_A_noaug": [+ {+ "perturbation": "noise70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.687,+ "acc_unperturbed": 0.9217,+ "mean_p_real": 0.362,+ "mean_p_unperturbed": 0.154+ },+ {+ "perturbation": "room70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.7217,+ "acc_unperturbed": 0.887,+ "mean_p_real": 0.317,+ "mean_p_unperturbed": 0.172+ },+ {+ "perturbation": "gated",+ "label": "real/pro",+ "n": 106,+ "[email protected]": 0.7264,+ "acc_unperturbed": 0.8113,+ "mean_p_real": 0.638,+ "mean_p_unperturbed": 0.715+ },+ {+ "perturbation": "short",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.9565,+ "acc_unperturbed": 0.9304,+ "mean_p_real": 0.138,+ "mean_p_unperturbed": 0.145+ },+ {+ "perturbation": "short",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.7196,+ "acc_unperturbed": 0.8879,+ "mean_p_real": 0.67,+ "mean_p_unperturbed": 0.772+ },+ {+ "perturbation": "mp3",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8696,+ "acc_unperturbed": 0.9304,+ "mean_p_real": 0.191,+ "mean_p_unperturbed": 0.159+ },+ {+ "perturbation": "mp3",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.8411,+ "acc_unperturbed": 0.8505,+ "mean_p_real": 0.741,+ "mean_p_unperturbed": 0.754+ }+ ],+ "robustness_A_clap": [+ {+ "perturbation": "noise70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8,+ "acc_unperturbed": 0.9304,+ "mean_p_real": 0.281,+ "mean_p_unperturbed": 0.154+ },+ {+ "perturbation": "room70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.7391,+ "acc_unperturbed": 0.8609,+ "mean_p_real": 0.313,+ "mean_p_unperturbed": 0.172+ },+ {+ "perturbation": "gated",+ "label": "real/pro",+ "n": 106,+ "[email protected]": 0.7453,+ "acc_unperturbed": 0.8113,+ "mean_p_real": 0.66,+ "mean_p_unperturbed": 0.715+ },+ {+ "perturbation": "short",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8957,+ "acc_unperturbed": 0.9043,+ "mean_p_real": 0.156,+ "mean_p_unperturbed": 0.145+ },+ {+ "perturbation": "short",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.7383,+ "acc_unperturbed": 0.9065,+ "mean_p_real": 0.679,+ "mean_p_unperturbed": 0.772+ },+ {+ "perturbation": "mp3",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8609,+ "acc_unperturbed": 0.887,+ "mean_p_real": 0.21,+ "mean_p_unperturbed": 0.159+ },+ {+ "perturbation": "mp3",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.8505,+ "acc_unperturbed": 0.8505,+ "mean_p_real": 0.746,+ "mean_p_unperturbed": 0.754+ }+ ],+ "robustness_M": [+ {+ "perturbation": "noise70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8348,+ "acc_unperturbed": 0.9478,+ "mean_p_real": 0.278,+ "mean_p_unperturbed": 0.154+ },+ {+ "perturbation": "room70",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.7739,+ "acc_unperturbed": 0.887,+ "mean_p_real": 0.299,+ "mean_p_unperturbed": 0.172+ },+ {+ "perturbation": "gated",+ "label": "real/pro",+ "n": 106,+ "[email protected]": 0.7547,+ "acc_unperturbed": 0.8396,+ "mean_p_real": 0.69,+ "mean_p_unperturbed": 0.715+ },+ {+ "perturbation": "short",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.9043,+ "acc_unperturbed": 0.9217,+ "mean_p_real": 0.152,+ "mean_p_unperturbed": 0.145+ },+ {+ "perturbation": "short",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.729,+ "acc_unperturbed": 0.8879,+ "mean_p_real": 0.696,+ "mean_p_unperturbed": 0.772+ },+ {+ "perturbation": "mp3",+ "label": "synth",+ "n": 115,+ "[email protected]": 0.8957,+ "acc_unperturbed": 0.9043,+ "mean_p_real": 0.195,+ "mean_p_unperturbed": 0.159+ },+ {+ "perturbation": "mp3",+ "label": "real/pro",+ "n": 107,+ "[email protected]": 0.8505,+ "acc_unperturbed": 0.8879,+ "mean_p_real": 0.782,+ "mean_p_unperturbed": 0.754+ }+ ],+ "ast_subset": {+ "ast_synth_classes_max": {+ "n": 244,+ "n_real": 118,+ "auc": 0.7555,+ "[email protected]": 0.4918,+ "brier": 0.4029,+ "logloss": 1.2974,+ "classes": [+ "Synthesizer",+ "Sine wave",+ "Beep, bleep",+ "Electronic tuner",+ "Chirp tone",+ "Dial tone",+ "Pink noise",+ "White noise",+ "Static",+ "Buzz",+ "Hum",+ "Theremin",+ "Video game music",+ "Ringtone",+ "Sound effect"+ ]+ },+ "ast_single:Synthesizer": {+ "auc_real_high": 0.8465+ },+ "ast_single:Sine wave": {+ "auc_real_high": 0.8253+ },+ "ast_single:Beep, bleep": {+ "auc_real_high": 0.7498+ },+ "ast_single:Electronic music": {+ "auc_real_high": 0.7551+ },+ "ast_single:Music": {+ "auc_real_high": 0.6642+ },+ "S": {+ "n": 244,+ "n_real": 118,+ "auc": 0.9566,+ "[email protected]": 0.877,+ "brier": 0.0845,+ "logloss": 0.2796+ },+ "S_ast": {+ "n": 244,+ "n_real": 118,+ "auc": 0.9533,+ "[email protected]": 0.8689,+ "brier": 0.0894,+ "logloss": 0.291+ },+ "S_astonly": {+ "n": 244,+ "n_real": 118,+ "auc": 0.7618,+ "[email protected]": 0.7418,+ "brier": 0.1978,+ "logloss": 0.5876+ },+ "A_on_subset": {+ "n": 244,+ "n_real": 118,+ "auc": 0.951,+ "[email protected]": 0.8975,+ "brier": 0.0884,+ "logloss": 0.2969+ },+ "clap_zero_shot_on_subset": {+ "n": 244,+ "n_real": 118,+ "auc": 0.8778,+ "[email protected]": 0.7582,+ "brier": 0.1861,+ "logloss": 0.651+ }+ },+ "final_model": {+ "fusion": {+ "heads": [+ "clap",+ "signal"+ ],+ "coef": [+ 1.212409,+ 0.0+ ],+ "intercept": -0.153358+ },+ "signal_coef": {+ "silence_frac": -0.106636,+ "floor_db": 0.317801,+ "spec_floor_db": 0.0,+ "log_flatness": 0.0,+ "peakiness_db": 0.0,+ "line_share": 0.0,+ "bandwidth_khz": 0.0,+ "quiet_flat": 0.898494,+ "tonal_frames": 0.0,+ "pitch_wobble": 0.199091,+ "env_resid_db": 0.007039,+ "texture_kurt": 0.0,+ "crest_db": 0.755868+ },+ "C": {+ "clap": 0.001,+ "signal": 0.1+ },+ "clap_direction": {+ "toward_real": [+ "a high quality studio recording",+ "a metal scrape",+ "an old typewriter",+ "a match being struck",+ "a knife chopping on a board",+ "a small object dropping on a table",+ "pizzicato strings",+ "a can opening"+ ],+ "toward_synth": [+ "a square wave chiptune blip",+ "a simple electronic synthesizer tone made by code",+ "a warm analog synth pad",+ "a cheap synthesized beep",+ "a computer-generated electronic tone",+ "filtered white noise",+ "an 8-bit video game sound effect",+ "an 8-bit sound effect"+ ],+ "how": "cosine between each vocabulary text embedding and the CLAP head weight direction (coef / scale)"+ }+ },+ "applications": {+ "clawd_film_real_soundtrack": {+ "final": {+ "p_real_mean": 0.942,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ },+ "held_out": {+ "p_real_mean": 0.91,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ }+ },+ "clawd_film_synth_soundtrack": {+ "final": {+ "p_real_mean": 0.327,+ "windows": 15,+ "windows_lean_synth": 14,+ "windows_le_lo": 1,+ "verdict": "likely code-synthesized"+ },+ "held_out": {+ "p_real_mean": 0.334,+ "windows": 15,+ "windows_lean_synth": 13,+ "windows_le_lo": 1,+ "verdict": "likely code-synthesized"+ }+ },+ "clawd_soundtrack_wav": {+ "final": {+ "p_real_mean": 0.933,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ },+ "held_out": {+ "p_real_mean": 0.901,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ }+ },+ "clawd_stem_pro_sfx": {+ "final": {+ "p_real_mean": 0.787,+ "windows": 15,+ "windows_lean_synth": 2,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ },+ "held_out": {+ "p_real_mean": 0.646,+ "windows": 15,+ "windows_lean_synth": 5,+ "windows_le_lo": 0,+ "verdict": "uncertain"+ }+ },+ "clawd_stem_pro_amb": {+ "final": {+ "p_real_mean": 0.882,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ },+ "held_out": {+ "p_real_mean": 0.84,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ }+ },+ "clawd_stem_pro_music": {+ "final": {+ "p_real_mean": 0.915,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ },+ "held_out": {+ "p_real_mean": 0.811,+ "windows": 15,+ "windows_lean_synth": 0,+ "windows_le_lo": 0,+ "verdict": "real or pro-produced"+ }+ }+ },+ "held_out_model": {+ "clips": 1273,+ "excluded": "accepted Clawd assets (incl. assets/music) and the retired score",+ "file_thresholds": {+ "hi": 0.77,+ "lo": 0.4,+ "files": 119,+ "synth_files": 23,+ "how": "lo: 92.5% precision on held-out long files (scheme M); hi: max(that, the clip hi)"+ }+ },+ "windows_ui_sounds": {+ "files": 70,+ "median_p_real": 0.238,+ "real or pro-produced": 6,+ "uncertain": 32,+ "likely code-synthesized": 32,+ "lowest": [+ [+ "Windows Hardware Insert.wav",+ 0.006+ ],+ [+ "Windows Hardware Remove.wav",+ 0.007+ ],+ [+ "Alarm10.wav",+ 0.009+ ],+ [+ "Windows Notify System Generic.wav",+ 0.009+ ],+ [+ "Windows Background.wav",+ 0.013+ ]+ ],+ "highest": [+ [+ "Ring08.wav",+ 0.861+ ],+ [+ "Speech Off.wav",+ 0.885+ ],+ [+ "ding.wav",+ 0.893+ ],+ [+ "Windows Recycle.wav",+ 0.898+ ],+ [+ "Windows Default.wav",+ 0.946+ ]+ ]+ },+ "layers": {+ "clips": 369,+ "beds": 184,+ "bed_p_real_mean": 0.804,+ "beds_lean_synth": 0.114,+ "per_level": [+ {+ "level_db": -12,+ "n": 93,+ "base": {+ "mean_p_real": 0.673,+ "lean_synth": 0.269,+ "le_lo": 0.172,+ "ge_hi": 0.538+ },+ "train": {+ "mean_p_real": 0.591,+ "lean_synth": 0.344,+ "le_lo": 0.204,+ "ge_hi": 0.441+ }+ },+ {+ "level_db": -6,+ "n": 91,+ "base": {+ "mean_p_real": 0.621,+ "lean_synth": 0.33,+ "le_lo": 0.121,+ "ge_hi": 0.429+ },+ "train": {+ "mean_p_real": 0.536,+ "lean_synth": 0.418,+ "le_lo": 0.132,+ "ge_hi": 0.253+ }+ },+ {+ "level_db": 0,+ "n": 94,+ "base": {+ "mean_p_real": 0.633,+ "lean_synth": 0.298,+ "le_lo": 0.149,+ "ge_hi": 0.479+ },+ "train": {+ "mean_p_real": 0.524,+ "lean_synth": 0.457,+ "le_lo": 0.17,+ "ge_hi": 0.266+ }+ },+ {+ "level_db": 6,+ "n": 91,+ "base": {+ "mean_p_real": 0.622,+ "lean_synth": 0.286,+ "le_lo": 0.143,+ "ge_hi": 0.396+ },+ "train": {+ "mean_p_real": 0.512,+ "lean_synth": 0.484,+ "le_lo": 0.154,+ "ge_hi": 0.253+ }+ }+ ],+ "per_bed_group_base": {+ "ambience": 0.167,+ "film": 0.167,+ "foley": 0.214,+ "garden": 0.133,+ "keyboard": 0.133,+ "magic": 0.833,+ "motion": 0.682,+ "music": 0.222,+ "toy": 0.227+ },+ "per_layer_family_base": {+ "cinetic": 0.375,+ "generic": 0.221,+ "retired": 0.295+ },+ "M_layers": {+ "n": 1484,+ "n_real": 691,+ "auc": 0.9281,+ "[email protected]": 0.8935,+ "brier": 0.0926,+ "logloss": 0.3313,+ "hi": 0.68,+ "lo": 0.05,+ "band": {+ "coverage": 0.5081,+ "precision_real": 0.9249,+ "precision_synth": 0.8359,+ "real_called_synth": 0.0304,+ "synth_called_real": 0.0593,+ "uncertain_frac": 0.4919+ }+ },+ "summary": "code-synthesized layers at -6..+6 dB inside real beds: 0.30 of windows lean synthesized (p_real < 0.5), 0.14 at or below lo; the beds alone: 0.114"+ }+}\ No newline at end of filediff --git a/eval/realness/run_eval.py b/eval/realness/run_eval.pynew file mode 100644index 0000000..d6a3ef5--- /dev/null+++ b/eval/realness/run_eval.py@@ -0,0 +1,832 @@+"""Held-out evaluation of the realness detector: grouped cross-validation, baselines, robustness, file level.++ OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py [--stage cv|layers|apps|level|report]++Needs out/realness/manifest.json (build_data.py) and out/realness/feats (extract.py; --ast 240 for the AST rows).+Stage cv caches out-of-group predictions per scheme in out/realness/oof/ (time-boxed: run again until it says done);+stage layers scores the layered clips (real bed + one code-synthesized layer) with models that never saw the bed's group+nor the layer's family; stage apps scores whole application files (the Clawd film with its real and its synthesized+soundtrack, the Windows UI sounds); stage level measures how much the input level moves p_real (final model; a fixed+subset at its own level, -20 dB, normalised to -25 dBFS RMS, lifted as at inference, at -56 dBFS); stage report+writes eval/realness/results.json and RESULTS.md (generated: every number in it comes from this run). Every clip is scored by a model that never saw its CV group:+ A groups = real asset category (ambience, foley, garden, keyboard, magic, motion, toy, music) or the films;+ synthesized family (retired score, cinetic engine, generic generators): the headline+ A_noaug the same without the augmented training copies; A_clap / A_signal: one head only (ablation)+ B synthesized groups = one generator type (cinetic:whoosh, generic:beep, retired:stem_pad ...)+ C the retired score is never in any training set: real recordings must still rank above it+ D neither the retired score nor the cinetic engine is ever trained on (only generic generators as negatives)+ M as A, but music is held out one provider at a time (mk / sb / ch / other): the grouping the final model is+ trained with (each source weighs the same, so music counts as four sources); verdict thresholds come from M+ M_layers as M, also trained on the layered clips of the other groups (tested, not adopted: see RESULTS.md)+ S* the stratified AST subset: clap+signal vs +AST head vs AST head alone+"""+import argparse+import hashlib+import json+import os+import sys+import time+from types import SimpleNamespace++import numpy as np+from sklearn.metrics import roc_auc_score++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, HERE)+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402+import common as C # noqa: E402+from earshot import realness # noqa: E402++OOF = os.path.join(C.OUT, 'oof')+SCHEMES = {'A': {}, 'A_noaug': {'augment': False}, 'A_clap': {'heads': ('clap',)}, 'A_signal': {'heads': ('signal',)},+ 'C': {'exclude': ('retired',)},+ 'D': {'exclude': ('retired', 'cinetic')}, 'B': {'groups': 'type', 'c_clap': 0.001, 'inner': 3},+ 'M': {'groups': 'music_src'}, 'M_layers': {'groups': 'music_src', 'layers': True},+ 'S': {'ast': True}, 'S_ast': {'ast': True, 'heads': ('clap', 'signal', 'ast')},+ 'S_astonly': {'ast': True, 'heads': ('ast',)}}+APPS = {'clawd_film_real_soundtrack': f'{EP.FILM}/out/Clawd_1080p60.mp4',+ 'clawd_film_synth_soundtrack': f'{EP.FILM}/out/superseded/Clawd_1080p60_synth-audio.mp4',+ 'clawd_soundtrack_wav': f'{EP.FILM}/out/audio/soundtrack.wav',+ 'clawd_stem_pro_sfx': f'{EP.FILM}/out/audio/stems_pro/sfx.wav',+ 'clawd_stem_pro_amb': f'{EP.FILM}/out/audio/stems_pro/amb.wav',+ 'clawd_stem_pro_music': f'{EP.FILM}/out/audio/stems_pro/music.wav'}+PRO_SYNTH = 'C:/Windows/Media' # professionally made synthetic UI / alert sounds (never trained on)+ASSETS = EP.ASSETS + '/'+++def music_source(src):+ """provider prefix of a music file name (mk = Mixkit, sb, ch ...; rare providers pooled)"""+ v = os.path.basename(src).split('-')[0].split('_')[0]+ return v if v in ('mk', 'sb', 'ch') else 'other'+++def groups_of(d, kind='fam'):+ if kind == 'music_src':+ g = groups_of(d)+ return np.array([f'music:{music_source(s)}' if gi == 'music' else gi for gi, s in zip(g, d.src)])+ if kind == 'type':+ return np.where(d.y == 1, d.group, np.char.add(np.char.add(d.family.astype(str), ':'), d.type.astype(str)))+ return np.where(d.y == 1, d.group, d.family)+++def bed_groups_of(d, kind='fam'):+ """the group of a layered clip's real bed under the same grouping (other clips: their own group)"""+ g = groups_of(d, kind).astype(object)+ for i in np.where(d.is_layer)[0]:+ b = d.bed_group[i]+ g[i] = f'music:{music_source(d.bed_src[i])}' if b == 'music' and kind == 'music_src' else b+ return g+++def run_layers():+ """the layer test: every layered clip (real bed + one code-synthesized layer) is scored by models that never saw+ its bed's group nor its layer's family (scheme-M grouping, the final model's CLAP C): 'base' trains as the final+ model does without layered clips, 'train' also trains on the layered clips of the other groups"""+ d = C.load()+ g = groups_of(d, 'music_src')+ bg = bed_groups_of(d, 'music_src')+ c = realness.load_model()['C']['clap']+ Lm = d.is_layer+ pairs = sorted(set(zip(bg[Lm], g[Lm])))+ for variant in ('base', 'train'):+ path = os.path.join(OOF, f'layers_{variant}.npz')+ if os.path.exists(path):+ continue+ t = time.time()+ p = np.full(len(d), np.nan)+ usable = (d.perturb == '') | d.augment | (Lm & (variant == 'train'))+ for gb, gl in pairs:+ te = Lm & (bg == gb) & (g == gl)+ tr = usable & ~np.isin(g, [gb, gl]) & ~np.isin(bg, [gb, gl])+ p[te] = C.predict(C.fit_model(d.sub(tr), g[tr], c_clap=c), d.sub(te))+ np.savez(path, ids=d.ids, p=p)+ print(f'layers {variant}: {int(Lm.sum())} clips, {len(pairs)} folds, {time.time() - t:.0f} s', flush=True)+ if variant == 'base':+ print('run again for the train variant')+ return+++def run_cv(budget):+ os.makedirs(OOF, exist_ok=True)+ t0 = time.time()+ left = []+ for name, cfg in SCHEMES.items():+ path = os.path.join(OOF, name + '.npz')+ if os.path.exists(path):+ continue+ if time.time() - t0 > budget:+ left.append(name)+ continue+ if cfg.get('ast') and not os.path.exists(os.path.join(C.FEATS, 'ast.npz')):+ print(f'{name}: skipped (no AST features; run extract.py --ast 240)')+ continue+ d = C.load(require_ast=bool(cfg.get('ast')), robust=not cfg.get('ast'))+ g = groups_of(d, cfg.get('groups', 'fam'))+ tm = ~np.isin(d.family, cfg['exclude']) if cfg.get('exclude') else None+ kw = {k: cfg[k] for k in ('heads', 'c_clap', 'augment', 'aug_weight', 'inner', 'layers') if k in cfg}+ if cfg.get('layers'):+ kw['bed_groups'] = bed_groups_of(d, cfg.get('groups', 'fam'))+ t = time.time()+ p, L = C.logo(d, g, tm, **kw)+ np.savez(path, ids=d.ids, p=p, **{'L_' + k: v for k, v in L.items()})+ print(f'{name}: {len(d)} clips, {len(set(g))} folds, {time.time() - t:.0f} s', flush=True)+ print('cv done' if not left else f'run again for: {", ".join(left)}')+++def prompt_embeddings():+ """CLAP text embeddings of the zero-shot prompts, read from earshot's text cache when present"""+ from earshot import config+ texts = realness.REAL_PROMPTS + realness.SYNTH_PROMPTS+ for d in sorted(os.listdir(os.path.join(config.EMB_CACHE, 'clap'))) if os.path.isdir(os.path.join(config.EMB_CACHE, 'clap')) else []:+ f = os.path.join(config.EMB_CACHE, 'clap', d, 'text.npz')+ if os.path.exists(f) and config.CLAP_MODEL.replace('/', '--') in d:+ z = np.load(f, allow_pickle=False)+ keys = [hashlib.sha1(t.encode('utf-8')).hexdigest()[:24] for t in texts]+ if all(k in z.files for k in keys):+ return np.stack([z[k] for k in keys])+ from earshot.models import clap+ return clap.get().embed_text(texts)+++def app_files():+ """{name: path}: the application files plus every top-level wav of C:/Windows/Media (name 'winui:<file>')"""+ out = dict(APPS)+ if os.path.isdir(PRO_SYNTH):+ out.update({f'winui:{f}': os.path.join(PRO_SYNTH, f) for f in sorted(os.listdir(PRO_SYNTH))+ if f.lower().endswith('.wav')})+ return out+++def run_apps():+ """windows of whole application files and of the Windows UI sounds -> CLAP + signal features, exactly as+ earshot.realness scores them (quiet sources lifted), cached in out/realness/feats/apps.npz"""+ from earshot import audio+ from earshot.models import clap+ c = clap.get()+ ids, E, S = [], [], []+ for name, path in app_files().items():+ x = realness.lift_quiet(audio.load(path, mono=True, cache=False))[0]+ spans = realness._clips(x)+ live = [(t0, w) for t0, _, w in spans if realness.audible(w)]+ E += list(c.embed_audio([w for _, w in live]))+ S += [json.dumps(realness.signal_features(w)) for _, w in live]+ ids += [f'{name}@{t0:.0f}' for t0, _ in live]+ if not name.startswith('winui:'):+ print(name, len(live), 'windows', flush=True)+ np.savez(os.path.join(C.FEATS, 'apps.npz'), ids=np.array(ids), emb=np.array(E, np.float32), sig=np.array(S))+++def run_level():+ """p_real of a fixed subset of clean clips under level changes (final model, CLAP head + fusion; the CLAP head is+ the only level-dependent input, signal features are peak-normalised): own level, 20 dB quieter, normalised to+ -25 dBFS RMS, own level after realness.lift_quiet (what inference does), and moved to -56 dBFS RMS without / with+ the lift (the old cliff: the first build scored sources between -60 and -30 dBFS RMS as they were)"""+ import soundfile as sf+ from earshot.models import clap+ man = json.load(open(os.path.join(C.OUT, 'manifest.json')))+ m = realness.load_model()+ c = clap.get()+ at = lambda x, r, db: (x * 10 ** ((db - r) / 20)).astype(np.float32) # noqa: E731+ conds = {'orig': lambda x, r: x, '-20dB': lambda x, r: x * 10 ** (-1.0), 'norm-25': lambda x, r: at(x, r, -25),+ 'lifted': lambda x, r: realness.lift_quiet(x)[0], 'at-56': lambda x, r: at(x, r, -56),+ 'at-56_lifted': lambda x, r: realness.lift_quiet(at(x, r, -56))[0]}+ out = {}+ for k, clips in {'real': man['real'][::12], 'cinetic': man['cinetic'][::9], 'generic': man['generic'][::8]}.items():+ X = [sf.read(os.path.join(C.OUT, cl['wav']), dtype='float32')[0] for cl in clips]+ rms = np.array([20 * np.log10(np.sqrt(np.mean(x ** 2)) + 1e-12) for x in X])+ P = {}+ for g, fn in conds.items():+ lc = np.array([realness.head_logit(m['heads']['clap'], e)[0]+ for e in c.embed_audio([fn(x, r) for x, r in zip(X, rms)])])+ P[g] = 1 / (1 + np.exp(-(m['fusion']['intercept'] + m['fusion']['coef'][0] * lc)))+ lab = k == 'real'+ out[k] = {'clips': len(X), 'rms_db_median': round(float(np.median(rms)), 1),+ 'lifted_clips': int(np.sum(rms < realness.LIFT_BELOW_DB)),+ **{f'mean_p_{g}': round(float(v.mean()), 3) for g, v in P.items()},+ **{f'acc_{g}': round(float(np.mean((v >= 0.5) == lab)), 3) for g, v in P.items()},+ **{f'mean_abs_dp_{g}': round(float(np.abs(P[g] - P['orig']).mean()), 3) for g in conds if g != 'orig'}}+ print(k, out[k], flush=True)+ json.dump(out, open(os.path.join(C.FEATS, 'level.json'), 'w'), indent=1)+++# ---------------------------------------------------------------------------------------------- metrics+def auc(y, s):+ return round(float(roc_auc_score(y, s)), 4) if len(set(y)) == 2 else None+++def brier(y, p):+ return round(float(np.mean((p - y) ** 2)), 4)+++def acc(y, p, t=0.5):+ return round(float(np.mean((p >= t) == (y == 1))), 4)+++def metrics(y, p):+ return {'n': int(len(y)), 'n_real': int(y.sum()), 'auc': auc(y, p), '[email protected]': acc(y, p), 'brier': brier(y, p),+ 'logloss': round(float(-np.mean(y * np.log(p + 1e-9) + (1 - y) * np.log(1 - p + 1e-9))), 4)}+++TARGET = 0.925+++def thresholds(y, p, target=TARGET):+ """hi = lowest p above which >= target of clips are real; lo = highest p below which >= target are synthesized"""+ o = np.argsort(-p)+ prec = np.cumsum(y[o]) / np.arange(1, len(o) + 1)+ ok = np.where(prec >= target)[0]+ hi = float(p[o][ok[-1]]) if len(ok) else 1.0+ o2 = np.argsort(p)+ prec2 = np.cumsum(1 - y[o2]) / np.arange(1, len(o2) + 1)+ ok2 = np.where(prec2 >= target)[0]+ lo = float(p[o2][ok2[-1]]) if len(ok2) else 0.0+ return round(min(max(hi, 0.5), 0.95), 2), round(max(min(lo, 0.5), 0.05), 2)+++def band_stats(y, p, lo, hi):+ real, syn = p >= hi, p <= lo+ unc = ~(real | syn)+ return {'coverage': round(float(np.mean(~unc)), 4),+ 'precision_real': round(float(np.mean(y[real])), 4) if real.any() else None,+ 'precision_synth': round(float(np.mean(1 - y[syn])), 4) if syn.any() else None,+ 'real_called_synth': round(float(np.mean(syn[y == 1])), 4), 'synth_called_real': round(float(np.mean(real[y == 0])), 4),+ 'uncertain_frac': round(float(np.mean(unc)), 4)}+++def calibration(y, p, bins=10):+ out = []+ for i in range(bins):+ m = (p >= i / bins) & (p < (i + 1) / bins + (i == bins - 1))+ if m.sum():+ out.append({'bin': f'{i / bins:.1f}-{(i + 1) / bins:.1f}', 'n': int(m.sum()), 'mean_p': round(float(p[m].mean()), 3),+ 'frac_real': round(float(y[m].mean()), 3)})+ return out+++def per_group(d, p, g):+ rows = []+ for k in sorted(set(g)):+ m = (g == k) & (d.perturb == '')+ lab = int(d.y[m][0])+ other = (d.y == 1 - lab) & (d.perturb == '')+ yy = np.r_[d.y[m], d.y[other]]+ rows.append({'group': str(k), 'label': 'real/pro' if lab else 'synth', 'n': int(m.sum()),+ 'mean_p_real': round(float(p[m].mean()), 3), '[email protected]': acc(d.y[m], p[m]),+ 'auc_vs_other_class': auc(yy, np.r_[p[m], p[other]])})+ return rows+++def file_level(d, p):+ """(sources, labels, mean window p_real) of the clean sources with >= 2 windows (long files)"""+ srcs = {}+ for i in np.where(d.perturb == '')[0]:+ if '@' in d.ids[i]:+ srcs.setdefault(d.src[i], []).append(i)+ multi = {s: v for s, v in srcs.items() if len(v) >= 2}+ return (list(multi), np.array([d.y[v[0]] for v in multi.values()]), np.array([p[v].mean() for v in multi.values()]),+ [len(v) for v in multi.values()])+++def file_thresholds(fy, fp, clip_hi, clip_lo):+ """verdict thresholds for the mean window p_real of a long file: lo from the held-out files (averaging windows+ makes a low mean stronger evidence than one low clip); hi is never looser than the clip hi, because real files+ outnumber synthesized ones in the held-out set, which makes precision on the real side cheap"""+ fh, fl = thresholds(fy, fp)+ return {'hi': max(fh, clip_hi), 'lo': max(fl, clip_lo), 'files': int(len(fy)), 'synth_files': int((fy == 0).sum()),+ 'how': f'lo: {100 * TARGET:g}% precision on held-out long files (scheme M); hi: max(that, the clip hi)'}+++def load_oof(name, d):+ path = os.path.join(OOF, name + '.npz')+ if not os.path.exists(path):+ return None+ z = np.load(path, allow_pickle=False)+ m = dict(zip(z['ids'], z['p']))+ return np.array([m.get(i, np.nan) for i in d.ids])+++def zero_shot(d, P):+ """CLAP zero-shot: log-sum-exp over the real prompts minus over the synth prompts (logit scale 38.7)"""+ s = 38.7 * (d.Xc @ P.T)+ n = len(realness.REAL_PROMPTS)+ lse = lambda v: v.max(axis=1) + np.log(np.exp(v - v.max(axis=1, keepdims=True)).sum(axis=1)) # noqa: E731+ return lse(s[:, :n]) - lse(s[:, n:])+++def report():+ d = C.load()+ clean = d.perturb == ''+ dc = d.sub(clean)+ g = groups_of(d)+ res = {'data': {}, 'schemes': {}, 'baselines': {}}+ res['data'] = {'clips': int(clean.sum()), 'real': int(dc.y.sum()), 'synth': int((1 - dc.y).sum()),+ 'by_family': {f: int((dc.family == f).sum()) for f in sorted(set(dc.family))},+ 'real_groups': sorted(set(dc.group[dc.y == 1])), 'perturbed_clips': int((~clean).sum())}+ P = {k: load_oof(k, d) for k in ('A', 'A_noaug', 'A_clap', 'A_signal', 'B', 'C', 'D', 'M')}+ for k, p in P.items():+ if p is None:+ continue+ # D trains on generic negatives only, so its generic fold has no negatives to learn from: score real vs the+ # two never-trained-on families+ keep = clean & ((d.y == 1) | np.isin(d.family, SCHEMES[k]['exclude'])) if k == 'D' else clean+ res['schemes'][k] = metrics(d.y[keep], p[keep])+ if k == 'D':+ res['schemes'][k]['note'] = 'real vs retired + cinetic, neither ever trained on (generic generators were the only negatives)'+ pA = P['A']+ pT, tk = (P['M'], 'M') if P['M'] is not None else (pA, 'A')+ hi, lo = thresholds(dc.y, pT[clean])+ res['thresholds'] = {'hi': hi, 'lo': lo, 'scheme': tk, 'how': f'>= {100 * TARGET:g}% precision on each side'}+ res['band'] = band_stats(dc.y, pT[clean], lo, hi)+ res['band_alt'] = {}+ for t in (0.95, TARGET, 0.9):+ h, lw = thresholds(dc.y, pT[clean], t)+ res['band_alt'][f'{t:g}'] = dict(hi=h, lo=lw, **band_stats(dc.y, pT[clean], lw, h))+ res['calibration'] = calibration(dc.y, pT[clean])+ res['per_group_A'] = per_group(d, pA, g)+ if P['M'] is not None:+ res['per_group_M'] = per_group(d, P['M'], g)+ nm = clean & (d.group != 'music')+ res['non_music'] = {'A': metrics(d.y[nm], pA[nm]), 'M': metrics(d.y[nm], P['M'][nm])}+ gt = groups_of(d, 'type')+ res['per_type_A'] = [r for r in per_group(d, pA, gt) if r['label'] == 'synth']+ if P['B'] is not None:+ res['per_type_B'] = [r for r in per_group(d, P['B'], gt) if r['label'] == 'synth']+ if P['M'] is not None:+ mu = clean & (d.group == 'music')+ files = sorted(set(d.src[mu]))+ fp = np.array([P['M'][mu & (d.src == f)].mean() for f in files])+ res['music_M'] = {'windows': int(mu.sum()), 'mean_p_real': round(float(P['M'][mu].mean()), 3),+ '[email protected]': acc(d.y[mu], P['M'][mu]), 'files': len(files),+ '[email protected]': round(float(np.mean(fp >= 0.5)), 4),+ 'files_called_synth': int((fp <= lo).sum()), 'files_called_real': int((fp >= hi).sum()),+ 'scheme_A_mean_p_real': round(float(pA[mu].mean()), 3),+ 'per_provider': [r for r in per_group(d, P['M'], groups_of(d, 'music_src')) if r['group'].startswith('music:')]}+ # the required check: real recordings vs the retired score, retired never trained on+ real = clean & (d.y == 1)+ checks = {}+ for k, fams in (('C', ('retired',)), ('D', ('retired', 'cinetic'))):+ if P[k] is None:+ continue+ for f in fams:+ m = real | (clean & (d.family == f))+ checks[f'{k}: real vs {f}'] = {'auc': auc(d.y[m], P[k][m]), '[email protected]': acc(d.y[m], P[k][m]),+ 'synth_mean_p': round(float(P[k][clean & (d.family == f)].mean()), 3)}+ assets = clean & np.isin(d.family, ['real-asset', 'real-rejected'])+ m = assets | (clean & (d.family == 'retired'))+ checks[f'{k}: real recordings only (no music/films) vs retired'] = {'auc': auc(d.y[m], P[k][m])}+ res['retired_check'] = checks+ # file level: windows of the same long source averaged; scheme M also gives the file verdict thresholds+ for k in ('A', 'M'):+ if P[k] is None:+ continue+ fs, fy, fp, nw = file_level(d, P[k])+ r = dict(metrics(fy, fp), note='sources with >= 2 windows (films, music, ambience, retired stems, synthesized '+ 'mixes): mean of window p_real')+ r['files'] = sorted([{'src': '/'.join(str(EP.anon(s)).replace(chr(92), '/').split('/')[-2:]), 'label': int(y), 'p_real': round(float(q), 3), 'windows': w}+ for s, y, q, w in zip(fs, fy, fp, nw)], key=lambda r: r['p_real'])+ if k == 'M':+ r['thresholds'] = file_thresholds(fy, fp, hi, lo)+ r['band'] = band_stats(fy, fp, r['thresholds']['lo'], r['thresholds']['hi'])+ r['calibration'] = calibration(fy, fp, 5)+ res[f'file_level_{k}'] = r+ # robustness: perturbed copies scored by the fold model that never saw their group+ rob = []+ base = dict(zip(d.ids, pA))+ for kind in ('noise70', 'room70', 'gated', 'short', 'mp3'):+ for lab in (0, 1):+ m = (d.perturb == kind) & (d.y == lab)+ if not m.any():+ continue+ pb = np.array([base.get(b, np.nan) for b in d.base[m]])+ rob.append({'perturbation': kind, 'label': 'real/pro' if lab else 'synth', 'n': int(m.sum()),+ '[email protected]': acc(d.y[m], pA[m]), 'acc_unperturbed': acc(d.y[m], pb),+ 'mean_p_real': round(float(pA[m].mean()), 3), 'mean_p_unperturbed': round(float(np.nanmean(pb)), 3)})+ res['robustness_A'] = rob+ lp = os.path.join(C.FEATS, 'level.json')+ if os.path.exists(lp):+ res['level_sensitivity'] = json.load(open(lp))+ for k in ('A_noaug', 'A_clap', 'M'):+ if P[k] is not None:+ bk = dict(zip(d.ids, P[k]))+ res['robustness_' + k] = [dict(r, **{'[email protected]': acc(d.y[m], P[k][m]),+ 'acc_unperturbed': acc(d.y[m], np.array([bk[b] for b in d.base[m]])),+ 'mean_p_real': round(float(P[k][m].mean()), 3)})+ for r in rob for m in [(d.perturb == r['perturbation']) & (d.y == (r['label'] == 'real/pro'))]]+ # baselines (no training): CLAP zero-shot prompts, single signal measurements+ Pe = prompt_embeddings()+ zs = zero_shot(dc, Pe)+ res['baselines']['clap_zero_shot'] = dict(metrics(dc.y, 1 / (1 + np.exp(-zs))), prompts={+ 'real': realness.REAL_PROMPTS, 'synth': realness.SYNTH_PROMPTS})+ res['baselines']['signal_univariate_auc'] = {k: auc(dc.y[np.isfinite(dc.Xs[:, j])], C.SIGNS[k] * dc.Xs[np.isfinite(dc.Xs[:, j]), j])+ for j, k in enumerate(C.SIG_INPUTS)}+ # AST subset+ if os.path.exists(os.path.join(C.FEATS, 'ast.npz')):+ da = C.load(require_ast=True, robust=False)+ labels = json.load(open(os.path.join(C.FEATS, 'ast_labels.json'))) if os.path.exists(os.path.join(C.FEATS, 'ast_labels.json')) else None+ sub = {}+ if labels:+ idx = [labels.index(l) for l in realness.AST_SYNTH]+ smax = da.Xa[:, idx].max(axis=1)+ sub['ast_synth_classes_max'] = dict(metrics(da.y, 1 - smax), classes=realness.AST_SYNTH)+ for l in ('Synthesizer', 'Sine wave', 'Beep, bleep', 'Electronic music', 'Music'):+ sub[f'ast_single:{l}'] = {'auc_real_high': auc(da.y, -da.Xa[:, labels.index(l)])}+ for k in ('S', 'S_ast', 'S_astonly'):+ p = load_oof(k, da)+ if p is not None:+ sub[k] = metrics(da.y, p)+ m = np.isin(d.ids, da.ids) & clean+ sub['A_on_subset'] = metrics(d.y[m], pA[m])+ zsa = zero_shot(da, Pe)+ sub['clap_zero_shot_on_subset'] = metrics(da.y, 1 / (1 + np.exp(-zsa)))+ res['ast_subset'] = sub+ # final model internals+ if os.path.exists(realness.MODEL_PATH):+ m = json.load(open(realness.MODEL_PATH))+ res['final_model'] = {'fusion': m['fusion'], 'signal_coef': dict(zip(m['heads']['signal']['inputs'], m['heads']['signal']['coef'])),+ 'C': m.get('C'), 'clap_direction': m.get('clap_direction')}+ # application files: the final model (in-sample: it trained on the real soundtrack's source recordings and on+ # the retired score) and a model that never saw the accepted Clawd assets (the 83 recordings the real soundtrack+ # is built from are among them) nor the retired score; plus the Windows UI sounds (never trained on)+ ap = os.path.join(C.FEATS, 'apps.npz')+ if os.path.exists(ap) and os.path.exists(realness.MODEL_PATH):+ z = np.load(ap, allow_pickle=False)+ sig = [json.loads(s) for s in z['sig']]+ Xs = np.array([[np.nan if f is None else f.get(k, np.nan) for k in C.SIG_INPUTS] for f in sig], float)+ dd = SimpleNamespace(Xc=z['emb'], Xs=Xs, Xa=None)+ fin = json.load(open(realness.MODEL_PATH))+ fth = realness.thresholds_of(fin, True)+ src = np.array([s.replace(chr(92), '/') for s in d.src])+ accepted = np.array([s.startswith(ASSETS) and '/_rejected/' not in s for s in src])+ trh = d.sub(((d.perturb == '') | d.augment) & ~accepted & (d.family != 'retired'))+ mH = C.fit_model(trh, groups_of(trh, 'music_src'), c_clap=fin.get('C', {}).get('clap'))+ pp = {'final': C.predict(fin, dd), 'held_out': C.predict(mH, dd)}+ name_of = np.array([i.rsplit('@', 1)[0] for i in z['ids']])+ out = {}+ for name in APPS:+ mk = name_of == name+ out[name] = {k: {'p_real_mean': round(float(v[mk].mean()), 3), 'windows': int(mk.sum()),+ 'windows_lean_synth': int((v[mk] < 0.5).sum()), 'windows_le_lo': int((v[mk] <= lo).sum()),+ 'verdict': realness.verdict_of(float(v[mk].mean()), fin, mk.sum() > 1)}+ for k, v in pp.items()}+ res['applications'] = out+ res['held_out_model'] = {'clips': int(len(trh)), 'excluded': 'accepted Clawd assets (incl. assets/music) and '+ 'the retired score', 'file_thresholds': fth}+ ui = sorted(set(n for n in name_of if n.startswith('winui:')))+ if ui:+ fp = np.array([pp['final'][name_of == n].mean() for n in ui])+ vs = [realness.verdict_of(float(q), fin, (name_of == n).sum() > 1) for q, n in zip(fp, ui)]+ res['windows_ui_sounds'] = {'files': len(ui), 'median_p_real': round(float(np.median(fp)), 3),+ **{v: int(sum(x == v for x in vs)) for v in realness.VERDICTS[:3]},+ 'lowest': [[n[6:], round(float(q), 3)] for q, n in sorted(zip(fp, ui))[:5]],+ 'highest': [[n[6:], round(float(q), 3)] for q, n in sorted(zip(fp, ui))[-5:]]}+ # layer test: real beds + one code-synthesized layer (run_layers), and the variant trained on layered clips+ pl = {k: load_oof(f'layers_{k}', d) for k in ('base', 'train')}+ if pl['base'] is not None:+ man = {c['id']: c for part in json.load(open(os.path.join(C.OUT, 'manifest.json'))).values() for c in part}+ Lm = d.is_layer & np.isfinite(pl['base'])+ lev = np.array([man[i].get('level_db', 99) for i in d.ids])+ pbed = dict(zip(d.ids, P['M'])) if P['M'] is not None else {}+ bed = np.array([pbed.get(man[i].get('bed'), np.nan) for i in d.ids])+ beds = clean & (d.y == 1) & np.isin(d.ids, [man[i]['bed'] for i in d.ids[Lm]])+ lay = {'clips': int(Lm.sum()), 'beds': int(beds.sum()), 'bed_p_real_mean': round(float(np.nanmean(bed[Lm])), 3),+ 'beds_lean_synth': round(float(np.mean(P['M'][beds] < 0.5)), 3) if P['M'] is not None else None,+ 'per_level': []}+ for lv in sorted(set(lev[Lm])):+ mk = Lm & (lev == lv)+ row = {'level_db': int(lv), 'n': int(mk.sum())}+ for k, q in pl.items():+ if q is not None:+ row[k] = {'mean_p_real': round(float(q[mk].mean()), 3), 'lean_synth': round(float(np.mean(q[mk] < 0.5)), 3),+ 'le_lo': round(float(np.mean(q[mk] <= lo)), 3), 'ge_hi': round(float(np.mean(q[mk] >= hi)), 3)}+ lay['per_level'].append(row)+ lay['per_bed_group_base'] = {str(b): round(float(np.mean(pl['base'][Lm & (d.bed_group == b) & (lev >= -6)] < 0.5)), 3)+ for b in sorted(set(d.bed_group[Lm]))}+ lay['per_layer_family_base'] = {str(f): round(float(np.mean(pl['base'][Lm & (d.family == f) & (lev >= -6)] < 0.5)), 3)+ for f in sorted(set(d.family[Lm]))}+ pml = load_oof('M_layers', d)+ if pml is not None:+ h2, l2 = thresholds(dc.y, pml[clean])+ lay['M_layers'] = dict(metrics(dc.y, pml[clean]), hi=h2, lo=l2, band=band_stats(dc.y, pml[clean], l2, h2))+ top = [r for r in lay['per_level'] if r['level_db'] >= -6]+ lay['summary'] = (f"code-synthesized layers at -6..+6 dB inside real beds: "+ f"{np.mean([r['base']['lean_synth'] for r in top]):.2f} of windows lean synthesized "+ f"(p_real < 0.5), {np.mean([r['base']['le_lo'] for r in top]):.2f} at or below lo; "+ f"the beds alone: {lay['beds_lean_synth']}")+ res['layers'] = lay+ json.dump(res, open(os.path.join(HERE, 'results.json'), 'w'), indent=1)+ with open(os.path.join(HERE, 'RESULTS.md'), 'w', encoding='utf-8') as fh:+ fh.write(markdown(res))+ a = res['schemes']['A']+ print(f"scheme A: AUC {a['auc']}, [email protected] {a['[email protected]']}, Brier {a['brier']}; thresholds lo {lo} hi {hi}; "+ 'wrote results.json + RESULTS.md')+ return res+++# ---------------------------------------------------------------------------------------------- RESULTS.md+def _tab(head, rows):+ return '\n'.join(['| ' + ' | '.join(head) + ' |', '|' + '---|' * len(head)] ++ ['| ' + ' | '.join('-' if v is None else str(v) for v in r) + ' |' for r in rows]) + '\n'+++FAMILY_DOC = {'real-asset': 'Freesound / Mixkit recordings used in the Clawd film (7 categories)',+ 'real-rejected': 'real recordings rejected for quality (still real)',+ 'pro-film': 'audio of the 7 professional launch films (4 s windows)',+ 'pro-music': 'professionally produced music (assets/music, kept and rejected)',+ 'retired': 'the retired code-synthesized Clawd score: stems, beds, melody, events',+ 'cinetic': 'cinetic synth engine: generator types x seeds / parameters, short mixes, 6 full rendered scores',+ 'generic': 'independent numpy generators (14 types) + 6 generic code-made mixes'}+SCHEME_DOC = {'A': 'A: leave one real category / synth family out (headline)',+ 'A_noaug': 'A without augmented training copies', 'A_clap': 'A, CLAP head only',+ 'A_signal': 'A, signal-measurement head only', 'B': 'B: leave one synth generator type out',+ 'C': 'C: retired score never trained on', 'D': 'D: retired score and cinetic engine never trained on',+ 'M': 'M: as A, but music held out by provider (mk / sb / ch / other)'}+++def _level_md(res):+ lv = res.get('level_sensitivity')+ if not lv:+ return ''+ cs = [c for c in ('-20dB', 'norm-25', 'lifted', 'at-56', 'at-56_lifted') if f'mean_p_{c}' in next(iter(lv.values()))]+ return ('\nLevel (final model, so in-sample clips; it measures sensitivity, not accuracy): the same clips at their own '+ 'level, 20 dB quieter, normalised to -25 dBFS RMS, after the inference lift (sources under '+ f'{realness.LIFT_BELOW_DB:g} dBFS RMS are raised to {realness.LIFT_BELOW_DB:g} dBFS RMS, peak <= -1 dBFS), and '+ 'moved to -56 dBFS RMS without / with that lift (the first build only lifted sources under -60 dBFS RMS, '+ 'so -56 dBFS input was scored as it was).\n\n' ++ _tab(['clips', 'n', 'median RMS dBFS', 'lifted at inference', 'mean p_real own / ' + ' / '.join(cs),+ 'mean abs change ' + ' / '.join(cs), 'acc own / ' + ' / '.join(cs)],+ [[k, v['clips'], v['rms_db_median'], v.get('lifted_clips'),+ ' / '.join(str(v[f'mean_p_{c}']) for c in ['orig'] + cs),+ ' / '.join(str(v[f'mean_abs_dp_{c}']) for c in cs),+ ' / '.join(str(v[f'acc_{c}']) for c in ['orig'] + cs)] for k, v in lv.items()]))+++def _music_md(res):+ mm = res.get('music_M')+ if not mm:+ return ''+ return (f"""+### Pro music+Scheme A holds out all music at once, so the model scoring it has never heard real music while its negatives include+synthesized scores: a pessimistic estimate (music mean p_real {mm['scheme_A_mean_p_real']}). Scheme M holds out one+provider at a time (the model has heard other providers' music): {mm['windows']} windows of {mm['files']} files, mean+p_real {mm['mean_p_real']}, window [email protected] {mm['[email protected]']}, file [email protected] {mm['[email protected]']}; files (mean of windows)+called real: {mm['files_called_real']}, called code-synthesized: {mm['files_called_synth']}. The price of weighting music+as four sources, on everything except music: scheme A AUC {res['non_music']['A']['auc']} / Brier+{res['non_music']['A']['brier']} vs scheme M AUC {res['non_music']['M']['auc']} / Brier {res['non_music']['M']['brier']}.++""" + _tab(['provider fold', 'windows', 'mean p_real', '[email protected]'],+ [[r['group'], r['n'], r['mean_p_real'], r['[email protected]']] for r in mm['per_provider']]))+++def _apps_md(res):+ """the held-out note under the application table, the Windows UI sounds and the layer test"""+ o = []+ ho = res.get('held_out_model', {})+ o.append(f"""+The held-out model ({ho.get('clips')} training clips) never saw {ho.get('excluded')}: the real soundtrack is built from+83 of those accepted recordings, and the synthesized soundtrack is the retired score, so this column is out of sample+on both sides. The final model trained on both (in-sample, optimistic). Verdicts use the file thresholds for files+over 6 s.+""")+ ui = res.get('windows_ui_sounds')+ if ui:+ o.append(f"""+### Professionally made synthetic sounds (Windows UI / alert sounds, never trained on)+{ui['files']} top-level wav files of C:/Windows/Media (chimes, alarms, ringtones, notification tones; made by sound+designers, mostly synthesized): {ui['likely code-synthesized']} called 'likely code-synthesized', {ui['uncertain']}+'uncertain', {ui['real or pro-produced']} 'real or pro-produced' (median p_real {ui['median_p_real']}). Lowest:+{', '.join(f'{n} {q}' for n, q in ui['lowest'])}. Highest: {', '.join(f'{n} {q}' for n, q in ui['highest'])}.+The detector hears synthesis, not provenance: a clean, simple synthesized tone reads as code-made even when a+professional made it and it is licensed. An agent using a licensed UI-sound pack must not treat that verdict as proof.+""")+ lay = res.get('layers')+ if lay:+ S, ba, ml = res['schemes'], res['band'], lay.get('M_layers', {})+ o.append(f"""+### Layer test: a code-synthesized layer inside a real mix+{lay['clips']} clips: {lay['beds']} real 4 s beds (ambience, films, music, foley ...; mean p_real alone+{lay['bed_p_real_mean']}, {lay['beds_lean_synth']} of them under 0.5 alone) with one code-synthesized layer each (a+cinetic / generic / retired clip: one-shots 1-4 times, 4 s windows throughout), at a layer-to-bed RMS of -12 / -6 / 0 /++6 dB (layer RMS over the span it plays). Every clip is scored by a model that never saw its bed's group nor its+layer's family (`--stage layers`). 'base' = the shipped recipe; 'train' = also trained on the layered clips of the+other groups.++""" + _tab(['layer level', 'clips', 'base: mean p_real', 'base: < 0.5', 'base: <= lo', 'base: >= hi (called real)',+ 'train: mean p_real', 'train: < 0.5', 'train: <= lo'],+ [[f"{r['level_db']:+d} dB", r['n'], r['base']['mean_p_real'], r['base']['lean_synth'], r['base']['le_lo'],+ r['base']['ge_hi'], r.get('train', {}).get('mean_p_real'), r.get('train', {}).get('lean_synth'),+ r.get('train', {}).get('le_lo')] for r in lay['per_level']]) + f"""+Most layers are missed: the windows lean synthesized for only about a third of the clips, almost independent of the+layer level, and many are still called real. Per bed group (base, -6..+6 dB, share < 0.5): {lay['per_bed_group_base']}+(magic and motion beds already lean synthesized on their own). Per layer family: {lay['per_layer_family_base']}.+Training on layered clips (scheme M_layers: M plus the layered clips of the other groups) raised the held-out clip AUC+to {ml.get('auc')} (acc {ml.get('[email protected]')}, Brier {ml.get('brier')}; M: {S['M']['auc']} / {S['M']['[email protected]']} /+{S['M']['brier']}) and the layer detection above, but compressed the synthesized scores: at the same 92.5% precision+target its thresholds are lo {ml.get('lo')} / hi {ml.get('hi')} with {ml.get('band', {}).get('coverage')} firm-verdict+coverage (M: {ba['coverage']}). It was not adopted (decided after seeing these held-out numbers): judging source files+firmly is the main use, and even trained, most layers stay unflagged. A dedicated layer head (CLAP logistic, bed vs bed++ layer, same pair folds; a scratch experiment, not in results.json) reached AUC 0.82-0.84 with 40-46% of the layers+at -6..+6 dB found at 5% false alarms on beds, but 11-17% false alarms on film and music beds: not shipped either.+So `earshot realness` says plainly that a clean window does not rule out a code-made layer under a real mix.+""")+ return ''.join(o)+++def markdown(res):+ S, D, th, ba = res['schemes'], res['data'], res['thresholds'], res['band']+ A, fl, bl = S['A'], res['file_level_A'], res['baselines']+ fM = res.get('file_level_M', {})+ fth = fM.get('thresholds', {})+ fm = res.get('final_model', {})+ fu = fm.get('fusion', {'heads': [], 'coef': [0, 0]})+ fool = sorted((r for r in res['per_type_A'] if r['[email protected]'] < 0.75), key=lambda r: r['[email protected]'])+ weak_real = sorted((r for r in res['per_group_A'] if r['label'] == 'real/pro' and r['[email protected]'] < 0.8),+ key=lambda r: r['[email protected]'])+ typeB = {r['group']: r for r in res.get('per_type_B', [])}+ noaug = {(r['perturbation'], r['label']): r['[email protected]'] for r in res.get('robustness_A_noaug', [])}+ robM = {(r['perturbation'], r['label']): r['[email protected]'] for r in res.get('robustness_M', [])}+ grM = {r['group']: r for r in res.get('per_group_M', [])}+ o = [f"""# Realness detector: held-out evaluation++Generated by `eval/realness/run_eval.py --stage report`; every number below comes from that run (`results.json` holds+all of them). The question: is a sound a real recording / professionally produced audio (positive), or amateur audio+synthesized in code: oscillators, filtered noise, simple envelopes, algorithmic reverb, beeps (negative)?+Professionally produced electronic music and pro sound design count as positive.++## Data+{D['clips']} clean clips ({D['real']} real/pro, {D['synth']} synthesized) plus {D['perturbed_clips']} perturbed copies+(robustness tests and training-only augmentation). A clip is a whole sound up to 6 s, else a 4 s window: exactly the+clips `earshot.realness` scores at inference. Audio stays in `out/realness/` (never committed).++""", _tab(['family', 'clean clips', 'what'], [[f, n, FAMILY_DOC.get(f, '')] for f, n in D['by_family'].items()]),+ f"""+Tuning vs evaluation: model choices (regularisation C of the CLAP head, fusion weights) are fitted inside each training+fold by inner grouped CV; nothing is tuned on the held-out group. The verdict thresholds (lo {th['lo']} / hi {th['hi']})+are read from the scheme-{th.get('scheme', 'A')} held-out predictions ({th['how']}), so the band numbers are in-sample+for that choice (two numbers fitted on {A['n']} clips); the file thresholds for long files likewise come from held-out+long files. Design choices made by looking at held-out results across the three build rounds: augmentation on+(robustness below), the precision target, music weighted as four sources (scheme M), the feature set, the file+threshold rule, and not training on layered clips (layer test below).+The final model (`earshot/data/realness_model.json`) is trained on all groups with the scheme-M grouping. Robustness copies are only ever scored, never trained+on; augmentation copies (synthetic noise floor, gating, short excerpts, codec round trips, with randomised parameters+different from the robustness set, never the real room tone) are only trained on.++## Held-out results+Every clip is scored by a model that never saw its group (nor augmented copies of its clips).++""", _tab(['scheme', 'clips', 'ROC-AUC', '[email protected]', 'Brier', 'log loss'],+ [[SCHEME_DOC.get(k, k), v['n'], v['auc'], v['[email protected]'], v['brier'], v['logloss']] for k, v in S.items()]),+ """+D is scored on real vs the retired score and the cinetic engine only (its sole negatives were the generic generators).++### The required check: real recordings rank above the retired synthesized score+The retired score (the synthesized soundtrack the user rejected) is held out of every training set (scheme C), or held+out together with the whole cinetic engine that made it (scheme D):++""", _tab(['check', 'ROC-AUC', '[email protected]', 'mean p_real of the synth side'],+ [[k, v.get('auc'), v.get('[email protected]'), v.get('synth_mean_p')] for k, v in res['retired_check'].items()]),+ f"""+### Verdict bands (scheme {th.get('scheme', 'A')} held-out predictions)+`real or pro-produced` when p_real >= {th['hi']}, `likely code-synthesized` when p_real <= {th['lo']}, else `uncertain`.+Coverage {100 * ba['coverage']:.0f}% (the rest is uncertain); precision of the real verdict {ba['precision_real']}, of the+synth verdict {ba['precision_synth']}; {100 * ba['real_called_synth']:.1f}% of real clips are called synthesized and+{100 * ba['synth_called_real']:.1f}% of synthesized clips are called real. Other precision targets:++""", _tab(['precision target', 'lo', 'hi', 'coverage', 'real called synth', 'synth called real'],+ [[k, v['lo'], v['hi'], v['coverage'], v['real_called_synth'], v['synth_called_real']]+ for k, v in res.get('band_alt', {}).items()]),+ f"\nCalibration (scheme {th.get('scheme', 'A')}, clean clips):\n\n",+ _tab(['p_real bin', 'clips', 'mean p_real', 'fraction real'],+ [[r['bin'], r['n'], r['mean_p'], r['frac_real']] for r in res['calibration']]),+ _music_md(res),+ '\n### Per group (A: the whole group held out; M: the same, music held out by provider)\n\n',+ _tab(['group', 'label', 'clips', 'A mean p_real', 'A [email protected]', 'A AUC vs the other class', 'M mean p_real',+ 'M [email protected]'],+ [[r['group'], r['label'], r['n'], r['mean_p_real'], r['[email protected]'], r['auc_vs_other_class'],+ grM.get(r['group'], {}).get('mean_p_real'), grM.get(r['group'], {}).get('[email protected]')]+ for r in res['per_group_A']]),+ '\n### Synthesized generator types (A: the whole family held out; B: only that type held out)\n\n',+ _tab(['type', 'clips', 'A mean p_real', 'A acc', 'B mean p_real', 'B acc'],+ [[r['group'], r['n'], r['mean_p_real'], r['[email protected]'], typeB.get(r['group'], {}).get('mean_p_real'),+ typeB.get(r['group'], {}).get('[email protected]')] for r in res['per_type_A']]),+ f"""+### File level (long files: the mean of their window p_real)+Sources with >= 2 windows (films, music, ambience, retired stems, full synthesized mixes): {fl['n']} files+({fl['n'] - fl['n_real']} synthesized). Scheme A: AUC {fl['auc']}, [email protected] {fl['[email protected]']}. Scheme M (the final model's+grouping): AUC {fM.get('auc')}, [email protected] {fM.get('[email protected]')}. File verdict thresholds ({fth.get('how')}): lo {fth.get('lo')} /+hi {fth.get('hi')}; with them {fM.get('band', {}).get('coverage')} of the files get a firm verdict, real files called+synthesized {fM.get('band', {}).get('real_called_synth')}, synthesized files called real {fM.get('band', {}).get('synth_called_real')}.+Only {fth.get('synth_files')} synthesized long files from three code bases: weak calibration. Scheme M synthesized files,+lowest to highest: {', '.join(f"{r['src']} {r['p_real']}" for r in fM.get('files', []) if r['label'] == 0)}.+Scheme M real files under 0.6: {', '.join(f"{r['src']} {r['p_real']}" for r in fM.get('files', []) if r['label'] == 1 and r['p_real'] < 0.6) or 'none'}.++### Robustness (perturbed copies, scored by the fold model that never saw their group)+noise70 = pink noise at -70 dBFS RMS added to synthesized clips; room70 = a real room tone (quietest 20 s of a rain+recording) at -70 dBFS; gated = real clips gated to exact digital silence below -45 dB re peak; short = 0.15 s around+the peak; mp3 = 128 kbps round trip. The final model is trained with augmentation (A, M); A_noaug shows the trade-off.++""", _tab(['perturbation', 'label', 'clips', 'acc perturbed (A)', 'acc unperturbed (A)', 'acc perturbed (A_noaug)',+ 'acc perturbed (M)'],+ [[r['perturbation'], r['label'], r['n'], r['[email protected]'], r['acc_unperturbed'],+ noaug.get((r['perturbation'], r['label'])), robM.get((r['perturbation'], r['label']))]+ for r in res['robustness_A']]),+ _level_md(res),+ f"""+### Baselines (no training)+- CLAP zero-shot (log-sum-exp over 6 'real' prompts minus 6 'synth' prompts): AUC {bl['clap_zero_shot']['auc']},+ [email protected] {bl['clap_zero_shot']['[email protected]']} on all clean clips.+- Single signal measurements (AUC, physical sign): {', '.join(f'{k} {v}' for k, v in bl['signal_univariate_auc'].items())}.+"""]+ sub = res.get('ast_subset')+ if sub:+ o.append(f"""- AST, on a stratified subset of {sub['A_on_subset']['n']} clean clips (AST costs ~5x CLAP here; the subset predates+ the full synthesized mixes): max over the synth classes ({', '.join(sub['ast_synth_classes_max']['classes'])})+ AUC {sub['ast_synth_classes_max']['auc']}; single classes (high = real):+ {', '.join(f"{k.split(':')[1]} {v['auc_real_high']}" for k, v in sub.items() if k.startswith('ast_single'))}.+ On the same subset: scheme A {sub['A_on_subset']['auc']}, CLAP zero-shot {sub['clap_zero_shot_on_subset']['auc']},+ clap+signal heads refit on the subset {sub.get('S', {}).get('auc')}, plus an AST head {sub.get('S_ast', {}).get('auc')},+ AST head alone {sub.get('S_astonly', {}).get('auc')}. An AST head adds nothing, so AST is evidence only (`--ast`).+""")+ o.append(f"""+## What carries the decision+- Fusion weights (fitted on inner held-out head outputs): {dict(zip(fu['heads'], fu['coef']))}. The CLAP head carries+ the decision; the signal head ({S['A_signal']['auc']} AUC alone vs {S['A_clap']['auc']} for CLAP alone) gets weight+ {fu['coef'][1]}, so its measurements are reported as evidence (each with the share of training clips past that+ threshold that were of that side), not votes.+- The CLAP head's weight direction is closest to these vocabulary texts. Toward real:+ {', '.join(fm.get('clap_direction', {}).get('toward_real', []))}. Toward synthesized:+ {', '.join(fm.get('clap_direction', {}).get('toward_synth', []))}.+- Signal-head coefficients (standardised, sign-constrained to their physics): {fm.get('signal_coef')}.+- Exact digital silence is weak evidence: over half of the clips of both classes contain some (edited and gated library+ recordings too), so it is never a verdict reason, only context quoted with its training base rates.+- One rule overrides the model at inference (earshot.realness.pure_tone): a pure, dead-steady tone (one spectral line in+ every frame, crest <= 4.5 dB, no pitch drift) gets p_real <= 0.03, because bare test tones lie outside the training+ data. Checked by `eval/realness/tone_guard.py` (`tone_guard.json`): a 30 s 440 Hz sine gets 0.48 from the model+ alone and 0.03 with the rule; the rule fires on none of 4741 audible clips of the 345 real recordings.++## Application files (whole soundtracks, 4 s windows)+""")+ o.append(_tab(['file', 'held-out model: p_real (mean) / verdict', 'windows < 0.5 / <= lo / all (held-out)',+ 'final model (in-sample): p_real / verdict'],+ [[k, f"{v['held_out']['p_real_mean']} / {v['held_out']['verdict']}",+ f"{v['held_out']['windows_lean_synth']} / {v['held_out']['windows_le_lo']} / {v['held_out']['windows']}",+ f"{v['final']['p_real_mean']} / {v['final']['verdict']}"]+ for k, v in res.get('applications', {}).items()]))+ o.append(_apps_md(res))+ o.append(f"""++## Limits: what fools it+- Synthesized types it misses most (scheme A acc < 0.75, lowest first; acc / mean p_real): {', '.join(f"{r['group']} ({r['[email protected]']} / {r['mean_p_real']})" for r in fool) or 'none'}.+ Code-made noise textures that imitate nature (rain, wind beds) and telephone-like tones (DTMF, sirens) sound like+ recordings of real things to CLAP, and a generic code-made mix built on such beds can score real.+- Real groups it doubts (scheme A acc < 0.8; acc / mean p_real): {', '.join(f"{r['group']} ({r['[email protected]']} / {r['mean_p_real']})" for r in weak_real) or 'none'}.+ Library "magic" and "motion" sounds are themselves designed (often synthesized) by sound designers, and real+ electronic music sits near synthesized scores: read 'uncertain' there as "sounds designed", not "fake".+- Finished mixes are harder than sources, and a code-made layer under a louder real mix is usually missed (layer+ test). Judge the source files and stems, not only the final mix.+- Professionally made synthetic tones (UI sounds, chimes, alarms) are called code-synthesized: it hears synthesis, not+ provenance (Windows UI sounds above).+- A -70 dBFS noise floor added to synthesized audio lowers accuracy (robustness table): the detector mostly hears what+ a sound is, not microphone physics. Very short clips (< 0.2 s) of real sounds are harder.+- Training data: one sound library (Freesound / Mixkit picks for one film), 7 launch films, ~120 music windows; the+ negatives come from three code bases (cinetic, the retired score made with it, generic numpy generators). Synthesis+ from other tools (pro soft synths, granular, sample-based) was never seen: pro-produced synth music is meant to score+ real, and amateur code using techniques outside these three code bases may also score real.++## Reproduce+```+.venv/Scripts/python.exe eval/realness/build_data.py # clips -> out/realness/+.venv/Scripts/python.exe eval/realness/build_data.py --only layers # layer-test clips+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/extract.py --ast 240 # repeat until "done"+OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage cv # until "cv done"+OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage layers # twice+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage apps+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/train.py+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/run_eval.py --stage level # needs the final model+.venv/Scripts/python.exe eval/realness/run_eval.py --stage report # results.json + RESULTS.md+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/train.py # embeds the summary in the model+```+""")+ return ''.join(o)+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])+ ap.add_argument('--stage', default='cv', choices=['cv', 'layers', 'apps', 'level', 'report'])+ ap.add_argument('--budget', type=float, default=480)+ a = ap.parse_args(argv)+ if a.stage == 'cv':+ run_cv(a.budget)+ elif a.stage == 'layers':+ run_layers()+ elif a.stage == 'apps':+ run_apps()+ elif a.stage == 'level':+ run_level()+ else:+ report()+++if __name__ == '__main__':+ main()diff --git a/eval/realness/tone_guard.json b/eval/realness/tone_guard.jsonnew file mode 100644index 0000000..ce3cedd--- /dev/null+++ b/eval/realness/tone_guard.json@@ -0,0 +1,56 @@+{+ "sine_440hz_30s": {+ "p_real_model_only": 0.478,+ "p_real_with_rule": 0.03,+ "windows": 8,+ "rule_fired": 8+ },+ "real_recordings": {+ "files": 345,+ "audible_clips": 4741,+ "rule_fired": 0,+ "fired_on": [],+ "closest": [+ {+ "file": "magic/fs-549899_small-wine-glass-tap.ogg",+ "start": 0.0,+ "tests_passed": 3,+ "crest_db": 20.4,+ "line_share": 0.99,+ "tonal_frames": 1.0+ },+ {+ "file": "_rejected/motion/fs-428073_sub-drop-smooth.ogg",+ "start": 0.0,+ "tests_passed": 2,+ "crest_db": 5.5,+ "line_share": 1.0,+ "tonal_frames": 1.0+ },+ {+ "file": "_rejected/music/sb-quirkylane_whimsical-winds.mp3",+ "start": 16.0,+ "tests_passed": 2,+ "crest_db": 8.2,+ "line_share": 0.95,+ "tonal_frames": 0.99+ },+ {+ "file": "_rejected/music/mk-506_little-bells.mp3",+ "start": 0.0,+ "tests_passed": 2,+ "crest_db": 9.5,+ "line_share": 0.99,+ "tonal_frames": 1.0+ },+ {+ "file": "_rejected/music/sb-quirkylane_whimsical-winds.mp3",+ "start": 0.0,+ "tests_passed": 2,+ "crest_db": 9.9,+ "line_share": 1.0,+ "tonal_frames": 0.97+ }+ ]+ }+}\ No newline at end of filediff --git a/eval/realness/tone_guard.py b/eval/realness/tone_guard.pynew file mode 100644index 0000000..45e4871--- /dev/null+++ b/eval/realness/tone_guard.py@@ -0,0 +1,83 @@+"""Check the pure-tone rule in earshot.realness: what it does to a test tone, and that it never fires on a real recording.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/tone_guard.py # -> eval/realness/tone_guard.json++1. a 30 s 440 Hz sine at -10.5 dBFS (the eval/launch negative): mean p_real over its 4 s windows from the model alone+ (CLAP head + fusion, no rule) and with the rule.+2. every real recording of the Clawd project (the 219 used + 126 rejected Freesound / Mixkit files under+ EP.ASSETS, see eval/README.md, read-only), clipped exactly as realness() clips it (whole sound up to 6 s, else+ 4 s windows): on how many clips does pure_tone() fire, and which audible clips come closest (fewest-failed tests).+No model is needed for 2 (signal measurements only).+"""+import json+import os+import sys++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+from earshot import audio # noqa: E402+from earshot import realness as R # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++ASSETS = EP.ASSETS+++def _closeness(f):+ """how many of the rule's four tests a clip passes (4 = the rule fires)"""+ w = f.get('pitch_wobble', float('nan'))+ return (int(f.get('tonal_frames', 0) >= 0.95) + int(f.get('line_share', 0) >= 0.9) + int(f.get('crest_db', 99) <= 4.5)+ + int(not np.isfinite(w) or w <= -1.0))+++def real_files():+ out = []+ for root, dirs, files in os.walk(ASSETS):+ dirs.sort()+ out += [os.path.join(root, f) for f in sorted(files) if os.path.splitext(f)[1].lower() in audio.AUDIO_EXTS]+ return out+++def main():+ res = {}+ t = np.arange(30 * audio.SR) / audio.SR+ sine = (10 ** (-10.5 / 20) * np.sin(2 * np.pi * 440 * t)).astype(np.float32)+ m = R.load_model()+ rows = R.score_clips([c for _, _, c in R._clips(R.lift_quiet(sine)[0])], m)+ model_only = [R.fuse(m['fusion'], r['logits'])[0] for r in rows]+ res['sine_440hz_30s'] = {'p_real_model_only': round(float(np.mean(model_only)), 3),+ 'p_real_with_rule': round(float(np.mean([r['p_real'] for r in rows])), 3),+ 'windows': len(rows), 'rule_fired': sum(bool(r.get('tone')) for r in rows)}+ print('sine', res['sine_440hz_30s'], flush=True)+ files = real_files()+ fired, n_clips, close = [], 0, []+ for i, p in enumerate(files):+ x = R.lift_quiet(audio.load(p, mono=True))[0]+ for a, b, c in R._clips(x):+ if not R.audible(c):+ continue+ f = R.signal_features(c)+ if f is None:+ continue+ n_clips += 1+ k = _closeness(f)+ if R.pure_tone(f):+ fired.append({'file': os.path.relpath(p, ASSETS), 'start': a})+ elif k >= 2:+ close.append({'file': os.path.relpath(p, ASSETS).replace('\\', '/'), 'start': round(a, 1),+ 'tests_passed': k, 'crest_db': round(float(f.get('crest_db', 0)), 1),+ 'line_share': round(float(f.get('line_share', 0)), 2),+ 'tonal_frames': round(float(f.get('tonal_frames', 0)), 2)})+ if i % 50 == 0:+ print(f'{i}/{len(files)} files, {n_clips} clips, fired {len(fired)}', flush=True)+ close.sort(key=lambda r: (-r['tests_passed'], r['crest_db']))+ res['real_recordings'] = {'files': len(files), 'audible_clips': n_clips, 'rule_fired': len(fired),+ 'fired_on': fired[:20], 'closest': close[:5]}+ json.dump(res, open(os.path.join(HERE, 'tone_guard.json'), 'w'), indent=1)+ print(json.dumps(res['real_recordings'], indent=1)[:1500])+++if __name__ == '__main__':+ main()diff --git a/eval/realness/train.py b/eval/realness/train.pynew file mode 100644index 0000000..3dc4c4d--- /dev/null+++ b/eval/realness/train.py@@ -0,0 +1,140 @@+"""Fit the final realness model on every group and write earshot/data/realness_model.json (plain JSON, no pickle).++ OMP_NUM_THREADS=2 EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/realness/train.py++Uses all clean clips plus their augmented training copies (common.fit_model: C of the CLAP head and the fusion weights+from inner grouped CV), grouped like run_eval scheme M (each source weighs the same; music = four providers). Verdict+thresholds come from the held-out scheme-M predictions (out/realness/oof/M.npz, made by run_eval.py --stage cv):+hi / lo = the p_real above / below which >= 92.5% (run_eval.TARGET) of held-out clips were real / synthesized; the+file thresholds (for the mean of a long file's windows) come from the same predictions averaged per long source+(run_eval.file_thresholds). Layered clips (part 'layers') are not trained on: run_eval.py --stage layers tested it.+Also stores which vocabulary descriptions the CLAP head's direction points to (an explanation of what it learned).+"""+import hashlib+import json+import os+import sys++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, HERE)+import common as C # noqa: E402+import run_eval as R # noqa: E402+from earshot import config, realness # noqa: E402+++def rounded(h, nd=6):+ return {k: ([round(float(v), nd) for v in x] if isinstance(x, list) and x and not isinstance(x[0], list) else+ [[round(float(v), nd) for v in r] for r in x] if isinstance(x, list) and x and isinstance(x[0], list) else+ round(float(x), nd) if isinstance(x, float) else x) for k, x in h.items()}+++def clap_direction(head, k=8):+ """vocabulary terms most aligned with the CLAP head's real (+) and synth (-) directions"""+ from earshot.models import clap+ voc = json.load(open(os.path.join(config.DATA_DIR, 'vocab.json')))+ terms = sorted({t for v in voc['categories'].values() for t in v} | set(realness.REAL_PROMPTS + realness.SYNTH_PROMPTS))+ T = clap.get().embed_text(terms)+ w = np.asarray(head['coef']) / np.asarray(head['scale'])+ s = T @ (w / np.linalg.norm(w))+ o = np.argsort(s)+ return {'toward_real': [terms[i] for i in o[::-1][:k]], 'toward_synth': [terms[i] for i in o[:k]],+ 'how': 'cosine between each vocabulary text embedding and the CLAP head weight direction (coef / scale)'}+++def facts(d, min_prec=0.9, min_support=25):+ """per measurement and side, the loosest threshold past which >= min_prec of the clean training clips belong to+ that side (support >= min_support clips): the human-readable signal facts realness() reports. Exact digital+ silence is never a fact: edited / gated library recordings have it as often as synthesizers (see silence_base)"""+ out = []+ for j, k in enumerate(C.SIG_INPUTS):+ if k == 'silence_frac':+ continue+ v = d.Xs[:, j]+ ok = np.isfinite(v)+ v, y = v[ok], d.y[ok]+ for side in ('synth', 'real'):+ toward_real_is_big = C.SIGNS[k] > 0+ big = (side == 'real') == toward_real_is_big+ best = None+ for t in np.unique(np.percentile(v, np.arange(1, 50) if not big else np.arange(51, 100))):+ m = v >= t if big else v <= t+ prec = float(np.mean(y[m] == (side == 'real'))) if m.any() else 0.0+ if m.sum() >= min_support and prec >= min_prec and (best is None or m.sum() > best[2]):+ best = (float(t), prec, int(m.sum()))+ if best:+ out.append({'feature': k, 'side': side, 'op': '>=' if big else '<=', 'value': round(best[0], 4),+ 'precision': round(best[1], 3), 'support': best[2]})+ return out+++def silence_base(d, levels=(0.005, 0.02, 0.05, 0.1, 0.2, 0.3, 0.5)):+ """share of clean training clips per class with at least this fraction of exact digital silence"""+ v = d.Xs[:, C.SIG_INPUTS.index('silence_frac')]+ return [{'at_least': t, 'real': round(float(np.mean(v[d.y == 1] >= t)), 3),+ 'synth': round(float(np.mean(v[d.y == 0] >= t)), 3)} for t in levels]+++def main():+ d = C.load()+ train = (d.perturb == '') | d.augment+ dt = d.sub(train)+ g = R.groups_of(dt, 'music_src')+ m = C.fit_model(dt, g)+ clean = d.perturb == ''+ oof = R.load_oof('M', d)+ if oof is None:+ raise SystemExit('run run_eval.py --stage cv first (thresholds come from held-out predictions)')+ hi, lo = R.thresholds(d.y[clean], oof[clean])+ _, fy, fp, _ = R.file_level(d, oof)+ fth = R.file_thresholds(fy, fp, hi, lo)+ heads = {'clap': dict(rounded(m['heads']['clap']), inputs='clap_embedding_512'),+ 'signal': dict(rounded(m['heads']['signal']), inputs=list(C.SIG_INPUTS))}+ ids = sorted(dt.ids)+ res_path = os.path.join(HERE, 'results.json')+ ev = {}+ if os.path.exists(res_path):+ r = json.load(open(res_path))+ ev = {'scheme_A': r['schemes'].get('A'), 'scheme_M': r['schemes'].get('M'), 'band': r.get('band'),+ 'retired_check': r.get('retired_check'), 'layers': r.get('layers', {}).get('summary'),+ 'file_level_M': {k: r.get('file_level_M', {}).get(k) for k in ('n', 'auc', '[email protected]', 'band')},+ 'see': 'eval/realness/RESULTS.md'}+ model = {+ 'version': 1,+ 'trained': hashlib.sha1('|'.join(ids).encode()).hexdigest()[:12],+ 'about': 'p_real = sigmoid(fusion over head logits); heads are standardised logistic regressions. '+ 'clap head: laion/larger_clap_general audio embedding (repeat-padded, one window per clip <= 6 s, else '+ '4 s windows). signal head: earshot.realness.signal_features, coefficients sign-constrained to their '+ 'physical direction. Positives: real recordings (Freesound/Mixkit), pro launch-film soundtracks, pro '+ 'music; negatives: code-synthesized audio (cinetic synth engine, a retired synthesized film score, '+ 'generic numpy generators).',+ 'data': {'clips': int(len(dt)), 'clean': int((~dt.augment).sum()), 'augmented': int(dt.augment.sum()),+ 'real': int(dt.y[~dt.augment].sum()), 'synth': int((1 - dt.y[~dt.augment]).sum()),+ 'groups': sorted(set(g))},+ 'long_s': realness.LONG_S, 'win_s': realness.WIN_S, 'silent_db': realness.SILENT_DB,+ 'heads': heads,+ 'fusion': {'heads': m['fusion']['heads'], 'coef': [round(c, 6) for c in m['fusion']['coef']],+ 'intercept': round(m['fusion']['intercept'], 6)},+ 'C': m['C'],+ 'thresholds': {'hi': hi, 'lo': lo},+ 'file_thresholds': fth,+ 'prompts': {'real': realness.REAL_PROMPTS, 'synth': realness.SYNTH_PROMPTS},+ 'facts': facts(d.sub(clean)),+ 'silence_base': silence_base(d.sub(clean)),+ 'clap_direction': clap_direction(m['heads']['clap']),+ 'eval': ev,+ }+ with open(realness.MODEL_PATH, 'w', encoding='utf-8') as fh:+ json.dump(model, fh, indent=1)+ print(f"wrote {realness.MODEL_PATH}: {len(dt)} clips, C {m['C']}, fusion {model['fusion']}, thresholds lo {lo} hi {hi}, "+ f"file thresholds lo {fth['lo']} hi {fth['hi']}")+ print('signal coef:', dict(zip(C.SIG_INPUTS, heads['signal']['coef'])))+ print('clap direction:', model['clap_direction'])+ for q in model['facts']:+ print('fact:', q)+ print('silence base rates:', model['silence_base'])+++if __name__ == '__main__':+ main()diff --git a/eval/semantics/RESULTS.md b/eval/semantics/RESULTS.mdnew file mode 100644index 0000000..3976037--- /dev/null+++ b/eval/semantics/RESULTS.md@@ -0,0 +1,103 @@+# earshot semantics: evaluation results++All numbers come from `eval/semantics/run_eval.py` (commands below) with CLAP `laion/larger_clap_general` (bf16, CPU,+`EARSHOT_THREADS=2`). Raw outputs: `results.json` (tuning library), `heldout.json` (held-out library), `kind.json`+(query-kind reader), `find_repeatpad.json` (find), `timing.json`.++## Data, and what was tuned on what+| set | files | labels | used for |+|---|---|---|---|+| tuning library: `<assets>/<8 folders>/` | 219 | manifests (need -> 35 classes, `queries.json`) | choosing pad / templates / calibration, the verdict thresholds, find's heuristics. 67 of the 322 vocabulary terms were added after seeing its classes, so its numbers are **optimistic** |+| held-out library: `.../assets/_rejected/<8 folders>/` | 125 (126 minus a duplicate) | folder + file name only, written **before** any score on these files (`heldout_labels.json`) | nothing: the honest test of the shipped configuration |+| term-bias references (unsupervised: only their mean embedding) | tuning analysis: held-out library + 7 launch films + Clawd mixes + synth stems (148); shipped `vocab_stats.json`: tuning library + films + mixes + synth stems (241) | none | always disjoint from the files being scored (the Clawd mixes do contain some tuning clips) |++Vocabulary history: 217 terms written before any evaluation (`vocab_preregistered.json`), +67 after seeing the tuning+classes, +38 general everyday terms (vehicles, household, tools, animals, people) from general knowledge; frozen+before the held-out library was scored.++## Calibration variants (tuning library, 219 files)+Pooled AUC = one threshold for every (recording, class description) pair; lenient leaves out pairs of related classes+(Enter key vs 'a spacebar press'), strict counts them as wrong. Per-split (dev/test) rows for every variant are in+`results.json`.++| pad | templates | calibration | cat MAP | class MAP | class top-3 | tagging acc | AUC | AUC strict |+|---|---|---|---|---|---|---|---|---|+| repeatpad | 1 | raw cosine | 0.698 | 0.597 | 0.843 | 0.553 | 0.944 | 0.927 |+| repeatpad | 5 | raw cosine | 0.728 | 0.627 | 0.886 | 0.594 | 0.951 | 0.936 |+| repeatpad | 5 | per-sound z | 0.770 | **0.650** | **0.900** | 0.594 | 0.963 | 0.948 |+| repeatpad | 5 | term bias only | 0.728 | 0.627 | 0.886 | 0.580 | 0.954 | 0.939 |+| **repeatpad** | **5** | **term bias + per-sound z (shipped)** | **0.778** | 0.642 | 0.886 | 0.580 | **0.966** | **0.951** |+| repeatpad | 5 | std-divided + per-sound z | 0.768 | 0.629 | 0.900 | 0.578 | 0.964 | 0.949 |+| zero | 5 | shipped calibration | 0.758 | 0.631 | 0.843 | 0.591 | 0.964 | 0.949 |+| spaced | 5 | shipped calibration | 0.784 | 0.644 | 0.871 | 0.589 | 0.965 | 0.950 |++The per-sound z does most of the work; the term bias adds a little to the pooled AUC (the thing verdicts rely on) and+costs a little class MAP. Dev / test for the shipped variant: AUC 0.973 / 0.956, class MAP 0.678 / 0.639. Specific+needs (24 concrete queries): a correct file in the top 3 for 23/24 (miss: 'a cork pop', mouth pops rank first), at+rank 1 for 19/24. Reference-set sensitivity (dev / test AUC): folder-balanced 0.973 / 0.956, uniform 0.972 / 0.957,+without music-like refs 0.974 / 0.959, transductive upper bound 0.974 / 0.961, no term bias 0.971 / 0.953.++## Verdicts+Rule (on the tuning library, lenient labels, 0.5 z grid, top-down): strong = lowest z with LR(z >= t) >= 100,+plausible = lowest z below it whose band LR >= 20, weak = lowest z below that whose band LR >= 5. It gives+**strong z >= 3, plausible 2.5-3, weak 2-2.5, no match < 2**. LR = P(band | true pair) / P(band | wrong pair);+precision at a library base rate of ~3% and at even prior odds follow from it.++Per band (not cumulative), held-out library, shipped configuration (248 true pairs; 7614 wrong lenient / 8222 strict):++| verdict | z | LR lenient | precision @3% | precision even odds | LR strict | precision @3% strict | recall at or above |+|---|---|---|---|---|---|---|---|+| strong | >= 3 | 1289 (126 true / 3 wrong) | 0.98 | 1.00 | 51 | 0.61 | 0.51 |+| plausible | 2.5-3 | 21 | 0.41 | 0.95 | 9.1 | 0.22 | 0.63 |+| weak | 2-2.5 | 12 | 0.28 | 0.92 | 6.6 | 0.17 | 0.76 |+| no match | < 2 | 0.25 | 0.008 | 0.20 | 0.25 | 0.008 | - |++Tuning library, same bands (lenient / strict LR): strong 170 / 40, plausible 29 / 12, weak 7.1 / 4.1, no match 0.18.+The first build's 'weak' band (z 1-2) had LR 0.98: no evidence at all, which is why it moved to 2-2.5.++## Held-out library (125 files, nothing tuned on it)+| vocabulary | cat MAP | class MAP | class top-3 | tagging acc | AUC | AUC strict |+|---|---|---|---|---|---|---|+| shipped (322 terms) | 0.691 | 0.646 | 0.827 | 0.707 | **0.980** | **0.965** |+| pre-registered (217 terms) | 0.685 | **0.669** | 0.827 | 0.707 | 0.976 | 0.960 |+| raw cosine (no calibration) | 0.646 | 0.700 | 0.788 | 0.727 | 0.978 | 0.964 |++The vocabulary extension carries over to new recordings for verdicts (strong precision 0.926 -> 0.977 lenient,+0.543 -> 0.606 strict) but not for ranking files (class MAP 0.669 -> 0.646). Raw cosine ranks files for a class query+better here (class MAP 0.700) though worse on the tuning library (0.627); calibration is kept for its absolute verdicts.+Category AP: ambience 0.92, keyboard 0.98, magic 0.98, music 1.00, motion 0.82, garden 0.45, toy 0.20, foley 0.18.++Music mood (unvalidated tags): on 45 held-out music tracks, all chosen for a bright, playful film, the top mood tag was+a dark / sad / tense one for 18% (chance 31%; tuning library 0 of 8). A music-specific term bias did not help (20%),+so mood stays an explicitly unvalidated hint.++## find()+| test | result |+|---|---|+| Keychron, 17 single presses (`fs-789628`, manifest slices), top 17 | 16 usable trims on 16 distinct presses; the 17th is a false positive on the stray double hit at 9.54 s the manifest says to skip. Attack measured independently (first sample at -20 dB of the slice peak): trims start 5-54 ms before it (median 16 ms), never after |+| Mars board, 32 presses (`fs-609534`), top 10 | 9/10 usable trims, lead 6-8 ms |+| typing recording (`fs-709460`: slow presses 0-17 s, then fast typing), top 5 | single-press query: 2 on labelled slow presses, 3 strokes cut from the fast part; continuous-typing query: 5/5 windows in the fast part |+| 44 s scene: 8 real one-shots over room tone | top-1 on the right event 8/8; 7 tight trims (lead 5-66 ms), the small bell came back as a 5 s window |+| 300 s scene: a key press every ~1.3 s throughout + glass tings at 20 / 150 / 275 s, cork pop 205 s, whoosh 240 s | 5/5 found as tight trims (the first build never looked past ~95 s) |++Query kind (`kind.json`): thresholds from the tuning phrasings (event = lowest event score, texture = median texture+score). Generic test queries: 25/30 one-shots read as one-shots, 4 as either (both candidate kinds scored), 1 as+continuous ('a bird chirp': windows only); 14/20 continuous sounds windows-only, 6 either, none read as a one-shot.++## Timings (EARSHOT_THREADS=2, bf16, shared Ryzen 7 260; models warm, nothing cached)+model load 27 s; describe 0.56 s key 17.0 s; describe 31 s rain 52.5 s; match, new query on a cached file 2.0 s;+find 18 s file 29.9 s cold, 1.2 s new query; find 300 s file 61.8 s cold; rank 20 foley files (804 s of audio) 55.8 s+cold (2.8 s per file), 3.4 s new query. A CLI `match` on a cached file and query loads no model: 1.5 s wall.++## Commands+```+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py embed --set heldout --part 1/2 # and 2/2+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py stats+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py analyze+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py heldout+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py kind+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py find+EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/run_eval.py timing+```+(tuning-library and tuning-reference embeddings for the three pad modes were cached by `embed --set eval|tuneref --pad+repeatpad|zero|spaced` in the first build.)diff --git a/eval/semantics/find_repeatpad.json b/eval/semantics/find_repeatpad.jsonnew file mode 100644index 0000000..e7fd204--- /dev/null+++ b/eval/semantics/find_repeatpad.json@@ -0,0 +1,844 @@+{+ "pad": "repeatpad",+ "keychron_17_keys": {+ "hits": 16,+ "n": 17,+ "distinct_presses": 16,+ "in_unlabelled_part": 0,+ "false_positives_in_skip_region": 1,+ "median_lead_ms": 15.65,+ "lead_ms_range": [+ 5.3,+ 54.3+ ],+ "rows": [+ {+ "start": 10.601,+ "end": 10.855,+ "z": 5.37,+ "kind": "event",+ "tail_db": -38.3,+ "press": 10,+ "lead_ms": 12.5,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 5.933,+ "end": 6.295,+ "z": 5.26,+ "kind": "event",+ "tail_db": -40.5,+ "press": 6,+ "lead_ms": 41.0,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 0.206,+ "end": 0.43,+ "z": 5.25,+ "kind": "event",+ "tail_db": -40.3,+ "press": 0,+ "lead_ms": 46.4,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 4.939,+ "end": 5.145,+ "z": 5.23,+ "kind": "event",+ "tail_db": -31.6,+ "press": 5,+ "lead_ms": 15.5,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 14.755,+ "end": 15.05,+ "z": 5.23,+ "kind": "event",+ "tail_db": -31.1,+ "press": 13,+ "lead_ms": 5.4,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 3.945,+ "end": 4.19,+ "z": 5.18,+ "kind": "stroke",+ "tail_db": -28.1,+ "press": 4,+ "lead_ms": 54.3,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 7.72,+ "end": 8.07,+ "z": 5.08,+ "kind": "event",+ "tail_db": -40.8,+ "press": 8,+ "lead_ms": 10.4,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 11.897,+ "end": 12.195,+ "z": 5.04,+ "kind": "event",+ "tail_db": -39.2,+ "press": 11,+ "lead_ms": 35.8,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 13.462,+ "end": 13.665,+ "z": 5.03,+ "kind": "stroke",+ "tail_db": -31.3,+ "press": 12,+ "lead_ms": 15.8,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 8.593,+ "end": 8.88,+ "z": 5.02,+ "kind": "event",+ "tail_db": -39.5,+ "press": 9,+ "lead_ms": 5.3,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 15.66,+ "end": 15.89,+ "z": 4.95,+ "kind": "event",+ "tail_db": -35.2,+ "press": 14,+ "lead_ms": 14.1,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 6.854,+ "end": 7.135,+ "z": 4.95,+ "kind": "event",+ "tail_db": -37.5,+ "press": 7,+ "lead_ms": 5.6,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 2.971,+ "end": 3.205,+ "z": 4.9,+ "kind": "stroke",+ "tail_db": -41.8,+ "press": 3,+ "lead_ms": 37.5,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 16.986,+ "end": 17.2,+ "z": 4.89,+ "kind": "stroke",+ "tail_db": -28.1,+ "press": 16,+ "lead_ms": 5.5,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 2.033,+ "end": 2.36,+ "z": 4.84,+ "kind": "event",+ "tail_db": -41.7,+ "press": 2,+ "lead_ms": 34.2,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 9.544,+ "end": 9.832,+ "z": 4.82,+ "kind": "event",+ "tail_db": -15.0,+ "press": 9,+ "lead_ms": -945.7,+ "in_skip_region": true,+ "hit": false+ },+ {+ "start": 16.32,+ "end": 16.61,+ "z": 4.73,+ "kind": "event",+ "tail_db": -35.3,+ "press": 15,+ "lead_ms": 17.7,+ "in_skip_region": false,+ "hit": true+ }+ ],+ "seconds": 4.2,+ "seconds_cached": 0.17,+ "duration": 18.204,+ "n_candidates": 59,+ "query": "a single mechanical keyboard key press"+ },+ "creamy_typing_slow_then_fast": {+ "hits": 2,+ "n": 5,+ "distinct_presses": 2,+ "in_unlabelled_part": 3,+ "false_positives_in_skip_region": 0,+ "median_lead_ms": 13.8,+ "lead_ms_range": [+ 5.8,+ 21.8+ ],+ "rows": [+ {+ "start": 39.542,+ "end": 39.82,+ "z": 4.14,+ "kind": "event",+ "tail_db": -32.3,+ "press": null,+ "lead_ms": null,+ "in_skip_region": false,+ "hit": false+ },+ {+ "start": 13.235,+ "end": 13.345,+ "z": 4.07,+ "kind": "stroke",+ "tail_db": -37.0,+ "press": 14,+ "lead_ms": 5.8,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 11.118,+ "end": 11.425,+ "z": 3.73,+ "kind": "event",+ "tail_db": -36.5,+ "press": 11,+ "lead_ms": 21.8,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 42.444,+ "end": 42.56,+ "z": 3.66,+ "kind": "stroke",+ "tail_db": -29.7,+ "press": null,+ "lead_ms": null,+ "in_skip_region": false,+ "hit": false+ },+ {+ "start": 29.836,+ "end": 29.952,+ "z": 3.65,+ "kind": "stroke",+ "tail_db": -30.0,+ "press": null,+ "lead_ms": null,+ "in_skip_region": false,+ "hit": false+ }+ ],+ "seconds": 0.3,+ "seconds_cached": 0.28,+ "duration": 47.864,+ "n_candidates": 67,+ "query": "a single mechanical keyboard key press",+ "continuous_typing_query": {+ "in_fast_part": 5,+ "n": 5,+ "segments": [+ [+ 20.0,+ 25.0,+ 3.88,+ "window"+ ],+ [+ 32.5,+ 37.5,+ 3.83,+ "window"+ ],+ [+ 37.5,+ 42.5,+ 3.6,+ "window"+ ],+ [+ 42.864,+ 47.864,+ 3.5,+ "window"+ ],+ [+ 25.0,+ 30.0,+ 3.48,+ "window"+ ]+ ]+ }+ },+ "mars_32_keys": {+ "hits": 9,+ "n": 10,+ "distinct_presses": 9,+ "in_unlabelled_part": 0,+ "false_positives_in_skip_region": 0,+ "median_lead_ms": 6.3,+ "lead_ms_range": [+ 5.8,+ 8.1+ ],+ "rows": [+ {+ "start": 0.987,+ "end": 1.335,+ "z": 5.79,+ "kind": "event",+ "tail_db": -31.9,+ "press": 1,+ "lead_ms": 75.1,+ "in_skip_region": false,+ "hit": false+ },+ {+ "start": 5.031,+ "end": 5.52,+ "z": 5.64,+ "kind": "event",+ "tail_db": -39.3,+ "press": 5,+ "lead_ms": 6.7,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 25.369,+ "end": 25.945,+ "z": 5.43,+ "kind": "event",+ "tail_db": -30.8,+ "press": 25,+ "lead_ms": 6.3,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 0.114,+ "end": 0.715,+ "z": 5.33,+ "kind": "event",+ "tail_db": -28.9,+ "press": 0,+ "lead_ms": 8.1,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 4.121,+ "end": 4.64,+ "z": 5.29,+ "kind": "event",+ "tail_db": -45.3,+ "press": 4,+ "lead_ms": 6.0,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 2.061,+ "end": 2.775,+ "z": 5.24,+ "kind": "event",+ "tail_db": -32.7,+ "press": 2,+ "lead_ms": 5.8,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 26.963,+ "end": 27.405,+ "z": 5.19,+ "kind": "event",+ "tail_db": -34.7,+ "press": 27,+ "lead_ms": 7.2,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 3.053,+ "end": 3.775,+ "z": 5.17,+ "kind": "event",+ "tail_db": -35.7,+ "press": 3,+ "lead_ms": 6.1,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 9.979,+ "end": 10.28,+ "z": 5.16,+ "kind": "event",+ "tail_db": -42.6,+ "press": 10,+ "lead_ms": 6.0,+ "in_skip_region": false,+ "hit": true+ },+ {+ "start": 28.448,+ "end": 28.725,+ "z": 5.09,+ "kind": "event",+ "tail_db": -34.4,+ "press": 29,+ "lead_ms": 6.5,+ "in_skip_region": false,+ "hit": true+ }+ ],+ "seconds": 0.4,+ "seconds_cached": 0.3,+ "duration": 31.463,+ "n_candidates": 60,+ "query": "a single mechanical keyboard key press"+ },+ "scene_44s": {+ "found": 8,+ "n": 8,+ "top1": 8,+ "n_queries": 8,+ "tight_trims": 7,+ "seconds_all_queries": 19.6,+ "rows": [+ {+ "query": "a glass ting",+ "truth_starts": [+ 2.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 1.996,+ 2.775,+ 3.9,+ "stroke"+ ],+ [+ 36.995,+ 42.075,+ 3.64,+ "window"+ ],+ [+ 11.944,+ 12.145,+ 2.66,+ "stroke"+ ]+ ],+ "lead_ms": [+ 14.0+ ]+ },+ {+ "query": "a fast whoosh",+ "truth_starts": [+ 7.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 6.997,+ 7.17,+ 4.84,+ "stroke"+ ],+ [+ 11.944,+ 12.145,+ 1.9,+ "stroke"+ ],+ [+ 32.002,+ 32.123,+ 1.63,+ "stroke"+ ]+ ],+ "lead_ms": [+ 13.0+ ]+ },+ {+ "query": "a cork pop",+ "truth_starts": [+ 12.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 11.944,+ 12.145,+ 3.92,+ "stroke"+ ],+ [+ 21.995,+ 22.305,+ 2.09,+ "stroke"+ ],+ [+ 26.996,+ 27.06,+ 1.98,+ "stroke"+ ]+ ],+ "lead_ms": [+ 66.0+ ]+ },+ {+ "query": "a lego brick click",+ "truth_starts": [+ 17.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 17.005,+ 17.06,+ 3.39,+ "stroke"+ ],+ [+ 26.996,+ 27.06,+ 2.95,+ "stroke"+ ],+ [+ 32.5,+ 37.02,+ 2.62,+ "window"+ ]+ ],+ "lead_ms": [+ 5.0+ ]+ },+ {+ "query": "an object dropping into water",+ "truth_starts": [+ 22.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 21.995,+ 22.305,+ 3.49,+ "stroke"+ ],+ [+ 36.997,+ 37.495,+ 0.3,+ "stroke"+ ],+ [+ 0.905,+ 2.775,+ 0.27,+ "event"+ ]+ ],+ "lead_ms": [+ 15.0+ ]+ },+ {+ "query": "a light switch click",+ "truth_starts": [+ 27.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 26.996,+ 27.06,+ 2.63,+ "stroke"+ ],+ [+ 17.005,+ 17.06,+ 1.66,+ "stroke"+ ],+ [+ 32.002,+ 32.123,+ 1.62,+ "stroke"+ ]+ ],+ "lead_ms": [+ 14.0+ ]+ },+ {+ "query": "a spacebar press",+ "truth_starts": [+ 32.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 28,+ "segments": [+ [+ 32.002,+ 32.29,+ 2.81,+ "event"+ ],+ [+ 26.996,+ 27.06,+ 1.73,+ "stroke"+ ],+ [+ 17.005,+ 17.06,+ 1.35,+ "stroke"+ ]+ ],+ "lead_ms": [+ 8.0+ ]+ },+ {+ "query": "a small bell",+ "truth_starts": [+ 37.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "either",+ "n_candidates": 31,+ "segments": [+ [+ 35.0,+ 40.0,+ 3.71,+ "window"+ ],+ [+ 0.0,+ 5.0,+ 2.49,+ "window"+ ],+ [+ 17.5,+ 22.5,+ 0.93,+ "window"+ ]+ ],+ "lead_ms": []+ }+ ]+ },+ "long_300s": {+ "found": 5,+ "n": 5,+ "top1": 3,+ "n_queries": 3,+ "tight_trims": 5,+ "seconds_all_queries": 44.3,+ "rows": [+ {+ "query": "a glass ting",+ "truth_starts": [+ 20.0,+ 150.0,+ 275.0+ ],+ "found_in_top_n": 3,+ "n_truth": 3,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 88,+ "segments": [+ [+ 149.996,+ 150.795,+ 4.04,+ "stroke"+ ],+ [+ 19.996,+ 20.79,+ 3.99,+ "stroke"+ ],+ [+ 274.996,+ 275.775,+ 3.81,+ "stroke"+ ]+ ],+ "lead_ms": [+ 14.0,+ 14.0,+ 14.0+ ]+ },+ {+ "query": "a cork pop",+ "truth_starts": [+ 205.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 88,+ "segments": [+ [+ 204.997,+ 205.145,+ 3.38,+ "stroke"+ ],+ [+ 239.998,+ 240.17,+ 1.51,+ "stroke"+ ],+ [+ 149.996,+ 150.795,+ 1.34,+ "stroke"+ ]+ ],+ "lead_ms": [+ 13.0+ ]+ },+ {+ "query": "a fast whoosh",+ "truth_starts": [+ 240.0+ ],+ "found_in_top_n": 1,+ "n_truth": 1,+ "top1_hit": true,+ "query_kind": "event",+ "n_candidates": 88,+ "segments": [+ [+ 239.998,+ 240.17,+ 4.74,+ "stroke"+ ],+ [+ 204.997,+ 205.145,+ 1.36,+ "stroke"+ ],+ [+ 230.481,+ 230.63,+ 1.17,+ "stroke"+ ]+ ],+ "lead_ms": [+ 12.0+ ]+ }+ ]+ }+}\ No newline at end of filediff --git a/eval/semantics/heldout.json b/eval/semantics/heldout.jsonnew file mode 100644index 0000000..ecbbfdc--- /dev/null+++ b/eval/semantics/heldout.json@@ -0,0 +1,443 @@+{+ "n_files": 125,+ "n_with_class": 121,+ "per_category": {+ "ambience": 21,+ "foley": 6,+ "garden": 5,+ "keyboard": 20,+ "magic": 6,+ "motion": 15,+ "music": 45,+ "toy": 7+ },+ "thresholds": {+ "strong": 3.0,+ "plausible": 2.5,+ "weak": 2.0+ },+ "shipped": {+ "n_vocab": 322,+ "metrics": {+ "cat_map": 0.6913988597776911,+ "cat_p5": 0.7,+ "cat_p10": 0.625,+ "class_map": 0.6459201163247413,+ "class_top3": 0.8269230769230769,+ "tag_acc": 0.7066115702479339,+ "tag_mrr": 0.8070052286532279,+ "pooled_auc": 0.9795993373835973,+ "pooled_auc_strict": 0.9649479955430356+ },+ "detail": {+ "category": {+ "map": 0.6913988597776911,+ "p5": 0.7,+ "p10": 0.625,+ "rows": [+ {+ "cat": "ambience",+ "ap": 0.9244109997643022,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "foley",+ "ap": 0.17848260127671892,+ "p5": 0.0,+ "p10": 0.1+ },+ {+ "cat": "garden",+ "ap": 0.4472438672438672,+ "p5": 0.4,+ "p10": 0.3+ },+ {+ "cat": "keyboard",+ "ap": 0.9771264855687605,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "magic",+ "ap": 0.9761904761904762,+ "p5": 1.0,+ "p10": 0.6+ },+ {+ "cat": "motion",+ "ap": 0.8236778206778207,+ "p5": 1.0,+ "p10": 0.9+ },+ {+ "cat": "music",+ "ap": 1.0,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "toy",+ "ap": 0.20405862749958284,+ "p5": 0.2,+ "p10": 0.1+ }+ ]+ },+ "class": {+ "map": 0.6459201163247413,+ "top3": 0.8269230769230769,+ "n_queries": 52+ },+ "tagging": {+ "acc": 0.7066115702479339,+ "mrr": 0.8070052286532279+ },+ "pooled_auc": 0.9795993373835973,+ "pooled_auc_strict": 0.9649479955430356+ },+ "verdicts": {+ "lenient": {+ "n_pos": 248,+ "n_neg": 7614,+ "base_rate": 0.0315,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 126,+ "fp": 3,+ "lr": 1289.47,+ "precision": 0.977,+ "precision_even_odds": 0.999,+ "recall_at_or_above": 0.508+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 31,+ "fp": 45,+ "lr": 21.15,+ "precision": 0.408,+ "precision_even_odds": 0.955,+ "recall_at_or_above": 0.633+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 31,+ "fp": 80,+ "lr": 11.9,+ "precision": 0.279,+ "precision_even_odds": 0.922,+ "recall_at_or_above": 0.758+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 60,+ "fp": 7486,+ "lr": 0.25,+ "precision": 0.008,+ "precision_even_odds": 0.197,+ "recall_at_or_above": 1.0+ }+ },+ "strict": {+ "n_pos": 248,+ "n_neg": 8222,+ "base_rate": 0.0293,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 126,+ "fp": 82,+ "lr": 50.94,+ "precision": 0.606,+ "precision_even_odds": 0.981,+ "recall_at_or_above": 0.508+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 31,+ "fp": 113,+ "lr": 9.1,+ "precision": 0.215,+ "precision_even_odds": 0.901,+ "recall_at_or_above": 0.633+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 31,+ "fp": 156,+ "lr": 6.59,+ "precision": 0.166,+ "precision_even_odds": 0.868,+ "recall_at_or_above": 0.758+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 60,+ "fp": 7871,+ "lr": 0.25,+ "precision": 0.008,+ "precision_even_odds": 0.202,+ "recall_at_or_above": 1.0+ }+ }+ }+ },+ "preregistered_217": {+ "n_vocab": 217,+ "metrics": {+ "cat_map": 0.684794997075632,+ "cat_p5": 0.65,+ "cat_p10": 0.6375,+ "class_map": 0.6691226676530339,+ "class_top3": 0.8269230769230769,+ "tag_acc": 0.7066115702479339,+ "tag_mrr": 0.8070052286532279,+ "pooled_auc": 0.9757169517950803,+ "pooled_auc_strict": 0.9596587832801061+ },+ "detail": {+ "category": {+ "map": 0.684794997075632,+ "p5": 0.65,+ "p10": 0.6375,+ "rows": [+ {+ "cat": "ambience",+ "ap": 0.9595023655395172,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "foley",+ "ap": 0.1697744758228629,+ "p5": 0.0,+ "p10": 0.2+ },+ {+ "cat": "garden",+ "ap": 0.37741935483870964,+ "p5": 0.2,+ "p10": 0.3+ },+ {+ "cat": "keyboard",+ "ap": 0.9738565162907268,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "magic",+ "ap": 0.9761904761904762,+ "p5": 1.0,+ "p10": 0.6+ },+ {+ "cat": "motion",+ "ap": 0.8146956023831557,+ "p5": 1.0,+ "p10": 0.9+ },+ {+ "cat": "music",+ "ap": 1.0,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "toy",+ "ap": 0.20692118553960656,+ "p5": 0.0,+ "p10": 0.1+ }+ ]+ },+ "class": {+ "map": 0.6691226676530339,+ "top3": 0.8269230769230769,+ "n_queries": 52+ },+ "tagging": {+ "acc": 0.7066115702479339,+ "mrr": 0.8070052286532279+ },+ "pooled_auc": 0.9757169517950803,+ "pooled_auc_strict": 0.9596587832801061+ },+ "verdicts": {+ "lenient": {+ "n_pos": 248,+ "n_neg": 7614,+ "base_rate": 0.0315,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 126,+ "fp": 10,+ "lr": 386.84,+ "precision": 0.926,+ "precision_even_odds": 0.997,+ "recall_at_or_above": 0.508+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 32,+ "fp": 43,+ "lr": 22.85,+ "precision": 0.427,+ "precision_even_odds": 0.958,+ "recall_at_or_above": 0.637+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 25,+ "fp": 99,+ "lr": 7.75,+ "precision": 0.202,+ "precision_even_odds": 0.886,+ "recall_at_or_above": 0.738+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 65,+ "fp": 7462,+ "lr": 0.27,+ "precision": 0.009,+ "precision_even_odds": 0.211,+ "recall_at_or_above": 1.0+ }+ },+ "strict": {+ "n_pos": 248,+ "n_neg": 8222,+ "base_rate": 0.0293,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 126,+ "fp": 106,+ "lr": 39.41,+ "precision": 0.543,+ "precision_even_odds": 0.975,+ "recall_at_or_above": 0.508+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 32,+ "fp": 110,+ "lr": 9.64,+ "precision": 0.225,+ "precision_even_odds": 0.906,+ "recall_at_or_above": 0.637+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 25,+ "fp": 179,+ "lr": 4.63,+ "precision": 0.123,+ "precision_even_odds": 0.822,+ "recall_at_or_above": 0.738+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 65,+ "fp": 7827,+ "lr": 0.28,+ "precision": 0.008,+ "precision_even_odds": 0.216,+ "recall_at_or_above": 1.0+ }+ }+ }+ },+ "raw_cosine_metrics": {+ "cat_map": 0.6461843465427891,+ "cat_p5": 0.725,+ "cat_p10": 0.625,+ "class_map": 0.6999272877356606,+ "class_top3": 0.7884615384615384,+ "tag_acc": 0.7272727272727273,+ "tag_mrr": 0.8180022628886265,+ "pooled_auc": 0.9780206453307574,+ "pooled_auc_strict": 0.9640892648362772+ },+ "mood_check": {+ "heldout": {+ "n_tracks": 45,+ "top_mood_negative_share": 0.178,+ "chance_share": 0.312,+ "top_moods": {+ "playful and quirky music": 10,+ "triumphant and heroic music": 7,+ "sad and melancholic music": 6,+ "uplifting and hopeful music": 5,+ "romantic music": 3,+ "dreamy and ethereal music": 3,+ "celebratory and joyful music": 2,+ "dark and tense music": 2,+ "futuristic music": 2,+ "warm and cozy music": 2,+ "confident and cool music": 2,+ "energetic and driving music": 1+ }+ },+ "tuning": {+ "n_tracks": 8,+ "top_mood_negative_share": 0.0,+ "chance_share": 0.312,+ "top_moods": {+ "dreamy and ethereal music": 3,+ "playful and quirky music": 2,+ "celebratory and joyful music": 1,+ "romantic music": 1,+ "energetic and driving music": 1+ }+ }+ }+}\ No newline at end of filediff --git a/eval/semantics/heldout_labels.json b/eval/semantics/heldout_labels.jsonnew file mode 100644index 0000000..36ce579--- /dev/null+++ b/eval/semantics/heldout_labels.json@@ -0,0 +1,26 @@+{+ "_doc": "Held-out library for earshot semantics: the 126 real recordings in <assets>/_rejected/<folder>/ (rejected by the Clawd project for quality, not for content). Labels were assigned from the folder and the file name ONLY, written on 2026-10-15 before any score on these files was computed; classes are those of queries.json. A file listed with [] has no matching class (counts only for its category). Files never used for choosing the vocabulary, templates, calibration or thresholds. The same lego file sits in foley/ and toy/: it is counted once, as toy (its manifest category).",+ "classes": {+ "fs-181801": ["wind_grass"], "fs-243781": ["rain"], "fs-333510": ["rain"], "fs-346533": ["meadow"], "fs-349044": ["meadow"],+ "fs-360756": ["rain"], "fs-440807": ["meadow"], "fs-455828": ["room_tone"], "fs-456766": ["meadow"], "fs-511758": ["meadow"],+ "fs-637808": ["room_tone"], "fs-639459": ["meadow", "wind_grass"], "fs-656629": ["room_tone"], "fs-669486": ["rain"],+ "fs-751473": ["wind_grass"], "fs-847380": ["meadow", "wind_grass"], "fs-85145": ["rain"], "fs-869851": ["rain"],+ "mk-2464": ["meadow"], "mk-2467": ["meadow"], "mk-2474": ["rain"],+ "fs-151778": ["snap_lock"], "fs-162795": ["soft_landing"], "fs-162799": ["soft_landing"], "fs-203939": ["lego"],+ "fs-253327": ["stretch_pop"], "fs-531501": ["tiny_steps"], "fs-704170": ["small_pop"],+ "fs-445788": ["tree"], "fs-479160": [], "fs-530437": [], "fs-635012": ["tiny_crowd"], "fs-833834": [],+ "fs-380144": ["spacebar"], "fs-391310": ["enter"], "fs-412923": ["key_thock"], "fs-450281": ["typing"], "fs-450282": ["typing"],+ "fs-466049": ["spacebar"], "fs-499773": ["typing"], "fs-509171": ["key_thock"], "fs-546165": ["typing"],+ "fs-561695": ["key_thock", "spacebar"], "fs-628325": ["typing"], "fs-660387": ["key_thock"], "fs-766628": ["key_thock"],+ "fs-766629": ["key_thock"], "fs-766630": ["key_thock"], "fs-766633": ["key_thock"], "fs-813210": ["key_thock"],+ "fs-813215": ["key_thock"], "fs-843786": ["spacebar"], "fs-868632": ["typing"],+ "fs-175932": ["ting"], "fs-398496": ["ting"], "fs-447799": [], "mk-2936": ["ting"], "vsco-glock": ["chime_sweep"],+ "fs-22036": ["plunge"], "fs-315931": ["plunge"], "fs-377829": ["big_whoosh"], "fs-394457": ["small_whoosh"],+ "fs-416710": ["plunge"], "fs-428073": ["impact"], "fs-60013": ["small_whoosh"], "fs-67882": ["underwater"],+ "mk-1304": ["plunge"], "mk-1461": ["small_whoosh"], "mk-1474": ["big_whoosh"], "mk-166": ["small_whoosh"],+ "mk-2604": ["big_whoosh"], "mk-2605": ["small_whoosh"], "mk-2607": ["small_whoosh"],+ "fs-128192": ["snap_lock"], "fs-402758": ["lego"], "fs-584215": ["latch"], "fs-686167": ["snap_lock"],+ "fs-863290": ["lego"], "mk-2582": ["switch"]+ },+ "music_folder_class": "music"+}diff --git a/eval/semantics/kind.json b/eval/semantics/kind.jsonnew file mode 100644index 0000000..dcee032--- /dev/null+++ b/eval/semantics/kind.json@@ -0,0 +1,598 @@+{+ "threshold": -0.27,+ "tune": {+ "events": {+ "event": 31,+ "either": 0,+ "texture": 0+ },+ "textures": {+ "event": 0,+ "either": 6,+ "texture": 6+ },+ "events_read_as_texture": [],+ "textures_read_as_event": [],+ "rows": [+ {+ "query": "a soft plastic thud",+ "truth": "event",+ "read": "event",+ "score": 0.04+ },+ {+ "query": "a small object landing softly on a table",+ "truth": "event",+ "read": "event",+ "score": 0.119+ },+ {+ "query": "a finger squeaking on glass",+ "truth": "event",+ "read": "event",+ "score": -0.073+ },+ {+ "query": "a rubber stretch squeak and suction pop",+ "truth": "event",+ "read": "event",+ "score": -0.018+ },+ {+ "query": "a plastic case snapping shut",+ "truth": "event",+ "read": "event",+ "score": 0.185+ },+ {+ "query": "a satisfying plastic click as something locks into place",+ "truth": "event",+ "read": "event",+ "score": 0.298+ },+ {+ "query": "a small mouth pop",+ "truth": "event",+ "read": "event",+ "score": 0.283+ },+ {+ "query": "a short hollow pop like a cork or a finger pulled from the cheek",+ "truth": "event",+ "read": "event",+ "score": 0.417+ },+ {+ "query": "an apple falling on grass",+ "truth": "event",+ "read": "event",+ "score": 0.015+ },+ {+ "query": "a single mechanical keyboard key press",+ "truth": "event",+ "read": "event",+ "score": 0.223+ },+ {+ "query": "a deep thocky key press on a mechanical keyboard",+ "truth": "event",+ "read": "event",+ "score": 0.146+ },+ {+ "query": "a spacebar press",+ "truth": "event",+ "read": "event",+ "score": 0.145+ },+ {+ "query": "pressing the spacebar of a mechanical keyboard",+ "truth": "event",+ "read": "event",+ "score": 0.099+ },+ {+ "query": "an enter key press",+ "truth": "event",+ "read": "event",+ "score": 0.28+ },+ {+ "query": "a heavy enter key thock",+ "truth": "event",+ "read": "event",+ "score": 0.154+ },+ {+ "query": "a soft key click",+ "truth": "event",+ "read": "event",+ "score": 0.121+ },+ {+ "query": "a light tactile keyboard switch tick",+ "truth": "event",+ "read": "event",+ "score": 0.231+ },+ {+ "query": "a tiny bell ting",+ "truth": "event",+ "read": "event",+ "score": -0.073+ },+ {+ "query": "a single small bell or glass ting",+ "truth": "event",+ "read": "event",+ "score": -0.018+ },+ {+ "query": "a single celesta note",+ "truth": "event",+ "read": "event",+ "score": 0.144+ },+ {+ "query": "a plucked kalimba or celesta note",+ "truth": "event",+ "read": "event",+ "score": 0.09+ },+ {+ "query": "a fast small whoosh",+ "truth": "event",+ "read": "event",+ "score": 0.25+ },+ {+ "query": "a stick swung quickly through the air",+ "truth": "event",+ "read": "event",+ "score": 0.195+ },+ {+ "query": "a big soft whoosh",+ "truth": "event",+ "read": "event",+ "score": 0.07+ },+ {+ "query": "an object plunging into water",+ "truth": "event",+ "read": "event",+ "score": -0.035+ },+ {+ "query": "a deep cinematic impact",+ "truth": "event",+ "read": "event",+ "score": 0.037+ },+ {+ "query": "a big boomy hit",+ "truth": "event",+ "read": "event",+ "score": 0.082+ },+ {+ "query": "a latch closing",+ "truth": "event",+ "read": "event",+ "score": 0.162+ },+ {+ "query": "a box lid closing with a solid clunk",+ "truth": "event",+ "read": "event",+ "score": 0.205+ },+ {+ "query": "a light switch click",+ "truth": "event",+ "read": "event",+ "score": 0.29+ },+ {+ "query": "a switch flip or camera shutter click",+ "truth": "event",+ "read": "event",+ "score": 0.115+ },+ {+ "query": "a quiet computer room tone hum",+ "truth": "texture",+ "read": "texture",+ "score": -0.344+ },+ {+ "query": "the steady low hum of a computer in a quiet room",+ "truth": "texture",+ "read": "texture",+ "score": -0.377+ },+ {+ "query": "rain on a window",+ "truth": "texture",+ "read": "texture",+ "score": -0.3+ },+ {+ "query": "rain heard from inside a room, drops on the window glass",+ "truth": "texture",+ "read": "either",+ "score": -0.212+ },+ {+ "query": "birds singing in a spring meadow",+ "truth": "texture",+ "read": "texture",+ "score": -0.32+ },+ {+ "query": "a morning field recording of birdsong in the countryside",+ "truth": "texture",+ "read": "either",+ "score": -0.223+ },+ {+ "query": "wind blowing through grass and leaves",+ "truth": "texture",+ "read": "texture",+ "score": -0.321+ },+ {+ "query": "a breeze rustling tall grass and leaves",+ "truth": "texture",+ "read": "texture",+ "score": -0.334+ },+ {+ "query": "fast typing on a mechanical keyboard",+ "truth": "texture",+ "read": "either",+ "score": -0.164+ },+ {+ "query": "continuous typing on a keyboard",+ "truth": "texture",+ "read": "either",+ "score": -0.247+ },+ {+ "query": "an uplifting orchestral music track",+ "truth": "texture",+ "read": "either",+ "score": -0.185+ },+ {+ "query": "a cheerful music track",+ "truth": "texture",+ "read": "either",+ "score": -0.183+ }+ ]+ },+ "test": {+ "events": {+ "event": 25,+ "either": 4,+ "texture": 1+ },+ "textures": {+ "event": 0,+ "either": 6,+ "texture": 14+ },+ "events_read_as_texture": [+ [+ "a bird chirp",+ -0.288+ ]+ ],+ "textures_read_as_event": [],+ "rows": [+ {+ "query": "a dog bark",+ "truth": "event",+ "read": "either",+ "score": -0.203+ },+ {+ "query": "a car horn",+ "truth": "event",+ "read": "either",+ "score": -0.208+ },+ {+ "query": "a thunder clap",+ "truth": "event",+ "read": "either",+ "score": -0.249+ },+ {+ "query": "a bird chirp",+ "truth": "event",+ "read": "texture",+ "score": -0.288+ },+ {+ "query": "a small bell",+ "truth": "event",+ "read": "either",+ "score": -0.177+ },+ {+ "query": "a camera shutter",+ "truth": "event",+ "read": "event",+ "score": -0.015+ },+ {+ "query": "a doorbell",+ "truth": "event",+ "read": "event",+ "score": -0.071+ },+ {+ "query": "a footstep",+ "truth": "event",+ "read": "event",+ "score": -0.015+ },+ {+ "query": "a notification ding",+ "truth": "event",+ "read": "event",+ "score": -0.079+ },+ {+ "query": "a gunshot",+ "truth": "event",+ "read": "event",+ "score": 0.143+ },+ {+ "query": "a door slam",+ "truth": "event",+ "read": "event",+ "score": 0.079+ },+ {+ "query": "a cork pop",+ "truth": "event",+ "read": "event",+ "score": 0.345+ },+ {+ "query": "a hand clap",+ "truth": "event",+ "read": "event",+ "score": 0.138+ },+ {+ "query": "a sneeze",+ "truth": "event",+ "read": "event",+ "score": 0.049+ },+ {+ "query": "a cough",+ "truth": "event",+ "read": "event",+ "score": 0.048+ },+ {+ "query": "a glass breaking",+ "truth": "event",+ "read": "event",+ "score": -0.011+ },+ {+ "query": "a balloon pop",+ "truth": "event",+ "read": "event",+ "score": 0.203+ },+ {+ "query": "a mouse click",+ "truth": "event",+ "read": "event",+ "score": 0.057+ },+ {+ "query": "a sword hit",+ "truth": "event",+ "read": "event",+ "score": 0.154+ },+ {+ "query": "a punch",+ "truth": "event",+ "read": "event",+ "score": 0.217+ },+ {+ "query": "a coin drop",+ "truth": "event",+ "read": "event",+ "score": 0.067+ },+ {+ "query": "a single drum hit",+ "truth": "event",+ "read": "event",+ "score": 0.249+ },+ {+ "query": "a whoosh",+ "truth": "event",+ "read": "event",+ "score": 0.135+ },+ {+ "query": "a swipe sound",+ "truth": "event",+ "read": "event",+ "score": 0.15+ },+ {+ "query": "a UI pop",+ "truth": "event",+ "read": "event",+ "score": 0.293+ },+ {+ "query": "a can opening",+ "truth": "event",+ "read": "event",+ "score": 0.101+ },+ {+ "query": "a light switch",+ "truth": "event",+ "read": "event",+ "score": 0.238+ },+ {+ "query": "a knock on a door",+ "truth": "event",+ "read": "event",+ "score": 0.019+ },+ {+ "query": "a cat meow",+ "truth": "event",+ "read": "event",+ "score": -0.018+ },+ {+ "query": "a stapler",+ "truth": "event",+ "read": "event",+ "score": 0.099+ },+ {+ "query": "rain on a roof",+ "truth": "texture",+ "read": "texture",+ "score": -0.346+ },+ {+ "query": "a crackling fire",+ "truth": "texture",+ "read": "either",+ "score": -0.091+ },+ {+ "query": "city traffic",+ "truth": "texture",+ "read": "texture",+ "score": -0.32+ },+ {+ "query": "ocean waves",+ "truth": "texture",+ "read": "either",+ "score": -0.113+ },+ {+ "query": "wind in the trees",+ "truth": "texture",+ "read": "texture",+ "score": -0.397+ },+ {+ "query": "a quiet office ambience",+ "truth": "texture",+ "read": "either",+ "score": -0.266+ },+ {+ "query": "crowd chatter",+ "truth": "texture",+ "read": "texture",+ "score": -0.303+ },+ {+ "query": "a refrigerator hum",+ "truth": "texture",+ "read": "texture",+ "score": -0.345+ },+ {+ "query": "birdsong in a forest",+ "truth": "texture",+ "read": "texture",+ "score": -0.32+ },+ {+ "query": "an air conditioner",+ "truth": "texture",+ "read": "either",+ "score": -0.251+ },+ {+ "query": "a busy restaurant",+ "truth": "texture",+ "read": "texture",+ "score": -0.318+ },+ {+ "query": "a river flowing",+ "truth": "texture",+ "read": "texture",+ "score": -0.341+ },+ {+ "query": "crickets at night",+ "truth": "texture",+ "read": "texture",+ "score": -0.316+ },+ {+ "query": "a lo-fi music loop",+ "truth": "texture",+ "read": "either",+ "score": -0.265+ },+ {+ "query": "a drone pad",+ "truth": "texture",+ "read": "texture",+ "score": -0.428+ },+ {+ "query": "a steady engine idle",+ "truth": "texture",+ "read": "texture",+ "score": -0.372+ },+ {+ "query": "continuous typing",+ "truth": "texture",+ "read": "either",+ "score": -0.245+ },+ {+ "query": "a vacuum cleaner running",+ "truth": "texture",+ "read": "texture",+ "score": -0.36+ },+ {+ "query": "white noise",+ "truth": "texture",+ "read": "texture",+ "score": -0.359+ },+ {+ "query": "a heavy rainstorm",+ "truth": "texture",+ "read": "texture",+ "score": -0.365+ }+ ]+ }+}\ No newline at end of filediff --git a/eval/semantics/kind_queries.json b/eval/semantics/kind_queries.jsonnew file mode 100644index 0000000..c701349--- /dev/null+++ b/eval/semantics/kind_queries.json@@ -0,0 +1,33 @@+{+ "_doc": "Labelled query texts for find()'s query-kind reader (event = one short sound you would trim as a sample; texture = a continuous sound you would loop). 'tune' = clear-cut class phrasings from queries.json (the threshold may be set on these); 'test' = generic queries written for this check, never used to set anything.",+ "tune": {+ "event": ["a soft plastic thud", "a small object landing softly on a table", "a finger squeaking on glass",+ "a rubber stretch squeak and suction pop", "a plastic case snapping shut",+ "a satisfying plastic click as something locks into place", "a small mouth pop",+ "a short hollow pop like a cork or a finger pulled from the cheek", "an apple falling on grass",+ "a single mechanical keyboard key press", "a deep thocky key press on a mechanical keyboard",+ "a spacebar press", "pressing the spacebar of a mechanical keyboard", "an enter key press",+ "a heavy enter key thock", "a soft key click", "a light tactile keyboard switch tick", "a tiny bell ting",+ "a single small bell or glass ting", "a single celesta note", "a plucked kalimba or celesta note",+ "a fast small whoosh", "a stick swung quickly through the air", "a big soft whoosh",+ "an object plunging into water", "a deep cinematic impact", "a big boomy hit", "a latch closing",+ "a box lid closing with a solid clunk", "a light switch click", "a switch flip or camera shutter click"],+ "texture": ["a quiet computer room tone hum", "the steady low hum of a computer in a quiet room", "rain on a window",+ "rain heard from inside a room, drops on the window glass", "birds singing in a spring meadow",+ "a morning field recording of birdsong in the countryside", "wind blowing through grass and leaves",+ "a breeze rustling tall grass and leaves", "fast typing on a mechanical keyboard",+ "continuous typing on a keyboard", "an uplifting orchestral music track", "a cheerful music track"]+ },+ "test": {+ "event": ["a dog bark", "a car horn", "a thunder clap", "a bird chirp", "a small bell", "a camera shutter", "a doorbell",+ "a footstep", "a notification ding", "a gunshot", "a door slam", "a cork pop", "a hand clap", "a sneeze",+ "a cough", "a glass breaking", "a balloon pop", "a mouse click", "a sword hit", "a punch", "a coin drop",+ "a single drum hit", "a whoosh", "a swipe sound", "a UI pop", "a can opening", "a light switch",+ "a knock on a door", "a cat meow", "a stapler"],+ "texture": ["rain on a roof", "a crackling fire", "city traffic", "ocean waves", "wind in the trees",+ "a quiet office ambience", "crowd chatter", "a refrigerator hum", "birdsong in a forest",+ "an air conditioner", "a busy restaurant", "a river flowing", "crickets at night", "a lo-fi music loop",+ "a drone pad", "a steady engine idle", "continuous typing", "a vacuum cleaner running", "white noise",+ "a heavy rainstorm"]+ }+}diff --git a/eval/semantics/precision.json b/eval/semantics/precision.jsonnew file mode 100644index 0000000..9658121--- /dev/null+++ b/eval/semantics/precision.json@@ -0,0 +1,16 @@+{+ "torch": "2.14.1+cpu",+ "cpu_capability": "AVX512",+ "native_bf16": true,+ "threads": 2,+ "clips": 12,+ "used": [+ "fp32",+ "bf16"+ ],+ "cosine_mean": 0.99975,+ "cosine_min": 0.99925,+ "s_per_window_fp32": 1.26,+ "s_per_window_bf16": 0.904,+ "speedup_bf16": 1.39+}\ No newline at end of filediff --git a/eval/semantics/precision.py b/eval/semantics/precision.pynew file mode 100644index 0000000..7915066--- /dev/null+++ b/eval/semantics/precision.py@@ -0,0 +1,61 @@+"""bf16 vs fp32 CLAP on this CPU: embedding agreement and speed (models load twice; no cache is used or written).++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/semantics/precision.py # -> eval/semantics/precision.json++Input: the first 10 s of 12 real recordings (2 per category folder of the Clawd project's library, read-only), each+embedded in fp32 and in bf16; cosine between the two embeddings of the same clip, and seconds per 10 s window+(batch 4, after one warm-up batch). Whether bf16 is used at all depends on the CPU: earshot.models.clap.native_bf16.+"""+import json+import os+import sys+import time++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+from earshot import audio, config # noqa: E402+from earshot.models import clap # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++ASSETS = EP.ASSETS+CATS = ['ambience', 'foley', 'keyboard', 'motion', 'music', 'toy']+++def clips():+ out = []+ for c in CATS:+ fs = sorted(f for f in os.listdir(os.path.join(ASSETS, c)) if os.path.splitext(f)[1].lower() in audio.AUDIO_EXTS)+ for f in fs[:2]:+ out.append(audio.load(os.path.join(ASSETS, c, f), mono=True, end=10.0))+ return out+++def run(precision, xs):+ config.PRECISION = precision+ c = clap.Clap()+ c._forward(xs[:4], 'repeatpad', 4) # warm-up+ t = time.time()+ e = c._forward(xs, 'repeatpad', 4)+ return c.precision, e, (time.time() - t) / len(xs)+++def main():+ import torch+ xs = clips()+ p32, e32, s32 = run('fp32', xs)+ p16, e16, s16 = run('bf16', xs)+ cos = np.sum(e32 * e16, axis=1) / (np.linalg.norm(e32, axis=1) * np.linalg.norm(e16, axis=1))+ res = {'torch': torch.__version__, 'cpu_capability': torch.backends.cpu.get_cpu_capability(),+ 'native_bf16': clap.native_bf16(torch), 'threads': config.threads(), 'clips': len(xs),+ 'used': [p32, p16], 'cosine_mean': round(float(cos.mean()), 5), 'cosine_min': round(float(cos.min()), 5),+ 's_per_window_fp32': round(s32, 3), 's_per_window_bf16': round(s16, 3),+ 'speedup_bf16': round(s32 / s16, 2) if s16 else None}+ json.dump(res, open(os.path.join(HERE, 'precision.json'), 'w'), indent=1)+ print(res)+++if __name__ == '__main__':+ main()diff --git a/eval/semantics/queries.json b/eval/semantics/queries.jsonnew file mode 100644index 0000000..9ca52e8--- /dev/null+++ b/eval/semantics/queries.json@@ -0,0 +1,76 @@+{+ "_doc": "Evaluation queries for earshot semantics, written from the manifest need names BEFORE any scores were looked at. 'classes' maps every manifest need to a class with two phrasings (a short agent-style query and a descriptive one). 'categories' is one natural query per asset folder. 'specific' are concrete needs with their correct files (ids). Caveat: 67 vocabulary terms were added later after seeing these classes (earshot/data/vocab.json _doc), so numbers on the 219-file tuning library are optimistic; the held-out library (heldout_labels.json) is the honest test.",+ "categories": {+ "ambience": "a continuous ambient background sound of a place",+ "foley": "a close-up foley sound of a small object",+ "garden": "sounds of grass, leaves, soil and wood outdoors in a garden",+ "keyboard": "a mechanical computer keyboard",+ "magic": "a bright magical chime, bell or celesta note",+ "motion": "a whoosh of fast movement through the air",+ "music": "a piece of music",+ "toy": "plastic toy clicks, snaps and latches"+ },+ "classes": {+ "room_tone": {"needs": ["a_terminal_room_tone"], "q": ["a quiet computer room tone hum", "the steady low hum of a computer in a quiet room"]},+ "rain": {"needs": ["b_night_room_rain", "b_night_room_rain_detail_layer"], "q": ["rain on a window", "rain heard from inside a room, drops on the window glass"]},+ "meadow": {"needs": ["c_spring_morning_meadow"], "q": ["birds singing in a spring meadow", "a morning field recording of birdsong in the countryside"]},+ "wind_grass": {"needs": ["d_wind_grass_leaves_layer"], "q": ["wind blowing through grass and leaves", "a breeze rustling tall grass and leaves"]},+ "tiny_steps": {"needs": ["a_tiny_steps"], "q": ["light finger taps", "tiny footsteps made by fingertips tapping on a surface"]},+ "soft_landing": {"needs": ["b_soft_landings"], "q": ["a soft plastic thud", "a small object landing softly on a table"]},+ "glass_squeak": {"needs": ["c_glass_touch_squeak"], "q": ["a finger squeaking on glass", "a squeaky rub of a finger across a glass surface"]},+ "stretch_pop": {"needs": ["d_membrane_stretch_pop"], "q": ["a rubber stretch squeak and suction pop", "stretching rubber or a balloon, squeaks and a suction cup pop"]},+ "snap_lock": {"needs": ["e_snap_click_lock"], "q": ["a plastic case snapping shut", "a satisfying plastic click as something locks into place"]},+ "small_pop": {"needs": ["f_small_pops"], "q": ["a small mouth pop", "a short hollow pop like a cork or a finger pulled from the cheek"]},+ "grass_steps": {"needs": ["steps_in_grass"], "q": ["footsteps on grass", "walking through grass and dry leaves"]},+ "soil": {"needs": ["soil_press"], "q": ["digging in soil", "a trowel pushing into soft dirt"]},+ "tree": {"needs": ["tree_growth"], "q": ["wood creaking", "creaking wood, rustling branches and a tree shaking"]},+ "fruit_drop": {"needs": ["fruit_drops"], "q": ["an apple falling on grass", "an object dropping onto grass and leaves with a soft thud"]},+ "tiny_crowd": {"needs": ["tiny_crowd_steps"], "q": ["small animal running through grass", "many tiny fast footsteps scurrying"]},+ "key_thock": {"needs": ["a_key_thock"], "q": ["a single mechanical keyboard key press", "a deep thocky key press on a mechanical keyboard"]},+ "spacebar": {"needs": ["b_spacebar"], "q": ["a spacebar press", "pressing the spacebar of a mechanical keyboard"]},+ "typing": {"needs": ["c_typing"], "q": ["fast typing on a mechanical keyboard", "continuous typing on a keyboard"]},+ "enter": {"needs": ["c_enter"], "q": ["an enter key press", "a heavy enter key thock"]},+ "cursor_click": {"needs": ["d_cursor_click"], "q": ["a soft key click", "a light tactile keyboard switch tick"]},+ "ting": {"needs": ["a: pixel wink tiny bell/ting (57.5 s)"], "q": ["a tiny bell ting", "a single small bell or glass ting"]},+ "chime_sweep": {"needs": ["b: light wave across keyboard (53.9-55.3 s) / voxel extrude swell (4-6 s)"], "q": ["a wind chime glissando", "a shimmering sweep of bar chimes"]},+ "note": {"needs": ["d: leitmotif one-shots on key presses (G major / pentatonic); also c: leaf/growth sparkle (36-39 s)", "d: leitmotif one-shots on key presses (E-major notes for the film key: E, F#, G#m harmony at 12.5-26 s)", "d: leitmotif one-shots on key presses (alternative instrument)"], "q": ["a single celesta note", "a plucked kalimba or celesta note"]},+ "small_whoosh": {"needs": ["a_small_whoosh"], "q": ["a fast small whoosh", "a stick swung quickly through the air"]},+ "big_whoosh": {"needs": ["b_big_soft_whoosh", "c_fall_air_rush"], "q": ["a big soft whoosh", "a wide airy swoosh of wind passing by"]},+ "plunge": {"needs": ["c_dive_plunge"], "q": ["an object plunging into water", "a splash and bubbles of something dropping into water"]},+ "underwater": {"needs": ["c_underwater_entry"], "q": ["going underwater", "submerging underwater with muffled bubbles"]},+ "swell": {"needs": ["d_riser", "d_swell"], "q": ["a rising cymbal swell", "a cinematic riser building tension"]},+ "impact": {"needs": ["e_drop_impact"], "q": ["a deep cinematic impact", "a big boomy hit"]},+ "music": {"needs": ["*music"], "q": ["an uplifting orchestral music track", "a cheerful music track"]},+ "toy_clicks": {"needs": ["unfold"], "q": ["plastic clicks", "a plastic toy mechanism clicking and opening"]},+ "lego": {"needs": ["stacking"], "q": ["lego bricks clicking together", "toy blocks clicking and stacking"]},+ "dominoes": {"needs": ["stacking-cascade"], "q": ["dominoes falling", "a cascade of dominoes toppling over"]},+ "latch": {"needs": ["big-lock"], "q": ["a latch closing", "a box lid closing with a solid clunk"]},+ "switch": {"needs": ["blink"], "q": ["a light switch click", "a switch flip or camera shutter click"]}+ },+ "specific": [+ {"q": "a deep thock of a mechanical keyboard key", "classes": ["key_thock", "enter", "spacebar"]},+ {"q": "a whoosh", "classes": ["small_whoosh", "big_whoosh"]},+ {"q": "a glass ting", "ids": ["fs-170237", "fs-549899", "mk-3108"]},+ {"q": "footsteps in grass", "classes": ["grass_steps", "tiny_crowd"]},+ {"q": "rain on a window", "classes": ["rain"]},+ {"q": "a cork pop", "ids": ["fs-16064"]},+ {"q": "a pop", "classes": ["small_pop"]},+ {"q": "a light switch click", "classes": ["switch"]},+ {"q": "lego bricks clicking", "classes": ["lego"]},+ {"q": "dominoes falling", "classes": ["dominoes"]},+ {"q": "a celesta note", "ids_prefix": ["fs-4106"]},+ {"q": "a kalimba note", "ids_prefix": ["fs-6817"]},+ {"q": "birds singing in the morning", "classes": ["meadow"]},+ {"q": "an object dropping into water", "classes": ["plunge"]},+ {"q": "a cymbal swell", "classes": ["swell"]},+ {"q": "typing on a keyboard", "classes": ["typing"]},+ {"q": "a creaking wooden sound", "ids": ["fs-735401", "fs-346140", "fs-123712", "fs-506665", "mk-3157", "mk-178"]},+ {"q": "digging in soil", "classes": ["soil"]},+ {"q": "an apple falling on grass", "classes": ["fruit_drop"]},+ {"q": "a finger squeaking on glass", "classes": ["glass_squeak"]},+ {"q": "a computer hum", "classes": ["room_tone"]},+ {"q": "a camera shutter", "ids": ["mk-1133", "fs-270435"]},+ {"q": "a suction cup pop", "ids": ["fs-543200", "fs-386885", "fs-41616"]},+ {"q": "a deep cinematic hit", "classes": ["impact"]}+ ]+}diff --git a/eval/semantics/results.json b/eval/semantics/results.jsonnew file mode 100644index 0000000..932c3c3--- /dev/null+++ b/eval/semantics/results.json@@ -0,0 +1,3454 @@+{+ "n_eval": 219,+ "n_ref": 148,+ "n_vocab": 322,+ "chosen": {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "both",+ "all": {+ "cat_map": 0.7775335919864723,+ "cat_p5": 0.85,+ "cat_p10": 0.85,+ "class_map": 0.6424589753467237,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5799086757990868,+ "tag_mrr": 0.712406871471145,+ "pooled_auc": 0.9656601066427873,+ "pooled_auc_strict": 0.9508040386214509,+ "spec_top1": 0.7916666666666666,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.777586606292167,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6779892152094823,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7313790616536111,+ "pooled_auc": 0.9731773544405689,+ "pooled_auc_strict": 0.9595833926453143,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7932570388183038,+ "cat_p5": 0.825,+ "cat_p10": 0.7125,+ "class_map": 0.6390066424489291,+ "class_top3": 0.8793103448275862,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6876431706014001,+ "pooled_auc": 0.9558856317750858,+ "pooled_auc_strict": 0.9393954636620294,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ },+ "detail": {+ "category": {+ "map": 0.7775335919864723,+ "p5": 0.85,+ "p10": 0.85,+ "rows": [+ {+ "cat": "ambience",+ "ap": 0.9464513817222796,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "foley",+ "ap": 0.23496220076980903,+ "p5": 0.0,+ "p10": 0.2+ },+ {+ "cat": "garden",+ "ap": 0.7607732443514181,+ "p5": 1.0,+ "p10": 0.9+ },+ {+ "cat": "keyboard",+ "ap": 0.9541322251479479,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "magic",+ "ap": 1.0,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "motion",+ "ap": 0.671879351245011,+ "p5": 1.0,+ "p10": 1.0+ },+ {+ "cat": "music",+ "ap": 1.0,+ "p5": 1.0,+ "p10": 0.8+ },+ {+ "cat": "toy",+ "ap": 0.6520703326553127,+ "p5": 0.8,+ "p10": 0.9+ }+ ]+ },+ "class": {+ "map": 0.6424589753467237,+ "top3": 0.8857142857142857,+ "n_queries": 70+ },+ "specific": {+ "top1": 0.7916666666666666,+ "top3": 0.9583333333333334,+ "map": 0.7557323637715152,+ "rows": [+ {+ "q": "a deep thock of a mechanical keyboard key",+ "top1": true,+ "top3": true,+ "ap": 0.65227130473939,+ "best": "fs-789628_keychron-linear-individual-keys.ogg"+ },+ {+ "q": "a whoosh",+ "top1": true,+ "top3": true,+ "ap": 0.960530045351474,+ "best": "mk-1489_air-woosh.wav"+ },+ {+ "q": "a glass ting",+ "top1": true,+ "top3": true,+ "ap": 0.6752136752136751,+ "best": "fs-549899_small-wine-glass-tap.ogg"+ },+ {+ "q": "footsteps in grass",+ "top1": true,+ "top3": true,+ "ap": 0.8168168880455408,+ "best": "mk-1921_walking-on-grass.wav"+ },+ {+ "q": "rain on a window",+ "top1": true,+ "top3": true,+ "ap": 0.9761904761904762,+ "best": "fs-346642_rain-on-windows-interior.ogg"+ },+ {+ "q": "a cork pop",+ "top1": false,+ "top3": false,+ "ap": 0.25,+ "best": "fs-181907_mouth-plops-c214.ogg"+ },+ {+ "q": "a pop",+ "top1": true,+ "top3": true,+ "ap": 0.8692956349206349,+ "best": "fs-181907_mouth-plops-c214.ogg"+ },+ {+ "q": "a light switch click",+ "top1": false,+ "top3": true,+ "ap": 0.210921912624699,+ "best": "fs-151778_clip-in.ogg"+ },+ {+ "q": "lego bricks clicking",+ "top1": false,+ "top3": true,+ "ap": 0.43934645601312267,+ "best": "fs-326470_dominos-falling.ogg"+ },+ {+ "q": "dominoes falling",+ "top1": true,+ "top3": true,+ "ap": 1.0,+ "best": "fs-873306_various-domino-falling.ogg"+ },+ {+ "q": "a celesta note",+ "top1": true,+ "top3": true,+ "ap": 1.0,+ "best": "fs-410664_celesta-B5.ogg"+ },+ {+ "q": "a kalimba note",+ "top1": true,+ "top3": true,+ "ap": 0.8761904761904761,+ "best": "fs-681791_kalimba-D-high.ogg"+ },+ {+ "q": "birds singing in the morning",+ "top1": true,+ "top3": true,+ "ap": 1.0,+ "best": "mk-2472_morning-birds.wav"+ },+ {+ "q": "an object dropping into water",+ "top1": true,+ "top3": true,+ "ap": 0.8441043083900226,+ "best": "fs-272392_object-drops-in-water.ogg"+ },+ {+ "q": "a cymbal swell",+ "top1": true,+ "top3": true,+ "ap": 0.8928571428571428,+ "best": "fs-346023_violin-bow-on-cymbal.ogg"+ },+ {+ "q": "typing on a keyboard",+ "top1": true,+ "top3": true,+ "ap": 0.9166666666666666,+ "best": "fs-789627_keychron-linear-typing-66wpm.ogg"+ },+ {+ "q": "a creaking wooden sound",+ "top1": true,+ "top3": true,+ "ap": 0.649133022774327,+ "best": "fs-123712_creaking-wood-screws.ogg"+ },+ {+ "q": "digging in soil",+ "top1": true,+ "top3": true,+ "ap": 0.6083333333333334,+ "best": "fs-696531_trowel-soft-soil.ogg"+ },+ {+ "q": "an apple falling on grass",+ "top1": false,+ "top3": true,+ "ap": 0.6472222222222223,+ "best": "fs-559082_high-grass-movement.ogg"+ },+ {+ "q": "a finger squeaking on glass",+ "top1": true,+ "top3": true,+ "ap": 0.7784090909090909,+ "best": "fs-119916_glass-squeaking.ogg"+ },+ {+ "q": "a computer hum",+ "top1": true,+ "top3": true,+ "ap": 0.7333333333333334,+ "best": "fs-381258_computer-hum-gentle.ogg"+ },+ {+ "q": "a camera shutter",+ "top1": true,+ "top3": true,+ "ap": 1.0,+ "best": "fs-270435_shutter-click-canon.ogg"+ },+ {+ "q": "a suction cup pop",+ "top1": false,+ "top3": true,+ "ap": 0.34074074074074073,+ "best": "fs-181907_mouth-plops-c214.ogg"+ },+ {+ "q": "a deep cinematic hit",+ "top1": true,+ "top3": true,+ "ap": 1.0,+ "best": "mk-563_drum-deep-impact.wav"+ }+ ]+ },+ "tagging": {+ "acc": 0.5799086757990868,+ "mrr": 0.712406871471145+ },+ "pooled_auc": 0.9656601066427873,+ "pooled_auc_strict": 0.9508040386214509+ }+ },+ "variants": [+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "raw",+ "all": {+ "cat_map": 0.6980522113170913,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.7875,+ "class_map": 0.5970493925255879,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5525114155251142,+ "tag_mrr": 0.6852196436140946,+ "pooled_auc": 0.9435436000759796,+ "pooled_auc_strict": 0.9273817348182524,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6856773137576235,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6238167679338643,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7081935069387318,+ "pooled_auc": 0.9537306144352026,+ "pooled_auc_strict": 0.9380697508896797,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.697124334270687,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.5876740068290082,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.5157894736842106,+ "tag_mrr": 0.6552327062219364,+ "pooled_auc": 0.9308220171927477,+ "pooled_auc_strict": 0.9139352508335569,+ "spec_top1": 0.5714285714285714,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "sound",+ "all": {+ "cat_map": 0.7439785814201825,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.8,+ "class_map": 0.6094180530398979,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5525114155251142,+ "tag_mrr": 0.6852196436140946,+ "pooled_auc": 0.9564496160316943,+ "pooled_auc_strict": 0.9402166676955068,+ "spec_top1": 0.875,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7456934125575034,+ "cat_p5": 0.825,+ "cat_p10": 0.7250000000000001,+ "class_map": 0.6627912840247255,+ "class_top3": 0.8709677419354839,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7081935069387318,+ "pooled_auc": 0.965224040783472,+ "pooled_auc_strict": 0.9494149466192171,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.759221115646978,+ "cat_p5": 0.8,+ "cat_p10": 0.7,+ "class_map": 0.6008358348396999,+ "class_top3": 0.8103448275862069,+ "tag_acc": 0.5157894736842106,+ "tag_mrr": 0.6552327062219364,+ "pooled_auc": 0.9450525727188112,+ "pooled_auc_strict": 0.9283196962148214,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "center",+ "all": {+ "cat_map": 0.6980522113170913,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.7875,+ "class_map": 0.5970493925255879,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5684931506849316,+ "tag_mrr": 0.7059231884033613,+ "pooled_auc": 0.9489340809182677,+ "pooled_auc_strict": 0.9332191925904139,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6856773137576235,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6238167679338643,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.730970754156238,+ "pooled_auc": 0.9599833646364369,+ "pooled_auc_strict": 0.9452052194543298,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.697124334270687,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.5876740068290082,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.5263157894736842,+ "tag_mrr": 0.6732295236311856,+ "pooled_auc": 0.9349122844270794,+ "pooled_auc_strict": 0.9179002350566563,+ "spec_top1": 0.5714285714285714,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "both",+ "all": {+ "cat_map": 0.7610068662109295,+ "cat_p5": 0.85,+ "cat_p10": 0.825,+ "class_map": 0.6130333886405536,+ "class_top3": 0.9,+ "tag_acc": 0.5684931506849316,+ "tag_mrr": 0.7059231884033613,+ "pooled_auc": 0.9602390982850321,+ "pooled_auc_strict": 0.9444559432467534,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7534607877045285,+ "cat_p5": 0.825,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6746554502564531,+ "class_top3": 0.8709677419354839,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.730970754156238,+ "pooled_auc": 0.9693265360880065,+ "pooled_auc_strict": 0.9548673784104389,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7673374351307555,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6112840868653101,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5263157894736842,+ "tag_mrr": 0.6732295236311856,+ "pooled_auc": 0.9484651101885965,+ "pooled_auc_strict": 0.9309656804506956,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "term_sd",+ "all": {+ "cat_map": 0.6980522113170913,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.7875,+ "class_map": 0.5970493925255879,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5365296803652968,+ "tag_mrr": 0.6865289901992316,+ "pooled_auc": 0.946785452621296,+ "pooled_auc_strict": 0.9314520082355633,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6856773137576235,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6238167679338643,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7158915345407281,+ "pooled_auc": 0.9578196941239603,+ "pooled_auc_strict": 0.9434927639383155,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.697124334270687,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.5876740068290082,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.4789473684210526,+ "tag_mrr": 0.648203142848226,+ "pooled_auc": 0.9328622846040324,+ "pooled_auc_strict": 0.9161987889472272,+ "spec_top1": 0.5714285714285714,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t1",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7586182144969197,+ "cat_p5": 0.825,+ "cat_p10": 0.8125,+ "class_map": 0.595603722072477,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.5365296803652968,+ "tag_mrr": 0.6865289901992316,+ "pooled_auc": 0.9574583808205797,+ "pooled_auc_strict": 0.9412123731954445,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7479252739679811,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6719452435180059,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7158915345407281,+ "pooled_auc": 0.9660998121813791,+ "pooled_auc_strict": 0.9513580071174377,+ "spec_top1": 0.85,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7880552005178514,+ "cat_p5": 0.8,+ "cat_p10": 0.7,+ "class_map": 0.6064812958523895,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.4789473684210526,+ "tag_mrr": 0.648203142848226,+ "pooled_auc": 0.9462222316597124,+ "pooled_auc_strict": 0.9280394486388431,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "raw",+ "all": {+ "cat_map": 0.718118751880866,+ "cat_p5": 0.825,+ "cat_p10": 0.7875,+ "class_map": 0.6273662801196275,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7097763961710694,+ "pooled_auc": 0.9475559331922284,+ "pooled_auc_strict": 0.9320818702620396,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7153805164806042,+ "cat_p5": 0.825,+ "cat_p10": 0.675,+ "class_map": 0.6480120430856741,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.625,+ "tag_mrr": 0.7339496162100214,+ "pooled_auc": 0.9563938824792058,+ "pooled_auc_strict": 0.9414861209964412,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7204871594793816,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.6354754574664218,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5421052631578948,+ "tag_mrr": 0.6782239826465425,+ "pooled_auc": 0.9363978043679064,+ "pooled_auc_strict": 0.9201422156644822,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "sound",+ "all": {+ "cat_map": 0.7665978592943665,+ "cat_p5": 0.85,+ "cat_p10": 0.8375,+ "class_map": 0.6414165364178294,+ "class_top3": 0.9142857142857143,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7097763961710694,+ "pooled_auc": 0.9601956813741452,+ "pooled_auc_strict": 0.9447617206081534,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7634370504834116,+ "cat_p5": 0.875,+ "cat_p10": 0.7375,+ "class_map": 0.6690742346670663,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.625,+ "tag_mrr": 0.7339496162100214,+ "pooled_auc": 0.9687255165012074,+ "pooled_auc_strict": 0.9539074733096086,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7911520237903075,+ "cat_p5": 0.8,+ "cat_p10": 0.7125,+ "class_map": 0.638673491556023,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5421052631578948,+ "tag_mrr": 0.6782239826465425,+ "pooled_auc": 0.9492304316590046,+ "pooled_auc_strict": 0.9329641410850931,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "center",+ "all": {+ "cat_map": 0.718118751880866,+ "cat_p5": 0.825,+ "cat_p10": 0.7875,+ "class_map": 0.6273662801196275,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5707762557077626,+ "tag_mrr": 0.7029040570512325,+ "pooled_auc": 0.9517398987843265,+ "pooled_auc_strict": 0.9370941376685028,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7153805164806042,+ "cat_p5": 0.825,+ "cat_p10": 0.675,+ "class_map": 0.6480120430856741,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.6088709677419355,+ "tag_mrr": 0.7304551820728292,+ "pooled_auc": 0.9609621679635095,+ "pooled_auc_strict": 0.9471051008303677,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7204871594793816,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.6354754574664218,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.6669425886019905,+ "pooled_auc": 0.9400527673670465,+ "pooled_auc_strict": 0.9243419371734437,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "both",+ "all": {+ "cat_map": 0.7695786878859658,+ "cat_p5": 0.85,+ "cat_p10": 0.825,+ "class_map": 0.6427246569683582,+ "class_top3": 0.9,+ "tag_acc": 0.5707762557077626,+ "tag_mrr": 0.7029040570512325,+ "pooled_auc": 0.9637309372625638,+ "pooled_auc_strict": 0.9486622353913768,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 1.0+ },+ "dev": {+ "cat_map": 0.7662596317937798,+ "cat_p5": 0.825,+ "cat_p10": 0.7625,+ "class_map": 0.6750202686222422,+ "class_top3": 0.8709677419354839,+ "tag_acc": 0.6088709677419355,+ "tag_mrr": 0.7304551820728292,+ "pooled_auc": 0.9718663804668635,+ "pooled_auc_strict": 0.9581542111506525,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7937494731754101,+ "cat_p5": 0.8,+ "cat_p10": 0.7250000000000001,+ "class_map": 0.6407116440892073,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.6669425886019905,+ "pooled_auc": 0.9530924295992723,+ "pooled_auc_strict": 0.9362688268884375,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "term_sd",+ "all": {+ "cat_map": 0.718118751880866,+ "cat_p5": 0.825,+ "cat_p10": 0.7875,+ "class_map": 0.6273662801196275,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5593607305936074,+ "tag_mrr": 0.6987727253599382,+ "pooled_auc": 0.9498845042331489,+ "pooled_auc_strict": 0.9355914257667582,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7153805164806042,+ "cat_p5": 0.825,+ "cat_p10": 0.675,+ "class_map": 0.6480120430856741,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.5846774193548387,+ "tag_mrr": 0.7193196458691004,+ "pooled_auc": 0.9590576871478401,+ "pooled_auc_strict": 0.9455406880189798,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7204871594793816,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.6354754574664218,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5263157894736842,+ "tag_mrr": 0.6719535870111372,+ "pooled_auc": 0.9382903160733435,+ "pooled_auc_strict": 0.9229582646686851,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t3",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7641445315624935,+ "cat_p5": 0.85,+ "cat_p10": 0.825,+ "class_map": 0.6208798291934929,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5593607305936074,+ "tag_mrr": 0.6987727253599382,+ "pooled_auc": 0.9621942160534028,+ "pooled_auc_strict": 0.9466517605876735,+ "spec_top1": 0.875,+ "spec_top3": 1.0+ },+ "dev": {+ "cat_map": 0.7522572784268877,+ "cat_p5": 0.825,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6733560017280501,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.5846774193548387,+ "tag_mrr": 0.7193196458691004,+ "pooled_auc": 0.9696361685001341,+ "pooled_auc_strict": 0.9554733096085409,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7919940719791706,+ "cat_p5": 0.825,+ "cat_p10": 0.7000000000000001,+ "class_map": 0.6408222726285296,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5263157894736842,+ "tag_mrr": 0.6719535870111372,+ "pooled_auc": 0.9524810571876515,+ "pooled_auc_strict": 0.9351709909426539,+ "spec_top1": 0.9047619047619048,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "raw",+ "all": {+ "cat_map": 0.7277025148739027,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.825,+ "class_map": 0.627087687258443,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.593607305936073,+ "tag_mrr": 0.7149215845506086,+ "pooled_auc": 0.9509806116357321,+ "pooled_auc_strict": 0.9355483052580997,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7229405850544317,+ "cat_p5": 0.85,+ "cat_p10": 0.6875,+ "class_map": 0.6431871490679331,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7338462142037805,+ "pooled_auc": 0.9592218942849476,+ "pooled_auc_strict": 0.9444052194543298,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7199503932575724,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.647772223495713,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6902199626875211,+ "pooled_auc": 0.9405800871316282,+ "pooled_auc_strict": 0.9243786647759935,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "sound",+ "all": {+ "cat_map": 0.7702130140324235,+ "cat_p5": 0.85,+ "cat_p10": 0.8625,+ "class_map": 0.649617613261983,+ "class_top3": 0.9,+ "tag_acc": 0.593607305936073,+ "tag_mrr": 0.7149215845506086,+ "pooled_auc": 0.9633435770107457,+ "pooled_auc_strict": 0.9481022739789366,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7706841705015869,+ "cat_p5": 0.8999999999999999,+ "cat_p10": 0.7375,+ "class_map": 0.6757453275548259,+ "class_top3": 0.9032258064516129,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7338462142037805,+ "pooled_auc": 0.9712261872819963,+ "pooled_auc_strict": 0.9565115065243179,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7699222515537358,+ "cat_p5": 0.8,+ "cat_p10": 0.7125,+ "class_map": 0.6454645266686625,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6902199626875211,+ "pooled_auc": 0.9532366462462972,+ "pooled_auc_strict": 0.9373035871689726,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "center",+ "all": {+ "cat_map": 0.7277025148739027,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.825,+ "class_map": 0.627087687258443,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5799086757990868,+ "tag_mrr": 0.712406871471145,+ "pooled_auc": 0.9537319887658743,+ "pooled_auc_strict": 0.9391610473835383,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7229405850544317,+ "cat_p5": 0.85,+ "cat_p10": 0.6875,+ "class_map": 0.6431871490679331,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7313790616536111,+ "pooled_auc": 0.9622554333243896,+ "pooled_auc_strict": 0.9485385527876631,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7199503932575724,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.647772223495713,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6876431706014001,+ "pooled_auc": 0.9428848991899095,+ "pooled_auc_strict": 0.9272050933200475,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "both",+ "all": {+ "cat_map": 0.7775335919864723,+ "cat_p5": 0.85,+ "cat_p10": 0.85,+ "class_map": 0.6424589753467237,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5799086757990868,+ "tag_mrr": 0.712406871471145,+ "pooled_auc": 0.9656601066427873,+ "pooled_auc_strict": 0.9508040386214509,+ "spec_top1": 0.7916666666666666,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.777586606292167,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6779892152094823,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7313790616536111,+ "pooled_auc": 0.9731773544405689,+ "pooled_auc_strict": 0.9595833926453143,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7932570388183038,+ "cat_p5": 0.825,+ "cat_p10": 0.7125,+ "class_map": 0.6390066424489291,+ "class_top3": 0.8793103448275862,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6876431706014001,+ "pooled_auc": 0.9558856317750858,+ "pooled_auc_strict": 0.9393954636620294,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "term_sd",+ "all": {+ "cat_map": 0.7277025148739027,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.825,+ "class_map": 0.627087687258443,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5776255707762558,+ "tag_mrr": 0.7099758184972561,+ "pooled_auc": 0.9517821285140562,+ "pooled_auc_strict": 0.9377659098033947,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7229405850544317,+ "cat_p5": 0.85,+ "cat_p10": 0.6875,+ "class_map": 0.6431871490679331,+ "class_top3": 0.8064516129032258,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7242494221728093,+ "pooled_auc": 0.9603209015293802,+ "pooled_auc_strict": 0.9471311981020166,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7199503932575724,+ "cat_p5": 0.75,+ "cat_p10": 0.675,+ "class_map": 0.647772223495713,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6913450094891653,+ "pooled_auc": 0.9409676140727135,+ "pooled_auc_strict": 0.9258701248099745,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "repeatpad",+ "templates": "t5",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7684321393745663,+ "cat_p5": 0.85,+ "cat_p10": 0.8375,+ "class_map": 0.6294999047685299,+ "class_top3": 0.9,+ "tag_acc": 0.5776255707762558,+ "tag_mrr": 0.7099758184972561,+ "pooled_auc": 0.9640507977857375,+ "pooled_auc_strict": 0.9488335070257682,+ "spec_top1": 0.7916666666666666,+ "spec_top3": 1.0+ },+ "dev": {+ "cat_map": 0.7597971337607392,+ "cat_p5": 0.825,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6736543996724086,+ "class_top3": 0.9032258064516129,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7242494221728093,+ "pooled_auc": 0.9708870405151596,+ "pooled_auc_strict": 0.9569504151838671,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.795543349737912,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6447098756668317,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6913450094891653,+ "pooled_auc": 0.9551512770693762,+ "pooled_auc_strict": 0.9382640938182655,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "raw",+ "all": {+ "cat_map": 0.6966951514439714,+ "cat_p5": 0.9,+ "cat_p10": 0.75,+ "class_map": 0.6103205175324865,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5662100456621004,+ "tag_mrr": 0.688933104494091,+ "pooled_auc": 0.9470330646911972,+ "pooled_auc_strict": 0.9317642914982698,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.6999603235345861,+ "cat_p5": 0.775,+ "cat_p10": 0.6375000000000001,+ "class_map": 0.6421799826780327,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.704020998923095,+ "pooled_auc": 0.9547206868795278,+ "pooled_auc_strict": 0.9396664294187426,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.6923689595029523,+ "cat_p5": 0.7,+ "cat_p10": 0.6375,+ "class_map": 0.6342054217314276,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.531578947368421,+ "tag_mrr": 0.6692394317657068,+ "pooled_auc": 0.9374489048382473,+ "pooled_auc_strict": 0.9218795909503187,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "sound",+ "all": {+ "cat_map": 0.7296165061572158,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.825,+ "class_map": 0.5990407658743546,+ "class_top3": 0.8571428571428571,+ "tag_acc": 0.5662100456621004,+ "tag_mrr": 0.688933104494091,+ "pooled_auc": 0.9525578666015413,+ "pooled_auc_strict": 0.9376875363876573,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7343754496700743,+ "cat_p5": 0.825,+ "cat_p10": 0.6875,+ "class_map": 0.6395514066218183,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.704020998923095,+ "pooled_auc": 0.9570480279044808,+ "pooled_auc_strict": 0.9420365361803085,+ "spec_top1": 0.85,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7421549661065795,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.6013351535309124,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.531578947368421,+ "tag_mrr": 0.6692394317657068,+ "pooled_auc": 0.9466831940713687,+ "pooled_auc_strict": 0.9319892627652372,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "center",+ "all": {+ "cat_map": 0.6966951514439714,+ "cat_p5": 0.9,+ "cat_p10": 0.75,+ "class_map": 0.6103205175324865,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5684931506849316,+ "tag_mrr": 0.700051267624108,+ "pooled_auc": 0.9540455741886464,+ "pooled_auc_strict": 0.9391905508894626,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.6999603235345861,+ "cat_p5": 0.775,+ "cat_p10": 0.6375000000000001,+ "class_map": 0.6421799826780327,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.711020057978075,+ "pooled_auc": 0.9625156962704589,+ "pooled_auc_strict": 0.9483321470937129,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.6923689595029523,+ "cat_p5": 0.7,+ "cat_p10": 0.6375,+ "class_map": 0.6342054217314276,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6857341096884034,+ "pooled_auc": 0.9431299790133811,+ "pooled_auc_strict": 0.9273655769746675,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "both",+ "all": {+ "cat_map": 0.7425693181695112,+ "cat_p5": 0.85,+ "cat_p10": 0.8125,+ "class_map": 0.60919922426989,+ "class_top3": 0.8571428571428571,+ "tag_acc": 0.5684931506849316,+ "tag_mrr": 0.700051267624108,+ "pooled_auc": 0.9584685023879301,+ "pooled_auc_strict": 0.9439301756411793,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.743093620970253,+ "cat_p5": 0.825,+ "cat_p10": 0.6875,+ "class_map": 0.6592606009695009,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.711020057978075,+ "pooled_auc": 0.9637552991682318,+ "pooled_auc_strict": 0.9497508896797153,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.747603229487134,+ "cat_p5": 0.775,+ "cat_p10": 0.7125,+ "class_map": 0.6141318587911977,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6857341096884034,+ "pooled_auc": 0.9515237417761121,+ "pooled_auc_strict": 0.9362305024336028,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "term_sd",+ "all": {+ "cat_map": 0.6966951514439714,+ "cat_p5": 0.9,+ "cat_p10": 0.75,+ "class_map": 0.6103205175324865,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5662100456621004,+ "tag_mrr": 0.6955952165165974,+ "pooled_auc": 0.9520811285683274,+ "pooled_auc_strict": 0.9379881695480244,+ "spec_top1": 0.7083333333333334,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.6999603235345861,+ "cat_p5": 0.775,+ "cat_p10": 0.6375000000000001,+ "class_map": 0.6421799826780327,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.5766129032258065,+ "tag_mrr": 0.7039788209421436,+ "pooled_auc": 0.9599737053930776,+ "pooled_auc_strict": 0.9465689205219454,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.6923689595029523,+ "cat_p5": 0.7,+ "cat_p10": 0.6375,+ "class_map": 0.6342054217314276,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6846524065295687,+ "pooled_auc": 0.9417612480137032,+ "pooled_auc_strict": 0.9267723463508731,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t1",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7357901241335416,+ "cat_p5": 0.825,+ "cat_p10": 0.8,+ "class_map": 0.5762451701876967,+ "class_top3": 0.8142857142857143,+ "tag_acc": 0.5662100456621004,+ "tag_mrr": 0.6955952165165974,+ "pooled_auc": 0.9544702458482579,+ "pooled_auc_strict": 0.9397197983957355,+ "spec_top1": 0.875,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7328908579222572,+ "cat_p5": 0.8,+ "cat_p10": 0.675,+ "class_map": 0.6431110174811173,+ "class_top3": 0.8225806451612904,+ "tag_acc": 0.5766129032258065,+ "tag_mrr": 0.7039788209421436,+ "pooled_auc": 0.9587121008854307,+ "pooled_auc_strict": 0.9446315539739027,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7635682012315999,+ "cat_p5": 0.775,+ "cat_p10": 0.6749999999999999,+ "class_map": 0.6058761528294955,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6846524065295687,+ "pooled_auc": 0.9489826975414158,+ "pooled_auc_strict": 0.9332092579108062,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "raw",+ "all": {+ "cat_map": 0.7113266964725198,+ "cat_p5": 0.85,+ "cat_p10": 0.7749999999999999,+ "class_map": 0.6242070802503793,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7051300820155875,+ "pooled_auc": 0.9528507611527189,+ "pooled_auc_strict": 0.9377459381993843,+ "spec_top1": 0.75,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7082306928343325,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6412667770398142,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7232177412169984,+ "pooled_auc": 0.96023557821304,+ "pooled_auc_strict": 0.9453508896797153,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7129527355473899,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6568296279529588,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5473684210526316,+ "tag_mrr": 0.68152092684743,+ "pooled_auc": 0.9435396250721083,+ "pooled_auc_strict": 0.9281328644975025,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "sound",+ "all": {+ "cat_map": 0.7479251873265214,+ "cat_p5": 0.85,+ "cat_p10": 0.8374999999999999,+ "class_map": 0.6260961543743475,+ "class_top3": 0.8571428571428571,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7051300820155875,+ "pooled_auc": 0.9592016715510692,+ "pooled_auc_strict": 0.9445763780709368,+ "spec_top1": 0.875,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7428199736405021,+ "cat_p5": 0.875,+ "cat_p10": 0.7,+ "class_map": 0.6625634652022035,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7232177412169984,+ "pooled_auc": 0.9641454252750201,+ "pooled_auc_strict": 0.9495302491103202,+ "spec_top1": 0.8,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7732243482406984,+ "cat_p5": 0.8,+ "cat_p10": 0.7000000000000001,+ "class_map": 0.619360899639458,+ "class_top3": 0.8103448275862069,+ "tag_acc": 0.5473684210526316,+ "tag_mrr": 0.68152092684743,+ "pooled_auc": 0.9527509104228822,+ "pooled_auc_strict": 0.9381084007204997,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "center",+ "all": {+ "cat_map": 0.7113266964725198,+ "cat_p5": 0.85,+ "cat_p10": 0.7749999999999999,+ "class_map": 0.6242070802503793,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5730593607305936,+ "tag_mrr": 0.7024972971901691,+ "pooled_auc": 0.9569955497666341,+ "pooled_auc_strict": 0.9427986027139594,+ "spec_top1": 0.75,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7082306928343325,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6412667770398142,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7192411307330662,+ "pooled_auc": 0.9652535551381809,+ "pooled_auc_strict": 0.9514296559905101,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7129527355473899,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6568296279529588,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5368421052631579,+ "tag_mrr": 0.6806421881447032,+ "pooled_auc": 0.9465230516596417,+ "pooled_auc_strict": 0.9319054280202864,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "both",+ "all": {+ "cat_map": 0.7545544072897863,+ "cat_p5": 0.85,+ "cat_p10": 0.8374999999999999,+ "class_map": 0.6295778568154784,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5730593607305936,+ "tag_mrr": 0.7024972971901691,+ "pooled_auc": 0.9623862002062303,+ "pooled_auc_strict": 0.9479070969397451,+ "spec_top1": 0.9166666666666666,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7569497491591481,+ "cat_p5": 0.85,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6593895422627897,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7192411307330662,+ "pooled_auc": 0.9682763616850013,+ "pooled_auc_strict": 0.9543800711743772,+ "spec_top1": 0.85,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7768694680931955,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6320349511109097,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5368421052631579,+ "tag_mrr": 0.6806421881447032,+ "pooled_auc": 0.9545637933048086,+ "pooled_auc_strict": 0.9393882778267479,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "term_sd",+ "all": {+ "cat_map": 0.7113266964725198,+ "cat_p5": 0.85,+ "cat_p10": 0.7749999999999999,+ "class_map": 0.6242070802503793,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5707762557077626,+ "tag_mrr": 0.7004832234156526,+ "pooled_auc": 0.9547480462390101,+ "pooled_auc_strict": 0.9412084393946546,+ "spec_top1": 0.75,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7082306928343325,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6412667770398142,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7099978762277149,+ "pooled_auc": 0.9625784813522941,+ "pooled_auc_strict": 0.9494111506524318,+ "spec_top1": 0.6,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7129527355473899,+ "cat_p5": 0.75,+ "cat_p10": 0.6625,+ "class_map": 0.6568296279529588,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6880640976399084,+ "pooled_auc": 0.9446561981306691,+ "pooled_auc_strict": 0.9306151713741873,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t3",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7419979603370317,+ "cat_p5": 0.825,+ "cat_p10": 0.8125,+ "class_map": 0.6137491601963941,+ "class_top3": 0.8142857142857143,+ "tag_acc": 0.5707762557077626,+ "tag_mrr": 0.7004832234156526,+ "pooled_auc": 0.9591180600781505,+ "pooled_auc_strict": 0.9443836218322312,+ "spec_top1": 0.9166666666666666,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7383885442934307,+ "cat_p5": 0.8,+ "cat_p10": 0.7,+ "class_map": 0.6636712966205031,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.5806451612903226,+ "tag_mrr": 0.7099978762277149,+ "pooled_auc": 0.9639930238797961,+ "pooled_auc_strict": 0.9498642941874259,+ "spec_top1": 0.8,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7736227636847813,+ "cat_p5": 0.8,+ "cat_p10": 0.6875,+ "class_map": 0.6297619157865565,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6880640976399084,+ "pooled_auc": 0.9527447170699425,+ "pooled_auc_strict": 0.9371798311169024,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "raw",+ "all": {+ "cat_map": 0.7195432854073197,+ "cat_p5": 0.875,+ "cat_p10": 0.7875,+ "class_map": 0.6207924807074807,+ "class_top3": 0.8285714285714286,+ "tag_acc": 0.6004566210045662,+ "tag_mrr": 0.7122834126130668,+ "pooled_auc": 0.9552127428633452,+ "pooled_auc_strict": 0.9398409897200707,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7192035638255845,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6365957342756754,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7251502314078777,+ "pooled_auc": 0.96239602897773,+ "pooled_auc_strict": 0.9472868327402135,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7103068995186603,+ "cat_p5": 0.75,+ "cat_p10": 0.6375,+ "class_map": 0.6498509450129192,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5736842105263158,+ "tag_mrr": 0.6954888280808925,+ "pooled_auc": 0.9461948039538365,+ "pooled_auc_strict": 0.9304483003104281,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "sound",+ "all": {+ "cat_map": 0.7528268584243417,+ "cat_p5": 0.85,+ "cat_p10": 0.8500000000000001,+ "class_map": 0.631513081220779,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.6004566210045662,+ "tag_mrr": 0.7122834126130668,+ "pooled_auc": 0.9614140684359058,+ "pooled_auc_strict": 0.946796857316809,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7561693052788976,+ "cat_p5": 0.8999999999999999,+ "cat_p10": 0.725,+ "class_map": 0.6608365270149242,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7251502314078777,+ "pooled_auc": 0.9661851354977193,+ "pooled_auc_strict": 0.9514424673784104,+ "spec_top1": 0.8,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7558766882096197,+ "cat_p5": 0.8,+ "cat_p10": 0.6875,+ "class_map": 0.6321180749414527,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5736842105263158,+ "tag_mrr": 0.6954888280808925,+ "pooled_auc": 0.9552176344223018,+ "pooled_auc_strict": 0.9407823298713576,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "center",+ "all": {+ "cat_map": 0.7195432854073197,+ "cat_p5": 0.875,+ "cat_p10": 0.7875,+ "class_map": 0.6207924807074807,+ "class_top3": 0.8285714285714286,+ "tag_acc": 0.591324200913242,+ "tag_mrr": 0.713834516703891,+ "pooled_auc": 0.9582488738738739,+ "pooled_auc_strict": 0.9438378826226468,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7192035638255845,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6365957342756754,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7193428063719185,+ "pooled_auc": 0.9658282801180574,+ "pooled_auc_strict": 0.951911743772242,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7103068995186603,+ "cat_p5": 0.75,+ "cat_p10": 0.6375,+ "class_map": 0.6498509450129192,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5842105263157895,+ "tag_mrr": 0.7066447491372023,+ "pooled_auc": 0.9485571257179866,+ "pooled_auc_strict": 0.9336044788512884,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "both",+ "all": {+ "cat_map": 0.7583908419074668,+ "cat_p5": 0.85,+ "cat_p10": 0.8374999999999999,+ "class_map": 0.631183476723012,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.591324200913242,+ "tag_mrr": 0.713834516703891,+ "pooled_auc": 0.9637078720286552,+ "pooled_auc_strict": 0.9492772700148758,+ "spec_top1": 0.875,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7592935004031254,+ "cat_p5": 0.85,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6625594786345831,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7193428063719185,+ "pooled_auc": 0.9691033002414811,+ "pooled_auc_strict": 0.9551772241992883,+ "spec_top1": 0.85,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7741438076784091,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6288217086231476,+ "class_top3": 0.8793103448275862,+ "tag_acc": 0.5842105263157895,+ "tag_mrr": 0.7066447491372023,+ "pooled_auc": 0.9565430119514017,+ "pooled_auc_strict": 0.9414977196949373,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "term_sd",+ "all": {+ "cat_map": 0.7195432854073197,+ "cat_p5": 0.875,+ "cat_p10": 0.7875,+ "class_map": 0.6207924807074807,+ "class_top3": 0.8285714285714286,+ "tag_acc": 0.5753424657534246,+ "tag_mrr": 0.7039218843087036,+ "pooled_auc": 0.9559677900792358,+ "pooled_auc_strict": 0.9423465182231808,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7192035638255845,+ "cat_p5": 0.825,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6365957342756754,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5887096774193549,+ "tag_mrr": 0.7139980675215183,+ "pooled_auc": 0.9630893480010733,+ "pooled_auc_strict": 0.9499814946619217,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7103068995186603,+ "cat_p5": 0.75,+ "cat_p10": 0.6375,+ "class_map": 0.6498509450129192,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6907698135888196,+ "pooled_auc": 0.9468663403654433,+ "pooled_auc_strict": 0.9326407784974259,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.8571428571428571+ }+ },+ {+ "pad": "zero",+ "templates": "t5",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7440666122282521,+ "cat_p5": 0.85,+ "cat_p10": 0.8,+ "class_map": 0.6171414447933016,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.5753424657534246,+ "tag_mrr": 0.7039218843087036,+ "pooled_auc": 0.9602011084880061,+ "pooled_auc_strict": 0.9455557431675933,+ "spec_top1": 0.875,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7428221662902166,+ "cat_p5": 0.8,+ "cat_p10": 0.7124999999999999,+ "class_map": 0.6513844220634469,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5887096774193549,+ "tag_mrr": 0.7139980675215183,+ "pooled_auc": 0.9645468204990609,+ "pooled_auc_strict": 0.9504583629893238,+ "spec_top1": 0.8,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7663582183951299,+ "cat_p5": 0.8,+ "cat_p10": 0.7000000000000001,+ "class_map": 0.6256447411677083,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6907698135888196,+ "pooled_auc": 0.9544602758342446,+ "pooled_auc_strict": 0.9390521404208025,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "raw",+ "all": {+ "cat_map": 0.706922872057108,+ "cat_p5": 0.875,+ "cat_p10": 0.7875000000000001,+ "class_map": 0.6107295823937894,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.5616438356164384,+ "tag_mrr": 0.6909010612328351,+ "pooled_auc": 0.9446412338543363,+ "pooled_auc_strict": 0.9287270946884006,+ "spec_top1": 0.75,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6884970537189887,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.6423744153944477,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7138462062406248,+ "pooled_auc": 0.9530222699221894,+ "pooled_auc_strict": 0.9374448398576513,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7079575497135653,+ "cat_p5": 0.75,+ "cat_p10": 0.6625000000000001,+ "class_map": 0.6028376226217312,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.660951608801615,+ "pooled_auc": 0.93430268154487,+ "pooled_auc_strict": 0.917817198737848,+ "spec_top1": 0.6190476190476191,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "sound",+ "all": {+ "cat_map": 0.7490033176025689,+ "cat_p5": 0.8500000000000001,+ "cat_p10": 0.825,+ "class_map": 0.6199766764100778,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.5616438356164384,+ "tag_mrr": 0.6909010612328351,+ "pooled_auc": 0.956998772115489,+ "pooled_auc_strict": 0.940935040239756,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7477036476412733,+ "cat_p5": 0.825,+ "cat_p10": 0.7375,+ "class_map": 0.6691017688292265,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7138462062406248,+ "pooled_auc": 0.9636592433592702,+ "pooled_auc_strict": 0.9478372479240806,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7696312428996632,+ "cat_p5": 0.8,+ "cat_p10": 0.7,+ "class_map": 0.6155754649787434,+ "class_top3": 0.7931034482758621,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.660951608801615,+ "pooled_auc": 0.948381942306263,+ "pooled_auc_strict": 0.9320555321350554,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "center",+ "all": {+ "cat_map": 0.706922872057108,+ "cat_p5": 0.875,+ "cat_p10": 0.7875000000000001,+ "class_map": 0.6107295823937894,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.5776255707762558,+ "tag_mrr": 0.7087323497675215,+ "pooled_auc": 0.949996608053837,+ "pooled_auc_strict": 0.9345255170527238,+ "spec_top1": 0.75,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6884970537189887,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.6423744153944477,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.6129032258064516,+ "tag_mrr": 0.7366968087148714,+ "pooled_auc": 0.9584314462033807,+ "pooled_auc_strict": 0.9437812574139977,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7079575497135653,+ "cat_p5": 0.75,+ "cat_p10": 0.6625000000000001,+ "class_map": 0.6028376226217312,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.531578947368421,+ "tag_mrr": 0.6722313717730859,+ "pooled_auc": 0.9392547096025283,+ "pooled_auc_strict": 0.922591787069329,+ "spec_top1": 0.6190476190476191,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "both",+ "all": {+ "cat_map": 0.7673928880987917,+ "cat_p5": 0.85,+ "cat_p10": 0.825,+ "class_map": 0.6187623766919265,+ "class_top3": 0.8571428571428571,+ "tag_acc": 0.5776255707762558,+ "tag_mrr": 0.7087323497675215,+ "pooled_auc": 0.9614535846087051,+ "pooled_auc_strict": 0.9457649911096102,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7531848837739687,+ "cat_p5": 0.8,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6936505103141615,+ "class_top3": 0.9032258064516129,+ "tag_acc": 0.6129032258064516,+ "tag_mrr": 0.7366968087148714,+ "pooled_auc": 0.9673871746713174,+ "pooled_auc_strict": 0.9529214709371293,+ "spec_top1": 0.85,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7791705700145516,+ "cat_p5": 0.8,+ "cat_p10": 0.7250000000000001,+ "class_map": 0.6199613689407614,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.531578947368421,+ "tag_mrr": 0.6722313717730859,+ "pooled_auc": 0.9536958391285422,+ "pooled_auc_strict": 0.9363742191392328,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "term_sd",+ "all": {+ "cat_map": 0.706922872057108,+ "cat_p5": 0.875,+ "cat_p10": 0.7875000000000001,+ "class_map": 0.6107295823937894,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.54337899543379,+ "tag_mrr": 0.6884737470690645,+ "pooled_auc": 0.947702126071855,+ "pooled_auc_strict": 0.9326695195800395,+ "spec_top1": 0.75,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.6884970537189887,+ "cat_p5": 0.75,+ "cat_p10": 0.6499999999999999,+ "class_map": 0.6423744153944477,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7203287804617625,+ "pooled_auc": 0.9560660048296217,+ "pooled_auc_strict": 0.941944009489917,+ "spec_top1": 0.7,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7079575497135653,+ "cat_p5": 0.75,+ "cat_p10": 0.6625000000000001,+ "class_map": 0.6028376226217312,+ "class_top3": 0.7758620689655172,+ "tag_acc": 0.4789473684210526,+ "tag_mrr": 0.6468945455880694,+ "pooled_auc": 0.9369729014265946,+ "pooled_auc_strict": 0.920643627281902,+ "spec_top1": 0.6190476190476191,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t1",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7663527622016136,+ "cat_p5": 0.85,+ "cat_p10": 0.8125,+ "class_map": 0.6053154405539275,+ "class_top3": 0.8428571428571429,+ "tag_acc": 0.54337899543379,+ "tag_mrr": 0.6884737470690645,+ "pooled_auc": 0.9586613345272984,+ "pooled_auc_strict": 0.9423814685301989,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9166666666666666+ },+ "dev": {+ "cat_map": 0.7489604716512064,+ "cat_p5": 0.8,+ "cat_p10": 0.75,+ "class_map": 0.6948401730457527,+ "class_top3": 0.9032258064516129,+ "tag_acc": 0.592741935483871,+ "tag_mrr": 0.7203287804617625,+ "pooled_auc": 0.9639989267507378,+ "pooled_auc_strict": 0.9490481613285884,+ "spec_top1": 0.85,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7970968265684556,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6219079826197252,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.4789473684210526,+ "tag_mrr": 0.6468945455880694,+ "pooled_auc": 0.9516599955407858,+ "pooled_auc_strict": 0.9335222409586224,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "raw",+ "all": {+ "cat_map": 0.733627480353698,+ "cat_p5": 0.875,+ "cat_p10": 0.8125,+ "class_map": 0.631684550946142,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.591324200913242,+ "tag_mrr": 0.7090230796224233,+ "pooled_auc": 0.9471253256268316,+ "pooled_auc_strict": 0.9317075539868769,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.721332154539629,+ "cat_p5": 0.825,+ "cat_p10": 0.7,+ "class_map": 0.6685541340636711,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.625,+ "tag_mrr": 0.7321579254982588,+ "pooled_auc": 0.955209551918433,+ "pooled_auc_strict": 0.9403696322657177,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7408307618711064,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.6341914042688647,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5473684210526316,+ "tag_mrr": 0.6788260176371224,+ "pooled_auc": 0.9368516886619173,+ "pooled_auc_strict": 0.9206635879354617,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "sound",+ "all": {+ "cat_map": 0.7712992046064168,+ "cat_p5": 0.85,+ "cat_p10": 0.8375,+ "class_map": 0.6432533280537299,+ "class_top3": 0.9,+ "tag_acc": 0.591324200913242,+ "tag_mrr": 0.7090230796224233,+ "pooled_auc": 0.959616845761424,+ "pooled_auc_strict": 0.9441547048862647,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 1.0+ },+ "dev": {+ "cat_map": 0.7656542451859354,+ "cat_p5": 0.875,+ "cat_p10": 0.7375,+ "class_map": 0.6948406693679899,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.625,+ "tag_mrr": 0.7321579254982588,+ "pooled_auc": 0.9671735980681513,+ "pooled_auc_strict": 0.9523753262158956,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7943363173833515,+ "cat_p5": 0.775,+ "cat_p10": 0.7375,+ "class_map": 0.6240989688025537,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5473684210526316,+ "tag_mrr": 0.6788260176371224,+ "pooled_auc": 0.9499099309529624,+ "pooled_auc_strict": 0.9336116646865699,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "center",+ "all": {+ "cat_map": 0.733627480353698,+ "cat_p5": 0.875,+ "cat_p10": 0.8125,+ "class_map": 0.631684550946142,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5821917808219178,+ "tag_mrr": 0.7082388382353201,+ "pooled_auc": 0.9511218861934223,+ "pooled_auc_strict": 0.93656489016223,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.721332154539629,+ "cat_p5": 0.825,+ "cat_p10": 0.7,+ "class_map": 0.6685541340636711,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.6129032258064516,+ "tag_mrr": 0.7332454862192767,+ "pooled_auc": 0.9593807351757445,+ "pooled_auc_strict": 0.9456540925266904,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7408307618711064,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.6341914042688647,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5421052631578948,+ "tag_mrr": 0.6755985819194191,+ "pooled_auc": 0.9407968544845184,+ "pooled_auc_strict": 0.9251156121054178,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "both",+ "all": {+ "cat_map": 0.775188178344212,+ "cat_p5": 0.85,+ "cat_p10": 0.825,+ "class_map": 0.6431672182428108,+ "class_top3": 0.9,+ "tag_acc": 0.5821917808219178,+ "tag_mrr": 0.7082388382353201,+ "pooled_auc": 0.9633947953978075,+ "pooled_auc_strict": 0.9482859522158192,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7703344479824905,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6863326451282192,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.6129032258064516,+ "tag_mrr": 0.7332454862192767,+ "pooled_auc": 0.9702516769519721,+ "pooled_auc_strict": 0.9564493475682088,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7965686178706268,+ "cat_p5": 0.775,+ "cat_p10": 0.7375,+ "class_map": 0.6331310086379538,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5421052631578948,+ "tag_mrr": 0.6755985819194191,+ "pooled_auc": 0.9545230941283475,+ "pooled_auc_strict": 0.9377227608937263,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "term_sd",+ "all": {+ "cat_map": 0.733627480353698,+ "cat_p5": 0.875,+ "cat_p10": 0.8125,+ "class_map": 0.631684550946142,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5639269406392694,+ "tag_mrr": 0.699348237848909,+ "pooled_auc": 0.9491552358080972,+ "pooled_auc_strict": 0.9350142161508546,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.721332154539629,+ "cat_p5": 0.825,+ "cat_p10": 0.7,+ "class_map": 0.6685541340636711,+ "class_top3": 0.8548387096774194,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7250881387877356,+ "pooled_auc": 0.9572465790179769,+ "pooled_auc_strict": 0.9439345195729537,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7408307618711064,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.6341914042688647,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.6657508934655983,+ "pooled_auc": 0.9390768718966878,+ "pooled_auc_strict": 0.9237942168397655,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t3",+ "method": "both_sd",+ "all": {+ "cat_map": 0.7682368356425465,+ "cat_p5": 0.85,+ "cat_p10": 0.8125,+ "class_map": 0.6256953056101257,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.5639269406392694,+ "tag_mrr": 0.699348237848909,+ "pooled_auc": 0.961727653858678,+ "pooled_auc_strict": 0.9461088960786663,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7569966515589786,+ "cat_p5": 0.825,+ "cat_p10": 0.7375,+ "class_map": 0.6843928205030713,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7250881387877356,+ "pooled_auc": 0.9678078883820768,+ "pooled_auc_strict": 0.9535316725978648,+ "spec_top1": 0.9,+ "spec_top3": 0.95+ },+ "test": {+ "cat_map": 0.7923047240435662,+ "cat_p5": 0.8,+ "cat_p10": 0.725,+ "class_map": 0.6436344180966361,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5210526315789473,+ "tag_mrr": 0.6657508934655983,+ "pooled_auc": 0.9538444795990955,+ "pooled_auc_strict": 0.9364492711966173,+ "spec_top1": 0.9047619047619048,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "raw",+ "all": {+ "cat_map": 0.7403152944695368,+ "cat_p5": 0.875,+ "cat_p10": 0.8375,+ "class_map": 0.6306648972281436,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.6004566210045662,+ "tag_mrr": 0.7167575604039297,+ "pooled_auc": 0.9505566183653533,+ "pooled_auc_strict": 0.935182764384699,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7288654463711456,+ "cat_p5": 0.85,+ "cat_p10": 0.7,+ "class_map": 0.6623461312956833,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.625,+ "tag_mrr": 0.7350574492825074,+ "pooled_auc": 0.9579908773812718,+ "pooled_auc_strict": 0.9432763938315539,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7339504300278596,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.644370763116193,+ "class_top3": 0.8103448275862069,+ "tag_acc": 0.5684210526315789,+ "tag_mrr": 0.6928713896571546,+ "pooled_auc": 0.9411640318373732,+ "pooled_auc_strict": 0.9250150104114769,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "sound",+ "all": {+ "cat_map": 0.7754300541392372,+ "cat_p5": 0.85,+ "cat_p10": 0.8625,+ "class_map": 0.6522621153856848,+ "class_top3": 0.9142857142857143,+ "tag_acc": 0.6004566210045662,+ "tag_mrr": 0.7167575604039297,+ "pooled_auc": 0.9627605014653208,+ "pooled_auc_strict": 0.947461972250364,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 1.0+ },+ "dev": {+ "cat_map": 0.7717425709437424,+ "cat_p5": 0.8999999999999999,+ "cat_p10": 0.75,+ "class_map": 0.6858513149747846,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.625,+ "tag_mrr": 0.7350574492825074,+ "pooled_auc": 0.969741346927824,+ "pooled_auc_strict": 0.9549988137603797,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7797688455311863,+ "cat_p5": 0.8,+ "cat_p10": 0.7375,+ "class_map": 0.6508153250257953,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5684210526315789,+ "tag_mrr": 0.6928713896571546,+ "pooled_auc": 0.9538878330696734,+ "pooled_auc_strict": 0.9379255611338929,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "center",+ "all": {+ "cat_map": 0.7403152944695368,+ "cat_p5": 0.875,+ "cat_p10": 0.8375,+ "class_map": 0.6306648972281436,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7166557925025382,+ "pooled_auc": 0.9532756024096386,+ "pooled_auc_strict": 0.9387681212046387,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7288654463711456,+ "cat_p5": 0.85,+ "cat_p10": 0.7,+ "class_map": 0.6623461312956833,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.6048387096774194,+ "tag_mrr": 0.7341978621034598,+ "pooled_auc": 0.9607158572578481,+ "pooled_auc_strict": 0.9471805456702254,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7339504300278596,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.644370763116193,+ "class_top3": 0.8103448275862069,+ "tag_acc": 0.5684210526315789,+ "tag_mrr": 0.69375877533923,+ "pooled_auc": 0.9439333453661334,+ "pooled_auc_strict": 0.9281464377419231,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "both",+ "all": {+ "cat_map": 0.7842722653956207,+ "cat_p5": 0.85,+ "cat_p10": 0.8500000000000001,+ "class_map": 0.6437703791946205,+ "class_top3": 0.8714285714285714,+ "tag_acc": 0.589041095890411,+ "tag_mrr": 0.7166557925025382,+ "pooled_auc": 0.965359749810051,+ "pooled_auc_strict": 0.9504587719521214,+ "spec_top1": 0.7916666666666666,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7786263627759495,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6880803109400951,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.6048387096774194,+ "tag_mrr": 0.7341978621034598,+ "pooled_auc": 0.9716114837671049,+ "pooled_auc_strict": 0.9579487544483986,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.8002964532472743,+ "cat_p5": 0.8,+ "cat_p10": 0.7375,+ "class_map": 0.6358470829434432,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5684210526315789,+ "tag_mrr": 0.69375877533923,+ "pooled_auc": 0.9572817904806395,+ "pooled_auc_strict": 0.9407392148596686,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "term_sd",+ "all": {+ "cat_map": 0.7403152944695368,+ "cat_p5": 0.875,+ "cat_p10": 0.8375,+ "class_map": 0.6306648972281436,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5799086757990868,+ "tag_mrr": 0.7094754522618477,+ "pooled_auc": 0.9510448890155215,+ "pooled_auc_strict": 0.9370711400638849,+ "spec_top1": 0.8333333333333334,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7288654463711456,+ "cat_p5": 0.85,+ "cat_p10": 0.7,+ "class_map": 0.6623461312956833,+ "class_top3": 0.8387096774193549,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7250974680232269,+ "pooled_auc": 0.9583783203649048,+ "pooled_auc_strict": 0.9453580071174377,+ "spec_top1": 0.8,+ "spec_top3": 0.9+ },+ "test": {+ "cat_map": 0.7339504300278596,+ "cat_p5": 0.775,+ "cat_p10": 0.675,+ "class_map": 0.644370763116193,+ "class_top3": 0.8103448275862069,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6890846106364683,+ "pooled_auc": 0.9418727283666182,+ "pooled_auc_strict": 0.9265998863041173,+ "spec_top1": 0.7142857142857143,+ "spec_top3": 0.9523809523809523+ }+ },+ {+ "pad": "spaced",+ "templates": "t5",+ "method": "both_sd",+ "all": {+ "cat_map": 0.773238958326166,+ "cat_p5": 0.85,+ "cat_p10": 0.8,+ "class_map": 0.634303067677147,+ "class_top3": 0.8857142857142857,+ "tag_acc": 0.5799086757990868,+ "tag_mrr": 0.7094754522618477,+ "pooled_auc": 0.9635615095517204,+ "pooled_auc_strict": 0.9482228601031503,+ "spec_top1": 0.875,+ "spec_top3": 0.9583333333333334+ },+ "dev": {+ "cat_map": 0.7612631638485738,+ "cat_p5": 0.8,+ "cat_p10": 0.7375,+ "class_map": 0.6871950955434535,+ "class_top3": 0.9354838709677419,+ "tag_acc": 0.5967741935483871,+ "tag_mrr": 0.7250974680232269,+ "pooled_auc": 0.9690845183793936,+ "pooled_auc_strict": 0.9550320284697509,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7968612828098313,+ "cat_p5": 0.775,+ "cat_p10": 0.7374999999999999,+ "class_map": 0.6471411183926385,+ "class_top3": 0.8275862068965517,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6890846106364683,+ "pooled_auc": 0.9563881781279087,+ "pooled_auc_strict": 0.9393547439287676,+ "spec_top1": 0.8571428571428571,+ "spec_top3": 0.9047619047619048+ }+ }+ ],+ "thresholds": {+ "rule": "on the 219 tuning files (lenient labels), 0.5 z grid, top-down: strong = lowest t with LR(z >= t) >= 100; plausible = lowest t below it with band LR >= 20; weak = lowest t below that with band LR >= 5",+ "grid_lenient": [+ {+ "z": 0.0,+ "tp": 438,+ "fp": 5459,+ "recall": 0.9865,+ "fpr": 0.41107,+ "lr": 2.4,+ "precision": 0.0743+ },+ {+ "z": 0.25,+ "tp": 431,+ "fp": 4270,+ "recall": 0.9707,+ "fpr": 0.32154,+ "lr": 3.0,+ "precision": 0.0917+ },+ {+ "z": 0.5,+ "tp": 424,+ "fp": 3314,+ "recall": 0.955,+ "fpr": 0.24955,+ "lr": 3.8,+ "precision": 0.1134+ },+ {+ "z": 0.75,+ "tp": 419,+ "fp": 2477,+ "recall": 0.9437,+ "fpr": 0.18652,+ "lr": 5.1,+ "precision": 0.1447+ },+ {+ "z": 1.0,+ "tp": 411,+ "fp": 1765,+ "recall": 0.9257,+ "fpr": 0.13291,+ "lr": 7.0,+ "precision": 0.1889+ },+ {+ "z": 1.25,+ "tp": 400,+ "fp": 1214,+ "recall": 0.9009,+ "fpr": 0.09142,+ "lr": 9.9,+ "precision": 0.2478+ },+ {+ "z": 1.5,+ "tp": 388,+ "fp": 853,+ "recall": 0.8739,+ "fpr": 0.06423,+ "lr": 13.6,+ "precision": 0.3127+ },+ {+ "z": 1.75,+ "tp": 380,+ "fp": 567,+ "recall": 0.8559,+ "fpr": 0.0427,+ "lr": 20.0,+ "precision": 0.4013+ },+ {+ "z": 2.0,+ "tp": 365,+ "fp": 359,+ "recall": 0.8221,+ "fpr": 0.02703,+ "lr": 30.4,+ "precision": 0.5041+ },+ {+ "z": 2.25,+ "tp": 345,+ "fp": 235,+ "recall": 0.777,+ "fpr": 0.0177,+ "lr": 43.9,+ "precision": 0.5948+ },+ {+ "z": 2.5,+ "tp": 309,+ "fp": 123,+ "recall": 0.6959,+ "fpr": 0.00926,+ "lr": 75.1,+ "precision": 0.7153+ },+ {+ "z": 2.75,+ "tp": 279,+ "fp": 76,+ "recall": 0.6284,+ "fpr": 0.00572,+ "lr": 109.8,+ "precision": 0.7859+ },+ {+ "z": 3.0,+ "tp": 228,+ "fp": 40,+ "recall": 0.5135,+ "fpr": 0.00301,+ "lr": 170.5,+ "precision": 0.8507+ },+ {+ "z": 3.25,+ "tp": 193,+ "fp": 24,+ "recall": 0.4347,+ "fpr": 0.00181,+ "lr": 240.5,+ "precision": 0.8894+ },+ {+ "z": 3.5,+ "tp": 157,+ "fp": 11,+ "recall": 0.3536,+ "fpr": 0.00083,+ "lr": 426.9,+ "precision": 0.9345+ },+ {+ "z": 3.75,+ "tp": 132,+ "fp": 9,+ "recall": 0.2973,+ "fpr": 0.00068,+ "lr": 438.7,+ "precision": 0.9362+ },+ {+ "z": 4.0,+ "tp": 101,+ "fp": 5,+ "recall": 0.2275,+ "fpr": 0.00038,+ "lr": 604.2,+ "precision": 0.9528+ },+ {+ "z": 4.25,+ "tp": 72,+ "fp": 4,+ "recall": 0.1622,+ "fpr": 0.0003,+ "lr": 538.4,+ "precision": 0.9474+ },+ {+ "z": 4.5,+ "tp": 46,+ "fp": 2,+ "recall": 0.1036,+ "fpr": 0.00015,+ "lr": 687.9,+ "precision": 0.9583+ },+ {+ "z": 4.75,+ "tp": 31,+ "fp": 2,+ "recall": 0.0698,+ "fpr": 0.00015,+ "lr": 463.6,+ "precision": 0.9394+ },+ {+ "z": 5.0,+ "tp": 19,+ "fp": 0,+ "recall": 0.0428,+ "fpr": 0.0,+ "lr": null,+ "precision": 1.0+ },+ {+ "z": 5.25,+ "tp": 9,+ "fp": 0,+ "recall": 0.0203,+ "fpr": 0.0,+ "lr": null,+ "precision": 1.0+ },+ {+ "z": 5.5,+ "tp": 5,+ "fp": 0,+ "recall": 0.0113,+ "fpr": 0.0,+ "lr": null,+ "precision": 1.0+ },+ {+ "z": 5.75,+ "tp": 1,+ "fp": 0,+ "recall": 0.0023,+ "fpr": 0.0,+ "lr": null,+ "precision": 1.0+ },+ {+ "z": 6.0,+ "tp": 0,+ "fp": 0,+ "recall": 0.0,+ "fpr": 0.0,+ "lr": null,+ "precision": 0.0+ }+ ],+ "grid_strict": [+ {+ "z": 0.0,+ "tp": 438,+ "fp": 6812,+ "recall": 0.9865,+ "fpr": 0.45761,+ "lr": 2.2,+ "precision": 0.0604+ },+ {+ "z": 0.25,+ "tp": 431,+ "fp": 5548,+ "recall": 0.9707,+ "fpr": 0.3727,+ "lr": 2.6,+ "precision": 0.0721+ },+ {+ "z": 0.5,+ "tp": 424,+ "fp": 4504,+ "recall": 0.955,+ "fpr": 0.30257,+ "lr": 3.2,+ "precision": 0.086+ },+ {+ "z": 0.75,+ "tp": 419,+ "fp": 3548,+ "recall": 0.9437,+ "fpr": 0.23834,+ "lr": 4.0,+ "precision": 0.1056+ },+ {+ "z": 1.0,+ "tp": 411,+ "fp": 2718,+ "recall": 0.9257,+ "fpr": 0.18259,+ "lr": 5.1,+ "precision": 0.1314+ },+ {+ "z": 1.25,+ "tp": 400,+ "fp": 2056,+ "recall": 0.9009,+ "fpr": 0.13812,+ "lr": 6.5,+ "precision": 0.1629+ },+ {+ "z": 1.5,+ "tp": 388,+ "fp": 1564,+ "recall": 0.8739,+ "fpr": 0.10507,+ "lr": 8.3,+ "precision": 0.1988+ },+ {+ "z": 1.75,+ "tp": 380,+ "fp": 1174,+ "recall": 0.8559,+ "fpr": 0.07887,+ "lr": 10.9,+ "precision": 0.2445+ },+ {+ "z": 2.0,+ "tp": 365,+ "fp": 870,+ "recall": 0.8221,+ "fpr": 0.05844,+ "lr": 14.1,+ "precision": 0.2955+ },+ {+ "z": 2.25,+ "tp": 345,+ "fp": 645,+ "recall": 0.777,+ "fpr": 0.04333,+ "lr": 17.9,+ "precision": 0.3485+ },+ {+ "z": 2.5,+ "tp": 309,+ "fp": 415,+ "recall": 0.6959,+ "fpr": 0.02788,+ "lr": 25.0,+ "precision": 0.4268+ },+ {+ "z": 2.75,+ "tp": 279,+ "fp": 281,+ "recall": 0.6284,+ "fpr": 0.01888,+ "lr": 33.3,+ "precision": 0.4982+ },+ {+ "z": 3.0,+ "tp": 228,+ "fp": 190,+ "recall": 0.5135,+ "fpr": 0.01276,+ "lr": 40.2,+ "precision": 0.5455+ },+ {+ "z": 3.25,+ "tp": 193,+ "fp": 119,+ "recall": 0.4347,+ "fpr": 0.00799,+ "lr": 54.4,+ "precision": 0.6186+ },+ {+ "z": 3.5,+ "tp": 157,+ "fp": 72,+ "recall": 0.3536,+ "fpr": 0.00484,+ "lr": 73.1,+ "precision": 0.6856+ },+ {+ "z": 3.75,+ "tp": 132,+ "fp": 54,+ "recall": 0.2973,+ "fpr": 0.00363,+ "lr": 82.0,+ "precision": 0.7097+ },+ {+ "z": 4.0,+ "tp": 101,+ "fp": 37,+ "recall": 0.2275,+ "fpr": 0.00249,+ "lr": 91.5,+ "precision": 0.7319+ },+ {+ "z": 4.25,+ "tp": 72,+ "fp": 26,+ "recall": 0.1622,+ "fpr": 0.00175,+ "lr": 92.8,+ "precision": 0.7347+ },+ {+ "z": 4.5,+ "tp": 46,+ "fp": 15,+ "recall": 0.1036,+ "fpr": 0.00101,+ "lr": 102.8,+ "precision": 0.7541+ },+ {+ "z": 4.75,+ "tp": 31,+ "fp": 10,+ "recall": 0.0698,+ "fpr": 0.00067,+ "lr": 103.9,+ "precision": 0.7561+ },+ {+ "z": 5.0,+ "tp": 19,+ "fp": 4,+ "recall": 0.0428,+ "fpr": 0.00027,+ "lr": 159.3,+ "precision": 0.8261+ },+ {+ "z": 5.25,+ "tp": 9,+ "fp": 3,+ "recall": 0.0203,+ "fpr": 0.0002,+ "lr": 100.6,+ "precision": 0.75+ },+ {+ "z": 5.5,+ "tp": 5,+ "fp": 1,+ "recall": 0.0113,+ "fpr": 7e-05,+ "lr": 167.6,+ "precision": 0.8333+ },+ {+ "z": 5.75,+ "tp": 1,+ "fp": 1,+ "recall": 0.0023,+ "fpr": 7e-05,+ "lr": 33.5,+ "precision": 0.5+ },+ {+ "z": 6.0,+ "tp": 0,+ "fp": 0,+ "recall": 0.0,+ "fpr": 0.0,+ "lr": null,+ "precision": 0.0+ }+ ],+ "picked": {+ "strong": 3.0,+ "plausible": 2.5,+ "weak": 2.0+ },+ "shipped": {+ "strong": 3.0,+ "plausible": 2.5,+ "weak": 2.0+ },+ "bands_tuning": {+ "lenient": {+ "n_pos": 444,+ "n_neg": 13280,+ "base_rate": 0.0324,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 228,+ "fp": 40,+ "lr": 170.49,+ "precision": 0.851,+ "precision_even_odds": 0.994,+ "recall_at_or_above": 0.514+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 81,+ "fp": 83,+ "lr": 29.19,+ "precision": 0.494,+ "precision_even_odds": 0.967,+ "recall_at_or_above": 0.696+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 56,+ "fp": 236,+ "lr": 7.1,+ "precision": 0.192,+ "precision_even_odds": 0.877,+ "recall_at_or_above": 0.822+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 79,+ "fp": 12921,+ "lr": 0.18,+ "precision": 0.006,+ "precision_even_odds": 0.155,+ "recall_at_or_above": 1.0+ }+ },+ "strict": {+ "n_pos": 444,+ "n_neg": 14886,+ "base_rate": 0.029,+ "strong": {+ "z_range": [+ 3.0,+ null+ ],+ "tp": 228,+ "fp": 190,+ "lr": 40.23,+ "precision": 0.545,+ "precision_even_odds": 0.976,+ "recall_at_or_above": 0.514+ },+ "plausible": {+ "z_range": [+ 2.5,+ 3.0+ ],+ "tp": 81,+ "fp": 225,+ "lr": 12.07,+ "precision": 0.265,+ "precision_even_odds": 0.923,+ "recall_at_or_above": 0.696+ },+ "weak": {+ "z_range": [+ 2.0,+ 2.5+ ],+ "tp": 56,+ "fp": 455,+ "lr": 4.13,+ "precision": 0.11,+ "precision_even_odds": 0.805,+ "recall_at_or_above": 0.822+ },+ "no match": {+ "z_range": [+ null,+ 2.0+ ],+ "tp": 79,+ "fp": 14016,+ "lr": 0.19,+ "precision": 0.006,+ "precision_even_odds": 0.159,+ "recall_at_or_above": 1.0+ }+ }+ }+ },+ "reference_sensitivity": {+ "folder-balanced (used)": {+ "dev": {+ "cat_map": 0.777586606292167,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6779892152094823,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6008064516129032,+ "tag_mrr": 0.7313790616536111,+ "pooled_auc": 0.9731773544405689,+ "pooled_auc_strict": 0.9595833926453143,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7932570388183038,+ "cat_p5": 0.825,+ "cat_p10": 0.7125,+ "class_map": 0.6390066424489291,+ "class_top3": 0.8793103448275862,+ "tag_acc": 0.5526315789473685,+ "tag_mrr": 0.6876431706014001,+ "pooled_auc": 0.9558856317750858,+ "pooled_auc_strict": 0.9393954636620294,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ "uniform over files": {+ "dev": {+ "cat_map": 0.7718379693024295,+ "cat_p5": 0.85,+ "cat_p10": 0.75,+ "class_map": 0.6733486372216337,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6048387096774194,+ "tag_mrr": 0.7347743311927373,+ "pooled_auc": 0.9716329487523477,+ "pooled_auc_strict": 0.9580948991696323,+ "spec_top1": 0.9,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7943198773256326,+ "cat_p5": 0.825,+ "cat_p10": 0.7125,+ "class_map": 0.6435487807470288,+ "class_top3": 0.896551724137931,+ "tag_acc": 0.5421052631578948,+ "tag_mrr": 0.6877697542124378,+ "pooled_auc": 0.9566429903631428,+ "pooled_auc_strict": 0.9400701337523474,+ "spec_top1": 0.7619047619047619,+ "spec_top3": 0.9047619047619048+ }+ },+ "without music-like refs": {+ "dev": {+ "cat_map": 0.7799268851074638,+ "cat_p5": 0.875,+ "cat_p10": 0.7749999999999999,+ "class_map": 0.6769617486987801,+ "class_top3": 0.8870967741935484,+ "tag_acc": 0.6169354838709677,+ "tag_mrr": 0.7386258398859205,+ "pooled_auc": 0.9738985779447277,+ "pooled_auc_strict": 0.9605219454329774,+ "spec_top1": 0.8,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7712286243312736,+ "cat_p5": 0.8,+ "cat_p10": 0.7125,+ "class_map": 0.6346012267444308,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5631578947368421,+ "tag_mrr": 0.6975672928451159,+ "pooled_auc": 0.9592043841860696,+ "pooled_auc_strict": 0.9430243104791834,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9523809523809523+ }+ },+ "transductive (tuning set itself)": {+ "dev": {+ "cat_map": 0.7728621385713701,+ "cat_p5": 0.875,+ "cat_p10": 0.7625,+ "class_map": 0.6770566888086154,+ "class_top3": 0.9193548387096774,+ "tag_acc": 0.6290322580645161,+ "tag_mrr": 0.7471394689539851,+ "pooled_auc": 0.9737123691977462,+ "pooled_auc_strict": 0.9601024911032029,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7940628661543965,+ "cat_p5": 0.8,+ "cat_p10": 0.7125,+ "class_map": 0.6287646366614319,+ "class_top3": 0.8448275862068966,+ "tag_acc": 0.5736842105263158,+ "tag_mrr": 0.7073285970723643,+ "pooled_auc": 0.9612526144797052,+ "pooled_auc_strict": 0.945261500530155,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ },+ "no term bias": {+ "dev": {+ "cat_map": 0.7706841705015869,+ "cat_p5": 0.8999999999999999,+ "cat_p10": 0.7375,+ "class_map": 0.6757453275548259,+ "class_top3": 0.9032258064516129,+ "tag_acc": 0.6209677419354839,+ "tag_mrr": 0.7338462142037805,+ "pooled_auc": 0.9712261872819963,+ "pooled_auc_strict": 0.9565115065243179,+ "spec_top1": 0.85,+ "spec_top3": 1.0+ },+ "test": {+ "cat_map": 0.7699222515537358,+ "cat_p5": 0.8,+ "cat_p10": 0.7125,+ "class_map": 0.6454645266686625,+ "class_top3": 0.8620689655172413,+ "tag_acc": 0.5578947368421052,+ "tag_mrr": 0.6902199626875211,+ "pooled_auc": 0.9532366462462972,+ "pooled_auc_strict": 0.9373035871689726,+ "spec_top1": 0.8095238095238095,+ "spec_top3": 0.9047619047619048+ }+ }+ },+ "term_bias": {+ "highest": [+ [+ "a professionally mixed and mastered music track",+ 0.182+ ],+ [+ "a high quality studio recording",+ 0.172+ ],+ [+ "a glockenspiel",+ 0.124+ ],+ [+ "romantic music",+ 0.114+ ],+ [+ "upbeat corporate pop music",+ 0.113+ ],+ [+ "a dry close sound",+ 0.112+ ],+ [+ "a robotic synthetic sound",+ 0.11+ ],+ [+ "an enter key press",+ 0.107+ ],+ [+ "warm and cozy music",+ 0.106+ ],+ [+ "tech house music",+ 0.104+ ]+ ],+ "lowest": [+ [+ "a man speaking",+ -0.119+ ],+ [+ "people talking in the background",+ -0.12+ ],+ [+ "a man shouting",+ -0.124+ ],+ [+ "a crowd talking",+ -0.125+ ],+ [+ "people laughing",+ -0.13+ ]+ ]+ },+ "seconds": 66.7+}\ No newline at end of filediff --git a/eval/semantics/run_eval.py b/eval/semantics/run_eval.pynew file mode 100644index 0000000..33da419--- /dev/null+++ b/eval/semantics/run_eval.py@@ -0,0 +1,740 @@+"""Evaluate earshot semantics (calibration, pad mode, prompt ensembling, verdicts, find, query kind) on real recordings.++Data (read-only, never copied):+ tuning library - the 219 labelled recordings in <assets>/<category>/ + manifests (split by+ file id into dev / test halves). Variants, the verdict thresholds and find's heuristics were chosen+ on it, and 67 vocabulary terms were added after seeing its classes: numbers on it are optimistic.+ held-out library - the 126 rejected-for-quality real recordings in .../assets/_rejected/<category>/, labelled from+ folder + file name only, BEFORE any score on them was computed (heldout_labels.json). Nothing was+ chosen on it: these are the honest numbers.+ term-bias reference sets (unsupervised: only their mean embedding is used) - for the tuning analysis: the held-out+ library + 7 pro launch films + the Clawd soundtrack / stems + retired synth stems (disjoint from the+ 219 files, but the Clawd mixes contain some of them); shipped (data/vocab_stats.json): the 219+ tuning files + films + Clawd mixes + synth stems (disjoint from the held-out library).+Queries: queries.json (written from the manifest needs before scoring). Run from the repo root (EARSHOT_THREADS=2; data: eval/README.md):++ .venv/Scripts/python.exe eval/semantics/run_eval.py embed --set eval|tuneref|heldout [--pad P] [--part 1/3]+ .venv/Scripts/python.exe eval/semantics/run_eval.py stats # -> earshot/data/vocab_stats.json (shipped refs)+ .venv/Scripts/python.exe eval/semantics/run_eval.py analyze # tuning library -> results.json (variants, thresholds)+ .venv/Scripts/python.exe eval/semantics/run_eval.py heldout # held-out library, shipped config -> heldout.json+ .venv/Scripts/python.exe eval/semantics/run_eval.py kind # find's query-kind reader -> kind.json+ .venv/Scripts/python.exe eval/semantics/run_eval.py find # find() on labelled key presses + 2 built scenes+ .venv/Scripts/python.exe eval/semantics/run_eval.py timing # wall-clock per call (warm models, cold caches)+"""+import argparse+import glob+import hashlib+import json+import os+import sys+import time++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio, match, vocab # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++HERE = os.path.dirname(os.path.abspath(__file__))+OUT = os.path.join(os.path.dirname(os.path.dirname(HERE)), 'out', 'semantics')+CLAWD = EP.FILM+ASSETS = EP.ASSETS+CATS = ['ambience', 'foley', 'garden', 'keyboard', 'magic', 'motion', 'music', 'toy']+FILMS = [EP.ref(rid) for rid in EP.REF_IDS] # the 7 reference launch films+REF_MAX_WIN = 3+TEMPLATES = {'t1': ['{}'], 't3': ['{}', 'the sound of {}', 'a recording of {}'],+ 't5': ['{}', 'the sound of {}', 'a recording of {}', '{} sound effect', 'an audio clip of {}']}+# classes a file can legitimately be described by although its label is another one: 'lenient' scoring leaves these+# pairs out, 'strict' scoring counts them as wrong (an agent asking for a spacebar does not want an Enter key)+CONFUSABLE = [{'snap_lock', 'toy_clicks', 'latch', 'lego', 'switch', 'soft_landing', 'dominoes'},+ {'tiny_steps', 'soft_landing', 'cursor_click', 'key_thock', 'spacebar', 'enter', 'typing'},+ {'wind_grass', 'tree', 'grass_steps', 'meadow', 'tiny_crowd', 'fruit_drop'},+ {'small_whoosh', 'big_whoosh'}, {'fruit_drop', 'soft_landing'}, {'note', 'ting', 'chime_sweep'},+ {'swell', 'chime_sweep', 'impact'}, {'small_pop', 'stretch_pop'}, {'plunge', 'underwater'},+ {'room_tone', 'rain'}]+NEG_MOODS = ['dark and tense music', 'sad and melancholic music', 'mysterious music', 'aggressive music',+ 'suspenseful build-up music']+++def _json(name):+ with open(os.path.join(HERE, name), encoding='utf-8') as fh:+ return json.load(fh)+++def Q():+ return _json('queries.json')+++def _write(name, obj):+ with open(os.path.join(HERE, name), 'w', encoding='utf-8') as fh:+ json.dump(obj, fh, indent=1)+ print('wrote', name)+++def _split(fid):+ return 'dev' if int(hashlib.sha1(fid.encode()).hexdigest(), 16) % 2 == 0 else 'test'+++def eval_set():+ """[{id, path, cat, needs, classes, title, split}] for every media file in the 8 tuning asset folders"""+ q = Q()+ need2cls = {n: c for c, d in q['classes'].items() for n in d['needs']}+ by_id, by_name = {}, {}+ for m in glob.glob(f'{ASSETS}/manifests/*.json'):+ j = json.load(open(m, encoding='utf-8'))+ if isinstance(j, list):+ for e in j:+ if 'need' in e and 'file' in e and e.get('category'):+ by_id.setdefault(e['id'], []).append(e)+ by_name.setdefault(os.path.basename(e['file']), []).append(e)+ out = []+ for cat in CATS:+ for p in audio.list_media(f'{ASSETS}/{cat}', recursive=False):+ es = by_name.get(os.path.basename(p), [])+ fid = es[0]['id'] if es else os.path.basename(p).split('_')[0]+ es = by_id.get(fid, es)+ needs = sorted({e['need'] for e in es})+ cls = sorted({'music'} if cat == 'music' else {need2cls[n] for n in needs if n in need2cls})+ out.append({'id': fid, 'path': p, 'cat': cat, 'needs': needs, 'classes': cls,+ 'title': es[0].get('title') if es else None, 'split': _split(fid)})+ return out+++def heldout_set():+ """the _rejected library with the labels of heldout_labels.json (the lego file in foley/ and toy/ counted once)"""+ lab = _json('heldout_labels.json')+ out, seen = [], set()+ for cat in ['toy'] + [c for c in CATS if c != 'toy']:+ for p in audio.list_media(f'{ASSETS}/_rejected/{cat}', recursive=False):+ fid = os.path.basename(p).split('_')[0]+ if os.path.basename(p) in seen:+ continue+ seen.add(os.path.basename(p))+ cls = ['music'] if cat == 'music' else lab['classes'][fid]+ out.append({'id': fid, 'path': p, 'cat': cat, 'classes': cls, 'split': 'heldout'})+ return sorted(out, key=lambda f: (CATS.index(f['cat']), f['path']))+++def _others():+ p = sorted(glob.glob(f'{CLAWD}/out/audio/stems/*.wav')) + [f'{CLAWD}/audio/_retired_synth/bed.wav',+ f'{CLAWD}/out/audio/melody.wav',+ f'{CLAWD}/out/audio/ambience.wav']+ p += [f'{CLAWD}/out/audio/soundtrack.wav'] + sorted(glob.glob(f'{CLAWD}/out/audio/stems_pro/*.wav'))+ p += FILMS+ return [x for x in p if os.path.exists(x)]+++def tune_refs():+ """term-bias references for the tuning analysis (disjoint from the 219 tuning files)"""+ return audio.list_media(f'{ASSETS}/_rejected') + _others()+++def ship_refs():+ """term-bias references of the shipped stats (disjoint from the held-out library): (paths, max_win per path)"""+ ev = [f['path'] for f in eval_set()]+ ot = _others()+ return ev + ot, [vocab.MAX_WIN] * len(ev) + [REF_MAX_WIN] * len(ot)+++def embed_refs(paths, wins, pad=vocab.PAD):+ A = np.zeros((len(paths), 512), np.float32)+ for w in sorted(set(wins)):+ idx = [i for i, x in enumerate(wins) if x == w]+ A[idx] = vocab.embed_files([paths[i] for i in idx], pad=pad, max_win=w)+ return A+++def embeddings(which, pad):+ if which == 'eval':+ return vocab.embed_files([f['path'] for f in eval_set()], pad=pad)+ if which == 'heldout':+ return vocab.embed_files([f['path'] for f in heldout_set()], pad=pad)+ return vocab.embed_files(tune_refs(), pad=pad, max_win=REF_MAX_WIN)+++def cmd_embed(a):+ paths = {'eval': lambda: [f['path'] for f in eval_set()], 'heldout': lambda: [f['path'] for f in heldout_set()],+ 'tuneref': tune_refs}[a.set]()+ i, n = map(int, a.part.split('/'))+ paths = paths[(i - 1) * len(paths) // n: i * len(paths) // n]+ t = time.time()+ vocab.embed_files(paths, pad=a.pad, max_win=REF_MAX_WIN if a.set == 'tuneref' else vocab.MAX_WIN)+ print(f'{a.set} {a.pad} part {a.part}: {len(paths)} files in {time.time() - t:.1f} s')+++# ---------------------------------------------------------------------------------------------------- metrics+def ap(scores, rel):+ o = np.argsort(-scores)+ r = rel[o]+ if not r.any():+ return None+ hits = np.cumsum(r)+ return float((hits[r] / (np.nonzero(r)[0] + 1)).mean())+++def prec_at(scores, rel, k):+ return float(rel[np.argsort(-scores)[:k]].mean())+++def auc(pos, neg):+ if not len(pos) or not len(neg):+ return None+ allv = np.concatenate([pos, neg])+ r = allv.argsort().argsort() + 1.0+ return float((r[:len(pos)].sum() - len(pos) * (len(pos) + 1) / 2) / (len(pos) * len(neg)))+++def pair_labels(files, cls_list, strict=False):+ """(files x classes) +1 positive, -1 negative, 0 left out (confusable unless strict; files without a class)"""+ L = -np.ones((len(files), len(cls_list)), int)+ for i, f in enumerate(files):+ amb = set()+ for g in CONFUSABLE:+ if g & set(f['classes']):+ amb |= g+ for j, c in enumerate(cls_list):+ if c in f['classes']:+ L[i, j] = 1+ elif not f['classes'] or (c in amb and not strict):+ L[i, j] = 0+ return L+++def specific_rel(spec, files):+ if 'classes' in spec:+ return np.array([bool(set(spec['classes']) & set(f['classes'])) for f in files])+ if 'ids' in spec:+ return np.array([f['id'] in spec['ids'] for f in files])+ return np.array([any(f['id'].startswith(p) for p in spec['ids_prefix']) for f in files])+++def class_pairs(files, Z, cls, strict):+ """pooled (z, label) over both phrasings of every class query; Z = (files, 2 * classes) in phrasing-major order"""+ L = pair_labels(files, cls, strict)+ PL = np.concatenate([L, L], axis=1)+ return Z[PL != 0], PL[PL != 0]+++def evaluate(files, A, V, method, specific=True):+ """retrieval + tagging + pooled separability for one (embeddings, vocab/templates/stats, method)"""+ q = Q()+ cats = list(q['categories'])+ cls = list(q['classes'])+ texts = [q['categories'][c] for c in cats] + [q['classes'][c]['q'][k] for k in (0, 1) for c in cls] + \+ [s['q'] for s in q['specific']]+ r = V.calibrate(A, V.embed_text(texts), method)+ Z = r['z_q']+ nc, ncl = len(cats), len(cls)+ out = {}+ rows = []+ for j, c in enumerate(cats):+ rel = np.array([f['cat'] == c for f in files])+ rows.append({'cat': c, 'ap': ap(Z[:, j], rel), 'p5': prec_at(Z[:, j], rel, 5), 'p10': prec_at(Z[:, j], rel, 10)})+ rows = [x for x in rows if x['ap'] is not None]+ out['category'] = {'map': float(np.mean([x['ap'] for x in rows])), 'p5': float(np.mean([x['p5'] for x in rows])),+ 'p10': float(np.mean([x['p10'] for x in rows])), 'rows': rows}+ aps, top3 = [], []+ for k in (0, 1):+ for j, c in enumerate(cls):+ rel = np.array([c in f['classes'] for f in files])+ if not rel.any():+ continue+ s = Z[:, nc + k * ncl + j]+ aps.append(ap(s, rel))+ top3.append(bool(rel[np.argsort(-s)[:3]].any()))+ out['class'] = {'map': float(np.mean([x for x in aps if x is not None])), 'top3': float(np.mean(top3)),+ 'n_queries': len(aps)}+ if specific:+ spec = []+ for j, s in enumerate(q['specific']):+ rel = specific_rel(s, files)+ if not rel.any():+ continue+ sc = Z[:, nc + 2 * ncl + j]+ o = np.argsort(-sc)+ spec.append({'q': s['q'], 'top1': bool(rel[o[0]]), 'top3': bool(rel[o[:3]].any()), 'ap': ap(sc, rel),+ 'best': os.path.basename(files[o[0]]['path'])[:50]})+ out['specific'] = {'top1': float(np.mean([x['top1'] for x in spec])),+ 'top3': float(np.mean([x['top3'] for x in spec])),+ 'map': float(np.mean([x['ap'] for x in spec])), 'rows': spec}+ U = r['u_q'] # tagging: which class description wins for each file (per-sound z cannot change the order)+ acc, mrr = [], []+ for k in (0, 1):+ S = U[:, nc + k * ncl: nc + (k + 1) * ncl]+ for i, f in enumerate(files):+ if not f['classes']:+ continue+ o = list(np.argsort(-S[i]))+ rk = min(o.index(cls.index(c)) for c in f['classes']) + 1+ acc.append(rk == 1)+ mrr.append(1 / rk)+ out['tagging'] = {'acc': float(np.mean(acc)), 'mrr': float(np.mean(mrr))}+ Zc = Z[:, nc:nc + 2 * ncl]+ for strict in (False, True):+ z, lab = class_pairs(files, Zc, cls, strict)+ out['pooled_auc_strict' if strict else 'pooled_auc'] = auc(z[lab == 1], z[lab == -1])+ out['_Zc'] = Zc+ return out+++def _row(ev):+ r = {'cat_map': ev['category']['map'], 'cat_p5': ev['category']['p5'], 'cat_p10': ev['category']['p10'],+ 'class_map': ev['class']['map'], 'class_top3': ev['class']['top3'], 'tag_acc': ev['tagging']['acc'],+ 'tag_mrr': ev['tagging']['mrr'], 'pooled_auc': ev['pooled_auc'], 'pooled_auc_strict': ev['pooled_auc_strict']}+ if 'specific' in ev:+ r.update(spec_top1=ev['specific']['top1'], spec_top3=ev['specific']['top3'])+ return r+++def _clean(ev):+ return {k: v for k, v in ev.items() if not k.startswith('_')}+++class SdVocab(vocab.Vocab):+ """the rejected variants: (cos - mu) / sd, with mu, sd the exact (folder-weighted) mean / std of each description's+ cosine over the reference recordings, optionally followed by the per-sound z"""+ def __init__(self, templates, stats, R, paths, per_sound):+ super().__init__(templates, stats)+ w = vocab.group_weights(paths)+ self.R, self.w, self.per_sound = R, w / w.sum(), per_sound+ self.sd_E = self._sd(self.E)++ def _sd(self, Q):+ C = self.R @ np.atleast_2d(Q).T+ return np.sqrt(self.w @ (C - self.w @ C) ** 2) + 1e-9++ def calibrate(self, A, Q=None, method=None):+ A = np.atleast_2d(A)+ C = A @ self.E.T+ U = (C - self.mu_E) / self.sd_E+ mu, sd = (U.mean(1, keepdims=True), U.std(1, keepdims=True) + 1e-9) if self.per_sound else (0.0, 1.0)+ r = {'cos_t': C, 'u_t': U, 'z_t': (U - mu) / sd}+ if Q is not None:+ Cq = A @ np.atleast_2d(Q).T+ Uq = (Cq - self.bias(Q)) / self._sd(Q)+ r.update(cos_q=Cq, u_q=Uq, z_q=(Uq - mu) / sd, pct_q=(U[:, None, :] < Uq[:, :, None]).mean(-1))+ return r+++# ---------------------------------------------------------------------------------------------------- verdicts+def grid(z, lab, step=0.25):+ """cumulative (z >= t) recall / false-positive rate / likelihood ratio / precision at the data's base rate"""+ npos, nneg = max(1, int((lab == 1).sum())), max(1, int((lab == -1).sum()))+ out = []+ for t in np.arange(0.0, 6.01, step):+ sel = z >= t+ tp, fp = int(((lab == 1) & sel).sum()), int(((lab == -1) & sel).sum())+ out.append({'z': round(float(t), 2), 'tp': tp, 'fp': fp, 'recall': round(tp / npos, 4),+ 'fpr': round(fp / nneg, 5), 'lr': round((tp / npos) / max(fp / nneg, 1e-9), 1) if fp else None,+ 'precision': round(tp / max(1, tp + fp), 4)})+ return out+++def bands(z, lab, cuts):+ """per verdict band (not cumulative): pairs in it, likelihood ratio LR = P(band | true) / P(band | false),+ precision at the data's base rate and at even prior odds (LR / (LR + 1)), cumulative recall"""+ npos, nneg = max(1, int((lab == 1).sum())), max(1, int((lab == -1).sum()))+ edges = [np.inf] + list(cuts) + [-np.inf]+ names = [n for _, n in match.VERDICTS] + ['no match']+ out = {'n_pos': int((lab == 1).sum()), 'n_neg': int((lab == -1).sum()),+ 'base_rate': round(npos / (npos + nneg), 4)}+ for name, hi, lo in zip(names, edges[:-1], edges[1:]):+ m = (z >= lo) & (z < hi)+ tp, fp = int(((lab == 1) & m).sum()), int(((lab == -1) & m).sum())+ lr = (tp / npos) / (fp / nneg) if fp else None+ out[name] = {'z_range': [None if not np.isfinite(lo) else lo, None if not np.isfinite(hi) else hi],+ 'tp': tp, 'fp': fp, 'lr': None if lr is None else round(lr, 2),+ 'precision': round(tp / max(1, tp + fp), 3),+ 'precision_even_odds': None if lr is None else round(lr / (lr + 1), 3),+ 'recall_at_or_above': round(float(((lab == 1) & (z >= lo)).sum()) / npos, 3)}+ return out+++RULE = ('on the 219 tuning files (lenient labels), 0.5 z grid, top-down: strong = lowest t with LR(z >= t) >= 100; '+ 'plausible = lowest t below it with band LR >= 20; weak = lowest t below that with band LR >= 5')+++def _band_lr(z, lab, lo, hi):+ npos, nneg = max(1, int((lab == 1).sum())), max(1, int((lab == -1).sum()))+ m = (z >= lo) & (z < hi)+ tp, fp = int(((lab == 1) & m).sum()), int(((lab == -1) & m).sum())+ return (tp / npos) / (fp / nneg) if fp else (np.inf if tp else 0.0)+++def pick_thresholds(z, lab):+ """the verdict thresholds by RULE"""+ out, hi = {}, np.inf+ for name, lr_min in (('strong', 100), ('plausible', 20), ('weak', 5)):+ t = None+ for c in np.arange(6.0, -0.01, -0.5):+ if _band_lr(z, lab, c, hi) >= lr_min:+ t = float(c)+ elif t is not None:+ break+ out[name] = t+ hi = t if t is not None else hi+ return out+++def verdict_tables(files, Zc, cls, cuts):+ return {('strict' if s else 'lenient'): bands(*class_pairs(files, Zc, cls, s), cuts) for s in (False, True)}+++def mood_check(V, A, files):+ """music only: share of tracks whose best music_mood tag is a dark / sad / tense one. Every track in both libraries+ was picked for a bright, playful animated film (titles: happy, playful, cheer up, delighted, dreaming big ...),+ so a dark top mood is very likely wrong"""+ moods = V.cats['music_mood']+ idx = [V.names.index(t) for t in moods]+ mus = [i for i, f in enumerate(files) if f['cat'] == 'music']+ Z = V.calibrate(A[mus])['z_t'][:, idx]+ top = [moods[i] for i in np.argmax(Z, axis=1)]+ neg = [t in NEG_MOODS for t in top]+ return {'n_tracks': len(mus), 'top_mood_negative_share': round(float(np.mean(neg)), 3),+ 'chance_share': round(len(NEG_MOODS) / len(moods), 3),+ 'top_moods': {t: top.count(t) for t in sorted(set(top), key=top.count, reverse=True)}}+++def cmd_analyze(a):+ """tuning library: every variant (pad x templates x calibration) on all 219 files and per split, the verdict+ thresholds and the reference-set sensitivity"""+ files = eval_set()+ refs = tune_refs()+ sp = np.array([f['split'] for f in files])+ cls = list(Q()['classes'])+ out = {'n_eval': len(files), 'n_ref': len(refs), 'n_vocab': len(vocab.Vocab().names),+ 'chosen': {'pad': vocab.PAD, 'templates': a.templates, 'method': 'both'}, 'variants': []}+ t0 = time.time()+ for pad in vocab.PADS:+ A, R = embeddings('eval', pad), embeddings('tuneref', pad)+ for tk, tp in TEMPLATES.items():+ st = vocab.build_stats(R, refs, path=None, templates=tp)+ V = vocab.Vocab(tp, stats=st)+ vs = [(m, V, m) for m in vocab.METHODS] + [('term_sd', SdVocab(tp, st, R, refs, False), None),+ ('both_sd', SdVocab(tp, st, R, refs, True), None)]+ for name, VV, method in vs:+ ev = evaluate(files, A, VV, method)+ row = dict(pad=pad, templates=tk, method=name, all=_row(ev),+ **{k: _row(evaluate([f for f in files if f['split'] == k], A[sp == k], VV, method))+ for k in ('dev', 'test')})+ out['variants'].append(row)+ print(pad, tk, name, ' '.join(f'{k} {v:.3f}' for k, v in row['all'].items()), flush=True)+ if (pad, tk, name) == (vocab.PAD, a.templates, 'both'):+ out['chosen'].update(row, detail=_clean(ev))+ Zc = ev['_Zc']+ if pad != vocab.PAD:+ continue+ tp = TEMPLATES[a.templates]+ st = vocab.build_stats(R, refs, path=None, templates=tp)+ g = {s: grid(*class_pairs(files, Zc, cls, s)) for s in (False, True)}+ out['thresholds'] = {'rule': RULE, 'grid_lenient': g[False], 'grid_strict': g[True],+ 'picked': pick_thresholds(*class_pairs(files, Zc, cls, False))}+ out['thresholds']['shipped'] = {n: t for t, n in match.VERDICTS}+ out['thresholds']['bands_tuning'] = verdict_tables(files, Zc, cls, [t for t, _ in match.VERDICTS])+ w_uniform = dict(st, mean=list(R.mean(0)))+ musicish = np.array([p.endswith('.mp4') or '/out/audio' in p.replace(os.sep, '/') or '/music/' in+ p.replace(os.sep, '/') or '_retired_synth' in p for p in refs])+ sens = {'folder-balanced (used)': st, 'uniform over files': w_uniform,+ 'without music-like refs': vocab.build_stats(R[~musicish], [p for p, m in zip(refs, musicish) if not m],+ path=None, templates=tp),+ 'transductive (tuning set itself)': vocab.build_stats(A, [f['path'] for f in files], path=None,+ templates=tp),+ 'no term bias': False}+ out['reference_sensitivity'] = {}+ for name, s_ in sens.items():+ VV = vocab.Vocab(tp, stats=s_)+ out['reference_sensitivity'][name] = {k: _row(evaluate([f for f in files if f['split'] == k], A[sp == k],+ VV, 'both')) for k in ('dev', 'test')}+ mu = V.mu_E+ hub = np.argsort(-mu)+ out['term_bias'] = {'highest': [(V.names[i], round(float(mu[i]), 3)) for i in hub[:10]],+ 'lowest': [(V.names[i], round(float(mu[i]), 3)) for i in hub[-5:]]}+ out['seconds'] = round(time.time() - t0, 1)+ _write('results.json', out)+ print('thresholds picked by the rule:', out['thresholds']['picked'], 'shipped:', out['thresholds']['shipped'])+++def cmd_heldout(a):+ """held-out library with exactly the shipped configuration (vocabulary, templates, stats, thresholds), plus the+ pre-registered 217-term vocabulary for comparison (same templates and stats)"""+ files = heldout_set()+ A = embeddings('heldout', vocab.PAD)+ cls = list(Q()['classes'])+ cuts = [t for t, _ in match.VERDICTS]+ out = {'n_files': len(files), 'n_with_class': sum(bool(f['classes']) for f in files),+ 'per_category': {c: sum(f['cat'] == c for f in files) for c in CATS}, 'thresholds': {n: t for t, n in match.VERDICTS}}+ V = vocab.get()+ pre = vocab.Vocab(stats=None, categories=_json('vocab_preregistered.json')['categories'])+ for name, VV in (('shipped', V), ('preregistered_217', pre)):+ ev = evaluate(files, A, VV, 'both', specific=False)+ out[name] = dict(n_vocab=len(VV.names), metrics=_row(ev), detail=_clean(ev),+ verdicts=verdict_tables(files, ev['_Zc'], cls, cuts))+ print(name, len(VV.names), json.dumps({k: round(v, 3) for k, v in _row(ev).items()}), flush=True)+ raw = evaluate(files, A, V, 'raw', specific=False)+ out['raw_cosine_metrics'] = _row(raw)+ out['mood_check'] = {'heldout': mood_check(V, A, files),+ 'tuning': mood_check(V, embeddings('eval', vocab.PAD), eval_set())}+ print('mood', json.dumps(out['mood_check']))+ _write('heldout.json', out)+++def cmd_stats(a):+ paths, wins = ship_refs()+ meta = {'max_win': 'tuning files 6, others 3',+ 'reference_set': 'the 219 tuning recordings of the Clawd project + 7 professional launch films + Clawd '+ 'soundtrack/stems + retired code-synthesized stems (eval/semantics/run_eval.py ship_refs); '+ 'disjoint from the held-out library'}+ r = vocab.build_stats(embed_refs(paths, wins), paths, meta)+ print(f'wrote {vocab.STATS_PATH}: n_ref={r["n_ref"]} folders={r["n_folders"]}')+++# ---------------------------------------------------------------------------------------------------- query kind+def cmd_kind(a):+ """find's query-kind reader on labelled query texts: 'tune' = the class phrasings of the tuning library (the+ thresholds were set on these), 'test' = generic queries written for this check. The costly error is an event+ read as 'texture' (windows only: no tight trims); 'either' scores both candidate kinds"""+ from earshot import search+ v = vocab.get()+ lab = _json('kind_queries.json')+ out = {'threshold': search.KIND_TEXTURE}+ for part in ('tune', 'test'):+ rows = []+ for kind in ('event', 'texture'):+ for q in lab[part][kind]:+ k, d = search.query_kind(v, q)+ rows.append({'query': q, 'truth': kind, 'read': k, 'score': d})+ cnt = {t: {k: sum(1 for r in rows if r['truth'] == t and r['read'] == k) for k in ('event', 'either', 'texture')}+ for t in ('event', 'texture')}+ out[part] = {'events': cnt['event'], 'textures': cnt['texture'],+ 'events_read_as_texture': [(r['query'], r['score']) for r in rows+ if r['truth'] == 'event' and r['read'] == 'texture'],+ 'textures_read_as_event': [(r['query'], r['score']) for r in rows+ if r['truth'] == 'texture' and r['read'] == 'event'], 'rows': rows}+ print(part, {k: v_ for k, v_ in out[part].items() if k != 'rows'})+ _write('kind.json', out)+++# ---------------------------------------------------------------------------------------------------- find+KEYQ = 'a single mechanical keyboard key press'+SCENE = [('fs-170237', 'magic', 'a glass ting'), ('fs-719637', 'motion', 'a fast whoosh'),+ ('fs-16064', 'foley', 'a cork pop'), ('fs-670000', 'toy', 'a lego brick click'),+ ('fs-272392', 'motion', 'an object dropping into water'), ('mk-2585', 'toy', 'a light switch click'),+ ('fs-789630', 'keyboard', 'a spacebar press'), ('fs-452379', 'magic', 'a small bell')]+LONG_EVENTS = [(20.0, 'fs-170237', 'magic'), (150.0, 'fs-170237', 'magic'), (275.0, 'fs-170237', 'magic'),+ (205.0, 'fs-16064', 'foley'), (240.0, 'fs-719637', 'motion')]+LONG_QUERIES = [('a glass ting', [20.0, 150.0, 275.0]), ('a cork pop', [205.0]), ('a fast whoosh', [240.0])]+++def _manifest(fid):+ for m in glob.glob(f'{ASSETS}/manifests/*.json'):+ j = json.load(open(m, encoding='utf-8'))+ for e in j if isinstance(j, list) else []:+ if e.get('id') == fid and e.get('category'):+ return e+++def _attacks(x, slices):+ """attack inside every labelled slice, measured independently of find's trim rule: the first SAMPLE whose+ magnitude reaches 10% (-20 dB) of the slice's peak sample"""+ out = []+ for s, e in slices:+ seg = np.abs(x[int(s * audio.SR):int(e * audio.SR)])+ out.append(s + int(np.argmax(seg >= 0.1 * seg.max())) / audio.SR)+ return out+++def _score_keys(segs, x, slices, limit=None, skip=None):+ """a hit = the segment starts on a labelled key press as a usable trim (lead = attack - start between 0 and 60 ms:+ the attack is not clipped and there is little dead air) and ends before the next press"""+ att = _attacks(x, slices)+ rows = []+ for g in segs:+ k = int(np.argmin([abs(g['start'] - t) for t in att]))+ lead = att[k] - g['start']+ nxt = att[k + 1] if k + 1 < len(att) else 1e9+ labelled = limit is None or g['start'] < limit+ rows.append({'start': g['start'], 'end': g['end'], 'z': g['z'], 'kind': g['kind'], 'tail_db': g['tail_db'],+ 'press': k if labelled else None, 'lead_ms': round(1000 * lead, 1) if labelled else None,+ 'in_skip_region': bool(skip and skip[0] <= g['start'] < skip[1]),+ 'hit': bool(labelled and 0.0 <= lead <= 0.06 and g['end'] <= nxt + 0.005)})+ hits = [r for r in rows if r['hit']]+ return {'hits': len(hits), 'n': len(rows), 'distinct_presses': len({r['press'] for r in hits}),+ 'in_unlabelled_part': sum(r['press'] is None for r in rows),+ 'false_positives_in_skip_region': sum(r['in_skip_region'] for r in rows),+ 'median_lead_ms': float(np.median([r['lead_ms'] for r in hits])) if hits else None,+ 'lead_ms_range': [min(r['lead_ms'] for r in hits), max(r['lead_ms'] for r in hits)] if hits else None,+ 'rows': rows}+++def _oneshot(fid, cat, peak_db=-6.0):+ p = next(iter(glob.glob(f'{ASSETS}/{cat}/{fid}_*')))+ x = audio.load(p, mono=True)+ env = np.abs(x)+ i0 = max(0, int(np.argmax(env >= env.max() * 0.05)) - int(0.01 * audio.SR))+ x = x[i0: i0 + int(2.5 * audio.SR)].copy()+ x[-int(0.05 * audio.SR):] *= np.linspace(1, 0, int(0.05 * audio.SR))+ return x * 10 ** (peak_db / 20) / (np.abs(x).max() + 1e-12)+++def _bed(seconds):+ bed = audio.load(f'{ASSETS}/ambience/fs-664152_roomtone-basement-hifi.ogg', mono=True)+ bed = np.tile(bed, int(np.ceil(seconds * audio.SR / len(bed))))[: int(seconds * audio.SR)].copy()+ return bed * 10 ** (-55 / 20) / (np.sqrt(np.mean(bed ** 2)) + 1e-12)+++def build_scene(path):+ """real one-shots (first <= 2.5 s, peak -6 dBFS) at known times over a real basement room tone at -55 dBFS RMS"""+ import soundfile as sf+ bed = _bed(44)+ truth = []+ for k, (fid, cat, q) in enumerate(SCENE):+ x = _oneshot(fid, cat)+ t = 2.0 + 5.0 * k+ bed[int(t * audio.SR):int(t * audio.SR) + len(x)] += x+ truth.append({'id': fid, 'query': q, 'start': t, 'end': round(t + len(x) / audio.SR, 3)})+ os.makedirs(os.path.dirname(path), exist_ok=True)+ sf.write(path, bed, audio.SR, subtype='FLOAT')+ return truth+++def build_long(path):+ """300 s: room tone + a real key press every ~1.3 s all the way through (Keychron slices, -12 dBFS) + glass tings+ at 20 / 150 / 275 s, a cork pop at 205 s and a whoosh at 240 s (-6 dBFS): events must be found anywhere"""+ import soundfile as sf+ bed = _bed(300)+ kp = f'{ASSETS}/keyboard/fs-789628_keychron-linear-individual-keys.ogg'+ kx = audio.load(kp, mono=True)+ keys = [kx[int(s * audio.SR):int(e * audio.SR)] for s, e in _manifest('fs-789628')['slices']]+ busy = [(t, t + 2.6) for t, _, _ in LONG_EVENTS]+ t, k = 1.0, 0+ while t < 298:+ if not any(a - 0.7 < t < b for a, b in busy):+ x = keys[k % len(keys)] * 10 ** (-12 / 20) / (np.abs(keys[k % len(keys)]).max() + 1e-12)+ bed[int(t * audio.SR):int(t * audio.SR) + len(x)] += x+ t += 1.3 + 0.1 * (k % 3)+ k += 1+ for t, fid, cat in LONG_EVENTS:+ x = _oneshot(fid, cat)+ bed[int(t * audio.SR):int(t * audio.SR) + len(x)] += x+ os.makedirs(os.path.dirname(path), exist_ok=True)+ sf.write(path, bed, audio.SR, subtype='FLOAT')+++def cmd_find(a):+ from earshot import search+ vocab.PAD = a.pad+ out = {'pad': a.pad}+ k1 = f'{ASSETS}/keyboard/fs-789628_keychron-linear-individual-keys.ogg'+ k2 = f'{ASSETS}/keyboard/fs-709460_lubed-wsbrown-creamy-typing.ogg'+ k3 = f'{ASSETS}/keyboard/fs-609534_single-key-presses-multi.ogg'+ for name, p, top, limit, skip in (('keychron_17_keys', k1, 17, None, (9.3, 10.45)),+ ('creamy_typing_slow_then_fast', k2, 5, 17.3, None),+ ('mars_32_keys', k3, 10, None, None)):+ x = audio.load(p, mono=True)+ sl = _manifest(os.path.basename(p).split('_')[0])['slices']+ t = time.time()+ r = search.find(p, KEYQ, top=top)+ dt = time.time() - t+ t = time.time()+ search.find(p, KEYQ, top=top)+ res = dict(_score_keys(r['segments'], x, sl, limit, skip), seconds=round(dt, 1),+ seconds_cached=round(time.time() - t, 2), duration=r['duration'], n_candidates=r['n_candidates'],+ query=KEYQ)+ if limit:+ r2 = search.find(p, 'fast continuous typing on a keyboard', top=5)+ res['continuous_typing_query'] = {'in_fast_part': sum(g['start'] >= limit for g in r2['segments']), 'n': 5,+ 'segments': [(g['start'], g['end'], g['z'], g['kind']) for g in r2['segments']]}+ out[name] = res+ print(name, json.dumps({k: v for k, v in res.items() if k != 'rows'}), flush=True)+ for name, build, queries in (('scene_44s', build_scene, None), ('long_300s', build_long, LONG_QUERIES)):+ path = os.path.join(OUT, f'find_{name}.wav')+ truth = build(path)+ if queries is None:+ queries = [(tr['query'], [tr['start']]) for tr in truth]+ rows = []+ t = time.time()+ for q, starts in queries:+ r = search.find(path, q, top=max(3, len(starts)))+ segs = r['segments']+ got = [any(g['start'] - 0.05 <= s0 < g['end'] for g in segs) for s0 in starts]+ first = [g['start'] - 0.05 <= starts[0] < g['end'] for g in segs]+ rows.append({'query': q, 'truth_starts': starts, 'found_in_top_n': sum(got), 'n_truth': len(starts),+ 'top1_hit': bool(first and any(g['start'] - 0.05 <= s0 < g['end'] for s0 in starts+ for g in segs[:1])),+ 'query_kind': r['query_kind'], 'n_candidates': r['n_candidates'],+ 'segments': [[g['start'], g['end'], g['z'], g['kind']] for g in segs],+ 'lead_ms': [round(1000 * (s0 + 0.01 - g['start']), 1) for s0 in starts for g in segs+ if g['start'] - 0.05 <= s0 < g['end'] and g['kind'] != 'window']})+ print(name, rows[-1]['query'], rows[-1]['found_in_top_n'], '/', len(starts), rows[-1]['segments'][:3],+ flush=True)+ out[name] = {'found': sum(x['found_in_top_n'] for x in rows), 'n': sum(x['n_truth'] for x in rows),+ 'top1': sum(x['top1_hit'] for x in rows), 'n_queries': len(rows),+ 'tight_trims': sum(len(x['lead_ms']) for x in rows), 'seconds_all_queries': round(time.time() - t, 1),+ 'rows': rows}+ _write(f'find_{a.pad}.json', out)+++def cmd_timing(a):+ """wall-clock per call with warm models on fresh copies (nothing cached) in out/semantics/timing_*/"""+ import shutil+ from earshot import config, describe, search+ d = os.path.join(OUT, f'timing_{int(time.time())}')+ os.makedirs(d)+ cp = lambda p: shutil.copy(p, d) and os.path.join(d, os.path.basename(p)) # noqa: E731+ t = time.time()+ vocab.get()+ describe._ast()+ vocab._clap()+ out = {'threads': config.threads(), 'precision': config.PRECISION, 'model_load_s': round(time.time() - t, 1)}++ def tm(name, f, *args, **kw):+ t0 = time.time()+ f(*args, **kw)+ out[name] = round(time.time() - t0, 2)+ print(name, out[name], flush=True)+ key = cp(f'{ASSETS}/keyboard/fs-789629_keychron-linear-enter.ogg')+ tm('describe_short_cold', describe.describe, key)+ tm('match_cached_new_query', match.match, key, 'a deep thock of a heavy key')+ rain = cp(f'{ASSETS}/ambience/fs-346642_rain-on-windows-interior.ogg')+ tm('describe_31s_cold', describe.describe, rain)+ k17 = cp(f'{ASSETS}/keyboard/fs-789628_keychron-linear-individual-keys.ogg')+ tm('find_18s_cold', search.find, k17, KEYQ)+ tm('find_18s_cached_new_query', search.find, k17, 'a soft click')+ lng = cp(os.path.join(OUT, 'find_long_300s.wav'))+ tm('find_300s_cold', search.find, lng, 'a glass ting')+ rk = os.path.join(d, 'rank')+ os.makedirs(rk)+ src = [p for p in audio.list_media(f'{ASSETS}/foley', recursive=False)][:20]+ for p in src:+ shutil.copy(p, rk)+ tm('rank_20_foley_files_cold', search.rank, rk, 'a cork pop')+ out['rank_s_per_file'] = round(out['rank_20_foley_files_cold'] / len(src), 2)+ out['rank_files_total_s'] = round(sum(audio.duration(p) for p in src), 1)+ tm('rank_20_cached_new_query', search.rank, rk, 'a suction cup pop')+ _write('timing.json', out)+ print(json.dumps(out))+++def main():+ ap_ = argparse.ArgumentParser()+ sub = ap_.add_subparsers(dest='cmd', required=True)+ p = sub.add_parser('embed')+ p.add_argument('--set', choices=['eval', 'tuneref', 'heldout'], required=True)+ p.add_argument('--pad', default='repeatpad', choices=vocab.PADS)+ p.add_argument('--part', default='1/1')+ p.set_defaults(func=cmd_embed)+ p = sub.add_parser('analyze')+ p.add_argument('--templates', default='t5')+ p.set_defaults(func=cmd_analyze)+ for name, fn in (('stats', cmd_stats), ('heldout', cmd_heldout), ('kind', cmd_kind), ('timing', cmd_timing)):+ sub.add_parser(name).set_defaults(func=fn)+ p = sub.add_parser('find')+ p.add_argument('--pad', default=vocab.PAD, choices=vocab.PADS)+ p.set_defaults(func=cmd_find)+ a = ap_.parse_args()+ a.func(a)+++if __name__ == '__main__':+ main()diff --git a/eval/semantics/timing.json b/eval/semantics/timing.jsonnew file mode 100644index 0000000..dbdd85a--- /dev/null+++ b/eval/semantics/timing.json@@ -0,0 +1,15 @@+{+ "threads": 2,+ "precision": "bf16",+ "model_load_s": 27.0,+ "describe_short_cold": 16.95,+ "match_cached_new_query": 1.99,+ "describe_31s_cold": 52.45,+ "find_18s_cold": 29.88,+ "find_18s_cached_new_query": 1.15,+ "find_300s_cold": 61.83,+ "rank_20_foley_files_cold": 55.81,+ "rank_s_per_file": 2.79,+ "rank_files_total_s": 803.7,+ "rank_20_cached_new_query": 3.38+}\ No newline at end of filediff --git a/eval/semantics/vocab_preregistered.json b/eval/semantics/vocab_preregistered.jsonnew file mode 100644index 0000000..9b80317--- /dev/null+++ b/eval/semantics/vocab_preregistered.json@@ -0,0 +1,75 @@+{+ "_doc": "earshot's listening vocabulary: short caption-like phrases CLAP scores sounds against. 'templates' are prompt ensembles (each term is embedded through every template and averaged). Every term in every category also forms the background distribution used to calibrate scores (z = how far a match stands out from what an arbitrary description would score). Extend freely; keep phrases concrete (source + action + quality).",+ "templates": ["{}", "the sound of {}", "a recording of {}"],+ "categories": {+ "transitions": [+ "a fast whoosh", "a soft air whoosh", "a deep cinematic whoosh", "a short swish", "a swoosh passing by",+ "a rising riser building tension", "an uplifter sweep", "a reverse cymbal swell", "a downlifter falling sweep",+ "a tape stop slowing down", "a record scratch", "a quick swipe", "a whip pan transition", "a stutter glitch transition"+ ],+ "impacts": [+ "a deep cinematic boom", "a sub bass drop", "a punchy impact hit", "a big braam", "a heavy thud", "a soft thump",+ "a metallic clang", "a door slam", "a snare hit", "a kick drum", "a hand clap", "a gunshot", "an explosion",+ "a low rumble", "a body falling on the ground"+ ],+ "ui": [+ "a mouse click", "a soft UI button tap", "a crisp interface click", "a notification ding", "a chat message pop",+ "a phone vibration buzz", "a camera shutter click", "a toggle switch click", "a scroll wheel tick",+ "a success chime", "an error buzz", "a digital glitch", "a typing cursor blip", "a keyboard typing fast",+ "a single mechanical keyboard key press", "a deep thocky mechanical keyboard", "a clicky mechanical keyboard",+ "a laptop keyboard typing", "a spacebar press", "a touchscreen tap"+ ],+ "foley": [+ "a plastic click", "a plastic toy snapping together", "lego bricks clicking", "a cork pop", "a bubble pop",+ "a balloon squeak", "a rubber stretch", "a suction cup pop", "a finger tapping on wood", "a finger tapping on glass",+ "glass clinking", "a glass ting", "a wine glass ring", "coins dropping", "paper rustling", "a page turning",+ "a zipper", "cloth rustling", "footsteps on wood", "footsteps on grass", "footsteps on gravel", "high heels walking",+ "a small object dropping on a table", "dominoes falling", "a wooden block hit", "a wood creak", "a metal scrape",+ "water splashing", "a water drop", "a liquid bloop", "pouring water", "ice cubes in a glass", "leaves rustling",+ "a branch snapping", "an apple falling on grass", "digging in soil"+ ],+ "ambience": [+ "rain on a window", "heavy rain", "a thunderstorm", "wind blowing", "a gentle breeze in tall grass",+ "birds singing in the morning", "a forest ambience", "a city street with traffic", "a busy cafe", "an office ambience",+ "a quiet room tone", "a computer fan hum", "an electrical hum", "ocean waves", "a crackling fire", "crickets at night",+ "an airplane flying overhead", "a crowd cheering", "a dog barking", "people talking in the background"+ ],+ "music_genre": [+ "minimal electronic music", "future bass music", "lo-fi hip hop beat", "cinematic orchestral music",+ "epic trailer music", "ambient synth pad music", "upbeat corporate pop music", "synthwave music", "house music",+ "tech house music", "drum and bass music", "trap beat", "glitch hop music", "indie pop music", "solo piano music",+ "acoustic guitar music", "jazz music", "rock music with electric guitars", "hip hop beat with heavy bass",+ "chiptune video game music", "dubstep music", "classical string quartet", "funk music", "techno music"+ ],+ "music_mood": [+ "uplifting and hopeful music", "playful and quirky music", "dark and tense music", "sad and melancholic music",+ "calm and relaxing music", "energetic and driving music", "dreamy and ethereal music", "triumphant and heroic music",+ "mysterious music", "romantic music", "aggressive music", "warm and cozy music", "futuristic music",+ "confident and cool music", "suspenseful build-up music", "celebratory and joyful music"+ ],+ "instruments": [+ "a deep sub bass", "a punchy kick drum pattern", "crisp hi-hats", "a snare roll", "a synthesizer lead",+ "a warm analog synth pad", "a plucked synth arpeggio", "a grand piano", "an electric piano", "strings ensemble",+ "brass section", "a celesta", "a glockenspiel", "bells", "a marimba", "an acoustic guitar", "an electric guitar",+ "a bass guitar", "a choir", "a vocal chop", "orchestral percussion", "a music box"+ ],+ "voice": [+ "a man speaking", "a woman speaking", "a voice-over narration", "someone whispering", "singing vocals",+ "a crowd talking", "children laughing", "a person breathing", "a person coughing"+ ],+ "quality": [+ "a deep sound", "a bright sound", "a thin sound", "a warm sound", "a harsh sound", "a muffled sound",+ "a crisp sound", "a punchy sound", "a soft sound", "a loud sound", "a quiet sound", "a short sound", "a long sound",+ "a dry close sound", "a reverberant sound in a large hall", "a distant sound", "a tight snappy sound",+ "a boomy sound", "an airy sound", "a metallic sound", "a hollow sound", "a satisfying sound"+ ],+ "production": [+ "a real recording of a physical object", "a close-miked foley recording", "a field recording outdoors",+ "a professionally mixed and mastered music track", "a high quality studio recording",+ "a cheap synthesized beep", "a pure sine wave tone", "a square wave chiptune blip", "a computer-generated electronic tone",+ "white noise", "pink noise", "a retro video game sound effect", "an 8-bit sound effect", "a robotic synthetic sound",+ "a low quality recording with hiss", "a distorted clipped recording", "a low bitrate mp3 with artifacts",+ "a recording made on a phone", "silence"+ ]+ }+}diff --git a/eval/timeline/RESULTS.md b/eval/timeline/RESULTS.mdnew file mode 100644index 0000000..2708de4--- /dev/null+++ b/eval/timeline/RESULTS.md@@ -0,0 +1,227 @@+# timeline + sheet: evaluation++All numbers below come from the commands listed with them (results in `eval/timeline/results.json` and+`eval/timeline/label_tune.json`; media and scratch output stay in `out/timeline/`). Machine: Ryzen 7 260 laptop shared+with other builders running models, CPU only, `EARSHOT_THREADS=2`. Second round (after review); the `timing93low` and+`probe` entries of results.json are from the first round.++**Tuning vs evaluation.** Every threshold and design choice was tuned on data that the Clawd soundtrack does not+contain: synthetic mixes of real recordings (`mixes.py`: unused / rejected files of the Clawd asset library, 8 mixes of+30 s, music and ambience beds with real one-shots on top at known times and levels), the library's unused music+(`label_tune.py`, `pulse_tune.py`, `tempo_tune.py`) and, for the tag background only, four launch films the evaluation+does not use (ref_3, ref_5, ref_6, ref_7). The Clawd film (soundtrack, stems, `mix_report.json`+placements, `cues_all.json`) was used only for the evaluation below. The ref_1, ref_2 and ref_4 films were used+only to check that the output reads sensibly. One caveat: `earshot/data/vocab_stats.json` (owned by semantics)+includes the Clawd soundtrack among 148 references; timeline does not use it.++## Event detection (Clawd soundtrack vs every placed sound effect)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts detect`++147 unique placement onsets (attack / start aligned layers within 30 ms merged), a hit = a detection within 30 ms.+SNR = the placement's level over music + ambience at that moment (from the stems).++| | recall | n |+|---|---|---|+| all placements | **93.2%** | 147 |+| loud (SNR >= 0 dB) | 96.4% | 83 |+| quiet (SNR < 0 dB) | 89.1% | 64 |+| SNR >= +6 / 0..+6 / -6..0 / < -6 dB | 96.4% / 96.3% / 97.9% / 64.7% | 56 / 27 / 47 / 17 |+| the same detector on the sfx stem alone | 95.2% | 147 |++- 270 detections: **50.7%** are a placement onset, **81.1%** a placement or a music / ambience onset (found in the+ bed stems), 95.6% if the extra hits inside multi-hit effect samples also count (found by re-running the detector at+ looser settings on the sfx stem); 12 (4.4%) unexplained. (The first build's docstring quoted only the 96%.)+- Timing error vs the mixer's attack times: median 0.0 ms, median |error| 0.5 ms, 90th percentile |error| 11.0 ms.+- Peak-aligned whooshes: 9 of 14 have an onset or swell event within 120 ms of their peak.++Tuning (`.venv/Scripts/python.exe eval/timeline/tune.py`, synthetic mixes, 204 hits, first round): robust z 3.0 and+band contrast 3 dB give recall 80.7% (SNR >= 0: 97.9%), 95.4% of detections are hits or bed onsets.++## Event tags (CLAP, compact event vocabulary)++### Hub phrases (review finding) and the fix++The first build ranked phrases by z within each event. Phrases that score high on any short sound then win+everywhere: "a chat message pop" and "a spacebar press" labelled syllables, music notes and ambience.+`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/tag_bg.py` collects the events timeline would tag (its own+picking, plus every placed one-shot of the mixes) from the 8 tuning mixes and the 4 unused launch films: 536 events.+The per-phrase mean cosine of the fit half (mixes 0-3, ref_3, ref_5; 270 events) is subtracted before ranking,+scaled by TAG_CENTER; scored on the test half (mixes 4-7, ref_6, ref_7; 266 events: 69 placed one-shots,+92 bed events = music / ambience onsets >= 0.15 s from any one-shot, 80 film events):++| scoring | one-shot right kind | bed events called KEY / UI | film events called KEY / UI | most common phrase |+|---|---|---|---|---|+| z within the event (first build) | 42.0% | 60.9% | 27.5% | 26.7% "chat message pop" |+| minus 0.25 x background mean | 34.8% | 46.7% | 16.3% | 17.7% "chat message pop" |+| **minus 0.5 x background mean (shipped)** | 30.4% | **31.5%** | **1.3%** | 11.7% "orchestral stab" |+| minus 0.75 x | 29.0% | 17.4% | 0% | 9.4% "uplifter sweep" |+| minus 1.0 x | 34.8% | 9.8% | 0% | 14.3% "uplifter sweep" (a new hub) |+| (C - mean) / std per phrase | 29.0% | 9.8% | 0% | 15.4% "uplifter sweep" |++Honest reading: in a mix no rule beats a constant guess on the one-shots (always "UI" would be right for 50.7% of+them, always "FOLEY" for 63.8%, because the library's toys and foley fit those kinds); the first build's 42% was+largely the hub phrase being lucky. The shipped rule gives up some of those lucky hits for far fewer false KEY / UI+tags on music, ambience and voice. Scoring each kind by its phrases' mean was worse (20-26%); by the mean of its+two best phrases it gave 34.8% / 30.4% / 3.7% at the shipped damping, within noise of the shipped rule, so it was not+adopted (scratch analysis of `out/timeline/tag_bg_rows.npz`, which tag_bg.py writes). The only signal that predicted a+right tag was exposure: with the shipped scoring over all 536 events (background fitted on half of them),+one-shots at least 12 dB over their bed were right **50%** of the time (n=46) vs **25%** below (n=91); the tag's+margin over the runner-up did not predict correctness (31% vs 34% overall). So the transcript prints the kind plain+only for events >= 12 dB over their bed (EXPOSED_DB) and with '?' otherwise, and the sheet draws those labels pale.++### Clawd (evaluation only)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts tags`++Each placement time is tagged exactly as timeline tags a detected event; "kind" = the kind of the top phrase.+Strict = the right kind (KEY / WHOOSH or RISER / STEP); lenient also accepts UI for keys and FOLEY for steps.+Top-1 strict / lenient / top-3:++| placements | n | mix, shipped | mix, first build | sfx stem, shipped | sfx stem, first build |+|---|---|---|---|---|---|+| keyboard presses | 40 | 57.5% / 92.5% / 72.5% | 60.0% / 97.5% / 80.0% | 62.5% / 65.0% / 82.5% | 67.5% / 72.5% / 92.5% |+| keyboard, no celesta layer | 27 | 81.5% / 96.3% / 92.6% | 85.2% / 96.3% / 96.3% | 81.5% / 81.5% / 96.3% | 81.5% / 81.5% / 96.3% |+| whooshes | 9 | **55.6%** / 55.6% / 77.8% | 11.1% / 11.1% / 66.7% | 77.8% / 77.8% / 88.9% | 77.8% / 77.8% / 88.9% |+| grass footsteps | 21 | 0% / 0% / 0% | 0% / 0% / 0% | 0% / 4.8% / 33.3% | 0% / 0% / 9.5% |++False tags where there is no keyboard and no UI sound (20 tagged events each, timeline's own picking):++| | KEY / UI, shipped | KEY / UI, first build | most common, shipped |+|---|---|---|---|+| Clawd ambience stem (hum, rain, wind in grass, birds) | **35%** | 70% | "spacebar press" 4 |+| Clawd music stem (orchestral, celesta) | **5%** | 15% | "orchestral stab" 15 |++Grass footsteps read as "hi-hat" / "snare drum hit" (on the stem 19 of 21 "hi-hat"). Read tags as hints; the+measurements (time, level, prominence, length, centroid) are the facts.++On the reference films (refs part below, normal detail, 20 tagged each): ref_1 (tech house + voice-over) KEY / UI+30%, 9 VOICE, most common "spoken word" 6 (the review saw "chat message pop" 12 + "spacebar press" 4); ref_2 KEY / UI+60%, "chat message pop" 6 (review: 9); ref_4 5%, "drum hit" 8. ref_2 0-15 s at `--detail high`: "chat message+pop" 37 of 96 tagged (review: 71 of 96); 5 of the 16 AST-labelled events are now VOICE (AST speech >= 0.5), e.g.+`0:01.11 VOICE speech (AST 0.85) / chat message pop`.++Budget: one tag per burst (same band within 0.5 s) before the rest; on the Clawd soundtrack the first typing burst+(2.0-2.9 s) now takes 2 of the 20 tags instead of 10, and the summary's accents are at least 0.5 s apart.++Excerpt treatment (`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/tag_tune.py --vocab compact --variants+delta,repeat`, first round, first-build scoring): repeat-padded excerpt 39.3% / 63.8% top-1 / top-3 (kept), bed+subtracted 34.4% / 55.2%, zero-padded 25.2% / 59.5%, the one-shot alone without any bed (upper bound) 61.3% / 92.0%.++## Sync, beat grid, sections (Clawd soundtrack with its cue list)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts sync`++`timeline(soundtrack.wav, cues=cues_all.json)`: 153 cues at 60 fps.++- Reported: 88.9% of cues have an onset within 1 frame (chance 15.2%), 96.7% within 100 ms (chance 60.8%), median+ |offset| 0.5 ms; 30.1% land on a strong accent within 1 frame (chance 3.1%); 47.7% on the beat grid (chance 20%).+- Offsets against the truth (140 cues with an attack-aligned placement): within 5 ms of the true placement offset for+ **84.3%**, within 1 frame (16.7 ms) for 95.7%, within 40 ms for 98.6% (median error 0.0 ms, p90 7.1 ms).+- **"Without a hit" over all 153 cues** (review finding: the first build listed 5, of which 4 were sounded whooshes /+ risers; its "0 sounded cues reported as unsounded" excluded the 11 peak / end-aligned cues). Now a cut / cue with+ no onset is first checked for a level peak (a soft whoosh / swell) and reported as "level peak only, CHECK"+ instead. Result: **2 reported without a hit** (28.86 s and 50.50 s, both sounded whooshes, so 0 of 2 correct), 3+ reported "level peak only" (39.75 s riser end and 50.99 s whoosh: right; 48.12 s: the mixer lists it unsounded,+ though a lock-ripple layer starts at 48.115 s). The other unsounded cue (40.48 s) gets an unrelated onset within+ 100 ms ("near"). So the no-hit list is still not trustworthy for build-ups: the transcript says so on every+ whoosh / riser / swell cue ("a soft build-up can still be missed: look at the sheet").+- The 11 peak / end-aligned cues: 6.0 swell OK, 11.5 riser OK, 12.0 whoosh OK, 17.75 whoosh near, 27.5 whoosh near,+ 28.86 whoosh off, 39.75 riser peak, 48.0 pop OK, 50.5 whoosh no sound, 50.99 whoosh peak, 53.9 swell near.+- Beat grid: 120.0 BPM, "clear" (pulse clarity 0.48); the score is at 120 BPM: 87.6% of the beats from 0-52 s are+ within 40 ms of the 0.5 s grid and 92.3% of the downbeat guesses fall on even seconds.+- Sections: true music / ambience edits at 10.5, 12.0, 28.0, 40.0, 51.0, 52.2 s; detected boundaries 10.75, 28.65,+ 40.0, 55.2 s: 3 of 6 matched within 1.5 s, 3 of 4 detections correct. Genres are mostly wrong under the busy+ effects ("glitch hop music?", "lo-fi hip hop beat?"); instruments, rain and birds are right.++Level-peak tuning (`.venv/Scripts/python.exe eval/timeline/swell_tune.py --peaks`, synthetic mixes): 10 of 33 whoosh+peaks have no onset / swell within reach; a level peak rising >= 3 dB (local maximum over +-0.3 s, within 150 ms)+is found for 6 of those 10, and for 13 of the 56 unanswered times on a 0.25 s grid at least 0.6 s from any placed+sound (23%). At 4 dB: 3 of 10 vs 7 of 56; a +-0.15 s local maximum found no more whooshes and more empty times+(21 of 56). Shipped: 3 dB, +-0.3 s, labelled as a check, never as a hit.++Section merging (`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/section_tune.py --shipped`, re-run this+round with the new merge of neighbours that read alike): recall 88.9%, precision 84.2% (18 true bed switches, 19+detected), median boundary error 0.04 s. A single 146 s pop song (cheer-up, rejected library music) now reads as 8+sections instead of 11 (the review saw 13 before the label merge existed).++Section labels (`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/label_tune.py`, first round, 52 unused+library tracks + 26 mix beds): centring the section phrases made dark / aggressive moods the top mood for 78% of+music beds with effects on top vs 44% raw, so raw scoring was kept and a mood is named only at z >= 2.5. Music vs+ambience was right for 26 of 26 beds; the ambience keyword was in the top 2 for 16 of 17.++Pulse thresholds (`.venv/Scripts/python.exe eval/timeline/pulse_tune.py`, first round): clarity >= 0.15 for 94.2%+of 52 music excerpts and 0% of 35 ambience excerpts. Tempo octave (`.venv/Scripts/python.exe+eval/timeline/tempo_tune.py`, same 52 music excerpts, librosa's tracker as a second opinion, not ground truth):+timeline agrees with librosa within 4% on 33 of 52 (38 counting octave errors); switching to the double tempo when+it pulses nearly as clearly changed at most 1 agreement (34 at ratio 0.5), so the tempo is not switched; the double+tempo is reported as `tempo_alt` when it pulses at least half as clearly ("~68 or 135 BPM" for the cheer-up song,+which librosa calls 136).++## Film: Clawd_1080p60.mp4 (picture cuts + cues)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts film`++- `earshot.video.cuts` (frame difference, core) finds 26 visual changes; 6 are the blinking terminal cursor in the+ first 1.5 s. Of the 5 act changes (8, 14, 28, 40, 52 s) one has a detected change within 100 ms (8.0 s).+- Cut sync: 46.2% have an onset within 1 frame (chance 15.1%), 80.8% within 100 ms, median |offset| 18 ms; 26.9% on a+ strong accent (chance 3.0%); 65.4% on the beat grid (chance 20%); without a hit: 0.30, 0.80, 1.30 (cursor blinks),+ 30.00 and 51.85 s (51.85 has a whoosh peaking on it, mo.bubble.up at -8.5 dB: still missed; no level peak rises+ 3 dB there). Cue sync on the film's own audio: 86.3% within 1 frame, 4 level peaks only, 1 without a hit (50.50).+- Zoom: `earshot sheet Clawd_1080p60.mp4 --cues cues_all.json --from 44 --to 54` lists the same cues without a hit+ / level peak as the full analysis (1 without a hit, 50.50; cue 53.90 no longer flips to "without a hit"), because+ the zoom is analysed inside +-30 s of context; its accent threshold is ranked over that context and says so.+- Sheets: `out/timeline/eval/clawd_film_sheet.png`, `clawd_zoom_14-24.png`, `out/timeline/r2/film_zoom.png`, opened+ and checked: header lines no longer run off the edge (one sync line each for cuts and cues), labels never overlap,+ masked tags drawn pale with '?', frames at cuts, cue markers coloured by sync (violet = level peak only).++## Reference launch films (read-through check, measure only)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts refs` (detail normal; transcripts and+sheets in `out/timeline/eval/{ref_1,ref_2,ref_4}_{timeline.txt,sheet.png}`)++| film | loudness | events | beat grid | sections | cut sync: onset 1 frame (chance) / accent (chance) |+|---|---|---|---|---|---|+| ref_1, 93.1 s | -12.1 LUFS, TP +0.1 dBTP, LRA 1.7; lead-in 0.6 s, tail 3.5 s | 662 (7.1/s) | none (clarity 0.11) | 2: "tech house music, vocal chop", then "... bass guitar \| voice: voice-over narration" | 28 cuts: 71% (60%) / 25% (11%) |+| ref_2, 28.9 s | -14.8 LUFS, TP +1.0 dBTP, 1 clipped run at 4.94 s | 209 | none (0.11) | 3: house / techno, vocal chop + voice-over | 9 cuts: 67% (46%) / 22% (9%) |+| ref_4, 32.3 s | -14.8 LUFS, TP -0.8 dBTP, LRA 7.0; lead-in 1.1 s | 319 (9.9/s) | 122 BPM, clear (0.61) | 3 | 41 cuts: 80% (71%) / 22% (15%); on beat 24% (20%) |++They read sensibly: loudness, true-peak overs and clipping are flagged; intro / outro quiet is a lead-in / tail,+not a stop-down; voice-over is recognised; the ref_1 sheet at `--width 2400` (`out/timeline/r2/ref_1_2400.png`)+shows every lane title in full. Cut sync in these dense mixes is close to chance, which the chance levels make+visible instead of hiding.++## Edge cases (review findings), each run on a real file++- 10 s of digital silence (`ffmpeg -f lavfi -i anullsrc ...`): "DIGITAL SILENCE: every sample is zero.", one+ "digital silence" section, the gap "= whole file", integrated / true peak None, no models run; the sheet's+ spectrogram is black and the waveform says "digital silence" (`out/timeline/r2/silence.png`).+- 0.05 s noise burst: "click.wav is 0.05 s long: too short for a timeline (>= 0.1 s); for a one-shot use earshot+ measure / describe". 0.3 s tone: true peak -18.1 dBTP ("too short for a loudness measurement") and one event at+ 0:00.00 (a file that starts with sound now has an event at its first millisecond).+- mp3 with cover art (cheer-up): no longer treated as a video (0 frames, no frames lane, no cut method).+- Picture-only mp4: analysed as silence of the same length, its cuts listed and all unsounded ("NO AUDIO STREAM").+ An audio file as `--video`: "--video tone3.wav has no picture stream (an audio file?)".+- Frame cues without fps / with key "frame": "cues have frame numbers but no fps: pass {"fps": 60, "events": ...}";+ "frame", "time", "sec", "seconds" are accepted with fps.+- `--from 5` past the end of a 3 s file: "nothing to analyse between 5 s and the end: tone3.wav is 3.00 s long".+- Non-ASCII file names are backslash-escaped in console text (`t\xf6n.wav`), exact in JSON.+- Result cache: keyed also on the core decode / features / cut code; `--fresh` recomputes and stores (a sheet after+ a fresh timeline reuses it: `cached: True`, internal total 86.0 s of the fresh run).++## Wall-clock time (target: under ~2 minutes at detail normal, EARSHOT_THREADS=2)++`EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts timing60,timing93`+(a fresh CLI process each: imports, model loads, `--fresh` = no cached result and no cached model outputs; decoded+audio and the film's frame differences were cached)++| input | detail | wall clock | internal total | other Python processes | model windows (CLAP events / sections, AST events / sections) |+|---|---|---|---|---|---|+| Clawd soundtrack, 60 s | normal | **81.4 s** | 74.6 s | 2 | 20 / 30, 1 / 3 |+| ref_1 film, 93 s | normal | **93.3 s** | 86.0 s | 4 | 20 / 32, 1 / 3 |+| ref_1 film, 93 s (first round) | low | 76.6 s | | 6 | 12 / 24, 0 / 0 |++In the first round, with 8-12 other Python processes, the same normal-detail runs took 113.0 s and 101.3 s (+ ~9 s+when the frame differences are not cached). Model cost then (12 processes): CLAP load 23.3 s, 0.49 s per window;+AST load 1.8 s, 4.75 s per window. Cached results return immediately; after a code change, cached model outputs make a+re-run take 5-35 s.diff --git a/eval/timeline/label_tune.json b/eval/timeline/label_tune.jsonnew file mode 100644index 0000000..01b2c69--- /dev/null+++ b/eval/timeline/label_tune.json@@ -0,0 +1,80 @@+{+ "n_music_files": 52,+ "n_beds": 26,+ "raw": {+ "dark_or_aggressive_top_mood": 0.25,+ "aggressive_top_mood": 0.038,+ "hub_mood": [+ "confident and cool music",+ 0.231+ ],+ "distinct_moods": 10,+ "hub_genre": [+ "inspirational corporate background music",+ 0.519+ ],+ "distinct_genres": 6,+ "genre_keyword_agree": [+ 0.214,+ 14+ ],+ "instrument_keyword_in_top3": [+ 0.429,+ 7+ ],+ "bed_music_vs_ambience_correct": [+ 1.0,+ 26+ ],+ "ambience_keyword_in_top2": [+ 0.941,+ 17+ ],+ "bed_music_with_hits_dark_top_mood": [+ 0.444,+ 9+ ],+ "bed_music_with_hits_genre_keyword_agree": [+ 0.5,+ 2+ ]+ },+ "centred": {+ "dark_or_aggressive_top_mood": 0.25,+ "aggressive_top_mood": 0.058,+ "hub_mood": [+ "celebratory and joyful music",+ 0.269+ ],+ "distinct_moods": 9,+ "hub_genre": [+ "inspirational corporate background music",+ 0.308+ ],+ "distinct_genres": 14,+ "genre_keyword_agree": [+ 0.429,+ 14+ ],+ "instrument_keyword_in_top3": [+ 0.286,+ 7+ ],+ "bed_music_vs_ambience_correct": [+ 1.0,+ 26+ ],+ "ambience_keyword_in_top2": [+ 0.941,+ 17+ ],+ "bed_music_with_hits_dark_top_mood": [+ 0.778,+ 9+ ],+ "bed_music_with_hits_genre_keyword_agree": [+ 0.0,+ 2+ ]+ }+}\ No newline at end of filediff --git a/eval/timeline/label_tune.py b/eval/timeline/label_tune.pynew file mode 100644index 0000000..5eb885f--- /dev/null+++ b/eval/timeline/label_tune.py@@ -0,0 +1,147 @@+"""Choose how timeline scores section tags (genre / mood / instruments / ambience). Never uses the Clawd soundtrack.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/label_tune.py++Data: (1) music recordings of the Clawd library that the Clawd mix does not use (assets/music minus the film's track,+plus _rejected/music): bright / playful / uplifting tracks picked for a cute launch film; each scored like a timeline+section (mean of three 3 s repeat-padded CLAP windows at 25 / 50 / 75 % of the file). (2) the music and ambience beds+of the synthetic tuning mixes (out/timeline/mixes: real beds with real one-shots on top), section windows inside each+bed averaged. Variants: raw = per-sound z of the raw cosine over all vocab terms (first build); centred = the same+after removing each term's bias over earshot's reference set (cos - m.q, data/vocab_stats.json, as earshot.vocab does).+Metrics: share of tracks whose top mood is a dark / aggressive one (wrong for this library), the most common top mood+and genre (hub terms), genre / instrument keyword agreement with the file names, ambience keyword agreement for beds.+"""+import glob+import json+import os+import sys++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.path.dirname(os.path.dirname(HERE))+sys.path.insert(0, ROOT)+from earshot import audio, config, timeline as tl # noqa: E402+from earshot.models import clap # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++ASSETS = EP.ASSETS+SR = audio.SR+DARK = {'aggressive music', 'dark and tense music', 'suspenseful build-up music', 'mysterious music',+ 'sad and melancholic music'}+GENRE_KW = {'orch': {'cinematic orchestral music', 'uplifting orchestral film score', 'epic trailer music',+ 'classical string quartet'},+ 'piano': {'solo piano music', "playful children's piano music"},+ 'pop': {'upbeat corporate pop music', 'indie pop music', 'inspirational corporate background music'}}+INSTR_KW = {'piano': {'a grand piano', 'an electric piano'}, 'celesta': {'a celesta', 'a glockenspiel', 'bells',+ 'a music box'}, 'marimba': {'a marimba'}, 'ukulele': {'a ukulele'}, 'cello': {'strings ensemble'},+ 'bells': {'bells', 'a glockenspiel', 'a celesta', 'a music box'}}+AMB_KW = {'rain': {'rain on a window', 'heavy rain', 'rain on a roof', 'a thunderstorm'},+ 'wind': {'wind blowing', 'a gentle breeze in tall grass', 'wind in pine trees'},+ 'bird': {'birds singing in the morning', 'skylarks over a meadow', 'a forest ambience',+ 'a quiet countryside morning'},+ 'chorus': {'birds singing in the morning', 'skylarks over a meadow', 'a forest ambience'},+ 'meadow': {'skylarks over a meadow', 'a summer meadow with insects', 'a quiet countryside morning',+ 'a gentle breeze in tall grass', 'birds singing in the morning'},+ 'forest': {'a forest ambience', 'birds singing in the morning', 'wind in pine trees'},+ 'night': {'crickets at night', 'a quiet room at night'}, 'cricket': {'crickets at night'},+ 'room': {'a quiet room tone', 'a quiet room at night', 'an office ambience', 'a computer fan hum'},+ 'hum': {'a computer fan hum', 'an electrical hum'}, 'cafe': {'a busy cafe'},+ 'city': {'a city street with traffic'}, 'traffic': {'a city street with traffic'},+ 'ocean': {'ocean waves'}, 'wave': {'ocean waves'}, 'fire': {'a crackling fire'}}+++def music_files():+ fs = sorted(glob.glob(f'{ASSETS}/music/*.mp3')) + sorted(glob.glob(f'{ASSETS}/_rejected/music/*.mp3'))+ film = {os.path.basename(f) for f in json.load(open(f'{EP.FILM}/out/qa/mix_report.json'))['files_used']}+ return [f for f in fs if os.path.basename(f) not in film]+++def embed_music(c, files):+ out = []+ for f in files:+ d = audio.duration(f)+ A = tl._cached(c, audio.file_id(f), [(d * q, d * q + 3.0) for q in (0.25, 0.5, 0.75)], 'clap-repeat',+ lambda sp: c.embed_audio([audio.load(f, mono=True, start=a, end=b) for a, b in sp]))+ v = A.mean(axis=0)+ out.append(v / np.linalg.norm(v))+ return np.array(out)+++def embed_beds(c):+ out = []+ for jf in sorted(glob.glob(os.path.join(ROOT, 'out/timeline/mixes/mix_*.json'))):+ meta = json.load(open(jf))+ wav = jf[:-5] + '.wav'+ x = audio.load(wav)+ fid = audio.file_id(wav)+ for b in meta['beds']:+ st = np.arange(b['start'] + 0.25, b['end'] - 3.25 + 1e-6, 2.0)+ if not len(st):+ continue+ sp = [(round(s, 3), round(s + 3.0, 3)) for s in st]+ A = tl._cached(c, fid, sp, 'clap-repeat', lambda q: c.embed_audio([tl._seg(x, a, e) for a, e in q]))+ v = A.mean(axis=0)+ out.append((b['kind'], b['file'], v / np.linalg.norm(v)))+ return out+++def scorer(E, terms, cats, mvec, variant):+ bias = E @ mvec if variant == 'centred' else np.zeros(len(terms))++ def top(v, cat, k=1):+ u = E @ v - bias+ z = (u - u.mean()) / (u.std() + 1e-9)+ idx = [i for i in np.argsort(-z) if cats[i] == cat][:k]+ return [terms[i] for i in idx], [float(z[i]) for i in idx]+ return top+++def main():+ c = clap.get()+ terms, cats, E = tl._vocab_matrix(c)+ mvec = np.array(json.load(open(os.path.join(config.DATA_DIR, 'vocab_stats.json')))['mean'], np.float32)+ files = music_files()+ M = embed_music(c, files)+ beds = embed_beds(c)+ names = [os.path.basename(f).lower() for f in files]+ res = {'n_music_files': len(files), 'n_beds': len(beds)}+ for variant in ('raw', 'centred'):+ top = scorer(E, terms, cats, mvec, variant)+ moods = [top(v, 'music_mood')[0][0] for v in M]+ genres = [top(v, 'music_genre')[0][0] for v in M]+ g_ok = [genres[i] in s for i, n in enumerate(names) for kw, s in GENRE_KW.items() if kw in n]+ ins3 = [set(top(v, 'instruments', 3)[0]) for v in M]+ i_ok = [bool(ins3[i] & s) for i, n in enumerate(names) for kw, s in INSTR_KW.items() if kw in n]+ a_ok, mus_ok, bed_dark, bed_g = [], [], [], []+ for kind, f, v in beds:+ if kind == 'music':+ bed_dark.append(top(v, 'music_mood')[0][0] in DARK)+ bed_g += [top(v, 'music_genre')[0][0] in s for kw, s in GENRE_KW.items() if kw in f.lower()]+ gz, az = top(v, 'music_genre')[1][0], top(v, 'ambience')[1][0]+ mus_ok.append((gz >= az) == (kind == 'music'))+ if kind == 'ambience':+ hits = [s for kw, s in AMB_KW.items() if kw in f.lower()]+ if hits:+ a_ok.append(any(set(top(v, 'ambience', 2)[0]) & s for s in hits))+ hub_m = max(set(moods), key=moods.count)+ hub_g = max(set(genres), key=genres.count)+ res[variant] = {+ 'dark_or_aggressive_top_mood': round(float(np.mean([m in DARK for m in moods])), 3),+ 'aggressive_top_mood': round(float(np.mean([m == 'aggressive music' for m in moods])), 3),+ 'hub_mood': [hub_m, round(moods.count(hub_m) / len(moods), 3)], 'distinct_moods': len(set(moods)),+ 'hub_genre': [hub_g, round(genres.count(hub_g) / len(genres), 3)], 'distinct_genres': len(set(genres)),+ 'genre_keyword_agree': [round(float(np.mean(g_ok)), 3), len(g_ok)],+ 'instrument_keyword_in_top3': [round(float(np.mean(i_ok)), 3), len(i_ok)],+ 'bed_music_vs_ambience_correct': [round(float(np.mean(mus_ok)), 3), len(mus_ok)],+ 'ambience_keyword_in_top2': [round(float(np.mean(a_ok)), 3), len(a_ok)],+ 'bed_music_with_hits_dark_top_mood': [round(float(np.mean(bed_dark)), 3), len(bed_dark)],+ 'bed_music_with_hits_genre_keyword_agree': [round(float(np.mean(bed_g)), 3) if bed_g else None, len(bed_g)],+ }+ print(variant, json.dumps(res[variant]))+ json.dump(res, open(os.path.join(HERE, 'label_tune.json'), 'w'), indent=1)+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/mixes.py b/eval/timeline/mixes.pynew file mode 100644index 0000000..264d86a--- /dev/null+++ b/eval/timeline/mixes.py@@ -0,0 +1,159 @@+"""Tuning set for the timeline event detector: synthetic mixtures of REAL recordings with exact ground truth.++ .venv/Scripts/python.exe eval/timeline/mixes.py [--n 8] [--dur 30] [--out out/timeline/mixes]++Only recordings that the Clawd soundtrack does NOT use (unused files in the Clawd asset library + its _rejected+folder) go into these mixes, so detector thresholds tuned here are evaluated on the Clawd soundtrack without leakage.+Each mix: 2-4 beds (music or ambience, hard switches = section boundaries) and one-shot hits every 0.35-1.6 s at a+random level relative to the bed (seeded, deterministic). Ground truth per hit: attack time (first 1 ms within 20 dB+of the hit's peak, the Clawd mixer's own definition), peak-aligned time for whooshes, and the hit-to-bed SNR in dB.+Writes <out>/mix_<k>.wav, mix_<k>_bed.wav (the beds alone) + mix_<k>.json (media stays in out/, never in git).+"""+import argparse+import json+import os+import sys++import numpy as np+import soundfile as sf++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++SR = audio.SR+ASSETS = EP.ASSETS+MIX_REPORT = f'{EP.FILM}/out/qa/mix_report.json'+HITS = ('foley', 'garden', 'keyboard', 'magic', 'toy')+++def unused(cat):+ used = {os.path.basename(f) for f in json.load(open(MIX_REPORT))['files_used']}+ out = []+ for d in (os.path.join(ASSETS, cat), os.path.join(ASSETS, '_rejected', cat)):+ if os.path.isdir(d):+ out += [os.path.join(d, f) for f in sorted(os.listdir(d))+ if os.path.splitext(f)[1].lower() in audio.AUDIO_EXTS and f not in used]+ return out+++def env1(x):+ n = len(x) // 48+ return np.abs(x[: n * 48]).reshape(n, 48).max(axis=1)+++def attack_ms(x):+ e = env1(x)+ return int(np.argmax(e >= e.max() * 0.1)) if len(e) else 0+++def one_shots(path, kmax=3):+ """single isolated hits inside a recording: [(array, attack_s)]"""+ x = audio.load(path, mono=True)[: 60 * SR]+ if len(x) < SR // 10:+ return []+ r = np.sqrt(np.convolve(x.astype(np.float64) ** 2, np.ones(240) / 240, 'same')[::240] + 1e-12)+ db = 20 * np.log10(r)+ out, used = [], np.zeros(len(db), bool)+ for ip in np.argsort(-db):+ if len(out) >= kmax or db[ip] < db.max() - 25:+ break+ if used[max(0, ip - 60): ip + 60].any():+ continue+ used[max(0, ip - 60): ip + 60] = True+ pre, post = db[max(0, ip - 40): max(0, ip - 4)], db[ip + 12: ip + 100]+ if len(pre) < 10 or pre.max() > db[ip] - 20 or (len(post) and post.max() > db[ip] - 12):+ continue+ a = max(0, (ip - 40) * 240)+ seg = x[a: a + int(0.6 * SR)].copy()+ fade = int(0.05 * SR)+ seg[-fade:] *= np.linspace(1, 0, fade)+ out.append((seg, attack_ms(seg) / 1000))+ return out+++def whooshes(path):+ x = audio.load(path, mono=True)[: 4 * SR]+ if len(x) < SR // 5:+ return []+ fade = int(0.03 * SR)+ x = x.copy()+ x[-fade:] *= np.linspace(1, 0, fade)+ e = env1(x)+ return [(x, (int(np.argmax(e)) + 0.5) / 1000)]+++def bed(path, n, rng):+ x = audio.load(path, mono=True)+ if len(x) < n:+ x = np.tile(x, int(np.ceil(n / max(1, len(x)))) + 1)+ a = int(rng.uniform(0, max(1, len(x) - n - 1)))+ seg = x[a: a + n].copy()+ return seg / (np.sqrt(np.mean(seg ** 2)) + 1e-9)+++def build(k, dur, pools, rng):+ music, amb, hits, swoosh = pools+ nb = int(rng.integers(2, 5))+ cuts = np.sort(rng.uniform(4, dur - 4, nb - 1))+ while len(cuts) > 1 and np.min(np.diff(cuts)) < 4:+ cuts = np.sort(rng.uniform(4, dur - 4, nb - 1))+ edges = [0.0] + [round(float(c), 2) for c in cuts] + [float(dur)]+ y = np.zeros(int(dur * SR), np.float32)+ beds = []+ for j in range(nb):+ src = music if (j % 2 == 0) != (rng.uniform() < 0.3) else amb+ f = src[int(rng.integers(len(src)))]+ lvl = rng.uniform(-24, -16) if src is music else rng.uniform(-40, -28)+ a, b = int(edges[j] * SR), int(edges[j + 1] * SR)+ y[a:b] = bed(f, b - a, rng) * 10 ** (lvl / 20)+ beds.append({'start': edges[j], 'end': edges[j + 1], 'kind': 'music' if src is music else 'ambience',+ 'file': os.path.basename(f), 'rms_dbfs': round(lvl, 1)})+ bed_only = y.copy()+ gt, t = [], rng.uniform(0.3, 0.8)+ while t < dur - 0.8:+ is_w = rng.uniform() < 0.12+ pool = swoosh if is_w else hits+ sig, al, cat, name = pool[int(rng.integers(len(pool)))]+ lo = int(max(0, (t - 0.05) * SR))+ bed_db = 20 * np.log10(np.sqrt(np.mean(bed_only[lo: lo + int(0.3 * SR)] ** 2)) + 1e-9)+ snr = rng.uniform(-14, 14)+ pk = 20 * np.log10(np.sqrt(np.max(np.convolve(sig ** 2, np.ones(480) / 480, 'same'))) + 1e-9)+ g = 10 ** ((bed_db + snr - pk) / 20)+ s0 = int(round(t * SR)) - int(round(al * SR))+ a, b = max(0, s0), min(len(y), s0 + len(sig))+ y[a:b] += (sig[a - s0: b - s0] * g).astype(np.float32)+ gt.append({'t': round(float(t), 4), 'align': 'peak' if is_w else 'attack', 'snr_db': round(float(snr), 1),+ 'cat': cat, 'file': name})+ t += rng.uniform(0.35, 1.6) + (1.0 if is_w else 0.0)+ pk = np.abs(y).max()+ if pk > 0.89:+ y *= 0.89 / pk+ bed_only *= 0.89 / pk+ return y, bed_only, {'beds': beds, 'boundaries': edges[1:-1], 'events': gt, 'duration': dur}+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--n', type=int, default=8)+ ap.add_argument('--dur', type=float, default=30.0)+ ap.add_argument('--out', default='out/timeline/mixes')+ a = ap.parse_args()+ os.makedirs(a.out, exist_ok=True)+ rng = np.random.default_rng(20261014)+ music = unused('music')+ amb = unused('ambience')+ hits = [(x, al, c, os.path.basename(f)) for c in HITS for f in unused(c) for x, al in one_shots(f)]+ sw = [(x, al, 'motion', os.path.basename(f)) for f in unused('motion') for x, al in whooshes(f)]+ print(f'pools: {len(music)} music, {len(amb)} ambience, {len(hits)} one-shots, {len(sw)} whooshes')+ for k in range(a.n):+ y, b, meta = build(k, a.dur, (music, amb, hits, sw), rng)+ sf.write(os.path.join(a.out, f'mix_{k}.wav'), y, SR, subtype='FLOAT')+ sf.write(os.path.join(a.out, f'mix_{k}_bed.wav'), b, SR, subtype='FLOAT')+ json.dump(meta, open(os.path.join(a.out, f'mix_{k}.json'), 'w'), indent=1)+ print(f'mix_{k}: {len(meta["events"])} events, boundaries {meta["boundaries"]}')+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/pulse_tune.py b/eval/timeline/pulse_tune.pynew file mode 100644index 0000000..ae56894--- /dev/null+++ b/eval/timeline/pulse_tune.py@@ -0,0 +1,43 @@+"""Calibrate timeline's pulse-clarity thresholds: music beds should show a pulse, ambience beds should not.++ .venv/Scripts/python.exe eval/timeline/pulse_tune.py++Uses 20 s excerpts (from 10 s in) of the unused / rejected music and ambience recordings of the Clawd library (the+same pools as the synthetic mixes, never the Clawd soundtrack).+"""+import json+import os+import sys++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio, timeline as tl # noqa: E402+import mixes # noqa: E402+++def main():+ res = {}+ for cat in ('music', 'ambience'):+ vals = []+ for f in mixes.unused(cat):+ x = audio.load(f, mono=True, start=10.0, end=30.0)+ if len(x) < 8 * audio.SR:+ x = audio.load(f, mono=True, end=20.0)+ g = tl.music_grid(x)+ vals.append((g.get('pulse_clarity') or 0.0, g.get('tempo'), os.path.basename(f)))+ res[cat] = vals+ c = np.array([v[0] for v in vals])+ print(cat, len(c), 'clarity pct10/50/90:', np.round(np.percentile(c, [10, 50, 90]), 3).tolist())+ mu = np.array([v[0] for v in res['music']])+ am = np.array([v[0] for v in res['ambience']])+ for thr in (0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4):+ print(json.dumps({'thr': thr, 'music_with_pulse': round(float(np.mean(mu >= thr)), 3),+ 'ambience_with_pulse': round(float(np.mean(am >= thr)), 3)}))+ print('lowest-clarity music:', sorted(res['music'])[:5])+ print('highest-clarity ambience:', sorted(res['ambience'])[-5:])+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/results.json b/eval/timeline/results.jsonnew file mode 100644index 0000000..f1db825--- /dev/null+++ b/eval/timeline/results.json@@ -0,0 +1,826 @@+{+ "detect": {+ "n_truth_onsets": 147,+ "n_detections": 270,+ "detect_seconds": 3.27,+ "recall_all": 0.932,+ "recall_loud_snr_ge0": 0.964,+ "n_loud": 83,+ "recall_quiet_snr_lt0": 0.891,+ "n_quiet": 64,+ "recall_by_snr": {+ "snr>=+6": [+ 0.964,+ 56+ ],+ "0..+6": [+ 0.963,+ 27+ ],+ "-6..0": [+ 0.979,+ 47+ ],+ "<-6": [+ 0.647,+ 17+ ]+ },+ "recall_sfx_stem_alone": 0.952,+ "precision_vs_placements": 0.507,+ "precision_incl_music_amb_onsets": 0.811,+ "extra_onsets_inside_sfx_samples": 39,+ "precision_incl_all_real_onsets": 0.956,+ "unexplained_detections": 12,+ "timing_error_ms": {+ "median": 0.0,+ "median_abs": 0.5,+ "p90_abs": 11.0+ },+ "peak_aligned_whoosh_swell": {+ "n": 14,+ "found_within_120ms": 9+ },+ "missed_loud_examples": [+ 28.05,+ 35.467+ ],+ "_run_s": 11.9+ },+ "tags": {+ "keyboard": {+ "n": 40,+ "mix_shipped": {+ "top1_strict": 0.575,+ "top1_lenient": 0.925,+ "top3_strict": 0.725,+ "labels": {+ "spacebar press": 21,+ "chat message pop": 11,+ "toggle switch click": 2,+ "glockenspiel note": 2,+ "wooden block hit": 1+ }+ },+ "mix_first_build": {+ "top1_strict": 0.6,+ "top1_lenient": 0.975,+ "top3_strict": 0.8,+ "labels": {+ "spacebar press": 22,+ "chat message pop": 14,+ "laptop key click": 2,+ "wooden block hit": 1,+ "toggle switch click": 1+ }+ },+ "sfx_stem_shipped": {+ "top1_strict": 0.625,+ "top1_lenient": 0.65,+ "top3_strict": 0.825,+ "labels": {+ "mechanical keyboard key press": 19,+ "celesta note": 7,+ "wooden block hit": 5,+ "spacebar press": 3,+ "laptop key click": 3+ }+ },+ "sfx_stem_first_build": {+ "top1_strict": 0.675,+ "top1_lenient": 0.725,+ "top3_strict": 0.925,+ "labels": {+ "mechanical keyboard key press": 12,+ "spacebar press": 11,+ "wooden block hit": 5,+ "laptop key click": 4,+ "celesta note": 4+ }+ }+ },+ "whoosh": {+ "n": 9,+ "mix_shipped": {+ "top1_strict": 0.556,+ "top1_lenient": 0.556,+ "top3_strict": 0.778,+ "labels": {+ "uplifter sweep": 3,+ "water drop": 2,+ "breath": 1,+ "swoosh passing by": 1,+ "whip pan whoosh transition": 1+ }+ },+ "mix_first_build": {+ "top1_strict": 0.111,+ "top1_lenient": 0.111,+ "top3_strict": 0.667,+ "labels": {+ "orchestral stab": 3,+ "water drop": 2,+ "kick drum": 1,+ "breath": 1,+ "chat message pop": 1+ }+ },+ "sfx_stem_shipped": {+ "top1_strict": 0.778,+ "top1_lenient": 0.778,+ "top3_strict": 0.889,+ "labels": {+ "fast air whoosh": 3,+ "whip pan whoosh transition": 2,+ "uplifter sweep": 2,+ "bubble pop": 1,+ "water drop": 1+ }+ },+ "sfx_stem_first_build": {+ "top1_strict": 0.778,+ "top1_lenient": 0.778,+ "top3_strict": 0.889,+ "labels": {+ "whip pan whoosh transition": 3,+ "fast air whoosh": 2,+ "uplifter sweep": 2,+ "bubble pop": 1,+ "water drop": 1+ }+ }+ },+ "grass_steps": {+ "n": 21,+ "mix_shipped": {+ "top1_strict": 0.0,+ "top1_lenient": 0.0,+ "top3_strict": 0.0,+ "labels": {+ "snare drum hit": 6,+ "spacebar press": 5,+ "hi-hat": 3,+ "orchestral stab": 3,+ "rising riser building tension": 3+ }+ },+ "mix_first_build": {+ "top1_strict": 0.0,+ "top1_lenient": 0.0,+ "top3_strict": 0.0,+ "labels": {+ "chat message pop": 7,+ "spacebar press": 7,+ "hi-hat": 3,+ "orchestral stab": 3,+ "snare drum hit": 1+ }+ },+ "sfx_stem_shipped": {+ "top1_strict": 0.0,+ "top1_lenient": 0.048,+ "top3_strict": 0.333,+ "labels": {+ "hi-hat": 19,+ "wooden block hit": 1,+ "short swish": 1+ }+ },+ "sfx_stem_first_build": {+ "top1_strict": 0.0,+ "top1_lenient": 0.0,+ "top3_strict": 0.095,+ "labels": {+ "hi-hat": 19,+ "chat message pop": 2+ }+ }+ },+ "keyboard_without_celesta_layer": {+ "n": 27,+ "mix_shipped": {+ "top1_strict": 0.815,+ "top1_lenient": 0.963,+ "top3_strict": 0.926,+ "labels": {+ "spacebar press": 21,+ "toggle switch click": 2,+ "chat message pop": 2,+ "wooden block hit": 1,+ "mechanical keyboard key press": 1+ }+ },+ "mix_first_build": {+ "top1_strict": 0.852,+ "top1_lenient": 0.963,+ "top3_strict": 0.963,+ "labels": {+ "spacebar press": 22,+ "chat message pop": 2,+ "wooden block hit": 1,+ "toggle switch click": 1,+ "laptop key click": 1+ }+ },+ "sfx_stem_shipped": {+ "top1_strict": 0.815,+ "top1_lenient": 0.815,+ "top3_strict": 0.963,+ "labels": {+ "mechanical keyboard key press": 18,+ "wooden block hit": 5,+ "spacebar press": 3,+ "laptop key click": 1+ }+ },+ "sfx_stem_first_build": {+ "top1_strict": 0.815,+ "top1_lenient": 0.815,+ "top3_strict": 0.963,+ "labels": {+ "spacebar press": 11,+ "mechanical keyboard key press": 10,+ "wooden block hit": 5,+ "laptop key click": 1+ }+ }+ },+ "amb_stem_shipped": {+ "tagged": 20,+ "key_or_ui": 0.35,+ "kinds": {+ "KEY": 6,+ "FOLEY": 6,+ "WHOOSH": 2,+ "VOICE": 2,+ "NOTE": 1,+ "UI": 1,+ "DRUM": 1,+ "HIT": 1+ },+ "labels": {+ "spacebar press": 4,+ "small object dropping on a table": 3,+ "short swish": 2,+ "breath": 2+ }+ },+ "amb_stem_first_build": {+ "tagged": 20,+ "key_or_ui": 0.7,+ "kinds": {+ "KEY": 10,+ "UI": 4,+ "FOLEY": 4,+ "WHOOSH": 1,+ "VOICE": 1+ },+ "labels": {+ "spacebar press": 10,+ "chat message pop": 3,+ "leaves rustling": 2,+ "small object dropping on a table": 2+ }+ },+ "music_stem_shipped": {+ "tagged": 20,+ "key_or_ui": 0.05,+ "kinds": {+ "NOTE": 15,+ "RISER": 2,+ "VOICE": 2,+ "UI": 1+ },+ "labels": {+ "orchestral stab": 15,+ "rising riser building tension": 2,+ "short vocal sound": 2,+ "chat message pop": 1+ }+ },+ "music_stem_first_build": {+ "tagged": 20,+ "key_or_ui": 0.15,+ "kinds": {+ "NOTE": 16,+ "UI": 3,+ "RISER": 1+ },+ "labels": {+ "orchestral stab": 16,+ "chat message pop": 3,+ "rising riser building tension": 1+ }+ },+ "_run_s": 41.6+ },+ "sync": {+ "cues": {+ "n": 153,+ "hit_within_1_frame": 0.889,+ "hit_within_100ms": 0.967,+ "median_abs_offset_ms": 0.5,+ "answered_by_swell": 0,+ "answered_by_level_peak": 3,+ "no_sound": [+ 28.858,+ 50.5+ ],+ "frame_ms": 16.7,+ "chance_within_1_frame": 0.152,+ "chance_within_100ms": 0.608,+ "accent_min_db": 8.2,+ "accent_within_1_frame": 0.301,+ "accent_chance_within_1_frame": 0.031,+ "accent_within_100ms": 0.327,+ "accent_chance_within_100ms": 0.124,+ "on_beat": 0.477,+ "on_beat_chance": 0.2+ },+ "n_cues_scored": 140,+ "peak_or_end_aligned_cues": [+ {+ "t": 6.0,+ "kind": "swell",+ "status": "OK"+ },+ {+ "t": 11.5,+ "kind": "riser",+ "status": "OK"+ },+ {+ "t": 12.0,+ "kind": "whoosh",+ "status": "OK"+ },+ {+ "t": 17.746,+ "kind": "whoosh",+ "status": "near"+ },+ {+ "t": 27.5,+ "kind": "whoosh",+ "status": "near"+ },+ {+ "t": 28.858,+ "kind": "whoosh",+ "status": "off"+ },+ {+ "t": 39.75,+ "kind": "riser",+ "status": "peak"+ },+ {+ "t": 48.0,+ "kind": "pop",+ "status": "OK"+ },+ {+ "t": 50.5,+ "kind": "whoosh",+ "status": "no sound"+ },+ {+ "t": 50.99,+ "kind": "whoosh",+ "status": "peak"+ },+ {+ "t": 53.9,+ "kind": "swell",+ "status": "near"+ }+ ],+ "without_a_hit_vs_truth_all_cues": {+ "reported_without_a_hit": 2,+ "of_them_truly_unsounded": 0,+ "truly_unsounded": 2,+ "sounded_reported_without_a_hit": 2,+ "reported_level_peak_only": 3,+ "level_peak_only_truly_unsounded": 1+ },+ "offset_error_ms": {+ "within_5ms": 0.843,+ "within_1_frame_16.7ms": 0.957,+ "within_40ms": 0.986,+ "median_abs": 0.0,+ "p90_abs": 7.1+ },+ "sounded_cues_reported_without_sound": 0,+ "truly_unsounded_cues": [+ {+ "t": 40.48,+ "id": "swarm:hero:dodge",+ "truth": "unsounded",+ "status": "near"+ },+ {+ "t": 48.12,+ "id": "swarm:hero:cheer:done",+ "truth": "unsounded",+ "status": "peak"+ }+ ],+ "beat_grid": {+ "tempo": 120.0,+ "confidence": "clear",+ "pulse_clarity": 0.483,+ "n_beats_0_52s": 105,+ "beats_within_40ms_of_0.5s_grid": 0.876,+ "downbeats_on_even_seconds_60ms": 0.923+ },+ "sections": {+ "true_boundaries": [+ 10.5,+ 12.0,+ 28.0,+ 40.0,+ 51.0,+ 52.2+ ],+ "detected": [+ 10.75,+ 28.65,+ 40.0,+ 55.2+ ],+ "matched_within_1.5s": 3,+ "labels": [+ "0.0-10.75: glitch hop music?, glockenspiel",+ "10.75-28.65: lo-fi hip hop beat?, glockenspiel | also: raindrop, rain on surface, rain (AST)",+ "28.65-40.0: glitch hop music, orchestral percussion | also: crow, bird, caw (AST)",+ "40.0-55.2: inspirational corporate background music, orchestral percussion, aggressive",+ "55.2-60.0: inspirational corporate background music?, glockenspiel, suspenseful build-up"+ ]+ },+ "summary": "soundtrack.wav: 0:00.00-1:00.00 (60.0 s). Loudness -15.0 LUFS integrated, true peak -1.6 dBTP, LRA 10.4 LU; loudest 0:40.00-0:43.00 (-10.3 LUFS short-term). Quiet lead-in 2.0 s (drop). Beat grid: ~120 BPM (clear pulse, clarity 0.48, 120 beats). 5 section(s): 0:00.00-0:10.75 glitch hop music?, glockenspiel; 0:10.75-0:28.65 lo-fi hip hop beat?, glockenspiel | also: raindrop, rain on surface, rain (AST); 0:28.65-0:40.00 glitch hop music, orchestral percussion | also: crow, bird, caw (AST); 0:40.00-0:55.20 inspirational corporate background music, orchestral percussion, aggressive; 0:55.20-1:00.00 inspirational corporate background music?, glockenspiel, suspenseful build-up. 274 events (4.6/s); 20 tagged (8 key, 5 ui, 4 note, 3 foley; tags are hints). Strongest accents over a running bed: 0:02.06 +28 dB (spacebar press), 0:00.01 +26 dB (wooden block hit), 0:02.88 +22 dB (spacebar press). Sync to 153 cues: 89% have an onset within 1 frame (chance 15%), 97% within 100 ms, median |offset| 0 ms; 30% land on a strong accent (>= 8 dB over the bed) within 1 frame (chance 3%); 48% on the beat grid (chance 20%); 3 only by a level peak (soft whoosh / swell?); 2 without a hit: 0:28.86, 0:50.50.",+ "timing_s": {+ "decode_s": 0.03,+ "loudness_s": 4.76,+ "events_s": 3.71,+ "beats_s": 2.24,+ "cuts_s": 0.0,+ "tag_events_s": 2.52,+ "sections_s": 0.99,+ "total_s": 14.26+ },+ "model_calls": {+ "clap_event_windows": 20,+ "ast_event_windows": 1,+ "clap_section_windows": 30,+ "ast_section_windows": 3+ },+ "wall_s": 14.5,+ "_run_s": 14.5+ },+ "film": {+ "cuts": [+ 0.3,+ 0.5,+ 0.8,+ 1.0,+ 1.3,+ 1.5,+ 8.0,+ 9.0,+ 10.5,+ 10.767,+ 11.5,+ 17.5,+ 18.033,+ 20.0,+ 24.0,+ 28.333,+ 28.983,+ 30.0,+ 33.0,+ 36.0,+ 40.883,+ 42.0,+ 43.0,+ 48.0,+ 51.85,+ 59.0+ ],+ "act_changes": [+ 8.0,+ 14.0,+ 28.0,+ 40.0,+ 52.0+ ],+ "act_changes_found_100ms": 1,+ "cut_sync": {+ "n": 26,+ "hit_within_1_frame": 0.462,+ "hit_within_100ms": 0.808,+ "median_abs_offset_ms": 18.0,+ "answered_by_swell": 0,+ "answered_by_level_peak": 0,+ "no_sound": [+ 0.3,+ 0.8,+ 1.3,+ 30.0,+ 51.85+ ],+ "frame_ms": 16.7,+ "chance_within_1_frame": 0.151,+ "chance_within_100ms": 0.604,+ "accent_min_db": 8.2,+ "accent_within_1_frame": 0.269,+ "accent_chance_within_1_frame": 0.03,+ "accent_within_100ms": 0.346,+ "accent_chance_within_100ms": 0.12,+ "on_beat": 0.654,+ "on_beat_chance": 0.2+ },+ "cue_sync": {+ "n": 153,+ "hit_within_1_frame": 0.863,+ "hit_within_100ms": 0.967,+ "median_abs_offset_ms": 0.5,+ "answered_by_swell": 0,+ "answered_by_level_peak": 4,+ "no_sound": [+ 50.5+ ],+ "frame_ms": 16.7,+ "chance_within_1_frame": 0.152,+ "chance_within_100ms": 0.606,+ "accent_min_db": 8.2,+ "accent_within_1_frame": 0.275,+ "accent_chance_within_1_frame": 0.03,+ "accent_within_100ms": 0.314,+ "accent_chance_within_100ms": 0.12,+ "on_beat": 0.477,+ "on_beat_chance": 0.2+ },+ "summary": "Clawd_1080p60.mp4: 0:00.00-1:00.01 (60.0 s). Loudness -15.0 LUFS integrated, true peak -1.4 dBTP, LRA 10.4 LU; loudest 0:40.00-0:43.00 (-10.4 LUFS short-term). Quiet lead-in 2.0 s (drop). Beat grid: ~120 BPM (clear pulse, clarity 0.48, 121 beats). 5 section(s): 0:00.00-0:10.75 glitch hop music?, glockenspiel; 0:10.75-0:28.65 lo-fi hip hop beat?, glockenspiel | also: raindrop, rain on surface, rain (AST); 0:28.65-0:40.00 glitch hop music, orchestral percussion, futuristic | also: crow, bird, caw (AST); 0:40.00-0:55.20 inspirational corporate background music, orchestral percussion, aggressive; 0:55.20-1:00.01 inspirational corporate background music?, glockenspiel, suspenseful build-up. 272 events (4.5/s); 20 tagged (7 key, 5 note, 4 ui, 2 foley, 1 whoosh, 1 drum; tags are hints). Strongest accents over a running bed: 0:02.06 +28 dB (spacebar press), 0:00.01 +26 dB (wooden block hit), 0:02.88 +22 dB (spacebar press). Sync to 26 cuts: 46% have an onset within 1 frame (chance 15%), 81% within 100 ms, median |offset| 18 ms; 27% land on a strong accent (>= 8 dB over the bed) within 1 frame (chance 3%); 65% on the beat grid (chance 20%); 5 without a hit: 0:00.30, 0:00.80, 0:01.30, 0:30.00, 0:51.85. Sync to 153 cues: 86% have an onset within 1 frame (chance 15%), 97% within 100 ms, median |offset| 0 ms; 28% land on a strong accent (>= 8 dB over the bed) within 1 frame (chance 3%); 48% on the beat grid (chance 20%); 4 only by a level peak (soft whoosh / swell?); 1 without a hit: 0:50.50.",+ "timing_s": {+ "decode_s": 0.04,+ "loudness_s": 3.34,+ "events_s": 2.5,+ "beats_s": 1.26,+ "cuts_s": 0.17,+ "tag_events_s": 25.37,+ "sections_s": 1.67,+ "total_s": 34.37+ },+ "wall_s": 34.6,+ "sheet": "Wrote out/timeline/eval/clawd_film_sheet.png (2000x1331): 0:00.00-1:00.01, 7 frames, 16 of 20 tagged events labelled, 272 event ticks, 26 cut lines. Open it with an image viewer / Read tool.",+ "zoom_sheet": "Wrote out/timeline/eval/clawd_zoom_14-24.png (2000x1331): 0:14.00-0:24.00, 7 frames, 18 of 20 tagged events labelled, 54 event ticks, 3 cut lines. Open it with an image viewer / Read tool.",+ "_run_s": 62.8+ },+ "refs": {+ "ref_1": {+ "duration": 93.12,+ "summary": "ref_1.mp4: 0:00.00-1:33.12 (93.1 s). Loudness -12.1 LUFS integrated, true peak 0.1 dBTP, LRA 1.7 LU; loudest 0:56.50-0:59.50 (-11.0 LUFS short-term). True peak 0.1 dBTP is above -1 dBTP (risk of overs after encoding). Quiet lead-in 0.6 s (drop) and tail 3.5 s (drop). No steady beat grid (pulse clarity 0.11 < 0.15). 2 section(s): 0:00.00-0:05.70 tech house music, vocal chop, confident and cool; 0:05.70-1:33.12 tech house music, bass guitar, confident and cool | voice: voice-over narration. 662 events (7.1/s); 20 tagged (9 voice, 3 hit, 3 ui, 3 key, 1 whoosh, 1 note; tags are hints). Strongest accents over a running bed: 1:21.32 +35 dB, 0:09.78 +31 dB, 0:26.28 +30 dB. Sync to 28 cuts: 71% have an onset within 1 frame (chance 60%), 96% within 100 ms, median |offset| 26 ms; 25% land on a strong accent (>= 9 dB over the bed) within 1 frame (chance 11%); 1 without a hit: 0:39.87.",+ "n_events": 662,+ "tagged": 20,+ "key_or_ui": 0.3,+ "most_common_tag": [+ "spoken word",+ 6+ ],+ "kinds": {+ "VOICE": 9,+ "HIT": 3,+ "UI": 3,+ "KEY": 3,+ "WHOOSH": 1,+ "NOTE": 1+ },+ "cuts": 28,+ "sections": [+ "tech house music, vocal chop, confident and cool",+ "tech house music, bass guitar, confident and cool | voice: voice-over narration"+ ],+ "cut_sync": {+ "n": 28,+ "hit_within_1_frame": 0.714,+ "hit_within_100ms": 0.964,+ "median_abs_offset_ms": 26.0,+ "answered_by_swell": 0,+ "answered_by_level_peak": 0,+ "no_sound": [+ 39.873+ ],+ "frame_ms": 41.7,+ "chance_within_1_frame": 0.597,+ "chance_within_100ms": 0.934,+ "accent_min_db": 9.4,+ "accent_within_1_frame": 0.25,+ "accent_chance_within_1_frame": 0.114,+ "accent_within_100ms": 0.321,+ "accent_chance_within_100ms": 0.233+ },+ "timing_s": {+ "decode_s": 0.11,+ "loudness_s": 5.32,+ "events_s": 4.08,+ "beats_s": 2.27,+ "cuts_s": 0.15,+ "tag_events_s": 24.14,+ "sections_s": 3.06,+ "total_s": 39.14+ },+ "wall_s": 39.4,+ "sheet": "Wrote out/timeline/eval/ref_1_sheet.png (2000x1300): 0:00.00-1:33.12, 7 frames, 17 of 20 tagged events labelled, 662 event ticks, 28 cut lines. Open it with an image viewer / Read tool."+ },+ "ref_2": {+ "duration": 28.909,+ "summary": "ref_2.mp4: 0:00.00-0:28.91 (28.9 s). Loudness -14.8 LUFS integrated, true peak 1.0 dBTP, LRA 3.6 LU; loudest 0:20.50-0:23.50 (-12.4 LUFS short-term). CLIPPING: 1 runs of clipped samples (first at 0:04.94). No steady beat grid (pulse clarity 0.11 < 0.15). 3 section(s): 0:00.00-0:05.75 tech house music?, vocal chop | also: plop (AST) | voice: voice-over narration; 0:05.75-0:17.95 house music, vocal chop, confident and cool | also: new-age music, drip, ambient music (AST) | voice: voice-over narration; 0:17.95-0:28.91 techno music, vocal chop, confident and cool | also: spray (AST) | voice: voice-over narration. 209 events (7.2/s); 20 tagged (12 ui, 3 foley, 3 hit, 1 note, 1 drum; tags are hints). Strongest accents over a running bed: 0:00.25 +34 dB, 0:14.55 +28 dB (sub bass drop), 0:27.45 +20 dB (water drop). Sync to 9 cuts: 67% have an onset within 1 frame (chance 46%), 89% within 100 ms, median |offset| 26 ms; 22% land on a strong accent (>= 10 dB over the bed) within 1 frame (chance 9%); 1 met by a whoosh / riser peak.",+ "n_events": 209,+ "tagged": 20,+ "key_or_ui": 0.6,+ "most_common_tag": [+ "chat message pop",+ 6+ ],+ "kinds": {+ "UI": 12,+ "FOLEY": 3,+ "HIT": 3,+ "NOTE": 1,+ "DRUM": 1+ },+ "cuts": 9,+ "sections": [+ "tech house music?, vocal chop | also: plop (AST) | voice: voice-over narration",+ "house music, vocal chop, confident and cool | also: new-age music, drip, ambient music (AST) | voice: voice-over narration",+ "techno music, vocal chop, confident and cool | also: spray (AST) | voice: voice-over narration"+ ],+ "cut_sync": {+ "n": 9,+ "hit_within_1_frame": 0.667,+ "hit_within_100ms": 0.889,+ "median_abs_offset_ms": 26.0,+ "answered_by_swell": 1,+ "answered_by_level_peak": 0,+ "no_sound": [],+ "frame_ms": 33.3,+ "chance_within_1_frame": 0.461,+ "chance_within_100ms": 0.888,+ "accent_min_db": 10.3,+ "accent_within_1_frame": 0.222,+ "accent_chance_within_1_frame": 0.089,+ "accent_within_100ms": 0.333,+ "accent_chance_within_100ms": 0.191+ },+ "timing_s": {+ "decode_s": 0.06,+ "loudness_s": 1.56,+ "events_s": 1.33,+ "beats_s": 0.72,+ "cuts_s": 0.16,+ "tag_events_s": 0.78,+ "sections_s": 0.65,+ "total_s": 5.26+ },+ "wall_s": 5.4,+ "sheet": "Wrote out/timeline/eval/ref_2_sheet.png (2000x1300): 0:00.00-0:28.91, 7 frames, 17 of 20 tagged events labelled, 209 event ticks, 9 cut lines. Open it with an image viewer / Read tool."+ },+ "ref_4": {+ "duration": 32.256,+ "summary": "ref_4.mp4: 0:00.00-0:32.26 (32.3 s). Loudness -14.8 LUFS integrated, true peak -0.8 dBTP, LRA 7.0 LU; loudest 0:23.50-0:26.50 (-12.8 LUFS short-term). True peak -0.8 dBTP is above -1 dBTP (risk of overs after encoding). 1 stop-down(s)/gap(s): 0:31.85 0.3 s drop. Quiet lead-in 1.1 s (drop). Beat grid: ~122 BPM (clear pulse, clarity 0.61, 66 beats). 3 section(s): 0:00.00-0:09.75 glitch hop music, orchestral percussion | also: electronica (AST); 0:09.75-0:29.55 inspirational corporate background music, orchestral percussion | also: liquid, drip (AST); 0:29.55-0:32.26 tech house music, pizzicato strings, confident and cool. 319 events (9.9/s); 20 tagged (13 drum, 6 foley, 1 ui; tags are hints). Strongest accents over a running bed: 0:00.35 +30 dB, 0:32.21 +18 dB, 0:31.23 +17 dB. Sync to 41 cuts: 80% have an onset within 1 frame (chance 71%), 100% within 100 ms, median |offset| 28 ms; 22% land on a strong accent (>= 10 dB over the bed) within 1 frame (chance 15%); 24% on the beat grid (chance 20%).",+ "n_events": 319,+ "tagged": 20,+ "key_or_ui": 0.05,+ "most_common_tag": [+ "drum hit",+ 8+ ],+ "kinds": {+ "DRUM": 13,+ "FOLEY": 6,+ "UI": 1+ },+ "cuts": 41,+ "sections": [+ "glitch hop music, orchestral percussion | also: electronica (AST)",+ "inspirational corporate background music, orchestral percussion | also: liquid, drip (AST)",+ "tech house music, pizzicato strings, confident and cool"+ ],+ "cut_sync": {+ "n": 41,+ "hit_within_1_frame": 0.805,+ "hit_within_100ms": 1.0,+ "median_abs_offset_ms": 28.0,+ "answered_by_swell": 0,+ "answered_by_level_peak": 0,+ "no_sound": [],+ "frame_ms": 40.0,+ "chance_within_1_frame": 0.706,+ "chance_within_100ms": 0.975,+ "accent_min_db": 9.8,+ "accent_within_1_frame": 0.22,+ "accent_chance_within_1_frame": 0.148,+ "accent_within_100ms": 0.341,+ "accent_chance_within_100ms": 0.313,+ "on_beat": 0.244,+ "on_beat_chance": 0.204+ },+ "timing_s": {+ "decode_s": 0.04,+ "loudness_s": 1.91,+ "events_s": 1.73,+ "beats_s": 0.95,+ "cuts_s": 0.18,+ "tag_events_s": 2.27,+ "sections_s": 1.87,+ "total_s": 8.95+ },+ "wall_s": 9.2,+ "sheet": "Wrote out/timeline/eval/ref_4_sheet.png (2000x1300): 0:00.00-0:32.26, 7 frames, 18 of 20 tagged events labelled, 319 event ticks, 41 cut lines. Open it with an image viewer / Read tool."+ },+ "_run_s": 72.1+ },+ "timing60": {+ "soundtrack_60s": {+ "wall_s_cli_incl_imports_and_model_load": 81.4,+ "internal_timing_s": {+ "decode_s": 0.02,+ "loudness_s": 3.22,+ "events_s": 2.55,+ "beats_s": 1.59,+ "cuts_s": 0.0,+ "tag_events_s": 37.08,+ "sections_s": 30.12,+ "total_s": 74.59+ },+ "model_calls": {+ "clap_event_windows": 20,+ "ast_event_windows": 1,+ "clap_section_windows": 30,+ "ast_section_windows": 3+ },+ "threads": "2",+ "duration_s": 60.0,+ "other_python_processes_at_start": 2+ },+ "_run_s": 82.3+ },+ "timing93": {+ "ref_1_film_93s": {+ "wall_s_cli_incl_imports_and_model_load": 93.3,+ "internal_timing_s": {+ "decode_s": 0.02,+ "loudness_s": 5.64,+ "events_s": 4.58,+ "beats_s": 2.25,+ "cuts_s": 0.12,+ "tag_events_s": 37.38,+ "sections_s": 35.97,+ "total_s": 85.98+ },+ "model_calls": {+ "clap_event_windows": 20,+ "ast_event_windows": 1,+ "clap_section_windows": 32,+ "ast_section_windows": 3+ },+ "threads": "2",+ "duration_s": 93.12,+ "other_python_processes_at_start": 4+ },+ "_run_s": 94.2+ },+ "probe": {+ "python_processes_running": 12,+ "clap_load_s": 23.3,+ "clap_s_per_window": 0.49,+ "ast_load_s": 1.8,+ "ast_s_per_window": 4.75,+ "_run_s": 39.9+ },+ "timing93low": {+ "ref_1_film_93s_detail_low": {+ "wall_s_cli_incl_imports_and_model_load": 76.6,+ "internal_timing_s": {+ "decode_s": 0.14,+ "loudness_s": 7.3,+ "events_s": 5.57,+ "beats_s": 4.08,+ "cuts_s": 0.12,+ "tag_events_s": 36.84,+ "sections_s": 13.68,+ "total_s": 67.76+ },+ "model_calls": {+ "clap_event_windows": 12,+ "ast_event_windows": 0,+ "clap_section_windows": 24,+ "ast_section_windows": 0+ },+ "threads": "2",+ "duration_s": 93.12,+ "other_python_processes_at_start": 6+ },+ "_run_s": 77.2+ }+}\ No newline at end of filediff --git a/eval/timeline/run_eval.py b/eval/timeline/run_eval.pynew file mode 100644index 0000000..b14d0d1--- /dev/null+++ b/eval/timeline/run_eval.py@@ -0,0 +1,358 @@+"""Evaluate earshot.timeline / earshot.sheet against the Clawd film's ground truth (never used for tuning) and on the+user's reference launch films.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts detect,tags,sync,film,refs,timing60,timing93++Parts (each fits in one ~10 minute job; results merge into eval/timeline/results.json):+ detect event detection on soundtrack.wav vs every sfx placement of out/qa/mix_report.json (30 ms), split by how+ loud the placement is against the music + ambience stems at that moment (SNR); also on the sfx stem alone+ tags tag plausibility at the true times of keyboard / whoosh / grass-step placements, in the mix and in the stem;+ false KEY / UI tags on the ambience and music stems (no keyboard, no UI sounds in them); each for the+ shipped scoring (hub phrases damped, TAG_CENTER 0.5) and the first build's (TAG_CENTER 0)+ sync timeline(soundtrack, cues=cues_all.json): measured cue->sound offsets vs the true placement offsets; the+ 'without a hit' list vs the mixer's unsounded cues (all 153 cues); beat grid vs the 120 BPM score grid;+ section boundaries vs the music / ambience edits+ film timeline + sheet on Clawd_1080p60.mp4 (picture cuts vs act changes)+ refs timeline + sheet on 3 reference launch films+ timing60 / timing93 wall-clock of the CLI (fresh: no cached results or model outputs) for the 60 s soundtrack /+ the 93 s ref_1 film+Tuning data (out/timeline/mixes, made by mixes.py from recordings the Clawd mix does not use) is separate from this.+"""+import argparse+import json+import os+import subprocess+import sys+import time++from collections import Counter++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.path.dirname(os.path.dirname(HERE))+sys.path.insert(0, ROOT)+sys.path.insert(0, HERE)+from earshot import audio, timeline as tl # noqa: E402+from tune import match # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++CLAWD = f'{EP.FILM}/out'+SOUNDTRACK = f'{CLAWD}/audio/soundtrack.wav'+STEMS = {k: f'{CLAWD}/audio/stems_pro/{k}.wav' for k in ('sfx', 'amb', 'music')}+REPORT = f'{CLAWD}/qa/mix_report.json'+CUES = f'{CLAWD}/cues_all.json'+FILM = f'{CLAWD}/Clawd_1080p60.mp4'+REFS = {rid: EP.ref(rid) for rid in ('ref_1', 'ref_2', 'ref_4')} # 3 of the 7 reference launch films+OUT = os.path.join(ROOT, 'out', 'timeline', 'eval')+RESULTS = os.path.join(HERE, 'results.json')+SR = audio.SR+++def placements():+ return json.load(open(REPORT))['placements']+++def onsets_truth(P, aligns=('attack', 'start')):+ """unique onset times (placement layers within 30 ms merged) for placements with the given alignment"""+ ts = sorted({round(p['t'], 4) for p in P if p.get('align', 'attack') in aligns})+ out = []+ for t in ts:+ if out and t - out[-1][-1] < 0.03:+ out[-1].append(t)+ else:+ out.append([t])+ return [g[0] for g in out]+++def snr_db(t, sfx, bed):+ """placement level vs the bed at that moment: max 10 ms RMS of the sfx stem in [t, t+60 ms] over the RMS of+ music + ambience in [t-30, t+60 ms] (dB)"""+ a, b = int(t * SR), int((t + 0.06) * SR)+ s = sfx[a:b]+ if len(s) < 480:+ return None+ r = np.sqrt(np.convolve(s.astype(np.float64) ** 2, np.ones(480) / 480, 'valid').max())+ q = bed[max(0, a - int(0.03 * SR)): b]+ return float(20 * np.log10(r + 1e-9) - 20 * np.log10(np.sqrt(np.mean(q.astype(np.float64) ** 2)) + 1e-9))+++def part_detect():+ P = placements()+ x = audio.load(SOUNDTRACK)+ sfx = audio.load(STEMS['sfx'], mono=True)+ bed = audio.load(STEMS['music'], mono=True) + audio.load(STEMS['amb'], mono=True)+ gt = onsets_truth(P)+ snr = np.array([snr_db(t, sfx, bed) for t in gt], float)+ t = time.time()+ ev = tl.detect_events(x)+ secs = time.time() - t+ det = [e['t'] for e in ev if e['shape'] == 'hit']+ m = match(gt, det, 0.03)+ hit = np.zeros(len(gt), bool)+ hit[[i for i, _, _ in m]] = True+ dts = np.array([dt for _, _, dt in m]) * 1000+ used = {j for _, j, _ in m}+ rest = [d for j, d in enumerate(det) if j not in used]+ bed_on = [e['t'] for e in tl.detect_events(bed, zthr=2.0, min_contrast_db=1.0)]+ mb = match(bed_on, rest, 0.03)+ expl = len(mb)+ rest2 = [d for j, d in enumerate(rest) if j not in {j for _, j, _ in mb}]+ sfx_loose = [e['t'] for e in tl.detect_events(sfx, zthr=2.0, min_contrast_db=1.0)]+ inner = len(match(sfx_loose, rest2, 0.03))+ bins = [('snr>=+6', snr >= 6), ('0..+6', (snr >= 0) & (snr < 6)), ('-6..0', (snr >= -6) & (snr < 0)),+ ('<-6', snr < -6)]+ sfx_ev = [e['t'] for e in tl.detect_events(audio.load(STEMS['sfx'])) if e['shape'] == 'hit']+ ms = match(gt, sfx_ev, 0.03)+ hs = np.zeros(len(gt), bool)+ hs[[i for i, _, _ in ms]] = True+ pk = onsets_truth(P, ('peak',))+ sw = [e['t'] for e in ev]+ mp = match(pk, sw, 0.12)+ res = {+ 'n_truth_onsets': len(gt), 'n_detections': len(det), 'detect_seconds': round(secs, 2),+ 'recall_all': round(float(hit.mean()), 3),+ 'recall_loud_snr_ge0': round(float(hit[snr >= 0].mean()), 3), 'n_loud': int((snr >= 0).sum()),+ 'recall_quiet_snr_lt0': round(float(hit[snr < 0].mean()), 3), 'n_quiet': int((snr < 0).sum()),+ 'recall_by_snr': {k: [round(float(hit[v].mean()), 3) if v.any() else None, int(v.sum())] for k, v in bins},+ 'recall_sfx_stem_alone': round(float(hs.mean()), 3),+ 'precision_vs_placements': round(len(m) / max(1, len(det)), 3),+ 'precision_incl_music_amb_onsets': round((len(m) + expl) / max(1, len(det)), 3),+ 'extra_onsets_inside_sfx_samples': inner,+ 'precision_incl_all_real_onsets': round((len(m) + expl + inner) / max(1, len(det)), 3),+ 'unexplained_detections': len(det) - len(m) - expl - inner,+ 'timing_error_ms': {'median': round(float(np.median(dts)), 1), 'median_abs': round(float(np.median(np.abs(dts))), 1),+ 'p90_abs': round(float(np.percentile(np.abs(dts), 90)), 1)},+ 'peak_aligned_whoosh_swell': {'n': len(pk), 'found_within_120ms': len(mp)},+ 'missed_loud_examples': [round(gt[i], 3) for i in np.where(~hit & (snr >= 6))[0][:10]],+ }+ return res+++GROUPS = {'keyboard': (lambda p: str(p.get('sample', '')).startswith('kb.'), {'KEY'}, {'KEY', 'UI'}),+ 'whoosh': (lambda p: p.get('kind') == 'whoosh', {'WHOOSH', 'RISER'}, {'WHOOSH', 'RISER'}),+ 'grass_steps': (lambda p: str(p.get('sample', '')).startswith('gd.step'), {'STEP'}, {'STEP', 'FOLEY'})}+++def _tag(x, sid, evs, n, center):+ keep = tl.TAG_CENTER+ tl.TAG_CENTER = center+ try:+ tl.tag_events(x, sid, 0.0, evs, n, 0)+ finally:+ tl.TAG_CENTER = keep+++def part_tags():+ P = placements()+ out = {}+ srcs = {'mix': SOUNDTRACK, 'sfx_stem': STEMS['sfx']}+ music_layer = [p['t'] for p in P if str(p.get('sample', '')).startswith('mg.')]+ groups = dict(GROUPS, keyboard_without_celesta_layer=(+ lambda p: GROUPS['keyboard'][0](p) and not any(abs(p['t'] - t) < 0.03 for t in music_layer), {'KEY'}, {'KEY', 'UI'}))+ for g, (sel, strict, lenient) in groups.items():+ ts = sorted({round(p['t'], 3) for p in P if sel(p)})+ ts = [t for k, t in enumerate(ts) if k == 0 or t - ts[k - 1] >= 0.03]+ shape = 'swell' if g == 'whoosh' else 'hit'+ out[g] = {'n': len(ts)}+ for name, path in srcs.items():+ x = audio.load(path)+ for rule, center in (('shipped', tl.TAG_CENTER), ('first_build', 0.0)):+ evs = [{'t': t, 'shape': shape, 'band': 'full', 'rise_ms': 400, 'length_ms': 300, 'prominence_db': 0.0,+ 'contrast_db': 0.0} for t in ts]+ _tag(x, tl.span_id(path), evs, len(evs), center)+ k1 = [e['kind'] for e in evs]+ k3 = [{t['kind'] for t in e['tags']} for e in evs]+ out[g][f'{name}_{rule}'] = {+ 'top1_strict': round(float(np.mean([k in strict for k in k1])), 3),+ 'top1_lenient': round(float(np.mean([k in lenient for k in k1])), 3),+ 'top3_strict': round(float(np.mean([bool(k & strict) for k in k3])), 3),+ 'labels': dict(Counter(e['label'] for e in evs).most_common(5))}+ for stem in ('amb', 'music'):+ x = audio.load(STEMS[stem])+ for rule, center in (('shipped', tl.TAG_CENTER), ('first_build', 0.0)):+ ev = [e for e in tl.detect_events(x) if e['shape'] == 'hit']+ _tag(x, tl.span_id(STEMS[stem]), ev, 20, center)+ tg = [e for e in ev if e.get('kind')]+ out[f'{stem}_stem_{rule}'] = {+ 'tagged': len(tg), 'key_or_ui': round(float(np.mean([e['kind'] in ('KEY', 'UI') for e in tg])), 3),+ 'kinds': dict(Counter(e['kind'] for e in tg).most_common()),+ 'labels': dict(Counter(e['label'] for e in tg).most_common(4))}+ return out+++def part_sync():+ P = placements()+ t = time.time()+ r = tl.timeline(SOUNDTRACK, cues=CUES, detail='normal')+ wall = time.time() - t+ os.makedirs(OUT, exist_ok=True)+ open(os.path.join(OUT, 'soundtrack_timeline.txt'), 'w', encoding='utf-8').write(r['text'])+ by_id = {}+ for p in P:+ if p.get('event'):+ by_id.setdefault(p['event'], []).append(p)+ errs, peak_only, flagged, ok = [], [], [], []+ unsounded = {(round(u['t'], 2), u['id']) for u in json.load(open(REPORT))['unsounded_events']}+ nohit = {'no sound', 'off'}+ for it in r['sync']['cues']['items']:+ ps = [p for p in by_id.get(it.get('id'), []) if abs(p['t'] - it['t']) < 0.5]+ att = [p['t'] for p in ps if p.get('align', 'attack') in ('attack', 'start')]+ if (round(it['t'], 2), it.get('id')) in unsounded:+ flagged.append({'t': it['t'], 'id': it.get('id'), 'truth': 'unsounded', 'status': it['status']})+ continue+ if not att:+ peak_only.append({'t': it['t'], 'kind': it.get('kind'), 'status': it['status']})+ continue+ true_off = (min(att, key=lambda v: abs(v - it['t'])) - it['t']) * 1000+ if it['offset_ms'] is None:+ ok.append(None)+ continue+ errs.append(it['offset_ms'] - true_off)+ e = np.abs(np.array(errs))+ grid = r['music']+ beats = np.array([b for b in grid['beats'] if b < 52.0])+ on_grid = np.abs(beats / 0.5 - np.round(beats / 0.5)) * 0.5 if len(beats) else np.zeros(0)+ db = np.array([b for b in grid['downbeats'] if b < 50.5])+ true_bounds = [10.5, 12.0, 28.0, 40.0, 51.0, 52.2]+ det_bounds = [s['start'] for s in r['sections'][1:]]+ mb = match(true_bounds, det_bounds, 1.5)+ res = {+ 'cues': r['sync']['cues']['stats'], 'n_cues_scored': len(errs) + len(ok),+ 'peak_or_end_aligned_cues': peak_only,+ 'without_a_hit_vs_truth_all_cues': {+ 'reported_without_a_hit': len(r['sync']['cues']['stats']['no_sound']),+ 'of_them_truly_unsounded': sum(f['status'] in nohit for f in flagged),+ 'truly_unsounded': len(flagged),+ 'sounded_reported_without_a_hit': len(r['sync']['cues']['stats']['no_sound'])+ - sum(f['status'] in nohit for f in flagged),+ 'reported_level_peak_only': sum(it['status'] == 'peak' for it in r['sync']['cues']['items']),+ 'level_peak_only_truly_unsounded': sum(f['status'] == 'peak' for f in flagged)},+ 'offset_error_ms': {'within_5ms': round(float(np.mean(e <= 5)), 3), 'within_1_frame_16.7ms': round(float(np.mean(e <= 16.7)), 3),+ 'within_40ms': round(float(np.mean(e <= 40)), 3), 'median_abs': round(float(np.median(e)), 1),+ 'p90_abs': round(float(np.percentile(e, 90)), 1)},+ 'sounded_cues_reported_without_sound': len(ok),+ 'truly_unsounded_cues': flagged,+ 'beat_grid': {'tempo': grid.get('tempo'), 'confidence': grid.get('confidence'),+ 'pulse_clarity': grid.get('pulse_clarity'), 'n_beats_0_52s': int(len(beats)),+ 'beats_within_40ms_of_0.5s_grid': round(float(np.mean(on_grid <= 0.04)), 3) if len(beats) else None,+ 'downbeats_on_even_seconds_60ms': round(float(np.mean(np.abs(db / 2 - np.round(db / 2)) * 2 <= 0.06)), 3)+ if len(db) else None},+ 'sections': {'true_boundaries': true_bounds, 'detected': det_bounds, 'matched_within_1.5s': len(mb),+ 'labels': [f"{s['start']}-{s['end']}: {s['label']}" for s in r['sections']]},+ 'summary': r['summary'], 'timing_s': r['timing_s'], 'model_calls': r['model_calls'], 'wall_s': round(wall, 1),+ }+ return res+++def part_film():+ from earshot.sheet import sheet+ t = time.time()+ r = tl.timeline(FILM, cues=CUES, detail='normal')+ wall = time.time() - t+ os.makedirs(OUT, exist_ok=True)+ open(os.path.join(OUT, 'film_timeline.txt'), 'w', encoding='utf-8').write(r['text'])+ acts = [8.0, 14.0, 28.0, 40.0, 52.0]+ mc = match(acts, r['cuts'], 0.1)+ s = sheet(FILM, os.path.join(OUT, 'clawd_film_sheet.png'), tl=r)+ z = sheet(FILM, os.path.join(OUT, 'clawd_zoom_14-24.png'), cues=CUES, start=14.0, end=24.0)+ return {'cuts': r['cuts'], 'act_changes': acts, 'act_changes_found_100ms': len(mc),+ 'cut_sync': (r['sync'].get('cuts') or {}).get('stats'), 'cue_sync': r['sync']['cues']['stats'],+ 'summary': r['summary'], 'timing_s': r['timing_s'], 'wall_s': round(wall, 1),+ 'sheet': s['summary'], 'zoom_sheet': z['summary']}+++def part_refs():+ from earshot.sheet import sheet+ out = {}+ for name, path in REFS.items():+ t = time.time()+ r = tl.timeline(path, detail='normal')+ wall = time.time() - t+ open(os.path.join(OUT, f'{name}_timeline.txt'), 'w', encoding='utf-8').write(r['text'])+ s = sheet(path, os.path.join(OUT, f'{name}_sheet.png'), tl=r)+ tg = [e for e in r['events'] if e.get('label')]+ top = Counter(e['label'] for e in tg).most_common(1)+ out[name] = {'duration': r['duration'], 'summary': r['summary'], 'n_events': len(r['events']),+ 'tagged': len(tg), 'key_or_ui': round(float(np.mean([e['kind'] in ('KEY', 'UI') for e in tg])), 3)+ if tg else None, 'most_common_tag': [top[0][0], top[0][1]] if top else None,+ 'kinds': dict(Counter(e['kind'] for e in tg).most_common()),+ 'cuts': len(r['cuts']), 'sections': [s_['label'] for s_ in r['sections']],+ 'cut_sync': (r['sync'].get('cuts') or {}).get('stats'), 'timing_s': r['timing_s'],+ 'wall_s': round(wall, 1), 'sheet': s['summary']}+ return out+++def part_timing60():+ return _timing('soundtrack_60s', SOUNDTRACK)+++def part_timing93():+ return _timing('ref_1_film_93s', REFS['ref_1'])+++def part_timing93low():+ return _timing('ref_1_film_93s_detail_low', REFS['ref_1'], 'low')+++def _timing(name, path, detail='normal'):+ """wall-clock of a fresh CLI run (new process: imports, model load, no cached results or model outputs)"""+ env = dict(os.environ, EARSHOT_THREADS=os.environ.get('EARSHOT_THREADS', '2'))+ n_py = _n_python()+ t = time.time()+ p = subprocess.run([sys.executable, '-m', 'earshot', 'timeline', path, '--fresh', '--json', '--detail', detail],+ cwd=ROOT, env=env, capture_output=True, text=True, encoding='utf-8', errors='replace')+ wall = time.time() - t+ try:+ j = json.loads(p.stdout)+ except ValueError:+ return {name: {'error': p.stderr[-800:]}}+ return {name: {'wall_s_cli_incl_imports_and_model_load': round(wall, 1), 'internal_timing_s': j['timing_s'],+ 'model_calls': j['model_calls'], 'threads': env['EARSHOT_THREADS'], 'duration_s': j['duration'],+ 'other_python_processes_at_start': n_py}}+++def part_probe():+ """model load and per-window cost right now (the laptop is shared, so this varies with load)"""+ from earshot.models import ast, clap+ x = audio.load(SOUNDTRACK, mono=True)+ segs = [x[int(i * SR): int((i + 0.5) * SR)] for i in range(10, 18)]+ out = {'python_processes_running': _n_python()}+ t = time.time()+ c = clap.get()+ out['clap_load_s'] = round(time.time() - t, 1)+ t = time.time()+ c.embed_audio(segs, pad='zero')+ out['clap_s_per_window'] = round((time.time() - t) / len(segs), 2)+ t = time.time()+ a = ast.get()+ out['ast_load_s'] = round(time.time() - t, 1)+ t = time.time()+ a.probs(segs[:2])+ out['ast_s_per_window'] = round((time.time() - t) / 2, 2)+ return out+++def _n_python():+ p = subprocess.run(['tasklist', '/FI', 'IMAGENAME eq python.exe', '/NH'], capture_output=True, text=True)+ return sum(1 for ln in p.stdout.splitlines() if ln.lower().startswith('python'))+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--parts', default='detect,tags,sync')+ a = ap.parse_args()+ res = json.load(open(RESULTS)) if os.path.exists(RESULTS) else {}+ os.makedirs(OUT, exist_ok=True)+ for part in a.parts.split(','):+ t = time.time()+ res[part] = globals()[f'part_{part}']()+ res[part]['_run_s'] = round(time.time() - t, 1)+ print(part, json.dumps(res[part], indent=1)[:4000], flush=True)+ with open(RESULTS, 'w') as fh:+ json.dump(res, fh, indent=1)+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/section_tune.py b/eval/timeline/section_tune.pynew file mode 100644index 0000000..18e1163--- /dev/null+++ b/eval/timeline/section_tune.py@@ -0,0 +1,47 @@+"""Tune timeline's section merging on the synthetic mixes (bed switches = true boundaries). Never uses Clawd.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/section_tune.py [--shipped]++Boundary recall / precision within +-1.5 s for several CLAP merge thresholds and Foote-novelty thresholds (99 = no+novelty split); --shipped runs only the shipped setting (3 s windows, 2 s hop, merge 0.84, novelty z 3.0). CLAP+window embeddings are cached, so only the first run pays for the model. Labels come from CLAP only (no AST), as in+timeline's merge of neighbouring sections that read alike.+"""+import glob+import json+import os+import sys++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio, timeline as tl # noqa: E402+from tune import match # noqa: E402+++def main(d='out/timeline/mixes', tol=1.5):+ mixes = [(audio.load(f[:-5] + '.wav'), tl.span_id(f[:-5] + '.wav'), json.load(open(f)))+ for f in sorted(glob.glob(os.path.join(d, 'mix_*.json')))]+ grid = [(3.0, 2.0, mc, nz) for mc in (0.80, 0.84, 0.88) for nz in (2.0, 3.0, 4.0, 99.0)]+ if '--shipped' in sys.argv:+ grid = [(3.0, 2.0, 0.84, tl.NOVELTY_Z)]+ else:+ grid += [(4.0, 4.0, 0.84, 3.0), (3.0, 1.5, 0.84, 3.0)]+ for win, hop, mc, nz in grid:+ tp = n_gt = n_det = 0+ err = []+ for x, fid, meta in mixes:+ secs, _ = tl.sections(x, fid, 0.0, win, hop, ast_budget=0, merge_cos=mc, nov_z=nz)+ det = [s['start'] for s in secs[1:]]+ m = match(meta['boundaries'], det, tol)+ tp += len(m)+ n_gt += len(meta['boundaries'])+ n_det += len(det)+ err += [abs(dt) for _, _, dt in m]+ print(json.dumps({'win': win, 'hop': hop, 'merge_cos': mc, 'novelty_z': nz, 'recall': round(tp / n_gt, 3),+ 'precision': round(tp / max(1, n_det), 3), 'n_gt': n_gt, 'n_det': n_det,+ 'median_abs_err_s': round(float(np.median(err)), 2) if err else None}), flush=True)+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/swell_tune.py b/eval/timeline/swell_tune.pynew file mode 100644index 0000000..34db3b9--- /dev/null+++ b/eval/timeline/swell_tune.py@@ -0,0 +1,65 @@+"""Tune the build-up (whoosh / riser) detector on the synthetic mixes: whooshes are placed with their peak on the+ground-truth time. Never uses the Clawd mix.++ .venv/Scripts/python.exe eval/timeline/swell_tune.py [--peaks]++--peaks tunes the level-peak check sync uses where no onset answers a cut / cue: at every whoosh peak (should be+answered) and at times on a 0.25 s grid at least 0.6 s from any placed sound (nothing placed there), how often each+rise threshold finds a level peak among the times no onset / swell answers.+"""+import glob+import json+import os+import sys++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+from earshot import audio, timeline as tl # noqa: E402+from tune import match # noqa: E402+++def peaks(d='out/timeline/mixes'):+ rows = {'whoosh': [], 'empty': []}+ for f in sorted(glob.glob(os.path.join(d, 'mix_*.json'))):+ x = audio.load(f[:-5] + '.wav')+ meta, ev, lv = json.load(open(f)), tl.detect_events(x), tl.level_envs(x)+ pt = np.array([g['t'] for g in meta['events']])+ grid = [t for t in np.arange(1.0, meta['duration'] - 1.0, 0.25) if np.min(np.abs(pt - t)) >= 0.6]+ for kind, times in (('whoosh', [g['t'] for g in meta['events'] if g['align'] == 'peak']), ('empty', grid)):+ for it in tl.sync(times, ev)['items']:+ rows[kind].append((it['t'], it['status'], lv))+ for kind, r in rows.items():+ un = [(t, lv) for t, st, lv in r if st in ('no sound', 'off')]+ print(kind, json.dumps({'n': len(r), 'answered_by_onset_or_swell': round(1 - len(un) / len(r), 3),+ 'unanswered': len(un)}))+ for rise, local in ((3.0, 0.3), (4.0, 0.3), (5.0, 0.3), (6.0, 0.3), (8.0, 0.3), (3.0, 0.15), (4.0, 0.15)):+ got = [tl.level_peak(lv, t, min_rise_db=rise, local=local) is not None for t, lv in un]+ print(f' rise >= {rise} dB, local max over +-{local} s: level peak found for {sum(got)}/{len(un)} '+ f'unanswered ({100 * np.mean(got) if got else 0:.0f}%)')+++def main(d='out/timeline/mixes'):+ if '--peaks' in sys.argv:+ return peaks(d)+ mixes = []+ for f in sorted(glob.glob(os.path.join(d, 'mix_*.json'))):+ x = audio.load(f[:-5] + '.wav')+ mixes.append((x, [e for e in tl.detect_events(x) if e['shape'] == 'hit'], json.load(open(f))))+ for rise, mono, dur in [(r, m, d) for r in (5.0, 6.0, 8.0) for m in (0.5, 0.6) for d in (0.12, 0.25)]:+ found = n_gt = n_sw = near_any = 0+ for x, hits, meta in mixes:+ sw = tl._swells(x, hits, min_rise_db=rise, mono_frac=mono, min_dur=dur)+ gt = [e['t'] for e in meta['events'] if e['align'] == 'peak']+ found += len(match(gt, [s['t'] for s in sw], 0.12))+ n_gt += len(gt)+ n_sw += len(sw)+ near_any += sum(any(abs(s['t'] - g['t']) < 0.3 for g in meta['events']) for s in sw)+ print(json.dumps({'min_rise_db': rise, 'mono_frac': mono, 'min_dur_s': dur,+ 'whoosh_peaks_found_120ms': f'{found}/{n_gt}', 'swells': n_sw,+ 'swells_within_300ms_of_any_placed_sound': near_any}), flush=True)+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/tag_bg.py b/eval/timeline/tag_bg.pynew file mode 100644index 0000000..1412d26--- /dev/null+++ b/eval/timeline/tag_bg.py@@ -0,0 +1,147 @@+"""Remove hub phrases from timeline's event tags: per-phrase background statistics over real events, and the choice+of scoring rule and confidence threshold. Never uses the Clawd film or the three reference films used for evaluation.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/tag_bg.py [--per 40]++Background events: the events timeline would tag (its own picking, up to --per per source) in the 8 synthetic tuning+mixes (mixes.py) and in four launch films that the evaluation does not use (ref_3, ref_5,+ref_6, ref_7). Fit half = mixes 0-3 + ref_3 + ref_5; test half = mixes 4-7 + ref_6 + ref_7. Scoring rules over the+CLAP cosines C (events x phrases), with the fit half's per-phrase mean mu and std sd:+ row z of C within each event (the first build: phrases that score high on everything win: hubs)+ col (C - mu) / sd: how unusual the phrase's score is for this event compared with typical events+ cent z within each event of C - mu+Metrics (test half): one-shot top-1 / top-3 kind (FIT, as tag_tune.py), the share of bed events (music / ambience+onsets at least 0.15 s from any one-shot) called KEY or UI, and the share of the single most common phrase over all+test events (hubness). Then confidence thresholds on the top background z. Prints the TAG_BG table (all sources) for+earshot/timeline.py.+"""+import argparse+import json+import os+import sys+from collections import Counter++import numpy as np++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+sys.path.insert(0, HERE)+from earshot import audio, timeline as tl # noqa: E402+from earshot.models import clap # noqa: E402+from tag_tune import FIT # noqa: E402+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # eval/: local data locations+import evalpaths as EP # noqa: E402++FILMS = [EP.ref(rid) for rid in ('ref_3', 'ref_5', 'ref_6', 'ref_7')] # 4 reference films the evaluation does not use+UIK = {'KEY', 'UI'}+++def sources():+ out = [(f'out/timeline/mixes/mix_{k}.wav', 'fit' if k < 4 else 'test') for k in range(8)]+ return out + [(f, 'fit' if k < 2 else 'test') for k, f in enumerate(FILMS)]+++def events_of(path, per, c, E):+ """the events timeline would tag (+ every placed one-shot in a mix) with their cosines and ground truth"""+ x = audio.load(path)+ ev = [e for e in tl.detect_events(x) if e['shape'] == 'hit']+ dur = len(x) / tl.SR+ order = sorted(range(len(ev)), key=lambda i: (-ev[i]['prominence_db'], -ev[i]['contrast_db']))+ pick = set(tl._spread(order, [e['t'] for e in ev], dur, per))+ meta = json.load(open(path[:-4] + '.json')) if path.endswith('.wav') else None+ truth = {}+ if meta:+ det = np.array([e['t'] for e in ev])+ for g in meta['events']:+ k = int(np.argmin(np.abs(det - g['t'])))+ if g['align'] == 'attack' and abs(det[k] - g['t']) <= 0.03:+ truth[k] = (g['cat'], g['snr_db'])+ pick.add(k)+ pick = sorted(pick)+ spans = [tl._event_span(ev[i], dur) for i in pick]+ A = tl._cached(c, tl.span_id(path), spans, 'clap-repeat',+ lambda q: c.embed_audio([tl._seg(x, a, b) for a, b in q], pad='repeatpad'))+ rows = []+ for j, i in enumerate(pick):+ snr = np.nan+ if i in truth:+ role, snr = truth[i]+ elif meta and min(abs(ev[i]['t'] - g['t']) for g in meta['events']) >= 0.15:+ role = 'bed'+ elif meta:+ role = 'other'+ else:+ role = 'film'+ rows.append({'role': role, 'cos': A[j] @ E.T, 'prom': ev[i]['prominence_db'], 'snr': snr})+ return rows+++def score(rule, C, mu, sd):+ if rule == 'row':+ return (C - C.mean(1, keepdims=True)) / (C.std(1, keepdims=True) + 1e-9)+ if rule == 'col':+ return (C - mu) / sd+ D = C - mu+ return (D - D.mean(1, keepdims=True)) / (D.std(1, keepdims=True) + 1e-9)+++def report(rule, rows, terms, kinds, mu, sd):+ C = np.stack([r['cos'] for r in rows])+ Z = score(rule, C, mu, sd)+ top = np.argsort(-Z, axis=1)[:, :3]+ one = [j for j, r in enumerate(rows) if r['role'] in FIT]+ bed = [j for j, r in enumerate(rows) if r['role'] == 'bed']+ t1 = [kinds[top[j, 0]] in FIT[rows[j]['role']] for j in one]+ t3 = [any(kinds[k] in FIT[rows[j]['role']] for k in top[j]) for j in one]+ cnt = Counter(terms[top[j, 0]] for j in range(len(rows)))+ res = {'rule': rule, 'n_oneshots': len(one), 'top1': round(float(np.mean(t1)), 3),+ 'top3': round(float(np.mean(t3)), 3), 'n_bed': len(bed),+ 'bed_called_key_or_ui': round(float(np.mean([kinds[top[j, 0]] in UIK for j in bed])), 3),+ 'film_called_key_or_ui': round(float(np.mean([kinds[top[j, 0]] in UIK for j, r in enumerate(rows)+ if r['role'] == 'film'])), 3),+ 'most_common_share': round(cnt.most_common(1)[0][1] / len(rows), 3), 'most_common': cnt.most_common(4),+ 'per_category': {k: round(float(np.mean([t for t, j in zip(t1, one) if rows[j]['role'] == k])), 2)+ for k in sorted({rows[j]['role'] for j in one})}}+ print(json.dumps(res), flush=True)+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--per', type=int, default=40)+ a = ap.parse_args()+ c = clap.get()+ terms, kinds, E = tl._event_matrix(c)+ kinds = [str(k) for k in kinds]+ rows = {'fit': [], 'test': []}+ for path, half in sources():+ rows[half] += events_of(path, a.per, c, E)+ np.savez('out/timeline/tag_bg_rows.npz', **{f'{h}_{k}': np.array([r[k] for r in rows[h]])+ for h in rows for k in ('role', 'cos', 'prom', 'snr')})+ print('events: fit', len(rows['fit']), 'test', len(rows['test']), flush=True)+ F = np.stack([r['cos'] for r in rows['fit']])+ mu, sd = F.mean(0), F.std(0) + 1e-6+ for rule in ('row', 'col', 'cent'):+ report(rule, rows['test'], terms, kinds, mu, sd)+ C = np.stack([r['cos'] for r in rows['test']])+ Z = score('col', C, mu, sd)+ top = np.argmax(Z, axis=1)+ for thr in (0.5, 1.0, 1.5, 2.0, 2.5):+ conf = Z[np.arange(len(Z)), top] >= thr+ one = [j for j, r in enumerate(rows['test']) if r['role'] in FIT]+ bed = [j for j, r in enumerate(rows['test']) if r['role'] == 'bed']+ film = [j for j, r in enumerate(rows['test']) if r['role'] == 'film']+ ok = [kinds[top[j]] in FIT[rows['test'][j]['role']] for j in one if conf[j]]+ print(json.dumps({'col_threshold': thr, 'oneshot_coverage': round(float(np.mean(conf[one])), 3),+ 'oneshot_precision': round(float(np.mean(ok)), 3) if ok else None,+ 'bed_confident': round(float(np.mean(conf[bed])), 3),+ 'bed_confident_key_or_ui': round(float(np.mean([conf[j] and kinds[top[j]] in UIK+ for j in bed])), 3),+ 'film_confident': round(float(np.mean(conf[film])), 3)}), flush=True)+ A = np.stack([r['cos'] for h in rows.values() for r in h])+ mu, sd = A.mean(0), A.std(0)+ print(f'TAG_BG from {len(A)} events:')+ print(json.dumps({t: [round(float(m), 4), round(float(s), 4)] for t, m, s in zip(terms, mu, sd)}))+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/tag_tune.py b/eval/timeline/tag_tune.pynew file mode 100644index 0000000..9d1adc5--- /dev/null+++ b/eval/timeline/tag_tune.py@@ -0,0 +1,151 @@+"""Choose how timeline tags events: excerpt treatment x vocabulary, scored on the synthetic mixes (mixes.py; every+one-shot carries its library category). Never uses the Clawd mix.++ EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/tag_tune.py --vocab compact --variants delta,repeat,zero,iso++vocab json = all event categories of earshot/data/vocab.json (kind from the category), compact = timeline's own+ EVENT_VOCAB (kind = its key)+variants zero / repeat = CLAP on the event excerpt, zero- or repeat-padded to 10 s (repeat: what timeline ships);+ delta = zero-padded cosine minus the same for the bed just before the event (the first build);+ deltarep = the same with repeat padding;+ win = minus the 3 s section window around the event; debed = excerpt after subtracting the bed's average+ spectrum; iso = the one-shot alone (an upper bound: no bed at all)+--center subtract this share of each phrase's mean cosine over the file's events (hub-phrase correction)+A tag is plausible when the kind of the top phrase fits the one-shot's library category (FIT below).+Scores here are the first build's (z within each event); timeline now also damps hub phrases (tag_bg.py).+"""+import argparse+import glob+import json+import os+import sys+from collections import Counter++import numpy as np+from scipy import signal++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))+sys.path.insert(0, HERE)+from earshot import audio, timeline as tl # noqa: E402+from earshot.models import clap # noqa: E402+import mixes # noqa: E402++FIT = {'keyboard': {'KEY'}, 'motion': {'WHOOSH', 'RISER'}, 'garden': {'STEP', 'FOLEY'},+ 'toy': {'FOLEY', 'UI'}, 'magic': {'NOTE', 'UI'}, 'foley': {'FOLEY', 'STEP', 'UI'}}+JSON_CATS = ('transitions', 'impacts', 'ui', 'foley', 'instruments')+++def json_kind(term, cat):+ """kind of a vocab.json phrase, for comparing against the compact vocabulary"""+ if cat == 'transitions':+ return 'RISER' if any(w in term for w in ('riser', 'uplifter', 'swell')) else 'WHOOSH'+ if cat == 'impacts':+ return 'HIT'+ if cat == 'ui':+ return 'KEY' if any(w in term for w in ('keyboard', 'key press', 'spacebar', 'typing')) else 'UI'+ if cat == 'foley':+ return 'STEP' if 'footsteps' in term else 'FOLEY'+ return 'NOTE'+++def vocabulary(c, which):+ if which == 'compact':+ terms, kinds, E = tl._event_matrix(c)+ return terms, [str(k) for k in kinds], E+ terms, cats, E = tl._vocab_matrix(c)+ keep = [i for i, k in enumerate(cats) if k in JSON_CATS]+ return [terms[i] for i in keep], [json_kind(terms[i], cats[i]) for i in keep], E[keep]+++def debed(x, a, b, pre_s=0.5):+ """excerpt with the bed's average magnitude spectrum (pre_s before it) subtracted, -20 dB floor, original phase"""+ seg = tl._seg(x, a, b).astype(np.float64)+ pre = tl._seg(x, max(0.0, a - pre_s), max(0.0, a - 0.01)).astype(np.float64) if a > 0.1 else np.zeros(0)+ if len(pre) < 2048 or len(seg) < 1024:+ return seg.astype(np.float32)+ _, _, P = signal.stft(pre, nperseg=1024, noverlap=768)+ _, _, S = signal.stft(seg, nperseg=1024, noverlap=768)+ mag = np.abs(S)+ y = signal.istft(np.maximum(mag - np.abs(P).mean(axis=1, keepdims=True), 0.1 * mag) * np.exp(1j * np.angle(S)),+ nperseg=1024, noverlap=768)[1]+ return y[: len(seg)].astype(np.float32)+++def iso(g):+ for d in (os.path.join(mixes.ASSETS, g['cat']), os.path.join(mixes.ASSETS, '_rejected', g['cat'])):+ f = os.path.join(d, g['file'])+ if os.path.exists(f):+ got = mixes.whooshes(f) if g['cat'] == 'motion' else mixes.one_shots(f)+ return got[0][0] if got else np.zeros(4800, np.float32)+ return np.zeros(4800, np.float32)+++def cosines(v, c, E, x, fid, spans, picks):+ def emb(kind, sp, make, pad):+ return tl._cached(c, fid, sp, kind, lambda q: c.embed_audio([make(p, r) for p, r in q], pad=pad))++ def seg(p, r):+ return tl._seg(x, p, r)+ pre = [(max(0.0, p - (q - p) - 0.01), max(0.01, p - 0.01)) for p, q in spans]+ if v == 'iso':+ return c.embed_audio([iso(g) for g, _ in picks], pad='zero') @ E.T+ if v == 'delta':+ return emb('clap-zero', spans, seg, 'zero') @ E.T - emb('clap-zero', pre, seg, 'zero') @ E.T+ if v == 'deltarep':+ return emb('clap-repeat', spans, seg, 'repeatpad') @ E.T - emb('clap-repeat', pre, seg, 'repeatpad') @ E.T+ if v == 'win':+ dur = len(x) / tl.SR+ starts = list(np.arange(0.0, max(dur - 3.0, 0.0) + 1e-6, 2.0))+ if starts[-1] + 3.0 < dur - 0.5:+ starts.append(max(0.0, dur - 3.0))+ wsp = [(round(a, 3), round(min(dur, a + 3.0), 3)) for a in starts]+ W = emb('clap-repeat', wsp, seg, 'repeatpad')+ mids = np.array([(a + b) / 2 for a, b in wsp])+ ref = np.stack([W[int(np.argmin(np.abs(mids - (p + q) / 2)))] for p, q in spans])+ return emb('clap-repeat', spans, seg, 'repeatpad') @ E.T - ref @ E.T+ if v == 'debed':+ return emb('clap-debed', spans, lambda p, r: debed(x, p, r), 'zero') @ E.T+ return emb(f'clap-{v}', spans, seg, 'repeatpad' if v == 'repeat' else 'zero') @ E.T+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--dir', default='out/timeline/mixes')+ ap.add_argument('--vocab', default='compact', choices=('compact', 'json'))+ ap.add_argument('--variants', default='delta,repeat,zero')+ ap.add_argument('--center', type=float, default=0.0)+ a = ap.parse_args()+ c = clap.get()+ terms, kinds, E = vocabulary(c, a.vocab)+ rows = {v: [] for v in a.variants.split(',')}+ for f in sorted(glob.glob(os.path.join(a.dir, 'mix_*.json'))):+ meta = json.load(open(f))+ x = audio.load(f[:-5] + '.wav')+ ev = tl.detect_events(x)+ det = np.array([e['t'] for e in ev])+ picks = [(g, ev[int(np.argmin(np.abs(det - g['t'])))]) for g in meta['events']+ if np.min(np.abs(det - g['t'])) <= (0.12 if g['align'] == 'peak' else 0.03)]+ spans = [tl._event_span(e, len(x) / tl.SR) for _, e in picks]+ for v in rows:+ cos = cosines(v, c, E, x, audio.file_id(f[:-5] + '.wav'), spans, picks)+ if a.center:+ cos = cos - a.center * cos.mean(axis=0, keepdims=True)+ z = (cos - cos.mean(axis=1, keepdims=True)) / (cos.std(axis=1, keepdims=True) + 1e-9)+ for j, (g, _) in enumerate(picks):+ idx = np.argsort(-z[j])[:3]+ rows[v].append({'cat': g['cat'], 'snr': g['snr_db'], 'top1': kinds[idx[0]] in FIT[g['cat']],+ 'top3': any(kinds[i] in FIT[g['cat']] for i in idx), 'tag': terms[idx[0]]})+ for v, r in rows.items():+ t1, t3 = np.array([q['top1'] for q in r]), np.array([q['top3'] for q in r])+ snr = np.array([q['snr'] for q in r])+ print(v, json.dumps({'vocab': a.vocab, 'center': a.center, 'n': len(r), 'top1': round(float(t1.mean()), 3),+ 'top3': round(float(t3.mean()), 3), 'top1_snr_ge0': round(float(t1[snr >= 0].mean()), 3),+ 'top1_snr_lt0': round(float(t1[snr < 0].mean()), 3),+ 'per_category': {k: round(float(np.mean([q['top1'] for q in r if q['cat'] == k])), 2)+ for k in sorted({q['cat'] for q in r})},+ 'most_common_tags': Counter(q['tag'] for q in r).most_common(4)}), flush=True)+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/tempo_tune.py b/eval/timeline/tempo_tune.pynew file mode 100644index 0000000..5fd882f--- /dev/null+++ b/eval/timeline/tempo_tune.py@@ -0,0 +1,59 @@+"""Choose when timeline prefers the faster of two octave-related tempos (a two-beat pattern makes the autocorrelation+peak at the half tempo). Reference: librosa's beat tracker (not ground truth, an independent second opinion) on the+same 20 s excerpts of the unused / rejected library music as pulse_tune.py; never the Clawd soundtrack.++ .venv/Scripts/python.exe eval/timeline/tempo_tune.py++For each excerpt: timeline's tempo, the autocorrelation at its beat period and at half of it (the faster octave), and+librosa's tempo. Prints how often timeline agrees with librosa (within 4%, or at 2x / 0.5x) for each rule+"faster octave when ac(P/2) >= r * ac(P) and it is <= 180 BPM".+"""+import json+import os+import sys++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio, timeline as tl # noqa: E402+import mixes # noqa: E402+++def main():+ import librosa+ rows = []+ for f in mixes.unused('music'):+ x = audio.load(f, mono=True, start=10.0, end=30.0)+ if len(x) < 8 * audio.SR:+ x = audio.load(f, mono=True, end=20.0)+ cands = [p for env in tl._pulse_envs(x) for p in [tl._period(env, 60, 200)] if p]+ if not cands:+ continue+ P, clar, _, ac = max(cands, key=lambda c: c[2])+ y = audio.resample(x, audio.SR, 22050)+ lt = float(np.atleast_1d(librosa.beat.beat_track(y=y, sr=22050)[0])[0])+ rows.append({'file': os.path.basename(f), 'tempo': 6000 / P, 'clarity': clar,+ 'ac_half': float(ac[int(round(P / 2))]), 'librosa': lt})+ print(len(rows), 'music excerpts')++ def agree(a, b, oct_ok=False):+ r = a / b+ return abs(r - 1) <= 0.04 or (oct_ok and (abs(r - 2) <= 0.08 or abs(r - 0.5) <= 0.02))+ for ratio in (None, 1.0, 0.9, 0.8, 0.7, 0.6, 0.5):+ same = octv = faster = 0+ for q in rows:+ t = q['tempo']+ if ratio is not None and 2 * t <= 180 and q['ac_half'] >= ratio * q['clarity']:+ t, faster = 2 * t, faster + 1+ same += int(agree(t, q["librosa"]))+ octv += int(agree(t, q["librosa"], True))+ print(json.dumps({'rule_ratio': ratio, 'switched_to_faster': faster, 'agree_with_librosa': same,+ 'agree_incl_octave': octv, 'n': len(rows)}))+ print('timeline slower than librosa (before):', [(q['file'][:30], round(q['tempo']), round(q['librosa']),+ round(q['ac_half'] / max(q['clarity'], 1e-6), 2))+ for q in rows if agree(2 * q['tempo'], q['librosa'])])+++if __name__ == '__main__':+ main()diff --git a/eval/timeline/tune.py b/eval/timeline/tune.pynew file mode 100644index 0000000..b8879c8--- /dev/null+++ b/eval/timeline/tune.py@@ -0,0 +1,93 @@+"""Tune the timeline event detector on the synthetic mixes (eval/timeline/mixes.py), never on the Clawd soundtrack.++ .venv/Scripts/python.exe eval/timeline/tune.py [--dir out/timeline/mixes] [--grid]++Reports recall (hits found within 30 ms of their attack; whooshes within 120 ms of their peak), precision against+the placed hits, and the share of extra detections explained by onsets in the bed alone (music / ambience notes).+"""+import argparse+import glob+import json+import os+import sys++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))+from earshot import audio, timeline # noqa: E402+++def match(gt, det, tol):+ """greedy one-to-one matching by |dt|: list of (gt_index, det_index, dt)"""+ pairs = sorted((abs(d - g), i, j, d - g) for i, g in enumerate(gt) for j, d in enumerate(det) if abs(d - g) <= tol)+ gi, dj, out = set(), set(), []+ for _, i, j, dt in pairs:+ if i not in gi and j not in dj:+ gi.add(i); dj.add(j); out.append((i, j, dt))+ return out+++def score(mixes, zthr=None, prom=None):+ """mixes: [(events detected at the loosest thresholds, bed events, meta)]; filter, then score"""+ rows = {'hit': [], 'dt': [], 'snr': [], 'whoosh': [], 'n_det': 0, 'tp': 0, 'explained': 0}+ for ev, bed, meta in mixes:+ if zthr is not None:+ ev = [e for e in ev if e['shape'] == 'swell' or (e['z'] >= zthr and e['contrast_db'] >= prom)]+ det = [e['t'] for e in ev]+ hits = [e for e in meta['events'] if e['align'] == 'attack']+ wh = [e for e in meta['events'] if e['align'] == 'peak']+ m1 = match([e['t'] for e in hits], det, 0.03)+ found = {i for i, _, _ in m1}+ rows['hit'] += [i in found for i in range(len(hits))]+ rows['snr'] += [e['snr_db'] for e in hits]+ rows['dt'] += [dt for _, _, dt in m1]+ rest = [d for j, d in enumerate(det) if j not in {j for _, j, _ in m1}]+ m2 = match([e['t'] for e in wh], rest, 0.12)+ rows['whoosh'] += [i in {i for i, _, _ in m2} for i in range(len(wh))]+ rest_j = [j for j in range(len(det)) if j not in {j for _, j, _ in m1}]+ used = {j for _, j, _ in m1} | {rest_j[j] for _, j, _ in m2}+ rest_idx = [j for j in range(len(det)) if j not in used]+ bed_det = [e['t'] for e in bed]+ m3 = match(bed_det, [det[j] for j in rest_idx], 0.03)+ rows['n_det'] += len(det)+ rows['tp'] += len(m1) + len(m2)+ rows['explained'] += len(m3)+ h, snr = np.array(rows['hit']), np.array(rows['snr'])+ res = {'recall_hits': round(float(h.mean()), 3), 'recall_snr_ge0': round(float(h[snr >= 0].mean()), 3),+ 'recall_snr_-6_0': round(float(h[(snr < 0) & (snr >= -6)].mean()), 3),+ 'recall_snr_lt-6': round(float(h[snr < -6].mean()), 3),+ 'recall_whoosh': round(float(np.mean(rows['whoosh'])), 3) if rows['whoosh'] else None,+ 'precision_vs_placed': round(rows['tp'] / max(1, rows['n_det']), 3),+ 'precision_incl_bed_onsets': round((rows['tp'] + rows['explained']) / max(1, rows['n_det']), 3),+ 'n_det': rows['n_det'], 'n_gt': int(len(h) + len(rows['whoosh'])),+ 'median_dt_ms': round(float(np.median(rows['dt'])) * 1000, 1) if rows['dt'] else None,+ 'p90_abs_dt_ms': round(float(np.percentile(np.abs(rows['dt']), 90)) * 1000, 1) if rows['dt'] else None}+ return res+++def load(d, **kw):+ """detections per mix + every real onset in its bed alone (loosest settings, so extra detections that are+ genuine bed onsets count as explained)"""+ out = []+ for f in sorted(glob.glob(os.path.join(d, 'mix_*.json'))):+ x, b = audio.load(f[:-5] + '.wav'), audio.load(f[:-5] + '_bed.wav')+ bed = timeline.detect_events(b, zthr=2.0, min_contrast_db=1.0)+ out.append((timeline.detect_events(x, **kw), bed, json.load(open(f))))+ return out+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument('--dir', default='out/timeline/mixes')+ ap.add_argument('--grid', action='store_true')+ a = ap.parse_args()+ print('default', json.dumps(score(load(a.dir))))+ if a.grid:+ loose = load(a.dir, zthr=2.0, min_contrast_db=1.0)+ for z in (2.0, 2.5, 3.0, 3.5, 4.0, 5.0, 6.0):+ for p in (1.0, 3.0, 6.0):+ print(f'zthr {z} prom {p}', json.dumps(score(loose, z, p)))+++if __name__ == '__main__':+ main()diff --git a/pyproject.toml b/pyproject.tomlnew file mode 100644index 0000000..f8a0eb7--- /dev/null+++ b/pyproject.toml@@ -0,0 +1,63 @@+[build-system]+requires = ["setuptools>=69"]+build-backend = "setuptools.build_meta"++[project]+name = "earshot"+version = "0.1.0"+description = "Ears for coding agents: describe, match, search, compare, time-align and critique sound files an agent cannot hear."+readme = "README.md"+requires-python = ">=3.10"+license = { text = "MIT" }+authors = [{ name = "Leon Lin" }]+keywords = ["audio", "sound", "ai-agents", "mcp", "claude-code", "clap", "audio-analysis", "sound-design",+ "video", "procedural-audio", "cpu"]+classifiers = [+ "Development Status :: 3 - Alpha",+ "Intended Audience :: Developers",+ "License :: OSI Approved :: MIT License",+ "Operating System :: OS Independent",+ "Programming Language :: Python :: 3",+ "Programming Language :: Python :: 3 :: Only",+ "Programming Language :: Python :: 3.10",+ "Programming Language :: Python :: 3.11",+ "Programming Language :: Python :: 3.12",+ "Topic :: Multimedia :: Sound/Audio :: Analysis",+ "Topic :: Multimedia :: Sound/Audio :: Sound Synthesis",+ "Topic :: Scientific/Engineering :: Artificial Intelligence",+]+dependencies = [+ "numpy>=1.26",+ "scipy>=1.11",+ "soundfile>=0.12",+ "soxr>=0.3",+ "librosa>=0.10",+ "pyloudnorm>=0.1",+ "scikit-learn>=1.3",+ "pillow>=10",+ "matplotlib>=3.8",+ "torch>=2.6", # CPU wheels: pip install torch --index-url https://download.pytorch.org/whl/cpu+ "transformers>=4.40",+]++[project.urls]+Homepage = "https://github.com/Leonxlnx/earshot"+Repository = "https://github.com/Leonxlnx/earshot"+Issues = "https://github.com/Leonxlnx/earshot/issues"++[project.optional-dependencies]+mcp = ["mcp>=2"]+llm = ["requests>=2.31"]+dev = ["pytest>=8"]++[project.scripts]+earshot = "earshot.cli:main"++[tool.setuptools.packages.find]+include = ["earshot*"]++[tool.setuptools.package-data]+earshot = ["data/*.json", "data/*.npz"]++[tool.pytest.ini_options]+testpaths = ["tests"]diff --git a/skills/earshot/SKILL.md b/skills/earshot/SKILL.mdnew file mode 100644index 0000000..a668234--- /dev/null+++ b/skills/earshot/SKILL.md@@ -0,0 +1,179 @@+---+name: earshot+description: Listen with earshot whenever you must judge, choose, trim, verify or compare sound you cannot hear - picking a sound effect, music or ambience from candidates, finding the cleanest moment in a recording, checking that audio is real and not code-synthesized, reviewing a soundtrack or a video's audio (what happens when, sync to picture, loudness, mix), A/B-ing a mix change, or making a sound in code and improving it by listening (earshot forge). Use it before telling anyone how audio sounds.+---++# earshot: ears for an agent++You cannot hear. Never say how audio sounds, or that a sound "fits", without an earshot reading behind it, and quote+the number when you report ("strong match, z +3.9; p_real 0.96; -14.2 LUFS"). earshot runs locally on CPU (audio+never leaves the machine, except with the optional `llm` command) and caches every embedding and result on disk.+Cost: each CLI call is a new process that loads the models first (~20-40 s on a busy laptop, even for a cached file);+inside the MCP server or one Python script the models load once and repeat calls take seconds. For more than a+handful of calls, use MCP or a short script (`from earshot import api; api.find(...); api.realness(...)`).++## Setup (once)+```+pip install -e "/path/to/earshot[mcp,llm]" # or the repo venv: /path/to/earshot/.venv/bin/python -m earshot ...+earshot doctor # packages, ffmpeg, cached models; --load also times the models+```+Two ways in, same functions:+- **CLI**: `earshot <command> ... [--json] [--out result.json]`. Text output starts with a plain summary; times are+ m:ss.ss in text and seconds in JSON.+- **MCP** (models stay loaded between calls): `claude mcp add earshot --scope user -e EARSHOT_THREADS=3 --+ /path/to/earshot/.venv/bin/python -m earshot mcp`, or in any client's `.mcp.json`:+ `{"mcpServers": {"earshot": {"command": "/path/to/earshot/.venv/bin/python", "args": ["-m", "earshot", "mcp"],+ "env": {"EARSHOT_THREADS": "3", "GEMINI_API_KEY": "${GEMINI_API_KEY:-}"}}}}` (the key line only for the optional+ llm tool; Claude Code fills it from the user's environment, other clients may need the key set in the environment+ that launches them: never write a literal key into a project file).+ Tools: measure, describe, match, rank, find, compare, ab, realness, timeline, sheet, critique, doctor,+ llm_critique. Pass absolute paths; `full=true` returns all JSON. The server answers within a few seconds, longer on a cold or+ busy machine; if Claude Code shows no earshot tools, start it with `MCP_TIMEOUT=60000` (PowerShell:+ `$env:MCP_TIMEOUT=60000; claude`). `earshot mcp --warm` preloads the models at startup.+If the machine is shared, set `EARSHOT_THREADS=2-3` (or `--threads 2` on any command). The first model call of a+process loads CLAP + AST (~20-60 s); later calls in the same process are fast. Any ffmpeg-readable file works,+including the audio of an mp4 (a video with no audio track says so). Errors are one line on stderr:+`earshot <command>: error: ...`, exit 1.++## Loop 1: choose an asset+1. `earshot rank "a soft felt mallet hit on a wooden block" downloads/sfx/ --top 5` - shortlist from a folder.+2. `earshot find winner.ogg "a single clean mallet hit" --top 3` - the best moments as sample trims (start / end in+ seconds). Each segment has a kind: `stroke` (one onset until it decays) and `event` (strokes that belong together,+ e.g. a key's down + up) start 5 ms before the attack and end after the decay; `window` is a stretch whose edges+ find snapped onto the sound for one-shot queries (a plain stretch, to be crossfaded, for beds). `head` / `tail` =+ how loud the first 5 ms / last 10 ms are against the trim's peak (-30 dB or lower = clean); a trim marked+ 'head cut' or 'tail cut' starts or ends inside a sound: do not use it as is. `--kind event` returns onset-based+ trims only (hits, clicks, steps), `--kind texture` plain windows only (beds); the default reads the query.+3. `earshot describe winner.ogg --start 1.12 --end 1.48` - what it actually sounds like; check the top tags fit what+ you need and the shape (attack / decay) fits the picture. AudioSet labels are weak on one-shots under ~0.5 s:+ 'tagger unsure' or a near class (Zipper for a swish, Finger snapping for a click) is neutral; only an outright+ different class (Speech, Music, Engine, Vehicle) is a red flag.+4. `earshot realness winner.ogg --start 1.12 --end 1.48` (or a whole folder) - judge the trim you will use: a+ multi-take file's p_real is a mean over every take and the silence between them. Reject 'likely code-synthesized';+ read the evidence of an 'uncertain' before using it, and write why you kept it. The verdict thresholds depend on+ length (each summary names them): a sound up to 6 s uses the clip thresholds (lo 0.17 / hi 0.77), a longer file+ the mean of its 4 s windows with weakly calibrated file thresholds (lo 0.40 / hi 0.77), so compare verdicts, not+ raw p_real, across files of different lengths. Real recordings and professionally produced music and sound design+ usually read real, but realness hears how a sound sounds, not where it came from: clean professionally synthesized+ tones (UI sounds, chimes, synth stabs) also read as code-synthesized. If realness is unavailable (model missing),+ say plainly that real-vs-synthetic was not checked; describe's AudioSet classes (Sine wave, Beep, Synthesizer) are+ weaker evidence. Exact digital zeros alone prove nothing (59% of the project's licensed real recordings have some+ edit or fade padding).+5. To choose between two candidates: `earshot ab a.ogg b.ogg --query "a crisp fast air whoosh"`, with+ `--a-start/--a-end/--b-start/--b-end` for trims from find (A and B may be the same multi-take file). Optional:+ `earshot match winner.ogg "a deep thocky key press" --alts "a clicky blue switch" "a typewriter key"` to test one+ claim, or `earshot compare winner.ogg --refs approved/ [--start --end]` to check it fits the palette.++## Loop 2: review a mix or a film+1. `earshot timeline film.mp4 --cues cues.json` (or `soundtrack.wav --video film.mp4`) - the listening transcript:+ events with time, level and tags, sections, loudness curve, picture cuts, sync per cut / cue. `--detail low` is+ fast, `high` tags up to 96 events (most important first; the summary says how many were tagged, so an untagged+ event is not a missing sound). Read the sync stats: offset_ms = sound minus picture (negative = early), and the+ 'chance' fraction says what random edit points would score; only better-than-chance sync means anything. A cut+ listed with no hit can still carry a soft whoosh: check it on the sheet before adding a sound.+2. `earshot sheet film.mp4 -o sheet.png --cues cues.json` (`--from 10 --to 25` zooms, `--detail low` is faster) -+ an annotated PNG: frames, sections, labelled events, spectrogram, loudness, beats, cue markers coloured by sync+ (green = within a frame, amber = within 100 ms, red = no sound). OPEN THE PNG with your image reader: gaps,+ pile-ups, masking and late hits are obvious at a glance.+3. `earshot critique film.mp4` (`--fast` skips the AudioSet / realness passes) - scorecard against the measured+ range of professional launch films: each metric ok / low / high with its weight and actions with timestamps, and+ a verdict (launch-grade / close / needs work / not launch-grade). Fix critical rows first, re-run 1-3 on the new+ render. Its "onset on visual events" share counts new attacks within +-100 ms only (the references' definition);+ its fixes and tips name the cuts with no sound at all, by the same matcher as timeline. If the file is one of the+ profile's own reference films, critique says so and scores it against the others.+4. Exact meters any time (LUFS, true peak dBTP, LRA, noise floor, attack / decay, brightness, bands):+ `earshot measure mix.wav [--music]` or the MCP `measure` tool.++## Loop 3: A/B a change+- `earshot ab before.wav after.wav --query "warm punchy kick"` or `--refs pro_refs/` - which is closer, and how B+ differs from A in measured words (louder, brighter, longer decay). For sounds inside longer files pass the trims+ (`--a-start 1.217 --a-end 1.615 --b-start 7.881 --b-end 8.285`): a whole-file A/B of multi-take files compares every+ take plus the silence between them. Then `earshot timeline after.wav` and compare the event lines and sync stats+ with the old run. Change one thing at a time.++## Loop 4: make a sound in code (earshot forge)+Use this when the user wants a sound **made**, not chosen: an AI that hears can write synthesis code, listen, and+improve it. Every sample must be computed (no recorded audio inside, not even an impulse response); real recordings+are only the references the critic compares with.+1. Write or pick a recipe in `earshot/forge/recipes/<name>.py`: `SPACE` (parameter ranges), `DEFAULTS`, `TARGET`,+ `ALTS` (realistic confusions) and `render(params, r) -> 48 kHz array`, built from `earshot.forge.dsp` (modal+ resonators, contact excitations, noise colours, a time-varying SVF, a computed room, a mic with a noise floor). Model+ the physics of the real thing, measured on the references: what strikes what, what rings, how long, in what room.+2. Run one round: `earshot forge <recipe> --refs <real refs> --budget 144 --reps 2 --tone 1.5 --keep 4 --robust 8+ --dir <round dir>`, or in one Python script per round that loads the models once:+ `forge(recipe, critic=Critic(target, alts, refs, tone=1.5), reps=2, space={...})`.+ - `--tone` puts the model-free tonal balance and tail into the objective.+ - `--reps 2` ranks each candidate by 2 renders, against lucky renders.+ - It takes about 3-5 minutes per 160 renders on a busy laptop CPU, including a 30-60 s model warm-up.+3. Read what it heard. Each pick's summary says, for example, "sounds like: a cork pop, a glass ting", "decay +2.4+ outside the refs" or "tone 11.2 dB from the refs' median: +16 dB at 500-630 Hz (above every ref)". Then check the+ picks four ways:+ - the model-free `tone` words: which bands fall outside every reference, the energy shares, the tail, the length+ (the most trustworthy signal once the model scores are high)+ - `robust`: the mean over new seeds, not the lucky best render+ - `alt_zs`: which confusion is closest+ - spectrograms next to the references: `ffmpeg -i x.wav -lavfi showspectrumpic=s=900x360:scale=log:legend=0 x.png`++ The AudioSet tags in report.json are held out only while you do not pick or steer by them. Fix a selection rule+ (for example, the robust mean) before you look, and report the tags afterwards.+4. Change the recipe's structure, not only its ranges, for what is wrong (a missing release tick, a metallic ring, a+ tail too long). Then repeat. Keep a LOG of each round's change and numbers.+5. Report honestly, next to the real references scored by the same critic (`report.json` ref_scores,+ leave-one-out).++Goodhart, say it plainly:+- The search optimises earshot's own models. Made sounds routinely out-score real recordings on them, which means the+ critic is satisfied, not that the sound is better.+- **p_real is easy to fool**: modal resonators plus a room and a mic noise floor read "real".+- **Typicality rewards being an average** of the references.+- **The model checks saturate**: after a search, every model check, held-out phrasings and independent real sets+ included, puts the made sound inside the real ones. Only measurements and ears can still tell them apart.+- Judge by measured closeness to the references: tonal balance, bands outside every reference, tails, lengths. Also+ use robust re-renders and spectrograms.+- Ask for a human blind test (random order, loudness-matched, hidden key) before calling a made sound "as good as+ real".++In the first test (demo/forge-test) an independent judge found the key press 8-13 dB off the real tonal balance and+the pops band-unlimited with long hissy tails, while every model check called them real. Re-made by steering with the+model-free check, they now sit inside the real recordings' measured spread. The whoosh was re-made as a truly fast+one. The impact (darker, swell-led) and the click (short tails) were not re-made. No human has listened yet.++## Optional: a second opinion in words (audio LLM)+`earshot llm mix.wav --yes-upload [--with-timeline] [--prompt "Is the riser at 0:12 cheap?"]` asks Google Gemini for+a rubric critique (sync, realism, mix, loudness, what sounds cheap, fixes with times). It only runs when the user has+set GEMINI_API_KEY (or GOOGLE_API_KEY) AND explicitly agreed that the audio is uploaded to Google; ask first, never+pass --yes-upload / consent=true on your own. Only audio is sent (mono 64 kbps). Its timestamps are approximate and+its scores are one model's opinion: confirm every fix with timeline / ab.++## Reading the numbers+| number | what it is | trust |+|---|---|---|+| LUFS, dBTP, LRA, dBFS, ms, Hz | measurements (BS.1770 meter, spectrum, envelope) | exact; use them for levels |+| z (match / rank / find / describe tags) | how far a description stands out among the few hundred vocabulary descriptions scored on the same sound | verdicts: z >= 3 strong, 2.5-3 plausible, 2-2.5 weak, < 2 no match (held-out evidence per band in eval/semantics/RESULTS.md) |+| p_vs_alts | uncalibrated, sharp forced choice between your query and your alternatives | only says which wording fits best; judge by z |+| cosine | raw CLAP similarity | never a probability; compare only for the same sound and query |+| AudioSet labels (0..1) | independent per-class probabilities from the AST tagger | good for broad classes; hears only up to 8 kHz |+| p_real / verdict | real or pro-produced vs code-synthesized, from a small fitted model | strong evidence, not proof; read its evidence list; thresholds differ for clips (<= 6 s) and long files (named in each summary) |+| digital_silence_frac | share of exact-zero samples | weak alone: edit / fade padding is common in real recordings |+| typicality z (compare) | distance to the reference set vs the refs' own spread | >= -1 fits the set, < -2 outlier |+| sync offset_ms, chance | sound minus picture per cut / cue; what random edits would score | trust when well above chance |+| critique status | value vs the range of 7 professional launch films (loudness: web-delivery practice; true peak <= -1 dBTP for a master, <= 0 dBTP for decoded AAC / MP3, which already holds the codec overshoot) | a range, not a target: ok = inside what pros do |+Prefer measurements, then verdicts / z / probabilities; never present cosine as confidence.++## Writing queries+Describe the sound, not its use: source + material + action + character ("a heavy wooden door slamming in a small+room", not "a sound for the logo"). Give alternatives that are realistic confusions (thock vs click vs typewriter).+When a z is borderline, try two or three phrasings: trust a reading that holds up under all of them.++## Limits (say these plainly when they matter)+- Taste is not measured: earshot says what a sound is and how it measures, not whether it is beautiful or right for+ the story. The user decides on taste; give them the evidence and the shortlist.+- No speech recognition (it can tell there is speech, not what is said) and no lyric or language judgement.+- CLAP hears 50 Hz - 14 kHz and AST up to 8 kHz: check "air" and low-pass problems with measure (bandwidth_hz, bands).+- Very short sounds are padded to a 10 s window; isolated hits read best with find / describe on a trimmed segment.+- attack_ms / decay_ms come from smoothed envelopes (5 ms / 10 ms RMS): an instant attack reads ~4 ms, and very+ short clicks cannot read below ~10-20 ms of decay.+- Picture cuts come from a frame-difference detector: hard cuts and whips are reliable, slow crossfades are missed+ or found late, and motion inside a shot is not an event, so pass cue times (--cues) for animation.+- Mixed, dense material blurs tags: describe a stem or a segment rather than the whole mix.+- Licence, provenance and attribution are not audio questions: earshot cannot check them.diff --git a/skills/real-sound/SKILL.md b/skills/real-sound/SKILL.mdnew file mode 100644index 0000000..339c21d--- /dev/null+++ b/skills/real-sound/SKILL.md@@ -0,0 +1,198 @@+---+name: real-sound+description: Build premium, realistic soundtracks for code-made videos and animations from real licensed recordings - launch films, product and feature videos, UI walkthroughs, logo stings, explainers, trailers and animated shorts made with Remotion, HyperFrames, three.js, GSAP or any renderer. Use this whenever a video or animation needs sound design, SFX, foley, ambience, music, a soundtrack, a score, UI click / whoosh / riser / impact / logo-sting sounds, or a music edit to picture; whenever a synthesized or "random" sounding soundtrack must be replaced with real recordings; or when the user asks to source, license, credit, mix, master, critique or mux audio for a film. It covers cue lists from the picture, sourcing from Freesound / Mixkit / composer libraries with licence checks, a music edit at bar lines, a JSON sound design rendered by a deterministic mixer, measured QA (clicks, masking, harshness, sync, loudness) plus earshot listening, a multi-agent critique, mastering, credits and the mux.+---++# real-sound++You are the sound supervisor of a film you cannot hear. The soundtrack is built ONLY from real, professionally+recorded or produced audio (one licensed music track, real foley and SFX, real ambiences), placed on the picture's+cue frames by a deterministic mixer, measured instead of listened to, critiqued through five lenses, fixed by one+agent, mastered, credited and muxed. Code selects, trims, places, levels, pans, EQs, pitches by resampling,+time-stretches the music and masters; code never generates a sound.++The bar is a professional launch film: every sound caused by something on screen, one material palette, the music's+drop on the hero reveal, restraint everywhere else, a resolved logo, a tail into silence. Rules:+`references/sound-design.md` (and the earshot repo's `docs/launch-video-sound.md` for the user's reference profile).++`<skill>` below is this folder. **Python:** `{{PY}}` below (and in every prompt template) is ONE interpreter that has+numpy, scipy, soundfile and the earshot package: the earshot repo's venv+(`<earshot repo>/.venv/Scripts/python.exe` on Windows, `<earshot repo>/.venv/bin/python` elsewhere), or+any Python where `<python> -c "import earshot"` works. Bare `python` on Windows is often the Microsoft Store stub and+fails. Substitute the full path for `{{PY}}` when you run a command or hand a prompt to a sub-agent (the other+placeholders are listed in `references/process.md`). Run the scripts with `-I` (isolated: downloaded files can never+shadow a module); every script except `scripts/licences.py` (the licence policy module that audit.py and credits.py+import) prints `--help`.++## The process++| step | who | output | gate |+|---|---|---|---|+| 0. cue list | lead | `out/cues.json` | sync frames computed from the picture code |+| 1. brief | lead | `audio/SOUND_BRIEF.md` | every cue has a sound need (or "nothing") |+| 2. sourcing (per category, parallel) | sourcing agents | `audio/assets/<cat>/`, manifests | each file measured + earshot `realness` "real or pro-produced" on the trim used (an "uncertain" only with its evidence and a trim-level re-check written in the manifest) |+| 3. licence audit | auditor | `audit.json` | `audit.py` exit 0 for what the film uses |+| 4. music edit | music supervisor | `audio/design/music.json` | grid verified, drop on the reveal, splices clean |+| 5. design fragments (per category, parallel) | designers | `audio/design/<cat>.json` | solo render + qa clean |+| 6. integrate | lead | soundtrack, stems, report, trace | master gate (+-0.5 LU, <= -1.5 dBTP) |+| 7. QA + listening | lead | `out/qa/qa.json`, earshot timeline / sheet / critique | no qa fail |+| 8. critique (5 lenses, parallel, read-only) | critics | ranked issues + tested operations | |+| 9. fix | ONE fixer | the design | qa: no regression |+| 10. master, credits, mux | lead | `CREDITS.md`, description text, the film | credits exit 0, lag 0 |++The detail of every step, the roles and what each critic measures: `references/process.md`. Every sub-agent prompt+is a ready-to-fill brief in `templates/prompts/`.++### 0. Cue list from the picture+Every sound sits on a frame the picture code computes: a contact frame, a pop-in's 50 % frame, a move's velocity+peak, a riser's hit, the frame the wordmark becomes readable. Cinetic films already have `out/cues.json`+(`export-cues.ts`); for anything else write the same format (`templates/cues.json`) from the timeline constants.+Stable ids (`act:thing:n`) let rules match classes of action with a regex.++### 1. Brief+Copy `templates/SOUND_BRIEF.md` to `audio/SOUND_BRIEF.md`: concept (palette, the music's arc, places), the picture map+(every cue: what is on screen, the sound need), ambience zones, allowed licences, quality bar, file layout. It is the+contract every sub-agent gets verbatim. Pick the brand's personality row (`references/sound-design.md` §3).++### 2. Sourcing, per category (`templates/prompts/sourcing.md`)+Typical launch-film categories: `ui` (clicks, taps, toggles, pops), `keyboard`, `motion` (whooshes, risers),+`impacts` (reveal hits, the logo), `ambience`, `music`; a character film adds `foley`, its worlds, its objects.+Freesound (CC0 / CC BY; HQ preview `cdn.freesound.org/previews/<id//1000>/<id>_<user>-hq.ogg`), Mixkit (WAV+`assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`), composer libraries for music; licences read on each item's own+page (`references/licences.md`); polite sequential downloads; every file measured:+```+{{PY}} -I <skill>/scripts/analyze.py audio/assets/ui/* --png out/audio-review/ui --out out/audio-review/ui.json+{{PY}} -m earshot realness audio/assets/ui/ # gate: "real or pro-produced" (re-check trims: --start --end)+{{PY}} -m earshot rank "a single crisp trackpad click" audio/assets/ui/ --top 5+{{PY}} -m earshot match <file> "a soft cork pop" --alts "a gunshot" "a balloon burst"+{{PY}} -m earshot find <long take> "a single mechanical key press" --top 8 # trims; skip any marked head cut / tail cut+```++### 3. Licence audit (`templates/prompts/licence_audit.md`)+The auditor records what each page says (`licence_seen`, `licence_confirmed`) in+`audio/assets/manifests/audit.json`; the script checks the rest (classification, ids vs URLs, files decode, durations,+credit lines):+```+{{PY}} -I <skill>/scripts/audit.py --root . --report out/qa/mix_report.json [--fetch]+```+Allowed: CC0, public domain, CC BY 2.0-4.0, Mixkit free licences, Pixabay. Forbidden: NC, ND, SA, Sampling+, personal,+custom, disputed. Unverified = blocked.++### 4. Music (`templates/prompts/music_supervisor.md`)+One licensed track whose structure fits the film. Write the arc per film bar (`templates/arc.json`), then:+```+{{PY}} -I <skill>/scripts/arrange.py audio/assets/music/<track> --film-bpm 120 --arc audio/arc.json --out out/audio-review/music/arrange.json+```+It fits the tempo, moves the grid onto the measured attacks (beat trackers lag 20-50 ms), proposes the downbeat+phase, anchors the drop film bar on the strongest drop candidate and drafts segments at bar lines (a ready-to-paste+`music` block). The downbeat is the weak spot: when it prints LOW confidence (on one of the two tracks it was tested on+it was half a bar off), decide between the two printed grids from the first note, the section entrances and the chord+changes in the bar table, and re-run with `--phase N`. Then edit by judgement: harmony continues across each splice,+the drop's downbeat on the hero frame, the final chord on the logo, `tail_fade` into the end.++### 5. Design fragments, per category (`templates/prompts/designer.md`)+`audio/design/<category>.json` with samples (trims), slices (cut a take of many hits into single hits), pools, rules+on cue events, free placements, ambience beds, the music block. Reference with every key and real examples:+`references/design-format.md`; a complete launch-video example: `templates/design_fragment.json` (its file names are+placeholders such as `mk-0000_mouse-click.wav`: it renders once each one is replaced by a real recording from your+manifests; mix.py stops and lists the missing files until then).+Verify alone:+```+{{PY}} -I <skill>/scripts/mix.py --only ui --dump out/audio-review/ui/dump # every trim as WAV + numbers+{{PY}} -I <skill>/scripts/mix.py --only ui --trace out/audio-review/ui/adds.json --out out/audio-review/ui/solo.wav+{{PY}} -I <skill>/scripts/qa.py --only ui --report out/audio-review/ui/solo_report.json --stems out/audio-review/ui/solo_stems --trace out/audio-review/ui/adds.json --out out/audio-review/ui/qa.json+{{PY}} -m earshot timeline out/audio-review/ui/solo.wav --cues out/cues.json+```++### 6-7. Integrate, QA, listen+```+{{PY}} -I <skill>/scripts/mix.py --root . --design audio/sounddesign.json --cues out/cues.json --trace out/qa/adds.json+{{PY}} -I <skill>/scripts/qa.py --layers # clicks, edges, audibility, harshness, loudness, coverage, sync, density+{{PY}} -I <skill>/scripts/qa.py --listen --realness --video out/film.mp4 # + earshot timeline, sheet, critique, realness+{{PY}} -m earshot critique out/audio/soundtrack.wav --video out/film.mp4 # scorecard vs professional launch films+{{PY}} -m earshot sheet out/audio/soundtrack.wav -o out/qa/sheet.png --video out/film.mp4 --cues out/qa/cues_sounded.json+```+`qa.py` ties every finding to the rule and sample that caused it (the trace). It is usable without the models;+`--listen` adds earshot's transcript (`out/qa/qa_timeline.txt`), annotated sheet and scorecard.++### 8-9. Critique and fix+Five read-only critics in parallel, each with its prompt: `critic_sync.md`, `critic_edges.md`, `critic_masking.md`,+`critic_tonal.md`, `critic_director.md`. Each tests its fixes as a variant in its own folder:+```+{{PY}} -I <skill>/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json --out out/audio-review/critic_tonal/v1 --render --qa+```+and returns ranked issues with exact operations and before -> after numbers. Then ONE fixer (`fixer.md`) merges the+operations, resolves conflicts, checks the merged variant against the previous qa.json, and applies it+(`variant.py --apply`, originals backed up). Acknowledge confirmed deliberate exceptions with `"qa_ok": "<reason>"`.+At least one round; a second when levels or the music edit changed.++### 10. Master, credits, mux+`mix.py` masters with `master.py` (-14 LUFS, true-peak limiter, ceiling 0.77 linear; gate +-0.5 LU and <= -1.5 dBTP).+```+{{PY}} -I <skill>/scripts/audit.py --root . --report out/qa/mix_report.json # exit 0+{{PY}} -I <skill>/scripts/credits.py --root . --design audio/sounddesign.json --audit audio/assets/manifests/audit.json+node <skill>/scripts/mux.mjs --video out/film.mp4 --audio out/audio/soundtrack.wav --out out/film_sound.mp4+{{PY}} -m earshot timeline out/film_sound.mp4 --cues out/qa/cues_sounded.json # the finished file, heard+```+`CREDITS.md` + `out/CREDITS_description.txt` (paste into the video description: CC BY credits are required there).+`mux.mjs` stream-copies the picture and cuts the AAC audio to the picture's duration.++## Launch videos: the sound map++| picture | sound | sits on |+|---|---|---|+| cursor moves, hovers, scrolls, progress, streaming text | nothing (the music carries it) | |+| click / tap | ONE real click voice for the film, press + release ~85 ms later, same level every time | the press frame |+| typing | one real keyboard, sliced; <= ~12 keys/s, one per word above ~25 chars/s; music dips 6-10 dB | each sounded character |+| a card / chip / badge appears | a soft real pop or tick, climbing across a run | 50 % of its travel |+| toggle, lock-in, a counter's final value | a real latch / click / crisp tick | the contact / lock frame |+| camera-scale move, whip, zoom (~2 per film) | a real swish, apex on the velocity peak | velocity peak |+| hard cuts | nothing | |+| tension into the reveal | the music's build, or a real riser ending 30-250 ms before the hit | the hit |+| hero reveal | the music's DROP (edit the track so its downbeat lands here) + optionally one real impact | the frame |+| logo / lockup | music stops on a bar line or filters down, 0.2-0.7 s near-silence, ONE soft note in key, tail to silence | the frame the wordmark reads |++Levels: -14 LUFS, true peak <= -1 dBTP after AAC, LRA 5-9 LU, the loudest moment on the drop (30-60 % in), the logo+8-20 dB under it, about 5 designed effects per 10 s, effects dry and centred. Measured norms and the cheap tells:+`references/sound-design.md`.++## Tools++| need | tool |+|---|---|+| measure recordings (format, floor, bandwidth, onsets, spikes, tempo / key) + spectrogram PNGs | `scripts/analyze.py` |+| hear recordings: real vs synthesized, what it sounds like, best match, best take inside a long file | `{{PY}} -m earshot realness / describe / match / rank / find / compare / ab` |+| levels / brightness / onsets of one file or a stretch of it (`--start --end`) | `{{PY}} -m earshot measure` (Python: `from earshot import api; api.measure(...)`, likewise `api.find`, `api.realness` ...) |+| licence + file audit | `scripts/audit.py` (policy in `scripts/licences.py`) |+| music grid, downbeats, bar-line edit draft | `scripts/arrange.py` |+| render the design (+ solo `--only`, `--dump` trims, `--trace` for QA) | `scripts/mix.py` |+| master and loudness measurement | `scripts/master.py` (`--measure`) |+| objective QA tied to placements | `scripts/qa.py` (`--layers`, `--listen`, `--realness`) |+| critic variants, the fixer's apply | `scripts/variant.py` |+| credits for CREDITS.md and the video description | `scripts/credits.py` |+| put the soundtrack on the picture | `scripts/mux.mjs` |+| listen to a whole mix / film; annotated picture; launch-film scorecard | `{{PY}} -m earshot timeline / sheet / critique` |++## Files++```+audio/SOUND_BRIEF.md the contract (templates/SOUND_BRIEF.md)+audio/assets/<category>/ downloaded recordings (<source>-<id>_<slug>.<ext>; never commit or publish them)+audio/assets/manifests/<cat>.json one entry per recording (templates/manifest_entry.json)+audio/assets/manifests/audit.json licence audit (templates/audit_entry.json)+audio/sounddesign.json top level: includes, buses, master (templates/sounddesign.json)+audio/design/<category>.json one fragment per designer (templates/design_fragment.json)+out/cues.json the picture's cue events (templates/cues.json)+out/audio/soundtrack.wav, stems/ the master and its stems+out/qa/ mix_report.json, adds.json, cues_sounded.json, master.json, qa.json+out/audio-review/ per-category and per-critic work, variants, dumps (the audit trail)+CREDITS.md, out/CREDITS_description.txt+```++## References+- `references/process.md`: roles, steps, what each critic measures, thresholds and their sources, variants, rounds.+- `references/design-format.md`: every design key, defaults, processing order, outputs, real examples.+- `references/sound-design.md`: launch-video and film sound rules, levels, sync, variation, tuning, selection, cheap tells.+- `references/licences.md`: allowed / forbidden, verification per site, attribution, Content ID.+- `references/troubleshooting.md`: errors and fixes.+- Cinetic films: `docs/INTEGRATION.md` in the earshot repo (how the cue file, timeline and review loop connect).diff --git a/skills/real-sound/references/design-format.md b/skills/real-sound/references/design-format.mdnew file mode 100644index 0000000..c2c6ddd--- /dev/null+++ b/skills/real-sound/references/design-format.md@@ -0,0 +1,251 @@+# Design format: the JSON that mix.py renders++The soundtrack is data. One top-level design (`audio/sounddesign.json`) includes one fragment per category+(`audio/design/<category>.json`); `mix.py` places real recordings on the picture's cue events, mixes three buses (sfx,+amb, music) and masters. Nothing is synthesized: the only processing is trim, fade, gain, pan, EQ, resampling (pitch),+a gain envelope, Rubber Band tempo stretch of the music, summing and mastering.++Templates: `templates/sounddesign.json` (top level), `templates/design_fragment.json` (every feature; its file names+are placeholders, so it renders with `templates/cues.json` once each one is replaced by a real recording from your+manifests), `templates/cues.json` (the cue file). `python` below = the interpreter `{{PY}}` described in SKILL.md.++```+python -I <skill>/scripts/mix.py --root . --design audio/sounddesign.json --cues out/cues.json --trace out/qa/adds.json+python -I <skill>/scripts/mix.py --only keyboard # solo render of the fragments whose path contains "keyboard"+python -I <skill>/scripts/mix.py --only keyboard --dump out/audio-review/keyboard/dump # every sample as a WAV+```++Every relative path (design, cues, outputs, each sample's `file`) resolves against `--root`. Keys starting with `_` are+comments (write the reason for every non-obvious number there: `_doc`, `_about`, `_buses_doc`); `mix.py` ignores+unknown keys everywhere except inside `buses` (each key there must be `sfx`, `amb` or `music`).++## Contents+1. Top level and includes+2. The cue file+3. samples+4. slices+5. pools+6. rules (cue-driven placements)+7. place (free placements)+8. ambience+9. music+10. buses and master+11. What mix.py writes+12. qa_ok: acknowledging a deliberate exception+13. Worked examples from a shipped film++## 1. Top level and includes++| key | meaning |+|---|---|+| `duration` | film length in seconds (default: cue `total` / `fps`) |+| `cues` | default cue file (the `--cues` flag wins; default `out/cues.json`) |+| `include` | fragment paths; their `samples`, `slices`, `pools` are merged (a name defined twice is an error), `rules`, `place`, `ambience` are concatenated, `music` is taken when the top level has none |+| `samples`, `slices`, `pools`, `rules`, `place`, `ambience`, `music` | as below (allowed at the top level too) |+| `buses`, `master` | read from the top-level file only |++Name everything with the fragment's prefix (`kb.`, `ui.`, `mo.`, `am.`, `mu.`): fragments are built by different agents+in parallel and must never collide. Variation (cycle order, jitter, random pitch) is keyed on rule names, so a solo+render (`--only`) plays exactly the same sounds as the full mix.++## 2. The cue file++`{fps, total, events: [{f, kind, id?, weight?, pan?, variant?, dur?, apexFrac?, pitch?}], acts?, cue?}` or a list whose+first element is that object. This is Cinetic's `out/cues.json` format (`scripts/export-cues.ts`), so a Cinetic film+plugs in directly (`docs/INTEGRATION.md`). `f` is the absolute frame the sound belongs on (fractional is fine).+Compute cue frames from the picture code (contact frames, 50 % pop frames, velocity peaks); typed-in sync points land+8-11 frames off in practice. A design with no rules needs no cue file.++## 3. samples++`name -> {file, start?, end?, gain_db?, norm_peak_db?, mono?, fade_in?, fade_out?, eq?, pitch?, glide?, env?}`:+a trimmed piece of a recording.++| key | default | meaning |+|---|---|---|+| `file` | required | the recording (any format ffmpeg reads; decoded to 48 kHz float stereo, cached) |+| `start`, `end` | 0, end of file | the trim in source seconds. Start 3-5 ms before the attack; end after the decay, before the next event |+| `mono` | false | fold to mono (point sources: keys, clicks, steps). Keep stereo for whooshes, beds, wide impacts |+| `eq` | none | `{hp, hp_order (2), lp, lp_order (2), peq: [[f0, q, gain_db], ...], lowshelf: [f0, gain_db], highshelf: [f0, gain_db]}` (Butterworth HP/LP, RBJ biquads) |+| `pitch` | 0 | semitones by resampling (tape-style: pitch AND length change together). +2..+6 makes a hand-sized sound miniature; keep tuned material within +-2 |+| `glide` | none | `[from, to]` semitones: a varispeed glide across the sample (band-limited, no aliasing) |+| `env` | none | `[[t_from_sample_start, dB], ...]` piecewise-linear gain inside the sample (shape a riser, duck a tail) |+| `norm_peak_db` | none | normalise the processed sample's peak to this level (makes a pool even before rule gains) |+| `fade_in`, `fade_out` | 0.002, 0.012 | raised-cosine fades in seconds, applied after normalisation |+| `gain_db` | 0 | static gain after everything |++Processing order: mono -> eq -> pitch -> glide -> env -> norm_peak_db -> fades -> gain_db. `reverse` is refused (a+reversed recording is still real, but the design keeps to what the recording is).++## 4. slices++`name -> {file, start?, end?, delta?, min_gap?, gate_db?, pre?, len?, stop_before_next?, pick?, max?, ...sample opts}`:+cuts a recording of many hits (typing, clicks, steps, dominoes) into single hits at its detected onsets. The slices are+named `name#0`, `name#1` ... in the order they are kept.++| key | default | meaning |+|---|---|---|+| `delta` | 0.07 | onset detector sensitivity (librosa, hop 128, backtracked); lower finds softer hits |+| `min_gap` | 0.06 | seconds between onsets |+| `gate_db` | -18 | drop onsets whose first 30 ms peak is this far below the loudest (handling noise, far keys) |+| `pre` | 0.004 | seconds kept before each onset |+| `len` | 0.16 | maximum slice length |+| `stop_before_next` | true | end each slice `pre` before the next onset (never two hits in one slice) |+| `pick` | all | list of onset indices to keep, in this order (after `pick`, `name#0` is the first picked) |+| `max` | all | keep at most this many |+| sample opts | | `mono`, `eq`, `pitch`, `glide`, `env`, `norm_peak_db`, `gain_db`, `fade_in` (0.002), `fade_out` (0.02) |++Always `--dump` a slice group and read `_samples.json` (duration, peak, attack, head/tail level per slice) before+using it; qa.py's edges section flags slices that start mid-sound or are cut mid-decay.++## 5. pools++`name -> [sample names]`; `"group#*"` expands to every slice of a slice group (sorted by index). An unknown name or an+empty pool is an error. A rule plays from a pool: one name = always that sound; several = variation.++## 6. rules (cue-driven placements)++`[{name, match, pool, mode?, gain_db?, weight_db?, weight_ref?, jitter_db?, pitch?, pitch_ramp?, gain_ramp?, pan?,+pan_scale?, align?, offset?, bus?, no_cue?, cycle_start?}]`. Every cue event that matches a rule gets a sound from the+rule's pool; several rules matching one event LAYER it (a key: thock + bottom-out + tiny motif note).++| key | default | meaning |+|---|---|---|+| `name` | rule index | stable name: the hash key for jitter / pitch / hash picks (renaming a rule changes its variation) |+| `match` | {} | `kind` (regex, full match), `id` (regex, search), `from` / `to` (film seconds, `[from, to)`), `variant` (exact) |+| `pool` | required | the pool to play |+| `mode` | cycle | `cycle`: round-robin, never the same sound twice in a row (`cycle_start` = first index); `hash`: picked per event by a hash of id + frame + rule name; `fixed`: always the first |+| `gain_db` | 0 | level |+| `weight_db` | 0 | dB per doubling of the event's `weight` vs `weight_ref` (default 1.0): bigger events play louder |+| `jitter_db` | 0 | +- random level per event (hashed: deterministic) |+| `pitch` | [0, 0] | `[lo, hi]` random semitones per event (hashed) |+| `pitch_ramp`, `gain_ramp` | none | `[a, b]` added across the matched events in time order: a climbing build, a decrescendo |+| `pan` | "cue" | `"cue"` = the event's `pan` x `pan_scale`, or a number -1..1 |+| `align` | attack | which point of the sound lands on the cue: `attack` (first ms within 20 dB of the peak; impacts, clicks, keys), `peak` (5 ms envelope maximum; whooshes), `end` (risers into a hit), `start` |+| `offset` | 0 | seconds added to the cue time (a release 85 ms after a press; a tail that trails the move) |+| `bus` | sfx | `sfx`, `amb` or `music` |+| `no_cue` | false | the placement does not mark the event as sounded (sweeteners; the A/V audit ignores it) |++Cue events no rule sounds are listed as `unsounded_events` in the report.++## 7. place (free placements)++`[{sample, at, gain_db?, pan?, align?, offset?, pitch?, bus?}]`: sounds with no cue event (a crowd bed, a lift before+a logo, a one-off sweetener). `at` is film seconds; `align` as in rules.++## 8. ambience++`[{name?, file, src_start?, src_end?, at, until, fade_in?, fade_out?, gain_db?, eq?, width?, auto?}]`: beds on the amb+bus. `src_start..src_end` is the clean window of the recording (no voices, dogs, bumps); if the bed needs more than the+window it is looped with a 1.5 s equal-power crossfade (the report says `looped`). `fade_in` / `fade_out` default 1 s.+`width` scales the side signal (0 = mono, 1 = as recorded, 1.15 = a little wider). `auto` = `[[film t, dB], ...]` rides+inside the bed (duck birds under a run, open the room on a cut).++## 9. music++`{file, src_bpm, target_bpm?, stretch?, gain_db?, eq?, segments: [{src_from, src_to, at, fade_in?, fade_out?,+fade_in_at_start?, tail_fade?, gain_db?, label?}]}`++| key | default | meaning |+|---|---|---|+| `src_bpm` | required | the track's measured tempo (arrange.py) |+| `target_bpm` | 120 | the picture's tempo |+| `stretch` | rubberband | `none` plays the source untouched (use it when the track is within ~0.2 % of the film tempo); otherwise the whole file is time-stretched by target/src with Rubber Band (keep the ratio within about +-6 %) |+| `segments[].src_from`, `src_to` | | ORIGINAL source seconds (before any stretch), on the track's downbeats |+| `segments[].at` | | film seconds where `src_from` lands; nudge by a few ms so the measured downbeat transient sits on the picture's downbeat |+| `fade_in`, `fade_out` | 0.008, 0.06 | the segment starts `fade_in` before `src_from` (a pre-roll, so the downbeat's own attack is kept) and fades out over `fade_out` after `src_to` |+| `fade_in_at_start` | true | false = no fade at the head (a hard cut on purpose) |+| `tail_fade` | none | seconds of a longer musical fade over the end of the segment (the last chord rings out) |+| `gain_db`, `label` | | per-segment level; a label that says which bars and why |++Edit at bar lines: a splice from bar n's end to bar m's start works when the harmony continues (V -> I, the same+chord) and the energy of the next bar is what the picture wants. `arrange.py` drafts the segment list; check every+splice (qa.py edges section, earshot timeline) and write the bars and the reason in each `label`.++## 10. buses and master++`buses: {sfx|amb|music: {gain_db?, eq?, auto?}}`: after everything is placed, each bus gets eq, then the `auto` ride+(`[[film t, dB], ...]`, piecewise linear), then `gain_db` (the music bus first gets `music.eq` and `music.gain_db`).+Rides shape the arc: the music dips under typing, lifts into the drop, settles under the logo. Put each step on a+musical splice or under a transient, never in the middle of a sustained note.++`master: {lufs?, ceiling?, no_glue?, no_comp?, mono_hz?}` (defaults -14, 0.77, true, false, 120): `master.py` high-passes+at 28 Hz, folds below `mono_hz` to mono, compresses gently (2:1 above -14 dBFS, unless `no_comp`), then runs loudness+passes with a 4x-oversampled true-peak limiter to `ceiling` (linear), a 3 ms fade-in at sample 0, a fade onto the last+sample and 480 zero samples at the end. The gate passes at |LUFS - target| <= 0.5 and true peak <= -1.5 dBTP.++## 11. What mix.py writes++- `--out` (default `out/audio/soundtrack.wav`): the master, 48 kHz 24-bit; the unmastered sum `premaster.wav` next to+ it. With `--no-master` or `--only`, `--out` is the unmastered sum itself (solo default+ `out/audio-review/solo/<name>.wav`).+- `--stems` (default `out/audio/stems`): `sfx.wav`, `amb.wav`, `music.wav` (pre-master, they sum to the premaster).+- `--report` (default `out/qa/mix_report.json`): `placements` (t, event, kind, rule, sample, gain_db, pitch, pan,+ align), `unsounded_events`, `buses` (peak / RMS), `ambience` (looped, body level), `music_segments`, `files_used`+ (every recording that is audible: the input of `credits.py` and `audit.py --report`), `premaster_peak_dbfs`;+ `master.json` (loudness gate) next to it.+- `--cues-out` (default `out/qa/cues_sounded.json`): the cue events that received an sfx-bus sound (for an A/V audit).+- `--trace FILE`: every `Mix.add()` (bus, start sample, length, gain, pan, sample / name / label, rule, kind): qa.py+ rebuilds every placement from it to attribute clicks, masking and harshness to a rule and a sample.+- `--dump DIR`: every sample and slice as WAV + `_samples.json` (duration, peak, RMS, attack, head/tail level), then exits.++## 12. qa_ok: acknowledging a deliberate exception++A sample, rule, free placement or music segment may carry `"qa_ok": "reason"` (mix.py ignores it). qa.py then lists+that item's edge / sync / audibility / harshness finding as acknowledged instead of a warning: e.g. a riser choked+25 ms before a pop on purpose, a chime that rings past its cue, a hard music cut into a quiet breakdown. Use it only for+decisions a critic has already confirmed, and say why.++## 13. Worked examples from a shipped film++From the Clawd film (60 s, 264 placements, real recordings only), trimmed for reading.++A key thock from a recording of single presses, shaped for a deep "thocky" board:+```json+"kb.thock.1": {"file": "audio/assets/keyboard/fs-789628_keychron-linear-individual-keys.ogg", "start": 1.168,+ "end": 1.598, "mono": true, "eq": {"hp": 115, "hp_order": 6, "peq": [[190, 0.8, 3.0], [380, 0.9, 3.0],+ [3300, 1.3, -1.5]], "highshelf": [5000, -4.5]}, "norm_peak_db": -10.0, "fade_in": 0.004, "fade_out": 0.16}+```++Typing on 16th notes: one key per typed pixel, climbing a semitone across the run, never the same key twice:+```json+{"name": "kb.type", "match": {"kind": "pop", "id": "^genesis:type"}, "pool": "kb.type", "mode": "cycle",+ "jitter_db": 1.5, "pitch": [1.0, 2.0], "pitch_ramp": [0.0, 1.0], "pan": 0.0, "align": "attack"}+```++Dominoes sliced into single clacks, 13 picked after measuring every slice (--dump):+```json+"ty.dom": {"file": "audio/assets/toy/fs-329098_domino-pieces-hit.ogg", "start": 0.2, "end": 16.4, "pre": 0.004,+ "len": 0.3, "min_gap": 0.2, "gate_db": -12, "pick": [2, 10, 11, 3, 17, 22, 23, 21, 24, 9, 16, 19, 4], "mono": true,+ "norm_peak_db": -3, "fade_out": 0.04, "eq": {"hp": 250, "hp_order": 4}}+```++A stacking flurry that ends ON each row's lock (aligned by its end):+```json+{"name": "ty.row.flurry", "match": {"kind": "pop", "id": "^swarm:row:"}, "pool": "ty.p.row.flurry", "mode": "cycle",+ "gain_db": -20.5, "gain_ramp": [-3, 2], "pitch_ramp": [0, 4], "jitter_db": 1.5, "pan": 0.12, "align": "end", "offset": 0.004}+```++Birdsong ducked under the hero's run (an `auto` ride inside the bed):+```json+{"name": "am.meadow_birds", "file": "audio/assets/ambience/fs-677535_woodlark-field-spring.ogg", "src_start": 15.5,+ "at": 28.15, "until": 52.2, "fade_in": 1.05, "fade_out": 0.8, "gain_db": 2.4, "width": 0.38,+ "eq": {"hp": 300, "lp": 13000, "lowshelf": [700, -5.0]},+ "auto": [[28.15, 0.0], [31.7, 0.0], [32.15, -2.5], [32.32, -10.0], [32.78, -10.0], [32.95, -2.5]]}+```++A music edit at bar lines (a 120 BPM track, no stretch), each splice explained:+```json+"segments": [+ {"label": "open bars 1-6 (E -> B), film = src - 0.5", "src_from": 0.5, "src_to": 12.5, "at": 0.0, "fade_in": 0.01, "fade_out": 0.05},+ {"label": "theme bars 17-24 (entrance on E: V -> I across the splice)", "src_from": 32.5, "src_to": 48.455, "at": 11.995, "fade_in": 0.01, "fade_out": 0.045},+ {"label": "drop bars 67-72 (E-major tutti) on the hero reveal", "src_from": 132.489, "src_to": 143.465, "at": 39.995, "fade_in": 0.008, "fade_out": 0.01},+ {"label": "coda bars 83-87 (final E on 59.0)", "src_from": 164.489, "src_to": 173.498, "at": 50.981, "fade_in": 0.01, "fade_out": 0.01, "tail_fade": 0.7}+]+```++Bus rides with their reasons (the drop hits, then a diminuendo lets the story read over the tutti):+```json+"_buses_doc": "the opening is lifted (+4.5 music/sfx) so it is soft but present; the DROP hits at -1.5 for its first beat, then -3.5 by 42.45 and -4.5 by 44.3 so the unfold/crowd/stacking story reads; a 2.5 dB duck 47.9-48.7 under the giant lock",+"music": {"auto": [[0, 0], [1.9, 0], [2.1, 4.5], [11.9, 4.5], [11.99, 0], [27.95, 0], [28.0, 5.5], [39.99, 5.5], [40.0, -1.5], [40.5, -1.5], [42.45, -3.5], [44.3, -4.5], [47.9, -4.5], [47.98, -7.0], [48.3, -7.0], [48.7, -4.5]]}+```diff --git a/skills/real-sound/references/licences.md b/skills/real-sound/references/licences.mdnew file mode 100644index 0000000..19eb2be--- /dev/null+++ b/skills/real-sound/references/licences.md@@ -0,0 +1,94 @@+# Licences: what may go into a film, how to verify it, how to credit it++A launch video is commercial use, it is edited (trimmed, layered, synced to picture: legally a derivative), and it is+published on platforms that scan audio. Every recording in the film needs a licence that allows all three, read on the+item's own page, recorded in its manifest, confirmed by the licence audit and credited. When in doubt, leave it out:+there is always another recording.++## Allowed / forbidden++| licence | status | why | credit |+|---|---|---|---|+| CC0 1.0, Public Domain Mark | allowed | no conditions | courtesy only |+| CC BY 4.0 / 3.0 / 2.5 / 2.0 | allowed | commercial use and edits allowed with attribution | REQUIRED (title, author, source link, licence link, "edited") |+| Mixkit Sound Effects Free License / Stock Music Free License | allowed | free for commercial video, no attribution; read the current text at https://mixkit.co/license/ | courtesy only; never redistribute the raw files, never register the film with Content ID |+| Pixabay Content License | allowed | free for commercial use, no attribution (Pixabay blocks scripted access: the user downloads by hand) | courtesy only |+| CC BY-NC (any version) | forbidden | NonCommercial: a launch video is commercial | |+| CC BY-ND | forbidden | NoDerivatives: trimming and syncing to picture are derivatives | |+| CC BY-SA | forbidden | ShareAlike: the whole film would have to be released under CC BY-SA | |+| Sampling+, Sampling | forbidden | retired, restrictive | |+| "personal use", "editorial", "custom", "free with credit on social media", all rights reserved | forbidden | not a standard licence the audit can trust | |+| two sources disagree ("DISPUTED") | forbidden | the strictest reading wins: reject it | |+| "royalty free" with no licence text, YouTube rips, anything behind a login or paywall you would bypass, AI-generated or code-synthesized audio | not used | unknown terms, or not a real recording | |++`scripts/licences.py` encodes this table (`classify()`); audit.py and credits.py use it.++## Verify on the item's own page (never from a search result, an aggregator or memory)++**Freesound** (most foley, ambiences, whooshes):+- The page is `https://freesound.org/people/<user>/sounds/<id>/`; the licence is in the sidebar of THAT page ("Creative+ Commons 0", "Attribution 4.0", "Attribution NonCommercial 4.0" ...). Licences are per sound, not per user.+- Record: licence text as shown, page author, title, the date you read it (`licence_seen` in audit.json).+- Originals need a login; the public high-quality preview is `https://cdn.freesound.org/previews/<id // 1000>/<id>_<user_id>-hq.ogg`+ (Ogg Vorbis, about 192 kbps; `-hq.mp3` also exists). `<user_id>` is the uploader's number: it must belong to the same+ page. audit.py checks that the id in the entry, the page and the preview URL agree.+- Read the description: some authors ask for a specific credit form (e.g. "<title> by <name> (www.site) of+ Freesound.org"), or forbid putting raw files online. Copy the requested form into the manifest's `attribution` and+ any restriction into the audit's `agent_notes`.++**Mixkit** (lossless WAV effects, some music):+- No per-item page exists for sound effects: record the category page that lists the item and its title there.+- WAV download: `https://assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`. Read the licence text itself+ (https://mixkit.co/license/) once per session and quote what it allows in `licence_seen`.+- Do not publish the raw assets folder; do not register the film's audio with Content ID or a rights-management service.++**Music sites** (one licensed track carries the film):+- Composer libraries that release CC BY music (e.g. Scott Buckley, scottbuckley.com.au): the licence line is on the+ track's page; many ask for a specific credit line: use it verbatim.+- Aggregators (chosic.com and similar) are not authoritative: confirm on the artist's own page. A real case: an+ aggregator listed a track as CC BY 3.0 while the artist's page said "free with credit on social media" (copyrighted):+ rejected as disputed.+- Free Music Archive, Internet Archive, SoundCloud: the licence is per track and often NC or ND: read it every time.+- YouTube Audio Library, platform libraries: terms tied to one platform: ask the user before using.++**Pixabay**: scripted downloads are blocked (bot protection). If a Pixabay item is the best choice, give the user the+page link and ask them to download it into the assets folder; then audit it like any other file.++## Attribution (CC BY)++A complete credit names the Title, the Author, the Source (link to the item page), the Licence (name + link) and says+it was modified. The manifest's `attribution` holds the line the author asked for (or this default):++```+"Rain on Windows, Interior, A.wav" by InspectorJ (www.jshaw.co.uk) of Freesound.org+(https://freesound.org/people/InspectorJ/sounds/346642/), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)+```++`credits.py` writes `CREDITS.md` (music + every sound, grouped by licence, with the sentence saying how the recordings+were edited) and `out/CREDITS_description.txt` (paste it into the video description). It completes a requested line+with the source link and the licence link when the author's form leaves them out. Credits never need to be on screen.++## Content ID and CC BY music++Some composers register their CC BY tracks with YouTube Content ID through a distributor. A video using the track+correctly can still get an automatic claim. The licence still allows the use: put the credit in the description, and+if a claim arrives, dispute it citing the licence and the credit (many composers also clear claims on request). Choose+a track whose composer explains this on their site, and tell the user before publishing. Mixkit and Pixabay forbid the+reverse: never register a film that contains their items with Content ID.++## The audit (scripts/audit.py, prompts/licence_audit.md)++1. The licence auditor opens every in-scope item's page, reads the licence and writes `licence_seen` (quote + URL ++ date + page author/title) and `licence_confirmed: true` into `audio/assets/manifests/audit.json` (format:+ `templates/audit_entry.json`). Never confirm a page you did not read in this session.+2. `audit.py --report out/qa/mix_report.json` classifies every manifest entry, checks ids and URLs, checks every file+ (exists, codec, rate, channels, duration vs the manifest, decodes without errors), merges the auditor's fields and+ writes audit.json + a Markdown table. In-scope = audible in the film. Exit 1 while any in-scope recording is blocked.+3. `--fetch` compares the local bytes with `download_url` (sequential, 1.5 s apart, a clear User-Agent, in memory,+ nothing executed; never Pixabay). Re-encoded or edited files show up here.+4. `credits.py --audit audio/assets/manifests/audit.json` refuses to call the credits clean while a used recording is+ unconfirmed.++On the Clawd film this caught four used recordings (celesta notes added after the hand audit) with no audit record.+Their manifest notes do say the licence was read on the Freesound page, so it was a bookkeeping gap, not an unverified+licence: audit.py quotes such a note and asks the auditor to copy it into `licence_seen`.diff --git a/skills/real-sound/references/process.md b/skills/real-sound/references/process.mdnew file mode 100644index 0000000..58cad8e--- /dev/null+++ b/skills/real-sound/references/process.md@@ -0,0 +1,135 @@+# The process: how a real-recordings soundtrack gets built by agents who cannot hear++This is the process that built the Clawd film's soundtrack (60 s, 83 real recordings, 264 placements, reproduced+bit-identically by `scripts/mix.py`), generalised. Agents cannot listen, so every decision is measured, every claim is+backed by a command, and the work is split so that parallel agents never edit the same file.++## Contents+1. Roles and files+2. The steps+3. What each critic measures (and the qa.py section that automates it)+4. Thresholds and where they come from+5. Variants: how critics test a fix without touching the design+6. Rounds, stopping, reporting++## 1. Roles and files++| role | prompt | owns (writes) | reads |+|---|---|---|---|+| lead | SKILL.md | `audio/SOUND_BRIEF.md`, `audio/sounddesign.json` (top level: include list, buses, master), final reports | everything |+| sourcing agent (one per category) | `templates/prompts/sourcing.md` | `audio/assets/<category>/`, `audio/assets/manifests/<category>.json`, `out/audio-review/<category>/` | brief |+| licence auditor | `templates/prompts/licence_audit.md` | agent fields of `audio/assets/manifests/audit.json` | manifests, item pages |+| music supervisor | `templates/prompts/music_supervisor.md` | `audio/assets/manifests/music_choice.json`, `audio/design/music.json` | music candidates, picture map |+| designer (one per category) | `templates/prompts/designer.md` | `audio/design/<category>.json` | its manifest, cues, solo renders |+| critic (one per lens) | `templates/prompts/critic_*.md` | only `out/audio-review/critic_<lens>/` (variants, measurements) | the whole render, design, trace |+| fixer (exactly one) | `templates/prompts/fixer.md` | every fragment + top-level buses/master during the fix round | all critic reports |++**Filling a prompt.** The lead replaces every `{{NAME}}` before handing a prompt to a sub-agent; an unfilled+placeholder must never reach one. The common ones: `{{PY}}` = the full path of the Python that has numpy, scipy,+soundfile and earshot (see SKILL.md, e.g. `<earshot repo>/.venv/Scripts/python.exe`); `{{SKILL}}` = this skill's folder;+`{{ROOT}}` = the film project root; `{{CATEGORY}}` = the sound category the agent owns; `{{FILM}}` = the film's name;+`{{ROUND}}` = the critique round number; `{{RENDER}}` = the folder of the render under review (soundtrack.wav,+stems); `{{REPORT}}` / `{{TRACE}}` / `{{QA_JSON}}` = its mix report, placement trace and qa.py report; `{{VIDEO}}` =+the rendered film; `{{CUES}}` = the cue file. The rest (`{{NEED_DESCRIPTION}}`, `{{KEY}}`, `{{BPM}}`, `{{DROP_T}}` ...)+are explained in the prompt that uses them.++## 2. The steps++1. **Cue list from the picture.** `out/cues.json` (format in `templates/cues.json`; Cinetic writes it with+ `export-cues.ts`). Sync frames are computed from the picture code (contact frames, 50 % pops, velocity peaks), never+ typed. Check the event kinds and ids are stable enough for rules to match with a regex (`act:thing:n`).+2. **Brief.** `audio/SOUND_BRIEF.md` from `templates/SOUND_BRIEF.md`: concept, picture map (every cue with what is on+ screen and the sound need), ambience zones, licences, quality bar, file layout, manifest format. The brief is the+ contract every sub-agent receives verbatim.+3. **Sourcing, per category, in parallel.** 15-40 candidates per category, licences read on each item's page,+ downloads polite (sequential, a pause between requests), each file measured (`analyze.py`, spectrogram PNG) and+ heard through earshot (`realness` gate, `describe` / `match` / `rank` against the need, `find` for the best take),+ ranked per need in the manifest with notes saying exactly what to use and what to avoid.+4. **Licence audit.** The auditor reads every candidate's page and records `licence_seen`; `audit.py` checks the rest.+ Nothing unconfirmed reaches the design.+5. **Music supervisor.** Measures every music candidate (`analyze.py --music`, `arrange.py`), verifies the beat grid+ and the downbeats against transients (the beat tracker lags attacks by 20-50 ms; arrange.py corrects the phase),+ picks the winner with reasons, and writes the edit: segments at bar lines, the drop's downbeat on the hero frame, each+ splice labelled with its bars and why it works.+6. **Design fragments, per category, in parallel.** Each designer trims samples (earshot `find` gives trims; `--dump`+ checks them), builds pools and rules on the cue events, and verifies the fragment alone: `mix.py --only <category>`+ (solo render, same variation as the full mix), `qa.py --only <category>`, `earshot timeline` of the solo with the+ cues. A designer never touches another fragment or the top level.+7. **Integrate.** The lead runs the full mix with `--trace`, then `qa.py` and earshot (`timeline`, `sheet`, `critique`+ against the launch-film profile). Fix blockers (missing files, failed gate) before critique.+8. **Critique, in parallel, read-only.** Five lenses (sync, edges, masking, tonal, director), each with its own prompt.+ Each critic measures, tests its proposed fixes as variants in its own folder (section 5) and returns ranked issues+ with exact JSON operations. Critics never edit the design.+9. **One fixer.** Merges the critics' operations, resolves conflicts (masking wants a hit louder, tonal wants its+ 2-6 kHz softer: the fixer finds the change that satisfies both, e.g. louder low body + softer top), applies them,+ re-renders, re-runs qa.py and compares with the previous round (nothing that passed may now fail). One writer+ avoids lost edits and contradictory fixes.+10. **Master and gate.** `mix.py` masters (`master.py`): integrated LUFS within 0.5 of the target, true peak at most+ -1.5 dBTP on the WAV; then check again after the AAC encode (about -1 dBTP).+11. **Credits.** `audit.py --report out/qa/mix_report.json` (exit 0) then `credits.py --audit ...` (exit 0): CREDITS.md+ and the video-description text.+12. **Mux.** `mux.mjs` (stream copy of the picture, AAC 320k, cut to the picture's duration); verify the audio track of+ the MP4 against the WAV (lag 0 samples) and run `earshot timeline film.mp4 --cues out/qa/cues_sounded.json` on the+ finished file.++## 3. What each critic measures++| lens | question | measurements (Clawd critic scripts, now qa.py) | typical fixes |+|---|---|---|---|+| **sync** | does every sound land on its cause? | per event: attack, 5 ms peak and 50 ms apex of the rebuilt event vs its cue (qa.py sync); risers by their end; per-frame picture motion around the event (frame differences of the film, earshot timeline cut/cue sync); envelope shape of each dumped sample (attack, -3/-6 dB span); solo renders of one rule | `align` (attack/peak/end), `offset`, a trim that starts closer to the attack, a different take whose peak is earlier |+| **edges** (clicks) | is anything cut, clicked or truncated? | > 14 kHz bursts concentrated in 1 ms in the master and every stem, attributed to the layer carrying the burst (qa.py clicks); unfaded head / tail of each used sample vs its peak and the source just outside the trim; second hits inside a sample; slices; placements past the film ends; music splices (the source level just after a segment end vs what enters) (qa.py edges) | longer `fade_in`/`fade_out`, move `start` 3-5 ms before the attack, end after the decay, `stop_before_next`, a different splice bar, `tail_fade` |+| **masking** (audibility, density) | can every effect be heard, and is the effect layer dense or repetitive? | per event: 1/3-octave margin vs the bed and vs everything else over its loudest 60 ms and its first 20 ms (`read_db`), per layer (dead layers); onsets per second; sample and take reuse; per-rule regularity (inter-onset CV, gain and pitch spread, distinct samples) (qa.py audibility, density); A/B variants of crowd beds | gain, EQ that carves the effect's band (or the bed's), a brighter take, fewer layers, a bus ride under the moment, more takes in a pool, `jitter_db` |+| **tonal** | is it pleasant on phones and laptops? | per event 2-6 kHz 5 ms peak vs the music's loudest (qa.py harshness) and its centroid; band levels over time of master vs stems (2-6k, 6-10k, 10-16k, >16k); low end under 120 Hz (mono, the impact's low partials vs the music's at the drop: beating); stereo width per section; tuned samples vs the key; limiter gain reduction at candidate (lufs, ceiling) settings | `highshelf` / `peq` cuts, `norm_peak_db` down, `mono`, `lp`, a pitch to the key, `master.lufs` / `ceiling` |+| **director** | does the soundtrack tell the film's story? | momentary / short-term loudness of the master and each stem across the picture map's beats, LRA, section medians, the loudest moment's position, the drop vs the logo, every beat's sfx and music peaks, ambience sources per zone (qa.py loudness + earshot timeline + sheet + critique) | bus rides, the music edit, which events are sounded, the ending |++## 4. Thresholds and where they come from++| check | threshold | source |+|---|---|---|+| click | > 14 kHz sample > 9x its 20 ms RMS and > 6x its 4 ms RMS, >= 80 % of the burst's energy in 1 ms | Cinetic av-audit's detector, ported (the same 7 click times as av-audit on the Clawd master: a check of the port, not independent validation) |+| click at an edge = a cut | the carrying layer's start or end within 2 ms | qa.py (fail); inside a recording = warn (the recording's own crisp transient, or a defect in the source) |+| abrupt entry | source 20 ms before the trim within 20 dB of the trim's peak, fade-in < 5 ms, and no attack within 10 ms to cover it (or the source within 10 dB) | Clawd critic_clicks edges.py (-30 dB listing), tightened so attacks that follow at once are not flagged |+| chopped decay | unfaded tail > -35 dB re peak, the source after the trim > -35 dB, fade-out < 30 ms | Clawd critic_clicks edges.py |+| chopped music | the source keeps sounding after `src_to` (within 6 dB) and is 6 dB above what enters, fade-out < 100 ms, no `tail_fade` | qa.py (generalises the supervisor's splice listings) |+| audibility | read = max(60 ms margin vs everything, 20 ms margin vs everything) over bands within 30 dB of the effect's strongest: >= 6 clear, 3-6 ok, 0-3 barely, < 0 masked | Clawd critic_masking measure.py |+| harshness | an event's 2-6 kHz 5 ms RMS more than 6 dB over the music's loudest 2-6 kHz 5 ms RMS | qa.py (the critic listed both levels; the 6 dB margin is a judgement); centroid > 6.5 kHz noted (norm) |+| sync | impacts: attack >= -45 ms and peak <= +75 ms from the cue; whooshes and peak-aligned swells: apex within +-45 ms; risers and end- / attack-aligned swells: audible end (last sample above -20 dB re its peak) -250..+30 ms (checked at -300..+60); a deviation that one layer's rule offset removes is listed as deliberate, naming that rule | measured norms of pro launch films (Cinetic sound.md §13) |+| machine gun | a rule with >= 4 events, one sample, pitch sd < 0.1 st, gain sd < 0.5 dB | Clawd critic_density metrics.py (regularity table) |+| loudness | integrated within 0.5 LU of the target (fail), true peak <= -1.0 dBTP (fail), last 50 ms <= -45 dBFS RMS and last 10 ms <= -60 dBFS peak (fail), first 5 ms <= -40 dBFS (warn), LRA 4-10 LU (warn), loudest moment before 85 % of the runtime (warn) | master.py gate, Cinetic av-audit head/tail, norms |++What qa.py flagged on the finished Clawd mix (which had been through five critics and a supervisor): 7 clicks in the+master, all inside recordings (the detector is av-audit's, so these are the transients the critics had already heard+about and accepted as crisp), 5 of 153 events masked (busy stacking moments), 3 events hot at 2-6 kHz, 2 edge defects+(one a deliberate 25 ms choke, one a deliberate music cut into the meadow), no sync defect (4 events sit off their cue+by exactly a rule offset, listed as deliberate with the rule that explains each), 2 cue events unsounded. Treat findings as questions+for a critic, not as orders; acknowledge confirmed exceptions with `qa_ok` (design-format.md §12).++## 5. Variants: how critics test a fix without touching the design++A critic writes its proposed operations to a JSON list and lets `variant.py` copy the design (top level + fragments)+into its own folder, apply them, render with every output inside that folder and run qa.py on the result:++```+python -I <skill>/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json \+ --out out/audio-review/critic_tonal/v1 --render --qa [--cues out/cues.json]+```++It prints every change as old -> new (an operation that matches nothing writes nothing), then the mix's gate line and+the qa digest; `v1/qa.json` sits next to the base render's `out/qa/qa.json` for a before / after comparison. The+critic reports those numbers next to each proposed operation. Operations are exact and small:+`{"f": "keyboard", "sample": "kb.thock.1", "set": {"norm_peak_db": -10}}`,+`{"f": "motion", "rule": "mo.leap.air", "set": {"offset": 0.30}}`,+`{"sd": "buses.music.auto", "value": [[0, 0], ...]}` (all forms in `templates/prompts/fixer.md`). The fixer applies+the merged list to the real files with `variant.py --apply` (originals copied to `--backup` first).++Variants catch interactions nobody predicts by reading JSON. A real case on the Clawd design: a 4 dB high-shelf cut on+a plastic "toy open" sample made its event LOUDER at 2-6 kHz (-15.8 -> -15.2 dBFS), because `norm_peak_db`+renormalises the sample after EQ (design-format.md §3, processing order); the fix is a lower `norm_peak_db` with the cut.++## 6. Rounds, stopping, reporting++- One critique round + one fix round is the minimum; a second round when the first fix round changed levels or the+ music edit. Stop when qa.py shows no fail, every remaining warn is acknowledged with a reason, and the director lens+ has no P0.+- Every number in a report comes from a command that was run; the report lists the commands.+- Keep `out/audio-review/` (critic folders, variants, dumps): it is the audit trail of why each number is what it is.diff --git a/skills/real-sound/references/sound-design.md b/skills/real-sound/references/sound-design.mdnew file mode 100644index 0000000..430e611--- /dev/null+++ b/skills/real-sound/references/sound-design.md@@ -0,0 +1,123 @@+# Sound design rules: launch videos and code-made films++The rules the designers, critics and fixer work from. Numbers marked *(norm)* are measured norms of top-tier launch+films (from the Cinetic skill's measurements, `references/sound.md` §13 there); the rest is practice that held up on+shipped films. The earshot repo's `docs/launch-video-sound.md` profiles the user's own reference films in depth; use it+for the targets of a specific film, this file stands alone for the craft.++## 1. The bar++- **Every sound has a visible cause**, at a loudness that matches the cause: a caret blink whispers, the hero reveal+ hits. Not every picture event gets a sound; once a class of action is sounded, every instance is, at one level.+- **Real and close.** Professional recordings: close-miked, dry, clean (floor under -55 dBFS between events), full+ bandwidth (95 % rolloff above 12 kHz), no clipping, no room of their own, no background events.+- **One palette.** One material family per film (felt + wood, glass + soft plastic, metal + air), one click voice, one+ keyboard, one music track. A different material for every sound is the fastest way to sound cheap.+- **The music carries the film.** Effects punctuate; the music's own events (drop, stop, re-entry) carry the big+ moments and most cuts.+- **Restraint.** About 5 designed effects per 10 s *(norm; 0-28)*, clustered where the music is sparse (cold open,+ typing, logo build) and thinning out after the drop. Good sound design mostly disappears.++## 2. Launch-video grammar (picture event -> sound)++| picture | sound (real source) | where it sits | level / notes |+|---|---|---|---|+| cursor moves, hovers, scrolls, progress bars, streaming text, rolling counters | nothing | | the music carries continuous motion |+| cursor click / tap / press | one real click voice for the whole film: mouse, trackpad, phone tap on glass; press + release 60-110 ms later (about 85, release 0-3 dB quieter) *(norm)* | the press frame (clicks land 10-30 ms before the visible press in pro films) | about 3 dB under the music's short-term level, 12-20 dB over it above 3 kHz *(norm)*; bright but short (-20 dB within 6-10 ms); centred; the same level every time |+| typing | sliced real keyboard (mechanical or laptop, one board per film); space / return deeper and longer | each sounded key on its character frame; at most ~12 keys/s; above ~25 chars/s one key per word or silence *(norm)* | music dips 6-10 dB under the run; vary the key sample and level (sigma ~2 dB, no jump over 6 dB), not random pitch; most on-screen typing in pro films is silent *(norm: ~75 %)* |+| a card / chip / toast / badge appears | a soft real pop or tick: cork, mouth pop, plastic click, wood block, pitched up for size | 50 % of its travel | a run climbs (`pitch_ramp` +1..+3 st); +2..+9 dB over the music for word-like pops *(norm)* |+| toggle, switch, lock-in, snap into place | a real latch, switch, magnet clack, Lego click | the contact frame | short; in sync within a frame |+| a counter or metric locks on its value | one crisp tick on the final value only | the lock frame | about 8 dB under the music *(norm)* |+| a camera-scale move, whip, zoom into the product, logo zoom | a real swish (bamboo, stick, cloth, cape, air past the mic) | apex on the move's velocity peak, onset 100-450 ms ahead *(norm)* | about 2 per film, never on every cut; may be wide; travels with the motion (pan) |+| hard cuts | nothing | | about 85 % of cuts in pro films have no sound of their own *(norm)* |+| tension into the reveal | a real riser: cymbal roll (yarn mallets), orchestral swell, the music's own build | ends 30-250 ms before the hit (a beat of air), or stops on it | in about 40 % of films *(norm)*; never a crash cymbal on the hit |+| the hero reveal / the turn | the music's DROP lands on it (edit the track so its downbeat is on the frame); optionally one real impact: drum, taiko, kick, felt sub hit, with a short top transient | the contact / cut frame | the loudest moment of the film, at 30-60 % of the runtime, +2..+7 LU over integrated *(norm)* |+| the logo / lockup | the music stops on a bar line or filters down, 0.2-0.7 s of near-silence, then ONE soft resolved element: a bell, celesta or piano note in the track's key, a soft swell | the frame the wordmark becomes readable (its settle), never later | 8-20 dB under the film's peak *(norm)*; a 0.5-2.5 s tail to below -60 dBFS, then silence |+| product objects (a phone, a card, a box) | real foley of the object's material | contacts | dry, centred |++## 3. Music++- **One licensed track** at 110-130 BPM (or half-time 55-65) whose structure fits the film: soft intro, groove, a+ breakdown or stop at 40-80 % of the runtime, a drop for the hero reveal, a resolve. Steady tempo, 4/4,+ professionally mastered, full bandwidth, instrumental.+- **Avoid:** vocals, corporate ukulele + whistling, EDM brostep, lo-fi hiss, trailer braams on a product film,+ dark/tense moods for a friendly product, famous overused stock tracks.+- **Match the brand's personality:** calm/literary -> soft piano, felt, strings, no booms; fast/exact -> tight drums,+ plucks, a drop on true silence; bright/friendly -> marimba, claps, pops, major key; precise/expensive -> sparse piano,+ low strings, air; raw/mechanical -> ticks, latches, metallic hits; warm/kind -> plucked strings, soft pads.+- **Edit at bar lines** (`arrange.py`): the picture is cut on the grid (120 BPM = 2 s bars at 4/4), the music is cut on+ its downbeats, splices continue the harmony (V -> I, or the same chord), and the drop's downbeat lands on the hero+ frame. Stretch only when the tempo is off by more than ~0.2 %, and by at most ~6 %.+- **Shape with rides, not with new music:** dip under typing, lift into the drop, settle under the logo (`buses.music.auto`).++## 4. Levels, arc and mix++- -14 LUFS integrated for web and social; true peak at most -1 dBTP after the AAC encode (`master.ceiling` 0.77 gives+ that). -15/-16 only for a stated reason (a dynamic orchestral track the limiter would squash).+- LRA 5-9 LU for a film with an arc *(norm: 1-2 sounds crushed)*; the intro 5-12 LU under the body for its first+ 2-3 s, entering with intent on frame 0 (no fade over the first bar, no click on sample 0).+- The loudest momentary sits on the drop / hero reveal, not on the logo, not in the last 15 %.+- Effects are dry, mono and centred (+-0.15 pan); only whooshes, risers, ambiences and the music are wide. Low end+ below ~120 Hz is mono (master.py folds it).+- Every effect at least +3 dB over the bed in its own best band (qa.py `read_db`; >= 6 is clear); a layer that adds+ nothing measurable is a wasted placement (qa.py `--layers`).+- Nothing in the 2-6 kHz band much louder than the music's own loudest presence peaks (qa.py harshness): that band+ hurts first on phones and laptops. Effect centroids at or below 6.5 kHz *(norm)*; pings at 9-14 kHz fatigue.++## 5. Sync++- Impacts, clicks, keys, pops: `align: attack` on the contact / press / 50 % frame. Whooshes: `align: peak` on the+ velocity peak. Risers: `align: end` on (or 30-250 ms before) the hit. Beds and crowds: `align: start`.+- A sound that leads its picture by more than ~45 ms reads as "sound first"; one that peaks more than ~75 ms late reads+ as late *(norm: accents land -45..+75 ms, median +8)*. Move the picture to the beat, not the sound off its cause.+- Check slow-attack samples: an attack-aligned sample whose peak comes 80 ms after its first transient lands late+ (qa.py sync reports attack and peak per event).+- Cue frames are computed from the picture code; typed-in times land 8-11 frames off *(Cinetic, in practice)*.++## 6. Variation without randomness++- A repeated action draws from a pool of 4+ takes (`mode: cycle`: never the same take twice in a row), +-1-2 dB jitter,+ +-0.3-0.5 st pitch at most (real keyboards do not change pitch per key: vary the take, not the pitch).+- The UI click is the exception: one voice, one level, every time.+- A progression climbs: `pitch_ramp` across a run of pops or stacking blocks, `gain_ramp` into a build.+- Machine-gun (one sample, no variation, regular rhythm) is the second-fastest cheap tell; qa.py density flags it.++## 7. Tuning++- Tuned elements (chimes, bells, celesta, a motif) play in the track's key (`analyze.py --music` and `arrange.py`+ estimate it); tune a recording by resampling (`pitch`) within +-2 semitones; choose a recording near the right note+ rather than shifting far.+- Booms and subs on the track's root, or neutral (unpitched) at least; an untuned boom under a resolving chord sounds+ cheap even to people who cannot say why.++## 8. Film foley (character pieces, miniatures, worlds)++- **Scale through pitch and closeness:** a 1 cm character's steps are fingertip taps pitched +2..+6 st, close and dry;+ a giant's are slowed (-3..-6 st) heavy foley with a longer tail.+- **Layer by role:** body (the weight, 100-400 Hz) + transient (the crisp contact, 2-6 kHz, short) + detail (material:+ rattle, creak, rustle). Each layer is one rule on the same cue; check each adds something (`--layers`).+- **Ambience zones:** one bed per location, from clean windows of real field recordings (no voices, traffic, planes,+ dogs unless the story wants them), cut with the picture's cuts, ducked under key actions with `auto`.+- **Materials stay consistent:** the same key always sounds like the same key; the same surface like the same surface.++## 9. Choosing a recording (agents cannot listen: measure)++- `analyze.py` (format, floor, bandwidth, clipping, onsets, spikes in an ambience, tempo/key for music) and the+ spectrogram PNG; `earshot describe / match` (does it sound like the description? use earshot's verdict words:+ strong z >= 3, plausible 2.5-3, weak 2-2.5, no match below 2),+ `earshot rank` (sort candidates), `earshot find` (the best take inside a long recording: ready-made trims),+ `earshot compare --refs` / `earshot ab` (closer to approved references?), `earshot realness` (a real recording, not+ code-synthesized: require "real or pro-produced" on the trim you use; an "uncertain" needs its evidence read and+ written down).+- Prefer: lossless or HQ previews (Freesound -hq.ogg, Mixkit WAV), 44.1/48 kHz or higher, many takes of the same object+ (variation for free), a clean floor between takes.+- Reject: clipped, low-bitrate (a cliff above 16 kHz), roomy foley, anything with voices or bumps inside the window you+ need, anything whose licence you cannot read on its own page.++## 10. The cheap tells (what made the user call a soundtrack "random ass sounds")++Code-synthesized beeps, sine pings and noise sweeps; one sample machine-gunned; a whoosh on every cut; a boom on every+logo; sparkle / shimmer sweeps standing in for "magic"; reverb on tiny sounds; random pitch on typing; partial+coverage (some presses heard, others silent); clicks at cuts; music chopped mid-note at a splice; an audible ambience+loop; a different room on every sound; the logo louder than the drop; an ending that stops mid-note instead of a tail+into silence.diff --git a/skills/real-sound/references/troubleshooting.md b/skills/real-sound/references/troubleshooting.mdnew file mode 100644index 0000000..6c3de3a--- /dev/null+++ b/skills/real-sound/references/troubleshooting.md@@ -0,0 +1,66 @@+# Troubleshooting++## mix.py++| symptom | cause | fix |+|---|---|---|+| `cue file out/cues.json not found (the design has cue rules)` | no cue file at the default path | pass `--cues path` or set `"cues"` in the design |+| `set "duration" (seconds) in the design or "total" (frames) in the cue file` | neither is known | add `"duration"` to the top-level design |+| `KeyError: 'pool X: unknown sample Y'` | a typo, or a slice group without `#*` | fix the name; `"kb.type#*"` expands every slice |+| `ValueError: pool X is empty` / `slices F: no onsets found` | the slice window has no onsets above the gate | widen `start`/`end`, lower `delta` (0.05), lower `gate_db` (-24), check the file with `analyze.py` |+| `KeyError: 'fragment.json: samples "x" is defined twice'` | two fragments use one name | prefix every name with its category (`kb.`, `mo.`) |+| `KeyError: '_doc'` in the bus loop | a comment key inside `buses` | move comments to `_buses_doc` at the top level |+| `ValueError: reverse is not used` | a sample asks for `reverse` | find a recording that already has the shape (a real swell, a reverse cymbal recorded as such) |+| `rubberband failed` | the ffmpeg build has no librubberband | use `"stretch": "none"` when the track is within ~0.2 % of the film tempo, else ask the user for an ffmpeg with rubberband (`ffmpeg -filters | grep rubberband`) |+| `master gate FAILED` | integrated loudness or true peak missed | read `master.json` next to the report: lower `master.lufs` by 1 for a dynamic track, or trim the loudest peaks (often one impact or the drop's first beat) with a bus ride; `ceiling` 0.77 leaves AAC headroom |+| the first run is slow | every recording is decoded once (and the music stretched once) | later runs read `out/audio/cache` (keyed on path, size, mtime); `--cache DIR` moves it |+| a solo render "has no file" | older habit: looking for premaster.wav | `--only NAME` writes the unmastered mix to `--out` (default `out/audio-review/solo/<NAME>.wav`) |+| the solo sounds different from the full mix | a rule without `name` (variation keyed on its index) | name every rule; then solo and full mix play identical placements |+| an event is in `unsounded_events` | no rule matches its kind / id / time | add a rule, or accept it on purpose (qa.py coverage lists partial classes, which read as mistakes) |++## qa.py++| symptom | cause | fix |+|---|---|---|+| sections "skipped: needs the trace" | the mix ran without `--trace` | re-run `mix.py ... --trace out/qa/adds.json` |+| `trace (N placements) and report (M) disagree` | the trace and the report come from different runs or a different `--only` | re-render with both from one run; pass the same `--only` to qa.py |+| `placement(s) could not be rebuilt` | the design changed after the mix | re-render before qa |+| a click "inside a recording" | the recording itself has a crisp transient (a tick, a pop) or a defect (a digital click, a bump) | `--dump` the sample and look at its spectrogram (`analyze.py --png`); if it is a defect, trim around it or use another take; if it is the sound itself, acknowledge with `qa_ok` |+| a music splice "chops the music" | the outgoing segment ends while its note still sounds and the next one is quieter | end the segment where the chord has decayed, splice into a louder bar, add `tail_fade`, or acknowledge a deliberate hard cut with `qa_ok` |+| many "bright" notes | a tiny-object palette (plastic, glass, bells) is bright by nature | fine when it is the film's palette; otherwise a `highshelf` cut or a darker take |+| `--listen` says earshot is not importable | earshot is not installed in this Python | `pip install -e <earshot repo>`, or run qa.py without `--listen` and call `python -m earshot ...` separately |++## audit.py / credits.py++| symptom | fix |+|---|---|+| every recording "licence not verified" | the auditor has not recorded `licence_seen` / `licence_confirmed` in `audio/assets/manifests/audit.json` yet: run `templates/prompts/licence_audit.md` |+| `licence_seen reads as X but the manifest says Y` | the page and the manifest disagree: re-read the page, fix the manifest's `license`, or reject the item |+| `used in the film but in no manifest` | a file was added to a fragment without a manifest entry: add the entry (and audit it) or remove the sound |+| `entry id ... but the Freesound page is sound ...` | the entry, the page and the download URL describe different sounds: re-download from the right page |+| credits list "PROBLEMS" | a used recording is forbidden, unknown, unconfirmed or lacks author / title / page_url: fix before publishing |++## Sources and downloads++| symptom | fix |+|---|---|+| Freesound preview 404 | the preview folder is `id // 1000` and the number after `_` is the uploader's user id (take it from the page's player) |+| Pixabay returns 403 / a challenge page | Pixabay blocks scripted downloads: give the user the link and ask them to download the file |+| a Mixkit item has no page | Mixkit SFX have none: record the category page and the listed title |+| a file decodes with errors | re-download; prefer the lossless (Mixkit WAV) or HQ (Freesound -hq.ogg) version |++## Mux and delivery++| symptom | fix |+|---|---|+| true peak over -1 dBTP after AAC | lower `master.ceiling` (0.77 -> 0.72) and re-render; measure the MP4's decoded audio with `python -m earshot measure film.mp4` (`master.py --measure` reads WAV / FLAC only) |+| the audio ends early | the soundtrack is shorter than the picture: set `duration` to the picture's length (mux.mjs reports the mismatch) |+| sync looks off | `mux.mjs` stream-copies the picture and cuts the audio to the picture's duration; cross-correlate the MP4's decoded audio with the WAV (lag must be 0) and check the cue frames; never re-time the audio to fix a picture-side offset |++## Platform++- Run the scripts with `python -I` (isolated: downloaded files can never shadow a module); the scripts add their own+ folder to the path themselves.+- Console output is ASCII (Windows cp1252 consoles); quote paths with spaces (`--root "/path/to/my film"`).+- `mix.py`'s audio maths are frozen: the Clawd soundtrack reproduces bit-identically+ (`sha1 b48c5e8872fe96765624f88db18a0b49a0a05d68`). Change the DSP only with a new version and a new reference.diff --git a/skills/real-sound/scripts/analyze.py b/skills/real-sound/scripts/analyze.pynew file mode 100644index 0000000..45a474a--- /dev/null+++ b/skills/real-sound/scripts/analyze.py@@ -0,0 +1,268 @@+#!/usr/bin/env python+"""Objective analysis of downloaded audio assets (we cannot listen, so we measure).++ python -I <skill>/scripts/analyze.py FILE [FILE ...] [--music] [--png DIR] [--out OUT.json] [--json]++Prints one plain-language line per file (then a summary line; exit code 1 when a file is missing or does not decode);+--out writes every measurement as a JSON list, --json prints that list instead of the lines.+Every file is decoded with ffmpeg to 48 kHz float stereo (a mono source is duplicated at unity gain). Reports per file:+ format (codec, source rate, channels, bitrate), duration, peak / RMS / integrated LUFS, noise floor (10th percentile of+ 20 ms RMS), clipping, DC, the first transient (attack) time, onset times, spectral centroid, bandwidth (95% rolloff),+ and "spikes" (sudden events far above the median level: useful to catch dogs, voices, bumps inside an ambience).+With --music also: tempo + beat times (librosa), tempo stability, key estimate (Krumhansl), loudness per 2 s and per+bar, and novelty-based section boundaries.+With --png: a spectrogram + waveform image per file (ffmpeg showspectrumpic), to look at with an image viewer.++Reading the numbers (rules of thumb for picking recordings):+ foley / SFX noise_floor_dbfs < -55 between events, clipped_samples 0, rolloff95_hz > 12 kHz (not dull), a+ band_db_rel_1_4k['16-19k'] cliff far below '12-16k' means a low-bitrate encode; spikes_over_12db inside an+ ambience = single events (dog, voice, bump) to avoid or cut around.+ ambience level_std_db_100ms small (steady bed), side_to_mid_db > -12 (real stereo; null = mono), no spikes,+ >= 40 s usable.+ music tempo + ibi_std_ms small (steady grid), key, rms_db_per_bar (the arc), sections_at (edit points).+"""+import argparse, json, os, subprocess, sys, math, warnings+import numpy as np++SR = 48000+++def probe(path):+ try:+ out = subprocess.run(['ffprobe', '-v', 'error', '-select_streams', 'a:0', '-show_entries',+ 'stream=codec_name,sample_rate,channels,bit_rate:format=bit_rate,duration',+ '-of', 'json', path], capture_output=True, text=True, timeout=60).stdout+ j = json.loads(out)+ s = (j.get('streams') or [{}])[0]+ f = j.get('format') or {}+ brs = [int(b) for b in (s.get('bit_rate'), f.get('bit_rate')) if str(b).isdigit() and 0 < int(b) < 10 ** 7]+ return {'codec': s.get('codec_name'), 'src_rate': int(s.get('sample_rate', 0) or 0),+ 'channels': int(s.get('channels', 0) or 0), 'kbps': round(brs[0] / 1000) if brs else None}+ except Exception as e: # noqa+ return {'probe_error': str(e)}+++def decode(path, channels=2):+ """(n, 2) float32 at SR; a mono source is duplicated at unity gain (ffmpeg's own upmix is 3 dB down)"""+ ch = 1 if channels == 1 else 2+ p = subprocess.run(['ffmpeg', '-v', 'error', '-i', path, '-map', '0:a:0', '-f', 'f32le', '-acodec', 'pcm_f32le',+ '-ac', str(ch), '-ar', str(SR), '-'], capture_output=True, timeout=600)+ if p.returncode != 0 or not p.stdout:+ raise RuntimeError(p.stderr.decode(errors='replace').strip()[:400] or 'no audio decoded')+ x = np.frombuffer(p.stdout, dtype=np.float32).reshape(-1, ch)+ return np.repeat(x, 2, axis=1) if ch == 1 else x.copy()+++def db(x):+ return float(20 * np.log10(max(float(x), 1e-12)))+++def frame_rms(mono, win):+ n = len(mono) // win+ if n == 0:+ return np.array([np.sqrt(np.mean(mono ** 2) + 1e-20)])+ f = mono[: n * win].reshape(n, win)+ return np.sqrt(np.mean(f ** 2, axis=1) + 1e-20)+++KK_MAJ = np.array([6.35, 2.23, 3.48, 2.33, 4.38, 4.09, 2.52, 5.19, 2.39, 3.66, 2.29, 2.88])+KK_MIN = np.array([6.33, 2.68, 3.52, 5.38, 2.60, 3.53, 2.54, 4.75, 3.98, 2.69, 3.34, 3.17])+NAMES = ['C', 'C#', 'D', 'Eb', 'E', 'F', 'F#', 'G', 'Ab', 'A', 'Bb', 'B']+++def key_estimate(chroma_mean):+ best = []+ for mode, prof in (('major', KK_MAJ), ('minor', KK_MIN)):+ for k in range(12):+ r = np.corrcoef(chroma_mean, np.roll(prof, k))[0, 1]+ best.append((float(r), f'{NAMES[k]} {mode}'))+ best.sort(reverse=True)+ return [{'key': b[1], 'r': round(b[0], 3)} for b in best[:3]]+++def analyze(path, music=False):+ import librosa+ warnings.filterwarnings('ignore', message='n_fft=') # very short sounds: librosa pads, the numbers stay usable+ info = {'file': path.replace('\\', '/'), **probe(path)}+ y = decode(path, info.get('channels'))+ mono = y.mean(axis=1)+ n = len(mono)+ info['duration'] = round(n / SR, 3)+ if n < SR * 0.02:+ info['error'] = 'too short'+ return info+ peak = float(np.max(np.abs(y)))+ info['peak_dbfs'] = round(db(peak), 2)+ info['rms_dbfs'] = round(db(np.sqrt(np.mean(y ** 2))), 2)+ info['clipped_samples'] = int(np.sum(np.abs(y) >= 0.999))+ info['dc'] = round(float(np.mean(mono)), 5)+ try:+ import pyloudnorm as pyln+ if n > SR * 0.45:+ v = float(pyln.Meter(SR).integrated_loudness(y.astype(np.float64)))+ info['lufs'] = round(v, 2) if math.isfinite(v) else None # None: silence+ except Exception:+ pass+ r20 = frame_rms(mono, int(0.02 * SR))+ info['noise_floor_dbfs'] = round(db(np.percentile(r20, 10)), 1)+ info['loudest_20ms_dbfs'] = round(db(np.max(r20)), 1)+ info['dynamic_range_db'] = round(info['loudest_20ms_dbfs'] - info['noise_floor_dbfs'], 1)+ # stereo width: side/mid energy+ mid, side = (y[:, 0] + y[:, 1]) / 2, (y[:, 0] - y[:, 1]) / 2+ info['side_to_mid_db'] = (round(db(np.sqrt(np.mean(side ** 2))) - db(np.sqrt(np.mean(mid ** 2))), 1)+ if np.any(side) else None) # None: both channels identical (mono)+ # first transient: first 1 ms window whose envelope gets within 20 dB of the peak envelope+ r1 = frame_rms(mono, int(0.001 * SR))+ thr = np.max(r1) * 0.1+ idx = int(np.argmax(r1 >= thr))+ info['attack_at'] = round(idx * 0.001, 4)+ pk = int(np.argmax(r1))+ info['peak_at'] = round(pk * 0.001, 4)+ # spectral shape (on a 22.05k copy for speed)+ ys = librosa.resample(mono, orig_sr=SR, target_sr=22050) if n < SR * 400 else mono[:: 2]+ sr2 = 22050+ S = np.abs(librosa.stft(ys, n_fft=2048, hop_length=512))+ cen = librosa.feature.spectral_centroid(S=S, sr=sr2)[0]+ roll = librosa.feature.spectral_rolloff(S=S, sr=sr2, roll_percent=0.95)[0]+ act = r20[: len(cen)] if len(r20) else None+ info['centroid_hz'] = int(np.median(cen))+ info['rolloff95_hz'] = int(np.percentile(roll, 90))+ # high-band levels relative to 1-4 kHz (an encoder lowpass shows up as a cliff, e.g. 16-19k far below 12-16k)+ m30 = mono[: min(n, SR * 60)]+ Sf = np.abs(np.fft.rfft(m30 * np.hanning(len(m30)))) ** 2+ fr = np.fft.rfftfreq(len(m30), 1 / SR)+ band = lambda a, b: float(np.mean(Sf[(fr >= a) & (fr < b)]) + 1e-30)+ ref = band(1000, 4000)+ info['band_db_rel_1_4k'] = {k: round(10 * np.log10(band(a, b) / ref), 1) for k, (a, b) in+ {'40-120': (40, 120), '8-12k': (8000, 12000), '12-16k': (12000, 16000),+ '16-19k': (16000, 19000), '19-22k': (19000, 22000)}.items()}+ onsets = librosa.onset.onset_detect(y=ys, sr=sr2, units='time', backtrack=False)+ info['onset_count'] = int(len(onsets))+ info['onsets'] = [round(float(o), 3) for o in onsets[:40]]+ # spikes: 100 ms windows more than 12 dB above the median level+ r100 = frame_rms(mono, int(0.1 * SR))+ med = np.median(r100)+ sp = np.where(r100 > med * 4)[0]+ info['spikes_over_12db'] = [round(i * 0.1, 1) for i in sp[:30]]+ info['level_std_db_100ms'] = round(float(np.std(20 * np.log10(r100 + 1e-12))), 2)+ if music:+ tempo, beats = librosa.beat.beat_track(y=ys, sr=sr2, start_bpm=120, units='time', tightness=200)+ tempo = float(np.atleast_1d(tempo)[0])+ info['tempo'] = round(tempo, 2)+ if len(beats) > 4:+ ibi = np.diff(beats)+ info['tempo_from_ibi'] = round(60 / float(np.median(ibi)), 2)+ info['ibi_std_ms'] = round(float(np.std(ibi)) * 1000, 1)+ info['beats'] = [round(float(b), 3) for b in beats]+ chroma = librosa.feature.chroma_cqt(y=ys, sr=sr2, hop_length=512)+ info['key'] = key_estimate(chroma.mean(axis=1))+ r2 = frame_rms(mono, SR * 2)+ info['rms_db_per_2s'] = [round(db(v), 1) for v in r2]+ # loudness per bar (assuming 4/4 on the tracked beats)+ if len(beats) > 8:+ bars = beats[::4]+ pb = []+ for a, b in zip(bars[:-1], bars[1:]):+ seg = mono[int(a * SR): int(b * SR)]+ pb.append(round(db(np.sqrt(np.mean(seg ** 2) + 1e-20)), 1))+ info['bar_starts_if_beat0_is_downbeat'] = [round(float(b), 3) for b in bars]+ info['rms_db_per_bar'] = pb+ # sections: novelty on stacked chroma + mfcc+ try:+ mf = librosa.feature.mfcc(y=ys, sr=sr2, n_mfcc=13, hop_length=512)+ F = np.vstack([librosa.util.normalize(chroma, axis=1), librosa.util.normalize(mf, axis=1)])+ k = max(4, min(12, int(info['duration'] // 15)))+ bounds = librosa.segment.agglomerative(F, k)+ info['sections_at'] = [round(float(t), 2) for t in librosa.frames_to_time(bounds, sr=sr2, hop_length=512)]+ except Exception as e:+ info['sections_error'] = str(e)[:200]+ return info+++def png(path, outdir):+ os.makedirs(outdir, exist_ok=True)+ base = os.path.splitext(os.path.basename(path))[0]+ out = os.path.join(outdir, base + '_spec.png')+ p = subprocess.run(['ffmpeg', '-v', 'error', '-y', '-i', path, '-lavfi',+ 'showspectrumpic=s=1400x420:mode=combined:color=intensity:scale=log:legend=1', out],+ capture_output=True, timeout=300)+ if p.returncode or not os.path.exists(out):+ raise RuntimeError(p.stderr.decode(errors='replace').strip()[:200] or 'no image written')+ return out.replace('\\', '/')+++def describe(r):+ """one plain-language line: format, level, noise, timing, tone, and what looks wrong"""+ if r.get('error'):+ return f"{os.path.basename(r['file'])}: ERROR {r['error']}"+ fmt = f"{r.get('codec')} {r.get('kbps') or '?'} kbps {(r.get('src_rate') or 0) / 1000:g} kHz {r.get('channels')}ch"+ lufs = f"{r['lufs']} LUFS" if r.get('lufs') is not None else 'LUFS n.a.'+ s = (f"{os.path.basename(r['file'])}: {fmt}, {r['duration']} s, peak {r['peak_dbfs']} dBFS, {lufs}, noise floor "+ f"{r['noise_floor_dbfs']} dBFS, attack {r['attack_at']} s, centroid {r['centroid_hz']} Hz, rolloff95 "+ f"{r['rolloff95_hz']} Hz, " + ('mono' if r.get('side_to_mid_db') is None else f"side/mid {r['side_to_mid_db']} dB"))+ if r.get('tempo'):+ s += f", {r.get('tempo_from_ibi', r['tempo'])} BPM (ibi std {r.get('ibi_std_ms')} ms), key {r['key'][0]['key']}"+ warn = []+ if r['peak_dbfs'] <= -90:+ warn.append('digital silence')+ if r.get('clipped_samples'):+ warn.append(f"{r['clipped_samples']} clipped samples")+ if r.get('kbps') and r.get('codec') == 'mp3' and r['kbps'] < 192:+ warn.append('low-bitrate mp3')+ if r.get('src_rate') and r['src_rate'] < 44100:+ warn.append(f"sample rate {r['src_rate']} Hz")+ if r['rolloff95_hz'] < 8000 and r['peak_dbfs'] > -90:+ warn.append('dull (little above 8 kHz)')+ if r.get('spikes_over_12db'):+ warn.append(f"{len(r['spikes_over_12db'])} spike(s) > 12 dB over the median level")+ return s + ('; CHECK: ' + ', '.join(warn) if warn else '')+++def clean(o):+ if isinstance(o, dict):+ return {k: clean(v) for k, v in o.items()}+ if isinstance(o, list):+ return [clean(v) for v in o]+ return None if isinstance(o, float) and not math.isfinite(o) else o+++def main():+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ ap.add_argument('files', nargs='+')+ ap.add_argument('--music', action='store_true', help='also tempo, beats, key, loudness per bar, sections')+ ap.add_argument('--png', metavar='DIR', help='write a spectrogram image per file here')+ ap.add_argument('--out', metavar='OUT.json', help='write every measurement as a JSON list')+ ap.add_argument('--json', action='store_true', help='print the JSON list instead of one line per file')+ a = ap.parse_args()+ res = []+ for f in a.files:+ if not os.path.isfile(f):+ r = {'file': f, 'error': 'file not found'}+ else:+ try:+ r = analyze(f, a.music)+ except Exception as e: # noqa: BLE001+ r = {'file': f, 'error': str(e)[:400]}+ if a.png:+ try:+ r['png'] = png(f, a.png)+ except Exception as e: # noqa: BLE001+ r['png_error'] = str(e)[:200]+ r = clean(r)+ r['summary'] = describe(r)+ res.append(r)+ if not a.json:+ print(r['summary'].encode('ascii', 'replace').decode())+ bad = [os.path.basename(r['file']) for r in res if r.get('error')]+ summary = f"{len(res)} file(s) analysed, {len(bad)} failed" + (f" ({', '.join(bad[:6])})" if bad else '')+ if a.out:+ os.makedirs(os.path.dirname(os.path.abspath(a.out)), exist_ok=True)+ with open(a.out, 'w', encoding='utf-8') as fh:+ json.dump(res, fh, indent=1)+ summary += f'; wrote {a.out}'+ print(json.dumps(res, indent=1, ensure_ascii=True) if a.json else summary)+ return 1 if bad else 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/scripts/arrange.py b/skills/real-sound/scripts/arrange.pynew file mode 100644index 0000000..776a051--- /dev/null+++ b/skills/real-sound/scripts/arrange.py@@ -0,0 +1,351 @@+#!/usr/bin/env python+"""Fit a music track to the film's bar structure by editing it at bar lines (no generation).++ python -I <skill>/scripts/arrange.py TRACK [--bpm 123.97] [--phase auto|0..3]+ [--film-bpm 120] [--beats-per-bar 4] [--bars 30 | --arc arc.json | --arc "-14,-12,...,0,0,-7"]+ [--drop-film-bar 21] [--drop auto|none|SRC_BAR] [--start-bar SRC_BAR] [--end-bar SRC_BAR] [--phrase 2]+ [--film-offset 0] [--file audio/assets/music/track.mp3] [--out out/audio-review/arrange.json] [--json]++The film's arc is a target loudness per film bar in dB relative to the track's loud bars (0 = the loudest sections,+-14 = a soft intro). Give it as a comma list, or a JSON file (templates/arc.json: {"arc": [...], "film_bpm": 120,+"beats_per_bar": 4, "drop_film_bar": 21}), or let the script build a generic one from --bars and --drop-film-bar (soft+open, groove, breakdown, build, big section from the drop, resolve). Write the arc from the picture map in SOUND_BRIEF.md.++1. Beat grid: librosa beats, a robust linear fit beats ~ t0 + k*T over the whole track (precise BPM), then the phase is+ moved onto the measured attacks (a 23 ms-window spectral flux at 2.9 ms resolution, folded at the beat period): the+ beat tracker's onset envelope lags real attacks by 20-50 ms, and a splice cut on a lagging grid chops the downbeat's+ own attack.+2. Downbeat: which beat phase starts bars (low-band onsets + harmonic change), with a confidence (z-score gap) and a+ local check in 8-bar windows (sections where another phase clearly wins are listed). Measured on the two+ hand-labelled tracks of the Clawd project (both seen while building this): Born Of The Sky correct (confidence 0.93);+ Bring Me The Sky half a bar off (confidence 0.15, flagged LOW: its bass plays on beats 1 and 3 and its accents move+ by half a bar later in the track). Under LOW confidence the downbeat is a guess between the two printed grids:+ decide from the music (the first note, section entrances, chord changes in the bar table) and pass --phase N.+3. Bar features: chroma, MFCC, loudness, low-band energy, onset density; novelty = change from the previous bar.+4. Arrangement by dynamic programming over the film's bars (film bar k spans bar_len*(k-1)..bar_len*k s):+ cost = energy-curve mismatch (the film's arc) + jump costs (a jump must land on a bar that sounds like the one that+ would naturally follow: chroma/MFCC/loudness distance) + phrase penalties (jumps prefer phrase starts) ++ hard anchors (--drop-film-bar = --drop source bar, the last film bar = --end-bar, film bar 1 = --start-bar). By+ default (--drop auto) the drop film bar is anchored on the strongest drop candidate (a bar within 3 dB of the loud+ sections that stays loud, scored by its lift over the 4 bars before + novelty); --drop none lets the arc decide.+ On the two Clawd tracks the top candidate is a section start the music supervisor marked by hand (Born Of The Sky+ bar 50 = the crash into big section 1; Bring Me The Sky the re-entry at 127.8 s); developed on those same tracks.+Prints the summary, the bar table, the chosen map and a ready-to-paste "music" block for the design (source seconds,+film seconds); --json prints the full result as JSON instead, --out writes it to a file.+It is a first draft: check every splice (qa.py edges, earshot timeline) and the bar table, then fine-tune "at" by a few+ms so the measured downbeat transients land on the picture's downbeats.+"""+import argparse, json, os, subprocess, sys+import numpy as np++SR = 22050+++def default_arc(n, drop):+ """a generic short-film arc over n film bars with the big section starting at film bar `drop` (1-based):+ soft open -> groove -> breakdown -> build -> big section -> resolve"""+ arc = []+ big_until = drop + max(2, int(round((n - drop) * 0.6)))+ for k in range(1, n + 1):+ if k < drop:+ x = (k - 1) / max(1, drop - 2) # 0..1 up to the bar before the drop+ v = -14 + 10 * min(1.0, x / 0.45) # opening rises to the groove (-4)+ if 0.55 <= x < 0.8:+ v = -9.5 # breakdown before the build+ elif x >= 0.8:+ v = -9 + 4 * (x - 0.8) / 0.2 # build+ elif k < big_until:+ v = 0.0 # the big section+ else:+ v = -13 + 2 * min(3, n - k) # resolve to the end+ arc.append(round(v, 1))+ return arc+++def decode(path):+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-f', 'f32le', '-ac', '1', '-ar', str(SR), '-'],+ capture_output=True)+ if p.returncode != 0 or not p.stdout:+ sys.exit(f'cannot decode {path}: {p.stderr.decode(errors="replace")[:300]}')+ return np.frombuffer(p.stdout, np.float32).copy()+++def fit_grid(beats, T0):+ """robust fit of beat times to t0 + k*T, starting from period T0"""+ k = np.r_[0, np.cumsum(np.maximum(1, np.round(np.diff(beats) / T0)))]+ A = np.vstack([k, np.ones_like(k)]).T+ inl = np.ones(len(beats), bool)+ T, t0 = T0, float(beats[0])+ for _ in range(5):+ T, t0 = np.linalg.lstsq(A[inl], beats[inl], rcond=None)[0]+ res = beats - (t0 + k * T)+ inl = np.abs(res) < 0.035+ if inl.sum() < 8:+ inl[:] = True+ res = beats - (t0 + k * T)+ return float(T), float(t0), float(np.median(np.abs(res))), float(np.mean(np.abs(res) < 0.035))+++def flux(y, n_fft=512, hop=64, block=4096):+ """log-compressed spectral flux with a short (23 ms) window at 2.9 ms resolution, computed in blocks of frames+ (bounded memory); returns (flux, frame centre times)"""+ win = np.hanning(n_fft).astype(np.float32)+ nfr = max(0, (len(y) - n_fft) // hop + 1)+ out, prev = np.zeros(nfr, np.float32), None+ for f0 in range(0, nfr, block):+ f = np.arange(f0, min(nfr, f0 + block))+ L = np.log1p(100 * np.abs(np.fft.rfft(y[f[:, None] * hop + np.arange(n_fft)] * win, axis=1)))+ d = np.diff(L, axis=0, prepend=L[:1] if prev is None else prev[None])+ out[f] = np.maximum(0, d).sum(axis=1)+ prev = L[-1]+ return out, (np.arange(nfr) * hop + n_fft / 2) / SR+++def transient_phase(y, T, t0):+ """shift (s) that puts the grid on the measured attacks: the short-window flux folded at the beat period over the+ whole track, searched within +-min(100 ms, 0.3 beat); returns (shift, score ratio best / median)"""+ fl, tt = flux(y)+ hop_s = tt[1] - tt[0]+ search = min(0.1, 0.3 * T)+ offs = np.arange(-search, search + 1e-9, 0.001)+ k = np.arange(int(np.ceil((tt[0] - t0 + search) / T)), int((tt[-1] - t0 - search) / T))+ score = np.array([fl[np.round((t0 + o + k * T - tt[0]) / hop_s).astype(int)].mean() for o in offs])+ b = int(np.argmax(score))+ return float(offs[b]), float(score[b] / (np.median(score) + 1e-9))+++def local_phases(score, grid, bpb, phase, win=8, hop=4, min_gap=0.5):+ """stretches where another downbeat phase clearly wins (by >= min_gap, the global LOW threshold) in windows of+ `win` bars: [{'from', 'to', 'phase', 'gap'}]"""+ out, W = [], win * bpb+ for s in range(0, len(grid) - W + 1, hop * bpb):+ seg = score[s:s + W]+ sc = [float(np.mean(seg[(p - s) % bpb::bpb])) for p in range(bpb)]+ o = np.argsort(sc)[::-1]+ gap = sc[o[0]] - sc[o[1]]+ if o[0] == phase or gap < min_gap:+ continue+ t0, t1 = round(float(grid[s]), 2), round(float(grid[min(len(grid) - 1, s + W)]), 2)+ if out and out[-1]['phase'] == int(o[0]) and t0 <= out[-1]['to']:+ out[-1].update({'to': t1, 'gap': max(out[-1]['gap'], round(gap, 2))})+ else:+ out.append({'from': t0, 'to': t1, 'phase': int(o[0]), 'gap': round(gap, 2)})+ return out+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])+ ap.add_argument('track')+ ap.add_argument('--bpm', type=float, help='the track\'s tempo if you know it (the phase is still fitted)')+ ap.add_argument('--phase', default='auto', help='downbeat phase 0..beats-per-bar-1, or auto')+ ap.add_argument('--film-bpm', type=float, default=120.0, help='tempo the picture is cut to (after any stretch)')+ ap.add_argument('--beats-per-bar', type=int, default=4)+ ap.add_argument('--bars', type=int, default=30, help='film bars (ignored when --arc is given)')+ ap.add_argument('--arc', help='target dB per film bar: comma list or JSON file')+ ap.add_argument('--drop-film-bar', type=int, help='1-based film bar of the drop / big section')+ ap.add_argument('--drop', default='auto', help='source bar for the drop film bar: auto (the top drop candidate), '+ 'none, or a 0-based bar index')+ ap.add_argument('--end-bar', type=int, help='source bar index for the film\'s last bar')+ ap.add_argument('--start-bar', type=int, help='source bar index for film bar 1')+ ap.add_argument('--phrase', type=int, default=2, help='jumps prefer film bars that start a phrase of this length')+ ap.add_argument('--film-offset', type=float, default=0.0, help='film time of film bar 1 (s)')+ ap.add_argument('--jump', type=float, default=3.0, help='base cost per jump')+ ap.add_argument('--no-snap', action='store_true', help='keep the beat tracker\'s phase (skip the transient fit)')+ ap.add_argument('--file', help='the track path as the design should reference it (default: TRACK as given)')+ ap.add_argument('--out', help='write the full result as JSON here')+ ap.add_argument('--json', action='store_true', help='print the full result as JSON instead of the table')+ a = ap.parse_args(argv)+ if a.drop not in ('auto', 'none') and not a.drop.lstrip('-').isdigit():+ sys.exit(f'--drop must be auto, none or a source bar index, not {a.drop!r}')+ if not os.path.isfile(a.track):+ sys.exit(f'track not found: {a.track}')+ cfg = {}+ if a.arc and a.arc.strip().endswith('.json'):+ cfg = json.load(open(a.arc, encoding='utf-8'))+ arc_list = [float(v) for v in cfg['arc']]+ elif a.arc:+ arc_list = [float(v) for v in a.arc.split(',') if v.strip()]+ else:+ arc_list = None+ film_bpm = float(cfg.get('film_bpm', a.film_bpm))+ bpb = int(cfg.get('beats_per_bar', a.beats_per_bar))+ FILM_BARS = len(arc_list) if arc_list else a.bars+ drop_k = int(a.drop_film_bar or cfg.get('drop_film_bar') or round(FILM_BARS * 0.7))+ ARC = arc_list or default_arc(FILM_BARS, drop_k)+ bar_len = bpb * 60.0 / film_bpm+ import librosa+ y = decode(a.track)+ dur = len(y) / SR+ hop = 256+ oenv = librosa.onset.onset_strength(y=y, sr=SR, hop_length=hop)+ tempo, beats = librosa.beat.beat_track(onset_envelope=oenv, sr=SR, hop_length=hop, start_bpm=a.bpm or 120,+ tightness=400, units='time')+ beats = np.asarray(beats, float)+ if len(beats) < 8:+ sys.exit(f'only {len(beats)} beats tracked in {a.track}: not a steady-tempo track, or too short')+ T0 = 60.0 / a.bpm if a.bpm else float(np.median(np.diff(beats)))+ T, t0, med_res, inlier = fit_grid(beats, T0)+ if a.bpm: # trust a given tempo but keep the fitted phase+ T = 60.0 / a.bpm+ shift, snap_ratio = (0.0, None) if a.no_snap else transient_phase(y, T, t0)+ t0 += shift+ bpm = 60.0 / T+ first = t0 - np.floor(t0 / T) * T+ grid = np.arange(first, dur, T)+ # downbeat phase: low-band onset strength + chroma change at each beat+ S = np.abs(librosa.stft(y, n_fft=2048, hop_length=hop))+ freqs = librosa.fft_frequencies(sr=SR, n_fft=2048)+ low = S[freqs < 150].sum(axis=0)+ low_on = np.maximum(0, np.diff(low, prepend=low[0]))+ chroma = librosa.feature.chroma_stft(S=S ** 2, sr=SR, hop_length=hop)+ fr = lambda t: min(int(t * SR / hop), S.shape[1] - 1)+ beat_low = np.array([low_on[max(0, fr(t) - 2): fr(t) + 4].max() for t in grid])+ beat_chg = []+ for t in grid:+ a0, a1, b1 = fr(max(0, t - T)), fr(t), fr(min(dur, t + T))+ pa = chroma[:, a0:a1].mean(axis=1) if a1 > a0 else chroma[:, a1]+ pb = chroma[:, a1:b1].mean(axis=1) if b1 > a1 else chroma[:, a1]+ beat_chg.append(1 - float(np.dot(pa, pb) / (np.linalg.norm(pa) * np.linalg.norm(pb) + 1e-9)))+ beat_chg = np.array(beat_chg)+ zl = (beat_low - beat_low.mean()) / (beat_low.std() + 1e-9)+ zc = (beat_chg - beat_chg.mean()) / (beat_chg.std() + 1e-9)+ ph_score = [float(np.mean(zl[p::bpb]) + np.mean(zc[p::bpb])) for p in range(bpb)]+ order = np.argsort(ph_score)[::-1]+ phase = int(order[0]) if a.phase == 'auto' else int(a.phase)+ conf = ph_score[order[0]] - ph_score[order[1]]+ alt = int(order[1]) if order[0] == phase else int(order[0])+ local = local_phases(zl + zc, grid, bpb, phase)+ bars = grid[phase::bpb]+ bars = bars[bars + bpb * T <= dur + 1e-3]+ nb = len(bars)+ if nb < 2:+ sys.exit('the track is shorter than two bars')+ # bar features+ mf = librosa.feature.mfcc(S=librosa.power_to_db(S ** 2), sr=SR, n_mfcc=13)+ feats, loud, dens = [], [], []+ for b in bars:+ s0, s1 = int(b * SR), int((b + bpb * T) * SR)+ f0, f1 = fr(b), fr(b + bpb * T)+ loud.append(20 * np.log10(np.sqrt(np.mean(y[s0:s1] ** 2)) + 1e-9))+ dens.append(float((oenv[f0:f1] > np.median(oenv) * 1.5).mean()))+ c = chroma[:, f0:f1].mean(axis=1); c = c / (np.linalg.norm(c) + 1e-9)+ m = mf[1:, f0:f1].mean(axis=1); m = m / (np.linalg.norm(m) + 1e-9)+ feats.append(np.concatenate([c, m]))+ feats, loud = np.array(feats), np.array(loud)+ rel = loud - np.percentile(loud, 90)+ nov = np.r_[0, np.linalg.norm(np.diff(feats, axis=0), axis=1)]+ rise = np.r_[0, np.diff(loud)]+ # a drop: a loud bar (within 3 dB of the loud sections) that stays loud for 4 bars, lifted over the 4 bars before it+ lift = np.array([rel[i] - np.median(rel[max(0, i - 4):i]) if i else 0.0 for i in range(nb)])+ loud_now = (rel > -3) & (np.array([rel[i:i + 4].mean() for i in range(nb)]) > -4)+ drop_score = np.where(loud_now if loud_now.any() else rel > np.percentile(rel, 75), lift / 3 + nov * 2, -np.inf)+ drop_cands = [int(i) for i in np.argsort(drop_score)[::-1][:6] if np.isfinite(drop_score[i])]+ drop_src = drop_cands[0] if a.drop == 'auto' else None if a.drop == 'none' else int(a.drop)+ for what, v in (('--drop', drop_src), ('--start-bar', a.start_bar), ('--end-bar', a.end_bar)):+ if v is not None and not 0 <= v < nb:+ sys.exit(f'{what} {v} is outside the source bars 0..{nb - 1}')+ if not 1 <= drop_k <= FILM_BARS:+ sys.exit(f'--drop-film-bar {drop_k} is outside the film bars 1..{FILM_BARS}')+ # how different bar j sounds from bar i+ D = np.linalg.norm(feats[:, None] - feats[None], axis=2) + np.abs(loud[:, None] - loud[None]) / 6+ nxt = np.minimum(np.arange(nb) + 1, nb - 1)+ J = a.jump + 2.5 * D[nxt] # J[i, j]: jump from source bar i to j+ J[np.arange(nb - 1), np.arange(1, nb)] = 0.0 # the natural continuation is free+ J[np.arange(nb), np.arange(nb)] += 1.5 # repeating the same bar+ cont = np.zeros((nb, nb), bool)+ cont[np.arange(nb - 1), np.arange(1, nb)] = True+ INF = 1e18+ arc = np.array(ARC, float)+ cost = np.full((FILM_BARS, nb), INF)+ back = np.zeros((FILM_BARS, nb), int)+ cost[0] = np.abs(rel - arc[0]) / 3.0 + 0.4 * np.arange(nb) # prefer starting near the top+ if a.start_bar is not None:+ keep = cost[0, a.start_bar] - 0.4 * a.start_bar+ cost[0] = INF+ cost[0, a.start_bar] = keep+ for k in range(1, FILM_BARS):+ Ck = J + np.where(cont, 0.0, 1.0 if k % a.phrase else 0.0) # film bar k+1 not on a phrase start: jumps cost more+ tot = cost[k - 1][:, None] + Ck+ back[k] = np.argmin(tot, axis=0)+ cost[k] = tot[back[k], np.arange(nb)] + np.abs(rel - arc[k]) / 3.0+ if k == drop_k - 1 and drop_src is not None:+ cost[k, np.arange(nb) != drop_src] = INF+ if k == FILM_BARS - 1 and a.end_bar is not None:+ cost[k, np.arange(nb) != a.end_bar] = INF+ jend = int(np.argmin(cost[-1]))+ path = [jend]+ for k in range(FILM_BARS - 1, 0, -1):+ path.append(int(back[k, path[-1]]))+ path = path[::-1]+ segs = []+ for k, j in enumerate(path):+ if segs and j == segs[-1]['last'] + 1:+ segs[-1]['last'] = j+ segs[-1]['bars'] += 1+ else:+ segs.append({'first': j, 'last': j, 'film_bar': k + 1, 'bars': 1})+ out_segs = []+ for s in segs:+ src_from = float(bars[s['first']])+ out_segs.append({'src_from': round(src_from, 4), 'src_to': round(src_from + bpb * T * s['bars'], 4),+ 'at': round(a.film_offset + bar_len * (s['film_bar'] - 1), 4),+ 'src_bars': [s['first'], s['last']], 'film_bars': [s['film_bar'], s['film_bar'] + s['bars'] - 1]})+ ratio = film_bpm / bpm+ music = {'file': (a.file or a.track).replace('\\', '/'), 'src_bpm': round(bpm, 3), 'target_bpm': film_bpm,+ 'stretch': 'none' if abs(ratio - 1) < 0.002 else 'rubberband', 'gain_db': 0.0,+ 'segments': [{'label': f"src bars {s['src_bars'][0]}-{s['src_bars'][1]} -> film bars "+ f"{s['film_bars'][0]}-{s['film_bars'][1]}",+ 'src_from': s['src_from'], 'src_to': s['src_to'], 'at': s['at'],+ 'fade_in': 0.008, 'fade_out': 0.06} for s in out_segs]}+ if music['stretch'] == 'none':+ music['src_bpm'] = film_bpm # within 0.2 %: play it unstretched on the film grid+ low = conf < 0.5 and a.phase == 'auto'+ summary = (f'{bpm:.3f} BPM (grid residual {med_res * 1000:.1f} ms, {inlier:.0%} of beats on it), downbeats at '+ f'{bars[0]:.3f} + {bpb * T:.4f}*n s (phase {phase}' + (+ f', confidence {conf:.2f}' + (f': LOW, the other candidate is {grid[alt]:.3f} + {bpb * T:.4f}*n s '+ f'(--phase {alt})' if low else '') if a.phase == 'auto' else+ f' as given; the automatic choice is phase {int(order[0])}, confidence {conf:.2f}')+ + f'); {nb} source bars -> {FILM_BARS} film bars in {len(out_segs)} segment(s), drop film bar {drop_k} = '+ + (f'source bar {drop_src}' + (' (top drop candidate)' if a.drop == 'auto' else '') if drop_src is not None+ else 'chosen by the arc') + f'; stretch ratio {ratio:.4f}')+ if local:+ summary += ('; another downbeat phase wins locally at ' + ', '.join(+ f"{x['from']:.1f}-{x['to']:.1f} s (phase {x['phase']})" for x in local[:4]) + ': check the bar lines there')+ res = {'summary': summary, 'track': a.track, 'duration': round(dur, 3), 'bpm': round(bpm, 4), 'beat_period': round(T, 6),+ 'film_bpm': film_bpm, 'beats_per_bar': bpb, 'film_bars': FILM_BARS, 'drop_film_bar': drop_k, 'arc': ARC,+ 'stretch_ratio_to_film': round(ratio, 6), 't0': round(t0, 4), 'transient_shift_ms': round(shift * 1000, 1),+ 'transient_fit_ratio': round(snap_ratio, 2) if snap_ratio else None,+ 'grid_median_residual_ms': round(med_res * 1000, 1), 'grid_inlier_frac': round(inlier, 3),+ 'downbeat_phase': phase, 'phase_scores': [round(s, 3) for s in ph_score], 'phase_confidence': round(conf, 3),+ 'phase_low_confidence': low, 'phase_alternative': {'phase': alt, 'first_downbeat': round(float(grid[alt]), 3)},+ 'local_phase_disagreements': local, 'drop_source_bar': drop_src,+ 'bars': [{'i': i, 't': round(float(bars[i]), 3), 'rel_db': round(float(rel[i]), 1), 'novelty': round(float(nov[i]), 2),+ 'rise_db': round(float(rise[i]), 1), 'density': round(dens[i], 2)} for i in range(nb)],+ 'drop_candidates': drop_cands, 'map': path, 'segments': out_segs, 'music': music,+ 'total_cost': round(float(cost[-1, jend]), 2)}+ if a.out:+ os.makedirs(os.path.dirname(os.path.abspath(a.out)), exist_ok=True)+ json.dump(res, open(a.out, 'w'), indent=1)+ if a.json:+ print(json.dumps(res, indent=1))+ return res+ print(summary)+ print(f'transient fit moved the grid {shift * 1000:+.0f} ms (peak/median {snap_ratio or 0:.2f}); '+ f'phase scores {[round(s, 2) for s in ph_score]}; drop candidates (source bars) {drop_cands}')+ if low:+ print(f'LOW downbeat confidence: bars may start at {grid[alt]:.3f} + {bpb * T:.4f}*n s instead. Decide from the '+ 'music (first note, section entrances, chord changes) and re-run with --phase N.')+ print('bar t rel_dB nov rise dens')+ for b in res['bars']:+ mark = ' <drop?' if b['i'] in drop_cands[:3] else ''+ print(f"{b['i']:3d} {b['t']:7.2f} {b['rel_db']:6.1f} {b['novelty']:5.2f} {b['rise_db']:5.1f} {b['density']:5.2f}{mark}")+ print('map (film bar -> source bar):', ' '.join(f'{k + 1}:{j}' for k, j in enumerate(path)))+ print('design "music" block (check every splice, then fine-tune "at" by a few ms):')+ print(json.dumps(music, indent=1))+ if a.out:+ print(f'wrote {a.out}')+ return res+++if __name__ == '__main__':+ main()diff --git a/skills/real-sound/scripts/audit.py b/skills/real-sound/scripts/audit.pynew file mode 100644index 0000000..72e051c--- /dev/null+++ b/skills/real-sound/scripts/audit.py@@ -0,0 +1,400 @@+#!/usr/bin/env python+"""Licence + file audit of every recording in the manifests: is each one allowed, verified, credited and intact?++ python -I <skill>/scripts/audit.py [--root .] [--manifests audio/assets/manifests] [--report out/qa/mix_report.json]+ [--out audio/assets/manifests/audit.json] [--table out/qa/audit.md]+ [--fetch] [--fetch-all] [--decode scope|all|none] [--json]++Per manifest entry (references/licences.md has the policy; templates/manifest_entry.json the fields):+ licence classified from the manifest's "license": allowed (CC0, public domain, CC BY 2.0-4.0, Mixkit free+ licences, Pixabay) / forbidden (NC, ND, SA, Sampling+, personal, custom, editorial, disputed) / unknown+ verified the agent's own reading of the licence on the item's page: "licence_seen" (+ "licence_confirmed") merged+ from an existing audit.json (the --out file); the script never verifies a licence itself, it only checks+ that what the agent saw classifies the same as the manifest+ attribution CC BY items need a ready-to-paste "attribution" credit line (and author, title, page_url)+ urls Freesound / Mixkit ids agree between the entry id, the page and the download URL+ file exists, ffprobe codec / rate / channels / duration, decodes without errors (ffmpeg -f null; in-scope+ recordings unless --decode all), duration matches the manifest's analysis; with --fetch the bytes are+ compared with download_url (sequential, 1.5 s apart, in memory, never executed; only for recordings+ used in the film unless --fetch-all)+in_scope = used in the film (report.files_used) when --report is given, else the existing audit's value, else true.+Severity: blocker (in scope and forbidden / unknown / unverified licence, missing or broken file, CC BY without a+credit line), warning, ok. Exit code 1 when an in-scope recording has a blocker.+Writes audit.json (a list, one record per entry; agent notes kept under "agent_notes") and a Markdown table.+"""+import argparse, hashlib, json, os, re, shutil, subprocess, sys, time++sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+import licences as L # noqa: E402++TOOL = 'real-sound audit.py v2'+NEEDED = ('id', 'file', 'title', 'author', 'page_url', 'license')+NO_SCRIPTS = ('pixabay.com',) # sites that block scripted downloads: never fetch+UA = 'real-sound-licence-audit/1.0 (polite single-file check; contact: project owner)'+++def say(s):+ print(str(s).encode('ascii', 'replace').decode())+++COMPUTED = ('path', 'exists', 'codec', 'sample_rate', 'channels', 'kbps', 'duration_s', 'video', 'decodes_clean',+ 'max_volume_db', 'bytes_match_download_url', 'fetch_note')+AUDIT_KEYS = ('id', 'manifest', 'need', 'rank', 'in_scope', 'severity', 'licence_in_manifest', 'licence', 'licence_status',+ 'licence_seen', 'licence_confirmed', 'attribution_required', 'attribution_ok', 'attribution_in_manifest',+ 'file_ok', 'file_check', 'earlier_file_check', 'problems', 'warnings', 'agent_notes', 'audit_tool', 'problem',+ 'fix', 'duplicate_of')+SENT = r'(?:[^.]|\.(?=\d))' # inside one sentence ("4.0" does not end it)+READ_NOTE = re.compile(SENT + r'*\blicen[cs]e' + SENT + r'{0,40}\b(read|verified|checked|seen|confirmed)\b' + SENT + '*',+ re.I)+++def ffprobe(path):+ """first audio stream's codec / rate / channels / bit rate, the duration, and any real video stream (not cover art)"""+ p = subprocess.run(['ffprobe', '-v', 'error', '-show_entries', 'stream=codec_type,codec_name,sample_rate,channels,'+ 'bit_rate:stream_disposition=attached_pic:format=duration,bit_rate', '-of', 'json', path],+ capture_output=True, text=True, timeout=120)+ j = json.loads(p.stdout or '{}')+ st = j.get('streams') or []+ s = next((x for x in st if x.get('codec_type') == 'audio'), {})+ f = j.get('format') or {}+ brs = [int(b) for b in (s.get('bit_rate'), f.get('bit_rate')) if str(b).isdigit() and 0 < int(b) < 10 ** 7]+ video = [x.get('codec_name') for x in st if x.get('codec_type') == 'video'+ and not (x.get('disposition') or {}).get('attached_pic')]+ return {'codec': s.get('codec_name'), 'sample_rate': int(s.get('sample_rate') or 0), 'channels': int(s.get('channels') or 0),+ 'kbps': round(brs[0] / 1000) if brs else None, 'duration_s': round(float(f.get('duration') or 0), 3),+ **({'video': video[0]} if video else {})}+++def decodes_clean(path):+ """(decodes without errors, first error line, max volume dBFS) from one full ffmpeg decode of the audio stream"""+ p = subprocess.run(['ffmpeg', '-hide_banner', '-nostdin', '-loglevel', 'level+info', '-i', path, '-map', '0:a:0',+ '-af', 'volumedetect', '-f', 'null', '-'], capture_output=True, timeout=600)+ lines = p.stderr.decode(errors='replace').splitlines()+ err = [ln for ln in lines if '[error]' in ln or '[fatal]' in ln]+ vol = [float(m.group(1)) for ln in lines for m in [re.search(r'max_volume: (-?[\d.]+|-inf) dB', ln)] if m+ and m.group(1) != '-inf'] or [None]+ return p.returncode == 0 and not err, err[0][:200] if err else None, vol[-1]+++def url_checks(e):+ """id consistency for the two sites whose URLs encode it"""+ out = []+ page, dl, eid = e.get('page_url') or '', e.get('download_url') or '', str(e.get('id') or '')+ m = re.search(r'freesound\.org/people/([^/]+)/sounds/(\d+)', page)+ if m:+ sid = m.group(2)+ if re.match(r'fs-\d+', eid) and eid[3:] != sid:+ out.append(f'entry id {eid} but the Freesound page is sound {sid}')+ d = re.search(r'previews/(\d+)/(\d+)_(\d+)-(hq|lq)\.(ogg|mp3)', dl)+ if d:+ if d.group(2) != sid:+ out.append(f'download_url is sound {d.group(2)}, the page is sound {sid}')+ if d.group(1) != str(int(sid) // 1000):+ out.append(f'preview folder {d.group(1)} should be {int(sid) // 1000}')+ if d.group(4) == 'lq':+ out.append('low-quality Freesound preview (-lq): use the -hq preview')+ if e.get('author') and m.group(1).lower() != str(e['author']).lower():+ out.append(f'author "{e["author"]}" differs from the page owner "{m.group(1)}"')+ d = re.search(r'assets\.mixkit\.co/active_storage/sfx/(\d+)/(\d+)\.(wav|mp3)', dl)+ if d:+ if d.group(1) != d.group(2):+ out.append('Mixkit download_url ids disagree')+ if re.match(r'mk-\d+', eid) and eid[3:] != d.group(1):+ out.append(f'entry id {eid} but the Mixkit download is item {d.group(1)}')+ return out+++def fetch_match(url, path):+ """True / False when the bytes at url equal the local file; None + reason when not checked"""+ host = re.sub(r'^[a-z]+://', '', url.lower()).split('/')[0]+ if any(host.endswith(h) for h in NO_SCRIPTS):+ return None, f'{host} blocks scripted downloads: compare by hand'+ import urllib.request+ try:+ req = urllib.request.Request(url, headers={'User-Agent': UA})+ with urllib.request.urlopen(req, timeout=60) as r:+ data = r.read(200 * 1024 * 1024 + 1)+ except Exception as ex: # noqa: BLE001+ return None, f'fetch failed: {str(ex)[:160]}'+ local = open(path, 'rb').read()+ return hashlib.sha1(data).hexdigest() == hashlib.sha1(local).hexdigest(), None+++def agent_fields(old):+ """what the agent wrote into an earlier audit record (licence reading, notes, hand-recorded file checks such as a+ byte comparison); script findings are recomputed, earlier evidence is kept under earlier_file_check"""+ if not old:+ return {}+ keep = {k: old[k] for k in ('licence_seen', 'licence_confirmed') if k in old}+ fc = old.get('file_check') or {}+ earlier = dict(old.get('earlier_file_check') or {})+ if old.get('audit_tool'):+ notes = old.get('agent_notes') or []+ earlier.update({k: fc[k] for k in ('bytes_match_download_url', 'fetch_note') if k in fc})+ else: # a hand-written audit: its warnings / problem / fix are the agent's notes, its file checks earlier evidence+ notes = list(old.get('warnings') or []) + [old[k] for k in ('problem', 'fix') if old.get(k)]+ earlier.update({k: v for k, v in fc.items() if k not in COMPUTED or k in ('bytes_match_download_url', 'fetch_note')})+ keep.update({k: v for k, v in old.items() if k not in AUDIT_KEYS})+ if earlier:+ keep['earlier_file_check'] = earlier+ if notes:+ keep['agent_notes'] = notes+ if 'in_scope' in old:+ keep['_in_scope'] = old['in_scope']+ return keep+++def audit_entry(e, root, used, old, decode='scope'):+ path = os.path.join(root, e['file']) if not os.path.isabs(e['file']) else e['file']+ f = L.rel(e['file'], root)+ lic = L.classify(e.get('license'))+ ag = agent_fields(old)+ in_scope = (f.lower() in used) if used is not None else bool(ag.get('_in_scope', True))+ problems, warnings = [], []+ for k in NEEDED:+ if not e.get(k):+ (problems if k in ('file', 'license') else warnings).append(f'missing "{k}"')+ if lic['status'] == 'forbidden':+ problems.append(f'licence forbidden: {lic["reason"]}')+ elif lic['status'] == 'unknown':+ problems.append(f'licence unknown: {lic["reason"]}')+ seen = ag.get('licence_seen')+ confirmed = bool(ag.get('licence_confirmed')) and bool(seen)+ if not seen:+ m = READ_NOTE.search(str(e.get('notes') or ''))+ problems.append('no licence record in audit.json' + (+ f' (the manifest notes say: "{m.group(0).strip()[:90]}"): copy that reading into "licence_seen" and set '+ '"licence_confirmed"' if m else ': read the licence on the item page and record "licence_seen"'))+ else:+ sk = L.classify(seen)['key']+ if sk not in ('unknown', 'none', lic['key']):+ warnings.append(f'licence_seen reads as {sk} but the manifest says {lic["key"]}')+ if not ag.get('licence_confirmed'):+ problems.append('licence_seen recorded but licence_confirmed is not true')+ attr_ok = True+ if lic['attribution']:+ attr_ok = bool(e.get('attribution')) and bool(e.get('author')) and bool(e.get('page_url'))+ if not attr_ok:+ problems.append('CC BY needs "attribution" (a ready-to-paste credit line), "author" and "page_url"')+ else:+ at = e['attribution']+ if str(e['author']).split(' (')[0].lower() not in at.lower():+ warnings.append('the attribution line does not name the author')+ if not re.search(r'cc[ -]?by|attribution|creativecommons\.org', at, re.I):+ warnings.append('the attribution line does not name the licence (credits.py adds the licence link)')+ warnings += url_checks(e)+ fc = {'path': f, 'exists': os.path.isfile(path)}+ if not fc['exists']:+ problems.append('file missing')+ else:+ try:+ fc.update(ffprobe(path))+ except Exception as ex: # noqa: BLE001+ problems.append(f'ffprobe failed: {str(ex)[:120]}')+ if fc.get('video'):+ warnings.append(f"the file carries a {fc['video']} video stream: extract the audio to its own file and make "+ 'sure the licence covers the audio')+ if decode == 'all' or (decode == 'scope' and in_scope):+ ok, err, vol = decodes_clean(path)+ fc['decodes_clean'], fc['max_volume_db'] = ok, vol+ if not ok:+ problems.append(f'decode errors: {err}')+ if vol is not None and vol <= -90:+ problems.append(f'the recording is digital silence (max {vol} dB)')+ elif vol is not None and vol < -60:+ warnings.append(f'the recording is near-silent (max {vol} dB): the wrong file?')+ an = (e.get('analysis') or {}).get('duration')+ if an and fc.get('duration_s') and abs(float(an) - fc['duration_s']) > 0.05:+ warnings.append(f'duration {fc["duration_s"]} s differs from the manifest analysis ({an} s): file replaced?')+ if fc.get('sample_rate') and fc['sample_rate'] < 44100:+ warnings.append(f'sample rate {fc["sample_rate"]} Hz (dull above {fc["sample_rate"] // 2000} kHz)')+ if fc.get('codec') == 'mp3' and fc.get('kbps') and fc['kbps'] < 192:+ warnings.append(f'MP3 at {fc["kbps"]} kbps: look for a lossless or higher-rate source')+ r = {'id': e.get('id'), 'manifest': e['_manifest'], 'need': e.get('need'), 'rank': e.get('rank'),+ 'in_scope': in_scope, 'severity': None,+ 'licence_in_manifest': e.get('license'), 'licence': lic['key'], 'licence_status': lic['status'],+ 'licence_seen': seen, 'licence_confirmed': confirmed, 'attribution_required': lic['attribution'],+ 'attribution_ok': attr_ok, 'attribution_in_manifest': e.get('attribution'),+ 'file_ok': fc['exists'] and fc.get('decodes_clean', True) is not False, 'file_check': fc,+ **({'earlier_file_check': ag['earlier_file_check']} if ag.get('earlier_file_check') else {}),+ 'problems': problems, 'warnings': warnings, 'agent_notes': ag.get('agent_notes', []), 'audit_tool': TOOL,+ **{k: v for k, v in ag.items() if k not in AUDIT_KEYS and not k.startswith('_')}}+ r['severity'] = severity(r)+ return r+++def severity(r):+ sev = 'blocker' if r['problems'] else 'warning' if r['warnings'] else 'ok'+ return sev if r['in_scope'] else 'info' if sev != 'ok' else 'ok'+++def audit(root='.', manifests='audio/assets/manifests', report=None, existing=None, fetch=False, fetch_all=False,+ decode='scope'):+ """audit every manifest entry; returns {'summary', 'counts', 'entries', 'notes'}"""+ mdir = manifests if os.path.isabs(manifests) else os.path.join(root, manifests)+ if not os.path.isdir(mdir):+ raise FileNotFoundError(f'manifest folder not found: {mdir} (pass --manifests)')+ entries, notes = L.load_manifests(mdir)+ if not entries:+ raise FileNotFoundError(f'no manifest entries in {mdir}' + (f' ({"; ".join(notes)})' if notes else ''))+ used = None+ if report:+ rp = report if os.path.isabs(report) else os.path.join(root, report)+ used = {L.rel(u, root).lower() for u in json.load(open(rp, encoding='utf-8')).get('files_used', [])}+ old = {}+ existing = existing and (existing if os.path.isabs(existing) else os.path.join(root, existing))+ if existing and os.path.exists(existing):+ for r in json.load(open(existing, encoding='utf-8')):+ p = L.rel(((r.get('file_check') or {}).get('path') or ''), root).lower()+ old[(str(r.get('id')), p)] = old[(str(r.get('id')), '')] = r+ rows, first = [], {}+ for e in entries:+ f = L.rel(e['file'], root).lower()+ o = old.get((str(e.get('id')), f)) or old.get((str(e.get('id')), ''))+ if f in first: # the same recording in another manifest: audited once, under its first entry+ r0, e0 = first[f]+ note = f'duplicate of {e0.get("id")} in {os.path.basename(e0["_manifest"])}'+ lic = L.classify(e.get('license'))+ if lic['status'] != 'allowed':+ r0['problems'].append(f'{os.path.basename(e["_manifest"])} ({e.get("id")}) lists this file under a '+ f'{lic["status"]} licence "{e.get("license")}" ({lic["reason"]}): one recording has '+ 'one licence; fix the manifest or reject the file')+ elif lic['key'] != r0['licence']:+ r0['warnings'].append(f'{os.path.basename(e["_manifest"])} lists this file as {lic["key"]}, not '+ f'{r0["licence"]}')+ r0['severity'] = severity(r0)+ rows.append({**r0, 'id': e.get('id'), 'manifest': e['_manifest'], 'need': e.get('need'), 'rank': e.get('rank'),+ 'in_scope': False, 'severity': 'info', 'duplicate_of': e0.get('id'), 'problems': [],+ 'warnings': [note]})+ continue+ r = audit_entry(e, root, used, o, decode)+ first[f] = (r, e)+ rows.append(r)+ if fetch or fetch_all:+ n = 0+ for r, e in zip(rows, entries):+ if not (fetch_all or r['in_scope']) or not e.get('download_url') or not r['file_check']['exists']:+ continue+ if n:+ time.sleep(1.5)+ n += 1+ ok, why = fetch_match(e['download_url'], os.path.join(root, e['file']))+ r['file_check']['bytes_match_download_url'] = ok+ if why:+ r['file_check']['fetch_note'] = why+ if ok is False:+ r['warnings'].append('local bytes differ from download_url (re-encoded or edited file?)')+ r['severity'] = severity(r)+ if used is not None:+ listed = {L.rel(e['file'], root).lower() for e in entries}+ for u in sorted(used - listed):+ rows.append({'id': None, 'manifest': None, 'in_scope': True, 'severity': 'blocker', 'licence': 'none',+ 'licence_status': 'unknown', 'file_check': {'path': u}, 'problems': ['used in the film but in no manifest'],+ 'warnings': [], 'agent_notes': [], 'audit_tool': TOOL})+ sc = [r for r in rows if r['in_scope']]+ cnt = {k: sum(1 for r in sc if r['severity'] == k) for k in ('ok', 'warning', 'blocker')}+ by_lic = {}+ for r in sc:+ by_lic[r['licence']] = by_lic.get(r['licence'], 0) + 1+ scope = 'used in the film' if used is not None else 'in scope'+ summary = (f'{len(rows)} manifest entries, {len(sc)} {scope}: {cnt["ok"]} ok, {cnt["warning"]} with warnings, '+ f'{cnt["blocker"]} blocked. Licences {scope}: ' + ', '.join(f'{k} {v}' for k, v in sorted(by_lic.items())) + '.')+ n_notes = sum(1 for r in sc if r.get('agent_notes'))+ if n_notes:+ summary += f' {n_notes} of them carry auditor notes (credit forms, restrictions): read agent_notes.'+ now = sum(1 for r in sc if (r.get('file_check') or {}).get('bytes_match_download_url'))+ before = sum(1 for r in sc if not (r.get('file_check') or {}).get('bytes_match_download_url')+ and (r.get('earlier_file_check') or {}).get('bytes_match_download_url'))+ if now:+ summary += (f' Bytes match download_url: {now} checked in this run' + (f', {before} more recorded by an earlier '+ 'audit (earlier_file_check)' if before else '') + '.')+ elif before:+ summary += f' Bytes match download_url for {before} according to an earlier audit (not re-checked: --fetch).'+ if cnt['blocker']:+ summary += ' Fix every blocker before publishing (see the table).'+ return {'summary': summary, 'counts': {**cnt, 'entries': len(rows), 'in_scope': len(sc)}, 'licences_in_scope': by_lic,+ 'notes': notes, 'entries': rows}+++def table(rows):+ order = {'blocker': 0, 'warning': 1, 'info': 2, 'ok': 3}+ out = ['| id | in film | severity | licence | verified | credit | file | issues |', '|---|---|---|---|---|---|---|---|']+ for r in sorted(rows, key=lambda r: (not r['in_scope'], order.get(r['severity'], 9), str(r.get('id')))):+ fc = r.get('file_check') or {}+ fdesc = (f"{fc.get('codec')} {fc.get('sample_rate', 0) // 1000 if fc.get('sample_rate') else '?'}k "+ f"{fc.get('channels')}ch {fc.get('duration_s')}s") if fc.get('exists') else 'MISSING'+ issues = '; '.join(r.get('problems', []) + r.get('warnings', []))[:300].replace('|', '/')+ out.append(f"| {r.get('id')} | {'yes' if r['in_scope'] else 'no'} | {r['severity']} | {r.get('licence')} | "+ f"{'yes' if r.get('licence_confirmed') else 'NO'} | "+ f"{'n/a' if not r.get('attribution_required') else 'ok' if r.get('attribution_ok') else 'MISSING'} | "+ f"{fdesc} | {issues} |")+ return '\n'.join(out) + '\n'+++def backup(out):+ """before overwriting an audit: keep the previous one as <name>.prev.json, and a hand-written one (no audit_tool+ field) once as <name>.handwritten.json; returns the paths written"""+ if not os.path.exists(out):+ return []+ stem = os.path.splitext(out)[0]+ done = [stem + '.prev.json']+ shutil.copy(out, done[0])+ try:+ rows = json.load(open(out, encoding='utf-8'))+ hand = isinstance(rows, list) and rows and not any(isinstance(r, dict) and r.get('audit_tool') for r in rows)+ except ValueError:+ hand = True+ if hand and not os.path.exists(stem + '.handwritten.json'):+ shutil.copy(out, stem + '.handwritten.json')+ done.append(stem + '.handwritten.json')+ return done+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')+ ap.add_argument('--manifests', default='audio/assets/manifests')+ ap.add_argument('--report', help='mix_report.json: in_scope = the files the film uses')+ ap.add_argument('--out', help='audit JSON to write (default <manifests>/audit.json); agent fields are merged from it')+ ap.add_argument('--existing', help='read agent fields from this audit instead of --out')+ ap.add_argument('--table', help='Markdown table (default: --out with .md)')+ ap.add_argument('--fetch', action='store_true', help='compare bytes with download_url for recordings in the film')+ ap.add_argument('--fetch-all', action='store_true', help='... for every entry')+ ap.add_argument('--decode', choices=('scope', 'all', 'none'), default='scope',+ help='full decode check for in-scope recordings (default), all, or none')+ ap.add_argument('--json', action='store_true', help='print the full result as JSON')+ a = ap.parse_args(argv)+ root = os.path.abspath(a.root)+ mdir = a.manifests if os.path.isabs(a.manifests) else os.path.join(root, a.manifests)+ out = a.out or os.path.join(mdir, 'audit.json')+ out = os.path.normpath(out if os.path.isabs(out) else os.path.join(root, out))+ try:+ res = audit(root, a.manifests, a.report, a.existing or out, a.fetch, a.fetch_all, a.decode)+ except FileNotFoundError as e:+ say(f'audit.py: {e}')+ return 2+ os.makedirs(os.path.dirname(out), exist_ok=True)+ backups = backup(out)+ json.dump(res['entries'], open(out, 'w', encoding='utf-8'), indent=1, ensure_ascii=False)+ tab = a.table or os.path.splitext(out)[0] + '.md'+ tab = os.path.normpath(tab if os.path.isabs(tab) else os.path.join(root, tab))+ open(tab, 'w', encoding='utf-8').write('# Licence and file audit\n\n' + res['summary'] + '\n\n' + table(res['entries']))+ if a.json:+ print(json.dumps(res, indent=1, ensure_ascii=True))+ else:+ say(res['summary'])+ for n in res['notes']:+ say(f' note: {n}')+ groups = {}+ for r in res['entries']:+ if r['in_scope'] and r['severity'] != 'ok':+ for m in r['problems'] + r['warnings']:+ groups.setdefault((m in r['problems'], m), []).append(str(r.get('id')))+ for (blk, m), ids in sorted(groups.items(), key=lambda kv: (not kv[0][0], -len(kv[1]))):+ say(f" {'BLOCKER' if blk else 'warning'} x{len(ids)}: {m} [{', '.join(ids[:8])}{' ...' if len(ids) > 8 else ''}]")+ say(f'wrote {out} and {tab}' + (f' (previous audit kept as {", ".join(backups)})' if backups else ''))+ return 1 if res['counts']['blocker'] else 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/scripts/credits.py b/skills/real-sound/scripts/credits.pynew file mode 100644index 0000000..29bda4b--- /dev/null+++ b/skills/real-sound/scripts/credits.py@@ -0,0 +1,213 @@+#!/usr/bin/env python+"""Credits for every recording audible in the film, built from the mix report and the licence manifests.++ python -I <skill>/scripts/credits.py [--root .] [--report out/qa/mix_report.json] [--manifests audio/assets/manifests]+ [--design audio/sounddesign.json] [--audit audio/assets/manifests/audit.json]+ [--out CREDITS.md] [--desc out/CREDITS_description.txt] [--title "Film name"]++Reads report.files_used (written by mix.py: every recording that is audible), matches each file to its manifest entry+and writes:+ CREDITS.md music + every sound with title, author, link and licence (grouped by licence); a CC BY entry uses+ the manifest's "attribution" line (the form the author asked for) when there is one+ CREDITS_description.txt plain text to paste into the video description (CC BY credits are REQUIRED there; CC0 /+ Mixkit / Pixabay items are listed as a courtesy)+The processing sentence (trimmed, equalised, pitch-shifted ...) and the music edit sentence come from the design when+--design is given; the source names (Freesound.org, Mixkit ...) come from the manifests. With --audit, recordings the+licence audit did not confirm are listed as problems. Both texts say that every work was modified (CC BY 4.0+s.3(a)(1)(B)). Exit code 1 when anything needs fixing before publishing, 2 when the report or manifests are missing.+"""+import argparse, json, os, sys++sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+import licences as L # noqa: E402++ORDER = ['by4', 'by3', 'by25', 'by2', 'cc0', 'pd', 'mixkit-sfx', 'mixkit-music', 'pixabay']+SHORT = {'cc0': 'CC0', 'pd': 'public domain', 'mixkit-sfx': 'Mixkit free licence', 'mixkit-music': 'Mixkit free licence',+ 'pixabay': 'Pixabay licence'}+++def say(s):+ print(str(s).encode('ascii', 'replace').decode())+++def and_join(xs):+ xs = list(xs)+ return ' and '.join([', '.join(xs[:-1]), xs[-1]]) if len(xs) > 1 else ''.join(xs)+++def processing(design_path, root):+ """(sentence for the effects, sentence for the music edit, music stretched?) derived from the design"""+ if not design_path:+ return ('trimmed, layered, level-adjusted, equalised and panned for the film',+ 'Edited for the film: cut to fit the picture and levels adjusted.', False)+ import mix # read-only use of the design loader+ mix.ROOT = root+ d = mix.load_design(design_path)+ opts = list((d.get('samples') or {}).values()) + list((d.get('slices') or {}).values())+ rules, place = d.get('rules') or [], d.get('place') or []+ did = ['trimmed']+ if d.get('slices'):+ did.append('cut into single hits')+ did.append('layered' if len(rules) + len(place) > len(opts) * 0.5 or len(rules) > 1 else 'placed')+ did.append('level-adjusted')+ if any(s.get('eq') for s in opts) or any((b or {}).get('eq') for b in (d.get('buses') or {}).values()):+ did.append('equalised')+ if any(r.get('pan') not in (None, 0, 0.0) for r in rules) or any(p.get('pan') for p in place):+ did.append('panned')+ if any(s.get('pitch') or s.get('glide') for s in opts) or any(r.get('pitch') or r.get('pitch_ramp') for r in rules):+ did.append('pitch-shifted by resampling')+ fx = and_join(did)+ mu = d.get('music')+ if not mu:+ return fx + ' for the film', None, False+ segs = mu.get('segments') or []+ stretched = mu.get('stretch', 'rubberband') != 'none' and abs(mu.get('target_bpm', 120) / mu['src_bpm'] - 1) > 1e-6+ parts = ['cut and rearranged at bar lines to fit the picture' if len(segs) > 1 else 'trimmed to the picture']+ if stretched:+ parts.append(f"time-stretched from {mu['src_bpm']:g} to {mu.get('target_bpm', 120):g} BPM")+ parts.append('levels adjusted')+ if any(s.get('tail_fade') for s in segs):+ parts.append('a fade at the end')+ return fx + ' for the film', 'Edited for the film: ' + ', '.join(parts) + '.', stretched+++def credits(root='.', report='out/qa/mix_report.json', manifests='audio/assets/manifests', design=None, audit=None,+ title=None):+ """build the CREDITS.md and description texts; returns {'summary', 'markdown', 'description', 'rows', 'problems'}"""+ R = lambda p: p if os.path.isabs(p) else os.path.join(root, p)+ if not os.path.exists(R(report)):+ raise FileNotFoundError(f'mix report not found: {R(report)} (run mix.py first, or pass --report)')+ if not os.path.isdir(R(manifests)):+ raise FileNotFoundError(f'manifest folder not found: {R(manifests)} (pass --manifests)')+ rep = json.load(open(R(report), encoding='utf-8'))+ used = [L.rel(f, root) for f in rep.get('files_used', [])]+ entries, notes = L.load_manifests(R(manifests))+ by_file = {}+ for e in entries:+ by_file.setdefault(L.rel(e['file'], root).lower(), e)+ confirmed = None+ if audit and os.path.exists(R(audit)):+ confirmed = {L.rel((r.get('file_check') or {}).get('path') or '', root).lower(): r+ for r in json.load(open(R(audit), encoding='utf-8'))}+ rows, problems = [], []+ for f in used:+ e = by_file.get(f.lower())+ if not e:+ problems.append(f'no manifest entry for {f}')+ continue+ lic = L.classify(e.get('license'))+ if lic['status'] != 'allowed':+ problems.append(f'{f}: licence "{e.get("license")}" is {lic["status"]} ({lic["reason"]})')+ if lic['attribution'] and not (e.get('author') and e.get('title') and e.get('page_url')):+ problems.append(f'{f}: CC BY needs author, title and page_url for the credit')+ if confirmed is not None:+ a = confirmed.get(f.lower())+ if not a:+ problems.append(f'{f}: the licence audit has no record of this file (run audit.py, record licence_seen)')+ elif not a.get('licence_confirmed'):+ problems.append(f'{f}: audit.json does not confirm this licence (licence_seen / licence_confirmed missing)')+ rows.append({'file': f, 'id': e.get('id'), 'title': e.get('title'), 'author': e.get('author'),+ 'url': e.get('page_url'), 'lic': lic, 'attribution': e.get('attribution'),+ 'category': e.get('category'), 'source': L.source_name(e)})+ music = [r for r in rows if r['category'] == 'music' or '/music/' in '/' + r['file'].lower()]+ sfx = [r for r in rows if r not in music]+ rank = lambda k: ORDER.index(k) if k in ORDER else len(ORDER)+ sfx.sort(key=lambda r: (rank(r['lic']['key']), (r['author'] or '').lower(), r['title'] or ''))+ fx_did, mu_did, stretched = processing(design and R(design), root)++ def line(r):+ """the author's requested credit (manifest "attribution") or a built one, completed with the source link and+ the licence link when the requested form leaves them out (CC BY asks for both)"""+ name, url = r['lic']['name'] or r['lic']['key'], r['lic']['url']+ if not (r['lic']['attribution'] and r['attribution']):+ return f'"{r["title"]}" by {r["author"]} ({r["url"]}), licensed under {name}' + (f' ({url})' if url else '')+ base = r['attribution'].strip().rstrip('.')+ if r['url'] and 'http' not in base and 'www.' not in base:+ base += f' ({r["url"]})'+ if url and 'creativecommons.org' not in base:+ base += f' ({url})'+ return base++ srcs = {}+ for r in sfx:+ srcs[r['source']] = srcs.get(r['source'], 0) + 1+ src_txt = and_join(s for s, _ in sorted(srcs.items(), key=lambda kv: -kv[1]))+ md = [f'# Credits{": " + title if title else ""}', '',+ 'All credits live here and in the video description (CC BY credits are required there).', '']+ if music:+ md += ['## Music', '']+ for r in music:+ md.append(f'- {line(r)}.')+ if mu_did:+ md.append(f' {mu_did}')+ md.append('')+ if sfx:+ md += ['## Sound effects and ambience', '',+ f'Real recordings from {src_txt or "the sources below"}, {fx_did}. Nothing in the soundtrack is '+ 'synthesized.', '']+ cur = None+ for r in sfx:+ if r['lic']['key'] != cur:+ cur = r['lic']['key']+ md += ['', f'### {r["lic"]["name"] or cur}', '']+ md.append(f'- {line(r)}')+ tools = 'ffmpeg, NumPy/SciPy, soxr, librosa and pyloudnorm' + (', Rubber Band (music tempo)' if stretched else '')+ md += ['', '## Tools', '', f'Edited, mixed and mastered with the real-sound skill (mix.py, master.py) using {tools}.', '']+ if problems:+ md += ['', '## PROBLEMS (fix before publishing)', ''] + [f'- {p}' for p in problems]+ d = []+ if music: # CC BY 4.0 s.3(a)(1)(B): say that the work was modified+ d.append('Music:')+ d += [line(r) for r in music] + ([mu_did] if mu_did else []) + ['']+ by = [r for r in sfx if r['lic']['attribution']]+ if by:+ d.append('Sound effects (CC BY):')+ d += [line(r) for r in by]+ d += [f'All sound effects and ambiences were modified: {fx_did}.', '']+ rest = sorted({r['author'] for r in sfx if not r['lic']['attribution'] and r['author']}, key=str.lower)+ if rest:+ kinds = sorted({SHORT.get(r['lic']['key'], r['lic']['key']) for r in sfx if not r['lic']['attribution']})+ d.append(f'More sounds ({", ".join(kinds)}) by: ' + ', '.join(rest))+ if sfx and not by:+ d.append(f'All sound effects and ambiences were {fx_did}.')+ n_by = sum(1 for r in rows if r['lic']['attribution'])+ summary = (f'{len(rows)} recordings credited ({len(music)} music, {len(sfx)} sound effects / ambience; {n_by} CC BY '+ f'need the credit in the video description)' + (f'; {len(problems)} PROBLEM(S) to fix before publishing'+ if problems else '; no problems'))+ if any(r['lic']['attribution'] for r in music):+ summary += ('. CC BY music: put the credit in the description; if a Content ID claim appears, dispute it with '+ 'the licence link (references/licences.md)')+ return {'summary': summary, 'markdown': '\n'.join(md) + '\n', 'description': '\n'.join(d) + '\n', 'rows': [+ {k: (v['key'] if k == 'lic' else v) for k, v in r.items()} for r in rows], 'problems': problems, 'notes': notes}+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')+ ap.add_argument('--report', default='out/qa/mix_report.json')+ ap.add_argument('--manifests', default='audio/assets/manifests')+ ap.add_argument('--design', help='the design file (derives the processing and music-edit sentences)')+ ap.add_argument('--audit', help='audit.json from audit.py: unconfirmed licences become problems')+ ap.add_argument('--out', default='CREDITS.md')+ ap.add_argument('--desc', default='out/CREDITS_description.txt')+ ap.add_argument('--title', help='film name for the heading')+ a = ap.parse_args(argv)+ root = os.path.abspath(a.root)+ try:+ res = credits(root, a.report, a.manifests, a.design, a.audit, a.title)+ except FileNotFoundError as e:+ say(f'credits.py: {e}')+ return 2+ for p, text in ((a.out, res['markdown']), (a.desc, res['description'])):+ p = p if os.path.isabs(p) else os.path.join(root, p)+ os.makedirs(os.path.dirname(p) or '.', exist_ok=True)+ open(p, 'w', encoding='utf-8').write(text)+ say(res['summary'])+ say(f'wrote {a.out} and {a.desc}')+ for p in res['problems']:+ say(f'PROBLEM: {p}')+ return 1 if res['problems'] else 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/scripts/licences.py b/skills/real-sound/scripts/licences.pynew file mode 100644index 0000000..42cbcff--- /dev/null+++ b/skills/real-sound/scripts/licences.py@@ -0,0 +1,122 @@+"""Licence classification and manifest loading shared by audit.py and credits.py (real-sound skill).++ import licences as L+ L.classify('CC BY 4.0') # {'key': 'by4', 'status': 'allowed', 'name': 'CC BY 4.0', 'url': ..., 'attribution': True}+ L.classify('CC BY-NC 4.0') # status 'forbidden', reason 'NonCommercial: ...'+ entries, notes = L.load_manifests('audio/assets/manifests')++status is 'allowed' (CC0, public domain, CC BY 2.0-4.0, Mixkit free licences, Pixabay Content License), 'forbidden'+(NonCommercial, NoDerivatives, ShareAlike, Sampling+, personal-use, custom, editorial or disputed licences) or 'unknown'+(anything else: read the item's own page and fix the manifest). The policy and how to verify a licence on each site:+references/licences.md.+"""+import glob+import json+import os+import re++CC = 'https://creativecommons.org/'+FIXED = {+ 'cc0': ('CC0 1.0 (public domain dedication)', CC + 'publicdomain/zero/1.0/', False),+ 'pd': ('Public Domain Mark 1.0', CC + 'publicdomain/mark/1.0/', False),+ 'mixkit-sfx': ('Mixkit Sound Effects Free License', 'https://mixkit.co/license/#sfxFree', False),+ 'mixkit-music': ('Mixkit Stock Music Free License', 'https://mixkit.co/license/#musicFree', False),+ 'pixabay': ('Pixabay Content License', 'https://pixabay.com/service/license-summary/', False),+}+FORBID = [ # (pattern on the lower-cased licence text, reason)+ (r'disputed', 'disputed: the sources disagree about the licence'),+ (r'\bnc\b|non-?commercial', 'NonCommercial: a launch video is commercial use'),+ (r'\bnd\b|no-?deriv', 'NoDerivatives: syncing to picture, trimming and editing are derivatives'),+ (r'\bsa\b|share-?alike', 'ShareAlike: the whole film would have to be released under CC BY-SA'),+ (r'sampling', 'Sampling+ / Sampling licences are retired and restrict use'),+ (r'personal', 'personal use only'),+ (r'custom|free with credit|social media', 'a custom licence: not a standard licence the audit can trust'),+ (r'editorial', 'editorial use only'),+]+RESERVED = r'all rights reserved|\bcopyrighted\b' # forbidden alone; next to a free licence it is ambiguous+++def classify(text):+ """licence text (as written in a manifest or seen on a page) -> {key, status, name, url, attribution, reason}"""+ s = (text or '').strip().lower()+ if not s:+ return {'key': 'none', 'status': 'unknown', 'name': None, 'url': None, 'attribution': False,+ 'reason': 'no licence recorded'}+ for pat, why in FORBID:+ if re.search(pat, s.replace('_', ' ')):+ return {'key': 'forbidden', 'status': 'forbidden', 'name': text, 'url': None, 'attribution': False, 'reason': why}+ if 'restricted' in s:+ return {'key': 'restricted', 'status': 'unknown', 'name': text, 'url': None, 'attribution': False,+ 'reason': 'a restricted licence (e.g. Mixkit Restricted): read its terms for this use'}+ key = None+ if re.search(r'\bcc0\b|cc-0|creative commons 0|publicdomain/zero|\bzero\b|public domain dedication', s):+ key = 'cc0'+ elif 'public domain' in s or 'publicdomain/mark' in s:+ key = 'pd'+ elif 'mixkit' in s:+ key = 'mixkit-music' if 'music' in s else 'mixkit-sfx'+ elif 'pixabay' in s:+ key = 'pixabay'+ res = None+ if key:+ name, url, attr = FIXED[key]+ res = {'key': key, 'status': 'allowed', 'name': name, 'url': url, 'attribution': attr, 'reason': None}+ m = re.search(r'(?:\bby\b|attribution)[^0-9]{0,24}([1-4](?:\.[05])?)', s)+ if m and not res:+ ver = {'1': '1.0', '2': '2.0', '3': '3.0', '4': '4.0'}.get(m.group(1), m.group(1))+ if ver.startswith('1'):+ return {'key': 'by1', 'status': 'unknown', 'name': f'CC BY {ver}', 'url': None, 'attribution': True,+ 'reason': 'CC BY 1.0 is rare: confirm it on the item page'}+ res = {'key': 'by' + ver.replace('.0', '').replace('.', ''), 'status': 'allowed', 'name': f'CC BY {ver}',+ 'url': f'{CC}licenses/by/{ver}/', 'attribution': True, 'reason': None}+ reserved = re.search(RESERVED, s)+ if reserved and not res:+ return {'key': 'forbidden', 'status': 'forbidden', 'name': text, 'url': None, 'attribution': False,+ 'reason': 'all rights reserved'}+ if reserved and 'all rights reserved' in s: # a free licence and "all rights reserved" contradict each other+ return {'key': 'unknown', 'status': 'unknown', 'name': text, 'url': None, 'attribution': res['attribution'],+ 'reason': f'names {res["name"]} and "all rights reserved": read the item page'}+ return res or {'key': 'unknown', 'status': 'unknown', 'name': text, 'url': None, 'attribution': False,+ 'reason': 'not a licence this skill recognises: read the item page and record the exact licence'}+++def rel(path, root):+ """path relative to the project root with forward slashes (case-insensitive prefix match, for Windows)"""+ p = (path or '').replace('\\', '/')+ r = os.path.abspath(root).replace('\\', '/').rstrip('/') + '/'+ return p[len(r):] if p.lower().startswith(r.lower()) else p+++def load_manifests(folder, skip=('audit.json',)):+ """every manifest entry (dicts with a "file") from folder/*.json, each with "_manifest"; plus notes on skipped files.+ A manifest is a JSON list of entries, or an object holding one under "entries" / "items"."""+ entries, notes = [], []+ for m in sorted(glob.glob(os.path.join(folder, '*.json'))):+ if os.path.basename(m) in skip or os.path.basename(m).startswith('audit.'): # audits and their backups+ continue+ try:+ d = json.load(open(m, encoding='utf-8'))+ except Exception as e: # noqa: BLE001+ notes.append(f'{os.path.basename(m)}: not valid JSON ({e})')+ continue+ if isinstance(d, dict):+ d = d.get('entries') or d.get('items')+ if not isinstance(d, list):+ notes.append(f'{os.path.basename(m)}: not a list of entries (a decision record?), skipped')+ continue+ for e in d:+ if isinstance(e, dict) and e.get('file'):+ entries.append({**e, '_manifest': m.replace('\\', '/')})+ return entries, notes+++def source_name(entry):+ """a readable source name from the entry's page_url host (Freesound.org, Mixkit, ...) or its "source" field"""+ url = entry.get('page_url') or entry.get('download_url') or ''+ host = re.sub(r'^www\.', '', re.sub(r'^[a-z]+://', '', url.lower()).split('/')[0])+ known = {'freesound.org': 'Freesound.org', 'mixkit.co': 'Mixkit', 'pixabay.com': 'Pixabay',+ 'archive.org': 'the Internet Archive', 'commons.wikimedia.org': 'Wikimedia Commons', 'github.com': 'GitHub'}+ for k, v in known.items():+ if host.endswith(k):+ return v+ return host or (entry.get('source') or 'unknown source').split(' (')[0]diff --git a/skills/real-sound/scripts/master.py b/skills/real-sound/scripts/master.pynew file mode 100644index 0000000..cf9c565--- /dev/null+++ b/skills/real-sound/scripts/master.py@@ -0,0 +1,342 @@+#!/usr/bin/env python3+"""+master.py: cinetic's mix bus and mastering chain, plus loudness measurement.++Chain (finish): 28 Hz high-pass -> mono below 120 Hz -> 2:1 bus compressor above -14 dBFS+(5 ms RMS detector, 15 ms attack, 120 ms release) -> soft saturation + air -> loudness passes to+the target LUFS, each followed by a true-peak lookahead limiter (4x oversampled detection, 4 ms+lookahead, 80 ms release, ceiling 0.77 linear = -2.3 dBFS, which leaves room for AAC overshoot)+-> edge fades, always: a 3 ms raised-cosine fade-in from exact zero at sample 0 (a sound already at+level on sample 0 clicks and cuts in mid-action) and a raised-cosine fade-out of at least 10 ms+(60 ms by default) onto the last sample -> last 480 samples zeroed.++score.py imports sidechain_times(), finish() and measure(). Standalone use:++ python3 master.py mix.wav public/audio/soundtrack.wav --lufs -14 --ceiling 0.77+ python3 master.py music.wav ducked.wav --sidechain kick.wav --sc-depth 0.55 --lufs -14+ python3 master.py --measure public/audio/soundtrack.wav --json out/qa/loudness.json++Exit code 0 when the output meets the gate (|LUFS - target| <= --tol and true peak <= --max-tp),+1 when it does not or on error. The JSON report holds integrated LUFS, true peak (dBTP, 4x+oversampled), LRA, the loudest momentary block and its time, limiter gain reduction and tail level.+"""+import argparse+import json+import os+import sys++import numpy as np+from scipy import signal+from scipy.ndimage import maximum_filter1d, minimum_filter1d++SR = 48000+HEAD_FADE = 0.003 # s, raised-cosine fade-in at sample 0, applied after the limiter on every master+END_FADE_MIN = 0.010 # s, the shortest fade-out onto the last sample, whatever end_fade asks for+++# ------------------------------------------------------------------------------------------+# measurement+# ------------------------------------------------------------------------------------------+def _stereo(x):+ x = np.asarray(x, dtype=np.float64)+ return np.stack([x, x], axis=1) if x.ndim == 1 else x+++def _kw(x):+ """BS.1770 K-weighting (48 kHz coefficients): high shelf then RLB high-pass."""+ b1, a1 = [1.53512485958697, -2.69169618940638, 1.19839281085285], [1.0, -1.69065929318241, 0.73248077421585]+ b2, a2 = [1.0, -2.0, 1.0], [1.0, -1.99004745483398, 0.99007225036621]+ return signal.lfilter(b2, a2, signal.lfilter(b1, a1, x, axis=0), axis=0)+++def blocks(x, win=0.4, hop=0.1):+ """Loudness (LUFS) of sliding blocks; 0.4 s = momentary, 3 s = short-term. Returns (t_mid, L)."""+ y = _kw(_stereo(x))+ p = np.cumsum(np.concatenate([np.zeros((1, 2)), y ** 2]), axis=0)+ w, h = int(win * SR), int(hop * SR)+ if len(y) < w:+ return np.array([len(y) / SR / 2]), np.array([-70.0])+ starts = np.arange(0, len(y) - w + 1, h)+ ms = (p[starts + w] - p[starts]) / w+ L = -0.691 + 10 * np.log10(np.maximum(ms.sum(axis=1), 1e-12))+ return (starts + w / 2) / SR, L+++def integrated(x):+ import pyloudnorm as pyln+ return float(pyln.Meter(SR).integrated_loudness(_stereo(x)))+++def lra(x):+ """EBU loudness range: 10th-95th percentile of gated 3 s short-term loudness."""+ _, st = blocks(x, 3.0, 0.1)+ st = st[st > -70]+ if len(st) < 2:+ return 0.0+ rel = 10 * np.log10(np.mean(10 ** (st / 10))) - 20+ st = st[st > rel]+ return float(np.percentile(st, 95) - np.percentile(st, 10)) if len(st) > 1 else 0.0+++def true_peak_db(x):+ up = signal.resample_poly(_stereo(x), 4, 1, axis=0)+ return float(20 * np.log10(np.max(np.abs(up)) + 1e-12))+++def measure(x):+ x = _stereo(x)+ tm, m = blocks(x, 0.4, 0.1)+ k = int(np.argmax(m))+ tail = x[-int(0.05 * SR):]+ return {+ 'lufs': round(integrated(x), 2),+ 'true_peak_db': round(true_peak_db(x), 2),+ 'sample_peak_db': round(float(20 * np.log10(np.max(np.abs(x)) + 1e-12)), 2),+ 'lra': round(lra(x), 2),+ 'momentary_max': round(float(m[k]), 2),+ 'momentary_max_t': round(float(tm[k]), 3),+ 'tail_50ms_db': round(float(20 * np.log10(np.sqrt(np.mean(tail ** 2)) + 1e-12)), 1),+ 'last_480_zero': bool(np.all(x[-480:] == 0)),+ 'seconds': round(len(x) / SR, 4),+ }+++# ------------------------------------------------------------------------------------------+# dynamics+# ------------------------------------------------------------------------------------------+def sidechain_times(hits, n, depth=0.55, tau=0.14, attack=0.003, length=0.6):+ """Duck gain (n, 1) from (t_seconds, weight) pairs: each hit pulls the music down by+ depth*weight with a 3 ms attack and recovers with time constant tau."""+ g = np.ones(n)+ a = int(attack * SR)+ ramp = 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, max(a, 1)))+ seg_t = np.arange(int(length * SR)) / SR+ shape = np.exp(-seg_t / tau)+ shape[:a] *= ramp+ for t, w in hits:+ i = int(round(t * SR))+ if i >= n:+ continue+ j = min(n, i + len(shape))+ s0 = max(0, -i)+ g[max(i, 0):j] = np.minimum(g[max(i, 0):j], 1 - depth * w * shape[s0:j - i])+ return g[:, None]+++def sidechain_key(key, depth=0.55, tau=0.14, hop=48):+ """Duck gain (n, 1) that follows a key signal (a kick stem): instant attack, release tau."""+ k = np.abs(_stereo(key)).max(axis=1)+ nb = int(np.ceil(len(k) / hop))+ v = np.pad(k, (0, nb * hop - len(k))).reshape(nb, hop).max(axis=1)+ v = v / (np.percentile(v[v > 0], 99.9) if np.any(v > 0) else 1.0)+ rel = np.exp(-hop / (tau * SR))+ e = np.empty(nb)+ cur = 0.0+ for i, x in enumerate(v.tolist()):+ cur = x if x > cur else cur * rel+ e[i] = cur+ g = 1 - depth * np.clip(e, 0, 1)+ return np.interp(np.arange(len(k)), np.arange(nb) * hop + hop / 2, g)[:, None]+++def mono_below(x, fc=120):+ """Sum the stereo image to mono below fc so the low end is centred and translates to phones."""+ x = _stereo(x)+ mid = x.mean(axis=1, keepdims=True)+ side = signal.sosfilt(signal.butter(2, fc, 'high', fs=SR, output='sos'), (x[:, :1] - x[:, 1:]) / 2, axis=0)+ return np.concatenate([mid + side, mid - side], axis=1)+++def bus_comp(x, thr=-14.0, ratio=2.0, attack=0.015, release=0.12, rms=0.005, hop=48):+ """Glue compressor: true RMS detector (rms seconds), gain computed at a 1 ms control rate.+ Returns (y, gain_reduction_db per sample)."""+ x = _stereo(x)+ det = signal.sosfilt(signal.butter(1, 1 / (2 * np.pi * rms), 'low', fs=SR, output='sos'), np.mean(x ** 2, axis=1))+ nb = int(np.ceil(len(x) / hop))+ d = np.pad(det, (0, nb * hop - len(det)), mode='edge').reshape(nb, hop).max(axis=1)+ over = np.maximum(0, 10 * np.log10(np.maximum(d, 1e-12)) - thr)+ target = -over * (1 - 1 / ratio)+ att, rel = np.exp(-hop / (attack * SR)), np.exp(-hop / (release * SR))+ g = np.empty(nb)+ cur = 0.0+ for i, v in enumerate(target.tolist()):+ cur = v + (cur - v) * (att if v < cur else rel)+ g[i] = cur+ gr = np.interp(np.arange(len(x)), np.arange(nb) * hop + hop / 2, g)+ return x * (10 ** (gr / 20))[:, None], gr+++def limiter(x, ceiling=0.77, look=0.004, release=0.08, hop=16):+ """True-peak lookahead limiter. Detection on a 4x oversampled copy catches inter-sample+ overs; the gain is computed per 16-sample block (min over neighbouring blocks, so the+ interpolated gain never exceeds what any sample needs), attacks over the lookahead and+ releases with time constant `release`. Returns (y, gain per sample)."""+ x = _stereo(x)+ n = len(x)+ up = np.abs(signal.resample_poly(x, 4, 1, axis=0)).max(axis=1)+ a = np.pad(up, (0, max(0, 4 * n - len(up))))[:4 * n].reshape(n, 4).max(axis=1)+ la = max(hop, int(look * SR))+ peak = maximum_filter1d(a, size=la, origin=-(la // 2), mode='nearest') # max over [i, i+la)+ need = np.minimum(1.0, ceiling / np.maximum(peak, 1e-9))+ nb = int(np.ceil(n / hop))+ gb = np.pad(need, (0, nb * hop - n), mode='edge').reshape(nb, hop).min(axis=1)+ k = max(3, la // hop)+ win = np.hanning(k + 2)[1:-1]+ win /= win.sum()+ gb = np.minimum(gb, np.convolve(np.pad(gb, (len(win) // 2, len(win) - 1 - len(win) // 2), mode='edge'), win, mode='valid'))+ rel = np.exp(-hop / (release * SR))+ out = np.empty(nb)+ cur = 1.0+ for i, v in enumerate(gb.tolist()):+ cur = v if v < cur else v + (cur - v) * rel+ out[i] = cur+ out = minimum_filter1d(out, 3, mode='nearest')+ g = np.minimum(np.interp(np.arange(n), np.arange(nb) * hop + hop / 2, out), need)+ return x * g[:, None], g+++def loudness_passes(x, target=-14.0, ceiling=0.77, passes=3, tol=0.05, max_passes=8):+ """Normalize to target LUFS then limit; repeat (limiting lowers loudness) until within tol."""+ g = np.ones(len(x))+ for p in range(max_passes):+ x = x * 10 ** ((target - integrated(x)) / 20)+ x, g = limiter(x, ceiling)+ if p + 1 >= passes and abs(integrated(x) - target) <= tol:+ break+ return x, g+++def glue(x, drive=1.15, air=0.25):+ """Soft saturation and a gentle high shelf (air)."""+ x = np.tanh(x * 0.9 * drive) / np.tanh(drive)+ if air:+ x = x + signal.sosfilt(signal.butter(1, 2500, 'high', fs=SR, output='sos'), x, axis=0) * air+ return x+++def edge_fades(x, head=HEAD_FADE, tail=END_FADE_MIN):+ """Raised-cosine fade-in over `head` s from exact zero at sample 0, and fade-out over `tail` s+ to zero on the last sample. Run after the limiter, so nothing raises the edges again."""+ x = _stereo(x).copy()+ n = len(x)+ for d, rising in ((head, True), (tail, False)):+ k = min(n, int(round(d * SR)))+ if k > 1:+ w = 0.5 - 0.5 * np.cos(np.pi * np.arange(k) / (k - 1)) # 0 -> 1, raised cosine+ if rising:+ x[:k] *= w[:, None]+ else:+ x[-k:] *= w[::-1, None]+ return x+++def finish(mix, lufs=-14.0, ceiling=0.77, mono_hz=120, comp=True, drive=1.15, air=0.25,+ hp_hz=28, fade=0.0, end_fade=0.06, tail_zero=480):+ """Full chain for a summed mix. Returns (master, info) where info has the limiter trace."""+ x = _stereo(mix)+ if hp_hz:+ x = signal.sosfilt(signal.butter(2, hp_hz, 'high', fs=SR, output='sos'), x, axis=0)+ if mono_hz:+ x = mono_below(x, mono_hz)+ gr = np.zeros(len(x))+ if comp:+ x, gr = bus_comp(x)+ if drive:+ x = glue(x, drive, air)+ if fade and fade > 0:+ t = np.arange(len(x)) / SR+ x = x * (np.clip((len(x) / SR - 0.03 - t) / fade, 0, 1) ** 2)[:, None]+ x, g = loudness_passes(x, lufs, ceiling)+ x = edge_fades(x, HEAD_FADE, max(end_fade or 0.0, END_FADE_MIN))+ if tail_zero:+ x[-tail_zero:] = 0+ lim_db = -20 * np.log10(np.maximum(g, 1e-9))+ return x, {'limiter_gr_db': lim_db, 'comp_gr_db': -gr}+++def gate(report, target, tol, max_tp):+ fails = []+ if abs(report['lufs'] - target) > tol:+ fails.append(f"integrated {report['lufs']} LUFS not within {target}+-{tol}")+ if report['true_peak_db'] > max_tp:+ fails.append(f"true peak {report['true_peak_db']} dBTP above {max_tp}")+ if not report['last_480_zero']:+ fails.append('last 480 samples are not digital zero')+ return fails+++# ------------------------------------------------------------------------------------------+# CLI+# ------------------------------------------------------------------------------------------+def _read(path):+ import soundfile as sf+ x, sr = sf.read(path, always_2d=True, dtype='float64')+ if x.shape[1] == 1:+ x = np.repeat(x, 2, axis=1)+ x = x[:, :2]+ if sr != SR:+ from math import gcd+ g = gcd(SR, sr)+ x = signal.resample_poly(x, SR // g, sr // g, axis=0)+ return x+++def main(argv=None):+ ap = argparse.ArgumentParser(description='Master a WAV to target loudness, or measure one (--measure).')+ ap.add_argument('input', help='input WAV (any rate; resampled to 48 kHz)')+ ap.add_argument('output', nargs='?', help='output WAV (48 kHz, 24-bit)')+ ap.add_argument('--measure', action='store_true', help='only measure the input')+ ap.add_argument('--lufs', type=float, default=-14.0, help='target integrated loudness (default -14)')+ ap.add_argument('--ceiling', type=float, default=0.77, help='limiter ceiling, linear (default 0.77)')+ ap.add_argument('--tol', type=float, default=0.5, help='gate tolerance in LU (default 0.5)')+ ap.add_argument('--max-tp', type=float, default=-1.5, help='gate: max true peak dBTP (default -1.5)')+ ap.add_argument('--sidechain', metavar='KEY.wav', help='duck the input from this key signal first')+ ap.add_argument('--sc-depth', type=float, default=0.55)+ ap.add_argument('--sc-tau', type=float, default=0.14)+ ap.add_argument('--mono-hz', type=float, default=120, help='mono below this frequency; 0 disables')+ ap.add_argument('--no-comp', action='store_true', help='skip the bus compressor')+ ap.add_argument('--no-glue', action='store_true', help='skip saturation and air')+ ap.add_argument('--fade', type=float, default=0.0, help='quadratic fade-out over the last N seconds')+ ap.add_argument('--tail-zero', type=int, default=480, help='samples zeroed at the end (default 480)')+ ap.add_argument('--json', metavar='PATH', help="write the report JSON here ('-' for stdout)")+ a = ap.parse_args(argv)+ try:+ x = _read(a.input)+ if a.measure:+ rep = measure(x)+ fails = gate(rep, a.lufs, a.tol, a.max_tp)+ else:+ if not a.output:+ ap.error('output path required unless --measure')+ if a.sidechain:+ key = _read(a.sidechain)+ key = np.pad(key, ((0, max(0, len(x) - len(key))), (0, 0)))[:len(x)]+ x = x * sidechain_key(key, a.sc_depth, a.sc_tau)+ y, info = finish(x, a.lufs, a.ceiling, a.mono_hz, not a.no_comp,+ 0 if a.no_glue else 1.15, 0 if a.no_glue else 0.25,+ fade=a.fade, tail_zero=a.tail_zero)+ import soundfile as sf+ os.makedirs(os.path.dirname(os.path.abspath(a.output)), exist_ok=True)+ sf.write(a.output, y.astype(np.float32), SR, subtype='PCM_24')+ rep = measure(y)+ rep['limiter_max_gr_db'] = round(float(info['limiter_gr_db'].max()), 2)+ rep['output'] = a.output+ fails = gate(rep, a.lufs, a.tol, a.max_tp)+ rep['pass'] = not fails+ rep['fails'] = fails+ except Exception as e: # noqa: BLE001+ print('error:', e, file=sys.stderr)+ return 1+ if a.json == '-':+ print(json.dumps(rep, indent=1))+ elif a.json:+ os.makedirs(os.path.dirname(os.path.abspath(a.json)), exist_ok=True)+ json.dump(rep, open(a.json, 'w'), indent=1)+ print(f"{'PASS' if not fails else 'FAIL'} {rep['lufs']} LUFS TP {rep['true_peak_db']} dBTP "+ f"LRA {rep['lra']} LU loudest {rep['momentary_max']} LUFS-M at {rep['momentary_max_t']}s"+ + ''.join(f'\n - {f}' for f in fails))+ return 0 if not fails else 1+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/scripts/mix.py b/skills/real-sound/scripts/mix.pynew file mode 100644index 0000000..6406d22--- /dev/null+++ b/skills/real-sound/scripts/mix.py@@ -0,0 +1,690 @@+#!/usr/bin/env python+"""real-sound mixer: real recordings placed on the picture's cues. Nothing is synthesized here.++ python -I <skill>/scripts/mix.py [--root .] [--design audio/sounddesign.json] [--cues out/cues.json]+ [--out out/audio/soundtrack.wav] [--stems out/audio/stems] [--no-master]+ [--only keyboard] [--dump out/audio-review/dump] [--trace out/qa/adds.json]++Run it from the project root (or pass --root). Every relative path (design, cues, outputs, the "file" of every sample)+is relative to the root. Full reference with examples: references/design-format.md.++The cue file (JSON, from the picture's timeline): {fps, total (frames), events: [{f, kind, id?, weight?, pan?, ...}]}+(or a list whose first element is that object). A design without rules needs no cue file.++The design file (JSON) describes, in film seconds:+ duration film length in seconds (default: cue total / fps)+ cues optional default cue file path (the --cues flag wins)+ include [other design fragments to merge: samples / slices / pools / rules / place / ambience / music]+ samples name -> {file, start?, end?, gain_db?, norm_peak_db?, mono?, fade_in?, fade_out?, eq?, pitch?,+ glide?: [from, to] semitones (tape-style varispeed across the sample), env?: [[t, dB], ...]}+ a trimmed piece of a downloaded recording (file paths are relative to the project root)+ slices name -> {file, start?, end?, pre?, len?, min_gap?, max?, pick?, ...sample opts}+ cut a recording of many hits (typing, clicks) into single hits at its detected onsets: name#0, name#1...+ pools name -> [sample names; "slicegroup#*" expands to all of its slices]+ rules [{match: {kind?, id?, from?, to?}, pool, mode: cycle|hash|fixed, gain_db?, weight_db?, jitter_db?,+ pitch?: [lo, hi] semitones, pitch_ramp?: [a, b], gain_ramp?: [a, b] (across the matched events),+ pan?: "cue"|number, pan_scale?, align: attack|peak|end|start, offset?, bus?}]+ every cue event matching a rule gets a sound from the rule's pool (several rules may layer one event)+ place [{sample, at, gain_db?, pan?, align?, offset?, pitch?, bus?}] sounds with no cue event+ ambience [{name?, file, src_start?, src_end?, at, until, fade_in?, fade_out?, gain_db?, eq?, width?, auto?: [[t, db], ...]}]+ (src_start..src_end is the clean window; it is looped with a 1.5 s crossfade if the bed is longer)+ music {file, src_bpm, target_bpm?, stretch?: "rubberband"|"none", gain_db?, eq?,+ segments: [{src_from, src_to, at, fade_in?, fade_out?, gain_db?}]} src_* in ORIGINAL source seconds+ buses {sfx|amb|music: {gain_db?, eq?, auto?: [[t, db], ...]}}+ master {lufs?, ceiling?, no_glue?, no_comp?, mono_hz?}+Processing is limited to trimming, fades, gain, pan, EQ, resampling (pitch variation), tempo stretch of the music,+summing and mastering (master.py next to this script). Writes the soundtrack, stems (sfx.wav, music.wav, amb.wav),+out/qa/mix_report.json (every placement, unsounded cue events, bus levels, files_used for the credits) and+out/qa/cues_sounded.json (the cue events that received a sound, for an A/V sync audit).++--only NAME solo render: keep only the include fragments whose path contains NAME (no master; the unmastered mix is+ written to --out, default out/audio-review/solo/<NAME>.wav). Variation is keyed on rule names, so a solo+ render plays exactly the same sounds as the full mix.+--no-master write the unmastered mix to --out (with master: --out is mastered, the unmastered sum is premaster.wav+ next to it).+--dump DIR write every sample / slice of the design as its own WAV + _samples.json (check trims), then exit.+--trace FILE also write every Mix.add() (bus, start sample, length, gain, pan, sample, rule) for QA attribution.+"""+import argparse, hashlib, json, math, os, re, subprocess, sys, zlib+import numpy as np+import soundfile as sf+from scipy import signal++SR = 48000+ROOT = os.getcwd() # the project root (set by --root)+CACHE = os.path.join(ROOT, 'out', 'audio', 'cache')+MASTER = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'master.py')+++def P(p):+ return p if os.path.isabs(p) else os.path.join(ROOT, p)+++def db2a(d):+ return 10.0 ** (d / 20.0)+++# ------------------------------------------------------------------ decoding (cached)+_dec = {}+++def decode(path):+ path = P(path)+ if path in _dec:+ return _dec[path]+ os.makedirs(CACHE, exist_ok=True)+ st = os.stat(path)+ key = hashlib.sha1(f'{path}|{st.st_size}|{st.st_mtime}'.encode()).hexdigest()[:16]+ npy = os.path.join(CACHE, key + '.npy')+ if os.path.exists(npy):+ x = np.load(npy)+ else:+ p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-f', 'f32le', '-acodec', 'pcm_f32le',+ '-ac', '2', '-ar', str(SR), '-'], capture_output=True)+ if p.returncode != 0:+ raise RuntimeError(f'decode failed {path}: {p.stderr.decode(errors="replace")[:300]}')+ x = np.frombuffer(p.stdout, np.float32).reshape(-1, 2).copy()+ tmp = f'{npy}.{os.getpid()}.tmp.npy'+ np.save(tmp, x)+ os.replace(tmp, npy) # atomic: other mixers may read the cache at the same time+ _dec[path] = x+ return x+++def stretched(path, ratio):+ """Whole-file tempo change with Rubber Band (ffmpeg filter). ratio = target_bpm / src_bpm (speed factor)."""+ if abs(ratio - 1) < 1e-6:+ return decode(path)+ src = P(path)+ st = os.stat(src)+ key = hashlib.sha1(f'{src}|{st.st_size}|{st.st_mtime}|{ratio:.6f}|rb2'.encode()).hexdigest()[:16]+ out = os.path.join(CACHE, f'stretch_{key}.wav')+ if not os.path.exists(out):+ os.makedirs(CACHE, exist_ok=True)+ af = (f'rubberband=tempo={ratio:.6f}:transients=mixed:detector=compound:phase=laminar:window=standard:'+ f'smoothing=off:formant=shifted:pitchq=quality:channels=together')+ tmp = out.replace('.wav', f'.{os.getpid()}.tmp.wav')+ r = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-y', '-i', src, '-af', af, '-ar', str(SR), '-ac', '2',+ '-c:a', 'pcm_f32le', tmp], capture_output=True)+ if r.returncode != 0:+ raise RuntimeError('rubberband failed: ' + r.stderr.decode(errors='replace')[:400])+ os.replace(tmp, out)+ return decode(out)+++# ------------------------------------------------------------------ DSP helpers (processing only)+def rc_fade(n, rising=True):+ if n <= 0:+ return np.ones(0)+ w = 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, n))+ return w if rising else w[::-1]+++def apply_fades(x, fin, fout):+ x = x.copy()+ a, b = int(fin * SR), int(fout * SR)+ a, b = min(a, len(x)), min(b, len(x))+ if a:+ x[:a] *= rc_fade(a)[:, None]+ if b:+ x[len(x) - b:] *= rc_fade(b, False)[:, None]+ return x+++def biquad(kind, f0, q=0.707, gain_db=0.0):+ A = 10 ** (gain_db / 40)+ w0 = 2 * np.pi * f0 / SR+ al = np.sin(w0) / (2 * q)+ c = np.cos(w0)+ if kind == 'peak':+ b = [1 + al * A, -2 * c, 1 - al * A]; a = [1 + al / A, -2 * c, 1 - al / A]+ elif kind == 'lowshelf':+ s = 2 * np.sqrt(A) * al+ b = [A * ((A + 1) - (A - 1) * c + s), 2 * A * ((A - 1) - (A + 1) * c), A * ((A + 1) - (A - 1) * c - s)]+ a = [(A + 1) + (A - 1) * c + s, -2 * ((A - 1) + (A + 1) * c), (A + 1) + (A - 1) * c - s]+ elif kind == 'highshelf':+ s = 2 * np.sqrt(A) * al+ b = [A * ((A + 1) + (A - 1) * c + s), -2 * A * ((A - 1) + (A + 1) * c), A * ((A + 1) + (A - 1) * c - s)]+ a = [(A + 1) - (A - 1) * c + s, 2 * ((A - 1) - (A + 1) * c), (A + 1) - (A - 1) * c - s]+ else:+ raise ValueError(kind)+ return np.array(b) / a[0], np.array(a) / a[0]+++def eq(x, spec):+ if not spec:+ return x+ y = x.astype(np.float64)+ if spec.get('hp'):+ sos = signal.butter(spec.get('hp_order', 2), spec['hp'], 'highpass', fs=SR, output='sos')+ y = signal.sosfilt(sos, y, axis=0)+ if spec.get('lp'):+ sos = signal.butter(spec.get('lp_order', 2), spec['lp'], 'lowpass', fs=SR, output='sos')+ y = signal.sosfilt(sos, y, axis=0)+ for f0, q, g in spec.get('peq', []):+ b, a = biquad('peak', f0, q, g); y = signal.lfilter(b, a, y, axis=0)+ if spec.get('lowshelf'):+ f0, g = spec['lowshelf']; b, a = biquad('lowshelf', f0, 0.707, g); y = signal.lfilter(b, a, y, axis=0)+ if spec.get('highshelf'):+ f0, g = spec['highshelf']; b, a = biquad('highshelf', f0, 0.707, g); y = signal.lfilter(b, a, y, axis=0)+ return y.astype(np.float32)+++def pan_gains(p):+ p = max(-1.0, min(1.0, float(p)))+ th = (p + 1) * np.pi / 4+ return np.cos(th) * np.sqrt(2), np.sin(th) * np.sqrt(2)+++def resample_pitch(x, semis):+ if abs(semis) < 1e-4:+ return x+ import soxr+ r = 2 ** (semis / 12.0)+ return soxr.resample(x, SR * r, SR, quality='VHQ').astype(np.float32)+++def varispeed(x, s0, s1):+ """tape-style pitch glide from s0 to s1 semitones across the sample (resampling a recording; no synthesis)"""+ import soxr+ r0, r1 = 2 ** (s0 / 12.0), 2 ** (s1 / 12.0)+ up = 4+ xo = soxr.resample(x.astype(np.float64), SR, SR * up, quality='VHQ')+ rmax = max(r0, r1, 1.0)+ if rmax > 1.0: # band-limit before reading faster, so the glide never aliases+ sos = signal.butter(8, 0.95 * (SR / 2) / rmax, 'lowpass', fs=SR * up, output='sos')+ xo = signal.sosfiltfilt(sos, xo, axis=0)+ L = max(2, int(len(x) / ((r0 + r1) / 2)))+ r = np.linspace(r0, r1, L)+ pos = np.concatenate([[0.0], np.cumsum(r[:-1])]) * up+ pos = np.clip(pos, 0, len(xo) - 1.001)+ i = pos.astype(np.int64)+ f = (pos - i)[:, None]+ y = xo[i] * (1 - f) + xo[np.minimum(i + 1, len(xo) - 1)] * f+ return y.astype(np.float32)+++def env_index(x, mode):+ m = np.abs(x).max(axis=1)+ if mode == 'start':+ return 0+ if mode == 'end':+ nz = np.where(m > m.max() * 0.003)[0]+ return int(nz[-1]) if len(nz) else len(x) - 1+ w = int(0.001 * SR) if mode == 'attack' else int(0.005 * SR)+ n = len(m) // w+ if n == 0:+ return 0+ e = m[: n * w].reshape(n, w).max(axis=1)+ if mode == 'peak':+ return int(np.argmax(e) * w + w // 2)+ return int(np.argmax(e >= e.max() * 0.1) * w) # attack: first 1 ms within 20 dB of the peak+++def auto_curve(points, n, default=0.0):+ """piecewise-linear dB automation over film time -> linear gain per sample"""+ if not points:+ return None+ pts = sorted(points)+ t = np.arange(n) / SR+ d = np.interp(t, [p[0] for p in pts], [p[1] for p in pts])+ return db2a(d).astype(np.float32)+++# ------------------------------------------------------------------ samples+class Lib:+ def __init__(self, design):+ self.d = design+ self.samples = {}+ self.meta = {}+ for name, s in (design.get('samples') or {}).items():+ self.samples[name] = self._make(s)+ self.meta[name] = {'file': s['file'], 'start': s.get('start', 0), 'end': s.get('end'), 'pitch': s.get('pitch', 0)}+ for name, s in (design.get('slices') or {}).items():+ for i, (x, t0) in enumerate(self._slice(s)):+ self.samples[f'{name}#{i}'] = x+ self.meta[f'{name}#{i}'] = {'file': s['file'], 'start': round(t0, 4), 'pitch': s.get('pitch', 0)}+ self.pools = {}+ for name, items in (design.get('pools') or {}).items():+ out = []+ for it in items:+ if it.endswith('#*'):+ g = it[:-2]+ out += sorted([k for k in self.samples if k.startswith(g + '#')], key=lambda k: int(k.split('#')[1]))+ else:+ if it not in self.samples:+ raise KeyError(f'pool {name}: unknown sample {it}')+ out.append(it)+ if not out:+ raise ValueError(f'pool {name} is empty')+ self.pools[name] = out+ self._pitched = {}++ def _finish(self, x, s):+ if s.get('mono'):+ m = x.mean(axis=1, keepdims=True)+ x = np.repeat(m, 2, axis=1)+ if s.get('reverse'):+ raise ValueError('reverse is not used')+ x = eq(x, s.get('eq'))+ if s.get('pitch'):+ x = resample_pitch(x, s['pitch'])+ if s.get('glide'): # [from, to] semitones across the sample+ x = varispeed(x, *s['glide'])+ if s.get('env'): # [[t_seconds_from_sample_start, dB], ...] gain envelope inside the sample+ pts = sorted(s['env'])+ t = np.arange(len(x)) / SR+ x = x * db2a(np.interp(t, [p[0] for p in pts], [p[1] for p in pts]))[:, None].astype(np.float32)+ if s.get('norm_peak_db') is not None:+ pk = float(np.abs(x).max()) or 1.0+ x = x * (db2a(s['norm_peak_db']) / pk)+ x = apply_fades(x, s.get('fade_in', 0.002), s.get('fade_out', 0.012))+ return (x * db2a(s.get('gain_db', 0.0))).astype(np.float32)++ def _make(self, s):+ x = decode(s['file'])+ a = int(s.get('start', 0) * SR)+ b = int(s['end'] * SR) if s.get('end') is not None else len(x)+ return self._finish(x[a:b], s)++ def _slice(self, s):+ import librosa+ x = decode(s['file'])+ a = int(s.get('start', 0) * SR)+ b = int(s['end'] * SR) if s.get('end') is not None else len(x)+ seg = x[a:b]+ mono = seg.mean(axis=1)+ hop = 128+ on = librosa.onset.onset_detect(y=mono, sr=SR, hop_length=hop, backtrack=True, units='samples',+ delta=s.get('delta', 0.07), wait=int(s.get('min_gap', 0.06) * SR / hop))+ on = [int(o) for o in on]+ # loudness gate: drop onsets far below the loudest (handling noise, the far keys)+ pk = []+ for o in on:+ w = mono[o: o + int(0.03 * SR)]+ pk.append(float(np.abs(w).max()) if len(w) else 0)+ if pk:+ top = max(pk)+ on = [o for o, p in zip(on, pk) if p >= top * db2a(s.get('gate_db', -18))]+ pre, ln = int(s.get('pre', 0.004) * SR), int(s.get('len', 0.16) * SR)+ out = []+ for i, o in enumerate(on):+ st = max(0, o - pre)+ en = min(len(seg), o + ln)+ if s.get('stop_before_next', True) and i + 1 < len(on):+ en = min(en, on[i + 1] - pre)+ if en - st < int(0.02 * SR):+ continue+ out.append((self._finish(seg[st:en], {**s, 'fade_out': s.get('fade_out', 0.02)}), (a + st) / SR))+ if s.get('pick') is not None:+ out = [out[i] for i in s['pick'] if i < len(out)]+ if s.get('max'):+ out = out[: s['max']]+ if not out:+ raise ValueError(f'slices {s["file"]}: no onsets found')+ return out++ def get(self, name, semis=0.0):+ if abs(semis) < 1e-3:+ return self.samples[name]+ k = (name, round(semis, 2))+ if k not in self._pitched:+ self._pitched[k] = resample_pitch(self.samples[name], k[1])+ return self._pitched[k]+++# ------------------------------------------------------------------ the mix+def h01(s):+ return (zlib.crc32(s.encode()) & 0xffffffff) / 0xffffffff+++class Mix:+ def __init__(self, dur):+ self.n = int(round(dur * SR))+ self.bus = {k: np.zeros((self.n, 2), np.float32) for k in ('sfx', 'amb', 'music')}+ self.log = []++ def add(self, bus, x, start, gain=1.0, pan=0.0, **meta):+ self.log.append({'bus': bus, 'start': int(start), 'n': int(len(x)), 'gain': float(gain), 'pan': float(pan), **meta})+ if start >= self.n or start + len(x) <= 0:+ return+ a, b = max(0, start), min(self.n, start + len(x))+ seg = x[a - start: b - start] * gain+ if pan:+ gl, gr = pan_gains(pan)+ seg = seg * np.array([gl, gr], np.float32)+ self.bus[bus][a:b] += seg+++def match(rule, e, fps):+ m = rule.get('match', {})+ if 'kind' in m and not re.fullmatch(m['kind'], e['kind']):+ return False+ if 'id' in m and not re.search(m['id'], e.get('id', '')):+ return False+ t = e['f'] / fps+ if 'from' in m and t < m['from'] - 1e-6:+ return False+ if 'to' in m and t >= m['to'] - 1e-6:+ return False+ if 'variant' in m and e.get('variant') != m['variant']:+ return False+ return True+++def load_design(path, only=None):+ """the design file, with its "include" fragments merged in (samples / slices / pools / rules / place)"""+ if not os.path.exists(P(path)):+ sys.exit(f'design file not found: {P(path)}')+ d = json.load(open(P(path), encoding='utf-8'))+ for inc in d.get('include', []):+ if only and only not in inc:+ continue+ f = load_design(inc)+ for k in ('samples', 'slices', 'pools'):+ for name, v in (f.get(k) or {}).items():+ if name in d.setdefault(k, {}):+ raise KeyError(f'{inc}: {k} "{name}" is defined twice')+ d[k][name] = v+ for k in ('rules', 'place', 'ambience'):+ d[k] = (f.get(k) or []) + (d.get(k) or [])+ if f.get('music') and not d.get('music'):+ d['music'] = f['music']+ return d+++def missing_files(design):+ """every recording the design references that is not on disk, as 'kind name: path' lines"""+ refs = [(f'sample {n}', s.get('file')) for n, s in (design.get('samples') or {}).items()]+ refs += [(f'slices {n}', s.get('file')) for n, s in (design.get('slices') or {}).items()]+ refs += [(f"ambience {a.get('name') or i}", a.get('file')) for i, a in enumerate(design.get('ambience') or [])]+ if design.get('music'):+ refs.append(('music', design['music'].get('file')))+ return [f'{what}: {f}' for what, f in refs if not f or not os.path.isfile(P(f))]+++def check_files(design):+ miss = missing_files(design)+ if miss:+ sys.exit(f'{len(miss)} recording(s) referenced by the design are missing (paths relative to --root {ROOT}):\n '+ + '\n '.join(miss))+++def dump(args):+ """write every sample / slice of the design as its own WAV + measurements, to check the trims"""+ design = load_design(args.design, args.only)+ check_files(design)+ lib = Lib(design)+ os.makedirs(P(args.dump), exist_ok=True)+ rows = []+ for name, x in lib.samples.items():+ fn = re.sub(r'[^A-Za-z0-9._#-]+', '_', name).replace('#', '-') + '.wav'+ sf.write(os.path.join(P(args.dump), fn), x, SR, subtype='PCM_24')+ m = np.abs(x).max(axis=1)+ pk = float(m.max()) or 1e-9+ a = env_index(x, 'attack')+ rows.append({'name': name, 'file': fn, 'src': lib.meta.get(name), 'dur': round(len(x) / SR, 3),+ 'peak_dbfs': round(20 * math.log10(pk), 1),+ 'rms_dbfs': round(10 * math.log10(float(np.mean(x.astype(np.float64) ** 2)) + 1e-20), 1),+ 'attack_ms': round(a / SR * 1000, 1), 'peak_ms': round(env_index(x, 'peak') / SR * 1000, 1),+ 'head_db': round(20 * math.log10(float(m[: int(0.002 * SR)].max()) / pk + 1e-9), 1),+ 'tail_db': round(20 * math.log10(float(m[-int(0.02 * SR):].max()) / pk + 1e-9), 1)})+ for name, items in lib.pools.items():+ rows.append({'pool': name, 'size': len(items), 'items': items})+ out = os.path.join(P(args.dump), '_samples.json')+ json.dump(rows, open(out, 'w'), indent=1)+ print(f'dumped {len(lib.samples)} samples, {len(lib.pools)} pools -> {out}')+ for r in rows:+ if 'pool' in r:+ print(f" pool {r['pool']}: {r['size']}")+ else:+ print(f" {r['name']:<34} {r['dur']:6.3f}s pk {r['peak_dbfs']:6.1f} rms {r['rms_dbfs']:6.1f} "+ f"att {r['attack_ms']:6.1f}ms head {r['head_db']:6.1f} tail {r['tail_db']:6.1f}")+++def run(args):+ if args.dump:+ return dump(args)+ design = load_design(args.design, args.only)+ cue_path = args.cues or design.get('cues') or 'out/cues.json'+ if os.path.exists(P(cue_path)):+ cues = json.load(open(P(cue_path)))+ cue = cues[0] if isinstance(cues, list) else cues+ elif design.get('rules'):+ sys.exit(f'cue file {cue_path} not found (the design has cue rules); pass --cues')+ else:+ cue = {'fps': 60, 'events': []}+ fps = cue.get('fps', 60)+ if 'duration' not in design and 'total' not in cue:+ sys.exit('set "duration" (seconds) in the design or "total" (frames) in the cue file')+ dur = design.get('duration', cue.get('total', 0) / fps)+ check_files(design)+ lib = Lib(design)+ mix = Mix(dur)+ placed_events = []+ report = {'placements': [], 'unsounded_events': [], 'buses': {}}++ # --- cue-driven effects+ events = sorted(cue['events'], key=lambda e: e['f'])+ counters, last = {}, {}+ sounded = set()+ for ri, rule in enumerate(design.get('rules', [])):+ pool = lib.pools[rule['pool']]+ matched = [e for e in events if match(rule, e, fps)]+ rk = rule.get('name', ri) # per-event variation keyed on the rule's name: identical solo and merged+ for mi, e in enumerate(matched):+ mode = rule.get('mode', 'cycle')+ if mode == 'fixed':+ name = pool[0]+ elif mode == 'hash':+ name = pool[int(h01(f'{e.get("id")}|{e["f"]}|{rk}') * len(pool)) % len(pool)]+ else:+ c = counters.get(ri, int(rule.get('cycle_start', 0)))+ name = pool[c % len(pool)]+ if len(pool) > 1 and name == last.get(ri):+ c += 1; name = pool[c % len(pool)]+ counters[ri] = c + 1+ last[ri] = name+ lo, hi = rule.get('pitch', [0, 0])+ semis = lo + (hi - lo) * h01(f'p|{e.get("id")}|{e["f"]}|{rk}') if hi != lo else lo+ if rule.get('pitch_ramp'): # climbs across the matched events in time order (growth, stacking)+ r0, r1 = rule['pitch_ramp']+ semis += r0 + (r1 - r0) * (mi / max(1, len(matched) - 1))+ if rule.get('gain_ramp'):+ g0, g1 = rule['gain_ramp']+ rule_gain_ramp = g0 + (g1 - g0) * (mi / max(1, len(matched) - 1))+ else:+ rule_gain_ramp = 0.0+ g = rule.get('gain_db', 0.0) + rule_gain_ramp+ if rule.get('weight_db'):+ g += rule['weight_db'] * math.log2(max(e.get('weight', 1.0), 1e-3) / rule.get('weight_ref', 1.0))+ if rule.get('jitter_db'):+ g += (h01(f'g|{e.get("id")}|{e["f"]}|{rk}') * 2 - 1) * rule['jitter_db']+ pan = rule.get('pan', 'cue')+ pan = (e.get('pan', 0.0) * rule.get('pan_scale', 1.0)) if pan == 'cue' else float(pan)+ x = lib.get(name, semis)+ align = rule.get('align', 'attack')+ t = e['f'] / fps + rule.get('offset', 0.0)+ start = int(round(t * SR)) - env_index(x, align)+ mix.add(rule.get('bus', 'sfx'), x, start, db2a(g), pan, sample=name, rule=rule.get('name', ri), kind=e['kind'])+ report['placements'].append({'t': round(t, 4), 'event': e.get('id'), 'kind': e['kind'], 'rule': rule.get('name', ri),+ 'sample': name, 'gain_db': round(g, 2), 'pitch': round(semis, 2),+ 'pitch_total': round(semis + (lib.meta.get(name, {}).get('pitch') or 0), 2),+ 'pan': round(pan, 3), 'align': align})+ if rule.get('bus', 'sfx') == 'sfx' and not rule.get('no_cue'):+ key = (e['f'], e['kind'], e.get('id'))+ if key not in sounded:+ sounded.add(key)+ placed_events.append(e)+ for e in events:+ if (e['f'], e['kind'], e.get('id')) not in sounded:+ report['unsounded_events'].append({'t': round(e['f'] / fps, 3), 'kind': e['kind'], 'id': e.get('id')})++ # --- free placements+ for p in design.get('place', []):+ x = lib.get(p['sample'], p.get('pitch', 0.0))+ t = p['at'] + p.get('offset', 0.0)+ start = int(round(t * SR)) - env_index(x, p.get('align', 'attack'))+ mix.add(p.get('bus', 'sfx'), x, start, db2a(p.get('gain_db', 0.0)), p.get('pan', 0.0), sample=p['sample'], kind='place')+ report['placements'].append({'t': round(t, 4), 'event': None, 'kind': 'place', 'sample': p['sample'],+ 'gain_db': p.get('gain_db', 0.0), 'bus': p.get('bus', 'sfx')})++ # --- ambience beds+ for a in design.get('ambience', []):+ x = decode(a['file'])+ s0 = int(a.get('src_start', 0) * SR)+ need = int((a['until'] - a['at']) * SR)+ src = x[s0: int(a['src_end'] * SR) if a.get('src_end') is not None else len(x)] # the clean window (also the loop)+ looped = len(src) < need+ if looped: # loop with a 1.5 s equal-power crossfade+ xf = int(1.5 * SR)+ out = src.copy()+ while len(out) < need:+ nxt = src.copy()+ fo = np.sqrt(rc_fade(xf, False))[:, None]; fi = np.sqrt(rc_fade(xf))[:, None]+ out[-xf:] = out[-xf:] * fo + nxt[:xf] * fi+ out = np.concatenate([out, nxt[xf:]])+ src = out+ seg = src[:need]+ if a.get('width') is not None:+ m, s = (seg[:, 0] + seg[:, 1]) / 2, (seg[:, 0] - seg[:, 1]) / 2 * a['width']+ seg = np.stack([m + s, m - s], axis=1)+ seg = eq(seg, a.get('eq'))+ seg = apply_fades(seg, a.get('fade_in', 1.0), a.get('fade_out', 1.0))+ start = int(round(a['at'] * SR))+ if a.get('auto'):+ curve = auto_curve([[t - a['at'], d] for t, d in a['auto']], len(seg))+ seg = seg * curve[:, None]+ mix.add('amb', seg.astype(np.float32), start, db2a(a.get('gain_db', 0.0)), name=a.get('name') or a['file'], kind='ambience')+ body = seg[int(min(len(seg) - 1, a.get('fade_in', 1.0) * SR)): max(1, len(seg) - int(a.get('fade_out', 1.0) * SR))]+ report.setdefault('ambience', []).append({+ 'name': a.get('name'), 'file': a['file'], 'at': a['at'], 'until': a['until'], 'looped': looped,+ 'body_rms_dbfs': round(10 * math.log10(float(np.mean(body.astype(np.float64) ** 2)) + 1e-20)+ + a.get('gain_db', 0.0), 1) if len(body) else None})++ # --- music+ mu = design.get('music')+ if mu:+ ratio = mu.get('target_bpm', 120) / mu['src_bpm'] if mu.get('stretch', 'rubberband') != 'none' else 1.0+ x = stretched(mu['file'], ratio)+ k = 1.0 / ratio # source seconds -> stretched seconds+ for sg in mu['segments']:+ fin, fout = sg.get('fade_in', 0.008), sg.get('fade_out', 0.06)+ a = int(round(sg['src_from'] * k * SR)) - int(fin * SR)+ b = int(round(sg['src_to'] * k * SR)) + int(fout * SR)+ pre = int(fin * SR)+ if a < 0:+ pre += a; a = 0+ seg = apply_fades(x[a:b], fin if sg.get('fade_in_at_start', True) else 0, fout)+ if sg.get('tail_fade'): # a longer musical fade over the end of the segment+ n = int(sg['tail_fade'] * SR)+ seg[-n:] *= rc_fade(n, False)[:, None]+ start = int(round(sg['at'] * SR)) - pre+ mix.add('music', seg.astype(np.float32), start, db2a(sg.get('gain_db', 0.0)), label=sg.get('label'), kind='music')+ report.setdefault('music_segments', []).append({**sg, 'stretched_from': round(a / SR, 4), 'film_start': round(start / SR, 4)})+ if mu.get('eq'):+ mix.bus['music'] = eq(mix.bus['music'], mu['eq'])+ mix.bus['music'] *= db2a(mu.get('gain_db', 0.0))++ # --- buses+ for name, b in (design.get('buses') or {}).items():+ if name.startswith('_'): # a comment key+ continue+ if name not in mix.bus:+ sys.exit(f'buses: unknown bus "{name}" (sfx, amb or music; comment keys start with _)')+ y = eq(mix.bus[name], b.get('eq'))+ c = auto_curve(b.get('auto'), mix.n)+ if c is not None:+ y = y * c[:, None]+ mix.bus[name] = (y * db2a(b.get('gain_db', 0.0))).astype(np.float32)++ # every recording that is audible in the film (for CREDITS.md)+ used = {lib.meta[p['sample']]['file'] for p in report['placements'] if p.get('sample') in lib.meta}+ used |= {a['file'] for a in design.get('ambience', [])}+ if mu:+ used.add(mu['file'])+ report['files_used'] = sorted(u.replace('\\', '/') for u in used)+ total = mix.bus['sfx'] + mix.bus['amb'] + mix.bus['music']+ os.makedirs(P(args.stems), exist_ok=True)+ for name, y in mix.bus.items():+ sf.write(os.path.join(P(args.stems), f'{name}.wav'), y, SR, subtype='FLOAT')+ pk = float(np.abs(y).max())+ report['buses'][name] = {'peak_dbfs': round(20 * math.log10(max(pk, 1e-9)), 2),+ 'rms_dbfs': round(10 * math.log10(float(np.mean(y.astype(np.float64) ** 2)) + 1e-20), 2)}+ # mastered: the unmastered sum goes next to --out as premaster.wav; --no-master / --only: it IS the output+ pre = P(args.out) if args.no_master else os.path.join(os.path.dirname(P(args.out)), 'premaster.wav')+ sf.write(pre, total, SR, subtype='FLOAT')+ report['premaster_peak_dbfs'] = round(20 * math.log10(max(float(np.abs(total).max()), 1e-9)), 2)+ # cues that received an effect, for the A/V audit+ cp = {'fps': fps, 'total': cue.get('total', int(dur * fps)), 'events': [+ {k: v for k, v in e.items() if k in ('f', 'kind', 'weight', 'id', 'pan', 'dur', 'apexFrac')} for e in placed_events]}+ for e in cp['events']:+ if e['kind'] in ('riser', 'swell'):+ e['ends'] = True+ os.makedirs(os.path.dirname(P(args.cues_out)) or '.', exist_ok=True)+ json.dump(cp, open(P(args.cues_out), 'w'), indent=0)+ os.makedirs(os.path.dirname(P(args.report)), exist_ok=True)+ json.dump(report, open(P(args.report), 'w'), indent=1)+ if args.trace:+ os.makedirs(os.path.dirname(P(args.trace)) or '.', exist_ok=True)+ json.dump(mix.log, open(P(args.trace), 'w'), indent=0)+ print(f'premaster {pre} peak {report["premaster_peak_dbfs"]} dBFS placements {len(report["placements"])} '+ f'unsounded {len(report["unsounded_events"])}')+ for k, v in report['buses'].items():+ print(f' bus {k}: {v}')+ if not args.no_master and not np.any(total):+ print('the mix is digital silence (every bus is zero): nothing to master, no soundtrack written'+ + (f' ({P(args.out)} is from an earlier render)' if os.path.exists(P(args.out)) else '')+ + '. Check the trims (--dump) and the gains.')+ sys.exit(1)+ if not args.no_master:+ m = design.get('master', {})+ cmd = [sys.executable, '-I', MASTER, pre, P(args.out), '--lufs', str(m.get('lufs', -14)), '--ceiling',+ str(m.get('ceiling', 0.77)), '--json', os.path.join(os.path.dirname(P(args.report)), 'master.json')]+ if m.get('no_glue', True):+ cmd.append('--no-glue')+ if m.get('no_comp'):+ cmd.append('--no-comp')+ if m.get('mono_hz') is not None:+ cmd += ['--mono-hz', str(m['mono_hz'])]+ r = subprocess.run(cmd, capture_output=True, text=True)+ print(r.stdout[-1500:], r.stderr[-1500:])+ if r.returncode != 0:+ print('master gate FAILED')+ sys.exit(1)+++if __name__ == '__main__':+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])+ ap.add_argument('--root', default='.', help='project root: every relative path is resolved against it')+ ap.add_argument('--design', default='audio/sounddesign.json')+ ap.add_argument('--cues', help='cue file (default: the design\'s "cues", else out/cues.json)')+ ap.add_argument('--out', help='default out/audio/soundtrack.wav (solo: out/audio-review/solo/<only>.wav)')+ ap.add_argument('--stems', help='default out/audio/stems (solo: next to --out)')+ ap.add_argument('--no-master', action='store_true')+ ap.add_argument('--only', help='solo render of the include fragments whose path contains this text (no master)')+ ap.add_argument('--dump', help='write every sample of the design to this folder as WAV + _samples.json, then exit')+ ap.add_argument('--report', help='default out/qa/mix_report.json (solo: next to --out)')+ ap.add_argument('--cues-out', help='the cue events that received a sound (A/V audit); default out/qa/cues_sounded.json')+ ap.add_argument('--trace', help='write every Mix.add() record here (QA: click / masking attribution)')+ ap.add_argument('--cache', help='decode cache folder (default <root>/out/audio/cache)')+ ap.add_argument('--master-script', help='mastering script (default: master.py next to this file)')+ a = ap.parse_args()+ ROOT = os.path.abspath(a.root)+ CACHE = os.path.abspath(os.path.join(ROOT, a.cache)) if a.cache else os.path.join(ROOT, 'out', 'audio', 'cache')+ if a.master_script:+ MASTER = os.path.abspath(a.master_script)+ if a.only:+ a.no_master = True+ tag = re.sub(r'[^A-Za-z0-9._-]+', '_', a.only)+ a.out = a.out or f'out/audio-review/solo/{tag}.wav'+ base = os.path.splitext(a.out)[0]+ a.stems = a.stems or base + '_stems'+ a.report = a.report or base + '_report.json'+ a.cues_out = a.cues_out or base + '_cues.json'+ a.out = a.out or 'out/audio/soundtrack.wav'+ a.stems = a.stems or 'out/audio/stems'+ a.report = a.report or 'out/qa/mix_report.json'+ a.cues_out = a.cues_out or 'out/qa/cues_sounded.json'+ os.makedirs(os.path.dirname(P(a.out)) or '.', exist_ok=True)+ run(a)diff --git a/skills/real-sound/scripts/mux.mjs b/skills/real-sound/scripts/mux.mjsnew file mode 100644index 0000000..b91ee1f--- /dev/null+++ b/skills/real-sound/scripts/mux.mjs@@ -0,0 +1,72 @@+#!/usr/bin/env node+// Put a soundtrack on the finished picture without re-encoding the video (stream copy; AAC 48 kHz stereo audio).+// node <skill>/scripts/mux.mjs --video out/film.mp4 [--audio out/audio/soundtrack.wav] [--out out/film_sound.mp4]+// [--abr 320k] [--keep-old out/superseded/film_prev.mp4] [--root .]+// Relative paths resolve against --root (default: the current directory). The audio is cut to the video's own duration+// (ffprobe), so a soundtrack that runs long never extends the film; a length mismatch over one frame is reported.+// When --out is the --video file itself, the old file is moved to --keep-old first (default+// out/superseded/<name>_prev<ext>); an existing keep-old file is never overwritten. Exit 0 = written and verified.+import { spawnSync } from 'node:child_process';+import path from 'node:path';+import fs from 'node:fs';++const argv = process.argv.slice(2);+if (argv.includes('-h') || argv.includes('--help')) {+ console.log(fs.readFileSync(new URL(import.meta.url), 'utf8').split('\n').slice(1, 8).map((l) => l.replace(/^\/\/ ?/, '')).join('\n'));+ process.exit(0);+}+const opt = (k, d) => { const i = argv.indexOf(`--${k}`); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; };+const ROOT = path.resolve(opt('root', '.'));+const R = (p) => (path.isAbsolute(p) ? p : path.join(ROOT, p));+const rel = (p) => path.relative(ROOT, p) || p;+const fail = (m) => { console.error(`mux: ${m}`); process.exit(1); };++function probe(file) {+ const r = spawnSync('ffprobe', ['-v', 'error', '-show_entries', 'stream=codec_type,codec_name,r_frame_rate,duration:format=duration',+ '-of', 'json', file], { encoding: 'utf8' });+ if (r.status !== 0) fail(`ffprobe failed on ${file}: ${(r.stderr || '').slice(0, 300)}`);+ const j = JSON.parse(r.stdout || '{}');+ const v = (j.streams || []).find((s) => s.codec_type === 'video');+ const a = (j.streams || []).find((s) => s.codec_type === 'audio');+ const [n, d] = ((v && v.r_frame_rate) || '0/1').split('/').map(Number);+ const fmt = Number((j.format || {}).duration || 0);+ return { video: v, audio: a, fps: d ? n / d : 0, duration: Number((v && v.duration) || fmt), audioDuration: Number((a && a.duration) || fmt) };+}++const audio = R(opt('audio', 'out/audio/soundtrack.wav'));+let video = R(opt('video', 'out/film.mp4'));+const out = R(opt('out', video));+const abr = opt('abr', '320k');+if (!fs.existsSync(audio)) fail(`no audio ${audio} (pass --audio)`);+if (!fs.existsSync(video)) fail(`no video ${video} (pass --video)`);+const pv = probe(video);+if (!pv.video) fail(`${video} has no video stream`);+const pa = probe(audio);+const frame = pv.fps ? 1 / pv.fps : 1 / 60;+const diff = pa.audioDuration - pv.duration;+if (Math.abs(diff) > frame) {+ console.log(`note: the audio is ${diff > 0 ? 'longer' : 'SHORTER'} than the picture by ${Math.abs(diff).toFixed(3)} s ` ++ `(${(Math.abs(diff) / frame).toFixed(1)} frames)` + (diff > 0 ? ': it is cut at the picture\'s end' : ': the end will be silent'));+}+if (path.resolve(out) === path.resolve(video)) {+ const ext = path.extname(video);+ const keep = R(opt('keep-old', path.join('out', 'superseded', path.basename(video, ext) + '_prev' + ext)));+ fs.mkdirSync(path.dirname(keep), { recursive: true });+ if (fs.existsSync(keep)) fail(`refusing to overwrite ${keep}; pass --keep-old <new path>`);+ fs.renameSync(video, keep);+ video = keep;+ console.log(`previous version kept at ${rel(keep)}`);+}+fs.mkdirSync(path.dirname(out), { recursive: true });+const ext = path.extname(out) || '.mp4';+const tmp = out.slice(0, out.length - path.extname(out).length) + '.muxing' + ext;+const args = ['-v', 'error', '-y', '-i', video, '-i', audio, '-map', '0:v:0', '-map', '1:a:0', '-c:v', 'copy', '-c:a', 'aac',+ '-b:a', abr, '-ar', '48000', '-ac', '2', '-t', pv.duration.toFixed(6)];+if (/\.(mp4|m4v|mov)$/i.test(ext)) args.push('-movflags', '+faststart');+const r = spawnSync('ffmpeg', [...args, tmp], { stdio: 'inherit' });+if (r.status !== 0) { try { fs.unlinkSync(tmp); } catch { /* nothing to clean */ } fail('ffmpeg failed'); }+const po = probe(tmp);+if (!po.video || !po.audio) fail(`${tmp} lacks a video or audio stream`);+fs.renameSync(tmp, out);+console.log(`wrote ${rel(out)} (${(fs.statSync(out).size / 1e6).toFixed(1)} MB): video copied (${pv.video.codec_name}, ` ++ `${pv.fps.toFixed(3)} fps, ${pv.duration.toFixed(3)} s), audio ${po.audio.codec_name} ${abr} ${po.audioDuration.toFixed(3)} s`);diff --git a/skills/real-sound/scripts/qa.py b/skills/real-sound/scripts/qa.pynew file mode 100644index 0000000..b6fec71--- /dev/null+++ b/skills/real-sound/scripts/qa.py@@ -0,0 +1,965 @@+#!/usr/bin/env python+"""Objective QA of a real-sound mix: what the critic lenses measure, in one pass, tied to the placements that caused it.++ python -I <skill>/scripts/qa.py [--root .] [--design audio/sounddesign.json] [--report out/qa/mix_report.json]+ [--trace out/qa/adds.json] [--stems out/audio/stems] [--master out/audio/soundtrack.wav]+ [--cues out/cues.json] [--only NAME] [--cache out/audio/cache] [--video out/film.mp4]+ [--out out/qa/qa.json] [--layers] [--listen] [--realness] [--json]++Run it after `mix.py --trace out/qa/adds.json` (same design, same --only): the trace ties every click, masked hit and+harsh peak to the placement that caused it. With --only, the report / stems default to mix.py's solo outputs+(out/audio-review/solo/<only>_report.json, _stems) and there is no master (levels at stem scale). Without the trace+only clicks, loudness and coverage run. No models needed; --listen / --realness add earshot's ears when installed. Sections (status ok / warn / fail + a plain summary):+ clicks > 14 kHz bursts concentrated in 1 ms (the av-audit detector) in the master and each stem, attributed to+ the placements whose start / end / body they sit in+ edges used samples that start mid-sound or are cut mid-decay (unfaded head / tail level vs their peak, and the+ source just outside the trim), a second hit inside a sample, steps in a stem at a placement's start / end+ (second-difference ratio), placements cut by the film's ends+ audibility every cue event's effect (its layers summed, rebuilt from the trace) vs the bed (music + amb) and vs+ everything else, in 1/3-octave bands over its loudest 60 ms and its first 20 ms: read >= 6 dB clear, 3-6 ok,+ 0-3 barely, < 0 masked (margins capped at +-60 dB); an effect peaking under -70 dBFS is SILENT (a trim+ that missed the sound: fail); not judged when the render has no bed (a solo); --layers also lists single+ layers that add nothing measurable+ harshness each event's 2-6 kHz peak (5 ms RMS, master scale) vs the music's loudest 2-6 kHz 5 ms level+ loudness integrated LUFS vs the design's target, true peak, LRA, loudest momentary (time, share of the runtime),+ short-term curve, per-act medians when the cue file has acts, head and tail of the master+ coverage cue events with no sound (unsounded or placed but silent); classes of action sounded only in part+ sync impacts by attack / peak (-45..+75 ms), whooshes and peak-aligned swells by their apex (+-45 ms), risers+ and end-aligned swells by their audible end vs the hit (-250..+30 ms); measured norms of professional+ launch films. A deviation that exactly one layer's rule offset explains is listed apart as deliberate+ density events per second, the most reused samples, "machine-gun" rules (one sample, no pitch or level change)+ listen --listen: earshot timeline + sheet + critique of the master; --realness: earshot realness of every source+Thresholds and where they come from: references/process.md. Writes --out (JSON) and prints a digest. Exit code 1 when+a section fails (a click at a placement edge in the master, a silent effect, stems of pure digital silence, the+loudness gate), 2 on missing or mismatched inputs (one line says which flag to pass), else 0.+"""+import argparse, json, math, os, re, sys++import numpy as np+import soundfile as sf+from scipy import signal++sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+import master as M # noqa: E402+import mix # noqa: E402 (read-only: the design loader, the sample library, envelope helpers)++SR = mix.SR+LONG = ('whoosh', 'riser', 'swell', 'suck')+FC = 10 ** (np.arange(20, 42) / 10) # 1/3-octave centres 100 Hz .. 12.6 kHz+BLO, BHI = FC * 2 ** (-1 / 6), FC * 2 ** (1 / 6)+SILENT_DB = -70.0 # an event whose peak (master scale) is below this makes no sound: a trim that missed the recording+CAP_DB = 60.0 # margins against a (near-)silent bed are reported as at most +60 dB+++class InputError(Exception):+ pass+++def db(v):+ return 20 * math.log10(max(float(v), 1e-12))+++def pdb(p):+ return 10 * math.log10(max(float(p), 1e-20))+++def fmt_t(t):+ m = int(t // 60)+ return f'{m}:{t - 60 * m:05.2f}'+++def say(s):+ print(str(s).encode('ascii', 'replace').decode())+++def lufs(x):+ """integrated LUFS, or None for audio shorter than one 400 ms block or with no gated signal (silence)"""+ if x is None or len(x) < int(0.4 * SR):+ return None+ v = M.integrated(x)+ return v if math.isfinite(v) else None+++def clean(o):+ """JSON-safe: numpy scalars to Python, NaN / inf to None"""+ if isinstance(o, dict):+ return {k: clean(v) for k, v in o.items()}+ if isinstance(o, (list, tuple)):+ return [clean(v) for v in o]+ if isinstance(o, np.integer):+ return int(o)+ if isinstance(o, (float, np.floating)):+ return float(o) if math.isfinite(o) else None+ return o+++# ------------------------------------------------------------------ context+class Ctx:+ def __init__(self, a):+ self.root = os.path.abspath(a.root)+ mix.ROOT = self.root+ mix.CACHE = os.path.join(self.root, a.cache) if a.cache else os.path.join(self.root, 'out', 'audio', 'cache')+ R = self.R = lambda p: os.path.normpath(p if os.path.isabs(p) else os.path.join(self.root, p))++ def need(path, what, flag):+ if not os.path.exists(R(path)):+ raise InputError(f'{what} not found: {R(path)} ({flag})')+ return R(path)++ need(a.design, 'design file', 'pass --design')+ self.design = mix.load_design(a.design, a.only)+ self.report = json.load(open(need(a.report, 'mix report', 'run mix.py first or pass --report'), encoding='utf-8'))+ self.trace = json.load(open(R(a.trace))) if a.trace and os.path.exists(R(a.trace)) else None+ self.stems = {}+ for k in ('sfx', 'amb', 'music'):+ p = need(os.path.join(a.stems, k + '.wav'), f'stem {k}.wav', 'pass the --stems folder mix.py wrote')+ self.stems[k] = sf.read(p, dtype='float32', always_2d=True)[0]+ self.N = len(self.stems['sfx'])+ if {len(v) for v in self.stems.values()} != {self.N}:+ raise InputError(f'the stems in {R(a.stems)} have different lengths: re-render them with one mix.py run')+ if a.master and getattr(a, 'master_given', False):+ need(a.master, 'master', 'pass --master none to skip it')+ self.master = (sf.read(R(a.master), dtype='float64', always_2d=True)[0]+ if a.master and os.path.exists(R(a.master)) else None)+ if self.master is not None and abs(len(self.master) - self.N) > 1:+ raise InputError(f'the master {R(a.master)} ({len(self.master) / SR:.3f} s) does not match the stems '+ f'({self.N / SR:.3f} s): pass the --master rendered with these stems, or --master none')+ cp = a.cues or self.design.get('cues') or 'out/cues.json'+ c = json.load(open(R(cp))) if os.path.exists(R(cp)) else {'fps': 60, 'events': []}+ self.cues = c[0] if isinstance(c, list) else c+ self.fps = self.cues.get('fps', 60)+ m = self.design.get('master') or {}+ self.target = m.get('lufs', -14)+ pre = sum(self.stems[k].astype(np.float64) for k in self.stems)+ self.pre_lufs = lufs(pre)+ self.master_lufs = lufs(self.master)+ both = self.master_lufs is not None and self.pre_lufs is not None+ self.G = self.master_lufs - self.pre_lufs if both else 0.0 # no master (or silence): stem scale+ self.scale = ('master' if both else 'stem') if self.master is not None else 'stem'+ self.no_bed = not (np.any(self.stems['music']) or np.any(self.stems['amb']))+ self.lib = None+ self.only = a.only+++def bus_curve(design, bus, n):+ b = (design.get('buses') or {}).get(bus) or {}+ c = mix.auto_curve(b.get('auto'), n)+ return (np.ones(n) if c is None else c.astype(np.float64)) * mix.db2a(b.get('gain_db', 0.0))+++def build_layers(ctx):+ """every sfx-bus placement rebuilt as it sits in the stem (sample, pitch, gain, pan, bus ride), grouped into events"""+ recs = [a for a in ctx.trace if 'sample' in a]+ pl = ctx.report.get('placements', [])+ if len(recs) != len(pl):+ return None, f'trace ({len(recs)} placements) and report ({len(pl)}) disagree: re-run mix.py with --trace'+ ctx.lib = ctx.lib or mix.Lib(ctx.design)+ places = iter(ctx.design.get('place') or [])+ rules = {str(r.get('name', i)): r for i, r in enumerate(ctx.design.get('rules') or [])}+ curve = bus_curve(ctx.design, 'sfx', ctx.N)+ out, bad = [], 0+ for a, p in zip(recs, pl):+ if p['kind'] == 'place':+ pp = next(places)+ pitch, off, align, cue, rule = pp.get('pitch', 0.0), pp.get('offset', 0.0), pp.get('align', 'attack'), pp['at'], 'place'+ ok = pp.get('qa_ok')+ else:+ r = rules.get(str(p.get('rule')), {})+ pitch, off, align, rule = p.get('pitch', 0.0), r.get('offset', 0.0), p.get('align', 'attack'), str(p.get('rule'))+ cue, ok = p['t'] - off, r.get('qa_ok')+ x = ctx.lib.get(a['sample'], pitch)+ if len(x) != a['n']:+ bad += 1+ continue+ if a['bus'] != 'sfx':+ continue+ s0, s1 = max(0, a['start']), min(ctx.N, a['start'] + a['n'])+ if s1 <= s0:+ continue+ seg = x[s0 - a['start']: s1 - a['start']].astype(np.float64) * a['gain']+ if a['pan']:+ seg = seg * np.array(mix.pan_gains(a['pan']))+ seg = seg * curve[s0:s1, None]+ out.append({'sample': a['sample'], 'rule': rule, 'kind': p['kind'], 'event': p.get('event'), 'cue': cue,+ 'offset': off, 'align': align, 'start': a['start'], 'n': a['n'], 'a': s0, 'b': s1, 'sig': seg, 'qa_ok': ok,+ 'att': a['start'] + mix.env_index(x, 'attack'), 'peak_db': db(np.abs(seg).max())})+ ctx.layers = out+ groups, by_cue = [], {}+ for L in sorted(out, key=lambda L: L['att']):+ if L['kind'] != 'place': # layers of one cue event share its exact cue time+ k = (L['event'], round(L['cue'], 4))+ if k in by_cue:+ by_cue[k]['layers'].append(L)+ else:+ by_cue[k] = {'att0': L['att'], 'layers': [L]}+ groups.append(by_cue[k])+ continue+ for g in groups[::-1][:12]: # free placements: layers whose attacks sit within 140 ms+ if g.get('place') and abs(L['att'] - g['att0']) < 0.14 * SR:+ g['layers'].append(L)+ break+ else:+ groups.append({'place': True, 'att0': L['att'], 'layers': [L]})+ for g in groups:+ ls = g['layers']+ g['a'], g['b'] = min(L['a'] for L in ls), max(L['b'] for L in ls)+ g['sig'] = np.zeros((g['b'] - g['a'], 2))+ for L in ls:+ g['sig'][L['a'] - g['a']: L['b'] - g['a']] += L['sig']+ g['cue'] = min(L['cue'] for L in ls)+ g['loud'] = max(ls, key=lambda L: L['peak_db'])+ g['kind'] = g['loud']['kind']+ g['label'] = (ls[0]['event'] or 'place:' + ls[0]['sample'])+ g['long'] = any(L['kind'] in LONG or L['align'] == 'peak' for L in ls)+ g['builds'] = any(L['kind'] in ('riser', 'swell') for L in ls)+ g['qa_ok'] = next((L['qa_ok'] for L in ls if L['qa_ok']), None)+ g['peak_db'] = db(np.abs(g['sig']).max()) + ctx.G+ g['silent'] = g['peak_db'] < SILENT_DB+ note = f'{bad} placement(s) could not be rebuilt (sample length differs from the trace)' if bad else None+ return groups, note+++# ------------------------------------------------------------------ clicks+HP14 = signal.butter(4, 14000, 'high', fs=SR, output='sos')+++def detect_clicks(x):+ """> 14 kHz samples > 9x the RMS of their 20 ms and > 6x their 4 ms neighbourhood; conc >= 0.8 of the burst's+ > 14 kHz energy inside 1 ms = a click (a cut), else a designed attack"""+ out = []+ for ch in range(x.shape[1]):+ hp = signal.sosfilt(HP14, x[:, ch].astype(np.float64))+ e, e2 = np.abs(hp), hp ** 2+ loc = np.sqrt(signal.fftconvolve(e2, np.ones(960) / 960, mode='same').clip(0)) + 1e-5+ near = np.sqrt(signal.fftconvolve(e2, np.ones(192) / 192, mode='same').clip(0)) + 1e-5+ last = -10 ** 9+ for i in np.nonzero((e / loc > 9) & (e / near > 6) & (e > 0.01))[0]:+ if i - last > SR * 0.01:+ conc = float(e2[max(0, i - 24):i + 24].sum() / (e2[max(0, i - 240):i + 240].sum() + 1e-15))+ out.append({'t': round(i / SR, 4), 'i': int(i), 'ch': ch, 'level_db': round(db(e[i]), 1),+ 'conc': round(conc, 2), 'kind': 'click' if conc >= 0.8 else 'attack'})+ last = i+ out.sort(key=lambda c: (c['t'], c['ch']))+ return [c for k, c in enumerate(out) if k == 0 or c['t'] - out[k - 1]['t'] > 0.001]+++def owners(trace, i, bus=None, win=960):+ """trace records around sample i, edge-near (within 2 ms of a start / end) first"""+ res = []+ for a in trace or []:+ if bus and a['bus'] != bus:+ continue+ s, e = a['start'], a['start'] + a['n']+ if s - win <= i <= e + win:+ d = min(abs(i - s), abs(i - e))+ where = 'start' if abs(i - s) <= 96 else 'end' if abs(i - e) <= 96 else 'body'+ what = a.get('sample') or a.get('name') or a.get('label') or f"{a.get('kind', a['bus'])} at {s / SR:.3f} s"+ res.append((where == 'body', d, {'bus': a['bus'], 'what': what,+ 'rule': a.get('rule', a.get('kind')), 'where': where,+ 'from_start_ms': round((i - s) / SR * 1000, 1),+ 'from_end_ms': round((i - e) / SR * 1000, 1)}))+ return [r for _, _, r in sorted(res, key=lambda r: (r[0], r[1]))[:3]]+++def hf_owner(ctx, i, ch):+ """the rebuilt sfx layer carrying the > 14 kHz burst at sample i (its level in the stem, where in the layer)"""+ best = None+ for L in getattr(ctx, 'layers', None) or []:+ if not L['a'] - 480 <= i < L['b'] + 480:+ continue+ j = i - L['a']+ lo, hi = max(0, j - 480), min(len(L['sig']), j + 480)+ if hi - lo < 64:+ continue+ hp = signal.sosfilt(HP14, L['sig'][lo:hi, ch])+ c = j - lo+ v = float(np.abs(hp[max(0, c - 24):c + 24]).max()) if 0 <= c < len(hp) else 0.0+ if best is None or v > best[0]:+ off = i - L['start']+ where = 'start' if abs(off) <= 96 else 'end' if abs(i - L['start'] - L['n']) <= 96 else 'body'+ best = (v, {'bus': 'sfx', 'what': L['sample'], 'rule': L['rule'], 'where': where,+ 'from_start_ms': round(off / SR * 1000, 1), 'level_db': round(db(v), 1)})+ return best+++def check_clicks(ctx):+ sets = [('master', ctx.master)] if ctx.master is not None else []+ sets += [(k, ctx.stems[k]) for k in ('sfx', 'amb', 'music')]+ items, counts = [], {}+ for name, x in sets:+ found = detect_clicks(x)+ counts[name] = {'clicks': sum(c['kind'] == 'click' for c in found), 'attacks': sum(c['kind'] == 'attack' for c in found)}+ for c in found:+ if c['kind'] != 'click':+ continue+ own = hf_owner(ctx, c['i'], c['ch']) if name in ('master', 'sfx') else None+ scale = 10 ** (-ctx.G / 20) if name == 'master' else 1.0 # master level back to stem scale+ if own and own[0] >= 0.3 * 10 ** (c['level_db'] / 20) * scale:+ ow = [own[1]]+ else:+ ow = owners(ctx.trace, c['i'], None if name == 'master' else name)+ items.append({'in': name, **{k: v for k, v in c.items() if k != 'i'}, 'owners': ow,+ 'at_edge': bool(ow) and ow[0]['where'] in ('start', 'end')})+ mc = [c for c in items if c['in'] == 'master']+ edge = [c for c in items if c['at_edge']]+ sc = len(items) - len(mc)+ status = 'fail' if any(c['in'] == 'master' for c in edge) else 'warn' if items else 'ok'+ summ = (f'{len(mc)} click(s) in the master, {sc} in the stems; {len(edge)} at a placement edge (a cut: fix the trim / '+ 'fade)' + (', the rest inside a recording (its own crisp transient or a defect in the source)' if items else '')+ + ('' if items else ' (designed attacks: ' + ', '.join(f"{k} {v['attacks']}" for k, v in counts.items()) + ')'))+ return {'status': status, 'summary': summ, 'counts': counts, 'items': items}+++# ------------------------------------------------------------------ edges+def rms(z):+ return float(np.sqrt((np.asarray(z, np.float64) ** 2).mean())) if len(z) else 0.0+++def sample_edges(x, raw_pre, raw_post, rawpk):+ """unfaded head / tail 1 ms RMS vs the sample's peak, the source 20 ms outside the trim vs the trim's peak, and a+ second hit (a 5 ms envelope jump > 12 dB to within 18 dB of the peak) in the last 40 %"""+ m = np.abs(x).max(axis=1)+ pk = float(m.max()) or 1e-9+ w = 240+ n = len(m) // w+ e = m[:n * w].reshape(n, w).max(axis=1) if n else np.zeros(1)+ late = None+ for k in range(max(1, int(n * 0.6)), n):+ if e[k] > pk * 0.12 and e[k] > 4 * e[k - 1]:+ late = round(k * w / SR, 3)+ break+ return {'head_db': round(db(rms(x[:48]) / pk), 1), 'tail_db': round(db(rms(x[-48:]) / pk), 1),+ 'pre20_db': round(db(rms(raw_pre) / rawpk), 1), 'post20_db': round(db(rms(raw_post) / rawpk), 1),+ 'attack_ms': round(mix.env_index(x, 'attack') / SR * 1000, 1), 'late_onset': late}+++def judge_edges(r, fi, fo, trimmed_start):+ """(defects, notes): a defect is audible as a click or a chop; a note is a craft suggestion"""+ bad, notes = [], []+ if trimmed_start and r['pre20_db'] > -20:+ msg = f"the trim cuts into a sound already playing (source 20 ms before at {r['pre20_db']} dB re peak)"+ if fi < 0.005 and (r['attack_ms'] > 10 or r['pre20_db'] > -10): # exposed: no attack right after to cover it+ bad.append(msg + f', fade_in only {fi * 1000:.0f} ms')+ else:+ notes.append(msg + ', softened by the fade-in')+ elif r['head_db'] > -30:+ notes.append(f"the attack starts in the first ms (head {r['head_db']} dB): the {fi * 1000:.0f} ms fade-in dulls it; "+ 'start the trim 3-5 ms earlier')+ if r['tail_db'] > -35 and r['post20_db'] > -35:+ msg = f"the sound continues after the trim (tail {r['tail_db']} dB, source after {r['post20_db']} dB)"+ if fo < 0.03:+ bad.append(msg + f': chopped with a {fo * 1000:.0f} ms fade')+ else:+ notes.append(msg + ': decay faded early')+ if r['late_onset'] is not None:+ notes.append(f"a second hit inside the sample at {r['late_onset']} s (fine for a rattle, wrong for a single hit)")+ return bad, notes+++def check_splices(ctx):+ """music segment boundaries: a segment end that chops a note still sounding into quieter material, holes and overlaps+ between segments, segments that begin inside a sustained sound"""+ mu = ctx.design.get('music')+ if not mu or not mu.get('segments'):+ return []+ ratio = mu.get('target_bpm', 120) / mu['src_bpm'] if mu.get('stretch', 'rubberband') != 'none' else 1.0+ x = mix.stretched(mu['file'], ratio)+ k = 1.0 / ratio+ w = int(0.05 * SR)++ def lv(i0):+ return db(rms(x[max(0, i0):max(0, i0 + w)]))++ segs = sorted(mu['segments'], key=lambda sg: sg['at'])+ dur = ctx.N / SR+ out = []+ for n, sg in enumerate(segs):+ s0, e0 = int(round(sg['src_from'] * k * SR)), int(round(sg['src_to'] * k * SR))+ end_t = sg['at'] + (sg['src_to'] - sg['src_from']) * k+ nxt = segs[n + 1] if n + 1 < len(segs) else None+ out_l, cut_l = lv(e0 - w), lv(e0)+ in_l = lv(int(round(nxt['src_from'] * k * SR))) if nxt else -120.0+ item = {'segment': sg.get('label') or n, 'film_end': round(end_t, 3), 'out_db': round(out_l, 1),+ 'cut_off_db': round(cut_l, 1), 'next_in_db': round(in_l, 1) if nxt else None, 'issues': [], 'notes': []}+ fo = sg.get('fade_out', 0.06)+ if not sg.get('tail_fade') and fo < 0.1 and end_t < dur - 0.05 and cut_l > out_l - 6 and cut_l > in_l + 6:+ msg = (f'chops the music at {end_t:.3f} s: the source keeps sounding at {cut_l:.0f} dBFS and the next '+ f'material enters at {in_l:.0f} (fade_out {fo * 1000:.0f} ms)')+ if sg.get('qa_ok'):+ item['notes'].append(f"acknowledged ({sg['qa_ok']}): {msg}")+ else:+ item['issues'].append(msg)+ if nxt:+ gap = nxt['at'] - end_t+ if gap > 0.02:+ item['notes'].append(f'{gap * 1000:.0f} ms of no music before {nxt["at"]:.3f} s (deliberate silence?)')+ elif gap < -0.1:+ item['notes'].append(f'overlaps the next segment by {-gap * 1000:.0f} ms (two passages at once)')+ if s0 > w and lv(s0 - w) > lv(s0) - 3:+ item['notes'].append(f"begins inside a sustained sound (the source before {sg['src_from']} s is as loud): "+ 'no fresh attack on the splice')+ if item['issues'] or item['notes']:+ out.append(item)+ return out+++def check_edges(ctx):+ items = []+ used = {p.get('sample') for p in ctx.report.get('placements', [])}+ lib = ctx.lib = ctx.lib or mix.Lib(ctx.design)+ for name, s in (ctx.design.get('samples') or {}).items():+ if name not in used:+ continue+ raw = mix.decode(s['file'])+ a = int(s.get('start', 0) * SR)+ b = int(s['end'] * SR) if s.get('end') is not None else len(raw)+ fi, fo = s.get('fade_in', 0.002), s.get('fade_out', 0.012)+ trim_pk = float(np.abs(raw[a:b]).max()) if b - a > 0 else 0.0+ if db(trim_pk) < -60: # nothing to judge: the trim missed the sound+ why = ('empty (start >= end or past the file end)' if b - a <= 0 else 'digital silence' if trim_pk == 0+ else f'near-silent (peak {db(trim_pk):.0f} dBFS)')+ items.append({'what': 'sample', 'name': name, 'file': os.path.basename(s['file']), 'start': s.get('start', 0),+ 'end': s.get('end'), 'trim_peak_dbfs': round(db(trim_pk), 1), 'fade_in': fi, 'fade_out': fo,+ 'issues': [f'the trim is {why}: it misses the sound (file length {len(raw) / SR:.3f} s)'],+ 'notes': []})+ continue+ x = lib._finish(raw[a:b], {**s, 'fade_in': 0, 'fade_out': 0})+ r = sample_edges(x, raw[max(0, a - 960):a], raw[b:b + 960], trim_pk)+ bad, notes = judge_edges(r, fi, fo, a > 0)+ if bad and s.get('qa_ok'):+ notes, bad = notes + [f'acknowledged ({s["qa_ok"]}): ' + b_ for b_ in bad], []+ if bad or notes:+ items.append({'what': 'sample', 'name': name, 'file': os.path.basename(s['file']), 'start': s.get('start', 0),+ 'end': s.get('end'), **r, 'fade_in': fi, 'fade_out': fo, 'issues': bad, 'notes': notes})+ for name, s in (ctx.design.get('slices') or {}).items():+ if not any(u and u.startswith(name + '#') for u in used):+ continue+ try:+ sl = lib._slice({**s, 'fade_in': 0, 'fade_out': 0})+ except Exception as e: # noqa: BLE001+ items.append({'what': 'slices', 'name': name, 'issues': [f'cannot slice: {e}'], 'notes': []})+ continue+ fi, fo = s.get('fade_in', 0.002), s.get('fade_out', 0.02)+ bad, notes = set(), set()+ for x, _ in sl:+ r = sample_edges(x, x[:0], x[:0], float(np.abs(x).max()) or 1e-9)+ r['post20_db'] = r['tail_db'] # a slice ends where the next hit begins: its tail level is what is cut+ b_, n_ = judge_edges(r, fi, fo, False)+ bad |= {m.split(' (')[0] for m in b_}+ notes |= {m.split(' (')[0] for m in n_}+ if bad or notes:+ items.append({'what': 'slices', 'name': name, 'n': len(sl), 'issues': sorted(bad), 'notes': sorted(notes)})+ cut = []+ for a in ctx.trace or []:+ if a['bus'] != 'music' and (a['start'] < 0 or a['start'] + a['n'] > ctx.N):+ cut.append({'bus': a['bus'], 'what': a.get('sample') or a.get('name') or a.get('label'),+ 'start': round(a['start'] / SR, 3), 'end': round((a['start'] + a['n']) / SR, 3)})+ splices = check_splices(ctx)+ n_bad = sum(1 for it in items + splices if it['issues'])+ n_note = sum(1 for it in items + splices if it['notes'] and not it['issues'])+ summ = (f'{n_bad} defect(s) (abrupt entries, chopped decays, chopped music), {n_note} with craft notes only (used '+ f'samples, slices, music splices); {len(cut)} placement(s) run past the film ends')+ return {'status': 'warn' if n_bad else 'ok', 'summary': summ,+ 'items': sorted(items, key=lambda it: (not it['issues'], it['name'])), 'splices': splices,+ 'cut_by_film_ends': cut}+++# ------------------------------------------------------------------ audibility / harshness / sync+def bands(seg):+ w = signal.windows.tukey(len(seg), 0.25)[:, None]+ S = (np.abs(np.fft.rfft(seg * w, axis=0)) ** 2).sum(axis=1)+ f = np.fft.rfftfreq(len(seg), 1 / SR)+ c = np.concatenate([[0], np.cumsum(S)])+ ia, ib = np.searchsorted(f, BLO), np.searchsorted(f, BHI)+ return c[ib] - c[ia] + 1e-30+++def window_at(arr, a0, s, w):+ o = arr[max(0, s - a0): max(0, s - a0) + w]+ return np.pad(o, ((0, w - len(o)), (0, 0))) if len(o) < w else o+++def read_metrics(ctx, eff, a, b, att, long):+ """m60 vs bed, m60r / m20 vs everything else (best band within 30 dB of the effect's strongest)"""+ bed = ctx.stems['music'][a:b].astype(np.float64) + ctx.stems['amb'][a:b]+ rest = ctx.stems['sfx'][a:b] - eff + bed+ e2 = (eff ** 2).sum(axis=1)+ c = np.concatenate([[0], np.cumsum(e2)])++ def best(w, s0, s1):+ s0 = max(a, s0)+ s1 = max(s0, min(b - w, s1))+ cand = np.arange(s0, s1 + 1, 48)+ en = [c[min(len(c) - 1, s - a + w)] - c[max(0, s - a)] for s in cand]+ return int(cand[int(np.argmax(en))])++ w60, w20 = int(0.06 * SR), int(0.02 * SR)+ s60 = best(w60, att - 240, (b - w60) if long else att + int(0.3 * SR))+ s20 = s60 if long else best(w20, att - 144, att + int(0.08 * SR))+ E, B, Rr = (bands(window_at(z, a, s60, w60)) for z in (eff, bed, rest))+ E20, R20 = bands(window_at(eff, a, s20, w20)), bands(window_at(rest, a, s20, w20))++ def mb(Ep, Bp):+ ok = Ep >= Ep.max() * 1e-3+ v = np.where(ok, np.clip(10 * np.log10(Ep / Bp), -CAP_DB, CAP_DB), -99)+ k = int(np.argmax(v))+ return round(float(v[k]), 1), int(round(FC[k]))++ m60, f60 = mb(E, B)+ m60r, _ = mb(E, Rr)+ m20, f20 = mb(E20, R20)+ read = max(m60r, m20)+ verdict = 'clear' if read >= 6 else 'ok' if read >= 3 else 'barely' if read >= 0 else 'masked'+ return {'vs_bed_db': m60, 'vs_bed_hz': f60, 'vs_all_db': m60r, 'first20_db': m20, 'first20_hz': f20,+ 'read_db': read, 'verdict': verdict, 'covered_by': 'music/amb' if m60 < 3 else 'other effects'}+++def check_audibility(ctx, groups, layers_too=False):+ rows = []+ for g in groups:+ att = min(L['att'] for L in g['layers'])+ r = {'t': round(att / SR, 3), 'label': g['label'], 'kind': g['kind'], 'n_layers': len(g['layers']),+ 'rules': sorted({L['rule'] for L in g['layers']}), 'peak_db': round(max(g['peak_db'], -120), 1)}+ if g['silent']:+ r.update({'read_db': None, 'verdict': 'silent',+ 'samples': sorted({L['sample'] for L in g['layers']})})+ elif ctx.no_bed:+ r['verdict'] = 'not judged'+ else:+ r.update(read_metrics(ctx, g['sig'], g['a'], g['b'], att, g['long']))+ if g['qa_ok']:+ r['qa_ok'] = g['qa_ok']+ rows.append(r)+ cnt = {}+ for r in rows:+ cnt[r['verdict']] = cnt.get(r['verdict'], 0) + 1+ masked = [r for r in rows if r['verdict'] == 'masked' and not r.get('qa_ok')]+ silent = [r for r in rows if r['verdict'] == 'silent' and not r.get('qa_ok')]+ sil = (f'; {len(silent)} SILENT (peak under {SILENT_DB:.0f} dBFS: the trim missed the sound) at '+ + ', '.join(f"{r['t']:.2f} {r['label']} ({', '.join(r['samples'])})" for r in silent[:6]) if silent else '')+ if ctx.no_bed:+ return {'status': 'fail' if silent else 'skipped', 'counts': cnt, 'events': rows, 'items': silent,+ 'summary': f'{len(rows)} events, not judged against a bed: this render has no music or ambience (a '+ 'solo render); judge audibility on the full mix' + sil}+ judged = sorted((r for r in rows if r['read_db'] is not None), key=lambda r: r['read_db'])+ res = {'status': 'fail' if silent else 'warn' if masked else 'ok', 'counts': cnt,+ 'summary': f'{len(rows)} events: ' + ', '.join(f'{k} {cnt.get(k, 0)}' for k in ('clear', 'ok', 'barely', 'masked'))+ + sil + ('' if not masked else '; masked at ' + ', '.join(f"{r['t']:.2f} {r['label']} (by {r['covered_by']})"+ for r in masked[:6])),+ 'items': (silent + judged)[:40], 'events': rows}+ if layers_too:+ dead = []+ for g in groups:+ if len(g['layers']) < 2 or g['silent']:+ continue+ for L in g['layers']:+ if L['peak_db'] + ctx.G < SILENT_DB:+ dead.append({'t': round(L['att'] / SR, 3), 'rule': L['rule'], 'sample': L['sample'], 'read_db': None,+ 'silent': True})+ continue+ r = read_metrics(ctx, L['sig'], L['a'], L['b'], L['att'], g['long'])+ if r['read_db'] < 0:+ dead.append({'t': round(L['att'] / SR, 3), 'rule': L['rule'], 'sample': L['sample'], 'read_db': r['read_db']})+ res['dead_layers'] = dead+ res['summary'] += f'; {len(dead)} layer(s) in multi-layer events add nothing measurable (read < 0 dB)'+ return res+++def centroid(x):+ S = np.abs(np.fft.rfft(x * np.hanning(len(x)))) ** 2+ f = np.fft.rfftfreq(len(x), 1 / SR)+ return float((f * S).sum() / (S.sum() + 1e-30))+++def check_harshness(ctx, groups):+ sos = signal.butter(4, [2000, 6000], 'bandpass', fs=SR, output='sos')+ sol = signal.butter(4, [80, 400], 'bandpass', fs=SR, output='sos')+ mu = signal.sosfilt(sos, ctx.stems['music'].astype(np.float64).mean(axis=1))+ e5 = np.convolve(mu ** 2, np.ones(240) / 240, 'same')+ has_music = e5.max() > 1e-10 # a solo render without the music: presence is not judged+ ref = pdb(e5.max()) + ctx.G if has_music else None+ rows = []+ for g in groups:+ att = min(L['att'] for L in g['layers'])+ x = g['sig'].mean(axis=1)+ pres = signal.sosfilt(sos, np.r_[np.zeros(480), x])[480:]+ lo = signal.sosfilt(sol, np.r_[np.zeros(480), x])[480:]+ i0 = max(0, att - g['a'] - 480)+ i1 = len(x) if g['long'] else min(len(x), att - g['a'] + int(0.08 * SR))+ if i1 - i0 < 240:+ continue+ pk = pdb(np.convolve(pres[i0:i1] ** 2, np.ones(240) / 240, 'valid').max()) + ctx.G+ lk = pdb(np.convolve(lo[i0:i1] ** 2, np.ones(240) / 240, 'valid').max()) + ctx.G+ m0, m1 = max(0, att - SR // 4), min(len(mu), att + SR // 4)+ r = {'t': round(att / SR, 3), 'label': g['label'], 'rules': sorted({L['rule'] for L in g['layers']}),+ 'pres_db': round(pk, 1), 'pres_minus_low_db': round(pk - lk, 1), 'centroid_hz': int(centroid(x[i0:i1])),+ 'music_pres_db': round(pdb((mu[m0:m1] ** 2).mean()) + ctx.G, 1),+ 'over_music_max_db': round(pk - ref, 1) if has_music else None}+ r['issues'] = []+ if has_music and r['over_music_max_db'] > 6:+ r['issues'].append(f"2-6 kHz peak {r['pres_db']} dBFS, {r['over_music_max_db']:+.1f} dB over the music's loudest")+ r['bright'] = r['centroid_hz'] > 6500+ if g['qa_ok'] and r['issues']:+ r['qa_ok'], r['issues'] = g['qa_ok'], []+ rows.append(r)+ rows.sort(key=lambda r: -r['pres_db'])+ hot = [r for r in rows if r['issues']]+ bright = [r for r in rows if r['bright']]+ head = (f"{len(hot)} event(s) more than 6 dB over the music's loudest 2-6 kHz peak ({ref:.1f} dBFS)" if has_music else+ 'no music in this render: presence vs the music not judged')+ return {'status': 'warn' if hot else 'ok', 'music_max_db': round(ref, 1) if has_music else None,+ 'summary': (head + ('' if not hot else ': ' + ', '.join(f"{r['t']:.2f} {r['label']}" for r in hot[:6]))+ + f"; note: {len(bright)} bright event(s) (centroid > 6.5 kHz, above launch-film effects)"),+ 'items': hot[:20] + [r for r in rows if not r['issues']][:max(0, 10 - len(hot))],+ 'bright': [{'t': r['t'], 'label': r['label'], 'centroid_hz': r['centroid_hz']} for r in bright]}+++def sync_issue(test, r, s):+ """the sync verdict for a measured event shifted by s ms (a rule offset): None when it sits on its cue"""+ if test == 'end': # a riser stops on (or 30-250 ms before) its hit+ e = r['end_ms'] - s+ return f"its audible end is {e:+d} ms from its hit (want -250..+30)" if not -300 <= e <= 60 else None+ if test == 'apex':+ e = r['apex_ms'] - s+ return f"apex {e:+d} ms from the cue (want +-45)" if abs(e) > 45 else None+ a, p = r['attack_ms'] - s, r['peak_ms'] - s+ return (f"attack {a:+d} ms (sound before picture)" if a < -45 else f"peak {p:+d} ms (lands late)" if p > 75 else None)+++def check_sync(ctx, groups):+ """impacts by attack / peak, whooshes and peak-aligned swells by their apex, risers (and swells placed by their+ end or attack) by their audible end vs the hit; a deviation that one layer's rule offset explains is deliberate"""+ rows, deliberate, n = [], [], 0+ for g in groups:+ x = np.abs(g['sig']).max(axis=1)+ k = len(x) // 240+ if not k:+ continue+ align = g['loud']['align']+ test = (('apex' if align == 'peak' else 'end') if g['builds'] else None if align in ('start', 'end') else+ 'apex' if g['long'] else 'impact')+ if test is None: # placed by its start / end on purpose (not a build): no hit to judge+ continue+ n += 1+ pk = (int(np.argmax(x[:k * 240].reshape(k, 240).max(axis=1))) * 240 + 120 + g['a']) / SR+ p = np.convolve((g['sig'] ** 2).sum(axis=1), np.ones(2400) / 2400, 'same')+ apex = (int(np.argmax(p)) + g['a']) / SR+ att = g['loud']['att'] / SR+ r = {'t_cue': round(g['cue'], 3), 'label': g['label'], 'kind': g['kind'], 'test': test, 'align': align,+ 'attack_ms': round((att - g['cue']) * 1000), 'peak_ms': round((pk - g['cue']) * 1000),+ 'apex_ms': round((apex - g['cue']) * 1000), 'long': g['long']}+ if test == 'end':+ nz = np.nonzero(x > x.max() * 0.1)[0] # the audible body: above 10 % of the peak (-20 dB)+ r['end_ms'] = round(((nz[-1] if len(nz) else 0) + g['a']) / SR * 1000 - g['cue'] * 1000)+ r['issue'] = sync_issue(test, r, 0)+ if not r['issue']:+ continue+ offs = sorted({(round(L['offset'] * 1000), L['rule']) for L in g['layers'] if abs(L['offset']) >= 0.03})+ why = next(((o, rule) for o, rule in offs if not sync_issue(test, r, o)), None)+ if why:+ r.update({'offset_s': why[0] / 1000, 'offset_rule': why[1],+ 'explained': f"{re.sub(r' [(].*', '', r['issue'])} = rule {why[1]}'s offset {why[0] / 1000:+.3f} s"})+ deliberate.append(r)+ elif g['qa_ok']:+ r['qa_ok'] = g['qa_ok']+ deliberate.append(r)+ else:+ if offs:+ r['offsets_tried_s'] = [o / 1000 for o, _ in offs]+ rows.append(r)+ return {'status': 'warn' if rows else 'ok', 'checked': n, 'items': rows, 'deliberate': deliberate,+ 'summary': f'{n} events checked, {len(rows)} off their cue'+ + (': ' + ', '.join(f"{r['t_cue']:.2f} {r['label']} {r['issue']}" for r in rows[:5]) if rows else '')+ + f'; {len(deliberate)} more sit off the cue by exactly a rule offset or carry a qa_ok note (check those on '+ 'the frames)'}+++def check_density(ctx, groups):+ per_s = {}+ for g in groups:+ s = int(min(L['att'] for L in g['layers']) / SR)+ per_s[s] = per_s.get(s, 0) + 1+ pl = [p for p in ctx.report.get('placements', []) if p.get('kind') != 'place']+ use = {}+ for p in ctx.report.get('placements', []):+ use[p['sample']] = use.get(p['sample'], 0) + 1+ by = {}+ for p in pl:+ by.setdefault(str(p['rule']), []).append(p)+ rules, mg = [], []+ for k, v in by.items():+ if len(v) < 4:+ continue+ t = np.array(sorted(p['t'] for p in v))+ ioi = np.diff(t)+ g = np.array([p['gain_db'] for p in v])+ pt = np.array([p.get('pitch', 0) for p in v])+ r = {'rule': k, 'n': len(v), 'distinct_samples': len({p['sample'] for p in v}), 'ioi_s': round(float(ioi.mean()), 3),+ 'ioi_cv': round(float(ioi.std() / max(ioi.mean(), 1e-9)), 2), 'gain_sd_db': round(float(g.std()), 2),+ 'pitch_sd_st': round(float(pt.std()), 2)}+ rules.append(r)+ if r['distinct_samples'] == 1 and r['pitch_sd_st'] < 0.1 and r['gain_sd_db'] < 0.5:+ mg.append(r)+ n = ctx.N / SR+ busiest = sorted(per_s.items(), key=lambda kv: -kv[1])[:5]+ return {'status': 'warn' if mg else 'ok', 'events_per_s': [per_s.get(s, 0) for s in range(int(math.ceil(n)))],+ 'busiest_s': busiest, 'most_used_samples': sorted(use.items(), key=lambda kv: -kv[1])[:8], 'rules': rules,+ 'machine_gun': mg,+ 'summary': (f'{len(groups)} effect events in {n:.0f} s ({len(groups) / n * 10:.1f} per 10 s); busiest seconds '+ + ', '.join(f'{s}s:{c}' for s, c in busiest) + f'; {len(mg)} machine-gun rule(s)'+ + (': ' + ', '.join(r['rule'] for r in mg) if mg else ''))}+++# ------------------------------------------------------------------ loudness / coverage+def check_loudness(ctx):+ x = ctx.master+ if x is None:+ return {'status': 'skipped', 'summary': 'no master WAV (pass --master)'}+ if len(x) < int(0.4 * SR):+ return {'status': 'skipped', 'summary': f'the master is {len(x) / SR * 1000:.0f} ms long: loudness needs >= 400 ms'}+ if not np.any(x):+ return {'status': 'fail', 'lufs': None, 'summary': 'the master is digital silence (no gated loudness)'}+ I, tp, lr = M.integrated(x), M.true_peak_db(x), M.lra(x)+ tm, mom = M.blocks(x, 0.4, 0.1)+ ts, st = M.blocks(x, 3.0, 1.0)+ k = int(np.argmax(mom))+ dur = len(x) / SR+ head = db(np.abs(x[:240]).max())+ tail_rms = db(np.sqrt((x[-2400:] ** 2).mean()))+ tail_pk = db(np.abs(x[-480:]).max())+ r = {'lufs': round(I, 2), 'target': ctx.target, 'true_peak_dbtp': round(tp, 2), 'lra_lu': round(lr, 1),+ 'premaster_lufs': round(ctx.pre_lufs, 2) if ctx.pre_lufs is not None else None, 'loudest_momentary': round(float(mom[k]), 1),+ 'loudest_at': round(float(tm[k]), 2), 'loudest_frac': round(float(tm[k]) / dur, 2),+ 'short_term_per_s': [round(float(v), 1) for v in st], 'head_5ms_peak_dbfs': round(head, 1),+ 'tail_50ms_rms_dbfs': round(tail_rms, 1), 'tail_10ms_peak_dbfs': round(tail_pk, 1)}+ acts = ctx.cues.get('acts') or {}+ if acts:+ r['acts'] = {}+ for name, v in acts.items():+ t0, t1 = v['from'] / ctx.fps, (v['from'] + v['dur']) / ctx.fps+ sel = (tm >= t0) & (tm < t1)+ if sel.any():+ r['acts'][name] = {'from': round(t0, 2), 'momentary_median': round(float(np.median(mom[sel])), 1),+ 'momentary_max': round(float(mom[sel].max()), 1)}+ fails, warns = [], []+ if abs(I - ctx.target) > 0.5:+ fails.append(f'integrated {I:.1f} LUFS, target {ctx.target}')+ if tp > -1.0:+ fails.append(f'true peak {tp:.1f} dBTP (> -1.0: AAC will clip)')+ if tail_rms > -45 or tail_pk > -60:+ fails.append(f'the end is not silent (last 50 ms {tail_rms:.0f} dBFS RMS, last 10 ms peak {tail_pk:.0f})')+ if head > -40:+ warns.append(f'first 5 ms peak {head:.0f} dBFS: starts mid-sound or clicks on sample 0')+ if lr < 4:+ warns.append(f'LRA {lr:.1f} LU: flat (launch films with an arc sit at 5-9 LU)')+ elif lr > 10:+ warns.append(f'LRA {lr:.1f} LU: wider than launch films (5-9 LU); quiet passages may vanish on phones')+ if r['loudest_frac'] > 0.85:+ warns.append(f'the loudest moment is at {r["loudest_frac"]:.0%} of the runtime: peak at the drop, resolve at the end')+ r['status'] = 'fail' if fails else 'warn' if warns else 'ok'+ r['summary'] = (f'{I:.1f} LUFS (target {ctx.target}), true peak {tp:.1f} dBTP, LRA {lr:.1f} LU, loudest momentary '+ f'{mom[k]:.1f} LUFS at {fmt_t(tm[k])} ({r["loudest_frac"]:.0%} of the runtime)'+ + ('; ' + '; '.join(fails + warns) if fails or warns else ''))+ return r+++def stem_id(i):+ return re.sub(r'(:\d+)+$', '', str(i or ''))+++def check_coverage(ctx, groups=None):+ """cue events with no sound: unsounded in the report, or placed but silent (every layer below SILENT_DB)"""+ ev = ctx.cues.get('events') or []+ uns = list(ctx.report.get('unsounded_events', []))+ quiet = [{'t': round(g['cue'], 3), 'kind': g['kind'], 'id': g['layers'][0]['event']}+ for g in groups or [] if g['silent'] and not g.get('place') and not g['qa_ok']]+ uns += quiet+ tot, miss = {}, {}+ for e in ev:+ k = (e['kind'], stem_id(e.get('id')))+ tot[k] = tot.get(k, 0) + 1+ for e in uns:+ k = (e['kind'], stem_id(e.get('id')))+ miss.setdefault(k, []).append(e['t'])+ partial = [{'kind': k[0], 'id': k[1], 'unsounded': len(v), 'of': tot.get(k, len(v)), 'at': v[:12]}+ for k, v in miss.items() if len(v) < tot.get(k, len(v))]+ silent = [{'kind': k[0], 'id': k[1], 'n': len(v), 'at': v[:12]} for k, v in miss.items() if len(v) >= tot.get(k, len(v))]+ return {'status': 'warn' if partial or quiet else 'ok', 'partial': partial, 'unsounded_classes': silent,+ 'placed_but_silent': quiet,+ 'summary': (('solo render (events of the other fragments are expected to have no sound): ' if ctx.only else '')+ + f'{len(uns)} of {len(ev)} cue events have no sound'+ + (f' ({len(quiet)} of them placed but silent: ' + ', '.join(f"{q['t']:.2f} {q['id']}" for q in quiet[:5])+ + ')' if quiet else '') + f'; {len(partial)} class(es) of action sounded '+ f'only in part' + (': ' + ', '.join(f"{p['kind']} {p['id']} ({p['unsounded']}/{p['of']} silent)"+ for p in partial[:5]) if partial else '')+ + (f'; classes never sounded: ' + ', '.join(f"{s['kind']} {s['id']} x{s['n']}" for s in silent[:8])+ if silent else ''))}+++# ------------------------------------------------------------------ earshot+def listen(ctx, a):+ try:+ from earshot import api+ except Exception as e: # noqa: BLE001+ return {'status': 'skipped', 'summary': f'earshot is not importable here ({e}): pip install -e <earshot repo>'}+ out_dir = os.path.dirname(ctx.R(a.out))+ os.makedirs(out_dir, exist_ok=True)+ res, notes = {}, []+ video = ctx.R(a.video) if a.video else None+ if a.listen and ctx.master is not None:+ mp = ctx.R(a.master)+ cues_used = ctx.R(a.cues_sounded) if a.cues_sounded and os.path.exists(ctx.R(a.cues_sounded)) else None+ tl = {}+ steps = (('timeline', lambda: tl.setdefault('r', api.timeline(mp, video=video, cues=cues_used))),+ ('sheet', lambda: api.sheet(mp, os.path.join(out_dir, 'qa_sheet.png'), video=video, cues=cues_used,+ tl=tl.get('r'))),+ ('critique', lambda: api.critique(mp, video=video, use_ast=False, use_realness=False)))+ for name, call in steps: # the critique is the fast scorecard: realness of the sources is --realness's job+ try:+ r = call()+ res[name] = {'summary': r.get('summary'), **({'png': r.get('png') or r.get('path')} if name == 'sheet' else {})}+ if name == 'timeline' and r.get('text'):+ p = os.path.join(out_dir, 'qa_timeline.txt')+ open(p, 'w', encoding='utf-8').write(r['text'])+ res[name]['text_file'] = p+ if name == 'critique':+ res[name].update({'verdict': r.get('verdict'), 'score': r.get('score'), 'actions': r.get('actions'),+ 'off_range': [i for i in r.get('metrics') or [] if isinstance(i, dict)+ and i.get('status') in ('low', 'high')]})+ except Exception as e: # noqa: BLE001+ notes.append(f'{name}: {type(e).__name__}: {str(e)[:160]}')+ if a.realness:+ rows = []+ for f in ctx.report.get('files_used', []):+ try:+ r = api.realness(ctx.R(f))+ rows.append({'file': f, 'p_real': r.get('p_real'), 'verdict': r.get('verdict')})+ except Exception as e: # noqa: BLE001+ notes.append(f'realness {os.path.basename(f)}: {str(e)[:160]}')+ rows.sort(key=lambda r: (r['p_real'] is None, r['p_real'] if r['p_real'] is not None else 1))+ res['realness'] = {'files': rows, 'not_real': [r for r in rows if (r['p_real'] < 0.5 if r['p_real'] is not None+ else 'synth' in str(r['verdict']).lower())]}+ short = lambda k, v: (v['summary'][:360].rsplit(' ', 1)[0] + f" ... (all of it: {v['text_file']})"+ if k == 'timeline' and len(v['summary']) > 400 and v.get('text_file') else v['summary'])+ parts = [f"{k}: {short(k, v)}" for k, v in res.items() if k != 'realness' and v.get('summary')]+ if 'realness' in res:+ rr = res['realness']+ parts.append(f"realness: {len(rr['not_real'])} of {len(rr['files'])} sources judged not real" if rr['files'] else+ 'realness: not available (' + (notes[0] if notes else 'no sources') + ')')+ if notes:+ parts.append(f'{len(notes)} earshot call(s) failed (see notes)')+ bad = bool(res.get('realness', {}).get('not_real'))+ return {'status': 'warn' if bad or notes else 'ok', 'summary': '; '.join(parts) or 'nothing ran', **res, 'notes': notes}+++# ------------------------------------------------------------------ run+def run_qa(a):+ ctx = Ctx(a)+ groups, note = build_layers(ctx) if ctx.trace is not None else (None, None)+ live = [g for g in groups if not g['silent']] if groups else groups+ sec = {'clicks': check_clicks(ctx), 'loudness': check_loudness(ctx), 'coverage': check_coverage(ctx, groups)}+ if ctx.trace is None:+ msg = 'needs the trace: run mix.py with --trace out/qa/adds.json'+ for k in ('edges', 'audibility', 'harshness', 'sync', 'density'):+ sec[k] = {'status': 'skipped', 'summary': msg}+ else:+ sec['edges'] = check_edges(ctx)+ if groups is None:+ for k in ('audibility', 'harshness', 'sync', 'density'):+ sec[k] = {'status': 'skipped', 'summary': note}+ else:+ sec['audibility'] = check_audibility(ctx, groups, a.layers)+ sec['harshness'] = check_harshness(ctx, live)+ sec['sync'] = check_sync(ctx, live)+ sec['density'] = check_density(ctx, live)+ if a.listen or a.realness:+ sec['listen'] = listen(ctx, a)+ if not any(np.any(v) for v in ctx.stems.values()):+ sec['mix'] = {'status': 'fail', 'summary': 'every stem is digital silence: nothing was mixed (check the trims with '+ 'mix.py --dump, the gains and the cue matches)'}+ order = {'fail': 0, 'warn': 1, 'ok': 2, 'skipped': 3}+ worst = min((s['status'] for s in sec.values()), key=lambda s: order.get(s, 9))+ bad = [k for k, s in sec.items() if s['status'] in ('fail', 'warn')]+ skipped = [k for k, s in sec.items() if s['status'] == 'skipped']+ summary = (f"QA {worst.upper()}: " + (', '.join(f"{k} {sec[k]['status']}" for k in bad) if bad else+ 'every check that ran passed' if skipped else 'every check passed')+ + (f" (not run: {', '.join(skipped)})" if skipped else '')+ + (f". Master gain over the stems {ctx.G:+.1f} dB (levels below are at master scale)." if ctx.scale == 'master'+ else '. Master or premaster silent / too short: levels below are at stem scale.' if ctx.master is not None+ else '. No master: levels below are at stem scale.'))+ return {'summary': summary, 'status': worst, 'note': note, 'sections': sec,+ 'inputs': {'design': a.design, 'report': a.report, 'trace': a.trace, 'stems': a.stems, 'master': a.master,+ 'only': a.only}}+++def digest(res):+ lines = [res['summary']]+ for k, s in res['sections'].items():+ lines.append(f"[{s['status'].upper():7s}] {k}: {s['summary']}")+ if k == 'clicks':+ for c in s.get('items', [])[:6]:+ o = c['owners'][0] if c['owners'] else {}+ lines.append(f" {c['in']} {c['t']:.3f} ({fmt_t(c['t'])}) ch{c['ch']} conc {c['conc']}: "+ f"{o.get('bus', '?')}:{o.get('what', '?')} [{o.get('rule', '')}] at its {o.get('where', '?')}")+ elif k == 'edges':+ for it in [i for i in s.get('items', []) + s.get('splices', []) if i['issues']][:8]:+ lines.append(f" {it.get('what', 'music')} {it.get('name', it.get('segment'))}: {'; '.join(it['issues'])}")+ elif k == 'audibility':+ for r in s.get('items', [])[:6]:+ if r['verdict'] == 'silent':+ lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: SILENT, peak {r['peak_db']} dBFS "+ f"[{', '.join(r['samples'])[:80]}]")+ elif r['verdict'] in ('masked', 'barely'):+ lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: read {r['read_db']} dB ({r['verdict']}), "+ f"vs bed {r['vs_bed_db']} dB @ {r['vs_bed_hz']} Hz [{', '.join(r['rules'])[:80]}]")+ elif k == 'harshness':+ for r in [r for r in s.get('items', []) if r['issues']][:5]:+ lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: {'; '.join(r['issues'])} [{', '.join(r['rules'])[:70]}]")+ elif k == 'sync':+ for r in s.get('items', [])[:6]:+ lines.append(f" cue {r['t_cue']:.3f} ({fmt_t(r['t_cue'])}) {r['label']}: {r['issue']}")+ for r in s.get('deliberate', [])[:6]:+ lines.append(f" cue {r['t_cue']:.3f} ({fmt_t(r['t_cue'])}) {r['label']}: deliberate, "+ + (r.get('explained') or f"{r['issue']} (qa_ok: {r.get('qa_ok')})"))+ return '\n'.join(lines)+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')+ ap.add_argument('--design', default='audio/sounddesign.json')+ ap.add_argument('--report', help='default out/qa/mix_report.json (--only: out/audio-review/solo/<only>_report.json)')+ ap.add_argument('--trace', default='out/qa/adds.json', help='mix.py --trace output (attribution, layers)')+ ap.add_argument('--stems', help='default out/audio/stems (--only: out/audio-review/solo/<only>_stems)')+ ap.add_argument('--master', help='default out/audio/soundtrack.wav (--only: none); "none" skips it')+ ap.add_argument('--cues', help='cue file (default: the design\'s "cues", else out/cues.json)')+ ap.add_argument('--cues-sounded', default='out/qa/cues_sounded.json', help='for earshot timeline (--listen)')+ ap.add_argument('--only', help='the same --only the mix was rendered with')+ ap.add_argument('--cache', help='decode cache (default <root>/out/audio/cache, as mix.py)')+ ap.add_argument('--video', help='the film, for earshot (--listen)')+ ap.add_argument('--out', default='out/qa/qa.json')+ ap.add_argument('--layers', action='store_true', help='also measure every layer of multi-layer events (dead layers)')+ ap.add_argument('--listen', action='store_true', help='earshot timeline + sheet + critique of the master')+ ap.add_argument('--realness', action='store_true', help='earshot realness of every source recording used')+ ap.add_argument('--json', action='store_true', help='print the full result as JSON')+ a = ap.parse_args(argv)+ base = 'out/audio-review/solo/' + re.sub(r'[^A-Za-z0-9._-]+', '_', a.only) if a.only else None+ a.report = a.report or (base + '_report.json' if base else 'out/qa/mix_report.json')+ a.stems = a.stems or (base + '_stems' if base else 'out/audio/stems')+ a.master_given = bool(a.master) and a.master.lower() != 'none'+ a.master = None if (a.master or '').lower() == 'none' else a.master or (None if base else 'out/audio/soundtrack.wav')+ out = os.path.normpath(a.out if os.path.isabs(a.out) else os.path.join(os.path.abspath(a.root), a.out))+ os.makedirs(os.path.dirname(out), exist_ok=True)+ try:+ res = run_qa(a)+ except InputError as e:+ say(f'qa.py: {e}')+ return 2+ full = clean(res)+ json.dump(full, open(out, 'w'), indent=1)+ say(json.dumps(full, indent=1) if a.json else digest(res) + f'\nwrote {out}')+ return 1 if res['status'] == 'fail' else 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/scripts/variant.py b/skills/real-sound/scripts/variant.pynew file mode 100644index 0000000..18fa7ed--- /dev/null+++ b/skills/real-sound/scripts/variant.py@@ -0,0 +1,320 @@+#!/usr/bin/env python+"""Apply design operations: build a variant to test (critics) or apply the agreed fix (the fixer).++ python -I <skill>/scripts/variant.py --ops ops.json --out out/audio-review/critic_tonal/v1 [--render] [--qa]+ [--cues out/cues.json] [--cache DIR]+ python -I <skill>/scripts/variant.py --ops ops.json --apply [--backup out/audio-review/backup_round1]+ [--root .] [--design audio/sounddesign.json]++A variant is a copy of the top-level design and its fragments in --out (design/<fragment>.json, include paths+rewritten), with the operations applied; --render mixes it with every output inside --out (soundtrack.wav, stems/,+report.json, cues.json, adds.json) and --qa runs qa.py on it (qa.json). --apply edits the real files instead, after+copying them to --backup (never overwritten). Operations (a JSON list, applied in order; "f" = fragment file name+without .json, "sd" = a dotted path in the top-level design):+ {"f": "keyboard", "sample": "kb.thock.1", "set": {"norm_peak_db": -10}} also "slice": name+ {"f": "keyboard", "sample": "kb.thock.1", "eq": {"highshelf": [5000, -6]}} merge into its eq+ {"f": "motion", "rule": "mo.leap.air", "set": {"offset": 0.3}} {"..., "rule": name, "delete": true}+ {"f": "motion", "rule_add": {...}} {"f": "garden", "place": "gd.crowd_a", "set": {...}} {"f": ..., "place_add": {...}}+ {"f": "ambience", "amb": "am.room", "set": {"gain_db": -3}} {"f": "music", "segment": 2, "set": {...}}+ {"f": "toy", "pool": "ty.p.row", "value": ["ty.row.a", "ty.row.b"]}+ {"sd": "buses.music.auto", "value": [[0, 0], [12, -3]]} {"sd": "master.lufs", "value": -15}+Prints every change as old -> new (or unchanged); an operation that matches nothing is an error (nothing is+written). Only files whose content changes are written, and only the changed values are rewritten in place (hand+alignment, key order, comments and number spelling elsewhere stay as they were), so the fix reads as a small diff; the+files written are listed.+"""+import argparse, copy, difflib, json, os, re, shutil, subprocess, sys++HERE = os.path.dirname(os.path.abspath(__file__))+WS = re.compile(r'[ \t\r\n]*')+DEC = json.JSONDecoder()+++# ------------------------------------------------------------------ format-preserving JSON edits+def scan(t, i, path, sp, kids):+ """parse the JSON value at t[i:], recording sp[path] = (item_start, value_start, value_end) for every value and+ kids[path] = child keys / indices in order; returns the end index"""+ i = WS.match(t, i).end()+ item = sp.get(path, (i,))[0]+ if t[i] in '{[':+ close, kids[path] = '}' if t[i] == '{' else ']', []+ j = WS.match(t, i + 1).end()+ n = 0+ while t[j] != close:+ if close == '}':+ key, k = json.decoder.scanstring(t, j + 1)+ k = WS.match(t, k).end() + 1 # past the colon+ sub = path + (key,)+ else:+ k, sub = j, path + (n,)+ sp[sub] = (j,)+ kids[path].append(sub[-1])+ j = WS.match(t, scan(t, k, sub, sp, kids)).end()+ if t[j] == ',':+ j = WS.match(t, j + 1).end()+ n += 1+ sp[path] = (item, i, j + 1)+ return j + 1+ _, j = DEC.raw_decode(t, i)+ sp[path] = (item, i, j)+ return j+++def dump1(v):+ return json.dumps(v, ensure_ascii=False)+++def edits_for(t, old, new, path, sp, kids, out):+ """text edits (start, end, replacement) that turn the value at path from old into new, touching only what changed"""+ if old == new and type(old) is type(new):+ return+ if isinstance(old, dict) and isinstance(new, dict) and path in kids:+ ks = kids[path]+ m = 0+ while m < len(ks):+ if ks[m] in new:+ edits_for(t, old[ks[m]], new[ks[m]], path + (ks[m],), sp, kids, out)+ m += 1+ continue+ m1 = m # a run of deleted entries m..m1, removed with one separator+ while m1 + 1 < len(ks) and ks[m1 + 1] not in new:+ m1 += 1+ at = lambda n, w: sp[path + (ks[n],)][w]+ out.append((at(m, 0), at(m1 + 1, 0), '') if m1 + 1 < len(ks) else+ (at(m - 1, 2), at(m1, 2), '') if m else (at(m, 0), at(m1, 2), ''))+ m = m1 + 1+ add = [k for k in new if k not in old]+ if add:+ insert(t, path, sp, kids, [f'{dump1(k)}: {dump1(new[k])}' for k in add], out)+ return+ if isinstance(old, list) and isinstance(new, list) and path in kids:+ sm = difflib.SequenceMatcher(None, list(map(ident, old)), list(map(ident, new)), autojunk=False)+ for op, a0, a1, b0, b1 in sm.get_opcodes():+ if op == 'equal' or (op == 'replace' and a1 - a0 == b1 - b0): # the same items (or one for one): patch inside+ for d in range(a1 - a0):+ edits_for(t, old[a0 + d], new[b0 + d], path + (a0 + d,), sp, kids, out)+ elif op == 'insert':+ insert(t, path, sp, kids, [dump1(x) for x in new[b0:b1]], out, after=a0 - 1)+ else: # delete / uneven replace: the old items' whole stretch (with one separator) -> the new items+ s0 = sp[path + (a0,)][0]+ e0 = sp[path + (a1 - 1,)][2]+ txt = sep(t, path, sp, kids).join(dump1(x) for x in new[b0:b1])+ if not txt:+ if a1 < len(old):+ e0 = sp[path + (a1,)][0]+ elif a0:+ s0 = sp[path + (a0 - 1,)][2]+ out.append((s0, e0, txt))+ return+ _, s0, e0 = sp[path]+ out.append((s0, e0, dump1(new)))+++def ident(x):+ """what makes a list item the same item after an edit: a dict's name / sample / label, else its whole value"""+ k = [(key, dump1(x[key])) for key in ('name', 'sample', 'label') if isinstance(x, dict) and key in x]+ return json.dumps(k) if k else dump1(x)+++def sep(t, path, sp, kids):+ """the separator used between the container's items (',' + its line break and indent), from its last item"""+ ks = kids[path]+ if len(ks) < 2:+ return ', '+ a, b = sp[path + (ks[-2],)][2], sp[path + (ks[-1],)][0]+ return t[a:b] if t[a:b].strip() == ',' else ', '+++def insert(t, path, sp, kids, items, out, after=None):+ ks = kids[path]+ if not ks:+ _, s0, e0 = sp[path]+ out.append((s0 + 1, s0 + 1, ', '.join(items)))+ return+ s = sep(t, path, sp, kids)+ if after is None:+ after = len(ks) - 1+ if after < 0:+ out.append((sp[path + (ks[0],)][0],) * 2 + (s.join(items) + s,))+ else:+ e = sp[path + (ks[after],)][2]+ out.append((e, e, s + s.join(items)))+++def patch_text(text, new):+ """text of a JSON file with only the values that differ from `new` rewritten; None when that is not possible"""+ try:+ old = json.loads(text)+ sp, kids, out = {}, {}, []+ scan(text, 0, (), sp, kids)+ edits_for(text, old, new, (), sp, kids, out)+ out.sort()+ if any(out[n][1] > out[n + 1][0] for n in range(len(out) - 1)):+ return None+ for s0, e0, rep in reversed(out):+ text = text[:s0] + rep + text[e0:]+ return text if json.loads(text) == new else None+ except (ValueError, KeyError, IndexError):+ return None+++def write_json(path, new, base_text=None):+ """write `new` to path keeping base_text's formatting where possible; returns 'patched' or 'reformatted'"""+ txt = patch_text(base_text, new) if base_text is not None else None+ how = 'patched' if txt is not None else 'reformatted'+ if txt is None:+ txt = json.dumps(new, indent=1, ensure_ascii=False) + '\n'+ with open(path, 'w', encoding='utf-8', newline='') as fh:+ fh.write(txt)+ return how+++def find(items, key, name):+ for i, it in enumerate(items):+ if str(it.get(key)) == str(name):+ return i+ raise KeyError(f'no item with {key} = {name!r}')+++def change(what, old, new):+ if old == new and type(old) is type(new):+ return f'{what}: unchanged ({json.dumps(old)[:80]})'+ return f'{what}: {json.dumps(old)[:80]} -> {json.dumps(new)[:80]}'+++def apply_ops(top, frags, ops):+ """apply operations to in-memory copies; returns a list of change lines"""+ log = []+ for op in ops:+ if 'sd' in op:+ node, keys = top, op['sd'].split('.')+ for k in keys[:-1]:+ node = node.setdefault(k, {})+ log.append(change(f"sounddesign.{op['sd']}", node.get(keys[-1]), op['value']))+ node[keys[-1]] = op['value']+ continue+ f = op['f']+ if f not in frags:+ raise KeyError(f'no fragment "{f}" (have {", ".join(sorted(frags))})')+ d = frags[f]+ if 'sample' in op or 'slice' in op:+ kind = 'samples' if 'sample' in op else 'slices'+ name = op.get('sample') or op.get('slice')+ s = d.get(kind, {}).get(name)+ if s is None:+ raise KeyError(f'{f}: no {kind[:-1]} "{name}"')+ for k, v in (op.get('set') or {}).items():+ log.append(change(f'{f}.{name}.{k}', s.get(k), v))+ s[k] = v+ if 'eq' in op:+ old = copy.deepcopy(s.get('eq'))+ s.setdefault('eq', {}).update(op['eq'])+ log.append(change(f'{f}.{name}.eq', old, s['eq']))+ elif 'rule' in op or 'place' in op or 'amb' in op:+ kind, key, name = (('rules', 'name', op['rule']) if 'rule' in op else ('place', 'sample', op['place'])+ if 'place' in op else ('ambience', 'name', op['amb']))+ items = d.get(kind, [])+ i = find(items, key, name)+ if op.get('delete'):+ log.append(f'{f}.{kind}[{name}]: deleted')+ items.pop(i)+ continue+ for k, v in (op.get('set') or {}).items():+ log.append(change(f'{f}.{kind}[{name}].{k}', items[i].get(k), v))+ items[i][k] = v+ elif 'rule_add' in op or 'place_add' in op:+ kind = 'rules' if 'rule_add' in op else 'place'+ d.setdefault(kind, []).append(op.get('rule_add') or op.get('place_add'))+ log.append(f'{f}.{kind}: added {json.dumps(d[kind][-1])[:100]}')+ elif 'segment' in op:+ sg = d['music']['segments'][int(op['segment'])]+ for k, v in (op.get('set') or {}).items():+ log.append(change(f"{f}.music.segments[{op['segment']}].{k}", sg.get(k), v))+ sg[k] = v+ elif 'pool' in op:+ log.append(change(f"{f}.pools.{op['pool']}", d.get('pools', {}).get(op['pool']), op['value']))+ d.setdefault('pools', {})[op['pool']] = op['value']+ else:+ raise ValueError(f'unknown operation {op}')+ return log+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ ap.add_argument('--root', default='.')+ ap.add_argument('--design', default='audio/sounddesign.json')+ ap.add_argument('--ops', required=True, help='JSON list of operations (or [] for an unchanged copy)')+ ap.add_argument('--out', help='variant folder (critics)')+ ap.add_argument('--apply', action='store_true', help='edit the real design files (the fixer)')+ ap.add_argument('--backup', default='out/audio-review/backup_before_fix', help='with --apply: copy the originals here first')+ ap.add_argument('--render', action='store_true', help='mix the variant (outputs inside --out)')+ ap.add_argument('--qa', action='store_true', help='also run qa.py on the rendered variant')+ ap.add_argument('--cache', help='decode cache for --render / --qa (default: <root>/out/audio/cache, as mix.py)')+ ap.add_argument('--cues', help='cue file for the variant (written into its design; default: the design\'s "cues", '+ 'else out/cues.json)')+ a = ap.parse_args(argv)+ root = os.path.abspath(a.root)+ R = lambda p: os.path.normpath(p if os.path.isabs(p) else os.path.join(root, p))+ if bool(a.out) == bool(a.apply):+ sys.exit('give exactly one of --out (a variant) or --apply (edit the design)')+ if not os.path.exists(R(a.design)):+ sys.exit(f'design file not found: {R(a.design)} (pass --design)')+ text = {p: open(R(p), encoding='utf-8', newline='').read() for p in [a.design] + json.load(+ open(R(a.design), encoding='utf-8')).get('include', [])}+ top = json.loads(text[a.design])+ paths = {os.path.splitext(os.path.basename(p))[0]: p for p in top.get('include', [])}+ frags = {k: json.loads(text[p]) for k, p in paths.items()}+ ops = json.load(open(R(a.ops), encoding='utf-8')) if os.path.exists(R(a.ops)) else json.loads(a.ops)+ new_top, new_frags = copy.deepcopy(top), copy.deepcopy(frags)+ try:+ log = apply_ops(new_top, new_frags, ops)+ except (KeyError, ValueError, IndexError) as e:+ sys.exit(f'nothing written: {e}')+ if a.apply:+ bk = R(a.backup)+ if os.path.exists(bk):+ sys.exit(f'backup {bk} exists: pass a new --backup')+ os.makedirs(os.path.join(bk, 'design'))+ shutil.copy(R(a.design), os.path.join(bk, os.path.basename(a.design)))+ for k, p in paths.items():+ shutil.copy(R(p), os.path.join(bk, 'design', os.path.basename(p)))+ todo = [(a.design, new_top, top)] + [(p, new_frags[k], frags[k]) for k, p in paths.items()]+ written = [(p, write_json(R(p), new, text[p])) for p, new, old in todo if new != old]+ dest = f'the design (originals in {bk})'+ else:+ out = R(a.out)+ os.makedirs(os.path.join(out, 'design'), exist_ok=True)+ new_top['include'] = [os.path.join(out, 'design', os.path.basename(p)).replace('\\', '/') for p in top['include']]+ if a.cues or top.get('cues'):+ new_top['cues'] = R(a.cues or top['cues']).replace('\\', '/')+ dpath = os.path.join(out, 'sounddesign.json').replace('\\', '/')+ written = [(os.path.join(out, 'design', os.path.basename(p)), write_json(+ os.path.join(out, 'design', os.path.basename(p)), new_frags[k], text[p])) for k, p in paths.items()]+ written.append((dpath, write_json(dpath, new_top, text[a.design])))+ dest = dpath+ n = sum(1 for line in log if ': unchanged (' not in line)+ print(f'{n} change(s), {len(log) - n} already as asked -> {dest}' if n < len(log) else f'{n} change(s) -> {dest}')+ for line in log:+ print(' ' + line.encode('ascii', 'replace').decode())+ for p, how in written:+ print(f' wrote {p} ({how}' + (', only the changed values rewritten)' if how == 'patched' else+ ': could not patch in place, rewritten with indent 1)'))+ if a.out and (a.render or a.qa):+ o = lambda n: os.path.join(R(a.out), n)+ cmd = [sys.executable, '-I', os.path.join(HERE, 'mix.py'), '--root', root, '--design', dest, '--out', o('soundtrack.wav'),+ '--stems', o('stems'), '--report', o('report.json'), '--cues-out', o('cues.json'), '--trace', o('adds.json')]+ cache = ['--cache', R(a.cache)] if a.cache else []+ r = subprocess.run(cmd + cache, capture_output=True, text=True)+ print(r.stdout[-1200:], r.stderr[-800:])+ if r.returncode:+ return r.returncode+ if a.qa:+ r = subprocess.run([sys.executable, '-I', os.path.join(HERE, 'qa.py'), '--root', root, '--design', dest,+ '--report', o('report.json'), '--trace', o('adds.json'), '--stems', o('stems'),+ '--master', o('soundtrack.wav'), '--out', o('qa.json')] + cache, capture_output=True, text=True)+ print(r.stdout, r.stderr[-800:])+ return 0+++if __name__ == '__main__':+ sys.exit(main())diff --git a/skills/real-sound/templates/SOUND_BRIEF.md b/skills/real-sound/templates/SOUND_BRIEF.mdnew file mode 100644index 0000000..683533d--- /dev/null+++ b/skills/real-sound/templates/SOUND_BRIEF.md@@ -0,0 +1,79 @@+# SOUND BRIEF: {{FILM}} soundtrack from professional recordings++<!-- Copy to audio/SOUND_BRIEF.md and fill every {{FIELD}}. Every sub-agent (sourcing, music, design, critics, fixer)+receives this file verbatim: it is the contract. Delete the example rows. -->++The picture is final: `{{VIDEO}}`, {{DURATION}} s, {{FPS}} fps, cut on a {{BPM}} BPM grid (1 beat = {{BEAT}} s,+1 bar = {{BAR}} s). Cue events: `{{CUES}}` (computed from the picture code; never re-time them by hand).++The soundtrack uses ONLY real, professionally recorded or professionally produced audio from licensed libraries:+- one licensed music track, edited to the picture at bar lines;+- real foley and SFX recordings placed on the picture's cues;+- real field-recorded ambiences where the picture has a place.++Code may only select, trim, place, level, pan, EQ, crossfade, time-stretch the music (small ratios), pitch by+resampling and master these recordings. Code never generates a tone, a noise or a "sound" (no oscillators, no+synthesized noise, no synthesized reverb tails). The bar: {{BAR_REFERENCE}} (e.g. "a premium product launch film:+crisp, tactile, confident, every sound caused by something on screen"), not a game jam.++## Sound concept+<!-- 4-8 bullets: the palette (materials), the music's character and arc, the places. Example for a launch film: -->+- **One tactile UI palette.** {{PRODUCT}} is precise and calm: every interface sound comes from one family of real+ close-miked sources (a laptop trackpad click, soft felt-tip taps, one mechanical keyboard), dry and centred.+- **One music track carries the film.** {{MOOD}}: a soft intro under the cold open (no drums), a groove for the feature+ beats, a breakdown at {{BREAKDOWN_T}} s before the turn, the DROP on the hero reveal at {{DROP_T}} s, a resolved ending+ under the logo ({{LOGO_T}} s), tail to {{DURATION}} s. Instrumental only, {{TEMPO_RANGE}} BPM.+- **Restraint.** About 5 designed effects per 10 s, clustered in the cold open and the logo build; the music carries+ the cuts.+- **Places.** {{ZONES_SUMMARY}} (e.g. "a quiet office room tone under the UI part only; no ambience in the abstract+ sections").++## Picture map (film seconds; cue ids from `{{CUES}}`)+| t (s) | what happens on screen | cue (kind / id) | sound need |+|---|---|---|---|+| 0.0 | the cursor glides onto the empty search field | (none) | nothing: the music enters soft |+| 1.0 | click into the field | click / demo:cursor:click | the film's one click voice (press + release) |+| 2.0-3.5 | the query types itself (12 characters) | key / demo:type:key:n, demo:type:space | real keyboard, one key per sounded character, space deeper; music dips 6 dB |+| 4.0 | whip pan into the results | whoosh / demo:transition | one real swish, apex on the velocity peak |+| 5.5-6.0 | three result cards pop in | pop / demo:card:n | soft real pops climbing 2 st across the three |+| 7.0-8.0 | everything pulls back; the product mark flies in | riser / demo:build | the music's build (or a real cymbal roll ending 30 ms before 8.0) |+| **8.0** | **hero reveal: the whole product UI lands** | hit / demo:reveal | **the music's DROP on the frame**; a low real impact under it |+| 10.0 | the wordmark settles | bell / demo:lockup | music stops on the bar line; one soft celesta note in key; tail to the end |++Ambience zones: {{ZONES}} (e.g. "office room tone 0-8.2 s; none after the reveal").++## Licences: what is allowed+Allowed (record each in the manifest, read on the item's own page): CC0 1.0 and public domain; CC BY 3.0 / 4.0+(credits in CREDITS.md and the video description, never required on screen); Mixkit Sound Effects / Stock Music Free+License (read the current text at https://mixkit.co/license/); Pixabay Content License (manual download only).+NOT allowed: NonCommercial (BY-NC), NoDerivatives (BY-ND: syncing to picture IS a derivative), ShareAlike (BY-SA),+Sampling+, "personal use only", custom or unclear licences, disputed licences, YouTube rips, anything behind a login+or paywall you would have to bypass, AI-generated or code-synthesized audio. Never fabricate a licence.+Details: `<skill>/references/licences.md`.++## Quality bar+- SFX / foley: real recordings, close and clean. Prefer lossless WAV/FLAC (Mixkit WAV+ `assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`) or Freesound's HQ preview+ (`cdn.freesound.org/previews/<id//1000>/<id>_<userid>-hq.ogg`). No clipping, floor under -55 dBFS between events,+ 95 % rolloff above 12 kHz, no untreated room on foley, no voices / handling noise / background events.+- earshot: `realness` verdict "real or pro-produced" for every file used; `match` z >= 2.5 for the need it fills.+- Ambiences: clean (no people / traffic / planes / dogs unless the story wants them), no loud single events, >= 40 s+ usable, real stereo.+- Music: professionally produced and mastered, full bandwidth, instrumental, steady tempo, 4/4, {{TEMPO_RANGE}} BPM+ (stretched to {{BPM}} only if needed). Avoid: vocals, corporate ukulele + whistling, EDM brostep, lo-fi hiss,+ bombastic trailer music, moods against the brand, famous overused stock tracks.+- Mix: -14 LUFS integrated, true peak <= -1 dBTP after AAC, LRA 5-9 LU, the loudest moment on the drop, the logo+ 8-20 dB under it, a tail into silence.++## Files, manifests, tools+- Downloads: `audio/assets/<category>/` (audio files only; never a zip with programs; never run anything downloaded).+ Name: `<source>-<id>_<short-slug>.<ext>`, e.g. `fs-450282_mech-typing.ogg`, `mk-1386_keyboard-typing.wav`.+- Manifests: `audio/assets/manifests/<category>.json`, a list of entries shaped like+ `<skill>/templates/manifest_entry.json` (id, category, file, title, author, source, page_url, download_url, license,+ license_url, attribution, need, rank, best_segment, notes, analysis).+- Audit: `audio/assets/manifests/audit.json` (agent fields per `<skill>/templates/audit_entry.json`) + `audit.py`.+- Design: `audio/sounddesign.json` (lead) + `audio/design/<category>.json` (one designer each).+- Review output: `out/audio-review/<category or critic>/`. QA: `out/qa/`.+- Analyse: `python -I <skill>/scripts/analyze.py <files> [--music] [--png out/audio-review/<category>] [--out ....json]`;+ listen: `python -m earshot realness|describe|match|rank|find ...`.+- Be polite to the sites: sequential downloads, a pause between requests, at most ~40 files per category.diff --git a/skills/real-sound/templates/arc.json b/skills/real-sound/templates/arc.jsonnew file mode 100644index 0000000..7cd08a5--- /dev/null+++ b/skills/real-sound/templates/arc.json@@ -0,0 +1,7 @@+{+ "_doc": "Input for arrange.py --arc: the target loudness of each FILM bar in dB relative to the track's loud bars (0 = its biggest sections, -6 = a groove, -10 = a breakdown, -14 = a soft intro). One number per film bar (bar = beats_per_bar beats at film_bpm). Write it from the picture map in SOUND_BRIEF.md: soft under the cold open and typing, the groove for the feature beats, a breakdown before the turn, the build, the big section from the drop (the hero reveal), a resolve under the logo. This example: a 30 s launch film at 120 BPM = 15 bars, drop on film bar 9 (16.0 s), logo from bar 13.",+ "film_bpm": 120,+ "beats_per_bar": 4,+ "drop_film_bar": 9,+ "arc": [-14, -12, -7, -6, -6, -6, -10, -6, 0, 0, 0, -2, -8, -11, -14]+}diff --git a/skills/real-sound/templates/audit_entry.json b/skills/real-sound/templates/audit_entry.jsonnew file mode 100644index 0000000..ea0ad93--- /dev/null+++ b/skills/real-sound/templates/audit_entry.json@@ -0,0 +1,16 @@+{+ "_doc": "One record of audio/assets/manifests/audit.json. The licence auditor (prompts/licence_audit.md) writes ONLY the three agent fields below after reading the licence on the item's own page; audit.py computes everything else on every run and keeps the agent fields (matched by id + file). Never set licence_confirmed without having read the page in this session.",+ "id": "fs-000000",+ "licence_seen": "Freesound page https://freesound.org/people/uploader_name/sounds/000000/: licence 'Attribution 4.0' -> https://creativecommons.org/licenses/by/4.0/ (read 2026-10-14); page author 'uploader_name', title 'Trackpad click close.wav'; preview id/user match download_url",+ "licence_confirmed": true,+ "agent_notes": ["the author asks for credit as \"<title> by uploader_name (freesound.org)\": the manifest attribution follows it"],+ "_computed_by_audit_py": {+ "manifest": "audio/assets/manifests/ui.json", "need": "ui_click", "rank": 1, "in_scope": true, "severity": "ok",+ "licence_in_manifest": "CC BY 4.0", "licence": "by4", "licence_status": "allowed", "attribution_required": true,+ "attribution_ok": true, "attribution_in_manifest": "\"Trackpad click close.wav\" by uploader_name (...), licensed under CC BY 4.0 (...)",+ "file_ok": true,+ "file_check": {"path": "audio/assets/ui/fs-000000_trackpad-click.ogg", "exists": true, "codec": "vorbis", "sample_rate": 48000,+ "channels": 1, "kbps": 192, "duration_s": 0.84, "decodes_clean": true, "bytes_match_download_url": true},+ "problems": [], "warnings": [], "audit_tool": "real-sound audit.py v1"+ }+}diff --git a/skills/real-sound/templates/cues.json b/skills/real-sound/templates/cues.jsonnew file mode 100644index 0000000..0d615d5--- /dev/null+++ b/skills/real-sound/templates/cues.json@@ -0,0 +1,29 @@+{+ "_doc": "The picture's cue file: the contract between picture and sound (Cinetic's scripts/export-cues.ts writes this exact format from the picture code; write it from the timeline for any other engine). f = absolute frame the sound belongs on (fractional is fine); kind = what happens (tick, tock, key, click, pop, snap, land, hit, drop, whoosh, riser, swell, suck, bell, or your own word); id = a stable label rules match with a regex ('act:thing:n'); weight 0..2 (1 = normal), pan -1..1 (the object's screen x), variant (key: word/space/return/back; land: light; click: confirm), dur in frames for whoosh / riser / swell. Optional: acts {name: {from, dur}} and cue {name: frame} (qa.py reports loudness per act). mix.py also accepts a list whose first element is this object. Sync points belong where the eye reads them: an impact on its contact frame, a pop-in at 50 % of its travel, a whoosh on the move's velocity peak, a riser on the frame of the hit it leads into, a logo sound on the frame the wordmark becomes readable.",+ "fps": 60,+ "bpm": 120,+ "total": 720,+ "acts": {"ui": {"from": 0, "dur": 240}, "cards": {"from": 240, "dur": 240}, "reveal": {"from": 480, "dur": 240}},+ "cue": {"click": 60, "typeStart": 120, "transition": 240, "reveal": 480, "lockup": 600},+ "events": [+ {"f": 60, "kind": "click", "weight": 1, "pan": 0.2, "id": "demo:cursor:click"},+ {"f": 120, "kind": "key", "weight": 0.6, "id": "demo:type:key:0", "variant": "word"},+ {"f": 125, "kind": "key", "weight": 0.5, "id": "demo:type:key:1"},+ {"f": 130, "kind": "key", "weight": 0.5, "id": "demo:type:key:2"},+ {"f": 135, "kind": "key", "weight": 0.5, "id": "demo:type:key:3"},+ {"f": 141, "kind": "key", "weight": 0.5, "id": "demo:type:space", "variant": "space"},+ {"f": 146, "kind": "key", "weight": 0.6, "id": "demo:type:key:4", "variant": "word"},+ {"f": 151, "kind": "key", "weight": 0.5, "id": "demo:type:key:5"},+ {"f": 156, "kind": "key", "weight": 0.5, "id": "demo:type:key:6"},+ {"f": 162, "kind": "key", "weight": 0.5, "id": "demo:type:key:7"},+ {"f": 168, "kind": "key", "weight": 0.5, "id": "demo:type:key:8"},+ {"f": 210, "kind": "click", "weight": 1, "pan": 0.35, "id": "demo:cursor:submit", "variant": "confirm"},+ {"f": 240, "kind": "whoosh", "weight": 1, "pan": -0.6, "dur": 24, "apexFrac": 0.5, "id": "demo:transition"},+ {"f": 330, "kind": "pop", "weight": 0.8, "pan": -0.4, "id": "demo:card:0"},+ {"f": 345, "kind": "pop", "weight": 0.8, "pan": 0.0, "id": "demo:card:1"},+ {"f": 360, "kind": "pop", "weight": 0.8, "pan": 0.4, "id": "demo:card:2"},+ {"f": 480, "kind": "riser", "weight": 1, "dur": 120, "id": "demo:build"},+ {"f": 480, "kind": "hit", "weight": 1.5, "id": "demo:reveal"},+ {"f": 600, "kind": "bell", "weight": 0.7, "id": "demo:lockup"}+ ]+}diff --git a/skills/real-sound/templates/design_fragment.json b/skills/real-sound/templates/design_fragment.jsonnew file mode 100644index 0000000..f36a28c--- /dev/null+++ b/skills/real-sound/templates/design_fragment.json@@ -0,0 +1,81 @@+{+ "_doc": "A complete design showing every feature mix.py understands, for a 12 s launch-video beat (cursor click, typing, whoosh into a transition, cards popping in, riser -> hero reveal hit, logo sting). It works as a single-file design (mix.py --design this.json) or as one category fragment listed in sounddesign.json 'include' (then 'duration', 'buses' and 'master' are read only from the top-level file). Keys starting with _ are comments. Every 'file' is a real licensed recording listed in a manifest (audio/assets/manifests/*.json). Prefix every name with the fragment's category so fragments never collide. Reference: references/design-format.md.",+ "duration": 12.0,+ "cues": "out/cues.json",++ "samples": {+ "ui.press": {"_doc": "the press half of a real mouse click: trim 3-5 ms before the attack, end before the release", "file": "audio/assets/ui/mk-0000_mouse-click.wav", "start": 0.096, "end": 0.150, "mono": true, "eq": {"hp": 400}, "norm_peak_db": -12, "fade_in": 0.002, "fade_out": 0.008},+ "ui.release": {"file": "audio/assets/ui/mk-0000_mouse-click.wav", "start": 0.180, "end": 0.230, "mono": true, "eq": {"hp": 400}, "norm_peak_db": -15, "fade_in": 0.002, "fade_out": 0.008},+ "kb.space": {"file": "audio/assets/keyboard/fs-000001_mech-spacebar.ogg", "start": 0.48, "end": 0.78, "mono": true, "eq": {"hp": 90, "highshelf": [5000, -3]}, "norm_peak_db": -10, "fade_out": 0.08},+ "mo.whoosh": {"_doc": "a real swish (stick / cloth / bamboo past the mic); placed by its peak on the move's velocity peak", "file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.0, "end": 0.55, "eq": {"hp": 150, "lp": 12000}, "norm_peak_db": -6, "fade_in": 0.01, "fade_out": 0.12},+ "mo.whoosh.tail": {"file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.05, "end": 0.55, "pitch": -3, "eq": {"hp": 120, "lp": 6000}, "norm_peak_db": -8, "fade_in": 0.02, "fade_out": 0.2},+ "fx.pop": {"file": "audio/assets/ui/fs-000003_cork-pop.ogg", "start": 0.02, "end": 0.20, "mono": true, "eq": {"hp": 250, "peq": [[3200, 1.2, -2]]}, "norm_peak_db": -10, "fade_out": 0.04},+ "fx.riser": {"_doc": "a real cymbal roll / orchestral swell; aligned by its END so it stops on the reveal", "file": "audio/assets/motion/fs-000004_cymbal-roll.ogg", "start": 0.3, "end": 2.3, "eq": {"hp": 200}, "norm_peak_db": -9, "fade_in": 0.6, "fade_out": 0.03, "env": [[0, -12], [1.5, -2], [2.0, 0]]},+ "fx.hit": {"file": "audio/assets/impacts/mk-0005_deep-drum-impact.wav", "start": 0.0, "end": 1.6, "eq": {"hp": 30, "peq": [[300, 1.0, -3]]}, "norm_peak_db": -4, "fade_out": 0.4},+ "fx.hit.top": {"file": "audio/assets/impacts/fs-000006_wood-slam.ogg", "start": 0.01, "end": 0.35, "mono": true, "eq": {"hp": 500}, "norm_peak_db": -10, "fade_out": 0.08},+ "logo.chime": {"_doc": "the sting: one soft real bell / celesta note, in the music's key (resample with pitch to tune, within +-2 st)", "file": "audio/assets/magic/fs-000007_celesta-E5.ogg", "start": 0.0, "end": 2.4, "pitch": 0.0, "norm_peak_db": -9, "fade_out": 0.9},+ "logo.air": {"file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.0, "end": 0.5, "glide": [2, -1], "eq": {"hp": 300, "lp": 9000}, "norm_peak_db": -14, "fade_in": 0.05, "fade_out": 0.25}+ },++ "slices": {+ "kb.type": {"_doc": "one recording of a mechanical keyboard typing, cut at its onsets into single keys: kb.type#0, kb.type#1 ... (check them with --dump)", "file": "audio/assets/keyboard/fs-000008_mech-typing.ogg", "start": 0.5, "end": 9.0, "pre": 0.004, "len": 0.14, "min_gap": 0.07, "gate_db": -15, "mono": true, "eq": {"hp": 110, "highshelf": [5000, -4]}, "norm_peak_db": -12, "fade_out": 0.03, "max": 16}+ },++ "pools": {+ "ui.press": ["ui.press"],+ "ui.release": ["ui.release"],+ "kb.type": ["kb.type#*"],+ "kb.space": ["kb.space"],+ "mo.whoosh": ["mo.whoosh"],+ "mo.whoosh.tail": ["mo.whoosh.tail"],+ "fx.pop": ["fx.pop"],+ "fx.riser": ["fx.riser"],+ "fx.hit": ["fx.hit"],+ "fx.hit.top": ["fx.hit.top"],+ "logo.chime": ["logo.chime"]+ },++ "rules": [+ {"name": "ui.click press", "match": {"kind": "click"}, "pool": "ui.press", "mode": "fixed", "gain_db": 0, "pan": 0, "align": "attack", "_doc": "one voice at one level for every click in the film"},+ {"name": "ui.click release", "match": {"kind": "click"}, "pool": "ui.release", "mode": "fixed", "gain_db": -2, "pan": 0, "align": "attack", "offset": 0.085},+ {"name": "kb.type", "match": {"kind": "key", "id": "^demo:type:key"}, "pool": "kb.type", "mode": "cycle", "gain_db": -2, "jitter_db": 1.5, "pitch": [-0.3, 0.3], "pan": 0, "align": "attack"},+ {"name": "kb.space", "match": {"kind": "key", "id": "^demo:type:space"}, "pool": "kb.space", "mode": "fixed", "gain_db": -1, "pan": 0, "align": "attack"},+ {"name": "mo.whoosh", "match": {"kind": "whoosh"}, "pool": "mo.whoosh", "mode": "fixed", "align": "peak", "gain_db": -6, "pan": "cue", "pan_scale": 0.8},+ {"name": "mo.whoosh tail", "match": {"kind": "whoosh"}, "pool": "mo.whoosh.tail", "mode": "fixed", "align": "peak", "offset": 0.06, "gain_db": -12, "pan": "cue", "pan_scale": 0.5},+ {"name": "fx.pops", "match": {"kind": "pop", "from": 5.0, "to": 7.0}, "pool": "fx.pop", "mode": "cycle", "gain_db": -4, "jitter_db": 1.0, "pitch": [-0.2, 0.2], "pitch_ramp": [0, 2], "gain_ramp": [-1, 1], "pan": "cue", "pan_scale": 0.3, "align": "attack"},+ {"name": "fx.riser", "match": {"kind": "riser"}, "pool": "fx.riser", "mode": "fixed", "align": "end", "offset": -0.03, "gain_db": -6, "pan": 0, "_doc": "ends 30 ms before the reveal: a beat of air before the hit"},+ {"name": "fx.hit", "match": {"kind": "hit", "id": "^demo:reveal$"}, "pool": "fx.hit", "mode": "fixed", "gain_db": -2, "pan": 0, "align": "attack"},+ {"name": "fx.hit top", "match": {"kind": "hit", "id": "^demo:reveal$"}, "pool": "fx.hit.top", "mode": "fixed", "gain_db": -8, "pan": 0, "align": "attack", "offset": 0.004},+ {"name": "logo.chime", "match": {"kind": "bell", "id": "^demo:lockup$"}, "pool": "logo.chime", "mode": "fixed", "gain_db": -3, "pan": 0, "align": "attack", "_doc": "on the frame the wordmark becomes readable (its settle), never later"}+ ],++ "place": [+ {"sample": "logo.air", "at": 9.98, "align": "peak", "gain_db": -6, "pan": 0.0, "_doc": "a free placement (no cue event): a soft air lift that peaks just before the chime"}+ ],++ "ambience": [+ {"name": "am.room", "_doc": "a quiet real room tone under the UI part; src_start..src_end is a clean window (no events), looped with a 1.5 s crossfade if needed", "file": "audio/assets/ambience/fs-000009_quiet-office-roomtone.ogg", "src_start": 5.0, "src_end": 45.0, "at": 0.0, "until": 8.2, "fade_in": 0.3, "fade_out": 0.4, "gain_db": -6, "width": 0.8, "eq": {"hp": 80, "lp": 12000}, "auto": [[0, 0], [7.5, 0], [8.0, -8]]}+ ],++ "music": {+ "_doc": "one licensed track, edited at bar lines (arrange.py drafts the segments). src_* are ORIGINAL source seconds; 'at' is film time. 'at' can be nudged a few ms so the measured downbeat transient lands on the picture's downbeat.",+ "file": "audio/assets/music/sb-0000_bright-launch-track.mp3",+ "src_bpm": 120,+ "target_bpm": 120,+ "stretch": "none",+ "gain_db": -4,+ "eq": {"hp": 30},+ "segments": [+ {"label": "intro bars 1-4 (soft, no drums) under the UI", "src_from": 0.5, "src_to": 8.5, "at": 0.0, "fade_in": 0.15, "fade_out": 0.03},+ {"label": "drop bars 17-18 on the hero reveal, ring out under the logo", "src_from": 32.5, "src_to": 36.5, "at": 8.0, "fade_in": 0.008, "fade_out": 0.06, "tail_fade": 1.2}+ ]+ },++ "_buses_doc": "buses and master are read from the top-level design only. auto = [[film t, dB], ...] piecewise-linear fader rides: here the music dips 6 dB under the typing and comes back for the whoosh",+ "buses": {+ "music": {"gain_db": 0.0, "auto": [[0, 0], [1.9, 0], [2.0, -6], [3.6, -6], [3.8, 0], [12, 0]]},+ "amb": {"gain_db": 0.0},+ "sfx": {"gain_db": 0.0, "eq": {"hp": 40}}+ },+ "master": {"_doc": "read from the top-level design only. -14 LUFS for web/social; ceiling 0.77 linear (-2.3 dBFS): measure about -1 dBTP after AAC", "lufs": -14, "ceiling": 0.77, "no_glue": true}+}diff --git a/skills/real-sound/templates/manifest_entry.json b/skills/real-sound/templates/manifest_entry.jsonnew file mode 100644index 0000000..f161937--- /dev/null+++ b/skills/real-sound/templates/manifest_entry.json@@ -0,0 +1,42 @@+[+ {+ "_doc": "One object per downloaded recording, in audio/assets/manifests/<category>.json (a JSON list). Values here are placeholders: copy the shape, never the data. Every field is read from the item's own page; never guess a licence. 'attribution' is required for CC BY (paste the form the author asks for on their page, else: \"<title>\" by <author> (<page_url>), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)); null for CC0 / Mixkit / Pixabay. 'analysis' is what analyze.py measured; 'best_segment' the clean part in seconds.",+ "id": "fs-000000",+ "category": "ui",+ "file": "audio/assets/ui/fs-000000_trackpad-click.ogg",+ "title": "Trackpad click close.wav",+ "author": "uploader_name",+ "source": "freesound",+ "page_url": "https://freesound.org/people/uploader_name/sounds/000000/",+ "download_url": "https://cdn.freesound.org/previews/0/000000_0000000-hq.ogg",+ "license": "CC BY 4.0",+ "license_url": "https://creativecommons.org/licenses/by/4.0/",+ "attribution": "\"Trackpad click close.wav\" by uploader_name (https://freesound.org/people/uploader_name/sounds/000000/), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",+ "need": "ui_click",+ "rank": 1,+ "best_segment": [0.10, 0.32],+ "notes": "What it is and how it was recorded (from the page), what you measured, what to use and avoid: e.g. 'close-miked laptop trackpad, press at 0.12 + release at 0.21 (85 ms later, 3 dB quieter), floor -78 dBFS, no room; use as the film's one click voice'.",+ "analysis": {"duration": 0.84, "codec": "vorbis", "kbps": 192, "src_rate": 48000, "channels": 1, "peak_dbfs": -9.6,+ "lufs": -31.2, "noise_floor_dbfs": -78.0, "clipped_samples": 0, "attack_at": 0.118, "centroid_hz": 3300,+ "rolloff95_hz": 11800, "onsets": [0.118, 0.205]},+ "earshot": {"realness": 0.97, "match": {"query": "a single crisp trackpad click", "z": 3.4, "verdict": "strong"}}+ },+ {+ "id": "mk-0000",+ "category": "impacts",+ "file": "audio/assets/impacts/mk-0000_deep-drum-impact.wav",+ "title": "Deep drum impact",+ "author": "Mixkit",+ "source": "mixkit",+ "page_url": "https://mixkit.co/free-sound-effects/impact/",+ "download_url": "https://assets.mixkit.co/active_storage/sfx/0000/0000.wav",+ "license": "Mixkit Sound Effects Free License",+ "license_url": "https://mixkit.co/license/#sfxFree",+ "attribution": null,+ "need": "hero_reveal_hit",+ "rank": 2,+ "best_segment": [0.0, 1.6],+ "notes": "Mixkit has no per-item page for sound effects: record the category page that lists the item and the item's title there.",+ "analysis": {"duration": 2.1, "codec": "pcm_s16le", "src_rate": 44100, "channels": 2, "peak_dbfs": -1.2}+ }+]diff --git a/skills/real-sound/templates/prompts/critic_director.md b/skills/real-sound/templates/prompts/critic_director.mdnew file mode 100644index 0000000..7d89f3b--- /dev/null+++ b/skills/real-sound/templates/prompts/critic_director.md@@ -0,0 +1,36 @@+# Critic lens: director (does the soundtrack tell the film's story?)++You are one of five read-only critics of {{FILM}}'s soundtrack, and the one who judges it as a whole. Your lens:+**the arc and the story**. Would a viewer feel the hook, the turn, the reveal and the resolution with their eyes+closed? Is the loudest moment the hero reveal, is the logo resolved rather than loud, does the ending subtract into+silence? Is anything cheap, generic, empty or "random"?++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (concept, picture map,+ reference bar: {{BAR_REFERENCE}}). Render `{{RENDER}}`, film `{{VIDEO}}`, qa `{{QA_JSON}}`. Round {{ROUND}};+ changed: {{CHANGED}}; fixed: {{FIXED}}.+- You write ONLY in `out/audio-review/critic_director/`.++## Measure+1. `{{PY}} -m earshot timeline {{VIDEO}} --cues {{CUES}}` (the listening transcript: events, sections, loudness, sync)+ and `{{PY}} -m earshot sheet {{RENDER}}/soundtrack.wav -o out/audio-review/critic_director/sheet.png --video {{VIDEO}} --cues {{CUES}}`+ (an annotated picture: frames, sections, tagged events, spectrogram, loudness, cue sync; open it).+2. `{{PY}} -m earshot critique {{RENDER}}/soundtrack.wav --video {{VIDEO}}`: the scorecard against the profile of+ professional launch films (each metric, its range, what to do).+3. qa.py `loudness`: integrated, LRA, loudest momentary and its position, per-act medians, head and tail; the+ short-term curve against the picture map's beats (write a small plot in your folder if it helps: you can read PNGs).+4. Walk the picture map beat by beat: for each, what the transcript hears there, the sfx and music levels, whether the+ sound makes the picture feel better or is just there.+5. Name the weakest 3 seconds and why.++## Test your fixes+`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_director/ops_v1.json --out out/audio-review/critic_director/v1 --render --qa`+(bus rides, the music edit, which events are sounded, the ending). Report the arc numbers before -> after.++## Rules+- At most 12 issues, ranked by impact. Taste is allowed, but every claim needs a measurement behind it.+- Do not re-report fixed issues; do not propose synthesized sounds; do not propose on-screen text.++## Return (JSON only)+`{"lens": "director", "weakest_3s": {"from", "to", "why"}, "issues": [{"id", "priority", "t", "problem",+"evidence", "ops": [...], "verified"}], "overall": "...", "score": 1-10}`diff --git a/skills/real-sound/templates/prompts/critic_edges.md b/skills/real-sound/templates/prompts/critic_edges.mdnew file mode 100644index 0000000..fa31225--- /dev/null+++ b/skills/real-sound/templates/prompts/critic_edges.md@@ -0,0 +1,33 @@+# Critic lens: edges (clicks, cuts, truncations, splices)++You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **anything cut, clicked or chopped**. A+click at a trim, a sample that starts inside a sound, a decay cut short, a music splice that chops a note, a bed that+starts or stops abruptly: these are the defects that make a real-recordings soundtrack sound amateur.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}` (report `{{REPORT}}`, trace+ `{{TRACE}}`, stems, master), qa `{{QA_JSON}}`. Round {{ROUND}}; changed: {{CHANGED}}; already fixed: {{FIXED}}.+- You write ONLY in `out/audio-review/critic_edges/`.++## Measure+1. qa.py `clicks`: every > 14 kHz burst concentrated in 1 ms, in the master and each stem, with the layer that carries+ it and where (start / end = a cut: P0 or P1; body = inside the recording: decide whether it is the sound's own crisp+ transient or a defect in the source).+2. qa.py `edges`: used samples / slices with abrupt entries or chopped decays (defects) and craft notes (attacks+ starting in the first ms, decays faded early, second hits); music splices (chopped notes, holes, overlaps,+ segments starting inside a sustained sound); placements past the film ends.+3. For each doubtful sample: `mix.py --only <fragment> --dump out/audio-review/critic_edges/dump`, read+ `_samples.json` (head / tail dB, attack ms) and the source around the trim (`analyze.py <file> --png ...`).+4. The master's head (first 5 ms) and tail (last 50 ms) in qa.py loudness.++## Test your fixes+`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_edges/ops_v1.json --out out/audio-review/critic_edges/v1 --render --qa`+and report the click / edge counts before and after (a fix must not create a new click elsewhere).++## Typical fixes+`fade_in` 3-5 ms (never more than the attack allows), `start` 3-5 ms before the attack, `end` after the decay or before+the next hit, `fade_out` 20-80 ms, `stop_before_next`, a different take, a splice on another bar, `tail_fade`.++## Return (JSON only)+`{"lens": "edges", "issues": [{"id", "priority", "t", "what", "problem", "evidence", "ops": [...], "verified"}],+"overall": "...", "score": 1-10}`diff --git a/skills/real-sound/templates/prompts/critic_masking.md b/skills/real-sound/templates/prompts/critic_masking.mdnew file mode 100644index 0000000..94aa5e8--- /dev/null+++ b/skills/real-sound/templates/prompts/critic_masking.md@@ -0,0 +1,34 @@+# Critic lens: masking and density (can every sound be heard; is the effect layer dense or repetitive?)++You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **audibility and density**. An effect the+music covers is a wasted placement and a story beat lost; a layer that adds nothing is clutter; one sample fired like+a machine gun is cheap; a wall of effects under the full groove is noise.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}`, qa `{{QA_JSON}}` (run qa.py with+ `--layers` if it was not). Brief: `audio/SOUND_BRIEF.md`. Round {{ROUND}}; changed: {{CHANGED}}; fixed: {{FIXED}}.+- You write ONLY in `out/audio-review/critic_masking/`.++## Measure+1. qa.py `audibility`: per event `read_db` (>= 6 clear, 3-6 ok, 0-3 barely, < 0 masked), the band where it reads+ best, `covered_by` (music/amb or other effects); `dead_layers` (layers of multi-layer events that add nothing).+2. qa.py `density`: events per second (about 5 designed effects per 10 s is the norm for launch films; dense where the+ music is sparse, thin after the drop), the most reused samples, per-rule regularity (inter-onset CV, gain / pitch+ spread, distinct samples), machine-gun rules.+3. Key beats first: the hook (first 2 s), every story beat in the picture map, the reveal, the logo.+4. `{{PY}} -m earshot timeline {{RENDER}}/soundtrack.wav --cues {{CUES}}`: does the transcript hear an event on each+ key beat, with tags that read as the intended sound?++## Test your fixes+`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_masking/ops_v1.json --out out/audio-review/critic_masking/v1 --render --qa`+Report read_db before -> after for every touched event, and check nothing else got masked (a louder effect can hide+its neighbour; a bus duck can drop the music's own downbeat).++## Typical fixes+Rule `gain_db`; a short music duck (`buses.music.auto`) under one key moment; EQ that carves the effect's best band (a+`peq` boost on the effect or a cut on the bed, never both large); a brighter or tighter take; removing dead layers;+more takes in a pool; `jitter_db` / `pitch` for variation; dropping effects the music already carries.++## Return (JSON only)+`{"lens": "masking", "issues": [{"id", "priority", "t", "event", "problem", "evidence", "ops": [...], "verified"}],+"overall": "...", "score": 1-10}`diff --git a/skills/real-sound/templates/prompts/critic_sync.md b/skills/real-sound/templates/prompts/critic_sync.mdnew file mode 100644index 0000000..276e08b--- /dev/null+++ b/skills/real-sound/templates/prompts/critic_sync.md@@ -0,0 +1,40 @@+# Critic lens: sync (does every sound land on its cause?)++You are one of five read-only critics of {{FILM}}'s soundtrack. You cannot hear; you measure. Your lens: **timing**.+Every sound must sit on the frame where the eye reads its cause: impacts on contact, pop-ins at 50 % of their travel,+whooshes peaking on the velocity peak, risers ending on (or 30-250 ms before) their hit, the logo sound on the frame the+wordmark becomes readable.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (picture map).+- Render under review: `{{RENDER}}` (soundtrack, stems, report `{{REPORT}}`, trace `{{TRACE}}`), film `{{VIDEO}}`,+ cues `{{CUES}}`, qa `{{QA_JSON}}`. Round {{ROUND}}. Changed since last round (check hardest): {{CHANGED}}.+ Already fixed, do not re-report: {{FIXED}}.+- You write ONLY in `out/audio-review/critic_sync/`. Never edit the design.++## Measure+1. `qa.py` sync section (`{{QA_JSON}}`): per event attack / peak / apex / end vs its cue; deliberate offsets listed apart.+2. The picture: `{{PY}} -m earshot timeline {{VIDEO}} --cues {{CUES}}` (cut / cue sync, events with offsets) and frames+ around doubtful events (`ffmpeg -ss <t> -i {{VIDEO}} -frames:v 1 out/audio-review/critic_sync/f_<t>.png`, a few frames+ apart; open them) to see where the contact / peak really is. A cue can be wrong too: say so.+3. The sounds: `mix.py --only <fragment> --dump out/audio-review/critic_sync/dump` and the dumped samples' envelopes:+ a slow-rising sample aligned by `attack` peaks late; a whoosh aligned by `attack` instead of `peak` lands early.+4. Rule offsets (`offset` >= 30 ms) are deliberate: check each on the frames anyway.++## Test your fixes+Write the operations to `out/audio-review/critic_sync/ops_v1.json` and render the variant (format in+`templates/prompts/fixer.md`):+```+{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_sync/ops_v1.json --out out/audio-review/critic_sync/v1 --render --qa+```+Report the sync numbers before and after.++## Rules+- Move the picture's cue only when the cue is provably wrong (frames); otherwise move the sound (`align`, `offset`, trim).+- A sound may lead its cause by at most ~45 ms; impacts may land up to ~75 ms late; whoosh apex within +-45 ms.+- Rank by impact per minute of fixing. P0 = audible on a key beat (hook, reveal, logo); P1 = noticeable on a normal+ listen; P2 = a refinement.++## Return (JSON only)+`{"lens": "sync", "issues": [{"id", "priority", "t", "event", "problem", "evidence": "<numbers + command>",+"ops": [<operations>], "verified": "<before -> after numbers from the variant>"}], "overall": "...", "score": 1-10}`diff --git a/skills/real-sound/templates/prompts/critic_tonal.md b/skills/real-sound/templates/prompts/critic_tonal.mdnew file mode 100644index 0000000..56a5dfd--- /dev/null+++ b/skills/real-sound/templates/prompts/critic_tonal.md@@ -0,0 +1,36 @@+# Critic lens: tonal (pleasant on phones and laptops: harshness, brightness, low end, width, key, limiter)++You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **tone and comfort**. Most viewers hear the+film on a phone or a laptop: harsh 2-6 kHz spikes, ice-pick highs, a muddy or wide low end, out-of-key bells and a+limiter squashing the drop are what make a mix tiring or cheap.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}`, qa `{{QA_JSON}}`, the music's key+ {{KEY}}. Round {{ROUND}}; changed: {{CHANGED}}; fixed: {{FIXED}}.+- You write ONLY in `out/audio-review/critic_tonal/`.++## Measure+1. qa.py `harshness`: events whose 2-6 kHz 5 ms peak is more than 6 dB over the music's loudest 2-6 kHz peak; the+ `bright` list (centroid > 6.5 kHz: above launch-film effects; fine if it is the film's palette).+2. Band levels over time (master vs stems): write a short script in your folder (scipy band-pass 2-6k, 6-10k, 10-16k,+ >16 kHz, 20 ms frames) or use `analyze.py --png` spectrograms of the stems; look for effects that poke out of the+ music's spectral envelope.+3. Low end: the impact's partials under 200 Hz vs the music's at the same moment (beating, mud); master.py folds+ below 120 Hz to mono: check nothing important lives in the side signal.+4. Width: side/mid per section (effects should be near mono, only whooshes, beds and music wide).+5. Key: tuned samples (bells, celesta, motif notes) vs {{KEY}}: `{{PY}} -m earshot describe <dumped sample>` gives f0+ / note; `analyze.py --music` gives the track's key.+6. Limiter: `master.json` (gain reduction, loudest momentary); if the drop is squashed, test a lower `master.lufs` or+ a bus ride on the loudest peaks.++## Test your fixes+`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json --out out/audio-review/critic_tonal/v1 --render --qa`+Report the harshness numbers, band levels and audibility (a softer top must not mask the event) before -> after.++## Typical fixes+`highshelf` / `peq` cuts on the offending sample, `norm_peak_db` down a few dB, `lp` on a bright bed, `mono` on point+sources, `pitch` to the key (within +-2 st) or another note's recording, `master.lufs` -1 for a dynamic track.++## Return (JSON only)+`{"lens": "tonal", "issues": [{"id", "priority", "t", "what", "problem", "evidence", "ops": [...], "verified"}],+"overall": "...", "score": 1-10}`diff --git a/skills/real-sound/templates/prompts/designer.md b/skills/real-sound/templates/prompts/designer.mdnew file mode 100644index 0000000..cf88987--- /dev/null+++ b/skills/real-sound/templates/prompts/designer.md@@ -0,0 +1,49 @@+# Brief: design the {{CATEGORY}} fragment++You are the sound designer for **{{CATEGORY}}** in {{FILM}}. You turn the ranked recordings in+`audio/assets/manifests/{{CATEGORY}}.json` into `audio/design/{{CATEGORY}}.json`: trimmed samples, pools and rules on+the picture's cue events. You cannot hear: you verify every trim, level and sync point by measurement and by earshot.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md`.+- Format: `{{SKILL}}/references/design-format.md`; craft: `{{SKILL}}/references/sound-design.md`; an example of every+ feature: `{{SKILL}}/templates/design_fragment.json`.+- Your cue events (kind / id regex / times): {{EVENTS}}+- Prefix every name with `{{PREFIX}}.` (samples, slices, pools, rule names).+- You own ONLY `audio/design/{{CATEGORY}}.json` and `out/audio-review/{{CATEGORY}}/`. Never edit another fragment, the+ top-level design, the manifests or the assets.++## Steps+1. **Trims.** For each need, take the manifest's best take and its notes; get exact trims from earshot and check them:+ ```+ {{PY}} -m earshot find <file> "{{SINGLE_EVENT_DESCRIPTION}}" --top 8 # start / end ready to use+ {{PY}} -I {{SKILL}}/scripts/mix.py --only {{CATEGORY}} --dump out/audio-review/{{CATEGORY}}/dump+ ```+ Read `_samples.json`: attack 3-5 ms after the start, head and tail far below the peak (head < -30 dB, tail < -35 dB+ re peak unless faded), no second hit, a sensible duration. Use `slices` for recordings of many hits.+2. **Shape.** `mono` for point sources, `eq` (high-pass the rumble, tame 2-6 kHz harshness with a `highshelf` or `peq`),+ `pitch` for scale (+2..+6 st makes it smaller), `norm_peak_db` so a pool is even, fades long enough to never click.+3. **Rules.** One rule per layer per event class: `align: attack` for impacts / clicks / keys / pops, `peak` for+ whooshes, `end` for risers. Pools of 4+ takes with `mode: cycle` for repeated actions, `jitter_db` 1-2, small+ `pitch` ranges; one fixed voice for UI clicks. Name every rule (variation is keyed on the name). Write `_doc` for+ every non-obvious number.+4. **Verify alone**:+ ```+ {{PY}} -I {{SKILL}}/scripts/mix.py --only {{CATEGORY}} --trace out/audio-review/{{CATEGORY}}/adds.json --out out/audio-review/{{CATEGORY}}/solo.wav+ {{PY}} -I {{SKILL}}/scripts/qa.py --only {{CATEGORY}} --report out/audio-review/{{CATEGORY}}/solo_report.json --trace out/audio-review/{{CATEGORY}}/adds.json --stems out/audio-review/{{CATEGORY}}/solo_stems --out out/audio-review/{{CATEGORY}}/qa.json+ {{PY}} -m earshot timeline out/audio-review/{{CATEGORY}}/solo.wav --cues out/cues.json+ ```+ (`--only` writes the report and stems next to `--out`: `solo_report.json`, `solo_stems/`.) In qa.json: no clicks at+ an edge, no edge defects, sync within limits, no machine-gun rule, every event of your classes sounded. In the+ timeline: an event on every cue, tags that read as the intended sound.+5. **Levels in context.** When the lead has an integrated mix, read its qa.py audibility rows for your events+ (read_db >= 3, ideally >= 6) and adjust rule gains; never louder than the cause deserves.++## Rules+- Only real recordings from the manifest (licence confirmed in audit.json). Processing: trim, fade, gain, pan, EQ,+ resampling, gain envelope. Never synthesize.+- Every claim in your report comes from a command you ran.++## Return+The fragment's contents in a table (need -> sample(s) -> rule -> events), the qa.json summary of your solo render, the+earshot timeline lines for your key moments, what you are unsure about, commands run.diff --git a/skills/real-sound/templates/prompts/fixer.md b/skills/real-sound/templates/prompts/fixer.mdnew file mode 100644index 0000000..7cca30f--- /dev/null+++ b/skills/real-sound/templates/prompts/fixer.md@@ -0,0 +1,51 @@+# Brief: the fixer (one agent applies the critics' changes)++You are the only agent allowed to edit the design during this fix round of {{FILM}}. Five read-only critics (sync,+edges, masking, tonal, director) returned ranked issues with exact operations and the numbers of the variants they+tested. Your job: merge them into one consistent change, apply it, re-render, prove nothing regressed.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Design `audio/sounddesign.json` + `audio/design/*.json`.+- Critic reports: {{CRITIC_REPORTS}} (JSON). Previous qa: `{{QA_JSON}}`. Round {{ROUND}}.++## Operations (the format every critic used; `scripts/variant.py` applies them)+```+{"f": "<fragment file name without .json>", "sample": "<name>", "set": {...}} also "slice": name+{"f": "...", "sample": "<name>", "eq": {...}} merged into the sample's eq+{"f": "...", "rule": "<rule name>", "set": {...}} | {"f": "...", "rule": "<name>", "delete": true} | {"f": "...", "rule_add": {...}}+{"f": "...", "place": "<sample>", "set": {...}} | {"f": "...", "place_add": {...}}+{"f": "ambience", "amb": "<name>", "set": {...}} {"f": "music", "segment": <index>, "set": {...}}+{"f": "...", "pool": "<name>", "value": [...]} {"sd": "buses.music.auto", "value": [[t, dB], ...]}+```++## Steps+1. **Collect** every P0 / P1 with its operations; drop issues whose `verified` numbers do not show an improvement.+2. **Resolve conflicts.** Two critics touching the same sample, rule or ride: find the change that satisfies both (a+ hit that must be louder AND less harsh -> more low body, a `highshelf` cut on top); masking vs density (louder vs+ fewer): prefer removing a dead layer over raising a level; any bus ride change against the director's arc wins only+ with numbers. Write each decision down.+3. **Apply** all operations at once:+ ```+ {{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/fix_round{{ROUND}}.json --out out/audio-review/fix_round{{ROUND}} --render --qa+ ```+ compare its qa.json with `{{QA_JSON}}` section by section: every targeted issue improved, no new fail, no new click,+ no new masked event, loudness still on target. Iterate on the variant until it holds, then:+ ```+ {{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/fix_round{{ROUND}}.json --apply --backup out/audio-review/backup_round{{ROUND}}+ {{PY}} -I {{SKILL}}/scripts/mix.py --trace out/qa/adds.json+ {{PY}} -I {{SKILL}}/scripts/qa.py --layers+ ```+ `--apply` rewrites only the values that changed (it lists the files it wrote), so `diff -r` against the backup is+ the reviewable record of the fix; a line `unchanged (...)` means that operation was already in the design.+4. **Acknowledge** confirmed deliberate exceptions with `"qa_ok": "<reason>"` on the sample / rule / segment (one+ sentence each) so later rounds do not re-litigate them.+5. **Record** in each touched fragment's `_doc` / the top level's `_buses_doc` why the numbers are what they are.++## Rules+- You alone write the design this round; critics and designers do not.+- Never synthesize; never change `mix.py` or `master.py`; never loosen a qa threshold to pass.+- Every number in your report comes from a command you ran.++## Return+The applied operations (merged list), each conflict and how you resolved it, the qa.json summary before -> after per+section, issues you rejected and why, open questions for the next round, commands run.diff --git a/skills/real-sound/templates/prompts/licence_audit.md b/skills/real-sound/templates/prompts/licence_audit.mdnew file mode 100644index 0000000..197ad91--- /dev/null+++ b/skills/real-sound/templates/prompts/licence_audit.md@@ -0,0 +1,44 @@+# Brief: licence audit of every recording++You are the licence auditor for {{FILM}}. You decide nothing about sound; you make sure every recording that can reach+the film is allowed, verified on its own page, credited correctly and intact. Read+`{{SKILL}}/references/licences.md` first.++## Context+- Project root: `{{ROOT}}`. Python: `{{PY}}`. Skill: `{{SKILL}}`.+- Manifests: `audio/assets/manifests/*.json`. Audit file: `audio/assets/manifests/audit.json` (create it as `[]` if+ missing). Mix report (when a mix exists): `out/qa/mix_report.json`.+- You own ONLY the agent fields of `audit.json` (`licence_seen`, `licence_confirmed`, `agent_notes`) and may correct a+ manifest's `license`, `license_url`, `author`, `title`, `page_url`, `attribution` when the page says otherwise+ (say so in your report).++## Steps+1. Run the audit to get the list and the file checks:+ ```+ {{PY}} -I {{SKILL}}/scripts/audit.py --root . [--report out/qa/mix_report.json]+ ```+ Without a report every entry is in scope; with it, the recordings audible in the film are.+2. For every in-scope entry, open its `page_url` (Freesound item page; for Mixkit the licence text+ https://mixkit.co/license/ plus the category page listing the item; for music the composer's track page) and read:+ the licence as written, the author, the title, the requested credit form, any restriction (no raw redistribution,+ no stock use). Check the download URL belongs to the same item (Freesound: the preview's id and user id match the page).+3. Write into `audit.json` for that entry (match `id` and `file_check.path`; see `templates/audit_entry.json`):+ `licence_seen`: "<site> page <url>: licence '<exact text>' -> <licence url> (read <date>); page author '<x>', title '<y>'",+ `licence_confirmed`: true only if the licence is allowed AND you read it on the page in this session,+ `agent_notes`: requested credit forms, restrictions, doubts.+ If the page shows a different or forbidden licence, set `licence_confirmed: false`, fix the manifest, and list it.+4. Re-run audit.py (with `--fetch` once the film's recordings are final: it compares the local bytes with+ `download_url`, sequentially and politely) until it exits 0 for the in-scope set. Then:+ ```+ {{PY}} -I {{SKILL}}/scripts/credits.py --root . --audit audio/assets/manifests/audit.json --design audio/sounddesign.json+ ```++## Rules+- Never confirm a licence you did not read on the item's own page in this session. Aggregators are not sources.+- Disputed or unclear = rejected. CC BY-SA / NC / ND / Sampling+ = rejected.+- Be polite to the sites: one page at a time, a pause between requests.++## Return+The audit summary line, every blocker / warning with what you did, every manifest correction, items rejected (and a+replacement suggestion from the same manifest), the credit lines that follow an author's requested form, and the+commands you ran.diff --git a/skills/real-sound/templates/prompts/music_supervisor.md b/skills/real-sound/templates/prompts/music_supervisor.mdnew file mode 100644index 0000000..9f49050--- /dev/null+++ b/skills/real-sound/templates/prompts/music_supervisor.md@@ -0,0 +1,50 @@+# Brief: music supervisor (choose the track, verify its grid, edit it to the picture)++You are the music supervisor for {{FILM}}. One licensed track carries the film's emotion; your job is to pick it and+cut it to the picture at bar lines so that its own events (entrance, drop, stop, final chord) land on the picture's+beats. You cannot hear: you measure the grid, the bars, the sections and every splice.++## Context+- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (picture map, mood, arc).+- Picture grid: {{BPM}} BPM, {{BEATS_PER_BAR}}/4, bar = {{BAR}} s; film bars: {{FILM_BARS}}; key moments: drop / hero+ reveal at {{DROP_T}} s (film bar {{DROP_BAR}}), breakdown {{BREAKDOWN}}, logo at {{LOGO_T}} s, end {{DURATION}} s.+- Candidates: `audio/assets/music/` with manifests `audio/assets/manifests/music*.json` (licences audited).+- You own: `audio/assets/manifests/music_choice.json` (the decision record), `audio/design/music.json`,+ `out/audio-review/music/`.++## Steps+1. **Measure every candidate**:+ ```+ {{PY}} -I {{SKILL}}/scripts/analyze.py audio/assets/music/* --music --png out/audio-review/music --out out/audio-review/music.json+ {{PY}} -I {{SKILL}}/scripts/arrange.py <track> --film-bpm {{BPM}} --arc out/audio-review/music/arc.json --out out/audio-review/music/<id>_arrange.json+ {{PY}} -m earshot describe <track> # mood / instrumentation in words, sections+ {{PY}} -m earshot timeline <track> # events, sections, beat grid, loudness as a transcript+ ```+ Write the arc file first (`templates/arc.json`): one target dB per film bar from the picture map.+2. **Verify the grid by hand**, not by trust: the tempo (arrange.py fits a line through every beat; residual and+ inlier fraction say how steady), the downbeat phase (confidence; check against the low-band hits and the chord+ changes in the bar table and the spectrogram), and the transient offset (arrange.py moves the grid onto the measured+ attacks; a cut on a lagging grid chops the downbeat). Write the verified grid as `downbeats_s` in the decision record.+3. **Choose** with reasons: structure fits the film's arc (a soft opening, a drop that can land on the reveal, an+ ending that resolves), mood fits the brand, tempo needs no or little stretch (within ~6 %; `stretch: none` within+ 0.2 %), clean master, licence confirmed. Record the runner-up.+4. **Edit**: take arrange.py's segment draft, then fix it by musical judgement: splices from bar to bar where the+ harmony continues (V -> I, or the same chord), the drop's downbeat on the hero frame, the final chord on the logo+ or the cut to black, a `tail_fade` into the end. Nudge `at` by a few ms so the measured downbeat transient sits on+ the picture's downbeat. Label every segment with its bars, its chords and why.+5. **Verify** the edit alone and check every splice:+ ```+ {{PY}} -I {{SKILL}}/scripts/mix.py --only music --trace out/audio-review/music/adds.json --out out/audio-review/music/solo.wav+ {{PY}} -m earshot timeline out/audio-review/music/solo.wav --cues out/cues.json+ ```+ and read the qa.py edges section of the integrated mix (chopped splices, holes, segments starting inside a sustained+ note). Set `music.gain_db` so the big sections sit around -14 LUFS short-term before the master.++## Rules+- One track; real, professionally produced; licence confirmed in audit.json; never a rip.+- Edit only at bar lines; never stretch more than ~6 %; never loop a bar so often it becomes audible.+- Every number in the decision record comes from a command you ran.++## Return+The winner and runner-up with measured reasons, the verified grid, the segment list with splice reasons, the solo+render's timeline highlights (entrance, drop, final chord times), open risks (Content ID for CC BY music), commands run.diff --git a/skills/real-sound/templates/prompts/sourcing.md b/skills/real-sound/templates/prompts/sourcing.mdnew file mode 100644index 0000000..7e23aea--- /dev/null+++ b/skills/real-sound/templates/prompts/sourcing.md@@ -0,0 +1,60 @@+# Brief: source real recordings for {{CATEGORY}}++You are the sourcing agent for the **{{CATEGORY}}** sounds of {{FILM}}. You cannot hear, so you measure: every choice+you make is backed by numbers from the commands below. Read `{{ROOT}}/audio/SOUND_BRIEF.md` first (the contract) and+`{{SKILL}}/references/licences.md` + `{{SKILL}}/references/sound-design.md`.++## Context+- Project root: `{{ROOT}}`. Python: `{{PY}}` (run scripts with `-I`). Skill: `{{SKILL}}`.+- Your needs (from the picture map), each with its cue times and what it must sound like:+{{NEEDS}}+ <!-- e.g. "a_ui_click: the film's one click voice, 2 clicks at 1.0 and 3.5 s; crisp, short (earshot decay < 30 ms), dry,+ a clear press + release" / "b_whoosh: 2 camera-scale moves at 4.0 and 12.5 s; real air, 0.2-0.6 s, no tonal swoosh" -->+- You own ONLY: `audio/assets/{{CATEGORY}}/`, `audio/assets/manifests/{{CATEGORY}}.json`, `out/audio-review/{{CATEGORY}}/`.++## Steps+1. **Search** Freesound (CC0 / CC BY only: use its licence filter, then read each item's page anyway), Mixkit (lossless+ WAV), and for music the composers' own sites. Prefer recordings that contain many takes of the same object+ (variation for free), close-miked, dry, lossless or HQ, 44.1 kHz+. 15-40 candidates for the category, 3-6 per need.+2. **Read each item's own page**: title, author, licence (exact text), the author's requested credit form, anything the+ description says about the recording (mic, room, what happens when). Reject NC / ND / SA / Sampling+ / custom /+ unclear on sight. Never fabricate or guess a licence.+3. **Download politely**: sequentially, a pause between requests, the public HQ file+ (`https://cdn.freesound.org/previews/<id//1000>/<id>_<userid>-hq.ogg`, Mixkit+ `https://assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`). Save as `audio/assets/{{CATEGORY}}/<source>-<id>_<slug>.<ext>`.+ Audio files only; never run or unpack anything downloaded. Pixabay blocks scripts: list such items for the user.+4. **Measure every file**:+ ```+ {{PY}} -I {{SKILL}}/scripts/analyze.py audio/assets/{{CATEGORY}}/* --png out/audio-review/{{CATEGORY}} --out out/audio-review/{{CATEGORY}}.json+ {{PY}} -m earshot realness audio/assets/{{CATEGORY}}/+ ```+ Open the spectrogram PNGs of the best candidates (the Read tool shows images). Reject: clipped, floor above -55+ dBFS between events, a cliff above 16 kHz (low bitrate), room echo on foley, voices / bumps / dogs in the part you+ need, realness verdict "likely code-synthesized". An "uncertain" file may stay only when you re-run realness on+ the trim the designer will use (`--start --end`) and write its evidence and the trim's verdict in the manifest+ `notes` (designed library sounds - whooshes, chimes - often sit there).+5. **Listen through earshot** for each need:+ ```+ {{PY}} -m earshot rank "{{NEED_DESCRIPTION}}" audio/assets/{{CATEGORY}}/ --top 6+ {{PY}} -m earshot match <file> "{{NEED_DESCRIPTION}}" --alts "<what it must not be>" "<another wrong reading>"+ {{PY}} -m earshot find <long file> "{{SINGLE_EVENT_DESCRIPTION}}" --top 8 # the best takes inside it as trims (skip any marked head cut / tail cut)+ {{PY}} -m earshot describe <file>+ ```+ Quote earshot's own verdict words, not your own z cut-offs: strong (z >= 3), plausible (2.5-3), weak (2-2.5),+ no match (< 2). `p_vs_alts` only says which of your wordings wins. Use find's trims as they are, but never one+ marked 'head cut' / 'tail cut'; for two candidate takes, `{{PY}} -m earshot ab <a> <b> --query "{{NEED_DESCRIPTION}}"+ --a-start .. --a-end .. --b-start .. --b-end ..`. When approved references exist,+ `{{PY}} -m earshot compare <file> --refs <folder>`.+6. **Write the manifest** (`templates/manifest_entry.json` shape): every field, `attribution` for CC BY (the author's+ form, else the default line), `need`, `rank` (1 = best for that need), `best_segment`, `analysis` (from analyze.py),+ `earshot` (realness, match z), and `notes` that tell the designer exactly what to use: the times of the best takes,+ what to avoid (a bump at 9.6 s), how loud it is, suggested trims / EQ / pitch.++## Rules+- Real recordings only; no generated or synthesized audio, no YouTube rips, nothing behind a login.+- Every number in your notes comes from a command you ran.+- Do not write design JSON; do not touch other categories.++## Return+A short report: per need the ranked candidates (id, why, best segment, measured key numbers, realness, match z), the+rejected ones with the reason, any item the user must download by hand (Pixabay), and the commands you ran.diff --git a/skills/real-sound/templates/sounddesign.json b/skills/real-sound/templates/sounddesign.jsonnew file mode 100644index 0000000..78df807--- /dev/null+++ b/skills/real-sound/templates/sounddesign.json@@ -0,0 +1,22 @@+{+ "_doc": "The film's soundtrack (real-sound skill): professional recordings only, credited in CREDITS.md. Built by <skill>/scripts/mix.py from this file and one fragment per category in audio/design/ (each designer owns one fragment; names are prefixed with the category). Keys starting with _ are comments. Reference: <skill>/references/design-format.md.",+ "duration": 30.0,+ "cues": "out/cues.json",+ "include": [+ "audio/design/music.json",+ "audio/design/ambience.json",+ "audio/design/ui.json",+ "audio/design/keyboard.json",+ "audio/design/motion.json",+ "audio/design/impacts.json",+ "audio/design/logo.json"+ ],+ "_buses_doc": "Fader rides that shape the arc (film seconds, dB). Write a reason for every ride here: e.g. the music dips 6 dB under the typing (2.0-3.6), comes back for the transition, lifts 1.5 dB into the drop, and resolves 3 dB down under the logo so the sting reads. Keep steps on a musical splice or under a transient.",+ "buses": {+ "music": {"gain_db": 0.0, "auto": [[0, 0], [1.9, 0], [2.0, -6], [3.6, -6], [3.8, 0], [14.9, 0], [15.0, 1.5], [24.0, 1.5], [26.0, -3], [30.0, -3]]},+ "amb": {"gain_db": 0.0},+ "sfx": {"gain_db": 0.0}+ },+ "_master_doc": "-14 LUFS integrated for web / social (YouTube, X, LinkedIn normalise to about -14). Ceiling 0.77 linear (-2.3 dBFS) leaves room for the AAC encode (measure about -1 dBTP after the mux). Lower the target to -15 / -16 only for a reason you write here (e.g. a dynamic orchestral track whose tutti the limiter would squash at -14).",+ "master": {"lufs": -14, "ceiling": 0.77, "no_glue": true}+}diff --git a/tests/conftest.py b/tests/conftest.pynew file mode 100644index 0000000..ddb1724--- /dev/null+++ b/tests/conftest.py@@ -0,0 +1,49 @@+"""Shared pytest fixtures: synthetic test signals only (never committed media)."""+import numpy as np+import pytest+import soundfile as sf++SR = 48000+++def pytest_configure(config):+ config.addinivalue_line('markers', 'slow: loads the models or runs longer than ~10 s (run with --slow)')+++def pytest_addoption(parser):+ parser.addoption('--slow', action='store_true', help='also run tests that load the models')+++def pytest_collection_modifyitems(config, items):+ if config.getoption('--slow'):+ return+ skip = pytest.mark.skip(reason='needs --slow (loads models or runs long)')+ for it in items:+ if 'slow' in it.keywords:+ it.add_marker(skip)+++def wav(tmp_path):+ """write a float array (n,) or (n, 2) at 48 kHz to a temp WAV and return its path"""+ def make(x, name='x.wav'):+ p = tmp_path / name+ sf.write(str(p), np.asarray(x, np.float32), SR, subtype='FLOAT')+ return str(p)+ return make+++def click():+ """a short decaying noise burst (a click-like transient) with a faint noise floor"""+ rng = np.random.RandomState(0)+ n = int(0.25 * SR)+ t = np.arange(n) / SR+ x = rng.randn(n) * np.exp(-t / 0.01) * 0.5 + rng.randn(n) * 1e-4+ return x.astype(np.float32)+++def sine():+ t = np.arange(SR) / SR+ return (0.3 * np.sin(2 * np.pi * 880 * t)).astype(np.float32)diff --git a/tests/test_core.py b/tests/test_core.pynew file mode 100644index 0000000..e4aa446--- /dev/null+++ b/tests/test_core.py@@ -0,0 +1,176 @@+"""Fast tests for core behaviour fixed in the final review: envelope times, decode errors, cache fingerprints, CLI+output on any code page, find's trims, critique's reference match and window thresholds. Synthetic signals only."""+import io+import json+import os+import subprocess++import numpy as np+import pytest++from earshot import audio, cli, critique, features, profile, report, search, sheet++SR = 48000+++def _decaying(f, tau=0.5, dur=4.0, lead=0.1):+ t = np.arange(int(dur * SR)) / SR+ x = (0.8 * np.sin(2 * np.pi * f * t) * np.exp(-t / tau)).astype(np.float32)+ return np.concatenate([np.zeros(int(lead * SR), np.float32), x])+++# ---------------------------------------------------------------------------------------------- envelope[email protected]('f', [40, 80, 200, 1000])+def test_decay_is_right_at_any_pitch(f):+ true = 0.5 * np.log(10 ** 1.5) * 1000 # -30 dB of exp(-t / 0.5): 1727 ms+ r = features.measure(_decaying(f), light=True)+ assert abs(r['decay_ms'] - true) <= 0.02 * true+ assert r['attack_ms'] <= 6 # an instant onset: ~4 ms on the 5 ms envelope+++def test_decay_of_a_noise_burst_and_attack_of_a_steady_tone():+ rng = np.random.RandomState(0)+ n = 2 * SR+ burst = (rng.randn(n) * np.exp(-np.arange(n) / SR / 0.1) * 0.5).astype(np.float32)+ true = 0.1 * np.log(10 ** 1.5) * 1000 # 345 ms+ assert abs(features.measure(burst, light=True)['decay_ms'] - true) <= 0.05 * true+ tone = (0.5 * np.sin(2 * np.pi * 440 * np.arange(2 * SR) / SR)).astype(np.float32)+ r = features.measure(tone, light=True)+ assert r['attack_ms'] <= 2 and r['decay_ms'] is None # starts at full level, never decays+++def test_measure_rejects_impossible_ranges(wav, sine):+ p = wav(sine)+ with pytest.raises(ValueError, match='beyond the end'):+ features.measure(p, start=5.0)+ with pytest.raises(ValueError, match='greater than start'):+ features.measure(p, start=0.5, end=0.2)+++# ---------------------------------------------------------------------------------------------- audio+def test_file_id_sees_a_same_size_swap_with_the_old_timestamp(wav, sine):+ p = wav(sine, 'swap.wav')+ st = os.stat(p)+ a = audio.file_id(p)+ import soundfile as sf+ sf.write(p, (sine[::-1] * 0.5).astype(np.float32), SR, subtype='FLOAT') # same length, same size+ os.utime(p, ns=(st.st_atime_ns, st.st_mtime_ns))+ assert os.stat(p).st_size == st.st_size and os.stat(p).st_mtime_ns == st.st_mtime_ns+ assert audio.file_id(p) != a+++def test_decode_errors_are_one_readable_line(tmp_path):+ bad = tmp_path / 'corrupt.wav'+ bad.write_text('not audio')+ with pytest.raises(audio.AudioError) as e:+ audio.load(str(bad), cache=False)+ assert '\n' not in str(e.value) and 'cannot decode' in str(e.value)+ vid = tmp_path / 'silent render.mp4'+ r = subprocess.run(['ffmpeg', '-v', 'error', '-f', 'lavfi', '-i', 'color=c=black:s=64x64:d=1', '-an', str(vid)],+ capture_output=True)+ if r.returncode != 0:+ pytest.skip('ffmpeg cannot make a test video here')+ with pytest.raises(audio.AudioError, match='has no audio track'):+ audio.load(str(vid), cache=False)+++def test_missing_paths_are_named(tmp_path, wav, sine):+ ok = wav(sine)+ audio.check_paths([ok])+ with pytest.raises(audio.AudioError, match='nope.wav'):+ audio.check_paths([ok, str(tmp_path / 'nope.wav')])+++def test_replace_quietly_never_raises(tmp_path):+ tmp = tmp_path / 'x.tmp'+ tmp.write_text('1')+ assert audio.replace_quietly(str(tmp), str(tmp_path / 'x'))+ tmp.write_text('2')+ assert not audio.replace_quietly(str(tmp), str(tmp_path / 'no' / 'such' / 'dir' / 'x'), tries=1)+ assert not tmp.exists()+++# ---------------------------------------------------------------------------------------------- CLI output+def test_json_on_a_cp1252_stdout_is_valid(monkeypatch):+ buf = io.BytesIO()+ out = io.TextIOWrapper(buf, encoding='cp1252')+ monkeypatch.setattr('sys.stdout', out)+ report.emit({'summary': 'ok', 'file': 'キー音.wav'}, type('A', (), {'json': True, 'out': None})())+ out.flush()+ assert json.loads(buf.getvalue().decode('cp1252'))['file'] == 'キー音.wav'+++def test_threads_after_the_command_and_errors_on_stderr(wav, sine, capsys, monkeypatch):+ monkeypatch.setenv('EARSHOT_THREADS', '3') # restored after the test+ p = wav(sine)+ assert cli.main(['measure', p, '--light', '--threads', '2']) == 0+ assert os.environ.get('EARSHOT_THREADS') == '2'+ capsys.readouterr()+ assert cli.main(['measure', p + '.nope']) == 1+ err = capsys.readouterr().err.strip()+ assert err.startswith('earshot measure: error:') and '\n' not in err+++def test_profile_summary_survives_unmeasured_metrics():+ p = {'n': 2, 'metrics': {k: {'n': 0} for k in profile.METRICS}}+ assert 'LRA n.a.' in profile.profile_summary(p)+++def test_sheet_ticks_at_quarter_seconds_are_exact():+ assert sheet._tick_label(0.25, 0.25) == '0:00.25' and sheet._tick_label(0.75, 0.25) == '0:00.75'+ assert sheet._tick_label(0.5, 0.5) == '0:00.5' and sheet._tick_label(61.0, 1) == '1:01'+++# ---------------------------------------------------------------------------------------------- find trims+def _swish(n, rng, peak_at=0.1):+ t = np.arange(n) / SR+ env = np.exp(-((t - peak_at) / 0.05) ** 2)+ return (rng.randn(n) * env * 0.4).astype(np.float32)+++def test_events_never_merge_across_an_edit_gap():+ rng = np.random.RandomState(3)+ a, b = _swish(int(0.3 * SR), rng), _swish(int(0.3 * SR), rng)+ x = np.concatenate([np.zeros(SR // 10, np.float32), a, np.zeros(int(0.05 * SR), np.float32), b,+ np.zeros(SR // 2, np.float32)])+ env = search._env_db(x)+ assert search._edit_gaps(env, 0.35, 0.5)+ for s, e, k, _ in search.event_candidates(x, n=None):+ assert not (s < 0.42 and e > 0.47), (s, e, k) # nothing spans the 50 ms of digital silence+++def test_windows_snap_to_the_sound_and_multi_sound_windows_are_detected():+ rng = np.random.RandomState(4)+ n = int(1.2 * SR)+ t = np.arange(n) / SR+ rise = (rng.randn(n) * np.minimum(1.0, t / 0.8) ** 3 * np.exp(-np.maximum(0, t - 0.8) / 0.05) * 0.4)+ x = np.concatenate([np.zeros(SR // 5, np.float32), rise.astype(np.float32), np.zeros(SR // 2, np.float32)])+ env = search._env_db(x)+ floor = float(np.percentile(env, 10))+ a, b = search.snap_window(env, floor, 0.7, 1.2, 0.0, len(x) / SR) # starts mid-rise, ends mid-decay+ assert a < 0.5 and search._head_db(env, a, b) <= search.CUT_DB and search._tail_db(env, a, b) <= -30+ two = np.concatenate([_swish(int(0.3 * SR), rng), np.zeros(int(0.4 * SR), np.float32), _swish(int(0.3 * SR), rng)])+ e2 = search._env_db(two)+ f2 = float(np.percentile(e2, 10))+ assert search.multi_sound(e2, f2, 0.0, len(two) / SR) and not search.multi_sound(e2, f2, 0.0, 0.3)+++# ---------------------------------------------------------------------------------------------- critique+def test_a_reference_film_is_scored_leave_one_out():+ prof = profile.load_profile()+ g = prof['films'][4]+ f = {'duration': g['duration'], 'metrics': dict(g['metrics'])}+ assert critique.reference_match(f, prof) == g['id']+ loo = critique.leave_one_out(prof, g['id'])+ assert loo['n'] == prof['n'] - 1 and g['id'] not in [x['id'] for x in loo['films']]+ f['duration'] += 1.0+ assert critique.reference_match(f, prof) is None+++def test_critique_flags_windows_at_the_window_threshold():+ t = critique.real_thresholds(file=False)+ segs = [{'start': 4.0 * i, 'end': 4.0 * i + 4, 'p_real': p} for i, p in enumerate([0.3, 0.18, t['lo'] - 0.01])]+ f = {'realness': {'segments': segs}, 'events': {}, 'duration': 12.0, 'metrics': {}}+ _, flagged, tw = critique._realness_windows(f)+ assert tw == t and [q['p_real'] for q in flagged] == [t['lo'] - 0.01]+ assert critique._p(0.766, {'lo': 0.4, 'hi': 0.77}) == '0.766' and critique._p(0.81, {'lo': 0.4, 'hi': 0.77}) == '0.81'diff --git a/tests/test_forge.py b/tests/test_forge.pynew file mode 100644index 0000000..3b06a8b--- /dev/null+++ b/tests/test_forge.py@@ -0,0 +1,241 @@+"""Fast tests for earshot forge: a toy recipe and a fake critic exercise the search, diverse picks, rendering, robust+re-scoring, family spaces, forge() end to end, and the dsp building blocks. No models are loaded."""+import json+import math+import types++import numpy as np+import pytest++from earshot.forge import dsp, forge+from earshot.forge import search+from earshot.forge.search import _decode, _encode, diverse, optimize, render, robust++SR = 48000+++def _toy_render(p, r):+ """a decaying sine at p['f'] Hz with time constant p['tau'], plus a little noise (r gives natural variation)"""+ n = int(0.2 * SR)+ t = np.arange(n) / SR+ x = np.sin(2 * np.pi * p['f'] * t) * np.exp(-t / p['tau']) + 0.001 * r.standard_normal(n)+ return x * {'soft': 0.2, 'hard': 1.0}[p['shape']]+++def toy(**over):+ m = types.SimpleNamespace(__name__='earshot.forge.recipes.toy',+ SPACE={'f': (200.0, 4000.0, 'log'), 'tau': (0.005, 0.2, 'log'), 'shape': ['soft', 'hard'],+ 'n': (1, 9, 'int')},+ DEFAULTS={'f': 300.0, 'tau': 0.15, 'shape': 'soft', 'n': 3}, render=_toy_render)+ for k, v in over.items():+ setattr(m, k, v)+ return m+++class FakeCritic:+ """scores by closeness to 1 kHz and a 30 ms decay (no models); counts how many sounds it heard"""+ target, alts, refs = 'a 1 kHz ping', ['a low hum'], []++ def __init__(self):+ self.heard = 0++ def score(self, xs):+ rows = []+ for i, x in enumerate(xs):+ self.heard += 1+ m = np.asarray(x).mean(axis=1) if np.ndim(x) == 2 else np.asarray(x)+ F = np.abs(np.fft.rfft(m))+ f = np.fft.rfftfreq(len(m), 1 / SR)[int(np.argmax(F))]+ w = SR // 1000+ env = np.abs(m[:len(m) // w * w]).reshape(-1, w).max(axis=1) # 1 ms envelope+ tau = max(1e-3, np.argmax(env < env.max() * math.exp(-1)) / 1000)+ total = 5.0 - 4 * abs(math.log2(max(f, 1.0) / 1000)) - abs(math.log(tau / 0.03))+ rows.append({'i': i, 'total': total, 'passes': total > 3.5, 'match_z': total, 'alt_margin': 1.0,+ 'p_real': 0.9, 'ref_typ_z': None, 'feat_pen': 0.0, 'sounds_like': ['a ping'],+ 'summary': f'total {total:.2f}'})+ return rows++ def holdout(self, xs):+ return [{'audioset_top': [['Ping', 0.5]], 'audioset_synth_max': 0.0} for _ in xs]+++# ---------------------------------------------------------------------------------------------- search+def test_encode_decode_round_trip():+ space = toy().SPACE+ p = {'f': 1000.0, 'tau': 0.03, 'shape': 'hard', 'n': 7}+ q = _decode(space, _encode(space, p))+ assert abs(q['f'] - 1000.0) < 1e-6 and abs(q['tau'] - 0.03) < 1e-9+ assert q['shape'] == 'hard' and q['n'] == 7+++def test_render_peak_normalises_and_rejects_silence():+ x = render(toy(), {'f': 1000.0, 'tau': 0.03, 'shape': 'soft', 'n': 1}, 1)+ assert x.dtype == np.float32+ assert abs(20 * np.log10(np.abs(x).max()) + 1.0) < 1e-3+ silent = toy(render=lambda p, r: np.zeros(4800))+ assert render(silent, {}, 1) is None+++def test_render_logs_the_first_exception_once(capsys):+ search._failed.discard('broken')++ def boom(p, r):+ raise ValueError('bad mode count')+ bad = types.SimpleNamespace(__name__='broken', SPACE={}, render=boom)+ assert render(bad, {}, 1) is None and render(bad, {}, 2) is None+ err = capsys.readouterr().err+ assert err.count('bad mode count') == 1 and 'broken' in err+++def test_optimize_improves_on_the_default_and_is_deterministic():+ c = FakeCritic()+ rec = toy()+ base = c.score([render(rec, rec.DEFAULTS, 0)])[0]['total']+ a = optimize(rec, c, budget=96, pop=16, seed=3)+ b = optimize(rec, FakeCritic(), budget=96, pop=16, seed=3)+ assert len(a['history']) == 96 and a['generations'] == 6+ assert a['best'][0]['score']['total'] > base + 2+ assert a['best'][0]['params'] == b['best'][0]['params']+ tot = [h['score']['total'] for h in a['best']]+ assert tot == sorted(tot, reverse=True)+++def test_space_override_keeps_defaults_for_the_rest():+ rec = toy()+ res = optimize(rec, FakeCritic(), budget=32, pop=16, seed=1, space={'f': (900.0, 1100.0), 'shape': ['hard']})+ for h in res['history']:+ assert 900.0 <= h['params']['f'] <= 1100.0+ assert h['params']['shape'] == 'hard'+ assert h['params']['tau'] == rec.DEFAULTS['tau'] and h['params']['n'] == rec.DEFAULTS['n']+++def test_diverse_picks_differ_and_passing_come_first():+ res = optimize(toy(), FakeCritic(), budget=64, pop=16, seed=2)+ picks = diverse(res['history'], toy(), k=3, min_dist=0.08)+ assert len(picks) == 3+ for i in range(3):+ for j in range(i):+ assert np.linalg.norm(picks[i]['u'] - picks[j]['u']) / math.sqrt(len(picks[i]['u'])) >= 0.08+ passes = [p['score']['passes'] for p in picks]+ assert passes == sorted(passes, reverse=True)+++def test_robust_re_renders_on_new_seeds():+ c = FakeCritic()+ st = robust(toy(), c, {'f': 1000.0, 'tau': 0.03, 'shape': 'hard', 'n': 1}, seeds=4)+ assert c.heard == 4 and st['n'] == 4 and st['seeds'] == [5000, 5001, 5002, 5003] and st['failed'] == 0+ assert st['min'] <= st['mean'] <= st['max'] and st['pass_rate'] == 1.0+ flaky = toy(render=lambda p, r: _toy_render(p, r) if r.uniform() < 0.5 else np.zeros(100))+ st2 = robust(flaky, FakeCritic(), {'f': 1000.0, 'tau': 0.03, 'shape': 'hard', 'n': 1}, seeds=8)+ assert st2['failed'] + len(st2['totals']) == 8 and st2['pass_rate'] <= len(st2['totals']) / 8+++def test_forge_end_to_end_with_a_prebuilt_critic(tmp_path):+ rep = forge(toy(), budget=48, pop=16, keep=2, seed=4, out=str(tmp_path), log=None, critic=FakeCritic(), robust=3)+ assert rep['target'] == 'a 1 kHz ping' and len(rep['picks']) == 2+ for p in rep['picks']:+ assert p['robust']['n'] == 3 and p['holdout']['audioset_top'][0][0] == 'Ping'+ assert (tmp_path / (p['name'] + '.wav')).exists()+ disk = json.loads((tmp_path / 'report.json').read_text(encoding='utf-8'))+ assert disk['ref_scores'] == [] and len(disk['history']) == 48+++def test_recipes_are_listed_and_render_their_defaults():+ from earshot.forge import load_recipe, recipes+ names = recipes()+ assert {'keypress', 'click', 'pop', 'whoosh', 'impact'} <= set(names)+ for name in names:+ rec = load_recipe(name)+ x = render(rec, dict(getattr(rec, 'DEFAULTS', {}) or {}), 7)+ assert x is not None, name+ assert np.isfinite(x).all() and 0.01 < len(x) / SR < 12, name+++# ---------------------------------------------------------------------------------------------- dsp+def test_room_and_mic_accept_stereo():+ r = dsp.rng(1)+ hit = dsp.modal([620, 1490], [0.03, 0.02], [1, 0.5], 0.2, r)+ st = np.stack([hit, hit], axis=1)+ y = dsp.room(st, dsp.rng(2), rt60=0.2, wet=0.2)+ ym = dsp.room(hit, dsp.rng(2), rt60=0.2, wet=0.2)+ assert y.shape == (len(ym), 2)+ assert np.allclose(y[:, 0], ym) and np.allclose(y[:, 1], ym) # same room on both channels as on mono+ z = dsp.mic(y, dsp.rng(3), floor_db=-40)+ assert z.shape == y.shape+ assert not np.allclose(z[:, 0], z[:, 1]) # independent noise floor per channel+ zm = dsp.mic(ym, dsp.rng(3), floor_db=-40)+ assert zm.ndim == 1 and len(zm) == len(ym)+++def test_svf_band_gain_and_normalised_band():+ t = np.arange(SR // 2) / SR+ x = np.sin(2 * np.pi * 1000 * t)+ q = 8.0+ band = dsp.svf(x, 1000.0, q, 'band')[SR // 4:]+ bandn = dsp.svf(x, 1000.0, q, 'bandn')[SR // 4:]+ assert abs(np.abs(band).max() - q) < 0.1 * q+ assert abs(np.abs(bandn).max() - 1.0) < 0.05++[email protected]('mode', ['low', 'band', 'high'])+def test_svf_modes_are_unchanged_by_the_new_mode(mode):+ r = dsp.rng(5)+ x = r.standard_normal(4800)+ y = dsp.svf(x, 2000.0, 0.9, mode)+ assert np.isfinite(y).all() and y.shape == x.shape+++# ---------------------------------------------------------------------------------------------- after the judge+def test_optimize_reps_scores_each_candidate_on_several_seeds():+ c = FakeCritic()+ a = optimize(toy(), c, budget=32, pop=16, seed=5, reps=2)+ b = optimize(toy(), FakeCritic(), budget=32, pop=16, seed=5, reps=2)+ assert c.heard == 64 and len(a['history']) == 32+ h = a['best'][0]+ assert len(h['score']['total_reps']) == 2+ assert abs(h['score']['total'] - float(np.mean(h['score']['total_reps']))) < 1e-3+ assert a['best'][0]['params'] == b['best'][0]['params']+ one = optimize(toy(), FakeCritic(), budget=32, pop=16, seed=5) # reps=1 is the old search, unchanged+ assert 'total_reps' not in one['best'][0]['score']+++def _burst(r, lo, hi, n=SR // 4, boost=None):+ from earshot.forge import dsp+ t = np.arange(n) / SR+ x = dsp.butter(r.standard_normal(n), 'band', (lo, hi), 2) * np.exp(-t / 0.04)+ if boost is not None:+ x = x + 8.0 * dsp.butter(r.standard_normal(n), 'band', boost, 4) * np.exp(-t / 0.04)+ return x+++def test_tone_ref_finds_a_band_no_reference_has():+ from earshot.forge.tone import ToneRef, profile, shares+ r = np.random.default_rng(0)+ refs = [_burst(r, 300 * (1 + 0.1 * k), 6000) for k in range(5)]+ tr = ToneRef(refs)+ med, far = tr.spread+ assert 0 < med <= far+ good = tr.check(_burst(r, 320, 6000))+ bad = tr.check(_burst(r, 320, 6000, boost=(450, 650)))+ assert bad['tone_db'] > good['tone_db'] + 1.5+ assert any(w == 'above' and b in ('500 Hz', '630 Hz') for b, d, w in bad['outside'])+ assert tr.penalty(bad) > tr.penalty(good)+ assert 'above every ref' in bad['words']+ assert len(profile(refs[0])) == 28 and set(shares(refs[0])) == {'<150', '150-1k', '1-4k', '4-12k', '>12k'}+ loo = tr.check(refs[0], exclude=0) # a reference scored against the others only+ assert loo['tone_db'] == round(tr.ref_dist[0], 2)+++def test_keypress_and_pop_new_chains_and_old_params_unchanged():+ """round 13-15 / 8-9 keys change the sound; parameters without them (rounds 1-12 / 1-7) keep the old code path"""+ from earshot.forge import load_recipe+ from earshot.forge.tone import profile+ kp, pop = load_recipe('keypress'), load_recipe('pop')+ new = render(kp, kp.DEFAULTS, 3)+ old = render(kp, {k: v for k, v in kp.DEFAULTS.items() if k != 'hpf'}, 3)+ assert profile(new)[0] < profile(old)[0] - 6 # the session high-pass empties the 31 Hz band+ p = dict(pop.DEFAULTS)+ x = render(pop, p, 3)+ assert len(x) / SR < 0.45 # the edit: the kit's pops are 0.19-0.41 s+ legacy = {k: v for k, v in p.items() if k not in ('hp', 'lp', 'clip', 'cut_fade', 'cut_db')}+ assert len(render(pop, legacy, 3)) > len(x) # no edit keys: the old room-tail lengthdiff --git a/tests/test_interfaces.py b/tests/test_interfaces.pynew file mode 100644index 0000000..6153af0--- /dev/null+++ b/tests/test_interfaces.py@@ -0,0 +1,530 @@+"""interfaces: the optional audio-LLM critique (HTTP mocked: no network, no key needed) and the MCP server (in-process+tool functions + a real stdio round trip with the mcp client)."""+import base64+import json+import os+import sys++import numpy as np+import pytest++from earshot import cli, llm, mcp_server+from earshot.report import to_json++SR = 48000+KEY = 'unit-test-key-0123456789'+REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))+ANSWER = """```json+{"overall": {"score": 6, "verdict": "Real foley, but the riser is a preset."},+ "scores": {"sync": null, "realism": "5/10", "mix_balance": 7, "loudness": 8, "music": 6},+ "notes": {"realism": "the riser sounds synthetic"},+ "synthetic_or_cheap": [{"start": "0:01.5", "end": 2.0, "what": "riser", "why": "plain saw sweep"}],+ "issues": [{"start": 0.2, "end": 0.4, "severity": "low", "category": "mix", "problem": "hat too bright",+ "fix": "-3 dB shelf at 8 kHz"},+ {"start": 1.5, "end": 2.0, "severity": "HIGH", "category": "realism", "problem": "synthetic riser",+ "fix": "replace with a recorded whoosh"},],+ "strengths": ["clean key foley"], "top_fixes": ["replace the riser"]}+```"""+++def _bursts(seconds=3.0):+ """decaying noise bursts every 0.5 s over a faint floor (a recording-like test signal, made here)"""+ rng = np.random.RandomState(1)+ n = int(seconds * SR)+ x = rng.randn(n) * 1e-3+ for t0 in np.arange(0.1, seconds - 0.2, 0.5):+ i = int(t0 * SR)+ k = np.arange(int(0.15 * SR))+ x[i:i + len(k)] += rng.randn(len(k)) * np.exp(-k / (0.02 * SR)) * 0.4+ return x.astype(np.float32)+++class Resp:+ def __init__(self, status=200, payload=None, headers=None):+ self.status_code, self.payload, self.headers = status, payload or {}, headers or {}+ self.text = json.dumps(self.payload)+ self.ok = status < 400++ def json(self):+ return self.payload+++class FakeRequests:+ """stands in for the requests module: records calls, replays queued responses, never touches the network"""+ class RequestException(Exception):+ pass++ class Timeout(RequestException):+ pass++ def __init__(self, posts=(), gets=()):+ self.posts, self.gets, self.calls = list(posts), list(gets), []++ def post(self, url, headers=None, timeout=None, **kw):+ self.calls.append(('POST', url, dict(headers or {}), kw))+ r = self.posts.pop(0)+ if isinstance(r, Exception):+ raise r+ return r++ def get(self, url, headers=None, timeout=None):+ self.calls.append(('GET', url, dict(headers or {}), {}))+ return self.gets.pop(0)++ def delete(self, url, headers=None, timeout=None):+ self.calls.append(('DELETE', url, dict(headers or {}), {}))+ return Resp(200)+++def _answer(text=ANSWER):+ return Resp(200, {'candidates': [{'content': {'parts': [{'text': text}]}, 'finishReason': 'STOP'}],+ 'usageMetadata': {'promptTokenCount': 900, 'candidatesTokenCount': 300, 'totalTokenCount': 1200},+ 'modelVersion': 'gemini-3.8-flash'})+++def fake(monkeypatch):+ def install(*posts, gets=()):+ f = FakeRequests(posts, gets)+ monkeypatch.setattr(llm, '_requests', lambda: f)+ monkeypatch.setattr(llm.time, 'sleep', lambda s: f.calls.append(('SLEEP', s, {}, {})))+ return f+ return install+++def keyed(monkeypatch):+ monkeypatch.delenv('GOOGLE_API_KEY', raising=False)+ monkeypatch.delenv('EARSHOT_GEMINI_MODEL', raising=False)+ monkeypatch.setenv('GEMINI_API_KEY', KEY)+++# ---------------------------------------------------------------------------------------------------------- llm+def test_llm_off_without_key(monkeypatch, wav, fake):+ monkeypatch.delenv('GEMINI_API_KEY', raising=False)+ monkeypatch.delenv('GOOGLE_API_KEY', raising=False)+ f = fake()+ r = llm.critique(wav(_bursts()), consent=True)+ assert r['enabled'] is False and r['ok'] is False+ assert 'GEMINI_API_KEY' in r['summary'] and '--yes-upload' in r['text']+ assert f.calls == []+++def test_llm_unexpanded_config_placeholder_is_no_key(monkeypatch, wav, fake):+ monkeypatch.setenv('GEMINI_API_KEY', '${GEMINI_API_KEY:-}')+ monkeypatch.delenv('GOOGLE_API_KEY', raising=False)+ f = fake()+ r = llm.critique(wav(_bursts()), consent=True)+ assert r['enabled'] is False and 'no GEMINI_API_KEY' in r['summary'] and f.calls == []+++def test_llm_needs_consent(keyed, wav, fake):+ f = fake()+ r = llm.critique(wav(_bursts()))+ assert r['enabled'] is False and 'consent' in r['summary'] and '--yes-upload' in r['text']+ assert f.calls == [] and KEY not in to_json(r)+++def test_llm_inline_round_trip(keyed, wav, fake):+ f = fake(_answer())+ r = llm.critique(wav(_bursts(3.0)), consent=True, start=0.5)+ assert r['ok'] and r['uploaded']['method'] == 'inline' and abs(r['uploaded']['seconds'] - 2.5) < 0.01+ method, url, headers, kw = f.calls[0]+ assert url.endswith('/v1beta/models/gemini-3.8-flash:generateContent') and KEY not in url+ assert headers['x-goog-api-key'] == KEY+ part, prompt = kw['json']['contents'][0]['parts']+ mp3 = base64.b64decode(part['inline_data']['data'])+ assert part['inline_data']['mime_type'] == 'audio/mp3' and (mp3[:3] == b'ID3' or mp3[0] == 0xFF)+ assert 'Reply with ONLY this JSON' in prompt['text'] and 'LUFS' in prompt['text']+ assert kw['json']['generationConfig']['responseMimeType'] == 'application/json'+ c = r['critique']+ assert c['scores']['realism'] == 5.0 and c['scores']['sync'] is None and c['overall']['score'] == 6.0+ assert c['issues'][0]['severity'] == 'high' and c['issues'][0]['start'] == 2.0 # clip time 1.5 + start 0.5+ assert c['synthetic_or_cheap'][0]['start'] == 2.0+ assert r['text'].startswith(r['summary']) and 'replace the riser' in r['summary']+ assert KEY not in to_json(r)+++def test_llm_question_is_answered_in_json(keyed, wav, fake):+ f = fake(_answer('{"answer": "Yes, about 100 ms late.", "overall": {"score": 7, "verdict": "ok"}}'))+ r = llm.critique(wav(_bursts()), prompt='Is the hit at 1.1 s late?', consent=True)+ prompt = f.calls[0][3]['json']['contents'][0]['parts'][1]['text']+ assert 'Is the hit at 1.1 s late?' in prompt and '"answer"' in prompt+ assert r['critique']['answer'].startswith('Yes') and 'answer: Yes' in r['text']+++def test_llm_video_sends_audio_plus_cut_times(keyed, tmp_path, fake, monkeypatch):+ import subprocess+ vid = str(tmp_path / 'cut.mp4') # black 1 s -> white 1 s (one hard cut at 1.0 s) with a quiet tone+ subprocess.run(['ffmpeg', '-v', 'error', '-y', '-f', 'lavfi', '-i', 'color=c=black:s=64x64:r=25:d=1', '-f', 'lavfi',+ '-i', 'color=c=white:s=64x64:r=25:d=1', '-f', 'lavfi', '-i', 'sine=f=440:d=2', '-filter_complex',+ '[0:v][1:v]concat=n=2:v=1[v]', '-map', '[v]', '-map', '2:a', '-shortest', vid], check=True)+ import earshot.timeline as tl+ monkeypatch.setattr(tl, 'timeline', lambda path, **kw: {'text': f'TRANSCRIPT {kw["start"]}-{kw["end"]}'})+ f = fake(_answer(), _answer())+ llm.critique(vid, consent=True)+ part, prompt = f.calls[0][3]['json']['contents'][0]['parts']+ assert 'Picture cuts' in prompt['text'] and ': 1.' in prompt['text']+ mp3 = base64.b64decode(part['inline_data']['data'])+ assert mp3[:3] == b'ID3' or mp3[0] == 0xFF+ llm.critique(vid, consent=True, with_timeline=True)+ prompt = [c for c in f.calls if c[0] == 'POST'][1][3]['json']['contents'][0]['parts'][1]['text']+ assert 'TRANSCRIPT 0.0-2.0' in prompt and 'Picture cuts' not in prompt # the transcript already has the cuts++ def broken(path, **kw):+ raise IndexError('pop index out of range')+ monkeypatch.setattr(tl, 'timeline', broken)+ fake(_answer('{"overall": {"score": 6, "verdict": "ok"}}'))+ r = llm.critique(vid, consent=True, with_timeline=True)+ assert 'timeline context was NOT sent' in r['text'] and 'rates the soundtrack 6/10: ok. 0 issues' in r['summary']+++def test_llm_files_api_above_inline_limit(keyed, wav, fake, monkeypatch):+ monkeypatch.setattr(llm, 'INLINE_MAX_B64', 16)+ f = fake(Resp(200, {}, {'x-goog-upload-url': 'https://upload.test/u1'}),+ Resp(200, {'file': {'name': 'files/abc', 'uri': 'https://g.test/v1beta/files/abc',+ 'state': 'PROCESSING'}}),+ _answer(), gets=[Resp(200, {'name': 'files/abc', 'uri': 'https://g.test/v1beta/files/abc',+ 'state': 'ACTIVE'})])+ r = llm.critique(wav(_bursts()), consent=True)+ kinds = [(c[0], c[1].rsplit('/', 1)[-1]) for c in f.calls if c[0] != 'SLEEP']+ assert kinds == [('POST', 'files'), ('POST', 'u1'), ('GET', 'abc'), ('POST', 'gemini-3.8-flash:generateContent'),+ ('DELETE', 'abc')]+ start = f.calls[0]+ assert start[2]['X-Goog-Upload-Protocol'] == 'resumable' and start[2]['x-goog-api-key'] == KEY+ assert start[3]['json']['file']['display_name'] == 'earshot-review' # the user's file name is not sent+ gen = [c for c in f.calls if c[0] == 'POST' and c[1].endswith(':generateContent')][0]+ assert gen[3]['json']['contents'][0]['parts'][0]['file_data']['file_uri'].endswith('files/abc')+ assert r['ok'] and r['uploaded']['method'] == 'files_api' and r['uploaded']['files_api_deleted'] is True+++def test_llm_files_api_copy_deleted_when_processing_fails(keyed, wav, fake, monkeypatch):+ monkeypatch.setattr(llm, 'INLINE_MAX_B64', 16)+ f = fake(Resp(200, {}, {'x-goog-upload-url': 'https://upload.test/u1'}),+ Resp(200, {'file': {'name': 'files/abc', 'state': 'PROCESSING'}}),+ gets=[Resp(200, {'name': 'files/abc', 'state': 'FAILED'})])+ r = llm.critique(wav(_bursts()), consent=True)+ assert [c[0] for c in f.calls if c[0] != 'SLEEP'] == ['POST', 'POST', 'GET', 'DELETE']+ assert r['ok'] is False and 'FAILED' in r['summary'] and 'uploaded copy was deleted' in r['summary']+ assert r['text'] == r['summary']+++def test_llm_http_errors_are_readable(keyed, wav, fake):+ path = wav(_bursts())+ fake(Resp(404, {'error': {'message': 'models/gemini-x is not found'}}))+ r = llm.critique(path, consent=True, model='gemini-x')+ assert r['ok'] is False and 'HTTP 404' in r['summary'] and 'EARSHOT_GEMINI_MODEL' in r['summary']+ f = fake(Resp(429, {'error': {'message': 'quota'}}), _answer())+ r = llm.critique(path, consent=True)+ assert r['ok'] and [c[1] for c in f.calls if c[0] == 'SLEEP'] == [llm.RETRY_WAIT[0]]+ fake(Resp(200, {'promptFeedback': {'blockReason': 'SAFETY'}}))+ r = llm.critique(path, consent=True)+ assert r['ok'] is False and 'SAFETY' in r['summary'] and r['text'].startswith('LLM critique failed')+ bad = Resp(200)+ bad.json, bad.text = (lambda: json.loads('<html>')), '<html>proxy</html>'+ fake(bad)+ r = llm.critique(path, consent=True)+ assert r['ok'] is False and 'not JSON' in r['summary'] and 'proxy' in r['summary']+++def test_llm_network_failures_are_readable(keyed, wav, fake):+ path = wav(_bursts())+ err = FakeRequests.RequestException(f'connection refused for key {KEY}')+ f = fake(err, err, err)+ r = llm.critique(path, consent=True)+ assert r['ok'] is False and 'network error' in r['summary'] and len([c for c in f.calls if c[0] == 'SLEEP']) == 2+ f = fake(FakeRequests.Timeout())+ r = llm.critique(path, consent=True)+ assert r['ok'] is False and 'did not answer within' in r['summary'] and len(f.calls) == 1+ assert KEY not in to_json(r)+++def test_llm_unparseable_answer_keeps_raw(keyed, wav, fake):+ fake(_answer('I think the mix is fine overall.'))+ r = llm.critique(wav(_bursts()), consent=True)+ assert r['ok'] is False and r['enabled'] and r['raw'].startswith('I think') and 'parseable' in r['summary']+++def test_llm_model_from_env(keyed, wav, fake, monkeypatch):+ monkeypatch.setenv('EARSHOT_GEMINI_MODEL', 'gemini-test-model')+ f = fake(_answer())+ llm.critique(wav(_bursts()), consent=True)+ assert ':generateContent' in f.calls[0][1] and 'gemini-test-model' in f.calls[0][1]+++def test_parse_json_and_normalize():+ assert llm.parse_json('```json\n{"a": [1, 2,],}\n```') == {'a': [1, 2]}+ assert llm.parse_json('Here you go: {"x": "a } b", "y": {"z": 1}} hope it helps') == {'x': 'a } b', 'y': {'z': 1}}+ assert llm.parse_json('no json at all') is None+ assert [llm._sec(v) for v in ('1:02.5', 12, '3.5s', 'soon', True)] == [62.5, 12.0, 3.5, None, None]+ n = llm.normalize({'overall': 8, 'scores': {'realism': '9/10', 'sync': 14},+ 'issues': [{'start': 5, 'severity': 'low'}, {'start': 9, 'severity': 'bogus'},+ {'start': 99, 'end': 1, 'severity': 'high'}]}, offset=10, clip_len=20)+ assert n['overall']['score'] == 8.0 and n['scores']['realism'] == 9.0 and n['scores']['sync'] == 10.0+ got = [(i['severity'], i['start'], i['end']) for i in n['issues']]+ assert got == [('high', 30.0, 30.0), ('medium', 19.0, 19.0), ('low', 15.0, 15.0)]+++def test_encode_trims_and_is_small(wav, tmp_path):+ data, mime, info = llm.encode(wav(_bursts(5.0)), start=1.0, max_seconds=2.0)+ assert info['trimmed'] and info['start'] == 1.0 and info['seconds'] == 2.0 and mime == 'audio/mp3'+ p = tmp_path / 'clip.mp3'+ p.write_bytes(data)+ from earshot import audio+ pr = audio.probe(str(p))+ assert pr['channels'] == 1 and abs(pr['duration'] - 2.0) < 0.1 and len(data) < 2.0 * 64000 / 8 * 1.3+++def test_llm_cli_explains_how_to_enable(monkeypatch, wav, capsys):+ monkeypatch.delenv('GEMINI_API_KEY', raising=False)+ monkeypatch.delenv('GOOGLE_API_KEY', raising=False)+ assert cli.main(['llm', wav(_bursts())]) == 1+ out = capsys.readouterr().out+ assert 'GEMINI_API_KEY' in out and 'aistudio.google.com' in out+++# ---------------------------------------------------------------------------------------------------------- mcp+def _json_part(text):+ return json.loads(text.split('JSON (compact): ', 1)[1])+++def test_mcp_measure_tool_summary_first(wav):+ text = mcp_server.tool_measure(wav(_bursts(3.0)), light=True)+ first = text.splitlines()[0]+ assert first.startswith('3.000 s, ') and 'LUFS integrated' in first and 'dBTP' in first+ j = _json_part(text)+ assert 'true_peak_dbtp' in j and 'summary' not in j+++def test_measure_summary_wording():+ m = {'duration': 0.25, 'rms_dbfs': -20.0, 'true_peak_dbtp': 0.4, 'clipped_samples': 12, 'noise_floor_dbfs': -120.0,+ 'centroid_hz': 880, 'attack_ms': 0.0, 'decay_ms': 5.0, 'onset_count': 1, 'digital_silence_frac': 0.4}+ s = mcp_server.measure_summary(m)+ assert s.startswith('0.250 s, RMS -20.0 dBFS (too short for LUFS), true peak 0.4 dBTP, 12 clipped samples')+ assert 'attack 0 ms, decay 5 ms' in s and '40.0% exact digital zeros' in s and 'weak evidence' in s+ assert 'code-generated' not in s+ assert mcp_server.measure_summary({'duration': 0.001, 'error': 'too short'}) == '0.001 s: too short.'+ silent = mcp_server.measure_summary({'duration': 10.0, 'peak_dbfs': -240.0, 'lufs': -70.0, 'true_peak_dbtp': -240.0,+ 'noise_floor_dbfs': -200.0, 'digital_silence_frac': 1.0, 'attack_ms': 0.0})+ assert silent.startswith('10.000 s, digitally silent (peak -240 dBFS): nothing to measure') and 'LUFS' not in silent+++def test_mcp_measure_tool_on_digital_silence(wav):+ first = mcp_server.tool_measure(wav(np.zeros(SR * 2, np.float32)), light=True).splitlines()[0]+ assert first.startswith('2.000 s, digitally silent') and 'kHz' not in first+++def test_mcp_errors_are_readable(tmp_path):+ with pytest.raises(mcp_server.ToolFailure, match='not found'):+ mcp_server.tool_measure(str(tmp_path / 'missing.wav'))+ with pytest.raises(mcp_server.ToolFailure, match='folder'):+ mcp_server.tool_measure(str(tmp_path))+ with pytest.raises(mcp_server.ToolFailure, match='no audio/video files'):+ mcp_server.tool_rank([str(tmp_path)], 'a door slam')+ good = tmp_path / 'a.wav'+ good.write_bytes(b'RIFF')+ assert mcp_server._many(str(tmp_path)) == [str(good)] # one string is fine+ with pytest.raises(mcp_server.ToolFailure, match=r'no audio/video files found in .*typo_folder'):+ mcp_server._many([str(tmp_path), str(tmp_path / 'typo_folder')], 'refs')+ with pytest.raises(mcp_server.ToolFailure, match='not part of this install yet. Tools with their module present: '):+ mcp_server._run('nonexistent_tool_xyz', {})+++def test_mcp_module_that_fails_to_import_is_reported(monkeypatch):+ real = mcp_server.importlib.import_module++ def broken(name):+ if name == 'earshot.describe':+ raise SyntaxError('unterminated string literal')+ return real(name)+ monkeypatch.setattr(mcp_server.importlib, 'import_module', broken)+ with pytest.raises(mcp_server.ToolFailure, match=r'could not be loaded \(SyntaxError: unterminated'):+ mcp_server._api('describe')+++def test_mcp_tool_exception_becomes_readable_failure(wav):+ def boom(path):+ raise ValueError('decoder exploded')+ with pytest.raises(mcp_server.ToolFailure, match='describe failed: ValueError: decoder exploded'):+ mcp_server._run('describe', {'path': wav(_bursts())}, fn=boom)+++def test_mcp_call_passes_only_supported_kwargs():+ def f(path, top=3):+ return {'summary': f'{path} {top}'}+ r, ignored = mcp_server._call(f, {'path': 'x', 'top': None, 'win': 2.0})+ assert r['summary'] == 'x 3' and ignored == ['win']+ text = mcp_server._run('fake', {'path': 'x', 'win': 2.0}, fn=f)+ assert text.startswith('x 3') and 'ignored: win' in text+++def test_mcp_compact_json_is_bounded():+ r = {'summary': 's', 'text': 't' * 50, 'curve': list(np.linspace(0, 1, 5000)),+ 'events': [{'t': i / 7, 'label': 'x' * 200} for i in range(500)]}+ s = mcp_server.compact(r)+ assert len(s) <= mcp_server.JSON_MAX + 60 and '"summary"' not in s and 'more' in s+ out = mcp_server._format(r)+ assert out.startswith('s\n\nttt') and 'JSON (compact): ' in out+ wide = json.loads(mcp_server.compact({'a': {f'k{i}': 'x' * 100 for i in range(200)}, 'b': 1})) # valid JSON+ assert wide['b'] == 1 and wide['cut_fields'] == ['a'] and 'a' not in wide+++def test_mcp_sheet_returns_png_path_and_image(wav, tmp_path, monkeypatch):+ from PIL import Image++ def fake_sheet(path, out_png, video=None, detail='normal'):+ Image.new('RGB', (2400, 900), (20, 30, 40)).save(out_png)+ return {'summary': f'sheet drawn ({detail})', 'png': out_png}+ monkeypatch.setattr(mcp_server, '_api', lambda name: fake_sheet)+ out = str(tmp_path / 'sheet.png')+ text, img = mcp_server.tool_sheet(wav(_bursts()), out_png=out, detail='low')+ assert text.startswith(f'PNG: {out}') and 'sheet drawn (low)' in text and 'ignored' not in text+ with pytest.raises(mcp_server.ToolFailure, match='detail must be'):+ mcp_server.tool_sheet(wav(_bursts()), detail='max')+ import io+ im = Image.open(io.BytesIO(img))+ assert max(im.size) == mcp_server.IMG_INLINE_MAX and im.format == 'JPEG'+ Server, Img, ToolError, _ = mcp_server._sdk()+ res = mcp_server._adapt(mcp_server.tool_sheet, Img, ToolError)(path=wav(_bursts()), out_png=out)+ assert isinstance(res[0], str) and isinstance(res[1], Img)+ assert mcp_server.tool_sheet(wav(_bursts()), out_png=out, include_image=False).startswith('PNG: ')+++def test_mcp_realness_folder_rows_and_missing_model(wav, tmp_path, monkeypatch):+ import types+ a, b = wav(_bursts(), 'a.wav'), wav(_bursts(2.0), 'b.wav')+ seen = {}++ def many(paths, ast=None, start=None, end=None):+ seen['range'] = (start, end)+ return {'summary': f'{len(paths)} files judged', 'files': [+ {'path': p, 'p_real': 0.91, 'verdict': 'real or pro-produced', 'why': 'a continuous noise floor'}+ for p in paths]}+ mod = types.SimpleNamespace(MODEL_PATH=a, realness_many=many)+ monkeypatch.setitem(sys.modules, 'earshot.realness', mod)+ monkeypatch.setattr(mcp_server, '_api', lambda name: lambda path, start=None, end=None, ast=None: {'summary': '1'})+ text = mcp_server.tool_realness(str(tmp_path))+ assert text.startswith('2 files judged') and '0.91 real or pro-produced' in text and 'b.wav: a continuous' in text+ assert mcp_server.tool_realness(b).startswith('1')+ mcp_server.tool_realness(str(tmp_path), start=1.0) # a range applies to every file of a folder+ assert seen['range'] == (1.0, None)+ with pytest.raises(mcp_server.ToolFailure, match='not found'):+ mcp_server.tool_realness(str(tmp_path / 'nope.wav'))+ mod.MODEL_PATH = str(tmp_path / 'realness_model.json')+ with pytest.raises(mcp_server.ToolFailure, match='NOT checked') as e:+ mcp_server.tool_realness(a)+ assert 'train' not in str(e.value)+++def test_mcp_llm_tool_without_key(monkeypatch, wav):+ monkeypatch.delenv('GEMINI_API_KEY', raising=False)+ monkeypatch.delenv('GOOGLE_API_KEY', raising=False)+ text = mcp_server.tool_llm_critique(wav(_bursts()), consent=True)+ assert text.startswith('LLM critique is off') and 'GEMINI_API_KEY' in text+++def test_mcp_server_registers_all_tools():+ import anyio+ srv = mcp_server.build()+ tools = anyio.run(srv.list_tools)+ names = {t.name for t in tools}+ assert names == {'measure', 'describe', 'match', 'rank', 'find', 'compare', 'ab', 'realness', 'timeline', 'sheet',+ 'critique', 'doctor', 'llm_critique'}+ for t in tools:+ schema = getattr(t, 'input_schema', None) or t.inputSchema+ assert len(t.description) > 120 and 'Use ' in t.description+ assert all(p.get('description') for p in schema['properties'].values()), t.name+ llm_tool = next(t for t in tools if t.name == 'llm_critique')+ ann = llm_tool.annotations.model_dump() # snake_case in mcp 2.x+ assert ann.get('read_only_hint', ann.get('readOnlyHint')) is False+ assert ann.get('open_world_hint', ann.get('openWorldHint')) is True+++async def _stdio_session(calls, log, *args):+ """start `python -m earshot mcp` (or `python -m <args>`) as a subprocess and run calls(session) over stdio"""+ import anyio+ from mcp import ClientSession, StdioServerParameters, stdio_client+ params = StdioServerParameters(command=sys.executable, args=['-m', *(args or ('earshot', 'mcp'))], cwd=REPO,+ env={'EARSHOT_THREADS': '2', 'PYTHONIOENCODING': 'utf-8'})+ with anyio.fail_after(900), open(log, 'w', encoding='utf-8') as errlog:+ async with stdio_client(params, errlog=errlog) as (rd, wr):+ async with ClientSession(rd, wr) as s:+ with anyio.fail_after(300): # a server that answers on the wrong stream hangs here+ await s.initialize()+ return await calls(s)+++def _text(res):+ return '\n'.join(getattr(c, 'text', '') for c in res.content)+++def _is_error(res):+ return bool(getattr(res, 'is_error', None) or getattr(res, 'isError', None))+++def test_mcp_stdio_end_to_end(wav, tmp_path):+ """fast round trip over a real stdio connection (doctor imports torch: it is in the slow test below)"""+ import anyio+ path = wav(_bursts(3.0))++ async def calls(s):+ tools = await s.list_tools()+ return (tools, await s.call_tool('measure', {'path': path, 'light': True}),+ await s.call_tool('measure', {'path': path + '.missing'}),+ await s.call_tool('rank', {'paths': path + '.missing', 'query': 'a door'}),+ await s.call_tool('llm_critique', {'path': path}))+ tools, meas, missing, rank, llm_res = anyio.run(_stdio_session, calls, str(tmp_path / 'server.log'))+ assert len(tools.tools) == 13+ assert not _is_error(meas) and _text(meas).startswith('3.000 s, ') and 'dBTP' in _text(meas)+ assert _is_error(missing) and 'not found' in _text(missing)+ assert _is_error(rank) and 'no audio/video files found' in _text(rank) # a plain string passes the schema+ assert not _is_error(llm_res) and 'LLM critique is off' in _text(llm_res)+++def test_mcp_stdio_describe_keeps_models_warm(wav, tmp_path):+ import importlib.util+ import time+ import anyio+ if importlib.util.find_spec('earshot.describe') is None:+ pytest.skip('earshot.describe not built yet')+ p1, p2 = wav(_bursts(3.0), 'a.wav'), wav(_bursts(2.0)[::-1].copy(), 'b.wav')++ async def calls(s):+ doc = await s.call_tool('doctor', {})+ assert not _is_error(doc) and 'earshot' in _text(doc).lower()+ t0 = time.time()+ a = await s.call_tool('describe', {'path': p1})+ t1 = time.time()+ b = await s.call_tool('describe', {'path': p2, 'top': 3})+ t2 = time.time()+ sheet = None+ if importlib.util.find_spec('earshot.sheet'):+ sheet = await s.call_tool('sheet', {'path': p1, 'out_png': str(tmp_path / 'a_sheet.png')})+ return a, b, t1 - t0, t2 - t1, sheet+ a, b, cold, warm, sheet = anyio.run(_stdio_session, calls, str(tmp_path / 'server.log'))+ assert not _is_error(a) and not _is_error(b), _text(a) + _text(b)+ assert len(_text(a)) > 40 and warm < cold # the second file needs inference but no model load+ if sheet is not None:+ kinds = [c.type for c in sheet.content]+ assert not _is_error(sheet) and kinds == ['text', 'image'] and (tmp_path / 'a_sheet.png').is_file()+ assert (getattr(sheet.content[1], 'mime_type', None) or sheet.content[1].mimeType) == 'image/jpeg'+++def test_mcp_stdio_warm_start_answers_and_preloads(wav, tmp_path):+ """--warm loads the models in a thread while the transport starts (this once sent the initialize answer to+ stderr and the client hung); also covers the fast-start entry point python -m earshot.mcp_server"""+ import time+ import anyio+ log = tmp_path / 'server.log'+ path = wav(_bursts(2.0))++ async def calls(s):+ meas = await s.call_tool('measure', {'path': path, 'light': True})+ t0 = time.time()+ while 'models warm' not in log.read_text(encoding='utf-8', errors='replace') and time.time() - t0 < 300:+ await anyio.sleep(1)+ return meas, await s.call_tool('describe', {'path': path})+ meas, desc = anyio.run(_stdio_session, calls, str(log), 'earshot.mcp_server', '--warm')+ assert not _is_error(meas) and not _is_error(desc) and 'models warm' in log.read_text(encoding='utf-8')diff --git a/tests/test_launch.py b/tests/test_launch.pynew file mode 100644index 0000000..e3d99fc--- /dev/null+++ b/tests/test_launch.py@@ -0,0 +1,248 @@+"""Launch profile + critique: detectors on synthetic signals, scorecard rules, end-to-end on generated files."""+import json+import os+import subprocess++import numpy as np+import pytest++from earshot import config, video+from earshot import profile as P+from earshot.critique import SPEC, critique, real_thresholds, scorecard++SR = 48000++[email protected](autouse=True)+def _cache(tmp_path, monkeypatch):+ monkeypatch.setattr(config, 'EMB_CACHE', str(tmp_path / 'emb'))+ monkeypatch.setattr(config, 'DEC_CACHE', str(tmp_path / 'dec'))+++def _bed(sec, level=0.05, seed=0):+ rng = np.random.RandomState(seed)+ return (rng.randn(int(sec * SR)) * level).astype(np.float32)+++def _hit(x, t, amp=0.8, seed=1):+ rng = np.random.RandomState(seed)+ n = int(0.15 * SR)+ i = int(t * SR)+ x[i:i + n] += (rng.randn(n) * amp * np.exp(-np.arange(n) / SR / 0.03)).astype(np.float32)+ return x+++def _soundtrack(sec=12.0, hits=(2.0, 5.0, 8.0), lead=0.0, hole=None):+ x = _bed(sec)+ for t in hits:+ _hit(x, t)+ x = np.stack([x, 0.9 * x + 0.1 * _bed(sec, 0.05, 3)], axis=1)+ if lead:+ x[: int(lead * SR)] = 0+ if hole:+ x[int(hole[0] * SR): int(hole[1] * SR)] = 0+ return x+++def test_salient_hits_and_stopdown(wav):+ x = _soundtrack(hits=(2.0, 5.0, 8.0))+ x[int(4.5 * SR): int(5.0 * SR)] *= 0.01 # 0.5 s stop-down right before the hit at 5.0+ a = P._audio_pass(wav(x))+ hits = P.salient_hits(a['tf'], a['Lf'], a['meas']['lufs'], a['env'], a['e2'])+ assert len(hits) == 3+ for t in (2.0, 5.0, 8.0):+ assert np.min(np.abs(hits - t)) < 0.006+ assert np.min(np.abs(a['onsets'] - t)) < 0.006+ sd = P.stopdowns(a['tf'], a['Lf'])+ assert len(sd) == 1 and abs(sd[0][0] - 4.5) < 0.1 and 0.35 < sd[0][1] < 0.6+++def test_visual_events_relative_threshold():+ fps = 30.0+ d = np.full(300, 0.003, np.float32)+ d[100:140] = 0.02 + 0.005 * np.sin(np.arange(40)) # steady motion: no event+ d[60] = 0.4 # hard cut+ d[200] = 0.05 # small change on a still frame: an event+ ev = video.diff_events(d, fps)+ assert list(ev) == [60, 200]+++def test_near_and_chance():+ ev = [1.0, 2.0, 3.0]+ assert P.near_frac(ev, [1.02, 2.5, 3.09], 0.1) == pytest.approx(2 / 3)+ assert P.near_frac(ev, [], 0.1) == 0.0 and P.near_frac([], [1.0], 0.1) is None+ dense = np.arange(0, 10, 0.05)+ assert P.chance_frac(ev, dense, 0.1, 10.0) == pytest.approx(1.0)+++def test_film_metrics_arc_gap_and_ending(wav):+ x = _soundtrack(sec=12.0, hits=(2.0, 6.0, 11.0), lead=1.0, hole=(4.0, 5.0))+ x[int(11.4 * SR):] = 0+ f = P.film_metrics(wav(x), use_ast=False)+ m = f['metrics']+ assert m['first_sound_s'] == pytest.approx(1.0, abs=0.06)+ assert m['max_gap_s'] == pytest.approx(1.0, abs=0.08)+ assert m['end_silence_s'] == pytest.approx(0.6, abs=0.08)+ assert m['music_frac'] is None and m['cuts_with_sound'] is None+ assert m['hit_rate'] == pytest.approx(5 / 12, abs=0.01) # 3 hits + sound starting at 1.0 and after the hole+ json.dumps(f)+++def _prof(**stats):+ return {'n': 7, 'metrics': {k: {'n': 7, 'min': lo, 'max': hi, 'median': (lo + hi) / 2, 'p10': lo, 'p90': hi}+ for k, (lo, hi) in stats.items()}}+++def test_scorecard_rules():+ prof = _prof(onset_rate=(2.5, 8.0), music_frac=(0.67, 1.0), first_sound_s=(0.0, 0.5))+ vals = {'lufs_i': -14.0, 'true_peak': -0.5, 'onset_rate': 1.0, 'music_frac': 0.3, 'first_sound_s': 0.0}+ rows = {r['metric']: r for r in scorecard(vals, prof)['metrics']}+ assert rows['lufs_i']['status'] == 'ok' and rows['lufs_i']['basis'] == 'web'+ assert rows['true_peak']['status'] == 'high'+ assert {r['metric']: r for r in scorecard(vals, prof, lossy=True)['metrics']}['true_peak']['status'] == 'ok'+ assert rows['onset_rate']['status'] == 'low' and rows['onset_rate']['range'] == [pytest.approx(2.2), pytest.approx(8.3)]+ assert rows['music_frac']['status'] == 'low' and rows['music_frac']['weight'] == 'critical'+ assert rows['first_sound_s']['range'][0] is None and rows['first_sound_s']['status'] == 'ok'+ assert rows['band_sub']['status'] == 'n.a.' and rows['tempo']['weight'] == 'info'+ near = {r['metric']: r for r in scorecard({'lufs_i': -17.5}, prof)['metrics']}['lufs_i']+ far = {r['metric']: r for r in scorecard({'lufs_i': -25.0}, prof)['metrics']}['lufs_i']+ assert near['weight'] == 'major' and far['weight'] == 'critical'+ sc = scorecard({'lufs_i': -14.0, 'onset_rate': 5.0, 'music_frac': 0.9}, prof)+ assert sc['verdict'] == 'launch-grade' and sc['score'] == 1.0+ assert set(SPEC) - {'p_real'} <= set(P.METRICS)+++def test_scorecard_realness_and_density_floor():+ prof = _prof(hit_rate=(0.1, 1.66), music_frac=(0.67, 1.0))+ t = real_thresholds()+ row = lambda vals, k: {r['metric']: r for r in scorecard(vals, prof)['metrics']}[k]+ synth = row({'p_real': 0.3}, 'p_real')+ assert synth['status'] == 'low' and synth['weight'] == 'critical' and synth['range'][0] == t['hi']+ assert scorecard({'p_real': 0.3, 'lufs_i': -14.0, 'music_frac': 1.0}, prof)['verdict'] == 'needs work'+ if t['hi'] > 0.55:+ assert row({'p_real': 0.52}, 'p_real')['weight'] == 'minor'+ assert row({'p_real': 0.99}, 'p_real')['status'] == 'ok'+ floor = row({'hit_rate': 0.0}, 'hit_rate')+ assert floor['status'] == 'low' and floor['range'][0] == pytest.approx(0.05)+++def test_bundled_profile_is_anonymous():+ p = P.load_profile()+ assert p['n'] == 7 and [f['id'] for f in p['films']] == [f'ref_{i}' for i in range(1, 8)]+ s = json.dumps(p)+ assert '.mp4' not in s and 'Downloads' not in s and 'path' not in s+ for k, spec in SPEC.items():+ if spec[0] and k != 'p_real':+ assert p['metrics'][k]['n'] >= 6, k+++def test_critique_end_to_end_audio_only(wav):+ path = wav(_soundtrack(sec=12.0, hits=(2.0, 6.0, 9.0), lead=1.5))+ r = critique(path, use_ast=False, use_realness=False)+ json.dumps(r)+ assert r['verdict'] in ('launch-grade', 'close', 'needs work', 'not launch-grade')+ rows = {m['metric']: m for m in r['metrics']}+ assert rows['first_sound_s']['status'] == 'high'+ assert any(a['metric'] == 'first_sound_s' and '0:01.5' in a['action'] for a in r['actions'])+ assert rows['cuts_with_sound']['status'] == 'n.a.' and 'cuts_with_sound' in r['summary']+ r['text'].encode('ascii')+++def test_critique_refuses_silent_and_tiny(wav):+ for x in (np.zeros(8 * SR, np.float32), _bed(1.0)):+ path = wav(x)+ r = critique(path, use_realness=False)+ assert r['verdict'] == 'n.a.' and not r['actions'] and 'nothing to score' in r['summary']+ assert not os.path.exists(P.cache_paths(path)['audio']) # refused before the librosa pass+++def test_critique_mono_and_short_clip(wav):+ r = critique(wav(_bed(20.0)[:, None].repeat(2, axis=1)), use_ast=False, use_realness=False)+ rows = {m['metric']: m for m in r['metrics']}+ assert rows['side_to_mid_db']['value'] == P.MONO_DB and rows['side_to_mid_db']['status'] == 'low'+ assert any(a['metric'] == 'side_to_mid_db' and 'mono' in a['action'] for a in r['actions'])+ r = critique(wav(_soundtrack(sec=6.0, hits=(2.0, 4.0))), use_ast=False, use_realness=False)+ rows = {m['metric']: m for m in r['metrics']}+ assert all(rows[k]['status'] == 'n.a.' for k in P.SHAPE) and 'clip under 10 s' in r['summary']+ assert r['partial'] == ['realness', 'music presence'] and 'partial' in r['text'].splitlines()[1]+++def test_input_errors(wav, tmp_path, capsys):+ from earshot.cli import main+ a = wav(_soundtrack(sec=8.0, hits=(2.0, 5.0)))+ with pytest.raises(P.InputError, match='file not found'):+ critique(str(tmp_path / 'nope.wav'), use_ast=False, use_realness=False)+ with pytest.raises(P.InputError, match='folder'):+ critique(str(tmp_path), use_ast=False, use_realness=False)+ with pytest.raises(P.InputError, match='has no picture'):+ critique(a, video=wav(_bed(4.0), 'other.wav'), use_ast=False, use_realness=False)+ assert critique(a, video=a, use_ast=False, use_realness=False)['video'] is None # "its own picture": none+ with pytest.raises(P.InputError, match='profile not found'):+ critique(a, profile=str(tmp_path / 'p.json'), use_ast=False, use_realness=False)+ bad = tmp_path / 'bad.json'+ bad.write_text('{"n": 1}')+ with pytest.raises(P.InputError, match='not a profile'):+ critique(a, profile=str(bad), use_ast=False, use_realness=False)+ assert main(['critique', str(tmp_path / 'nope.wav'), '--fast']) == 1+ assert capsys.readouterr().err.strip() == f"earshot critique: error: file not found: {tmp_path / 'nope.wav'}"+++def test_profile_from_two_files(wav, tmp_path):+ a = wav(_soundtrack(sec=10.0, hits=(2.0, 4.0)), 'a.wav')+ b = wav(_soundtrack(sec=10.0, hits=(3.0, 7.0, 8.0)), 'b.wav')+ out = str(tmp_path / 'p.json')+ p = P.profile([a, b], styles=['one', 'two'], use_ast=False, out=out)+ q = json.load(open(out))+ assert q['n'] == 2 and q['films'][1]['style'] == 'two' and q['metrics']['hit_rate']['n'] == 2+ assert 'a.wav' not in json.dumps(q) and p['summary'].startswith('profile of 2')+ r = critique(a, profile=out, use_ast=False, use_realness=False)+ rows = {m['metric']: m for m in r['metrics']}+ assert r['profile_n'] == 2 and rows['music_frac']['status'] == 'n.a.' and rows['onset_rate']['status'] != 'n.a.'+ no_ref = {m['metric']: m for m in scorecard({'cuts_with_sound': 0.1}, q)['metrics']}['cuts_with_sound']+ assert no_ref['status'] == 'n.a.' and no_ref['basis'] == 'no reference data'+ assert any('only 2 reference' in w for w in q['warnings'])+++def test_profile_skips_unusable_files(wav, tmp_path):+ good = wav(_soundtrack(sec=10.0, hits=(2.0, 4.0)), 'good.wav')+ silent = wav(np.zeros(10 * SR, np.float32), 'silent.wav')+ tiny = wav(_bed(0.5), 'tiny.wav')+ p = P.profile([good, silent, tiny, str(tmp_path / 'missing.wav')], use_ast=False)+ assert p['n'] == 1 and p['metrics']['lufs_i']['min'] > -40 and '3 file(s) skipped' in p['warnings'][0]+ with pytest.raises(P.InputError, match='no usable reference films'):+ P.profile([silent, tiny], use_ast=False)+++def test_picture_sync_on_generated_film(tmp_path):+ film = str(tmp_path / 'f.mp4')+ src = ['-f', 'lavfi', '-i', 'color=c=red:s=160x90:r=25:d=2', '-f', 'lavfi', '-i', 'color=c=blue:s=160x90:r=25:d=2',+ '-f', 'lavfi', '-i', 'color=c=white:s=160x90:r=25:d=2']+ aud = ['-f', 'lavfi', '-i', "aevalsrc='0.02*sin(2*PI*220*t)+0.8*exp(-60*mod(t,2))*sin(2*PI*900*t)':s=48000:d=6"]+ flt = '[0:v][1:v][2:v]concat=n=3:v=1:a=0[v]'+ p = subprocess.run(['ffmpeg', '-v', 'error', '-y', *src, *aud, '-filter_complex', flt, '-map', '[v]', '-map', '3:a',+ '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-shortest', film], capture_output=True)+ if p.returncode != 0:+ pytest.skip('ffmpeg cannot write a test mp4 here')+ f = P.film_metrics(film, use_ast=False)+ assert f['lossy'] and f['fps'] == 25.0+ assert [round(t, 1) for t in f['events']['visual']] == [2.0, 4.0]+ assert f['metrics']['cuts_with_sound'] is None # fewer than 3 events: no sync shares+ assert f['metrics']['cut_rate'] == pytest.approx(2 / 6, abs=0.02)+ assert os.path.exists(film)+++def test_cli_critique_json(wav, capsys):+ from earshot.cli import main+ path = wav(_soundtrack(sec=8.0, hits=(2.0, 5.0)))+ main(['critique', path, '--fast', '--json'])+ r = json.loads(capsys.readouterr().out)+ assert r['scored'] > 10 and r['summary'] and r['partial'] == ['realness', 'music presence']+++def test_music_presence_with_ast(wav):+ t = np.arange(int(10 * SR)) / SR+ chord = sum(np.sin(2 * np.pi * f * t) for f in (261.6, 329.6, 392.0)) * 0.1+ env = 0.6 + 0.4 * (np.sin(2 * np.pi * 2 * t) > 0)+ f = P.film_metrics(wav((chord * env).astype(np.float32)), use_ast=True)+ assert f['metrics']['music_frac'] is not None and len(f['events']['music_windows']) == 2diff --git a/tests/test_real_sound.py b/tests/test_real_sound.pynew file mode 100644index 0000000..0cbba04--- /dev/null+++ b/tests/test_real_sound.py@@ -0,0 +1,356 @@+"""skills/real-sound: the scripts run end to end on a synthetic project (test signals only, never committed media).++The project is the design template (templates/design_fragment.json) + the cue template, with synthetic stand-ins at+every file path, so these tests also prove the templates render. A second tiny project carries deliberate defects+(a cut into a playing sine, a masked tick, a partly sounded class of action, a trim on digital silence, a riser that+ends long before its hit) that qa.py must catch. The end-to-end tests start the scripts as subprocesses (mix, qa,+arrange: tens of seconds each on this laptop) and are marked slow; the fast ones run in-process.+"""+import json+import os+import shutil+import subprocess+import sys++import numpy as np+import pytest+import soundfile as sf+from scipy import signal++SR = 48000+SKILL = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'skills', 'real-sound')+SCRIPTS = os.path.join(SKILL, 'scripts')+sys.path.insert(0, SCRIPTS)+import licences # noqa: E402+import variant # noqa: E402++RNG = np.random.RandomState(7)+++def run(script, *args, cwd=None):+ p = subprocess.run([sys.executable, '-I', os.path.join(SCRIPTS, script), *args], capture_output=True, text=True,+ cwd=cwd, timeout=600)+ return p.returncode, p.stdout + p.stderr+++def hits(dur, times, sr=SR, tau=0.012, amp=0.5, floor=1e-4):+ n = int(dur * sr)+ x = RNG.randn(n) * floor+ for t in times:+ i = int(t * sr)+ k = np.arange(min(n - i, int(0.25 * sr)))+ x[i:i + len(k)] += RNG.randn(len(k)) * np.exp(-k / sr / tau) * amp+ return x.astype(np.float32)+++def write(root, rel, x, sr=SR):+ p = os.path.join(root, rel)+ os.makedirs(os.path.dirname(p), exist_ok=True)+ sf.write(p, x, sr, format='WAV', subtype='PCM_16') # WAV bytes; ffmpeg probes content, not the extension+ return p+++def music(dur, sr=22050, start=0.5, bpm=120):+ """a steady test track: a low thump on every downbeat, a tick on every beat, a different chord per bar"""+ n = int(dur * sr)+ t = np.arange(n) / sr+ x = np.zeros(n)+ beat = 60 / bpm+ for k in range(int((dur - start) / beat)):+ i = int((start + k * beat) * sr)+ m = np.arange(min(n - i, int(0.3 * sr)))+ if k % 4 == 0:+ x[i:i + len(m)] += 0.6 * np.sin(2 * np.pi * 55 * m / sr) * np.exp(-m / sr / 0.12)+ x[i:i + len(m)] += 0.15 * RNG.randn(len(m)) * np.exp(-m / sr / 0.01)+ for b in range(int((dur - start) / (4 * beat))):+ a0, a1 = int((start + 4 * beat * b) * sr), int((start + 4 * beat * (b + 1)) * sr)+ for f in ([220, 277, 330], [196, 247, 294], [175, 220, 262], [165, 208, 247])[b % 4]:+ x[a0:a1] += 0.05 * np.sin(2 * np.pi * f * t[a0:a1])+ return x.astype(np.float32), sr++[email protected](scope='module')+def project(tmp_path_factory):+ """the design template + cue template with synthetic recordings at every path, mixed once"""+ root = str(tmp_path_factory.mktemp('rs_project'))+ write(root, 'audio/assets/ui/mk-0000_mouse-click.wav', hits(0.3, [0.1, 0.185]))+ write(root, 'audio/assets/keyboard/fs-000001_mech-spacebar.ogg', hits(1.0, [0.5], tau=0.03))+ write(root, 'audio/assets/motion/fs-000002_stick-swish.ogg',+ (RNG.randn(int(0.6 * SR)) * np.hanning(int(0.6 * SR)) * 0.3).astype(np.float32))+ write(root, 'audio/assets/ui/fs-000003_cork-pop.ogg', hits(0.3, [0.03], tau=0.02))+ write(root, 'audio/assets/motion/fs-000004_cymbal-roll.ogg',+ (RNG.randn(int(2.5 * SR)) * np.linspace(0.02, 0.3, int(2.5 * SR))).astype(np.float32))+ write(root, 'audio/assets/impacts/mk-0005_deep-drum-impact.wav', hits(2.0, [0.0], tau=0.25, amp=0.8))+ write(root, 'audio/assets/impacts/fs-000006_wood-slam.ogg', hits(0.5, [0.02], tau=0.04))+ tt = np.arange(3 * SR) / SR+ write(root, 'audio/assets/magic/fs-000007_celesta-E5.ogg',+ (0.4 * np.sin(2 * np.pi * 659.3 * tt) * np.exp(-tt / 0.5)).astype(np.float32))+ write(root, 'audio/assets/keyboard/fs-000008_mech-typing.ogg', hits(10.0, np.arange(0.6, 9.5, 0.17), tau=0.01))+ write(root, 'audio/assets/ambience/fs-000009_quiet-office-roomtone.ogg',+ (RNG.randn(46 * 8000) * 0.01).astype(np.float32), sr=8000)+ m, msr = music(40.0)+ write(root, 'audio/assets/music/sb-0000_bright-launch-track.mp3', m, sr=msr)+ os.makedirs(os.path.join(root, 'out'), exist_ok=True)+ shutil.copy(os.path.join(SKILL, 'templates', 'cues.json'), os.path.join(root, 'out', 'cues.json'))+ os.makedirs(os.path.join(root, 'audio'), exist_ok=True)+ shutil.copy(os.path.join(SKILL, 'templates', 'design_fragment.json'), os.path.join(root, 'audio', 'design.json'))+ code, out = run('mix.py', '--root', root, '--design', 'audio/design.json', '--trace', 'out/qa/adds.json')+ assert code == 0, out+ return root+++def test_template_mix_renders(project):+ rep = json.load(open(os.path.join(project, 'out/qa/mix_report.json')))+ cues = json.load(open(os.path.join(project, 'out/cues.json')))+ assert rep['unsounded_events'] == []+ assert len(rep['placements']) == 24 # 23 rule placements on 19 events (layered) + 1 free placement+ x, sr = sf.read(os.path.join(project, 'out/audio/soundtrack.wav'))+ assert sr == SR and abs(len(x) / SR - cues['total'] / cues['fps']) < 1e-3+ master = json.load(open(os.path.join(project, 'out/qa/master.json')))+ assert abs(master['lufs'] - (-14)) <= 0.5+ trace = json.load(open(os.path.join(project, 'out/qa/adds.json')))+ assert len([a for a in trace if 'sample' in a]) == len(rep['placements'])+ assert 'audio/assets/music/sb-0000_bright-launch-track.mp3' in rep['files_used']+++def test_solo_and_dump(project, tmp_path):+ out = str(tmp_path / 'solo' / 'design.wav')+ code, txt = run('mix.py', '--root', project, '--design', 'audio/design.json', '--only', 'design', '--out', out)+ assert code == 0, txt+ assert os.path.exists(out) # the unmastered solo mix is written to --out itself+ code, txt = run('mix.py', '--root', project, '--design', 'audio/design.json', '--dump', str(tmp_path / 'dump'))+ assert code == 0, txt+ rows = json.load(open(tmp_path / 'dump' / '_samples.json'))+ names = {r.get('name') for r in rows}+ assert 'ui.press' in names and any(n and n.startswith('kb.type#') for n in names)+++def test_qa_on_template(project):+ code, txt = run('qa.py', '--root', project, '--design', 'audio/design.json', '--out', 'out/qa/qa.json')+ res = json.load(open(os.path.join(project, 'out/qa/qa.json')))+ for k in ('clicks', 'loudness', 'coverage', 'edges', 'audibility', 'harshness', 'sync', 'density'):+ assert k in res['sections'] and res['sections'][k]['summary']+ assert res['sections']['coverage']['status'] == 'ok'+ assert res['sections']['loudness']['lufs'] == pytest.approx(-14, abs=0.5)+ assert txt.startswith('QA ')+++def test_qa_catches_defects(tmp_path):+ root = str(tmp_path)+ t = np.arange(3 * SR) / SR+ write(root, 'a/sine.wav', (0.3 * np.sin(2 * np.pi * 880 * t)).astype(np.float32))+ write(root, 'a/tick.wav', hits(0.3, [0.05], amp=0.5))+ write(root, 'a/silence.wav', np.zeros(SR, np.float32))+ ramp = RNG.randn(int(2.6 * SR)) * np.r_[np.linspace(0.01, 0.4, 2 * SR), np.full(int(0.6 * SR), 0.004)]+ write(root, 'a/riser.wav', ramp.astype(np.float32)) # its body ends 0.6 s before the file does+ bed = signal.sosfilt(signal.butter(2, 2000, 'low', fs=SR, output='sos'), RNG.randn(4 * SR)) * 0.3+ write(root, 'a/bed.wav', bed.astype(np.float32)) # a dark bed: masks the quiet tick, leaves the click audible+ cues = {'fps': 60, 'total': 240, 'events': [{'f': 60, 'kind': 'pop', 'id': 't:cut'},+ {'f': 120, 'kind': 'tick', 'id': 't:quiet'},+ {'f': 150, 'kind': 'land', 'id': 't:step:0'},+ {'f': 180, 'kind': 'land', 'id': 't:step:1'},+ {'f': 200, 'kind': 'boop', 'id': 't:silent'},+ {'f': 228, 'kind': 'riser', 'id': 't:riser'}]}+ design = {'duration': 4.0, 'samples': {+ 'cut': {'file': 'a/sine.wav', 'start': 1.0003, 'end': 1.3, 'fade_in': 0, 'fade_out': 0.05},+ 'quiet': {'file': 'a/tick.wav', 'norm_peak_db': -60},+ 'step': {'file': 'a/tick.wav', 'norm_peak_db': -6},+ 'sil': {'file': 'a/silence.wav', 'start': 0.2, 'end': 0.5},+ 'rise': {'file': 'a/riser.wav', 'fade_out': 0.01}},+ 'pools': {'cut': ['cut'], 'quiet': ['quiet'], 'step': ['step'], 'sil': ['sil'], 'rise': ['rise']},+ 'rules': [{'name': 'cut', 'match': {'kind': 'pop'}, 'pool': 'cut', 'mode': 'fixed'},+ {'name': 'quiet', 'match': {'kind': 'tick'}, 'pool': 'quiet', 'mode': 'fixed'},+ {'name': 'step', 'match': {'kind': 'land', 'to': 2.9}, 'pool': 'step', 'mode': 'fixed'},+ {'name': 'sil', 'match': {'kind': 'boop'}, 'pool': 'sil', 'mode': 'fixed'},+ {'name': 'rise', 'match': {'kind': 'riser'}, 'pool': 'rise', 'mode': 'fixed', 'align': 'end',+ 'gain_db': -12}],+ 'ambience': [{'file': 'a/bed.wav', 'at': 0, 'until': 4.0, 'fade_in': 0.2, 'fade_out': 0.2}],+ 'master': {'lufs': -14}}+ os.makedirs(os.path.join(root, 'out'), exist_ok=True)+ json.dump(cues, open(os.path.join(root, 'out/cues.json'), 'w'))+ json.dump(design, open(os.path.join(root, 'd.json'), 'w'))+ code, txt = run('mix.py', '--root', root, '--design', 'd.json', '--trace', 'out/qa/adds.json')+ assert code == 0, txt+ code, txt = run('qa.py', '--root', root, '--design', 'd.json', '--out', 'out/qa/qa.json')+ s = json.load(open(os.path.join(root, 'out/qa/qa.json')))['sections']+ cut = [c for c in s['clicks']['items'] if c['owners'] and c['owners'][0].get('what') == 'cut']+ assert cut and cut[0]['owners'][0]['where'] == 'start' # the cut into a playing sine clicks, at its start+ assert s['clicks']['status'] == 'fail' and code == 1+ edge = [it for it in s['edges']['items'] if it['name'] == 'cut']+ assert edge and any('already playing' in m for m in edge[0]['issues'])+ quiet = [r for r in s['audibility']['events'] if r['label'] == 't:quiet']+ assert quiet and quiet[0]['verdict'] == 'masked'+ assert s['coverage']['status'] == 'warn' and s['coverage']['partial'][0]['id'] == 't:step'+ silent = [r for r in s['audibility']['events'] if r['label'] == 't:silent']+ assert silent and silent[0]['verdict'] == 'silent' and s['audibility']['status'] == 'fail'+ assert [q['id'] for q in s['coverage']['placed_but_silent']] == ['t:silent']+ riser = [r for r in s['sync']['items'] if r['label'] == 't:riser']+ assert riser and riser[0]['test'] == 'end' and riser[0]['end_ms'] < -500 # the riser placed by its end is checked+ code, txt = run('qa.py', '--root', root, '--design', 'd.json', '--report', 'out/nope.json')+ assert code == 2 and 'mix report not found' in txt and 'Traceback' not in txt+++def test_licence_classes():+ c = licences.classify+ assert c('CC0 1.0')['status'] == 'allowed' and c('Creative Commons 0')['key'] == 'cc0'+ assert c('CC BY 4.0')['key'] == 'by4' and c('Attribution 4.0')['attribution']+ assert c('Mixkit Sound Effects Free License')['key'] == 'mixkit-sfx'+ assert c('Mixkit Stock Music Free License')['key'] == 'mixkit-music'+ assert c('Pixabay Content License')['status'] == 'allowed'+ for bad in ('CC BY-NC 4.0', 'CC BY-ND 4.0', 'CC BY-SA 4.0', 'Sampling+ 1.0', 'personal use only',+ 'DISPUTED: chosic says CC BY 3.0; artist says custom'):+ assert c(bad)['status'] == 'forbidden', bad+ assert c('Royalty free')['status'] == 'unknown' and c('')['status'] == 'unknown'+ assert c('Creative Commons Attribution 4.0 International (copyrighted by the author)')['key'] == 'by4'+ assert c('All rights reserved')['status'] == 'forbidden' and c('CC BY 4.0, all rights reserved')['status'] == 'unknown'+++def manifests(root):+ m = os.path.join(root, 'audio/assets/manifests')+ os.makedirs(m, exist_ok=True)+ ents = [+ {'id': 'fs-111', 'category': 'ui', 'file': 'a/tick.wav', 'title': 'Tick', 'author': 'alice', 'license': 'CC BY 4.0',+ 'page_url': 'https://freesound.org/people/alice/sounds/111/',+ 'download_url': 'https://cdn.freesound.org/previews/0/111_42-hq.ogg',+ 'attribution': '"Tick" by alice (https://freesound.org/s/111/), licensed under CC BY 4.0'},+ {'id': 'fs-222', 'category': 'ui', 'file': 'a/sine.wav', 'title': 'Tone', 'author': 'bob', 'license': 'CC BY-NC 4.0',+ 'page_url': 'https://freesound.org/people/bob/sounds/223/'},+ {'id': 'mk-5', 'category': 'amb', 'file': 'a/bed.wav', 'title': 'Bed', 'author': 'Mixkit',+ 'license': 'Mixkit Sound Effects Free License', 'page_url': 'https://mixkit.co/free-sound-effects/'},+ ]+ json.dump(ents, open(os.path.join(m, 'ui.json'), 'w'))+ json.dump([dict(ents[2], category='dup')], open(os.path.join(m, 'zz.json'), 'w'))+ return m+++def test_audit_and_credits(tmp_path):+ root = str(tmp_path)+ write(root, 'a/tick.wav', hits(0.3, [0.05]))+ write(root, 'a/sine.wav', hits(0.3, [0.05]))+ write(root, 'a/bed.wav', hits(1.0, [0.1]))+ m = manifests(root)+ os.makedirs(os.path.join(root, 'out/qa'), exist_ok=True)+ json.dump({'files_used': ['a/tick.wav', 'a/bed.wav', 'a/sine.wav']}, open(os.path.join(root, 'out/qa/mix_report.json'), 'w'))+ code, txt = run('audit.py', '--root', root, '--report', 'out/qa/mix_report.json')+ rows = {r['id']: r for r in json.load(open(os.path.join(m, 'audit.json'))) if not r.get('duplicate_of')}+ assert code == 1+ assert rows['fs-222']['severity'] == 'blocker' and any('forbidden' in p for p in rows['fs-222']['problems'])+ assert any('page is sound 223' in w for w in rows['fs-222']['warnings']) # the id does not match the page+ assert any('no licence record' in p for p in rows['fs-111']['problems'])+ assert rows['mk-5']['in_scope'] and rows['mk-5']['file_check']['decodes_clean']+ assert sum(1 for r in json.load(open(os.path.join(m, 'audit.json'))) if r.get('duplicate_of')) == 1+ agent = json.load(open(os.path.join(m, 'audit.json')))+ for r in agent: # the auditor records what it read on each page; audit.py keeps it on the next run+ if r['id'] in ('fs-111', 'mk-5'):+ r['licence_seen'] = 'page shows Attribution 4.0' if r['id'] == 'fs-111' else 'Mixkit Sound Effects Free License (read)'+ r['licence_confirmed'] = True+ json.dump(agent, open(os.path.join(m, 'audit.json'), 'w'))+ code, txt = run('audit.py', '--root', root, '--report', 'out/qa/mix_report.json')+ rows = {r['id']: r for r in json.load(open(os.path.join(m, 'audit.json'))) if not r.get('duplicate_of')}+ assert rows['fs-111']['severity'] == 'ok' and rows['mk-5']['severity'] == 'ok' and rows['fs-222']['severity'] == 'blocker'+ code, txt = run('credits.py', '--root', root, '--audit', 'audio/assets/manifests/audit.json')+ assert code == 1 and 'PROBLEM' in txt # the NC recording is used+ md = open(os.path.join(root, 'CREDITS.md'), encoding='utf-8').read()+ assert 'by alice' in md and 'creativecommons.org/licenses/by/4.0/' in md # the licence link is completed+ desc = open(os.path.join(root, 'out/CREDITS_description.txt'), encoding='utf-8').read()+ assert 'Sound effects (CC BY):' in desc and 'Mixkit' in desc+ assert 'were modified' in desc # CC BY 4.0 s.3(a)(1)(B): the description says the works were changed+ assert os.path.exists(os.path.join(m, 'audit.prev.json')) # the earlier audit is kept before it is rewritten+++def test_audit_inputs(tmp_path):+ code, txt = run('audit.py', '--root', str(tmp_path), '--manifests', 'nope', '--out', 'a.json')+ assert code == 2 and 'manifest folder not found' in txt+ m = tmp_path / 'm'+ m.mkdir()+ write(str(tmp_path), 'a/x.wav', hits(0.3, [0.05]))+ json.dump([{'id': 'fs-1', 'file': 'a/x.wav', 'title': 'x', 'author': 'a', 'license': 'CC0 1.0',+ 'page_url': 'https://freesound.org/people/a/sounds/1/'},+ {'id': 'fs-2', 'file': 'a/x.wav', 'title': 'x', 'author': 'a', 'license': 'CC BY-SA 4.0'}], open(m / 's.json', 'w'))+ code, txt = run('audit.py', '--root', str(tmp_path), '--manifests', 'm', '--out', 'a.json', '--decode', 'none')+ rows = json.load(open(tmp_path / 'a.json'))+ assert code == 1 and any('forbidden licence' in p for p in rows[0]['problems']) # the same file, a second licence+++def test_arrange_grid(tmp_path):+ x, sr = music(24.0)+ p = str(tmp_path / 'track.wav')+ sf.write(p, x, sr)+ code, txt = run('arrange.py', p, '--bars', '6', '--drop-film-bar', '4', '--out', str(tmp_path / 'a.json'))+ assert code == 0, txt+ r = json.load(open(tmp_path / 'a.json'))+ assert r['drop_source_bar'] == r['drop_candidates'][0] and r['map'][3] == r['drop_source_bar']+ assert abs(r['bpm'] - 120) < 0.5+ ph = (r['bars'][0]['t'] - 0.5) % 2.0+ assert min(ph, 2.0 - ph) < 0.02, r['bars'][0] # downbeats on the thumps (0.5 + 2k s), within 20 ms+ assert len(r['map']) == 6 and r['music']['segments'][0]['at'] == 0.0++[email protected](shutil.which('node') is None or shutil.which('ffmpeg') is None, reason='needs node and ffmpeg')+def test_mux(tmp_path):+ v, a, o = str(tmp_path / 'v.mp4'), str(tmp_path / 'a.wav'), str(tmp_path / 'o.mp4')+ subprocess.run(['ffmpeg', '-v', 'error', '-y', '-f', 'lavfi', '-i', 'testsrc=size=160x90:rate=30:duration=2', '-c:v',+ 'mpeg4', v], check=True)+ sf.write(a, hits(2.5, [0.5, 1.0]), SR)+ p = subprocess.run(['node', os.path.join(SCRIPTS, 'mux.mjs'), '--video', v, '--audio', a, '--out', o],+ capture_output=True, text=True)+ assert p.returncode == 0, p.stderr+ assert 'longer than the picture' in p.stdout+ j = json.loads(subprocess.run(['ffprobe', '-v', 'error', '-show_entries', 'stream=codec_type,codec_name:format=duration',+ '-of', 'json', o], capture_output=True, text=True).stdout)+ kinds = {s['codec_type']: s['codec_name'] for s in j['streams']}+ assert kinds == {'video': 'mpeg4', 'audio': 'aac'} and abs(float(j['format']['duration']) - 2.0) < 0.1+++def test_templates_parse():+ tdir = os.path.join(SKILL, 'templates')+ for f in os.listdir(tdir):+ if f.endswith('.json'):+ json.load(open(os.path.join(tdir, f), encoding='utf-8'))+ arc = json.load(open(os.path.join(tdir, 'arc.json')))+ assert len(arc['arc']) >= arc['drop_film_bar']+ top = json.load(open(os.path.join(tdir, 'sounddesign.json')))+ assert top['master']['lufs'] == -14 and all(i.startswith('audio/design/') for i in top['include'])+++def test_variant_patch_keeps_format():+ text = ('{\n "rules": [\n {"name": "a", "gain_db": -3.0, "pitch": [1, 1]},\n'+ ' {"name": "b", "gain_db": -6},\n {"name": "c", "gain_db": -9}\n ],\n'+ ' "samples": {"s1": {"file": "x.ogg", "end": 0.250}}\n}\n')+ new = json.loads(text)+ del new['rules'][1]+ new['rules'].append({'name': 'd'}) # a delete plus an add keeps the length: must not patch pairwise+ new['rules'][0]['gain_db'] = -4+ new['samples']['s1']['eq'] = {'lp': 9000}+ out = variant.patch_text(text, new)+ assert json.loads(out) == new+ assert '{"name": "a", "gain_db": -4, "pitch": [1, 1]}' in out and '"end": 0.250' in out # spacing and 0.250 kept+ assert '{"name": "c", "gain_db": -9}' in out and '"b"' not in out+ assert variant.change('x', 0.3, 0.3).endswith('unchanged (0.3)')+++def test_variant_ops(project, tmp_path):+ top = {'duration': 12.0, 'cues': 'out/cues.json', 'include': ['audio/design.json'], 'master': {'lufs': -14}}+ json.dump(top, open(os.path.join(project, 'audio/top.json'), 'w'))+ ops = [{'f': 'design', 'rule': 'fx.hit', 'set': {'gain_db': -5}}, {'f': 'design', 'sample': 'fx.pop', 'eq': {'lp': 9000}},+ {'sd': 'master.lufs', 'value': -15}]+ json.dump(ops, open(tmp_path / 'ops.json', 'w'))+ out = str(tmp_path / 'v1')+ code, txt = run('variant.py', '--root', project, '--design', 'audio/top.json', '--ops', str(tmp_path / 'ops.json'), '--out', out)+ assert code == 0 and '3 change(s)' in txt, txt+ v = json.load(open(os.path.join(out, 'design', 'design.json')))+ assert [r for r in v['rules'] if r['name'] == 'fx.hit'][0]['gain_db'] == -5+ assert v['samples']['fx.pop']['eq'] == {'hp': 250, 'peq': [[3200, 1.2, -2]], 'lp': 9000}+ assert json.load(open(os.path.join(out, 'sounddesign.json')))['master']['lufs'] == -15+ orig = json.load(open(os.path.join(project, 'audio/design.json')))+ assert [r for r in orig['rules'] if r['name'] == 'fx.hit'][0]['gain_db'] == -2 # the design itself is untouched+ json.dump([{'f': 'design', 'rule': 'no.such.rule', 'set': {'gain_db': 0}}], open(tmp_path / 'bad.json', 'w'))+ code, txt = run('variant.py', '--root', project, '--design', 'audio/top.json', '--ops', str(tmp_path / 'bad.json'),+ '--out', str(tmp_path / 'v2'))+ assert code != 0 and 'nothing written' in txt and not os.path.exists(tmp_path / 'v2')diff --git a/tests/test_realness.py b/tests/test_realness.pynew file mode 100644index 0000000..f6a149e--- /dev/null+++ b/tests/test_realness.py@@ -0,0 +1,222 @@+"""realness: synthetic signals only; model-free unless marked slow (the CLAP model is stubbed)."""+import json++import numpy as np+import pytest++from earshot import realness as rl++SR = 48000+++def tone(f=880.0, dur=1.0, amp=0.3):+ t = np.arange(int(dur * SR)) / SR+ return (amp * np.sin(2 * np.pi * f * t) * np.exp(-t / 0.4)).astype(np.float32)+++def noisy_hit(dur=1.0, seed=0):+ """decaying noise transients over a -70 dBFS-ish noise floor (microphone-like)"""+ rng = np.random.RandomState(seed)+ n = int(dur * SR)+ x = rng.randn(n) * 3e-4+ for t0 in (0.1, 0.45):+ a, k = int(t0 * SR), int(0.2 * SR)+ x[a:a + k] += rng.randn(k) * np.exp(-np.arange(k) / SR / 0.02) * 0.4+ return x.astype(np.float32)+++class FakeClap:+ """an embedding that drives the CLAP head to score +4 for rough (noise-like) clips and -4 for smooth (tonal) ones,+ and fixed unit text vectors"""+ def __init__(self, m):+ h = m['heads']['clap']+ self.mean, self.scale, self.coef = (np.asarray(h[k], np.float64) for k in ('mean', 'scale', 'coef'))+ self.b = h['intercept']++ def embed_audio(self, xs, **kw):+ out = []+ for x in xs:+ k = (4.0 if np.std(np.diff(x)) > 0.5 * np.std(x) else -4.0) - self.b+ out.append(self.mean + self.scale * self.coef * k / (self.coef @ self.coef))+ return np.stack(out).astype(np.float32)++ def embed_text(self, texts):+ rng = np.random.RandomState(1)+ v = rng.randn(len(texts), len(self.mean))+ return v / np.linalg.norm(v, axis=1, keepdims=True)+++def fake_clap(monkeypatch):+ from earshot.models import clap+ monkeypatch.setattr(clap, 'get', lambda: FakeClap(rl.load_model()))+++def test_model_is_plain_json_and_complete():+ m = json.load(open(rl.MODEL_PATH, encoding='utf-8'))+ assert set(m['heads']) == {'clap', 'signal'} and m['fusion']['heads'] == ['clap', 'signal']+ assert len(m['heads']['clap']['coef']) == 512+ assert m['heads']['signal']['inputs'] == list(m['heads']['signal']['inputs'])+ assert len(m['heads']['signal']['coef']) == len(m['heads']['signal']['inputs'])+ assert 0 < m['thresholds']['lo'] < 0.5 <= m['thresholds']['hi'] < 1+ ft = m['file_thresholds']+ assert m['thresholds']['lo'] <= ft['lo'] < 0.5 <= m['thresholds']['hi'] <= ft['hi'] < 1 and ft['synth_files'] > 0+ assert m['long_s'] == rl.LONG_S and m['win_s'] == rl.WIN_S+++def test_signal_features_separate_tone_from_recording():+ ft, fr = rl.signal_features(tone()), rl.signal_features(noisy_hit())+ assert set(ft) == set(rl.load_model()['heads']['signal']['inputs'])+ assert ft['tonal_frames'] > 0.8 > fr['tonal_frames']+ assert ft['line_share'] > fr['line_share']+ assert ft['spec_floor_db'] < fr['spec_floor_db']+ assert ft['pitch_wobble'] < -1.0+++def test_signal_features_ignore_level():+ a, b = rl.signal_features(noisy_hit()), rl.signal_features(noisy_hit() * 0.1)+ for k in a:+ if k != 'silence_frac' and np.isfinite(a[k]):+ assert a[k] == pytest.approx(b[k], abs=1e-3), k+++def test_silent_and_tiny_clips_are_not_judged():+ assert rl.signal_features(np.zeros(SR, np.float32)) is None+ assert rl.signal_features(np.ones(100, np.float32) * 0.1) is None+ assert not rl.audible(np.full(SR, 1e-4, np.float32)) and rl.audible(tone())+++def test_clips_windows():+ assert len(rl._clips(np.zeros(5 * SR))) == 1+ spans = rl._clips(np.zeros(int(13.55 * SR)))+ assert [round(s[0], 2) for s in spans] == [0.0, 4.0, 8.0, 9.55]+ spans = rl._clips(np.zeros(int(60.01 * SR)))+ assert len(spans) == 15 and spans[-1][0] == 56.0+++def test_head_logit_and_fuse():+ m = rl.load_model()+ h = m['heads']['signal']+ logit, z = rl.head_logit(h, h['mean'])+ assert np.allclose(z, 0) and logit == pytest.approx(h['intercept'])+ p, s = rl.fuse(m['fusion'], {'clap': 2.0, 'signal': -1.0})+ assert 0 < p < 1 and p == pytest.approx(1 / (1 + np.exp(-s)))+ t = m['thresholds']+ assert rl.verdict_of(t['hi'], m) == 'real or pro-produced'+ assert rl.verdict_of(t['lo'], m) == 'likely code-synthesized'+ assert rl.verdict_of((t['lo'] + t['hi']) / 2, m) == 'uncertain'+ ft = m['file_thresholds']+ assert rl.verdict_of(ft['lo'], m, file=True) == 'likely code-synthesized'+ assert rl.verdict_of(ft['lo'] + 0.01, m, file=True) == 'uncertain'+++def test_digital_silence_is_never_a_verdict_reason():+ m = rl.load_model()+ assert all(q['feature'] != 'silence_frac' for q in m['facts'])+ assert all(q['precision'] >= 0.9 for q in m['facts'])+ assert rl._silence_line(m, {'silence_frac': 0.01}) is None+ line = rl._silence_line(m, {'silence_frac': 0.38})+ assert 'weak evidence' in line and 'real/pro' in line and 'no room tone' not in line+ facts = rl._facts(m, {'silence_frac': 0.99, 'crest_db': 3.0})+ assert facts and all('silence' not in t for _, _, t in facts)+++def test_realness_silent_file_needs_no_model(wav):+ r = rl.realness(wav(np.zeros(SR, np.float32)), cache=False)+ assert r['verdict'] == 'silent' and r['p_real'] is None and 'silent' in r['summary']+++def noise(seconds, amp=0.1):+ return (np.random.RandomState(3).randn(int(seconds * SR)) * amp).astype(np.float32)+++def test_realness_structure_with_stub(wav, fake_clap):+ r = rl.realness(wav(tone(dur=1.0)), cache=False)+ assert r['p_real'] < 0.5 and r['verdict'] in ('uncertain', 'likely code-synthesized')+ assert r['evidence'][0].startswith('CLAP head') and 'segments' not in r+ assert 'closest reference prompt' in r['evidence'][0] and 'weak context' in r['evidence'][0]+ assert 'nearest description' not in r['summary'] and 'calibrated on held-out clips' in r['note']+ json.dumps(r)+ x = np.concatenate([noisy_hit(dur=4.0), np.zeros(int(0.5 * SR), np.float32), tone(dur=4.0)])+ r = rl.realness(wav(x, 'long.wav'), cache=False)+ assert len(r['segments']) == 3 and r['segments'][-1]['end'] == pytest.approx(8.5, abs=1e-3)+ assert r['evidence'][0].startswith('evidence below is from the most')+ assert 'windows' in r['summary'] and r['summary'].isascii() and 'file-level thresholds' in r['note']+ text = rl._text(r)+ assert text.isascii() and 'p_real' in text+++def test_long_file_reports_lean_windows_in_file_time(wav, fake_clap):+ x = np.concatenate([np.zeros(2 * SR, np.float32), tone(dur=4.0), noise(4.0), tone(dur=4.0, f=660.0)])+ path = wav(x, 'tnt.wav')+ r = rl.realness(path, cache=False)+ segs = r['segments']+ assert [s['start'] for s in segs] == [0.0, 4.0, 8.0, 10.0]+ assert r['lean_synth_windows'] == sum(s['p_real'] is not None and s['p_real'] < 0.5 for s in segs) >= 2+ assert 'lean synthesized' in r['summary'] and 'Leaning synthesized:' in r['summary']+ r = rl.realness(path, start=2.0, end=14.0, cache=False)+ assert r['range'] == [2.0, 14.0] and [s['start'] for s in r['segments']] == [2.0, 6.0, 10.0]+ assert r['segments'][1]['p_real'] > 0.5 > r['segments'][0]['p_real']+ assert '[0:02.00-0:14.00]' in r['summary'] and '0:02.00-0:06.00' in r['summary']+ clean = rl.realness(wav(noise(9.0), 'n.wav'), cache=False)+ assert clean['lean_synth_windows'] == 0 and 'does not rule out a code-made layer' in clean['summary']+++def test_bad_ranges_raise_clear_errors(wav):+ path = wav(tone(dur=1.5), 'short.wav')+ with pytest.raises(ValueError, match='beyond the end'):+ rl.realness(path, start=5, end=6, cache=False)+ with pytest.raises(ValueError, match='greater than start'):+ rl.realness(path, start=1, end=0.5, cache=False)+++def test_many_counts_silent_error_and_missing_separately(wav, tmp_path, fake_clap):+ bad = tmp_path / 'corrupt.wav'+ bad.write_text('not audio')+ many = rl.realness_many([wav(noise(1.0), 'a.wav'), wav(np.zeros(SR, np.float32), 'b.wav'), str(bad),+ str(tmp_path / 'nope.wav')])+ v = [f['verdict'] for f in many['files']]+ assert v.count('silent') == 1 and v.count('error') == 2 and many['counts']['real or pro-produced'] <= 1+ assert '1 silent (not judged)' in many['summary'] and '2 could not be read' in many['summary']+ assert many['summary'].startswith('4 files:')+ table = rl._table(many)+ assert table.isascii() and len(table.splitlines()) == 7 # summary, thresholds line, header, 4 rows+ assert 'b.wav: b.wav' not in table and 'no such file or folder' in table+ assert 'Verdict thresholds' in table.splitlines()[1]+++def test_many_applies_the_range_to_every_file(wav, fake_clap):+ many = rl.realness_many([wav(noise(2.0), 'a.wav'), wav(tone(dur=2.0), 'b.wav')], start=0.5, end=1.0)+ assert all(f['range'] == [0.5, 1.0] for f in many['files']) and 'Range 0:00.50-0:01.00' in many['summary']+++def test_cli_missing_path_is_an_error(tmp_path, capsys):+ from earshot import cli+ assert cli.main(['realness', str(tmp_path / 'nope.wav')]) == 1+ assert 'no such file or folder' in capsys.readouterr().err+++def test_realness_flags_a_code_beep(wav):+ t = np.arange(int(0.6 * SR)) / SR+ x = np.r_[np.zeros(int(0.1 * SR)), 0.5 * np.sin(2 * np.pi * 880 * t) * np.exp(-t / 0.15), np.zeros(int(0.3 * SR))]+ r = rl.realness(wav(x.astype(np.float32), 'beep.wav'), cache=False)+ assert r['p_real'] < 0.5 and r['verdict'] != 'real or pro-produced'+ assert any('CLAP head' in e for e in r['evidence'])+++def rms_db(x):+ return 20 * np.log10(np.sqrt(np.mean(np.square(x, dtype=np.float64))))+++def test_quiet_sources_are_lifted_smoothly_not_called_silent():+ x = noise(1.0)++ def at(db):+ return (x * 10 ** ((db - rms_db(x)) / 20)).astype(np.float32)+ y, g, rdb, pdb = rl.lift_quiet(at(-75))+ assert rdb < rl.SILENT_DB and rl.audible(y) and rms_db(y) == pytest.approx(rl.LIFT_BELOW_DB, abs=0.01)+ assert np.abs(y).max() <= 10 ** (-1 / 20) + 1e-6+ assert rl.lift_quiet(at(rl.LIFT_BELOW_DB - 0.5))[1] == pytest.approx(0.5, abs=0.01) # no cliff+ assert rl.lift_quiet(at(rl.LIFT_BELOW_DB + 0.5))[1] == 0.0+ assert rl.lift_quiet(np.zeros(SR, np.float32))[1] == 0.0diff --git a/tests/test_semantics.py b/tests/test_semantics.pynew file mode 100644index 0000000..510de1c--- /dev/null+++ b/tests/test_semantics.py@@ -0,0 +1,298 @@+"""Tests for the semantics modules (vocab, describe, match, search, compare). Fast tests use synthetic signals and fake+embeddings; tests marked slow load CLAP / AST (run with --slow)."""+import json++import numpy as np+import pytest++from earshot import compare, describe, features, match, search, vocab+from earshot.report import to_json++SR = 48000+++def _fake_vocab(T=40, seed=0):+ rng = np.random.RandomState(seed)+ v = vocab.Vocab.__new__(vocab.Vocab)+ v.names = [f'term {i}' for i in range(T)]+ v.cats = {'a': v.names[:T // 2], 'b': v.names[T // 2:]}+ v.cat_of = {t: ('a' if i < T // 2 else 'b') for i, t in enumerate(v.names)}+ v.E = vocab._norm(rng.randn(T, 512).astype(np.float32))+ v.m = vocab._norm(rng.randn(512).astype(np.float32)) * 0.3+ v.mu_E = v.bias(v.E)+ return v+++def _clicks(times, dur=4.0, amp=0.5, seed=1):+ rng = np.random.RandomState(seed)+ x = rng.randn(int(dur * SR)).astype(np.float32) * 1e-4+ n = int(0.08 * SR)+ burst = rng.randn(n) * np.exp(-np.arange(n) / SR / 0.008) * amp+ for t in times:+ a = int(t * SR)+ x[a:a + n] += burst.astype(np.float32)+ return x+++# ---------------------------------------------------------------------------------------------------- vocab+def test_vocab_json_schema():+ d = vocab.load_vocab()+ assert set(d) >= {'templates', 'categories'}+ assert all('{}' in t for t in d['templates'])+ terms = [t for ts in d['categories'].values() for t in ts]+ assert len(terms) == len(set(terms)) and len(terms) > 200+ assert set(vocab.UNVALIDATED) <= set(d['categories'])+++def test_stats_file_matches_vocab_and_model():+ s = vocab.load_stats()+ assert s is not None and len(s['mean']) == 512 and s['pad'] == vocab.PAD+ assert set(vocab.load_vocab()['templates']) == set(s['templates'])+ assert s['n_ref'] >= 100+++def test_spans_for():+ assert vocab.spans_for(0.0, 3.0) == [(0.0, 3.0)]+ sp = vocab.spans_for(0.0, 25.0, max_win=None)+ assert [round(a, 3) for a, _ in sp] == [0.0, 5.0, 10.0, 15.0] and sp[-1][1] == pytest.approx(25.0)+ sp = vocab.spans_for(2.0, 12.5, max_win=None) # last window flush with the end+ assert sp[0] == (2.0, 12.0) and sp[-1][1] == pytest.approx(12.5)+ sp = vocab.spans_for(0.0, 170.0, max_win=6)+ assert len(sp) == 6 and sp[0][0] == 0.0 and sp[-1][1] == pytest.approx(170.0)+ assert all(b - a == pytest.approx(10.0) for a, b in sp)+++def test_group_weights_balance_folders(tmp_path):+ paths = [str(tmp_path / 'a' / f'{i}.wav') for i in range(4)] + [str(tmp_path / 'b' / 'x.wav')]+ w = vocab.group_weights(paths)+ assert w[:4].sum() == pytest.approx(w[4])+++def test_calibrate_math():+ v = _fake_vocab()+ rng = np.random.RandomState(3)+ A = vocab._norm(rng.randn(5, 512).astype(np.float32))+ r = v.calibrate(A, v.E[:3], 'both')+ assert np.allclose(r['z_t'].mean(1), 0, atol=1e-5) and np.allclose(r['z_t'].std(1), 1, atol=1e-4)+ assert np.allclose(r['z_q'], r['z_t'][:, :3], atol=1e-5) # a query equal to a term scores like the term+ assert np.allclose(r['cos_q'], A @ v.E[:3].T)+ raw = v.calibrate(A, v.E[:3], 'raw')+ assert np.allclose(raw['z_q'], raw['cos_q'])+ c = v.calibrate(A, None, 'center')+ assert np.allclose(c['u_t'], A @ v.E.T - v.E @ v.m, atol=1e-5)+ assert ((r['pct_q'] >= 0) & (r['pct_q'] <= 1)).all()+++def test_top_terms_filters_categories():+ v = _fake_vocab()+ a = v.E[25] * 0.9 + v.E[3] * 0.1+ a /= np.linalg.norm(a)+ top = v.top_terms(a, k=3)+ assert top[0][0] == 'term 25' and top[0][2] > 2+ assert all(c == 'a' for _, c, _, _ in v.top_terms(a, k=5, cats=('a',)))+ assert all(c != 'b' for _, c, _, _ in v.top_terms(a, k=5, exclude=('b',)))+++# ---------------------------------------------------------------------------------------------------- match+def test_verdict_thresholds_ordered():+ zs = [t for t, _ in match.VERDICTS]+ assert zs == sorted(zs, reverse=True)+ assert match.verdict(zs[0] + 0.01) == 'strong' and match.verdict(-5) == 'no match'+ assert set(match.VERDICT_EVIDENCE) == {name for _, name in match.VERDICTS} | {'no match'}+ lrs = [match.VERDICT_EVIDENCE[n]['lr'] for _, n in match.VERDICTS] + [match.VERDICT_EVIDENCE['no match']['lr']]+ assert lrs == sorted(lrs, reverse=True) and lrs[-1] < 1 # bands, each stronger evidence than the next+++def test_z_text_never_contradicts_verdict():+ assert vocab.zf(2.96) == '+2.9' and vocab.zf(3.0) == '+3.0' and vocab.zf(-0.04) == '-0.1'+ for z in np.arange(1.5, 3.5, 0.01):+ assert match.verdict(float(vocab.zf(z))) == match.verdict(z) or float(vocab.zf(z)) < z+++def test_check_rejects_bad_ranges(wav, click):+ p = wav(click)+ assert vocab.check(p) == pytest.approx(0.25)+ with pytest.raises(ValueError):+ vocab.check(p, start=1.0)+ with pytest.raises(ValueError):+ vocab.check(p, start=0.1, end=0.05)+ with pytest.raises(Exception):+ vocab.check(p + '.missing.wav')+ with pytest.raises(ValueError):+ match.match(p, [])+++def test_readable_skips_bad_files(wav, click, tmp_path):+ bad = tmp_path / 'not_audio.wav'+ bad.write_bytes(b'this is not audio')+ ok, skipped = vocab.readable([wav(click), str(bad)])+ assert len(ok) == 1 and skipped[0]['path'] == str(bad) and skipped[0]['error']+++def test_softmax_is_a_distribution():+ p = match.softmax([0.3, 0.25, 0.1], 38.7)+ assert p.sum() == pytest.approx(1.0) and p[0] > p[1] > p[2]+++# ---------------------------------------------------------------------------------------------------- search+def test_event_candidates_land_on_clicks():+ times = [0.5, 1.5, 2.6]+ x = _clicks(times)+ c = search.event_candidates(x)+ strokes = sorted(s for s in c if s[2] == 'stroke')+ assert len(strokes) == 3+ for (s, e, _, _), t in zip(strokes, times):+ assert t - 0.015 <= s <= t + 0.002 # trim starts just before the attack, never after it+ assert 0.02 < e - s < 0.5 # ends after the decay, long before the next click+++def test_down_up_strokes_are_paired():+ x = _clicks([0.5, 0.62, 1.8, 1.92], amp=0.5)+ c = search.event_candidates(x)+ pairs = sorted((round(s, 2), round(e, 2)) for s, e, k, _ in c if k == 'event')+ assert len(pairs) == 2 and pairs[0][0] == pytest.approx(0.49, abs=0.02) and pairs[0][1] > 0.62+++def test_select_round_robin_covers_time():+ cands = [(t, t + 0.1, 'stroke', 50.0) for t in np.arange(0, 4, 0.05)] + [(10.0, 10.1, 'stroke', 1.0)]+ sel = search._select(cands, n=6, bucket=2.0)+ assert len(sel) == 6 and any(s[0] >= 10 for s in sel)+++def test_window_candidates_cover_everything():+ w = search.window_candidates(18.0)+ assert w[0][:2] == (0.0, 3.0) and w[-1][1] == pytest.approx(18.0)+ assert search.window_candidates(0.5) == [(0.0, 0.5, 'window', 0.0)]+ for dur in (44.0, 300.0, 600.0, 1800.0): # no gaps anywhere, whatever the length+ w = search.window_candidates(dur)+ assert w[0][0] == 0.0 and w[-1][1] == pytest.approx(dur)+ assert all(b[0] <= a[1] + 1e-6 for a, b in zip(w, w[1:]))+ assert all(b - a <= search.MAX_WIN_S + 1e-6 for a, b, _, _ in w)+ assert len(search.window_candidates(300.0)) <= search.MAX_WINDOWS+ w = search.window_candidates(12.0, t0=5.1) # a --start/--end range in local time+ assert w[0][0] == 5.1 and w[-1][1] == pytest.approx(12.0)+++def test_event_candidates_span_a_long_file():+ x = _clicks(list(np.arange(0.5, 299, 1.0)), dur=300.0, amp=0.3)+ allc = search.event_candidates(x, n=None)+ assert max(c[0] for c in allc) > 290 # every onset is a candidate before any selection+ assert len(search.event_candidates(x)) <= search.MAX_EVENTS+++def test_dedupe():+ c = [(0.0, 1.0, 'stroke', 1), (0.01, 1.0, 'event', 1), (2.0, 3.0, 'stroke', 1)]+ assert len(search._dedupe(c)) == 2+++def test_tail_db():+ x = _clicks([0.5])+ env = search._env_db(x)+ assert search._tail_db(env, 0.495, 0.7) < -30+ assert search._tail_db(env, 0.495, 0.505) > -15+++# ---------------------------------------------------------------------------------------------------- compare / describe+def _f(x):+ return compare.feats(features.measure(x, light=True), x)+++def test_differences_in_words(click, sine):+ fa, fb = _f(sine * 0.1), _f(click)+ keys, txt = compare.differences(fa, fb)+ assert 'brightness_hz' in keys and 'brighter (brightness' in txt and len(keys) <= 4+ keys, txt = compare.differences(fa, dict(fa, loudness_lufs=fa['loudness_lufs'] + 6))+ assert keys == ['loudness_lufs'] and txt.startswith('6 dB louder')+ assert compare.differences(fa, fa)[0] == []+++def test_trims_of_the_same_sound_read_the_same(click):+ short = click[: int(0.05 * SR)]+ for pad_s in (0.3, 0.5, 2.0): # extra silence after (and before) the same click+ padded = np.concatenate([np.zeros(int(0.1 * SR), np.float32), short, np.zeros(int(pad_s * SR), np.float32)])+ keys, txt = compare.differences(_f(short), _f(padded))+ assert keys == [], (pad_s, txt)+ st = np.stack([short, short], axis=1) # dual mono = mono: no width difference+ assert compare.differences(_f(short), _f(st))[0] == []+ assert _f(short)['width_db'] == compare.MONO_DB and 'noise_floor_dbfs' not in _f(short)+++def test_closer_is_none_when_too_close():+ assert compare._closer(2.9, 2.91) == (None, 'Neither is closer')+ assert compare._closer(1.0, 2.5)[0] == 'b' and compare._closer(2.0, 1.6)[0] == 'a'+++def test_typicality_flags_outlier():+ rng = np.random.RandomState(0)+ c = vocab._norm(rng.randn(512))+ R = vocab._norm(c + 0.3 * vocab._norm(rng.randn(12, 512)))+ inlier = vocab._norm(c + 0.3 * vocab._norm(rng.randn(512)))+ outlier = vocab._norm(rng.randn(512))+ assert compare._typicality(inlier, R, 3)[1] > -1+ assert compare._typicality(outlier, R, 3)[1] < -2+++def test_shape_words(click, sine):+ assert 'short transient' in describe.shape(features.measure(click))+ s = describe.shape(features.measure(sine))+ assert 'tonal' in s and 'A5' in s+++def test_distinct_windows_do_not_overlap_or_repeat():+ rng = np.random.RandomState(0)+ W = vocab._norm(np.repeat(rng.randn(1, 512), 6, axis=0) + 0.01 * rng.randn(6, 512))+ W[4] = vocab._norm(rng.randn(512))+ spans = [(5.0 * i, 5.0 * i + 10) for i in range(6)]+ tags = ['x', 'x', 'y', 'x', 'z', 'x']+ idx = describe.distinct_windows(spans, W, tags, n=3)+ assert 4 in idx and len(idx) >= 2+ assert all(spans[a][1] <= spans[b][0] for a, b in zip(idx, idx[1:]))+ assert len({tags[i] for i in idx}) == len(idx)+ assert describe.distinct_windows(spans, W, ['x'] * 6) == [] # uniform: nothing reads differently+++def test_measurements_hide_what_is_not_measurable(click):+ m = describe.measurements(features.measure(click))+ assert m['width'] == 'mono' and 'noise_floor_dbfs' not in m and 'centroid_hz' not in m+++# ---------------------------------------------------------------------------------------------------- with models+def test_match_and_describe_end_to_end(wav, click):+ p = wav(np.concatenate([click, np.zeros(SR // 4, np.float32)]))+ r = match.match(p, ['a short click', 'a choir singing'], alts=['music'])+ json.loads(to_json(r))+ assert {'query', 'cosine', 'z', 'percentile', 'rank_in_vocab', 'p_vs_alts', 'verdict', 'summary'} <= set(r)+ assert r['results'][0]['z'] >= r['results'][1]['z']+ d = describe.describe(p)+ json.loads(to_json(d))+ assert d['top_tags'] and d['audioset'] and 'Sounds most like' in d['summary']+ assert all(t['category'] not in vocab.UNVALIDATED for t in d['top_tags'])+ tiny = describe.describe(wav(click[:100], 'tiny.wav'))+ assert 'too short' in tiny['summary'] and tiny['top_tags'] == []+ silent = describe.describe(wav(np.zeros(SR, np.float32), 'silence.wav'))+ assert 'near-silence' in silent['summary']+++def test_rank_skips_unreadable_and_flags_silence(wav, click, tmp_path):+ wav(click, 'click.wav')+ wav(np.zeros(SR, np.float32), 'silence.wav')+ (tmp_path / 'broken.wav').write_bytes(b'not audio at all')+ r = search.rank([str(tmp_path)], 'a short click')+ json.loads(to_json(r))+ assert r['n_files'] == 2 and len(r['skipped']) == 1+ assert r['results'][-1]['name'] == 'silence.wav' and r['results'][-1]['near_silent']+ assert 'unreadable' in r['summary'] and 'near-silent' in r['summary']+++def test_find_segments_hold_the_clicks(wav):+ times = [1.0, 3.0, 5.0]+ r = search.find(wav(_clicks(times, dur=6.0)), 'a single click', top=3)+ json.loads(to_json(r))+ assert r['query_kind'] == 'event' and len(r['segments']) == 3+ held = sorted(next(t for t in times if s['start'] <= t < s['end']) for s in r['segments'])+ assert held == times # three non-overlapping segments, one per click+ r = search.find(wav(_clicks(times, dur=6.0), 'r.wav'), 'a single click', top=3, start=2.9, end=6.0)+ assert r['segments'] and all(s['start'] >= 2.9 - search.PRE_ROLL - 1e-3 for s in r['segments'])diff --git a/tests/test_timeline.py b/tests/test_timeline.pynew file mode 100644index 0000000..4d9575e--- /dev/null+++ b/tests/test_timeline.py@@ -0,0 +1,274 @@+"""timeline + sheet: synthetic signals only; model-free unless marked slow (the CLAP / AST steps are stubbed)."""+import json+import os++import numpy as np+import pytest++from earshot import sheet as sheet_mod+from earshot import timeline as tl++SR = 48000+++def click_track(times, dur, amp=0.5, bed=0.01, seed=0, sr=SR):+ """decaying noise bursts at `times` over a quiet noise bed"""+ rng = np.random.RandomState(seed)+ x = rng.randn(int(dur * sr)).astype(np.float32) * bed+ n = int(0.05 * sr)+ env = np.exp(-np.arange(n) / sr / 0.008)+ for t in times:+ a = int(round(t * sr))+ x[a:a + n] += (rng.randn(n) * env * amp)[: len(x) - a]+ return x+++def no_models(monkeypatch):+ """replace the model steps (CLAP tags, sections) so the pipeline runs in milliseconds"""+ def fake_tags(x, fid, t0, events, n_tag, n_ast, near=(), reuse=True, win=None):+ for e in sorted(events, key=lambda e: -e['prominence_db'])[:n_tag]:+ e.update(kind='HIT', label='punchy impact hit', tags=[{'tag': 'a punchy impact hit', 'kind': 'HIT', 'z': 3.0}])+ return {'tagged': min(n_tag, len(events)), 'ast': 0, 'clap_windows': 0}++ def fake_sections(x, fid, t0, win, hop, ast_budget=4, reuse=True, **kw):+ d = len(x) / SR+ return [{'start': 0.0, 'end': round(d, 2), 'label': 'test bed', 'lufs': -20.0, 'windows': 1}], \+ {'windows': 0, 'ast': 0}+ monkeypatch.setattr(tl, 'tag_events', fake_tags)+ monkeypatch.setattr(tl, 'sections', fake_sections)+++def test_detects_clicks_at_their_attack():+ times = [0.5, 1.25, 2.0, 2.6, 3.33]+ ev = tl.detect_events(click_track(times, 4.0))+ hits = [e['t'] for e in ev if e['shape'] == 'hit']+ for t in times:+ assert min(abs(np.array(hits) - t)) <= 0.003, (t, hits)+ assert len(hits) <= len(times) + 1+ e = min(ev, key=lambda e: abs(e['t'] - 1.25))+ assert e['prominence_db'] > 20 and -12 < e['peak_dbfs'] < 6 and e['length_ms'] < 200+++def test_quiet_click_is_less_prominent(wav):+ x = click_track([1.0], 2.0, amp=0.5) + click_track([1.5], 2.0, amp=0.05, bed=0.0, seed=1)+ ev = {round(e['t'], 1): e for e in tl.detect_events(x)}+ assert ev[1.0]['prominence_db'] > ev[1.5]['prominence_db'] + 10+++def test_loudness_gaps_and_clipping(sine):+ x = np.concatenate([np.tile(sine, 3), np.zeros(SR), np.tile(sine, 2)])+ lo = tl.loudness(x)+ gaps = lo['silences']+ assert len(gaps) == 1 and gaps[0]['kind'] == 'digital silence'+ assert abs(gaps[0]['start'] - 3.0) < 0.15 and abs(gaps[0]['end'] - 4.0) < 0.15+ assert lo['clipping']['runs'] == 0+ clipped = np.clip(np.tile(sine, 6) * 5, -1, 1)+ assert tl.loudness(clipped)['clipping']['runs'] > 10+++def test_beat_grid_on_a_click_track():+ beats = np.arange(0.25, 12.0, 0.5)+ x = click_track(beats, 12.5, amp=0.3)+ t = np.arange(int(0.12 * SR)) / SR+ thump = (np.sin(2 * np.pi * 60 * t) * np.exp(-t / 0.04) * 0.6).astype(np.float32)+ for b in beats[::4]: # accented low thump on every 4th beat = downbeat+ a = int(b * SR)+ x[a:a + len(thump)] += thump[: len(x) - a]+ g = tl.music_grid(x)+ assert abs(g['tempo'] - 120) < 2 and g['confidence'] == 'clear'+ d = [min(abs(beats - b)) for b in g['beats'] if 1.0 < b < 11.5]+ assert np.median(d) < 0.015+ assert min(abs(beats[::4] - g['downbeats'][1])) < 0.03+++def test_no_beat_grid_in_noise():+ rng = np.random.RandomState(3)+ assert tl.music_grid((rng.randn(8 * SR) * 0.1).astype(np.float32))['confidence'] == 'none'+++def test_read_cues_formats(tmp_path):+ f = tmp_path / 'cues.json'+ f.write_text(json.dumps({'fps': 60, 'events': [{'f': 120, 'kind': 'hit', 'id': 'a'}, {'f': 30, 'kind': 'tick'}]}))+ cues, fps = tl.read_cues(str(f))+ assert fps == 60 and [c['t'] for c in cues] == [0.5, 2.0] and cues[1]['id'] == 'a'+ assert [c['t'] for c in tl.read_cues('1.5, 3')[0]] == [1.5, 3.0]+ assert [c['t'] for c in tl.read_cues('[{"t": 2.25}]')[0]] == [2.25]+ assert tl.read_cues([4, 1])[0][0]['t'] == 1+ assert tl.read_cues(None) == ([], None)+++def test_sync_offsets_status_and_chance():+ ev = [{'t': t, 'prominence_db': p, 'shape': 'hit'} for t, p in ((1.000, 12), (2.030, 12), (3.300, 2), (5.0, 12))]+ s = tl.sync([1.004, 2.0, 3.25, 4.0], ev, beats=[1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0], frame_s=1 / 60)+ it = s['items']+ assert it[0]['status'] == 'OK' and it[0]['offset_ms'] == pytest.approx(-4.0)+ assert it[1]['status'] == 'near' and it[1]['offset_ms'] == pytest.approx(30.0)+ assert it[2]['status'] == 'near'+ assert it[3]['status'] == 'no sound' and it[3]['offset_ms'] is None+ st = s['stats']+ assert st['hit_within_1_frame'] == 0.25 and st['no_sound'] == [4.0] and 0 < st['chance_within_1_frame'] < 0.1+ assert st['on_beat'] == 0.75 and st['on_beat_chance'] == 0.2+ assert st['accent_min_db'] == 12.0 and [i['accent_offset_ms'] for i in it] == [-4.0, 30.0, None, None]+ assert st['accent_within_1_frame'] == 0.25 and st['accent_within_100ms'] == 0.5+ assert st['accent_chance_within_1_frame'] < st['chance_within_1_frame']+++def test_video_cuts_and_frames(wav, no_models, tmp_path, monkeypatch):+ """a tiny film: black -> white picture cut at 1.5 s with a click on it; its own audio and cuts are used"""+ import subprocess+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ a = wav(click_track([0.6, 1.5, 2.4], 3.0), 'a.wav')+ mp4 = str(tmp_path / 'film.mp4')+ subprocess.run(['ffmpeg', '-y', '-v', 'error', '-f', 'lavfi', '-i', 'color=c=black:s=96x54:r=30:d=1.5', '-f', 'lavfi',+ '-i', 'color=c=white:s=96x54:r=30:d=1.5', '-i', a, '-filter_complex', '[0:v][1:v]concat=n=2:v=1:a=0[v]',+ '-map', '[v]', '-map', '2:a', '-c:v', 'mpeg4', '-c:a', 'aac', mp4], check=True)+ r = tl.timeline(mp4)+ assert r['video'] == mp4 and r['fps'] == 30 and [round(c, 1) for c in r['cuts']] == [1.5]+ it = r['sync']['cuts']['items'][0]+ assert it['status'] in ('OK', 'near') and abs(it['offset_ms']) < 50+ s = sheet_mod.sheet(mp4, str(tmp_path / 'film.png'), tl=r, width=1200)+ assert s['thumbnails'] >= 3 and s['cuts'] == 1+++def test_timeline_end_to_end_without_models(wav, no_models, tmp_path, monkeypatch):+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ p = wav(click_track([0.5, 1.5, 2.5, 3.5, 4.5, 5.5], 6.0), 'clicks.wav')+ r = tl.timeline(p, cues=[1.5, 2.52, 5.0])+ assert r['summary'] and r['text'].startswith(r['summary'])+ r['text'].encode('ascii')+ assert len([e for e in r['events'] if e['shape'] == 'hit' and e['prominence_db'] > 10]) == 6+ st = r['sync']['cues']['stats']+ assert st['n'] == 3 and st['no_sound'] == [5.0]+ assert 'NO HIT' in r['text'] and 'cue -20 ms OK' in r['text']+ json.dumps(r)+ assert tl.timeline(p, cues=[1.5, 2.52, 5.0])['cached'] is True+ z = tl.timeline(p, start=2.0, end=4.0, cache=False)+ assert z['start'] == 2.0 and z['end'] == 4.0+ assert sorted(round(e['t'], 1) for e in z['events'] if e['prominence_db'] > 10) == [2.5, 3.5]+++def test_sheet_png(wav, no_models, tmp_path, monkeypatch):+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ p = wav(click_track(np.arange(0.5, 8, 0.5), 8.0), 'beat.wav')+ out = tmp_path / 'sheet.png'+ r = sheet_mod.sheet(p, str(out), cues='1.0,2.0', width=1400)+ from PIL import Image+ im = Image.open(out)+ assert im.size == (r['width'], r['height']) and r['width'] == 1400+ assert r['labelled_events'] > 3 and 'Wrote' in r['summary']+ z = sheet_mod.sheet(p, str(tmp_path / 'zoom.png'), start=2.0, end=4.0)+ assert z['start'] == 2.0 and os.path.exists(z['png'])+++def test_label_staggering_never_overlaps():+ items = [((1, -k), 1.0 + 0.01 * k, f'HIT label number {k}') for k in range(30)]+ placed = tl_rows = sheet_mod._place_labels(items, 0.0, 10.0, 1800, 10.0, 4)+ assert 0 < len(placed) <= 30+ rows = {}+ for row, t, a, text in tl_rows:+ b = a + (len(text) * 10.0 + 16) / 180.0+ for a2, b2 in rows.get(row, []):+ assert b <= a2 + 1e-9 or a >= b2 - 1e-9+ rows.setdefault(row, []).append((a, b))+++def test_cli_registers():+ import argparse+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest='command')+ tl.add_cli(sub)+ sheet_mod.add_cli(sub)+ a = ap.parse_args(['timeline', 'x.wav', '--cues', '1,2', '--from', '1', '--to', '3', '--detail', 'low'])+ assert (a.start, a.end, a.detail, a.cues) == (1.0, 3.0, 'low', '1,2')+ b = ap.parse_args(['sheet', 'x.mp4', '-o', 'o.png', '--width', '1600'])+ assert b.png == 'o.png' and b.width == 1600+++def test_timeline_with_models(wav, tmp_path, monkeypatch):+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ p = wav(click_track([0.5, 1.5, 2.5, 3.5, 4.5, 5.5], 6.0), 'clicks.wav')+ r = tl.timeline(p, detail='low')+ tagged = [e for e in r['events'] if e.get('label')]+ assert tagged and all(e['kind'] in tl.EVENT_VOCAB for e in tagged)+ assert r['sections'] and r['sections'][0]['label']+ assert r['model_calls']['clap_event_windows'] >= len(tagged)+++def test_tag_background_covers_the_vocabulary():+ phrases = [t for ts in tl.EVENT_VOCAB.values() for t in ts]+ assert sorted(phrases) == sorted(tl.TAG_BG) and len(set(phrases)) == len(phrases)+++def test_digital_silence_and_edge_gaps(sine):+ lo = tl.loudness(np.zeros((5 * SR, 2), np.float32))+ assert lo['digital_silence'] and lo['integrated_lufs'] is None and lo['true_peak_dbtp'] is None+ assert lo['silences'][0]['where'] == 'whole file' and min(lo['momentary']['lufs']) >= -70+ x = np.concatenate([np.zeros(SR), np.tile(sine, 3), np.zeros(SR), np.tile(sine, 2), np.zeros(SR)])+ g = tl.loudness(x)['silences']+ assert [q.get('where') for q in g] == ['lead-in', None, 'tail'] and g[0]['start'] == 0.0+ assert [q.get('where') for q in tl.loudness(x, at_start=False, at_end=False)['silences']] == [None] * 3+++def test_level_peak_answers_a_soft_swell():+ rng = np.random.RandomState(5)+ t = np.arange(6 * SR) / SR+ x = (rng.randn(len(t)) * (0.01 + 0.2 * np.exp(-0.5 * ((t - 3.0) / 0.25) ** 2))).astype(np.float32)+ lv = tl.level_envs(x)+ pk = tl.level_peak(lv, 3.05)+ assert pk and abs(pk['t'] - 3.0) < 0.06 and pk['rise_db'] > 10+ assert tl.level_peak(lv, 1.0) is None+ it = tl.sync([3.05, 1.0], [], levels=lv)['items']+ assert it[0]['status'] == 'peak' and it[1]['status'] == 'no sound'+++def test_cue_errors_and_aliases():+ assert tl.read_cues('{"fps": 50, "events": [{"frame": 100}]}')[0][0]['t'] == 2.0+ assert tl.read_cues([{'seconds': 1.25}])[0][0]['t'] == 1.25+ with pytest.raises(ValueError, match='no fps'):+ tl.read_cues('{"events": [{"f": 120, "kind": "hit"}]}')+ with pytest.raises(ValueError, match='without a time'):+ tl.read_cues([{'kind': 'hit'}])+++def test_cover_art_is_not_a_video():+ art = {'has_video': True, 'video': {'fps': 90000.0}}+ assert not tl._is_video('song.mp3', art) and not tl._is_video('song.mp4', art)+ assert tl._is_video('film.mp4', {'has_video': True, 'video': {'fps': 30.0}})+ assert not tl._is_video('song.m4a', {'has_video': True, 'video': {'fps': 30.0}})+++def test_burst_takes_one_tag_slot():+ ev = [{'t': 1.0 + 0.06 * k, 'band': 'high', 'prominence_db': 30.0 - k * 0.1, 'contrast_db': 20.0} for k in range(10)]+ ev += [{'t': t, 'band': 'full', 'prominence_db': 10.0, 'contrast_db': 8.0} for t in (3.0, 5.0, 7.0, 9.0)]+ pick, _ = tl._pick(ev, 6, (), 0.0, 10.0)+ assert sum(ev[i]['t'] < 2 for i in pick) <= 2 and {3.0, 5.0, 7.0, 9.0} <= {ev[i]['t'] for i in pick}+++def test_zoom_keeps_full_file_verdicts(wav, no_models, tmp_path, monkeypatch):+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ p = wav(click_track(np.arange(0.5, 12, 0.5), 12.0), 'grid.wav')+ cues = [2.0, 4.02, 6.5, 7.97]+ full = {i['t']: (i['status'], i['offset_ms']) for i in tl.timeline(p, cues=cues)['sync']['cues']['items']}+ z = tl.timeline(p, cues=cues, start=4.0, end=8.0)+ assert z['context'] == [0.0, 12.0] and z['start'] == 4.0+ for i in z['sync']['cues']['items']:+ assert full[i['t']] == (i['status'], i['offset_ms'])+ assert all(4.0 <= e['t'] < 8.0 for e in z['events']) and all(4.0 <= b <= 8.0 for b in z['music']['beats'])+++def test_picture_only_video(no_models, tmp_path, monkeypatch):+ import subprocess+ monkeypatch.setattr(tl.config, 'EMB_CACHE', str(tmp_path / 'emb'))+ mp4 = str(tmp_path / 'silent.mp4')+ subprocess.run(['ffmpeg', '-y', '-v', 'error', '-f', 'lavfi', '-i', 'color=c=black:s=96x54:r=30:d=1.5', '-f', 'lavfi',+ '-i', 'color=c=white:s=96x54:r=30:d=1.5', '-filter_complex', '[0:v][1:v]concat=n=2:v=1:a=0[v]',+ '-map', '[v]', '-c:v', 'mpeg4', mp4], check=True)+ r = tl.timeline(mp4)+ assert r['has_audio'] is False and [round(c, 1) for c in r['cuts']] == [1.5]+ assert r['sync']['cuts']['items'][0]['status'] == 'no sound' and 'NO AUDIO STREAM' in r['summary']+++def test_ascii_keeps_non_ascii_names_recoverable():+ assert tl._ascii('t\xf6n.wav') == r't\xf6n.wav'