Skip to content

Ears for coding agents: let AI hear, review and make sound. CPU-only, open source.

Describe what realness measures instead of one user's taste (0.2.1)

Leonxlnx04f2d030aa63
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 22ed498..8d2ee7f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,19 @@
# Changelog
+## 0.2.1 - 2026-10-17
+
+Two fixes from a careful outside review (thanks, @markblundeberg).
+
+### Fixed
+
+- **Agent-facing text no longer states one user's preferences as instructions** (#1). The `realness` tool description
+ and the MCP server instructions told every agent to check every sound because "the user rejects synthetic-sounding
+ audio"; the earshot skill said to reject 'likely code-synthesized' sounds outright, and the real-sound skill and
+ several docs called the bundled launch-film profile "the user's". They now say what realness measures and leave the
+ decision to the current user: use it when real or professionally produced sound is wanted, and treat a synthesized
+ verdict as information when synthesized sound is the point. `critique`'s realness fix is scoped the same way. A
+ test keeps such phrasing out of tool descriptions, the server instructions, skills and docs.
+
## 0.2.0 - 2026-10-16
forge v2 and `earshot fit`.
diff --git a/docs/INTEGRATION.md b/docs/INTEGRATION.md
index e258783..6e1ed59 100644
--- a/docs/INTEGRATION.md
+++ b/docs/INTEGRATION.md
@@ -86,7 +86,7 @@ earshot adds hearing to the measure and lens steps:
| av-audit | `python3 scripts/av-audit.py out/preview.mp4 --cues out/qa/cues_sounded.json --stems out/stems` | Cinetic's own onset/sync audit on the real soundtrack; `cues_sounded.json` (written by mix.py) lists only events that received a sound, so deliberate silences are not counted as misses |
| sound-sync lens (`assets/critics/sound-sync.md`) | hand the lens `qa.json`, the earshot transcript, sheet and scorecard with `av.json` | the lens can name the exact rule / sample to change; with real recordings its fixes are design operations (`<rs>/templates/prompts/fixer.md`), not score.py parameters |
| verifier (`assets/critics/verifier.md`) | refute a sound issue with earshot evidence (`earshot match` on the placed sample, the transcript at that time) | fewer false P1s from reading code alone |
-| a synthesized score already exists | `python -m earshot realness public/audio/soundtrack.wav` (score.py's output) | shows (per 4 s window) where the synthesized score reads as code-made: the case for switching |
+| a synthesized score already exists | `python -m earshot realness public/audio/soundtrack.wav` (score.py's output) | shows (per 4 s window) where the synthesized score reads as code-made: useful when the brief asks for recorded sound |
## 4. A minimal change list for a future Cinetic
diff --git a/docs/LIMITS.md b/docs/LIMITS.md
index 33933ce..5082d49 100644
--- a/docs/LIMITS.md
+++ b/docs/LIMITS.md
@@ -22,7 +22,7 @@ earshot gives an agent measured evidence about sound, not taste. Read this befor
them apart as unvalidated hints. Instruments, rain, birds and voice are usually right.
- **"Uncertain" from realness** on designed library sounds ("magic" chimes/celesta, "motion" whooshes) and on real
electronic music: they sit between real and synthesized. Read it as "sounds designed", not "fake".
-- **The launch scorecard ranges** come from 7 films the user picked. They describe that style (modern SaaS / AI
+- **The launch scorecard ranges** come from 7 films the author picked. They describe that style (modern SaaS / AI
launch films, kinetic type, UI demos), not every good film. A cinematic short or a quiet piano piece will break
rules that do not apply to it. Build your own profile from films you admire (`earshot profile`).
- **The "cuts without a hit" list** (timeline, sheet, critique tips): soft build-ups (whooshes, risers, swells) can
diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md
index aebf107..b73257f 100644
--- a/docs/VALIDATION.md
+++ b/docs/VALIDATION.md
@@ -3,7 +3,7 @@
Every number here comes from a command in this repo, run on a Ryzen 7 260 laptop CPU (bf16, `EARSHOT_THREADS` 2-3,
other jobs running). Full tables, data descriptions and "what was tuned on what" live in each area's results file;
this page is the summary. Data: the Clawd film's licensed recordings (219 used + 125 rejected Freesound/Mixkit files),
-its finished 60 s soundtrack with every placement logged, the retired code-synthesized score the user rejected, renders
+its finished 60 s soundtrack with every placement logged, the code-synthesized score that project retired, renders
of the Cinetic synth engine, independent numpy generators, and 7 professional launch films (measured locally, never
committed).
@@ -50,7 +50,7 @@ committed).
|---|---|---|
| leave one real category / synth family out (headline) | 0.922 | 0.860 |
| leave one synth generator type out | 0.958 | 0.902 |
-| retired (user-rejected) score never trained on: real vs retired | **0.948** | 0.862 |
+| retired score never trained on: real vs retired | **0.948** | 0.862 |
| retired score + the whole engine that made it never trained on: real vs retired | 0.909 | 0.839 |
| zero-shot CLAP prompts (no training, baseline) | 0.858 | 0.749 |
diff --git a/docs/launch-video-sound.md b/docs/launch-video-sound.md
index 6c4d6be..c58bd21 100644
--- a/docs/launch-video-sound.md
+++ b/docs/launch-video-sound.md
@@ -1,11 +1,13 @@
# Launch-film sound: the playbook for coding agents
You cannot hear the film you are making. This page tells you what professional launch films do with sound, how to
-build that from real recordings, and how to check your result with earshot until the numbers say it is there.
+build that from real recordings, and how to check your result with earshot until the numbers say it is there. It is
+written for films meant to sound like those references; when synthesized sound is the point of a film, read the
+realness numbers as information, not as defects.
How to read it:
-- **[measured]** = measured by earshot on the 7 professional launch films the user chose as the quality bar (modern
+- **[measured]** = measured by earshot on the 7 professional launch films the author chose as the quality bar (modern
SaaS / AI product launches and motion reels, web-delivered MP4s with 128 kbps AAC audio). Numbers come from
`earshot/data/launch_profile.json`; the commands and the leave-one-out check are in `eval/launch/RESULTS.md`.
Seven films are a small sample: treat the numbers as "what pros did here", not as laws.
diff --git a/earshot/__init__.py b/earshot/__init__.py
index 5542aaa..7a92afd 100644
--- a/earshot/__init__.py
+++ b/earshot/__init__.py
@@ -12,7 +12,7 @@ or explicitly: `from earshot import api` / `from earshot.describe import describ
"""
import importlib
-__version__ = '0.2.0'
+__version__ = '0.2.1'
# modules that are callable as their main function (see _callable.py)
_CALLABLE = {'describe', 'match', 'compare', 'realness', 'timeline', 'sheet', 'critique', 'profile'}
diff --git a/earshot/critique.py b/earshot/critique.py
index 4a486df..5d1a990 100644
--- a/earshot/critique.py
+++ b/earshot/critique.py
@@ -55,7 +55,8 @@ SPEC = {
'a master above -1 dBTP clips once encoded to AAC / MP3 (a decoded lossy file above 0 dBTP already '
'clips on playback): crackle on the biggest hits'),
'p_real': (3, 0.0, 'lower', None,
- 'code-synthesized sound reads as cheap and generic; launch films use real recordings and produced music'),
+ 'the reference launch films use real recordings and produced music; next to them, code-synthesized '
+ 'sound reads as cheap and generic'),
'music_frac': (3, 0.05, 'lower', None,
'the references are music-led: one continuous track carries the energy and hides edit seams'),
'cuts_with_sound': (2, 0.05, 'lower', None,
@@ -239,8 +240,9 @@ def _actions(key, st, v, f, a, refs):
more = f' (+{len(flagged) - len(pick)} more)' if flagged and len(flagged) > len(pick) else ''
where = (f"; 4 s windows at or below the window threshold {tw['lo']:.2f} (code-synthesized): {spans}{more}"
if flagged else f'; lowest-scoring 4 s windows: {spans}' if spans else '')
- fix = ('replace oscillator / noise layers with real recordings and produced music (skills/real-sound has the '
- 'sourcing and licence workflow)' if v < 0.5 else 'find and replace the layer that sounds synthetic')
+ fix = ('if the film should sound recorded or produced, replace oscillator / noise layers with real recordings '
+ 'and produced music (skills/real-sound has the sourcing and licence workflow)' if v < 0.5 else
+ 'if the film should sound recorded or produced, find and replace the layer that sounds synthetic')
return [f'{head}{where}: {fix}; judge each stem with `earshot realness stem.wav` (a finished mix is harder to '
'judge than its sources)']
if key == 'music_frac':
diff --git a/earshot/mcp_server.py b/earshot/mcp_server.py
index edb6a6c..38c5ebc 100644
--- a/earshot/mcp_server.py
+++ b/earshot/mcp_server.py
@@ -42,7 +42,8 @@ from .report import _clean, emit
INSTRUCTIONS = """earshot gives you ears: you cannot hear, so use these tools whenever you must judge, choose or check
sound (sound effects, music, ambience, a mix, the audio of a video). Use absolute paths.
-Loops: choose an asset = rank (many files) -> find (best moment in the winner) -> describe + realness (verify).
+Loops: choose an asset = rank (many files) -> find (best moment in the winner) -> describe (verify; + realness when
+the user wants real or professionally produced sound).
Review a mix or film = timeline (what happens when, sync to cuts/cues) -> sheet (look at the picture) -> critique
(scorecard vs professional launch films). A/B a change = ab on before/after renders, then timeline on the new one.
measure = exact meters (LUFS, dBTP, noise floor), no models. llm_critique = optional Gemini opinion, uploads audio:
@@ -368,11 +369,13 @@ def tool_realness(path: Annotated[str, Field(description='an audio/video file, o
ast: Annotated[bool, Field(description='also run the AudioSet tagger for synth-class evidence '
'(slower)')] = False,
full: Full = False) -> str:
- """Real recording / professionally produced audio, or cheap code-synthesized sound (oscillators, filtered noise,
- simple envelopes, beeps)? p_real (a probability) with a verdict and the evidence in words; files over 6 s also get
- per-window verdicts; a folder or glob gives one row per file. Electronic music and pro sound design count as real:
- it flags amateur procedural audio. Use on every sound before it goes into a film and on finished mixes: the user
- rejects synthetic-sounding audio. Strong evidence, not proof: read the evidence list."""
+ """Does it sound like a real recording / professionally produced audio, or like simple code-synthesized sound
+ (oscillators, filtered noise, simple envelopes, beeps)? p_real (a probability) with a verdict and the evidence in
+ words; files over 6 s also get per-window verdicts; a folder or glob gives one row per file. Electronic music and
+ pro sound design count as real: it flags simple procedural audio. Use when the user wants real or professionally
+ produced sound (choosing recordings, checking a finished mix), or asks whether a sound reads as code-synthesized.
+ Synthesized audio may be intended: report the verdict, and treat it as a defect only when the user asked for real
+ sound. Strong evidence, not proof: read the evidence list."""
fn = _api('realness')
mod = sys.modules.get('earshot.realness')
model = getattr(mod, 'MODEL_PATH', None)
diff --git a/eval/forge/RESULTS.md b/eval/forge/RESULTS.md
index 4ec4259..dc69654 100644
--- a/eval/forge/RESULTS.md
+++ b/eval/forge/RESULTS.md
@@ -134,8 +134,8 @@ How to read it:
progress: they are already above real.
4. **Tails.** Made clicks have t40 0.56x the bench median, against the real refs' 0.86-1.33. Made whooshes are at
0.51x, against 0.6-1.08. Both are short tails. Made impacts are at 0.35-0.62x the single held-out impact's 3.45 s
- tail. But the two forge impacts are just as short (0.38x, 0.54x), so one bench ref cannot settle it. The user's ear
- ("the tail died too fast") points the same way.
+ tail. But the two forge impacts are just as short (0.38x, 0.54x), so one bench ref cannot settle it. One listener's
+ note ("the tail died too fast") points the same way.
5. **Phrasings are saturated.** Phrase z and margin are above real in every category, as expected, so don't read
progress from them. AudioSet naming is mixed: made sounds beat real for pop (1 vs 0.29 / 0.67) and impact (1 vs
0.5), but fall below it for click (0 vs 0.12 / 0), typing_key (0.33 vs 0.53 / 0.42) and whoosh_fast (0.33 vs 0.4 /
diff --git a/eval/launch/RESULTS.md b/eval/launch/RESULTS.md
index 61e71ec..3bac0cc 100644
--- a/eval/launch/RESULTS.md
+++ b/eval/launch/RESULTS.md
@@ -1,6 +1,6 @@
# Launch profile + critique: evaluation
-Generated by `eval/launch/run_eval.py` (numbers only; the films are the user's local reference files, never copied). Profile = `earshot/data/launch_profile.json`, built from the 7 references below.
+Generated by `eval/launch/run_eval.py` (numbers only; the films are local reference files of the author, never copied). Profile = `earshot/data/launch_profile.json`, built from the 7 references below.
## Commands
@@ -108,7 +108,7 @@ Caught: 13 of 14 known-bad mixes have the expected metric flagged (missed: squas
## The Clawd film: real soundtrack vs the retired synthesized soundtrack
-Reading: the scorecard rates the real soundtrack close (90%) and the synthesized one needs work (80%). The structural metrics separate them only a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has one loudness peak that sticks out. The difference the user hears most - synthesized timbre - is not structural; it is what p_real measures: real 0.94 vs synth 0.33 (earshot.realness, verdict thresholds lo 0.4 / hi 0.77 at this run); below 0.5 the p_real row is critical, which is what moves the synth version's verdict. Caveat: the realness model (another module) was trained partly on these 7 launch films, on the Clawd project's real asset library and on the retired synthesized stems (eval/realness/), so p_real on the Clawd films is not an independent test.
+Reading: the scorecard rates the real soundtrack close (90%) and the synthesized one needs work (80%). The structural metrics separate them only a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has one loudness peak that sticks out. The difference a listener hears most - synthesized timbre - is not structural; it is what p_real measures: real 0.94 vs synth 0.33 (earshot.realness, verdict thresholds lo 0.4 / hi 0.77 at this run); below 0.5 the p_real row is critical, which is what moves the synth version's verdict. Caveat: the realness model (another module) was trained partly on these 7 launch films, on the Clawd project's real asset library and on the retired synthesized stems (eval/realness/), so p_real on the Clawd films is not an independent test.
### clawd_real: Clawd short film with its real-recordings soundtrack (licensed music, foley, ambience)
diff --git a/eval/launch/run_eval.py b/eval/launch/run_eval.py
index bc289cc..d8e48de 100644
--- a/eval/launch/run_eval.py
+++ b/eval/launch/run_eval.py
@@ -244,7 +244,7 @@ def _fmt(k, v):
def report(films, loo_rows, per_metric, cl, det, ht, neg=None):
L = ['# Launch profile + critique: evaluation', '',
- 'Generated by `eval/launch/run_eval.py` (numbers only; the films are the user\'s local reference files, '
+ 'Generated by `eval/launch/run_eval.py` (numbers only; the films are local reference files of the author, '
'never copied). Profile = `earshot/data/launch_profile.json`, built from the 7 references below.', '',
'## Commands', '',
'```',
@@ -358,7 +358,7 @@ def report(films, loo_rows, per_metric, cl, det, ht, neg=None):
f"synthesized one {s_['verdict']} ({100 * s_['score']:.0f}%). The structural metrics separate them only "
'a little: both are loud enough, continuous and on the picture; the synth version fades away at the end, '
'is sparser and has a long fade, while the real one starts late (a 1 s near-silent cold open) and has '
- 'one loudness peak that sticks out. The difference the user hears most - synthesized timbre - is not '
+ 'one loudness peak that sticks out. The difference a listener hears most - synthesized timbre - is not '
'structural; it is what p_real measures' +
(f": real {pr:.2f} vs synth {ps:.2f} (earshot.realness, verdict thresholds lo {real_thresholds()['lo']} "
f"/ hi {real_thresholds()['hi']} at this run); below 0.5 the p_real row is critical, which is what "
diff --git a/eval/realness/RESULTS.md b/eval/realness/RESULTS.md
index c53b8f7..4232868 100644
--- a/eval/realness/RESULTS.md
+++ b/eval/realness/RESULTS.md
@@ -48,8 +48,8 @@ Every clip is scored by a model that never saw its group (nor augmented copies o
D is scored on real vs the retired score and the cinetic engine only (its sole negatives were the generic generators).
### The required check: real recordings rank above the retired synthesized score
-The retired score (the synthesized soundtrack the user rejected) is held out of every training set (scheme C), or held
-out together with the whole cinetic engine that made it (scheme D):
+The retired score (the synthesized soundtrack the Clawd project replaced) is held out of every training set
+(scheme C), or held out together with the whole cinetic engine that made it (scheme D):
| check | ROC-AUC | [email protected] | mean p_real of the synth side |
|---|---|---|---|
diff --git a/eval/realness/run_eval.py b/eval/realness/run_eval.py
index d6a3ef5..37b49e3 100644
--- a/eval/realness/run_eval.py
+++ b/eval/realness/run_eval.py
@@ -681,8 +681,8 @@ Every clip is scored by a model that never saw its group (nor augmented copies o
D is scored on real vs the retired score and the cinetic engine only (its sole negatives were the generic generators).
### The required check: real recordings rank above the retired synthesized score
-The retired score (the synthesized soundtrack the user rejected) is held out of every training set (scheme C), or held
-out together with the whole cinetic engine that made it (scheme D):
+The retired score (the synthesized soundtrack the Clawd project replaced) is held out of every training set
+(scheme C), or held out together with the whole cinetic engine that made it (scheme D):
""", _tab(['check', 'ROC-AUC', '[email protected]', 'mean p_real of the synth side'],
[[k, v.get('auc'), v.get('[email protected]'), v.get('synth_mean_p')] for k, v in res['retired_check'].items()]),
diff --git a/eval/timeline/run_eval.py b/eval/timeline/run_eval.py
index b14d0d1..e7df9ae 100644
--- a/eval/timeline/run_eval.py
+++ b/eval/timeline/run_eval.py
@@ -1,5 +1,5 @@
"""Evaluate earshot.timeline / earshot.sheet against the Clawd film's ground truth (never used for tuning) and on the
-user's reference launch films.
+7 reference launch films of the launch profile.
EARSHOT_THREADS=2 .venv/Scripts/python.exe eval/timeline/run_eval.py --parts detect,tags,sync,film,refs,timing60,timing93
diff --git a/pyproject.toml b/pyproject.toml
index d5471a8..50cdee4 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "earshot"
-version = "0.2.0"
+version = "0.2.1"
description = "Ears for coding agents: describe, match, search, compare, time-align and critique sound files an agent cannot hear."
readme = "README.md"
requires-python = ">=3.10"
diff --git a/skills/earshot/SKILL.md b/skills/earshot/SKILL.md
index a668234..f658a92 100644
--- a/skills/earshot/SKILL.md
+++ b/skills/earshot/SKILL.md
@@ -1,6 +1,6 @@
---
name: earshot
-description: Listen with earshot whenever you must judge, choose, trim, verify or compare sound you cannot hear - picking a sound effect, music or ambience from candidates, finding the cleanest moment in a recording, checking that audio is real and not code-synthesized, reviewing a soundtrack or a video's audio (what happens when, sync to picture, loudness, mix), A/B-ing a mix change, or making a sound in code and improving it by listening (earshot forge). Use it before telling anyone how audio sounds.
+description: Listen with earshot whenever you must judge, choose, trim, verify or compare sound you cannot hear - picking a sound effect, music or ambience from candidates, finding the cleanest moment in a recording, checking whether audio reads as a real recording or as code-synthesized, reviewing a soundtrack or a video's audio (what happens when, sync to picture, loudness, mix), A/B-ing a mix change, or making a sound in code and improving it by listening (earshot forge). Use it before telling anyone how audio sounds.
---
# earshot: ears for an agent
@@ -49,8 +49,10 @@ including the audio of an mp4 (a video with no audio track says so). Errors are
'tagger unsure' or a near class (Zipper for a swish, Finger snapping for a click) is neutral; only an outright
different class (Speech, Music, Engine, Vehicle) is a red flag.
4. `earshot realness winner.ogg --start 1.12 --end 1.48` (or a whole folder) - judge the trim you will use: a
- multi-take file's p_real is a mean over every take and the silence between them. Reject 'likely code-synthesized';
- read the evidence of an 'uncertain' before using it, and write why you kept it. The verdict thresholds depend on
+ multi-take file's p_real is a mean over every take and the silence between them. When the user wants real or
+ professionally produced sound, do not use a 'likely code-synthesized' trim, and read the evidence of an
+ 'uncertain' one before using it (write why you kept it). When synthesized sound is the point (procedural music, UI
+ tones, sounds made with earshot forge), realness is information, not a gate. The verdict thresholds depend on
length (each summary names them): a sound up to 6 s uses the clip thresholds (lo 0.17 / hi 0.77), a longer file
the mean of its 4 s windows with weakly calibrated file thresholds (lo 0.40 / hi 0.77), so compare verdicts, not
raw p_real, across files of different lengths. Real recordings and professionally produced music and sound design
diff --git a/skills/real-sound/SKILL.md b/skills/real-sound/SKILL.md
index 339c21d..edb8bb4 100644
--- a/skills/real-sound/SKILL.md
+++ b/skills/real-sound/SKILL.md
@@ -13,7 +13,8 @@ time-stretches the music and masters; code never generates a sound.
The bar is a professional launch film: every sound caused by something on screen, one material palette, the music's
drop on the hero reveal, restraint everywhere else, a resolved logo, a tail into silence. Rules:
-`references/sound-design.md` (and the earshot repo's `docs/launch-video-sound.md` for the user's reference profile).
+`references/sound-design.md` (and the earshot repo's `docs/launch-video-sound.md` for the bundled reference profile:
+7 professional launch films).
`<skill>` below is this folder. **Python:** `{{PY}}` below (and in every prompt template) is ONE interpreter that has
numpy, scipy, soundfile and the earshot package: the earshot repo's venv
diff --git a/skills/real-sound/references/sound-design.md b/skills/real-sound/references/sound-design.md
index 430e611..ba00f0f 100644
--- a/skills/real-sound/references/sound-design.md
+++ b/skills/real-sound/references/sound-design.md
@@ -2,8 +2,8 @@
The rules the designers, critics and fixer work from. Numbers marked *(norm)* are measured norms of top-tier launch
films (from the Cinetic skill's measurements, `references/sound.md` §13 there); the rest is practice that held up on
-shipped films. The earshot repo's `docs/launch-video-sound.md` profiles the user's own reference films in depth; use it
-for the targets of a specific film, this file stands alone for the craft.
+shipped films. The earshot repo's `docs/launch-video-sound.md` profiles 7 professional reference launch films in depth
+(earshot's bundled launch profile); use it for the targets of a specific film, this file stands alone for the craft.
## 1. The bar
@@ -114,7 +114,7 @@ for the targets of a specific film, this file stands alone for the craft.
- Reject: clipped, low-bitrate (a cliff above 16 kHz), roomy foley, anything with voices or bumps inside the window you
need, anything whose licence you cannot read on its own page.
-## 10. The cheap tells (what made the user call a soundtrack "random ass sounds")
+## 10. The cheap tells (what makes a soundtrack sound like random stock effects)
Code-synthesized beeps, sine pings and noise sweeps; one sample machine-gunned; a whoosh on every cut; a boom on every
logo; sparkle / shimmer sweeps standing in for "magic"; reverb on tiny sounds; random pitch on typing; partial
diff --git a/tests/test_interfaces.py b/tests/test_interfaces.py
index 6153af0..f876b23 100644
--- a/tests/test_interfaces.py
+++ b/tests/test_interfaces.py
@@ -1,8 +1,10 @@
"""interfaces: the optional audio-LLM critique (HTTP mocked: no network, no key needed) and the MCP server (in-process
tool functions + a real stdio round trip with the mcp client)."""
import base64
+import glob
import json
import os
+import re
import sys
import numpy as np
@@ -440,6 +442,36 @@ def test_mcp_server_registers_all_tools():
assert ann.get('open_world_hint', ann.get('openWorldHint')) is True
+# one person's taste written as a rule for every agent (issue #1); conditional, generic phrasing such as "when the user
+# wants real sound" stays allowed
+PREFERENCE_AS_RULE = re.compile(
+ r"\bthe user (?:rejects|rejected|hates|hated)\b|\buser-rejected\b"
+ r"|\bthe user's (?:own )?(?:reference (?:profile|films)|ear)\b"
+ r"|\bthe user (?:chose|picked) as\b|\bfilms the user (?:chose|picked)\b|random ass", re.I)
+
+
+def test_agent_text_states_no_preference_as_a_rule():
+ """issue #1: tool descriptions, server instructions, skills and docs say what earshot measures and leave the
+ decision to the current user's own instructions (synthesized audio may be the point of a film)"""
+ import anyio
+ texts = {f'tool {t.name}': t.description for t in anyio.run(mcp_server.build().list_tools)}
+ texts['INSTRUCTIONS'] = mcp_server.INSTRUCTIONS
+ for pattern in ('README.md', 'docs/*.md', 'skills/**/*.md', 'eval/*/RESULTS.md'):
+ for path in glob.glob(os.path.join(REPO, pattern), recursive=True):
+ with open(path, encoding='utf-8') as f:
+ texts[os.path.relpath(path, REPO)] = f.read()
+ assert len(texts) > 30 and any(k.startswith('skills') for k in texts)
+ hits = [f'{k}: {m.group(0)!r}' for k, v in texts.items()
+ for m in PREFERENCE_AS_RULE.finditer(' '.join(v.split()))] # joined lines: phrases split by a wrap count
+ assert not hits, hits
+ realness = ' '.join(texts['tool realness'].split())
+ assert 'Use when the user wants real or professionally produced sound' in realness
+ assert 'Synthesized audio may be intended' in realness
+ assert 'realness when the user wants real' in ' '.join(mcp_server.INSTRUCTIONS.split())
+ skill = ' '.join(texts[os.path.join('skills', 'earshot', 'SKILL.md')].split())
+ assert "Reject 'likely code-synthesized'" not in skill and 'realness is information, not a gate' in skill
+
+
async def _stdio_session(calls, log, *args):
"""start `python -m earshot mcp` (or `python -m <args>`) as a subprocess and run calls(session) over stdio"""
import anyio