Skip to content

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

Refactor Whistle Module and CLI Integration

HenryNdubuaku8881c1ceada5
diff --git a/README.md b/README.md
index f39432d..1ae78af 100644
--- a/README.md
+++ b/README.md
@@ -5,6 +5,7 @@ A foundation model for mobiles, wearables, robots, smart home, automotive and mi
- **Tool calls**: given the functions your app exposes, Needle picks the right ones and fills every argument from what the user said. Ask for two things and you get two calls in order; ask for something no tool covers and you get an empty list, not a guess.
- **Structured extraction**: declare a shape, hand over messy text, get typed fields back: an invoice, a booking, a notification, a form. The decode grammar guarantees the output parses, and extraction generalises to classification.
- **Text embedding**: the same model returns a vector for a sentence, so an app can search, match and route locally.
+- **Speech**: [Whistle](#whistle), our speech-to-text model, shares Needle's container and engine. One build gives Needle audio input: the clip goes in, the tool calls come out.
![Needle 3 at a glance](assets/model.svg)
@@ -43,6 +44,35 @@ print(agent.run("what's it like in Lagos right now?")["results"])
Every turn returns one JSON object with `function_calls`, the model's `reasoning` and a calibrated `confidence`; an off-topic request returns an empty list rather than a guess. `needle.Needle(tools=[...], generation=2)` keeps running Needle 2 for existing deployments.
+## Whistle
+
+![One engine, three ways to load it](assets/whistle.svg)
+
+Whistle is our speech-to-text model, one 16.9 MB file on the CPU: 16 kHz mono audio, up to 30 seconds in one pass, in English, German, French, Spanish, Italian, Dutch and Polish. It shares Needle's `.cact` container, its quantisation and its C++ engine, so the two are one runtime, and its decoder is laddered the same way, with `--audio-depth N` running the N-layer rung of the same weights.
+
+```python
+import needle
+
+print(needle.transcribe("clip.wav")["text"])
+# turn off the kitchen lights
+```
+
+Every call returns the text, the language, the milliseconds to the first token and the decoder's tokens per second after it. `word_timestamps=True` adds each word with its start, end and probability, `keywords=[...]` favours the names and product words your users say, and `language="de"` forces the language instead of detecting it. `needle.Whistle()` is the same model as an object, for `embed(audio)` or to hold one tuned `.cact`. Silence and steady noise return an empty transcript rather than an invented sentence. `needle whistle playground` transcribes from the microphone, and `needle whistle compare` puts Whistle next to Whisper and Moonshine on the same clip.
+
+At 16.9 MB Whistle is 8.6x smaller than Whisper base and 6.6x quicker to the first token. It is ahead on LibriSpeech test-clean and test-other, on SPGISpeech, on Earnings-22 and on the FLEURS average. Whisper base is ahead on TED-LIUM, on AMI and on the MLS average.
+
+![Whistle against Whisper and Moonshine](assets/whistle-benchmarks.svg)
+
+Word error rate on the full test splits, scored with the Whisper normalizers. Whisper and Moonshine figures are the ones their authors published. Latency and decode are 10 s of audio on an Apple M4 Pro, at each engine's defaults. The per-benchmark caveats are on [Hugging Face](https://huggingface.co/Cactus-Compute/whistle).
+
+```sh
+needle --model needle3.cact --model whistle.cact --tools tools.json --audio clip.wav
+```
+
+One engine holds both models: `needle_load` reads whichever one a `.cact` carries, and `needle_complete` takes a clip wherever it takes text. It transcribes, answers the transcript against your tools, and returns one JSON object with the tool calls and the speech fields, the speech ones prefixed `audio_`. The transcription stays inside the engine, so audio in and tool calls out is one call.
+
+The benchmarks against Whisper and Moonshine, the architecture and the interactive demo are at [cactuscompute.com/whistle](https://cactuscompute.com/whistle); the weights and every platform engine are on [Hugging Face](https://huggingface.co/Cactus-Compute/whistle).
+
## Guides
- [How to design tools for Needle 3](https://cactuscompute.com/blog/designing-tools-for-needle): one tool per action, names users would say, formats in descriptions, constraints in the grammar, triggers.
@@ -103,7 +133,7 @@ Or hand the key to Claude Code or Codex with [cactuscompute.com/llms.txt](https:
## Deploy
-Every deployment target ships a prebuilt engine under 1 MB that loads the `needle3.cact` weights at start. `needle build --platform <folder> [--layers N]` fetches that engine and puts the weights beside it.
+Every deployment target ships a prebuilt engine that loads the `needle3.cact` weights at start. `needle build --platform <folder> [--layers N]` fetches that engine and puts the weights beside it.
![One engine per platform folder](assets/deploy.svg)
diff --git a/assets/deploy.svg b/assets/deploy.svg
index 01b6159..4ecbc51 100644
--- a/assets/deploy.svg
+++ b/assets/deploy.svg
@@ -1,8 +1,8 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 529" width="900" height="529" role="img" aria-label="The thirteen platform folders Needle ships, each with an engine under 1 MB">
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 529" width="900" height="529" role="img" aria-label="The thirteen platform folders Needle ships">
<style>svg{--t0:0.0s}.s{animation-duration:14.3s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}@keyframes t0{0%{stroke:#232326;fill:#141416}0.00%{stroke:#232326;fill:#141416}1.05%{stroke:#e8593d;fill:#1d1512}6.99%{stroke:#e8593d;fill:#1d1512}7.69%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t0{animation-name:t0}@keyframes f0{0%{fill:#85827a}0.00%{fill:#85827a}1.05%{fill:#e8593d}6.99%{fill:#e8593d}7.69%{fill:#85827a}100%{fill:#85827a}}.f0{animation-name:f0}@keyframes c0{0%{opacity:0}0.00%{opacity:0}0.84%{opacity:1}7.13%{opacity:1}7.69%{opacity:0}100%{opacity:0}}.c0{animation-name:c0}@keyframes t1{0%{stroke:#232326;fill:#141416}7.69%{stroke:#232326;fill:#141416}8.74%{stroke:#e8593d;fill:#1d1512}14.69%{stroke:#e8593d;fill:#1d1512}15.38%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t1{animation-name:t1}@keyframes f1{0%{fill:#85827a}7.69%{fill:#85827a}8.74%{fill:#e8593d}14.69%{fill:#e8593d}15.38%{fill:#85827a}100%{fill:#85827a}}.f1{animation-name:f1}@keyframes c1{0%{opacity:0}7.69%{opacity:0}8.53%{opacity:1}14.83%{opacity:1}15.38%{opacity:0}100%{opacity:0}}.c1{animation-name:c1}@keyframes t2{0%{stroke:#232326;fill:#141416}15.38%{stroke:#232326;fill:#141416}16.43%{stroke:#e8593d;fill:#1d1512}22.38%{stroke:#e8593d;fill:#1d1512}23.08%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t2{animation-name:t2}@keyframes f2{0%{fill:#85827a}15.38%{fill:#85827a}16.43%{fill:#e8593d}22.38%{fill:#e8593d}23.08%{fill:#85827a}100%{fill:#85827a}}.f2{animation-name:f2}@keyframes c2{0%{opacity:0}15.38%{opacity:0}16.22%{opacity:1}22.52%{opacity:1}23.08%{opacity:0}100%{opacity:0}}.c2{animation-name:c2}@keyframes t3{0%{stroke:#232326;fill:#141416}23.08%{stroke:#232326;fill:#141416}24.13%{stroke:#e8593d;fill:#1d1512}30.07%{stroke:#e8593d;fill:#1d1512}30.77%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t3{animation-name:t3}@keyframes f3{0%{fill:#85827a}23.08%{fill:#85827a}24.13%{fill:#e8593d}30.07%{fill:#e8593d}30.77%{fill:#85827a}100%{fill:#85827a}}.f3{animation-name:f3}@keyframes c3{0%{opacity:0}23.08%{opacity:0}23.92%{opacity:1}30.21%{opacity:1}30.77%{opacity:0}100%{opacity:0}}.c3{animation-name:c3}@keyframes t4{0%{stroke:#232326;fill:#141416}30.77%{stroke:#232326;fill:#141416}31.82%{stroke:#e8593d;fill:#1d1512}37.76%{stroke:#e8593d;fill:#1d1512}38.46%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t4{animation-name:t4}@keyframes f4{0%{fill:#85827a}30.77%{fill:#85827a}31.82%{fill:#e8593d}37.76%{fill:#e8593d}38.46%{fill:#85827a}100%{fill:#85827a}}.f4{animation-name:f4}@keyframes c4{0%{opacity:0}30.77%{opacity:0}31.61%{opacity:1}37.90%{opacity:1}38.46%{opacity:0}100%{opacity:0}}.c4{animation-name:c4}@keyframes t5{0%{stroke:#232326;fill:#141416}38.46%{stroke:#232326;fill:#141416}39.51%{stroke:#e8593d;fill:#1d1512}45.45%{stroke:#e8593d;fill:#1d1512}46.15%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t5{animation-name:t5}@keyframes f5{0%{fill:#85827a}38.46%{fill:#85827a}39.51%{fill:#e8593d}45.45%{fill:#e8593d}46.15%{fill:#85827a}100%{fill:#85827a}}.f5{animation-name:f5}@keyframes c5{0%{opacity:0}38.46%{opacity:0}39.30%{opacity:1}45.59%{opacity:1}46.15%{opacity:0}100%{opacity:0}}.c5{animation-name:c5}@keyframes t6{0%{stroke:#232326;fill:#141416}46.15%{stroke:#232326;fill:#141416}47.20%{stroke:#e8593d;fill:#1d1512}53.15%{stroke:#e8593d;fill:#1d1512}53.85%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t6{animation-name:t6}@keyframes f6{0%{fill:#85827a}46.15%{fill:#85827a}47.20%{fill:#e8593d}53.15%{fill:#e8593d}53.85%{fill:#85827a}100%{fill:#85827a}}.f6{animation-name:f6}@keyframes c6{0%{opacity:0}46.15%{opacity:0}46.99%{opacity:1}53.29%{opacity:1}53.85%{opacity:0}100%{opacity:0}}.c6{animation-name:c6}@keyframes t7{0%{stroke:#232326;fill:#141416}53.85%{stroke:#232326;fill:#141416}54.90%{stroke:#e8593d;fill:#1d1512}60.84%{stroke:#e8593d;fill:#1d1512}61.54%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t7{animation-name:t7}@keyframes f7{0%{fill:#85827a}53.85%{fill:#85827a}54.90%{fill:#e8593d}60.84%{fill:#e8593d}61.54%{fill:#85827a}100%{fill:#85827a}}.f7{animation-name:f7}@keyframes c7{0%{opacity:0}53.85%{opacity:0}54.69%{opacity:1}60.98%{opacity:1}61.54%{opacity:0}100%{opacity:0}}.c7{animation-name:c7}@keyframes t8{0%{stroke:#232326;fill:#141416}61.54%{stroke:#232326;fill:#141416}62.59%{stroke:#e8593d;fill:#1d1512}68.53%{stroke:#e8593d;fill:#1d1512}69.23%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t8{animation-name:t8}@keyframes f8{0%{fill:#85827a}61.54%{fill:#85827a}62.59%{fill:#e8593d}68.53%{fill:#e8593d}69.23%{fill:#85827a}100%{fill:#85827a}}.f8{animation-name:f8}@keyframes c8{0%{opacity:0}61.54%{opacity:0}62.38%{opacity:1}68.67%{opacity:1}69.23%{opacity:0}100%{opacity:0}}.c8{animation-name:c8}@keyframes t9{0%{stroke:#232326;fill:#141416}69.23%{stroke:#232326;fill:#141416}70.28%{stroke:#e8593d;fill:#1d1512}76.22%{stroke:#e8593d;fill:#1d1512}76.92%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t9{animation-name:t9}@keyframes f9{0%{fill:#85827a}69.23%{fill:#85827a}70.28%{fill:#e8593d}76.22%{fill:#e8593d}76.92%{fill:#85827a}100%{fill:#85827a}}.f9{animation-name:f9}@keyframes c9{0%{opacity:0}69.23%{opacity:0}70.07%{opacity:1}76.36%{opacity:1}76.92%{opacity:0}100%{opacity:0}}.c9{animation-name:c9}@keyframes t10{0%{stroke:#232326;fill:#141416}76.92%{stroke:#232326;fill:#141416}77.97%{stroke:#e8593d;fill:#1d1512}83.92%{stroke:#e8593d;fill:#1d1512}84.62%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t10{animation-name:t10}@keyframes f10{0%{fill:#85827a}76.92%{fill:#85827a}77.97%{fill:#e8593d}83.92%{fill:#e8593d}84.62%{fill:#85827a}100%{fill:#85827a}}.f10{animation-name:f10}@keyframes c10{0%{opacity:0}76.92%{opacity:0}77.76%{opacity:1}84.06%{opacity:1}84.62%{opacity:0}100%{opacity:0}}.c10{animation-name:c10}@keyframes t11{0%{stroke:#232326;fill:#141416}84.62%{stroke:#232326;fill:#141416}85.66%{stroke:#e8593d;fill:#1d1512}91.61%{stroke:#e8593d;fill:#1d1512}92.31%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t11{animation-name:t11}@keyframes f11{0%{fill:#85827a}84.62%{fill:#85827a}85.66%{fill:#e8593d}91.61%{fill:#e8593d}92.31%{fill:#85827a}100%{fill:#85827a}}.f11{animation-name:f11}@keyframes c11{0%{opacity:0}84.62%{opacity:0}85.45%{opacity:1}91.75%{opacity:1}92.31%{opacity:0}100%{opacity:0}}.c11{animation-name:c11}@keyframes t12{0%{stroke:#232326;fill:#141416}92.31%{stroke:#232326;fill:#141416}93.36%{stroke:#e8593d;fill:#1d1512}99.30%{stroke:#e8593d;fill:#1d1512}100.00%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t12{animation-name:t12}@keyframes f12{0%{fill:#85827a}92.31%{fill:#85827a}93.36%{fill:#e8593d}99.30%{fill:#e8593d}100.00%{fill:#85827a}100%{fill:#85827a}}.f12{animation-name:f12}@keyframes c12{0%{opacity:0}92.31%{opacity:0}93.15%{opacity:1}99.44%{opacity:1}100.00%{opacity:0}100%{opacity:0}}.c12{animation-name:c12}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}}</style>
<rect width="900" height="529" rx="18" fill="#0e0e10"/>
<text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">One engine per platform folder</text>
-<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">each under 1 MB, each loading the same needle3.cact</text>
+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">each loading any .cact you hand it</text>
<rect class="s t0" x="40" y="72" width="266" height="62" rx="9" fill="#141416" stroke="#232326" stroke-width="1.2"/>
<text x="54" y="94" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Mac (Apple Silicon)</text>
<text class="s f0" x="54" y="111" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">macos-arm64</text>
diff --git a/assets/whistle-benchmarks.svg b/assets/whistle-benchmarks.svg
new file mode 100644
index 0000000..20c189b
--- /dev/null
+++ b/assets/whistle-benchmarks.svg
@@ -0,0 +1,99 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 576" width="900" height="576" role="img" aria-label="Whistle against Whisper tiny, Whisper base and Moonshine tiny v2 on word error rate, size and speed">
+<style>
+svg{--t0:0.0s}.s{animation-duration:7.0s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}.s{animation-iteration-count:1;animation-fill-mode:forwards}@keyframes b0{0%{transform:scaleY(0)}3.00%{transform:scaleY(0)}7.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b0{animation-name:b0;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w0{0%{transform:scaleX(0)}3.00%{transform:scaleX(0)}7.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w0{animation-name:w0;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v0{0%{opacity:0}4.40%{opacity:0}8.00%{opacity:1}100%{opacity:1}}.v0{animation-name:v0}@keyframes b1{0%{transform:scaleY(0)}8.60%{transform:scaleY(0)}13.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b1{animation-name:b1;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w1{0%{transform:scaleX(0)}8.60%{transform:scaleX(0)}13.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w1{animation-name:w1;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v1{0%{opacity:0}10.00%{opacity:0}13.60%{opacity:1}100%{opacity:1}}.v1{animation-name:v1}@keyframes b2{0%{transform:scaleY(0)}14.20%{transform:scaleY(0)}18.60%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b2{animation-name:b2;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w2{0%{transform:scaleX(0)}14.20%{transform:scaleX(0)}18.60%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w2{animation-name:w2;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v2{0%{opacity:0}15.60%{opacity:0}19.20%{opacity:1}100%{opacity:1}}.v2{animation-name:v2}@keyframes b3{0%{transform:scaleY(0)}19.80%{transform:scaleY(0)}24.20%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b3{animation-name:b3;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w3{0%{transform:scaleX(0)}19.80%{transform:scaleX(0)}24.20%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w3{animation-name:w3;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v3{0%{opacity:0}21.20%{opacity:0}24.80%{opacity:1}100%{opacity:1}}.v3{animation-name:v3}@keyframes b4{0%{transform:scaleY(0)}25.40%{transform:scaleY(0)}29.80%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b4{animation-name:b4;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w4{0%{transform:scaleX(0)}25.40%{transform:scaleX(0)}29.80%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w4{animation-name:w4;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v4{0%{opacity:0}26.80%{opacity:0}30.40%{opacity:1}100%{opacity:1}}.v4{animation-name:v4}@keyframes b5{0%{transform:scaleY(0)}31.00%{transform:scaleY(0)}35.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b5{animation-name:b5;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w5{0%{transform:scaleX(0)}31.00%{transform:scaleX(0)}35.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w5{animation-name:w5;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v5{0%{opacity:0}32.40%{opacity:0}36.00%{opacity:1}100%{opacity:1}}.v5{animation-name:v5}@keyframes b6{0%{transform:scaleY(0)}36.60%{transform:scaleY(0)}41.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b6{animation-name:b6;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w6{0%{transform:scaleX(0)}36.60%{transform:scaleX(0)}41.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w6{animation-name:w6;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v6{0%{opacity:0}38.00%{opacity:0}41.60%{opacity:1}100%{opacity:1}}.v6{animation-name:v6}@keyframes b7{0%{transform:scaleY(0)}42.20%{transform:scaleY(0)}46.60%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b7{animation-name:b7;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w7{0%{transform:scaleX(0)}42.20%{transform:scaleX(0)}46.60%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w7{animation-name:w7;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v7{0%{opacity:0}43.60%{opacity:0}47.20%{opacity:1}100%{opacity:1}}.v7{animation-name:v7}@keyframes b8{0%{transform:scaleY(0)}47.80%{transform:scaleY(0)}52.20%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b8{animation-name:b8;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w8{0%{transform:scaleX(0)}47.80%{transform:scaleX(0)}52.20%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w8{animation-name:w8;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v8{0%{opacity:0}49.20%{opacity:0}52.80%{opacity:1}100%{opacity:1}}.v8{animation-name:v8}@keyframes b9{0%{transform:scaleY(0)}53.40%{transform:scaleY(0)}57.80%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b9{animation-name:b9;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w9{0%{transform:scaleX(0)}53.40%{transform:scaleX(0)}57.80%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w9{animation-name:w9;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v9{0%{opacity:0}54.80%{opacity:0}58.40%{opacity:1}100%{opacity:1}}.v9{animation-name:v9}@keyframes b10{0%{transform:scaleY(0)}59.00%{transform:scaleY(0)}63.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b10{animation-name:b10;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w10{0%{transform:scaleX(0)}59.00%{transform:scaleX(0)}63.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w10{animation-name:w10;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v10{0%{opacity:0}60.40%{opacity:0}64.00%{opacity:1}100%{opacity:1}}.v10{animation-name:v10}@keyframes b11{0%{transform:scaleY(0)}64.60%{transform:scaleY(0)}69.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b11{animation-name:b11;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w11{0%{transform:scaleX(0)}64.60%{transform:scaleX(0)}69.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w11{animation-name:w11;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v11{0%{opacity:0}66.00%{opacity:0}69.60%{opacity:1}100%{opacity:1}}.v11{animation-name:v11}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}.b0,.w0{transform:none}.v0{opacity:1}}</style>
+<rect width="900" height="576" rx="18" fill="#0e0e10"/>
+<text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">Whistle against Whisper and Moonshine</text>
+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">word error rate %, lower is better</text>
+<rect x="40" y="63" width="10" height="10" rx="2.5" fill="#e8593d"/>
+<text x="56" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Whistle</text>
+<rect x="106" y="63" width="10" height="10" rx="2.5" fill="#7b776f"/>
+<text x="122" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Whisper base</text>
+<rect x="203" y="63" width="10" height="10" rx="2.5" fill="#a7a299"/>
+<text x="219" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Moonshine tiny v2</text>
+<line x1="40" y1="364" x2="860" y2="364" stroke="#232326" stroke-width="1"/>
+<line x1="120" y1="233.3" x2="860" y2="233.3" stroke="#232326" stroke-width="1"/>
+<text x="110" y="236.8" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">10</text>
+<line x1="120" y1="164.7" x2="860" y2="164.7" stroke="#232326" stroke-width="1"/>
+<text x="110" y="168.2" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">20</text>
+<line x1="120" y1="96.0" x2="860" y2="96.0" stroke="#232326" stroke-width="1"/>
+<text x="110" y="99.5" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">30</text>
+<line x1="120" y1="302" x2="860" y2="302" stroke="#85827a" stroke-width="1"/>
+<rect class="s b0" x="130.1" y="272.4" width="18.0" height="29.6" rx="2" fill="#e8593d"/>
+<text class="s v0" x="139.1" y="260.4" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">4.31</text>
+<rect class="s b0" x="152.1" y="268.4" width="18.0" height="33.6" rx="2" fill="#7b776f"/>
+<rect class="s b0" x="174.1" y="271.0" width="18.0" height="31.0" rx="2" fill="#a7a299"/>
+<text class="s v0" x="161.1" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">LibriSpeech</text>
+<text class="s v0" x="161.1" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">test-clean</text>
+<rect class="s b1" x="212.3" y="230.0" width="18.0" height="72.0" rx="2" fill="#e8593d"/>
+<text class="s v1" x="221.3" y="213.6" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">10.49</text>
+<rect class="s b1" x="234.3" y="226.5" width="18.0" height="75.5" rx="2" fill="#7b776f"/>
+<rect class="s b1" x="256.3" y="221.6" width="18.0" height="80.4" rx="2" fill="#a7a299"/>
+<text class="s v1" x="243.3" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">LibriSpeech</text>
+<text class="s v1" x="243.3" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">test-other</text>
+<rect class="s b2" x="294.6" y="249.5" width="18.0" height="52.5" rx="2" fill="#e8593d"/>
+<text class="s v2" x="303.6" y="241.1" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">7.65</text>
+<line class="s v2" x1="320.6" y1="297" x2="330.6" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<rect class="s b2" x="338.6" y="249.1" width="18.0" height="52.9" rx="2" fill="#a7a299"/>
+<text class="s v2" x="325.6" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">SPGISpeech</text>
+<rect class="s b3" x="376.8" y="171.5" width="18.0" height="130.5" rx="2" fill="#e8593d"/>
+<text class="s v3" x="385.8" y="148.1" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">19.01</text>
+<line class="s v3" x1="402.8" y1="297" x2="412.8" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<rect class="s b3" x="420.8" y="156.1" width="18.0" height="145.9" rx="2" fill="#a7a299"/>
+<text class="s v3" x="407.8" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">Earnings-22</text>
+<rect class="s b4" x="459.0" y="123.0" width="18.0" height="179.0" rx="2" fill="#e8593d"/>
+<text class="s v4" x="468.0" y="115.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">26.07</text>
+<rect class="s b4" x="481.0" y="154.4" width="18.0" height="147.6" rx="2" fill="#7b776f"/>
+<rect class="s b4" x="503.0" y="145.6" width="18.0" height="156.4" rx="2" fill="#a7a299"/>
+<text class="s v4" x="490.0" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">AMI</text>
+<rect class="s b5" x="541.2" y="145.0" width="18.0" height="157.0" rx="2" fill="#e8593d"/>
+<text class="s v5" x="550.2" y="137.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">22.87</text>
+<line class="s v5" x1="567.2" y1="297" x2="577.2" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<line class="s v5" x1="589.2" y1="297" x2="599.2" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<text class="s v5" x="572.2" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">AMI</text>
+<text class="s v5" x="572.2" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">cleaned</text>
+<rect class="s b6" x="623.4" y="249.7" width="18.0" height="52.3" rx="2" fill="#e8593d"/>
+<text class="s v6" x="632.4" y="241.7" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">7.61</text>
+<rect class="s b6" x="645.4" y="267.7" width="18.0" height="34.3" rx="2" fill="#7b776f"/>
+<rect class="s b6" x="667.4" y="263.3" width="18.0" height="38.7" rx="2" fill="#a7a299"/>
+<text class="s v6" x="654.4" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">TED-LIUM</text>
+<rect class="s b7" x="705.7" y="155.1" width="18.0" height="146.9" rx="2" fill="#e8593d"/>
+<text class="s v7" x="714.7" y="125.8" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">21.4</text>
+<rect class="s b7" x="727.7" y="133.8" width="18.0" height="168.2" rx="2" fill="#7b776f"/>
+<line class="s v7" x1="753.7" y1="297" x2="763.7" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<text class="s v7" x="736.7" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">FLEURS</text>
+<rect class="s b8" x="787.9" y="131.0" width="18.0" height="171.0" rx="2" fill="#e8593d"/>
+<text class="s v8" x="796.9" y="123.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">24.9</text>
+<rect class="s b8" x="809.9" y="143.4" width="18.0" height="158.6" rx="2" fill="#7b776f"/>
+<line class="s v8" x1="835.9" y1="297" x2="845.9" y2="297" stroke="#4a4844" stroke-width="1.5"/>
+<text class="s v8" x="818.9" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">MLS</text>
+<text x="40" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Size</text>
+<text x="40" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">MB, smaller is better</text>
+<line x1="40" y1="430" x2="287" y2="430" stroke="#232326" stroke-width="1"/>
+<rect class="s w9" x="40" y="446" width="21.5" height="9" rx="2" fill="#e8593d"/>
+<text class="s v9" x="287" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">16.9</text>
+<rect class="s w9" x="40" y="463" width="184.7" height="9" rx="2" fill="#7b776f"/>
+<text class="s v9" x="287" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">145.3</text>
+<rect class="s w9" x="40" y="480" width="53.3" height="9" rx="2" fill="#a7a299"/>
+<text class="s v9" x="287" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">41.9</text>
+<text x="327" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Time to first token</text>
+<text x="327" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">ms on a 10 s clip, smaller is better</text>
+<line x1="327" y1="430" x2="573" y2="430" stroke="#232326" stroke-width="1"/>
+<rect class="s w10" x="327" y="446" width="28.0" height="9" rx="2" fill="#e8593d"/>
+<text class="s v10" x="573" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">11.1</text>
+<rect class="s w10" x="327" y="463" width="184.7" height="9" rx="2" fill="#7b776f"/>
+<text class="s v10" x="573" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">73.2</text>
+<rect class="s w10" x="327" y="480" width="57.5" height="9" rx="2" fill="#a7a299"/>
+<text class="s v10" x="573" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">22.8</text>
+<text x="613" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Decode</text>
+<text x="613" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">tokens/s on a 10 s clip, larger is better</text>
+<line x1="613" y1="430" x2="860" y2="430" stroke="#232326" stroke-width="1"/>
+<rect class="s w11" x="613" y="446" width="184.7" height="9" rx="2" fill="#e8593d"/>
+<text class="s v11" x="860" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">1,319</text>
+<rect class="s w11" x="613" y="463" width="37.2" height="9" rx="2" fill="#7b776f"/>
+<text class="s v11" x="860" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">266</text>
+<rect class="s w11" x="613" y="480" width="36.7" height="9" rx="2" fill="#a7a299"/>
+<text class="s v11" x="860" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">262</text>
+<text x="40" y="524" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">Whistle's figures are labelled; every bar is to the same scale. Whisper and Moonshine word error rates</text>
+<text x="40" y="538" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">as published; Whisper's AMI figure is AMI-IHM. FLEURS and MLS average the seven languages, MLS the six</text>
+<text x="40" y="552" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">without English. Speed is 10 s of audio on an Apple M4 Pro, at each engine's defaults.</text>
+</svg>
diff --git a/assets/whistle.svg b/assets/whistle.svg
new file mode 100644
index 0000000..607de11
--- /dev/null
+++ b/assets/whistle.svg
@@ -0,0 +1,189 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 300" width="900" height="300" role="img" aria-label="One Needle engine with three loadouts: the speech model alone, both models together, and the text model alone">
+<style>
+svg{--t0:0.0s}.s{animation-duration:16.5s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}@keyframes s0r0{0%{opacity:0;transform:translateX(-8px)}1.20%{opacity:0;transform:translateX(-8px)}3.80%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r0{animation-name:s0r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s0r1{0%{opacity:0;transform:translateX(-8px)}3.40%{opacity:0;transform:translateX(-8px)}6.00%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r1{animation-name:s0r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s0r2{0%{opacity:0;transform:translateX(-8px)}5.60%{opacity:0;transform:translateX(-8px)}8.20%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r2{animation-name:s0r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep0{0%{opacity:0;transform:translateX(0)}3.20%{opacity:0;transform:translateX(0)}4.00%{opacity:1;transform:translateX(0)}30.33%{opacity:1;transform:translateX(500px)}31.13%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep0{animation-name:sweep0;animation-timing-function:linear}@keyframes s1r0{0%{opacity:0;transform:translateX(-8px)}34.53%{opacity:0;transform:translateX(-8px)}37.13%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r0{animation-name:s1r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s1r1{0%{opacity:0;transform:translateX(-8px)}36.73%{opacity:0;transform:translateX(-8px)}39.33%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r1{animation-name:s1r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s1r2{0%{opacity:0;transform:translateX(-8px)}38.93%{opacity:0;transform:translateX(-8px)}41.53%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r2{animation-name:s1r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep1{0%{opacity:0;transform:translateX(0)}36.53%{opacity:0;transform:translateX(0)}37.33%{opacity:1;transform:translateX(0)}63.67%{opacity:1;transform:translateX(500px)}64.47%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep1{animation-name:sweep1;animation-timing-function:linear}@keyframes s2r0{0%{opacity:0;transform:translateX(-8px)}67.87%{opacity:0;transform:translateX(-8px)}70.47%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r0{animation-name:s2r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s2r1{0%{opacity:0;transform:translateX(-8px)}70.07%{opacity:0;transform:translateX(-8px)}72.67%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r1{animation-name:s2r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s2r2{0%{opacity:0;transform:translateX(-8px)}72.27%{opacity:0;transform:translateX(-8px)}74.87%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r2{animation-name:s2r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep2{0%{opacity:0;transform:translateX(0)}69.87%{opacity:0;transform:translateX(0)}70.67%{opacity:1;transform:translateX(0)}97.00%{opacity:1;transform:translateX(500px)}97.80%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep2{animation-name:sweep2;animation-timing-function:linear}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}.s0r0,.s0r1,.s0r2{opacity:1;transform:none}.sweep0{opacity:1}}</style>
+<rect width="900" height="300" rx="18" fill="#0e0e10"/>
+<text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">One engine, three ways to load it</text>
+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">needle.cact, whistle.cact, or both</text>
+<line x1="40" y1="66" x2="860" y2="66" stroke="#232326" stroke-width="1"/>
+<line x1="40" y1="118" x2="860" y2="118" stroke="#232326" stroke-width="1"/>
+<line x1="40" y1="176" x2="860" y2="176" stroke="#232326" stroke-width="1"/>
+<line x1="40" y1="234" x2="860" y2="234" stroke="#232326" stroke-width="1"/>
+<g class="s s0r0">
+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>
+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model whistle.cact</text>
+</g>
+<g class="s s0r1">
+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>
+<rect class="s s0r1" x="150.0" y="141.8" width="3.7" height="10.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="157.4" y="140.8" width="3.7" height="12.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="164.7" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="172.1" y="138.8" width="3.7" height="16.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="179.4" y="138.0" width="3.7" height="18.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="186.8" y="137.3" width="3.7" height="19.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="194.1" y="136.9" width="3.7" height="20.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="201.5" y="136.8" width="3.7" height="20.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="208.8" y="137.1" width="3.7" height="19.9" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="216.2" y="137.7" width="3.7" height="18.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="223.5" y="138.6" width="3.7" height="16.7" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="230.9" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="238.2" y="141.1" width="3.7" height="11.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="245.6" y="142.4" width="3.7" height="9.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="252.9" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="260.3" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="267.6" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="275.0" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="282.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="289.7" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="297.1" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="304.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="311.8" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="319.1" y="145.3" width="3.7" height="3.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="326.5" y="144.2" width="3.7" height="5.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="333.8" y="142.8" width="3.7" height="8.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="341.2" y="141.1" width="3.7" height="11.9" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="348.5" y="139.2" width="3.7" height="15.7" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="355.9" y="137.2" width="3.7" height="19.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="363.2" y="135.3" width="3.7" height="23.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="370.6" y="133.6" width="3.7" height="26.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="377.9" y="132.3" width="3.7" height="29.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="385.3" y="131.4" width="3.7" height="31.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="392.6" y="131.0" width="3.7" height="32.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="400.0" y="131.1" width="3.7" height="31.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="407.4" y="131.7" width="3.7" height="30.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="414.7" y="132.8" width="3.7" height="28.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="422.1" y="134.2" width="3.7" height="25.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="429.4" y="135.8" width="3.7" height="22.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="436.8" y="137.5" width="3.7" height="19.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="444.1" y="139.1" width="3.7" height="15.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="451.5" y="140.6" width="3.7" height="12.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="458.8" y="141.8" width="3.7" height="10.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="466.2" y="142.9" width="3.7" height="8.3" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="473.5" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="480.9" y="144.2" width="3.7" height="5.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="488.2" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="495.6" y="145.1" width="3.7" height="3.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="502.9" y="145.5" width="3.7" height="3.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="510.3" y="144.9" width="3.7" height="4.2" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="517.6" y="144.1" width="3.7" height="5.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="525.0" y="143.2" width="3.7" height="7.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="532.4" y="142.3" width="3.7" height="9.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="539.7" y="141.4" width="3.7" height="11.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="547.1" y="140.7" width="3.7" height="12.6" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="554.4" y="140.1" width="3.7" height="13.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="561.8" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="569.1" y="139.7" width="3.7" height="14.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="576.5" y="140.0" width="3.7" height="14.1" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="583.8" y="140.4" width="3.7" height="13.2" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="591.2" y="141.0" width="3.7" height="11.9" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="598.5" y="141.8" width="3.7" height="10.5" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="605.9" y="142.5" width="3.7" height="9.0" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="613.2" y="143.1" width="3.7" height="7.8" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="620.6" y="143.6" width="3.7" height="6.9" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="627.9" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="635.3" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>
+<rect class="s s0r1" x="642.6" y="143.6" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<text x="672" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">clip.wav</text>
+</g>
+<rect class="s sweep0" x="150" y="128" width="1.5" height="38" fill="#e8593d"/>
+<g class="s s0r2">
+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>
+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"text":"turn off the kitchen lights","language":"en",</text>
+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "ttft_ms":11.1,"decode_tps":1319.0}</text>
+</g>
+<g class="s s1r0">
+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>
+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model needle3.cact --model whistle.cact --tools tools.json</text>
+</g>
+<g class="s s1r1">
+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>
+<rect class="s s1r1" x="150.0" y="141.8" width="3.7" height="10.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="157.4" y="140.8" width="3.7" height="12.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="164.7" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="172.1" y="138.8" width="3.7" height="16.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="179.4" y="138.0" width="3.7" height="18.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="186.8" y="137.3" width="3.7" height="19.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="194.1" y="136.9" width="3.7" height="20.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="201.5" y="136.8" width="3.7" height="20.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="208.8" y="137.1" width="3.7" height="19.9" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="216.2" y="137.7" width="3.7" height="18.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="223.5" y="138.6" width="3.7" height="16.7" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="230.9" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="238.2" y="141.1" width="3.7" height="11.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="245.6" y="142.4" width="3.7" height="9.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="252.9" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="260.3" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="267.6" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="275.0" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="282.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="289.7" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="297.1" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="304.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="311.8" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="319.1" y="145.3" width="3.7" height="3.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="326.5" y="144.2" width="3.7" height="5.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="333.8" y="142.8" width="3.7" height="8.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="341.2" y="141.1" width="3.7" height="11.9" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="348.5" y="139.2" width="3.7" height="15.7" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="355.9" y="137.2" width="3.7" height="19.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="363.2" y="135.3" width="3.7" height="23.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="370.6" y="133.6" width="3.7" height="26.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="377.9" y="132.3" width="3.7" height="29.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="385.3" y="131.4" width="3.7" height="31.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="392.6" y="131.0" width="3.7" height="32.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="400.0" y="131.1" width="3.7" height="31.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="407.4" y="131.7" width="3.7" height="30.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="414.7" y="132.8" width="3.7" height="28.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="422.1" y="134.2" width="3.7" height="25.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="429.4" y="135.8" width="3.7" height="22.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="436.8" y="137.5" width="3.7" height="19.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="444.1" y="139.1" width="3.7" height="15.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="451.5" y="140.6" width="3.7" height="12.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="458.8" y="141.8" width="3.7" height="10.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="466.2" y="142.9" width="3.7" height="8.3" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="473.5" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="480.9" y="144.2" width="3.7" height="5.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="488.2" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="495.6" y="145.1" width="3.7" height="3.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="502.9" y="145.5" width="3.7" height="3.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="510.3" y="144.9" width="3.7" height="4.2" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="517.6" y="144.1" width="3.7" height="5.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="525.0" y="143.2" width="3.7" height="7.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="532.4" y="142.3" width="3.7" height="9.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="539.7" y="141.4" width="3.7" height="11.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="547.1" y="140.7" width="3.7" height="12.6" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="554.4" y="140.1" width="3.7" height="13.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="561.8" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="569.1" y="139.7" width="3.7" height="14.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="576.5" y="140.0" width="3.7" height="14.1" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="583.8" y="140.4" width="3.7" height="13.2" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="591.2" y="141.0" width="3.7" height="11.9" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="598.5" y="141.8" width="3.7" height="10.5" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="605.9" y="142.5" width="3.7" height="9.0" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="613.2" y="143.1" width="3.7" height="7.8" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="620.6" y="143.6" width="3.7" height="6.9" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="627.9" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="635.3" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>
+<rect class="s s1r1" x="642.6" y="143.6" width="3.7" height="6.7" rx="1" fill="#85827a"/>
+<text x="672" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">clip.wav</text>
+</g>
+<rect class="s sweep1" x="150" y="128" width="1.5" height="38" fill="#e8593d"/>
+<g class="s s1r2">
+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>
+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"function_calls":[{"name":"set_lights","arguments":{"room":"kitchen","on":false}}],</text>
+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "confidence":0.94,"audio_text":"turn off the kitchen lights","audio_language":"en"}</text>
+</g>
+<g class="s s2r0">
+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>
+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model needle3.cact --tools tools.json</text>
+</g>
+<g class="s s2r1">
+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>
+<text x="150" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">turn off the kitchen lights</text>
+</g>
+<g class="s s2r2">
+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>
+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"function_calls":[{"name":"set_lights","arguments":{"room":"kitchen","on":false}}],</text>
+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "confidence":0.94}</text>
+</g>
+<text x="40" y="268" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">one binary: needle_load reads whichever model the .cact holds, and needle_complete answers text or a clip</text>
+</svg>
diff --git a/llms.txt b/llms.txt
index fea901d..2ad06d9 100644
--- a/llms.txt
+++ b/llms.txt
@@ -26,6 +26,8 @@ pip install "cactus-needle[train]" # adds JAX + training deps for finetune
- `needle.tool` - decorator that turns a function into a tool schema (attached as `fn._needle_tool`); `@needle.tool(triggers=[r"\b(turn|switch)\b.*\b(on|off)\b"])` adds case-insensitive regexes that route a matching request to this tool and require a call; a triggered call ships even below the confidence floor (raw schemas take the same `"triggers"` list beside `description`).
- `needle.Field(default=..., *, description, enum, const, ge, le, gt, lt, multiple_of, min_length, max_length, pattern, format, min_items, max_items, unique_items)` - per-argument constraints; attach inline with `typing.Annotated`.
- `needle.extract(text, schema, system=None, max_new_tokens=512, weights=None)` - one-shot extraction; returns a Pydantic instance if `schema` is a model, else a dict (or `None` if nothing matched). `weights` selects a tuned `.cact` and defaults to whatever the engine already has loaded, so extraction inherits the active tuned model unless you pass another path.
+- `needle.transcribe(audio, language=None, keywords=None, word_timestamps=False, weights=None) -> dict` - speech to text in English, German, French, Spanish, Italian, Dutch and Polish, on the same engine and the same install as Needle. `audio` is a WAV path or 16 kHz mono float samples in [-1, 1], at most 30 s. Returns `{"text", "language", "ttft_ms", "decode_tps"}`, plus `"words"` with `start`, `end` and `probability` when `word_timestamps` is set. `language` is one of `en de fr es it nl pl`, or `None` to detect it; `keywords` is a list or a newline-separated string of words and phrases to favour. Silence and steady noise return an empty text and language. 16 kHz WAV files and raw samples need nothing extra; other sample rates need the `[mic]` extra for soxr. `whistle.cact` is fetched from `Cactus-Compute/whistle` on first use.
+- `needle.Whistle(weights=None, buffer_size=262144)` - the same model as an object, for `whistle.embed(audio)` (the encoder output, one row of floats per 80 ms frame, flattened) or to hold one tuned `.cact`. One model per process, not thread-safe.
## Defining tools (three equivalent ways)
@@ -234,9 +236,9 @@ Adapt by swapping the Literal values for your own and keeping the shapes: closed
- `needle run --checkpoint <base.safetensors> --query "..." --tools tools.json` - JAX reference inference from a checkpoint (dev path; normal inference is the Python `Needle` API above).
- `needle generate-data` / `needle finetune` / `needle build [checkpoint] [--lora adapter] [--layers N] [--platform <folder>] [--out <path>]` - the fine-tuning pipeline; fine-tuning runs at the full 20 layers, `build` merges the adapter, slices the `--layers` rung (default 20), exports at 4 bits, and fetches the published `needle3.cact` on every run (without an adapter at full depth it ships that published 2-bit archive as is). `--platform` also downloads that platform's engine and header and makes `--out` a folder holding them plus `needle3.cact`.
- `needle playground` - the browser UI locally (hosted at https://cactuscompute.com/needle).
-- `needle whistle playground [clip.wav] [--language en] [--keywords "Siobhan, Krzysztof"] [--word-timestamps] [--weights whistle.cact]` - Whistle speech to text in the terminal: press Enter to speak, Enter again to stop, or pass a WAV file; `/language`, `/keywords`, `/timestamps` and `/file` change the settings between recordings. Needs the `[whistle]` extra. The Whistle engine and `whistle.cact` are fetched from `Cactus-Compute/whistle` on first use; `NEEDLE_WHISTLE_LIB_PATH` and `NEEDLE_WHISTLE_WEIGHTS` point at local ones.
-- `needle whistle compare [clip.wav] [--weights whistle.cact]` - the same English speech through Whistle, Whisper tiny and base (openai-whisper) and Moonshine tiny v2 (moonshine-voice), one line per model with its weight size, time to first token, decode tokens per second, total time and transcript; `/file` runs a WAV between recordings. Whistle's timing comes from the engine's own JSON. Needs the `[whistle,whistle-compare]` extras.
-- `needle whistle fetch [--platform-tag <tag>] [--out <dir>]` / `needle whistle download whistle | <platform> [--out <dir>]` - the Whistle counterparts of `needle fetch` and `needle download`: pre-download the Whistle engine library, the `whistle.cact` weights, or a platform folder with the `whistle` binary, `libwhistle.a` and `whistle.h`.
+- `needle whistle playground [clip.wav] [--language en] [--keywords "Siobhan, Krzysztof"] [--word-timestamps] [--weights whistle.cact]` - Whistle speech to text in the terminal: press Enter to speak, Enter again to stop, or pass a WAV file; `/language`, `/keywords`, `/timestamps` and `/file` change the settings between recordings. Recording from the microphone needs the `[mic]` extra. `whistle.cact` is fetched from `Cactus-Compute/whistle` on first use and runs on the same engine as Needle; `NEEDLE_WHISTLE_WEIGHTS` points at a local copy and `NEEDLE3_LIB_PATH` at a local engine.
+- `needle whistle compare [clip.wav] [--weights whistle.cact]` - the same English speech through Whistle, Whisper tiny and base (openai-whisper) and Moonshine tiny v2 (moonshine-voice), one line per model with its weight size, time to first token, decode tokens per second, total time and transcript; `/file` runs a WAV between recordings. Whistle's timing comes from the engine's own JSON. Needs the `[mic,compare]` extras.
+- `needle download whistle [--out <dir>]` - pre-download the `whistle.cact` speech weights. There is one engine for both models, so `needle fetch` and `needle download <platform>` cover the library and the platform folders.
- `needle fetch [--platform-tag <tag>] [--out <dir>]` - pre-download the inference engine for this machine (or another platform) into the cache; prints the path. For air-gapped devices, also see `NEEDLE3_LIB_PATH` and `HF_HUB_OFFLINE` at https://cactuscompute.com/blog/needle-python-docs.
- `needle platform finetune | generate | jobs | models | files | billing` - the hosted platform from the CLI (section above).
- `needle download needle3 | needle3.safetensors | model-<id> | <org>/<repo>[/<file>.cact] | <platform> [--out <dir>]` - pull the base Needle 3 archive, a checkpoint to fine-tune, a platform-trained model by id (`--depth N` for one size), a published `.cact` (single-archive repos need only `<org>/<repo>`), or pass a platform folder name (`macos-arm64`, `linux-x86_64`, `wasm`, `wasm-component`, ...) to fetch that platform's engine files into `<out>/<platform>/`. Native runners are marked executable; `wasm-component` contains the WASI Preview 2 component and WIT contract.
diff --git a/needle/__init__.py b/needle/__init__.py
index d1ea2c7..e838d39 100644
--- a/needle/__init__.py
+++ b/needle/__init__.py
@@ -11,11 +11,11 @@ import warnings
from .agent.tools import Field, build_schema, pydantic_schema, tool, _is_pydantic_model
from ._telemetry import track as _track
from ._worker import FineTuneWorker
-from .whistle import Whistle
+from .agent.whistle import Whistle, transcribe
-__version__ = "3.0.1"
+__version__ = "3.1.0"
__all__ = ["Needle", "Whistle", "ExtractionValidationError", "tool", "Field", "extract",
- "__version__"]
+ "transcribe", "__version__"]
class ExtractionValidationError(ValueError):
@@ -168,11 +168,13 @@ def _lib(generation=2):
lib.needle_init.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p]
lib.needle_init.restype = ctypes.c_int
lib.needle_complete.argtypes = [
- ctypes.c_char_p, ctypes.c_int, ctypes.c_char_p, ctypes.c_int]
+ ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int,
+ ctypes.c_int, ctypes.c_char_p, ctypes.c_int]
lib.needle_complete.restype = ctypes.c_int
if generation >= 3:
lib.needle_embed.argtypes = [
- ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int]
+ ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int,
+ ctypes.POINTER(ctypes.c_float), ctypes.c_int]
lib.needle_embed.restype = ctypes.c_int
lib.needle_reset.argtypes = []
lib.needle_reset.restype = None
@@ -286,7 +288,7 @@ class Needle:
else:
lib = _lib(self._generation)
rc = lib.needle_complete(
- text.encode("utf-8"), int(max_new_tokens), self._buffer,
+ text.encode("utf-8"), None, 0, int(max_new_tokens), self._buffer,
len(self._buffer))
if rc < 0:
detail = self._buffer.value.decode("utf-8", "replace")
@@ -312,11 +314,11 @@ class Needle:
if self._worker is not None:
return self._worker.embed(text)
lib = _lib(self._generation)
- dim = lib.needle_embed(text.encode("utf-8"), None, 0)
+ dim = lib.needle_embed(text.encode("utf-8"), None, 0, None, 0)
if dim <= 0:
raise RuntimeError(f"needle_embed failed (code {dim})")
output = (ctypes.c_float * dim)()
- rc = lib.needle_embed(text.encode("utf-8"), output, dim)
+ rc = lib.needle_embed(text.encode("utf-8"), None, 0, output, dim)
if rc != dim:
raise RuntimeError(f"needle_embed failed (code {rc})")
return list(output)
diff --git a/needle/_worker.py b/needle/_worker.py
index df76cb7..4e461a9 100644
--- a/needle/_worker.py
+++ b/needle/_worker.py
@@ -52,7 +52,8 @@ def _load_library(path, generation=2):
lib.needle_init.argtypes = [ctypes.c_char_p, ctypes.c_char_p,
ctypes.c_char_p]
lib.needle_init.restype = ctypes.c_int
- lib.needle_complete.argtypes = [ctypes.c_char_p, ctypes.c_int,
+ lib.needle_complete.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_float),
+ ctypes.c_int, ctypes.c_int,
ctypes.c_char_p, ctypes.c_int]
lib.needle_complete.restype = ctypes.c_int
if int(generation) >= 3:
@@ -96,7 +97,7 @@ def _child():
operation = request.get("operation")
if operation == "complete":
code = lib.needle_complete(
- request["text"].encode("utf-8"),
+ request["text"].encode("utf-8"), None, 0,
int(request["max_new_tokens"]), output, len(output))
if code < 0:
_write_message(protocol, {
diff --git a/needle/agent/fetch.py b/needle/agent/fetch.py
index 44c79fa..b7bc93f 100644
--- a/needle/agent/fetch.py
+++ b/needle/agent/fetch.py
@@ -5,33 +5,27 @@ import sys
import zipfile
WHISTLE = "whistle"
-NEEDLE3_WHISTLE = "needle3_whistle"
ENGINE_REPOS = {
2: "Cactus-Compute/needle2",
3: "Cactus-Compute/needle3",
- NEEDLE3_WHISTLE: "Cactus-Compute/needle3",
WHISTLE: "Cactus-Compute/whistle",
}
ENGINE_VERSIONS = {
2: "2.0.4",
- 3: "3.0.3",
- NEEDLE3_WHISTLE: "3.0.0",
- WHISTLE: "1.0.0",
+ 3: "3.1.0",
+ WHISTLE: "2.0.0",
}
BASE_WEIGHTS = {
2: "needle2.cact",
3: "needle3.cact",
- NEEDLE3_WHISTLE: "needle3.cact",
WHISTLE: "whistle.cact",
}
-# Engines named instead of numbered: one library per name, beside the generations.
-LIB_PATH_ENVS = {
- NEEDLE3_WHISTLE: "NEEDLE3_WHISTLE_LIB_PATH",
- WHISTLE: "NEEDLE_WHISTLE_LIB_PATH",
-}
-NAMED_ENGINES = tuple(LIB_PATH_ENVS)
+# Weights published on their own, with no engine of their own: one engine runs them all.
+WEIGHTS_ONLY = (WHISTLE,)
+LIB_PATH_ENVS = {}
+NAMED_ENGINES = ()
CHECKPOINT_PREFIX = "checkpoints"
# Backwards-compatible aliases for callers that explicitly fetch Needle 2.
@@ -145,6 +139,8 @@ def unpublished_engine_wheels():
missing, listings = [], {}
for generation in ENGINE_VERSIONS:
+ if generation in WEIGHTS_ONLY:
+ continue
repo = ENGINE_REPOS[generation]
if repo not in listings:
# A repo that is private or not created yet holds none of its wheels.
@@ -216,12 +212,12 @@ def base_weights(generation=2):
def repo_prefix(generation=2):
- """Where an engine lives inside its repo. needle3_whistle shares needle3's."""
- return generation + "/" if generation == NEEDLE3_WHISTLE else ""
+ """Where an engine lives inside its repo."""
+ return ""
def cache_dir(generation=2):
- folder = generation if generation in NAMED_ENGINES else f"v{int(generation)}"
+ folder = generation if generation in NAMED_ENGINES + WEIGHTS_ONLY else f"v{int(generation)}"
return os.path.join(os.path.expanduser("~"), ".cache", "cactus-needle",
folder, engine_version(generation))
diff --git a/needle/agent/whistle.py b/needle/agent/whistle.py
new file mode 100644
index 0000000..9f854dc
--- /dev/null
+++ b/needle/agent/whistle.py
@@ -0,0 +1,365 @@
+from __future__ import annotations
+
+import array
+import ctypes
+import json
+import os
+import sys
+import time
+import wave
+
+SAMPLE_RATE = 16000
+LANGUAGES = ("en", "de", "fr", "es", "it", "nl", "pl")
+
+_handle = None
+_loaded = None
+
+
+def _weights_path():
+ from .. import _base_weights_path
+ from . import fetch
+
+ return os.environ.get("NEEDLE_WHISTLE_WEIGHTS") or _base_weights_path(fetch.WHISTLE)
+
+
+def _lib():
+ global _handle
+ if _handle is None:
+ from .. import _load_cdll
+
+ lib = _load_cdll(3)
+ samples = ctypes.POINTER(ctypes.c_float)
+ lib.needle_load.argtypes = [ctypes.c_char_p, ctypes.c_uint64]
+ lib.needle_load.restype = ctypes.c_int
+ lib.needle_transcribe.argtypes = [samples, ctypes.c_int, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int,
+ ctypes.c_char_p, ctypes.c_int]
+ lib.needle_transcribe.restype = ctypes.c_int
+ lib.needle_embed.argtypes = [ctypes.c_char_p, samples, ctypes.c_int, samples, ctypes.c_int]
+ lib.needle_embed.restype = ctypes.c_int
+ lib.needle_last_error.argtypes = []
+ lib.needle_last_error.restype = ctypes.c_char_p
+ _handle = lib
+ return _handle
+
+
+def _read_wav(path):
+ with wave.open(path, "rb") as source:
+ channels, width, rate = source.getnchannels(), source.getsampwidth(), source.getframerate()
+ raw = source.readframes(source.getnframes())
+ if width == 3:
+ values = [int.from_bytes(raw[i:i + 3], "little", signed=True) for i in range(0, len(raw), 3)]
+ else:
+ values = array.array({1: "B", 2: "h", 4: "i"}[width])
+ values.frombytes(raw)
+ if sys.byteorder == "big":
+ values.byteswap()
+ offset, scale = (128, 128.0) if width == 1 else (0, float(1 << (8 * width - 1)))
+ scale *= channels
+ mono = [(sum(frame) - offset * channels) / scale for frame in zip(*(values[c::channels] for c in range(channels)))]
+ if rate == SAMPLE_RATE or not mono:
+ return mono
+ try:
+ import numpy
+ import soxr
+ except ImportError:
+ raise RuntimeError(f"resampling {rate} Hz audio to {SAMPLE_RATE} Hz needs soxr: pip install cactus-needle[mic]") from None
+ return soxr.resample(numpy.asarray(mono, numpy.float32), rate, SAMPLE_RATE, quality="HQ")
+
+
+def _samples(audio):
+ if isinstance(audio, (str, os.PathLike)):
+ audio = _read_wav(os.fspath(audio))
+ data = array.array("f")
+ if isinstance(audio, (bytes, bytearray)):
+ data.frombytes(audio)
+ elif isinstance(audio, array.array) and audio.typecode == "f":
+ data = audio
+ elif hasattr(audio, "tobytes") and getattr(audio, "dtype", None) is not None:
+ data.frombytes(audio.astype("float32", copy=False).tobytes())
+ else:
+ data.extend(audio)
+ return (ctypes.c_float * len(data)).from_buffer(data) if len(data) else (ctypes.c_float * 1)(), len(data)
+
+
+class Whistle:
+ """Speech-to-text in English, German, French, Spanish, Italian, Dutch and Polish.
+
+ Audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.
+ One model is loaded per process; it is not thread-safe.
+ """
+
+ def __init__(self, weights=None, buffer_size=1 << 18):
+ self.weights = os.fspath(weights) if weights is not None else _weights_path()
+ self._buffer = ctypes.create_string_buffer(buffer_size)
+ self._bind()
+
+ def _bind(self):
+ global _loaded
+ lib = _lib()
+ if _loaded == self.weights:
+ return lib
+ with open(self.weights, "rb") as handle:
+ data = handle.read()
+ if lib.needle_load(data, len(data)) < 0:
+ _loaded = None
+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))
+ _loaded = self.weights
+ return lib
+
+ def transcribe(self, audio, language=None, keywords=None, word_timestamps=False) -> dict:
+ """Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start, end and probability when word_timestamps is set.
+
+ language is one of LANGUAGES, or None to detect it. keywords are words or phrases for keyword biasing.
+ """
+ lib = self._bind()
+ samples, count = _samples(audio)
+ if keywords is not None and not isinstance(keywords, str):
+ keywords = "\n".join(keywords)
+ code = lib.needle_transcribe(samples, count, language.encode("utf-8") if language else None,
+ keywords.encode("utf-8") if keywords else None, int(bool(word_timestamps)),
+ self._buffer, len(self._buffer))
+ if code < 0:
+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))
+ return json.loads(self._buffer.value.decode("utf-8", "replace"))
+
+ def embed(self, audio) -> list[float]:
+ """The encoder output, one row of floats per 80 ms frame, flattened."""
+ lib = self._bind()
+ samples, count = _samples(audio)
+ size = lib.needle_embed(None, samples, count, None, 0)
+ if size < 0:
+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))
+ output = (ctypes.c_float * size)()
+ if lib.needle_embed(None, samples, count, output, size) != size:
+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))
+ return list(output)
+
+
+_shared = {}
+
+
+def _shared_model(weights):
+ key = os.fspath(weights) if weights is not None else None
+ if key not in _shared:
+ _shared[key] = Whistle(weights=weights)
+ return _shared[key]
+
+
+def transcribe(audio, language=None, keywords=None, word_timestamps=False, weights=None) -> dict:
+ """Speech to text, on the model this process already has loaded.
+
+ audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.
+ Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start,
+ end and probability when word_timestamps is set. language is one of LANGUAGES,
+ or None to detect it; keywords are words or phrases for keyword biasing.
+ """
+ from .._telemetry import track
+
+ track("transcribe", {"tuned": bool(weights), "timestamps": bool(word_timestamps)})
+ return _shared_model(weights).transcribe(audio, language=language, keywords=keywords,
+ word_timestamps=word_timestamps)
+
+
+PLAYGROUND_HELP = """ Enter speak, Enter again to stop
+ /language de force a language (en de fr es it nl pl), bare to detect it
+ /keywords Siobhan, Krzysztof
+ /timestamps toggle word times
+ /file clip.wav transcribe a file
+ /quit leave"""
+COMPARE_HELP = """ Enter speak, Enter again to stop
+ /file clip.wav run a file through every model
+ /quit leave"""
+COMPARE_INSTALL = 'pip install "cactus-needle[mic,compare]"'
+CLEAR_ABOVE = "\x1b[1A\x1b[2K"
+LIMIT_SECONDS = 30
+
+
+def audio_path(rest):
+ """The path in a /file line, as a terminal hands it over: quoted, or with escaped spaces."""
+ path = rest.strip()
+ if len(path) > 1 and path[0] == path[-1] and path[0] in "'\"":
+ return path[1:-1]
+ return path.replace("\\ ", " ")
+
+
+def status(name, size, ttft, tokens, total):
+ ttft = f"{ttft * 1000:.0f} ms" if ttft is not None else "-"
+ tokens = f"{tokens:.0f} tok/s" if tokens is not None else "-"
+ spent = f"{total:.1f} s" if total >= 10 else f"{total * 1000:.0f} ms"
+ return f" {name:<18}{size / 1e6:>4.0f} MB ttft {ttft:>6} decode {tokens:>10} total {spent:>7}"
+
+
+def record():
+ try:
+ import numpy
+ import sounddevice
+ import soxr
+ except ImportError as error:
+ raise RuntimeError(f'{error.name} is not installed: pip install "cactus-needle[mic]"') from None
+ chunks = []
+ try:
+ rate = int(sounddevice.query_devices(kind="input")["default_samplerate"])
+ with sounddevice.InputStream(samplerate=rate, channels=1, dtype="float32", callback=lambda data, *_: chunks.append(data.copy())):
+ print(CLEAR_ABOVE + "● recording, Enter to stop", end="", flush=True)
+ try:
+ input()
+ except (EOFError, KeyboardInterrupt):
+ print()
+ except sounddevice.PortAudioError as error:
+ raise RuntimeError(f"no microphone: {error}") from None
+ print(CLEAR_ABOVE, end="", flush=True)
+ heard = numpy.concatenate(chunks)[:, 0] if chunks else numpy.zeros(0, numpy.float32)
+ audio = heard[:LIMIT_SECONDS * rate]
+ if len(audio) < len(heard):
+ print(f" keeping the first {LIMIT_SECONDS} s")
+ return audio if rate == SAMPLE_RATE else soxr.resample(audio, rate, SAMPLE_RATE, quality="HQ")
+
+
+def prompt():
+ try:
+ return input("› ").strip()
+ except (EOFError, KeyboardInterrupt):
+ print()
+ return "/quit"
+
+
+def show_transcript(whistle, audio, state):
+ started = time.perf_counter()
+ try:
+ result = whistle.transcribe(audio, language=state["language"], keywords=state["keywords"], word_timestamps=state["timestamps"])
+ except (RuntimeError, OSError, EOFError) as error:
+ print(f" {error}")
+ return
+ total = time.perf_counter() - started
+ print(result["text"] or "(no speech)")
+ for word in result.get("words", []):
+ print(f" {word['start']:6.2f} - {word['end']:5.2f} {word['word']:<20} {word['probability']:.2f}")
+ print(status("whistle", os.path.getsize(whistle.weights), result["ttft_ms"] / 1000, result["decode_tps"], total),
+ " ", result["language"] or "no speech")
+
+
+def playground(args):
+ state = {"language": args.language, "keywords": [k.strip() for k in args.keywords.split(",") if k.strip()], "timestamps": args.word_timestamps}
+ print("whistle playground: downloading and initializing the model...", flush=True)
+ whistle = Whistle(weights=args.weights)
+ if args.audio:
+ return show_transcript(whistle, args.audio, state)
+ print(PLAYGROUND_HELP)
+ while True:
+ line = prompt()
+ command, _, rest = line.partition(" ")
+ if command == "/quit":
+ return
+ if command == "/language":
+ language = rest.strip()
+ if language and language not in LANGUAGES:
+ print(" language is one of", " ".join(LANGUAGES))
+ continue
+ state["language"] = language or None
+ print(" language", state["language"] or "detected")
+ elif command in ("/keywords", "/keyword"):
+ state["keywords"] = [k.strip() for k in rest.split(",") if k.strip()]
+ print(" keywords", ", ".join(state["keywords"]) or "none")
+ elif command == "/timestamps":
+ state["timestamps"] = not state["timestamps"]
+ print(" word timestamps", "on" if state["timestamps"] else "off")
+ elif command == "/file":
+ show_transcript(whistle, audio_path(rest), state)
+ elif line == "":
+ try:
+ show_transcript(whistle, record(), state)
+ except RuntimeError as error:
+ print(f" {error}")
+ else:
+ print(PLAYGROUND_HELP)
+
+
+def rate(marks):
+ return (len(marks) - 1) / (marks[-1] - marks[0]) if len(marks) > 1 and marks[-1] > marks[0] else 0.0
+
+
+def load_whistle(weights):
+ whistle = Whistle(weights=weights)
+
+ def transcribe(audio):
+ result = whistle.transcribe(audio, language="en")
+ return result["text"], result["ttft_ms"] / 1000, result["decode_tps"]
+
+ return os.path.getsize(whistle.weights), transcribe
+
+
+def load_whisper(size):
+ import whisper
+
+ model = whisper.load_model(size, device="cpu")
+ cache = os.path.join(os.getenv("XDG_CACHE_HOME", os.path.join(os.path.expanduser("~"), ".cache")), "whisper")
+ marks = []
+ model.decoder.register_forward_hook(lambda *_: marks.append(time.perf_counter()))
+
+ def transcribe(audio):
+ marks.clear()
+ started = time.perf_counter()
+ text = model.transcribe(audio, language="en", fp16=False)["text"].strip()
+ return text, marks[0] - started, rate(marks)
+
+ return os.path.getsize(os.path.join(cache, f"{size}.pt")), transcribe
+
+
+def load_moonshine():
+ import moonshine_voice
+
+ path, arch = moonshine_voice.get_model_for_language("en", moonshine_voice.ModelArch.TINY_STREAMING)
+ model = moonshine_voice.Transcriber(model_path=path, model_arch=arch)
+
+ def transcribe(audio):
+ lines = model.transcribe_without_streaming(audio.tolist(), SAMPLE_RATE).lines
+ return " ".join(line.text.strip() for line in lines).strip(), None, None
+
+ return sum(os.path.getsize(os.path.join(path, f)) for f in os.listdir(path)), transcribe
+
+
+def run_models(models, audio):
+ for name, size, transcribe in models:
+ started = time.perf_counter()
+ text, ttft, tokens = transcribe(audio)
+ print(status(name, size, ttft, tokens, time.perf_counter() - started), " ", text or "(no speech)")
+
+
+def compare(args):
+ try:
+ import moonshine_voice # noqa: F401
+ import numpy
+ import whisper # noqa: F401
+ except ImportError as error:
+ raise SystemExit(f"{error.name} is not installed: {COMPARE_INSTALL}")
+ print("whistle compare: downloading and initializing the models...", flush=True)
+ models = [
+ ("whistle", *load_whistle(args.weights)),
+ ("whisper tiny", *load_whisper("tiny")),
+ ("whisper base", *load_whisper("base")),
+ ("moonshine tiny v2", *load_moonshine()),
+ ]
+ beat = numpy.arange(SAMPLE_RATE) * (numpy.pi / SAMPLE_RATE)
+ warmup = (numpy.sin(440 * beat) * numpy.abs(numpy.sin(3 * beat))).astype(numpy.float32)
+ for _, _, transcribe in models:
+ transcribe(warmup)
+ if args.audio:
+ return run_models(models, numpy.asarray(_read_wav(args.audio), numpy.float32))
+ print(COMPARE_HELP)
+ while True:
+ line = prompt()
+ command, _, rest = line.partition(" ")
+ if command == "/quit":
+ return
+ if command == "/file":
+ try:
+ run_models(models, numpy.asarray(_read_wav(audio_path(rest)), numpy.float32))
+ except OSError as error:
+ print(f" {error}")
+ elif line == "":
+ try:
+ run_models(models, numpy.asarray(record(), numpy.float32))
+ except RuntimeError as error:
+ print(f" {error}")
+ else:
+ print(COMPARE_HELP)
diff --git a/needle/cli.py b/needle/cli.py
index 7ab4000..71521c0 100644
--- a/needle/cli.py
+++ b/needle/cli.py
@@ -14,7 +14,7 @@ HELP = """usage: needle <command> [options]
download needle3 | needle3.safetensors | <platform> | model-<id> | <org>/<repo>[/<file>.cact]
fetch fetch the engine library for this platform
playground serve the browser playground
- whistle speech to text: playground | compare | fetch | download
+ whistle speech to text: playground | compare
needle <command> --help for the options of one command.
Check the readme for the rest."""
@@ -265,7 +265,7 @@ def main():
p.add_argument("--port", type=int, default=7860)
p.add_argument("--host", type=str, default="127.0.0.1")
- from .whistle import LANGUAGES
+ from .agent.whistle import LANGUAGES
p = sub.add_parser("whistle")
verbs = p.add_subparsers(dest="verb")
v = verbs.add_parser("playground")
@@ -284,14 +284,6 @@ def main():
help="A WAV file to run through every model once (default: press Enter to speak)")
v.add_argument("--weights", type=str, default=None,
help="whistle.cact to load (default: the published weights)")
- v = verbs.add_parser("fetch")
- v.add_argument("--out", type=str, default=None,
- help="Directory to place the engine (default: the cache)")
- v.add_argument("--platform-tag", type=str, default=None,
- help="Fetch the build for another device, e.g. manylinux2014_aarch64")
- v = verbs.add_parser("download")
- v.add_argument("spec", type=str, help="whistle (the weights) or a platform folder (e.g. macos-arm64)")
- v.add_argument("--out", type=str, default=".", help="Directory to place the files")
args = parser.parse_args()
@@ -303,9 +295,6 @@ def main():
track("cli:" + args.command,
{"generation": getattr(args, "generation", 2)})
- if args.command == "whistle" and args.verb in ("fetch", "download"):
- args.command, args.generation = args.verb, "whistle"
-
if args.command == "run":
from .model.run import main as run_main
run_main(args)
@@ -341,7 +330,7 @@ def main():
path = fetch.fetch_weights(target, args.out)
print(f" {'weights':<9} {path} {os.path.getsize(path) / 1e6:.2f} MB")
if target == fetch.WHISTLE:
- print(f" {'next':<9} needle.Whistle(weights={path!r})")
+ print(f" {'next':<9} needle.transcribe('clip.wav', weights={path!r})")
else:
print(f" {'next':<9} needle.Needle(weights={path!r}, tools=[...])")
elif kind == "hosted":
@@ -389,9 +378,9 @@ def main():
playground_main(args)
elif args.command == "whistle":
if args.verb == "playground":
- from .whistle.playground import main as whistle_main
+ from .agent.whistle import playground as whistle_main
elif args.verb == "compare":
- from .whistle.compare import main as whistle_main
+ from .agent.whistle import compare as whistle_main
else:
- raise SystemExit("needle whistle playground | compare | fetch | download")
+ raise SystemExit("needle whistle playground | compare")
whistle_main(args)
diff --git a/needle/whistle/__init__.py b/needle/whistle/__init__.py
deleted file mode 100644
index bc4be7c..0000000
--- a/needle/whistle/__init__.py
+++ /dev/null
@@ -1,135 +0,0 @@
-from __future__ import annotations
-
-import array
-import ctypes
-import json
-import os
-import sys
-import wave
-
-SAMPLE_RATE = 16000
-LANGUAGES = ("en", "de", "fr", "es", "it", "nl", "pl")
-
-_handle = None
-_loaded = None
-
-
-def _weights_path():
- from .. import _base_weights_path
- from ..agent import fetch
-
- return os.environ.get("NEEDLE_WHISTLE_WEIGHTS") or _base_weights_path(fetch.WHISTLE)
-
-
-def _lib():
- global _handle
- if _handle is None:
- from .. import _load_cdll
- from ..agent import fetch
-
- lib = _load_cdll(fetch.WHISTLE)
- samples = ctypes.POINTER(ctypes.c_float)
- lib.whistle_load.argtypes = [ctypes.c_char_p, ctypes.c_uint64]
- lib.whistle_load.restype = ctypes.c_int
- lib.whistle_transcribe.argtypes = [samples, ctypes.c_int, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int,
- ctypes.c_char_p, ctypes.c_int]
- lib.whistle_transcribe.restype = ctypes.c_int
- lib.whistle_embed.argtypes = [samples, ctypes.c_int, samples, ctypes.c_int]
- lib.whistle_embed.restype = ctypes.c_int
- lib.whistle_last_error.argtypes = []
- lib.whistle_last_error.restype = ctypes.c_char_p
- _handle = lib
- return _handle
-
-
-def _read_wav(path):
- with wave.open(path, "rb") as source:
- channels, width, rate = source.getnchannels(), source.getsampwidth(), source.getframerate()
- raw = source.readframes(source.getnframes())
- if width == 3:
- values = [int.from_bytes(raw[i:i + 3], "little", signed=True) for i in range(0, len(raw), 3)]
- else:
- values = array.array({1: "B", 2: "h", 4: "i"}[width])
- values.frombytes(raw)
- if sys.byteorder == "big":
- values.byteswap()
- offset, scale = (128, 128.0) if width == 1 else (0, float(1 << (8 * width - 1)))
- scale *= channels
- mono = [(sum(frame) - offset * channels) / scale for frame in zip(*(values[c::channels] for c in range(channels)))]
- if rate == SAMPLE_RATE or not mono:
- return mono
- try:
- import numpy
- import soxr
- except ImportError:
- raise RuntimeError(f"resampling {rate} Hz audio to {SAMPLE_RATE} Hz needs soxr: pip install cactus-needle[whistle]") from None
- return soxr.resample(numpy.asarray(mono, numpy.float32), rate, SAMPLE_RATE, quality="HQ")
-
-
-def _samples(audio):
- if isinstance(audio, (str, os.PathLike)):
- audio = _read_wav(os.fspath(audio))
- data = array.array("f")
- if isinstance(audio, (bytes, bytearray)):
- data.frombytes(audio)
- elif isinstance(audio, array.array) and audio.typecode == "f":
- data = audio
- elif hasattr(audio, "tobytes") and getattr(audio, "dtype", None) is not None:
- data.frombytes(audio.astype("float32", copy=False).tobytes())
- else:
- data.extend(audio)
- return (ctypes.c_float * len(data)).from_buffer(data) if len(data) else (ctypes.c_float * 1)(), len(data)
-
-
-class Whistle:
- """Speech-to-text in English, German, French, Spanish, Italian, Dutch and Polish.
-
- Audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.
- One model is loaded per process; it is not thread-safe.
- """
-
- def __init__(self, weights=None, buffer_size=1 << 18):
- self.weights = os.fspath(weights) if weights is not None else _weights_path()
- self._buffer = ctypes.create_string_buffer(buffer_size)
- self._bind()
-
- def _bind(self):
- global _loaded
- lib = _lib()
- if _loaded == self.weights:
- return lib
- with open(self.weights, "rb") as handle:
- data = handle.read()
- if lib.whistle_load(data, len(data)) < 0:
- _loaded = None
- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))
- _loaded = self.weights
- return lib
-
- def transcribe(self, audio, language=None, keywords=None, word_timestamps=False) -> dict:
- """Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start, end and probability when word_timestamps is set.
-
- language is one of LANGUAGES, or None to detect it. keywords are words or phrases for keyword biasing.
- """
- lib = self._bind()
- samples, count = _samples(audio)
- if keywords is not None and not isinstance(keywords, str):
- keywords = "\n".join(keywords)
- code = lib.whistle_transcribe(samples, count, language.encode("utf-8") if language else None,
- keywords.encode("utf-8") if keywords else None, int(bool(word_timestamps)),
- self._buffer, len(self._buffer))
- if code < 0:
- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))
- return json.loads(self._buffer.value.decode("utf-8", "replace"))
-
- def embed(self, audio) -> list[float]:
- """The encoder output, one row of floats per 80 ms frame, flattened."""
- lib = self._bind()
- samples, count = _samples(audio)
- size = lib.whistle_embed(samples, count, None, 0)
- if size < 0:
- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))
- output = (ctypes.c_float * size)()
- if lib.whistle_embed(samples, count, output, size) != size:
- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))
- return list(output)
diff --git a/needle/whistle/compare.py b/needle/whistle/compare.py
deleted file mode 100644
index c975260..0000000
--- a/needle/whistle/compare.py
+++ /dev/null
@@ -1,106 +0,0 @@
-import os
-import time
-
-from .playground import audio_path, prompt, record, status
-
-HELP = """ Enter speak, Enter again to stop
- /file clip.wav run a file through every model
- /quit leave"""
-INSTALL = 'pip install "cactus-needle[whistle,whistle-compare]"'
-
-
-def rate(marks):
- return (len(marks) - 1) / (marks[-1] - marks[0]) if len(marks) > 1 and marks[-1] > marks[0] else 0.0
-
-
-def load_whistle(weights):
- from . import Whistle
-
- whistle = Whistle(weights=weights)
-
- def transcribe(audio):
- result = whistle.transcribe(audio, language="en")
- return result["text"], result["ttft_ms"] / 1000, result["decode_tps"]
-
- return os.path.getsize(whistle.weights), transcribe
-
-
-def load_whisper(size):
- import whisper
-
- model = whisper.load_model(size, device="cpu")
- cache = os.path.join(os.getenv("XDG_CACHE_HOME", os.path.join(os.path.expanduser("~"), ".cache")), "whisper")
- marks = []
- model.decoder.register_forward_hook(lambda *_: marks.append(time.perf_counter()))
-
- def transcribe(audio):
- marks.clear()
- started = time.perf_counter()
- text = model.transcribe(audio, language="en", fp16=False)["text"].strip()
- return text, marks[0] - started, rate(marks)
-
- return os.path.getsize(os.path.join(cache, f"{size}.pt")), transcribe
-
-
-def load_moonshine():
- import moonshine_voice
-
- from . import SAMPLE_RATE
-
- path, arch = moonshine_voice.get_model_for_language("en", moonshine_voice.ModelArch.TINY_STREAMING)
- model = moonshine_voice.Transcriber(model_path=path, model_arch=arch)
-
- def transcribe(audio):
- lines = model.transcribe_without_streaming(audio.tolist(), SAMPLE_RATE).lines
- return " ".join(line.text.strip() for line in lines).strip(), None, None
-
- return sum(os.path.getsize(os.path.join(path, f)) for f in os.listdir(path)), transcribe
-
-
-def compare(models, audio):
- for name, size, transcribe in models:
- started = time.perf_counter()
- text, ttft, tokens = transcribe(audio)
- print(status(name, size, ttft, tokens, time.perf_counter() - started), " ", text or "(no speech)")
-
-
-def main(args):
- from . import SAMPLE_RATE, _read_wav
-
- try:
- import moonshine_voice # noqa: F401
- import numpy
- import whisper # noqa: F401
- except ImportError as error:
- raise SystemExit(f"{error.name} is not installed: {INSTALL}")
- print("whistle compare: downloading and initializing the models...", flush=True)
- models = [
- ("whistle", *load_whistle(args.weights)),
- ("whisper tiny", *load_whisper("tiny")),
- ("whisper base", *load_whisper("base")),
- ("moonshine tiny v2", *load_moonshine()),
- ]
- beat = numpy.arange(SAMPLE_RATE) * (numpy.pi / SAMPLE_RATE)
- warmup = (numpy.sin(440 * beat) * numpy.abs(numpy.sin(3 * beat))).astype(numpy.float32)
- for _, _, transcribe in models:
- transcribe(warmup)
- if args.audio:
- return compare(models, numpy.asarray(_read_wav(args.audio), numpy.float32))
- print(HELP)
- while True:
- line = prompt()
- command, _, rest = line.partition(" ")
- if command == "/quit":
- return
- if command == "/file":
- try:
- compare(models, numpy.asarray(_read_wav(audio_path(rest)), numpy.float32))
- except OSError as error:
- print(f" {error}")
- elif line == "":
- try:
- compare(models, numpy.asarray(record(), numpy.float32))
- except RuntimeError as error:
- print(f" {error}")
- else:
- print(HELP)
diff --git a/needle/whistle/playground.py b/needle/whistle/playground.py
deleted file mode 100644
index ea283b4..0000000
--- a/needle/whistle/playground.py
+++ /dev/null
@@ -1,115 +0,0 @@
-import os
-import time
-
-HELP = """ Enter speak, Enter again to stop
- /language de force a language (en de fr es it nl pl), bare to detect it
- /keywords Siobhan, Krzysztof
- /timestamps toggle word times
- /file clip.wav transcribe a file
- /quit leave"""
-CLEAR_ABOVE = "\x1b[1A\x1b[2K"
-LIMIT_SECONDS = 30
-
-
-def audio_path(rest):
- """The path in a /file line, as a terminal hands it over: quoted, or with escaped spaces."""
- path = rest.strip()
- if len(path) > 1 and path[0] == path[-1] and path[0] in "'\"":
- return path[1:-1]
- return path.replace("\\ ", " ")
-
-
-def status(name, size, ttft, tokens, total):
- ttft = f"{ttft * 1000:.0f} ms" if ttft is not None else "-"
- tokens = f"{tokens:.0f} tok/s" if tokens is not None else "-"
- spent = f"{total:.1f} s" if total >= 10 else f"{total * 1000:.0f} ms"
- return f" {name:<18}{size / 1e6:>4.0f} MB ttft {ttft:>6} decode {tokens:>10} total {spent:>7}"
-
-
-def record():
- from . import SAMPLE_RATE
-
- try:
- import numpy
- import sounddevice
- import soxr
- except ImportError as error:
- raise RuntimeError(f'{error.name} is not installed: pip install "cactus-needle[whistle]"') from None
- chunks = []
- try:
- rate = int(sounddevice.query_devices(kind="input")["default_samplerate"])
- with sounddevice.InputStream(samplerate=rate, channels=1, dtype="float32", callback=lambda data, *_: chunks.append(data.copy())):
- print(CLEAR_ABOVE + "● recording, Enter to stop", end="", flush=True)
- try:
- input()
- except (EOFError, KeyboardInterrupt):
- print()
- except sounddevice.PortAudioError as error:
- raise RuntimeError(f"no microphone: {error}") from None
- print(CLEAR_ABOVE, end="", flush=True)
- heard = numpy.concatenate(chunks)[:, 0] if chunks else numpy.zeros(0, numpy.float32)
- audio = heard[:LIMIT_SECONDS * rate]
- if len(audio) < len(heard):
- print(f" keeping the first {LIMIT_SECONDS} s")
- return audio if rate == SAMPLE_RATE else soxr.resample(audio, rate, SAMPLE_RATE, quality="HQ")
-
-
-def prompt():
- try:
- return input("› ").strip()
- except (EOFError, KeyboardInterrupt):
- print()
- return "/quit"
-
-
-def transcribe(whistle, audio, state):
- started = time.perf_counter()
- try:
- result = whistle.transcribe(audio, language=state["language"], keywords=state["keywords"], word_timestamps=state["timestamps"])
- except (RuntimeError, OSError, EOFError) as error:
- print(f" {error}")
- return
- total = time.perf_counter() - started
- print(result["text"] or "(no speech)")
- for word in result.get("words", []):
- print(f" {word['start']:6.2f} - {word['end']:5.2f} {word['word']:<20} {word['probability']:.2f}")
- print(status("whistle", os.path.getsize(whistle.weights), result["ttft_ms"] / 1000, result["decode_tps"], total),
- " ", result["language"] or "no speech")
-
-
-def main(args):
- from . import LANGUAGES, Whistle
-
- state = {"language": args.language, "keywords": [k.strip() for k in args.keywords.split(",") if k.strip()], "timestamps": args.word_timestamps}
- print("whistle playground: downloading and initializing the model...", flush=True)
- whistle = Whistle(weights=args.weights)
- if args.audio:
- return transcribe(whistle, args.audio, state)
- print(HELP)
- while True:
- line = prompt()
- command, _, rest = line.partition(" ")
- if command == "/quit":
- return
- if command == "/language":
- language = rest.strip()
- if language and language not in LANGUAGES:
- print(" language is one of", " ".join(LANGUAGES))
- continue
- state["language"] = language or None
- print(" language", state["language"] or "detected")
- elif command in ("/keywords", "/keyword"):
- state["keywords"] = [k.strip() for k in rest.split(",") if k.strip()]
- print(" keywords", ", ".join(state["keywords"]) or "none")
- elif command == "/timestamps":
- state["timestamps"] = not state["timestamps"]
- print(" word timestamps", "on" if state["timestamps"] else "off")
- elif command == "/file":
- transcribe(whistle, audio_path(rest), state)
- elif line == "":
- try:
- transcribe(whistle, record(), state)
- except RuntimeError as error:
- print(f" {error}")
- else:
- print(HELP)
diff --git a/pyproject.toml b/pyproject.toml
index 3cee9c7..82a4dec 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "cactus-needle"
-version = "3.0.1"
+version = "3.1.0"
description = "A 14MB foundation tool-calling model for tiny devices: inference, LoRA finetuning, and build."
requires-python = ">=3.9"
readme = "README.md"
@@ -25,8 +25,8 @@ metal = [
"jaxlib==0.4.38",
"jax-metal"
]
-whistle = ["numpy", "soxr", "sounddevice"]
-whistle-compare = ["openai-whisper", "moonshine-voice"]
+mic = ["numpy", "soxr", "sounddevice"]
+compare = ["openai-whisper", "moonshine-voice"]
test = ["pytest", "pydantic"]
[project.scripts]
diff --git a/tests/test_fetch.py b/tests/test_fetch.py
index 0d76862..46c1a17 100644
--- a/tests/test_fetch.py
+++ b/tests/test_fetch.py
@@ -110,7 +110,8 @@ def test_unpublished_engine_wheels_lists_every_missing_tag(monkeypatch):
published = {}
for generation, version in fetch.ENGINE_VERSIONS.items():
- # needle3 hosts both its own engine and needle3_whistle, so the sets merge.
+ if generation in fetch.WEIGHTS_ONLY:
+ continue
published.setdefault(fetch.ENGINE_REPOS[generation], set()).update(
fetch.engine_wheel(version, tag, generation) for tag in fetch.WHEEL_TAGS)
monkeypatch.setattr("huggingface_hub.list_repo_files",
@@ -122,16 +123,8 @@ def test_unpublished_engine_wheels_lists_every_missing_tag(monkeypatch):
published[repo3].remove(dropped)
assert fetch.unpublished_engine_wheels() == [repo3 + "/" + dropped]
- audio = fetch.engine_wheel(fetch.ENGINE_VERSIONS[fetch.NEEDLE3_WHISTLE], "win_arm64", fetch.NEEDLE3_WHISTLE)
- published[repo3].remove(audio)
- assert fetch.unpublished_engine_wheels() == [repo3 + "/" + dropped, repo3 + "/" + audio]
- published[repo3].add(audio)
-
- whistle = fetch.ENGINE_REPOS[fetch.WHISTLE]
- published[whistle].clear()
- missing = fetch.unpublished_engine_wheels()
- assert len(missing) == 1 + len(fetch.WHEEL_TAGS)
- assert missing[-1] == f"{whistle}/python/cactus_whistle-{fetch.ENGINE_VERSIONS[fetch.WHISTLE]}-py3-none-win_arm64.whl"
+ assert fetch.WHISTLE in fetch.WEIGHTS_ONLY
+ assert not any(fetch.ENGINE_REPOS[fetch.WHISTLE] in path for path in fetch.unpublished_engine_wheels())
def test_a_repo_that_cannot_be_listed_holds_none_of_its_wheels(monkeypatch):
@@ -143,7 +136,8 @@ def test_a_repo_that_cannot_be_listed_holds_none_of_its_wheels(monkeypatch):
monkeypatch.setattr("huggingface_hub.list_repo_files", refuse)
missing = fetch.unpublished_engine_wheels()
- assert len(missing) == len(fetch.ENGINE_VERSIONS) * len(fetch.WHEEL_TAGS)
+ engines = len(fetch.ENGINE_VERSIONS) - len(fetch.WEIGHTS_ONLY)
+ assert len(missing) == engines * len(fetch.WHEEL_TAGS)
def test_release_gate_runs_before_the_publish_step():
diff --git a/tests/test_whistle.py b/tests/test_whistle.py
index d98e625..14a0c7e 100644
--- a/tests/test_whistle.py
+++ b/tests/test_whistle.py
@@ -20,54 +20,72 @@ requires_whistle = pytest.mark.skipif(not (_engine_available("whistle") and os.p
reason="Whistle engine or whistle.cact not installed (auto-fetched from HF on first real use)")
-def test_engine_and_weights_resolve_like_needle(tmp_path, monkeypatch):
- import zipfile
+class _Stub:
+ class _Fn:
+ argtypes = None
+ restype = None
+
+ def __getattr__(self, name):
+ return _Stub._Fn()
+
+
+def test_speech_weights_come_from_their_own_repo(tmp_path, monkeypatch):
+ """One engine runs both models, so only the weights have a channel of their own."""
import needle
- from needle import whistle
+ from needle.agent import whistle
from needle.agent import fetch
+ assert fetch.WHISTLE in fetch.WEIGHTS_ONLY and fetch.NAMED_ENGINES == ()
monkeypatch.setattr(os.path, "expanduser", lambda path: str(tmp_path))
monkeypatch.setattr(needle, "__file__", str(tmp_path / "package" / "__init__.py"))
monkeypatch.setattr(fetch, "_register_download", lambda generation: None)
monkeypatch.delenv("NEEDLE_WHISTLE_WEIGHTS", raising=False)
- monkeypatch.setenv("NEEDLE_WHISTLE_LIB_PATH", "/opt/libwhistle.so")
- assert needle._library_path(fetch.WHISTLE) == "/opt/libwhistle.so"
- monkeypatch.delenv("NEEDLE_WHISTLE_LIB_PATH")
-
- lib = fetch.lib_name(fetch.WHISTLE)
- wheel, weights, fetched = tmp_path / "engine.whl", tmp_path / "published.cact", []
- with zipfile.ZipFile(wheel, "w") as archive:
- archive.writestr("needle/" + lib, b"engine")
+
+ weights, fetched = tmp_path / "published.cact", []
weights.write_bytes(b"weights")
monkeypatch.setattr("huggingface_hub.hf_hub_download",
- lambda **kwargs: fetched.append((kwargs["repo_id"], kwargs["filename"])) or str(weights if kwargs["filename"].endswith(".cact") else wheel))
+ lambda **kwargs: fetched.append((kwargs["repo_id"], kwargs["filename"])) or str(weights))
cache = tmp_path / ".cache" / "cactus-needle" / "whistle" / fetch.ENGINE_VERSIONS[fetch.WHISTLE]
- assert lib.startswith("libwhistle.") and needle._library_path(fetch.WHISTLE) == str(cache / lib) and (cache / lib).read_bytes() == b"engine"
- assert whistle._weights_path() == str(cache / "whistle.cact") and (cache / "whistle.cact").read_bytes() == b"weights"
- assert fetched == [("Cactus-Compute/whistle", f"python/cactus_whistle-{fetch.ENGINE_VERSIONS[fetch.WHISTLE]}-py3-none-{fetch._platform_tag()}.whl"),
- ("Cactus-Compute/whistle", "whistle.cact")]
- assert needle._library_path(fetch.WHISTLE) and whistle._weights_path() and len(fetched) == 2
+ assert whistle._weights_path() == str(cache / "whistle.cact")
+ assert (cache / "whistle.cact").read_bytes() == b"weights"
+ assert fetched == [("Cactus-Compute/whistle", "whistle.cact")]
monkeypatch.setenv("NEEDLE_WHISTLE_WEIGHTS", "/opt/whistle.cact")
assert whistle._weights_path() == "/opt/whistle.cact"
-def test_cli_fetches_the_whistle_engine_and_weights(tmp_path, monkeypatch, capsys):
- import sys
- import needle._telemetry
- import needle.cli
- from needle.agent import fetch
+def test_module_level_transcribe_reuses_one_model(monkeypatch):
+ import needle
+ from needle.agent import whistle
calls = []
- monkeypatch.setattr(needle._telemetry, "track", lambda *a, **k: None)
- monkeypatch.setattr(fetch, "fetch_library", lambda version, dest, tag=None, generation=2: calls.append((version, tag, generation)) or os.path.join(dest, "libwhistle.so"))
- monkeypatch.setattr(fetch, "fetch_weights", lambda generation, dest: calls.append((generation, dest)) or __file__)
- monkeypatch.setattr(sys, "argv", ["needle", "whistle", "fetch", "--out", str(tmp_path), "--platform-tag", "manylinux2014_aarch64"])
- needle.cli.main()
- monkeypatch.setattr(sys, "argv", ["needle", "whistle", "download", "whistle", "--out", str(tmp_path)])
- needle.cli.main()
- assert calls == [(fetch.ENGINE_VERSIONS[fetch.WHISTLE], "manylinux2014_aarch64", fetch.WHISTLE), (fetch.WHISTLE, str(tmp_path))]
- out = capsys.readouterr().out
- assert "NEEDLE_WHISTLE_LIB_PATH" in out and "needle.Whistle(weights=" in out
+
+ class _Fake:
+ def __init__(self, weights=None):
+ calls.append(weights)
+ self.weights = weights
+
+ def transcribe(self, audio, language=None, keywords=None, word_timestamps=False):
+ return {"text": audio, "language": language, "words": word_timestamps}
+
+ monkeypatch.setattr(whistle, "Whistle", _Fake)
+ monkeypatch.setattr(whistle, "_shared", {})
+ monkeypatch.setattr("needle._telemetry.track", lambda *a, **k: None)
+ assert needle.transcribe("a.wav")["text"] == "a.wav"
+ assert needle.transcribe("b.wav", language="de")["language"] == "de"
+ assert calls == [None]
+ needle.transcribe("c.wav", weights="tuned.cact")
+ assert calls == [None, "tuned.cact"]
+
+
+def test_the_speech_model_loads_into_needle_s_engine(monkeypatch):
+ from needle.agent import whistle
+
+ loaded = []
+ monkeypatch.setattr(whistle, "_handle", None)
+ monkeypatch.setattr("needle._load_cdll", lambda generation: loaded.append(generation) or _Stub())
+ whistle._lib()
+ monkeypatch.setattr(whistle, "_handle", None)
+ assert loaded == [3]
def _tone(seconds, rate=16000):
@@ -88,7 +106,7 @@ def _write_wav(path, samples, rate, channels=1, width=2):
@pytest.mark.parametrize("rate,channels,width", [(16000, 1, 2), (8000, 1, 2), (44100, 2, 2), (16000, 1, 1), (22050, 2, 3), (48000, 1, 4)])
def test_wav_files_become_16k_mono_floats(tmp_path, rate, channels, width):
- from needle.whistle import _read_wav
+ from needle.agent.whistle import _read_wav
if rate != 16000:
pytest.importorskip("soxr")
@@ -102,7 +120,7 @@ def test_wav_files_become_16k_mono_floats(tmp_path, rate, channels, width):
def test_samples_accept_lists_arrays_and_float32_bytes():
- from needle.whistle import _samples
+ from needle.agent.whistle import _samples
values = [0.0, 0.5, -0.25]
inputs = [values, array.array("f", values), struct.pack("<3f", *values), (v for v in values)]
@@ -125,7 +143,7 @@ def test_silence_is_an_empty_transcript():
@requires_whistle
def test_transcribe_returns_text_language_and_timed_words(tmp_path):
from needle import Whistle
- from needle.whistle import LANGUAGES
+ from needle.agent.whistle import LANGUAGES
whistle = Whistle()
sound = [v * (0.2 + 0.8 * abs(math.sin(math.pi * 3 * i / 16000))) for i, v in enumerate(_tone(3))]
@@ -204,7 +222,7 @@ class _Microphone:
def test_record_resamples_the_microphone_to_16_khz_and_keeps_30_s(monkeypatch, capsys):
numpy = pytest.importorskip("numpy")
pytest.importorskip("soxr")
- from needle.whistle.playground import record
+ from needle.agent.whistle import record
microphone = _Microphone(48000, 1)
monkeypatch.setitem(sys.modules, "sounddevice", microphone)
@@ -219,26 +237,26 @@ def test_record_resamples_the_microphone_to_16_khz_and_keeps_30_s(monkeypatch, c
def test_record_says_what_is_missing(monkeypatch):
- from needle.whistle.playground import record
+ from needle.agent.whistle import record
monkeypatch.setitem(sys.modules, "sounddevice", None)
- with pytest.raises(RuntimeError, match=r"cactus-needle\[whistle\]"):
+ with pytest.raises(RuntimeError, match=r"cactus-needle\[mic\]"):
record()
def test_rate_counts_steps_after_the_first_mark():
- from needle.whistle.compare import rate
+ from needle.agent.whistle import rate
assert rate([1.0, 1.5, 2.0]) == 2.0
assert rate([1.0]) == 0.0 and rate([]) == 0.0 and rate([2.0, 2.0]) == 0.0
def test_compare_prints_one_line_per_model_with_dashes_for_missing_timing(capsys):
- from needle.whistle.compare import compare
+ from needle.agent.whistle import run_models
models = [("whistle", 17e6, lambda audio: ("hello there", 0.013, 1264.0)),
("moonshine tiny v2", 45e6, lambda audio: ("", None, None))]
- compare(models, [0.0] * 16000)
+ run_models(models, [0.0] * 16000)
lines = capsys.readouterr().out.splitlines()
assert len(lines) == 2
assert lines[0].startswith(" whistle 17 MB ttft 13 ms decode 1264 tok/s total")
@@ -248,21 +266,21 @@ def test_compare_prints_one_line_per_model_with_dashes_for_missing_timing(capsys
def test_playground_and_compare_share_one_status_line():
- from needle.whistle.playground import status
+ from needle.agent.whistle import status
assert status("whistle", 17e6, 0.0137, 1264.4, 0.048) == " whistle 17 MB ttft 14 ms decode 1264 tok/s total 48 ms"
assert status("moonshine tiny v2", 145e6, None, None, 0.21) == " moonshine tiny v2 145 MB ttft - decode - total 210 ms"
def test_status_reads_a_slow_run_in_seconds():
- from needle.whistle.playground import status
+ from needle.agent.whistle import status
assert status("whistle", 17e6, 0.0137, 1264.4, 12.5).endswith("total 12.5 s")
assert status("whistle", 17e6, 0.0137, 1264.4, 9.9).endswith("total 9900 ms")
def test_audio_path_takes_a_path_as_a_terminal_hands_it_over():
- from needle.whistle.playground import audio_path
+ from needle.agent.whistle import audio_path
assert audio_path(" clip.wav ") == "clip.wav"
assert audio_path('"my clips/a b.wav"') == "my clips/a b.wav"
@@ -271,11 +289,11 @@ def test_audio_path_takes_a_path_as_a_terminal_hands_it_over():
def test_playground_prints_text_words_and_timing(capsys):
- from needle.whistle.playground import transcribe
+ from needle.agent.whistle import show_transcript
whistle = _Whistle({"text": "hello there", "language": "en", "ttft_ms": 12.6, "decode_tps": 900.4,
"words": [{"word": "hello", "start": 0.1, "end": 0.4, "probability": 0.98}]})
- transcribe(whistle, "clip.wav", {"language": "en", "keywords": ["Siobhan"], "timestamps": True})
+ show_transcript(whistle, "clip.wav", {"language": "en", "keywords": ["Siobhan"], "timestamps": True})
out = capsys.readouterr().out.splitlines()
assert whistle.calls == [("clip.wav", {"language": "en", "keywords": ["Siobhan"], "word_timestamps": True})]
assert out[0] == "hello there"
@@ -284,62 +302,61 @@ def test_playground_prints_text_words_and_timing(capsys):
def test_playground_reports_engine_errors_instead_of_raising(capsys):
- from needle.whistle.playground import transcribe
+ from needle.agent.whistle import show_transcript
class Broken:
def transcribe(self, audio, **options):
raise RuntimeError("audio limit is 30 s")
- transcribe(Broken(), "long.wav", {"language": None, "keywords": [], "timestamps": False})
+ show_transcript(Broken(), "long.wav", {"language": None, "keywords": [], "timestamps": False})
assert capsys.readouterr().out == " audio limit is 30 s\n"
def test_playground_keeps_the_language_when_the_code_is_not_one_of_ours(monkeypatch, capsys):
- from needle.whistle import playground
+ from needle.agent import whistle
lines = iter(["/language de", "/language EN", "/language", "/quit"])
- monkeypatch.setattr(playground, "prompt", lambda: next(lines))
- monkeypatch.setattr("needle.whistle.Whistle", lambda weights=None: _Whistle({}))
- playground.main(type("Args", (), {"audio": None, "language": None, "keywords": "",
- "word_timestamps": False, "weights": None})())
+ monkeypatch.setattr(whistle, "prompt", lambda: next(lines))
+ monkeypatch.setattr("needle.agent.whistle.Whistle", lambda weights=None: _Whistle({}))
+ whistle.playground(type("Args", (), {"audio": None, "language": None, "keywords": "",
+ "word_timestamps": False, "weights": None})())
out = capsys.readouterr().out
assert "language de" in out and "language is one of en de fr es it nl pl" in out and "language detected" in out
def test_compare_runs_a_file_the_same_way_the_playground_does(monkeypatch, capsys):
pytest.importorskip("numpy")
- from needle.whistle import compare as whistle_compare
+ from needle.agent import whistle
seen, read = [], []
- monkeypatch.setattr(whistle_compare, "compare", lambda models, audio: seen.append(len(audio)))
- monkeypatch.setattr("needle.whistle._read_wav", lambda path: read.append(path) or [0.0] * 16000)
+ monkeypatch.setattr(whistle, "run_models", lambda models, audio: seen.append(len(audio)))
+ monkeypatch.setattr("needle.agent.whistle._read_wav", lambda path: read.append(path) or [0.0] * 16000)
lines = iter(['/file "my clips/a b.wav"', "/quit"])
- monkeypatch.setattr(whistle_compare, "prompt", lambda: next(lines))
- monkeypatch.setattr(whistle_compare, "load_whistle", lambda weights: (17e6, lambda audio: ("", None, None)))
- monkeypatch.setattr(whistle_compare, "load_whisper", lambda size: (76e6, lambda audio: ("", None, None)))
- monkeypatch.setattr(whistle_compare, "load_moonshine", lambda: (45e6, lambda audio: ("", None, None)))
- whistle_compare.main(type("Args", (), {"audio": None, "weights": None})())
+ monkeypatch.setattr(whistle, "prompt", lambda: next(lines))
+ monkeypatch.setattr(whistle, "load_whistle", lambda weights: (17e6, lambda audio: ("", None, None)))
+ monkeypatch.setattr(whistle, "load_whisper", lambda size: (76e6, lambda audio: ("", None, None)))
+ monkeypatch.setattr(whistle, "load_moonshine", lambda: (45e6, lambda audio: ("", None, None)))
+ whistle.compare(type("Args", (), {"audio": None, "weights": None})())
assert read == ["my clips/a b.wav"] and seen == [16000]
def test_compare_needs_its_extra(monkeypatch):
- from needle.whistle import compare as whistle_compare
+ from needle.agent import whistle
monkeypatch.setitem(sys.modules, "whisper", None)
- with pytest.raises(SystemExit, match=r"cactus-needle\[whistle,whistle-compare\]"):
- whistle_compare.main(type("Args", (), {"audio": None, "weights": None})())
+ with pytest.raises(SystemExit, match=r"cactus-needle\[mic,compare\]"):
+ whistle.compare(type("Args", (), {"audio": None, "weights": None})())
def test_cli_routes_the_whistle_commands(monkeypatch):
import needle._telemetry
import needle.cli
- import needle.whistle.playground
- import needle.whistle.compare
+ import needle.agent.whistle
seen = []
monkeypatch.setattr(needle._telemetry, "track", lambda *a, **k: None)
- monkeypatch.setattr(needle.whistle.playground, "main", lambda args: seen.append(("playground", args)))
- monkeypatch.setattr(needle.whistle.compare, "main", lambda args: seen.append(("compare", args)))
+ monkeypatch.setattr(needle.agent.whistle, "playground", lambda args: seen.append(("playground", args)))
+ monkeypatch.setattr(needle.agent.whistle, "compare", lambda args: seen.append(("compare", args)))
monkeypatch.setattr(sys, "argv", ["needle", "whistle", "playground", "clip.wav", "--language", "de", "--keywords", "Siobhan, Krzysztof", "--word-timestamps"])
needle.cli.main()
monkeypatch.setattr(sys, "argv", ["needle", "whistle", "compare", "--weights", "w.cact"])
@@ -348,35 +365,5 @@ def test_cli_routes_the_whistle_commands(monkeypatch):
assert (seen[0][1].audio, seen[0][1].language, seen[0][1].keywords, seen[0][1].word_timestamps) == ("clip.wav", "de", "Siobhan, Krzysztof", True)
assert (seen[1][1].audio, seen[1][1].weights) == (None, "w.cact")
monkeypatch.setattr(sys, "argv", ["needle", "whistle"])
- with pytest.raises(SystemExit, match="needle whistle playground \\| compare \\| fetch \\| download"):
+ with pytest.raises(SystemExit, match="needle whistle playground \\| compare"):
needle.cli.main()
-
-
-def test_needle3_whistle_resolves_inside_the_needle3_repo(tmp_path, monkeypatch):
- """The combined engine shares needle3's repo, so every path it fetches is prefixed."""
- import zipfile
- import needle
- from needle.agent import fetch
-
- audio = fetch.NEEDLE3_WHISTLE
- assert fetch.engine_repo(audio) == fetch.engine_repo(3)
- assert fetch.engine_version(audio) == "3.0.0"
- assert fetch.lib_name(audio) == fetch.lib_name(3).replace("needle", audio)
- assert fetch.engine_wheel("3.0.0", "win_arm64", audio).startswith(audio + "/python/")
- assert fetch.engine_wheel("3.0.3", "win_arm64", 3).startswith("python/")
-
- monkeypatch.setattr(os.path, "expanduser", lambda path: str(tmp_path))
- monkeypatch.setattr(needle, "__file__", str(tmp_path / "package" / "__init__.py"))
- monkeypatch.setattr(fetch, "_register_download", lambda generation: None)
- monkeypatch.setenv(fetch.lib_path_env(audio), "/opt/libneedle3_whistle.so")
- assert needle._library_path(audio) == "/opt/libneedle3_whistle.so"
- monkeypatch.delenv(fetch.lib_path_env(audio))
-
- wheel, fetched = tmp_path / "engine.whl", []
- with zipfile.ZipFile(wheel, "w") as archive:
- archive.writestr("needle/" + fetch.lib_name(audio), b"engine")
- monkeypatch.setattr("huggingface_hub.hf_hub_download",
- lambda **kwargs: fetched.append(kwargs["filename"]) or str(wheel))
- resolved = needle._library_path(audio)
- assert resolved == str(tmp_path / ".cache" / "cactus-needle" / audio / "3.0.0" / fetch.lib_name(audio))
- assert fetched == [fetch.engine_wheel("3.0.0", fetch._platform_tag(), audio)]
diff --git a/tests/test_worker.py b/tests/test_worker.py
index c853e5c..32cb3dc 100644
--- a/tests/test_worker.py
+++ b/tests/test_worker.py
@@ -45,8 +45,10 @@ int needle_init(const char* system, const char* tools, const char* index) {
return 7;
}
-int needle_complete(const char* input, int max_new_tokens, char* output,
- int capacity) {
+int needle_complete(const char* input, const float* pcm, int samples,
+ int max_new_tokens, char* output, int capacity) {
+ (void)pcm;
+ (void)samples;
(void)max_new_tokens;
snprintf(output, (size_t)capacity,
"{\"type\":\"text\",\"model\":%d,\"resets\":%d,"