Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Refactor Whistle Module and CLI Integration
HenryNdubuaku8881c1ceada5
diff --git a/README.md b/README.mdindex f39432d..1ae78af 100644--- a/README.md+++ b/README.md@@ -5,6 +5,7 @@ A foundation model for mobiles, wearables, robots, smart home, automotive and mi- **Tool calls**: given the functions your app exposes, Needle picks the right ones and fills every argument from what the user said. Ask for two things and you get two calls in order; ask for something no tool covers and you get an empty list, not a guess.- **Structured extraction**: declare a shape, hand over messy text, get typed fields back: an invoice, a booking, a notification, a form. The decode grammar guarantees the output parses, and extraction generalises to classification.- **Text embedding**: the same model returns a vector for a sentence, so an app can search, match and route locally.+- **Speech**: [Whistle](#whistle), our speech-to-text model, shares Needle's container and engine. One build gives Needle audio input: the clip goes in, the tool calls come out.@@ -43,6 +44,35 @@ print(agent.run("what's it like in Lagos right now?")["results"])Every turn returns one JSON object with `function_calls`, the model's `reasoning` and a calibrated `confidence`; an off-topic request returns an empty list rather than a guess. `needle.Needle(tools=[...], generation=2)` keeps running Needle 2 for existing deployments.+## Whistle++++Whistle is our speech-to-text model, one 16.9 MB file on the CPU: 16 kHz mono audio, up to 30 seconds in one pass, in English, German, French, Spanish, Italian, Dutch and Polish. It shares Needle's `.cact` container, its quantisation and its C++ engine, so the two are one runtime, and its decoder is laddered the same way, with `--audio-depth N` running the N-layer rung of the same weights.++```python+import needle++print(needle.transcribe("clip.wav")["text"])+# turn off the kitchen lights+```++Every call returns the text, the language, the milliseconds to the first token and the decoder's tokens per second after it. `word_timestamps=True` adds each word with its start, end and probability, `keywords=[...]` favours the names and product words your users say, and `language="de"` forces the language instead of detecting it. `needle.Whistle()` is the same model as an object, for `embed(audio)` or to hold one tuned `.cact`. Silence and steady noise return an empty transcript rather than an invented sentence. `needle whistle playground` transcribes from the microphone, and `needle whistle compare` puts Whistle next to Whisper and Moonshine on the same clip.++At 16.9 MB Whistle is 8.6x smaller than Whisper base and 6.6x quicker to the first token. It is ahead on LibriSpeech test-clean and test-other, on SPGISpeech, on Earnings-22 and on the FLEURS average. Whisper base is ahead on TED-LIUM, on AMI and on the MLS average.++++Word error rate on the full test splits, scored with the Whisper normalizers. Whisper and Moonshine figures are the ones their authors published. Latency and decode are 10 s of audio on an Apple M4 Pro, at each engine's defaults. The per-benchmark caveats are on [Hugging Face](https://huggingface.co/Cactus-Compute/whistle).++```sh+needle --model needle3.cact --model whistle.cact --tools tools.json --audio clip.wav+```++One engine holds both models: `needle_load` reads whichever one a `.cact` carries, and `needle_complete` takes a clip wherever it takes text. It transcribes, answers the transcript against your tools, and returns one JSON object with the tool calls and the speech fields, the speech ones prefixed `audio_`. The transcription stays inside the engine, so audio in and tool calls out is one call.++The benchmarks against Whisper and Moonshine, the architecture and the interactive demo are at [cactuscompute.com/whistle](https://cactuscompute.com/whistle); the weights and every platform engine are on [Hugging Face](https://huggingface.co/Cactus-Compute/whistle).+## Guides- [How to design tools for Needle 3](https://cactuscompute.com/blog/designing-tools-for-needle): one tool per action, names users would say, formats in descriptions, constraints in the grammar, triggers.@@ -103,7 +133,7 @@ Or hand the key to Claude Code or Codex with [cactuscompute.com/llms.txt](https:## Deploy-Every deployment target ships a prebuilt engine under 1 MB that loads the `needle3.cact` weights at start. `needle build --platform <folder> [--layers N]` fetches that engine and puts the weights beside it.+Every deployment target ships a prebuilt engine that loads the `needle3.cact` weights at start. `needle build --platform <folder> [--layers N]` fetches that engine and puts the weights beside it.diff --git a/assets/deploy.svg b/assets/deploy.svgindex 01b6159..4ecbc51 100644--- a/assets/deploy.svg+++ b/assets/deploy.svg@@ -1,8 +1,8 @@-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 529" width="900" height="529" role="img" aria-label="The thirteen platform folders Needle ships, each with an engine under 1 MB">+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 529" width="900" height="529" role="img" aria-label="The thirteen platform folders Needle ships"><style>svg{--t0:0.0s}.s{animation-duration:14.3s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}@keyframes t0{0%{stroke:#232326;fill:#141416}0.00%{stroke:#232326;fill:#141416}1.05%{stroke:#e8593d;fill:#1d1512}6.99%{stroke:#e8593d;fill:#1d1512}7.69%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t0{animation-name:t0}@keyframes f0{0%{fill:#85827a}0.00%{fill:#85827a}1.05%{fill:#e8593d}6.99%{fill:#e8593d}7.69%{fill:#85827a}100%{fill:#85827a}}.f0{animation-name:f0}@keyframes c0{0%{opacity:0}0.00%{opacity:0}0.84%{opacity:1}7.13%{opacity:1}7.69%{opacity:0}100%{opacity:0}}.c0{animation-name:c0}@keyframes t1{0%{stroke:#232326;fill:#141416}7.69%{stroke:#232326;fill:#141416}8.74%{stroke:#e8593d;fill:#1d1512}14.69%{stroke:#e8593d;fill:#1d1512}15.38%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t1{animation-name:t1}@keyframes f1{0%{fill:#85827a}7.69%{fill:#85827a}8.74%{fill:#e8593d}14.69%{fill:#e8593d}15.38%{fill:#85827a}100%{fill:#85827a}}.f1{animation-name:f1}@keyframes c1{0%{opacity:0}7.69%{opacity:0}8.53%{opacity:1}14.83%{opacity:1}15.38%{opacity:0}100%{opacity:0}}.c1{animation-name:c1}@keyframes t2{0%{stroke:#232326;fill:#141416}15.38%{stroke:#232326;fill:#141416}16.43%{stroke:#e8593d;fill:#1d1512}22.38%{stroke:#e8593d;fill:#1d1512}23.08%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t2{animation-name:t2}@keyframes f2{0%{fill:#85827a}15.38%{fill:#85827a}16.43%{fill:#e8593d}22.38%{fill:#e8593d}23.08%{fill:#85827a}100%{fill:#85827a}}.f2{animation-name:f2}@keyframes c2{0%{opacity:0}15.38%{opacity:0}16.22%{opacity:1}22.52%{opacity:1}23.08%{opacity:0}100%{opacity:0}}.c2{animation-name:c2}@keyframes t3{0%{stroke:#232326;fill:#141416}23.08%{stroke:#232326;fill:#141416}24.13%{stroke:#e8593d;fill:#1d1512}30.07%{stroke:#e8593d;fill:#1d1512}30.77%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t3{animation-name:t3}@keyframes f3{0%{fill:#85827a}23.08%{fill:#85827a}24.13%{fill:#e8593d}30.07%{fill:#e8593d}30.77%{fill:#85827a}100%{fill:#85827a}}.f3{animation-name:f3}@keyframes c3{0%{opacity:0}23.08%{opacity:0}23.92%{opacity:1}30.21%{opacity:1}30.77%{opacity:0}100%{opacity:0}}.c3{animation-name:c3}@keyframes t4{0%{stroke:#232326;fill:#141416}30.77%{stroke:#232326;fill:#141416}31.82%{stroke:#e8593d;fill:#1d1512}37.76%{stroke:#e8593d;fill:#1d1512}38.46%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t4{animation-name:t4}@keyframes f4{0%{fill:#85827a}30.77%{fill:#85827a}31.82%{fill:#e8593d}37.76%{fill:#e8593d}38.46%{fill:#85827a}100%{fill:#85827a}}.f4{animation-name:f4}@keyframes c4{0%{opacity:0}30.77%{opacity:0}31.61%{opacity:1}37.90%{opacity:1}38.46%{opacity:0}100%{opacity:0}}.c4{animation-name:c4}@keyframes t5{0%{stroke:#232326;fill:#141416}38.46%{stroke:#232326;fill:#141416}39.51%{stroke:#e8593d;fill:#1d1512}45.45%{stroke:#e8593d;fill:#1d1512}46.15%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t5{animation-name:t5}@keyframes f5{0%{fill:#85827a}38.46%{fill:#85827a}39.51%{fill:#e8593d}45.45%{fill:#e8593d}46.15%{fill:#85827a}100%{fill:#85827a}}.f5{animation-name:f5}@keyframes c5{0%{opacity:0}38.46%{opacity:0}39.30%{opacity:1}45.59%{opacity:1}46.15%{opacity:0}100%{opacity:0}}.c5{animation-name:c5}@keyframes t6{0%{stroke:#232326;fill:#141416}46.15%{stroke:#232326;fill:#141416}47.20%{stroke:#e8593d;fill:#1d1512}53.15%{stroke:#e8593d;fill:#1d1512}53.85%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t6{animation-name:t6}@keyframes f6{0%{fill:#85827a}46.15%{fill:#85827a}47.20%{fill:#e8593d}53.15%{fill:#e8593d}53.85%{fill:#85827a}100%{fill:#85827a}}.f6{animation-name:f6}@keyframes c6{0%{opacity:0}46.15%{opacity:0}46.99%{opacity:1}53.29%{opacity:1}53.85%{opacity:0}100%{opacity:0}}.c6{animation-name:c6}@keyframes t7{0%{stroke:#232326;fill:#141416}53.85%{stroke:#232326;fill:#141416}54.90%{stroke:#e8593d;fill:#1d1512}60.84%{stroke:#e8593d;fill:#1d1512}61.54%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t7{animation-name:t7}@keyframes f7{0%{fill:#85827a}53.85%{fill:#85827a}54.90%{fill:#e8593d}60.84%{fill:#e8593d}61.54%{fill:#85827a}100%{fill:#85827a}}.f7{animation-name:f7}@keyframes c7{0%{opacity:0}53.85%{opacity:0}54.69%{opacity:1}60.98%{opacity:1}61.54%{opacity:0}100%{opacity:0}}.c7{animation-name:c7}@keyframes t8{0%{stroke:#232326;fill:#141416}61.54%{stroke:#232326;fill:#141416}62.59%{stroke:#e8593d;fill:#1d1512}68.53%{stroke:#e8593d;fill:#1d1512}69.23%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t8{animation-name:t8}@keyframes f8{0%{fill:#85827a}61.54%{fill:#85827a}62.59%{fill:#e8593d}68.53%{fill:#e8593d}69.23%{fill:#85827a}100%{fill:#85827a}}.f8{animation-name:f8}@keyframes c8{0%{opacity:0}61.54%{opacity:0}62.38%{opacity:1}68.67%{opacity:1}69.23%{opacity:0}100%{opacity:0}}.c8{animation-name:c8}@keyframes t9{0%{stroke:#232326;fill:#141416}69.23%{stroke:#232326;fill:#141416}70.28%{stroke:#e8593d;fill:#1d1512}76.22%{stroke:#e8593d;fill:#1d1512}76.92%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t9{animation-name:t9}@keyframes f9{0%{fill:#85827a}69.23%{fill:#85827a}70.28%{fill:#e8593d}76.22%{fill:#e8593d}76.92%{fill:#85827a}100%{fill:#85827a}}.f9{animation-name:f9}@keyframes c9{0%{opacity:0}69.23%{opacity:0}70.07%{opacity:1}76.36%{opacity:1}76.92%{opacity:0}100%{opacity:0}}.c9{animation-name:c9}@keyframes t10{0%{stroke:#232326;fill:#141416}76.92%{stroke:#232326;fill:#141416}77.97%{stroke:#e8593d;fill:#1d1512}83.92%{stroke:#e8593d;fill:#1d1512}84.62%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t10{animation-name:t10}@keyframes f10{0%{fill:#85827a}76.92%{fill:#85827a}77.97%{fill:#e8593d}83.92%{fill:#e8593d}84.62%{fill:#85827a}100%{fill:#85827a}}.f10{animation-name:f10}@keyframes c10{0%{opacity:0}76.92%{opacity:0}77.76%{opacity:1}84.06%{opacity:1}84.62%{opacity:0}100%{opacity:0}}.c10{animation-name:c10}@keyframes t11{0%{stroke:#232326;fill:#141416}84.62%{stroke:#232326;fill:#141416}85.66%{stroke:#e8593d;fill:#1d1512}91.61%{stroke:#e8593d;fill:#1d1512}92.31%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t11{animation-name:t11}@keyframes f11{0%{fill:#85827a}84.62%{fill:#85827a}85.66%{fill:#e8593d}91.61%{fill:#e8593d}92.31%{fill:#85827a}100%{fill:#85827a}}.f11{animation-name:f11}@keyframes c11{0%{opacity:0}84.62%{opacity:0}85.45%{opacity:1}91.75%{opacity:1}92.31%{opacity:0}100%{opacity:0}}.c11{animation-name:c11}@keyframes t12{0%{stroke:#232326;fill:#141416}92.31%{stroke:#232326;fill:#141416}93.36%{stroke:#e8593d;fill:#1d1512}99.30%{stroke:#e8593d;fill:#1d1512}100.00%{stroke:#232326;fill:#141416}100%{stroke:#232326;fill:#141416}}.t12{animation-name:t12}@keyframes f12{0%{fill:#85827a}92.31%{fill:#85827a}93.36%{fill:#e8593d}99.30%{fill:#e8593d}100.00%{fill:#85827a}100%{fill:#85827a}}.f12{animation-name:f12}@keyframes c12{0%{opacity:0}92.31%{opacity:0}93.15%{opacity:1}99.44%{opacity:1}100.00%{opacity:0}100%{opacity:0}}.c12{animation-name:c12}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}}</style><rect width="900" height="529" rx="18" fill="#0e0e10"/><text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">One engine per platform folder</text>-<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">each under 1 MB, each loading the same needle3.cact</text>+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">each loading any .cact you hand it</text><rect class="s t0" x="40" y="72" width="266" height="62" rx="9" fill="#141416" stroke="#232326" stroke-width="1.2"/><text x="54" y="94" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Mac (Apple Silicon)</text><text class="s f0" x="54" y="111" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">macos-arm64</text>diff --git a/assets/whistle-benchmarks.svg b/assets/whistle-benchmarks.svgnew file mode 100644index 0000000..20c189b--- /dev/null+++ b/assets/whistle-benchmarks.svg@@ -0,0 +1,99 @@+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 576" width="900" height="576" role="img" aria-label="Whistle against Whisper tiny, Whisper base and Moonshine tiny v2 on word error rate, size and speed">+<style>+svg{--t0:0.0s}.s{animation-duration:7.0s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}.s{animation-iteration-count:1;animation-fill-mode:forwards}@keyframes b0{0%{transform:scaleY(0)}3.00%{transform:scaleY(0)}7.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b0{animation-name:b0;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w0{0%{transform:scaleX(0)}3.00%{transform:scaleX(0)}7.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w0{animation-name:w0;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v0{0%{opacity:0}4.40%{opacity:0}8.00%{opacity:1}100%{opacity:1}}.v0{animation-name:v0}@keyframes b1{0%{transform:scaleY(0)}8.60%{transform:scaleY(0)}13.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b1{animation-name:b1;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w1{0%{transform:scaleX(0)}8.60%{transform:scaleX(0)}13.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w1{animation-name:w1;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v1{0%{opacity:0}10.00%{opacity:0}13.60%{opacity:1}100%{opacity:1}}.v1{animation-name:v1}@keyframes b2{0%{transform:scaleY(0)}14.20%{transform:scaleY(0)}18.60%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b2{animation-name:b2;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w2{0%{transform:scaleX(0)}14.20%{transform:scaleX(0)}18.60%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w2{animation-name:w2;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v2{0%{opacity:0}15.60%{opacity:0}19.20%{opacity:1}100%{opacity:1}}.v2{animation-name:v2}@keyframes b3{0%{transform:scaleY(0)}19.80%{transform:scaleY(0)}24.20%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b3{animation-name:b3;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w3{0%{transform:scaleX(0)}19.80%{transform:scaleX(0)}24.20%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w3{animation-name:w3;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v3{0%{opacity:0}21.20%{opacity:0}24.80%{opacity:1}100%{opacity:1}}.v3{animation-name:v3}@keyframes b4{0%{transform:scaleY(0)}25.40%{transform:scaleY(0)}29.80%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b4{animation-name:b4;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w4{0%{transform:scaleX(0)}25.40%{transform:scaleX(0)}29.80%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w4{animation-name:w4;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v4{0%{opacity:0}26.80%{opacity:0}30.40%{opacity:1}100%{opacity:1}}.v4{animation-name:v4}@keyframes b5{0%{transform:scaleY(0)}31.00%{transform:scaleY(0)}35.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b5{animation-name:b5;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w5{0%{transform:scaleX(0)}31.00%{transform:scaleX(0)}35.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w5{animation-name:w5;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v5{0%{opacity:0}32.40%{opacity:0}36.00%{opacity:1}100%{opacity:1}}.v5{animation-name:v5}@keyframes b6{0%{transform:scaleY(0)}36.60%{transform:scaleY(0)}41.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b6{animation-name:b6;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w6{0%{transform:scaleX(0)}36.60%{transform:scaleX(0)}41.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w6{animation-name:w6;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v6{0%{opacity:0}38.00%{opacity:0}41.60%{opacity:1}100%{opacity:1}}.v6{animation-name:v6}@keyframes b7{0%{transform:scaleY(0)}42.20%{transform:scaleY(0)}46.60%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b7{animation-name:b7;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w7{0%{transform:scaleX(0)}42.20%{transform:scaleX(0)}46.60%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w7{animation-name:w7;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v7{0%{opacity:0}43.60%{opacity:0}47.20%{opacity:1}100%{opacity:1}}.v7{animation-name:v7}@keyframes b8{0%{transform:scaleY(0)}47.80%{transform:scaleY(0)}52.20%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b8{animation-name:b8;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w8{0%{transform:scaleX(0)}47.80%{transform:scaleX(0)}52.20%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w8{animation-name:w8;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v8{0%{opacity:0}49.20%{opacity:0}52.80%{opacity:1}100%{opacity:1}}.v8{animation-name:v8}@keyframes b9{0%{transform:scaleY(0)}53.40%{transform:scaleY(0)}57.80%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b9{animation-name:b9;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w9{0%{transform:scaleX(0)}53.40%{transform:scaleX(0)}57.80%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w9{animation-name:w9;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v9{0%{opacity:0}54.80%{opacity:0}58.40%{opacity:1}100%{opacity:1}}.v9{animation-name:v9}@keyframes b10{0%{transform:scaleY(0)}59.00%{transform:scaleY(0)}63.40%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b10{animation-name:b10;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w10{0%{transform:scaleX(0)}59.00%{transform:scaleX(0)}63.40%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w10{animation-name:w10;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v10{0%{opacity:0}60.40%{opacity:0}64.00%{opacity:1}100%{opacity:1}}.v10{animation-name:v10}@keyframes b11{0%{transform:scaleY(0)}64.60%{transform:scaleY(0)}69.00%{transform:scaleY(1)}100%{transform:scaleY(1)}}.b11{animation-name:b11;transform-box:fill-box;transform-origin:bottom;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes w11{0%{transform:scaleX(0)}64.60%{transform:scaleX(0)}69.00%{transform:scaleX(1)}100%{transform:scaleX(1)}}.w11{animation-name:w11;transform-box:fill-box;transform-origin:left center;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes v11{0%{opacity:0}66.00%{opacity:0}69.60%{opacity:1}100%{opacity:1}}.v11{animation-name:v11}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}.b0,.w0{transform:none}.v0{opacity:1}}</style>+<rect width="900" height="576" rx="18" fill="#0e0e10"/>+<text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">Whistle against Whisper and Moonshine</text>+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">word error rate %, lower is better</text>+<rect x="40" y="63" width="10" height="10" rx="2.5" fill="#e8593d"/>+<text x="56" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Whistle</text>+<rect x="106" y="63" width="10" height="10" rx="2.5" fill="#7b776f"/>+<text x="122" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Whisper base</text>+<rect x="203" y="63" width="10" height="10" rx="2.5" fill="#a7a299"/>+<text x="219" y="72" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#85827a">Moonshine tiny v2</text>+<line x1="40" y1="364" x2="860" y2="364" stroke="#232326" stroke-width="1"/>+<line x1="120" y1="233.3" x2="860" y2="233.3" stroke="#232326" stroke-width="1"/>+<text x="110" y="236.8" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">10</text>+<line x1="120" y1="164.7" x2="860" y2="164.7" stroke="#232326" stroke-width="1"/>+<text x="110" y="168.2" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">20</text>+<line x1="120" y1="96.0" x2="860" y2="96.0" stroke="#232326" stroke-width="1"/>+<text x="110" y="99.5" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">30</text>+<line x1="120" y1="302" x2="860" y2="302" stroke="#85827a" stroke-width="1"/>+<rect class="s b0" x="130.1" y="272.4" width="18.0" height="29.6" rx="2" fill="#e8593d"/>+<text class="s v0" x="139.1" y="260.4" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">4.31</text>+<rect class="s b0" x="152.1" y="268.4" width="18.0" height="33.6" rx="2" fill="#7b776f"/>+<rect class="s b0" x="174.1" y="271.0" width="18.0" height="31.0" rx="2" fill="#a7a299"/>+<text class="s v0" x="161.1" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">LibriSpeech</text>+<text class="s v0" x="161.1" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">test-clean</text>+<rect class="s b1" x="212.3" y="230.0" width="18.0" height="72.0" rx="2" fill="#e8593d"/>+<text class="s v1" x="221.3" y="213.6" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">10.49</text>+<rect class="s b1" x="234.3" y="226.5" width="18.0" height="75.5" rx="2" fill="#7b776f"/>+<rect class="s b1" x="256.3" y="221.6" width="18.0" height="80.4" rx="2" fill="#a7a299"/>+<text class="s v1" x="243.3" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">LibriSpeech</text>+<text class="s v1" x="243.3" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">test-other</text>+<rect class="s b2" x="294.6" y="249.5" width="18.0" height="52.5" rx="2" fill="#e8593d"/>+<text class="s v2" x="303.6" y="241.1" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">7.65</text>+<line class="s v2" x1="320.6" y1="297" x2="330.6" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<rect class="s b2" x="338.6" y="249.1" width="18.0" height="52.9" rx="2" fill="#a7a299"/>+<text class="s v2" x="325.6" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">SPGISpeech</text>+<rect class="s b3" x="376.8" y="171.5" width="18.0" height="130.5" rx="2" fill="#e8593d"/>+<text class="s v3" x="385.8" y="148.1" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">19.01</text>+<line class="s v3" x1="402.8" y1="297" x2="412.8" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<rect class="s b3" x="420.8" y="156.1" width="18.0" height="145.9" rx="2" fill="#a7a299"/>+<text class="s v3" x="407.8" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">Earnings-22</text>+<rect class="s b4" x="459.0" y="123.0" width="18.0" height="179.0" rx="2" fill="#e8593d"/>+<text class="s v4" x="468.0" y="115.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">26.07</text>+<rect class="s b4" x="481.0" y="154.4" width="18.0" height="147.6" rx="2" fill="#7b776f"/>+<rect class="s b4" x="503.0" y="145.6" width="18.0" height="156.4" rx="2" fill="#a7a299"/>+<text class="s v4" x="490.0" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">AMI</text>+<rect class="s b5" x="541.2" y="145.0" width="18.0" height="157.0" rx="2" fill="#e8593d"/>+<text class="s v5" x="550.2" y="137.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">22.87</text>+<line class="s v5" x1="567.2" y1="297" x2="577.2" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<line class="s v5" x1="589.2" y1="297" x2="599.2" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<text class="s v5" x="572.2" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">AMI</text>+<text class="s v5" x="572.2" y="336" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#4a4844">cleaned</text>+<rect class="s b6" x="623.4" y="249.7" width="18.0" height="52.3" rx="2" fill="#e8593d"/>+<text class="s v6" x="632.4" y="241.7" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">7.61</text>+<rect class="s b6" x="645.4" y="267.7" width="18.0" height="34.3" rx="2" fill="#7b776f"/>+<rect class="s b6" x="667.4" y="263.3" width="18.0" height="38.7" rx="2" fill="#a7a299"/>+<text class="s v6" x="654.4" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">TED-LIUM</text>+<rect class="s b7" x="705.7" y="155.1" width="18.0" height="146.9" rx="2" fill="#e8593d"/>+<text class="s v7" x="714.7" y="125.8" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">21.4</text>+<rect class="s b7" x="727.7" y="133.8" width="18.0" height="168.2" rx="2" fill="#7b776f"/>+<line class="s v7" x1="753.7" y1="297" x2="763.7" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<text class="s v7" x="736.7" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">FLEURS</text>+<rect class="s b8" x="787.9" y="131.0" width="18.0" height="171.0" rx="2" fill="#e8593d"/>+<text class="s v8" x="796.9" y="123.0" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8593d">24.9</text>+<rect class="s b8" x="809.9" y="143.4" width="18.0" height="158.6" rx="2" fill="#7b776f"/>+<line class="s v8" x1="835.9" y1="297" x2="845.9" y2="297" stroke="#4a4844" stroke-width="1.5"/>+<text class="s v8" x="818.9" y="322" text-anchor="middle" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">MLS</text>+<text x="40" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Size</text>+<text x="40" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">MB, smaller is better</text>+<line x1="40" y1="430" x2="287" y2="430" stroke="#232326" stroke-width="1"/>+<rect class="s w9" x="40" y="446" width="21.5" height="9" rx="2" fill="#e8593d"/>+<text class="s v9" x="287" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">16.9</text>+<rect class="s w9" x="40" y="463" width="184.7" height="9" rx="2" fill="#7b776f"/>+<text class="s v9" x="287" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">145.3</text>+<rect class="s w9" x="40" y="480" width="53.3" height="9" rx="2" fill="#a7a299"/>+<text class="s v9" x="287" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">41.9</text>+<text x="327" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Time to first token</text>+<text x="327" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">ms on a 10 s clip, smaller is better</text>+<line x1="327" y1="430" x2="573" y2="430" stroke="#232326" stroke-width="1"/>+<rect class="s w10" x="327" y="446" width="28.0" height="9" rx="2" fill="#e8593d"/>+<text class="s v10" x="573" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">11.1</text>+<rect class="s w10" x="327" y="463" width="184.7" height="9" rx="2" fill="#7b776f"/>+<text class="s v10" x="573" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">73.2</text>+<rect class="s w10" x="327" y="480" width="57.5" height="9" rx="2" fill="#a7a299"/>+<text class="s v10" x="573" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">22.8</text>+<text x="613" y="404" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#e8e6df">Decode</text>+<text x="613" y="420" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="9.5" fill="#4a4844">tokens/s on a 10 s clip, larger is better</text>+<line x1="613" y1="430" x2="860" y2="430" stroke="#232326" stroke-width="1"/>+<rect class="s w11" x="613" y="446" width="184.7" height="9" rx="2" fill="#e8593d"/>+<text class="s v11" x="860" y="454" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#e8e6df">1,319</text>+<rect class="s w11" x="613" y="463" width="37.2" height="9" rx="2" fill="#7b776f"/>+<text class="s v11" x="860" y="471" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">266</text>+<rect class="s w11" x="613" y="480" width="36.7" height="9" rx="2" fill="#a7a299"/>+<text class="s v11" x="860" y="488" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" fill="#85827a">262</text>+<text x="40" y="524" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">Whistle's figures are labelled; every bar is to the same scale. Whisper and Moonshine word error rates</text>+<text x="40" y="538" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">as published; Whisper's AMI figure is AMI-IHM. FLEURS and MLS average the seven languages, MLS the six</text>+<text x="40" y="552" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">without English. Speed is 10 s of audio on an Apple M4 Pro, at each engine's defaults.</text>+</svg>diff --git a/assets/whistle.svg b/assets/whistle.svgnew file mode 100644index 0000000..607de11--- /dev/null+++ b/assets/whistle.svg@@ -0,0 +1,189 @@+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 300" width="900" height="300" role="img" aria-label="One Needle engine with three loadouts: the speech model alone, both models together, and the text model alone">+<style>+svg{--t0:0.0s}.s{animation-duration:16.5s;animation-iteration-count:infinite;animation-fill-mode:both;animation-delay:calc(0s - var(--t0))}@keyframes s0r0{0%{opacity:0;transform:translateX(-8px)}1.20%{opacity:0;transform:translateX(-8px)}3.80%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r0{animation-name:s0r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s0r1{0%{opacity:0;transform:translateX(-8px)}3.40%{opacity:0;transform:translateX(-8px)}6.00%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r1{animation-name:s0r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s0r2{0%{opacity:0;transform:translateX(-8px)}5.60%{opacity:0;transform:translateX(-8px)}8.20%{opacity:1;transform:translateX(0)}31.13%{opacity:1;transform:translateX(0)}32.73%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s0r2{animation-name:s0r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep0{0%{opacity:0;transform:translateX(0)}3.20%{opacity:0;transform:translateX(0)}4.00%{opacity:1;transform:translateX(0)}30.33%{opacity:1;transform:translateX(500px)}31.13%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep0{animation-name:sweep0;animation-timing-function:linear}@keyframes s1r0{0%{opacity:0;transform:translateX(-8px)}34.53%{opacity:0;transform:translateX(-8px)}37.13%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r0{animation-name:s1r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s1r1{0%{opacity:0;transform:translateX(-8px)}36.73%{opacity:0;transform:translateX(-8px)}39.33%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r1{animation-name:s1r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s1r2{0%{opacity:0;transform:translateX(-8px)}38.93%{opacity:0;transform:translateX(-8px)}41.53%{opacity:1;transform:translateX(0)}64.47%{opacity:1;transform:translateX(0)}66.07%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s1r2{animation-name:s1r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep1{0%{opacity:0;transform:translateX(0)}36.53%{opacity:0;transform:translateX(0)}37.33%{opacity:1;transform:translateX(0)}63.67%{opacity:1;transform:translateX(500px)}64.47%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep1{animation-name:sweep1;animation-timing-function:linear}@keyframes s2r0{0%{opacity:0;transform:translateX(-8px)}67.87%{opacity:0;transform:translateX(-8px)}70.47%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r0{animation-name:s2r0;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s2r1{0%{opacity:0;transform:translateX(-8px)}70.07%{opacity:0;transform:translateX(-8px)}72.67%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r1{animation-name:s2r1;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes s2r2{0%{opacity:0;transform:translateX(-8px)}72.27%{opacity:0;transform:translateX(-8px)}74.87%{opacity:1;transform:translateX(0)}97.80%{opacity:1;transform:translateX(0)}99.40%{opacity:0;transform:translateX(0)}100%{opacity:0;transform:translateX(-8px)}}.s2r2{animation-name:s2r2;animation-timing-function:cubic-bezier(.2,.8,.2,1)}@keyframes sweep2{0%{opacity:0;transform:translateX(0)}69.87%{opacity:0;transform:translateX(0)}70.67%{opacity:1;transform:translateX(0)}97.00%{opacity:1;transform:translateX(500px)}97.80%{opacity:0;transform:translateX(500px)}100%{opacity:0;transform:translateX(0)}}.sweep2{animation-name:sweep2;animation-timing-function:linear}@media (prefers-reduced-motion:reduce){.s{animation:none!important}.c0{opacity:1}.s0r0,.s0r1,.s0r2{opacity:1;transform:none}.sweep0{opacity:1}}</style>+<rect width="900" height="300" rx="18" fill="#0e0e10"/>+<text x="40" y="38" font-family="Inter, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="15" font-weight="600" fill="#e8e6df">One engine, three ways to load it</text>+<text x="860" y="38" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" fill="#85827a">needle.cact, whistle.cact, or both</text>+<line x1="40" y1="66" x2="860" y2="66" stroke="#232326" stroke-width="1"/>+<line x1="40" y1="118" x2="860" y2="118" stroke="#232326" stroke-width="1"/>+<line x1="40" y1="176" x2="860" y2="176" stroke="#232326" stroke-width="1"/>+<line x1="40" y1="234" x2="860" y2="234" stroke="#232326" stroke-width="1"/>+<g class="s s0r0">+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model whistle.cact</text>+</g>+<g class="s s0r1">+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>+<rect class="s s0r1" x="150.0" y="141.8" width="3.7" height="10.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="157.4" y="140.8" width="3.7" height="12.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="164.7" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="172.1" y="138.8" width="3.7" height="16.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="179.4" y="138.0" width="3.7" height="18.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="186.8" y="137.3" width="3.7" height="19.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="194.1" y="136.9" width="3.7" height="20.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="201.5" y="136.8" width="3.7" height="20.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="208.8" y="137.1" width="3.7" height="19.9" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="216.2" y="137.7" width="3.7" height="18.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="223.5" y="138.6" width="3.7" height="16.7" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="230.9" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="238.2" y="141.1" width="3.7" height="11.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="245.6" y="142.4" width="3.7" height="9.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="252.9" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="260.3" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="267.6" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="275.0" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="282.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="289.7" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="297.1" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="304.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="311.8" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="319.1" y="145.3" width="3.7" height="3.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="326.5" y="144.2" width="3.7" height="5.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="333.8" y="142.8" width="3.7" height="8.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="341.2" y="141.1" width="3.7" height="11.9" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="348.5" y="139.2" width="3.7" height="15.7" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="355.9" y="137.2" width="3.7" height="19.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="363.2" y="135.3" width="3.7" height="23.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="370.6" y="133.6" width="3.7" height="26.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="377.9" y="132.3" width="3.7" height="29.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="385.3" y="131.4" width="3.7" height="31.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="392.6" y="131.0" width="3.7" height="32.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="400.0" y="131.1" width="3.7" height="31.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="407.4" y="131.7" width="3.7" height="30.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="414.7" y="132.8" width="3.7" height="28.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="422.1" y="134.2" width="3.7" height="25.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="429.4" y="135.8" width="3.7" height="22.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="436.8" y="137.5" width="3.7" height="19.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="444.1" y="139.1" width="3.7" height="15.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="451.5" y="140.6" width="3.7" height="12.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="458.8" y="141.8" width="3.7" height="10.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="466.2" y="142.9" width="3.7" height="8.3" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="473.5" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="480.9" y="144.2" width="3.7" height="5.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="488.2" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="495.6" y="145.1" width="3.7" height="3.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="502.9" y="145.5" width="3.7" height="3.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="510.3" y="144.9" width="3.7" height="4.2" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="517.6" y="144.1" width="3.7" height="5.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="525.0" y="143.2" width="3.7" height="7.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="532.4" y="142.3" width="3.7" height="9.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="539.7" y="141.4" width="3.7" height="11.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="547.1" y="140.7" width="3.7" height="12.6" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="554.4" y="140.1" width="3.7" height="13.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="561.8" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="569.1" y="139.7" width="3.7" height="14.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="576.5" y="140.0" width="3.7" height="14.1" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="583.8" y="140.4" width="3.7" height="13.2" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="591.2" y="141.0" width="3.7" height="11.9" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="598.5" y="141.8" width="3.7" height="10.5" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="605.9" y="142.5" width="3.7" height="9.0" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="613.2" y="143.1" width="3.7" height="7.8" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="620.6" y="143.6" width="3.7" height="6.9" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="627.9" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="635.3" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>+<rect class="s s0r1" x="642.6" y="143.6" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<text x="672" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">clip.wav</text>+</g>+<rect class="s sweep0" x="150" y="128" width="1.5" height="38" fill="#e8593d"/>+<g class="s s0r2">+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"text":"turn off the kitchen lights","language":"en",</text>+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "ttft_ms":11.1,"decode_tps":1319.0}</text>+</g>+<g class="s s1r0">+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model needle3.cact --model whistle.cact --tools tools.json</text>+</g>+<g class="s s1r1">+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>+<rect class="s s1r1" x="150.0" y="141.8" width="3.7" height="10.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="157.4" y="140.8" width="3.7" height="12.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="164.7" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="172.1" y="138.8" width="3.7" height="16.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="179.4" y="138.0" width="3.7" height="18.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="186.8" y="137.3" width="3.7" height="19.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="194.1" y="136.9" width="3.7" height="20.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="201.5" y="136.8" width="3.7" height="20.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="208.8" y="137.1" width="3.7" height="19.9" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="216.2" y="137.7" width="3.7" height="18.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="223.5" y="138.6" width="3.7" height="16.7" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="230.9" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="238.2" y="141.1" width="3.7" height="11.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="245.6" y="142.4" width="3.7" height="9.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="252.9" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="260.3" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="267.6" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="275.0" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="282.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="289.7" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="297.1" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="304.4" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="311.8" y="145.5" width="3.7" height="3.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="319.1" y="145.3" width="3.7" height="3.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="326.5" y="144.2" width="3.7" height="5.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="333.8" y="142.8" width="3.7" height="8.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="341.2" y="141.1" width="3.7" height="11.9" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="348.5" y="139.2" width="3.7" height="15.7" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="355.9" y="137.2" width="3.7" height="19.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="363.2" y="135.3" width="3.7" height="23.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="370.6" y="133.6" width="3.7" height="26.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="377.9" y="132.3" width="3.7" height="29.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="385.3" y="131.4" width="3.7" height="31.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="392.6" y="131.0" width="3.7" height="32.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="400.0" y="131.1" width="3.7" height="31.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="407.4" y="131.7" width="3.7" height="30.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="414.7" y="132.8" width="3.7" height="28.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="422.1" y="134.2" width="3.7" height="25.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="429.4" y="135.8" width="3.7" height="22.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="436.8" y="137.5" width="3.7" height="19.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="444.1" y="139.1" width="3.7" height="15.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="451.5" y="140.6" width="3.7" height="12.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="458.8" y="141.8" width="3.7" height="10.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="466.2" y="142.9" width="3.7" height="8.3" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="473.5" y="143.7" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="480.9" y="144.2" width="3.7" height="5.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="488.2" y="144.7" width="3.7" height="4.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="495.6" y="145.1" width="3.7" height="3.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="502.9" y="145.5" width="3.7" height="3.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="510.3" y="144.9" width="3.7" height="4.2" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="517.6" y="144.1" width="3.7" height="5.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="525.0" y="143.2" width="3.7" height="7.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="532.4" y="142.3" width="3.7" height="9.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="539.7" y="141.4" width="3.7" height="11.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="547.1" y="140.7" width="3.7" height="12.6" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="554.4" y="140.1" width="3.7" height="13.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="561.8" y="139.8" width="3.7" height="14.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="569.1" y="139.7" width="3.7" height="14.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="576.5" y="140.0" width="3.7" height="14.1" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="583.8" y="140.4" width="3.7" height="13.2" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="591.2" y="141.0" width="3.7" height="11.9" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="598.5" y="141.8" width="3.7" height="10.5" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="605.9" y="142.5" width="3.7" height="9.0" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="613.2" y="143.1" width="3.7" height="7.8" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="620.6" y="143.6" width="3.7" height="6.9" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="627.9" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="635.3" y="143.8" width="3.7" height="6.4" rx="1" fill="#85827a"/>+<rect class="s s1r1" x="642.6" y="143.6" width="3.7" height="6.7" rx="1" fill="#85827a"/>+<text x="672" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">clip.wav</text>+</g>+<rect class="s sweep1" x="150" y="128" width="1.5" height="38" fill="#e8593d"/>+<g class="s s1r2">+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"function_calls":[{"name":"set_lights","arguments":{"room":"kitchen","on":false}}],</text>+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "confidence":0.94,"audio_text":"turn off the kitchen lights","audio_language":"en"}</text>+</g>+<g class="s s2r0">+<text x="40" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">LOAD</text>+<text x="150" y="98" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">needle --model needle3.cact --tools tools.json</text>+</g>+<g class="s s2r1">+<text x="40" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">IN</text>+<text x="150" y="152" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12.5" fill="#e8e6df">turn off the kitchen lights</text>+</g>+<g class="s s2r2">+<text x="40" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="1.5" fill="#e8593d">OUT</text>+<text x="150" y="206" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve">{"function_calls":[{"name":"set_lights","arguments":{"room":"kitchen","on":false}}],</text>+<text x="150" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="12" fill="#e8e6df" xml:space="preserve"> "confidence":0.94}</text>+</g>+<text x="40" y="268" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10.5" fill="#4a4844">one binary: needle_load reads whichever model the .cact holds, and needle_complete answers text or a clip</text>+</svg>diff --git a/llms.txt b/llms.txtindex fea901d..2ad06d9 100644--- a/llms.txt+++ b/llms.txt@@ -26,6 +26,8 @@ pip install "cactus-needle[train]" # adds JAX + training deps for finetune- `needle.tool` - decorator that turns a function into a tool schema (attached as `fn._needle_tool`); `@needle.tool(triggers=[r"\b(turn|switch)\b.*\b(on|off)\b"])` adds case-insensitive regexes that route a matching request to this tool and require a call; a triggered call ships even below the confidence floor (raw schemas take the same `"triggers"` list beside `description`).- `needle.Field(default=..., *, description, enum, const, ge, le, gt, lt, multiple_of, min_length, max_length, pattern, format, min_items, max_items, unique_items)` - per-argument constraints; attach inline with `typing.Annotated`.- `needle.extract(text, schema, system=None, max_new_tokens=512, weights=None)` - one-shot extraction; returns a Pydantic instance if `schema` is a model, else a dict (or `None` if nothing matched). `weights` selects a tuned `.cact` and defaults to whatever the engine already has loaded, so extraction inherits the active tuned model unless you pass another path.+- `needle.transcribe(audio, language=None, keywords=None, word_timestamps=False, weights=None) -> dict` - speech to text in English, German, French, Spanish, Italian, Dutch and Polish, on the same engine and the same install as Needle. `audio` is a WAV path or 16 kHz mono float samples in [-1, 1], at most 30 s. Returns `{"text", "language", "ttft_ms", "decode_tps"}`, plus `"words"` with `start`, `end` and `probability` when `word_timestamps` is set. `language` is one of `en de fr es it nl pl`, or `None` to detect it; `keywords` is a list or a newline-separated string of words and phrases to favour. Silence and steady noise return an empty text and language. 16 kHz WAV files and raw samples need nothing extra; other sample rates need the `[mic]` extra for soxr. `whistle.cact` is fetched from `Cactus-Compute/whistle` on first use.+- `needle.Whistle(weights=None, buffer_size=262144)` - the same model as an object, for `whistle.embed(audio)` (the encoder output, one row of floats per 80 ms frame, flattened) or to hold one tuned `.cact`. One model per process, not thread-safe.## Defining tools (three equivalent ways)@@ -234,9 +236,9 @@ Adapt by swapping the Literal values for your own and keeping the shapes: closed- `needle run --checkpoint <base.safetensors> --query "..." --tools tools.json` - JAX reference inference from a checkpoint (dev path; normal inference is the Python `Needle` API above).- `needle generate-data` / `needle finetune` / `needle build [checkpoint] [--lora adapter] [--layers N] [--platform <folder>] [--out <path>]` - the fine-tuning pipeline; fine-tuning runs at the full 20 layers, `build` merges the adapter, slices the `--layers` rung (default 20), exports at 4 bits, and fetches the published `needle3.cact` on every run (without an adapter at full depth it ships that published 2-bit archive as is). `--platform` also downloads that platform's engine and header and makes `--out` a folder holding them plus `needle3.cact`.- `needle playground` - the browser UI locally (hosted at https://cactuscompute.com/needle).-- `needle whistle playground [clip.wav] [--language en] [--keywords "Siobhan, Krzysztof"] [--word-timestamps] [--weights whistle.cact]` - Whistle speech to text in the terminal: press Enter to speak, Enter again to stop, or pass a WAV file; `/language`, `/keywords`, `/timestamps` and `/file` change the settings between recordings. Needs the `[whistle]` extra. The Whistle engine and `whistle.cact` are fetched from `Cactus-Compute/whistle` on first use; `NEEDLE_WHISTLE_LIB_PATH` and `NEEDLE_WHISTLE_WEIGHTS` point at local ones.-- `needle whistle compare [clip.wav] [--weights whistle.cact]` - the same English speech through Whistle, Whisper tiny and base (openai-whisper) and Moonshine tiny v2 (moonshine-voice), one line per model with its weight size, time to first token, decode tokens per second, total time and transcript; `/file` runs a WAV between recordings. Whistle's timing comes from the engine's own JSON. Needs the `[whistle,whistle-compare]` extras.-- `needle whistle fetch [--platform-tag <tag>] [--out <dir>]` / `needle whistle download whistle | <platform> [--out <dir>]` - the Whistle counterparts of `needle fetch` and `needle download`: pre-download the Whistle engine library, the `whistle.cact` weights, or a platform folder with the `whistle` binary, `libwhistle.a` and `whistle.h`.+- `needle whistle playground [clip.wav] [--language en] [--keywords "Siobhan, Krzysztof"] [--word-timestamps] [--weights whistle.cact]` - Whistle speech to text in the terminal: press Enter to speak, Enter again to stop, or pass a WAV file; `/language`, `/keywords`, `/timestamps` and `/file` change the settings between recordings. Recording from the microphone needs the `[mic]` extra. `whistle.cact` is fetched from `Cactus-Compute/whistle` on first use and runs on the same engine as Needle; `NEEDLE_WHISTLE_WEIGHTS` points at a local copy and `NEEDLE3_LIB_PATH` at a local engine.+- `needle whistle compare [clip.wav] [--weights whistle.cact]` - the same English speech through Whistle, Whisper tiny and base (openai-whisper) and Moonshine tiny v2 (moonshine-voice), one line per model with its weight size, time to first token, decode tokens per second, total time and transcript; `/file` runs a WAV between recordings. Whistle's timing comes from the engine's own JSON. Needs the `[mic,compare]` extras.+- `needle download whistle [--out <dir>]` - pre-download the `whistle.cact` speech weights. There is one engine for both models, so `needle fetch` and `needle download <platform>` cover the library and the platform folders.- `needle fetch [--platform-tag <tag>] [--out <dir>]` - pre-download the inference engine for this machine (or another platform) into the cache; prints the path. For air-gapped devices, also see `NEEDLE3_LIB_PATH` and `HF_HUB_OFFLINE` at https://cactuscompute.com/blog/needle-python-docs.- `needle platform finetune | generate | jobs | models | files | billing` - the hosted platform from the CLI (section above).- `needle download needle3 | needle3.safetensors | model-<id> | <org>/<repo>[/<file>.cact] | <platform> [--out <dir>]` - pull the base Needle 3 archive, a checkpoint to fine-tune, a platform-trained model by id (`--depth N` for one size), a published `.cact` (single-archive repos need only `<org>/<repo>`), or pass a platform folder name (`macos-arm64`, `linux-x86_64`, `wasm`, `wasm-component`, ...) to fetch that platform's engine files into `<out>/<platform>/`. Native runners are marked executable; `wasm-component` contains the WASI Preview 2 component and WIT contract.diff --git a/needle/__init__.py b/needle/__init__.pyindex d1ea2c7..e838d39 100644--- a/needle/__init__.py+++ b/needle/__init__.py@@ -11,11 +11,11 @@ import warningsfrom .agent.tools import Field, build_schema, pydantic_schema, tool, _is_pydantic_modelfrom ._telemetry import track as _trackfrom ._worker import FineTuneWorker-from .whistle import Whistle+from .agent.whistle import Whistle, transcribe-__version__ = "3.0.1"+__version__ = "3.1.0"__all__ = ["Needle", "Whistle", "ExtractionValidationError", "tool", "Field", "extract",- "__version__"]+ "transcribe", "__version__"]class ExtractionValidationError(ValueError):@@ -168,11 +168,13 @@ def _lib(generation=2):lib.needle_init.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p]lib.needle_init.restype = ctypes.c_intlib.needle_complete.argtypes = [- ctypes.c_char_p, ctypes.c_int, ctypes.c_char_p, ctypes.c_int]+ ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int,+ ctypes.c_int, ctypes.c_char_p, ctypes.c_int]lib.needle_complete.restype = ctypes.c_intif generation >= 3:lib.needle_embed.argtypes = [- ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int]+ ctypes.c_char_p, ctypes.POINTER(ctypes.c_float), ctypes.c_int,+ ctypes.POINTER(ctypes.c_float), ctypes.c_int]lib.needle_embed.restype = ctypes.c_intlib.needle_reset.argtypes = []lib.needle_reset.restype = None@@ -286,7 +288,7 @@ class Needle:else:lib = _lib(self._generation)rc = lib.needle_complete(- text.encode("utf-8"), int(max_new_tokens), self._buffer,+ text.encode("utf-8"), None, 0, int(max_new_tokens), self._buffer,len(self._buffer))if rc < 0:detail = self._buffer.value.decode("utf-8", "replace")@@ -312,11 +314,11 @@ class Needle:if self._worker is not None:return self._worker.embed(text)lib = _lib(self._generation)- dim = lib.needle_embed(text.encode("utf-8"), None, 0)+ dim = lib.needle_embed(text.encode("utf-8"), None, 0, None, 0)if dim <= 0:raise RuntimeError(f"needle_embed failed (code {dim})")output = (ctypes.c_float * dim)()- rc = lib.needle_embed(text.encode("utf-8"), output, dim)+ rc = lib.needle_embed(text.encode("utf-8"), None, 0, output, dim)if rc != dim:raise RuntimeError(f"needle_embed failed (code {rc})")return list(output)diff --git a/needle/_worker.py b/needle/_worker.pyindex df76cb7..4e461a9 100644--- a/needle/_worker.py+++ b/needle/_worker.py@@ -52,7 +52,8 @@ def _load_library(path, generation=2):lib.needle_init.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p]lib.needle_init.restype = ctypes.c_int- lib.needle_complete.argtypes = [ctypes.c_char_p, ctypes.c_int,+ lib.needle_complete.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_float),+ ctypes.c_int, ctypes.c_int,ctypes.c_char_p, ctypes.c_int]lib.needle_complete.restype = ctypes.c_intif int(generation) >= 3:@@ -96,7 +97,7 @@ def _child():operation = request.get("operation")if operation == "complete":code = lib.needle_complete(- request["text"].encode("utf-8"),+ request["text"].encode("utf-8"), None, 0,int(request["max_new_tokens"]), output, len(output))if code < 0:_write_message(protocol, {diff --git a/needle/agent/fetch.py b/needle/agent/fetch.pyindex 44c79fa..b7bc93f 100644--- a/needle/agent/fetch.py+++ b/needle/agent/fetch.py@@ -5,33 +5,27 @@ import sysimport zipfileWHISTLE = "whistle"-NEEDLE3_WHISTLE = "needle3_whistle"ENGINE_REPOS = {2: "Cactus-Compute/needle2",3: "Cactus-Compute/needle3",- NEEDLE3_WHISTLE: "Cactus-Compute/needle3",WHISTLE: "Cactus-Compute/whistle",}ENGINE_VERSIONS = {2: "2.0.4",- 3: "3.0.3",- NEEDLE3_WHISTLE: "3.0.0",- WHISTLE: "1.0.0",+ 3: "3.1.0",+ WHISTLE: "2.0.0",}BASE_WEIGHTS = {2: "needle2.cact",3: "needle3.cact",- NEEDLE3_WHISTLE: "needle3.cact",WHISTLE: "whistle.cact",}-# Engines named instead of numbered: one library per name, beside the generations.-LIB_PATH_ENVS = {- NEEDLE3_WHISTLE: "NEEDLE3_WHISTLE_LIB_PATH",- WHISTLE: "NEEDLE_WHISTLE_LIB_PATH",-}-NAMED_ENGINES = tuple(LIB_PATH_ENVS)+# Weights published on their own, with no engine of their own: one engine runs them all.+WEIGHTS_ONLY = (WHISTLE,)+LIB_PATH_ENVS = {}+NAMED_ENGINES = ()CHECKPOINT_PREFIX = "checkpoints"# Backwards-compatible aliases for callers that explicitly fetch Needle 2.@@ -145,6 +139,8 @@ def unpublished_engine_wheels():missing, listings = [], {}for generation in ENGINE_VERSIONS:+ if generation in WEIGHTS_ONLY:+ continuerepo = ENGINE_REPOS[generation]if repo not in listings:# A repo that is private or not created yet holds none of its wheels.@@ -216,12 +212,12 @@ def base_weights(generation=2):def repo_prefix(generation=2):- """Where an engine lives inside its repo. needle3_whistle shares needle3's."""- return generation + "/" if generation == NEEDLE3_WHISTLE else ""+ """Where an engine lives inside its repo."""+ return ""def cache_dir(generation=2):- folder = generation if generation in NAMED_ENGINES else f"v{int(generation)}"+ folder = generation if generation in NAMED_ENGINES + WEIGHTS_ONLY else f"v{int(generation)}"return os.path.join(os.path.expanduser("~"), ".cache", "cactus-needle",folder, engine_version(generation))diff --git a/needle/agent/whistle.py b/needle/agent/whistle.pynew file mode 100644index 0000000..9f854dc--- /dev/null+++ b/needle/agent/whistle.py@@ -0,0 +1,365 @@+from __future__ import annotations++import array+import ctypes+import json+import os+import sys+import time+import wave++SAMPLE_RATE = 16000+LANGUAGES = ("en", "de", "fr", "es", "it", "nl", "pl")++_handle = None+_loaded = None+++def _weights_path():+ from .. import _base_weights_path+ from . import fetch++ return os.environ.get("NEEDLE_WHISTLE_WEIGHTS") or _base_weights_path(fetch.WHISTLE)+++def _lib():+ global _handle+ if _handle is None:+ from .. import _load_cdll++ lib = _load_cdll(3)+ samples = ctypes.POINTER(ctypes.c_float)+ lib.needle_load.argtypes = [ctypes.c_char_p, ctypes.c_uint64]+ lib.needle_load.restype = ctypes.c_int+ lib.needle_transcribe.argtypes = [samples, ctypes.c_int, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int,+ ctypes.c_char_p, ctypes.c_int]+ lib.needle_transcribe.restype = ctypes.c_int+ lib.needle_embed.argtypes = [ctypes.c_char_p, samples, ctypes.c_int, samples, ctypes.c_int]+ lib.needle_embed.restype = ctypes.c_int+ lib.needle_last_error.argtypes = []+ lib.needle_last_error.restype = ctypes.c_char_p+ _handle = lib+ return _handle+++def _read_wav(path):+ with wave.open(path, "rb") as source:+ channels, width, rate = source.getnchannels(), source.getsampwidth(), source.getframerate()+ raw = source.readframes(source.getnframes())+ if width == 3:+ values = [int.from_bytes(raw[i:i + 3], "little", signed=True) for i in range(0, len(raw), 3)]+ else:+ values = array.array({1: "B", 2: "h", 4: "i"}[width])+ values.frombytes(raw)+ if sys.byteorder == "big":+ values.byteswap()+ offset, scale = (128, 128.0) if width == 1 else (0, float(1 << (8 * width - 1)))+ scale *= channels+ mono = [(sum(frame) - offset * channels) / scale for frame in zip(*(values[c::channels] for c in range(channels)))]+ if rate == SAMPLE_RATE or not mono:+ return mono+ try:+ import numpy+ import soxr+ except ImportError:+ raise RuntimeError(f"resampling {rate} Hz audio to {SAMPLE_RATE} Hz needs soxr: pip install cactus-needle[mic]") from None+ return soxr.resample(numpy.asarray(mono, numpy.float32), rate, SAMPLE_RATE, quality="HQ")+++def _samples(audio):+ if isinstance(audio, (str, os.PathLike)):+ audio = _read_wav(os.fspath(audio))+ data = array.array("f")+ if isinstance(audio, (bytes, bytearray)):+ data.frombytes(audio)+ elif isinstance(audio, array.array) and audio.typecode == "f":+ data = audio+ elif hasattr(audio, "tobytes") and getattr(audio, "dtype", None) is not None:+ data.frombytes(audio.astype("float32", copy=False).tobytes())+ else:+ data.extend(audio)+ return (ctypes.c_float * len(data)).from_buffer(data) if len(data) else (ctypes.c_float * 1)(), len(data)+++class Whistle:+ """Speech-to-text in English, German, French, Spanish, Italian, Dutch and Polish.++ Audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.+ One model is loaded per process; it is not thread-safe.+ """++ def __init__(self, weights=None, buffer_size=1 << 18):+ self.weights = os.fspath(weights) if weights is not None else _weights_path()+ self._buffer = ctypes.create_string_buffer(buffer_size)+ self._bind()++ def _bind(self):+ global _loaded+ lib = _lib()+ if _loaded == self.weights:+ return lib+ with open(self.weights, "rb") as handle:+ data = handle.read()+ if lib.needle_load(data, len(data)) < 0:+ _loaded = None+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))+ _loaded = self.weights+ return lib++ def transcribe(self, audio, language=None, keywords=None, word_timestamps=False) -> dict:+ """Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start, end and probability when word_timestamps is set.++ language is one of LANGUAGES, or None to detect it. keywords are words or phrases for keyword biasing.+ """+ lib = self._bind()+ samples, count = _samples(audio)+ if keywords is not None and not isinstance(keywords, str):+ keywords = "\n".join(keywords)+ code = lib.needle_transcribe(samples, count, language.encode("utf-8") if language else None,+ keywords.encode("utf-8") if keywords else None, int(bool(word_timestamps)),+ self._buffer, len(self._buffer))+ if code < 0:+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))+ return json.loads(self._buffer.value.decode("utf-8", "replace"))++ def embed(self, audio) -> list[float]:+ """The encoder output, one row of floats per 80 ms frame, flattened."""+ lib = self._bind()+ samples, count = _samples(audio)+ size = lib.needle_embed(None, samples, count, None, 0)+ if size < 0:+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))+ output = (ctypes.c_float * size)()+ if lib.needle_embed(None, samples, count, output, size) != size:+ raise RuntimeError(lib.needle_last_error().decode("utf-8", "replace"))+ return list(output)+++_shared = {}+++def _shared_model(weights):+ key = os.fspath(weights) if weights is not None else None+ if key not in _shared:+ _shared[key] = Whistle(weights=weights)+ return _shared[key]+++def transcribe(audio, language=None, keywords=None, word_timestamps=False, weights=None) -> dict:+ """Speech to text, on the model this process already has loaded.++ audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.+ Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start,+ end and probability when word_timestamps is set. language is one of LANGUAGES,+ or None to detect it; keywords are words or phrases for keyword biasing.+ """+ from .._telemetry import track++ track("transcribe", {"tuned": bool(weights), "timestamps": bool(word_timestamps)})+ return _shared_model(weights).transcribe(audio, language=language, keywords=keywords,+ word_timestamps=word_timestamps)+++PLAYGROUND_HELP = """ Enter speak, Enter again to stop+ /language de force a language (en de fr es it nl pl), bare to detect it+ /keywords Siobhan, Krzysztof+ /timestamps toggle word times+ /file clip.wav transcribe a file+ /quit leave"""+COMPARE_HELP = """ Enter speak, Enter again to stop+ /file clip.wav run a file through every model+ /quit leave"""+COMPARE_INSTALL = 'pip install "cactus-needle[mic,compare]"'+CLEAR_ABOVE = "\x1b[1A\x1b[2K"+LIMIT_SECONDS = 30+++def audio_path(rest):+ """The path in a /file line, as a terminal hands it over: quoted, or with escaped spaces."""+ path = rest.strip()+ if len(path) > 1 and path[0] == path[-1] and path[0] in "'\"":+ return path[1:-1]+ return path.replace("\\ ", " ")+++def status(name, size, ttft, tokens, total):+ ttft = f"{ttft * 1000:.0f} ms" if ttft is not None else "-"+ tokens = f"{tokens:.0f} tok/s" if tokens is not None else "-"+ spent = f"{total:.1f} s" if total >= 10 else f"{total * 1000:.0f} ms"+ return f" {name:<18}{size / 1e6:>4.0f} MB ttft {ttft:>6} decode {tokens:>10} total {spent:>7}"+++def record():+ try:+ import numpy+ import sounddevice+ import soxr+ except ImportError as error:+ raise RuntimeError(f'{error.name} is not installed: pip install "cactus-needle[mic]"') from None+ chunks = []+ try:+ rate = int(sounddevice.query_devices(kind="input")["default_samplerate"])+ with sounddevice.InputStream(samplerate=rate, channels=1, dtype="float32", callback=lambda data, *_: chunks.append(data.copy())):+ print(CLEAR_ABOVE + "● recording, Enter to stop", end="", flush=True)+ try:+ input()+ except (EOFError, KeyboardInterrupt):+ print()+ except sounddevice.PortAudioError as error:+ raise RuntimeError(f"no microphone: {error}") from None+ print(CLEAR_ABOVE, end="", flush=True)+ heard = numpy.concatenate(chunks)[:, 0] if chunks else numpy.zeros(0, numpy.float32)+ audio = heard[:LIMIT_SECONDS * rate]+ if len(audio) < len(heard):+ print(f" keeping the first {LIMIT_SECONDS} s")+ return audio if rate == SAMPLE_RATE else soxr.resample(audio, rate, SAMPLE_RATE, quality="HQ")+++def prompt():+ try:+ return input("› ").strip()+ except (EOFError, KeyboardInterrupt):+ print()+ return "/quit"+++def show_transcript(whistle, audio, state):+ started = time.perf_counter()+ try:+ result = whistle.transcribe(audio, language=state["language"], keywords=state["keywords"], word_timestamps=state["timestamps"])+ except (RuntimeError, OSError, EOFError) as error:+ print(f" {error}")+ return+ total = time.perf_counter() - started+ print(result["text"] or "(no speech)")+ for word in result.get("words", []):+ print(f" {word['start']:6.2f} - {word['end']:5.2f} {word['word']:<20} {word['probability']:.2f}")+ print(status("whistle", os.path.getsize(whistle.weights), result["ttft_ms"] / 1000, result["decode_tps"], total),+ " ", result["language"] or "no speech")+++def playground(args):+ state = {"language": args.language, "keywords": [k.strip() for k in args.keywords.split(",") if k.strip()], "timestamps": args.word_timestamps}+ print("whistle playground: downloading and initializing the model...", flush=True)+ whistle = Whistle(weights=args.weights)+ if args.audio:+ return show_transcript(whistle, args.audio, state)+ print(PLAYGROUND_HELP)+ while True:+ line = prompt()+ command, _, rest = line.partition(" ")+ if command == "/quit":+ return+ if command == "/language":+ language = rest.strip()+ if language and language not in LANGUAGES:+ print(" language is one of", " ".join(LANGUAGES))+ continue+ state["language"] = language or None+ print(" language", state["language"] or "detected")+ elif command in ("/keywords", "/keyword"):+ state["keywords"] = [k.strip() for k in rest.split(",") if k.strip()]+ print(" keywords", ", ".join(state["keywords"]) or "none")+ elif command == "/timestamps":+ state["timestamps"] = not state["timestamps"]+ print(" word timestamps", "on" if state["timestamps"] else "off")+ elif command == "/file":+ show_transcript(whistle, audio_path(rest), state)+ elif line == "":+ try:+ show_transcript(whistle, record(), state)+ except RuntimeError as error:+ print(f" {error}")+ else:+ print(PLAYGROUND_HELP)+++def rate(marks):+ return (len(marks) - 1) / (marks[-1] - marks[0]) if len(marks) > 1 and marks[-1] > marks[0] else 0.0+++def load_whistle(weights):+ whistle = Whistle(weights=weights)++ def transcribe(audio):+ result = whistle.transcribe(audio, language="en")+ return result["text"], result["ttft_ms"] / 1000, result["decode_tps"]++ return os.path.getsize(whistle.weights), transcribe+++def load_whisper(size):+ import whisper++ model = whisper.load_model(size, device="cpu")+ cache = os.path.join(os.getenv("XDG_CACHE_HOME", os.path.join(os.path.expanduser("~"), ".cache")), "whisper")+ marks = []+ model.decoder.register_forward_hook(lambda *_: marks.append(time.perf_counter()))++ def transcribe(audio):+ marks.clear()+ started = time.perf_counter()+ text = model.transcribe(audio, language="en", fp16=False)["text"].strip()+ return text, marks[0] - started, rate(marks)++ return os.path.getsize(os.path.join(cache, f"{size}.pt")), transcribe+++def load_moonshine():+ import moonshine_voice++ path, arch = moonshine_voice.get_model_for_language("en", moonshine_voice.ModelArch.TINY_STREAMING)+ model = moonshine_voice.Transcriber(model_path=path, model_arch=arch)++ def transcribe(audio):+ lines = model.transcribe_without_streaming(audio.tolist(), SAMPLE_RATE).lines+ return " ".join(line.text.strip() for line in lines).strip(), None, None++ return sum(os.path.getsize(os.path.join(path, f)) for f in os.listdir(path)), transcribe+++def run_models(models, audio):+ for name, size, transcribe in models:+ started = time.perf_counter()+ text, ttft, tokens = transcribe(audio)+ print(status(name, size, ttft, tokens, time.perf_counter() - started), " ", text or "(no speech)")+++def compare(args):+ try:+ import moonshine_voice # noqa: F401+ import numpy+ import whisper # noqa: F401+ except ImportError as error:+ raise SystemExit(f"{error.name} is not installed: {COMPARE_INSTALL}")+ print("whistle compare: downloading and initializing the models...", flush=True)+ models = [+ ("whistle", *load_whistle(args.weights)),+ ("whisper tiny", *load_whisper("tiny")),+ ("whisper base", *load_whisper("base")),+ ("moonshine tiny v2", *load_moonshine()),+ ]+ beat = numpy.arange(SAMPLE_RATE) * (numpy.pi / SAMPLE_RATE)+ warmup = (numpy.sin(440 * beat) * numpy.abs(numpy.sin(3 * beat))).astype(numpy.float32)+ for _, _, transcribe in models:+ transcribe(warmup)+ if args.audio:+ return run_models(models, numpy.asarray(_read_wav(args.audio), numpy.float32))+ print(COMPARE_HELP)+ while True:+ line = prompt()+ command, _, rest = line.partition(" ")+ if command == "/quit":+ return+ if command == "/file":+ try:+ run_models(models, numpy.asarray(_read_wav(audio_path(rest)), numpy.float32))+ except OSError as error:+ print(f" {error}")+ elif line == "":+ try:+ run_models(models, numpy.asarray(record(), numpy.float32))+ except RuntimeError as error:+ print(f" {error}")+ else:+ print(COMPARE_HELP)diff --git a/needle/cli.py b/needle/cli.pyindex 7ab4000..71521c0 100644--- a/needle/cli.py+++ b/needle/cli.py@@ -14,7 +14,7 @@ HELP = """usage: needle <command> [options]download needle3 | needle3.safetensors | <platform> | model-<id> | <org>/<repo>[/<file>.cact]fetch fetch the engine library for this platformplayground serve the browser playground- whistle speech to text: playground | compare | fetch | download+ whistle speech to text: playground | compareneedle <command> --help for the options of one command.Check the readme for the rest."""@@ -265,7 +265,7 @@ def main():p.add_argument("--port", type=int, default=7860)p.add_argument("--host", type=str, default="127.0.0.1")- from .whistle import LANGUAGES+ from .agent.whistle import LANGUAGESp = sub.add_parser("whistle")verbs = p.add_subparsers(dest="verb")v = verbs.add_parser("playground")@@ -284,14 +284,6 @@ def main():help="A WAV file to run through every model once (default: press Enter to speak)")v.add_argument("--weights", type=str, default=None,help="whistle.cact to load (default: the published weights)")- v = verbs.add_parser("fetch")- v.add_argument("--out", type=str, default=None,- help="Directory to place the engine (default: the cache)")- v.add_argument("--platform-tag", type=str, default=None,- help="Fetch the build for another device, e.g. manylinux2014_aarch64")- v = verbs.add_parser("download")- v.add_argument("spec", type=str, help="whistle (the weights) or a platform folder (e.g. macos-arm64)")- v.add_argument("--out", type=str, default=".", help="Directory to place the files")args = parser.parse_args()@@ -303,9 +295,6 @@ def main():track("cli:" + args.command,{"generation": getattr(args, "generation", 2)})- if args.command == "whistle" and args.verb in ("fetch", "download"):- args.command, args.generation = args.verb, "whistle"-if args.command == "run":from .model.run import main as run_mainrun_main(args)@@ -341,7 +330,7 @@ def main():path = fetch.fetch_weights(target, args.out)print(f" {'weights':<9} {path} {os.path.getsize(path) / 1e6:.2f} MB")if target == fetch.WHISTLE:- print(f" {'next':<9} needle.Whistle(weights={path!r})")+ print(f" {'next':<9} needle.transcribe('clip.wav', weights={path!r})")else:print(f" {'next':<9} needle.Needle(weights={path!r}, tools=[...])")elif kind == "hosted":@@ -389,9 +378,9 @@ def main():playground_main(args)elif args.command == "whistle":if args.verb == "playground":- from .whistle.playground import main as whistle_main+ from .agent.whistle import playground as whistle_mainelif args.verb == "compare":- from .whistle.compare import main as whistle_main+ from .agent.whistle import compare as whistle_mainelse:- raise SystemExit("needle whistle playground | compare | fetch | download")+ raise SystemExit("needle whistle playground | compare")whistle_main(args)diff --git a/needle/whistle/__init__.py b/needle/whistle/__init__.pydeleted file mode 100644index bc4be7c..0000000--- a/needle/whistle/__init__.py+++ /dev/null@@ -1,135 +0,0 @@-from __future__ import annotations--import array-import ctypes-import json-import os-import sys-import wave--SAMPLE_RATE = 16000-LANGUAGES = ("en", "de", "fr", "es", "it", "nl", "pl")--_handle = None-_loaded = None---def _weights_path():- from .. import _base_weights_path- from ..agent import fetch-- return os.environ.get("NEEDLE_WHISTLE_WEIGHTS") or _base_weights_path(fetch.WHISTLE)---def _lib():- global _handle- if _handle is None:- from .. import _load_cdll- from ..agent import fetch-- lib = _load_cdll(fetch.WHISTLE)- samples = ctypes.POINTER(ctypes.c_float)- lib.whistle_load.argtypes = [ctypes.c_char_p, ctypes.c_uint64]- lib.whistle_load.restype = ctypes.c_int- lib.whistle_transcribe.argtypes = [samples, ctypes.c_int, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int,- ctypes.c_char_p, ctypes.c_int]- lib.whistle_transcribe.restype = ctypes.c_int- lib.whistle_embed.argtypes = [samples, ctypes.c_int, samples, ctypes.c_int]- lib.whistle_embed.restype = ctypes.c_int- lib.whistle_last_error.argtypes = []- lib.whistle_last_error.restype = ctypes.c_char_p- _handle = lib- return _handle---def _read_wav(path):- with wave.open(path, "rb") as source:- channels, width, rate = source.getnchannels(), source.getsampwidth(), source.getframerate()- raw = source.readframes(source.getnframes())- if width == 3:- values = [int.from_bytes(raw[i:i + 3], "little", signed=True) for i in range(0, len(raw), 3)]- else:- values = array.array({1: "B", 2: "h", 4: "i"}[width])- values.frombytes(raw)- if sys.byteorder == "big":- values.byteswap()- offset, scale = (128, 128.0) if width == 1 else (0, float(1 << (8 * width - 1)))- scale *= channels- mono = [(sum(frame) - offset * channels) / scale for frame in zip(*(values[c::channels] for c in range(channels)))]- if rate == SAMPLE_RATE or not mono:- return mono- try:- import numpy- import soxr- except ImportError:- raise RuntimeError(f"resampling {rate} Hz audio to {SAMPLE_RATE} Hz needs soxr: pip install cactus-needle[whistle]") from None- return soxr.resample(numpy.asarray(mono, numpy.float32), rate, SAMPLE_RATE, quality="HQ")---def _samples(audio):- if isinstance(audio, (str, os.PathLike)):- audio = _read_wav(os.fspath(audio))- data = array.array("f")- if isinstance(audio, (bytes, bytearray)):- data.frombytes(audio)- elif isinstance(audio, array.array) and audio.typecode == "f":- data = audio- elif hasattr(audio, "tobytes") and getattr(audio, "dtype", None) is not None:- data.frombytes(audio.astype("float32", copy=False).tobytes())- else:- data.extend(audio)- return (ctypes.c_float * len(data)).from_buffer(data) if len(data) else (ctypes.c_float * 1)(), len(data)---class Whistle:- """Speech-to-text in English, German, French, Spanish, Italian, Dutch and Polish.-- Audio is a WAV file path or 16 kHz mono float samples in [-1, 1], at most 30 s.- One model is loaded per process; it is not thread-safe.- """-- def __init__(self, weights=None, buffer_size=1 << 18):- self.weights = os.fspath(weights) if weights is not None else _weights_path()- self._buffer = ctypes.create_string_buffer(buffer_size)- self._bind()-- def _bind(self):- global _loaded- lib = _lib()- if _loaded == self.weights:- return lib- with open(self.weights, "rb") as handle:- data = handle.read()- if lib.whistle_load(data, len(data)) < 0:- _loaded = None- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))- _loaded = self.weights- return lib-- def transcribe(self, audio, language=None, keywords=None, word_timestamps=False) -> dict:- """Returns {"text", "language", "ttft_ms", "decode_tps"}, plus "words" with start, end and probability when word_timestamps is set.-- language is one of LANGUAGES, or None to detect it. keywords are words or phrases for keyword biasing.- """- lib = self._bind()- samples, count = _samples(audio)- if keywords is not None and not isinstance(keywords, str):- keywords = "\n".join(keywords)- code = lib.whistle_transcribe(samples, count, language.encode("utf-8") if language else None,- keywords.encode("utf-8") if keywords else None, int(bool(word_timestamps)),- self._buffer, len(self._buffer))- if code < 0:- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))- return json.loads(self._buffer.value.decode("utf-8", "replace"))-- def embed(self, audio) -> list[float]:- """The encoder output, one row of floats per 80 ms frame, flattened."""- lib = self._bind()- samples, count = _samples(audio)- size = lib.whistle_embed(samples, count, None, 0)- if size < 0:- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))- output = (ctypes.c_float * size)()- if lib.whistle_embed(samples, count, output, size) != size:- raise RuntimeError(lib.whistle_last_error().decode("utf-8", "replace"))- return list(output)diff --git a/needle/whistle/compare.py b/needle/whistle/compare.pydeleted file mode 100644index c975260..0000000--- a/needle/whistle/compare.py+++ /dev/null@@ -1,106 +0,0 @@-import os-import time--from .playground import audio_path, prompt, record, status--HELP = """ Enter speak, Enter again to stop- /file clip.wav run a file through every model- /quit leave"""-INSTALL = 'pip install "cactus-needle[whistle,whistle-compare]"'---def rate(marks):- return (len(marks) - 1) / (marks[-1] - marks[0]) if len(marks) > 1 and marks[-1] > marks[0] else 0.0---def load_whistle(weights):- from . import Whistle-- whistle = Whistle(weights=weights)-- def transcribe(audio):- result = whistle.transcribe(audio, language="en")- return result["text"], result["ttft_ms"] / 1000, result["decode_tps"]-- return os.path.getsize(whistle.weights), transcribe---def load_whisper(size):- import whisper-- model = whisper.load_model(size, device="cpu")- cache = os.path.join(os.getenv("XDG_CACHE_HOME", os.path.join(os.path.expanduser("~"), ".cache")), "whisper")- marks = []- model.decoder.register_forward_hook(lambda *_: marks.append(time.perf_counter()))-- def transcribe(audio):- marks.clear()- started = time.perf_counter()- text = model.transcribe(audio, language="en", fp16=False)["text"].strip()- return text, marks[0] - started, rate(marks)-- return os.path.getsize(os.path.join(cache, f"{size}.pt")), transcribe---def load_moonshine():- import moonshine_voice-- from . import SAMPLE_RATE-- path, arch = moonshine_voice.get_model_for_language("en", moonshine_voice.ModelArch.TINY_STREAMING)- model = moonshine_voice.Transcriber(model_path=path, model_arch=arch)-- def transcribe(audio):- lines = model.transcribe_without_streaming(audio.tolist(), SAMPLE_RATE).lines- return " ".join(line.text.strip() for line in lines).strip(), None, None-- return sum(os.path.getsize(os.path.join(path, f)) for f in os.listdir(path)), transcribe---def compare(models, audio):- for name, size, transcribe in models:- started = time.perf_counter()- text, ttft, tokens = transcribe(audio)- print(status(name, size, ttft, tokens, time.perf_counter() - started), " ", text or "(no speech)")---def main(args):- from . import SAMPLE_RATE, _read_wav-- try:- import moonshine_voice # noqa: F401- import numpy- import whisper # noqa: F401- except ImportError as error:- raise SystemExit(f"{error.name} is not installed: {INSTALL}")- print("whistle compare: downloading and initializing the models...", flush=True)- models = [- ("whistle", *load_whistle(args.weights)),- ("whisper tiny", *load_whisper("tiny")),- ("whisper base", *load_whisper("base")),- ("moonshine tiny v2", *load_moonshine()),- ]- beat = numpy.arange(SAMPLE_RATE) * (numpy.pi / SAMPLE_RATE)- warmup = (numpy.sin(440 * beat) * numpy.abs(numpy.sin(3 * beat))).astype(numpy.float32)- for _, _, transcribe in models:- transcribe(warmup)- if args.audio:- return compare(models, numpy.asarray(_read_wav(args.audio), numpy.float32))- print(HELP)- while True:- line = prompt()- command, _, rest = line.partition(" ")- if command == "/quit":- return- if command == "/file":- try:- compare(models, numpy.asarray(_read_wav(audio_path(rest)), numpy.float32))- except OSError as error:- print(f" {error}")- elif line == "":- try:- compare(models, numpy.asarray(record(), numpy.float32))- except RuntimeError as error:- print(f" {error}")- else:- print(HELP)diff --git a/needle/whistle/playground.py b/needle/whistle/playground.pydeleted file mode 100644index ea283b4..0000000--- a/needle/whistle/playground.py+++ /dev/null@@ -1,115 +0,0 @@-import os-import time--HELP = """ Enter speak, Enter again to stop- /language de force a language (en de fr es it nl pl), bare to detect it- /keywords Siobhan, Krzysztof- /timestamps toggle word times- /file clip.wav transcribe a file- /quit leave"""-CLEAR_ABOVE = "\x1b[1A\x1b[2K"-LIMIT_SECONDS = 30---def audio_path(rest):- """The path in a /file line, as a terminal hands it over: quoted, or with escaped spaces."""- path = rest.strip()- if len(path) > 1 and path[0] == path[-1] and path[0] in "'\"":- return path[1:-1]- return path.replace("\\ ", " ")---def status(name, size, ttft, tokens, total):- ttft = f"{ttft * 1000:.0f} ms" if ttft is not None else "-"- tokens = f"{tokens:.0f} tok/s" if tokens is not None else "-"- spent = f"{total:.1f} s" if total >= 10 else f"{total * 1000:.0f} ms"- return f" {name:<18}{size / 1e6:>4.0f} MB ttft {ttft:>6} decode {tokens:>10} total {spent:>7}"---def record():- from . import SAMPLE_RATE-- try:- import numpy- import sounddevice- import soxr- except ImportError as error:- raise RuntimeError(f'{error.name} is not installed: pip install "cactus-needle[whistle]"') from None- chunks = []- try:- rate = int(sounddevice.query_devices(kind="input")["default_samplerate"])- with sounddevice.InputStream(samplerate=rate, channels=1, dtype="float32", callback=lambda data, *_: chunks.append(data.copy())):- print(CLEAR_ABOVE + "● recording, Enter to stop", end="", flush=True)- try:- input()- except (EOFError, KeyboardInterrupt):- print()- except sounddevice.PortAudioError as error:- raise RuntimeError(f"no microphone: {error}") from None- print(CLEAR_ABOVE, end="", flush=True)- heard = numpy.concatenate(chunks)[:, 0] if chunks else numpy.zeros(0, numpy.float32)- audio = heard[:LIMIT_SECONDS * rate]- if len(audio) < len(heard):- print(f" keeping the first {LIMIT_SECONDS} s")- return audio if rate == SAMPLE_RATE else soxr.resample(audio, rate, SAMPLE_RATE, quality="HQ")---def prompt():- try:- return input("› ").strip()- except (EOFError, KeyboardInterrupt):- print()- return "/quit"---def transcribe(whistle, audio, state):- started = time.perf_counter()- try:- result = whistle.transcribe(audio, language=state["language"], keywords=state["keywords"], word_timestamps=state["timestamps"])- except (RuntimeError, OSError, EOFError) as error:- print(f" {error}")- return- total = time.perf_counter() - started- print(result["text"] or "(no speech)")- for word in result.get("words", []):- print(f" {word['start']:6.2f} - {word['end']:5.2f} {word['word']:<20} {word['probability']:.2f}")- print(status("whistle", os.path.getsize(whistle.weights), result["ttft_ms"] / 1000, result["decode_tps"], total),- " ", result["language"] or "no speech")---def main(args):- from . import LANGUAGES, Whistle-- state = {"language": args.language, "keywords": [k.strip() for k in args.keywords.split(",") if k.strip()], "timestamps": args.word_timestamps}- print("whistle playground: downloading and initializing the model...", flush=True)- whistle = Whistle(weights=args.weights)- if args.audio:- return transcribe(whistle, args.audio, state)- print(HELP)- while True:- line = prompt()- command, _, rest = line.partition(" ")- if command == "/quit":- return- if command == "/language":- language = rest.strip()- if language and language not in LANGUAGES:- print(" language is one of", " ".join(LANGUAGES))- continue- state["language"] = language or None- print(" language", state["language"] or "detected")- elif command in ("/keywords", "/keyword"):- state["keywords"] = [k.strip() for k in rest.split(",") if k.strip()]- print(" keywords", ", ".join(state["keywords"]) or "none")- elif command == "/timestamps":- state["timestamps"] = not state["timestamps"]- print(" word timestamps", "on" if state["timestamps"] else "off")- elif command == "/file":- transcribe(whistle, audio_path(rest), state)- elif line == "":- try:- transcribe(whistle, record(), state)- except RuntimeError as error:- print(f" {error}")- else:- print(HELP)diff --git a/pyproject.toml b/pyproject.tomlindex 3cee9c7..82a4dec 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "cactus-needle"-version = "3.0.1"+version = "3.1.0"description = "A 14MB foundation tool-calling model for tiny devices: inference, LoRA finetuning, and build."requires-python = ">=3.9"readme = "README.md"@@ -25,8 +25,8 @@ metal = ["jaxlib==0.4.38","jax-metal"]-whistle = ["numpy", "soxr", "sounddevice"]-whistle-compare = ["openai-whisper", "moonshine-voice"]+mic = ["numpy", "soxr", "sounddevice"]+compare = ["openai-whisper", "moonshine-voice"]test = ["pytest", "pydantic"][project.scripts]diff --git a/tests/test_fetch.py b/tests/test_fetch.pyindex 0d76862..46c1a17 100644--- a/tests/test_fetch.py+++ b/tests/test_fetch.py@@ -110,7 +110,8 @@ def test_unpublished_engine_wheels_lists_every_missing_tag(monkeypatch):published = {}for generation, version in fetch.ENGINE_VERSIONS.items():- # needle3 hosts both its own engine and needle3_whistle, so the sets merge.+ if generation in fetch.WEIGHTS_ONLY:+ continuepublished.setdefault(fetch.ENGINE_REPOS[generation], set()).update(fetch.engine_wheel(version, tag, generation) for tag in fetch.WHEEL_TAGS)monkeypatch.setattr("huggingface_hub.list_repo_files",@@ -122,16 +123,8 @@ def test_unpublished_engine_wheels_lists_every_missing_tag(monkeypatch):published[repo3].remove(dropped)assert fetch.unpublished_engine_wheels() == [repo3 + "/" + dropped]- audio = fetch.engine_wheel(fetch.ENGINE_VERSIONS[fetch.NEEDLE3_WHISTLE], "win_arm64", fetch.NEEDLE3_WHISTLE)- published[repo3].remove(audio)- assert fetch.unpublished_engine_wheels() == [repo3 + "/" + dropped, repo3 + "/" + audio]- published[repo3].add(audio)-- whistle = fetch.ENGINE_REPOS[fetch.WHISTLE]- published[whistle].clear()- missing = fetch.unpublished_engine_wheels()- assert len(missing) == 1 + len(fetch.WHEEL_TAGS)- assert missing[-1] == f"{whistle}/python/cactus_whistle-{fetch.ENGINE_VERSIONS[fetch.WHISTLE]}-py3-none-win_arm64.whl"+ assert fetch.WHISTLE in fetch.WEIGHTS_ONLY+ assert not any(fetch.ENGINE_REPOS[fetch.WHISTLE] in path for path in fetch.unpublished_engine_wheels())def test_a_repo_that_cannot_be_listed_holds_none_of_its_wheels(monkeypatch):@@ -143,7 +136,8 @@ def test_a_repo_that_cannot_be_listed_holds_none_of_its_wheels(monkeypatch):monkeypatch.setattr("huggingface_hub.list_repo_files", refuse)missing = fetch.unpublished_engine_wheels()- assert len(missing) == len(fetch.ENGINE_VERSIONS) * len(fetch.WHEEL_TAGS)+ engines = len(fetch.ENGINE_VERSIONS) - len(fetch.WEIGHTS_ONLY)+ assert len(missing) == engines * len(fetch.WHEEL_TAGS)def test_release_gate_runs_before_the_publish_step():diff --git a/tests/test_whistle.py b/tests/test_whistle.pyindex d98e625..14a0c7e 100644--- a/tests/test_whistle.py+++ b/tests/test_whistle.py@@ -20,54 +20,72 @@ requires_whistle = pytest.mark.skipif(not (_engine_available("whistle") and os.preason="Whistle engine or whistle.cact not installed (auto-fetched from HF on first real use)")-def test_engine_and_weights_resolve_like_needle(tmp_path, monkeypatch):- import zipfile+class _Stub:+ class _Fn:+ argtypes = None+ restype = None++ def __getattr__(self, name):+ return _Stub._Fn()+++def test_speech_weights_come_from_their_own_repo(tmp_path, monkeypatch):+ """One engine runs both models, so only the weights have a channel of their own."""import needle- from needle import whistle+ from needle.agent import whistlefrom needle.agent import fetch+ assert fetch.WHISTLE in fetch.WEIGHTS_ONLY and fetch.NAMED_ENGINES == ()monkeypatch.setattr(os.path, "expanduser", lambda path: str(tmp_path))monkeypatch.setattr(needle, "__file__", str(tmp_path / "package" / "__init__.py"))monkeypatch.setattr(fetch, "_register_download", lambda generation: None)monkeypatch.delenv("NEEDLE_WHISTLE_WEIGHTS", raising=False)- monkeypatch.setenv("NEEDLE_WHISTLE_LIB_PATH", "/opt/libwhistle.so")- assert needle._library_path(fetch.WHISTLE) == "/opt/libwhistle.so"- monkeypatch.delenv("NEEDLE_WHISTLE_LIB_PATH")-- lib = fetch.lib_name(fetch.WHISTLE)- wheel, weights, fetched = tmp_path / "engine.whl", tmp_path / "published.cact", []- with zipfile.ZipFile(wheel, "w") as archive:- archive.writestr("needle/" + lib, b"engine")++ weights, fetched = tmp_path / "published.cact", []weights.write_bytes(b"weights")monkeypatch.setattr("huggingface_hub.hf_hub_download",- lambda **kwargs: fetched.append((kwargs["repo_id"], kwargs["filename"])) or str(weights if kwargs["filename"].endswith(".cact") else wheel))+ lambda **kwargs: fetched.append((kwargs["repo_id"], kwargs["filename"])) or str(weights))cache = tmp_path / ".cache" / "cactus-needle" / "whistle" / fetch.ENGINE_VERSIONS[fetch.WHISTLE]- assert lib.startswith("libwhistle.") and needle._library_path(fetch.WHISTLE) == str(cache / lib) and (cache / lib).read_bytes() == b"engine"- assert whistle._weights_path() == str(cache / "whistle.cact") and (cache / "whistle.cact").read_bytes() == b"weights"- assert fetched == [("Cactus-Compute/whistle", f"python/cactus_whistle-{fetch.ENGINE_VERSIONS[fetch.WHISTLE]}-py3-none-{fetch._platform_tag()}.whl"),- ("Cactus-Compute/whistle", "whistle.cact")]- assert needle._library_path(fetch.WHISTLE) and whistle._weights_path() and len(fetched) == 2+ assert whistle._weights_path() == str(cache / "whistle.cact")+ assert (cache / "whistle.cact").read_bytes() == b"weights"+ assert fetched == [("Cactus-Compute/whistle", "whistle.cact")]monkeypatch.setenv("NEEDLE_WHISTLE_WEIGHTS", "/opt/whistle.cact")assert whistle._weights_path() == "/opt/whistle.cact"-def test_cli_fetches_the_whistle_engine_and_weights(tmp_path, monkeypatch, capsys):- import sys- import needle._telemetry- import needle.cli- from needle.agent import fetch+def test_module_level_transcribe_reuses_one_model(monkeypatch):+ import needle+ from needle.agent import whistlecalls = []- monkeypatch.setattr(needle._telemetry, "track", lambda *a, **k: None)- monkeypatch.setattr(fetch, "fetch_library", lambda version, dest, tag=None, generation=2: calls.append((version, tag, generation)) or os.path.join(dest, "libwhistle.so"))- monkeypatch.setattr(fetch, "fetch_weights", lambda generation, dest: calls.append((generation, dest)) or __file__)- monkeypatch.setattr(sys, "argv", ["needle", "whistle", "fetch", "--out", str(tmp_path), "--platform-tag", "manylinux2014_aarch64"])- needle.cli.main()- monkeypatch.setattr(sys, "argv", ["needle", "whistle", "download", "whistle", "--out", str(tmp_path)])- needle.cli.main()- assert calls == [(fetch.ENGINE_VERSIONS[fetch.WHISTLE], "manylinux2014_aarch64", fetch.WHISTLE), (fetch.WHISTLE, str(tmp_path))]- out = capsys.readouterr().out- assert "NEEDLE_WHISTLE_LIB_PATH" in out and "needle.Whistle(weights=" in out++ class _Fake:+ def __init__(self, weights=None):+ calls.append(weights)+ self.weights = weights++ def transcribe(self, audio, language=None, keywords=None, word_timestamps=False):+ return {"text": audio, "language": language, "words": word_timestamps}++ monkeypatch.setattr(whistle, "Whistle", _Fake)+ monkeypatch.setattr(whistle, "_shared", {})+ monkeypatch.setattr("needle._telemetry.track", lambda *a, **k: None)+ assert needle.transcribe("a.wav")["text"] == "a.wav"+ assert needle.transcribe("b.wav", language="de")["language"] == "de"+ assert calls == [None]+ needle.transcribe("c.wav", weights="tuned.cact")+ assert calls == [None, "tuned.cact"]+++def test_the_speech_model_loads_into_needle_s_engine(monkeypatch):+ from needle.agent import whistle++ loaded = []+ monkeypatch.setattr(whistle, "_handle", None)+ monkeypatch.setattr("needle._load_cdll", lambda generation: loaded.append(generation) or _Stub())+ whistle._lib()+ monkeypatch.setattr(whistle, "_handle", None)+ assert loaded == [3]def _tone(seconds, rate=16000):@@ -88,7 +106,7 @@ def _write_wav(path, samples, rate, channels=1, width=2):@pytest.mark.parametrize("rate,channels,width", [(16000, 1, 2), (8000, 1, 2), (44100, 2, 2), (16000, 1, 1), (22050, 2, 3), (48000, 1, 4)])def test_wav_files_become_16k_mono_floats(tmp_path, rate, channels, width):- from needle.whistle import _read_wav+ from needle.agent.whistle import _read_wavif rate != 16000:pytest.importorskip("soxr")@@ -102,7 +120,7 @@ def test_wav_files_become_16k_mono_floats(tmp_path, rate, channels, width):def test_samples_accept_lists_arrays_and_float32_bytes():- from needle.whistle import _samples+ from needle.agent.whistle import _samplesvalues = [0.0, 0.5, -0.25]inputs = [values, array.array("f", values), struct.pack("<3f", *values), (v for v in values)]@@ -125,7 +143,7 @@ def test_silence_is_an_empty_transcript():@requires_whistledef test_transcribe_returns_text_language_and_timed_words(tmp_path):from needle import Whistle- from needle.whistle import LANGUAGES+ from needle.agent.whistle import LANGUAGESwhistle = Whistle()sound = [v * (0.2 + 0.8 * abs(math.sin(math.pi * 3 * i / 16000))) for i, v in enumerate(_tone(3))]@@ -204,7 +222,7 @@ class _Microphone:def test_record_resamples_the_microphone_to_16_khz_and_keeps_30_s(monkeypatch, capsys):numpy = pytest.importorskip("numpy")pytest.importorskip("soxr")- from needle.whistle.playground import record+ from needle.agent.whistle import recordmicrophone = _Microphone(48000, 1)monkeypatch.setitem(sys.modules, "sounddevice", microphone)@@ -219,26 +237,26 @@ def test_record_resamples_the_microphone_to_16_khz_and_keeps_30_s(monkeypatch, cdef test_record_says_what_is_missing(monkeypatch):- from needle.whistle.playground import record+ from needle.agent.whistle import recordmonkeypatch.setitem(sys.modules, "sounddevice", None)- with pytest.raises(RuntimeError, match=r"cactus-needle\[whistle\]"):+ with pytest.raises(RuntimeError, match=r"cactus-needle\[mic\]"):record()def test_rate_counts_steps_after_the_first_mark():- from needle.whistle.compare import rate+ from needle.agent.whistle import rateassert rate([1.0, 1.5, 2.0]) == 2.0assert rate([1.0]) == 0.0 and rate([]) == 0.0 and rate([2.0, 2.0]) == 0.0def test_compare_prints_one_line_per_model_with_dashes_for_missing_timing(capsys):- from needle.whistle.compare import compare+ from needle.agent.whistle import run_modelsmodels = [("whistle", 17e6, lambda audio: ("hello there", 0.013, 1264.0)),("moonshine tiny v2", 45e6, lambda audio: ("", None, None))]- compare(models, [0.0] * 16000)+ run_models(models, [0.0] * 16000)lines = capsys.readouterr().out.splitlines()assert len(lines) == 2assert lines[0].startswith(" whistle 17 MB ttft 13 ms decode 1264 tok/s total")@@ -248,21 +266,21 @@ def test_compare_prints_one_line_per_model_with_dashes_for_missing_timing(capsysdef test_playground_and_compare_share_one_status_line():- from needle.whistle.playground import status+ from needle.agent.whistle import statusassert status("whistle", 17e6, 0.0137, 1264.4, 0.048) == " whistle 17 MB ttft 14 ms decode 1264 tok/s total 48 ms"assert status("moonshine tiny v2", 145e6, None, None, 0.21) == " moonshine tiny v2 145 MB ttft - decode - total 210 ms"def test_status_reads_a_slow_run_in_seconds():- from needle.whistle.playground import status+ from needle.agent.whistle import statusassert status("whistle", 17e6, 0.0137, 1264.4, 12.5).endswith("total 12.5 s")assert status("whistle", 17e6, 0.0137, 1264.4, 9.9).endswith("total 9900 ms")def test_audio_path_takes_a_path_as_a_terminal_hands_it_over():- from needle.whistle.playground import audio_path+ from needle.agent.whistle import audio_pathassert audio_path(" clip.wav ") == "clip.wav"assert audio_path('"my clips/a b.wav"') == "my clips/a b.wav"@@ -271,11 +289,11 @@ def test_audio_path_takes_a_path_as_a_terminal_hands_it_over():def test_playground_prints_text_words_and_timing(capsys):- from needle.whistle.playground import transcribe+ from needle.agent.whistle import show_transcriptwhistle = _Whistle({"text": "hello there", "language": "en", "ttft_ms": 12.6, "decode_tps": 900.4,"words": [{"word": "hello", "start": 0.1, "end": 0.4, "probability": 0.98}]})- transcribe(whistle, "clip.wav", {"language": "en", "keywords": ["Siobhan"], "timestamps": True})+ show_transcript(whistle, "clip.wav", {"language": "en", "keywords": ["Siobhan"], "timestamps": True})out = capsys.readouterr().out.splitlines()assert whistle.calls == [("clip.wav", {"language": "en", "keywords": ["Siobhan"], "word_timestamps": True})]assert out[0] == "hello there"@@ -284,62 +302,61 @@ def test_playground_prints_text_words_and_timing(capsys):def test_playground_reports_engine_errors_instead_of_raising(capsys):- from needle.whistle.playground import transcribe+ from needle.agent.whistle import show_transcriptclass Broken:def transcribe(self, audio, **options):raise RuntimeError("audio limit is 30 s")- transcribe(Broken(), "long.wav", {"language": None, "keywords": [], "timestamps": False})+ show_transcript(Broken(), "long.wav", {"language": None, "keywords": [], "timestamps": False})assert capsys.readouterr().out == " audio limit is 30 s\n"def test_playground_keeps_the_language_when_the_code_is_not_one_of_ours(monkeypatch, capsys):- from needle.whistle import playground+ from needle.agent import whistlelines = iter(["/language de", "/language EN", "/language", "/quit"])- monkeypatch.setattr(playground, "prompt", lambda: next(lines))- monkeypatch.setattr("needle.whistle.Whistle", lambda weights=None: _Whistle({}))- playground.main(type("Args", (), {"audio": None, "language": None, "keywords": "",- "word_timestamps": False, "weights": None})())+ monkeypatch.setattr(whistle, "prompt", lambda: next(lines))+ monkeypatch.setattr("needle.agent.whistle.Whistle", lambda weights=None: _Whistle({}))+ whistle.playground(type("Args", (), {"audio": None, "language": None, "keywords": "",+ "word_timestamps": False, "weights": None})())out = capsys.readouterr().outassert "language de" in out and "language is one of en de fr es it nl pl" in out and "language detected" in outdef test_compare_runs_a_file_the_same_way_the_playground_does(monkeypatch, capsys):pytest.importorskip("numpy")- from needle.whistle import compare as whistle_compare+ from needle.agent import whistleseen, read = [], []- monkeypatch.setattr(whistle_compare, "compare", lambda models, audio: seen.append(len(audio)))- monkeypatch.setattr("needle.whistle._read_wav", lambda path: read.append(path) or [0.0] * 16000)+ monkeypatch.setattr(whistle, "run_models", lambda models, audio: seen.append(len(audio)))+ monkeypatch.setattr("needle.agent.whistle._read_wav", lambda path: read.append(path) or [0.0] * 16000)lines = iter(['/file "my clips/a b.wav"', "/quit"])- monkeypatch.setattr(whistle_compare, "prompt", lambda: next(lines))- monkeypatch.setattr(whistle_compare, "load_whistle", lambda weights: (17e6, lambda audio: ("", None, None)))- monkeypatch.setattr(whistle_compare, "load_whisper", lambda size: (76e6, lambda audio: ("", None, None)))- monkeypatch.setattr(whistle_compare, "load_moonshine", lambda: (45e6, lambda audio: ("", None, None)))- whistle_compare.main(type("Args", (), {"audio": None, "weights": None})())+ monkeypatch.setattr(whistle, "prompt", lambda: next(lines))+ monkeypatch.setattr(whistle, "load_whistle", lambda weights: (17e6, lambda audio: ("", None, None)))+ monkeypatch.setattr(whistle, "load_whisper", lambda size: (76e6, lambda audio: ("", None, None)))+ monkeypatch.setattr(whistle, "load_moonshine", lambda: (45e6, lambda audio: ("", None, None)))+ whistle.compare(type("Args", (), {"audio": None, "weights": None})())assert read == ["my clips/a b.wav"] and seen == [16000]def test_compare_needs_its_extra(monkeypatch):- from needle.whistle import compare as whistle_compare+ from needle.agent import whistlemonkeypatch.setitem(sys.modules, "whisper", None)- with pytest.raises(SystemExit, match=r"cactus-needle\[whistle,whistle-compare\]"):- whistle_compare.main(type("Args", (), {"audio": None, "weights": None})())+ with pytest.raises(SystemExit, match=r"cactus-needle\[mic,compare\]"):+ whistle.compare(type("Args", (), {"audio": None, "weights": None})())def test_cli_routes_the_whistle_commands(monkeypatch):import needle._telemetryimport needle.cli- import needle.whistle.playground- import needle.whistle.compare+ import needle.agent.whistleseen = []monkeypatch.setattr(needle._telemetry, "track", lambda *a, **k: None)- monkeypatch.setattr(needle.whistle.playground, "main", lambda args: seen.append(("playground", args)))- monkeypatch.setattr(needle.whistle.compare, "main", lambda args: seen.append(("compare", args)))+ monkeypatch.setattr(needle.agent.whistle, "playground", lambda args: seen.append(("playground", args)))+ monkeypatch.setattr(needle.agent.whistle, "compare", lambda args: seen.append(("compare", args)))monkeypatch.setattr(sys, "argv", ["needle", "whistle", "playground", "clip.wav", "--language", "de", "--keywords", "Siobhan, Krzysztof", "--word-timestamps"])needle.cli.main()monkeypatch.setattr(sys, "argv", ["needle", "whistle", "compare", "--weights", "w.cact"])@@ -348,35 +365,5 @@ def test_cli_routes_the_whistle_commands(monkeypatch):assert (seen[0][1].audio, seen[0][1].language, seen[0][1].keywords, seen[0][1].word_timestamps) == ("clip.wav", "de", "Siobhan, Krzysztof", True)assert (seen[1][1].audio, seen[1][1].weights) == (None, "w.cact")monkeypatch.setattr(sys, "argv", ["needle", "whistle"])- with pytest.raises(SystemExit, match="needle whistle playground \\| compare \\| fetch \\| download"):+ with pytest.raises(SystemExit, match="needle whistle playground \\| compare"):needle.cli.main()---def test_needle3_whistle_resolves_inside_the_needle3_repo(tmp_path, monkeypatch):- """The combined engine shares needle3's repo, so every path it fetches is prefixed."""- import zipfile- import needle- from needle.agent import fetch-- audio = fetch.NEEDLE3_WHISTLE- assert fetch.engine_repo(audio) == fetch.engine_repo(3)- assert fetch.engine_version(audio) == "3.0.0"- assert fetch.lib_name(audio) == fetch.lib_name(3).replace("needle", audio)- assert fetch.engine_wheel("3.0.0", "win_arm64", audio).startswith(audio + "/python/")- assert fetch.engine_wheel("3.0.3", "win_arm64", 3).startswith("python/")-- monkeypatch.setattr(os.path, "expanduser", lambda path: str(tmp_path))- monkeypatch.setattr(needle, "__file__", str(tmp_path / "package" / "__init__.py"))- monkeypatch.setattr(fetch, "_register_download", lambda generation: None)- monkeypatch.setenv(fetch.lib_path_env(audio), "/opt/libneedle3_whistle.so")- assert needle._library_path(audio) == "/opt/libneedle3_whistle.so"- monkeypatch.delenv(fetch.lib_path_env(audio))-- wheel, fetched = tmp_path / "engine.whl", []- with zipfile.ZipFile(wheel, "w") as archive:- archive.writestr("needle/" + fetch.lib_name(audio), b"engine")- monkeypatch.setattr("huggingface_hub.hf_hub_download",- lambda **kwargs: fetched.append(kwargs["filename"]) or str(wheel))- resolved = needle._library_path(audio)- assert resolved == str(tmp_path / ".cache" / "cactus-needle" / audio / "3.0.0" / fetch.lib_name(audio))- assert fetched == [fetch.engine_wheel("3.0.0", fetch._platform_tag(), audio)]diff --git a/tests/test_worker.py b/tests/test_worker.pyindex c853e5c..32cb3dc 100644--- a/tests/test_worker.py+++ b/tests/test_worker.py@@ -45,8 +45,10 @@ int needle_init(const char* system, const char* tools, const char* index) {return 7;}-int needle_complete(const char* input, int max_new_tokens, char* output,- int capacity) {+int needle_complete(const char* input, const float* pcm, int samples,+ int max_new_tokens, char* output, int capacity) {+ (void)pcm;+ (void)samples;(void)max_new_tokens;snprintf(output, (size_t)capacity,"{\"type\":\"text\",\"model\":%d,\"resets\":%d,"