Skip to content

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

Add Needle 2 Playground with server and UI components

HenryNdubuaku6ca91869a0ce
diff --git a/MANIFEST.in b/MANIFEST.in
index 8f5eacc..f63b52e 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,3 +1,3 @@
-include README.md LICENSE
+include README.md LICENSE llms.txt
recursive-include needle/model *.model *.vocab
-include needle/agent/index.html
+include needle/playground/index.html needle/playground/app.js needle/playground/style.css
diff --git a/README.md b/README.md
index 937380d..580abb9 100644
--- a/README.md
+++ b/README.md
@@ -2,10 +2,11 @@
# Needle 2
-Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges 300-700 on sub-$200 phones such as the Samsung A-Series. With a peak session RAM around 28MB, Needle reaches microcontrollers like the ESP32-P4; others have reported running it on an ESP32-S3 in about 11MB.
+Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in about 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16.
-- **Self-contained**: model baked into the binary, no runtime, no downloads, no network.
-- **Runs everywhere**: ARM64, x86-64, ARMv7, RISC-V, and WebAssembly, on Apple, Windows, Linux, Android, Raspberry Pi.
+This repository is the Python package: inference, LoRA fine-tuning, and export. `pip install cactus-needle`, describe your tools, and call them from Python. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build.
+
+- **Self-contained**: weights baked into a single 14MB engine; no separate model files to manage, and inference does no network.
- **Simple contract**: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token.
- **Confidence-gated**: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it.
- **Tool retrieval**: declare a large catalogue and a built-in retrieval head renders only the top five tools per turn, with the grammar constrained to that subset.
@@ -23,7 +24,7 @@ Needle 2 is a Simple Attention Network, our dense small-model recipe: a Hadamard
Each block carries its update rule. Here x̂ is the RMS-normalised flattening of the four residual streams, H the orthonormal Walsh-Hadamard transform (a fixed matrix, applied in n log n time with no weights to read), (kₜ, vₜ) rows gathered from hashed n-gram tables, and P the doubly-stochastic normalisation of the routing logits A, computed by Sinkhorn iteration; a, b, g and all σ-gates are learned and input-dependent. Both attention and MLP residuals are sandwich-normed and gated, the engram sites fire at two layers, and decoding is constrained by a byte-level grammar compiled from the declared schemas.
-## Quickstart with Python
+## Quickstart
```sh
pip install cactus-needle
@@ -95,7 +96,7 @@ invoice = needle.extract("Invoice from Acme Corp, $1,200.00, due 2026-09-01", In
print(invoice.vendor, invoice.total) # -> Acme Corp 1200.0
```
-**By hand** - the decorator just builds a JSON schema; you can pass that schema directly, which is exactly what Needle consumes. This is how you set descriptions and constraints without the decorator (and `tools.json` for the CLI is the same shape):
+**By hand** - the decorator just builds a JSON schema; you can pass that schema directly, which is exactly what Needle consumes. This is how you set descriptions and constraints without the decorator:
```python
tools = [{
@@ -136,11 +137,21 @@ With a large catalogue, persist tool embeddings across runs with `needle.Needle(
"reasoning": "'living room' -> room; 'dim' -> on true, brightness 30",
"confidence": 0.94,
"prefill_tps": 4300.0,
- "decode_tps": 850.0,
- "peak_ram_mb": 28.0
+ "decode_tps": 850.0
}
```
+## Playground
+
+Try any model in the browser: pick a preset, edit the tools or prompt, and Run. Follow-up queries continue the same conversation.
+
+```sh
+needle playground # base model, http://127.0.0.1:7860
+needle playground --weights my.cact # a tuned model
+```
+
+The server downloads and initializes the model before serving, so the first query is instant. The **Finetune on these tools** button runs the fine-tuning pipeline below from the UI and hands back a downloadable `.cact`.
+
## Behaviour
Needle solves every problem as a function call. The context declares what may be called; the model answers with calls. Performing an action and extracting structured data are the same operation, the only difference is what you declare.
@@ -151,91 +162,45 @@ Needle solves every problem as a function call. The context declares what may be
- After you execute a call, pass the result back as the next `complete()`. The model continues from it, and later arguments may depend on earlier results: `search_for_contact` first, then `send_instant_message` with the returned `contact_id`. A final step may answer in plain text from the results: `"type": "respond"` with empty `function_calls`.
- A session shares one toolset. Later turns are bare queries against the same tools; `reset()` rewinds the conversation and keeps the tools loaded.
-## System facts
+## Extraction
-An optional system turn carries environment state as facts, never instructions:
+Extraction is not a separate mode - it is tool calling with one tool. Declare the record as the only schema and pass the content where the query goes; the returned call's `arguments` are the extracted fields. With one declared tool the grammar admits exactly one call of that name, so schema conformance is guaranteed rather than requested. Use the `extract()` helper for a typed result (shown in Quickstart), or pass a plain schema and read the call:
+```python
+receipt = [{
+ "name": "receipt",
+ "description": "A purchase receipt shared as text",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "merchant": {"type": "string"},
+ "total": {"type": "number"},
+ "currency": {"type": "string"},
+ "line_items": {"type": "array", "items": {"type": "object"}},
+ },
+ "required": ["merchant", "total"],
+ },
+}]
+agent = needle.Needle(tools=receipt)
+print(agent.complete("GreenMart receipt: oat milk 3.50, total 7.75 paid by visa")["function_calls"])
+# -> [{"name": "receipt", "arguments": {"merchant": "GreenMart", "total": 7.75}}]
```
-date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%
-```
-
-Recognized keys are `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, and `assistant`. The model resolves relative language against them: "tomorrow at 7" becomes an absolute time only when a `date:` fact licenses it, otherwise the human phrase passes through verbatim. `assistant:` declares the identity the model binds to. Pass the turn with `--system system.txt` on the CLI or `needle.Needle(tools=tools, system="date: ...")` in Python. Needle trains with and without the turn, so omitting it is safe; instructions placed there do not steer the model.
-
-## Deploy Needle
-
-Download the folder for your platform from the release:
-
-| your device | folder | command-line | library |
-| --- | --- | --- | --- |
-| Mac (Apple Silicon) | `macos-arm64` | `needle` | `libneedle.a` |
-| Linux x86-64 (PC, server, AMD) | `linux-x86_64` | `needle` | `libneedle.a` |
-| Linux ARM64 (Raspberry Pi, server) | `linux-arm64` | `needle` | `libneedle.a` |
-| Linux ARMv7 (32-bit) | `linux-armv7` | `needle` | `libneedle.a` |
-| Linux RISC-V | `linux-riscv64` | `needle` | `libneedle.a` |
-| Linux MIPS32el (Ingenic cameras, routers) | `linux-mipsel` | `needle` | `libneedle.a` |
-| Windows x64 | `windows-x86_64` | `needle.exe` | `libneedle.a` |
-| Windows ARM | `windows-arm64` | `needle.exe` | `libneedle.a` |
-| Android | `android-arm64` / `android-armv7` / `android-riscv64` | `needle` | `libneedle.a` |
-| iOS / watchOS / tvOS | `ios-arm64` / `watchos-arm64` / `tvos-arm64` | - | `libneedle.a` |
-| Browser / Node (WebAssembly) | `wasm` | - | `needle.js` + `needle.wasm` |
-To run it, use the command-line binary. On macOS, Linux, or Android:
+Because it is the same operation, everything else applies unchanged: `confidence` gates the extraction, unsupported input returns the empty call `[]`, and fine-tuning uses the same data format (the record as the tool, the passage as the query).
-```sh
-# answer one query and exit
-./needle --tools tools.json --prompt "dim the living room to 30"
+## System facts
-# or an HTTP server on localhost:8080 (POST /complete {"input": "..."})
-./needle --tools tools.json --serve
+An optional system turn carries environment state as facts, never instructions:
-# with a large tool catalogue, persist tool embeddings across runs
-./needle --tools tools.json --tool-index tools.idx --serve
+```python
+agent = needle.Needle(tools=tools, system="date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%")
```
-`tools.json` is a JSON array of the functions the assistant may call:
-
-```json
-[
- {
- "name": "set_lights",
- "description": "Turn a room's lights on or off and set brightness",
- "parameters": {
- "type": "object",
- "properties": {
- "room": { "type": "string" },
- "on": { "type": "boolean" },
- "brightness": { "type": "integer", "description": "0 to 100" }
- },
- "required": ["room", "on"]
- }
- },
- {
- "name": "play_music",
- "description": "Play music matching a mood, genre, or artist",
- "parameters": {
- "type": "object",
- "properties": { "query": { "type": "string" } },
- "required": ["query"]
- }
- },
- {
- "name": "send_message",
- "description": "Text a contact",
- "parameters": {
- "type": "object",
- "properties": {
- "to": { "type": "string" },
- "body": { "type": "string" }
- },
- "required": ["to", "body"]
- }
- }
-]
-```
+Recognized keys are `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, and `assistant`. The model resolves relative language against them: "tomorrow at 7" becomes an absolute time only when a `date:` fact licenses it, otherwise the human phrase passes through verbatim. `assistant:` declares the identity the model binds to. Needle trains with and without the turn, so omitting it is safe; instructions placed there do not steer the model.
## Tool retrieval
-Five or fewer declared tools render directly. Above that, retrieval engages: at init every tool schema is embedded once by a built-in contrastive head, each turn embeds the query, and only the five highest-scoring tools enter the context, with the grammar rebuilt over just that subset. An unselected tool is unreachable, not merely unlikely. `--tool-index <path>` (CLI) or `tool_index_path` (Python) persists the embeddings on disk, keyed by a fingerprint over the schemas and the model; a matching fingerprint loads instantly, a changed schema re-embeds only what changed.
+Five or fewer declared tools render directly. Above that, retrieval engages: at init every tool schema is embedded once by a built-in contrastive head, each turn embeds the query, and only the five highest-scoring tools enter the context, with the grammar rebuilt over just that subset. An unselected tool is unreachable, not merely unlikely. `tool_index_path` persists the embeddings on disk, keyed by a fingerprint over the schemas and the model; a matching fingerprint loads instantly, a changed schema re-embeds only what changed.
## Confidence
@@ -243,60 +208,45 @@ The `confidence` field is the minimum of two signals: a calibrated post-hoc head
## Fine-tuning
-Needle fine-tunes with LoRA on the frozen base and merges the adapter at export, so a run is cheap and the adapter is tiny. Your data is a JSONL of tool-call examples, one per line: `{"query": ..., "tools": [...], "answers": [{"name": ..., "arguments": {...}}], "reasoning": "..."}`.
+Needle fine-tunes with LoRA on the frozen base and merges the adapter at export, so a run is cheap and the tuned model is still a single `.cact` that runs on the same engine. The workflow is: (optionally) synthesize data, LoRA fine-tune, then build a tuned `.cact`.
-```sh
-# optionally synthesize training data first (needs OPENROUTER_API_KEY)
-needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl
+**Data format.** A JSONL file, one example per line. `reasoning` is optional; an off-topic example has `answers: []`.
-# LoRA fine-tune; --generate N expands your set with more examples before training
-needle finetune data.jsonl --epochs 3 --generate 300
-
-# merge the adapter into the base and export a tuned .cact
-needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my_needle.cact
+```json
+{"query": "dim the kitchen to 10", "tools": [{"name": "set_lights", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "brightness": {"type": "integer"}}, "required": ["room"]}}], "answers": [{"name": "set_lights", "arguments": {"room": "kitchen", "brightness": 10}}], "reasoning": "'kitchen' -> room; 'dim to 10' -> brightness 10"}
```
-The engine binary is downloaded from Hugging Face and is weights-agnostic, so a tuned `.cact` runs on it directly - no recompilation:
+**1. Synthesize data (optional).** Needs `OPENROUTER_API_KEY`. Seed from a tool schema file, or expand an existing set:
-```python
-import needle
-agent = needle.Needle(weights="my_needle.cact", tools=[...])
-agent.run("...")
+```sh
+export OPENROUTER_API_KEY=sk-or-...
+needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl
+needle generate-data --augment data.jsonl --num-samples 500 # expand an existing JSONL
```
-Set `NEEDLE_HF_REPO=<you>/<model>` and pass `--upload` to `needle build` to publish your tuned `.cact`. Try any model in the browser playground with `needle playground` (http://127.0.0.1:7860).
-
-## Extraction
+**2. LoRA fine-tune.** The base checkpoint auto-downloads from Hugging Face if you do not pass `--checkpoint`. `--generate N` first synthesizes N more examples from the tools in your data (also needs `OPENROUTER_API_KEY`).
-Extraction is the same exchange as tool calling: declare the record schema as the only tool and pass the content as the prompt; the passage sits where the query sits, and the returned call's `arguments` are the extracted fields. With one declared tool the grammar admits exactly one call of that name, the `tool_choice` equivalent, so schema conformance is guaranteed rather than requested. There is no separate JSON mode.
+```sh
+needle finetune data.jsonl --epochs 3
+needle finetune data.jsonl --epochs 3 --generate 300 --lora-rank 16 --lora-alpha 32
+```
-`schema.json` describes the record to extract:
+Key options: `--lora-rank` (default 16), `--lora-alpha` (32), `--lr` (1e-4), `--batch-size` (16), `--max-len` (1024), `--checkpoint <base.pkl>`, `--out <adapter.pkl>`. The adapter is written to `checkpoints/needle_lora.pkl`.
-```json
-[
- {
- "name": "receipt",
- "description": "A purchase receipt shared as text",
- "parameters": {
- "type": "object",
- "properties": {
- "merchant": { "type": "string" },
- "total": { "type": "number" },
- "currency": { "type": "string" },
- "line_items": { "type": "array", "items": { "type": "object" } }
- },
- "required": ["merchant", "total"]
- }
- }
-]
-```
+**3. Build a tuned `.cact`.** Merge the adapter into the base and quantize. The base auto-downloads if absent.
```sh
-./needle --tools schema.json --prompt "GreenMart receipt: oat milk 3.50, total 7.75 paid by visa"
+needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my_needle.cact
```
-```json
-{ "type": "call", "function_calls": [ { "name": "receipt", "arguments": { "merchant": "GreenMart", "total": 7.75 } } ] }
+Add `--bits 2` (default 4) for a smaller model, or set `NEEDLE_HF_REPO=<you>/<model>` and pass `--upload` to publish the `.cact`.
+
+**4. Run it.** The engine is weights-agnostic, so a tuned `.cact` runs on it directly - no recompilation:
+
+```python
+import needle
+agent = needle.Needle(weights="my_needle.cact", tools=[...])
+agent.run("...")
```
## Citation
@@ -314,4 +264,4 @@ Needle 2 is built by the Cactus Compute team. If you use it in your work, please
}
```
-Reach out on [email protected] for partnerships, collaborations, synergies and deploying Needle2 in your product.
\ No newline at end of file
+Reach out on [email protected] for partnerships, collaborations, synergies and deploying Needle2 in your product.
diff --git a/llms.txt b/llms.txt
new file mode 100644
index 0000000..77a68fd
--- /dev/null
+++ b/llms.txt
@@ -0,0 +1,200 @@
+# Needle 2 (cactus-needle)
+
+> Needle 2 is an open 45M-parameter on-device model for tool calling, device control, and structured extraction. This Python package (`cactus-needle`) is inference + LoRA fine-tuning + export. Text goes in, a JSON tool call comes back; a byte-level grammar compiled from your schemas constrains every token, so the call is always well-formed. The inference engine is a 14MB binary fetched once from Hugging Face and cached; a full session runs in ~28MB RAM.
+
+This file is written for AI coding assistants. It is enough to write correct Needle code without reading the source. Copy the patterns; do not invent API that is not listed here.
+
+## Install
+
+```sh
+pip install cactus-needle
+```
+
+`import needle` is lightweight (no JAX). JAX is imported lazily and only by fine-tuning/export/build. The engine binary auto-downloads from Hugging Face on first `Needle(...)` use.
+
+## Core API
+
+- `needle.Needle(tools=None, system=None, weights=None, tool_index_path=None, buffer_size=65536)` - create an agent bound to one toolset.
+ - `tools`: list of decorated functions, Pydantic models, raw JSON-schema dicts, or a JSON string.
+ - `system`: optional environment-facts string (see System facts).
+ - `weights`: path to a tuned `.cact` to load instead of the baked base model.
+ - `tool_index_path`: path to persist tool embeddings when you declare many tools.
+- `agent.run(query, max_steps=8, max_new_tokens=256) -> dict` - full agentic loop: model picks calls, Needle executes your Python functions, feeds results back, returns the final response.
+- `agent.complete(text, max_new_tokens=256) -> dict` - one turn; you execute the call and feed the result back yourself via the next `complete(...)`.
+- `agent.reset()` - rewind the conversation, keep the tools loaded.
+- `needle.tool` - decorator that turns a function into a tool schema (attached as `fn._needle_tool`).
+- `needle.Field(default=..., *, description, enum, const, ge, le, gt, lt, multiple_of, min_length, max_length, pattern, format, min_items, max_items, unique_items)` - per-argument constraints; attach inline with `typing.Annotated`.
+- `needle.extract(text, schema, system=None, max_new_tokens=256)` - one-shot extraction; returns a Pydantic instance if `schema` is a model, else a dict (or `None` if nothing matched).
+
+## Defining tools (three equivalent ways)
+
+Decorator - signature gives types, docstring is the description, Google-style `Args:` gives per-argument docs, a default makes an argument optional, `Literal[...]` becomes a fixed choice set:
+
+```python
+import needle
+from typing import Literal, Annotated
+
+def set_thermostat(temperature: int, mode: Literal["heat", "cool", "auto"] = "auto"):
+ """Set the thermostat.
+
+ Args:
+ temperature: target temperature in Celsius
+ """
+ return {"temperature": temperature, "mode": mode}
+
+def send_money(amount: Annotated[float, needle.Field(gt=0, le=10000)], to: str):
+ "Send money to a handle."
+ return {"sent": amount, "to": to}
+
+agent = needle.Needle(tools=[set_thermostat, send_money])
+```
+
+Raw JSON schema (what the engine actually consumes):
+
+```python
+tools = [{
+ "name": "set_lights",
+ "description": "Turn a room's lights on/off and set brightness",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "room": {"type": "string"},
+ "on": {"type": "boolean"},
+ "brightness": {"type": "integer", "minimum": 0, "maximum": 100},
+ },
+ "required": ["room", "on"],
+ },
+}]
+agent = needle.Needle(tools=tools)
+```
+
+Pydantic model (mainly for extraction):
+
+```python
+from pydantic import BaseModel
+class Invoice(BaseModel):
+ vendor: str
+ total: float
+invoice = needle.extract("Invoice from Acme Corp, $1,200.00", Invoice) # -> Invoice(...)
+```
+
+## Response shape
+
+Every turn returns one dict:
+
+```json
+{
+ "type": "call",
+ "success": true,
+ "error": null,
+ "error_code": null,
+ "function_calls": [{"name": "set_lights", "arguments": {"room": "living room", "on": true}}],
+ "reasoning": "'living room' -> room; 'dim' -> on true",
+ "confidence": 0.94,
+ "prefill_tps": 4300.0,
+ "decode_tps": 850.0
+}
+```
+
+- `type` is `"call"` when there are tool calls, `"respond"` for a final plain-text answer from results, `"refuse"`/empty `function_calls` for off-topic input.
+- `function_calls` is a list of `{"name", "arguments"}`. Read `arguments` directly; it is grammar-guaranteed to match your schema.
+- `reasoning` is a short unconstrained derivation of each argument from its source span.
+- `confidence` is a calibrated score in [0,1] (see Confidence gating).
+
+## Behaviour contract (important for correct code)
+
+- Off-topic / unsupported request -> empty `function_calls` (a refusal). There is no free-text fallback. Always handle the empty case.
+- Arguments contain only values evidenced in the input. Optional fields with no evidence are omitted, not guessed. Do not assume a key exists.
+- Multi-turn: repeated `complete(...)` on the same agent continue one conversation; later arguments can depend on earlier tool results. Feed each result back as the next `complete(json.dumps(result))`.
+- One toolset per session. To change tools, make a new `Needle(...)`. `reset()` clears history but keeps the tools.
+
+Driving the loop manually:
+
+```python
+import json
+r = agent.complete("dim the living room to 30")
+if r["type"] == "call":
+ out = set_lights(**r["function_calls"][0]["arguments"])
+ r = agent.complete(json.dumps(out)) # feed result back
+```
+
+## Confidence gating
+
+`confidence` is the min of a post-hoc calibration head and the decode probability of the call. Pick a threshold per product; act at/above it, escalate below it:
+
+```python
+r = agent.complete(user_text)
+calls = r.get("function_calls") or []
+if calls and r["confidence"] >= 0.8:
+ execute(calls[0])
+else:
+ escalate_or_reask()
+```
+
+## System facts (optional)
+
+Pass environment state as facts, never instructions. Recognized keys: `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, `assistant`.
+
+```python
+agent = needle.Needle(tools=tools, system="date: 2026-07-21 Tue 14:30; locale: en-US; device: phone")
+```
+
+Relative language ("tomorrow at 7") resolves only when a `date:` fact licenses it. Omitting the system turn is safe.
+
+## Tool retrieval (many tools)
+
+With more than 5 declared tools, a built-in retrieval head renders only the top-5 per turn and constrains the grammar to that subset. Persist embeddings across runs:
+
+```python
+agent = needle.Needle(tools=big_catalogue, tool_index_path="tools.idx")
+```
+
+## Fine-tuning (CLI)
+
+LoRA on the frozen base, merged at export. Data is JSONL, one example per line; `reasoning` optional, off-topic example has `answers: []`:
+
+```json
+{"query": "dim the kitchen to 10", "tools": [{"name": "set_lights", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "brightness": {"type": "integer"}}, "required": ["room"]}}], "answers": [{"name": "set_lights", "arguments": {"room": "kitchen", "brightness": 10}}]}
+```
+
+```sh
+export OPENROUTER_API_KEY=sk-or-...
+needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl # optional
+needle finetune data.jsonl --epochs 3 --generate 300 # LoRA; auto-downloads base
+needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my.cact
+```
+
+Run a tuned model - the engine is weights-agnostic:
+
+```python
+agent = needle.Needle(weights="my.cact", tools=[...])
+agent.run("...")
+```
+
+Options: `finetune --lora-rank 16 --lora-alpha 32 --lr 1e-4 --batch-size 16 --max-len 1024 --checkpoint <base.pkl> --out <adapter.pkl>`; `build --bits 2|4 --upload` (with `NEEDLE_HF_REPO=<you>/<model>`).
+
+## Playground (browser)
+
+```sh
+needle playground # base model at http://127.0.0.1:7860
+needle playground --weights my.cact # a tuned model
+```
+
+Preset demos, editable tools/prompt, multi-turn follow-ups, and a "Finetune on these tools" button that runs the pipeline above and returns a downloadable `.cact`.
+
+## CLI summary
+
+- `needle run --checkpoint <base.pkl> --query "..." --tools tools.json` - JAX reference inference from a checkpoint (dev path; normal inference is the Python `Needle` API above).
+- `needle generate-data` / `needle finetune` / `needle build` - the fine-tuning pipeline.
+- `needle playground` - browser UI.
+
+## Common mistakes to avoid
+
+- Do not expect free-text answers to arbitrary questions; unsupported input returns an empty call. Handle it.
+- Do not read `arguments` keys that were not evidenced in the input; optional fields may be absent.
+- Do not create one `Needle` per turn for a conversation; reuse the instance so context carries. New tools = new instance.
+- `import needle` does not import JAX; only `needle finetune`/`build` do.
+- `weights=` expects a `.cact` (from `needle build`), not a `.pkl` checkpoint.
+```
diff --git a/needle/agent/index.html b/needle/agent/index.html
deleted file mode 100644
index 7affee3..0000000
--- a/needle/agent/index.html
+++ /dev/null
@@ -1,94 +0,0 @@
-<!doctype html>
-<html lang="en">
-<head>
-<meta charset="utf-8">
-<meta name="viewport" content="width=device-width, initial-scale=1">
-<title>Needle 2 playground</title>
-<style>
- :root { --bg:#0d0d0f; --panel:#161619; --line:#26262b; --fg:#e7e7ea; --dim:#8a8a92; --red:#ff5a36; }
- * { box-sizing: border-box; }
- body { margin:0; background:var(--bg); color:var(--fg); font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace; }
- header { padding:20px 24px; border-bottom:1px solid var(--line); }
- header b { color:var(--red); }
- .wrap { display:grid; grid-template-columns:1fr 1fr; gap:0; height:calc(100vh - 62px); }
- .col { padding:20px 24px; overflow:auto; }
- .col.left { border-right:1px solid var(--line); }
- .lbl { color:var(--dim); text-transform:uppercase; font-size:11px; letter-spacing:.08em; margin:0 0 8px; }
- textarea, input { width:100%; background:var(--panel); color:var(--fg); border:1px solid var(--line);
- border-radius:8px; padding:12px; font:inherit; resize:vertical; }
- textarea { height:340px; }
- #query { margin:14px 0; }
- button { background:var(--panel); color:var(--fg); border:1px solid var(--line); border-radius:999px;
- padding:6px 12px; margin:0 6px 8px 0; cursor:pointer; font:inherit; }
- button:hover { border-color:var(--red); }
- #run { background:var(--red); border-color:var(--red); color:#fff; padding:8px 22px; border-radius:8px; }
- pre { background:var(--panel); border:1px solid var(--line); border-radius:8px; padding:14px; white-space:pre-wrap;
- word-break:break-word; margin:0; }
- .meta { color:var(--dim); margin-top:10px; font-size:12px; }
-</style>
-</head>
-<body>
-<header>Play with <b>Needle 2</b> right here. Pick a preset, edit the tools or prompt, and Run.</header>
-<div class="wrap">
- <div class="col left">
- <p class="lbl">Tools JSON</p>
- <textarea id="tools"></textarea>
- <div id="presets" style="margin-top:14px"></div>
- </div>
- <div class="col">
- <p class="lbl">Query</p>
- <input id="query" placeholder="what do you want Needle to do?">
- <button id="run">Run</button>
- <p class="lbl" style="margin-top:22px">Result</p>
- <pre id="out">-</pre>
- <div class="meta" id="meta"></div>
- </div>
-</div>
-<script>
-var PRESETS = {"home": {"tools": [{"name": "set_lights", "description": "Turn lights on or off or dim them.", "parameters": {"type": "object", "properties": {"room": {"type": "string", "description": "Which room."}, "state": {"type": "string", "enum": ["on", "off"]}, "brightness": {"type": "integer", "description": "Percent 1-100."}}, "required": ["room", "state"]}}, {"name": "set_thermostat", "description": "Set the target temperature.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "temperature": {"type": "number"}, "mode": {"type": "string", "enum": ["heat", "cool", "auto"]}}, "required": ["room", "temperature"]}}, {"name": "lock_door", "description": "Lock a door.", "parameters": {"type": "object", "properties": {"door": {"type": "string", "description": "Which door."}}, "required": ["door"]}}], "q": "dim the bedroom lights to 20 percent and lock the front door"}, "robot": {"tools": [{"name": "move", "description": "Drive the robot in a direction.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["forward", "backward", "left", "right"]}, "distance_m": {"type": "number", "description": "Distance in meters."}}, "required": ["direction", "distance_m"]}}, {"name": "rotate", "description": "Rotate the robot in place.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["left", "right"]}, "degrees": {"type": "number"}}, "required": ["direction", "degrees"]}}, {"name": "gripper", "description": "Open or close the gripper.", "parameters": {"type": "object", "properties": {"action": {"type": "string", "enum": ["open", "close"]}}, "required": ["action"]}}], "q": "move forward 2 meters, turn left 90 degrees, then close the gripper"}, "device": {"tools": [{"name": "open_website", "description": "Open a website in a new tab.", "parameters": {"type": "object", "properties": {"url": {"type": "string", "description": "The site to open."}}, "required": ["url"]}}, {"name": "set_theme", "description": "Change the page accent color.", "parameters": {"type": "object", "properties": {"accent_color": {"type": "string", "description": "A color name or hex."}}, "required": ["accent_color"]}}, {"name": "speak", "description": "Say something out loud through the device speakers.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}], "q": "make the accent green, open github.com, and say hello"}, "gallery": {"tools": [{"name": "create_album", "description": "Create a new photo album.", "parameters": {"type": "object", "properties": {"name": {"type": "string"}}, "required": ["name"]}}, {"name": "move_photos", "description": "Move photos into an album.", "parameters": {"type": "object", "properties": {"album": {"type": "string"}, "filter": {"type": "string", "description": "Which photos, e.g. 'last weekend', 'screenshots'."}}, "required": ["album", "filter"]}}, {"name": "delete_photos", "description": "Delete photos.", "parameters": {"type": "object", "properties": {"filter": {"type": "string"}}, "required": ["filter"]}}], "q": "create an album called Summer 2026 and move the photos from last weekend into it"}, "email": {"tools": [{"name": "send_email", "description": "Send an email.", "parameters": {"type": "object", "properties": {"to": {"type": "string", "description": "Email address."}, "subject": {"type": "string"}, "body": {"type": "string"}}, "required": ["to", "subject", "body"]}}], "q": "send the receipt to [email protected] with subject expenses: Blue Bottle Coffee, $14.50, August 4th"}, "currency": {"tools": [{"name": "convert_currency", "description": "Convert an amount between currencies.", "parameters": {"type": "object", "properties": {"amount": {"type": "number"}, "from_currency": {"type": "string", "description": "ISO code, e.g. USD."}, "to_currency": {"type": "string", "description": "ISO code, e.g. EUR."}}, "required": ["amount", "from_currency", "to_currency"]}}], "q": "how much is 250 dollars in euros"}, "document": {"tools": [{"name": "record_booking", "description": "Record the details of a hotel booking.", "parameters": {"type": "object", "properties": {"hotel": {"type": "string"}, "confirmation_number": {"type": "string"}, "guest_name": {"type": "string"}, "room_type": {"type": "string"}, "check_in": {"type": "string"}, "check_out": {"type": "string"}, "total": {"type": "number"}, "currency": {"type": "string"}}, "required": ["confirmation_number", "check_in", "check_out", "total"]}}], "q": "extract the booking from this email: Dear Mr Okafor, thank you for choosing the Harbor Light Hotel. This email confirms reservation HL-88213 for a deluxe sea-view room, checking in on 14 September 2026 and checking out on 18 September 2026. The total for your stay is 642.50 euros, payable at checkout. Breakfast is included from 7am, and our shuttle meets the 9:40 ferry on request. We look forward to welcoming you. Warm regards, Reception."}, "sentiment": {"tools": [{"name": "classify_sentiment", "description": "Classify the sentiment of a message.", "parameters": {"type": "object", "properties": {"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral", "mixed"]}}, "required": ["sentiment"]}}], "q": "classify the sentiment of this message: this is the worst purchase I have ever made, it broke in a day and support ignored me"}, "crowded": {"tools": [{"name": "set_lights", "description": "Control lights.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "state": {"type": "string", "enum": ["on", "off"]}}, "required": ["room", "state"]}}, {"name": "set_thermostat", "description": "Set temperature.", "parameters": {"type": "object", "properties": {"temperature": {"type": "number"}}, "required": ["temperature"]}}, {"name": "lock_door", "description": "Lock a door.", "parameters": {"type": "object", "properties": {"door": {"type": "string"}}, "required": ["door"]}}, {"name": "play_music", "description": "Play music.", "parameters": {"type": "object", "properties": {"genre": {"type": "string"}}, "required": []}}, {"name": "set_timer", "description": "Set a timer.", "parameters": {"type": "object", "properties": {"time_human": {"type": "string"}}, "required": ["time_human"]}}, {"name": "send_message", "description": "Send a text.", "parameters": {"type": "object", "properties": {"recipient": {"type": "string"}, "message": {"type": "string"}}, "required": ["recipient", "message"]}}, {"name": "create_note", "description": "Create a note.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}, {"name": "start_vacuum", "description": "Start the robot vacuum.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}}, "required": []}}, {"name": "water_plants", "description": "Run the irrigation system.", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}, "required": []}}, {"name": "open_blinds", "description": "Open or close the blinds.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "position": {"type": "string", "enum": ["open", "closed", "half"]}}, "required": ["room", "position"]}}, {"name": "preheat_oven", "description": "Preheat the oven.", "parameters": {"type": "object", "properties": {"temperature_c": {"type": "number"}}, "required": ["temperature_c"]}}, {"name": "charge_car", "description": "Start charging the car.", "parameters": {"type": "object", "properties": {"limit_percent": {"type": "integer"}}, "required": []}}], "q": "close the bedroom blinds halfway"}, "repeat": {"tools": [{"name": "add_to_list", "description": "Add one item to the shopping list.", "parameters": {"type": "object", "properties": {"item": {"type": "string"}, "quantity": {"type": "integer"}}, "required": ["item"]}}], "q": "add 2 milk and 6 eggs to the shopping list"}, "array": {"tools": [{"name": "add_items", "description": "Add items to the shopping list.", "parameters": {"type": "object", "properties": {"items": {"type": "array", "items": {"type": "string"}}}, "required": ["items"]}}], "q": "add milk, eggs, and bread to the shopping list"}, "flight": {"tools": [{"name": "search_flights", "description": "Search for flights.", "parameters": {"type": "object", "properties": {"from_city": {"type": "string"}, "to_city": {"type": "string"}, "date": {"type": "string"}, "direct_only": {"type": "boolean", "description": "Only non-stop flights."}, "passengers": {"type": "integer"}}, "required": ["from_city", "to_city"]}}], "q": "find non-stop flights from lagos to nairobi on december 3rd for two passengers"}, "refuse": {"tools": [{"name": "move", "description": "Drive the robot in a direction.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["forward", "backward", "left", "right"]}, "distance_m": {"type": "number", "description": "Distance in meters."}}, "required": ["direction", "distance_m"]}}, {"name": "rotate", "description": "Rotate the robot in place.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["left", "right"]}, "degrees": {"type": "number"}}, "required": ["direction", "degrees"]}}, {"name": "gripper", "description": "Open or close the gripper.", "parameters": {"type": "object", "properties": {"action": {"type": "string", "enum": ["open", "close"]}}, "required": ["action"]}}], "q": "write me a short poem about the moon"}};
-var LABELS = {"home": "Smart home", "robot": "Robot", "gallery": "Gallery", "device": "Device control", "email": "Extract email", "currency": "Currency", "document": "Document extraction", "sentiment": "Sentiment", "crowded": "12 tools routing", "repeat": "Repeated calls", "array": "Array argument", "flight": "Flight form", "refuse": "Off-topic refusal"};
-var toolsBox = document.getElementById("tools");
-var queryBox = document.getElementById("query");
-var out = document.getElementById("out");
-var meta = document.getElementById("meta");
-
-function load(key) {
- var p = PRESETS[key];
- toolsBox.value = JSON.stringify(p.tools, null, 2);
- queryBox.value = p.q;
-}
-var box = document.getElementById("presets");
-Object.keys(LABELS).forEach(function (key) {
- if (!PRESETS[key]) return;
- var b = document.createElement("button");
- b.textContent = LABELS[key];
- b.onclick = function () { load(key); };
- box.appendChild(b);
-});
-
-document.getElementById("run").onclick = async function () {
- out.textContent = "running...";
- meta.textContent = "";
- try {
- var tools = JSON.parse(toolsBox.value);
- var res = await fetch("/complete", {
- method: "POST", headers: {"Content-Type": "application/json"},
- body: JSON.stringify({tools: tools, query: queryBox.value}),
- });
- var d = await res.json();
- out.textContent = JSON.stringify(d.function_calls || d, null, 2);
- if (d.confidence !== undefined) {
- meta.textContent = "confidence " + d.confidence.toFixed(4)
- + " \u00b7 " + (d.decode_tps || 0).toFixed(0) + " tok/s"
- + " \u00b7 " + (d.peak_ram_mb || 0).toFixed(1) + " MB"
- + (d.reasoning ? " \u2014 " + d.reasoning : "");
- }
- } catch (e) {
- out.textContent = "error: " + e;
- }
-};
-
-load("home");
-</script>
-</body>
-</html>
diff --git a/needle/agent/server.py b/needle/agent/server.py
deleted file mode 100644
index cb49986..0000000
--- a/needle/agent/server.py
+++ /dev/null
@@ -1,56 +0,0 @@
-import json
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-
-_INDEX = Path(__file__).parent / "index.html"
-_CACHE = {}
-
-
-def _agent(tools_json):
- from .. import Needle
- if tools_json not in _CACHE:
- _CACHE.clear()
- _CACHE[tools_json] = Needle(tools=tools_json)
- return _CACHE[tools_json]
-
-
-class _Handler(BaseHTTPRequestHandler):
- def _send(self, code, body, ctype="application/json"):
- data = body if isinstance(body, bytes) else body.encode("utf-8")
- self.send_response(code)
- self.send_header("Content-Type", ctype)
- self.send_header("Content-Length", str(len(data)))
- self.end_headers()
- self.wfile.write(data)
-
- def do_GET(self):
- if self.path.split("?")[0] in ("/", "/index.html"):
- self._send(200, _INDEX.read_bytes(), "text/html")
- else:
- self._send(404, b"not found", "text/plain")
-
- def do_POST(self):
- if self.path != "/complete":
- self._send(404, b"not found", "text/plain")
- return
- length = int(self.headers.get("Content-Length", 0))
- try:
- request = json.loads(self.rfile.read(length) or b"{}")
- tools = request.get("tools", [])
- tools_json = tools if isinstance(tools, str) else json.dumps(tools)
- result = _agent(tools_json).complete(request.get("query", ""))
- self._send(200, json.dumps(result))
- except Exception as exc:
- self._send(500, json.dumps({"error": str(exc)}))
-
- def log_message(self, *args):
- pass
-
-
-def main(args):
- server = ThreadingHTTPServer((args.host, args.port), _Handler)
- print(f"needle playground: http://{args.host}:{args.port}")
- try:
- server.serve_forever()
- except KeyboardInterrupt:
- server.shutdown()
diff --git a/needle/cli.py b/needle/cli.py
index 2ba520e..7b75c69 100644
--- a/needle/cli.py
+++ b/needle/cli.py
@@ -37,8 +37,6 @@ def _install_xla_log_filter():
return
_log_filter_installed = True
- # 1. Dup the original stderr fd; rebind Python's sys.stderr to it so
- # all Python-level writes bypass the pipe.
py_stderr_fd = os.dup(2)
try:
sys.stderr.flush()
@@ -126,14 +124,18 @@ def main():
help="Generate N extra examples via OpenRouter before training (0 = off)")
p.add_argument("--model", type=str, default="deepseek/deepseek-v4-flash",
help="OpenRouter model for --generate")
+ p.add_argument("--workers", type=int, default=8,
+ help="Concurrent OpenRouter requests when generating (default: 8)")
p.add_argument("--checkpoint-dir", type=str, default="checkpoints")
p.add_argument("--out", type=str, default=None, help="Output adapter path (.pkl)")
p = sub.add_parser("generate-data", add_help=False)
p.add_argument("--tools", type=str, default=None, help="Tool schemas JSON to seed generation")
p.add_argument("--augment", type=str, default=None, help="Existing JSONL to expand")
- p.add_argument("--num-samples", type=int, default=200)
+ p.add_argument("--num-samples", type=int, default=100)
p.add_argument("--batch-size", type=int, default=25)
+ p.add_argument("--workers", type=int, default=16,
+ help="Concurrent OpenRouter requests (default: 16)")
p.add_argument("--model", type=str, default="deepseek/deepseek-v4-flash")
p.add_argument("--output", type=str, default=None)
@@ -145,7 +147,8 @@ def main():
p.add_argument("--bits", type=str, default=None, choices=["2", "4"])
p = sub.add_parser("playground", add_help=False)
- p.add_argument("--checkpoint", type=str, default=None)
+ p.add_argument("--weights", type=str, default=None,
+ help="Tuned .cact to serve (defaults to the base model from HuggingFace)")
p.add_argument("--port", type=int, default=7860)
p.add_argument("--host", type=str, default="127.0.0.1")
@@ -168,5 +171,5 @@ def main():
from .model.finetune import build_main
build_main(args)
elif args.command == "playground":
- from .agent.server import main as ui_main
- ui_main(args)
+ from .playground.server import main as playground_main
+ playground_main(args)
diff --git a/needle/model/finetune.py b/needle/model/finetune.py
index 65d85ff..75e20b6 100644
--- a/needle/model/finetune.py
+++ b/needle/model/finetune.py
@@ -1,7 +1,9 @@
+import concurrent.futures
import json
import os
import pickle
import urllib.request
+from concurrent.futures import ThreadPoolExecutor
import numpy as np
@@ -18,24 +20,28 @@ OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
DEFAULT_MODEL = "deepseek/deepseek-v4-flash"
_GEN_SYSTEM = (
- "You generate training data for a tool-calling model. Given a set of tool "
- "schemas, produce realistic, diverse user requests and the exact tool calls "
- "that satisfy them. Return only JSON."
+ "You generate training data for a tool-calling and extraction model. Given a "
+ "set of tool or record schemas, produce realistic, diverse inputs paired with "
+ "the exact calls that satisfy them. Return only JSON."
)
-_GEN_TEMPLATE = """Tools available (JSON schemas):
+_GEN_TEMPLATE = """Schemas available (JSON):
{tools}
Produce {n} varied examples as a JSON array. Each element is an object:
- {{"query": "<a natural user request>",
- "reasoning": "<one short line deriving each argument from the query>",
- "answers": [{{"name": "<tool name>", "arguments": {{...}}}}]}}
+ {{"query": "<a natural user request to act on, or a passage of text to extract from>",
+ "reasoning": "<one short line deriving each argument from its source span in the query>",
+ "answers": [{{"name": "<schema name>", "arguments": {{...}}}}]}}
Rules:
-- Use only the tools above; arguments must match their schemas exactly.
-- Cover single-call, multi-call, and about {refusals} off-topic requests that no
- tool can serve (for those, "answers" is []).
-- Vary phrasing, values, and which tools are used. Return ONLY the JSON array."""
+- Use only the schemas above; arguments must match them exactly and contain only
+ values evidenced in the query.
+- For an action tool the query is a command. For a record/extraction schema (its
+ fields describe an entity), the query is a natural passage that contains those
+ fields and the call extracts them.
+- Cover single-call, multi-call, and about {refusals} off-topic inputs that no
+ schema can serve (for those, "answers" is []).
+- Vary phrasing, values, and which schemas are used. Return ONLY the JSON array."""
def _openrouter(messages, model, api_key, temperature=0.9):
@@ -76,6 +82,59 @@ def generate_examples(tools, n=25, model=DEFAULT_MODEL, api_key=None, refusals=3
return rows
+def _dedup_key(example):
+ answers = example.get("answers", example.get("function_calls", []))
+ return (example.get("query", "").strip().lower(), json.dumps(answers, sort_keys=True))
+
+
+def generate_dataset(tools, num_samples, model=DEFAULT_MODEL, batch_size=25,
+ api_key=None, workers=8, progress=None):
+ api_key = api_key or os.environ.get("OPENROUTER_API_KEY")
+ if not api_key:
+ raise RuntimeError("set OPENROUTER_API_KEY to generate data")
+
+ target = int(num_samples * 1.3)
+ max_submissions = max(1, target // batch_size * 3)
+ seen, rows, failed, submitted = set(), [], 0, 0
+ pool = ThreadPoolExecutor(max_workers=workers)
+ pending = set()
+
+ def _submit():
+ nonlocal submitted
+ pending.add(pool.submit(generate_examples, tools, batch_size,
+ model=model, api_key=api_key))
+ submitted += 1
+
+ for _ in range(min(workers, max(1, -(-target // batch_size)))):
+ _submit()
+
+ try:
+ while pending and len(rows) < num_samples:
+ done, pending = concurrent.futures.wait(
+ pending, return_when=concurrent.futures.FIRST_COMPLETED)
+ for future in done:
+ try:
+ for example in future.result():
+ key = _dedup_key(example)
+ if key not in seen:
+ seen.add(key)
+ rows.append(example)
+ except Exception as exc:
+ failed += 1
+ print(f" batch failed: {exc}", flush=True)
+ if len(rows) < num_samples and submitted < max_submissions:
+ _submit()
+ done_count = min(len(rows), num_samples)
+ if progress:
+ progress(done_count, num_samples)
+ else:
+ print(f" generated {done_count}/{num_samples} (failed={failed})", flush=True)
+ finally:
+ pool.shutdown(wait=False, cancel_futures=True)
+
+ return rows[:num_samples]
+
+
def _collect_tools(examples):
seen, tools = set(), []
for example in examples:
@@ -87,42 +146,38 @@ def _collect_tools(examples):
return tools
-def augment_jsonl(path, num_samples, model=DEFAULT_MODEL, batch_size=25, out_path=None):
+def augment_jsonl(path, num_samples, model=DEFAULT_MODEL, batch_size=25, out_path=None, workers=8):
with open(path) as handle:
examples = [json.loads(line) for line in handle if line.strip()]
tools = _collect_tools(examples)
if not tools:
raise RuntimeError("no tool schemas found in " + path)
out_path = out_path or path.replace(".jsonl", "") + ".augmented.jsonl"
- generated = []
- while len(generated) < num_samples:
- generated.extend(generate_examples(tools, n=min(batch_size, num_samples - len(generated)),
- model=model or DEFAULT_MODEL))
- print(f" generated {len(generated)}/{num_samples}")
+ generated = generate_dataset(tools, num_samples, model=model or DEFAULT_MODEL,
+ batch_size=batch_size, workers=workers)
with open(out_path, "w") as handle:
- for example in examples + generated[:num_samples]:
+ for example in examples + generated:
handle.write(json.dumps(example) + "\n")
- print(f"wrote {len(examples) + min(len(generated), num_samples)} examples -> {out_path}")
+ print(f"wrote {len(examples) + len(generated)} examples -> {out_path}")
return out_path
def generate_main(args):
model = args.model or DEFAULT_MODEL
+ workers = getattr(args, "workers", 8)
if args.tools:
with open(args.tools) as handle:
tools = json.load(handle)
- rows, out = [], args.output or "needle_data.jsonl"
- while len(rows) < args.num_samples:
- rows.extend(generate_examples(tools, n=min(args.batch_size, args.num_samples - len(rows)),
- model=model))
- print(f" generated {len(rows)}/{args.num_samples}")
+ out = args.output or "needle_data.jsonl"
+ rows = generate_dataset(tools, args.num_samples, model=model,
+ batch_size=args.batch_size, workers=workers)
with open(out, "w") as handle:
- for row in rows[:args.num_samples]:
+ for row in rows:
handle.write(json.dumps(row) + "\n")
- print(f"wrote {min(len(rows), args.num_samples)} examples -> {out}")
+ print(f"wrote {len(rows)} examples -> {out}")
elif args.augment:
augment_jsonl(args.augment, args.num_samples, model=model,
- batch_size=args.batch_size, out_path=args.output)
+ batch_size=args.batch_size, out_path=args.output, workers=workers)
else:
raise SystemExit("pass --tools <schemas.json> or --augment <data.jsonl>")
@@ -212,7 +267,8 @@ def finetune_local(args):
base_path = args.checkpoint or DEFAULT_BASE
data_path = args.jsonl_path
if getattr(args, "generate", 0):
- data_path = augment_jsonl(data_path, args.generate, model=getattr(args, "model", None))
+ data_path = augment_jsonl(data_path, args.generate, model=getattr(args, "model", None),
+ workers=getattr(args, "workers", 8))
params, config = load_checkpoint(base_path)
params = jax.device_put(params)
diff --git a/needle/playground/__init__.py b/needle/playground/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/needle/playground/app.js b/needle/playground/app.js
new file mode 100644
index 0000000..f1b56c8
--- /dev/null
+++ b/needle/playground/app.js
@@ -0,0 +1,331 @@
+var PRESETS = {"home": {"tools": [{"name": "set_lights", "description": "Turn lights on or off or dim them.", "parameters": {"type": "object", "properties": {"room": {"type": "string", "description": "Which room."}, "state": {"type": "string", "enum": ["on", "off"]}, "brightness": {"type": "integer", "description": "Percent 1-100."}}, "required": ["room", "state"]}}, {"name": "set_thermostat", "description": "Set the target temperature.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "temperature": {"type": "number"}, "mode": {"type": "string", "enum": ["heat", "cool", "auto"]}}, "required": ["room", "temperature"]}}, {"name": "lock_door", "description": "Lock a door.", "parameters": {"type": "object", "properties": {"door": {"type": "string", "description": "Which door."}}, "required": ["door"]}}], "q": "dim the bedroom lights to 20 percent and lock the front door"}, "robot": {"tools": [{"name": "move", "description": "Drive the robot in a direction.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["forward", "backward", "left", "right"]}, "distance_m": {"type": "number", "description": "Distance in meters."}}, "required": ["direction", "distance_m"]}}, {"name": "rotate", "description": "Rotate the robot in place.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["left", "right"]}, "degrees": {"type": "number"}}, "required": ["direction", "degrees"]}}, {"name": "gripper", "description": "Open or close the gripper.", "parameters": {"type": "object", "properties": {"action": {"type": "string", "enum": ["open", "close"]}}, "required": ["action"]}}], "q": "move forward 2 meters, turn left 90 degrees, then close the gripper"}, "device": {"tools": [{"name": "open_website", "description": "Open a website in a new tab.", "parameters": {"type": "object", "properties": {"url": {"type": "string", "description": "The site to open."}}, "required": ["url"]}}, {"name": "set_theme", "description": "Change the page accent color.", "parameters": {"type": "object", "properties": {"accent_color": {"type": "string", "description": "A color name or hex."}}, "required": ["accent_color"]}}, {"name": "speak", "description": "Say something out loud through the device speakers.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}], "q": "make the accent green, open github.com, and say hello"}, "gallery": {"tools": [{"name": "create_album", "description": "Create a new photo album.", "parameters": {"type": "object", "properties": {"name": {"type": "string"}}, "required": ["name"]}}, {"name": "move_photos", "description": "Move photos into an album.", "parameters": {"type": "object", "properties": {"album": {"type": "string"}, "filter": {"type": "string", "description": "Which photos, e.g. 'last weekend', 'screenshots'."}}, "required": ["album", "filter"]}}, {"name": "delete_photos", "description": "Delete photos.", "parameters": {"type": "object", "properties": {"filter": {"type": "string"}}, "required": ["filter"]}}], "q": "create an album called Summer 2026 and move the photos from last weekend into it"}, "email": {"tools": [{"name": "send_email", "description": "Send an email.", "parameters": {"type": "object", "properties": {"to": {"type": "string", "description": "Email address."}, "subject": {"type": "string"}, "body": {"type": "string"}}, "required": ["to", "subject", "body"]}}], "q": "send the receipt to [email protected] with subject expenses: Blue Bottle Coffee, $14.50, August 4th"}, "currency": {"tools": [{"name": "convert_currency", "description": "Convert an amount between currencies.", "parameters": {"type": "object", "properties": {"amount": {"type": "number"}, "from_currency": {"type": "string", "description": "ISO code, e.g. USD."}, "to_currency": {"type": "string", "description": "ISO code, e.g. EUR."}}, "required": ["amount", "from_currency", "to_currency"]}}], "q": "how much is 250 dollars in euros"}, "document": {"tools": [{"name": "record_booking", "description": "Record the details of a hotel booking.", "parameters": {"type": "object", "properties": {"hotel": {"type": "string"}, "confirmation_number": {"type": "string"}, "guest_name": {"type": "string"}, "room_type": {"type": "string"}, "check_in": {"type": "string"}, "check_out": {"type": "string"}, "total": {"type": "number"}, "currency": {"type": "string"}}, "required": ["confirmation_number", "check_in", "check_out", "total"]}}], "q": "extract the booking from this email: Dear Mr Okafor, thank you for choosing the Harbor Light Hotel. This email confirms reservation HL-88213 for a deluxe sea-view room, checking in on 14 September 2026 and checking out on 18 September 2026. The total for your stay is 642.50 euros, payable at checkout. Breakfast is included from 7am, and our shuttle meets the 9:40 ferry on request. We look forward to welcoming you. Warm regards, Reception."}, "sentiment": {"tools": [{"name": "classify_sentiment", "description": "Classify the sentiment of a message.", "parameters": {"type": "object", "properties": {"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral", "mixed"]}}, "required": ["sentiment"]}}], "q": "classify the sentiment of this message: this is the worst purchase I have ever made, it broke in a day and support ignored me"}, "crowded": {"tools": [{"name": "set_lights", "description": "Control lights.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "state": {"type": "string", "enum": ["on", "off"]}}, "required": ["room", "state"]}}, {"name": "set_thermostat", "description": "Set temperature.", "parameters": {"type": "object", "properties": {"temperature": {"type": "number"}}, "required": ["temperature"]}}, {"name": "lock_door", "description": "Lock a door.", "parameters": {"type": "object", "properties": {"door": {"type": "string"}}, "required": ["door"]}}, {"name": "play_music", "description": "Play music.", "parameters": {"type": "object", "properties": {"genre": {"type": "string"}}, "required": []}}, {"name": "set_timer", "description": "Set a timer.", "parameters": {"type": "object", "properties": {"time_human": {"type": "string"}}, "required": ["time_human"]}}, {"name": "send_message", "description": "Send a text.", "parameters": {"type": "object", "properties": {"recipient": {"type": "string"}, "message": {"type": "string"}}, "required": ["recipient", "message"]}}, {"name": "create_note", "description": "Create a note.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}, {"name": "start_vacuum", "description": "Start the robot vacuum.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}}, "required": []}}, {"name": "water_plants", "description": "Run the irrigation system.", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}, "required": []}}, {"name": "open_blinds", "description": "Open or close the blinds.", "parameters": {"type": "object", "properties": {"room": {"type": "string"}, "position": {"type": "string", "enum": ["open", "closed", "half"]}}, "required": ["room", "position"]}}, {"name": "preheat_oven", "description": "Preheat the oven.", "parameters": {"type": "object", "properties": {"temperature_c": {"type": "number"}}, "required": ["temperature_c"]}}, {"name": "charge_car", "description": "Start charging the car.", "parameters": {"type": "object", "properties": {"limit_percent": {"type": "integer"}}, "required": []}}], "q": "close the bedroom blinds halfway"}, "repeat": {"tools": [{"name": "add_to_list", "description": "Add one item to the shopping list.", "parameters": {"type": "object", "properties": {"item": {"type": "string"}, "quantity": {"type": "integer"}}, "required": ["item"]}}], "q": "add 2 milk and 6 eggs to the shopping list"}, "array": {"tools": [{"name": "add_items", "description": "Add items to the shopping list.", "parameters": {"type": "object", "properties": {"items": {"type": "array", "items": {"type": "string"}}}, "required": ["items"]}}], "q": "add milk, eggs, and bread to the shopping list"}, "flight": {"tools": [{"name": "search_flights", "description": "Search for flights.", "parameters": {"type": "object", "properties": {"from_city": {"type": "string"}, "to_city": {"type": "string"}, "date": {"type": "string"}, "direct_only": {"type": "boolean", "description": "Only non-stop flights."}, "passengers": {"type": "integer"}}, "required": ["from_city", "to_city"]}}], "q": "find non-stop flights from lagos to nairobi on december 3rd for two passengers"}, "refuse": {"tools": [{"name": "move", "description": "Drive the robot in a direction.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["forward", "backward", "left", "right"]}, "distance_m": {"type": "number", "description": "Distance in meters."}}, "required": ["direction", "distance_m"]}}, {"name": "rotate", "description": "Rotate the robot in place.", "parameters": {"type": "object", "properties": {"direction": {"type": "string", "enum": ["left", "right"]}, "degrees": {"type": "number"}}, "required": ["direction", "degrees"]}}, {"name": "gripper", "description": "Open or close the gripper.", "parameters": {"type": "object", "properties": {"action": {"type": "string", "enum": ["open", "close"]}}, "required": ["action"]}}], "q": "write me a short poem about the moon"}};
+var LABELS = {"home": "Smart home", "robot": "Robot", "gallery": "Gallery", "device": "Device control", "email": "Extract email", "currency": "Currency", "document": "Document extraction", "sentiment": "Sentiment", "crowded": "12 tools routing", "repeat": "Repeated calls", "array": "Array argument", "flight": "Flight form", "refuse": "Off-topic refusal"};
+
+var _toastTimer = null;
+
+function showError(msg) {
+ if (_toastTimer) clearTimeout(_toastTimer);
+ document.getElementById("toastMsg").textContent = msg;
+ document.getElementById("toast").classList.add("visible");
+ _toastTimer = setTimeout(dismissToast, 8000);
+}
+
+function dismissToast() {
+ document.getElementById("toast").classList.remove("visible");
+}
+
+function loadToolsFile(input) {
+ var file = input.files[0];
+ if (!file) return;
+ var reader = new FileReader();
+ reader.onload = function () {
+ try {
+ JSON.parse(reader.result);
+ document.getElementById("tools").value = reader.result;
+ newChat();
+ } catch (e) {
+ showError("Invalid JSON file");
+ }
+ };
+ reader.readAsText(file);
+ input.value = "";
+}
+
+function fetchModelName() {
+ fetch("/model").then(function (r) { return r.json(); })
+ .then(function (d) { document.getElementById("modelName").textContent = d.name || ""; })
+ .catch(function () {});
+}
+
+function loadModelFile(input) {
+ var file = input.files[0];
+ if (!file) return;
+ var name = document.getElementById("modelName");
+ name.textContent = "loading " + file.name + "...";
+ fetch("/load-model", { method: "POST", headers: { "X-Filename": file.name }, body: file })
+ .then(function (r) { return r.json(); })
+ .then(function (d) {
+ if (d.error) { showError(d.error); fetchModelName(); }
+ else { name.textContent = d.name; newChat(); }
+ })
+ .catch(function (e) { showError("Upload failed: " + e.message); fetchModelName(); });
+ input.value = "";
+}
+
+var conversation = document.getElementById("conversation");
+var emptyState = document.getElementById("emptyState");
+
+function addTurn(query, data) {
+ if (emptyState) { emptyState.remove(); emptyState = null; }
+ var turn = document.createElement("div");
+ turn.className = "turn";
+
+ var q = document.createElement("div");
+ q.className = "turn-query";
+ q.textContent = query;
+ turn.appendChild(q);
+
+ var pre = document.createElement("pre");
+ pre.className = "turn-result";
+ var calls = data.function_calls;
+ if (data.type === "refuse" || (Array.isArray(calls) && calls.length === 0)) {
+ pre.classList.add("refused");
+ pre.textContent = "no tool call (off-topic / refused)";
+ } else {
+ pre.textContent = JSON.stringify(calls || data, null, 2);
+ }
+ turn.appendChild(pre);
+
+ if (data.reasoning) {
+ var reason = document.createElement("div");
+ reason.className = "turn-reasoning";
+ var label = document.createElement("div");
+ label.className = "turn-reasoning-label";
+ label.textContent = "Reasoning trace";
+ var body = document.createElement("div");
+ body.className = "turn-reasoning-body";
+ body.textContent = data.reasoning;
+ reason.appendChild(label);
+ reason.appendChild(body);
+ turn.appendChild(reason);
+ }
+
+ var bits = [];
+ if (data.confidence !== undefined && data.confidence !== null)
+ bits.push("confidence " + Number(data.confidence).toFixed(4));
+ if (data.decode_tps) bits.push(Math.round(data.decode_tps) + " tok/s");
+ if (bits.length) {
+ var meta = document.createElement("div");
+ meta.className = "turn-meta";
+ meta.textContent = bits.join(" · ");
+ turn.appendChild(meta);
+ }
+
+ conversation.appendChild(turn);
+ conversation.scrollTop = conversation.scrollHeight;
+}
+
+async function send() {
+ var input = document.getElementById("query");
+ var btn = document.getElementById("sendBtn");
+ var query = input.value.trim();
+ if (!query) return;
+ var tools = document.getElementById("tools").value.trim() || "[]";
+ try { JSON.parse(tools); } catch (e) { showError("Invalid tools JSON"); return; }
+
+ input.disabled = true;
+ btn.disabled = true;
+ try {
+ var r = await fetch("/complete", {
+ method: "POST",
+ headers: { "Content-Type": "application/json" },
+ body: JSON.stringify({ query: query, tools: tools }),
+ });
+ var data = await r.json();
+ if (data.error) { showError(data.error); }
+ else { addTurn(query, data); input.value = ""; }
+ } catch (e) {
+ showError("Request failed: " + e.message);
+ } finally {
+ input.disabled = false;
+ btn.disabled = false;
+ input.focus();
+ }
+}
+
+function newChat() {
+ fetch("/reset", { method: "POST" }).catch(function () {});
+ conversation.innerHTML = "";
+ emptyState = document.createElement("div");
+ emptyState.className = "empty";
+ emptyState.id = "emptyState";
+ emptyState.textContent = "Pick a preset or type a query, then Run.";
+ conversation.appendChild(emptyState);
+}
+
+function applyPreset(key) {
+ var p = PRESETS[key];
+ document.getElementById("tools").value = JSON.stringify(p.tools, null, 2);
+ document.getElementById("query").value = p.q;
+ newChat();
+ document.getElementById("query").focus();
+}
+
+var presetBox = document.getElementById("presets");
+Object.keys(LABELS).forEach(function (key) {
+ if (!PRESETS[key]) return;
+ var b = document.createElement("button");
+ b.className = "preset-btn";
+ b.textContent = LABELS[key];
+ b.onclick = function () { applyPreset(key); };
+ presetBox.appendChild(b);
+});
+
+function togglePanel() {
+ var sidebar = document.getElementById("sidebar");
+ sidebar.classList.toggle("open");
+ document.querySelector(".tools-toggle span").textContent =
+ sidebar.classList.contains("open") ? "Query" : "Tools";
+}
+
+var _pollTimer = null;
+var _ftRunning = false;
+
+function openFinetuneModal() {
+ var tools = document.getElementById("tools").value.trim() || "[]";
+ try { JSON.parse(tools); } catch (e) { showError("Invalid tools JSON"); return; }
+ _resetModal();
+ document.getElementById("modalOverlay").classList.add("visible");
+ document.getElementById("ftApiKey").focus();
+}
+
+function closeModal(e) {
+ if (e && e.target && e.target !== document.getElementById("modalOverlay")) return;
+ if (_ftRunning) return;
+ document.getElementById("modalOverlay").classList.remove("visible");
+}
+
+function _resetModal() {
+ document.getElementById("ftSteps").classList.remove("visible");
+ document.getElementById("ftProgress").textContent = "";
+ var start = document.getElementById("ftStartBtn");
+ start.disabled = false;
+ start.textContent = "Start Finetune";
+ start.style.display = "";
+ document.getElementById("modalCloseBtn").style.display = "";
+ var dl = document.getElementById("ftDownload");
+ if (dl) dl.remove();
+ document.querySelectorAll(".modal-step").forEach(function (s) {
+ s.classList.remove("active", "done");
+ });
+}
+
+async function startFinetune() {
+ var apiKey = document.getElementById("ftApiKey").value.trim();
+ if (!apiKey) { showError("OpenRouter API key is required"); return; }
+ var tools = document.getElementById("tools").value.trim() || "[]";
+ try { JSON.parse(tools); } catch (e) { showError("Invalid tools JSON"); return; }
+ var samples = parseInt(document.getElementById("ftSamples").value, 10) || 200;
+
+ var btn = document.getElementById("ftStartBtn");
+ btn.disabled = true;
+ btn.textContent = "Starting...";
+ document.getElementById("modalCloseBtn").style.display = "none";
+ document.getElementById("ftSteps").classList.add("visible");
+ _ftRunning = true;
+
+ try {
+ var r = await fetch("/finetune", {
+ method: "POST",
+ headers: { "Content-Type": "application/json" },
+ body: JSON.stringify({ tools: tools, api_key: apiKey, samples: samples }),
+ });
+ var data = await r.json();
+ if (data.error) { showError(data.error); _ftRunning = false; _resetModal(); return; }
+ _pollTimer = setInterval(pollFinetune, 2000);
+ } catch (e) {
+ showError("Request failed: " + e.message);
+ _ftRunning = false;
+ _resetModal();
+ }
+}
+
+function _updateSteps(current) {
+ var steps = document.querySelectorAll(".modal-step");
+ var past = true;
+ steps.forEach(function (s) {
+ var name = s.getAttribute("data-step");
+ s.classList.remove("active", "done");
+ if (name === current) { s.classList.add("active"); past = false; }
+ else if (past) { s.classList.add("done"); }
+ });
+}
+
+var _stepLabels = {
+ "generating data": "Generating data...",
+ "training": "Training...",
+ "building": "Building .cact...",
+};
+
+async function pollFinetune() {
+ try {
+ var r = await fetch("/finetune/status");
+ var data = await r.json();
+ _updateSteps(data.step);
+ var btn = document.getElementById("ftStartBtn");
+ btn.textContent = _stepLabels[data.step] || data.step;
+ if (data.log && data.log.length)
+ document.getElementById("ftProgress").textContent = data.log[data.log.length - 1];
+
+ if (!data.running) {
+ clearInterval(_pollTimer);
+ _pollTimer = null;
+ _ftRunning = false;
+ document.getElementById("modalCloseBtn").style.display = "";
+ if (data.step === "done") {
+ document.querySelectorAll(".modal-step").forEach(function (s) {
+ s.classList.remove("active"); s.classList.add("done");
+ });
+ btn.style.display = "none";
+ fetchModelName();
+ if (data.checkpoint) {
+ var dl = document.createElement("a");
+ dl.id = "ftDownload";
+ dl.className = "modal-download";
+ dl.href = "/download/" + data.checkpoint;
+ dl.download = data.checkpoint;
+ dl.textContent = "Download " + data.checkpoint;
+ document.getElementById("ftFooter").appendChild(dl);
+ }
+ } else {
+ showError("Finetune failed — " + (data.error || "unknown error"));
+ btn.textContent = "Retry";
+ btn.disabled = false;
+ }
+ }
+ } catch (e) {
+ clearInterval(_pollTimer);
+ _pollTimer = null;
+ _ftRunning = false;
+ document.getElementById("modalCloseBtn").style.display = "";
+ showError("Lost connection to server");
+ document.getElementById("ftStartBtn").textContent = "Retry";
+ document.getElementById("ftStartBtn").disabled = false;
+ }
+}
+
+document.getElementById("query").addEventListener("keydown", function (e) {
+ if (e.key === "Enter" && !e.shiftKey) { e.preventDefault(); send(); }
+});
+document.addEventListener("keydown", function (e) { if (e.key === "Escape") closeModal(); });
+
+(function () {
+ var handle = document.getElementById("resizeHandle");
+ var sidebar = document.getElementById("sidebar");
+ var dragging = false;
+ handle.addEventListener("mousedown", function (e) {
+ if (window.innerWidth <= 768) return;
+ e.preventDefault();
+ dragging = true;
+ handle.classList.add("active");
+ document.body.style.cursor = "col-resize";
+ document.body.style.userSelect = "none";
+ });
+ window.addEventListener("mousemove", function (e) {
+ if (!dragging) return;
+ sidebar.style.width = Math.min(Math.max(e.clientX, 200), window.innerWidth * 0.6) + "px";
+ });
+ window.addEventListener("mouseup", function () {
+ if (!dragging) return;
+ dragging = false;
+ handle.classList.remove("active");
+ document.body.style.cursor = "";
+ document.body.style.userSelect = "";
+ });
+ window.addEventListener("resize", function () {
+ if (window.innerWidth <= 768) sidebar.style.width = "";
+ });
+})();
+
+fetchModelName();
+applyPreset("home");
diff --git a/needle/playground/index.html b/needle/playground/index.html
new file mode 100644
index 0000000..cc2c02c
--- /dev/null
+++ b/needle/playground/index.html
@@ -0,0 +1,111 @@
+<!DOCTYPE html>
+<html lang="en" class="dark">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <title>Needle 2 Playground | Cactus</title>
+ <link rel="preconnect" href="https://fonts.googleapis.com">
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+ <link href="https://fonts.googleapis.com/css2?family=Geist:wght@400;500;600;700&family=Geist+Mono:wght@400;500&display=swap" rel="stylesheet">
+ <link rel="stylesheet" href="/style.css">
+</head>
+<body>
+ <header>
+ <div class="header-border">
+ <div class="header-inner">
+ <a class="header-logo" href="https://cactuscompute.com">
+ <img src="https://cactuscompute.com/assets/cactus_white.png" alt="Cactus" width="32" height="32">
+ <span>Cactus</span>
+ </a>
+ <nav class="header-nav">
+ <span class="nav-current">Needle 2 Playground</span>
+ </nav>
+ <div class="header-spacer"></div>
+ <button class="tools-toggle" onclick="togglePanel()" aria-label="Toggle tools panel">
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12.22 2h-.44a2 2 0 0 0-2 2v.18a2 2 0 0 1-1 1.73l-.43.25a2 2 0 0 1-2 0l-.15-.08a2 2 0 0 0-2.73.73l-.22.38a2 2 0 0 0 .73 2.73l.15.1a2 2 0 0 1 1 1.72v.51a2 2 0 0 1-1 1.74l-.15.09a2 2 0 0 0-.73 2.73l.22.38a2 2 0 0 0 2.73.73l.15-.08a2 2 0 0 1 2 0l.43.25a2 2 0 0 1 1 1.73V20a2 2 0 0 0 2 2h.44a2 2 0 0 0 2-2v-.18a2 2 0 0 1 1-1.73l.43-.25a2 2 0 0 1 2 0l.15.08a2 2 0 0 0 2.73-.73l.22-.39a2 2 0 0 0-.73-2.73l-.15-.08a2 2 0 0 1-1-1.74v-.5a2 2 0 0 1 1-1.74l.15-.09a2 2 0 0 0 .73-2.73l-.22-.38a2 2 0 0 0-2.73-.73l-.15.08a2 2 0 0 1-2 0l-.43-.25a2 2 0 0 1-1-1.73V4a2 2 0 0 0-2-2z"/><circle cx="12" cy="12" r="3"/></svg>
+ <span>Tools</span>
+ </button>
+ </div>
+ </div>
+ </header>
+ <main>
+ <aside id="sidebar">
+ <div class="panel-section">
+ <label class="panel-label">Presets</label>
+ <div class="presets" id="presets"></div>
+ </div>
+ <div class="panel-section panel-tools">
+ <div class="panel-header">
+ <label class="panel-label">Tools JSON</label>
+ <label class="file-btn">
+ Upload
+ <input type="file" id="toolsFile" accept=".json" onchange="loadToolsFile(this)">
+ </label>
+ </div>
+ <textarea id="tools" spellcheck="false" placeholder="Paste tool definitions here..."></textarea>
+ <button class="btn-finetune" onclick="openFinetuneModal()">Finetune on these tools</button>
+ </div>
+ </aside>
+ <div class="resize-handle" id="resizeHandle"></div>
+ <section class="query-area">
+ <div class="query-section">
+ <div class="panel-header">
+ <label class="panel-label">Query</label>
+ <span class="model-name" id="modelName"></span>
+ <label class="file-btn">
+ Load model
+ <input type="file" id="modelFile" accept=".cact" onchange="loadModelFile(this)">
+ </label>
+ </div>
+ <div class="input-row">
+ <input id="query" type="text" placeholder="e.g. dim the bedroom lights to 20 percent" autofocus>
+ <button class="btn-primary" id="sendBtn" onclick="send()">Run</button>
+ <button class="btn-ghost" id="resetBtn" onclick="newChat()" title="Start a fresh conversation">New chat</button>
+ </div>
+ <p class="hint">Runs continue the same conversation, so you can ask follow-ups. Picking a preset or "New chat" resets the session.</p>
+ </div>
+ <div class="result-section">
+ <label class="panel-label">Conversation</label>
+ <div class="conversation" id="conversation">
+ <div class="empty" id="emptyState">Pick a preset or type a query, then Run.</div>
+ </div>
+ </div>
+ </section>
+ </main>
+
+ <div class="modal-overlay" id="modalOverlay" onclick="closeModal(event)">
+ <div class="modal" onclick="event.stopPropagation()">
+ <div class="modal-header">
+ <span class="modal-title">Finetune on your tools</span>
+ <button class="modal-close" id="modalCloseBtn" onclick="closeModal()">&times;</button>
+ </div>
+ <div class="modal-body">
+ <p class="modal-note">We generate tool-calling data for the tools in the sidebar with OpenRouter, LoRA-finetune the base model, and merge + export a tuned <span class="mono">.cact</span> you can download and run.</p>
+ <div class="modal-field">
+ <label class="modal-label">OpenRouter API Key</label>
+ <input type="password" id="ftApiKey" class="modal-input" placeholder="sk-or-..." autocomplete="off" spellcheck="false">
+ </div>
+ <div class="modal-field">
+ <label class="modal-label">Synthetic examples</label>
+ <input type="number" id="ftSamples" class="modal-input modal-input-sm" value="200" min="25" step="25">
+ </div>
+ <div class="modal-steps" id="ftSteps">
+ <div class="modal-step" data-step="generating data"><span class="step-dot"></span>Generating data</div>
+ <div class="modal-step" data-step="training"><span class="step-dot"></span>Training</div>
+ <div class="modal-step" data-step="building"><span class="step-dot"></span>Building .cact</div>
+ </div>
+ <div class="ft-progress mono" id="ftProgress"></div>
+ </div>
+ <div class="modal-footer" id="ftFooter">
+ <button class="btn-primary modal-start" id="ftStartBtn" onclick="startFinetune()">Start Finetune</button>
+ </div>
+ </div>
+ </div>
+
+ <div class="toast" id="toast">
+ <span class="toast-msg" id="toastMsg"></span>
+ <button class="toast-close" onclick="dismissToast()">&times;</button>
+ </div>
+ <script src="/app.js"></script>
+</body>
+</html>
diff --git a/needle/playground/server.py b/needle/playground/server.py
new file mode 100644
index 0000000..9e87be1
--- /dev/null
+++ b/needle/playground/server.py
@@ -0,0 +1,178 @@
+import json
+import os
+import tempfile
+import threading
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+
+_DIR = Path(__file__).parent
+_STATIC = {"/": "index.html", "/index.html": "index.html",
+ "/app.js": "app.js", "/style.css": "style.css"}
+_CTYPE = {".html": "text/html; charset=utf-8",
+ ".js": "application/javascript; charset=utf-8",
+ ".css": "text/css; charset=utf-8"}
+_DOWNLOADS = Path(tempfile.gettempdir())
+
+
+class Engine:
+ def __init__(self, weights=None):
+ self.weights = weights
+ self.name = os.path.basename(weights) if weights else "needle-2 (base)"
+ self.lock = threading.Lock()
+ self.tools_json = None
+ self.agent = None
+
+ def load(self):
+ from .. import Needle
+ self.agent = Needle(tools="[]", weights=self.weights)
+ self.tools_json = "[]"
+
+ def complete(self, tools_json, query):
+ from .. import Needle
+ with self.lock:
+ if self.agent is None or tools_json != self.tools_json:
+ self.agent = Needle(tools=tools_json, weights=self.weights)
+ self.tools_json = tools_json
+ return self.agent.complete(query)
+
+ def reset(self):
+ from .. import _lib
+ with self.lock:
+ if self.agent is not None:
+ _lib().needle_reset()
+ self.agent = None
+ self.tools_json = None
+
+ def load_weights(self, path):
+ with self.lock:
+ self.weights = path
+ self.name = os.path.basename(path)
+ self.agent = None
+ self.tools_json = None
+ self.load()
+
+
+_FT = {"running": False, "step": "", "log": [], "checkpoint": None, "error": None}
+
+
+def _finetune_worker(tools_json, api_key, samples, engine):
+ import types
+ try:
+ _FT.update(running=True, step="generating data", log=[], checkpoint=None, error=None)
+ from ..model.finetune import generate_dataset, finetune_local, build_main, DEFAULT_BASE
+
+ tools = json.loads(tools_json)
+ rows = generate_dataset(
+ tools, samples, api_key=api_key,
+ progress=lambda done, total: _FT["log"].append(f"generated {done}/{total}"))
+ data_path = str(_DOWNLOADS / "needle_playground_data.jsonl")
+ with open(data_path, "w") as handle:
+ for row in rows:
+ handle.write(json.dumps(row) + "\n")
+
+ _FT["step"] = "training"
+ adapter = str(_DOWNLOADS / "needle_playground_lora.pkl")
+ finetune_local(types.SimpleNamespace(
+ jsonl_path=data_path, checkpoint=None, epochs=3, batch_size=16, lr=1e-4,
+ lora_rank=16, lora_alpha=32.0, max_len=1024, generate=0, model=None,
+ checkpoint_dir=str(_DOWNLOADS), out=adapter))
+
+ _FT["step"] = "building"
+ out = str(_DOWNLOADS / "needle_tuned.cact")
+ build_main(types.SimpleNamespace(checkpoint=DEFAULT_BASE, lora=adapter,
+ out=out, upload=False, bits=None))
+ engine.load_weights(out)
+ _FT.update(running=False, step="done", checkpoint=os.path.basename(out))
+ except Exception as exc:
+ _FT.update(running=False, step="failed", error=str(exc))
+ _FT["log"].append(str(exc))
+
+
+class _Handler(BaseHTTPRequestHandler):
+ engine = None
+
+ def _send(self, code, body, ctype="application/json"):
+ data = body if isinstance(body, (bytes, bytearray)) else body.encode("utf-8")
+ self.send_response(code)
+ self.send_header("Content-Type", ctype)
+ self.send_header("Content-Length", str(len(data)))
+ self.end_headers()
+ self.wfile.write(data)
+
+ def _json_body(self):
+ length = int(self.headers.get("Content-Length", 0))
+ return json.loads(self.rfile.read(length) or b"{}")
+
+ def do_GET(self):
+ path = self.path.split("?")[0]
+ if path in _STATIC:
+ f = _DIR / _STATIC[path]
+ self._send(200, f.read_bytes(), _CTYPE[f.suffix])
+ elif path == "/model":
+ self._send(200, json.dumps({"name": self.engine.name}))
+ elif path == "/finetune/status":
+ self._send(200, json.dumps(_FT))
+ elif path.startswith("/download/"):
+ name = os.path.basename(path[len("/download/"):])
+ f = _DOWNLOADS / name
+ if f.exists():
+ self._send(200, f.read_bytes(), "application/octet-stream")
+ else:
+ self._send(404, b"not found", "text/plain")
+ else:
+ self._send(404, b"not found", "text/plain")
+
+ def do_POST(self):
+ try:
+ if self.path == "/complete":
+ body = self._json_body()
+ tools = body.get("tools", [])
+ tools_json = tools if isinstance(tools, str) else json.dumps(tools)
+ result = self.engine.complete(tools_json, body.get("query", ""))
+ self._send(200, json.dumps(result))
+ elif self.path == "/reset":
+ self.engine.reset()
+ self._send(200, json.dumps({"ok": True}))
+ elif self.path == "/load-model":
+ name = os.path.basename(self.headers.get("X-Filename", "model.cact"))
+ length = int(self.headers.get("Content-Length", 0))
+ dest = _DOWNLOADS / name
+ dest.write_bytes(self.rfile.read(length))
+ self.engine.load_weights(str(dest))
+ self._send(200, json.dumps({"name": self.engine.name}))
+ elif self.path == "/finetune":
+ if _FT["running"]:
+ self._send(200, json.dumps({"error": "a finetune is already running"}))
+ return
+ body = self._json_body()
+ api_key = (body.get("api_key") or "").strip()
+ if not api_key:
+ self._send(200, json.dumps({"error": "OpenRouter API key is required"}))
+ return
+ tools = body.get("tools", "[]")
+ tools_json = tools if isinstance(tools, str) else json.dumps(tools)
+ samples = int(body.get("samples", 200))
+ threading.Thread(target=_finetune_worker,
+ args=(tools_json, api_key, samples, self.engine),
+ daemon=True).start()
+ self._send(200, json.dumps({"ok": True}))
+ else:
+ self._send(404, b"not found", "text/plain")
+ except Exception as exc:
+ self._send(200, json.dumps({"error": str(exc)}))
+
+ def log_message(self, *args):
+ pass
+
+
+def main(args):
+ engine = Engine(weights=getattr(args, "weights", None))
+ print("needle playground: downloading and initializing the model...", flush=True)
+ engine.load()
+ _Handler.engine = engine
+ server = ThreadingHTTPServer((args.host, args.port), _Handler)
+ print(f"needle playground ready: http://{args.host}:{args.port} ({engine.name})", flush=True)
+ try:
+ server.serve_forever()
+ except KeyboardInterrupt:
+ server.shutdown()
diff --git a/needle/playground/style.css b/needle/playground/style.css
new file mode 100644
index 0000000..48a1a86
--- /dev/null
+++ b/needle/playground/style.css
@@ -0,0 +1,513 @@
+:root {
+ --background: #000000;
+ --foreground: #ffffff;
+ --card: #0a0a0a;
+ --primary: #ffffff;
+ --primary-foreground: #000000;
+ --muted-foreground: #a3a3a3;
+ --accent: #262626;
+ --accent-foreground: #ffffff;
+ --destructive: #FF3B30;
+ --brand: #ff5a36;
+ --border: #262626;
+ --input: #171717;
+ --ring: #262626;
+ --radius: 0.5rem;
+}
+
+*, *::before, *::after {
+ box-sizing: border-box;
+ margin: 0;
+ padding: 0;
+ border-color: var(--border);
+}
+
+body {
+ background: var(--background);
+ color: var(--foreground);
+ font-family: 'Geist', -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
+ font-size: 14px;
+ line-height: 1.5;
+ height: 100vh;
+ display: flex;
+ flex-direction: column;
+ -webkit-font-smoothing: antialiased;
+ -moz-osx-font-smoothing: grayscale;
+}
+
+.mono { font-family: 'Geist Mono', 'SFMono-Regular', 'Menlo', monospace; }
+
+header {
+ position: sticky;
+ top: 0;
+ z-index: 50;
+ background: rgba(0, 0, 0, 0.8);
+ backdrop-filter: blur(12px);
+ -webkit-backdrop-filter: blur(12px);
+ flex-shrink: 0;
+}
+
+.header-border { border-bottom: 1px solid var(--border); }
+
+.header-inner {
+ width: 100%;
+ height: 64px;
+ padding: 0 24px;
+ display: flex;
+ align-items: center;
+}
+
+.header-logo {
+ display: flex;
+ align-items: center;
+ gap: 8px;
+ text-decoration: none;
+ color: var(--foreground);
+}
+
+.header-logo img { height: 32px; width: auto; object-fit: contain; }
+.header-logo span { font-size: 18px; font-weight: 600; letter-spacing: -0.02em; }
+
+.header-nav {
+ display: flex;
+ align-items: center;
+ margin-left: 12px;
+ padding-left: 12px;
+ border-left: 1px solid var(--border);
+}
+
+.nav-current { color: var(--muted-foreground); font-size: 14px; }
+.header-spacer { flex: 1; }
+
+.tools-toggle {
+ display: none;
+ align-items: center;
+ gap: 6px;
+ height: 32px;
+ padding: 0 12px;
+ background: transparent;
+ color: var(--muted-foreground);
+ border: 1px solid var(--border);
+ border-radius: calc(var(--radius) - 2px);
+ font-family: inherit;
+ font-size: 13px;
+ font-weight: 500;
+ cursor: pointer;
+ transition: all 150ms;
+}
+
+.tools-toggle:hover { color: var(--foreground); background: var(--accent); }
+
+main { flex: 1; display: flex; overflow: hidden; width: 100%; }
+
+aside {
+ width: 360px;
+ min-width: 240px;
+ max-width: 60vw;
+ flex-shrink: 0;
+ padding: 20px;
+ display: flex;
+ flex-direction: column;
+ gap: 18px;
+ overflow-y: auto;
+}
+
+.resize-handle {
+ width: 5px;
+ flex-shrink: 0;
+ cursor: col-resize;
+ background: transparent;
+ border-left: 1px solid var(--border);
+ transition: background 150ms;
+}
+
+.resize-handle:hover, .resize-handle.active { background: var(--accent); }
+
+.panel-section { display: flex; flex-direction: column; gap: 8px; }
+.panel-tools { flex: 1; min-height: 0; }
+
+.panel-header { display: flex; align-items: center; justify-content: space-between; }
+.panel-label { font-size: 13px; font-weight: 600; letter-spacing: -0.01em; }
+
+.presets { display: flex; flex-wrap: wrap; gap: 6px; }
+
+.preset-btn {
+ padding: 5px 10px;
+ background: var(--card);
+ color: var(--muted-foreground);
+ border: 1px solid var(--border);
+ border-radius: 999px;
+ font-family: inherit;
+ font-size: 12px;
+ font-weight: 500;
+ cursor: pointer;
+ transition: all 150ms;
+}
+
+.preset-btn:hover { color: var(--foreground); border-color: var(--brand); }
+
+.file-btn {
+ font-size: 12px;
+ font-weight: 500;
+ color: var(--muted-foreground);
+ cursor: pointer;
+ transition: color 150ms;
+}
+
+.file-btn:hover { color: var(--foreground); }
+.file-btn input { display: none; }
+
+.model-name { font-size: 12px; font-family: 'Geist Mono', monospace; color: var(--muted-foreground); }
+
+.btn-finetune {
+ flex-shrink: 0;
+ height: 36px;
+ padding: 0 14px;
+ background: var(--primary);
+ color: var(--primary-foreground);
+ border: none;
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 13px;
+ font-weight: 500;
+ cursor: pointer;
+ transition: background 150ms;
+ width: 100%;
+}
+
+.btn-finetune:hover { background: rgba(255, 255, 255, 0.9); }
+.btn-finetune:disabled { opacity: 0.5; pointer-events: none; }
+
+textarea {
+ flex: 1;
+ min-height: 200px;
+ font-family: 'Geist Mono', 'SFMono-Regular', 'Menlo', 'Monaco', 'Consolas', monospace;
+ font-size: 13px;
+ line-height: 1.6;
+ resize: none;
+ padding: 12px;
+ background: var(--card);
+ color: var(--foreground);
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ outline: none;
+ transition: border-color 150ms;
+}
+
+textarea:focus { border-color: var(--ring); box-shadow: 0 0 0 3px rgba(38, 38, 38, 0.5); }
+textarea::placeholder { color: var(--muted-foreground); }
+
+.query-area { flex: 1; display: flex; flex-direction: column; padding: 20px; gap: 20px; min-width: 0; }
+.query-section { display: flex; flex-direction: column; gap: 8px; flex-shrink: 0; }
+
+.input-row { display: flex; gap: 8px; }
+
+.input-row input {
+ flex: 1;
+ height: 40px;
+ padding: 0 14px;
+ background: var(--input);
+ color: var(--foreground);
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 14px;
+ outline: none;
+ transition: border-color 150ms;
+}
+
+.input-row input:focus { border-color: var(--ring); box-shadow: 0 0 0 3px rgba(38, 38, 38, 0.5); }
+.input-row input::placeholder { color: var(--muted-foreground); }
+
+.btn-primary {
+ height: 40px;
+ padding: 0 20px;
+ background: var(--primary);
+ color: var(--primary-foreground);
+ border: none;
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 14px;
+ font-weight: 500;
+ cursor: pointer;
+ transition: background 150ms;
+ white-space: nowrap;
+}
+
+.btn-primary:hover { background: rgba(255, 255, 255, 0.9); }
+.btn-primary:disabled { opacity: 0.5; pointer-events: none; }
+
+.btn-ghost {
+ height: 40px;
+ padding: 0 16px;
+ background: transparent;
+ color: var(--muted-foreground);
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 14px;
+ cursor: pointer;
+ transition: all 150ms;
+ white-space: nowrap;
+}
+
+.btn-ghost:hover { color: var(--foreground); background: var(--accent); }
+
+.hint { font-size: 12px; color: var(--muted-foreground); }
+
+.result-section { flex: 1; min-height: 0; display: flex; flex-direction: column; gap: 8px; }
+
+.conversation {
+ flex: 1;
+ min-height: 0;
+ padding: 8px;
+ background: var(--card);
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ overflow-y: auto;
+ display: flex;
+ flex-direction: column;
+ gap: 12px;
+}
+
+.empty { color: var(--muted-foreground); padding: 12px; font-size: 13px; }
+
+.turn { display: flex; flex-direction: column; gap: 6px; padding: 10px; border-bottom: 1px solid var(--border); }
+.turn:last-child { border-bottom: none; }
+
+.turn-query { color: var(--foreground); font-size: 13px; font-weight: 500; }
+.turn-query::before { content: "› "; color: var(--brand); }
+
+.turn-result {
+ margin: 0;
+ padding: 12px;
+ background: var(--input);
+ border: 1px solid var(--border);
+ border-radius: calc(var(--radius) - 2px);
+ font-family: 'Geist Mono', 'SFMono-Regular', 'Menlo', 'Monaco', 'Consolas', monospace;
+ font-size: 13px;
+ line-height: 1.6;
+ color: var(--foreground);
+ white-space: pre-wrap;
+ word-break: break-word;
+}
+
+.turn-result.refused { color: var(--muted-foreground); font-style: italic; }
+
+.turn-reasoning {
+ font-size: 13px;
+ color: var(--foreground);
+ line-height: 1.55;
+ padding: 8px 12px;
+ background: var(--input);
+ border-left: 2px solid var(--brand);
+ border-radius: calc(var(--radius) - 2px);
+}
+
+.turn-reasoning-label {
+ font-size: 10px;
+ font-weight: 600;
+ text-transform: uppercase;
+ letter-spacing: 0.06em;
+ color: var(--brand);
+ margin-bottom: 4px;
+}
+
+.turn-reasoning-body { color: var(--foreground); }
+
+.turn-meta { font-size: 12px; color: var(--muted-foreground); font-family: 'Geist Mono', monospace; }
+
+.modal-overlay {
+ position: fixed;
+ inset: 0;
+ z-index: 60;
+ display: none;
+ align-items: center;
+ justify-content: center;
+ background: rgba(0, 0, 0, 0.6);
+ padding: 16px;
+}
+
+.modal-overlay.visible { display: flex; }
+
+.modal {
+ width: 100%;
+ max-width: 520px;
+ max-height: 85vh;
+ background: var(--card);
+ border: 1px solid var(--border);
+ border-radius: 12px;
+ display: flex;
+ flex-direction: column;
+ overflow: hidden;
+}
+
+.modal-header {
+ display: flex;
+ align-items: center;
+ justify-content: space-between;
+ padding: 16px 20px;
+ border-bottom: 1px solid var(--border);
+ flex-shrink: 0;
+}
+
+.modal-title { font-size: 15px; font-weight: 600; }
+
+.modal-close {
+ width: 28px;
+ height: 28px;
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ background: none;
+ border: none;
+ color: var(--muted-foreground);
+ font-size: 20px;
+ cursor: pointer;
+ border-radius: calc(var(--radius) - 2px);
+ transition: color 150ms, background 150ms;
+ padding: 0;
+ line-height: 1;
+}
+
+.modal-close:hover { color: var(--foreground); background: var(--accent); }
+
+.modal-body { padding: 20px; overflow-y: auto; display: flex; flex-direction: column; gap: 16px; }
+.modal-note { font-size: 13px; color: var(--muted-foreground); line-height: 1.5; }
+.modal-field { display: flex; flex-direction: column; gap: 6px; }
+.modal-label { font-size: 13px; font-weight: 500; color: var(--muted-foreground); }
+
+.modal-input {
+ height: 36px;
+ padding: 0 12px;
+ background: var(--input);
+ color: var(--foreground);
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 13px;
+ outline: none;
+ transition: border-color 150ms;
+ width: 100%;
+}
+
+.modal-input:focus { border-color: var(--ring); box-shadow: 0 0 0 3px rgba(38, 38, 38, 0.5); }
+.modal-input::placeholder { color: var(--muted-foreground); }
+.modal-input-sm { font-family: 'Geist Mono', monospace; }
+
+.modal-steps { display: none; gap: 4px; }
+.modal-steps.visible { display: flex; flex-direction: column; }
+
+.modal-step { display: flex; align-items: center; gap: 8px; font-size: 13px; color: var(--muted-foreground); padding: 4px 0; }
+.modal-step.active { color: var(--foreground); }
+.modal-step.done { color: #4ade80; }
+
+.step-dot { width: 8px; height: 8px; border-radius: 50%; background: var(--accent); flex-shrink: 0; transition: background 200ms; }
+.modal-step.active .step-dot { background: var(--foreground); }
+.modal-step.done .step-dot { background: #4ade80; }
+
+.ft-progress { min-height: 20px; color: var(--foreground); font-size: 13px; }
+
+.modal-footer { padding: 16px 20px; border-top: 1px solid var(--border); flex-shrink: 0; }
+.modal-start { width: 100%; }
+
+.modal-download {
+ display: block;
+ width: 100%;
+ height: 40px;
+ line-height: 40px;
+ text-align: center;
+ background: var(--primary);
+ color: var(--primary-foreground);
+ border-radius: var(--radius);
+ font-family: inherit;
+ font-size: 14px;
+ font-weight: 500;
+ text-decoration: none;
+ transition: background 150ms;
+}
+
+.modal-download:hover { background: rgba(255, 255, 255, 0.9); }
+
+.toast {
+ position: fixed;
+ top: 80px;
+ right: 24px;
+ display: flex;
+ align-items: flex-start;
+ gap: 10px;
+ max-width: 420px;
+ padding: 12px 14px;
+ background: #1a0000;
+ border: 1px solid rgba(255, 59, 48, 0.3);
+ border-radius: var(--radius);
+ color: #ff6b6b;
+ font-size: 13px;
+ line-height: 1.5;
+ z-index: 100;
+ opacity: 0;
+ transform: translateX(20px);
+ pointer-events: none;
+ transition: opacity 200ms, transform 200ms;
+}
+
+.toast.visible { opacity: 1; transform: translateX(0); pointer-events: auto; }
+.toast-msg { flex: 1; word-break: break-word; }
+
+.toast-close {
+ flex-shrink: 0;
+ width: 20px;
+ height: 20px;
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ background: none;
+ border: none;
+ color: #ff6b6b;
+ font-size: 16px;
+ cursor: pointer;
+ opacity: 0.6;
+ transition: opacity 150ms;
+ padding: 0;
+ line-height: 1;
+}
+
+.toast-close:hover { opacity: 1; }
+
+::-webkit-scrollbar { width: 6px; }
+::-webkit-scrollbar-track { background: transparent; }
+::-webkit-scrollbar-thumb { background: var(--accent); border-radius: 3px; }
+
+@media (max-width: 768px) {
+ .header-inner { padding: 0 16px; height: 56px; }
+ .header-nav { display: none; }
+ .tools-toggle { display: flex; }
+ main { flex-direction: column; min-height: 0; }
+ .resize-handle { display: none !important; }
+ aside {
+ width: 100% !important;
+ min-width: 0 !important;
+ max-width: none !important;
+ flex-shrink: 0;
+ display: none !important;
+ padding: 16px;
+ overflow-y: auto;
+ }
+ aside.open { display: flex !important; flex: 1; max-height: none; }
+ aside.open .panel-tools { flex: 1; min-height: 0; }
+ aside.open textarea { flex: 1; min-height: 0; }
+ aside.open ~ .resize-handle, aside.open ~ .query-area { display: none !important; }
+ .query-area { flex: 1; min-height: 0; padding: 16px; gap: 16px; }
+ .result-section { flex: 1; min-height: 0; }
+ .modal { max-width: none; max-height: none; height: 100%; border-radius: 0; }
+ .modal-overlay { padding: 0; }
+}
+
+@media (max-width: 480px) {
+ .header-inner { padding: 0 12px; }
+ .header-logo span { display: none; }
+ aside.open { padding: 12px; }
+ .query-area { padding: 12px; }
+ .btn-primary { padding: 0 14px; }
+ .btn-ghost { display: none; }
+}
diff --git a/pyproject.toml b/pyproject.toml
index f55e74f..5595fe0 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -27,7 +27,7 @@ include = ["needle*"]
[tool.setuptools.package-data]
"needle.model" = ["*.model", "*.vocab"]
-"needle.agent" = ["index.html"]
+"needle.playground" = ["index.html", "app.js", "style.css"]
[build-system]
requires = ["setuptools>=68.0"]
diff --git a/tests/test_generate.py b/tests/test_generate.py
index c205269..60650ef 100644
--- a/tests/test_generate.py
+++ b/tests/test_generate.py
@@ -43,17 +43,48 @@ def test_generate_examples_requires_api_key(monkeypatch):
finetune.generate_examples([{"name": "f"}], n=1)
+def _unique_generator():
+ counter = {"i": 0}
+
+ def fake(tools, n=1, **k):
+ rows = []
+ for _ in range(n):
+ counter["i"] += 1
+ rows.append({"tools": tools, "query": "gen %d" % counter["i"], "answers": []})
+ return rows
+ return fake
+
+
+def test_generate_dataset_hits_target_concurrently(monkeypatch):
+ from needle.model import finetune
+
+ monkeypatch.setenv("OPENROUTER_API_KEY", "test-key")
+ monkeypatch.setattr(finetune, "generate_examples", _unique_generator())
+ rows = finetune.generate_dataset([{"name": "f"}], 10, batch_size=3, workers=4)
+ assert len(rows) == 10
+ assert len({finetune._dedup_key(r) for r in rows}) == 10
+
+
+def test_generate_dataset_dedups_and_terminates(monkeypatch):
+ from needle.model import finetune
+
+ monkeypatch.setenv("OPENROUTER_API_KEY", "test-key")
+ monkeypatch.setattr(finetune, "generate_examples",
+ lambda tools, n=1, **k: [{"tools": tools, "query": "same", "answers": []}])
+ rows = finetune.generate_dataset([{"name": "f"}], 5, batch_size=1, workers=2)
+ assert len(rows) == 1
+
+
def test_augment_jsonl_appends_generated(monkeypatch, tmp_path):
from needle.model import finetune
+ monkeypatch.setenv("OPENROUTER_API_KEY", "test-key")
tools = [{"name": "f", "parameters": {"type": "object", "properties": {}}}]
src = tmp_path / "seed.jsonl"
with open(src, "w") as handle:
handle.write(json.dumps({"tools": tools, "query": "seed", "answers": []}) + "\n")
- monkeypatch.setattr(finetune, "generate_examples",
- lambda tools, n=1, **k: [{"tools": tools, "query": "gen",
- "answers": []} for _ in range(n)])
+ monkeypatch.setattr(finetune, "generate_examples", _unique_generator())
out = finetune.augment_jsonl(str(src), num_samples=3, out_path=str(tmp_path / "out.jsonl"))
lines = [line for line in open(out) if line.strip()]
assert len(lines) == 1 + 3