Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Implement platform integration for fine-tuning and model management
HenryNdubuakuf189b23ebf34
diff --git a/README.md b/README.mdindex f5f574a..f39432d 100644--- a/README.md+++ b/README.md@@ -62,13 +62,44 @@ Needle was designed to be customised. Its capacity is a ladder, and a subnetwork+Two ways to fine-tune, from the same package:++| | Local, `needle finetune` | Platform, `needle platform finetune` |+| --- | --- | --- |+| What trains | LoRA adapters on the attention projections, base frozen, merged at export | The full model, every depth from 2 layers up |+| What it keeps | Your data only | Your data reinforced with Needle's original dataset, so nothing already learned is unlearned |+| Confidence | Head untouched; `confidence` is `None` | Head fine-tuned with the model, calibrated on your tools |+| Precision | 4-bit | 2-bit, the same post-training as the shipped model |+| Data | Your JSONL, `query`/`answers` or chat format | Yours, or generated from your tool definitions, 100 to 10,000 examples per run |+| Scores | Validation loss | Validation and test accuracy for every depth |+| Compute | Your machine, JAX on CPU, CUDA or Metal | Cactus GPUs |+| Runs from | The CLI | The CLI, Python, the [dashboard](https://cactuscompute.com/dashboard), or a coding agent holding your key |++Local:+```shpip install "cactus-needle[train]"needle finetune data.jsonl --epochs 10 --out adapter.safetensorsneedle build --lora adapter.safetensors --layers 8 --out tuned.cact```-Local fine-tuning trains and exports at 4 bits; the [fine-tuning guide](https://cactuscompute.com/blog/finetuning-needle) has the rest. The 2-bit post-training and quantisation behind the shipped model, enriched with Cactus proprietary datasets, run on the [Cactus Platform](https://cactuscompute.com/dashboard).+Platform, with a key from the [console](https://cactuscompute.com/dashboard/api-keys) in `NEEDLE_API_KEY`. One command uploads the files, trains and scores every size, and downloads the `.cact` files; once a job is submitted it can also be followed on the dashboard:++```sh+export NEEDLE_API_KEY=needle_ft_...+needle platform generate --tools tools.json --examples 1000 --out ./data+needle platform finetune data/train.jsonl data/validation.jsonl data/test.jsonl --suffix smart-home --out ./models+```++```python+from needle.platform import Platform++client = Platform()+job = client.wait(client.finetune(["train.jsonl"], ["validation.jsonl"], ["test.jsonl"], suffix="smart-home"))+paths = client.download(job["fine_tuned_model"], "models", depth=8)+```++Or hand the key to Claude Code or Codex with [cactuscompute.com/llms.txt](https://cactuscompute.com/llms.txt) and let the agent run the loop. `needle platform jobs | models | files | billing` list what the account holds, `needle download model-<id>` fetches a model by id, and the [fine-tuning guide](https://cactuscompute.com/blog/finetuning-needle) covers the data format and how to read the scores.## Deploydiff --git a/llms.txt b/llms.txtindex 3d8b5f8..24f3400 100644--- a/llms.txt+++ b/llms.txt@@ -128,7 +128,7 @@ if r["type"] == "call":## Confidence gating-`confidence` is the min of a post-hoc calibration head and the decode probability of the call. Calibration holds for the base model only: an agent constructed with `weights=` reports `confidence` as None (fine-tuning does not update the head). Pick a threshold per product; act at/above it, escalate below it:+`confidence` is the min of a post-hoc calibration head and the decode probability of the call. The head ships inside the archive when it was trained for those weights: the base model and platform fine-tunes have it, so they report a calibrated score; local `needle finetune` does not train it, `needle build --lora` leaves it out, and an agent constructed with such weights reports `confidence` as None. Pick a threshold per product; act at/above it, escalate below it:```pythonr = agent.complete(user_text)@@ -187,6 +187,31 @@ agent.run("...")Options: `finetune --lora-rank 16 --lora-alpha 32 --lr 1e-4 --batch-size 16 --max-len 1024 --val-split 0.1 --checkpoint <base.safetensors or .pkl> --checkpoint-dir <dir> --out <adapter.safetensors or .pkl> --generate <n> --model <id> --workers <n>`; `--model` defaults to `deepseek/deepseek-flash-latest`, `--workers` defaults to `8`, and `--checkpoint-dir` defaults to `checkpoints`. `--generate` synthesizes extra examples through the configured OpenRouter endpoint before training. `build --bits 2|4 --upload` (with `NEEDLE_HF_REPO=<you>/<model>`).+## Platform (hosted fine-tuning)++The hosted platform at https://cactuscompute.com/v1 trains every size from 2 layers up on GPUs, scores each on your test file, and serves the `.cact` files; the API reference is https://cactuscompute.com/llms/api. The package drives it with a key from https://cactuscompute.com/dashboard/api-keys in `NEEDLE_API_KEY` (never print it). Files are chat-format JSONL: `messages` (OpenAI chat turns, tool calls on the assistant turn as `tool_calls` with `arguments` as a JSON string) and OpenAI-form `tools`.++```sh+needle platform finetune train.jsonl validation.jsonl test.jsonl [--max-depth N] [--suffix name] [--out dir] [--depth N] [--no-wait]+needle platform generate --tools tools.json --examples 1000 [--description "..."] [--message "..."] [--suffix name] [--out dir]+needle platform jobs [ftjob-id [--wait] [--out dir] [--depth N]] | models [model-id] | files | billing+needle download model-<id> [--depth N] [--out dir]+```++```python+from needle.platform import Platform, PlatformError+client = Platform() # key from NEEDLE_API_KEY, or Platform(api_key=...)+client.billing() # plan, limits, usage; a fine-tune spends one job at submission+job = client.finetune(["train.jsonl"], ["validation.jsonl"], ["test.jsonl"], max_depth=20, suffix="smart-home")+job = client.wait(job, poll=10) # raises PlatformError(code) if the job fails+job["evaluations"] # [{depth, validation: {correct, total, excluded}, test: {...}}, ...]+client.download(job["fine_tuned_model"], "models") # every size as <name>-<depth>L.cact; depth=8 for one+gen = client.wait(client.generate(tools, 1000, description="...", messages=["..."], suffix="smart-home"))+client.download_file(gen["files"][0]["id"], "train.jsonl")+```++Submission prints the job's dashboard page, https://cactuscompute.com/dashboard/jobs/<id>; Ctrl-C leaves the job running and `needle platform jobs <id> --wait --out dir` resumes the wait and the download. `PlatformError.code` is the API's stable code (`unauthorized`, `subscription_required`, `quota_exceeded`, `job_active`, `rate_limited`, `invalid_request`, ...), with `.param` and `.url` when the API sends them; `rate_limited` is retried after `Retry-After` automatically. Ask before every `finetune` or `generate`: both spend allowance at submission. Platform files also train locally: `needle finetune train.jsonl` reads single-turn chat-format lines as well as the `query`/`answers` form.+## Playground (browser)Hosted at https://cactuscompute.com/needle: preset demos, editable tools/prompt, multi-turn follow-ups, and a "Finetune on these tools" button that runs the pipeline above and returns a downloadable `.cact`. `needle playground [--weights my.cact]` serves the same UI locally at http://127.0.0.1:7860. Local fine-tuning is 4-bit; the 2-bit post-training and quantisation behind the shipped model, enriched with Cactus proprietary datasets, run on the Cactus Platform at https://cactuscompute.com/dashboard.@@ -208,7 +233,8 @@ Adapt by swapping the Literal values for your own and keeping the shapes: closed- `needle generate-data` / `needle finetune` / `needle build [checkpoint] [--lora adapter] [--layers N] [--platform <folder>] [--out <path>]` - the fine-tuning pipeline; fine-tuning runs at the full 20 layers, `build` merges the adapter, slices the `--layers` rung (default 20), exports at 4 bits, and fetches the published `needle3.cact` on every run (without an adapter at full depth it ships that published 2-bit archive as is). `--platform` also downloads that platform's engine and header and makes `--out` a folder holding them plus `needle3.cact`.- `needle playground` - the browser UI locally (hosted at https://cactuscompute.com/needle).- `needle fetch [--platform-tag <tag>] [--out <dir>]` - pre-download the inference engine for this machine (or another platform) into the cache; prints the path. For air-gapped devices, also see `NEEDLE3_LIB_PATH` and `HF_HUB_OFFLINE` at https://cactuscompute.com/blog/needle-python-docs.-- `needle download needle3 | needle3.safetensors | <org>/<repo>[/<file>.cact] | <platform> [--out <dir>]` - pull the base Needle 3 archive, a checkpoint to fine-tune, a published `.cact` (single-archive repos need only `<org>/<repo>`), or pass a platform folder name (`macos-arm64`, `linux-x86_64`, `wasm`, `wasm-component`, ...) to fetch that platform's engine files into `<out>/<platform>/`. Native runners are marked executable; `wasm-component` contains the WASI Preview 2 component and WIT contract.+- `needle platform finetune | generate | jobs | models | files | billing` - the hosted platform from the CLI (section above).+- `needle download needle3 | needle3.safetensors | model-<id> | <org>/<repo>[/<file>.cact] | <platform> [--out <dir>]` - pull the base Needle 3 archive, a checkpoint to fine-tune, a platform-trained model by id (`--depth N` for one size), a published `.cact` (single-archive repos need only `<org>/<repo>`), or pass a platform folder name (`macos-arm64`, `linux-x86_64`, `wasm`, `wasm-component`, ...) to fetch that platform's engine files into `<out>/<platform>/`. Native runners are marked executable; `wasm-component` contains the WASI Preview 2 component and WIT contract.## Common mistakes to avoiddiff --git a/needle/__init__.py b/needle/__init__.pyindex cd24b20..9d3e7e5 100644--- a/needle/__init__.py+++ b/needle/__init__.py@@ -41,6 +41,40 @@ def _weight_generation(path):"cannot choose a compatible Needle engine") from exc+_CACT_HEADER = "<48If"+_CACT_RECORD = "<BBHIIIIQQII"+_CACT_FP16, _CACT_RAW = 1, 4+_CONFIDENCE_HEAD_CODE = 2+++def _confidence_head_present(path):+ """Whether a Needle 3 archive carries a confidence head, read from its head manifest."""+ import struct++ with open(path, "rb") as handle:+ header = handle.read(struct.calcsize(_CACT_HEADER))+ if len(header) < struct.calcsize(_CACT_HEADER):+ return False+ fields = struct.unpack(_CACT_HEADER, header)+ num_tensors, codebook = fields[1], fields[2]+ handle.seek(codebook * 4, 1)+ size = struct.calcsize(_CACT_RECORD)+ records = [struct.unpack(_CACT_RECORD, handle.read(size)) for _ in range(num_tensors)]+ if not records or records[-1][0] != _CACT_RAW:+ return False+ for heads in (1, 2, 3):+ index = num_tensors - 2 - 6 * heads+ if index < 0:+ continue+ dtype, ndim, _, length = records[index][:4]+ if dtype != _CACT_FP16 or ndim != 1 or length != heads:+ continue+ handle.seek(records[index][7])+ codes = struct.unpack(f"<{heads}e", handle.read(2 * heads))+ return _CONFIDENCE_HEAD_CODE in {int(round(c)) for c in codes}+ return False++def _library_path(generation=2):from .agent import fetch@@ -132,9 +166,11 @@ class Needle:self._generation = int(generation or 3)self._worker = Noneself._closed = False- if self._tuned:- warnings.warn("finetuning does not update the confidence head, so scores are "- "uncalibrated for tuned weights; this agent reports confidence as None",+ self._calibrated = (not self._tuned+ or (self._generation >= 3 and _confidence_head_present(self._weights)))+ if not self._calibrated:+ warnings.warn("these weights carry no confidence head trained for them, so this "+ "agent reports confidence as None; platform fine-tunes keep the head",stacklevel=2)self._system_text = _with_date_fact(system or "") if auto_date else (system or "")self._system = self._system_text.encode("utf-8")@@ -220,7 +256,7 @@ class Needle:raise RuntimeError(f"engine returned an unparseable envelope ({err}); this is an "f"engine bug - please report it with the prompt and schema") from err- if self._tuned:+ if not self._calibrated:response["confidence"] = Noneif ground:_annotate_ungrounded(response, self._tool_schemas, self._seen_years,diff --git a/needle/cli.py b/needle/cli.pyindex bc649f6..89a6859 100644--- a/needle/cli.py+++ b/needle/cli.py@@ -10,7 +10,8 @@ HELP = """usage: needle <command> [options]finetune train a LoRA adapter on JSONL data (--layers N for a rung)generate-data synthesise training data via OpenRouterbuild export a checkpoint (+ adapter) to a .cact archive- download needle3 | needle3.safetensors | <platform> | <org>/<repo>[/<file>.cact]+ platform fine-tune, generate data and download models on cactuscompute.com+ download needle3 | needle3.safetensors | <platform> | model-<id> | <org>/<repo>[/<file>.cact]fetch fetch the engine library for this platformplayground serve the browser playground@@ -31,6 +32,8 @@ def _download_target(spec):if "/" in spec:return "hub", spec+ if spec.startswith("model-"):+ return "hosted", specif spec in fetch.PLATFORMS:return "platform", specname = spec[:-5] if spec.endswith(".cact") else spec@@ -41,7 +44,8 @@ def _download_target(spec):raise SystemExit(f"unknown download {spec!r}: pass needle3 (base weights), needle3.safetensors "f"(the checkpoint to fine-tune), a platform ("- + ", ".join(fetch.PLATFORMS) + "), or <org>/<repo>[/<file>.cact]")+ + ", ".join(fetch.PLATFORMS) + "), model-<id> from cactuscompute.com, "+ "or <org>/<repo>[/<file>.cact]")_ABSL_LOG_START = re.compile(rb"^[EIWF]\d{4} \d\d:\d\d:\d\d")@@ -200,12 +204,50 @@ def main():p.add_argument("spec", type=str,help="needle3 (base weights), needle3.safetensors (the checkpoint to ""fine-tune), a platform "- "folder (e.g. macos-arm64), or a Hugging Face spec: "+ "folder (e.g. macos-arm64), a fine-tuned model-<id> from "+ "cactuscompute.com, or a Hugging Face spec: ""<org>/<repo>/<file>.cact, or <org>/<repo> if it holds one archive")p.add_argument("--out", type=str, default=".", help="Directory to place the files")+ p.add_argument("--depth", type=int, default=None,+ help="One size of a fine-tuned model-<id> (default: every size)")p.add_argument("--generation", type=int, choices=[2, 3], default=3,help="Engine generation when downloading a platform build (default: 3)")+ p = sub.add_parser("platform")+ verbs = p.add_subparsers(dest="verb")+ v = verbs.add_parser("finetune")+ v.add_argument("train", type=str, help="Training .jsonl (a path, or a file-<id> already uploaded)")+ v.add_argument("validation", type=str, help="Validation .jsonl or file-<id>")+ v.add_argument("test", type=str, help="Test .jsonl or file-<id>")+ v.add_argument("--max-depth", type=int, default=None,+ help="Largest size to train, 2 up to the base depth (default: the base depth)")+ v.add_argument("--suffix", type=str, default=None, help="Name for the job and its models")+ v.add_argument("--out", type=str, default=".", help="Directory for the downloaded .cact files")+ v.add_argument("--depth", type=int, default=None,+ help="Download one size only (default: every size)")+ v.add_argument("--no-wait", action="store_true",+ help="Submit and print the job id without polling or downloading")+ v = verbs.add_parser("generate")+ v.add_argument("--tools", type=str, required=True, help="Tool definitions .json, flat or OpenAI form")+ v.add_argument("--examples", type=int, default=1000, help="Examples to generate, 100 to 10,000")+ v.add_argument("--description", type=str, default=None, help="What the product does")+ v.add_argument("--message", action="append", default=None,+ help="An example user message; repeat for several")+ v.add_argument("--suffix", type=str, default=None, help="Name for the job and its files")+ v.add_argument("--out", type=str, default=".", help="Directory for the generated .jsonl files")+ v.add_argument("--no-wait", action="store_true",+ help="Submit and print the job id without polling or downloading")+ v = verbs.add_parser("jobs")+ v.add_argument("job_id", type=str, nargs="?", default=None, help="One job to show in full")+ v.add_argument("--wait", action="store_true", help="Poll that job until it ends")+ v.add_argument("--out", type=str, default=None,+ help="Download what the finished job produced into this directory")+ v.add_argument("--depth", type=int, default=None, help="One size of a fine-tuned model (default: every size)")+ v = verbs.add_parser("models")+ v.add_argument("model_id", type=str, nargs="?", default=None, help="One model to show with its sizes")+ v = verbs.add_parser("files")+ verbs.add_parser("billing")+p = sub.add_parser("fetch")p.add_argument("--out", type=str, default=None,help="Directory to place the engine (default: the cache)")@@ -264,6 +306,11 @@ def main():path = fetch.fetch_weights(target, args.out)print(f" {'weights':<9} {path} {os.path.getsize(path) / 1e6:.2f} MB")print(f" {'next':<9} needle.Needle(weights={path!r}, tools=[...])")+ elif kind == "hosted":+ from .platform import Platform+ for path in Platform().download(target, args.out, depth=args.depth):+ print(f" {'weights':<9} {path} {os.path.getsize(path) / 1e6:.2f} MB")+ print(f" {'next':<9} needle.Needle(weights=<path>, tools=[...], auto_date=False)")elif kind == "checkpoint":generation = 2 if target.startswith("needle2") else 3path = fetch.fetch_checkpoint(target, os.path.join(args.out, fetch.CHECKPOINT_PREFIX),@@ -297,6 +344,9 @@ def main():print(f" {'engine':<9} {path}")print(f" {'deploy':<9} copy to ~/.cache/cactus-needle/v{generation}/{version}/ "f"on the device, or point NEEDLE{generation}_LIB_PATH at the file")+ elif args.command == "platform":+ from .platform import main as platform_main+ platform_main(args)elif args.command == "playground":from .playground.server import main as playground_mainplayground_main(args)diff --git a/needle/model/finetune.py b/needle/model/finetune.pyindex bffba09..ee64b24 100644--- a/needle/model/finetune.py+++ b/needle/model/finetune.py@@ -224,8 +224,35 @@ def _encode(tokenizer, example, max_len):return ids + [PAD_ID] * pad, mask + [0.0] * pad-def fit_max_len(path, tokenizer, cap):- longest = 0+def from_chat(example):+ """A platform chat-format line (`messages` + OpenAI-form `tools`) as a local example.++ Handles one user turn answered by one assistant turn, with an optional system+ turn in front; returns None for anything else.+ """+ messages = example.get("messages")+ if not isinstance(messages, list) or "query" in example:+ return None+ turns = [m for m in messages if m.get("role") != "system"]+ if len(turns) != 2 or turns[0].get("role") != "user" or turns[1].get("role") != "assistant":+ return None+ answers = []+ for call in turns[1].get("tool_calls") or []:+ function = call.get("function") or call+ arguments = function.get("arguments", {})+ if isinstance(arguments, str):+ arguments = json.loads(arguments) if arguments.strip() else {}+ answers.append({"name": function.get("name"), "arguments": arguments})+ tools = [t.get("function", t) if isinstance(t, dict) else t for t in example.get("tools") or []]+ out = {"query": turns[0].get("content") or "", "tools": tools, "answers": answers}+ system = [m.get("content") for m in messages if m.get("role") == "system"]+ if system and system[0]:+ out["system"] = system[0]+ return out+++def read_examples(path, report=False):+ skipped = 0with open(path) as handle:for line in handle:line = line.strip()@@ -233,9 +260,20 @@ def fit_max_len(path, tokenizer, cap):continueexample = json.loads(line)if "query" not in example:+ example = from_chat(example)+ if example is None:+ skipped += 1continue- prompt, target = render_example(example)- longest = max(longest, len(tokenizer.encode(prompt)) + len(tokenizer.encode(target)) + 2)+ yield example+ if skipped and report:+ print(f" skipped {skipped} line(s) that are neither query/answers nor single-turn chat format")+++def fit_max_len(path, tokenizer, cap):+ longest = 0+ for example in read_examples(path):+ prompt, target = render_example(example)+ longest = max(longest, len(tokenizer.encode(prompt)) + len(tokenizer.encode(target)) + 2)bucket = 128while bucket < min(longest, cap):bucket *= 2@@ -244,17 +282,10 @@ def fit_max_len(path, tokenizer, cap):def load_jsonl(path, tokenizer, max_len):seqs, masks = [], []- with open(path) as handle:- for line in handle:- line = line.strip()- if not line:- continue- example = json.loads(line)- if "query" not in example:- continue- ids, mask = _encode(tokenizer, example, max_len)- seqs.append(ids)- masks.append(mask)+ for example in read_examples(path, report=True):+ ids, mask = _encode(tokenizer, example, max_len)+ seqs.append(ids)+ masks.append(mask)return np.array(seqs, np.int32), np.array(masks, np.float32)@@ -427,7 +458,7 @@ def finetune_local(args, progress=None):})print(f" {'adapter':<9} {out}")print(f" {'next':<9} needle build {base_path} --lora {out}")- print(f" {'note':<9} confidence reports None with tuned weights; the head is not tuned")+ print(f" {'note':<9} local tuning leaves the confidence head untrained; needle build drops it and confidence reports None")def build_main(args):@@ -435,7 +466,7 @@ def build_main(args):import jax.numpy as jnpfrom ..agent import fetchfrom .run import load_checkpoint- from .architecture import effective_kv_window+ from .architecture import ConfidenceHead, effective_kv_windowfrom .export import read_layers, read_tokenizer_blob, write_exportfrom .quantize import WEIGHT_BITS@@ -468,6 +499,9 @@ def build_main(args):for key, v in adapter["lora"].items()}params = merge_lora(params, lora, adapter["scale"])print(f" {'merged':<9} {len(lora)} weight groups {args.lora}")+ if ConfidenceHead.key in params:+ params = {k: v for k, v in params.items() if k != ConfidenceHead.key}+ print(f" {'dropped':<9} the confidence head; it is not trained locally, so confidence reports None")params, config = rung(params, config, layers)print(f" {'depth':<9} {config.num_layers} layers")out = out or (os.path.splitext(os.path.basename(checkpoint))[0] + ".cact")diff --git a/needle/platform.py b/needle/platform.pynew file mode 100644index 0000000..6aa01f3--- /dev/null+++ b/needle/platform.py@@ -0,0 +1,429 @@+from __future__ import annotations++import json+import os+import shutil+import ssl+import time+import urllib.error+import urllib.parse+import urllib.request++BASE_URL = "https://cactuscompute.com/v1"+KEYS_URL = "https://cactuscompute.com/dashboard/api-keys"+JOBS_URL = "https://cactuscompute.com/dashboard/jobs"+BASE_MODEL = "needle-3"+TERMINAL = ("succeeded", "failed", "cancelled")+++class PlatformError(RuntimeError):+ def __init__(self, code, message, status=0, param=None, url=None, retry_after=0):+ super().__init__(message)+ self.code = code+ self.status = status+ self.param = param+ self.url = url+ self.retry_after = retry_after++ def __str__(self):+ text = f"{self.code}: {self.args[0]}"+ if self.param:+ text += f" (param: {self.param})"+ if self.url:+ text += f" -> {self.url}"+ return text+++class _NoRedirect(urllib.request.HTTPRedirectHandler):+ def redirect_request(self, req, fp, code, msg, headers, newurl):+ return None+++def _ssl_context():+ try:+ import certifi+ return ssl.create_default_context(cafile=certifi.where())+ except ImportError:+ return ssl.create_default_context()+++def flat_tools(tools):+ """OpenAI-form tool definitions to the flat form `needle.Needle(tools=...)` takes."""+ return [t["function"] if t.get("type") == "function" and "function" in t else t+ for t in tools]+++def openai_tools(tools):+ """Flat tool definitions to the OpenAI form the platform takes."""+ return [t if t.get("type") == "function" else {"type": "function", "function": t}+ for t in tools]+++class Platform:+ """Client for the hosted fine-tuning API at cactuscompute.com/v1.++ The key comes from `api_key` or `NEEDLE_API_KEY`; create one in the console+ under API Keys. Every method raises `PlatformError` with the API's stable+ error `code` on failure.+ """++ def __init__(self, api_key=None, base_url=BASE_URL, timeout=60, retries=3):+ self.api_key = api_key or os.environ.get("NEEDLE_API_KEY")+ self.base_url = base_url.rstrip("/")+ self.timeout = timeout+ self.retries = retries+ https = urllib.request.HTTPSHandler(context=_ssl_context())+ self._opener = urllib.request.build_opener(_NoRedirect, https)+ self._follower = urllib.request.build_opener(https)++ def _send(self, method, url, body=None, headers=None, auth=True):+ headers = dict(headers or {})+ if auth and self.api_key:+ headers["Authorization"] = f"Bearer {self.api_key}"+ data = None+ if body is not None and not isinstance(body, (bytes, bytearray)):+ data = json.dumps(body).encode("utf-8")+ headers["Content-Type"] = "application/json"+ elif body is not None:+ data = bytes(body)+ request = urllib.request.Request(url, data=data, method=method, headers=headers)+ for attempt in range(self.retries + 1):+ try:+ with self._opener.open(request, timeout=self.timeout) as response:+ return response.status, dict(response.headers), response.read()+ except urllib.error.HTTPError as error:+ payload = error.read()+ if error.code in (301, 302, 303, 307, 308):+ return error.code, dict(error.headers), payload+ code, message, param, link = _parse_error(payload, error.code)+ if error.code == 401 and not self.api_key:+ message, link = "set NEEDLE_API_KEY to a key from the console", link or KEYS_URL+ retry_after = _seconds(error.headers.get("Retry-After"))+ if error.code == 429 and code == "rate_limited" and attempt < self.retries:+ time.sleep(max(retry_after, 1))+ continue+ raise PlatformError(code, message, error.code, param, link, retry_after) from None+ except urllib.error.URLError as error:+ raise PlatformError("network_error", str(error.reason)) from None+ raise PlatformError("rate_limited", "the API kept answering 429", 429)++ def _call(self, method, path, body=None, params=None):+ url = f"{self.base_url}/{path}"+ if params:+ url += "?" + urllib.parse.urlencode({k: v for k, v in params.items() if v is not None})+ status, headers, payload = self._send(method, url, body)+ return json.loads(payload.decode("utf-8")) if payload else {}++ def _signed_link(self, path):+ status, headers, _ = self._send("GET", f"{self.base_url}/{path}")+ link = headers.get("Location") or headers.get("location")+ if status not in (301, 302, 303, 307, 308) or not link:+ raise PlatformError("unexpected_response", f"expected a redirect from {path}, got {status}", status)+ return link++ def _fetch_to(self, link, dest):+ os.makedirs(os.path.dirname(os.path.abspath(dest)), exist_ok=True)+ request = urllib.request.Request(link, method="GET")+ with self._follower.open(request, timeout=self.timeout) as response, open(dest, "wb") as out:+ shutil.copyfileobj(response, out)+ return dest++ def plans(self):+ return self._call("GET", "plans")++ def billing(self):+ return self._call("GET", "billing")++ def tool_schemas(self, description):+ return self._call("POST", "tool_schemas", {"description": description})["tools"]++ def files(self, search=None, limit=100):+ return self._call("GET", "files", params={"search": search, "limit": limit})["data"]++ def file(self, file_id):+ return self._call("GET", f"files/{file_id}")++ def delete_file(self, file_id):+ return self._call("DELETE", f"files/{file_id}")++ def upload(self, path, name=None):+ """Upload a .jsonl file in the platform's three steps; returns the file record."""+ size = os.path.getsize(path)+ reservation = self._call("POST", "files", {"name": name or os.path.basename(path), "bytes": size})+ with open(path, "rb") as handle:+ self._send("PUT", reservation["url"], handle.read(),+ {"Content-Type": "application/octet-stream"}, auth=False)+ return self._call("POST", f"files/{reservation['id']}/complete")++ def download_file(self, file_id, dest):+ return self._fetch_to(self._signed_link(f"files/{file_id}/content"), dest)++ def generate(self, tools, examples, description=None, messages=None, suffix=None):+ body = {"tools": openai_tools(tools), "examples": int(examples)}+ if description:+ body["description"] = description+ if messages:+ body["messages"] = list(messages)+ if suffix:+ body["suffix"] = suffix+ return self._call("POST", "generations", body)++ def generation(self, job_id):+ return self._call("GET", f"generations/{job_id}")++ def generations(self, limit=20):+ return self._call("GET", "generations", params={"limit": limit})["data"]++ def cancel_generation(self, job_id):+ return self._call("POST", f"generations/{job_id}/cancel")++ def _file_ids(self, items):+ ids = []+ for item in items if isinstance(items, (list, tuple)) else [items]:+ if isinstance(item, dict):+ ids.append(item["id"])+ elif isinstance(item, str) and item.startswith("file-"):+ ids.append(item)+ elif isinstance(item, str) and os.path.exists(item):+ ids.append(self.upload(item)["id"])+ else:+ raise PlatformError("invalid_request", f"{item!r} is neither a file id nor a path", 400)+ return ids++ def finetune(self, training_files, validation_files, test_files, max_depth=None, suffix=None):+ """Start a fine-tune. Each group takes paths (uploaded first), file ids, or file records."""+ if max_depth is None:+ max_depth = self.model(BASE_MODEL)["depth"]+ body = {+ "model": BASE_MODEL,+ "training_files": self._file_ids(training_files),+ "validation_files": self._file_ids(validation_files),+ "test_files": self._file_ids(test_files),+ "max_depth": int(max_depth),+ }+ if suffix:+ body["suffix"] = suffix+ return self._call("POST", "fine_tuning/jobs", body)++ def job(self, job_id):+ return self._call("GET", f"fine_tuning/jobs/{job_id}")++ def jobs(self, limit=20):+ return self._call("GET", "fine_tuning/jobs", params={"limit": limit})["data"]++ def cancel(self, job_id):+ return self._call("POST", f"fine_tuning/jobs/{job_id}/cancel")++ def wait(self, job, poll=10, on_update=None):+ """Poll a fine-tune or generation until it ends; returns the record, raises if it failed."""+ job_id = job["id"] if isinstance(job, dict) else job+ fetch = self.generation if (isinstance(job, dict) and job.get("object") == "generation") else self.job+ while True:+ record = fetch(job_id)+ if on_update:+ on_update(record)+ status = record.get("status")+ if status in TERMINAL:+ break+ time.sleep(poll)+ if status == "succeeded":+ return record+ error = record.get("error") or {}+ raise PlatformError(error.get("code") or status, error.get("message") or f"job {job_id} {status}")++ def models(self, search=None, limit=100):+ return self._call("GET", "models", params={"search": search, "limit": limit})["data"]++ def model(self, model_id):+ return self._call("GET", f"models/{model_id}")++ def delete_model(self, model_id):+ return self._call("DELETE", f"models/{model_id}")++ def download(self, model_id, out=".", depth=None):+ """Download a model's .cact files into `out`, every size or one `depth`; returns the paths."""+ record = self.model(model_id)+ stem = record.get("name") or record["id"]+ variants = record.get("variants") or [{"id": record["id"], "depth": record.get("depth")}]+ if depth is not None:+ variants = [v for v in variants if v.get("depth") == int(depth)]+ if not variants:+ raise PlatformError("not_found", f"{model_id} has no {depth}-layer size", 404)+ paths = []+ for variant in variants:+ dest = os.path.join(out, f"{stem}-{variant['depth']}L.cact")+ paths.append(self._fetch_to(self._signed_link(f"models/{variant['id']}/content"), dest))+ return paths+++def _seconds(value):+ try:+ return int(float(value or 0))+ except ValueError:+ return 0+++def _parse_error(payload, status):+ try:+ error = json.loads(payload.decode("utf-8")).get("error") or {}+ except (ValueError, AttributeError):+ error = {}+ return (error.get("code") or f"http_{status}", error.get("message") or f"HTTP {status}",+ error.get("param"), error.get("url"))+++def _print(label, value):+ print(f" {label:<9} {value}")+++def _print_usage(client):+ billing = client.billing()+ limits, usage = billing.get("limits") or {}, billing.get("usage") or {}+ _print("plan", f"{billing.get('plan')}: {usage.get('jobs', 0)}/{limits.get('jobs', '?')} fine-tunes, "+ f"{usage.get('examples', 0)}/{limits.get('examples', '?')} generated examples this period")+++def _print_job(job):+ line = f"{job['id']} {job.get('status')}"+ if job.get("name"):+ line += f" {job['name']}"+ if job.get("max_depth"):+ line += f" max_depth {job['max_depth']}"+ if job.get("error"):+ line += f" error {job['error'].get('code')}"+ print(" " + line)+++def _print_evaluations(job):+ rows = job.get("evaluations") or []+ if not rows:+ return+ print(f" {'depth':>5} {'validation':>12} {'test':>12}")+ for row in sorted(rows, key=lambda r: -r["depth"]):+ v, t = row["validation"], row["test"]+ print(f" {row['depth']:>4}L {v['correct']:>5}/{v['total']:<6} {t['correct']:>5}/{t['total']:<6}")+++def _progress(kind):+ seen = {}++ def show(record):+ key = (record.get("status"), record.get("completed_steps"))+ if key in seen:+ return+ seen[key] = True+ status = record.get("status")+ if status == "running" and record.get("total_steps"):+ print(f" {kind:<9} running, step {record.get('completed_steps', 0)}/{record['total_steps']}", flush=True)+ else:+ print(f" {kind:<9} {status}", flush=True)+ return show+++def _cmd_finetune(client, args):+ _print_usage(client)+ groups = {}+ for label, item in (("train", args.train), ("validation", args.validation), ("test", args.test)):+ if item.startswith("file-"):+ groups[label] = item+ else:+ record = client.upload(item)+ _print("upload", f"{record['id']} {record['filename']} {record['bytes'] / 1e6:.2f} MB")+ groups[label] = record["id"]+ job = client.finetune([groups["train"]], [groups["validation"]], [groups["test"]],+ max_depth=args.max_depth, suffix=args.suffix)+ _print("job", f"{job['id']} max_depth {job.get('max_depth')}")+ _print("watch", f"{JOBS_URL}/{job['id']} (Ctrl-C leaves the job running; needle platform jobs {job['id']} --wait --out DIR resumes)")+ if args.no_wait:+ return+ job = client.wait(job, on_update=_progress("job"))+ _print_evaluations(job)+ paths = client.download(job["fine_tuned_model"], args.out, depth=args.depth)+ for path in paths:+ _print("weights", f"{path} {os.path.getsize(path) / 1e6:.2f} MB")+ _print("next", f"needle.Needle(weights={paths[-1]!r}, tools=[...], auto_date=False)")+++def _cmd_generate(client, args):+ _print_usage(client)+ with open(args.tools) as handle:+ tools = json.load(handle)+ job = client.generate(tools, args.examples, description=args.description,+ messages=args.message, suffix=args.suffix)+ _print("job", f"{job['id']} {args.examples} examples")+ _print("watch", f"{JOBS_URL}/{job['id']} (Ctrl-C leaves the job running; needle platform jobs {job['id']} --wait --out DIR resumes)")+ if args.no_wait:+ return+ job = client.wait(job, on_update=_progress("job"))+ for record in job.get("files") or []:+ dest = client.download_file(record["id"], os.path.join(args.out, record["filename"]))+ _print(record["group"], f"{dest} {os.path.getsize(dest) / 1e6:.2f} MB")+ _print("next", "needle platform finetune <train.jsonl> <validation.jsonl> <test.jsonl>")+++def _cmd_jobs(client, args):+ if not args.job_id:+ for job in client.jobs():+ _print_job(job)+ return+ try:+ job = client.job(args.job_id)+ except PlatformError as error:+ if error.status != 404:+ raise+ job = client.generation(args.job_id)+ if args.wait and job.get("status") not in TERMINAL:+ job = client.wait(job, on_update=_progress("job"))+ _print_job(job)+ _print_evaluations(job)+ if job.get("fine_tuned_model"):+ _print("model", job["fine_tuned_model"])+ if args.out:+ for path in client.download(job["fine_tuned_model"], args.out, depth=args.depth):+ _print("weights", f"{path} {os.path.getsize(path) / 1e6:.2f} MB")+ return+ for record in job.get("files") or []:+ if args.out and job.get("object") == "generation" and job.get("status") == "succeeded":+ dest = client.download_file(record["id"], os.path.join(args.out, record["filename"]))+ _print(record.get("group", "file"), f"{dest} {os.path.getsize(dest) / 1e6:.2f} MB")+ else:+ _print(record.get("group", "file"), f"{record['id']} {record['filename']}")+++def _cmd_models(client, args):+ if args.model_id:+ record = client.model(args.model_id)+ _print("model", f"{record['id']} {record.get('name') or ''} {record.get('depth')} layers")+ for variant in record.get("variants") or []:+ size = f" {variant['bytes'] / 1e6:.2f} MB" if variant.get("bytes") else ""+ _print(f"{variant['depth']}L", f"{variant['id']}{size}")+ _print("next", f"needle download {record['id']} [--depth N]")+ return+ for record in client.models():+ _print("model", f"{record['id']} {record.get('name') or ''} {record.get('depth')} layers")+++def _cmd_files(client, args):+ for record in client.files():+ _print("file", f"{record['id']} {record['filename']} {record['bytes'] / 1e6:.2f} MB")+++def _cmd_billing(client, args):+ billing = client.billing()+ _print("plan", billing.get("plan"))+ for key, limit in (billing.get("limits") or {}).items():+ _print(key, f"{(billing.get('usage') or {}).get(key, 0)} / {limit}")+ if billing.get("period_end"):+ _print("resets", time.strftime("%Y-%m-%d", time.gmtime(billing["period_end"])))+++def main(args):+ client = Platform()+ verbs = {"finetune": _cmd_finetune, "generate": _cmd_generate, "jobs": _cmd_jobs,+ "models": _cmd_models, "files": _cmd_files, "billing": _cmd_billing}+ if not args.verb:+ raise SystemExit("needle platform finetune | generate | jobs | models | files | billing")+ try:+ verbs[args.verb](client, args)+ except PlatformError as error:+ raise SystemExit(f"platform error {error}")diff --git a/tests/conftest.py b/tests/conftest.pyindex d3d7c19..9d2bac4 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -98,3 +98,37 @@ def published_base(tiny_base_archive, monkeypatch):monkeypatch.setattr(fetch, "fetch_weights", fake_fetch)return calls+++def fake_cact():+ return _fake_cact+++def _fake_cact(path, heads):+ import struct++ header_fmt, record_fmt = "<48If", "<BBHIIIIQQII"+ body = bytearray()+ records = []++ def add(dtype, shape, blob):+ records.append((dtype, len(shape), 0, *(list(shape) + [0] * (4 - len(shape))), len(body), len(blob), 0, 0))+ body.extend(blob)++ add(1, (4,), struct.pack("<4e", 0, 0, 0, 0))+ if heads:+ add(1, (len(heads),), struct.pack(f"<{len(heads)}e", *heads))+ for _ in heads:+ for _ in range(6):+ add(1, (2,), struct.pack("<2e", 0, 0))+ add(4, (3,), b"tok")+ head = struct.calcsize(header_fmt) + struct.calcsize(record_fmt) * len(records)+ fields = [0x05E12A84, len(records), 0] + [0] * 45+ out = bytearray(struct.pack(header_fmt, *fields, 0.0))+ for rec in records:+ rec = list(rec)+ rec[7] += head+ out += struct.pack(record_fmt, *rec)+ out += body+ path.write_bytes(bytes(out))diff --git a/tests/test_platform.py b/tests/test_platform.pynew file mode 100644index 0000000..aa9690e--- /dev/null+++ b/tests/test_platform.py@@ -0,0 +1,223 @@+import json+import threading+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer++import pytest+++class FakePlatform(BaseHTTPRequestHandler):+ state = None++ def log_message(self, *args):+ pass++ def _body(self):+ length = int(self.headers.get("Content-Length") or 0)+ return self.rfile.read(length) if length else b""++ def _json(self, status, payload, headers=None):+ data = json.dumps(payload).encode()+ self.send_response(status)+ self.send_header("Content-Type", "application/json")+ for key, value in (headers or {}).items():+ self.send_header(key, value)+ self.send_header("Content-Length", str(len(data)))+ self.end_headers()+ self.wfile.write(data)++ def _error(self, status, code, headers=None, **extra):+ self._json(status, {"error": {"message": code, "type": "platform_error", "code": code, **extra}}, headers)++ def do_GET(self):+ s = self.state+ s["requests"].append(("GET", self.path, self.headers.get("Authorization")))+ if self.path.startswith("/blob/"):+ self.send_response(200)+ self.send_header("Content-Length", "5")+ self.end_headers()+ self.wfile.write(b"CACT!")+ elif self.path.startswith("/v1/") and self.headers.get("Authorization") != "Bearer needle_ft_test":+ self._error(401, "unauthorized", url="https://cactuscompute.com/dashboard/api-keys")+ elif self.path == "/v1/billing":+ if s["rate_limit_once"]:+ s["rate_limit_once"] = False+ self._error(429, "rate_limited", {"Retry-After": "1"})+ else:+ self._json(200, {"plan": "standard", "limits": {"jobs": 10}, "usage": {"jobs": 1}})+ elif self.path == "/v1/models/needle-3":+ self._json(200, {"id": "needle-3", "object": "model", "owned_by": "cactus", "depth": 20, "variants": []})+ elif self.path == "/v1/models/model-1":+ self._json(200, {"id": "model-1", "object": "model", "owned_by": "me", "name": "smart-home", "depth": 8,+ "variants": [{"id": "model-1-8", "depth": 8, "is_full": True, "bytes": 5},+ {"id": "model-1-4", "depth": 4, "is_full": False, "bytes": 5}]})+ elif self.path.startswith("/v1/models/model-1-") and self.path.endswith("/content"):+ self.send_response(307)+ self.send_header("Location", f"http://{self.headers['Host']}/blob/{self.path.split('/')[3]}")+ self.send_header("Content-Length", "0")+ self.end_headers()+ elif self.path == "/v1/fine_tuning/jobs/ftjob-1":+ s["polls"] += 1+ status = "running" if s["polls"] < 2 else "succeeded"+ self._json(200, {"id": "ftjob-1", "object": "fine_tuning.job", "status": status, "created_at": 0,+ "completed_steps": min(s["polls"], 5), "total_steps": 5, "max_depth": 8,+ "fine_tuned_model": "model-1" if status == "succeeded" else None,+ "evaluations": [{"depth": 8, "validation": {"correct": 9, "total": 10, "excluded": 0},+ "test": {"correct": 8, "total": 10, "excluded": 0}}] if status == "succeeded" else []})+ else:+ self._error(404, "not_found")++ def do_POST(self):+ s = self.state+ body = self._body()+ s["requests"].append(("POST", self.path, self.headers.get("Authorization")))+ if self.path == "/v1/files":+ payload = json.loads(body)+ s["reserved"] = payload+ self._json(201, {"id": "0123", "url": f"http://{self.headers['Host']}/upload/0123", "expires_at": "2026-01-01T00:00:00Z"})+ elif self.path == "/v1/files/0123/complete":+ self._json(201, {"id": "file-0123", "object": "file", "filename": s["reserved"]["name"],+ "bytes": s["reserved"]["bytes"], "created_at": 0, "purpose": "fine-tune"})+ elif self.path == "/v1/fine_tuning/jobs":+ s["submitted"] = json.loads(body)+ if s["submitted"].get("suffix") == "broke":+ self._error(402, "quota_exceeded", param="jobs", url="https://cactuscompute.com/dashboard/usage")+ else:+ self._json(201, {"id": "ftjob-1", "object": "fine_tuning.job", "status": "queued", "created_at": 0,+ "max_depth": s["submitted"]["max_depth"]})+ else:+ self._error(404, "not_found")++ def do_PUT(self):+ s = self.state+ body = self._body()+ s["requests"].append(("PUT", self.path, self.headers.get("Authorization")))+ s["uploaded"] = body+ self.send_response(200)+ self.send_header("Content-Length", "0")+ self.end_headers()+++def server():+ state = {"requests": [], "polls": 0, "rate_limit_once": False}+ FakePlatform.state = state+ httpd = ThreadingHTTPServer(("127.0.0.1", 0), FakePlatform)+ thread = threading.Thread(target=httpd.serve_forever, daemon=True)+ thread.start()+ try:+ yield f"http://127.0.0.1:{httpd.server_address[1]}/v1", state+ finally:+ httpd.shutdown()+++def test_missing_key_points_at_the_console(server, monkeypatch):+ from needle.platform import Platform, PlatformError++ base, state = server+ monkeypatch.delenv("NEEDLE_API_KEY", raising=False)+ with pytest.raises(PlatformError) as caught:+ Platform(base_url=base).billing()+ assert caught.value.code == "unauthorized"+ assert "NEEDLE_API_KEY" in str(caught.value)+ assert "dashboard/api-keys" in caught.value.url+ assert state["requests"][-1][2] is None+++def test_finetune_uploads_submits_waits_and_downloads(server, tmp_path):+ from needle.platform import Platform++ base, state = server+ train = tmp_path / "train.jsonl"+ train.write_text('{"messages":[],"tools":[]}\n')+ client = Platform(api_key="needle_ft_test", base_url=base)++ job = client.finetune([str(train)], ["file-val"], ["file-test"], suffix="smart-home")+ assert job["id"] == "ftjob-1"+ assert state["reserved"] == {"name": "train.jsonl", "bytes": train.stat().st_size}+ assert state["uploaded"] == train.read_bytes()+ put = next(r for r in state["requests"] if r[0] == "PUT")+ assert put[1] == "/upload/0123" and put[2] is None+ assert state["submitted"] == {"model": "needle-3", "training_files": ["file-0123"],+ "validation_files": ["file-val"], "test_files": ["file-test"],+ "max_depth": 20, "suffix": "smart-home"}++ seen = []+ done = client.wait(job, poll=0, on_update=lambda r: seen.append(r["status"]))+ assert seen == ["running", "succeeded"]+ assert done["fine_tuned_model"] == "model-1"++ paths = client.download("model-1", str(tmp_path / "out"))+ assert sorted(p.split("/")[-1] for p in paths) == ["smart-home-4L.cact", "smart-home-8L.cact"]+ assert all(open(p, "rb").read() == b"CACT!" for p in paths)+ blob = next(r for r in state["requests"] if r[1].startswith("/blob/"))+ assert blob[2] is None++ only = client.download("model-1", str(tmp_path / "one"), depth=4)+ assert [p.split("/")[-1] for p in only] == ["smart-home-4L.cact"]+++def test_errors_carry_code_param_and_url(server, tmp_path):+ from needle.platform import Platform, PlatformError++ base, state = server+ client = Platform(api_key="needle_ft_test", base_url=base)+ with pytest.raises(PlatformError) as caught:+ client.finetune(["file-a"], ["file-b"], ["file-c"], max_depth=8, suffix="broke")+ assert (caught.value.code, caught.value.status, caught.value.param) == ("quota_exceeded", 402, "jobs")+ assert "dashboard/usage" in str(caught.value)++ state["rate_limit_once"] = True+ assert client.billing()["plan"] == "standard"+++def test_download_target_routes_hosted_models():+ from needle.cli import _download_target++ assert _download_target("model-abc") == ("hosted", "model-abc")+++def test_tool_form_conversions():+ from needle.platform import flat_tools, openai_tools++ flat = [{"name": "set_lights", "parameters": {"type": "object", "properties": {}}}]+ wrapped = openai_tools(flat)+ assert wrapped == [{"type": "function", "function": flat[0]}]+ assert flat_tools(wrapped) == flat+ assert openai_tools(wrapped) == wrapped+++def test_local_finetune_reads_chat_format(tmp_path):+ from needle.model.finetune import from_chat, read_examples++ chat = {"messages": [{"role": "system", "content": "date: 2026-09-19"},+ {"role": "user", "content": "turn the kitchen lights off"},+ {"role": "assistant", "content": "", "tool_calls": [+ {"id": "call_1", "type": "function",+ "function": {"name": "set_lights", "arguments": "{\"room\":\"kitchen\",\"state\":\"off\"}"}}]}],+ "tools": [{"type": "function", "function": {"name": "set_lights", "parameters": {"type": "object"}}}]}+ example = from_chat(chat)+ assert example == {"query": "turn the kitchen lights off", "system": "date: 2026-09-19",+ "tools": [{"name": "set_lights", "parameters": {"type": "object"}}],+ "answers": [{"name": "set_lights", "arguments": {"room": "kitchen", "state": "off"}}]}+ silent = {"messages": [{"role": "user", "content": "hi"}, {"role": "assistant", "content": "Hello."}], "tools": []}+ assert from_chat(silent)["answers"] == []+ multi = {"messages": [{"role": "user", "content": "a"}, {"role": "assistant", "content": "b"},+ {"role": "user", "content": "c"}, {"role": "assistant", "content": "d"}], "tools": []}+ assert from_chat(multi) is None++ path = tmp_path / "mixed.jsonl"+ path.write_text("\n".join(json.dumps(x) for x in (chat, silent, multi,+ {"query": "q", "tools": [], "answers": []})) + "\n")+ rows = list(read_examples(str(path)))+ assert [r["query"] for r in rows] == ["turn the kitchen lights off", "hi", "q"]+++def test_confidence_head_detected_from_manifest(tmp_path, fake_cact):+ import needle++ fake_cact(tmp_path / "platform.cact", [1, 2])+ fake_cact(tmp_path / "local.cact", [1])+ fake_cact(tmp_path / "bare.cact", [])+ assert needle._confidence_head_present(str(tmp_path / "platform.cact")) is True+ assert needle._confidence_head_present(str(tmp_path / "local.cact")) is False+ assert needle._confidence_head_present(str(tmp_path / "bare.cact")) is Falsediff --git a/tests/test_weights.py b/tests/test_weights.pyindex d90ac89..6f15696 100644--- a/tests/test_weights.py+++ b/tests/test_weights.py@@ -277,3 +277,17 @@ def test_tool_index_path_agrees_between_the_base_and_tuned_paths(engine, tuned,if isinstance(call, tuple) and call[0] == "worker_start"]assert startedassert base._tool_index_path.decode("utf-8") == os.fspath(index)+++def test_platform_tuned_archive_keeps_confidence(engine, tmp_path, fake_cact):+ import needle++ fake_cact(tmp_path / "platform.cact", [1, 2])+ fake_cact(tmp_path / "local.cact", [1])+ with warnings.catch_warnings():+ warnings.simplefilter("error")+ calibrated = needle.Needle(tools="[]", weights=str(tmp_path / "platform.cact"))+ assert calibrated.complete("hello").get("confidence") is not None+ with pytest.warns(UserWarning, match="confidence as None"):+ uncalibrated = needle.Needle(tools="[]", weights=str(tmp_path / "local.cact"))+ assert uncalibrated.complete("hello").get("confidence", "kept") is None