Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Separate training dependencies from the runtime package (#97)
nimakalantaree221ce7c135
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yamlindex 7763826..f080eac 100644--- a/.github/workflows/release.yaml+++ b/.github/workflows/release.yaml@@ -40,7 +40,7 @@ jobs:- uses: actions/setup-python@v5with:python-version: "3.12"- - run: pip install -e ".[test]"+ - run: pip install -e ".[test,train]"- run: pytest -q -m "not slow"release:@@ -49,11 +49,11 @@ jobs:runs-on: ubuntu-latestpermissions:contents: write # pushes the vX.Y.Z release tag- id-token: write+ id-token: writesteps:- uses: actions/checkout@v4with:- fetch-depth: 0+ fetch-depth: 0- uses: actions/setup-python@v5with:python-version: "3.12"diff --git a/README.md b/README.mdindex c3c7f45..d407c57 100644--- a/README.md+++ b/README.md@@ -30,6 +30,13 @@ Each block carries its update rule. Here x̂ is the RMS-normalised flattening ofpip install cactus-needle```+The runtime package does not install the training stack. Add the `train` extra+when using fine-tuning or checkpoint export:++```sh+pip install "cactus-needle[train]"+```+Needle reads your tool descriptions to decide what to call and how to fill arguments, so describing them well is the whole game.**Simple**: decorate a function. The signature gives the argument types, the docstring is the tool description, and `run()` completes the loop: model picks the call, Needle executes your function, feeds the result back, and returns the final response with the executed tool results attached as `results`.@@ -119,13 +126,13 @@ Key options: `--epochs` (default 3), `--lora-rank` (16), `--lora-alpha` (32), `-Training is plain JAX and runs on any accelerator jax supports. On an NVIDIA machine install the CUDA build and the same command trains on the GPU:```sh-pip install "cactus-needle[gpu]"+pip install "cactus-needle[train,gpu]"```On Apple Silicon the `metal` extra trains on the GPU:```sh-pip install "cactus-needle[metal]"+pip install "cactus-needle[train,metal]"```**3. Build a tuned `.cact`.** Merge the adapter into the base and quantize. The base auto-downloads if absent.diff --git a/doc/finetuning.md b/doc/finetuning.mdindex 1e15d43..4129914 100644--- a/doc/finetuning.md+++ b/doc/finetuning.md@@ -4,6 +4,12 @@ Needle finetunes with LoRA adapters on the frozen base model: rank 16 on the fiv## Data format+Install the training dependencies before running the commands in this guide:++```sh+pip install "cactus-needle[train]"+```+One JSON object per line. `query` and `tools` describe the turn, `answers` lists the exact calls the model should emit, `reasoning` is one short line deriving each argument from its source span in the query. `reasoning` is optional but include it: the model produces the derivation before the call, and examples that show where each value comes from teach grounding, not just tool selection.```json@@ -46,13 +52,13 @@ The playground button labelled Finetune on these tools runs the same pipeline frTraining is plain JAX, so it runs on any accelerator jax supports. On an NVIDIA machine install the CUDA build and the same command trains on the GPU, nothing else changes:```sh-pip install "cactus-needle[gpu]"+pip install "cactus-needle[train,gpu]"```Apple GPUs train through the jax metal plugin, which does not work past jax 0.4.38, so the `metal` extra pins an older stack:```sh-pip install "cactus-needle[metal]"+pip install "cactus-needle[train,metal]"```Needle detects the Metal backend and adapts automatically: manual attention, no rematerialisation, unrolled layer stack, and the `ENABLE_PJRT_COMPATIBILITY` variable the plugin requires is set for you. Measured on an M5 Max: 0.71 seconds per step against 2.90 on CPU at the same shape, about 4 times faster, with a one time compile of about 23 seconds. Training runs in float32 on every backend.diff --git a/pyproject.toml b/pyproject.tomlindex 64853ff..3c33134 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -7,17 +7,23 @@ readme = "README.md"license = { text = "Apache-2.0" }dependencies = ["huggingface_hub",- "numpy",- "jax",- "jaxlib",- "flax>=0.10.2",- "optax",- "sentencepiece",][project.optional-dependencies]+train = [+ "numpy",+ "jax",+ "jaxlib",+ "flax>=0.10.2",+ "optax",+ "sentencepiece"+]gpu = ["jax[cuda12]"]-metal = ["jax==0.4.38", "jaxlib==0.4.38", "jax-metal", "flax==0.10.2", "optax==0.2.4"]+metal = [+ "jax==0.4.38",+ "jaxlib==0.4.38",+ "jax-metal"+]test = ["pytest", "pydantic"][project.scripts]diff --git a/requirements-train.txt b/requirements-train.txtnew file mode 100644index 0000000..3b6ac3f--- /dev/null+++ b/requirements-train.txt@@ -0,0 +1,9 @@+-r requirements.txt++# Training and checkpoint export dependencies+numpy+jax+jaxlib+flax>=0.10.2+optax+sentencepiece\ No newline at end of filediff --git a/requirements.txt b/requirements.txtindex 558faaf..72a6100 100644--- a/requirements.txt+++ b/requirements.txt@@ -1,7 +1,2 @@+# Runtime dependencieshuggingface_hub-numpy-jax-jaxlib-flax-optax-sentencepiecediff --git a/setup b/setupindex 27752b5..1c674b4 100755--- a/setup+++ b/setup@@ -20,11 +20,11 @@ fisource "$VENV_DIR/bin/activate"pip install --upgrade pip -q-pip install -e . -q+pip install -e ".[train]" -qif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi >/dev/null 2>&1; then- echo "NVIDIA GPU detected; installing jax[cuda12]..."- pip install -U "jax[cuda12]" -q+ echo "NVIDIA GPU detected; installing gpu extras..."+ pip install -e ".[gpu]" -qfineedle --helpdiff --git a/setup.bat b/setup.batindex 5310c17..788ca6d 100644--- a/setup.bat+++ b/setup.bat@@ -19,6 +19,6 @@ if not exist "%VENV_DIR%\Scripts\python.exe" %PYTHON_CMD% -m venv "%VENV_DIR%"call "%VENV_DIR%\Scripts\activate.bat"python -m pip install --upgrade pip -q-python -m pip install -e . -q+python -m pip install -e ".[train]" -qneedle --helpdiff --git a/tests/test_packaging.py b/tests/test_packaging.pynew file mode 100644index 0000000..7a3f1a7--- /dev/null+++ b/tests/test_packaging.py@@ -0,0 +1,32 @@+from __future__ import annotations++from pathlib import Path++import tomllib+++def test_training_dependencies_are_not_required_for_runtime() -> None:+ root = Path(__file__).parents[1]+ project = tomllib.loads((root / "pyproject.toml").read_text())+ runtime = set(project["project"]["dependencies"])+ training = set(project["project"]["optional-dependencies"]["train"])+ extras = project["project"]["optional-dependencies"]++ assert runtime.isdisjoint(training)+ assert extras["gpu"]+ assert extras["metal"]+ assert not set(extras["gpu"]).intersection(training)+ assert not set(extras["metal"]).intersection(training)++ requirements = {+ line.strip()+ for line in (root / "requirements.txt").read_text().splitlines()+ if line.strip() and not line.startswith("#")+ }+ train_requirements = {+ line.strip()+ for line in (root / "requirements-train.txt").read_text().splitlines()+ if line.strip() and not line.startswith("#") and not line.startswith("-r ")+ }+ assert requirements == runtime+ assert train_requirements == training\ No newline at end of file