Skip to content

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

Run the unsloth_zoo suite beside Core instead of inside it (#10919)

Daniel Han93958760c477
diff --git a/.github/actions/core-cpu-setup/action.yml b/.github/actions/core-cpu-setup/action.yml
new file mode 100644
index 00000000000..395f2713550
--- /dev/null
+++ b/.github/actions/core-cpu-setup/action.yml
@@ -0,0 +1,220 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
+
+# The CPU install preamble that every `Core` matrix cell runs before it can
+# execute a single test: node, uv, the __from_pyproject__ spec resolution, the
+# ~2 GB pip install, and the shallow unsloth_zoo clone.
+#
+# Extracted because `consolidated-tests-ci.yml` now has TWO matrix jobs over the
+# same three (transformers, TRL) combos rather than one. The zoo suite was 457s
+# of a 17.2 minute serial cell, so it moved into its own job and now runs beside
+# the rest of Core instead of after it. Both jobs need a byte-identical
+# environment: the whole point of the split is that the two halves test the same
+# stack they used to test together, and the cheapest way for that claim to go
+# quietly false is two copies of a 120-line install block drifting apart one
+# pinned version at a time.
+#
+# So there is one copy, here, and the two call sites pass only what genuinely
+# differs between them: nothing, today, beyond the matrix specs both already
+# receive from the same matrix definition.
+#
+# What deliberately stays at the CALLER:
+# - actions/checkout, because `uses: ./.github/actions/...` cannot resolve
+# before the repository is on disk.
+# - actions/setup-python, because the pip cache restore below it probes
+# `python -m pip cache dir` and so has to run after the interpreter is
+# selected, while the install steps in here have to run after the cache is
+# restored. Splitting this action in two to swallow both halves would buy
+# nothing over leaving four pinned lines in the caller.
+# - the pip-cache-restore / pip-cache-save pair, which
+# tests/studio/test_cache_budget_discipline.py and
+# tests/studio/test_pip_cache_naming.py both read off the JOB's step list.
+# Hiding them in here would make those guards silently vacuous for the two
+# jobs that hold the largest pip entries in the repo, which is the exact
+# failure shape that file exists to reject.
+#
+# tests/studio/test_zoo_suite_parallel_isolation.py asserts that the two callers
+# agree on all of the above, so the four lines this action does not own cannot
+# drift either.
+
+name: Core CPU setup
+description: >-
+ Install the CPU test environment shared by every Core matrix cell: node 22, uv,
+ the matrix-resolved transformers + TRL pins, torch CPU, unsloth (editable) and
+ a shallow unsloth_zoo checkout installed editable. Requires actions/checkout,
+ actions/setup-python and the pip cache restore to have run first.
+
+inputs:
+ transformers-spec:
+ description: >-
+ The matrix cell's `transformers` requirement, or the literal
+ `__from_pyproject__` sentinel to take whatever pyproject.toml pins.
+ required: true
+ trl-spec:
+ description: >-
+ The matrix cell's `trl` requirement, or `__from_pyproject__` as above.
+ required: true
+ unsloth-zoo-ref:
+ description: unsloth_zoo git ref to clone and install editable.
+ required: true
+
+runs:
+ using: composite
+ steps:
+ # Node 22 unblocks tests/studio/test_chat_preset_builtin_invariants.py's
+ # `node --experimental-strip-types` subprocess. Cheap to install; keeps
+ # the consolidated job self-sufficient even if studio-backend-ci.yml
+ # changes its node setup.
+ #
+ # Installed for the zoo job too, which has no node test of its own. A few
+ # seconds against a preamble measured in minutes, and it is what makes
+ # "the two jobs run the same environment" true without qualification.
+ - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+ with:
+ node-version: '22'
+
+ - name: Install uv (some unsloth_zoo dev tooling expects it on PATH)
+ shell: bash
+ run: pip install uv
+
+ - name: Resolve matrix specs (handle __from_pyproject__ sentinel)
+ # The pyproject cell uses a sentinel; resolve the real `transformers`
+ # and `trl` constraints from the project's pyproject.toml at job time.
+ # unsloth's pyproject puts the LLM stack pins in
+ # [project.optional-dependencies] under the `huggingfacenotorch`
+ # extra (top-level [project.dependencies] is just typer/pydantic/etc.),
+ # so we walk every optional extra and pick the first matching spec.
+ # Other cells pass their spec through unchanged.
+ #
+ # RESOLVED_* go to $GITHUB_ENV, which is the job's environment file: the
+ # install step below reads them, and so does every later step in the
+ # CALLING job, exactly as when these lines lived in the workflow.
+ shell: bash
+ env:
+ MATRIX_TRANSFORMERS_SPEC: ${{ inputs.transformers-spec }}
+ MATRIX_TRL_SPEC: ${{ inputs.trl-spec }}
+ run: |
+ set -euxo pipefail
+ python <<'PY' >> "$GITHUB_ENV"
+ import os, re, tomllib
+ spec_t = os.environ["MATRIX_TRANSFORMERS_SPEC"]
+ spec_r = os.environ["MATRIX_TRL_SPEC"]
+
+ def _pkg_name(spec: str) -> str:
+ m = re.match(r"\s*([A-Za-z0-9_.-]+)", spec)
+ return (m.group(1).lower() if m else "")
+
+ if spec_t == "__from_pyproject__" or spec_r == "__from_pyproject__":
+ with open("pyproject.toml", "rb") as f:
+ doc = tomllib.load(f)
+ proj = doc.get("project", {})
+ # Try top-level deps first, then all optional extras.
+ all_deps: list[str] = list(proj.get("dependencies", []))
+ for _name, dep_list in proj.get("optional-dependencies", {}).items():
+ all_deps.extend(dep_list)
+
+ if spec_t == "__from_pyproject__":
+ spec_t = next((x for x in all_deps if _pkg_name(x) == "transformers"),
+ "transformers")
+ if spec_r == "__from_pyproject__":
+ spec_r = next((x for x in all_deps if _pkg_name(x) == "trl"),
+ "trl")
+ print(f"RESOLVED_TRANSFORMERS_SPEC={spec_t}")
+ print(f"RESOLVED_TRL_SPEC={spec_r}")
+ PY
+ # Echo to logs so the matrix cell label maps cleanly to a spec.
+ grep RESOLVED_ "$GITHUB_ENV" || true
+
+ - name: Install runtime deps (mirrors studio-backend-ci.yml + mlx-ci.yml)
+ # The shape matches studio-backend-ci.yml's "Repo tests (CPU)" install
+ # so we inherit the same CPU-spoof harness in tests/conftest.py and
+ # the same import-chain guarantees, plus the extra deps that the
+ # tests/saving + tests/utils Bucket-A files transitively need but
+ # which Repo tests (CPU) does not require because it --ignores
+ # those directories:
+ # - protobuf + sentencepiece: tests/saving/test_fix_sentencepiece_gguf_robustness.py
+ # does `from transformers.utils import sentencepiece_model_pb2`,
+ # which imports `google.protobuf`. Not pulled by transformers'
+ # base install.
+ # - triton: unsloth/_gpu_init.py:232 does an unconditional
+ # `import triton`. The triton PyPI wheel installs cleanly on
+ # Linux x86_64 even without CUDA (the import succeeds; runtime
+ # GPU work is what would fail, which we never do here).
+ # transformers + trl are matrix-parameterized.
+ shell: bash
+ run: |
+ set -euxo pipefail
+ python -m pip install --upgrade pip
+ pip install -r studio/backend/requirements/studio.txt
+ # CPU torch FIRST, before anything that depends on torch. peft, accelerate and
+ # datasets all declare a torch dependency, so installing them first made pip
+ # resolve it from PyPI: the default CUDA build, plus 14 nvidia_* wheels and
+ # cuda-toolkit, roughly 2.3 GB, which the pinned CPU wheel below then replaced
+ # seconds later. Same end state either way; this just stops paying for the
+ # download. torchvision: unsloth_zoo.vision_utils imports it at module scope.
+ pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple \
+ 'torch>=2.4,<2.11' 'torchvision<0.26'
+ pip install \
+ python-multipart aiofiles sqlalchemy cryptography \
+ pyyaml jinja2 mammoth unpdf requests typer \
+ 'numpy<3' pytest==9.0.3 pytest-asyncio pytest-xdist pytest-timeout httpx \
+ protobuf sentencepiece triton \
+ psutil packaging tqdm safetensors datasets \
+ 'peft>=0.18,<0.20' 'accelerate>=0.34,<2' \
+ ipython
+ # transformers + trl from the matrix combo.
+ pip install "$RESOLVED_TRANSFORMERS_SPEC"
+ pip install "$RESOLVED_TRL_SPEC"
+ # bitsandbytes: hard import in unsloth/models/_utils.py. Recent
+ # versions ship a CPU build that imports cleanly on Linux.
+ pip install 'bitsandbytes>=0.45'
+ # unsloth itself, editable, no-deps so pip does not fight the
+ # explicit torch CPU-index install above.
+ pip install -e . --no-deps
+ echo "::group::Installed transformers + trl + torch + unsloth versions"
+ pip show transformers
+ pip show trl
+ pip show torch
+ pip show unsloth
+ echo "::endgroup::"
+
+ - name: Clone unsloth_zoo @ ${{ inputs.unsloth-zoo-ref }}
+ # We need the repository tree (the wheel does not ship tests/), so
+ # clone shallow then editable-install so unsloth_zoo.* imports
+ # resolve to the cloned tree. We use `pip show` for the location
+ # check rather than `import unsloth_zoo` because the latter calls
+ # device_type.get_device_type() at module load and raises on a
+ # GPU-less runner; pytest steps below route through the existing
+ # tests/conftest.py spoof which handles that.
+ shell: bash
+ env:
+ UNSLOTH_ZOO_REF: ${{ inputs.unsloth-zoo-ref }}
+ run: |
+ set -euxo pipefail
+ # github.com occasionally 500s on the git fetch; retry so a
+ # single upstream blip does not fail CI.
+ # fetch + checkout rather than `clone --branch`: the ref is a commit SHA once
+ # resolve-zoo-ref has pinned it, and --branch takes only an advertised branch or
+ # tag, so it would fail with "Remote branch <sha> not found in upstream origin".
+ # tests/studio/studiobench/runtime/selftest/test_studiobench_lifecycle_refs.py
+ # ::test_clone_branch_cannot_take_a_commit asserts that rule. This form takes a
+ # branch, a tag or a commit, and stays shallow.
+ for attempt in 1 2 3; do
+ rm -rf "$RUNNER_TEMP/unsloth-zoo"
+ if git init -q "$RUNNER_TEMP/unsloth-zoo" \
+ && git -C "$RUNNER_TEMP/unsloth-zoo" remote add origin \
+ https://github.com/unslothai/unsloth-zoo \
+ && git -C "$RUNNER_TEMP/unsloth-zoo" fetch -q --depth=1 origin "$UNSLOTH_ZOO_REF" \
+ && git -C "$RUNNER_TEMP/unsloth-zoo" checkout -q FETCH_HEAD; then
+ break
+ fi
+ if [ "$attempt" -eq 3 ]; then
+ echo "::error::git clone unsloth-zoo failed after 3 attempts"
+ exit 1
+ fi
+ delay=$((5 * attempt))
+ echo "::warning::clone failed (attempt $attempt/3), retrying in ${delay}s..."
+ sleep "$delay"
+ done
+ pip install -e "$RUNNER_TEMP/unsloth-zoo" --no-deps
+ pip show unsloth_zoo
diff --git a/.github/workflows/consolidated-tests-ci.yml b/.github/workflows/consolidated-tests-ci.yml
index 78b15d4d4af..222880c957f 100644
--- a/.github/workflows/consolidated-tests-ci.yml
+++ b/.github/workflows/consolidated-tests-ci.yml
@@ -1,9 +1,18 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
-# One consolidated CPU-only job that runs every test_* function the existing
-# CI does not already cover from this repo plus the full unsloth_zoo@main
-# CPU test suite plus unsloth_zoo.compiler.test_apply_fused_lm_head.
+# CPU-only coverage for every test_* function the existing CI does not already
+# cover from this repo, plus the full unsloth_zoo@main CPU test suite, plus
+# unsloth_zoo.compiler.test_apply_fused_lm_head.
+#
+# Two matrix jobs over the SAME three (transformers, TRL) combos, not one:
+# `consolidated` for this repo's own coverage and `consolidated-zoo` for the
+# unsloth_zoo suite. They were one job until the zoo suite, 418s mean over 24
+# measured jobs, was 7.6 of its 17.2 minutes and every pull request waited
+# through both halves end to end. Nothing was dropped in the split; the two
+# halves now run side by side and the pip install preamble, which is the only
+# thing they duplicate, has one definition in
+# .github/actions/core-cpu-setup.
#
# Why a separate workflow:
# - studio-backend-ci.yml's "Repo tests (CPU)" job already auto-discovers
@@ -49,6 +58,9 @@ on:
- 'pyproject.toml'
- '.github/workflows/consolidated-tests-ci.yml'
- '.github/scripts/retry-with-apt-lock.sh'
+ # Both Core jobs `uses:` this action, so a change to it must run them.
+ - '.github/actions/core-cpu-setup/**'
+ # Both Core jobs `uses:` this action, so a change to it must run them.
push:
branches: [main]
paths:
@@ -59,6 +71,9 @@ on:
- 'pyproject.toml'
- '.github/workflows/consolidated-tests-ci.yml'
- '.github/scripts/retry-with-apt-lock.sh'
+ # Both Core jobs `uses:` this action, so a change to it must run them.
+ - '.github/actions/core-cpu-setup/**'
+ # Both Core jobs `uses:` this action, so a change to it must run them.
workflow_dispatch:
inputs:
unsloth_zoo_ref:
@@ -78,7 +93,97 @@ permissions:
contents: read
jobs:
+ resolve-zoo-ref:
+ # Both Core halves clone unsloth_zoo. Before the split they shared one checkout per
+ # matrix cell, so the compatibility checks and the suite always saw the same revision.
+ # Cloning a mutable branch twice loses that: a push to unsloth_zoo between the two clones
+ # has `consolidated` check revision A while `consolidated-zoo` runs revision B under one
+ # matrix label, and the run is then not reproducible from anything it recorded.
+ #
+ # Resolving once costs a gating job. That is affordable here and was not while Core sat on
+ # ubuntu-latest: on the pinned label a trivial job waited 11s to 162s across the runs in
+ # #10914, against the roughly seven minutes the split takes off this workflow.
+ #
+ # llama-cpp-smoke still resolves its own ref. It shares no verdict with these two, so a
+ # different revision there cannot make one cell disagree with itself.
+ name: Resolve the unsloth_zoo revision
+ runs-on: ubuntu-24.04
+ timeout-minutes: 5
+ outputs:
+ sha: ${{ steps.resolve.outputs.sha }}
+ steps:
+ - name: Resolve
+ id: resolve
+ env:
+ REF: ${{ inputs.unsloth_zoo_ref || 'main' }}
+ run: |
+ set -euo pipefail
+ # A ref that is already a commit has no remote head to look up, so pass it through.
+ if printf '%s' "$REF" | grep -Eq '^[0-9a-f]{40}$'; then
+ sha="$REF"
+ else
+ # Retried like the clone in core-cpu-setup, and for the same reason: github.com
+ # occasionally 500s. This job gates both Core halves, so an unretried blip here
+ # would skip all six matrix jobs rather than fail one step.
+ # `git ls-remote` takes PATTERNS, not one ref, so a bare name can print more than
+ # one record: a name that is both a branch and a tag prints both, and an annotated
+ # tag prints the tag object and the commit it peels to. Taking every record puts a
+ # newline inside `sha`, and `sha=<a>\n<b>` makes the second line of $GITHUB_OUTPUT a
+ # bare value that GitHub rejects, failing this job and skipping all six Core cells.
+ # So ask for the exact refs and pick one, in git's own order: a branch beats a tag,
+ # and an annotated tag resolves to the commit rather than to the tag object.
+ resolve_ref() {
+ local ref="$1" out want line uniq found
+ out="$(git ls-remote https://github.com/unslothai/unsloth-zoo \
+ "refs/heads/$ref" "refs/tags/$ref^{}" "refs/tags/$ref" "$ref")" || return 1
+ for want in "refs/heads/$ref" "refs/tags/$ref^{}" "refs/tags/$ref"; do
+ line="$(printf '%s\n' "$out" | awk -v w="$want" '$2 == w { print $1; exit }')"
+ if [ -n "$line" ]; then
+ printf '%s\n' "$line"
+ return 0
+ fi
+ done
+ # Not a plain branch or tag name. The trailing pattern above still covers a fully
+ # qualified ref, but only one commit may answer to it.
+ uniq="$(printf '%s\n' "$out" | awk 'NF { print $1 }' | sort -u)"
+ found="$(printf '%s\n' "$uniq" | grep -c . || true)"
+ # No match is the same shape as a failed lookup and stays retryable; several
+ # matches is a statement about the remote that retrying cannot change.
+ if [ "$found" -eq 0 ]; then
+ return 1
+ fi
+ if [ "$found" -ne 1 ]; then
+ return 2
+ fi
+ printf '%s\n' "$uniq"
+ }
+ sha=""
+ for attempt in 1 2 3; do
+ if sha="$(resolve_ref "$REF")"; then
+ if [ -n "$sha" ]; then break; fi
+ elif [ "$?" -eq 2 ]; then
+ # Retrying cannot disambiguate a name several commits answer to, and warning
+ # about a network blip that did not happen would send the reader the wrong way.
+ echo "::error::unsloth_zoo ref '$REF' matches more than one commit on the remote"
+ exit 1
+ fi
+ if [ "$attempt" -eq 3 ]; then
+ break
+ fi
+ delay=$((5 * attempt))
+ echo "::warning::ls-remote failed (attempt $attempt/3), retrying in ${delay}s..."
+ sleep "$delay"
+ done
+ fi
+ if [ -z "$sha" ]; then
+ echo "::error::could not resolve unsloth_zoo ref '$REF' to a commit"
+ exit 1
+ fi
+ echo "sha=$sha" >> "$GITHUB_OUTPUT"
+ echo "unsloth_zoo $REF is $sha"
+
consolidated:
+ needs: resolve-zoo-ref
# Matrix: three (transformers, TRL) combos cover the failure surface the
# PR cares about:
# 1. transformers==4.57.6 + TRL latest <1.0.0 (the just-before-5.x line)
@@ -119,12 +224,11 @@ jobs:
# the runtime check shim handles patches that legitimately cannot
# run on a CPU-only runner (real CUDA dispatcher, runtime args).
env:
- UNSLOTH_ZOO_REF: ${{ inputs.unsloth_zoo_ref || 'main' }}
- MATRIX_TRANSFORMERS_SPEC: ${{ matrix.combo.transformers_spec }}
- MATRIX_TRL_SPEC: ${{ matrix.combo.trl_spec }}
+ UNSLOTH_ZOO_REF: ${{ needs.resolve-zoo-ref.outputs.sha }}
MATRIX_COMBO_ID: ${{ matrix.combo.id }}
- # Hoisted to job-level so every step (Sanity, Bucket-A, unsloth_zoo
- # pytest, test_apply_fused_lm_head) inherits it. transformers' bundled
+ # Hoisted to job-level so every step (Sanity, Bucket-A, the static and
+ # runtime checks) inherits it, and repeated in `consolidated-zoo` for the
+ # same reason. transformers' bundled
# *_pb2.py was generated against an older protoc; the C++ protobuf
# 4+/5+/6 implementation rejects them with "Descriptors cannot be
# created directly". The pure-Python parser bypasses the check; the
@@ -157,138 +261,16 @@ jobs:
pyproject.toml
studio/backend/requirements/*.txt
- # Node 22 unblocks tests/studio/test_chat_preset_builtin_invariants.py's
- # `node --experimental-strip-types` subprocess. Cheap to install; keeps
- # the consolidated job self-sufficient even if studio-backend-ci.yml
- # changes its node setup.
- - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+ # node + uv + spec resolution + the ~2 GB install + the unsloth_zoo
+ # clone. Shared verbatim with the `consolidated-zoo` job below, which
+ # runs the same three combos against the same stack; see the action for
+ # why it owns these five steps and not the four above it.
+ - name: Core CPU setup
+ uses: ./.github/actions/core-cpu-setup
with:
- node-version: '22'
-
- - name: Install uv (some unsloth_zoo dev tooling expects it on PATH)
- run: pip install uv
-
- - name: Resolve matrix specs (handle __from_pyproject__ sentinel)
- # The pyproject cell uses a sentinel; resolve the real `transformers`
- # and `trl` constraints from the project's pyproject.toml at job time.
- # unsloth's pyproject puts the LLM stack pins in
- # [project.optional-dependencies] under the `huggingfacenotorch`
- # extra (top-level [project.dependencies] is just typer/pydantic/etc.),
- # so we walk every optional extra and pick the first matching spec.
- # Other cells pass their spec through unchanged.
- run: |
- set -euxo pipefail
- python <<'PY' >> "$GITHUB_ENV"
- import os, re, tomllib
- spec_t = os.environ["MATRIX_TRANSFORMERS_SPEC"]
- spec_r = os.environ["MATRIX_TRL_SPEC"]
-
- def _pkg_name(spec: str) -> str:
- m = re.match(r"\s*([A-Za-z0-9_.-]+)", spec)
- return (m.group(1).lower() if m else "")
-
- if spec_t == "__from_pyproject__" or spec_r == "__from_pyproject__":
- with open("pyproject.toml", "rb") as f:
- doc = tomllib.load(f)
- proj = doc.get("project", {})
- # Try top-level deps first, then all optional extras.
- all_deps: list[str] = list(proj.get("dependencies", []))
- for _name, dep_list in proj.get("optional-dependencies", {}).items():
- all_deps.extend(dep_list)
-
- if spec_t == "__from_pyproject__":
- spec_t = next((x for x in all_deps if _pkg_name(x) == "transformers"),
- "transformers")
- if spec_r == "__from_pyproject__":
- spec_r = next((x for x in all_deps if _pkg_name(x) == "trl"),
- "trl")
- print(f"RESOLVED_TRANSFORMERS_SPEC={spec_t}")
- print(f"RESOLVED_TRL_SPEC={spec_r}")
- PY
- # Echo to logs so the matrix cell label maps cleanly to a spec.
- grep RESOLVED_ "$GITHUB_ENV" || true
-
- - name: Install runtime deps (mirrors studio-backend-ci.yml + mlx-ci.yml)
- # The shape matches studio-backend-ci.yml's "Repo tests (CPU)" install
- # so we inherit the same CPU-spoof harness in tests/conftest.py and
- # the same import-chain guarantees, plus the extra deps that the
- # tests/saving + tests/utils Bucket-A files transitively need but
- # which Repo tests (CPU) does not require because it --ignores
- # those directories:
- # - protobuf + sentencepiece: tests/saving/test_fix_sentencepiece_gguf_robustness.py
- # does `from transformers.utils import sentencepiece_model_pb2`,
- # which imports `google.protobuf`. Not pulled by transformers'
- # base install.
- # - triton: unsloth/_gpu_init.py:232 does an unconditional
- # `import triton`. The triton PyPI wheel installs cleanly on
- # Linux x86_64 even without CUDA (the import succeeds; runtime
- # GPU work is what would fail, which we never do here).
- # transformers + trl are matrix-parameterized.
- run: |
- set -euxo pipefail
- python -m pip install --upgrade pip
- pip install -r studio/backend/requirements/studio.txt
- # CPU torch FIRST, before anything that depends on torch. peft, accelerate and
- # datasets all declare a torch dependency, so installing them first made pip
- # resolve it from PyPI: the default CUDA build, plus 14 nvidia_* wheels and
- # cuda-toolkit, roughly 2.3 GB, which the pinned CPU wheel below then replaced
- # seconds later. Same end state either way; this just stops paying for the
- # download. torchvision: unsloth_zoo.vision_utils imports it at module scope.
- pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple \
- 'torch>=2.4,<2.11' 'torchvision<0.26'
- pip install \
- python-multipart aiofiles sqlalchemy cryptography \
- pyyaml jinja2 mammoth unpdf requests typer \
- 'numpy<3' pytest==9.0.3 pytest-asyncio pytest-xdist pytest-timeout httpx \
- protobuf sentencepiece triton \
- psutil packaging tqdm safetensors datasets \
- 'peft>=0.18,<0.20' 'accelerate>=0.34,<2' \
- ipython
- # transformers + trl from the matrix combo.
- pip install "$RESOLVED_TRANSFORMERS_SPEC"
- pip install "$RESOLVED_TRL_SPEC"
- # bitsandbytes: hard import in unsloth/models/_utils.py. Recent
- # versions ship a CPU build that imports cleanly on Linux.
- pip install 'bitsandbytes>=0.45'
- # unsloth itself, editable, no-deps so pip does not fight the
- # explicit torch CPU-index install above.
- pip install -e . --no-deps
- echo "::group::Installed transformers + trl + torch + unsloth versions"
- pip show transformers
- pip show trl
- pip show torch
- pip show unsloth
- echo "::endgroup::"
-
- - name: Clone unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }}
- # We need the repository tree (the wheel does not ship tests/), so
- # clone shallow then editable-install so unsloth_zoo.* imports
- # resolve to the cloned tree. We use `pip show` for the location
- # check rather than `import unsloth_zoo` because the latter calls
- # device_type.get_device_type() at module load and raises on a
- # GPU-less runner; pytest steps below route through the existing
- # tests/conftest.py spoof which handles that.
- run: |
- set -euxo pipefail
- # github.com occasionally 500s on the git fetch; retry so a
- # single upstream blip does not fail CI.
- for attempt in 1 2 3; do
- rm -rf "$RUNNER_TEMP/unsloth-zoo"
- if git clone --depth=1 --branch="$UNSLOTH_ZOO_REF" \
- https://github.com/unslothai/unsloth-zoo \
- "$RUNNER_TEMP/unsloth-zoo"; then
- break
- fi
- if [ "$attempt" -eq 3 ]; then
- echo "::error::git clone unsloth-zoo failed after 3 attempts"
- exit 1
- fi
- delay=$((5 * attempt))
- echo "::warning::clone failed (attempt $attempt/3), retrying in ${delay}s..."
- sleep "$delay"
- done
- pip install -e "$RUNNER_TEMP/unsloth-zoo" --no-deps
- pip show unsloth_zoo
+ transformers-spec: ${{ matrix.combo.transformers_spec }}
+ trl-spec: ${{ matrix.combo.trl_spec }}
+ unsloth-zoo-ref: ${{ env.UNSLOTH_ZOO_REF }}
- name: Sanity — collection only (both repos)
# Catches import-time breakage before we run the suite. Cheap; bails
@@ -463,117 +445,6 @@ jobs:
# until attention_dispatch.py predefined flash_attn_varlen_func as None; it
# monkeypatches that name, so it no longer needs flash_attn on this runner.
- - name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)
- # 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip
- # cases below auto-skip on a GPU-less runner; deselect them
- # explicitly so the no-CUDA outcome is "deselected", not "skipped",
- # making intent visible in the report. Env inherited from job block.
- #
- # test_get_peft_model_passes_finetune_last_n_layers_through is
- # deselected because unsloth_zoo/mlx/loader.py at line 2972 calls
- # model.trainable_parameters() on the fake-model fixture, which
- # the test never stubbed; this fails on every platform regardless
- # of CUDA. Tracked upstream as an unsloth_zoo bug; deselecting
- # here unblocks unsloth CI until the loader fixture is fixed.
- #
- # A test that hangs rather than fails used to consume the whole 35 minute
- # job budget and report nothing usable: the job ended as "cancelled", with
- # no test name, no traceback, and every later step marked skipped. That is
- # how unsloth_zoo's gather_qmm guard walking a chain forever under the mlx
- # shim presented. PYTEST_TIMEOUT is read by pytest-timeout on every pytest
- # invocation in this step, so one setting covers the xdist run and each
- # serial file below, instead of the same flag on six command lines.
- #
- # 5 minutes. The slowest honest test in the suite is about 48s locally
- # (test_fla_hopper_tile.py), and CI runs roughly 1.65x slower than that
- # box, measured on the one test long enough to compare, so call it 80s
- # here. That leaves well over 3x headroom, while a genuine hang is now
- # named and stack-dumped in 5 minutes rather than silently eating all 35.
- #
- # This only became a sane number once unsloth_zoo stopped rebuilding
- # dir(transformers.models) on every call (unsloth-zoo#1139): before that,
- # test_every_shipped_model_type_accepted alone ran past 600s, so any
- # useful timeout would have failed a passing test.
- env:
- PYTEST_TIMEOUT: 300
- working-directory: ${{ runner.temp }}/unsloth-zoo
- run: |
- # Keep tests within each file on one worker. MLX shims contaminate sys.modules,
- # so exclude that family from xdist and run it separately below. The other
- # ignores either leak process state or use timing assertions that need a quiet
- # process.
- python -m pytest -q --tb=short -n 4 --dist loadfile tests/ \
- --ignore-glob='tests/test_mlx_*.py' \
- --ignore=tests/test_moe_bnb4bit_per_expert_conversions.py \
- --ignore=tests/test_hf_xet_fallback.py \
- --ignore=tests/test_gemma3_forced_float32_boundary_dtype.py \
- --deselect tests/test_unsloth_zoo_lora_merge.py::test_active_merge_device_returns_string_on_cuda_host \
- --deselect tests/test_unsloth_zoo_lora_merge.py::test_merge_lora_moves_cpu_inputs_to_active_device
-
- # Run contamination-sensitive files in separate processes. Keep collecting
- # results after failures, and treat pytest exit 5 as a module-level skip.
- rc=0
- _keep() {
- if [ "$1" = 5 ]; then
- # Named: an expected module-level skip and a file that stopped collecting
- # for a new reason both exit 5.
- echo "note: ${2:-that run} collected nothing (module-level skip), not a failure"
- else
- rc=1
- fi
- }
- python -m pytest -q --tb=short tests/test_mlx_generate.py || _keep "$?" tests/test_mlx_generate.py
- python -m pytest -q --tb=short tests/test_moe_bnb4bit_per_expert_conversions.py || _keep "$?" tests/test_moe_bnb4bit_per_expert_conversions.py
- python -m pytest -q --tb=short tests/test_hf_xet_fallback.py || _keep "$?" tests/test_hf_xet_fallback.py
- python -m pytest -q --tb=short tests/test_mlx_neftune_quant_map.py || _keep "$?" tests/test_mlx_neftune_quant_map.py
- python -m pytest -q --tb=short tests/test_mlx_gated_delta_vjp.py || _keep "$?" tests/test_mlx_gated_delta_vjp.py
- python -m pytest -q --tb=short tests/test_gemma3_forced_float32_boundary_dtype.py || _keep "$?" tests/test_gemma3_forced_float32_boundary_dtype.py
-
- # Run the remaining MLX files serially in alphabetical order. The three isolated
- # above also contaminate this group, so exclude them from its file list.
- mlx_group=$(ls tests/test_mlx_*.py |
- grep -vE 'tests/test_mlx_(generate|neftune_quant_map|gated_delta_vjp)\.py$')
- # Passed unquoted below to word-split into file arguments, so an empty list
- # would pass none and pytest would collect the whole rootdir instead.
- if [ -z "$mlx_group" ]; then
- echo "::error::the mlx group glob matched no files"
- exit 1
- fi
- python -m pytest -q --tb=short $mlx_group \
- --deselect tests/test_mlx_finetune_last_n_layers.py::test_get_peft_model_passes_finetune_last_n_layers_through \
- || _keep "$?" "the mlx group"
- exit "$rc"
-
- - name: unsloth_zoo — test_apply_fused_lm_head (lives in compiler.py)
- # `test_apply_fused_lm_head` lives at unsloth_zoo/compiler.py:1983,
- # not under tests/, so pytest's default discovery does not pick it up.
- # We route it through pytest by writing a one-shot shim test file
- # inside the unsloth checkout's tests/ — pytest then walks UP and
- # picks up tests/conftest.py, whose GPU-spoof harness (lines 84-141)
- # patches torch.cuda.is_available, torch.cuda.memory.mem_get_info,
- # torch.cuda.get_device_capability, and is_bf16_supported. That full
- # spoof is required because unsloth_zoo/temporary_patches/gpt_oss.py
- # at module load reads torch.cuda.memory.mem_get_info(0), which
- # bare `is_available = True` doesn't cover. Env inherited.
- run: |
- set -euxo pipefail
- cat > tests/_zoo_apply_fused_lm_head_shim.py <<'PY'
- # Auto-generated by .github/workflows/consolidated-tests-ci.yml.
- # Wraps unsloth_zoo.compiler.test_apply_fused_lm_head so that
- # tests/conftest.py's GPU-spoof harness applies before the import.
- # _zoo_aggressive_cuda_spoof extends conftest's harness with deeper
- # patches (see tests/_zoo_aggressive_cuda_spoof.py).
- import sys, pathlib
- sys.path.insert(0, str(pathlib.Path(__file__).parent))
- import _zoo_aggressive_cuda_spoof as _spoof
- _spoof.apply()
- from unsloth_zoo.compiler import test_apply_fused_lm_head as _zoo_test
- def test_zoo_apply_fused_lm_head_runs():
- _zoo_test()
- PY
- python -m pytest -q --tb=short tests/_zoo_apply_fused_lm_head_shim.py
- rm -f tests/_zoo_apply_fused_lm_head_shim.py
-
- name: Static checks — unsloth/trainer.py + unsloth/models/rl.py against latest pip TRL
# AST-only sanity: confirm both files parse and that every TRL symbol
# they reference still exists in the installed `trl`. Catches API
@@ -2343,6 +2214,244 @@ jobs:
echo "Consolidated job done. Coverage:"
echo " - 24 unsloth Bucket-A tests under tests/saving/ + tests/utils/"
echo " - 168 dataset_num_proc worker-bound guards (#2693)"
+ echo " - the unsloth_zoo suite + test_apply_fused_lm_head run in the"
+ echo " sibling 'Core zoo' job, on the same three combos"
+
+ - name: Save the pip cache
+ if: always()
+ uses: ./.github/actions/pip-cache-save
+ with:
+ dir: ${{ steps.pip-cache.outputs.dir }}
+ key: ${{ steps.pip-cache.outputs.key }}
+ cache-hit: ${{ steps.pip-cache.outputs.cache-hit }}
+
+ # The unsloth_zoo suite, lifted out of the `consolidated` cell above.
+ #
+ # Measured over 24 real jobs, the "unsloth_zoo @ <ref> - full pytest (CPU)"
+ # step runs 418s mean, 341-464s range, call it 457s at the top of the band.
+ # It used to sit serially inside a cell whose total is ~17.2 min, so it was
+ # ~7.6 min of pure critical path on 100% of pull requests: Core's
+ # `on.pull_request` paths filter matches `unsloth/**`, `studio/**`,
+ # `tests/**` and `pyproject.toml`, which is every source change this repo
+ # makes.
+ #
+ # Moving it into its own job over the SAME three combos removes no work, only
+ # the ordering. Core's critical path goes from ~17.2 min to roughly 10, the
+ # longer of the two halves, and the price is three more runners that each
+ # re-pay the install preamble. That trade is the explicit objective: fewer
+ # minutes of waiting per pull request, not fewer runner-minutes.
+ #
+ # A separate WORKFLOW file was the alternative and is worse on two counts.
+ # The combo list would have to be duplicated across files rather than sitting
+ # beside the job it mirrors, and the two halves would fall into different
+ # concurrency groups, so a force-push would cancel one and leave the other
+ # running against the commit it superseded.
+ consolidated-zoo:
+ needs: resolve-zoo-ref
+ # Byte-identical to `consolidated`'s matrix, deliberately: the coverage
+ # claim for this split is that the same suite still runs under the same
+ # three (transformers, TRL) pins. GitHub Actions supports no YAML anchors
+ # and no way to share a matrix between jobs, so it is a copy, and
+ # tests/studio/test_zoo_suite_parallel_isolation.py fails if the two copies
+ # stop agreeing.
+ strategy:
+ fail-fast: false
+ matrix:
+ combo:
+ - id: t4576-trl0latest
+ label: "HF=4.57.6 + TRL<1"
+ transformers_spec: "transformers==4.57.6"
+ trl_spec: "trl>=0.18.2,<1.0.0"
+ - id: tlatest5-trl1latest
+ label: "HF=latest + TRL=latest"
+ transformers_spec: "transformers>=5,<6"
+ trl_spec: "trl>=1,<2"
+ - id: pyproject
+ label: "HF=default + TRL=default"
+ transformers_spec: "__from_pyproject__"
+ trl_spec: "__from_pyproject__"
+ name: "Core zoo (${{ matrix.combo.label }})"
+ runs-on: ubuntu-24.04
+ # 25 rather than `consolidated`'s 35. This job runs the ~7.6 min zoo suite
+ # on top of the preamble and nothing else, and the cap is what turns a hung
+ # test into a bounded failure; see PYTEST_TIMEOUT in the step below, which
+ # is the mechanism that actually names the culprit.
+ timeout-minutes: 25
+ # Same contract as `consolidated`: no job-level or per-step
+ # `continue-on-error`, so a failing test fails the cell.
+ env:
+ UNSLOTH_ZOO_REF: ${{ needs.resolve-zoo-ref.outputs.sha }}
+ MATRIX_COMBO_ID: ${{ matrix.combo.id }}
+ # Identical to `consolidated`'s env block, and asserted identical by
+ # tests/studio/test_zoo_suite_parallel_isolation.py. The reasoning for
+ # each entry lives there; repeating it here would give it two homes.
+ PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
+ PYTHONPATH: ${{ github.workspace }}/studio
+ UNSLOTH_COMPILE_DISABLE: '1'
+ UNSLOTH_IS_PRESENT: '1'
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: '3.12'
+
+ - name: Restore the pip cache
+ id: pip-cache
+ # Its own `name`, not `consolidated`'s. Two jobs sharing one is the
+ # defect tests/studio/test_pip_cache_naming.py exists to reject: the
+ # save is gated on `cache-hit != 'true'`, so whichever job finished
+ # first on main would write the entry and the other would restore it
+ # and never save its own. The key-files are the same because the
+ # install is the same, which that file also allows for explicitly.
+ #
+ # The first run on a pull request therefore misses and downloads. The
+ # entry is written by the next push to main and warm from then on, the
+ # same lifecycle every other key here has.
+ uses: ./.github/actions/pip-cache-restore
+ with:
+ name: consolidated-zoo
+ key-files: |
+ pyproject.toml
+ studio/backend/requirements/*.txt
+
+ # The identical preamble `consolidated` runs, from the one definition
+ # both share. Anything added to the install belongs in that action, not
+ # in either caller.
+ - name: Core CPU setup
+ uses: ./.github/actions/core-cpu-setup
+ with:
+ transformers-spec: ${{ matrix.combo.transformers_spec }}
+ trl-spec: ${{ matrix.combo.trl_spec }}
+ unsloth-zoo-ref: ${{ env.UNSLOTH_ZOO_REF }}
+
+ - name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)
+ # 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip
+ # cases below auto-skip on a GPU-less runner; deselect them
+ # explicitly so the no-CUDA outcome is "deselected", not "skipped",
+ # making intent visible in the report. Env inherited from job block.
+ #
+ # test_get_peft_model_passes_finetune_last_n_layers_through is
+ # deselected because unsloth_zoo/mlx/loader.py at line 2972 calls
+ # model.trainable_parameters() on the fake-model fixture, which
+ # the test never stubbed; this fails on every platform regardless
+ # of CUDA. Tracked upstream as an unsloth_zoo bug; deselecting
+ # here unblocks unsloth CI until the loader fixture is fixed.
+ #
+ # A test that hangs rather than fails used to consume the whole job
+ # budget and report nothing usable: the job ended as "cancelled", with
+ # no test name, no traceback, and every later step marked skipped. That is
+ # how unsloth_zoo's gather_qmm guard walking a chain forever under the mlx
+ # shim presented. PYTEST_TIMEOUT is read by pytest-timeout on every pytest
+ # invocation in this step, so one setting covers the xdist run and each
+ # serial file below, instead of the same flag on six command lines.
+ #
+ # 5 minutes. The slowest honest test in the suite is about 48s locally
+ # (test_fla_hopper_tile.py), and CI runs roughly 1.65x slower than that
+ # box, measured on the one test long enough to compare, so call it 80s
+ # here. That leaves well over 3x headroom, while a genuine hang is now
+ # named and stack-dumped in 5 minutes rather than silently eating the
+ # job's whole 25 minute cap.
+ #
+ # This only became a sane number once unsloth_zoo stopped rebuilding
+ # dir(transformers.models) on every call (unsloth-zoo#1139): before that,
+ # test_every_shipped_model_type_accepted alone ran past 600s, so any
+ # useful timeout would have failed a passing test.
+ env:
+ PYTEST_TIMEOUT: 300
+ working-directory: ${{ runner.temp }}/unsloth-zoo
+ run: |
+ # Keep tests within each file on one worker. MLX shims contaminate sys.modules,
+ # so exclude that family from xdist and run it separately below. The other
+ # ignores either leak process state or use timing assertions that need a quiet
+ # process.
+ python -m pytest -q --tb=short -n 4 --dist loadfile tests/ \
+ --ignore-glob='tests/test_mlx_*.py' \
+ --ignore=tests/test_moe_bnb4bit_per_expert_conversions.py \
+ --ignore=tests/test_hf_xet_fallback.py \
+ --ignore=tests/test_gemma3_forced_float32_boundary_dtype.py \
+ --deselect tests/test_unsloth_zoo_lora_merge.py::test_active_merge_device_returns_string_on_cuda_host \
+ --deselect tests/test_unsloth_zoo_lora_merge.py::test_merge_lora_moves_cpu_inputs_to_active_device
+
+ # Run contamination-sensitive files in separate processes. Keep collecting
+ # results after failures, and treat pytest exit 5 as a module-level skip.
+ rc=0
+ _keep() {
+ if [ "$1" = 5 ]; then
+ # Named: an expected module-level skip and a file that stopped collecting
+ # for a new reason both exit 5.
+ echo "note: ${2:-that run} collected nothing (module-level skip), not a failure"
+ else
+ rc=1
+ fi
+ }
+ python -m pytest -q --tb=short tests/test_mlx_generate.py || _keep "$?" tests/test_mlx_generate.py
+ python -m pytest -q --tb=short tests/test_moe_bnb4bit_per_expert_conversions.py || _keep "$?" tests/test_moe_bnb4bit_per_expert_conversions.py
+ python -m pytest -q --tb=short tests/test_hf_xet_fallback.py || _keep "$?" tests/test_hf_xet_fallback.py
+ python -m pytest -q --tb=short tests/test_mlx_neftune_quant_map.py || _keep "$?" tests/test_mlx_neftune_quant_map.py
+ python -m pytest -q --tb=short tests/test_mlx_gated_delta_vjp.py || _keep "$?" tests/test_mlx_gated_delta_vjp.py
+ python -m pytest -q --tb=short tests/test_gemma3_forced_float32_boundary_dtype.py || _keep "$?" tests/test_gemma3_forced_float32_boundary_dtype.py
+
+ # Run the remaining MLX files serially in alphabetical order. The three isolated
+ # above also contaminate this group, so exclude them from its file list.
+ mlx_group=$(ls tests/test_mlx_*.py |
+ grep -vE 'tests/test_mlx_(generate|neftune_quant_map|gated_delta_vjp)\.py$')
+ # Passed unquoted below to word-split into file arguments, so an empty list
+ # would pass none and pytest would collect the whole rootdir instead.
+ if [ -z "$mlx_group" ]; then
+ echo "::error::the mlx group glob matched no files"
+ exit 1
+ fi
+ python -m pytest -q --tb=short $mlx_group \
+ --deselect tests/test_mlx_finetune_last_n_layers.py::test_get_peft_model_passes_finetune_last_n_layers_through \
+ || _keep "$?" "the mlx group"
+ exit "$rc"
+
+ - name: unsloth_zoo — test_apply_fused_lm_head (lives in compiler.py)
+ # `test_apply_fused_lm_head` lives at unsloth_zoo/compiler.py:1983,
+ # not under tests/, so pytest's default discovery does not pick it up.
+ # We route it through pytest by writing a one-shot shim test file
+ # inside the unsloth checkout's tests/ — pytest then walks UP and
+ # picks up tests/conftest.py, whose GPU-spoof harness (lines 84-141)
+ # patches torch.cuda.is_available, torch.cuda.memory.mem_get_info,
+ # torch.cuda.get_device_capability, and is_bf16_supported. That full
+ # spoof is required because unsloth_zoo/temporary_patches/gpt_oss.py
+ # at module load reads torch.cuda.memory.mem_get_info(0), which
+ # bare `is_available = True` doesn't cover. Env inherited.
+ run: |
+ set -euxo pipefail
+ cat > tests/_zoo_apply_fused_lm_head_shim.py <<'PY'
+ # Auto-generated by .github/workflows/consolidated-tests-ci.yml.
+ # Wraps unsloth_zoo.compiler.test_apply_fused_lm_head so that
+ # tests/conftest.py's GPU-spoof harness applies before the import.
+ # _zoo_aggressive_cuda_spoof extends conftest's harness with deeper
+ # patches (see tests/_zoo_aggressive_cuda_spoof.py).
+ import sys, pathlib
+ sys.path.insert(0, str(pathlib.Path(__file__).parent))
+ import _zoo_aggressive_cuda_spoof as _spoof
+ _spoof.apply()
+ from unsloth_zoo.compiler import test_apply_fused_lm_head as _zoo_test
+ def test_zoo_apply_fused_lm_head_runs():
+ _zoo_test()
+ PY
+ python -m pytest -q --tb=short tests/_zoo_apply_fused_lm_head_shim.py
+ rm -f tests/_zoo_apply_fused_lm_head_shim.py
+
+ - name: Summary
+ if: always()
+ run: |
+ echo "::group::Versions"
+ python -c "import sys, platform; print(sys.version); print(platform.platform())"
+ python -c "import torch; print('torch', torch.__version__, 'cuda?', torch.cuda.is_available())"
+ python -c "import transformers; print('transformers', transformers.__version__)"
+ # `pip show` instead of `import unsloth_zoo` — its __init__ raises
+ # without an accelerator and the spoof harness only kicks in under
+ # pytest. Cheap and accurate.
+ pip show unsloth_zoo
+ echo "::endgroup::"
+ echo "Core zoo job done. Coverage:"
echo " - unsloth_zoo @ ${UNSLOTH_ZOO_REF} pytest tests/ (5 GPU cases deselected)"
echo " - unsloth_zoo.compiler.test_apply_fused_lm_head"
diff --git a/.github/workflows/workflow-trigger-lint.yml b/.github/workflows/workflow-trigger-lint.yml
index 5a9799decd0..3f4ae63c57a 100644
--- a/.github/workflows/workflow-trigger-lint.yml
+++ b/.github/workflows/workflow-trigger-lint.yml
@@ -134,6 +134,7 @@ jobs:
tests/studio/test_backend_ci_matrix.py \
tests/studio/test_backend_ci_parallel_isolation.py \
tests/studio/test_cache_budget_discipline.py \
+ tests/studio/test_local_actions_are_in_path_filters.py \
tests/studio/test_chat_ui_shards_cover_everything.py \
tests/studio/test_ui_shard_engines.py \
tests/studio/test_zoo_suite_parallel_isolation.py \
diff --git a/tests/studio/test_cache_budget_discipline.py b/tests/studio/test_cache_budget_discipline.py
index d1d9bed8757..75cd132585e 100644
--- a/tests/studio/test_cache_budget_discipline.py
+++ b/tests/studio/test_cache_budget_discipline.py
@@ -63,6 +63,13 @@ WORKFLOWS = REPO / ".github" / "workflows"
# splits the halves so the save can be gated on the default branch.
PIP_CACHE_JOBS = {
("consolidated-tests-ci.yml", "consolidated"),
+ # The unsloth_zoo half of Core, split out so its 418s suite runs beside the rest
+ # instead of after it. It runs the same install as `consolidated` (both call
+ # .github/actions/core-cpu-setup), which is exactly the torch/transformers-class
+ # download this allowlist exists for, and it needs its OWN name rather than sharing
+ # `consolidated`'s: the save is gated on `cache-hit != 'true'`, so a shared key means
+ # whichever job finishes first on main writes it and the other never saves.
+ ("consolidated-tests-ci.yml", "consolidated-zoo"),
("consolidated-tests-ci.yml", "llama-cpp-smoke"),
("mlx-ci.yml", "dispatch"),
("notebooks-ci.yml", "api-introspect"),
diff --git a/tests/studio/test_local_actions_are_in_path_filters.py b/tests/studio/test_local_actions_are_in_path_filters.py
new file mode 100644
index 00000000000..805a647d869
--- /dev/null
+++ b/tests/studio/test_local_actions_are_in_path_filters.py
@@ -0,0 +1,225 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""A path-filtered workflow must list the local actions it `uses:`.
+
+Factoring a workflow's setup into `.github/actions/<name>/` moves real logic outside every
+path in that workflow's `paths:` filter. The workflow then does not run when only the action
+changes, so a PR can break the install preamble of a job and watch the job not run. The
+failure is silent in the worst way: the checks that would have caught it are simply absent,
+and the PR is green because nothing looked.
+
+`.github/scripts/retry-with-apt-lock.sh` is already carried in two filters for this reason.
+This makes that a rule rather than a thing someone remembered once.
+
+A workflow with no `paths:` filter runs on everything and is not at risk, so it is skipped.
+"""
+
+import re
+from pathlib import Path
+
+import pytest
+import yaml
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+_WORKFLOWS = REPO_ROOT / ".github" / "workflows"
+
+# The one action pair still missing from the filters of the six workflows that use it
+# (consolidated-tests-ci, mlx-ci, notebooks-ci, studio-backend-ci, studio-export-capability-ci,
+# version-compat-ci). Every other local action in the repo is already listed, by its
+# action.yml, which is the convention this guard enforces.
+#
+# Waived rather than fixed here because adding them means touching six workflows unrelated to
+# this change. The test below deletes the waiver's right to exist as soon as that is done.
+_PRE_EXISTING = {
+ ".github/actions/pip-cache-restore",
+ ".github/actions/pip-cache-save",
+}
+
+
+def _triggers(doc: dict) -> dict:
+ # `on:` is the YAML 1.1 boolean True once parsed, unless it was quoted.
+ return doc.get(True) or doc.get("on") or {}
+
+
+def _resolve(used: str) -> str:
+ """The action's real location in this repo, given a `uses:` path.
+
+ Some workflows check the repo into a subdirectory and so say
+ `uses: ./unsloth/.github/actions/pip-cache-restore`. No such directory exists here; the
+ action being referenced is the repo-root one. Taking the literal string would put a path
+ under `unsloth/` into the check, where a `unsloth/**` filter entry satisfies it and the
+ guard passes on a workflow that would still skip. So leading segments are dropped until
+ the result is a directory that actually holds an action.
+ """
+ parts = used.split("/")
+ for start in range(len(parts)):
+ candidate = "/".join(parts[start:])
+ if (REPO_ROOT / candidate / "action.yml").is_file():
+ return candidate
+ return used
+
+
+def _local_actions(node) -> set:
+ """Every `uses: ./path` in the workflow, at any depth."""
+ found = set()
+ if isinstance(node, dict):
+ uses = node.get("uses")
+ if isinstance(uses, str) and uses.startswith("./"):
+ found.add(_resolve(uses[2:].rstrip("/")))
+ for value in node.values():
+ found |= _local_actions(value)
+ elif isinstance(node, list):
+ for value in node:
+ found |= _local_actions(value)
+ return found
+
+
+def _matches(pattern: str, path: str) -> bool:
+ """One `paths:` glob against one file path, with GitHub's wildcard semantics.
+
+ `path` is a FILE inside the action, not the action directory, because that is what
+ GitHub matches a filter against. The distinction decides real cases: a bare
+ `.github/actions/foo` entry selects only a file literally at that path, so it does NOT
+ cover `.github/actions/foo/action.yml` and the workflow still skips.
+ """
+ regex = re.escape(pattern).replace(r"\*\*", ".*").replace(r"\*", "[^/]*")
+ return re.fullmatch(regex, path) is not None
+
+
+def _selects(paths: list, path: str) -> bool:
+ """Does the whole `paths:` list select `path`?
+
+ The list is ordered and `!` negates, with the LAST matching pattern deciding, so the
+ entries cannot be tested independently: `.github/actions/**` followed by
+ `!.github/actions/foo/**` does not select foo. This repo already uses negation
+ (startup-profile-ci.yml excludes `!studio/backend/tests/**`), so reading `!` as a
+ literal character would quietly pass a workflow that really does skip.
+ """
+ selected = False
+ for entry in paths:
+ entry = str(entry).strip("'\"")
+ negated = entry.startswith("!")
+ if _matches(entry[1:] if negated else entry, path):
+ selected = not negated
+ return selected
+
+
+def _action_files(action: str, root: Path = REPO_ROOT) -> list:
+ """Every file GitHub could report as changed for an edit to this action.
+
+ The whole directory, not just the manifest. A composite action that grows a helper
+ script keeps working with a filter naming only `action.yml`, and a helper-only change
+ would then skip every workflow that uses it, which is the exact failure this guard
+ exists to prevent. Listing the directory means such a change has to be covered too,
+ by a `/**` entry or by naming the file.
+ """
+ directory = root / action
+ if not directory.is_dir():
+ return [f"{action}/action.yml"]
+ return sorted(
+ str(path.relative_to(root)).replace("\\", "/")
+ for path in directory.rglob("*")
+ if path.is_file()
+ )
+
+
+def _workflows() -> list:
+ return sorted(_WORKFLOWS.glob("*.yml"))
+
+
[email protected]("workflow", _workflows(), ids = lambda p: p.name)
+def test_a_path_filtered_workflow_lists_the_actions_it_uses(workflow):
+ doc = yaml.safe_load(workflow.read_text(encoding = "utf-8"))
+ if not isinstance(doc, dict):
+ pytest.skip("not a workflow mapping")
+ actions = _local_actions(doc.get("jobs") or {})
+ if not actions:
+ pytest.skip("uses no local action")
+
+ for event, spec in _triggers(doc).items():
+ paths = (spec or {}).get("paths") if isinstance(spec, dict) else None
+ if not paths:
+ continue # unfiltered: it already runs on any change
+ missing = sorted(
+ a
+ for a in actions - _PRE_EXISTING
+ if not all(_selects(paths, f) for f in _action_files(a))
+ )
+ assert not missing, (
+ f"{workflow.name} `{event}` is path-filtered but does not list {missing}, so a "
+ f"change to only that action runs none of the jobs that depend on it"
+ )
+
+
+def test_an_ordered_negation_is_not_read_as_a_literal_bang():
+ """`paths:` is ordered and `!` negates, with the last match deciding.
+
+ Testing the entries independently would accept `.github/actions/**` followed by
+ `!.github/actions/foo/**` as covering foo, while GitHub skips the workflow on a
+ foo-only change. This repo already negates in startup-profile-ci.yml, so the case is
+ reachable rather than theoretical.
+ """
+ action = ".github/actions/foo/action.yml"
+ assert _selects([".github/actions/**"], action)
+ assert not _selects([".github/actions/**", "!.github/actions/foo/**"], action)
+ assert _selects(["!.github/actions/foo/**", ".github/actions/**"], action)
+ # The negation this repo actually carries, and a sibling it must not touch.
+ assert not _selects(
+ ["studio/backend/**", "!studio/backend/tests/**"], "studio/backend/tests/x.py"
+ )
+ assert _selects(["studio/backend/**", "!studio/backend/tests/**"], "studio/backend/main.py")
+
+
+def test_a_helper_file_beside_the_manifest_is_checked_too(tmp_path):
+ """Every action here is a lone action.yml today, so this is the case the tree cannot
+ show: an action that grows a helper script.
+
+ A filter naming only `<action>/action.yml` still selects the manifest, so checking the
+ manifest alone would pass while a helper-only change skipped every consuming workflow.
+ """
+ action = ".github/actions/grown"
+ (tmp_path / action).mkdir(parents = True)
+ (tmp_path / action / "action.yml").write_text("name: grown\n")
+ (tmp_path / action / "run.sh").write_text("echo hi\n")
+
+ files = _action_files(action, root = tmp_path)
+ assert files == [f"{action}/action.yml", f"{action}/run.sh"]
+
+ manifest_only = [f"{action}/action.yml"]
+ assert _selects(manifest_only, files[0])
+ assert not all(
+ _selects(manifest_only, f) for f in files
+ ), "naming only the manifest must not satisfy the guard once the action has a helper"
+ assert all(_selects([f"{action}/**"], f) for f in files)
+
+
+def test_the_pre_existing_list_does_not_outlive_the_problem():
+ """An entry that is now listed everywhere it is used must leave the waiver, or the
+ waiver quietly re-permits a regression someone already paid to fix."""
+ still_unlisted = set()
+ for workflow in _workflows():
+ doc = yaml.safe_load(workflow.read_text(encoding = "utf-8"))
+ if not isinstance(doc, dict):
+ continue
+ actions = _local_actions(doc.get("jobs") or {})
+ for spec in _triggers(doc).values():
+ paths = (spec or {}).get("paths") if isinstance(spec, dict) else None
+ if not paths:
+ continue
+ still_unlisted |= {
+ a for a in actions if not all(_selects(paths, f) for f in _action_files(a))
+ }
+ settled = sorted(_PRE_EXISTING - still_unlisted)
+ assert not settled, f"these are now listed everywhere and must leave _PRE_EXISTING: {settled}"
+
+
+def test_the_guard_sees_a_workflow_that_uses_a_local_action():
+ """Vacuity: if nothing in the tree uses a local action, everything above skips."""
+ users = [
+ w
+ for w in _workflows()
+ if _local_actions(yaml.safe_load(w.read_text(encoding = "utf-8")) or {})
+ ]
+ assert users, "no workflow uses a local action, so this guard checks nothing"
diff --git a/tests/studio/test_zoo_suite_parallel_isolation.py b/tests/studio/test_zoo_suite_parallel_isolation.py
index e2c20ac87f1..1d59e9b07ec 100644
--- a/tests/studio/test_zoo_suite_parallel_isolation.py
+++ b/tests/studio/test_zoo_suite_parallel_isolation.py
@@ -1,6 +1,23 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
-"""Guard the parallel zoo run and the serial reruns for tests that cannot share workers."""
+"""Guard the parallel zoo run, the serial reruns for tests that cannot share workers,
+and the split that moved all of them into their own job.
+
+The zoo suite used to run inside the `consolidated` cell, serially, for 418s of that
+cell's 17.2 minutes. It now runs in `consolidated-zoo`, a second matrix job over the
+same three (transformers, TRL) combos. Two things that were previously true by
+construction have to be asserted now that there are two jobs:
+
+ - the suite still runs under all three pins, and still runs at all. A matrix that
+ loses a combo, or a job whose steps drift away from the zoo ones, reduces coverage
+ without failing anything.
+ - the two jobs still install the same environment. The install lives in
+ .github/actions/core-cpu-setup so there is one copy of it, but the four steps above
+ that action (checkout, setup-python, the pip cache restore) and the job-level `env`
+ and `runs-on` are per-job and can drift silently. A zoo job on a different
+ transformers than the cell it was split out of would still be green, and would be
+ testing something nobody asked for.
+"""
from __future__ import annotations
@@ -8,10 +25,18 @@ import re
from pathlib import Path
import pytest
+import yaml
WORKFLOW = (
Path(__file__).resolve().parents[2] / ".github" / "workflows" / "consolidated-tests-ci.yml"
)
+ACTION = (
+ Path(__file__).resolve().parents[2] / ".github" / "actions" / "core-cpu-setup" / "action.yml"
+)
+
+# The two halves of Core. Both run the same matrix; only their test steps differ.
+CORE_JOBS = ("consolidated", "consolidated-zoo")
+SETUP_ACTION = "./.github/actions/core-cpu-setup"
# (ignored path, why it cannot share a worker with the rest of the suite)
ISOLATED = [
@@ -187,6 +212,149 @@ def test_an_empty_mlx_group_stops_the_step_instead_of_collecting_everything() ->
)
+def _doc() -> dict:
+ return yaml.safe_load(WORKFLOW.read_text(encoding = "utf-8"))
+
+
+def _job(jid: str) -> dict:
+ jobs = _doc()["jobs"]
+ assert jid in jobs, f"{jid} is gone from the workflow; the split it guards is undone"
+ return jobs[jid]
+
+
+def _job_commands(jid: str) -> list[str]:
+ """Every `python -m pytest ...` invocation inside one job, continuations resolved."""
+ found = []
+ for step in _job(jid)["steps"]:
+ joined = re.sub(r"\\\s*\n\s*", " ", str(step.get("run", "")))
+ found += [
+ " ".join(line.split())
+ for line in joined.splitlines()
+ if "python -m pytest" in line and not line.lstrip().startswith("#")
+ ]
+ return found
+
+
+def _preamble(jid: str) -> list[dict]:
+ """The steps up to and including the shared setup action, which every job repeats."""
+ out: list[dict] = []
+ for step in _job(jid)["steps"]:
+ out.append(step)
+ if str(step.get("uses", "")) == SETUP_ACTION:
+ return out
+ raise AssertionError(
+ f"{jid} never calls {SETUP_ACTION}, so it no longer installs the environment "
+ f"the other half of Core installs"
+ )
+
+
+def test_the_zoo_suite_runs_beside_core_and_not_inside_it() -> None:
+ """The point of the split. Back inside `consolidated` it is 7.6 min of serial wait."""
+ inside = [c for c in _job_commands("consolidated") if ZOO_MARKER in c]
+ assert not inside, (
+ f"the zoo suite is running inside the `consolidated` cell again ({inside}), so "
+ f"every pull request waits through it before the rest of Core can finish"
+ )
+ beside = [c for c in _job_commands("consolidated-zoo") if ZOO_MARKER in c]
+ assert (
+ len(beside) == 1
+ ), f"expected the parallel zoo run in the `consolidated-zoo` job, found {len(beside)}"
+
+
+def test_the_zoo_job_runs_every_zoo_step_that_left_the_cell() -> None:
+ """Two steps moved. A move that drops one is a silent deletion of its tests."""
+ zoo = " \n".join(_job_commands("consolidated-zoo"))
+ assert "_zoo_apply_fused_lm_head_shim.py" in zoo, (
+ "unsloth_zoo.compiler.test_apply_fused_lm_head moved out of the consolidated "
+ "cell but is not run by the zoo job either, so it runs nowhere"
+ )
+ for path, _ in ISOLATED:
+ assert path in zoo, (
+ f"{path}'s serial rerun is not in the zoo job. It is ignored by the parallel "
+ f"run, so wherever its rerun went, it has to have gone with it"
+ )
+
+
[email protected]("jid", CORE_JOBS)
+def test_both_halves_of_core_run_the_same_three_combos(jid: str) -> None:
+ """Coverage is '3 pins x the same suite'. A matrix that drifts quietly ends that."""
+ expected = _doc()["jobs"]["consolidated"]["strategy"]["matrix"]["combo"]
+ assert [c["id"] for c in expected] == [
+ "t4576-trl0latest",
+ "tlatest5-trl1latest",
+ "pyproject",
+ ], "the Core combo ids changed; update this guard deliberately, not by accident"
+ assert _job(jid)["strategy"]["matrix"]["combo"] == expected, (
+ f"{jid}'s matrix no longer matches `consolidated`'s. GitHub Actions has no way to "
+ f"share a matrix between jobs, so these are two copies, and a copy that drifts "
+ f"means the two halves of Core are testing different (transformers, TRL) pins "
+ f"while still reporting as one gate"
+ )
+
+
+def test_both_halves_of_core_share_one_install_preamble() -> None:
+ """The install has one definition; the four steps around it are still per-job.
+
+ Checkout, setup-python and the pip cache restore are duplicated by necessity, and a
+ difference in any of them (a different interpreter, a cache scoped to other files)
+ makes the zoo job test a stack the cell it was split from never runs.
+ """
+ a, b = (_preamble(jid) for jid in CORE_JOBS)
+ assert len(a) == len(b), (
+ f"the two Core jobs run {len(a)} and {len(b)} preamble steps. They install the "
+ f"same environment, so their preambles have to be the same steps in the same order"
+ )
+ for left, right in zip(a, b):
+ # The pip cache `name` is the one field that MUST differ: a shared name is a
+ # shared key, and only the first job to finish on main would ever save.
+ # tests/studio/test_pip_cache_naming.py owns that rule.
+ left, right = dict(left), dict(right)
+ if "pip-cache-restore" in str(left.get("uses", "")):
+ left["with"] = {k: v for k, v in left["with"].items() if k != "name"}
+ right["with"] = {k: v for k, v in right["with"].items() if k != "name"}
+ assert left == right, (
+ f"the Core preambles have drifted at step "
+ f"{left.get('name') or left.get('uses')!r}:\n consolidated: {left}\n"
+ f" consolidated-zoo: {right}"
+ )
+
+
+def test_both_halves_of_core_share_one_environment_and_one_runner() -> None:
+ """`env` and `runs-on` are job-level and cannot be factored into the action.
+
+ `runs-on` is included on purpose. The label is not cosmetic here: measured on this
+ repo, `ubuntu-latest` queues behind the org's backlog for a median 44.9 min while any
+ other Ubuntu label walks past in minutes, so two halves of one gate on two different
+ labels would make the split buy nothing.
+ """
+ a, b = (_job(jid) for jid in CORE_JOBS)
+ assert a["env"] == b["env"], (
+ f"the two Core jobs no longer share a job-level env:\n only in consolidated: "
+ f"{ {k: v for k, v in a['env'].items() if b['env'].get(k) != v} }\n"
+ f" only in consolidated-zoo: "
+ f"{ {k: v for k, v in b['env'].items() if a['env'].get(k) != v} }"
+ )
+ assert a["runs-on"] == b["runs-on"], (
+ f"the two halves of Core run on different labels ({a['runs-on']} vs "
+ f"{b['runs-on']}), so one of them queues behind a backlog the other skips"
+ )
+
+
+def test_the_shared_preamble_is_not_also_inlined() -> None:
+ """A caller that re-adds an install step is how one definition becomes two."""
+ body = "\n".join(str(step.get("run", "")) for jid in CORE_JOBS for step in _job(jid)["steps"])
+ for marker in ("pip install -e .", "download.pytorch.org/whl/cpu", "git clone"):
+ assert marker not in body, (
+ f"{marker!r} is inlined in a Core job again. The install lives in "
+ f"{SETUP_ACTION} so both halves cannot drift apart; a second copy is the "
+ f"drift, and it is silent until the two jobs disagree about a version"
+ )
+ action = yaml.safe_load(ACTION.read_text(encoding = "utf-8"))
+ run = "\n".join(str(step.get("run", "")) for step in action["runs"]["steps"])
+ for marker in ("pip install -e .", "download.pytorch.org/whl/cpu", "git clone"):
+ assert marker in run, f"{SETUP_ACTION} no longer does {marker!r}"
+
+
def test_a_skipped_isolated_file_is_named_in_the_log() -> None:
"""Exit 5 is tolerated, so the file that produced it has to be identifiable: an
expected module-level skip and a file that stopped collecting for a new reason both