Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Run the unsloth_zoo suite beside Core instead of inside it (#10919)
Daniel Han93958760c477
diff --git a/.github/actions/core-cpu-setup/action.yml b/.github/actions/core-cpu-setup/action.ymlnew file mode 100644index 00000000000..395f2713550--- /dev/null+++ b/.github/actions/core-cpu-setup/action.yml@@ -0,0 +1,220 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.++# The CPU install preamble that every `Core` matrix cell runs before it can+# execute a single test: node, uv, the __from_pyproject__ spec resolution, the+# ~2 GB pip install, and the shallow unsloth_zoo clone.+#+# Extracted because `consolidated-tests-ci.yml` now has TWO matrix jobs over the+# same three (transformers, TRL) combos rather than one. The zoo suite was 457s+# of a 17.2 minute serial cell, so it moved into its own job and now runs beside+# the rest of Core instead of after it. Both jobs need a byte-identical+# environment: the whole point of the split is that the two halves test the same+# stack they used to test together, and the cheapest way for that claim to go+# quietly false is two copies of a 120-line install block drifting apart one+# pinned version at a time.+#+# So there is one copy, here, and the two call sites pass only what genuinely+# differs between them: nothing, today, beyond the matrix specs both already+# receive from the same matrix definition.+#+# What deliberately stays at the CALLER:+# - actions/checkout, because `uses: ./.github/actions/...` cannot resolve+# before the repository is on disk.+# - actions/setup-python, because the pip cache restore below it probes+# `python -m pip cache dir` and so has to run after the interpreter is+# selected, while the install steps in here have to run after the cache is+# restored. Splitting this action in two to swallow both halves would buy+# nothing over leaving four pinned lines in the caller.+# - the pip-cache-restore / pip-cache-save pair, which+# tests/studio/test_cache_budget_discipline.py and+# tests/studio/test_pip_cache_naming.py both read off the JOB's step list.+# Hiding them in here would make those guards silently vacuous for the two+# jobs that hold the largest pip entries in the repo, which is the exact+# failure shape that file exists to reject.+#+# tests/studio/test_zoo_suite_parallel_isolation.py asserts that the two callers+# agree on all of the above, so the four lines this action does not own cannot+# drift either.++name: Core CPU setup+description: >-+ Install the CPU test environment shared by every Core matrix cell: node 22, uv,+ the matrix-resolved transformers + TRL pins, torch CPU, unsloth (editable) and+ a shallow unsloth_zoo checkout installed editable. Requires actions/checkout,+ actions/setup-python and the pip cache restore to have run first.++inputs:+ transformers-spec:+ description: >-+ The matrix cell's `transformers` requirement, or the literal+ `__from_pyproject__` sentinel to take whatever pyproject.toml pins.+ required: true+ trl-spec:+ description: >-+ The matrix cell's `trl` requirement, or `__from_pyproject__` as above.+ required: true+ unsloth-zoo-ref:+ description: unsloth_zoo git ref to clone and install editable.+ required: true++runs:+ using: composite+ steps:+ # Node 22 unblocks tests/studio/test_chat_preset_builtin_invariants.py's+ # `node --experimental-strip-types` subprocess. Cheap to install; keeps+ # the consolidated job self-sufficient even if studio-backend-ci.yml+ # changes its node setup.+ #+ # Installed for the zoo job too, which has no node test of its own. A few+ # seconds against a preamble measured in minutes, and it is what makes+ # "the two jobs run the same environment" true without qualification.+ - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0+ with:+ node-version: '22'++ - name: Install uv (some unsloth_zoo dev tooling expects it on PATH)+ shell: bash+ run: pip install uv++ - name: Resolve matrix specs (handle __from_pyproject__ sentinel)+ # The pyproject cell uses a sentinel; resolve the real `transformers`+ # and `trl` constraints from the project's pyproject.toml at job time.+ # unsloth's pyproject puts the LLM stack pins in+ # [project.optional-dependencies] under the `huggingfacenotorch`+ # extra (top-level [project.dependencies] is just typer/pydantic/etc.),+ # so we walk every optional extra and pick the first matching spec.+ # Other cells pass their spec through unchanged.+ #+ # RESOLVED_* go to $GITHUB_ENV, which is the job's environment file: the+ # install step below reads them, and so does every later step in the+ # CALLING job, exactly as when these lines lived in the workflow.+ shell: bash+ env:+ MATRIX_TRANSFORMERS_SPEC: ${{ inputs.transformers-spec }}+ MATRIX_TRL_SPEC: ${{ inputs.trl-spec }}+ run: |+ set -euxo pipefail+ python <<'PY' >> "$GITHUB_ENV"+ import os, re, tomllib+ spec_t = os.environ["MATRIX_TRANSFORMERS_SPEC"]+ spec_r = os.environ["MATRIX_TRL_SPEC"]++ def _pkg_name(spec: str) -> str:+ m = re.match(r"\s*([A-Za-z0-9_.-]+)", spec)+ return (m.group(1).lower() if m else "")++ if spec_t == "__from_pyproject__" or spec_r == "__from_pyproject__":+ with open("pyproject.toml", "rb") as f:+ doc = tomllib.load(f)+ proj = doc.get("project", {})+ # Try top-level deps first, then all optional extras.+ all_deps: list[str] = list(proj.get("dependencies", []))+ for _name, dep_list in proj.get("optional-dependencies", {}).items():+ all_deps.extend(dep_list)++ if spec_t == "__from_pyproject__":+ spec_t = next((x for x in all_deps if _pkg_name(x) == "transformers"),+ "transformers")+ if spec_r == "__from_pyproject__":+ spec_r = next((x for x in all_deps if _pkg_name(x) == "trl"),+ "trl")+ print(f"RESOLVED_TRANSFORMERS_SPEC={spec_t}")+ print(f"RESOLVED_TRL_SPEC={spec_r}")+ PY+ # Echo to logs so the matrix cell label maps cleanly to a spec.+ grep RESOLVED_ "$GITHUB_ENV" || true++ - name: Install runtime deps (mirrors studio-backend-ci.yml + mlx-ci.yml)+ # The shape matches studio-backend-ci.yml's "Repo tests (CPU)" install+ # so we inherit the same CPU-spoof harness in tests/conftest.py and+ # the same import-chain guarantees, plus the extra deps that the+ # tests/saving + tests/utils Bucket-A files transitively need but+ # which Repo tests (CPU) does not require because it --ignores+ # those directories:+ # - protobuf + sentencepiece: tests/saving/test_fix_sentencepiece_gguf_robustness.py+ # does `from transformers.utils import sentencepiece_model_pb2`,+ # which imports `google.protobuf`. Not pulled by transformers'+ # base install.+ # - triton: unsloth/_gpu_init.py:232 does an unconditional+ # `import triton`. The triton PyPI wheel installs cleanly on+ # Linux x86_64 even without CUDA (the import succeeds; runtime+ # GPU work is what would fail, which we never do here).+ # transformers + trl are matrix-parameterized.+ shell: bash+ run: |+ set -euxo pipefail+ python -m pip install --upgrade pip+ pip install -r studio/backend/requirements/studio.txt+ # CPU torch FIRST, before anything that depends on torch. peft, accelerate and+ # datasets all declare a torch dependency, so installing them first made pip+ # resolve it from PyPI: the default CUDA build, plus 14 nvidia_* wheels and+ # cuda-toolkit, roughly 2.3 GB, which the pinned CPU wheel below then replaced+ # seconds later. Same end state either way; this just stops paying for the+ # download. torchvision: unsloth_zoo.vision_utils imports it at module scope.+ pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple \+ 'torch>=2.4,<2.11' 'torchvision<0.26'+ pip install \+ python-multipart aiofiles sqlalchemy cryptography \+ pyyaml jinja2 mammoth unpdf requests typer \+ 'numpy<3' pytest==9.0.3 pytest-asyncio pytest-xdist pytest-timeout httpx \+ protobuf sentencepiece triton \+ psutil packaging tqdm safetensors datasets \+ 'peft>=0.18,<0.20' 'accelerate>=0.34,<2' \+ ipython+ # transformers + trl from the matrix combo.+ pip install "$RESOLVED_TRANSFORMERS_SPEC"+ pip install "$RESOLVED_TRL_SPEC"+ # bitsandbytes: hard import in unsloth/models/_utils.py. Recent+ # versions ship a CPU build that imports cleanly on Linux.+ pip install 'bitsandbytes>=0.45'+ # unsloth itself, editable, no-deps so pip does not fight the+ # explicit torch CPU-index install above.+ pip install -e . --no-deps+ echo "::group::Installed transformers + trl + torch + unsloth versions"+ pip show transformers+ pip show trl+ pip show torch+ pip show unsloth+ echo "::endgroup::"++ - name: Clone unsloth_zoo @ ${{ inputs.unsloth-zoo-ref }}+ # We need the repository tree (the wheel does not ship tests/), so+ # clone shallow then editable-install so unsloth_zoo.* imports+ # resolve to the cloned tree. We use `pip show` for the location+ # check rather than `import unsloth_zoo` because the latter calls+ # device_type.get_device_type() at module load and raises on a+ # GPU-less runner; pytest steps below route through the existing+ # tests/conftest.py spoof which handles that.+ shell: bash+ env:+ UNSLOTH_ZOO_REF: ${{ inputs.unsloth-zoo-ref }}+ run: |+ set -euxo pipefail+ # github.com occasionally 500s on the git fetch; retry so a+ # single upstream blip does not fail CI.+ # fetch + checkout rather than `clone --branch`: the ref is a commit SHA once+ # resolve-zoo-ref has pinned it, and --branch takes only an advertised branch or+ # tag, so it would fail with "Remote branch <sha> not found in upstream origin".+ # tests/studio/studiobench/runtime/selftest/test_studiobench_lifecycle_refs.py+ # ::test_clone_branch_cannot_take_a_commit asserts that rule. This form takes a+ # branch, a tag or a commit, and stays shallow.+ for attempt in 1 2 3; do+ rm -rf "$RUNNER_TEMP/unsloth-zoo"+ if git init -q "$RUNNER_TEMP/unsloth-zoo" \+ && git -C "$RUNNER_TEMP/unsloth-zoo" remote add origin \+ https://github.com/unslothai/unsloth-zoo \+ && git -C "$RUNNER_TEMP/unsloth-zoo" fetch -q --depth=1 origin "$UNSLOTH_ZOO_REF" \+ && git -C "$RUNNER_TEMP/unsloth-zoo" checkout -q FETCH_HEAD; then+ break+ fi+ if [ "$attempt" -eq 3 ]; then+ echo "::error::git clone unsloth-zoo failed after 3 attempts"+ exit 1+ fi+ delay=$((5 * attempt))+ echo "::warning::clone failed (attempt $attempt/3), retrying in ${delay}s..."+ sleep "$delay"+ done+ pip install -e "$RUNNER_TEMP/unsloth-zoo" --no-deps+ pip show unsloth_zoodiff --git a/.github/workflows/consolidated-tests-ci.yml b/.github/workflows/consolidated-tests-ci.ymlindex 78b15d4d4af..222880c957f 100644--- a/.github/workflows/consolidated-tests-ci.yml+++ b/.github/workflows/consolidated-tests-ci.yml@@ -1,9 +1,18 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-# One consolidated CPU-only job that runs every test_* function the existing-# CI does not already cover from this repo plus the full unsloth_zoo@main-# CPU test suite plus unsloth_zoo.compiler.test_apply_fused_lm_head.+# CPU-only coverage for every test_* function the existing CI does not already+# cover from this repo, plus the full unsloth_zoo@main CPU test suite, plus+# unsloth_zoo.compiler.test_apply_fused_lm_head.+#+# Two matrix jobs over the SAME three (transformers, TRL) combos, not one:+# `consolidated` for this repo's own coverage and `consolidated-zoo` for the+# unsloth_zoo suite. They were one job until the zoo suite, 418s mean over 24+# measured jobs, was 7.6 of its 17.2 minutes and every pull request waited+# through both halves end to end. Nothing was dropped in the split; the two+# halves now run side by side and the pip install preamble, which is the only+# thing they duplicate, has one definition in+# .github/actions/core-cpu-setup.## Why a separate workflow:# - studio-backend-ci.yml's "Repo tests (CPU)" job already auto-discovers@@ -49,6 +58,9 @@ on:- 'pyproject.toml'- '.github/workflows/consolidated-tests-ci.yml'- '.github/scripts/retry-with-apt-lock.sh'+ # Both Core jobs `uses:` this action, so a change to it must run them.+ - '.github/actions/core-cpu-setup/**'+ # Both Core jobs `uses:` this action, so a change to it must run them.push:branches: [main]paths:@@ -59,6 +71,9 @@ on:- 'pyproject.toml'- '.github/workflows/consolidated-tests-ci.yml'- '.github/scripts/retry-with-apt-lock.sh'+ # Both Core jobs `uses:` this action, so a change to it must run them.+ - '.github/actions/core-cpu-setup/**'+ # Both Core jobs `uses:` this action, so a change to it must run them.workflow_dispatch:inputs:unsloth_zoo_ref:@@ -78,7 +93,97 @@ permissions:contents: readjobs:+ resolve-zoo-ref:+ # Both Core halves clone unsloth_zoo. Before the split they shared one checkout per+ # matrix cell, so the compatibility checks and the suite always saw the same revision.+ # Cloning a mutable branch twice loses that: a push to unsloth_zoo between the two clones+ # has `consolidated` check revision A while `consolidated-zoo` runs revision B under one+ # matrix label, and the run is then not reproducible from anything it recorded.+ #+ # Resolving once costs a gating job. That is affordable here and was not while Core sat on+ # ubuntu-latest: on the pinned label a trivial job waited 11s to 162s across the runs in+ # #10914, against the roughly seven minutes the split takes off this workflow.+ #+ # llama-cpp-smoke still resolves its own ref. It shares no verdict with these two, so a+ # different revision there cannot make one cell disagree with itself.+ name: Resolve the unsloth_zoo revision+ runs-on: ubuntu-24.04+ timeout-minutes: 5+ outputs:+ sha: ${{ steps.resolve.outputs.sha }}+ steps:+ - name: Resolve+ id: resolve+ env:+ REF: ${{ inputs.unsloth_zoo_ref || 'main' }}+ run: |+ set -euo pipefail+ # A ref that is already a commit has no remote head to look up, so pass it through.+ if printf '%s' "$REF" | grep -Eq '^[0-9a-f]{40}$'; then+ sha="$REF"+ else+ # Retried like the clone in core-cpu-setup, and for the same reason: github.com+ # occasionally 500s. This job gates both Core halves, so an unretried blip here+ # would skip all six matrix jobs rather than fail one step.+ # `git ls-remote` takes PATTERNS, not one ref, so a bare name can print more than+ # one record: a name that is both a branch and a tag prints both, and an annotated+ # tag prints the tag object and the commit it peels to. Taking every record puts a+ # newline inside `sha`, and `sha=<a>\n<b>` makes the second line of $GITHUB_OUTPUT a+ # bare value that GitHub rejects, failing this job and skipping all six Core cells.+ # So ask for the exact refs and pick one, in git's own order: a branch beats a tag,+ # and an annotated tag resolves to the commit rather than to the tag object.+ resolve_ref() {+ local ref="$1" out want line uniq found+ out="$(git ls-remote https://github.com/unslothai/unsloth-zoo \+ "refs/heads/$ref" "refs/tags/$ref^{}" "refs/tags/$ref" "$ref")" || return 1+ for want in "refs/heads/$ref" "refs/tags/$ref^{}" "refs/tags/$ref"; do+ line="$(printf '%s\n' "$out" | awk -v w="$want" '$2 == w { print $1; exit }')"+ if [ -n "$line" ]; then+ printf '%s\n' "$line"+ return 0+ fi+ done+ # Not a plain branch or tag name. The trailing pattern above still covers a fully+ # qualified ref, but only one commit may answer to it.+ uniq="$(printf '%s\n' "$out" | awk 'NF { print $1 }' | sort -u)"+ found="$(printf '%s\n' "$uniq" | grep -c . || true)"+ # No match is the same shape as a failed lookup and stays retryable; several+ # matches is a statement about the remote that retrying cannot change.+ if [ "$found" -eq 0 ]; then+ return 1+ fi+ if [ "$found" -ne 1 ]; then+ return 2+ fi+ printf '%s\n' "$uniq"+ }+ sha=""+ for attempt in 1 2 3; do+ if sha="$(resolve_ref "$REF")"; then+ if [ -n "$sha" ]; then break; fi+ elif [ "$?" -eq 2 ]; then+ # Retrying cannot disambiguate a name several commits answer to, and warning+ # about a network blip that did not happen would send the reader the wrong way.+ echo "::error::unsloth_zoo ref '$REF' matches more than one commit on the remote"+ exit 1+ fi+ if [ "$attempt" -eq 3 ]; then+ break+ fi+ delay=$((5 * attempt))+ echo "::warning::ls-remote failed (attempt $attempt/3), retrying in ${delay}s..."+ sleep "$delay"+ done+ fi+ if [ -z "$sha" ]; then+ echo "::error::could not resolve unsloth_zoo ref '$REF' to a commit"+ exit 1+ fi+ echo "sha=$sha" >> "$GITHUB_OUTPUT"+ echo "unsloth_zoo $REF is $sha"+consolidated:+ needs: resolve-zoo-ref# Matrix: three (transformers, TRL) combos cover the failure surface the# PR cares about:# 1. transformers==4.57.6 + TRL latest <1.0.0 (the just-before-5.x line)@@ -119,12 +224,11 @@ jobs:# the runtime check shim handles patches that legitimately cannot# run on a CPU-only runner (real CUDA dispatcher, runtime args).env:- UNSLOTH_ZOO_REF: ${{ inputs.unsloth_zoo_ref || 'main' }}- MATRIX_TRANSFORMERS_SPEC: ${{ matrix.combo.transformers_spec }}- MATRIX_TRL_SPEC: ${{ matrix.combo.trl_spec }}+ UNSLOTH_ZOO_REF: ${{ needs.resolve-zoo-ref.outputs.sha }}MATRIX_COMBO_ID: ${{ matrix.combo.id }}- # Hoisted to job-level so every step (Sanity, Bucket-A, unsloth_zoo- # pytest, test_apply_fused_lm_head) inherits it. transformers' bundled+ # Hoisted to job-level so every step (Sanity, Bucket-A, the static and+ # runtime checks) inherits it, and repeated in `consolidated-zoo` for the+ # same reason. transformers' bundled# *_pb2.py was generated against an older protoc; the C++ protobuf# 4+/5+/6 implementation rejects them with "Descriptors cannot be# created directly". The pure-Python parser bypasses the check; the@@ -157,138 +261,16 @@ jobs:pyproject.tomlstudio/backend/requirements/*.txt- # Node 22 unblocks tests/studio/test_chat_preset_builtin_invariants.py's- # `node --experimental-strip-types` subprocess. Cheap to install; keeps- # the consolidated job self-sufficient even if studio-backend-ci.yml- # changes its node setup.- - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0+ # node + uv + spec resolution + the ~2 GB install + the unsloth_zoo+ # clone. Shared verbatim with the `consolidated-zoo` job below, which+ # runs the same three combos against the same stack; see the action for+ # why it owns these five steps and not the four above it.+ - name: Core CPU setup+ uses: ./.github/actions/core-cpu-setupwith:- node-version: '22'-- - name: Install uv (some unsloth_zoo dev tooling expects it on PATH)- run: pip install uv-- - name: Resolve matrix specs (handle __from_pyproject__ sentinel)- # The pyproject cell uses a sentinel; resolve the real `transformers`- # and `trl` constraints from the project's pyproject.toml at job time.- # unsloth's pyproject puts the LLM stack pins in- # [project.optional-dependencies] under the `huggingfacenotorch`- # extra (top-level [project.dependencies] is just typer/pydantic/etc.),- # so we walk every optional extra and pick the first matching spec.- # Other cells pass their spec through unchanged.- run: |- set -euxo pipefail- python <<'PY' >> "$GITHUB_ENV"- import os, re, tomllib- spec_t = os.environ["MATRIX_TRANSFORMERS_SPEC"]- spec_r = os.environ["MATRIX_TRL_SPEC"]-- def _pkg_name(spec: str) -> str:- m = re.match(r"\s*([A-Za-z0-9_.-]+)", spec)- return (m.group(1).lower() if m else "")-- if spec_t == "__from_pyproject__" or spec_r == "__from_pyproject__":- with open("pyproject.toml", "rb") as f:- doc = tomllib.load(f)- proj = doc.get("project", {})- # Try top-level deps first, then all optional extras.- all_deps: list[str] = list(proj.get("dependencies", []))- for _name, dep_list in proj.get("optional-dependencies", {}).items():- all_deps.extend(dep_list)-- if spec_t == "__from_pyproject__":- spec_t = next((x for x in all_deps if _pkg_name(x) == "transformers"),- "transformers")- if spec_r == "__from_pyproject__":- spec_r = next((x for x in all_deps if _pkg_name(x) == "trl"),- "trl")- print(f"RESOLVED_TRANSFORMERS_SPEC={spec_t}")- print(f"RESOLVED_TRL_SPEC={spec_r}")- PY- # Echo to logs so the matrix cell label maps cleanly to a spec.- grep RESOLVED_ "$GITHUB_ENV" || true-- - name: Install runtime deps (mirrors studio-backend-ci.yml + mlx-ci.yml)- # The shape matches studio-backend-ci.yml's "Repo tests (CPU)" install- # so we inherit the same CPU-spoof harness in tests/conftest.py and- # the same import-chain guarantees, plus the extra deps that the- # tests/saving + tests/utils Bucket-A files transitively need but- # which Repo tests (CPU) does not require because it --ignores- # those directories:- # - protobuf + sentencepiece: tests/saving/test_fix_sentencepiece_gguf_robustness.py- # does `from transformers.utils import sentencepiece_model_pb2`,- # which imports `google.protobuf`. Not pulled by transformers'- # base install.- # - triton: unsloth/_gpu_init.py:232 does an unconditional- # `import triton`. The triton PyPI wheel installs cleanly on- # Linux x86_64 even without CUDA (the import succeeds; runtime- # GPU work is what would fail, which we never do here).- # transformers + trl are matrix-parameterized.- run: |- set -euxo pipefail- python -m pip install --upgrade pip- pip install -r studio/backend/requirements/studio.txt- # CPU torch FIRST, before anything that depends on torch. peft, accelerate and- # datasets all declare a torch dependency, so installing them first made pip- # resolve it from PyPI: the default CUDA build, plus 14 nvidia_* wheels and- # cuda-toolkit, roughly 2.3 GB, which the pinned CPU wheel below then replaced- # seconds later. Same end state either way; this just stops paying for the- # download. torchvision: unsloth_zoo.vision_utils imports it at module scope.- pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple \- 'torch>=2.4,<2.11' 'torchvision<0.26'- pip install \- python-multipart aiofiles sqlalchemy cryptography \- pyyaml jinja2 mammoth unpdf requests typer \- 'numpy<3' pytest==9.0.3 pytest-asyncio pytest-xdist pytest-timeout httpx \- protobuf sentencepiece triton \- psutil packaging tqdm safetensors datasets \- 'peft>=0.18,<0.20' 'accelerate>=0.34,<2' \- ipython- # transformers + trl from the matrix combo.- pip install "$RESOLVED_TRANSFORMERS_SPEC"- pip install "$RESOLVED_TRL_SPEC"- # bitsandbytes: hard import in unsloth/models/_utils.py. Recent- # versions ship a CPU build that imports cleanly on Linux.- pip install 'bitsandbytes>=0.45'- # unsloth itself, editable, no-deps so pip does not fight the- # explicit torch CPU-index install above.- pip install -e . --no-deps- echo "::group::Installed transformers + trl + torch + unsloth versions"- pip show transformers- pip show trl- pip show torch- pip show unsloth- echo "::endgroup::"-- - name: Clone unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }}- # We need the repository tree (the wheel does not ship tests/), so- # clone shallow then editable-install so unsloth_zoo.* imports- # resolve to the cloned tree. We use `pip show` for the location- # check rather than `import unsloth_zoo` because the latter calls- # device_type.get_device_type() at module load and raises on a- # GPU-less runner; pytest steps below route through the existing- # tests/conftest.py spoof which handles that.- run: |- set -euxo pipefail- # github.com occasionally 500s on the git fetch; retry so a- # single upstream blip does not fail CI.- for attempt in 1 2 3; do- rm -rf "$RUNNER_TEMP/unsloth-zoo"- if git clone --depth=1 --branch="$UNSLOTH_ZOO_REF" \- https://github.com/unslothai/unsloth-zoo \- "$RUNNER_TEMP/unsloth-zoo"; then- break- fi- if [ "$attempt" -eq 3 ]; then- echo "::error::git clone unsloth-zoo failed after 3 attempts"- exit 1- fi- delay=$((5 * attempt))- echo "::warning::clone failed (attempt $attempt/3), retrying in ${delay}s..."- sleep "$delay"- done- pip install -e "$RUNNER_TEMP/unsloth-zoo" --no-deps- pip show unsloth_zoo+ transformers-spec: ${{ matrix.combo.transformers_spec }}+ trl-spec: ${{ matrix.combo.trl_spec }}+ unsloth-zoo-ref: ${{ env.UNSLOTH_ZOO_REF }}- name: Sanity — collection only (both repos)# Catches import-time breakage before we run the suite. Cheap; bails@@ -463,117 +445,6 @@ jobs:# until attention_dispatch.py predefined flash_attn_varlen_func as None; it# monkeypatches that name, so it no longer needs flash_attn on this runner.- - name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)- # 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip- # cases below auto-skip on a GPU-less runner; deselect them- # explicitly so the no-CUDA outcome is "deselected", not "skipped",- # making intent visible in the report. Env inherited from job block.- #- # test_get_peft_model_passes_finetune_last_n_layers_through is- # deselected because unsloth_zoo/mlx/loader.py at line 2972 calls- # model.trainable_parameters() on the fake-model fixture, which- # the test never stubbed; this fails on every platform regardless- # of CUDA. Tracked upstream as an unsloth_zoo bug; deselecting- # here unblocks unsloth CI until the loader fixture is fixed.- #- # A test that hangs rather than fails used to consume the whole 35 minute- # job budget and report nothing usable: the job ended as "cancelled", with- # no test name, no traceback, and every later step marked skipped. That is- # how unsloth_zoo's gather_qmm guard walking a chain forever under the mlx- # shim presented. PYTEST_TIMEOUT is read by pytest-timeout on every pytest- # invocation in this step, so one setting covers the xdist run and each- # serial file below, instead of the same flag on six command lines.- #- # 5 minutes. The slowest honest test in the suite is about 48s locally- # (test_fla_hopper_tile.py), and CI runs roughly 1.65x slower than that- # box, measured on the one test long enough to compare, so call it 80s- # here. That leaves well over 3x headroom, while a genuine hang is now- # named and stack-dumped in 5 minutes rather than silently eating all 35.- #- # This only became a sane number once unsloth_zoo stopped rebuilding- # dir(transformers.models) on every call (unsloth-zoo#1139): before that,- # test_every_shipped_model_type_accepted alone ran past 600s, so any- # useful timeout would have failed a passing test.- env:- PYTEST_TIMEOUT: 300- working-directory: ${{ runner.temp }}/unsloth-zoo- run: |- # Keep tests within each file on one worker. MLX shims contaminate sys.modules,- # so exclude that family from xdist and run it separately below. The other- # ignores either leak process state or use timing assertions that need a quiet- # process.- python -m pytest -q --tb=short -n 4 --dist loadfile tests/ \- --ignore-glob='tests/test_mlx_*.py' \- --ignore=tests/test_moe_bnb4bit_per_expert_conversions.py \- --ignore=tests/test_hf_xet_fallback.py \- --ignore=tests/test_gemma3_forced_float32_boundary_dtype.py \- --deselect tests/test_unsloth_zoo_lora_merge.py::test_active_merge_device_returns_string_on_cuda_host \- --deselect tests/test_unsloth_zoo_lora_merge.py::test_merge_lora_moves_cpu_inputs_to_active_device-- # Run contamination-sensitive files in separate processes. Keep collecting- # results after failures, and treat pytest exit 5 as a module-level skip.- rc=0- _keep() {- if [ "$1" = 5 ]; then- # Named: an expected module-level skip and a file that stopped collecting- # for a new reason both exit 5.- echo "note: ${2:-that run} collected nothing (module-level skip), not a failure"- else- rc=1- fi- }- python -m pytest -q --tb=short tests/test_mlx_generate.py || _keep "$?" tests/test_mlx_generate.py- python -m pytest -q --tb=short tests/test_moe_bnb4bit_per_expert_conversions.py || _keep "$?" tests/test_moe_bnb4bit_per_expert_conversions.py- python -m pytest -q --tb=short tests/test_hf_xet_fallback.py || _keep "$?" tests/test_hf_xet_fallback.py- python -m pytest -q --tb=short tests/test_mlx_neftune_quant_map.py || _keep "$?" tests/test_mlx_neftune_quant_map.py- python -m pytest -q --tb=short tests/test_mlx_gated_delta_vjp.py || _keep "$?" tests/test_mlx_gated_delta_vjp.py- python -m pytest -q --tb=short tests/test_gemma3_forced_float32_boundary_dtype.py || _keep "$?" tests/test_gemma3_forced_float32_boundary_dtype.py-- # Run the remaining MLX files serially in alphabetical order. The three isolated- # above also contaminate this group, so exclude them from its file list.- mlx_group=$(ls tests/test_mlx_*.py |- grep -vE 'tests/test_mlx_(generate|neftune_quant_map|gated_delta_vjp)\.py$')- # Passed unquoted below to word-split into file arguments, so an empty list- # would pass none and pytest would collect the whole rootdir instead.- if [ -z "$mlx_group" ]; then- echo "::error::the mlx group glob matched no files"- exit 1- fi- python -m pytest -q --tb=short $mlx_group \- --deselect tests/test_mlx_finetune_last_n_layers.py::test_get_peft_model_passes_finetune_last_n_layers_through \- || _keep "$?" "the mlx group"- exit "$rc"-- - name: unsloth_zoo — test_apply_fused_lm_head (lives in compiler.py)- # `test_apply_fused_lm_head` lives at unsloth_zoo/compiler.py:1983,- # not under tests/, so pytest's default discovery does not pick it up.- # We route it through pytest by writing a one-shot shim test file- # inside the unsloth checkout's tests/ — pytest then walks UP and- # picks up tests/conftest.py, whose GPU-spoof harness (lines 84-141)- # patches torch.cuda.is_available, torch.cuda.memory.mem_get_info,- # torch.cuda.get_device_capability, and is_bf16_supported. That full- # spoof is required because unsloth_zoo/temporary_patches/gpt_oss.py- # at module load reads torch.cuda.memory.mem_get_info(0), which- # bare `is_available = True` doesn't cover. Env inherited.- run: |- set -euxo pipefail- cat > tests/_zoo_apply_fused_lm_head_shim.py <<'PY'- # Auto-generated by .github/workflows/consolidated-tests-ci.yml.- # Wraps unsloth_zoo.compiler.test_apply_fused_lm_head so that- # tests/conftest.py's GPU-spoof harness applies before the import.- # _zoo_aggressive_cuda_spoof extends conftest's harness with deeper- # patches (see tests/_zoo_aggressive_cuda_spoof.py).- import sys, pathlib- sys.path.insert(0, str(pathlib.Path(__file__).parent))- import _zoo_aggressive_cuda_spoof as _spoof- _spoof.apply()- from unsloth_zoo.compiler import test_apply_fused_lm_head as _zoo_test- def test_zoo_apply_fused_lm_head_runs():- _zoo_test()- PY- python -m pytest -q --tb=short tests/_zoo_apply_fused_lm_head_shim.py- rm -f tests/_zoo_apply_fused_lm_head_shim.py-- name: Static checks — unsloth/trainer.py + unsloth/models/rl.py against latest pip TRL# AST-only sanity: confirm both files parse and that every TRL symbol# they reference still exists in the installed `trl`. Catches API@@ -2343,6 +2214,244 @@ jobs:echo "Consolidated job done. Coverage:"echo " - 24 unsloth Bucket-A tests under tests/saving/ + tests/utils/"echo " - 168 dataset_num_proc worker-bound guards (#2693)"+ echo " - the unsloth_zoo suite + test_apply_fused_lm_head run in the"+ echo " sibling 'Core zoo' job, on the same three combos"++ - name: Save the pip cache+ if: always()+ uses: ./.github/actions/pip-cache-save+ with:+ dir: ${{ steps.pip-cache.outputs.dir }}+ key: ${{ steps.pip-cache.outputs.key }}+ cache-hit: ${{ steps.pip-cache.outputs.cache-hit }}++ # The unsloth_zoo suite, lifted out of the `consolidated` cell above.+ #+ # Measured over 24 real jobs, the "unsloth_zoo @ <ref> - full pytest (CPU)"+ # step runs 418s mean, 341-464s range, call it 457s at the top of the band.+ # It used to sit serially inside a cell whose total is ~17.2 min, so it was+ # ~7.6 min of pure critical path on 100% of pull requests: Core's+ # `on.pull_request` paths filter matches `unsloth/**`, `studio/**`,+ # `tests/**` and `pyproject.toml`, which is every source change this repo+ # makes.+ #+ # Moving it into its own job over the SAME three combos removes no work, only+ # the ordering. Core's critical path goes from ~17.2 min to roughly 10, the+ # longer of the two halves, and the price is three more runners that each+ # re-pay the install preamble. That trade is the explicit objective: fewer+ # minutes of waiting per pull request, not fewer runner-minutes.+ #+ # A separate WORKFLOW file was the alternative and is worse on two counts.+ # The combo list would have to be duplicated across files rather than sitting+ # beside the job it mirrors, and the two halves would fall into different+ # concurrency groups, so a force-push would cancel one and leave the other+ # running against the commit it superseded.+ consolidated-zoo:+ needs: resolve-zoo-ref+ # Byte-identical to `consolidated`'s matrix, deliberately: the coverage+ # claim for this split is that the same suite still runs under the same+ # three (transformers, TRL) pins. GitHub Actions supports no YAML anchors+ # and no way to share a matrix between jobs, so it is a copy, and+ # tests/studio/test_zoo_suite_parallel_isolation.py fails if the two copies+ # stop agreeing.+ strategy:+ fail-fast: false+ matrix:+ combo:+ - id: t4576-trl0latest+ label: "HF=4.57.6 + TRL<1"+ transformers_spec: "transformers==4.57.6"+ trl_spec: "trl>=0.18.2,<1.0.0"+ - id: tlatest5-trl1latest+ label: "HF=latest + TRL=latest"+ transformers_spec: "transformers>=5,<6"+ trl_spec: "trl>=1,<2"+ - id: pyproject+ label: "HF=default + TRL=default"+ transformers_spec: "__from_pyproject__"+ trl_spec: "__from_pyproject__"+ name: "Core zoo (${{ matrix.combo.label }})"+ runs-on: ubuntu-24.04+ # 25 rather than `consolidated`'s 35. This job runs the ~7.6 min zoo suite+ # on top of the preamble and nothing else, and the cap is what turns a hung+ # test into a bounded failure; see PYTEST_TIMEOUT in the step below, which+ # is the mechanism that actually names the culprit.+ timeout-minutes: 25+ # Same contract as `consolidated`: no job-level or per-step+ # `continue-on-error`, so a failing test fails the cell.+ env:+ UNSLOTH_ZOO_REF: ${{ needs.resolve-zoo-ref.outputs.sha }}+ MATRIX_COMBO_ID: ${{ matrix.combo.id }}+ # Identical to `consolidated`'s env block, and asserted identical by+ # tests/studio/test_zoo_suite_parallel_isolation.py. The reasoning for+ # each entry lives there; repeating it here would give it two homes.+ PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python+ PYTHONPATH: ${{ github.workspace }}/studio+ UNSLOTH_COMPILE_DISABLE: '1'+ UNSLOTH_IS_PRESENT: '1'+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ with:+ persist-credentials: false++ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0+ with:+ python-version: '3.12'++ - name: Restore the pip cache+ id: pip-cache+ # Its own `name`, not `consolidated`'s. Two jobs sharing one is the+ # defect tests/studio/test_pip_cache_naming.py exists to reject: the+ # save is gated on `cache-hit != 'true'`, so whichever job finished+ # first on main would write the entry and the other would restore it+ # and never save its own. The key-files are the same because the+ # install is the same, which that file also allows for explicitly.+ #+ # The first run on a pull request therefore misses and downloads. The+ # entry is written by the next push to main and warm from then on, the+ # same lifecycle every other key here has.+ uses: ./.github/actions/pip-cache-restore+ with:+ name: consolidated-zoo+ key-files: |+ pyproject.toml+ studio/backend/requirements/*.txt++ # The identical preamble `consolidated` runs, from the one definition+ # both share. Anything added to the install belongs in that action, not+ # in either caller.+ - name: Core CPU setup+ uses: ./.github/actions/core-cpu-setup+ with:+ transformers-spec: ${{ matrix.combo.transformers_spec }}+ trl-spec: ${{ matrix.combo.trl_spec }}+ unsloth-zoo-ref: ${{ env.UNSLOTH_ZOO_REF }}++ - name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)+ # 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip+ # cases below auto-skip on a GPU-less runner; deselect them+ # explicitly so the no-CUDA outcome is "deselected", not "skipped",+ # making intent visible in the report. Env inherited from job block.+ #+ # test_get_peft_model_passes_finetune_last_n_layers_through is+ # deselected because unsloth_zoo/mlx/loader.py at line 2972 calls+ # model.trainable_parameters() on the fake-model fixture, which+ # the test never stubbed; this fails on every platform regardless+ # of CUDA. Tracked upstream as an unsloth_zoo bug; deselecting+ # here unblocks unsloth CI until the loader fixture is fixed.+ #+ # A test that hangs rather than fails used to consume the whole job+ # budget and report nothing usable: the job ended as "cancelled", with+ # no test name, no traceback, and every later step marked skipped. That is+ # how unsloth_zoo's gather_qmm guard walking a chain forever under the mlx+ # shim presented. PYTEST_TIMEOUT is read by pytest-timeout on every pytest+ # invocation in this step, so one setting covers the xdist run and each+ # serial file below, instead of the same flag on six command lines.+ #+ # 5 minutes. The slowest honest test in the suite is about 48s locally+ # (test_fla_hopper_tile.py), and CI runs roughly 1.65x slower than that+ # box, measured on the one test long enough to compare, so call it 80s+ # here. That leaves well over 3x headroom, while a genuine hang is now+ # named and stack-dumped in 5 minutes rather than silently eating the+ # job's whole 25 minute cap.+ #+ # This only became a sane number once unsloth_zoo stopped rebuilding+ # dir(transformers.models) on every call (unsloth-zoo#1139): before that,+ # test_every_shipped_model_type_accepted alone ran past 600s, so any+ # useful timeout would have failed a passing test.+ env:+ PYTEST_TIMEOUT: 300+ working-directory: ${{ runner.temp }}/unsloth-zoo+ run: |+ # Keep tests within each file on one worker. MLX shims contaminate sys.modules,+ # so exclude that family from xdist and run it separately below. The other+ # ignores either leak process state or use timing assertions that need a quiet+ # process.+ python -m pytest -q --tb=short -n 4 --dist loadfile tests/ \+ --ignore-glob='tests/test_mlx_*.py' \+ --ignore=tests/test_moe_bnb4bit_per_expert_conversions.py \+ --ignore=tests/test_hf_xet_fallback.py \+ --ignore=tests/test_gemma3_forced_float32_boundary_dtype.py \+ --deselect tests/test_unsloth_zoo_lora_merge.py::test_active_merge_device_returns_string_on_cuda_host \+ --deselect tests/test_unsloth_zoo_lora_merge.py::test_merge_lora_moves_cpu_inputs_to_active_device++ # Run contamination-sensitive files in separate processes. Keep collecting+ # results after failures, and treat pytest exit 5 as a module-level skip.+ rc=0+ _keep() {+ if [ "$1" = 5 ]; then+ # Named: an expected module-level skip and a file that stopped collecting+ # for a new reason both exit 5.+ echo "note: ${2:-that run} collected nothing (module-level skip), not a failure"+ else+ rc=1+ fi+ }+ python -m pytest -q --tb=short tests/test_mlx_generate.py || _keep "$?" tests/test_mlx_generate.py+ python -m pytest -q --tb=short tests/test_moe_bnb4bit_per_expert_conversions.py || _keep "$?" tests/test_moe_bnb4bit_per_expert_conversions.py+ python -m pytest -q --tb=short tests/test_hf_xet_fallback.py || _keep "$?" tests/test_hf_xet_fallback.py+ python -m pytest -q --tb=short tests/test_mlx_neftune_quant_map.py || _keep "$?" tests/test_mlx_neftune_quant_map.py+ python -m pytest -q --tb=short tests/test_mlx_gated_delta_vjp.py || _keep "$?" tests/test_mlx_gated_delta_vjp.py+ python -m pytest -q --tb=short tests/test_gemma3_forced_float32_boundary_dtype.py || _keep "$?" tests/test_gemma3_forced_float32_boundary_dtype.py++ # Run the remaining MLX files serially in alphabetical order. The three isolated+ # above also contaminate this group, so exclude them from its file list.+ mlx_group=$(ls tests/test_mlx_*.py |+ grep -vE 'tests/test_mlx_(generate|neftune_quant_map|gated_delta_vjp)\.py$')+ # Passed unquoted below to word-split into file arguments, so an empty list+ # would pass none and pytest would collect the whole rootdir instead.+ if [ -z "$mlx_group" ]; then+ echo "::error::the mlx group glob matched no files"+ exit 1+ fi+ python -m pytest -q --tb=short $mlx_group \+ --deselect tests/test_mlx_finetune_last_n_layers.py::test_get_peft_model_passes_finetune_last_n_layers_through \+ || _keep "$?" "the mlx group"+ exit "$rc"++ - name: unsloth_zoo — test_apply_fused_lm_head (lives in compiler.py)+ # `test_apply_fused_lm_head` lives at unsloth_zoo/compiler.py:1983,+ # not under tests/, so pytest's default discovery does not pick it up.+ # We route it through pytest by writing a one-shot shim test file+ # inside the unsloth checkout's tests/ — pytest then walks UP and+ # picks up tests/conftest.py, whose GPU-spoof harness (lines 84-141)+ # patches torch.cuda.is_available, torch.cuda.memory.mem_get_info,+ # torch.cuda.get_device_capability, and is_bf16_supported. That full+ # spoof is required because unsloth_zoo/temporary_patches/gpt_oss.py+ # at module load reads torch.cuda.memory.mem_get_info(0), which+ # bare `is_available = True` doesn't cover. Env inherited.+ run: |+ set -euxo pipefail+ cat > tests/_zoo_apply_fused_lm_head_shim.py <<'PY'+ # Auto-generated by .github/workflows/consolidated-tests-ci.yml.+ # Wraps unsloth_zoo.compiler.test_apply_fused_lm_head so that+ # tests/conftest.py's GPU-spoof harness applies before the import.+ # _zoo_aggressive_cuda_spoof extends conftest's harness with deeper+ # patches (see tests/_zoo_aggressive_cuda_spoof.py).+ import sys, pathlib+ sys.path.insert(0, str(pathlib.Path(__file__).parent))+ import _zoo_aggressive_cuda_spoof as _spoof+ _spoof.apply()+ from unsloth_zoo.compiler import test_apply_fused_lm_head as _zoo_test+ def test_zoo_apply_fused_lm_head_runs():+ _zoo_test()+ PY+ python -m pytest -q --tb=short tests/_zoo_apply_fused_lm_head_shim.py+ rm -f tests/_zoo_apply_fused_lm_head_shim.py++ - name: Summary+ if: always()+ run: |+ echo "::group::Versions"+ python -c "import sys, platform; print(sys.version); print(platform.platform())"+ python -c "import torch; print('torch', torch.__version__, 'cuda?', torch.cuda.is_available())"+ python -c "import transformers; print('transformers', transformers.__version__)"+ # `pip show` instead of `import unsloth_zoo` — its __init__ raises+ # without an accelerator and the spoof harness only kicks in under+ # pytest. Cheap and accurate.+ pip show unsloth_zoo+ echo "::endgroup::"+ echo "Core zoo job done. Coverage:"echo " - unsloth_zoo @ ${UNSLOTH_ZOO_REF} pytest tests/ (5 GPU cases deselected)"echo " - unsloth_zoo.compiler.test_apply_fused_lm_head"diff --git a/.github/workflows/workflow-trigger-lint.yml b/.github/workflows/workflow-trigger-lint.ymlindex 5a9799decd0..3f4ae63c57a 100644--- a/.github/workflows/workflow-trigger-lint.yml+++ b/.github/workflows/workflow-trigger-lint.yml@@ -134,6 +134,7 @@ jobs:tests/studio/test_backend_ci_matrix.py \tests/studio/test_backend_ci_parallel_isolation.py \tests/studio/test_cache_budget_discipline.py \+ tests/studio/test_local_actions_are_in_path_filters.py \tests/studio/test_chat_ui_shards_cover_everything.py \tests/studio/test_ui_shard_engines.py \tests/studio/test_zoo_suite_parallel_isolation.py \diff --git a/tests/studio/test_cache_budget_discipline.py b/tests/studio/test_cache_budget_discipline.pyindex d1d9bed8757..75cd132585e 100644--- a/tests/studio/test_cache_budget_discipline.py+++ b/tests/studio/test_cache_budget_discipline.py@@ -63,6 +63,13 @@ WORKFLOWS = REPO / ".github" / "workflows"# splits the halves so the save can be gated on the default branch.PIP_CACHE_JOBS = {("consolidated-tests-ci.yml", "consolidated"),+ # The unsloth_zoo half of Core, split out so its 418s suite runs beside the rest+ # instead of after it. It runs the same install as `consolidated` (both call+ # .github/actions/core-cpu-setup), which is exactly the torch/transformers-class+ # download this allowlist exists for, and it needs its OWN name rather than sharing+ # `consolidated`'s: the save is gated on `cache-hit != 'true'`, so a shared key means+ # whichever job finishes first on main writes it and the other never saves.+ ("consolidated-tests-ci.yml", "consolidated-zoo"),("consolidated-tests-ci.yml", "llama-cpp-smoke"),("mlx-ci.yml", "dispatch"),("notebooks-ci.yml", "api-introspect"),diff --git a/tests/studio/test_local_actions_are_in_path_filters.py b/tests/studio/test_local_actions_are_in_path_filters.pynew file mode 100644index 00000000000..805a647d869--- /dev/null+++ b/tests/studio/test_local_actions_are_in_path_filters.py@@ -0,0 +1,225 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0++"""A path-filtered workflow must list the local actions it `uses:`.++Factoring a workflow's setup into `.github/actions/<name>/` moves real logic outside every+path in that workflow's `paths:` filter. The workflow then does not run when only the action+changes, so a PR can break the install preamble of a job and watch the job not run. The+failure is silent in the worst way: the checks that would have caught it are simply absent,+and the PR is green because nothing looked.++`.github/scripts/retry-with-apt-lock.sh` is already carried in two filters for this reason.+This makes that a rule rather than a thing someone remembered once.++A workflow with no `paths:` filter runs on everything and is not at risk, so it is skipped.+"""++import re+from pathlib import Path++import pytest+import yaml+++REPO_ROOT = Path(__file__).resolve().parents[2]+_WORKFLOWS = REPO_ROOT / ".github" / "workflows"++# The one action pair still missing from the filters of the six workflows that use it+# (consolidated-tests-ci, mlx-ci, notebooks-ci, studio-backend-ci, studio-export-capability-ci,+# version-compat-ci). Every other local action in the repo is already listed, by its+# action.yml, which is the convention this guard enforces.+#+# Waived rather than fixed here because adding them means touching six workflows unrelated to+# this change. The test below deletes the waiver's right to exist as soon as that is done.+_PRE_EXISTING = {+ ".github/actions/pip-cache-restore",+ ".github/actions/pip-cache-save",+}+++def _triggers(doc: dict) -> dict:+ # `on:` is the YAML 1.1 boolean True once parsed, unless it was quoted.+ return doc.get(True) or doc.get("on") or {}+++def _resolve(used: str) -> str:+ """The action's real location in this repo, given a `uses:` path.++ Some workflows check the repo into a subdirectory and so say+ `uses: ./unsloth/.github/actions/pip-cache-restore`. No such directory exists here; the+ action being referenced is the repo-root one. Taking the literal string would put a path+ under `unsloth/` into the check, where a `unsloth/**` filter entry satisfies it and the+ guard passes on a workflow that would still skip. So leading segments are dropped until+ the result is a directory that actually holds an action.+ """+ parts = used.split("/")+ for start in range(len(parts)):+ candidate = "/".join(parts[start:])+ if (REPO_ROOT / candidate / "action.yml").is_file():+ return candidate+ return used+++def _local_actions(node) -> set:+ """Every `uses: ./path` in the workflow, at any depth."""+ found = set()+ if isinstance(node, dict):+ uses = node.get("uses")+ if isinstance(uses, str) and uses.startswith("./"):+ found.add(_resolve(uses[2:].rstrip("/")))+ for value in node.values():+ found |= _local_actions(value)+ elif isinstance(node, list):+ for value in node:+ found |= _local_actions(value)+ return found+++def _matches(pattern: str, path: str) -> bool:+ """One `paths:` glob against one file path, with GitHub's wildcard semantics.++ `path` is a FILE inside the action, not the action directory, because that is what+ GitHub matches a filter against. The distinction decides real cases: a bare+ `.github/actions/foo` entry selects only a file literally at that path, so it does NOT+ cover `.github/actions/foo/action.yml` and the workflow still skips.+ """+ regex = re.escape(pattern).replace(r"\*\*", ".*").replace(r"\*", "[^/]*")+ return re.fullmatch(regex, path) is not None+++def _selects(paths: list, path: str) -> bool:+ """Does the whole `paths:` list select `path`?++ The list is ordered and `!` negates, with the LAST matching pattern deciding, so the+ entries cannot be tested independently: `.github/actions/**` followed by+ `!.github/actions/foo/**` does not select foo. This repo already uses negation+ (startup-profile-ci.yml excludes `!studio/backend/tests/**`), so reading `!` as a+ literal character would quietly pass a workflow that really does skip.+ """+ selected = False+ for entry in paths:+ entry = str(entry).strip("'\"")+ negated = entry.startswith("!")+ if _matches(entry[1:] if negated else entry, path):+ selected = not negated+ return selected+++def _action_files(action: str, root: Path = REPO_ROOT) -> list:+ """Every file GitHub could report as changed for an edit to this action.++ The whole directory, not just the manifest. A composite action that grows a helper+ script keeps working with a filter naming only `action.yml`, and a helper-only change+ would then skip every workflow that uses it, which is the exact failure this guard+ exists to prevent. Listing the directory means such a change has to be covered too,+ by a `/**` entry or by naming the file.+ """+ directory = root / action+ if not directory.is_dir():+ return [f"{action}/action.yml"]+ return sorted(+ str(path.relative_to(root)).replace("\\", "/")+ for path in directory.rglob("*")+ if path.is_file()+ )+++def _workflows() -> list:+ return sorted(_WORKFLOWS.glob("*.yml"))++[email protected]("workflow", _workflows(), ids = lambda p: p.name)+def test_a_path_filtered_workflow_lists_the_actions_it_uses(workflow):+ doc = yaml.safe_load(workflow.read_text(encoding = "utf-8"))+ if not isinstance(doc, dict):+ pytest.skip("not a workflow mapping")+ actions = _local_actions(doc.get("jobs") or {})+ if not actions:+ pytest.skip("uses no local action")++ for event, spec in _triggers(doc).items():+ paths = (spec or {}).get("paths") if isinstance(spec, dict) else None+ if not paths:+ continue # unfiltered: it already runs on any change+ missing = sorted(+ a+ for a in actions - _PRE_EXISTING+ if not all(_selects(paths, f) for f in _action_files(a))+ )+ assert not missing, (+ f"{workflow.name} `{event}` is path-filtered but does not list {missing}, so a "+ f"change to only that action runs none of the jobs that depend on it"+ )+++def test_an_ordered_negation_is_not_read_as_a_literal_bang():+ """`paths:` is ordered and `!` negates, with the last match deciding.++ Testing the entries independently would accept `.github/actions/**` followed by+ `!.github/actions/foo/**` as covering foo, while GitHub skips the workflow on a+ foo-only change. This repo already negates in startup-profile-ci.yml, so the case is+ reachable rather than theoretical.+ """+ action = ".github/actions/foo/action.yml"+ assert _selects([".github/actions/**"], action)+ assert not _selects([".github/actions/**", "!.github/actions/foo/**"], action)+ assert _selects(["!.github/actions/foo/**", ".github/actions/**"], action)+ # The negation this repo actually carries, and a sibling it must not touch.+ assert not _selects(+ ["studio/backend/**", "!studio/backend/tests/**"], "studio/backend/tests/x.py"+ )+ assert _selects(["studio/backend/**", "!studio/backend/tests/**"], "studio/backend/main.py")+++def test_a_helper_file_beside_the_manifest_is_checked_too(tmp_path):+ """Every action here is a lone action.yml today, so this is the case the tree cannot+ show: an action that grows a helper script.++ A filter naming only `<action>/action.yml` still selects the manifest, so checking the+ manifest alone would pass while a helper-only change skipped every consuming workflow.+ """+ action = ".github/actions/grown"+ (tmp_path / action).mkdir(parents = True)+ (tmp_path / action / "action.yml").write_text("name: grown\n")+ (tmp_path / action / "run.sh").write_text("echo hi\n")++ files = _action_files(action, root = tmp_path)+ assert files == [f"{action}/action.yml", f"{action}/run.sh"]++ manifest_only = [f"{action}/action.yml"]+ assert _selects(manifest_only, files[0])+ assert not all(+ _selects(manifest_only, f) for f in files+ ), "naming only the manifest must not satisfy the guard once the action has a helper"+ assert all(_selects([f"{action}/**"], f) for f in files)+++def test_the_pre_existing_list_does_not_outlive_the_problem():+ """An entry that is now listed everywhere it is used must leave the waiver, or the+ waiver quietly re-permits a regression someone already paid to fix."""+ still_unlisted = set()+ for workflow in _workflows():+ doc = yaml.safe_load(workflow.read_text(encoding = "utf-8"))+ if not isinstance(doc, dict):+ continue+ actions = _local_actions(doc.get("jobs") or {})+ for spec in _triggers(doc).values():+ paths = (spec or {}).get("paths") if isinstance(spec, dict) else None+ if not paths:+ continue+ still_unlisted |= {+ a for a in actions if not all(_selects(paths, f) for f in _action_files(a))+ }+ settled = sorted(_PRE_EXISTING - still_unlisted)+ assert not settled, f"these are now listed everywhere and must leave _PRE_EXISTING: {settled}"+++def test_the_guard_sees_a_workflow_that_uses_a_local_action():+ """Vacuity: if nothing in the tree uses a local action, everything above skips."""+ users = [+ w+ for w in _workflows()+ if _local_actions(yaml.safe_load(w.read_text(encoding = "utf-8")) or {})+ ]+ assert users, "no workflow uses a local action, so this guard checks nothing"diff --git a/tests/studio/test_zoo_suite_parallel_isolation.py b/tests/studio/test_zoo_suite_parallel_isolation.pyindex e2c20ac87f1..1d59e9b07ec 100644--- a/tests/studio/test_zoo_suite_parallel_isolation.py+++ b/tests/studio/test_zoo_suite_parallel_isolation.py@@ -1,6 +1,23 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-"""Guard the parallel zoo run and the serial reruns for tests that cannot share workers."""+"""Guard the parallel zoo run, the serial reruns for tests that cannot share workers,+and the split that moved all of them into their own job.++The zoo suite used to run inside the `consolidated` cell, serially, for 418s of that+cell's 17.2 minutes. It now runs in `consolidated-zoo`, a second matrix job over the+same three (transformers, TRL) combos. Two things that were previously true by+construction have to be asserted now that there are two jobs:++ - the suite still runs under all three pins, and still runs at all. A matrix that+ loses a combo, or a job whose steps drift away from the zoo ones, reduces coverage+ without failing anything.+ - the two jobs still install the same environment. The install lives in+ .github/actions/core-cpu-setup so there is one copy of it, but the four steps above+ that action (checkout, setup-python, the pip cache restore) and the job-level `env`+ and `runs-on` are per-job and can drift silently. A zoo job on a different+ transformers than the cell it was split out of would still be green, and would be+ testing something nobody asked for.+"""from __future__ import annotations@@ -8,10 +25,18 @@ import refrom pathlib import Pathimport pytest+import yamlWORKFLOW = (Path(__file__).resolve().parents[2] / ".github" / "workflows" / "consolidated-tests-ci.yml")+ACTION = (+ Path(__file__).resolve().parents[2] / ".github" / "actions" / "core-cpu-setup" / "action.yml"+)++# The two halves of Core. Both run the same matrix; only their test steps differ.+CORE_JOBS = ("consolidated", "consolidated-zoo")+SETUP_ACTION = "./.github/actions/core-cpu-setup"# (ignored path, why it cannot share a worker with the rest of the suite)ISOLATED = [@@ -187,6 +212,149 @@ def test_an_empty_mlx_group_stops_the_step_instead_of_collecting_everything() ->)+def _doc() -> dict:+ return yaml.safe_load(WORKFLOW.read_text(encoding = "utf-8"))+++def _job(jid: str) -> dict:+ jobs = _doc()["jobs"]+ assert jid in jobs, f"{jid} is gone from the workflow; the split it guards is undone"+ return jobs[jid]+++def _job_commands(jid: str) -> list[str]:+ """Every `python -m pytest ...` invocation inside one job, continuations resolved."""+ found = []+ for step in _job(jid)["steps"]:+ joined = re.sub(r"\\\s*\n\s*", " ", str(step.get("run", "")))+ found += [+ " ".join(line.split())+ for line in joined.splitlines()+ if "python -m pytest" in line and not line.lstrip().startswith("#")+ ]+ return found+++def _preamble(jid: str) -> list[dict]:+ """The steps up to and including the shared setup action, which every job repeats."""+ out: list[dict] = []+ for step in _job(jid)["steps"]:+ out.append(step)+ if str(step.get("uses", "")) == SETUP_ACTION:+ return out+ raise AssertionError(+ f"{jid} never calls {SETUP_ACTION}, so it no longer installs the environment "+ f"the other half of Core installs"+ )+++def test_the_zoo_suite_runs_beside_core_and_not_inside_it() -> None:+ """The point of the split. Back inside `consolidated` it is 7.6 min of serial wait."""+ inside = [c for c in _job_commands("consolidated") if ZOO_MARKER in c]+ assert not inside, (+ f"the zoo suite is running inside the `consolidated` cell again ({inside}), so "+ f"every pull request waits through it before the rest of Core can finish"+ )+ beside = [c for c in _job_commands("consolidated-zoo") if ZOO_MARKER in c]+ assert (+ len(beside) == 1+ ), f"expected the parallel zoo run in the `consolidated-zoo` job, found {len(beside)}"+++def test_the_zoo_job_runs_every_zoo_step_that_left_the_cell() -> None:+ """Two steps moved. A move that drops one is a silent deletion of its tests."""+ zoo = " \n".join(_job_commands("consolidated-zoo"))+ assert "_zoo_apply_fused_lm_head_shim.py" in zoo, (+ "unsloth_zoo.compiler.test_apply_fused_lm_head moved out of the consolidated "+ "cell but is not run by the zoo job either, so it runs nowhere"+ )+ for path, _ in ISOLATED:+ assert path in zoo, (+ f"{path}'s serial rerun is not in the zoo job. It is ignored by the parallel "+ f"run, so wherever its rerun went, it has to have gone with it"+ )++[email protected]("jid", CORE_JOBS)+def test_both_halves_of_core_run_the_same_three_combos(jid: str) -> None:+ """Coverage is '3 pins x the same suite'. A matrix that drifts quietly ends that."""+ expected = _doc()["jobs"]["consolidated"]["strategy"]["matrix"]["combo"]+ assert [c["id"] for c in expected] == [+ "t4576-trl0latest",+ "tlatest5-trl1latest",+ "pyproject",+ ], "the Core combo ids changed; update this guard deliberately, not by accident"+ assert _job(jid)["strategy"]["matrix"]["combo"] == expected, (+ f"{jid}'s matrix no longer matches `consolidated`'s. GitHub Actions has no way to "+ f"share a matrix between jobs, so these are two copies, and a copy that drifts "+ f"means the two halves of Core are testing different (transformers, TRL) pins "+ f"while still reporting as one gate"+ )+++def test_both_halves_of_core_share_one_install_preamble() -> None:+ """The install has one definition; the four steps around it are still per-job.++ Checkout, setup-python and the pip cache restore are duplicated by necessity, and a+ difference in any of them (a different interpreter, a cache scoped to other files)+ makes the zoo job test a stack the cell it was split from never runs.+ """+ a, b = (_preamble(jid) for jid in CORE_JOBS)+ assert len(a) == len(b), (+ f"the two Core jobs run {len(a)} and {len(b)} preamble steps. They install the "+ f"same environment, so their preambles have to be the same steps in the same order"+ )+ for left, right in zip(a, b):+ # The pip cache `name` is the one field that MUST differ: a shared name is a+ # shared key, and only the first job to finish on main would ever save.+ # tests/studio/test_pip_cache_naming.py owns that rule.+ left, right = dict(left), dict(right)+ if "pip-cache-restore" in str(left.get("uses", "")):+ left["with"] = {k: v for k, v in left["with"].items() if k != "name"}+ right["with"] = {k: v for k, v in right["with"].items() if k != "name"}+ assert left == right, (+ f"the Core preambles have drifted at step "+ f"{left.get('name') or left.get('uses')!r}:\n consolidated: {left}\n"+ f" consolidated-zoo: {right}"+ )+++def test_both_halves_of_core_share_one_environment_and_one_runner() -> None:+ """`env` and `runs-on` are job-level and cannot be factored into the action.++ `runs-on` is included on purpose. The label is not cosmetic here: measured on this+ repo, `ubuntu-latest` queues behind the org's backlog for a median 44.9 min while any+ other Ubuntu label walks past in minutes, so two halves of one gate on two different+ labels would make the split buy nothing.+ """+ a, b = (_job(jid) for jid in CORE_JOBS)+ assert a["env"] == b["env"], (+ f"the two Core jobs no longer share a job-level env:\n only in consolidated: "+ f"{ {k: v for k, v in a['env'].items() if b['env'].get(k) != v} }\n"+ f" only in consolidated-zoo: "+ f"{ {k: v for k, v in b['env'].items() if a['env'].get(k) != v} }"+ )+ assert a["runs-on"] == b["runs-on"], (+ f"the two halves of Core run on different labels ({a['runs-on']} vs "+ f"{b['runs-on']}), so one of them queues behind a backlog the other skips"+ )+++def test_the_shared_preamble_is_not_also_inlined() -> None:+ """A caller that re-adds an install step is how one definition becomes two."""+ body = "\n".join(str(step.get("run", "")) for jid in CORE_JOBS for step in _job(jid)["steps"])+ for marker in ("pip install -e .", "download.pytorch.org/whl/cpu", "git clone"):+ assert marker not in body, (+ f"{marker!r} is inlined in a Core job again. The install lives in "+ f"{SETUP_ACTION} so both halves cannot drift apart; a second copy is the "+ f"drift, and it is silent until the two jobs disagree about a version"+ )+ action = yaml.safe_load(ACTION.read_text(encoding = "utf-8"))+ run = "\n".join(str(step.get("run", "")) for step in action["runs"]["steps"])+ for marker in ("pip install -e .", "download.pytorch.org/whl/cpu", "git clone"):+ assert marker in run, f"{SETUP_ACTION} no longer does {marker!r}"++def test_a_skipped_isolated_file_is_named_in_the_log() -> None:"""Exit 5 is tolerated, so the file that produced it has to be identifiable: anexpected module-level skip and a file that stopped collecting for a new reason both