Skip to content

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

CI: run lint checks in parallel and overlap independent setup in Windows smoke and Backend CI (#13710)

Daniel Hand975be3d4a81
diff --git a/.github/workflows/lint-ci.yml b/.github/workflows/lint-ci.yml
index 69056019807..061457a6b36 100644
--- a/.github/workflows/lint-ci.yml
+++ b/.github/workflows/lint-ci.yml
@@ -150,58 +150,347 @@ jobs:
bash .github/scripts/retry-with-apt-lock.sh sudo sh -c \
'apt-get install -y --no-install-recommends shellcheck || { apt-get update -qq && apt-get install -y --no-install-recommends shellcheck; }'
- - name: Python AST/syntax check (every committed .py must compile)
- # python -m compileall uses the same parser the interpreter
- # uses, so anything broken here would also crash at
- # `import X` on a user's machine. Sub-second across 350+
- # files. Hard gate.
- run: |
- python -m compileall -q -j 0 \
- unsloth unsloth_cli studio tests cli.py unsloth-cli.py
-
- - name: Dynamic-execution gate self-test
- run: python scripts/lint_exec_literals.py --self-test
-
- - name: No exec, eval or compile of a value that is not written out
- run: python scripts/lint_exec_literals.py
-
- - name: Scanner-shape gate self-test
- run: python scripts/lint_av_shapes.py --self-test
-
- # New lines only; existing ones live in scripts/av_shapes_baseline.json.
- - name: No new code shapes that heuristic scanners quarantine
- run: python scripts/lint_av_shapes.py
-
- - name: Risky-loader gate self-test
- run: python scripts/lint_risky_loaders.py --self-test
-
- # Dynamic imports, trust_remote_code turned on in code, unpinned code fetches and
- # unsafe deserialization. Existing sites live in scripts/risky_loaders_baseline.json.
- - name: No new risky loader call sites
- run: python scripts/lint_risky_loaders.py
-
- - name: Python ruff check (whole repo)
- # The narrow rule set in pyproject.toml [tool.ruff.lint]
- # selects E9 / F63 / F7 / F82 -- syntax errors, broken
- # comparisons, undefined names. The whole repo passes today,
- # so this is a hard gate.
- run: |
- ruff check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
-
- - name: Import-hoist verifier self-test
- # scripts/verify_import_hoist.py is a scope-aware (LEGB) AST
- # resolver that gates import-hoisting / alias-rename refactors
- # against two bugs ruff and pyflakes both miss:
- # 1. dangling alias -- `from a import b as _b` hoisted to
- # `from a import b` but a leftover `_b` reference now
- # resolves to nothing (or to some other module-level `_b`).
- # 2. rename clash -- `_b -> b` silently re-points at a
- # different object already named `b` in that scope.
- # This step runs the tool's 8 negative-control cases so a
- # regression in the verifier itself fails before we trust it on
- # a diff. Hermetic, stdlib-only, sub-second. Hard gate.
- run: |
- python scripts/verify_import_hoist.py --self-test
+ # Every check below is read-only against the checkout and independent of the
+ # others, so they run as one parallel group: the job waits for all of them at
+ # the end of the group and fails if any one failed (an informational step keeps
+ # its continue-on-error). They used to run one after another, about 150 s of
+ # the job, and a failure in an early one hid every result after it.
+ #
+ # The two PR-only diff checks are NOT in the group: both run `git fetch` into
+ # this shallow clone, and two concurrent fetches race on .git/shallow.lock.
+ - parallel:
+ - name: Python AST/syntax check (every committed .py must compile)
+ # python -m compileall uses the same parser the interpreter
+ # uses, so anything broken here would also crash at
+ # `import X` on a user's machine. Sub-second across 350+
+ # files. Hard gate.
+ run: |
+ python -m compileall -q -j 0 \
+ unsloth unsloth_cli studio tests cli.py unsloth-cli.py
+
+ - name: Dynamic-execution gate self-test
+ run: python scripts/lint_exec_literals.py --self-test
+
+ - name: No exec, eval or compile of a value that is not written out
+ run: python scripts/lint_exec_literals.py
+
+ - name: Scanner-shape gate self-test
+ run: python scripts/lint_av_shapes.py --self-test
+
+ # New lines only; existing ones live in scripts/av_shapes_baseline.json.
+ - name: No new code shapes that heuristic scanners quarantine
+ run: python scripts/lint_av_shapes.py
+
+ - name: Risky-loader gate self-test
+ run: python scripts/lint_risky_loaders.py --self-test
+
+ # Dynamic imports, trust_remote_code turned on in code, unpinned code fetches and
+ # unsafe deserialization. Existing sites live in scripts/risky_loaders_baseline.json.
+ - name: No new risky loader call sites
+ run: python scripts/lint_risky_loaders.py
+
+ - name: Python ruff check (whole repo)
+ # The narrow rule set in pyproject.toml [tool.ruff.lint]
+ # selects E9 / F63 / F7 / F82 -- syntax errors, broken
+ # comparisons, undefined names. The whole repo passes today,
+ # so this is a hard gate.
+ run: |
+ ruff check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
+
+ - name: Import-hoist verifier self-test
+ # scripts/verify_import_hoist.py is a scope-aware (LEGB) AST
+ # resolver that gates import-hoisting / alias-rename refactors
+ # against two bugs ruff and pyflakes both miss:
+ # 1. dangling alias -- `from a import b as _b` hoisted to
+ # `from a import b` but a leftover `_b` reference now
+ # resolves to nothing (or to some other module-level `_b`).
+ # 2. rename clash -- `_b -> b` silently re-points at a
+ # different object already named `b` in that scope.
+ # This step runs the tool's 8 negative-control cases so a
+ # regression in the verifier itself fails before we trust it on
+ # a diff. Hermetic, stdlib-only, sub-second. Hard gate.
+ run: |
+ python scripts/verify_import_hoist.py --self-test
+
+ - name: Duplicate-definition verifier self-test
+ # scripts/lint_duplicate_definitions.py refuses a name bound twice in one
+ # scope: two top-level copies of a def / class / constant, or a name
+ # imported twice. That is what a bad merge resolution leaves behind, and
+ # every other check here passes it -- compileall parses both copies, the
+ # ruff rule set (E9 / F63 / F7 / F82) has no F811, and the two copies
+ # usually differ only in their comments.
+ # Runs the tool's self-test cases, positive and negative, so a regression in the
+ # rule fails before we trust it on a diff. Stdlib-only, sub-second.
+ run: |
+ python scripts/lint_duplicate_definitions.py --self-test
+
+ - name: Duplicate definitions (whole tree)
+ # The diff-scoped step passes pre-existing findings, so the backlog regrew
+ # to 38 while it was the only gate. Runs on push too.
+ run: |
+ python scripts/lint_duplicate_definitions.py \
+ unsloth unsloth_cli studio scripts tests cli.py unsloth-cli.py
+
+ - name: No silent llama-server parallel-slot downgrades
+ # #7717 clamped --parallel to 1 whenever MTP resolved, costing batched
+ # callers 4x throughput. Launch fewer slots only for a real capability
+ # or VRAM limit, marked '# allow-slot-clamp: <reason>'. Self-test first,
+ # so a regression in the rule fails before we trust it on the tree.
+ run: |
+ python scripts/lint_no_parallel_clamp.py --self-test
+ python scripts/lint_no_parallel_clamp.py
+
+ - name: No duplicate TypeScript import bindings
+ # #8470 duplicated an import line in model-selector.tsx; tsc rejects it
+ # with TS2300 and every job that builds the frontend goes red behind
+ # it. eslint is not run in CI, so nothing saw it until tsc did, inside
+ # the slowest job in the matrix. Self-test first, so a regression in
+ # the rule fails before we trust it.
+ run: |
+ python scripts/lint_no_duplicate_ts_imports.py --self-test
+ python scripts/lint_no_duplicate_ts_imports.py
+
+ - name: No leftover debugger / pdb / breakpoint calls
+ # Catches the "I'll just stick a breakpoint() here" mistake
+ # before it ships. AST-based so commented-out debugger
+ # markers don't false-positive (a bare grep would; there
+ # are three commented `# breakpoint()` markers in
+ # unsloth/models/rl* today). Sub-second.
+ run: |
+ python <<'PY'
+ import ast, pathlib, sys
+
+ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
+ "unsloth_compiled_cache", "node_modules",
+ "unsloth.egg-info"}
+
+ bad = []
+ scanned = 0
+ for path in sorted(pathlib.Path(".").rglob("*.py")):
+ if any(part in SKIP_PARTS for part in path.parts):
+ continue
+ scanned += 1
+ try:
+ tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"))
+ except SyntaxError:
+ continue # compileall step above already failed this
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.Call):
+ continue
+ fn = node.func
+ if isinstance(fn, ast.Name) and fn.id == "breakpoint":
+ bad.append((path, node.lineno, "breakpoint()"))
+ elif (isinstance(fn, ast.Attribute) and fn.attr == "set_trace"
+ and isinstance(fn.value, ast.Name)
+ and fn.value.id in {"pdb", "ipdb"}):
+ bad.append((path, node.lineno, f"{fn.value.id}.set_trace()"))
+
+ if bad:
+ for path, lineno, what in bad:
+ print(f"::error file={path},line={lineno}::leftover {what} -- remove before merging")
+ sys.exit(1)
+ print(f"no leftover debugger calls (scanned {scanned} files)")
+ PY
+
+ - name: License-header drift (informational; whole repo)
+ # Three header families are accepted across the repo:
+ # 1. SPDX one-liner: `# SPDX-License-Identifier: ...`
+ # Used across studio/ (AGPL-3.0-only) and a few new
+ # files elsewhere.
+ # 2. Apache-2.0 long form, marker phrase
+ # "Licensed under the Apache License". Used across
+ # unsloth/ and unsloth_cli/.
+ # 3. GNU long form, marker phrase "General Public License".
+ # That single substring covers GPL, LGPL ("GNU Lesser
+ # General Public License") and AGPL ("GNU Affero
+ # General Public License") preambles, all three of
+ # which appear in unsloth/kernels/* (LGPL/AGPL) without
+ # the SPDX line.
+ # Empty files (mainly empty __init__.py) are skipped.
+ # Surfaced as a warning; cleaning up the actual misses is a
+ # follow-up PR, not a CI fix.
+ continue-on-error: true
+ run: |
+ python <<'PY'
+ import pathlib
+
+ ACCEPTED = (
+ "SPDX-License-Identifier", # any SPDX line
+ "Licensed under the Apache License", # Apache-2.0 long form
+ "General Public License", # GPL / LGPL / AGPL long form
+ )
+ # "vendor" is upstream third-party source: it carries its own licence.
+ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
+ "unsloth_compiled_cache", "node_modules",
+ "unsloth.egg-info", "vendor"}
+
+ studio_missing = []
+ other_missing = []
+ for path in sorted(pathlib.Path(".").rglob("*.py")):
+ if any(part in SKIP_PARTS for part in path.parts):
+ continue
+ text = path.read_text(encoding="utf-8", errors="replace")
+ if not text.strip():
+ continue # empty __init__.py etc.
+ head = "\n".join(text.splitlines()[:25])
+ if any(marker in head for marker in ACCEPTED):
+ continue
+ if "studio" in path.parts:
+ studio_missing.append(path)
+ else:
+ other_missing.append(path)
+
+ total = len(studio_missing) + len(other_missing)
+ if total == 0:
+ print("every committed .py has a recognised license header")
+ else:
+ print(f"::warning::{total} Python files have no recognised license "
+ f"header (SPDX / Apache-2.0 / GNU long form): "
+ f"studio={len(studio_missing)}, other={len(other_missing)}")
+ for path in (studio_missing + other_missing)[:30]:
+ print(f" {path}")
+ if total > 30:
+ print(f" ... and {total - 30} more")
+ PY
+
+ - name: Shell scripts parse cleanly (`bash -n`)
+ # Same idea as Python's compileall: parse-only check that
+ # every committed *.sh would not blow up at `bash script.sh`
+ # invocation time on a release box. tests/sh/ is the largest
+ # cluster (the install.sh shape tests).
+ run: |
+ shopt -s globstar
+ fail=0
+ for f in $(git ls-files '*.sh'); do
+ if ! bash -n "$f"; then
+ echo "::error file=$f::shell parse error"
+ fail=1
+ fi
+ done
+ if [ "$fail" -ne 0 ]; then
+ exit 1
+ fi
+ n=$(git ls-files '*.sh' | wc -l)
+ echo "$n shell scripts parse cleanly"
+
+ - name: YAML files parse cleanly (yaml.safe_load)
+ # Catches truncated workflow files, broken indents in
+ # dependabot.yml / pre-commit configs, etc. Includes
+ # .github/workflows/*.yml so a typo in the file we just
+ # added shows up immediately.
+ run: |
+ python <<'PY'
+ import pathlib, sys, yaml
+
+ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
+ "node_modules", "unsloth_compiled_cache",
+ "unsloth.egg-info"}
+
+ bad = []
+ scanned = 0
+ for path in sorted(list(pathlib.Path(".").rglob("*.yml"))
+ + list(pathlib.Path(".").rglob("*.yaml"))):
+ if any(part in SKIP_PARTS for part in path.parts):
+ continue
+ scanned += 1
+ try:
+ with path.open("r", encoding="utf-8") as fh:
+ list(yaml.safe_load_all(fh))
+ except Exception as exc:
+ bad.append((path, exc))
+
+ if bad:
+ for path, exc in bad:
+ print(f"::error file={path}::YAML parse failed: {exc}")
+ sys.exit(1)
+ print(f"{scanned} YAML files parse cleanly")
+ PY
+
+ - name: JSON files parse cleanly (json.loads)
+ # Catches malformed package.json, biome.json, etc. Skips:
+ # - huge npm/bun lockfiles (machine-generated, slow to
+ # parse, no value).
+ # - tsconfig*.json: TypeScript convention is JSONC (JSON
+ # with `/* ... */` comments), which standard json.loads
+ # rejects. Strip-and-validate would need json5 or a
+ # hand-rolled comment scrubber for marginal value, since
+ # `tsc --noEmit` already validates these in Frontend CI.
+ run: |
+ python <<'PY'
+ import fnmatch, json, pathlib, sys
+
+ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
+ "node_modules", "unsloth_compiled_cache",
+ "unsloth.egg-info"}
+ SKIP_NAMES = {"package-lock.json", "bun.lock"}
+ SKIP_PATTERNS = ("tsconfig*.json",)
+
+ bad = []
+ scanned = 0
+ for path in sorted(pathlib.Path(".").rglob("*.json")):
+ if any(part in SKIP_PARTS for part in path.parts):
+ continue
+ if path.name in SKIP_NAMES:
+ continue
+ if any(fnmatch.fnmatch(path.name, pat) for pat in SKIP_PATTERNS):
+ continue
+ scanned += 1
+ try:
+ json.loads(path.read_text(encoding="utf-8"))
+ except Exception as exc:
+ bad.append((path, exc))
+
+ if bad:
+ for path, exc in bad:
+ print(f"::error file={path}::JSON parse failed: {exc}")
+ sys.exit(1)
+ print(f"{scanned} JSON files parse cleanly")
+ PY
+
+ - name: codespell typo check (informational)
+ # Catches typos in code, comments, and docs across the repo.
+ # Skips lockfiles, generated assets, binary artefacts, and
+ # the LICENSE files (US/UK spelling drift in legal text is
+ # not ours to second-guess). The ignore-words-list pulls
+ # out short identifiers + valid technical terms that
+ # codespell's default dictionary would otherwise flag
+ # (e.g. `ans` as a math-quiz variable name in
+ # tests/utils/aime_eval.py, `parm`/`parms` in PyTorch
+ # nn.Module idioms). Non-blocking until the surfaced typos
+ # are fixed; drop continue-on-error after the cleanup.
+ continue-on-error: true
+ run: |
+ codespell \
+ --skip='*.lock,*.lockb,*.json,*.svg,*.png,*.jpg,*.jpeg,*.gif,*.ico,*.woff*,*.ttf,*.eot,*.zip,*.gz,*.gguf,*.safetensors,*.bin,node_modules,.git,build,dist,unsloth_compiled_cache,unsloth.egg-info,target,studio/frontend/dist,*.pyc,*-licenses.txt,LICENSE*' \
+ --ignore-words-list='ans,bu,hel,fo,te,ot,hist,ned,sav,recurser,datas,nin,parm,parms,checkin,nd,fr,inout,donot,uint' \
+ --quiet-level=2
+
+ - name: shellcheck on committed *.sh (informational)
+ # Goes beyond `bash -n` (which only parses): catches subtle
+ # shell bugs like unquoted variable expansions, useless
+ # `cat`, command substitutions inside `[[`, etc. The
+ # install/setup scripts are critical-path so the signal is
+ # worth surfacing. Non-blocking until install.sh's
+ # hand-rolled patterns get cleaned up; drop continue-on-error
+ # afterwards.
+ continue-on-error: true
+ run: |
+ # Exclude SC1090 ("source not followable") -- legitimate
+ # for installer scripts that source files at runtime
+ # paths shellcheck cannot resolve statically.
+ # SC2034 ("variable assigned but never used") fires on
+ # the export-only assignment idiom we use in install.sh.
+ shellcheck -e SC1090,SC2034 $(git ls-files '*.sh')
+
+ - name: ruff format drift (informational)
+ # The canonical formatter is scripts/run_ruff_format.py
+ # = ruff format + scripts/enforce_kwargs_spacing.py, so plain
+ # `ruff format --check` reports the kwarg-spacing diff as
+ # drift. Surface the count for visibility but keep
+ # non-blocking until the custom pipeline is wired in here.
+ continue-on-error: true
+ run: |
+ ruff format --check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
- name: Import-hoist / alias-rename safety (changed Python files)
# Runs the verifier in compare mode on every in-place-modified
@@ -263,18 +552,6 @@ jobs:
python scripts/verify_import_hoist.py \
--before "$MERGE_BASE" --after "$HEAD_SHA" "${CHANGED[@]}"
- - name: Duplicate-definition verifier self-test
- # scripts/lint_duplicate_definitions.py refuses a name bound twice in one
- # scope: two top-level copies of a def / class / constant, or a name
- # imported twice. That is what a bad merge resolution leaves behind, and
- # every other check here passes it -- compileall parses both copies, the
- # ruff rule set (E9 / F63 / F7 / F82) has no F811, and the two copies
- # usually differ only in their comments.
- # Runs the tool's self-test cases, positive and negative, so a regression in the
- # rule fails before we trust it on a diff. Stdlib-only, sub-second.
- run: |
- python scripts/lint_duplicate_definitions.py --self-test
-
- name: Duplicate definitions (changed Python files)
# Same merge-base reasoning as the import-hoist step above: diff the merge
# base against the PR head SHA so the range is the PR's own commits and
@@ -334,273 +611,6 @@ jobs:
python scripts/lint_duplicate_definitions.py \
--before "$MERGE_BASE" --after "$HEAD_SHA" "${CHANGED[@]}"
- - name: Duplicate definitions (whole tree)
- # The diff-scoped step passes pre-existing findings, so the backlog regrew
- # to 38 while it was the only gate. Runs on push too.
- run: |
- python scripts/lint_duplicate_definitions.py \
- unsloth unsloth_cli studio scripts tests cli.py unsloth-cli.py
-
- - name: No silent llama-server parallel-slot downgrades
- # #7717 clamped --parallel to 1 whenever MTP resolved, costing batched
- # callers 4x throughput. Launch fewer slots only for a real capability
- # or VRAM limit, marked '# allow-slot-clamp: <reason>'. Self-test first,
- # so a regression in the rule fails before we trust it on the tree.
- run: |
- python scripts/lint_no_parallel_clamp.py --self-test
- python scripts/lint_no_parallel_clamp.py
-
- - name: No duplicate TypeScript import bindings
- # #8470 duplicated an import line in model-selector.tsx; tsc rejects it
- # with TS2300 and every job that builds the frontend goes red behind
- # it. eslint is not run in CI, so nothing saw it until tsc did, inside
- # the slowest job in the matrix. Self-test first, so a regression in
- # the rule fails before we trust it.
- run: |
- python scripts/lint_no_duplicate_ts_imports.py --self-test
- python scripts/lint_no_duplicate_ts_imports.py
-
- - name: No leftover debugger / pdb / breakpoint calls
- # Catches the "I'll just stick a breakpoint() here" mistake
- # before it ships. AST-based so commented-out debugger
- # markers don't false-positive (a bare grep would; there
- # are three commented `# breakpoint()` markers in
- # unsloth/models/rl* today). Sub-second.
- run: |
- python <<'PY'
- import ast, pathlib, sys
-
- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
- "unsloth_compiled_cache", "node_modules",
- "unsloth.egg-info"}
-
- bad = []
- scanned = 0
- for path in sorted(pathlib.Path(".").rglob("*.py")):
- if any(part in SKIP_PARTS for part in path.parts):
- continue
- scanned += 1
- try:
- tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"))
- except SyntaxError:
- continue # compileall step above already failed this
- for node in ast.walk(tree):
- if not isinstance(node, ast.Call):
- continue
- fn = node.func
- if isinstance(fn, ast.Name) and fn.id == "breakpoint":
- bad.append((path, node.lineno, "breakpoint()"))
- elif (isinstance(fn, ast.Attribute) and fn.attr == "set_trace"
- and isinstance(fn.value, ast.Name)
- and fn.value.id in {"pdb", "ipdb"}):
- bad.append((path, node.lineno, f"{fn.value.id}.set_trace()"))
-
- if bad:
- for path, lineno, what in bad:
- print(f"::error file={path},line={lineno}::leftover {what} -- remove before merging")
- sys.exit(1)
- print(f"no leftover debugger calls (scanned {scanned} files)")
- PY
-
- - name: License-header drift (informational; whole repo)
- # Three header families are accepted across the repo:
- # 1. SPDX one-liner: `# SPDX-License-Identifier: ...`
- # Used across studio/ (AGPL-3.0-only) and a few new
- # files elsewhere.
- # 2. Apache-2.0 long form, marker phrase
- # "Licensed under the Apache License". Used across
- # unsloth/ and unsloth_cli/.
- # 3. GNU long form, marker phrase "General Public License".
- # That single substring covers GPL, LGPL ("GNU Lesser
- # General Public License") and AGPL ("GNU Affero
- # General Public License") preambles, all three of
- # which appear in unsloth/kernels/* (LGPL/AGPL) without
- # the SPDX line.
- # Empty files (mainly empty __init__.py) are skipped.
- # Surfaced as a warning; cleaning up the actual misses is a
- # follow-up PR, not a CI fix.
- continue-on-error: true
- run: |
- python <<'PY'
- import pathlib
-
- ACCEPTED = (
- "SPDX-License-Identifier", # any SPDX line
- "Licensed under the Apache License", # Apache-2.0 long form
- "General Public License", # GPL / LGPL / AGPL long form
- )
- # "vendor" is upstream third-party source: it carries its own licence.
- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
- "unsloth_compiled_cache", "node_modules",
- "unsloth.egg-info", "vendor"}
-
- studio_missing = []
- other_missing = []
- for path in sorted(pathlib.Path(".").rglob("*.py")):
- if any(part in SKIP_PARTS for part in path.parts):
- continue
- text = path.read_text(encoding="utf-8", errors="replace")
- if not text.strip():
- continue # empty __init__.py etc.
- head = "\n".join(text.splitlines()[:25])
- if any(marker in head for marker in ACCEPTED):
- continue
- if "studio" in path.parts:
- studio_missing.append(path)
- else:
- other_missing.append(path)
-
- total = len(studio_missing) + len(other_missing)
- if total == 0:
- print("every committed .py has a recognised license header")
- else:
- print(f"::warning::{total} Python files have no recognised license "
- f"header (SPDX / Apache-2.0 / GNU long form): "
- f"studio={len(studio_missing)}, other={len(other_missing)}")
- for path in (studio_missing + other_missing)[:30]:
- print(f" {path}")
- if total > 30:
- print(f" ... and {total - 30} more")
- PY
-
- - name: Shell scripts parse cleanly (`bash -n`)
- # Same idea as Python's compileall: parse-only check that
- # every committed *.sh would not blow up at `bash script.sh`
- # invocation time on a release box. tests/sh/ is the largest
- # cluster (the install.sh shape tests).
- run: |
- shopt -s globstar
- fail=0
- for f in $(git ls-files '*.sh'); do
- if ! bash -n "$f"; then
- echo "::error file=$f::shell parse error"
- fail=1
- fi
- done
- if [ "$fail" -ne 0 ]; then
- exit 1
- fi
- n=$(git ls-files '*.sh' | wc -l)
- echo "$n shell scripts parse cleanly"
-
- - name: YAML files parse cleanly (yaml.safe_load)
- # Catches truncated workflow files, broken indents in
- # dependabot.yml / pre-commit configs, etc. Includes
- # .github/workflows/*.yml so a typo in the file we just
- # added shows up immediately.
- run: |
- python <<'PY'
- import pathlib, sys, yaml
-
- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
- "node_modules", "unsloth_compiled_cache",
- "unsloth.egg-info"}
-
- bad = []
- scanned = 0
- for path in sorted(list(pathlib.Path(".").rglob("*.yml"))
- + list(pathlib.Path(".").rglob("*.yaml"))):
- if any(part in SKIP_PARTS for part in path.parts):
- continue
- scanned += 1
- try:
- with path.open("r", encoding="utf-8") as fh:
- list(yaml.safe_load_all(fh))
- except Exception as exc:
- bad.append((path, exc))
-
- if bad:
- for path, exc in bad:
- print(f"::error file={path}::YAML parse failed: {exc}")
- sys.exit(1)
- print(f"{scanned} YAML files parse cleanly")
- PY
-
- - name: JSON files parse cleanly (json.loads)
- # Catches malformed package.json, biome.json, etc. Skips:
- # - huge npm/bun lockfiles (machine-generated, slow to
- # parse, no value).
- # - tsconfig*.json: TypeScript convention is JSONC (JSON
- # with `/* ... */` comments), which standard json.loads
- # rejects. Strip-and-validate would need json5 or a
- # hand-rolled comment scrubber for marginal value, since
- # `tsc --noEmit` already validates these in Frontend CI.
- run: |
- python <<'PY'
- import fnmatch, json, pathlib, sys
-
- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
- "node_modules", "unsloth_compiled_cache",
- "unsloth.egg-info"}
- SKIP_NAMES = {"package-lock.json", "bun.lock"}
- SKIP_PATTERNS = ("tsconfig*.json",)
-
- bad = []
- scanned = 0
- for path in sorted(pathlib.Path(".").rglob("*.json")):
- if any(part in SKIP_PARTS for part in path.parts):
- continue
- if path.name in SKIP_NAMES:
- continue
- if any(fnmatch.fnmatch(path.name, pat) for pat in SKIP_PATTERNS):
- continue
- scanned += 1
- try:
- json.loads(path.read_text(encoding="utf-8"))
- except Exception as exc:
- bad.append((path, exc))
-
- if bad:
- for path, exc in bad:
- print(f"::error file={path}::JSON parse failed: {exc}")
- sys.exit(1)
- print(f"{scanned} JSON files parse cleanly")
- PY
-
- - name: codespell typo check (informational)
- # Catches typos in code, comments, and docs across the repo.
- # Skips lockfiles, generated assets, binary artefacts, and
- # the LICENSE files (US/UK spelling drift in legal text is
- # not ours to second-guess). The ignore-words-list pulls
- # out short identifiers + valid technical terms that
- # codespell's default dictionary would otherwise flag
- # (e.g. `ans` as a math-quiz variable name in
- # tests/utils/aime_eval.py, `parm`/`parms` in PyTorch
- # nn.Module idioms). Non-blocking until the surfaced typos
- # are fixed; drop continue-on-error after the cleanup.
- continue-on-error: true
- run: |
- codespell \
- --skip='*.lock,*.lockb,*.json,*.svg,*.png,*.jpg,*.jpeg,*.gif,*.ico,*.woff*,*.ttf,*.eot,*.zip,*.gz,*.gguf,*.safetensors,*.bin,node_modules,.git,build,dist,unsloth_compiled_cache,unsloth.egg-info,target,studio/frontend/dist,*.pyc,*-licenses.txt,LICENSE*' \
- --ignore-words-list='ans,bu,hel,fo,te,ot,hist,ned,sav,recurser,datas,nin,parm,parms,checkin,nd,fr,inout,donot,uint' \
- --quiet-level=2
-
- - name: shellcheck on committed *.sh (informational)
- # Goes beyond `bash -n` (which only parses): catches subtle
- # shell bugs like unquoted variable expansions, useless
- # `cat`, command substitutions inside `[[`, etc. The
- # install/setup scripts are critical-path so the signal is
- # worth surfacing. Non-blocking until install.sh's
- # hand-rolled patterns get cleaned up; drop continue-on-error
- # afterwards.
- continue-on-error: true
- run: |
- # Exclude SC1090 ("source not followable") -- legitimate
- # for installer scripts that source files at runtime
- # paths shellcheck cannot resolve statically.
- # SC2034 ("variable assigned but never used") fires on
- # the export-only assignment idiom we use in install.sh.
- shellcheck -e SC1090,SC2034 $(git ls-files '*.sh')
-
- - name: ruff format drift (informational)
- # The canonical formatter is scripts/run_ruff_format.py
- # = ruff format + scripts/enforce_kwargs_spacing.py, so plain
- # `ruff format --check` reports the kwarg-spacing diff as
- # drift. Surface the count for visibility but keep
- # non-blocking until the custom pipeline is wired in here.
- continue-on-error: true
- run: |
- ruff format --check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
# Collects the two lanes launched before the lint steps. This step is the whole
# reason the backgrounding is safe: without it a lane's failure would be invisible
diff --git a/.github/workflows/studio-backend-ci.yml b/.github/workflows/studio-backend-ci.yml
index 2342f45bd06..dcb3d90c727 100644
--- a/.github/workflows/studio-backend-ci.yml
+++ b/.github/workflows/studio-backend-ci.yml
@@ -183,6 +183,13 @@ jobs:
- name: Install bubblewrap, so the sandbox tests run instead of skipping
# Bound apt retries; 25 minutes covers both calls' 1450s worst case.
+ #
+ # Background: apt (system packages) and the pip install below (this job's
+ # Python) touch nothing in common, so the ~10 s of apt overlaps the ~80 s of
+ # pip. Collected right before the first pytest step, which is the first thing
+ # that needs bwrap.
+ id: bubblewrap
+ background: true
timeout-minutes: 25
env:
RETRY_ATTEMPTS: '2'
@@ -239,6 +246,9 @@ jobs:
# extensions under torch < 2.11 and the pure-Python constructors still import.
pip install torchao
+ - name: Collect the bubblewrap install
+ wait: bubblewrap
+
- name: Backend tests
if: matrix.scope == 'full'
working-directory: studio/backend
diff --git a/.github/workflows/studio-windows-inference-smoke.yml b/.github/workflows/studio-windows-inference-smoke.yml
index f2f87bc3f0c..7fcae4bc6e4 100644
--- a/.github/workflows/studio-windows-inference-smoke.yml
+++ b/.github/workflows/studio-windows-inference-smoke.yml
@@ -161,7 +161,17 @@ jobs:
# Fast GPU-free gate: parse install.ps1 + setup.ps1 and run the PowerShell
# unit tests (CUDA-toolkit + torch-flavor helpers) before the heavy GGUF smoke.
+ #
+ # Background, together with the uninstall test below: about 60 s of pure
+ # PowerShell that reads only the checkout, so it overlaps the node/python setup
+ # and the ~90 s uv cache restore instead of running ahead of them. Both are
+ # collected by the wait step right before Install Unsloth, so a failure still
+ # stops the job before the install, as it did when they ran first; they are not
+ # overlapped with the install itself, because test_mirror_fallback.ps1 asserts
+ # deadlines and transfer speeds that a CPU-bound install next to it could skew.
- name: PowerShell installer unit tests
+ id: ps-installer-units
+ background: true
shell: pwsh
run: |
foreach ($f in @('install.ps1', 'studio/setup.ps1')) {
@@ -194,6 +204,8 @@ jobs:
# uninstall.ps1: native uninstall must keep the shared unsloth.ico while a
# WSL shortcut still references it (dual install), else that shortcut blanks.
- name: uninstall.ps1 unit test (dual-install icon preserve)
+ id: ps-uninstall-unit
+ background: true
shell: pwsh
run: |
$errs = $null
@@ -267,6 +279,9 @@ jobs:
id: fe-dist
uses: ./.github/actions/frontend-dist-restore
+ - name: Collect the PowerShell unit tests
+ wait: [ps-installer-units, ps-uninstall-unit]
+
- name: Install Unsloth (--local, --no-torch)
id: install
timeout-minutes: 25
diff --git a/.github/workflows/studio-windows-update-smoke.yml b/.github/workflows/studio-windows-update-smoke.yml
index 4e075783f74..a4ddeedd142 100644
--- a/.github/workflows/studio-windows-update-smoke.yml
+++ b/.github/workflows/studio-windows-update-smoke.yml
@@ -114,7 +114,14 @@ jobs:
persist-credentials: false
# This workflow triggers on uninstall.ps1; neither test really uninstalls.
+ #
+ # These three unit-test steps run in the background: they read only the
+ # checkout, and together take about 20 s that now overlaps the node/python
+ # setup and the ~90 s uv cache restore. The wait step right before Install
+ # Unsloth collects them, so a failure still stops the job before the install.
- name: uninstall.ps1 unit tests (argument guard, dual-install icon preserve)
+ id: ps-uninstall-units
+ background: true
shell: pwsh
run: |
# Propagate each child test's failure before continuing.
@@ -126,6 +133,8 @@ jobs:
# Both entry points prepend the 5.1 system module dir, and Refresh-Environment
# must not reload PSModulePath back over it before the uv installer runs.
- name: PSModulePath normalization unit tests
+ id: ps-modulepath-units
+ background: true
shell: pwsh
run: pwsh -NoProfile -File tests/studio/test_psmodulepath_normalization.ps1
@@ -133,6 +142,8 @@ jobs:
# Windows PowerShell 5.1: the shell the README one-liner lands in, handled
# nowhere else in CI, and it differs from pwsh 7 on native stderr.
- name: uninstall.ps1 argument guard under Windows PowerShell 5.1
+ id: ps51-uninstall-guard
+ background: true
shell: powershell
run: |
powershell -NoProfile -File tests/studio/test_uninstall_arg_guard.ps1
@@ -223,6 +234,9 @@ jobs:
id: fe-dist
uses: ./.github/actions/frontend-dist-restore
+ - name: Collect the PowerShell unit tests
+ wait: [ps-uninstall-units, ps-modulepath-units, ps51-uninstall-guard]
+
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env: