Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
CI: run lint checks in parallel and overlap independent setup in Windows smoke and Backend CI (#13710)
Daniel Hand975be3d4a81
diff --git a/.github/workflows/lint-ci.yml b/.github/workflows/lint-ci.ymlindex 69056019807..061457a6b36 100644--- a/.github/workflows/lint-ci.yml+++ b/.github/workflows/lint-ci.yml@@ -150,58 +150,347 @@ jobs:bash .github/scripts/retry-with-apt-lock.sh sudo sh -c \'apt-get install -y --no-install-recommends shellcheck || { apt-get update -qq && apt-get install -y --no-install-recommends shellcheck; }'- - name: Python AST/syntax check (every committed .py must compile)- # python -m compileall uses the same parser the interpreter- # uses, so anything broken here would also crash at- # `import X` on a user's machine. Sub-second across 350+- # files. Hard gate.- run: |- python -m compileall -q -j 0 \- unsloth unsloth_cli studio tests cli.py unsloth-cli.py-- - name: Dynamic-execution gate self-test- run: python scripts/lint_exec_literals.py --self-test-- - name: No exec, eval or compile of a value that is not written out- run: python scripts/lint_exec_literals.py-- - name: Scanner-shape gate self-test- run: python scripts/lint_av_shapes.py --self-test-- # New lines only; existing ones live in scripts/av_shapes_baseline.json.- - name: No new code shapes that heuristic scanners quarantine- run: python scripts/lint_av_shapes.py-- - name: Risky-loader gate self-test- run: python scripts/lint_risky_loaders.py --self-test-- # Dynamic imports, trust_remote_code turned on in code, unpinned code fetches and- # unsafe deserialization. Existing sites live in scripts/risky_loaders_baseline.json.- - name: No new risky loader call sites- run: python scripts/lint_risky_loaders.py-- - name: Python ruff check (whole repo)- # The narrow rule set in pyproject.toml [tool.ruff.lint]- # selects E9 / F63 / F7 / F82 -- syntax errors, broken- # comparisons, undefined names. The whole repo passes today,- # so this is a hard gate.- run: |- ruff check unsloth unsloth_cli studio tests cli.py unsloth-cli.py-- - name: Import-hoist verifier self-test- # scripts/verify_import_hoist.py is a scope-aware (LEGB) AST- # resolver that gates import-hoisting / alias-rename refactors- # against two bugs ruff and pyflakes both miss:- # 1. dangling alias -- `from a import b as _b` hoisted to- # `from a import b` but a leftover `_b` reference now- # resolves to nothing (or to some other module-level `_b`).- # 2. rename clash -- `_b -> b` silently re-points at a- # different object already named `b` in that scope.- # This step runs the tool's 8 negative-control cases so a- # regression in the verifier itself fails before we trust it on- # a diff. Hermetic, stdlib-only, sub-second. Hard gate.- run: |- python scripts/verify_import_hoist.py --self-test+ # Every check below is read-only against the checkout and independent of the+ # others, so they run as one parallel group: the job waits for all of them at+ # the end of the group and fails if any one failed (an informational step keeps+ # its continue-on-error). They used to run one after another, about 150 s of+ # the job, and a failure in an early one hid every result after it.+ #+ # The two PR-only diff checks are NOT in the group: both run `git fetch` into+ # this shallow clone, and two concurrent fetches race on .git/shallow.lock.+ - parallel:+ - name: Python AST/syntax check (every committed .py must compile)+ # python -m compileall uses the same parser the interpreter+ # uses, so anything broken here would also crash at+ # `import X` on a user's machine. Sub-second across 350++ # files. Hard gate.+ run: |+ python -m compileall -q -j 0 \+ unsloth unsloth_cli studio tests cli.py unsloth-cli.py++ - name: Dynamic-execution gate self-test+ run: python scripts/lint_exec_literals.py --self-test++ - name: No exec, eval or compile of a value that is not written out+ run: python scripts/lint_exec_literals.py++ - name: Scanner-shape gate self-test+ run: python scripts/lint_av_shapes.py --self-test++ # New lines only; existing ones live in scripts/av_shapes_baseline.json.+ - name: No new code shapes that heuristic scanners quarantine+ run: python scripts/lint_av_shapes.py++ - name: Risky-loader gate self-test+ run: python scripts/lint_risky_loaders.py --self-test++ # Dynamic imports, trust_remote_code turned on in code, unpinned code fetches and+ # unsafe deserialization. Existing sites live in scripts/risky_loaders_baseline.json.+ - name: No new risky loader call sites+ run: python scripts/lint_risky_loaders.py++ - name: Python ruff check (whole repo)+ # The narrow rule set in pyproject.toml [tool.ruff.lint]+ # selects E9 / F63 / F7 / F82 -- syntax errors, broken+ # comparisons, undefined names. The whole repo passes today,+ # so this is a hard gate.+ run: |+ ruff check unsloth unsloth_cli studio tests cli.py unsloth-cli.py++ - name: Import-hoist verifier self-test+ # scripts/verify_import_hoist.py is a scope-aware (LEGB) AST+ # resolver that gates import-hoisting / alias-rename refactors+ # against two bugs ruff and pyflakes both miss:+ # 1. dangling alias -- `from a import b as _b` hoisted to+ # `from a import b` but a leftover `_b` reference now+ # resolves to nothing (or to some other module-level `_b`).+ # 2. rename clash -- `_b -> b` silently re-points at a+ # different object already named `b` in that scope.+ # This step runs the tool's 8 negative-control cases so a+ # regression in the verifier itself fails before we trust it on+ # a diff. Hermetic, stdlib-only, sub-second. Hard gate.+ run: |+ python scripts/verify_import_hoist.py --self-test++ - name: Duplicate-definition verifier self-test+ # scripts/lint_duplicate_definitions.py refuses a name bound twice in one+ # scope: two top-level copies of a def / class / constant, or a name+ # imported twice. That is what a bad merge resolution leaves behind, and+ # every other check here passes it -- compileall parses both copies, the+ # ruff rule set (E9 / F63 / F7 / F82) has no F811, and the two copies+ # usually differ only in their comments.+ # Runs the tool's self-test cases, positive and negative, so a regression in the+ # rule fails before we trust it on a diff. Stdlib-only, sub-second.+ run: |+ python scripts/lint_duplicate_definitions.py --self-test++ - name: Duplicate definitions (whole tree)+ # The diff-scoped step passes pre-existing findings, so the backlog regrew+ # to 38 while it was the only gate. Runs on push too.+ run: |+ python scripts/lint_duplicate_definitions.py \+ unsloth unsloth_cli studio scripts tests cli.py unsloth-cli.py++ - name: No silent llama-server parallel-slot downgrades+ # #7717 clamped --parallel to 1 whenever MTP resolved, costing batched+ # callers 4x throughput. Launch fewer slots only for a real capability+ # or VRAM limit, marked '# allow-slot-clamp: <reason>'. Self-test first,+ # so a regression in the rule fails before we trust it on the tree.+ run: |+ python scripts/lint_no_parallel_clamp.py --self-test+ python scripts/lint_no_parallel_clamp.py++ - name: No duplicate TypeScript import bindings+ # #8470 duplicated an import line in model-selector.tsx; tsc rejects it+ # with TS2300 and every job that builds the frontend goes red behind+ # it. eslint is not run in CI, so nothing saw it until tsc did, inside+ # the slowest job in the matrix. Self-test first, so a regression in+ # the rule fails before we trust it.+ run: |+ python scripts/lint_no_duplicate_ts_imports.py --self-test+ python scripts/lint_no_duplicate_ts_imports.py++ - name: No leftover debugger / pdb / breakpoint calls+ # Catches the "I'll just stick a breakpoint() here" mistake+ # before it ships. AST-based so commented-out debugger+ # markers don't false-positive (a bare grep would; there+ # are three commented `# breakpoint()` markers in+ # unsloth/models/rl* today). Sub-second.+ run: |+ python <<'PY'+ import ast, pathlib, sys++ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",+ "unsloth_compiled_cache", "node_modules",+ "unsloth.egg-info"}++ bad = []+ scanned = 0+ for path in sorted(pathlib.Path(".").rglob("*.py")):+ if any(part in SKIP_PARTS for part in path.parts):+ continue+ scanned += 1+ try:+ tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"))+ except SyntaxError:+ continue # compileall step above already failed this+ for node in ast.walk(tree):+ if not isinstance(node, ast.Call):+ continue+ fn = node.func+ if isinstance(fn, ast.Name) and fn.id == "breakpoint":+ bad.append((path, node.lineno, "breakpoint()"))+ elif (isinstance(fn, ast.Attribute) and fn.attr == "set_trace"+ and isinstance(fn.value, ast.Name)+ and fn.value.id in {"pdb", "ipdb"}):+ bad.append((path, node.lineno, f"{fn.value.id}.set_trace()"))++ if bad:+ for path, lineno, what in bad:+ print(f"::error file={path},line={lineno}::leftover {what} -- remove before merging")+ sys.exit(1)+ print(f"no leftover debugger calls (scanned {scanned} files)")+ PY++ - name: License-header drift (informational; whole repo)+ # Three header families are accepted across the repo:+ # 1. SPDX one-liner: `# SPDX-License-Identifier: ...`+ # Used across studio/ (AGPL-3.0-only) and a few new+ # files elsewhere.+ # 2. Apache-2.0 long form, marker phrase+ # "Licensed under the Apache License". Used across+ # unsloth/ and unsloth_cli/.+ # 3. GNU long form, marker phrase "General Public License".+ # That single substring covers GPL, LGPL ("GNU Lesser+ # General Public License") and AGPL ("GNU Affero+ # General Public License") preambles, all three of+ # which appear in unsloth/kernels/* (LGPL/AGPL) without+ # the SPDX line.+ # Empty files (mainly empty __init__.py) are skipped.+ # Surfaced as a warning; cleaning up the actual misses is a+ # follow-up PR, not a CI fix.+ continue-on-error: true+ run: |+ python <<'PY'+ import pathlib++ ACCEPTED = (+ "SPDX-License-Identifier", # any SPDX line+ "Licensed under the Apache License", # Apache-2.0 long form+ "General Public License", # GPL / LGPL / AGPL long form+ )+ # "vendor" is upstream third-party source: it carries its own licence.+ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",+ "unsloth_compiled_cache", "node_modules",+ "unsloth.egg-info", "vendor"}++ studio_missing = []+ other_missing = []+ for path in sorted(pathlib.Path(".").rglob("*.py")):+ if any(part in SKIP_PARTS for part in path.parts):+ continue+ text = path.read_text(encoding="utf-8", errors="replace")+ if not text.strip():+ continue # empty __init__.py etc.+ head = "\n".join(text.splitlines()[:25])+ if any(marker in head for marker in ACCEPTED):+ continue+ if "studio" in path.parts:+ studio_missing.append(path)+ else:+ other_missing.append(path)++ total = len(studio_missing) + len(other_missing)+ if total == 0:+ print("every committed .py has a recognised license header")+ else:+ print(f"::warning::{total} Python files have no recognised license "+ f"header (SPDX / Apache-2.0 / GNU long form): "+ f"studio={len(studio_missing)}, other={len(other_missing)}")+ for path in (studio_missing + other_missing)[:30]:+ print(f" {path}")+ if total > 30:+ print(f" ... and {total - 30} more")+ PY++ - name: Shell scripts parse cleanly (`bash -n`)+ # Same idea as Python's compileall: parse-only check that+ # every committed *.sh would not blow up at `bash script.sh`+ # invocation time on a release box. tests/sh/ is the largest+ # cluster (the install.sh shape tests).+ run: |+ shopt -s globstar+ fail=0+ for f in $(git ls-files '*.sh'); do+ if ! bash -n "$f"; then+ echo "::error file=$f::shell parse error"+ fail=1+ fi+ done+ if [ "$fail" -ne 0 ]; then+ exit 1+ fi+ n=$(git ls-files '*.sh' | wc -l)+ echo "$n shell scripts parse cleanly"++ - name: YAML files parse cleanly (yaml.safe_load)+ # Catches truncated workflow files, broken indents in+ # dependabot.yml / pre-commit configs, etc. Includes+ # .github/workflows/*.yml so a typo in the file we just+ # added shows up immediately.+ run: |+ python <<'PY'+ import pathlib, sys, yaml++ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",+ "node_modules", "unsloth_compiled_cache",+ "unsloth.egg-info"}++ bad = []+ scanned = 0+ for path in sorted(list(pathlib.Path(".").rglob("*.yml"))+ + list(pathlib.Path(".").rglob("*.yaml"))):+ if any(part in SKIP_PARTS for part in path.parts):+ continue+ scanned += 1+ try:+ with path.open("r", encoding="utf-8") as fh:+ list(yaml.safe_load_all(fh))+ except Exception as exc:+ bad.append((path, exc))++ if bad:+ for path, exc in bad:+ print(f"::error file={path}::YAML parse failed: {exc}")+ sys.exit(1)+ print(f"{scanned} YAML files parse cleanly")+ PY++ - name: JSON files parse cleanly (json.loads)+ # Catches malformed package.json, biome.json, etc. Skips:+ # - huge npm/bun lockfiles (machine-generated, slow to+ # parse, no value).+ # - tsconfig*.json: TypeScript convention is JSONC (JSON+ # with `/* ... */` comments), which standard json.loads+ # rejects. Strip-and-validate would need json5 or a+ # hand-rolled comment scrubber for marginal value, since+ # `tsc --noEmit` already validates these in Frontend CI.+ run: |+ python <<'PY'+ import fnmatch, json, pathlib, sys++ SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",+ "node_modules", "unsloth_compiled_cache",+ "unsloth.egg-info"}+ SKIP_NAMES = {"package-lock.json", "bun.lock"}+ SKIP_PATTERNS = ("tsconfig*.json",)++ bad = []+ scanned = 0+ for path in sorted(pathlib.Path(".").rglob("*.json")):+ if any(part in SKIP_PARTS for part in path.parts):+ continue+ if path.name in SKIP_NAMES:+ continue+ if any(fnmatch.fnmatch(path.name, pat) for pat in SKIP_PATTERNS):+ continue+ scanned += 1+ try:+ json.loads(path.read_text(encoding="utf-8"))+ except Exception as exc:+ bad.append((path, exc))++ if bad:+ for path, exc in bad:+ print(f"::error file={path}::JSON parse failed: {exc}")+ sys.exit(1)+ print(f"{scanned} JSON files parse cleanly")+ PY++ - name: codespell typo check (informational)+ # Catches typos in code, comments, and docs across the repo.+ # Skips lockfiles, generated assets, binary artefacts, and+ # the LICENSE files (US/UK spelling drift in legal text is+ # not ours to second-guess). The ignore-words-list pulls+ # out short identifiers + valid technical terms that+ # codespell's default dictionary would otherwise flag+ # (e.g. `ans` as a math-quiz variable name in+ # tests/utils/aime_eval.py, `parm`/`parms` in PyTorch+ # nn.Module idioms). Non-blocking until the surfaced typos+ # are fixed; drop continue-on-error after the cleanup.+ continue-on-error: true+ run: |+ codespell \+ --skip='*.lock,*.lockb,*.json,*.svg,*.png,*.jpg,*.jpeg,*.gif,*.ico,*.woff*,*.ttf,*.eot,*.zip,*.gz,*.gguf,*.safetensors,*.bin,node_modules,.git,build,dist,unsloth_compiled_cache,unsloth.egg-info,target,studio/frontend/dist,*.pyc,*-licenses.txt,LICENSE*' \+ --ignore-words-list='ans,bu,hel,fo,te,ot,hist,ned,sav,recurser,datas,nin,parm,parms,checkin,nd,fr,inout,donot,uint' \+ --quiet-level=2++ - name: shellcheck on committed *.sh (informational)+ # Goes beyond `bash -n` (which only parses): catches subtle+ # shell bugs like unquoted variable expansions, useless+ # `cat`, command substitutions inside `[[`, etc. The+ # install/setup scripts are critical-path so the signal is+ # worth surfacing. Non-blocking until install.sh's+ # hand-rolled patterns get cleaned up; drop continue-on-error+ # afterwards.+ continue-on-error: true+ run: |+ # Exclude SC1090 ("source not followable") -- legitimate+ # for installer scripts that source files at runtime+ # paths shellcheck cannot resolve statically.+ # SC2034 ("variable assigned but never used") fires on+ # the export-only assignment idiom we use in install.sh.+ shellcheck -e SC1090,SC2034 $(git ls-files '*.sh')++ - name: ruff format drift (informational)+ # The canonical formatter is scripts/run_ruff_format.py+ # = ruff format + scripts/enforce_kwargs_spacing.py, so plain+ # `ruff format --check` reports the kwarg-spacing diff as+ # drift. Surface the count for visibility but keep+ # non-blocking until the custom pipeline is wired in here.+ continue-on-error: true+ run: |+ ruff format --check unsloth unsloth_cli studio tests cli.py unsloth-cli.py- name: Import-hoist / alias-rename safety (changed Python files)# Runs the verifier in compare mode on every in-place-modified@@ -263,18 +552,6 @@ jobs:python scripts/verify_import_hoist.py \--before "$MERGE_BASE" --after "$HEAD_SHA" "${CHANGED[@]}"- - name: Duplicate-definition verifier self-test- # scripts/lint_duplicate_definitions.py refuses a name bound twice in one- # scope: two top-level copies of a def / class / constant, or a name- # imported twice. That is what a bad merge resolution leaves behind, and- # every other check here passes it -- compileall parses both copies, the- # ruff rule set (E9 / F63 / F7 / F82) has no F811, and the two copies- # usually differ only in their comments.- # Runs the tool's self-test cases, positive and negative, so a regression in the- # rule fails before we trust it on a diff. Stdlib-only, sub-second.- run: |- python scripts/lint_duplicate_definitions.py --self-test-- name: Duplicate definitions (changed Python files)# Same merge-base reasoning as the import-hoist step above: diff the merge# base against the PR head SHA so the range is the PR's own commits and@@ -334,273 +611,6 @@ jobs:python scripts/lint_duplicate_definitions.py \--before "$MERGE_BASE" --after "$HEAD_SHA" "${CHANGED[@]}"- - name: Duplicate definitions (whole tree)- # The diff-scoped step passes pre-existing findings, so the backlog regrew- # to 38 while it was the only gate. Runs on push too.- run: |- python scripts/lint_duplicate_definitions.py \- unsloth unsloth_cli studio scripts tests cli.py unsloth-cli.py-- - name: No silent llama-server parallel-slot downgrades- # #7717 clamped --parallel to 1 whenever MTP resolved, costing batched- # callers 4x throughput. Launch fewer slots only for a real capability- # or VRAM limit, marked '# allow-slot-clamp: <reason>'. Self-test first,- # so a regression in the rule fails before we trust it on the tree.- run: |- python scripts/lint_no_parallel_clamp.py --self-test- python scripts/lint_no_parallel_clamp.py-- - name: No duplicate TypeScript import bindings- # #8470 duplicated an import line in model-selector.tsx; tsc rejects it- # with TS2300 and every job that builds the frontend goes red behind- # it. eslint is not run in CI, so nothing saw it until tsc did, inside- # the slowest job in the matrix. Self-test first, so a regression in- # the rule fails before we trust it.- run: |- python scripts/lint_no_duplicate_ts_imports.py --self-test- python scripts/lint_no_duplicate_ts_imports.py-- - name: No leftover debugger / pdb / breakpoint calls- # Catches the "I'll just stick a breakpoint() here" mistake- # before it ships. AST-based so commented-out debugger- # markers don't false-positive (a bare grep would; there- # are three commented `# breakpoint()` markers in- # unsloth/models/rl* today). Sub-second.- run: |- python <<'PY'- import ast, pathlib, sys-- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",- "unsloth_compiled_cache", "node_modules",- "unsloth.egg-info"}-- bad = []- scanned = 0- for path in sorted(pathlib.Path(".").rglob("*.py")):- if any(part in SKIP_PARTS for part in path.parts):- continue- scanned += 1- try:- tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"))- except SyntaxError:- continue # compileall step above already failed this- for node in ast.walk(tree):- if not isinstance(node, ast.Call):- continue- fn = node.func- if isinstance(fn, ast.Name) and fn.id == "breakpoint":- bad.append((path, node.lineno, "breakpoint()"))- elif (isinstance(fn, ast.Attribute) and fn.attr == "set_trace"- and isinstance(fn.value, ast.Name)- and fn.value.id in {"pdb", "ipdb"}):- bad.append((path, node.lineno, f"{fn.value.id}.set_trace()"))-- if bad:- for path, lineno, what in bad:- print(f"::error file={path},line={lineno}::leftover {what} -- remove before merging")- sys.exit(1)- print(f"no leftover debugger calls (scanned {scanned} files)")- PY-- - name: License-header drift (informational; whole repo)- # Three header families are accepted across the repo:- # 1. SPDX one-liner: `# SPDX-License-Identifier: ...`- # Used across studio/ (AGPL-3.0-only) and a few new- # files elsewhere.- # 2. Apache-2.0 long form, marker phrase- # "Licensed under the Apache License". Used across- # unsloth/ and unsloth_cli/.- # 3. GNU long form, marker phrase "General Public License".- # That single substring covers GPL, LGPL ("GNU Lesser- # General Public License") and AGPL ("GNU Affero- # General Public License") preambles, all three of- # which appear in unsloth/kernels/* (LGPL/AGPL) without- # the SPDX line.- # Empty files (mainly empty __init__.py) are skipped.- # Surfaced as a warning; cleaning up the actual misses is a- # follow-up PR, not a CI fix.- continue-on-error: true- run: |- python <<'PY'- import pathlib-- ACCEPTED = (- "SPDX-License-Identifier", # any SPDX line- "Licensed under the Apache License", # Apache-2.0 long form- "General Public License", # GPL / LGPL / AGPL long form- )- # "vendor" is upstream third-party source: it carries its own licence.- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",- "unsloth_compiled_cache", "node_modules",- "unsloth.egg-info", "vendor"}-- studio_missing = []- other_missing = []- for path in sorted(pathlib.Path(".").rglob("*.py")):- if any(part in SKIP_PARTS for part in path.parts):- continue- text = path.read_text(encoding="utf-8", errors="replace")- if not text.strip():- continue # empty __init__.py etc.- head = "\n".join(text.splitlines()[:25])- if any(marker in head for marker in ACCEPTED):- continue- if "studio" in path.parts:- studio_missing.append(path)- else:- other_missing.append(path)-- total = len(studio_missing) + len(other_missing)- if total == 0:- print("every committed .py has a recognised license header")- else:- print(f"::warning::{total} Python files have no recognised license "- f"header (SPDX / Apache-2.0 / GNU long form): "- f"studio={len(studio_missing)}, other={len(other_missing)}")- for path in (studio_missing + other_missing)[:30]:- print(f" {path}")- if total > 30:- print(f" ... and {total - 30} more")- PY-- - name: Shell scripts parse cleanly (`bash -n`)- # Same idea as Python's compileall: parse-only check that- # every committed *.sh would not blow up at `bash script.sh`- # invocation time on a release box. tests/sh/ is the largest- # cluster (the install.sh shape tests).- run: |- shopt -s globstar- fail=0- for f in $(git ls-files '*.sh'); do- if ! bash -n "$f"; then- echo "::error file=$f::shell parse error"- fail=1- fi- done- if [ "$fail" -ne 0 ]; then- exit 1- fi- n=$(git ls-files '*.sh' | wc -l)- echo "$n shell scripts parse cleanly"-- - name: YAML files parse cleanly (yaml.safe_load)- # Catches truncated workflow files, broken indents in- # dependabot.yml / pre-commit configs, etc. Includes- # .github/workflows/*.yml so a typo in the file we just- # added shows up immediately.- run: |- python <<'PY'- import pathlib, sys, yaml-- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",- "node_modules", "unsloth_compiled_cache",- "unsloth.egg-info"}-- bad = []- scanned = 0- for path in sorted(list(pathlib.Path(".").rglob("*.yml"))- + list(pathlib.Path(".").rglob("*.yaml"))):- if any(part in SKIP_PARTS for part in path.parts):- continue- scanned += 1- try:- with path.open("r", encoding="utf-8") as fh:- list(yaml.safe_load_all(fh))- except Exception as exc:- bad.append((path, exc))-- if bad:- for path, exc in bad:- print(f"::error file={path}::YAML parse failed: {exc}")- sys.exit(1)- print(f"{scanned} YAML files parse cleanly")- PY-- - name: JSON files parse cleanly (json.loads)- # Catches malformed package.json, biome.json, etc. Skips:- # - huge npm/bun lockfiles (machine-generated, slow to- # parse, no value).- # - tsconfig*.json: TypeScript convention is JSONC (JSON- # with `/* ... */` comments), which standard json.loads- # rejects. Strip-and-validate would need json5 or a- # hand-rolled comment scrubber for marginal value, since- # `tsc --noEmit` already validates these in Frontend CI.- run: |- python <<'PY'- import fnmatch, json, pathlib, sys-- SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",- "node_modules", "unsloth_compiled_cache",- "unsloth.egg-info"}- SKIP_NAMES = {"package-lock.json", "bun.lock"}- SKIP_PATTERNS = ("tsconfig*.json",)-- bad = []- scanned = 0- for path in sorted(pathlib.Path(".").rglob("*.json")):- if any(part in SKIP_PARTS for part in path.parts):- continue- if path.name in SKIP_NAMES:- continue- if any(fnmatch.fnmatch(path.name, pat) for pat in SKIP_PATTERNS):- continue- scanned += 1- try:- json.loads(path.read_text(encoding="utf-8"))- except Exception as exc:- bad.append((path, exc))-- if bad:- for path, exc in bad:- print(f"::error file={path}::JSON parse failed: {exc}")- sys.exit(1)- print(f"{scanned} JSON files parse cleanly")- PY-- - name: codespell typo check (informational)- # Catches typos in code, comments, and docs across the repo.- # Skips lockfiles, generated assets, binary artefacts, and- # the LICENSE files (US/UK spelling drift in legal text is- # not ours to second-guess). The ignore-words-list pulls- # out short identifiers + valid technical terms that- # codespell's default dictionary would otherwise flag- # (e.g. `ans` as a math-quiz variable name in- # tests/utils/aime_eval.py, `parm`/`parms` in PyTorch- # nn.Module idioms). Non-blocking until the surfaced typos- # are fixed; drop continue-on-error after the cleanup.- continue-on-error: true- run: |- codespell \- --skip='*.lock,*.lockb,*.json,*.svg,*.png,*.jpg,*.jpeg,*.gif,*.ico,*.woff*,*.ttf,*.eot,*.zip,*.gz,*.gguf,*.safetensors,*.bin,node_modules,.git,build,dist,unsloth_compiled_cache,unsloth.egg-info,target,studio/frontend/dist,*.pyc,*-licenses.txt,LICENSE*' \- --ignore-words-list='ans,bu,hel,fo,te,ot,hist,ned,sav,recurser,datas,nin,parm,parms,checkin,nd,fr,inout,donot,uint' \- --quiet-level=2-- - name: shellcheck on committed *.sh (informational)- # Goes beyond `bash -n` (which only parses): catches subtle- # shell bugs like unquoted variable expansions, useless- # `cat`, command substitutions inside `[[`, etc. The- # install/setup scripts are critical-path so the signal is- # worth surfacing. Non-blocking until install.sh's- # hand-rolled patterns get cleaned up; drop continue-on-error- # afterwards.- continue-on-error: true- run: |- # Exclude SC1090 ("source not followable") -- legitimate- # for installer scripts that source files at runtime- # paths shellcheck cannot resolve statically.- # SC2034 ("variable assigned but never used") fires on- # the export-only assignment idiom we use in install.sh.- shellcheck -e SC1090,SC2034 $(git ls-files '*.sh')-- - name: ruff format drift (informational)- # The canonical formatter is scripts/run_ruff_format.py- # = ruff format + scripts/enforce_kwargs_spacing.py, so plain- # `ruff format --check` reports the kwarg-spacing diff as- # drift. Surface the count for visibility but keep- # non-blocking until the custom pipeline is wired in here.- continue-on-error: true- run: |- ruff format --check unsloth unsloth_cli studio tests cli.py unsloth-cli.py# Collects the two lanes launched before the lint steps. This step is the whole# reason the backgrounding is safe: without it a lane's failure would be invisiblediff --git a/.github/workflows/studio-backend-ci.yml b/.github/workflows/studio-backend-ci.ymlindex 2342f45bd06..dcb3d90c727 100644--- a/.github/workflows/studio-backend-ci.yml+++ b/.github/workflows/studio-backend-ci.yml@@ -183,6 +183,13 @@ jobs:- name: Install bubblewrap, so the sandbox tests run instead of skipping# Bound apt retries; 25 minutes covers both calls' 1450s worst case.+ #+ # Background: apt (system packages) and the pip install below (this job's+ # Python) touch nothing in common, so the ~10 s of apt overlaps the ~80 s of+ # pip. Collected right before the first pytest step, which is the first thing+ # that needs bwrap.+ id: bubblewrap+ background: truetimeout-minutes: 25env:RETRY_ATTEMPTS: '2'@@ -239,6 +246,9 @@ jobs:# extensions under torch < 2.11 and the pure-Python constructors still import.pip install torchao+ - name: Collect the bubblewrap install+ wait: bubblewrap+- name: Backend testsif: matrix.scope == 'full'working-directory: studio/backenddiff --git a/.github/workflows/studio-windows-inference-smoke.yml b/.github/workflows/studio-windows-inference-smoke.ymlindex f2f87bc3f0c..7fcae4bc6e4 100644--- a/.github/workflows/studio-windows-inference-smoke.yml+++ b/.github/workflows/studio-windows-inference-smoke.yml@@ -161,7 +161,17 @@ jobs:# Fast GPU-free gate: parse install.ps1 + setup.ps1 and run the PowerShell# unit tests (CUDA-toolkit + torch-flavor helpers) before the heavy GGUF smoke.+ #+ # Background, together with the uninstall test below: about 60 s of pure+ # PowerShell that reads only the checkout, so it overlaps the node/python setup+ # and the ~90 s uv cache restore instead of running ahead of them. Both are+ # collected by the wait step right before Install Unsloth, so a failure still+ # stops the job before the install, as it did when they ran first; they are not+ # overlapped with the install itself, because test_mirror_fallback.ps1 asserts+ # deadlines and transfer speeds that a CPU-bound install next to it could skew.- name: PowerShell installer unit tests+ id: ps-installer-units+ background: trueshell: pwshrun: |foreach ($f in @('install.ps1', 'studio/setup.ps1')) {@@ -194,6 +204,8 @@ jobs:# uninstall.ps1: native uninstall must keep the shared unsloth.ico while a# WSL shortcut still references it (dual install), else that shortcut blanks.- name: uninstall.ps1 unit test (dual-install icon preserve)+ id: ps-uninstall-unit+ background: trueshell: pwshrun: |$errs = $null@@ -267,6 +279,9 @@ jobs:id: fe-distuses: ./.github/actions/frontend-dist-restore+ - name: Collect the PowerShell unit tests+ wait: [ps-installer-units, ps-uninstall-unit]+- name: Install Unsloth (--local, --no-torch)id: installtimeout-minutes: 25diff --git a/.github/workflows/studio-windows-update-smoke.yml b/.github/workflows/studio-windows-update-smoke.ymlindex 4e075783f74..a4ddeedd142 100644--- a/.github/workflows/studio-windows-update-smoke.yml+++ b/.github/workflows/studio-windows-update-smoke.yml@@ -114,7 +114,14 @@ jobs:persist-credentials: false# This workflow triggers on uninstall.ps1; neither test really uninstalls.+ #+ # These three unit-test steps run in the background: they read only the+ # checkout, and together take about 20 s that now overlaps the node/python+ # setup and the ~90 s uv cache restore. The wait step right before Install+ # Unsloth collects them, so a failure still stops the job before the install.- name: uninstall.ps1 unit tests (argument guard, dual-install icon preserve)+ id: ps-uninstall-units+ background: trueshell: pwshrun: |# Propagate each child test's failure before continuing.@@ -126,6 +133,8 @@ jobs:# Both entry points prepend the 5.1 system module dir, and Refresh-Environment# must not reload PSModulePath back over it before the uv installer runs.- name: PSModulePath normalization unit tests+ id: ps-modulepath-units+ background: trueshell: pwshrun: pwsh -NoProfile -File tests/studio/test_psmodulepath_normalization.ps1@@ -133,6 +142,8 @@ jobs:# Windows PowerShell 5.1: the shell the README one-liner lands in, handled# nowhere else in CI, and it differs from pwsh 7 on native stderr.- name: uninstall.ps1 argument guard under Windows PowerShell 5.1+ id: ps51-uninstall-guard+ background: trueshell: powershellrun: |powershell -NoProfile -File tests/studio/test_uninstall_arg_guard.ps1@@ -223,6 +234,9 @@ jobs:id: fe-distuses: ./.github/actions/frontend-dist-restore+ - name: Collect the PowerShell unit tests+ wait: [ps-uninstall-units, ps-modulepath-units, ps51-uninstall-guard]+- name: Install Unsloth (--local, --no-torch)shell: pwshenv: