Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Say when a failing pull_request job is testing a stale merge ref (#11386)
Daniel Han8090d9523fe0
diff --git a/.github/actions/merge-ref-age/action.yml b/.github/actions/merge-ref-age/action.ymlnew file mode 100644index 00000000000..acf9d9db735--- /dev/null+++ b/.github/actions/merge-ref-age/action.yml@@ -0,0 +1,68 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.++# Say, on a failure, how far behind the base branch this run's merge ref is.+#+# A pull_request run does not test the PR's head. It tests `refs/pull/N/merge`, a+# merge of the head into the base as the base stood WHEN THE RUN WAS CREATED. That+# ref is not recomputed because the base later moved, so a failure fixed on main+# ten minutes after a run started keeps failing that run forever, and re-running it+# does not help: a re-run replays the same merge commit.+#+# That is not hypothetical here. #11372 gave three test files `encoding = "utf-8"`+# and merged at 14:35 UTC. Runs created at 13:39, 14:26 and 14:29 on #11375, #11357+# and #11373 all kept failing `tests/test_source_read_encoding.py` on exactly those+# three lines, and all three were read as defects in branches that had never touched+# the files. The only cure is a push to the PR, which is what makes GitHub build a+# new merge ref.+#+# The cost of not saying this is that someone debugs a fix that already landed. The+# step is `if: failure()` so a green job pays nothing for it, and it only speaks when+# the base has actually moved, so it does not add noise to the common case where the+# failure is real.+#+# It NEVER changes the job's result. A stale merge ref does not make a failure+# untrue -- the merge really does fail -- and deciding that for the reader would be+# worse than the confusion it is fixing. It reports and gets out of the way.++name: Report the age of this run's merge ref+description: >-+ On a pull_request run, emit a notice saying how many commits the base branch+ has gained since this run's merge ref was built. Advisory only: it never fails,+ and never changes the outcome of the job it sits in.++runs:+ using: composite+ steps:+ - name: Compare the merge ref's base against the branch tip+ shell: bash+ env:+ # `gh` needs a token, and checkout runs with persist-credentials: false+ # in these jobs, so there is nothing in the git config to fall back on.+ # The compare endpoint only wants `contents: read`.+ GH_TOKEN: ${{ github.token }}+ BASE_SHA: ${{ github.event.pull_request.base.sha }}+ BASE_REF: ${{ github.event.pull_request.base.ref }}+ REPOSITORY: ${{ github.repository }}+ run: |+ set -u+ if [ -z "${BASE_SHA:-}" ] || [ -z "${BASE_REF:-}" ]; then+ # Not a pull_request event. Nothing to compare against.+ exit 0+ fi+ # base.sha is the commit the merge ref was built on; BASE_REF is where the+ # branch is now. `ahead_by` on base...ref is how far the branch has moved+ # since, which is the number this is about. Asking the API rather than the+ # clone on purpose: these checkouts are shallow, and deepening them enough+ # to hold both commits costs more than the answer is worth.+ behind=$(gh api "repos/${REPOSITORY}/compare/${BASE_SHA}...${BASE_REF}" --jq '.ahead_by' 2>/dev/null) || behind=""+ case "${behind}" in+ ''|*[!0-9]*)+ # No answer, or not a number. Say nothing rather than guess.+ exit 0+ ;;+ esac+ if [ "${behind}" -eq 0 ]; then+ exit 0+ fi+ echo "::notice title=This run's merge ref is ${behind} commit(s) behind ${BASE_REF}::A pull_request run tests a merge of this branch into ${BASE_REF} as it stood when the run was created, and ${BASE_REF} has gained ${behind} commit(s) since. If this failure was fixed on ${BASE_REF} in that window, the fix is not in this run, and re-running will not pick it up because a re-run replays the same merge commit. Push to the branch to have the merge ref rebuilt. This notice does not mean the failure is spurious."diff --git a/.github/workflows/studio-backend-ci.yml b/.github/workflows/studio-backend-ci.ymlindex 2321d6d36e0..5eacca9d927 100644--- a/.github/workflows/studio-backend-ci.yml+++ b/.github/workflows/studio-backend-ci.yml@@ -437,6 +437,14 @@ jobs:tests/test_recommended_folders_permission.py \tests/test_hf_cache_settings.py+ - name: Why a failure here might already be fixed on the base branch+ # `if: failure()` so a green job never runs it, and the action itself stays+ # quiet unless the base branch has actually moved since this run's merge ref+ # was built. See .github/actions/merge-ref-age for the three pull requests+ # that were misread this way.+ if: failure() && github.event_name == 'pull_request'+ uses: ./.github/actions/merge-ref-age+- name: Save the pip cacheif: always()uses: ./.github/actions/pip-cache-save@@ -719,6 +727,14 @@ jobs:if: matrix.shard == 'rest'run: python tests/validate_studio_features.py+ - name: Why a failure here might already be fixed on the base branch+ # `if: failure()` so a green job never runs it, and the action itself stays+ # quiet unless the base branch has actually moved since this run's merge ref+ # was built. See .github/actions/merge-ref-age for the three pull requests+ # that were misread this way.+ if: failure() && github.event_name == 'pull_request'+ uses: ./.github/actions/merge-ref-age+- name: Save the pip cacheif: always()uses: ./.github/actions/pip-cache-savediff --git a/.github/workflows/workflow-trigger-lint.yml b/.github/workflows/workflow-trigger-lint.ymlindex fa466866bc7..6f7583c9c6b 100644--- a/.github/workflows/workflow-trigger-lint.yml+++ b/.github/workflows/workflow-trigger-lint.yml@@ -160,6 +160,7 @@ jobs:tests/studio/test_main_runs_survive_merge_bursts.py \tests/studio/test_pull_requests_cancel_superseded_runs.py \tests/studio/test_probe_cost_claim_is_measured.py \+ tests/studio/test_merge_ref_age_is_reported.py \tests/studio/test_pester_bootstrap_hardening.py \tests/studio/test_playwright_suites_run_in_ci.py \tests/studio/test_sdk_installs_are_major_bounded.py \diff --git a/tests/studio/test_merge_ref_age_is_reported.py b/tests/studio/test_merge_ref_age_is_reported.pynew file mode 100644index 00000000000..efd4d0074cf--- /dev/null+++ b/tests/studio/test_merge_ref_age_is_reported.py@@ -0,0 +1,104 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0++"""A failing pytest job says how old its merge ref is, and saying so cannot fail the job.++A `pull_request` run tests `refs/pull/N/merge`, a merge of the branch into the base as+the base stood when the run was created. GitHub does not rebuild that ref because the+base later moved, and a re-run replays the same merge commit, so a fix that lands on+main after a run starts is invisible to that run for as long as the branch sits still.++That cost real time: #11372 added `encoding = "utf-8"` to three test files and merged at+14:35 UTC, and runs created at 13:39, 14:26 and 14:29 on #11375, #11357 and #11373 went+on failing `tests/test_source_read_encoding.py` on exactly those three lines. All three+branches had never touched the files.++`.github/actions/merge-ref-age` prints a notice that says so. What this file keeps is+the two properties that make it worth having, because both are easy to lose:++ 1. Every job here that runs pytest carries it, and carries it gated on `failure()`.+ A job added later that does not is a job where the same confusion returns.++ 2. It cannot fail a job. It runs exactly when a job is already failing, so a bug in+ it would land as a second, misleading error on top of the real one. The action's+ script therefore never exits non-zero and never emits `::error::`.++It deliberately does NOT check the wording of the notice. The message is prose, prose+guards have to parse English to be right, and the value here is that the step exists and+is harmless, not that any particular sentence survives.+"""++from __future__ import annotations++import re+from pathlib import Path++import pytest+import yaml++ROOT = Path(__file__).resolve().parents[2]+WORKFLOW = ROOT / ".github" / "workflows" / "studio-backend-ci.yml"+ACTION = ROOT / ".github" / "actions" / "merge-ref-age" / "action.yml"++_USES = "./.github/actions/merge-ref-age"+++def _jobs() -> dict:+ return yaml.safe_load(WORKFLOW.read_text(encoding = "utf-8"))["jobs"]+++def _jobs_that_run_pytest() -> list[str]:+ names = []+ for name, job in _jobs().items():+ if any("pytest" in str(step.get("run", "")) for step in job.get("steps", [])):+ names.append(name)+ return names+++def test_some_job_here_runs_pytest():+ """Rule 1 below is vacuous if nothing matches it, which is how such rules die."""+ assert _jobs_that_run_pytest(), (+ f"{WORKFLOW.name} no longer has a job that runs pytest, so either the suites "+ f"moved and this guard should move with them, or it is reading the wrong file"+ )++[email protected]("job_name", _jobs_that_run_pytest())+def test_every_pytest_job_reports_the_age_of_its_merge_ref(job_name):+ steps = _jobs()[job_name]["steps"]+ reporting = [step for step in steps if step.get("uses") == _USES]+ assert reporting, (+ f"job {job_name!r} in {WORKFLOW.name} runs pytest but does not use {_USES}. A "+ f"failure in it cannot tell the reader that the base branch moved since the "+ f"merge ref was built, which is how a fix that already landed gets debugged "+ f"a second time"+ )+ for step in reporting:+ condition = str(step.get("if", ""))+ assert "failure()" in condition, (+ f"job {job_name!r} runs {_USES} under {condition!r}. It is meant to speak "+ f"only when the job has already failed; on a green job the notice is noise "+ f"and the API call is waste"+ )+++def test_reporting_the_age_cannot_fail_the_job():+ """It runs on the failure path, where a second error would bury the first one."""+ action = ACTION.read_text(encoding = "utf-8")+ body = yaml.safe_load(action)+ scripts = [str(step.get("run", "")) for step in body["runs"]["steps"]]+ assert any(scripts), f"{ACTION} has no script left to check"++ for script in scripts:+ # Comments explain the exits; the code is what runs.+ code = "\n".join(line for line in script.splitlines() if not line.lstrip().startswith("#"))+ bad = re.findall(r"^\s*exit\s+(?!0\b)\S+", code, re.M)+ assert not bad, (+ f"{ACTION} exits non-zero ({bad}). It runs only when the job is already "+ f"failing, so a non-zero exit here adds a second error on top of the real "+ f"one and points the reader at the wrong thing"+ )+ assert "::error" not in code, (+ f"{ACTION} emits ::error::. Advisory only: a stale merge ref does not make "+ f"the failure untrue, and annotating it as an error says it does"+ )