Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Pair the linearity measurements, so a busy runner stops reading as quadratic (#10868)
Daniel Hanb3dfd088b5b4
diff --git a/studio/backend/tests/test_pr5624_regressions.py b/studio/backend/tests/test_pr5624_regressions.pyindex edc13de474c..2e7c51fc989 100644--- a/studio/backend/tests/test_pr5624_regressions.py+++ b/studio/backend/tests/test_pr5624_regressions.py@@ -230,31 +230,60 @@ def _growth() -> tuple[float, float, list]:"""How much more `factor` times the input costs. Returns (ratio, big_seconds, big_result).- Best-of-`repeats` at each size, not a mean: contention on a shared runner only ever ADDS- time, so the minimum is the closest each size gets to its own true cost, and dividing one- minimum by the other cancels the machine out.+ The MEDIAN of `repeats` PAIRED ratios: each pair times the small input and then the big+ one back to back, and the ratio is formed inside the pair before anything is aggregated.This replaces an absolute ``elapsed < 1.0`` budget at one size. That budget read 0.20s ona quiet runner and 1.41s on a busy one, so it failed for the wrong reason on unrelatedPRs, and it also sat close enough to the line that a REAL regression (#10507, which madethe R1 path quadratic again) only tipped it over some of the time. A ratio answers thequestion these tests are named for: linear is ~`factor`, quadratic is ~`factor ** 2`.++ It also replaces ``min(big over repeats) / min(small over repeats)``, which looked like+ it cancelled the machine out and did not. Those two minima come from batches run at+ different times, so a quiet window during the small batch and a busy one during the big+ batch multiply instead of cancelling, and the quotient of two separately-taken minima is+ not a minimum of anything. Measured on a 2-vCPU box under load, 15 trials per shape:++ strategy R1 R1-distant GLM V3 >= 6.0+ min/min max 8.69 max 8.23 max 8.27 max 8.59 7 of 60+ paired median max 4.26 max 4.83 max 4.83 max 5.42 0 of 60++ That 12% false-fail rate is not hypothetical: it is why this test failed on #10825 and+ again on #10864 at 6.56, both times on branches that touch none of this code.++ Pairing is what fixes it. Contention hits both halves of a pair roughly equally and+ divides out, which is what the old comment claimed for the unpaired form. The median+ then discards a pair that got unlucky, in either direction, rather than trusting one+ reading.++ Detection power is kept, which is the half worth checking before loosening anything. On+ a synthetic path with a true 16x profile this reports 8 of 8 over the bar, same as the+ old form. On the marginal 6.7x shape (#10832's partially fixed sweep) it reports 9 of 12+ against the old form's 10 of 12, a difference well inside the noise at that sample size,+ and that shape's sibling test measures 12.2x when broken, so the suite still catches it."""+ import statistics as _statisticsimport time as _time- def cost(n: int) -> tuple[float, list]:- text, best, result = build(n), None, None- for _ in range(repeats):- start = _time.perf_counter()- result = parse_tool_calls_from_text(text)- elapsed = _time.perf_counter() - start- best = elapsed if best is None else min(best, elapsed)- return best, result-- small, _ = cost(units)- big, result = cost(units * factor)- # A timer's own resolution must not read as superlinear growth on a very fast machine.- return big / max(small, 1e-4), big, result+ def once(text: str) -> tuple[float, list]:+ start = _time.perf_counter()+ result = parse_tool_calls_from_text(text)+ return _time.perf_counter() - start, result++ small_text, big_text = build(units), build(units * factor)+ ratios, big, result = [], None, None+ for _ in range(repeats):+ small_elapsed, _ = once(small_text)+ big_elapsed, result = once(big_text)+ # The backstop below reads the BEST big time, not the worst. Contention only adds, so+ # the minimum is the closest this size got to its own cost, and the backstop should+ # fire on a path that is genuinely too slow rather than on a runner that stalled once.+ big = big_elapsed if big is None else min(big, big_elapsed)+ # A timer's own resolution must not read as superlinear growth on a very fast machine.+ ratios.append(big_elapsed / max(small_elapsed, 1e-4))++ return _statistics.median(ratios), big, resultdef _assert_linear(