Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Maxwell sm_52 floor for the opt-in old-card build (#3218)
Niko122161ede87e1bed
diff --git a/CMakeLists.txt b/CMakeLists.txtindex 6356e27b6..e18e61618 100644--- a/CMakeLists.txt+++ b/CMakeLists.txt@@ -213,9 +213,16 @@ elseif(STRATA_ENABLE_CUDA)"CMAKE_CUDA_ARCHITECTURES is '${CMAKE_CUDA_ARCHITECTURES}'. (A Pascal sm_60 or Volta sm_70 build is ""-DSTRATA_EXPERIMENTAL_SM60=ON; upstream does not support it.)")endif()- if(_base LESS 60)+ if(_base LESS 52)message(FATAL_ERROR "CMAKE_CUDA_ARCHITECTURES '${CMAKE_CUDA_ARCHITECTURES}': even the experimental build "- "needs compute capability 6.0 (sm_60) or newer")+ "needs compute capability 5.2 (sm_52, Maxwell) or newer")+ endif()+ # Maxwell (sm_52, #3218): one reporter built and ran this on two Quadro M6000 (CUDA 12.x, Qwen3.8-Flash-Next+ # IQ1, about 22 t/s). It has no native half-precision arithmetic, so it is slower per byte than Pascal; only+ # this opt-in build gets the floor. Warn, do not refuse.+ if(_base LESS 60)+ message(WARNING "Building for sm_${_base} (Maxwell): community-tested on one reporter's Quadro M6000 only, "+ "no native half-precision arithmetic, CUDA 12.x toolkit required (docs/OLDER_GPUS.md).")endif()# #220 / #224: RTX 50 engines built with CUDA 12.8 crashed in the prompt path on Linux (13.0 and newer did not)if(_base GREATER_EQUAL 120 AND CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 13.0)diff --git a/docs/OLDER_GPUS.md b/docs/OLDER_GPUS.mdindex 92c26f9c8..88036809c 100644--- a/docs/OLDER_GPUS.md+++ b/docs/OLDER_GPUS.md@@ -21,6 +21,7 @@ unchanged. Numbers are the reporters' own, on one machine each, unless the text| Cards | Compute capability | How it runs | What is different on it || --- | --- | --- | --- |+| Quadro M6000, GTX 900 series (Maxwell) | 5.2 | a source build with CUDA 12.x | Opt-in, one reporter, [below](#maxwell-sm_52). No native half-precision arithmetic. Not in the ready-made CUDA 12 engine || Tesla P100 | 6.0 | the CUDA 12 engine | `__dp4a` emulated (bit-exact); BF16 projections through fp32; seven scheduling switches on by default ([below](#tesla-p100-sm_60)); opt-in `STRATA_Q8_SM60=1` ([below](#pascal-gp100-a-q8_0-decode-kernel-opt-in)) || Tesla P40 / P4, GTX 10 series | 6.1 | the CUDA 12 engine | BF16 projections through fp32 (cuBLAS has no BF16 GEMM there); dense weights dequantized by a wide kernel; expert mode 8 and its own interleaved-matvec table; opt-in `STRATA_DENSE_F32=1` ([below](#tesla-p40-sm_61-what-was-measured-and-changed)) || Tesla V100, Titan V | 7.0 | the CUDA 12 engine | BF16 projections on the FP16 tensor cores; prompt attention on `mma.m8n8k4`; a leaner attention kernel. See [NVIDIA_V100.md](NVIDIA_V100.md) |@@ -41,6 +42,22 @@ too and cannot hold any of the model.| Radeon PRO V520 / Pro 5600M (RDNA1) | gfx1011 | built by hand: `-DCMAKE_HIP_ARCHITECTURES=gfx1011`; builds with 0 errors, untested on hardware || Instinct MI50 / MI60, Radeon VII | gfx906 (wave64) | built by hand: `-DSTRATA_HIP_GFX906=ON` |+### Maxwell (sm_52)++One reporter ran Strata v0.1.44.1 on two Quadro M6000 24 GB (Ubuntu, driver 580) with Qwen3.8-Flash-Next at IQ1 and saw about+22 tokens a second. Nobody has checked the answer text on these cards yet, and Maxwell has no native half-precision arithmetic,+so treat it as a test. The ready-made CUDA 12 engine holds no sm_52 code; build from source with a CUDA 12.x toolkit (CUDA 13+cannot compile for Maxwell):++```+cmake -B build -G Ninja -DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=52+cmake --build build -j+```++CMake prints a warning for sm_52 and goes on. A release build still needs sm_75 or newer, and without `STRATA_EXPERIMENTAL_SM60=ON`+the build stops as before. For `setup.py` to list a Maxwell card as usable, set `STRATA_EXPERIMENTAL_SM52=1`; setup warns and uses+the CUDA 12 path. If you try it, please post a short chat and one code answer from the card.+## NVIDIA: the CUDA 12 engineCUDA 13 dropped Pascal and Volta and cannot compile for them. Setup therefore keeps a second engine, built with CUDA 12.9 anddiff --git a/setup.py b/setup.pyindex 55f8941f2..de04c5971 100644--- a/setup.py+++ b/setup.py@@ -1548,7 +1548,12 @@ def experimental_sm60() -> bool:def sm60_card(arch) -> bool:- return 60 <= int(arch) <= 70+ """Pascal (6.x) and Volta (7.0) cards; Maxwell 5.2 (#3218) only for a tester who set STRATA_EXPERIMENTAL_SM52=1,+ because the ready-made CUDA 12 engine holds no sm_52 code: a Maxwell card is built from source, by that tester."""+ a = int(arch)+ if a == 52:+ return os.environ.get("STRATA_EXPERIMENTAL_SM52", "").strip() == "1"+ return 60 <= a <= 70def old_gpus_opt_in(found, named=(), cuda=None, other=False):@@ -8881,6 +8886,10 @@ def main() -> int:if OLD_GPUS and a.backend != "hip":warn(f"older NVIDIA GPUs (Pascal / Volta) can be used ({OLD_GPUS}): experimental, through a second engine ""built with CUDA 12 (docs/OLDER_GPUS.md)")+ if any(int(g["arch"]) == 52 for g in found):+ warn("a Maxwell card (compute capability 5.2) has no fast half-precision path; it was tested by one reporter "+ "only, on a source build (-DSTRATA_EXPERIMENTAL_SM60=ON, CUDA 12.x). Check that the answer text reads "+ "correctly (docs/OLDER_GPUS.md)")nv_ok = any(gpu_problem(g) is None for g in found)hip = a.backend == "hip" or (a.backend is None and not nv_ok and bool(amd_ok))if a.backend is None and nv_ok and amd_ok:diff --git a/tools/test_setup_older_gpus.py b/tools/test_setup_older_gpus.pyindex d11419e15..f66d121ef 100644--- a/tools/test_setup_older_gpus.py+++ b/tools/test_setup_older_gpus.py@@ -88,6 +88,21 @@ class OptIn(unittest.TestCase):self.assertIn("only kind", setup.old_gpus_opt_in([V100]))self.assertIsNone(setup.old_gpus_opt_in([V100], other=True)) # an AMD card it can use insteadself.assertIsNone(setup.old_gpus_opt_in([card(0, "GTX 980", 4.0, "52")]))+ # #3218: Maxwell only for a tester who sets STRATA_EXPERIMENTAL_SM52=1 (the default stays unchanged)+ m6000 = card(0, "Quadro M6000", 24.0, "52")+ self.assertFalse(setup.sm60_card("52"))+ self.assertTrue(setup.sm60_card("60") and setup.sm60_card("70") and not setup.sm60_card("75"))+ self.assertIn("not supported", setup.gpu_problem(m6000))+ self.assertNotIn("experimental: choose it", setup.gpu_problem(m6000))+ with mock.patch.dict(os.environ, {"STRATA_EXPERIMENTAL_SM52": "1"}):+ self.assertTrue(setup.sm60_card("52"))+ self.assertFalse(setup.sm60_card("50"))+ self.assertIn("experimental: choose it", setup.gpu_problem(m6000))+ self.assertIn("only kind", setup.old_gpus_opt_in([m6000]))+ self.assertIsNone(setup.old_gpus_opt_in([rtx, m6000])) # a newer card keeps winning+ self.assertEqual(setup.cuda_choice([52])[0], 12)+ with mock.patch.dict(os.environ, {"STRATA_EXPERIMENTAL_SM60": "1"}):+ self.assertIsNone(setup.gpu_problem(m6000)) # warned, not refusedwith mock.patch.dict(os.environ, {"STRATA_EXPERIMENTAL_SM60": "1"}):self.assertEqual(setup.old_gpus_opt_in([rtx, v100]), "STRATA_EXPERIMENTAL_SM60=1")