Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
tools: commit run_f8_bench.sh (BENCH_PEAK_GBPS=273 default for GB10)
monotophic669b488e5524
diff --git a/c/tools/run_f8_bench.sh b/c/tools/run_f8_bench.shnew file mode 100755index 00000000..8a181ef3--- /dev/null+++ b/c/tools/run_f8_bench.sh@@ -0,0 +1,24 @@+#!/usr/bin/env bash+# Build + run the fmt=8 kernel bench (c/tests/bench_fp8_cuda.cu) on a CUDA+# host. JSON on stdout, build output and progress on stderr, so the caller can+# pipe stdout straight into a file or jq. Extra make variables pass through:+#+# bash c/tools/run_f8_bench.sh # -arch=native (default)+# bash c/tools/run_f8_bench.sh CUDA_ARCH=sm_121 # explicit arch+#+# The bench times the old per-(o,s) fmt=8 kernels against the COLI_CUDA_F8_WARP+# rework (LUT decode and, where the toolchain has cuda_fp8.h, the cvt decode)+# at S in {1,4,8,32} on the census expert shapes, reporting weight-goodput+# GB/s and % of the BENCH_PEAK_GBPS roofline.+#+# BENCH_PEAK_GBPS defaults to 273 — the GB10 Spark wire (256-bit LPDDR5X x+# 8533 MT/s), this harness's primary host. It must be an explicit number+# because cudaDevAttrMemoryClockRate is device-inconsistent: on GB10 it+# returns the EFFECTIVE 8,533,000 kHz rate, so the usual 2x DDR factor would+# double-count to ~546 GB/s. Override for any other host, or set it empty+# (BENCH_PEAK_GBPS=) to get pct_peak=-1 plus the raw attributes to judge from.+set -euo pipefail+export BENCH_PEAK_GBPS="${BENCH_PEAK_GBPS-273}"+cd "$(dirname "$0")/.."+make fp8_bench CUDA=1 "$@" 1>&2+./fp8_bench