Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #625 from bharathgnana/fix/anthropic-stop-sequences
c031307 Vincenzo Fornaro committed 3 months ago
Merge pull request #627 from ZacharyZcR/feat/fmt6-cuda-kernel
8245766 Vincenzo Fornaro committed 3 months ago
cuda: fmt=6 (E8/IQ3) decode, so E8 containers can reach VRAM
ce25594 ZacharyZcR committed 3 months ago
Apply the implicit GLM stop sequences on /v1/messages too
033df1a bharathgnana committed 3 months ago
Merge pull request #624 from JustVugg/fix/622-metal-prefill-docs-debug-logits
6fcc555 Vincenzo Fornaro committed 3 months ago
metal/oracle: scope the token-exact claim to decode + implement DEBUG_LOGITS top-5 dump (#622)
a9b8a47 JustVugg committed 3 months ago
Merge pull request #623 from manjumihir/fix/macos-hwinfo
9c45c58 Vincenzo Fornaro committed 3 months ago
Merge pull request #615 from bherald/contrib/doctor-deep-preflight
2d1cb27 Vincenzo Fornaro committed 3 months ago
Merge pull request #618 from 4ny3l/fix-windows-portability-security
6ec554b Vincenzo Fornaro committed 3 months ago
Merge pull request #617 from EgonRuiter/perf/simd-matrix-reductions-fix442
5a906fe Vincenzo Fornaro committed 3 months ago
telemetry: read real hwinfo on macOS (sysctl + host_statistics64)
751e141 manjumihir committed 3 months ago
Add opt-in per-expert causal-ablation harness (ABLATE_SCORE) for moe()
ae81fa7 jeswr committed 3 months ago
fix(st.h): prevent integer overflow in st_read_slice_f32 bounds check
c1869cc 4ny3l committed 3 months ago
perf(simd): add OpenMP SIMD reductions to hot-path matrix multiplication loops (#442)
961e81f Egon Ruiter committed 3 months ago
Merge pull request #614 from ZacharyZcR/fix/586-missing-tensor-diagnostics
21e7a35 Vincenzo Fornaro committed 3 months ago
Merge pull request #613 from silios/docs/benchmark-i5-13600k-5070ti
7cf3448 Vincenzo Fornaro committed 3 months ago
Merge pull request #611 from NeuralNotwerk/fix/hip-pic-link
9640282 Vincenzo Fornaro committed 3 months ago
loader: diagnose a missing core tensor instead of just naming it
3e5d68a ZacharyZcR committed 3 months ago
docs(benchmarks): i5-13600K + RTX 5070 Ti datapoint - 0.98 tok/s, hit 54.7% (#605)
5eb7ee4 Silios committed 3 months ago
Merge pull request #610 from ZacharyZcR/fix/597-http-keepalive-framing
fb0a726 Vincenzo Fornaro committed 3 months ago