Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
metal: fix fmt=0 (raw f32) NULL-scale segfault in coli_metal_matmul
90ec4f6 njloof committed 2 months ago
added block on recursion reading array on array, added Q 4/5/8 on dequant, added tests
9a81c3e matteo committed 2 months ago
tests: drop the per-test _putenv_s helpers now that compat.h's setenv works
c6d6b10 texasich committed 2 months ago
build: restore the missing recipe for bench_inkling_shared_batch
65753ec texasich committed 2 months ago
Merge pull request #1428 from JustVugg/fix/windows-setenv-putenv
899e7c2 Vincenzo Fornaro committed 2 months ago
fix(compat): setenv must be visible to getenv in the same process (Windows)
51a7f70 JustVugg committed 2 months ago
Merge pull request #1423 from kreuzzelg/fix/ehit-lazy-alloc-race
585a745 Vincenzo Fornaro committed 2 months ago
improved tests in case main library is not installed, skip in that case
a4eaaa9 matteo committed 2 months ago
engines: publish the lazy HITS table under a lock (the parallel warmstart segfaulted on first touch)
fc6e7bb kreuzzelg committed 2 months ago
added general tool wrapper, gguf dequant and special olmoe conversion file, added tests
c054523 matteo committed 2 months ago
Merge pull request #1418 from dajiaohuang/fix/v4-test-env-windows
6f9117f Vincenzo Fornaro committed 2 months ago
Merge pull request #1414 from dajiaohuang/fix/glm53-serve-cancel
50b139e Vincenzo Fornaro committed 2 months ago
Merge pull request #1415 from dajiaohuang/fix/fp8-e2e-windows-cflags
63f3b51 Vincenzo Fornaro committed 2 months ago
Merge pull request #1419 from dajiaohuang/fix/registry-test-windows
01d886c Vincenzo Fornaro committed 2 months ago
test(v4): make the ExpertStore knobs reach getenv() on Windows
9d86d9b dajiaohuang committed 2 months ago
test(v4): make the expert-store registry test build on Windows
de5886a dajiaohuang committed 2 months ago
fix(glm53 serve): honor CANCEL while the turn is still running (#1332)
8e09479 dajiaohuang committed 2 months ago
test(fp8): mirror the Windows CFLAGS in the e2e harness build (#1413)
8ae8758 dajiaohuang committed 2 months ago
Merge pull request #1404 from Petsku01/fix/qwen36-tier-int8-decode-offer
173c757 Vincenzo Fornaro committed 2 months ago
Merge pull request #1411 from JustVugg/fix/vram-prefix-against-headroom
9305123 Vincenzo Fornaro committed 2 months ago
cuda tier: size the VRAM prefix against the measured headroom, not the requested budget (#1405)
6cbd5aa JustVugg committed 2 months ago
Merge pull request #1399 from SebaWag/feat/trunk-resident-layers
80eda61 Vincenzo Fornaro committed 2 months ago
fix(trunk): guard mmap views in segment/edge destroy; gate the knob to the full-model load
da90821 SebaWag committed 2 months ago
Merge pull request #1410 from JustVugg/fix/vram-tier-not-starved-by-ram-pin
1c094d6 Vincenzo Fornaro committed 2 months ago
cuda tier: size the VRAM prefix from the VRAM budget on a single GPU, and put the dense trunk on a card that holds it (#1409)
1af1a2a JustVugg committed 2 months ago
Merge pull request #1408 from JustVugg/build/msvc-x64-guard
516bf3b Vincenzo Fornaro committed 2 months ago
Merge pull request #1407 from JustVugg/ci/portable-cuda-dll-artifact
a2ff4f2 Vincenzo Fornaro committed 2 months ago