Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
Merge pull request #456 from monotophic/convert/resume-guard
bb2bae8 Vincenzo Fornaro committed 4 months ago
Merge pull request #437 from 4ny3l/fix-tool-calling-401
3ddbc65 Vincenzo Fornaro committed 4 months ago
fix(resource_plan): take last two lscpu fields, not [1]/[2] (#332 review)
171aa69 woolcoxm committed 4 months ago
resource_plan: stop setting OMP_PROC_BIND/OMP_PLACES from --auto-tier (#325)
bd3efb1 woolcoxm committed 4 months ago
resource_plan: fix --auto-tier pinning decode to one core (#325)
f27a89e woolcoxm committed 4 months ago
fix(tests): skip efficiency tests when glm_tiny fixture is absent (CI #360)
cf126cb woolcoxm committed 4 months ago
tests: efficiency suite β tiny-model regression gates + full-model optimization dossier
d43b545 woolcoxm committed 4 months ago
tools: fmt=5 index codec β deployable bytes for the E8/IQ3 container (#452 step 2)
e483461 ZacharyZcR committed 4 months ago
convert: mirror the --indir resume params guard onto --repo download loops
42a5417 monotophic committed 4 months ago
tools: IQ3_XXS-codebook scheme in the quant ablation (#452 step 1)
ae4e31a ZacharyZcR committed 4 months ago
numa: per-layer pin arenas β fix the PIN_GB=all VMA explosion (#419)
536d8bf ZacharyZcR committed 4 months ago
cuda: grouped-int4 (fmt=4) support in the expert-group kernels (#334)
a74e3e0 ZacharyZcR committed 4 months ago
Merge pull request #449 from cdhdt/fix/rss-guard-uaf
e9b3614 Vincenzo Fornaro committed 4 months ago
rss_guard: free the evicted expert slab under g_pilot_mx (use-after-free)
9c5ab39 cdhdt committed 4 months ago
Merge pull request #445 from ZacharyZcR/fix/pin-budget-release-host
4b70482 Vincenzo Fornaro committed 4 months ago
metal: parallel top-8 expert selection (r_top8_par), default ON
9f30916 monotophic committed 4 months ago
pin: with CUDA_RELEASE_HOST the VRAM prefix must not consume the RAM pin budget
453d140 ZacharyZcR committed 4 months ago
idot: independent accumulators on the AVX-VNNI int8/int4 dot kernels
c4b2cf6 cdhdt committed 4 months ago
bench: microbench for AVX-VNNI idot accumulator A/B (not a gate)
9c698b2 cdhdt committed 4 months ago
Merge pull request #393 from ZacharyZcR/feat/auto-tune
17fbdfa Vincenzo Fornaro committed 4 months ago
Merge pull request #438 from monotophic/e7/disk-class-instr
a7ef058 Vincenzo Fornaro committed 4 months ago
Merge pull request #422 from nbeerbower/fix-mtp-probe-expert-count
fbabaa4 Vincenzo Fornaro committed 4 months ago
PROF: DISK-CLASS β per-load cold/warm disk instrumentation (re-derived onto #391 split)
1f00142 monotophic committed 4 months ago
fix(engine): filter non-EOS stop tokens in serve mode to prevent premature halt on tool calls
26bd8b4 Colibri Developer committed 4 months ago
fix(api): pass tools and tool_choice as parameters to generation() to prevent duplicate extraction
70fb5b0 Colibri Developer committed 4 months ago