Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40)
nalepy89d95fc73bc7
diff --git a/README.md b/README.mdindex 327ef940..4af26674 100644--- a/README.md+++ b/README.md@@ -112,6 +112,44 @@ variables keep precedence over automatic values.The engine at runtime is pure C — python is only used by the one-time converter.+### Windows 11 (native, no WSL)++colibrì builds and runs natively on Windows 11 x86-64 with MinGW-w64. The port adds+a `_WIN32` compatibility layer in `c/compat.h` that maps POSIX I/O to the Windows API+(pread → ReadFile+OVERLAPPED, posix_fadvise no-op, aligned allocation, MoveFileEx rename,+GlobalMemoryStatusEx RAM detection). All platform differences stay in `compat.h`; the+engine source is unchanged.++**Toolchain:** GCC via [winlibs](https://winlibs.com/) or MSYS2 MinGW-w64. Tested with+GCC 16.1.0 (x86_64-ucrt-posix-seh).++```powershell+# One-time toolchain install (pick one):+scoop install mingw-winlibs # portable, no shell needed+# or: pacman -S mingw-w64-x86_64-gcc make # via MSYS2++# Build (from c/ directory):+make glm.exe # GLM-5.2 engine (static, no DLL dependencies)+make olmoe.exe # OLMoE engine (same shims)+make iobench.exe # disk I/O benchmark+make test-c # run C tests+make test-python # run Python tests (requires python)++# Verify (tiny model, 2.4 MB):+pip install torch transformers safetensors huggingface_hub+python tools/make_glm_oracle.py # generate tiny oracle+SNAP=./glm_tiny TF=1 ./glm.exe 64 16 16 # expect "32/32 posizioni"++# Run with real model:+SNAP=D:\glm52_i4 ./glm.exe 64 4 16 # batch inference+python coli chat --model D:\glm52_i4 # interactive chat+python coli serve --model D:\glm52_i4 # OpenAI-compatible API+```++**Status:** Phase 1 complete (compiles, correct, static-linked). O_DIRECT (Phase 2),+GPU via `LoadLibrary` on `coli_cuda.dll` (Phases G0–G2), and full-model validation+are separate workstreams. See `PORT_WINDOWS_PLAN.md` for the full plan.+### OpenAI-compatible API`coli serve` keeps one model process loaded and exposes a text-only OpenAI-compatible@@ -277,7 +315,7 @@ thrashing. Persistent `.coli_usage` remains the long-term signal and is not deca## Got a better machine? Try it — here's what to expect-colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, NVMe behind a WSL2 VHDX that caps random reads at ~1 GB/s). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4 — never a network/9p mount).+colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, NVMe behind a WSL2 VHDX that caps random reads at ~1 GB/s). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), macOS, or **Windows 11 natively (MinGW-w64)**; gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4/NTFS — never a network/9p mount).**How to test it, in order:**diff --git a/c/.gitignore b/c/.gitignorenew file mode 100644index 00000000..782930be--- /dev/null+++ b/c/.gitignore@@ -0,0 +1,4 @@+*.exe+glm_tiny/+olmoe_hf/+olmoe_i4/diff --git a/c/Makefile b/c/Makefileindex 6fe3ab47..94120e8b 100644--- a/c/Makefile+++ b/c/Makefile@@ -1,4 +1,7 @@UNAME_S := $(shell uname -s)+MINGW := $(findstring MINGW,$(UNAME_S))+MSYS := $(findstring MSYS,$(UNAME_S))+IS_WIN := $(MINGW)$(MSYS)ifeq ($(UNAME_S),Darwin)# --- macOS / Apple Silicon ---@@ -17,6 +20,18 @@ OMPL =endifCFLAGS = -O3 $(OMPC) -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-functionLDFLAGS = -lm $(OMPL)+EXE =+else ifneq ($(IS_WIN),)+# --- Windows 11 x86-64 (MinGW-w64 / MSYS2) ---+# GCC + libgomp + winpthreads: pthread, OpenMP, clock_gettime, opendir/readdir,+# AVX2 intrinsics — tutto gratis, nessun porting.+# ARCH default = x86-64-v3 (binario portabile con AVX2). Per max velocita'+# su QUESTA macchina: make ARCH=native+CC = gcc+ARCH ?= x86-64-v3+CFLAGS = -D_FILE_OFFSET_BITS=64 -O3 -march=$(ARCH) -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function+LDFLAGS = -lm -fopenmp -static+EXE = .exeelse# --- Linux x86-64 (percorso originale, invariato) ---CC = gcc@@ -26,6 +41,7 @@ CC = gccARCH ?= nativeCFLAGS = -O3 -march=$(ARCH) -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-functionLDFLAGS = -lm -fopenmp+EXE =endif# CUDA=1 adds an opt-in backend for resident tensors. The default build remains@@ -37,20 +53,27 @@ CUDA_ARCH ?= nativeNVCCFLAGS ?= -O3 -std=c++17 -arch=$(CUDA_ARCH) -Xcompiler=-Wall,-WextraPYTHON ?= python3CUDA_OBJ =-TEST_BINS = tests/test_json tests/test_st tests/test_tier+TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE)ifeq ($(CUDA),1)ifeq ($(UNAME_S),Darwin)$(error CUDA=1 is supported only on Linux)endif+ifneq ($(IS_WIN),)+# GPU: stub only in Phase 1 (G0). G1 builds coli_cuda.dll with MSVC+nvcc.+$(error CUDA=1 on Windows requires G1: build coli_cuda.dll with MSVC+nvcc (see PORT_WINDOWS_PLAN.md §8))+endifCFLAGS += -DCOLI_CUDALDFLAGS += -L$(CUDA_HOME)/lib64 -Wl,-rpath,$(CUDA_HOME)/lib64 -lcudart -lstdc++CUDA_OBJ = backend_cuda.oendif-all: glm+all: glm$(EXE)-glm: glm.c st.h json.h tok.h tok_unicode.h compat.h $(CUDA_OBJ)- $(CC) $(CFLAGS) glm.c $(CUDA_OBJ) -o glm $(LDFLAGS)+# phony 'glm' → 'glm.exe' on Windows (so 'make glm' and 'coli build' work on every platform)+glm: glm$(EXE)++glm$(EXE): glm.c st.h json.h tok.h tok_unicode.h compat.h $(CUDA_OBJ)+ $(CC) $(CFLAGS) glm.c $(CUDA_OBJ) -o glm$(EXE) $(LDFLAGS)backend_cuda.o: backend_cuda.cu backend_cuda.h@command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }@@ -58,23 +81,26 @@ backend_cuda.o: backend_cuda.cu backend_cuda.hcuda-test: backend_cuda.cu backend_cuda.h tests/test_backend_cuda.cu@command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }- $(NVCC) $(NVCCFLAGS) backend_cuda.cu tests/test_backend_cuda.cu -o backend_cuda_test- ./backend_cuda_test+ $(NVCC) $(NVCCFLAGS) backend_cuda.cu tests/test_backend_cuda.cu -o backend_cuda_test$(EXE)+ ./backend_cuda_test$(EXE)-olmoe: olmoe.c st.h json.h- $(CC) $(CFLAGS) olmoe.c -o olmoe $(LDFLAGS)+olmoe$(EXE): olmoe.c st.h json.h compat.h+ $(CC) $(CFLAGS) olmoe.c -o olmoe$(EXE) $(LDFLAGS)# binario portabile da distribuire su altre macchine x86-64portable:- $(MAKE) glm ARCH=x86-64-v3+ $(MAKE) glm$(EXE) ARCH=x86-64-v3++iobench$(EXE): iobench.c compat.h+ $(CC) $(CFLAGS) iobench.c -o iobench$(EXE) $(LDFLAGS)-tests/test_json: tests/test_json.c json.h+tests/test_json$(EXE): tests/test_json.c json.h$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)-tests/test_st: tests/test_st.c st.h json.h compat.h+tests/test_st$(EXE): tests/test_st.c st.h json.h compat.h$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)-tests/test_tier: tests/test_tier.c tier.h+tests/test_tier$(EXE): tests/test_tier.c tier.h$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)test-c: $(TEST_BINS)@@ -92,7 +118,7 @@ check:$(MAKE) testclean:- rm -f olmoe glm backend_cuda.o backend_cuda_test $(TEST_BINS)+ rm -f olmoe$(EXE) glm$(EXE) iobench$(EXE) backend_cuda.o backend_cuda_test$(EXE) $(TEST_BINS)rm -rf tests/__pycache__-.PHONY: all cuda-test portable test-c test-python test check clean+.PHONY: all glm cuda-test portable test-c test-python test check cleandiff --git a/c/coli b/c/coliindex d9bff3d7..32a80dcf 100755--- a/c/coli+++ b/c/coli@@ -21,9 +21,15 @@ Config via env o flag (validi anche dopo il sottocomando):"""import os, sys, subprocess, argparse, json, time, signal, shutil, threading, re, codecs, tempfile, textwrap+# Windows: forza output UTF-8 (console cp1252 tronca Unicode box-drawing/emoji)+if sys.platform == "win32":+ for s in (sys.stdout, sys.stderr):+ try: s.reconfigure(encoding="utf-8")+ except (AttributeError, OSError): pass+HERE = os.path.dirname(os.path.abspath(__file__))TOOLS = os.path.join(HERE, "tools")-GLM = os.path.join(HERE, "glm")+GLM = os.path.join(HERE, "glm" + (".exe" if sys.platform == "win32" else ""))DEF_MODEL = os.environ.get("COLI_MODEL", "/home/vincenzo/glm52_i4")END = b"\x01\x01END\x01\x01\n"READY = b"\x01\x01READY\x01\x01\n"@@ -299,8 +305,11 @@ def cmd_info(a):av=int(re.search(r'MemAvailable:\s+(\d+)',mi).group(1))/1e6row("RAM", f"{tot:.0f} GB totali · {av:.1f} GB disponibili")except Exception: pass- fs=os.statvfs(a.model if os.path.isdir(a.model) else HERE)- row("disco", f"{fs.f_bavail*fs.f_frsize/1e9:.0f} GB liberi")+ try:+ fs = shutil.disk_usage(a.model if os.path.isdir(a.model) else HERE)+ row("disco", f"{fs.free/1e9:.0f} GB liberi")+ except OSError:+ row("disco", "? GB (non disponibile)")row("motore", "pronto ✓" if os.path.exists(GLM) else "da compilare (coli build)")knobs=[]if a.ram: knobs.append(f"ram {a.ram}GB")@@ -447,7 +456,7 @@ def cmd_bench(a):need_model(a.model)banner("bench")# python con `tokenizers`: l'ambiente del progetto se c'e', altrimenti quello corrente- venv_py=os.path.join(HERE,"mio_env","bin","python3")+ venv_py = os.path.join(HERE, "mio_env", "Scripts" if sys.platform == "win32" else "bin", "python3")py = venv_py if os.path.exists(venv_py) else sys.executabletasks = ",".join(a.tasks) if a.tasks else "hellaswag,arc_challenge,mmlu"# dataset mancanti -> li scarica una volta (fetch_benchmarks.py li mette in --data come JSONL)@@ -466,7 +475,7 @@ def cmd_bench(a):def cmd_convert(a):banner("convert")# python con torch/safetensors: l'ambiente del progetto se c'e', altrimenti quello corrente- venv_py=os.path.join(HERE,"mio_env","bin","python3")+ venv_py = os.path.join(HERE, "mio_env", "Scripts" if sys.platform == "win32" else "bin", "python3")py = venv_py if os.path.exists(venv_py) else sys.executablebase=[py, os.path.join(TOOLS,"convert_fp8_to_int4.py"),"--repo", a.repo, "--outdir", a.model, "--ebits", str(a.ebits), "--io-bits", str(a.io_bits)]diff --git a/c/compat.h b/c/compat.hindex 6663d390..bb0b8990 100644--- a/c/compat.h+++ b/c/compat.h@@ -1,4 +1,5 @@-/* compat.h — shim di portabilita' per piattaforme non-Linux (oggi: macOS / Apple Silicon).+/* compat.h — shim di portabilita' per piattaforme non-Linux (oggi: macOS / Apple Silicon,+ * Windows 11 x86-64 via MinGW-w64).* Su Linux questo header e' un NO-OP totale: nessun simbolo definito o ridefinito,* zero impatto sul percorso x86 esistente.* Regola: ogni differenza di piattaforma vive QUI; i .c restano puliti. */@@ -47,4 +48,191 @@ static inline int compat_open_direct(const char *path){}#endif /* __APPLE__ */+/* ===================================================================+ * Windows 11 x86-64 (MinGW-w64 / MSYS2)+ * ===================================================================+ * pread -> compat_pread (ReadFile + OVERLAPPED su raw handle:+ * thread-safe, 64-bit offset, no CRT+ * text-mode translation — NEVER use+ * _read/_lseeki64 which are racy AND+ * corrupt 0x0A bytes in binary files).+ * posix_fadvise -> no-op (advisory only; macOS already no-ops DONTNEED).+ * posix_memalign->_aligned_malloc(free must be compat_aligned_free).+ * rename -> compat_rename (MoveFileEx MOVEFILE_REPLACE_EXISTING;+ * CRT rename fails EEXIST if dest exists,+ * breaking stats atomic-write every turn).+ * meminfo -> compat_meminfo (GlobalMemoryStatusEx: ullTotalPhys,+ * ullAvailPhys — approx MemAvailable).+ * getpid -> _getpid+ * =================================================================== */+#ifdef _WIN32++/* Belt-and-braces: 64-bit off_t mandatory — model is 370 GB, every pread+ * region can exceed 2 GB. 32-bit off_t silently wraps >4 GB offsets into the+ * first 4 GB → reads wrong weight bytes → silent token corruption. */+#if !defined(_FILE_OFFSET_BITS) || _FILE_OFFSET_BITS < 64+#error "_FILE_OFFSET_BITS=64 required on Windows (add -D_FILE_OFFSET_BITS=64 to CFLAGS)"+#endif++#ifndef WIN32_LEAN_AND_MEAN+#define WIN32_LEAN_AND_MEAN+#endif+#include <windows.h>+#include <io.h>+#include <process.h>+#include <malloc.h>+#include <fcntl.h>+#include <errno.h>++/* --- O_BINARY: belt-and-braces vs CRT text-mode (0x0A byte corruption) --- */+#ifndef O_BINARY+#define O_BINARY 0x8000+#endif+/* All open() calls for model data must use binary mode. The compat_pread+ * wrapper already bypasses CRT via ReadFile on the raw OS handle, so this+ * is defense-in-depth: if anyone adds a future CRT-based read path, O_BINARY+ * prevents 0x0A bytes from being silently translated to \r\n. */+#define COMPAT_O_RDONLY (O_RDONLY | O_BINARY)++/* --- posix_fadvise: no-op (advisory only; safe to ignore) --- */+#ifndef POSIX_FADV_NORMAL+#define POSIX_FADV_NORMAL 0+#define POSIX_FADV_RANDOM 1+#define POSIX_FADV_SEQUENTIAL 2+#define POSIX_FADV_WILLNEED 3+#define POSIX_FADV_DONTNEED 4+#define POSIX_FADV_NOREUSE 5+#endif+#define posix_fadvise(fd,off,len,advice) do{(void)(fd);(void)(off);(void)(len);(void)(advice);}while(0)++/* --- pread -> ReadFile + OVERLAPPED su raw OS handle ---+ * Thread-safe (no shared seek position). Gestisce offset >4 GB e chunking+ * per letture >2 GB (anche se i tensori individuali sono nell'ordine dei+ * MB-centinaia di MB, il wrapper e' robusto per ogni taglia). */+static inline ssize_t compat_pread(int fd, void *buf, size_t n, off_t off){+ intptr_t osfh = _get_osfhandle(fd);+ if(osfh == -1 || osfh == -2){ errno = EBADF; return -1; }+ HANDLE h = (HANDLE)osfh;+ size_t total = 0;+ while(total < n){+ size_t chunk = n - total;+ DWORD chunk32 = (chunk > 0x7FFFFFFF) ? 0x7FFFFFFF : (DWORD)chunk;+ OVERLAPPED ov = {0};+ ov.Offset = (DWORD)( (off + (off_t)total) & 0xFFFFFFFFULL);+ ov.OffsetHigh = (DWORD)(((off + (off_t)total) >> 32) & 0xFFFFFFFFULL);+ DWORD rd = 0;+ if(!ReadFile(h, (char*)buf + total, chunk32, &rd, &ov)){+ DWORD err = GetLastError();+ if(err == ERROR_HANDLE_EOF) break; /* past EOF → return bytes read (0 if none, matching POSIX pread) */+ if(err == ERROR_INVALID_HANDLE || err == ERROR_INVALID_FUNCTION) errno = EBADF;+ else errno = EIO;+ return -1;+ }+ total += rd;+ if(rd == 0 || rd < chunk32) break; /* EOF or partial (file truncated) */+ }+ return (ssize_t)total;+}+#define pread(fd,buf,n,off) compat_pread(fd,buf,n,off)++/* --- posix_memalign -> _aligned_malloc ---+ * ATTN: memoria allocata con _aligned_malloc DEVE essere liberata con+ * _aligned_free, NON con free(). Vedi compat_aligned_free sotto.+ * Audit: l'unico sito che libera memoria aligned e' free(s->slab) in+ * glm.c:892 (cambiato in compat_aligned_free). s->fslab usa falloc()+ * (malloc semplice) -> il suo free() resta plain. */+#ifndef ENOMEM+#define ENOMEM 12+#endif+static inline int compat_posix_memalign(void **memptr, size_t alignment, size_t size){+ if(alignment < sizeof(void*)) alignment = sizeof(void*);+ *memptr = _aligned_malloc(size, alignment);+ return *memptr ? 0 : ENOMEM;+}+#define posix_memalign(memptr,alignment,size) compat_posix_memalign(memptr,alignment,size)++/* matching free per memoria aligned di _aligned_malloc */+#define compat_aligned_free _aligned_free++/* --- meminfo: GlobalMemoryStatusEx ---+ * ullAvailPhys ~ MemAvailable di Linux (include standby/free/zero pages —+ * pagine recuperabili senza swap). Guida il cap automatico della cache+ * expert: se sbagliato, la cache e' mis-sized → swap thrash o OOM. */+static inline void compat_meminfo(double *total_gb, double *avail_gb){+ MEMORYSTATUSEX msx = {0};+ msx.dwLength = sizeof(msx);+ if(GlobalMemoryStatusEx(&msx)){+ *total_gb = (double)msx.ullTotalPhys / 1e9;+ *avail_gb = (double)msx.ullAvailPhys / 1e9;+ } else {+ *total_gb = 0; *avail_gb = 0;+ }+}++/* --- rename -> MoveFileEx (CRT rename EEXIST se destinazione esiste) ---+ * stats_dump_q chiama rename(tmp, path) OGNI turno di serve: dopo il primo+ * write il file esiste gia', e CRT rename fallisce silenziosamente,+ * affamando la pipeline REPIN/heat/PIN del suo segnale persistente. */+static inline int compat_rename(const char *old, const char *new){+ return MoveFileExA(old, new, MOVEFILE_REPLACE_EXISTING) ? 0 : -1;+}+#define rename(old,new) compat_rename(old,new)++/* --- getpid -> _getpid --- */+#define getpid() _getpid()++/* --- rss_gb: getrusage -> GetProcessMemoryInfo ---+ * ru_maxrss in KB (come Linux): rss_gb() divide per 1e6 → GB corretti. */+#include <psapi.h>+#pragma comment(lib, "psapi.lib")+struct rusage { long ru_maxrss; };+#define RUSAGE_SELF 0+static inline int getrusage(int who, struct rusage *r){+ (void)who;+ PROCESS_MEMORY_COUNTERS_EX pmc = {0};+ pmc.cb = sizeof(pmc);+ if(GetProcessMemoryInfo(GetCurrentProcess(), (PROCESS_MEMORY_COUNTERS*)&pmc, sizeof(pmc))){+ r->ru_maxrss = (long)(pmc.PeakWorkingSetSize / 1024); /* ru_maxrss = peak, not current */+ return 0;+ }+ r->ru_maxrss = 0; return -1;+}++/* --- getline -> compat_getline (fgets + realloc) --- */+#include <sys/types.h> /* ssize_t */+static inline ssize_t compat_getline(char **lineptr, size_t *n, FILE *stream){+ if(!lineptr || !n || !stream){ errno = EINVAL; return -1; }+ if(!*lineptr || !*n){ *n = 128; free(*lineptr); *lineptr = malloc(*n); if(!*lineptr) return -1; }+ size_t pos = 0; int c;+ while((c = fgetc(stream)) != EOF){+ if(pos + 1 >= *n){ size_t nn = *n * 2; char *np = realloc(*lineptr, nn); if(!np) return -1; *lineptr = np; *n = nn; }+ (*lineptr)[pos++] = (char)c;+ if(c == '\n') break;+ }+ if(pos == 0) return -1;+ (*lineptr)[pos] = '\0';+ return (ssize_t)pos;+}+#define getline(lineptr,n,stream) compat_getline(lineptr,n,stream)++/* --- setenv -> SetEnvironmentVariableA (POSIX setenv assente su Windows) --- */+static inline int compat_setenv(const char *name, const char *value, int overwrite){+ if(!overwrite && getenv(name)) return 0;+ return SetEnvironmentVariableA(name, value) ? 0 : -1;+}+#define setenv(name,value,overwrite) compat_setenv(name,value,overwrite)++#endif /* _WIN32 */++/* --- compat_aligned_free su piattaforme diverse da Windows ---+ * Su Linux/macOS, posix_memalign usa free() normale. */+#ifndef compat_aligned_free+#define compat_aligned_free free+#endif++/* --- COMPAT_O_RDONLY: O_RDONLY con O_BINARY su Windows, O_RDONLY puro altrove --- */+#ifndef COMPAT_O_RDONLY+#define COMPAT_O_RDONLY O_RDONLY+#endif+#endif /* COMPAT_H */diff --git a/c/glm.c b/c/glm.cindex 4b5e9dcd..3a296107 100644--- a/c/glm.c+++ b/c/glm.c@@ -25,8 +25,8 @@#include <limits.h>#include <pthread.h> /* thread I/O del PILOTA */#include <unistd.h>-#include <sys/resource.h>#if defined(__APPLE__) || defined(__linux__)+#include <sys/resource.h>#include <sys/mman.h> /* mlock: inchioda le pagine in RAM / wire pages into RAM */#endif#include "st.h"@@ -905,7 +905,7 @@ static void expert_load(Model *m, int layer, int eid, ESlot *s){/* rialloca se lo slot (riusato tra layer) e' troppo piccolo per QUESTO expert:* pread oltre la mappatura = short-read o CORRUZIONE silenziosa dei vicini */if(!s->slab || wtot+8192 > s->slab_cap){- free(s->slab);+ compat_aligned_free(s->slab);if(posix_memalign((void**)&s->slab,4096,wtot+8192)){fprintf(stderr,"OOM slab\n");exit(1);}s->slab_cap=wtot+8192;}@@ -2309,6 +2309,10 @@ static double mem_available_gb(void){if(host_statistics64(mach_host_self(),HOST_VM_INFO64,(host_info64_t)&vm,&cnt)!=KERN_SUCCESS) return 0;return ((double)vm.free_count+(double)vm.inactive_count+(double)vm.purgeable_count)* (double)sysconf(_SC_PAGESIZE) / 1e9;+#elif defined(_WIN32)+ double total, avail;+ compat_meminfo(&total, &avail);+ return avail;#elseFILE *f=fopen("/proc/meminfo","r"); if(!f) return 0;char ln[256]; double kb=0;@@ -2527,9 +2531,15 @@ int main(int argc, char **argv){int *tf=read_arr(ref,"tf_pred",&(int){0});int *pred=malloc(nfull*sizeof(int)); double tt=now_s();forward_all(&m, full, nfull, pred); double tdt=now_s()-tt;- int ok=0; for(int i=0;i<nfull;i++) ok+=(pred[i]==tf[i]);+ int ok=0; for(int i=0;i<nfull;i++){+ if(pred[i]==tf[i]) ok++;+ else fprintf(stderr,"[ORACLE] mismatch pos=%d expected=%d got=%d\n",i,tf[i],pred[i]);+ }printf("PREFILL (teacher-forcing) C vs oracolo: %d/%d posizioni | %.1f pos/s\n",ok,nfull,nfull/tdt);+ if(ok<nfull) fprintf(stderr,+ "[ORACLE] %d/%d mismatches — run: TF=1 DEBUG_LOGITS=1 for top-5 logit dump\n",+ nfull-ok,nfull);profile_print(&m,tdt);#ifdef COLI_CUDAif(g_cuda_enabled) cuda_stats_print();diff --git a/c/iobench.c b/c/iobench.cindex cfacb879..39d16a0f 100644--- a/c/iobench.c+++ b/c/iobench.c@@ -12,6 +12,7 @@#include <time.h>#include <errno.h>#include <string.h>+#include "compat.h"#ifdef _OPENMP#include <omp.h>#endifdiff --git a/c/olmoe.c b/c/olmoe.cindex 3d2a07d4..84b9f0a3 100644--- a/c/olmoe.c+++ b/c/olmoe.c@@ -12,7 +12,9 @@#include <string.h>#include <math.h>#include <time.h>+#if defined(__APPLE__) || defined(__linux__)#include <sys/resource.h>+#endif#include "st.h"/* ---------- config ---------- */diff --git a/c/resource_plan.py b/c/resource_plan.pyindex 7949379b..22ac79ec 100644--- a/c/resource_plan.py+++ b/c/resource_plan.py@@ -4,6 +4,7 @@import jsonimport osimport re+import shutilimport statisticsimport subprocessfrom pathlib import Path@@ -110,8 +111,11 @@ def build_plan(model, ram_gb=0, context=4096, gpu_indices=None, vram_gb=0,cfg = info["config"]available_memory = memory_available() if available_memory is None else available_memoryif available_disk is None:- fs = os.statvfs(info["path"])- available_disk = fs.f_bavail * fs.f_frsize+ try:+ usage = shutil.disk_usage(info["path"])+ available_disk = usage.free+ except OSError:+ available_disk = 500 * GBgpus = discover_gpus() if gpus is None else gpusif gpu_indices is not None:wanted = set(gpu_indices)diff --git a/c/setup.sh b/c/setup.shindex adeec0d7..12fc5003 100755--- a/c/setup.sh+++ b/c/setup.sh@@ -1,24 +1,34 @@#!/usr/bin/env bash-# colibrì — installazione su una macchina nuova (Linux x86-64).+# colibrì — installazione su una macchina nuova (Linux x86-64, macOS, Windows/MinGW).# Compila il motore e fa un self-test. Il MODELLO (~372 GB int4) va copiato a parte# o rigenerato con: coli convert --model <dir-su-ext4/NVMe>set -ecd "$(dirname "$0")"echo "🐦 colibrì — setup"+UNAME_S=$(uname -s)+# 1) dipendenzecommand -v make >/dev/null || { echo "manca make"; exit 1; }-if [ "$(uname -s)" = "Darwin" ]; then+case "$UNAME_S" in+Darwin)command -v clang >/dev/null || { echo "manca clang (xcode-select --install)"; exit 1; }echo " clang: $(clang --version | head -1) · $(sysctl -n hw.ncpu) core"echo -n " OpenMP: "if brew --prefix libomp >/dev/null 2>&1; then echo "ok (libomp)"else echo "libomp assente -> build single-thread (consigliato: brew install libomp)"; fi-else+ ;;+MINGW*|MSYS*)+ command -v gcc >/dev/null || { echo "manca gcc (MinGW-w64). Installa: pacman -S mingw-w64-x86_64-gcc make"; exit 1; }+ echo " gcc: $(gcc -dumpversion) · MinGW-w64"+ echo -n " OpenMP: "; echo 'int main(){return 0;}' | gcc -fopenmp -xc - -o /tmp/_omp 2>/dev/null && echo ok || { echo "manca libgomp (pacman -S mingw-w64-x86_64-gcc)"; exit 1; }+ ;;+*)command -v gcc >/dev/null || { echo "manca gcc (es: sudo apt install build-essential)"; exit 1; }echo " gcc: $(gcc -dumpversion) · $(nproc) core"echo -n " OpenMP: "; echo 'int main(){return 0;}' | gcc -fopenmp -xc - -o /tmp/_omp 2>/dev/null && echo ok || { echo "manca (libgomp)"; exit 1; }-fi+ ;;+esac# 2) build: nativa (veloce, per QUESTA macchina). Per un binario da distribuire: make portableecho " compilo (ARCH=${ARCH:-native})…"@@ -31,11 +41,18 @@ if [ -d glm_tiny ] && [ -f ref_glm.json ]; thenfi# 4) info macchina (la velocità dipende da QUESTI due numeri, non dalla GPU)-if [ "$(uname -s)" = "Darwin" ]; then+case "$UNAME_S" in+Darwin)ram=$(( $(sysctl -n hw.memsize) / 1000000000 ))-else+ ;;+MINGW*|MSYS*)+ # MSYS2 fornisce /proc/meminfo come symlink (più affidabile di wmic, deprecato)ram=$(awk '/MemTotal/{printf "%.0f", $2/1e6}' /proc/meminfo 2>/dev/null || echo "?")-fi+ ;;+*)+ ram=$(awk '/MemTotal/{printf "%.0f", $2/1e6}' /proc/meminfo 2>/dev/null || echo "?")+ ;;+esacecho " RAM: ${ram} GB (più RAM = più expert in cache = più veloce)"echoecho "pronto. Prossimi passi:"diff --git a/c/st.h b/c/st.hindex 61b84f17..69ed9f12 100644--- a/c/st.h+++ b/c/st.h@@ -76,11 +76,11 @@ static inline float f16_to_f32(uint16_t h) {static int st_open_fd(shards *S, const char *path) {for (int i = 0; i < S->nfd; i++) if (!strcmp(S->paths[i], path)) return S->fds[i];- int fd = open(path, O_RDONLY);+ int fd = open(path, COMPAT_O_RDONLY);if (fd < 0) { perror(path); exit(1); }S->paths[S->nfd] = strdup(path); S->fds[S->nfd] = fd;#ifdef O_DIRECT- S->dfds[S->nfd] = open(path, O_RDONLY | O_DIRECT); /* eager: lookup poi thread-safe */+ S->dfds[S->nfd] = open(path, COMPAT_O_RDONLY | O_DIRECT); /* eager: lookup poi thread-safe */#elif defined(__APPLE__)S->dfds[S->nfd] = compat_open_direct(path); /* macOS: F_NOCACHE ~ O_DIRECT */#elsediff --git a/c/tests/audit_win_shims.c b/c/tests/audit_win_shims.cnew file mode 100644index 00000000..8bc3cbe2--- /dev/null+++ b/c/tests/audit_win_shims.c@@ -0,0 +1,44 @@+/* audit: verify compat_pread >4GB offset + compat_rename replace-existing (Windows) */+#include <stdio.h>+#include <stdlib.h>+#include <string.h>+#include <fcntl.h>+#include <sys/stat.h>+#include <unistd.h>+#include "../compat.h"+#include <winioctl.h>++int main(void){+ /* --- rename second-write test --- */+ FILE *f = fopen("aud_tmp1", "wb"); fputs("v1", f); fclose(f);+ if(rename("aud_tmp1", "aud_dst")){ printf("rename #1 FAIL\n"); return 1; }+ f = fopen("aud_tmp2", "wb"); fputs("v2", f); fclose(f);+ if(rename("aud_tmp2", "aud_dst")){ printf("rename #2 (dest exists) FAIL <- CRT EEXIST bug\n"); return 1; }+ char buf[8] = {0};+ f = fopen("aud_dst", "rb"); fread(buf, 1, 2, f); fclose(f);+ remove("aud_dst");+ if(strcmp(buf, "v2")){ printf("rename content FAIL (%s)\n", buf); return 1; }+ printf("rename replace-existing: ok\n");++ /* --- pread >4GB offset test (sparse file) --- */+ const char *p = "aud_big.bin";+ int fd = open(p, O_RDWR | O_CREAT | O_BINARY, 0644);+ if(fd < 0){ printf("open FAIL\n"); return 1; }+ /* mark sparse so 5GB costs ~nothing on NTFS */+ HANDLE h = (HANDLE)_get_osfhandle(fd);+ DWORD br; DeviceIoControl(h, FSCTL_SET_SPARSE, NULL, 0, NULL, 0, &br, NULL);+ off_t off = (off_t)5 * 1024 * 1024 * 1024; /* 5 GiB */+ const char magic[] = "COLIBRI64";+ OVERLAPPED ov = {0};+ ov.Offset = (DWORD)(off & 0xFFFFFFFFULL); ov.OffsetHigh = (DWORD)(off >> 32);+ if(!WriteFile(h, magic, sizeof(magic), &br, &ov)){ printf("write@5GB FAIL\n"); close(fd); remove(p); return 1; }+ char rd[sizeof(magic)] = {0};+ ssize_t n = pread(fd, rd, sizeof(magic), off);+ close(fd); remove(p);+ if(n != sizeof(magic) || memcmp(rd, magic, sizeof(magic))){+ printf("pread >4GB FAIL (n=%lld data=%s) <- OVERLAPPED 64-bit fill broken\n", (long long)n, rd);+ return 1;+ }+ printf("pread >4GB offset: ok\n");+ return 0;+}diff --git a/c/tools/convert_olmoe.py b/c/tools/convert_olmoe.pynew file mode 100644index 00000000..60d87444--- /dev/null+++ b/c/tools/convert_olmoe.py@@ -0,0 +1,115 @@+#!/usr/bin/env python3+"""Convert OLMoE HuggingFace checkpoint to colibri int4 format.++Downloads or converts a local OLMoE checkpoint (e.g., allenai/OLMoE-1B-7B-0125-Instruct).+Dense weights stay as-is (engine reads BF16/F16 → F32 on load).+Expert weights get row-wise int8 quantization with float32 scales.++Usage:+ python tools/convert_olmoe.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ./olmoe_i4+ python tools/convert_olmoe.py --model ./OLMoE-1B-7B-0125-Instruct --out ./olmoe_i4+"""++import argparse, json, math, os, struct, sys+from pathlib import Path++# Windows: force UTF-8 output+if sys.platform == "win32":+ for s in (sys.stdout, sys.stderr):+ try: s.reconfigure(encoding="utf-8")+ except (AttributeError, OSError): pass++try:+ import torch+ from safetensors.torch import load_file, save_file+except ImportError as exc:+ sys.exit(f"Missing dependencies: {exc}. Install: pip install torch safetensors")+++EXPERT_KEY_RE = r"model\.layers\.\d+\.mlp\.experts\.\d+\.(gate_proj|up_proj|down_proj)\.weight"+++def quantize_row(w: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:+ """Row-wise int8 quantization. Returns (int8_weights, float32_scales)."""+ w_f32 = w.float()+ row_max = w_f32.abs().amax(dim=1, keepdim=True).clamp(min=1e-12)+ scales = row_max / 127.0+ q = (w_f32 / scales).round().clamp(-128, 127).to(torch.int8)+ return q, scales.squeeze(1)+++def is_expert_weight(name: str) -> bool:+ import re+ return bool(re.search(EXPERT_KEY_RE, name))+++def main():+ ap = argparse.ArgumentParser(description="Convert OLMoE HF checkpoint -> colibri int4")+ src = ap.add_mutually_exclusive_group(required=True)+ src.add_argument("--repo", help="HuggingFace repo ID")+ src.add_argument("--model", help="Local HF checkpoint directory")+ ap.add_argument("--out", required=True, help="Output directory for int4 model")+ ap.add_argument("--ebits", type=int, default=4,+ help="Expert quant bits (4 or 8, default 4)")+ args = ap.parse_args()++ if args.repo:+ from huggingface_hub import snapshot_download+ print(f"Downloading {args.repo}...")+ src_dir = snapshot_download(args.repo, local_files_only=True, max_workers=4)+ if not any(Path(src_dir).glob("*.safetensors")):+ print("Downloading safetensors...")+ src_dir = snapshot_download(args.repo, max_workers=4)+ else:+ src_dir = args.model++ src = Path(src_dir)+ if not src.is_dir():+ sys.exit(f"Model directory not found: {src}")+ if not (src / "config.json").is_file():+ sys.exit(f"config.json missing in {src}")++ out = Path(args.out)+ out.mkdir(parents=True, exist_ok=True)++ # Copy config.json+ import shutil+ shutil.copy2(src / "config.json", out / "config.json")+ print(f"config.json -> {out}")++ # Process safetensors+ shards = sorted(src.glob("*.safetensors"))+ if not shards:+ sys.exit(f"No safetensors found in {src}")++ expert_count = 0+ total_expert_f32 = total_expert_q = 0++ for si, shard in enumerate(shards, 1):+ print(f"[{si}/{len(shards)}] {shard.name}...", end=" ", flush=True)+ tensors = load_file(str(shard))+ out_tensors = {}+ for name, tensor in tensors.items():+ if is_expert_weight(name):+ expert_count += 1+ q, scales = quantize_row(tensor)+ total_expert_f32 += tensor.numel() * tensor.element_size()+ total_expert_q += q.numel() * 1 + scales.numel() * 4+ out_tensors[name] = q+ out_tensors[name + ".qs"] = scales+ else:+ out_tensors[name] = tensor++ out_shard = out / shard.name+ save_file(out_tensors, str(out_shard))+ ratio = total_expert_q / max(total_expert_f32, 1) * 100+ print(f"ok")++ print(f"\nDone. {expert_count} expert tensors quantized.")+ print(f"Expert storage: {total_expert_f32/1e9:.1f} GB -> {total_expert_q/1e9:.1f} GB ({ratio:.0f}%)")+ print(f"Model ready at: {out}")+ print(f"\nRun: SNAP={out} ./olmoe.exe 32 4 16")+++if __name__ == "__main__":+ main()