Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)
woolcoxm2319b942d2f2
diff --git a/.gitignore b/.gitignoreindex b208456f..203a3228 100644--- a/.gitignore+++ b/.gitignore@@ -11,15 +11,26 @@ desktop/src-tauri/gen/# binari compilati (si rigenerano con make / coli build)c/glm+c/glm.exec/olmoe+c/olmoe.exec/iobench+c/iobench.exec/tok_testc/backend_cuda.oc/backend_cuda_test+c/backend_loader.o+c/coli_cuda.dll+c/coli_cuda.lib+c/coli_cuda.expc/tests/test_json+c/tests/test_json.exec/tests/test_st+c/tests/test_st.exec/tests/test_tier+c/tests/test_tier.exec/tests/test_grammar+c/tests/test_grammar.exe# oracoli tiny generati (make_glm_oracle.py) e dati benchmark scaricatic/glm_tiny/diff --git a/README.md b/README.mdindex 21900f19..58f6745a 100644--- a/README.md+++ b/README.md@@ -160,6 +160,12 @@ make iobench.exe # disk I/O benchmarkmake test-c # run C testsmake test-python # run Python tests (requires python)+# AVX-VNNI: Intel Alder Lake+ (and Meteor Lake+) CPUs have a 128-bit int8+# dot-product instruction (VPDPBUSD) the engine can use for ~1.3x faster+# quantized matmul. The x86-64-v3 default (portable AVX2) compiles it out;+# build for THIS machine to enable it:+make glm.exe ARCH=native # banner prints "idot: avx-vnni"+# Verify (tiny model, 2.4 MB):pip install torch transformers safetensors huggingface_hubpython tools/make_glm_oracle.py # generate tiny oracle@@ -171,9 +177,49 @@ python coli chat --model D:\glm52_i4 # interactive chatpython coli serve --model D:\glm52_i4 # OpenAI-compatible API```-**Status:** Phase 1 complete (compiles, correct, static-linked). O_DIRECT (Phase 2),-GPU via `LoadLibrary` on `coli_cuda.dll` (Phases G0–G2), and full-model validation-are separate workstreams. See `PORT_WINDOWS_PLAN.md` for the full plan.+**Warmup (overnight cache priming):** the engine's expert cache learns from+your workload. The included `warmup.ps1` script runs `coli run` in a loop with+diverse prompts to build the `.coli_usage` histogram unattended, so the next+real session starts with a large, accurate hot-expert pin. Each run saves usage+atomically on clean completion.++```powershell+.\warmup.ps1 -Rounds 1 -Ngen 32 # ~60-90 min, durable progress+```++**NVIDIA GPU (optional, via runtime DLL):** on Windows the engine is built with+MinGW gcc but CUDA kernels require MSVC + nvcc. The split is clean: build the+CUDA backend into a standalone `coli_cuda.dll` (nvcc + MSVC), then the host+`glm.exe` loads it at runtime via `LoadLibrary` (`c/backend_loader.c`). The host+never links cudart directly; if the DLL is absent the engine falls back to CPU+without error.++```powershell+# Prerequisites: CUDA Toolkit + MSVC Build Tools (cl.exe) + nvcc on PATH.+# Build the DLL from a shell with the MSVC environment set (vcvars64.bat or+# "x64 Native Tools Command Prompt for VS"):+make cuda-dll CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8" CUDA_ARCH=sm_120++# Build the host with the runtime loader (CUDA_DLL=1 adds -DCOLI_CUDA and+# links backend_loader.o instead of cudart):+make glm.exe CUDA_DLL=1 ARCH=native++# Run with the GPU expert tier (8 GB VRAM budget here; scale to your free VRAM):+$env:COLI_CUDA="1"; $env:COLI_GPU="0"; $env:CUDA_EXPERT_GB="8"+python coli chat --model D:\glm52_i4 --topp 0.7+```++The DLL exports 11 `extern "C"` symbols (`coli_cuda_init`, `coli_cuda_matmul`,+etc.); `backend_loader.c` resolves them via `GetProcAddress` on first use.+`ColiCudaTensor*` is opaque to the host (stored, never dereferenced), so the+MSVC-allocated struct is safe across the ABI boundary. `CUDA_ARCH` must match+your GPU's compute capability (e.g. `sm_120` for Blackwell / RTX 50-series,+`sm_89` for Ada / RTX 40-series).++**Status:** Phase 1 complete (compiles, correct, static-linked). The Windows+GPU tier (runtime `coli_cuda.dll` via `LoadLibrary`) is implemented and+verified on RTX 50-series (sm_120). O_DIRECT (Phase 2) and full-model+validation against the transformers oracle remain separate workstreams.### OpenAI-compatible APIdiff --git a/c/Makefile b/c/Makefileindex 58dbc7f8..9d63979d 100644--- a/c/Makefile+++ b/c/Makefile@@ -35,9 +35,12 @@ EXE =else ifneq ($(IS_WIN),)# --- Windows 11 x86-64 (MinGW-w64 / MSYS2) ---# GCC + libgomp + winpthreads: pthread, OpenMP, clock_gettime, opendir/readdir,-# AVX2 intrinsics — tutto gratis, nessun porting.-# ARCH default = x86-64-v3 (binario portabile con AVX2). Per max velocita'-# su QUESTA macchina: make ARCH=native+# AVX2 intrinsics - tutto gratis, nessun porting.+# ARCH default = x86-64-v3 (portable binary with AVX2). For max speed on THIS+# machine use ARCH=native: on AVX-VNNI CPUs (Intel Alder Lake+, Meteor Lake+)+# it also unlocks the 128-bit VPDPBUSD int8/int4 dot kernel (dot_i8i8/dot_i4i8),+# which the x86-64-v3 baseline does not define. The #ifdef guards in glm.c mean+# a v3 build simply compiles out the VNNI path - safe on any x86-64.CC = gccARCH ?= x86-64-v3CFLAGS = -D_FILE_OFFSET_BITS=64 -O3 -march=$(ARCH) -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function@@ -70,7 +73,17 @@ endif# CUDA=1 adds an opt-in backend for resident tensors. The default build remains# pure C and keeps the original zero-dependency runtime.+#+# Two paths:+# - Linux/macOS: CUDA=1 links backend_cuda.o directly (cudart via -l).+# - Windows: CUDA_DLL=1 builds a standalone coli_cuda.dll (nvcc+MSVC),+# then the host glm.exe loads it at runtime via backend_loader.c+# (LoadLibrary/GetProcAddress). MinGW gcc cannot compile .cu+# (nvcc needs cl.exe), and cross-linking MSVC objects into a+# gcc binary is fragile — the DLL split keeps the toolchains+# clean. See backend_loader.c and README "cuda-dll" below.CUDA ?= 0+CUDA_DLL ?= 0CUDA_HOME ?= /usr/local/cudaNVCC ?= $(CUDA_HOME)/bin/nvccCUDA_ARCH ?= native@@ -78,13 +91,23 @@ NVCCFLAGS ?= -O3 -std=c++17 -arch=$(CUDA_ARCH) -Xcompiler=-Wall,-WextraPYTHON ?= python3CUDA_OBJ =TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE) tests/test_grammar$(EXE) tests/test_decode_batch$(EXE) tests/test_idot$(EXE) tests/test_i4_acc512$(EXE)++# Windows CUDA DLL path: host links the loader, NOT cudart.+ifneq ($(IS_WIN),)+ifeq ($(CUDA_DLL),1)+CFLAGS += -DCOLI_CUDA+CUDA_OBJ = backend_loader.o+endif+endif++# Linux CUDA direct-link path (unchanged).ifeq ($(CUDA),1)ifeq ($(UNAME_S),Darwin)$(error CUDA=1 is supported only on Linux)endififneq ($(IS_WIN),)-# GPU: stub only in Phase 1 (G0). G1 builds coli_cuda.dll with MSVC+nvcc.-$(error CUDA=1 on Windows requires G1: build coli_cuda.dll with MSVC+nvcc (see PORT_WINDOWS_PLAN.md §8))+# On Windows use CUDA_DLL=1 (runtime DLL), not CUDA=1 (direct link).+$(error On Windows use: make CUDA_DLL=1 cuda-dll (see backend_loader.c))endifCFLAGS += -DCOLI_CUDALDFLAGS += -L$(CUDA_HOME)/lib64 -Wl,-rpath,$(CUDA_HOME)/lib64 -lcudart -lstdc++@@ -113,6 +136,22 @@ glm: glm$(EXE)glm$(EXE): glm.c st.h json.h tok.h tok_unicode.h compat.h grammar.h $(CUDA_OBJ) $(METAL_OBJ)$(CC) $(CFLAGS) glm.c $(CUDA_OBJ) $(METAL_OBJ) -o glm$(EXE) $(LDFLAGS)+# Windows runtime loader object: resolves coli_cuda_* from coli_cuda.dll.+backend_loader.o: backend_loader.c backend_cuda.h compat.h+ $(CC) $(CFLAGS) -c backend_loader.c -o $@++# Windows CUDA DLL: compile backend_cuda.cu with nvcc (+MSVC cl.exe as host+# compiler, required by nvcc on Windows) into coli_cuda.dll. Run this from a+# shell that has the MSVC environment set (e.g. after vcvars64.bat, or from a+# "x64 Native Tools Command Prompt"). COLI_CUDA_BUILDING_DLL enables+# __declspec(dllexport) so the 11 API symbols are exported.+cuda-dll: backend_cuda.cu backend_cuda.h+ @command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }+ @command -v cl >/dev/null 2>&1 || { echo "cl.exe (MSVC) not in PATH — run vcvars64.bat first" >&2; exit 1; }+ $(NVCC) $(NVCCFLAGS) -shared -DCOLI_CUDA_BUILDING_DLL \+ -L"$(CUDA_HOME)/lib/x64" -lcudart \+ backend_cuda.cu -o coli_cuda.dll+backend_cuda.o: backend_cuda.cu backend_cuda.h@command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }$(NVCC) $(NVCCFLAGS) -c backend_cuda.cu -o $@@@ -180,7 +219,7 @@ check:$(MAKE) testclean:- rm -f olmoe$(EXE) glm$(EXE) iobench$(EXE) backend_cuda.o backend_cuda_test$(EXE) backend_cuda_bench$(EXE) backend_metal.o backend_metal_test $(TEST_BINS)+ rm -f olmoe$(EXE) glm$(EXE) iobench$(EXE) backend_cuda.o backend_loader.o backend_cuda_test$(EXE) backend_cuda_bench$(EXE) backend_metal.o backend_metal_test coli_cuda.dll coli_cuda.lib coli_cuda.exp $(TEST_BINS)rm -rf tests/__pycache__-.PHONY: all glm cuda-test cuda-bench portable test-c test-python test check clean+.PHONY: all glm cuda-test cuda-bench cuda-dll portable test-c test-python test check cleandiff --git a/c/backend_cuda.h b/c/backend_cuda.hindex 2a1979c2..81125559 100644--- a/c/backend_cuda.h+++ b/c/backend_cuda.h@@ -4,6 +4,16 @@#include <stddef.h>#include <stdint.h>+/* COLI_CUDA_DLLEXPORT marks functions exported from coli_cuda.dll on Windows.+ * Define COLI_CUDA_BUILDING_DLL when compiling the .cu into the DLL (so the+ * functions are __declspec(dllexport)); the host loader does NOT include this+ * header's declarations — it resolves symbols at runtime via GetProcAddress. */+#if defined(_WIN32) && defined(COLI_CUDA_BUILDING_DLL)+#define COLI_CUDA_DLLEXPORT __declspec(dllexport)+#else+#define COLI_CUDA_DLLEXPORT+#endif+#ifdef __cplusplusextern "C" {#endif@@ -14,18 +24,18 @@ extern "C" {typedef struct ColiCudaTensor ColiCudaTensor;/* Devices are CUDA ordinals, not positions in the input list. */-int coli_cuda_init(const int *devices, int count);-void coli_cuda_shutdown(void);-int coli_cuda_device_count(void);-int coli_cuda_device_at(int index);-int coli_cuda_mem_info(int device, size_t *free_bytes, size_t *total_bytes);+COLI_CUDA_DLLEXPORT int coli_cuda_init(const int *devices, int count);+COLI_CUDA_DLLEXPORT void coli_cuda_shutdown(void);+COLI_CUDA_DLLEXPORT int coli_cuda_device_count(void);+COLI_CUDA_DLLEXPORT int coli_cuda_device_at(int index);+COLI_CUDA_DLLEXPORT int coli_cuda_mem_info(int device, size_t *free_bytes, size_t *total_bytes);/* device < 0 returns aggregate statistics for all configured devices. */-void coli_cuda_stats(int device, size_t *tensor_count, size_t *tensor_bytes);-void coli_cuda_group_stats(uint64_t *calls, uint64_t *experts, uint64_t *rows,+COLI_CUDA_DLLEXPORT void coli_cuda_stats(int device, size_t *tensor_count, size_t *tensor_bytes);+COLI_CUDA_DLLEXPORT void coli_cuda_group_stats(uint64_t *calls, uint64_t *experts, uint64_t *rows,double *h2d_ms, double *kernel_ms, double *d2h_ms);/* Upload without executing, so capacity failures happen during model startup. */-int coli_cuda_tensor_upload(ColiCudaTensor **tensor,+COLI_CUDA_DLLEXPORT int coli_cuda_tensor_upload(ColiCudaTensor **tensor,const void *weights, const float *scales,int fmt, int I, int O, int device);@@ -35,7 +45,7 @@ int coli_cuda_tensor_upload(ColiCudaTensor **tensor,* The first successful call uploads W and its row scales; later calls reuse it.* Returns 1 on success and 0 when CUDA is not initialized or the format is invalid.*/-int coli_cuda_matmul(ColiCudaTensor **tensor,+COLI_CUDA_DLLEXPORT int coli_cuda_matmul(ColiCudaTensor **tensor,float *y, const float *x,const void *weights, const float *scales,int fmt, int S, int I, int O, int device);@@ -43,25 +53,25 @@ int coli_cuda_matmul(ColiCudaTensor **tensor,/* Fused expert pipeline: y = down(silu(gate(x)) * up(x)). All three tensors* must already be resident on one device. Activations cross PCIe once in* each direction instead of once per matrix. */-int coli_cuda_expert_mlp(ColiCudaTensor *gate, ColiCudaTensor *up,+COLI_CUDA_DLLEXPORT int coli_cuda_expert_mlp(ColiCudaTensor *gate, ColiCudaTensor *up,ColiCudaTensor *down, float *y, const float *x, int S);/* Packed group of same-shaped experts. Inputs and outputs contain sum(rows)* consecutive [D] rows in call order. */-int coli_cuda_expert_group(ColiCudaTensor *const *gates,+COLI_CUDA_DLLEXPORT int coli_cuda_expert_group(ColiCudaTensor *const *gates,ColiCudaTensor *const *ups,ColiCudaTensor *const *downs,const int *rows, int count,float *y, const float *x);/* Decode-only MLA weight-absorption core for one token. kv_b is [H*(Q+V),K]. */-int coli_cuda_attention_absorb(ColiCudaTensor *kv_b,float *ctx,const float *q,+COLI_CUDA_DLLEXPORT int coli_cuda_attention_absorb(ColiCudaTensor *kv_b,float *ctx,const float *q,const float *latent,const float *rope,int H,int Q,int R,int V,int K,int T,float attention_scale);-void coli_cuda_tensor_free(ColiCudaTensor *tensor);-size_t coli_cuda_tensor_bytes(const ColiCudaTensor *tensor);-int coli_cuda_tensor_device(const ColiCudaTensor *tensor);+COLI_CUDA_DLLEXPORT void coli_cuda_tensor_free(ColiCudaTensor *tensor);+COLI_CUDA_DLLEXPORT size_t coli_cuda_tensor_bytes(const ColiCudaTensor *tensor);+COLI_CUDA_DLLEXPORT int coli_cuda_tensor_device(const ColiCudaTensor *tensor);#ifdef __cplusplus}diff --git a/c/backend_loader.c b/c/backend_loader.cnew file mode 100644index 00000000..6fea70c1--- /dev/null+++ b/c/backend_loader.c@@ -0,0 +1,219 @@+/* backend_loader.c — Windows runtime loader for coli_cuda.dll.+ *+ * Why this exists: the engine is built with MinGW-w64 (gcc), but CUDA kernels+ * must be compiled with MSVC + nvcc. We cannot link a CUDA .o into a gcc binary+ * reliably across the MSVC/MinGW ABI, and nvcc requires cl.exe as its host+ * compiler. The clean cross-toolchain split is: build the CUDA backend into a+ * standalone coli_cuda.dll with nvcc+MSVC, then load it here at runtime via+ * LoadLibrary/GetProcAddress. The host (glm.exe) never links cudart directly.+ *+ * On Linux this file is not compiled (the Makefile links backend_cuda.o+ * directly). On Windows, when COLI_CUDA is defined, glm.c calls the+ * coli_cuda_* wrappers below, which forward through function pointers resolved+ * from the DLL at first use. If the DLL is absent, every call safely returns+ * the "not initialized" sentinel (0 / no-op) and the engine falls back to CPU.+ *+ * ABI note: ColiCudaTensor* is opaque to the host (it stores the pointer,+ * never dereferences it), so the MSVC-allocated struct is safe to pass across+ * the boundary as an opaque handle. All scalar types (int, size_t, pointers)+ * agree between MSVC and MinGW-w64 on x86-64.+ */+#ifdef _WIN32++#include <stdio.h>+#include <stddef.h>+#include <windows.h>++#include "backend_cuda.h"++/* Function-pointer typedefs matching each exported symbol. */+typedef int (*fn_init)(const int *devices, int count);+typedef void (*fn_shutdown)(void);+typedef int (*fn_device_count)(void);+typedef int (*fn_device_at)(int index);+typedef int (*fn_mem_info)(int device, size_t *free_bytes, size_t *total_bytes);+typedef void (*fn_stats)(int device, size_t *tensor_count, size_t *tensor_bytes);+typedef void (*fn_group_stats)(uint64_t *calls, uint64_t *experts, uint64_t *rows,+ double *h2d_ms, double *kernel_ms, double *d2h_ms);+typedef int (*fn_expert_mlp)(ColiCudaTensor *gate, ColiCudaTensor *up,+ ColiCudaTensor *down, float *y, const float *x, int S);+typedef int (*fn_expert_group)(ColiCudaTensor *const *gates, ColiCudaTensor *const *ups,+ ColiCudaTensor *const *downs, const int *rows, int count,+ float *y, const float *x);+typedef int (*fn_attention_absorb)(ColiCudaTensor *kv_b, float *ctx, const float *q,+ const float *latent, const float *rope, int H, int Q,+ int R, int V, int K, int T, float attention_scale);+typedef int (*fn_tensor_upload)(ColiCudaTensor **tensor, const void *weights,+ const float *scales, int fmt, int I, int O, int device);+typedef int (*fn_matmul)(ColiCudaTensor **tensor, float *y, const float *x,+ const void *weights, const float *scales,+ int fmt, int S, int I, int O, int device);+typedef void (*fn_tensor_free)(ColiCudaTensor *tensor);+typedef size_t (*fn_tensor_bytes)(const ColiCudaTensor *tensor);+typedef int (*fn_tensor_device)(const ColiCudaTensor *tensor);++/* Resolved pointers, plus a flag so we attempt the load at most once. */+static struct {+ int loaded; /* 1 = load attempted (success or fail), 0 = not yet */+ int available; /* 1 = DLL loaded and all symbols resolved */+ HMODULE dll;+ fn_init init;+ fn_shutdown shutdown;+ fn_device_count device_count;+ fn_device_at device_at;+ fn_mem_info mem_info;+ fn_stats stats;+ fn_group_stats group_stats;+ fn_expert_mlp expert_mlp;+ fn_expert_group expert_group;+ fn_attention_absorb attention_absorb;+ fn_tensor_upload tensor_upload;+ fn_matmul matmul;+ fn_tensor_free tensor_free;+ fn_tensor_bytes tensor_bytes;+ fn_tensor_device tensor_device;+} g_cuda;++/* Resolve the DLL and all 11 symbols. Returns 1 on success, 0 otherwise.+ * Idempotent: the first call (success or fail) sticks; later calls are no-ops+ * that return the cached result. The engine treats a 0 return as "CUDA+ * unavailable" and falls back to the CPU path without aborting. */+static int coli_cuda_load(void){+ if(g_cuda.loaded) return g_cuda.available;+ g_cuda.loaded = 1;++ /* Search the model directory first (so a DLL shipped next to the model+ * wins), then the engine directory, then the default DLL search path. */+ g_cuda.dll = LoadLibraryA("coli_cuda.dll");+ if(!g_cuda.dll){+ fprintf(stderr, "[CUDA] coli_cuda.dll not found; GPU tier disabled "+ "(CPU path remains active).\n");+ return 0;+ }++ #define RESOLVE(name, type) \+ /* GetProcAddress returns FARPROC (void(*)(void)); casting it to a \+ * specific function-pointer type is the standard LoadLibrary idiom. \+ * -Wcast-function-type flags it but it is safe: the DLL exported \+ * the symbol with extern "C" and the exact signature we expect. */ \+ _Pragma("GCC diagnostic push") \+ _Pragma("GCC diagnostic ignored \"-Wcast-function-type\"") \+ g_cuda.name = (type)GetProcAddress(g_cuda.dll, "coli_cuda_" #name); \+ _Pragma("GCC diagnostic pop") \+ if(!g_cuda.name){ \+ fprintf(stderr, "[CUDA] coli_cuda.dll missing symbol coli_cuda_" #name "\n"); \+ FreeLibrary(g_cuda.dll); g_cuda.dll=NULL; return 0; }++ RESOLVE(init, fn_init)+ RESOLVE(shutdown, fn_shutdown)+ RESOLVE(device_count, fn_device_count)+ RESOLVE(device_at, fn_device_at)+ RESOLVE(mem_info, fn_mem_info)+ RESOLVE(stats, fn_stats)+ RESOLVE(group_stats, fn_group_stats)+ RESOLVE(expert_mlp, fn_expert_mlp)+ RESOLVE(expert_group, fn_expert_group)+ RESOLVE(attention_absorb, fn_attention_absorb)+ RESOLVE(tensor_upload, fn_tensor_upload)+ RESOLVE(matmul, fn_matmul)+ RESOLVE(tensor_free, fn_tensor_free)+ RESOLVE(tensor_bytes, fn_tensor_bytes)+ RESOLVE(tensor_device, fn_tensor_device)+ #undef RESOLVE++ g_cuda.available = 1;+ return 1;+}++/* ---- Public wrappers: match backend_cuda.h signatures exactly.+ * Each forwards to the resolved pointer; if the DLL never loaded, return the+ * "not initialized" result the engine already handles (init returns 0, matmul+ * returns 0 so the caller marks the tensor cuda_failed and uses CPU). ---- */++int coli_cuda_init(const int *devices, int count){+ if(!coli_cuda_load()) return 0;+ return g_cuda.init(devices, count);+}++void coli_cuda_shutdown(void){+ if(g_cuda.available && g_cuda.shutdown) g_cuda.shutdown();+}++int coli_cuda_device_count(void){+ if(!g_cuda.available) return 0;+ return g_cuda.device_count();+}++int coli_cuda_device_at(int index){+ if(!g_cuda.available) return -1;+ return g_cuda.device_at(index);+}++int coli_cuda_mem_info(int device, size_t *free_bytes, size_t *total_bytes){+ if(!g_cuda.available){ if(free_bytes)*free_bytes=0; if(total_bytes)*total_bytes=0; return 0; }+ return g_cuda.mem_info(device, free_bytes, total_bytes);+}++void coli_cuda_stats(int device, size_t *tensor_count, size_t *tensor_bytes){+ if(!g_cuda.available){ if(tensor_count)*tensor_count=0; if(tensor_bytes)*tensor_bytes=0; return; }+ g_cuda.stats(device, tensor_count, tensor_bytes);+}++void coli_cuda_group_stats(uint64_t *calls, uint64_t *experts, uint64_t *rows,+ double *h2d_ms, double *kernel_ms, double *d2h_ms){+ if(!g_cuda.available){+ if(calls)*calls=0; if(experts)*experts=0; if(rows)*rows=0;+ if(h2d_ms)*h2d_ms=0; if(kernel_ms)*kernel_ms=0; if(d2h_ms)*d2h_ms=0;+ return;+ }+ g_cuda.group_stats(calls, experts, rows, h2d_ms, kernel_ms, d2h_ms);+}++int coli_cuda_expert_mlp(ColiCudaTensor *gate, ColiCudaTensor *up,+ ColiCudaTensor *down, float *y, const float *x, int S){+ if(!g_cuda.available) return 0;+ return g_cuda.expert_mlp(gate, up, down, y, x, S);+}++int coli_cuda_expert_group(ColiCudaTensor *const *gates, ColiCudaTensor *const *ups,+ ColiCudaTensor *const *downs, const int *rows, int count,+ float *y, const float *x){+ if(!g_cuda.available) return 0;+ return g_cuda.expert_group(gates, ups, downs, rows, count, y, x);+}++int coli_cuda_attention_absorb(ColiCudaTensor *kv_b, float *ctx, const float *q,+ const float *latent, const float *rope, int H, int Q,+ int R, int V, int K, int T, float attention_scale){+ if(!g_cuda.available) return 0;+ return g_cuda.attention_absorb(kv_b, ctx, q, latent, rope, H, Q, R, V, K, T, attention_scale);+}++int coli_cuda_tensor_upload(ColiCudaTensor **tensor, const void *weights,+ const float *scales, int fmt, int I, int O, int device){+ if(!g_cuda.available) return 0;+ return g_cuda.tensor_upload(tensor, weights, scales, fmt, I, O, device);+}++int coli_cuda_matmul(ColiCudaTensor **tensor, float *y, const float *x,+ const void *weights, const float *scales,+ int fmt, int S, int I, int O, int device){+ if(!g_cuda.available) return 0;+ return g_cuda.matmul(tensor, y, x, weights, scales, fmt, S, I, O, device);+}++void coli_cuda_tensor_free(ColiCudaTensor *tensor){+ if(g_cuda.available && g_cuda.tensor_free) g_cuda.tensor_free(tensor);+}++size_t coli_cuda_tensor_bytes(const ColiCudaTensor *tensor){+ if(!g_cuda.available) return 0;+ return g_cuda.tensor_bytes(tensor);+}++int coli_cuda_tensor_device(const ColiCudaTensor *tensor){+ if(!g_cuda.available) return -1;+ return g_cuda.tensor_device(tensor);+}++#endif /* _WIN32 */diff --git a/c/coli b/c/coliindex 8c8e5573..7ff7812c 100755--- a/c/coli+++ b/c/coli@@ -383,13 +383,38 @@ def cmd_chat(a):banner(f"chat · {os.path.basename(a.model)} · ram {a.ram or '-'}GB · topp {a.topp or 'off'}")errlog=tempfile.NamedTemporaryFile(mode="w+", suffix=".log", delete=False)e=env_for(a); e["SERVE"]="1"+ # stderr -> PIPE, NOT stderr=errlog (file). On Windows/MinGW, pointing the+ # child's stderr at a file/DEVNULL handle stalls the CRT so stdout (the byte+ # protocol coli reads one byte at a time) never flushes and chat hangs at+ # ~10 GB resident. A PIPE whose read end nobody drains still works: the+ # engine emits only ~400 bytes of status to stderr, which fits comfortably+ # in the OS pipe buffer, so it never blocks. We snapshot stderr into errlog+ # once the READY sentinel arrives, so the status-line display below works+ # exactly as before. (Do NOT add a concurrent stderr drain thread: on+ # Windows, reading two child pipes simultaneously deadlocks CPython's IO.)p=subprocess.Popen([GLM,str(a.cap)], env=e, stdin=subprocess.PIPE,- stdout=subprocess.PIPE, stderr=errlog, bufsize=0)+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=0)sp=Spinner("waking the giant (744B)…"); sp.start()st=stream_turn(p, READY, lambda b: None)sp.stop()if st is None:+ try: errlog.write(p.stderr.read().decode("utf-8","replace"))+ except (OSError, ValueError): passerrlog.seek(0); print(errlog.read()[-1500:]); sys.exit("the engine exited while loading")+ # READY received. Drain the child's stderr into errlog without blocking:+ # the engine is still alive (blocked on stdin), so a plain read() would+ # hang forever waiting for EOF. A short bounded drain grabs the ~400 bytes+ # of load-time status ([RAM_GB], [MTP], ...) that were already emitted.+ _drain_box={"done":False}+ def _drain():+ try: errlog.write(p.stderr.read().decode("utf-8","replace"))+ except (OSError, ValueError): pass+ _drain_box["done"]=True+ threading.Thread(target=_drain, daemon=True).start()+ _drain_box["th"]=threading.current_thread()+ for _ in range(20): # up to ~1s for the load-status lines+ if _drain_box["done"]: break+ time.sleep(0.05)errlog.flush()try:elog=open(errlog.name).read()diff --git a/c/glm.c b/c/glm.cindex bce736eb..7f2d4913 100644--- a/c/glm.c+++ b/c/glm.c@@ -27,7 +27,9 @@#include <stdatomic.h> /* PIPE ready-flags/job queue + PILOT_REAL cross-layer handshake */#include <sched.h> /* sched_yield: PIPE spin / PILOT barrier */#include <unistd.h>-#include <sys/select.h>+#if defined(__APPLE__) || defined(__linux__)+#include <sys/select.h> /* select() serve-loop polling (#68); not on native MinGW */+#endif#if defined(__APPLE__) || defined(__linux__)#include <sys/resource.h>#include <sys/mman.h> /* mlock: inchioda le pagine in RAM / wire pages into RAM */@@ -439,6 +441,8 @@ static void matmul_i2(float *y, const float *x, const uint8_t *q2, const float ** RMS per matmul (attivazione int8), IDOT=0 torna al percorso f32 esatto. */#if defined(__AVX512VNNI__) && defined(__AVX512BW__)#define IDOT_KERNEL "avx512-vnni"+#elif defined(__AVXVNNI__) && defined(__AVX2__)+#define IDOT_KERNEL "avx-vnni"#elif defined(__AVX2__)#define IDOT_KERNEL "avx2"#elif defined(__ARM_NEON)@@ -475,6 +479,12 @@ static inline int hsum256_i32(__m256i v){return _mm_cvtsi128_si32(lo);}#endif+#if defined(__AVXVNNI__) && defined(__AVX2__)+/* hsum di un __m128i a 4 lane s32 (l'AVX-VNNI 128-bit accumula su 4 lane). */+static inline int hsum128_i32(__m128i v){+ v=_mm_hadd_epi32(v,v); v=_mm_hadd_epi32(v,v); return _mm_cvtsi128_si32(v);+}+#endif/* dot int8·int8: trucco del segno (|w| unsigned × x·sign(w) signed). Sicuro:* coppie <= 128*127*2 = 32512 < 32767, accumulo s32 fino a I=16384. */static inline int32_t dot_i8i8(const int8_t *w, const int8_t *x, int I){@@ -493,6 +503,18 @@ static inline int32_t dot_i8i8(const int8_t *w, const int8_t *x, int I){acc=_mm512_dpbusd_epi32(acc,_mm512_abs_epi8(wv),xs);}sum=_mm512_reduce_add_epi32(acc);+#elif defined(__AVXVNNI__) && defined(__AVX2__)+ /* AVX-VNNI 128-bit: vpdpbusd u8*s8 -> s32, 16 byte/iter. Stesso trucco del+ * segno della variante 512-bit: |w| via abs, segno piegato in x con maschera+ * (w==0 -> product 0). __AVX2__ serve per _mm_sign_epi8 / abs. */+ __m128i acc=_mm_setzero_si128();+ for(;i+16<=I;i+=16){+ __m128i wv=_mm_loadu_si128((const __m128i*)(w+i));+ __m128i xv=_mm_loadu_si128((const __m128i*)(x+i));+ __m128i xs=_mm_sign_epi8(xv,wv); /* x * sign(w); _mm_sign zona __AVX2__ */+ acc=_mm_dpbusd_epi32(acc,_mm_abs_epi8(wv),xs);+ }+ sum=hsum128_i32(acc);#elif defined(__AVX2__)__m256i acc=_mm256_setzero_si256(); const __m256i ones=_mm256_set1_epi16(1);for(;i+32<=I;i+=32){@@ -563,6 +585,23 @@ static inline int32_t dot_i4i8(const uint8_t *w4, const int8_t *x, int I){acc=_mm512_dpbusd_epi32(acc,_mm512_abs_epi8(wv),xs);}sum=_mm512_reduce_add_epi32(acc);+#elif defined(__AVXVNNI__) && defined(__AVX2__)+ /* AVX-VNNI 128-bit, int4: 16 byte = 32 nibble -> int8 [-8,7] in due half+ * (n0/n1), ciascuno alimentato a un vpdpbusd da 16 byte. Stesso unpack+ * 128-bit del ramo AVX2 sotto; 32 elementi/iter come li. */+ const __m128i m4=_mm_set1_epi8(0x0F); const __m128i b8=_mm_set1_epi8(8);+ __m128i acc=_mm_setzero_si128();+ for(;i+32<=I;i+=32){+ __m128i by=_mm_loadu_si128((const __m128i*)(w4+(i>>1))); /* 16 byte = 32 nibble */+ __m128i lo=_mm_and_si128(by,m4), hi=_mm_and_si128(_mm_srli_epi16(by,4),m4);+ __m128i n0=_mm_unpacklo_epi8(lo,hi), n1=_mm_unpackhi_epi8(lo,hi); /* nibble in ordine */+ __m128i w0=_mm_sub_epi8(n0,b8), w1=_mm_sub_epi8(n1,b8);+ __m128i x0=_mm_loadu_si128((const __m128i*)(x+i));+ __m128i x1=_mm_loadu_si128((const __m128i*)(x+i+16));+ acc=_mm_dpbusd_epi32(acc,_mm_abs_epi8(w0),_mm_sign_epi8(x0,w0));+ acc=_mm_dpbusd_epi32(acc,_mm_abs_epi8(w1),_mm_sign_epi8(x1,w1));+ }+ sum=hsum128_i32(acc);#elif defined(__AVX2__)const __m128i m4=_mm_set1_epi8(0x0F); const __m256i b8=_mm256_set1_epi8(8);const __m256i ones=_mm256_set1_epi16(1);@@ -1128,7 +1167,9 @@ static pthread_mutex_t g_map_mtx = PTHREAD_MUTEX_INITIALIZER; /* expert_load estatic void *map_of_fd(int fd){pthread_mutex_lock(&g_map_mtx);for(int i=0;i<g_nmaps;i++) if(g_maps[i].fd==fd){ void *b=g_maps[i].base; pthread_mutex_unlock(&g_map_mtx); return b; }- void *base=NULL; struct stat st;+ void *base=NULL;+#if defined(__APPLE__) || defined(__linux__)+ struct stat st;if(g_nmaps<512 && fstat(fd,&st)==0){size_t len=((size_t)st.st_size+16383)&~(size_t)16383;void *p=mmap(NULL,len,PROT_READ,MAP_SHARED,fd,0);@@ -1139,6 +1180,7 @@ static void *map_of_fd(int fd){#endif}}+#endifpthread_mutex_unlock(&g_map_mtx);return base;}@@ -1198,7 +1240,9 @@ static int expert_load(Model *m, int layer, int eid, ESlot *s, int fatal){* residency. This is pread's I/O without the copy and without the slab. */for(int k=0;k<3;k++){char *p=(char*)bw[k]+tw[k]->off; size_t n=(size_t)tw[k]->nbytes;+#if defined(__APPLE__) || defined(__linux__)madvise((void*)((uintptr_t)p & ~16383UL), n+16384, MADV_WILLNEED);+#endifvolatile char acc=0;for(size_t i=0;i<n;i+=4096) acc+=p[i];acc+=p[n-1]; (void)acc;@@ -3082,6 +3126,7 @@ static int mux_submit(Model *m, Tok *T, ServeCtx *ctx, ServeReq *req, int nctx,}static void run_serve_mux(Model *m, const char *snap){+#if defined(__APPLE__) || defined(__linux__)char tkp[2048]; snprintf(tkp,sizeof(tkp),"%s/tokenizer.json",snap);Tok T; tok_load(&T,tkp); int eos=tok_id_of(&T,"<|endoftext|>"); stops_arm(&m->c,eos);g_draft=0; /* one scheduler owns every forward; MTP/speculation is not ragged-safe */@@ -3123,9 +3168,33 @@ static void run_serve_mux(Model *m, const char *snap){usage_save(m);for(int i=0;i<nctx;i++) serve_ctx_free(m,&ctx[i]); free(ctx); free(req);m->kv=NULL; m->Lc=m->Rc=m->Ic=NULL; m->kv_start=NULL; m->max_t=0;+#else+ /* SERVE_BATCH (continuous batching) uses select() on stdin, a Unix-ism.+ * Not yet ported to native Windows — fall back to the single-sequence+ * serve path (run_serve). Remove this stub once select()-free polling+ * (e.g. WaitForSingleObject on the stdin handle) is implemented. */+ (void)snap;+ fprintf(stderr,"[SERVE_BATCH] continuous-batching serve is not yet available on "+ "native Windows; use the default serve path (omit SERVE_BATCH).\n");+#endif}static void run_serve(Model *m, const char *snap){+ /* Serve mode speaks a byte protocol over BOTH stdout and stdin:+ * stdout: \x01\x01READY\x01\x01\n, STAT lines, \x01\x01END\x01\x01\n+ * stdin: text lines plus \x02RESET / \x02MORE control bytes.+ * 'coli' matches the sentinels with endswith() and a "^STAT ..." regex,+ * so they must arrive byte-exact (LF, no CR). On Windows the CRT opens+ * both handles in TEXT mode: stdout translates '\n'->'\r\n' (so the READY+ * sentinel never matches and chat hangs at ~10 GB resident), and stdin+ * translates '\r\n'->'\n' and rejects writes of raw bytes with EINVAL,+ * breaking the control protocol. Put BOTH handles in BINARY mode so the+ * protocol bytes are exact in both directions. No-op on Linux/macOS. */+#ifdef _WIN32+ _setmode(_fileno(stdin), _O_BINARY);+ _setmode(_fileno(stdout), _O_BINARY);+ setvbuf(stdout, NULL, _IONBF, 0);+#endifchar tkp[2048]; snprintf(tkp,sizeof(tkp),"%s/tokenizer.json",snap);Tok T; tok_load(&T,tkp);int eos=tok_id_of(&T,"<|endoftext|>");diff --git a/c/resource_plan.py b/c/resource_plan.pyindex 1c3a5723..22da7e0b 100644--- a/c/resource_plan.py+++ b/c/resource_plan.py@@ -7,6 +7,7 @@ import reimport shutilimport statisticsimport subprocess+import sysfrom pathlib import Path@@ -76,11 +77,46 @@ def analyze_model(model):def memory_available():+ # Linux (and MSYS2/Git-Bash CPython where /proc exists): MemAvailable.try:text = Path("/proc/meminfo").read_text()return int(re.search(r"MemAvailable:\s+(\d+)", text).group(1)) * 1024except (OSError, AttributeError):- return 0+ pass+ # Windows native CPython: GlobalMemoryStatusEx -> ullAvailPhys.+ # Same definition the C engine uses (compat_meminfo in compat.h):+ # standby/free/zero pages, i.e. reclaimable without swapping.+ if sys.platform == "win32":+ try:+ import ctypes++ class MEMORYSTATUSEX(ctypes.Structure):+ _fields_ = [("dwLength", ctypes.c_ulong),+ ("dwMemoryLoad", ctypes.c_ulong),+ ("ullTotalPhys", ctypes.c_ulonglong),+ ("ullAvailPhys", ctypes.c_ulonglong),+ ("ullTotalVirtual", ctypes.c_ulonglong),+ ("ullAvailVirtual", ctypes.c_ulonglong),+ ("ullAvailExtendedVirtual", ctypes.c_ulonglong)]++ stat = MEMORYSTATUSEX(dwLength=ctypes.sizeof(MEMORYSTATUSEX))+ kernel32 = ctypes.windll.kernel32+ kernel32.GlobalMemoryStatusEx.argtypes = [ctypes.c_void_p]+ kernel32.GlobalMemoryStatusEx.restype = ctypes.c_int+ if kernel32.GlobalMemoryStatusEx(ctypes.byref(stat)) and stat.ullAvailPhys:+ return stat.ullAvailPhys+ # Fallback (e.g. sandboxed callers where GlobalMemoryStatusEx reports+ # nothing): total installed RAM in KB. Less precise than ullAvailPhys+ # — it ignores standby/reclaimable pages — but never returns 0 on a+ # real machine, which keeps the expert cache from being mis-sized.+ total_kb = ctypes.c_ulonglong(0)+ kernel32.GetPhysicallyInstalledSystemMemory.argtypes = [ctypes.c_void_p]+ kernel32.GetPhysicallyInstalledSystemMemory.restype = ctypes.c_int+ if kernel32.GetPhysicallyInstalledSystemMemory(ctypes.byref(total_kb)):+ return total_kb.value * 1024+ except OSError:+ pass+ return 0def discover_gpus():diff --git a/c/tests/test_resource_plan.py b/c/tests/test_resource_plan.pyindex d44f6614..5a897732 100644--- a/c/tests/test_resource_plan.py+++ b/c/tests/test_resource_plan.py@@ -6,7 +6,14 @@ import tempfileimport unittestfrom pathlib import Path-from resource_plan import GB, analyze_model, build_plan, environment_for_plan, format_plan+from resource_plan import (+ GB,+ analyze_model,+ build_plan,+ environment_for_plan,+ format_plan,+ memory_available,+)def write_shard(path, tensors):@@ -53,6 +60,12 @@ class ResourcePlanTest(unittest.TestCase):self.assertEqual(info["expert_count"], 2)self.assertEqual(info["per_cap_bytes"], 60)+ def test_memory_available_is_positive(self):+ # Regression: on native Windows CPython, /proc/meminfo does not exist,+ # so the Linux-only path returned 0 and the expert cache was sized to+ # 0 slots/layer. The value must be a sane positive number of bytes.+ self.assertGreater(memory_available(), 0)+def test_builds_bounded_three_tier_plan(self):gpus = [{"index": 0, "name": "test-gpu", "total_bytes": 12 * GB,"free_bytes": 10 * GB}]diff --git a/c/warmup.ps1 b/c/warmup.ps1new file mode 100644index 00000000..2301a2fe--- /dev/null+++ b/c/warmup.ps1@@ -0,0 +1,146 @@+# warmup.ps1 - overnight expert-cache warmup for colibri+#+# Runs `coli run` in a loop with diverse prompts so the engine records which+# routed experts your workload actually uses into .coli_usage. At startup the+# engine pins the hottest experts into RAM; the more history it has, the bigger+# and more accurate that pin gets. This does NOT load random experts - it loads+# whatever the model actually routes to for these prompts, then promotes the+# frequent ones.+#+# Usage (from the c\ directory):+# .\warmup.ps1 # defaults: model next to repo, 3 rounds+# .\warmup.ps1 -Model D:\glm52_i4 -Rounds 10 -Ngen 400+#+# Let it run while you sleep. Each iteration logs selections count + hit rate.+# Ctrl-C is safe: each run saves usage atomically only on clean completion, so+# the file is never corrupted (but a killed mid-generation run saves nothing).+#+# Why diverse prompts? Expert routing is content-dependent. Coding prompts+# activate different experts than poetry or math. A spread of topics builds a+# general-purpose pin that helps whatever YOU ask later. If you only ever warm+# on one topic, the pin overfits to that topic.++param(+ [string]$Model = (Resolve-Path (Join-Path $PSScriptRoot "..\glm52_i4")).Path,+ [int]$Rounds = 3,+ # Default 32 (not 500): on a cold QLC cache a 500-token run takes hours and+ # a killed mid-generation run saves nothing (usage_save runs only on clean+ # completion). 32 tokens finishes in ~5-10 min even cold, so usage saves+ # frequently and the loop accumulates selections steadily overnight. Each+ # 32-token prompt still records ~90k expert selections.+ [int]$Ngen = 32,+ [string]$Log = (Join-Path $PSScriptRoot "warmup.log")+)++# "Continue" (not "Stop"): the engine writes status to stderr, which "Stop"+# treats as a fatal error and aborts the whole warmup loop on every prompt.+$ErrorActionPreference = "Continue"+$Coli = Join-Path $PSScriptRoot "coli"++if (-not (Test-Path $Coli)) { Write-Error "coli not found at $Coli - run from the c\ directory"; exit 1 }+if (-not (Test-Path $Model)) { Write-Error "model not found at $Model"; exit 1 }++# Diverse prompts across domains - each touches a different expert distribution.+# Kept open-ended ("explain", "write", "list") so generation runs to NGEN tokens+# and routes through many experts rather than stopping early on a short answer.+$Prompts = @(+ "Explain how a transformer neural network works, covering attention, feed-forward layers, and backpropagation in detail.",+ "Write a Python function that implements quicksort with in-place partitioning, including comments explaining each step.",+ "Describe the causes and major events of the French Revolution in chronological order.",+ "What is the difference between TCP and UDP? Explain handshakes, reliability, and use cases.",+ "Write a short story about a lighthouse keeper who discovers a message in a bottle.",+ "Explain the theory of general relativity, including the equivalence principle and gravitational time dilation.",+ "List and describe the major organ systems of the human body and their primary functions.",+ "How does photosynthesis work? Explain the light-dependent reactions and the Calvin cycle.",+ "Write a C program that reads a file line by line and counts word frequency using a hash table.",+ "Summarize the plot of Shakespeare's Hamlet, act by act.",+ "Explain the difference between supervised, unsupervised, and reinforcement learning with examples of each.",+ "What causes climate change? Describe the greenhouse effect, carbon cycle, and major greenhouse gases.",+ "Write a recipe for a classic French onion soup, with step-by-step instructions.",+ "Describe how the internet works, from typing a URL to rendering a webpage, including DNS, TCP, HTTP, and browsers.",+ "Explain database normalization, including first, second, and third normal forms with examples.",+ "What is quantum entanglement? Explain it as if to a curious high school student.",+ "Write a poem about the ocean and the passage of time.",+ "Describe the water cycle, including evaporation, condensation, precipitation, and transpiration.",+ "How do vaccines work? Explain the immune response, antibodies, and mRNA vaccine technology.",+ "Explain the Big Bang theory and the evidence supporting it, including cosmic microwave background and redshift.",+ "Write a Python class for a binary search tree with insert, search, and inorder traversal methods.",+ "What are the major branches of philosophy? Describe epistemology, ethics, metaphysics, and logic.",+ "Explain how a CPU executes an instruction, covering fetch, decode, execute, and writeback.",+ "Describe the life cycle of a star, from protostar to main sequence to red giant and beyond.",+ "How does public key cryptography work? Explain RSA, including key generation, encryption, and signing.",+ "Write a dialogue between two characters debating whether artificial intelligence can be conscious.",+ "Explain the economic concepts of supply and demand, elasticity, and market equilibrium.",+ "What is CRISPR gene editing and how does it work? Explain Cas9, guide RNA, and applications.",+ "Describe the major causes and consequences of World War I.",+ "How does a compiler work? Explain lexing, parsing, semantic analysis, optimization, and code generation."+)++function Get-Selections {+ $u = Join-Path $Model ".coli_usage"+ if (-not (Test-Path $u)) { return 0 }+ $tot = 0+ Get-Content $u | ForEach-Object {+ $p = $_ -split '\s+'+ if ($p.Count -eq 3) { $tot += [int]$p[2] }+ }+ return $tot+}++$start = Get-Date+$baseline = Get-Selections+$line = "=" * 72+"$line" | Tee-Object -FilePath $Log -Append+"colibri warmup - started $start" | Tee-Object -FilePath $Log -Append+" model: $Model" | Tee-Object -FilePath $Log -Append+" rounds: $Rounds x $($Prompts.Count) prompts" | Tee-Object -FilePath $Log -Append+" ngen: $Ngen tokens/prompt" | Tee-Object -FilePath $Log -Append+" baseline: $baseline selections" | Tee-Object -FilePath $Log -Append+"$line" | Tee-Object -FilePath $Log -Append++$iter = 0+$total = $Rounds * $Prompts.Count+for ($r = 1; $r -le $Rounds; $r++) {+ for ($i = 0; $i -lt $Prompts.Count; $i++) {+ $iter+++ $prompt = $Prompts[$i]+ $now = Get-Date -Format "HH:mm:ss"+ $sel = Get-Selections+ $header = "[$now] round $r/$Rounds prompt {0,2}/$($Prompts.Count) (iter $iter/$total) selections: $sel" -f ($i+1)+ $header | Tee-Object -FilePath $Log -Append+ " prompt: $($prompt.Substring(0, [Math]::Min(70, $prompt.Length)))..." | Tee-Object -FilePath $Log -Append++ $t0 = Get-Date+ # coli run writes status to stderr (normal) and may exit non-zero on+ # EOS-early; neither is a real failure for our purpose. Relax the+ # error preference and collect ALL output streams so stderr text+ # doesn't abort the loop.+ $prev = $ErrorActionPreference+ $ErrorActionPreference = "Continue"+ try {+ $output = & python $Coli run --model $Model --ngen $Ngen $prompt 2>&1 |+ Select-Object -Last 4+ } catch {+ $output = @(" (engine run threw: $($_.Exception.Message))")+ }+ $ErrorActionPreference = $prev+ $elapsed = ((Get-Date) - $t0).TotalSeconds+ $after = Get-Selections+ $delta = $after - $sel++ $output | ForEach-Object { " $_" | Tee-Object -FilePath $Log -Append }+ " -> {0:N0}s, +{1} selections (now {2})" -f $elapsed, $delta, $after | Tee-Object -FilePath $Log -Append+ "" | Tee-Object -FilePath $Log -Append+ }+}++$end = Get-Date+$final = Get-Selections+$gain = $final - $baseline+$duration = ($end - $start).ToString("hh\:mm\:ss")+"$line" | Tee-Object -FilePath $Log -Append+"colibri warmup - finished $end" | Tee-Object -FilePath $Log -Append+" duration: $duration" | Tee-Object -FilePath $Log -Append+" selections: $baseline -> $final (+$gain)" | Tee-Object -FilePath $Log -Append+" next: python coli chat --model $Model" | Tee-Object -FilePath $Log -Append+"$line" | Tee-Object -FilePath $Log -Append