Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

deepseek_v4: the macOS available-memory comment described different code

JustVuggd135b7131f37
diff --git a/c/deepseek_v4.c b/c/deepseek_v4.c
index e2a8bf27..66a83141 100644
--- a/c/deepseek_v4.c
+++ b/c/deepseek_v4.c
@@ -1060,7 +1060,25 @@ uint64_t coli_v4_os_available_memory(void) {
/* No /proc and no _SC_AVPHYS_PAGES on macOS. "Available" is what the
* kernel could hand out without swapping: free + inactive pages -- the
* same approximation Activity Monitor reports, and the closest analogue
- * of Linux's MemAvailable (which also counts reclaimable cache). */
+ * of Linux's MemAvailable (which also counts reclaimable cache).
+ *
+ * Two things a caller must know before trusting this number.
+ *
+ * It does NOT add purgeable_count, while inkling.c, kimi_k3.c, compat.h and
+ * telemetry.h all do. The same machine therefore reports a smaller figure
+ * here than through any other engine. Keep the two in mind together: they
+ * are not interchangeable.
+ *
+ * And "could hand out without swapping" is not "will keep resident".
+ * macOS answers memory pressure by COMPRESSING anonymous pages rather than
+ * swapping them, so a budget this function accepts can still end up half
+ * compressed, and every cache hit then pays a decompression. Swap stays at
+ * zero throughout, so a swap-based check sees nothing wrong. Reported and
+ * measured in issue #1614: on a 48 GB machine a 32 GiB budget decoded
+ * SLOWER than a 16 GiB one (0.96 vs 1.31 tok/s) while the hit rate rose
+ * monotonically. Sizing a cache from this number alone is therefore
+ * unsafe on Darwin; a fix needs vm.compressor_page_count, which this
+ * function does not read. */
mach_msg_type_number_t count = HOST_VM_INFO64_COUNT;
vm_statistics64_data_t vm;
vm_size_t page = 0;
@@ -13897,9 +13915,14 @@ static void v4_hwinfo_emit(void) {
* contract the web UI was built against. Hence bytes/1e9, not bytes/2^30.
*
* Availability reuses coli_v4_os_available_memory() rather than repeating the detection:
- * it already carries a Darwin branch (free + inactive + purgeable pages, the MemAvailable
- * equivalent). Declared extern because the amalgamation compiles this file once per
- * -DCOLI_V4_UNIT_*, so the definition need not be in this unit. */
+ * it already carries a Darwin branch. Declared extern because the amalgamation compiles
+ * this file once per -DCOLI_V4_UNIT_*, so the definition need not be in this unit.
+ *
+ * That branch returns free + inactive, NOT free + inactive + purgeable: this engine is
+ * deliberately one term more conservative than inkling.c, kimi_k3.c, compat.h and
+ * telemetry.h, which all add purgeable_count. Whoever changes one of the two formulas
+ * should know the other exists, or the same machine will report two different "free RAM"
+ * figures depending on which engine is asked. */
{
extern uint64_t coli_v4_os_available_memory(void);
if (!cpu[0]) {
@@ -13929,9 +13952,14 @@ static void v4_hwinfo_emit(void) {
* contract the web UI was built against. Hence bytes/1e9, not bytes/2^30.
*
* Availability reuses coli_v4_os_available_memory() rather than repeating the detection:
- * it already carries a Darwin branch (free + inactive + purgeable pages, the MemAvailable
- * equivalent). Declared extern because the amalgamation compiles this file once per
- * -DCOLI_V4_UNIT_*, so the definition need not be in this unit. */
+ * it already carries a Darwin branch. Declared extern because the amalgamation compiles
+ * this file once per -DCOLI_V4_UNIT_*, so the definition need not be in this unit.
+ *
+ * That branch returns free + inactive, NOT free + inactive + purgeable: this engine is
+ * deliberately one term more conservative than inkling.c, kimi_k3.c, compat.h and
+ * telemetry.h, which all add purgeable_count. Whoever changes one of the two formulas
+ * should know the other exists, or the same machine will report two different "free RAM"
+ * figures depending on which engine is asked. */
{
extern uint64_t coli_v4_os_available_memory(void);
if (!cpu[0]) {