Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
metal: the Engram rows are read with two threads (more gave different words run to run; the cause is not found yet); a failed Engram row read is reported
Niko1221bc6c32555b7b
diff --git a/docs/DEEPSEEK_MAC.md b/docs/DEEPSEEK_MAC.mdindex 4453965d6..86c63645d 100644--- a/docs/DEEPSEEK_MAC.md+++ b/docs/DEEPSEEK_MAC.md@@ -76,6 +76,8 @@ Full quality. That is what was measured and scored.- Smaller Macs: nothing under 128 GB was run with DeepSeek. The plan screen shows how much of the model your Mac keeps in memoryand warns when that is under a quarter of it. That estimate follows the engine's own rule and is not a measurement.- The 1.75 bpw set shares the 2.00 set's kernel but was not run on a Mac, so it is not offered there.+- The Mac reads the Engram table with two threads. With more, the same prompt gave slightly different words from run to run+ (in 3 to 9 of 16 runs); with one or two threads it did not. The cause is under study.- Two runs of the same prompt can differ in the last digits of a score, because the engine moves experts between memory and theSSD as it learns what the prompt needs. The answers stay equally good.diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cuindex 51f1eac08..11df0048c 100644--- a/src/dsv41/ds4_model.cu+++ b/src/dsv41/ds4_model.cu@@ -2757,6 +2757,12 @@ struct Model::Impl {#endifconst unsigned hw = std::max(1u, std::thread::hardware_concurrency());int nth = (int) std::min(16u, std::max(4u, hw / 2));+#if defined(STRATA_METAL)+ // A Mac reads the Engram rows with two threads: with more, a decode of the same prompt gave slightly different words run to run+ // (3 to 9 runs of 16 differed from the serial reference, against none of 24 with one or two threads; the cause is not found yet). DS4_ENGRAM_THREADS=N sets another number.+ nth = 2;+ if (std::getenv("DS4_ENGRAM_THREADS") == nullptr) std::fprintf(stderr, "ds4: Engram rows are read with 2 threads (DS4_ENGRAM_THREADS=N sets another number)\n");+#endifif (const char* e = std::getenv("DS4_ENGRAM_THREADS")) nth = std::max(1, std::atoi(e));const uint64_t avail = ram_avail();uint64_t mb = avail == ~uint64_t{0} ? 64 : std::min<uint64_t>(256, std::max<uint64_t>(16, (avail >> 20) / 64));@@ -6159,6 +6165,10 @@ struct Model::Impl {for (size_t i = 0; i < b.reads.size(); ++i) {const EgRead& r = b.reads[i];if (!b.ok[i]) { // (a failed read: zeros, its gate adds nothing; not kept)+#if defined(STRATA_METAL)+ std::fprintf(stderr, "ds4: WARNING: an Engram row could not be read (file %d, offset %llu, %u bytes): it counts as zeros\n", r.file,+ (unsigned long long) r.off, r.n);+#endifstd::memset(r.dst, 0, (size_t) hd);std::memset(r.scales, 0, (size_t) hd / 32);continue;