Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

metal: the Engram rows are read with two threads (more gave different words run to run; the cause is not found yet); a failed Engram row read is reported

Niko1221bc6c32555b7b
diff --git a/docs/DEEPSEEK_MAC.md b/docs/DEEPSEEK_MAC.md
index 4453965d6..86c63645d 100644
--- a/docs/DEEPSEEK_MAC.md
+++ b/docs/DEEPSEEK_MAC.md
@@ -76,6 +76,8 @@ Full quality. That is what was measured and scored.
- Smaller Macs: nothing under 128 GB was run with DeepSeek. The plan screen shows how much of the model your Mac keeps in memory
and warns when that is under a quarter of it. That estimate follows the engine's own rule and is not a measurement.
- The 1.75 bpw set shares the 2.00 set's kernel but was not run on a Mac, so it is not offered there.
+- The Mac reads the Engram table with two threads. With more, the same prompt gave slightly different words from run to run
+ (in 3 to 9 of 16 runs); with one or two threads it did not. The cause is under study.
- Two runs of the same prompt can differ in the last digits of a score, because the engine moves experts between memory and the
SSD as it learns what the prompt needs. The answers stay equally good.
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cu
index 51f1eac08..11df0048c 100644
--- a/src/dsv41/ds4_model.cu
+++ b/src/dsv41/ds4_model.cu
@@ -2757,6 +2757,12 @@ struct Model::Impl {
#endif
const unsigned hw = std::max(1u, std::thread::hardware_concurrency());
int nth = (int) std::min(16u, std::max(4u, hw / 2));
+#if defined(STRATA_METAL)
+ // A Mac reads the Engram rows with two threads: with more, a decode of the same prompt gave slightly different words run to run
+ // (3 to 9 runs of 16 differed from the serial reference, against none of 24 with one or two threads; the cause is not found yet). DS4_ENGRAM_THREADS=N sets another number.
+ nth = 2;
+ if (std::getenv("DS4_ENGRAM_THREADS") == nullptr) std::fprintf(stderr, "ds4: Engram rows are read with 2 threads (DS4_ENGRAM_THREADS=N sets another number)\n");
+#endif
if (const char* e = std::getenv("DS4_ENGRAM_THREADS")) nth = std::max(1, std::atoi(e));
const uint64_t avail = ram_avail();
uint64_t mb = avail == ~uint64_t{0} ? 64 : std::min<uint64_t>(256, std::max<uint64_t>(16, (avail >> 20) / 64));
@@ -6159,6 +6165,10 @@ struct Model::Impl {
for (size_t i = 0; i < b.reads.size(); ++i) {
const EgRead& r = b.reads[i];
if (!b.ok[i]) { // (a failed read: zeros, its gate adds nothing; not kept)
+#if defined(STRATA_METAL)
+ std::fprintf(stderr, "ds4: WARNING: an Engram row could not be read (file %d, offset %llu, %u bytes): it counts as zeros\n", r.file,
+ (unsigned long long) r.off, r.n);
+#endif
std::memset(r.dst, 0, (size_t) hd);
std::memset(r.scales, 0, (size_t) hd / 32);
continue;