Skip to content

armgit / janas

Public Imported from GitHub
0

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

Release 2026-10-30 (2): fma() rounded once on the GPUs whose driver splits it (RADV before RDNA2, issue #48) - a probe when the GPU opens, and shaders whose fma goes through float64; float conversions held exact in the shaders

camaurifaae71d71e59
diff --git a/ChangeLog.md b/ChangeLog.md
index 1468324..bf2071d 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -2,6 +2,11 @@
> Curated, user-facing summary of completed work, newest first.
+## [2026-10-30] - The GPU's products the CPU's to the bit on older AMD GPUs
+
+- **`fma()` rounded once where the driver splits it** (issue #48): Mesa's RADV lowers every 32-bit `fma` into a multiply and an add, two roundings, on the AMD GPUs before RDNA2 - Polaris, Vega, the Renoir and Cezanne APUs, RDNA1 - even where the shader asks for one operation, so the GPU's products came out a unit in the last place off the CPU's on about half the rows, and the self-check gave them to the CPU. When the GPU opens, a probe now computes `fma()` on 1,024 triples where one rounding and two differ; where the driver splits it, every shader that calls `fma()` is built from a variant whose `fma()` is rounded once through float64 (the product exact in a double, the sum rounded to odd, then to a float: Boldo and Melquiond's method; the same as the C library's `fmaf` on 200 million triples), provided the variant passes the same probe. On the Intel Arc of the development laptop, with the variants forced (`JANAS_GPU_XFMA=1`): `gpu_check`'s 3,980 products and 96 BF16 ones the CPU's to the bit.
+- **Float conversions held exact in the shaders**: Mesa's NIR took `double(float(i))` for `double(i)`, the integer's rounding to a float lost; every conversion is now decorated as `fma()` already was (`tools/spv_fma.c`).
+
## [2026-10-29] - Speech in every sound file OpenAI accepts
- **Word times** (`timestamp_granularities[]=word` in janas-server's transcriptions, `words` in the library's parameters with `janas_llm_speech_words` and `janas_llm_speech_word`, the FreeBASIC header too), found as OpenAI's open-source Whisper finds them: the window's text decoded again keeping the attention of the model's alignment heads over the sound (from the checkpoint; OpenAI's default, the decoder's second half, without them), its softmax standardized and median-filtered, the path of least cost through it by dynamic time warping, tokens joined into words as OpenAI's tokenizer joins them (characters for Chinese, Japanese, Thai, Lao, Burmese and Cantonese), punctuation joined to its word, OpenAI's corrections at pauses and segment ends, and the next window starting at the last word's end. On John F. Kennedy's 11 seconds the words end where whisper.cpp's token times (the same method, in half precision) put them, within 20-40 ms.
diff --git a/build.sh b/build.sh
index 1db08bb..a5146ae 100755
--- a/build.sh
+++ b/build.sh
@@ -246,6 +246,7 @@ mkdir -p "$OBJ" "$BIN"
SPV="$OBJ/spv"
mkdir -p "$SPV"
GPU_MISSING=""
+XFMA_NAMES=() # the shaders with an _xfma variant
VK_INC=()
if [ "$OS" = windows ] && [ -d /usr/include/vulkan ] &&
! echo '#include <vulkan/vulkan.h>' | "$CC" -E -x c - >/dev/null 2>&1; then
@@ -304,8 +305,56 @@ if [ -n "$GLSLANG" ]; then
fi
rm -f "$SPV/$name.spv"
fi
+ # a shader that calls fma() gets an _xfma variant, fma() rounded
+ # once in float64 (shaders/xfma.glsl after its #extension lines),
+ # for the drivers that split it (gpu_fma.c)
+ grep -q 'fma(' "$comp" || continue
+ xout="$SPV/${name}_xfma.h"
+ if [ ! -f "$xout" ] || [ "$comp" -nt "$xout" ] ||
+ [ "$SPV_FMA" -nt "$xout" ] ||
+ [ src/llm/shaders/xfma.glsl -nt "$xout" ]; then
+ echo "SPV $comp (xfma)"
+ last=$(grep -n '^#extension\|^#version' "$comp" | tail -1 | cut -d: -f1)
+ { head -n "$last" "$comp"; cat src/llm/shaders/xfma.glsl
+ echo "#line $((last + 1))"; tail -n "+$((last + 1))" "$comp"; } \
+ >"$SPV/${name}_xfma.comp"
+ if ! glslangValidator -V --target-env vulkan1.3 \
+ "$SPV/${name}_xfma.comp" -o "$SPV/${name}_xfma.spv" >/dev/null ||
+ ! "$SPV_FMA" "$SPV/${name}_xfma.spv" "janas_spv_${name}_xfma" \
+ "$xout"; then
+ rm -f "$xout"
+ GLSLANG=""
+ GPU_MISSING="${GPU_MISSING:+$GPU_MISSING and }a glslangValidator that compiles $comp with float64"
+ break
+ fi
+ rm -f "$SPV/${name}_xfma.comp" "$SPV/${name}_xfma.spv"
+ fi
+ XFMA_NAMES+=("$name")
done
fi
+# the table of the variants (gpu_fma.c), rewritten only when it changes
+if [ -n "$GLSLANG" ]; then
+ {
+ echo "/* Generated by build.sh: the shaders' _xfma variants. */"
+ for name in "${XFMA_NAMES[@]}"; do
+ echo "#include \"${name}_xfma.h\""
+ echo "extern const uint32_t janas_spv_${name}[];"
+ done
+ echo "static const struct janas_xfma {"
+ echo " const uint32_t *code, *xfma;"
+ echo " size_t bytes;"
+ echo "} janas_xfma_table[] = {"
+ for name in "${XFMA_NAMES[@]}"; do
+ echo " {janas_spv_${name}, janas_spv_${name}_xfma, sizeof(janas_spv_${name}_xfma)},"
+ done
+ echo "};"
+ } >"$SPV/xfma_table.h.new"
+ if cmp -s "$SPV/xfma_table.h.new" "$SPV/xfma_table.h"; then
+ rm -f "$SPV/xfma_table.h.new"
+ else
+ mv "$SPV/xfma_table.h.new" "$SPV/xfma_table.h"
+ fi
+fi
[ -z "$GPU_MISSING" ] ||
echo "note: building without GPU support: it needs $GPU_MISSING"
# the reason, for the engine to give when asked for the GPU
diff --git a/docs/hardware.md b/docs/hardware.md
index 89a70e1..1fea756 100644
--- a/docs/hardware.md
+++ b/docs/hardware.md
@@ -38,7 +38,7 @@ The [hardware report form](https://github.com/prabanta-dev/janas/issues/new?temp
**Whether a GPU can run the GPU backend**, and why not: `bin/x86_64-linux/vkcheck` lists what the backend needs with the answer of the GPU's driver, the decision the engine would take, and three short tests on the device (zero-copy, the int8 dot product, the float operations that keep the GPU's results the CPU's to the bit). It builds alone where Janas does not build yet, macOS with MoltenVK included: `cc -O2 tools/vkcheck.c -o vkcheck -ldl -lm` (on macOS without `-ldl -lm`, with `-I/opt/homebrew/include`). Its output is welcome in a hardware report.
-**The engine checks the GPU by itself**: when it opens the GPU it runs a product of every type against the CPU's and leaves out any shader the driver gets wrong (the status line says which; once per driver, kept in `~/.cache/janas/gpu_selfcheck`; `JANAS_GPU_SELFCHECK=0` turns it off).
+**The engine checks the GPU by itself**: when it opens the GPU it runs a product of every type against the CPU's and leaves out any shader the driver gets wrong (the status line says which; once per driver, kept in `~/.cache/janas/gpu_selfcheck`; `JANAS_GPU_SELFCHECK=0` turns it off). Before that it checks that the driver rounds `fma()` once, as the CPU does: Mesa's RADV splits it into a multiply and an add on the AMD GPUs before RDNA2 (Polaris, Vega, the Renoir and Cezanne APUs, RDNA1), and there the engine builds its shaders with an `fma()` of their own, rounded once through float64, so that their products stay the CPU's to the bit (`gpu_check`'s device line ends with `fma split by the driver, rounded once in float64`; `JANAS_GPU_XFMA=1` takes them on any GPU with float64, `=0` never).
**Whether the GPU computes what the CPU does**: `gpu_check`, in the packages from the release of 24 October 2026, runs every kind of product the GPU takes - each weight type, the rows that end inside a group, several vector counts, the BF16 products - on the GPU Janas would use, all its rows on the GPU and then shared with the CPU as in the engine, and compares each with the CPU's bit for bit, as a card with memory of its own works where it is one. Nothing stops at the first error: the report names the shader of every wrong product, how far its results are (a few units in the last place: the driver rounds differently; many: wrong values), the first wrong row in hexadecimal, and then runs the wrong products again with the other shaders the engine could take and with every other subgroup size the device allows, so that it isolates the culprit by itself. Its last line says `ok` or `FAILED`. Under a minute on most machines. `vkcheck` and `gpu_check` also come alone, without the engine, in `janas-diag-linux-x86_64.tar.gz` and `janas-diag-windows-x86_64.zip` (a few megabytes, no model needed); their whole output is what an issue about the GPU needs.
diff --git a/src/common/env_names.c b/src/common/env_names.c
index 1122f3f..d937373 100644
--- a/src/common/env_names.c
+++ b/src/common/env_names.c
@@ -63,6 +63,7 @@ const char *const JANAS_ENV_NAMES[] = {"JANAS_ALPHAVANTAGE_KEY",
"JANAS_GPU_ARENA_INPLACE",
"JANAS_GPU_EXPERT_MIN",
"JANAS_GPU_EXPERT_SHARE",
+ "JANAS_GPU_FMA_N",
"JANAS_GPU_NO_ROW",
"JANAS_GPU_NOSPIN",
"JANAS_GPU_Q16",
@@ -81,6 +82,7 @@ const char *const JANAS_ENV_NAMES[] = {"JANAS_ALPHAVANTAGE_KEY",
"JANAS_GPU_VERBOSE",
"JANAS_GPU_WHOLE",
"JANAS_GPU_XCOH",
+ "JANAS_GPU_XFMA",
"JANAS_HC_MIX",
"JANAS_HEAD_BITS",
"JANAS_IMAGE_MAX_TOKENS",
diff --git a/src/llm/gpu.c b/src/llm/gpu.c
index 29837d7..813090c 100644
--- a/src/llm/gpu.c
+++ b/src/llm/gpu.c
@@ -258,6 +258,7 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
g->host_import =
has_extension(g, VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME);
g->discrete = type == VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU;
+ g->has_f64 = f2.features.shaderFloat64; /* gpu_fma.c */
if (!g->discrete && !g->host_import)
REJECT("integrated, without VK_EXT_external_memory_host")
/* the subgroup size the shader is built for: 8 where the size can
@@ -409,10 +410,14 @@ static VkResult make_pipeline(struct janas_gpu *g, const uint32_t *code,
VkPipeline *pipe, uint32_t kind, uint32_t nv,
uint32_t sg)
{
+ /* the _xfma variant where the driver splits fma() (gpu_fma.c); made[]
+ keeps code, the name janas_gpu_last_shader looks up */
+ size_t mb = bytes;
+ const uint32_t *mc = janas_g_spv(g, code, &mb);
VkShaderModuleCreateInfo smi = {
.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO,
- .codeSize = bytes,
- .pCode = code};
+ .codeSize = mb,
+ .pCode = mc};
VkResult r = g->vkCreateShaderModule(g->dev, &smi, NULL, sm);
if (r != VK_SUCCESS)
return r;
@@ -577,8 +582,12 @@ struct janas_gpu *janas_gpu_create(char *err, size_t err_len)
.pNext = &f13,
.bufferDeviceAddress = VK_TRUE};
const char *ext[] = {VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME};
+ VkPhysicalDeviceFeatures2 f2 = {
+ .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2,
+ .pNext = &f12,
+ .features = {.shaderFloat64 = g->has_f64 ? VK_TRUE : VK_FALSE}};
VkDeviceCreateInfo dci = {.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO,
- .pNext = &f12,
+ .pNext = &f2,
.queueCreateInfoCount = 1,
.pQueueCreateInfos = &qci,
.enabledExtensionCount = g->host_import ? 1 : 0,
@@ -593,6 +602,26 @@ struct janas_gpu *janas_gpu_create(char *err, size_t err_len)
VK_DEVICE_FUNCS(LOAD_D)
#undef LOAD_D
g->vkGetDeviceQueue(g->dev, family, 0, &g->queue);
+ VkCommandPoolCreateInfo cpci = {
+ .sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO,
+ .flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT,
+ .queueFamilyIndex = family};
+ VkCommandBufferAllocateInfo cba = {
+ .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
+ .level = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
+ .commandBufferCount = 1};
+ VkFenceCreateInfo fci = {.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO};
+ if (g->vkCreateCommandPool(g->dev, &cpci, NULL, &g->cp) != VK_SUCCESS ||
+ (cba.commandPool = g->cp,
+ g->vkAllocateCommandBuffers(g->dev, &cba, &g->cb) != VK_SUCCESS) ||
+ g->vkCreateFence(g->dev, &fci, NULL, &g->fence) != VK_SUCCESS ||
+ janas_g_make_stage(g, &g->xs, X_STAGE_BYTES,
+ !getenv("JANAS_GPU_XCOH")) != 0 ||
+ janas_g_make_stage(g, &g->ys, Y_STAGE_BYTES, 1) != 0 ||
+ janas_g_make_stage(g, &g->ts, 64 << 10, 1) != 0) {
+ snprintf(err, err_len, "cannot create the GPU buffers");
+ goto fail;
+ }
VkPushConstantRange pcr = {.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.size = JANAS_G_PUSH_BYTES};
@@ -601,8 +630,13 @@ struct janas_gpu *janas_gpu_create(char *err, size_t err_len)
.pushConstantRangeCount = 1,
.pPushConstantRanges = &pcr};
if ((r = g->vkCreatePipelineLayout(g->dev, &pli, NULL, &g->pl)) !=
- VK_SUCCESS ||
- (r = janas_g_make_pipeline(g, janas_spv_q4k_matvec,
+ VK_SUCCESS) {
+ set_err(err, err_len, "cannot create the shaders", r);
+ goto fail;
+ }
+ /* before any shader is built: whether they need their _xfma variants */
+ janas_g_fma_probe(g);
+ if ((r = janas_g_make_pipeline(g, janas_spv_q4k_matvec,
sizeof(janas_spv_q4k_matvec), &g->sm,
&g->pipe, 0)) != VK_SUCCESS ||
(r = janas_g_make_pipeline(g, janas_spv_q6k_matvec,
@@ -908,26 +942,6 @@ struct janas_gpu *janas_gpu_create(char *err, size_t err_len)
set_err(err, err_len, "cannot create the shaders", r);
goto fail;
}
- VkCommandPoolCreateInfo cpci = {
- .sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO,
- .flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT,
- .queueFamilyIndex = family};
- VkCommandBufferAllocateInfo cba = {
- .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
- .level = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
- .commandBufferCount = 1};
- VkFenceCreateInfo fci = {.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO};
- if (g->vkCreateCommandPool(g->dev, &cpci, NULL, &g->cp) != VK_SUCCESS ||
- (cba.commandPool = g->cp,
- g->vkAllocateCommandBuffers(g->dev, &cba, &g->cb) != VK_SUCCESS) ||
- g->vkCreateFence(g->dev, &fci, NULL, &g->fence) != VK_SUCCESS ||
- janas_g_make_stage(g, &g->xs, X_STAGE_BYTES,
- !getenv("JANAS_GPU_XCOH")) != 0 ||
- janas_g_make_stage(g, &g->ys, Y_STAGE_BYTES, 1) != 0 ||
- janas_g_make_stage(g, &g->ts, 64 << 10, 1) != 0) {
- snprintf(err, err_len, "cannot create the GPU buffers");
- goto fail;
- }
pthread_mutex_init(&g->qlock, NULL);
const char *ka = getenv("JANAS_GPU_KEEPALIVE_US");
g->keep_us = ka ? (unsigned)atoi(ka) : 500;
diff --git a/src/llm/gpu_fma.c b/src/llm/gpu_fma.c
new file mode 100644
index 0000000..9838c63
--- /dev/null
+++ b/src/llm/gpu_fma.c
@@ -0,0 +1,185 @@
+/* SPDX-License-Identifier: GPL-3.0-or-later
+ Copyright (C) 2026 Maurizio Cammalleri */
+/*
+ * gpu_fma.c - whether the driver rounds fma() once, as the CPU does, and
+ * the shaders' _xfma variants where it does not. RADV splits every 32-bit
+ * fma into a multiply and an add on the AMD GPUs before RDNA2 (Polaris,
+ * Vega, the Renoir APUs, RDNA1: NIR's lower_ffma32, NoContraction or not),
+ * two roundings: products 1 ulp off the CPU's on half the rows, issue #48.
+ * When the GPU opens, fma_probe.comp computes fma() of triples where one
+ * rounding and two differ (c = -a*b rounded: once gives a*b's lost bits,
+ * twice gives 0); if the driver splits it, every shader that calls fma()
+ * is built from its variant (shaders/xfma.glsl: fma rounded once through
+ * float64), provided the variant passes the same probe. JANAS_GPU_XFMA=1
+ * takes the variants anyway, =0 never (both to test them).
+ */
+#include "gpu.h"
+
+#if defined(JANAS_GPU_SPV) && __has_include(<vulkan/vulkan.h>)
+#include "gpu_impl.h"
+
+#include <math.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "fma_probe.h" /* janas_spv_fma_probe, generated by build.sh */
+#include "xfma_table.h" /* janas_xfma_table, generated by build.sh */
+
+enum { N_PROBE = 1024 };
+
+const uint32_t *janas_g_spv(const struct janas_gpu *g, const uint32_t *code,
+ size_t *bytes)
+{
+ if (g->xfma)
+ for (size_t i = 0;
+ i < sizeof(janas_xfma_table) / sizeof(*janas_xfma_table); i++)
+ if (janas_xfma_table[i].code == code) {
+ *bytes = janas_xfma_table[i].bytes;
+ return janas_xfma_table[i].xfma;
+ }
+ return code;
+}
+
+static uint32_t next(uint32_t *s)
+{
+ *s ^= *s << 13;
+ *s ^= *s >> 17;
+ *s ^= *s << 5;
+ return *s;
+}
+
+/* A float of about 1 in magnitude, either sign, all 24 bits used. */
+static float operand(uint32_t *s)
+{
+ uint32_t r = next(s);
+ float m = 1.0f + (float)(r & 0x7fffff) / 8388608.0f;
+ return ldexpf(r >> 31 ? -m : m, (int)(r >> 23 & 7) - 3);
+}
+
+/* How many of the probe's triples the GPU's fma() gets wrong (against the
+ C library's, rounded once), -1 if it could not run. */
+static int probe(struct janas_gpu *g)
+{
+ const char *e = getenv("JANAS_GPU_FMA_N"); /* more triples, to look */
+ int np = e && atoi(e) > N_PROBE ? atoi(e) : N_PROBE;
+ size_t in_bytes = 3 * (size_t)np * sizeof(float),
+ bytes = in_bytes + (size_t)np * sizeof(float);
+ struct stage st = {0};
+ VkShaderModule sm = VK_NULL_HANDLE;
+ VkPipeline pipe = VK_NULL_HANDLE;
+ int wrong = -1;
+ if (janas_g_make_stage(g, &st, bytes, 0) != 0)
+ return -1;
+ if (janas_g_make_pipeline(g, janas_spv_fma_probe,
+ sizeof(janas_spv_fma_probe), &sm, &pipe,
+ 0) != VK_SUCCESS)
+ goto done;
+ float *abc = (float *)st.map, *y = abc + 3 * (size_t)np;
+ uint32_t seed = 0x9e3779b9u;
+ for (int i = 0; i < np; i++) {
+ float a = operand(&seed), b = operand(&seed), c;
+ if (i % 4 == 0) /* c = -(a*b rounded): one rounding and two part */
+ c = -(a * b);
+ else if (i % 4 == 1) /* any c */
+ c = operand(&seed);
+ else if (i % 4 == 2) { /* a product's: scale, integer, sum */
+ a = ldexpf(a, -12);
+ b = (float)(int)(next(&seed) % 2000000) - 1e6f;
+ c = ldexpf(operand(&seed), 8);
+ } else /* a*b just off -c: the sum loses bits */
+ c = -(a * b) * (1.0f + ldexpf((float)(next(&seed) & 255), -22));
+ abc[3 * i] = a;
+ abc[3 * i + 1] = b;
+ abc[3 * i + 2] = c;
+ }
+ memset(y, 0, (size_t)np * sizeof(float));
+ VkCommandBufferBeginInfo bb = {
+ .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
+ .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
+ if (g->vkResetCommandBuffer(g->cb, 0) != VK_SUCCESS ||
+ g->vkBeginCommandBuffer(g->cb, &bb) != VK_SUCCESS)
+ goto done;
+ g->vkCmdBindPipeline(g->cb, VK_PIPELINE_BIND_POINT_COMPUTE, pipe);
+ struct {
+ uint32_t abc[2], y[2];
+ uint32_t n;
+ } pc = {.n = (uint32_t)np};
+ janas_g_push_addr(pc.abc, st.addr);
+ janas_g_push_addr(pc.y, st.addr + in_bytes);
+ g->vkCmdPushConstants(g->cb, g->pl, VK_SHADER_STAGE_COMPUTE_BIT, 0,
+ sizeof(pc), &pc);
+ g->vkCmdDispatch(g->cb, (uint32_t)(np + 63) / 64, 1, 1);
+ VkMemoryBarrier mb = {.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
+ .srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
+ .dstAccessMask = VK_ACCESS_HOST_READ_BIT};
+ g->vkCmdPipelineBarrier(g->cb, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,
+ VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &mb, 0, NULL, 0,
+ NULL);
+ if (g->vkEndCommandBuffer(g->cb) != VK_SUCCESS)
+ goto done;
+ VkSubmitInfo sub = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO,
+ .commandBufferCount = 1,
+ .pCommandBuffers = &g->cb};
+ if (g->vkQueueSubmit(g->queue, 1, &sub, g->fence) != VK_SUCCESS)
+ goto done;
+ VkResult r =
+ g->vkWaitForFences(g->dev, 1, &g->fence, VK_TRUE, 5000000000ull);
+ g->vkResetFences(g->dev, 1, &g->fence);
+ if (r != VK_SUCCESS)
+ goto done;
+ wrong = 0;
+ for (int i = 0; i < np; i++) {
+ float want = fmaf(abc[3 * i], abc[3 * i + 1], abc[3 * i + 2]);
+ if (memcmp(&want, &y[i], sizeof(float)) != 0 && wrong++ < 4 && e)
+ fprintf(stderr, "gpu fma: fma(%a, %a, %a) = %a, the GPU's %a\n",
+ abc[3 * i], abc[3 * i + 1], abc[3 * i + 2], want, y[i]);
+ }
+done:
+ if (pipe)
+ g->vkDestroyPipeline(g->dev, pipe, NULL);
+ if (sm)
+ g->vkDestroyShaderModule(g->dev, sm, NULL);
+ janas_g_free_stage(g, &st);
+ return wrong;
+}
+
+void janas_g_fma_probe(struct janas_gpu *g)
+{
+ const char *e = getenv("JANAS_GPU_XFMA");
+ int forced = e && *e ? atoi(e) > 0 : -1;
+ g->xfma = 0;
+ int plain = forced < 0 ? probe(g) : -1;
+ int verbose = getenv("JANAS_GPU_VERBOSE") != NULL;
+ if (forced == 0 || (forced < 0 && plain <= 0)) {
+ if (verbose)
+ fprintf(stderr, "gpu fma: %s\n",
+ forced == 0 ? "the shaders' own (JANAS_GPU_XFMA=0)"
+ : plain == 0 ? "rounded once"
+ : "probe failed");
+ return;
+ }
+ int with = -1;
+ if (g->has_f64) {
+ g->xfma = 1;
+ with = probe(g);
+ if (with != 0 && forced < 0)
+ g->xfma = 0;
+ }
+ size_t n = strlen(g->about);
+ snprintf(g->about + n, sizeof(g->about) - n, "; fma %s",
+ forced > 0 ? (g->xfma ? "in float64 (JANAS_GPU_XFMA=1)"
+ : "the shaders' own (no float64)")
+ : g->xfma ? "split by the driver, rounded once in float64"
+ : g->has_f64 ? "split by the driver (float64 did not mend it)"
+ : "split by the driver (no float64)");
+ if (verbose || (forced < 0 && !g->xfma))
+ fprintf(stderr,
+ "gpu fma: %d of the probe's triples split by the driver; %s "
+ "(%d wrong with them)\n",
+ plain,
+ g->xfma ? "the shaders' float64 variants" : "the shaders' own",
+ with);
+}
+#endif
diff --git a/src/llm/gpu_impl.h b/src/llm/gpu_impl.h
index b435421..6752407 100644
--- a/src/llm/gpu_impl.h
+++ b/src/llm/gpu_impl.h
@@ -315,6 +315,9 @@ struct janas_gpu {
got wrong whatever the shader, and a line for the user */
int off_types[16], n_off;
char check_note[256];
+ /* gpu_fma.c: shaders with float64 (shaderFloat64), and the _xfma
+ variants taken because this driver splits fma() */
+ int has_f64, xfma;
};
/* The products checked against the CPU's when the GPU opens, and the
@@ -322,6 +325,13 @@ struct janas_gpu {
void janas_g_selfcheck(struct janas_gpu *g);
/* Whether the check left the type out. */
int janas_g_type_off(const struct janas_gpu *g, int type);
+/* Whether this driver rounds fma() once, as the CPU does: if not, the
+ shaders' _xfma variants from then on (g->xfma), when the device has
+ float64 and they pass the same probe (gpu_fma.c). Needs g->cb. */
+void janas_g_fma_probe(struct janas_gpu *g);
+/* The SPIR-V to build for code: its _xfma variant where g->xfma. */
+const uint32_t *janas_g_spv(const struct janas_gpu *g, const uint32_t *code,
+ size_t *bytes);
/* A buffer in the device's own memory (no host mapping), for shaders and
copies; 0, or -1. */
int janas_g_make_local(struct janas_gpu *g, struct stage *s, size_t bytes);
diff --git a/src/llm/gpu_selfcheck.c b/src/llm/gpu_selfcheck.c
index 10787e7..acea65d 100644
--- a/src/llm/gpu_selfcheck.c
+++ b/src/llm/gpu_selfcheck.c
@@ -149,9 +149,10 @@ static int check_one(struct janas_gpu *g, struct janas_pool *pool,
/* The cache's key for this device and driver, and its file. */
static void cache_key(const struct janas_gpu *g, char *key, size_t len)
{
- snprintf(key, len, "%04x:%04x:%08x:sg%u:%s:%s", g->vendor_id, g->device_id,
- g->driver_version, g->sg,
- janas_gpu_shares_host(g) ? "shared" : "copy", JANAS_VERSION);
+ snprintf(key, len, "%04x:%04x:%08x:sg%u:%s:%s%s", g->vendor_id,
+ g->device_id, g->driver_version, g->sg,
+ janas_gpu_shares_host(g) ? "shared" : "copy", JANAS_VERSION,
+ g->xfma ? ":xfma" : "");
}
/* What a saved check says: 1 found (switches and types applied). */
diff --git a/src/llm/shaders/fma_probe.comp b/src/llm/shaders/fma_probe.comp
new file mode 100644
index 0000000..038903c
--- /dev/null
+++ b/src/llm/shaders/fma_probe.comp
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-3.0-or-later
+ Copyright (C) 2026 Maurizio Cammalleri */
+#version 450
+// y[i] = fma(a, b, c) of n triples (gpu_fma.c: whether this driver rounds
+// fma() once, as the CPU does).
+#extension GL_EXT_buffer_reference : require
+#extension GL_EXT_buffer_reference_uvec2 : require
+
+layout(local_size_x = 64) in;
+
+layout(buffer_reference, std430, buffer_reference_align = 4) readonly buffer F { float v[]; };
+layout(buffer_reference, std430, buffer_reference_align = 4) writeonly buffer FO { float v[]; };
+
+layout(push_constant) uniform P { uvec2 abc, y; uint n; } p;
+
+void main()
+{
+ uint i = gl_GlobalInvocationID.x;
+ if (i >= p.n)
+ return;
+ F t = F(p.abc);
+ FO(p.y).v[i] = fma(t.v[3u * i], t.v[3u * i + 1u], t.v[3u * i + 2u]);
+}
diff --git a/src/llm/shaders/xfma.glsl b/src/llm/shaders/xfma.glsl
new file mode 100644
index 0000000..4a915e7
--- /dev/null
+++ b/src/llm/shaders/xfma.glsl
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-3.0-or-later
+ Copyright (C) 2026 Maurizio Cammalleri */
+// fma() rounded once where the driver splits it into a multiply and an add
+// (RADV before RDNA2: NIR lowers every 32-bit ffma, NoContraction or not;
+// issue #48). build.sh puts this after the #extension lines of every
+// shader that calls fma(), as its _xfma variant; gpu_fma.c takes those
+// variants where its probe finds fma() split.
+//
+// a*b is exact in a double; s = a*b + c rounded to a double, and err the
+// part of the sum s lost (TwoSum); s made odd when it lost something
+// (rounding to odd, 53 bits), then rounded to a float: with 53 >= 24 + 2
+// bits that is the sum rounded once (Boldo and Melquiond, 2008).
+
+float janas_xfma(float a, float b, float c)
+{
+ precise double p = double(a) * double(b);
+ precise double dc = double(c);
+ precise double s = p + dc;
+ precise double bv = s - p;
+ precise double err = (p - (s - bv)) + (dc - bv);
+ if (err != 0.0) {
+ uvec2 u = unpackDouble2x32(s);
+ if ((u.x & 1u) == 0u) {
+ // one step towards err: away from zero if err has s's sign
+ if ((err > 0.0) == (s > 0.0))
+ u.x += 1u; // even: never carries
+ else {
+ if (u.x == 0u)
+ u.y -= 1u;
+ u.x -= 1u;
+ }
+ s = packDouble2x32(u);
+ }
+ }
+ return float(s);
+}
+
+#define fma janas_xfma
diff --git a/tools/spv_fma.c b/tools/spv_fma.c
index 0832e2e..fe81a0a 100644
--- a/tools/spv_fma.c
+++ b/tools/spv_fma.c
@@ -14,6 +14,11 @@
* to move a block's scale in one expert in eighty (24 Oct 2026). The
* shaders write fma() where the CPU fuses, so every one of them is meant
* as one operation.
+ *
+ * Every OpFConvert too: undecorated, NIR takes double(float(i)) for
+ * double(i), the integer's float rounding lost (nir_opt_algebraic's
+ * ~f2f64(i2f32(a)) -> i2f64(a)), and the _xfma variants (shaders/xfma.glsl)
+ * came out off the CPU's on Q6_K_P's large sums (30 Oct 2026).
*/
#include <stdint.h>
#include <stdio.h>
@@ -24,6 +29,7 @@ enum {
OP_EXT_INST_IMPORT = 11,
OP_EXT_INST = 12,
OP_DECORATE = 71,
+ OP_FCONVERT = 115,
OP_TYPE_FIRST = 19, /* OpTypeVoid .. OpTypeForwardPointer: after the */
OP_TYPE_LAST = 39, /* annotations, so the new ones go before them */
GLSL_FMA = 50,
@@ -51,8 +57,8 @@ int main(int argc, char **argv)
return 1;
}
fclose(f);
- /* the GLSL.std.450 set, the Fma results, those already decorated, and
- where the types start */
+ /* the GLSL.std.450 set, the Fma and FConvert results, those already
+ decorated, and where the types start */
uint32_t glsl = 0, *fma = malloc(n * 4), nf = 0;
uint8_t *done = calloc(w[3], 1); /* w[3]: the ids' bound */
size_t types = 0;
@@ -67,8 +73,9 @@ int main(int argc, char **argv)
if (op == OP_EXT_INST_IMPORT && wc >= 3 &&
strncmp((const char *)&w[i + 2], "GLSL.std.450", 13) == 0)
glsl = w[i + 1];
- if (op == OP_EXT_INST && wc >= 5 && w[i + 3] == glsl &&
- w[i + 4] == GLSL_FMA)
+ if ((op == OP_EXT_INST && wc >= 5 && w[i + 3] == glsl &&
+ w[i + 4] == GLSL_FMA) ||
+ (op == OP_FCONVERT && wc == 4))
fma[nf++] = w[i + 2];
if (op == OP_DECORATE && wc >= 3 && w[i + 2] == DEC_NO_CONTRACTION &&
w[i + 1] < w[3])
@@ -87,8 +94,8 @@ int main(int argc, char **argv)
return 1;
}
fprintf(o,
- "#pragma once\n/* %s, its %u fma() NoContraction (spv_fma) "
- "*/\nconst uint32_t %s[] = {\n",
+ "#pragma once\n/* %s, its %u fma() and float conversions "
+ "NoContraction (spv_fma) */\nconst uint32_t %s[] = {\n",
argv[1], nf, argv[2]);
size_t col = 0;
#define PUT(x) fprintf(o, "%s0x%08x,", col++ % 8 ? "" : "\n\t", (unsigned)(x))