Skip to content

armgit / janas

Public Imported from GitHub
0

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

Release 2026-10-11 (7): Q3_K products of a prompt on the GPU; the engine named Janas-LLM throughout

camauri60a56d6029a6
diff --git a/ChangeLog.md b/ChangeLog.md
index e8eb1f8..671c9bd 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -4,6 +4,8 @@
## [2026-10-11] - A binary package, and the models already converted
+- **Q3_K weights in a prompt's products on the GPU** (the fast prompt's tiles): as the CPU's to seven digits, 420-635 GMAC/s against 267 on twelve threads. A mixture's Q3_K experts stay on the CPU, where they are faster: on unsloth's 3-bit Qwen3-30B-A3B, which still reads a prompt slower than llama.cpp (102 tok/s against 112).
+- **Measured on 11 October 2026**: Qwen3-Coder-Next reads 66 tok/s and writes 22 where llama.cpp manages 14 and 2.2; Qwen3.6-35B-A3B reads at 104 tok/s against 79. The README's tables give the operating system of the test machine (Debian 13, kernel 6.12.107, Mesa 26.1.6), and the engine is called Janas-LLM where the engine is meant.
- **A binary package for Linux x86-64**, built by GitHub Actions on Ubuntu 22.04 and attached to each release: `janas-linux-x86_64.tar.gz`, at a link that always gives the newest (`releases/latest/download/janas-linux-x86_64.tar.gz`). The programs and the converters, Janas's services in `services/` with their pages, the libraries and their headers, the SHA-256 of every file; no script. It runs with the C library of Ubuntu 22.04, Debian 12 and anything newer, the GPU included. `janas-chat` finds the services in `services/` beside it.
- **The converted models are a promise**: the `.jns` files on Hugging Face (format version 2) will be read by every later version of Janas.
- **`janas-get` downloads the `.jns` files themselves**, from the project's repositories on Hugging Face (`prabanta-dev/<model>-JNS`): no conversion on your machine, half the room on disk, every file checked against the same fingerprints of MODELS.md. `--from-gguf` keeps the old way (the original GGUF, converted here), and `janas-get` takes it by itself where a converted file is not on Hugging Face; `janas-get hf:<owner>/<repo>/<file.jns>` takes any `.jns`. `janas-get list` shows the size of the converted files.
diff --git a/README.md b/README.md
index 4d15097..2239d13 100644
--- a/README.md
+++ b/README.md
@@ -65,7 +65,7 @@ Designed and written by **Maurizio "camauri" Cammalleri**.
| | |
|---|---|
-| **Stage** | Early development. The public API and the command line **change without notice**, and have changed several times a week. The models' files are the exception: the `.jns` files published on Hugging Face (format version 2) will be read by every later version of Janas. |
+| **Stage** | Early development. The public API and the command line **change without notice**, and have changed several times a week. The models' files are the exception: the `.jns` files published on Hugging Face (format version 2) will be read by every later version of Janas-LLM. |
| **Stability** | **Not verified.** The engine is deliberately hard on the machine: it fills the free memory with its expert cache and reads the disk at full speed with `O_DIRECT`. On a machine with less headroom than it reckons, the desktop can be pushed into swap and become slow or unresponsive. Using a GPU has, once, caused a driver reset that took the desktop with it. |
| **Data** | It writes only in `~/.cache/janas` (what it learns about your machine, and the system messages it has read, so as not to read them again) and where you tell it to put a converted model. It does not touch the model files it reads. |
| **Hardware** | Developed and run on **one** machine (below). On anything else it is untested: it may be slower, it may refuse the model, it may misbehave. |
@@ -134,7 +134,7 @@ The last column is a draft guessing the next tokens and the model checking them,
The comparison with llama.cpp at the top of this page was measured so: llama.cpp (build `2b18470` of 18 September 2026) with its Vulkan backend on the same integrated GPU, everything on it (`llama-bench -p 1024 -n 16 -ngl 99 -t 12`; the largest model otherwise, below), against `janas-bench --prompt 1024 --gen 16` on the same files the same days (9-11 October 2026; llama.cpp's best configuration, Janas-LLM's own choice; on 11 October three rounds each, `llama-bench -r 3` and `janas-bench --rounds 3`, for Qwen3-0.6B, Qwen3.5-0.8B, Qwen3.6-35B-A3B and the models from other hands; Qwen3-Coder-Next as Qwen3-Next-80B, below).
-On Qwen3-Next-80B-A3B, whose file is larger than this laptop's 30 GB, llama.cpp with its usual build ran out of memory as soon as the GPU took part, even with the experts left on the CPU (`-ngl 99 -ncmoe 48`): its CPU backend copies the weights into memory of its own to repack them, and the Vulkan backend copies those it leaves to the CPU into buffers the system cannot reclaim. It ran with a build without repacking (`GGML_CPU_REPACK=OFF`) and two options more, `--no-host 1 -lm mmap`, which leave the experts in the file, read through the page cache: 16 tok/s reading with the GPU, and 2.8 writing on the CPU alone (with the GPU, 2.0); these are its two figures in the table. Janas ran the same model with no option of its own, the experts it uses most in a cache of about 19 GB and the rest read from the disk. Janas's figures are one round each; the writing column is without drafts on both sides. The two tools do not measure in exactly the same way (`llama-bench` writes without a prompt before it; `janas-bench` after the 1,024 tokens), and it is one machine. llama.cpp's own CPU backend did worse on this laptop than its Vulkan one at reading (Qwen3-4B 70 tok/s on 20 threads) and as well at writing.
+On Qwen3-Next-80B-A3B, whose file is larger than this laptop's 30 GB, llama.cpp with its usual build ran out of memory as soon as the GPU took part, even with the experts left on the CPU (`-ngl 99 -ncmoe 48`): its CPU backend copies the weights into memory of its own to repack them, and the Vulkan backend copies those it leaves to the CPU into buffers the system cannot reclaim. It ran with a build without repacking (`GGML_CPU_REPACK=OFF`) and two options more, `--no-host 1 -lm mmap`, which leave the experts in the file, read through the page cache: 16 tok/s reading with the GPU, and 2.8 writing on the CPU alone (with the GPU, 2.0); these are its two figures in the table. Janas-LLM ran the same model with no option of its own, the experts it uses most in a cache of about 19 GB and the rest read from the disk. Janas-LLM's figures are one round each; the writing column is without drafts on both sides. The two tools do not measure in exactly the same way (`llama-bench` writes without a prompt before it; `janas-bench` after the 1,024 tokens), and it is one machine. llama.cpp's own CPU backend did worse on this laptop than its Vulkan one at reading (Qwen3-4B 70 tok/s on 20 threads) and as well at writing.
**Try it on an idle machine first.** The expert cache takes the memory that is free when the model is opened, and what is left of the model is read from disk while it answers, so everything else running takes its share. It degrades gently rather than breaking: measured with a browser and an editor open, the same model kept about three quarters of its speed and waited five times longer on the disk. But the first thing you see should be the machine's real speed, and any measurement you mean to send to others has to be taken with the machine to itself.
@@ -172,7 +172,7 @@ bin/x86_64-linux/janas-chat ~/.local/share/janas/models/gemma-4-e4b-it-q4km.jns
A download stopped halfway resumes where it stopped, and a file already there is checked rather than fetched again. `--from-gguf` takes the other way: the original GGUF the catalog names, downloaded and converted on your machine to the very same fingerprints (the GGUF goes once converted, `--keep` keeps it); `janas-get` takes it by itself where a converted file is not on Hugging Face. Any other file: `janas-get hf:<owner>/<repo>/<file.jns|file.gguf>`, checked against the SHA-256 Hugging Face lists for it. The memory column of `janas-get list` is the smallest machine each model is meant for. The rest of this section is the same by hand.
-Janas reads its own format, `.jns`, converted from a **Q4_K_M** GGUF (Q3_K, IQ3_S, IQ4_NL, IQ4_XS, Q4_0, Q4_1 and Q5_1 convert too, and so do unsloth's "UD" files made of them; IQ2 and IQ1 types, not yet). Keep both in `models/` next to the sources — the tools take a path, so anywhere works, but that is where these examples put them, and where the project keeps its own.
+Janas-LLM reads its own format, `.jns`, converted from a **Q4_K_M** GGUF (Q3_K, IQ3_S, IQ4_NL, IQ4_XS, Q4_0, Q4_1 and Q5_1 convert too, and so do unsloth's "UD" files made of them; IQ2 and IQ1 types, not yet). Keep both in `models/` next to the sources — the tools take a path, so anywhere works, but that is where these examples put them, and where the project keeps its own.
| Model | GGUF converted here | GGUF | `.jns` | Memory it likes |
|---|---|---|---|---|
@@ -572,9 +572,9 @@ curl http://127.0.0.1:8080/v1/chat/completions \
-d '{"messages": [{"role": "user", "content": "What is the capital of Sardinia?"}]}'
```
-A client that lets you set the address of an OpenAI-compatible API should need nothing else: the base URL is `http://127.0.0.1:8080/v1`, and the key whatever you gave `--api-key` (anything, if you gave none). It has been tried here with `curl`, its own test page, the `openai` Python SDK and Open WebUI 0.11.4 (driven through its own API: streamed chat, titles, tags, follow-up suggestions, and a document indexed with Janas's embeddings and asked about), and by a user with AnythingLLM.
+A client that lets you set the address of an OpenAI-compatible API should need nothing else: the base URL is `http://127.0.0.1:8080/v1`, and the key whatever you gave `--api-key` (anything, if you gave none). It has been tried here with `curl`, its own test page, the `openai` Python SDK and Open WebUI 0.11.4 (driven through its own API: streamed chat, titles, tags, follow-up suggestions, and a document indexed with Janas-LLM's embeddings and asked about), and by a user with AnythingLLM.
-**Models that reason, and the limits a client sets.** Qwen3 and its successors reason before they answer, unless told not to. OpenAI's `max_completion_tokens` (and `max_output_tokens` in the Responses API) counts the reasoning too, and Janas takes it so. The older `max_tokens` of a chat, which OpenAI does not take at all for the models that reason, is taken for the answer alone: a client that sends it means the answer, and counted with the reasoning a small limit was spent before a word of it was written (Open WebUI asks for a chat's title with 1000, and got none).
+**Models that reason, and the limits a client sets.** Qwen3 and its successors reason before they answer, unless told not to. OpenAI's `max_completion_tokens` (and `max_output_tokens` in the Responses API) counts the reasoning too, and `janas-server` takes it so. The older `max_tokens` of a chat, which OpenAI does not take at all for the models that reason, is taken for the answer alone: a client that sends it means the answer, and counted with the reasoning a small limit was spent before a word of it was written (Open WebUI asks for a chat's title with 1000, and got none).
**With Open WebUI**, start the server with `--think off`: the titles, tags and suggestions it asks for after every reply are then quick, and a model that reasons can still be asked to (`reasoning_effort`). Its built-in tools, when they are on, add about 5,000 tokens to every prompt; the server reads them once and keeps them computed, but the first reply of a conversation waits for them.
diff --git a/src/llm/gpu.c b/src/llm/gpu.c
index e450c61..98e20b7 100644
--- a/src/llm/gpu.c
+++ b/src/llm/gpu.c
@@ -29,6 +29,7 @@
#include "q4k_mmq.h" /* janas_spv_q4k_mmq */
#include "q6kp_mmq.h" /* janas_spv_q6kp_mmq */
#include "q6k_mmq.h" /* janas_spv_q6k_mmq */
+#include "q3k_mmq.h" /* janas_spv_q3k_mmq */
#include "q80_mmq.h" /* janas_spv_q80_mmq */
#include "q5k_mmq.h" /* janas_spv_q5k_mmq */
#include "dense_mm.h" /* janas_spv_dense_mm */
@@ -518,6 +519,9 @@ struct janas_gpu *janas_gpu_create(char *err, size_t err_len)
(r = make_pipeline(g, janas_spv_q6k_mmq, sizeof(janas_spv_q6k_mmq),
&g->sm_mm6s, &g->pipe_mm6s, 0, NV_PER_GROUP,
g->sg)) != VK_SUCCESS ||
+ (r = make_pipeline(g, janas_spv_q3k_mmq, sizeof(janas_spv_q3k_mmq),
+ &g->sm_mm3, &g->pipe_mm3, 0, NV_PER_GROUP, g->sg)) !=
+ VK_SUCCESS ||
(r = make_pipeline(g, janas_spv_q80_mmq, sizeof(janas_spv_q80_mmq),
&g->sm_mm80, &g->pipe_mm80, 0, NV_PER_GROUP,
g->sg)) != VK_SUCCESS ||
@@ -779,6 +783,8 @@ void janas_gpu_destroy(struct janas_gpu *g)
g->vkDestroyPipeline(g->dev, g->pipe_mm6, NULL);
if (g->pipe_mm6s)
g->vkDestroyPipeline(g->dev, g->pipe_mm6s, NULL);
+ if (g->pipe_mm3)
+ g->vkDestroyPipeline(g->dev, g->pipe_mm3, NULL);
if (g->pipe_mm80)
g->vkDestroyPipeline(g->dev, g->pipe_mm80, NULL);
if (g->pipe_mm5)
@@ -803,6 +809,8 @@ void janas_gpu_destroy(struct janas_gpu *g)
}
if (g->sm_mm6s)
g->vkDestroyShaderModule(g->dev, g->sm_mm6s, NULL);
+ if (g->sm_mm3)
+ g->vkDestroyShaderModule(g->dev, g->sm_mm3, NULL);
if (g->sm_mm80)
g->vkDestroyShaderModule(g->dev, g->sm_mm80, NULL);
if (g->sm_mm6)
@@ -1405,6 +1413,7 @@ static int fast_type(const struct janas_matvec_task *t)
{
return t->type == JANAS_Q4_K || t->type == JANAS_Q6_K ||
t->type == JANAS_Q8_0 || t->type == JANAS_Q5_K ||
+ t->type == JANAS_Q3_K ||
(t->type == JANAS_Q6_K_P && plane_level(t) == 3);
}
@@ -1472,6 +1481,11 @@ int janas_g_record_task(struct janas_gpu *g, VkPipeline *bound,
want = g->pipe_mm6s;
nvg = 64;
rows_wg = 64;
+ } else if (t->type == JANAS_Q3_K && nv >= JANAS_GPU_FAST_N && g->pipe_mm3 &&
+ mm_on()) {
+ want = g->pipe_mm3;
+ nvg = 64;
+ rows_wg = 64;
} else if (t->type == JANAS_Q5_K && nv >= JANAS_GPU_FAST_N && g->pipe_mm5 &&
mm_on()) {
want = g->pipe_mm5;
diff --git a/src/llm/gpu_impl.h b/src/llm/gpu_impl.h
index ee0ec44..2dc0b4e 100644
--- a/src/llm/gpu_impl.h
+++ b/src/llm/gpu_impl.h
@@ -149,8 +149,9 @@ struct janas_gpu {
VkPipeline pipe32[4];
VkPipeline pipex4, pipe3, pipei3; /* IQ4_XS, Q3_K, IQ3_S */
/* Q4_K on many vectors in tiles (q4k_mm.comp), MM_ROWS x MM_VECS */
- VkShaderModule sm_mm, sm_mm6, sm_mm6s, sm_mm80, sm_mm5, sm_dm[2];
- VkPipeline pipe_mm, pipe_mm6, pipe_mm6s, pipe_mm80, pipe_mm5, pipe_dm[2];
+ VkShaderModule sm_mm, sm_mm6, sm_mm6s, sm_mm80, sm_mm5, sm_mm3, sm_dm[2];
+ VkPipeline pipe_mm, pipe_mm6, pipe_mm6s, pipe_mm80, pipe_mm5, pipe_mm3,
+ pipe_dm[2];
int only_fast; /* janas_gpu_set_only_fast */
/* a mixture's experts in one dispatch (experts_id): Q4_K, Q6_K_P */
VkShaderModule sm_id4, sm_id6, sm_id6s;
diff --git a/src/llm/shaders/q3k_mmq.comp b/src/llm/shaders/q3k_mmq.comp
new file mode 100644
index 0000000..efc0d57
--- /dev/null
+++ b/src/llm/shaders/q3k_mmq.comp
@@ -0,0 +1,145 @@
+/* SPDX-License-Identifier: GPL-3.0-or-later
+ Copyright (C) 2026 Maurizio Cammalleri */
+#version 450
+// The fast prompt for Q3_K: the tiling of q4k_mmq.comp, as q6k_mmq.comp.
+// A group of 32 weights is two groups of 16 with a 6-bit scale each (less
+// 32); a weight's three bits are two of qs and one of hmask (quant.h), put
+// together unsigned (0-7) from the 110-byte blocks read a word at a time
+// and realigned; the -4 comes off with the activations' sums of 16. NOT bit
+// for bit with the CPU: rounded group by group.
+#extension GL_EXT_control_flow_attributes : require
+#extension GL_EXT_integer_dot_product : require
+#extension GL_EXT_buffer_reference : require
+#extension GL_EXT_buffer_reference_uvec2 : require
+
+layout(local_size_x = 128) in;
+layout(constant_id = 0) const uint NV = 8; // unused: the table's constants
+layout(constant_id = 1) const uint SG = 8;
+
+const uint BM = 64u, BN = 64u, KS = 4u;
+const uint WM = 32u, WN = 32u, TM = 2u, TN = 2u, WARP = 32u;
+const uint WNITER = (WM * WN) / (WARP * TM * TN); // 8
+const uint WSUBN = WN / WNITER; // 4
+const uint XB = 20u;
+
+layout(buffer_reference, std430, buffer_reference_align = 16) readonly buffer U4 { uvec4 v[]; };
+layout(buffer_reference, std430, buffer_reference_align = 4) readonly buffer U1 { uint v[]; };
+layout(buffer_reference, std430, buffer_reference_align = 4) writeonly buffer F { float v[]; };
+
+layout(push_constant) uniform P { uvec2 w, x, y; uint rows, nb, nv, xs, ys; } p;
+
+struct ACache { uint qs[8]; vec2 dsc; }; // a group: three-bit values, d * scale of each 16
+struct BCache { int qs[8]; vec2 d8s; float d8; }; // quants, 4 x the sums of 16, d8
+shared ACache sa[BM * KS];
+shared BCache sb[BN * KS];
+
+// word i (0-27) of the block at byte off from w, realigned
+uint hword(U1 W, uint off, uint i)
+{
+ uint a = (off >> 2) + i;
+ return (off & 2u) != 0u ? (W.v[a] >> 16) | (W.v[a + 1u] << 16) : W.v[a];
+}
+
+void main()
+{
+ U4 X = U4(p.x);
+ U1 W = U1(p.w);
+ F Y = F(p.y);
+ uint id = gl_LocalInvocationID.x;
+ uint r0 = gl_WorkGroupID.x * BM, v0 = gl_WorkGroupID.y * BN;
+ uint warp = id / WARP, tiw = id % WARP;
+ uint tiwr = tiw % (WM / TM), tiwc = tiw / (WM / TM);
+ uint warp_r = warp % (BM / WM), warp_c = warp / (BM / WM);
+ float sums[WNITER * TN * TM];
+ [[unroll]] for (uint i = 0; i < WNITER * TN * TM; i++) sums[i] = 0.0;
+ uint ngroups = p.nb * 8u;
+ for (uint g0 = 0; g0 < ngroups; g0 += KS) {
+ // weights: 4 invocations a row, 2 words each of the group's 8
+ {
+ uint iqs = id % 4u, rl = id / 4u;
+ [[unroll]] for (uint l = 0; l < BM; l += 32u) {
+ uint r = min(r0 + rl + l, p.rows - 1u);
+ // the 4 groups staged (j = 0-3 of one n) share their words
+ // of qs (two bits each, shifted by 2j) and of hmask (bit
+ // g = 4n + j): each read once
+ uint b = g0 / 8u, gb = g0 % 8u, n = gb / 4u; // gb: in the block
+ uint off = (r * p.nb + b) * 110u;
+ [[unroll]] for (uint t = 0; t < 2u; t++) {
+ uint kw = 2u * iqs + t;
+ uint q = hword(W, off, 8u + 8u * n + kw), hm = hword(W, off, kw);
+ [[unroll]] for (uint k = 0; k < KS; k++)
+ sa[k * BM + rl + l].qs[kw] = ((q >> (2u * k)) & 0x03030303u) |
+ (((hm >> (gb + k)) & 0x01010101u) << 2);
+ }
+ if (iqs == 0u) {
+ float d = unpackHalf2x16(hword(W, off, 27u) & 0xffffu).x;
+ uint s0 = hword(W, off, 24u), s1 = hword(W, off, 25u), s2 = hword(W, off, 26u);
+ [[unroll]] for (uint k = 0; k < KS; k++) {
+ // scales 2g and 2g + 1: the low four bits from byte
+ // i % 8 (its low nibble below 8, high above), the top
+ // two from byte 8 + i % 4 at bits 2 (i / 4)
+ float sc[2];
+ [[unroll]] for (uint h = 0; h < 2u; h++) {
+ uint i = 2u * (gb + k) + h, lb = i % 8u;
+ uint lo = ((lb < 4u ? s0 : s1) >> (8u * (lb % 4u))) & 0xffu;
+ lo = i < 8u ? lo & 15u : lo >> 4;
+ uint hi = (s2 >> (8u * (i % 4u) + 2u * (i / 4u))) & 3u;
+ sc[h] = float(int(lo | (hi << 4)) - 32);
+ }
+ sa[k * BM + rl + l].dsc = vec2(d * sc[0], d * sc[1]);
+ }
+ }
+ }
+ }
+ // activations: 2 invocations a vector, each half a group's quants
+ {
+ uint iqs = id % 2u, c = id / 2u;
+ uint v = min(v0 + c, p.nv - 1u);
+ [[unroll]] for (uint k = 0; k < KS; k++) {
+ uint g = g0 + k, b = g / 8u, G = g % 8u;
+ uint xo = (v * p.xs + b) * XB;
+ uvec4 q = X.v[xo + 2u * G + iqs];
+ sb[k * BN + c].qs[4u * iqs] = int(q.x);
+ sb[k * BN + c].qs[4u * iqs + 1u] = int(q.y);
+ sb[k * BN + c].qs[4u * iqs + 2u] = int(q.z);
+ sb[k * BN + c].qs[4u * iqs + 3u] = int(q.w);
+ if (iqs == 0u) {
+ uvec4 bs = X.v[xo + 16u + G / 4u];
+ uint bw = G % 4u == 0u ? bs.x : G % 4u == 1u ? bs.y : G % 4u == 2u ? bs.z : bs.w;
+ sb[k * BN + c].d8s = vec2(4.0 * float(bitfieldExtract(int(bw), 0, 16)),
+ 4.0 * float(bitfieldExtract(int(bw), 16, 16)));
+ sb[k * BN + c].d8 = uintBitsToFloat(X.v[xo + 18u].x);
+ }
+ }
+ }
+ barrier();
+ for (uint k = 0; k < KS; k++) {
+ ACache ca[TM];
+ [[unroll]] for (uint cr = 0; cr < TM; cr++)
+ ca[cr] = sa[k * BM + warp_r * WM + tiwr * TM + cr];
+ [[unroll]] for (uint wsic = 0; wsic < WNITER; wsic++)
+ [[unroll]] for (uint cc = 0; cc < TN; cc++) {
+ BCache cb = sb[k * BN + warp_c * WN + wsic * WSUBN + tiwc * TN + cc];
+ [[unroll]] for (uint cr = 0; cr < TM; cr++) {
+ int q0 = 0, q1 = 0;
+ [[unroll]] for (uint i = 0; i < 4u; i++) {
+ q0 += dotPacked4x8EXT(int(ca[cr].qs[i]), cb.qs[i]);
+ q1 += dotPacked4x8EXT(int(ca[cr].qs[i + 4u]), cb.qs[i + 4u]);
+ }
+ sums[(wsic * TN + cc) * TM + cr] +=
+ cb.d8 * (ca[cr].dsc.x * (float(q0) - cb.d8s.x) +
+ ca[cr].dsc.y * (float(q1) - cb.d8s.y));
+ }
+ }
+ }
+ barrier();
+ }
+ [[unroll]] for (uint wsic = 0; wsic < WNITER; wsic++)
+ [[unroll]] for (uint cc = 0; cc < TN; cc++)
+ [[unroll]] for (uint cr = 0; cr < TM; cr++) {
+ uint row = r0 + warp_r * WM + tiwr * TM + cr;
+ uint v = v0 + warp_c * WN + wsic * WSUBN + tiwc * TN + cc;
+ if (row < p.rows && v < p.nv)
+ Y.v[v * p.ys + row] = sums[(wsic * TN + cc) * TM + cr];
+ }
+}
diff --git a/tests/bench_gemm.c b/tests/bench_gemm.c
index 8086f98..1bbd9f6 100644
--- a/tests/bench_gemm.c
+++ b/tests/bench_gemm.c
@@ -8,7 +8,7 @@
* all of them, so beyond a few tokens the product is bound by arithmetic and
* the figure of merit is multiply-accumulates per second.
*
- * Usage: bench_gemm [rows cols q4k|q6k|q6kp1|q6kp2|q6kp3 threads [gpu]]
+ * Usage: bench_gemm [rows cols q4k|q6k|q3k|q6kp1|q6kp2|q6kp3 threads [gpu]]
* q6kpL is Q6_K in bit planes with L of the three planes in memory.
* Defaults: 12288 x 2048 Q4_K (Qwen3-Next ssm_in), 12 threads on CPUs 0-11.
* gpu: the products of a prompt's blocks on the GPU (the fast prompt's
@@ -40,6 +40,8 @@ int main(int argc, char **argv)
int type = JANAS_Q4_K;
if (!strcmp(kind, "q6k"))
type = JANAS_Q6_K;
+ else if (!strcmp(kind, "q3k"))
+ type = JANAS_Q3_K;
else if (!strncmp(kind, "q6kp", 4)) {
type = JANAS_Q6_K_P;
level = kind[4] ? kind[4] - '0' : 3;
@@ -80,6 +82,8 @@ int main(int argc, char **argv)
((struct janas_block_q6kp *)w)[b].d = janas_fp32_to_fp16(0.001f);
} else if (type == JANAS_Q6_K) {
((struct janas_block_q6k *)w)[b].d = janas_fp32_to_fp16(0.001f);
+ } else if (type == JANAS_Q3_K) {
+ ((struct janas_block_q3k *)w)[b].d = janas_fp32_to_fp16(0.001f);
} else {
((struct janas_block_q4k *)w)[b].d = janas_fp32_to_fp16(0.001f);
((struct janas_block_q4k *)w)[b].dmin = janas_fp32_to_fp16(0.0005f);