Skip to content

armgit / janas

Public Imported from GitHub
0

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

Release 2026-09-26: why no GPU is used

camauri791b07d565f8
diff --git a/ChangeLog.md b/ChangeLog.md
index 097eb6f..98c0868 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -2,6 +2,19 @@
> Curated, user-facing summary of completed work, newest first.
+## [2026-09-26] - Why there is no GPU
+
+- **"GPU: none usable" now says why**: built without GPU support
+ (`glslangValidator` missing at build time), no Vulkan loader, Vulkan
+ seeing no GPU, the GPU turned down with its name and what its driver
+ lacks (Vulkan below 1.3, a feature, the subgroup size), or turned off
+ (eco mode, `JANAS_GPU=0`). The same reason is in the model's description
+ (`janas_llm_describe`). A tester's RTX 5080 was reported as "none
+ usable" with nothing more to go on.
+- **`janas-try` reports what Vulkan sees** (each device's name, kind and
+ version), whether Janas will be built with GPU support, and whether it
+ runs under WSL, before building and in the report.
+
## [2026-09-26] - Gemma 4, first part (not yet usable)
- **Gemma 4 (`gemma4`) runs its forward pass**, on Gemma-4-12B-it: layers
diff --git a/INSTALL.md b/INSTALL.md
index 9054b9d..7f33108 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -22,7 +22,7 @@ sudo apt install libvulkan-dev glslang-tools vulkan-tools
On Fedora: `sudo dnf install gcc make` and, for the GPU, `vulkan-loader-devel glslang vulkan-tools`. On Arch: `sudo pacman -S base-devel` and `vulkan-headers vulkan-icd-loader glslang vulkan-tools`.
-The GPU also needs its Vulkan driver: Mesa (`mesa-vulkan-drivers` on Debian/Ubuntu) for Intel and AMD, the proprietary driver for NVIDIA. `vulkaninfo --summary` should list your GPU.
+The GPU also needs its Vulkan driver: Mesa (`mesa-vulkan-drivers` on Debian/Ubuntu) for Intel and AMD, the proprietary driver for NVIDIA. `vulkaninfo --summary` should list your GPU with Vulkan 1.3 or later. When the engine takes none, `janas-bench` and the model's description say why: a build without GPU support (`glslangValidator` was missing: install it and build again), no Vulkan loader, a GPU whose driver lacks something (named), or the GPU turned off.
Then, in the source directory:
diff --git a/src/bench/main.c b/src/bench/main.c
index dcbb61d..734944f 100644
--- a/src/bench/main.c
+++ b/src/bench/main.c
@@ -241,8 +241,11 @@ int main(int argc, char **argv)
char desc[256];
int32_t dl;
janas_llm_describe(llm, desc, sizeof(desc), &dl);
- out("GPU: %s\nmodel: %s\n", gpu ? janas_gpu_name(gpu) : "none usable",
- desc);
+ if (gpu)
+ out("GPU: %s\n", janas_gpu_name(gpu));
+ else
+ out("GPU: none usable: %s\n", janas_llm_model_gpu_why(m));
+ out("model: %s\n", desc);
/* the prompt: the text once, repeated only when more is asked for */
struct janas_tokenizer *tok = janas_llm_internal_tokenizer(llm);
diff --git a/src/llm/api.c b/src/llm/api.c
index ff8c300..edc066b 100644
--- a/src/llm/api.c
+++ b/src/llm/api.c
@@ -566,8 +566,10 @@ int32_t janas_llm_describe(const janas_llm *llm, char *buf, int32_t cap,
* runs. The same goes for the attention, which the context decides.
*/
janas_llm *w = (janas_llm *)llm;
- char gpubuf[96];
- const char *gpu = ", GPU not present";
+ char gpubuf[224];
+ snprintf(gpubuf, sizeof(gpubuf), ", no GPU (%s)",
+ janas_llm_model_gpu_why(llm->m));
+ const char *gpu = gpubuf;
if (janas_llm_model_gpu(llm->m)) {
/* which passes it is given, in the words /mode uses for them */
int use = janas_llm_model_gpu_used(llm->m);
diff --git a/src/llm/gpu.c b/src/llm/gpu.c
index 76f6288..c4a98db 100644
--- a/src/llm/gpu.c
+++ b/src/llm/gpu.c
@@ -290,6 +290,15 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
/* a discrete GPU first (memory and power of its own), then an
integrated one; JANAS_GPU_DEVICE=integrated|discrete restricts */
const char *want_dev = getenv("JANAS_GPU_DEVICE");
+ /* the first device turned down and why, for the message: a tester
+ with a GPU the engine does not take has to be able to tell why */
+ char why[160] = "";
+#define REJECT(what) \
+ { \
+ if (!why[0]) \
+ snprintf(why, sizeof(why), "%.80s: %s", pp.deviceName, what); \
+ continue; \
+ }
for (int pass = 0; pass < 2; pass++)
for (uint32_t i = 0; i < n; i++) {
g->pd = pds[i];
@@ -298,10 +307,12 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
VkPhysicalDeviceType type =
pass == 0 ? VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
: VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU;
- if (pp.deviceType != type || pp.apiVersion < VK_API_VERSION_1_3 ||
+ if (pp.deviceType != type ||
(want_dev &&
strcmp(want_dev, pass == 0 ? "discrete" : "integrated") != 0))
continue;
+ if (pp.apiVersion < VK_API_VERSION_1_3)
+ REJECT("its driver offers Vulkan below 1.3")
VkPhysicalDeviceVulkan13Properties p13 = {
.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_PROPERTIES};
@@ -325,7 +336,7 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
if (!f12.bufferDeviceAddress || !f13.shaderIntegerDotProduct ||
!(p11.subgroupSupportedOperations &
VK_SUBGROUP_FEATURE_BALLOT_BIT))
- continue;
+ REJECT("no buffer addresses, integer dot product or ballot")
/* sharing host memory: needed by an integrated GPU (it reads the
weights where they are), optional for a discrete one (it gets a
copy in its own memory) */
@@ -333,7 +344,7 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
has_extension(g, VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME);
g->discrete = type == VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU;
if (!g->discrete && !g->host_import)
- continue;
+ REJECT("integrated, without VK_EXT_external_memory_host")
/* the subgroup size the shader is built for: 8 where the size can
be chosen (Intel: fewer registers per lane), else the device's
own if it never varies (NVIDIA: 32); JANAS_GPU_SG forces one */
@@ -350,9 +361,9 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
else if (p13.minSubgroupSize == p13.maxSubgroupSize)
sg = p11.subgroupSize;
else
- continue; /* varying and not selectable */
+ REJECT("subgroup size varying and not selectable")
if (sg < 4 || sg > ROWS_PER_GROUP || (sg & (sg - 1)) != 0)
- continue;
+ REJECT("subgroup size not usable")
g->sg = sg;
g->sg_control = control;
uint32_t nf = 0;
@@ -367,8 +378,18 @@ static int pick_device(struct janas_gpu *g, uint32_t *family, char *err,
snprintf(g->name, sizeof(g->name), "%s", pp.deviceName);
return 0;
}
+ REJECT("no compute queue")
}
- snprintf(err, err_len, "no GPU with the needed features");
+#undef REJECT
+ if (why[0])
+ snprintf(err, err_len, "%s", why);
+ else if (n == 0)
+ snprintf(err, err_len, "Vulkan sees no GPU");
+ else if (want_dev)
+ snprintf(err, err_len, "no %.20s GPU, as JANAS_GPU_DEVICE asks",
+ want_dev);
+ else
+ snprintf(err, err_len, "Vulkan sees no integrated or discrete GPU");
return -1;
}
diff --git a/src/llm/model.h b/src/llm/model.h
index 5db2973..06f298e 100644
--- a/src/llm/model.h
+++ b/src/llm/model.h
@@ -268,6 +268,9 @@ int janas_llm_model_set_experts(struct janas_llm_model *m, uint32_t n);
/* The GPU in use, or NULL. */
struct janas_gpu *janas_llm_model_gpu(struct janas_llm_model *m);
+/* Without one, why: the driver's or the build's reason, or that it was
+ turned off ("" when there is a GPU). */
+const char *janas_llm_model_gpu_why(const struct janas_llm_model *m);
/* The expert cache, for statistics and warm-up. */
struct janas_expert_cache *janas_llm_model_cache(struct janas_llm_model *m);
diff --git a/src/llm/model_impl.h b/src/llm/model_impl.h
index fba5200..15a25ea 100644
--- a/src/llm/model_impl.h
+++ b/src/llm/model_impl.h
@@ -74,6 +74,7 @@ struct janas_llm_model {
struct janas_gpu *gpu; /* optional, for products over blocks */
int resident_gpu; /* the resident region belongs to the GPU */
int gpu_off; /* weights not shared with it: CPU only */
+ char gpu_why[160]; /* no GPU: why (janas_llm_model_gpu_why) */
struct janas_expert_cache *cache;
const struct janas_arch *a; /* what this architecture has (arch.h) */
diff --git a/src/llm/model_load.c b/src/llm/model_load.c
index cfaf53d..f9d7989 100644
--- a/src/llm/model_load.c
+++ b/src/llm/model_load.c
@@ -475,9 +475,15 @@ struct janas_llm_model *janas_llm_model_load(const char *path,
/* resident region into RAM; with the GPU, memory from its driver
(JANAS_GPU_ALLOC) or imported at the end */
const char *ge = getenv("JANAS_GPU");
+ snprintf(m->gpu_why, sizeof(m->gpu_why), "%s",
+ ge ? "turned off by JANAS_GPU=0" : "turned off (eco mode)");
if (ge ? atoi(ge) > 0 : o->use_gpu) {
- char gerr[256];
+ /* the reason is kept: a tester with a GPU the engine does not take
+ has to be able to tell a build without GPU support from a driver
+ without Vulkan 1.3 */
+ char gerr[256] = "";
m->gpu = janas_gpu_create(gerr, sizeof(gerr));
+ snprintf(m->gpu_why, sizeof(m->gpu_why), "%.150s", m->gpu ? "" : gerr);
if (m->gpu && getenv("JANAS_GPU_ALLOC") &&
(m->resident = janas_gpu_alloc(m->gpu, m->j.h.resident_bytes)))
m->resident_gpu = 1;
@@ -965,6 +971,11 @@ struct janas_gpu *janas_llm_model_gpu(struct janas_llm_model *m)
return m->gpu;
}
+const char *janas_llm_model_gpu_why(const struct janas_llm_model *m)
+{
+ return m->gpu ? "" : m->gpu_why;
+}
+
uint32_t janas_llm_model_n_ctx(const struct janas_llm_model *m)
{
return m->n_ctx;
diff --git a/tools/janas-try.sh b/tools/janas-try.sh
index 53ce10e..040165d 100755
--- a/tools/janas-try.sh
+++ b/tools/janas-try.sh
@@ -169,6 +169,24 @@ machine()
esac
KERNEL=$(uname -r)
DISTRO=$( . /etc/os-release 2>/dev/null && echo "${PRETTY_NAME:-unknown}")
+ grep -qi microsoft /proc/version 2>/dev/null && DISTRO="$DISTRO (WSL)"
+ # the GPU: whether the build can have it (the shaders need
+ # glslangValidator) and what Vulkan sees - name, kind, version
+ if command -v glslangValidator >/dev/null; then
+ GPU_BUILD="with GPU support"
+ else
+ GPU_BUILD="without GPU support (glslangValidator not installed)"
+ fi
+ if command -v vulkaninfo >/dev/null; then
+ GPU_VK=$(vulkaninfo --summary 2>/dev/null | awk '
+ /apiVersion/ { v = $3 }
+ /deviceType/ { t = $3; sub("PHYSICAL_DEVICE_TYPE_", "", t) }
+ /deviceName/ { sub(/^[^=]*= */, ""); printf "%s%s (%s, Vulkan %s)",
+ n++ ? "; " : "", $0, tolower(t), v }')
+ [ -n "$GPU_VK" ] || GPU_VK="Vulkan sees no device"
+ else
+ GPU_VK="vulkaninfo not installed"
+ fi
CC_VER=$( (${CC:-gcc} --version 2>/dev/null || clang --version 2>/dev/null) |
head -1)
# mains when an adapter (or a USB-C source) is plugged in, battery when
@@ -204,6 +222,7 @@ preflight()
note "memory: $(gib "$MEM_TOTAL") GiB, $(gib "$MEM_AVAIL") GiB free now"
note "disk: $DISK, $(gib "$DISK_FREE") GiB free for the models"
note "system: $DISTRO, Linux $KERNEL, $CC_VER"
+ note "GPU: $GPU_VK; Janas will be built $GPU_BUILD"
note "power: $POWER; load now $LOAD"
case " $FLAGS " in *" avx2 "*) ;; *)
note "no AVX2: Janas runs on its portable path, much slower";;
@@ -450,6 +469,7 @@ report()
echo "- Disk: $DISK"
echo "- System: $DISTRO, Linux $KERNEL"
echo "- Compiler: $CC_VER"
+ echo "- GPU seen by Vulkan: $GPU_VK; Janas built $GPU_BUILD"
echo "- Power: $POWER"
echo "- Load when it started: $LOAD_START over $THREADS threads (the speed lines below say how idle the CPU was before each measurement)"
echo