Skip to content

armgit / janas

Public Imported from GitHub
0

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

Release 2026-10-09: Qwen3-30B-A3B in unsloth's three-bit mix, smaller and closer to the original

camauri4b73e1da62cd
diff --git a/ChangeLog.md b/ChangeLog.md
index f55a6d8..9e2a917 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -2,6 +2,10 @@
> Curated, user-facing summary of completed work, newest first.
+## [2026-10-09] - Qwen3-30B-A3B in three bits
+
+- **`qwen3-30b-a3b-q3`: the same model in unsloth's dynamic three-bit mix, smaller and closer to the original.** Measured against Qwen's own Q8_0 (400 tokens of llama.cpp's logits), unsloth's UD-Q3_K_XL agrees on the most likely token more often than Qwen's Q4_K_M that the catalog already had: 386 against 376 on code, 360 against 335 on Italian prose, with 27% fewer bytes of experts (12.8 GB against 17.6). With little memory it is also faster: with the expert cache held at 2 GiB, 19.8 tokens/s against 13.6. Held whole in memory it is 5% slower, since its Q3_K product is slower than Q4_K's. `qwen3-30b-a3b` is unchanged. The same trade does not pay on Qwen3.6-35B-A3B, whose Q4_K_M is already made with an importance matrix: bartowski's Q3_K_M lost 26 positions of 400 on Italian for 2-12% more speed, so it is not in the catalog.
+
## [2026-10-08] - Light experts not in memory left out, not the lightest of all
- **With a small expert cache, only what would cost a read is given up.** When the cache holds less than a fifth of a model's experts, the engine used to drop the lightest experts of every token, six of ten or eight, whether they were in memory or not. Now it keeps every expert in memory and leaves out only the light ones it would have to read from disk (below 0.8/k of a token's weight, 0.95/k under a tenth), the other experts taking their share. Qwen3-Next-80B against llama.cpp, 400 tokens: with a 2 GiB cache 26.4 tokens/s instead of 20.9 at about the same agreement (380 against 383 most likely tokens); with 4 GiB the same speed (27.7) with a mean logit difference of 0.36 instead of 0.53; on Italian prose with 8 GiB 372 tokens agreeing instead of 361, and faster (23.7 against 21.4 tokens/s). On Qwen3-30B-A3B and Qwen3.6-35B-A3B with 1.5-3.5 GiB of cache it was 9-37% faster than the old way on code and on Italian prose, as close to llama.cpp or closer in most cases. `/experts n` asks for an exact number again and turns it off; `JANAS_MISS_SKIP` sets the weight for measures.
diff --git a/MODELS.md b/MODELS.md
index 5df4380..3f45211 100644
--- a/MODELS.md
+++ b/MODELS.md
@@ -113,6 +113,13 @@ e40ec6c58ef2390c7a4ce8c9b7b6b384ed4e55c9980f646d1c98e83e2c018115 qwen3.5-9b-q4k
3fa0185b7eecd43e222d420f68aae28a86b3b90392fd7e4052b5cd337318e895 qwen3-30b-a3b-q4km.jns (jns_planes)
```
+**Qwen3-30B-A3B in three bits** (`qwen3-30b-a3b-q3`), from [`unsloth/Qwen3-30B-A3B-GGUF`](https://huggingface.co/unsloth/Qwen3-30B-A3B-GGUF), unsloth's dynamic UD-Q3_K_XL. Converted in one step: its down matrices are Q3_K and Q4_K, so there are no planes to cut.
+
+```
+f1b8111d3e49a0238fcaa77ec9ee24eb9805266b5e85bf8e54c12dedb02cf4ff Qwen3-30B-A3B-UD-Q3_K_XL.gguf
+7a77e62a3d6b1df321f455472e0b970a09eb3534ec76982a6e169087d4897656 qwen3-30b-a3b-udq3kxl.jns (gguf2jns)
+```
+
**Qwen3-Embedding-0.6B**, from [`Qwen/Qwen3-Embedding-0.6B-GGUF`](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF), for `janas-server --embedding-model`. It is published in Q8_0 and converted as it is, in one step: its matrices are not Q6_K, so there are no planes to cut. Two conversions give the same bytes.
```
diff --git a/README.md b/README.md
index 1f53d9d..bf35a6b 100644
--- a/README.md
+++ b/README.md
@@ -134,6 +134,7 @@ Janas reads its own format, `.jns`, converted from a **Q4_K_M** GGUF (Q3_K, IQ3_
| **Qwen3-Coder-Next** | [Qwen](https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF) | 48.4 GB | 48.4 GB | 32 GB |
| **Qwen3-Coder-30B-A3B-Instruct** — for code completion in the editor ([docs/code-completion.md](docs/code-completion.md)) | [unsloth](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF) | 18.6 GB | 18.6 GB | 32 GB, to keep it whole in memory |
| **Qwen3-30B-A3B** | [Qwen](https://huggingface.co/Qwen/Qwen3-30B-A3B-GGUF) | 18.6 GB | 18.6 GB | 16 GB |
+| **Qwen3-30B-A3B in three bits** (`qwen3-30b-a3b-q3`) — unsloth's dynamic UD-Q3_K_XL: against Qwen's own Q8_0 it agrees more often than the Q4_K_M above (400 tokens: code 386 against 376, Italian 360 against 335), with 27% fewer bytes of experts; with the expert cache held at 2 GiB on this 32 GB machine, 19.8 tokens/s against 13.6; held whole in memory, 5% slower (its Q3_K product is slower than Q4_K's) | [unsloth](https://huggingface.co/unsloth/Qwen3-30B-A3B-GGUF) | 13.8 GB | 13.8 GB | 8-16 GB |
| **Qwen3.5-9B** — dense; its prediction block from the checkpoint ([MODELS.md](MODELS.md#the-fingerprints)) | [unsloth](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) | 5.7 GB | 5.7 GB | 16 GB |
| **Qwen3.5-2B** — dense, small and quick; a prediction block too | [unsloth](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF) | 1.3 GB | 1.3 GB | 8 GB |
| **Qwen3-4B** — dense, and the smallest here that answers well | [Qwen](https://huggingface.co/Qwen/Qwen3-4B-GGUF) | 2.5 GB | 2.5 GB | 8 GB |
diff --git a/src/common/catalog.c b/src/common/catalog.c
index e3f4eee..b6ca211 100644
--- a/src/common/catalog.c
+++ b/src/common/catalog.c
@@ -114,6 +114,23 @@ const struct janas_model_entry janas_catalog[] = {
"3fa0185b7eecd43e222d420f68aae28a86b3b90392fd7e4052b5cd337318e895",
.ram_gb = 16,
.fast = "--head 4 --attention fast --bits 4"},
+ /* the same model in unsloth's dynamic three-bit mix, quantized with an
+ importance matrix: nearer Qwen's own Q8_0 than Qwen's Q4_K_M (400
+ tokens, most likely token agreeing: code 386 against 376, Italian
+ 360 against 335) with 27% fewer bytes of experts; with a cache of
+ 2 GiB 19.8 tokens/s against 13.6, held whole in memory 5% slower
+ (9 Oct 2026). No fast profile measured yet. */
+ {.name = "qwen3-30b-a3b-q3",
+ .what = "Qwen3-30B-A3B, unsloth's 3-bit mix: smaller, as good",
+ .repo = "unsloth/Qwen3-30B-A3B-GGUF",
+ .parts = {"Qwen3-30B-A3B-UD-Q3_K_XL.gguf"},
+ .part_sha =
+ {"f1b8111d3e49a0238fcaa77ec9ee24eb9805266b5e85bf8e54c12dedb02cf4ff"},
+ .gguf_gb = 13.8,
+ .out = "qwen3-30b-a3b-udq3kxl.jns",
+ .flat_sha =
+ "7a77e62a3d6b1df321f455472e0b970a09eb3534ec76982a6e169087d4897656",
+ .ram_gb = 8},
{.name = "qwen3.6-35b-a3b",
.what = "Qwen3.6-35B-A3B, mixture of experts, MTP inside",
.repo = "bartowski/Qwen_Qwen3.6-35B-A3B-GGUF",