37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
C avx2ccpu-inferenceggufinference-enginelinuxllmlocal-llmmixture-of-expertsmoenvmequantizationqwen3simdspeculative-decodingvulkan
/* SPDX-License-Identifier: GPL-3.0-or-later
Copyright (C) 2026 Maurizio Cammalleri */
:root {
--bg: #f7f7f5;
--panel: #ffffff;
--text: #1d1d1f;
--muted: #6b6b70;
--line: #e2e2e0;
--accent: #c2410c;
--accent-text: #ffffff;
--user: #fff1e8;
--code: #f1f1ef;
--error: #b91c1c;
color-scheme: light dark;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #141416;
--panel: #1d1d20;
--text: #ececee;
--muted: #9a9aa2;
--line: #2e2e33;
--accent: #f97316;
--accent-text: #141416;
--user: #2a1d14;
--code: #26262b;
--error: #f87171;
}
}
* { box-sizing: border-box; }
html, body { height: 100%; margin: 0; }
body {
display: flex;
flex-direction: column;
background: var(--bg);
color: var(--text);
font: 16px/1.5 system-ui, -apple-system, "Segoe UI", sans-serif;
}
header, footer {
background: var(--panel);
border-color: var(--line);
padding: 10px 16px;
}
header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
border-bottom: 1px solid var(--line);
}
.brand { display: flex; align-items: center; gap: 12px; min-width: 0; }
.logo {
flex: none;
display: grid;
place-items: center;
width: 36px;
height: 36px;
border-radius: 10px;
background: var(--accent);
color: var(--accent-text);
font-weight: 700;
}
.name { font-weight: 600; }
.desc, .versions, .meta { color: var(--muted); font-size: 13px; }
.desc { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
main {
flex: 1;
overflow-y: auto;
padding: 16px;
display: flex;
flex-direction: column;
gap: 14px;
}
.message { max-width: 760px; width: 100%; margin: 0 auto; }
.message.user {
background: var(--user);
border-radius: 14px;
padding: 10px 14px;
white-space: pre-wrap;
overflow-wrap: anywhere;
}
.message.user.unanswered { opacity: 0.6; }
.answer p { margin: 0 0 0.6em; }
.answer pre {
background: var(--code);
border-radius: 8px;
padding: 10px 12px;
overflow-x: auto;
font-size: 14px;
}
.answer code { font-family: ui-monospace, "Cascadia Code", Menlo, monospace; }
.answer p code { background: var(--code); border-radius: 4px; padding: 0 4px; }
.reasoning { color: var(--muted); font-size: 14px; margin-bottom: 6px; }
.reasoning summary { cursor: pointer; }
.reasoning-text { white-space: pre-wrap; border-left: 2px solid var(--line); padding-left: 10px; }
.meta.error { color: var(--error); }
footer { border-top: 1px solid var(--line); }
.composer { display: flex; gap: 8px; max-width: 760px; margin: 8px auto 4px; }
textarea {
width: 100%;
resize: vertical;
font: inherit;
color: var(--text);
background: var(--bg);
border: 1px solid var(--line);
border-radius: 10px;
padding: 8px 10px;
}
button {
font: inherit;
border: 0;
border-radius: 10px;
padding: 8px 16px;
background: var(--accent);
color: var(--accent-text);
cursor: pointer;
}
button.ghost { background: transparent; color: var(--text); border: 1px solid var(--line); }
button:disabled { opacity: 0.5; cursor: default; }
#settings { max-width: 760px; margin: 0 auto; font-size: 14px; }
#settings summary { cursor: pointer; color: var(--muted); }
#settings label { display: block; margin: 6px 0 2px; }
.row { display: flex; flex-wrap: wrap; gap: 4px 24px; align-items: center; }
.versions { max-width: 760px; margin: 0 auto; }
@media (max-width: 520px) {
.composer { flex-wrap: wrap; }
.composer button { flex: 1; }
header { padding: 8px 12px; }
main { padding: 12px; }
}