Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
DeepSeek settings: Speed (Faster / Full quality), default shown by the engine's own 16 GB rule, passed as --experts 5|6 only when chosen
4e4921f Niko1221 committed 26 days ago
launch: the 16 GiB rule for the Speed label is compared in whole MiB; the catalog's vram_gb is already GiB (nvidia-smi MiB / 1024, AMD bytes / 2**30), so a 16 GB card (16376 MiB) counts as 16 GiB or less; tests with exact MiB values
bd7acc0 Niko1221 committed 26 days ago
launch: DeepSeek Speed choice, Faster (--experts 5, the default on a PC with 16 GB of graphics memory or less) or Full quality (--experts 6): a Speed step in the guided screens, --speed and --full-quality, plain words in --plan and the summary, the quality note; tests, docs
bc1a3ba Niko1221 committed 26 days ago
DeepSeek engine: on a Windows integrated GPU the experts leave room for the context's later GPU allocations
eb4736c Niko1221 committed 26 days ago
metal: commit the queued GPU work before the CPU lane runs (they overlap)
7c77f38 Niko1221 committed 26 days ago
Monitor: with All GPUs, a card that cannot report a value is left out and the card that does report is named (35 C . RTX 5070) instead of blanking the figure
66b43d4 Niko1221 committed 26 days ago
App update: a job an older host left running is finished when the app's program file is newer than the job
91eb05a Niko1221 committed 26 days ago
chat model picker: a downloaded model that is not loaded uses the same two-line row as the API models
4228e6e Niko1221 committed 26 days ago
metal: copies and fills ride the context's command buffer, events mark it, buffers chained by a shared event, decode matvecs for IQ2_XS/IQ2_S/IQ3_XXS
42a982e Niko1221 committed 26 days ago
metal: decode matvecs for Q5_K and IQ2_XXS (every lane busy, 16-byte loads), lane-parallel hc_split for hc 4, DS4_PROF per-kernel GPU table
3e51b1c Niko1221 committed 26 days ago
test: serving never reaches the DeepSeek engine check (only launch and setup import it), and 'serve --plan runs no program' means no installer or fetch: a PC's read-only hardware probes (nvidia-smi, Get-CimInstance) no longer fail it
a3e4cb6 Niko1221 committed 26 days ago
test: launch_engine_first gives the runner a Qwen catalog (a bare Mock answered 'DeepSeek' to the 0.2.4 multi-file question and skipped the engine step); the launch code was right
dc2c422 Niko1221 committed 26 days ago
metal: the IQ2_XXS matvec read its sub-blocks 4 bytes apart (8), IQ2_XS signs, IQ3_XXS matvec added; more dumps and checks
9dbb13e Niko1221 committed 26 days ago
DeepSeek decode: fewer launches a layer - the int8 roundings of one GEMV call in one launch (identical inputs once) and hc_mix + hc_split in one launch (the token's last mix block runs the split); same arithmetic
60d856a Niko1221 committed 26 days ago
Feed reads: one more try when the first times out or the connection fails (the first wait is 12 s, the second the full 30 s); about one github.com request in 25 hung for 30 s and the first-run card said offline
abab458 Niko1221 committed 26 days ago
Load of DeepSeek: the start wait is at most the 600 s Control allows
9cf91f3 Niko1221 committed 26 days ago
Load of DeepSeek: the app adds no Flash-Next options (request slots, RAM budget, output clamp, resource profile, memory retry) to the DeepSeek engine and waits as long as its config says
f30d191 Niko1221 committed 26 days ago
Host: the per-model settings routes accept a dotted id; a test walks a dotted model id through list, load and unload
75ee28f Niko1221 committed 26 days ago
metal: the ring backup flag, decode takes the host-routed path, split commits sync the streams first, sparse_attn sizes its binding without a host read of device data
860e384 Niko1221 committed 26 days ago
Host: run config file names with a dot (DeepSeek) pass the app's own checks, so the model can be loaded
fe2b9f4 Niko1221 committed 26 days ago
test: the own-GGUF check finds tests.terminal when tools/ is run on its own
54768d2 Niko1221 committed 26 days ago
Installed models: an id with a dot (deepseek-v4.1-flash-ist-das-2.00bpw) is listed; the install finished but the app never showed the model
4499c69 Niko1221 committed 26 days ago
test: the Mac build-tools test follows find_tool's version check
beeea0a Niko1221 committed 26 days ago
test: the --kv choices check reads the last line of the error (Python 3.12 and 3.13 print the list differently)
569f7c7 Niko1221 committed 26 days ago
First-run card: one plain line when the drive for the engine is short on space
ff84ff8 Niko1221 committed 26 days ago