Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
skill: guardrail llama.cpp update workflow with patch commands (#998)
Davide Eynardc6d052c86829
diff --git a/.llamafile_plugin/.claude-plugin/marketplace.json b/.llamafile_plugin/.claude-plugin/marketplace.jsonindex 145d3bc0..0415c161 100644--- a/.llamafile_plugin/.claude-plugin/marketplace.json+++ b/.llamafile_plugin/.claude-plugin/marketplace.json@@ -9,7 +9,7 @@{"name": "llamafile","description": "Build guidance and commands for the llamafile project",- "version": "0.1.2",+ "version": "0.1.6","author": {"name": "Mozilla AI","email": "[email protected]"diff --git a/.llamafile_plugin/.claude-plugin/plugin.json b/.llamafile_plugin/.claude-plugin/plugin.jsonindex 52f49d78..a4725755 100644--- a/.llamafile_plugin/.claude-plugin/plugin.json+++ b/.llamafile_plugin/.claude-plugin/plugin.json@@ -1,5 +1,5 @@{"name": "llamafile",- "version": "0.1.2",+ "version": "0.1.6","description": "Build guidance and commands for the llamafile project"}diff --git a/docs/commands/generate-patches.md b/docs/commands/generate-patches.mdnew file mode 100644index 00000000..7cd28c67--- /dev/null+++ b/docs/commands/generate-patches.md@@ -0,0 +1,37 @@+---+description: Regenerate llamafile patches from in-place submodule edits+---++# Generate Patches++Capture the current in-place edits of a submodule as patch files, using the+project's `generate_patches.sh` tool. This is the **only** sanctioned way to+produce patches — never hand-craft them with `git diff` (the tool rewrites the+`a/`,`b/` paths to be repo-rooted, strips the volatile `index` line, names files+by convention, and routes new/untracked files into `llamafile-files/`; a raw+`git diff` gets all of that wrong).++Run it for the `llama.cpp` submodule (the common case). The tool is general and+works for any submodule — swap the directory and output path for `whisper.cpp`+or `stable-diffusion.cpp`.++The subshell keeps the working directory restored even if the tool fails, and+`echo y` answers the script's confirmation prompt non-interactively:++```bash+( cd llama.cpp && echo y | ../tools/generate_patches.sh --output-dir ../llama.cpp.patches )+```++Output lands in `llama.cpp.patches/patches/` (modified files) and+`llama.cpp.patches/llamafile-files/` (new files, including `BUILD.mk`).++The tool **only writes/overwrites — it never deletes**. If you dropped a patch+during a bump (the file is no longer modified, e.g. upstream absorbed the+change), its old `.patch` will still be sitting in `patches/` and will keep+being applied by `setup`. `git rm` each dropped patch by hand, and confirm the+final count (`ls llama.cpp.patches/patches | wc -l`) matches your intent.++IMPORTANT: only run this **after** the in-place edits are proven to work+(a clean build succeeds and llamafile runs as expected). Generating patches+from unproven edits bakes in breakage. After generating, verify the patch set+round-trips with `llamafile:verify-clean`.diff --git a/docs/commands/verify-clean.md b/docs/commands/verify-clean.mdnew file mode 100644index 00000000..d9716a94--- /dev/null+++ b/docs/commands/verify-clean.md@@ -0,0 +1,38 @@+---+description: Clean round-trip — reset, re-apply patches, clean build, and test+---++# Verify Clean++Verify the repository from a clean slate: reset, re-pull submodules and+re-apply patches, then do a **clean** build and run the tests. Use this:++- as the final verification after `llamafile:generate-patches` (confirms the+ regenerated patches re-apply cleanly and still build), and+- any time patches or submodules change and you need a trustworthy result.++Why a clean build: after a `reset-repo`/`setup`, submodule sources change+without their timestamps necessarily moving, so an incremental `make` can link+stale objects. Always `make clean` first here — do not rebuild incrementally.++`make setup` both pulls submodules and applies patches, and it cannot pull+submodules onto a dirty tree — that is exactly why `reset-repo` runs first.++```bash+# ensure the toolchain is available+if [ ! -d .cosmocc/4.0.2 ]; then+ build/download-cosmocc.sh .cosmocc/4.0.2 4.0.2 85b8c37a406d862e656ad4ec14be9f6ce474c1b436b9615e91a55208aced3f44+fi+MAKE=.cosmocc/4.0.2/bin/make++$MAKE reset-repo # clean: drop all local changes, reset submodules+$MAKE setup # pull submodules + apply patches (+ fetch UI assets)+$MAKE clean # drop stale build outputs+$MAKE -j$(nproc) # clean build; on mac use -j$(sysctl -n hw.physicalcpu)+$MAKE check # unit tests+```++A successful round-trip proves the committed patch set is internally+consistent. It does NOT cover GPU runtime backends, non-host platforms, the+web UI, or long-run stability — see the handoff checklist in+`update_llamacpp.md`.diff --git a/docs/skills/llamafile/SKILL.md b/docs/skills/llamafile/SKILL.mdindex d1990f5f..000e79c1 100644--- a/docs/skills/llamafile/SKILL.md+++ b/docs/skills/llamafile/SKILL.md@@ -1,7 +1,7 @@---name: llamafiledescription: This skill should be used when the user asks to "build llamafile", "rebuild llamafile", "run llamafile", "run llamafile tests", "debug llamafile", "set up llamafile", "update patches", "fix patch conflict", "update llama.cpp", "pull latest llama.cpp", "sync upstream llama.cpp", "reset submodules", "write a test for llamafile", "how does llamafile work", "llamafile architecture", or needs guidance on the llamafile build system, patch workflow, submodule integration, cosmocc toolchain, or development practices.-version: 0.1.4+version: 0.1.6---# Llamafile Development Guide@@ -48,6 +48,18 @@ make reset-repo # Warning: removes all local changesWARNING: this command removes all local changes. Do not run it without first generating patches from any modifications.+### Patch & Upstream-Update Commands++When changing submodule patches or bumping llama.cpp, use the dedicated commands+rather than ad-hoc `git diff`/`git apply`:++- `llamafile:generate-patches` — regenerate patches from in-place submodule edits+ (the only sanctioned way to produce patches).+- `llamafile:verify-clean` — clean round-trip (`reset-repo` → `setup` → clean+ build → `check`); the verification after generating patches or any patch change.++For the full llama.cpp bump procedure, follow `update_llamacpp.md` step by step.+## Core Workflowsdiff --git a/docs/skills/llamafile/testing.md b/docs/skills/llamafile/testing.mdindex f77d74a5..72d3e9ea 100644--- a/docs/skills/llamafile/testing.md+++ b/docs/skills/llamafile/testing.md@@ -29,22 +29,52 @@ Run a newly compiled llamafile executable this way:./o/llamafile/llamafile --model gguf_model.gguf --server```+#### CLI mode (single prompt, non-interactive)++For a scriptable one-shot generation (e.g. smoke tests), use `--cli`:++```sh+./o/llamafile/llamafile --cli -m gguf_model.gguf -p "What is the capital of France?"+```++The wrapper's flags differ from raw llama.cpp — modes are `--cli`, `--server`,+`--chat`, and the default combined TUI+server. `--cli` requires `-p`; `--gpu`+takes `auto|apple|amd|nvidia|disable`. When unsure, run `--help` (and+`--cli --help` etc.) rather than guessing flags.+#### Verbose modeWhen debugging, the `--verbose` argument is particularly useful as it addsmore verbose logging.-#### Where can I find GGUF model weights files?+#### GPU/Metal: stale per-version runtime cache-Look for available gguf files in `~/llamafiles/`. Depending on the kind of-test, prefer:+llamafile compiles GPU backends at runtime and caches them **keyed by the+version string** under `~/.llamafile/v/<VERSION>/` (on macOS this holds the+extracted `ggml-metal.*` sources and the compiled `ggml-metal.dylib`). The key+is the version alone — not the binary's contents or mtime.-- `gpt-oss-20b-MXFP4.gguf` for agentic tests-- `Ministral-3-3B-Instruct-2512-Q4_K_M.gguf` for multimodal tests-(also look for corresponding `mmproj` projector weights or ask for them)-- `Qwen3-0.6B-Q8_0.gguf` for any other tests+Consequence: if you rebuild after changing GPU/Metal code (e.g. a llama.cpp+bump) **without bumping the llamafile version**, the new binary finds the+existing `v/<VERSION>/` dir and reuses the **stale** dylib instead of+recompiling. That ABI/code mismatch looks exactly like a regression — the+server never reaches `/health`, GPU CLI runs fail — while a `--gpu disable`+(CPU) run works fine, masking the cause.+Before validating GPU/Metal on a dev machine, force a fresh compile:++- bump the llamafile version (preferred — a release does this anyway), or+- `rm -rf ~/.llamafile/v/<VERSION>` to drop the stale cache.++Corollary: a CPU-only smoke test (`--gpu disable`) does **not** exercise this+path. Always run at least one default/GPU invocation when verifying a build on+GPU-capable hardware.++#### Where can I find GGUF model weights files?++Look for available gguf files in `~/ggufs/`. If you don't find any or the+directory is not present, ask the user where you can find them.### Run All Unit Tests@@ -54,12 +84,26 @@ Run `llamafile:check` to run all unit tests from the test suite.### Run Integration Tests```sh+# pre-bundled llamafile./tests/integration/run_tests.sh --executable model_name.llamafile++# direct build: executable + model (+ mmproj for multimodal)+./tests/integration/run_tests.sh \+ --executable ./o/llamafile/llamafile \+ --model ~/path/to/model.gguf \+ --mmproj ~/path/to/mmproj.gguf```- executable can be a pre-bundled llamafile or just the server executable- if running the server executable, `--model` (and `--mmproj` for multimodal models) can be specified too- different tests are run to verify the model/server capabilities+- select categories with `-m` (markers: `cli`, `tui`, `server`, `combined`,+ `multimodal`, `tool_calling`, `thinking`, `gpu`, `cpu`)+- **reasoning models** (e.g. Qwen3.x) need `-m "not thinking"` — otherwise the+ thinking-block assertions derail unrelated tests. Such a model run this way+ is expected to pass the whole suite.+- the suite drives the **default GPU path** by default, so it also exercises+ the Metal/GPU cache caveat above — see "stale per-version cache"- more information and a user manual are available in `tests/integration/README.md`### Run Specific Testdiff --git a/docs/skills/llamafile/update_llamacpp.md b/docs/skills/llamafile/update_llamacpp.mdindex 3feb996f..c2d21de6 100644--- a/docs/skills/llamafile/update_llamacpp.md+++ b/docs/skills/llamafile/update_llamacpp.md@@ -4,108 +4,231 @@ llamafile relies on llama.cpp for many of its functionalities. Keeping it up-to-with the latest version upstream is generally a good practice, as it brings bothbugfixes and support for recent models and features.-This document describes the steps to keep llamafile updated with upstream. At a high-level,-a llama.cpp consists in the following tasks--1. Updating the submodule-2. Verifying and updating the current patches (important: never create patches manually,- but run `../tools/generate-patches.sh --output-dir ../llama.cpp.patches` from the- llama.cpp directory)-3. Updating build dependencies, to make sure new/changed deps are taken into account-4. Updating llamafile code: llamafile is built on top of llama.cpp and you want to- make sure its code still works with the new, updated submodule. NOTE that this also- includes GPU acceleration libs (cuda, rocm, vulkan .sh and .bat scripts in the- `llamafile/` directory)--## Step 1: Update the submodule--The output of this step is a new branch with the submodule checked out-at its latest commit id.+This document is the canonical procedure for a llama.cpp bump. It is built by+**composing small, single-purpose tools** — do not improvise around them.++## Tools and their one job each++| Tool | One job | Run from |+|------|---------|----------|+| `make reset-repo` | Clean slate: drop all local changes, reset submodules | repo root |+| `make setup` | Pull submodules **and** apply patches (+ fetch UI assets). Needs a clean tree (can't pull onto a dirty one). Doubles as the patch-**application** test. | repo root |+| `tools/check_patches.sh` | Triage **only**: of the pre-existing patches, which still apply to the freshly-bumped submodule (line-number fuzz tolerated) and which need hand-work. | repo root |+| `llamafile:generate-patches` | Regenerate **all** patches from in-place submodule edits. The only sanctioned way to produce patches. | (wraps the `cd`) |+| `llamafile:verify-clean` | Clean round-trip: `reset-repo` → `setup` → clean build → `check`. The post-generate verification. | repo root |+| `llamafile:build` / `llamafile:check` | Build all targets / run unit tests. | repo root |++### DO / DON'T++- **DON'T** craft or edit patches with `git diff` / `git apply`. Patch *production*+ is `llamafile:generate-patches`; patch *triage* is `check_patches.sh`; patch+ *round-trip verification* is `llamafile:verify-clean`.+- **DON'T** hand-roll `for p in patches/*.patch; do git apply --check ...` loops.+ That is what `check_patches.sh` (forward, pre-edit) and `verify-clean` (full+ round-trip, post-generate) already do.+- **DON'T** rebuild incrementally after a reset/setup — always clean build+ (`verify-clean` does this). Stale objects link silently otherwise.+- **DON'T** run `generate-patches` until the in-place edits are *proven* (clean+ build succeeds and llamafile runs as expected).+- **DO** use `git diff $OLD_ID..$COMMIT_ID` for **upstream-drift recon only**+ (seeing what changed upstream to drive BUILD.mk / integration work). That is+ the one legitimate ad-hoc `git diff` use.+- **DO** use `git apply --reject <patch>` to *reconcile* a drifted patch (Step 3):+ it applies every hunk that still fits and drops only the stragglers as `.rej`+ files, so a 50-hunk mechanical patch (e.g. the `GGML_CALL` ones) collapses to+ hand-editing the 1–2 hunks that actually moved. This is distinct from — and+ doesn't violate — the DON'Ts above: `check_patches.sh` triages (file-level,+ pre-edit), `generate-patches` produces, and `--reject` is just a precise way to+ *apply* during reconciliation. Delete the `.rej` files once reconciled (they'd+ otherwise be picked up as untracked files).++## The procedure++### Step 1 — Bump the submodule++Creates a new branch with the submodule at its latest commit. The tree is now+clean and patches are **not** yet applied (fresh upstream code).```bash-# make sure the submodule is initializedgit submodule update --init llama.cpp-# check current commitcd llama.cppOLD_ID=`git rev-parse HEAD`--# checkout latest commitgit fetch origin masterCOMMIT_ID=`git rev-parse origin/master`git checkout origin/master--# create new branch for mergingcd ..+git checkout -b llamacpp_$COMMIT_IDgit add llama.cppgit commit -m "Update llama.cpp submodule to $COMMIT_ID"--# this branch becomes the starting point of a new PR```-## Step 2: Verify and update patches--Review the patches in `llama.cpp.patches/patches/` as follows:--- As a first pass, run `tools/check_patches.sh` to check if applying any of the-patches causes an error. Directly apply all and only the patches you see working.--- Any patch that has conflicts due to upstream changes has to be inspected-in detail and updated. Useful references are:- - the file the patch refers to- - the patch description in `llama.cpp.patches/README.md`--- To update patches that have conflicts, first edit the new llama.cpp code-in-place, then call the `generate_patches` script (more info in `development.md`).--At the end of this step, your patches should all work (i.e. it should be possible-to apply them without conflicts). Note that you might still not have a working build,-but you should at least be able to run `make setup` without any errors.--## Step 3: Update BUILD.mk dependencies--- Review `llama.cpp/BUILD.mk` for any new source files or dependencies added upstream-- Remove references to any deleted source files-- Ensure all new dependencies are properly included-- Check the upstream changes for new/removed files in `llama.cpp/src/`, `llama.cpp/common/`, `llama.cpp/ggml/`, `llama.cpp/tools`, (all the relevant subdirectories you'd find in `llama.cpp/BUILD.mk`)--Useful references:--- check changes in each dir-```bash-cd llama.cpp-git diff --stat --summary $OLD_ID -- src/-```--- the `llama.cpp/CMakeLists.txt` file, showing what files are included in the latest llama.cpp build--At the end of this step, the `llama.cpp/BUILD.mk` file should include all the-updated dependencies to build, at least, the `o//llama.cpp/server/llama-server`-target.--## Step 4: Update llamafile integration code--- Check if the llamafile code that calls llama.cpp server/main needs updates-- Review `llamafile/` for any API changes in llama.cpp that need to be reflected-- Pay attention to changes in `llama.cpp/include/` for API modifications-- Also check GPU acceleration libraries code (cuda, rocm, vulkan .sh and .bat- scripts in the `llamafile/` directory)--At the end of this step, you should be able to build all targets in this repo,-i.e. the following verification step should return a successful result--## Verification--After making changes, verify the build works:-```sh-# llamafile:clean-# llamafile:build-```--+Keep `$OLD_ID` and `$COMMIT_ID` — you need them for drift recon below.++### Step 2 — Triage the existing patches++Run `tools/check_patches.sh` from the repo root. It reports, for each existing+patch, whether it still applies to the bumped submodule. This is *triage only*:+it tells you which patches are free and which need reconciliation. A patch may+be accepted despite line shifts — that fuzz is fine and welcome.++A patch that **fails** triage has three possible fates, not one — decide which+before editing:+- **Reconcile** — upstream moved the code; reproduce the patch's intent against+ the new source (the common case).+- **Drop as obsolete** — upstream **absorbed** the change, so the patch is now+ redundant (this bump: upstream added the `<algorithm>` include `ngram-mod`+ patched in, and replaced the async shader-compile the Vulkan patch was+ rewriting). Applying it would duplicate/conflict. Don't reconcile it — delete+ it (see Step 5 for the cleanup gotcha).+- **Split** — part still applies, part is obsolete; keep the live hunks, drop the+ rest (`git apply --reject` makes this visible).++### Step 3 — Reconcile (edit llama.cpp in place)++Make the submodule build and work against the new upstream, editing files+**in place** (never editing patch files):++- Apply the patches that triage showed as clean; for the conflicting ones,+ hand-edit the new llama.cpp code to reproduce each patch's intent (use+ `git apply --reject` to isolate just the drifted hunks — see DO/DON'T).+- **BUILD.mk source lists:** add new upstream sources, drop deleted ones, fix+ renames. Drive this from drift recon:+ `cd llama.cpp && git diff --stat --summary $OLD_ID..$COMMIT_ID -- src/ common/ ggml/ tools/`+ and cross-check against `llama.cpp/CMakeLists.txt` (the `src/models/` and+ `tools/mtmd/models/` dirs are globbed there, so *every* new `.cpp` is a real+ TU). A new source usually has to be listed in **more than one place** — see the+ "keep-in-sync" list under "Recurring breakage". Missing one fails as a+ link-time `undefined reference`, often only at `verify-clean`, not as a compile+ error.+- **llamafile integration:** reconcile any upstream API change that llamafile's+ **own** code calls — these live outside the patch set (in `llamafile/`) and+ surface as *compile errors at build*, not in triage. The usual suspects are the+ `chatbot_*` files and the server bridge; this bump, `mtmd_helper_bitmap_init_*`+ gained a `placeholder` arg and changed return type, breaking+ `chatbot_eval.cpp`/`chatbot_cli.cpp`. When the build errors in a `llamafile/`+ file, grep `llamafile/` for the changed symbol.+- **Buildable dirty tree:** `make setup` does more than apply patches — it copies+ `llamafile-files/` into the submodule (`BUILD.mk`, `common/license.cpp`),+ removes the upstream `Makefile`, and fetches the UI. If you reconcile by hand+ (applying patches directly rather than via `setup`, e.g. because some patches+ don't yet apply), replicate those steps or the build fails on a missing+ `BUILD.mk`/`license.cpp`. Also: a full `make` needs **all three** submodules+ initialized (`reset-repo` deinits whisper/sd); to prove *just* llama.cpp,+ `make o/$(MODE)/llama.cpp`.++### Step 4 — Prove the reconciliation works (on the dirty tree)++Before generating any patches, prove the in-place edits actually work:++- Clean build (`llamafile:clean` then `llamafile:build`) and `llamafile:check`.+- Run llamafile as expected — ideally the integration tests. Runtime / GPU /+ platform validation is your hardware's job (see handoff checklist).++Only proceed past this gate once the build is green and llamafile runs. Patches+generated from unproven edits bake in breakage.++### Step 5 — Regenerate the patches++Run `llamafile:generate-patches`. It rewrites the full patch set from your+proven in-place edits (refreshing line numbers — this resilience is welcome).+New/untracked files (including `BUILD.mk`) are routed to `llamafile-files/`.++**Gotcha — it only writes, never deletes.** A patch you dropped as obsolete+(Step 2) has no corresponding edit, so `generate-patches` simply doesn't+regenerate it — but the **old `.patch` file lingers** and keeps getting applied+by `setup`. Manually `git rm llama.cpp.patches/patches/<dropped>.patch` for each+one. Sanity-check: `ls llama.cpp.patches/patches | wc -l` should equal what you+expect (old count − dropped + added).++Update `llama.cpp.patches/README.md` for any patch added/removed/materially+reworked.++### Step 6 — Verify by clean round-trip++Run `llamafile:verify-clean`. `reset-repo` → `setup` re-applies the **new**+patches onto a clean tree (erroring if any patch is broken), then a clean build+and `check`. A green round-trip proves the committed patch set is internally+consistent.++## Host verification is necessary, not sufficient++`verify-clean` only exercises the CPU build on the host. It does **not** cover+the things that broke in every past bump. Hand these off for testing on real+hardware/platforms and report them in the PR:++- [ ] CUDA / ROCm smoke test on a Linux GPU box+- [ ] Windows smoke test (incl. GPU DSO extraction — see "Permission denied" class)+- [ ] macOS Metal runtime compile + run — first bump the llamafile version or+ `rm -rf ~/.llamafile/v/<VERSION>`, else you test a **stale cached Metal+ dylib** from before the bump (looks like a Metal regression; see+ testing.md "stale per-version cache")+- [ ] Web UI serves — **host-checkable, do it in-session**: `llama-server` starts+ in *router mode with no model*, so launch it and `curl` the routes. `GET /`+ → 200 `index.html`; the hashed bundles under `/_app/immutable/...` (read the+ paths out of `index.html` — they're content-hashed, not `/bundle.js`) → 200.+ With a gzip build (`llama_ui_use_gzip()`), send `Accept-Encoding: gzip` or+ assets return **415**, and expect `Content-Encoding: gzip` on the response.+- [ ] Long-run stability (the `cv.wait` / futex class only shows after hours)++## Recurring breakage (check these proactively)++Distilled from PRs #941, #951, #983 — these recur almost every bump and are the+bulk of the manual follow-up. Check them in Step 3/4 instead of waiting for them+to surface during your testing:++- **Keep-in-sync coupling points** — a new/renamed source often must be declared+ in several places that don't validate each other. When you touch BUILD.mk+ source lists, walk this whole registry:+ - **`llamafile/BUILD.mk`** — the TUI binary (`o/$(MODE)/llamafile/llamafile`)+ relinks an *explicit* server-object list (`LLAMAFILE_SERVER_SUPPORT_OBJS`)+ that is **separate** from llama.cpp's `TOOL_SERVER_SRCS`. A new+ `tools/server/*` (or mtmd) source must be added to **both**; missing the+ second is a link-time `undefined reference` that only shows at the+ `verify-clean` link step, not as a compile error (this bump:+ `server-schema.cpp`).+ - **GPU runtime build scripts** — the host build never exercises them (GPU DSOs+ compile at runtime on the target). The cuda/rocm scripts collect sources by+ glob (`collect_gpu_sources` over `ggml-cuda/*.cu`) and `vulkan.sh` globs+ `*.comp`, so *top-level* sources are auto-picked — but verify the glob still+ covers any new **subdir** / non-globbed path, and mirror anything explicit+ into `build-functions.sh`, `cuda.sh`/`.bat`, `rocm.sh`/`.bat`,+ `vulkan.sh`/`.bat`, and the Metal runtime-compile bundle.+ - **`fetch-ui-assets.sh` ↔ `tools/ui/embed.cpp`** — fetch's required-asset+ check must mirror embed.cpp's `required_check[]` (see Web UI below).+- **Web UI shipping** changes upstream most cycles, but the *structure* is+ durable: the embed + serve halves are **upstream** files — `tools/ui/embed.cpp`+ and `tools/server/server-http.cpp` — which llamafile patches **neither** (they+ usually already handle the new format). Only **two llamafile glue pieces** ever+ need touching: `fetch-ui-assets.sh` (download/verify/extract the assets) and+ the UI block in `llamafile-files/BUILD.mk` (invoke `embed`). Each bump, re-read+ embed.cpp's CLI and its `required_check[]` and make fetch match — the interface+ drifts: b9747 changed `embed` from `<name> <path>` pairs to+ `<out_cpp> <out_h> [<asset_dir>]` (recursive over a dir), and moved the HF+ bucket from 4 flat files to a SvelteKit `dist.tar.gz` of hashed+ `_app/immutable/*`. fetch now extracts the tarball and builds a `dist/_gzip/`+ mirror so embed emits gzip-encoded assets (binary growth ~5MB vs ~17MB raw).+ Symptom of a missed bump: `fetch-ui-assets.sh` 404s and the server builds+ UI-less (degrades gracefully — easy to not notice).+- **TinyBLAS vs ggml quant block formats / cuBLAS API** (`QK*` block sizes,+ strided-batched gemm). Diff ggml quant headers and cuBLAS call sites.+- **`GGML_CALL` annotations** on any new/renamed backend callback (the+ meta-backend / vulkan callback breakage).+- **Untimed `cv.wait()`** added upstream on server threads → needs the+ `wait_for(30s)` loop + widened sigmask treatment (the futex/EINTR family,+ hit in both #941 and #983). Only reproduces after a long run — your bug, not+ the build's.+- **chat-template / reasoning refactors** → re-check llamafile's+ `--reasoning`/thinking wiring in `chatbot_*`.++When a runtime bug can't be reproduced in-session (GPU / long-run / platform),+state hypotheses *as* hypotheses, don't turn guessed numbers into evidence, and+propose a measurement/instrumentation step before a fix.## Reference- **Upstream changes:** https://github.com/ggerganov/llama.cpp/compare/$OLD_ID...$COMMIT_ID-- **Example PR with similar updates:** https://github.com/mozilla-ai/llamafile/pull/847+- **Example PRs:** [#941](https://github.com/mozilla-ai/llamafile/pull/941),+ [#951](https://github.com/mozilla-ai/llamafile/pull/951),+ [#983](https://github.com/mozilla-ai/llamafile/pull/983)