Skip to content

Distribute and run LLMs with a single file.

13.8 KB
Raw Blame

CLI Arguments and Flags

llamafile accepts two layers of command-line options:

  1. Wrapper flags added by llamafile itself, such as --server, --chat, --cli, and --gpu.
  2. The bundled llama.cpp flags that are passed through to chat, CLI, and server mode.

The examples on this page use llamafile as the executable name. This is the name of the standalone runtime distributed in the releases. A pre-built, model-bundled llamafile normally has a model-specific filename instead; use that filename in the same commands. For example, these commands display the same kind of help:

llamafile --help
./Qwen3.5-0.8B-Q8_0.llamafile --help

The ./ prefix runs a downloaded file from the current directory on macOS, Linux, and BSD. On Windows, use the filename after adding the .exe suffix.

Discovering Available Options

Command-line options are scoped by execution mode rather than collected in a single mode-independent list. The complete user-facing interface is the union of the general and mode-specific help views:

llamafile --help
llamafile --server --help
llamafile --chat --help
llamafile --cli --help

The top-level command is the reference for general llamafile wrapper flags. Each mode-specific command selects the matching llama.cpp parser and adds that mode's wrapper flags:

Command Options shown
llamafile --help Default combined-mode options and general llamafile wrapper flags.
llamafile --server --help HTTP server, API, Web UI, slot, and server sandbox options.
llamafile --chat --help Interactive chat, conversation, and multimodal options.
llamafile --cli --help Single-prompt, output, sampling, and multimodal options.

For fuller descriptions and upstream examples, see the llama.cpp CLI argument reference and server argument reference. Those pages track the latest llama.cpp development version, so an option may differ from the version bundled in a particular llamafile. The help output from that llamafile is authoritative for its accepted options.

An option listed for one mode is not necessarily available in another. For example, --host, --port, and --slot-save-path configure the server, while --nothink is a CLI-mode option.

Since v0.10.4, each llama.cpp option list is generated by the same parser that accepts arguments for that mode. This keeps accepted options and their help entries in sync. Llamafile-specific wrapper options appear in the top-level help or in the relevant mode introduction.

Releases before v0.10.4 printed abbreviated hand-written help for the default, chat, and CLI modes. In those versions, --server --help provides the broadest option list, although it also contains server-only options.

Additional Accepted Parameter Spellings

The --gpu MODE help uses the canonical backend names but does not spell out all accepted compatibility aliases. Values are case-insensitive:

Backend Accepted MODE values
Automatic selection auto
NVIDIA CUDA nvidia, cublas
AMD ROCm/HIP amd, rocm, rocblas, hip
Apple Metal apple, metal
Vulkan vulkan, vk
CPU only disable, disabled

Prefer the first value in each row in scripts. The aliases are retained for compatibility.

Commonly Used Flags

Flag What it does
-t, --threads N Number of CPU threads to use during generation.
-tb, --threads-batch N Number of CPU threads to use during prompt and batch processing.
-c, --ctx-size N Context window size. 0 means to use the model default.
-b, --batch-size N Logical maximum batch size.
-ub, --ubatch-size N Physical maximum batch size.
-lm, --load-mode MODE How the weights are read: auto (default), none, mmap, mlock, mmap+mlock, dio. Replaces --mmap, --no-mmap, --mlock and -dio; see Model Loading Flags.
--repeat-penalty N Penalize repeating tokens during sampling. 1.0 disables the penalty.
-ngl, --gpu-layers, --n-gpu-layers N Number of layers to offload to GPU.
--host HOST Server bind address.
--port PORT Server listen port.

Model Loading Flags

llama.cpp b11100 replaced the separate --mmap, --no-mmap, --mlock, -dio/--direct-io and -ndio/--no-direct-io flags with a single -lm, --load-mode MODE, and no longer accepts the old flags. Use --load-mode in new command lines and .args files, as llama.cpp does.

So that existing llamafiles and .args files keep working, llamafile still accepts the old flags. It prints a warning and replaces them with the equivalent --load-mode, keeping their original meaning: memory mapping is on unless --no-mmap is given, --mlock locks the model in RAM on top of that, and direct I/O takes precedence over memory mapping.

Old flags Equivalent
--mmap --load-mode mmap
--no-mmap --load-mode none
--mlock --load-mode mmap+mlock
--no-mmap --mlock --load-mode mlock
-dio, --direct-io --load-mode dio
-ndio, --no-direct-io --load-mode mmap

Several old flags combine into one mode, applied in command-line order. If you mix them with --load-mode, whichever comes last wins. The old LLAMA_ARG_MMAP, LLAMA_ARG_MLOCK and LLAMA_ARG_DIO environment variables are not translated; use LLAMA_ARG_LOAD_MODE instead.

Wrapper and Mode Flags

  • Mode selection: --server, --chat, --cli
  • General wrapper flags: --gpu MODE, --unsecure, --version, --help
  • Chat-only wrapper flags: --nologo, --ascii
  • CLI-only wrapper flags: --nothink
  • Server-only wrapper flag: --confine-reads

--unsecure disables the pledge/SECCOMP sandbox in modes where that sandbox is active. --confine-reads adds filesystem read confinement and only applies to --server mode.

Model, prompt, GPU-layer, and logging options such as -m, -p, -ngl, and --verbose come from the bundled llama.cpp parser rather than the llamafile wrapper.

Shared Flags Accepted by Chat, CLI, and Server

General, CPU, Context, and Batching

  • General help and shell integration: -h, --help, --usage, --version, -cl, --cache-list, --completion-bash, --verbose-prompt
  • CPU scheduling and affinity: -t, --threads, -tb, --threads-batch, -C, --cpu-mask, -Cr, --cpu-range, --cpu-strict, --prio, --poll, -Cb, --cpu-mask-batch, -Crb, --cpu-range-batch, --cpu-strict-batch, --prio-batch, --poll-batch
  • Context and batching: -c, --ctx-size, -n, --predict, --n-predict, -b, --batch-size, -ub, --ubatch-size, --keep, --swa-full, -fa, --flash-attn
  • Prompt input: -p, --prompt, -f, --file, -bf, --binary-file, -e, --escape, --no-escape
  • RoPE and YaRN: --rope-scaling, --rope-scale, --rope-freq-base, --rope-freq-scale, --yarn-orig-ctx, --yarn-ext-factor, --yarn-attn-factor, --yarn-beta-slow, --yarn-beta-fast

Memory, KV Cache, and Offload

  • KV cache and host memory: -kvo, --kv-offload, -nkvo, --no-kv-offload, --repack, -nr, --no-repack, --no-host
  • Cache storage and defragmentation: -ctk, --cache-type-k, -ctv, --cache-type-v, -dt, --defrag-thold
  • Memory mapping and residency: -lm, --load-mode, -lzm, --lazy-mode, --numa, --check-tensors, --op-offload, --no-op-offload
  • Draft-model cache types: -ctkd, --cache-type-k-draft, -ctvd, --cache-type-v-draft

Devices, GPU, Tensors, and Adapters

  • Devices and tensor placement: -dev, --device, --list-devices, -ot, --override-tensor
  • CPU MoE controls: -cmoe, --cpu-moe, -ncmoe, --n-cpu-moe
  • GPU offload controls: -ngl, --gpu-layers, --n-gpu-layers, -sm, --split-mode, -ts, --tensor-split, -mg, --main-gpu
  • Automatic fitting: -fit, --fit, -fitt, --fit-target, -fitc, --fit-ctx
  • Adapters and metadata overrides: --lora, --lora-scaled, --control-vector, --control-vector-scaled, --control-vector-layer-range, --override-kv

Model Selection, Downloads, and Logging

  • Model path and downloads: -m, --model, -mu, --model-url, -dr, --docker-repo
  • Hugging Face and related model selectors: -hf, -hfr, --hf-repo, -hfd, -hfrd, --hf-repo-draft, -hff, --hf-file, -hfv, -hfrv, --hf-repo-v, -hffv, --hf-file-v, -hft, --hf-token
  • Logging and diagnostics: --log-disable, --log-file, --log-colors, -v, --verbose, --log-verbose, --offline, -lv, --verbosity, --log-verbosity, --log-prefix, --log-timestamps, --perf, --no-perf

Sampling Flags

  • Sampling order and randomness: --samplers, -s, --seed, --sampler-seq, --sampling-seq, --ignore-eos
  • Core sampling controls: --temp, --top-k, --top-p, --min-p, --top-nsigma, --xtc-probability, --xtc-threshold, --typical
  • Repetition controls: --repeat-last-n, --repeat-penalty, --presence-penalty, --frequency-penalty
  • DRY and dynamic temperature: --dry-multiplier, --dry-base, --dry-allowed-length, --dry-penalty-last-n, --dry-sequence-breaker, --dynatemp-range, --dynatemp-exp
  • Mirostat and constrained output: --mirostat, --mirostat-lr, --mirostat-ent, -l, --logit-bias, --grammar, --grammar-file, -j, --json-schema, -jf, --json-schema-file

CLI and Chat-Specific Flags

  • Output and interaction: --display-prompt, --no-display-prompt, -co, --color, --show-timings, --no-show-timings, -cnv, --conversation, -no-cnv, --no-conversation, -st, --single-turn, -mli, --multiline-input, --simple-io
  • System prompt and stopping: -sys, --system-prompt, -sysf, --system-prompt-file, -r, --reverse-prompt, -sp, --special
  • Context and cache management: --ctx-checkpoints, --swa-checkpoints, -cram, --cache-ram, --context-shift, --no-context-shift, --warmup, --no-warmup
  • Parallel decode and multimodal input: -np, --parallel, -mm, --mmproj, -mmu, --mmproj-url, --mmproj-auto, --no-mmproj, --no-mmproj-auto, --mmproj-offload, --no-mmproj-offload, --image, --audio, --image-min-tokens, --image-max-tokens
  • Draft and speculative decoding: -otd, --override-tensor-draft, -cmoed, --cpu-moe-draft, -ncmoed, --n-cpu-moe-draft, --draft, --draft-n, --draft-max, --draft-min, --draft-n-min, --draft-p-min, -cd, --ctx-size-draft, -devd, --device-draft, -ngld, --gpu-layers-draft, --n-gpu-layers-draft, -md, --model-draft, --spec-replace
  • Chat templates and reasoning controls: --chat-template-kwargs, --jinja, --no-jinja, --reasoning-format, --reasoning-budget, --chat-template, --chat-template-file
  • Built-in defaults: --gpt-oss-20b-default, --gpt-oss-120b-default, --vision-gemma-4b-default, --vision-gemma-12b-default

-np, --parallel in CLI and chat mode controls the number of parallel sequences to decode.

Server-Only Flags

  • Server runtime and batching: -kvu, --kv-unified, --spm-infill, --pooling, -np, --parallel, -cb, --cont-batching, -nocb, --no-cont-batching, --threads-http, --cache-reuse
  • Network and API configuration: -a, --alias, --host, --port, --path, --api-prefix, --api-key, --api-key-file, -to, --timeout
  • Web UI and monitoring: --webui-config, --webui-config-file, --webui, --no-webui, --metrics, --props, --slots, --no-slots, --slot-save-path
  • Built-in tools and agent mode: --tools, -ag, --agent, -no-ag, --no-agent, --ui-mcp-proxy, --webui-mcp-proxy, --no-ui-mcp-proxy, --no-webui-mcp-proxy
  • Router and media: --media-path, --models-dir, --models-preset, --models-max, --models-autoload, --no-models-autoload
  • Embeddings, reranking, and TLS: --embedding, --embeddings, --rerank, --reranking, --ssl-key-file, --ssl-cert-file
  • Chat templating and slot behavior: --chat-template-kwargs, --jinja, --no-jinja, --reasoning-format, --reasoning-budget, --chat-template, --chat-template-file, --prefill-assistant, --no-prefill-assistant, -sps, --slot-prompt-similarity, --lora-init-without-apply, --sleep-idle-seconds
  • Draft, speculative decoding, and TTS: -td, --threads-draft, -tbd, --threads-batch-draft, --draft, --draft-n, --draft-max, --draft-min, --draft-n-min, --draft-p-min, -cd, --ctx-size-draft, -devd, --device-draft, -ngld, --gpu-layers-draft, --n-gpu-layers-draft, -md, --model-draft, --spec-replace, -mv, --model-vocoder, --tts-use-guide-tokens
  • Built-in server defaults: --embd-gemma-default, --fim-qwen-1.5b-default, --fim-qwen-3b-default, --fim-qwen-7b-default, --fim-qwen-7b-spec, --fim-qwen-14b-spec, --fim-qwen-30b-default, --gpt-oss-20b-default, --gpt-oss-120b-default, --vision-gemma-4b-default, --vision-gemma-12b-default

-np, --parallel in server mode controls the number of server slots rather than the number of parallel decode sequences.

See Built-in local tools for the tools enabled by --tools, their parameters and limits, and their interaction with llamafile's sandbox.

The Web UI is inherited from llama.cpp and may refer to its upstream server executable as llama-server. In those instructions, use the name of the llamafile executable instead. For example, llama-server --tools all becomes ./ModelName.llamafile --server --tools all for a model-bundled llamafile.

Notes

  • The lists above are a readable index, not a second source of truth. The help output from your executable reflects the exact bundled llama.cpp version.
  • If you are unsure whether a flag is shared, CLI-only, or server-only, check the help for that mode rather than relying on another mode's option list.