Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Clarify Linux GPU offload diagnostics (#967)
adminlip18db09f5ca08
diff --git a/docs/support.md b/docs/support.mdindex 28fc4199..e783212a 100644--- a/docs/support.md+++ b/docs/support.md@@ -72,6 +72,13 @@ benefit of letting you run llamafiles greater than 4GB on Windows.On Linux, NVIDIA users will need to install the CUDA SDK (ideally usingthe shell script installer) and ROCm users need to install the HIP SDK.They're detected by looking to see if `nvcc` or `hipcc` are on the PATH.+For AMD systems, make sure the executable directory containing `hipcc` is+on your `PATH` and that it can be executed by your user; a `hipcc:+Permission denied` message means ROCm was found but can't be run, so GPU+offload will not be available until the SDK permissions or installation+are fixed. Running with `--gpu amd` or `--gpu nvidia` is a useful way to+turn an otherwise quiet CPU fallback into an explicit startup error while+you diagnose the toolchain.If you have both an AMD GPU *and* an NVIDIA GPU in your machine, thenyou may need to qualify which one you want used, by passing either