Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Add WSL CUDA to GPU section (#105)
Chandler658b18a2edfe
diff --git a/README.md b/README.mdindex 2b4ecd14..5423a542 100644--- a/README.md+++ b/README.md@@ -238,7 +238,10 @@ On Apple Silicon, everything should just work if Xcode is installed.On Windows, GPU should just work so long as (1) you're using our releasebinaries, and (2) you pass the `-ngl 35` flag. You also need an NVIDIA-graphics card that supports CUDA. There's no support yet for AMD GPUs.+graphics card that supports CUDA. There's no support yet for AMD GPUs.+You can also use CUDA via WSL by enabling [Nvidia CUDA on WSL](https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl)+and running your llamafiles inside of WSL. This will also allow you to use+llamafiles greater than 4GB on Windows.On Linux, Nvidia cuBLAS GPU support will be compiled on the fly if (1)you have the `cc` compiler installed, (2) you pass the `-ngl 35` flag to