Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Update sever README build/testing instructions (#461)
Vicki Boykis9cd8d70942a0
diff --git a/llama.cpp/server/README.md b/llama.cpp/server/README.mdold mode 100644new mode 100755index 1c92a204..6a233345--- a/llama.cpp/server/README.md+++ b/llama.cpp/server/README.md@@ -34,7 +34,7 @@ Command line options:- `--grp-attn-w`: Set the group attention width to extend context size through self-extend(default: 512), used together with group attention factor `--grp-attn-n`## Build-server is build alongside everything else from the root of the project+The server is vendored from `llama.cpp` and built at the root of the `llamafile` as part of the compilation of `llama.cpp/main/main`. To build it, compile the project at root.- Using `make`:@@ -48,6 +48,14 @@ server is build alongside everything else from the root of the projectcmake --build . --config Release```+Then, to test your compiled artifacts, run the `--server` CLI command at the root of the repo where you pointed your compiled artifacts to from your root `sudo make install` command.++Example:++```+$PREFIX/bin/llamafile --server -m /mistral-7b-instruct-v0.1.Q4_K_M.gguf+```+## Quick StartTo get started right away, run the following command, making sure to use the correct path for the model you have: