Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Recognize cache_prompt parameter in OpenAI API
Georgi Gerganov8cb9fd8479fc
diff --git a/llama.cpp/README.llamafile b/llama.cpp/README.llamafileindex 9510b544..bf6ab964 100644--- a/llama.cpp/README.llamafile+++ b/llama.cpp/README.llamafile@@ -20,8 +20,8 @@ ORIGINallocation, scheduling, backends, etc. So far, we've curated up to:https://github.com/ggerganov/llama.cpp- 8d6d9f033b8101f929e445cf45b39e1557ca7934- Fri Dec 1 11:41:56 2023 +0200+ fe680e3d1080a765e5d3150ffd7bab189742898d+ Thu Dec 7 22:26:54 2023 +0200See the llamafile git history re: cherry-picks for further details.diff --git a/llama.cpp/server/server.cpp b/llama.cpp/server/server.cppindex a81b498b..172e4a26 100644--- a/llama.cpp/server/server.cpp+++ b/llama.cpp/server/server.cpp@@ -2380,6 +2380,7 @@ json oaicompat_completion_params_parse(// Map OpenAI parameters to llama.cpp parametersllama_params["prompt"] = format_chatml(body["messages"]); // OpenAI 'messages' to llama.cpp 'prompt'+ llama_params["cache_prompt"] = json_value(body, "cache_prompt", false);llama_params["temperature"] = json_value(body, "temperature", 0.8);llama_params["top_k"] = json_value(body, "top_k", 40);llama_params["top_p"] = json_value(body, "top_p", 0.95);