Skip to content

armgit / PS5LM

Public Imported from GitHub
0

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

llama.cpp: b11827, one patch fewer to carry (#39)

Mert Cobanov8a2da9164a68
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 650b8c4..93b1f68 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -50,6 +50,12 @@ v0.1.x are the upstream payload releases by cobanov.
`ps5lm-app-fleet` zip any more: every PS5LM can be a fleet worker, and stays the normal app
unless `/data/PS5LM/fleet-worker.txt` asks for one, as before. `PS5LM_FLEET` is gone from
`scripts/build-app.sh`, and CI builds one title again.
+- llama.cpp moves to b11827 (970b51ce7, 2026-10-22), 219 commits on, among them ggml-vulkan's
+ fixes for GPU-visible host memory (#30468) and concurrent first device init (#29766), its strided
+ f16 KV copy on AMD (#27703), and upstream's own `tojson` floats as Python writes them (#30362,
+ #30483), so `patches/0006-jinja-tojson-floats-as-python.patch` is gone; `patches/0005` is
+ re-based on the new `server-http.cpp`, and `patches/0009` (introspection) on the new
+ `server-context.cpp`. It builds and links for the console as before.
- The planner counts the compute buffers' growth with the prompt batch (`-ub`, about 38 bytes a
token per unit of feed-forward width, measured with ggml-vulkan on the console's shader paths),
and the batch for language models is one constant, `kPromptUbatch`, still 64 until
diff --git a/docs/FLEET.md b/docs/FLEET.md
index d38dde5..953a771 100644
--- a/docs/FLEET.md
+++ b/docs/FLEET.md
@@ -118,7 +118,7 @@ memory. IP validation is not authentication.
## Revisions and build
Developed on PS5LM `c4595784ce75c1d58777b11ef812d3e087e74296`, with llama.cpp
-`04f7226a2c12c7f382c545a6db5a034266aa53cb` and UI
+`970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` and UI
`bf44d80384d5b8f8f4f213cf98d748887211328d`. Build host and workers from the same
checkout and patch set. The host manifest checks the local binary checksum and
llama revision. A remote handshake checks **protocol compatibility**, not a remote
diff --git a/docs/INTROSPECTION.md b/docs/INTROSPECTION.md
index 0145ed6..e84bf0d 100644
--- a/docs/INTROSPECTION.md
+++ b/docs/INTROSPECTION.md
@@ -18,7 +18,7 @@ The field also works on `/completion` itself, so an existing client only has to
add one key.
Implementation: `patches/0009-introspection-logits-and-activations.patch` over
-llama.cpp `04f7226`. Logits come from the context's own output buffer
+llama.cpp `970b51ce7`. Logits come from the context's own output buffer
(`llama_get_logits_ith`), so no work is repeated; activations are copied out of
the compute graph while it runs, through the graph-eval callback llama.cpp
already exposes (`llama_context_params.cb_eval`). Nothing is captured unless a
diff --git a/docs/MODELS.md b/docs/MODELS.md
index f84cc74..ab0a17d 100644
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@ -1,6 +1,6 @@
# Models
-Models worth running on a PS5, from what is popular on Ollama in October 2026, with the GGUF sizes on Hugging Face. Every architecture below loads in the pinned llama.cpp (`b11327`).
+Models worth running on a PS5, from what is popular on Ollama in October 2026, with the GGUF sizes on Hugging Face. Every architecture below loads in the pinned llama.cpp (`b11827`).
The console has 16 GB shared by everything. How much one PS5LM process can hold is what `probes/memprobe` measures. Until then: ProsperoAI already keeps a 6.37 GiB model resident on this console, so up to about 6 GiB is safe, and anything above 8 GiB waits for the probe.
diff --git a/docs/PORTING.md b/docs/PORTING.md
index 4ab1a22..862b023 100644
--- a/docs/PORTING.md
+++ b/docs/PORTING.md
@@ -1,6 +1,6 @@
# Porting notes
-What it took to build llama.cpp for the PS5, and why. Upstream is pinned at `b11327` (2026-10-01).
+What it took to build llama.cpp for the PS5, and why. Upstream is pinned at `b11827` (2026-10-22).
## The target
diff --git a/fleet/controller.py b/fleet/controller.py
index 16958a7..64505b3 100644
--- a/fleet/controller.py
+++ b/fleet/controller.py
@@ -19,7 +19,7 @@ import sys
from .models import ModelError, fetch_model, inspect_file
from .planner import FleetError, MIB, inventory, plan
-LLAMA_REVISION = '04f7226a2c12c7f382c545a6db5a034266aa53cb'
+LLAMA_REVISION = '970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc'
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_BINARY = ROOT / 'build/fleet-host/bin/llama-server'
diff --git a/patches/0005-listener-survives-network-drops.patch b/patches/0005-listener-survives-network-drops.patch
index e5decc4..2730849 100644
--- a/patches/0005-listener-survives-network-drops.patch
+++ b/patches/0005-listener-survives-network-drops.patch
@@ -1,5 +1,5 @@
diff --git a/tools/server/server-http.cpp b/tools/server/server-http.cpp
-index 33cbb156c..7ac78f9ad 100644
+index cf147c7e7..a5f26d018 100644
--- a/tools/server/server-http.cpp
+++ b/tools/server/server-http.cpp
@@ -10,6 +10,7 @@
@@ -8,9 +8,9 @@ index 33cbb156c..7ac78f9ad 100644
#include <string>
+#include <atomic>
#include <thread>
+ #include <mutex>
- //
-@@ -23,6 +24,7 @@ public:
+@@ -24,6 +25,7 @@ public:
std::vector<std::thread> threads; // one thread per listener
std::unique_ptr<httplib::ThreadPool> pool; // single pool shared among all listeners
int n_threads_http = 0;
@@ -18,7 +18,7 @@ index 33cbb156c..7ac78f9ad 100644
};
class server_http_task_queue : public httplib::TaskQueue {
-@@ -528,9 +530,23 @@ bool server_http_context::start() {
+@@ -529,9 +531,23 @@ bool server_http_context::start() {
pimpl->pool = std::make_unique<httplib::ThreadPool>(pimpl->n_threads_http, pimpl->n_threads_http + 1024);
for (size_t i = 0; i < pimpl->servers.size(); ++i) {
const auto & srv = pimpl->servers[i];
@@ -45,7 +45,7 @@ index 33cbb156c..7ac78f9ad 100644
}
});
srv->wait_until_ready();
-@@ -546,6 +562,7 @@ bool server_http_context::start() {
+@@ -547,6 +563,7 @@ bool server_http_context::start() {
}
void server_http_context::stop() const {
diff --git a/patches/0006-jinja-tojson-floats-as-python.patch b/patches/0006-jinja-tojson-floats-as-python.patch
deleted file mode 100644
index d2c138d..0000000
--- a/patches/0006-jinja-tojson-floats-as-python.patch
+++ /dev/null
@@ -1,69 +0,0 @@
-diff --git a/common/jinja/value.cpp b/common/jinja/value.cpp
-index 6edd8b9..d467942 100644
---- a/common/jinja/value.cpp
-+++ b/common/jinja/value.cpp
-@@ -5,6 +5,8 @@
- // for converting from JSON to jinja values
- #include "json.h"
-
-+#include <charconv>
-+#include <cmath>
- #include <sstream>
- #include <string>
- #include <cctype>
-@@ -1519,6 +1521,46 @@ void global_from_json(context & ctx, const common_json & json_obj, bool mark_inp
-
- // recursively convert value to JSON string
- // TODO: avoid circular references
-+// A float as Python's json.dumps writes it (repr): the shortest digits that read back the same
-+// value, fixed notation for exponents -4 to 15 with ".0" on whole numbers, else 1e+16 style.
-+// ostream's default (6 significant digits) wrote 0.0 as 0 and 1234567.89 as 1.23457e+06.
-+static std::string float_to_json(double v) {
-+ if (std::isnan(v)) {
-+ return "NaN";
-+ }
-+ if (std::isinf(v)) {
-+ return v > 0 ? "Infinity" : "-Infinity";
-+ }
-+ char buf[64];
-+ const auto res = std::to_chars(buf, buf + sizeof(buf), v, std::chars_format::scientific);
-+ const std::string sci(buf, res.ptr); // [-]d[.ddd]e(+|-)dd
-+ const size_t e_pos = sci.find('e');
-+ const bool neg = sci[0] == '-';
-+ std::string digits;
-+ for (size_t i = neg ? 1 : 0; i < e_pos; i++) {
-+ if (sci[i] != '.') {
-+ digits += sci[i];
-+ }
-+ }
-+ const int exp = std::stoi(sci.substr(e_pos + 1));
-+ std::string out = neg ? "-" : "";
-+ if (exp >= -4 && exp < 16) {
-+ if (exp < 0) {
-+ out += "0." + std::string(-exp - 1, '0') + digits;
-+ } else {
-+ if (digits.size() <= (size_t) exp + 1) {
-+ digits.append(exp + 1 - digits.size(), '0');
-+ }
-+ const std::string frac = digits.substr(exp + 1);
-+ out += digits.substr(0, exp + 1) + "." + (frac.empty() ? "0" : frac);
-+ }
-+ } else {
-+ out += digits.substr(0, 1) + (digits.size() > 1 ? "." + digits.substr(1) : "");
-+ out += (exp < 0 ? "e-" : "e+") + std::string(std::abs(exp) < 10 ? "0" : "") + std::to_string(std::abs(exp));
-+ }
-+ return out;
-+}
-+
- static void value_to_json_internal(std::ostringstream & oss, const value & val, int curr_lvl, int indent, const std::string_view item_sep, const std::string_view key_sep) {
- auto indent_str = [indent, curr_lvl]() -> std::string {
- return (indent > 0) ? std::string(curr_lvl * indent, ' ') : "";
-@@ -1534,7 +1576,7 @@ static void value_to_json_internal(std::ostringstream & oss, const value & val,
- } else if (is_val<value_int>(val)) {
- oss << val->as_int();
- } else if (is_val<value_float>(val)) {
-- oss << val->as_float();
-+ oss << float_to_json(val->as_float());
- } else if (is_val<value_string>(val)) {
- oss << "\"";
- for (char c : val->as_string().str()) {
diff --git a/patches/0009-introspection-logits-and-activations.patch b/patches/0009-introspection-logits-and-activations.patch
index 6ffa8de..44dc433 100644
--- a/patches/0009-introspection-logits-and-activations.patch
+++ b/patches/0009-introspection-logits-and-activations.patch
@@ -1,5 +1,5 @@
diff --git a/tools/server/CMakeLists.txt b/tools/server/CMakeLists.txt
-index 70b8d7d..7da0f33 100644
+index e0b9d0ec1..7cef7a7ee 100644
--- a/tools/server/CMakeLists.txt
+++ b/tools/server/CMakeLists.txt
@@ -25,6 +25,8 @@ add_library(${TARGET} STATIC
@@ -12,10 +12,10 @@ index 70b8d7d..7da0f33 100644
if (BUILD_SHARED_LIBS)
diff --git a/tools/server/server-context.cpp b/tools/server/server-context.cpp
-index 60c5633..c785782 100644
+index a0acf1655..6dba147c4 100644
--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
-@@ -292,6 +292,13 @@ struct server_slot {
+@@ -325,6 +325,13 @@ struct server_slot {
std::vector<completion_token_output> generated_token_probs;
@@ -29,7 +29,7 @@ index 60c5633..c785782 100644
bool has_next_token = true;
bool has_new_line = false;
bool truncated = false;
-@@ -397,6 +404,13 @@ struct server_slot {
+@@ -435,6 +442,13 @@ struct server_slot {
generated_token_probs.clear();
json_schema = json();
@@ -43,7 +43,7 @@ index 60c5633..c785782 100644
task_prev = std::move(task);
task.reset();
-@@ -953,6 +967,9 @@ public:
+@@ -1023,6 +1037,9 @@ public:
server_decision_context decision;
@@ -53,7 +53,7 @@ index 60c5633..c785782 100644
server_queue queue_tasks;
server_response queue_results;
-@@ -1019,6 +1036,14 @@ private:
+@@ -1089,6 +1106,14 @@ private:
int slots_debug = 0; // env: LLAMA_SERVER_SLOTS_DEBUG
int slots_n_diff = 0; // env: LLAMA_SERVER_SLOTS_N_DIFF
@@ -68,7 +68,7 @@ index 60c5633..c785782 100644
int n_empty_consecutive = 0;
std::unique_ptr<server_prompt_cache> prompt_cache;
-@@ -1213,11 +1238,33 @@ private:
+@@ -1284,11 +1309,33 @@ private:
params_base.load_progress_callback_user_data = &load_progress_text;
}
@@ -103,7 +103,7 @@ index 60c5633..c785782 100644
if (model_tgt == nullptr) {
SRV_ERR("failed to load model, '%s'\n", params_base.model.path.c_str());
return false;
-@@ -1230,6 +1277,13 @@ private:
+@@ -1301,6 +1348,13 @@ private:
vocab = llama_model_get_vocab(model_tgt);
@@ -117,7 +117,7 @@ index 60c5633..c785782 100644
try {
decision.init(model_tgt);
} catch (const std::exception & e) {
-@@ -2290,6 +2344,27 @@ private:
+@@ -2411,6 +2465,27 @@ private:
res->generation_params = slot.task->params; // copy the parameters
@@ -145,7 +145,7 @@ index 60c5633..c785782 100644
queue_results.send(std::move(res));
}
-@@ -3231,13 +3306,35 @@ private:
+@@ -3513,13 +3588,35 @@ private:
int32_t off_next = 0;
int32_t n_batch = llama_n_batch(ctx_tgt);
for (int32_t off = 0; off < batch.size(); off = off_next) {
@@ -183,7 +183,7 @@ index 60c5633..c785782 100644
#ifdef DEBUG_TIMINGS
llama_synchronize(ctx_tgt);
#endif
-@@ -3978,13 +4075,21 @@ private:
+@@ -4353,13 +4450,21 @@ private:
break;
}
@@ -206,7 +206,7 @@ index 60c5633..c785782 100644
/* is_prompt = */ true);
if (!slot.task->decision.order.empty()) {
batch.set_decision_order(batch.size() - 1, slot.task->decision.order[slot.prompt.n_tokens()]);
-@@ -4286,6 +4391,159 @@ private:
+@@ -4661,6 +4766,159 @@ private:
return true;
}
@@ -366,7 +366,7 @@ index 60c5633..c785782 100644
void post_decode(int32_t n_batch_tokens, int32_t off) {
// for checking if a given batch index is inside the current sub-batch
auto is_inside_view = [&](int32_t idx) {
-@@ -4842,6 +5100,34 @@ std::unique_ptr<server_res_generator> server_routes::handle_completions_impl(
+@@ -5230,6 +5488,34 @@ std::unique_ptr<server_res_generator> server_routes::handle_completions_impl(
task.apply_chat_session(chat_session);
@@ -401,7 +401,7 @@ index 60c5633..c785782 100644
task.id_slot = json_value(data, "id_slot", -1);
sse_ping_interval = task.params.sse_ping_interval;
-@@ -5690,6 +5976,49 @@ void server_routes::init_routes() {
+@@ -6283,6 +6569,49 @@ void server_routes::init_routes() {
return handle_embeddings_impl(req, TASK_RESPONSE_TYPE_OAI_EMBD);
};
@@ -449,15 +449,15 @@ index 60c5633..c785782 100644
+ };
+
this->post_rerank = [this](const server_http_req & req) {
- auto res = create_response();
- if (!params.embedding || params.pooling_type != LLAMA_POOLING_TYPE_RANK) {
+ return handle_rerank_impl(req);
+ };
diff --git a/tools/server/server-context.h b/tools/server/server-context.h
-index ed23f7d..d2ae01d 100644
+index 4a8e327fc..3d2637a32 100644
--- a/tools/server/server-context.h
+++ b/tools/server/server-context.h
-@@ -154,6 +154,8 @@ struct server_routes {
- server_http_context::handler_t post_embeddings_oai;
+@@ -159,6 +159,8 @@ struct server_routes {
server_http_context::handler_t post_rerank;
+ server_http_context::handler_t post_tts;
server_http_context::handler_t post_systemone;
+ // PS5LM fork: raw logits and per-layer activations
+ server_http_context::handler_t post_introspect;
@@ -466,7 +466,7 @@ index ed23f7d..d2ae01d 100644
diff --git a/tools/server/server-introspect.cpp b/tools/server/server-introspect.cpp
new file mode 100644
-index 0000000..af9c2ee
+index 000000000..af9c2ee80
--- /dev/null
+++ b/tools/server/server-introspect.cpp
@@ -0,0 +1,624 @@
@@ -1096,7 +1096,7 @@ index 0000000..af9c2ee
+}
diff --git a/tools/server/server-introspect.h b/tools/server/server-introspect.h
new file mode 100644
-index 0000000..a80f7e1
+index 000000000..a80f7e1d7
--- /dev/null
+++ b/tools/server/server-introspect.h
@@ -0,0 +1,205 @@
@@ -1306,7 +1306,7 @@ index 0000000..a80f7e1
+ }
+};
diff --git a/tools/server/server-schema.cpp b/tools/server/server-schema.cpp
-index 54e126a..30efbbb 100644
+index 1e2dce888..a3609b179 100644
--- a/tools/server/server-schema.cpp
+++ b/tools/server/server-schema.cpp
@@ -187,6 +187,17 @@ std::vector<std::unique_ptr<field>> make_llama_cmpl_schema(const common_params &
@@ -1328,7 +1328,7 @@ index 54e126a..30efbbb 100644
->set_desc("Return probabilities of top n_probs tokens after applying the sampling chain"));
diff --git a/tools/server/server-task.cpp b/tools/server/server-task.cpp
-index 3f30da2..542d559 100644
+index d7ac2a1dc..180984684 100644
--- a/tools/server/server-task.cpp
+++ b/tools/server/server-task.cpp
@@ -351,6 +351,10 @@ json server_task_result_cmpl_final::to_json_non_oaicompat() {
@@ -1343,18 +1343,18 @@ index 3f30da2..542d559 100644
}
diff --git a/tools/server/server-task.h b/tools/server/server-task.h
-index 50926ec..93b29ec 100644
+index 810cefbfe..97d31fd9c 100644
--- a/tools/server/server-task.h
+++ b/tools/server/server-task.h
-@@ -10,6 +10,7 @@
-
+@@ -11,6 +11,7 @@
// TODO: prevent including the whole server-common.h as we only use server_tokens
#include "server-common.h"
+ #include "mtmd-helper.h"
+#include "server-introspect.h" // PS5LM fork: introspection request/response types
enum server_task_type {
-@@ -99,6 +100,12 @@ struct task_params {
+@@ -111,6 +112,12 @@ struct task_params {
// Embeddings
int32_t embd_normalize = 2; // (-1=none, 0=max absolute int16, 1=taxicab, 2=Euclidean/L2, >2=p-norm)
@@ -1367,7 +1367,7 @@ index 50926ec..93b29ec 100644
json format_logit_bias(const std::vector<llama_logit_bias> & logit_bias) const;
json to_json(bool only_metrics = false) const;
};
-@@ -384,6 +391,9 @@ struct server_task_result_cmpl_final : server_task_result {
+@@ -407,6 +414,9 @@ struct server_task_result_cmpl_final : server_task_result {
task_params generation_params;
@@ -1378,10 +1378,10 @@ index 50926ec..93b29ec 100644
bool verbose = false;
task_response_type res_type = TASK_RESPONSE_TYPE_NONE;
diff --git a/tools/server/server.cpp b/tools/server/server.cpp
-index 871e827..12dd9bd 100644
+index ce44405b8..ba39d0e63 100644
--- a/tools/server/server.cpp
+++ b/tools/server/server.cpp
-@@ -241,6 +241,8 @@ int llama_server(common_params & params, int argc, char ** argv, server_child &
+@@ -258,6 +258,8 @@ int llama_server(common_params & params, int argc, char ** argv, server_child &
routes.post_systemone = models_routes->proxy_post;
routes.post_tokenize = models_routes->proxy_post;
routes.post_detokenize = models_routes->proxy_post;
@@ -1390,7 +1390,7 @@ index 871e827..12dd9bd 100644
routes.post_apply_template = models_routes->proxy_post;
routes.post_chat_completions_tok = models_routes->proxy_post;
routes.post_responses_tok_oai = models_routes->proxy_post;
-@@ -289,6 +291,9 @@ int llama_server(common_params & params, int argc, char ** argv, server_child &
+@@ -308,6 +310,9 @@ int llama_server(common_params & params, int argc, char ** argv, server_child &
ctx_http.post("/v1/systemone", ex_wrapper(routes.post_systemone));
ctx_http.post("/tokenize", ex_wrapper(routes.post_tokenize));
ctx_http.post("/detokenize", ex_wrapper(routes.post_detokenize));
diff --git a/tests/fleet_rpc_no_comm_test.py b/tests/fleet_rpc_no_comm_test.py
index 9645ef6..75e07e4 100644
--- a/tests/fleet_rpc_no_comm_test.py
+++ b/tests/fleet_rpc_no_comm_test.py
@@ -67,7 +67,8 @@ def run(binary):
time.sleep(0.05)
sock.settimeout(1)
hello = request(sock, 14, bytes(24), 28)
- assert hello[:3] == bytes([8, 0, 0]), hello[:3]
+ # Only the major version breaks compatibility; upstream bumps the patch per new ggml op.
+ assert hello[0] == 8, hello[:3]
for rank in (0, 1):
body = struct.pack('<IIII64s', 0, rank, 2, comm_port, b'127.0.0.1')
assert request(sock, 20, body, 1) == b'\0', f'rank {rank} was not refused'
diff --git a/third_party/llama.cpp b/third_party/llama.cpp
index 04f7226..970b51c 160000
--- a/third_party/llama.cpp
+++ b/third_party/llama.cpp
@@ -1 +1 @@
-Subproject commit 04f7226a2c12c7f382c545a6db5a034266aa53cb
+Subproject commit 970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc