llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
App: load models straight from USB, library in sections, no auto-load
Abhirama39dc6f25080
diff --git a/README.md b/README.mdindex 4a3176d..15af440 100644--- a/README.md+++ b/README.md@@ -99,7 +99,7 @@ scripts/build-app.sh # build/app-ps5/title/PPSA99581Copy `build/app-ps5/title/PPSA99581` to `/data/homebrew/PPSA99581` and a model to `/data/PS5LM/models`, then start **PS5LM** from the home screen. The chat is at `http://<console IP>:8081` once the model is on the GPU (about 90 s).-**On the TV** the app shows a dashboard drawn with [ps5-homebrew-ui](https://github.com/blackbearreloaded/ps5-homebrew-ui) (a submodule in `third_party/`): generation speed, how busy the GPU is, CPU use, the memory pool the GPU and CPU share and how the model, its KV cache and the rest split it, context in use, and the model. Cross on the Model tile opens the **model library**: every `.gguf` in `/data/PS5LM/models` and on USB drives (`/mnt/usbN` or `/mnt/usbN/PS5LM/models`), each with the context and KV cache type the planner (`ps5/app/model_plan.cpp`) picks for the memory free, and the sampling settings of its family when it is a model we know (Qwen 3.5/3.6/3.8, Gemma 4, gpt-oss, Granite, Nemotron, Mistral, Llama). Cross loads one, or imports one from USB into internal storage.+**On the TV** the app shows a dashboard drawn with [ps5-homebrew-ui](https://github.com/blackbearreloaded/ps5-homebrew-ui) (a submodule in `third_party/`): generation speed, how busy the GPU is, CPU use, the memory pool the GPU and CPU share and how the model, its KV cache and the rest split it, context in use, and the model. Cross on the Model tile opens the **model library**: every `.gguf` in `/data/PS5LM/models` and on USB drives (`/mnt/usbN` or `/mnt/usbN/PS5LM/models`), each with the context and KV cache type the planner (`ps5/app/model_plan.cpp`) picks for the memory free, and the sampling settings of its family when it is a model we know (Qwen 3.5/3.6/3.8, Gemma 4, gpt-oss, Granite, Nemotron, Mistral, Llama). It lists internal storage and USB drives in separate sections; Cross loads a model from either. llama.cpp reads the whole file into GPU memory (no mmap), so a model on USB loads at the drive's speed and the drive is not touched afterwards.`/data/PS5LM/app-args.txt` (one argument per line) overrides the planner. The app logs to `/data/PS5LM/app.log` and `llama.log`.diff --git a/docs/CONSOLE.md b/docs/CONSOLE.mdindex 79fecfc..fc564bf 100644--- a/docs/CONSOLE.md+++ b/docs/CONSOLE.md@@ -98,10 +98,10 @@ What a title has to do differently from a payload, each found by a crash:### Switching models-Unloading a model in the running app leaves part of its GPU memory held: with Qwen3.8-27B UD-IQ2_XXS at 64k, the pool went from 11.01 GiB free before loading to 5.77 GiB free after `llama_server` returned, and ggml's Vulkan figure fell the same way, so the next model could not get its KV cache. The app therefore changes model by restarting itself: it writes the choice to `/data/PS5LM/model.txt` and calls `sceSystemServiceLoadExec("/app0/eboot.bin", NULL)`. The system replaces the process (`Kill for LoadExec(0x11d) => 0`, then a new `EXEC`) without the crash reporter, and the new process starts from the whole pool (11.01 GiB free). Where the memory is held is still open.+Unloading a model in the running app leaves part of its GPU memory held: with Qwen3.8-27B UD-IQ2_XXS at 64k, the pool went from 11.01 GiB free before loading to 5.77 GiB free after `llama_server` returned, and ggml's Vulkan figure fell the same way, so the next model could not get its KV cache. The app therefore changes model by restarting itself: it writes the choice to `/data/PS5LM/model.txt` (read once by the new process, then deleted, so a plain launch loads nothing and opens the library) and calls `sceSystemServiceLoadExec("/app0/eboot.bin", NULL)`. The system replaces the process (`Kill for LoadExec(0x11d) => 0`, then a new `EXEC`) without the crash reporter, and the new process starts from the whole pool (11.01 GiB free). Where the memory is held is still open.### The dashboard on the console-ps5-homebrew-ui draws through ps5-opengl in the same title as RADV: the GL stack is linked as one object with only its GL/EGL API global (4,399 symbol names collide with RADV's Mesa otherwise), `sceAgcInit` runs once for both drivers (`--wrap`), and the GL runtime's thread-local initialiser `_ZTH23_mesa_glapi_tls_Context` is an empty function, as ProsperoAI has it. The screen costs about 0.35 GiB of the pool (11.38 to 11.03 GiB free). The app can be driven from a PC without a controller: `scripts/ps5lm-app.sh close|deploy|launch|screenshot|load|unload|import`.+ps5-homebrew-ui draws through ps5-opengl in the same title as RADV: the GL stack is linked as one object with only its GL/EGL API global (4,399 symbol names collide with RADV's Mesa otherwise), `sceAgcInit` runs once for both drivers (`--wrap`), and the GL runtime's thread-local initialiser `_ZTH23_mesa_glapi_tls_Context` is an empty function, as ProsperoAI has it. The screen costs about 0.35 GiB of the pool (11.38 to 11.03 GiB free). The app can be driven from a PC without a controller: `scripts/ps5lm-app.sh close|deploy|launch|screenshot|load|unload`.ShadowMountPlus registers a title once and copies its icon and backgrounds then; replacing them later needs a re-registration. Uninstalling to force one hit "Register failed: TitleDir bridge unavailable" after ShadowMountPlus had logged `stale cave bridge detached`; reloading ShadowMountPlus fixed it.diff --git a/ps5/app/dashboard.cpp b/ps5/app/dashboard.cppindex 3cf1df6..9682744 100644--- a/ps5/app/dashboard.cpp+++ b/ps5/app/dashboard.cpp@@ -182,14 +182,11 @@ void Dashboard::update(const hui::InputFrame & input, float dt, ui::Feedback & f}if (input.is_pressed(Action::confirm) && rows > 0) {const ModelRow & m = live_.models[(size_t) library_cursor_];- if (m.on_usb && m.import_progress < 0) {- request_ = { Request::import, m.file };- feedback.play(hui::audio::Cue::select);- } else if (m.current) {+ if (m.current) {request_ = { Request::unload, m.file };library_open_ = false;feedback.play(hui::audio::Cue::back);- } else if (!m.on_usb && m.fits) {+ } else if (m.fits) {request_ = { Request::load, m.file };library_open_ = false;feedback.play(hui::audio::Cue::complete);@@ -227,14 +224,7 @@ void Dashboard::update(const hui::InputFrame & input, float dt, ui::Feedback & f}}if (input.is_pressed(Action::confirm) && focus_ == kModel) {- library_open_ = true;- library_cursor_ = 0;- for (int i = 0; i < rows; ++i) {- if (live_.models[(size_t) i].current) {- library_cursor_ = i;- }- }- cursor_y_.snap((float) library_cursor_);+ open_library();feedback.play(hui::audio::Cue::open);}}@@ -248,7 +238,7 @@ void Dashboard::update(const hui::InputFrame & input, float dt, ui::Feedback & frefusal_.update(dt, 9.0f);library_.target = library_open_ ? 1.0f : 0.0f;library_.update(dt, 13.0f);- cursor_y_.target = (float) library_cursor_;+ cursor_y_.target = (float) slot(library_cursor_);cursor_y_.update(dt, 20.0f);}@@ -491,7 +481,7 @@ void Dashboard::draw_content(gfx::DrawList & list, int tile, const Rect & r, floif (L.model_label.empty()) {ui::text(list, f.semibold, "No model loaded", r.x + kPad, r.y + 104, 34, kInk);ui::text(list, f.regular, "Cross opens the library", r.x + kPad, r.y + 138, 20, kInk.with_alpha(kMuted));- std::snprintf(text, sizeof(text), "%zu installed", L.models.size());+ std::snprintf(text, sizeof(text), "%zu found", L.models.size());ui::text(list, f.regular, text, r.x + kPad, r.y + 352, 20, kInk.with_alpha(kFaint));break;}@@ -510,7 +500,7 @@ void Dashboard::draw_content(gfx::DrawList & list, int tile, const Rect & r, flocaps(list, f, rows[k][0], r.x + kPad, y, 15, kInk.with_alpha(kMuted));ui::text(list, f.mono, rows[k][1], r.x + r.w - kPad, y + 2, 22, kInk, gfx::Align::right);}- std::snprintf(text, sizeof(text), "%zu installed", L.models.size());+ std::snprintf(text, sizeof(text), "%zu found", L.models.size());ui::text(list, f.regular, text, r.x + kPad, r.y + 352, 20, kInk.with_alpha(kFaint));break;}@@ -578,7 +568,7 @@ void Dashboard::draw_library(gfx::DrawList & list, std::uint32_t glass) const {draw_panel(list, panel, 40, 1.0f, kLime);caps(list, f, "MODEL LIBRARY", panel.x + 56, panel.y + 60, 20, kLime);ui::text(list, f.regular,- f.regular.font->fit("Settings fit the memory free now. Add models over FTP to /data/PS5LM/models, or import them from USB.",+ f.regular.font->fit("Settings fit the memory free now. Models go in /data/PS5LM/models, or PS5LM/models on USB.",20, panel.w - 112),panel.x + 56, panel.y + 98, 20, kInk.with_alpha(kMuted));@@ -587,46 +577,74 @@ void Dashboard::draw_library(gfx::DrawList & list, std::uint32_t glass) const {if (rows == 0) {ui::text(list, f.regular, "No models found yet", panel.cx(), panel.cy(), 26, kInk.with_alpha(kFaint), gfx::Align::center);}- // Eight rows show; the list scrolls to keep the cursor among them.- const int first = std::max(0, library_cursor_ - 7);+ // Eight lines show, section headers included; the list scrolls to keep+ // the cursor among them.+ const int first = rows > 0 ? std::max(0, slot(library_cursor_) - 7) : 0;const Rect hl{ panel.x + 36, top + (cursor_y_.value - (float) first) * kRowH, panel.w - 72, kRowH - 8 };if (rows > 0) {list.glow(hl, 18, 18, kLime.with_alpha(0.25f));list.bordered_rect(hl, 18, kLime.with_alpha(0.1f), 2.0f, kLime.with_alpha(0.7f));}- for (int i = first; i < rows && i < first + 8; ++i) {+ for (int i = 0; i < rows; ++i) {const ModelRow & m = live_.models[(size_t) i];- const float y = top + (float) (i - first) * kRowH;+ const int s = slot(i);+ if (i == 0 || m.on_usb != live_.models[(size_t) i - 1].on_usb) {+ const int h = s - 1;+ if (h >= first && h < first + 8) {+ const float hy = top + (float) (h - first) * kRowH;+ caps(list, f, m.on_usb ? "USB DRIVE" : "INTERNAL STORAGE", panel.x + 60, hy + 50, 16,+ m.on_usb ? kAmber : kCyan);+ list.rounded_rect({ panel.x + 60, hy + 62, panel.w - 120, 1 }, 0, kInk.with_alpha(0.12f));+ }+ }+ if (s < first || s >= first + 8) {+ continue;+ }+ const float y = top + (float) (s - first) * kRowH;const float ink = m.fits ? 1.0f : 0.45f;ui::text(list, f.semibold, f.semibold.font->fit(m.label, 24, 620), panel.x + 60, y + 32, 24, kInk.with_alpha(ink));std::string sub = m.plan;if (!m.preset.empty()) {sub = m.preset + " preset \xC2\xB7 " + sub;}- if (m.on_usb) {- sub = "On USB \xC2\xB7 " + sub;- }ui::text(list, f.regular, f.regular.font->fit(sub, 18, 700), panel.x + 60, y + 58, 18, kInk.with_alpha(kMuted * ink));char size[24];std::snprintf(size, sizeof(size), "%.1f GiB", m.size_gib);ui::text(list, f.mono, size, panel.x + panel.w - 260, y + 44, 22, kInk.with_alpha(ink), gfx::Align::right);- if (m.import_progress >= 0) {- bar(list, { panel.x + panel.w - 200, y + 34, 140, 10 }, m.import_progress, kAmber);- } else {- const char * tag = m.current ? "LOADED" : m.on_usb ? "IMPORT" : m.fits ? "LOAD" : "TOO BIG";- const Color tc = m.current ? kCyan : m.on_usb ? kAmber : m.fits ? kLime : kRose;- caps(list, f, tag, panel.x + panel.w - 60, y + 44, 15, tc, gfx::Align::right);- }+ const char * tag = m.current ? "LOADED" : m.fits ? "LOAD" : "TOO BIG";+ const Color tc = m.current ? kCyan : m.fits ? kLime : kRose;+ caps(list, f, tag, panel.x + panel.w - 60, y + 44, 15, tc, gfx::Align::right);}list.pop_opacity();}+void Dashboard::open_library() {+ library_open_ = true;+ library_cursor_ = 0;+ for (int i = 0; i < (int) live_.models.size(); ++i) {+ if (live_.models[(size_t) i].current) {+ library_cursor_ = i;+ }+ }+ cursor_y_.snap((float) slot(library_cursor_));+}++// Rows come sorted by path, so internal (/data) ones precede USB (/mnt) ones;+// each group gets a header line above it.+int Dashboard::slot(int row) const {+ int s = row + 1;+ if (row > 0 && live_.models[(size_t) row].on_usb && !live_.models[0].on_usb) {+ ++s;+ }+ return s;+}+void Dashboard::draw_hints(DashboardFrame & frame) const {const ui::GlyphStyle style = ui::GlyphStyle::dark();if (library_.value > 0.5f) {const bool on = library_cursor_ < (int) live_.models.size();const ModelRow * m = on ? &live_.models[(size_t) library_cursor_] : nullptr;- const char * act = !m ? "Load" : m->current ? "Unload" : m->on_usb ? "Import" : "Load";+ const char * act = !m ? "Load" : m->current ? "Unload" : "Load";const ui::Hint hints[] = { { ui::Button::dpad, "Choose" }, { ui::Button::cross, act }, { ui::Button::circle, "Back" } };ui::draw_hints(frame.overlay, fonts_, style, hints, 3, kGridX + kGridW, true);} else {diff --git a/ps5/app/dashboard.hpp b/ps5/app/dashboard.hppindex 68a7ed0..1f508f9 100644--- a/ps5/app/dashboard.hpp+++ b/ps5/app/dashboard.hpp@@ -36,13 +36,13 @@ class Dashboard {explicit Dashboard(const hui::ui::Fonts & fonts) : fonts_(fonts) { ring_.snap(ring_rect()); }void set_live(const Live & live) { live_ = live; }+ void open_library(); // over the dashboard, cursor on the loaded modelvoid update(const hui::InputFrame & input, float dt, hui::ui::Feedback & feedback);void draw(DashboardFrame & frame) const;- // What the user asked for in the library, once: load a model, or import- // one from a USB drive. Empty path: nothing.+ // What the user asked for in the library, once: load or unload a model.struct Request {- enum Kind { none, load, unload, import } kind = none;+ enum Kind { none, load, unload } kind = none;std::string path;};Request take_request() {@@ -76,7 +76,8 @@ class Dashboard {bool library_open_ = false;int library_cursor_ = 0;hui::tween::Spring library_; // 0 closed, 1 open- hui::tween::Spring cursor_y_; // the highlight's row+ hui::tween::Spring cursor_y_; // the highlight's slot+ int slot(int row) const; // a model row's line in the list, after the section headersmutable char size_text_[64] = {};};diff --git a/ps5/app/main.cpp b/ps5/app/main.cppindex 764566c..bd283a8 100644--- a/ps5/app/main.cpp+++ b/ps5/app/main.cpp@@ -254,12 +254,6 @@ struct Models {pthread_t server{};bool server_started = false;- // Import from USB, one at a time.- std::mutex import_mutex;- std::string importing; // source path- std::atomic<float> import_progress{ -1 };- std::atomic<bool> import_done{ false };-void scan(StatsCollector & stats) {rows.clear();for (const auto & path : find_models(kModelDirs)) {@@ -289,12 +283,8 @@ struct Models {void publish(StatsCollector & stats) {std::vector<ModelRow> out = rows;- std::lock_guard<std::mutex> lock(import_mutex);for (auto & r : out) {r.current = r.file == running;- if (r.file == importing) {- r.import_progress = import_progress;- }}stats.set_models(out);}@@ -348,50 +338,6 @@ struct Models {running.clear();log_memory("after unloading");}-- void begin_import(const std::string & from) {- {- std::lock_guard<std::mutex> lock(import_mutex);- if (!importing.empty()) {- return;- }- importing = from;- }- import_progress = 0;- import_done = false;- std::thread([this, from] {- const std::string to = std::string(kModels) + "/" + base_name(from) + ".gguf";- const std::string part = to + ".part";- FILE * in = std::fopen(from.c_str(), "rb");- FILE * out = in ? std::fopen(part.c_str(), "wb") : nullptr;- bool ok = in && out;- if (ok) {- std::fseek(in, 0, SEEK_END);- const double total = (double) std::ftell(in);- std::fseek(in, 0, SEEK_SET);- std::vector<char> buf(8u << 20);- double done = 0;- for (size_t n; ok && (n = std::fread(buf.data(), 1, buf.size(), in)) > 0;) {- ok = std::fwrite(buf.data(), 1, n, out) == n;- done += (double) n;- import_progress = (float) (done / total);- }- ok = ok && std::fflush(out) == 0 && fsync(fileno(out)) == 0;- }- if (in) std::fclose(in);- if (out) std::fclose(out);- ok = ok && std::rename(part.c_str(), to.c_str()) == 0;- if (!ok) {- std::remove(part.c_str());- }- std::printf("ps5lm-app: import %s: %s\n", from.c_str(), ok ? "done" : "failed");- notify(ok ? "PS5LM: model imported" : "PS5LM: import failed (space on /data?)");- std::lock_guard<std::mutex> lock(import_mutex);- importing.clear();- import_progress = -1;- import_done = true;- }).detach();- }};bool load_font(hui::gfx::Renderer & renderer, const char * name, hui::gfx::Font * font, hui::ui::FontRef * ref) {@@ -458,8 +404,9 @@ int main(int, char **) {stats.start(kPort);models.scan(stats);- // Which model: app-args.txt verbatim, else model.txt (the library's last- // choice), else the largest that fits with at least 8k of context.+ // Which model: app-args.txt verbatim, else model.txt, which the library+ // writes just before restarting the app for a switch and which is used+ // once. A plain launch loads nothing and opens the library instead.Live model_part;std::vector<std::string> args = read_lines(kArgs);std::string path;@@ -473,18 +420,9 @@ int main(int, char **) {std::printf("ps5lm-app: app-args.txt overrides the planner\n");} else {const auto chosen = read_lines(kChoice);- double best = -1;- // "none": the user unloaded the model; start without one.- for (const auto & r : chosen.empty() || chosen[0] != "none" ? models.rows : std::vector<ModelRow>{}) {- if (!r.fits || r.on_usb) {- continue;- }- if (!chosen.empty() && r.file == chosen[0]) {- path = r.file;- break;- }- if (r.size_gib > best) {- best = r.size_gib;+ unlink(kChoice);+ for (const auto & r : models.rows) {+ if (r.fits && !chosen.empty() && r.file == chosen[0]) {path = r.file;}}@@ -497,7 +435,9 @@ int main(int, char **) {models.start(path, args, stats, model_part);} else {stats.set_state(ServerState::no_model);- notify("PS5LM: no model fits; copy a .gguf to /data/PS5LM/models or import one from USB");+ if (models.rows.empty()) {+ notify("PS5LM: no models; copy a .gguf to /data/PS5LM/models or PS5LM/models on a USB drive");+ }}// Input and sound, as ps5-homebrew-ui's own app does.@@ -511,6 +451,7 @@ int main(int, char **) {sounds.load(std::string(kAssets) + "/audio/sfx");Dashboard dash(fonts);+ bool library_shown = false; // opened once at launch when no model is loadedDashboardFrame frame;hui::ui::Feedback feedback;hui::PadSample samples[64];@@ -536,10 +477,13 @@ int main(int, char **) {if (n % 30 == 0) {models.publish(stats);}- if (models.import_done.exchange(false)) {- models.scan(stats);- }dash.set_live(live);+ if (!library_shown && !live.models.empty()) {+ library_shown = true;+ if (live.state == ServerState::no_model) {+ dash.open_library();+ }+ }feedback.clear();dash.update(input, dt, feedback);for (const auto & cue : feedback.cues) {@@ -551,19 +495,15 @@ int main(int, char **) {pad.tick(dt);Dashboard::Request req = dash.take_request();- // The same requests from the PC (scripts/ps5lm-app.sh load|import).+ // The same requests from the PC (scripts/ps5lm-app.sh load|unload).if (n % 60 == 30 && req.kind == Dashboard::Request::none) {if (std::string p = take_request_file("/data/PS5LM/load"); !p.empty()) {req = { Dashboard::Request::load, p };} else if (!take_request_file("/data/PS5LM/unload").empty()) {req = { Dashboard::Request::unload, "" };- } else if (std::string q = take_request_file("/data/PS5LM/import"); !q.empty()) {- req = { Dashboard::Request::import, q };}}- if (req.kind == Dashboard::Request::import) {- models.begin_import(req.path);- } else if (req.kind == Dashboard::Request::unload || req.kind == Dashboard::Request::load) {+ if (req.kind == Dashboard::Request::unload || req.kind == Dashboard::Request::load) {// Unloading in place leaves part of the model's GPU memory held// (about 5 GiB of a 6.8 GiB model, measured), so a model change// restarts the app: the system replaces the process (LoadExec ondiff --git a/ps5/app/preview_main.cpp b/ps5/app/preview_main.cppindex 10712ab..131ad13 100644--- a/ps5/app/preview_main.cpp+++ b/ps5/app/preview_main.cpp@@ -114,7 +114,6 @@ ps5lm::Live sample(float seconds, bool generating) {usb.plan = "64k context, q8_0 cache: 9.1 of 11.0 GiB";usb.preset = "Gemma 4";usb.size_gib = 5.30;- usb.import_progress = 0.42f;L.models.push_back(usb);if (g_loading >= 0) {L.state = ps5lm::ServerState::loading;@@ -230,7 +229,7 @@ int main(int argc, char ** argv) {step(30, nav(hui::Direction::up), false);shot("04-library-choose");for (int i = 0; i < 3; ++i) {- step(12, nav(hui::Direction::down), false);+ step(25, nav(hui::Direction::down), false);}shot("05-library-usb");return ok ? 0 : 1;diff --git a/ps5/app/stats.hpp b/ps5/app/stats.hppindex 2423777..0ed16c9 100644--- a/ps5/app/stats.hpp+++ b/ps5/app/stats.hpp@@ -28,8 +28,7 @@ struct ModelRow {double size_gib = 0;bool fits = false;bool current = false;- bool on_usb = false; // found on a USB drive: importable to internal storage- float import_progress = -1; // 0..1 while it is being copied in+ bool on_usb = false; // on a USB drive; loads from there like internal ones};struct Live {diff --git a/scripts/ps5lm-app.sh b/scripts/ps5lm-app.shindex 08b0982..9e7b4da 100755--- a/scripts/ps5lm-app.sh+++ b/scripts/ps5lm-app.sh@@ -7,7 +7,6 @@# scripts/ps5lm-app.sh cycle close, deploy, launch# scripts/ps5lm-app.sh screenshot [file] what the TV shows, as a BMP# scripts/ps5lm-app.sh load <model path> switch the model, as the library does-# scripts/ps5lm-app.sh import <path> copy a .gguf into /data/PS5LM/models# scripts/ps5lm-app.sh unload unload the model## The app is never killed: it polls /data/PS5LM/quit and leaves through@@ -74,12 +73,11 @@ screenshot() {case ${1:-} inload) request load "${2:?model path on the console}" ;;- import) request import "${2:?file path on the console}" ;;unload) request unload ;;screenshot) screenshot "${2:-}" ;;close) close ;;deploy) deploy ;;launch) launch ;;cycle) close; deploy; launch ;;- *) echo "usage: $0 close|deploy|launch|cycle|screenshot|load|unload|import" >&2; exit 2 ;;+ *) echo "usage: $0 close|deploy|launch|cycle|screenshot|load|unload" >&2; exit 2 ;;esac