llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Add the model list, the launcher plan and console setup scripts
Mert Cobanov3bed43489b2e
diff --git a/README.md b/README.mdindex 9325743..9039594 100644--- a/README.md+++ b/README.md@@ -18,7 +18,7 @@ Earlier PS5 LLM work hand-writes GPU kernels for one model at a time. PS5LM port| `ps5/compat` | the few libc functions the console lacks || `probes/memprobe` | Phase 0: measures memory, bandwidth and compute on the console || `scripts/` | SDK setup, the llama.cpp cross build, sending payloads |-| `docs/` | [roadmap](docs/ROADMAP.md), [porting notes](docs/PORTING.md), [research](docs/RESEARCH.md) |+| `docs/` | [roadmap](docs/ROADMAP.md), [models that fit](docs/MODELS.md), [porting notes](docs/PORTING.md), [research](docs/RESEARCH.md) |## Building@@ -44,7 +44,9 @@ scripts/build-llama.sh # llama-cli, llama-server, llama-bench for theYou need a PS5 you own on firmware 7.00 to 13.60, jailbroken, with an ELF loader listening on port 9021 ([elfldr](https://github.com/ps5-payload-dev/elfldr)).```sh+scripts/host-relapse.sh # serves the Relapse exploit page; open it in the PS5's browserexport PS5_HOST=192.168.1.50 # your console+scripts/console-setup.sh # FTP, kernel log and shell payloadsscripts/send.sh probes/memprobe/memprobe.elf```diff --git a/docs/MODELS.md b/docs/MODELS.mdnew file mode 100644index 0000000..9117e07--- /dev/null+++ b/docs/MODELS.md@@ -0,0 +1,37 @@+# Models++Models worth running on a PS5, from what is popular on Ollama in October 2026, with the GGUF sizes on Hugging Face. Every architecture below loads in the pinned llama.cpp (`b11327`).++The console has 16 GB shared by everything. How much one PS5LM process can hold is what `probes/memprobe` measures. Until then: ProsperoAI already keeps a 6.37 GiB model resident on this console, so up to about 6 GiB is safe, and anything above 8 GiB waits for the probe.++## Fits for sure (up to 6 GiB)++| Model | Ollama name | Quant | Size | Architecture | Notes |+|---|---|---|---|---|---|+| Granite 4.2 3B | `granite4.2:3b` | Q4_K_M | 2.09 GiB | granite | the smoke test model: small and fast |+| Nemotron 3 Nano 4B | | Q4_K_M | 2.64 GiB | nemotron_h | hybrid Mamba, 1M context |+| Gemma 4 E4B | | Q4_0 | 4.28 GiB | gemma4 | |+| Granite 4.2 8B | `granite4.2:8b` | Q4_K_M | 4.98 GiB | granite | |+| Qwen3.5 9B | | Q4_K_M | 5.29 GiB | qwen35 | the model ProsperoAI runs: a direct comparison |+| Ornith 1.5 9B | `ornith-1.5:9b` | Q4_K_M | 5.38 GiB | qwen35 | among Ollama's most pulled |+| Gemma 4 12B | | Q3_K_M / IQ4_XS | 5.30 / 5.94 GiB | gemma4 | |++## Fits if the probe allows (6 to 11 GiB)++| Model | Ollama name | Quant | Size | Architecture | Notes |+|---|---|---|---|---|---|+| **Qwen 3.8 27B** | `qwen3.8` | UD-IQ2_XXS / UD-Q2_K_XL | 6.77 / 9.15 GiB | qwen35 | the headline target |+| Muse Glimmer 30B | `muse-glimmer` | IQ2_XXS | 8.31 GiB | muse-glimmer | |+| Qwen 3.6 27B | `qwen3.6:27b` | UD-IQ2_XXS | 8.74 GiB | qwen35 | Ollama's most pulled model |+| Qwen 3.6 35B-A3B | `qwen3.6:35b` | UD-IQ2_XXS | 10.02 GiB | qwen35moe | MoE, 3B active |+| gpt-oss 20B | `gpt-oss:20b` | MXFP4 | 10.7 GiB | gpt-oss | MoE, 3.6B active; every quant is about this size |++## Does not fit++- Nemotron 3.5 Lightning 30B-A3B: the smallest GGUF is 18 GiB.+- Ornith 1.5 35B-A3B: only a 20 GiB Q4_K_M is published.+- Qwen 3.8 Flash-Next (180B), and the cloud-only models (MiniMax M3, GLM 5.x, DeepSeek V4).++## Speed++Decoding reads the active weights once per token, so tokens per second ≈ memory bandwidth / active bytes per token. A dense 27B at 2 to 3 bits reads 7 to 9 GiB per token; a MoE with 3B active reads around 1 GiB. On the CPU the MoE models (Qwen 3.6 35B-A3B, gpt-oss 20B) should be several times faster than the dense 27Bs, at a similar quality. Real numbers come with `llama-bench` on the console.diff --git a/docs/ROADMAP.md b/docs/ROADMAP.mdindex 1bb878c..ed63240 100644--- a/docs/ROADMAP.md+++ b/docs/ROADMAP.md@@ -26,6 +26,7 @@ Decoding is bound by memory bandwidth: tokens per second ≈ effective bandwidth- [x] ps5-payload-dev SDK v0.43, pinned by checksum (`scripts/setup-sdk.sh`)- [x] macOS host toolchain: Homebrew `llvm@21` and `lld` (`scripts/env.sh`)- [x] `probes/memprobe` builds+- [x] `scripts/host-relapse.sh` serves the exploit page from macOS or Linux; `scripts/console-setup.sh` sends ftpsrv, klogsrv and shsrv after it- [ ] Console jailbroken (Relapse, firmware 13.60 or lower), with elfldr (9021), ftpsrv (2121) and klogsrv (3232)- [ ] `scripts/send.sh probes/memprobe/memprobe.elf`: record the results in `docs/CONSOLE.md`- direct, flexible and malloc ceilings for one payload process@@ -49,6 +50,18 @@ Expect 1 to 3 tok/s here. Slow, but it is Qwen 3.8 running on a PS5, and the cla**Done when** Qwen 3.8 answers a prompt on the console.+## Phase 1.5: the launcher and the model library++An Ollama-like way to get models and chat, without a PC in the loop. One payload, `ps5lm.elf`, kept in the loader's autoload:++- [ ] A small web server (port 8090) with the **model library** page, opened in the PS5's own browser on start (`sceSystemServiceLaunchWebBrowser`, as the SDK's `browser` sample and PS5SX2 do). The same page works from a phone or laptop on the network.+- [ ] The library: the curated list in [MODELS.md](MODELS.md) as `models.json`, each marked fits / tight / too big from the memprobe numbers, plus whatever is already in `/data/PS5LM/models` or on a USB drive.+- [ ] **Download** from Hugging Face straight to `/data/PS5LM/models`, with progress and resume. HTTPS through OpenSSL from ps5-payload-dev's pacbrew repository, or Sony's own `libSceHttp2` and `libSceSsl`. The same TLS turns on llama.cpp's `-hf` downloads.+- [ ] **Run**: start `llama-server` with the chosen model through the local elfldr (`127.0.0.1:9021`), stopping the one before. llama-server's own router mode spawns child processes, which a payload cannot do, so the launcher does the routing.+- [ ] **Chat** in llama-server's built-in web UI, on the TV through the PS5 browser or from any device on the network, and the OpenAI-compatible API for other apps. If the PS5's WebKit is too old for that UI, the launcher serves a plain chat page instead.++**Done when** a model can be picked, downloaded and chatted with using only the DualSense.+## Phase 2: fit and speed on the CPU- [ ] Weights in direct memory if malloc's ceiling is too low: a ggml CPU buffer type on `sceKernelAllocateDirectMemory`, as a patch in `patches/`diff --git a/scripts/console-setup.sh b/scripts/console-setup.shnew file mode 100755index 0000000..b8479ab--- /dev/null+++ b/scripts/console-setup.sh@@ -0,0 +1,36 @@+#!/usr/bin/env bash+# Sends the helper payloads PS5LM needs to a freshly jailbroken console:+# ftpsrv (FTP on 2121), klogsrv (kernel log on 3232) and shsrv (shell on 2323).+# Run it after the jailbreak, once elfldr listens on 9021. They stay up until+# the console reboots.+#+# PS5_HOST=192.168.1.50 scripts/console-setup.sh+set -euo pipefail++root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"+host=${PS5_HOST:?set PS5_HOST to the console IP}+port=${PS5_PORT:-9021}+dir="$root/.deps/payloads"+mkdir -p "$dir"++payloads=(+ "ftpsrv/v0.21.1/ftpsrv-ps5.elf"+ "klogsrv/v0.9/klogsrv-ps5.elf"+ "shsrv/v0.20/shsrv-ps5.elf"+)++for p in "${payloads[@]}"; do+ repo=${p%%/*}+ rest=${p#*/}+ tag=${rest%%/*}+ name=${rest#*/}+ if [ ! -f "$dir/$name" ]; then+ curl -fsSL -o "$dir/$name" "https://github.com/ps5-payload-dev/$repo/releases/download/$tag/$name"+ fi+ # -u: send the file and hang up; the payload keeps running on the console.+ socat -u "FILE:$dir/$name" "TCP:$host:$port"+ echo "console-setup: sent $name"+ sleep 1+done++echo "console-setup: FTP ftp://$host:2121, kernel log: nc $host 3232, shell: nc $host 2323"diff --git a/scripts/host-relapse.sh b/scripts/host-relapse.shnew file mode 100755index 0000000..54cdad4--- /dev/null+++ b/scripts/host-relapse.sh@@ -0,0 +1,33 @@+#!/usr/bin/env bash+# Serves the Relapse exploit page (firmware 7.00 to 13.60) from this machine,+# for opening in the PS5's browser on the same network. Relapse's own serve.py+# finds the local IP with Windows' ipconfig, so this does the same on macOS+# and Linux. After a successful run, elfldr listens on the console's port 9021.+#+# scripts/host-relapse.sh (port 8000)+set -euo pipefail++root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"+dir="$root/.deps/Relapse-Exploit"+port=${RELAPSE_PORT:-8000}++if [ ! -d "$dir/.git" ]; then+ git clone --depth 1 https://github.com/ntfargo/Relapse-Exploit "$dir"+else+ git -C "$dir" pull -q --ff-only+fi++ip=""+if [ "$(uname -s)" = "Darwin" ]; then+ for ifc in en0 en1 en2; do+ ip=$(ipconfig getifaddr "$ifc" 2>/dev/null || true)+ [ -n "$ip" ] && break+ done+else+ ip=$(hostname -I 2>/dev/null | awk '{print $1}')+fi++echo "host-relapse: open http://${ip:-<this machine>}:$port/ in the PS5's browser"+echo "host-relapse: if the browser stalls, reload; if the console hangs, reboot and retry"+cd "$dir"+exec python3 -m http.server "$port" --bind 0.0.0.0