Skip to content

armgit / PS5LM

Public Imported from GitHub
0

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

llama.cpp tools link for the PS5; add roadmap and research notes

Mert Cobanovb2cc03715ff1
diff --git a/README.md b/README.md
index d6798af..9325743 100644
--- a/README.md
+++ b/README.md
@@ -4,7 +4,7 @@
The first target is **Qwen 3.8** (27B, hybrid Gated DeltaNet attention), the newest open Qwen. No one has run it on a PS5 yet.
-> Status: early work, nothing runs on the console yet. See [docs/ROADMAP.md](docs/ROADMAP.md).
+> **Status:** `llama-cli`, `llama-server` and `llama-bench` cross-compile and link for the PS5 with no changes to llama.cpp's source. Nothing has run on a console yet: that is next. See [docs/ROADMAP.md](docs/ROADMAP.md).
## Why llama.cpp
@@ -15,10 +15,10 @@ Earlier PS5 LLM work hand-writes GPU kernels for one model at a time. PS5LM port
| Path | What |
|---|---|
| `third_party/llama.cpp` | upstream llama.cpp, pinned as a submodule |
-| `patches/` | the PS5 changes to llama.cpp, kept small |
+| `ps5/compat` | the few libc functions the console lacks |
| `probes/memprobe` | Phase 0: measures memory, bandwidth and compute on the console |
| `scripts/` | SDK setup, the llama.cpp cross build, sending payloads |
-| `docs/` | roadmap and research notes |
+| `docs/` | [roadmap](docs/ROADMAP.md), [porting notes](docs/PORTING.md), [research](docs/RESEARCH.md) |
## Building
@@ -36,7 +36,7 @@ scripts/setup-sdk.sh
source scripts/env.sh
make -C probes/memprobe # Phase 0 probe
-scripts/build-llama.sh # llama.cpp for the PS5 (work in progress)
+scripts/build-llama.sh # llama-cli, llama-server, llama-bench for the PS5
```
## Running on the console
@@ -50,6 +50,14 @@ scripts/send.sh probes/memprobe/memprobe.elf
The probe streams its results back and also writes them to `/data/PS5LM/memprobe.txt` on the console.
+With [ftpsrv](https://github.com/ps5-payload-dev/ftpsrv) running too (port 2121), `scripts/run.sh` uploads a llama.cpp tool and starts it with arguments:
+
+```sh
+# copy a model to /data/PS5LM/models first (FTP or USB)
+scripts/run.sh llama-cli -m /data/PS5LM/models/model.gguf -p "Hello from a PS5" -n 64
+scripts/run.sh llama-server -m /data/PS5LM/models/model.gguf --host 0.0.0.0 --port 8081
+```
+
## Credits
- [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp), which does the actual work.
diff --git a/docs/PORTING.md b/docs/PORTING.md
new file mode 100644
index 0000000..32b2018
--- /dev/null
+++ b/docs/PORTING.md
@@ -0,0 +1,32 @@
+# Porting notes
+
+What it took to build llama.cpp for the PS5, and why. Upstream is pinned at `b11327` (2026-10-01).
+
+## The target
+
+The SDK compiles with clang's own PS5 target, `x86_64-sie-ps5`, which defines `__PROSPERO__`, `__SCE__` and `__FreeBSD__ 9`. Its CMake toolchain (`prospero-cmake`) presents the console as FreeBSD on x86_64. Programs link dynamically against stubs of Sony's libraries (`libSceLibcInternal`, `libkernel_web`, `libSceNet`, ...) and run as payloads under elfldr.
+
+ggml and libllama build for it as they are: no source changes so far.
+
+## Build fixes (all in `scripts/build-llama.sh`)
+
+1. **`posix_spawn_file_actions_addchdir_np` is missing.** The SDK's libc predates it, and `common/subproc.cpp` (through `vendor/sheredom/subprocess.h`) calls it. The header lets the caller turn it off: `-DSUBPROCESS_HAVE_CWD=0`. Nothing on the console spawns processes.
+2. **There is no `libm`.** ggml links `m` unless `MATH_LIBRARY` is set; the math functions live in Sony's libc, so `-DMATH_LIBRARY=SceLibcInternal`.
+3. **`accept4` is declared but not exported.** The SDK's `sys/socket.h` declares it (FreeBSD 10), libkernel exports only `accept`, and cpp-httplib's server loop calls `accept4(..., SOCK_CLOEXEC)`. `ps5/compat/compat.c` implements it on `accept` and `fcntl`, and goes last on every link line through `CMAKE_C_STANDARD_LIBRARIES` and `CMAKE_CXX_STANDARD_LIBRARIES`.
+
+Build options: AVX, AVX2, FMA, F16C and BMI2 on, AVX-512 off (Zen 2), no OpenMP (ggml's own thread pool), static libraries, no OpenSSL.
+
+## Output
+
+`build/llama-ps5/bin/`: `llama-cli` (20 MB), `llama-server` (20 MB), `llama-bench` (13 MB), position independent FreeBSD ELFs that import from `libSceLibcInternal.sprx`, `libkernel_web.sprx` and `libSceNet.sprx`. The only weak import is `__cxa_thread_atexit_impl`, which libc++abi checks for before use.
+
+`llvm-readelf` warns that `PT_DYNAMIC` is larger than `.dynamic`: the SDK's linker script puts `.dynsym`, `.dynstr` and `.rela.dyn` in the same segment, and the SDK's own samples have the same layout.
+
+## Not verified yet
+
+None of this has run on a console. Things to watch in Phase 0 and 1:
+
+- whether `mmap` of a multi-GiB model file works in a payload, or llama.cpp needs `--no-mmap`
+- whether a single `posix_memalign` of 7 to 9 GiB succeeds
+- how many cores the payload process may use, and whether `std::thread::hardware_concurrency()` reports them
+- stdout buffering through elfldr's socket
diff --git a/docs/RESEARCH.md b/docs/RESEARCH.md
new file mode 100644
index 0000000..d8b1482
--- /dev/null
+++ b/docs/RESEARCH.md
@@ -0,0 +1,41 @@
+# Research notes
+
+What is known about the platform and the prior work, as of October 2026, with sources.
+
+## The console
+
+- **CPU:** 8 Zen 2 cores (16 threads) at up to 3.5 GHz, 3.85 GHz on the Pro. AVX2, FMA, F16C, BMI2; no AVX-512.
+- **Memory:** 16 GB GDDR6 shared by CPU and GPU, 448 GB/s peak (576 GB/s on the Pro). How much of the CPU side bandwidth a process sees is a Phase 0 measurement.
+- **GPU:** RDNA 2 class, programmed through Sony's private AGC interface. There is no public graphics or compute API.
+- **OS:** a FreeBSD derivative. Big allocations go through Sony's direct memory (`sceKernelAllocateDirectMemory`, `sceKernelMapDirectMemory`). In a native app the flexible memory pool behind ordinary allocations is small (see PS5SX2 below); for a payload process, Phase 0 will tell.
+
+## The jailbreak
+
+[Relapse](https://github.com/ntfargo/Relapse-Exploit) (ntfargo, 29 September 2026) covers firmware 7.00 to 13.60 on the PS5 and the PS5 Pro. It starts in the browser (a JavaScriptCore info leak and a structured clone object-pool mismatch that corrupts a typed array) and reaches kernel read/write through a use-after-free race in `aio_multi_wait`. It does not survive a reboot. Firmware 14.00 (16 September 2026) closes it.
+
+On top of it: kstuff (fake-signed executables), a HEN (etaHEN or OnionHEN) and [elfldr](https://github.com/ps5-payload-dev/elfldr), which runs ELF payloads sent to port 9021. Tools from [ps5-payload-dev](https://github.com/ps5-payload-dev): ftpsrv (2121), klogsrv (3232), shsrv (2323), gdbsrv, websrv (8080).
+
+## The SDK
+
+[ps5-payload-dev/sdk](https://github.com/ps5-payload-dev/sdk) v0.43 (29 August 2026). Clang's `x86_64-sie-ps5` target with any host LLVM from 15 to 21; macOS works with Homebrew's LLVM. Ships libc++, a CMake toolchain and Meson cross files. Payloads take arguments through elfldr as `file:/path/to/elf?args=...`.
+
+## Prior work this project builds on
+
+**[PS5SX2](https://github.com/Swordpdf/PS5SX2)**, PCSX2 on the PS5 (September 2026). The best public notes on the console's memory as a native app sees it:
+
+- `mmap` in the app sandbox fails; flexible memory (`sceKernelMapFlexibleMemory`) is only ~190 to 448 MiB
+- large data goes in direct memory; the Vulkan driver offers a 12 GiB device heap with `PS5VK_WIDE_MEMORY`
+- JIT and other rights come from the HEN: the app writes `{"PID":n}` to `/download0/etahen_jailbreak`
+
+**[ProsperoAI](https://github.com/blackbearreloaded/ProsperoAI)** (GPL-3.0), LLM chat on the PS5 GPU (September 2026). Runs Qwen3.5 9B Q4_0 (6.37 GiB) at about 30 tok/s decode and Mistral 7B Q4_0 at 25 to 29 tok/s, with a 4,096 token context. It writes AGC kernels by hand for each model (`src/backends/qwen35`, `src/backends/mistral`) from per-model recipes and supports Q4_0 only, so a new model or quant means new kernels. It vendors llama.cpp's CPU libraries built for the PS5. Proof that a 6+ GiB model and fast GPU decode work on this console.
+
+**[PS5_Vulkan](https://github.com/mihawk-99/PS5_Vulkan)** (Mihawk-99), a Vulkan driver for the PS5's GPU built on Mesa 26.2's RADV. Shaders go SPIR-V → NIR → ACO → AGC (the PSBC compiler from the ps5-opengl SDK). Reports Vulkan 1.4 and 1,569,390 passing CTS cases with no failures; no async compute queues yet. Runs vkQuake at 120 fps in 4K, RetroArch's Vulkan cores and the Eden emulator.
+
+**[ps5-native-app-boilerplate](https://github.com/blackbearreloaded/ps5-native-app-boilerplate)** (GPL-3.0): the build of a native app (`eboot.bin` as an fSELF, `sce_sys/param.json`, `sce_module/libc.prx`) for ShadowMountPlus.
+
+## Qwen 3.8
+
+- **Qwen3.8-27B** (14 August 2026, Apache 2.0): dense, hybrid attention (Gated DeltaNet and gated attention), vision encoder, 262K context, a native MTP block. GGUFs from [unsloth](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) use the `qwen35` architecture and need llama.cpp from its release week or later.
+- **Qwen3.8-Flash-Next**: 180B total, 6B active (MoE, 512 experts).
+- **Qwen3.8 2.4T-A95B** and the hosted **Qwen3.8-Max**.
+- No model below 27B in this generation.
diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md
new file mode 100644
index 0000000..1bb878c
--- /dev/null
+++ b/docs/ROADMAP.md
@@ -0,0 +1,98 @@
+# PS5LM roadmap
+
+The goal: llama.cpp running natively on a jailbroken PS5, able to load any GGUF model, with **Qwen 3.8 27B** as the first model to run on the console.
+
+The plan goes CPU first, GPU second. The CPU path needs nothing but the payload SDK and gets Qwen 3.8 talking on the console soonest. The GPU path is where the speed is, and it is the part nobody has done with llama.cpp yet.
+
+## The model and the memory
+
+Qwen 3.8's smallest open model is the 27B (dense, hybrid Gated DeltaNet and gated attention, GGUF architecture `qwen35`). The PS5 has 16 GB of GDDR6 shared by the CPU, the GPU and the system, so only the low-bit quants fit ([unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)):
+
+| Quant | Size | Role |
+|---|---|---|
+| UD-IQ1_M | 6.27 GiB | fallback if memory is tight |
+| **UD-IQ2_XXS** | 6.77 GiB | first light candidate |
+| UD-IQ2_S | 7.80 GiB | |
+| **UD-Q2_K_XL** | 9.15 GiB | main target: K-quants are the cheapest to decode on CPU and GPU |
+| UD-IQ3_XXS | 10.18 GiB | stretch, if a process can hold ~11 GiB |
+| UD-Q4_K_M | 15.33 GiB | does not fit |
+
+Only one layer in four keeps a KV cache (the others are Gated DeltaNet with a fixed state), so long contexts stay cheap. The vision encoder (`mmproj`, 0.86 GiB) is left out until the text model runs.
+
+Decoding is bound by memory bandwidth: tokens per second ≈ effective bandwidth / bytes read per token (about the model size). The GDDR6 peaks at 448 GB/s (576 GB/s on the Pro), so the GPU ceiling for Q2_K_XL is around 45 tok/s. What the CPU can pull is what Phase 0 measures.
+
+## Phase 0: toolchain and console facts
+
+- [x] ps5-payload-dev SDK v0.43, pinned by checksum (`scripts/setup-sdk.sh`)
+- [x] macOS host toolchain: Homebrew `llvm@21` and `lld` (`scripts/env.sh`)
+- [x] `probes/memprobe` builds
+- [ ] Console jailbroken (Relapse, firmware 13.60 or lower), with elfldr (9021), ftpsrv (2121) and klogsrv (3232)
+- [ ] `scripts/send.sh probes/memprobe/memprobe.elf`: record the results in `docs/CONSOLE.md`
+ - direct, flexible and malloc ceilings for one payload process
+ - the largest single allocation (llama.cpp's CPU backend puts all weights in one buffer)
+ - read bandwidth with 1 to 8 threads, on malloc and on direct memory
+ - AVX2 FMA GFLOPS
+ - file read and mmap speed on a GGUF in `/data/PS5LM/models`
+
+**Done when** we know how many GiB one process can hold and how fast it can read them.
+
+## Phase 1: llama.cpp on the CPU, first light
+
+- [x] ggml and libllama cross-compile with no source changes
+- [x] `llama-cli`, `llama-server` and `llama-bench` link ([PORTING.md](PORTING.md) lists the three build fixes)
+- [ ] Smoke test with a small model (Qwen3.5 0.8B or SmolLM2) through `scripts/run.sh llama-cli`
+- [ ] `llama-server` on the console: chat from a laptop or phone with llama.cpp's built-in web UI (port 8081, since websrv takes 8080)
+- [ ] `llama-bench` numbers for the small model
+- [ ] **First light: Qwen3.8-27B generates text on the PS5.** UD-Q2_K_XL if Phase 0 says it fits, UD-IQ2_XXS if not. A video of it, posted with the repo.
+
+Expect 1 to 3 tok/s here. Slow, but it is Qwen 3.8 running on a PS5, and the claim we want first.
+
+**Done when** Qwen 3.8 answers a prompt on the console.
+
+## Phase 2: fit and speed on the CPU
+
+- [ ] Weights in direct memory if malloc's ceiling is too low: a ggml CPU buffer type on `sceKernelAllocateDirectMemory`, as a patch in `patches/`
+- [ ] Loading: mmap or `--no-mmap`, from `/data` or a USB drive, whichever Phase 0 shows is faster
+- [ ] Threads: the best `-t` for the cores a payload is allowed, with pinning
+- [ ] Qwen 3.8's native MTP block for speculative decoding
+- [ ] Context budget: 8k to 32k tokens
+
+**Done when** Q2_K_XL runs at the best speed the CPU allows, and the numbers are in `docs/CONSOLE.md`.
+
+## Phase 3: the GPU through Vulkan
+
+The route: ggml's Vulkan backend on [Mihawk-99's PS5_Vulkan](https://github.com/mihawk-99/PS5_Vulkan), a RADV based driver that programs the PS5's GPU (AGC) directly and passes the Vulkan CTS. llama.cpp keeps its own kernels, so every model and quant comes along.
+
+- [ ] Build the driver on Linux (an OrbStack Ubuntu machine is enough)
+- [ ] App shell: `eboot.bin` with `sce_sys/param.json`, from [ps5-native-app-boilerplate](https://github.com/blackbearreloaded/ps5-native-app-boilerplate), mounted by ShadowMountPlus, jailbroken at start through the HEN request PS5SX2 uses
+- [ ] ggml-vulkan cross-compiled, its shaders compiled to SPIR-V on the host
+- [ ] `test-backend-ops` on the console: every op the `qwen35` graph uses matches the CPU
+- [ ] Weights in device memory (the driver's 12 GiB heap, `PS5VK_WIDE_MEMORY`)
+- [ ] Qwen3.8-27B decoding on the GPU, target 15 tok/s or more
+
+**Done when** the GPU decodes at least five times faster than the CPU.
+
+## Phase 4: the app
+
+- [ ] A chat screen on the TV, with the DualSense and the system keyboard
+- [ ] Model picker over `/data/PS5LM/models` and USB drives
+- [ ] `llama-server` always on, so the PS5 serves an OpenAI-compatible API to the home network
+- [ ] Releases: payload ELFs and the app folder, with an install guide
+
+**Done when** someone who is not a developer can install it and chat.
+
+## Later
+
+- Qwen3.8-Flash-Next (180B total, 6B active) with experts streamed from the SSD
+- Vision input for the 27B (`mmproj`)
+- Upstream the small portability fixes to llama.cpp
+
+## Risks
+
+| Risk | Plan B |
+|---|---|
+| A payload process gets far less than 7 GiB | Move to the native app shell (Phase 3's) sooner, where direct memory is larger; IQ1_M meanwhile |
+| The CPU is too slow for a 27B | Still first light; Phase 3 is the fix |
+| ggml-vulkan needs a feature the driver lacks | Its feature switches (`GGML_VK_DISABLE_*`), or fix it in the driver with its author |
+| Someone else gets there first | Keep Phase 1 short and publish as soon as it talks |
+| A firmware update closes the exploit | Keep the console on 13.60 or lower, automatic updates off |
diff --git a/ps5/compat/compat.c b/ps5/compat/compat.c
new file mode 100644
index 0000000..5185d4f
--- /dev/null
+++ b/ps5/compat/compat.c
@@ -0,0 +1,31 @@
+/* Functions the PS5's libc and libkernel don't export, built on ones they do.
+ * scripts/build-llama.sh links this archive last into every llama.cpp tool.
+ *
+ * SPDX-License-Identifier: GPL-3.0-or-later
+ */
+#include <errno.h>
+#include <fcntl.h>
+#include <sys/socket.h>
+#include <unistd.h>
+
+/* The SDK's headers declare accept4 (FreeBSD 10), but libkernel only exports
+ * accept. cpp-httplib's server loop calls accept4(..., SOCK_CLOEXEC). */
+int accept4(int s, struct sockaddr *__restrict addr, socklen_t *__restrict addrlen, int flags) {
+ const int fd = accept(s, addr, addrlen);
+ if (fd < 0)
+ return fd;
+ if ((flags & SOCK_CLOEXEC) && fcntl(fd, F_SETFD, FD_CLOEXEC) == -1)
+ goto fail;
+ if (flags & SOCK_NONBLOCK) {
+ const int fl = fcntl(fd, F_GETFL);
+ if (fl == -1 || fcntl(fd, F_SETFL, fl | O_NONBLOCK) == -1)
+ goto fail;
+ }
+ return fd;
+
+fail:;
+ const int saved = errno;
+ close(fd);
+ errno = saved;
+ return -1;
+}
diff --git a/scripts/build-llama.sh b/scripts/build-llama.sh
index 086d9ad..d601880 100755
--- a/scripts/build-llama.sh
+++ b/scripts/build-llama.sh
@@ -25,11 +25,23 @@ fi
# console spawns processes with a working directory, so subprocess.h goes without.
ps5_flags="-DSUBPROCESS_HAVE_CWD=0"
# There is no libm: the math functions are in Sony's libc (libSceLibcInternal).
+
+# ps5/compat fills in the libc functions the console lacks. It goes at the very
+# end of every link line (CMAKE_*_STANDARD_LIBRARIES), after the objects that
+# need it.
+compat_dir="$out/ps5compat"
+mkdir -p "$compat_dir"
+"$PS5_PAYLOAD_SDK/bin/prospero-clang" -O2 -Wall -Wextra -Werror -c \
+ "$root/ps5/compat/compat.c" -o "$compat_dir/compat.o"
+rm -f "$compat_dir/libps5compat.a"
+"$PS5_PAYLOAD_SDK/bin/prospero-ar" rcs "$compat_dir/libps5compat.a" "$compat_dir/compat.o"
"$PS5_PAYLOAD_SDK/bin/prospero-cmake" -S "$src" -B "$out" -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_FLAGS="$ps5_flags" \
-DCMAKE_CXX_FLAGS="$ps5_flags" \
-DMATH_LIBRARY=SceLibcInternal \
+ -DCMAKE_C_STANDARD_LIBRARIES="$compat_dir/libps5compat.a" \
+ -DCMAKE_CXX_STANDARD_LIBRARIES="$compat_dir/libps5compat.a" \
-DBUILD_SHARED_LIBS=OFF \
-DGGML_NATIVE=OFF \
-DGGML_BACKEND_DL=OFF \
diff --git a/scripts/run.sh b/scripts/run.sh
new file mode 100755
index 0000000..291fe45
--- /dev/null
+++ b/scripts/run.sh
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# Uploads a llama.cpp tool to the console over FTP and starts it through elfldr
+# with arguments, streaming its output back.
+#
+# scripts/run.sh llama-cli -m /data/PS5LM/models/model.gguf -p "Hello" -n 64
+# scripts/run.sh llama-server -m /data/PS5LM/models/model.gguf --host 0.0.0.0 --port 8081
+#
+# Needs elfldr (port 9021) and ftpsrv (port 2121) running on the console.
+set -euo pipefail
+
+root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
+tool=${1:?usage: run.sh TOOL [ARGS...]}
+shift
+host=${PS5_HOST:?set PS5_HOST to the console IP}
+port=${PS5_PORT:-9021}
+ftp_port=${PS5_FTP_PORT:-2121}
+remote_dir=/data/PS5LM/bin
+
+elf="$root/build/llama-ps5/bin/$tool"
+[ -f "$elf" ] || elf="$tool"
+[ -f "$elf" ] || { echo "run.sh: no $tool, build it first" >&2; exit 1; }
+name=$(basename "$elf")
+
+curl -sS --ftp-create-dirs -T "$elf" "ftp://$host:$ftp_port$remote_dir/$name"
+
+# elfldr splits args on spaces; a space inside one argument is escaped with a
+# backslash, and the whole string is URL-encoded.
+args=$(python3 -c '
+import sys, urllib.parse
+parts = [a.replace("\\", "\\\\").replace(" ", "\\ ") for a in sys.argv[1:]]
+print(urllib.parse.quote(" ".join(parts), safe=""))
+' "$@")
+
+echo "file:$remote_dir/$name?args=$args" | socat -t 9999999 - "TCP:$host:$port"