Skip to content

armgit / PS5LM

Public Imported from GitHub
0

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

docs: what ran on a console for v0.11.0

Mert Cobanov05d592c747a3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1d502a5..f2f5f73 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,7 +19,10 @@ v0.1.x are the upstream payload releases by cobanov.
- **Ternary Bonsai 2**: a **Bonsai 2** preset (PrismML's sampling, 64k context at most) and
**Bonsai 2 27B** in Suggested, upstream's `Q2_0` (7.1 GiB). The app's llama.cpp already reads the
`prism.hadamard.*` rotation those files carry: checked on a PC with the 2B and the 27B, on the CPU
- and on ggml-vulkan with the console's shader paths. Not yet run on the console.
+ and on ggml-vulkan with the console's shader paths. On a console, fetched from Suggested (the
+ download carried on across two restarts of the app): a 64k context with a q8_0 cache, 1.36 GiB
+ of the pool left once it serves, about 17 tokens a second, right answers with and without
+ thinking.
- `scripts/bonsai-pq2-to-q2.py`: PrismML's `PQ2_0` files, which upstream llama.cpp (b11827 too)
refuses, to `Q2_0`, with no loss: each 128-weight block becomes two 64-weight blocks with the same
scale and codes. Python only, about 80 s for the 27B; the 2B and the 27B come out byte for byte
@@ -34,7 +37,10 @@ v0.1.x are the upstream payload releases by cobanov.
storage, or how to open full storage (OnionHEN's `exact_title_ids`, etaHEN, other HENs) and start
again, which asks the HEN once more. **Settings > Sound and storage > Where PS5LM keeps its
files** shows which one is in use and how full it is. Models saved there stay in the library
- once full storage opens.
+ once full storage opens. Checked on a console with `/data` made unwritable to the app: the
+ question on the screen, a model searched for, fetched to `/download0/PS5LM/models` and loaded
+ from there, the choice kept across the restart, and the model still in the library once `/data`
+ opened again.
- Two probes for later decisions, out of the app's way (`ps5/app/probes.cpp`, run from the PC):
`scripts/ps5lm-app.sh bench-read <model file>` measures a drive's sequential and random read
@@ -46,7 +52,9 @@ v0.1.x are the upstream payload releases by cobanov.
blank before. The API page lists the endpoint, and docs/CLI.md says what it takes.
- **A PC's GPU as a fleet worker.** `PS5LM_FLEET_GPU=metal` (or `cuda`, `vulkan`)
`scripts/build-fleet-host.sh` adds that backend, so `ggml-rpc-server -d MTL0` serves the PC's
- GPU next to the consoles. The default host build stays CPU only.
+ GPU next to the consoles. The default host build stays CPU only. Run with a console: a PS5 and a
+ Mac's GPU (Metal), and a PS5 and an RTX 3090 over CUDA and over Vulkan, each serving
+ `qwen2.5-0.5b-instruct-q4_k_m.gguf` with blocks 0–19 on the PS5. See docs/FLEET.md.
- `scripts/ps5lm-app.sh launch` works again, with PS5LM's own launcher payload
(`ps5/app/launch_app.c`, built by `build-app.sh`) instead of the removed
ps5-homebrew-dev-protocol.
@@ -64,12 +72,15 @@ v0.1.x are the upstream payload releases by cobanov.
`server-context.cpp`. It builds and links for the console as before.
- The planner counts the compute buffers' growth with the prompt batch (`-ub`, about 38 bytes a
token per unit of feed-forward width, measured with ggml-vulkan on the console's shader paths),
- and the batch for language models is one constant, `kPromptUbatch`, still 64 until
- `scripts/bench-ubatch.py` has measured the others on a console. See docs/CONSOLE.md.
+ and the batch for language models is one constant, `kPromptUbatch`, 64. `scripts/bench-ubatch.py`
+ measured the others on a console (Qwen 3.8 27B UD-IQ2_XXS, 64k, 2,048-token prompt): 124.6
+ tokens a second at 64, 107.1 at 128, 122.8 at 256, 102.0 at 512, and the larger sizes leave less
+ memory, so it stays 64. See docs/CONSOLE.md.
- `downloadDataSize` in `sce_sys/param.json` is 16 GiB (16384 MiB, was 256 MiB): the size of
the app's own storage. The app counts what it holds itself, as a title's `statvfs` is a stub.
- Whether the system sets the whole 16 GiB aside on internal storage at the first start, or
- only what is written, is not known yet; lowering it later frees nothing.
+ The system makes it an 18.8 GB image, `/user/download/PPSA99575/download0.dat`, at the first
+ start; whether all of that is taken on the drive at once was not measured. Lowering it later
+ frees nothing.
- A download is refused before it starts, with the reason on the TV, when the drive has no room
for it, the downloads already queued to that drive, and 1 GiB to spare. A model and its image
projector are queued together or not at all.
@@ -80,13 +91,19 @@ v0.1.x are the upstream payload releases by cobanov.
neither the GL stack nor RADV uses what the blocks configure. The planner reads the memory free,
so every model's plan grows with it: with the dashboard up, v0.9.1 had 11.04 GiB free before a
model and gave Qwen 3.8 27B UD-Q2_K_XL a 32k context, not the 64k it was suggested with; with
- the 12 GiB pool it plans 64k again. Not yet measured on a console: `app.log` now records the pool
- and the flexible memory at start (`memory at start`) and once the model serves (`memory when
- ready`).
+ the 12 GiB pool it plans 64k again. On a console the pool is 12.0000 GiB and the planner has
+ 11.6 GiB for a model (11.0 before); `app.log` records the pool and the flexible memory at start
+ (`memory at start`) and once the model serves (`memory when ready`).
- Every line the app writes to `app.log` starts with the console's local time, as
`22:56:01 ps5lm-app: start` (#40). Lines from the graphics stack keep their own form.
### Fixed
+- **A black screen on OnionHEN** with `PPSA99575` in `exact_title_ids` (#23, #32, #54). OnionHEN
+ opens `/data` with a full app jailbreak that also moves the process out of its sandbox, so
+ `/app0` and `/download0` were gone and the fonts did not load. Once `/data` is open, the app
+ finds its sandbox in `/mnt/sandbox/<title id>_<slot>` and reads its own files from there;
+ `app.log` says when it runs outside its sandbox. Unchanged on etaHEN, which keeps the sandbox.
+ Not yet run on an OnionHEN console.
- A download that meets a full drive anyway stops at once, "Failed: App storage is full", instead
of retrying twenty times.
- At the end of a download, Models > Downloads said "starting" for the minutes the SHA-256 check
@@ -98,18 +115,24 @@ v0.1.x are the upstream payload releases by cobanov.
not first. `patches/0004` folded such messages into the first one only for the Anthropic API
(Claude Code); it now does so for Chat Completions, Responses and Anthropic alike, with
`tests/server_system_messages_test.py` checking all three on a host build. docs/CLI.md's codex
- setup is the one codex 0.166 reads: the profile in its own `~/.codex/ps5.config.toml`.
+ setup is the one codex 0.166 reads: the profile in its own `~/.codex/ps5.config.toml`. Checked
+ with codex 0.166 against a console.
- Models with sliding-window attention got far less context than fits (#57): the planner counted
every attention layer at full length. It now reads `<arch>.attention.sliding_window_pattern` and
counts the window layers at their window, as llama.cpp keeps them. Spark-X2.6 4B (27 of 36 layers
- keep 512 tokens) plans 256k with a q8_0 cache instead of 128k with q4_0. **Longest context** in
- Settings goes to 192k and 256k, and the planner tries both.
+ keep 512 tokens) plans 256k with a q8_0 cache instead of 128k with q4_0; on a console the server
+ took the 262,144-token context with 2 GiB of the pool left, and found a needle in a 52,821-token
+ prompt. **Longest context** in Settings goes to 192k and 256k, and the planner tries both.
- A console serving a model went into rest mode an hour after the last button press: requests
from the network are not controller input. Rest mode ends the app, and on a jailbroken console the
controllers do not connect again until a restart. While a model is loaded, a download is queued
or running, or the app is a fleet worker, it now tells the system every 30 s that it is in use
(`sceSystemServicePowerTick`, as a video player does); with nothing to do, the console's own
rest-mode setting applies. `app.log` says when it starts doing so.
+- `scripts/bench-ubatch.py` took the old app's last lines, as it stopped, for the restarted one,
+ so a size that loads was reported as not loading, or its prompt went to the stopping server;
+ it never removed `app-args.txt` (ftpsrv answers a delete with 226, which ftplib refuses), and an
+ error mid-run left the test arguments for every later start. All three are fixed.
## [0.10.0] - 2026-11-01
diff --git a/README.md b/README.md
index 6786f52..e859d23 100644
--- a/README.md
+++ b/README.md
@@ -200,7 +200,8 @@ memory, not a copy on each, so a model too large for one PS5 can run on two or
more. The PC can add its CPU or GPU as another worker. The worker is part of the
app and starts only when `/data/PS5LM/fleet-worker.txt` asks for it (v0.10.0 had
it in a separate `ps5lm-app-fleet` zip). Two-worker CPU inference is tested in
-CI, and a PS5 worker has served a model together with a PC, but it is
+CI, and a PS5 worker has served a model together with a PC's CPU, a Mac's GPU and an
+RTX 3090 (CUDA and Vulkan), but it is
**experimental**: Qwen 3.8's `qwen35` layout is not admitted by
this first planner yet, and RPC has no authentication, so it belongs on an
isolated network you trust. See [validation and limits](docs/FLEET-VALIDATION.md).
diff --git a/docs/FLEET.md b/docs/FLEET.md
index 953a771..d02b9b0 100644
--- a/docs/FLEET.md
+++ b/docs/FLEET.md
@@ -2,9 +2,9 @@
**Status: host controller, and a PS5 worker that is part of the app and starts
only when you opt in; CPU RPC inference and the title's cross-build are tested
-in CI. On hardware, one PS5 worker has served one model together with a Linux
-CPU worker (2026-11-01, below); more consoles at once are not validated by the
-maintainers.** This is not a performance claim. Up to v0.10.0 the worker was a
+in CI. On hardware, one PS5 worker has served one model together with a PC's
+CPU, a Mac's GPU (Metal) and an RTX 3090 (CUDA and Vulkan) (2026-11-01 and
+11-02, below); more consoles at once are not validated by the maintainers.** This is not a performance claim. Up to v0.10.0 the worker was a
separate build, `ps5lm-app-fleet-v0.10.0.zip`; from v0.11.0 every PS5LM has it.
What it is for: a model that does not fit in one console's memory runs across
@@ -183,6 +183,14 @@ M-series Mac, the controller on the Linux PC. Blocks 0–22 on the PS5, block 23
and the output on the Mac's GPU; it answered correctly at 13 tokens/s (7.4 with
the PC's CPU in that place). One small model, so not a performance figure.
+On 2026-11-02, the same PS5 worker with an RTX 3090 on the Linux PC, from
+`PS5LM_FLEET_GPU=cuda` and `vulkan` host builds (`ggml-rpc-server -d CUDA0`,
+`-d Vulkan0`), the controller on the same PC: blocks 0–19 on the PS5, blocks
+20–23 and the output on the 3090 (`--tensor-split 39,11`). Both answered
+correctly, at 28 to 30 tokens/s on a longer reply. The controller's binary must
+run on the PC it is started on: a `build/fleet-host` made inside a newer Linux
+container stops at `GLIBC_2.43 not found` on an older host.
+
## Configure workers (only after building and validating the title)
On each console, explicitly create `/data/PS5LM/fleet-worker.txt` containing
diff --git a/site/index.html b/site/index.html
index 0134f83..03d2090 100644
--- a/site/index.html
+++ b/site/index.html
@@ -281,12 +281,12 @@
<p class="sub">What people ask before they install it</p>
<ul class="points two">
<li><b>Does it run on the GPU?</b><span>Yes, every layer. llama.cpp's Vulkan backend runs on RADV, the open Mesa driver, ported to the PS5 by PS5_Vulkan.</span></li>
- <li><b>How big a model can it run?</b><span>About 11 GB of model and context. Qwen 3.8 27B at 2-bit fits with a 64k context; the planner tells you what fits before you download.</span></li>
+ <li><b>How big a model can it run?</b><span>About 11.6 GB of model and context. Qwen 3.8 27B at 2-bit fits with a 64k context; the planner tells you what fits before you download.</span></li>
<li><b>Does this run games or need any Sony code?</b><span>No. PS5LM runs open-weight models and nothing else. No games, keys, firmware or Sony SDK files are part of it.</span></li>
<li><b>Can it break my console?</b><span>It writes only to its own folders, never to system files. Under a long load it runs warm and the fan spins up, as in a game; the Console page shows both.</span></li>
<li><b>Is it the same as PS5LM Plus?</b><span>PS5LM grew from PS5LM Plus, the native GPU app by abhi245y, and now lives under its own title ID, PPSA99575, so the two install side by side.</span></li>
<li><b>Which model for coding agents?</b><span>A smaller model with a long context: prompts are read several times faster. <a href="https://github.com/cobanov/PS5LM/blob/main/docs/CLI.md">docs/CLI.md</a> has the trade-offs.</span></li>
- <li><b>Can several consoles run one model?</b><span>Experimentally, with the fleet build, a separate zip. A PC runs the controller and each PS5 holds a run of one dense llama or qwen2 model's layers on its GPU. So far one PS5 has served a model together with a PC. RPC has no authentication, so keep it on an isolated network: <a href="https://github.com/cobanov/PS5LM/blob/main/docs/FLEET.md">docs/FLEET.md</a>.</span></li>
+ <li><b>Can several consoles run one model?</b><span>Experimentally, and every PS5LM can be a worker. A PC runs the controller and each PS5 holds a run of one dense llama or qwen2 model's layers on its GPU; the PC can add its CPU or its GPU (Metal, CUDA or Vulkan) as one more worker. So far one PS5 has served a model together with a PC's CPU, a Mac's GPU and an RTX 3090. RPC has no authentication, so keep it on an isolated network: <a href="https://github.com/cobanov/PS5LM/blob/main/docs/FLEET.md">docs/FLEET.md</a>.</span></li>
<li><b>Can I see inside the model?</b><span>If you turn it on. <code>POST /v1/introspect</code> returns the logits where you ask, the top candidates and one layer's activations, and can zero, scale or add to a channel. It is off by default, under Settings &gt; Developer: <a href="https://github.com/cobanov/PS5LM/blob/main/docs/INTROSPECTION.md">docs/INTROSPECTION.md</a>.</span></li>
</ul>
</section>
@@ -337,7 +337,7 @@
</div>
<div class="legal">
<span>Made by <a href="https://cobanov.dev">Mert Cobanov</a></span>
- <span>v0.10.0. Not affiliated with Sony Interactive Entertainment; PlayStation and PS5 are its trademarks</span>
+ <span>v0.11.0. Not affiliated with Sony Interactive Entertainment; PlayStation and PS5 are its trademarks</span>
</div>
</div>
</footer>
diff --git a/site/llms.txt b/site/llms.txt
index 8a12fb1..2a2f92a 100644
--- a/site/llms.txt
+++ b/site/llms.txt
@@ -16,10 +16,10 @@ Key facts:
- Needs a PS5 on firmware 7.00 to 13.60 with kstuff-lite, etaHEN and ShadowMountPlus; elfldr and ftpsrv are
recommended. Models live in `/data/PS5LM/models` or `PS5LM/models` on a USB drive.
- Measured on a PS5 Slim: Qwen 3.5 9B Q4_K_M at 55 tokens a second, Qwen 3.8 27B UD-Q2_K_XL at about 20 with a
- 64k context, d1 Omni 600M answers three questions in 0.3 s. About 11 GB of model and context fits.
-- Fleet (experimental, since v0.10.0): a separate build, `ps5lm-app-fleet-vX.Y.Z.zip`, splits one dense `llama`
- or `qwen2` GGUF across several consoles by layers over llama.cpp's RPC. A Linux or macOS PC runs the controller
- and can add its CPU as a worker. RPC has no authentication: isolated networks only.
+ 64k context, d1 Omni 600M answers three questions in 0.3 s. About 11.6 GB of model and context fits.
+- Fleet (experimental, since v0.10.0; part of every build from v0.11.0): splits one dense `llama` or `qwen2`
+ GGUF across several consoles by layers over llama.cpp's RPC. A Linux or macOS PC runs the controller and can
+ add its CPU or its GPU (Metal, CUDA, Vulkan) as a worker. RPC has no authentication: isolated networks only.
## Docs