llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Site: v0.1 download, install steps and the new measurements
Mert Cobanov6fccd547e09f
diff --git a/site/assets/style.css b/site/assets/style.cssindex 8c60bd1..ae63c12 100644--- a/site/assets/style.css+++ b/site/assets/style.css@@ -262,3 +262,6 @@ li { margin: 0.4rem 0; }}.lessons { margin-top: 2.5rem; }++.steps-list { max-width: var(--measure); color: var(--ink-soft); padding-left: 1.3rem; margin: 0 0 1.25rem; }+.steps-list li { margin: 0.5rem 0; }diff --git a/site/index.html b/site/index.htmlindex d237780..e2b9436 100644--- a/site/index.html+++ b/site/index.html@@ -9,7 +9,7 @@<link rel="canonical" href="https://ps5lm.cobanov.dev/"><link rel="preload" as="font" type="font/woff2" href="/assets/fonts/recursive-sans-v1.woff2" crossorigin><link rel="preload" as="font" type="font/woff2" href="/assets/fonts/recursive-mono-v1.woff2" crossorigin>-<link rel="stylesheet" href="/assets/style.css?v=1">+<link rel="stylesheet" href="/assets/style.css?v=2"><meta property="og:type" content="website"><meta property="og:url" content="https://ps5lm.cobanov.dev/"><meta property="og:title" content="PS5LM: llama.cpp on a PlayStation 5">@@ -28,7 +28,7 @@<a href="#how">How it works</a><a href="#numbers">Numbers</a><a href="#roadmap">Roadmap</a>- <a href="#try">Try it</a>+ <a href="#install">Install</a><a href="#faq">FAQ</a></nav><a class="btn" href="https://github.com/cobanov/PS5LM">GitHub</a>@@ -42,10 +42,10 @@<h1 class="display">llama.cpp, running on a PlayStation 5.</h1><p class="lede">PS5LM runs open language models on the console's own CPU. You chat with them in the PS5's browser, typing with the DualSense.</p><div class="cta">- <a class="btn btn-primary" href="https://github.com/cobanov/PS5LM">Source on GitHub</a>- <a class="btn" href="#how">How it works</a>+ <a class="btn btn-primary" href="https://github.com/cobanov/PS5LM/releases/latest">Download v0.1</a>+ <a class="btn" href="https://github.com/cobanov/PS5LM">Source on GitHub</a></div>- <p class="hero-note">Open source under GPL-3.0. Runs on a PS5 you own, on firmware 13.60 or lower.</p>+ <p class="hero-note">One payload, <code>ps5lm.elf</code>: pick a model, it downloads on the console, chat. Open source under GPL-3.0, for a PS5 you own on firmware 13.60 or lower.</p></div><figure class="demo"><video id="demo" src="/media/ps5lm-demo.mp4" poster="/media/poster.jpg" muted loop playsinline preload="metadata"@@ -77,8 +77,8 @@<h2>Measured on the console</h2><div class="grid"><div class="stat">- <p class="value">13 to 22 tok/s</p>- <p>Qwen3.5 0.8B generating text, with the prompt read at 38 to 67 tokens a second.</p>+ <p class="value">13 to 21 tok/s</p>+ <p>Qwen3.5 0.8B generating text. Qwen3.5 2B does about 9.</p></div><div class="stat"><p class="value">5 of 16 cores</p>@@ -89,15 +89,16 @@<p>The memory one homebrew program can hold, from the pool the home screen also uses.</p></div><div class="stat">- <p class="value">4 seconds</p>- <p>To load a 0.5 GB model from the console's SSD.</p>+ <p class="value">12 MB/s</p>+ <p>Downloading a model from Hugging Face on the console, over 16 connections.</p></div></div><h3 class="lessons">What the console taught us</h3><ul><li>llama.cpp starts one busy-waiting thread per core. On a PS5 that takes every core homebrew gets, and the console shuts itself down. The PS5 build leaves two cores free.</li><li>llama.cpp's own web interface renders blank in the PS5 browser, so PS5LM ships a plain chat page.</li>- <li>A model mapped from disk gets paged out once the browser opens; reading it into memory keeps generation fast.</li>+ <li>While the browser is in front, the system pages a background program's memory out to disk, and a model then crawls. Locking it in memory took Qwen3.5 2B from 0.9 to 9 tokens a second; locking 2.7 GB froze the console, so the limit is 1.6 GB for now.</li>+ <li>The kernel caps a connection's receive buffer at 64 KB, so one download stream to the CDN stops at 0.9 MB/s. Sixteen in parallel get about 12 MB/s.</li></ul><p>The full notes are in <a href="https://github.com/cobanov/PS5LM/blob/main/docs/CONSOLE.md">docs/CONSOLE.md</a>.</p></div>@@ -109,26 +110,23 @@<ul class="roadmap"><li><span class="state done">done</span><span>llama.cpp builds for the PS5 and runs small models on its CPU</span></li><li><span class="state done">done</span><span>Chat in the PS5's own browser, with the DualSense and the on-screen keyboard</span></li>- <li><span class="state">next</span><span>The PS5LM app: one payload from Payload Manager. Pick a model that fits, download it from Hugging Face on the console, chat.</span></li>+ <li><span class="state done">done</span><span>The PS5LM app (v0.1): one payload, a model library in the PS5 browser, downloads on the console, chat</span></li>+ <li><span class="state">next</span><span>Listed in Payload Manager, and more models measured on more consoles</span></li><li><span class="state">later</span><span>Bigger models, Qwen 3.8 27B first, in a native app that gets more memory than a payload</span></li><li><span class="state">later</span><span>The GPU, through llama.cpp's Vulkan backend on a community Vulkan driver for the PS5</span></li></ul></div></section>- <section id="try">+ <section id="install"><div class="wrap">- <h2>Try it</h2>- <p>Today this is a developer setup: you build on a Mac or Linux machine and start the server on the console from there. The one-step app is next.</p>- <pre class="panel"><span class="dim"># on your computer</span>-git clone --recursive https://github.com/cobanov/PS5LM-cd PS5LM-scripts/setup-sdk.sh && source scripts/env.sh-scripts/build-llama.sh--<span class="dim"># console jailbroken, ftpsrv and shsrv running, a model in /data/PS5LM/models</span>-PS5_HOST=192.168.1.50 scripts/ps5-chat.sh model.gguf</pre>- <p>The <a href="https://github.com/cobanov/PS5LM#readme">README</a> has the details, and <a href="https://github.com/cobanov/PS5LM/blob/main/docs/MODELS.md">docs/MODELS.md</a> lists models that fit.</p>+ <h2>Install</h2>+ <ol class="steps-list">+ <li>Jailbreak a PS5 you own on firmware 7.00 to 13.60, with a payload loader such as Payload Manager.</li>+ <li>Download <code>ps5lm.elf</code> from the <a href="https://github.com/cobanov/PS5LM/releases/latest">latest release</a> and load it like any payload.</li>+ <li>The PS5 browser opens the library. Press Download on a model, then Run, then Open chat.</li>+ </ol>+ <p>From a phone or computer on the same network, the library is at <code>http://<console IP>:8082</code> and the chat at port 8081, which also speaks the OpenAI API. Building from source is in the <a href="https://github.com/cobanov/PS5LM#readme">README</a>.</p></div></section>@@ -140,7 +138,7 @@ PS5_HOST=192.168.1.50 scripts/ps5-chat.sh model.gguf</pre><h3>Why the CPU and not the GPU?</h3><p>The CPU path needs nothing but the open payload SDK, so it came first. The GPU is the next big step, through a community Vulkan driver for the PS5.</p><h3>How big a model can it run?</h3>- <p>A homebrew payload can hold about 6 GB including the context. Models up to 3B parameters have run so far, and 4-bit models up to about 8B should fit. Bigger ones need a native app, which is on the roadmap.</p>+ <p>Up to 1.6 GB of model for now, which is Qwen3.5 2B at 4-bit. The model has to stay locked in memory to run at full speed, and that memory is shared with the home screen. Bigger models need a native app, which is on the roadmap.</p><h3>Can it break my console?</h3><p>It does not write to system files. Pushing the CPU or memory too hard can freeze the console until you hold the power button; the defaults are set to avoid that.</p></div>