Skip to content

armgit / PS5LM

Public Imported from GitHub
0

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

Chat with a model in the PS5's own browser

Mert Cobanov65667a23077f
diff --git a/README.md b/README.md
index 09fc1df..55eca3c 100644
--- a/README.md
+++ b/README.md
@@ -4,7 +4,7 @@
The first target is **Qwen 3.8** (27B, hybrid Gated DeltaNet attention), the newest open Qwen. No one has run it on a PS5 yet.
-> **Status:** llama.cpp runs on a PS5. Qwen3.5 0.8B generates 14 tokens/s on two of the console's CPU cores (firmware 13.60, PS5 Slim). Next: the chat UI in the PS5's browser, then bigger models. See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CONSOLE.md](docs/CONSOLE.md).
+> **Status:** llama.cpp runs on a PS5. Qwen3.5 0.8B generates 14 tokens/s on two of the console's CPU cores (firmware 13.60, PS5 Slim). There is a chat page in the PS5's own browser (`scripts/ps5-chat.sh`). Next: bigger models. See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CONSOLE.md](docs/CONSOLE.md).
## Why llama.cpp
diff --git a/docs/CONSOLE.md b/docs/CONSOLE.md
index 4c4109e..aeae680 100644
--- a/docs/CONSOLE.md
+++ b/docs/CONSOLE.md
@@ -41,6 +41,15 @@ First run, Qwen3.5 0.8B Q4_K_M, `-t 2 --poll 0 -lm none`, 512 context:
`llama-server` loads Granite 4.2 3B (2.1 GiB) with mmap, so mapping a model file works in a payload.
+Chat through `llama-server` and `ps5/webui` in the PS5's browser (browser open, `-c 4096 -np 1 -lm none`, 3 threads from the safe defaults):
+
+| Model | Prompt | Generation |
+|---|---|---|
+| Qwen3.5 0.8B Q4_K_M, thinking off | 67 tok/s | 13.1 tok/s |
+| Granite 4.2 3B Q4_K_M | 0.6 tok/s | 0.2 tok/s |
+
+Granite is 4 times bigger but 65 times slower: something in that architecture's CPU path, to look into. llama.cpp's own web UI renders blank in the PS5 browser; `ps5/webui` (plain ES5) works.
+
At exit, Sony's libc prints `[SceLibc] A heap error is detected` (SceLibcInternalHeap) after all work is done. Not investigated yet.
## shsrv quirks
diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md
index cd3f408..2c36e93 100644
--- a/docs/ROADMAP.md
+++ b/docs/ROADMAP.md
@@ -43,7 +43,7 @@ Decoding is bound by memory bandwidth: tokens per second ≈ effective bandwidth
- [x] `llama-cli`, `llama-server` and `llama-bench` link ([PORTING.md](PORTING.md) lists the three build fixes)
- [x] Smoke test: Qwen3.5 0.8B generates text on the PS5 CPU, 14.2 tok/s on two threads
- [x] Safe CPU defaults for the PS5 (`patches/0001`): threads = allowed CPUs minus two, no busy-waiting
-- [ ] `llama-server` on the console: chat from a laptop or phone with llama.cpp's built-in web UI (port 8081, since websrv takes 8080)
+- [x] `llama-server` on the console with a chat page in the PS5's own browser (`scripts/ps5-chat.sh`, `ps5/webui`); phones and laptops use the same page on port 8081
- [ ] `llama-bench` numbers for the small model
- [ ] **First light: Qwen3.8-27B generates text on the PS5.** UD-Q2_K_XL if Phase 0 says it fits, UD-IQ2_XXS if not. A video of it, posted with the repo.
diff --git a/ps5/webui/index.html b/ps5/webui/index.html
new file mode 100644
index 0000000..1ffe403
--- /dev/null
+++ b/ps5/webui/index.html
@@ -0,0 +1,174 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>PS5LM</title>
+<!--
+ The chat page llama-server serves on the PS5 (llama-server --path www).
+ llama.cpp's own web UI renders blank in the PS5 browser, so this page sticks
+ to plain ES5 and XMLHttpRequest, with no external files. Sized for a TV.
+ SPDX-License-Identifier: GPL-3.0-or-later
+-->
+<style>
+ :root {
+ --bg: #0b0d12;
+ --panel: #151923;
+ --text: #e8ebf2;
+ --muted: #8a93a6;
+ --accent: #3d7bfd;
+ --user: #1f2a44;
+ --border: #262c3a;
+ }
+ html, body { margin: 0; height: 100%; background: var(--bg); color: var(--text);
+ font-family: -apple-system, "Segoe UI", Helvetica, Arial, sans-serif; }
+ body { display: flex; flex-direction: column; }
+ header { display: flex; align-items: baseline; gap: 16px; padding: 18px 32px;
+ border-bottom: 1px solid var(--border); }
+ header h1 { margin: 0; font-size: 30px; letter-spacing: 0.5px; }
+ header span { color: var(--muted); font-size: 20px; }
+ #log { flex: 1; overflow-y: auto; padding: 24px 32px; }
+ .msg { max-width: 1100px; margin: 0 0 18px; padding: 16px 22px; border-radius: 14px;
+ font-size: 26px; line-height: 1.45; white-space: pre-wrap; word-wrap: break-word; }
+ .user { background: var(--user); margin-left: auto; }
+ .bot { background: var(--panel); border: 1px solid var(--border); }
+ .think { color: var(--muted); font-style: italic; font-size: 22px; }
+ .meta { color: var(--muted); font-size: 18px; margin-top: 8px; }
+ form { display: flex; gap: 14px; padding: 18px 32px 26px; border-top: 1px solid var(--border); }
+ #input { flex: 1; font-size: 26px; padding: 16px 18px; border-radius: 12px;
+ border: 1px solid var(--border); background: var(--panel); color: var(--text); }
+ button { font-size: 24px; padding: 0 30px; border-radius: 12px; border: 0;
+ background: var(--accent); color: #fff; }
+ button.stop { background: #a83a3a; }
+</style>
+</head>
+<body>
+<header><h1>PS5LM</h1><span id="model">llama.cpp on PlayStation 5</span></header>
+<div id="log"></div>
+<form id="form">
+ <input id="input" type="text" autocomplete="off" placeholder="Type a message">
+ <button id="send" type="submit">Send</button>
+</form>
+<script>
+(function () {
+ var log = document.getElementById('log');
+ var form = document.getElementById('form');
+ var input = document.getElementById('input');
+ var send = document.getElementById('send');
+ var SYSTEM = 'You are a helpful assistant running on a PlayStation 5. ' +
+ 'Answer in one to three short sentences, in the language the user writes in.';
+ var messages = [];
+ var xhr = null;
+
+ function bubble(cls, text) {
+ var div = document.createElement('div');
+ div.className = 'msg ' + cls;
+ div.textContent = text || '';
+ log.appendChild(div);
+ log.scrollTop = log.scrollHeight;
+ return div;
+ }
+
+ function busy(on) {
+ send.textContent = on ? 'Stop' : 'Send';
+ send.className = on ? 'stop' : '';
+ }
+
+ // The model name, for the header.
+ var info = new XMLHttpRequest();
+ info.open('GET', '/v1/models');
+ info.onload = function () {
+ try {
+ var id = JSON.parse(info.responseText).data[0].id;
+ document.getElementById('model').textContent = id.split('/').pop() + ' on PlayStation 5';
+ } catch (e) {}
+ };
+ info.send();
+
+ function ask(text) {
+ messages.push({ role: 'user', content: text });
+ // Keep the request small: the console's context is 4096 tokens.
+ if (messages.length > 12) messages = messages.slice(messages.length - 12);
+ bubble('user', text);
+ var out = bubble('bot', '');
+ var think = null;
+ var answer = '';
+ var reasoning = '';
+ var seen = 0;
+ var pending = '';
+ var started = Date.now();
+
+ xhr = new XMLHttpRequest();
+ xhr.open('POST', '/v1/chat/completions');
+ xhr.setRequestHeader('Content-Type', 'application/json');
+
+ // Server-sent events arrive in pieces; parse whole "data:" lines as they come.
+ xhr.onprogress = function () {
+ pending += xhr.responseText.substring(seen);
+ seen = xhr.responseText.length;
+ var lines = pending.split('\n');
+ pending = lines.pop();
+ for (var i = 0; i < lines.length; i++) {
+ var line = lines[i].replace(/^\s+|\s+$/g, '');
+ if (line.indexOf('data:') !== 0) continue;
+ var data = line.substring(5).replace(/^\s+/, '');
+ if (data === '[DONE]') continue;
+ var chunk;
+ try { chunk = JSON.parse(data); } catch (e) { continue; }
+ var delta = chunk.choices && chunk.choices[0] && chunk.choices[0].delta;
+ if (delta && delta.reasoning_content) {
+ if (!think) { think = document.createElement('div'); think.className = 'think'; out.appendChild(think); }
+ reasoning += delta.reasoning_content;
+ think.textContent = reasoning;
+ }
+ if (delta && delta.content) {
+ answer += delta.content;
+ if (!out.lastChild || out.lastChild.nodeType !== 3) out.appendChild(document.createTextNode(''));
+ out.lastChild.nodeValue = answer;
+ }
+ if (chunk.timings) {
+ var meta = document.createElement('div');
+ meta.className = 'meta';
+ meta.textContent = chunk.timings.predicted_n + ' tokens, ' +
+ chunk.timings.predicted_per_second.toFixed(1) + ' tok/s, ' +
+ ((Date.now() - started) / 1000).toFixed(1) + ' s';
+ out.appendChild(meta);
+ }
+ log.scrollTop = log.scrollHeight;
+ }
+ };
+
+ xhr.onloadend = function () {
+ xhr.onprogress();
+ if (xhr.status && xhr.status !== 200 && !answer) out.textContent = 'Error ' + xhr.status + ': ' + xhr.responseText;
+ if (answer) messages.push({ role: 'assistant', content: answer });
+ xhr = null;
+ busy(false);
+ input.focus();
+ };
+
+ // Small models on four CPU cores: short answers, no thinking phase.
+ xhr.send(JSON.stringify({
+ messages: [{ role: 'system', content: SYSTEM }].concat(messages),
+ stream: true,
+ max_tokens: 256,
+ chat_template_kwargs: { enable_thinking: false }
+ }));
+ busy(true);
+ }
+
+ form.onsubmit = function (e) {
+ e.preventDefault();
+ if (xhr) { xhr.abort(); return; }
+ var text = input.value.replace(/^\s+|\s+$/g, '');
+ if (!text) return;
+ input.value = '';
+ ask(text);
+ };
+
+ bubble('bot', 'Hi! I am a language model running on this PlayStation 5, on its own CPU. Ask me anything.');
+ input.focus();
+})();
+</script>
+</body>
+</html>
diff --git a/scripts/ps5-chat.sh b/scripts/ps5-chat.sh
new file mode 100755
index 0000000..028a7a7
--- /dev/null
+++ b/scripts/ps5-chat.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+# Starts llama-server on the console and opens its chat UI in the PS5's own
+# browser. Needs ftpsrv (2121) and shsrv (2323) running on the console, and the
+# model already in /data/PS5LM/models.
+#
+# PS5_HOST=192.168.8.52 scripts/ps5-chat.sh [model.gguf]
+set -euo pipefail
+
+root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
+host=${PS5_HOST:?set PS5_HOST to the console IP}
+model=${1:-granite3b.gguf}
+port=${PS5LM_PORT:-8081}
+logdir="$root/build/console"
+mkdir -p "$logdir"
+
+# Upload the current build and the chat page. shsrv needs absolute paths and
+# short command lines (keep model file names short).
+curl -sS --disable-epsv --ftp-create-dirs -T "$root/build/llama-ps5/bin/llama-server" \
+ "ftp://$host:2121/data/PS5LM/bin/llama-server"
+curl -sS --disable-epsv --ftp-create-dirs -T "$root/ps5/webui/index.html" \
+ "ftp://$host:2121/data/PS5LM/www/index.html"
+
+# Keep the shell session that runs the server open in the background; its output
+# goes to build/console/server.log. The context is capped: the model's own
+# default (131K for Granite) would not fit in a payload's ~6 GiB. The model is
+# read into memory (-lm none): mapped from the file, its pages get evicted once
+# the browser opens, and generation drops to a fraction of a token a second.
+# --path www serves ps5/webui instead of llama.cpp's UI, which the PS5 browser
+# renders blank.
+nohup bash -c "
+ { sleep 1
+ printf 'chmod 755 /data/PS5LM/bin/llama-server\n'; sleep 1
+ printf 'cd /data/PS5LM\n'; sleep 1
+ printf '/data/PS5LM/bin/llama-server -m models/$model -c 4096 -np 1 -lm none --path www --host 0.0.0.0 --port $port\n'
+ sleep 86400
+ } | nc $host 2323
+" > "$logdir/server.log" 2>&1 &
+echo "ps5-chat: llama-server starting (log: $logdir/server.log)"
+
+for _ in $(seq 1 90); do
+ sleep 2
+ code=$(curl -s -m 3 -o /dev/null -w "%{http_code}" "http://$host:$port/health" || true)
+ [ "$code" = "200" ] && break
+done
+if [ "$code" != "200" ]; then
+ echo "ps5-chat: server not healthy (last status $code), see $logdir/server.log" >&2
+ exit 1
+fi
+echo "ps5-chat: server ready"
+
+{ sleep 1; printf 'browse http://127.0.0.1:%s/\n' "$port"; sleep 3; printf 'exit\n'; sleep 1; } \
+ | nc -w 5 "$host" 2323 > /dev/null 2>&1 || true
+echo "ps5-chat: chat UI opened in the PS5 browser; from another device: http://$host:$port/"