llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
v0.8.1: the chat server survives network drops
Abhiramfb15fb722f62
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b0e8cb8..8108e85 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,27 @@ v0.1.x are the upstream payload releases by cobanov.## [Unreleased]+## [0.8.1] - 2026-10-13++### Fixed+- The chat server stopped answering after the network dropped: when the+ console's link went down, accept() failed with `SCE_NET_EINACTIVEDISABLED`+ (errno 163, the same `0x804101a3` a dropped download reported), cpp-httplib+ took that as fatal and closed the listening socket, and the model stayed+ loaded with nothing to reach it. llama-server now binds again once a second+ until it works (`patches/0005-listener-survives-network-drops.patch`).+ Checked on the console: the listener came back within a second after each+ of dozens of drops. A reply streaming at the moment of a drop is still cut.++### Added+- README: the web UI with a sample chat.++### Found+- The repeated "Ethernet disconnected" notifications are the console's link+ going down and up (`eth0: link state changed to DOWN` in the kernel log,+ up to eight times in a few seconds), with or without PS5LM running: a cable,+ port or switch problem, not the app.+## [0.8.0] - 2026-10-13### Addeddiff --git a/README.md b/README.mdindex 0e40c09..c5186b0 100644--- a/README.md+++ b/README.md@@ -54,6 +54,11 @@ For comparison, the 27B on the CPU as a payload, streamed from disk, made about**On your network**+<p align="center">+ <img src="docs/media/app/webui-chat.jpg" width="820" alt="llama.cpp's web UI served by the PS5, answering with Llama 3.2 3B">+ <br><sub>The chat in a PC's browser, served by the PS5: Llama 3.2 3B on the console's GPU at 105 tok/s, explaining that a games console cannot run a chatbot on its own.</sub>+</p>+- **Chat:** llama.cpp's web UI at `http://<console IP>:8081`, with file tools and, for vision models such as Gemma 4, images (put the model's `mmproj` file beside it).- **API:** OpenAI Chat Completions and Responses, and Anthropic Messages, on the same port. opencode, codex, Claude Code and aider work against it: see [docs/CLI.md](docs/CLI.md).diff --git a/docs/media/app/webui-chat.jpg b/docs/media/app/webui-chat.jpgnew file mode 100644index 0000000..683db8fBinary files /dev/null and b/docs/media/app/webui-chat.jpg differdiff --git a/patches/0005-listener-survives-network-drops.patch b/patches/0005-listener-survives-network-drops.patchnew file mode 100644index 0000000..e5decc4--- /dev/null+++ b/patches/0005-listener-survives-network-drops.patch@@ -0,0 +1,55 @@+diff --git a/tools/server/server-http.cpp b/tools/server/server-http.cpp+index 33cbb156c..7ac78f9ad 100644+--- a/tools/server/server-http.cpp++++ b/tools/server/server-http.cpp+@@ -10,6 +10,7 @@+ #include <future>+ #include <memory>+ #include <string>++#include <atomic>+ #include <thread>++ //+@@ -23,6 +24,7 @@ public:+ std::vector<std::thread> threads; // one thread per listener+ std::unique_ptr<httplib::ThreadPool> pool; // single pool shared among all listeners+ int n_threads_http = 0;++ std::atomic<bool> stopping{false}; // stop() was asked: a listener that ends stays ended+ };++ class server_http_task_queue : public httplib::TaskQueue {+@@ -528,9 +530,23 @@ bool server_http_context::start() {+ pimpl->pool = std::make_unique<httplib::ThreadPool>(pimpl->n_threads_http, pimpl->n_threads_http + 1024);+ for (size_t i = 0; i < pimpl->servers.size(); ++i) {+ const auto & srv = pimpl->servers[i];+- pimpl->threads.emplace_back([srv = srv.get(), addr = listening_addresses[i]] {+- if (!srv->listen_after_bind()) {+- SRV_ERR("listener on %s stopped unexpectedly\n", addr.c_str());++ // PS5LM: when the network drops (link down, a new DHCP address),++ // accept() fails with an error httplib takes as fatal and closes the++ // listening socket: the model stays loaded with nothing to reach it.++ // Bind again, once a second, until it works or stop() is asked.++ const std::string host = pimpl->hosts[i];++ const int bind_port = port;++ pimpl->threads.emplace_back([srv = srv.get(), addr = listening_addresses[i], host, bind_port,++ stopping = &pimpl->stopping] {++ while (!srv->listen_after_bind() && !*stopping) {++ SRV_ERR("listener on %s stopped unexpectedly (errno %d); listening again\n", addr.c_str(), errno);++ do {++ std::this_thread::sleep_for(std::chrono::seconds(1));++ srv->stop(); // clears the failed state; the socket is closed already++ } while (!*stopping && !srv->bind_to_port(host, bind_port));++ if (!*stopping) {++ SRV_INF("listener on %s is back\n", addr.c_str());++ }+ }+ });+ srv->wait_until_ready();+@@ -546,6 +562,7 @@ bool server_http_context::start() {+ }++ void server_http_context::stop() const {++ pimpl->stopping = true;+ for (const auto & srv : pimpl->servers) {+ if (srv) {+ srv->stop();diff --git a/ps5/app/version.hpp b/ps5/app/version.hppindex c60081c..fce3fef 100644--- a/ps5/app/version.hpp+++ b/ps5/app/version.hpp@@ -4,4 +4,4 @@// The app's version, semver. The one place it is set: scripts read it from// here (scripts/release-app.sh), and git tags are v<this>. Bump it with an// entry in CHANGELOG.md.-#define PS5LM_VERSION "0.8.0"+#define PS5LM_VERSION "0.8.1"