llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Plans for the next phase, one branch each
Abhiram9be116a05cb5
diff --git a/docs/plans/README.md b/docs/plans/README.mdnew file mode 100644index 0000000..6fc2b2d--- /dev/null+++ b/docs/plans/README.md@@ -0,0 +1,15 @@+# Plans++The next phase, split into one goal per branch. Each branch starts from+`main`, merges back into `main` when its "done when" holds on the console,+and bumps the version in `ps5/app/version.hpp` with a CHANGELOG entry.++| Order | Branch | Plan | Version |+|---|---|---|---|+| 1 | `feature/dashboard-pages` | [Dashboard pages](dashboard-pages.md): widgets, model details, Settings and Logs pages on L1/R1 | 0.3.0 |+| 2 | `feature/cli-access` | [CLI access](cli-access.md): opencode, codex, Claude Code and others against the console | 0.4.0 |+| 3 | `feature/chat-tools` | [Chat tools](chat-tools.md): tools, files, images in the browser chat; a scratch folder with a size warning | 0.5.0 |+| 4 | `feature/model-market` | [Model market](model-market.md): search and download from Hugging Face, with fit shown | 0.6.0 |++`main` is this fork's integration branch. Upstream pull requests go from a+branch cut from `cobanov/PS5LM:main` with only the commits meant for it.diff --git a/docs/plans/chat-tools.md b/docs/plans/chat-tools.mdnew file mode 100644index 0000000..1bf3b86--- /dev/null+++ b/docs/plans/chat-tools.md@@ -0,0 +1,25 @@+# Chat tools (`feature/chat-tools`, 0.5.0)++## Work++- **Tools in the browser chat:** llama-server's web UI with tool calling+ (`--jinja`), and tools served by the app: create, read and list files,+ fetch a URL, run a calculation. Files live in a scratch folder.+- **Scratch folder:** `/data/PS5LM/scratch` by default (changeable in+ Settings). Settings shows its size and has a Clear button; the app notifies+ when it passes a limit (default 2 GiB).+- **Images:** vision models with their `mmproj` file (Gemma 4, Qwen VL),+ loaded together; the planner counts the projector's memory. The library+ pairs a model with its `mmproj` file automatically.++## Done when++On the console: a chat creates a file through a tool and it appears in the+scratch folder; a picture sent to a vision model gets a correct description;+the size warning fires when the folder passes the limit, and Clear empties it.++## Risks++- Writes to `/data` from the app run at 1 to 2 MB/s (measured); fine for+ chat files, not for large outputs.+- Tools that run code are out of scope: no shell on the console.diff --git a/docs/plans/cli-access.md b/docs/plans/cli-access.mdnew file mode 100644index 0000000..114e50b--- /dev/null+++ b/docs/plans/cli-access.md@@ -0,0 +1,30 @@+# CLI access (`feature/cli-access`, 0.4.0)++The console already serves llama-server's OpenAI API at `:8081`, and the+pinned llama.cpp (b11327) also has `/v1/messages` (Anthropic's API). Most of+this goal is making that dependable and documented.++## Work++- A stable model name in the API (`--alias`, from the model's family) so+ tools can be configured once.+- An optional API key (`--api-key`, from Settings), shown on the dashboard.+- Tool calling on: `--jinja` with the model's chat template, checked with a+ tool-call request per preset family.+- `docs/CLI.md`: configuration for opencode, codex, Claude Code+ (`ANTHROPIC_BASE_URL`), t3 code, Continue, aider; with the context length to+ set and what works and what does not.+- The dashboard shows the address and the model name to paste.++## Done when++opencode and Claude Code each complete a small coding task (read a file,+edit it) against the console, and codex at least chats. Logged in+`docs/CLI.md` with the model used.++## Risks++- Long prompts from coding agents: prompt processing runs at 35 to 50 tok/s+ for the 27B, so a 20k-token agent prompt takes minutes. Document which+ models suit agents (smaller, faster ones).+- Only one slot (`-np 1`): concurrent requests queue.diff --git a/docs/plans/dashboard-pages.md b/docs/plans/dashboard-pages.mdnew file mode 100644index 0000000..1d90eca--- /dev/null+++ b/docs/plans/dashboard-pages.md@@ -0,0 +1,55 @@+# Dashboard pages (`feature/dashboard-pages`, 0.3.0)++## Widgets++The home page has eight tiles; some repeat each other:++- MEMORY's ring and GPU MEMORY's bar show the same 11.4 GiB pool (GPU and+ CPU share it).+- GPU's "model on the GPU" and CPU's "CPU heap" lines repeat GPU MEMORY's+ Model and Other segments.+- SESSION and CONTEXT both count tokens.++Proposed eight:++| Tile | Shows |+|---|---|+| GENERATION | tok/s history, min/avg/max, prompt speed (the large tile, unchanged) |+| GPU | busy %, history |+| CPU | use of the app's 13 CPUs, history |+| MEMORY | one tile: % in use and the model / KV cache / other / free bar |+| THERMALS | SoC and CPU temperature, fan duty, SoC power |+| CONTEXT | fill bar, plus tokens in and out, replies and uptime (SESSION folded in) |+| MODEL | small card; Cross opens a detail view (all arguments, sampling, layers, trained context, cache); the library moves to its own action |+| STORAGE | free space on /data and on USB, models found |++THERMALS: Marice/ps5-exporter reads `sceKernelGetCpuTemperature`,+`sceKernelGetSocSensorTemperature`, `sceKernelGetCurrentFanDuty` (0..1024)+and `sceKernelGetSocPowerConsumption` from a payload. First check each+resolves inside a title (some imports are null in titles, as `isatty` was).+If the fan is not exposed, ship temperatures only.++## Pages, switched with L1/R1++1. **Dashboard**: the tiles above.+2. **Settings**: stored in `/data/PS5LM/settings.json`. Default model and+ auto-load at launch, context cap, KV cache type override, sampling+ overrides per model, the server port, sounds on or off.+3. **Logs**: `app.log` and `llama.log`, scrollable, newest at the bottom,+ with a filter for errors.++Reuse ps5-homebrew-ui's `concepts/settings.cpp` and `concepts/terminal.cpp`+before writing new components.++## Done when++On the console: the eight tiles show live values (THERMALS at least+temperatures), L1/R1 move between the three pages, a setting changed on the+Settings page survives a restart and takes effect, and the Logs page scrolls+the live log. The PC preview renders every page.++## Risks++- Thermal calls may not resolve in a title.+- Reading a large `llama.log` every frame: read new bytes only, keep the last+ few thousand lines.diff --git a/docs/plans/model-market.md b/docs/plans/model-market.mdnew file mode 100644index 0000000..ba1b0c1--- /dev/null+++ b/docs/plans/model-market.md@@ -0,0 +1,37 @@+# Model market (`feature/model-market`, 0.6.0)++A page that searches Hugging Face (and other GGUF hosts later) and downloads+models to the console, showing whether each one fits before download.++## First: write speed++The app writes to `/data` at about 1 to 2 MB/s sustained (measured on an+import), so a 5 GB model would take about an hour. Before anything else,+measure from the app: raw `write()` against stdio, larger chunks, `O_DIRECT`,+and writing to a USB drive instead. ftpsrv writes to `/data` much faster, so+the limit is in how the app writes or where it is allowed to. Picking the+download folder (USB by default if it is faster) depends on this.++## Work++- **Search:** the Hugging Face API (`/api/models?search=…&filter=gguf`), then+ the repository's file list for each `.gguf` and its size.+- **Fit:** the planner on the file's size and, where the GGUF header can be+ fetched with a range request, its real metadata; rows show the context it+ would get, or TOO BIG.+- **Download:** HTTPS from the app (the SDK's TLS, or a small client),+ resumable with range requests, checked against the file's SHA-256 from the+ API; progress on the page and in a notification.+- **Location:** default `/data/PS5LM/models`, changeable in Settings (USB).+- **Page:** a store-like page (ps5-homebrew-ui's `concepts/store.cpp`), with+ curated picks from the presets first.++## Done when++On the console: search finds a model, the page shows its fit, the download+finishes with a verified checksum, and the model loads from the library.++## Risks++- TLS in a title: certificate checks need a CA bundle.+- Write speed, above.