Skip to content

armgit / PS5LM

Public Imported from GitHub
0

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

Memory, debug report, app storage, Bonsai 2, Responses feed, probes (#38)

Mert Cobanovf615153b14b9
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6a0729d..5a2069a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,11 +7,70 @@ v0.1.x are the upstream payload releases by cobanov.
## [Unreleased]
+### Added
+- **Settings > Debug report**: one file to attach to a GitHub issue, `/data/PS5LM/debug-report.txt`
+ and a copy on a USB drive when one is in. It holds the app's version, the system software, where
+ the app keeps its files and the room on each drive, the models and their plans, the downloads,
+ the settings, `env.txt` and `app-args.txt`, the last 256 KiB of `app.log` and `llama.log`, and
+ which payload ports answer (ELF loader, FTP, kernel log, shell, ps5-exporter, ...). API keys,
+ tokens and passwords are taken out of everything in it. The kernel log goes in only when
+ **Include the kernel log** is on (off by default), at most its last 1 MiB, read from klogsrv
+ or etaHEN's. Before the first report a dialog says what it holds.
+- **Ternary Bonsai 2**: a **Bonsai 2** preset (PrismML's sampling, 64k context at most) and
+ **Bonsai 2 27B** in Suggested, upstream's `Q2_0` (7.1 GiB). The app's llama.cpp already reads the
+ `prism.hadamard.*` rotation those files carry: checked on a PC with the 2B and the 27B, on the CPU
+ and on ggml-vulkan with the console's shader paths. Not yet run on the console.
+- `scripts/bonsai-pq2-to-q2.py`: PrismML's `PQ2_0` files, which upstream llama.cpp (b11827 too)
+ refuses, to `Q2_0`, with no loss: each 128-weight block becomes two 64-weight blocks with the same
+ scale and codes. Python only, about 80 s for the 27B; the 2B and the 27B come out byte for byte
+ as the `Q2_0` files PrismML publishes. The library says what to do with a `PQ2_0` file (and that a
+ `PTQ1_0` one does not load) instead of failing at load.
+- `scripts/bench-ubatch.py`: prompt and generation speed and memory of one model at several prompt
+ batch sizes on the console (2,048-token prompt, each size a restart with the planner's arguments),
+ as a table; and `scripts/ps5lm-app.sh memory`, which logs the memory pools on request.
+- **The app's own storage** when the HEN keeps `/data` closed (#16, #23, #24). The app used to
+ stop there: no log, no settings, every download failing. It now keeps its files in
+ `/download0/PS5LM` instead, and on the first such start asks, over the screen: use the app's
+ storage, or how to open full storage (OnionHEN's `exact_title_ids`, etaHEN, other HENs) and start
+ again, which asks the HEN once more. **Settings > Sound and storage > Where PS5LM keeps its
+ files** shows which one is in use and how full it is. Models saved there stay in the library
+ once full storage opens.
+
+- Two probes for later decisions, out of the app's way (`ps5/app/probes.cpp`, run from the PC):
+ `scripts/ps5lm-app.sh bench-read <model file>` measures a drive's sequential and random read
+ speed at the sizes MoE experts come in (64 KiB to 4 MiB, one and four threads, cached and
+ `O_DIRECT`), and `bench-memory` logs what the system keeps of the title's memory and the
+ largest block that really allocates. The download and write probes moved there from `main.cpp`.
+- The TV's request feed shows OpenAI Responses requests (`/v1/responses`, what codex sends): the
+ question from `input`, the answer from the response or its `output_text` deltas. They showed
+ blank before. The API page lists the endpoint, and docs/CLI.md says what it takes.
+
### Changed
- **The fleet worker is part of the app.** There is no separate fleet build or
`ps5lm-app-fleet` zip any more: every PS5LM can be a fleet worker, and stays the normal app
unless `/data/PS5LM/fleet-worker.txt` asks for one, as before. `PS5LM_FLEET` is gone from
`scripts/build-app.sh`, and CI builds one title again.
+- The planner counts the compute buffers' growth with the prompt batch (`-ub`, about 38 bytes a
+ token per unit of feed-forward width, measured with ggml-vulkan on the console's shader paths),
+ and the batch for language models is one constant, `kPromptUbatch`, still 64 until
+ `scripts/bench-ubatch.py` has measured the others on a console. See docs/CONSOLE.md.
+- `downloadDataSize` in `sce_sys/param.json` is 16 GiB (16384 MiB, was 256 MiB): the size of
+ the app's own storage. The app counts what it holds itself, as a title's `statvfs` is a stub.
+ Whether the system sets the whole 16 GiB aside on internal storage at the first start, or
+ only what is written, is not known yet; lowering it later frees nothing.
+- A download is refused before it starts, with the reason on the TV, when the drive has no room
+ for it, the downloads already queued to that drive, and 1 GiB to spare. A model and its image
+ projector are queued together or not at all.
+- More memory for models, expected about 0.56 GiB: `sce_sys/param.json` no longer declares the
+ `amm` block and the 256 MiB CPU and GPU page tables it carried over from ps5-homebrew-ui's
+ template. With them the system gave the app 11.44 GiB of direct memory (the 12,280,922,112 bytes
+ in #23's log); titles that leave them out, PS5_Vulkan's and PS5_RetroArch's, get 12 GiB, and
+ neither the GL stack nor RADV uses what the blocks configure. The planner reads the memory free,
+ so every model's plan grows with it: with the dashboard up, v0.9.1 had 11.04 GiB free before a
+ model and gave Qwen 3.8 27B UD-Q2_K_XL a 32k context, not the 64k it was suggested with; with
+ the 12 GiB pool it plans 64k again. Not yet measured on a console: `app.log` now records the pool
+ and the flexible memory at start (`memory at start`) and once the model serves (`memory when
+ ready`).
### Added
- **A PC's GPU as a fleet worker.** `PS5LM_FLEET_GPU=metal` (or `cuda`, `vulkan`)
@@ -21,6 +80,10 @@ v0.1.x are the upstream payload releases by cobanov.
(`ps5/app/launch_app.c`, built by `build-app.sh`) instead of the removed
ps5-homebrew-dev-protocol.
+### Fixed
+- A download that meets a full drive anyway stops at once, "Failed: App storage is full", instead
+ of retrying twenty times.
+
## [0.10.0] - 2026-11-01
### Added
diff --git a/README.md b/README.md
index 127cf37..6786f52 100644
--- a/README.md
+++ b/README.md
@@ -44,8 +44,8 @@ no PC while it runs, no cloud and no account.
- **It runs on the GPU.** llama.cpp's Vulkan backend on the console's RDNA 2 GPU, with 11.4 GB of the
shared memory for the model and its context. Every architecture and quantization llama.cpp knows
comes along.
-- **It answers your network.** OpenAI's `/v1/chat/completions`, Anthropic's `/v1/messages` (Claude Code
- works against it) and a chat page in any browser. Point a client at `http://<console>:8081` and use
+- **It answers your network.** OpenAI's `/v1/chat/completions` and `/v1/responses` (Codex works against
+ it), Anthropic's `/v1/messages` (Claude Code too) and a chat page in any browser. Point a client at `http://<console>:8081` and use
any model name: the loaded one answers.
- **It decides, too.** Load a decision model and the console answers TypeSafe's `/v1/systemone`:
choice, score and yes-or-no questions about text and images, the API Ollaya serves on a PC.
@@ -67,6 +67,17 @@ no PC while it runs, no cloud and no account.
| Qwen 3.8 27B, UD-Q2_K_XL | 9.2 GB | about 20 tokens a second, 64k context |
| d1 Omni 600M, a decision model | 1.1 GB | 3 questions in 0.3 s; a 720p image and 7 questions in 2.9 s |
+### Ternary Bonsai 2
+
+PrismML's Bonsai 2 27B is Qwen 3.8 27B in ternary weights, 7.1 GB as `Q2_0`, which leaves a 64k context
+plenty of room. Its `Q2_0` file is in **Models > Suggested**, and the app's llama.cpp applies the
+model's Hadamard rotation itself. PrismML's main repository ships `PQ2_0` instead, a type only their
+fork reads; convert it on a PC, losslessly (the result is byte for byte PrismML's own `Q2_0`):
+
+```sh
+python3 scripts/bonsai-pq2-to-q2.py Ternary-Bonsai-2-27B-PQ2_0.gguf # writes Ternary-Bonsai-2-27B-Q2_0.gguf
+```
+
## Decision models, powered by Ollaya
[Ollaya](https://github.com/ollaya-dev/ollaya) runs open decision models on a PC the way Ollama runs
@@ -121,7 +132,18 @@ opened the console's storage to it:
mode 777 (in FileZilla: File permissions, applied to subdirectories). Reported on etaHEN 2.6b,
firmware 10.00.
-Using it from coding tools (opencode, Codex, Claude Code) is in [docs/CLI.md](docs/CLI.md).
+If the HEN has not opened the console's storage (`/data`) to PS5LM, the app says so on its first
+start and offers its own storage instead: up to 16 GB in `/download0/PS5LM`, for its settings, logs
+and models (a download that does not fit is refused before it starts). Models saved in the app's storage stay in the
+library. **Settings > Sound and storage** shows which one is in use.
+
+When something goes wrong, **Settings > Debug report** writes `/data/PS5LM/debug-report.txt` (and a copy
+to a USB drive, when one is in): the app's version, the system software, storage, models, settings, the
+end of both logs and which payload ports answer, with API keys and tokens taken out. Attach it to your
+[issue](https://github.com/cobanov/PS5LM/issues). The kernel log goes in only if you switch it on there.
+
+Using it from coding tools (opencode, Codex, Claude Code) is in
+[docs/CLI.md](docs/CLI.md).
| Where | Button | Does |
|---|---|---|
@@ -134,6 +156,7 @@ Using it from coding tools (opencode, Codex, Claude Code) is in [docs/CLI.md](do
| Console | L2 / R2 | the last two minutes / the last hour |
| Settings | D-pad, Cross | choose a category and a row; flip it, or pick from its list |
| Settings, Diagnostics | Square, Triangle | app or server log; errors only |
+| Settings, Debug report | Cross | write one file to attach to an issue |
## Looking inside the model
diff --git a/docs/BUILDING.md b/docs/BUILDING.md
index 5626935..7f43666 100644
--- a/docs/BUILDING.md
+++ b/docs/BUILDING.md
@@ -12,8 +12,8 @@ works too.
`libc.prx` and registered through ShadowMountPlus. Titles may not `exit()`, some libc imports
resolve to nothing, and `getaddrinfo`, `accept4`, `HOME` and `localeconv` need replacing
(`ps5/app/compat_app.c`); each was found by a crash.
-- **The GPU.** ggml-vulkan on RADV with no Vulkan loader, and RADV's unified heap for 11.44 GiB in one
- heap. A GPU submission is costly here (its words copied, a cache flush, a state reset); measured,
+- **The GPU.** ggml-vulkan on RADV with no Vulkan loader, and RADV's unified heap for the whole direct-memory
+ pool in one heap. A GPU submission is costly here (its words copied, a cache flush, a state reset); measured,
fewer of them did not make generation faster.
- **The screen.** ps5-homebrew-ui through ps5-opengl, in the same process as RADV. The two Mesa builds
share 4,399 symbol names, so the GL stack is linked with only its API visible.
diff --git a/docs/CLI.md b/docs/CLI.md
index d252cd4..dfc74f9 100644
--- a/docs/CLI.md
+++ b/docs/CLI.md
@@ -66,6 +66,23 @@ model = "ps5"
Then `codex -p ps5`.
+### The Responses API
+
+`/v1/responses` is llama-server's own, on the same path as chat: `input` as a string or as messages
+(`input_text`, `input_image`; not `input_file`), `instructions`, `temperature`, `top_p`, `max_output_tokens`, `stream`
+(the `response.*` events, text arriving as `response.output_text.delta`), tools, and
+`reasoning.effort`. Each request stands alone: `previous_response_id` is refused, so send the whole
+conversation every time, as codex does. A reply cut off by `max_output_tokens` still reports
+`"status": "completed"`.
+
+```sh
+curl http://192.168.1.19:8081/v1/responses -H 'Content-Type: application/json' -d '{
+ "instructions": "Answer in one sentence.",
+ "input": "Why does a PS5 have a GPU?",
+ "temperature": 0.7, "max_output_tokens": 200, "stream": true
+}'
+```
+
## Claude Code
```sh
diff --git a/docs/CONSOLE.md b/docs/CONSOLE.md
index efac549..b56399f 100644
--- a/docs/CONSOLE.md
+++ b/docs/CONSOLE.md
@@ -113,12 +113,14 @@ PS5LM's app (`ps5/app`, PPSA99575) is a title, not a payload: it gets the consol
| What | Value |
|---|---|
-| Direct memory for the title | 12 GiB; RADV reports one 11.44 GiB device heap with `radv_enable_unified_heap_on_apu` (8 GiB without) |
+| Direct memory for the title | 11.4375 GiB up to v0.9.1, 12 GiB expected after; RADV reports it all as one device heap with `radv_enable_unified_heap_on_apu` (2/3 of it without) |
| Device | PlayStation 5 GPU (RADV NAVI21), Vulkan 1.4 |
| Qwen3.8-27B UD-IQ2_XXS, `-ngl 999 -lm none -c 4096 -ub 64` | loads in 82 s |
| Prompt | 35 to 50 tok/s |
| Generation | 21 tok/s on a short prompt, 9 tok/s after the web UI's 371-token prompt |
+The pool is what `sceKernelGetDirectMemorySize()` returns, and `sce_sys/param.json` sets it. Up to v0.9.1 the app's file carried the `amm` block (`pagetableMemorySizeInMib` 256, 512 GiB address ranges) and the `kernel` block (256 MiB CPU and 256 MiB GPU page tables) of ps5-homebrew-ui's template, and the pool was 11.4375 GiB (12,280,922,112 bytes, from a user's log in #23). PS5_Vulkan's and PS5_RetroArch's titles declare no `amm` block and no page-table sizes and get 12 GiB (PS5_Vulkan measured on system software 6.02 to 12.70, PS5_RetroArch's probe on 12.09); that probe mapped an 11.875 GiB block from it and touched every word, so the default page tables hold a pool-sized mapping. Neither the GL stack nor RADV calls an AMM function (only `sceKernelAllocateDirectMemory` and `sceKernelMapDirectMemory`). The blocks are gone since; the app logs the pool and the flexible memory at start (`memory at start` in `app.log`). Not yet measured on 13.60. The flexible pool (448 MiB, the system's default) is the other part of the title's budget: ProsperoEden declares 320 MiB in `kernel.flexibleMemorySize` and its direct pool grows by the 128 MiB; PS5LM keeps the default until `app.log` shows how much of it the app leaves free.
+
What a title has to do differently from a payload, each found by a crash:
- **Never exit.** `exit()` and `_Exit` end in SIGSYS; the title waits for the shell to close it, as ps5-native-app-boilerplate's examples do. It hides the splash screen itself and posts notifications to show its state.
@@ -127,6 +129,44 @@ What a title has to do differently from a payload, each found by a crash:
- **No `HOME`**: llama.cpp throws looking for its cache directory without one.
- C++ exceptions and RTTI are off by default for the PS5 target and must be turned on for llama.cpp.
+### Prompt batch
+
+llama-server's `-ub` is how many prompt tokens go through the GPU at once. The planner gives language
+models 64 (`kPromptUbatch` in `ps5/app/model_plan.hpp`), what the console has run since the first
+release; decision models and image encoders get 2048. A larger batch needs larger compute buffers.
+Measured on a PC (RTX 3090, ggml-vulkan with cooperative matrices, integer dot products and bf16 off,
+so it takes the console's shader paths): Qwen3.8-27B UD-Q2_K_XL at 64k with a q4_0 cache, flash
+attention on, a 2,048-token prompt and 128 tokens generated, the faster of two runs.
+
+| `-ub` | Device compute buffer | Host compute buffer | Prompt, PC | Generation, PC |
+|---|---|---|---|---|
+| 64 | 27 MiB | 13 MiB | 270 tok/s | 35.1 tok/s |
+| 128 | 54 MiB | 26 MiB | 307 tok/s | 34.9 tok/s |
+| 256 | 108 MiB | 52 MiB | 332 tok/s | 34.4 tok/s |
+| 512 | 216 MiB | 104 MiB | 355 tok/s | 34.2 tok/s |
+
+The speeds are the PC's, not the console's. The buffers carry over: on the console both come from the
+pool, about 38 bytes a token for each unit of feed-forward width, which the planner now counts. With
+the 11.04 GiB a v0.9.1 console had free before a model (#23), the 27B gets 32k at any of these sizes;
+with the 12 GiB pool it keeps 64k at 64 to 256 (11.4 of 11.6 GiB expected) and only just fits at 512.
+`scripts/bench-ubatch.py` measures speed and memory on the console, size by size.
+
+### Probes
+
+Measurements the app makes only when the PC asks, each on a thread of its own, with the results in
+`app.log` (`ps5/app/probes.cpp`). They are for decisions not taken yet, such as reading MoE experts
+from a drive instead of holding them in memory (Qwen3.8-Flash-Next on the roadmap).
+
+| `scripts/ps5lm-app.sh` | Logs |
+|---|---|
+| `bench-read <model file>` | Sequential reads (8 MiB, up to 2 GiB), then three seconds each of random reads of 64 KiB, 256 KiB, 1 MiB and 4 MiB with one and four threads, through the cache and with `O_DIRECT`: MB/s, reads a second, median and p99 latency |
+| `bench-memory` | The direct pool and what the system keeps of a nominal 12 GiB, its largest free block, the flexible pool; with no model loaded, also the largest block that allocates and maps, given back at once |
+| `bench-write` | Write speed by method to the data folder and a USB drive |
+| `memory` | One `memory asked` line: pools, CPU heap and ggml's devices |
+
+Point `bench-read` at a model of several GiB on the drive in question (internal storage, the M.2
+drive or USB), so the cache holds little of it; the `O_DIRECT` lines are the drive itself.
+
### Switching models
Unloading a model in the running app leaves part of its GPU memory held: with Qwen3.8-27B UD-IQ2_XXS at 64k, the pool went from 11.01 GiB free before loading to 5.77 GiB free after `llama_server` returned, and ggml's Vulkan figure fell the same way, so the next model could not get its KV cache. The app therefore changes model by restarting itself: it writes the choice to `/data/PS5LM/model.txt` (read once by the new process, then deleted, so a plain launch loads nothing and opens the library) and calls `sceSystemServiceLoadExec("/app0/eboot.bin", NULL)`. The system replaces the process (`Kill for LoadExec(0x11d) => 0`, then a new `EXEC`) without the crash reporter, and the new process starts from the whole pool (11.01 GiB free). Where the memory is held is still open.
diff --git a/docs/MODELS.md b/docs/MODELS.md
index 9117e07..f84cc74 100644
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@ -23,6 +23,7 @@ The console has 16 GB shared by everything. How much one PS5LM process can hold
| **Qwen 3.8 27B** | `qwen3.8` | UD-IQ2_XXS / UD-Q2_K_XL | 6.77 / 9.15 GiB | qwen35 | the headline target |
| Muse Glimmer 30B | `muse-glimmer` | IQ2_XXS | 8.31 GiB | muse-glimmer | |
| Qwen 3.6 27B | `qwen3.6:27b` | UD-IQ2_XXS | 8.74 GiB | qwen35 | Ollama's most pulled model |
+| Bonsai 2 27B | | Q2_0 | 7.10 GiB | qwen35 | Qwen 3.8 27B in ternary weights (PrismML); 64k with a q8_0 cache in 10.3 GiB. PrismML's PQ2_0 converts losslessly with `scripts/bonsai-pq2-to-q2.py`; PTQ1_0 does not load |
| Qwen 3.6 35B-A3B | `qwen3.6:35b` | UD-IQ2_XXS | 10.02 GiB | qwen35moe | MoE, 3B active |
| gpt-oss 20B | `gpt-oss:20b` | MXFP4 | 10.7 GiB | gpt-oss | MoE, 3.6B active; every quant is about this size |
diff --git a/ps5/app/activity.cpp b/ps5/app/activity.cpp
index f7d1cdd..d3913e8 100644
--- a/ps5/app/activity.cpp
+++ b/ps5/app/activity.cpp
@@ -59,7 +59,8 @@ std::string text_of(const json & content) {
if (content.is_array()) {
for (const auto & part : content) {
const std::string type = part.value("type", "");
- if (type == "text" && part.contains("text") && part["text"].is_string()) {
+ if ((type == "text" || type == "input_text" || type == "output_text") && part.contains("text") &&
+ part["text"].is_string()) {
out += (out.empty() ? "" : " ") + part["text"].get<std::string>();
} else if (type == "image_url" || type == "image" || type == "input_image") {
out += (out.empty() ? "" : " ") + std::string("[image]");
@@ -69,7 +70,9 @@ std::string text_of(const json & content) {
return out;
}
-// What was asked: the last user message, a decision request's state, or the prompt.
+// What was asked: the last user message, a decision request's state, or the
+// prompt. Chat and Anthropic requests carry "messages", Responses "input"
+// (a string, or items with a role).
std::string asked_from(const std::string & path, const json & j) {
if (!j.is_object()) {
return {};
@@ -85,13 +88,18 @@ std::string asked_from(const std::string & path, const json & j) {
}
return head + (state.empty() || state == "null" ? "" : ": " + state);
}
- if (j.contains("messages") && j["messages"].is_array()) {
- for (auto it = j["messages"].rbegin(); it != j["messages"].rend(); ++it) {
- if (it->is_object() && it->value("role", "") == "user" && it->contains("content")) {
- return text_of((*it)["content"]);
+ for (const char * key : { "messages", "input" }) {
+ if (j.contains(key) && j[key].is_array()) {
+ for (auto it = j[key].rbegin(); it != j[key].rend(); ++it) {
+ if (it->is_object() && it->value("role", "") == "user" && it->contains("content")) {
+ return text_of((*it)["content"]);
+ }
}
}
}
+ if (j.contains("input") && j["input"].is_string()) {
+ return j["input"].get<std::string>();
+ }
if (j.contains("prompt")) {
return j["prompt"].is_string() ? j["prompt"].get<std::string>() : j["prompt"].dump();
}
@@ -157,6 +165,20 @@ std::string answer_from(const json & j) {
}
return {};
}
+ // OpenAI Responses: a text delta, or a whole response's message text. The
+ // stream's closing events repeat the text and are left out.
+ if (j.value("type", "") == "response.output_text.delta") {
+ return j.contains("delta") && j["delta"].is_string() ? j["delta"].get<std::string>() : "";
+ }
+ if (j.value("object", "") == "response" && j.contains("output") && j["output"].is_array()) {
+ std::string out;
+ for (const auto & item : j["output"]) {
+ if (item.is_object() && item.value("type", "") == "message" && item.contains("content")) {
+ out += text_of(item["content"]);
+ }
+ }
+ return out;
+ }
// Anthropic: a text delta, or a whole message's text blocks.
if (j.value("type", "") == "content_block_delta" && j.contains("delta") && j["delta"].is_object()) {
return j["delta"].value("type", "") == "text_delta" ? j["delta"].value("text", "") : "";
diff --git a/ps5/app/activity_test.cpp b/ps5/app/activity_test.cpp
new file mode 100644
index 0000000..c500a69
--- /dev/null
+++ b/ps5/app/activity_test.cpp
@@ -0,0 +1,87 @@
+// Host check for activity.cpp, with payloads llama-server sent:
+// c++ -std=c++20 -O1 -Ithird_party/llama.cpp/vendor ps5/app/activity_test.cpp ps5/app/activity.cpp
+// ps5/app/stats.cpp ps5/app/storage.cpp -o build/activity_test -pthread && build/activity_test
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "stats.hpp"
+
+#include <cstdio>
+#include <cstdlib>
+#include <string>
+
+extern "C" {
+void ps5lm_activity_begin(int id, const char * method, const char * path, const char * remote, const char * body,
+ size_t body_len);
+void ps5lm_activity_data(int id, const char * data, size_t len);
+void ps5lm_activity_end(int id, int status);
+}
+
+namespace {
+
+int g_failed = 0;
+
+// One request through the hooks, its answer in pieces of `chunk` bytes, as a
+// stream arrives; what the Home feed would show.
+ps5lm::Exchange run(ps5lm::StatsCollector & stats, int id, const char * path, const std::string & body,
+ const std::string & answer, size_t chunk) {
+ ps5lm_activity_begin(id, "POST", path, "192.168.1.24", body.data(), body.size());
+ for (size_t at = 0; at < answer.size(); at += chunk) {
+ const std::string piece = answer.substr(at, chunk);
+ ps5lm_activity_data(id, piece.data(), piece.size());
+ }
+ ps5lm_activity_end(id, 200);
+ for (const auto & e : stats.snapshot().activity) {
+ if (e.id == id) {
+ return e;
+ }
+ }
+ return {};
+}
+
+void expect(const ps5lm::Exchange & e, const char * asked, const char * answered) {
+ if (e.asked != asked || e.answered != answered || e.status != 200) {
+ std::printf("FAIL %s\n asked: \"%s\" (want \"%s\")\n answered: \"%s\" (want \"%s\")\n", e.path.c_str(),
+ e.asked.c_str(), asked, e.answered.c_str(), answered);
+ ++g_failed;
+ }
+}
+
+} // namespace
+
+int main() {
+ ps5lm::StatsCollector stats;
+ ps5lm::set_activity_sink(&stats);
+
+ // OpenAI Responses, whole.
+ expect(run(stats, 1, "/v1/responses", R"({"input":"Say hi in one word.","max_output_tokens":30})",
+ R"({"object":"response","status":"completed","output":[{"content":[{"type":"output_text","annotations":[],)"
+ R"("logprobs":[],"text":"Hi."}],"id":"msg_1","role":"assistant","status":"completed","type":"message"}]})", 1 << 20),
+ "Say hi in one word.", "Hi.");
+
+ // OpenAI Responses, streamed: the deltas once, not again from the closing events.
+ const std::string sse =
+ "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_1\",\"object\":\"response\",\"status\":\"in_progress\"}}\n\n"
+ "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"item\":{\"content\":[],\"id\":\"msg_1\",\"role\":\"assistant\",\"status\":\"in_progress\",\"type\":\"message\"}}\n\n"
+ "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"msg_1\",\"part\":{\"type\":\"output_text\",\"text\":\"\"}}\n\n"
+ "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"msg_1\",\"delta\":\"One\"}\n\n"
+ "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"msg_1\",\"delta\":\", two\"}\n\n"
+ "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"msg_1\",\"delta\":\", three.\"}\n\n"
+ "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"msg_1\",\"text\":\"One, two, three.\"}\n\n"
+ "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"msg_1\",\"part\":{\"type\":\"output_text\",\"text\":\"One, two, three.\"}}\n\n"
+ "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"item\":{\"type\":\"message\",\"content\":[{\"type\":\"output_text\",\"text\":\"One, two, three.\"}],\"role\":\"assistant\"}}\n\n"
+ "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp_1\",\"object\":\"response\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"content\":[{\"type\":\"output_text\",\"text\":\"One, two, three.\"}],\"role\":\"assistant\"}]}}\n\n";
+ const std::string items = R"({"input":[{"role":"user","content":[{"type":"input_text","text":"Count to three."}]}],"stream":true})";
+ expect(run(stats, 2, "/v1/responses", items, sse, 37), "Count to three.", "One, two, three.");
+ expect(run(stats, 3, "/v1/responses", items, sse, 1 << 20), "Count to three.", "One, two, three.");
+
+ // Chat Completions and Anthropic Messages, as before.
+ expect(run(stats, 4, "/v1/chat/completions", R"({"messages":[{"role":"system","content":"Be brief."},{"role":"user","content":"Hello?"}],"stream":true})",
+ "data: {\"choices\":[{\"delta\":{\"content\":\"Hel\"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"lo!\"}}]}\n\ndata: [DONE]\n\n", 11),
+ "Hello?", "Hello!");
+ expect(run(stats, 5, "/v1/messages", R"({"messages":[{"role":"user","content":[{"type":"text","text":"Hi Claude"}]}],"stream":true})",
+ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"Hi \"}}\n\n"
+ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"there\"}}\n\n", 5),
+ "Hi Claude", "Hi there");
+
+ std::printf(g_failed ? "activity_test: %d failed\n" : "activity_test: all passed\n", g_failed);
+ return g_failed ? EXIT_FAILURE : EXIT_SUCCESS;
+}
diff --git a/ps5/app/dashboard.cpp b/ps5/app/dashboard.cpp
index 3fe9e7e..83af23d 100644
--- a/ps5/app/dashboard.cpp
+++ b/ps5/app/dashboard.cpp
@@ -79,7 +79,8 @@ constexpr int kMarketChipCount = 7;
// ---- Settings -------------------------------------------------------------------
-enum class Setting : int { auto_load, default_model, ctx_cap, kv_type, sounds, tools, introspect, scratch_limit, clear_scratch, download_dir };
+enum class Setting : int { auto_load, default_model, ctx_cap, kv_type, sounds, tools, introspect, scratch_limit, clear_scratch, download_dir,
+ storage, debug_report, report_kernel_log };
struct SettingRow {
Setting id;
@@ -109,16 +110,21 @@ constexpr SettingCategory kCategories[] = {
3 },
{ "Sound and storage",
{ { Setting::sounds, "Sounds", "Interface sounds on the TV" },
- { Setting::download_dir, "Save downloads to", "Internal storage, or PS5LM/models on an M.2 or USB drive" } },
- 2 },
+ { Setting::download_dir, "Save downloads to", "Internal storage, or PS5LM/models on an M.2 or USB drive" },
+ { Setting::storage, "Where PS5LM keeps its files", "Full storage, or its own while the HEN keeps /data closed" } },
+ 3 },
{ "Developer",
{ { Setting::introspect, "Logits and activations API",
"Off: /v1/introspect is closed. On: the raw numbers. A change restarts the model" } },
1 },
+ { "Debug report",
+ { { Setting::debug_report, "Create a debug report", "One file to attach to a GitHub issue, with no keys or tokens in it" },
+ { Setting::report_kernel_log, "Include the kernel log", "Its last 1 MiB, read from klogsrv; off unless you switch it on" } },
+ 2 },
{ "Diagnostics", {}, 0 },
};
-constexpr int kCategoryCount = 6;
-constexpr int kDiagnostics = 5;
+constexpr int kCategoryCount = 7;
+constexpr int kDiagnostics = 6;
constexpr uint32_t kCtxSteps[] = { 4096, 8192, 16384, 32768, 65536, 131072 };
constexpr const char * kKvSteps[] = { "auto", "f16", "q8_0", "q4_0" };
@@ -403,22 +409,42 @@ std::string setting_text(const Settings & s, Setting id, const std::vector<Model
return d.name;
}
}
- return s.download_dir.rfind("/mnt/ext", 0) == 0 ? "M.2 drive, not found" : "USB drive, not found";
+ return drive_name(s.download_dir) + ", not found";
}
default: return "";
}
}
-bool is_toggle(Setting id) { return id == Setting::auto_load || id == Setting::sounds || id == Setting::tools || id == Setting::introspect; }
+bool is_toggle(Setting id) {
+ return id == Setting::auto_load || id == Setting::sounds || id == Setting::tools || id == Setting::introspect
+ || id == Setting::report_kernel_log;
+}
-bool toggle_value(const Settings & s, Setting id) {
+bool & toggle_ref(Settings & s, Setting id) {
switch (id) {
- case Setting::auto_load: return s.auto_load;
- case Setting::sounds: return s.sounds;
- case Setting::tools: return s.tools;
- case Setting::introspect: return s.introspect;
- default: return false;
+ case Setting::auto_load: return s.auto_load;
+ case Setting::sounds: return s.sounds;
+ case Setting::introspect: return s.introspect;
+ case Setting::report_kernel_log: return s.report_kernel_log;
+ default: return s.tools;
+ }
+}
+
+bool toggle_value(const Settings & s, Setting id) {
+ return is_toggle(id) && toggle_ref(const_cast<Settings &>(s), id);
+}
+
+// A row that does something when pressed, with no value of its own.
+bool is_action(Setting id) { return id == Setting::clear_scratch || id == Setting::debug_report; }
+
+// Used and whole size of the app's storage, from its drive, as the row shows it.
+std::string app_storage_text(const std::vector<Drive> & drives, double size_gib) {
+ for (const auto & d : drives) {
+ if (d.name == "App storage" && d.total > 0) {
+ return format("App storage, %.1f", d.total - d.free) + format(" of %.0f GB used", d.total);
+ }
}
+ return format("App storage, %.0f GB", size_gib);
}
// A row's choices, for the list X opens under it: each with the settings it
@@ -586,7 +612,7 @@ void Dashboard::refilter_logs() {
void Dashboard::update(const hui::InputFrame & raw, float dt, ui::Feedback & feedback) {
const hui::InputFrame none{};
- const bool dialog = market_confirm_ || market_space_ || picker_;
+ const bool dialog = market_confirm_ || market_space_ || picker_ || report_dialog_ || storage_dialog_;
const hui::InputFrame & input = busy() ? none : raw; // busy: the screen says so, buttons wait
age_ += dt;
clock_ += dt;
@@ -615,7 +641,9 @@ void Dashboard::update(const hui::InputFrame & raw, float dt, ui::Feedback & fee
// L1/R1 walk the dock, unless a question is open. A message being typed
// takes every button until it is sent or dropped.
const int step = input.is_pressed(Action::page_next) ? 1 : input.is_pressed(Action::page_prev) ? -1 : 0;
- if (prompt_.is_open()) {
+ if (storage_dialog_) {
+ update_storage_dialog(input, feedback);
+ } else if (prompt_.is_open()) {
if (prompt_.handle(input, feedback) == ui::Event::activated) {
request_ = { Request::ask, prompt_.text() };
}
@@ -940,6 +968,18 @@ void Dashboard::update_settings(const hui::InputFrame & input, ui::Feedback & fe
update_picker(input, feedback);
return;
}
+ // What a debug report holds, before the first one.
+ if (report_dialog_) {
+ if (input.is_pressed(Action::confirm)) {
+ report_dialog_ = false;
+ request_ = { Request::debug_report, "" };
+ feedback.play(Cue::select);
+ } else if (input.is_pressed(Action::back)) {
+ report_dialog_ = false;
+ feedback.play(Cue::modal_close);
+ }
+ return;
+ }
if (!in_rows_) {
if (input.nav == Direction::up || input.nav == Direction::down) {
const int next = setting_category_ + (input.nav == Direction::down ? 1 : -1);
@@ -977,11 +1017,26 @@ void Dashboard::update_settings(const hui::InputFrame & input, ui::Feedback & fe
if (id == Setting::clear_scratch) {
request_ = { Request::clear_scratch, "" };
feedback.play(Cue::select);
+ } else if (id == Setting::storage) {
+ if (full_storage_) {
+ feedback.play(Cue::error); // nothing to choose
+ } else {
+ storage_dialog_ = true;
+ storage_howto_ = false;
+ feedback.play(Cue::modal_open);
+ }
+ } else if (id == Setting::debug_report) {
+ if (report_busy_) {
+ feedback.play(Cue::error); // one is being written
+ } else if (!settings_.report_explained) {
+ report_dialog_ = true;
+ feedback.play(Cue::modal_open);
+ } else {
+ request_ = { Request::debug_report, "" };
+ feedback.play(Cue::select);
+ }
} else if (is_toggle(id)) {
- bool & on = id == Setting::auto_load ? settings_.auto_load
- : id == Setting::sounds ? settings_.sounds
- : id == Setting::tools ? settings_.tools
- : settings_.introspect;
+ bool & on = toggle_ref(settings_, id);
on = !on;
request_ = { Request::settings, "" };
feedback.play(Cue::toggle);
@@ -1128,7 +1183,7 @@ void Dashboard::draw_hints(gfx::DrawList & l) const {
layout.cy = 1018;
layout.item_gap = 36;
std::vector<ui::Hint> h;
- if (market_confirm_ || market_space_ || prompt_.visible()) {
+ if (market_confirm_ || market_space_ || report_dialog_ || storage_dialog_ || prompt_.visible()) {
return; // the dialog has its own
}
if (picker_ && page_ == kSettingsPage) {
@@ -1181,6 +1236,13 @@ void Dashboard::draw_hints(gfx::DrawList & l) const {
const Setting id = kCategories[setting_category_].rows[setting_row_].id;
if (id == Setting::clear_scratch) {
h = { { ui::Button::circle, "Back" }, { ui::Button::cross, "Clear" } };
+ } else if (id == Setting::debug_report) {
+ h = { { ui::Button::circle, "Back" }, { ui::Button::cross, report_busy_ ? "Writing it" : "Create" } };
+ } else if (id == Setting::storage) {
+ h = { { ui::Button::circle, "Back" } };
+ if (!full_storage_) {
+ h.push_back({ ui::Button::cross, "Options" });
+ }
} else if (is_toggle(id)) {
h = { { ui::Button::circle, "Back" }, { ui::Button::cross, "Switch" } };
} else {
@@ -1632,6 +1694,7 @@ void Dashboard::draw_api(gfx::DrawList & l) const {
struct Ep { const char * method; const char * path; const char * about; bool decision; };
const Ep all[] = {
{ "POST", "/v1/chat/completions", "OpenAI chat, streaming and tools", false },
+ { "POST", "/v1/responses", "OpenAI Responses, for Codex and the Agents SDK", false },
{ "POST", "/v1/messages", "Anthropic Messages, for Claude Code and its SDKs", false },
{ "POST", "/v1/completions", "Plain text completion", false },
{ "POST", "/v1/systemone", "TypeSafe decisions: choice, score and yes-or-no questions", true },
@@ -1644,7 +1707,7 @@ void Dashboard::draw_api(gfx::DrawList & l) const {
put(l, f.mono, e.method, kLeft, y, 24, kInk3);
put(l, f.mono, e.path, kLeft + 96, y, 24, here ? kInk : kInk3);
put(l, f.regular, e.about, kLeft + 96, y + 34, 20, here ? kInk2 : kInk3);
- y += 84;
+ y += 72;
}
// An example for what is loaded, in a quiet panel.
@@ -1811,11 +1874,17 @@ void Dashboard::draw_settings(gfx::DrawList & l) const {
const SettingRow & r = c.rows[i];
const float y = kListY + kRowPitch * (float) i;
put(l, f.regular, r.label, kListX + 28, y + 36, 26, kInk);
- put(l, f.regular, r.help, kListX + 28, y + 66, 20, kInk3);
+ // The debug report's row says where the last one went, in place of its help.
+ const bool status = r.id == Setting::debug_report && !report_status_.empty();
+ put(l, f.regular, f.regular.font->fit(status ? report_status_ : std::string(r.help), 20, kRight - kListX - 140),
+ kListX + 28, y + 66, 20, status && report_status_.rfind("Failed", 0) == 0 ? kRed : status ? kInk2 : kInk3);
const float cy = y + kRowH * 0.5f;
if (is_toggle(r.id)) {
toggle_switch(l, kRight - 28, cy, toggle_value(settings_, r.id));
- } else if (r.id != Setting::clear_scratch) {
+ } else if (r.id == Setting::storage) {
+ put(l, f.regular, full_storage_ ? "Full storage, /data/PS5LM" : app_storage_text(live_.drives, app_storage_gib_),
+ kRight - 28, y + 50, 24, in_rows_ && i == setting_row_ ? kInk : kInk2, gfx::Align::right);
+ } else if (!is_action(r.id)) {
const bool focus = in_rows_ && i == setting_row_;
const Color c = focus ? kInk : kInk2;
put(l, f.regular, setting_text(settings_, r.id, live_.models, live_.drives), kRight - 68, y + 50, 24, c,
@@ -1906,6 +1975,14 @@ void sheet(gfx::DrawList & l, const Rect & r) {
} // namespace
void Dashboard::draw_dialogs(gfx::DrawList & l) const {
+ if (storage_dialog_) {
+ draw_storage_dialog(l);
+ return;
+ }
+ if (page_ == kSettingsPage && report_dialog_) {
+ draw_report_dialog(l);
+ return;
+ }
if (!(page_ == kModelsPage && (market_confirm_ || market_space_))) {
return;
}
@@ -1933,6 +2010,114 @@ void Dashboard::draw_dialogs(gfx::DrawList & l) const {
}
}
+// Full storage closed: the app's own storage, or how to open /data to it.
+void Dashboard::update_storage_dialog(const hui::InputFrame & input, ui::Feedback & feedback) {
+ if (!storage_howto_) {
+ if (input.is_pressed(Action::confirm)) {
+ storage_dialog_ = false;
+ request_ = { Request::storage_app, "" };
+ feedback.play(Cue::select);
+ } else if (input.is_pressed(Action::north)) {
+ storage_howto_ = true;
+ feedback.play(Cue::tab);
+ } else if (input.is_pressed(Action::back)) {
+ storage_dialog_ = false; // asked again at the next start
+ feedback.play(Cue::modal_close);
+ }
+ return;
+ }
+ if (input.is_pressed(Action::confirm)) {
+ storage_dialog_ = false;
+ request_ = { Request::restart, "" };
+ feedback.play(Cue::launch);
+ } else if (input.is_pressed(Action::back)) {
+ storage_howto_ = false;
+ feedback.play(Cue::back);
+ }
+}
+
+void Dashboard::draw_storage_dialog(gfx::DrawList & l) const {
+ const ui::Fonts & f = fonts_;
+ const Rect r{ 960 - 580, 540 - 300, 1160, 600 };
+ sheet(l, r);
+ const float text_w = r.w - 112;
+ float y = r.y + 150;
+ const auto para = [&](const std::string & text, Color c) {
+ for (const auto & line : wrap(f.regular, text, 23, text_w)) {
+ put(l, f.regular, line, r.x + 56, y, 23, c);
+ y += 34;
+ }
+ y += 18;
+ };
+ std::vector<ui::Hint> h;
+ if (!storage_howto_) {
+ put(l, f.semibold, "Full storage is closed to PS5LM", r.x + 56, r.y + 86, 34, kInk);
+ para("The HEN did not open the console's storage (/data) to the app, so PS5LM cannot keep its models, "
+ "settings and logs there.", kInk2);
+ para("It can use its own storage instead: up to " + format("%.0f GB", app_storage_gib_) +
+ ", enough for one large model or a few small ones. Downloads that do not fit are refused before they start.",
+ kInk2);
+ para("Or open full storage to it and start again; models saved in the app's storage stay in the library.", kInk3);
+ h = { { ui::Button::circle, "Not now" }, { ui::Button::triangle, "How to open it" },
+ { ui::Button::cross, "Use the app's storage" } };
+ } else {
+ put(l, f.semibold, "Open full storage to PS5LM", r.x + 56, r.y + 86, 34, kInk);
+ para("OnionHEN: over FTP, add PPSA99575 to exact_title_ids in the [app_jailbreak] section of "
+ "/data/OnionHEN/config.ini.", kInk2);
+ para("etaHEN opens /data to homebrew before it starts; if it did not, load etaHEN again, up to date.", kInk2);
+ para("Other HENs: allow PPSA99575 in their app jailbreak list.", kInk2);
+ para("Then start PS5LM again: it asks the HEN for /data at every start.", kInk3);
+ h = { { ui::Button::circle, "Back" }, { ui::Button::cross, "Start again" } };
+ }
+ ui::GlyphStyle style = ui::GlyphStyle::dark();
+ style.label = kInk2;
+ ui::HintLayout layout;
+ layout.size = 32;
+ layout.text_size = 22;
+ layout.cy = r.y + r.h - 56;
+ ui::draw_hints(l, fonts_, style, h.data(), (int) h.size(), r.x + r.w - 56, true, layout);
+}
+
+// Before the first debug report: what goes in it, and what never does.
+void Dashboard::draw_report_dialog(gfx::DrawList & l) const {
+ const ui::Fonts & f = fonts_;
+ const Rect r{ 960 - 560, 540 - 330, 1120, 660 };
+ sheet(l, r);
+ put(l, f.semibold, "What a debug report holds", r.x + 56, r.y + 86, 34, kInk);
+ const char * items[] = {
+ "The app's version and the console's system software",
+ "Where PS5LM keeps its files, the drives and their free space",
+ "The models on this PS5, their plans, and the downloads",
+ "Your settings and the server's arguments",
+ "The last 256 KiB of app.log and of llama.log",
+ "Which payload ports answer on this console (ELF loader, FTP, kernel log, ...)",
+ };
+ float y = r.y + 150;
+ for (const char * it : items) {
+ l.circle(r.x + 66, y - 8, 4, kInk3);
+ put(l, f.regular, it, r.x + 86, y, 22, kInk2);
+ y += 40;
+ }
+ put(l, f.regular, settings_.report_kernel_log ? "And the last 1 MiB of the kernel log, as you switched on."
+ : "Not the kernel log, unless you switch it on below.",
+ r.x + 86, y, 22, kInk2);
+ y += 56;
+ for (const auto & line : wrap(f.regular, "API keys and tokens are taken out. The logs can still name your models and the "
+ "addresses of the devices that used the server; read the file before you post it.",
+ 22, r.w - 112)) {
+ put(l, f.regular, line, r.x + 56, y, 22, kInk3);
+ y += 32;
+ }
+ ui::GlyphStyle style = ui::GlyphStyle::dark();
+ style.label = kInk2;
+ ui::HintLayout layout;
+ layout.size = 32;
+ layout.text_size = 22;
+ layout.cy = r.y + r.h - 56;
+ const ui::Hint h[] = { { ui::Button::circle, "Cancel" }, { ui::Button::cross, "Create the report" } };
+ ui::draw_hints(l, fonts_, style, h, 2, r.x + r.w - 56, true, layout);
+}
+
// Slow work, often the app restarting for a model: the whole screen, the
// name and what is happening, and a bar that keeps moving so a pause never
// looks like a hang.
diff --git a/ps5/app/dashboard.hpp b/ps5/app/dashboard.hpp
index 0e66b28..6b1d151 100644
--- a/ps5/app/dashboard.hpp
+++ b/ps5/app/dashboard.hpp
@@ -61,6 +61,22 @@ class Dashboard {
void set_logs(std::vector<std::string> app, std::vector<std::string> llama);
bool wants_logs() const; // the logs are on screen: keep them fresh
void set_market(MarketView view) { market_ = std::move(view); }
+ // Where the app keeps its files: full storage (/data), or its own, of
+ // `app_gib`. `ask`: full storage is closed and no choice was made yet, so
+ // the question comes up over whatever page is open.
+ void set_storage(bool full, double app_gib, bool ask) {
+ full_storage_ = full;
+ app_storage_gib_ = app_gib;
+ if (ask) {
+ storage_dialog_ = true;
+ storage_howto_ = false;
+ }
+ }
+ // The debug report's row: being written, or where the last one went.
+ void set_report_status(bool busy, std::string status) {
+ report_busy_ = busy;
+ report_status_ = std::move(status);
+ }
int page() const { return page_; }
void update(const hui::InputFrame & input, float dt, hui::ui::Feedback & feedback);
void draw(DashboardFrame & frame) const;
@@ -69,7 +85,8 @@ class Dashboard {
// changed (read them with settings()).
struct Request {
enum Kind { none, load, unload, settings, clear_scratch, market_search, market_open, market_plan, market_download,
- market_download_unload, download_pause, download_resume, download_remove, suggested_download, ask } kind = none;
+ market_download_unload, download_pause, download_resume, download_remove, suggested_download, ask,
+ debug_report, storage_app, restart } kind = none;
std::string path; // a model, a market query, the folder a download goes to (empty: the setting), or what to ask
int repo = -1, file = -1; // a market file, a download's index (repo), or a suggestion's (repo)
};
@@ -108,6 +125,9 @@ class Dashboard {
void draw_picker(hui::gfx::DrawList & list) const;
void draw_logs(hui::gfx::DrawList & list, const hui::gfx::Rect & area) const;
void draw_dialogs(hui::gfx::DrawList & list) const;
+ void draw_report_dialog(hui::gfx::DrawList & list) const;
+ void draw_storage_dialog(hui::gfx::DrawList & list) const;
+ void update_storage_dialog(const hui::InputFrame & input, hui::ui::Feedback & feedback);
void draw_busy(hui::gfx::DrawList & list) const;
void draw_hints(hui::gfx::DrawList & list) const;
@@ -154,6 +174,13 @@ class Dashboard {
bool in_rows_ = false; // the focus is on the rows, not the categories
bool picker_ = false; // a row's choices are open in a list under it
int picker_cursor_ = 0;
+ bool full_storage_ = true; // /data is open to the app
+ double app_storage_gib_ = 16;
+ bool storage_dialog_ = false; // full storage is closed: use the app's own, or how to open it
+ bool storage_howto_ = false; // ... showing how to open it
+ bool report_dialog_ = false; // what a debug report holds, before the first
+ bool report_busy_ = false; // one is being written
+ std::string report_status_; // "Saved to ...", or why it failed
// Console.
bool console_hour_ = false; // the charts show the last hour, not the last two minutes
diff --git a/ps5/app/main.cpp b/ps5/app/main.cpp
index de4897e..d07fde7 100644
--- a/ps5/app/main.cpp
+++ b/ps5/app/main.cpp
@@ -8,12 +8,16 @@
// the interface sounds. Models come from /data/PS5LM/models and USB drives; the
// planner (model_plan.cpp) picks each one's context and cache for the memory
// free. /data/PS5LM/app-args.txt, one argument per line, overrides all of it.
-// The app logs to /data/PS5LM/app.log and llama.log.
+// The app logs to /data/PS5LM/app.log and llama.log. When the HEN keeps /data
+// closed, all of it lives in the app's own storage, /download0/PS5LM, instead.
//
// SPDX-License-Identifier: GPL-3.0-or-later
#include <pthread.h>
+#include <arpa/inet.h>
#include <fcntl.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/statvfs.h>
@@ -21,10 +25,12 @@
#include <algorithm>
#include <atomic>
+#include <chrono>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
+#include <ctime>
#include <exception>
#include <filesystem>
#include <fstream>
@@ -46,7 +52,10 @@
#include "display.hpp"
#include "ggml-backend.h"
#include "model_plan.hpp"
+#include "probes.hpp"
+#include "report.hpp"
#include "settings.hpp"
+#include "storage.hpp"
#include "stats.hpp"
#include "suggested.hpp"
#include "version.hpp"
@@ -101,28 +110,32 @@ namespace {
using namespace ps5lm;
-constexpr const char * kLog = "/data/PS5LM/app.log";
-constexpr const char * kArgs = "/data/PS5LM/app-args.txt";
-constexpr const char * kChoice = "/data/PS5LM/model.txt";
-constexpr const char * kEnv = "/data/PS5LM/env.txt"; // driver and ggml switches, for measuring
-constexpr const char * kTextOnly = "/data/PS5LM/text-only"; // the last load's projector failed: load without it once
-constexpr const char * kQuit = "/data/PS5LM/quit";
-constexpr const char * kSettings = "/data/PS5LM/settings.json";
-constexpr const char * kFleetWorker = "/data/PS5LM/fleet-worker.txt";
-constexpr const char * kScratch = "/data/PS5LM/scratch";
+// The app's files, in /data/PS5LM with full storage, else in the app's own
+// storage (storage.hpp). Set once at start, before any thread starts.
+struct Files {
+ std::string log, args, choice, env, text_only, quit, settings, scratch, llama_log, models;
+ explicit Files(const std::string & root)
+ : log(root + "/app.log"), args(root + "/app-args.txt"), choice(root + "/model.txt"),
+ env(root + "/env.txt"), // driver and ggml switches, for measuring
+ text_only(root + "/text-only"), // the last load's projector failed: load without it once
+ quit(root + "/quit"), settings(root + "/settings.json"), scratch(root + "/scratch"),
+ llama_log(root + "/llama.log"), models(root + "/models") {}
+};
+Files g_files(kFullRoot);
+constexpr const char * kFleetWorker = "/data/PS5LM/fleet-worker.txt"; // fleet needs full storage
// The file tools only: a title cannot start processes (no shell command).
constexpr const char * kTools = "read_file,write_file,edit_file,file_glob_search,grep_search,get_info";
-constexpr const char * kLlamaLog = "/data/PS5LM/llama.log";
-constexpr const char * kModels = "/data/PS5LM/models";
constexpr const char * kAssets = "/app0/assets";
constexpr int kPort = 8081;
-const std::vector<std::string> kModelDirs = { kModels, "/mnt/ext0/PS5LM/models", "/mnt/usb0/PS5LM/models",
- "/mnt/usb1/PS5LM/models", "/mnt/usb0", "/mnt/usb1" };
+// Both homes' models are listed: a model saved to the app's storage still
+// loads once full storage opens.
+const std::vector<std::string> kModelDirs = { "/data/PS5LM/models", "/download0/PS5LM/models", "/mnt/ext0/PS5LM/models",
+ "/mnt/usb0/PS5LM/models", "/mnt/usb1/PS5LM/models", "/mnt/usb0", "/mnt/usb1" };
// The last lines of a log, up to about 64 KiB of it, oldest first.
-std::vector<std::string> tail_lines(const char * path) {
+std::vector<std::string> tail_lines(const std::string & path) {
std::vector<std::string> lines;
- FILE * f = std::fopen(path, "rb");
+ FILE * f = std::fopen(path.c_str(), "rb");
if (!f) {
return lines;
}
@@ -140,86 +153,6 @@ std::vector<std::string> tail_lines(const char * path) {
return lines;
}
-// Probe: download a URL over HTTPS to /data/PS5LM/fetch.out, logging the
-// status, size and speed (scripts/ps5lm-app.sh fetch <url>).
-void probe_fetch(std::string url) {
- FILE * out = std::fopen("/data/PS5LM/fetch.out", "wb");
- const FetchResult r = https_get(url, "", [&](const char * p, size_t n) { return out && std::fwrite(p, 1, n, out) == n; });
- if (out) {
- std::fclose(out);
- }
- std::printf("ps5lm-app: fetch %s: status %d, length %lld, %llu bytes in %.1f s (%.2f MB/s)%s%s\n", url.c_str(),
- r.status, (long long) r.length, (unsigned long long) r.bytes, r.seconds,
- r.seconds > 0 ? r.bytes / 1e6 / r.seconds : 0.0, r.error.empty() ? "" : ", ", r.error.c_str());
-}
-
-// Probe: how fast the app can write, by method and place, fsync included
-// (scripts/ps5lm-app.sh bench-write). 128 MiB each.
-void probe_write() {
- const size_t total = 128u << 20;
- std::vector<char> buf(8u << 20, 'x');
- const char * dirs[] = { "/data/PS5LM", "/mnt/usb0" };
- for (const char * dir : dirs) {
- struct statvfs v = {};
- if (statvfs(dir, &v) != 0) {
- continue;
- }
- const std::string path = std::string(dir) + "/write-bench.tmp";
- struct Method {
- const char * name;
- size_t chunk;
- bool stdio;
- int flags;
- } methods[] = {
- { "stdio 8M", 8u << 20, true, 0 },
- { "write 1M", 1u << 20, false, 0 },
- { "write 8M", 8u << 20, false, 0 },
- { "write 8M direct", 8u << 20, false, O_DIRECT },
- };
- for (const auto & m : methods) {
- const auto t0 = std::chrono::steady_clock::now();
- bool ok = true;
- if (m.stdio) {
- FILE * f = std::fopen(path.c_str(), "wb");
- for (size_t done = 0; f && ok && done < total; done += m.chunk) {
- ok = std::fwrite(buf.data(), 1, m.chunk, f) == m.chunk;
- }
- ok = f && ok && std::fflush(f) == 0 && fsync(fileno(f)) == 0;
- if (f) {
- std::fclose(f);
- }
- } else {
- const int fd = open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC | m.flags, 0666);
- for (size_t done = 0; fd >= 0 && ok && done < total; done += m.chunk) {
- ok = write(fd, buf.data(), m.chunk) == (ssize_t) m.chunk;
- }
- ok = fd >= 0 && ok && fsync(fd) == 0;
- if (fd >= 0) {
- close(fd);
- }
- }
- const double s = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
- std::printf("ps5lm-app: write-bench %s %s: %s, %.1f MB/s\n", dir, m.name, ok ? "ok" : strerror(errno),
- ok ? total / 1e6 / s : 0.0);
- unlink(path.c_str());
- }
- }
- std::printf("ps5lm-app: write-bench done\n");
-}
-
-// Bytes under a folder, or 0.
-uint64_t folder_bytes(const char * path) {
- std::error_code ec;
- uint64_t total = 0;
- for (auto it = std::filesystem::recursive_directory_iterator(path, ec);
- !ec && it != std::filesystem::recursive_directory_iterator(); it.increment(ec)) {
- if (it->is_regular_file(ec)) {
- total += it->file_size(ec);
- }
- }
- return total;
-}
-
// A system notification on the TV.
void notify(const char * text) {
struct {
@@ -238,8 +171,8 @@ void notify(const char * text) {
// Open means the models folder lists and a file can be written there.
bool data_open() {
mkdir("/data/PS5LM", 0777);
- mkdir(kModels, 0777);
- DIR * d = opendir(kModels);
+ mkdir("/data/PS5LM/models", 0777);
+ DIR * d = opendir("/data/PS5LM/models");
if (!d) {
return false;
}
@@ -282,10 +215,10 @@ std::string open_data() {
// itself the way the system expects; killing a title that is rendering has
// been followed by lost consoles (ps5-homebrew-ui's AGENTS.md).
void quit_if_asked() {
- if (access(kQuit, F_OK) != 0) {
+ if (access(g_files.quit.c_str(), F_OK) != 0) {
return;
}
- unlink(kQuit);
+ unlink(g_files.quit.c_str());
std::printf("ps5lm-app: quit requested\n");
std::fflush(nullptr);
sceSystemServiceLoadExec("exit", nullptr);
@@ -333,29 +266,34 @@ bool save_screen(const char * path, int width, int height) {
}
// A request file from the PC: its first line, and the file is consumed.
-std::string take_request_file(const char * path) {
- if (access(path, F_OK) != 0) {
+std::string take_request_file(const std::string & path) {
+ if (access(path.c_str(), F_OK) != 0) {
return {};
}
std::string line;
std::ifstream(path) >> line;
- unlink(path);
+ unlink(path.c_str());
return line.empty() ? std::string("-") : line;
}
-// Where the memory is: the pool's free part, the CPU heap, and what ggml
-// says about each device (its GPU figure wraps once the unified heap is
-// over-committed; the raw numbers still show a change).
-void log_memory(const char * when) {
+// Where the memory is: the title's direct-memory pool and its free part,
+// the flexible pool libc and the system libraries use, the CPU heap, and
+// what ggml says about each device (its GPU figure wraps once the unified
+// heap is over-committed; the raw numbers still show a change). `devices`
+// false before the GPU opens: asking ggml would open it.
+void log_memory(const char * when, bool devices = true) {
int64_t at = 0;
- size_t avail = 0;
+ size_t avail = 0, flex = 0, flex_free = 0;
const int64_t pool = sceKernelGetDirectMemorySize();
sceKernelAvailableDirectMemorySize(0, pool, 2 * 1024 * 1024, &at, &avail);
+ sceKernelConfiguredFlexibleMemorySize(&flex);
+ sceKernelAvailableFlexibleMemorySize(&flex_free);
struct ps5_heap_stats heap = {};
ps5_heap_stats(&heap);
- std::printf("ps5lm-app: memory %s: pool free %.2f GiB, cpu heap %.2f GiB (peak %.2f)", when, avail / 1073741824.0,
+ std::printf("ps5lm-app: memory %s: pool %.4f GiB, pool free %.2f GiB, flexible %zu MiB (%zu free), cpu heap %.2f GiB "
+ "(peak %.2f)", when, pool / 1073741824.0, avail / 1073741824.0, flex >> 20, flex_free >> 20,
heap.mapped_bytes / 1073741824.0, heap.peak_bytes / 1073741824.0);
- for (size_t i = 0; i < ggml_backend_dev_count(); i++) {
+ for (size_t i = 0; devices && i < ggml_backend_dev_count(); i++) {
size_t free = 0, total = 0;
ggml_backend_dev_memory(ggml_backend_dev_get(i), &free, &total);
std::printf(", %s free %zu total %zu", ggml_backend_dev_name(ggml_backend_dev_get(i)), free, total);
@@ -363,7 +301,7 @@ void log_memory(const char * when) {
std::printf("\n");
}
-std::vector<std::string> read_lines(const char * path) {
+std::vector<std::string> read_lines(const std::string & path) {
std::vector<std::string> out;
std::ifstream in(path);
for (std::string line; std::getline(in, line);) {
@@ -395,6 +333,224 @@ std::string base_name(const std::string & path) {
return n;
}
+// ---- the debug report ---------------------------------------------------------------
+
+std::string read_text(const std::string & path, size_t max_bytes = 4u << 20) {
+ std::ifstream in(path, std::ios::binary);
+ if (!in) {
+ return {};
+ }
+ in.seekg(0, std::ios::end);
+ const std::streamoff size = in.tellg();
+ const std::streamoff from = size > (std::streamoff) max_bytes ? size - (std::streamoff) max_bytes : 0;
+ in.seekg(from);
+ std::string text((size_t) (size - from), '\0');
+ in.read(text.data(), (std::streamsize) text.size());
+ text.resize((size_t) in.gcount());
+ return text;
+}
+
+// Whether something listens on a port of this console. The connection is
+// reset at once (SO_LINGER 0), so an ELF loader sees an error, never a payload.
+bool port_open(int port) {
+ const int fd = socket(AF_INET, SOCK_STREAM, 0);
+ if (fd < 0) {
+ return false;
+ }
+ sockaddr_in to = {};
+#ifdef __FreeBSD__
+ to.sin_len = sizeof(to);
+#endif
+ to.sin_family = AF_INET;
+ to.sin_port = htons((uint16_t) port);
+ to.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+ const bool open = connect(fd, (sockaddr *) &to, sizeof(to)) == 0;
+ const linger reset = { 1, 0 };
+ setsockopt(fd, SOL_SOCKET, SO_LINGER, &reset, sizeof(reset));
+ close(fd);
+ return open;
+}
+
+// The kernel log, from klogsrv (3232) or etaHEN's (9081): what it sends in a
+// few seconds, the last `max_bytes` of it.
+std::string kernel_log(size_t max_bytes) {
+ for (int port : { 3232, 9081 }) {
+ const int fd = socket(AF_INET, SOCK_STREAM, 0);
+ if (fd < 0) {
+ continue;
+ }
+ timeval tv = { 1, 0 };
+ setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
+ sockaddr_in to = {};
+#ifdef __FreeBSD__
+ to.sin_len = sizeof(to);
+#endif
+ to.sin_family = AF_INET;
+ to.sin_port = htons((uint16_t) port);
+ to.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+ std::string text;
+ if (connect(fd, (sockaddr *) &to, sizeof(to)) == 0) {
+ const auto until = std::chrono::steady_clock::now() + std::chrono::seconds(4);
+ char buf[65536];
+ while (std::chrono::steady_clock::now() < until) {
+ const ssize_t got = recv(fd, buf, sizeof(buf), 0);
+ if (got <= 0) {
+ break; // quiet for a second, or closed
+ }
+ text.append(buf, (size_t) got);
+ if (text.size() > 2 * max_bytes) {
+ text = last_bytes(text, max_bytes);
+ }
+ }
+ close(fd);
+ return text.empty() ? "klog on port " + std::to_string(port) + " sent nothing in 4 s\n"
+ : last_bytes(text, max_bytes);
+ }
+ close(fd);
+ }
+ return "No kernel log server answers (klogsrv on 3232, etaHEN on 9081).\n";
+}
+
+const char * state_name(ServerState s) {
+ switch (s) {
+ case ServerState::loading: return "loading";
+ case ServerState::ready: return "ready";
+ case ServerState::generating: return "generating";
+ case ServerState::failed: return "stopped with an error";
+ default: return "no model";
+ }
+}
+
+// Where the files are, in a line for the log and the debug report.
+std::string storage_line(const std::string & data) {
+ if (!app_storage()) {
+ return data_root() + ", full storage (/data " + data + ")";
+ }
+ char t[160];
+ std::snprintf(t, sizeof(t), "%s, the app's own storage: %.1f of %.0f GiB used (/data ", data_root().c_str(),
+ app_storage_used() / 1073741824.0, app_storage_size() / 1073741824.0);
+ return t + data + ")";
+}
+
+// What the report is made from, copied on the main thread.
+struct ReportJob {
+ Live live;
+ std::vector<Download> downloads;
+ Settings settings;
+ std::string storage; // the data folder and how /data ended up
+ std::string dir; // where the report goes
+};
+
+std::mutex g_report_mutex;
+std::string g_report_status; // for the Settings row
+std::atomic<bool> g_report_busy{ false };
+
+void write_report(ReportJob job) {
+ const auto gib = [](double v) { char t[32]; std::snprintf(t, sizeof(t), "%.2f GiB", v); return std::string(t); };
+ ReportInput in;
+ const std::time_t now = std::time(nullptr);
+ std::tm tm = {};
+ char when[32] = "unknown";
+ if (localtime_r(&now, &tm)) {
+ std::strftime(when, sizeof(when), "%Y-%m-%d %H:%M", &tm);
+ }
+ in.created = when;
+ in.version = PS5LM_VERSION;
+ struct ps5_kernel_sw_version sw = {};
+ sw.size = sizeof(sw);
+ if (sceKernelGetSystemSwVersion(&sw) == 0) {
+ sw.text[sizeof(sw.text) - 1] = '\0';
+ char t[64];
+ std::snprintf(t, sizeof(t), "%s (0x%08x)", sw.text, sw.version);
+ in.firmware = t;
+ in.firmware.erase(0, in.firmware.find_first_not_of(' '));
+ }
+ in.storage = job.storage;
+ for (const auto & d : job.live.drives) {
+ in.drives.push_back(d.name + ": " + (d.total > 0 ? gib(d.free) + " free of " + gib(d.total) : "space unknown (ps5-exporter is not running)") +
+ ", models in " + d.models);
+ }
+ size_t flex = 0, flex_free = 0;
+ sceKernelConfiguredFlexibleMemorySize(&flex);
+ sceKernelAvailableFlexibleMemorySize(&flex_free);
+ in.memory = { "Direct memory pool " + gib(job.live.pool_gib) + ", " + gib(job.live.free_gib) + " free, CPU heap " + gib(job.live.heap_gib),
+ "Flexible memory " + std::to_string(flex >> 20) + " MiB, " + std::to_string(flex_free >> 20) + " MiB free" };
+ for (size_t i = 0; i < ggml_backend_dev_count(); i++) {
+ size_t free = 0, total = 0;
+ ggml_backend_dev_memory(ggml_backend_dev_get(i), &free, &total);
+ in.memory.push_back(std::string(ggml_backend_dev_name(ggml_backend_dev_get(i))) + " (" +
+ ggml_backend_dev_description(ggml_backend_dev_get(i)) + "): " + gib(free / 1073741824.0) + " free of " +
+ gib(total / 1073741824.0));
+ }
+ const Live & L = job.live;
+ in.server = { std::string("State: ") + state_name(L.state), "Address: " + L.address };
+ if (!L.model_label.empty()) {
+ in.server.push_back("Model: " + L.model_file + (L.model_arch.empty() ? "" : " (" + L.model_arch + ")"));
+ in.server.push_back("Plan: " + L.plan);
+ in.server.push_back("Requests answered: " + std::to_string(L.served) + ", tokens generated: " + std::to_string(L.tokens));
+ }
+ in.args = L.args;
+ in.ports = report_ports();
+ for (const auto & [port, what] : in.ports) {
+ if (port_open(port)) {
+ in.open_ports.push_back(port);
+ }
+ }
+ for (const auto & m : L.models) {
+ in.models.push_back(m.file + " (" + m.drive + ", " + gib(m.size_gib) + (m.arch.empty() ? "" : ", " + m.arch) + "): " +
+ m.plan + (m.current ? " [loaded]" : ""));
+ }
+ for (const auto & d : job.downloads) {
+ static const char * states[] = { "queued", "running", "retrying", "paused", "failed", "done" };
+ in.downloads.push_back(d.repo + "/" + d.file + " to " + d.dir + ": " + states[(int) d.state] + ", " +
+ std::to_string(d.done >> 20) + " of " + std::to_string(d.bytes >> 20) + " MiB" +
+ (d.note.empty() ? "" : ", " + d.note));
+ }
+ in.settings_json = read_text(g_files.settings);
+ for (const std::string & f : { g_files.env, g_files.args }) {
+ const std::string text = read_text(f, 64u << 10);
+ if (!text.empty()) {
+ in.files.emplace_back(f.substr(f.find_last_of('/') + 1), text);
+ }
+ }
+ in.logs = { { "app.log, the last 256 KiB", last_bytes(read_text(g_files.log), 256u << 10) },
+ { "llama.log, the last 256 KiB", last_bytes(read_text(g_files.llama_log), 256u << 10) } };
+ in.kernel_log_asked = job.settings.report_kernel_log;
+ if (in.kernel_log_asked) {
+ in.kernel_log = kernel_log(1u << 20);
+ }
+
+ const std::string text = build_report(in);
+ const std::string path = job.dir + "/debug-report.txt";
+ bool ok = false;
+ {
+ std::ofstream out(path, std::ios::binary | std::ios::trunc);
+ ok = (bool) (out << text);
+ }
+ // A copy on a USB drive, for a console without FTP.
+ std::string usb;
+ for (const char * drive : { "/mnt/usb0", "/mnt/usb1" }) {
+ std::ofstream out(std::string(drive) + "/PS5LM-debug-report.txt", std::ios::binary | std::ios::trunc);
+ if (out && (out << text)) {
+ usb = drive;
+ break;
+ }
+ }
+ char status[256];
+ if (ok) {
+ std::snprintf(status, sizeof(status), "Saved to %s%s, %zu KB", path.c_str(),
+ usb.empty() ? "" : (" and " + usb + "/PS5LM-debug-report.txt").c_str(), (text.size() + 1023) / 1024);
+ } else {
+ std::snprintf(status, sizeof(status), "Failed: cannot write %s%s", path.c_str(),
+ usb.empty() ? "" : ("; saved to " + usb + "/PS5LM-debug-report.txt").c_str());
+ }
+ std::printf("ps5lm-app: debug report: %s\n", status);
+ notify((std::string("PS5LM: debug report ") + (ok ? "saved to " + path : status)).c_str());
+ std::lock_guard<std::mutex> lock(g_report_mutex);
+ g_report_status = status;
+ g_report_busy = false;
+}
+
// ---- the server thread ------------------------------------------------------------
std::atomic<int> g_server_rc{ -1 }; // llama-server's exit code once it returns; -1 while it runs
@@ -550,7 +706,7 @@ struct Models {
void start(const std::string & path, std::vector<std::string> args, StatsCollector & stats, const Live & model_part) {
// llama.cpp's own log, written as it goes (it survives a crash), and
// the metrics the dashboard reads.
- args.insert(args.begin(), { "--log-file", "/data/PS5LM/llama.log" });
+ args.insert(args.begin(), { "--log-file", g_files.llama_log });
if (std::find(args.begin(), args.end(), "--metrics") == args.end()) {
args.push_back("--metrics");
}
@@ -558,9 +714,9 @@ struct Models {
// patches/0003 (LLAMA_TOOLS_ROOT) and working there by default.
if (settings.tools && std::find(args.begin(), args.end(), "--tools") == args.end()) {
std::error_code ec;
- std::filesystem::create_directories(kScratch, ec);
- setenv("LLAMA_TOOLS_ROOT", kScratch, 1);
- chdir(kScratch);
+ std::filesystem::create_directories(g_files.scratch, ec);
+ setenv("LLAMA_TOOLS_ROOT", g_files.scratch.c_str(), 1);
+ chdir(g_files.scratch.c_str());
args.insert(args.end(), { "--tools", kTools });
}
for (const auto & a : args) {
@@ -630,29 +786,37 @@ bool load_font(hui::gfx::Renderer & renderer, const char * name, hui::gfx::Font
} // namespace
int main(int, char **) {
- // /data first: the log, the settings and the models live there.
+ // /data first: the log, the settings and the models live there when the
+ // HEN opens it, else in the app's own storage, /download0/PS5LM.
const std::string data = open_data();
+ const bool full = data.rfind("closed", 0) != 0;
+ set_data_root(full ? kFullRoot : kAppRoot);
+ g_files = Files(data_root());
+ mkdir(data_root().c_str(), 0777);
// stdout and stderr to a file the PC can fetch over FTP.
- if (std::freopen(kLog, "w", stdout) != nullptr) {
+ if (std::freopen(g_files.log.c_str(), "w", stdout) != nullptr) {
setvbuf(stdout, nullptr, _IOLBF, 0);
}
- if (std::freopen(kLog, "a", stderr) != nullptr) {
+ if (std::freopen(g_files.log.c_str(), "a", stderr) != nullptr) {
setvbuf(stderr, nullptr, _IONBF, 0);
}
std::printf("ps5lm-app: version " PS5LM_VERSION "\n");
std::printf("ps5lm-app: /data %s\n", data.c_str());
- if (data.rfind("closed", 0) == 0) {
- notify("PS5LM cannot write to /data. On OnionHEN, add PPSA99575 to app_jailbreak.exact_title_ids "
- "in /data/OnionHEN/config.ini and start PS5LM again.");
+ std::printf("ps5lm-app: files in %s\n", storage_line(data).c_str());
+ // The first start without full storage asks on the screen what to do;
+ // the notification is for a screen that does not come up.
+ if (!full && storage_choice().empty()) {
+ notify("PS5LM: full storage (/data) is closed to the app, so it keeps its files in its own storage. "
+ "On OnionHEN, add PPSA99575 to app_jailbreak.exact_title_ids in /data/OnionHEN/config.ini.");
}
// One device-local heap over the whole pool (RADV splits an APU's 2/3 to
// 1/3), and a HOME for llama.cpp, which throws without one.
setenv("radv_enable_unified_heap_on_apu", "true", 1);
- setenv("HOME", "/data/PS5LM", 0);
- setenv("LLAMA_CACHE", "/data/PS5LM/cache", 0);
+ setenv("HOME", data_root().c_str(), 0);
+ setenv("LLAMA_CACHE", data_path("cache").c_str(), 0);
// Driver and backend switches to measure with, one NAME=value a line in
// env.txt, set before the GPU opens; only the driver's, ggml's and llama's.
- for (const auto & line : read_lines(kEnv)) {
+ for (const auto & line : read_lines(g_files.env)) {
const size_t eq = line.find('=');
const std::string name = line.substr(0, eq == std::string::npos ? 0 : eq);
bool known = false;
@@ -664,8 +828,11 @@ int main(int, char **) {
}
std::printf("ps5lm-app: env %s%s\n", line.c_str(), known ? "" : " (ignored)");
}
- mkdir(kModels, 0777);
+ mkdir(g_files.models.c_str(), 0777);
std::printf("ps5lm-app: start\n");
+ // What the system gave the title, before the screen and the GPU take any
+ // of it: the pool size follows sce_sys/param.json.
+ log_memory("at start", false);
// The screen first: it takes from the same memory as the model.
Display display;
@@ -777,12 +944,16 @@ int main(int, char **) {
stats.start(kPort);
set_activity_sink(&stats); // llama-server's request hooks (patches/0007) report here
Models models;
- models.settings = load_settings(kSettings);
+ models.settings = load_settings(g_files.settings.c_str());
+ // Without full storage, downloads go where the app can write.
+ if (app_storage() && models.settings.download_dir.rfind(kFullRoot, 0) == 0) {
+ models.settings.download_dir = g_files.models;
+ }
// The model about to load, by name, for Home while the GPU opens.
{
- std::vector<std::string> next = read_lines(kArgs);
+ std::vector<std::string> next = read_lines(g_files.args);
const auto m = std::find(next.begin(), next.end(), "-m");
- next = m != next.end() && m + 1 != next.end() ? std::vector<std::string>{ *(m + 1) } : read_lines(kChoice);
+ next = m != next.end() && m + 1 != next.end() ? std::vector<std::string>{ *(m + 1) } : read_lines(g_files.choice);
if (next.empty() && models.settings.auto_load) {
next = { models.settings.default_model };
}
@@ -819,7 +990,7 @@ int main(int, char **) {
// once, else the default model when Settings asks for it at launch. A
// plain launch otherwise loads nothing and opens the library.
Live model_part;
- std::vector<std::string> args = read_lines(kArgs);
+ std::vector<std::string> args = read_lines(g_files.args);
std::string path;
if (!args.empty()) {
const auto m = std::find(args.begin(), args.end(), "-m");
@@ -832,8 +1003,8 @@ int main(int, char **) {
model_part.args = args;
std::printf("ps5lm-app: app-args.txt overrides the planner\n");
} else {
- auto chosen = read_lines(kChoice);
- unlink(kChoice);
+ auto chosen = read_lines(g_files.choice);
+ unlink(g_files.choice.c_str());
if (chosen.empty() && models.settings.auto_load && !models.settings.default_model.empty()) {
chosen = { models.settings.default_model }; // "none" after an unload keeps it from coming back
}
@@ -848,8 +1019,8 @@ int main(int, char **) {
}
// A projector this llama.cpp cannot read stops the server (#15); the
// restart after it loads the same model text only, this once.
- if (access(kTextOnly, F_OK) == 0) {
- unlink(kTextOnly);
+ if (access(g_files.text_only.c_str(), F_OK) == 0) {
+ unlink(g_files.text_only.c_str());
const auto mm = std::find(args.begin(), args.end(), "--mmproj");
if (mm != args.end()) {
std::printf("ps5lm-app: loading without %s\n", mm + 1 != args.end() ? (mm + 1)->c_str() : "the projector");
@@ -867,7 +1038,7 @@ int main(int, char **) {
} else {
stats.set_state(ServerState::no_model);
if (models.rows.empty()) {
- notify("PS5LM: no models; copy a .gguf to /data/PS5LM/models or PS5LM/models on a USB drive");
+ notify(("PS5LM: no models; copy a .gguf to " + g_files.models + " or PS5LM/models on a USB drive").c_str());
}
}
stats.set_phase("");
@@ -912,9 +1083,10 @@ int main(int, char **) {
market.start(models.budget_gib);
int saved_seen = 0; // downloads finished, as last seen
dash.set_settings(models.settings);
+ dash.set_storage(!app_storage(), app_storage_size() / 1073741824.0, app_storage() && storage_choice().empty());
bool scratch_warned = false;
// Downloads left queued or running carry on, on their own page.
- unlink("/data/PS5LM/download.txt"); // the queue file replaced this one (v0.7.0)
+ unlink(data_path("download.txt").c_str()); // the queue file replaced this one (v0.7.0)
market.load_queue();
for (const auto & d : market.snapshot().downloads) {
// Running already, maybe: the queue's thread starts it at once.
@@ -941,19 +1113,20 @@ int main(int, char **) {
stats.set_state(ServerState::failed);
const bool projector = std::find(g_server_args.begin(), g_server_args.end(), "--mmproj") != g_server_args.end();
if (live.state == ServerState::loading && projector && restart_at == 0) {
- std::ofstream(kChoice) << models.running << "\n";
- std::ofstream(kTextOnly) << models.running << "\n";
+ std::ofstream(g_files.choice) << models.running << "\n";
+ std::ofstream(g_files.text_only) << models.running << "\n";
notify("PS5LM: the image projector did not load; loading the model text only");
dash.set_busy("Loading " + base_name(models.running) + " text only",
- "Its image projector did not load, see /data/PS5LM/llama.log");
+ "Its image projector did not load, see " + g_files.llama_log);
restart_at = n + 3;
} else {
- notify("PS5LM: the server stopped, see /data/PS5LM/llama.log");
+ notify(("PS5LM: the server stopped, see " + g_files.llama_log).c_str());
}
}
const bool serving = live.state == ServerState::ready || live.state == ServerState::generating;
if (serving && !was_serving) {
notify(live.decision.empty() ? "PS5LM: ready, chat in a browser on this network" : "PS5LM: ready for decisions at /v1/systemone");
+ log_memory("when ready"); // the weights, the cache and the compute buffers are all allocated by now
}
was_serving = serving;
if (n % 30 == 0) {
@@ -965,7 +1138,7 @@ int main(int, char **) {
// testing the screens without a controller in hand.
hui::InputFrame in = input;
if (n % 20 == 10) {
- const std::string key = take_request_file("/data/PS5LM/press");
+ const std::string key = take_request_file(data_path("press"));
const struct {
const char * name;
hui::Action action;
@@ -1007,7 +1180,7 @@ int main(int, char **) {
// The scratch folder: its size for the dashboard, and one warning each
// time it grows past the limit.
if (n % 600 == 300) {
- const double gib = folder_bytes(kScratch) / 1073741824.0;
+ const double gib = folder_bytes(g_files.scratch) / 1073741824.0;
stats.set_scratch(gib);
const bool over = gib > models.settings.scratch_limit_gib;
if (over && !scratch_warned) {
@@ -1029,7 +1202,7 @@ int main(int, char **) {
dash.set_market(std::move(mv));
}
if (dash.wants_logs() && n % 60 == 0) {
- dash.set_logs(tail_lines(kLog), tail_lines(kLlamaLog));
+ dash.set_logs(tail_lines(g_files.log), tail_lines(g_files.llama_log));
}
// A settings rescan in the background: say so until it lands.
@@ -1058,7 +1231,7 @@ int main(int, char **) {
// reads every model's header, seconds with a USB drive.
const Settings was = models.settings;
models.settings = dash.settings();
- save_settings(kSettings, models.settings);
+ save_settings(g_files.settings.c_str(), models.settings);
if (was.ctx_cap != models.settings.ctx_cap || was.kv_type != models.settings.kv_type) {
models.rescan(stats);
}
@@ -1066,7 +1239,7 @@ int main(int, char **) {
// The server reads LLAMA_INTROSPECT only when it starts: restart with the
// same model, so that the switch never says something the server does not do.
if (was.introspect != models.settings.introspect && !models.running.empty() && restart_at == 0) {
- std::ofstream(kChoice) << models.running << "\n";
+ std::ofstream(g_files.choice) << models.running << "\n";
std::printf("ps5lm-app: restarting %s with the introspection API %s\n", models.running.c_str(),
models.settings.introspect ? "on" : "off");
dash.set_busy(std::string(models.settings.introspect ? "Opening" : "Closing") + " the logits and activations API",
@@ -1081,19 +1254,24 @@ int main(int, char **) {
case Dashboard::Request::market_plan: market.plan_file(req.repo, req.file); break;
case Dashboard::Request::market_download: {
MarketJob job;
- if (market.job(req.repo, req.file, req.path.empty() ? models.settings.download_dir : req.path, &job)) {
- market.enqueue(job, free_gib(job.dir));
+ if (market.job(req.repo, req.file, req.path.empty() ? models.settings.download_dir : req.path, &job) &&
+ !market.enqueue(job, free_gib(job.dir))) {
+ notify(("PS5LM: " + market.snapshot().note).c_str());
}
break;
}
case Dashboard::Request::suggested_download: {
const Suggestion & s = kSuggestions[std::clamp(req.repo, 0, kSuggestionCount - 1)];
const std::string dir = req.path.empty() ? models.settings.download_dir : req.path;
+ std::vector<MarketJob> jobs;
for (const SuggestedFile * f : { &s.model, &s.mmproj }) {
if (f->file) {
- market.enqueue({ s.repo, f->file, f->sha256, dir, f->bytes, f->save_as ? f->save_as : "" }, free_gib(dir));
+ jobs.push_back({ s.repo, f->file, f->sha256, dir, f->bytes, f->save_as ? f->save_as : "" });
}
}
+ if (!market.enqueue(jobs, free_gib(dir))) {
+ notify(("PS5LM: " + market.snapshot().note).c_str());
+ }
dash.show_downloads();
break;
}
@@ -1106,7 +1284,7 @@ int main(int, char **) {
MarketJob job;
if (market.job(req.repo, req.file, req.path.empty() ? models.settings.download_dir : req.path, &job) &&
market.enqueue(job, free_gib(job.dir))) {
- std::ofstream(kChoice) << "none\n";
+ std::ofstream(g_files.choice) << "none\n";
std::printf("ps5lm-app: restarting to download %s/%s without a model\n", job.repo.c_str(), job.file.c_str());
dash.set_busy("Unloading " + base_name(models.running), "The app restarts without it, then the download starts");
restart_at = n + 3;
@@ -1117,24 +1295,63 @@ int main(int, char **) {
}
if (req.kind == Dashboard::Request::clear_scratch) {
std::error_code ec;
- for (const auto & e : std::filesystem::directory_iterator(kScratch, ec)) {
+ for (const auto & e : std::filesystem::directory_iterator(g_files.scratch, ec)) {
std::filesystem::remove_all(e.path(), ec);
}
- stats.set_scratch(folder_bytes(kScratch) / 1073741824.0);
+ stats.set_scratch(folder_bytes(g_files.scratch) / 1073741824.0);
notify("PS5LM: scratch folder cleared");
std::printf("ps5lm-app: scratch folder cleared\n");
}
+ // Full storage closed: the app's own storage is the answer from now on,
+ // or a fresh start asks the HEN again.
+ if (req.kind == Dashboard::Request::storage_app) {
+ set_storage_choice("app");
+ std::printf("ps5lm-app: the app's own storage, chosen\n");
+ } else if (req.kind == Dashboard::Request::restart) {
+ if (!models.running.empty()) {
+ std::ofstream(g_files.choice) << models.running << "\n";
+ }
+ std::printf("ps5lm-app: restarting to ask for /data again\n");
+ dash.set_busy("Starting again", "PS5LM asks the HEN for full storage once more");
+ restart_at = n + 3;
+ }
+ // A debug report, on a thread of its own: the ports and the kernel log
+ // take seconds. The first one was explained on the screen; remember it.
+ if (req.kind == Dashboard::Request::debug_report && !g_report_busy.exchange(true)) {
+ if (!models.settings.report_explained) {
+ models.settings.report_explained = true;
+ dash.set_settings(models.settings);
+ save_settings(g_files.settings.c_str(), models.settings);
+ }
+ {
+ std::lock_guard<std::mutex> lock(g_report_mutex);
+ g_report_status = "Writing the report";
+ }
+ ReportJob job{ stats.snapshot(), market.snapshot().downloads, models.settings,
+ storage_line(data), data_root() };
+ std::thread(write_report, std::move(job)).detach();
+ }
+ if (n % 10 == 5) {
+ std::lock_guard<std::mutex> lock(g_report_mutex);
+ dash.set_report_status(g_report_busy, g_report_status);
+ }
// The same requests from the PC (scripts/ps5lm-app.sh load|unload).
if (n % 60 == 30 && req.kind == Dashboard::Request::none) {
- if (std::string p = take_request_file("/data/PS5LM/load"); !p.empty()) {
+ if (std::string p = take_request_file(data_path("load")); !p.empty()) {
req = { Dashboard::Request::load, p };
- } else if (!take_request_file("/data/PS5LM/unload").empty()) {
+ } else if (!take_request_file(data_path("unload")).empty()) {
req = { Dashboard::Request::unload, "" };
- } else if (std::string u = take_request_file("/data/PS5LM/fetch"); !u.empty()) {
+ } else if (std::string u = take_request_file(data_path("fetch")); !u.empty()) {
std::thread(probe_fetch, u).detach();
- } else if (!take_request_file("/data/PS5LM/bench-write").empty()) {
+ } else if (!take_request_file(data_path("bench-write")).empty()) {
std::thread(probe_write).detach();
- } else if (std::string q = take_request_file("/data/PS5LM/search"); !q.empty()) {
+ } else if (!take_request_file(data_path("memory")).empty()) {
+ log_memory("asked"); // scripts/bench-ubatch.py: what a prompt left allocated
+ } else if (std::string f = take_request_file(data_path("bench-read")); !f.empty()) {
+ std::thread(probe_read, f).detach();
+ } else if (!take_request_file(data_path("bench-memory")).empty()) {
+ std::thread(probe_memory, !models.running.empty()).detach();
+ } else if (std::string q = take_request_file(data_path("search")); !q.empty()) {
market.search(q == "-" ? std::string() : q); // a query typed on the PC
}
}
@@ -1144,9 +1361,9 @@ int main(int, char **) {
// restarts the app: the system replaces the process (LoadExec on
// our own eboot.bin, "Kill for LoadExec" in the log) and the new
// one starts from the whole pool, with model.txt saying what to load.
- std::ofstream(kChoice) << (req.kind == Dashboard::Request::load ? req.path : std::string("none")) << "\n";
+ std::ofstream(g_files.choice) << (req.kind == Dashboard::Request::load ? req.path : std::string("none")) << "\n";
if (req.kind == Dashboard::Request::load) {
- unlink(kArgs); // the library's choice replaces a manual override
+ unlink(g_files.args.c_str()); // the library's choice replaces a manual override
}
std::printf("ps5lm-app: restarting for %s\n", req.kind == Dashboard::Request::load ? req.path.c_str() : "no model");
const bool loaded = !models.running.empty();
@@ -1162,8 +1379,8 @@ int main(int, char **) {
if (ui_ok) {
render();
- if (n % 60 == 45 && !take_request_file("/data/PS5LM/screenshot").empty()) {
- const bool ok = save_screen("/data/PS5LM/screen.bmp", display.width(), display.height());
+ if (n % 60 == 45 && !take_request_file(data_path("screenshot")).empty()) {
+ const bool ok = save_screen(data_path("screen.bmp").c_str(), display.width(), display.height());
std::printf("ps5lm-app: screenshot %s\n", ok ? "saved" : "failed");
}
}
diff --git a/ps5/app/market.cpp b/ps5/app/market.cpp
index f440253..f4f230e 100644
--- a/ps5/app/market.cpp
+++ b/ps5/app/market.cpp
@@ -4,6 +4,7 @@
#include "fetch.hpp"
#include "ftp_writer.hpp"
#include "model_plan.hpp"
+#include "storage.hpp"
#include <nlohmann/json.hpp>
@@ -11,6 +12,7 @@ extern "C" {
#include "hash/sha256/sha256.h"
}
+#include <cerrno>
#include <cstdio>
#include <algorithm>
#include <chrono>
@@ -225,7 +227,6 @@ bool Market::job(int repo, int file, const std::string & dir, MarketJob * out) {
namespace {
-constexpr const char * kQueue = "/data/PS5LM/downloads.json";
// Pieces at once, each on its own template (made once, one after the
// other). Measured on the console (CHANGELOG 0.8.0): one connection gets 3
@@ -268,11 +269,11 @@ void Market::save_queue() {
j.push_back({ { "repo", d.repo }, { "file", d.file }, { "sha256", d.sha256 }, { "dir", d.dir }, { "bytes", d.bytes },
{ "state", state_name(d.state) }, { "note", d.note }, { "resume", d.resume }, { "save_as", d.save_as } });
}
- std::ofstream(kQueue) << j.dump(1) << "\n";
+ std::ofstream(data_path("downloads.json")) << j.dump(1) << "\n";
}
void Market::load_queue() {
- std::ifstream in(kQueue);
+ std::ifstream in(data_path("downloads.json"));
const nlohmann::json j = nlohmann::json::parse(in, nullptr, false);
if (!j.is_array()) {
return;
@@ -303,29 +304,51 @@ void Market::load_queue() {
}
}
-bool Market::enqueue(const MarketJob & job, double free) {
+bool Market::enqueue(const MarketJob & job, double free) { return enqueue(std::vector<MarketJob>{ job }, free); }
+
+bool Market::enqueue(const std::vector<MarketJob> & jobs, double free) {
std::lock_guard<std::mutex> lock(mutex_);
+ if (jobs.empty()) {
+ return false;
+ }
+ for (const auto & job : jobs) {
+ for (const auto & d : view_.downloads) {
+ if (d.repo == job.repo && d.file == job.file && d.state != DownloadState::done) {
+ view_.note = job.file + " is in Downloads already";
+ return false;
+ }
+ }
+ }
+ // Room for the files, the downloads queued to the same drive before them,
+ // and 1 GiB to spare, so a download never fills a drive (a full /data
+ // upsets the system, not only this app) and never stops halfway for want
+ // of room. A model and its image projector are asked for together.
+ const std::string & dir = jobs[0].dir;
+ double need = 0, queued = 0;
+ for (const auto & job : jobs) {
+ need += job.bytes / 1073741824.0;
+ }
for (const auto & d : view_.downloads) {
- if (d.repo == job.repo && d.file == job.file && d.state != DownloadState::done) {
- view_.note = job.file + " is in Downloads already";
- return false;
+ if (d.dir == dir && d.state != DownloadState::done) {
+ queued += (double) (d.bytes - std::min(d.done, d.bytes)) / 1073741824.0;
}
}
- // Room for the file and 1 GiB to spare, so a download never fills a
- // drive (a full /data upsets the system, not only this app).
- if (free >= 0 && free < job.bytes / 1073741824.0 + 1.0) {
- char text[256];
- std::snprintf(text, sizeof(text), "Not enough space: %s needs %.1f GiB and 1 GiB spare, %s has %.1f GiB free",
- job.file.c_str(), job.bytes / 1073741824.0, job.dir.c_str(), free);
+ if (free >= 0 && free - queued < need + 1.0) {
+ char text[320];
+ std::snprintf(text, sizeof(text), "Not enough space: %s%s needs %.1f GiB and 1 GiB spare, %s has %.1f GiB free%s",
+ jobs[0].file.c_str(), jobs.size() > 1 ? " and its image projector" : "", need, drive_name(dir).c_str(),
+ std::max(0.0, free - queued), queued > 0 ? " after the downloads queued there" : "");
view_.note = text;
- std::printf("ps5lm-app: download %s/%s: %s\n", job.repo.c_str(), job.file.c_str(), text);
+ std::printf("ps5lm-app: download %s/%s: %s\n", jobs[0].repo.c_str(), jobs[0].file.c_str(), text);
return false;
}
- Download d;
- static_cast<MarketJob &>(d) = job;
- d.resume = d.done = 0; // ponytail: a stray partial file is fetched again, its progress is not known
- view_.downloads.push_back(d);
- view_.note = job.file + " added to Downloads";
+ for (const auto & job : jobs) {
+ Download d;
+ static_cast<MarketJob &>(d) = job;
+ d.resume = d.done = 0; // ponytail: a stray partial file is fetched again, its progress is not known
+ view_.downloads.push_back(d);
+ }
+ view_.note = jobs[0].file + " added to Downloads";
save_queue();
if (!working_.exchange(true)) {
std::thread([this] { worker(); }).detach();
@@ -462,7 +485,13 @@ bool Market::fetch(Download & d) {
}
FtpWriter ftp;
std::string ftp_error;
- const bool via_ftp = connect_ftp_writer(ftp, ftp_error);
+ // The app's own storage (/download0) exists only in its sandbox; a payload
+ // writing that path would write somewhere else.
+ const bool own_storage = d.dir.rfind("/download0", 0) == 0;
+ if (own_storage) {
+ ftp_error = "the app's own storage is not visible to ftpsrv";
+ }
+ const bool via_ftp = !own_storage && connect_ftp_writer(ftp, ftp_error);
const std::string direct_note = via_ftp ? "" : "written without ftpsrv, slower after about 2 GB";
if (!via_ftp) {
std::printf("ps5lm-app: download %s: the app writes the file itself, which gets slow after about 2.2 GB (%s)\n",
@@ -485,6 +514,7 @@ bool Market::fetch(Download & d) {
std::atomic<int> failures{ 0 }; // in a row, across connections
std::atomic<int> waiting{ 0 }; // connections waiting to try again
std::atomic<bool> fatal{ false };
+ std::atomic<int> write_errno{ 0 }; // a write refused for want of room ends the download
std::string last_error;
// Through ftpsrv: once the pieces before k are written, k at its place. A
// piece counts once ftpsrv confirmed it (226); a failed one is cut off by
@@ -544,6 +574,7 @@ bool Market::fetch(Download & d) {
if (via_ftp) {
buf.insert(buf.end(), p, p + n);
} else if (pwrite(fd, p, n, (off_t) (off + got)) != (ssize_t) n) {
+ write_errno = errno;
return false;
}
got += n;
@@ -576,6 +607,12 @@ bool Market::fetch(Download & d) {
const int f = ++failures;
std::printf("ps5lm-app: download %s: piece at %llu stopped after %llu bytes: %s\n", d.file.c_str(),
(unsigned long long) off, (unsigned long long) got, text);
+ const int werr = write_errno;
+ if (werr == ENOSPC || werr == EDQUOT || werr == EFBIG) {
+ last_error = drive_name(d.dir) + " is full";
+ fatal = true;
+ return;
+ }
// A 4xx other than 408/429 will not get better by asking again.
if ((r.status >= 400 && r.status < 500 && r.status != 408 && r.status != 429) || r.status == 200 || f > 20 ||
(fetched && ftp_disk_full(write_error))) {
diff --git a/ps5/app/market.hpp b/ps5/app/market.hpp
index db35b65..37b1d5f 100644
--- a/ps5/app/market.hpp
+++ b/ps5/app/market.hpp
@@ -77,6 +77,7 @@ class Market {
// (the check is skipped). False, with the reason in the view's note, when
// it does not fit or is queued already.
bool enqueue(const MarketJob & job, double free_gib);
+ bool enqueue(const std::vector<MarketJob> & jobs, double free_gib); // all on one drive, all or none
void pause(int index); // stops it, keeping the partial file
void resume(int index); // a paused or failed one goes back in the queue
void remove(int index); // cancels it and deletes the partial file; a finished one only leaves the list
diff --git a/ps5/app/model_plan.cpp b/ps5/app/model_plan.cpp
index 3abef0d..e748ea4 100644
--- a/ps5/app/model_plan.cpp
+++ b/ps5/app/model_plan.cpp
@@ -212,6 +212,16 @@ ModelInfo parse_model_info(Reader r, const std::string & path, uint64_t file_byt
if (!strs["general.name"].empty()) {
m.name = strs["general.name"];
}
+ // PrismML's own packings of Bonsai models (their fork's types 142 and 143,
+ // file types 141 and 143), which this llama.cpp refuses at load.
+ if (ints.count("prism.hadamard.version") && ints.count("general.file_type")) {
+ const uint64_t ftype = ints["general.file_type"];
+ if (ftype == 141 || ftype == 143) {
+ m.error = ftype == 141 ? "PrismML's PQ2_0: convert it to Q2_0 on a PC with scripts/bonsai-pq2-to-q2.py"
+ : "PrismML's PTQ1_0, which llama.cpp cannot load; get its Q2_0 file";
+ return m;
+ }
+ }
const std::string a = m.arch + ".";
m.decision = strs[a + "decision.type"];
auto num = [&](const std::string & k, uint64_t def = 0) {
@@ -221,6 +231,7 @@ ModelInfo parse_model_info(Reader r, const std::string & path, uint64_t file_byt
// block_count includes the MTP blocks some files carry (Qwen 3.8's
// UD-Q2_K_XL has 65); they are not part of the main stack.
m.n_layer = (uint32_t) (num("block_count") - num("nextn_predict_layers"));
+ m.n_ff = (uint32_t) num("feed_forward_length");
const uint64_t n_head = num("attention.head_count");
m.n_head_kv = (uint32_t) num("attention.head_count_kv", n_head);
const uint64_t n_embd = num("embedding_length");
@@ -276,6 +287,9 @@ double kv_bytes_per_token(const ModelInfo & m, const std::string & type) {
// out (context, cache type) are not here.
const std::vector<Preset> & presets() {
static const std::vector<Preset> table = {
+ // PrismML's model card (thinking mode); min_p 0.05 is not in the file.
+ { "bonsai", "Bonsai 2", "Qwen 3.8 27B in ternary weights; Q2_0 is 7.1 GiB and keeps 64k with room to spare",
+ { "--temp", "1.0", "--top-p", "0.95", "--top-k", "20", "--min-p", "0.05" }, 65536 },
{ "qwen3.8", "Qwen 3.8", "27B hybrid Gated DeltaNet; UD-Q2_K_XL holds 64k context on the GPU",
{ "--temp", "0.6", "--top-p", "0.95", "--top-k", "20", "--min-p", "0" } },
{ "qwen3.6", "Qwen 3.6", "MoE 35B-A3B or dense 27B; the MoE decodes fastest",
@@ -313,7 +327,7 @@ const Preset * find_preset(const ModelInfo & m) {
return nullptr;
}
-Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap, const char * kv_force) {
+Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap, const char * kv_force, uint32_t ubatch) {
Plan p;
const Preset * preset = find_preset(m);
if (preset && preset->max_ctx) {
@@ -336,8 +350,15 @@ Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap, const
// encoders (mtmd caps an image to the ubatch) need a whole prompt or image in
// one ubatch, so they get 2048, the size #13 ran on the console.
const bool wide = !m.decision.empty() || !m.mmproj.empty();
+ p.ubatch = wide ? 2048 : std::max<uint32_t>(ubatch, 1);
+ // Compute buffers grow with the ubatch, by about 38 bytes a token for each
+ // unit of feed-forward width: on Qwen 3.8 27B (17,408 wide), at 64k with a
+ // q4_0 cache, ggml-vulkan's device and host compute buffers took 41 MiB
+ // at -ub 64 and 321 MiB at 512, measured on a PC with the console's shader
+ // paths. The 0.75 GiB above holds the 64 the console was calibrated at.
+ const double ub_extra = !wide && p.ubatch > 64 ? (p.ubatch - 64) * (double) m.n_ff * 38.0 / kGiB : 0.0;
// shortcut: 0.5 GiB more compute buffer for the 2048 ubatch is a guess; measure on the console.
- p.reserve_gib = 0.75 + p.state_gib + (m.mmproj.empty() ? 0.0 : 0.25) + (wide ? 0.5 : 0.0);
+ p.reserve_gib = 0.75 + p.state_gib + (m.mmproj.empty() ? 0.0 : 0.25) + (wide ? 0.5 : 0.0) + ub_extra;
// The longest context first, and at each length the best cache that fits.
static const uint32_t ctxs[] = { 131072, 65536, 32768, 16384, 8192, 4096 };
@@ -384,7 +405,7 @@ Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap, const
p.args = { "-m", m.path, "-ngl", "999", "-fit", "off", "-lm", "none",
"-c", std::to_string(p.ctx), "-fa", "on", "-ctk", p.kv_type, "-ctv", p.kv_type,
- "-b", wide ? "2048" : "256", "-ub", wide ? "2048" : "64", "-np", "1",
+ "-b", std::to_string(std::max<uint32_t>(256, p.ubatch)), "-ub", std::to_string(p.ubatch), "-np", "1",
// One prompt checkpoint and no RAM cache: llama-server's defaults
// (32 checkpoints, 8 GiB) are for a PC, and the CPU shares this pool.
"-ctxcp", "1", "-cram", "0",
diff --git a/ps5/app/model_plan.hpp b/ps5/app/model_plan.hpp
index f76c942..2656a6a 100644
--- a/ps5/app/model_plan.hpp
+++ b/ps5/app/model_plan.hpp
@@ -24,6 +24,7 @@ struct ModelInfo {
std::string decision; // "<arch>.decision.type" (openjev, clef, lfm2-d1, ...): answers /v1/systemone, not chat
uint64_t file_bytes = 0;
uint32_t n_layer = 0;
+ uint32_t n_ff = 0; // feed-forward width, the widest layer's
uint32_t n_head_kv = 0;
uint32_t key_length = 0; // per KV head
uint32_t value_length = 0;
@@ -64,9 +65,16 @@ struct Preset {
const std::vector<Preset> & presets();
const Preset * find_preset(const ModelInfo & m);
+// llama-server's -ub for language models: how many prompt tokens go through
+// the GPU at once. 64 is what the console has run since the first release;
+// scripts/bench-ubatch.py measures the others there. Decision models and
+// image encoders take 2048 whatever this says (a whole prompt or image in one).
+constexpr uint32_t kPromptUbatch = 64;
+
struct Plan {
bool fits = false;
uint32_t ctx = 0;
+ uint32_t ubatch = 0;
std::string kv_type; // cache type for K and V
double weights_gib = 0;
double kv_gib = 0;
@@ -79,7 +87,9 @@ struct Plan {
// budget_gib: device memory free before loading (the app reads it from ggml).
// kv_force: one cache type ("f16", "q8_0", "q4_0") instead of the best that fits.
-Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap = 65536, const char * kv_force = nullptr);
+// ubatch: the prompt batch for a language model (its compute buffers grow with it).
+Plan plan_model(const ModelInfo & m, double budget_gib, uint32_t ctx_cap = 65536, const char * kv_force = nullptr,
+ uint32_t ubatch = kPromptUbatch);
// Every .gguf in the given folders (not recursive), sorted by name. Split
// files are listed by their first part only.
diff --git a/ps5/app/model_plan_test.cpp b/ps5/app/model_plan_test.cpp
index 681ac10..8bfcbea 100644
--- a/ps5/app/model_plan_test.cpp
+++ b/ps5/app/model_plan_test.cpp
@@ -62,6 +62,36 @@ int main(int argc, char ** argv) {
const ps5lm::Plan cp = ps5lm::plan_model(c, budget);
assert(cp.fits && cp.ctx > 4096 && has_args(cp, "-ub", "64"));
+ // A larger prompt batch costs compute buffer, counted by the feed-forward width.
+ Gguf wide_ff = chat;
+ wide_ff.kv("qwen35.feed_forward_length", 12288u);
+ const ps5lm::ModelInfo w = ps5lm::read_model_header("Qwen3.5-9B-Q4_K_M.gguf", wide_ff.bytes(), 5ull << 30);
+ const ps5lm::Plan w64 = ps5lm::plan_model(w, budget), w512 = ps5lm::plan_model(w, budget, 65536, nullptr, 512);
+ assert(w.n_ff == 12288 && has_args(w64, "-ub", "64") && has_args(w512, "-ub", "512") && has_args(w512, "-b", "512"));
+ assert(w512.reserve_gib - w64.reserve_gib > 0.18 && w512.reserve_gib - w64.reserve_gib < 0.21);
+
+ // Bonsai 2: upstream's Q2_0 loads with its preset (64k at most, PrismML's
+ // sampling); PrismML's own PQ2_0 and PTQ1_0 say what to do instead.
+ const auto bonsai = [](uint32_t ftype) {
+ Gguf g;
+ g.kv("general.architecture", "qwen35").kv("general.name", "Ternary-Bonsai-2-27B")
+ .kv("qwen35.block_count", 64u).kv("qwen35.attention.head_count", 24u)
+ .kv("qwen35.attention.head_count_kv", 4u).kv("qwen35.embedding_length", 5120u)
+ .kv("qwen35.attention.key_length", 256u).kv("qwen35.attention.value_length", 256u)
+ .kv("qwen35.full_attention_interval", 4u).kv("qwen35.context_length", 262144u)
+ .kv("prism.hadamard.version", 1u).kv("general.file_type", ftype);
+ return g.bytes();
+ };
+ const ps5lm::ModelInfo b = ps5lm::read_model_header("Ternary-Bonsai-2-27B-Q2_0.gguf", bonsai(41), 7626008928ull);
+ const ps5lm::Preset * bp = ps5lm::find_preset(b);
+ assert(b.ok && bp && std::string(bp->label) == "Bonsai 2");
+ const ps5lm::Plan bpl = ps5lm::plan_model(b, 12.0, 131072);
+ assert(bpl.fits && bpl.ctx == 65536 && has_args(bpl, "--min-p", "0.05"));
+ const ps5lm::ModelInfo pq = ps5lm::read_model_header("Ternary-Bonsai-2-27B-PQ2_0.gguf", bonsai(141), 7206168928ull);
+ assert(!pq.ok && pq.error.find("bonsai-pq2-to-q2.py") != std::string::npos);
+ const ps5lm::ModelInfo pt = ps5lm::read_model_header("Ternary-Bonsai-2-27B-PTQ1_0.gguf", bonsai(143), 5946648928ull);
+ assert(!pt.ok && pt.error.find("PTQ1_0") != std::string::npos);
+
if (argc < 2) {
std::puts("model_plan_test: ok (synthetic only)");
return 0;
diff --git a/ps5/app/preview_main.cpp b/ps5/app/preview_main.cpp
index 412f2f0..07514f9 100644
--- a/ps5/app/preview_main.cpp
+++ b/ps5/app/preview_main.cpp
@@ -452,9 +452,28 @@ int main(int argc, char ** argv) {
shot("18-settings-context");
step(20, press(A::back), false);
step(20, press(A::back), false);
- for (int i = 0; i < 3; ++i) {
+ for (int i = 0; i < 2; ++i) {
step(15, nav(D::down), false);
}
+ // Sound and storage, on a console where the HEN kept /data closed.
+ dash.set_storage(false, 16, false);
+ step(30, press(A::confirm), false);
+ step(20, nav(D::down), false);
+ step(20, nav(D::down), false);
+ shot("18-settings-app-storage");
+ step(20, press(A::back), false);
+ dash.set_storage(true, 16, false);
+ step(15, nav(D::down), false);
+ step(30, press(A::confirm), false);
+ shot("19-settings-report");
+ step(30, press(A::confirm), false);
+ shot("19-settings-report-dialog");
+ step(20, press(A::back), false);
+ dash.set_report_status(false, "Saved to /data/PS5LM/debug-report.txt and /mnt/usb0/PS5LM-debug-report.txt, 214 KB");
+ step(20, {}, false);
+ shot("19-settings-report-saved");
+ step(20, press(A::back), false);
+ step(15, nav(D::down), false);
step(30, press(A::confirm), false);
shot("19-settings-diagnostics");
step(20, press(A::back), false);
@@ -465,6 +484,14 @@ int main(int argc, char ** argv) {
step(60, {}, true);
shot("20-home-message");
step(30, press(A::back), true);
+ // The first start without full storage.
+ dash.set_storage(false, 16, true);
+ step(30, {}, false);
+ shot("20-storage-closed");
+ step(20, press(A::north), false);
+ shot("20-storage-howto");
+ step(20, press(A::back), false);
+ step(20, press(A::confirm), false);
dash.set_busy("Loading Qwen3.8 27B", "Restarting with the whole memory for it, about 90 seconds");
step(30, {}, false);
shot("21-busy");
diff --git a/ps5/app/probes.cpp b/ps5/app/probes.cpp
new file mode 100644
index 0000000..b5fd4ea
--- /dev/null
+++ b/ps5/app/probes.cpp
@@ -0,0 +1,245 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "probes.hpp"
+
+#include "fetch.hpp"
+#include "storage.hpp"
+
+#include <fcntl.h>
+#include <sys/stat.h>
+#include <sys/statvfs.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <atomic>
+#include <cerrno>
+#include <chrono>
+#include <cstdint>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <thread>
+#include <vector>
+
+#ifdef __PROSPERO__
+#include <ps5platform/kernel.h>
+#endif
+
+namespace ps5lm {
+
+// Probe: download a URL over HTTPS to fetch.out in the data folder, logging the
+// status, size and speed (scripts/ps5lm-app.sh fetch <url>).
+void probe_fetch(std::string url) {
+ FILE * out = std::fopen(data_path("fetch.out").c_str(), "wb");
+ const FetchResult r = https_get(url, "", [&](const char * p, size_t n) { return out && std::fwrite(p, 1, n, out) == n; });
+ if (out) {
+ std::fclose(out);
+ }
+ std::printf("ps5lm-app: fetch %s: status %d, length %lld, %llu bytes in %.1f s (%.2f MB/s)%s%s\n", url.c_str(),
+ r.status, (long long) r.length, (unsigned long long) r.bytes, r.seconds,
+ r.seconds > 0 ? r.bytes / 1e6 / r.seconds : 0.0, r.error.empty() ? "" : ", ", r.error.c_str());
+}
+
+// Probe: how fast the app can write, by method and place, fsync included
+// (scripts/ps5lm-app.sh bench-write). 128 MiB each.
+void probe_write() {
+ const size_t total = 128u << 20;
+ std::vector<char> buf(8u << 20, 'x');
+ for (const std::string & dir : { data_root(), std::string("/mnt/usb0") }) {
+ struct statvfs v = {};
+ if (statvfs(dir.c_str(), &v) != 0) {
+ continue;
+ }
+ const std::string path = std::string(dir) + "/write-bench.tmp";
+ struct Method {
+ const char * name;
+ size_t chunk;
+ bool stdio;
+ int flags;
+ } methods[] = {
+ { "stdio 8M", 8u << 20, true, 0 },
+ { "write 1M", 1u << 20, false, 0 },
+ { "write 8M", 8u << 20, false, 0 },
+ { "write 8M direct", 8u << 20, false, O_DIRECT },
+ };
+ for (const auto & m : methods) {
+ const auto t0 = std::chrono::steady_clock::now();
+ bool ok = true;
+ if (m.stdio) {
+ FILE * f = std::fopen(path.c_str(), "wb");
+ for (size_t done = 0; f && ok && done < total; done += m.chunk) {
+ ok = std::fwrite(buf.data(), 1, m.chunk, f) == m.chunk;
+ }
+ ok = f && ok && std::fflush(f) == 0 && fsync(fileno(f)) == 0;
+ if (f) {
+ std::fclose(f);
+ }
+ } else {
+ const int fd = open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC | m.flags, 0666);
+ for (size_t done = 0; fd >= 0 && ok && done < total; done += m.chunk) {
+ ok = write(fd, buf.data(), m.chunk) == (ssize_t) m.chunk;
+ }
+ ok = fd >= 0 && ok && fsync(fd) == 0;
+ if (fd >= 0) {
+ close(fd);
+ }
+ }
+ const double s = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
+ std::printf("ps5lm-app: write-bench %s %s: %s, %.1f MB/s\n", dir.c_str(), m.name, ok ? "ok" : strerror(errno),
+ ok ? total / 1e6 / s : 0.0);
+ unlink(path.c_str());
+ }
+ }
+ std::printf("ps5lm-app: write-bench done\n");
+}
+
+// The read probe's pieces, on one thread or several: random offsets in the
+// file, aligned to the piece, each read timed.
+namespace {
+
+struct ReadResult {
+ double seconds = 0;
+ uint64_t bytes = 0;
+ std::vector<double> ms; // each read
+ int err = 0;
+};
+
+ReadResult read_random(int fd, uint64_t size, size_t piece, int threads, double seconds) {
+ ReadResult total;
+ std::vector<ReadResult> part((size_t) threads);
+ std::vector<std::thread> pool;
+ const auto t0 = std::chrono::steady_clock::now();
+ for (int t = 0; t < threads; ++t) {
+ pool.emplace_back([&, t] {
+ void * buf = nullptr;
+ if (posix_memalign(&buf, 1u << 16, piece) != 0) {
+ part[(size_t) t].err = ENOMEM;
+ return;
+ }
+ uint64_t x = 0x9e3779b97f4a7c15ull * (uint64_t) (t + 1);
+ const uint64_t slots = size / piece;
+ while (std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count() < seconds) {
+ x ^= x << 13, x ^= x >> 7, x ^= x << 17;
+ const auto r0 = std::chrono::steady_clock::now();
+ const ssize_t got = pread(fd, buf, piece, (off_t) ((x % slots) * piece));
+ if (got != (ssize_t) piece) {
+ part[(size_t) t].err = got < 0 ? errno : EIO;
+ break;
+ }
+ part[(size_t) t].ms.push_back(std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - r0).count());
+ part[(size_t) t].bytes += piece;
+ }
+ std::free(buf);
+ });
+ }
+ for (auto & th : pool) {
+ th.join();
+ }
+ total.seconds = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
+ for (auto & p : part) {
+ total.bytes += p.bytes;
+ total.ms.insert(total.ms.end(), p.ms.begin(), p.ms.end());
+ total.err = total.err ? total.err : p.err;
+ }
+ std::sort(total.ms.begin(), total.ms.end());
+ return total;
+}
+
+} // namespace
+
+void probe_read(std::string path) {
+ struct stat st = {};
+ if (stat(path.c_str(), &st) != 0 || st.st_size < (256ll << 20)) {
+ std::printf("ps5lm-app: read-bench %s: %s\n", path.c_str(),
+ st.st_size ? "under 256 MiB; give it a model of several GiB" : strerror(errno));
+ return;
+ }
+ const uint64_t size = (uint64_t) st.st_size;
+ std::printf("ps5lm-app: read-bench %s, %.2f GiB (%s)\n", path.c_str(), size / 1073741824.0, drive_name(path).c_str());
+ // Sequentially, 8 MiB at a time, up to 2 GiB from the start.
+ {
+ const int fd = open(path.c_str(), O_RDONLY);
+ std::vector<char> buf(8u << 20);
+ const uint64_t want = std::min<uint64_t>(size, 2ull << 30);
+ uint64_t done = 0;
+ const auto t0 = std::chrono::steady_clock::now();
+ while (fd >= 0 && done < want) {
+ const ssize_t got = read(fd, buf.data(), buf.size());
+ if (got <= 0) {
+ break;
+ }
+ done += (uint64_t) got;
+ }
+ const double s = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
+ std::printf("ps5lm-app: read-bench sequential 8 MiB: %.2f GiB in %.1f s, %.0f MB/s\n", done / 1073741824.0, s,
+ s > 0 ? done / 1e6 / s : 0.0);
+ if (fd >= 0) {
+ close(fd);
+ }
+ }
+ // At random, as experts would be fetched: a few seconds for each piece size,
+ // one thread and four, through the cache and then around it.
+ for (const bool direct : { false, true }) {
+ const int fd = open(path.c_str(), O_RDONLY | (direct ? O_DIRECT : 0));
+ if (fd < 0) {
+ std::printf("ps5lm-app: read-bench %s: cannot open (%s)\n", direct ? "O_DIRECT" : "cached", strerror(errno));
+ continue;
+ }
+ for (const size_t piece : { (size_t) 64 << 10, (size_t) 256 << 10, (size_t) 1 << 20, (size_t) 4 << 20 }) {
+ for (const int threads : { 1, 4 }) {
+ const ReadResult r = read_random(fd, size, piece, threads, 3.0);
+ const size_t n = r.ms.size();
+ std::printf("ps5lm-app: read-bench random %s %4zu KiB x%d: %.0f MB/s, %.0f reads/s, %.2f ms median, %.2f ms p99%s%s\n",
+ direct ? "O_DIRECT" : "cached ", piece >> 10, threads, r.seconds > 0 ? r.bytes / 1e6 / r.seconds : 0.0,
+ r.seconds > 0 ? n / r.seconds : 0.0, n ? r.ms[n / 2] : 0.0, n ? r.ms[std::min(n - 1, n * 99 / 100)] : 0.0,
+ r.err ? ", stopped: " : "", r.err ? strerror(r.err) : "");
+ }
+ }
+ close(fd);
+ }
+ std::printf("ps5lm-app: read-bench done\n");
+}
+
+void probe_memory(bool model_loaded) {
+#ifdef __PROSPERO__
+ const int64_t pool = sceKernelGetDirectMemorySize();
+ int64_t start = 0;
+ size_t largest = 0, flex = 0, flex_free = 0;
+ sceKernelAvailableDirectMemorySize(0, pool, 2u << 20, &start, &largest);
+ sceKernelConfiguredFlexibleMemorySize(&flex);
+ sceKernelAvailableFlexibleMemorySize(&flex_free);
+ std::printf("ps5lm-app: memory-probe: direct pool %.4f GiB (the system keeps %.4f GiB of a nominal 12), largest free "
+ "block %.2f GiB at 0x%llx; flexible %zu MiB, %zu MiB free\n",
+ pool / 1073741824.0, 12.0 - pool / 1073741824.0, largest / 1073741824.0, (unsigned long long) start,
+ flex >> 20, flex_free >> 20);
+ if (model_loaded) {
+ std::printf("ps5lm-app: memory-probe: a model is loaded; not allocating (unload it to measure the usable block)\n");
+ return;
+ }
+ // The largest block that really allocates and maps, in 64 MiB steps down
+ // from the largest free one, given back at once.
+ constexpr size_t kStep = 64u << 20;
+ for (size_t len = largest / kStep * kStep; len >= kStep; len -= kStep) {
+ int64_t phys = 0;
+ if (sceKernelAllocateDirectMemory(0, pool, len, 2u << 20, PS5_KERNEL_DIRECT_TYPE_CPU, &phys) != 0) {
+ continue;
+ }
+ void * addr = nullptr;
+ const int mapped = sceKernelMapDirectMemory(&addr, len, PS5_KERNEL_PROT_CPU_READ | PS5_KERNEL_PROT_CPU_WRITE, 0,
+ phys, 2u << 20);
+ if (mapped == 0) {
+ static_cast<volatile char *>(addr)[0] = 1; // touched, at both ends
+ static_cast<volatile char *>(addr)[len - 1] = 1;
+ sceKernelMunmap(addr, len);
+ }
+ sceKernelReleaseDirectMemory(phys, len);
+ std::printf("ps5lm-app: memory-probe: allocated %.2f GiB in one block%s\n", len / 1073741824.0,
+ mapped == 0 ? ", mapped and touched" : ", but mapping it failed");
+ return;
+ }
+ std::printf("ps5lm-app: memory-probe: no 64 MiB block allocates\n");
+#else
+ (void) model_loaded;
+#endif
+}
+
+} // namespace ps5lm
diff --git a/ps5/app/probes.hpp b/ps5/app/probes.hpp
new file mode 100644
index 0000000..7f847da
--- /dev/null
+++ b/ps5/app/probes.hpp
@@ -0,0 +1,30 @@
+// Measurements the app makes only when the PC asks (scripts/ps5lm-app.sh),
+// each on a thread of its own, with its results in app.log. None of them is
+// part of serving a model.
+//
+// SPDX-License-Identifier: GPL-3.0-or-later
+#pragma once
+
+#include <string>
+
+namespace ps5lm {
+
+// Downloads a URL over HTTPS to fetch.out in the data folder: status, size, speed.
+void probe_fetch(std::string url);
+
+// Write speed by method (stdio, write, O_DIRECT) to the data folder and a USB drive.
+void probe_write();
+
+// Read speed of a file, the way streaming MoE experts from a drive would read
+// it: sequentially, then at random offsets in pieces of 64 KiB to 4 MiB with
+// one and four threads, through the cache and with O_DIRECT. Point it at a
+// model of several GiB, so the cache can hold little of it.
+void probe_read(std::string path);
+
+// The title's memory as the system gave it: the direct pool and its largest
+// free block, the flexible pool, and the largest block the app can allocate
+// and map now, given back at once. `model_loaded`: skip that last part, so it
+// never takes memory from a running model.
+void probe_memory(bool model_loaded);
+
+} // namespace ps5lm
diff --git a/ps5/app/report.cpp b/ps5/app/report.cpp
new file mode 100644
index 0000000..4b36549
--- /dev/null
+++ b/ps5/app/report.cpp
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "report.hpp"
+
+#include <algorithm>
+#include <cctype>
+#include <cstring>
+
+namespace ps5lm {
+
+namespace {
+
+constexpr const char * kRemoved = "[removed]";
+
+std::string lower(std::string s) {
+ for (auto & c : s) {
+ c = (char) std::tolower((unsigned char) c);
+ }
+ return s;
+}
+
+bool ends_with(const std::string & s, const char * tail) {
+ const size_t n = std::strlen(tail);
+ return s.size() >= n && s.compare(s.size() - n, n, tail) == 0;
+}
+
+bool name_char(char c) { return std::isalnum((unsigned char) c) || c == '_' || c == '-' || c == '.'; }
+
+// llama-server's options whose value is a secret.
+bool secret_flag(const std::string & word) {
+ static const char * flags[] = { "--api-key", "--api-key-file", "--hf-token", "-hft", "--ssl-key-file" };
+ for (const char * f : flags) {
+ if (word == f) {
+ return true;
+ }
+ }
+ return false;
+}
+
+// A key whose value is a secret: "api_key", "hf_token", "client_secret",
+// "password". Not "max_tokens", "n_tokens" or "eos_token_id".
+bool secret_key(std::string key) {
+ key = lower(key);
+ const size_t dot = key.find_last_of('.');
+ if (dot != std::string::npos) {
+ key = key.substr(dot + 1); // "env.HF_TOKEN", "x.api_key"
+ }
+ static const char * exact[] = { "token", "apikey", "api_key", "api-key", "x-api-key", "authorization", "secret",
+ "password", "passwd", "bearer" };
+ for (const char * e : exact) {
+ if (key == e) {
+ return true;
+ }
+ }
+ static const char * tails[] = { "_api_key", "-api-key", "_apikey", "_secret", "-secret", "_password", "-password",
+ "access_token", "auth_token", "hf_token", "api_token", "refresh_token", "id_token",
+ "bearer_token", "github_token", "-token" };
+ for (const char * t : tails) {
+ if (ends_with(key, t)) {
+ return true;
+ }
+ }
+ return false;
+}
+
+// Where a value that starts at `at` ends: a quote closes a quoted one, else
+// space or punctuation does.
+size_t value_end(const std::string & s, size_t at) {
+ if (at < s.size() && (s[at] == '"' || s[at] == '\'')) {
+ const char q = s[at];
+ const size_t close = s.find(q, at + 1);
+ return close == std::string::npos ? s.size() : close + 1;
+ }
+ size_t e = at;
+ while (e < s.size() && !std::isspace((unsigned char) s[e]) && s[e] != ',' && s[e] != ';' && s[e] != '&' && s[e] != '}' &&
+ s[e] != ')') {
+ ++e;
+ }
+ return e;
+}
+
+// Replaces s[from, to) with [removed], keeping the quotes of a quoted value.
+void blank(std::string & s, size_t from, size_t to) {
+ if (to <= from) {
+ return;
+ }
+ if (to - from >= 2 && (s[from] == '"' || s[from] == '\'') && s[to - 1] == s[from]) {
+ ++from;
+ --to;
+ }
+ s.replace(from, to - from, kRemoved);
+}
+
+// Strings shaped like an access token, whatever key they are under.
+void redact_shapes(std::string & s) {
+ struct Shape {
+ const char * prefix;
+ size_t min_tail;
+ };
+ static const Shape shapes[] = { { "hf_", 30 }, { "sk-", 20 }, { "ghp_", 30 }, { "gho_", 30 }, { "ghs_", 30 },
+ { "github_pat_", 30 }, { "xoxb-", 20 }, { "xoxp-", 20 } };
+ for (const auto & sh : shapes) {
+ const size_t plen = std::strlen(sh.prefix);
+ for (size_t at = s.find(sh.prefix); at != std::string::npos; at = s.find(sh.prefix, at + 1)) {
+ if (at > 0 && (std::isalnum((unsigned char) s[at - 1]) || s[at - 1] == '_')) {
+ continue; // inside a word: "mask_hf_x"
+ }
+ size_t e = at + plen;
+ while (e < s.size() && (std::isalnum((unsigned char) s[e]) || s[e] == '_' || s[e] == '-')) {
+ ++e;
+ }
+ if (e - at - plen >= sh.min_tail) {
+ s.replace(at, e - at, kRemoved);
+ }
+ }
+ }
+}
+
+// One line: flags and their values, "Header: value", key=value and
+// "key": "value".
+std::string redact_line(std::string s, bool & next_is_secret) {
+ // The value of a flag on the line before ("ps5lm-app: arg --api-key", then
+ // "ps5lm-app: arg <key>"), or app-args.txt's one argument a line.
+ if (next_is_secret) {
+ next_is_secret = false;
+ size_t end = s.size();
+ while (end > 0 && std::isspace((unsigned char) s[end - 1])) {
+ --end;
+ }
+ size_t start = end;
+ while (start > 0 && !std::isspace((unsigned char) s[start - 1])) {
+ --start;
+ }
+ if (end > start) {
+ s.replace(start, end - start, kRemoved);
+ }
+ }
+ // Words: "--api-key X", "--api-key=X", and a flag ending the line.
+ for (size_t at = 0; at < s.size();) {
+ while (at < s.size() && std::isspace((unsigned char) s[at])) {
+ ++at;
+ }
+ size_t e = at;
+ while (e < s.size() && !std::isspace((unsigned char) s[e])) {
+ ++e;
+ }
+ std::string word = s.substr(at, e - at);
+ const size_t eq = word.find('=');
+ if (eq != std::string::npos && secret_flag(word.substr(0, eq))) {
+ blank(s, at + eq + 1, e);
+ e = at + eq + 1 + std::strlen(kRemoved);
+ } else if (!word.empty() && secret_flag(word)) {
+ size_t v = e;
+ while (v < s.size() && std::isspace((unsigned char) s[v])) {
+ ++v;
+ }
+ if (v < s.size()) {
+ const size_t ve = value_end(s, v);
+ blank(s, v, ve);
+ e = v + std::strlen(kRemoved);
+ } else {
+ next_is_secret = true;
+ }
+ }
+ at = e;
+ }
+ // "name: value", "name=value", "name": "value", for secret names.
+ for (size_t at = 0; at < s.size(); ++at) {
+ if (!name_char(s[at]) || (at > 0 && name_char(s[at - 1]))) {
+ continue;
+ }
+ size_t e = at;
+ while (e < s.size() && name_char(s[e])) {
+ ++e;
+ }
+ const std::string name = s.substr(at, e - at);
+ size_t p = e;
+ if (p < s.size() && (s[p] == '"' || s[p] == '\'')) {
+ ++p; // the end of a quoted key
+ }
+ // The separator right after the name: "token=", "token:", "token":,
+ // not llama.cpp's "EOS token = 248046".
+ if (p >= s.size() || (s[p] != ':' && s[p] != '=') || !secret_key(name)) {
+ at = e - 1;
+ continue;
+ }
+ const bool header = s[p] == ':' && (lower(name) == "authorization" || ends_with(lower(name), "api-key"));
+ ++p;
+ while (p < s.size() && s[p] == ' ') {
+ ++p;
+ }
+ if (header) {
+ blank(s, p, s.size()); // "Authorization: Bearer <token>": all of it
+ break;
+ }
+ const size_t ve = value_end(s, p);
+ if (ve > p) {
+ blank(s, p, ve);
+ }
+ at = p;
+ }
+ redact_shapes(s);
+ return s;
+}
+
+void add_lines(std::string & out, const std::vector<std::string> & lines) {
+ for (const auto & l : lines) {
+ out += "- " + l + "\n";
+ }
+ if (lines.empty()) {
+ out += "- none\n";
+ }
+}
+
+void add_block(std::string & out, const std::string & title, const std::string & text) {
+ out += "\n## " + title + "\n\n```text\n" + text;
+ if (!text.empty() && text.back() != '\n') {
+ out += "\n";
+ }
+ out += "```\n";
+}
+
+} // namespace
+
+const std::vector<std::pair<int, std::string>> & report_ports() {
+ static const std::vector<std::pair<int, std::string>> ports = {
+ { 9021, "ELF loader (elfldr, etaHEN)" }, { 9020, "ELF loader (older HENs)" }, { 2121, "ftpsrv" },
+ { 1337, "etaHEN FTP" }, { 3232, "klogsrv" }, { 9081, "etaHEN kernel log" },
+ { 2323, "shsrv" }, { 8080, "websrv" }, { 9100, "ps5-exporter" },
+ { 12800, "Payload Manager" }, { 8081, "PS5LM's server" },
+ };
+ return ports;
+}
+
+std::string redact_secrets(const std::string & text) {
+ std::string out;
+ out.reserve(text.size());
+ bool next = false;
+ size_t at = 0;
+ while (at < text.size()) {
+ size_t nl = text.find('\n', at);
+ const bool has_nl = nl != std::string::npos;
+ if (!has_nl) {
+ nl = text.size();
+ }
+ out += redact_line(text.substr(at, nl - at), next);
+ if (has_nl) {
+ out += '\n';
+ }
+ at = nl + 1;
+ }
+ return out;
+}
+
+std::string last_bytes(const std::string & text, size_t max_bytes) {
+ if (text.size() <= max_bytes) {
+ return text;
+ }
+ const size_t from = text.size() - max_bytes;
+ const size_t nl = text.find('\n', from);
+ return nl == std::string::npos ? text.substr(from) : text.substr(nl + 1);
+}
+
+std::string build_report(const ReportInput & in) {
+ std::string out = "# PS5LM debug report\n\n";
+ out += "Attach this file to a GitHub issue. API keys and tokens are removed from it.\n\n";
+ out += "- Created: " + in.created + " (the console's clock)\n";
+ out += "- PS5LM: " + in.version + "\n";
+ out += "- System software: " + (in.firmware.empty() ? std::string("unknown") : in.firmware) + "\n";
+ out += "- Storage: " + in.storage + "\n";
+
+ out += "\n## Drives\n\n";
+ add_lines(out, in.drives);
+ out += "\n## Memory\n\n";
+ add_lines(out, in.memory);
+ out += "\n## Server\n\n";
+ add_lines(out, in.server);
+ if (!in.args.empty()) {
+ std::string args;
+ for (const auto & a : in.args) {
+ args += a + "\n";
+ }
+ add_block(out, "llama-server arguments", redact_secrets(args));
+ }
+
+ out += "\n## Payload ports\n\n";
+ for (const auto & [port, what] : in.ports) {
+ const bool open = std::find(in.open_ports.begin(), in.open_ports.end(), port) != in.open_ports.end();
+ out += "- " + std::to_string(port) + " (" + what + "): " + (open ? "open" : "closed") + "\n";
+ }
+
+ out += "\n## Models\n\n";
+ add_lines(out, in.models);
+ out += "\n## Downloads\n\n";
+ add_lines(out, in.downloads);
+
+ add_block(out, "settings.json", redact_secrets(in.settings_json));
+ for (const auto & [name, text] : in.files) {
+ add_block(out, name, redact_secrets(text));
+ }
+ for (const auto & [name, text] : in.logs) {
+ add_block(out, name, redact_secrets(text));
+ }
+ if (in.kernel_log_asked) {
+ add_block(out, "Kernel log", redact_secrets(in.kernel_log));
+ } else {
+ out += "\n## Kernel log\n\nNot included (Settings > Debug report > Include the kernel log).\n";
+ }
+ return out;
+}
+
+} // namespace ps5lm
diff --git a/ps5/app/report.hpp b/ps5/app/report.hpp
new file mode 100644
index 0000000..8232cc0
--- /dev/null
+++ b/ps5/app/report.hpp
@@ -0,0 +1,56 @@
+// The debug report: one text file to attach to a GitHub issue, made from
+// Settings > Debug report. It holds what a bug report needs and nothing
+// secret: the app's version, the console's system software, where the app
+// keeps its files and the room on each drive, the models and their plans, the
+// settings, the last part of both logs, which payload ports answer, and the
+// kernel log only when the user switched it on. API keys and tokens are
+// removed from everything that goes in (redact_secrets).
+//
+// SPDX-License-Identifier: GPL-3.0-or-later
+#pragma once
+
+#include <cstddef>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace ps5lm {
+
+// What the report says, gathered by the app (main.cpp); building it is plain
+// text work, checked on the PC by report_test.cpp.
+struct ReportInput {
+ std::string created; // "2026-10-23 14:32", the console's clock
+ std::string version; // the app's
+ std::string firmware; // "13.60" and the raw word, or empty
+ std::string storage; // where settings, logs and models live, and why
+ std::vector<std::string> drives; // one line each
+ std::vector<std::string> memory; // the pools and ggml's devices
+ std::vector<std::string> server; // state, address, model and plan
+ std::vector<std::string> args; // llama-server's arguments
+ std::vector<std::pair<int, std::string>> ports; // port, what usually listens there
+ std::vector<int> open_ports;
+ std::vector<std::string> models; // one line each: file, drive, size, plan
+ std::vector<std::string> downloads; // one line each
+ std::string settings_json;
+ std::vector<std::pair<std::string, std::string>> files; // other small files: name, text (env.txt, app-args.txt)
+ std::vector<std::pair<std::string, std::string>> logs; // name, its last part
+ bool kernel_log_asked = false;
+ std::string kernel_log; // when asked: the last part of it, or why there is none
+};
+
+// Payload ports worth knowing about, and what usually listens on each.
+const std::vector<std::pair<int, std::string>> & report_ports();
+
+// The text with API keys, tokens and passwords replaced by [removed]: the
+// value after --api-key and its kin, Authorization and API-key headers,
+// key=value and "key": "value" pairs whose name says key, token, secret or
+// password, and strings shaped like Hugging Face (hf_...), OpenAI (sk-...)
+// or GitHub (ghp_...) tokens.
+std::string redact_secrets(const std::string & text);
+
+// The last `max_bytes` of a text, from the start of a line.
+std::string last_bytes(const std::string & text, size_t max_bytes);
+
+std::string build_report(const ReportInput & in);
+
+} // namespace ps5lm
diff --git a/ps5/app/report_test.cpp b/ps5/app/report_test.cpp
new file mode 100644
index 0000000..1986af9
--- /dev/null
+++ b/ps5/app/report_test.cpp
@@ -0,0 +1,103 @@
+// Host check for report.cpp:
+// c++ -std=c++17 -O1 ps5/app/report_test.cpp ps5/app/report.cpp -o build/report_test && build/report_test
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "report.hpp"
+
+#include <cstdio>
+#include <cstdlib>
+#include <string>
+
+namespace {
+
+int g_failed = 0;
+
+void expect(const std::string & in, const std::string & want) {
+ const std::string got = ps5lm::redact_secrets(in);
+ if (got != want) {
+ std::printf("FAIL\n in: %s\n got: %s\n want: %s\n", in.c_str(), got.c_str(), want.c_str());
+ ++g_failed;
+ }
+}
+
+void expect_kept(const std::string & in) { expect(in, in); }
+
+} // namespace
+
+int main() {
+ using ps5lm::build_report;
+ using ps5lm::last_bytes;
+
+ // Flags with a secret value, in every form the app writes them.
+ expect("--api-key sk-local-123", "--api-key [removed]");
+ expect("-m model.gguf --api-key=abc --port 8081", "-m model.gguf --api-key=[removed] --port 8081");
+ expect("ps5lm-app: arg --api-key\nps5lm-app: arg hunter2\nps5lm-app: arg --port",
+ "ps5lm-app: arg --api-key\nps5lm-app: arg [removed]\nps5lm-app: arg --port");
+ expect("--hf-token\nhf_abcdefghijklmnopqrstuvwxyz0123456789\n-m\nx.gguf", "--hf-token\n[removed]\n-m\nx.gguf");
+ expect("--api-key-file /data/PS5LM/key.txt", "--api-key-file [removed]");
+
+ // Headers and key-value pairs.
+ expect("Authorization: Bearer abc.def.ghi", "Authorization: [removed]");
+ expect("x-api-key: 12345", "x-api-key: [removed]");
+ expect("{\"api_key\": \"s3cret\", \"model\": \"ps5\"}", "{\"api_key\": \"[removed]\", \"model\": \"ps5\"}");
+ expect("HF_TOKEN=hf_x password=pw1 user=me", "HF_TOKEN=[removed] password=[removed] user=me");
+ expect("\"client_secret\":\"abc\"", "\"client_secret\":\"[removed]\"");
+
+ // Token shapes anywhere.
+ expect("got hf_ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456 from env", "got [removed] from env");
+ expect("key sk-ant-api03-ABCDEFGHIJKLMNOPQRSTUV end", "key [removed] end");
+ expect("ghp_0123456789abcdefghijklmnopqrstuvwxyz", "[removed]");
+
+ // What llama.cpp and the app log every day stays as it is.
+ expect_kept("slot update_slots: id 0 | task 3 | n_tokens = 512, memory_seq_rm [512, end)");
+ expect_kept("srv log_server_r: request: POST /v1/chat/completions 192.168.8.12 200");
+ expect_kept("{\"max_tokens\": 512, \"stream\": true, \"temperature\": 0.7}");
+ expect_kept("print_info: EOS token = 248046 '<|im_end|>'");
+ expect_kept("llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 248046");
+ expect_kept("ps5lm-app: arg -ctk\nps5lm-app: arg q8_0");
+ expect_kept("prompt_tokens_total 9120\nllamacpp:tokens_predicted_total 4410");
+ expect_kept("mask_hf_ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456");
+ expect_kept("load_tensors: Vulkan0 model buffer size = 9369.58 MiB");
+
+ // The last part of a log starts at a line.
+ if (last_bytes("one\ntwo\nthree\n", 8) != "three\n" || last_bytes("short", 100) != "short") {
+ std::printf("FAIL last_bytes\n");
+ ++g_failed;
+ }
+
+ // The report itself: every section, secrets out of the logs and settings,
+ // the kernel log only when asked.
+ ps5lm::ReportInput in;
+ in.created = "2026-10-23 14:32";
+ in.version = "0.9.2";
+ in.firmware = "13.60 (0x13600000)";
+ in.storage = "/data/PS5LM, full storage";
+ in.drives = { "Internal storage: 120.4 of 825.0 GiB free, models in /data/PS5LM/models" };
+ in.args = { "-m", "/data/PS5LM/models/x.gguf", "--api-key", "topsecret" };
+ in.ports = ps5lm::report_ports();
+ in.open_ports = { 9021, 8081 };
+ in.settings_json = "{\"ctx_cap\": 65536}";
+ in.logs = { { "app.log (last 256 KiB)", "ps5lm-app: arg --api-key\nps5lm-app: arg topsecret\n" } };
+ const std::string r = build_report(in);
+ const char * must[] = { "# PS5LM debug report", "System software: 13.60", "## Drives", "## Payload ports",
+ "- 9021 (ELF loader (elfldr, etaHEN)): open", "- 2121 (ftpsrv): closed", "## Models\n\n- none",
+ "## settings.json", "app.log (last 256 KiB)", "Not included" };
+ for (const char * m : must) {
+ if (r.find(m) == std::string::npos) {
+ std::printf("FAIL report lacks: %s\n", m);
+ ++g_failed;
+ }
+ }
+ if (r.find("topsecret") != std::string::npos) {
+ std::printf("FAIL report holds the API key\n%s\n", r.c_str());
+ ++g_failed;
+ }
+ in.kernel_log_asked = true;
+ in.kernel_log = "[KERNEL] eth0: link state changed to UP\n";
+ if (build_report(in).find("## Kernel log\n\n```text\n[KERNEL] eth0") == std::string::npos) {
+ std::printf("FAIL kernel log missing when asked\n");
+ ++g_failed;
+ }
+
+ std::printf(g_failed ? "report_test: %d failed\n" : "report_test: all passed\n", g_failed);
+ return g_failed ? EXIT_FAILURE : EXIT_SUCCESS;
+}
diff --git a/ps5/app/sce_sys/param.json b/ps5/app/sce_sys/param.json
index 14a380d..4109eb8 100644
--- a/ps5/app/sce_sys/param.json
+++ b/ps5/app/sce_sys/param.json
@@ -2,11 +2,6 @@
"ageLevel": {
"default": 0
},
- "amm": {
- "multimapVaRangeInGib": 512,
- "pagetableMemorySizeInMib": 256,
- "vaRangeInGib": 512
- },
"applicationCategoryType": 0,
"applicationDrmType": "free",
"attribute": 0,
@@ -16,7 +11,7 @@
"contentBadgeType": 1,
"contentId": "UP9000-PPSA99575_00-PS5LMVULKAN00001",
"contentVersion": "01.010.000",
- "downloadDataSize": 256,
+ "downloadDataSize": 16384,
"gameIntent": {
"permittedIntents": [
{
@@ -24,10 +19,6 @@
}
]
},
- "kernel": {
- "cpuPageTableSize": 268435456,
- "gpuPageTableSize": 268435456
- },
"localizedParameters": {
"defaultLanguage": "en-US",
"en-US": {
diff --git a/ps5/app/settings.cpp b/ps5/app/settings.cpp
index 4d14c8d..56026eb 100644
--- a/ps5/app/settings.cpp
+++ b/ps5/app/settings.cpp
@@ -26,6 +26,8 @@ Settings load_settings(const char * path) {
s.introspect = j.value("introspect", s.introspect);
s.scratch_limit_gib = j.value("scratch_limit_gib", s.scratch_limit_gib);
s.download_dir = j.value("download_dir", s.download_dir);
+ s.report_kernel_log = j.value("report_kernel_log", s.report_kernel_log);
+ s.report_explained = j.value("report_explained", s.report_explained);
return s;
}
@@ -35,6 +37,7 @@ bool save_settings(const char * path, const Settings & s) {
{ "kv_type", s.kv_type }, { "sounds", s.sounds }, { "tools", s.tools },
{ "introspect", s.introspect },
{ "scratch_limit_gib", s.scratch_limit_gib }, { "download_dir", s.download_dir },
+ { "report_kernel_log", s.report_kernel_log }, { "report_explained", s.report_explained },
};
std::ofstream out(path);
out << j.dump(2) << "\n";
diff --git a/ps5/app/settings.hpp b/ps5/app/settings.hpp
index 2f7be58..5875e5c 100644
--- a/ps5/app/settings.hpp
+++ b/ps5/app/settings.hpp
@@ -20,6 +20,8 @@ struct Settings {
bool introspect = false; // POST /v1/introspect: raw logits and activations (off by default)
uint32_t scratch_limit_gib = 2; // warn when the scratch folder grows past this
std::string download_dir = "/data/PS5LM/models"; // where the market saves models
+ bool report_kernel_log = false; // the debug report includes the kernel log
+ bool report_explained = false; // what a debug report holds was shown before the first
bool operator==(const Settings &) const = default;
};
diff --git a/ps5/app/stats.cpp b/ps5/app/stats.cpp
index 0c62387..eeb2add 100644
--- a/ps5/app/stats.cpp
+++ b/ps5/app/stats.cpp
@@ -82,14 +82,15 @@ std::vector<Drive> drives_from_metrics(const std::string & text) {
out.push_back({ drive_name(mount), mount + "/PS5LM/models", free, total });
}
}
+ // Without full storage the app's own storage stands in for internal storage.
+ if (app_storage()) {
+ out[0] = { "App storage", std::string(kAppRoot) + "/models", app_storage_free_gib(), app_storage_size() / kGiB };
+ }
return out;
}
std::vector<Drive> mounted_drives() { return drives_from_metrics(http_get(9100, "/metrics", 2)); }
-std::string drive_name(const std::string & path) {
- return path.rfind("/mnt/ext", 0) == 0 ? "M.2 drive" : path.rfind("/mnt/usb", 0) == 0 ? "USB drive" : "Internal storage";
-}
namespace {
diff --git a/ps5/app/stats.hpp b/ps5/app/stats.hpp
index 57ceb2d..6b95ed8 100644
--- a/ps5/app/stats.hpp
+++ b/ps5/app/stats.hpp
@@ -10,6 +10,8 @@
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
+#include "storage.hpp"
+
#include <cstdint>
#include <mutex>
#include <string>
@@ -154,8 +156,6 @@ class StatsCollector {
std::vector<Drive> mounted_drives();
std::vector<Drive> drives_from_metrics(const std::string & text);
-// The kind of drive holding `path`: "Internal storage", "M.2 drive", "USB drive".
-std::string drive_name(const std::string & path);
bool send_payload(const char * path, int port);
// Where llama-server's request hooks (activity.cpp) report.
diff --git a/ps5/app/storage.cpp b/ps5/app/storage.cpp
new file mode 100644
index 0000000..c1ad08d
--- /dev/null
+++ b/ps5/app/storage.cpp
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "storage.hpp"
+
+#include <algorithm>
+#include <cstdlib>
+#include <filesystem>
+#include <fstream>
+#include <sstream>
+
+namespace ps5lm {
+
+namespace {
+
+std::string g_root = kFullRoot;
+
+constexpr const char * kParam = "/app0/sce_sys/param.json";
+constexpr const char * kChoice = "/download0/PS5LM/storage-choice";
+constexpr uint64_t kDefaultSize = 16ull << 30;
+
+} // namespace
+
+void set_data_root(const std::string & root) { g_root = root; }
+
+const std::string & data_root() { return g_root; }
+
+bool app_storage() { return g_root == kAppRoot; }
+
+std::string data_path(const std::string & name) { return g_root + "/" + name; }
+
+uint64_t folder_bytes(const std::string & path) {
+ std::error_code ec;
+ uint64_t total = 0;
+ for (auto it = std::filesystem::recursive_directory_iterator(path, ec);
+ !ec && it != std::filesystem::recursive_directory_iterator(); it.increment(ec)) {
+ if (it->is_regular_file(ec)) {
+ total += it->file_size(ec);
+ }
+ }
+ return total;
+}
+
+uint64_t app_storage_size() {
+ // "downloadDataSize": 16384, in MiB.
+ static const uint64_t size = [] {
+ std::stringstream text;
+ text << std::ifstream(kParam).rdbuf();
+ const std::string json = text.str();
+ const size_t key = json.find("\"downloadDataSize\"");
+ const size_t colon = key == std::string::npos ? key : json.find(':', key);
+ const uint64_t mib = colon == std::string::npos ? 0 : std::strtoull(json.c_str() + colon + 1, nullptr, 10);
+ return mib ? mib << 20 : kDefaultSize;
+ }();
+ return size;
+}
+
+// Everything in /download0 counts, the HEN's request file and the app's notes included.
+uint64_t app_storage_used() { return folder_bytes("/download0"); }
+
+double app_storage_free_gib() {
+ const uint64_t size = app_storage_size(), used = app_storage_used();
+ return (double) (size > used ? size - used : 0) / 1073741824.0;
+}
+
+std::string drive_name(const std::string & path) {
+ return path.rfind("/mnt/ext", 0) == 0 ? "M.2 drive" : path.rfind("/mnt/usb", 0) == 0 ? "USB drive"
+ : path.rfind("/download0", 0) == 0 ? "App storage" : "Internal storage";
+}
+
+std::string storage_choice() {
+ std::string choice;
+ std::ifstream(kChoice) >> choice;
+ return choice;
+}
+
+void set_storage_choice(const std::string & choice) {
+ std::error_code ec;
+ std::filesystem::create_directories(kAppRoot, ec);
+ std::ofstream(kChoice) << choice << "\n";
+}
+
+} // namespace ps5lm
diff --git a/ps5/app/storage.hpp b/ps5/app/storage.hpp
new file mode 100644
index 0000000..714f73e
--- /dev/null
+++ b/ps5/app/storage.hpp
@@ -0,0 +1,47 @@
+// Where PS5LM keeps its files. With full storage the HEN opened /data to the
+// app and everything lives in /data/PS5LM, as it always has. Without it the app
+// uses its own storage, /download0/PS5LM: the system gives every title a
+// /download0 sized by downloadDataSize in sce_sys/param.json (16 GiB for this
+// app), and the app keeps its settings, logs and models there instead.
+//
+// A title's statvfs is a stub, so the room left in the app's storage is
+// counted here: its size less what the files in it take. (Whether the system
+// sets the whole size aside on internal storage up front is not known; a
+// download that meets a full drive anyway stops with that reason.)
+//
+// SPDX-License-Identifier: GPL-3.0-or-later
+#pragma once
+
+#include <cstdint>
+#include <string>
+
+namespace ps5lm {
+
+constexpr const char * kFullRoot = "/data/PS5LM";
+constexpr const char * kAppRoot = "/download0/PS5LM";
+
+// The folder in use, chosen once at start (main.cpp) before any thread reads it.
+void set_data_root(const std::string & root);
+const std::string & data_root();
+bool app_storage(); // the root is the app's own storage
+std::string data_path(const std::string & name); // data_root() + "/" + name
+
+// The app's own storage: its size from sce_sys/param.json (16 GiB when the file
+// cannot be read), the bytes its files take, and what is left.
+uint64_t app_storage_size();
+uint64_t app_storage_used();
+double app_storage_free_gib();
+
+// The choice made on the first start without full storage: "app" once the
+// user chose the app's storage, empty while they have not.
+std::string storage_choice();
+void set_storage_choice(const std::string & choice);
+
+// Bytes in the files under a folder, or 0.
+uint64_t folder_bytes(const std::string & path);
+
+// The kind of drive holding `path`: "Internal storage", "App storage",
+// "M.2 drive", "USB drive".
+std::string drive_name(const std::string & path);
+
+} // namespace ps5lm
diff --git a/ps5/app/suggested.hpp b/ps5/app/suggested.hpp
index 6f947b8..934cd4a 100644
--- a/ps5/app/suggested.hpp
+++ b/ps5/app/suggested.hpp
@@ -32,6 +32,16 @@ inline constexpr Suggestion kSuggestions[] = {
{ "Qwen 3.8 27B", false, "The strongest model that fits, with a 64k context", 20, "unsloth/Qwen3.8-27B-GGUF",
{ "Qwen3.8-27B-UD-Q2_K_XL.gguf", 9828981664ull, "fd4730dd8aad070517978752b63d530aeb1740d2283cab9fa24f1e404032ddb0", nullptr },
{} },
+ // Bonsai 2 27B, PrismML's ternary Qwen 3.8, as upstream's Q2_0: the app's
+ // llama.cpp applies its Hadamard rotation (prism.hadamard.*). PrismML keeps
+ // this file in its -dev repo under a "prism-fork-required" name from before
+ // upstream had the rotation; it is byte for byte what scripts/bonsai-pq2-to-q2.py
+ // makes from the main repo's PQ2_0, and it answers correctly with
+ // ggml-vulkan on a PC. Not yet run on the console.
+ { "Bonsai 2 27B", false, "Qwen 3.8 27B in ternary weights, with a 64k context", 0, "prism-ml/Ternary-Bonsai-2-27B-gguf-dev",
+ { "Ternary-Bonsai-2-27B-Q2_0-prism-fork-required.gguf", 7626008928ull,
+ "4f99aed01b8a877e153f9aa6569a4440fe17c59701cc0953e36d7f460549e70e", "Ternary-Bonsai-2-27B-Q2_0.gguf" },
+ {} },
{ "Qwen 3.5 9B", false, "Fast all-rounder that leaves room for long chats", 55, "unsloth/Qwen3.5-9B-GGUF",
{ "Qwen3.5-9B-Q4_K_M.gguf", 5680522464ull, "03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8", nullptr },
{} },
diff --git a/scripts/bench-ubatch.py b/scripts/bench-ubatch.py
new file mode 100755
index 0000000..9d73a72
--- /dev/null
+++ b/scripts/bench-ubatch.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python3
+# Prompt and generation speed, and memory, of one model at several prompt
+# batch sizes (llama-server's -ub), on the console, to choose the planner's
+# kPromptUbatch (ps5/app/model_plan.hpp).
+#
+# PS5_HOST=192.168.1.19 scripts/bench-ubatch.py /data/PS5LM/models/Qwen3.8-27B-UD-Q2_K_XL.gguf
+# PS5_HOST=192.168.1.19 scripts/bench-ubatch.py <model on the console> 64 128 256 512
+#
+# The app loads the model with the planner first, and its arguments (from
+# app.log) are kept for every size; only -ub, -b and the log verbosity change.
+# Each size restarts the app with them in app-args.txt, sends the same prompt
+# (2,048 tokens of the LICENSE) twice with 128 tokens to generate, keeps the
+# faster run, and reads what the app logged about memory: the pool free once
+# the model serves and again after the prompt, and llama.cpp's buffers. A size
+# that does not load is reported as such. At the end app-args.txt is removed
+# and the model loads with the planner again.
+#
+# Needs the app running with /data open, ftpsrv on 2121, Python 3.8+.
+#
+# SPDX-License-Identifier: GPL-3.0-or-later
+import ftplib
+import io
+import json
+import os
+import re
+import sys
+import time
+import urllib.error
+import urllib.request
+
+HOST = os.environ.get("PS5_HOST") or sys.exit("bench-ubatch: set PS5_HOST to the console's address")
+FTP_PORT = int(os.environ.get("PS5_FTP_PORT", "2121"))
+API = "http://%s:8081" % HOST
+DATA = "/data/PS5LM"
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+PROMPT_TOKENS, GENERATE = 2048, 128
+
+
+def ftp():
+ f = ftplib.FTP()
+ f.connect(HOST, FTP_PORT, timeout=30)
+ f.login()
+ return f
+
+
+def put(name, text):
+ with ftp() as f:
+ f.storbinary("STOR %s/%s" % (DATA, name), io.BytesIO(text.encode()))
+
+
+def get(name):
+ out = io.BytesIO()
+ try:
+ with ftp() as f:
+ f.retrbinary("RETR %s/%s" % (DATA, name), out.write)
+ except ftplib.error_perm:
+ return ""
+ return out.getvalue().decode("utf-8", "replace")
+
+
+def remove(name):
+ try:
+ with ftp() as f:
+ f.delete("%s/%s" % (DATA, name))
+ except ftplib.error_perm:
+ pass
+
+
+def post(path, body, timeout=3600):
+ req = urllib.request.Request(API + path, json.dumps(body).encode(), {"Content-Type": "application/json"})
+ return json.load(urllib.request.urlopen(req, timeout=timeout))
+
+
+def this_run(log):
+ """app.log from the app's latest start."""
+ at = log.rfind("ps5lm-app: version")
+ return log[at:] if at >= 0 else log
+
+
+def wait_ready(started_after, timeout=1200):
+ """Until the server answers /health, or the new process logs that it stopped."""
+ t0 = time.time()
+ while time.time() - t0 < timeout:
+ time.sleep(5)
+ log = get("app.log")
+ if "ps5lm-app: version" not in log:
+ continue
+ run = this_run(log)
+ if run == started_after:
+ continue # the old process is still there
+ if "llama_server returned" in run:
+ return False, run
+ try:
+ with urllib.request.urlopen(API + "/health", timeout=5) as r:
+ if r.status == 200:
+ return True, run
+ except (urllib.error.URLError, OSError):
+ pass
+ return False, this_run(get("app.log"))
+
+
+def planner_args(run):
+ args = [l.split("ps5lm-app: arg ", 1)[1] for l in run.splitlines() if l.startswith("ps5lm-app: arg ")]
+ out, skip = [], False
+ for a in args: # what the app adds itself
+ if skip:
+ skip = False
+ elif a in ("--log-file", "--tools"):
+ skip = True
+ elif a != "--metrics":
+ out.append(a)
+ return out
+
+
+def with_ubatch(args, ub):
+ out = list(args)
+ for flag, value in (("-ub", ub), ("-b", max(256, ub))):
+ if flag in out:
+ out[out.index(flag) + 1] = str(value)
+ else:
+ out += [flag, str(value)]
+ return out + ["-lv", "4"] # llama.cpp's buffer sizes in llama.log
+
+
+def pool_free(run, when):
+ m = re.findall(r"memory %s: pool [0-9.]+ GiB, pool free ([0-9.]+) GiB" % re.escape(when), run)
+ return float(m[-1]) if m else None
+
+
+def buffers(llama_log):
+ sizes = {}
+ for line in llama_log.splitlines():
+ m = re.search(r"(\S+) (model|KV|RS|compute) buffer size = +([0-9.]+) MiB", line)
+ if m:
+ sizes["%s %s" % (m.group(1), m.group(2))] = float(m.group(3))
+ return sizes
+
+
+def main():
+ if len(sys.argv) < 2:
+ sys.exit("usage: bench-ubatch.py <model path on the console> [ubatch ...]")
+ model = sys.argv[1]
+ sizes = [int(x) for x in sys.argv[2:]] or [64, 128, 256, 512]
+ prompt_text = open(os.path.join(ROOT, "LICENSE"), encoding="utf-8").read()
+
+ print("bench-ubatch: loading %s with the planner" % model)
+ before = this_run(get("app.log"))
+ remove("app-args.txt")
+ put("load", model + "\n")
+ ok, run = wait_ready(before)
+ if not ok:
+ sys.exit("bench-ubatch: the model did not load with the planner; see app.log and llama.log")
+ base = planner_args(run)
+ print("bench-ubatch: the planner's arguments: %s" % " ".join(base))
+
+ rows = []
+ for ub in sizes:
+ args = with_ubatch(base, ub)
+ before = run
+ put("app-args.txt", "\n".join(args) + "\n")
+ put("unload", "-\n") # restarts the app, which then reads app-args.txt
+ ok, run = wait_ready(before)
+ if not ok:
+ rows.append((ub, None, None, None, None, "did not load (out of memory? see llama.log)"))
+ print("bench-ubatch: -ub %d did not load" % ub)
+ continue
+ tokens = post("/tokenize", {"content": prompt_text})["tokens"][:PROMPT_TOKENS]
+ post("/completion", {"prompt": tokens[:64], "n_predict": 4, "cache_prompt": False}) # shaders, buffers
+ best = None
+ for _ in range(2):
+ t = post("/completion", {"prompt": tokens, "n_predict": GENERATE, "cache_prompt": False,
+ "temperature": 0, "ignore_eos": True})["timings"]
+ best = t if best is None or t["prompt_per_second"] > best["prompt_per_second"] else best
+ put("memory", "-\n")
+ time.sleep(4)
+ run = this_run(get("app.log"))
+ buf = buffers(get("llama.log"))
+ compute = sum(v for k, v in buf.items() if k.endswith("compute"))
+ rows.append((ub, best["prompt_per_second"], best["predicted_per_second"], pool_free(run, "when ready"),
+ pool_free(run, "asked"), "compute buffers %.0f MiB" % compute))
+ print("bench-ubatch: -ub %d: prompt %.1f tok/s, generation %.1f tok/s" % (ub, rows[-1][1], rows[-1][2]))
+
+ remove("app-args.txt")
+ put("load", model + "\n")
+ print()
+ print("| -ub | prompt tok/s (%d tokens) | generation tok/s | pool free when ready | pool free after the prompt | |" % PROMPT_TOKENS)
+ print("|---|---|---|---|---|---|")
+ for ub, pp, tg, ready, after, note in rows:
+ f = lambda v, fmt: fmt % v if v is not None else "-"
+ print("| %d | %s | %s | %s | %s | %s |" % (ub, f(pp, "%.1f"), f(tg, "%.1f"), f(ready, "%.2f GiB"),
+ f(after, "%.2f GiB"), note))
+ print("\nbench-ubatch: the model loads with the planner again")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/bonsai-pq2-to-q2.py b/scripts/bonsai-pq2-to-q2.py
new file mode 100755
index 0000000..99e70f0
--- /dev/null
+++ b/scripts/bonsai-pq2-to-q2.py
@@ -0,0 +1,233 @@
+#!/usr/bin/env python3
+# Converts PrismML's PQ2_0 GGUF files (Ternary Bonsai 2) to upstream llama.cpp's
+# Q2_0, with no loss: the app's llama.cpp loads Q2_0 and applies the model's
+# Hadamard rotation (prism.hadamard.* metadata) itself, but refuses PQ2_0, a type
+# only PrismML's fork knows.
+#
+# scripts/bonsai-pq2-to-q2.py Ternary-Bonsai-2-27B-PQ2_0.gguf
+# scripts/bonsai-pq2-to-q2.py in.gguf out.gguf
+#
+# Python 3.8 or newer and nothing else; about a minute for the 27B, and 6% more
+# disk than the input. The output goes to <name with Q2_0>.gguf beside the input.
+#
+# Both types hold the same ternary weights with an fp16 scale per block, and
+# use the same 2-bit codes (0, 1, 2 for -1, 0, +1, the lowest bits first). A
+# PQ2_0 block is 128 weights (the scale, then 32 bytes of codes); a Q2_0 block
+# is 64 (the scale, then 16 bytes). Each PQ2_0 block becomes two Q2_0 blocks
+# with its scale and one half of its codes each, so every weight dequantizes to
+# the same value. Code 3, which ternary weights never use, stops the conversion.
+# Every other tensor and every key is copied as it is, except general.file_type.
+# Checked against PrismML's own Q2_0 files: the 2B converts to the same bytes.
+#
+# SPDX-License-Identifier: GPL-3.0-or-later
+import hashlib
+import os
+import struct
+import sys
+
+PQ2_0, Q2_0 = 142, 42 # ggml type ids: PrismML's fork, upstream
+FTYPE_PQ2_0, FTYPE_Q2_0 = 141, 41 # general.file_type
+GGML_TYPE_COUNT = 43 # upstream knows the types below this
+PQ_BLOCK, PQ_BYTES = 128, 34
+Q_BLOCK, Q_BYTES = 64, 18
+CHUNK = 1 << 20 # blocks converted at a time
+
+# Scalar sizes of GGUF metadata types (8 is a string, 9 an array).
+SCALAR = {0: 1, 1: 1, 2: 2, 3: 2, 4: 4, 5: 4, 6: 4, 7: 1, 10: 8, 11: 8, 12: 8}
+
+# For each byte of codes, 1 when one of its four codes is 3.
+HAS_CODE_3 = bytes(1 if any((b >> s) & 3 == 3 for s in (0, 2, 4, 6)) else 0 for b in range(256))
+
+
+def fail(text):
+ sys.exit("bonsai-pq2-to-q2: " + text)
+
+
+class Reader:
+ def __init__(self, f):
+ self.f = f
+
+ def take(self, n):
+ b = self.f.read(n)
+ if len(b) != n:
+ fail("the file ends inside its header")
+ return b
+
+ def u32(self):
+ return struct.unpack("<I", self.take(4))[0]
+
+ def u64(self):
+ return struct.unpack("<Q", self.take(8))[0]
+
+ def string(self):
+ return self.take(self.u64())
+
+
+def skip_value(r, t, raw):
+ """Reads one metadata value of type t, appending its bytes to raw."""
+ if t == 8:
+ n = r.take(8)
+ raw += n + r.take(struct.unpack("<Q", n)[0])
+ elif t == 9:
+ head = r.take(12)
+ raw += head
+ et, n = struct.unpack("<IQ", head)
+ if et == 8:
+ for _ in range(n):
+ skip_value(r, 8, raw)
+ elif et in SCALAR:
+ raw += r.take(n * SCALAR[et])
+ else:
+ fail("unknown array element type %d" % et)
+ elif t in SCALAR:
+ raw += r.take(SCALAR[t])
+ else:
+ fail("unknown metadata type %d" % t)
+
+
+def to_q2_0(block_bytes):
+ """PQ2_0 blocks to Q2_0 blocks: the scale twice, the codes split in halves."""
+ n = len(block_bytes) // PQ_BYTES
+ out = bytearray(n * 2 * Q_BYTES)
+ for j in range(2): # the fp16 scale
+ out[j::2 * Q_BYTES] = block_bytes[j::PQ_BYTES]
+ out[Q_BYTES + j::2 * Q_BYTES] = block_bytes[j::PQ_BYTES]
+ for j in range(16): # codes of weights 0..63, then 64..127
+ out[2 + j::2 * Q_BYTES] = block_bytes[2 + j::PQ_BYTES]
+ out[Q_BYTES + 2 + j::2 * Q_BYTES] = block_bytes[18 + j::PQ_BYTES]
+ codes = bytearray(out)
+ codes[0::Q_BYTES] = bytes(2 * n)
+ codes[1::Q_BYTES] = bytes(2 * n)
+ if 1 in codes.translate(HAS_CODE_3):
+ fail("a block holds code 3, which is not a ternary weight; this is not a file this script knows")
+ return out
+
+
+def main():
+ if len(sys.argv) not in (2, 3) or sys.argv[1] in ("-h", "--help"):
+ sys.exit("usage: bonsai-pq2-to-q2.py IN.gguf [OUT.gguf]")
+ src = sys.argv[1]
+ if len(sys.argv) == 3:
+ dst = sys.argv[2]
+ else:
+ base = os.path.basename(src)
+ name = base.replace("PQ2_0", "Q2_0") if "PQ2_0" in base else base[:-5] + "-Q2_0.gguf"
+ dst = os.path.join(os.path.dirname(src), name)
+ if os.path.abspath(dst) == os.path.abspath(src):
+ fail("the output would replace the input")
+
+ f = open(src, "rb")
+ r = Reader(f)
+ if r.take(4) != b"GGUF":
+ fail("%s is not a GGUF file" % src)
+ version = r.u32()
+ if version != 3:
+ fail("GGUF version %d; this script reads version 3" % version)
+ n_tensors, n_kv = r.u64(), r.u64()
+
+ kvs = [] # (raw key bytes, type, raw value bytes)
+ alignment = 32
+ for _ in range(n_kv):
+ key = r.string()
+ t = r.u32()
+ raw = bytearray()
+ skip_value(r, t, raw)
+ if key == b"general.alignment" and t == 4:
+ alignment = struct.unpack("<I", raw)[0]
+ if key == b"general.file_type" and t == 4 and struct.unpack("<I", raw)[0] == FTYPE_PQ2_0:
+ raw = bytearray(struct.pack("<I", FTYPE_Q2_0))
+ kvs.append((key, t, bytes(raw)))
+
+ tensors = [] # [name, dims, type, offset]
+ for _ in range(n_tensors):
+ name = r.string()
+ dims = [r.u64() for _ in range(r.u32())]
+ t, off = r.u32(), r.u64()
+ tensors.append([name, dims, t, off])
+ data_start = (f.tell() + alignment - 1) // alignment * alignment
+ file_size = os.fstat(f.fileno()).st_size
+
+ # Each tensor's bytes run to the next tensor's start (padding included),
+ # so types this script does not size itself are copied whole.
+ by_offset = sorted(tensors, key=lambda x: x[3])
+ span = {}
+ for i, x in enumerate(by_offset):
+ end = by_offset[i + 1][3] if i + 1 < len(by_offset) else file_size - data_start
+ span[x[0]] = end - x[3]
+
+ converted = 0
+ for name, dims, t, off in tensors:
+ n = 1
+ for d in dims:
+ n *= d
+ if t == PQ2_0:
+ if dims[0] % PQ_BLOCK:
+ fail("%s: rows of %d weights, not whole PQ2_0 blocks" % (name.decode(), dims[0]))
+ if n // PQ_BLOCK * PQ_BYTES > span[name]:
+ fail("%s: the file holds less data than its shape needs" % name.decode())
+ converted += 1
+ elif t >= GGML_TYPE_COUNT:
+ fail("%s has type %d, which upstream llama.cpp does not know either (PrismML's PTQ1_0 is 143); "
+ "only PQ2_0 (142) converts" % (name.decode(), t))
+ if not converted:
+ fail("%s has no PQ2_0 tensors: nothing to convert" % src)
+
+ # The new layout: tensors in the order listed, each at an aligned offset.
+ sizes, out_off, at = {}, {}, 0
+ for name, dims, t, off in tensors:
+ n = 1
+ for d in dims:
+ n *= d
+ sizes[name] = n // Q_BLOCK * Q_BYTES if t == PQ2_0 else span[name]
+ out_off[name] = at
+ at = (at + sizes[name] + alignment - 1) // alignment * alignment
+
+ header = bytearray(b"GGUF" + struct.pack("<IQQ", version, n_tensors, n_kv))
+ for key, t, raw in kvs:
+ header += struct.pack("<Q", len(key)) + key + struct.pack("<I", t) + raw
+ for name, dims, t, off in tensors:
+ header += struct.pack("<Q", len(name)) + name + struct.pack("<I", len(dims))
+ header += struct.pack("<%dQ" % len(dims), *dims)
+ header += struct.pack("<IQ", Q2_0 if t == PQ2_0 else t, out_off[name])
+ header += bytes((-len(header)) % alignment)
+
+ part = dst + ".part"
+ sha = hashlib.sha256()
+ with open(part, "wb") as out:
+ def emit(b):
+ out.write(b)
+ sha.update(b)
+ emit(header)
+ for i, (name, dims, t, off) in enumerate(tensors):
+ f.seek(data_start + off)
+ if t == PQ2_0:
+ left = sizes[name] // (2 * Q_BYTES) * PQ_BYTES
+ while left:
+ take = min(left, CHUNK * PQ_BYTES)
+ emit(to_q2_0(f.read(take)))
+ left -= take
+ else:
+ left = sizes[name]
+ while left:
+ b = f.read(min(left, 64 << 20))
+ if not b:
+ fail("%s: the file ends inside its data" % name.decode())
+ emit(b)
+ left -= len(b)
+ pad = (-sizes[name]) % alignment
+ if pad:
+ emit(bytes(pad))
+ if sys.stderr.isatty():
+ sys.stderr.write("\rbonsai-pq2-to-q2: %d of %d tensors" % (i + 1, len(tensors)))
+ out.flush()
+ os.fsync(out.fileno())
+ os.replace(part, dst)
+ if sys.stderr.isatty():
+ sys.stderr.write("\n")
+ print("bonsai-pq2-to-q2: %s: %d PQ2_0 tensors as Q2_0, %d others copied, %.2f GiB" %
+ (dst, converted, len(tensors) - converted, os.path.getsize(dst) / 1073741824))
+ print("sha256 %s" % sha.hexdigest())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/build-app.sh b/scripts/build-app.sh
index 32c3d17..43a6aa8 100755
--- a/scripts/build-app.sh
+++ b/scripts/build-app.sh
@@ -110,6 +110,9 @@ cxx -fexceptions -fcxx-exceptions -c "$root/ps5/app/model_plan.cpp" -o "$obj/mod
cxx -fexceptions -fcxx-exceptions -I "$llama/ggml/include" \
-c "$root/ps5/app/fleet_worker.cpp" -o "$obj/fleet_worker.o"
cxx -std=c++20 -fexceptions -fcxx-exceptions -I "$llama/vendor" -c "$root/ps5/app/settings.cpp" -o "$obj/settings.o"
+cxx -std=c++20 -fexceptions -fcxx-exceptions -c "$root/ps5/app/report.cpp" -o "$obj/report.o"
+cxx -std=c++20 -fexceptions -fcxx-exceptions -c "$root/ps5/app/storage.cpp" -o "$obj/storage.o"
+cxx -std=c++20 -fexceptions -fcxx-exceptions -c "$root/ps5/app/probes.cpp" -o "$obj/probes.o"
cxx -std=c++20 -fexceptions -fcxx-exceptions -c "$root/ps5/app/fetch.cpp" -o "$obj/fetch.o"
cxx -std=c++20 -fexceptions -fcxx-exceptions -I "$llama/vendor" -c "$root/ps5/app/market.cpp" -o "$obj/market.o"
cxx -std=c++20 -fexceptions -fcxx-exceptions -c "$root/ps5/app/ftp_writer.cpp" -o "$obj/ftp_writer.o"
@@ -212,7 +215,7 @@ mapfile -t libs < <(find "$out/llama" -name '*.a' | sort)
"$sdk/bin/prospero-lld" "${radv_linker_script[@]}" --eh-frame-hdr "${radv_link_flags[@]}" \
--version-script "$vk/tooling/native/app-symbols.map" --exclude-libs=ALL \
-e _start -o "$out/llvm-pie.elf" \
- "$obj/app_crt.o" "$obj/app_cpp_runtime.o" "$obj/main.o" "$obj/display.o" "$obj/model_plan.o" "$obj/fleet_worker.o" "$obj/settings.o" "$obj/fetch.o" "$obj/market.o" "$obj/ftp_writer.o" "$obj/activity.o" "$obj/sha256.o" "$obj/dashboard.o" "$obj/stats.o" \
+ "$obj/app_crt.o" "$obj/app_cpp_runtime.o" "$obj/main.o" "$obj/display.o" "$obj/model_plan.o" "$obj/fleet_worker.o" "$obj/settings.o" "$obj/report.o" "$obj/storage.o" "$obj/probes.o" "$obj/fetch.o" "$obj/market.o" "$obj/ftp_writer.o" "$obj/activity.o" "$obj/sha256.o" "$obj/dashboard.o" "$obj/stats.o" \
--start-group "${libs[@]}" "$obj/libps5lm_kit.a" "$obj/libps5app_compat.a" "$out/gl/libps5lm_gl.a" --end-group \
"$out/stubs/libSceAgc.so" "$out/stubs/libSceAgcDriver.so" \
"${radv_link_inputs[@]}" \
diff --git a/scripts/preview-app-ui.sh b/scripts/preview-app-ui.sh
index 856957e..6bf5178 100755
--- a/scripts/preview-app-ui.sh
+++ b/scripts/preview-app-ui.sh
@@ -19,7 +19,7 @@ cc=${HOST_CC:-clang}
# host platform functions, and the app's dashboard.
mapfile -t sources < <(find "$kit/src/gfx" "$kit/src/ui" "$kit/src/core" "$kit/src/audio" "$kit/src/third_party" \
-type f \( -name '*.cpp' -o -name '*.c' \) | sort)
-sources+=("$kit/host/platform_host.cpp" "$root/ps5/app/dashboard.cpp" "$root/ps5/app/stats.cpp"
+sources+=("$kit/host/platform_host.cpp" "$root/ps5/app/dashboard.cpp" "$root/ps5/app/stats.cpp" "$root/ps5/app/storage.cpp"
"$root/ps5/app/preview_main.cpp")
compile() {
diff --git a/scripts/ps5lm-app.sh b/scripts/ps5lm-app.sh
index c31d9ef..5b3daa5 100755
--- a/scripts/ps5lm-app.sh
+++ b/scripts/ps5lm-app.sh
@@ -11,6 +11,9 @@
# scripts/ps5lm-app.sh press <button> press a button (up, down, cross, l1, r1, ...)
# scripts/ps5lm-app.sh fetch <url> probe: download over HTTPS (system TLS), see app.log
# scripts/ps5lm-app.sh bench-write probe: write speed by method, see app.log
+# scripts/ps5lm-app.sh memory log the memory pools now, see app.log
+# scripts/ps5lm-app.sh bench-read <file> probe: sequential and random read speed of a model file, see app.log
+# scripts/ps5lm-app.sh bench-memory probe: the pools and the largest block that allocates, see app.log
# scripts/ps5lm-app.sh search <words> search Get models from the PC
#
# The app is never killed: it polls /data/PS5LM/quit and leaves through
@@ -77,6 +80,9 @@ case ${1:-} in
unload) request unload ;;
fetch) request fetch "${2:?https URL}" ;;
bench-write) request bench-write ;;
+ memory) request memory ;;
+ bench-read) request bench-read "${2:?a model file on the console, several GiB}" ;;
+ bench-memory) request bench-memory ;;
search) request search "${2:?search words}" ;;
press) request press "${2:?up|down|left|right|cross|circle|triangle|square|l1|r1|l2|r2}" ;;
screenshot) screenshot "${2:-}" ;;
@@ -84,5 +90,5 @@ case ${1:-} in
deploy) deploy ;;
launch) launch ;;
cycle) close; deploy; launch ;;
- *) echo "usage: $0 close|deploy|launch|cycle|screenshot|load|unload|press|fetch|bench-write|search" >&2; exit 2 ;;
+ *) echo "usage: $0 close|deploy|launch|cycle|screenshot|load|unload|press|fetch|bench-write|bench-read|bench-memory|memory|search" >&2; exit 2 ;;
esac