llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
ci: validate the existing PS5 app with a pinned public native build (#36)
Nicholas Tindleecbcb8cc54fa
diff --git a/.github/workflows/native-build.yml b/.github/workflows/native-build.ymlnew file mode 100644index 0000000..45a7c93--- /dev/null+++ b/.github/workflows/native-build.yml@@ -0,0 +1,75 @@+name: Native app validation (not PS5 hardware)+on:+ pull_request:+ push:+ branches: [main, 'ci/**']+permissions:+ contents: read+concurrency:+ group: native-app-${{ github.workflow }}-${{ github.ref }}+ cancel-in-progress: true+jobs:+ host-checks:+ name: Existing model planner and script syntax+ runs-on: ubuntu-24.04+ timeout-minutes: 5+ steps:+ - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1+ with:+ persist-credentials: false+ - name: Shell and Python syntax (no submodules needed)+ run: |+ for script in scripts/*.sh ps5/app/ps5cc ps5/app/ps5cxx; do bash -n "$script"; done+ PYTHONPYCACHEPREFIX=build/pycache python3 -m py_compile scripts/*.py+ - name: Build-script job limit and compiler-failure regressions+ run: PYTHONPYCACHEPREFIX=build/pycache python3 -m unittest discover -s tests -p 'test_build_app.py' -v+ - name: Existing synthetic GGUF model-planner regression test+ run: |+ mkdir -p build+ c++ -std=c++17 -O1 ps5/app/model_plan_test.cpp ps5/app/model_plan.cpp -o build/model_plan_test+ build/model_plan_test++ ps5-title:+ name: Full PS5 title cross-build and static package checks+ runs-on: ubuntu-24.04+ timeout-minutes: 45+ container:+ # Official Debian image; trixie supplies compatible LLVM19/SPIRV host tools.+ image: debian:trixie-20261020-slim@sha256:62bd423aeee912639ca8ee681c2b762cb3e25137e0a92b20be455898a9f034ad+ options: --cpus 2+ defaults:+ run:+ shell: bash+ env:+ DEBIAN_FRONTEND: noninteractive+ BUILD_JOBS: '2'+ LLVM_CONFIG: /usr/bin/llvm-config-19+ steps:+ - name: Public cross-build prerequisites+ run: |+ apt-get update+ apt-get install -y --no-install-recommends \+ build-essential git ca-certificates wget curl unzip rsync cmake ninja-build \+ clang-19 llvm-19-dev lld-19 libclang-rt-19-dev libclang-19-dev libclang-cpp19-dev \+ libllvmspirvlib-19-dev libclc-19-dev spirv-tools spirv-headers glslang-tools \+ meson pkgconf flex bison python3 python3-mako python3-yaml python3-packaging python3-setuptools \+ zlib1g-dev libzstd-dev libffi-dev libedit-dev libxml2-dev libvulkan-dev glslc ffmpeg xxd+ echo /usr/lib/llvm-19/bin >> "$GITHUB_PATH"+ - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1+ with:+ persist-credentials: false+ - name: Pinned llama.cpp and UI sources+ run: |+ # Container root differs from the runner owner. Trust only this checkout,+ # in the disposable job container; never use safe.directory='*'.+ git config --global --add safe.directory "$GITHUB_WORKSPACE"+ git submodule update --init --depth 1+ - name: Build patched RADV and title, inspect ELF and FSELF+ run: scripts/test-native-build.sh+ - name: State the full-build validation boundary+ if: always()+ run: |+ echo '## Native PS5 title cross-build' >> "$GITHUB_STEP_SUMMARY"+ echo 'Builds the pinned public SDK/runtime, patched RADV, UI and existing app title; checks ELF/FSELF structure.' >> "$GITHUB_STEP_SUMMARY"+ echo 'Build tools and checksum-verified public SDK/driver dependencies download; no model or proprietary firmware is needed.' >> "$GITHUB_STEP_SUMMARY"+ echo 'This is static cross-build validation, not console boot, Vulkan execution, PS5 emulation or model inference.' >> "$GITHUB_STEP_SUMMARY"diff --git a/docs/BUILDING.md b/docs/BUILDING.mdindex a18ef33..bb4673f 100644--- a/docs/BUILDING.md+++ b/docs/BUILDING.md@@ -27,6 +27,10 @@ works too.## Build+For the automated, pinned-source Debian build and its validation limits, see+[CI validation](CI.md). `scripts/test-native-build.sh` builds and statically checks+the existing app without a console, model download, or proprietary firmware.+On Arch, the host packages (or run them in an `archlinux` container with the repository mounted):```shdiff --git a/docs/CI.md b/docs/CI.mdnew file mode 100644index 0000000..3e12f55--- /dev/null+++ b/docs/CI.md@@ -0,0 +1,84 @@+# CI validation++[`native-build.yml`](../.github/workflows/native-build.yml) checks the existing app+on pull requests and pushes to `main` or `ci/**`. It uses ordinary Ubuntu hosted+runners, a read-only token, no secrets, and no console connection. Superseded runs+are cancelled. There are two independent jobs:++- **Host checks** (5-minute limit): shell/Python syntax, the existing synthetic+ GGUF `model_plan_test.cpp`, and regressions for dashboard compiler job limits+ and propagation of failed background compilers. No submodule or model download+ is needed for these checks.+- **Full PS5 title cross-build** (45-minute limit, two compiler jobs): bootstrap+ the public SDK, reproduce clean-room `libc.prx`, build patched RADV and the+ existing llama-server/native dashboard, package the title folder, and inspect+ its ELF/FSELF structure and integrity. It does not execute the resulting title.++The second job downloads build tools, source dependencies and checksum-verified+public SDK archives. It does **not** download a trained model or proprietary+console firmware. Only the optional browser UI is omitted: its upstream fallback+fetches a floating `latest` release. The native console dashboard is built.+Normal `scripts/build-app.sh` builds keep the prebuilt browser UI enabled;+`PS5LM_PREBUILT_UI=OFF` explicitly opts out and clears legacy CMake aliases.++## Reproduce the checks++Host checks need Bash, Python 3 and a C++17 compiler:++```sh+for script in scripts/*.sh ps5/app/ps5cc ps5/app/ps5cxx; do bash -n "$script"; done+PYTHONPYCACHEPREFIX=build/pycache python3 -m py_compile scripts/*.py+PYTHONPYCACHEPREFIX=build/pycache python3 -m unittest discover -s tests -p 'test_build_app.py' -v+mkdir -p build+c++ -std=c++17 -O1 ps5/app/model_plan_test.cpp ps5/app/model_plan.cpp -o build/model_plan_test+build/model_plan_test+```++For the full build, use the digest-pinned Debian trixie container and exact apt+package list in the workflow, or an x86-64 Linux host with equivalent tools:++```sh+git submodule update --init --depth 1+export PATH=/usr/lib/llvm-19/bin:$PATH+LLVM_CONFIG=/usr/bin/llvm-config-19 BUILD_JOBS=2 scripts/test-native-build.sh+```++The script keeps dependencies in `.deps/native-ci-src` (override with+`PS5LM_NATIVE_SOURCES`) and validates existing dependency revisions before reuse.+It uses upstream's checked SDK setup and `make libc`, which rebuilds the runtime+twice and compares the recorded raw/signed hashes. That target also produces the+same host packaging tool required by the app, without building an unrelated demo+title. It applies this repository's three Mesa patches with fixed commit identity+and dates. The removed OpenGL release is fetched from the mirror already+documented in [BUILDING.md](BUILDING.md), checked against the original archive+hash, then checked by the pinned UI fetcher's file manifest.++| Input | Pin |+|---|---|+| Debian container | `trixie-20261020-slim@sha256:62bd423aeee912639ca8ee681c2b762cb3e25137e0a92b20be455898a9f034ad` |+| PS5_Vulkan | `f410e67a0241b3e8edcdbcd2a9b285dfc4a412ed` |+| PS5_PayloadSDK fork | `b83202be73e930050e00de0f1b4d9ec46c0391bf` |+| Mesa base | `1ba9874f478657ccf6da72ce55f0260164cc0bd0` |+| Mesa after this repository's patches | `9e0a7c63bc6d4b6bcb60831163bb3fe9f46e91ea` |+| llama.cpp / UI | The checkout's gitlink revisions (verified before building) |+| OpenGL SDK archive SHA-256 | `f93643c04c843d56143b00951df1f8042ea7706ae9f19e4e9abf158e1ead77c5` |++`build/native-ci/result.json` records the source pins and SHA-256 hashes of the+packaged files after successful inspections. The title is in+`build/app-ps5/title/PPSA99575`. A failed check exits nonzero; there is no automatic+deployment, release publication or binary execution. CI publishes logs, not a+release package.++## What a green result means++It proves the checked source can build/link/package with these public tools and+pass static ELF/FSELF checks, including the native UI and Vulkan driver build.+It does **not** prove the title boots on a PS5, the GPU executes inference, models+fit memory, or networking/performance works on hardware. A generic QEMU x86-64+guest cannot supply the PS5 loader, system services or AGC GPU interface; it would+not close that gap. Hardware smoke tests remain separate.++Source pins and archive checksums make the build inputs explicit, but apt package+updates and build paths/compiler versions can change title bytes. Only the+upstream clean-room runtime has a fixed byte-reproduction check; the title hashes+identify a particular build, not a promised reproducible release binary.diff --git a/scripts/build-app.sh b/scripts/build-app.shindex 495f410..70c66a7 100755--- a/scripts/build-app.sh+++ b/scripts/build-app.sh@@ -9,6 +9,8 @@# tools/build-radv.sh release and `make app` (for the host tool and libc.prx).## scripts/build-app.sh configure, build and package+# PS5LM_PREBUILT_UI=OFF ... omit the optional downloaded browser UI+# CMAKE_BUILD_PARALLEL_LEVEL also bounds the native dashboard compilers.set -euo pipefailroot="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"@@ -76,7 +78,9 @@ cmake -S "$llama" -B "$out/llama" -G Ninja \-DVulkan_GLSLC_EXECUTABLE="$(command -v glslc)" -DSPIRV-Headers_DIR="$headers/cmake" \-DMATH_LIBRARY=SceLibcInternal \-DLLAMA_OPENSSL=OFF -DLLAMA_BUILD_COMMON=ON -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF \- -DLLAMA_BUILD_TOOLS=ON -DLLAMA_BUILD_SERVER=ON > "$out.configure.log" 2>&1 ||+ -DLLAMA_BUILD_TOOLS=ON -DLLAMA_BUILD_SERVER=ON \+ -ULLAMA_BUILD_WEBUI -ULLAMA_USE_PREBUILT_WEBUI \+ -DLLAMA_BUILD_UI=OFF -DLLAMA_USE_PREBUILT_UI="${PS5LM_PREBUILT_UI:-ON}" > "$out.configure.log" 2>&1 ||{ tail -30 "$out.configure.log" >&2; exit 1; }cmake --build "$out/llama" --target llama-server-impl@@ -134,6 +138,9 @@ mkdir -p "$obj/kit"mapfile -t kit_sources < <(find "$kit/src/gfx" "$kit/src/ui" "$kit/src/core" "$kit/src/audio" "$kit/src/third_party" \"$kit/src/platform/ps5" -type f \( -name '*.cpp' -o -name '*.c' \) ! -name display_egl.cpp | sort)kit_objects=()+kit_pending=()+jobs_limit=${CMAKE_BUILD_PARALLEL_LEVEL:-$(nproc)}+[[ $jobs_limit =~ ^[1-9][0-9]*$ ]] || { echo "build-app: invalid job limit" >&2; exit 2; }for src in "${kit_sources[@]}"; doo="$obj/kit/$(echo "${src#"$kit/src/"}" | tr '/' '_').o"kit_objects+=("$o")@@ -143,9 +150,14 @@ for src in "${kit_sources[@]}"; doelsecxx "${kitflags[@]}" -w -c "$src" -o "$o" &fi- (( $(jobs -r | wc -l) >= $(nproc) )) && wait -n+ # Retain each PID: bare wait can hide an already-finished compiler failure.+ kit_pending+=("$!")+ if (( ${#kit_pending[@]} >= jobs_limit )); then+ wait "${kit_pending[0]}"+ kit_pending=("${kit_pending[@]:1}")+ fidone-wait+for pid in "${kit_pending[@]}"; do wait "$pid"; donerm -f "$obj/libps5lm_kit.a""$sdk/bin/llvm-ar" rcs "$obj/libps5lm_kit.a" "${kit_objects[@]}"for src in display dashboard stats; dodiff --git a/scripts/test-native-build.sh b/scripts/test-native-build.shnew file mode 100755index 0000000..93a6335--- /dev/null+++ b/scripts/test-native-build.sh@@ -0,0 +1,148 @@+#!/usr/bin/env bash+# Full native cross-build and static package validation, NOT console execution.+set -euo pipefail+root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"+cd "$root"+output="$root/build/native-ci"+mkdir -p "$output/tools"+# Clear stale success evidence even if a preflight check below fails.+rm -f "$output/result.json"+export PS5LM_NATIVE_SOURCES="${PS5LM_NATIVE_SOURCES:-$root/.deps/native-ci-src}"+export BUILD_JOBS="${BUILD_JOBS:-2}"+[[ $BUILD_JOBS =~ ^[1-9][0-9]*$ ]] || { echo 'BUILD_JOBS must be a positive integer' >&2; exit 2; }+export CMAKE_BUILD_PARALLEL_LEVEL="$BUILD_JOBS"+# SDK archive wrappers resolve this as a path, not a PATH command name.+llvm_config=$(command -v "${LLVM_CONFIG:-llvm-config}") || {+ echo "title-check: missing LLVM_CONFIG tool: ${LLVM_CONFIG:-llvm-config}" >&2; exit 2;+}+export LLVM_CONFIG="$llvm_config"+for tool in git clang clang++ make wget unzip sha256sum timeout python3 cmake ninja meson rsync glslangValidator glslc pkg-config curl ffmpeg; do+ command -v "$tool" >/dev/null || { echo "title-check: missing $tool" >&2; exit 2; }+done+ui="$root/third_party/ps5-homebrew-ui"+ui_revision=$(git ls-tree HEAD third_party/ps5-homebrew-ui | awk '{print $3}')+[[ -f "$ui/tools/fetch-opengl-sdk.sh" && $(git -C "$ui" rev-parse HEAD) == "$ui_revision" ]] || {+ echo 'title-check: initialize the pinned UI submodule first' >&2; exit 2;+}+llama="$root/third_party/llama.cpp"+llama_revision=$(git ls-tree HEAD third_party/llama.cpp | awk '{print $3}')+[[ -f "$llama/CMakeLists.txt" && $(git -C "$llama" rev-parse HEAD) == "$llama_revision" ]] || {+ echo 'title-check: initialize the pinned llama.cpp submodule first' >&2; exit 2;+}+vk="$PS5LM_NATIVE_SOURCES/PS5_Vulkan"+sdk_fork="$PS5LM_NATIVE_SOURCES/PS5_PayloadSDK"+vk_revision=f410e67a0241b3e8edcdbcd2a9b285dfc4a412ed+sdk_revision=b83202be73e930050e00de0f1b4d9ec46c0391bf+fetch_pinned() {+ local url=$1 directory=$2 revision=$3+ if [[ ! -d "$directory/.git" ]]; then+ [[ ! -e "$directory" ]] || { echo "native-check: refusing existing non-repo $directory" >&2; exit 2; }+ mkdir -p "$directory"+ git -C "$directory" init -q+ timeout 120 git -C "$directory" fetch --depth 1 "$url" "$revision"+ git -C "$directory" checkout -q --detach FETCH_HEAD+ fi+ [[ $(git -C "$directory" rev-parse HEAD) == "$revision" ]] || {+ echo "native-check: unexpected revision in $directory" >&2; exit 2;+ }+ git -C "$directory" diff --quiet HEAD -- || {+ echo "native-check: tracked dependency source is modified: $directory" >&2; exit 2;+ }+}+fetch_pinned https://github.com/mihawk-99/PS5_Vulkan.git "$vk" "$vk_revision"+fetch_pinned https://github.com/mihawk-99/PS5_PayloadSDK.git "$sdk_fork" "$sdk_revision"++# Upstream scripts verify the v0.42 SDK release and zlib archive SHA-256s.+# These are public homebrew sources/tools, not proprietary firmware dumps.+export PS5_PAYLOAD_SDK_FORK="$sdk_fork"+timeout 300 bash "$vk/tools/setup-native-dependencies.sh"+export PS5_VULKAN_ROOT="$vk"+export PS5_PAYLOAD_SDK="$vk/.deps/native/ps5-payload-sdk"+[[ $(cat "$PS5_PAYLOAD_SDK/.ps5-sdk-revision") == "$sdk_revision" ]]+# Rebuild twice and verify upstream's recorded clean-room runtime checksums.+# This also builds the host packaging tool without linking an unrelated demo.+timeout 180 make -C "$vk" libc++mesa="$PS5LM_NATIVE_SOURCES/PS5_Mesa"+mesa_base=1ba9874f478657ccf6da72ce55f0260164cc0bd0+mesa_patched=9e0a7c63bc6d4b6bcb60831163bb3fe9f46e91ea+if [[ ! -d "$mesa/.git" ]]; then+ [[ ! -e "$mesa" ]] || { echo "title-check: refusing existing non-repo $mesa" >&2; exit 2; }+ mkdir -p "$mesa"+ git -C "$mesa" init -q+ timeout 180 git -C "$mesa" fetch --depth 1 https://github.com/mihawk-99/PS5_Mesa.git "$mesa_base"+ git -C "$mesa" checkout -q --detach FETCH_HEAD+fi+git -C "$mesa" diff --quiet HEAD -- || { echo 'title-check: modified Mesa sources' >&2; exit 2; }+if [[ $(git -C "$mesa" rev-parse HEAD) == "$mesa_base" ]]; then+ git -C "$mesa" -c user.name='PS5LM CI' -c user.email='[email protected]' \+ -c commit.gpgsign=false am --committer-date-is-author-date "$root"/patches/mesa/*.patch+fi+[[ $(git -C "$mesa" rev-parse HEAD) == "$mesa_patched" ]] || {+ echo 'title-check: unexpected patched Mesa revision' >&2; exit 2;+}+for patch in "$root"/patches/mesa/*.patch; do+ git -C "$mesa" apply --reverse --check "$patch"+done+export PS5_MESA_FORK="$mesa" PS5_MESA_REVISION="$mesa_patched"+export PS5_VULKAN_ROOT="$vk" PS5_PAYLOAD_SDK="$vk/.deps/native/ps5-payload-sdk"+# Upstream build-radv invokes Ninja without a job limit; bound both host and target stages.+ninja_real="$(command -v ninja)"+printf '#!/bin/sh\nexec "%s" -j "%s" "$@"\n' "$ninja_real" "$BUILD_JOBS" > "$output/tools/ninja"+chmod +x "$output/tools/ninja"+export NINJA="$output/tools/ninja"+timeout 1800 bash "$vk/tools/build-radv.sh" release++# make libc already built this exact host tool, without the unrelated demo title.+mkdir -p "$vk/build/host"+cp "$vk/build/runtime-shim/ps5-native-tool" "$vk/build/host/ps5-native-tool"+# The original OpenGL release was removed. Use the mirror documented in BUILDING.md,+# with the original archive hash; the pinned UI fetcher also verifies every SDK file.+glcache="$ui/.deps/ps5-opengl"+glarchive="$glcache/ps5-opengl-sdk-1.0.0.tar.gz"+if [[ ! -f "$glcache/ps5-opengl-sdk-1.0.0/sdk/manifest.sha256" ]]; then+ mkdir -p "$glcache"+ curl -fL --retry 2 --max-time 300 \+ https://github.com/MyLuxy/overdrive-deps/releases/download/ps5-opengl-v1.0.0/ps5-opengl-sdk-1.0.0.tar.gz \+ -o "$glarchive.part"+ echo "f93643c04c843d56143b00951df1f8042ea7706ae9f19e4e9abf158e1ead77c5 $glarchive.part" | sha256sum --check --status+ mv "$glarchive.part" "$glarchive"+fi+glsdk=$(bash "$ui/tools/fetch-opengl-sdk.sh")+export PS5_OPENGL_SDK="$glsdk"+# Cross-build validation omits llama.cpp's optional browser UI (which otherwise+# falls back to a floating Hugging Face "latest" download). Keep the console UI.+[[ ! -e "$root/third_party/llama.cpp/tools/ui/dist/index.html" ]] || {+ echo 'title-check: remove custom source-tree browser assets or use a clean checkout' >&2; exit 2;+}+rm -rf -- "$root/build/app-ps5/llama/tools/ui/dist"+export PS5LM_PREBUILT_UI=OFF+timeout 1800 bash scripts/build-app.sh+bash "$vk/tools/inspect.sh" "$root/build/app-ps5/eboot.elf"+title_id=$(python3 -c 'import json; print(json.load(open("ps5/app/sce_sys/param.json"))["titleId"])')+app="$root/build/app-ps5/title/$title_id"+bash "$vk/tools/inspect.sh" "$app/eboot.bin"+"$vk/build/host/ps5-native-tool" self --inspect --file "$app/eboot.bin"+python3 - "$output" "$app" "$mesa_base" "$mesa_patched" "$ui_revision" "$llama_revision" "$vk_revision" "$sdk_revision" <<'PY'+import hashlib+import json+from pathlib import Path+import sys++output, app = map(Path, sys.argv[1:3])+files = ('eboot.bin', 'sce_module/libc.prx', 'sce_sys/param.json', 'sce_sys/icon0.png', 'sce_sys/pic0.dds', 'sce_sys/pic1.dds')+result = {+ 'status': 'passed',+ 'scope': 'Complete native title cross-build and static ELF/FSELF validation; NOT console/Vulkan execution',+ 'mesa_base_revision': sys.argv[3],+ 'mesa_patched_revision': sys.argv[4],+ 'ui_revision': sys.argv[5],+ 'llama_revision': sys.argv[6],+ 'ps5_vulkan_revision': sys.argv[7],+ 'ps5_payload_sdk_revision': sys.argv[8],+ 'optional_browser_ui': 'omitted; native console dashboard is built',+ 'files_sha256': {name: hashlib.sha256((app / name).read_bytes()).hexdigest() for name in files},+}+(output / 'result.json').write_text(json.dumps(result, indent=2) + '\n')+print(json.dumps(result, indent=2))+PYdiff --git a/tests/test_build_app.py b/tests/test_build_app.pynew file mode 100644index 0000000..7900d11--- /dev/null+++ b/tests/test_build_app.py@@ -0,0 +1,203 @@+"""Exercise build-app.sh's real UI compiler queue without a PS5 SDK.++The extracted fragment is deliberately bounded by stable surrounding commands;+if the build script is reorganized, update this harness rather than testing a+copy of the scheduling algorithm.+"""++import json+import os+from pathlib import Path+import shlex+import subprocess+import sys+import tempfile+import time+import unittest+++ROOT = Path(__file__).resolve().parents[1]++# One locked record makes overlapping fake compiler processes observable without+# timing assertions. Both the C and C++ paths use this executable.+COMPILER = r'''+import fcntl+import json+import os+from pathlib import Path+import sys+import time++source = Path(sys.argv[sys.argv.index('-c') + 1])+output = Path(sys.argv[sys.argv.index('-o') + 1])+state_path = Path(os.environ['QUEUE_STATE'])+def record(starting):+ with state_path.open('r+') as stream:+ fcntl.flock(stream, fcntl.LOCK_EX)+ state = json.load(stream)+ state['active'] += 1 if starting else -1+ state['peak'] = max(state['peak'], state['active'])+ if starting:+ state['started'].append(source.name)+ stream.seek(0)+ json.dump(state, stream)+ stream.truncate()++record(True)+failed = source.name == os.environ.get('FAIL_SOURCE')+try:+ # A failure can finish before the parent reaches wait; successful compilers+ # stay alive briefly so the concurrency bound is meaningfully exercised.+ if not failed:+ time.sleep(0.08)+ output.write_text('fake object\n')+finally:+ record(False)+sys.exit(17 if failed else 0)+'''+++class BuildAppQueueTests(unittest.TestCase):+ @classmethod+ def setUpClass(cls):+ script = (ROOT / 'scripts/build-app.sh').read_text()+ start = script.index('kit_objects=()\n')+ end = script.index('rm -f "$obj/libps5lm_kit.a"', start)+ cls.queue = script[start:end]++ def run_queue(self, jobs='2', fail=None, cached=(), count=5, host_cpus=2):+ with tempfile.TemporaryDirectory(prefix='ps5lm-build-queue-') as directory:+ root = Path(directory)+ kit = root / 'kit'+ obj = root / 'obj'+ (kit / 'src').mkdir(parents=True)+ (obj / 'kit').mkdir(parents=True)+ (root / 'ps5/app').mkdir(parents=True)+ compiler = root / 'ps5/app/ps5cc'+ compiler.write_text(f'#!{sys.executable}\n' + COMPILER)+ compiler.chmod(0o755)+ state_path = root / 'state.json'+ state_path.write_text(json.dumps({'active': 0, 'peak': 0, 'started': []}))+ sources = []+ for index in range(count):+ extension = 'c' if index % 2 else 'cpp'+ source = kit / 'src' / f'{index}.{extension}'+ source.write_text('// fake input\n')+ sources.append(source)+ if index in cached:+ output = obj / 'kit' / (source.name + '.o')+ output.write_text('cached object\n')+ # Avoid depending on filesystem timestamp granularity.+ os.utime(output, (time.time() + 10, time.time() + 10))++ setup = '\n'.join([+ 'set -euo pipefail',+ f'root={shlex.quote(str(root))}',+ f'kit={shlex.quote(str(kit))}',+ f'obj={shlex.quote(str(obj))}',+ 'kitflags=()',+ 'kit_sources=(' + ' '.join(shlex.quote(str(p)) for p in sources) + ')',+ 'cxx() { "$root/ps5/app/ps5cc" "$@"; }',+ # Keep the default deterministic, independent of runner size.+ f'nproc() {{ printf "{host_cpus}\\n"; }}',+ ])+ env = os.environ.copy()+ env.pop('CMAKE_BUILD_PARALLEL_LEVEL', None)+ env.pop('FAIL_SOURCE', None)+ if jobs is not None:+ env['CMAKE_BUILD_PARALLEL_LEVEL'] = jobs+ if fail is not None:+ env['FAIL_SOURCE'] = sources[fail].name+ env['QUEUE_STATE'] = str(state_path)+ completed = subprocess.run(+ ['bash', '-c', setup + '\n' + self.queue],+ env=env, capture_output=True, text=True, timeout=10,+ )+ state = json.loads(state_path.read_text())+ objects = {p.name: p.read_text() for p in (obj / 'kit').glob('*.o')}+ return completed, state, objects++ def test_success_respects_job_limit(self):+ for jobs in ('1', '2', '3'):+ with self.subTest(jobs=jobs):+ result, state, objects = self.run_queue(jobs=jobs)+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertEqual(len(state['started']), 5)+ self.assertEqual(state['active'], 0)+ self.assertGreaterEqual(state['peak'], 1)+ self.assertLessEqual(state['peak'], int(jobs))+ self.assertEqual(len(objects), 5)++ def test_fast_compiler_failures_are_not_lost(self):+ # The old nproc/bare-wait loop loses failures on a many-core host;+ # CMAKE_BUILD_PARALLEL_LEVEL must take precedence over that host size.+ for position in (0, 2, 4):+ with self.subTest(position=position):+ result, state, _ = self.run_queue(fail=position, host_cpus=64)+ self.assertEqual(result.returncode, 17, result.stderr)+ self.assertEqual(state['active'], 0)+ self.assertLessEqual(state['peak'], 2)++ def test_cached_objects_skip_compilation(self):+ result, state, objects = self.run_queue(cached=(0, 2, 4))+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertEqual(sorted(state['started']), ['1.c', '3.c'])+ self.assertEqual(state['active'], 0)+ for index in (0, 2, 4):+ self.assertEqual(objects[f'{index}.cpp.o'], 'cached object\n')++ def test_all_cached_or_empty_sources_need_no_wait(self):+ for count, cached in ((0, ()), (5, range(5))):+ with self.subTest(count=count):+ result, state, _ = self.run_queue(count=count, cached=cached)+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertEqual(state['started'], [])+ self.assertEqual(state['active'], 0)++ def test_invalid_job_limits_fail_before_compiling(self):+ for jobs in ('0', '-1', '01', 'two'):+ with self.subTest(jobs=jobs):+ result, state, _ = self.run_queue(jobs=jobs)+ self.assertEqual(result.returncode, 2, result.stderr)+ self.assertIn('invalid job limit', result.stderr)+ self.assertEqual(state['started'], [])++ def test_default_uses_nproc(self):+ result, state, _ = self.run_queue(jobs=None)+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertLessEqual(state['peak'], 2)+ self.assertEqual(len(state['started']), 5)+++class NativeBuildPreflightTests(unittest.TestCase):+ def test_early_failure_removes_previous_success_report(self):+ cases = (+ ({'BUILD_JOBS': '0'}, 'BUILD_JOBS must be a positive integer'),+ ({'BUILD_JOBS': '2', 'LLVM_CONFIG': 'ps5lm-nonexistent-llvm-config-for-test'},+ 'LLVM_CONFIG'),+ )+ for overrides, message in cases:+ with self.subTest(overrides=overrides):+ with tempfile.TemporaryDirectory(prefix='ps5lm-native-preflight-') as directory:+ root = Path(directory)+ (root / 'scripts').mkdir()+ script = root / 'scripts/test-native-build.sh'+ script.write_text((ROOT / 'scripts/test-native-build.sh').read_text())+ report = root / 'build/native-ci/result.json'+ report.parent.mkdir(parents=True)+ report.write_text('{"status": "passed"}\n')+ env = os.environ.copy()+ env.update(overrides)+ result = subprocess.run(+ ['bash', str(script)], env=env,+ capture_output=True, text=True, timeout=5,+ )+ self.assertEqual(result.returncode, 2, result.stderr)+ self.assertIn(message, result.stderr)+ self.assertFalse(report.exists())+ # These failures must occur before any dependency bootstrap.+ self.assertFalse((root / '.deps').exists())+++if __name__ == '__main__':+ unittest.main()