Play your Steam library's Windows games on a jailbroken PS5.
C++
Download the runtime kit on the first start
Leonardo Lobatoe50d52b762db
diff --git a/.github/ISSUE_TEMPLATE/install-problem.yml b/.github/ISSUE_TEMPLATE/install-problem.ymlindex bc14060..e60c916 100644--- a/.github/ISSUE_TEMPLATE/install-problem.yml+++ b/.github/ISSUE_TEMPLATE/install-problem.yml@@ -19,7 +19,8 @@ body:attributes:label: Where it failsoptions:- - Copying Playden with console_title_deploy.py+ - Copying Playden to the console (FTP or console_title_deploy.py)+ - Downloading the compatibility kit in Playden- Copying the kit with console_kit_upload.py- Playden doesn't start, or stays on the splash screen- Playden starts and shows an error (runtime, kit or storage)diff --git a/README.md b/README.mdindex 5cf0b8a..c7515b4 100644--- a/README.md+++ b/README.md@@ -92,7 +92,8 @@ you sign in:(port 2120), and optionally [ps5-web-file-manager](https://github.com/owendswang/ps5-web-file-manager)(port 8888).- **Windows games:** a Steam account with games, or game folders you already have (see [Import Windows games](#import-windows-games)).-- **A computer on the same network as the PS5**, to copy Playden and the kit to it. The kit upload needs Python 3.+- **A computer on the same network as the PS5**, to copy Playden to it. Python 3 is needed only for the optional+ install tools.- **To build it yourself (optional):** a Mac with Apple Silicon. Linux should also work, but we have not tested ityet. See [Build from source](#build-from-source).@@ -111,13 +112,9 @@ Other versions can work, but these are the ones we know.## Install-Download both files from the latest [release](../../releases):--- `playden-ps5-0.6.0.zip`: the Playden app (`FAKE77031`) and the console tools used below.-- `playden-kit-0.12.zip`: the runtime kit 0.12, with its source list and license texts.--Unzip both files next to each other. Installing takes two copies over the network: the app folder, then the-runtime kit folder. After that, Playden updates itself (see [Update Playden](#update-playden)).+Download `playden-ps5-<version>.zip` from the latest [release](../../releases): the Playden app (`FAKE77031`) and+the console tools used below. Unzip it. Installing takes one copy over the network, the app folder. Playden+downloads its runtime kit on its first start, and after that it updates itself (see [Update Playden](#update-playden)).Like other PS5 homebrew, Playden is a folder in `/data/homebrew/`, andShadowMountPlus adds its tile to the Home screen.@@ -147,8 +144,8 @@ ShadowMountPlus adds its tile to the Home screen.The app folder keeps empty `playden-data` and `logs` folders, and a `MOVED.txt` in each old place says wherethe data went.2. Wait about 10 seconds. ShadowMountPlus adds the Playden tile to the Home screen.-3. Install the runtime kit (next section).-4. Start Playden and sign in to Steam.+3. Start Playden. It offers to download the runtime kit (next section); choose Download.+4. Sign in to Steam while the kit downloads.### Update Playden@@ -177,20 +174,22 @@ You can also update by hand: copy the new `FAKE77031` folder over the old one, o`--install`. The command copies only the changed files, keeps a backup of what it replaced, and puts those threefolders' modes right if they changed.-### Install the runtime kit+### The runtime kit-This release uses **kit 0.12**. Upload the kit folder from `playden-kit-0.12.zip` with the upload tool. This step-needs Python 3. A plain FTP copy does not work for the kit: Playden checks each kit file's permissions against the-kit's manifest, and a plain copy doesn't set them. Run it from the `playden-ps5-<version>` folder:+The runtime kit (the "compatibility kit" in Playden) is what runs Windows games: Wine, Mesa and DXVK. Playden+downloads the kit its version was made for from this repository's releases, on its first start and whenever the+installed kit isn't that one. It installs in the background, shows on the Queue screen, and games can be prepared+as soon as it is in place. If you choose Later, or the download fails, Settings › About › Compatibility kit starts+it again. Games pick up a new kit when you press Play.++You can also install the kit by hand, with Python 3: download `playden-kit-<version>.zip` from the release, unzip it+next to `playden-ps5-<version>`, and run this from the `playden-ps5-<version>` folder. A plain FTP copy does not+work for the kit, because Playden checks each kit file's permissions against the kit's manifest.```sh-python3 tools/console_kit_upload.py ../playden-kit-0.12 --host <PS5 IP> --replace+python3 tools/console_kit_upload.py ../playden-kit-<version> --host <PS5 IP> --replace```-The upload checks every file before it switches to the new kit, and creates `/data/playden/kits` on a console that-doesn't have it yet. Playden accepts only the kit version it was tested with. Games pick up a new kit when you press-Play.-The kit is built on a **Linux x86-64 machine or VM** from prospero-win, Wine, Mesa and DXVK. To build it yourself,see [Build the kit](CONTRIBUTING.md#build-the-kit). The scripts are a record of how we built it, not a turnkey buildyet, so most people should use the release download.diff --git a/src/homebrew/controller.cpp b/src/homebrew/controller.cppindex 3e6a51d..5582753 100644--- a/src/homebrew/controller.cpp+++ b/src/homebrew/controller.cpp@@ -2131,17 +2131,39 @@ void Controller::TickSelfUpdate() {else if (request == "restart") self_update_->RequestRestart();else if (request == "rollback") self_update_->RequestRollback();else if (request == "acknowledge") self_update_->AcknowledgeFinished();+ else if (request == "kit-check") self_update_->CheckKit();+ else if (request == "kit-install") self_update_->InstallKit();+ else if (request == "kit-cancel") self_update_->Cancel();+ }+ // The runtime started without a usable kit (a first run), or with another kit than the one this+ // version was made for: the running version's kit is looked for once, after the startup update+ // check, before sign-in.+ if (!kit_check_asked_ && runtime_ && !self_update_->Busy()) {+ const auto runtime = runtime_->Shared();+ const bool other_kit = runtime->kit && runtime->kit->manifest_sha256 != runtime->kit->expected_manifest_sha256;+ if (runtime->status == RuntimeHostStatus::kReady && (!runtime->kit_available || other_kit)) {+ kit_check_asked_ = true;+ self_update_->CheckKit();+ }}const auto snapshot = self_update_->Snapshot();+ if (snapshot.kit.phase == update::KitInstallPhase::kInstalled && !kit_reload_asked_ && runtime_) {+ kit_reload_asked_ = true;+ runtime_->ReloadKit();+ }const auto now = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now().time_since_epoch()).count();- const bool downloading = snapshot.phase == update::UpdatePhase::kDownloading;+ const bool kit_downloading = snapshot.kit.phase == update::KitInstallPhase::kDownloading;+ const bool downloading = snapshot.phase == update::UpdatePhase::kDownloading || kit_downloading;+ const std::string rate_part = kit_downloading ? std::string("kit-only") : snapshot.part;+ const std::uint64_t rate_done = kit_downloading ? snapshot.kit.done : snapshot.done;+ const std::uint64_t rate_total = kit_downloading ? snapshot.kit.total : snapshot.total;if (downloading) {- if (self_update_rate_part_ != snapshot.part) {+ if (self_update_rate_part_ != rate_part) {self_update_rate_.Reset();- self_update_rate_part_ = snapshot.part;+ self_update_rate_part_ = rate_part;}- self_update_rate_.Record(snapshot.done, now);+ self_update_rate_.Record(rate_done, now);} else if (!self_update_rate_part_.empty()) {self_update_rate_.Reset();self_update_rate_part_.clear();@@ -2160,7 +2182,7 @@ void Controller::TickSelfUpdate() {live.done = snapshot.done;live.total = snapshot.total;if (downloading) {- const auto estimate = self_update_rate_.Estimate(now, snapshot.done, snapshot.total);+ const auto estimate = self_update_rate_.Estimate(now, rate_done, rate_total);live.bytes_per_second = estimate.bytes_per_second;live.eta_seconds = estimate.eta_seconds;}@@ -2176,6 +2198,12 @@ void Controller::TickSelfUpdate() {live.quiet = snapshot.quiet;live.finished_message = snapshot.finished_message;live.finished_failed = snapshot.finished_failed;+ live.kit_phase = update::KitInstallPhaseName(snapshot.kit.phase);+ live.kit_replaces = snapshot.kit.replaces;+ live.kit_download_bytes = snapshot.kit.download_bytes;+ live.kit_done = snapshot.kit.done;+ live.kit_total = snapshot.kit.total;+ live.kit_error = snapshot.kit.error;model_.UpdateSelfUpdate(live);}diff --git a/src/homebrew/controller.hpp b/src/homebrew/controller.hppindex 8e32aa8..e3b1422 100644--- a/src/homebrew/controller.hpp+++ b/src/homebrew/controller.hpp@@ -120,6 +120,8 @@ class Controller {std::uint64_t self_update_revision_ = ~0ull;steam::DownloadRate self_update_rate_;std::string self_update_rate_part_;+ bool kit_check_asked_ = false; // the runtime started without a kit: CheckKit asked once+ bool kit_reload_asked_ = false; // the kit-only install finished: the runtime asked to admit itvoid TickCloud();void TickRuntime();Simulation& model_;diff --git a/src/homebrew/ps5.cpp b/src/homebrew/ps5.cppindex 8b5cdef..e1b4d4d 100644--- a/src/homebrew/ps5.cpp+++ b/src/homebrew/ps5.cpp@@ -379,6 +379,9 @@ int main() {update_config.installed_root = std::string("/data/homebrew/") + PLAYDEN_PS5_TITLE_ID;update_config.work_root = "/data/playden/self-update";update_config.kits_root = "/data/playden/kits";+ // The kit this build was made for: downloaded at the first start (or over a kit it can't use).+ update_config.kit_id = playden::runtime::kDevelopmentKitId;+ update_config.kit_manifest_sha256 = playden::runtime::kExpectedKitManifestSha256;update_config.appmeta_root = std::string("/user/appmeta/") + PLAYDEN_PS5_TITLE_ID;update_config.registered_app_root = std::string("/user/app/") + PLAYDEN_PS5_TITLE_ID;update_config.layout_journal = "/data/playden/runtime-state/layout-0.6.json";diff --git a/src/homebrew/runtime_host.cpp b/src/homebrew/runtime_host.cppindex 319905d..eefefc2 100644--- a/src/homebrew/runtime_host.cpp+++ b/src/homebrew/runtime_host.cpp@@ -620,6 +620,15 @@ void RuntimeHost::MeasureSaves() {wake_.notify_all();}+void RuntimeHost::ReloadKit() {+ {+ std::lock_guard<std::mutex> lock(mutex_);+ if (stopping_) return;+ kit_reload_requested_ = true;+ }+ wake_.notify_all();+}+void RuntimeHost::Maintain(std::uint32_t app_id, RuntimeMaintenance action) {{std::lock_guard<std::mutex> lock(mutex_);@@ -3681,6 +3690,10 @@ std::string RuntimeHost::DeleteKept(Composition& c, std::uint32_t app) {}void RuntimeHost::Run() {+ while (RunServices()) Log("kit reload: starting the runtime services again");+}++bool RuntimeHost::RunServices() {const auto hold = [this] { return config_.hold_effective_uid ? config_.hold_effective_uid() : nullptr; };auto startup_hold = hold();auto unavailable = [this](std::string reason) {@@ -3903,6 +3916,7 @@ void RuntimeHost::Run() {PublishPrepared(composition.get());}startup_hold.reset();+ bool reload = false;for (;;) {std::vector<std::uint32_t> plays;std::vector<std::pair<std::uint32_t, bool>> removals;@@ -3929,11 +3943,19 @@ void RuntimeHost::Run() {(!kept_delete_requests_.empty() && !snapshot_.handoff_ready) ||!scan_requests_.empty() || !prepared_dismissals_.empty() || prepared_clear_requested_ ||(!maintenance_requests_.empty() && !snapshot_.handoff_ready) ||- (!relocation_requests_.empty() && !snapshot_.handoff_ready);+ (!relocation_requests_.empty() && !snapshot_.handoff_ready) ||+ (kit_reload_requested_ && !snapshot_.handoff_ready && !launch_poll_);};if (launch_poll_) wake_.wait_for(lock, kLaunchPollInterval, ready);else wake_.wait(lock, [&] { return ready() || launch_poll_; });if (stopping_) break;+ // Between batches, with no launch to hand off or follow: the services hold no work, and PS5+ // titles run their own copy of the runtime, so the kit's folder can change under them.+ if (kit_reload_requested_ && !snapshot_.handoff_ready && !launch_poll_) {+ kit_reload_requested_ = false;+ reload = true;+ break;+ }plays.swap(play_requests_);// Removal changes install records that Handoff reads, like preparation.if (!snapshot_.handoff_ready) removals.swap(remove_requests_);@@ -4175,7 +4197,15 @@ void RuntimeHost::Run() {}ShareNumbers(nullptr);composition.reset();- Publish([](RuntimeSnapshot& s) { s.status = RuntimeHostStatus::kStopped; });+ Publish([&](RuntimeSnapshot& s) {+ s.status = reload ? RuntimeHostStatus::kStarting : RuntimeHostStatus::kStopped;+ // Refused for want of a kit: the services starting again with one answer anew.+ if (reload) {+ s.refusals.clear();+ s.play_refusals.clear();+ }+ });+ return reload;}} // namespace playden::homebrewdiff --git a/src/homebrew/runtime_host.hpp b/src/homebrew/runtime_host.hppindex 90cd19d..5622b91 100644--- a/src/homebrew/runtime_host.hpp+++ b/src/homebrew/runtime_host.hpp@@ -466,6 +466,10 @@ class RuntimeHost {void ScanPrograms(std::uint32_t runtime_number, std::string install_path);// Measures each game's Windows user folders (where saves of games without Steam Cloud live).void MeasureSaves();+ // A kit was put in place while Playden runs (the first-run kit download): the worker stops its+ // services and starts them again to admit it, publishing kStarting in between, once no launch is+ // being handed off or followed. Queued requests wait for it.+ void ReloadKit();// Queues maintenance of a prepared game. Each action is refused while the game's// title runs or its state is unknown.void Maintain(std::uint32_t runtime_number, RuntimeMaintenance action);@@ -500,6 +504,8 @@ class RuntimeHost {struct Composition;static void* Entry(void* self);void Run();+ // Starts the services and runs requests until Shutdown (false) or ReloadKit (true).+ bool RunServices();void Publish(const std::function<void(RuntimeSnapshot&)>& change);// With `only`, that install's title is checked again and the others keep their last// check (about two seconds per title on the console).@@ -689,6 +695,7 @@ class RuntimeHost {bool launch_poll_ = false;bool cleanup_requested_ = false;bool measure_requested_ = false;+ bool kit_reload_requested_ = false; // guarded by mutex_bool prepares_held_ = false;std::vector<PreparedGame> prepared_dismissals_; // guarded by mutex_bool prepared_clear_requested_ = false; // guarded by mutex_diff --git a/src/homebrew/simulation.cpp b/src/homebrew/simulation.cppindex 0936c05..54fc651 100644--- a/src/homebrew/simulation.cpp+++ b/src/homebrew/simulation.cpp@@ -495,6 +495,7 @@ std::string action_label(const std::string& key) {{"prep-clear-completed", "Clear completed"}, {"storage-close", "Close"}, {"collection-new", "New collection"}, {"collection-manage", "Manage collections"},{"about-version", "Version"}, {"self-update-install", "Download and install"},{"self-update-restart", "Restart Playden"}, {"self-update-cancel", "Cancel update"},+ {"kit-install", "Download"}, {"kit-cancel", "Cancel download"},{"self-update-back-confirm", "Go back"}, {"release-notes-ok", "OK"},{"about-build", "Build"}, {"about-framework", "Framework"}, {"about-platform", "Platform"},{"about-kit", "Compatibility kit"}, {"about-kit-source", "Built from"},@@ -1600,6 +1601,8 @@ void Simulation::UpdateSelfUpdate(const LiveSelfUpdate& update) {before.bytes_per_second != update.bytes_per_second || before.previous != update.previous ||before.quiet != update.quiet || before.current_notes_url != update.current_notes_url ||before.whats_new != update.whats_new ||+ before.kit_phase != update.kit_phase || before.kit_done != update.kit_done ||+ before.kit_total != update.kit_total || before.kit_error != update.kit_error ||!update.finished_message.empty();if (!changed) {show_update_prompt();@@ -1633,6 +1636,20 @@ void Simulation::UpdateSelfUpdate(const LiveSelfUpdate& update) {push_notice({"self-update-check", "Playden is up to date", "", false, kLiveNoticeSeconds, false,"Version " + update.current});}+ // The kit-only install. An update on offer or under way brings the kit itself, so it asks first.+ const bool update_brings_kit = update.phase == "available" || update.phase == "downloading" ||+ update.phase == "unpacking" || update.phase == "staged";+ if (update.kit_phase == "checking") kit_prompted_ = false;+ if (update.kit_phase == "available" && !kit_prompted_ && !update_brings_kit && pending_update_prompt_.empty()) {+ kit_prompted_ = true;+ pending_update_prompt_ = "kit";+ } else if (update.kit_phase == "failed" && before.kit_phase != "failed" &&+ (before.kit_phase == "downloading" || before.kit_phase == "unpacking")) {+ show_alert("Compatibility kit download stopped", update.kit_error);+ } else if (update.kit_phase == "installed" && before.kit_phase != "installed") {+ push_notice({"kit-installed", "Compatibility kit installed", "", false, kLiveNoticeSeconds, false,+ "Games can be prepared and played now"});+ }sync_self_update_job();show_update_prompt();rebuild_view();@@ -1647,11 +1664,28 @@ std::vector<std::string> Simulation::TakeSelfUpdateRequests() {// The update's prompts wait, like alerts, until no other dialog, sheet or keyboard is open.void Simulation::show_update_prompt() {const auto route = state_.navigation.route;+ // The kit's offer shows before sign-in too (Connect Steam and the sign-in screens, not while the+ // saved account loads): it downloads while the player signs in.+ const bool kit = pending_update_prompt_ == "kit";+ const bool before_sign_in = (route == Route::Startup && !state_.startup_loading) || route == Route::SignIn;if (pending_update_prompt_.empty() || !state_.navigation.modals.empty() || state_.editor.active ||- route == Route::Startup || route == Route::SignIn || route == Route::Session)+ route == Route::Session || (route == Route::Startup && !(kit && before_sign_in)) ||+ (route == Route::SignIn && !kit))return;const auto& update = state_.self_update;- if (pending_update_prompt_ == "available" && update.phase == "available") {+ if (kit && update.kit_phase == "available") {+ ModalEntry prompt{"confirmation", update.kit_replaces ? "Replace the compatibility kit?" : "Download the compatibility kit?",+ state_.navigation.focus_key, {"kit-install", "ready-later"}, 0, false};+ const std::string size = update.kit_download_bytes ? "The download is " + format_bytes(update.kit_download_bytes) + ". "+ : std::string();+ prompt.body = (update.kit_replaces+ ? "The installed compatibility kit isn't the one Playden " + update.current ++ " was made for, so games may not run as this version expects. Playden can put the "+ "right one in its place. "+ : std::string("Playden needs its compatibility kit to prepare and play Windows games. ")) ++ size + "You can sign in and keep using Playden while it downloads.";+ open_modal(std::move(prompt));+ } else if (pending_update_prompt_ == "available" && update.phase == "available") {ModalEntry prompt{"confirmation", "Playden " + update.available + " is available", state_.navigation.focus_key,{"self-update-install", "ready-later"}, 0, false};prompt.body = (update.notes.empty() ? std::string() : update.notes + "\n\n") + "The download is " +@@ -1678,6 +1712,24 @@ void Simulation::sync_self_update_job() {[](const Job& job) { return job.kind == "self-update"; }),state_.jobs.end());const auto& update = state_.self_update;+ if (update.kit_phase == "downloading" || update.kit_phase == "unpacking") {+ Job job;+ job.id = "kit-install";+ job.kind = "self-update";+ job.queue_order = -1;+ job.title = "Compatibility kit";+ const bool downloading = update.kit_phase == "downloading";+ job.state = downloading ? JobState::Downloading : JobState::Verifying;+ job.stage_text = downloading ? "Downloading the compatibility kit" : "Unpacking the compatibility kit";+ job.state_text = downloading ? "Downloading" : "Unpacking";+ if (downloading) {+ job.bytes_per_second = update.bytes_per_second;+ job.eta_seconds = update.eta_seconds;+ }+ job.completed_bytes = update.kit_done;+ job.total_bytes = update.kit_total;+ state_.jobs.push_back(std::move(job));+ }if (update.phase != "downloading" && update.phase != "unpacking" && update.phase != "staged") return;const bool kit = update.part == "kit";Job job;@@ -5068,6 +5120,32 @@ void Simulation::rebuild_view() {: "Version " + update.current;}}+ // The kit-only install, while it has something to offer or say.+ if (live()) {+ const auto& update = state_.self_update;+ const auto& phase = update.kit_phase;+ // An update on offer or under way brings the kit itself: its own row says so.+ const bool update_brings_kit = update.phase == "available" || update.phase == "downloading" ||+ update.phase == "unpacking" || update.phase == "staged";+ if ((phase == "available" && !update_brings_kit) || phase == "checking" || phase == "downloading" ||+ phase == "unpacking" || phase == "failed") {+ auto& kit = add("settings-kit", RowKind::Action, "Compatibility kit", "kit");+ if (phase == "available") {+ kit.value = "Download";+ kit.subtitle = update.kit_replaces ? "The installed kit isn't the one this version was made for"+ : "Needed to prepare and play games";+ } else if (phase == "checking") {+ kit.value = "Checking…";+ kit.subtitle = "Looking for this version's kit on GitHub";+ } else if (phase == "failed") {+ kit.value = "Try again";+ kit.subtitle = update.kit_error;+ } else {+ kit.value = update.kit_total ? std::to_string(update.kit_done * 100 / update.kit_total) + "%" : "Working…";+ kit.subtitle = phase == "downloading" ? "Downloading the compatibility kit" : "Unpacking the compatibility kit";+ }+ }+ }if (kOfferGoBack && live() && !state_.self_update.previous.empty() &&state_.self_update.phase != "downloading" && state_.self_update.phase != "unpacking") {auto& back = add("settings-update-back", RowKind::Action, "Go back to Playden " + state_.self_update.previous,@@ -6218,6 +6296,8 @@ void Simulation::activate() {}} else if (action_name == "install-estimate-retry") {state_.notices.push_back({"estimate", "Fixture estimate is still unavailable", "install-estimate-retry", true, 0});+ } else if (action_name == "job-action" && selected.key == "job-kit-install") {+ open_modal({"job-actions", "Compatibility kit", selected.key, {"kit-cancel", "cancel"}, 0, false});} else if (action_name == "job-action" && selected.key == "job-playden-update") {const auto& phase = state_.self_update.phase;std::vector<std::string> actions;@@ -7305,6 +7385,16 @@ void Simulation::activate() {} else if (phase != "checking") self_update_requests_.push_back("check");}else if (live() && action_name == "self-update-install") { close_modal(); self_update_requests_.push_back("install"); }+ else if (live() && action_name == "kit") {+ const auto& phase = state_.self_update.kit_phase;+ if (phase == "available") { pending_update_prompt_ = "kit"; show_update_prompt(); }+ else if (phase == "downloading" || phase == "unpacking") {+ state_.navigation.route = Route::Downloads;+ state_.navigation.focus_key = "job-kit-install";+ } else if (phase == "failed" || phase == "idle") self_update_requests_.push_back("kit-check");+ }+ else if (live() && action_name == "kit-install") { close_modal(); self_update_requests_.push_back("kit-install"); }+ else if (live() && action_name == "kit-cancel") { close_modal(); self_update_requests_.push_back("kit-cancel"); }else if (live() && action_name == "self-update-cancel") { close_modal(); self_update_requests_.push_back("cancel"); }else if (live() && action_name == "self-update-restart") { close_modal(); request_self_update_restart(); }else if (live() && action_name == "self-update-notes") open_release_notes(false);diff --git a/src/homebrew/simulation.hpp b/src/homebrew/simulation.hppindex aad62ba..6b28867 100644--- a/src/homebrew/simulation.hpp+++ b/src/homebrew/simulation.hpp@@ -120,6 +120,14 @@ struct LiveSelfUpdate {bool quiet = false; // the available version is one the player went back from: no dialogstd::string finished_message; // the previous run's update result, shown oncebool finished_failed = false;+ // The kit-only install (first run, or a kit that isn't this version's): idle, checking, not-needed,+ // unavailable, available, downloading, unpacking, installed, failed.+ std::string kit_phase = "idle";+ bool kit_replaces = false; // another kit is installed: this one replaces it+ std::uint64_t kit_download_bytes = 0;+ std::uint64_t kit_done = 0;+ std::uint64_t kit_total = 0;+ std::string kit_error;};// Live: what Game settings needs to choose a game's program, projected from the runtime host.// A program a Steam launch option names, as a path in the game folder.@@ -880,6 +888,7 @@ private:std::vector<std::string> self_update_requests_;std::string self_update_prompted_; // "available 0.3.0" or "staged 0.3.0": each prompt shows oncestd::string pending_update_prompt_; // a prompt waiting for the open dialog to close+ bool kit_prompted_ = false; // the kit was offered since its last check (a cancel doesn't offer it again)void show_update_prompt();void sync_self_update_job();void request_self_update_restart(const std::string& request = "restart");diff --git a/src/update/release.cpp b/src/update/release.cppindex 246b4d1..ffb1b50 100644--- a/src/update/release.cpp+++ b/src/update/release.cpp@@ -83,9 +83,9 @@ const ReleaseAsset* Release::Find(std::string_view name) const {return nullptr;}-std::optional<Release> ParseRelease(std::string_view json, std::string* error) {- const auto doc = ParseObject(json, error);- if (!doc) return std::nullopt;+namespace {++std::optional<Release> ReleaseFrom(const Json* doc, std::string* error) {Release release;const auto* tag = Text(*doc, "tag_name");const auto version = tag ? ParseVersion(*tag) : std::nullopt;@@ -119,6 +119,30 @@ std::optional<Release> ParseRelease(std::string_view json, std::string* error) {return release;}+} // namespace++std::optional<Release> ParseRelease(std::string_view json, std::string* error) {+ const auto doc = ParseObject(json, error);+ if (!doc) return std::nullopt;+ return ReleaseFrom(&*doc, error);+}++std::optional<std::vector<Release>> ParseReleases(std::string_view json, std::string* error) {+ steam::json::Limits limits;+ limits.max_bytes = 4 << 20;+ limits.max_values = 1 << 18;+ const auto doc = steam::json::Parse(json, limits);+ if (!doc || !doc->is_array()) {+ if (error) *error = "not a JSON array";+ return std::nullopt;+ }+ std::vector<Release> releases;+ for (const auto& item : doc->array())+ if (item.is_object())+ if (auto release = ReleaseFrom(&item, nullptr)) releases.push_back(std::move(*release)); // a tag that isn't a version is skipped+ return releases;+}+std::optional<UpdateManifest> ParseUpdateManifest(std::string_view json, std::string* error) {auto refuse = [&](const char* reason) -> std::optional<UpdateManifest> {if (error) *error = reason;diff --git a/src/update/release.h b/src/update/release.hindex 7885dee..6c59f52 100644--- a/src/update/release.h+++ b/src/update/release.h@@ -33,6 +33,8 @@ struct Release {// The JSON of GET /repos/<owner>/<repo>/releases/latest. The tag must be a version ("v0.3.0").std::optional<Release> ParseRelease(std::string_view json, std::string* error);+// The JSON of GET /repos/<owner>/<repo>/releases: newest first, releases whose tag isn't a version left out.+std::optional<std::vector<Release>> ParseReleases(std::string_view json, std::string* error);struct KitPart {std::string kit_id; // stays the same across revisions ("hl2-fw1360-dev1")diff --git a/src/update/update_service.cpp b/src/update/update_service.cppindex 542a3cc..3e359a0 100644--- a/src/update/update_service.cpp+++ b/src/update/update_service.cpp@@ -79,6 +79,21 @@ const char* UpdatePhaseName(UpdatePhase phase) {return "?";}+const char* KitInstallPhaseName(KitInstallPhase phase) {+ switch (phase) {+ case KitInstallPhase::kIdle: return "idle";+ case KitInstallPhase::kChecking: return "checking";+ case KitInstallPhase::kNotNeeded: return "not-needed";+ case KitInstallPhase::kUnavailable: return "unavailable";+ case KitInstallPhase::kAvailable: return "available";+ case KitInstallPhase::kDownloading: return "downloading";+ case KitInstallPhase::kUnpacking: return "unpacking";+ case KitInstallPhase::kInstalled: return "installed";+ case KitInstallPhase::kFailed: return "failed";+ }+ return "?";+}+UpdateService::UpdateService(UpdateConfig config) : config_(std::move(config)), shared_(std::make_shared<Shared>()) {if (!config_.as_root) config_.as_root = [](const std::function<bool()>& work) { return work(); };if (!config_.now) config_.now = [] { return static_cast<std::int64_t>(time(nullptr)); };@@ -169,15 +184,19 @@ bool UpdateService::PrepareWorkRoot() {return ok;}-// A kit left by going back: its files are the app user's, removed by CleanUpLeftovers, but its folder-// sits in the root-owned kits folder. The empty folder goes as root here, at startup, before the-// runtime and the UI run (root is process-wide).-void UpdateService::RemoveEmptyReplacedKits() {+// A kit left by going back, or replaced by the kit-only install (perhaps uploaded over FTP, so+// root's files): it goes as root here, at startup, before the runtime and the UI run (root is+// process-wide). Once, after a kit was replaced; a large kit takes a few seconds.+void UpdateService::RemoveReplacedKits() {for (const auto& name : ListFolder(config_.kits_root)) {if (name.compare(0, 10, ".replaced-") != 0) continue;const std::string path = config_.kits_root + "/" + name;- if (!IsFolder(path) || !ListFolder(path).empty()) continue;- if (rmdir(path.c_str()) != 0) config_.as_root([&] { return rmdir(path.c_str()) == 0; });+ if (!IsFolder(path)) continue;+ if (ListFolder(path).empty() && rmdir(path.c_str()) == 0) continue;+ config_.as_root([&] {+ RemoveTree(path);+ return !Exists(path);+ });if (Exists(path)) Log("could not remove " + path);}}@@ -333,7 +352,11 @@ DownloadOptions UpdateService::Downloads(const std::string& part) {options.cancel = &shared_->cancel;options.wait_before_retry = config_.wait_before_retry;options.log = [this](const std::string& line) { Log(line); };- if (!part.empty())+ if (part == "kit-only")+ options.progress = [this](std::uint64_t done, std::uint64_t total) {+ Set([&](UpdateSnapshot& s) { s.kit.done = done; s.kit.total = total; });+ };+ else if (!part.empty())options.progress = [this, part](std::uint64_t done, std::uint64_t total) {Set([&](UpdateSnapshot& s) { s.part = part; s.done = done; s.total = total; });};@@ -354,7 +377,7 @@ std::string UpdateService::ReadyKit(const KitPart& kit) const {void UpdateService::Start() {FindPrevious();LoadCurrentNotes();- RemoveEmptyReplacedKits();+ RemoveReplacedKits();std::string text;if (!ReadSmallFile(StatePath(), &text)) {CleanUpLeftovers(); // what a start closed before its cleanup finished left behind@@ -584,7 +607,7 @@ void UpdateService::DoInstall() {"free space");}if (kit_needed) {- if (!StageKit(release, manifest)) return;+ if (!StageKit(*kit, manifest.kit, false)) return;kit_folder = ReadyKit(manifest.kit);}if (!StageApp(release, manifest)) return;@@ -600,70 +623,92 @@ void UpdateService::DoInstall() {});}-bool UpdateService::StageKit(const Release& release, const UpdateManifest& manifest) {- const auto* asset = release.Find(manifest.kit.asset);- const std::string local_zip = config_.local_release_dir + "/" + asset->name;+bool UpdateService::StageKit(const ReleaseAsset& asset, const KitPart& kit, bool kit_only) {+ const std::string local_zip = config_.local_release_dir + "/" + asset.name;const bool local = !config_.local_release_dir.empty() && Exists(local_zip);- const std::string zip = local ? local_zip : config_.work_root + "/downloads/" + asset->name;- Set([&](UpdateSnapshot& s) {- s.phase = UpdatePhase::kDownloading;- s.part = "kit";- s.done = 0;- s.total = asset->size;- });+ const std::string zip = local ? local_zip : config_.work_root + "/downloads/" + asset.name;+ // The update reports in the snapshot's own fields, the kit-only install in snapshot.kit.+ const auto step = [&](bool unpacking, std::uint64_t total) {+ Set([&](UpdateSnapshot& s) {+ if (kit_only) {+ s.kit.phase = unpacking ? KitInstallPhase::kUnpacking : KitInstallPhase::kDownloading;+ s.kit.done = 0;+ s.kit.total = total;+ return;+ }+ s.phase = unpacking ? UpdatePhase::kUnpacking : UpdatePhase::kDownloading;+ s.part = "kit";+ s.done = 0;+ s.total = total;+ });+ };+ const auto cancelled = [&] {+ Set([&](UpdateSnapshot& s) {+ if (kit_only) s.kit.phase = KitInstallPhase::kAvailable;+ else { s.phase = UpdatePhase::kAvailable; s.part.clear(); }+ });+ };+ const auto fail = [&](const std::string& message, const std::string& detail) {+ if (kit_only) FailKit(message, detail);+ else Fail(message, detail);+ };+ step(false, asset.size);const auto download = local ? DownloadResult{true, false, {}}- : DownloadAsset(asset->url, asset->size, asset->sha256, zip, Downloads("kit"));+ : DownloadAsset(asset.url, asset.size, asset.sha256, zip, Downloads(kit_only ? "kit-only" : "kit"));if (!download.ok) {if (download.cancelled) {- Set([](UpdateSnapshot& s) { s.phase = UpdatePhase::kAvailable; s.part.clear(); });+ cancelled();return false;}- Fail("The kit download failed.", download.error);+ fail("The kit download failed.", download.error);return false;}- Set([&](UpdateSnapshot& s) {- s.phase = UpdatePhase::kUnpacking;- s.part = "kit";- s.done = 0;- s.total = 0;- });- const std::string staging = config_.kits_root + "/.staging-" + manifest.kit.kit_id + "-" + manifest.kit.revision;+ step(true, 0);+ const std::string staging = config_.kits_root + "/.staging-" + kit.kit_id + "-" + kit.revision;RemoveOwnedTree(staging);- config_.as_root([&] { return MakeFolders(staging) && chmod(staging.c_str(), 0777) == 0; });+ config_.as_root([&] {+ // A first start has no kits folder: made like the kit upload tool makes it.+ if (!IsFolder(config_.kits_root) && (!MakeFolders(config_.kits_root) || chmod(config_.kits_root.c_str(), 0755) != 0))+ return false;+ return MakeFolders(staging) && chmod(staging.c_str(), 0777) == 0;+ });if (!IsFolder(staging) || access(staging.c_str(), W_OK) != 0) {- Fail("Playden could not create the kit folder.", staging);+ fail("Playden could not create the kit folder.", staging);return false;}ZipExtractOptions unzip;- unzip.inner_prefix = manifest.kit.prefix;+ unzip.inner_prefix = kit.prefix;unzip.cancel = &shared_->cancel;unzip.fix_mode = [this](const std::string& path, unsigned mode) {return config_.as_root([&] { return chmod(path.c_str(), mode) == 0; });};unzip.log = [this](const std::string& line) { Log(line); };- unzip.progress = [this](std::uint64_t done, std::uint64_t total) {- Set([&](UpdateSnapshot& s) { s.done = done; s.total = total; });+ unzip.progress = [this, kit_only](std::uint64_t done, std::uint64_t total) {+ Set([&](UpdateSnapshot& s) {+ if (kit_only) { s.kit.done = done; s.kit.total = total; }+ else { s.done = done; s.total = total; }+ });};const auto unpacked = ExtractZip(zip, staging, unzip);if (!unpacked.ok) {RemoveOwnedTree(staging);if (unpacked.cancelled) {- Set([](UpdateSnapshot& s) { s.phase = UpdatePhase::kAvailable; s.part.clear(); });+ cancelled();return false;}- Fail("The kit could not be unpacked.", unpacked.error);+ fail("The kit could not be unpacked.", unpacked.error);return false;}Log("kit unpacked: " + std::to_string(unpacked.files) + " files, " + std::to_string(unpacked.rewritten) +" written again, " + std::to_string(unpacked.mode_fixed) + " root-owned given their mode");const auto sha = config_.verify_kit ? config_.verify_kit(staging) : std::nullopt;- if (!sha || *sha != manifest.kit.manifest_sha256) {+ if (!sha || *sha != kit.manifest_sha256) {RemoveOwnedTree(staging);- Fail("The new kit did not pass its check.", sha ? "manifest " + *sha : "not a complete kit");+ fail("The new kit did not pass its check.", sha ? "manifest " + *sha : "not a complete kit");return false;}// Ready next to the installed kit; it replaces it when Playden quits for the update.- const std::string final_folder = config_.kits_root + "/.next-" + manifest.kit.kit_id + "-" + manifest.kit.revision;+ const std::string final_folder = config_.kits_root + "/.next-" + kit.kit_id + "-" + kit.revision;const std::string replaced = final_folder + ".old";const bool moved = config_.as_root([&] {if (Exists(final_folder) && rename(final_folder.c_str(), replaced.c_str()) != 0) return false;@@ -672,13 +717,167 @@ bool UpdateService::StageKit(const Release& release, const UpdateManifest& manifRemoveOwnedTree(replaced);if (!moved) {RemoveOwnedTree(staging);- Fail("Playden could not put the new kit in place.", final_folder);+ fail("Playden could not put the new kit in place.", final_folder);return false;}if (!local) unlink(zip.c_str());return true;}+void UpdateService::FailKit(const std::string& message, const std::string& detail) {+ Log("kit install failed: " + message + (detail.empty() ? "" : " (" + detail + ")"));+ Set([&](UpdateSnapshot& s) {+ s.kit.phase = KitInstallPhase::kFailed;+ s.kit.error = message;+ });+}++// Every release, newest first: GET .../releases next to the configured .../releases/latest. Console+// testing reads release.json from local_release_dir as the only one.+std::optional<std::vector<Release>> UpdateService::FetchReleases(std::string* error) {+ if (!config_.local_release_dir.empty() && Exists(config_.local_release_dir + "/release.json")) {+ std::string text;+ if (!ReadSmallFile(config_.local_release_dir + "/release.json", &text, 2 << 20)) {+ *error = "cannot read release.json";+ return std::nullopt;+ }+ auto release = ParseRelease(text, error);+ if (!release) return std::nullopt;+ return std::vector<Release>{std::move(*release)};+ }+ std::string url = config_.latest_release_url;+ if (url.size() > 7 && url.compare(url.size() - 7, 7, "/latest") == 0) url.erase(url.size() - 7);+ auto options = Downloads("");+ options.max_attempts = 2;+ const auto body = FetchSmall(url + "?per_page=100", 4 << 20, options, "application/vnd.github+json", error);+ if (!body) return std::nullopt;+ return ParseReleases(*body, error);+}++void UpdateService::CheckKit() {+ if (config_.kit_id.empty() || config_.kit_manifest_sha256.empty()) return;+ Run([this] { DoCheckKit(); });+}++void UpdateService::InstallKit() {+ Run([this] {+ if (!kit_asset_ || !kit_part_) DoCheckKit();+ if (Snapshot().kit.phase != KitInstallPhase::kAvailable) return;+ DoInstallKit();+ });+}++void UpdateService::DoCheckKit() {+ const auto phase = Snapshot().phase;+ // A staged update brings its own kit when Playden restarts; one downloading may too.+ if (phase == UpdatePhase::kStaged || phase == UpdatePhase::kDownloading || phase == UpdatePhase::kUnpacking) return;+ const std::string installed = config_.kits_root + "/" + config_.kit_id;+ const auto installed_sha = config_.verify_kit ? config_.verify_kit(installed) : std::nullopt;+ if (installed_sha && *installed_sha == config_.kit_manifest_sha256) {+ Set([](UpdateSnapshot& s) { s.kit = {}; s.kit.phase = KitInstallPhase::kNotNeeded; });+ return;+ }+ Log("kit check: " + (installed_sha ? "the installed kit is another one (" + *installed_sha + ")"+ : std::string("no complete kit is installed")));+ Set([](UpdateSnapshot& s) {+ s.kit = {};+ s.kit.phase = KitInstallPhase::kChecking;+ });+ const auto unavailable = [&](const std::string& why) {+ Log("kit check: " + why);+ Set([](UpdateSnapshot& s) { s.kit.phase = KitInstallPhase::kUnavailable; });+ };+ std::string error;+ const auto releases = FetchReleases(&error);+ if (!releases) return FailKit("Playden could not reach GitHub to download its compatibility kit.", error);+ const Release* own = nullptr;+ for (const auto& release : *releases)+ if (!release.draft && release.version == config_.current) own = &release;+ if (!own) return unavailable("no release has version " + config_.current.ToString());+ const auto* manifest_asset = own->Find(kUpdateManifestAsset);+ if (!manifest_asset) return unavailable("release " + own->tag + " has no update information");+ auto options = Downloads("");+ options.max_attempts = 2;+ const bool local = !config_.local_release_dir.empty() && Exists(config_.local_release_dir + "/" + kUpdateManifestAsset);+ std::optional<std::string> manifest_text;+ if (!local) {+ manifest_text = FetchSmall(manifest_asset->url, 256 << 10, options, "application/octet-stream", &error);+ } else if (std::string text; ReadSmallFile(config_.local_release_dir + "/" + kUpdateManifestAsset, &text, 256 << 10)) {+ manifest_text = std::move(text);+ } else {+ error = std::string("cannot read ") + kUpdateManifestAsset;+ }+ if (!manifest_text) return FailKit("Playden could not read the kit information.", error);+ const auto manifest = ParseUpdateManifest(*manifest_text, &error);+ if (!manifest) return FailKit("The kit information of " + own->tag + " is not valid.", error);+ if (manifest->kit.kit_id != config_.kit_id || manifest->kit.manifest_sha256 != config_.kit_manifest_sha256)+ return unavailable("release " + own->tag + " names kit " + manifest->kit.kit_id + " " + manifest->kit.manifest_sha256);+ // A release whose kit did not change may leave the kit zip out: an older release has the same zip.+ const Release* source = own;+ const ReleaseAsset* zip = own->Find(manifest->kit.asset);+ for (const auto& release : *releases) {+ if (zip) break;+ if (release.draft || config_.current < release.version) continue;+ if ((zip = release.Find(manifest->kit.asset))) source = &release;+ }+ if (!zip) return unavailable("no release has " + manifest->kit.asset);+ if (zip->sha256.empty()) return FailKit("The kit on GitHub has no checksum. Install it by hand.", source->tag);+ const bool staged = !ReadyKit(manifest->kit).empty();+ kit_asset_ = *zip;+ kit_part_ = manifest->kit;+ Log("kit " + manifest->kit.asset + " available from " + source->tag + ": " ++ (staged ? std::string("already on the console") : std::to_string(zip->size) + " bytes"));+ Set([&](UpdateSnapshot& s) {+ s.kit.phase = KitInstallPhase::kAvailable;+ s.kit.replaces = installed_sha.has_value();+ s.kit.version = source->version.ToString();+ s.kit.download_bytes = staged ? 0 : zip->size;+ });+}++void UpdateService::DoInstallKit() {+ const auto phase = Snapshot().phase;+ // A staged update's kit may be the same one: taking it would leave the update without its kit.+ if (phase == UpdatePhase::kStaged || phase == UpdatePhase::kDownloading || phase == UpdatePhase::kUnpacking) {+ Log("kit install skipped: an update brings the kit");+ return;+ }+ const auto& kit = *kit_part_;+ const std::string installed = config_.kits_root + "/" + kit.kit_id;+ std::string folder = ReadyKit(kit);+ if (folder.empty()) {+ if (!PrepareWorkRoot() || !MakeFolders(config_.work_root + "/downloads"))+ return FailKit("Playden could not create its download folder.", config_.work_root);+ const std::uint64_t needed = kit_asset_->size * 4; // zip, unpacked files and some room to spare+ if (config_.free_bytes) {+ const auto free = config_.free_bytes(config_.work_root);+ if (free && *free < needed)+ return FailKit("The kit needs " + Gigabytes(needed) + " of free space; " + Gigabytes(*free) + " is free.",+ "free space");+ }+ if (!StageKit(*kit_asset_, kit, true)) return;+ folder = ReadyKit(kit);+ if (folder.empty()) return FailKit("The new kit did not pass its check.", "not ready after unpacking");+ }+ if (folder != installed) {+ // No game can use the kit in place (none was admitted), so it is swapped now; the one it+ // replaces is a ".replaced-" leftover, removed at a later start.+ const std::string aside = config_.kits_root + "/.replaced-" + kit.kit_id + "-" + std::to_string(config_.now());+ const bool moved = config_.as_root([&] {+ if (Exists(installed) && rename(installed.c_str(), aside.c_str()) != 0) return false;+ if (rename(folder.c_str(), installed.c_str()) == 0) return true;+ if (Exists(aside)) rename(aside.c_str(), installed.c_str());+ return false;+ });+ if (!moved) return FailKit("Playden could not put the kit in place.", installed);+ }+ Log("kit " + kit.kit_id + " " + kit.revision + " installed");+ Set([](UpdateSnapshot& s) {+ s.kit.phase = KitInstallPhase::kInstalled;+ s.kit.done = s.kit.total;+ });+}+bool UpdateService::StageApp(const Release& release, const UpdateManifest& manifest) {const auto* asset = release.Find(manifest.app_asset);const std::string local_zip = config_.local_release_dir + "/" + asset->name;diff --git a/src/update/update_service.h b/src/update/update_service.hindex 315ac19..42c10a0 100644--- a/src/update/update_service.h+++ b/src/update/update_service.h@@ -10,6 +10,7 @@#include <mutex>#include <optional>#include <string>+#include <vector>#include "update/asset_download.h"#include "update/release.h"@@ -24,6 +25,10 @@ struct UpdateConfig {std::string installed_root; // /data/homebrew/FAKE77031std::string work_root; // /data/playden/self-updatestd::string kits_root; // /data/playden/kits; the kit lives at <kits_root>/<kit_id>+ // The kit this build was made for (runtime/kit_trust.h): CheckKit offers it when the installed kit+ // is missing, incomplete or another one. Empty: no kit-only install.+ std::string kit_id;+ std::string kit_manifest_sha256;std::string appmeta_root; // /user/appmeta/FAKE77031std::string registered_app_root; // /user/app/FAKE77031// The 0.6 layout move's record (/data/playden/runtime-state/layout-0.6.json). While it exists,@@ -59,6 +64,31 @@ enum class UpdatePhase {const char* UpdatePhaseName(UpdatePhase phase);+// The kit-only install (CheckKit, InstallKit): the running version's kit, put in place at once.+enum class KitInstallPhase {+ kIdle,+ kChecking,+ kNotNeeded, // the installed kit is this version's+ kUnavailable, // no release on GitHub has this version's kit (a development build)+ kAvailable, // can be downloaded+ kDownloading,+ kUnpacking,+ kInstalled, // in place: the runtime can admit it now+ kFailed,+};++const char* KitInstallPhaseName(KitInstallPhase phase);++struct KitInstallSnapshot {+ KitInstallPhase phase = KitInstallPhase::kIdle;+ bool replaces = false; // another, complete kit is installed: this one replaces it+ std::string version; // the release that has the kit ("0.6.0")+ std::uint64_t download_bytes = 0; // 0 when a copy is ready on the console already+ std::uint64_t done = 0;+ std::uint64_t total = 0;+ std::string error; // for kFailed: one short sentence+};+struct UpdateSnapshot {std::uint64_t revision = 0;UpdatePhase phase = UpdatePhase::kIdle;@@ -86,6 +116,7 @@ struct UpdateSnapshot {// One-shot result of the previous run's update ("Playden was updated to 0.3.0."), shown once.std::string finished_message;bool finished_failed = false;+ KitInstallSnapshot kit;};class UpdateService {@@ -103,6 +134,12 @@ class UpdateService {// Downloads, unpacks and stages the available release in the background.void Install();void Cancel();+ // Looks for the running version's kit when the installed one is missing, incomplete or another+ // one. Skipped while an update is staged (its restart brings a kit). In the background.+ void CheckKit();+ // Downloads the kit CheckKit found and puts it in place at once (the installed one, if any, is+ // moved aside). In the background.+ void InstallKit();// Marks a staged update to be applied when Playden quits.void RequestRestart();// Marks the previous version (and its kit) to be put back when Playden quits.@@ -140,7 +177,12 @@ class UpdateService {// The folder holding a checked copy of the release's kit: the installed kit, a kit staged// earlier (".next-<kit_id>-<revision>"), or empty when it must be downloaded.std::string ReadyKit(const KitPart& kit) const;- bool StageKit(const Release& release, const UpdateManifest& manifest);+ // Downloads and unpacks the kit into ".next-<kit_id>-<revision>"; kit_only reports in snapshot.kit.+ bool StageKit(const ReleaseAsset& asset, const KitPart& kit, bool kit_only);+ void DoCheckKit();+ void DoInstallKit();+ void FailKit(const std::string& message, const std::string& detail);+ std::optional<std::vector<Release>> FetchReleases(std::string* error);bool StageApp(const Release& release, const UpdateManifest& manifest);void Fail(const std::string& message, const std::string& detail);void Set(const std::function<void(UpdateSnapshot&)>& change);@@ -151,7 +193,7 @@ class UpdateService {std::string StagedFields() const;bool PrepareWorkRoot();bool RemoveOwnedTree(const std::string& path);- void RemoveEmptyReplacedKits();+ void RemoveReplacedKits();void CleanUpLeftovers();UpdateConfig config_;@@ -164,6 +206,9 @@ class UpdateService {Version staged_version_;std::string staged_kit_;std::string staged_kit_id_;+ // What CheckKit found: the kit zip (from the running version's release or an older one with the same zip).+ std::optional<ReleaseAsset> kit_asset_;+ std::optional<KitPart> kit_part_;};} // namespace playden::updatediff --git a/tests/homebrew_live_model_test.cpp b/tests/homebrew_live_model_test.cppindex bc59a07..d3c9ad8 100644--- a/tests/homebrew_live_model_test.cpp+++ b/tests/homebrew_live_model_test.cpp@@ -646,6 +646,96 @@ void live_download_telemetry() {}+// The kit-only install: offered before sign-in, downloaded while the player signs in, and listed on+// the Queue and in Settings until it is in place.+void live_kit_install(const std::filesystem::path& base) {+ using namespace playden::homebrew;+ std::filesystem::create_directories(base);+ auto model = make_live(base);+ check(model.state().navigation.route == Route::Startup, "kit: a first start shows Connect Steam");+ LiveSelfUpdate update;+ update.current = "0.6.0";+ update.kit_phase = "checking";+ model.UpdateSelfUpdate(update);+ check(model.state().navigation.modals.empty(), "kit: nothing is offered while it is looked for");+ update.kit_phase = "available";+ update.kit_download_bytes = 300000000;+ model.UpdateSelfUpdate(update);+ check(!model.state().navigation.modals.empty() &&+ model.state().navigation.modals.back().title == "Download the compatibility kit?" &&+ model.state().navigation.modals.back().body.find("sign in") != std::string::npos,+ "kit: offered before sign-in, with the download size");+ check(model.action("cross") && model.state().navigation.modals.empty() &&+ model.TakeSelfUpdateRequests() == std::vector<std::string>{"kit-install"},+ "kit: Download asks for the kit");++ update.kit_phase = "downloading";+ update.kit_done = 100;+ update.kit_total = 400;+ model.UpdateSelfUpdate(update);+ // Cancelled: the kit is available again, but not offered again until the next check.+ update.kit_phase = "available";+ model.UpdateSelfUpdate(update);+ check(model.state().navigation.modals.empty(), "kit: a cancelled download isn't offered again");+ update.kit_phase = "downloading";+ model.UpdateSelfUpdate(update);+ const auto job = std::find_if(model.state().jobs.begin(), model.state().jobs.end(),+ [](const Job& item) { return item.id == "kit-install"; });+ check(job != model.state().jobs.end() && job->title == "Compatibility kit" && job->completed_bytes == 100 &&+ job->stage_text == "Downloading the compatibility kit",+ "kit: the download shows on the Queue");++ // Signed in while it downloads: Settings shows the progress.+ model.UpdateAccount({AuthState::SignedIn, "account", "Owner", {}, {}, false});+ check(model.UpdateLibrary({{}, false, {}, false, true, 100}), "kit: library delivered");+ for (int i = 0; i < 6 && model.state().navigation.route != Route::Settings; ++i) model.action("r1");+ check(find_row(model, "settings-kit") && find_row(model, "settings-kit")->value == "25%",+ "kit: Settings shows the progress");++ update.kit_phase = "installed";+ model.UpdateSelfUpdate(update);+ check(std::none_of(model.state().jobs.begin(), model.state().jobs.end(),+ [](const Job& item) { return item.id == "kit-install"; }) &&+ !find_row(model, "settings-kit") &&+ std::any_of(model.state().notices.begin(), model.state().notices.end(),+ [](const Notice& notice) { return notice.id == "kit-installed"; }),+ "kit: once in place, the job and row go and a notice says so");++ // Another kit is installed: the offer says it replaces it; a failed download says why.+ auto other = make_live(base / "other");+ LiveSelfUpdate replace;+ replace.current = "0.6.0";+ replace.kit_phase = "available";+ replace.kit_replaces = true;+ other.UpdateSelfUpdate(replace);+ check(!other.state().navigation.modals.empty() &&+ other.state().navigation.modals.back().title == "Replace the compatibility kit?",+ "kit: replacing another kit is said");+ other.action("circle");+ replace.kit_phase = "downloading";+ other.UpdateSelfUpdate(replace);+ replace.kit_phase = "failed";+ replace.kit_error = "The kit download failed.";+ other.UpdateSelfUpdate(replace);+ // Like every alert, it waits for sign-in.+ other.UpdateAccount({AuthState::SignedIn, "account", "Owner", {}, {}, false});+ check(other.UpdateLibrary({{}, false, {}, false, true, 100}), "kit: library delivered");+ other.action("r1"); // a queued alert shows after the next press+ check(!other.state().navigation.modals.empty() &&+ other.state().navigation.modals.back().title == "Compatibility kit download stopped",+ "kit: a failed download says why");++ // An update on offer brings the kit itself: the kit isn't offered on top of it.+ auto both = make_live(base / "both");+ LiveSelfUpdate newer;+ newer.current = "0.6.0";+ newer.phase = "available";+ newer.available = "0.7.0";+ newer.kit_phase = "available";+ both.UpdateSelfUpdate(newer);+ check(both.state().navigation.modals.empty(), "kit: no kit offer while a newer Playden is on offer");+}+// Playden's own update: the prompts, the Downloads job, the Settings row and the requests.void live_self_update(const std::filesystem::path& base) {using namespace playden::homebrew;@@ -1067,6 +1157,7 @@ int main() {failed_signout_preserves_account_until_external_clear(base);account_commands_dispatch_refresh_and_switch_barriers(base);live_self_update(base);+ live_kit_install(base / "kit");std::filesystem::remove_all(base, error);std::cout << "homebrew_live_model_test passed\n";diff --git a/tests/homebrew_runtime_host_test.cpp b/tests/homebrew_runtime_host_test.cppindex 2f6e4ee..0c309de 100644--- a/tests/homebrew_runtime_host_test.cpp+++ b/tests/homebrew_runtime_host_test.cpp@@ -177,6 +177,42 @@ void NoKitRefusesPreparationWithoutDurableEffects() {CHECK(fs::is_empty(fixture.root / "data/homebrew"));}+// A kit put in place while Playden runs (the first-run kit download) is admitted by ReloadKit: the+// services start again with it, also over a kit already in use (one this version wasn't made for).+void ReloadKitAdmitsAKitInstalledWhileRunning() {+ Fixture fixture;+ auto config = fixture.Config();+ const auto kit = AnyTitleKit(fixture.root / "kit");+ std::atomic<bool> installed{false};+ std::atomic<int> admissions{0};+ config.admit_kit = [&](std::string* reason) -> std::optional<RuntimeKit> {+ ++admissions;+ if (installed) return kit;+ *reason = "No verified compatibility kit is installed.";+ return std::nullopt;+ };+ RuntimeHost host(std::move(config));+ CHECK(host.Start());+ CHECK(!WaitFor(host, Started).kit_available);+ host.Prepare(Hl2());+ WaitFor(host, [](const RuntimeSnapshot& s) { return s.refusals.count(220) != 0; });+ installed = true;+ host.ReloadKit();+ const auto reloaded = WaitFor(host, [](const RuntimeSnapshot& s) { return Started(s) && s.kit_available; });+ CHECK(reloaded.status == RuntimeHostStatus::kReady && reloaded.kit_reason.empty());+ CHECK(reloaded.refusals.empty()); // the refusal for want of a kit is gone+ CHECK(admissions == 2);+ // Preparation goes past the kit check now (this fixture has no game folder to prepare).+ host.Prepare(Hl2());+ const auto done = WaitFor(host, [](const RuntimeSnapshot& s) { return s.prepares_handled == 2; }, 60);+ CHECK(!done.refusals.count(220) || done.refusals.at(220) != "No verified compatibility kit is installed.");+ host.ReloadKit();+ WaitFor(host, [&](const RuntimeSnapshot& s) { return Started(s) && admissions == 3; });+ CHECK(host.Snapshot().kit_available);+ host.Shutdown();+ CHECK(host.Snapshot().status == RuntimeHostStatus::kStopped);+}+// Root is process-wide on the console: while the worker starts up or works through a batch it holds// the effective UID, so a root raise elsewhere (an update's cleanup) can't fail its ownership checks.// Waiting for work, it holds nothing.@@ -552,6 +588,7 @@ void PreparesExistingDownloadToReady() {(!s.installs.empty() && s.installs[0].state == PreparationState::kReady));}, 60);if (done.refusals.count(220)) std::fprintf(stderr, "Prepare refused: %s\n", done.refusals.at(220).c_str());+ if (done.refusals.count(220)) std::fprintf(stderr, "refusal: %s\n", done.refusals.at(220).c_str());CHECK(!done.refusals.count(220));CHECK(done.installs.size() == 1);const auto& ready = done.installs[0];@@ -674,6 +711,7 @@ void PrepareFixesRootOwnedHomebrewFolder() {(!s.installs.empty() && s.installs[0].state == PreparationState::kReady));}, 60);if (done.refusals.count(220)) std::fprintf(stderr, "Prepare refused: %s\n", done.refusals.at(220).c_str());+ if (done.refusals.count(220)) std::fprintf(stderr, "refusal: %s\n", done.refusals.at(220).c_str());CHECK(!done.refusals.count(220));CHECK(done.installs.size() == 1 && done.installs[0].state == PreparationState::kReady);CHECK(raises == 1); // only the folder that failed its probe@@ -1930,7 +1968,8 @@ void OwnedPublicationOpenedUpRecovers(int route) {host.Prepare(Hl2());const auto done = WaitFor(host, [before](const RuntimeSnapshot& s) { return s.prepares_handled > before; }, 60);if (done.refusals.count(220)) std::fprintf(stderr, "Update refused: %s\n", done.refusals.at(220).c_str());- CHECK(!done.refusals.count(220));+ if (done.refusals.count(220)) std::fprintf(stderr, "refusal: %s\n", done.refusals.at(220).c_str());+ CHECK(!done.refusals.count(220));} else if (route == 4) {host.Play(220);const auto done = WaitFor(host, [](const RuntimeSnapshot& s) { return s.handoff_ready || s.play_refusals.count(220); }, 30);@@ -2991,7 +3030,8 @@ void PreparesGameOnUnownedDrive() {CHECK(done.installs.empty());} else {if (done.refusals.count(220)) std::fprintf(stderr, "Prepare refused: %s\n", done.refusals.at(220).c_str());- CHECK(!done.refusals.count(220));+ if (done.refusals.count(220)) std::fprintf(stderr, "refusal: %s\n", done.refusals.at(220).c_str());+ CHECK(!done.refusals.count(220));CHECK(done.installs.size() == 1 && done.installs[0].state == PreparationState::kReady);const auto metadata = Read(fixture.root / "data/playden/games/steam-220/library/profiles/.playden-publication");CHECK(metadata.find("schema=3\n") != std::string::npos);@@ -5171,6 +5211,7 @@ int main(int argc, char** argv) {g_runtime_dir = argv[1];WorkerMovesGamesUnderPublicationLease();NoKitRefusesPreparationWithoutDurableEffects();+ ReloadKitAdmitsAKitInstalledWhileRunning();QueuedPreparationIsPublishedUntilTaken();PrepareNextReordersWaitingPreparations();HeldPreparationsWait();diff --git a/tests/self_update_service_test.cpp b/tests/self_update_service_test.cppindex 2349086..af6a9b6 100644--- a/tests/self_update_service_test.cpp+++ b/tests/self_update_service_test.cpp@@ -22,6 +22,7 @@ using zip_fixture::Entry;namespace {const std::string kLatest = "https://api.github.com/repos/o/r/releases/latest";+const std::string kReleases = "https://api.github.com/repos/o/r/releases?per_page=100";const std::string kDownload = "https://github.com/o/r/releases/download/v0.3.0/";const std::string kKitSha = std::string(64, 'c');@@ -46,16 +47,20 @@ void Put(const std::string& path, const std::string& text) {struct Server {std::map<std::string, std::string> files;std::string latest;+ std::string releases; // the list, newest firstint kit_downloads = 0;+ int requests = 0;HttpResponse operator()(const HttpRequest& request) {HttpResponse response;std::string range;for (const auto& [key, value] : request.headers)if (key == "Range") range = value;- if (request.url == kLatest) {+ ++requests;+ if (request.url == kLatest || request.url == kReleases) {+ const std::string& body = request.url == kLatest ? latest : releases;response.status_code = 200;- response.body.assign(latest.begin(), latest.end());+ response.body.assign(body.begin(), body.end());return response;}const auto found = files.find(request.url);@@ -157,6 +162,26 @@ void Wait(const UpdateService& service) {CHECK(!service.Busy());}+// The kit-only install for version 0.3.0, on a console with `installed` as its kit (empty: none).+struct KitOnly {+ Fixture fixture;+ std::string kits;+ explicit KitOnly(const std::string& name, const std::string& installed) {+ fixture.root = zip_fixture::TempFolder(name.c_str());+ kits = fixture.root + "/playden/kits";+ if (!installed.empty()) {+ Put(kits + "/kitid/manifest.json", installed);+ Put(kits + "/kitid/runtime/wine.bin", "old");+ }+ }+ UpdateConfig Config() {+ auto config = fixture.Config({0, 3, 0});+ config.kit_id = "kitid";+ config.kit_manifest_sha256 = kKitSha;+ return config;+ }+};+void Installed(const std::string& root) {Put(root + "/homebrew/FAKE77031/eboot.bin", "eboot 0.2.0");Put(root + "/homebrew/FAKE77031/sce_sys/param.json", "old param");@@ -171,6 +196,118 @@ void Installed(const std::string& root) {} // namespaceint main() {+ // First run on a console without a kit: the running version's kit is found and put in place at once.+ {+ KitOnly test("kit-fresh", "");+ Publish(test.fixture.server, "v0.3.0");+ test.fixture.server.releases = "[" + test.fixture.server.latest + "]";+ UpdateService service(test.Config());+ service.Start();+ service.CheckKit();+ Wait(service);+ auto kit = service.Snapshot().kit;+ CHECK(kit.phase == KitInstallPhase::kAvailable && !kit.replaces && kit.version == "0.3.0");+ CHECK(kit.download_bytes == test.fixture.server.files[kDownload + "playden-kit-h-r9.zip"].size());+ CHECK(service.Snapshot().phase == UpdatePhase::kIdle); // the update's own state is untouched+ service.InstallKit();+ Wait(service);+ kit = service.Snapshot().kit;+ CHECK(kit.phase == KitInstallPhase::kInstalled);+ CHECK(Read(test.kits + "/kitid/manifest.json") == kKitSha && Read(test.kits + "/kitid/runtime/wine.bin").size() == 100);+ CHECK(!Exists(test.kits + "/.next-kitid-r9") && !Exists(test.kits + "/.staging-kitid-r9"));+ CHECK(ListFolder(test.fixture.root + "/playden/self-update/downloads").empty());+ // Installed: the next check needs nothing, and asks GitHub for nothing.+ const int requests = test.fixture.server.requests;+ service.CheckKit();+ Wait(service);+ CHECK(service.Snapshot().kit.phase == KitInstallPhase::kNotNeeded && test.fixture.server.requests == requests);+ service.Shutdown();+ }+ // Another kit is installed: the right one replaces it, and the old one is left as a ".replaced-" leftover.+ {+ KitOnly test("kit-replace", std::string(64, 'b'));+ Publish(test.fixture.server, "v0.3.0");+ test.fixture.server.releases = "[" + test.fixture.server.latest + "]";+ UpdateService service(test.Config());+ service.InstallKit(); // checks first+ Wait(service);+ const auto kit = service.Snapshot().kit;+ CHECK(kit.phase == KitInstallPhase::kInstalled && kit.replaces);+ CHECK(Read(test.kits + "/kitid/manifest.json") == kKitSha);+ CHECK(Read(test.kits + "/.replaced-kitid-1000/manifest.json") == std::string(64, 'b'));+ service.Shutdown();+ }+ // The running version's release left the kit zip out (its kit did not change): an older release has it.+ {+ KitOnly test("kit-older", "");+ Publish(test.fixture.server, "v0.2.0");+ const std::string older = test.fixture.server.latest;+ Publish(test.fixture.server, "v0.3.0", "0.2.0", "00.003.000", false);+ Publish(test.fixture.server, "v0.4.0"); // newer than this version: never its kit's source+ const std::string newer = test.fixture.server.latest;+ Publish(test.fixture.server, "v0.3.0", "0.2.0", "00.003.000", false);+ test.fixture.server.releases = "[" + newer + "," + test.fixture.server.latest + "," + older + "]";+ UpdateService service(test.Config());+ service.CheckKit();+ Wait(service);+ const auto kit = service.Snapshot().kit;+ CHECK(kit.phase == KitInstallPhase::kAvailable && kit.version == "0.2.0");+ service.Shutdown();+ }+ // A staged update brings its kit at the restart: the kit-only install leaves it alone, even when asked.+ {+ KitOnly test("kit-staged", "");+ Installed(test.fixture.root);+ unlink((test.kits + "/kitid/manifest.json").c_str()); // no complete kit+ Publish(test.fixture.server, "v0.3.0");+ test.fixture.server.releases = "[" + test.fixture.server.latest + "]";+ auto config = test.Config();+ config.current = {0, 2, 0};+ UpdateService service(std::move(config));+ service.Check(true);+ Wait(service);+ service.Install();+ Wait(service);+ CHECK(service.Snapshot().phase == UpdatePhase::kStaged && Exists(test.kits + "/.next-kitid-r9"));+ service.CheckKit();+ Wait(service);+ CHECK(service.Snapshot().kit.phase == KitInstallPhase::kIdle);+ service.InstallKit();+ Wait(service);+ CHECK(service.Snapshot().kit.phase == KitInstallPhase::kIdle && Exists(test.kits + "/.next-kitid-r9"));+ service.Shutdown();+ }+ // A replaced kit (here one uploaded over FTP) is removed at the next start.+ {+ KitOnly test("kit-leftover", "");+ Put(test.kits + "/.replaced-kitid-5/manifest.json", std::string(64, 'b'));+ Put(test.kits + "/.replaced-kitid-5/runtime/wine.bin", "old");+ UpdateService service(test.Config());+ service.Start();+ CHECK(!Exists(test.kits + "/.replaced-kitid-5"));+ service.Shutdown();+ }+ // A development build has no release, and a release made with another kit doesn't offer it.+ {+ KitOnly test("kit-none", "");+ Publish(test.fixture.server, "v0.2.0");+ test.fixture.server.releases = "[" + test.fixture.server.latest + "]";+ UpdateService service(test.Config());+ service.CheckKit();+ Wait(service);+ CHECK(service.Snapshot().kit.phase == KitInstallPhase::kUnavailable);+ auto other = test.Config();+ other.kit_manifest_sha256 = std::string(64, 'd');+ Publish(test.fixture.server, "v0.3.0");+ test.fixture.server.releases = "[" + test.fixture.server.latest + "]";+ UpdateService another(std::move(other));+ another.CheckKit();+ Wait(another);+ CHECK(another.Snapshot().kit.phase == KitInstallPhase::kUnavailable);+ CHECK(!Exists(test.kits + "/kitid"));+ service.Shutdown();+ another.Shutdown();+ }// Full run: check, install (kit and app), restart, swap, then confirm on the next start.{Fixture fixture;