Skip to content

Play your own Switch games on iPhone/iPad — one-click macOS app, no terminal. | 把 Switch 游戏装进 iPhone:图形界面一键构建,不用命令行。Bring your own game.

Move the engine selection, the engine sync and the engine failure into an EngineSyncStore (#387)

v0lumehif04d4acde242
diff --git a/frontend-qt/nspb_qt/engine.py b/frontend-qt/nspb_qt/engine.py
index 872d71c5..89ef6814 100644
--- a/frontend-qt/nspb_qt/engine.py
+++ b/frontend-qt/nspb_qt/engine.py
@@ -1,7 +1,7 @@
"""Where the engine of a checkout is, and whether it needs `tools/engine.py sync` (LNPANDY/NX2Apple#214).
The engine is not committed: it is built into `<checkout>/engine/` (seedds/nxnative at `.upstream-ref` plus `patches/`) by
-`tools/engine.py sync`. The rule, the same as the Mac launcher's (frontend/Sources/NX2Apple/Engine.swift). The sync is for
+`tools/engine.py sync`. The rule, the same as the Mac launcher's (frontend/Sources/NXCore/EngineLayout.swift). The sync is for
checkouts (a release carries the engine tree instead, #227): it downloads the pinned commit from GitHub, and a failed one becomes a
failure row (`failure_kind`):
diff --git a/frontend/ARCHITECTURE.md b/frontend/ARCHITECTURE.md
index 891bfa50..6dbef446 100644
--- a/frontend/ARCHITECTURE.md
+++ b/frontend/ARCHITECTURE.md
@@ -13,16 +13,17 @@ The plan and the steps are in [LNPANDY/NX2Apple#387](https://github.com/LNPANDY/
## Where it stands
-Step 1 of #387 made the targets and moved the files that already fit. Step 2 moved the language out of `BuildModel` into the `Localizer` store. Step 3 moved the build log into the `BuildLog` store. Step 4 moved the pure logic into `NXCore`. Step 5 added the first services. Step 6 added the first store, `AppUpdateStore`, the reference for the others. Step 7 moves the rest one store per PR, the repository first. The rest still lives in the app target and moves with the later steps.
+Step 1 of #387 made the targets and moved the files that already fit. Step 2 moved the language out of `BuildModel` into the `Localizer` store. Step 3 moved the build log into the `BuildLog` store. Step 4 moved the pure logic into `NXCore`. Step 5 added the first services. Step 6 added the first store, `AppUpdateStore`, the reference for the others. Step 7 moves the rest one store per PR: the repository, then the engine sync. The rest still lives in the app target and moves with the later steps.
- `BuildModel` is the model of the whole launcher and reaches into every layer, so the files that use it cannot leave the app target yet. It no longer owns the language: it holds the `Localizer` it was given and forwards `lang` and `t()` to it, for the text it writes itself (alerts, status, failures).
-- `NXCore` holds the value types and static functions that need only Foundation: app and engine update parsing, `Localization.swift`, the release notes, the device and signing tools, the cloud folder and saves migration, the bundled tools, input staging, the build target and the pipe reader. Step 4 added four types: `TitleID` (parse a title ID from a file name, its owner, the update version, the build key), `GameInputs` (classify files into the four rows, row mismatch, allowed file types), `BuildProgress` (stage lines, progress details, durations, the steps of Xcode and the install, failure info) and `Toolchain` (probing Python, the venv, dotnet and Xcode, and the dependency rows `Dep`). `Stage` and `StepState` moved with `BuildProgress`. Step 5 added the first services (step 6 added `ReleaseSource`, see below): `ProcessRunner` (a protocol, with `SystemProcessRunner` as the real one), `ProcessGroups` (the process groups that Stop and quitting end) and `GitCheckout` (a protocol, with `SystemGitCheckout`, plus `GitRules` for the decisions that need no git). `ProcessRunner` is where a build process starts: `capture` runs to the end and keeps the output, `stream` hands out lines, `start` returns at once and can lead a process group. `BuildModel` holds a runner and a checkout, takes them in `init`, and uses them for the clone, fetch, pull, adoption of a ZIP copy, the build run, the tools of a stage, the archive step and `openOnDevice`. `Toolchain`, `Diagnostics`, `OlderTools`, `Uninstall`, `EngineLayout.runSync` and the update staging take a runner or a checkout as a parameter with the real one as default. Tests pass in a fake. These still start a `Process` themselves and move with the step that owns them: `AppSignature` (codesign) with the device store, `DeviceTools.run` (devicectl with a timeout and input) and `SaveTransfer` with the device store, `ShellInstaller.run` (a blocking read with start and end callbacks) with the build store, `EngineUpdater` (ditto) with the engine store, the administrator prompt in `LiveOnboardingDriver` and the game start in `LiveConsoleDriver`. The tests that stand in for `xcodebuild` and `devicectl` through `BuildModel.xcrunPath` stay too: `ShellInstaller` and `DeviceTools` run those tools themselves, so they move with the device store. What reads `MetalPin`, `EngineLayout` or `OnboardingDefaults` (`detectDepsSync`, `metalDep`, `needsIOSPlatform`) and the handling of the build's JSON events stay in `BuildModel` until the later store steps.
+- `NXCore` holds the value types and static functions that need only Foundation: app and engine update parsing, `Localization.swift`, the release notes, the device and signing tools, the cloud folder and saves migration, the bundled tools, input staging, the build target and the pipe reader. Step 4 added four types: `TitleID` (parse a title ID from a file name, its owner, the update version, the build key), `GameInputs` (classify files into the four rows, row mismatch, allowed file types), `BuildProgress` (stage lines, progress details, durations, the steps of Xcode and the install, failure info) and `Toolchain` (probing Python, the venv, dotnet and Xcode, and the dependency rows `Dep`). `Stage` and `StepState` moved with `BuildProgress`. Step 5 added the first services (step 6 added `ReleaseSource`, see below): `ProcessRunner` (a protocol, with `SystemProcessRunner` as the real one), `ProcessGroups` (the process groups that Stop and quitting end) and `GitCheckout` (a protocol, with `SystemGitCheckout`, plus `GitRules` for the decisions that need no git). `ProcessRunner` is where a build process starts: `capture` runs to the end and keeps the output, `stream` hands out lines, `start` returns at once and can lead a process group. `BuildModel` holds a runner and a checkout, takes them in `init`, and uses them for the clone, fetch, pull, adoption of a ZIP copy, the build run, the tools of a stage, the archive step and `openOnDevice`. `Toolchain`, `Diagnostics`, `OlderTools`, `Uninstall`, `EngineLayout.runSync` and the update staging take a runner or a checkout as a parameter with the real one as default. Tests pass in a fake. These still start a `Process` themselves and move with the step that owns them: `AppSignature` (codesign) with the device store, `DeviceTools.run` (devicectl with a timeout and input) and `SaveTransfer` with the device store, `ShellInstaller.run` (a blocking read with start and end callbacks) with the build store, `EngineInstaller` (ditto, in `NXCore` since the engine sync store) still starts it itself, the administrator prompt in `LiveOnboardingDriver` and the game start in `LiveConsoleDriver`. The tests that stand in for `xcodebuild` and `devicectl` through `BuildModel.xcrunPath` stay too: `ShellInstaller` and `DeviceTools` run those tools themselves, so they move with the device store. What reads `MetalPin`, `EngineLayout` or `OnboardingDefaults` (`detectDepsSync`, `metalDep`, `needsIOSPlatform`) and the handling of the build's JSON events stay in `BuildModel` until the later store steps.
- Step 6 moved the self-update out of `BuildModel`. `AppUpdate` (`NXCore`) now also holds the release parsing (`release(fromJSON:)`, `check(latest:current:)`), the version compare (`isNewer`), the asset names, the staging of a download (`stage`, `StageFailure`) and the install, which clears the quarantine flag through a `ProcessRunner`. `ReleaseSource` is the second service (a protocol, with `GitHubReleases` as the real one): the latest release and the download. `Uninstall` moved to `NXCore` too, because the store holds its plan. `AppUpdate.relaunch` still starts its detached shell itself, and the store gets it as a closure.
- `NXCoreTests` tests `NXCore` alone. It links no other target, so these tests run without SwiftUI. New tests of pure logic go there.
- `NXModel` holds `Localizer`: `lang` and `t()`. The app creates one with each `BuildModel`, and `ContentView` hands it to every view as an environment object. A view that shows text declares `@EnvironmentObject var loc: Localizer` and calls `loc.t(...)`. A language change redraws these views and nothing else.
- `NXModel` holds `AppUpdateStore`: the release check, the staged update, the update sheet, the uninstall plan and `appVersion`. `BuildModel` creates it with its localizer, runner and release source, and the views that show the update (the launcher's top bar and activity line, the Settings section, the update sheet) observe it and not `BuildModel`. `NXModelTests` tests it with a fake runner and a fake release source, without SwiftUI.
- `NXModel` holds `RepoStore` (step 7, repository): the repository path, the status line under it, how far it is behind origin/main, the clone, the update check, the pull, the automatic update when the app opens and the adoption of a ZIP copy. `BuildModel` creates it with its `Localizer` and `GitCheckout` and wires four closures: whether a build runs, whether this is a release app (which never clones or pulls), the build log, and `didMove`, which starts the engine sync after a clone, a pull or an adoption. The views that show the repository (Settings "Tools", the library's activity line, the setup hint) observe the store. `RepoLocation` is the narrow protocol (`path`, `isRepo`, `isGit`) for a store that needs only where the repository is. `NXCore` got the rules the store needs: `RepoLayout` (is it a repository, the tools a build needs, which checkout to use at start) and the file-system part of `EngineLayout` (`marker`, `root`, `isCheckout`, `hasEngineTool`). The stale check and the sync of `EngineLayout` stay in the app target with the engine.
-- Still in `BuildModel`, going to their store: `repoPath`, `repoOK` and `repoIsGit` are forwarders to `RepoStore`, because the build code (start, archive, install, diagnostics, the engine switch) reads the path and is not a store yet. `BuildModel` also forwards the store's `objectWillChange` to its own, because `canBuild` and the build views read the repository through it. Both go with the build store (step 7, build). `syncEngine` and its state read the path and `pythonPath` and go to the engine sync store. The engine switch of the release app (`EngineUpdateModel`) sets the path to the installed tree and goes there too. `BuildModel` is not `@MainActor` yet, so it reaches the store through `MainActor.assumeIsolated`.
+- `NXModel` holds `EngineSyncStore` (step 7, engine sync): the engine selection (channel, engine ID, version, the one kept), the sync of a checkout's `engine/` (`tools/engine.py sync`, its status line and its failure row with Try again, the calls that wait for a running sync) and the engine of the release app (seeding the app's own engine, the check of the selected channel, the install, the channel switch after a confirmation, an engine that waits for the running build). `BuildModel` creates it with its localizer and runner, the `RepoStore` as the `RepoLocation` to read, itself as the `PythonLocation` (a narrow protocol with `pythonPath`, the dependency store takes it over) and closures: where the build reads its tools from (`setRepoPath`, an installed engine tree), whether a build runs, the automatic update and the log. `RepoStore.didMove` starts its sync. The views that show the engine (Settings "Tools", the sync banner, the failure row's Try again, the launcher's checks after a sync) observe the store. `NXCore` got what the store needs: the stale check and the sync of `EngineLayout`, `EngineUpdater.swift` (`EngineChannel`, `EngineSelection`, `EngineUpdate`, `EngineInstaller`, `EngineTransport`) and `BuildFailureKind` with `BuildFailure` (the sync's failure row). The classifier `BuildFailureClassifier` stays in the app target. `NXModelTests` tests the store with a fake runner, without SwiftUI.
+- Still in `BuildModel`, going to their store: `repoPath`, `repoOK` and `repoIsGit` are forwarders to `RepoStore`, because the build code (start, archive, install, diagnostics) reads the path and is not a store yet. `BuildModel` also forwards the objectWillChange of `RepoStore` and of `EngineSyncStore` to its own, because `canBuild` and the build views read the repository and the engine's paths through it. Both go with the build store (step 7, build). The synchronous calls of the build (`startBuild` waits for the sync, `Diagnostics` reads the selection) and `pythonPath` (the `PythonLocation`, with `deps`, which go to the dependency store) stay. `BuildModel` is not `@MainActor` yet, so it reaches the stores through `MainActor.assumeIsolated`.
- `NXModel` also holds `BuildLog`: the log text, its tail trimming and the line buffer. `BuildModel` holds it in a `let` and does not publish it, and only `BuildLogView` (in `NXLauncher`) observes it. A log line redraws the log and no game card. `LauncherPerformanceTests` writes 1000 lines during a build and allows 0 cards. `BuildModel` is not `@MainActor` yet, so it reaches the log through `BuildLog.onMain`.
- `NXLauncher` holds the views and styles that do not need `BuildModel`: `LauncherPalette`, `Typography`, `LauncherStyles`, `LauncherDropdown`, `PadFocus` and `EmbeddedFonts`.
- `NX2Apple` holds everything else, including `BuildModel`.
diff --git a/frontend/Sources/NX2Apple/BuildFailure.swift b/frontend/Sources/NX2Apple/BuildFailure.swift
index 62d61fcd..d36e0d26 100644
--- a/frontend/Sources/NX2Apple/BuildFailure.swift
+++ b/frontend/Sources/NX2Apple/BuildFailure.swift
@@ -1,47 +1,5 @@
import Foundation
-
-/// The early stops of the NSP script that the launcher can turn into a next step. The script prints a
-/// `[RED]` line and exits 1 for each (a `BuildError`), and the launcher used to show only the last line of
-/// the log, cut to 200 characters. The classifier reads the whole log tail, because for a damaged NSP the
-/// last line is only "nx_extract exit code 3" and the message is a few lines above it.
-enum BuildFailureKind: String, CaseIterable {
- case rosetta, translator, zbic, iosSDK, cltSelected, otherMetal, shaderRegression, damagedNSP, a32Program
- /// The inputs: the keys file, the game folder, a drive that is gone.
- case keys, severalGames, noTitleID, driveMissing
- /// The build's and the install's own steps.
- case diskFull, teamTaken, noSigningAccount, deviceTransfer
- /// The build folder is in iCloud Drive or another cloud folder (CloudFolder.swift): the launcher stops before the build, and the
- /// codesign line of xcodebuild is the same stop for a build that got past that check.
- case cloudFolder
- /// The shared app's install: a device this Mac never paired, and game data that went over only in part.
- case devicePairing, shellDataIncomplete
- /// The engine's online sync (tools/engine.py sync, Engine.swift): no network or GitHub unreachable, or any other stop of it.
- case engineOffline, engineSync
- case unknown
-
- /// The script's override for this stop, offered as "Build anyway". Only these three have one that the launcher
- /// may offer: the others cannot be built around (a 32-bit program, a missing SDK).
- var overrideFlag: String? {
- switch self {
- case .otherMetal: return "--allow-other-metal"
- case .shaderRegression: return "--allow-shader-regressions"
- case .damagedNSP: return "--allow-damaged"
- default: return nil
- }
- }
-
- /// L10n keys: what happened, and the next step.
- var whatKey: String { "fail.\(rawValue).what" }
- var nextKey: String { "fail.\(rawValue).next" }
-}
-
-struct BuildFailure: Equatable {
- var kind: BuildFailureKind
- var gameName: String? // a32Program: the game, when the message names it
- /// What the message adds to its row: the GB a full disk needs (diskFull), the drive's name (driveMissing), the folder
- /// of "no NSP with a readable title ID" (noTitleID).
- var detail: String? = nil
-}
+import NXCore
enum BuildFailureClassifier {
/// One row per message of the script. The pattern is a stable part of the text in scripts/build_ipa_from_nsp.py,
diff --git a/frontend/Sources/NX2Apple/BuildFailureView.swift b/frontend/Sources/NX2Apple/BuildFailureView.swift
index ff0a79f8..ad78b4da 100644
--- a/frontend/Sources/NX2Apple/BuildFailureView.swift
+++ b/frontend/Sources/NX2Apple/BuildFailureView.swift
@@ -7,12 +7,15 @@ import NXLauncher
/// button that does it (a dependency's own install command, or "Build anyway" for the three stops that have an override).
struct BuildFailureCard: View {
@ObservedObject var m: BuildModel
+ /// Where the engine's failure row retries. nil for a failure of a build.
+ var engine: EngineSyncStore?
@EnvironmentObject var loc: Localizer
/// The failure to show: the build's (`m.failure`) unless the caller passes another one, such as the engine sync's.
var failure: BuildFailure?
- init(m: BuildModel, failure: BuildFailure? = nil) {
+ init(m: BuildModel, engine: EngineSyncStore? = nil, failure: BuildFailure? = nil) {
self.m = m
+ self.engine = engine
self.failure = failure
}
@@ -67,7 +70,7 @@ struct BuildFailureCard: View {
case .shellDataIncomplete:
fix(kind, "shell.retry")
case .engineOffline, .engineSync:
- Button(loc.t("fail.retry")) { m.retryEngineSync() }.disabled(m.engineSyncing)
+ Button(loc.t("fail.retry")) { engine?.retrySync() }.disabled(engine?.syncing ?? false)
default: EmptyView()
}
if kind.overrideFlag != nil {
@@ -91,11 +94,12 @@ struct BuildFailureCard: View {
/// Above every page of the launcher while the engine is fetched and patched (tools/engine.py sync, run for a checkout at launch and after
/// updates), and its failure row when that stops: what happened, what to do, and Try again.
struct EngineSyncBanner: View {
- @ObservedObject var m: BuildModel
+ @ObservedObject var engine: EngineSyncStore
+ let m: BuildModel
@EnvironmentObject var loc: Localizer
var body: some View {
- if m.engineSyncing {
+ if engine.syncing {
HStack(spacing: 10) {
ProgressView().controlSize(.small)
Text(loc.t("engine.syncing")).font(.manrope(13)).foregroundStyle(LauncherPalette.building)
@@ -104,8 +108,8 @@ struct EngineSyncBanner: View {
}
.padding(.horizontal, 24).padding(.vertical, 10)
.accessibilityIdentifier("engine.syncing")
- } else if let f = m.engineFailure {
- BuildFailureCard(m: m, failure: f).padding(.horizontal, 24).padding(.vertical, 10)
+ } else if let f = engine.failure {
+ BuildFailureCard(m: m, engine: engine, failure: f).padding(.horizontal, 24).padding(.vertical, 10)
}
}
}
diff --git a/frontend/Sources/NX2Apple/ContentView.swift b/frontend/Sources/NX2Apple/ContentView.swift
index e7078022..a6f9837c 100644
--- a/frontend/Sources/NX2Apple/ContentView.swift
+++ b/frontend/Sources/NX2Apple/ContentView.swift
@@ -56,7 +56,7 @@ struct Slot: Identifiable {
// MARK: - Model
-final class BuildModel: ObservableObject {
+final class BuildModel: ObservableObject, PythonLocation {
@Published var gameDir = ""
@Published var slots: [Slot] = [
Slot(id: "base", kind: .base, title: "data.base", optional: false),
@@ -206,9 +206,9 @@ final class BuildModel: ObservableObject {
let store = RepoStore(
localizer: localizer, git: gitCheckout,
buildRunning: { [unowned self] in running },
- isReleaseApp: { [unowned self] in usesEngineStore },
+ isReleaseApp: { [unowned self] in MainActor.assumeIsolated { engineSync.usesEngineStore } },
log: { [unowned self] in append($0) },
- didMove: { [unowned self] in syncEngine() }) // the pull may have moved the pin or the patches
+ didMove: { [unowned self] in MainActor.assumeIsolated { engineSync.syncEngine() } }) // the pull may have moved the pin or the patches
// The build state that reads the repository (`canBuild`, the setup rows) is still here: a change of the repository
// redraws what observes this model, as it did when the repository's state was published by it. Goes with the build store.
repoChanges = store.objectWillChange.sink { [unowned self] _ in objectWillChange.send() }
@@ -224,23 +224,25 @@ final class BuildModel: ObservableObject {
var repoOK: Bool { MainActor.assumeIsolated { repo.isRepo } }
var repoIsGit: Bool { MainActor.assumeIsolated { repo.isGit } }
- // ---- The release app's engine (EngineUpdater.swift, EngineUpdateModel.swift) ----
-
- /// The engine tree inside the app bundle. nil for a launcher built from the repository, which keeps its checkout and git.
- /// A release app builds from the engine store instead, and never clones or pulls.
- var engineBundle: URL? = EngineBundle.tree(in: Bundle.main.resourceURL)
- var engineStore = EngineStore()
- var engineTransport: EngineTransport = URLSessionEngineTransport()
- @Published var engineSelection = EngineSelection.load()
- @Published var engineBusy = false
- @Published var engineStatus = ""
- /// A newer engine the launch check found while automatic updates are off: Settings offers it.
- @Published var engineAvailable: EngineUpdate.Engine?
- /// The channel the player picked and is asked to confirm (the switch can change the shader chain).
- @Published var confirmingChannel: EngineChannel?
- /// An installed engine that waits for the running build to end before the build tools switch to it.
- var pendingEngine: (engine: EngineUpdate.Engine, channel: EngineChannel)?
- var usesEngineStore: Bool { engineBundle != nil }
+ // ---- The engine (`EngineSyncStore`, NXModel) ----
+ // It owns the engine selection, the sync of a checkout's engine/ and the engine switch of the release app. The model creates it
+ // with its services and wires what reaches back here: the repository and the interpreter to read, where the build's tools
+ // are read from, whether a build runs, the automatic update and the log.
+
+ /// Created on first use, on the main thread, because its closures reach back to this model.
+ private(set) lazy var engineSync: EngineSyncStore = MainActor.assumeIsolated {
+ let store = EngineSyncStore(
+ localizer: localizer, runner: runner, repo: repo, python: self,
+ setRepoPath: { [unowned self] in repoPath = $0 },
+ buildRunning: { [unowned self] in running },
+ autoUpdate: { [unowned self] in MainActor.assumeIsolated { repo.autoUpdate } },
+ log: { [unowned self] in append($0) })
+ // The build state that reads the engine's paths (`canBuild`, the setup rows) is still here: a change of the engine redraws
+ // what observes this model, as it did when the engine's state was published by it. Goes with the build store.
+ engineChanges = store.objectWillChange.sink { [unowned self] _ in objectWillChange.send() }
+ return store
+ }
+ private var engineChanges: AnyCancellable?
/// The PATH of the processes that run the build, installing packages and writing the RomFS image: the bundled tools in front of
/// `base` (BundledTools.swift), which is Homebrew's folders and the system's. One place, so that no environment forgets the bundle.
@@ -248,48 +250,6 @@ final class BuildModel: ObservableObject {
BundledTools.path(base, python: python)
}
- /// What waits for the running `tools/engine.py sync` (nil: none runs), and whether another one is due when it ends.
- private var engineSyncWaiting: [(Bool) -> Void]?
- private var engineSyncAgain = false
- /// A sync runs: the launcher shows a status line with a spinner (EngineSyncBanner).
- @Published var engineSyncing = false
- /// How the last sync stopped (no network or GitHub unreachable, or another stop), as a failure row with a retry button. nil: none.
- @Published var engineFailure: BuildFailure?
-
- /// `tools/engine.py sync` when engine/ is stale (#214, Engine.swift): at launch, after a clone, a pull or an adoption, and before a
- /// build that has no other engine. Checkouts only: a release app carries its engine tree (#227) and never syncs. A no-op for a current engine/ or a folder
- /// that is an engine tree. `then(ok)` runs on the main queue.
- func syncEngine(then: ((Bool) -> Void)? = nil) {
- if engineSyncWaiting != nil { engineSyncAgain = true; if let then { engineSyncWaiting?.append(then) }; return }
- guard EngineLayout.isStale(repoPath) else { engineFailure = nil; then?(true); return }
- engineSyncWaiting = then.map { [$0] } ?? []
- engineSyncing = true; engineFailure = nil
- let repo = repoPath, chosen = pythonPath
- append("$ python3 tools/engine.py sync\n")
- var tail = "" // what failureKind reads when the sync stops (only touched on the main queue)
- DispatchQueue.global().async {
- let py = chosen.isEmpty ? Toolchain.findPython().path : chosen
- let ok = EngineLayout.runSync(repo: repo, python: py, runner: self.runner) { line in
- DispatchQueue.main.async { self.append(line + "\n"); tail = String((tail + line + "\n").suffix(4000)) }
- }
- DispatchQueue.main.async {
- let waiting = self.engineSyncWaiting ?? [], again = self.engineSyncAgain && ok
- self.engineSyncWaiting = nil; self.engineSyncAgain = false
- self.engineSyncing = false
- self.engineFailure = ok ? nil : BuildFailure(kind: EngineLayout.failureKind(tail), gameName: nil)
- if again { self.syncEngine { ok in waiting.forEach { $0(ok) } } } else { waiting.forEach { $0(ok) } }
- self.objectWillChange.send() // repoOK and the engine's paths may have changed
- }
- }
- }
-
- /// The failure row's Try again: clears the row and runs the sync again.
- func retryEngineSync() {
- guard !engineSyncing else { return }
- engineFailure = nil
- syncEngine()
- }
-
// ---- App self-update (GitHub Releases) ----
// `AppUpdateStore` (NXModel) owns it: the check, the staged update, the update sheet, the uninstall plan and the version.
// The model creates it with its services and wires the two things that need AppKit, quitting and the quit notification.
@@ -427,10 +387,12 @@ final class BuildModel: ObservableObject {
let py = pythonPath.isEmpty ? Toolchain.findPython().path : pythonPath
// engine/ is the only engine and is stale or missing (#214): make it, then start again
if EngineLayout.needsSyncBeforeBuild(repoPath) {
- syncEngine { ok in
- self.running = false
- if ok && !EngineLayout.needsSyncBeforeBuild(self.repoPath) { self.startBuild() }
- else { self.fail(String(format: self.t("msg.toolsOld"), "engine/ (python3 tools/engine.py sync, see the log)")) }
+ MainActor.assumeIsolated {
+ engineSync.syncEngine { ok in
+ self.running = false
+ if ok && !EngineLayout.needsSyncBeforeBuild(self.repoPath) { self.startBuild() }
+ else { self.fail(String(format: self.t("msg.toolsOld"), "engine/ (python3 tools/engine.py sync, see the log)")) }
+ }
}
return
}
@@ -1575,6 +1537,7 @@ struct BuilderView: View {
@ObservedObject var m: BuildModel
@ObservedObject var updates: AppUpdateStore // the app's self-update (Settings, "NX2Apple" section)
@ObservedObject var repo: RepoStore // the repository (Settings, "Tools" section, and the setup hint)
+ @ObservedObject var engine: EngineSyncStore // the engine (Settings, "Tools" section)
@EnvironmentObject var loc: Localizer
@ObservedObject var library: GameLibrary
var settings = false
@@ -2097,9 +2060,9 @@ struct BuilderView: View {
}
private var tools: some View {
- group(loc.t("section.tools"), footnote: m.usesEngineStore ? m.engineStatus
+ group(loc.t("section.tools"), footnote: engine.usesEngineStore ? engine.status
: (repo.status.isEmpty ? (repo.isRepo ? "" : loc.t("repo.firstHint")) : repo.status)) {
- if m.usesEngineStore {
+ if engine.usesEngineStore {
engineRows
} else {
row(loc.t("repo.title"),
@@ -2203,39 +2166,39 @@ struct BuilderView: View {
}
}
- /// The release app's engine (EngineUpdateModel.swift): which one the build uses, its channel, and the automatic update.
+ /// The release app's engine (EngineSyncStore): which one the build uses, its channel, and the automatic update.
@ViewBuilder private var engineRows: some View {
- let idle = !(m.engineBusy || m.running)
- row(loc.t("engine.title"), detail: m.engineDetail,
- dot: m.engineSelection.engineID.isEmpty ? LauncherPalette.building : LauncherPalette.ready, help: repo.path) {
- if m.engineBusy { ProgressView().controlSize(.small) }
- if let next = m.engineAvailable {
- Button(String(format: loc.t("engine.install"), next.version)) { m.installAvailableEngine() }.disabled(!idle)
- .padTarget("set.engineInstall", scope: "main") { if idle { m.installAvailableEngine() } }
+ let idle = !(engine.busy || m.running)
+ row(loc.t("engine.title"), detail: engine.detail,
+ dot: engine.selection.engineID.isEmpty ? LauncherPalette.building : LauncherPalette.ready, help: repo.path) {
+ if engine.busy { ProgressView().controlSize(.small) }
+ if let next = engine.available {
+ Button(String(format: loc.t("engine.install"), next.version)) { engine.installAvailable() }.disabled(!idle)
+ .padTarget("set.engineInstall", scope: "main") { if idle { engine.installAvailable() } }
}
- Button(loc.t("repo.check")) { m.checkEngineUpdate() }.disabled(!idle)
- .padTarget("set.engineCheck", scope: "main") { if idle { m.checkEngineUpdate() } }
+ Button(loc.t("repo.check")) { engine.checkForUpdate() }.disabled(!idle)
+ .padTarget("set.engineCheck", scope: "main") { if idle { engine.checkForUpdate() } }
}
divider
// A different channel is asked about first (the alert below): the switch can recompile every game's shaders
row(loc.t("engine.update"), detail: loc.t("engine.channelNote"), dot: LauncherPalette.ready) {
LauncherDropdown(options: EngineChannel.allCases.map { (value: $0.rawValue, label: loc.t($0.labelKey)) },
- selection: m.engineSelection.channel.rawValue) { v in
- if let channel = EngineChannel(rawValue: v) { m.requestEngineChannel(channel) }
+ selection: engine.selection.channel.rawValue) { v in
+ if let channel = EngineChannel(rawValue: v) { engine.requestChannel(channel) }
}
.disabled(!idle)
.padTarget("set.engineChannel", scope: "main") {
- if idle { m.requestEngineChannel(m.engineSelection.channel == .stable ? .experimental : .stable) }
+ if idle { engine.requestChannel(engine.selection.channel == .stable ? .experimental : .stable) }
}
}
- .alert(loc.t("engine.switch.title"), isPresented: Binding(get: { m.confirmingChannel != nil },
- set: { if !$0 { m.cancelEngineChannel() } })) {
- if let channel = m.confirmingChannel {
- Button(String(format: loc.t("engine.switch.go"), loc.t(channel.labelKey))) { m.confirmEngineChannel(channel) }
+ .alert(loc.t("engine.switch.title"), isPresented: Binding(get: { engine.confirmingChannel != nil },
+ set: { if !$0 { engine.cancelChannel() } })) {
+ if let channel = engine.confirmingChannel {
+ Button(String(format: loc.t("engine.switch.go"), loc.t(channel.labelKey))) { engine.confirmChannel(channel) }
}
- Button(loc.t("engine.switch.cancel"), role: .cancel) { m.cancelEngineChannel() }
+ Button(loc.t("engine.switch.cancel"), role: .cancel) { engine.cancelChannel() }
} message: {
- Text(m.confirmingChannel.flatMap { m.engineSelection.channel.switchWarningKey(to: $0) }.map { loc.t($0) } ?? "")
+ Text(engine.confirmingChannel.flatMap { engine.selection.channel.switchWarningKey(to: $0) }.map { loc.t($0) } ?? "")
}
divider
row(loc.t("shellCache.row"), detail: loc.t("shellCache.note"), detail2: m.shellCachePath) {
diff --git a/frontend/Sources/NX2Apple/Diagnostics.swift b/frontend/Sources/NX2Apple/Diagnostics.swift
index 2e5a5df6..4da29fc9 100644
--- a/frontend/Sources/NX2Apple/Diagnostics.swift
+++ b/frontend/Sources/NX2Apple/Diagnostics.swift
@@ -300,7 +300,7 @@ extension BuildModel {
let repo = repoOK ? repoPath : "", python = pythonPath.isEmpty ? Toolchain.findPython().path : pythonPath, dotnet = dotnetPath
let rows = depsChecked ? deps.map(Diagnostics.dependency) : [], checked = depsChecked
let command = lastBuildCommand, log = logTail, xcrun = xcrunPath
- let sel = engineSelection
+ let (sel, usesEngineStore) = MainActor.assumeIsolated { (engineSync.selection, engineSync.usesEngineStore) }
let engine = usesEngineStore && !sel.engineID.isEmpty ? "\(sel.channel.rawValue) \(sel.version) (\(sel.engineID))" : nil
DispatchQueue.global(qos: .userInitiated).async {
let git = Diagnostics.git(repo)
diff --git a/frontend/Sources/NX2Apple/Engine.swift b/frontend/Sources/NX2Apple/Engine.swift
deleted file mode 100644
index 9057972b..00000000
--- a/frontend/Sources/NX2Apple/Engine.swift
+++ /dev/null
@@ -1,95 +0,0 @@
-import CryptoKit
-import Foundation
-import NXCore
-import NXLauncher
-
-/// Where the engine of a checkout is, and whether it needs `tools/engine.py sync` (LNPANDY/NX2Apple#214).
-///
-/// The engine is not committed: it is built into `<checkout>/engine/` (seedds/nxnative at `.upstream-ref` plus `patches/`) by
-/// `tools/engine.py sync`. The rule, the same as the Qt front end's (frontend-qt/nspb_qt/engine.py):
-///
-/// - a checkout of this repository (it has `tools/engine.py` and `.upstream-ref`): `engine/`, which the build syncs first when it is
-/// stale or missing,
-/// - else a folder that is an engine tree itself (`dm/` at its root: the engine a release carries or installs, #227, or a
-/// seedds/nxnative clone): that folder,
-/// - else `engine/` as it is.
-///
-/// `stampKey` is tools/engine.py's digest of the pin and the series, compared with the stamp sync wrote into engine/.git, so the
-/// launcher tells a stale engine without starting Python. The sync is for a repository checkout (a developer's,
-/// or one a launcher cloned): it downloads the pinned engine from GitHub (about 80 MB) and shows a failure row when that does not work.
-/// A release app carries the engine tree instead (`Contents/Resources/engine`, built by `tools/engine.py tree`, #227) and takes updates
-/// from the release's `latest.json`: no git, no sync.
-extension EngineLayout {
- // `marker`, `hasEngineTool`, `root` and `isCheckout` are in NXCore (EngineLayout.swift): the repository store needs them.
-
- static func series(_ repo: String) -> [String]? {
- guard let text = try? String(contentsOfFile: repo + "/patches/series", encoding: .utf8) else { return nil }
- return text.split(separator: "\n", omittingEmptySubsequences: false).compactMap { line in
- let s = (line.split(separator: "#", maxSplits: 1, omittingEmptySubsequences: false).first ?? "").trimmingCharacters(in: .whitespaces)
- return s.isEmpty ? nil : s
- }
- }
-
- static func hex(_ d: SHA256.Digest) -> String { d.map { String(format: "%02x", $0) }.joined() }
-
- /// tools/engine.py's stamp_key: sha256 of "pin <sha>\n", then "patch <name>\n<sha256 of the file>\n" for each patch in order.
- static func stampKey(_ repo: String) -> String? {
- guard let pin = try? String(contentsOfFile: repo + "/.upstream-ref", encoding: .utf8), let names = series(repo) else { return nil }
- var h = SHA256()
- h.update(data: Data(("pin " + pin.trimmingCharacters(in: .whitespacesAndNewlines) + "\n").utf8))
- for name in names {
- guard let data = FileManager.default.contents(atPath: repo + "/patches/" + name) else { return nil }
- h.update(data: Data(("patch " + name + "\n" + hex(SHA256.hash(data: data)) + "\n").utf8))
- }
- return hex(h.finalize())
- }
-
- /// HEAD of engine/ read from its files (no git process): the detached id, or the id its branch points to.
- static func head(_ eng: String) -> String {
- func read(_ p: String) -> String? { (try? String(contentsOfFile: eng + "/.git/" + p, encoding: .utf8))?.trimmingCharacters(in: .whitespacesAndNewlines) }
- guard let h = read("HEAD") else { return "" }
- guard h.hasPrefix("ref: ") else { return h }
- let ref = String(h.dropFirst(5))
- if let id = read(ref) { return id }
- for line in (read("packed-refs") ?? "").split(separator: "\n") where line.hasSuffix(" " + ref) {
- return String(line.split(separator: " ").first ?? "")
- }
- return ""
- }
-
- /// True when the checkout can make engine/ and engine/ is missing or not built from the current pin and series.
- static func isStale(_ repo: String) -> Bool {
- guard hasEngineTool(repo) else { return false }
- let eng = repo + "/engine"
- guard let text = try? String(contentsOfFile: eng + "/.git/nx2apple-stamp", encoding: .utf8) else { return true }
- var stamp: [String: String] = [:]
- for line in text.split(separator: "\n") {
- let kv = line.split(separator: " ", maxSplits: 1)
- if kv.count == 2 { stamp[String(kv[0])] = String(kv[1]) }
- }
- return stamp["key"] != stampKey(repo) || stamp["head"] != head(eng)
- }
-
- /// engine/ is the only engine and it is stale or missing: the build syncs it first.
- static func needsSyncBeforeBuild(_ repo: String) -> Bool {
- hasEngineTool(repo) && isStale(repo)
- }
-
- /// `python tools/engine.py sync`
- static func syncArguments(repo: String, python: String) -> [String] {
- [python, repo + "/tools/engine.py", "sync"]
- }
-
- /// What a failed sync was, from its output: the fetch of the pinned commit (no network, GitHub unreachable, a proxy) says
- /// "cannot fetch" (tools/engine.py), anything else is a sync that could not build the engine.
- static func failureKind(_ log: String) -> BuildFailureKind {
- log.contains("cannot fetch ") ? .engineOffline : .engineSync
- }
-
- /// Run the sync and stream its lines (blocking: call it off the main thread). git comes from the system's or Homebrew's PATH.
- static func runSync(repo: String, python: String, runner: ProcessRunner = SystemProcessRunner(), line: @escaping (String) -> Void) -> Bool {
- let env = ["PATH": BuildModel.toolPath("/usr/bin:/bin:/opt/homebrew/bin:/usr/local/bin", python: python), "HOME": NSHomeDirectory(),
- "LANG": "en_US.UTF-8", "TMPDIR": NSTemporaryDirectory()].merging(BundledTools.environment(for: python)) { $1 }
- return runner.runCaptured(syncArguments(repo: repo, python: python), env: env, line: line)
- }
-}
diff --git a/frontend/Sources/NX2Apple/EngineUpdateModel.swift b/frontend/Sources/NX2Apple/EngineUpdateModel.swift
deleted file mode 100644
index 96e94841..00000000
--- a/frontend/Sources/NX2Apple/EngineUpdateModel.swift
+++ /dev/null
@@ -1,151 +0,0 @@
-import Foundation
-import NXCore
-import NXModel
-import NXLauncher
-
-// The release app's engine in the model: which tree the build reads, the launch check, the channel switch and what
-// Settings says. The rules (what is verified, where a tree comes from) are in EngineUpdater.swift. A launcher built
-// from the repository has no engine in its bundle (`engineBundle` is nil), and everything here leaves it alone.
-extension BuildModel {
- /// At launch: build from the selected engine in the store, copy the app's own engine there on the first start,
- /// then ask the selected channel for a newer one. On the stable channel a new app brings its own engine, which
- /// then replaces the one an older app left selected, without the network.
- func prepareEngine() {
- guard let bundled = engineBundle, !engineBusy, let id = EngineStore.identity(of: bundled) else { return }
- let selected = engineSelection.engineID
- let appMovedOn = engineSelection.channel == .stable && selected != id && engineSelection.version != AppUpdateStore.appVersion
- if !appMovedOn, engineStore.isUsable(selected), let dir = engineStore.directory(for: selected) {
- repoPath = dir.path
- checkEngineUpdate(automatic: true)
- return
- }
- engineBusy = true
- engineStatus = t("engine.preparing")
- let store = engineStore
- DispatchQueue.global(qos: .userInitiated).async {
- let result = Result { try EngineInstaller.seed(from: bundled, engineID: id, store: store) }
- DispatchQueue.main.async {
- self.engineBusy = false
- switch result {
- case .success:
- // The app's engine is the stable one of this release. A player on experimental gets theirs from the check.
- let engine = EngineUpdate.Engine(id: id, version: AppUpdateStore.appVersion)
- if self.running { // a build started from the old folder while the copy ran
- self.pendingEngine = (engine, self.engineSelection.channel)
- self.engineStatus = ""
- } else {
- self.switchEngine(to: engine, channel: self.engineSelection.channel, announce: false)
- self.checkEngineUpdate(automatic: true)
- }
- case .failure:
- self.engineStatus = self.t("engine.seedFail")
- }
- }
- }
- }
-
- /// Ask the selected channel for its engine. `automatic` is the launch check: it installs only with automatic
- /// updates on, and it says nothing when the engine is current or the network is down.
- func checkEngineUpdate(automatic: Bool = false) {
- runEngineUpdate(channel: engineSelection.channel, install: automatic ? MainActor.assumeIsolated { repo.autoUpdate } : true, quiet: automatic)
- }
-
- /// Install the engine the launch check found (automatic updates off).
- func installAvailableEngine() {
- runEngineUpdate(channel: engineSelection.channel, install: true, quiet: false)
- }
-
- /// The channel picker: a different channel is confirmed first, because the switch can change the shader chain.
- func requestEngineChannel(_ channel: EngineChannel) {
- guard engineSelection.channel.switchWarningKey(to: channel) != nil, !engineBusy, !running else { return }
- confirmingChannel = channel
- }
-
- /// The alert's button passes the channel it showed, so the order of the alert's own dismissal does not matter.
- func confirmEngineChannel(_ channel: EngineChannel) {
- confirmingChannel = nil
- runEngineUpdate(channel: channel, install: true, quiet: false)
- }
-
- func cancelEngineChannel() { confirmingChannel = nil }
-
- /// The selected channel is only changed once its engine is in place, so Settings never names a channel whose
- /// engine the build does not use.
- private func runEngineUpdate(channel: EngineChannel, install: Bool, quiet: Bool) {
- guard usesEngineStore, !engineBusy else { return }
- guard !running else {
- if !quiet { engineStatus = t("engine.waitForBuild") }
- return
- }
- engineBusy = true
- if !quiet { engineStatus = t(channel == engineSelection.channel ? "engine.checking" : "engine.switching") }
- let store = engineStore, bundled = engineBundle, current = engineSelection.engineID, transport = engineTransport
- DispatchQueue.global(qos: .userInitiated).async {
- let outcome = EngineUpdate.run(channel: channel, appVersion: AppUpdateStore.appVersion, current: current, store: store,
- bundled: bundled, install: install, transport: transport)
- DispatchQueue.main.async {
- self.engineBusy = false
- self.applyEngineOutcome(outcome, channel: channel, quiet: quiet)
- }
- }
- }
-
- func applyEngineOutcome(_ outcome: EngineUpdate.Outcome, channel: EngineChannel, quiet: Bool) {
- switch outcome {
- case .current(let engine):
- engineAvailable = nil
- if engineSelection.channel != channel || engineSelection.version != engine.version {
- engineSelection.channel = channel
- engineSelection.version = engine.version
- engineSelection.save()
- }
- if !quiet { engineStatus = String(format: t("engine.upToDate"), engine.version) }
- case .available(let engine):
- engineAvailable = engine
- engineStatus = String(format: t("engine.available"), engine.version)
- case .installed(let engine, _):
- engineAvailable = nil
- if running {
- pendingEngine = (engine, channel)
- engineStatus = String(format: t("engine.waitsForBuild"), engine.version)
- } else {
- switchEngine(to: engine, channel: channel)
- }
- case .failed(let failure):
- if quiet, failure == .network || failure == .noManifest { return }
- let key = EngineUpdate.key(failure)
- engineStatus = failure == .noManifest ? String(format: t(key), t(channel.labelKey)) : t(key)
- }
- }
-
- /// Point the build at an installed tree, keep the one before it, and say whether the shader chain changed.
- /// `announce` off (the app's own engine at launch): only a changed shader chain is worth a line.
- func switchEngine(to engine: EngineUpdate.Engine, channel: EngineChannel, announce: Bool = true) {
- guard let dir = engineStore.directory(for: engine.id) else { return }
- let old = repoPath
- engineSelection.channel = channel
- engineSelection.switched(to: engine.id, version: engine.version)
- engineSelection.save()
- repoPath = dir.path
- engineStore.prune(keeping: engineSelection.kept)
- if let change = EngineUpdate.chainChange(from: old, to: dir.path) {
- engineStatus = String(format: t("engine.chainChanged"), engine.version, change.old, change.new)
- } else {
- engineStatus = announce ? String(format: t("engine.installed"), engine.version) : ""
- }
- }
-
- /// The build that held an installed engine back has ended (LauncherView watches `running`).
- func applyPendingEngine() {
- guard !running, let pending = pendingEngine else { return }
- pendingEngine = nil
- switchEngine(to: pending.engine, channel: pending.channel)
- }
-
- /// What Settings shows for the engine: its version and the pin it was built from.
- var engineDetail: String {
- let s = engineSelection
- guard !s.engineID.isEmpty else { return t("engine.none") }
- return String(format: t("engine.detail"), s.version.isEmpty ? "?" : s.version, String(s.engineID.prefix(8)))
- }
-}
diff --git a/frontend/Sources/NX2Apple/FreeSpace.swift b/frontend/Sources/NX2Apple/FreeSpace.swift
index 4b408b02..685d4bda 100644
--- a/frontend/Sources/NX2Apple/FreeSpace.swift
+++ b/frontend/Sources/NX2Apple/FreeSpace.swift
@@ -97,14 +97,7 @@ enum BuildFolder {
/// The shader chain version of the checkout the launcher builds with: DMSH_CHAIN in dm/gfx/nvn/dmsh_chain.h, read as
/// the NSP script's chain_of() does (`#define DMSH_CHAIN "(.*)"`). nil when the header is missing or has none, and then no
/// shader version counts as old.
- nonisolated static func currentChain(repo: String) -> String? {
- guard !repo.isEmpty, let text = try? String(contentsOfFile: EngineLayout.root(repo) + "/dm/gfx/nvn/dmsh_chain.h", encoding: .utf8),
- let range = text.range(of: #"#define DMSH_CHAIN "(.*)""#, options: .regularExpression) else { return nil }
- let line = text[range]
- guard let open = line.firstIndex(of: "\""), let close = line.lastIndex(of: "\""), open < close else { return nil }
- let chain = String(line[line.index(after: open)..<close])
- return chain.isEmpty ? nil : chain
- }
+ nonisolated static func currentChain(repo: String) -> String? { EngineLayout.shaderChain(repo: repo) }
/// The entries of the build folder that hold the libraries of chain versions other than `chain`, relative to it:
/// shader/cache/b-<old> (Mac) and shader/ios/b-<old> (iPhone) with shader/ios/b-<old>.key and shader/mac-b-<old>.key, the
diff --git a/frontend/Sources/NX2Apple/LauncherView.swift b/frontend/Sources/NX2Apple/LauncherView.swift
index db3b4fe8..455615df 100644
--- a/frontend/Sources/NX2Apple/LauncherView.swift
+++ b/frontend/Sources/NX2Apple/LauncherView.swift
@@ -45,6 +45,7 @@ struct ContentView: View {
@ObservedObject private var loc: Localizer // the strings: handed on to every view below through the environment
@ObservedObject private var updates: AppUpdateStore // the app's self-update: the update icon, the sheet, the uninstall dialog
@ObservedObject private var repo: RepoStore // the repository: the activity line, and the checks that follow a change of it
+ @ObservedObject private var engine: EngineSyncStore // the engine: the sync banner, and the checks that follow a sync
@StateObject private var library: GameLibrary
@State private var page: LauncherPage = .library
@State private var search = ""
@@ -99,7 +100,7 @@ struct ContentView: View {
_m = StateObject(wrappedValue: model)
_loc = ObservedObject(wrappedValue: model.localizer)
_updates = ObservedObject(wrappedValue: model.appUpdates)
- _repo = ObservedObject(wrappedValue: model.repo)
+ _repo = ObservedObject(wrappedValue: model.repo); _engine = ObservedObject(wrappedValue: model.engineSync)
_library = StateObject(wrappedValue: library ?? GameLibrary())
_page = State(initialValue: initialPage)
_activeBuildID = State(initialValue: activeBuildID)
@@ -172,11 +173,11 @@ struct ContentView: View {
VStack(spacing: 0) {
topBar
Divider().opacity(0.3)
- EngineSyncBanner(m: m)
+ EngineSyncBanner(engine: engine, m: m)
if page == .library {
libraryPage
} else {
- BuilderView(m: m, updates: updates, repo: repo, library: library, settings: page == .settings, onViewGame: { page = .library },
+ BuilderView(m: m, updates: updates, repo: repo, engine: engine, library: library, settings: page == .settings, onViewGame: { page = .library },
onOpenSettings: { page = .settings }, onChooseLibraryFolder: chooseLibraryFolder,
onOpenControls: { controlsOpen = true }, onOpenControllers: { controllerSheet = true }, onRunSetup: onRunSetup, controlChanges: controls.changeCount,
padMerge: Binding(get: { controls.padMerge }, set: { setPadMerge($0) }))
@@ -215,8 +216,8 @@ struct ContentView: View {
if performStartupChecks { library.discover(); fetchArtwork(); m.refreshDevices(); checkTools() }
if performStartupChecks {
m.checkDeps()
- if m.usesEngineStore { m.prepareEngine() } // the release app: its engine store, never git (EngineUpdateModel.swift)
- else { m.syncEngine() } // a checkout whose engine/ is missing or older than its pin (#214)
+ if engine.usesEngineStore { engine.prepareEngine() } // the release app: its engine store, never git
+ else { engine.syncEngine() } // a checkout whose engine/ is missing or older than its pin (#214)
if repo.isRepo && repo.isGit { repo.checkForUpdates(silent: true) } else { repo.adoptAtLaunch() }
updates.checkForUpdate(silent: true)
BuildNotifier.shared.watch(m, library: library) // BuildNotifications.swift
@@ -236,8 +237,7 @@ struct ContentView: View {
.onChange(of: repo.path) { _ in refreshArtwork(); if performStartupChecks { checkTools() } }
// A clone, pull or update check has finished (the automatic pull at launch among them): the checkout may have moved
.onReceive(repo.$busy.removeDuplicates().dropFirst()) { busy in if !busy && performStartupChecks { checkTools() } }
- // The engine was made or moved by a sync (its HEAD is what ToolsAge compares)
- .onReceive(m.$engineSyncing.removeDuplicates().dropFirst()) { syncing in if !syncing && performStartupChecks { checkTools() } }
+ .onReceive(engine.$syncing.removeDuplicates().dropFirst()) { syncing in if !syncing && performStartupChecks { checkTools() } } // a sync made or moved the engine (its HEAD is what ToolsAge compares)
.onChange(of: library.selectedID) { _ in resignNote = nil } // the last re-sign's note belongs to its game
.onChange(of: buildProgress) { progress in if m.running { DockProgress.update(progress) } }
.onReceive(m.$running.removeDuplicates()) { running in
@@ -255,7 +255,7 @@ struct ContentView: View {
DispatchQueue.main.async {
if m.done { recordBuild() }
activeBuildID = nil
- m.applyPendingEngine() // an engine installed during the build takes over once it is recorded
+ engine.applyPendingEngine() // an engine installed during the build takes over once it is recorded
}
}
}
diff --git a/frontend/Sources/NXCore/BuildFailureKind.swift b/frontend/Sources/NXCore/BuildFailureKind.swift
new file mode 100644
index 00000000..2edcebae
--- /dev/null
+++ b/frontend/Sources/NXCore/BuildFailureKind.swift
@@ -0,0 +1,50 @@
+import Foundation
+
+/// The early stops of the NSP script that the launcher can turn into a next step. The script prints a
+/// `[RED]` line and exits 1 for each (a `BuildError`), and the launcher used to show only the last line of
+/// the log, cut to 200 characters. The classifier reads the whole log tail, because for a damaged NSP the
+/// last line is only "nx_extract exit code 3" and the message is a few lines above it.
+package enum BuildFailureKind: String, CaseIterable {
+ case rosetta, translator, zbic, iosSDK, cltSelected, otherMetal, shaderRegression, damagedNSP, a32Program
+ /// The inputs: the keys file, the game folder, a drive that is gone.
+ case keys, severalGames, noTitleID, driveMissing
+ /// The build's and the install's own steps.
+ case diskFull, teamTaken, noSigningAccount, deviceTransfer
+ /// The build folder is in iCloud Drive or another cloud folder (CloudFolder.swift): the launcher stops before the build, and the
+ /// codesign line of xcodebuild is the same stop for a build that got past that check.
+ case cloudFolder
+ /// The shared app's install: a device this Mac never paired, and game data that went over only in part.
+ case devicePairing, shellDataIncomplete
+ /// The engine's online sync (tools/engine.py sync, EngineLayout.swift): no network or GitHub unreachable, or any other stop of it.
+ case engineOffline, engineSync
+ case unknown
+
+ /// The script's override for this stop, offered as "Build anyway". Only these three have one that the launcher
+ /// may offer: the others cannot be built around (a 32-bit program, a missing SDK).
+ package var overrideFlag: String? {
+ switch self {
+ case .otherMetal: return "--allow-other-metal"
+ case .shaderRegression: return "--allow-shader-regressions"
+ case .damagedNSP: return "--allow-damaged"
+ default: return nil
+ }
+ }
+
+ /// L10n keys: what happened, and the next step.
+ package var whatKey: String { "fail.\(rawValue).what" }
+ package var nextKey: String { "fail.\(rawValue).next" }
+}
+
+package struct BuildFailure: Equatable {
+ package var kind: BuildFailureKind
+ package var gameName: String? // a32Program: the game, when the message names it
+ /// What the message adds to its row: the GB a full disk needs (diskFull), the drive's name (driveMissing), the folder
+ /// of "no NSP with a readable title ID" (noTitleID).
+ package var detail: String? = nil
+
+ package init(kind: BuildFailureKind, gameName: String?, detail: String? = nil) {
+ self.kind = kind
+ self.gameName = gameName
+ self.detail = detail
+ }
+}
diff --git a/frontend/Sources/NXCore/EngineLayout.swift b/frontend/Sources/NXCore/EngineLayout.swift
index edb3961a..a1b851d3 100644
--- a/frontend/Sources/NXCore/EngineLayout.swift
+++ b/frontend/Sources/NXCore/EngineLayout.swift
@@ -1,7 +1,22 @@
+import CryptoKit
import Foundation
-/// Where the engine of a checkout is: the part of `EngineLayout` that needs only the file system (the rest, the stale check and the
-/// sync, is in the app target, `Engine.swift`, which explains the rule). `RepoLayout` and `RepoStore` need these.
+/// Where the engine of a checkout is, and whether it needs `tools/engine.py sync` (LNPANDY/NX2Apple#214).
+///
+/// The engine is not committed: it is built into `<checkout>/engine/` (seedds/nxnative at `.upstream-ref` plus `patches/`) by
+/// `tools/engine.py sync`. The rule, the same as the Qt front end's (frontend-qt/nspb_qt/engine.py):
+///
+/// - a checkout of this repository (it has `tools/engine.py` and `.upstream-ref`): `engine/`, which the build syncs first when it is
+/// stale or missing,
+/// - else a folder that is an engine tree itself (`dm/` at its root: the engine a release carries or installs, #227, or a
+/// seedds/nxnative clone): that folder,
+/// - else `engine/` as it is.
+///
+/// `stampKey` is tools/engine.py's digest of the pin and the series, compared with the stamp sync wrote into engine/.git, so the
+/// launcher tells a stale engine without starting Python. The sync is for a repository checkout (a developer's,
+/// or one a launcher cloned): it downloads the pinned engine from GitHub (about 80 MB) and shows a failure row when that does not work.
+/// A release app carries the engine tree instead (`Contents/Resources/engine`, built by `tools/engine.py tree`, #227) and takes updates
+/// from the release's `latest.json`: no git, no sync.
package enum EngineLayout {
/// The file that tells an engine tree.
package static let marker = "dm/tools/nx_extract/Program.cs"
@@ -26,4 +41,86 @@ package enum EngineLayout {
let fm = FileManager.default
return !dir.isEmpty && (fm.fileExists(atPath: dir + "/" + marker) || fm.fileExists(atPath: dir + "/engine/" + marker) || hasEngineTool(dir))
}
+
+ package static func series(_ repo: String) -> [String]? {
+ guard let text = try? String(contentsOfFile: repo + "/patches/series", encoding: .utf8) else { return nil }
+ return text.split(separator: "\n", omittingEmptySubsequences: false).compactMap { line in
+ let s = (line.split(separator: "#", maxSplits: 1, omittingEmptySubsequences: false).first ?? "").trimmingCharacters(in: .whitespaces)
+ return s.isEmpty ? nil : s
+ }
+ }
+
+ static func hex(_ d: SHA256.Digest) -> String { d.map { String(format: "%02x", $0) }.joined() }
+
+ /// tools/engine.py's stamp_key: sha256 of "pin <sha>\n", then "patch <name>\n<sha256 of the file>\n" for each patch in order.
+ package static func stampKey(_ repo: String) -> String? {
+ guard let pin = try? String(contentsOfFile: repo + "/.upstream-ref", encoding: .utf8), let names = series(repo) else { return nil }
+ var h = SHA256()
+ h.update(data: Data(("pin " + pin.trimmingCharacters(in: .whitespacesAndNewlines) + "\n").utf8))
+ for name in names {
+ guard let data = FileManager.default.contents(atPath: repo + "/patches/" + name) else { return nil }
+ h.update(data: Data(("patch " + name + "\n" + hex(SHA256.hash(data: data)) + "\n").utf8))
+ }
+ return hex(h.finalize())
+ }
+
+ /// HEAD of engine/ read from its files (no git process): the detached id, or the id its branch points to.
+ package static func head(_ eng: String) -> String {
+ func read(_ p: String) -> String? { (try? String(contentsOfFile: eng + "/.git/" + p, encoding: .utf8))?.trimmingCharacters(in: .whitespacesAndNewlines) }
+ guard let h = read("HEAD") else { return "" }
+ guard h.hasPrefix("ref: ") else { return h }
+ let ref = String(h.dropFirst(5))
+ if let id = read(ref) { return id }
+ for line in (read("packed-refs") ?? "").split(separator: "\n") where line.hasSuffix(" " + ref) {
+ return String(line.split(separator: " ").first ?? "")
+ }
+ return ""
+ }
+
+ /// True when the checkout can make engine/ and engine/ is missing or not built from the current pin and series.
+ package static func isStale(_ repo: String) -> Bool {
+ guard hasEngineTool(repo) else { return false }
+ let eng = repo + "/engine"
+ guard let text = try? String(contentsOfFile: eng + "/.git/nx2apple-stamp", encoding: .utf8) else { return true }
+ var stamp: [String: String] = [:]
+ for line in text.split(separator: "\n") {
+ let kv = line.split(separator: " ", maxSplits: 1)
+ if kv.count == 2 { stamp[String(kv[0])] = String(kv[1]) }
+ }
+ return stamp["key"] != stampKey(repo) || stamp["head"] != head(eng)
+ }
+
+ /// engine/ is the only engine and it is stale or missing: the build syncs it first.
+ package static func needsSyncBeforeBuild(_ repo: String) -> Bool {
+ hasEngineTool(repo) && isStale(repo)
+ }
+
+ /// `python tools/engine.py sync`
+ package static func syncArguments(repo: String, python: String) -> [String] {
+ [python, repo + "/tools/engine.py", "sync"]
+ }
+
+ /// What a failed sync was, from its output: the fetch of the pinned commit (no network, GitHub unreachable, a proxy) says
+ /// "cannot fetch" (tools/engine.py), anything else is a sync that could not build the engine.
+ package static func failureKind(_ log: String) -> BuildFailureKind {
+ log.contains("cannot fetch ") ? .engineOffline : .engineSync
+ }
+
+ /// Run the sync and stream its lines (blocking: call it off the main thread). git comes from the system's or Homebrew's PATH.
+ package static func runSync(repo: String, python: String, runner: ProcessRunner = SystemProcessRunner(), line: @escaping (String) -> Void) -> Bool {
+ let env = ["PATH": BundledTools.path("/usr/bin:/bin:/opt/homebrew/bin:/usr/local/bin", python: python), "HOME": NSHomeDirectory(),
+ "LANG": "en_US.UTF-8", "TMPDIR": NSTemporaryDirectory()].merging(BundledTools.environment(for: python)) { $1 }
+ return runner.runCaptured(syncArguments(repo: repo, python: python), env: env, line: line)
+ }
+
+ /// The shader chain version of the checkout: DMSH_CHAIN in dm/gfx/nvn/dmsh_chain.h, read as the NSP script's chain_of() does
+ /// (`#define DMSH_CHAIN "(.*)"`). nil when the header is missing or has none.
+ package static func shaderChain(repo: String) -> String? {
+ guard !repo.isEmpty, let text = try? String(contentsOfFile: root(repo) + "/dm/gfx/nvn/dmsh_chain.h", encoding: .utf8),
+ let range = text.range(of: #"#define DMSH_CHAIN "(.*)""#, options: .regularExpression) else { return nil }
+ let line = text[range]
+ guard let open = line.firstIndex(of: "\""), let close = line.lastIndex(of: "\""), open < close else { return nil }
+ let chain = String(line[line.index(after: open)..<close])
+ return chain.isEmpty ? nil : chain
+ }
}
diff --git a/frontend/Sources/NX2Apple/EngineUpdater.swift b/frontend/Sources/NXCore/EngineUpdater.swift
similarity index 84%
rename from frontend/Sources/NX2Apple/EngineUpdater.swift
rename to frontend/Sources/NXCore/EngineUpdater.swift
index a2bced0f..e998d6ce 100644
--- a/frontend/Sources/NX2Apple/EngineUpdater.swift
+++ b/frontend/Sources/NXCore/EngineUpdater.swift
@@ -1,6 +1,4 @@
import Foundation
-import NXCore
-import NXLauncher
/// Which engine the launcher follows, and where its manifest lives.
///
@@ -8,18 +6,18 @@ import NXLauncher
/// - `experimental` ("实验版") is the tree built from the newest upstream pin. The release workflow publishes it
/// from main after an upstream bump passed CI, as a pre-release under a fixed tag. It is opt-in and never
/// carries the app. CI compiles it, applies the patches and runs the host tests. Nobody has played it.
-enum EngineChannel: String, CaseIterable {
+package enum EngineChannel: String, CaseIterable {
case stable, experimental
/// The pre-release tag the experimental engine is published under. The tag is fixed, so the launcher's URL
/// never changes. The assets inside are uploaded again with each publish.
- static let experimentalTag = "engine-experimental"
+ package static let experimentalTag = "engine-experimental"
/// Where this repository's release assets live.
- static let releaseBase = "https://github.com/LNPANDY/NX2Apple/releases/download/"
+ package static let releaseBase = "https://github.com/LNPANDY/NX2Apple/releases/download/"
/// The manifest for this channel. `version` is the running app's release version, which names the
/// stable tag (`v0.7.2`); the experimental channel ignores it.
- func manifestURL(version: String) -> URL? {
+ package func manifestURL(version: String) -> URL? {
switch self {
case .stable: return URL(string: Self.releaseBase + "v" + version + "/latest.json")
case .experimental: return URL(string: Self.releaseBase + Self.experimentalTag + "/latest.json")
@@ -27,29 +25,29 @@ enum EngineChannel: String, CaseIterable {
}
/// The detached signature next to the manifest (`latest.json.sig`, base64 of the Ed25519 signature).
- func signatureURL(version: String) -> URL? {
+ package func signatureURL(version: String) -> URL? {
manifestURL(version: version).flatMap { URL(string: $0.absoluteString + ".sig") }
}
/// The localization key of this channel's name in Settings.
- var labelKey: String { "engine.channel." + rawValue }
+ package var labelKey: String { "engine.channel." + rawValue }
/// The localization key of the warning shown before switching from `self` to `other`, nil when nothing changes.
/// Each switch can bring another shader chain, and then every game compiles its shaders again on its next build.
- func switchWarningKey(to other: EngineChannel) -> String? {
+ package func switchWarningKey(to other: EngineChannel) -> String? {
guard other != self else { return nil }
return other == .experimental ? "engine.switch.toExperimental" : "engine.switch.toStable"
}
/// Where an asset of this channel lives when the manifest names no source. A manifest that lists
/// sources wins: that is how a mirror is introduced without a new launcher.
- func assetURL(_ name: String, version: String, sources: [String]) -> URL? {
+ package func assetURL(_ name: String, version: String, sources: [String]) -> URL? {
assetURLs(name, version: version, sources: sources).first
}
/// Every place an asset is tried, in order: the manifest's sources, then this channel's release on GitHub
/// when no source already is it.
- func assetURLs(_ name: String, version: String, sources: [String]) -> [URL] {
+ package func assetURLs(_ name: String, version: String, sources: [String]) -> [URL] {
let tag = self == .stable ? "v" + version : Self.experimentalTag
var bases = sources.map { $0.hasSuffix("/") ? $0 : $0 + "/" }
let github = Self.releaseBase + tag + "/"
@@ -60,9 +58,9 @@ enum EngineChannel: String, CaseIterable {
/// The engine tree the release app carries in `Contents/Resources/engine` (the release workflow puts it there).
/// A launcher built from the repository has none, and keeps working on its checkout with git.
-enum EngineBundle {
+package enum EngineBundle {
/// The bundled tree, when it is complete and names itself. nil: no engine in this app.
- static func tree(in resources: URL?) -> URL? {
+ package static func tree(in resources: URL?) -> URL? {
guard let dir = resources?.appendingPathComponent("engine", isDirectory: true),
let id = EngineStore.identity(of: dir), EngineStore.isComplete(dir, id: id) else { return nil }
return dir
@@ -72,8 +70,8 @@ enum EngineBundle {
/// Installing an engine tree into the store. Every tree is unpacked or copied into a hidden folder beside the others
/// and renamed into place only when it is complete, so an interrupted download never leaves a tree that looks usable.
/// A tree is never changed once it is in place: the same id is the same files.
-enum EngineInstaller {
- enum Failure: Error, Equatable {
+package enum EngineInstaller {
+ package enum Failure: Error, Equatable {
case unpack // ditto failed: not a zip, or out of space
case incomplete(String) // the tree lacks what the launcher runs by path
case wrongEngine(String) // the tree's .engine-id is not the one the manifest names
@@ -84,7 +82,7 @@ enum EngineInstaller {
/// Unpack `zip` into `<store>/<engineID>/`. The archive's root is the tree itself
/// (`scripts/`, `dm/`, `.upstream-ref`, `.engine-id`, ...), as `tools/engine.py tree` builds it.
@discardableResult
- static func install(zip: URL, engineID: String, store: EngineStore) throws -> URL {
+ package static func install(zip: URL, engineID: String, store: EngineStore) throws -> URL {
guard store.directory(for: engineID) != nil else { throw Failure.badID(engineID) }
if store.isUsable(engineID), let dest = store.directory(for: engineID) { return dest }
let fm = FileManager.default
@@ -116,7 +114,7 @@ enum EngineInstaller {
/// Copy a tree that is already on disk (the one in the app bundle) into the store. The bundle is read-only and
/// signed, so the build never runs in it.
@discardableResult
- static func seed(from tree: URL, engineID: String, store: EngineStore) throws -> URL {
+ package static func seed(from tree: URL, engineID: String, store: EngineStore) throws -> URL {
guard store.directory(for: engineID) != nil else { throw Failure.badID(engineID) }
if store.isUsable(engineID), let dest = store.directory(for: engineID) { return dest }
let fm = FileManager.default
@@ -151,25 +149,32 @@ enum EngineInstaller {
}
/// Which channel and engine the launcher is on, remembered between launches.
-struct EngineSelection: Equatable {
- var channel: EngineChannel
- var engineID: String // empty: nothing installed by the launcher yet
- var version: String // the manifest's version of that engine, for Settings
- var previousID: String // the one kept for a build that was already running
+package struct EngineSelection: Equatable {
+ package var channel: EngineChannel
+ package var engineID: String // empty: nothing installed by the launcher yet
+ package var version: String // the manifest's version of that engine, for Settings
+ package var previousID: String // the one kept for a build that was already running
private static let channelKey = "engineChannel"
private static let idKey = "engineID"
private static let versionKey = "engineVersion"
private static let previousKey = "enginePreviousID"
- static func load(_ defaults: UserDefaults = .standard) -> EngineSelection {
+ package init(channel: EngineChannel, engineID: String, version: String, previousID: String) {
+ self.channel = channel
+ self.engineID = engineID
+ self.version = version
+ self.previousID = previousID
+ }
+
+ package static func load(_ defaults: UserDefaults = .standard) -> EngineSelection {
EngineSelection(channel: EngineChannel(rawValue: defaults.string(forKey: channelKey) ?? "") ?? .stable,
engineID: defaults.string(forKey: idKey) ?? "",
version: defaults.string(forKey: versionKey) ?? "",
previousID: defaults.string(forKey: previousKey) ?? "")
}
- func save(_ defaults: UserDefaults = .standard) {
+ package func save(_ defaults: UserDefaults = .standard) {
defaults.set(channel.rawValue, forKey: Self.channelKey)
defaults.set(engineID, forKey: Self.idKey)
defaults.set(version, forKey: Self.versionKey)
@@ -177,27 +182,27 @@ struct EngineSelection: Equatable {
}
/// Move to a newly installed engine, keeping the one we came from.
- mutating func switched(to id: String, version: String) {
+ package mutating func switched(to id: String, version: String) {
if !engineID.isEmpty, engineID != id { previousID = engineID }
engineID = id
self.version = version
}
/// The trees the store keeps: this one and the one before it.
- var kept: Set<String> { Set([engineID, previousID].filter { !$0.isEmpty }) }
+ package var kept: Set<String> { Set([engineID, previousID].filter { !$0.isEmpty }) }
}
// ---- Fetching ----
/// What a request for one file gave.
-enum EngineFetch<Value> {
+package enum EngineFetch<Value> {
case ok(Value)
case missing // HTTP 404: the release has no such file
case failed // offline, a timeout, any other status
}
/// The network, as the updater sees it. Tests replace it with files in memory.
-protocol EngineTransport {
+package protocol EngineTransport {
func data(from url: URL) -> EngineFetch<Data>
/// Download to `destination` (which does not exist yet). true when the whole file arrived.
func download(from url: URL, to destination: URL) -> Bool
@@ -205,7 +210,7 @@ protocol EngineTransport {
/// URLSession, waited on from a background queue as the app update does (BuildModel.downloadAndStage).
/// Nothing is cached: the experimental release replaces its files under the same names.
-struct URLSessionEngineTransport: EngineTransport {
+package struct URLSessionEngineTransport: EngineTransport {
private let session: URLSession = {
let c = URLSessionConfiguration.ephemeral
c.requestCachePolicy = .reloadIgnoringLocalAndRemoteCacheData
@@ -213,7 +218,9 @@ struct URLSessionEngineTransport: EngineTransport {
return URLSession(configuration: c)
}()
- func data(from url: URL) -> EngineFetch<Data> {
+ package init() {}
+
+ package func data(from url: URL) -> EngineFetch<Data> {
var result: EngineFetch<Data> = .failed
let sem = DispatchSemaphore(value: 0)
session.dataTask(with: url) { data, response, _ in
@@ -225,7 +232,7 @@ struct URLSessionEngineTransport: EngineTransport {
return result
}
- func download(from url: URL, to destination: URL) -> Bool {
+ package func download(from url: URL, to destination: URL) -> Bool {
var ok = false
let sem = DispatchSemaphore(value: 0)
session.downloadTask(with: url) { local, response, _ in
@@ -242,16 +249,21 @@ struct URLSessionEngineTransport: EngineTransport {
/// One engine update: read the channel's manifest, check its signature, and put its engine into the store, from the
/// store itself, from the app bundle or from a download whose SHA-256 matches. Runs on a background queue, touches
/// no UI state, and returns what the model shows.
-enum EngineUpdate {
+package enum EngineUpdate {
/// An engine as Settings names it.
- struct Engine: Equatable {
- let id: String
- let version: String
+ package struct Engine: Equatable {
+ package let id: String
+ package let version: String
+
+ package init(id: String, version: String) {
+ self.id = id
+ self.version = version
+ }
}
- enum Via: Equatable { case store, bundle, download }
+ package enum Via: Equatable { case store, bundle, download }
- enum Failure: Error, Equatable {
+ package enum Failure: Error, Equatable {
case network // the manifest did not arrive
case noManifest // this channel has no latest.json
case unsigned(EngineManifest.Signature) // a key is built in and the signature is absent or wrong
@@ -262,16 +274,16 @@ enum EngineUpdate {
case install(EngineInstaller.Failure)
}
- enum Outcome: Equatable {
+ package enum Outcome: Equatable {
case current(Engine) // the selected tree is the channel's engine
case available(Engine) // a newer one exists and `install` was off
case installed(Engine, Via)
case failed(Failure)
}
- static let assetPrefix = ".download-"
+ package static let assetPrefix = ".download-"
- static func run(channel: EngineChannel, appVersion: String, current: String, store: EngineStore, bundled: URL?,
+ package static func run(channel: EngineChannel, appVersion: String, current: String, store: EngineStore, bundled: URL?,
publicKey: String = EngineManifest.publicKey, install: Bool, transport: EngineTransport) -> Outcome {
guard let manifestURL = channel.manifestURL(version: appVersion) else { return .failed(.noManifest) }
let bundledID = bundled.flatMap { EngineStore.identity(of: $0) }
@@ -342,13 +354,13 @@ enum EngineUpdate {
/// The shader chain before and after a switch, when it changed. Every game then compiles its shaders again on
/// its next build, which the player is told.
- static func chainChange(from old: String, to new: String) -> (old: String, new: String)? {
- guard let a = BuildFolder.currentChain(repo: old), let b = BuildFolder.currentChain(repo: new), a != b else { return nil }
+ package static func chainChange(from old: String, to new: String) -> (old: String, new: String)? {
+ guard let a = EngineLayout.shaderChain(repo: old), let b = EngineLayout.shaderChain(repo: new), a != b else { return nil }
return (a, b)
}
/// The localization key of a failure's message.
- static func key(_ failure: Failure) -> String {
+ package static func key(_ failure: Failure) -> String {
switch failure {
case .network: return "engine.fail.network"
case .noManifest: return "engine.fail.noManifest"
diff --git a/frontend/Sources/NXModel/EngineSyncStore.swift b/frontend/Sources/NXModel/EngineSyncStore.swift
new file mode 100644
index 00000000..99cac3be
--- /dev/null
+++ b/frontend/Sources/NXModel/EngineSyncStore.swift
@@ -0,0 +1,296 @@
+import Foundation
+import Combine
+import NXCore
+
+/// Where the Python interpreter is, for a store that only needs to read it. `BuildModel` provides it today, the dependency
+/// store takes it over. Empty: not found yet, and the user of it looks for one itself.
+package protocol PythonLocation: AnyObject {
+ var pythonPath: String { get }
+}
+
+/// The engine (LNPANDY/NX2Apple#387, step 7): which engine the launcher builds with and how it gets it. It follows
+/// `AppUpdateStore` (`frontend/ARCHITECTURE.md`, "Stores").
+///
+/// Two kinds of launcher have an engine, and the store does both:
+/// - A checkout of the repository (a developer's, or one the launcher cloned) has no engine in git. `syncEngine` runs
+/// `tools/engine.py sync` when `engine/` is missing or older than the pin, at launch, after a clone, a pull or an adoption and
+/// before a build. It shows a status line while it runs and a failure row when it stops (`syncing`, `failure`).
+/// - A release app carries an engine tree in its bundle (`bundle`) and builds from the engine store instead. It seeds the app's
+/// own engine there on the first start, asks the selected channel for a newer one (`checkForUpdate`), and switches between
+/// channels after a confirmation. `selection` is the channel and the engine it is on, `status` what Settings says.
+///
+/// - What it works with comes through `init`: the `Localizer`, a `ProcessRunner` for the sync, the repository (`RepoLocation`) and
+/// the interpreter (`PythonLocation`) to read, the engine store and the network (`EngineTransport`), the defaults it saves the
+/// selection in. Where the build reads its tools from changes with the engine (`setRepoPath`), whether a build runs, whether the
+/// automatic update is on and where the log goes are closures the app wires.
+/// - The actions check their own conditions: no check or switch while one runs or a build runs.
+/// - The work blocks, so it runs off the main actor. The method that starts it sets its state first and returns the `Task`.
+@MainActor package final class EngineSyncStore: ObservableObject {
+ // ---- The engine of a release app ----
+
+ /// The channel and the engine the launcher is on, saved when it changes.
+ @Published package var selection: EngineSelection
+ /// A seed, a check or an install runs (the buttons of Settings disable).
+ @Published package private(set) var busy = false
+ /// The line under the tools section of Settings.
+ @Published package var status = ""
+ /// A newer engine the launch check found while automatic updates are off: Settings offers it.
+ @Published package private(set) var available: EngineUpdate.Engine?
+ /// The channel the player picked and is asked to confirm (the switch can change the shader chain).
+ @Published package var confirmingChannel: EngineChannel?
+ /// An installed engine that waits for the running build to end before the build tools switch to it.
+ package var pendingEngine: (engine: EngineUpdate.Engine, channel: EngineChannel)?
+ /// The engine tree inside the app bundle. nil for a launcher built from the repository, which keeps its checkout and git.
+ /// A release app builds from the engine store instead, and never clones or pulls.
+ package var bundle: URL?
+ package var usesEngineStore: Bool { bundle != nil }
+
+ // ---- The engine of a checkout ----
+
+ /// A sync runs: the launcher shows a status line with a spinner.
+ @Published package private(set) var syncing = false
+ /// How the last sync stopped (no network or GitHub unreachable, or another stop), as a failure row with a retry button. nil: none.
+ @Published package var failure: BuildFailure?
+
+ /// What waits for the running `tools/engine.py sync` (nil: none runs), and whether another one is due when it ends.
+ private var syncWaiting: [(Bool) -> Void]?
+ private var syncAgain = false
+
+ private let localizer: Localizer
+ private let runner: ProcessRunner
+ private let repo: RepoLocation
+ private let python: PythonLocation
+ private let engineStore: EngineStore
+ private let transport: EngineTransport
+ private let defaults: UserDefaults
+ private let appVersion: String
+ private let setRepoPath: (String) -> Void
+ private let buildRunning: () -> Bool
+ private let autoUpdate: () -> Bool
+ private let log: (String) -> Void
+
+ /// - Parameters:
+ /// - bundle: the engine tree of the app. nil: a launcher built from the repository.
+ /// - setRepoPath: the build reads its tools from this folder from now on (an installed engine tree).
+ /// - buildRunning: whether a build runs (the model's flag).
+ /// - autoUpdate: the automatic update is on: the launch check installs by itself.
+ /// - log: a line for the build log (the sync's output).
+ package init(localizer: Localizer, runner: ProcessRunner, repo: RepoLocation, python: PythonLocation,
+ engineStore: EngineStore = EngineStore(), transport: EngineTransport = URLSessionEngineTransport(),
+ bundle: URL? = EngineBundle.tree(in: Bundle.main.resourceURL), defaults: UserDefaults = .standard,
+ appVersion: String = AppUpdateStore.appVersion,
+ setRepoPath: @escaping (String) -> Void = { _ in }, buildRunning: @escaping () -> Bool = { false },
+ autoUpdate: @escaping () -> Bool = { true }, log: @escaping (String) -> Void = { _ in }) {
+ self.localizer = localizer
+ self.runner = runner
+ self.repo = repo
+ self.python = python
+ self.engineStore = engineStore
+ self.transport = transport
+ self.bundle = bundle
+ self.defaults = defaults
+ self.appVersion = appVersion
+ self.setRepoPath = setRepoPath
+ self.buildRunning = buildRunning
+ self.autoUpdate = autoUpdate
+ self.log = log
+ selection = EngineSelection.load(defaults)
+ }
+
+ private func t(_ key: String) -> String { localizer.t(key) }
+
+ // ---- The checkout's engine: tools/engine.py sync ----
+
+ /// `tools/engine.py sync` when engine/ is stale (#214): at launch, after a clone, a pull or an adoption, and before a build that
+ /// has no other engine. Checkouts only: a release app carries its engine tree (#227) and never syncs. A no-op for a current engine/
+ /// or a folder that is an engine tree. `then(ok)` runs on the main actor.
+ @discardableResult
+ package func syncEngine(then: ((Bool) -> Void)? = nil) -> Task<Void, Never> {
+ if syncWaiting != nil {
+ syncAgain = true
+ if let then { syncWaiting?.append(then) }
+ return Task {}
+ }
+ let path = repo.path
+ guard EngineLayout.isStale(path) else { failure = nil; then?(true); return Task {} }
+ syncWaiting = then.map { [$0] } ?? []
+ syncing = true; failure = nil
+ let chosen = python.pythonPath, runner = self.runner, log = self.log
+ log("$ python3 tools/engine.py sync\n")
+ let tail = Tail() // what failureKind reads when the sync stops
+ return Task {
+ let ok = await Task.detached {
+ let py = chosen.isEmpty ? Toolchain.findPython().path : chosen
+ return EngineLayout.runSync(repo: path, python: py, runner: runner) { line in
+ tail.append(line + "\n")
+ DispatchQueue.main.async { log(line + "\n") }
+ }
+ }.value
+ let waiting = syncWaiting ?? [], again = syncAgain && ok
+ syncWaiting = nil; syncAgain = false
+ syncing = false
+ failure = ok ? nil : BuildFailure(kind: EngineLayout.failureKind(tail.text), gameName: nil)
+ if again { syncEngine { ok in waiting.forEach { $0(ok) } } } else { waiting.forEach { $0(ok) } }
+ }
+ }
+
+ /// The failure row's Try again: clears the row and runs the sync again.
+ @discardableResult
+ package func retrySync() -> Task<Void, Never> {
+ guard !syncing else { return Task {} }
+ failure = nil
+ return syncEngine()
+ }
+
+ /// The last 4000 characters of what the sync printed, written from the background thread that runs it.
+ private final class Tail: @unchecked Sendable {
+ private let lock = NSLock()
+ private var value = ""
+ func append(_ s: String) { lock.lock(); value = String((value + s).suffix(4000)); lock.unlock() }
+ var text: String { lock.lock(); defer { lock.unlock() }; return value }
+ }
+
+ // ---- The release app's engine ----
+
+ /// At launch: build from the selected engine in the store, copy the app's own engine there on the first start,
+ /// then ask the selected channel for a newer one. On the stable channel a new app brings its own engine, which
+ /// then replaces the one an older app left selected, without the network.
+ @discardableResult
+ package func prepareEngine() -> Task<Void, Never> {
+ guard let bundled = bundle, !busy, let id = EngineStore.identity(of: bundled) else { return Task {} }
+ let selected = selection.engineID
+ let appMovedOn = selection.channel == .stable && selected != id && selection.version != appVersion
+ if !appMovedOn, engineStore.isUsable(selected), let dir = engineStore.directory(for: selected) {
+ setRepoPath(dir.path)
+ return checkForUpdate(automatic: true)
+ }
+ busy = true
+ status = t("engine.preparing")
+ let store = engineStore
+ return Task {
+ let result = await Task.detached { Result { try EngineInstaller.seed(from: bundled, engineID: id, store: store) } }.value
+ busy = false
+ switch result {
+ case .success:
+ // The app's engine is the stable one of this release. A player on experimental gets theirs from the check.
+ let engine = EngineUpdate.Engine(id: id, version: appVersion)
+ if buildRunning() { // a build started from the old folder while the copy ran
+ pendingEngine = (engine, selection.channel)
+ status = ""
+ } else {
+ switchEngine(to: engine, channel: selection.channel, announce: false)
+ await checkForUpdate(automatic: true).value
+ }
+ case .failure:
+ status = t("engine.seedFail")
+ }
+ }
+ }
+
+ /// Ask the selected channel for its engine. `automatic` is the launch check: it installs only with automatic
+ /// updates on, and it says nothing when the engine is current or the network is down.
+ @discardableResult
+ package func checkForUpdate(automatic: Bool = false) -> Task<Void, Never> {
+ runEngineUpdate(channel: selection.channel, install: automatic ? autoUpdate() : true, quiet: automatic)
+ }
+
+ /// Install the engine the launch check found (automatic updates off).
+ @discardableResult
+ package func installAvailable() -> Task<Void, Never> {
+ runEngineUpdate(channel: selection.channel, install: true, quiet: false)
+ }
+
+ /// The channel picker: a different channel is confirmed first, because the switch can change the shader chain.
+ package func requestChannel(_ channel: EngineChannel) {
+ guard selection.channel.switchWarningKey(to: channel) != nil, !busy, !buildRunning() else { return }
+ confirmingChannel = channel
+ }
+
+ /// The alert's button passes the channel it showed, so the order of the alert's own dismissal does not matter.
+ @discardableResult
+ package func confirmChannel(_ channel: EngineChannel) -> Task<Void, Never> {
+ confirmingChannel = nil
+ return runEngineUpdate(channel: channel, install: true, quiet: false)
+ }
+
+ package func cancelChannel() { confirmingChannel = nil }
+
+ /// The selected channel is only changed once its engine is in place, so Settings never names a channel whose
+ /// engine the build does not use.
+ private func runEngineUpdate(channel: EngineChannel, install: Bool, quiet: Bool) -> Task<Void, Never> {
+ guard usesEngineStore, !busy else { return Task {} }
+ guard !buildRunning() else {
+ if !quiet { status = t("engine.waitForBuild") }
+ return Task {}
+ }
+ busy = true
+ if !quiet { status = t(channel == selection.channel ? "engine.checking" : "engine.switching") }
+ let store = engineStore, bundled = bundle, current = selection.engineID, transport = self.transport, version = appVersion
+ return Task {
+ let outcome = await Task.detached {
+ EngineUpdate.run(channel: channel, appVersion: version, current: current, store: store,
+ bundled: bundled, install: install, transport: transport)
+ }.value
+ busy = false
+ applyOutcome(outcome, channel: channel, quiet: quiet)
+ }
+ }
+
+ package func applyOutcome(_ outcome: EngineUpdate.Outcome, channel: EngineChannel, quiet: Bool) {
+ switch outcome {
+ case .current(let engine):
+ available = nil
+ if selection.channel != channel || selection.version != engine.version {
+ selection.channel = channel
+ selection.version = engine.version
+ selection.save(defaults)
+ }
+ if !quiet { status = String(format: t("engine.upToDate"), engine.version) }
+ case .available(let engine):
+ available = engine
+ status = String(format: t("engine.available"), engine.version)
+ case .installed(let engine, _):
+ available = nil
+ if buildRunning() {
+ pendingEngine = (engine, channel)
+ status = String(format: t("engine.waitsForBuild"), engine.version)
+ } else {
+ switchEngine(to: engine, channel: channel)
+ }
+ case .failed(let failure):
+ if quiet, failure == .network || failure == .noManifest { return }
+ let key = EngineUpdate.key(failure)
+ status = failure == .noManifest ? String(format: t(key), t(channel.labelKey)) : t(key)
+ }
+ }
+
+ /// Point the build at an installed tree, keep the one before it, and say whether the shader chain changed.
+ /// `announce` off (the app's own engine at launch): only a changed shader chain is worth a line.
+ package func switchEngine(to engine: EngineUpdate.Engine, channel: EngineChannel, announce: Bool = true) {
+ guard let dir = engineStore.directory(for: engine.id) else { return }
+ let old = repo.path
+ selection.channel = channel
+ selection.switched(to: engine.id, version: engine.version)
+ selection.save(defaults)
+ setRepoPath(dir.path)
+ engineStore.prune(keeping: selection.kept)
+ if let change = EngineUpdate.chainChange(from: old, to: dir.path) {
+ status = String(format: t("engine.chainChanged"), engine.version, change.old, change.new)
+ } else {
+ status = announce ? String(format: t("engine.installed"), engine.version) : ""
+ }
+ }
+
+ /// The build that held an installed engine back has ended (the launcher watches `running`).
+ package func applyPendingEngine() {
+ guard !buildRunning(), let pending = pendingEngine else { return }
+ pendingEngine = nil
+ switchEngine(to: pending.engine, channel: pending.channel)
+ }
+
+ /// What Settings shows for the engine: its version and the pin it was built from.
+ package var detail: String {
+ guard !selection.engineID.isEmpty else { return t("engine.none") }
+ return String(format: t("engine.detail"), selection.version.isEmpty ? "?" : selection.version, String(selection.engineID.prefix(8)))
+ }
+}
diff --git a/frontend/Tests/NX2AppleTests/DropFilesTests.swift b/frontend/Tests/NX2AppleTests/DropFilesTests.swift
index 9e39eb79..fe9a55cd 100644
--- a/frontend/Tests/NX2AppleTests/DropFilesTests.swift
+++ b/frontend/Tests/NX2AppleTests/DropFilesTests.swift
@@ -138,7 +138,7 @@ final class DropFilesTests: XCTestCase {
model.deps = [Dep(id: "test", name: "Xcode", hint: "", present: true, installCmd: [])]
let library = GameLibrary(file: temporary.appendingPathComponent("preview.json"))
// the page as a drag shows it: the highlight drawn over the Builds page
- let page = BuilderView(m: model, updates: model.appUpdates, repo: model.repo, library: library).environmentObject(PadFocus()).environmentObject(model.localizer)
+ let page = BuilderView(m: model, updates: model.appUpdates, repo: model.repo, engine: model.engineSync, library: library).environmentObject(PadFocus()).environmentObject(model.localizer)
.modifier(GameFilesDrop(enabled: true, label: model.t("drop.hint"), targeted: true) { _ in })
.frame(width: 1100, height: 760).background(LauncherPalette.background).environment(\.colorScheme, .dark)
let host = NSHostingView(rootView: page)
diff --git a/frontend/Tests/NX2AppleTests/EngineLayoutTests.swift b/frontend/Tests/NX2AppleTests/EngineLayoutTests.swift
index f1686389..83fb68d6 100644
--- a/frontend/Tests/NX2AppleTests/EngineLayoutTests.swift
+++ b/frontend/Tests/NX2AppleTests/EngineLayoutTests.swift
@@ -6,7 +6,7 @@ import XCTest
/// #214: which engine the launcher builds from. A checkout of this repository builds from its engine/ (synced first when it is stale or
/// missing, there is no committed tree), a folder that is an engine tree itself (an installed release's engine) from that folder.
/// The stamp key matches tools/engine.py's.
-final class EngineLayoutTests: XCTestCase {
+@MainActor final class EngineLayoutTests: XCTestCase {
private var root: URL!
private var repo: String { root.path }
private let pin = String(repeating: "a", count: 40)
@@ -118,8 +118,8 @@ final class EngineLayoutTests: XCTestCase {
private func waitForSync(_ m: BuildModel, file: StaticString = #filePath, line: UInt = #line) {
let deadline = Date().addingTimeInterval(20)
- while m.engineSyncing && Date() < deadline { RunLoop.current.run(until: Date().addingTimeInterval(0.05)) }
- XCTAssertFalse(m.engineSyncing, "the sync did not end", file: file, line: line)
+ while m.engineSync.syncing && Date() < deadline { RunLoop.current.run(until: Date().addingTimeInterval(0.05)) }
+ XCTAssertFalse(m.engineSync.syncing, "the sync did not end", file: file, line: line)
}
func testNoNetworkShowsTheStatusThenTheOfflineRowAndRetryRunsTheSyncAgain() throws {
@@ -130,26 +130,26 @@ final class EngineLayoutTests: XCTestCase {
print('engine: cannot fetch aaaaaaaaaaaa from https://github.com/seedds/nxnative.git: Could not resolve host: github.com')
sys.exit(2)
""")
- XCTAssertNil(m.engineFailure)
- m.syncEngine()
- XCTAssertTrue(m.engineSyncing, "the status line shows while the engine is fetched")
+ XCTAssertNil(m.engineSync.failure)
+ m.engineSync.syncEngine()
+ XCTAssertTrue(m.engineSync.syncing, "the status line shows while the engine is fetched")
waitForSync(m)
- XCTAssertEqual(m.engineFailure?.kind, .engineOffline)
+ XCTAssertEqual(m.engineSync.failure?.kind, .engineOffline)
XCTAssertTrue(m.logTail.contains("cannot fetch"), "the log has the tool's line")
- m.retryEngineSync()
- XCTAssertNil(m.engineFailure, "Try again clears the row")
- XCTAssertTrue(m.engineSyncing)
+ m.engineSync.retrySync()
+ XCTAssertNil(m.engineSync.failure, "Try again clears the row")
+ XCTAssertTrue(m.engineSync.syncing)
waitForSync(m)
- XCTAssertEqual(m.engineFailure?.kind, .engineOffline)
+ XCTAssertEqual(m.engineSync.failure?.kind, .engineOffline)
XCTAssertEqual(try String(contentsOfFile: counter, encoding: .utf8), "xx", "the retry ran the tool again")
}
func testASyncThatWorksLeavesNoRow() throws {
let m = try model(engineScript: "print('engine: ready')\n")
- m.engineFailure = BuildFailure(kind: .engineOffline, gameName: nil)
- m.syncEngine()
+ m.engineSync.failure = BuildFailure(kind: .engineOffline, gameName: nil)
+ m.engineSync.syncEngine()
waitForSync(m)
- XCTAssertNil(m.engineFailure)
+ XCTAssertNil(m.engineSync.failure)
}
func testOfflineRowsHaveARetryButtonAndTheStatusLineHasText() {
diff --git a/frontend/Tests/NX2AppleTests/EngineUpdateModelTests.swift b/frontend/Tests/NX2AppleTests/EngineUpdateModelTests.swift
index 4bff5102..902e1107 100644
--- a/frontend/Tests/NX2AppleTests/EngineUpdateModelTests.swift
+++ b/frontend/Tests/NX2AppleTests/EngineUpdateModelTests.swift
@@ -3,17 +3,15 @@ import XCTest
@testable import NXCore
@testable import NXLauncher
-/// The engine in the model (EngineUpdateModel.swift): what the release app does with an update check's outcome, and
-/// that it never runs git. Nothing here touches the network or the store.
+/// The engine in the model: that the release app never runs git, and the engine strings. What the app does with an update check's
+/// outcome is `EngineSyncStore`, tested in NXModelTests. Nothing here touches the network or the store.
final class EngineUpdateModelTests: XCTestCase {
private let root = URL(fileURLWithPath: "/nonexistent-engine-bundle")
- private func id(_ c: Character) -> String { String(repeating: c, count: 40) + "-0123456789ab" }
-
/// A launcher built from the repository keeps git, the release app never touches it.
@MainActor func testTheReleaseAppNeverRunsGit() {
let m = BuildModel()
- m.engineBundle = root
+ m.engineSync.bundle = root
m.repo.clone()
m.repo.pull()
m.repo.autoPull(3)
@@ -23,45 +21,6 @@ final class EngineUpdateModelTests: XCTestCase {
XCTAssertEqual(m.repo.status, "")
}
- func testAnEngineInstalledDuringABuildWaits() {
- let m = BuildModel()
- m.lang = .en
- m.engineBundle = root
- m.running = true
- let path = m.repoPath
- m.applyEngineOutcome(.installed(.init(id: id("a"), version: "0.7.4"), .download), channel: .stable, quiet: false)
- XCTAssertEqual(m.repoPath, path, "the running build keeps its tools")
- XCTAssertEqual(m.pendingEngine?.engine.id, id("a"))
- XCTAssertEqual(m.engineStatus, String(format: L10n.string("engine.waitsForBuild", .en), "0.7.4"))
- m.checkEngineUpdate()
- XCTAssertFalse(m.engineBusy, "no check while a build runs")
- }
-
- func testTheLaunchCheckIsQuietWhenOffline() {
- let m = BuildModel()
- m.lang = .en
- m.applyEngineOutcome(.failed(.network), channel: .stable, quiet: true)
- XCTAssertEqual(m.engineStatus, "")
- m.applyEngineOutcome(.failed(.digest), channel: .stable, quiet: true)
- XCTAssertEqual(m.engineStatus, L10n.string("engine.fail.digest", .en))
- m.applyEngineOutcome(.failed(.noManifest), channel: .experimental, quiet: false)
- XCTAssertEqual(m.engineStatus, "The Experimental channel has no engine for this app version yet")
- }
-
- func testChangingTheChannelAsksFirst() {
- let m = BuildModel()
- m.engineBundle = root
- m.engineSelection.channel = .stable
- m.requestEngineChannel(.stable)
- XCTAssertNil(m.confirmingChannel, "the same channel asks nothing")
- m.requestEngineChannel(.experimental)
- XCTAssertEqual(m.confirmingChannel, .experimental)
- XCTAssertEqual(m.engineSelection.channel, .stable, "nothing changes before the answer")
- m.cancelEngineChannel()
- XCTAssertNil(m.confirmingChannel)
- XCTAssertEqual(m.engineSelection.channel, .stable)
- }
-
// MARK: Strings
func testEveryEngineStringIsTranslated() {
diff --git a/frontend/Tests/NX2AppleTests/EngineUpdaterTests.swift b/frontend/Tests/NX2AppleTests/EngineUpdaterTests.swift
index 63a5d5cc..ba43a136 100644
--- a/frontend/Tests/NX2AppleTests/EngineUpdaterTests.swift
+++ b/frontend/Tests/NX2AppleTests/EngineUpdaterTests.swift
@@ -4,7 +4,7 @@ import XCTest
@testable import NXCore
@testable import NXLauncher
-/// Channels, asset URLs, the installer and the update check (EngineUpdater.swift, EngineUpdateModel.swift). The
+/// Channels, asset URLs, the installer and the update check (EngineUpdater.swift in NXCore, EngineSyncStore in NXModel). The
/// archive is built here with `ditto`, so the unpack path is real. Nothing touches the network.
final class EngineUpdaterTests: XCTestCase {
private var root: URL!
diff --git a/frontend/Tests/NX2AppleTests/SourceSizeTests.swift b/frontend/Tests/NX2AppleTests/SourceSizeTests.swift
index f3a34c38..977b62bd 100644
--- a/frontend/Tests/NX2AppleTests/SourceSizeTests.swift
+++ b/frontend/Tests/NX2AppleTests/SourceSizeTests.swift
@@ -7,7 +7,7 @@ final class SourceSizeTests: XCTestCase {
/// Path under frontend/Sources → its line count when the limit came in. Lower the number when the file shrinks.
private static let aboveTheLimit: [String: Int] = [
- "NX2Apple/ContentView.swift": 2265,
+ "NX2Apple/ContentView.swift": 2228,
"NX2Apple/LauncherView.swift": 967,
"NXCore/Localization.swift": 4237,
]
diff --git a/frontend/Tests/NXModelTests/EngineSyncStoreTests.swift b/frontend/Tests/NXModelTests/EngineSyncStoreTests.swift
new file mode 100644
index 00000000..1a81506e
--- /dev/null
+++ b/frontend/Tests/NXModelTests/EngineSyncStoreTests.swift
@@ -0,0 +1,263 @@
+import XCTest
+@testable import NXModel
+@testable import NXCore
+
+/// `EngineSyncStore` without SwiftUI: a fake `ProcessRunner` stands in for `tools/engine.py sync`, the checkouts and the engine store are
+/// folders in a temporary directory and the defaults are a suite of their own. The model's wiring is tested in NX2AppleTests
+/// (EngineLayoutTests runs a real script, EngineUpdateModelTests the release app's rules through `BuildModel`).
+@MainActor final class EngineSyncStoreTests: XCTestCase {
+ private final class FakeRepo: RepoLocation {
+ var path = ""
+ var isRepo: Bool { RepoLayout.isRepo(path) }
+ var isGit = true
+ }
+
+ private final class FakePython: PythonLocation {
+ var pythonPath = "/usr/bin/python3"
+ }
+
+ private var temporary: URL!
+ private var suite: String!
+ private var defaults: UserDefaults!
+ private let runner = FakeProcessRunner()
+ private let localizer = Localizer(.en)
+ private let repo = FakeRepo()
+ private let python = FakePython()
+ private var log: [String] = []
+ private var building = false
+ private var autoUpdate = true
+ private var movedTo: [String] = []
+
+ override func setUpWithError() throws {
+ temporary = FileManager.default.temporaryDirectory.appendingPathComponent("nxmodel-engine-tests-" + UUID().uuidString)
+ try FileManager.default.createDirectory(at: temporary, withIntermediateDirectories: true)
+ suite = "nxmodel-engine-tests-" + UUID().uuidString
+ defaults = UserDefaults(suiteName: suite)
+ }
+
+ override func tearDownWithError() throws {
+ try FileManager.default.removeItem(at: temporary)
+ defaults.removePersistentDomain(forName: suite)
+ }
+
+ private func id(_ c: Character) -> String { String(repeating: c, count: 40) + "-0123456789ab" }
+
+ private func touch(_ rel: String, in dir: URL, _ text: String = "x") throws {
+ let url = dir.appendingPathComponent(rel)
+ try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true)
+ try text.write(to: url, atomically: true, encoding: .utf8)
+ }
+
+ /// A checkout that can make engine/ and has none: stale.
+ private func staleCheckout() throws -> String {
+ let dir = temporary.appendingPathComponent("checkout")
+ try touch("tools/engine.py", in: dir)
+ try touch(".upstream-ref", in: dir, String(repeating: "a", count: 40) + "\n")
+ try touch("patches/series", in: dir, "0001-a.patch\n")
+ try touch("patches/0001-a.patch", in: dir)
+ return dir.path
+ }
+
+ /// A complete engine tree in the store, so the store takes it for usable.
+ private func install(_ engineID: String, in store: EngineStore) throws {
+ let dir = try XCTUnwrap(store.directory(for: engineID))
+ for file in EngineStore.requiredFiles { try touch(file, in: dir) }
+ try touch(EngineStore.idFile, in: dir, engineID)
+ }
+
+ private func makeStore(bundle: URL? = nil, engines: EngineStore? = nil) -> EngineSyncStore {
+ EngineSyncStore(localizer: localizer, runner: runner, repo: repo, python: python,
+ engineStore: engines ?? EngineStore(root: temporary.appendingPathComponent("engines")),
+ bundle: bundle, defaults: defaults, appVersion: "1.2.3",
+ setRepoPath: { [unowned self] in repo.path = $0; movedTo.append($0) },
+ buildRunning: { [unowned self] in building }, autoUpdate: { [unowned self] in autoUpdate },
+ log: { [unowned self] in log.append($0) })
+ }
+
+ private func t(_ key: String) -> String { localizer.t(key) }
+
+ // ---- The sync of a checkout's engine ----
+
+ func testAStaleEngineIsSyncedWithTheChosenPythonAndLogged() async throws {
+ repo.path = try staleCheckout()
+ runner.streamAnswer = { _ in (["engine: ready"], true) }
+ let store = makeStore()
+ let task = store.syncEngine()
+ XCTAssertTrue(store.syncing, "the status line shows from the call on")
+ XCTAssertNil(store.failure)
+ await task.value
+ XCTAssertFalse(store.syncing)
+ XCTAssertNil(store.failure)
+ XCTAssertEqual(runner.launches.map(\.arguments), [[repo.path + "/tools/engine.py", "sync"]])
+ XCTAssertEqual(runner.launches.first?.executable, "/usr/bin/python3")
+ XCTAssertEqual(log.first, "$ python3 tools/engine.py sync\n")
+ try await Task.sleep(nanoseconds: 50_000_000) // the lines reach the log through the main queue
+ XCTAssertTrue(log.contains("engine: ready\n"))
+ }
+
+ func testACurrentEngineRunsNothingAndClearsTheRow() throws {
+ repo.path = temporary.path // no tools/engine.py: nothing to make
+ let store = makeStore()
+ store.failure = BuildFailure(kind: .engineOffline, gameName: nil)
+ var answers: [Bool] = []
+ store.syncEngine { answers.append($0) }
+ XCTAssertEqual(answers, [true], "then runs at once")
+ XCTAssertNil(store.failure)
+ XCTAssertFalse(store.syncing)
+ XCTAssertTrue(runner.launches.isEmpty)
+ }
+
+ func testAFailedFetchIsTheOfflineRowAndAnyOtherStopIsTheGeneralOne() async throws {
+ repo.path = try staleCheckout()
+ let store = makeStore()
+ runner.streamAnswer = { _ in (["engine: cannot fetch aaaaaaaaaaaa from https://github.com/seedds/nxnative.git: Could not resolve host"], false) }
+ await store.syncEngine().value
+ XCTAssertEqual(store.failure?.kind, .engineOffline)
+ runner.streamAnswer = { _ in (["0003-x.patch does not apply to the pin aaaaaaaaaaaa"], false) }
+ await store.syncEngine().value
+ XCTAssertEqual(store.failure?.kind, .engineSync)
+ }
+
+ func testTryAgainClearsTheRowAndRunsTheSyncAgain() async throws {
+ repo.path = try staleCheckout()
+ runner.streamAnswer = { _ in (["engine: cannot fetch x"], false) }
+ let store = makeStore()
+ await store.syncEngine().value
+ XCTAssertEqual(store.failure?.kind, .engineOffline)
+ let retry = store.retrySync()
+ XCTAssertNil(store.failure, "Try again clears the row")
+ XCTAssertTrue(store.syncing)
+ await retry.value
+ XCTAssertEqual(store.failure?.kind, .engineOffline)
+ XCTAssertEqual(runner.launches.count, 2, "the retry ran the tool again")
+ }
+
+ func testASyncThatWorksLeavesNoRow() async throws {
+ repo.path = try staleCheckout()
+ let store = makeStore()
+ store.failure = BuildFailure(kind: .engineOffline, gameName: nil)
+ await store.syncEngine().value
+ XCTAssertNil(store.failure)
+ }
+
+ func testACallDuringASyncWaitsForItAndOneMoreRunFollows() async throws {
+ repo.path = try staleCheckout()
+ let store = makeStore()
+ var answers: [String] = []
+ let first = store.syncEngine { answers.append("first \($0)") }
+ store.syncEngine { answers.append("second \($0)") } // the build asks while the launch sync runs
+ await first.value
+ try await Task.sleep(nanoseconds: 200_000_000)
+ XCTAssertEqual(runner.launches.count, 2, "what changed during the run is checked once more")
+ XCTAssertEqual(answers.sorted(), ["first true", "second true"])
+ XCTAssertFalse(store.syncing)
+ }
+
+ // ---- The release app's engine ----
+
+ func testAnEngineInstalledDuringABuildWaitsAndTheEndOfTheBuildSwitches() throws {
+ let engines = EngineStore(root: temporary.appendingPathComponent("engines"))
+ try install(id("a"), in: engines)
+ let store = makeStore(bundle: temporary, engines: engines)
+ repo.path = "/before"
+ building = true
+ store.applyOutcome(.installed(.init(id: id("a"), version: "0.7.4"), .download), channel: .stable, quiet: false)
+ XCTAssertEqual(repo.path, "/before", "the running build keeps its tools")
+ XCTAssertEqual(store.pendingEngine?.engine.id, id("a"))
+ XCTAssertEqual(store.status, String(format: t("engine.waitsForBuild"), "0.7.4"))
+ store.applyPendingEngine()
+ XCTAssertNotNil(store.pendingEngine, "still running")
+ building = false
+ store.applyPendingEngine()
+ XCTAssertNil(store.pendingEngine)
+ XCTAssertEqual(repo.path, engines.directory(for: id("a"))?.path)
+ XCTAssertEqual(store.selection.engineID, id("a"))
+ XCTAssertEqual(store.selection.version, "0.7.4")
+ XCTAssertEqual(EngineSelection.load(defaults).engineID, id("a"), "the selection is saved")
+ XCTAssertEqual(store.status, String(format: t("engine.installed"), "0.7.4"))
+ }
+
+ func testNoCheckRunsWhileABuildRuns() {
+ let store = makeStore(bundle: temporary)
+ building = true
+ store.checkForUpdate()
+ XCTAssertFalse(store.busy)
+ XCTAssertEqual(store.status, t("engine.waitForBuild"))
+ store.checkForUpdate(automatic: true)
+ XCTAssertFalse(store.busy)
+ }
+
+ func testALauncherBuiltFromTheRepositoryNeverChecksForAnEngine() {
+ let store = makeStore(bundle: nil)
+ XCTAssertFalse(store.usesEngineStore)
+ store.checkForUpdate()
+ XCTAssertFalse(store.busy)
+ XCTAssertEqual(store.status, "")
+ }
+
+ func testTheLaunchCheckIsQuietWhenOffline() {
+ let store = makeStore(bundle: temporary)
+ store.applyOutcome(.failed(.network), channel: .stable, quiet: true)
+ XCTAssertEqual(store.status, "")
+ store.applyOutcome(.failed(.digest), channel: .stable, quiet: true)
+ XCTAssertEqual(store.status, t("engine.fail.digest"))
+ store.applyOutcome(.failed(.noManifest), channel: .experimental, quiet: false)
+ XCTAssertEqual(store.status, "The Experimental channel has no engine for this app version yet")
+ }
+
+ func testAnAvailableEngineIsOfferedAndTheCurrentOneClearsTheOffer() {
+ let store = makeStore(bundle: temporary)
+ store.applyOutcome(.available(.init(id: id("b"), version: "0.8.0")), channel: .stable, quiet: true)
+ XCTAssertEqual(store.available?.version, "0.8.0")
+ XCTAssertEqual(store.status, String(format: t("engine.available"), "0.8.0"))
+ store.applyOutcome(.current(.init(id: id("b"), version: "0.8.0")), channel: .experimental, quiet: false)
+ XCTAssertNil(store.available)
+ XCTAssertEqual(store.selection.channel, .experimental)
+ XCTAssertEqual(EngineSelection.load(defaults).channel, .experimental)
+ XCTAssertEqual(store.status, String(format: t("engine.upToDate"), "0.8.0"))
+ }
+
+ func testChangingTheChannelAsksFirst() {
+ let store = makeStore(bundle: temporary)
+ store.selection.channel = .stable
+ store.requestChannel(.stable)
+ XCTAssertNil(store.confirmingChannel, "the same channel asks nothing")
+ store.requestChannel(.experimental)
+ XCTAssertEqual(store.confirmingChannel, .experimental)
+ XCTAssertEqual(store.selection.channel, .stable, "nothing changes before the answer")
+ store.cancelChannel()
+ XCTAssertNil(store.confirmingChannel)
+ XCTAssertEqual(store.selection.channel, .stable)
+ }
+
+ func testNoChannelSwitchIsAskedForWhileABuildRuns() {
+ let store = makeStore(bundle: temporary)
+ building = true
+ store.requestChannel(.experimental)
+ XCTAssertNil(store.confirmingChannel)
+ }
+
+ func testTheLaunchUsesTheSelectedEngineInTheStoreAndPointsTheBuildAtIt() throws {
+ let engines = EngineStore(root: temporary.appendingPathComponent("engines"))
+ let bundled = temporary.appendingPathComponent("bundled")
+ for file in EngineStore.requiredFiles { try touch(file, in: bundled) }
+ try touch(EngineStore.idFile, in: bundled, id("a"))
+ try install(id("a"), in: engines)
+ EngineSelection(channel: .stable, engineID: id("a"), version: "1.2.3", previousID: "").save(defaults)
+ autoUpdate = false
+ building = true // the launch check then ends at once, without the network
+ let store = makeStore(bundle: bundled, engines: engines)
+ store.prepareEngine()
+ XCTAssertEqual(repo.path, engines.directory(for: id("a"))?.path)
+ XCTAssertFalse(store.busy)
+ }
+
+ func testTheDetailNamesTheVersionAndThePin() {
+ let store = makeStore(bundle: temporary)
+ XCTAssertEqual(store.detail, t("engine.none"))
+ store.selection = EngineSelection(channel: .stable, engineID: id("c"), version: "", previousID: "")
+ XCTAssertEqual(store.detail, String(format: t("engine.detail"), "?", "cccccccc"))
+ store.selection.version = "1.2.3"
+ XCTAssertEqual(store.detail, String(format: t("engine.detail"), "1.2.3", "cccccccc"))
+ }
+}
diff --git a/frontend/Tests/NXModelTests/Fakes.swift b/frontend/Tests/NXModelTests/Fakes.swift
index a83f9cb4..15205ccc 100644
--- a/frontend/Tests/NXModelTests/Fakes.swift
+++ b/frontend/Tests/NXModelTests/Fakes.swift
@@ -7,6 +7,8 @@ final class FakeProcessRunner: ProcessRunner, @unchecked Sendable {
private var recorded: [ProcessLaunch] = []
/// The answer for a launch. The default is a run that exits 0 and prints nothing.
var answer: (ProcessLaunch) -> ProcessResult = { _ in ProcessResult() }
+ /// The lines a streamed launch prints, and whether it exits 0. The default is a run that prints nothing and succeeds.
+ var streamAnswer: (ProcessLaunch) -> (lines: [String], ok: Bool) = { _ in ([], true) }
var launches: [ProcessLaunch] { lock.lock(); defer { lock.unlock() }; return recorded }
/// Each launch as one line: the tool's name and its arguments.
@@ -19,7 +21,9 @@ final class FakeProcessRunner: ProcessRunner, @unchecked Sendable {
func stream(_ launch: ProcessLaunch, line: @escaping (String) -> Void) -> Bool {
lock.lock(); recorded.append(launch); lock.unlock()
- return true
+ let result = streamAnswer(launch)
+ result.lines.forEach(line)
+ return result.ok
}
func start(_ launch: ProcessLaunch, onOutput: @escaping (Data) -> Void, onExit: @escaping (Int32) -> Void) throws -> RunningProcess {