Run a virtual iPhone on an Apple Silicon Mac.
Swift
Wait for first-boot data migration in the setup boot with progress
Jacky40f4f3e27701
diff --git a/Research/Host/machine_templates.md b/Research/Host/machine_templates.mdindex b64516fd..a11e2930 100644--- a/Research/Host/machine_templates.md+++ b/Research/Host/machine_templates.md@@ -425,8 +425,8 @@ setup boot is the guest's first boot). It talks to vphoned through| --- | --- | --- | --- || connect | `ping` every second | 300 s | vphoned answers || 0 snapshot | `apfs.snapshot.delete {force}` | 120 s | no `orig-fs.disabled.rn-*` in `remaining`/`after` (none to begin with is fine) |-| a skip | `setup.skip {force}`, any refusal retried | 120 s | `setup_done` |-| b settle | `setup.settle {timeout_s ≤ 110}` repeated | 600 s overall | `settled` |+| a skip | `setup.skip {force}`, any refusal retried | 600 s | `setup_done` |+| b settle | `setup.settle {timeout_s ≤ 30}` repeated, a progress line after each | 600 s overall | `settled` (system apps expanded and data migration finished) || c apps | `apps.remove_system {bundle_ids, force}` | 300 s | every app `removed`, `absent` or `unregistered_stale` || d/e profile | `services.profile.apply {profile, groups, force}`, `services.profile` | 180 s | no `failed`; the record holds `signin_followup` and the extra groups; followupd and appleidsetupd disabled || f reboot | `processes.list` (launchd's `start_time`), `system.reboot {force}` | 300 s | vphoned answers with another launchd start time |@@ -436,9 +436,64 @@ setup boot is the guest's first boot). It talks to vphoned through`setup.skip` is retried on any refusal: right after vphoned first answers ona guest that just finished its first boot, it failed with "Cannot allocate-memory", then for 87 s with "relaunch action ignored and launchd stop failed:-144 Requestor lacks required entitlement" (SpringBoard's restart), before it-went through (p4-src, 27.0, 2026-10-08). The keys it writes are idempotent.+memory". The keys it writes are idempotent.++The setup boot is the guest's first boot, and the first boot's data migration+runs until 60–180 s after the VM starts (`Research/Guest/setup_assistant_skip.md`,+"The first boot after a restore"). Until it ends, SpringBoard has not decided+whether to run Setup, and FrontBoard ignores the request to restart it. Until+2026-10-09 `setup.skip` always restarted SpringBoard. FrontBoard ignored the+relaunch, the `launchctl stop` fallback failed with "144 Requestor lacks+required entitlement", and the step was retried until migration ended. That+took 45–98 s of every build (p4-src 87 s on 2026-10-08; 49, 56, 63, 79 and+98 s in later builds). vphoned now writes the keys and leaves SpringBoard alone+while migration has not finished (`respring: {restarted: false, reason:+"data_migration_pending"}`, logged as `SpringBoard left alone`). SpringBoard+reads the keys when migration ends. The settle step waits for the end of+migration (`data_migration_done`) as well as for the app expansion, so no+later step, and no template frozen from the boot, can come before it. Before+this change, that guarantee was only a side effect of the skip's retries:+`staged_system_apps` empties 26–67 s before migration ends.++This makes the steps report where the time goes, but it does not shorten the+boot. The wait is the migration (`SpringBoard.migrator`, which waits for+PosterBoard's poster migration), and every clone would otherwise repeat it.+Right after it ends, LaunchServices drops 7 records (265 → 258 apps) and+installd works for about 10 s, which the settle window covers before apps are+removed. Measured on `ss-raw` (27.0, never booted, reverted from a snapshot+between runs, 2026-10-09):++| Run | vphoned | setup-skip | settle | VM start → remove-apps | migration end → remove-apps |+| --- | --- | --- | --- | --- | --- |+| A | before | 98 s (11 refusals) | 11 s | 114 s | — |+| A2 | before | 56 s (6) | 10 s | 72 s | — |+| A3 | before | 54 s (6) | 10 s | 70 s | 14 s |+| N1 | after | 0 s | 64 s | 70 s | — |+| N2 | after | 0 s | 80 s | 86 s | 15 s |+| e2e (`vm create`, template 8e02beba26ce) | after | 0 s | 64 s | 69 s | — |++2.9.0 failed every creation on a slower Mac (a MacBook Air, 2026-10-09):+`setup.skip` was refused 14 times and its 120 s deadline ran out before data+migration ended. vphoned now passes the skip at once, and the settle step+waits for migration for up to 600 s, printing `waiting for the guest's+first-boot data migration (N s)` after each call of at most 30 s. The skip's+own deadline is 600 s too: a 2.9.0 vphoned (refused until migration ends)+gets through, and its refusals print `waiting for the guest's first-boot data+migration before SpringBoard can restart (N s)` at most every 30 s instead of+one error line each.++The migration's length varies between runs, and so does the boot. Measured+from the end of migration, the old and the new flow both reach remove-apps+about 15 s later. The reboot step took either about 10 s or about 33 s (4 of 6+runs), with either vphoned. The guest's crash reports show why: a+`panic-full` "initproc failed to start … Library not loaded:+/usr/lib/libSystem.B.dylib … (no such file, no dyld cache)" right at a boot,+after which the VM boots again. It happened on the setup boot's first boot+(N3: a few seconds lost) and on the boot after its reboot (the e2e build: the+33 s reboot). The cause is not known yet. Every run also left `duetexpertd`+crash reports, every 10 s after the reboot ("Failed to initialize datavault+for DuetExpertCenterAsset"), and one `PridePosterExtension` crash during+migration.Retries: a call that did not reach vphoned (no socket, guest not connected,timeout) or that vphoned refused with `retryable: true` or `reason: busy`diff --git a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneTemplateSetupBoot.swift b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneTemplateSetupBoot.swiftindex 04aaa663..60f45369 100644--- a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneTemplateSetupBoot.swift+++ b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneTemplateSetupBoot.swift@@ -96,10 +96,17 @@ public struct VPhoneTemplateSetupTimeouts: Equatable, Sendable {/// few seconds; the first-boot check of `vm create` allows 300.public var connect: TimeInterval = 300public var snapshot: TimeInterval = 120- public var skipSetup: TimeInterval = 120- /// Overall, across `setup.settle` calls of at most 110 s each. P0 saw- /// 92 s after `setup.skip` on a guest whose apps were already expanded.+ /// vphoned writes the keys at once. A vphoned that also has to restart+ /// SpringBoard is refused until the first boot's data migration ends,+ /// which took up to 98 s on a fast Mac and more than 120 s on a MacBook+ /// Air, so the deadline is sized like the settle step's.+ public var skipSetup: TimeInterval = 600+ /// Overall, across `setup.settle` calls of at most `settleCallLimit`+ /// seconds each. The first boot's data migration ended 60–180 s after+ /// the VM started on a fast Mac (2026-10-09); a slow host takes longer.public var settle: TimeInterval = 600+ /// How often a waiting step prints its progress.+ public var progressInterval: TimeInterval = 30public var removeApps: TimeInterval = 300public var serviceProfile: TimeInterval = 180/// From `system.reboot` until vphoned of the new boot answers (P1: 20 s).@@ -317,8 +324,10 @@ public final class VPhoneTemplateSetupBoot {/// (step e), and the labels it must have turned off.public static let signInFollowUpGroup = "signin_followup"public static let signInFollowUpLabels = ["com.apple.appleidsetupd", "com.apple.followupd"]- /// `setup.settle` answers within 110 s; the host waits at most 120 s.- static let settleCallLimit = 110+ /// One `setup.settle` call waits at most this long (vphoned's cap is+ /// 110 s, the host's read timeout 120 s), so the step prints progress+ /// between calls.+ static let settleCallLimit = 30private let machine: VPhoneTemplateSetupMachineprivate let plan: VPhoneTemplateSetupPlan@@ -419,23 +428,56 @@ public final class VPhoneTemplateSetupBoot {: " deleted \(outcome.deletedSnapshots.joined(separator: ", "))")}- /// a. `setup.skip` writes purplebuddy's keys and restarts SpringBoard.- /// Right after vphoned first answers, SpringBoard and cfprefsd may not be- /// up yet: on a fresh 27.0 guest it failed with "Cannot allocate memory"- /// seconds into the boot, then for 87 s with SpringBoard's restart- /// refused ("Requestor lacks required entitlement"). It is idempotent, so- /// any refusal is retried until the deadline.+ /// a. `setup.skip` writes purplebuddy's keys. On the setup boot (the+ /// guest's first) data migration is still running: SpringBoard decides+ /// whether to run Setup only when it ends, so vphoned leaves SpringBoard+ /// alone and the step takes a moment. Before vphoned did that, it asked+ /// FrontBoard to restart SpringBoard, which FrontBoard ignores until+ /// migration ends, and the fallback `launchctl stop` failed with launchd+ /// status 144 ("Requestor lacks required entitlement"); the step was+ /// retried for 45–98 s, as long as migration had left. Waiting for+ /// migration is now part of the settle step. Right after vphoned first+ /// answers, cfprefsd may not be up yet ("Cannot allocate memory" seconds+ /// into a fresh 27.0 boot). The skip is idempotent, so any refusal is+ /// retried until the deadline.private func skipSetup() throws {- let result = try call("setup.skip", ["force": true], deadline: deadline(timeouts.skipSetup), retryingRefusals: true)+ let start = clock.now()+ var lastProgress = start+ let result = try call(+ "setup.skip", ["force": true], deadline: deadline(timeouts.skipSetup), retryingRefusals: true,+ ) { [self] error in+ // A vphoned that restarts SpringBoard is refused until data+ // migration ends: report the wait, not every refusal.+ guard Self.isSpringBoardRestartRefusal(error.message) else { return " setup.skip: \(error.message); retrying" }+ let now = clock.now()+ guard now.timeIntervalSince(lastProgress) >= timeouts.progressInterval || lastProgress == start else { return nil }+ lastProgress = now+ return " waiting for the guest's first-boot data migration before SpringBoard can restart (\(Int(now.timeIntervalSince(start))) s)"+ }guard result["setup_done"] as? Bool == true else {throw StepError("setup.skip did not report setup_done")}outcome.setupSkipped = true+ let respring = result["respring"] as? [String: Any]+ if respring?["restarted"] as? Bool == false {+ log(" SpringBoard left alone: \(respring?["reason"] as? String ?? "not restarted"); it reads the keys when data migration ends")+ }}- /// b. `setup.settle` waits at most 110 s per call; it is called again- /// until the overall deadline.+ /// The refusal of a SpringBoard restart FrontBoard ignored: 2.9.0's+ /// "relaunch action ignored and launchd stop failed: 144 …", or the+ /// later "… SpringBoard did not restart: …".+ static func isSpringBoardRestartRefusal(_ message: String) -> Bool {+ message.contains("relaunch action ignored") || message.contains("SpringBoard did not restart")+ }++ /// b. `setup.settle` waits at most `settleCallLimit` seconds per call;+ /// it is called again until the overall deadline, printing what it waits+ /// for in between. It settles once the system apps are expanded and data+ /// migration has ended, which is also when SpringBoard reads the keys+ /// `setup.skip` wrote.private func settle() throws {+ let start = clock.now()let end = deadline(timeouts.settle)var reasons: [String] = []while true {@@ -446,16 +488,29 @@ public final class VPhoneTemplateSetupBoot {}let wait = Int(min(Double(Self.settleCallLimit), max(10, remaining.rounded(.up))))let result = try call("setup.settle", ["timeout_s": wait], deadline: end, timeout: Double(wait) + 20)+ let elapsed = Int(clock.now().timeIntervalSince(start))if result["settled"] as? Bool == true {outcome.settled = true- log(" settled after \(result["elapsed_s"].map { "\($0)" } ?? "?") s")+ log(" settled after \(elapsed) s")return}reasons = strings(result["reasons"])- log(" not settled yet: \(reasons.joined(separator: "; "))")+ log(Self.settleProgress(reasons, elapsed: elapsed))}}+ static let migrationReason = "data migration has not finished"++ /// One progress line of the settle step.+ static func settleProgress(_ reasons: [String], elapsed: Int) -> String {+ let others = reasons.filter { $0 != migrationReason }+ guard others.count < reasons.count else {+ return " waiting for first-boot work (\(elapsed) s): \(reasons.joined(separator: "; "))"+ }+ return " waiting for the guest's first-boot data migration (\(elapsed) s)"+ + (others.isEmpty ? "" : "; also \(others.joined(separator: "; "))")+ }+/// c. `apps.remove_system` backs each container up, unregisters it and/// removes it, then resprings once.private func removeApps() throws {@@ -717,6 +772,7 @@ public final class VPhoneTemplateSetupBoot {deadline: Date,timeout: TimeInterval = 60,retryingRefusals: Bool = false,+ retryLine: ((VPhoneGuestCallError) -> String?)? = nil,) throws -> [String: Any] {var attempts = 0while true {@@ -728,7 +784,9 @@ public final class VPhoneTemplateSetupBoot {guard clock.now().addingTimeInterval(timeouts.retryInterval) < deadline else {throw StepError("\(method): \(error.message) (\(attempts) attempt(s))")}- log(" \(method): \(error.message); retrying")+ if let line = retryLine.map({ $0(error) }) ?? " \(method): \(error.message); retrying" {+ log(line)+ }clock.sleep(timeouts.retryInterval)} catch let error as VPhoneGuestCallError {// Kept whole for a caller that reads its detail.diff --git a/VPhoneKit/VPhoneCoreKitTests/Bundle/TemplateSetupBootTests.swift b/VPhoneKit/VPhoneCoreKitTests/Bundle/TemplateSetupBootTests.swiftindex 7333a3ea..3cf0e7c0 100644--- a/VPhoneKit/VPhoneCoreKitTests/Bundle/TemplateSetupBootTests.swift+++ b/VPhoneKit/VPhoneCoreKitTests/Bundle/TemplateSetupBootTests.swift@@ -32,7 +32,15 @@ private final class FakeGuest: VPhoneTemplateSetupMachine {var connectDelayPolls = 2var snapshotBusyAnswers = 0var skipFailures = 0+ /// vphoned of 2026-10-09 on: the first boot's data migration is still+ /// running, so `setup.skip` writes the keys and leaves SpringBoard alone.+ var migrationPending = false+ var skipFailureDetail: [String: Any] = ["code": "failed", "message": "Cannot allocate memory"]var unsettledAnswers = 1+ /// How long one refused `setup.skip` takes in the guest (2.9.0's+ /// vphoned waited 5 s for SpringBoard's pid to change).+ var skipFailureSeconds: TimeInterval = 0+ var unsettledReasons = ["staged_system_apps not empty"]var refusedApps: Set<String> = []/// Apps vphoned had to unregister more than once, as it reports them.var unregisterAttempts: [String: Int] = [:]@@ -106,17 +114,21 @@ private final class FakeGuest: VPhoneTemplateSetupMachine {#expect(params["force"] as? Bool == true)if skipFailures > 0 {skipFailures -= 1- throw refused("Cannot allocate memory", ["code": "failed", "message": "Cannot allocate memory"])+ clock.sleep(skipFailureSeconds)+ throw refused(skipFailureDetail["message"] as? String ?? "", skipFailureDetail)}setupDone = true- return ["setup_done": true, "setup_version": 11]+ let respring: [String: Any] = migrationPending+ ? ["restarted": false, "reason": "data_migration_pending"]+ : ["restarted": true, "method": "frontboard_relaunch", "previous_pid": 36, "pid": 551]+ return ["setup_done": true, "setup_version": 11, "respring": respring]case "setup.settle":let wait = params["timeout_s"] as? Int ?? 90- #expect(wait <= 110)+ #expect(wait <= 30)if unsettledAnswers > 0 {unsettledAnswers -= 1clock.sleep(TimeInterval(wait))- return ["settled": false, "reasons": ["staged_system_apps not empty"], "elapsed_s": wait]+ return ["settled": false, "reasons": unsettledReasons, "elapsed_s": wait]}return ["settled": true, "elapsed_s": 10.2, "reasons": [String]()]case "apps.remove_system":@@ -301,6 +313,37 @@ struct TemplateSetupBootTests {#expect(outcome.setupSkipped)}+ @Test func `setup skip during data migration leaves SpringBoard alone and says so`() throws {+ let guest = FakeGuest(clock: FakeClock())+ guest.migrationPending = true+ var lines: [String] = []+ let boot = VPhoneTemplateSetupBoot(+ machine: guest,+ plan: VPhoneTemplateSetupPlan(slimming: VPhoneTemplateSlimmingRequest.defaultSlimming, requiresEveryApp: true),+ clock: guest.clock,+ log: { lines.append($0) },+ )+ let outcome = try boot.run()+ #expect(guest.calls.count(where: { $0 == "setup.skip" }) == 1)+ #expect(outcome.setupSkipped)+ #expect(outcome.isComplete)+ #expect(lines.contains { $0.contains("SpringBoard left alone: data_migration_pending") })+ // The settle step, which waits for migration, still comes before any slimming.+ #expect(inOrder(guest.calls, ["setup.skip", "setup.settle", "apps.remove_system"]))+ }++ @Test func `a SpringBoard restart vphoned reports as retryable is retried`() throws {+ let guest = FakeGuest(clock: FakeClock())+ guest.skipFailures = 2+ guest.skipFailureDetail = [+ "code": "command_failed", "reason": "busy", "retryable": true,+ "message": "Setup Assistant's keys are written, but SpringBoard did not restart: relaunch action ignored",+ ]+ let outcome = try run(guest).get()+ #expect(guest.calls.count(where: { $0 == "setup.skip" }) == 3)+ #expect(outcome.setupSkipped)+ }+@Test func `setup skip that keeps failing fails its step`() throws {let guest = FakeGuest(clock: FakeClock())guest.skipFailures = 1000@@ -317,7 +360,7 @@ struct TemplateSetupBootTests {#expect(outcome.snapshotsGone)}- @Test func `settle is called again until it settles, each call within 110 s`() throws {+ @Test func `settle is called again until it settles, each call within 30 s`() throws {let guest = FakeGuest(clock: FakeClock())guest.unsettledAnswers = 4let outcome = try run(guest).get()@@ -335,8 +378,72 @@ struct TemplateSetupBootTests {#expect(!guest.stopped)#expect(!failure.outcome.isComplete)#expect(!guest.calls.contains("apps.remove_system"))- // Bounded: 600 s of 110 s calls.- #expect(guest.calls.count(where: { $0 == "setup.settle" }) <= 7)+ // Bounded: 600 s of 30 s calls.+ #expect(guest.calls.count(where: { $0 == "setup.settle" }) <= 21)+ }++ /// The slower Mac of the 2.9.0 report: data migration ran past the old+ /// 120 s skip deadline. The skip now passes at once and the settle step+ /// waits, printing progress, well past 120 s.+ @Test func `a slow data migration is waited for in settle with progress, not failed`() throws {+ let guest = FakeGuest(clock: FakeClock())+ guest.migrationPending = true+ guest.unsettledAnswers = 10+ guest.unsettledReasons = ["data migration has not finished"]+ var lines: [String] = []+ let boot = VPhoneTemplateSetupBoot(+ machine: guest,+ plan: VPhoneTemplateSetupPlan(slimming: VPhoneTemplateSlimmingRequest.defaultSlimming, requiresEveryApp: true),+ clock: guest.clock,+ log: { lines.append($0) },+ )+ let outcome = try boot.run()+ #expect(outcome.isComplete)+ #expect(try #require(outcome.durations[.settle]) > 300)+ #expect(try #require(outcome.durations[.skipSetup]) < 5)+ let progress = lines.filter { $0.hasPrefix(" waiting for the guest's first-boot data migration (") }+ #expect(progress.count == 10)+ #expect(progress.allSatisfy { !$0.contains(";") })+ #expect(lines.contains { $0.hasPrefix(" settled after ") })+ }++ @Test func `settle progress names data migration first`() {+ #expect(VPhoneTemplateSetupBoot.settleProgress(["data migration has not finished"], elapsed: 42)+ == " waiting for the guest's first-boot data migration (42 s)")+ #expect(VPhoneTemplateSetupBoot.settleProgress(["staged_system_apps has 3 entries", "data migration has not finished"], elapsed: 12)+ == " waiting for the guest's first-boot data migration (12 s); also staged_system_apps has 3 entries")+ #expect(VPhoneTemplateSetupBoot.settleProgress(["app count changed: 250 → 258"], elapsed: 70)+ == " waiting for first-boot work (70 s): app count changed: 250 → 258")+ }++ /// 2.9.0's vphoned restarts SpringBoard in every skip and is refused+ /// until migration ends: 14 refusals (about 130 s) failed the step on the+ /// MacBook Air. Now the step outlasts them and reports the wait every 30 s+ /// instead of every refusal.+ @Test func `a 2_9_0 vphoned refused until migration ends passes the skip step`() throws {+ let guest = FakeGuest(clock: FakeClock())+ guest.skipFailures = 30+ guest.skipFailureSeconds = 5.5+ guest.skipFailureDetail = [+ "code": "failed",+ "message": "relaunch action ignored and launchd stop failed: 144 Requestor lacks required entitlement",+ ]+ var lines: [String] = []+ let boot = VPhoneTemplateSetupBoot(+ machine: guest,+ plan: VPhoneTemplateSetupPlan(slimming: VPhoneTemplateSlimmingRequest.defaultSlimming, requiresEveryApp: true),+ clock: guest.clock,+ log: { lines.append($0) },+ )+ let outcome = try boot.run()+ #expect(outcome.isComplete)+ let skip = try #require(outcome.durations[.skipSetup])+ #expect(skip > 250 && skip < 600)+ #expect(!lines.contains { $0.contains("144") })+ let progress = lines.filter { $0.contains("waiting for the guest's first-boot data migration before SpringBoard can restart") }+ // The first refusal, then one line per 30 s at most.+ #expect(progress.count >= 2 && progress.count <= Int(skip / 30) + 1, "\(progress)")+ #expect(progress.first?.hasSuffix(" s)") == true, "\(progress)")}@Test func `a refused app fails a build whose key promises it`() throws {