Run a virtual iPhone on an Apple Silicon Mac.
Swift
Let a clone wait out a reader instead of failing on Launchpad's meter
Jacky35f92d1f0859
diff --git a/Research/Host/machine_identity_and_clone.md b/Research/Host/machine_identity_and_clone.mdindex 36fb7294..d30051e3 100644--- a/Research/Host/machine_identity_and_clone.md+++ b/Research/Host/machine_identity_and_clone.md@@ -104,7 +104,9 @@ moment. The gigalocker and the anti-replay records advance together, so a copyof a running machine can pair a disk with SEP state from another moment.`vm clone` refuses while the source runs (`VPhoneBundleActivity.requireStopped`looks for a process holding the disk image, `SEPStorage` or `nvram.bin` open, or-a control socket that answers). Host-side writes to the disk image by `cfw+a control socket that answers). A holder that runs no machine, such as+Launchpad's disk meter, is waited out for up to 2 s first; a VM holder refuses+at once (`Research/Host/machine_templates.md`, Disk use). Host-side writes to the disk image by `cfwinstall` or `--update-environment` do not break the pairing: a source whosedisk image was modified hours after its `SEPStorage` cloned and booted fine.diff --git a/Research/Host/machine_templates.md b/Research/Host/machine_templates.mdindex 56d0b949..699d425e 100644--- a/Research/Host/machine_templates.md+++ b/Research/Host/machine_templates.md@@ -196,7 +196,8 @@ Launchpad mapping running machines' images about every 30 s, so a forcedfor the machine (starting, running, stopping), the last `lsof` names no otherprocess for its disk, and Launchpad has no operation on it (a creation, atemplate build, an export, an install or update, a shutdown or stop) or on a-whole library (an import, a template deletion). It asks again right before+whole library (an import, a template deletion). Templates are opened unless a+library-wide operation runs or a creation is saving or cloning one (below). It asks again right beforeeach file is mapped, so a machine started during a pass is not opened. Anyother machine keeps the ranges last mapped, and a stopped one is opened onlywhen a file's size, mtime or ctime changed since then: once after each run.@@ -218,13 +219,30 @@ image) is named and never signalled. Until 2026-10-09 `vm stop` SIGINTed every`lsof` holder when it found no `vphone-vm` and SIGKILLed every holder leftafter the timeout, so a stop that hit the meter's 0.1 s window could killLaunchpad (PR #633 retest, N2). `cfw install`, `update-environment` and-`update-kernel` still refuse any other holder, as do clone, snapshot, revert,-adopt, trim and the setup boot (`VPhoneBundleActivity.requireStopped`, over+`update-kernel` still refuse any other holder, as do snapshot, revert, adopt,+trim and the setup boot (`VPhoneBundleActivity.requireStopped`, over`Disk.img`, `SEPStorage` and `nvram.bin`): a file another process has open isnot standing still. Their refusals name each holder (`process 900vphone-launchpad`) and say when none of them runs the VM, so a reader is nottaken for a running machine.+A clone (`vm clone`, and `vm create --template` cloning a template) only reads+its source, so it waits out a holder that runs no machine: while every holder+is such a process and the control socket is not live, each of its two checks+(before and after the copy) asks again every 0.1 s for up to 2 s, then refuses+as before (`requireStopped(_:waitingForReaders:)`). A VM process holding the+source, or a live socket, refuses at once, so a clone still cannot take a+machine that is running or starting; a template is never booted. Writes keep+the strict check. In Launchpad 2.9.0 New Machine failed at its clone step with+"VM '<id>' is in use: … open in process <pid> vphone-launchpad": the adopt+step refreshed the machine list, the meter saw the new template folder, never+mapped, and opened its `Disk.img` just as `vm create --template` checked it;+retrying the step passed at once. Launchpad now also leaves every template+unopened while a creation of its own is at its adopt or clone step+(`VPhoneLaunchpadDiskAccess.templatesMayOpen`, asked before each file as+above), and the wait covers a mapping that was already under way, or another+reader, when the clone starts.+Launchpad judges the holders `lsof` lists the same way(`VPhoneLaunchpadDiskHolder`, a copy of the `VPhoneProcessHolder` rule, sinceLaunchpad does not link VPhoneCoreKit; both are tested against one table, seediff --git a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleActivity.swift b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleActivity.swiftindex a84cd39e..cfd4e6b7 100644--- a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleActivity.swift+++ b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleActivity.swift@@ -30,10 +30,61 @@ public enum VPhoneBundleActivity {/// The error names each holder's executable, so a refusal caused by, say,/// a backup tool reading `Disk.img` is not mistaken for a running VM.public static func requireStopped(_ bundle: VPhoneBundle) throws {+ try requireStopped(bundle, waitingForReaders: 0)+ }++ /// How long an operation that only reads a machine waits for holders+ /// that do not run it to close its files: Launchpad's disk meter holds an+ /// image for 0.1 s or less, longer on a slow Mac.+ public static let readerWait: TimeInterval = 2++ /// `requireStopped` for an operation that only reads the machine's files,+ /// a clone's source: while every holder is a process that does not run a+ /// machine (Launchpad's disk meter, Spotlight, a backup tool), it checks+ /// again every 0.1 s for up to `wait` seconds before refusing. A VM+ /// process holding a file, or a live control socket, refuses at once.+ /// Operations that write the files keep `requireStopped`.+ public static func requireStopped(_ bundle: VPhoneBundle, waitingForReaders wait: TimeInterval) throws {let urls = stateFileNames(of: bundle).map { bundle.url.appendingPathComponent($0) }- let pids = processesHolding(urls)- if !pids.isEmpty || controlSocketIsLive(bundle.url.appendingPathComponent("vphone.sock")) {- throw VPhoneBundleActivityError.running(name: bundle.name, pids: pids)+ let socket = bundle.url.appendingPathComponent("vphone.sock")+ try requireStopped(+ name: bundle.name,+ waitingForReaders: wait,+ holders: { processesHolding(urls).map { VPhoneProcessHolder(pid: $0) } },+ socketIsLive: { controlSocketIsLive(socket) },+ sleep: { Thread.sleep(forTimeInterval: $0) },+ )+ }++ /// The decision behind `requireStopped`, with the process table, the+ /// socket probe and the clock injected. With no holders the socket+ /// decides. Holders refuse at once when one runs a machine or the wait+ /// is over; otherwise, while the socket is not live, they are asked again+ /// after `interval`. With `wait` 0 the socket is not probed for a held+ /// machine, as before the wait existed.+ static func requireStopped(+ name: String,+ waitingForReaders wait: TimeInterval,+ interval: TimeInterval = 0.1,+ holders: () -> [VPhoneProcessHolder],+ socketIsLive: () -> Bool,+ sleep: (TimeInterval) -> Void,+ ) throws {+ var waited: TimeInterval = 0+ while true {+ let found = holders()+ if found.isEmpty {+ if socketIsLive() {+ throw VPhoneBundleActivityError.running(name: name, holders: [])+ }+ return+ }+ // Compared with a margin, so 20 steps of 0.1 s make 2 s.+ if waited + interval / 2 >= wait || found.contains(where: \.runsMachine) || socketIsLive() {+ throw VPhoneBundleActivityError.running(name: name, holders: found)+ }+ sleep(interval)+ waited += interval}}diff --git a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleOperations.swift b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleOperations.swiftindex 2f3e1795..f8209cc1 100644--- a/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleOperations.swift+++ b/VPhoneKit/VPhoneCoreKit/Bundle/VPhoneBundleOperations.swift@@ -192,17 +192,22 @@ public enum VPhoneBundleOperations {}/// `afterCopy` runs between the copy and the second running check, so- /// tests can start the source while it is being copied.+ /// tests can start the source while it is being copied. `readerWait` is+ /// how long each check waits out holders that do not run the source.static func clone(bundleNamed name: String,to newName: String,in library: VPhoneLibrary,newIdentity: Bool,+ readerWait: TimeInterval = VPhoneBundleActivity.readerWait,afterCopy: () throws -> Void,) throws -> VPhoneBundle {try requireValidName(newName)let source = try library.bundle(named: name)- return try clone(source, sourceName: name, to: newName, in: library, newIdentity: newIdentity, afterCopy: afterCopy)+ return try clone(+ source, sourceName: name, to: newName, in: library, newIdentity: newIdentity,+ readerWait: readerWait, afterCopy: afterCopy,+ )}/// Clones a machine folder that need not be in `library`, such as a@@ -224,6 +229,7 @@ public enum VPhoneBundleOperations {to newName: String,in library: VPhoneLibrary,newIdentity: Bool,+ readerWait: TimeInterval = VPhoneBundleActivity.readerWait,afterCopy: () throws -> Void,) throws -> VPhoneBundle {try requireValidName(newName)@@ -237,7 +243,12 @@ public enum VPhoneBundleOperations {// copying anything, and again after it: the source may have started// while it was copied (off APFS that takes minutes), and the copy may// then hold state files from different moments.- try VPhoneBundleActivity.requireStopped(source)+ //+ // A clone only reads the source, so a holder that runs no machine+ // (Launchpad's disk meter mapping a template that just appeared) is+ // waited out for up to `readerWait` instead of failing the clone. A+ // VM process holding the source, or its live socket, refuses at once.+ try VPhoneBundleActivity.requireStopped(source, waitingForReaders: readerWait)// Roll back a half-made copy, so a retry with the same name is not// blocked by the alreadyExists check.@@ -257,7 +268,7 @@ public enum VPhoneBundleOperations {}}try afterCopy()- try VPhoneBundleActivity.requireStopped(source)+ try VPhoneBundleActivity.requireStopped(source, waitingForReaders: readerWait)var clone = try VPhoneBundle.load(at: dst)if newIdentity {clone = try resetIdentity(of: clone, clonedFrom: name)diff --git a/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleActivityTests.swift b/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleActivityTests.swiftindex e3f0df83..1f5ab1f3 100644--- a/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleActivityTests.swift+++ b/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleActivityTests.swift@@ -121,6 +121,105 @@ struct BundleActivityTests {}}+ // MARK: - Waiting for readers++ private static let meter = VPhoneProcessHolder(pid: 900, executablePath: "/Applications/vphone-launchpad.app/Contents/MacOS/vphone-launchpad")+ private static let vm = VPhoneProcessHolder(pid: 410, executablePath: "/opt/VPhone.bundle/Contents/MacOS/vphone-vm")+ private static let service = VPhoneProcessHolder(+ pid: 412,+ executablePath: "/System/Library/Frameworks/Virtualization.framework/Versions/A/XPCServices/"+ + "com.apple.Virtualization.VirtualMachine.xpc/Contents/MacOS/com.apple.Virtualization.VirtualMachine",+ )++ /// Runs the decision over `answers`, one per probe (the last repeats),+ /// and returns how many times it slept and probed the socket.+ private func decide(+ wait: TimeInterval,+ _ answers: [[VPhoneProcessHolder]],+ socketIsLive: Bool = false,+ ) -> (error: VPhoneBundleActivityError?, sleeps: [TimeInterval], probes: Int, socketProbes: Int) {+ var probes = 0+ var socketProbes = 0+ var sleeps: [TimeInterval] = []+ var error: VPhoneBundleActivityError?+ do {+ try VPhoneBundleActivity.requireStopped(+ name: "tpl",+ waitingForReaders: wait,+ holders: {+ defer { probes += 1 }+ return answers[min(probes, answers.count - 1)]+ },+ socketIsLive: {+ socketProbes += 1+ return socketIsLive+ },+ sleep: { sleeps.append($0) },+ )+ } catch let refusal as VPhoneBundleActivityError {+ error = refusal+ } catch {+ Issue.record(error)+ }+ return (error, sleeps, probes, socketProbes)+ }++ @Test func `a reader that closes within the wait lets a clone through`() {+ let found = decide(wait: 2, [[Self.meter], [Self.meter], []])+ #expect(found.error == nil)+ #expect(found.sleeps == [0.1, 0.1])+ #expect(found.probes == 3)+ }++ @Test func `a reader that stays refuses once the wait is over`() {+ let found = decide(wait: 2, [[Self.meter]])+ #expect(found.error == VPhoneBundleActivityError.running(name: "tpl", holders: [Self.meter]))+ // 20 waits of 0.1 s, 21 probes: about 2 s, not more.+ #expect(found.sleeps.count == 20)+ #expect(found.probes == 21)+ }++ @Test func `a VM holding the files refuses at once, with or without readers`() {+ for holders in [[Self.vm], [Self.service], [Self.meter, Self.service]] {+ let found = decide(wait: 2, [holders, []])+ #expect(found.error == VPhoneBundleActivityError.running(name: "tpl", holders: holders))+ #expect(found.sleeps.isEmpty)+ }+ // A VM that starts during the wait refuses on the next probe.+ let starting = decide(wait: 2, [[Self.meter], [Self.meter, Self.vm], []])+ #expect(starting.error == VPhoneBundleActivityError.running(name: "tpl", holders: [Self.meter, Self.vm]))+ #expect(starting.sleeps.count == 1)+ }++ @Test func `a live control socket refuses at once`() {+ let held = decide(wait: 2, [[Self.meter], []], socketIsLive: true)+ #expect(held.error == VPhoneBundleActivityError.running(name: "tpl", holders: [Self.meter]))+ #expect(held.sleeps.isEmpty)+ let free = decide(wait: 2, [[]], socketIsLive: true)+ #expect(free.error == VPhoneBundleActivityError.running(name: "tpl", holders: []))+ }++ @Test func `without a wait any holder refuses as before`() {+ let found = decide(wait: 0, [[Self.meter], []])+ #expect(found.error == VPhoneBundleActivityError.running(name: "tpl", holders: [Self.meter]))+ #expect(found.sleeps.isEmpty)+ #expect(found.socketProbes == 0)+ #expect(decide(wait: 0, [[]]).error == nil)+ }++ @Test func `a held state file is waited for and then refused`() throws {+ let (root, bundle) = try makeBundle()+ defer { try? FileManager.default.removeItem(at: root) }+ let fd = open(bundle.url.appendingPathComponent("Disk.img").path, O_RDONLY)+ try #require(fd >= 0)+ defer { close(fd) }+ let began = Date()+ #expect(throws: VPhoneBundleActivityError.running(name: "vm", pids: [getpid()])) {+ try VPhoneBundleActivity.requireStopped(bundle, waitingForReaders: 0.3)+ }+ #expect(Date().timeIntervalSince(began) >= 0.25)+ }+@Test func `a control socket path that is not a socket counts as stopped`() throws {let (root, bundle) = try makeBundle()defer { try? FileManager.default.removeItem(at: root) }diff --git a/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleOperationsTests.swift b/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleOperationsTests.swiftindex dc784699..ecbae664 100644--- a/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleOperationsTests.swift+++ b/VPhoneKit/VPhoneCoreKitTests/Bundle/BundleOperationsTests.swift@@ -541,12 +541,35 @@ struct BundleOperationsTests {defer { close(fd) }for newIdentity in [false, true] {#expect(throws: VPhoneBundleActivityError.running(name: "src", pids: [getpid()])) {- try VPhoneBundleOperations.clone(bundleNamed: "src", to: "dst", in: lib, newIdentity: newIdentity)+ // This process runs no machine: the clone waits it out first.+ try VPhoneBundleOperations.clone(bundleNamed: "src", to: "dst", in: lib, newIdentity: newIdentity, readerWait: 0.3) {}}#expect(!FileManager.default.fileExists(atPath: lib.url(forName: "dst").path))}}+ @Test func `clone waits out a reader that closes the source soon`() throws {+ let root = try makeRoot()+ defer { try? FileManager.default.removeItem(at: root) }+ let lib = VPhoneLibrary(root: root)+ let src = try makeBootedSource(named: "src", in: lib)+ let disk = src.url.appendingPathComponent("Disk.img")++ // Like Launchpad's disk meter mapping a template it has just seen+ // appear: New Machine of Launchpad 2.9.0 failed its clone step on it.+ let fd = open(disk.path, O_RDONLY | O_EVTONLY)+ try #require(fd >= 0)+ #expect(VPhoneBundleActivity.processesHolding([disk]).contains(getpid()))+ let closer = Thread {+ Thread.sleep(forTimeInterval: 0.3)+ close(fd)+ }+ closer.start()+ let clone = try VPhoneBundleOperations.clone(bundleNamed: "src", to: "dst", in: lib)+ #expect(FileManager.default.fileExists(atPath: clone.url.appendingPathComponent("Disk.img").path))+ #expect(!VPhoneBundleActivity.processesHolding([disk]).contains(getpid()))+ }+@Test func `clone refuses a source that starts while it is copied and leaves no copy`() throws {let root = try makeRoot()defer { try? FileManager.default.removeItem(at: root) }@@ -561,7 +584,7 @@ struct BundleOperationsTests {}}#expect(throws: VPhoneBundleActivityError.running(name: "src", pids: [getpid()])) {- try VPhoneBundleOperations.clone(bundleNamed: "src", to: "dst", in: lib, newIdentity: false) {+ try VPhoneBundleOperations.clone(bundleNamed: "src", to: "dst", in: lib, newIdentity: false, readerWait: 0.2) {// The source starts after the first check passed.fd = open(src.url.appendingPathComponent(name).path, O_RDONLY)}diff --git a/VPhoneLaunchpad/Tests/TemplatesTests.swift b/VPhoneLaunchpad/Tests/TemplatesTests.swiftindex fbeb4d14..b09a7686 100644--- a/VPhoneLaunchpad/Tests/TemplatesTests.swift+++ b/VPhoneLaunchpad/Tests/TemplatesTests.swift@@ -543,6 +543,16 @@ struct TemplatesTests {precondition(!Access(isLibraryBusy: true).mayOpen, "An import or template deletion")precondition(!Access(isLaunched: true, isHeld: true).mayOpen, "Running")+ // Templates are left alone while a creation saves or clones one: the+ // clone refuses a template the meter holds open (2.9.0, New Machine+ // failed at its clone step right after the template was saved).+ precondition(Step.allCases.filter(\.needsTemplatesToItself) == [.adoptTemplate, .cloneTemplate], "Save and clone")+ precondition(Access.templatesMayOpen(isLibraryBusy: false, creationSteps: []), "Nothing runs")+ precondition(Access.templatesMayOpen(isLibraryBusy: false, creationSteps: [.restore, .setUpTemplate, .firstBoot]), "Other steps")+ precondition(!Access.templatesMayOpen(isLibraryBusy: false, creationSteps: [.adoptTemplate]), "Saving a template")+ precondition(!Access.templatesMayOpen(isLibraryBusy: false, creationSteps: [.restore, .cloneTemplate]), "Cloning a template")+ precondition(!Access.templatesMayOpen(isLibraryBusy: true, creationSteps: []), "A template deletion")+typealias Meter = VPhoneLaunchpadDiskMeter// An unchanged file is never opened, whatever else holds.for mayOpen in [true, false] {diff --git a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadCreationPlan.swift b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadCreationPlan.swiftindex cb58fd08..2144d150 100644--- a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadCreationPlan.swift+++ b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadCreationPlan.swift@@ -58,6 +58,13 @@ nonisolated enum VPhoneLaunchpadCreationStep: Int, CaseIterable, Identifiable, Cvar needsRoot: Bool {self == .installCFW || self == .applyGuestPatches}++ /// The step moves a build into `.templates` or clones a template, and+ /// the clone refuses a template another process holds open. The disk+ /// meter leaves templates alone meanwhile (`VPhoneLaunchpadDiskAccess`).+ var needsTemplatesToItself: Bool {+ self == .adoptTemplate || self == .cloneTemplate+ }}// MARK: - Plandiff --git a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadDiskUsage.swift b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadDiskUsage.swiftindex 81c9e4bf..4e6291f5 100644--- a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadDiskUsage.swift+++ b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadDiskUsage.swift@@ -66,6 +66,16 @@ nonisolated struct VPhoneLaunchpadDiskAccess: Hashable, Sendable {var mayOpen: Bool {!isLaunched && !isHeld && !isBusy && !isLibraryBusy}++ /// Whether the meter may open a template's files: no operation on a+ /// whole library runs (a template deletion), and no creation is at a+ /// step in `creationSteps` (the current step of each running one) that+ /// saves or clones a template. Mapping the image of a template saved a+ /// moment before made `vm create --template` refuse it as in use+ /// (Launchpad 2.9.0, New Machine failed at its clone step).+ static func templatesMayOpen(isLibraryBusy: Bool, creationSteps: [VPhoneLaunchpadCreationStep]) -> Bool {+ !isLibraryBusy && !creationSteps.contains(where: \.needsTemplatesToItself)+ }}// MARK: - Measuringdiff --git a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadMachineLibrary.swift b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadMachineLibrary.swiftindex fd1c8c08..f54c6ceb 100644--- a/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadMachineLibrary.swift+++ b/VPhoneLaunchpad/VPhoneLaunchpad/Machines/VPhoneLaunchpadMachineLibrary.swift@@ -844,7 +844,7 @@ final class VPhoneLaunchpadMachineLibrary {}let libraryRoots = roots.filter { $0 == libraryRoot || VPhoneLaunchpadMachineLocations.isAvailable($0) }let meter = diskMeter- let templatesMayOpen = globalActivity == nil+ let templatesMayOpen = self.templatesMayOpenlet mayOpenNow: @Sendable (String) async -> Bool = { [weak self] folder inawait self?.mayOpenDisk(inFolder: folder) ?? false}@@ -886,16 +886,27 @@ final class VPhoneLaunchpadMachineLibrary {)}+ /// Whether the meter may open template files now: not while a+ /// library-wide operation (a template deletion) runs, nor while a+ /// creation saves a template or clones one, which `vm create --template`+ /// refuses while another process holds the template open.+ private var templatesMayOpen: Bool {+ VPhoneLaunchpadDiskAccess.templatesMayOpen(+ isLibraryBusy: globalActivity != nil,+ creationSteps: creations.values.filter(\.isRunning).compactMap(\.current),+ )+ }+/// `diskAccess` for a measured folder, asked right before the meter- /// opens one of its files. A template folder may be opened unless a- /// library-wide operation (a template deletion) runs; a folder that is- /// neither a listed machine nor a template is not opened.+ /// opens one of its files. A template folder may be opened while+ /// `templatesMayOpen`; a folder that is neither a listed machine nor a+ /// template is not opened.private func mayOpenDisk(inFolder folder: String) -> Bool {if let machine = machines.first(where: { $0.path.url.path == folder }) {return diskAccess(of: machine.path).mayOpen}let isTemplate = URL(fileURLWithPath: folder).deletingLastPathComponent().lastPathComponent == ".templates"- return isTemplate && globalActivity == nil+ return isTemplate && templatesMayOpen}@concurrent