Skip to content

Run a virtual iPhone on an Apple Silicon Mac.

Swift

Restart a locationd that deadlocked as it started

Jacky1c77108ff52f
diff --git a/Research/Guest/location_simulation_26_4_failure.md b/Research/Guest/location_simulation_26_4_failure.md
index 4b6b50d5..2cd66ecd 100644
--- a/Research/Guest/location_simulation_26_4_failure.md
+++ b/Research/Guest/location_simulation_26_4_failure.md
@@ -48,13 +48,12 @@ one ran. StageBundle.sh now signs the helper with `designated => identifier
cdhash-signed helper was running, and the helper went from `NotDetermined` to
`AuthorizedAlways` without a prompt.
-**locationd deadlocks in a headless guest.** Started with `--headless`, the VM's
-locationd deadlocked within a second of starting on every boot, with sync off
-and no location request sent; with a window it never did. A spindump has its
-`CLClientManagerSilo` and `CLAuthSyncManagerSilo` queues "blocked by queue in a
-transient state" and every client (vphoned, Maps, navd, nearbyd) waiting on
-`com.apple.locationd.synchronous`. Location RPCs then time out after 120 s. Not
-solved here.
+**locationd can deadlock as it starts.** It was first seen on headless starts,
+where the VM's locationd hung within a second on every boot tried; more boots
+showed windowed starts hang too, less often. Every CoreLocation client then
+waits on `com.apple.locationd.synchronous` and location RPCs time out after
+120 s. vphoned now restarts such a locationd; see
+`Research/Guest/locationd_startup_deadlock.md`.
Sync Host Location is now a machine setting, `syncsHostLocation` in
`config.plist`, off when absent, in a window and headless alike (headless used
diff --git a/Research/Guest/locationd_startup_deadlock.md b/Research/Guest/locationd_startup_deadlock.md
new file mode 100644
index 00000000..8d4eef8e
--- /dev/null
+++ b/Research/Guest/locationd_startup_deadlock.md
@@ -0,0 +1,147 @@
+# locationd startup deadlock (iOS 27.0)
+
+On an iOS 27.0 guest, locationd can deadlock about a second after it starts.
+Every CoreLocation client then blocks on `com.apple.locationd.synchronous` for
+good: vphoned's `location.current` and `location.set` time out after 120 s
+("The machine closed the connection without answering"), and Maps, navd,
+nearbyd, PerfPowerServices, usernotificationsd and the rest sit on the same
+port. vphoned now restarts such a locationd (`GuestLocationdWatchdog`).
+
+Measured on 2026-10-07 on a new `standard` VM: iPhone17,3, iOS 27.0 (24A435),
+cloudOS 26.4 (23E5207q) kernel, 8 vCPUs, 8 GiB, Location Services off, no
+location RPC sent, Sync Host Location off.
+
+## It is not a headless problem
+
+It was first seen on headless starts (`vphone-launchpad-cli vm start <vm>
+--headless`) while windowed starts of the same VM worked. Boots of the
+unmodified guest (upstream vphoned, no watchdog) show that both kinds hang:
+
+| Start | Boots | locationd hung |
+| --- | --- | --- |
+| headless | 15 | 6 |
+| windowed | 5 | 1 |
+
+A boot counts as hung when `location.current` did not answer within 15 s at
+about 60 s of uptime (a healthy locationd answers at once, here with
+`location_services_off`). locationd's own CPU time tells the two apart as
+well: about 0.15–0.22 s for a hung instance, 0.4–1.1 s for a healthy one. One
+headless boot that streamed locationd's log from the moment vphoned answered
+did not hang, possibly because logging slows locationd down.
+
+Nothing in `VPhoneVirtualMachineAppDelegate`'s headless branch reaches
+locationd: the headless `onConnect` sends the same `time.timezone` and
+`audio.host_latency` calls as the windowed one and only skips the menu's
+battery and Low Power Mode sync. The deadlock also forms before most of
+those calls arrive.
+
+## The wait cycle
+
+From a guest spindump (`/usr/sbin/spindump -notarget 3 10`, run as a launchd
+job through vphoned; see the guest-profiling procedure). locationd is not in
+the shared cache: pull `/usr/libexec/locationd` out of the IPSW
+(`ipsw extract --files --pattern 'usr/libexec/locationd$'`), take
+`0x100000000 + <offset>` and read the ObjC name with
+`ipsw macho disass <locationd> --vaddr <function start> --count 1`
+(`ipsw macho a2s` returns only `func_…`). Shared cache frames resolve with
+`ipsw dyld a2s` (slide 0 on this boot).
+
+| Thread (queue) | Doing | Waits on |
+| --- | --- | --- |
+| main | CFRunLoop source `CLLocationController::sendTechStatusNotification` → `[silo sync:]` | `CLLocationControllerSilo` (turnstile on its thread) |
+| `CLLocationControllerSilo` | `-[CLLocationControllerAdapter beginService]` → `dispatch_once` → the controller's constructor (`sub_1007428b8`) → `+[CLSimulatedLocationProviderAdapter performSyncOnSilo:invoker:]` | thread event: the main-queue hierarchy |
+| `CLClientManagerSilo` | `beginService` → `dispatch_once` → LocationAuthorizationFramework `iterateAllAnchorKeyPathsWithBlock:` → `+[CLCapabilityMonitor performSyncOnSilo:invoker:]` | "queue in a transient state" |
+| `CLNetworkLocationProviderSilo`, `CLWifiTilesManagerSilo` | `beginService` → `+[CLDataProtectionManagerAdapter performSyncOnSilo:invoker:]` | transient / thread event |
+| `CLCountryTrackerSilo`, `CLGeoIPCountryTrackerSilo` | `beginService` → `initInUniverse:` → `performSyncOnSilo:invoker:` | thread event / transient |
+| `CLInternalServiceSilo` | XPC `getAuthorizationStatusForBundleID:orBundlePath:replyBlock:` → `+[CLClientManagerAdapter performSyncOnSilo:invoker:]` | `CLClientManagerSilo` |
+
+LocationSupport groups silos into cohorts, layers of the sync graph read from
+`CoreLocation.framework/Support/syncget_graph_iOS.plist` (logged as
+`#Cohorting CohortId assignment for silo`). A silo may `sync:` only into a
+lower layer. Everything above waits on cohort 0
+(`CLSimulatedLocationProviderSilo`, `CLCapabilityMonitorSilo`,
+`CLDataProtectionManagerSilo`, `CLDaemonStatusSilo`, …), the waits into it are
+thread-event waits (the main queue's hierarchy has no workloop), and no thread
+is draining it. The main thread itself sits at the top, inside a run-loop
+callback that synced into cohort 6.
+
+The tech-status source is created by the controller's constructor itself
+(`CFRunLoopSourceCreate` with `sendTechStatusNotification` as `perform`, added
+to the main run loop in common modes, next to a second source for
+`checkActiveProviders`). `CLLocationController::signalTechStatusChanged`
+(`sub_10006f794`) signals both sources and wakes the main run loop. Its callers
+include `CLLocationController::startLocation` and the GPS, Wi-Fi and indoor
+"active state" notifications. When it fires while the constructor is still
+running, and the main thread is idle, the main thread enters the callback,
+blocks on the controller's silo, and the constructor's next `sync:` into
+cohort 0 can never run. In a healthy start `#techstatus,posting notification`
+comes after `#Manufacturing service complete … CLLocationController`.
+
+## A patch was tried and is not enough
+
+Both callbacks only capture the controller pointer, and nothing after the
+`sync:` uses the block's result, so delivering them with `async:` is safe. A
+prototype retargeted the two `bl objc_msgSend$sync:` (24A435:
+`0x100074d68`, `0x100077e98`) to the existing `objc_msgSend$async:` stub
+(`0x101b23800`) and re-hashed the two code pages. With it the first boot hung
+again, through a third path: the main thread was draining the main queue in a
+silo timer (`shouldFire`, `sub_1004217ac`) that syncs into
+`+[CLDaemonStatusAdapter performSyncOnSilo:invoker:]` while
+`CLClientManagerSilo` waits on the same silo. Each main-thread entry point
+would need its own patch, and the set is not known, so no patch was added.
+
+The prototype went in through a host mount of the stopped VM's system volume
+as the invoking user (`hdiutil attach -nomount`, `mount_apfs -o
+rw,nosuid,nodev,nobrowse,noowners`, overwrite in place) and was reverted the
+same way; the original binary was restored byte for byte.
+
+## Restarts and a second race
+
+A locationd that launchd starts again does not always come up either. Killed
+every ~40 s on a running guest with `processes.kill … "signal":"TERM"`, five
+of seven new instances hung. (`processes.kill` takes a signal *name*; a
+numeric `"signal":9` falls back to TERM.) The same guest also logged
+locationd crash reports with `EXC_ARM_PAC_FAIL` 0.13 s after launch, the
+faulting thread draining a `Cohort:8` queue and calling a block whose invoke
+pointer was garbage: a second startup race, in the unmodified binary.
+
+Why the guest hits these so often is open. The guest runs an iOS 27 userland
+on the cloudOS 26.4 kernel (xnu 25.4), and both failures sit in libdispatch
+workloop and `dispatch_sync` paths. Nothing here tests that; it is the
+difference from a real iPhone most likely to matter.
+
+## Workaround: vphoned restarts it
+
+`VPhoneDaemon/Daemon/GuestLocationdWatchdog.swift`, started from `main.swift`:
+
+- calls `CLLocationManager.locationServicesEnabled()`, the cheapest
+ synchronous request locationd serves, every 15 s for the first five minutes
+ and every 60 s after;
+- when no answer comes within 10 s, sends SIGKILL to `/usr/libexec/locationd`
+ (a deadlocked one never finishes its SIGTERM handler), waits for the stuck
+ request to return, and asks again after one second, which has launchd start
+ a new locationd;
+- stops after 20 restarts in one vphoned lifetime and logs every action as
+ `vphoned: locationd watchdog: …`.
+
+Requests that were waiting on the dead locationd fail instead of blocking.
+
+### Verification
+
+Same VM, alternating between a bundle with the watchdog and one without
+(upstream 2.7.0), each bound with `vm set-bundle --update-environment`. A boot
+is probed with `location.current` at 65–75 s of uptime; a relaunch is a
+`processes.kill … "signal":"KILL"` of the running locationd, probed 10 s after
+launchd starts the next one.
+
+| vphoned | Boots with a working locationd | Relaunches with a working locationd |
+| --- | --- | --- |
+| without the watchdog | 3 of 5 | 3 of 6 |
+| with the watchdog | 14 of 14 | 6 of 6 |
+
+The watchdog's lines are in `/var/log/vphoned.log`, vphoned's stdout and
+stderr; `logs.syslog` does not return them. Over those runs it killed a hung
+locationd six times, once twice in a row because the first new instance hung
+as well. launchd's count of locationd launches (`services.print
+com.apple.locationd`, `runs =`) stayed at 3–4 on boots it did not act on.
+
diff --git a/Research/README.md b/Research/README.md
index fd78e49d..918d4e9f 100644
--- a/Research/README.md
+++ b/Research/README.md
@@ -20,7 +20,7 @@ This directory keeps the evidence behind firmware patches, restore and the self-
| Other patches and captures | [Launchd jetsam](Patches/cfw_patch_launchd_jetsam.md), [user-mode hypervisor references](Patches/hv_vmm_present_usermode_xrefs.md), [reference capture](Patches/patch_reference_capture.md) |
| Restore | [DFU probe](Restore/virtual_dfu_probe.md), [in-process restore](Restore/native_restore_architecture.md) |
| Host and archives | [Binary split](Host/host_binary_split.md), [runtime dependency tiers](Host/runtime_dependency_tiers.md), [archive extraction contracts](Host/archive_extraction_contracts.md), [libarchive validation](Host/libarchive_xcframework_validation.md), [guest MAC, fixed address, port forwarding and names](Host/network_fixed_address.md), [machine identity and `vm clone --new-identity`](Host/machine_identity_and_clone.md) |
-| Guest interaction and VM identity | [DevMode XPC](Guest/devmode_xpc_protocol.md), [keyboard events](Guest/keyboard_event_pipeline.md), [machine identifier](Guest/machine_identifier_storage_analysis.md), [Auto-Lock on the Lock Screen](Guest/lock_screen_idle_timer.md), [display refresh rate](Guest/display_refresh_rate.md), [GPU acceleration](Guest/gpu_acceleration.md), [RootHide bootstrap base](Guest/roothide_bootstrap_base.md), [the Mac's .local name](Guest/mac_name_resolution.md) |
+| Guest interaction and VM identity | [DevMode XPC](Guest/devmode_xpc_protocol.md), [keyboard events](Guest/keyboard_event_pipeline.md), [machine identifier](Guest/machine_identifier_storage_analysis.md), [Auto-Lock on the Lock Screen](Guest/lock_screen_idle_timer.md), [display refresh rate](Guest/display_refresh_rate.md), [GPU acceleration](Guest/gpu_acceleration.md), [RootHide bootstrap base](Guest/roothide_bootstrap_base.md), [the Mac's .local name](Guest/mac_name_resolution.md), [locationd startup deadlock](Guest/locationd_startup_deadlock.md) |
| Historical project records | [Migration ledger](History/intg_update_status.md), [manifest refactoring summary](History/manifest_and_refactoring_summary.md) |
`KernelSymbols/` holds symbol datasets and indexes; `Reference/` holds source references when present. They are evidence inputs, not steps for running the distributed app. The files in `History/` are preserved snapshots and may describe scripts or variants that have since been removed.
diff --git a/Skills/vphone-guest-control/references/finding-docs.md b/Skills/vphone-guest-control/references/finding-docs.md
index 46a794ea..8fae44d7 100644
--- a/Skills/vphone-guest-control/references/finding-docs.md
+++ b/Skills/vphone-guest-control/references/finding-docs.md
@@ -40,6 +40,7 @@ note's command lines.
| Developer Mode, DDI mount errors | `Research/Guest/devmode_xpc_protocol.md` |
| App install fails, signature, profile checks, UDID | `Research/Guest/xcode_install_signature_gate.md` |
| Location simulation returns `unavailable` | `Research/Guest/location_simulation_26_4_failure.md` |
+| `location.*` times out after 120 s, or every CoreLocation client hangs | `Research/Guest/locationd_startup_deadlock.md` |
| Camera / virtual camera | `Research/Guest/virtual_camera_transport.md` |
| Machine identity, clone behaving like the same device | `Research/Guest/machine_identifier_storage_analysis.md` |
| What a patch does, whether it is on, why a preset differs | `Research/0_binary_patch_comparison.md`, `Research/KernelCustomFirmwarePatches/README.md` |
diff --git a/VPhoneDaemon/Daemon/GuestLocationdWatchdog.swift b/VPhoneDaemon/Daemon/GuestLocationdWatchdog.swift
new file mode 100644
index 00000000..b16e2a16
--- /dev/null
+++ b/VPhoneDaemon/Daemon/GuestLocationdWatchdog.swift
@@ -0,0 +1,90 @@
+import CoreLocation
+import Darwin
+import Foundation
+import IcliKit
+
+/// Restarts a locationd that deadlocked while it was starting.
+///
+/// iOS 27.0's locationd can deadlock about a second after launch. Its main
+/// thread runs a run-loop callback (`CLLocationController::sendTechStatusNotification`,
+/// `checkActiveProviders`, or a silo timer) that waits synchronously on a silo
+/// still being constructed, and that construction waits for a silo served by
+/// the main queue. Nothing in locationd runs again, and every CoreLocation
+/// client blocks on `com.apple.locationd.synchronous` for good: `location.*`
+/// here, Maps, navd, nearbyd and the rest. It happened on about four boots in
+/// ten, headless or windowed alike, and on about half of the restarts after
+/// one. See Research/Guest/locationd_startup_deadlock.md.
+///
+/// vphoned asks locationd whether Location Services are on, the cheapest
+/// synchronous request it serves. When no answer comes within `timeout`,
+/// vphoned kills the running locationd and asks again at once, which has
+/// launchd start a new one; the requests that were waiting fail instead of
+/// blocking. A restart that deadlocks too is caught the same way.
+final class GuestLocationdWatchdog: @unchecked Sendable {
+ static let shared = GuestLocationdWatchdog()
+
+ private static let executable = "/usr/libexec/locationd"
+ /// A healthy locationd answers in milliseconds, also while it starts.
+ private static let timeout: DispatchTimeInterval = .seconds(10)
+ /// Often while locationd is most likely to be starting, then rarely.
+ private static let startupInterval: DispatchTimeInterval = .seconds(15)
+ private static let startupPeriod: TimeInterval = 300
+ private static let interval: DispatchTimeInterval = .seconds(60)
+ /// A locationd that deadlocks on every start is not helped by more restarts.
+ private static let maxRestarts = 20
+
+ /// Runs the checks; owns `restarts`.
+ private let queue = DispatchQueue(label: "vphoned.locationd-watchdog", qos: .utility)
+ /// Runs the request, which blocks for as long as locationd does not answer.
+ private let probeQueue = DispatchQueue(label: "vphoned.locationd-probe", qos: .utility)
+ private let started = Date()
+ private var restarts = 0
+
+ func startOnStartup() {
+ queue.asyncAfter(deadline: .now() + Self.startupInterval) { self.check() }
+ }
+
+ private func check() {
+ let answered = DispatchSemaphore(value: 0)
+ probeQueue.async {
+ _ = CLLocationManager.locationServicesEnabled()
+ answered.signal()
+ }
+ var restarted = false
+ if answered.wait(timeout: .now() + Self.timeout) == .timedOut {
+ restarted = restartLocationd()
+ // The request returns once its locationd is gone. Waiting for it
+ // keeps a second request from queueing behind a stuck first one.
+ _ = answered.wait(timeout: .now() + Self.timeout)
+ }
+ let next: DispatchTimeInterval = if restarted {
+ .seconds(1)
+ } else if Date().timeIntervalSince(started) < Self.startupPeriod {
+ Self.startupInterval
+ } else {
+ Self.interval
+ }
+ queue.asyncAfter(deadline: .now() + next) { self.check() }
+ }
+
+ /// Whether a locationd was killed.
+ private func restartLocationd() -> Bool {
+ let rows = (try? listProcesses(filter: "locationd"))?["processes"] as? [[String: Any]] ?? []
+ guard let pid = rows.first(where: { $0["executable"] as? String == Self.executable })?["pid"] as? Int else {
+ NSLog("vphoned: locationd watchdog: no answer, and no locationd is running")
+ return false
+ }
+ guard restarts < Self.maxRestarts else {
+ NSLog("vphoned: locationd watchdog: locationd %d does not answer; restarted %d times already, leaving it", pid, restarts)
+ return false
+ }
+ restarts += 1
+ // A deadlocked locationd does not finish its SIGTERM handler.
+ guard kill(pid_t(pid), SIGKILL) == 0 else {
+ NSLog("vphoned: locationd watchdog: kill(%d): %@", pid, String(cString: strerror(errno)))
+ return false
+ }
+ NSLog("vphoned: locationd watchdog: locationd %d did not answer, killed it (restart %d)", pid, restarts)
+ return true
+ }
+}
diff --git a/VPhoneDaemon/Daemon/main.swift b/VPhoneDaemon/Daemon/main.swift
index ce75466e..397d59bf 100644
--- a/VPhoneDaemon/Daemon/main.swift
+++ b/VPhoneDaemon/Daemon/main.swift
@@ -26,6 +26,8 @@ _ = vp_hid_load()
GuestIrisinInstaller.refreshBootstrapOnStartup()
GuestAPI.restoreUSBSerialOnStartup()
GuestStaticNames.shared.restoreOnStartup()
+// locationd can deadlock as it starts; this restarts it.
+GuestLocationdWatchdog.shared.startOnStartup()
let group = MultiThreadedEventLoopGroup(numberOfThreads: 2)
let filePool = NIOThreadPool(numberOfThreads: 2)