Skip to content

Run a virtual iPhone on an Apple Silicon Mac.

Swift

Keep a banded stream from deadlocking the host GPU

世界66956a87ec30
diff --git a/Research/0_binary_patch_comparison.md b/Research/0_binary_patch_comparison.md
index 0ddff6f3..3ee90ebc 100644
--- a/Research/0_binary_patch_comparison.md
+++ b/Research/0_binary_patch_comparison.md
@@ -565,7 +565,7 @@ do NOT execute these).
| 23 | No binary change: `libvcamcaptured.dylib`, which SystemHook already loads into `cameracaptured`, rewrites the `bl PrewarmThreadSafeSBPs` in `__FigCapturePreloadShadersInternal_block_invoke_2` to `nop` in the daemon's own memory. This runs from the dylib's constructor and only when the paravirtual Metal driver is installed — **all bases** | none (guest dylib `VPhoneGuestComponents/VCamCaptured/Prewarm/`, installed at `/usr/lib/libvcamcaptured.dylib`) | **Stop `cameracaptured` crashing at every boot, and the SpringBoard freeze that follows.** At launch the daemon preloads its shaders. `PrewarmThreadSafeSBPs` runs each video processor's `-prewarm`, and NRFV3's reaches `-[ToneMappingCurves initWithWithContext:]`, which fills textures made from a shared `MTLHeap` with `-replaceRegion:…`. The cloudOS 26.4 `AppleParavirtGPUMetalIOGPUFamily` never sets `_dimension` (a texture's CPU layout) in `-initWithHeap:resource:offset:length:descriptor:`, and `-replaceRegion:` loads it unchecked, so the daemon faults with `KERN_INVALID_ADDRESS` at `0xc` (measured on `iPad16,1 26.6.2 (23G90)`, `standard`: launchd at `successive crashes = 6`). While the daemon crash-loops, SpringBoard's main thread blocked in a synchronous XPC to it as soon as an app started recording. The daemon does have a Metal device, so `kernel-exp-paravirt_user_clients` does not help. Prewarming is only an optimisation, and nothing waits on that function: no return value, no global, no signal. The rest of the preload (processor flags, data migration, and the deferred shader cache copy, whose semaphore deferred processing waits on) is kept. The call site is found from the exported `FigCapturePreloadShaders`: its `b` to the internal function, that function's one `adrp/add/pacia x16` block invoke, then the block's one `ldr x0, [xN, #0x20]; bl` into CMCapture's `__text`. That gives `0x1aeb2fd68` on 23G90 and `0x1b0759d10` on 24A435. Any mismatch leaves the prewarm in place and logs the step that failed. Host test: `make -C VPhoneGuestComponents test-vcam-prewarm`. Validation: `vcamcaptured.log` has `gpu prewarm: … skipped`, no new `cameracaptured` crash report, and `successive crashes = 0`. See `Research/Guest/gpu_acceleration.md` ("Heap Textures Have No CPU Layout"). | Y | Y | Y |
| 24 | No binary change: both spawn hooks insert `libbatteryhealthfix.dylib` into `Settings` (`/Applications/Preferences.app/Preferences`) beside SystemHook, where it interposes `IOPSCopyPowerSourcesInfo`, `IOPSCopyPowerSourcesByType`, `IOPSCopyPowerSourcesByTypePrecise`, `notify_register_check` / `notify_get_state` / `notify_cancel` and `MGGetBoolAnswer` — **all bases** | none (new guest dylib `VPhoneGuestComponents/BatteryHealthFix/libbatteryhealthfix.c`, installed at `/usr/lib/libbatteryhealthfix.dylib`) | **Stop Settings → Battery → Battery Health & Charging loading forever.** The VM's battery is Virtualization.framework's synthetic power source (charge and connectivity only), and the health half of a real battery's description comes from AppleSmartBattery, which no VM has. Measured on `iPhone17,3 27.0 (24A435)`: powerd's internal battery description has one `BatteryPacks` entry holding only `PackID`, `Cycle count` and `Transport Type`, no `Battery Service State`/`Flags` or `Maximum Capacity Percent` anywhere, and `com.apple.thermalmonitor.ageAwareMitigationState` stays 0. `-[BatteryHealthUIController specifiers]` asks `-[BatteryHealthUIInformation batteryDataUnavailable]`, which answers YES while `+getManagementState` is 0, while no pack parses (`-[BatteryHealthUIPackInformation initWithPowerSourceInformation:]` drops a pack without service flags and state), or while the overall genuine status is -1 — and it is -1 because `MGGetBoolAnswer("D6/BMDrlb8V3WSiqL8gL+w")` (HasBatteryModuleAuth, `+[PLGestaltUtilities hasBatteryModuleAuth]`) answers 1 on the guest, sending `+genuineBatteryStatusForPackIndex:` to an `AppleBatteryAuth`/`AppleAuthCPRelay` service that does not exist. The controller then shows its global spinner and re-asks on a timer. iPadOS 26.6.2 (23G90) has the same gate in `+[BatteryUIResourceClass batteryDataUnavailable]`, reading the flat keys of `IOPSCopyPowerSourcesByType`'s array, the same notify name and the same MobileGestalt key. The hook answers as a new battery without battery authentication: each internal-battery description is copied with `Battery Service State` 0, `Battery Service Flags` 0, `Maximum Capacity Percent` 100 and `Cycle count` 0 at the top level, and each of powerd's packs (or one new pack) completed with a `BatteryHealth` dictionary of the same three numbers plus `Date of manufacture`/`Date of first use` (the birth time of `/private/var/mobile`, i.e. the restore); keys already present are never replaced, and a description that already has them is passed through. A mitigation state of 0 on a token registered for the age-aware name reads as 2 ("supporting normal peak performance"). HasBatteryModuleAuth reads as no only while `IOServiceGetMatchingService("AppleBatteryAuth")` finds nothing. Interposing reaches the callers because `BatteryUsageUI.bundle` is a standalone Mach-O bound through dyld's interposing table (cache-to-cache calls are not; see `MISFix/MISFixSignature.c`), and inserting it — rather than SystemHook `dlopen`ing it like row 21 — is what puts the table in place before the bundle loads. `vpInsertedLibraryFor` in `Shared/InjectionEnvironment.h` picks it or libmisfix for a path; the two target sets do not overlap. Ships with `system-launchdaemons-boot-environment`; existing guests get it from the vphoned environment update on their next start with the new bundle (offline `cfw update-environment` only replaces libraries a VM already has). launchd starts Settings itself, so it is the launchd hook that inserts it: a guest updated only by that online sync needs one more reboot before launchd runs the new hook, which the update already reports as `reboot_required`; an offline environment update replaces the hook before boot. **Validated (2026-10-04, `bhtest-iphone`, `iPhone17,3 27.0 (24A435)` + cloudOS 26.4):** before, the page showed only the spinner and the two charging switches; after, Maximum Capacity 100%, Peak Performance Capability "Your battery is currently supporting normal peak performance", the two switches, no spinner; the hook's log shows all three answers taken once per process. Re-checked on the final bundle after a reboot of the same guest, with the launchd hook logging `inserted+batteryhealthfix` for each Settings launch. **iPad16,1 26.6.2 (23G90), `bhtest-ipad`, created with the new bundle:** the hook is inserted and stays inert, and Settings runs without a crash; that iPad has no Battery Health page at all, because `-[BatteryUIController setUpBatteryHealthGroup]` offers the row only on an iPhone or, through `+[PLBatteryUIBackendModel shouldShowModifiedHealthController]`, on an iPad that supports the fixed charge limit with the `PerfPowerServices/battery_health_details` feature on — the model decides, as on hardware. The 26.x gate inside the page is from static analysis; no 26.x iPhone image was at hand to run it. | Y | Y | Y |
| 25 | No binary change: `libvcamcaptured.dylib` stops every other thread in `cameracaptured` (`task_threads` + `thread_suspend`, self excluded, one patcher at a time) around each `vcc_patch_word` / `vcc_patch_two_nops` text rewrite — **all bases** | none (guest dylib `VPhoneGuestComponents/VCamCaptured/Image/VCamCodePatch.m`, installed at `/usr/lib/libvcamcaptured.dylib`) | **Stop the hook's own runtime patching from crashing `cameracaptured`, which froze the whole VM when the camera was opened.** The hook rewrites CMCapture shared-cache `__text` through `vm_protect(RW|COPY)` → write → `vm_protect(RX)`; for the whole flip the 16K page carries no execute permission, and the pages it touches (the `-12783` error-code sites, the master-clock gate, the prewarm call) are live XPC server code. Two crash reports caught a thread fetching an instruction in a page mid-flip while the patching thread was in `vcc_patch_word` on the same boot: `ipad-mini-01` `cameracaptured-2026-10-03-045638.ips` died in `__FigFlashlightGetClassID_block_invoke` on `com.apple.coremedia.flashlight(34)-messages`, and `ipad-pro-13` `cameracaptured-2026-10-04-225558.ips` died in `captureSourceServer_handleRegisterNotificationForProprietaryDefaultChangesMessage` on `com.apple.coremedia.capturesource(523)-messages` — the register-for-source-changes XPC that camera enumeration itself sends. Both are `EXC_BAD_ACCESS (SIGBUS)`, `KERN_PROTECTION_FAILURE` on an instruction fetch, in a region snapshotted `r-x/rwx`, `SM=COW` (the rwx max-protection is the flip's fingerprint). The card: with row 23 removing the boot crash, any later `cameracaptured` restart (jetsam, another crash) while a camera client reconnects sends its first XPCs straight into the +3 s reinstall window; one SIGBUS → launchd throttles → every AVCapture client, SpringBoard's main thread included, blocks forever in a synchronous XPC — the "VM 卡死" report of 2026-10-05. The stopped section is exactly two `vm_protect` calls, the word write and `sys_icache_invalidate`: no locks, no allocation, no logging, so a frozen thread holding any userspace lock cannot deadlock the patcher; every `vcc_log` moved after the resume, and a mutex serialises patchers (constructor prewarm patch vs the +3 s install block). All five runtime text-patching call sites (`vcc_scan_and_patch` ×2, the prewarm NOP, the session-hook pair, the source-installation NOPs) funnel through the two fixed functions. Validation: kill-and-relaunch stress with Camera.app auto-reconnecting (the same collision that produced both crashes) produces no new `cameracaptured` crash report, and the boot log's `patch_word OK` lines are unchanged. See `Research/Guest/virtual_camera_transport.md` ("The patch race that froze the whole VM"). | Y | Y | Y |
-| 26 | No binary change: both spawn hooks insert `libdevicehubfix.dylib` into `cryptexd` and the DDI's `dtremotedisplayd` and `dtdeviceinfod`. In `cryptexd` it interposes `fcntl` and reports an `EPERM` from `F_SETPROTECTIONCLASS` as `ENOTSUP`; in `dtremotedisplayd` it interposes `MediaStreamSupportedFeatures.current` (answers `0x8c`) and `CurrentDevice.isCustomerRestricted` (answers false) from the DDI's `CoreDeviceUtilities`; in `dtdeviceinfod` it interposes `MGCopyAnswer` and fills a missing `FramebufferIdentifier` for `phone11` chrome — **all bases** | none (new guest dylib `VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c`, installed at `/usr/lib/libdevicehubfix.dylib`) | **Let Xcode 27's DeviceHub show the guest's screen.** On iOS 27 `cryptexd` stages each DDI asset with `open(O_RDWR)` + `fcntl(F_SETPROTECTIONCLASS, 4)` and accepts only `ENOTSUP` as a failure; a guest answers `EPERM` (`set protection class: [1: Operation not permitted]`, `staging failed`), so CoreDevice shows the device as "connected (no DDI)" for good. With the DDI mounted, DeviceHub asks `dtremotedisplayd` for its media stream features and gets 0 from CoreDeviceUtilities' policy: `isCustomerRestricted` (`isProductionFused && !hasInternalOSBuild && hasInternalDDI == false`) refuses every pre-27.0 device, and remote control is limited to iPhone, Apple Watch and Apple TV, which refuses every iPad. `0x8c` is what a Mac host reports for itself (mirrored primary display 0x4, system audio 0x8, display information 0x80). The 27.x iPhone guest reports `phone11` chrome with no `FramebufferIdentifier`, so DeviceHub drew the screen over the bezel corners; the hook answers only that missing key with Xcode's matching mask `4E5532ED-1470-47D1-BDF4-7AA90C26957A` from `/Library/Developer/DeviceKit/chrome_map.plist`. Interposing reaches all of them because `cryptexd`, `dtremotedisplayd`, `dtdeviceinfod` and `CoreDeviceUtilities` are standalone images; the two Swift symbols are weak flat-namespace imports, resolving to nothing outside `dtremotedisplayd`. On 27.x the view also needs row 11's conditional dispatch, or the capture display stays black. Ships with `system-launchdaemons-boot-environment`; an existing guest gets it from the environment update. Details: `Research/Guest/devicehub_screen_viewing.md`. **Validated (2026-10-06/07):** `iPad16,1 26.6.2 (23G90)`: DeviceHub receives `supportedFeatures: 140`, `MediaStreamStart` succeeds and the view shows the live home screen. `iPhone17,3 27.0.1 (24A446)`: the DDI mounts (`F_SETPROTECTIONCLASS EPERM, reported as ENOTSUP`), and with row 11's conditional dispatch DeviceHub shows the home screen and Settings live, clipped inside the chrome (`Found FramebufferMaskIdentifier from displayInfo`). Open: latency that grows over a long session and occasional stalls after a cold start, both upstream of the Mac receiver. | Y | Y | Y |
+| 26 | No binary change: both spawn hooks insert `libdevicehubfix.dylib` into `cryptexd`, the DDI's `dtremotedisplayd` and `dtdeviceinfod`, and `avconferenced`. In `cryptexd` it interposes `fcntl` and reports an `EPERM` from `F_SETPROTECTIONCLASS` as `ENOTSUP`; in `dtremotedisplayd` it interposes `MediaStreamSupportedFeatures.current` (answers `0x8c`) and `CurrentDevice.isCustomerRestricted` (answers false) from the DDI's `CoreDeviceUtilities`; in `dtdeviceinfod` it interposes `MGCopyAnswer` and fills a missing `FramebufferIdentifier` for `phone11` chrome; in `avconferenced` it replaces `VCPSideCarMetal`'s two band reads through the Objective-C runtime and runs one synchronous scaler transform before the first read of each frame — **all bases** | none (new guest dylib `VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c`, installed at `/usr/lib/libdevicehubfix.dylib`) | **Let Xcode 27's DeviceHub show the guest's screen.** On iOS 27 `cryptexd` stages each DDI asset with `open(O_RDWR)` + `fcntl(F_SETPROTECTIONCLASS, 4)` and accepts only `ENOTSUP` as a failure; a guest answers `EPERM` (`set protection class: [1: Operation not permitted]`, `staging failed`), so CoreDevice shows the device as "connected (no DDI)" for good. With the DDI mounted, DeviceHub asks `dtremotedisplayd` for its media stream features and gets 0 from CoreDeviceUtilities' policy: `isCustomerRestricted` (`isProductionFused && !hasInternalOSBuild && hasInternalDDI == false`) refuses every pre-27.0 device, and remote control is limited to iPhone, Apple Watch and Apple TV, which refuses every iPad. `0x8c` is what a Mac host reports for itself (mirrored primary display 0x4, system audio 0x8, display information 0x80). The 27.x iPhone guest reports `phone11` chrome with no `FramebufferIdentifier`, so DeviceHub drew the screen over the bezel corners; the hook answers only that missing key with Xcode's matching mask `4E5532ED-1470-47D1-BDF4-7AA90C26957A` from `/Library/Developer/DeviceKit/chrome_map.plist`. Interposing reaches all of them because `cryptexd`, `dtremotedisplayd`, `dtdeviceinfod` and `CoreDeviceUtilities` are standalone images; the two Swift symbols are weak flat-namespace imports, resolving to nothing outside `dtremotedisplayd`. On 27.x the view also needs row 11's conditional dispatch, or the capture display stays black. A stream that splits each frame into four bands (`tilesPerFrame` 4) deadlocked the host GPU within 16–35 s of screen activity: `VCPSideCarMetal` reads every frame once per band while the paravirtual scaler's write of that frame can still be unregistered on the host, and a band read that registers the first of the frame's two planes before the write and the second after it waits on the write, which waits on it (`Timeout in timestamp wait`, the guest loses its GPU). Waiting for one synchronous transform puts the band reads behind the frame's write. `VideoProcessing` is in the shared cache, so this part is a method replacement, not an interpose. Ships with `system-launchdaemons-boot-environment`; an existing guest gets it from the environment update. Details: `Research/Guest/devicehub_screen_viewing.md`. **Validated (2026-10-06/07):** `iPad16,1 26.6.2 (23G90)`: DeviceHub receives `supportedFeatures: 140`, `MediaStreamStart` succeeds and the view shows the live home screen. `iPhone17,3 27.0.1 (24A446)`: the DDI mounts (`F_SETPROTECTIONCLASS EPERM, reported as ENOTSUP`), and with row 11's conditional dispatch DeviceHub shows the home screen and Settings live, clipped inside the chrome (`Found FramebufferMaskIdentifier from displayInfo`). **Band reads validated (2026-10-07/08, `iPhone17,3 27.0.1 (24A446)`):** a four-band stream with screen activity deadlocked after 18 s without the barrier and ran 60, 180 and 300 s with it without a GPU error; in the host trace of the 180 s run no IOSurface access waited, and every band read of 6258 frames registered after the frame's write was released, a median 0.76 ms after it. Open: latency that grows over a long session and occasional stalls after a cold start, both upstream of the Mac receiver. | Y | Y | Y |
| 27 | No binary change: both spawn hooks insert `libdevicename.dylib` into `configd` and `lockdownd` (after `libmisfix.dylib`, the first process to take two inserted libraries). In `configd` it interposes `SCDynamicStoreSetMultiple` and `SCDynamicStoreSetValue` and pins `Setup:/System`'s `ComputerName`; in `lockdownd` it interposes `SCPreferencesSetComputerName`, `SCPreferencesSetHostName` and `SCPreferencesSetLocalHostName` and refuses every rename with `kSCStatusAccessError`. Inert unless `/var/db/vphone/devicename.plist` holds a valid `DeviceName` — **all bases** | none (new guest dylib `VPhoneGuestComponents/DeviceName/libdevicename.c`, installed at `/usr/lib/libdevicename.dylib`; vphoned `device.name.get`/`device.name.set`) | **Pin the guest's device name to the VM's name.** The name is `System/System/ComputerName` in `preferences.plist`; configd's PreferencesMonitor (`updateConfiguration`, 24A435 `0x100062098`) flattens it into `Setup:/System` with one `SCDynamicStoreSetMultiple`, and every reader (`SCDynamicStoreCopyComputerName` in lockdownd's `copy_device_name`, MobileGestalt `UserAssignedDeviceName`, `UIDevice.name`) reads the store. The hook rewrites that publication, not the preferences read: the monitor's model-change path (`sub_10006155c`) writes the `kSCPrefSystem` value it reads back to disk, and the publication route leaves `preferences.plist` with the guest's own name. A publication that leaves `Setup:/System` out as unchanged gets the store's value pinned; one that removes it gets the pinned name instead. `vphone-vm` sends the VM's name to vphoned's `device.name.set` after every connect; vphoned writes the file (one rename, 0644) and applies the preferences unchanged, so configd publishes again and the new pin, or the guest's own name when the file is removed, is live within a second. The file outlives a reboot, so configd's first publication at boot carries it. The library re-reads the file when its inode, size or mtime change. lockdownd's `set_device_name` (`0x1000073b0`) is the only caller of the three setters; it logs their failure and still commits, and the `SetValue DeviceName` handler ignores the result, so a host rename reports success and the name stays. The pinned name itself is refused too: lockdownd runs `set_device_name(copy_device_name())` whenever it starts, and a lockdownd restarted after configd published would otherwise write the pinned name and derived host names into `preferences.plist` (seen once). An NVRAM variable was the first channel and never reached the guest: iBoot keeps only the variables it knows (even `vphone-vm`'s `boot-args` write is replaced). Interposing works because both daemons are standalone images. Ships with `system-launchdaemons-boot-environment` and the environment update; takes effect at the next boot. Verified on iOS 27.0 (24A435) on 2026-10-07: pin at boot and live, host rename refused, `vm rename` followed, preferences untouched. CoreDevice (`devicectl`, DeviceHub) reads the name when the device appears and can lag one boot after a rename. Details: `Research/Guest/device_name_pinning.md`. | Y | Y | Y |
diff --git a/Research/Guest/devicehub_screen_viewing.md b/Research/Guest/devicehub_screen_viewing.md
index e772a2c5..78f6b239 100644
--- a/Research/Guest/devicehub_screen_viewing.md
+++ b/Research/Guest/devicehub_screen_viewing.md
@@ -16,7 +16,10 @@ Mac busy restoring or downloading) the view can stall for seconds or fall
seconds behind and stay there until the stream restarts. A single stream on a
quiet Mac showed no growing delay in two hours, though two short encode stalls
still occurred (see
-[Known issue](#known-issue-stalls-and-growing-latency-under-load)).
+[Known issue](#known-issue-stalls-and-growing-latency-under-load)). A stream
+that splits each frame into bands deadlocked the host GPU until the library
+also loaded into `avconferenced` (see
+[Banded streams](#banded-streams-deadlock-the-host-gpu)).
## What stopped it
@@ -168,7 +171,7 @@ to an iPhone 16 (1179x2556) did not change the overflow either.
### `libdevicehubfix.dylib`
`VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c`, installed as
-`/usr/lib/libdevicehubfix.dylib`. Both spawn hooks insert it into three
+`/usr/lib/libdevicehubfix.dylib`. Both spawn hooks insert it into four
processes only (`vpIsDeviceHubFixTarget` in `Shared/InjectionEnvironment.h`):
- **`cryptexd`**: interposes `fcntl`. An `EPERM` from `F_SETPROTECTIONCLASS`
@@ -186,12 +189,17 @@ processes only (`vpIsDeviceHubFixTarget` in `Shared/InjectionEnvironment.h`):
`phone11`, it answers with Xcode's matching mask `4E5532ED-…`, retained under
the Copy ownership rule. Existing answers and other chromes are unchanged.
This is display metadata, not a change to the guest's hardware identity.
+- **`avconferenced`**: replaces `VCPSideCarMetal`'s two band reads and runs
+ one synchronous scaler transform before the first read of each frame (see
+ [Banded streams](#banded-streams-deadlock-the-host-gpu)).
Interposing works because `cryptexd`, `dtremotedisplayd`, `dtdeviceinfod` and
`CoreDeviceUtilities` are standalone images, not shared-cache ones, so dyld
binds their imports through the interposing table. `CoreDeviceUtilities` lives
on the DDI and not in the SDK, so its two symbols are weak flat-namespace
imports (`-Wl,-U`) and resolve to nothing outside `dtremotedisplayd`.
+`VideoProcessing` is in the shared cache, which an interpose does not reach,
+so the band reads are replaced through the Objective-C runtime.
The library ships with `system-launchdaemons-boot-environment`. An existing
guest gets it from the environment update when it next starts on a bundle
@@ -276,7 +284,7 @@ showed the guest live, 30–40 ms behind the VZ window.
refuses pairing (`kAMDUserDeniedPairingError`, from `devicectl manage pair`
and from Xcode alike) without showing a trust alert. That happens before
any of this code is involved and is left for separate work.
-- **Tests.** `InjectionEnvironmentTests` covers the three targets and their
+- **Tests.** `InjectionEnvironmentTests` covers the four targets and their
near-miss suffixes. The seven real-cache force-kern tests (pristine 24A435
cache via `VPHONE_DSC_PRISTINE`) cover conditional routing, the preserved
null check, symbol discovery, dry run and idempotence, and pass. `ARM64CompareBranchTests` pins the new encoder.
@@ -284,6 +292,122 @@ showed the guest live, 30–40 ms behind the VZ window.
Touch input sent from inside DeviceHub reaches the guest (checked by hand),
but has no automated check.
+## Banded streams deadlock the host GPU
+
+Fixed. A stream can split each frame into bands that are encoded separately
+(`tilesPerFrame` in the stream configuration). With four bands and a changing
+screen, the 27.0.1 iPhone guest froze within 16–35 s: the picture stopped
+while touch still worked, then the guest turned green and the VZ window
+stopped, until the VM was restarted. The Mac logged `Timeout in timestamp
+wait` and restarted its GPU, and the guest never got it back. A still screen
+produces no new frames and did not freeze; a single-band stream never froze.
+
+Measured with macOS 27.0.1 (26A434) on an M2 Mac and `iPhone17,3` 27.0.1
+(24A446) over cloudOS 26.4 (23E5207q), 2026-10-07 and 2026-10-08.
+
+### Who writes the frame and who reads it
+
+- backboardd's CoreAnimation display thread
+ (`com.apple.coreanimation.display.primary`) writes each capture frame (420f,
+ 1320x2868) with the M2 scaler, through
+ `IOSurfaceAcceleratorTransformSurface`.
+- `avconferenced` gets the frame before that transform has run on the Mac and
+ does not wait for it: IOSurface implicit synchronization orders its reads
+ after the write.
+- With bands, VideoProcessing's `VCPSideCarMetal` reads the frame on the GPU
+ once per band, in `temporalTransitionScore:previousFrame:forRegion:`
+ (current and previous frame) and `copyFromFrame:toTile:origin:size:withFence:`.
+ Each command buffer samples the frame's two planes as two textures. With one
+ band neither method runs and the GPU does not read the frame.
+
+Both run on the Mac. The guest's Metal commands are replayed by one
+`ParavirtualizedGraphicsGPUTask` process per guest process, and scaler
+requests (`AppleM2ScalerParavirtDriver`) by the VM process. Both end in the
+Mac's IOSurface implicit synchronization, which the guest cannot see.
+
+### Host synchronization
+
+From the Mac's kdebug class `0x8521` (see [Measuring](#measuring)):
+
+- The VM process runs scaler requests one at a time, and registers the write
+ 0.25–0.8 ms after taking a request.
+- The GPU task registers a command buffer's accesses on its own path, and can
+ register a band read inside that window, before the write.
+- A command buffer registers each texture on its own, so the frame's two
+ planes are two registrations, and another registration can land between
+ them.
+- Once a writer waits, new readers queue behind it.
+
+At the deadlock, a band read registered plane 0 of the current frame, the
+scaler's write registered 1 µs later and waited for it, and the same command
+buffer's plane 1 registration 1 µs after that queued behind the write. The
+surface was never released again; the scaler queue and the GPU FIFO stopped
+together. In two traced runs a band read registered before the write in 26 of
+779 and 20 of 564 frames. Most such inversions are harmless, because the write
+waits for the read; only the 1–2 µs between two plane registrations
+deadlocks, which fits a freeze after tens of seconds.
+
+### The fix
+
+Before the first band read of each frame (keyed by IOSurface ID),
+`avconferenced` runs one synchronous transform between two scratch surfaces,
+64x64 to 32x32. When `avconferenced` reads a frame, the VM process has always
+taken the request that writes it already (779 of 779 frames, at least 234 µs
+earlier), and a synchronous transform returns only after the host released its
+write. The scaler runs requests in order, so when the barrier returns the
+frame's write is done and no band read can meet it.
+
+Neither method takes a floating-point argument, and the score method returns a
+C++ future through `x8`, so the replacement saves `x0`–`x8`, calls the barrier
+with the frame and continues into the original. If either method is missing,
+nothing is replaced and the library logs `VCPSideCarMetal band reads not
+found`.
+
+Ruled out:
+
+- `IOSurfaceLock` before the reads returns within microseconds without waiting
+ for the host's write, and still deadlocked after 25 s.
+- Waiting after the write in backboardd: QuartzCore reaches the scaler through
+ a branch island, not its GOT, so an interpose does not see the call.
+- One band per frame avoids the deadlock but gives up the lower latency bands
+ are for.
+- Leaving the scaler out of the VM configuration
+ (`_VZMacScalerAcceleratorDeviceConfiguration`) would change every guest
+ process that uses it; not tried.
+
+### Validation
+
+Four bands while swiping back and forth on the home screen, unless noted:
+
+| `avconferenced` | Duration | Result |
+| --- | --- | --- |
+| unchanged | 18 s | deadlock |
+| `IOSurfaceLock` before the reads | 25 s | deadlock |
+| barrier | 60 s | no scaler write waited, no GPU error |
+| barrier, after a reboot, traced | 180 s | no IOSurface access waited; every band read of 6258 frames after the write's release |
+| barrier, after another reboot | 300 s | no GPU error, about 41 fps |
+| barrier, one band | 90 s | about 55 fps as before, no barrier transform |
+
+The barrier relies on two host behaviors that were observed, not documented:
+scaler requests run one at a time and in order, and a frame's request is taken
+before `avconferenced` reads the frame. A surface reused for two consecutive
+frames would skip the second barrier; this was not seen, since
+`temporalTransitionScore` holds the previous frame, and all 74651 band-read
+command buffers of the 180 s run came after their frame's barrier.
+
+### Cost
+
+- Only `avconferenced`, and only when frames are split into bands: the two
+ methods are not called otherwise. A single-band stream keeps its frame rate
+ and the host sees no barrier transform.
+- `avconferenced` waits in the barrier about 2 ms a frame, but without it most
+ band reads waited on the host for the same write. What the barrier adds is
+ the time from the write's release to the first band read: 760 µs median,
+ 442 µs at least.
+- Four bands ran at 35–41 fps with the barrier, against about 29 fps without
+ it before the deadlock.
+- The host runs one extra 64x64 to 32x32 transform per frame.
+
## Known issue: stalls and growing latency under load
Not fixed. What a user sees and what helps:
@@ -420,6 +544,9 @@ seconds.
- Touch input sent from DeviceHub has no automated check.
- The `0x14` port offset in the conditional dispatch is fixed, not derived;
check it on the next 27.x build.
+- The band-read barrier was tested on the 27.0.1 iPhone only. It depends on
+ two private `VCPSideCarMetal` method names; if they change, the library logs
+ that they were not found and banded streams can deadlock again.
## Measuring
@@ -451,6 +578,13 @@ seconds.
`Health: VCVideoPlayer` (presentation times) and `VCPDec` (decoder input).
Player addresses are reused across streams, so start a stream's series at
its `displayLinkTickCount=0` line rather than matching the pointer.
+- **Host IOSurface synchronization:** `sudo ktrace dump -b 1024 -T 210s -f
+ C0x85 <file>` while streaming, then `ktrace trace -R <file> -N --ndjson`. In
+ class `0x8521`, `0x85210000` registers an access, `0x85210004` waits,
+ `0x85210008` is granted and `0x8521000c` releases; `arg1` is 1 for the
+ scaler and other non-GPU access in the VM process and 2 for the GPU, `arg2`
+ the surface ID, `arg3` 1 for a read and 2 for a write. `0x8521001c` is the
+ scaler looking up a surface as it takes a request. A run writes 1–1.5 GB.
- **Guest logs:** vphoned `logs.syslog` takes `level` `all`, `error` or
`fault`; info lines are not persisted. `avconferenced` prints
`fvdp_statsReport`, `VCPEnc` and `Health: VideoTransmitter` every second.
diff --git a/VPhoneExecutable/VPhoneCommand/FirmwarePatcher/PatchSets/FirmwareGuestSystemPatchSet.swift b/VPhoneExecutable/VPhoneCommand/FirmwarePatcher/PatchSets/FirmwareGuestSystemPatchSet.swift
index a1e35626..017c0a78 100644
--- a/VPhoneExecutable/VPhoneCommand/FirmwarePatcher/PatchSets/FirmwareGuestSystemPatchSet.swift
+++ b/VPhoneExecutable/VPhoneCommand/FirmwarePatcher/PatchSets/FirmwareGuestSystemPatchSet.swift
@@ -207,8 +207,10 @@ public enum FirmwareGuestSystemPatchSet {
haptic hardware, libbatteryhealthfix.dylib, inserted into Settings, which gives \
the virtual battery the health record Battery Health waits for, and \
libdevicehubfix.dylib, inserted into cryptexd, which then stages the iOS 27 \
- developer disk image, and into the image's dtremotedisplayd, which then reports \
- the media stream features Xcode's DeviceHub needs to show the screen. \
+ developer disk image, into the image's dtremotedisplayd and dtdeviceinfod, \
+ which then report the media stream features and the screen mask Xcode's \
+ DeviceHub needs to show the screen, and into avconferenced, where a stream \
+ split into bands would otherwise deadlock the host GPU. \
libdevicename.dylib, inserted into configd and lockdownd, does nothing until \
vphoned stores a device name (the host sends the VM's name); then configd \
publishes that name and lockdownd refuses to rename the device.
diff --git a/VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c b/VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c
index 8aa6dacf..573df52a 100644
--- a/VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c
+++ b/VPhoneGuestComponents/DeviceHubFix/libdevicehubfix.c
@@ -1,9 +1,10 @@
// libdevicehubfix.c — let Xcode's DeviceHub show a vphone guest's screen.
//
// DeviceHub (Xcode 27) views a device through the developer disk image's
-// dtremotedisplayd. On a vphone guest it needs three fixes, one per process:
+// dtremotedisplayd. On a vphone guest it needs four fixes, one per process:
// cryptexd mounts the DDI, dtremotedisplayd reports media stream features,
-// and dtdeviceinfod supplies the framebuffer mask.
+// dtdeviceinfod supplies the framebuffer mask, and avconferenced keeps a
+// banded stream from deadlocking the host GPU.
//
// ## cryptexd: the DDI does not mount on iOS 27
//
@@ -63,6 +64,40 @@
// Xcode's /Library/Developer/DeviceKit/chrome_map.plist; existing answers and
// other chromes pass through, and the result keeps the Copy ownership rule.
//
+// ## avconferenced: a banded stream deadlocks the host GPU
+//
+// A RemoteDesktop stream may split each frame into bands (tilesPerFrame 4).
+// avconferenced then reads every captured frame on the GPU once per band, in
+// VideoProcessing's VCPSideCarMetal: `temporalTransitionScore:previousFrame:
+// forRegion:` and `copyFromFrame:toTile:origin:size:withFence:` each commit a
+// command buffer that samples the frame's two 420f planes as two textures.
+// The frame is written by backboardd's CoreAnimation display thread with the
+// paravirtual M2 scaler (`IOSurfaceAcceleratorTransformSurface`), and reaches
+// avconferenced while that transform is still in flight: IOSurface implicit
+// synchronization is what orders the band reads after the write.
+//
+// On the host the two reach that synchronization by different paths. The
+// VM process runs scaler requests one at a time and registers the write
+// 0.25–0.8 ms after taking a request; the band command buffers come through
+// ParavirtualizedGraphics' GPU task, which registers each texture's surface on
+// its own, and can get there first. Once a writer waits, the host queues new
+// readers behind it. When the scaler's write registers between the two plane
+// registrations of one command buffer, the buffer holds the first plane and
+// waits for the writer, which waits for that read: the scaler and the GPU
+// FIFO both stop ("Timeout in timestamp wait"), the host restarts the GPU and
+// the guest loses it for good. With on-screen activity this hit 3% of frames
+// and deadlocked within 16–35 s (macOS 27.0.1 on M2, iOS 27.0.1 guest with
+// cloudOS 26.4 paravirtual drivers). A single-band stream never reads the
+// frame on the GPU.
+//
+// The request that produced a frame is already on the host's scaler queue
+// when avconferenced gets the frame, and a synchronous transform returns only
+// after the host finished it. So before the first band read of each frame one
+// synchronous transform between two scratch surfaces is queued behind it;
+// when it returns the frame's write is done and released, and no band read
+// can meet it. That costs one scaler round trip per frame, 0.5–1 ms past the
+// write the band reads used to wait for on the host anyway.
+//
// ## Reach
//
// The interposes reach their callers because cryptexd, dtremotedisplayd,
@@ -70,11 +105,16 @@
// ones: dyld binds their imports through the interposing table.
// CoreDeviceUtilities is on the DDI and not in the SDK, so its two symbols are
// weak flat-namespace imports; outside dtremotedisplayd they resolve to nothing
-// and dyld skips those entries. The spawn hooks insert this into those three
-// processes only — `vpIsDeviceHubFixTarget` in Shared/InjectionEnvironment.h.
+// and dyld skips those entries. VideoProcessing is in the shared cache, so the
+// band reads are replaced through the Objective-C runtime instead. The spawn
+// hooks insert this into those four processes only — `vpIsDeviceHubFixTarget`
+// in Shared/InjectionEnvironment.h.
+#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
+#include <objc/runtime.h>
+#include <pthread.h>
#include <stdarg.h>
#include <stdatomic.h>
#include <stdio.h>
@@ -82,6 +122,8 @@
#include <sys/stat.h>
#include <unistd.h>
#include <CoreFoundation/CoreFoundation.h>
+#include <CoreVideo/CoreVideo.h>
+#include <IOSurface/IOSurfaceRef.h>
extern CFTypeRef MGCopyAnswer(CFStringRef key, CFDictionaryRef options);
@@ -187,7 +229,119 @@ static CFTypeRef vpDisplayAnswer(CFStringRef key, CFDictionaryRef options) {
return result;
}
-int vphone_devicehubfix_version(void) { return 2; }
+// MARK: - avconferenced
+
+#define VP_VIDEO_PROCESSING "/System/Library/PrivateFrameworks/VideoProcessing.framework/VideoProcessing"
+
+extern int IOSurfaceAcceleratorCreate(CFAllocatorRef allocator, CFDictionaryRef properties, void **accelerator);
+extern int IOSurfaceAcceleratorTransformSurface(void *accelerator, IOSurfaceRef source, IOSurfaceRef destination,
+ CFDictionaryRef options, void *rectangles, void *completion,
+ void *swap, void *command);
+
+__attribute__((used)) static IMP vpOriginalTransitionScore;
+__attribute__((used)) static IMP vpOriginalCopyFromFrame;
+
+static void *vpScaler;
+static CVPixelBufferRef vpScalerSource;
+static CVPixelBufferRef vpScalerDestination;
+
+static CVPixelBufferRef vpScratchBuffer(size_t width, size_t height) {
+ CFDictionaryRef surface = CFDictionaryCreate(NULL, NULL, NULL, 0, &kCFTypeDictionaryKeyCallBacks,
+ &kCFTypeDictionaryValueCallBacks);
+ const void *keys[] = {kCVPixelBufferIOSurfacePropertiesKey};
+ const void *values[] = {surface};
+ CFDictionaryRef attributes = CFDictionaryCreate(NULL, keys, values, 1, &kCFTypeDictionaryKeyCallBacks,
+ &kCFTypeDictionaryValueCallBacks);
+ CVPixelBufferRef buffer = NULL;
+ CVPixelBufferCreate(NULL, width, height, kCVPixelFormatType_32BGRA, attributes, &buffer);
+ CFRelease(attributes);
+ CFRelease(surface);
+ return buffer;
+}
+
+static void vpCreateScalerBarrier(void) {
+ if (IOSurfaceAcceleratorCreate(NULL, NULL, &vpScaler) != 0)
+ vpScaler = NULL;
+ vpScalerSource = vpScratchBuffer(64, 64);
+ vpScalerDestination = vpScratchBuffer(32, 32);
+ if (!vpScaler || !vpScalerSource || !vpScalerDestination)
+ vpLog("scaler barrier %s, %s", "unavailable", "a banded stream can deadlock");
+}
+
+__attribute__((used)) static void vpAwaitScaler(CVPixelBufferRef frame) {
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+ static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
+ static _Atomic uint32_t awaited;
+ IOSurfaceRef surface = frame ? CVPixelBufferGetIOSurface(frame) : NULL;
+ if (!surface)
+ return;
+ uint32_t id = IOSurfaceGetID(surface);
+ if (atomic_load(&awaited) == id)
+ return;
+ pthread_once(&once, vpCreateScalerBarrier);
+ pthread_mutex_lock(&lock);
+ if (atomic_load(&awaited) != id) {
+ if (vpScaler && vpScalerSource && vpScalerDestination)
+ IOSurfaceAcceleratorTransformSurface(vpScaler, CVPixelBufferGetIOSurface(vpScalerSource),
+ CVPixelBufferGetIOSurface(vpScalerDestination), NULL, NULL,
+ NULL, NULL, NULL);
+ atomic_store(&awaited, id);
+ }
+ pthread_mutex_unlock(&lock);
+}
+
+// Calls vpAwaitScaler with the frame (x2) and continues into the original
+// with x0–x8 intact. Neither method takes a floating-point argument (region,
+// origin and size are passed by reference), and the score method returns a
+// C++ future indirectly, through x8.
+#define VP_AWAIT_SCALER_THEN(name, original) \
+ __attribute__((naked)) static void name(void) { \
+ __asm__ volatile("pacibsp\n" \
+ "stp x29, x30, [sp, #-0x60]!\n" \
+ "mov x29, sp\n" \
+ "stp x0, x1, [sp, #0x10]\n" \
+ "stp x2, x3, [sp, #0x20]\n" \
+ "stp x4, x5, [sp, #0x30]\n" \
+ "stp x6, x7, [sp, #0x40]\n" \
+ "str x8, [sp, #0x50]\n" \
+ "mov x0, x2\n" \
+ "bl _vpAwaitScaler\n" \
+ "ldp x0, x1, [sp, #0x10]\n" \
+ "ldp x2, x3, [sp, #0x20]\n" \
+ "ldp x4, x5, [sp, #0x30]\n" \
+ "ldp x6, x7, [sp, #0x40]\n" \
+ "ldr x8, [sp, #0x50]\n" \
+ "ldp x29, x30, [sp], #0x60\n" \
+ "autibsp\n" \
+ "adrp x16, _" #original "@PAGE\n" \
+ "ldr x16, [x16, _" #original "@PAGEOFF]\n" \
+ "braaz x16\n"); \
+ }
+
+VP_AWAIT_SCALER_THEN(vpTransitionScore, vpOriginalTransitionScore)
+VP_AWAIT_SCALER_THEN(vpCopyFromFrame, vpOriginalCopyFromFrame)
+
+__attribute__((constructor)) static void vpInstallScalerBarrier(void) {
+ if (strcmp(getprogname(), "avconferenced") != 0)
+ return;
+ dlopen(VP_VIDEO_PROCESSING, RTLD_LAZY | RTLD_LOCAL);
+ Class sideCar = objc_getClass("VCPSideCarMetal");
+ Method score = sideCar ? class_getInstanceMethod(sideCar, sel_registerName(
+ "temporalTransitionScore:previousFrame:forRegion:"))
+ : NULL;
+ Method copy = sideCar ? class_getInstanceMethod(sideCar, sel_registerName(
+ "copyFromFrame:toTile:origin:size:withFence:"))
+ : NULL;
+ if (!score || !copy) {
+ vpLog("VCPSideCarMetal band reads %s, %s", "not found", "a banded stream can deadlock");
+ return;
+ }
+ vpOriginalTransitionScore = method_setImplementation(score, (IMP)vpTransitionScore);
+ vpOriginalCopyFromFrame = method_setImplementation(copy, (IMP)vpCopyFromFrame);
+ vpLog("VCPSideCarMetal band reads %s, %s", "found", "they wait for the scaler");
+}
+
+int vphone_devicehubfix_version(void) { return 3; }
__attribute__((used, section("__DATA,__interpose"))) static const struct {
const void *replacement;
diff --git a/VPhoneGuestComponents/Makefile b/VPhoneGuestComponents/Makefile
index 53ae7242..a47bef03 100644
--- a/VPhoneGuestComponents/Makefile
+++ b/VPhoneGuestComponents/Makefile
@@ -58,7 +58,8 @@ BATTERYHEALTHFIX_SOURCES := $(wildcard BatteryHealthFix/*.c)
BATTERYHEALTHFIX_INCLUDES := $(wildcard BatteryHealthFix/*.h)
# CoreDeviceUtilities lives on the developer disk image, not in the SDK, and
# the image is mounted only after the host enables developer services. The two
-# interposed Swift getters are weak flat-namespace imports.
+# interposed Swift getters are weak flat-namespace imports. CoreVideo, IOSurface
+# and libobjc are for the avconferenced band reads it waits on the scaler for.
DEVICEHUBFIX := $(STAGE)/devicehubfix/libdevicehubfix.dylib
DEVICEHUBFIX_SOURCES := $(wildcard DeviceHubFix/*.c)
DEVICEHUBFIX_INCLUDES := $(wildcard DeviceHubFix/*.h)
@@ -249,7 +250,7 @@ $(DEVICEHUBFIX): $(DEVICEHUBFIX_SOURCES) $(DEVICEHUBFIX_INCLUDES) DeviceHubFix |
$(CLANG) $(GUEST_C_FLAGS) -dynamiclib \
-install_name /usr/lib/libdevicehubfix.dylib \
-o $@ $(DEVICEHUBFIX_SOURCES) $(DEVICEHUBFIX_DYNAMIC_LOOKUP_SYMS) \
- -framework CoreFoundation -lMobileGestalt \
+ -framework CoreFoundation -framework CoreVideo -framework IOSurface -lMobileGestalt -lobjc \
-Wl,-not_for_dyld_shared_cache
@codesign --force --sign - $@
diff --git a/VPhoneGuestComponents/Shared/InjectionEnvironment.h b/VPhoneGuestComponents/Shared/InjectionEnvironment.h
index fd1e8150..1c27a7b3 100644
--- a/VPhoneGuestComponents/Shared/InjectionEnvironment.h
+++ b/VPhoneGuestComponents/Shared/InjectionEnvironment.h
@@ -94,13 +94,15 @@ static const char *vpBatteryHealthFixFor(const char *path) {
#define VP_DEVICEHUB_FIX "/usr/lib/libdevicehubfix.dylib"
#endif
-// cryptexd, which stages and grafts the developer disk image, and the DDI's
-// display server. The DDI is mounted under /System/Developer, so the
-// suffixes are matched like the other targets.
+// cryptexd, which stages and grafts the developer disk image, the DDI's
+// display server, and avconferenced, which captures and encodes the stream.
+// The DDI is mounted under /System/Developer, so the suffixes are matched like
+// the other targets.
static int vpIsDeviceHubFixTarget(const char *path) {
return path && (vpPathHasSuffix(path, "/usr/libexec/cryptexd") ||
vpPathHasSuffix(path, "/usr/libexec/dtremotedisplayd") ||
- vpPathHasSuffix(path, "/usr/libexec/dtdeviceinfod"));
+ vpPathHasSuffix(path, "/usr/libexec/dtdeviceinfod") ||
+ vpPathHasSuffix(path, "/usr/libexec/avconferenced"));
}
static const char *vpDeviceHubFixFor(const char *path) {
diff --git a/VPhoneGuestComponents/Tests/InjectionEnvironmentTests.c b/VPhoneGuestComponents/Tests/InjectionEnvironmentTests.c
index 9a9904bb..7f5dd404 100644
--- a/VPhoneGuestComponents/Tests/InjectionEnvironmentTests.c
+++ b/VPhoneGuestComponents/Tests/InjectionEnvironmentTests.c
@@ -80,8 +80,10 @@ static void namesTheDeviceHubFixTarget(void) {
assert(vpIsDeviceHubFixTarget("/usr/libexec/dtremotedisplayd"));
assert(vpIsDeviceHubFixTarget("/usr/libexec/cryptexd"));
assert(vpIsDeviceHubFixTarget("/System/Developer/usr/libexec/dtdeviceinfod"));
+ assert(vpIsDeviceHubFixTarget("/usr/libexec/avconferenced"));
assert(!vpIsDeviceHubFixTarget("/usr/libexec/dtdeviceinfodx"));
assert(!vpIsDeviceHubFixTarget("/usr/libexec/cryptexdx"));
+ assert(!vpIsDeviceHubFixTarget("/usr/libexec/avconferencedx"));
assert(!vpIsDeviceHubFixTarget("/System/Developer/usr/libexec/dtremotedisplaydx"));
assert(!vpIsDeviceHubFixTarget("/System/Developer/usr/libexec/remoted"));
assert(!vpIsDeviceHubFixTarget(NULL));