Run a virtual iPhone on an Apple Silicon Mac.
Swift
Duo: wake the panel from black, not from its last frame
Jackybde6440c410b
diff --git a/Research/Guest/iphone_duo_bringup.md b/Research/Guest/iphone_duo_bringup.mdindex 2a0d2424..5f274c2b 100644--- a/Research/Guest/iphone_duo_bringup.md+++ b/Research/Guest/iphone_duo_bringup.md@@ -5090,3 +5090,18 @@ DeviceHub's slider as it animates, and taking every read reshaped the windowat each step of a sweep. The cover drawn in the window's open shape,sideways, now lasts a fraction of the half second it did. A fold while the screen sleeps lights it, and thepanel's first 0.2 s are its last frame from before sleep.++The fade to black stays for a fold that starts with the panel off or lit+less than a second ago. A fold made while the screen sleeps (DeviceHub, a+guest-side request) runs only as the screen comes on, since SpringBoard's+pose provider commits on display frames; without the fade, the effect's+carried-over inner content (its Home Screen, sideways on the cover) showed+for about half a second after the wake. And coming on, the panel first shows+its last frame from before it went off (the unlocked Home Screen on a plain+wake), which nothing in the guest replaces: SpringBoard stops compositing+before it turns the panel off, and neither `-[CAWindowServerDisplay+presentSurface:withOptions:]` nor `acquireFrozenSurface` reached the shown+surface (`willUnblank` never runs: the panel goes off by display state). So+`display.orientation` reports `screen_off`, and the VM window covers the+panel with black while it is off and for 0.3 s after it comes on. Both wakes+now go from black to the lock screen (60 fps capture).diff --git a/Research/vphoned_http_api.md b/Research/vphoned_http_api.mdindex 7483bcfd..7a027121 100644--- a/Research/vphoned_http_api.md+++ b/Research/vphoned_http_api.md@@ -497,7 +497,7 @@ request carries `"force": true`.| Area | Methods || --- | --- || Device | `device.snapshot`, `device.info` (snapshot plus network, screen, rotation, brightness, volume, low power, Developer Mode, agent, `device_name` `{name, own_name}`: the pinned name and the guest's own ComputerName), `device.screen`, `device.network`, `device.displays` (a flat IOService probe of every display-ish object with its small properties — what the display stack bound; built for the foldable display work: `AppleParavirtGPU` `DisplayPortCount`, and each `AppleParavirtDisplay` width/height/`primary`), `device.ioreg {plane}`, `device.environment`, `device.basebin {archive?}` |-| Display, audio | `display.pose` (read-only SpringBoard pose snapshot: orientation, device_orientation, hinge, tabletop; reports unavailable for an unsupported ABI), `display.modes` (read-only CoreAnimation display bounds, native size, current/available modes, and virtual-mode selector availability; availability does not prove a mode change is supported), `display.brightness {value?}`, `display.rotation {orientation?}`, `display.posture {hinge, orientation?, tabletop?}` (a foldable guest's pose through SpringBoard's test-automation override, see below; capability `device_pose`), `display.orientation` (`{degrees, source, fold_angle?}`: SpringBoard's interface orientation without a screen capture, for polling; `degrees` is null while SpringBoard has no answer; on a foldable `fold_angle` is the virtual hinge's settled angle, which every fold leaves it at, so a host can follow folds it did not ask for; capability `display_orientation`), `display.rotation_lock {locked}`, `display.auto_lock` (`{auto_lock_seconds, never, lock_screen_minimum_seconds}`), `display.auto_lock.set {seconds}` (writes the mobile UserSettings input through cfprefsd, posts profiled's effective-settings notification, and returns an immediate readback; the caller must verify the effective value), `screen.unlock {passcode?, timeout?}` (`{locked, screen_off, was_locked, was_screen_off}`: the display on and the Lock Screen passed, see below; capability `screen_unlock`), `audio.volume {value?, category?}`, `audio.state` |+| Display, audio | `display.pose` (read-only SpringBoard pose snapshot: orientation, device_orientation, hinge, tabletop; reports unavailable for an unsupported ABI), `display.modes` (read-only CoreAnimation display bounds, native size, current/available modes, and virtual-mode selector availability; availability does not prove a mode change is supported), `display.brightness {value?}`, `display.rotation {orientation?}`, `display.posture {hinge, orientation?, tabletop?}` (a foldable guest's pose through SpringBoard's test-automation override, see below; capability `device_pose`), `display.orientation` (`{degrees, source, fold_angle?, screen_off?}`: SpringBoard's interface orientation without a screen capture, for polling; `degrees` is null while SpringBoard has no answer; on a foldable `fold_angle` is the virtual hinge's angle, so a host can follow folds it did not ask for, and `screen_off` whether SpringBoard has the screen off, so a host can keep the panel's frame from before off its window as it comes on; capability `display_orientation`), `display.rotation_lock {locked}`, `display.auto_lock` (`{auto_lock_seconds, never, lock_screen_minimum_seconds}`), `display.auto_lock.set {seconds}` (writes the mobile UserSettings input through cfprefsd, posts profiled's effective-settings notification, and returns an immediate readback; the caller must verify the effective value), `screen.unlock {passcode?, timeout?}` (`{locked, screen_off, was_locked, was_screen_off}`: the display on and the Lock Screen passed, see below; capability `screen_unlock`), `audio.volume {value?, category?}`, `audio.state` || Input | `input.touch`, `input.hid {page, usage, down?, long_press?}`, `input.button {name}`, `input.key {name}`, `input.type {text, delay_ms?}`, `input.paste {text}`, `input.tap`, `input.double_tap`, `input.long_press`, `input.swipe`, `input.drag {points}`, `input.touch_sequence {events}` — gesture coordinates are screen points; `input.pointer {x, y, seq?, buttons?}`, `input.scroll {phase, dx?, dy?, x?, y?}` (capabilities `pointer_scroll` and `pointer_buttons`, see below) || UI | `ui.tree` (alias `accessibility.tree`), `ui.element_at`, `ui.tap_element`, `ui.wait`, `ui.wait_gone`, `ui.ocr {languages?, min_confidence?}`, `ui.describe`, `screen.screenshot` || Processes | `processes.list {filter?}`, `processes.kill {pid, signal?}` **force**, `process.spawn {executable, arguments?, timeout_seconds?}`, `memory.jetsam`, `memory.pressure` (only the three kernel memory sysctls, for polling) |diff --git a/VPhoneDaemon/Daemon/GuestAPI+Device.swift b/VPhoneDaemon/Daemon/GuestAPI+Device.swiftindex 2bbbc438..883f3598 100644--- a/VPhoneDaemon/Daemon/GuestAPI+Device.swift+++ b/VPhoneDaemon/Daemon/GuestAPI+Device.swift@@ -176,7 +176,12 @@ extension GuestAPI {"degrees": interfaceRotationDegrees().map { $0 as Any } ?? NSNull(),"source": "springboard",]- if let angle = settledFoldAngle() { reply["fold_angle"] = angle }+ if let angle = settledFoldAngle() {+ reply["fold_angle"] = angle+ // The window covers the panel while it is off: coming on, it+ // shows its last frame from before until the guest draws again.+ reply["screen_off"] = GuestScreenUnlock.screenIsOff+ }return reply}diff --git a/VPhoneDaemon/Daemon/GuestScreenUnlock.swift b/VPhoneDaemon/Daemon/GuestScreenUnlock.swiftindex c7acaab3..db8d9231 100644--- a/VPhoneDaemon/Daemon/GuestScreenUnlock.swift+++ b/VPhoneDaemon/Daemon/GuestScreenUnlock.swift@@ -173,6 +173,9 @@ enum GuestScreenUnlock {// MARK: - State+ /// Whether SpringBoard has the screen off.+ static var screenIsOff: Bool { state().screenOff }+private struct State {let locked: Boollet screenOff: Booldiff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineWindowController.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineWindowController.swiftindex e8a83206..06b1738b 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineWindowController.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineWindowController.swift@@ -287,6 +287,28 @@ class VPhoneVirtualMachineWindowController: NSObject, NSWindowDelegate {private var minimumDuoPanelSize: CGSize?private var orientationPollInFlight = falseprivate var lastPolledFoldAngle: Double?+ private var screenRevealWork: DispatchWorkItem?++ /// Covers a foldable guest's panel while its screen is off, and for a+ /// moment after it comes on: its display first shows the frame it showed+ /// before going off (after a fold made asleep, the other panel's picture,+ /// sideways), until the guest draws again.+ private func followGuestScreen(off: Bool) {+ guard let container = displayContainer else { return }+ if off {+ screenRevealWork?.cancel()+ screenRevealWork = nil+ container.isScreenDark = true+ return+ }+ guard container.isScreenDark, screenRevealWork == nil else { return }+ let work = DispatchWorkItem { [weak self] in+ self?.displayContainer?.isScreenDark = false+ self?.screenRevealWork = nil+ }+ screenRevealWork = work+ DispatchQueue.main.asyncAfter(deadline: .now() + 0.3, execute: work)+ }/// Asks vphoned for the interface orientation once a second (a foldable/// guest four times a second). Guests without `display.orientation` stay@@ -308,6 +330,9 @@ class VPhoneVirtualMachineWindowController: NSObject, NSWindowDelegate {if foldAngle == lastPolledFoldAngle { control.followGuestFold(angle: foldAngle) }lastPolledFoldAngle = foldAngle}+ if let screenOff = result["screen_off"] as? Bool {+ followGuestScreen(off: screenOff)+ }guard !control.isChangingOrientation,let degrees = (result["degrees"] as? NSNumber)?.intValue,let orientation = VPhoneDisplayOrientation(degrees: degrees)diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneDisplayContainerView.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneDisplayContainerView.swiftindex 06cb5aa9..fa6d2418 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneDisplayContainerView.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneDisplayContainerView.swift@@ -28,6 +28,23 @@ final class VPhoneDisplayContainerView: NSView {private static let turnDuration: CFTimeInterval = 0.35+ /// Black over the panel, above the VM view's layer rather than in its+ /// place, so clicks still reach the VM view (a tap wakes the guest).+ private let darkLayer = CALayer()++ /// Covers the panel while the guest's screen is off: coming on, the+ /// guest's display shows its last frame from before until it draws again,+ /// which after a fold made asleep was the other panel's picture.+ var isScreenDark = false {+ didSet {+ guard isScreenDark != oldValue else { return }+ CATransaction.begin()+ CATransaction.setDisableActions(true)+ darkLayer.opacity = isScreenDark ? 1 : 0+ CATransaction.commit()+ }+ }+/// One animated turn: the panel's angle and the window frame move/// together, so the window reshapes as the panel turns.private struct Turn {@@ -48,6 +65,10 @@ final class VPhoneDisplayContainerView: NSView {viewportView.layer?.masksToBounds = trueaddSubview(viewportView)viewportView.addSubview(displayView)+ darkLayer.backgroundColor = NSColor.black.cgColor+ darkLayer.opacity = 0+ darkLayer.zPosition = 1000+ viewportView.layer?.addSublayer(darkLayer)installFoldChrome()}@@ -249,6 +270,10 @@ let backing = backingPanelSize.width > 0 ? backingPanelSize : panelSizelet full = NSSize(width: backing.width * scale, height: backing.height * scale)displayView.frameCenterRotation = 0displayView.frame = NSRect(x: 0, y: size.height - full.height, width: full.width, height: full.height)+ CATransaction.begin()+ CATransaction.setDisableActions(true)+ darkLayer.frame = viewportView.bounds+ CATransaction.commit()viewportView.frameCenterRotation = angleif let vmView = displayView as? VPhoneVirtualMachineView {vmView.enclosingDisplayRotation = anglediff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.mindex 2878952e..1518653d 100644--- a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m@@ -39,8 +39,12 @@ static unsigned int vpStateControlDisplayId(id control) {return *(unsigned int *)((uint8_t *)(__bridge void *)control + ivar_getOffset(ivar));}+// When the panel last came on (system uptime), for vpFollowContentModeTransition.+static NSTimeInterval vpPanelOnAt;+static void vpApplyPanelState(id control, long long state, long long *applied, id completion) {if (!control || (state == *applied && !completion)) return;+ if (control == vpPrimaryStateControl && state && !*applied) vpPanelOnAt = NSProcessInfo.processInfo.systemUptime;*applied = state;((void (*)(id, SEL, long long, id))vpOriginalTransitionDisplayState)(control,sel_registerName("transitionToDisplayState:withCompletion:"), state, completion);@@ -122,7 +126,22 @@ static void vpFollowPanelOwner(void) {dispatch_async(dispatch_get_main_queue(), ^{ vpApplyPanelOwner(); });}+// SpringBoard's fold effect fades the panel being folded away to black and+// the arriving one in from black: two screens on hardware, about 0.12 s of+// black on this one panel each time it opens. Without the fade the cover dims+// and blurs and the inner display takes over at the swap. A fold made while+// the screen is dark runs only as it comes on (the pose provider commits on+// display frames), and there the fade hides what the effect carries over from+// the inner display (its Home Screen, sideways on the cover), so it stays for+// a fold that starts with the panel off or just lit.+static void vpSetFoldFadeToBlack(BOOL enabled) {+ id settings = ((id (*)(id, SEL))objc_msgSend)(objc_getClass("SBDeviceContinuousFoldInteractionDomain"), sel_registerName("rootSettings"));+ SEL setFade = sel_registerName("setFadeToBlackEnabled:");+ if ([settings respondsToSelector:setFade]) ((void (*)(id, SEL, BOOL))objc_msgSend)(settings, setFade, enabled);+}+static void vpFollowContentModeTransition(id self, SEL selector, id transitionCoordinator) {+ vpSetFoldFadeToBlack(!vpPrimaryAppliedState || NSProcessInfo.processInfo.systemUptime - vpPanelOnAt < 1);((void (*)(id, SEL, id))vpOriginalPerformContentModeTransition)(self, selector, transitionCoordinator);vpFollowPanelOwner();}@@ -164,14 +183,7 @@ void vpInstallSharedPanel(void) {vpOriginalTransitionDisplayState = method_setImplementation(transition, (IMP)vpSharedScanoutTransition);vpInstallContentModeOwnership();vpFollowInnerPages();- // SpringBoard's fold effect fades the panel being folded away to black and- // the arriving one in from black: two screens on hardware, about 0.12 s of- // black on this one panel each time it opens. Without the fade the cover- // dims and blurs and the inner display takes over at the swap.- id settings =((id (*)(id, SEL))objc_msgSend)(objc_getClass("SBDeviceContinuousFoldInteractionDomain"), sel_registerName("rootSettings"));- SEL setFade = sel_registerName("setFadeToBlackEnabled:");- if ([settings respondsToSelector:setFade]) ((void (*)(id, SEL, BOOL))objc_msgSend)(settings, setFade, NO);- vpFoldDisplayLog([NSString stringWithFormat:@"inner display: fold fade to black off=%d", [settings respondsToSelector:setFade]]);+ vpSetFoldFadeToBlack(NO);vpFoldDisplayLog(@"inner display: shared scanout installed");}