Run a virtual iPhone on an Apple Silicon Mac.
Swift
Carry the keys the virtual keyboard drops: 🌐, JIS input keys, Esc on iPad
Jacky8ef23f43bcaf
diff --git a/Research/Guest/keyboard_event_pipeline.md b/Research/Guest/keyboard_event_pipeline.mdindex f93e5544..0149777f 100644--- a/Research/Guest/keyboard_event_pipeline.md+++ b/Research/Guest/keyboard_event_pipeline.md@@ -249,3 +249,61 @@ Dynamic(vm)._processHIDReports(UnsafeRawPointer(vecPtr), forDevice: deviceId, de| `-[VZVirtualMachine(_VZHIDAdditions) _processHIDReports:forDevice:deviceType:]` | 0x2301b2310 || `-[VZVirtualMachineView _sendKeyEventsToVirtualMachine:]` | -- || `-[_VZHIDEventMonitor getHIDReportsFromHIDEvent:]` | 0x2301b2af0 |++---++## What Never Reaches the Guest (2026-10-02)++Measured on macOS 27.0.1 (Virtualization 259), against the two lookup tables+`-[_VZKeyboard sendKeyEvents:]` reads (179 entries each, at `adrp`+`add` in+the method's prologue; read from the shared cache, whose slide is one per+boot) and the report descriptor of `VZUSBKeyboardConfiguration`, found in+`com.apple.Virtualization.VirtualMachine` by its `05 01 09 06 A1 01` prefix:++```+05 01 09 06 A1 01 Generic Desktop / Keyboard+05 07 19 E0 29 E7 ... 81 02 8 modifier bits, page 7 0xE0-0xE7+... 19 00 29 91 ... 95 06 6 keys, page 7 0x00-0x91+05 08 19 01 29 03 ... 3 LEDs+C0+```++A boot-protocol keyboard. Of the 179 virtual key codes, 115 have an index;+the rest are dropped before any report is built:++| Mac key (`kVK_*`) | Code | Why it is lost | What vphone sends instead (vphoned) |+| --- | --- | --- | --- |+| fn / 🌐 (`Function`) | 0x3F | has an index (0x55) but no report field | Apple vendor top-case 0xFF / 0x03, held while fn is |+| JIS 英数 (`JIS_Eisu`) | 0x66 | no index | page 7 0x91 (LANG2) |+| JIS かな (`JIS_Kana`) | 0x68 | no index | page 7 0x90 (LANG1) |+| JIS ¥ (`JIS_Yen`) | 0x5D | no index | page 7 0x89 (International3) |+| JIS _ (`JIS_Underscore`) | 0x5E | no index | page 7 0x87 (International1) |+| JIS keypad , (`JIS_KeypadComma`) | 0x5F | no index | page 7 0x85 (Keypad Comma) |+| Context Menu | 0x6E | no index | page 7 0x65 (Application) |+| Help / Insert | 0x72 | no index | page 7 0x49 (Insert) |+| Volume Up / Down / Mute | 0x48 / 0x49 / 0x4A | no index | consumer 0xE9 / 0xEA / 0xE2 |++`VPhoneApplication.sendEvent(_:)` sends these through vphoned's `input.hid`+(`VPhoneGuestKeyMap`), since nothing the virtual keyboard emits can carry+them. Everything else — letters, digits, punctuation, Return, Tab, Space,+Delete, Forward Delete, Caps Lock, all eight modifiers, F1–F20, arrows,+Home / End / Page Up / Page Down, the keypad and ISO § — goes through the+virtual keyboard unchanged.++On an iPadOS 26.6.2 guest with English and Simplified Pinyin enabled,+measured by injecting each usage through vphoned in Notes:++- page 7 `E0` + `2C` (⌃Space) switches the input source;+- vendor `FF` / `03` (🌐) switches the input source;+- page 7 `39` (Caps Lock) types capitals in Pinyin rather than switching —+ iPadOS's default for Chinese input.++Esc: `VPhoneApplication` turns Esc into the back gesture on an iPhone guest,+which has no Esc key. On an iPad guest Esc goes through as Esc, where it+cancels a composition, a menu or a sheet.++Host shortcuts still apply before the guest sees anything: with+`AppleFnUsageType = 1` (System Settings › Keyboard › "Press 🌐 key to: Change+Input Source") the Mac switches its own input source on the same fn press,+and a ⌃Space or ⌘Space bound to Spotlight or input-source switching on the+host is the host's unless the VM view's `capturesSystemKeys` takes it first.diff --git a/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings b/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstringsindex 1520245b..a109ed18 100644--- a/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings+++ b/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings@@ -17188,6 +17188,40 @@}}},+ "Switch Guest Input Source" : {+ "localizations" : {+ "en" : {+ "stringUnit" : {+ "state" : "translated",+ "value" : "Switch Guest Input Source"+ }+ },+ "ja" : {+ "stringUnit" : {+ "state" : "translated",+ "value" : "ゲストの入力ソースを切り替える"+ }+ },+ "ko" : {+ "stringUnit" : {+ "state" : "translated",+ "value" : "게스트 입력 소스 전환"+ }+ },+ "vi" : {+ "stringUnit" : {+ "state" : "translated",+ "value" : "Chuyển nguồn nhập trên máy khách"+ }+ },+ "zh-Hans" : {+ "stringUnit" : {+ "state" : "translated",+ "value" : "切换客体输入法"+ }+ }+ }+ },"The guest removes them from its keychain. This cannot be undone." : {"localizations" : {"en" : {diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swiftindex 0bce0617..f6f920df 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift@@ -62,6 +62,7 @@ extension VPhoneMenuController {}menu.addItem(NSMenuItem.separator())menu.addItem(makeItem("Open Guest Spotlight", action: #selector(sendSpotlight), symbol: "magnifyingglass"))+ menu.addItem(makeItem("Switch Guest Input Source", action: #selector(sendGlobe), symbol: "globe"))// Trackpad scroll and pinch arrive as ordinary NSEvents; the view turns// them into guest touches. Off hands both back to AppKit untouched.let trackpadItem = makeItem(@@ -121,6 +122,10 @@ extension VPhoneMenuController {keySender.sendSpotlight()}+ @objc func sendGlobe() {+ keySender.sendGlobe()+ }+// MARK: - Rotate/// The four interface orientations, checked by the one the window lastdiff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swiftindex 4b15039c..36928c2f 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift@@ -1,21 +1,33 @@import AppKit+import VPhoneCoreKit// MARK: - Application/// Offers each key press in the VM window to the main menu before the VM view-/// can take it, and takes Esc for the guest's back gesture.+/// can take it, takes Esc for the guest's back gesture, and carries the keys the+/// virtual keyboard cannot.////// `capturesSystemKeys` makes `VZVirtualMachineView` install a local event/// monitor that sends every key to the guest and swallows it. AppKit runs/// local monitors in the order they were added, and the view adds its own/// again each time it becomes first responder or its window becomes key, so/// a monitor of ours always runs after it. `sendEvent(_:)` runs before any-/// local monitor, so both the menu lookup and the Esc handling live here.+/// local monitor, so the menu lookup, the Esc handling and the dropped keys+/// all live here.final class VPhoneApplication: NSApplication {/// Keys whose key-down a menu item took. Their key-up is dropped too, so/// the guest never sees a release without a press.private var menuKeyCodes = Set<UInt16>()+ /// Keys whose key-down went to the guest through vphoned, so their key-up+ /// goes the same way.+ private var forwardedKeyCodes = Set<UInt16>()++ /// The guest 🌐 is held through, while it is held. fn's release only arrives+ /// while the window is key, so leaving the window lets go of it here.+ private weak var globeHolder: VPhoneGuestControl?+ private var resignObserver: NSObjectProtocol?+/// `kVK_Escape`. Intercepted here instead of as a menu key equivalent:/// AppKit's matching for a modifier-less Esc is not dependable, and a missed/// match forwards the key to the guest as a plain Escape — which is what@@ -23,23 +35,31 @@ final class VPhoneApplication: NSApplication {private static let escapeKeyCode: UInt16 = 53override func sendEvent(_ event: NSEvent) {- guard event.type == .keyDown || event.type == .keyUp,- event.window?.firstResponder is VPhoneVirtualMachineView+ guard event.type == .keyDown || event.type == .keyUp || event.type == .flagsChanged,+ let view = event.window?.firstResponder as? VPhoneVirtualMachineViewelse {super.sendEvent(event)return}+ if event.type == .flagsChanged {+ forwardGlobe(event, view: view)+ super.sendEvent(event)+ return+ }// Esc replays the guest's back gesture: iOS has no back key, and a// forwarded Escape only reads as cancel. The press and its release both- // stop here, so the guest sees neither.- if event.keyCode == Self.escapeKeyCode {- if event.type == .keyDown,- let view = event.window?.firstResponder as? VPhoneVirtualMachineView- {+ // stop here, so the guest sees neither. An iPad has a real Esc key —+ // it cancels a composition, a sheet or a menu — so there it goes+ // through like any other key.+ if event.keyCode == Self.escapeKeyCode, view.escapeIsBackGesture {+ if event.type == .keyDown {view.performBackGesture()}return}+ if forwardDroppedKey(event, view: view) {+ return+ }if event.type == .keyUp {if menuKeyCodes.remove(event.keyCode) == nil {super.sendEvent(event)@@ -52,4 +72,58 @@ final class VPhoneApplication: NSApplication {}super.sendEvent(event)}++ // MARK: - Keys the Virtual Keyboard Drops++ /// fn / 🌐 reaches AppKit only as a flags change, and the virtual USB+ /// keyboard has no field for it. vphoned presses the guest's 🌐 for it, held+ /// for as long as fn is, so 🌐 alone switches the input source and 🌐+ /// shortcuts work. The event still goes on to the view, which keeps the+ /// virtual keyboard's own modifier state in step.+ private func forwardGlobe(_ event: NSEvent, view: VPhoneVirtualMachineView) {+ guard event.keyCode == VPhoneGuestKeyMap.functionKeyCode, let control = view.control else { return }+ let globe = VPhoneGuestKeyMap.globe+ if event.modifierFlags.contains(.function) {+ guard globeHolder == nil else { return }+ globeHolder = control+ control.sendHIDDown(page: globe.page, usage: globe.usage)+ resignObserver = NotificationCenter.default.addObserver(+ forName: NSWindow.didResignKeyNotification,+ object: event.window,+ queue: .main,+ ) { [weak self] _ in+ MainActor.assumeIsolated { self?.releaseGlobe() }+ }+ } else {+ releaseGlobe()+ }+ }++ private func releaseGlobe() {+ if let resignObserver {+ NotificationCenter.default.removeObserver(resignObserver)+ }+ resignObserver = nil+ guard let holder = globeHolder else { return }+ globeHolder = nil+ let globe = VPhoneGuestKeyMap.globe+ holder.sendHIDUp(page: globe.page, usage: globe.usage)+ }++ /// Sends a key `_VZKeyboard` would drop — the JIS input-mode and character+ /// keys, Context Menu, Insert, volume — through vphoned. Returns whether the+ /// event was one of them; it then goes no further.+ private func forwardDroppedKey(_ event: NSEvent, view: VPhoneVirtualMachineView) -> Bool {+ guard let usage = VPhoneGuestKeyMap.usage(forDroppedKeyCode: event.keyCode),+ let control = view.control+ else { return false }+ if event.type == .keyDown {+ // The guest repeats a held key itself.+ guard !event.isARepeat, forwardedKeyCodes.insert(event.keyCode).inserted else { return true }+ control.sendHIDDown(page: usage.page, usage: usage.usage)+ } else if forwardedKeyCodes.remove(event.keyCode) != nil {+ control.sendHIDUp(page: usage.page, usage: usage.usage)+ }+ return true+ }}diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swiftindex b9d64211..41f3ae21 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift@@ -2,6 +2,7 @@ import AppKitimport Dynamicimport Foundationimport Virtualization+import VPhoneCoreKit// MARK: - Key Helper@@ -69,6 +70,14 @@ class VPhoneVirtualMachineKeySender {control.sendHIDUp(page: 0x07, usage: 0xE3) // Cmd up}+ /// A 🌐 tap: the guest switches to its next input source. The Mac's fn key+ /// does the same while the VM window is key; see `VPhoneApplication`.+ func sendGlobe() {+ guard requireConnection() else { return }+ let globe = VPhoneGuestKeyMap.globe+ control.sendHIDPress(page: globe.page, usage: globe.usage)+ }+// MARK: - Type ASCII from Clipboardfunc typeFromClipboard() {diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swiftindex d157dfe7..7d887e02 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift@@ -31,6 +31,7 @@ extension VPhoneBootCommand {screenPPI: manifest.screenConfig.pixelsPerInch,screenScale: manifest.screenConfig.scale,kernelDebugPort: kernelDebugPort,+ isPadGuest: manifest.guestDevice.isPad,)}}diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swiftindex 880fff8c..7810a4c9 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift@@ -128,6 +128,7 @@ class VPhoneVirtualMachineAppDelegate: NSObject, NSApplicationDelegate {.resolvingSymlinksInPath().path,)+ wc.captureView?.escapeIsBackGesture = !options.isPadGuestwindowController = wclet fileWC = VPhoneFileWindowController()diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swiftindex 9923b4e7..0d4b59b2 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift@@ -31,6 +31,8 @@ class VPhoneVirtualMachine: NSObject, VZVirtualMachineDelegate {var screenPPI: Int = 460var screenScale: Double = 3.0var kernelDebugPort: Int?+ /// The guest runs iPadOS: Esc is a key there, not the back gesture.+ var isPadGuest = false}private struct DeviceIdentity {diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swiftindex f107af91..754adf58 100644--- a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift@@ -7,6 +7,9 @@ import VPhoneCoreKitclass VPhoneVirtualMachineView: VZVirtualMachineView {var keySender: VPhoneVirtualMachineKeySender?weak var control: VPhoneGuestControl?+ /// Whether Esc replays the back gesture (an iPhone, which has no Esc key)+ /// or reaches the guest as Esc (an iPad). See `VPhoneApplication`.+ var escapeIsBackGesture = true/// Whether trackpad scroll/pinch gestures are replayed as guest touches.var trackpadGesturesEnabled = VPhoneTrackpadGestures.isEnabled {diff --git a/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swift b/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swiftnew file mode 100644index 00000000..6efcb312--- /dev/null+++ b/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swift@@ -0,0 +1,56 @@+// MARK: - Guest Key Map++/// Mac keys the virtual USB keyboard cannot carry, and the HID usage the guest+/// gets for each instead, by way of vphoned.+///+/// `VZUSBKeyboardConfiguration` is a boot-protocol keyboard: its report holds+/// the eight page-7 modifiers (0xE0–0xE7) and six page-7 keys in 0x00–0x91,+/// nothing else. `_VZKeyboard` first maps each Mac virtual key code to an+/// internal index through a 179-entry table, and drops codes the table has no+/// index for (macOS 27.0.1, Virtualization 259). Measured against that table+/// and the report descriptor, the keys that never reach a guest are:+///+/// - **fn / 🌐** (`kVK_Function`, 0x3F): has an index, but no report field. On+/// an iPad, 🌐 switches the input source and starts the 🌐 shortcuts; Apple's+/// keyboards send it as Apple vendor top-case page 0xFF, usage 0x03, which+/// iPadOS honours from any keyboard service.+/// - the JIS keys 英数 / かな / ¥ / _ / keypad `,`, which switch the Japanese+/// input mode and type their characters;+/// - Context Menu, Help/Insert, and the volume and mute keys.+public enum VPhoneGuestKeyMap {+ public struct Usage: Hashable, Sendable {+ public let page: UInt32+ public let usage: UInt32++ public init(page: UInt32, usage: UInt32) {+ self.page = page+ self.usage = usage+ }+ }++ /// `kVK_Function`, which AppKit reports in a flags-changed event.+ public static let functionKeyCode: UInt16 = 0x3F++ /// The 🌐 key of an Apple keyboard: vendor top-case page, keyboard fn.+ public static let globe = Usage(page: 0xFF, usage: 0x03)++ /// Key-down / key-up codes `_VZKeyboard` drops, with the usage to send.+ public static let droppedKeys: [UInt16: Usage] = [+ 0x66: Usage(page: 0x07, usage: 0x91), // JIS 英数 → LANG2+ 0x68: Usage(page: 0x07, usage: 0x90), // JIS かな → LANG1+ 0x5D: Usage(page: 0x07, usage: 0x89), // JIS ¥ → International3+ 0x5E: Usage(page: 0x07, usage: 0x87), // JIS _ → International1+ 0x5F: Usage(page: 0x07, usage: 0x85), // JIS keypad , → Keypad Comma+ 0x6E: Usage(page: 0x07, usage: 0x65), // Context Menu → Application+ 0x72: Usage(page: 0x07, usage: 0x49), // Help / Insert → Insert+ 0x48: Usage(page: 0x0C, usage: 0xE9), // Volume Up+ 0x49: Usage(page: 0x0C, usage: 0xEA), // Volume Down+ 0x4A: Usage(page: 0x0C, usage: 0xE2), // Mute+ ]++ /// The usage a key-down or key-up with `keyCode` needs sent by hand, or nil+ /// for a key the virtual keyboard carries itself.+ public static func usage(forDroppedKeyCode keyCode: UInt16) -> Usage? {+ droppedKeys[keyCode]+ }+}diff --git a/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swift b/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swiftnew file mode 100644index 00000000..3f9e3e87--- /dev/null+++ b/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swift@@ -0,0 +1,33 @@+import Testing+@testable import VPhoneCoreKit++/// The keys the virtual USB keyboard drops, and what the guest gets instead.+@Suite("Guest key map")+struct GuestKeyMapTests {+ @Test+ func `fn becomes the Apple keyboard 🌐 usage`() {+ #expect(VPhoneGuestKeyMap.functionKeyCode == 0x3F)+ #expect(VPhoneGuestKeyMap.globe == .init(page: 0xFF, usage: 0x03))+ }++ @Test+ func `the JIS input-mode keys become LANG1 and LANG2`() {+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: 0x68) == .init(page: 0x07, usage: 0x90))+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: 0x66) == .init(page: 0x07, usage: 0x91))+ }++ @Test+ func `keys the virtual keyboard carries are left to it`() {+ // A, Space, Escape, Caps Lock, Left Control, F1, Up Arrow, kVK_Function.+ for keyCode: UInt16 in [0x00, 0x31, 0x35, 0x39, 0x3B, 0x7A, 0x7E, 0x3F] {+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: keyCode) == nil)+ }+ }++ @Test+ func `every forwarded usage is on a page the guest accepts from vphoned`() {+ for usage in VPhoneGuestKeyMap.droppedKeys.values {+ #expect([0x07, 0x0C].contains(usage.page))+ }+ }+}