Skip to content

Run a virtual iPhone on an Apple Silicon Mac.

Swift

Carry the keys the virtual keyboard drops: 🌐, JIS input keys, Esc on iPad

Jacky8ef23f43bcaf
diff --git a/Research/Guest/keyboard_event_pipeline.md b/Research/Guest/keyboard_event_pipeline.md
index f93e5544..0149777f 100644
--- a/Research/Guest/keyboard_event_pipeline.md
+++ b/Research/Guest/keyboard_event_pipeline.md
@@ -249,3 +249,61 @@ Dynamic(vm)._processHIDReports(UnsafeRawPointer(vecPtr), forDevice: deviceId, de
| `-[VZVirtualMachine(_VZHIDAdditions) _processHIDReports:forDevice:deviceType:]` | 0x2301b2310 |
| `-[VZVirtualMachineView _sendKeyEventsToVirtualMachine:]` | -- |
| `-[_VZHIDEventMonitor getHIDReportsFromHIDEvent:]` | 0x2301b2af0 |
+
+---
+
+## What Never Reaches the Guest (2026-10-02)
+
+Measured on macOS 27.0.1 (Virtualization 259), against the two lookup tables
+`-[_VZKeyboard sendKeyEvents:]` reads (179 entries each, at `adrp`+`add` in
+the method's prologue; read from the shared cache, whose slide is one per
+boot) and the report descriptor of `VZUSBKeyboardConfiguration`, found in
+`com.apple.Virtualization.VirtualMachine` by its `05 01 09 06 A1 01` prefix:
+
+```
+05 01 09 06 A1 01 Generic Desktop / Keyboard
+05 07 19 E0 29 E7 ... 81 02 8 modifier bits, page 7 0xE0-0xE7
+... 19 00 29 91 ... 95 06 6 keys, page 7 0x00-0x91
+05 08 19 01 29 03 ... 3 LEDs
+C0
+```
+
+A boot-protocol keyboard. Of the 179 virtual key codes, 115 have an index;
+the rest are dropped before any report is built:
+
+| Mac key (`kVK_*`) | Code | Why it is lost | What vphone sends instead (vphoned) |
+| --- | --- | --- | --- |
+| fn / 🌐 (`Function`) | 0x3F | has an index (0x55) but no report field | Apple vendor top-case 0xFF / 0x03, held while fn is |
+| JIS 英数 (`JIS_Eisu`) | 0x66 | no index | page 7 0x91 (LANG2) |
+| JIS かな (`JIS_Kana`) | 0x68 | no index | page 7 0x90 (LANG1) |
+| JIS ¥ (`JIS_Yen`) | 0x5D | no index | page 7 0x89 (International3) |
+| JIS _ (`JIS_Underscore`) | 0x5E | no index | page 7 0x87 (International1) |
+| JIS keypad , (`JIS_KeypadComma`) | 0x5F | no index | page 7 0x85 (Keypad Comma) |
+| Context Menu | 0x6E | no index | page 7 0x65 (Application) |
+| Help / Insert | 0x72 | no index | page 7 0x49 (Insert) |
+| Volume Up / Down / Mute | 0x48 / 0x49 / 0x4A | no index | consumer 0xE9 / 0xEA / 0xE2 |
+
+`VPhoneApplication.sendEvent(_:)` sends these through vphoned's `input.hid`
+(`VPhoneGuestKeyMap`), since nothing the virtual keyboard emits can carry
+them. Everything else — letters, digits, punctuation, Return, Tab, Space,
+Delete, Forward Delete, Caps Lock, all eight modifiers, F1–F20, arrows,
+Home / End / Page Up / Page Down, the keypad and ISO § — goes through the
+virtual keyboard unchanged.
+
+On an iPadOS 26.6.2 guest with English and Simplified Pinyin enabled,
+measured by injecting each usage through vphoned in Notes:
+
+- page 7 `E0` + `2C` (⌃Space) switches the input source;
+- vendor `FF` / `03` (🌐) switches the input source;
+- page 7 `39` (Caps Lock) types capitals in Pinyin rather than switching —
+ iPadOS's default for Chinese input.
+
+Esc: `VPhoneApplication` turns Esc into the back gesture on an iPhone guest,
+which has no Esc key. On an iPad guest Esc goes through as Esc, where it
+cancels a composition, a menu or a sheet.
+
+Host shortcuts still apply before the guest sees anything: with
+`AppleFnUsageType = 1` (System Settings › Keyboard › "Press 🌐 key to: Change
+Input Source") the Mac switches its own input source on the same fn press,
+and a ⌃Space or ⌘Space bound to Spotlight or input-source switching on the
+host is the host's unless the VM view's `capturesSystemKeys` takes it first.
diff --git a/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings b/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings
index 1520245b..a109ed18 100644
--- a/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings
+++ b/VPhoneExecutable/VPhoneVirtualization/Resources/Localizable.xcstrings
@@ -17188,6 +17188,40 @@
}
}
},
+ "Switch Guest Input Source" : {
+ "localizations" : {
+ "en" : {
+ "stringUnit" : {
+ "state" : "translated",
+ "value" : "Switch Guest Input Source"
+ }
+ },
+ "ja" : {
+ "stringUnit" : {
+ "state" : "translated",
+ "value" : "ゲストの入力ソースを切り替える"
+ }
+ },
+ "ko" : {
+ "stringUnit" : {
+ "state" : "translated",
+ "value" : "게스트 입력 소스 전환"
+ }
+ },
+ "vi" : {
+ "stringUnit" : {
+ "state" : "translated",
+ "value" : "Chuyển nguồn nhập trên máy khách"
+ }
+ },
+ "zh-Hans" : {
+ "stringUnit" : {
+ "state" : "translated",
+ "value" : "切换客体输入法"
+ }
+ }
+ }
+ },
"The guest removes them from its keychain. This cannot be undone." : {
"localizations" : {
"en" : {
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift
index 0bce0617..f6f920df 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/Menu/VPhoneMenuDevice.swift
@@ -62,6 +62,7 @@ extension VPhoneMenuController {
}
menu.addItem(NSMenuItem.separator())
menu.addItem(makeItem("Open Guest Spotlight", action: #selector(sendSpotlight), symbol: "magnifyingglass"))
+ menu.addItem(makeItem("Switch Guest Input Source", action: #selector(sendGlobe), symbol: "globe"))
// Trackpad scroll and pinch arrive as ordinary NSEvents; the view turns
// them into guest touches. Off hands both back to AppKit untouched.
let trackpadItem = makeItem(
@@ -121,6 +122,10 @@ extension VPhoneMenuController {
keySender.sendSpotlight()
}
+ @objc func sendGlobe() {
+ keySender.sendGlobe()
+ }
+
// MARK: - Rotate
/// The four interface orientations, checked by the one the window last
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift
index 4b15039c..36928c2f 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneApplication.swift
@@ -1,21 +1,33 @@
import AppKit
+import VPhoneCoreKit
// MARK: - Application
/// Offers each key press in the VM window to the main menu before the VM view
-/// can take it, and takes Esc for the guest's back gesture.
+/// can take it, takes Esc for the guest's back gesture, and carries the keys the
+/// virtual keyboard cannot.
///
/// `capturesSystemKeys` makes `VZVirtualMachineView` install a local event
/// monitor that sends every key to the guest and swallows it. AppKit runs
/// local monitors in the order they were added, and the view adds its own
/// again each time it becomes first responder or its window becomes key, so
/// a monitor of ours always runs after it. `sendEvent(_:)` runs before any
-/// local monitor, so both the menu lookup and the Esc handling live here.
+/// local monitor, so the menu lookup, the Esc handling and the dropped keys
+/// all live here.
final class VPhoneApplication: NSApplication {
/// Keys whose key-down a menu item took. Their key-up is dropped too, so
/// the guest never sees a release without a press.
private var menuKeyCodes = Set<UInt16>()
+ /// Keys whose key-down went to the guest through vphoned, so their key-up
+ /// goes the same way.
+ private var forwardedKeyCodes = Set<UInt16>()
+
+ /// The guest 🌐 is held through, while it is held. fn's release only arrives
+ /// while the window is key, so leaving the window lets go of it here.
+ private weak var globeHolder: VPhoneGuestControl?
+ private var resignObserver: NSObjectProtocol?
+
/// `kVK_Escape`. Intercepted here instead of as a menu key equivalent:
/// AppKit's matching for a modifier-less Esc is not dependable, and a missed
/// match forwards the key to the guest as a plain Escape — which is what
@@ -23,23 +35,31 @@ final class VPhoneApplication: NSApplication {
private static let escapeKeyCode: UInt16 = 53
override func sendEvent(_ event: NSEvent) {
- guard event.type == .keyDown || event.type == .keyUp,
- event.window?.firstResponder is VPhoneVirtualMachineView
+ guard event.type == .keyDown || event.type == .keyUp || event.type == .flagsChanged,
+ let view = event.window?.firstResponder as? VPhoneVirtualMachineView
else {
super.sendEvent(event)
return
}
+ if event.type == .flagsChanged {
+ forwardGlobe(event, view: view)
+ super.sendEvent(event)
+ return
+ }
// Esc replays the guest's back gesture: iOS has no back key, and a
// forwarded Escape only reads as cancel. The press and its release both
- // stop here, so the guest sees neither.
- if event.keyCode == Self.escapeKeyCode {
- if event.type == .keyDown,
- let view = event.window?.firstResponder as? VPhoneVirtualMachineView
- {
+ // stop here, so the guest sees neither. An iPad has a real Esc key —
+ // it cancels a composition, a sheet or a menu — so there it goes
+ // through like any other key.
+ if event.keyCode == Self.escapeKeyCode, view.escapeIsBackGesture {
+ if event.type == .keyDown {
view.performBackGesture()
}
return
}
+ if forwardDroppedKey(event, view: view) {
+ return
+ }
if event.type == .keyUp {
if menuKeyCodes.remove(event.keyCode) == nil {
super.sendEvent(event)
@@ -52,4 +72,58 @@ final class VPhoneApplication: NSApplication {
}
super.sendEvent(event)
}
+
+ // MARK: - Keys the Virtual Keyboard Drops
+
+ /// fn / 🌐 reaches AppKit only as a flags change, and the virtual USB
+ /// keyboard has no field for it. vphoned presses the guest's 🌐 for it, held
+ /// for as long as fn is, so 🌐 alone switches the input source and 🌐
+ /// shortcuts work. The event still goes on to the view, which keeps the
+ /// virtual keyboard's own modifier state in step.
+ private func forwardGlobe(_ event: NSEvent, view: VPhoneVirtualMachineView) {
+ guard event.keyCode == VPhoneGuestKeyMap.functionKeyCode, let control = view.control else { return }
+ let globe = VPhoneGuestKeyMap.globe
+ if event.modifierFlags.contains(.function) {
+ guard globeHolder == nil else { return }
+ globeHolder = control
+ control.sendHIDDown(page: globe.page, usage: globe.usage)
+ resignObserver = NotificationCenter.default.addObserver(
+ forName: NSWindow.didResignKeyNotification,
+ object: event.window,
+ queue: .main,
+ ) { [weak self] _ in
+ MainActor.assumeIsolated { self?.releaseGlobe() }
+ }
+ } else {
+ releaseGlobe()
+ }
+ }
+
+ private func releaseGlobe() {
+ if let resignObserver {
+ NotificationCenter.default.removeObserver(resignObserver)
+ }
+ resignObserver = nil
+ guard let holder = globeHolder else { return }
+ globeHolder = nil
+ let globe = VPhoneGuestKeyMap.globe
+ holder.sendHIDUp(page: globe.page, usage: globe.usage)
+ }
+
+ /// Sends a key `_VZKeyboard` would drop — the JIS input-mode and character
+ /// keys, Context Menu, Insert, volume — through vphoned. Returns whether the
+ /// event was one of them; it then goes no further.
+ private func forwardDroppedKey(_ event: NSEvent, view: VPhoneVirtualMachineView) -> Bool {
+ guard let usage = VPhoneGuestKeyMap.usage(forDroppedKeyCode: event.keyCode),
+ let control = view.control
+ else { return false }
+ if event.type == .keyDown {
+ // The guest repeats a held key itself.
+ guard !event.isARepeat, forwardedKeyCodes.insert(event.keyCode).inserted else { return true }
+ control.sendHIDDown(page: usage.page, usage: usage.usage)
+ } else if forwardedKeyCodes.remove(event.keyCode) != nil {
+ control.sendHIDUp(page: usage.page, usage: usage.usage)
+ }
+ return true
+ }
}
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift
index b9d64211..41f3ae21 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/UserInterface/VPhoneVirtualMachineKeySender.swift
@@ -2,6 +2,7 @@ import AppKit
import Dynamic
import Foundation
import Virtualization
+import VPhoneCoreKit
// MARK: - Key Helper
@@ -69,6 +70,14 @@ class VPhoneVirtualMachineKeySender {
control.sendHIDUp(page: 0x07, usage: 0xE3) // Cmd up
}
+ /// A 🌐 tap: the guest switches to its next input source. The Mac's fn key
+ /// does the same while the VM window is key; see `VPhoneApplication`.
+ func sendGlobe() {
+ guard requireConnection() else { return }
+ let globe = VPhoneGuestKeyMap.globe
+ control.sendHIDPress(page: globe.page, usage: globe.usage)
+ }
+
// MARK: - Type ASCII from Clipboard
func typeFromClipboard() {
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift
index d157dfe7..7d887e02 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneBootCommand+VirtualMachine.swift
@@ -31,6 +31,7 @@ extension VPhoneBootCommand {
screenPPI: manifest.screenConfig.pixelsPerInch,
screenScale: manifest.screenConfig.scale,
kernelDebugPort: kernelDebugPort,
+ isPadGuest: manifest.guestDevice.isPad,
)
}
}
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift
index 880fff8c..7810a4c9 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneVirtualMachineAppDelegate.swift
@@ -128,6 +128,7 @@ class VPhoneVirtualMachineAppDelegate: NSObject, NSApplicationDelegate {
.resolvingSymlinksInPath()
.path,
)
+ wc.captureView?.escapeIsBackGesture = !options.isPadGuest
windowController = wc
let fileWC = VPhoneFileWindowController()
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift
index 9923b4e7..0d4b59b2 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachine.swift
@@ -31,6 +31,8 @@ class VPhoneVirtualMachine: NSObject, VZVirtualMachineDelegate {
var screenPPI: Int = 460
var screenScale: Double = 3.0
var kernelDebugPort: Int?
+ /// The guest runs iPadOS: Esc is a key there, not the back gesture.
+ var isPadGuest = false
}
private struct DeviceIdentity {
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift
index f107af91..754adf58 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VirtualMachine/VPhoneVirtualMachineView.swift
@@ -7,6 +7,9 @@ import VPhoneCoreKit
class VPhoneVirtualMachineView: VZVirtualMachineView {
var keySender: VPhoneVirtualMachineKeySender?
weak var control: VPhoneGuestControl?
+ /// Whether Esc replays the back gesture (an iPhone, which has no Esc key)
+ /// or reaches the guest as Esc (an iPad). See `VPhoneApplication`.
+ var escapeIsBackGesture = true
/// Whether trackpad scroll/pinch gestures are replayed as guest touches.
var trackpadGesturesEnabled = VPhoneTrackpadGestures.isEnabled {
diff --git a/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swift b/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swift
new file mode 100644
index 00000000..6efcb312
--- /dev/null
+++ b/VPhoneKit/VPhoneCoreKit/VirtualMachine/VPhoneGuestKeyMap.swift
@@ -0,0 +1,56 @@
+// MARK: - Guest Key Map
+
+/// Mac keys the virtual USB keyboard cannot carry, and the HID usage the guest
+/// gets for each instead, by way of vphoned.
+///
+/// `VZUSBKeyboardConfiguration` is a boot-protocol keyboard: its report holds
+/// the eight page-7 modifiers (0xE0–0xE7) and six page-7 keys in 0x00–0x91,
+/// nothing else. `_VZKeyboard` first maps each Mac virtual key code to an
+/// internal index through a 179-entry table, and drops codes the table has no
+/// index for (macOS 27.0.1, Virtualization 259). Measured against that table
+/// and the report descriptor, the keys that never reach a guest are:
+///
+/// - **fn / 🌐** (`kVK_Function`, 0x3F): has an index, but no report field. On
+/// an iPad, 🌐 switches the input source and starts the 🌐 shortcuts; Apple's
+/// keyboards send it as Apple vendor top-case page 0xFF, usage 0x03, which
+/// iPadOS honours from any keyboard service.
+/// - the JIS keys 英数 / かな / ¥ / _ / keypad `,`, which switch the Japanese
+/// input mode and type their characters;
+/// - Context Menu, Help/Insert, and the volume and mute keys.
+public enum VPhoneGuestKeyMap {
+ public struct Usage: Hashable, Sendable {
+ public let page: UInt32
+ public let usage: UInt32
+
+ public init(page: UInt32, usage: UInt32) {
+ self.page = page
+ self.usage = usage
+ }
+ }
+
+ /// `kVK_Function`, which AppKit reports in a flags-changed event.
+ public static let functionKeyCode: UInt16 = 0x3F
+
+ /// The 🌐 key of an Apple keyboard: vendor top-case page, keyboard fn.
+ public static let globe = Usage(page: 0xFF, usage: 0x03)
+
+ /// Key-down / key-up codes `_VZKeyboard` drops, with the usage to send.
+ public static let droppedKeys: [UInt16: Usage] = [
+ 0x66: Usage(page: 0x07, usage: 0x91), // JIS 英数 → LANG2
+ 0x68: Usage(page: 0x07, usage: 0x90), // JIS かな → LANG1
+ 0x5D: Usage(page: 0x07, usage: 0x89), // JIS ¥ → International3
+ 0x5E: Usage(page: 0x07, usage: 0x87), // JIS _ → International1
+ 0x5F: Usage(page: 0x07, usage: 0x85), // JIS keypad , → Keypad Comma
+ 0x6E: Usage(page: 0x07, usage: 0x65), // Context Menu → Application
+ 0x72: Usage(page: 0x07, usage: 0x49), // Help / Insert → Insert
+ 0x48: Usage(page: 0x0C, usage: 0xE9), // Volume Up
+ 0x49: Usage(page: 0x0C, usage: 0xEA), // Volume Down
+ 0x4A: Usage(page: 0x0C, usage: 0xE2), // Mute
+ ]
+
+ /// The usage a key-down or key-up with `keyCode` needs sent by hand, or nil
+ /// for a key the virtual keyboard carries itself.
+ public static func usage(forDroppedKeyCode keyCode: UInt16) -> Usage? {
+ droppedKeys[keyCode]
+ }
+}
diff --git a/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swift b/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swift
new file mode 100644
index 00000000..3f9e3e87
--- /dev/null
+++ b/VPhoneKit/VPhoneCoreKitTests/VirtualMachine/GuestKeyMapTests.swift
@@ -0,0 +1,33 @@
+import Testing
+@testable import VPhoneCoreKit
+
+/// The keys the virtual USB keyboard drops, and what the guest gets instead.
+@Suite("Guest key map")
+struct GuestKeyMapTests {
+ @Test
+ func `fn becomes the Apple keyboard 🌐 usage`() {
+ #expect(VPhoneGuestKeyMap.functionKeyCode == 0x3F)
+ #expect(VPhoneGuestKeyMap.globe == .init(page: 0xFF, usage: 0x03))
+ }
+
+ @Test
+ func `the JIS input-mode keys become LANG1 and LANG2`() {
+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: 0x68) == .init(page: 0x07, usage: 0x90))
+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: 0x66) == .init(page: 0x07, usage: 0x91))
+ }
+
+ @Test
+ func `keys the virtual keyboard carries are left to it`() {
+ // A, Space, Escape, Caps Lock, Left Control, F1, Up Arrow, kVK_Function.
+ for keyCode: UInt16 in [0x00, 0x31, 0x35, 0x39, 0x3B, 0x7A, 0x7E, 0x3F] {
+ #expect(VPhoneGuestKeyMap.usage(forDroppedKeyCode: keyCode) == nil)
+ }
+ }
+
+ @Test
+ func `every forwarded usage is on a page the guest accepts from vphoned`() {
+ for usage in VPhoneGuestKeyMap.droppedKeys.values {
+ #expect([0x07, 0x0C].contains(usage.page))
+ }
+ }
+}