Skip to content

Run a virtual iPhone on an Apple Silicon Mac.

Swift

Duo: show the inner display's own pages instead of rendering it twice

Jackyd2d4c34313b1
diff --git a/Research/Guest/iphone_duo_bringup.md b/Research/Guest/iphone_duo_bringup.md
index 18876e33..8dfbbb6f 100644
--- a/Research/Guest/iphone_duo_bringup.md
+++ b/Research/Guest/iphone_duo_bringup.md
@@ -4900,3 +4900,68 @@ Reveal and validation on a new guest: `device.input_devices` lists
`MGGetLogicalDeviceDisplayCount()` is 2, `+[CMAngleManager isAvailable]` is
YES and `SBFDevicePoseProvider`'s description reads `.partiallyOpen` at
~99.4° six seconds after `{"hinge":"partially-open","position":"book"}`.
+
+## 2026-10-30: the panel hosts the inner display's own pages
+
+Open, the guest lagged badly. A spindump of backboardd on an idle Home Screen
+(iOS 27.1, M5 Pro host, no DeviceHub viewer) had the render server thread in
+`_XRenderDisplaySurface → VirtualServer::render_surface → ShadowNode::apply →
+blur_surface → … -[MTLCommandBuffer waitUntilCompleted]` in 116 of 125
+samples, SpringBoard's inner-panel queue inside `CARenderServerRenderDisplay`
+in 118 of 126, and the inner display's own `VirtualServer` thread waiting on
+the render server's lock. The compositor above re-rendered the whole inner
+display, blurs included, for every panel frame, while the inner display was
+rendering itself as well: `VirtualDisplay::begin_update` /
+`finish_update_` draw only damage, into three pages in turn
+(`displayed_surface` returns the last finished one).
+
+Ruled out as a "did the inner display change" signal:
+`CARenderServerGetDirtyFrameCountByIndex` and `GetFrameCounterByIndex` count
+only the LCD (index 0); every other display, the inner one included, reads 0,
+so gating the copy on it froze the panel on its first frame.
+`updateRequestCallback` is a bare `ret` for non-IOMFB displays,
+`frameInfoCallback` is IOMFB-only, `CADisplay.seed` is configuration, and
+`-acquireFrozenSurface` is `Display::acquire_frozen_surface`, which returns
+NULL unless an `IOMFBDisplay` overrides it.
+
+**The fix** (`FoldDisplayInnerPages.m`): backboardd shows the inner display's
+last finished page in a layer of a context of its own (`remoteContextWithOptions:`
+with `displayable` NO and `ignoresHitTest` YES) and publishes the context id
+as the state of the Darwin notification `com.vphone.folddisplay.inner-pages`;
+SpringBoard's compositor layer is a `CALayerHost` of that context. A 120 Hz
+timer on a backboardd queue reads the shown page under the two mutexes
+`acquireFrozenSurface` takes and commits only when it changed. Nothing in
+QuartzCore hands out the pages, so the path is read from QuartzCore itself and
+checked at each step: the `_impl` type encoding (Server after the 64-byte
+mutex); the instructions of `-acquireFrozenSurface` (server lock at +0x18,
+C++ display at +0x60 on 27.1); the one vtable function with
+`displayed_surface`'s body (slot 63); and the one pair of
+`CA::WindowServer::IOSurface` getters for an object and the 32-bit value after
+it (`iosurface()` +0x128 and `id()` +0x130, slots 21 and 24), checked against
+`IOSurfaceGetID` before first use. The page is 2853×2007 BGRA. If a step does
+not match, backboardd publishes 0 and SpringBoard keeps the
+`CARenderServerRenderDisplay` path; it switches when the state changes while
+the panel is shown (tested both ways with vphoned `notify.post`).
+
+Measured on a clone of duospike-inner (duopanel-perf), the same session, CPU
+from `processes.list` over 8–10 s, panel rate from
+`CARenderServerGetFrameCounterByIndex(0, 0)` every 2 s:
+
+| | upstream compositor | hosted pages |
+|---|---|---|
+| idle Home Screen: backboardd / SpringBoard | 60–77% / 10–18% | 5–8% / 2–3% |
+| idle Home Screen: panel frames per second | 37–46 (nothing changed) | 10 (what changed) |
+| Home Screen page swipes: panel fps | 7.5–13.5 | 34–44 |
+| scrolling Settings: panel fps, backboardd | 45–79, 91% | 111–117, 51% |
+| folded closed: backboardd / SpringBoard | | 3% / 1% |
+
+Without SpringBoard's 120 Hz display link the panel still reaches the inner
+display's rate: the render server composites the LCD on each commit of the
+hosted context. Checked on the same VM: Settings launch and taps, Home,
+unlock, fold closed (cover shows natively) and open again, backboardd
+restarts (a new context id each time, SpringBoard restarts with it); no
+backboardd or SpringBoard crash reports.
+
+Trap seen while probing: the CA `IOSurface` object in a page is a C++ object,
+not an `IOSurfaceRef`; `CFRetain` on it is a PAC failure that crash-loops
+backboardd.
diff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplayBackboard.m b/VPhoneGuestComponents/FoldDisplay/FoldDisplayBackboard.m
index a50a4128..564aea42 100644
--- a/VPhoneGuestComponents/FoldDisplay/FoldDisplayBackboard.m
+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplayBackboard.m
@@ -87,6 +87,7 @@ static BOOL vpAddInnerRenderDisplayToServer(id server) {
((void (*)(id, SEL, BOOL))objc_msgSend)(display, setEnabled, YES);
vpTagUnknownDisplay(display);
vpInnerRenderDisplay = display;
+ dispatch_async(dispatch_get_main_queue(), ^{ vpShareInnerDisplayPages(display); });
vpFoldDisplayLog([NSString stringWithFormat:@"inner display: render display added %@ uniqueId=%@", display,
[display respondsToSelector:sel_registerName("uniqueId")] ? [display valueForKey:@"uniqueId"] : @"-"]);
return YES;
diff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplayInnerPages.m b/VPhoneGuestComponents/FoldDisplay/FoldDisplayInnerPages.m
new file mode 100644
index 00000000..470d30e0
--- /dev/null
+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplayInnerPages.m
@@ -0,0 +1,291 @@
+// backboardd: the inner display's own frames, for SpringBoard to show on the
+// shared panel (FoldDisplayPanel.m).
+//
+// The inner display is a CAWindowServerVirtualDisplay. The render server
+// draws it like any display: only when one of its contexts commits, only the
+// damaged area, into three pages it takes turns with. Having the render
+// server draw it again into a surface of SpringBoard's on every panel frame
+// (CARenderServerRenderDisplay) is a second, full render: 120 times a second,
+// blurs included, on the render server's one thread, idle or not. That kept
+// backboardd near a core and held up every display's frames.
+//
+// So backboardd shows the page the inner display last finished in a layer of
+// a context of its own, and SpringBoard hosts that context. It commits only
+// when the inner display has finished a new page, so an idle inner display
+// costs the panel nothing. The page being drawn is never the one shown: it is
+// the one finished two pages before.
+//
+// No API hands out a display's pages; acquireFrozenSurface, the nearest,
+// answers only for hardware displays. They are read the way QuartzCore reads
+// them, each step taken from QuartzCore and checked before use:
+// - the CAWindowServerDisplay's _impl, whose type encoding puts the Server
+// right after the impl's mutex;
+// - -[CAWindowServerDisplay acquireFrozenSurface], which loads the server
+// from the impl, locks the server's mutex and loads the C++ display from
+// the server: the offsets come from those instructions;
+// - the C++ display's virtual displayed_surface(bool), the one function in
+// its vtable with that body (load the shown page's index, load the page,
+// load its surface), which returns the shown page's CA::WindowServer::
+// IOSurface;
+// - that surface's virtual iosurface() and id(), the one pair of getters for
+// an object and a 32-bit value right after it, checked against each other
+// with IOSurfaceGetID before the first use.
+// Both mutexes are held while the page is read, as acquireFrozenSurface holds
+// them. If a step does not match, backboardd publishes no context and
+// SpringBoard draws the inner display itself.
+
+#import "FoldDisplayPrivate.h"
+#include <notify.h>
+#include <pthread.h>
+#include <ptrauth.h>
+
+typedef struct __IOSurface *VPSurfaceRef;
+
+// MARK: - instruction shapes
+
+static const uint32_t vpReturn = 0xD65F03C0u;
+
+// ldr xt, [xn, #offset]
+static BOOL vpLoad64(uint32_t word, unsigned *rt, unsigned *rn, unsigned *offset) {
+ if ((word & 0xFFC00000u) != 0xF9400000u) return NO;
+ *rt = word & 31;
+ *rn = (word >> 5) & 31;
+ *offset = ((word >> 10) & 0xFFF) * 8;
+ return YES;
+}
+
+// add xd, xn, #value
+static BOOL vpAdd64(uint32_t word, unsigned *rd, unsigned *rn, unsigned *value) {
+ if ((word & 0xFF800000u) != 0x91000000u) return NO;
+ *rd = word & 31;
+ *rn = (word >> 5) & 31;
+ *value = ((word >> 10) & 0xFFF) << ((word >> 22) & 1 ? 12 : 0);
+ return YES;
+}
+
+// mov wd, #value
+static BOOL vpMove32(uint32_t word, unsigned rd, unsigned *value) {
+ if ((word & 0xFFE0001Fu) != (0x52800000u | rd)) return NO;
+ *value = (word >> 5) & 0xFFFF;
+ return YES;
+}
+
+// ldr w0|x0, [x0, #offset]; ret
+static BOOL vpGetter(const uint32_t *code, BOOL wide, unsigned *offset) {
+ if ((code[0] & 0xFFC003FFu) != (wide ? 0xF9400000u : 0xB9400000u) || code[1] != vpReturn) return NO;
+ *offset = ((code[0] >> 10) & 0xFFF) * (wide ? 8 : 4);
+ return YES;
+}
+
+// mov w8, #pages; add x8, x0, x8; mov w9, #index; add x9, x0, x9;
+// ldrb w9, [x9]; mov w10, #8; orr x9, x10, x9, lsl #4; ldr x8, [x8, x9];
+// cbz x8, 1f; ldr x0, [x8, #surface]; ret
+static BOOL vpDisplayedSurfaceBody(const uint32_t *code) {
+ unsigned pages, index, rt, rn, surface;
+ return vpMove32(code[0], 8, &pages) && code[1] == 0x8B080008u && vpMove32(code[2], 9, &index) &&
+ code[3] == 0x8B090009u && code[4] == 0x39400129u && code[5] == 0x5280010Au && code[6] == 0xAA091149u &&
+ code[7] == 0xF8696908u && code[8] == 0xB4000068u && vpLoad64(code[9], &rt, &rn, &surface) && rt == 0 &&
+ rn == 8 && code[10] == vpReturn;
+}
+
+// MARK: - the inner display's pages
+
+static BOOL vpInQuartzCore(const void *address) {
+ Dl_info info;
+ return dladdr(address, &info) && info.dli_fname && strstr(info.dli_fname, "/QuartzCore");
+}
+
+static const uint32_t *vpCode(const void *pointer) {
+ return ptrauth_strip((const uint32_t *)pointer, ptrauth_key_function_pointer);
+}
+
+static void **vpVirtualTable(void *object) {
+ return ptrauth_strip(*(void ***)object, ptrauth_key_process_independent_data);
+}
+
+static BOOL vpVirtualSlot(void **table, int slot) {
+ return table[slot] && vpInQuartzCore(vpCode(table[slot]));
+}
+
+static void *vpCallable(const uint32_t *code) {
+ return ptrauth_sign_unauthenticated((void *)code, ptrauth_key_function_pointer, 0);
+}
+
+static pthread_mutex_t *vpImplLock, *vpServerLock;
+static void *vpDisplay;
+static void *(*vpDisplayedSurface)(void *, bool);
+static VPSurfaceRef (*vpSurfaceIOSurface)(void *);
+static uint32_t (*vpSurfaceGetID)(VPSurfaceRef);
+static BOOL vpSurfaceGetterResolved, vpSurfaceGetterFailed;
+
+static BOOL vpResolveInnerDisplay(id display) {
+ Ivar implIvar = class_getInstanceVariable(objc_getClass("CAWindowServerDisplay"), "_impl");
+ const char *implType = implIvar ? ivar_getTypeEncoding(implIvar) : "missing";
+ static const char serverAfterMutex[] = "^{CAWindowServerDisplayImpl={Mutex={_opaque_pthread_mutex_t=q[56c]}}^{Server}";
+ if (strncmp(implType, serverAfterMutex, sizeof(serverAfterMutex) - 1) != 0) {
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: impl layout unavailable (%.120s)", implType]);
+ return NO;
+ }
+ Method acquire = class_getInstanceMethod(objc_getClass("CAWindowServerDisplay"), sel_registerName("acquireFrozenSurface"));
+ if (!acquire || strcmp(method_getTypeEncoding(acquire), "^{__IOSurface=}16@0:8") != 0) {
+ vpFoldDisplayLog(@"inner pages: acquireFrozenSurface unavailable");
+ return NO;
+ }
+ const uint32_t *code = vpCode((void *)method_getImplementation(acquire));
+ unsigned server = 32, lockOffset = 0, displayOffset = 0;
+ for (int index = 0; index < 32 && !displayOffset; index++) {
+ unsigned rt, rn, value;
+ if (server == 32 && vpLoad64(code[index], &rt, &rn, &value) && value == sizeof(pthread_mutex_t))
+ server = rt;
+ else if (server != 32 && !lockOffset && vpAdd64(code[index], &rt, &rn, &value) && rt == 0 && rn == server)
+ lockOffset = value;
+ else if (server != 32 && lockOffset && vpLoad64(code[index], &rt, &rn, &value) && rt == 0 && rn == server)
+ displayOffset = value;
+ }
+ if (!displayOffset) {
+ vpFoldDisplayLog(@"inner pages: server path not found in acquireFrozenSurface");
+ return NO;
+ }
+ uint8_t *impl = *(uint8_t **)((uint8_t *)(__bridge void *)display + ivar_getOffset(implIvar));
+ uint8_t *serverObject = impl ? *(uint8_t **)(impl + sizeof(pthread_mutex_t)) : NULL;
+ void *cppDisplay = serverObject ? *(void **)(serverObject + displayOffset) : NULL;
+ if (!cppDisplay || !vpInQuartzCore(vpVirtualTable(cppDisplay))) {
+ vpFoldDisplayLog(@"inner pages: display object unavailable");
+ return NO;
+ }
+ void **table = vpVirtualTable(cppDisplay);
+ int matches = 0, found = -1, slot = 0;
+ for (; slot < 512 && vpVirtualSlot(table, slot); slot++)
+ if (vpDisplayedSurfaceBody(vpCode(table[slot]))) { found = slot; matches++; }
+ if (matches != 1) {
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: %d displayed-surface functions in %d slots", matches, slot]);
+ return NO;
+ }
+ vpImplLock = (pthread_mutex_t *)impl;
+ vpServerLock = (pthread_mutex_t *)(serverObject + lockOffset);
+ vpDisplay = cppDisplay;
+ vpDisplayedSurface = vpCallable(vpCode(table[found]));
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: display at server +0x%x, lock +0x%x, displayed surface slot %d",
+ displayOffset, lockOffset, found]);
+ return YES;
+}
+
+// Under both locks, with the first page the inner display shows.
+static BOOL vpResolveSurfaceGetter(void *surface) {
+ void **table = vpVirtualTable(surface);
+ int pairs = 0, objectSlot = -1, identifierSlot = -1;
+ for (int slot = 0; slot < 256 && vpVirtualSlot(table, slot); slot++) {
+ unsigned objectOffset, identifierOffset;
+ if (!vpGetter(vpCode(table[slot]), YES, &objectOffset)) continue;
+ for (int other = 0; other < 256 && vpVirtualSlot(table, other); other++)
+ if (vpGetter(vpCode(table[other]), NO, &identifierOffset) && identifierOffset == objectOffset + 8) {
+ objectSlot = slot;
+ identifierSlot = other;
+ pairs++;
+ }
+ }
+ if (pairs != 1) {
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: %d surface getter pairs", pairs]);
+ return NO;
+ }
+ VPSurfaceRef (*object)(void *) = vpCallable(vpCode(table[objectSlot]));
+ uint32_t (*identifier)(void *) = vpCallable(vpCode(table[identifierSlot]));
+ VPSurfaceRef iosurface = object(surface);
+ if (!iosurface || vpSurfaceGetID(iosurface) != identifier(surface)) {
+ vpFoldDisplayLog(@"inner pages: surface getters disagree");
+ return NO;
+ }
+ vpSurfaceIOSurface = object;
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: surface getters at slots %d and %d", objectSlot, identifierSlot]);
+ return YES;
+}
+
+// The page the inner display last finished, retained; NULL before its first.
+static VPSurfaceRef vpCopyShownPage(void) {
+ pthread_mutex_lock(vpImplLock);
+ pthread_mutex_lock(vpServerLock);
+ void *surface = vpDisplayedSurface(vpDisplay, false);
+ if (surface && !vpSurfaceGetterResolved && !vpSurfaceGetterFailed) {
+ vpSurfaceGetterResolved = vpResolveSurfaceGetter(surface);
+ vpSurfaceGetterFailed = !vpSurfaceGetterResolved;
+ }
+ VPSurfaceRef page = surface && vpSurfaceGetterResolved ? vpSurfaceIOSurface(surface) : NULL;
+ if (page) CFRetain(page);
+ pthread_mutex_unlock(vpServerLock);
+ pthread_mutex_unlock(vpImplLock);
+ return page;
+}
+
+// MARK: - the context SpringBoard hosts
+
+static id vpPagesContext, vpPagesLayer;
+static dispatch_source_t vpPagesTimer;
+static VPSurfaceRef vpShownPage;
+
+static void vpPublishPagesContext(uint32_t contextId) {
+ int token;
+ if (notify_register_check(vpInnerPagesNotification, &token) != NOTIFY_STATUS_OK) return;
+ notify_set_state(token, contextId);
+ notify_cancel(token);
+ notify_post(vpInnerPagesNotification);
+}
+
+// On the pages queue, at the inner display's top rate. The context is
+// published with its first page, so a host never shows it empty.
+static void vpShowNewestPage(void) {
+ VPSurfaceRef page = vpCopyShownPage();
+ if (vpSurfaceGetterFailed) {
+ dispatch_source_cancel(vpPagesTimer);
+ return;
+ }
+ if (!page || page == vpShownPage) {
+ if (page) CFRelease(page);
+ return;
+ }
+ Class transaction = objc_getClass("CATransaction");
+ ((void (*)(id, SEL))objc_msgSend)(transaction, sel_registerName("begin"));
+ ((void (*)(id, SEL, BOOL))objc_msgSend)(transaction, sel_registerName("setDisableActions:"), YES);
+ ((void (*)(id, SEL, id))objc_msgSend)(vpPagesLayer, sel_registerName("setContents:"), (__bridge id)page);
+ ((void (*)(id, SEL))objc_msgSend)(transaction, sel_registerName("commit"));
+ BOOL first = !vpShownPage;
+ if (vpShownPage) CFRelease(vpShownPage);
+ vpShownPage = page;
+ if (first) {
+ uint32_t contextId = ((uint32_t (*)(id, SEL))objc_msgSend)(vpPagesContext, sel_registerName("contextId"));
+ vpPublishPagesContext(contextId);
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: context %u published", contextId]);
+ }
+}
+
+// From the main queue once the render server runs, never from its display
+// detection: the context is a client of that server.
+void vpShareInnerDisplayPages(id display) {
+ vpPublishPagesContext(0);
+ void *iosurface = dlopen("/System/Library/Frameworks/IOSurface.framework/IOSurface", RTLD_NOW);
+ vpSurfaceGetID = iosurface ? dlsym(iosurface, "IOSurfaceGetID") : NULL;
+ Method create = class_getClassMethod(objc_getClass("CAContext"), sel_registerName("remoteContextWithOptions:"));
+ if (!vpSurfaceGetID || !create || strcmp(method_getTypeEncoding(create), "@24@0:8@16") != 0) {
+ vpFoldDisplayLog(@"inner pages: context ABI unavailable");
+ return;
+ }
+ if (!vpResolveInnerDisplay(display)) return;
+ // Hosted only: never drawn on a display by itself, never hit-tested.
+ CFStringRef *displayable = dlsym(RTLD_DEFAULT, "kCAContextDisplayable");
+ CFStringRef *ignoresHitTest = dlsym(RTLD_DEFAULT, "kCAContextIgnoresHitTest");
+ NSMutableDictionary *options = [NSMutableDictionary dictionary];
+ if (displayable) options[(__bridge NSString *)*displayable] = @NO;
+ if (ignoresHitTest) options[(__bridge NSString *)*ignoresHitTest] = @YES;
+ vpPagesContext = ((id (*)(id, SEL, id))objc_msgSend)(objc_getClass("CAContext"), sel_registerName("remoteContextWithOptions:"), options);
+ // In the inner display's pixels; SpringBoard's host scales and turns them.
+ CGRect bounds = ((CGRect (*)(id, SEL))objc_msgSend)(display, sel_registerName("bounds"));
+ vpPagesLayer = [objc_getClass("CALayer") new];
+ ((void (*)(id, SEL, CGRect))objc_msgSend)(vpPagesLayer, sel_registerName("setFrame:"), (CGRect){CGPointZero, bounds.size});
+ ((void (*)(id, SEL, id))objc_msgSend)(vpPagesContext, sel_registerName("setLayer:"), vpPagesLayer);
+ dispatch_queue_t queue = dispatch_queue_create("com.vphone.folddisplay.inner-pages", DISPATCH_QUEUE_SERIAL);
+ vpPagesTimer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, queue);
+ dispatch_source_set_timer(vpPagesTimer, DISPATCH_TIME_NOW, NSEC_PER_SEC / 120, NSEC_PER_SEC / 1200);
+ dispatch_source_set_event_handler(vpPagesTimer, ^{ vpShowNewestPage(); });
+ dispatch_resume(vpPagesTimer);
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner pages: sharing {%.0f, %.0f}, %lu context options",
+ bounds.size.width, bounds.size.height, (unsigned long)options.count]);
+}
diff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m
index 11575b99..0a28649f 100644
--- a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m
+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPanel.m
@@ -3,8 +3,10 @@
// composites the inner display onto it.
#import "FoldDisplayPrivate.h"
+#include <notify.h>
static void vpShowInnerDisplay(BOOL shown);
+static void vpFollowInnerPages(void);
// MARK: - power
@@ -132,6 +134,7 @@ void vpInstallSharedPanel(void) {
}
vpOriginalTransitionDisplayState = method_setImplementation(transition, (IMP)vpSharedScanoutTransition);
vpInstallContentModeOwnership();
+ vpFollowInnerPages();
vpFoldDisplayLog(@"inner display: shared scanout installed");
}
@@ -139,10 +142,8 @@ void vpInstallSharedPanel(void) {
// The render server's own clone of a display (addClone:options:) shows
// nothing on this guest's paravirtual panel (black with every option), so
-// SpringBoard shows the inner display there itself: on every primary frame
-// it has the render server draw the inner display into a surface
-// (CARenderServerRenderDisplay) and shows that surface in a layer at the top
-// of Main's root scene window (UIRootSceneWindow, the window Main's scenes
+// SpringBoard shows the inner display there itself, in a layer at the top of
+// Main's root scene window (UIRootSceneWindow, the window Main's scenes
// are hosted in). It is not in a window of SB-main, because open,
// SpringBoard gives the cover (Main) auxiliary content and takes its scene's
// windows off screen. Like every window it works in points, so UIKit's
@@ -164,12 +165,18 @@ void vpInstallSharedPanel(void) {
// backdrop are drawn against the primary display: holes beside a launching
// app's icon, a black frame going home, and ghosting glass.
//
-// Three surfaces take turns, so the render server never draws into the one
-// the panel shows or the one it showed last. Each frame of the inner display
-// reaches the panel one primary frame later.
+// The layer hosts backboardd's context that shows the page the inner display
+// last finished (FoldDisplayInnerPages.m), so the panel shows each frame the
+// inner display draws, and only those. Where backboardd publishes no context,
+// SpringBoard has the render server draw the inner display into a surface of
+// its own on every primary frame (CARenderServerRenderDisplay), a second full
+// render each frame. Three surfaces take turns then, so the render server
+// never draws into the one the panel shows or the one it showed last.
static id vpInnerPanelRoot, vpInnerPanelImage, vpInnerPanelRefresh;
static void *vpInnerPanelSurfaces[3];
static int vpInnerPanelShown;
+// The hosted context, 0 while SpringBoard draws the inner display itself.
+static uint32_t vpInnerPanelContextId;
static BOOL vpInnerPanelRendering; // main thread only
static dispatch_queue_t vpInnerPanelQueue;
static void (*vpRenderDisplay)(mach_port_t, CFStringRef, void *, int32_t, int32_t);
@@ -220,6 +227,16 @@ static CGSize vpInnerDisplayPixels(void) {
return CGSizeZero;
}
+// backboardd's context for the inner display's pages, 0 for none.
+static uint32_t vpInnerPagesContext(void) {
+ int token;
+ uint64_t state = 0;
+ if (notify_register_check(vpInnerPagesNotification, &token) != NOTIFY_STATUS_OK) return 0;
+ notify_get_state(token, &state);
+ notify_cancel(token);
+ return (uint32_t)state;
+}
+
static void vpReleaseInnerPanelSurfaces(void) {
for (int index = 0; index < 3; index++) {
if (vpInnerPanelSurfaces[index]) CFRelease(vpInnerPanelSurfaces[index]);
@@ -243,17 +260,18 @@ static void vpShowInnerDisplay(BOOL shown) {
return;
}
if (vpInnerPanelRoot) return;
+ uint32_t hosted = vpInnerPagesContext();
id rootWindow = vpMainRootSceneWindow();
CGSize pixels = vpInnerDisplayPixels();
void *(*createSurface)(CFDictionaryRef) =
dlsym(dlopen("/System/Library/Frameworks/IOSurface.framework/IOSurface", RTLD_NOW), "IOSurfaceCreate");
if (!vpRenderDisplay) vpRenderDisplay = dlsym(RTLD_DEFAULT, "CARenderServerRenderDisplay");
- if (!rootWindow || pixels.width < 1 || !createSurface || !vpRenderDisplay) {
+ if (!rootWindow || pixels.width < 1 || (!hosted && (!createSurface || !vpRenderDisplay))) {
vpFoldDisplayLog([NSString stringWithFormat:@"inner display: show unavailable root=%@ pixels={%.0f, %.0f}",
rootWindow, pixels.width, pixels.height]);
return;
}
- if (!vpInnerPanelSurfaces[0]) {
+ if (!hosted && !vpInnerPanelSurfaces[0]) {
NSDictionary *properties = @{@"IOSurfaceWidth": @(pixels.width), @"IOSurfaceHeight": @(pixels.height),
@"IOSurfaceBytesPerElement": @4, @"IOSurfacePixelFormat": @((uint32_t)'BGRA')};
for (int index = 0; index < 3; index++) vpInnerPanelSurfaces[index] = createSurface((__bridge CFDictionaryRef)properties);
@@ -279,33 +297,50 @@ static void vpShowInnerDisplay(BOOL shown) {
CGAffineTransform transform = CGAffineTransformConcat(
turned ? CGAffineTransformMakeRotation(-M_PI_2) : CGAffineTransformIdentity,
CGAffineTransformMakeScale(panel.size.width / upright.width, panel.size.height / upright.height));
+ // backboardd's layer is in the inner display's pixels.
+ if (hosted) transform = CGAffineTransformConcat(CGAffineTransformMakeScale(1 / pointScale, 1 / pointScale), transform);
id root = [objc_getClass("CALayer") new];
((void (*)(id, SEL, CGRect))objc_msgSend)(root, sel_registerName("setFrame:"), panel);
((void (*)(id, SEL, CGColorRef))objc_msgSend)(root, sel_registerName("setBackgroundColor:"),
((CGColorRef (*)(id, SEL))objc_msgSend)(vpReadObject(objc_getClass("UIColor"), "blackColor"), sel_registerName("CGColor")));
((void (*)(id, SEL, double))objc_msgSend)(root, sel_registerName("setZPosition:"), 1.0e9);
- id image = [objc_getClass("CALayer") new];
- ((void (*)(id, SEL, CGRect))objc_msgSend)(image, sel_registerName("setBounds:"), (CGRect){CGPointZero, inner});
+ id image = [objc_getClass(hosted ? "CALayerHost" : "CALayer") new];
+ if (hosted) ((void (*)(id, SEL, uint32_t))objc_msgSend)(image, sel_registerName("setContextId:"), hosted);
+ ((void (*)(id, SEL, CGRect))objc_msgSend)(image, sel_registerName("setBounds:"), (CGRect){CGPointZero, hosted ? pixels : inner});
((void (*)(id, SEL, CGPoint))objc_msgSend)(image, sel_registerName("setPosition:"), CGPointMake(CGRectGetMidX(panel), CGRectGetMidY(panel)));
((void (*)(id, SEL, CGAffineTransform))objc_msgSend)(image, sel_registerName("setAffineTransform:"), transform);
((void (*)(id, SEL, id))objc_msgSend)(root, sel_registerName("addSublayer:"), image);
((void (*)(id, SEL, id))objc_msgSend)(rootLayer, sel_registerName("addSublayer:"), root);
vpInnerPanelRoot = root;
vpInnerPanelImage = image;
+ vpInnerPanelContextId = hosted;
+ NSInteger fps = ((NSInteger (*)(id, SEL))objc_msgSend)(vpReadObject(rootWindow, "screen"), sel_registerName("maximumFramesPerSecond"));
+ vpFoldDisplayLog([NSString stringWithFormat:@"inner display: shown=1 panel={%.0f, %.0f} inner={%.0f, %.0f} turned=%d scale={%.3f, %.3f} fps=%ld context=%u",
+ panel.size.width, panel.size.height, inner.width, inner.height, turned,
+ panel.size.width / upright.width, panel.size.height / upright.height, (long)fps, hosted]);
+ if (hosted) return;
vpInnerPanelRefresh = ((id (*)(id, SEL, id, SEL))objc_msgSend)(objc_getClass("CADisplayLink"),
sel_registerName("displayLinkWithTarget:selector:"), [VPhoneInnerPanelRefresh new], @selector(refresh:));
// The panel shows the inner display at the panel's top rate, asked for
// rather than left to the system's choice for a link with no range.
- NSInteger fps = ((NSInteger (*)(id, SEL))objc_msgSend)(vpReadObject(rootWindow, "screen"), sel_registerName("maximumFramesPerSecond"));
SEL setRange = sel_registerName("setPreferredFrameRateRange:");
if (fps > 60 && [vpInnerPanelRefresh respondsToSelector:setRange])
((void (*)(id, SEL, VPhoneFrameRateRange))objc_msgSend)(vpInnerPanelRefresh, setRange, (VPhoneFrameRateRange){fps, fps, fps});
((void (*)(id, SEL, id, id))objc_msgSend)(vpInnerPanelRefresh, sel_registerName("addToRunLoop:forMode:"),
NSRunLoop.mainRunLoop, NSRunLoopCommonModes);
- vpFoldDisplayLog([NSString stringWithFormat:@"inner display: shown=1 panel={%.0f, %.0f} inner={%.0f, %.0f} turned=%d scale={%.3f, %.3f} fps=%ld",
- panel.size.width, panel.size.height, inner.width, inner.height, turned,
- panel.size.width / upright.width, panel.size.height / upright.height, (long)fps]);
} @catch (NSException *exception) {
vpFoldDisplayLog([NSString stringWithFormat:@"inner display: show exception %@", exception.reason]);
}
}
+
+// backboardd publishes its context with the inner display's first page,
+// which can come after the panel is shown, and takes it back if it cannot
+// read a page.
+static void vpFollowInnerPages(void) {
+ int token;
+ notify_register_dispatch(vpInnerPagesNotification, &token, dispatch_get_main_queue(), ^(int unused) {
+ if (!vpInnerPanelRoot || vpInnerPagesContext() == vpInnerPanelContextId) return;
+ vpShowInnerDisplay(NO);
+ vpShowInnerDisplay(YES);
+ });
+}
diff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPrivate.h b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPrivate.h
index fff3f000..5ebb71b8 100644
--- a/VPhoneGuestComponents/FoldDisplay/FoldDisplayPrivate.h
+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplayPrivate.h
@@ -23,6 +23,8 @@
// MARK: - FoldDisplaySupport.m
extern NSString *const vpInnerDisplayName;
+/// Its state is the context id backboardd shows the inner display's pages in, 0 for none.
+extern const char *const vpInnerPagesNotification;
void vpFoldDisplayLog(NSString *line);
id vpReadObject(id object, const char *name);
id vpReadObjectIvar(id object, const char *name);
@@ -31,6 +33,10 @@ id vpReadObjectIvar(id object, const char *name);
void vpInstallInnerRenderDisplay(void);
+// MARK: - FoldDisplayInnerPages.m
+
+void vpShareInnerDisplayPages(id display);
+
// MARK: - FoldDisplayAdapter.m
BOOL vpIsInnerDisplay(id display);
diff --git a/VPhoneGuestComponents/FoldDisplay/FoldDisplaySupport.m b/VPhoneGuestComponents/FoldDisplay/FoldDisplaySupport.m
index 215fcbc0..9ef4a3c1 100644
--- a/VPhoneGuestComponents/FoldDisplay/FoldDisplaySupport.m
+++ b/VPhoneGuestComponents/FoldDisplay/FoldDisplaySupport.m
@@ -4,6 +4,7 @@
#import "FoldDisplayPrivate.h"
NSString *const vpInnerDisplayName = @"VPhone Duo Inner";
+const char *const vpInnerPagesNotification = "com.vphone.folddisplay.inner-pages";
void vpFoldDisplayLog(NSString *line) {
int fd = open("/var/mobile/Library/Caches/vphone-folddisplay.log", O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0666);