Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

Metal, step one: a device that exists (metal=1), and what Metal asks of it first

shimoku455af7eae22c
diff --git a/docs/metal-plan.md b/docs/metal-plan.md
new file mode 100644
index 0000000..7e4d468
--- /dev/null
+++ b/docs/metal-plan.md
@@ -0,0 +1,74 @@
+# Metal on the Adreno: the plan, and where it stands
+
+Read `docs/ru/metal.md` (why Metal is not simply missing, and what is on each side of the wall) and `docs/air.md` (the
+shader compiler) first. This file is the order of work, with how each step is checked and where.
+
+## The three parts
+
+| Part | What it is | State |
+|---|---|---|
+| **A. The shader compiler** | Apple's shader bitcode (AIR, in a `.metallib`) into Mesa's NIR, so that ir3 (the compiler behind the OpenGL ES that already runs) makes Adreno code | largely done: 321 of the system's 456 shader modules reach ir3, 54 have run on the phone's GPU (`docs/air.md`, `docs/air-report.md`) |
+| **B. The device** | what Metal loads: a bundle with a subclass of Metal's private `_MTLDevice`, and the objects it hands out (buffers, libraries, pipeline states, queues, command buffers, encoders: 53 classes and about 1200 methods in Apple's own) | step 1 is written (below); nothing past it |
+| **C. The kernel side** | submitting work to the GPU for a Metal client, with memory and fences | the GPU already has a user client for Mesa (`drivers/gpu/hakios.gpu.cpp`, `HakiosGPUUserClient`); it is not connected to B |
+
+What "real Metal" would be, and why games are out of reach, is argued in `docs/ru/metal.md` section 9.4: that has not
+changed. What this plan aims at is the part that is within reach: **Metal clients of the system itself** (the window
+server's compositor, CoreImage, the camera's viewfinder) running on the Adreno through B and C, with shaders from A.
+
+## The order, and how each step is checked
+
+| # | Step | Checked by | Where |
+|---|---|---|---|
+| 1 | **A device that exists.** `metal=1` makes the (so far empty) `IOAcceleratorES` service name `HakiosMetal`; the bundle makes `HakiosMetalDevice`, a `_MTLDevice` subclass; `MTLCreateSystemDefaultDevice()` returns it | `metal_test` (`RC_JOBS=rc.jobs.metal`) prints the device and `HakiosMetal:` lines say which messages Metal sent that the device did not answer | QEMU, no GPU needed |
+| 2 | **Learn what Metal asks first**, from step 1's "not implemented" lines: properties, family support, queue, buffer | the log, in order | QEMU |
+| 3 | **Buffers and textures** over the GPU memory the kernel hands out (`hakios.gpu.cpp`: `kMsmGemNew`), `IOSurface`-backed | a program writes a buffer and reads it back | phone |
+| 4 | **A library and a compute pipeline**: `newLibraryWithData:` takes a `.metallib`, the function goes through A, the kernel runs | the CPU reference in `tools/air` and then the phone | phone |
+| 5 | **Command queue, command buffer, compute encoder, commit and wait**, on fences | a compute kernel dispatched through the Metal API gives the numbers `tools/air/cpuref.py` expects | phone |
+| 6 | **Render pipeline and render command encoder** for the three vertex and 271 fragment shaders A already handles | a triangle through Metal | phone |
+| 7 | **Present**: a drawable, and the frame reaching the screen without being dropped | the frame counters (below) | phone |
+
+Without a phone, steps 1 and 2 are all that can be done and checked. Steps 3 to 7 each need the GPU.
+
+## Step 1: what is in the tree
+
+- `xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp`: with boot-arg `metal=1` the `IOAcceleratorES` service gets the
+ properties `MetalPluginName` = `HakiosMetal` and `MetalPluginClassName` = `HakiosMetalDevice`. Without the argument
+ nothing changes: **the properties stay off by default, on purpose**. With them a program that asks for Metal gets a
+ device, and CoreAnimation would pick Metal for the screen; until that device can draw, the screen without it is the
+ working one.
+- `userland/src/HakiosMetal.m`, `userland/src/HakiosMetal.plist`: the bundle, built into `/redwood/lib/HakiosMetal.bundle` and
+ linked at `/System/Library/Extensions/HakiosMetal.bundle` (`userland/build.sh`).
+- `userland/src/metal_test.m` and `userland/rc.jobs.metal`: the check that already existed.
+
+## What steps 1 and 2 found (QEMU, 27 October 2026, `metal=1`, a clean image)
+
+Run in a loop of rebuild-image, boot, read the log; each pass gave the next requirement:
+
+1. **Metal found the service, loaded the bundle and made the class**: `HakiosMetal: the class HakiosMetalDevice is registered`
+ in `backboardd` and `mediaserverd`, then `a device for the accelerator port N (made)`: `-initWithAcceleratorPort:` was
+ called and `_MTLDevice`'s own `-init` returned an object. So the discovery chain of `docs/ru/metal.md` section 1 is as described.
+2. `MTLAddDevice` then **asserted that the class conforms to the protocol `MTLDevice`**, and, once it did, **to the private
+ protocol `MTLDeviceSPI`** (neither is declared by `_MTLDevice` itself, so the class declares them).
+3. With both declared, the first messages sent to the device, in this order: **`-featureProfile`**, **`-maxBufferLength`**,
+ and then `backboardd` tries to **create a command queue** (`[Metal] Failed to create MTLCommandQueue`) and exits. The
+ window server restarts and fails the same way (launchd: "service has crashed N times in a row"). That is what a Metal
+ device that exists and cannot do anything does to the system, and the reason `metal=1` is off by default.
+
+So the road from here is step 3 onwards, and the first thing the window server needs is a **command queue** (step 5), with
+buffers and textures under it; none of that can be tried without the GPU.
+
+## Frame skips
+
+A dropped frame is not a Metal problem, and Metal would not cure it: the screen of this port is drawn today by
+CoreAnimation's OpenGL ES renderer through Mesa (`gpu=1`), and its pace depends on the submit path, the fences and the
+present. What to measure first, on the phone, is already counted: `gGPUStat` in `drivers/gpu/hakios.gpu.cpp` (submissions,
+waits, what each cost, every five seconds). Whether the framebuffer stand-in (`MiniIOMobileFramebuffer.cpp`) counts
+frames and drops is not known to me: it records when the first frame was shown and keeps a small copy of frames
+(`record()`), and a counter of presented and dropped frames would be the first thing to add. A frame skip that is not seen
+in those numbers is in the compositor, not the driver. This needs a stable GPU run on the phone first; the
+`gpu=1` hang being bisected (see `docs/driver-review/README.md`) comes before any of it.
+
+## What is not claimed
+
+Nothing here draws. No Metal client has used this device for anything: the first one that tried (the window server) asked
+for a command queue and stopped. Everything above was seen in QEMU; none of it was run on the phone.
diff --git a/userland/build.sh b/userland/build.sh
index 91d7756..f5ac08b 100755
--- a/userland/build.sh
+++ b/userland/build.sh
@@ -232,6 +232,8 @@ if [ -d $IOS/rootfs/usr/lib ]; then
-o obj/libhakios.media.dylib src/inject_media.c src/camera_device.c src/camera_isp.c
# the output device of the audio server: a plug-in for CoreAudio's HAL
$DYN -bundle -framework CoreFoundation -o obj/HakiosAudio src/audio_plugin.c
+ # the plug-in Metal loads when the kernel names it (boot-arg metal=1): a device that exists, src/HakiosMetal.m
+ $DYN -bundle -framework Foundation -framework Metal -lobjc -o obj/HakiosMetal src/HakiosMetal.m
if [ -n "$GL" ]; then
# the OpenGL ES library, and the table of functions that hands CoreAnimation's calls to it
# (Mesa says beside the library what it was built from: one from other patches is built again)
@@ -287,6 +289,11 @@ if [ -d $IOS/rootfs/usr/lib ]; then
cp obj/HakiosAudio root/lib/HakiosAudio.driver/
cp src/audio_plugin.plist root/lib/HakiosAudio.driver/Info.plist
fi
+ if [ -f obj/HakiosMetal ]; then
+ mkdir -p root/lib/HakiosMetal.bundle
+ cp obj/HakiosMetal root/lib/HakiosMetal.bundle/
+ cp src/HakiosMetal.plist root/lib/HakiosMetal.bundle/Info.plist
+ fi
if [ -n "$GL" ]; then
cp obj/libOSMesa.8.dylib root/lib/
cp obj/gl_test root/bin/
@@ -605,6 +612,9 @@ if [ -f $IOS/system.manifest ] && [ -f $CACHE/dyld_shared_cache_arm64 ] &&
# our audio device, where the audio server looks for such plug-ins (and for no others: Apple's are left out)
mkdir -p sysroot/System/Library/Audio/Plug-Ins/HAL
ln -s /redwood/lib/HakiosAudio.driver sysroot/System/Library/Audio/Plug-Ins/HAL/HakiosAudio.driver
+ # the plug-in Metal opens when the accelerator names it (boot-arg metal=1; without it nothing names this bundle)
+ mkdir -p sysroot/System/Library/Extensions
+ [ -d root/lib/HakiosMetal.bundle ] && ln -sfn /redwood/lib/HakiosMetal.bundle sysroot/System/Library/Extensions/HakiosMetal.bundle
# the OS cryptex: on a device a separate volume grafted at /System/Cryptexes/OS. It holds the dyld
# shared cache (dyld looks there first) and a few libraries and resources outside the cache.
mkdir -p sysroot/System/Cryptexes
diff --git a/userland/src/HakiosMetal.m b/userland/src/HakiosMetal.m
new file mode 100644
index 0000000..f57157d
--- /dev/null
+++ b/userland/src/HakiosMetal.m
@@ -0,0 +1,169 @@
+/*
+ * HakiosMetal: the plug-in Metal loads when the kernel names it (boot-arg metal=1), STEP ONE: a device that exists.
+ *
+ * What Metal does (docs/ru/metal.md section 1): it finds the IOKit service of class IOAcceleratorES, reads its
+ * properties MetalPluginName ("HakiosMetal") and MetalPluginClassName ("HakiosMetalDevice"), opens
+ * /System/Library/Extensions/HakiosMetal.bundle, takes that class, insists that it is a subclass of Metal's private
+ * _MTLDevice, and sends it -initWithAcceleratorPort:. That object is the system's one device.
+ *
+ * What this is, and is not. It is the first link of that chain and nothing more: HakiosMetalDevice is a subclass of
+ * _MTLDevice that answers -initWithAcceleratorPort: and -name. It cannot make a buffer, a library, a pipeline or a command
+ * queue yet, and says so: a message it does not know is logged ("HakiosMetal: not implemented: -newBufferWithLength:...")
+ * and answers nil/0/NO instead of crashing, so that the question "what does iOS ask a device first, and in what order?" can
+ * be read off the log rather than guessed. The compiler for the shaders is a separate piece and is largely done
+ * (docs/air.md: Apple's shader bitcode into Mesa's NIR); the command queue and the kernel side are not started (docs/metal-plan.md).
+ *
+ * _MTLDevice is not in the SDK's Metal.tbd (it lives in the system's own copy of the framework), so the class is made at run
+ * time, in the constructor that runs when the bundle is loaded, by objc_allocateClassPair on the _MTLDevice that is there.
+ * If there is none, nothing is registered, and Metal drops this plug-in as it does any that is not a _MTLDevice.
+ *
+ * Status: written without having been run. Whether _MTLDevice's own -init is safe to call, what Metal asks of the device right
+ * after it is made, and whether -name is enough for MTLCreateSystemDefaultDevice() to hand it out, are what the first run (QEMU,
+ * userland/rc.jobs.metal with metal=1) is for.
+ */
+#import <Foundation/Foundation.h>
+#import <Metal/Metal.h>
+#import <objc/message.h>
+#import <objc/runtime.h>
+
+static Class gBase; /* Metal's private _MTLDevice */
+static unsigned int gPort; /* the IOAcceleratorES service this device was made for */
+
+static void
+say(NSString * text)
+{
+ NSLog(@"HakiosMetal: %@", text);
+}
+
+/* -initWithAcceleratorPort: the fifth step of how Metal finds a device. */
+static id
+deviceInit(id self, SEL _cmd, unsigned int port)
+{
+ struct objc_super sup = { self, gBase };
+ id (*send)(struct objc_super *, SEL) = (id (*)(struct objc_super *, SEL))objc_msgSendSuper;
+
+ self = send(&sup, @selector(init));
+ gPort = port;
+ say([NSString stringWithFormat:@"a device for the accelerator port %u (%@)", port, self ? @"made" : @"_MTLDevice's init gave nil"]);
+ return self;
+}
+
+static id
+deviceName(__unused id self, __unused SEL _cmd)
+{
+ return @"HakiOS Adreno 642L (Metal: a device that exists, nothing more)";
+}
+
+static uint64_t
+deviceRegistryID(__unused id self, __unused SEL _cmd)
+{
+ return gPort;
+}
+
+/* the protocols the class declares (the ones Metal checks for), for the signatures below */
+static Protocol * gProtocols[4];
+static unsigned gProtocolCount;
+
+/*
+ * A message the class does not have, but a protocol it declares does: Metal sends the whole of MTLDevice and the private
+ * MTLDeviceSPI, and a method that is missing would end the process ("unrecognized selector": -featureProfile was the first).
+ * The type of the answer is read from the protocol's own description of the method, so that zero comes back in the right
+ * size, and the message is logged once. What a method should answer is learned one message at a time from this log.
+ */
+static id
+deviceSignature(id self, SEL _cmd, SEL selector)
+{
+ for (unsigned i = 0; i < gProtocolCount; i++) {
+ for (int required = 1; required >= 0; required--) {
+ struct objc_method_description d = protocol_getMethodDescription(gProtocols[i], selector, required, YES);
+
+ if (d.name != NULL && d.types != NULL) {
+ id (*make)(Class, SEL, const char *) = (id (*)(Class, SEL, const char *))objc_msgSend;
+
+ return make(objc_getClass("NSMethodSignature"), sel_registerName("signatureWithObjCTypes:"), d.types);
+ }
+ }
+ }
+ struct objc_super sup = { self, gBase };
+ id (*send)(struct objc_super *, SEL, SEL) = (id (*)(struct objc_super *, SEL, SEL))objc_msgSendSuper;
+
+ (void)_cmd;
+ return send(&sup, sel_registerName("methodSignatureForSelector:"), selector);
+}
+
+static void
+deviceForward(id self, SEL _cmd, id invocation)
+{
+ static NSMutableSet * seen;
+ SEL selector = ((SEL (*)(id, SEL))objc_msgSend)(invocation, sel_registerName("selector"));
+ id signature = ((id (*)(id, SEL))objc_msgSend)(invocation, sel_registerName("methodSignature"));
+ NSUInteger length = ((NSUInteger (*)(id, SEL))objc_msgSend)(signature, sel_registerName("methodReturnLength"));
+ NSString * name = NSStringFromSelector(selector);
+
+ (void)self;
+ (void)_cmd;
+ @synchronized(gBase) {
+ if (seen == nil) {
+ seen = [NSMutableSet new];
+ }
+ if (![seen containsObject:name]) {
+ [seen addObject:name];
+ say([NSString stringWithFormat:@"asked and not implemented: -%@ (returns %lu bytes; answered with zero)", name, (unsigned long)length]);
+ }
+ }
+ if (length != 0) {
+ void * zero = calloc(1, length);
+
+ ((void (*)(id, SEL, void *))objc_msgSend)(invocation, sel_registerName("setReturnValue:"), zero);
+ free(zero);
+ }
+}
+
+__attribute__((constructor)) static void
+registerDevice(void)
+{
+ Class device;
+
+ gBase = objc_getClass("_MTLDevice");
+ if (gBase == Nil) {
+ say(@"this system has no _MTLDevice: no device class is registered");
+ return;
+ }
+ if (objc_getClass("HakiosMetalDevice") != Nil) {
+ return;
+ }
+ device = objc_allocateClassPair(gBase, "HakiosMetalDevice", 0);
+ if (device == Nil) {
+ say(@"could not make the class HakiosMetalDevice");
+ return;
+ }
+ /*
+ * The first run said it: MTLAddDevice asserts [newDevice conformsToProtocol:@protocol(MTLDevice)] and aborts the process
+ * (backboardd) when the class does not declare it. _MTLDevice itself does not, so the class does.
+ */
+ class_addProtocol(device, @protocol(MTLDevice));
+ gProtocols[gProtocolCount++] = @protocol(MTLDevice);
+ /*
+ * ... and the second run said that MTLAddDevice asserts the private protocol MTLDeviceSPI as well (line 835). Private
+ * protocols are not in the SDK, so they are taken by name from the system's own Metal; each next assertion of this kind
+ * that a run shows is a name for this list.
+ */
+ static const char * const privateProtocols[] = { "MTLDeviceSPI", NULL };
+ for (const char * const * name = privateProtocols; *name != NULL; name++) {
+ Protocol * protocol = objc_getProtocol(*name);
+
+ if (protocol != NULL && gProtocolCount < sizeof(gProtocols) / sizeof(gProtocols[0])) {
+ class_addProtocol(device, protocol);
+ gProtocols[gProtocolCount++] = protocol;
+ } else {
+ say([NSString stringWithFormat:@"the protocol %s is not in this system's Metal", *name]);
+ }
+ }
+ class_addMethod(device, @selector(initWithAcceleratorPort:), (IMP)deviceInit, "@@:I");
+ class_addMethod(device, @selector(name), (IMP)deviceName, "@@:");
+ class_addMethod(device, @selector(registryID), (IMP)deviceRegistryID, "Q@:");
+ class_addMethod(device, sel_registerName("methodSignatureForSelector:"), (IMP)deviceSignature, "@@::");
+ class_addMethod(device, sel_registerName("forwardInvocation:"), (IMP)deviceForward, "v@:@");
+ objc_registerClassPair(device);
+ say(@"the class HakiosMetalDevice is registered (a subclass of _MTLDevice)");
+}
diff --git a/userland/src/HakiosMetal.plist b/userland/src/HakiosMetal.plist
new file mode 100644
index 0000000..0d56648
--- /dev/null
+++ b/userland/src/HakiosMetal.plist
@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+ <key>CFBundleIdentifier</key>
+ <string>com.hakios.metal</string>
+ <key>CFBundleName</key>
+ <string>HakiosMetal</string>
+ <key>CFBundleExecutable</key>
+ <string>HakiosMetal</string>
+ <key>CFBundlePackageType</key>
+ <string>BNDL</string>
+ <key>CFBundleInfoDictionaryVersion</key>
+ <string>6.0</string>
+ <key>CFBundleVersion</key>
+ <string>1</string>
+ <key>CFBundleSupportedPlatforms</key>
+ <array>
+ <string>iPhoneOS</string>
+ </array>
+ <!-- the class Metal takes (it also names it in MetalPluginClassName); made when the bundle is loaded, src/HakiosMetal.m -->
+ <key>NSPrincipalClass</key>
+ <string>HakiosMetalDevice</string>
+</dict>
+</plist>
diff --git a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
index 46f4c59..5a165d8 100644
--- a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
+++ b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
@@ -1541,7 +1541,20 @@ publishAccelerator(IOService * provider)
IOAcceleratorES * accelerator = OSTypeAlloc(IOAcceleratorES);
if (accelerator != NULL && accelerator->init() && accelerator->attach(provider)) {
+ uint32_t metal = 0;
+
accelerator->setName("sgx");
+ /*
+ * Boot-arg metal=1: say where Metal's device is (docs/metal-plan.md, step 1). Metal reads these two
+ * properties, opens /System/Library/Extensions/<name>.bundle and takes the class (userland/src/HakiosMetal.m).
+ * Never by default: with them a program that asks for Metal gets a device, and CoreAnimation would choose
+ * Metal for the screen; until that device can draw, a screen drawn without it is the working one.
+ */
+ if (PE_parse_boot_argn("metal", &metal, sizeof(metal)) && metal != 0) {
+ accelerator->setProperty("MetalPluginName", "HakiosMetal");
+ accelerator->setProperty("MetalPluginClassName", "HakiosMetalDevice");
+ IOLog("MiniIOMobileFramebuffer: metal=1: the accelerator names HakiosMetal as its Metal plug-in\n");
+ }
accelerator->registerService();
}
OSSafeReleaseNULL(accelerator);