Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.
C
Metal, step one: a device that exists (metal=1), and what Metal asks of it first
shimoku455af7eae22c
diff --git a/docs/metal-plan.md b/docs/metal-plan.mdnew file mode 100644index 0000000..7e4d468--- /dev/null+++ b/docs/metal-plan.md@@ -0,0 +1,74 @@+# Metal on the Adreno: the plan, and where it stands++Read `docs/ru/metal.md` (why Metal is not simply missing, and what is on each side of the wall) and `docs/air.md` (the+shader compiler) first. This file is the order of work, with how each step is checked and where.++## The three parts++| Part | What it is | State |+|---|---|---|+| **A. The shader compiler** | Apple's shader bitcode (AIR, in a `.metallib`) into Mesa's NIR, so that ir3 (the compiler behind the OpenGL ES that already runs) makes Adreno code | largely done: 321 of the system's 456 shader modules reach ir3, 54 have run on the phone's GPU (`docs/air.md`, `docs/air-report.md`) |+| **B. The device** | what Metal loads: a bundle with a subclass of Metal's private `_MTLDevice`, and the objects it hands out (buffers, libraries, pipeline states, queues, command buffers, encoders: 53 classes and about 1200 methods in Apple's own) | step 1 is written (below); nothing past it |+| **C. The kernel side** | submitting work to the GPU for a Metal client, with memory and fences | the GPU already has a user client for Mesa (`drivers/gpu/hakios.gpu.cpp`, `HakiosGPUUserClient`); it is not connected to B |++What "real Metal" would be, and why games are out of reach, is argued in `docs/ru/metal.md` section 9.4: that has not+changed. What this plan aims at is the part that is within reach: **Metal clients of the system itself** (the window+server's compositor, CoreImage, the camera's viewfinder) running on the Adreno through B and C, with shaders from A.++## The order, and how each step is checked++| # | Step | Checked by | Where |+|---|---|---|---|+| 1 | **A device that exists.** `metal=1` makes the (so far empty) `IOAcceleratorES` service name `HakiosMetal`; the bundle makes `HakiosMetalDevice`, a `_MTLDevice` subclass; `MTLCreateSystemDefaultDevice()` returns it | `metal_test` (`RC_JOBS=rc.jobs.metal`) prints the device and `HakiosMetal:` lines say which messages Metal sent that the device did not answer | QEMU, no GPU needed |+| 2 | **Learn what Metal asks first**, from step 1's "not implemented" lines: properties, family support, queue, buffer | the log, in order | QEMU |+| 3 | **Buffers and textures** over the GPU memory the kernel hands out (`hakios.gpu.cpp`: `kMsmGemNew`), `IOSurface`-backed | a program writes a buffer and reads it back | phone |+| 4 | **A library and a compute pipeline**: `newLibraryWithData:` takes a `.metallib`, the function goes through A, the kernel runs | the CPU reference in `tools/air` and then the phone | phone |+| 5 | **Command queue, command buffer, compute encoder, commit and wait**, on fences | a compute kernel dispatched through the Metal API gives the numbers `tools/air/cpuref.py` expects | phone |+| 6 | **Render pipeline and render command encoder** for the three vertex and 271 fragment shaders A already handles | a triangle through Metal | phone |+| 7 | **Present**: a drawable, and the frame reaching the screen without being dropped | the frame counters (below) | phone |++Without a phone, steps 1 and 2 are all that can be done and checked. Steps 3 to 7 each need the GPU.++## Step 1: what is in the tree++- `xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp`: with boot-arg `metal=1` the `IOAcceleratorES` service gets the+ properties `MetalPluginName` = `HakiosMetal` and `MetalPluginClassName` = `HakiosMetalDevice`. Without the argument+ nothing changes: **the properties stay off by default, on purpose**. With them a program that asks for Metal gets a+ device, and CoreAnimation would pick Metal for the screen; until that device can draw, the screen without it is the+ working one.+- `userland/src/HakiosMetal.m`, `userland/src/HakiosMetal.plist`: the bundle, built into `/redwood/lib/HakiosMetal.bundle` and+ linked at `/System/Library/Extensions/HakiosMetal.bundle` (`userland/build.sh`).+- `userland/src/metal_test.m` and `userland/rc.jobs.metal`: the check that already existed.++## What steps 1 and 2 found (QEMU, 27 October 2026, `metal=1`, a clean image)++Run in a loop of rebuild-image, boot, read the log; each pass gave the next requirement:++1. **Metal found the service, loaded the bundle and made the class**: `HakiosMetal: the class HakiosMetalDevice is registered`+ in `backboardd` and `mediaserverd`, then `a device for the accelerator port N (made)`: `-initWithAcceleratorPort:` was+ called and `_MTLDevice`'s own `-init` returned an object. So the discovery chain of `docs/ru/metal.md` section 1 is as described.+2. `MTLAddDevice` then **asserted that the class conforms to the protocol `MTLDevice`**, and, once it did, **to the private+ protocol `MTLDeviceSPI`** (neither is declared by `_MTLDevice` itself, so the class declares them).+3. With both declared, the first messages sent to the device, in this order: **`-featureProfile`**, **`-maxBufferLength`**,+ and then `backboardd` tries to **create a command queue** (`[Metal] Failed to create MTLCommandQueue`) and exits. The+ window server restarts and fails the same way (launchd: "service has crashed N times in a row"). That is what a Metal+ device that exists and cannot do anything does to the system, and the reason `metal=1` is off by default.++So the road from here is step 3 onwards, and the first thing the window server needs is a **command queue** (step 5), with+buffers and textures under it; none of that can be tried without the GPU.++## Frame skips++A dropped frame is not a Metal problem, and Metal would not cure it: the screen of this port is drawn today by+CoreAnimation's OpenGL ES renderer through Mesa (`gpu=1`), and its pace depends on the submit path, the fences and the+present. What to measure first, on the phone, is already counted: `gGPUStat` in `drivers/gpu/hakios.gpu.cpp` (submissions,+waits, what each cost, every five seconds). Whether the framebuffer stand-in (`MiniIOMobileFramebuffer.cpp`) counts+frames and drops is not known to me: it records when the first frame was shown and keeps a small copy of frames+(`record()`), and a counter of presented and dropped frames would be the first thing to add. A frame skip that is not seen+in those numbers is in the compositor, not the driver. This needs a stable GPU run on the phone first; the+`gpu=1` hang being bisected (see `docs/driver-review/README.md`) comes before any of it.++## What is not claimed++Nothing here draws. No Metal client has used this device for anything: the first one that tried (the window server) asked+for a command queue and stopped. Everything above was seen in QEMU; none of it was run on the phone.diff --git a/userland/build.sh b/userland/build.shindex 91d7756..f5ac08b 100755--- a/userland/build.sh+++ b/userland/build.sh@@ -232,6 +232,8 @@ if [ -d $IOS/rootfs/usr/lib ]; then-o obj/libhakios.media.dylib src/inject_media.c src/camera_device.c src/camera_isp.c# the output device of the audio server: a plug-in for CoreAudio's HAL$DYN -bundle -framework CoreFoundation -o obj/HakiosAudio src/audio_plugin.c+ # the plug-in Metal loads when the kernel names it (boot-arg metal=1): a device that exists, src/HakiosMetal.m+ $DYN -bundle -framework Foundation -framework Metal -lobjc -o obj/HakiosMetal src/HakiosMetal.mif [ -n "$GL" ]; then# the OpenGL ES library, and the table of functions that hands CoreAnimation's calls to it# (Mesa says beside the library what it was built from: one from other patches is built again)@@ -287,6 +289,11 @@ if [ -d $IOS/rootfs/usr/lib ]; thencp obj/HakiosAudio root/lib/HakiosAudio.driver/cp src/audio_plugin.plist root/lib/HakiosAudio.driver/Info.plistfi+ if [ -f obj/HakiosMetal ]; then+ mkdir -p root/lib/HakiosMetal.bundle+ cp obj/HakiosMetal root/lib/HakiosMetal.bundle/+ cp src/HakiosMetal.plist root/lib/HakiosMetal.bundle/Info.plist+ fiif [ -n "$GL" ]; thencp obj/libOSMesa.8.dylib root/lib/cp obj/gl_test root/bin/@@ -605,6 +612,9 @@ if [ -f $IOS/system.manifest ] && [ -f $CACHE/dyld_shared_cache_arm64 ] &&# our audio device, where the audio server looks for such plug-ins (and for no others: Apple's are left out)mkdir -p sysroot/System/Library/Audio/Plug-Ins/HALln -s /redwood/lib/HakiosAudio.driver sysroot/System/Library/Audio/Plug-Ins/HAL/HakiosAudio.driver+ # the plug-in Metal opens when the accelerator names it (boot-arg metal=1; without it nothing names this bundle)+ mkdir -p sysroot/System/Library/Extensions+ [ -d root/lib/HakiosMetal.bundle ] && ln -sfn /redwood/lib/HakiosMetal.bundle sysroot/System/Library/Extensions/HakiosMetal.bundle# the OS cryptex: on a device a separate volume grafted at /System/Cryptexes/OS. It holds the dyld# shared cache (dyld looks there first) and a few libraries and resources outside the cache.mkdir -p sysroot/System/Cryptexesdiff --git a/userland/src/HakiosMetal.m b/userland/src/HakiosMetal.mnew file mode 100644index 0000000..f57157d--- /dev/null+++ b/userland/src/HakiosMetal.m@@ -0,0 +1,169 @@+/*+ * HakiosMetal: the plug-in Metal loads when the kernel names it (boot-arg metal=1), STEP ONE: a device that exists.+ *+ * What Metal does (docs/ru/metal.md section 1): it finds the IOKit service of class IOAcceleratorES, reads its+ * properties MetalPluginName ("HakiosMetal") and MetalPluginClassName ("HakiosMetalDevice"), opens+ * /System/Library/Extensions/HakiosMetal.bundle, takes that class, insists that it is a subclass of Metal's private+ * _MTLDevice, and sends it -initWithAcceleratorPort:. That object is the system's one device.+ *+ * What this is, and is not. It is the first link of that chain and nothing more: HakiosMetalDevice is a subclass of+ * _MTLDevice that answers -initWithAcceleratorPort: and -name. It cannot make a buffer, a library, a pipeline or a command+ * queue yet, and says so: a message it does not know is logged ("HakiosMetal: not implemented: -newBufferWithLength:...")+ * and answers nil/0/NO instead of crashing, so that the question "what does iOS ask a device first, and in what order?" can+ * be read off the log rather than guessed. The compiler for the shaders is a separate piece and is largely done+ * (docs/air.md: Apple's shader bitcode into Mesa's NIR); the command queue and the kernel side are not started (docs/metal-plan.md).+ *+ * _MTLDevice is not in the SDK's Metal.tbd (it lives in the system's own copy of the framework), so the class is made at run+ * time, in the constructor that runs when the bundle is loaded, by objc_allocateClassPair on the _MTLDevice that is there.+ * If there is none, nothing is registered, and Metal drops this plug-in as it does any that is not a _MTLDevice.+ *+ * Status: written without having been run. Whether _MTLDevice's own -init is safe to call, what Metal asks of the device right+ * after it is made, and whether -name is enough for MTLCreateSystemDefaultDevice() to hand it out, are what the first run (QEMU,+ * userland/rc.jobs.metal with metal=1) is for.+ */+#import <Foundation/Foundation.h>+#import <Metal/Metal.h>+#import <objc/message.h>+#import <objc/runtime.h>++static Class gBase; /* Metal's private _MTLDevice */+static unsigned int gPort; /* the IOAcceleratorES service this device was made for */++static void+say(NSString * text)+{+ NSLog(@"HakiosMetal: %@", text);+}++/* -initWithAcceleratorPort: the fifth step of how Metal finds a device. */+static id+deviceInit(id self, SEL _cmd, unsigned int port)+{+ struct objc_super sup = { self, gBase };+ id (*send)(struct objc_super *, SEL) = (id (*)(struct objc_super *, SEL))objc_msgSendSuper;++ self = send(&sup, @selector(init));+ gPort = port;+ say([NSString stringWithFormat:@"a device for the accelerator port %u (%@)", port, self ? @"made" : @"_MTLDevice's init gave nil"]);+ return self;+}++static id+deviceName(__unused id self, __unused SEL _cmd)+{+ return @"HakiOS Adreno 642L (Metal: a device that exists, nothing more)";+}++static uint64_t+deviceRegistryID(__unused id self, __unused SEL _cmd)+{+ return gPort;+}++/* the protocols the class declares (the ones Metal checks for), for the signatures below */+static Protocol * gProtocols[4];+static unsigned gProtocolCount;++/*+ * A message the class does not have, but a protocol it declares does: Metal sends the whole of MTLDevice and the private+ * MTLDeviceSPI, and a method that is missing would end the process ("unrecognized selector": -featureProfile was the first).+ * The type of the answer is read from the protocol's own description of the method, so that zero comes back in the right+ * size, and the message is logged once. What a method should answer is learned one message at a time from this log.+ */+static id+deviceSignature(id self, SEL _cmd, SEL selector)+{+ for (unsigned i = 0; i < gProtocolCount; i++) {+ for (int required = 1; required >= 0; required--) {+ struct objc_method_description d = protocol_getMethodDescription(gProtocols[i], selector, required, YES);++ if (d.name != NULL && d.types != NULL) {+ id (*make)(Class, SEL, const char *) = (id (*)(Class, SEL, const char *))objc_msgSend;++ return make(objc_getClass("NSMethodSignature"), sel_registerName("signatureWithObjCTypes:"), d.types);+ }+ }+ }+ struct objc_super sup = { self, gBase };+ id (*send)(struct objc_super *, SEL, SEL) = (id (*)(struct objc_super *, SEL, SEL))objc_msgSendSuper;++ (void)_cmd;+ return send(&sup, sel_registerName("methodSignatureForSelector:"), selector);+}++static void+deviceForward(id self, SEL _cmd, id invocation)+{+ static NSMutableSet * seen;+ SEL selector = ((SEL (*)(id, SEL))objc_msgSend)(invocation, sel_registerName("selector"));+ id signature = ((id (*)(id, SEL))objc_msgSend)(invocation, sel_registerName("methodSignature"));+ NSUInteger length = ((NSUInteger (*)(id, SEL))objc_msgSend)(signature, sel_registerName("methodReturnLength"));+ NSString * name = NSStringFromSelector(selector);++ (void)self;+ (void)_cmd;+ @synchronized(gBase) {+ if (seen == nil) {+ seen = [NSMutableSet new];+ }+ if (![seen containsObject:name]) {+ [seen addObject:name];+ say([NSString stringWithFormat:@"asked and not implemented: -%@ (returns %lu bytes; answered with zero)", name, (unsigned long)length]);+ }+ }+ if (length != 0) {+ void * zero = calloc(1, length);++ ((void (*)(id, SEL, void *))objc_msgSend)(invocation, sel_registerName("setReturnValue:"), zero);+ free(zero);+ }+}++__attribute__((constructor)) static void+registerDevice(void)+{+ Class device;++ gBase = objc_getClass("_MTLDevice");+ if (gBase == Nil) {+ say(@"this system has no _MTLDevice: no device class is registered");+ return;+ }+ if (objc_getClass("HakiosMetalDevice") != Nil) {+ return;+ }+ device = objc_allocateClassPair(gBase, "HakiosMetalDevice", 0);+ if (device == Nil) {+ say(@"could not make the class HakiosMetalDevice");+ return;+ }+ /*+ * The first run said it: MTLAddDevice asserts [newDevice conformsToProtocol:@protocol(MTLDevice)] and aborts the process+ * (backboardd) when the class does not declare it. _MTLDevice itself does not, so the class does.+ */+ class_addProtocol(device, @protocol(MTLDevice));+ gProtocols[gProtocolCount++] = @protocol(MTLDevice);+ /*+ * ... and the second run said that MTLAddDevice asserts the private protocol MTLDeviceSPI as well (line 835). Private+ * protocols are not in the SDK, so they are taken by name from the system's own Metal; each next assertion of this kind+ * that a run shows is a name for this list.+ */+ static const char * const privateProtocols[] = { "MTLDeviceSPI", NULL };+ for (const char * const * name = privateProtocols; *name != NULL; name++) {+ Protocol * protocol = objc_getProtocol(*name);++ if (protocol != NULL && gProtocolCount < sizeof(gProtocols) / sizeof(gProtocols[0])) {+ class_addProtocol(device, protocol);+ gProtocols[gProtocolCount++] = protocol;+ } else {+ say([NSString stringWithFormat:@"the protocol %s is not in this system's Metal", *name]);+ }+ }+ class_addMethod(device, @selector(initWithAcceleratorPort:), (IMP)deviceInit, "@@:I");+ class_addMethod(device, @selector(name), (IMP)deviceName, "@@:");+ class_addMethod(device, @selector(registryID), (IMP)deviceRegistryID, "Q@:");+ class_addMethod(device, sel_registerName("methodSignatureForSelector:"), (IMP)deviceSignature, "@@::");+ class_addMethod(device, sel_registerName("forwardInvocation:"), (IMP)deviceForward, "v@:@");+ objc_registerClassPair(device);+ say(@"the class HakiosMetalDevice is registered (a subclass of _MTLDevice)");+}diff --git a/userland/src/HakiosMetal.plist b/userland/src/HakiosMetal.plistnew file mode 100644index 0000000..0d56648--- /dev/null+++ b/userland/src/HakiosMetal.plist@@ -0,0 +1,25 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>CFBundleIdentifier</key>+ <string>com.hakios.metal</string>+ <key>CFBundleName</key>+ <string>HakiosMetal</string>+ <key>CFBundleExecutable</key>+ <string>HakiosMetal</string>+ <key>CFBundlePackageType</key>+ <string>BNDL</string>+ <key>CFBundleInfoDictionaryVersion</key>+ <string>6.0</string>+ <key>CFBundleVersion</key>+ <string>1</string>+ <key>CFBundleSupportedPlatforms</key>+ <array>+ <string>iPhoneOS</string>+ </array>+ <!-- the class Metal takes (it also names it in MetalPluginClassName); made when the bundle is loaded, src/HakiosMetal.m -->+ <key>NSPrincipalClass</key>+ <string>HakiosMetalDevice</string>+</dict>+</plist>diff --git a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cppindex 46f4c59..5a165d8 100644--- a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp+++ b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp@@ -1541,7 +1541,20 @@ publishAccelerator(IOService * provider)IOAcceleratorES * accelerator = OSTypeAlloc(IOAcceleratorES);if (accelerator != NULL && accelerator->init() && accelerator->attach(provider)) {+ uint32_t metal = 0;+accelerator->setName("sgx");+ /*+ * Boot-arg metal=1: say where Metal's device is (docs/metal-plan.md, step 1). Metal reads these two+ * properties, opens /System/Library/Extensions/<name>.bundle and takes the class (userland/src/HakiosMetal.m).+ * Never by default: with them a program that asks for Metal gets a device, and CoreAnimation would choose+ * Metal for the screen; until that device can draw, a screen drawn without it is the working one.+ */+ if (PE_parse_boot_argn("metal", &metal, sizeof(metal)) && metal != 0) {+ accelerator->setProperty("MetalPluginName", "HakiosMetal");+ accelerator->setProperty("MetalPluginClassName", "HakiosMetalDevice");+ IOLog("MiniIOMobileFramebuffer: metal=1: the accelerator names HakiosMetal as its Metal plug-in\n");+ }accelerator->registerService();}OSSafeReleaseNULL(accelerator);