Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

Several QEMUs at once, a phone-sized guest, parallel workspaces

vcvkkc8b5f43fb9dc
diff --git a/README.md b/README.md
index 4ec2385..1dadb03 100644
--- a/README.md
+++ b/README.md
@@ -37,7 +37,7 @@
образ системы. **Файлов iOS в репозитории нет** — скрипт сам скачивает их с серверов Apple (из IPSW iOS 16.5
для iPhone X) и раскладывает.
-Нужно: Linux x86-64, ~12 ГБ свободного места (готовое занимает ~8 ГБ), 16 ГБ ОЗУ. Пакеты (Arch):
+Нужно: Linux x86-64, ~12 ГБ свободного места (готовое занимает ~8 ГБ), от 4 ГБ свободной памяти. Пакеты (Arch):
```sh
sudo pacman -S --needed git python 7zip lz4 gcc qemu-system-aarch64 qemu-ui-gtk edk2-aarch64
@@ -69,9 +69,11 @@ qemu/run-ios.sh 1200 -display gtk > logs/qemu.log # сеанс на 20 ми
Замечания:
-- Память гостя — 12 ГБ, отображённые из разрежённого файла `qemu/ram.img` (образ системы лежит в нём по нужному
- смещению). На btrfs/xfs это reflink и места не занимает, на ext4 образ копируется (ещё 2,5 ГБ). Если QEMU не
- может выделить память — `sudo sysctl vm.overcommit_memory=1`.
+- Память гостя — 7 ГБ (ядру достаётся то же окно 4,2 ГБ, что и на телефоне), отображённые из разрежённого файла
+ `qemu/ram.img` (образ системы лежит в нём по нужному смещению). На btrfs/xfs это reflink и места не занимает,
+ на ext4 образ копируется (ещё 2,5 ГБ). Один запущенный QEMU занимает 2,5–3 ГБ памяти компьютера.
+- Несколько QEMU сразу: `QEMU_INSTANCE=1 qemu/run-ios.sh …` (у каждого номера свои порты; `touch.py`,
+ `screendump.sh`, `stop.sh` понимают ту же переменную). `qemu/stop.sh` завершает QEMU досрочно.
- Прошивка UEFI для aarch64 ищется там, куда её кладёт Arch. В Debian/Ubuntu (пакет `qemu-efi-aarch64`):
`QEMU_EFI=/usr/share/AAVMF/AAVMF_CODE.fd QEMU_VARS=/usr/share/AAVMF/AAVMF_VARS.fd qemu/run-ios.sh …`
(проверялось только на Arch).
@@ -212,7 +214,7 @@ Userland не изменён ни в одном байте: всё, чего е
| `mkpackfs.py` | сборщик образов packfs (формат описан в `bsd/miscfs/packfs/packfs.h` ядра) |
| `qemu/` | запуск в QEMU, касания, снимки экрана, разбор стеков |
| `uefi/`, `mkxnuimg.py` | порт UEFI на redwood и упаковка загрузочного образа для `fastboot boot` |
-| `tools/` | `recording.py` (запись экрана → mp4), `update-prebuilt.sh` |
+| `tools/` | `recording.py` (запись экрана → mp4), `update-prebuilt.sh`, `new-workspace.sh` (вторая рабочая копия со своим ядром и QEMU) |
| `prebuilt/` | собранные ядро, загрузчик, наши программы и куски UEFI |
Скрипты не зависят от того, куда склонирован репозиторий.
diff --git a/qemu/run-ios.sh b/qemu/run-ios.sh
index 849d51c..5c88a7a 100755
--- a/qemu/run-ios.sh
+++ b/qemu/run-ios.sh
@@ -4,6 +4,12 @@
# same arrangement `xl ramdisk` + `xl run` make on the phone.
# usage: run-ios.sh [seconds] [extra qemu args...]
# env: ROOT=<small image>, IMG=<system image>, CMDLINE=<extra boot-args>
+# QEMU_RAM_GB=<n> the guest's RAM, default 7: memory then ends where the phone's does and the
+# kernel gets the same 4.2 GB window (the system image included) as on the phone.
+# The host needs about 2.5 GB per running instance.
+# QEMU_INSTANCE=<n> to run several at once: each number has its own monitor and QMP ports
+# (4444 + 10n, 4445 + 10n); the file qemu/instance sets it for a working copy.
+# touch.py, screendump.sh and stop.sh follow the same rule.
# The machine has a display (ramfb; without one SpringBoard does not start): the loader sets a mode of the
# size of the phone's screen. qemu/screendump.sh saves a picture of it (through the monitor on port 4444).
# Add "-display gtk" to see it in a window and use it with the mouse: the pointer with the left button
@@ -20,10 +26,19 @@ IMG=${IMG:-userland/system.packfs}
ROOT=${ROOT:-userland/root.packfs}
[ -f "$IMG" ] && [ -f "$ROOT" ] || { echo "run-ios.sh: $IMG or $ROOT is missing (userland/build.sh makes them)"; exit 1; }
SECS=${1:-60}; shift
-RAM_BASE=0x40000000 RD_ADDR=0xFC000000 RAM_SIZE=$((12 << 30))
+RAM_BASE=0x40000000 RD_ADDR=0xFC000000
+RAM_GB=${QEMU_RAM_GB:-7}
+RAM_SIZE=$((RAM_GB << 30)) FILE_SIZE=$((12 << 30))
+INSTANCE=${QEMU_INSTANCE:-$(cat qemu/instance 2>/dev/null || echo 0)}
+# an instance takes 2.5-3 GB of host memory; starting one that does not fit gets something killed
+AVAILABLE=$(awk '/^MemAvailable:/ { print int($2 / 1024) }' /proc/meminfo)
+if [ "${AVAILABLE:-9999}" -lt 3000 ] && [ -z "$QEMU_FORCE" ]; then
+ echo "run-ios.sh: only $AVAILABLE MB of memory available on this machine: stop another QEMU first (QEMU_FORCE=1 to start anyway)"
+ exit 1
+fi
if [ ! -f qemu/ram.img ] || [ "$IMG" -nt qemu/ram.img ]; then
- python3 - "$IMG" qemu/ram.img $((RD_ADDR - RAM_BASE)) $RAM_SIZE <<'PY' || exit 1
+ python3 - "$IMG" qemu/ram.img $((RD_ADDR - RAM_BASE)) $FILE_SIZE <<'PY' || exit 1
import os, sys
src = os.open(sys.argv[1], os.O_RDONLY)
dst = os.open(sys.argv[2], os.O_RDWR | os.O_CREAT | os.O_TRUNC, 0o644)
@@ -36,8 +51,9 @@ PY
fi
xnuloader/mkpayload.py -o qemu/payload.bin kernel=$K ramdisk=$ROOT --rdres "$(printf '%x' "$(stat -c %s "$IMG")")" \
--cmdline "loader_return_after=$SECS epoch=$(date +%s) virtio_input=1 $CMDLINE" >/dev/null || exit 1
-QEMU_MEM=12G exec qemu/run.sh $((SECS + 40)) \
+QEMU_MEM=${RAM_GB}G exec qemu/run.sh $((SECS + 40)) \
-object memory-backend-file,id=ram,size=$RAM_SIZE,mem-path="$PWD/qemu/ram.img",share=off \
-machine memory-backend=ram \
-global virtio-mmio.force-legacy=false -device virtio-tablet-device -device virtio-keyboard-device \
- -device ramfb -monitor tcp:127.0.0.1:4444,server,nowait -qmp tcp:127.0.0.1:4445,server,nowait "$@"
+ -device ramfb -monitor tcp:127.0.0.1:$((4444 + 10 * INSTANCE)),server,nowait \
+ -qmp tcp:127.0.0.1:$((4445 + 10 * INSTANCE)),server,nowait "$@"
diff --git a/qemu/run.sh b/qemu/run.sh
index 969fd65..70939ef 100755
--- a/qemu/run.sh
+++ b/qemu/run.sh
@@ -1,7 +1,9 @@
#!/bin/bash
# Boot the XNU loader + kernel payload under QEMU (virt, GICv3, UEFI at EL1).
# usage: run.sh [seconds] [extra qemu args...]
-# env: QEMU_EFI, QEMU_VARS the UEFI firmware for aarch64 guests (default: where Arch's edk2-aarch64
+# env: QEMU_TB_MB megabytes for QEMU's cache of translated code (default 384; QEMU's own
+# default of 1 GB mostly costs host memory)
+# QEMU_EFI, QEMU_VARS the UEFI firmware for aarch64 guests (default: where Arch's edk2-aarch64
# puts it; Debian/Ubuntu: /usr/share/AAVMF/AAVMF_CODE.fd and AAVMF_VARS.fd)
cd "$(dirname "$0")"
T=${1:-20}; shift
@@ -13,6 +15,7 @@ cp "${QEMU_VARS:-/usr/share/edk2/aarch64/QEMU_VARS.fd}" vars.fd 2>/dev/null
L=../xnuloader/loader.efi; [ -f $L ] || L=../prebuilt/loader.efi
mkdir -p esp/EFI/BOOT && cp -u $L esp/EFI/BOOT/BOOTAA64.EFI
timeout "$T" qemu-system-aarch64 -M virt,gic-version=3 -cpu cortex-a76 -smp 1 -m ${QEMU_MEM:-6G} \
+ -accel tcg,tb-size=${QEMU_TB_MB:-384} \
-drive if=pflash,format=raw,readonly=on,file="$EFI" \
-drive if=pflash,format=raw,file=vars.fd \
-drive file=fat:rw:esp,format=raw,if=virtio \
diff --git a/qemu/screendump.sh b/qemu/screendump.sh
index f2cc36d..ab4a438 100755
--- a/qemu/screendump.sh
+++ b/qemu/screendump.sh
@@ -3,10 +3,11 @@
# usage: screendump.sh [out.png] (FULL=1: the whole mode, not only the phone-sized corner)
out=$(realpath -m "${1:-$(dirname "$(readlink -f "$0")")/../logs/qemu-screen.png}") # QEMU writes it: absolute
mkdir -p "$(dirname "$out")"
-python3 - "$out" <<'PY'
+here=$(dirname "$(readlink -f "$0")")
+python3 - "$out" "${QEMU_INSTANCE:-$(cat "$here/instance" 2>/dev/null || echo 0)}" <<'PY'
import socket, struct, sys, time, zlib, os
out = sys.argv[1]; ppm = out + '.ppm'
-s = socket.create_connection(('127.0.0.1', 4444)); s.settimeout(2)
+s = socket.create_connection(('127.0.0.1', 4444 + 10 * int(sys.argv[2]))); s.settimeout(2)
s.sendall(b'\n'); time.sleep(0.3)
s.sendall(f'screendump {ppm}\n'.encode()); time.sleep(1.5)
d = open(ppm, 'rb').read(); os.unlink(ppm)
diff --git a/qemu/stop.sh b/qemu/stop.sh
new file mode 100755
index 0000000..9830f05
--- /dev/null
+++ b/qemu/stop.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Stop the QEMU of this working copy (and only it): "quit" on its monitor.
+# The instance is QEMU_INSTANCE, or the number in qemu/instance, or 0 (see run-ios.sh).
+here=$(dirname "$(readlink -f "$0")")
+port=$((4444 + 10 * ${QEMU_INSTANCE:-$(cat "$here/instance" 2>/dev/null || echo 0)}))
+python3 - $port <<'PY'
+import socket, sys, time
+try:
+ s = socket.create_connection(('127.0.0.1', int(sys.argv[1])), timeout=2)
+except OSError:
+ sys.exit(0) # nothing is running there
+s.sendall(b'\nquit\n'); time.sleep(0.5)
+PY
diff --git a/qemu/touch.py b/qemu/touch.py
index 2927b0e..58346ca 100755
--- a/qemu/touch.py
+++ b/qemu/touch.py
@@ -14,16 +14,26 @@ Positions are fractions of the screen: 0 0 is the top left corner, 1 1 the botto
Example, the "swipe up to open" gesture: touch.py swipe 0.5 0.98 0.5 0.4 250
"""
import json
+import os
import socket
import sys
import time
+
+def instance():
+ """Which QEMU: QEMU_INSTANCE, or the number in the file "instance" next to this script, or 0."""
+ try:
+ return int(os.environ.get('QEMU_INSTANCE') or
+ open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'instance')).read())
+ except (OSError, ValueError):
+ return 0
+
KEYS = {'lock': 'l', 'home': 'h', 'volup': 'up', 'voldown': 'down', 'fivefingers': 'f11'}
class Qmp:
- def __init__(self, port=4445):
- self.sock = socket.create_connection(('127.0.0.1', port))
+ def __init__(self, port=None):
+ self.sock = socket.create_connection(('127.0.0.1', port or 4445 + 10 * instance()))
self.file = self.sock.makefile('rw')
self.file.readline() # the greeting
self.command('qmp_capabilities')
diff --git a/qemu/where.py b/qemu/where.py
index e537608..4bf3ddb 100755
--- a/qemu/where.py
+++ b/qemu/where.py
@@ -1,8 +1,12 @@
#!/usr/bin/env python3
-"""Ask a running QEMU (monitor on tcp:127.0.0.1:4444) where the guest CPU is, a few times."""
+"""Ask a running QEMU (its monitor: port 4444, or 4444 + 10 * QEMU_INSTANCE) where the guest CPU is, a few times."""
import socket, time, re, subprocess, sys, os
n = int(sys.argv[1]) if len(sys.argv) > 1 else 3
-s = socket.create_connection(('127.0.0.1', 4444)); s.settimeout(2)
+try:
+ instance = int(os.environ.get('QEMU_INSTANCE') or open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'instance')).read())
+except (OSError, ValueError):
+ instance = 0
+s = socket.create_connection(('127.0.0.1', 4444 + 10 * instance)); s.settimeout(2)
def cmd(c):
s.sendall(c.encode() + b'\n'); time.sleep(0.4)
out = b''
diff --git a/tools/kernel-delta.sh b/tools/kernel-delta.sh
new file mode 100755
index 0000000..5e57931
--- /dev/null
+++ b/tools/kernel-delta.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# In a workspace made by new-workspace.sh: print, as a patch, what was changed in the kernel tree
+# since the workspace was made (new files included). Apply it elsewhere with "git apply".
+set -e
+cd "$(dirname "$(readlink -f "$0")")/../xnu-src/xnu-8796.121.2"
+git rev-parse -q --verify refs/redwood/baseline >/dev/null || { echo "kernel-delta.sh: not a workspace (no baseline)" >&2; exit 1; }
+export GIT_INDEX_FILE=$PWD/.git/index.delta
+rm -f "$GIT_INDEX_FILE"
+git read-tree refs/redwood/baseline
+git add -A . 2>/dev/null
+git diff --cached --binary refs/redwood/baseline
+rm -f "$GIT_INDEX_FILE"
diff --git a/tools/new-workspace.sh b/tools/new-workspace.sh
new file mode 100755
index 0000000..2b036fb
--- /dev/null
+++ b/tools/new-workspace.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+# A second working copy of everything, for working on something else at the same time: its own
+# branch of this repository (a git worktree), its own kernel tree and build, its own images and
+# its own QEMU instance. What is big is shared with this copy through reflinks and hard links
+# (btrfs or xfs), so a workspace costs about 0.5 GB, most of it the kernel's build output.
+#
+# usage: tools/new-workspace.sh NAME INSTANCE [DIR]
+# NAME the branch, and the directory ../darwin-redwood-NAME unless DIR is given
+# INSTANCE the QEMU instance number of the copy (1, 2, ...; see qemu/run-ios.sh)
+#
+# The kernel tree is copied as it is now, uncommitted work included, and that state is recorded
+# (refs/redwood/baseline in the copy's kernel tree): tools/kernel-delta.sh, run in the workspace,
+# prints what was changed in the kernel since. Files below ios165/rootfs and ios165/cryptex are
+# hard links to this copy's: replace them, never edit them in place.
+# The phone is not part of a workspace: there is one, and ./xl belongs to the main copy.
+set -e
+ROOT=$(cd "$(dirname "$(readlink -f "$0")")/.." && pwd)
+NAME=${1:?usage: new-workspace.sh NAME INSTANCE [DIR]}
+INSTANCE=${2:?usage: new-workspace.sh NAME INSTANCE [DIR]}
+WS=${3:-$(dirname "$ROOT")/darwin-redwood-$NAME}
+[ -e "$WS" ] && { echo "new-workspace.sh: $WS already exists"; exit 1; }
+
+git -C "$ROOT" worktree add -q -b "$NAME" "$WS"
+cd "$WS"
+
+# the kernel tree, without build output (it is full of absolute paths)
+mkdir -p xnu-src
+cp -a --reflink=auto "$ROOT/xnu-src/xnu-8796.121.2" xnu-src/
+K=xnu-src/xnu-8796.121.2
+rm -rf $K/BUILD
+(
+ cd $K
+ export GIT_INDEX_FILE=$PWD/.git/index.baseline
+ git add -A . 2>/dev/null
+ git update-ref refs/redwood/baseline "$(git commit-tree "$(git write-tree)" -p HEAD -m 'the state the workspace started from')"
+ rm -f "$GIT_INDEX_FILE"
+)
+# what the kernel build needs besides the tree
+cp -a --reflink=auto "$ROOT/xnu-deps/." xnu-deps/
+# the files of iOS and the system image made of them
+for d in rootfs cryptex "20F66__iPhone10,3_6"; do cp -al "$ROOT/ios165/$d" ios165/; done
+for f in system.files system.manifest var.skel .done-rootfs .done-cryptex a2s.json; do
+ [ -e "$ROOT/ios165/$f" ] && cp -a "$ROOT/ios165/$f" ios165/
+done
+for f in rootfs.dmg rootfs.slt extracted; do
+ [ -e "$ROOT/ios165/$f" ] && ln -s "$ROOT/ios165/$f" ios165/
+done
+cp -a --reflink=auto "$ROOT/userland/obj" "$ROOT/userland/system.packfs" "$ROOT/userland/root.packfs" "$ROOT/userland/hello" userland/
+[ -f "$ROOT/xnuloader/loader.efi" ] && cp -a "$ROOT/xnuloader/loader.efi" xnuloader/
+[ -f "$ROOT/qemu/kernel.syms" ] && cp -a "$ROOT/qemu/kernel.syms" qemu/
+echo "$INSTANCE" > qemu/instance
+mkdir -p logs
+
+echo "new-workspace.sh: building the kernel in $WS"
+./xnu-build.sh -j"$(nproc)" -k > xnu-build.log 2>&1 || true
+[ -f $K/BUILD/obj/DEVELOPMENT_ARM64_BCM2837/mach.development.bcm2837 ] ||
+ { echo "new-workspace.sh: the kernel did not build, see $WS/xnu-build.log"; exit 1; }
+echo "new-workspace.sh: $WS is ready (branch $NAME, QEMU instance $INSTANCE)"
diff --git a/userland/mkjobs.py b/userland/mkjobs.py
index c583938..ae4648e 100755
--- a/userland/mkjobs.py
+++ b/userland/mkjobs.py
@@ -11,7 +11,8 @@ everything the daemon has to say -- SpringBoard alone says a megabyte in half a
costs time and pushes everybody else out of the log that comes back from the phone.
The program itself must be in the system image; a warning is printed if it was not extracted.
-With --rest, the other jobs of iOS follow, but strictly on demand: whatever starts them by
+With --rest, the other jobs of iOS follow (their standard error on the console as well, their os_log
+only when named in OSLOG), but strictly on demand: whatever starts them by
itself (RunAtLoad, KeepAlive) is taken out, so that launchd runs one only when a program asks
for one of its Mach services. That is how an application finds the daemons it needs without
all 490 of them being started at boot. Left out: jobs without Mach services (nothing could
@@ -66,6 +67,9 @@ if rest:
for key in ('RunAtLoad', 'KeepAlive', 'StartInterval', 'StartCalendarInterval', 'WatchPaths',
'QueueDirectories', 'StartOnMount', 'LaunchEvents'):
job.pop(key, None)
+ job['StandardErrorPath'] = '/dev/console'
+ if label in oslog:
+ job.setdefault('EnvironmentVariables', {})['OS_ACTIVITY_DT_MODE'] = 'YES'
with open(f'{dest}/{name}', 'wb') as f:
plistlib.dump(job, f, fmt=plistlib.FMT_BINARY)
count += 1