Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.
C
sysimage/unpack.sh: the IPSW is the user's own; no script downloads Apple's files any more
shimoku7a397f799b05
diff --git a/.gitignore b/.gitignoreindex 9f5664a..51680ca 100644--- a/.gitignore+++ b/.gitignore@@ -63,7 +63,7 @@ __pycache__/!/userland/var//sysimage/*-!/sysimage/fetch.sh+!/sysimage/unpack.sh!/sysimage/*.py# of xnu-deps only what is ours; setup.sh fetches and builds the restdiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 87ca5b1..74128f5 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -4,8 +4,8 @@kernel, loader and user space stay device-neutral and take their numbers from the device treeand the phone's own Android.- Nothing of Apple's in the tree: no iOS files, images, caches, screenshots or headers derived- from the firmware. The build fetches what it needs from the user's own IPSW- (`sysimage/fetch.sh`, `xnuloader/mkidentity.sh`). CI refuses pictures and iOS files.+ from the firmware. The build takes what it needs from the IPSW the user got+ themselves (`sysimage/unpack.sh`, `xnuloader/mkidentity.sh`); no script downloads Apple's files. CI refuses pictures and iOS files.- Nothing of the vendor's either: firmware is pulled from the user's phone (`make firmware`).- Kernel changes are made in `xnu-src/` and exported with `xnu/export.sh` before committing;the patch must apply to Apple's `xnu-8796.121.2` tag (CI checks).diff --git a/LEGAL.md b/LEGAL.mdindex 45e4422..57ed30f 100644--- a/LEGAL.md+++ b/LEGAL.md@@ -56,9 +56,9 @@ program that can write to flash, and CI fails if one appears that is not listed.## What this repository does NOT contain and never distributes- **No part of iOS**: no iOS binary, image, cache, font, picture or screenshot. The user-space- half of the system image is built on your machine from the iOS 16.5 IPSW, which- `sysimage/fetch.sh` downloads from Apple's public servers under Apple's license, to your- machine only. Nothing of it is committed here.+ half of the system image is built on your machine from the iOS 16.5 IPSW, which you+ get yourself, under Apple's license, and hand to `sysimage/unpack.sh` on your machine; no script+ here downloads it. Nothing of it is committed here.- **No device firmware**: the GPU, Wi-Fi, audio DSP, modem, haptics and camera firmware is thevendor's (Qualcomm/Xiaomi). It is pulled from your own phone (`make firmware`) and is git-ignored.- **No keys, certificates, activation data, or anything to impersonate a genuine Apple device todiff --git a/Makefile b/Makefileindex 3d05fce..d44af8f 100644--- a/Makefile+++ b/Makefile@@ -2,7 +2,7 @@# below it; run the script itself for its options.## make help-# make fetch the OS files from the iOS 16.5 IPSW into sysimage/ (sysimage/fetch.sh)+# make unpack IPSW=F the OS files out of your own iOS 16.5 IPSW into sysimage/ (sysimage/unpack.sh)# make images userland/root.apfs and userland/system.apfs (userland/build.sh)# make kernel the kernel from source (xnu/apply.sh, xnu-deps/setup.sh, xnu-build.sh)# make loader the boot loader, with identity.h made from your IPSW (xnuloader/build.sh)@@ -50,7 +50,7 @@ IMAGE_OPTS ?= REST=1ESP_DIR ?= build/espPAYLOAD ?= qemu/payload.bin-.PHONY: devices help fetch images kernel loader loader-open hello qemu qemu-open phone prebuilt clean all doctor defconfig config savedefconfig esp+.PHONY: devices help unpack images kernel loader loader-open hello qemu qemu-open phone prebuilt clean all doctor defconfig config savedefconfig esphelp:@awk '{ if (!/^#/) exit; sub(/^# ?/, ""); print }' Makefile@@ -60,8 +60,8 @@ doctor:all: kernel loader images-fetch:- sysimage/fetch.sh+unpack:+ sysimage/unpack.sh $(IPSW)images:$(IMAGE_OPTS) userland/build.shdiff --git a/NOTICE b/NOTICEindex 9bb9970..ca2335c 100644--- a/NOTICE+++ b/NOTICE@@ -22,8 +22,8 @@ In the repository:Fetched at build time, never committed:-- the iOS 16.5 IPSW (Apple): the user space of the system image; sysimage/fetch.sh downloads it from- Apple's public servers; its license is Apple's.+- the iOS 16.5 IPSW (Apple): the user space of the system image; the user gets it and+ sysimage/unpack.sh unpacks it on their machine; nothing downloads it; its license is Apple's.- the iPhoneOS 16.5 SDK from github.com/theos/sdks, and theos's cctools (Apple's open-source cctools): thekernel's and user space's build tools.- the phone's firmware (GPU, Wi-Fi, audio DSP, modem, haptics, cameras): the vendor's, pulled from thediff --git a/README.md b/README.mdindex 4c794d0..b4ea8c2 100644--- a/README.md+++ b/README.md@@ -62,9 +62,8 @@ make phone SECS=600 # a 10-minute session on the phone, in RAM| Repository | What ||---|---|| [hakios](https://github.com/oosexgang/hakios) | **common**: the kernel port, every common driver, the loader, user space, the tools, this documentation, and `devices/qemu` |-| [hakios-redwood](https://github.com/oosexgang/hakios-redwood) | POCO X5 Pro 5G: its defconfig, status, UEFI platform, firmware script, and its SoC's layer (`soc/`: the Qualcomm SC7280 family) |+| [hakios-sm7325](https://github.com/oosexgang/hakios-sm7325) | POCO X5 Pro 5G: its defconfig, status, UEFI platform, firmware script, and its SoC's layer (`soc/`: the Qualcomm SC7280 family) || [hakios-uefi](https://github.com/oosexgang/hakios-uefi) | HakiOS's UEFI: Mu-Silicium with HakiOS's boot path, banner and name |-| [hakios-web](https://github.com/oosexgang/hakios-web) | the website, with this documentation and the FAQ |`devices/manifest.conf` in the common repository names each one and the revision it is at; `tools/hakios-device.py sync` clones theminto `devices/` and `uefi/`. One repository per device: it carries its SoC's layer, and `tools/hakios-device.py new` makes one. [How the repositories fit](docs/porting/repositories.md).@@ -85,8 +84,8 @@ In it: only our own code and data: the patch to Apple's open-source XNU (`xnu/haloader, our programs and libraries, the tools, `prebuilt/` (built from this source; the loader **without** any Apple data:`make loader` makes the one that starts iOS, on your machine, from your own IPSW), and the device and SoC directories.-Never in it: any file of iOS (you build the system image on your machine from the IPSW, which `sysimage/fetch.sh` downloads from-Apple's public servers without decrypting or circumventing anything); the phone's firmware (pulled from your own phone into+Never in it: any file of iOS (you build the system image on your machine from an IPSW you get yourself and hand to+`sysimage/unpack.sh`; nothing is downloaded for you, decrypted or circumvented); the phone's firmware (pulled from your own phone into`fw-*/`, which git ignores); keys, certificates, activation, or anything that would make the phone pass as an iPhone to Apple.## Independent, and Apple-free at the corediff --git a/README.uk.md b/README.uk.mdindex 8e27e68..c2fd560 100644--- a/README.uk.md+++ b/README.uk.md@@ -46,8 +46,8 @@ HakiOS — це адаптація ядра Darwin (відкритий XNU віЧого немає і не буде:- жодного файлу iOS: ні бінарників, ні образів, ні кешів, ні шрифтів, ні картинок, ні знімків- екрана. Образ системи ви збираєте на своїй машині з IPSW iOS 16.5, який `sysimage/fetch.sh`- завантажує з публічних серверів Apple, нічого не розшифровуючи й не обходячи;+ екрана. Образ системи ви збираєте на своїй машині з IPSW iOS 16.5, який ви берете самі й+ передаєте `sysimage/unpack.sh`; за вас нічого не завантажується, не розшифровується й не обходиться;- прошивок телефона (GPU, Wi-Fi, аудіо-DSP, модем, вібро, камери): вони вендорські і знімаютьсяз вашого власного телефона в `fw-*/`, які git ігнорує;- ключів, сертифікатів, активації чи будь-чого, що видавало б телефон за iPhone перед службами@@ -79,7 +79,7 @@ Linux x86-64, близько 12 ГБ на диску, 7-Zip 23 або новішgit clone https://github.com/oosexgang/hakios.git hakioscd hakiosmake doctor # чого бракує цьому комп'ютеру (дисплей QEMU, UEFI, пам'ять) і що робити-make fetch # IPSW iOS 16.5 із серверів Apple, потрібне розпаковується (7,4 ГБ)+make unpack IPSW=~/ваш.ipsw # ваш власний IPSW iOS 16.5, потрібне розпаковується (7,4 ГБ)make images # образи APFS: userland/root.apfs і userland/system.apfsmake loader # завантажувач з ідентичністю iPhone з ВАШОЇ копії IPSW (потрібні clang і lld)make qemu # сеанс на 20 хвилинdiff --git a/README.zh.md b/README.zh.mdindex 57560cb..bb25f74 100644--- a/README.zh.md+++ b/README.zh.md@@ -39,7 +39,7 @@ Linux 电脑上的 QEMU(`virt`,aarch64)里运行,不需要任何 Apple永远不会有的东西:- 任何 iOS 文件:没有二进制、镜像、缓存、字体、图片或截图。系统镜像由你在自己机器上用 iOS 16.5- 的 IPSW 构建,`sysimage/fetch.sh` 从 Apple 的公开服务器为你下载它,不解密、不绕过任何东西;+ 的 IPSW 构建,IPSW 由你自己获取并交给 `sysimage/unpack.sh`;项目不替你下载、不解密、不绕过任何东西;- 手机的固件(GPU、Wi-Fi、音频 DSP、调制解调器、振动、摄像头):属于厂商,从你自己的手机取到`fw-*/`,git 忽略这些目录;- 密钥、证书、激活,或任何让手机在 Apple 服务面前冒充 iPhone 的东西。这里无法登录 Apple 账户,@@ -71,7 +71,7 @@ Linux x86-64,约 12 GB 磁盘,7-Zip 23 或更新,`lz4`,`qemu-system-aarcgit clone https://github.com/oosexgang/hakios.git hakioscd hakiosmake doctor # 检查本机缺什么(QEMU 显示设备、UEFI 固件、内存)以及怎么办-make fetch # 从 Apple 服务器取 iOS 16.5 IPSW,解包所需部分(7.4 GB)+make unpack IPSW=~/your.ipsw # 你自己获取的 iOS 16.5 IPSW,解包所需部分(7.4 GB)make images # APFS 镜像:userland/root.apfs 和 userland/system.apfsmake loader # 引导程序,iPhone 的标识取自你自己的 IPSW 副本(需要 clang 和 lld)make qemu # 20 分钟的会话diff --git a/THIRD_PARTY_LICENSES.md b/THIRD_PARTY_LICENSES.mdindex f723705..66ab459 100644--- a/THIRD_PARTY_LICENSES.md+++ b/THIRD_PARTY_LICENSES.md@@ -27,7 +27,7 @@ This supersedes the summary in `NOTICE`, which stays as the short, canonical lis| Component | Fetched by | Origin | License ||---|---|---|---|-| iOS 16.5 IPSW (user space of the system image) | `sysimage/fetch.sh` | Apple | Apple's — user supplies, user's own copy |+| iOS 16.5 IPSW (user space of the system image) | `sysimage/unpack.sh` | Apple | Apple's — user supplies, user's own copy || iPhoneOS 16.5 SDK | `xnu-deps/setup.sh` (github.com/theos/sdks) | Apple (headers) via theos | Apple's / theos || Apple open sources (AvailabilityVersions, libdispatch, libpthread, cctools, bootstrap_cmds) | `xnu-deps/setup.sh` | Apple | APSL 2.0 || MacOSX13.3 SDK (Kernel.framework DriverKit headers only) | `xnu-deps/setup.sh` | Apple via joseluisq/macosx-sdks | Apple's |diff --git a/docs/getting-started/building.md b/docs/getting-started/building.mdindex 8dbfcb0..e95c827 100644--- a/docs/getting-started/building.md+++ b/docs/getting-started/building.md@@ -42,7 +42,7 @@ make loader # xnuloader/build.sh: clang and lld-link, an EFI applicatio```The header `xnuloader/identity.h`, the properties by which iOS tells which iPhone it is, is not kept in the repository:-`xnuloader/mkidentity.sh` makes it at build time from the device tree in your own IPSW. `make loader-open` makes the placeholder+`xnuloader/mkidentity.sh` makes it at build time from the device tree in your own IPSW (`sysimage/unpack.sh` leaves it in `sysimage/`). `make loader-open` makes the placeholderloader that boots the open kernel only.## User space and the imagesdiff --git a/docs/getting-started/qemu.md b/docs/getting-started/qemu.mdindex f73a13c..25dc8c8 100644--- a/docs/getting-started/qemu.md+++ b/docs/getting-started/qemu.md@@ -22,7 +22,7 @@ Linux x86-64, about 12 GB of disk, 7-Zip 23 or newer, `lz4`, `qemu-system-aarch6```shmake doctor # QEMU's display device, UEFI firmware, memory: what is missing, and what to do-sysimage/fetch.sh # the iOS 16.5 IPSW from Apple's servers, the needed parts unpacked (7.4 GB)+sysimage/unpack.sh ~/iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw # YOUR IPSW: the needed parts unpacked (7.4 GB)REST=1 userland/build.sh # the APFS images: userland/root.apfs and userland/system.apfsmake loader # the boot loader, with the iPhone's identity taken from YOUR copy of the IPSWmkdir -p logsdiff --git a/docs/legal-review.md b/docs/legal-review.mdindex 8f31457..5573cac 100644--- a/docs/legal-review.md+++ b/docs/legal-review.md@@ -42,7 +42,7 @@ gate anything that contains Apple's code behind it.**### What the build does that touches Apple's code (the exposure that is yours, not the user's)-- `sysimage/fetch.sh` downloads parts of the IPSW from Apple's public CDN and `userland/build.sh`+- `sysimage/unpack.sh` unpacks parts of an IPSW the user supplies (nothing is downloaded) and `userland/build.sh`assembles a modified root image: it selects parts of iOS, replaces launchd with `stage0`, injectsour dylibs into SpringBoard and other Apple processes (`userland/src/inject*.c`), and`xnuloader/mkidentity.py` copies the iPhone's device-tree identity so the system "believes it isdiff --git a/docs/port-pkg110.md b/docs/port-pkg110.mdindex b47951d..97a9e97 100644--- a/docs/port-pkg110.md+++ b/docs/port-pkg110.md@@ -81,9 +81,9 @@ the boot test itself: `./xl enter` does a `fastboot boot`, which a truly lockedfrom `/fw` of the system image, which `MODEM=1 userland/build.sh` fills from `fw-modem/`(`hakios.mpss.c:1045-1110`). The modem's own file systems (modemst1/2, fsc, fsg) do come from thephone's UFS (`ufs=2`), but in addition to the image, not instead of it. So the project's standard- path applies: `sysimage/fetch.sh` (iOS 16.5 from Apple's public CDN, ~7.4 GB) and+ path applies: `sysimage/unpack.sh YOUR.ipsw` (iOS 16.5, which you supply; ~7.4 GB unpacked) and`PREBUILT=1 REST=1 MODEM=1 SYSTEM=1 userland/build.sh`.-- **Build in WSL's own ext4, not on `/mnt/d`.** `fetch.sh` relies on 7z leaving files named+- **Build in WSL's own ext4, not on `/mnt/d`.** `unpack.sh` relies on 7z leaving files named`NAME:com.apple.decmpfs` (finished by `unzbm`); `:` is not a legal NTFS file-name character, anddrvfs is slow on tens of thousands of small files. A local clone in `~/hakios` plus a copy of`fw-*` is used; the finished `root.apfs`/`system.apfs` are copied back to the D: checkout for `xl`.@@ -104,7 +104,7 @@ the boot test itself: `./xl enter` does a `fastboot boot`, which a truly lockedIn WSL's ext4 (`~/hakios`, a local clone plus `fw-*`):-- `sysimage/fetch.sh`: ~9 min. `done: 2.9G rootfs, 26M cryptex, 11M App cryptex, 2.6G dyld shared+- `sysimage/unpack.sh`: ~9 min. `done: 2.9G rootfs, 26M cryptex, 11M App cryptex, 2.6G dyld sharedcache` (5.7 GB in `sysimage/`); `unzbm` built by `cc`, the LZBITMAP files finished, PingFangtrimmed 74 -> 26 MB.- `PREBUILT=1 REST=1 MODEM=1 SYSTEM=1 userland/build.sh`: ~4 min, exit 0. The small image gotdiff --git a/docs/reference/tools.md b/docs/reference/tools.mdindex 102da0a..92489b2 100644--- a/docs/reference/tools.md+++ b/docs/reference/tools.md@@ -8,7 +8,7 @@ Every `make` target is a script below it; run the script itself for its options|---|---|| `make` | the list below || `make doctor` | checks this machine (and a phone on `adb`) for what HakiOS needs |-| `make fetch` | the OS files from the iOS 16.5 IPSW into `sysimage/` |+| `make unpack IPSW=F` | the OS files out of your own iOS 16.5 IPSW into `sysimage/` (nothing is downloaded) || `make images` | `userland/root.apfs` and `userland/system.apfs` || `make kernel` | the kernel from source, with the `.config`'s drivers || `make loader` / `make loader-open` | the boot loader with your IPSW's identity / the placeholder that boots the open kernel only |diff --git a/docs/ru/notes.md b/docs/ru/notes.mdindex c0fac64..caff9ae 100644--- a/docs/ru/notes.md+++ b/docs/ru/notes.md@@ -42,8 +42,8 @@## Быстрый старт: QEMU, без сборки ядраВ `prebuilt/` лежат собранные из этого же репозитория ядро, загрузчик и наши программы, так что нужен только-образ системы. **Файлов iOS в репозитории нет** — скрипт сам скачивает их с серверов Apple (из IPSW iOS 16.5-для iPhone X) и раскладывает.+образ системы. **Файлов iOS в репозитории нет** — IPSW iOS 16.5 для iPhone X вы берёте сами,+скрипт только распаковывает из него нужное.Нужно: Linux x86-64, ~12 ГБ свободного места (готовое занимает ~8 ГБ), от 4 ГБ свободной памяти. Пакеты (Arch):@@ -56,7 +56,7 @@ sudo pacman -S --needed git python 7zip lz4 gcc qemu-system-aarch64 qemu-ui-gtk```shgit clone [email protected]:vcvkk/darwin-redwood.gitcd darwin-redwood-sysimage/fetch.sh # скачивает 7,4 ГБ с CDN Apple и распаковывает нужное (минут 5–15)+sysimage/unpack.sh ~/iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw # ваш IPSW: распаковывает нужноеREST=1 userland/build.sh # собирает образы APFS: userland/root.apfs и userland/system.apfs (2,5 ГБ)mkdir -p logsqemu/run-ios.sh 1200 -display gtk > logs/qemu.log # сеанс на 20 минут@@ -65,9 +65,8 @@ qemu/run-ios.sh 1200 -display gtk > logs/qemu.log # сеанс на 20 миЧерез 2–3 минуты в окне появится экран блокировки.**Сколько это занимает.** Измерено 20 октября на этой машине (Linux x86-64, 12 потоков, NVMe), проходом-по этим самым командам из чистого клона: `sysimage/fetch.sh` — 1 мин 45 с, из них скачивание криптексов-2,7 ГБ (образ корня, 4,7 ГБ, в этом прогоне уже лежал рядом; с нуля к этому времени добавится его-скачивание — на том же канале около трёх минут); `REST=1 userland/build.sh` — 1 мин 52 с; QEMU до+по этим самым командам из чистого клона: `sysimage/unpack.sh` — 1 мин 45 с+(тогда скрипт ещё сам качал образы; теперь IPSW вы кладёте сами, и остаётся только распаковка); `REST=1 userland/build.sh` — 1 мин 52 с; QEMU доэкрана блокировки — ещё 2–3 минуты. То есть около восьми минут от пустого каталога до iOS, если каналотдаёт порядка 25 МБ/с; сборка ядра и загрузчика в это не входит и не нужна — они в `prebuilt/`.@@ -197,7 +196,7 @@ codec DMA TX_CDC_DMA_TX_3 и звуковой процессор в поток,CoreGraphics в них не рисовал — плитки оставались нулями. Теперь загрузчик пишет в дерево sRGB, как и есть унашего кадрового буфера, и WebKit рисует в BGRA (`xnuloader/loader.c`, `dt_identity()`; проверка —`RC_JOBS=rc.jobs.safari`). Цветные эмодзи и китайский шрифт PingFang (130 и 74 МБ) целиком в образ системы не- помещаются, и их знаки выходили квадратиками с вопросом; теперь `sysimage/fetch.sh` кладёт их в образ уменьшенными+ помещаются, и их знаки выходили квадратиками с вопросом; теперь `sysimage/unpack.sh` кладёт их в образ уменьшенными(`tools/font_trim.py`): эмодзи — без самых крупных картинок (остаются 40, 64 и 96 точек; 59 МБ), PingFang — собычным и полужирным начертаниями упрощённого, традиционного и гонконгского письма (26 МБ). Проверка —`RC_JOBS=rc.jobs.fonts` (`font_test`: к какому шрифту система идёт за эмодзи, китайским, японским и корейским и@@ -234,7 +233,7 @@ codec DMA TX_CDC_DMA_TX_3 и звуковой процессор в поток,## На телефонеНужен именно **POCO X5 Pro 5G (redwood)** с разблокированным загрузчиком, `adb` и `fastboot` на компьютере-(`sudo pacman -S android-tools`), и образы из предыдущего раздела (`sysimage/fetch.sh`, `REST=1 userland/build.sh`).+(`sudo pacman -S android-tools`), и образы из предыдущего раздела (`sysimage/unpack.sh`, `REST=1 userland/build.sh`).**Прошивка самого телефона.** В репозитории её нет и быть не может: она вендорская и берётся с того самогоаппарата, на котором всё будет работать, из его Android (подробности — в разделах по каждому устройству):@@ -1710,7 +1709,7 @@ xnuloader/build.sh # clang + lld-link → xnuloader/loader.efi (EFI-пр````xnuloader/identity.h` генерируется при сборке `xnuloader/mkidentity.sh` (через `mkidentity.py`) из дерева устройств-iPhone X в вашем IPSW (так система узнаёт, «какой она iPhone»); в репозитории его нет. Вручную: `python3 sysimage/fetch-from-ipsw.py Firmware/all_flash/DeviceTree.d22ap.im4p dt.im4p`,+iPhone X в вашем IPSW (так система узнаёт, «какой она iPhone»); в репозитории его нет. Вручную: `python3 sysimage/ipsw-member.py ВАШ.ipsw Firmware/all_flash/DeviceTree.d22ap.im4p dt.im4p`,`ipsw img4 im4p extract dt.im4p -o dt.bin`, `xnuloader/mkidentity.py dt.bin > xnuloader/identity.h`. Кроме свойствкорня и `/product` оттуда же берётся узел `/product/haptics` (`closed-loop`, `supports-3rd-party-haptics`): по немуMobileGestalt отвечает `DeviceSupportsClosedLoopHaptics`, и iOS играет вибрацию и отклики волнами на устройстве@@ -3235,7 +3234,7 @@ iOS, которого она ждёт, — это уже другое: созд## Про файлы Apple и лицензии-- Файлов iOS в репозитории нет: `sysimage/fetch.sh` скачивает их из IPSW с CDN Apple на вашу машину.+- Файлов iOS в репозитории нет: IPSW вы берёте сами, `sysimage/unpack.sh` распаковывает его на вашей машине; ничего не скачивается.- `xnu/` — производная работа от XNU, распространяется на условиях APSL 2.0, как и сам XNU; `prebuilt/mach.*` —ядро, собранное из этих исходников.- `xnu/files/bsd/miscfs/apfs/libzbitmap/` — декодер LZBITMAP libzbitmap (Corellium LLC, GPL-2.0+ OR MIT; здесь —diff --git a/docs/windows.md b/docs/windows.mdindex 31234b7..0c9783c 100644--- a/docs/windows.md+++ b/docs/windows.md@@ -7,7 +7,7 @@ Everything here was done on a real machine and phone; `tools/doctor.sh` checks t| Job | Where | Why ||---|---|---|-| Build the images (`sysimage/fetch.sh`, `userland/build.sh`) | **WSL 2, in its own filesystem** (`~/hakios`) | `fetch.sh` makes files named `NAME:com.apple.decmpfs`; NTFS forbids `:`. The Windows drive is also slow for tens of thousands of small files. |+| Build the images (`sysimage/unpack.sh`, `userland/build.sh`) | **WSL 2, in its own filesystem** (`~/hakios`) | `unpack.sh` makes files named `NAME:com.apple.decmpfs`; NTFS forbids `:`. The Windows drive is also slow for tens of thousands of small files. || Drive the phone (`./xl`, `tools/modem-cycle-win.sh`) | **Git Bash**, in your Windows checkout | `fastboot` and `adb` are Windows programs that see the phone's USB; WSL 2 does not, without usbipd. |So: build in WSL, copy `userland/root.apfs` and `userland/system.apfs` to the Windows checkout, run `xl`@@ -41,7 +41,7 @@ from Git Bash.```sh# WSL, in ~/hakios (a clone in WSL's own filesystem, with fw-*/ copied in):-sysimage/fetch.sh # iOS 16.5 from Apple's servers, ~9 min+sysimage/unpack.sh ~/iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw # your own IPSW, ~5 minPREBUILT=1 REST=1 MODEM=1 SYSTEM=1 userland/build.sh # root.apfs + system.apfs, ~4 min, no theos neededcp userland/root.apfs userland/system.apfs /mnt/d/.../hakios/userland/diff --git a/sysimage/fetch-from-ipsw.py b/sysimage/fetch-from-ipsw.pydeleted file mode 100755index b5648f3..0000000--- a/sysimage/fetch-from-ipsw.py+++ /dev/null@@ -1,109 +0,0 @@-#!/usr/bin/env python3-"""Fetch one member of the iOS 16.5 iPhone X firmware archive without downloading the rest.--usage: fetch-from-ipsw.py MEMBER OUTPUT--The IPSW is a ZIP64 archive on Apple's CDN; its directory is read with HTTP range requests and-the member is streamed (and inflated, if it is compressed) straight to OUTPUT. A download that-breaks off is taken up again: a member stored without compression (the big disk images are)-continues where it stopped, also when OUTPUT is the partial file of an earlier run; a compressed-one starts over.-"""-import http.client-import os-import struct-import sys-import time-import urllib.request-import zlib--ATTEMPTS = 20--URL = ('https://updates.cdn-apple.com/2023SpringFCS/fullrestores/032-85130/6AB0D10A-8AF9-4F6C-8CF5-A1B315CABE88/'- 'iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw')-SIZE = 6165698198---def fetch(start, end):- request = urllib.request.Request(URL, headers={'Range': f'bytes={start}-{end}'})- return urllib.request.urlopen(request, timeout=120)---def directory():- tail = fetch(SIZE - 65536, SIZE - 1).read()- at = tail.rfind(b'PK\x06\x06')- cd_size, cd_offset = struct.unpack('<QQ', tail[at + 40:at + 56])- cd = fetch(cd_offset, cd_offset + cd_size - 1).read()- p = 0- while cd[p:p + 4] == b'PK\x01\x02':- method, = struct.unpack('<H', cd[p + 10:p + 12])- csize, usize, nl, el, cl = struct.unpack('<IIHHH', cd[p + 20:p + 34])- offset, = struct.unpack('<I', cd[p + 42:p + 46])- name = cd[p + 46:p + 46 + nl].decode()- extra = cd[p + 46 + nl:p + 46 + nl + el]- q = 0- while q + 4 <= len(extra):- tag, length = struct.unpack('<HH', extra[q:q + 4])- if tag == 1:- values = list(struct.unpack('<' + 'Q' * (length // 8), extra[q + 4:q + 4 + length // 8 * 8]))- if usize == 0xFFFFFFFF:- usize = values.pop(0)- if csize == 0xFFFFFFFF:- csize = values.pop(0)- if offset == 0xFFFFFFFF:- offset = values.pop(0)- q += 4 + length- yield name, method, csize, usize, offset- p += 46 + nl + el + cl---def download(data, method, csize, output):- """One attempt at the member whose bytes start at DATA in the archive; returns the size of OUTPUT."""- if method == 0:- have = os.path.getsize(output) if os.path.exists(output) else 0- if have > csize:- have = 0- if have < csize:- with fetch(data + have, data + csize - 1) as stream, open(output, 'ab' if have else 'wb') as out:- while chunk := stream.read(1 << 20):- out.write(chunk)- return os.path.getsize(output)- inflater = zlib.decompressobj(-15)- written = 0- with fetch(data, data + csize - 1) as stream, open(output, 'wb') as out:- while chunk := stream.read(1 << 20):- chunk = inflater.decompress(chunk)- out.write(chunk)- written += len(chunk)- tail = inflater.flush()- out.write(tail)- return written + len(tail)---def main():- member, output = sys.argv[1], sys.argv[2]- for name, method, csize, usize, offset in directory():- if name != member:- continue- if method not in (0, 8):- sys.exit(f'{member}: compression method {method} is not supported')- header = fetch(offset, offset + 29).read()- nl, el = struct.unpack('<HH', header[26:30])- data = offset + 30 + nl + el- written = -1- for attempt in range(1, ATTEMPTS + 1):- try:- written = download(data, method, csize, output)- except (OSError, http.client.HTTPException, zlib.error) as error:- print(f'{member}: {error}', file=sys.stderr)- if written == usize:- break- print(f'{member}: the download broke off, trying again ({attempt} of {ATTEMPTS})', file=sys.stderr)- time.sleep(5)- print(f'{member}: {written} bytes (expected {usize})')- sys.exit(0 if written == usize else 1)- sys.exit(f'{member}: not in the archive')---if __name__ == '__main__':- main()diff --git a/sysimage/ipsw-member.py b/sysimage/ipsw-member.pynew file mode 100755index 0000000..069bbc7--- /dev/null+++ b/sysimage/ipsw-member.py@@ -0,0 +1,35 @@+#!/usr/bin/env python3+"""Copy one member out of an IPSW the user has on their own disk.++usage: ipsw-member.py IPSW MEMBER OUTPUT++An IPSW is a ZIP64 archive; Python's zipfile reads it as it is. The member is streamed to OUTPUT+(inflated if it is compressed), so the 4.7 GB root filesystem image never has to fit in memory.+Nothing is downloaded: the project ships no file of Apple's and fetches none. The IPSW is the+user's, got by the user from wherever they get their firmware.+"""+import shutil+import sys+import zipfile+++def main():+ if len(sys.argv) != 4:+ sys.exit(__doc__.split('\n\n')[1])+ ipsw, member, output = sys.argv[1:]+ try:+ archive = zipfile.ZipFile(ipsw)+ except (OSError, zipfile.BadZipFile) as error:+ sys.exit(f'ipsw-member.py: {ipsw}: {error}')+ with archive:+ try:+ info = archive.getinfo(member)+ except KeyError:+ sys.exit(f'ipsw-member.py: {member} is not in {ipsw} (is it the iOS 16.5 20F66 IPSW for iPhone10,3/iPhone10,6?)')+ with archive.open(info) as stream, open(output, 'wb') as out:+ shutil.copyfileobj(stream, out, 1 << 20)+ print(f'{member}: {info.file_size} bytes')+++if __name__ == '__main__':+ main()diff --git a/sysimage/select-system.py b/sysimage/select-system.pyindex 99a2229..e5cda12 100755--- a/sysimage/select-system.py+++ b/sysimage/select-system.py@@ -25,7 +25,7 @@ The whole root is 7.9 GB and the kernel on the phone has about 4 GB of RAM for eare left out, and so are a few very big daemons that have nothing to do with the display;- /Library (the wallpapers, regional tables) without its sounds;- of pictures, what the user interface itself is drawn with: small images, asset catalogs of- moderate size; the fonts, the two biggest (colour emoji and Chinese) made smaller by fetch.sh;+ moderate size; the fonts, the two biggest (colour emoji and Chinese) made smaller by unpack.sh;- the empty LANGUAGE.lproj directories of the bundles whose strings are in one .loctable filefor all languages: they are how CoreFoundation knows which languages a bundle has, andwithout them every string comes out as its key.@@ -64,7 +64,7 @@ SKIP_APP_EXT = ('.mov', '.mp4', '.m4a', '.caf', '.wav', '.heic', '.aiff', '.meta# (the one with all the SF Symbols alone is 170 MB and is left out)MAX_ASSET_CATALOG = 4 << 20# fonts: all but the very big ones; those two -- the colour emoji, 130 MB, and PingFang, the Chinese faces, 74 MB ---# are taken and made smaller after extraction (fetch.sh, ../tools/font_trim.py): the emoji without their biggest+# are taken and made smaller after extraction (unpack.sh, ../tools/font_trim.py): the emoji without their biggest# bitmaps (59 MB), PingFang with the regular and semibold faces for simplified, traditional and Hong Kong Chinese,# both those programs name and the hidden ones (".PingFangSC-Regular") the system font falls back to# (26 MB). Without them Safari and the rest drew emoji and Chinese characters as boxes with a question mark.diff --git a/sysimage/fetch.sh b/sysimage/unpack.shsimilarity index 55%rename from sysimage/fetch.shrename to sysimage/unpack.shindex ee15b25..f19de4c 100755--- a/sysimage/fetch.sh+++ b/sysimage/unpack.sh@@ -1,25 +1,44 @@#!/bin/bash-# Get the files of iOS 16.5 (20F66, iPhone X) that the system image is made of, straight from-# Apple's servers, and lay them out the way ../userland/build.sh expects them:+# Unpack the files of iOS 16.5 (20F66, iPhone X) that the system image is made of, out of an IPSW+# you have on your own disk, and lay them out the way ../userland/build.sh expects them:## rootfs/ the chosen part of the root filesystem (see select-system.py)# system.files/.manifest what was chosen, with owners, modes and symbolic links# var.skel the skeleton of /private/var (see mkvarskel.py)# cryptex/ the OS cryptex without the dyld shared cache-# cryptex-app.dmg the App cryptex as Apple ships it (10 MB, an APFS container: Safari's+# cryptex-app.dmg the App cryptex as it is in the IPSW (10 MB, an APFS container: Safari's# daemons, its settings and the bundles around it, which the root volume's# launchd jobs and links reach through /System/Cryptexes/App)# cryptex-app/ ... and its files# 20F66__iPhone10,3_6/ the dyld shared cache (arm64)+# DeviceTree.d22ap.im4p the device tree, for ../xnuloader/mkidentity.sh#-# Nothing of this is in the repository: the two disk images are fetched out of the IPSW on-# Apple's CDN (fetch-from-ipsw.py reads single members of the archive, 4.7 GB + 2.7 GB instead of-# all 6.2 GB) and unpacked with 7z. Needs: python3, 7z (7-Zip 23 or newer: it reads APFS), cc.-# About 8 GB of disk while it runs, 5.3 GB afterwards.+# The project ships no file of Apple's and downloads none. The IPSW is yours: get+# iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw yourself (6 165 698 198 bytes) and give its path:#-# KEEP=1 keep the downloaded disk images (rootfs.dmg is needed again only after changing-# select-system.py; with it in place a second run downloads nothing)+# sysimage/unpack.sh ~/Downloads/iPhone10,3,iPhone10,6_16.5_20F66_Restore.ipsw+# IPSW=... sysimage/unpack.sh the same, from the environment (make unpack IPSW=...)+#+# The disk images are copied out of the IPSW (ipsw-member.py) and unpacked with 7z. Needs: python3,+# 7z (7-Zip 23 or newer: it reads APFS), cc. About 8 GB of disk while it runs, 5.3 GB afterwards.+#+# KEEP=1 keep the copied disk images (rootfs.dmg is needed again only after changing+# select-system.py; with it in place a second run does not read the IPSW for it)set -e+IPSW=${1:-${IPSW:-}}+if [ -z "$IPSW" ]; then+ sed -n '16,20p' "$0" | sed 's/^# \{0,1\}//' >&2+ echo "unpack.sh: give the path of your iOS 16.5 IPSW" >&2+ exit 1+fi+[ -f "$IPSW" ] || { echo "unpack.sh: $IPSW: no such file" >&2; exit 1; }+IPSW=$(cd "$(dirname "$IPSW")" && pwd)/$(basename "$IPSW")+size=$(wc -c < "$IPSW" | tr -d ' ')+# select-system.py and the checks below are made for this one build; another IPSW would unpack, then fail somewhere later+if [ "$size" != 6165698198 ] && [ -z "${ANY_IPSW:-}" ]; then+ echo "unpack.sh: $IPSW is $size bytes, not the 6165698198 of iOS 16.5 20F66 for iPhone10,3/10,6 (ANY_IPSW=1 tries it anyway)" >&2+ exit 1+ficd "$(dirname "$0")"ROOTFS_DMG=096-04088-077.dmg # the root filesystem, APFSCRYPTEX_DMG=096-04666-078.dmg # the OS cryptex: dyld shared cache and a few libraries@@ -28,7 +47,7 @@ CACHE=20F66__iPhone10,3_6UNZBM=../xnu-deps/host/bin/unzbmfor tool in python3 7z cc; do- command -v $tool >/dev/null || { echo "fetch.sh: $tool is not installed"; exit 1; }+ command -v $tool >/dev/null || { echo "unpack.sh: $tool is not installed"; exit 1; }done# 7z leaves files compressed with LZBITMAP as an empty file and two attribute files; unzbm finishes themif [ ! -x $UNZBM ]; then@@ -39,21 +58,21 @@ unzbm_all() { # unzbm_all DIR: expand every LZBITMAP file below DIRfind "$1" -type f -name '*:com.apple.decmpfs' | sed 's/:com\.apple\.decmpfs$//' | tr '\n' '\0' | xargs -0 -r -n 200 $UNZBMlocal leftleft=$(find "$1" -name '*:com.apple.*' | wc -l)- [ "$left" -eq 0 ] || { echo "fetch.sh: $left files in $1 are still compressed"; exit 1; }+ [ "$left" -eq 0 ] || { echo "unpack.sh: $left files in $1 are still compressed"; exit 1; }}# --- the root filesystem --------------------------------------------------------------------------if [ ! -f .done-rootfs ] || [ select-system.py -nt .done-rootfs ]; thenif [ ! -f rootfs.dmg ]; then- echo "fetch.sh: downloading the root filesystem image (4.7 GB)"- python3 fetch-from-ipsw.py $ROOTFS_DMG rootfs.dmg.part+ echo "unpack.sh: copying the root filesystem image out of the IPSW (4.7 GB)"+ python3 ipsw-member.py "$IPSW" $ROOTFS_DMG rootfs.dmg.partmv rootfs.dmg.part rootfs.dmgfi- echo "fetch.sh: listing the image"+ echo "unpack.sh: listing the image"7z l -slt rootfs.dmg > rootfs.sltpython3 select-system.py rootfs.slt system.files system.manifestpython3 mkvarskel.py rootfs.slt > var.skel- echo "fetch.sh: extracting $(wc -l < system.files) files"+ echo "unpack.sh: extracting $(wc -l < system.files) files"rm -rf rootfs7z x -y -orootfs rootfs.dmg @system.files > extract-system.log# the two fonts too big for the image as they are, made smaller in place (see select-system.py)@@ -68,11 +87,11 @@ fi# --- the OS cryptex -------------------------------------------------------------------------------if [ ! -f .done-cryptex ]; thenif [ ! -f cryptex-os.dmg ]; then- echo "fetch.sh: downloading the OS cryptex image (2.7 GB)"- python3 fetch-from-ipsw.py $CRYPTEX_DMG cryptex-os.dmg.part+ echo "unpack.sh: copying the OS cryptex image out of the IPSW (2.7 GB)"+ python3 ipsw-member.py "$IPSW" $CRYPTEX_DMG cryptex-os.dmg.partmv cryptex-os.dmg.part cryptex-os.dmgfi- echo "fetch.sh: extracting the cryptex"+ echo "unpack.sh: extracting the cryptex"rm -rf cryptex cryptex.cache $CACHE# 7z reports every LZBITMAP file as an error ("Unsupported Method"): expected, unzbm_all checks the result7z x -y -ocryptex cryptex-os.dmg '-xr!com.apple.dyld' '-x!System/DriverKit' > extract-cryptex.log 2>&1 || true@@ -82,7 +101,7 @@ if [ ! -f .done-cryptex ]; thenunzbm_all cryptex.cachemv cryptex.cache/System/Library/Caches/com.apple.dyld $CACHErm -rf cryptex.cache- [ "$(ls $CACHE | wc -l)" -eq 43 ] || { echo "fetch.sh: the dyld shared cache is incomplete"; exit 1; }+ [ "$(ls $CACHE | wc -l)" -eq 43 ] || { echo "unpack.sh: the dyld shared cache is incomplete"; exit 1; }touch .done-cryptex[ -n "$KEEP" ] || rm -f cryptex-os.dmgfi@@ -92,15 +111,17 @@ fi# and unpacked (the system image has its files at /System/Cryptexes/App)if [ ! -f .done-cryptex-app ]; thenif [ ! -f cryptex-app.dmg ]; then- echo "fetch.sh: downloading the App cryptex image (10 MB)"- python3 fetch-from-ipsw.py $APP_CRYPTEX_DMG cryptex-app.dmg.part+ echo "unpack.sh: copying the App cryptex image out of the IPSW (10 MB)"+ python3 ipsw-member.py "$IPSW" $APP_CRYPTEX_DMG cryptex-app.dmg.partmv cryptex-app.dmg.part cryptex-app.dmgfi- echo "fetch.sh: extracting the App cryptex"+ echo "unpack.sh: extracting the App cryptex"rm -rf cryptex-app7z x -y -ocryptex-app cryptex-app.dmg > extract-cryptex-app.log 2>&1 || trueunzbm_all cryptex-app- [ -f cryptex-app/usr/libexec/webbookmarksd ] || { echo "fetch.sh: the App cryptex is incomplete"; exit 1; }+ [ -f cryptex-app/usr/libexec/webbookmarksd ] || { echo "unpack.sh: the App cryptex is incomplete"; exit 1; }touch .done-cryptex-appfi-echo "fetch.sh: done: $(du -sh rootfs | cut -f1) rootfs, $(du -sh cryptex | cut -f1) cryptex, $(du -sh cryptex-app | cut -f1) App cryptex, $(du -sh $CACHE | cut -f1) dyld shared cache"+# --- the device tree, for xnuloader/mkidentity.sh -----------------------------------------------+[ -f DeviceTree.d22ap.im4p ] || python3 ipsw-member.py "$IPSW" Firmware/all_flash/DeviceTree.d22ap.im4p DeviceTree.d22ap.im4p+echo "unpack.sh: done: $(du -sh rootfs | cut -f1) rootfs, $(du -sh cryptex | cut -f1) cryptex, $(du -sh cryptex-app | cut -f1) App cryptex, $(du -sh $CACHE | cut -f1) dyld shared cache"diff --git a/tools/apfs/run-tests.sh b/tools/apfs/run-tests.shindex 7605c0c..fa9ac46 100755--- a/tools/apfs/run-tests.sh+++ b/tools/apfs/run-tests.sh@@ -61,7 +61,7 @@ echo "== 2. the UDIF reader on sysimage/rootfs.dmg"if [ -f "$ROOT/sysimage/rootfs.dmg" ]; thenpython3 "$HERE/dmg.py" info "$ROOT/sysimage/rootfs.dmg" | tail -n 14 || fail=1else- echo " skipped: sysimage/rootfs.dmg is not here (run sysimage/fetch.sh with KEEP=1)"+ echo " skipped: sysimage/rootfs.dmg is not here (run sysimage/unpack.sh with KEEP=1)"fiechodiff --git a/tools/apfs/verify.py b/tools/apfs/verify.pyindex 175c87d..1651385 100755--- a/tools/apfs/verify.py+++ b/tools/apfs/verify.py@@ -4,7 +4,7 @@# Two independent references, both already in sysimage/:## listing sysimage/rootfs.slt -- what 7-Zip's own APFS reader saw in the same image-# (`7z l -slt rootfs.dmg`, made by sysimage/fetch.sh). Covers every entry in+# (`7z l -slt rootfs.dmg`, made by sysimage/unpack.sh). Covers every entry in# the volume: path, mode, owner, size, symbolic-link target, inode number.## content sysimage/rootfs/ -- the files 7-Zip actually extracted, compared byte for@@ -36,7 +36,7 @@ import apfsSYSTEM_OBJ_ID_MARK = 0x0FFFFFFF00000000-# sysimage/fetch.sh cuts these two fonts down after extracting them, so what is on disk in+# sysimage/unpack.sh cuts these two fonts down after extracting them, so what is on disk in# the reference tree is deliberately not what is in the imageTRIMMED = {"System/Library/Fonts/CoreAddition/AppleColorEmoji-160px.ttc",@@ -241,7 +241,7 @@ def cmd_content(image, refdir, limit, do_all, stratified):print("volume %r, reference tree %s" % (v.volname, refdir))mine = build_tree(v, note=True)- # every regular file that exists in both, minus the two fonts fetch.sh rewrites+ # every regular file that exists in both, minus the two fonts unpack.sh rewritescands = []for root, _dirs, files in os.walk(refdir):for fn in files:diff --git a/tools/apfs/write-tests.sh b/tools/apfs/write-tests.shindex 54bec97..637b737 100755--- a/tools/apfs/write-tests.sh+++ b/tools/apfs/write-tests.sh@@ -8,7 +8,7 @@# check-image.sh pins, cloned into xnu-deps/build/apfsprogs and built there: the# reader that has nothing to do with us and checks the space manager's bitmaps# against every block anything refers to-# 7-Zip a third APFS reader, already needed by sysimage/fetch.sh+# 7-Zip a third APFS reader, already needed by sysimage/unpack.sh## usage: tools/apfs/write-tests.sh [-k] [TEST ...] (-k: keep the images in apfs-work)# The tests: baseline writes refusals volume-gates extent-gates power-failure real-imagesdiff --git a/tools/doctor.sh b/tools/doctor.shindex 5f1bacd..2f470f9 100755--- a/tools/doctor.sh+++ b/tools/doctor.sh@@ -37,7 +37,7 @@ elsebad "python3 does not run" "install Python 3 (on Windows: python.org, not the Microsoft Store stub)"fi-# 7-Zip 23 or newer: sysimage/fetch.sh unpacks APFS with it+# 7-Zip 23 or newer: sysimage/unpack.sh unpacks APFS with itz=""; for t in 7zz 7z 7za; do have $t && { z=$t; break; }; doneif [ -n "$z" ]; thenzv=$($z 2>&1 | head -3 | grep -oE '[0-9]+\.[0-9]+' | head -1)@@ -46,7 +46,7 @@ elseneed "7-Zip not found" "install 7-Zip 23 or newer (apt: 7zip, brew: sevenzip)"fi-if have cc || have clang; then ok "C compiler ($(have cc && echo cc || echo clang)): builds unzbm for sysimage/fetch.sh"; else need "no C compiler" "install clang or gcc"; fi+if have cc || have clang; then ok "C compiler ($(have cc && echo cc || echo clang)): builds unzbm for sysimage/unpack.sh"; else need "no C compiler" "install clang or gcc"; fihave git && ok "git" || bad "git not found" "install git"if have lld-link; then ok "lld-link: the boot loader can be built (xnuloader/build.sh)"; else warn "lld-link not found" "needed to build the boot loader with the iPhone identity from your own IPSW (make loader), which the iOS run needs (apt: lld; brew: lld); the open kernel (make qemu-open) runs with prebuilt/loader.efi, which has no Apple data"; fihave lz4 && ok "lz4" || warn "lz4 not found" "needed by some image steps (apt: lz4)"@@ -63,10 +63,10 @@ fikb=$(df -Pk . 2>/dev/null | awk 'NR == 2 { print $4 }')if [ -n "$kb" ] && [ "$kb" -ge 12582912 ]; then ok "disk: $((kb / 1048576)) GB free here"; else warn "disk: ${kb:+$((kb / 1048576)) GB }free here, 12 GB wanted" "free some space, or work in another directory"; fi-# the trap: fetch.sh makes files named NAME:com.apple.decmpfs, and ':' is not a legal NTFS name character+# the trap: unpack.sh makes files named NAME:com.apple.decmpfs, and ':' is not a legal NTFS name charactercase "$host:$PWD" inwsl:/mnt/*) warn "this checkout is on a Windows drive (/mnt/...)" "build in WSL's own filesystem: git clone it to ~/hakios and copy fw-* there (docs/windows.md)" ;;-gitbash:*) warn "Git Bash cannot run sysimage/fetch.sh (NTFS forbids ':' in file names)" "build the images in WSL, copy them back; run xl from here (docs/windows.md)" ;;+gitbash:*) warn "Git Bash cannot run sysimage/unpack.sh (NTFS forbids ':' in file names)" "build the images in WSL, copy them back; run xl from here (docs/windows.md)" ;;esac# rebuilding the kernel or userland needs theos's SDK and its Linux toolchain's cctools linker (xnu-deps/setup.sh)@@ -101,7 +101,7 @@ fihead1 "What is already built here"if [ -f fw-modem/modem.mdt ]; then ok "phone firmware pulled (fw-*/)"; else warn "no fw-modem/ yet" "with the phone on adb: devices/redwood/pull-firmware.sh, or pull-firmware-root.sh if that gets nothing"; fi-if [ -f userland/system.apfs ] && [ -f userland/root.apfs ]; then ok "userland/system.apfs and root.apfs"; else warn "no iOS images yet" "make fetch, then PREBUILT=1 REST=1 MODEM=1 SYSTEM=1 userland/build.sh (needs iOS 16.5 from Apple's servers, ~8 GB)"; fi+if [ -f userland/system.apfs ] && [ -f userland/root.apfs ]; then ok "userland/system.apfs and root.apfs"; else warn "no iOS images yet" "make unpack IPSW=your.ipsw (iOS 16.5 20F66 for iPhone X, which you get yourself), then PREBUILT=1 REST=1 MODEM=1 SYSTEM=1 userland/build.sh (~8 GB)"; fihead1 "Phone"adb=""diff --git a/tools/font_trim.py b/tools/font_trim.pyindex 1a57846..5272e44 100755--- a/tools/font_trim.py+++ b/tools/font_trim.py@@ -14,7 +14,7 @@ the phone's memory, made from the files themselves (nothing of them is in the rematches REGEX, and only the tables they use.Both write a TrueType collection that CoreText reads as it reads the original; DST may be SRC.-sysimage/fetch.sh makes both for the system image this way.+sysimage/unpack.sh makes both for the system image this way."""import osimport rediff --git a/userland/mkjobs.py b/userland/mkjobs.pyindex cc41251..7e1f42f 100755--- a/userland/mkjobs.py+++ b/userland/mkjobs.py@@ -239,7 +239,7 @@ src_root, dest, labels = args[0], args[1], args[2:]oslog = os.environ.get('OSLOG', '').split()src = f'{src_root}/System/Library/LaunchDaemons'# A program in a cryptex -- on a device a volume of its own, grafted at /System/Cryptexes/NAME (eight of-# Safari's daemons are in the App cryptex) -- is looked for where fetch.sh unpacked that cryptex, beside+# Safari's daemons are in the App cryptex) -- is looked for where unpack.sh unpacked that cryptex, beside# the root filesystem; the images have it at the same place (build.sh).CRYPTEXES = {'/System/Cryptexes/OS/': 'cryptex', '/System/Cryptexes/App/': 'cryptex-app'}diff --git a/xnuloader/mkidentity.sh b/xnuloader/mkidentity.shindex 1b22100..5f06014 100755--- a/xnuloader/mkidentity.sh+++ b/xnuloader/mkidentity.sh@@ -3,13 +3,15 @@# from the device tree in the IPSW (Firmware/all_flash/DeviceTree.d22ap.im4p) at build time, on# the builder's machine, from their own copy of the firmware. The header is not in the repository.#-# sysimage/fetch-from-ipsw.py the one file out of the IPSW on Apple's CDN (34 KB, HTTP ranges)+# sysimage/ipsw-member.py the one file out of your IPSW (34 KB); sysimage/unpack.sh leaves it+# in sysimage/ too# xnuloader/im4p.py its payload: an LZFSE stream# lzfse(1) / compression_tool(1) / python3 -m lzfse / xnu-deps/host/bin/lzfse (built from the# BSD-licensed github.com/lzfse/lzfse when nothing else is there)# xnuloader/mkidentity.py the header from the decoded tree#-# usage: xnuloader/mkidentity.sh [DeviceTree.im4p] (an im4p already at hand skips the download)+# usage: xnuloader/mkidentity.sh [DeviceTree.im4p | your.ipsw]+# (with no argument: sysimage/DeviceTree.d22ap.im4p, or $IPSW)# xnuloader/mkidentity.sh --placeholder (no IPSW: an identity.h with no Apple data,# enough to build and boot the open kernel under# QEMU with an Apple-free user space, not iOS)@@ -24,10 +26,18 @@ if [ "$1" = --placeholder ] || [ "${PLACEHOLDER:-}" = 1 ]; thenfiWORK=$(mktemp -d "${TMPDIR:-/tmp}/mkidentity.XXXXXX")trap 'rm -rf "$WORK"' EXIT-IM4P=${1:-$WORK/dt.im4p}-if [ ! -f "$IM4P" ]; then- python3 "$TOP/sysimage/fetch-from-ipsw.py" Firmware/all_flash/DeviceTree.d22ap.im4p "$IM4P"+# nothing is downloaded: the device tree comes from the IPSW you have+SRC=${1:-}+[ -n "$SRC" ] || { [ -f "$TOP/sysimage/DeviceTree.d22ap.im4p" ] && SRC=$TOP/sysimage/DeviceTree.d22ap.im4p; } || SRC=${IPSW:-}+if [ -z "$SRC" ] || [ ! -f "$SRC" ]; then+ echo "mkidentity.sh: no device tree. Run sysimage/unpack.sh YOUR.ipsw first, or give the IPSW or" >&2+ echo " DeviceTree.d22ap.im4p as the argument (or --placeholder for the open kernel without iOS)" >&2+ exit 1fi+case $SRC in+*.ipsw) python3 "$TOP/sysimage/ipsw-member.py" "$SRC" Firmware/all_flash/DeviceTree.d22ap.im4p "$WORK/dt.im4p"; IM4P=$WORK/dt.im4p ;;+*) IM4P=$SRC ;;+esacpython3 "$HERE/im4p.py" "$IM4P" "$WORK/dt.lzfse"decode() {if command -v lzfse >/dev/null; then lzfse -decode -i "$1" -o "$2"